1 | Hi again, | 1 | Hi, |
---|---|---|---|
2 | 2 | ||
3 | This patch series intent is to introduce a generator that produces a Go | 3 | This new version removed the translate_fn() from patch 1 because it |
4 | module for Go applications to interact over QMP with QEMU. | 4 | wasn't removing the sign-extension for pentry as we thought it would. |
5 | A more detailed explanation is given in the commit msg of patch 1. | ||
5 | 6 | ||
6 | Previous version (10 Jan 2025) | 7 | We're now retrieving the 'lowaddr' value from load_elf_ram_sym() and |
7 | https://lists.gnu.org/archive/html/qemu-devel/2025-01/msg01530.html | 8 | using it when we're running a 32-bit CPU. This worked with 32 bit |
9 | 'virt' machine booting with the -kernel option. | ||
8 | 10 | ||
9 | The generated code was mostly tested using existing examples in the QAPI | 11 | If this approach doesn't work for the Xvisor use case, IMO we should |
10 | documentation, 192 instances that might have multiple QMP messages each. | 12 | just filter kernel_load_addr bits directly as we were doing a handful of |
13 | versions ago. | ||
11 | 14 | ||
12 | You can find the the tests and the generated code in my personal repo, | 15 | Patches are based on current riscv-to-apply.next. |
13 | main branch: | ||
14 | 16 | ||
15 | https://gitlab.com/victortoso/qapi-go | 17 | Changes from v9: |
18 | - patch 1: | ||
19 | - removed the translate_fn() callback | ||
20 | - return 'kernel_low' when running a 32-bit CPU | ||
21 | - v9 link: https://lists.gnu.org/archive/html/qemu-devel/2023-01/msg04509.html | ||
16 | 22 | ||
17 | If you want to see the generated code from QEMU's master but per patch: | 23 | Daniel Henrique Barboza (3): |
24 | hw/riscv: handle 32 bit CPUs kernel_addr in riscv_load_kernel() | ||
25 | hw/riscv/boot.c: consolidate all kernel init in riscv_load_kernel() | ||
26 | hw/riscv/boot.c: make riscv_load_initrd() static | ||
18 | 27 | ||
19 | https://gitlab.com/victortoso/qapi-go/-/commits/qapi-golang-v4-by-patch | 28 | hw/riscv/boot.c | 96 +++++++++++++++++++++++--------------- |
20 | 29 | hw/riscv/microchip_pfsoc.c | 12 +---- | |
21 | If you rather see the diff between v9.1.0, v9.2.0 and latest: | 30 | hw/riscv/opentitan.c | 4 +- |
22 | 31 | hw/riscv/sifive_e.c | 4 +- | |
23 | https://gitlab.com/victortoso/qapi-go/-/commits/qapi-golang-v4-by-tags | 32 | hw/riscv/sifive_u.c | 12 +---- |
24 | 33 | hw/riscv/spike.c | 14 ++---- | |
25 | 34 | hw/riscv/virt.c | 12 +---- | |
26 | ################# | 35 | include/hw/riscv/boot.h | 3 +- |
27 | # Changes in v4 # | 36 | 8 files changed, 76 insertions(+), 81 deletions(-) |
28 | ################# | ||
29 | |||
30 | 1. Daniel wrote a demo on top of v3 and proposed changes that would | ||
31 | result in more interesting module to build on top: | ||
32 | https://lists.gnu.org/archive/html/qemu-devel/2025-01/msg03052.html | ||
33 | |||
34 | I've implemented all the suggestions that are relevant for this | ||
35 | introductory series, they are: | ||
36 | |||
37 | a. New struct type Message, that shall be used for a 1st level | ||
38 | unmarshalling of the JSON message. | ||
39 | b. Removal of Marshal/Unmarshal code in both Events and Comands, | ||
40 | together with utility code that is not relevant anymore. | ||
41 | c. Declaration of 3 new interfaces: | ||
42 | i. Events | ||
43 | ii. Commands | ||
44 | iii. CommandsAsync | ||
45 | |||
46 | 2. I've moved the code to a new folder: scripts/qapi/golang. This | ||
47 | allowed me to move templates out of golang.py, keeping go related | ||
48 | code self-contained in the new directory. | ||
49 | |||
50 | 3. As mentioned in (2), created protocol.go and utils.go that are 100% | ||
51 | hand generated Go code. Message mentioned in (1a) is under | ||
52 | protocol.go | ||
53 | |||
54 | 4. Defined license using SPDX-License-Identifier. | ||
55 | a. Every Go source code written by hand is 100% MIT-0 | ||
56 | b. Every Go source code generated is dual licensed as MIT-0 and | ||
57 | GPL-2.0-or-later | ||
58 | c. The binary code is expected to be MIT-0 only but not really | ||
59 | relevant for this series. | ||
60 | |||
61 | If you want more information, please check the thread: | ||
62 | https://lists.gnu.org/archive/html/qemu-devel/2024-11/msg01621.html | ||
63 | |||
64 | 5. I've renamed the generated files. | ||
65 | a. Any type related file is now prefixed with "gen_type_" | ||
66 | b. Any interface related file is prefixed as "gen_iface_" | ||
67 | |||
68 | 6. Relevant changes were made to the doc but it is not complete. I plan | ||
69 | that follow-up proposals would add to the documentation. | ||
70 | |||
71 | 7. Improvements to the generator were made to. | ||
72 | |||
73 | 8. Also worth to mention that resulting generated code does not have any | ||
74 | diff with gofmt and goimport tools, as requested in the past. | ||
75 | |||
76 | ################ | ||
77 | # Expectations # | ||
78 | ################ | ||
79 | |||
80 | As is, this still is a PoC that works. I'd like to have the generated | ||
81 | code included in QEMU's gitlab [0] in order to write library and tools | ||
82 | on top. Initial version should be considered alpha. Moving to | ||
83 | beta/stable would require functional libraries and tools, but this work | ||
84 | needs to be merged before one commit to that. | ||
85 | |||
86 | [0] https://lists.gnu.org/archive/html/qemu-devel/2023-09/msg07024.html | ||
87 | |||
88 | |||
89 | ################## | ||
90 | # Follow-up work # | ||
91 | ################## | ||
92 | |||
93 | When this is merged we would need to: | ||
94 | 1. Create gitlab's repo | ||
95 | 2. Add unit test and CI to new repos | ||
96 | 3. Have first alpha relase/tag | ||
97 | 4. Start working on top :) | ||
98 | |||
99 | |||
100 | Thanks for the time looking at this. I appreciate it. | ||
101 | |||
102 | Victor Toso (11): | ||
103 | qapi: golang: first level unmarshalling type | ||
104 | qapi: golang: Generate enum type | ||
105 | qapi: golang: Generate alternate types | ||
106 | qapi: golang: Generate struct types | ||
107 | qapi: golang: structs: Address nullable members | ||
108 | qapi: golang: Generate union type | ||
109 | qapi: golang: Generate event type | ||
110 | qapi: golang: Generate Event interface | ||
111 | qapi: golang: Generate command type | ||
112 | qapi: golang: Generate Command sync/async interfaces | ||
113 | docs: add notes on Golang code generator | ||
114 | |||
115 | docs/devel/index-build.rst | 1 + | ||
116 | docs/devel/qapi-golang-code-gen.rst | 420 ++++++++ | ||
117 | scripts/qapi/golang/__init__.py | 0 | ||
118 | scripts/qapi/golang/golang.py | 1444 +++++++++++++++++++++++++++ | ||
119 | scripts/qapi/golang/protocol.go | 48 + | ||
120 | scripts/qapi/golang/utils.go | 38 + | ||
121 | scripts/qapi/main.py | 2 + | ||
122 | 7 files changed, 1953 insertions(+) | ||
123 | create mode 100644 docs/devel/qapi-golang-code-gen.rst | ||
124 | create mode 100644 scripts/qapi/golang/__init__.py | ||
125 | create mode 100644 scripts/qapi/golang/golang.py | ||
126 | create mode 100644 scripts/qapi/golang/protocol.go | ||
127 | create mode 100644 scripts/qapi/golang/utils.go | ||
128 | 37 | ||
129 | -- | 38 | -- |
130 | 2.48.1 | 39 | 2.39.1 | diff view generated by jsdifflib |
1 | This patch handles QAPI enum types and generates its equivalent in Go. | 1 | load_elf_ram_sym() will sign-extend 32 bit addresses. If a 32 bit QEMU |
---|---|---|---|
2 | We sort the output based on enum's type name. | 2 | guest happens to be running in a hypervisor that are using 64 bits to |
3 | encode its address, kernel_entry can be padded with '1's and create | ||
4 | problems [1]. | ||
3 | 5 | ||
4 | Enums are being handled as strings in Golang. | 6 | Using a translate_fn() callback in load_elf_ram_sym() to filter the |
7 | padding from the address doesn't work. A more detailed explanation can | ||
8 | be found in [2]. The short version is that glue(load_elf, SZ), from | ||
9 | include/hw/elf_ops.h, will calculate 'pentry' (mapped into the | ||
10 | 'kernel_load_base' var in riscv_load_Kernel()) before using | ||
11 | translate_fn(), and will not recalculate it after executing it. This | ||
12 | means that the callback does not prevent the padding from | ||
13 | kernel_load_base to appear. | ||
5 | 14 | ||
6 | 1. For each QAPI enum, we will define a string type in Go to be the | 15 | Let's instead use a kernel_low var to capture the 'lowaddr' value from |
7 | assigned type of this specific enum. | 16 | load_elf_ram_sim(), and return it when we're dealing with 32 bit CPUs. |
8 | 17 | ||
9 | 2. Naming: CamelCase will be used in any identifier that we want to | 18 | [1] https://lists.gnu.org/archive/html/qemu-devel/2023-01/msg02281.html |
10 | export, which is everything. | 19 | [2] https://lists.gnu.org/archive/html/qemu-devel/2023-02/msg00099.html |
11 | 20 | ||
12 | Example: | 21 | Signed-off-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> |
22 | --- | ||
23 | hw/riscv/boot.c | 15 +++++++++++---- | ||
24 | hw/riscv/microchip_pfsoc.c | 3 ++- | ||
25 | hw/riscv/opentitan.c | 3 ++- | ||
26 | hw/riscv/sifive_e.c | 3 ++- | ||
27 | hw/riscv/sifive_u.c | 3 ++- | ||
28 | hw/riscv/spike.c | 3 ++- | ||
29 | hw/riscv/virt.c | 3 ++- | ||
30 | include/hw/riscv/boot.h | 1 + | ||
31 | 8 files changed, 24 insertions(+), 10 deletions(-) | ||
13 | 32 | ||
14 | qapi: | 33 | diff --git a/hw/riscv/boot.c b/hw/riscv/boot.c |
15 | | ## | ||
16 | | # @DisplayProtocol: | ||
17 | | # | ||
18 | | # Display protocols which support changing password options. | ||
19 | | # | ||
20 | | # Since: 7.0 | ||
21 | | ## | ||
22 | | { 'enum': 'DisplayProtocol', | ||
23 | | 'data': [ 'vnc', 'spice' ] } | ||
24 | |||
25 | go: | ||
26 | | // Display protocols which support changing password options. | ||
27 | | // | ||
28 | | // Since: 7.0 | ||
29 | | type DisplayProtocol string | ||
30 | | | ||
31 | | const ( | ||
32 | | DisplayProtocolVnc DisplayProtocol = "vnc" | ||
33 | | DisplayProtocolSpice DisplayProtocol = "spice" | ||
34 | | ) | ||
35 | |||
36 | Signed-off-by: Victor Toso <victortoso@redhat.com> | ||
37 | --- | ||
38 | scripts/qapi/golang/golang.py | 185 +++++++++++++++++++++++++++++++++- | ||
39 | 1 file changed, 183 insertions(+), 2 deletions(-) | ||
40 | |||
41 | diff --git a/scripts/qapi/golang/golang.py b/scripts/qapi/golang/golang.py | ||
42 | index XXXXXXX..XXXXXXX 100644 | 34 | index XXXXXXX..XXXXXXX 100644 |
43 | --- a/scripts/qapi/golang/golang.py | 35 | --- a/hw/riscv/boot.c |
44 | +++ b/scripts/qapi/golang/golang.py | 36 | +++ b/hw/riscv/boot.c |
45 | @@ -XXX,XX +XXX,XX @@ | 37 | @@ -XXX,XX +XXX,XX @@ target_ulong riscv_load_firmware(const char *firmware_filename, |
46 | # Just for type hint on self | 38 | } |
47 | from __future__ import annotations | 39 | |
48 | 40 | target_ulong riscv_load_kernel(MachineState *machine, | |
49 | -import os, shutil | 41 | + RISCVHartArrayState *harts, |
50 | +import os, shutil, textwrap | 42 | target_ulong kernel_start_addr, |
51 | from typing import List, Optional | 43 | symbol_fn_t sym_cb) |
52 | 44 | { | |
53 | from ..schema import ( | 45 | const char *kernel_filename = machine->kernel_filename; |
54 | @@ -XXX,XX +XXX,XX @@ | 46 | - uint64_t kernel_load_base, kernel_entry; |
55 | ) | 47 | + uint64_t kernel_load_base, kernel_entry, kernel_low; |
56 | from ..source import QAPISourceInfo | 48 | |
57 | 49 | g_assert(kernel_filename != NULL); | |
58 | +TEMPLATE_GENERATED_HEADER = """ | 50 | |
59 | +/* | 51 | @@ -XXX,XX +XXX,XX @@ target_ulong riscv_load_kernel(MachineState *machine, |
60 | + * Copyright 2025 Red Hat, Inc. | 52 | * the (expected) load address load address. This allows kernels to have |
61 | + * SPDX-License-Identifier: (MIT-0 and GPL-2.0-or-later) | 53 | * separate SBI and ELF entry points (used by FreeBSD, for example). |
62 | + */ | 54 | */ |
55 | - if (load_elf_ram_sym(kernel_filename, NULL, NULL, NULL, | ||
56 | - NULL, &kernel_load_base, NULL, NULL, 0, | ||
57 | + if (load_elf_ram_sym(kernel_filename, NULL, NULL, NULL, NULL, | ||
58 | + &kernel_load_base, &kernel_low, NULL, 0, | ||
59 | EM_RISCV, 1, 0, NULL, true, sym_cb) > 0) { | ||
60 | - return kernel_load_base; | ||
61 | + kernel_entry = kernel_load_base; | ||
63 | + | 62 | + |
64 | +/**************************************************************************** | 63 | + if (riscv_is_32bit(harts)) { |
65 | + * THIS CODE HAS BEEN GENERATED. DO NOT CHANGE IT DIRECTLY * | 64 | + kernel_entry = kernel_low; |
66 | + ****************************************************************************/ | ||
67 | +package {package_name} | ||
68 | +""" | ||
69 | + | ||
70 | +TEMPLATE_GO_IMPORTS = """ | ||
71 | +import ( | ||
72 | +{imports} | ||
73 | +) | ||
74 | +""" | ||
75 | + | ||
76 | +TEMPLATE_ENUM = """ | ||
77 | +type {name} string | ||
78 | + | ||
79 | +const ( | ||
80 | +{fields} | ||
81 | +) | ||
82 | +""" | ||
83 | + | ||
84 | + | ||
85 | +# Takes the documentation object of a specific type and returns | ||
86 | +# that type's documentation and its member's docs. | ||
87 | +def qapi_to_golang_struct_docs(doc: QAPIDoc) -> (str, Dict[str, str]): | ||
88 | + if doc is None: | ||
89 | + return "", {} | ||
90 | + | ||
91 | + cmt = "// " | ||
92 | + fmt = textwrap.TextWrapper( | ||
93 | + width=70, initial_indent=cmt, subsequent_indent=cmt | ||
94 | + ) | ||
95 | + main = fmt.fill(doc.body.text) | ||
96 | + | ||
97 | + for section in doc.sections: | ||
98 | + # TODO is not a relevant section to Go applications | ||
99 | + if section.tag in ["TODO"]: | ||
100 | + continue | ||
101 | + | ||
102 | + if main != "": | ||
103 | + # Give empty line as space for the tag. | ||
104 | + main += "\n//\n" | ||
105 | + | ||
106 | + tag = "" if section.tag is None else f"{section.tag}: " | ||
107 | + text = section.text.replace(" ", " ") | ||
108 | + main += fmt.fill(f"{tag}{text}") | ||
109 | + | ||
110 | + fields = {} | ||
111 | + for key, value in doc.args.items(): | ||
112 | + if len(value.text) > 0: | ||
113 | + fields[key] = " ".join(value.text.replace("\n", " ").split()) | ||
114 | + | ||
115 | + return main, fields | ||
116 | + | ||
117 | |||
118 | def gen_golang(schema: QAPISchema, output_dir: str, prefix: str) -> None: | ||
119 | vis = QAPISchemaGenGolangVisitor(prefix) | ||
120 | @@ -XXX,XX +XXX,XX @@ def gen_golang(schema: QAPISchema, output_dir: str, prefix: str) -> None: | ||
121 | vis.write(output_dir) | ||
122 | |||
123 | |||
124 | +def qapi_to_field_name_enum(name: str) -> str: | ||
125 | + return name.title().replace("-", "") | ||
126 | + | ||
127 | + | ||
128 | +def fetch_indent_blocks_over_enum_with_docs( | ||
129 | + name: str, members: List[QAPISchemaEnumMember], docfields: Dict[str, str] | ||
130 | +) -> Tuple[int]: | ||
131 | + maxname = 0 | ||
132 | + blocks: List[int] = [0] | ||
133 | + for member in members: | ||
134 | + # For simplicity, every time we have doc, we add a new indent block | ||
135 | + hasdoc = member.name is not None and member.name in docfields | ||
136 | + | ||
137 | + enum_name = f"{name}{qapi_to_field_name_enum(member.name)}" | ||
138 | + maxname = ( | ||
139 | + max(maxname, len(enum_name)) if not hasdoc else len(enum_name) | ||
140 | + ) | ||
141 | + | ||
142 | + if hasdoc: | ||
143 | + blocks.append(maxname) | ||
144 | + else: | ||
145 | + blocks[-1] = maxname | ||
146 | + | ||
147 | + return blocks | ||
148 | + | ||
149 | + | ||
150 | +def generate_content_from_dict(data: dict[str, str]) -> str: | ||
151 | + content = "" | ||
152 | + | ||
153 | + for name in sorted(data): | ||
154 | + content += data[name] | ||
155 | + | ||
156 | + return content.replace("\n\n\n", "\n\n") | ||
157 | + | ||
158 | + | ||
159 | +def generate_template_imports(words: List[str]) -> str: | ||
160 | + if len(words) == 0: | ||
161 | + return "" | ||
162 | + | ||
163 | + if len(words) == 1: | ||
164 | + return '\nimport "{words[0]}"\n' | ||
165 | + | ||
166 | + return TEMPLATE_GO_IMPORTS.format( | ||
167 | + imports="\n".join(f'\t"{w}"' for w in words) | ||
168 | + ) | ||
169 | + | ||
170 | + | ||
171 | class QAPISchemaGenGolangVisitor(QAPISchemaVisitor): | ||
172 | # pylint: disable=too-many-arguments | ||
173 | def __init__(self, _: str): | ||
174 | super().__init__() | ||
175 | gofiles = ("protocol.go",) | ||
176 | + # Map each qapi type to the necessary Go imports | ||
177 | + types = { | ||
178 | + "enum": [], | ||
179 | + } | 65 | + } |
180 | + | 66 | + |
181 | self.schema: QAPISchema | 67 | + return kernel_entry; |
182 | self.golang_package_name = "qapi" | 68 | } |
183 | self.duplicate = list(gofiles) | 69 | |
184 | + self.enums: dict[str, str] = {} | 70 | if (load_uimage_as(kernel_filename, &kernel_entry, NULL, NULL, |
185 | + self.docmap = {} | 71 | diff --git a/hw/riscv/microchip_pfsoc.c b/hw/riscv/microchip_pfsoc.c |
186 | + | 72 | index XXXXXXX..XXXXXXX 100644 |
187 | + self.types = dict.fromkeys(types, "") | 73 | --- a/hw/riscv/microchip_pfsoc.c |
188 | + self.types_import = types | 74 | +++ b/hw/riscv/microchip_pfsoc.c |
189 | 75 | @@ -XXX,XX +XXX,XX @@ static void microchip_icicle_kit_machine_init(MachineState *machine) | |
190 | def visit_begin(self, schema: QAPISchema) -> None: | 76 | kernel_start_addr = riscv_calc_kernel_start_addr(&s->soc.u_cpus, |
191 | self.schema = schema | 77 | firmware_end_addr); |
192 | 78 | ||
193 | + # iterate once in schema.docs to map doc objects to its name | 79 | - kernel_entry = riscv_load_kernel(machine, kernel_start_addr, NULL); |
194 | + for doc in schema.docs: | 80 | + kernel_entry = riscv_load_kernel(machine, &s->soc.u_cpus, |
195 | + if doc.symbol is None: | 81 | + kernel_start_addr, NULL); |
196 | + continue | 82 | |
197 | + self.docmap[doc.symbol] = doc | 83 | if (machine->initrd_filename) { |
198 | + | 84 | riscv_load_initrd(machine, kernel_entry); |
199 | + for qapitype, imports in self.types_import.items(): | 85 | diff --git a/hw/riscv/opentitan.c b/hw/riscv/opentitan.c |
200 | + self.types[qapitype] = TEMPLATE_GENERATED_HEADER[1:].format( | 86 | index XXXXXXX..XXXXXXX 100644 |
201 | + package_name=self.golang_package_name | 87 | --- a/hw/riscv/opentitan.c |
202 | + ) | 88 | +++ b/hw/riscv/opentitan.c |
203 | + self.types[qapitype] += generate_template_imports(imports) | 89 | @@ -XXX,XX +XXX,XX @@ static void opentitan_board_init(MachineState *machine) |
204 | + | 90 | } |
205 | def visit_end(self) -> None: | 91 | |
206 | del self.schema | 92 | if (machine->kernel_filename) { |
207 | + self.types["enum"] += generate_content_from_dict(self.enums) | 93 | - riscv_load_kernel(machine, memmap[IBEX_DEV_RAM].base, NULL); |
208 | 94 | + riscv_load_kernel(machine, &s->soc.cpus, | |
209 | def visit_object_type( | 95 | + memmap[IBEX_DEV_RAM].base, NULL); |
210 | self, | 96 | } |
211 | @@ -XXX,XX +XXX,XX @@ def visit_enum_type( | 97 | } |
212 | members: List[QAPISchemaEnumMember], | 98 | |
213 | prefix: Optional[str], | 99 | diff --git a/hw/riscv/sifive_e.c b/hw/riscv/sifive_e.c |
214 | ) -> None: | 100 | index XXXXXXX..XXXXXXX 100644 |
215 | - pass | 101 | --- a/hw/riscv/sifive_e.c |
216 | + assert name not in self.enums | 102 | +++ b/hw/riscv/sifive_e.c |
217 | + doc = self.docmap.get(name, None) | 103 | @@ -XXX,XX +XXX,XX @@ static void sifive_e_machine_init(MachineState *machine) |
218 | + maindoc, docfields = qapi_to_golang_struct_docs(doc) | 104 | memmap[SIFIVE_E_DEV_MROM].base, &address_space_memory); |
219 | + | 105 | |
220 | + # The logic below is to generate QAPI enums as blocks of Go consts | 106 | if (machine->kernel_filename) { |
221 | + # each with its own type for type safety inside Go applications. | 107 | - riscv_load_kernel(machine, memmap[SIFIVE_E_DEV_DTIM].base, NULL); |
222 | + # | 108 | + riscv_load_kernel(machine, &s->soc.cpus, |
223 | + # Block of const() blocks are vertically indented so we have to | 109 | + memmap[SIFIVE_E_DEV_DTIM].base, NULL); |
224 | + # first iterate over all names to calculate space between | 110 | } |
225 | + # $var_name and $var_type. This is achieved by helper function | 111 | } |
226 | + # @fetch_indent_blocks_over_enum_with_docs() | 112 | |
227 | + # | 113 | diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c |
228 | + # A new indentation block is defined by empty line or a comment. | 114 | index XXXXXXX..XXXXXXX 100644 |
229 | + | 115 | --- a/hw/riscv/sifive_u.c |
230 | + indent_block = iter( | 116 | +++ b/hw/riscv/sifive_u.c |
231 | + fetch_indent_blocks_over_enum_with_docs(name, members, docfields) | 117 | @@ -XXX,XX +XXX,XX @@ static void sifive_u_machine_init(MachineState *machine) |
232 | + ) | 118 | kernel_start_addr = riscv_calc_kernel_start_addr(&s->soc.u_cpus, |
233 | + maxname = next(indent_block) | 119 | firmware_end_addr); |
234 | + fields = "" | 120 | |
235 | + for index, member in enumerate(members): | 121 | - kernel_entry = riscv_load_kernel(machine, kernel_start_addr, NULL); |
236 | + # For simplicity, every time we have doc, we go to next indent block | 122 | + kernel_entry = riscv_load_kernel(machine, &s->soc.u_cpus, |
237 | + hasdoc = member.name is not None and member.name in docfields | 123 | + kernel_start_addr, NULL); |
238 | + | 124 | |
239 | + if hasdoc: | 125 | if (machine->initrd_filename) { |
240 | + maxname = next(indent_block) | 126 | riscv_load_initrd(machine, kernel_entry); |
241 | + | 127 | diff --git a/hw/riscv/spike.c b/hw/riscv/spike.c |
242 | + enum_name = f"{name}{qapi_to_field_name_enum(member.name)}" | 128 | index XXXXXXX..XXXXXXX 100644 |
243 | + name2type = " " * (maxname - len(enum_name) + 1) | 129 | --- a/hw/riscv/spike.c |
244 | + | 130 | +++ b/hw/riscv/spike.c |
245 | + if hasdoc: | 131 | @@ -XXX,XX +XXX,XX @@ static void spike_board_init(MachineState *machine) |
246 | + docstr = ( | 132 | kernel_start_addr = riscv_calc_kernel_start_addr(&s->soc[0], |
247 | + textwrap.TextWrapper(width=80) | 133 | firmware_end_addr); |
248 | + .fill(docfields[member.name]) | 134 | |
249 | + .replace("\n", "\n\t// ") | 135 | - kernel_entry = riscv_load_kernel(machine, kernel_start_addr, |
250 | + ) | 136 | + kernel_entry = riscv_load_kernel(machine, &s->soc[0], |
251 | + fields += f"""\t// {docstr}\n""" | 137 | + kernel_start_addr, |
252 | + | 138 | htif_symbol_callback); |
253 | + fields += f"""\t{enum_name}{name2type}{name} = "{member.name}"\n""" | 139 | |
254 | + | 140 | if (machine->initrd_filename) { |
255 | + if maindoc != "": | 141 | diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c |
256 | + maindoc = f"\n{maindoc}" | 142 | index XXXXXXX..XXXXXXX 100644 |
257 | + | 143 | --- a/hw/riscv/virt.c |
258 | + self.enums[name] = maindoc + TEMPLATE_ENUM.format( | 144 | +++ b/hw/riscv/virt.c |
259 | + name=name, fields=fields[:-1] | 145 | @@ -XXX,XX +XXX,XX @@ static void virt_machine_done(Notifier *notifier, void *data) |
260 | + ) | 146 | kernel_start_addr = riscv_calc_kernel_start_addr(&s->soc[0], |
261 | 147 | firmware_end_addr); | |
262 | def visit_array_type( | 148 | |
263 | self, | 149 | - kernel_entry = riscv_load_kernel(machine, kernel_start_addr, NULL); |
264 | @@ -XXX,XX +XXX,XX @@ def write(self, outdir: str) -> None: | 150 | + kernel_entry = riscv_load_kernel(machine, &s->soc[0], |
265 | srcpath = os.path.join(srcdir, filename) | 151 | + kernel_start_addr, NULL); |
266 | dstpath = os.path.join(targetpath, filename) | 152 | |
267 | shutil.copyfile(srcpath, dstpath) | 153 | if (machine->initrd_filename) { |
268 | + | 154 | riscv_load_initrd(machine, kernel_entry); |
269 | + # Types to be generated | 155 | diff --git a/include/hw/riscv/boot.h b/include/hw/riscv/boot.h |
270 | + for qapitype, content in self.types.items(): | 156 | index XXXXXXX..XXXXXXX 100644 |
271 | + gofile = f"gen_type_{qapitype}.go" | 157 | --- a/include/hw/riscv/boot.h |
272 | + pathname = os.path.join(targetpath, gofile) | 158 | +++ b/include/hw/riscv/boot.h |
273 | + | 159 | @@ -XXX,XX +XXX,XX @@ target_ulong riscv_load_firmware(const char *firmware_filename, |
274 | + with open(pathname, "w", encoding="utf8") as outfile: | 160 | hwaddr firmware_load_addr, |
275 | + outfile.write(content) | 161 | symbol_fn_t sym_cb); |
162 | target_ulong riscv_load_kernel(MachineState *machine, | ||
163 | + RISCVHartArrayState *harts, | ||
164 | target_ulong firmware_end_addr, | ||
165 | symbol_fn_t sym_cb); | ||
166 | void riscv_load_initrd(MachineState *machine, uint64_t kernel_entry); | ||
276 | -- | 167 | -- |
277 | 2.48.1 | 168 | 2.39.1 | diff view generated by jsdifflib |
1 | The goal of this patch is converge discussions into a documentation, | 1 | The microchip_icicle_kit, sifive_u, spike and virt boards are now doing |
---|---|---|---|
2 | to make it easy and explicit design decisions, known issues and what | 2 | the same steps when '-kernel' is used: |
3 | else might help a person interested in how the Go module is generated. | 3 | |
4 | 4 | - execute load_kernel() | |
5 | Signed-off-by: Victor Toso <victortoso@redhat.com> | 5 | - load init_rd() |
6 | - write kernel_cmdline | ||
7 | |||
8 | Let's fold everything inside riscv_load_kernel() to avoid code | ||
9 | repetition. To not change the behavior of boards that aren't calling | ||
10 | riscv_load_init(), add an 'load_initrd' flag to riscv_load_kernel() and | ||
11 | allow these boards to opt out from initrd loading. | ||
12 | |||
13 | Cc: Palmer Dabbelt <palmer@dabbelt.com> | ||
14 | Reviewed-by: Bin Meng <bmeng@tinylab.org> | ||
15 | Reviewed-by: Alistair Francis <alistair.francis@wdc.com> | ||
16 | Signed-off-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> | ||
6 | --- | 17 | --- |
7 | docs/devel/index-build.rst | 1 + | 18 | hw/riscv/boot.c | 21 ++++++++++++++++++--- |
8 | docs/devel/qapi-golang-code-gen.rst | 420 ++++++++++++++++++++++++++++ | 19 | hw/riscv/microchip_pfsoc.c | 11 +---------- |
9 | 2 files changed, 421 insertions(+) | 20 | hw/riscv/opentitan.c | 3 ++- |
10 | create mode 100644 docs/devel/qapi-golang-code-gen.rst | 21 | hw/riscv/sifive_e.c | 3 ++- |
11 | 22 | hw/riscv/sifive_u.c | 11 +---------- | |
12 | diff --git a/docs/devel/index-build.rst b/docs/devel/index-build.rst | 23 | hw/riscv/spike.c | 11 +---------- |
13 | index XXXXXXX..XXXXXXX 100644 | 24 | hw/riscv/virt.c | 11 +---------- |
14 | --- a/docs/devel/index-build.rst | 25 | include/hw/riscv/boot.h | 1 + |
15 | +++ b/docs/devel/index-build.rst | 26 | 8 files changed, 27 insertions(+), 45 deletions(-) |
16 | @@ -XXX,XX +XXX,XX @@ some of the basics if you are adding new files and targets to the build. | 27 | |
17 | kconfig | 28 | diff --git a/hw/riscv/boot.c b/hw/riscv/boot.c |
18 | docs | 29 | index XXXXXXX..XXXXXXX 100644 |
19 | qapi-code-gen | 30 | --- a/hw/riscv/boot.c |
20 | + qapi-golang-code-gen | 31 | +++ b/hw/riscv/boot.c |
21 | control-flow-integrity | 32 | @@ -XXX,XX +XXX,XX @@ target_ulong riscv_load_firmware(const char *firmware_filename, |
22 | diff --git a/docs/devel/qapi-golang-code-gen.rst b/docs/devel/qapi-golang-code-gen.rst | 33 | target_ulong riscv_load_kernel(MachineState *machine, |
23 | new file mode 100644 | 34 | RISCVHartArrayState *harts, |
24 | index XXXXXXX..XXXXXXX | 35 | target_ulong kernel_start_addr, |
25 | --- /dev/null | 36 | + bool load_initrd, |
26 | +++ b/docs/devel/qapi-golang-code-gen.rst | 37 | symbol_fn_t sym_cb) |
27 | @@ -XXX,XX +XXX,XX @@ | 38 | { |
28 | +========================== | 39 | const char *kernel_filename = machine->kernel_filename; |
29 | +QAPI Golang code generator | 40 | uint64_t kernel_load_base, kernel_entry, kernel_low; |
30 | +========================== | 41 | + void *fdt = machine->fdt; |
42 | |||
43 | g_assert(kernel_filename != NULL); | ||
44 | |||
45 | @@ -XXX,XX +XXX,XX @@ target_ulong riscv_load_kernel(MachineState *machine, | ||
46 | kernel_entry = kernel_low; | ||
47 | } | ||
48 | |||
49 | - return kernel_entry; | ||
50 | + goto out; | ||
51 | } | ||
52 | |||
53 | if (load_uimage_as(kernel_filename, &kernel_entry, NULL, NULL, | ||
54 | NULL, NULL, NULL) > 0) { | ||
55 | - return kernel_entry; | ||
56 | + goto out; | ||
57 | } | ||
58 | |||
59 | if (load_image_targphys_as(kernel_filename, kernel_start_addr, | ||
60 | current_machine->ram_size, NULL) > 0) { | ||
61 | - return kernel_start_addr; | ||
62 | + kernel_entry = kernel_start_addr; | ||
63 | + goto out; | ||
64 | } | ||
65 | |||
66 | error_report("could not load kernel '%s'", kernel_filename); | ||
67 | exit(1); | ||
31 | + | 68 | + |
32 | +.. | 69 | +out: |
33 | + Copyright (C) 2025 Red Hat, Inc. | 70 | + if (load_initrd && machine->initrd_filename) { |
34 | + | 71 | + riscv_load_initrd(machine, kernel_entry); |
35 | + This work is licensed under the terms of the GNU GPL, version 2 or | ||
36 | + later. See the COPYING file in the top-level directory. | ||
37 | + | ||
38 | + | ||
39 | +Introduction | ||
40 | +============ | ||
41 | + | ||
42 | +This document provides information of how the generated Go code maps | ||
43 | +with the QAPI specification, clarifying design decisions when needed. | ||
44 | + | ||
45 | + | ||
46 | +Scope of the generated Go code | ||
47 | +============================== | ||
48 | + | ||
49 | +The scope is to provide data structures that can interpret and be used | ||
50 | +to generate valid QMP messages. These data structures are generated | ||
51 | +from a QAPI schema and should be able to handle QMP messages from the | ||
52 | +same schema. | ||
53 | + | ||
54 | +We also provide interfaces for Commands and Events which allows an | ||
55 | +abstraction for client and server applications with the possibility of | ||
56 | +custom back end implantations. | ||
57 | + | ||
58 | +The generated Go code is a Go module with data structs that uses Go | ||
59 | +standard library ``encoding/json``, implementing its field tags and | ||
60 | +Marshal interface whenever needed. | ||
61 | + | ||
62 | + | ||
63 | +QAPI Documentation | ||
64 | +================== | ||
65 | + | ||
66 | +The documentation included in QAPI schema such as type and type's | ||
67 | +fields information, comments, examples and more, they are converted | ||
68 | +and embed in the Go generated source code. Metadata information that | ||
69 | +might not be relevant to developers are excluded (e.g: TODOs) | ||
70 | + | ||
71 | + | ||
72 | +QAPI types to Go structs | ||
73 | +======================== | ||
74 | + | ||
75 | +Enum | ||
76 | +---- | ||
77 | + | ||
78 | +Enums are mapped as strings in Go, using a specified string type per | ||
79 | +Enum to help with type safety in the Go application. | ||
80 | + | ||
81 | +:: | ||
82 | + | ||
83 | + { 'enum': 'HostMemPolicy', | ||
84 | + 'data': [ 'default', 'preferred', 'bind', 'interleave' ] } | ||
85 | + | ||
86 | +.. code-block:: go | ||
87 | + | ||
88 | + // Host memory policy types | ||
89 | + // | ||
90 | + // Since: 2.1 | ||
91 | + type HostMemPolicy string | ||
92 | + | ||
93 | + const ( | ||
94 | + // restore default policy, remove any nondefault policy | ||
95 | + HostMemPolicyDefault HostMemPolicy = "default" | ||
96 | + // set the preferred host nodes for allocation | ||
97 | + HostMemPolicyPreferred HostMemPolicy = "preferred" | ||
98 | + // a strict policy that restricts memory allocation to the host nodes specified | ||
99 | + HostMemPolicyBind HostMemPolicy = "bind" | ||
100 | + // memory allocations are interleaved across the set of host nodes specified | ||
101 | + HostMemPolicyInterleave HostMemPolicy = "interleave" | ||
102 | + ) | ||
103 | + | ||
104 | + | ||
105 | +Struct | ||
106 | +------ | ||
107 | + | ||
108 | +The mapping between a QAPI struct in Go struct is very straightforward. | ||
109 | + - Each member of the QAPI struct has its own field in a Go struct. | ||
110 | + - Optional members are pointers type with 'omitempty' field tag set | ||
111 | + | ||
112 | +One important design decision was to _not_ embed base struct, copying | ||
113 | +the base members to the original struct. This reduces the complexity | ||
114 | +for the Go application. | ||
115 | + | ||
116 | +:: | ||
117 | + | ||
118 | + { 'struct': 'BlockExportOptionsNbdBase', | ||
119 | + 'data': { '*name': 'str', '*description': 'str' } } | ||
120 | + | ||
121 | + { 'struct': 'BlockExportOptionsNbd', | ||
122 | + 'base': 'BlockExportOptionsNbdBase', | ||
123 | + 'data': { '*bitmaps': ['BlockDirtyBitmapOrStr'], | ||
124 | + '*allocation-depth': 'bool' } } | ||
125 | + | ||
126 | +.. code-block:: go | ||
127 | + | ||
128 | + // An NBD block export (distinct options used in the NBD branch of | ||
129 | + // block-export-add). | ||
130 | + // | ||
131 | + // Since: 5.2 | ||
132 | + type BlockExportOptionsNbd struct { | ||
133 | + // Export name. If unspecified, the @device parameter is used as | ||
134 | + // the export name. (Since 2.12) | ||
135 | + Name *string `json:"name,omitempty"` | ||
136 | + // Free-form description of the export, up to 4096 bytes. (Since | ||
137 | + // 5.0) | ||
138 | + Description *string `json:"description,omitempty"` | ||
139 | + // Also export each of the named dirty bitmaps reachable from | ||
140 | + // @device, so the NBD client can use NBD_OPT_SET_META_CONTEXT | ||
141 | + // with the metadata context name "qemu:dirty-bitmap:BITMAP" to | ||
142 | + // inspect each bitmap. Since 7.1 bitmap may be specified by | ||
143 | + // node/name pair. | ||
144 | + Bitmaps []BlockDirtyBitmapOrStr `json:"bitmaps,omitempty"` | ||
145 | + // Also export the allocation depth map for @device, so the NBD | ||
146 | + // client can use NBD_OPT_SET_META_CONTEXT with the metadata | ||
147 | + // context name "qemu:allocation-depth" to inspect allocation | ||
148 | + // details. (since 5.2) | ||
149 | + AllocationDepth *bool `json:"allocation-depth,omitempty"` | ||
150 | + } | 72 | + } |
151 | + | 73 | + |
152 | + | 74 | + if (fdt && machine->kernel_cmdline && *machine->kernel_cmdline) { |
153 | +Union | 75 | + qemu_fdt_setprop_string(fdt, "/chosen", "bootargs", |
154 | +----- | 76 | + machine->kernel_cmdline); |
155 | + | ||
156 | +Unions in QAPI are bounded to a Enum type which provides all possible | ||
157 | +branches of the union. The most important caveat here is that the Union | ||
158 | +does not need to have a complex type implemented for all possible | ||
159 | +branches of the Enum. Receiving a enum value of a empty branch is valid. | ||
160 | + | ||
161 | +The generated Go struct will then define a field for each | ||
162 | +Enum value. The type for Enum values of empty branch is bool. Only one | ||
163 | +field can be set at time. | ||
164 | + | ||
165 | +:: | ||
166 | + | ||
167 | + { 'union': 'ImageInfoSpecificQCow2Encryption', | ||
168 | + 'base': 'ImageInfoSpecificQCow2EncryptionBase', | ||
169 | + 'discriminator': 'format', | ||
170 | + 'data': { 'luks': 'QCryptoBlockInfoLUKS' } } | ||
171 | + | ||
172 | + { 'struct': 'ImageInfoSpecificQCow2EncryptionBase', | ||
173 | + 'data': { 'format': 'BlockdevQcow2EncryptionFormat'}} | ||
174 | + | ||
175 | + { 'enum': 'BlockdevQcow2EncryptionFormat', | ||
176 | + 'data': [ 'aes', 'luks' ] } | ||
177 | + | ||
178 | +.. code-block:: go | ||
179 | + | ||
180 | + type ImageInfoSpecificQCow2Encryption struct { | ||
181 | + // Variants fields | ||
182 | + Luks *QCryptoBlockInfoLUKS `json:"-"` | ||
183 | + // Empty branched enum fields | ||
184 | + Aes bool `json:"-"` | ||
185 | + } | 77 | + } |
186 | + | 78 | + |
187 | + func (s ImageInfoSpecificQCow2Encryption) MarshalJSON() ([]byte, error) { | 79 | + return kernel_entry; |
188 | + // ... | 80 | } |
189 | + // Logic for branched Enum | 81 | |
190 | + if s.Luks != nil && err == nil { | 82 | void riscv_load_initrd(MachineState *machine, uint64_t kernel_entry) |
191 | + if len(bytes) != 0 { | 83 | diff --git a/hw/riscv/microchip_pfsoc.c b/hw/riscv/microchip_pfsoc.c |
192 | + err = errors.New(`multiple variant fields set`) | 84 | index XXXXXXX..XXXXXXX 100644 |
193 | + } else if err = unwrapToMap(m, s.Luks); err == nil { | 85 | --- a/hw/riscv/microchip_pfsoc.c |
194 | + m["format"] = BlockdevQcow2EncryptionFormatLuks | 86 | +++ b/hw/riscv/microchip_pfsoc.c |
195 | + bytes, err = json.Marshal(m) | 87 | @@ -XXX,XX +XXX,XX @@ static void microchip_icicle_kit_machine_init(MachineState *machine) |
196 | + } | 88 | firmware_end_addr); |
197 | + } | 89 | |
198 | + | 90 | kernel_entry = riscv_load_kernel(machine, &s->soc.u_cpus, |
199 | + // Logic for unbranched Enum | 91 | - kernel_start_addr, NULL); |
200 | + if s.Aes && err == nil { | 92 | - |
201 | + if len(bytes) != 0 { | 93 | - if (machine->initrd_filename) { |
202 | + err = errors.New(`multiple variant fields set`) | 94 | - riscv_load_initrd(machine, kernel_entry); |
203 | + } else { | 95 | - } |
204 | + m["format"] = BlockdevQcow2EncryptionFormatAes | 96 | - |
205 | + bytes, err = json.Marshal(m) | 97 | - if (machine->kernel_cmdline && *machine->kernel_cmdline) { |
206 | + } | 98 | - qemu_fdt_setprop_string(machine->fdt, "/chosen", |
207 | + } | 99 | - "bootargs", machine->kernel_cmdline); |
208 | + | 100 | - } |
209 | + // ... | 101 | + kernel_start_addr, true, NULL); |
210 | + // Handle errors | 102 | |
211 | + } | 103 | /* Compute the fdt load address in dram */ |
212 | + | 104 | fdt_load_addr = riscv_compute_fdt_addr(memmap[MICROCHIP_PFSOC_DRAM_LO].base, |
213 | + | 105 | diff --git a/hw/riscv/opentitan.c b/hw/riscv/opentitan.c |
214 | + func (s *ImageInfoSpecificQCow2Encryption) UnmarshalJSON(data []byte) error { | 106 | index XXXXXXX..XXXXXXX 100644 |
215 | + // ... | 107 | --- a/hw/riscv/opentitan.c |
216 | + | 108 | +++ b/hw/riscv/opentitan.c |
217 | + switch tmp.Format { | 109 | @@ -XXX,XX +XXX,XX @@ static void opentitan_board_init(MachineState *machine) |
218 | + case BlockdevQcow2EncryptionFormatLuks: | 110 | |
219 | + s.Luks = new(QCryptoBlockInfoLUKS) | 111 | if (machine->kernel_filename) { |
220 | + if err := json.Unmarshal(data, s.Luks); err != nil { | 112 | riscv_load_kernel(machine, &s->soc.cpus, |
221 | + s.Luks = nil | 113 | - memmap[IBEX_DEV_RAM].base, NULL); |
222 | + return err | 114 | + memmap[IBEX_DEV_RAM].base, |
223 | + } | 115 | + false, NULL); |
224 | + case BlockdevQcow2EncryptionFormatAes: | 116 | } |
225 | + s.Aes = true | 117 | } |
226 | + | 118 | |
227 | + default: | 119 | diff --git a/hw/riscv/sifive_e.c b/hw/riscv/sifive_e.c |
228 | + return fmt.Errorf("error: unmarshal: ImageInfoSpecificQCow2Encryption: received unrecognized value: '%s'", | 120 | index XXXXXXX..XXXXXXX 100644 |
229 | + tmp.Format) | 121 | --- a/hw/riscv/sifive_e.c |
230 | + } | 122 | +++ b/hw/riscv/sifive_e.c |
231 | + return nil | 123 | @@ -XXX,XX +XXX,XX @@ static void sifive_e_machine_init(MachineState *machine) |
232 | + } | 124 | |
233 | + | 125 | if (machine->kernel_filename) { |
234 | + | 126 | riscv_load_kernel(machine, &s->soc.cpus, |
235 | +Alternate | 127 | - memmap[SIFIVE_E_DEV_DTIM].base, NULL); |
236 | +--------- | 128 | + memmap[SIFIVE_E_DEV_DTIM].base, |
237 | + | 129 | + false, NULL); |
238 | +Like Unions, alternates can have branches. Unlike Unions, they don't | 130 | } |
239 | +have a discriminator field and each branch should be a different class | 131 | } |
240 | +of Type entirely (e.g: You can't have two branches of type int in one | 132 | |
241 | +Alternate). | 133 | diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c |
242 | + | 134 | index XXXXXXX..XXXXXXX 100644 |
243 | +While the marshalling is similar to Unions, the unmarshalling uses a | 135 | --- a/hw/riscv/sifive_u.c |
244 | +try-and-error approach, trying to fit the data payload in one of the | 136 | +++ b/hw/riscv/sifive_u.c |
245 | +Alternate fields. | 137 | @@ -XXX,XX +XXX,XX @@ static void sifive_u_machine_init(MachineState *machine) |
246 | + | 138 | firmware_end_addr); |
247 | +The biggest caveat is handling Alternates that can take JSON Null as | 139 | |
248 | +value. The issue lies on ``encoding/json`` library limitation where | 140 | kernel_entry = riscv_load_kernel(machine, &s->soc.u_cpus, |
249 | +unmarshalling JSON Null data to a Go struct which has the 'omitempty' | 141 | - kernel_start_addr, NULL); |
250 | +field as it will bypass the Marshal interface. The same happens when | 142 | - |
251 | +marshalling, if the field tag 'omitempty' is used, a nil pointer would | 143 | - if (machine->initrd_filename) { |
252 | +never be translated to null JSON value. The problem here is that we do | 144 | - riscv_load_initrd(machine, kernel_entry); |
253 | +use pointer to type plus ``omitempty`` field to express a QAPI | 145 | - } |
254 | +optional member. | 146 | - |
255 | + | 147 | - if (machine->kernel_cmdline && *machine->kernel_cmdline) { |
256 | +In order to handle JSON Null, the generator needs to do the following: | 148 | - qemu_fdt_setprop_string(machine->fdt, "/chosen", "bootargs", |
257 | + - Read the QAPI schema prior to generate any code and cache | 149 | - machine->kernel_cmdline); |
258 | + all alternate types that can take JSON Null | 150 | - } |
259 | + - For all Go structs that should be considered optional and they type | 151 | + kernel_start_addr, true, NULL); |
260 | + are one of those alternates, do not set ``omitempty`` and implement | 152 | } else { |
261 | + Marshal interface for this Go struct, to properly handle JSON Null | 153 | /* |
262 | + - In the Alternate, uses a boolean 'IsNull' to express a JSON Null | 154 | * If dynamic firmware is used, it doesn't know where is the next mode |
263 | + and implement the AbsentAlternate interface, to help structs know | 155 | diff --git a/hw/riscv/spike.c b/hw/riscv/spike.c |
264 | + if a given Alternate type should be considered Absent (not set) or | 156 | index XXXXXXX..XXXXXXX 100644 |
265 | + any other possible Value, including JSON Null. | 157 | --- a/hw/riscv/spike.c |
266 | + | 158 | +++ b/hw/riscv/spike.c |
267 | +:: | 159 | @@ -XXX,XX +XXX,XX @@ static void spike_board_init(MachineState *machine) |
268 | + | 160 | |
269 | + { 'alternate': 'BlockdevRefOrNull', | 161 | kernel_entry = riscv_load_kernel(machine, &s->soc[0], |
270 | + 'data': { 'definition': 'BlockdevOptions', | 162 | kernel_start_addr, |
271 | + 'reference': 'str', | 163 | - htif_symbol_callback); |
272 | + 'null': 'null' } } | 164 | - |
273 | + | 165 | - if (machine->initrd_filename) { |
274 | +.. code-block:: go | 166 | - riscv_load_initrd(machine, kernel_entry); |
275 | + | 167 | - } |
276 | + // Reference to a block device. | 168 | - |
277 | + // | 169 | - if (machine->kernel_cmdline && *machine->kernel_cmdline) { |
278 | + // Since: 2.9 | 170 | - qemu_fdt_setprop_string(machine->fdt, "/chosen", "bootargs", |
279 | + type BlockdevRefOrNull struct { | 171 | - machine->kernel_cmdline); |
280 | + // defines a new block device inline | 172 | - } |
281 | + Definition *BlockdevOptions | 173 | + true, htif_symbol_callback); |
282 | + // references the ID of an existing block device. An empty string | 174 | } else { |
283 | + // means that no block device should be referenced. Deprecated; | 175 | /* |
284 | + // use null instead. | 176 | * If dynamic firmware is used, it doesn't know where is the next mode |
285 | + Reference *string | 177 | diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c |
286 | + // No block device should be referenced (since 2.10) | 178 | index XXXXXXX..XXXXXXX 100644 |
287 | + IsNull bool | 179 | --- a/hw/riscv/virt.c |
288 | + } | 180 | +++ b/hw/riscv/virt.c |
289 | + | 181 | @@ -XXX,XX +XXX,XX @@ static void virt_machine_done(Notifier *notifier, void *data) |
290 | + func (s *BlockdevRefOrNull) ToAnyOrAbsent() (any, bool) { | 182 | firmware_end_addr); |
291 | + if s != nil { | 183 | |
292 | + if s.IsNull { | 184 | kernel_entry = riscv_load_kernel(machine, &s->soc[0], |
293 | + return nil, false | 185 | - kernel_start_addr, NULL); |
294 | + } else if s.Definition != nil { | 186 | - |
295 | + return *s.Definition, false | 187 | - if (machine->initrd_filename) { |
296 | + } else if s.Reference != nil { | 188 | - riscv_load_initrd(machine, kernel_entry); |
297 | + return *s.Reference, false | 189 | - } |
298 | + } | 190 | - |
299 | + } | 191 | - if (machine->kernel_cmdline && *machine->kernel_cmdline) { |
300 | + | 192 | - qemu_fdt_setprop_string(machine->fdt, "/chosen", "bootargs", |
301 | + return nil, true | 193 | - machine->kernel_cmdline); |
302 | + } | 194 | - } |
303 | + | 195 | + kernel_start_addr, true, NULL); |
304 | + func (s BlockdevRefOrNull) MarshalJSON() ([]byte, error) { | 196 | } else { |
305 | + if s.IsNull { | 197 | /* |
306 | + return []byte("null"), nil | 198 | * If dynamic firmware is used, it doesn't know where is the next mode |
307 | + } else if s.Definition != nil { | 199 | diff --git a/include/hw/riscv/boot.h b/include/hw/riscv/boot.h |
308 | + return json.Marshal(s.Definition) | 200 | index XXXXXXX..XXXXXXX 100644 |
309 | + } else if s.Reference != nil { | 201 | --- a/include/hw/riscv/boot.h |
310 | + return json.Marshal(s.Reference) | 202 | +++ b/include/hw/riscv/boot.h |
311 | + } | 203 | @@ -XXX,XX +XXX,XX @@ target_ulong riscv_load_firmware(const char *firmware_filename, |
312 | + return []byte("{}"), nil | 204 | target_ulong riscv_load_kernel(MachineState *machine, |
313 | + } | 205 | RISCVHartArrayState *harts, |
314 | + | 206 | target_ulong firmware_end_addr, |
315 | + func (s *BlockdevRefOrNull) UnmarshalJSON(data []byte) error { | 207 | + bool load_initrd, |
316 | + // Check for json-null first | 208 | symbol_fn_t sym_cb); |
317 | + if string(data) == "null" { | 209 | void riscv_load_initrd(MachineState *machine, uint64_t kernel_entry); |
318 | + s.IsNull = true | 210 | uint64_t riscv_compute_fdt_addr(hwaddr dram_start, uint64_t dram_size, |
319 | + return nil | ||
320 | + } | ||
321 | + // Check for BlockdevOptions | ||
322 | + { | ||
323 | + s.Definition = new(BlockdevOptions) | ||
324 | + if err := StrictDecode(s.Definition, data); err == nil { | ||
325 | + return nil | ||
326 | + } | ||
327 | + s.Definition = nil | ||
328 | + } | ||
329 | + | ||
330 | + // Check for string | ||
331 | + { | ||
332 | + s.Reference = new(string) | ||
333 | + if err := StrictDecode(s.Reference, data); err == nil { | ||
334 | + return nil | ||
335 | + } | ||
336 | + s.Reference = nil | ||
337 | + } | ||
338 | + | ||
339 | + return fmt.Errorf("Can't convert to BlockdevRefOrNull: %s", string(data)) | ||
340 | + } | ||
341 | + | ||
342 | + | ||
343 | +Event | ||
344 | +----- | ||
345 | + | ||
346 | +Each event is mapped to its own struct with. | ||
347 | + | ||
348 | +:: | ||
349 | + | ||
350 | + { 'event': 'SHUTDOWN', | ||
351 | + 'data': { 'guest': 'bool', | ||
352 | + 'reason': 'ShutdownCause' } } | ||
353 | + | ||
354 | +.. code-block:: go | ||
355 | + | ||
356 | + // Emitted when the virtual machine has shut down, indicating that | ||
357 | + // qemu is about to exit. | ||
358 | + // | ||
359 | + // .. note:: If the command-line option "-no-shutdown" has been | ||
360 | + // specified, qemu will not exit, and a STOP event will eventually | ||
361 | + // follow the SHUTDOWN event. | ||
362 | + // | ||
363 | + // Since: 0.12 | ||
364 | + // | ||
365 | + // .. qmp-example:: <- { "event": "SHUTDOWN", "data": { | ||
366 | + // "guest": true, "reason": "guest-shutdown" }, "timestamp": { | ||
367 | + // "seconds": 1267040730, "microseconds": 682951 } } | ||
368 | + type ShutdownEvent struct { | ||
369 | + // If true, the shutdown was triggered by a guest request (such as | ||
370 | + // a guest-initiated ACPI shutdown request or other hardware- | ||
371 | + // specific action) rather than a host request (such as sending | ||
372 | + // qemu a SIGINT). (since 2.10) | ||
373 | + Guest bool `json:"guest"` | ||
374 | + // The @ShutdownCause which resulted in the SHUTDOWN. (since 4.0) | ||
375 | + Reason ShutdownCause `json:"reason"` | ||
376 | + } | ||
377 | + | ||
378 | + | ||
379 | +Command | ||
380 | +------- | ||
381 | + | ||
382 | +Each commands is mapped to its own struct. If the command has a boxed | ||
383 | +data struct, the option struct will be embed in the command struct. | ||
384 | + | ||
385 | +The return value is always a well defined type and it is part of first | ||
386 | +layer unmarshalling type, Message. | ||
387 | + | ||
388 | +:: | ||
389 | + | ||
390 | + { 'command': 'set_password', | ||
391 | + 'boxed': true, | ||
392 | + 'data': 'SetPasswordOptions' } | ||
393 | + | ||
394 | + { 'union': 'SetPasswordOptions', | ||
395 | + 'base': { 'protocol': 'DisplayProtocol', | ||
396 | + 'password': 'str', | ||
397 | + '*connected': 'SetPasswordAction' }, | ||
398 | + 'discriminator': 'protocol', | ||
399 | + 'data': { 'vnc': 'SetPasswordOptionsVnc' } } | ||
400 | + | ||
401 | +.. code-block:: go | ||
402 | + | ||
403 | + // Set the password of a remote display server. | ||
404 | + // Errors: - If Spice is not enabled, DeviceNotFound | ||
405 | + // | ||
406 | + // Since: 0.14 | ||
407 | + // | ||
408 | + // .. qmp-example:: -> { "execute": "set_password", "arguments": { | ||
409 | + // "protocol": "vnc", "password": "secret" } | ||
410 | + // } <- { "return": {} } | ||
411 | + type SetPasswordCommand struct { | ||
412 | + SetPasswordOptions | ||
413 | + } | ||
414 | + | ||
415 | +Now an example of a command without boxed type. | ||
416 | + | ||
417 | +:: | ||
418 | + | ||
419 | + { 'command': 'set_link', | ||
420 | + 'data': {'name': 'str', 'up': 'bool'} } | ||
421 | + | ||
422 | +.. code-block:: go | ||
423 | + | ||
424 | + // Sets the link status of a virtual network adapter. | ||
425 | + // | ||
426 | + // Errors: - If @name is not a valid network device, DeviceNotFound | ||
427 | + // | ||
428 | + // Since: 0.14 | ||
429 | + // | ||
430 | + // .. note:: Not all network adapters support setting link status. | ||
431 | + // This command will succeed even if the network adapter does not | ||
432 | + // support link status notification. .. qmp-example:: -> { | ||
433 | + // "execute": "set_link", "arguments": { "name": "e1000.0", "up": | ||
434 | + // false } } <- { "return": {} } | ||
435 | + type SetLinkCommand struct { | ||
436 | + // the device name of the virtual network adapter | ||
437 | + Name string `json:"name"` | ||
438 | + // true to set the link status to be up | ||
439 | + Up bool `json:"up"` | ||
440 | + } | ||
441 | + | ||
442 | +Known issues | ||
443 | +============ | ||
444 | + | ||
445 | +- Type names might not follow proper Go convention. Andrea suggested an | ||
446 | + annotation to the QAPI schema that could solve it. | ||
447 | + https://lists.gnu.org/archive/html/qemu-devel/2022-05/msg00127.html | ||
448 | -- | 211 | -- |
449 | 2.48.1 | 212 | 2.39.1 | diff view generated by jsdifflib |
1 | This first patch introduces protocol.go. It introduces the Message Go | 1 | The only remaining caller is riscv_load_kernel_and_initrd() which |
---|---|---|---|
2 | struct type that can unmarshall any QMP message. | 2 | belongs to the same file. |
3 | 3 | ||
4 | It does not handle deeper than 1st layer of the JSON object, that is, | 4 | Signed-off-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> |
5 | with: | 5 | Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> |
6 | Reviewed-by: Bin Meng <bmeng@tinylab.org> | ||
7 | Reviewed-by: Alistair Francis <alistair.francis@wdc.com> | ||
8 | --- | ||
9 | hw/riscv/boot.c | 80 ++++++++++++++++++++--------------------- | ||
10 | include/hw/riscv/boot.h | 1 - | ||
11 | 2 files changed, 40 insertions(+), 41 deletions(-) | ||
6 | 12 | ||
7 | 1. { | 13 | diff --git a/hw/riscv/boot.c b/hw/riscv/boot.c |
8 | "execute": "query-machines", | 14 | index XXXXXXX..XXXXXXX 100644 |
9 | "arguments": { "compat-props": true } | 15 | --- a/hw/riscv/boot.c |
10 | } | 16 | +++ b/hw/riscv/boot.c |
11 | 17 | @@ -XXX,XX +XXX,XX @@ target_ulong riscv_load_firmware(const char *firmware_filename, | |
12 | 2. { | 18 | exit(1); |
13 | "event": "BALLOON_CHANGE", | 19 | } |
14 | "data": { "actual": 944766976 }, | 20 | |
15 | "timestamp": { | 21 | +static void riscv_load_initrd(MachineState *machine, uint64_t kernel_entry) |
16 | "seconds": 1267020223, | 22 | +{ |
17 | "microseconds": 435656 | 23 | + const char *filename = machine->initrd_filename; |
18 | } | 24 | + uint64_t mem_size = machine->ram_size; |
19 | } | 25 | + void *fdt = machine->fdt; |
20 | 26 | + hwaddr start, end; | |
21 | We will be able to know it is a query-machine command or a | 27 | + ssize_t size; |
22 | balloon-change event. Specific data type to handle arguments/data will | ||
23 | be introduced further in the series. | ||
24 | |||
25 | This patch also introduces the Visitor skeleton with a proper write() | ||
26 | function to copy-over the protocol.go to the target destination. | ||
27 | |||
28 | Note, you can execute any patch of this series with: | ||
29 | |||
30 | python3 ./scripts/qapi-gen.py -o /tmp/out qapi/qapi-schema.json | ||
31 | |||
32 | Signed-off-by: Victor Toso <victortoso@redhat.com> | ||
33 | --- | ||
34 | scripts/qapi/golang/__init__.py | 0 | ||
35 | scripts/qapi/golang/golang.py | 135 ++++++++++++++++++++++++++++++++ | ||
36 | scripts/qapi/golang/protocol.go | 48 ++++++++++++ | ||
37 | scripts/qapi/main.py | 2 + | ||
38 | 4 files changed, 185 insertions(+) | ||
39 | create mode 100644 scripts/qapi/golang/__init__.py | ||
40 | create mode 100644 scripts/qapi/golang/golang.py | ||
41 | create mode 100644 scripts/qapi/golang/protocol.go | ||
42 | |||
43 | diff --git a/scripts/qapi/golang/__init__.py b/scripts/qapi/golang/__init__.py | ||
44 | new file mode 100644 | ||
45 | index XXXXXXX..XXXXXXX | ||
46 | diff --git a/scripts/qapi/golang/golang.py b/scripts/qapi/golang/golang.py | ||
47 | new file mode 100644 | ||
48 | index XXXXXXX..XXXXXXX | ||
49 | --- /dev/null | ||
50 | +++ b/scripts/qapi/golang/golang.py | ||
51 | @@ -XXX,XX +XXX,XX @@ | ||
52 | +""" | ||
53 | +Golang QAPI generator | ||
54 | +""" | ||
55 | + | 28 | + |
56 | +# Copyright (c) 2025 Red Hat Inc. | 29 | + g_assert(filename != NULL); |
57 | +# | ||
58 | +# Authors: | ||
59 | +# Victor Toso <victortoso@redhat.com> | ||
60 | +# | ||
61 | +# This work is licensed under the terms of the GNU GPL, version 2. | ||
62 | +# See the COPYING file in the top-level directory. | ||
63 | + | 30 | + |
64 | +# Just for type hint on self | 31 | + /* |
65 | +from __future__ import annotations | 32 | + * We want to put the initrd far enough into RAM that when the |
33 | + * kernel is uncompressed it will not clobber the initrd. However | ||
34 | + * on boards without much RAM we must ensure that we still leave | ||
35 | + * enough room for a decent sized initrd, and on boards with large | ||
36 | + * amounts of RAM we must avoid the initrd being so far up in RAM | ||
37 | + * that it is outside lowmem and inaccessible to the kernel. | ||
38 | + * So for boards with less than 256MB of RAM we put the initrd | ||
39 | + * halfway into RAM, and for boards with 256MB of RAM or more we put | ||
40 | + * the initrd at 128MB. | ||
41 | + */ | ||
42 | + start = kernel_entry + MIN(mem_size / 2, 128 * MiB); | ||
66 | + | 43 | + |
67 | +import os, shutil | 44 | + size = load_ramdisk(filename, start, mem_size - start); |
68 | +from typing import List, Optional | 45 | + if (size == -1) { |
46 | + size = load_image_targphys(filename, start, mem_size - start); | ||
47 | + if (size == -1) { | ||
48 | + error_report("could not load ramdisk '%s'", filename); | ||
49 | + exit(1); | ||
50 | + } | ||
51 | + } | ||
69 | + | 52 | + |
70 | +from ..schema import ( | 53 | + /* Some RISC-V machines (e.g. opentitan) don't have a fdt. */ |
71 | + QAPISchema, | 54 | + if (fdt) { |
72 | + QAPISchemaBranches, | 55 | + end = start + size; |
73 | + QAPISchemaEnumMember, | 56 | + qemu_fdt_setprop_cell(fdt, "/chosen", "linux,initrd-start", start); |
74 | + QAPISchemaFeature, | 57 | + qemu_fdt_setprop_cell(fdt, "/chosen", "linux,initrd-end", end); |
75 | + QAPISchemaIfCond, | 58 | + } |
76 | + QAPISchemaObjectType, | ||
77 | + QAPISchemaObjectTypeMember, | ||
78 | + QAPISchemaType, | ||
79 | + QAPISchemaVariants, | ||
80 | + QAPISchemaVisitor, | ||
81 | +) | ||
82 | +from ..source import QAPISourceInfo | ||
83 | + | ||
84 | + | ||
85 | +def gen_golang(schema: QAPISchema, output_dir: str, prefix: str) -> None: | ||
86 | + vis = QAPISchemaGenGolangVisitor(prefix) | ||
87 | + schema.visit(vis) | ||
88 | + vis.write(output_dir) | ||
89 | + | ||
90 | + | ||
91 | +class QAPISchemaGenGolangVisitor(QAPISchemaVisitor): | ||
92 | + # pylint: disable=too-many-arguments | ||
93 | + def __init__(self, _: str): | ||
94 | + super().__init__() | ||
95 | + gofiles = ("protocol.go",) | ||
96 | + self.schema: QAPISchema | ||
97 | + self.golang_package_name = "qapi" | ||
98 | + self.duplicate = list(gofiles) | ||
99 | + | ||
100 | + def visit_begin(self, schema: QAPISchema) -> None: | ||
101 | + self.schema = schema | ||
102 | + | ||
103 | + def visit_end(self) -> None: | ||
104 | + del self.schema | ||
105 | + | ||
106 | + def visit_object_type( | ||
107 | + self, | ||
108 | + name: str, | ||
109 | + info: Optional[QAPISourceInfo], | ||
110 | + ifcond: QAPISchemaIfCond, | ||
111 | + features: List[QAPISchemaFeature], | ||
112 | + base: Optional[QAPISchemaObjectType], | ||
113 | + members: List[QAPISchemaObjectTypeMember], | ||
114 | + branches: Optional[QAPISchemaBranches], | ||
115 | + ) -> None: | ||
116 | + pass | ||
117 | + | ||
118 | + def visit_alternate_type( | ||
119 | + self, | ||
120 | + name: str, | ||
121 | + info: Optional[QAPISourceInfo], | ||
122 | + ifcond: QAPISchemaIfCond, | ||
123 | + features: List[QAPISchemaFeature], | ||
124 | + variants: QAPISchemaVariants, | ||
125 | + ) -> None: | ||
126 | + pass | ||
127 | + | ||
128 | + def visit_enum_type( | ||
129 | + self, | ||
130 | + name: str, | ||
131 | + info: Optional[QAPISourceInfo], | ||
132 | + ifcond: QAPISchemaIfCond, | ||
133 | + features: List[QAPISchemaFeature], | ||
134 | + members: List[QAPISchemaEnumMember], | ||
135 | + prefix: Optional[str], | ||
136 | + ) -> None: | ||
137 | + pass | ||
138 | + | ||
139 | + def visit_array_type( | ||
140 | + self, | ||
141 | + name: str, | ||
142 | + info: Optional[QAPISourceInfo], | ||
143 | + ifcond: QAPISchemaIfCond, | ||
144 | + element_type: QAPISchemaType, | ||
145 | + ) -> None: | ||
146 | + pass | ||
147 | + | ||
148 | + def visit_command( | ||
149 | + self, | ||
150 | + name: str, | ||
151 | + info: Optional[QAPISourceInfo], | ||
152 | + ifcond: QAPISchemaIfCond, | ||
153 | + features: List[QAPISchemaFeature], | ||
154 | + arg_type: Optional[QAPISchemaObjectType], | ||
155 | + ret_type: Optional[QAPISchemaType], | ||
156 | + gen: bool, | ||
157 | + success_response: bool, | ||
158 | + boxed: bool, | ||
159 | + allow_oob: bool, | ||
160 | + allow_preconfig: bool, | ||
161 | + coroutine: bool, | ||
162 | + ) -> None: | ||
163 | + pass | ||
164 | + | ||
165 | + def visit_event( | ||
166 | + self, | ||
167 | + name: str, | ||
168 | + info: Optional[QAPISourceInfo], | ||
169 | + ifcond: QAPISchemaIfCond, | ||
170 | + features: List[QAPISchemaFeature], | ||
171 | + arg_type: Optional[QAPISchemaObjectType], | ||
172 | + boxed: bool, | ||
173 | + ) -> None: | ||
174 | + pass | ||
175 | + | ||
176 | + def write(self, outdir: str) -> None: | ||
177 | + godir = "go" | ||
178 | + targetpath = os.path.join(outdir, godir) | ||
179 | + os.makedirs(targetpath, exist_ok=True) | ||
180 | + | ||
181 | + # Content to be copied over | ||
182 | + srcdir = os.path.dirname(os.path.realpath(__file__)) | ||
183 | + for filename in self.duplicate: | ||
184 | + srcpath = os.path.join(srcdir, filename) | ||
185 | + dstpath = os.path.join(targetpath, filename) | ||
186 | + shutil.copyfile(srcpath, dstpath) | ||
187 | diff --git a/scripts/qapi/golang/protocol.go b/scripts/qapi/golang/protocol.go | ||
188 | new file mode 100644 | ||
189 | index XXXXXXX..XXXXXXX | ||
190 | --- /dev/null | ||
191 | +++ b/scripts/qapi/golang/protocol.go | ||
192 | @@ -XXX,XX +XXX,XX @@ | ||
193 | +/* | ||
194 | + * Copyright 2025 Red Hat, Inc. | ||
195 | + * SPDX-License-Identifier: MIT-0 | ||
196 | + * | ||
197 | + * Authors: | ||
198 | + * Victor Toso <victortoso@redhat.com> | ||
199 | + * Daniel P. Berrange <berrange@redhat.com> | ||
200 | + */ | ||
201 | +package qapi | ||
202 | + | ||
203 | +import ( | ||
204 | + "encoding/json" | ||
205 | + "time" | ||
206 | +) | ||
207 | + | ||
208 | +/* Union of data for command, response, error, or event, | ||
209 | + * since when receiving we don't know upfront which we | ||
210 | + * must deserialize */ | ||
211 | +type Message struct { | ||
212 | + QMP *json.RawMessage `json:"QMP,omitempty"` | ||
213 | + Execute string `json:"execute,omitempty"` | ||
214 | + ExecOOB string `json:"exec-oob,omitempty"` | ||
215 | + Event string `json:"event,omitempty"` | ||
216 | + Error *json.RawMessage `json:"error,omitempty"` | ||
217 | + Return *json.RawMessage `json:"return,omitempty"` | ||
218 | + ID string `json:"id,omitempty"` | ||
219 | + Timestamp *Timestamp `json:"timestamp,omitempty"` | ||
220 | + Data *json.RawMessage `json:"data,omitempty"` | ||
221 | + Arguments *json.RawMessage `json:"arguments,omitempty"` | ||
222 | +} | 59 | +} |
223 | + | 60 | + |
224 | +type QAPIError struct { | 61 | target_ulong riscv_load_kernel(MachineState *machine, |
225 | + Class string `json:"class"` | 62 | RISCVHartArrayState *harts, |
226 | + Description string `json:"desc"` | 63 | target_ulong kernel_start_addr, |
227 | +} | 64 | @@ -XXX,XX +XXX,XX @@ out: |
228 | + | 65 | return kernel_entry; |
229 | +func (err *QAPIError) Error() string { | 66 | } |
230 | + return err.Description | 67 | |
231 | +} | 68 | -void riscv_load_initrd(MachineState *machine, uint64_t kernel_entry) |
232 | + | 69 | -{ |
233 | +type Timestamp struct { | 70 | - const char *filename = machine->initrd_filename; |
234 | + Seconds int `json:"seconds"` | 71 | - uint64_t mem_size = machine->ram_size; |
235 | + MicroSeconds int `json:"microseconds"` | 72 | - void *fdt = machine->fdt; |
236 | +} | 73 | - hwaddr start, end; |
237 | + | 74 | - ssize_t size; |
238 | +func (t *Timestamp) AsTime() time.Time { | 75 | - |
239 | + return time.Unix(int64(t.Seconds), int64(t.MicroSeconds)*1000) | 76 | - g_assert(filename != NULL); |
240 | +} | 77 | - |
241 | diff --git a/scripts/qapi/main.py b/scripts/qapi/main.py | 78 | - /* |
79 | - * We want to put the initrd far enough into RAM that when the | ||
80 | - * kernel is uncompressed it will not clobber the initrd. However | ||
81 | - * on boards without much RAM we must ensure that we still leave | ||
82 | - * enough room for a decent sized initrd, and on boards with large | ||
83 | - * amounts of RAM we must avoid the initrd being so far up in RAM | ||
84 | - * that it is outside lowmem and inaccessible to the kernel. | ||
85 | - * So for boards with less than 256MB of RAM we put the initrd | ||
86 | - * halfway into RAM, and for boards with 256MB of RAM or more we put | ||
87 | - * the initrd at 128MB. | ||
88 | - */ | ||
89 | - start = kernel_entry + MIN(mem_size / 2, 128 * MiB); | ||
90 | - | ||
91 | - size = load_ramdisk(filename, start, mem_size - start); | ||
92 | - if (size == -1) { | ||
93 | - size = load_image_targphys(filename, start, mem_size - start); | ||
94 | - if (size == -1) { | ||
95 | - error_report("could not load ramdisk '%s'", filename); | ||
96 | - exit(1); | ||
97 | - } | ||
98 | - } | ||
99 | - | ||
100 | - /* Some RISC-V machines (e.g. opentitan) don't have a fdt. */ | ||
101 | - if (fdt) { | ||
102 | - end = start + size; | ||
103 | - qemu_fdt_setprop_cell(fdt, "/chosen", "linux,initrd-start", start); | ||
104 | - qemu_fdt_setprop_cell(fdt, "/chosen", "linux,initrd-end", end); | ||
105 | - } | ||
106 | -} | ||
107 | - | ||
108 | /* | ||
109 | * This function makes an assumption that the DRAM interval | ||
110 | * 'dram_base' + 'dram_size' is contiguous. | ||
111 | diff --git a/include/hw/riscv/boot.h b/include/hw/riscv/boot.h | ||
242 | index XXXXXXX..XXXXXXX 100644 | 112 | index XXXXXXX..XXXXXXX 100644 |
243 | --- a/scripts/qapi/main.py | 113 | --- a/include/hw/riscv/boot.h |
244 | +++ b/scripts/qapi/main.py | 114 | +++ b/include/hw/riscv/boot.h |
245 | @@ -XXX,XX +XXX,XX @@ | 115 | @@ -XXX,XX +XXX,XX @@ target_ulong riscv_load_kernel(MachineState *machine, |
246 | from .error import QAPIError | 116 | target_ulong firmware_end_addr, |
247 | from .events import gen_events | 117 | bool load_initrd, |
248 | from .features import gen_features | 118 | symbol_fn_t sym_cb); |
249 | +from .golang import golang | 119 | -void riscv_load_initrd(MachineState *machine, uint64_t kernel_entry); |
250 | from .introspect import gen_introspect | 120 | uint64_t riscv_compute_fdt_addr(hwaddr dram_start, uint64_t dram_size, |
251 | from .schema import QAPISchema | 121 | MachineState *ms); |
252 | from .types import gen_types | 122 | void riscv_load_fdt(hwaddr fdt_addr, void *fdt); |
253 | @@ -XXX,XX +XXX,XX @@ def generate(schema_file: str, | ||
254 | gen_commands(schema, output_dir, prefix, gen_tracing) | ||
255 | gen_events(schema, output_dir, prefix) | ||
256 | gen_introspect(schema, output_dir, prefix, unmask) | ||
257 | + golang.gen_golang(schema, output_dir, prefix) | ||
258 | |||
259 | |||
260 | def main() -> int: | ||
261 | -- | 123 | -- |
262 | 2.48.1 | 124 | 2.39.1 |
125 | |||
126 | diff view generated by jsdifflib |
Deleted patch | |||
---|---|---|---|
1 | This patch handles QAPI alternate types and generates data structures | ||
2 | in Go that handles it. | ||
3 | 1 | ||
4 | Alternate types are similar to Union but without a discriminator that | ||
5 | can be used to identify the underlying value on the wire. | ||
6 | |||
7 | 1. Over the wire, we need to infer underlying value by its type | ||
8 | |||
9 | 2. Pointer to types are mapped as optional. Absent value can be a | ||
10 | valid value. | ||
11 | |||
12 | 3. We use Go's standard 'encoding/json' library with its Marshal | ||
13 | and Unmarshal interfaces. | ||
14 | |||
15 | 4. As an exceptional but valid case, there are types that accept | ||
16 | JSON NULL as value. Due to limitations with Go's standard library | ||
17 | (point 3) combined with Absent being a possibility (point 2), we | ||
18 | translante NULL values to a boolean field called 'IsNull'. See the | ||
19 | second example and docs/devel/qapi-golang-code-gen.rst under | ||
20 | Alternate section. | ||
21 | |||
22 | * First example: | ||
23 | |||
24 | qapi: | ||
25 | | ## | ||
26 | | # @BlockdevRef: | ||
27 | | # | ||
28 | | # Reference to a block device. | ||
29 | | # | ||
30 | | # @definition: defines a new block device inline | ||
31 | | # | ||
32 | | # @reference: references the ID of an existing block device | ||
33 | | # | ||
34 | | # Since: 2.9 | ||
35 | | ## | ||
36 | | { 'alternate': 'BlockdevRef', | ||
37 | | 'data': { 'definition': 'BlockdevOptions', | ||
38 | | 'reference': 'str' } } | ||
39 | |||
40 | go: | ||
41 | | // Reference to a block device. | ||
42 | | // | ||
43 | | // Since: 2.9 | ||
44 | | type BlockdevRef struct { | ||
45 | | // defines a new block device inline | ||
46 | | Definition *BlockdevOptions | ||
47 | | // references the ID of an existing block device | ||
48 | | Reference *string | ||
49 | | } | ||
50 | | | ||
51 | | func (s BlockdevRef) MarshalJSON() ([]byte, error) { | ||
52 | | ... | ||
53 | | } | ||
54 | | | ||
55 | | func (s *BlockdevRef) UnmarshalJSON(data []byte) error { | ||
56 | | ... | ||
57 | | } | ||
58 | |||
59 | usage: | ||
60 | | input := `{"driver":"qcow2","data-file":"/some/place/my-image"}` | ||
61 | | k := BlockdevRef{} | ||
62 | | err := json.Unmarshal([]byte(input), &k) | ||
63 | | if err != nil { | ||
64 | | panic(err) | ||
65 | | } | ||
66 | | // *k.Definition.Qcow2.DataFile.Reference == "/some/place/my-image" | ||
67 | |||
68 | * Second example: | ||
69 | |||
70 | qapi: | ||
71 | | { 'alternate': 'StrOrNull', | ||
72 | | 'data': { 's': 'str', | ||
73 | | 'n': 'null' } } | ||
74 | |||
75 | | // This is a string value or the explicit lack of a string (null | ||
76 | | // pointer in C). Intended for cases when 'optional absent' already | ||
77 | | // has a different meaning. | ||
78 | | // | ||
79 | | // Since: 2.10 | ||
80 | | type StrOrNull struct { | ||
81 | | // the string value | ||
82 | | S *string | ||
83 | | // no string value | ||
84 | | IsNull bool | ||
85 | | } | ||
86 | | | ||
87 | | // Helper function to get its underlying Go value or absent of value | ||
88 | | func (s *StrOrNull) ToAnyOrAbsent() (any, bool) { | ||
89 | | ... | ||
90 | | } | ||
91 | | | ||
92 | | func (s StrOrNull) MarshalJSON() ([]byte, error) { | ||
93 | | ... | ||
94 | | } | ||
95 | | | ||
96 | | func (s *StrOrNull) UnmarshalJSON(data []byte) error { | ||
97 | | ... | ||
98 | | } | ||
99 | |||
100 | Signed-off-by: Victor Toso <victortoso@redhat.com> | ||
101 | --- | ||
102 | scripts/qapi/golang/golang.py | 306 +++++++++++++++++++++++++++++++++- | ||
103 | scripts/qapi/golang/utils.go | 26 +++ | ||
104 | 2 files changed, 329 insertions(+), 3 deletions(-) | ||
105 | create mode 100644 scripts/qapi/golang/utils.go | ||
106 | |||
107 | diff --git a/scripts/qapi/golang/golang.py b/scripts/qapi/golang/golang.py | ||
108 | index XXXXXXX..XXXXXXX 100644 | ||
109 | --- a/scripts/qapi/golang/golang.py | ||
110 | +++ b/scripts/qapi/golang/golang.py | ||
111 | @@ -XXX,XX +XXX,XX @@ | ||
112 | from __future__ import annotations | ||
113 | |||
114 | import os, shutil, textwrap | ||
115 | -from typing import List, Optional | ||
116 | +from typing import List, Optional, Tuple | ||
117 | |||
118 | from ..schema import ( | ||
119 | QAPISchema, | ||
120 | + QAPISchemaAlternateType, | ||
121 | QAPISchemaBranches, | ||
122 | QAPISchemaEnumMember, | ||
123 | QAPISchemaFeature, | ||
124 | @@ -XXX,XX +XXX,XX @@ | ||
125 | ) | ||
126 | from ..source import QAPISourceInfo | ||
127 | |||
128 | +FOUR_SPACES = " " | ||
129 | + | ||
130 | TEMPLATE_GENERATED_HEADER = """ | ||
131 | /* | ||
132 | * Copyright 2025 Red Hat, Inc. | ||
133 | @@ -XXX,XX +XXX,XX @@ | ||
134 | ) | ||
135 | """ | ||
136 | |||
137 | +TEMPLATE_ALTERNATE_CHECK_INVALID_JSON_NULL = """ | ||
138 | + // Check for json-null first | ||
139 | + if string(data) == "null" {{ | ||
140 | + return errors.New(`null not supported for {name}`) | ||
141 | + }}""" | ||
142 | + | ||
143 | +TEMPLATE_ALTERNATE_NULLABLE_CHECK = """ | ||
144 | + }} else if s.{var_name} != nil {{ | ||
145 | + return *s.{var_name}, false""" | ||
146 | + | ||
147 | +TEMPLATE_ALTERNATE_MARSHAL_CHECK = """ | ||
148 | + if s.{var_name} != nil {{ | ||
149 | + return json.Marshal(s.{var_name}) | ||
150 | + }} else """ | ||
151 | + | ||
152 | +TEMPLATE_ALTERNATE_UNMARSHAL_CHECK = """ | ||
153 | + // Check for {var_type} | ||
154 | + {{ | ||
155 | + s.{var_name} = new({var_type}) | ||
156 | + if err := strictDecode(s.{var_name}, data); err == nil {{ | ||
157 | + return nil | ||
158 | + }} | ||
159 | + s.{var_name} = nil | ||
160 | + }} | ||
161 | + | ||
162 | +""" | ||
163 | + | ||
164 | +TEMPLATE_ALTERNATE_NULLABLE_MARSHAL_CHECK = """ | ||
165 | + if s.IsNull { | ||
166 | + return []byte("null"), nil | ||
167 | + } else """ | ||
168 | + | ||
169 | +TEMPLATE_ALTERNATE_NULLABLE_UNMARSHAL_CHECK = """ | ||
170 | + // Check for json-null first | ||
171 | + if string(data) == "null" { | ||
172 | + s.IsNull = true | ||
173 | + return nil | ||
174 | + }""" | ||
175 | + | ||
176 | +TEMPLATE_ALTERNATE_METHODS = """ | ||
177 | +func (s {name}) MarshalJSON() ([]byte, error) {{ | ||
178 | +{marshal_check_fields} | ||
179 | + return {marshal_return_default} | ||
180 | +}} | ||
181 | + | ||
182 | +func (s *{name}) UnmarshalJSON(data []byte) error {{ | ||
183 | +{unmarshal_check_fields} | ||
184 | + return fmt.Errorf("Can't convert to {name}: %s", string(data)) | ||
185 | +}} | ||
186 | +""" | ||
187 | + | ||
188 | |||
189 | # Takes the documentation object of a specific type and returns | ||
190 | # that type's documentation and its member's docs. | ||
191 | @@ -XXX,XX +XXX,XX @@ def gen_golang(schema: QAPISchema, output_dir: str, prefix: str) -> None: | ||
192 | vis.write(output_dir) | ||
193 | |||
194 | |||
195 | +def qapi_to_field_name(name: str) -> str: | ||
196 | + return name.title().replace("_", "").replace("-", "") | ||
197 | + | ||
198 | + | ||
199 | def qapi_to_field_name_enum(name: str) -> str: | ||
200 | return name.title().replace("-", "") | ||
201 | |||
202 | |||
203 | +def qapi_schema_type_to_go_type(qapitype: str) -> str: | ||
204 | + schema_types_to_go = { | ||
205 | + "str": "string", | ||
206 | + "null": "nil", | ||
207 | + "bool": "bool", | ||
208 | + "number": "float64", | ||
209 | + "size": "uint64", | ||
210 | + "int": "int64", | ||
211 | + "int8": "int8", | ||
212 | + "int16": "int16", | ||
213 | + "int32": "int32", | ||
214 | + "int64": "int64", | ||
215 | + "uint8": "uint8", | ||
216 | + "uint16": "uint16", | ||
217 | + "uint32": "uint32", | ||
218 | + "uint64": "uint64", | ||
219 | + "any": "any", | ||
220 | + "QType": "QType", | ||
221 | + } | ||
222 | + | ||
223 | + prefix = "" | ||
224 | + if qapitype.endswith("List"): | ||
225 | + prefix = "[]" | ||
226 | + qapitype = qapitype[:-4] | ||
227 | + | ||
228 | + qapitype = schema_types_to_go.get(qapitype, qapitype) | ||
229 | + return prefix + qapitype | ||
230 | + | ||
231 | + | ||
232 | +# Helper for Alternate generation | ||
233 | +def qapi_field_to_alternate_go_field( | ||
234 | + member_name: str, type_name: str | ||
235 | +) -> Tuple[str, str, str]: | ||
236 | + # Nothing to generate on null types. We update some | ||
237 | + # variables to handle json-null on marshalling methods. | ||
238 | + if type_name == "null": | ||
239 | + return "IsNull", "bool", "" | ||
240 | + | ||
241 | + # On Alternates, fields are optional represented in Go as pointer | ||
242 | + return ( | ||
243 | + qapi_to_field_name(member_name), | ||
244 | + qapi_schema_type_to_go_type(type_name), | ||
245 | + "*", | ||
246 | + ) | ||
247 | + | ||
248 | + | ||
249 | +def fetch_indent_blocks_over_args( | ||
250 | + args: List[dict[str:str]], | ||
251 | +) -> Tuple[int, int]: | ||
252 | + maxname, maxtype = 0, 0 | ||
253 | + blocks: tuple(int, int) = [] | ||
254 | + for arg in args: | ||
255 | + if "comment" in arg or "doc" in arg: | ||
256 | + blocks.append((maxname, maxtype)) | ||
257 | + maxname, maxtype = 0, 0 | ||
258 | + | ||
259 | + if "comment" in arg: | ||
260 | + # They are single blocks | ||
261 | + continue | ||
262 | + | ||
263 | + if "type" not in arg: | ||
264 | + # Embed type are on top of the struct and the following | ||
265 | + # fields do not consider it for formatting | ||
266 | + blocks.append((maxname, maxtype)) | ||
267 | + maxname, maxtype = 0, 0 | ||
268 | + continue | ||
269 | + | ||
270 | + maxname = max(maxname, len(arg.get("name", ""))) | ||
271 | + maxtype = max(maxtype, len(arg.get("type", ""))) | ||
272 | + | ||
273 | + blocks.append((maxname, maxtype)) | ||
274 | + return blocks | ||
275 | + | ||
276 | + | ||
277 | def fetch_indent_blocks_over_enum_with_docs( | ||
278 | name: str, members: List[QAPISchemaEnumMember], docfields: Dict[str, str] | ||
279 | ) -> Tuple[int]: | ||
280 | @@ -XXX,XX +XXX,XX @@ def fetch_indent_blocks_over_enum_with_docs( | ||
281 | return blocks | ||
282 | |||
283 | |||
284 | +# Helper function for boxed or self contained structures. | ||
285 | +def generate_struct_type( | ||
286 | + type_name, | ||
287 | + type_doc: str = "", | ||
288 | + args: List[dict[str:str]] = None, | ||
289 | + indent: int = 0, | ||
290 | +) -> str: | ||
291 | + base_indent = FOUR_SPACES * indent | ||
292 | + | ||
293 | + with_type = "" | ||
294 | + if type_name != "": | ||
295 | + with_type = f"\n{base_indent}type {type_name}" | ||
296 | + | ||
297 | + if type_doc != "": | ||
298 | + # Append line jump only if type_doc exists | ||
299 | + type_doc = f"\n{type_doc}" | ||
300 | + | ||
301 | + if args is None: | ||
302 | + # No args, early return | ||
303 | + return f"""{type_doc}{with_type} struct{{}}""" | ||
304 | + | ||
305 | + # The logic below is to generate fields of the struct. | ||
306 | + # We have to be mindful of the different indentation possibilities between | ||
307 | + # $var_name $var_type $var_tag that are vertically indented with gofmt. | ||
308 | + # | ||
309 | + # So, we first have to iterate over all args and find all indent blocks | ||
310 | + # by calculating the spaces between (1) member and type and between (2) | ||
311 | + # the type and tag. (1) and (2) is the tuple present in List returned | ||
312 | + # by the helper function fetch_indent_blocks_over_args. | ||
313 | + inner_indent = base_indent + FOUR_SPACES | ||
314 | + doc_indent = inner_indent + "// " | ||
315 | + fmt = textwrap.TextWrapper( | ||
316 | + width=70, initial_indent=doc_indent, subsequent_indent=doc_indent | ||
317 | + ) | ||
318 | + | ||
319 | + indent_block = iter(fetch_indent_blocks_over_args(args)) | ||
320 | + maxname, maxtype = next(indent_block) | ||
321 | + members = " {\n" | ||
322 | + for index, arg in enumerate(args): | ||
323 | + if "comment" in arg: | ||
324 | + maxname, maxtype = next(indent_block) | ||
325 | + members += f""" // {arg["comment"]}\n""" | ||
326 | + # comments are single blocks, so we can skip to next arg | ||
327 | + continue | ||
328 | + | ||
329 | + name2type = "" | ||
330 | + if "doc" in arg: | ||
331 | + maxname, maxtype = next(indent_block) | ||
332 | + members += fmt.fill(arg["doc"]) | ||
333 | + members += "\n" | ||
334 | + | ||
335 | + name = arg["name"] | ||
336 | + if "type" in arg: | ||
337 | + namelen = len(name) | ||
338 | + name2type = " " * max(1, (maxname - namelen + 1)) | ||
339 | + | ||
340 | + type2tag = "" | ||
341 | + if "tag" in arg: | ||
342 | + typelen = len(arg["type"]) | ||
343 | + type2tag = " " * max(1, (maxtype - typelen + 1)) | ||
344 | + | ||
345 | + gotype = arg.get("type", "") | ||
346 | + tag = arg.get("tag", "") | ||
347 | + members += ( | ||
348 | + f"""{inner_indent}{name}{name2type}{gotype}{type2tag}{tag}\n""" | ||
349 | + ) | ||
350 | + | ||
351 | + members += f"{base_indent}}}\n" | ||
352 | + return f"""{type_doc}{with_type} struct{members}""" | ||
353 | + | ||
354 | + | ||
355 | +def generate_template_alternate( | ||
356 | + self: QAPISchemaGenGolangVisitor, | ||
357 | + name: str, | ||
358 | + variants: Optional[QAPISchemaVariants], | ||
359 | +) -> str: | ||
360 | + args: List[dict[str:str]] = [] | ||
361 | + nullable = name in self.accept_null_types | ||
362 | + if nullable: | ||
363 | + # Examples in QEMU QAPI schema: StrOrNull and BlockdevRefOrNull | ||
364 | + marshal_return_default = """[]byte("{}"), nil""" | ||
365 | + marshal_check_fields = TEMPLATE_ALTERNATE_NULLABLE_MARSHAL_CHECK[1:] | ||
366 | + unmarshal_check_fields = TEMPLATE_ALTERNATE_NULLABLE_UNMARSHAL_CHECK | ||
367 | + else: | ||
368 | + marshal_return_default = f'nil, errors.New("{name} has empty fields")' | ||
369 | + marshal_check_fields = "" | ||
370 | + unmarshal_check_fields = ( | ||
371 | + TEMPLATE_ALTERNATE_CHECK_INVALID_JSON_NULL.format(name=name) | ||
372 | + ) | ||
373 | + | ||
374 | + doc = self.docmap.get(name, None) | ||
375 | + content, docfields = qapi_to_golang_struct_docs(doc) | ||
376 | + if variants: | ||
377 | + for var in variants.variants: | ||
378 | + var_name, var_type, isptr = qapi_field_to_alternate_go_field( | ||
379 | + var.name, var.type.name | ||
380 | + ) | ||
381 | + args.append( | ||
382 | + { | ||
383 | + "name": f"{var_name}", | ||
384 | + "type": f"{isptr}{var_type}", | ||
385 | + "doc": docfields.get(var.name, ""), | ||
386 | + } | ||
387 | + ) | ||
388 | + # Null is special, handled first | ||
389 | + if var.type.name == "null": | ||
390 | + assert nullable | ||
391 | + continue | ||
392 | + | ||
393 | + skip_indent = 1 + len(FOUR_SPACES) | ||
394 | + if marshal_check_fields == "": | ||
395 | + skip_indent = 1 | ||
396 | + marshal_check_fields += TEMPLATE_ALTERNATE_MARSHAL_CHECK[ | ||
397 | + skip_indent: | ||
398 | + ].format(var_name=var_name) | ||
399 | + unmarshal_check_fields += TEMPLATE_ALTERNATE_UNMARSHAL_CHECK[ | ||
400 | + :-1 | ||
401 | + ].format(var_name=var_name, var_type=var_type) | ||
402 | + | ||
403 | + content += string_to_code(generate_struct_type(name, args=args)) | ||
404 | + content += string_to_code( | ||
405 | + TEMPLATE_ALTERNATE_METHODS.format( | ||
406 | + name=name, | ||
407 | + marshal_check_fields=marshal_check_fields[:-6], | ||
408 | + marshal_return_default=marshal_return_default, | ||
409 | + unmarshal_check_fields=unmarshal_check_fields[1:], | ||
410 | + ) | ||
411 | + ) | ||
412 | + return "\n" + content | ||
413 | + | ||
414 | + | ||
415 | def generate_content_from_dict(data: dict[str, str]) -> str: | ||
416 | content = "" | ||
417 | |||
418 | @@ -XXX,XX +XXX,XX @@ def generate_content_from_dict(data: dict[str, str]) -> str: | ||
419 | return content.replace("\n\n\n", "\n\n") | ||
420 | |||
421 | |||
422 | +def string_to_code(text: str) -> str: | ||
423 | + DOUBLE_BACKTICK = "``" | ||
424 | + result = "" | ||
425 | + for line in text.splitlines(): | ||
426 | + # replace left four spaces with tabs | ||
427 | + limit = len(line) - len(line.lstrip()) | ||
428 | + result += line[:limit].replace(FOUR_SPACES, "\t") | ||
429 | + | ||
430 | + # work with the rest of the line | ||
431 | + if line[limit : limit + 2] == "//": | ||
432 | + # gofmt tool does not like comments with backticks. | ||
433 | + result += line[limit:].replace(DOUBLE_BACKTICK, '"') | ||
434 | + else: | ||
435 | + result += line[limit:] | ||
436 | + result += "\n" | ||
437 | + | ||
438 | + return result | ||
439 | + | ||
440 | + | ||
441 | def generate_template_imports(words: List[str]) -> str: | ||
442 | if len(words) == 0: | ||
443 | return "" | ||
444 | @@ -XXX,XX +XXX,XX @@ class QAPISchemaGenGolangVisitor(QAPISchemaVisitor): | ||
445 | # pylint: disable=too-many-arguments | ||
446 | def __init__(self, _: str): | ||
447 | super().__init__() | ||
448 | - gofiles = ("protocol.go",) | ||
449 | + gofiles = ("protocol.go", "utils.go") | ||
450 | # Map each qapi type to the necessary Go imports | ||
451 | types = { | ||
452 | + "alternate": ["encoding/json", "errors", "fmt"], | ||
453 | "enum": [], | ||
454 | } | ||
455 | |||
456 | @@ -XXX,XX +XXX,XX @@ def __init__(self, _: str): | ||
457 | self.golang_package_name = "qapi" | ||
458 | self.duplicate = list(gofiles) | ||
459 | self.enums: dict[str, str] = {} | ||
460 | + self.alternates: dict[str, str] = {} | ||
461 | + self.accept_null_types = [] | ||
462 | self.docmap = {} | ||
463 | |||
464 | self.types = dict.fromkeys(types, "") | ||
465 | @@ -XXX,XX +XXX,XX @@ def __init__(self, _: str): | ||
466 | def visit_begin(self, schema: QAPISchema) -> None: | ||
467 | self.schema = schema | ||
468 | |||
469 | + # We need to be aware of any types that accept JSON NULL | ||
470 | + for name, entity in self.schema._entity_dict.items(): | ||
471 | + if not isinstance(entity, QAPISchemaAlternateType): | ||
472 | + # Assume that only Alternate types accept JSON NULL | ||
473 | + continue | ||
474 | + | ||
475 | + for var in entity.alternatives.variants: | ||
476 | + if var.type.name == "null": | ||
477 | + self.accept_null_types.append(name) | ||
478 | + break | ||
479 | + | ||
480 | # iterate once in schema.docs to map doc objects to its name | ||
481 | for doc in schema.docs: | ||
482 | if doc.symbol is None: | ||
483 | @@ -XXX,XX +XXX,XX @@ def visit_begin(self, schema: QAPISchema) -> None: | ||
484 | def visit_end(self) -> None: | ||
485 | del self.schema | ||
486 | self.types["enum"] += generate_content_from_dict(self.enums) | ||
487 | + self.types["alternate"] += generate_content_from_dict(self.alternates) | ||
488 | |||
489 | def visit_object_type( | ||
490 | self, | ||
491 | @@ -XXX,XX +XXX,XX @@ def visit_alternate_type( | ||
492 | features: List[QAPISchemaFeature], | ||
493 | variants: QAPISchemaVariants, | ||
494 | ) -> None: | ||
495 | - pass | ||
496 | + assert name not in self.alternates | ||
497 | + self.alternates[name] = generate_template_alternate( | ||
498 | + self, name, variants | ||
499 | + ) | ||
500 | |||
501 | def visit_enum_type( | ||
502 | self, | ||
503 | diff --git a/scripts/qapi/golang/utils.go b/scripts/qapi/golang/utils.go | ||
504 | new file mode 100644 | ||
505 | index XXXXXXX..XXXXXXX | ||
506 | --- /dev/null | ||
507 | +++ b/scripts/qapi/golang/utils.go | ||
508 | @@ -XXX,XX +XXX,XX @@ | ||
509 | +/* | ||
510 | + * Copyright 2025 Red Hat, Inc. | ||
511 | + * SPDX-License-Identifier: MIT-0 | ||
512 | + * | ||
513 | + * Authors: | ||
514 | + * Victor Toso <victortoso@redhat.com> | ||
515 | + */ | ||
516 | +package qapi | ||
517 | + | ||
518 | +import ( | ||
519 | + "encoding/json" | ||
520 | + "strings" | ||
521 | +) | ||
522 | + | ||
523 | +// Creates a decoder that errors on unknown Fields | ||
524 | +// Returns nil if successfully decoded @from payload to @into type | ||
525 | +// Returns error if failed to decode @from payload to @into type | ||
526 | +func strictDecode(into interface{}, from []byte) error { | ||
527 | + dec := json.NewDecoder(strings.NewReader(string(from))) | ||
528 | + dec.DisallowUnknownFields() | ||
529 | + | ||
530 | + if err := dec.Decode(into); err != nil { | ||
531 | + return err | ||
532 | + } | ||
533 | + return nil | ||
534 | +} | ||
535 | -- | ||
536 | 2.48.1 | diff view generated by jsdifflib |
Deleted patch | |||
---|---|---|---|
1 | This patch handles QAPI struct types and generates the equivalent | ||
2 | types in Go. The following patch adds extra logic when a member of the | ||
3 | struct has a Type that can take JSON Null value (e.g: StrOrNull in | ||
4 | QEMU) | ||
5 | 1 | ||
6 | The highlights of this implementation are: | ||
7 | |||
8 | 1. Generating a Go struct that requires a @base type, the @base type | ||
9 | fields are copied over to the Go struct. The advantage of this | ||
10 | approach is to not have embed structs in any of the QAPI types. | ||
11 | Note that embedding a @base type is recursive, that is, if the | ||
12 | @base type has a @base, all of those fields will be copied over. | ||
13 | |||
14 | 2. About the Go struct's fields: | ||
15 | |||
16 | i) They can be either by Value or Reference. | ||
17 | |||
18 | ii) Every field that is marked as optional in the QAPI specification | ||
19 | are translated to Reference fields in its Go structure. This | ||
20 | design decision is the most straightforward way to check if a | ||
21 | given field was set or not. Exception only for types that can | ||
22 | take JSON Null value. | ||
23 | |||
24 | iii) Mandatory fields are always by Value with the exception of QAPI | ||
25 | arrays, which are handled by Reference (to a block of memory) by | ||
26 | Go. | ||
27 | |||
28 | iv) All the fields are named with Uppercase due Golang's export | ||
29 | convention. | ||
30 | |||
31 | Example: | ||
32 | |||
33 | qapi: | ||
34 | | ## | ||
35 | | # @BlockdevCreateOptionsFile: | ||
36 | | # | ||
37 | | # Driver specific image creation options for file. | ||
38 | | # | ||
39 | | # @filename: Filename for the new image file | ||
40 | | # | ||
41 | | # @size: Size of the virtual disk in bytes | ||
42 | | # | ||
43 | | # @preallocation: Preallocation mode for the new image (default: off; | ||
44 | | # allowed values: off, falloc (if CONFIG_POSIX_FALLOCATE), full | ||
45 | | # (if CONFIG_POSIX)) | ||
46 | | # | ||
47 | | # @nocow: Turn off copy-on-write (valid only on btrfs; default: off) | ||
48 | | # | ||
49 | | # @extent-size-hint: Extent size hint to add to the image file; 0 for | ||
50 | | # not adding an extent size hint (default: 1 MB, since 5.1) | ||
51 | | # | ||
52 | | # Since: 2.12 | ||
53 | | ## | ||
54 | | { 'struct': 'BlockdevCreateOptionsFile', | ||
55 | | 'data': { 'filename': 'str', | ||
56 | | 'size': 'size', | ||
57 | | '*preallocation': 'PreallocMode', | ||
58 | | '*nocow': 'bool', | ||
59 | | '*extent-size-hint': 'size'} } | ||
60 | |||
61 | go: | ||
62 | | // Driver specific image creation options for file. | ||
63 | | // | ||
64 | | // Since: 2.12 | ||
65 | | type BlockdevCreateOptionsFile struct { | ||
66 | | // Filename for the new image file | ||
67 | | Filename string `json:"filename"` | ||
68 | | // Size of the virtual disk in bytes | ||
69 | | Size uint64 `json:"size"` | ||
70 | | // Preallocation mode for the new image (default: off; allowed | ||
71 | | // values: off, falloc (if CONFIG_POSIX_FALLOCATE), full (if | ||
72 | | // CONFIG_POSIX)) | ||
73 | | Preallocation *PreallocMode `json:"preallocation,omitempty"` | ||
74 | | // Turn off copy-on-write (valid only on btrfs; default: off) | ||
75 | | Nocow *bool `json:"nocow,omitempty"` | ||
76 | | // Extent size hint to add to the image file; 0 for not adding an | ||
77 | | // extent size hint (default: 1 MB, since 5.1) | ||
78 | | ExtentSizeHint *uint64 `json:"extent-size-hint,omitempty"` | ||
79 | | } | ||
80 | |||
81 | Signed-off-by: Victor Toso <victortoso@redhat.com> | ||
82 | --- | ||
83 | scripts/qapi/golang/golang.py | 193 +++++++++++++++++++++++++++++++++- | ||
84 | 1 file changed, 192 insertions(+), 1 deletion(-) | ||
85 | |||
86 | diff --git a/scripts/qapi/golang/golang.py b/scripts/qapi/golang/golang.py | ||
87 | index XXXXXXX..XXXXXXX 100644 | ||
88 | --- a/scripts/qapi/golang/golang.py | ||
89 | +++ b/scripts/qapi/golang/golang.py | ||
90 | @@ -XXX,XX +XXX,XX @@ def gen_golang(schema: QAPISchema, output_dir: str, prefix: str) -> None: | ||
91 | vis.write(output_dir) | ||
92 | |||
93 | |||
94 | +def qapi_name_is_base(name: str) -> bool: | ||
95 | + return qapi_name_is_object(name) and name.endswith("-base") | ||
96 | + | ||
97 | + | ||
98 | +def qapi_name_is_object(name: str) -> bool: | ||
99 | + return name.startswith("q_obj_") | ||
100 | + | ||
101 | + | ||
102 | def qapi_to_field_name(name: str) -> str: | ||
103 | return name.title().replace("_", "").replace("-", "") | ||
104 | |||
105 | @@ -XXX,XX +XXX,XX @@ def qapi_to_field_name_enum(name: str) -> str: | ||
106 | return name.title().replace("-", "") | ||
107 | |||
108 | |||
109 | +def qapi_to_go_type_name(name: str) -> str: | ||
110 | + # We want to keep CamelCase for Golang types. We want to avoid removing | ||
111 | + # already set CameCase names while fixing uppercase ones, eg: | ||
112 | + # 1) q_obj_SocketAddress_base -> SocketAddressBase | ||
113 | + # 2) q_obj_WATCHDOG-arg -> WatchdogArg | ||
114 | + | ||
115 | + if qapi_name_is_object(name): | ||
116 | + # Remove q_obj_ prefix | ||
117 | + name = name[6:] | ||
118 | + | ||
119 | + # Handle CamelCase | ||
120 | + words = list(name.replace("_", "-").split("-")) | ||
121 | + name = words[0] | ||
122 | + if name.islower() or name.isupper(): | ||
123 | + name = name.title() | ||
124 | + | ||
125 | + name += "".join(word.title() for word in words[1:]) | ||
126 | + | ||
127 | + return name | ||
128 | + | ||
129 | + | ||
130 | def qapi_schema_type_to_go_type(qapitype: str) -> str: | ||
131 | schema_types_to_go = { | ||
132 | "str": "string", | ||
133 | @@ -XXX,XX +XXX,XX @@ def generate_struct_type( | ||
134 | return f"""{type_doc}{with_type} struct{members}""" | ||
135 | |||
136 | |||
137 | +def get_struct_field( | ||
138 | + self: QAPISchemaGenGolangVisitor, | ||
139 | + qapi_name: str, | ||
140 | + qapi_type_name: str, | ||
141 | + field_doc: str, | ||
142 | + is_optional: bool, | ||
143 | + is_variant: bool, | ||
144 | +) -> dict[str:str]: | ||
145 | + field = qapi_to_field_name(qapi_name) | ||
146 | + member_type = qapi_schema_type_to_go_type(qapi_type_name) | ||
147 | + | ||
148 | + optional = "" | ||
149 | + if is_optional: | ||
150 | + if member_type not in self.accept_null_types: | ||
151 | + optional = ",omitempty" | ||
152 | + | ||
153 | + # Use pointer to type when field is optional | ||
154 | + isptr = "*" if is_optional and member_type[0] not in "*[" else "" | ||
155 | + | ||
156 | + fieldtag = ( | ||
157 | + '`json:"-"`' if is_variant else f'`json:"{qapi_name}{optional}"`' | ||
158 | + ) | ||
159 | + arg = { | ||
160 | + "name": f"{field}", | ||
161 | + "type": f"{isptr}{member_type}", | ||
162 | + "tag": f"{fieldtag}", | ||
163 | + } | ||
164 | + if field_doc != "": | ||
165 | + arg["doc"] = field_doc | ||
166 | + | ||
167 | + return arg | ||
168 | + | ||
169 | + | ||
170 | +def recursive_base( | ||
171 | + self: QAPISchemaGenGolangVisitor, | ||
172 | + base: Optional[QAPISchemaObjectType], | ||
173 | +) -> List[dict[str:str]]: | ||
174 | + fields: List[dict[str:str]] = [] | ||
175 | + | ||
176 | + if not base: | ||
177 | + return fields | ||
178 | + | ||
179 | + if base.base is not None: | ||
180 | + embed_base = self.schema.lookup_entity(base.base.name) | ||
181 | + fields = recursive_base(self, embed_base) | ||
182 | + | ||
183 | + doc = self.docmap.get(base.name, None) | ||
184 | + _, docfields = qapi_to_golang_struct_docs(doc) | ||
185 | + | ||
186 | + for member in base.local_members: | ||
187 | + field_doc = docfields.get(member.name, "") | ||
188 | + field = get_struct_field( | ||
189 | + self, | ||
190 | + member.name, | ||
191 | + member.type.name, | ||
192 | + field_doc, | ||
193 | + member.optional, | ||
194 | + False, | ||
195 | + ) | ||
196 | + fields.append(field) | ||
197 | + | ||
198 | + return fields | ||
199 | + | ||
200 | + | ||
201 | +# Helper function that is used for most of QAPI types | ||
202 | +def qapi_to_golang_struct( | ||
203 | + self: QAPISchemaGenGolangVisitor, | ||
204 | + name: str, | ||
205 | + info: Optional[QAPISourceInfo], | ||
206 | + __: QAPISchemaIfCond, | ||
207 | + ___: List[QAPISchemaFeature], | ||
208 | + base: Optional[QAPISchemaObjectType], | ||
209 | + members: List[QAPISchemaObjectTypeMember], | ||
210 | + variants: Optional[QAPISchemaVariants], | ||
211 | + indent: int = 0, | ||
212 | + doc_enabled: bool = True, | ||
213 | +) -> str: | ||
214 | + fields = recursive_base(self, base) | ||
215 | + | ||
216 | + doc = self.docmap.get(name, None) | ||
217 | + type_doc, docfields = qapi_to_golang_struct_docs(doc) | ||
218 | + if not doc_enabled: | ||
219 | + type_doc = "" | ||
220 | + | ||
221 | + if members: | ||
222 | + for member in members: | ||
223 | + field_doc = docfields.get(member.name, "") if doc_enabled else "" | ||
224 | + field = get_struct_field( | ||
225 | + self, | ||
226 | + member.name, | ||
227 | + member.type.name, | ||
228 | + field_doc, | ||
229 | + member.optional, | ||
230 | + False, | ||
231 | + ) | ||
232 | + fields.append(field) | ||
233 | + | ||
234 | + exists = {} | ||
235 | + if variants: | ||
236 | + fields.append({"comment": "Variants fields"}) | ||
237 | + for variant in variants.variants: | ||
238 | + if variant.type.is_implicit(): | ||
239 | + continue | ||
240 | + | ||
241 | + exists[variant.name] = True | ||
242 | + field_doc = docfields.get(variant.name, "") if doc_enabled else "" | ||
243 | + field = get_struct_field( | ||
244 | + self, | ||
245 | + variant.name, | ||
246 | + variant.type.name, | ||
247 | + field_doc, | ||
248 | + True, | ||
249 | + True, | ||
250 | + ) | ||
251 | + fields.append(field) | ||
252 | + | ||
253 | + type_name = qapi_to_go_type_name(name) | ||
254 | + content = string_to_code( | ||
255 | + generate_struct_type( | ||
256 | + type_name, type_doc=type_doc, args=fields, indent=indent | ||
257 | + ) | ||
258 | + ) | ||
259 | + return content | ||
260 | + | ||
261 | + | ||
262 | def generate_template_alternate( | ||
263 | self: QAPISchemaGenGolangVisitor, | ||
264 | name: str, | ||
265 | @@ -XXX,XX +XXX,XX @@ def __init__(self, _: str): | ||
266 | types = { | ||
267 | "alternate": ["encoding/json", "errors", "fmt"], | ||
268 | "enum": [], | ||
269 | + "struct": ["encoding/json"], | ||
270 | } | ||
271 | |||
272 | self.schema: QAPISchema | ||
273 | @@ -XXX,XX +XXX,XX @@ def __init__(self, _: str): | ||
274 | self.duplicate = list(gofiles) | ||
275 | self.enums: dict[str, str] = {} | ||
276 | self.alternates: dict[str, str] = {} | ||
277 | + self.structs: dict[str, str] = {} | ||
278 | self.accept_null_types = [] | ||
279 | self.docmap = {} | ||
280 | |||
281 | @@ -XXX,XX +XXX,XX @@ def visit_end(self) -> None: | ||
282 | del self.schema | ||
283 | self.types["enum"] += generate_content_from_dict(self.enums) | ||
284 | self.types["alternate"] += generate_content_from_dict(self.alternates) | ||
285 | + self.types["struct"] += generate_content_from_dict(self.structs) | ||
286 | |||
287 | def visit_object_type( | ||
288 | self, | ||
289 | @@ -XXX,XX +XXX,XX @@ def visit_object_type( | ||
290 | members: List[QAPISchemaObjectTypeMember], | ||
291 | branches: Optional[QAPISchemaBranches], | ||
292 | ) -> None: | ||
293 | - pass | ||
294 | + # Do not handle anything besides struct. | ||
295 | + if ( | ||
296 | + name == self.schema.the_empty_object_type.name | ||
297 | + or not isinstance(name, str) | ||
298 | + or info.defn_meta not in ["struct"] | ||
299 | + ): | ||
300 | + return | ||
301 | + | ||
302 | + # Base structs are embed | ||
303 | + if qapi_name_is_base(name): | ||
304 | + return | ||
305 | + | ||
306 | + # visit all inner objects as well, they are not going to be | ||
307 | + # called by python's generator. | ||
308 | + if branches: | ||
309 | + for branch in branches.variants: | ||
310 | + assert isinstance(branch.type, QAPISchemaObjectType) | ||
311 | + self.visit_object_type( | ||
312 | + self, | ||
313 | + branch.type.name, | ||
314 | + branch.type.info, | ||
315 | + branch.type.ifcond, | ||
316 | + branch.type.base, | ||
317 | + branch.type.local_members, | ||
318 | + branch.type.branches, | ||
319 | + ) | ||
320 | + | ||
321 | + # Save generated Go code to be written later | ||
322 | + if info.defn_meta == "struct": | ||
323 | + assert name not in self.structs | ||
324 | + self.structs[name] = string_to_code( | ||
325 | + qapi_to_golang_struct( | ||
326 | + self, name, info, ifcond, features, base, members, branches | ||
327 | + ) | ||
328 | + ) | ||
329 | |||
330 | def visit_alternate_type( | ||
331 | self, | ||
332 | -- | ||
333 | 2.48.1 | diff view generated by jsdifflib |
Deleted patch | |||
---|---|---|---|
1 | Explaining why this is needed needs some context, so taking the | ||
2 | example of StrOrNull alternate type and considering a simplified | ||
3 | struct that has two fields: | ||
4 | 1 | ||
5 | qapi: | ||
6 | | { 'struct': 'MigrationExample', | ||
7 | | 'data': { '*label': 'StrOrNull', | ||
8 | | 'target': 'StrOrNull' } } | ||
9 | |||
10 | We have an optional member 'label' which can have three JSON values: | ||
11 | 1. A string: { "target": "a.host.com", "label": "happy" } | ||
12 | 2. A null : { "target": "a.host.com", "label": null } | ||
13 | 3. Absent : { "target": null} | ||
14 | |||
15 | The member 'target' is not optional, hence it can't be absent. | ||
16 | |||
17 | A Go struct that contains an optional type that can be JSON Null like | ||
18 | 'label' in the example above, will need extra care when Marshaling and | ||
19 | Unmarshaling from JSON. | ||
20 | |||
21 | This patch handles this very specific case: | ||
22 | - It implements the Marshaler interface for these structs to properly | ||
23 | handle these values. | ||
24 | - It adds the interface AbsentAlternate() and implement it for any | ||
25 | Alternate that can be JSON Null. See its uses in map_and_set() | ||
26 | |||
27 | Signed-off-by: Victor Toso <victortoso@redhat.com> | ||
28 | --- | ||
29 | scripts/qapi/golang/golang.py | 290 ++++++++++++++++++++++++++++++++-- | ||
30 | 1 file changed, 279 insertions(+), 11 deletions(-) | ||
31 | |||
32 | diff --git a/scripts/qapi/golang/golang.py b/scripts/qapi/golang/golang.py | ||
33 | index XXXXXXX..XXXXXXX 100644 | ||
34 | --- a/scripts/qapi/golang/golang.py | ||
35 | +++ b/scripts/qapi/golang/golang.py | ||
36 | @@ -XXX,XX +XXX,XX @@ | ||
37 | ) | ||
38 | """ | ||
39 | |||
40 | +TEMPLATE_ALTERNATE = """ | ||
41 | +// Only implemented on Alternate types that can take JSON NULL as value. | ||
42 | +// | ||
43 | +// This is a helper for the marshalling code. It should return true only when | ||
44 | +// the Alternate is empty (no members are set), otherwise it returns false and | ||
45 | +// the member set to be Marshalled. | ||
46 | +type AbsentAlternate interface { | ||
47 | + ToAnyOrAbsent() (any, bool) | ||
48 | +} | ||
49 | +""" | ||
50 | + | ||
51 | TEMPLATE_ALTERNATE_CHECK_INVALID_JSON_NULL = """ | ||
52 | // Check for json-null first | ||
53 | if string(data) == "null" {{ | ||
54 | @@ -XXX,XX +XXX,XX @@ | ||
55 | return nil | ||
56 | }""" | ||
57 | |||
58 | +TEMPLATE_ALTERNATE_NULLABLE = """ | ||
59 | +func (s *{name}) ToAnyOrAbsent() (any, bool) {{ | ||
60 | + if s != nil {{ | ||
61 | + if s.IsNull {{ | ||
62 | + return nil, false | ||
63 | +{absent_check_fields} | ||
64 | + }} | ||
65 | + }} | ||
66 | + | ||
67 | + return nil, true | ||
68 | +}} | ||
69 | +""" | ||
70 | + | ||
71 | TEMPLATE_ALTERNATE_METHODS = """ | ||
72 | func (s {name}) MarshalJSON() ([]byte, error) {{ | ||
73 | {marshal_check_fields} | ||
74 | @@ -XXX,XX +XXX,XX @@ | ||
75 | """ | ||
76 | |||
77 | |||
78 | +TEMPLATE_STRUCT_WITH_NULLABLE_MARSHAL = """ | ||
79 | +func (s {type_name}) MarshalJSON() ([]byte, error) {{ | ||
80 | + m := make(map[string]any) | ||
81 | +{map_members}{map_special} | ||
82 | + return json.Marshal(&m) | ||
83 | +}} | ||
84 | + | ||
85 | +func (s *{type_name}) UnmarshalJSON(data []byte) error {{ | ||
86 | + tmp := {struct}{{}} | ||
87 | + | ||
88 | + if err := json.Unmarshal(data, &tmp); err != nil {{ | ||
89 | + return err | ||
90 | + }} | ||
91 | + | ||
92 | +{set_members}{set_special} | ||
93 | + return nil | ||
94 | +}} | ||
95 | +""" | ||
96 | + | ||
97 | + | ||
98 | # Takes the documentation object of a specific type and returns | ||
99 | # that type's documentation and its member's docs. | ||
100 | def qapi_to_golang_struct_docs(doc: QAPIDoc) -> (str, Dict[str, str]): | ||
101 | @@ -XXX,XX +XXX,XX @@ def get_struct_field( | ||
102 | qapi_name: str, | ||
103 | qapi_type_name: str, | ||
104 | field_doc: str, | ||
105 | + within_nullable_struct: bool, | ||
106 | is_optional: bool, | ||
107 | is_variant: bool, | ||
108 | -) -> dict[str:str]: | ||
109 | +) -> Tuple[dict[str:str], bool]: | ||
110 | field = qapi_to_field_name(qapi_name) | ||
111 | member_type = qapi_schema_type_to_go_type(qapi_type_name) | ||
112 | + is_nullable = False | ||
113 | |||
114 | optional = "" | ||
115 | if is_optional: | ||
116 | - if member_type not in self.accept_null_types: | ||
117 | + if member_type in self.accept_null_types: | ||
118 | + is_nullable = True | ||
119 | + else: | ||
120 | optional = ",omitempty" | ||
121 | |||
122 | # Use pointer to type when field is optional | ||
123 | isptr = "*" if is_optional and member_type[0] not in "*[" else "" | ||
124 | |||
125 | + if within_nullable_struct: | ||
126 | + # Within a struct which has a field of type that can hold JSON NULL, | ||
127 | + # we have to _not_ use a pointer, otherwise the Marshal methods are | ||
128 | + # not called. | ||
129 | + isptr = "" if member_type in self.accept_null_types else isptr | ||
130 | + | ||
131 | fieldtag = ( | ||
132 | '`json:"-"`' if is_variant else f'`json:"{qapi_name}{optional}"`' | ||
133 | ) | ||
134 | @@ -XXX,XX +XXX,XX @@ def get_struct_field( | ||
135 | if field_doc != "": | ||
136 | arg["doc"] = field_doc | ||
137 | |||
138 | - return arg | ||
139 | + return arg, is_nullable | ||
140 | + | ||
141 | + | ||
142 | +# This helper is used whithin a struct that has members that accept JSON NULL. | ||
143 | +def map_and_set( | ||
144 | + is_nullable: bool, field: str, field_is_optional: bool, name: str | ||
145 | +) -> Tuple[str, str]: | ||
146 | + mapstr = "" | ||
147 | + setstr = "" | ||
148 | + if is_nullable: | ||
149 | + mapstr = f""" | ||
150 | + if val, absent := s.{field}.ToAnyOrAbsent(); !absent {{ | ||
151 | + m["{name}"] = val | ||
152 | + }} | ||
153 | +""" | ||
154 | + setstr += f""" | ||
155 | + if _, absent := (&tmp.{field}).ToAnyOrAbsent(); !absent {{ | ||
156 | + s.{field} = &tmp.{field} | ||
157 | + }} | ||
158 | +""" | ||
159 | + elif field_is_optional: | ||
160 | + mapstr = f""" | ||
161 | + if s.{field} != nil {{ | ||
162 | + m["{name}"] = s.{field} | ||
163 | + }} | ||
164 | +""" | ||
165 | + setstr = f""" s.{field} = tmp.{field}\n""" | ||
166 | + else: | ||
167 | + mapstr = f""" m["{name}"] = s.{field}\n""" | ||
168 | + setstr = f""" s.{field} = tmp.{field}\n""" | ||
169 | + | ||
170 | + return mapstr, setstr | ||
171 | + | ||
172 | + | ||
173 | +def recursive_base_nullable( | ||
174 | + self: QAPISchemaGenGolangVisitor, base: Optional[QAPISchemaObjectType] | ||
175 | +) -> Tuple[List[dict[str:str]], str, str, str, str]: | ||
176 | + fields: List[dict[str:str]] = [] | ||
177 | + map_members = "" | ||
178 | + set_members = "" | ||
179 | + map_special = "" | ||
180 | + set_special = "" | ||
181 | + | ||
182 | + if not base: | ||
183 | + return fields, map_members, set_members, map_special, set_special | ||
184 | + | ||
185 | + doc = self.docmap.get(base.name, None) | ||
186 | + _, docfields = qapi_to_golang_struct_docs(doc) | ||
187 | + | ||
188 | + if base.base is not None: | ||
189 | + embed_base = self.schema.lookup_entity(base.base.name) | ||
190 | + ( | ||
191 | + fields, | ||
192 | + map_members, | ||
193 | + set_members, | ||
194 | + map_special, | ||
195 | + set_special, | ||
196 | + ) = recursive_base_nullable(self, embed_base) | ||
197 | + | ||
198 | + for member in base.local_members: | ||
199 | + field_doc = docfields.get(member.name, "") | ||
200 | + field, _ = get_struct_field( | ||
201 | + self, | ||
202 | + member.name, | ||
203 | + member.type.name, | ||
204 | + field_doc, | ||
205 | + True, | ||
206 | + member.optional, | ||
207 | + False, | ||
208 | + ) | ||
209 | + fields.append(field) | ||
210 | + | ||
211 | + member_type = qapi_schema_type_to_go_type(member.type.name) | ||
212 | + nullable = member_type in self.accept_null_types | ||
213 | + field_name = qapi_to_field_name(member.name) | ||
214 | + tomap, toset = map_and_set( | ||
215 | + nullable, field_name, member.optional, member.name | ||
216 | + ) | ||
217 | + if nullable: | ||
218 | + map_special += tomap | ||
219 | + set_special += toset | ||
220 | + else: | ||
221 | + map_members += tomap | ||
222 | + set_members += toset | ||
223 | + | ||
224 | + return fields, map_members, set_members, map_special, set_special | ||
225 | + | ||
226 | + | ||
227 | +# Helper function. This is executed when the QAPI schema has members | ||
228 | +# that could accept JSON NULL (e.g: StrOrNull in QEMU"s QAPI schema). | ||
229 | +# This struct will need to be extended with Marshal/Unmarshal methods to | ||
230 | +# properly handle such atypical members. | ||
231 | +# | ||
232 | +# Only the Marshallaing methods are generated but we do need to iterate over | ||
233 | +# all the members to properly set/check them in those methods. | ||
234 | +def struct_with_nullable_generate_marshal( | ||
235 | + self: QAPISchemaGenGolangVisitor, | ||
236 | + name: str, | ||
237 | + base: Optional[QAPISchemaObjectType], | ||
238 | + members: List[QAPISchemaObjectTypeMember], | ||
239 | + variants: Optional[QAPISchemaVariants], | ||
240 | +) -> str: | ||
241 | + ( | ||
242 | + fields, | ||
243 | + map_members, | ||
244 | + set_members, | ||
245 | + map_special, | ||
246 | + set_special, | ||
247 | + ) = recursive_base_nullable(self, base) | ||
248 | + | ||
249 | + doc = self.docmap.get(name, None) | ||
250 | + _, docfields = qapi_to_golang_struct_docs(doc) | ||
251 | + | ||
252 | + if members: | ||
253 | + for member in members: | ||
254 | + field_doc = docfields.get(member.name, "") | ||
255 | + field, _ = get_struct_field( | ||
256 | + self, | ||
257 | + member.name, | ||
258 | + member.type.name, | ||
259 | + field_doc, | ||
260 | + True, | ||
261 | + member.optional, | ||
262 | + False, | ||
263 | + ) | ||
264 | + fields.append(field) | ||
265 | + | ||
266 | + member_type = qapi_schema_type_to_go_type(member.type.name) | ||
267 | + nullable = member_type in self.accept_null_types | ||
268 | + tomap, toset = map_and_set( | ||
269 | + nullable, | ||
270 | + qapi_to_field_name(member.name), | ||
271 | + member.optional, | ||
272 | + member.name, | ||
273 | + ) | ||
274 | + if nullable: | ||
275 | + map_special += tomap | ||
276 | + set_special += toset | ||
277 | + else: | ||
278 | + map_members += tomap | ||
279 | + set_members += toset | ||
280 | + | ||
281 | + if variants: | ||
282 | + for variant in variants.variants: | ||
283 | + if variant.type.is_implicit(): | ||
284 | + continue | ||
285 | + | ||
286 | + field, _ = get_struct_field( | ||
287 | + self, | ||
288 | + variant.name, | ||
289 | + variant.type.name, | ||
290 | + True, | ||
291 | + variant.optional, | ||
292 | + True, | ||
293 | + ) | ||
294 | + fields.append(field) | ||
295 | + | ||
296 | + member_type = qapi_schema_type_to_go_type(variant.type.name) | ||
297 | + nullable = member_type in self.accept_null_types | ||
298 | + tomap, toset = map_and_set( | ||
299 | + nullable, | ||
300 | + qapi_to_field_name(variant.name), | ||
301 | + variant.optional, | ||
302 | + variant.name, | ||
303 | + ) | ||
304 | + if nullable: | ||
305 | + map_special += tomap | ||
306 | + set_special += toset | ||
307 | + else: | ||
308 | + map_members += tomap | ||
309 | + set_members += toset | ||
310 | + | ||
311 | + type_name = qapi_to_go_type_name(name) | ||
312 | + struct = generate_struct_type("", args=fields, indent=1) | ||
313 | + return string_to_code( | ||
314 | + TEMPLATE_STRUCT_WITH_NULLABLE_MARSHAL.format( | ||
315 | + struct=struct[1:-1], | ||
316 | + type_name=type_name, | ||
317 | + map_members=map_members, | ||
318 | + map_special=map_special, | ||
319 | + set_members=set_members, | ||
320 | + set_special=set_special, | ||
321 | + ) | ||
322 | + ) | ||
323 | |||
324 | |||
325 | def recursive_base( | ||
326 | self: QAPISchemaGenGolangVisitor, | ||
327 | base: Optional[QAPISchemaObjectType], | ||
328 | -) -> List[dict[str:str]]: | ||
329 | + discriminator: Optional[str] = None, | ||
330 | +) -> Tuple[List[dict[str:str]], bool]: | ||
331 | fields: List[dict[str:str]] = [] | ||
332 | + with_nullable = False | ||
333 | |||
334 | if not base: | ||
335 | - return fields | ||
336 | + return fields, with_nullable | ||
337 | |||
338 | if base.base is not None: | ||
339 | embed_base = self.schema.lookup_entity(base.base.name) | ||
340 | - fields = recursive_base(self, embed_base) | ||
341 | + fields, with_nullable = recursive_base(self, embed_base, discriminator) | ||
342 | |||
343 | doc = self.docmap.get(base.name, None) | ||
344 | _, docfields = qapi_to_golang_struct_docs(doc) | ||
345 | |||
346 | for member in base.local_members: | ||
347 | + if discriminator and member.name == discriminator: | ||
348 | + continue | ||
349 | + | ||
350 | field_doc = docfields.get(member.name, "") | ||
351 | - field = get_struct_field( | ||
352 | + field, nullable = get_struct_field( | ||
353 | self, | ||
354 | member.name, | ||
355 | member.type.name, | ||
356 | field_doc, | ||
357 | + False, | ||
358 | member.optional, | ||
359 | False, | ||
360 | ) | ||
361 | fields.append(field) | ||
362 | + with_nullable = True if nullable else with_nullable | ||
363 | |||
364 | - return fields | ||
365 | + return fields, with_nullable | ||
366 | |||
367 | |||
368 | # Helper function that is used for most of QAPI types | ||
369 | @@ -XXX,XX +XXX,XX @@ def qapi_to_golang_struct( | ||
370 | indent: int = 0, | ||
371 | doc_enabled: bool = True, | ||
372 | ) -> str: | ||
373 | - fields = recursive_base(self, base) | ||
374 | + discriminator = None if not variants else variants.tag_member.name | ||
375 | + fields, with_nullable = recursive_base(self, base, discriminator) | ||
376 | |||
377 | doc = self.docmap.get(name, None) | ||
378 | type_doc, docfields = qapi_to_golang_struct_docs(doc) | ||
379 | @@ -XXX,XX +XXX,XX @@ def qapi_to_golang_struct( | ||
380 | if members: | ||
381 | for member in members: | ||
382 | field_doc = docfields.get(member.name, "") if doc_enabled else "" | ||
383 | - field = get_struct_field( | ||
384 | + field, nullable = get_struct_field( | ||
385 | self, | ||
386 | member.name, | ||
387 | member.type.name, | ||
388 | field_doc, | ||
389 | + False, | ||
390 | member.optional, | ||
391 | False, | ||
392 | ) | ||
393 | fields.append(field) | ||
394 | + with_nullable = True if nullable else with_nullable | ||
395 | |||
396 | exists = {} | ||
397 | if variants: | ||
398 | @@ -XXX,XX +XXX,XX @@ def qapi_to_golang_struct( | ||
399 | |||
400 | exists[variant.name] = True | ||
401 | field_doc = docfields.get(variant.name, "") if doc_enabled else "" | ||
402 | - field = get_struct_field( | ||
403 | + field, nullable = get_struct_field( | ||
404 | self, | ||
405 | variant.name, | ||
406 | variant.type.name, | ||
407 | field_doc, | ||
408 | + False, | ||
409 | True, | ||
410 | True, | ||
411 | ) | ||
412 | fields.append(field) | ||
413 | + with_nullable = True if nullable else with_nullable | ||
414 | |||
415 | type_name = qapi_to_go_type_name(name) | ||
416 | content = string_to_code( | ||
417 | @@ -XXX,XX +XXX,XX @@ def qapi_to_golang_struct( | ||
418 | type_name, type_doc=type_doc, args=fields, indent=indent | ||
419 | ) | ||
420 | ) | ||
421 | + if with_nullable: | ||
422 | + content += struct_with_nullable_generate_marshal( | ||
423 | + self, name, base, members, variants | ||
424 | + ) | ||
425 | return content | ||
426 | |||
427 | |||
428 | @@ -XXX,XX +XXX,XX @@ def generate_template_alternate( | ||
429 | name: str, | ||
430 | variants: Optional[QAPISchemaVariants], | ||
431 | ) -> str: | ||
432 | + absent_check_fields = "" | ||
433 | args: List[dict[str:str]] = [] | ||
434 | nullable = name in self.accept_null_types | ||
435 | if nullable: | ||
436 | @@ -XXX,XX +XXX,XX @@ def generate_template_alternate( | ||
437 | assert nullable | ||
438 | continue | ||
439 | |||
440 | + if nullable: | ||
441 | + absent_check_fields += string_to_code( | ||
442 | + TEMPLATE_ALTERNATE_NULLABLE_CHECK[1:].format( | ||
443 | + var_name=var_name | ||
444 | + ) | ||
445 | + ) | ||
446 | skip_indent = 1 + len(FOUR_SPACES) | ||
447 | if marshal_check_fields == "": | ||
448 | skip_indent = 1 | ||
449 | @@ -XXX,XX +XXX,XX @@ def generate_template_alternate( | ||
450 | ].format(var_name=var_name, var_type=var_type) | ||
451 | |||
452 | content += string_to_code(generate_struct_type(name, args=args)) | ||
453 | + if nullable: | ||
454 | + content += string_to_code( | ||
455 | + TEMPLATE_ALTERNATE_NULLABLE.format( | ||
456 | + name=name, absent_check_fields=absent_check_fields[:-1] | ||
457 | + ) | ||
458 | + ) | ||
459 | content += string_to_code( | ||
460 | TEMPLATE_ALTERNATE_METHODS.format( | ||
461 | name=name, | ||
462 | @@ -XXX,XX +XXX,XX @@ def visit_begin(self, schema: QAPISchema) -> None: | ||
463 | ) | ||
464 | self.types[qapitype] += generate_template_imports(imports) | ||
465 | |||
466 | + self.types["alternate"] += string_to_code(TEMPLATE_ALTERNATE) | ||
467 | + | ||
468 | def visit_end(self) -> None: | ||
469 | del self.schema | ||
470 | self.types["enum"] += generate_content_from_dict(self.enums) | ||
471 | -- | ||
472 | 2.48.1 | diff view generated by jsdifflib |
Deleted patch | |||
---|---|---|---|
1 | This patch handles QAPI union types and generates the equivalent data | ||
2 | structures and methods in Go to handle it. | ||
3 | 1 | ||
4 | The QAPI union type has two types of fields: The @base and the | ||
5 | @Variants members. The @base fields can be considered common members | ||
6 | for the union while only one field maximum is set for the @Variants. | ||
7 | |||
8 | In the QAPI specification, it defines a @discriminator field, which is | ||
9 | an Enum type. The purpose of the @discriminator is to identify which | ||
10 | @variant type is being used. | ||
11 | |||
12 | For the @discriminator's enum that are not handled by the QAPI Union, | ||
13 | we add in the Go struct a separate block as "Unbranched enum fields". | ||
14 | The rationale for this extra block is to allow the user to pass that | ||
15 | enum value under the discriminator, without extra payload. | ||
16 | |||
17 | The union types implement the Marshaler and Unmarshaler interfaces to | ||
18 | seamless decode from JSON objects to Golang structs and vice versa. | ||
19 | |||
20 | qapi: | ||
21 | | ## | ||
22 | | # @SetPasswordOptions: | ||
23 | | # | ||
24 | | # Options for set_password. | ||
25 | | # | ||
26 | | # @protocol: | ||
27 | | # - 'vnc' to modify the VNC server password | ||
28 | | # - 'spice' to modify the Spice server password | ||
29 | | # | ||
30 | | # @password: the new password | ||
31 | | # | ||
32 | | # @connected: How to handle existing clients when changing the | ||
33 | | # password. If nothing is specified, defaults to 'keep'. For | ||
34 | | # VNC, only 'keep' is currently implemented. | ||
35 | | # | ||
36 | | # Since: 7.0 | ||
37 | | ## | ||
38 | | { 'union': 'SetPasswordOptions', | ||
39 | | 'base': { 'protocol': 'DisplayProtocol', | ||
40 | | 'password': 'str', | ||
41 | | '*connected': 'SetPasswordAction' }, | ||
42 | | 'discriminator': 'protocol', | ||
43 | | 'data': { 'vnc': 'SetPasswordOptionsVnc' } } | ||
44 | |||
45 | go: | ||
46 | | // Options for set_password. | ||
47 | | // | ||
48 | | // Since: 7.0 | ||
49 | | type SetPasswordOptions struct { | ||
50 | | // the new password | ||
51 | | Password string `json:"password"` | ||
52 | | // How to handle existing clients when changing the password. If | ||
53 | | // nothing is specified, defaults to 'keep'. For VNC, only 'keep' | ||
54 | | // is currently implemented. | ||
55 | | Connected *SetPasswordAction `json:"connected,omitempty"` | ||
56 | | // Variants fields | ||
57 | | Vnc *SetPasswordOptionsVnc `json:"-"` | ||
58 | | // Unbranched enum fields | ||
59 | | Spice bool `json:"-"` | ||
60 | | } | ||
61 | | | ||
62 | | func (s SetPasswordOptions) MarshalJSON() ([]byte, error) { | ||
63 | | ... | ||
64 | | } | ||
65 | |||
66 | | | ||
67 | | func (s *SetPasswordOptions) UnmarshalJSON(data []byte) error { | ||
68 | | ... | ||
69 | | } | ||
70 | |||
71 | Signed-off-by: Victor Toso <victortoso@redhat.com> | ||
72 | --- | ||
73 | scripts/qapi/golang/golang.py | 208 +++++++++++++++++++++++++++++++++- | ||
74 | scripts/qapi/golang/utils.go | 12 ++ | ||
75 | 2 files changed, 217 insertions(+), 3 deletions(-) | ||
76 | |||
77 | diff --git a/scripts/qapi/golang/golang.py b/scripts/qapi/golang/golang.py | ||
78 | index XXXXXXX..XXXXXXX 100644 | ||
79 | --- a/scripts/qapi/golang/golang.py | ||
80 | +++ b/scripts/qapi/golang/golang.py | ||
81 | @@ -XXX,XX +XXX,XX @@ | ||
82 | """ | ||
83 | |||
84 | |||
85 | +TEMPLATE_UNION_CHECK_VARIANT_FIELD = """ | ||
86 | + if s.{field} != nil && err == nil {{ | ||
87 | + if len(bytes) != 0 {{ | ||
88 | + err = errors.New(`multiple variant fields set`) | ||
89 | + }} else if err = unwrapToMap(m, s.{field}); err == nil {{ | ||
90 | + m["{discriminator}"] = {go_enum_value} | ||
91 | + bytes, err = json.Marshal(m) | ||
92 | + }} | ||
93 | + }} | ||
94 | +""" | ||
95 | + | ||
96 | +TEMPLATE_UNION_CHECK_UNBRANCHED_FIELD = """ | ||
97 | + if s.{field} && err == nil {{ | ||
98 | + if len(bytes) != 0 {{ | ||
99 | + err = errors.New(`multiple variant fields set`) | ||
100 | + }} else {{ | ||
101 | + m["{discriminator}"] = {go_enum_value} | ||
102 | + bytes, err = json.Marshal(m) | ||
103 | + }} | ||
104 | + }} | ||
105 | +""" | ||
106 | + | ||
107 | +TEMPLATE_UNION_DRIVER_VARIANT_CASE = """ | ||
108 | + case {go_enum_value}: | ||
109 | + s.{field} = new({member_type}) | ||
110 | + if err := json.Unmarshal(data, s.{field}); err != nil {{ | ||
111 | + s.{field} = nil | ||
112 | + return err | ||
113 | + }}""" | ||
114 | + | ||
115 | +TEMPLATE_UNION_DRIVER_UNBRANCHED_CASE = """ | ||
116 | + case {go_enum_value}: | ||
117 | + s.{field} = true | ||
118 | +""" | ||
119 | + | ||
120 | +TEMPLATE_UNION_METHODS = """ | ||
121 | +func (s {type_name}) MarshalJSON() ([]byte, error) {{ | ||
122 | + var bytes []byte | ||
123 | + var err error | ||
124 | + m := make(map[string]any) | ||
125 | + {{ | ||
126 | + type Alias {type_name} | ||
127 | + v := Alias(s) | ||
128 | + unwrapToMap(m, &v) | ||
129 | + }} | ||
130 | +{check_fields} | ||
131 | + if err != nil {{ | ||
132 | + return nil, fmt.Errorf("marshal {type_name} due:'%s' struct='%+v'", err, s) | ||
133 | + }} else if len(bytes) == 0 {{ | ||
134 | + return nil, fmt.Errorf("marshal {type_name} unsupported, struct='%+v'", s) | ||
135 | + }} | ||
136 | + return bytes, nil | ||
137 | +}} | ||
138 | + | ||
139 | +func (s *{type_name}) UnmarshalJSON(data []byte) error {{ | ||
140 | +{base_type_def} | ||
141 | + tmp := struct {{ | ||
142 | + {base_type_name} | ||
143 | + }}{{}} | ||
144 | + | ||
145 | + if err := json.Unmarshal(data, &tmp); err != nil {{ | ||
146 | + return err | ||
147 | + }} | ||
148 | +{base_type_assign_unmarshal} | ||
149 | + switch tmp.{discriminator} {{ | ||
150 | +{driver_cases} | ||
151 | + default: | ||
152 | + return fmt.Errorf("unmarshal {type_name} received unrecognized value '%s'", | ||
153 | + tmp.{discriminator}) | ||
154 | + }} | ||
155 | + return nil | ||
156 | +}} | ||
157 | +""" | ||
158 | + | ||
159 | + | ||
160 | # Takes the documentation object of a specific type and returns | ||
161 | # that type's documentation and its member's docs. | ||
162 | def qapi_to_golang_struct_docs(doc: QAPIDoc) -> (str, Dict[str, str]): | ||
163 | @@ -XXX,XX +XXX,XX @@ def qapi_name_is_object(name: str) -> bool: | ||
164 | return name.startswith("q_obj_") | ||
165 | |||
166 | |||
167 | +def qapi_base_name_to_parent(name: str) -> str: | ||
168 | + if qapi_name_is_base(name): | ||
169 | + name = name[6:-5] | ||
170 | + return name | ||
171 | + | ||
172 | + | ||
173 | def qapi_to_field_name(name: str) -> str: | ||
174 | return name.title().replace("_", "").replace("-", "") | ||
175 | |||
176 | @@ -XXX,XX +XXX,XX @@ def recursive_base( | ||
177 | embed_base = self.schema.lookup_entity(base.base.name) | ||
178 | fields, with_nullable = recursive_base(self, embed_base, discriminator) | ||
179 | |||
180 | - doc = self.docmap.get(base.name, None) | ||
181 | + doc = self.docmap.get(qapi_base_name_to_parent(base.name), None) | ||
182 | _, docfields = qapi_to_golang_struct_docs(doc) | ||
183 | |||
184 | for member in base.local_members: | ||
185 | @@ -XXX,XX +XXX,XX @@ def qapi_to_golang_struct( | ||
186 | fields.append(field) | ||
187 | with_nullable = True if nullable else with_nullable | ||
188 | |||
189 | + if info.defn_meta == "union" and variants: | ||
190 | + enum_name = variants.tag_member.type.name | ||
191 | + enum_obj = self.schema.lookup_entity(enum_name) | ||
192 | + if len(exists) != len(enum_obj.members): | ||
193 | + fields.append({"comment": "Unbranched enum fields"}) | ||
194 | + for member in enum_obj.members: | ||
195 | + if member.name in exists: | ||
196 | + continue | ||
197 | + | ||
198 | + field_doc = ( | ||
199 | + docfields.get(member.name, "") if doc_enabled else "" | ||
200 | + ) | ||
201 | + field, nullable = get_struct_field( | ||
202 | + self, member.name, "bool", field_doc, False, False, True | ||
203 | + ) | ||
204 | + fields.append(field) | ||
205 | + with_nullable = True if nullable else with_nullable | ||
206 | + | ||
207 | type_name = qapi_to_go_type_name(name) | ||
208 | content = string_to_code( | ||
209 | generate_struct_type( | ||
210 | @@ -XXX,XX +XXX,XX @@ def qapi_to_golang_struct( | ||
211 | return content | ||
212 | |||
213 | |||
214 | +def qapi_to_golang_methods_union( | ||
215 | + self: QAPISchemaGenGolangVisitor, | ||
216 | + name: str, | ||
217 | + base: Optional[QAPISchemaObjectType], | ||
218 | + variants: Optional[QAPISchemaVariants], | ||
219 | +) -> str: | ||
220 | + type_name = qapi_to_go_type_name(name) | ||
221 | + | ||
222 | + assert base | ||
223 | + base_type_assign_unmarshal = "" | ||
224 | + base_type_name = qapi_to_go_type_name(base.name) | ||
225 | + base_type_def = qapi_to_golang_struct( | ||
226 | + self, | ||
227 | + base.name, | ||
228 | + base.info, | ||
229 | + base.ifcond, | ||
230 | + base.features, | ||
231 | + base.base, | ||
232 | + base.members, | ||
233 | + base.branches, | ||
234 | + indent=1, | ||
235 | + doc_enabled=False, | ||
236 | + ) | ||
237 | + | ||
238 | + discriminator = qapi_to_field_name(variants.tag_member.name) | ||
239 | + for member in base.local_members: | ||
240 | + field = qapi_to_field_name(member.name) | ||
241 | + if field == discriminator: | ||
242 | + continue | ||
243 | + base_type_assign_unmarshal += f""" | ||
244 | + s.{field} = tmp.{field}""" | ||
245 | + | ||
246 | + driver_cases = "" | ||
247 | + check_fields = "" | ||
248 | + exists = {} | ||
249 | + enum_name = variants.tag_member.type.name | ||
250 | + if variants: | ||
251 | + for var in variants.variants: | ||
252 | + if var.type.is_implicit(): | ||
253 | + continue | ||
254 | + | ||
255 | + field = qapi_to_field_name(var.name) | ||
256 | + enum_value = qapi_to_field_name_enum(var.name) | ||
257 | + member_type = qapi_schema_type_to_go_type(var.type.name) | ||
258 | + go_enum_value = f"""{enum_name}{enum_value}""" | ||
259 | + exists[go_enum_value] = True | ||
260 | + | ||
261 | + check_fields += TEMPLATE_UNION_CHECK_VARIANT_FIELD.format( | ||
262 | + field=field, | ||
263 | + discriminator=variants.tag_member.name, | ||
264 | + go_enum_value=go_enum_value, | ||
265 | + ) | ||
266 | + driver_cases += TEMPLATE_UNION_DRIVER_VARIANT_CASE.format( | ||
267 | + go_enum_value=go_enum_value, | ||
268 | + field=field, | ||
269 | + member_type=member_type, | ||
270 | + ) | ||
271 | + | ||
272 | + enum_obj = self.schema.lookup_entity(enum_name) | ||
273 | + if len(exists) != len(enum_obj.members): | ||
274 | + for member in enum_obj.members: | ||
275 | + value = qapi_to_field_name_enum(member.name) | ||
276 | + go_enum_value = f"""{enum_name}{value}""" | ||
277 | + | ||
278 | + if go_enum_value in exists: | ||
279 | + continue | ||
280 | + | ||
281 | + field = qapi_to_field_name(member.name) | ||
282 | + | ||
283 | + check_fields += TEMPLATE_UNION_CHECK_UNBRANCHED_FIELD.format( | ||
284 | + field=field, | ||
285 | + discriminator=variants.tag_member.name, | ||
286 | + go_enum_value=go_enum_value, | ||
287 | + ) | ||
288 | + driver_cases += TEMPLATE_UNION_DRIVER_UNBRANCHED_CASE.format( | ||
289 | + go_enum_value=go_enum_value, | ||
290 | + field=field, | ||
291 | + ) | ||
292 | + | ||
293 | + return string_to_code( | ||
294 | + TEMPLATE_UNION_METHODS.format( | ||
295 | + type_name=type_name, | ||
296 | + check_fields=check_fields[1:], | ||
297 | + base_type_def=base_type_def[1:], | ||
298 | + base_type_name=base_type_name, | ||
299 | + base_type_assign_unmarshal=base_type_assign_unmarshal, | ||
300 | + discriminator=discriminator, | ||
301 | + driver_cases=driver_cases[1:], | ||
302 | + ) | ||
303 | + ) | ||
304 | + | ||
305 | + | ||
306 | def generate_template_alternate( | ||
307 | self: QAPISchemaGenGolangVisitor, | ||
308 | name: str, | ||
309 | @@ -XXX,XX +XXX,XX @@ def __init__(self, _: str): | ||
310 | "alternate": ["encoding/json", "errors", "fmt"], | ||
311 | "enum": [], | ||
312 | "struct": ["encoding/json"], | ||
313 | + "union": ["encoding/json", "errors", "fmt"], | ||
314 | } | ||
315 | |||
316 | self.schema: QAPISchema | ||
317 | @@ -XXX,XX +XXX,XX @@ def __init__(self, _: str): | ||
318 | self.enums: dict[str, str] = {} | ||
319 | self.alternates: dict[str, str] = {} | ||
320 | self.structs: dict[str, str] = {} | ||
321 | + self.unions: dict[str, str] = {} | ||
322 | self.accept_null_types = [] | ||
323 | self.docmap = {} | ||
324 | |||
325 | @@ -XXX,XX +XXX,XX @@ def visit_end(self) -> None: | ||
326 | self.types["enum"] += generate_content_from_dict(self.enums) | ||
327 | self.types["alternate"] += generate_content_from_dict(self.alternates) | ||
328 | self.types["struct"] += generate_content_from_dict(self.structs) | ||
329 | + self.types["union"] += generate_content_from_dict(self.unions) | ||
330 | |||
331 | def visit_object_type( | ||
332 | self, | ||
333 | @@ -XXX,XX +XXX,XX @@ def visit_object_type( | ||
334 | members: List[QAPISchemaObjectTypeMember], | ||
335 | branches: Optional[QAPISchemaBranches], | ||
336 | ) -> None: | ||
337 | - # Do not handle anything besides struct. | ||
338 | + # Do not handle anything besides struct and unions. | ||
339 | if ( | ||
340 | name == self.schema.the_empty_object_type.name | ||
341 | or not isinstance(name, str) | ||
342 | - or info.defn_meta not in ["struct"] | ||
343 | + or info.defn_meta not in ["struct", "union"] | ||
344 | ): | ||
345 | return | ||
346 | |||
347 | @@ -XXX,XX +XXX,XX @@ def visit_object_type( | ||
348 | self, name, info, ifcond, features, base, members, branches | ||
349 | ) | ||
350 | ) | ||
351 | + else: | ||
352 | + assert name not in self.unions | ||
353 | + self.unions[name] = qapi_to_golang_struct( | ||
354 | + self, name, info, ifcond, features, base, members, branches | ||
355 | + ) | ||
356 | + self.unions[name] += qapi_to_golang_methods_union( | ||
357 | + self, name, base, branches | ||
358 | + ) | ||
359 | |||
360 | def visit_alternate_type( | ||
361 | self, | ||
362 | diff --git a/scripts/qapi/golang/utils.go b/scripts/qapi/golang/utils.go | ||
363 | index XXXXXXX..XXXXXXX 100644 | ||
364 | --- a/scripts/qapi/golang/utils.go | ||
365 | +++ b/scripts/qapi/golang/utils.go | ||
366 | @@ -XXX,XX +XXX,XX @@ package qapi | ||
367 | |||
368 | import ( | ||
369 | "encoding/json" | ||
370 | + "fmt" | ||
371 | "strings" | ||
372 | ) | ||
373 | |||
374 | @@ -XXX,XX +XXX,XX @@ func strictDecode(into interface{}, from []byte) error { | ||
375 | } | ||
376 | return nil | ||
377 | } | ||
378 | + | ||
379 | +// This helper is used to move struct's fields into a map. | ||
380 | +// This function is useful to merge JSON objects. | ||
381 | +func unwrapToMap(m map[string]any, data any) error { | ||
382 | + if bytes, err := json.Marshal(&data); err != nil { | ||
383 | + return fmt.Errorf("unwrapToMap: %s", err) | ||
384 | + } else if err := json.Unmarshal(bytes, &m); err != nil { | ||
385 | + return fmt.Errorf("unwrapToMap: %s, data=%s", err, string(bytes)) | ||
386 | + } | ||
387 | + return nil | ||
388 | +} | ||
389 | -- | ||
390 | 2.48.1 | diff view generated by jsdifflib |
Deleted patch | |||
---|---|---|---|
1 | This patch handles QAPI event types and generates data structures in | ||
2 | Go that handles it. | ||
3 | 1 | ||
4 | Note that the timestamp is part of the first layer of unmarshal, so it | ||
5 | s a member of protocol.go's Message type. | ||
6 | |||
7 | Example: | ||
8 | qapi: | ||
9 | | ## | ||
10 | | # @MEMORY_DEVICE_SIZE_CHANGE: | ||
11 | | # | ||
12 | | # Emitted when the size of a memory device changes. Only emitted for | ||
13 | | # memory devices that can actually change the size (e.g., virtio-mem | ||
14 | | # due to guest action). | ||
15 | | # | ||
16 | | # @id: device's ID | ||
17 | | # | ||
18 | | # @size: the new size of memory that the device provides | ||
19 | | # | ||
20 | | # @qom-path: path to the device object in the QOM tree (since 6.2) | ||
21 | | # | ||
22 | | # .. note:: This event is rate-limited. | ||
23 | | # | ||
24 | | # Since: 5.1 | ||
25 | | # | ||
26 | | # .. qmp-example:: | ||
27 | | # | ||
28 | | # <- { "event": "MEMORY_DEVICE_SIZE_CHANGE", | ||
29 | | # "data": { "id": "vm0", "size": 1073741824, | ||
30 | | # "qom-path": "/machine/unattached/device[2]" }, | ||
31 | | # "timestamp": { "seconds": 1588168529, "microseconds": 201316 } } | ||
32 | | ## | ||
33 | | { 'event': 'MEMORY_DEVICE_SIZE_CHANGE', | ||
34 | | 'data': { '*id': 'str', 'size': 'size', 'qom-path' : 'str'} } | ||
35 | |||
36 | go: | ||
37 | | // Emitted when the size of a memory device changes. Only emitted for | ||
38 | | // memory devices that can actually change the size (e.g., virtio-mem | ||
39 | | // due to guest action). | ||
40 | | // | ||
41 | | // .. note:: This event is rate-limited. | ||
42 | | // | ||
43 | | // Since: 5.1 | ||
44 | | // | ||
45 | | // .. qmp-example:: <- { "event": "MEMORY_DEVICE_SIZE_CHANGE", | ||
46 | | // "data": { "id": "vm0", "size": 1073741824, "qom-path": | ||
47 | | // "/machine/unattached/device[2]" }, "timestamp": { "seconds": | ||
48 | | // 1588168529, "microseconds": 201316 } } | ||
49 | | type MemoryDeviceSizeChangeEvent struct { | ||
50 | | // device's ID | ||
51 | | Id *string `json:"id,omitempty"` | ||
52 | | // the new size of memory that the device provides | ||
53 | | Size uint64 `json:"size"` | ||
54 | | // path to the device object in the QOM tree (since 6.2) | ||
55 | | QomPath string `json:"qom-path"` | ||
56 | | } | ||
57 | |||
58 | Signed-off-by: Victor Toso <victortoso@redhat.com> | ||
59 | --- | ||
60 | scripts/qapi/golang/golang.py | 49 ++++++++++++++++++++++++++++++++--- | ||
61 | 1 file changed, 46 insertions(+), 3 deletions(-) | ||
62 | |||
63 | diff --git a/scripts/qapi/golang/golang.py b/scripts/qapi/golang/golang.py | ||
64 | index XXXXXXX..XXXXXXX 100644 | ||
65 | --- a/scripts/qapi/golang/golang.py | ||
66 | +++ b/scripts/qapi/golang/golang.py | ||
67 | @@ -XXX,XX +XXX,XX @@ def qapi_to_field_name_enum(name: str) -> str: | ||
68 | return name.title().replace("-", "") | ||
69 | |||
70 | |||
71 | -def qapi_to_go_type_name(name: str) -> str: | ||
72 | +def qapi_to_go_type_name(name: str, meta: Optional[str] = None) -> str: | ||
73 | # We want to keep CamelCase for Golang types. We want to avoid removing | ||
74 | # already set CameCase names while fixing uppercase ones, eg: | ||
75 | # 1) q_obj_SocketAddress_base -> SocketAddressBase | ||
76 | @@ -XXX,XX +XXX,XX @@ def qapi_to_go_type_name(name: str) -> str: | ||
77 | |||
78 | name += "".join(word.title() for word in words[1:]) | ||
79 | |||
80 | + # Handle specific meta suffix | ||
81 | + types = ["event"] | ||
82 | + if meta in types: | ||
83 | + name = name[:-3] if name.endswith("Arg") else name | ||
84 | + name += meta.title().replace(" ", "") | ||
85 | + | ||
86 | return name | ||
87 | |||
88 | |||
89 | @@ -XXX,XX +XXX,XX @@ def qapi_to_golang_struct( | ||
90 | fields.append(field) | ||
91 | with_nullable = True if nullable else with_nullable | ||
92 | |||
93 | - type_name = qapi_to_go_type_name(name) | ||
94 | + type_name = qapi_to_go_type_name(name, info.defn_meta) | ||
95 | + | ||
96 | content = string_to_code( | ||
97 | generate_struct_type( | ||
98 | type_name, type_doc=type_doc, args=fields, indent=indent | ||
99 | @@ -XXX,XX +XXX,XX @@ def generate_template_alternate( | ||
100 | return "\n" + content | ||
101 | |||
102 | |||
103 | +def generate_template_event(events: dict[str, Tuple[str, str]]) -> str: | ||
104 | + content = "" | ||
105 | + for name in sorted(events): | ||
106 | + type_name, gocode = events[name] | ||
107 | + content += gocode | ||
108 | + | ||
109 | + return content | ||
110 | + | ||
111 | + | ||
112 | def generate_content_from_dict(data: dict[str, str]) -> str: | ||
113 | content = "" | ||
114 | |||
115 | @@ -XXX,XX +XXX,XX @@ def __init__(self, _: str): | ||
116 | types = { | ||
117 | "alternate": ["encoding/json", "errors", "fmt"], | ||
118 | "enum": [], | ||
119 | + "event": [], | ||
120 | "struct": ["encoding/json"], | ||
121 | "union": ["encoding/json", "errors", "fmt"], | ||
122 | } | ||
123 | |||
124 | self.schema: QAPISchema | ||
125 | + self.events: dict[str, Tuple[str, str]] = {} | ||
126 | self.golang_package_name = "qapi" | ||
127 | self.duplicate = list(gofiles) | ||
128 | self.enums: dict[str, str] = {} | ||
129 | @@ -XXX,XX +XXX,XX @@ def visit_end(self) -> None: | ||
130 | self.types["alternate"] += generate_content_from_dict(self.alternates) | ||
131 | self.types["struct"] += generate_content_from_dict(self.structs) | ||
132 | self.types["union"] += generate_content_from_dict(self.unions) | ||
133 | + self.types["event"] += generate_template_event(self.events) | ||
134 | |||
135 | def visit_object_type( | ||
136 | self, | ||
137 | @@ -XXX,XX +XXX,XX @@ def visit_event( | ||
138 | arg_type: Optional[QAPISchemaObjectType], | ||
139 | boxed: bool, | ||
140 | ) -> None: | ||
141 | - pass | ||
142 | + assert name == info.defn_name | ||
143 | + assert name not in self.events | ||
144 | + type_name = qapi_to_go_type_name(name, info.defn_meta) | ||
145 | + | ||
146 | + if isinstance(arg_type, QAPISchemaObjectType): | ||
147 | + content = string_to_code( | ||
148 | + qapi_to_golang_struct( | ||
149 | + self, | ||
150 | + name, | ||
151 | + info, | ||
152 | + arg_type.ifcond, | ||
153 | + arg_type.features, | ||
154 | + arg_type.base, | ||
155 | + arg_type.members, | ||
156 | + arg_type.branches, | ||
157 | + ) | ||
158 | + ) | ||
159 | + else: | ||
160 | + doc = self.docmap.get(name, None) | ||
161 | + type_doc, _ = qapi_to_golang_struct_docs(doc) | ||
162 | + content = string_to_code( | ||
163 | + generate_struct_type(type_name, type_doc=type_doc) | ||
164 | + ) | ||
165 | + | ||
166 | + self.events[name] = (type_name, content) | ||
167 | |||
168 | def write(self, outdir: str) -> None: | ||
169 | godir = "go" | ||
170 | -- | ||
171 | 2.48.1 | diff view generated by jsdifflib |
Deleted patch | |||
---|---|---|---|
1 | The Event interface is an abstraction that can be used by client and | ||
2 | server to the manager the Event types albeit with a different | ||
3 | implementation for sending and receiving. | ||
4 | 1 | ||
5 | The implementation of client/server is not part of this series. | ||
6 | |||
7 | Signed-off-by: Victor Toso <victortoso@redhat.com> | ||
8 | --- | ||
9 | scripts/qapi/golang/golang.py | 38 ++++++++++++++++++++++++++++++++--- | ||
10 | 1 file changed, 35 insertions(+), 3 deletions(-) | ||
11 | |||
12 | diff --git a/scripts/qapi/golang/golang.py b/scripts/qapi/golang/golang.py | ||
13 | index XXXXXXX..XXXXXXX 100644 | ||
14 | --- a/scripts/qapi/golang/golang.py | ||
15 | +++ b/scripts/qapi/golang/golang.py | ||
16 | @@ -XXX,XX +XXX,XX @@ | ||
17 | }} | ||
18 | """ | ||
19 | |||
20 | +TEMPLATE_EVENT = """ | ||
21 | +type Event interface {{ | ||
22 | +{methods} | ||
23 | +}} | ||
24 | +""" | ||
25 | + | ||
26 | |||
27 | # Takes the documentation object of a specific type and returns | ||
28 | # that type's documentation and its member's docs. | ||
29 | @@ -XXX,XX +XXX,XX @@ def generate_template_alternate( | ||
30 | return "\n" + content | ||
31 | |||
32 | |||
33 | -def generate_template_event(events: dict[str, Tuple[str, str]]) -> str: | ||
34 | +def generate_template_event(events: dict[str, Tuple[str, str]]) -> (str, str): | ||
35 | content = "" | ||
36 | + methods = "" | ||
37 | for name in sorted(events): | ||
38 | type_name, gocode = events[name] | ||
39 | + methods += f"\t{type_name}({type_name}, time.Time) error\n" | ||
40 | content += gocode | ||
41 | |||
42 | - return content | ||
43 | + iface = string_to_code(TEMPLATE_EVENT.format(methods=methods[:-1])) | ||
44 | + return content, iface | ||
45 | |||
46 | |||
47 | def generate_content_from_dict(data: dict[str, str]) -> str: | ||
48 | @@ -XXX,XX +XXX,XX @@ def __init__(self, _: str): | ||
49 | "struct": ["encoding/json"], | ||
50 | "union": ["encoding/json", "errors", "fmt"], | ||
51 | } | ||
52 | + interfaces = { | ||
53 | + "event": ["time"], | ||
54 | + } | ||
55 | |||
56 | self.schema: QAPISchema | ||
57 | self.events: dict[str, Tuple[str, str]] = {} | ||
58 | @@ -XXX,XX +XXX,XX @@ def __init__(self, _: str): | ||
59 | self.types = dict.fromkeys(types, "") | ||
60 | self.types_import = types | ||
61 | |||
62 | + self.interfaces = dict.fromkeys(interfaces, "") | ||
63 | + self.interface_imports = interfaces | ||
64 | + | ||
65 | def visit_begin(self, schema: QAPISchema) -> None: | ||
66 | self.schema = schema | ||
67 | |||
68 | @@ -XXX,XX +XXX,XX @@ def visit_begin(self, schema: QAPISchema) -> None: | ||
69 | continue | ||
70 | self.docmap[doc.symbol] = doc | ||
71 | |||
72 | + for qapitype, imports in self.interface_imports.items(): | ||
73 | + self.interfaces[qapitype] = TEMPLATE_GENERATED_HEADER[1:].format( | ||
74 | + package_name=self.golang_package_name | ||
75 | + ) | ||
76 | + self.interfaces[qapitype] += generate_template_imports(imports) | ||
77 | + | ||
78 | for qapitype, imports in self.types_import.items(): | ||
79 | self.types[qapitype] = TEMPLATE_GENERATED_HEADER[1:].format( | ||
80 | package_name=self.golang_package_name | ||
81 | @@ -XXX,XX +XXX,XX @@ def visit_end(self) -> None: | ||
82 | self.types["alternate"] += generate_content_from_dict(self.alternates) | ||
83 | self.types["struct"] += generate_content_from_dict(self.structs) | ||
84 | self.types["union"] += generate_content_from_dict(self.unions) | ||
85 | - self.types["event"] += generate_template_event(self.events) | ||
86 | + | ||
87 | + evtype, eviface = generate_template_event(self.events) | ||
88 | + self.types["event"] += evtype | ||
89 | + self.interfaces["event"] += eviface | ||
90 | |||
91 | def visit_object_type( | ||
92 | self, | ||
93 | @@ -XXX,XX +XXX,XX @@ def write(self, outdir: str) -> None: | ||
94 | |||
95 | with open(pathname, "w", encoding="utf8") as outfile: | ||
96 | outfile.write(content) | ||
97 | + | ||
98 | + # Interfaces to be generated | ||
99 | + for qapitype, content in self.interfaces.items(): | ||
100 | + gofile = f"gen_iface_{qapitype}.go" | ||
101 | + pathname = os.path.join(targetpath, gofile) | ||
102 | + | ||
103 | + with open(pathname, "w", encoding="utf8") as outfile: | ||
104 | + outfile.write(content) | ||
105 | -- | ||
106 | 2.48.1 | diff view generated by jsdifflib |
Deleted patch | |||
---|---|---|---|
1 | This patch handles QAPI command types and generates data structures in | ||
2 | Go that handles it. | ||
3 | 1 | ||
4 | Note that command's id is part of the first layer of unmarshal, so it | ||
5 | is a member of protocol.go's Message type. | ||
6 | |||
7 | qapi: | ||
8 | | ## | ||
9 | | # @add-fd: | ||
10 | | # | ||
11 | | # Add a file descriptor, that was passed via SCM rights, to an fd set. | ||
12 | | # | ||
13 | | # @fdset-id: The ID of the fd set to add the file descriptor to. | ||
14 | | # | ||
15 | | # @opaque: A free-form string that can be used to describe the fd. | ||
16 | | # | ||
17 | | # Returns: | ||
18 | | # @AddfdInfo | ||
19 | | # | ||
20 | | # Errors: | ||
21 | | # - If file descriptor was not received, GenericError | ||
22 | | # - If @fdset-id is a negative value, GenericError | ||
23 | | # | ||
24 | | # .. note:: The list of fd sets is shared by all monitor connections. | ||
25 | | # | ||
26 | | # .. note:: If @fdset-id is not specified, a new fd set will be | ||
27 | | # created. | ||
28 | | # | ||
29 | | # Since: 1.2 | ||
30 | | # | ||
31 | | # .. qmp-example:: | ||
32 | | # | ||
33 | | # -> { "execute": "add-fd", "arguments": { "fdset-id": 1 } } | ||
34 | | # <- { "return": { "fdset-id": 1, "fd": 3 } } | ||
35 | | ## | ||
36 | | { 'command': 'add-fd', | ||
37 | | 'data': { '*fdset-id': 'int', | ||
38 | | '*opaque': 'str' }, | ||
39 | | 'returns': 'AddfdInfo' } | ||
40 | |||
41 | go: | ||
42 | | // Add a file descriptor, that was passed via SCM rights, to an fd | ||
43 | | // set. | ||
44 | | // | ||
45 | | // Returns: @AddfdInfo | ||
46 | | // | ||
47 | | // Errors: - If file descriptor was not received, GenericError - | ||
48 | | // If @fdset-id is a negative value, GenericError | ||
49 | | // | ||
50 | | // .. note:: The list of fd sets is shared by all monitor connections. | ||
51 | | // .. note:: If @fdset-id is not specified, a new fd set will be | ||
52 | | // created. | ||
53 | | // | ||
54 | | // Since: 1.2 | ||
55 | | // | ||
56 | | // .. qmp-example:: -> { "execute": "add-fd", "arguments": { | ||
57 | | // "fdset-id": 1 } } <- { "return": { "fdset-id": 1, "fd": 3 } } | ||
58 | | type AddFdCommand struct { | ||
59 | | // The ID of the fd set to add the file descriptor to. | ||
60 | | FdsetId *int64 `json:"fdset-id,omitempty"` | ||
61 | | // A free-form string that can be used to describe the fd. | ||
62 | | Opaque *string `json:"opaque,omitempty"` | ||
63 | | } | ||
64 | |||
65 | Signed-off-by: Victor Toso <victortoso@redhat.com> | ||
66 | --- | ||
67 | scripts/qapi/golang/golang.py | 52 +++++++++++++++++++++++++++++++++-- | ||
68 | 1 file changed, 50 insertions(+), 2 deletions(-) | ||
69 | |||
70 | diff --git a/scripts/qapi/golang/golang.py b/scripts/qapi/golang/golang.py | ||
71 | index XXXXXXX..XXXXXXX 100644 | ||
72 | --- a/scripts/qapi/golang/golang.py | ||
73 | +++ b/scripts/qapi/golang/golang.py | ||
74 | @@ -XXX,XX +XXX,XX @@ def qapi_to_go_type_name(name: str, meta: Optional[str] = None) -> str: | ||
75 | name += "".join(word.title() for word in words[1:]) | ||
76 | |||
77 | # Handle specific meta suffix | ||
78 | - types = ["event"] | ||
79 | + types = ["event", "command"] | ||
80 | if meta in types: | ||
81 | name = name[:-3] if name.endswith("Arg") else name | ||
82 | name += meta.title().replace(" ", "") | ||
83 | @@ -XXX,XX +XXX,XX @@ def generate_template_alternate( | ||
84 | return "\n" + content | ||
85 | |||
86 | |||
87 | +def generate_template_command(commands: dict[str, Tuple[str, str]]) -> str: | ||
88 | + content = "" | ||
89 | + for name in sorted(commands): | ||
90 | + type_name, gocode = commands[name] | ||
91 | + content += gocode | ||
92 | + | ||
93 | + return content | ||
94 | + | ||
95 | + | ||
96 | def generate_template_event(events: dict[str, Tuple[str, str]]) -> (str, str): | ||
97 | content = "" | ||
98 | methods = "" | ||
99 | @@ -XXX,XX +XXX,XX @@ def __init__(self, _: str): | ||
100 | # Map each qapi type to the necessary Go imports | ||
101 | types = { | ||
102 | "alternate": ["encoding/json", "errors", "fmt"], | ||
103 | + "command": [], | ||
104 | "enum": [], | ||
105 | "event": [], | ||
106 | "struct": ["encoding/json"], | ||
107 | @@ -XXX,XX +XXX,XX @@ def __init__(self, _: str): | ||
108 | |||
109 | self.schema: QAPISchema | ||
110 | self.events: dict[str, Tuple[str, str]] = {} | ||
111 | + self.commands: dict[str, Tuple[str, str]] = {} | ||
112 | self.golang_package_name = "qapi" | ||
113 | self.duplicate = list(gofiles) | ||
114 | self.enums: dict[str, str] = {} | ||
115 | @@ -XXX,XX +XXX,XX @@ def visit_end(self) -> None: | ||
116 | self.types["event"] += evtype | ||
117 | self.interfaces["event"] += eviface | ||
118 | |||
119 | + self.types["command"] += generate_template_command(self.commands) | ||
120 | + | ||
121 | def visit_object_type( | ||
122 | self, | ||
123 | name: str, | ||
124 | @@ -XXX,XX +XXX,XX @@ def visit_command( | ||
125 | allow_preconfig: bool, | ||
126 | coroutine: bool, | ||
127 | ) -> None: | ||
128 | - pass | ||
129 | + assert name == info.defn_name | ||
130 | + assert name not in self.commands | ||
131 | + | ||
132 | + type_name = qapi_to_go_type_name(name, info.defn_meta) | ||
133 | + | ||
134 | + doc = self.docmap.get(name, None) | ||
135 | + type_doc, _ = qapi_to_golang_struct_docs(doc) | ||
136 | + | ||
137 | + content = "" | ||
138 | + if boxed or not arg_type or not qapi_name_is_object(arg_type.name): | ||
139 | + args: List[dict[str:str]] = [] | ||
140 | + if arg_type: | ||
141 | + args.append( | ||
142 | + { | ||
143 | + "name": f"{arg_type.name}", | ||
144 | + } | ||
145 | + ) | ||
146 | + content += string_to_code( | ||
147 | + generate_struct_type(type_name, type_doc=type_doc, args=args) | ||
148 | + ) | ||
149 | + else: | ||
150 | + assert isinstance(arg_type, QAPISchemaObjectType) | ||
151 | + content += string_to_code( | ||
152 | + qapi_to_golang_struct( | ||
153 | + self, | ||
154 | + name, | ||
155 | + arg_type.info, | ||
156 | + arg_type.ifcond, | ||
157 | + arg_type.features, | ||
158 | + arg_type.base, | ||
159 | + arg_type.members, | ||
160 | + arg_type.branches, | ||
161 | + ) | ||
162 | + ) | ||
163 | + | ||
164 | + self.commands[name] = (type_name, content) | ||
165 | |||
166 | def visit_event( | ||
167 | self, | ||
168 | -- | ||
169 | 2.48.1 | diff view generated by jsdifflib |
Deleted patch | |||
---|---|---|---|
1 | The Command interface is an abstraction that can be used by client and | ||
2 | server to the manager the Command types albeit with a different | ||
3 | implementation for sending and receiving. | ||
4 | 1 | ||
5 | The proposal for Sync is defined as Command while the Async is named | ||
6 | CommandAsync. | ||
7 | |||
8 | The implementation of client/server is not part of this series. | ||
9 | |||
10 | Signed-off-by: Victor Toso <victortoso@redhat.com> | ||
11 | --- | ||
12 | scripts/qapi/golang/golang.py | 56 +++++++++++++++++++++++++++++++---- | ||
13 | 1 file changed, 50 insertions(+), 6 deletions(-) | ||
14 | |||
15 | diff --git a/scripts/qapi/golang/golang.py b/scripts/qapi/golang/golang.py | ||
16 | index XXXXXXX..XXXXXXX 100644 | ||
17 | --- a/scripts/qapi/golang/golang.py | ||
18 | +++ b/scripts/qapi/golang/golang.py | ||
19 | @@ -XXX,XX +XXX,XX @@ | ||
20 | }} | ||
21 | """ | ||
22 | |||
23 | +TEMPLATE_COMMAND = """ | ||
24 | +// Synchronous interface of all available commands | ||
25 | +type Commands interface {{ | ||
26 | +{methods} | ||
27 | +}} | ||
28 | + | ||
29 | +{callbacks} | ||
30 | + | ||
31 | +// Asynchronous interface of all available commands | ||
32 | +type CommandsAsync interface {{ | ||
33 | +{async_methods} | ||
34 | +}} | ||
35 | +""" | ||
36 | + | ||
37 | |||
38 | # Takes the documentation object of a specific type and returns | ||
39 | # that type's documentation and its member's docs. | ||
40 | @@ -XXX,XX +XXX,XX @@ def generate_template_alternate( | ||
41 | return "\n" + content | ||
42 | |||
43 | |||
44 | -def generate_template_command(commands: dict[str, Tuple[str, str]]) -> str: | ||
45 | +def generate_template_command( | ||
46 | + commands: dict[str, Tuple[str, str, str]] | ||
47 | +) -> (str, str): | ||
48 | content = "" | ||
49 | + methods = "" | ||
50 | + async_methods = "" | ||
51 | + callbacks = "" | ||
52 | + | ||
53 | for name in sorted(commands): | ||
54 | - type_name, gocode = commands[name] | ||
55 | + type_name, gocode, retarg = commands[name] | ||
56 | content += gocode | ||
57 | |||
58 | - return content | ||
59 | + name = type_name[:-7] | ||
60 | + cbname = f"{name}Complete" | ||
61 | + syncret = "error" | ||
62 | + cbarg = "error" | ||
63 | + if retarg != "": | ||
64 | + cbarg = f"{retarg}, error" | ||
65 | + syncret = f"({retarg}, error)" | ||
66 | + methods += f"\t{name}({type_name}) {syncret}\n" | ||
67 | + async_methods += f"\t{name}({type_name}, {cbname}) error\n" | ||
68 | + callbacks += f"type {cbname} func({cbarg})\n" | ||
69 | + | ||
70 | + iface = string_to_code( | ||
71 | + TEMPLATE_COMMAND.format( | ||
72 | + methods=methods[:-1], | ||
73 | + callbacks=callbacks[:-1], | ||
74 | + async_methods=async_methods[:-1], | ||
75 | + ) | ||
76 | + ) | ||
77 | + return content, iface | ||
78 | |||
79 | |||
80 | def generate_template_event(events: dict[str, Tuple[str, str]]) -> (str, str): | ||
81 | @@ -XXX,XX +XXX,XX @@ def __init__(self, _: str): | ||
82 | "union": ["encoding/json", "errors", "fmt"], | ||
83 | } | ||
84 | interfaces = { | ||
85 | + "command": [], | ||
86 | "event": ["time"], | ||
87 | } | ||
88 | |||
89 | self.schema: QAPISchema | ||
90 | self.events: dict[str, Tuple[str, str]] = {} | ||
91 | - self.commands: dict[str, Tuple[str, str]] = {} | ||
92 | + self.commands: dict[str, Tuple[str, str, str]] = {} | ||
93 | self.golang_package_name = "qapi" | ||
94 | self.duplicate = list(gofiles) | ||
95 | self.enums: dict[str, str] = {} | ||
96 | @@ -XXX,XX +XXX,XX @@ def visit_end(self) -> None: | ||
97 | self.types["event"] += evtype | ||
98 | self.interfaces["event"] += eviface | ||
99 | |||
100 | - self.types["command"] += generate_template_command(self.commands) | ||
101 | + cmdtype, cmdiface = generate_template_command(self.commands) | ||
102 | + self.types["command"] += cmdtype | ||
103 | + self.interfaces["command"] += cmdiface | ||
104 | |||
105 | def visit_object_type( | ||
106 | self, | ||
107 | @@ -XXX,XX +XXX,XX @@ def visit_command( | ||
108 | ) | ||
109 | ) | ||
110 | |||
111 | - self.commands[name] = (type_name, content) | ||
112 | + retarg = "" | ||
113 | + if ret_type: | ||
114 | + retarg = qapi_schema_type_to_go_type(ret_type.name) | ||
115 | + self.commands[name] = (type_name, content, retarg) | ||
116 | |||
117 | def visit_event( | ||
118 | self, | ||
119 | -- | ||
120 | 2.48.1 | diff view generated by jsdifflib |