[PoCv2 00/15] Rust binding for QAPI (qemu-ga only, for now)

marcandre.lureau@redhat.com posted 15 patches 3 years, 6 months ago
Test checkpatch failed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20201011203513.1621355-1-marcandre.lureau@redhat.com
.cargo/config                |   5 +
.gitmodules                  |   3 +
.travis.yml                  |  18 +-
Cargo.toml                   |   5 +
configure                    |  26 ++
include/qemu/osdep.h         |  10 -
meson.build                  |  29 ++-
migration/dirtyrate.c        |   3 +-
qga/Cargo.toml               |  20 ++
qga/commands-posix.c         | 159 -------------
qga/commands-win32.c         |  76 ------
qga/commands.c               |  34 +--
qga/lib.rs                   |   5 +
qga/meson.build              |  30 ++-
qga/qapi.rs                  |   6 +
qga/qapi_sys.rs              |   5 +
qga/qmp/hostname.rs          |   9 +
qga/qmp/mod.rs               |  61 +++++
qga/qmp/vcpus.rs             | 161 +++++++++++++
rust/common/Cargo.toml       |  11 +
rust/common/src/error.rs     | 109 +++++++++
rust/common/src/lib.rs       |  10 +
rust/common/src/qemu.rs      |  30 +++
rust/common/src/sys.rs       |  58 +++++
rust/common/src/translate.rs | 309 ++++++++++++++++++++++++
rust/vendored                |   1 +
scripts/cargo_wrapper.py     | 102 ++++++++
scripts/qapi-gen.py          |  18 +-
scripts/qapi/rs.py           | 204 ++++++++++++++++
scripts/qapi/rs_sys.py       | 254 ++++++++++++++++++++
scripts/qapi/rs_types.py     | 447 +++++++++++++++++++++++++++++++++++
scripts/qapi/schema.py       |  14 +-
tests/test-bitmap.c          |   1 -
tests/test-qga.c             |   4 +
util/oslib-posix.c           |  35 ---
util/oslib-win32.c           |  13 -
36 files changed, 1962 insertions(+), 323 deletions(-)
create mode 100644 .cargo/config
create mode 100644 Cargo.toml
create mode 100644 qga/Cargo.toml
create mode 100644 qga/lib.rs
create mode 100644 qga/qapi.rs
create mode 100644 qga/qapi_sys.rs
create mode 100644 qga/qmp/hostname.rs
create mode 100644 qga/qmp/mod.rs
create mode 100644 qga/qmp/vcpus.rs
create mode 100644 rust/common/Cargo.toml
create mode 100644 rust/common/src/error.rs
create mode 100644 rust/common/src/lib.rs
create mode 100644 rust/common/src/qemu.rs
create mode 100644 rust/common/src/sys.rs
create mode 100644 rust/common/src/translate.rs
create mode 160000 rust/vendored
create mode 100644 scripts/cargo_wrapper.py
create mode 100644 scripts/qapi/rs.py
create mode 100644 scripts/qapi/rs_sys.py
create mode 100644 scripts/qapi/rs_types.py
[PoCv2 00/15] Rust binding for QAPI (qemu-ga only, for now)
Posted by marcandre.lureau@redhat.com 3 years, 6 months ago
From: Marc-André Lureau <marcandre.lureau@redhat.com>

Hi,

Among the QEMU developers, there is a desire to use Rust. (see previous
thread from Stefan "Why QEMU should move from C to Rust", the rust-vmm
related projects and other experiments).

Thanks to our QAPI type system and the associate code generator, it is
relatively straightforward to create Rust bindings for the generated C
types (also called sys/ffi binding) and functions. (rust-bindgen could
probably do a similar job, but it would probably bring other issues).
This provides an important internal API already.

Slightly more complicated is to expose a Rust API for those, and provide
convenient conversions C<->Rust. Taking inspiration from glib-rs
binding, I implemented a simplified version of the FromGlib/ToGlib
traits, with simpler ownership model, sufficient for QAPI needs.

The usage is relatively simple:

- from_qemu_none(ptr: *const sys::P) -> T
  Return a Rust type T for a const ffi pointer P.

- from_qemu_full(ptr: *mut sys::P) -> T
  Return a Rust type T for a ffi pointer P, taking ownership.

- T::to_qemu_none() -> Stash<P>
  Returns a borrowed ffi pointer P (using a Stash to destroy "glue"
  storage data, if any).

- T::to_qemu_full() -> P
  Returns a ffi pointer P. (P resources are leaked/passed to C/ffi)

With those traits, it's relatively easy to implement the QMP callbacks. With
enough interest, we could eventually add new commands in Rust, and start
rewriting QGA in Rust, as it is a simple service. See qga/qmp/ for some
examples. QEMU would be the next obvious target.

My biggest pain-point right now is the handling of 'if' conditions. I tried
different approaches, but none of them are satisfying. I am planning to tackle
this next again, along with full QEMU API schema support and hopefully some t=
est
integration.

v2:
 - split the original patch in smaller patches and more digestable form
 - dropped the DBus interface experiment from this series
 - various build-sys improvements, new configure options --with-rust(-target)
 - various attempts at better meson integration, finally satisfied enough wit=
h the
   current solution, which handles getting link flags from Rust sanely.
   (more meson stuff to come to handle config-host/features mapping etc).
 - rebased, QGA QMP now uses unions, added support for it.
 - start a common crate (which re-surfaced issues with foreign types and trai=
ts,
   worked around with a NewPtr wrapper)
 - explicit errors when ifcond are presents (after various unsucessful attemp=
ts,
   I will try to tackle it in v3)
 - mingw cross compilation support
 - some attempts to add it to CI
 - actually implement {get,set}-vcpus
 - vendor the external crates

Marc-Andr=C3=A9 Lureau (15):
  mingw: fix error __USE_MINGW_ANSI_STDIO redefined
  scripts/qapi: teach c_param_type() to return const argument type
  build-sys: add --with-rust{-target} & basic build infrastructure
  build-sys: add a cargo-wrapper script
  qga/rust: build and link an empty static library
  rust: provide a common crate for QEMU
  scripts/qapi: add Rust sys bindings generation
  qga/rust: generate QGA QAPI sys bindings
  scripts/qapi: add generation of Rust bindings for types
  qga/rust: build Rust types
  qga: add qmp! macro helper
  qga: implement get-host-name in Rust
  qga: implement {get,set}-vcpus in Rust
  travis: add Rust
  rust: use vendored-sources

 .cargo/config                |   5 +
 .gitmodules                  |   3 +
 .travis.yml                  |  18 +-
 Cargo.toml                   |   5 +
 configure                    |  26 ++
 include/qemu/osdep.h         |  10 -
 meson.build                  |  29 ++-
 migration/dirtyrate.c        |   3 +-
 qga/Cargo.toml               |  20 ++
 qga/commands-posix.c         | 159 -------------
 qga/commands-win32.c         |  76 ------
 qga/commands.c               |  34 +--
 qga/lib.rs                   |   5 +
 qga/meson.build              |  30 ++-
 qga/qapi.rs                  |   6 +
 qga/qapi_sys.rs              |   5 +
 qga/qmp/hostname.rs          |   9 +
 qga/qmp/mod.rs               |  61 +++++
 qga/qmp/vcpus.rs             | 161 +++++++++++++
 rust/common/Cargo.toml       |  11 +
 rust/common/src/error.rs     | 109 +++++++++
 rust/common/src/lib.rs       |  10 +
 rust/common/src/qemu.rs      |  30 +++
 rust/common/src/sys.rs       |  58 +++++
 rust/common/src/translate.rs | 309 ++++++++++++++++++++++++
 rust/vendored                |   1 +
 scripts/cargo_wrapper.py     | 102 ++++++++
 scripts/qapi-gen.py          |  18 +-
 scripts/qapi/rs.py           | 204 ++++++++++++++++
 scripts/qapi/rs_sys.py       | 254 ++++++++++++++++++++
 scripts/qapi/rs_types.py     | 447 +++++++++++++++++++++++++++++++++++
 scripts/qapi/schema.py       |  14 +-
 tests/test-bitmap.c          |   1 -
 tests/test-qga.c             |   4 +
 util/oslib-posix.c           |  35 ---
 util/oslib-win32.c           |  13 -
 36 files changed, 1962 insertions(+), 323 deletions(-)
 create mode 100644 .cargo/config
 create mode 100644 Cargo.toml
 create mode 100644 qga/Cargo.toml
 create mode 100644 qga/lib.rs
 create mode 100644 qga/qapi.rs
 create mode 100644 qga/qapi_sys.rs
 create mode 100644 qga/qmp/hostname.rs
 create mode 100644 qga/qmp/mod.rs
 create mode 100644 qga/qmp/vcpus.rs
 create mode 100644 rust/common/Cargo.toml
 create mode 100644 rust/common/src/error.rs
 create mode 100644 rust/common/src/lib.rs
 create mode 100644 rust/common/src/qemu.rs
 create mode 100644 rust/common/src/sys.rs
 create mode 100644 rust/common/src/translate.rs
 create mode 160000 rust/vendored
 create mode 100644 scripts/cargo_wrapper.py
 create mode 100644 scripts/qapi/rs.py
 create mode 100644 scripts/qapi/rs_sys.py
 create mode 100644 scripts/qapi/rs_types.py

--=20
2.28.0



Re: [PoCv2 00/15] Rust binding for QAPI (qemu-ga only, for now)
Posted by no-reply@patchew.org 3 years, 6 months ago
Patchew URL: https://patchew.org/QEMU/20201011203513.1621355-1-marcandre.lureau@redhat.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Message-id: 20201011203513.1621355-1-marcandre.lureau@redhat.com
Subject: [PoCv2 00/15] Rust binding for QAPI (qemu-ga only, for now)

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 * [new tag]         patchew/20201011203513.1621355-1-marcandre.lureau@redhat.com -> patchew/20201011203513.1621355-1-marcandre.lureau@redhat.com
Switched to a new branch 'test'
e5abacf rust: use vendored-sources
f8125dc travis: add Rust
0bb6095b qga: implement {get,set}-vcpus in Rust
6f1cb07 qga: implement get-host-name in Rust
6cc3f63 qga: add qmp! macro helper
abc816c qga/rust: build Rust types
8273b34 scripts/qapi: add generation of Rust bindings for types
3068424 qga/rust: generate QGA QAPI sys bindings
8d23431 scripts/qapi: add Rust sys bindings generation
eb4bb2d rust: provide a common crate for QEMU
f19d443 qga/rust: build and link an empty static library
03097d9 build-sys: add a cargo-wrapper script
d8ec366 build-sys: add --with-rust{-target} & basic build infrastructure
e524eda scripts/qapi: teach c_param_type() to return const argument type
e68868d mingw: fix error __USE_MINGW_ANSI_STDIO redefined

=== OUTPUT BEGIN ===
1/15 Checking commit e68868d9d68b (mingw: fix error __USE_MINGW_ANSI_STDIO redefined)
2/15 Checking commit e524eda108f7 (scripts/qapi: teach c_param_type() to return const argument type)
WARNING: line over 80 characters
#27: FILE: scripts/qapi/schema.py:171:
+    # The argument should be considered const, since no ownership is given to the callee,

WARNING: line over 80 characters
#28: FILE: scripts/qapi/schema.py:172:
+    # but qemu C code frequently tweaks it. Set const=True for a stricter declaration.

total: 0 errors, 2 warnings, 28 lines checked

Patch 2/15 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
3/15 Checking commit d8ec36655d29 (build-sys: add --with-rust{-target} & basic build infrastructure)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#29: 
new file mode 100644

total: 0 errors, 1 warnings, 85 lines checked

Patch 3/15 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
4/15 Checking commit 03097d99222b (build-sys: add a cargo-wrapper script)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#39: 
new file mode 100644

WARNING: line over 80 characters
#82: FILE: scripts/cargo_wrapper.py:39:
+        target_dir, args.target_triple, args.build_type, "lib" + package_name + ".a"

WARNING: line over 80 characters
#119: FILE: scripts/cargo_wrapper.py:76:
+            "Environment: " + " ".join(["{}={}".format(k, v) for k, v in env.items()])

total: 0 errors, 3 warnings, 108 lines checked

Patch 4/15 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
5/15 Checking commit f19d443c4992 (qga/rust: build and link an empty static library)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#29: 
new file mode 100644

total: 0 errors, 1 warnings, 38 lines checked

Patch 5/15 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
6/15 Checking commit eb4bb2d04ac5 (rust: provide a common crate for QEMU)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#32: 
new file mode 100644

total: 0 errors, 1 warnings, 533 lines checked

Patch 6/15 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
7/15 Checking commit 8d23431862a8 (scripts/qapi: add Rust sys bindings generation)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#79: 
new file mode 100644

WARNING: line over 80 characters
#182: FILE: scripts/qapi/rs.py:99:
+    value = ''.join(word.title() for word in filter(None, re.split("[-_]+", value)))

ERROR: line over 90 characters
#270: FILE: scripts/qapi/rs_sys.py:55:
+                     rs_systype=rs_systype(memb.type.c_type(), ''), rs_name=rs_name(memb.name))

WARNING: line over 80 characters
#332: FILE: scripts/qapi/rs_sys.py:117:
+                ret += gen_rs_sys_object(v.type.name, v.type.ifcond, v.type.base,

WARNING: line over 80 characters
#429: FILE: scripts/qapi/rs_sys.py:214:
+                 rs_name=rs_name(name), rs_systype=rs_systype(element_type.c_type(), ''))

total: 1 errors, 4 warnings, 422 lines checked

Patch 7/15 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

8/15 Checking commit 306842468f88 (qga/rust: generate QGA QAPI sys bindings)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#61: 
new file mode 100644

total: 0 errors, 1 warnings, 37 lines checked

Patch 8/15 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
9/15 Checking commit 8273b3417e30 (scripts/qapi: add generation of Rust bindings for types)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#179: 
new file mode 100644

WARNING: line over 80 characters
#227: FILE: scripts/qapi/rs_types.py:44:
+''', var_name=var_name, rs_type=rs_type(type_name, ''), rs_systype=rs_systype(type_name))

ERROR: line over 90 characters
#231: FILE: scripts/qapi/rs_types.py:48:
+                 (stash_.0 as *mut std::ffi::c_void, %(rs_name)sCEnum::%(var_name)s(stash_.1))

WARNING: line over 80 characters
#252: FILE: scripts/qapi/rs_types.py:69:
+    fn to_qemu_none(&'a self) -> Stash<'a, *mut std::ffi::c_void, %(rs_name)sEnum> {

ERROR: line over 90 characters
#368: FILE: scripts/qapi/rs_types.py:185:
+            arms += mcgen('%(rs_name)sEnum::%(kind_name)s(_) => qapi_sys::%(rs_name)sUnion { %(var_name)s: %(var_type)s { data: u_ptr_ as *mut _ } },',

ERROR: line over 90 characters
#369: FILE: scripts/qapi/rs_types.py:186:
+                          rs_name=rs_name(name), kind_name=to_camel_case(var.name), var_name=rs_name(var.name), var_type=rs_systype(var.type.c_name()))

WARNING: line over 80 characters
#395: FILE: scripts/qapi/rs_types.py:212:
+    fn to_qemu_none(&'a self) -> Stash<'a, *mut qapi_sys::%(rs_name)s, %(rs_name)s> {

WARNING: line over 80 characters
#406: FILE: scripts/qapi/rs_types.py:223:
+            let ptr = sys::g_malloc0(std::mem::size_of::<*const %(rs_name)s>()) as *mut _;

ERROR: line over 90 characters
#414: FILE: scripts/qapi/rs_types.py:231:
+                 sys_memb=', '.join(sys_memb), memb_none=memb_none, memb_full=memb_full, stash=', '.join(stash))

ERROR: line over 90 characters
#459: FILE: scripts/qapi/rs_types.py:276:
+            %(enum)s::%(variant)s => { %(rs_name)sEnum::%(variant)s(from_qemu_none(sys.u.%(memb)s.data as *const _)) },

ERROR: line over 90 characters
#487: FILE: scripts/qapi/rs_types.py:304:
+                     rs_type=rs_type(memb.type.c_type(), '', optional=memb.optional), rs_name=rsname)

total: 6 errors, 5 warnings, 570 lines checked

Patch 9/15 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

10/15 Checking commit abc816c08984 (qga/rust: build Rust types)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#30: 
new file mode 100644

total: 0 errors, 1 warnings, 15 lines checked

Patch 10/15 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
11/15 Checking commit 6cc3f639e6bc (qga: add qmp! macro helper)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#22: 
new file mode 100644

total: 0 errors, 1 warnings, 39 lines checked

Patch 11/15 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
12/15 Checking commit 6f1cb07dcab6 (qga: implement get-host-name in Rust)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#92: 
new file mode 100644

total: 0 errors, 1 warnings, 150 lines checked

Patch 12/15 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
13/15 Checking commit 0bb6095b4294 (qga: implement {get,set}-vcpus in Rust)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#339: 
new file mode 100644

total: 0 errors, 1 warnings, 472 lines checked

Patch 13/15 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
14/15 Checking commit f8125dcffce4 (travis: add Rust)
15/15 Checking commit e5abacf1af78 (rust: use vendored-sources)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#17: 
new file mode 100644

total: 0 errors, 1 warnings, 33 lines checked

Patch 15/15 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20201011203513.1621355-1-marcandre.lureau@redhat.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com