[PATCH 00/19] rust: QObject and QAPI bindings

Paolo Bonzini posted 19 patches 1 month ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20251010151006.791038-1-pbonzini@redhat.com
Maintainers: Paolo Bonzini <pbonzini@redhat.com>, "Marc-André Lureau" <marcandre.lureau@redhat.com>, "Daniel P. Berrangé" <berrange@redhat.com>, "Philippe Mathieu-Daudé" <philmd@linaro.org>, Markus Armbruster <armbru@redhat.com>, Michael Roth <michael.roth@amd.com>, Manos Pitsidianakis <manos.pitsidianakis@linaro.org>, "Alex Bennée" <alex.bennee@linaro.org>, Thomas Huth <thuth@redhat.com>
docs/devel/rust.rst                           |   1 +
meson.build                                   |   4 +-
rust/util/wrapper.h                           |   8 +
qapi/meson.build                              |   6 +
rust/Cargo.lock                               |   2 +
rust/Cargo.toml                               |   2 +
rust/chardev/meson.build                      |   2 +-
rust/hw/timer/hpet/src/device.rs              |  21 +-
rust/hw/timer/hpet/src/fw_cfg.rs              |   7 +-
rust/meson.build                              |   4 +
rust/tests/meson.build                        |  25 +-
rust/tests/tests/integration.rs               |   2 +
rust/tests/tests/qapi.rs                      | 444 +++++++++++++
rust/util/Cargo.toml                          |   2 +
rust/util/meson.build                         |  31 +-
rust/util/src/error.rs                        | 222 +++----
rust/util/src/lib.rs                          |   2 +
rust/util/src/qobject/deserialize.rs          | 134 ++++
rust/util/src/qobject/deserializer.rs         | 371 +++++++++++
rust/util/src/qobject/error.rs                |  58 ++
rust/util/src/qobject/mod.rs                  | 369 +++++++++++
rust/util/src/qobject/serialize.rs            |  59 ++
rust/util/src/qobject/serializer.rs           | 585 ++++++++++++++++++
scripts/archive-source.sh                     |   3 +
scripts/make-release                          |   2 +-
scripts/qapi/backend.py                       |  27 +-
scripts/qapi/common.py                        |  16 +
scripts/qapi/gen.py                           |   6 +-
scripts/qapi/main.py                          |   4 +-
scripts/qapi/rs.py                            | 181 ++++++
scripts/qapi/rs_types.py                      | 387 ++++++++++++
scripts/qapi/schema.py                        |   4 +
scripts/rust/rustc_args.py                    |  16 +-
subprojects/.gitignore                        |   3 +
.../packagefiles/serde-1-rs/meson.build       |  36 ++
.../packagefiles/serde-1.0.226-include.patch  |  16 +
.../packagefiles/serde_core-1-rs/meson.build  |  25 +
.../serde_core-1.0.226-include.patch          |  15 +
.../serde_derive-1-rs/meson.build             |  35 ++
.../serde_derive-1.0.226-include.patch        |  11 +
subprojects/serde-1-rs.wrap                   |  11 +
subprojects/serde_core-1-rs.wrap              |  11 +
subprojects/serde_derive-1-rs.wrap            |  11 +
43 files changed, 3042 insertions(+), 139 deletions(-)
create mode 100644 rust/tests/tests/integration.rs
create mode 100644 rust/tests/tests/qapi.rs
create mode 100644 rust/util/src/qobject/deserialize.rs
create mode 100644 rust/util/src/qobject/deserializer.rs
create mode 100644 rust/util/src/qobject/error.rs
create mode 100644 rust/util/src/qobject/mod.rs
create mode 100644 rust/util/src/qobject/serialize.rs
create mode 100644 rust/util/src/qobject/serializer.rs
create mode 100644 scripts/qapi/rs.py
create mode 100644 scripts/qapi/rs_types.py
create mode 100644 subprojects/packagefiles/serde-1-rs/meson.build
create mode 100644 subprojects/packagefiles/serde-1.0.226-include.patch
create mode 100644 subprojects/packagefiles/serde_core-1-rs/meson.build
create mode 100644 subprojects/packagefiles/serde_core-1.0.226-include.patch
create mode 100644 subprojects/packagefiles/serde_derive-1-rs/meson.build
create mode 100644 subprojects/packagefiles/serde_derive-1.0.226-include.patch
create mode 100644 subprojects/serde-1-rs.wrap
create mode 100644 subprojects/serde_core-1-rs.wrap
create mode 100644 subprojects/serde_derive-1-rs.wrap
[PATCH 00/19] rust: QObject and QAPI bindings
Posted by Paolo Bonzini 1 month ago
This adds two related parts of the Rust bindings:

- QAPI code generator that creates Rust structs from the JSON
  description.  The structs are *not* ABI compatible with the
  C ones, instead they use native Rust data types.

- QObject bindings and (de)serialization support, which can be used to
  convert QObjects to and from QAPI structs.

Unfortunately Rust code is not able to use visitors, other than by
creating an intermediate QObject.  This is because of the different
architecture of serde vs. QAPI visitors, and because visitor's
dual-purpose functions, where the same function is used by both input and
output visitors, rely heavily on the structs using the same representation
as the visitor arguments (for example NUL-terminated strings).

The serde format implementation was co-authored by me and Marc-André.
Marc-André did all the bug fixing and integration testing.

As an example of how this would be used, the marshaling functions for
QMP commands would look like this:

fn qmp_marshal_query_stats(args: *mut QDict,
    retp: *mut *mut QObject, errp: *mut *mut Error)
{
    let qobj = unsafe { QObject::cloned_from_raw(args.cast()) };

    let result = from_qobject::<StatsFilter>(qobj)
         .map_err(anyhow::Error::from)
         .and_then(qmp_query_stats)
         .and_then(|ret| to_qobject::<Vec<StatsResult>>(ret).map_err(anyhow::Error::from));

    match qmp_marshal_query_stats_rs(qobj) {
        Ok(ret) => unsafe { *retp = ret.into_raw(); },
        Err(e) => unsafe { crate::Error::from(e).propagate(errp) },
    }
}

As a small extra, patches 1 and 2 rework a bit the error implementation
so that it is possible to convert any Rust error into a QEMU one.  This
is because we noticed that we had to add several From<> implementations
to convert e.g. NulError or serde errors into util::Error.

The price is that it's a bit harder to convert *strings* into errors;
therefore, the first patch adds a macro wrapper for
"if !cond { return Err(...) }", where the dots build an error from a
formatted string.

Paolo

Marc-André Lureau (8):
  rust/qobject: add Display/Debug
  scripts/qapi: add QAPISchemaIfCond.rsgen()
  scripts/qapi: generate high-level Rust bindings
  scripts/qapi: add serde attributes
  scripts/qapi: strip trailing whitespaces
  scripts/rustc_args: add --no-strict-cfg
  rust/util: build QAPI types
  rust/tests: QAPI integration tests

Paolo Bonzini (11):
  util: add ensure macro
  rust/util: use anyhow's native chaining capabilities
  rust: do not add qemuutil to Rust crates
  rust/qobject: add basic bindings
  subprojects: add serde
  rust/qobject: add Serialize implementation
  rust/qobject: add Serializer (to_qobject) implementation
  rust/qobject: add Deserialize implementation
  rust/qobject: add Deserializer (from_qobject) implementation
  rust/util: replace Error::err_or_unit/err_or_else with
    Error::with_errp
  rust/qobject: add from/to JSON bindings for QObject

 docs/devel/rust.rst                           |   1 +
 meson.build                                   |   4 +-
 rust/util/wrapper.h                           |   8 +
 qapi/meson.build                              |   6 +
 rust/Cargo.lock                               |   2 +
 rust/Cargo.toml                               |   2 +
 rust/chardev/meson.build                      |   2 +-
 rust/hw/timer/hpet/src/device.rs              |  21 +-
 rust/hw/timer/hpet/src/fw_cfg.rs              |   7 +-
 rust/meson.build                              |   4 +
 rust/tests/meson.build                        |  25 +-
 rust/tests/tests/integration.rs               |   2 +
 rust/tests/tests/qapi.rs                      | 444 +++++++++++++
 rust/util/Cargo.toml                          |   2 +
 rust/util/meson.build                         |  31 +-
 rust/util/src/error.rs                        | 222 +++----
 rust/util/src/lib.rs                          |   2 +
 rust/util/src/qobject/deserialize.rs          | 134 ++++
 rust/util/src/qobject/deserializer.rs         | 371 +++++++++++
 rust/util/src/qobject/error.rs                |  58 ++
 rust/util/src/qobject/mod.rs                  | 369 +++++++++++
 rust/util/src/qobject/serialize.rs            |  59 ++
 rust/util/src/qobject/serializer.rs           | 585 ++++++++++++++++++
 scripts/archive-source.sh                     |   3 +
 scripts/make-release                          |   2 +-
 scripts/qapi/backend.py                       |  27 +-
 scripts/qapi/common.py                        |  16 +
 scripts/qapi/gen.py                           |   6 +-
 scripts/qapi/main.py                          |   4 +-
 scripts/qapi/rs.py                            | 181 ++++++
 scripts/qapi/rs_types.py                      | 387 ++++++++++++
 scripts/qapi/schema.py                        |   4 +
 scripts/rust/rustc_args.py                    |  16 +-
 subprojects/.gitignore                        |   3 +
 .../packagefiles/serde-1-rs/meson.build       |  36 ++
 .../packagefiles/serde-1.0.226-include.patch  |  16 +
 .../packagefiles/serde_core-1-rs/meson.build  |  25 +
 .../serde_core-1.0.226-include.patch          |  15 +
 .../serde_derive-1-rs/meson.build             |  35 ++
 .../serde_derive-1.0.226-include.patch        |  11 +
 subprojects/serde-1-rs.wrap                   |  11 +
 subprojects/serde_core-1-rs.wrap              |  11 +
 subprojects/serde_derive-1-rs.wrap            |  11 +
 43 files changed, 3042 insertions(+), 139 deletions(-)
 create mode 100644 rust/tests/tests/integration.rs
 create mode 100644 rust/tests/tests/qapi.rs
 create mode 100644 rust/util/src/qobject/deserialize.rs
 create mode 100644 rust/util/src/qobject/deserializer.rs
 create mode 100644 rust/util/src/qobject/error.rs
 create mode 100644 rust/util/src/qobject/mod.rs
 create mode 100644 rust/util/src/qobject/serialize.rs
 create mode 100644 rust/util/src/qobject/serializer.rs
 create mode 100644 scripts/qapi/rs.py
 create mode 100644 scripts/qapi/rs_types.py
 create mode 100644 subprojects/packagefiles/serde-1-rs/meson.build
 create mode 100644 subprojects/packagefiles/serde-1.0.226-include.patch
 create mode 100644 subprojects/packagefiles/serde_core-1-rs/meson.build
 create mode 100644 subprojects/packagefiles/serde_core-1.0.226-include.patch
 create mode 100644 subprojects/packagefiles/serde_derive-1-rs/meson.build
 create mode 100644 subprojects/packagefiles/serde_derive-1.0.226-include.patch
 create mode 100644 subprojects/serde-1-rs.wrap
 create mode 100644 subprojects/serde_core-1-rs.wrap
 create mode 100644 subprojects/serde_derive-1-rs.wrap

-- 
2.51.0


Re: [PATCH 00/19] rust: QObject and QAPI bindings
Posted by Paolo Bonzini 2 weeks ago
On 10/10/25 17:09, Paolo Bonzini wrote:
> This adds two related parts of the Rust bindings:
> 
> - QAPI code generator that creates Rust structs from the JSON
>    description.  The structs are *not* ABI compatible with the
>    C ones, instead they use native Rust data types.
> 
> - QObject bindings and (de)serialization support, which can be used to
>    convert QObjects to and from QAPI structs.
> 
> Unfortunately Rust code is not able to use visitors, other than by
> creating an intermediate QObject.  This is because of the different
> architecture of serde vs. QAPI visitors, and because visitor's
> dual-purpose functions, where the same function is used by both input and
> output visitors, rely heavily on the structs using the same representation
> as the visitor arguments (for example NUL-terminated strings).
> 
> The serde format implementation was co-authored by me and Marc-André.
> Marc-André did all the bug fixing and integration testing.
> 
> As an example of how this would be used, the marshaling functions for
> QMP commands would look like this:
> 
> fn qmp_marshal_query_stats(args: *mut QDict,
>      retp: *mut *mut QObject, errp: *mut *mut Error)
> {
>      let qobj = unsafe { QObject::cloned_from_raw(args.cast()) };
> 
>      let result = from_qobject::<StatsFilter>(qobj)
>           .map_err(anyhow::Error::from)
>           .and_then(qmp_query_stats)
>           .and_then(|ret| to_qobject::<Vec<StatsResult>>(ret).map_err(anyhow::Error::from));
> 
>      match qmp_marshal_query_stats_rs(qobj) {
>          Ok(ret) => unsafe { *retp = ret.into_raw(); },
>          Err(e) => unsafe { crate::Error::from(e).propagate(errp) },
>      }
> }
> 
> As a small extra, patches 1 and 2 rework a bit the error implementation
> so that it is possible to convert any Rust error into a QEMU one.  This
> is because we noticed that we had to add several From<> implementations
> to convert e.g. NulError or serde errors into util::Error.
> 
> The price is that it's a bit harder to convert *strings* into errors;
> therefore, the first patch adds a macro wrapper for
> "if !cond { return Err(...) }", where the dots build an error from a
> formatted string.
Ping.

Paolo