[PATCH 06/14] tracetool: Add Rust format support

Paolo Bonzini posted 14 patches 2 months, 3 weeks ago
There is a newer version of this series
[PATCH 06/14] tracetool: Add Rust format support
Posted by Paolo Bonzini 2 months, 3 weeks ago
From: Tanish Desai <tanishdesai37@gmail.com>

Generating .rs files makes it possible to support tracing in rust.
This support comprises a new format, and common code that converts
the C expressions in trace-events to Rust.  In particular, types
need to be converted, and PRI macros expanded.  Fortunately, all
common platforms have a known mapping of 8/16/32/64-bit integers
to char/short/int/"long long": even if int64_t is equal to long,
it is fine to change the format string from PRIx64's expansion
"%lx" to "%llx".  This makes it possible to have a static mapping
from PRI macros to their expansion.

As of this commit no backend generates Rust code, but it is already
possible to use tracetool to generate Rust sources; they are not
functional but they compile and contain tracepoint functions.

Signed-off-by: Tanish Desai <tanishdesai37@gmail.com>
[Move Rust argument conversion from Event to Arguments; string
 support. - Paolo]
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 scripts/tracetool/__init__.py  | 156 +++++++++++++++++++++++++++++++++
 scripts/tracetool/format/rs.py |  76 ++++++++++++++++
 2 files changed, 232 insertions(+)
 create mode 100644 scripts/tracetool/format/rs.py

diff --git a/scripts/tracetool/__init__.py b/scripts/tracetool/__init__.py
index 1d5238a0843..0b8ec707332 100644
--- a/scripts/tracetool/__init__.py
+++ b/scripts/tracetool/__init__.py
@@ -31,6 +31,49 @@ def error(*lines):
     error_write(*lines)
     sys.exit(1)
 
+FMT_TOKEN = re.compile(r'''(?:
+                       " ( (?: [^"\\] | \\[\\"abfnrt] |            # a string literal
+                               \\x[0-9a-fA-F][0-9a-fA-F]) *? ) "
+                       | ( PRI [duixX] (?:8|16|32|64|PTR|MAX) )    # a PRIxxx macro
+                       | \s+                                       # spaces (ignored)
+                       )''', re.X)
+
+PRI_SIZE_MAP = {
+    '8':  'hh',
+    '16': 'h',
+    '32': '',
+    '64': 'll',
+    'PTR': 't',
+    'MAX': 'j',
+}
+
+def expand_format_string(c_fmt, prefix=""):
+    def pri_macro_to_fmt(pri_macro):
+        assert pri_macro.startswith("PRI")
+        fmt_type = pri_macro[3]  # 'd', 'i', 'u', or 'x'
+        fmt_size = pri_macro[4:]  # '8', '16', '32', '64', 'PTR', 'MAX'
+
+        size = PRI_SIZE_MAP.get(fmt_size, None)
+        if size is None:
+            raise Exception(f"unknown macro {pri_macro}")
+        return size + fmt_type
+
+    result = prefix
+    pos = 0
+    while pos < len(c_fmt):
+        m = FMT_TOKEN.match(c_fmt, pos)
+        if not m:
+            print("No match at position", pos, ":", repr(c_fmt[pos:]), file=sys.stderr)
+            raise Exception("syntax error in trace file")
+        if m[1]:
+            substr = m[1]
+        elif m[2]:
+            substr = pri_macro_to_fmt(m[2])
+        else:
+            substr = ""
+        result += substr
+        pos = m.end()
+    return result
 
 out_lineno = 1
 out_filename = '<none>'
@@ -90,6 +133,48 @@ def out(*lines, **kwargs):
     "ptrdiff_t",
 ]
 
+C_TYPE_KEYWORDS = {"int", "short", "long", "unsigned", "char"}
+
+C_TO_RUST_TYPE_MAP = {
+    "int": "std::ffi::c_int",
+    "long": "std::ffi::c_long",
+    "long long": "std::ffi::c_longlong",
+    "short": "std::ffi::c_short",
+    "char": "std::ffi::c_char",
+    "bool": "bool",
+    "unsigned": "std::ffi::c_uint",
+    "unsigned long": "std::ffi::c_long",
+    "unsigned long long": "std::ffi::c_ulonglong",
+    "unsigned short": "std::ffi::c_ushort",
+    "unsigned char": "u8",
+    "int8_t": "i8",
+    "uint8_t": "u8",
+    "int16_t": "i16",
+    "uint16_t": "u16",
+    "int32_t": "i32",
+    "uint32_t": "u32",
+    "int64_t": "i64",
+    "uint64_t": "u64",
+    "void": "()",
+    "size_t": "usize",
+    "ssize_t": "isize",
+    "uintptr_t": "usize",
+    "ptrdiff_t": "isize",
+}
+
+# Rust requires manual casting of <32-bit types when passing them to
+# variable-argument functions.
+RUST_VARARGS_SMALL_TYPES = {
+    "std::ffi::c_short",
+    "std::ffi::c_ushort",
+    "std::ffi::c_char",
+    "i8",
+    "u8",
+    "i16",
+    "u16",
+    "bool",
+}
+
 def validate_type(name):
     bits = name.split(" ")
     for bit in bits:
@@ -105,6 +190,40 @@ def validate_type(name):
                              "other complex pointer types should be "
                              "declared as 'void *'" % name)
 
+def c_type_to_rust(name):
+    ptr = False
+    const = False
+    name = name.rstrip()
+    if name[-1] == '*':
+        name = name[:-1].rstrip()
+        ptr = True
+        if name[-1] == '*':
+            # pointers to pointers are the same as void*
+            name = "void"
+
+    bits = iter(name.split())
+    bit = next(bits)
+    if bit == "const":
+        const = True
+        bit = next(bits)
+
+    if bit in C_TYPE_KEYWORDS:
+        if bit == 'signed':
+            bit = ''
+        rest = list(bits)
+        if rest and rest[-1] == 'int':
+            rest = rest[:-1]
+        name = bit + ' ' + ' '.join(rest)
+    else:
+        if list(bits):
+            raise ValueError("Invalid type '%s'." % name)
+        name = bit
+
+    ty = C_TO_RUST_TYPE_MAP[name.strip()]
+    if ptr:
+        ty = f'*{"const" if const else "mut"} {ty}'
+    return ty
+
 class Arguments:
     """Event arguments description."""
 
@@ -197,6 +316,43 @@ def casted(self):
         """List of argument names casted to their type."""
         return ["(%s)%s" % (type_, name) for type_, name in self._args]
 
+    def rust_decl_extern(self):
+        """Return a Rust argument list for an extern "C" function"""
+        return ", ".join((f"_{name}: {c_type_to_rust(type_)}"
+                          for type_, name in self._args))
+
+    def rust_decl(self):
+        """Return a Rust argument list for a tracepoint function"""
+        def decl_type(type_):
+            if type_ == "const char *":
+                return "&std::ffi::CStr"
+            return c_type_to_rust(type_)
+
+        return ", ".join((f"_{name}: {decl_type(type_)}"
+                          for type_, name in self._args))
+
+    def rust_call_extern(self):
+        """Return a Rust argument list for a call to an extern "C" function"""
+        def rust_cast(name, type_):
+            if type_ == "const char *":
+                return f"_{name}.as_ptr()"
+            return "_{name}"
+
+        return ", ".join((rust_cast(name, type_) for type_, name in self._args))
+
+    def rust_call_varargs(self):
+        """Return a Rust argument list for a call to a C varargs function"""
+        def rust_cast(name, type_):
+            if type_ == "const char *":
+                return f"_{name}.as_ptr()"
+
+            type_ = c_type_to_rust(type_)
+            if type_ in RUST_VARARGS_SMALL_TYPES:
+                return f"_{name} as std::ffi::c_int"
+            return f"_{name} /* as {type_} */"
+
+        return ", ".join((rust_cast(name, type_) for type_, name in self._args))
+
 
 class Event(object):
     """Event description.
diff --git a/scripts/tracetool/format/rs.py b/scripts/tracetool/format/rs.py
new file mode 100644
index 00000000000..bc8b2be5971
--- /dev/null
+++ b/scripts/tracetool/format/rs.py
@@ -0,0 +1,76 @@
+# -*- coding: utf-8 -*-
+
+"""
+trace-DIR.rs
+"""
+
+__author__     = "Tanish Desai <tanishdesai37@gmail.com>"
+__copyright__  = "Copyright 2025, Tanish Desai <tanishdesai37@gmail.com>"
+__license__    = "GPL version 2 or (at your option) any later version"
+
+__maintainer__ = "Stefan Hajnoczi"
+__email__      = "stefanha@redhat.com"
+
+
+from tracetool import out
+
+
+def generate(events, backend, group):
+    out('// This file is autogenerated by tracetool, do not edit.',
+        '',
+        '#[allow(unused_imports)]',
+        'use std::ffi::c_char;',
+        '#[allow(unused_imports)]',
+        'use qemu_api::bindings;',
+        '',
+        '#[inline(always)]',
+        'fn trace_event_get_state_dynamic_by_id(_id: u16) -> bool {',
+        '    unsafe { (trace_events_enabled_count != 0) && (_id != 0) }',
+        '}',
+        '',
+        'extern "C" {',
+        '    static mut trace_events_enabled_count: u32;',
+        '}',)
+
+    out('extern "C" {')
+
+    for e in events:
+        out('    static mut %s: u16;' % e.api(e.QEMU_DSTATE))
+    out('}')
+
+    # static state
+    for e in events:
+        if 'disable' in e.properties:
+            enabled = "false"
+        else:
+            enabled = "true"
+        if "tcg-exec" in e.properties:
+            # a single define for the two "sub-events"
+            out('const _TRACE_%(name)s_ENABLED: bool = %(enabled)s;',
+                name=e.original.name.upper(),
+				enabled=enabled)
+        out('const _TRACE_%s_ENABLED: bool = %s;' % (e.name.upper(), enabled))
+
+    backend.generate_begin(events, group)
+
+    for e in events:
+        out('',
+			'#[inline(always)]',
+            '#[allow(dead_code)]',
+            'pub fn %(api)s(%(args)s)',
+            '{',
+            api=e.api(e.QEMU_TRACE),
+            args=e.args.rust_decl())
+
+        if "disable" not in e.properties:
+            backend.generate(e, group, check_trace_event_get_state=False)
+            if backend.check_trace_event_get_state:
+                event_id = 'TRACE_' + e.name.upper()
+                out('    if trace_event_get_state_dynamic_by_id(unsafe { _%(event_id)s_DSTATE}) {',
+                    event_id = event_id,
+                    api=e.api())
+                backend.generate(e, group, check_trace_event_get_state=True)
+                out('    }')
+        out('}')
+
+    backend.generate_end(events, group)
-- 
2.50.1
Re: [PATCH 06/14] tracetool: Add Rust format support
Posted by Daniel P. Berrangé 2 months, 2 weeks ago
On Fri, Aug 22, 2025 at 02:26:47PM +0200, Paolo Bonzini wrote:
> From: Tanish Desai <tanishdesai37@gmail.com>
> 
> Generating .rs files makes it possible to support tracing in rust.
> This support comprises a new format, and common code that converts
> the C expressions in trace-events to Rust.  In particular, types
> need to be converted, and PRI macros expanded.  Fortunately, all
> common platforms have a known mapping of 8/16/32/64-bit integers
> to char/short/int/"long long": even if int64_t is equal to long,
> it is fine to change the format string from PRIx64's expansion
> "%lx" to "%llx".  This makes it possible to have a static mapping
> from PRI macros to their expansion.
> 
> As of this commit no backend generates Rust code, but it is already
> possible to use tracetool to generate Rust sources; they are not
> functional but they compile and contain tracepoint functions.
> 
> Signed-off-by: Tanish Desai <tanishdesai37@gmail.com>
> [Move Rust argument conversion from Event to Arguments; string
>  support. - Paolo]
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  scripts/tracetool/__init__.py  | 156 +++++++++++++++++++++++++++++++++
>  scripts/tracetool/format/rs.py |  76 ++++++++++++++++
>  2 files changed, 232 insertions(+)
>  create mode 100644 scripts/tracetool/format/rs.py


> diff --git a/scripts/tracetool/format/rs.py b/scripts/tracetool/format/rs.py
> new file mode 100644
> index 00000000000..bc8b2be5971
> --- /dev/null
> +++ b/scripts/tracetool/format/rs.py
> @@ -0,0 +1,76 @@
> +# -*- coding: utf-8 -*-
> +
> +"""
> +trace-DIR.rs
> +"""
> +
> +__author__     = "Tanish Desai <tanishdesai37@gmail.com>"
> +__copyright__  = "Copyright 2025, Tanish Desai <tanishdesai37@gmail.com>"
> +__license__    = "GPL version 2 or (at your option) any later version"
> +
> +__maintainer__ = "Stefan Hajnoczi"
> +__email__      = "stefanha@redhat.com"
> +
> +
> +from tracetool import out
> +
> +
> +def generate(events, backend, group):
> +    out('// This file is autogenerated by tracetool, do not edit.',

Should add

        '/* SPDX-License-Identifier: GPL-2.0-or-later */',

and update the reference output in the later patch.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
Re: [PATCH 06/14] tracetool: Add Rust format support
Posted by Manos Pitsidianakis 2 months, 3 weeks ago
On Fri, Aug 22, 2025 at 3:31 PM Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> From: Tanish Desai <tanishdesai37@gmail.com>
>
> Generating .rs files makes it possible to support tracing in rust.
> This support comprises a new format, and common code that converts
> the C expressions in trace-events to Rust.  In particular, types
> need to be converted, and PRI macros expanded.  Fortunately, all
> common platforms have a known mapping of 8/16/32/64-bit integers
> to char/short/int/"long long": even if int64_t is equal to long,
> it is fine to change the format string from PRIx64's expansion
> "%lx" to "%llx".  This makes it possible to have a static mapping
> from PRI macros to their expansion.
>
> As of this commit no backend generates Rust code, but it is already
> possible to use tracetool to generate Rust sources; they are not
> functional but they compile and contain tracepoint functions.
>
> Signed-off-by: Tanish Desai <tanishdesai37@gmail.com>
> [Move Rust argument conversion from Event to Arguments; string
>  support. - Paolo]
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  scripts/tracetool/__init__.py  | 156 +++++++++++++++++++++++++++++++++
>  scripts/tracetool/format/rs.py |  76 ++++++++++++++++
>  2 files changed, 232 insertions(+)
>  create mode 100644 scripts/tracetool/format/rs.py
>
> diff --git a/scripts/tracetool/__init__.py b/scripts/tracetool/__init__.py
> index 1d5238a0843..0b8ec707332 100644
> --- a/scripts/tracetool/__init__.py
> +++ b/scripts/tracetool/__init__.py
> @@ -31,6 +31,49 @@ def error(*lines):
>      error_write(*lines)
>      sys.exit(1)
>
> +FMT_TOKEN = re.compile(r'''(?:
> +                       " ( (?: [^"\\] | \\[\\"abfnrt] |            # a string literal
> +                               \\x[0-9a-fA-F][0-9a-fA-F]) *? ) "
> +                       | ( PRI [duixX] (?:8|16|32|64|PTR|MAX) )    # a PRIxxx macro
> +                       | \s+                                       # spaces (ignored)
> +                       )''', re.X)
> +
> +PRI_SIZE_MAP = {
> +    '8':  'hh',
> +    '16': 'h',
> +    '32': '',
> +    '64': 'll',
> +    'PTR': 't',
> +    'MAX': 'j',
> +}
> +
> +def expand_format_string(c_fmt, prefix=""):
> +    def pri_macro_to_fmt(pri_macro):
> +        assert pri_macro.startswith("PRI")
> +        fmt_type = pri_macro[3]  # 'd', 'i', 'u', or 'x'
> +        fmt_size = pri_macro[4:]  # '8', '16', '32', '64', 'PTR', 'MAX'
> +
> +        size = PRI_SIZE_MAP.get(fmt_size, None)
> +        if size is None:
> +            raise Exception(f"unknown macro {pri_macro}")
> +        return size + fmt_type
> +
> +    result = prefix
> +    pos = 0
> +    while pos < len(c_fmt):
> +        m = FMT_TOKEN.match(c_fmt, pos)
> +        if not m:
> +            print("No match at position", pos, ":", repr(c_fmt[pos:]), file=sys.stderr)
> +            raise Exception("syntax error in trace file")
> +        if m[1]:
> +            substr = m[1]
> +        elif m[2]:
> +            substr = pri_macro_to_fmt(m[2])
> +        else:
> +            substr = ""
> +        result += substr
> +        pos = m.end()
> +    return result
>
>  out_lineno = 1
>  out_filename = '<none>'
> @@ -90,6 +133,48 @@ def out(*lines, **kwargs):
>      "ptrdiff_t",
>  ]
>
> +C_TYPE_KEYWORDS = {"int", "short", "long", "unsigned", "char"}
> +
> +C_TO_RUST_TYPE_MAP = {
> +    "int": "std::ffi::c_int",
> +    "long": "std::ffi::c_long",
> +    "long long": "std::ffi::c_longlong",
> +    "short": "std::ffi::c_short",
> +    "char": "std::ffi::c_char",
> +    "bool": "bool",
> +    "unsigned": "std::ffi::c_uint",
> +    "unsigned long": "std::ffi::c_long",
> +    "unsigned long long": "std::ffi::c_ulonglong",
> +    "unsigned short": "std::ffi::c_ushort",
> +    "unsigned char": "u8",
> +    "int8_t": "i8",
> +    "uint8_t": "u8",
> +    "int16_t": "i16",
> +    "uint16_t": "u16",
> +    "int32_t": "i32",
> +    "uint32_t": "u32",
> +    "int64_t": "i64",
> +    "uint64_t": "u64",
> +    "void": "()",
> +    "size_t": "usize",
> +    "ssize_t": "isize",
> +    "uintptr_t": "usize",
> +    "ptrdiff_t": "isize",
> +}
> +
> +# Rust requires manual casting of <32-bit types when passing them to
> +# variable-argument functions.
> +RUST_VARARGS_SMALL_TYPES = {
> +    "std::ffi::c_short",
> +    "std::ffi::c_ushort",
> +    "std::ffi::c_char",
> +    "i8",
> +    "u8",
> +    "i16",
> +    "u16",
> +    "bool",
> +}
> +
>  def validate_type(name):
>      bits = name.split(" ")
>      for bit in bits:
> @@ -105,6 +190,40 @@ def validate_type(name):
>                               "other complex pointer types should be "
>                               "declared as 'void *'" % name)
>
> +def c_type_to_rust(name):
> +    ptr = False
> +    const = False
> +    name = name.rstrip()
> +    if name[-1] == '*':
> +        name = name[:-1].rstrip()
> +        ptr = True
> +        if name[-1] == '*':
> +            # pointers to pointers are the same as void*
> +            name = "void"
> +
> +    bits = iter(name.split())
> +    bit = next(bits)
> +    if bit == "const":
> +        const = True
> +        bit = next(bits)
> +
> +    if bit in C_TYPE_KEYWORDS:
> +        if bit == 'signed':
> +            bit = ''
> +        rest = list(bits)
> +        if rest and rest[-1] == 'int':
> +            rest = rest[:-1]
> +        name = bit + ' ' + ' '.join(rest)
> +    else:
> +        if list(bits):
> +            raise ValueError("Invalid type '%s'." % name)
> +        name = bit
> +
> +    ty = C_TO_RUST_TYPE_MAP[name.strip()]
> +    if ptr:
> +        ty = f'*{"const" if const else "mut"} {ty}'
> +    return ty
> +
>  class Arguments:
>      """Event arguments description."""
>
> @@ -197,6 +316,43 @@ def casted(self):
>          """List of argument names casted to their type."""
>          return ["(%s)%s" % (type_, name) for type_, name in self._args]
>
> +    def rust_decl_extern(self):
> +        """Return a Rust argument list for an extern "C" function"""
> +        return ", ".join((f"_{name}: {c_type_to_rust(type_)}"
> +                          for type_, name in self._args))
> +
> +    def rust_decl(self):
> +        """Return a Rust argument list for a tracepoint function"""
> +        def decl_type(type_):
> +            if type_ == "const char *":
> +                return "&std::ffi::CStr"
> +            return c_type_to_rust(type_)
> +
> +        return ", ".join((f"_{name}: {decl_type(type_)}"
> +                          for type_, name in self._args))
> +
> +    def rust_call_extern(self):
> +        """Return a Rust argument list for a call to an extern "C" function"""
> +        def rust_cast(name, type_):
> +            if type_ == "const char *":
> +                return f"_{name}.as_ptr()"
> +            return "_{name}"

Missing f-prefix for f-string

> +
> +        return ", ".join((rust_cast(name, type_) for type_, name in self._args))
> +
> +    def rust_call_varargs(self):
> +        """Return a Rust argument list for a call to a C varargs function"""
> +        def rust_cast(name, type_):
> +            if type_ == "const char *":
> +                return f"_{name}.as_ptr()"
> +
> +            type_ = c_type_to_rust(type_)
> +            if type_ in RUST_VARARGS_SMALL_TYPES:
> +                return f"_{name} as std::ffi::c_int"
> +            return f"_{name} /* as {type_} */"
> +
> +        return ", ".join((rust_cast(name, type_) for type_, name in self._args))
> +
>
>  class Event(object):
>      """Event description.
> diff --git a/scripts/tracetool/format/rs.py b/scripts/tracetool/format/rs.py
> new file mode 100644
> index 00000000000..bc8b2be5971
> --- /dev/null
> +++ b/scripts/tracetool/format/rs.py
> @@ -0,0 +1,76 @@
> +# -*- coding: utf-8 -*-
> +
> +"""
> +trace-DIR.rs
> +"""
> +
> +__author__     = "Tanish Desai <tanishdesai37@gmail.com>"
> +__copyright__  = "Copyright 2025, Tanish Desai <tanishdesai37@gmail.com>"
> +__license__    = "GPL version 2 or (at your option) any later version"
> +
> +__maintainer__ = "Stefan Hajnoczi"
> +__email__      = "stefanha@redhat.com"
> +
> +
> +from tracetool import out
> +
> +
> +def generate(events, backend, group):
> +    out('// This file is autogenerated by tracetool, do not edit.',
> +        '',
> +        '#[allow(unused_imports)]',
> +        'use std::ffi::c_char;',
> +        '#[allow(unused_imports)]',
> +        'use qemu_api::bindings;',
> +        '',
> +        '#[inline(always)]',
> +        'fn trace_event_get_state_dynamic_by_id(_id: u16) -> bool {',
> +        '    unsafe { (trace_events_enabled_count != 0) && (_id != 0) }',
> +        '}',
> +        '',
> +        'extern "C" {',
> +        '    static mut trace_events_enabled_count: u32;',
> +        '}',)
> +
> +    out('extern "C" {')
> +
> +    for e in events:
> +        out('    static mut %s: u16;' % e.api(e.QEMU_DSTATE))
> +    out('}')
> +
> +    # static state
> +    for e in events:
> +        if 'disable' in e.properties:
> +            enabled = "false"
> +        else:
> +            enabled = "true"
> +        if "tcg-exec" in e.properties:
> +            # a single define for the two "sub-events"
> +            out('const _TRACE_%(name)s_ENABLED: bool = %(enabled)s;',
> +                name=e.original.name.upper(),
> +                               enabled=enabled)
> +        out('const _TRACE_%s_ENABLED: bool = %s;' % (e.name.upper(), enabled))
> +
> +    backend.generate_begin(events, group)
> +
> +    for e in events:
> +        out('',
> +                       '#[inline(always)]',
> +            '#[allow(dead_code)]',
> +            'pub fn %(api)s(%(args)s)',
> +            '{',
> +            api=e.api(e.QEMU_TRACE),
> +            args=e.args.rust_decl())
> +
> +        if "disable" not in e.properties:
> +            backend.generate(e, group, check_trace_event_get_state=False)
> +            if backend.check_trace_event_get_state:
> +                event_id = 'TRACE_' + e.name.upper()
> +                out('    if trace_event_get_state_dynamic_by_id(unsafe { _%(event_id)s_DSTATE}) {',
> +                    event_id = event_id,
> +                    api=e.api())
> +                backend.generate(e, group, check_trace_event_get_state=True)
> +                out('    }')
> +        out('}')
> +
> +    backend.generate_end(events, group)
> --
> 2.50.1
>
>
Re: [PATCH 06/14] tracetool: Add Rust format support
Posted by Manos Pitsidianakis 2 months, 3 weeks ago
On Fri, Aug 22, 2025 at 3:31 PM Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> From: Tanish Desai <tanishdesai37@gmail.com>
>
> Generating .rs files makes it possible to support tracing in rust.
> This support comprises a new format, and common code that converts
> the C expressions in trace-events to Rust.  In particular, types
> need to be converted, and PRI macros expanded.  Fortunately, all
> common platforms have a known mapping of 8/16/32/64-bit integers
> to char/short/int/"long long": even if int64_t is equal to long,
> it is fine to change the format string from PRIx64's expansion
> "%lx" to "%llx".  This makes it possible to have a static mapping
> from PRI macros to their expansion.
>
> As of this commit no backend generates Rust code, but it is already
> possible to use tracetool to generate Rust sources; they are not
> functional but they compile and contain tracepoint functions.
>
> Signed-off-by: Tanish Desai <tanishdesai37@gmail.com>
> [Move Rust argument conversion from Event to Arguments; string
>  support. - Paolo]
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  scripts/tracetool/__init__.py  | 156 +++++++++++++++++++++++++++++++++
>  scripts/tracetool/format/rs.py |  76 ++++++++++++++++
>  2 files changed, 232 insertions(+)
>  create mode 100644 scripts/tracetool/format/rs.py
>
> diff --git a/scripts/tracetool/__init__.py b/scripts/tracetool/__init__.py
> index 1d5238a0843..0b8ec707332 100644
> --- a/scripts/tracetool/__init__.py
> +++ b/scripts/tracetool/__init__.py
> @@ -31,6 +31,49 @@ def error(*lines):
>      error_write(*lines)
>      sys.exit(1)
>
> +FMT_TOKEN = re.compile(r'''(?:
> +                       " ( (?: [^"\\] | \\[\\"abfnrt] |            # a string literal
> +                               \\x[0-9a-fA-F][0-9a-fA-F]) *? ) "
> +                       | ( PRI [duixX] (?:8|16|32|64|PTR|MAX) )    # a PRIxxx macro
> +                       | \s+                                       # spaces (ignored)
> +                       )''', re.X)
> +
> +PRI_SIZE_MAP = {
> +    '8':  'hh',
> +    '16': 'h',
> +    '32': '',
> +    '64': 'll',
> +    'PTR': 't',
> +    'MAX': 'j',
> +}
> +
> +def expand_format_string(c_fmt, prefix=""):

(Pedantic comment) let's put type annotations in function signatures
when possible, these seem to handle `str` mostly so this should be
simple. This should catch basic errors like passing/returning `str |
None` or wrong types altogether.

> +    def pri_macro_to_fmt(pri_macro):
> +        assert pri_macro.startswith("PRI")
> +        fmt_type = pri_macro[3]  # 'd', 'i', 'u', or 'x'
> +        fmt_size = pri_macro[4:]  # '8', '16', '32', '64', 'PTR', 'MAX'
> +
> +        size = PRI_SIZE_MAP.get(fmt_size, None)
> +        if size is None:
> +            raise Exception(f"unknown macro {pri_macro}")
> +        return size + fmt_type
> +
> +    result = prefix
> +    pos = 0
> +    while pos < len(c_fmt):
> +        m = FMT_TOKEN.match(c_fmt, pos)
> +        if not m:
> +            print("No match at position", pos, ":", repr(c_fmt[pos:]), file=sys.stderr)
> +            raise Exception("syntax error in trace file")
> +        if m[1]:
> +            substr = m[1]
> +        elif m[2]:
> +            substr = pri_macro_to_fmt(m[2])
> +        else:
> +            substr = ""
> +        result += substr
> +        pos = m.end()
> +    return result
>
>  out_lineno = 1
>  out_filename = '<none>'
> @@ -90,6 +133,48 @@ def out(*lines, **kwargs):
>      "ptrdiff_t",
>  ]
>
> +C_TYPE_KEYWORDS = {"int", "short", "long", "unsigned", "char"}
> +
> +C_TO_RUST_TYPE_MAP = {
> +    "int": "std::ffi::c_int",
> +    "long": "std::ffi::c_long",
> +    "long long": "std::ffi::c_longlong",
> +    "short": "std::ffi::c_short",
> +    "char": "std::ffi::c_char",
> +    "bool": "bool",
> +    "unsigned": "std::ffi::c_uint",
> +    "unsigned long": "std::ffi::c_long",
> +    "unsigned long long": "std::ffi::c_ulonglong",
> +    "unsigned short": "std::ffi::c_ushort",
> +    "unsigned char": "u8",
> +    "int8_t": "i8",
> +    "uint8_t": "u8",
> +    "int16_t": "i16",
> +    "uint16_t": "u16",
> +    "int32_t": "i32",
> +    "uint32_t": "u32",
> +    "int64_t": "i64",
> +    "uint64_t": "u64",
> +    "void": "()",
> +    "size_t": "usize",
> +    "ssize_t": "isize",
> +    "uintptr_t": "usize",
> +    "ptrdiff_t": "isize",
> +}
> +
> +# Rust requires manual casting of <32-bit types when passing them to
> +# variable-argument functions.
> +RUST_VARARGS_SMALL_TYPES = {
> +    "std::ffi::c_short",
> +    "std::ffi::c_ushort",
> +    "std::ffi::c_char",
> +    "i8",
> +    "u8",
> +    "i16",
> +    "u16",
> +    "bool",
> +}
> +
>  def validate_type(name):
>      bits = name.split(" ")
>      for bit in bits:
> @@ -105,6 +190,40 @@ def validate_type(name):
>                               "other complex pointer types should be "
>                               "declared as 'void *'" % name)
>
> +def c_type_to_rust(name):
> +    ptr = False
> +    const = False
> +    name = name.rstrip()
> +    if name[-1] == '*':
> +        name = name[:-1].rstrip()
> +        ptr = True
> +        if name[-1] == '*':
> +            # pointers to pointers are the same as void*
> +            name = "void"
> +
> +    bits = iter(name.split())
> +    bit = next(bits)
> +    if bit == "const":
> +        const = True
> +        bit = next(bits)
> +
> +    if bit in C_TYPE_KEYWORDS:
> +        if bit == 'signed':
> +            bit = ''
> +        rest = list(bits)
> +        if rest and rest[-1] == 'int':
> +            rest = rest[:-1]
> +        name = bit + ' ' + ' '.join(rest)
> +    else:
> +        if list(bits):
> +            raise ValueError("Invalid type '%s'." % name)
> +        name = bit
> +
> +    ty = C_TO_RUST_TYPE_MAP[name.strip()]
> +    if ptr:
> +        ty = f'*{"const" if const else "mut"} {ty}'
> +    return ty
> +
>  class Arguments:
>      """Event arguments description."""
>
> @@ -197,6 +316,43 @@ def casted(self):
>          """List of argument names casted to their type."""
>          return ["(%s)%s" % (type_, name) for type_, name in self._args]
>
> +    def rust_decl_extern(self):
> +        """Return a Rust argument list for an extern "C" function"""
> +        return ", ".join((f"_{name}: {c_type_to_rust(type_)}"
> +                          for type_, name in self._args))
> +
> +    def rust_decl(self):
> +        """Return a Rust argument list for a tracepoint function"""
> +        def decl_type(type_):
> +            if type_ == "const char *":
> +                return "&std::ffi::CStr"
> +            return c_type_to_rust(type_)
> +
> +        return ", ".join((f"_{name}: {decl_type(type_)}"
> +                          for type_, name in self._args))
> +
> +    def rust_call_extern(self):
> +        """Return a Rust argument list for a call to an extern "C" function"""
> +        def rust_cast(name, type_):
> +            if type_ == "const char *":
> +                return f"_{name}.as_ptr()"
> +            return "_{name}"
> +
> +        return ", ".join((rust_cast(name, type_) for type_, name in self._args))
> +
> +    def rust_call_varargs(self):
> +        """Return a Rust argument list for a call to a C varargs function"""
> +        def rust_cast(name, type_):
> +            if type_ == "const char *":
> +                return f"_{name}.as_ptr()"
> +
> +            type_ = c_type_to_rust(type_)
> +            if type_ in RUST_VARARGS_SMALL_TYPES:
> +                return f"_{name} as std::ffi::c_int"
> +            return f"_{name} /* as {type_} */"
> +
> +        return ", ".join((rust_cast(name, type_) for type_, name in self._args))
> +
>
>  class Event(object):
>      """Event description.
> diff --git a/scripts/tracetool/format/rs.py b/scripts/tracetool/format/rs.py
> new file mode 100644
> index 00000000000..bc8b2be5971
> --- /dev/null
> +++ b/scripts/tracetool/format/rs.py
> @@ -0,0 +1,76 @@
> +# -*- coding: utf-8 -*-

This was probably copied verbatim from other tracetool files, but IIUC
it's not needed, utf-8 is the default encoding if there's no `coding`
declaration.

Also, missing SPDX header as a new file

> +
> +"""
> +trace-DIR.rs

(what's "trace-DIR"?)

> +"""
> +
> +__author__     = "Tanish Desai <tanishdesai37@gmail.com>"
> +__copyright__  = "Copyright 2025, Tanish Desai <tanishdesai37@gmail.com>"
> +__license__    = "GPL version 2 or (at your option) any later version"
> +
> +__maintainer__ = "Stefan Hajnoczi"
> +__email__      = "stefanha@redhat.com"
> +
> +
> +from tracetool import out
> +
> +
> +def generate(events, backend, group):
> +    out('// This file is autogenerated by tracetool, do not edit.',

Let's use `@generated comments` https://generated.at/

> +        '',
> +        '#[allow(unused_imports)]',
> +        'use std::ffi::c_char;',
> +        '#[allow(unused_imports)]',
> +        'use qemu_api::bindings;',
> +        '',
> +        '#[inline(always)]',
> +        'fn trace_event_get_state_dynamic_by_id(_id: u16) -> bool {',
> +        '    unsafe { (trace_events_enabled_count != 0) && (_id != 0) }',
> +        '}',
> +        '',
> +        'extern "C" {',
> +        '    static mut trace_events_enabled_count: u32;',
> +        '}',)
> +
> +    out('extern "C" {')
> +
> +    for e in events:
> +        out('    static mut %s: u16;' % e.api(e.QEMU_DSTATE))
> +    out('}')
> +
> +    # static state
> +    for e in events:
> +        if 'disable' in e.properties:
> +            enabled = "false"
> +        else:
> +            enabled = "true"
> +        if "tcg-exec" in e.properties:
> +            # a single define for the two "sub-events"
> +            out('const _TRACE_%(name)s_ENABLED: bool = %(enabled)s;',
> +                name=e.original.name.upper(),

What's the difference between e.original.name and e.name?

> +                               enabled=enabled)
> +        out('const _TRACE_%s_ENABLED: bool = %s;' % (e.name.upper(), enabled))
> +
> +    backend.generate_begin(events, group)
> +
> +    for e in events:
> +        out('',
> +                       '#[inline(always)]',
> +            '#[allow(dead_code)]',
> +            'pub fn %(api)s(%(args)s)',
> +            '{',
> +            api=e.api(e.QEMU_TRACE),
> +            args=e.args.rust_decl())
> +
> +        if "disable" not in e.properties:
> +            backend.generate(e, group, check_trace_event_get_state=False)
> +            if backend.check_trace_event_get_state:
> +                event_id = 'TRACE_' + e.name.upper()
> +                out('    if trace_event_get_state_dynamic_by_id(unsafe { _%(event_id)s_DSTATE}) {',
> +                    event_id = event_id,
> +                    api=e.api())
> +                backend.generate(e, group, check_trace_event_get_state=True)
> +                out('    }')
> +        out('}')
> +
> +    backend.generate_end(events, group)
> --
> 2.50.1
>
>

-- 
Manos Pitsidianakis
Emulation and Virtualization Engineer at Linaro Ltd
Re: [PATCH 06/14] tracetool: Add Rust format support
Posted by Paolo Bonzini 2 months, 3 weeks ago
On 8/25/25 09:03, Manos Pitsidianakis wrote:
>> +def expand_format_string(c_fmt, prefix=""):
> 
> (Pedantic comment) let's put type annotations in function signatures
> when possible, these seem to handle `str` mostly so this should be
> simple. This should catch basic errors like passing/returning `str |
> None` or wrong types altogether.

Not pedantic, I have a full conversion of tracetool to add mypy 
annotations... but it's on top of these patches. :)

I can reorder them and post the conversion first, depending on what the 
maintainers prefer.

>> @@ -0,0 +1,76 @@
>> +# -*- coding: utf-8 -*-
> 
> This was probably copied verbatim from other tracetool files, but IIUC
> it's not needed, utf-8 is the default encoding if there's no `coding`
> declaration.
> 
> Also, missing SPDX header as a new file

Ok I can do that for all files, too.

>> +        if "tcg-exec" in e.properties:
>> +            # a single define for the two "sub-events"
>> +            out('const _TRACE_%(name)s_ENABLED: bool = %(enabled)s;',
>> +                name=e.original.name.upper(),
> 
> What's the difference between e.original.name and e.name?

Good point---anything to do with e.original is dead code.  I'll add a 
patch in front to drop it.

Paolo