Some distros prefer to avoid vendored crate sources, and instead use
local sources from e.g. ``/usr/share/cargo/registry``.  Add a
script, inspired by the Mesa spec file(*), that automatically
performs this task.  The script is meant to be invoked after unpacking
the QEMU tarball.

(*) This is the hack that Mesa uses:

    export MESON_PACKAGE_CACHE_DIR="%{cargo_registry}/"
    %define inst_crate_nameversion() %(basename %{cargo_registry}/%{1}-*)
    %define rewrite_wrap_file() sed -e "/source.*/d" -e "s/%{1}-.*/%{inst_crate_nameversion %{1}}/" -i subprojects/%{1}.wrap
    %rewrite_wrap_file proc-macro2
    ... more %rewrite_wrap_file invocations follow ...

Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
---
 docs/about/build-platforms.rst           |   8 +
 scripts/get-wraps-from-cargo-registry.py | 191 +++++++++++++++++++++++
 2 files changed, 199 insertions(+)
 create mode 100755 scripts/get-wraps-from-cargo-registry.py

diff --git a/docs/about/build-platforms.rst b/docs/about/build-platforms.rst
index 8ecbd6b26f7..8671c3be9cd 100644
--- a/docs/about/build-platforms.rst
+++ b/docs/about/build-platforms.rst
@@ -127,6 +127,14 @@ Rust build dependencies
   (or newer) package.  The path to ``rustc`` and ``rustdoc`` must be
   provided manually to the configure script.
 
+  Some distros prefer to avoid vendored crate sources, and instead use
+  local sources from e.g. ``/usr/share/cargo/registry``.  QEMU includes a
+  script, ``scripts/get-wraps-from-cargo-registry.py``, that automatically
+  performs this task.  The script is meant to be invoked after unpacking
+  the QEMU tarball.  QEMU also includes ``rust/Cargo.toml`` and
+  ``rust/Cargo.lock`` files that can be used to compute QEMU's build
+  dependencies, e.g. using ``cargo2rpm -p rust/Cargo.toml buildrequires``.
+
 Optional build dependencies
   Build components whose absence does not affect the ability to build QEMU
   may not be available in distros, or may be too old for our requirements.
diff --git a/scripts/get-wraps-from-cargo-registry.py b/scripts/get-wraps-from-cargo-registry.py
new file mode 100755
index 00000000000..6b76d00a6d9
--- /dev/null
+++ b/scripts/get-wraps-from-cargo-registry.py
@@ -0,0 +1,191 @@
+#!/usr/bin/env python3
+
+"""
+get-wraps-from-cargo-registry.py - Update Meson subprojects from a global registry
+"""
+
+# Copyright (C) 2025 Red Hat, Inc.
+#
+# Author: Paolo Bonzini <pbonzini@redhat.com>
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import argparse
+import configparser
+import filecmp
+import glob
+import os
+import subprocess
+import sys
+
+
+def get_name_and_semver(namever: str) -> tuple[str, str]:
+    """Split a subproject name into its name and semantic version parts"""
+    parts = namever.rsplit("-", 1)
+    if len(parts) != 2:
+        return namever, ""
+
+    return parts[0], parts[1]
+
+
+class UpdateSubprojects:
+    cargo_registry: str
+    top_srcdir: str
+    dry_run: bool
+    changes: int = 0
+
+    def find_installed_crate(self, namever: str) -> str | None:
+        """Find installed crate matching name and semver prefix"""
+        name, semver = get_name_and_semver(namever)
+
+        # exact version match
+        path = os.path.join(self.cargo_registry, f"{name}-{semver}")
+        if os.path.exists(path):
+            return f"{name}-{semver}"
+
+        # semver match
+        matches = sorted(glob.glob(f"{path}.*"))
+        return os.path.basename(matches[0]) if matches else None
+
+    def compare_build_rs(self, orig_dir: str, registry_namever: str) -> None:
+        """Warn if the build.rs in the original directory differs from the registry version."""
+        orig_build_rs = os.path.join(orig_dir, "build.rs")
+        new_build_rs = os.path.join(self.cargo_registry, registry_namever, "build.rs")
+
+        msg = None
+        if os.path.isfile(orig_build_rs) != os.path.isfile(new_build_rs):
+            if os.path.isfile(orig_build_rs):
+                msg = f"build.rs removed in {registry_namever}"
+            if os.path.isfile(new_build_rs):
+                msg = f"build.rs added in {registry_namever}"
+
+        elif os.path.isfile(orig_build_rs) and not filecmp.cmp(orig_build_rs, new_build_rs):
+            msg = f"build.rs changed from {orig_dir} to {registry_namever}"
+
+        if msg:
+            print(f"⚠️  Warning: {msg}")
+            print("   This may affect the build process - please review the differences.")
+
+    def update_subproject(self, wrap_file: str, registry_namever: str) -> None:
+        """Modify [wrap-file] section to point to self.cargo_registry."""
+        assert wrap_file.endswith("-rs.wrap")
+        wrap_name = wrap_file[:-5]
+
+        env = os.environ.copy()
+        env["MESON_PACKAGE_CACHE_DIR"] = self.cargo_registry
+
+        config = configparser.ConfigParser()
+        config.read(wrap_file)
+        if "wrap-file" not in config:
+            return
+
+        # do not download the wrap, always use the local copy
+        orig_dir = config["wrap-file"]["directory"]
+        if os.path.exists(orig_dir) and orig_dir != registry_namever:
+            self.compare_build_rs(orig_dir, registry_namever)
+        if self.dry_run:
+            if orig_dir == registry_namever:
+                print(f"Will install {orig_dir} from registry.")
+            else:
+                print(f"Will replace {orig_dir} with {registry_namever}.")
+            self.changes += 1
+            return
+
+        config["wrap-file"]["directory"] = registry_namever
+        for key in list(config["wrap-file"].keys()):
+            if key.startswith("source"):
+                del config["wrap-file"][key]
+
+        # replace existing directory with installed version
+        if os.path.exists(orig_dir):
+            subprocess.run(
+                ["meson", "subprojects", "purge", "--confirm", wrap_name],
+                cwd=self.top_srcdir,
+                env=env,
+                check=True,
+            )
+
+        with open(wrap_file, "w") as f:
+            config.write(f)
+
+        if orig_dir == registry_namever:
+            print(f"Installing {orig_dir} from registry.")
+        else:
+            print(f"Replacing {orig_dir} with {registry_namever}.")
+
+        if orig_dir != registry_namever:
+            patch_dir = config["wrap-file"]["patch_directory"]
+            patch_dir = os.path.join("packagefiles", patch_dir)
+            _, ver = registry_namever.rsplit("-", 1)
+            subprocess.run(
+                ["meson", "rewrite", "kwargs", "set", "project", "/", "version", ver],
+                cwd=patch_dir,
+                env=env,
+                check=True,
+            )
+
+        subprocess.run(
+            ["meson", "subprojects", "download", wrap_name],
+            cwd=self.top_srcdir,
+            env=env,
+            check=True,
+        )
+
+    @staticmethod
+    def parse_cmdline() -> argparse.Namespace:
+        parser = argparse.ArgumentParser(
+            description="Replace Meson subprojects with packages in a Cargo registry"
+        )
+        parser.add_argument(
+            "--cargo-registry",
+            default=os.environ.get("CARGO_REGISTRY"),
+            help="Path to Cargo registry (default: CARGO_REGISTRY env var)",
+        )
+        parser.add_argument(
+            "--dry-run",
+            action="store_true",
+            default=False,
+            help="Do not actually replace anything",
+        )
+
+        args = parser.parse_args()
+        if not args.cargo_registry:
+            print("error: CARGO_REGISTRY environment variable not set and --cargo-registry not provided")
+            sys.exit(1)
+
+        return args
+
+    def __init__(self, args: argparse.Namespace):
+        self.cargo_registry = args.cargo_registry
+        self.dry_run = args.dry_run
+        self.top_srcdir = os.getcwd()
+
+    def main(self) -> None:
+        if not os.path.exists("subprojects"):
+            print("'subprojects' directory not found, nothing to do.")
+            return
+
+        os.chdir("subprojects")
+        for wrap_file in sorted(glob.glob("*-rs.wrap")):
+            namever = wrap_file[:-8]  # Remove '-rs.wrap'
+
+            registry_namever = self.find_installed_crate(namever)
+            if not registry_namever:
+                print(f"No installed crate found for {wrap_file}")
+                continue
+
+            self.update_subproject(wrap_file, registry_namever)
+
+        if self.changes:
+            if self.dry_run:
+                print("Rerun without --dry-run to apply changes.")
+            else:
+                print(f"✨ {self.changes} subproject(s) updated!")
+        else:
+            print("No changes.")
+
+
+if __name__ == "__main__":
+    args = UpdateSubprojects.parse_cmdline()
+    UpdateSubprojects(args).main()
-- 
2.50.1

On Tue, Jul 22, 2025 at 11:36 AM Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> Some distros prefer to avoid vendored crate sources, and instead use
> local sources from e.g. ``/usr/share/cargo/registry``.  Add a
> script, inspired by the Mesa spec file(*), that automatically
> performs this task.  The script is meant to be invoked after unpacking
> the QEMU tarball.
>
> (*) This is the hack that Mesa uses:
>
>     export MESON_PACKAGE_CACHE_DIR="%{cargo_registry}/"
>     %define inst_crate_nameversion() %(basename %{cargo_registry}/%{1}-*)
>     %define rewrite_wrap_file() sed -e "/source.*/d" -e "s/%{1}-.*/%{inst_crate_nameversion %{1}}/" -i subprojects/%{1}.wrap
>     %rewrite_wrap_file proc-macro2
>     ... more %rewrite_wrap_file invocations follow ...
>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  docs/about/build-platforms.rst           |   8 +
>  scripts/get-wraps-from-cargo-registry.py | 191 +++++++++++++++++++++++
>  2 files changed, 199 insertions(+)
>  create mode 100755 scripts/get-wraps-from-cargo-registry.py
>
> diff --git a/docs/about/build-platforms.rst b/docs/about/build-platforms.rst
> index 8ecbd6b26f7..8671c3be9cd 100644
> --- a/docs/about/build-platforms.rst
> +++ b/docs/about/build-platforms.rst
> @@ -127,6 +127,14 @@ Rust build dependencies
>    (or newer) package.  The path to ``rustc`` and ``rustdoc`` must be
>    provided manually to the configure script.
>
> +  Some distros prefer to avoid vendored crate sources, and instead use
> +  local sources from e.g. ``/usr/share/cargo/registry``.  QEMU includes a
> +  script, ``scripts/get-wraps-from-cargo-registry.py``, that automatically
> +  performs this task.  The script is meant to be invoked after unpacking
> +  the QEMU tarball.  QEMU also includes ``rust/Cargo.toml`` and
> +  ``rust/Cargo.lock`` files that can be used to compute QEMU's build
> +  dependencies, e.g. using ``cargo2rpm -p rust/Cargo.toml buildrequires``.
> +
>  Optional build dependencies
>    Build components whose absence does not affect the ability to build QEMU
>    may not be available in distros, or may be too old for our requirements.
> diff --git a/scripts/get-wraps-from-cargo-registry.py b/scripts/get-wraps-from-cargo-registry.py
> new file mode 100755
> index 00000000000..6b76d00a6d9
> --- /dev/null
> +++ b/scripts/get-wraps-from-cargo-registry.py
> @@ -0,0 +1,191 @@
> +#!/usr/bin/env python3
> +
> +"""
> +get-wraps-from-cargo-registry.py - Update Meson subprojects from a global registry
> +"""
> +
> +# Copyright (C) 2025 Red Hat, Inc.
> +#
> +# Author: Paolo Bonzini <pbonzini@redhat.com>
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2 or
> +# later. See the COPYING file in the top-level directory.
> +

Nit, missing:

# SPDX-License-Identifier:

> +import argparse
> +import configparser
> +import filecmp
> +import glob
> +import os
> +import subprocess
> +import sys
> +
> +
> +def get_name_and_semver(namever: str) -> tuple[str, str]:
> +    """Split a subproject name into its name and semantic version parts"""
> +    parts = namever.rsplit("-", 1)
> +    if len(parts) != 2:
> +        return namever, ""
> +
> +    return parts[0], parts[1]
> +
> +
> +class UpdateSubprojects:
> +    cargo_registry: str
> +    top_srcdir: str
> +    dry_run: bool
> +    changes: int = 0
> +
> +    def find_installed_crate(self, namever: str) -> str | None:
> +        """Find installed crate matching name and semver prefix"""
> +        name, semver = get_name_and_semver(namever)
> +
> +        # exact version match
> +        path = os.path.join(self.cargo_registry, f"{name}-{semver}")
> +        if os.path.exists(path):
> +            return f"{name}-{semver}"
> +
> +        # semver match
> +        matches = sorted(glob.glob(f"{path}.*"))
> +        return os.path.basename(matches[0]) if matches else None
> +
> +    def compare_build_rs(self, orig_dir: str, registry_namever: str) -> None:
> +        """Warn if the build.rs in the original directory differs from the registry version."""
> +        orig_build_rs = os.path.join(orig_dir, "build.rs")
> +        new_build_rs = os.path.join(self.cargo_registry, registry_namever, "build.rs")
> +
> +        msg = None
> +        if os.path.isfile(orig_build_rs) != os.path.isfile(new_build_rs):
> +            if os.path.isfile(orig_build_rs):
> +                msg = f"build.rs removed in {registry_namever}"
> +            if os.path.isfile(new_build_rs):
> +                msg = f"build.rs added in {registry_namever}"
> +
> +        elif os.path.isfile(orig_build_rs) and not filecmp.cmp(orig_build_rs, new_build_rs):
> +            msg = f"build.rs changed from {orig_dir} to {registry_namever}"
> +
> +        if msg:
> +            print(f"⚠️  Warning: {msg}")
> +            print("   This may affect the build process - please review the differences.")
> +
> +    def update_subproject(self, wrap_file: str, registry_namever: str) -> None:
> +        """Modify [wrap-file] section to point to self.cargo_registry."""
> +        assert wrap_file.endswith("-rs.wrap")
> +        wrap_name = wrap_file[:-5]
> +
> +        env = os.environ.copy()
> +        env["MESON_PACKAGE_CACHE_DIR"] = self.cargo_registry
> +
> +        config = configparser.ConfigParser()
> +        config.read(wrap_file)
> +        if "wrap-file" not in config:
> +            return
> +
> +        # do not download the wrap, always use the local copy
> +        orig_dir = config["wrap-file"]["directory"]
> +        if os.path.exists(orig_dir) and orig_dir != registry_namever:
> +            self.compare_build_rs(orig_dir, registry_namever)
> +        if self.dry_run:
> +            if orig_dir == registry_namever:
> +                print(f"Will install {orig_dir} from registry.")
> +            else:
> +                print(f"Will replace {orig_dir} with {registry_namever}.")
> +            self.changes += 1
> +            return
> +
> +        config["wrap-file"]["directory"] = registry_namever
> +        for key in list(config["wrap-file"].keys()):
> +            if key.startswith("source"):
> +                del config["wrap-file"][key]
> +
> +        # replace existing directory with installed version
> +        if os.path.exists(orig_dir):
> +            subprocess.run(
> +                ["meson", "subprojects", "purge", "--confirm", wrap_name],
> +                cwd=self.top_srcdir,
> +                env=env,
> +                check=True,
> +            )
> +
> +        with open(wrap_file, "w") as f:
> +            config.write(f)
> +
> +        if orig_dir == registry_namever:
> +            print(f"Installing {orig_dir} from registry.")
> +        else:
> +            print(f"Replacing {orig_dir} with {registry_namever}.")
> +
> +        if orig_dir != registry_namever:
> +            patch_dir = config["wrap-file"]["patch_directory"]
> +            patch_dir = os.path.join("packagefiles", patch_dir)
> +            _, ver = registry_namever.rsplit("-", 1)
> +            subprocess.run(
> +                ["meson", "rewrite", "kwargs", "set", "project", "/", "version", ver],
> +                cwd=patch_dir,
> +                env=env,
> +                check=True,
> +            )
> +
> +        subprocess.run(
> +            ["meson", "subprojects", "download", wrap_name],
> +            cwd=self.top_srcdir,
> +            env=env,
> +            check=True,
> +        )
> +
> +    @staticmethod
> +    def parse_cmdline() -> argparse.Namespace:
> +        parser = argparse.ArgumentParser(
> +            description="Replace Meson subprojects with packages in a Cargo registry"
> +        )
> +        parser.add_argument(
> +            "--cargo-registry",
> +            default=os.environ.get("CARGO_REGISTRY"),
> +            help="Path to Cargo registry (default: CARGO_REGISTRY env var)",
> +        )
> +        parser.add_argument(
> +            "--dry-run",
> +            action="store_true",
> +            default=False,
> +            help="Do not actually replace anything",
> +        )
> +
> +        args = parser.parse_args()
> +        if not args.cargo_registry:
> +            print("error: CARGO_REGISTRY environment variable not set and --cargo-registry not provided")
> +            sys.exit(1)
> +
> +        return args
> +
> +    def __init__(self, args: argparse.Namespace):
> +        self.cargo_registry = args.cargo_registry
> +        self.dry_run = args.dry_run
> +        self.top_srcdir = os.getcwd()
> +
> +    def main(self) -> None:
> +        if not os.path.exists("subprojects"):
> +            print("'subprojects' directory not found, nothing to do.")
> +            return
> +
> +        os.chdir("subprojects")
> +        for wrap_file in sorted(glob.glob("*-rs.wrap")):
> +            namever = wrap_file[:-8]  # Remove '-rs.wrap'
> +
> +            registry_namever = self.find_installed_crate(namever)
> +            if not registry_namever:
> +                print(f"No installed crate found for {wrap_file}")
> +                continue
> +
> +            self.update_subproject(wrap_file, registry_namever)
> +
> +        if self.changes:
> +            if self.dry_run:
> +                print("Rerun without --dry-run to apply changes.")
> +            else:
> +                print(f"✨ {self.changes} subproject(s) updated!")
> +        else:
> +            print("No changes.")
> +
> +
> +if __name__ == "__main__":
> +    args = UpdateSubprojects.parse_cmdline()
> +    UpdateSubprojects(args).main()
> --
> 2.50.1
>
>

On Tue, Jul 22, 2025 at 4:35 AM Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> Some distros prefer to avoid vendored crate sources, and instead use
> local sources from e.g. ``/usr/share/cargo/registry``.  Add a
> script, inspired by the Mesa spec file(*), that automatically
> performs this task.  The script is meant to be invoked after unpacking
> the QEMU tarball.
>
> (*) This is the hack that Mesa uses:
>
>     export MESON_PACKAGE_CACHE_DIR="%{cargo_registry}/"
>     %define inst_crate_nameversion() %(basename %{cargo_registry}/%{1}-*)
>     %define rewrite_wrap_file() sed -e "/source.*/d" -e "s/%{1}-.*/%{inst_crate_nameversion %{1}}/" -i subprojects/%{1}.wrap
>     %rewrite_wrap_file proc-macro2
>     ... more %rewrite_wrap_file invocations follow ...
>
> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
> ---
>  docs/about/build-platforms.rst           |   8 +
>  scripts/get-wraps-from-cargo-registry.py | 191 +++++++++++++++++++++++
>  2 files changed, 199 insertions(+)
>  create mode 100755 scripts/get-wraps-from-cargo-registry.py
>

I am impressed how much code my three lines in the Mesa spec file produced.

The code looks good to me and seems to do what it says on the tin.

Reviewed-by: Neal Gompa <ngompa@fedoraproject.org>



-- 
Neal Gompa (FAS: ngompa)