From nobody Wed Apr 1 22:06:11 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1771956982; cv=none; d=zohomail.com; s=zohoarc; b=fGcQAFgLsNDooLnqQXKvEAHyiKvOJ9Zp7gbs6e/gfFgy7j+lo+N0p9w2Qhuja0Ji2fuBjjW7fat2xxXR6hVib77otuFduIUXHPfSR9JiF0v5Vj15QC2zsK7ebWnknxvaIsV2WhKaBBGMf3Vql9LnWd13QIp5tUwpuVRBoBTpeTk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1771956982; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=A8Jn0yMcBwEzyMs58lSknWJcgq5eOazfgHuIFhiIiiE=; b=cHXVcMKFP7zAW857K3ctWHqmpB2E9Kbf/u+26aq1LapG+fvmDQwW8bSNBl47fg4/z6R3xZAmsxl8NG3ACBhncys2chg/kv97fE9LCtuWe0iuJpWZsx7a13b1QxkpBKH17edizemv47d5eITYlXmOZQ8MiLHGGzaBZkIvf8Z3B1Q= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1771956982462981.5362429241156; Tue, 24 Feb 2026 10:16:22 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1vuwwn-0003Bo-VY; Tue, 24 Feb 2026 13:15:53 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vuwwa-00039E-SU for qemu-devel@nongnu.org; Tue, 24 Feb 2026 13:15:37 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1vuwwQ-0002JC-Jr for qemu-devel@nongnu.org; Tue, 24 Feb 2026 13:15:29 -0500 Received: from mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-433-S7Z_zR7dNjG0VuMPNkAdjA-1; Tue, 24 Feb 2026 13:15:14 -0500 Received: from mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B5DA91800365; Tue, 24 Feb 2026 18:15:11 +0000 (UTC) Received: from jsnow-thinkpadp16vgen1.westford.csb (unknown [10.22.89.212]) by mx-prod-int-06.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 04E141800666; Tue, 24 Feb 2026 18:15:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1771956924; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=A8Jn0yMcBwEzyMs58lSknWJcgq5eOazfgHuIFhiIiiE=; b=bsVBF3c+hzvUUpW9bOZstw2J5LR0f7wzm0oD/Z8oT6Dcfwel78vhCS/9QY1B/oTR+8vlUQ vBPgeLqEuM5goaFSBF9d0VAhWvo+/JpvjINTJeIjgR/me2l3phBEza8qdKgMWV7m8fnNi+ btrkzkrQYKgkfPxMF6qjc++CdhhwL+c= X-MC-Unique: S7Z_zR7dNjG0VuMPNkAdjA-1 X-Mimecast-MFC-AGG-ID: S7Z_zR7dNjG0VuMPNkAdjA_1771956912 From: John Snow To: qemu-devel@nongnu.org Cc: Vladimir Sementsov-Ogievskiy , Kyle Evans , Thomas Huth , =?UTF-8?q?Marc-Andr=C3=A9=20Lureau?= , John Snow , Warner Losh , Markus Armbruster , Michael Roth , Fabiano Rosas , =?UTF-8?q?Alex=20Benn=C3=A9e?= , Maksim Davydov , Peter Maydell , =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= , Peter Xu , Cleber Rosa , Ed Maste , qemu-block@nongnu.org, Li-Wen Hsu , Hanna Reitz , Mauro Carvalho Chehab , Kevin Wolf , Paolo Bonzini , Yonggang Luo , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Subject: [PULL v2 04/19] python/mkvenv: add mechanism to install local package(s) Date: Tue, 24 Feb 2026 13:14:23 -0500 Message-ID: <20260224181440.832943-5-jsnow@redhat.com> In-Reply-To: <20260224181440.832943-1-jsnow@redhat.com> References: <20260224181440.832943-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.93 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -10 X-Spam_score: -1.1 X-Spam_bar: - X-Spam_report: (-1.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.358, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.659, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1771956990068158500 Content-Type: text/plain; charset="utf-8" Currently, we "implicitly" install the local 'qemu' python package for 'make check-venv' with some logic inside tests/Makefile.include. I would like to make this installation explicit in pythondeps.toml instead. This patch adds a path constraint that can be used in lieu of version constraints to specify that a package should be installed from the source tree instead of from PyPI or vendored packages. This is done to allow us to install the python packages hosted inside of the tree while also processing dependencies; i.e. so that our "qemu" package can specify that it needs "qemu.qmp", which soon will not be included in qemu.git. This also has the benefit of being able to specify in a declarative configuration file that our pyvenv environment *will* have our local python packages installed and available without any PYTHONPATH hacks, which should simplify iotests, device-crash-test and functional tests without needing to manage local inclusion paths in environment variables. On the downsides, installing packages through mkvenv/ensuregroup means that there are extra steps we need to take in order to install a local package *offline*; namely we must disable build isolation (so we have access to setuptools) and we must also include python3-wheel in QEMU's build dependencies in order for "make check" to run successfully when in an offline, isolated environment. These extra dependencies are handled in a forthcoming commit; for now, nothing is utilizing this new pathway. Reviewed-by: Thomas Huth Message-ID: <20260218213416.674483-6-jsnow@redhat.com> Signed-off-by: John Snow --- python/scripts/mkvenv.py | 37 +++++++++++++++++++++++++++++++------ 1 file changed, 31 insertions(+), 6 deletions(-) diff --git a/python/scripts/mkvenv.py b/python/scripts/mkvenv.py index b4662d33e6e..8ed6a354505 100644 --- a/python/scripts/mkvenv.py +++ b/python/scripts/mkvenv.py @@ -662,6 +662,7 @@ def pip_install( args: Sequence[str], online: bool =3D False, wheels_dir: Optional[Union[str, Path]] =3D None, + env: Optional[Dict[str, str]] =3D None, ) -> None: """ Use pip to install a package or package(s) as specified in @args. @@ -687,6 +688,7 @@ def pip_install( full_args +=3D list(args) subprocess.run( full_args, + env=3Denv, check=3DTrue, ) =20 @@ -733,9 +735,14 @@ def _do_ensure( :param wheels_dir: If specified, search this path for packages. """ absent =3D [] + local_packages =3D [] present =3D [] canary =3D None for name, info in group.items(): + if "path" in info: + pkgpath =3D Path(__file__).parents[2].joinpath(info["path"]) + local_packages.append(str(pkgpath)) + continue constraint =3D _make_version_constraint(info, False) matcher =3D Matcher(name + constraint) print(f"mkvenv: checking for {matcher}", file=3Dsys.stderr) @@ -770,15 +777,33 @@ def _do_ensure( print(f"mkvenv: installing {', '.join(absent)}", file=3Dsys.st= derr) try: pip_install(args=3Dabsent, online=3Donline, wheels_dir=3Dw= heels_dir) - return None + absent =3D [] except subprocess.CalledProcessError: pass =20 - return diagnose( - absent[0], - online, - wheels_dir, - canary, + if absent: + return diagnose( + absent[0], + online, + wheels_dir, + canary, + ) + + # Handle local packages separately and last so we can use different + # installation arguments (-e), and so that any dependencies that may + # be covered above will be handled according to the depfile + # specifications. + if local_packages: + print(f"mkvenv: installing {', '.join(local_packages)}", + file=3Dsys.stderr) + env =3D dict(os.environ) + env['PIP_CONFIG_SETTINGS'] =3D "editable_mode=3Dcompat" + pip_install( + args=3D["--no-build-isolation", + "-e"] + local_packages, + online=3Donline, + wheels_dir=3Dwheels_dir, + env=3Denv, ) =20 return None --=20 2.53.0