From nobody Mon Feb 9 03:47:26 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1654900103; cv=none; d=zohomail.com; s=zohoarc; b=SHABLW2dwewMHxM1/wKBJxgYCKX9j3t9+aRo8vkHeje0nxID0glW07ahDSDd3UP1FSsOLdAufrhncSj6gi4BlYc3QVikm6ZkpB+9kgoIUw+kPIas39bZguiVRKIs+V2hUnWY8K/Q0yUOQAzkvWgbVk44t2V8O64dnKviYj5KKco= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1654900103; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Hpl9Hcmja7wCqA/6pHrgfp7z/GTa2UgFHuwdaUlLYz0=; b=kibCbj1xixpoOKr7bMfY/xyWQ3dZoM9jIi8rnBCfRtUDoJeiyI6OAM4Ul5TK45Kyh7zspt63XzPzs+QLcFcElQw3mbdCDLcluA2X/Z5WVgs0V/OMURsfLVzD1t98jWk8mzX4k2Hm+ovVLClvRAueZOKDbxZOTV0etclrbUzi8GA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1654900103339253.06402272967182; Fri, 10 Jun 2022 15:28:23 -0700 (PDT) Received: from localhost ([::1]:33200 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nzn78-0003VU-BA for importer@patchew.org; Fri, 10 Jun 2022 18:28:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47060) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzn56-0000hv-2C for qemu-devel@nongnu.org; Fri, 10 Jun 2022 18:26:16 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:52631) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzn52-0005UC-8K for qemu-devel@nongnu.org; Fri, 10 Jun 2022 18:26:15 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-32-lKHzOdYSPHipkgrRWJBDZA-1; Fri, 10 Jun 2022 18:26:08 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 93289811E75; Fri, 10 Jun 2022 22:26:07 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.34.139]) by smtp.corp.redhat.com (Postfix) with ESMTP id 10AAE40D2962; Fri, 10 Jun 2022 22:26:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654899971; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Hpl9Hcmja7wCqA/6pHrgfp7z/GTa2UgFHuwdaUlLYz0=; b=gVayIGFIN0/aD2Khar2kfcL0WsrM+pQnrK+YsBLCpDVMYoZh03J/ZIlWnLTqahnfFzVa7L HqxcxnkrzpMVCDqKcmCrA0UGHuL1XuiE1+5VTYVYzKOzr1bLiAsyU3GDQDfnx+xro8Z2oj Y5bWA8O5xhFf5nHOjEL9NZMy5I3X8Sk= X-MC-Unique: lKHzOdYSPHipkgrRWJBDZA-1 From: John Snow To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Warner Losh , Beraldo Leal , Wainer dos Santos Moschetta , Kevin Wolf , Hanna Reitz , Daniel Berrange , Kyle Evans , John Snow , qemu-block@nongnu.org, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Thomas Huth , Cleber Rosa , Paolo Bonzini Subject: [RFC PATCH v2 1/7] tests: create optional tests/venv dependency groups Date: Fri, 10 Jun 2022 18:25:59 -0400 Message-Id: <20220610222605.2259132-2-jsnow@redhat.com> In-Reply-To: <20220610222605.2259132-1-jsnow@redhat.com> References: <20220610222605.2259132-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1654900105240100001 Content-Type: text/plain; charset="utf-8" This patch uses a dummy package and setup.cfg/setup.py files to manage optional dependency groups for the test venv specification. Now, there's a core set of dependencies which for now includes just "qemu" (but soon, qemu.qmp) and a separate, optional 'avocado' group that includes avocado-framework and pycdlib. Practical upshot: We install only a minimum of things for the majority of check-* targets, but allow optional add-ons to be processed when running avocado tests. This will spare downstreams from having to add more dependencies than is necessary as a build dependencies when invoking "make check". (We also keep both sets of dependencies in one file, which is helpful for review to ensure that different option groups don't conflict with one another.) NOTE: There is a non-fatal caveat introduced by this patch on Ubuntu 20.04 systems; see the subsequent commit "tests: Remove spurious pip warnings on Ubuntu20.04" for more information. Signed-off-by: John Snow --- tests/Makefile.include | 21 ++++++++++++++------- tests/requirements.txt | 6 ------ tests/setup.cfg | 20 ++++++++++++++++++++ tests/setup.py | 16 ++++++++++++++++ 4 files changed, 50 insertions(+), 13 deletions(-) delete mode 100644 tests/requirements.txt create mode 100644 tests/setup.cfg create mode 100644 tests/setup.py diff --git a/tests/Makefile.include b/tests/Makefile.include index 3accb83b132..82c697230e0 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -81,13 +81,13 @@ clean-tcg: $(CLEAN_TCG_TARGET_RULES) =20 # Python venv for running tests =20 -.PHONY: check-venv check-avocado check-acceptance check-acceptance-depreca= ted-warning +.PHONY: check-venv check-venv-avocado check-avocado check-acceptance \ + check-acceptance-deprecated-warning =20 # Build up our target list from the filtered list of ninja targets TARGETS=3D$(patsubst libqemu-%.fa, %, $(filter libqemu-%.fa, $(ninja-targe= ts))) =20 TESTS_VENV_DIR=3D$(BUILD_DIR)/tests/venv -TESTS_VENV_REQ=3D$(SRC_PATH)/tests/requirements.txt TESTS_RESULTS_DIR=3D$(BUILD_DIR)/tests/results TESTS_PYTHON=3D$(TESTS_VENV_DIR)/bin/python3 ifndef AVOCADO_TESTS @@ -108,10 +108,16 @@ quiet-venv-pip =3D $(quiet-@)$(call quiet-command-run= , \ $(TESTS_PYTHON) -m pip -q --disable-pip-version-check $1, \ "VENVPIP","$1") =20 -$(TESTS_VENV_DIR): $(TESTS_VENV_REQ) +# Core dependencies for tests/venv +$(TESTS_VENV_DIR): $(SRC_PATH)/tests/setup.cfg $(SRC_PATH)/python/setup.cfg $(call quiet-command, $(PYTHON) -m venv $@, VENV, $@) $(call quiet-venv-pip,install -e "$(SRC_PATH)/python/") - $(call quiet-venv-pip,install -r $(TESTS_VENV_REQ)) + $(call quiet-venv-pip,install "$(SRC_PATH)/tests/") + $(call quiet-command, touch $@) + +# Optional avocado dependencies for tests/venv +$(TESTS_VENV_DIR)/avocado: $(TESTS_VENV_DIR) + $(call quiet-venv-pip,install "$(SRC_PATH)/tests/[avocado]") $(call quiet-command, touch $@) =20 $(TESTS_RESULTS_DIR): @@ -119,6 +125,7 @@ $(TESTS_RESULTS_DIR): MKDIR, $@) =20 check-venv: $(TESTS_VENV_DIR) +check-venv-avocado: $(TESTS_VENV_DIR)/avocado =20 FEDORA_31_ARCHES_TARGETS=3D$(patsubst %-softmmu,%, $(filter %-softmmu,$(TA= RGETS))) FEDORA_31_ARCHES_CANDIDATES=3D$(patsubst ppc64,ppc64le,$(FEDORA_31_ARCHES_= TARGETS)) @@ -126,16 +133,16 @@ FEDORA_31_ARCHES :=3D x86_64 aarch64 ppc64le s390x FEDORA_31_DOWNLOAD=3D$(filter $(FEDORA_31_ARCHES),$(FEDORA_31_ARCHES_CANDI= DATES)) =20 # download one specific Fedora 31 image -get-vm-image-fedora-31-%: check-venv +get-vm-image-fedora-31-%: check-venv-avocado $(call quiet-command, \ $(TESTS_PYTHON) -m avocado vmimage get \ --distro=3Dfedora --distro-version=3D31 --arch=3D$*, \ "AVOCADO", "Downloading avocado tests VM image for $*") =20 # download all vm images, according to defined targets -get-vm-images: check-venv $(patsubst %,get-vm-image-fedora-31-%, $(FEDORA_= 31_DOWNLOAD)) +get-vm-images: check-venv-avocado $(patsubst %,get-vm-image-fedora-31-%, $= (FEDORA_31_DOWNLOAD)) =20 -check-avocado: check-venv $(TESTS_RESULTS_DIR) get-vm-images +check-avocado: check-venv-avocado $(TESTS_RESULTS_DIR) get-vm-images $(call quiet-command, \ $(TESTS_PYTHON) -m avocado \ --show=3D$(AVOCADO_SHOW) run --job-results-dir=3D$(TESTS_RESUL= TS_DIR) \ diff --git a/tests/requirements.txt b/tests/requirements.txt deleted file mode 100644 index 0ba561b6bdf..00000000000 --- a/tests/requirements.txt +++ /dev/null @@ -1,6 +0,0 @@ -# Add Python module requirements, one per line, to be installed -# in the tests/venv Python virtual environment. For more info, -# refer to: https://pip.pypa.io/en/stable/user_guide/#id1 -# Note that qemu.git/python/ is always implicitly installed. -avocado-framework=3D=3D88.1 -pycdlib=3D=3D1.11.0 diff --git a/tests/setup.cfg b/tests/setup.cfg new file mode 100644 index 00000000000..263a5de01af --- /dev/null +++ b/tests/setup.cfg @@ -0,0 +1,20 @@ +# This file represents a "dummy" package that expresses +# the dependencies necessary to run the various python-based +# test suites in the qemu.git tree. +[metadata] +name =3D qemu.dummy_tests +version =3D 0.0.0 +description =3D qemu.git testing dummy package + +[options] +packages =3D +python_requires =3D >=3D 3.6 +install_requires =3D + qemu # Note: A special setup script will always install 'qemu' from + # qemu.git/python. It doesn't have a meaningful/real version. + +[options.extras_require] +# Extra dependencies required by the avocado tests. +avocado =3D + avocado-framework<89.0,>=3D88.1 + pycdlib<2.0,>=3D1.11.0 diff --git a/tests/setup.py b/tests/setup.py new file mode 100644 index 00000000000..c99fe5d1725 --- /dev/null +++ b/tests/setup.py @@ -0,0 +1,16 @@ +# This setup file is just-enough-config to allow pip to bootstrap a +# testing environment. It is not meant to be executed directly. +# See also: setup.cfg + +import setuptools +import pkg_resources + + +def main(): + # https://medium.com/@daveshawley/safely-using-setup-cfg-for-metadata-= 1babbe54c108 + pkg_resources.require('setuptools>=3D39.2') + setuptools.setup() + + +if __name__ =3D=3D '__main__': + main() --=20 2.34.3 From nobody Mon Feb 9 03:47:26 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1654900088; cv=none; d=zohomail.com; s=zohoarc; b=JxAyfQqchPe30970WS5oonzyKRpb+2Ny19oi0Am6jR2C27kPoKJSMsgMwGU/H63LKlg5En+wGDRTEQdAdMwl6g7UNflxJMz9c+08jRlCihUsyaGuF1CHc1wxBWNM3X00Z7dlwG9xapTXmd9ziBmIrUxQxruvm6bOe3kCsjsBTgs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1654900088; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=anYhfcvgMMDENUPqkU/tUzCABSO9Egx8LCG3jbKLIcY=; b=YxwYxgszMSUt3WQWrPc8hnURXHqCSgLXbAQYMP3MIC7OlJLTFblLNsLHQD2azCuYbn4wph/7q4nxM1qiubaoabyz4HTwsTeLpFqJurYizrEWpvDZdEa/sxUyY1Qi8cwjnEhY28wv0ZS+2lUiZIedf6FmcAXf1rcjJPGPn9cXRtw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 16549000886261001.044208810724; Fri, 10 Jun 2022 15:28:08 -0700 (PDT) Received: from localhost ([::1]:60824 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nzn6s-00037U-Md for importer@patchew.org; Fri, 10 Jun 2022 18:28:06 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47038) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzn55-0000hs-SU for qemu-devel@nongnu.org; Fri, 10 Jun 2022 18:26:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:56996) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzn51-0005Tw-Dr for qemu-devel@nongnu.org; Fri, 10 Jun 2022 18:26:15 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-661-Kp4X-yEQNzWeCV1Juc1LKg-1; Fri, 10 Jun 2022 18:26:09 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2E12585A580; Fri, 10 Jun 2022 22:26:08 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.34.139]) by smtp.corp.redhat.com (Postfix) with ESMTP id A116240D2962; Fri, 10 Jun 2022 22:26:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654899970; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=anYhfcvgMMDENUPqkU/tUzCABSO9Egx8LCG3jbKLIcY=; b=OlRldXXzCJ332M207hEFBSz/80oapaVbis0kNah6o+pjWua9QxRUfR0h9XQjDsFg9B8Hw8 RDWp/qLqAgm/lS25244cXoel85BvaNs83prvCd4Z5QZg1g8yK3Dp7PrymASexOly8478qh QekDLNpIBuZX1mBOtGxMphsMnQmi8qM= X-MC-Unique: Kp4X-yEQNzWeCV1Juc1LKg-1 From: John Snow To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Warner Losh , Beraldo Leal , Wainer dos Santos Moschetta , Kevin Wolf , Hanna Reitz , Daniel Berrange , Kyle Evans , John Snow , qemu-block@nongnu.org, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Thomas Huth , Cleber Rosa , Paolo Bonzini Subject: [RFC PATCH v2 2/7] tests: pythonize test venv creation Date: Fri, 10 Jun 2022 18:26:00 -0400 Message-Id: <20220610222605.2259132-3-jsnow@redhat.com> In-Reply-To: <20220610222605.2259132-1-jsnow@redhat.com> References: <20220610222605.2259132-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1654900089452100001 Content-Type: text/plain; charset="utf-8" This splits the venv creation logic out of the Makefile and into a Python script. One reason for doing this is to be able to call the venv bootstrapper from places outside of the Makefile, e.g. configure and iotests. Another reason is to be able to add "offline" logic to modify the behavior of the script in certain circumstances. RFC: I don't like how I have an entire "enter_venv" function here, and by the end of the series, completely separate logic for entering a venv in testenv.py -- but they both operate in different ways: this patch offers a method that alters the current ENV immediately, whereas the testenv.py method alters a copied ENV that is explicitly passed to subprocesses. Still, there's a bit more boilerplate than I'd like, but it's probably fine and it probably won't need to be updated much, but... less code is usually better. But, even if I did write it more generically here; iotests wouldn't be able to use it anyway, because importing it would mean more sys.path hacking. So... eh? Signed-off-by: John Snow --- tests/Makefile.include | 16 ++-- tests/mkvenv.py | 186 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 192 insertions(+), 10 deletions(-) create mode 100644 tests/mkvenv.py diff --git a/tests/Makefile.include b/tests/Makefile.include index 82c697230e0..d8af6a38112 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -104,21 +104,17 @@ else AVOCADO_CMDLINE_TAGS=3D$(addprefix -t , $(AVOCADO_TAGS)) endif =20 -quiet-venv-pip =3D $(quiet-@)$(call quiet-command-run, \ - $(TESTS_PYTHON) -m pip -q --disable-pip-version-check $1, \ - "VENVPIP","$1") - # Core dependencies for tests/venv $(TESTS_VENV_DIR): $(SRC_PATH)/tests/setup.cfg $(SRC_PATH)/python/setup.cfg - $(call quiet-command, $(PYTHON) -m venv $@, VENV, $@) - $(call quiet-venv-pip,install -e "$(SRC_PATH)/python/") - $(call quiet-venv-pip,install "$(SRC_PATH)/tests/") - $(call quiet-command, touch $@) + $(call quiet-command, \ + $(PYTHON) "$(SRC_PATH)/tests/mkvenv.py" "$@", \ + MKVENV, $@) =20 # Optional avocado dependencies for tests/venv $(TESTS_VENV_DIR)/avocado: $(TESTS_VENV_DIR) - $(call quiet-venv-pip,install "$(SRC_PATH)/tests/[avocado]") - $(call quiet-command, touch $@) + $(call quiet-command, \ + $(PYTHON) "$(SRC_PATH)/tests/mkvenv.py" --opt avocado "$<", \ + MKVENV, $@) =20 $(TESTS_RESULTS_DIR): $(call quiet-command, mkdir -p $@, \ diff --git a/tests/mkvenv.py b/tests/mkvenv.py new file mode 100644 index 00000000000..0667106d6aa --- /dev/null +++ b/tests/mkvenv.py @@ -0,0 +1,186 @@ +""" +mkvenv - QEMU venv bootstrapping utility + +usage: mkvenv.py [-h] [--offline] [--opt OPT] target + +Bootstrap QEMU testing venv. + +positional arguments: + target Target directory to install virtual environment into. + +optional arguments: + -h, --help show this help message and exit + --offline Use system packages and work entirely offline. + --opt OPT Install an optional dependency group. +""" + +# Copyright (C) 2022 Red Hat, Inc. +# +# Authors: +# John Snow +# +# This work is licensed under the terms of the GNU GPL, version 2 or +# later. See the COPYING file in the top-level directory. + +# Do not add any dependencies from outside the stdlib: +# This script must be usable on its own! + +import argparse +from contextlib import contextmanager +import logging +import os +from pathlib import Path +import subprocess +import sys +from typing import Iterable, Iterator, Union +import venv + + +def make_venv( + venv_path: Union[str, Path], + system_site_packages: bool =3D False +) -> None: + """ + Create a venv using the python3 'venv' module. + + Identical to: + ``python3 -m venv --clear [--system-site-packages] {venv_path}`` + + :param venv_path: The target directory + :param system_site_packages: If True, allow system packages in venv. + """ + logging.debug("%s: make_venv(venv_path=3D%s, system_site_packages=3D%s= )", + __file__, str(venv_path), system_site_packages) + venv.create( + str(venv_path), + clear=3DTrue, + symlinks=3Dos.name !=3D "nt", # Same as venv CLI's default. + with_pip=3DTrue, + system_site_packages=3Dsystem_site_packages, + ) + + +@contextmanager +def enter_venv(venv_dir: Union[str, Path]) -> Iterator[None]: + """Scoped activation of an existing venv.""" + venv_keys =3D ('PATH', 'PYTHONHOME', 'VIRTUAL_ENV') + old =3D {k: v for k, v in os.environ.items() if k in venv_keys} + vdir =3D Path(venv_dir).resolve() + + def _delete_keys() -> None: + for k in venv_keys: + try: + del os.environ[k] + except KeyError: + pass + + try: + _delete_keys() + + os.environ['VIRTUAL_ENV'] =3D str(vdir) + os.environ['PATH'] =3D os.pathsep.join([ + str(vdir / "bin/"), + old['PATH'], + ]) + + logging.debug("PATH =3D> %s", os.environ['PATH']) + logging.debug("VIRTUAL_ENV =3D> %s", os.environ['VIRTUAL_ENV']) + + yield + + finally: + _delete_keys() + for key, val in old.items(): + os.environ[key] =3D val + + +def install(*args: str, offline: bool =3D False) -> None: + """Shorthand for pip install; expected to be used under a venv.""" + cli_args =3D ['pip3', '--disable-pip-version-check', '-q', 'install'] + if offline: + cli_args.append('--no-index') + cli_args +=3D args + print(f" VENVPIP install {' '.join(args)}") + logging.debug(' '.join(cli_args)) + subprocess.run(cli_args, check=3DTrue) + + +def make_qemu_venv( + venv_dir: str, + options: Iterable[str], + offline: bool =3D False +) -> None: + """ + Make and initialize a qemu testing venv. + + Forcibly re-creates the venv if it already exists, unless optional + dependency groups are specified - in which case, just install the + extra dependency groups. + + :param venv_dir: The target directory to install to. + :param options: + Optional dependency groups of the testing package to install. + (e.g. 'avocado'.) + :param offline: + If true, force offline mode. System packages will be usable from + the venv and dependencies will not be fetched from PyPI. + """ + venv_path =3D Path(venv_dir).resolve() + src_root =3D Path(__file__).joinpath('../..') + pysrc_path =3D src_root.joinpath('python/').resolve() + test_src_path =3D src_root.joinpath('tests/').resolve() + + logging.debug("make_qemu_venv(%s)", venv_dir) + logging.debug("sys.executable: %s", sys.executable) + logging.debug("resolved: %s", str(Path(sys.executable).resolve()= )) + logging.debug("venv_dir: %s", str(venv_path)) + logging.debug("python source: %s", str(pysrc_path)) + logging.debug("tests source: %s", str(test_src_path)) + + do_initialize =3D not venv_path.exists() or not options + if do_initialize: + make_venv(venv_path, system_site_packages=3Doffline) + + with enter_venv(venv_path): + if do_initialize: + install("-e", str(pysrc_path), offline=3Doffline) + install(str(test_src_path), offline=3Doffline) + venv_path.touch() + + for option in options: + dummy =3D venv_path / option + install(f"{test_src_path!s}/[{option}]", offline=3Doffline) + dummy.touch() + + +def main() -> int: + """CLI interface to make_qemu_venv. See module docstring.""" + if os.environ.get('DEBUG'): + logging.basicConfig(level=3Dlogging.DEBUG) + + parser =3D argparse.ArgumentParser( + description=3D"Bootstrap QEMU testing venv.") + parser.add_argument( + '--offline', + action=3D'store_true', + help=3D"Use system packages and work entirely offline.", + ) + parser.add_argument( + '--opt', + type=3Dstr, + action=3D'append', + help=3D"Install an optional dependency group.", + ) + parser.add_argument( + 'target', + type=3Dstr, + action=3D'store', + help=3D"Target directory to install virtual environment into.", + ) + args =3D parser.parse_args() + make_qemu_venv(args.target, args.opt or (), args.offline) + return 0 + + +if __name__ =3D=3D '__main__': + sys.exit(main()) --=20 2.34.3 From nobody Mon Feb 9 03:47:26 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1654900342; cv=none; d=zohomail.com; s=zohoarc; b=DWsmYnVaIGAjwlrzG4uGNv/ViJc/S17RycrXbyhMGRKDidkIxaipoAQjFDN0Xr6RtLfv+KXUA3cF+5RdD2Ujeel77xpRXLmhj4JHyKyzruxtLkRWxp2GHZLyVFEqCDntCUjyO3YX9snpiSFC7vMhv+kIE18rF13wNfDV++9SC60= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1654900342; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=STX+HDAQ+qzf8HD6UcpV60/VFSD4Vcst0FqCUJoq2SM=; b=byT/Ey2NbrNAgXvyeV3Lv5NeZhd4fmnSnAuUIKULJZI9TPO3vqXpo73myyOHzB+x73QarHtvwF/eip27mHTRcezn+QyUHbowshds7JSusO8HTgT/KE//AJtoVsB1H3UYg4vCKXk3d3UPozUMMx7oyk/RcHUni8sB7Ap6KagfuVk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1654900342695232.09231990185685; Fri, 10 Jun 2022 15:32:22 -0700 (PDT) Received: from localhost ([::1]:39024 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nznAz-0008KR-N4 for importer@patchew.org; Fri, 10 Jun 2022 18:32:21 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47102) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzn56-0000j0-Ke for qemu-devel@nongnu.org; Fri, 10 Jun 2022 18:26:16 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:22060) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzn52-0005Ua-Sa for qemu-devel@nongnu.org; Fri, 10 Jun 2022 18:26:16 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-518-UBwKrirDOg-rXPfnkBtupw-1; Fri, 10 Jun 2022 18:26:09 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id BE5FC85A582; Fri, 10 Jun 2022 22:26:08 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.34.139]) by smtp.corp.redhat.com (Postfix) with ESMTP id 3B6CF40D2962; Fri, 10 Jun 2022 22:26:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654899972; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=STX+HDAQ+qzf8HD6UcpV60/VFSD4Vcst0FqCUJoq2SM=; b=e6mUD+Y++7gzBJDoLZRmK9qMf9uhRAPxU+fTaQ0zN9BmN+StQb0PPQMZ4aBjVDwl0trmX7 lxgeSGPuAjceZMhc6RD3EABYPiAzyJeBfSyhYH7+H7Gk4sAw2x9b2Mf8k7sHaYhDAZtn4p JhI2PohErddzeOECu2xB5X2Fg5Ypw9o= X-MC-Unique: UBwKrirDOg-rXPfnkBtupw-1 From: John Snow To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Warner Losh , Beraldo Leal , Wainer dos Santos Moschetta , Kevin Wolf , Hanna Reitz , Daniel Berrange , Kyle Evans , John Snow , qemu-block@nongnu.org, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Thomas Huth , Cleber Rosa , Paolo Bonzini Subject: [RFC PATCH v2 3/7] tests: Remove spurious pip warnings on Ubuntu20.04 Date: Fri, 10 Jun 2022 18:26:01 -0400 Message-Id: <20220610222605.2259132-4-jsnow@redhat.com> In-Reply-To: <20220610222605.2259132-1-jsnow@redhat.com> References: <20220610222605.2259132-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1654900342923100001 Content-Type: text/plain; charset="utf-8" The version of pip ("20.0.2") that ships with Ubuntu 20.04 has a bug where it will try to attempt building a wheel even if the "wheel" python package that enables it to do so is not installed. Even though pip continues gracefully from source, The result is a lot of irrelevant failure output. Upstream pip 20.0.2 does not have this problem, and pip 20.1 introduces a new info message that informs a user that wheel building is being skipped. On this version, the output can be silenced by passing --no-binary to coax pip into skipping that step to begin with. Note, this error does not seem to show up for the "qemu" package because we install that package in editable mode. (I think, but did not test, that installing an empty package in editable mode caused more problems than it fixed.) Signed-off-by: John Snow --- tests/mkvenv.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/mkvenv.py b/tests/mkvenv.py index 0667106d6aa..78f8d0382e0 100644 --- a/tests/mkvenv.py +++ b/tests/mkvenv.py @@ -144,7 +144,8 @@ def make_qemu_venv( with enter_venv(venv_path): if do_initialize: install("-e", str(pysrc_path), offline=3Doffline) - install(str(test_src_path), offline=3Doffline) + install("--no-binary", "qemu.dummy-tests", + str(test_src_path), offline=3Doffline) venv_path.touch() =20 for option in options: --=20 2.34.3 From nobody Mon Feb 9 03:47:26 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1654900470; cv=none; d=zohomail.com; s=zohoarc; b=Uj6nGfh1G4uCjqyPZyhiGMS+rZK1lFAvLOUTdcR9iluJsqv+wNZPkpSCTgfBhqBMX0f8Q5hZLjTxi+BIKYPBW95HLbaYXeJ8WHIxu9snwHe4W1ehl+OAd4AI4uReFCyBXPwU9L7m/ZNQiN7/yPrSfk3pt2gj/fbKxv+ZMgM8/yY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1654900470; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=B1ysuS0Y51qktc7HoHw3hcZsl4RHIYhU/hRbczMkewY=; b=mMW9OxEEJH+M/UaVEs6QO78X/ZAaUzICjiaANAqu8vIOE54gnbMwy7xlww637je1UTgWDYY3jZFzeD4TmvWyUMtVc7Y6fSDjkExnoY5EVcGRWeM4f8Xzv43VGNVKw3lFtbVDlKMTPC4kf/n1+2BBO/GYVDBIQmFToOP7STMi5qE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1654900470297768.8865260807888; Fri, 10 Jun 2022 15:34:30 -0700 (PDT) Received: from localhost ([::1]:43798 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nznD3-0003mP-5S for importer@patchew.org; Fri, 10 Jun 2022 18:34:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47100) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzn56-0000iw-Km for qemu-devel@nongnu.org; Fri, 10 Jun 2022 18:26:16 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:57383) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzn53-0005V7-BI for qemu-devel@nongnu.org; Fri, 10 Jun 2022 18:26:16 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-237-hkm3Ha4-P-W2aa3ILua4Dg-1; Fri, 10 Jun 2022 18:26:09 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 59288811E75; Fri, 10 Jun 2022 22:26:09 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.34.139]) by smtp.corp.redhat.com (Postfix) with ESMTP id CBEAE40E80E0; Fri, 10 Jun 2022 22:26:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654899972; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=B1ysuS0Y51qktc7HoHw3hcZsl4RHIYhU/hRbczMkewY=; b=cDzeJlOF3q+rNGE/ArgVx4/vjJIRqDtl3WwaUBr4USzaBY1CrGi6voKZlxTkFu52npKpZT I4FEnJf8qRkH+4+kz/Vy3S72L6M33hABa1VrNKY0MMP1/nktkz/5iAZ6HGDKAOjVZsVYII 7V/Fq4sUSGs74eoJLpj8lTXHTbBf8bE= X-MC-Unique: hkm3Ha4-P-W2aa3ILua4Dg-1 From: John Snow To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Warner Losh , Beraldo Leal , Wainer dos Santos Moschetta , Kevin Wolf , Hanna Reitz , Daniel Berrange , Kyle Evans , John Snow , qemu-block@nongnu.org, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Thomas Huth , Cleber Rosa , Paolo Bonzini Subject: [RFC PATCH v2 4/7] tests/vm: add venv pre-requisites to VM building recipes Date: Fri, 10 Jun 2022 18:26:02 -0400 Message-Id: <20220610222605.2259132-5-jsnow@redhat.com> In-Reply-To: <20220610222605.2259132-1-jsnow@redhat.com> References: <20220610222605.2259132-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1654900471451100001 Content-Type: text/plain; charset="utf-8" Ubuntu needs "python3-venv" in order to create virtual environments, and NetBSD needs "py37-pip" in order to do the same. Signed-off-by: John Snow --- tests/vm/netbsd | 1 + tests/vm/ubuntu.i386 | 9 ++++++--- 2 files changed, 7 insertions(+), 3 deletions(-) diff --git a/tests/vm/netbsd b/tests/vm/netbsd index 45aa9a7fda7..53fe508e487 100755 --- a/tests/vm/netbsd +++ b/tests/vm/netbsd @@ -31,6 +31,7 @@ class NetBSDVM(basevm.BaseVM): "pkgconf", "xz", "python37", + "py37-pip", "ninja-build", =20 # gnu tools diff --git a/tests/vm/ubuntu.i386 b/tests/vm/ubuntu.i386 index 47681b6f87d..40fd5ec86da 100755 --- a/tests/vm/ubuntu.i386 +++ b/tests/vm/ubuntu.i386 @@ -16,9 +16,12 @@ import basevm import ubuntuvm =20 DEFAULT_CONFIG =3D { - 'install_cmds' : "apt-get update,"\ - "apt-get build-dep -y qemu,"\ - "apt-get install -y libfdt-dev language-pack-en ninja= -build", + 'install_cmds' : ( + "apt-get update," + "apt-get build-dep -y qemu," + "apt-get install -y libfdt-dev language-pack-en ninja-build," + "apt-get install -y python3-venv" + ), } =20 class UbuntuX86VM(ubuntuvm.UbuntuVM): --=20 2.34.3 From nobody Mon Feb 9 03:47:26 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1654900466; cv=none; d=zohomail.com; s=zohoarc; b=jJrqKiMbiz7PC2wU2MaGeGhXCAhwqu0OMeriKbqfIyY5LKB+SgzvDywMDaB0Jv2sagA+MRP9Vvqzs12wriCChUN6j2hhnH3kkW3H9jNMLM319Wu0sZ+MPOqj5UTYC9abxj3+7XDcRlF9UJb9q0ZfXjO9pF3GwxeA59DOHiO/BiM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1654900466; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=bjWxx+HtDYkpHG8IIiAqT8hxb/16pFET6uQ6IklA3os=; b=RzTtp9dDYq/HyOu/koejV8RKColUXeJfrPZrjbsT/bG12ahlA7xU+DKKXg38EQoygcdl/+mGxEstEWeLdH1974Hh25E79Hy7IST7n+xrCpVmh/f3FYVHTuEaNRu6MWFUZPQ4QirEqm358GCXJ4SxJGo/KEP59nat48HSKVB9jws= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1654900466420502.81209134944334; Fri, 10 Jun 2022 15:34:26 -0700 (PDT) Received: from localhost ([::1]:43580 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nznCz-0003dV-5x for importer@patchew.org; Fri, 10 Jun 2022 18:34:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47146) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzn57-0000lK-Nx for qemu-devel@nongnu.org; Fri, 10 Jun 2022 18:26:17 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:59210) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzn54-0005W1-9f for qemu-devel@nongnu.org; Fri, 10 Jun 2022 18:26:17 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-416-x1P8rO70OhuH9EmfRHY5Vg-1; Fri, 10 Jun 2022 18:26:10 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E98668027EE; Fri, 10 Jun 2022 22:26:09 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.34.139]) by smtp.corp.redhat.com (Postfix) with ESMTP id 66C0440D2962; Fri, 10 Jun 2022 22:26:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654899973; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=bjWxx+HtDYkpHG8IIiAqT8hxb/16pFET6uQ6IklA3os=; b=DSBxhZ/OgdMhBmE+OrGpI5aHuntG6Yi8zZNIw35hVm23HsVVyLiyifuxyPjU3acWbb9FkF 7WhaNGXNO01yLdmBYDHWOWpqQGP8YogJ9SeeESH/13ZL9XD+RbWRPRt7pEO6P9Cv5XKCCf zMJ23WA6GyJixfuzh/elRnb/YwE0cUM= X-MC-Unique: x1P8rO70OhuH9EmfRHY5Vg-1 From: John Snow To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Warner Losh , Beraldo Leal , Wainer dos Santos Moschetta , Kevin Wolf , Hanna Reitz , Daniel Berrange , Kyle Evans , John Snow , qemu-block@nongnu.org, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Thomas Huth , Cleber Rosa , Paolo Bonzini Subject: [RFC PATCH v2 5/7] tests: add 'check-venv' as a dependency of 'make check' Date: Fri, 10 Jun 2022 18:26:03 -0400 Message-Id: <20220610222605.2259132-6-jsnow@redhat.com> In-Reply-To: <20220610222605.2259132-1-jsnow@redhat.com> References: <20220610222605.2259132-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1654900467798100001 Content-Type: text/plain; charset="utf-8" This patch adds the 'check-venv' target as a requisite of all meson driven check-* targets. As of this commit, it will only install the "qemu" namespace package from the source tree, and nothing else. In the future, the "qemu" namespace package in qemu.git will begin to require an external qemu.qmp package, and this would be installed into this environment as well. The avocado test dependencies will *not* be pulled into this venv by default, but they may be added in at a later point in time by running 'make check-avocado' or, without running the tests, 'make check-venv-avocado'. Signed-off-by: John Snow --- tests/Makefile.include | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/Makefile.include b/tests/Makefile.include index d8af6a38112..d484a335be5 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -155,6 +155,9 @@ check-acceptance-deprecated-warning: =20 check-acceptance: check-acceptance-deprecated-warning | check-avocado =20 +# The do-meson-check and do-meson-bench targets are defined in Makefile.mt= est +do-meson-check do-meson-bench: check-venv + # Consolidated targets =20 .PHONY: check check-clean get-vm-images --=20 2.34.3 From nobody Mon Feb 9 03:47:26 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1654900336; cv=none; d=zohomail.com; s=zohoarc; b=DBJMcK8ZZVMsxFgioMCLYWigMycJxD0GSv9uzLs6vEGQI7Uvt8L/pQaCLHO7Hq5rnl7oTjlYQ+93Y6dKxenycyearstBcPz8jQb42TdIgqAG6ylb5pUN72GvT8gcirDeW8/b/xWQJ+ytYeo0dVuE6ig4b+aYJvLMUeNsP1juw68= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1654900336; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=62eiebJ4CRcXipkHxKOY6aVMeIBXkh9jFMURuOSkro4=; b=BCINO9c60PL//uaL3Zkm3V9atEUuSN/UTRTnLATt4J092eRR0kcX+4j9f48oNaB7Pfayw4yCj9hBIn6OXapZGtb11sjeEpW5Ez6RQM0f7hVcY4R4uiDxidCCE0NEVYDd+/Hwijy4Mwfd6PysAGbCujiZNJFjE9Ql9+XXwvPz4ww= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 165490033648129.95323394024001; Fri, 10 Jun 2022 15:32:16 -0700 (PDT) Received: from localhost ([::1]:38780 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nznAt-00089X-8o for importer@patchew.org; Fri, 10 Jun 2022 18:32:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47032) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzn55-0000hm-SV for qemu-devel@nongnu.org; Fri, 10 Jun 2022 18:26:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:33182) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzn52-0005UJ-Im for qemu-devel@nongnu.org; Fri, 10 Jun 2022 18:26:14 -0400 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-315-iHV5NKanN-STDPj1bYMNDA-1; Fri, 10 Jun 2022 18:26:11 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8730A3815D22; Fri, 10 Jun 2022 22:26:10 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.34.139]) by smtp.corp.redhat.com (Postfix) with ESMTP id 02D3040D296B; Fri, 10 Jun 2022 22:26:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654899972; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=62eiebJ4CRcXipkHxKOY6aVMeIBXkh9jFMURuOSkro4=; b=aVlq+Pd/Mwi0Y/Vsy3xPi1EYBN200j+P00F2bDQRI3XUzGq9bEfoPmYJfkDbieWIs3b/pm TLRpyRGPPPFNAKUVoLybbucDQDceaN4xWXHmk/k0U6/BcA52rVqe5Rgy6aYkU8mhm6EJN8 Duzud3U/LUabbsRt1yQRd5laxjuRSYI= X-MC-Unique: iHV5NKanN-STDPj1bYMNDA-1 From: John Snow To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Warner Losh , Beraldo Leal , Wainer dos Santos Moschetta , Kevin Wolf , Hanna Reitz , Daniel Berrange , Kyle Evans , John Snow , qemu-block@nongnu.org, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Thomas Huth , Cleber Rosa , Paolo Bonzini Subject: [RFC PATCH v2 6/7] iotests: use tests/venv for running tests Date: Fri, 10 Jun 2022 18:26:04 -0400 Message-Id: <20220610222605.2259132-7-jsnow@redhat.com> In-Reply-To: <20220610222605.2259132-1-jsnow@redhat.com> References: <20220610222605.2259132-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1654900336877100001 Content-Type: text/plain; charset="utf-8" Essentially, the changes to testenv.py here mimic the changes that occur wh= en you "source venv/bin/activate.fish" or similar. (1) update iotest's internal notion of which python binary to use, (2) export the VIRTUAL_ENV variable, (3) front-load the venv/bin directory to PATH. If the venv directory isn't found, raise a friendly exception that tries to give the human operator a friendly clue as to what's gone wrong. The subsequent commit attempts to address this shortcoming by teaching iotests how to invoke the venv bootstrapper in this circumstance instead. Signed-off-by: John Snow --- tests/qemu-iotests/testenv.py | 25 +++++++++++++++++++------ 1 file changed, 19 insertions(+), 6 deletions(-) diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py index a864c74b123..29404ac94be 100644 --- a/tests/qemu-iotests/testenv.py +++ b/tests/qemu-iotests/testenv.py @@ -65,8 +65,9 @@ class TestEnv(ContextManager['TestEnv']): # lot of them. Silence pylint: # pylint: disable=3Dtoo-many-instance-attributes =20 - env_variables =3D ['PYTHONPATH', 'TEST_DIR', 'SOCK_DIR', 'SAMPLE_IMG_D= IR', - 'PYTHON', 'QEMU_PROG', 'QEMU_IMG_PROG', + env_variables =3D ['PYTHONPATH', 'VIRTUAL_ENV', 'PYTHON', 'PATH', + 'TEST_DIR', 'SOCK_DIR', 'SAMPLE_IMG_DIR', + 'QEMU_PROG', 'QEMU_IMG_PROG', 'QEMU_IO_PROG', 'QEMU_NBD_PROG', 'QSD_PROG', 'QEMU_OPTIONS', 'QEMU_IMG_OPTIONS', 'QEMU_IO_OPTIONS', 'QEMU_IO_OPTIONS_NO_FMT', @@ -102,18 +103,29 @@ def get_env(self) -> Dict[str, str]: =20 def init_directories(self) -> None: """Init directory variables: + VIRTUAL_ENV + PATH PYTHONPATH TEST_DIR SOCK_DIR SAMPLE_IMG_DIR """ + venv_path =3D Path(self.build_root, 'tests/venv/') + if not venv_path.exists(): + raise FileNotFoundError( + f"Virtual environment \"{venv_path!s}\" isn't found." + " (Maybe you need to run 'make check-venv'" + " from the build dir?)" + ) + self.virtual_env: str =3D str(venv_path) =20 - # Path where qemu goodies live in this source tree. - qemu_srctree_path =3D Path(__file__, '../../../python').resolve() + self.path =3D os.pathsep.join(( + str(venv_path / 'bin'), + os.environ['PATH'], + )) =20 self.pythonpath =3D os.pathsep.join(filter(None, ( self.source_iotests, - str(qemu_srctree_path), os.getenv('PYTHONPATH'), ))) =20 @@ -138,7 +150,7 @@ def init_binaries(self) -> None: PYTHON (for bash tests) QEMU_PROG, QEMU_IMG_PROG, QEMU_IO_PROG, QEMU_NBD_PROG, QSD_PR= OG """ - self.python =3D sys.executable + self.python: str =3D os.path.join(self.virtual_env, 'bin', 'python= 3') =20 def root(*names: str) -> str: return os.path.join(self.build_root, *names) @@ -300,6 +312,7 @@ def print_env(self, prefix: str =3D '') -> None: {prefix}GDB_OPTIONS -- {GDB_OPTIONS} {prefix}VALGRIND_QEMU -- {VALGRIND_QEMU} {prefix}PRINT_QEMU_OUTPUT -- {PRINT_QEMU} +{prefix}VIRTUAL_ENV -- {VIRTUAL_ENV} {prefix}""" =20 args =3D collections.defaultdict(str, self.get_env()) --=20 2.34.3 From nobody Mon Feb 9 03:47:26 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1654900562; cv=none; d=zohomail.com; s=zohoarc; b=gzozbqEBMTc3Sbi64vl9Ru1ZN9OoSqo8ZG3eav6Prq5z045aViZBHY7pY3nXWXGbLIY9n7Mja5E6X2tUCUQhf5eSnj7XIRNBXuz9RMPLjnNdzbErVIth8ZSFun5NymTCTUaHCctHvK/emxPeY5uYQz/lvpx2DmeUISs/KpsDJ+k= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1654900562; h=Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=0f3qVSnmvQZOUE7GmzmLW3QsKQrZDOm39K1IEFxZUeU=; b=G51LNsv9A8929tciP0cCOfBtn3Yts+kN2H7qOaMyZbp0xaaIMPQ+/I873WOS+/0B3gVieAorXpG/kmV6KP8LcyGrlwXMR4a0PG3fxg20pMSevf/bm8iOiw50LL6JHYGwLfsxaPgtfZy9kWnM5GY57L5d7sR5Fk7W3rxZgKkmsnA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1654900562329666.7442658894664; Fri, 10 Jun 2022 15:36:02 -0700 (PDT) Received: from localhost ([::1]:47290 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1nznEW-00067z-RH for importer@patchew.org; Fri, 10 Jun 2022 18:36:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:47158) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzn58-0000nQ-Fw for qemu-devel@nongnu.org; Fri, 10 Jun 2022 18:26:18 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]:58162) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1nzn55-0005Wl-Ci for qemu-devel@nongnu.org; Fri, 10 Jun 2022 18:26:18 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-633-RAlBHuh2MyiRwN8wHPRMuw-1; Fri, 10 Jun 2022 18:26:11 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 24FAF85A580; Fri, 10 Jun 2022 22:26:11 +0000 (UTC) Received: from scv.redhat.com (unknown [10.22.34.139]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9479440E80E0; Fri, 10 Jun 2022 22:26:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1654899974; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0f3qVSnmvQZOUE7GmzmLW3QsKQrZDOm39K1IEFxZUeU=; b=MkkhUZM9gFwtOekzGOowQou2wjMnYpkTaNhG4xZz8o3sljt5RzjYylrWoXkbYz9v7G403R wUsWamKalFzN8zALazcYttHD8iDXi2rWUyu8XbFwUfXX5jG/8JHo6qSv1wxM6mNvmJw2vo oTkqQs/bdDhc6KVV5eVOVghCycc2XuE= X-MC-Unique: RAlBHuh2MyiRwN8wHPRMuw-1 From: John Snow To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , Warner Losh , Beraldo Leal , Wainer dos Santos Moschetta , Kevin Wolf , Hanna Reitz , Daniel Berrange , Kyle Evans , John Snow , qemu-block@nongnu.org, =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Thomas Huth , Cleber Rosa , Paolo Bonzini Subject: [RFC PATCH v2 7/7] iotests: self-bootstrap testing venv Date: Fri, 10 Jun 2022 18:26:05 -0400 Message-Id: <20220610222605.2259132-8-jsnow@redhat.com> In-Reply-To: <20220610222605.2259132-1-jsnow@redhat.com> References: <20220610222605.2259132-1-jsnow@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.84 on 10.11.54.2 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=jsnow@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.082, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1654900564157100001 Content-Type: text/plain; charset="utf-8" When iotests are invoked manually from e.g. $build/tests/qemu-iotests/check, it is not necessarily guaranteed that we'll have run 'check-venv' yet. With this patch, teach testenv.py how to create its own environment. Note: this self-bootstrapping is fairly rudimentary and will miss certain triggers to refresh the venv. It will miss when new dependencies are added to either python/setup.cfg or tests/setup.cfg. It can be coaxed into updating by running 'make check', 'make check-block', 'make check-venv', etc. Signed-off-by: John Snow --- tests/qemu-iotests/testenv.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/qemu-iotests/testenv.py b/tests/qemu-iotests/testenv.py index 29404ac94be..e985eaf3e97 100644 --- a/tests/qemu-iotests/testenv.py +++ b/tests/qemu-iotests/testenv.py @@ -112,10 +112,10 @@ def init_directories(self) -> None: """ venv_path =3D Path(self.build_root, 'tests/venv/') if not venv_path.exists(): - raise FileNotFoundError( - f"Virtual environment \"{venv_path!s}\" isn't found." - " (Maybe you need to run 'make check-venv'" - " from the build dir?)" + mkvenv =3D Path(self.source_iotests, '../mkvenv.py') + subprocess.run( + (sys.executable, str(mkvenv), str(venv_path)), + check=3DTrue, ) self.virtual_env: str =3D str(venv_path) =20 --=20 2.34.3