1. Patchew
  2. QEMU
  3. [Qemu-devel] [PATCH v6 0/3] Bootstrap Python venv and acceptance/functional tests
  4. View patch

[Qemu-devel] [PATCH v6 1/3] Bootstrap Python venv for tests

Cleber Rosa posted 3 patches 7 years ago
There is a newer version of this series
[Qemu-devel] [PATCH v6 1/3] Bootstrap Python venv for tests
Posted by Cleber Rosa 7 years ago 
A number of QEMU tests are written in Python, and may benefit
from an untainted Python venv.

By using make rules, tests that depend on specific Python libs
can set that rule as a requirement, along with rules that require
the presence or installation of specific libraries.

The tests/venv-requirements.txt is supposed to contain the
Python requirements that should be added to the venv created
by check-venv.

Signed-off-by: Cleber Rosa <crosa@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
---
 tests/Makefile.include | 26 ++++++++++++++++++++++++++
 tests/requirements.txt |  3 +++
 2 files changed, 29 insertions(+)
 create mode 100644 tests/requirements.txt

diff --git a/tests/Makefile.include b/tests/Makefile.include
index 5eadfd52f9..4b88a119e0 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -12,6 +12,7 @@ check-help:
 	@echo " $(MAKE) check-block          Run block tests"
 	@echo " $(MAKE) check-tcg            Run TCG tests"
 	@echo " $(MAKE) check-report.html    Generates an HTML test report"
+	@echo " $(MAKE) check-venv           Creates a Python venv for tests"
 	@echo " $(MAKE) check-clean          Clean the tests"
 	@echo
 	@echo "Please note that HTML reports do not regenerate if the unit tests"
@@ -1017,6 +1018,30 @@ check-decodetree:
           ./check.sh "$(PYTHON)" "$(SRC_PATH)/scripts/decodetree.py", \
           TEST, decodetree.py)
 
+# Python venv for running tests
+
+.PHONY: check-venv
+
+TESTS_VENV_DIR=$(BUILD_DIR)/tests/venv
+TESTS_VENV_REQ=$(SRC_PATH)/tests/requirements.txt
+
+PYTHON_VER := $(findstring Python 3,$(shell $(PYTHON) -V))
+ifeq ($(PYTHON_VER),Python 3)
+$(TESTS_VENV_DIR): $(TESTS_VENV_REQ)
+	$(call quiet-command, \
+            $(PYTHON) -m venv --system-site-packages $@, \
+            VENV, $@)
+	$(call quiet-command, \
+            $(TESTS_VENV_DIR)/bin/python -m pip -q install -r $(TESTS_VENV_REQ), \
+            PIP, $(TESTS_VENV_REQ))
+	$(call quiet-command, touch $@)
+else
+$(TESTS_VENV_DIR):
+       $(error "venv directory for tests requires Python 3")
+endif
+
+check-venv: $(TESTS_VENV_DIR)
+
 # Consolidated targets
 
 .PHONY: check-qapi-schema check-qtest check-unit check check-clean
@@ -1030,6 +1055,7 @@ check-clean:
 	rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y)
 	rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y))
 	rm -f tests/test-qapi-gen-timestamp
+	rm -rf $(TESTS_VENV_DIR)
 
 clean: check-clean
 
diff --git a/tests/requirements.txt b/tests/requirements.txt
new file mode 100644
index 0000000000..d39f9d1576
--- /dev/null
+++ b/tests/requirements.txt
@@ -0,0 +1,3 @@
+# Add Python module requirements, one per line, to be installed
+# in the tests/venv Python virtual environment. For more info,
+# refer to: https://pip.pypa.io/en/stable/user_guide/#id1
-- 
2.17.1
Re: [Qemu-devel] [PATCH v6 1/3] Bootstrap Python venv for tests
Posted by Wainer dos Santos Moschetta 7 years ago 
On 10/17/2018 01:05 PM, Cleber Rosa wrote:
> A number of QEMU tests are written in Python, and may benefit
> from an untainted Python venv.
>
> By using make rules, tests that depend on specific Python libs
> can set that rule as a requirement, along with rules that require
> the presence or installation of specific libraries.
>
> The tests/venv-requirements.txt is supposed to contain the
> Python requirements that should be added to the venv created
> by check-venv.
>
> Signed-off-by: Cleber Rosa <crosa@redhat.com>
> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> Acked-by: Stefan Hajnoczi <stefanha@redhat.com>
> ---
>   tests/Makefile.include | 26 ++++++++++++++++++++++++++
>   tests/requirements.txt |  3 +++
>   2 files changed, 29 insertions(+)
>   create mode 100644 tests/requirements.txt
>
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index 5eadfd52f9..4b88a119e0 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -12,6 +12,7 @@ check-help:
>   	@echo " $(MAKE) check-block          Run block tests"
>   	@echo " $(MAKE) check-tcg            Run TCG tests"
>   	@echo " $(MAKE) check-report.html    Generates an HTML test report"
> +	@echo " $(MAKE) check-venv           Creates a Python venv for tests"
>   	@echo " $(MAKE) check-clean          Clean the tests"
>   	@echo
>   	@echo "Please note that HTML reports do not regenerate if the unit tests"
> @@ -1017,6 +1018,30 @@ check-decodetree:
>             ./check.sh "$(PYTHON)" "$(SRC_PATH)/scripts/decodetree.py", \
>             TEST, decodetree.py)
>   
> +# Python venv for running tests
> +
> +.PHONY: check-venv
> +
> +TESTS_VENV_DIR=$(BUILD_DIR)/tests/venv
> +TESTS_VENV_REQ=$(SRC_PATH)/tests/requirements.txt
> +
> +PYTHON_VER := $(findstring Python 3,$(shell $(PYTHON) -V))
> +ifeq ($(PYTHON_VER),Python 3)
> +$(TESTS_VENV_DIR): $(TESTS_VENV_REQ)
> +	$(call quiet-command, \
> +            $(PYTHON) -m venv --system-site-packages $@, \
> +            VENV, $@)
> +	$(call quiet-command, \
> +            $(TESTS_VENV_DIR)/bin/python -m pip -q install -r $(TESTS_VENV_REQ), \
> +            PIP, $(TESTS_VENV_REQ))
> +	$(call quiet-command, touch $@)
> +else
> +$(TESTS_VENV_DIR):
> +       $(error "venv directory for tests requires Python 3")

Above line is indented with space instead of tab. Thus, it breaks the build:

# make -j4
Python 2.7.15
/root/qemu/tests/Makefile.include:1040: *** "venv directory for tests 
requires Python 3".  Stop.

Also make check-help prints python's version when the build was 
configured with py2 (or the system lacks py3):

# make check-help
Python 2.7.15
Regression testing targets:

  make check                Run all tests
  make check-qtest-TARGET   Run qtest tests for given target
<output omitted>

I tested this patch series on Fedora 28 x86_64. It works fine except for 
above issues, so once fixing them:

Acked-by: Wainer dos Santos Moschetta <wainersm@redhat.com>


> +endif
> +
> +check-venv: $(TESTS_VENV_DIR)
> +
>   # Consolidated targets
>   
>   .PHONY: check-qapi-schema check-qtest check-unit check check-clean
> @@ -1030,6 +1055,7 @@ check-clean:
>   	rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y)
>   	rm -rf $(sort $(foreach target,$(SYSEMU_TARGET_LIST), $(check-qtest-$(target)-y)) $(check-qtest-generic-y))
>   	rm -f tests/test-qapi-gen-timestamp
> +	rm -rf $(TESTS_VENV_DIR)
>   
>   clean: check-clean
>   
> diff --git a/tests/requirements.txt b/tests/requirements.txt
> new file mode 100644
> index 0000000000..d39f9d1576
> --- /dev/null
> +++ b/tests/requirements.txt
> @@ -0,0 +1,3 @@
> +# Add Python module requirements, one per line, to be installed
> +# in the tests/venv Python virtual environment. For more info,
> +# refer to: https://pip.pypa.io/en/stable/user_guide/#id1

Patchew 2.1-devel-b67e4c2

© 2016 - 2025 Red Hat, Inc.