tests: Run device-crash-test on "make check"

[Qemu-devel] [PATCH 8/8] tests: Run device-crash-test on "make check"

Posted by Eduardo Habkost 7 years, 7 months ago

Run a subset of tests using device-crash-test on "make check", to
help us catch device crashes earlier.

This also add a "check-device-crash-test-full" rule, that will
check all machine/device combinations.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
---
 tests/Makefile.include | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index ef9b88c369..0ba641b8d4 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -942,6 +942,17 @@ check-decodetree:
           ./check.sh "$(PYTHON)" "$(SRC_PATH)/scripts/decodetree.py", \
           TEST, decodetree.py)
 
+CRASH_TEST = $(SRC_PATH)/scripts/device-crash-test
+CRASH_TEST_OPTIONS = $(if $(V),-v,-q)
+CRASH_TEST_BINARIES = $(foreach TARGET,$(TARGETS), $(TARGET)-softmmu/qemu-system-$(TARGET))
+
+.PHONY: check-device-crash-quick
+check-device-crash-quick:
+	$(CRASH_TEST) $(CRASH_TEST_OPTIONS) -t machine=DEFAULT accel=tcg -- $(CRASH_TEST_BINARIES)
+
+check-device-crash-full:
+	$(CRASH_TEST) $(CRASH_TEST_OPTIONS) -F $(CRASH_TEST_BINARIES)
+
 # Consolidated targets
 
 .PHONY: check-qapi-schema check-qtest check-unit check check-clean
@@ -950,7 +961,7 @@ check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS))
 check-unit: $(patsubst %,check-%, $(check-unit-y))
 check-speed: $(patsubst %,check-%, $(check-speed-y))
 check-block: $(patsubst %,check-%, $(check-block-y))
-check: check-qapi-schema check-unit check-qtest check-decodetree
+check: check-qapi-schema check-unit check-qtest check-decodetree check-device-crash-quick
 check-clean:
 	$(MAKE) -C tests/tcg clean
 	rm -rf $(check-unit-y) tests/*.o $(QEMU_IOTESTS_HELPERS-y)
-- 
2.14.3

Re: [Qemu-devel] [PATCH 8/8] tests: Run device-crash-test on "make check"

Posted by Eduardo Habkost 7 years, 7 months ago

On Fri, Mar 09, 2018 at 05:28:27PM -0300, Eduardo Habkost wrote:
> Run a subset of tests using device-crash-test on "make check", to
> help us catch device crashes earlier.
> 
> This also add a "check-device-crash-test-full" rule, that will
> check all machine/device combinations.
> 
> Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
> ---
>  tests/Makefile.include | 13 ++++++++++++-
>  1 file changed, 12 insertions(+), 1 deletion(-)
> 
> diff --git a/tests/Makefile.include b/tests/Makefile.include
> index ef9b88c369..0ba641b8d4 100644
> --- a/tests/Makefile.include
> +++ b/tests/Makefile.include
> @@ -942,6 +942,17 @@ check-decodetree:
>            ./check.sh "$(PYTHON)" "$(SRC_PATH)/scripts/decodetree.py", \
>            TEST, decodetree.py)
>  
> +CRASH_TEST = $(SRC_PATH)/scripts/device-crash-test
> +CRASH_TEST_OPTIONS = $(if $(V),-v,-q)
> +CRASH_TEST_BINARIES = $(foreach TARGET,$(TARGETS), $(TARGET)-softmmu/qemu-system-$(TARGET))
> +
> +.PHONY: check-device-crash-quick
> +check-device-crash-quick:
> +	$(CRASH_TEST) $(CRASH_TEST_OPTIONS) -t machine=DEFAULT accel=tcg -- $(CRASH_TEST_BINARIES)
> +
> +check-device-crash-full:
> +	$(CRASH_TEST) $(CRASH_TEST_OPTIONS) -F $(CRASH_TEST_BINARIES)
> +
>  # Consolidated targets
>  
>  .PHONY: check-qapi-schema check-qtest check-unit check check-clean
> @@ -950,7 +961,7 @@ check-qtest: $(patsubst %,check-qtest-%, $(QTEST_TARGETS))
>  check-unit: $(patsubst %,check-%, $(check-unit-y))
>  check-speed: $(patsubst %,check-%, $(check-speed-y))
>  check-block: $(patsubst %,check-%, $(check-block-y))
> -check: check-qapi-schema check-unit check-qtest check-decodetree
> +check: check-qapi-schema check-unit check-qtest check-decodetree check-device-crash-quick

Making device-crash-test work with Python 3 will take more work
than I have expected, so I won't try to include it on "make
check" yet.

I will send a version that only adds the "check-device-crash-*"
rules, so people can start running "make check-device-crash-quick"
on their scripts if they know the host has Python 2.7 available.

-- 
Eduardo

[Qemu-devel] [PATCH v2 8/8] tests: Add check-device-crash-* rules

Posted by Eduardo Habkost 7 years, 7 months ago

Add check-device-crash-quick and check-device-crash-full rules,
that can be used to run device-crash-test on the built QEMU
binaries.

The plan was to run check-device-crash-quick on "make check",
but qemu.py/qmp.py still don't work with Python 3.

Signed-off-by: Eduardo Habkost <ehabkost@redhat.com>
---
Changes v1 -> v2:
* Don't try to run the script on "make check" yet because it
  won't work if only Python 3 is available

---
 tests/Makefile.include | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/tests/Makefile.include b/tests/Makefile.include
index ef9b88c369..19f5ee73dc 100644
--- a/tests/Makefile.include
+++ b/tests/Makefile.include
@@ -942,6 +942,17 @@ check-decodetree:
           ./check.sh "$(PYTHON)" "$(SRC_PATH)/scripts/decodetree.py", \
           TEST, decodetree.py)
 
+CRASH_TEST = $(SRC_PATH)/scripts/device-crash-test
+CRASH_TEST_OPTIONS = $(if $(V),-v,-q)
+CRASH_TEST_BINARIES = $(foreach TARGET,$(TARGETS), $(TARGET)-softmmu/qemu-system-$(TARGET))
+
+.PHONY: check-device-crash-quick
+check-device-crash-quick:
+	$(CRASH_TEST) $(CRASH_TEST_OPTIONS) -t machine=DEFAULT accel=tcg -- $(CRASH_TEST_BINARIES)
+
+check-device-crash-full:
+	$(CRASH_TEST) $(CRASH_TEST_OPTIONS) -F $(CRASH_TEST_BINARIES)
+
 # Consolidated targets
 
 .PHONY: check-qapi-schema check-qtest check-unit check check-clean
-- 
2.14.3