[PATCH RFC v2 0/1] tests: add wasmtime testsuite

Ilya Leoshkevich posted 1 patch 1 year, 4 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20221221190544.311471-1-iii@linux.ibm.com
Maintainers: Paolo Bonzini <pbonzini@redhat.com>, "Alex Bennée" <alex.bennee@linaro.org>, Thomas Huth <thuth@redhat.com>, "Philippe Mathieu-Daudé" <philmd@linaro.org>, Wainer dos Santos Moschetta <wainersm@redhat.com>, Beraldo Leal <bleal@redhat.com>
Makefile                                     |  1 +
docs/devel/testing.rst                       |  9 +++
tests/Makefile.include                       |  6 ++
tests/third-party/Makefile.include           | 50 ++++++++++++
tests/third-party/wasmtime/Dockerfile        | 32 ++++++++
tests/third-party/wasmtime/Makefile.include  | 49 ++++++++++++
tests/third-party/wasmtime/avocado-wrapper   | 38 +++++++++
tests/third-party/wasmtime/avocado.cfg       |  3 +
tests/third-party/wasmtime/json2tap          | 77 ++++++++++++++++++
tests/third-party/wasmtime/run-tests-wrapper | 82 ++++++++++++++++++++
tests/third-party/wasmtime/test              | 48 ++++++++++++
11 files changed, 395 insertions(+)
create mode 100644 tests/third-party/Makefile.include
create mode 100644 tests/third-party/wasmtime/Dockerfile
create mode 100644 tests/third-party/wasmtime/Makefile.include
create mode 100755 tests/third-party/wasmtime/avocado-wrapper
create mode 100644 tests/third-party/wasmtime/avocado.cfg
create mode 100755 tests/third-party/wasmtime/json2tap
create mode 100755 tests/third-party/wasmtime/run-tests-wrapper
create mode 100755 tests/third-party/wasmtime/test
[PATCH RFC v2 0/1] tests: add wasmtime testsuite
Posted by Ilya Leoshkevich 1 year, 4 months ago
Hi,

I made some updates based on the feedback from Alex.

At the moment it mostly works for me on top of 6394578984da: aarch64,
riscv64 and s390x are clean, but there are some failures on x86_64.
With qemu-user vma rework it unfortunately fails in more places;
I haven't analyzed these failures yet.

v1: https://lists.gnu.org/archive/html/qemu-devel/2022-07/msg00657.html
    https://lists.gnu.org/archive/html/qemu-devel/2022-12/msg02612.html

v1 -> v2:
* Drop gitlab jobs.
* Move to tests/third-party/.
* Use avocado. To be honest, I'm not sure how much value it brings
  here; I hoped that TAPRunner would at least print the number of
  passed tests, but it only checks whether at least one test failed).
* Add various check-* Makefile targets.
* Add json -> TAP 14 conversion.
* Add documentation.
* Move test binaries to the host system.
  This prevents unnecessary full rebuilds of the Docker image.
* Add riscv64; bump Rust and Wasmtime versions.
* Do not use docker.py; unfortunately this leads to reimplementing some
  of its features: docker command detection based on $(ENGINE) and
  injecting the current user into the image.
* Disable core dumps.

Best regards,
Ilya

Ilya Leoshkevich (1):
  tests: add wasmtime testsuite

 Makefile                                     |  1 +
 docs/devel/testing.rst                       |  9 +++
 tests/Makefile.include                       |  6 ++
 tests/third-party/Makefile.include           | 50 ++++++++++++
 tests/third-party/wasmtime/Dockerfile        | 32 ++++++++
 tests/third-party/wasmtime/Makefile.include  | 49 ++++++++++++
 tests/third-party/wasmtime/avocado-wrapper   | 38 +++++++++
 tests/third-party/wasmtime/avocado.cfg       |  3 +
 tests/third-party/wasmtime/json2tap          | 77 ++++++++++++++++++
 tests/third-party/wasmtime/run-tests-wrapper | 82 ++++++++++++++++++++
 tests/third-party/wasmtime/test              | 48 ++++++++++++
 11 files changed, 395 insertions(+)
 create mode 100644 tests/third-party/Makefile.include
 create mode 100644 tests/third-party/wasmtime/Dockerfile
 create mode 100644 tests/third-party/wasmtime/Makefile.include
 create mode 100755 tests/third-party/wasmtime/avocado-wrapper
 create mode 100644 tests/third-party/wasmtime/avocado.cfg
 create mode 100755 tests/third-party/wasmtime/json2tap
 create mode 100755 tests/third-party/wasmtime/run-tests-wrapper
 create mode 100755 tests/third-party/wasmtime/test

-- 
2.38.1
PING: [PATCH RFC v2 0/1] tests: add wasmtime testsuite
Posted by Ilya Leoshkevich 1 year, 2 months ago
On Wed, 2022-12-21 at 20:05 +0100, Ilya Leoshkevich wrote:
> Hi,
> 
> I made some updates based on the feedback from Alex.
> 
> At the moment it mostly works for me on top of 6394578984da: aarch64,
> riscv64 and s390x are clean, but there are some failures on x86_64.
> With qemu-user vma rework it unfortunately fails in more places;
> I haven't analyzed these failures yet.
> 
> v1:
> https://lists.gnu.org/archive/html/qemu-devel/2022-07/msg00657.html
>    
> https://lists.gnu.org/archive/html/qemu-devel/2022-12/msg02612.html
> 
> v1 -> v2:
> * Drop gitlab jobs.
> * Move to tests/third-party/.
> * Use avocado. To be honest, I'm not sure how much value it brings
>   here; I hoped that TAPRunner would at least print the number of
>   passed tests, but it only checks whether at least one test failed).
> * Add various check-* Makefile targets.
> * Add json -> TAP 14 conversion.
> * Add documentation.
> * Move test binaries to the host system.
>   This prevents unnecessary full rebuilds of the Docker image.
> * Add riscv64; bump Rust and Wasmtime versions.
> * Do not use docker.py; unfortunately this leads to reimplementing
> some
>   of its features: docker command detection based on $(ENGINE) and
>   injecting the current user into the image.
> * Disable core dumps.
> 
> Best regards,
> Ilya
> 
> Ilya Leoshkevich (1):
>   tests: add wasmtime testsuite
> 
>  Makefile                                     |  1 +
>  docs/devel/testing.rst                       |  9 +++
>  tests/Makefile.include                       |  6 ++
>  tests/third-party/Makefile.include           | 50 ++++++++++++
>  tests/third-party/wasmtime/Dockerfile        | 32 ++++++++
>  tests/third-party/wasmtime/Makefile.include  | 49 ++++++++++++
>  tests/third-party/wasmtime/avocado-wrapper   | 38 +++++++++
>  tests/third-party/wasmtime/avocado.cfg       |  3 +
>  tests/third-party/wasmtime/json2tap          | 77 ++++++++++++++++++
>  tests/third-party/wasmtime/run-tests-wrapper | 82
> ++++++++++++++++++++
>  tests/third-party/wasmtime/test              | 48 ++++++++++++
>  11 files changed, 395 insertions(+)
>  create mode 100644 tests/third-party/Makefile.include
>  create mode 100644 tests/third-party/wasmtime/Dockerfile
>  create mode 100644 tests/third-party/wasmtime/Makefile.include
>  create mode 100755 tests/third-party/wasmtime/avocado-wrapper
>  create mode 100644 tests/third-party/wasmtime/avocado.cfg
>  create mode 100755 tests/third-party/wasmtime/json2tap
>  create mode 100755 tests/third-party/wasmtime/run-tests-wrapper
>  create mode 100755 tests/third-party/wasmtime/test
> 

Hi,

I would like to ping this patch
(https://patchew.org/QEMU/20221221190544.311471-1-iii@linux.ibm.com/)

Best regards,
Ilya