Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---

Hi Paolo, I've used "should_fail: true" because from the docs that
seems like the proper thing, but that produces

28/39 qemu:decodetree / err_width1  EXPECTEDFAIL  0.08s  exit status 1

which isn't nearly so reassuring as OK.  I wonder if it would be better
to add a --test-fail switch to decodetree.py, to invert the exit value?

r~

---
 tests/decode/check.sh    | 24 ----------------
 tests/decode/meson.build | 60 ++++++++++++++++++++++++++++++++++++++++
 tests/meson.build        |  5 +---
 3 files changed, 61 insertions(+), 28 deletions(-)
 delete mode 100755 tests/decode/check.sh
 create mode 100644 tests/decode/meson.build

diff --git a/tests/decode/check.sh b/tests/decode/check.sh
deleted file mode 100755
index 95445a0115..0000000000
--- a/tests/decode/check.sh
+++ /dev/null
@@ -1,24 +0,0 @@
-#!/bin/sh
-# This work is licensed under the terms of the GNU LGPL, version 2 or later.
-# See the COPYING.LIB file in the top-level directory.
-
-PYTHON=$1
-DECODETREE=$2
-E=0
-
-# All of these tests should produce errors
-for i in err_*.decode; do
-    if $PYTHON $DECODETREE $i > /dev/null 2> /dev/null; then
-        # Pass, aka failed to fail.
-        echo FAIL: $i 1>&2
-        E=1
-    fi
-done
-
-for i in succ_*.decode; do
-    if ! $PYTHON $DECODETREE $i > /dev/null 2> /dev/null; then
-        echo FAIL:$i 1>&2
-    fi
-done
-
-exit $E
diff --git a/tests/decode/meson.build b/tests/decode/meson.build
new file mode 100644
index 0000000000..e2e7ccef56
--- /dev/null
+++ b/tests/decode/meson.build
@@ -0,0 +1,60 @@
+err_tests = [
+    'err_argset1.decode',
+    'err_argset2.decode',
+    'err_field1.decode',
+    'err_field2.decode',
+    'err_field3.decode',
+    'err_field4.decode',
+    'err_field5.decode',
+    'err_field6.decode',
+    'err_init1.decode',
+    'err_init2.decode',
+    'err_init3.decode',
+    'err_init4.decode',
+    'err_overlap1.decode',
+    'err_overlap2.decode',
+    'err_overlap3.decode',
+    'err_overlap4.decode',
+    'err_overlap5.decode',
+    'err_overlap6.decode',
+    'err_overlap7.decode',
+    'err_overlap8.decode',
+    'err_overlap9.decode',
+    'err_pattern_group_empty.decode',
+    'err_pattern_group_ident1.decode',
+    'err_pattern_group_ident2.decode',
+    'err_pattern_group_nest1.decode',
+    'err_pattern_group_nest2.decode',
+    'err_pattern_group_nest3.decode',
+    'err_pattern_group_overlap1.decode',
+    'err_width1.decode',
+    'err_width2.decode',
+    'err_width3.decode',
+    'err_width4.decode',
+]
+
+succ_tests = [
+    'succ_argset_type1.decode',
+    'succ_function.decode',
+    'succ_ident1.decode',
+    'succ_pattern_group_nest1.decode',
+    'succ_pattern_group_nest2.decode',
+    'succ_pattern_group_nest3.decode',
+    'succ_pattern_group_nest4.decode',
+]
+
+suite = 'decodetree'
+
+decodetree = find_program(meson.project_source_root() / 'scripts/decodetree.py')
+
+foreach t: err_tests
+    test(fs.replace_suffix(t, ''),
+         decodetree, args: ['-o', '/dev/null', files(t)],
+         should_fail: true, suite: suite)
+endforeach
+
+foreach t: succ_tests
+    test(fs.replace_suffix(t, ''),
+         decodetree, args: ['-o', '/dev/null', files(t)],
+         suite: suite)
+endforeach
diff --git a/tests/meson.build b/tests/meson.build
index 8e318ec513..083f2990bd 100644
--- a/tests/meson.build
+++ b/tests/meson.build
@@ -74,10 +74,7 @@ if have_tools and have_vhost_user and 'CONFIG_LINUX' in config_host
              dependencies: [qemuutil, vhost_user])
 endif
 
-test('decodetree', sh,
-     args: [ files('decode/check.sh'), config_host['PYTHON'], files('../scripts/decodetree.py') ],
-     workdir: meson.current_source_dir() / 'decode',
-     suite: 'decodetree')
+subdir('decode')
 
 if 'CONFIG_TCG' in config_all
   subdir('fp')
-- 
2.34.1

On 5/23/23 19:48, Richard Henderson wrote:
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
> 
> Hi Paolo, I've used "should_fail: true" because from the docs that
> seems like the proper thing, but that produces
> 
> 28/39 qemu:decodetree / err_width1  EXPECTEDFAIL  0.08s  exit status 1
> 
> which isn't nearly so reassuring as OK.  I wonder if it would be better
> to add a --test-fail switch to decodetree.py, to invert the exit value?

Yeah, the documentation is incorrect and it would need a
success_exitcode argument or something like that.  Maybe it's easier to
change the shell script to emit TAP, and use

   protocol: 'tap', verbose: true

in the test() invocation?

Should be as easy as

--- a/tests/decode/check.sh
+++ b/tests/decode/check.sh
@@ -1,24 +1,29 @@
-#!/bin/sh
+#!/bin/bash
  # This work is licensed under the terms of the GNU LGPL, version 2 or later.
  # See the COPYING.LIB file in the top-level directory.
  
  PYTHON=$1
  DECODETREE=$2
-E=0
+j=0
  
  # All of these tests should produce errors
  for i in err_*.decode; do
+    let j++
      if $PYTHON $DECODETREE $i > /dev/null 2> /dev/null; then
          # Pass, aka failed to fail.
-        echo FAIL: $i 1>&2
-        E=1
+        echo not ok $j $i
+    else
+        echo ok $j $i
      fi
  done
  
  for i in succ_*.decode; do
-    if ! $PYTHON $DECODETREE $i > /dev/null 2> /dev/null; then
-        echo FAIL:$i 1>&2
+    let j++
+    if $PYTHON $DECODETREE $i > /dev/null 2> /dev/null; then
+        echo ok $j $i
+    else
+        echo not ok $j $i
      fi
  done
  
-exit $E
+echo 1..$j