Part 3 was supposed to be the refactor of the command parser but since
I wanted to add few tests I've noticed that there's a lot of old cruft
and many tests are skipped.

This series refactors virshtest and optimizes it to run multiple
commands with one virsh instance. Doing this allows us to do more
testing in the same run time allowing us to reduce the amount of
"expensive" tests.

Further down this removes all of the 'shell' test infra which was used
for virsh.

Peter Krempa (35):
  virshtest: Prepare for testing against output files
  virshtest: Allow to test failure of commands
  virshtest: Filter multiple occurences of string to drop in
    testFilterLine
  virshtest: Add support for testing commands read from input file and
    adapt alias tests
  virshtest: Adapt 'echo --split' tests to DO_TEST_SCRIPT
  virshtest: Drop some redundant 'echo' cases
  virshtest: Adapt some 'escaping' tests via DO_TEST_SCRIPT
  vrishtest: Prepare for simpler testing - echo tests
  virshtest: Test against output files ("echo" tests)
  virshtest: Drop string form of expected output data of "echo" tests
  virshtest: Replace list and nodeinfo tests by equivalent
    VIR_TEST_SCRIPT variant
  virshtest: Adapt tests for domain id lookup and state query to
    DO_TEST_SCRIPT
  virshtest: Adapt 'blkiotune' tests to DO_TEST_SCRIPT
  virshtest: Adapt 'iothread' tests to DO_TEST_SCRIPT
  virshtest: Drop support for testing against hardcoded strings
  virshtest: Adapt argument parsing tests from 'virsh-optparse'
  virshtest: Adapt 'snapshot-create-as' arg handling tests from
    'virsh-optparse'
  virshtest: Adapt numeric option arg handling tests from
    'virsh-optparse'
  virshtest: Adapt 'event' option arg handling tests from
    'virsh-optparse'
  virshtest: Reimplement 'virsh-output' test
  qemuxmlconftest: Adapt XMLs from 'virsh-cpuset' and
    'virsh-define-dev-segfault' cases
  virshtest: Adapt 'virsh-vcpupin' test
  virshtest: Adapt 'virsh-int-overflow'
  virshtest: Adapt 'virsh-schedinfo'
  virshtest: Adapt 'virsh-start' and 'virsh-undefine' tests as
    'lifecycle' case
  virsh: Fix '--name' and '--parent' used together in
    '(snapshot|checkpoint)-list' command
  vsh: Allow non-interactive use of 'cd' command
  virshtest: Adapt 'virsh-snapshot' test
  virshtest: Adapt 'virsh-checkpoint' test
  virshtest: Adapt 'virsh-read-bufsiz' and 'virsh-read-non-seekable'
  virshtest: Adapt 'libvirtd-pool'
  tests: Re-implement '(virsh|virt-admin)-self-test' directly in meson
  tests: Reimplement 'libvirtd-fail' case directly in meson
  virshtest: Adapt virsh-uriprecedence test case
  tests: Drop 'test-lib.sh'

 build-aux/syntax-check.mk                     |   6 +-
 docs/manpages/virsh.rst                       |  17 +-
 src/meson.build                               |   5 +
 tests/libvirtd-fail                           |  16 -
 tests/libvirtd-pool                           |  38 -
 tests/meson.build                             |  39 +-
 .../console-compat-crash.x86_64-latest.args   |  46 +
 .../console-compat-crash.x86_64-latest.xml    |  65 ++
 .../console-compat-crash.xml}                 |  36 +-
 .../cpuset-invalid.x86_64-latest.err          |   1 +
 tests/qemuxmlconfdata/cpuset-invalid.xml      |  12 +
 tests/qemuxmlconftest.c                       |   2 +
 tests/test-lib.sh                             | 280 ------
 tests/virsh-checkpoint                        | 178 ----
 tests/virsh-cpuset                            |  46 -
 tests/virsh-int-overflow                      |  18 -
 tests/virsh-optparse                          | 292 ------
 tests/virsh-output                            |  29 -
 tests/virsh-output-commands                   |  94 --
 tests/virsh-output.out                        | 496 ----------
 tests/virsh-read-bufsiz                       |  49 -
 tests/virsh-read-non-seekable                 |  51 -
 tests/virsh-schedinfo                         |  40 -
 tests/virsh-self-test                         |  48 -
 tests/virsh-snapshot                          | 233 -----
 tests/virsh-start                             |  41 -
 tests/virsh-undefine                          |  76 --
 tests/virsh-uriprecedence                     |  97 --
 tests/virsh-vcpupin                           | 100 --
 tests/virshtest.c                             | 878 +++++++-----------
 tests/virshtestdata/argument-assignment.in    |  15 +
 tests/virshtestdata/argument-assignment.out   |  91 ++
 tests/virshtestdata/attach-disk.in            |  79 ++
 tests/virshtestdata/attach-disk.out           | 343 +++++++
 tests/virshtestdata/blkiotune.in              |   3 +
 tests/virshtestdata/blkiotune.out             |  15 +
 tests/virshtestdata/checkpoint-c2.xml         |  41 +
 tests/virshtestdata/checkpoint-c3.xml         |  38 +
 tests/virshtestdata/checkpoint-redefine.out   |  13 +
 tests/virshtestdata/checkpoint.in             |  35 +
 tests/virshtestdata/checkpoint.out            | 133 +++
 tests/virshtestdata/dash-dash-argument-1.out  |   1 +
 tests/virshtestdata/dash-dash-argument-2.out  |   1 +
 tests/virshtestdata/dash-dash-argument-3.out  |   1 +
 tests/virshtestdata/dash-dash-argument-4.out  |   1 +
 tests/virshtestdata/dash-dash-argument-5.out  |   1 +
 tests/virshtestdata/dash-dash-argument-6.out  |   1 +
 tests/virshtestdata/domain-id-overflow.out    |   3 +
 tests/virshtestdata/domain-id.in              |  18 +
 tests/virshtestdata/domain-id.out             |  73 ++
 tests/virshtestdata/echo-alias-argv.out       |   1 +
 tests/virshtestdata/echo-alias.in             |   3 +
 tests/virshtestdata/echo-alias.out            |   3 +
 tests/virshtestdata/echo-escaping-1.out       |   2 +
 tests/virshtestdata/echo-escaping-2.out       |   2 +
 tests/virshtestdata/echo-escaping-3.out       |   2 +
 tests/virshtestdata/echo-escaping.in          |  11 +
 tests/virshtestdata/echo-escaping.out         |  11 +
 tests/virshtestdata/echo-quote-removal-1.out  |   1 +
 tests/virshtestdata/echo-quote-removal-2.out  |   1 +
 tests/virshtestdata/echo-quote-removal-3.out  |   1 +
 tests/virshtestdata/echo-quote-removal-4.out  |   1 +
 tests/virshtestdata/echo-quote-removal-5.out  |   1 +
 tests/virshtestdata/echo-quote-removal-6.out  |   1 +
 tests/virshtestdata/echo-quote-removal-7.out  |   2 +
 tests/virshtestdata/echo-quote-removal-8.out  |   1 +
 tests/virshtestdata/echo-split.in             |   5 +
 tests/virshtestdata/echo-split.out            |  24 +
 tests/virshtestdata/info-custom.in            |   2 +
 tests/virshtestdata/info-custom.out           |  15 +
 tests/virshtestdata/info-default.in           |   2 +
 tests/virshtestdata/info-default.out          |  13 +
 tests/virshtestdata/iothreads.in              |  14 +
 tests/virshtestdata/iothreads.out             |  51 +
 tests/virshtestdata/lifecycle.in              |  23 +
 tests/virshtestdata/lifecycle.out             | 106 +++
 tests/virshtestdata/multiple-commands-1.out   |   2 +
 tests/virshtestdata/multiple-commands-10.out  |   1 +
 tests/virshtestdata/multiple-commands-11.out  |   1 +
 tests/virshtestdata/multiple-commands-12.out  |   1 +
 tests/virshtestdata/multiple-commands-2.out   |   2 +
 tests/virshtestdata/multiple-commands-3.out   |   2 +
 tests/virshtestdata/multiple-commands-4.out   |   2 +
 tests/virshtestdata/multiple-commands-5.out   |   3 +
 tests/virshtestdata/multiple-commands-6.out   |   1 +
 tests/virshtestdata/multiple-commands-7.out   |   2 +
 tests/virshtestdata/multiple-commands-8.out   |   2 +
 tests/virshtestdata/multiple-commands-9.out   |   1 +
 tests/virshtestdata/numeric-parsing-event.in  |  26 +
 tests/virshtestdata/numeric-parsing-event.out |  10 +
 tests/virshtestdata/numeric-parsing.in        |  43 +
 tests/virshtestdata/numeric-parsing.out       |  12 +
 tests/virshtestdata/pool-define-as.out        |  12 +
 tests/virshtestdata/read-big-pipe.out         |   7 +
 .../schedinfo-invalid-argument.out            |   5 +
 tests/virshtestdata/snapshot-create-args.in   |   9 +
 tests/virshtestdata/snapshot-create-args.out  |  82 ++
 tests/virshtestdata/snapshot-redefine.out     |  17 +
 tests/virshtestdata/snapshot-s2.xml           |  43 +
 tests/virshtestdata/snapshot-s3.xml           |  40 +
 tests/virshtestdata/snapshot.in               |  54 ++
 tests/virshtestdata/snapshot.out              | 173 ++++
 .../uriprecedence-LIBVIRT_DEFAULT_URI.out     |   5 +
 ...riprecedence-VIRSH_DEFAULT_CONNECT_URI.out |   5 +
 tests/virshtestdata/uriprecedence-param.out   |   5 +
 .../uriprecedence-xdg-config.out              |   5 +
 .../bad/libvirt/libvirt.conf                  |   1 +
 .../good/libvirt/libvirt.conf                 |   1 +
 tests/virshtestdata/vcpupin.in                |  23 +
 tests/virshtestdata/vcpupin.out               |  26 +
 tests/virt-admin-self-test                    |   1 -
 tools/meson.build                             |   4 +-
 tools/virsh-checkpoint.c                      |  18 +-
 tools/virsh-snapshot.c                        |  18 +-
 tools/vsh.c                                   |   5 -
 115 files changed, 2352 insertions(+), 2852 deletions(-)
 delete mode 100755 tests/libvirtd-fail
 delete mode 100755 tests/libvirtd-pool
 create mode 100644 tests/qemuxmlconfdata/console-compat-crash.x86_64-latest.args
 create mode 100644 tests/qemuxmlconfdata/console-compat-crash.x86_64-latest.xml
 rename tests/{virsh-define-dev-segfault => qemuxmlconfdata/console-compat-crash.xml} (59%)
 mode change 100755 => 100644
 create mode 100644 tests/qemuxmlconfdata/cpuset-invalid.x86_64-latest.err
 create mode 100644 tests/qemuxmlconfdata/cpuset-invalid.xml
 delete mode 100644 tests/test-lib.sh
 delete mode 100755 tests/virsh-checkpoint
 delete mode 100755 tests/virsh-cpuset
 delete mode 100755 tests/virsh-int-overflow
 delete mode 100755 tests/virsh-optparse
 delete mode 100755 tests/virsh-output
 delete mode 100755 tests/virsh-output-commands
 delete mode 100644 tests/virsh-output.out
 delete mode 100755 tests/virsh-read-bufsiz
 delete mode 100755 tests/virsh-read-non-seekable
 delete mode 100755 tests/virsh-schedinfo
 delete mode 100755 tests/virsh-self-test
 delete mode 100755 tests/virsh-snapshot
 delete mode 100755 tests/virsh-start
 delete mode 100755 tests/virsh-undefine
 delete mode 100755 tests/virsh-uriprecedence
 delete mode 100755 tests/virsh-vcpupin
 create mode 100644 tests/virshtestdata/argument-assignment.in
 create mode 100644 tests/virshtestdata/argument-assignment.out
 create mode 100755 tests/virshtestdata/attach-disk.in
 create mode 100644 tests/virshtestdata/attach-disk.out
 create mode 100644 tests/virshtestdata/blkiotune.in
 create mode 100644 tests/virshtestdata/blkiotune.out
 create mode 100644 tests/virshtestdata/checkpoint-c2.xml
 create mode 100644 tests/virshtestdata/checkpoint-c3.xml
 create mode 100644 tests/virshtestdata/checkpoint-redefine.out
 create mode 100755 tests/virshtestdata/checkpoint.in
 create mode 100644 tests/virshtestdata/checkpoint.out
 create mode 100644 tests/virshtestdata/dash-dash-argument-1.out
 create mode 100644 tests/virshtestdata/dash-dash-argument-2.out
 create mode 100644 tests/virshtestdata/dash-dash-argument-3.out
 create mode 100644 tests/virshtestdata/dash-dash-argument-4.out
 create mode 100644 tests/virshtestdata/dash-dash-argument-5.out
 create mode 100644 tests/virshtestdata/dash-dash-argument-6.out
 create mode 100644 tests/virshtestdata/domain-id-overflow.out
 create mode 100644 tests/virshtestdata/domain-id.in
 create mode 100644 tests/virshtestdata/domain-id.out
 create mode 100644 tests/virshtestdata/echo-alias-argv.out
 create mode 100644 tests/virshtestdata/echo-alias.in
 create mode 100644 tests/virshtestdata/echo-alias.out
 create mode 100644 tests/virshtestdata/echo-escaping-1.out
 create mode 100644 tests/virshtestdata/echo-escaping-2.out
 create mode 100644 tests/virshtestdata/echo-escaping-3.out
 create mode 100644 tests/virshtestdata/echo-escaping.in
 create mode 100644 tests/virshtestdata/echo-escaping.out
 create mode 100644 tests/virshtestdata/echo-quote-removal-1.out
 create mode 100644 tests/virshtestdata/echo-quote-removal-2.out
 create mode 100644 tests/virshtestdata/echo-quote-removal-3.out
 create mode 100644 tests/virshtestdata/echo-quote-removal-4.out
 create mode 100644 tests/virshtestdata/echo-quote-removal-5.out
 create mode 100644 tests/virshtestdata/echo-quote-removal-6.out
 create mode 100644 tests/virshtestdata/echo-quote-removal-7.out
 create mode 100644 tests/virshtestdata/echo-quote-removal-8.out
 create mode 100644 tests/virshtestdata/echo-split.in
 create mode 100644 tests/virshtestdata/echo-split.out
 create mode 100644 tests/virshtestdata/info-custom.in
 create mode 100644 tests/virshtestdata/info-custom.out
 create mode 100644 tests/virshtestdata/info-default.in
 create mode 100644 tests/virshtestdata/info-default.out
 create mode 100644 tests/virshtestdata/iothreads.in
 create mode 100644 tests/virshtestdata/iothreads.out
 create mode 100644 tests/virshtestdata/lifecycle.in
 create mode 100644 tests/virshtestdata/lifecycle.out
 create mode 100644 tests/virshtestdata/multiple-commands-1.out
 create mode 100644 tests/virshtestdata/multiple-commands-10.out
 create mode 100644 tests/virshtestdata/multiple-commands-11.out
 create mode 100644 tests/virshtestdata/multiple-commands-12.out
 create mode 100644 tests/virshtestdata/multiple-commands-2.out
 create mode 100644 tests/virshtestdata/multiple-commands-3.out
 create mode 100644 tests/virshtestdata/multiple-commands-4.out
 create mode 100644 tests/virshtestdata/multiple-commands-5.out
 create mode 100644 tests/virshtestdata/multiple-commands-6.out
 create mode 100644 tests/virshtestdata/multiple-commands-7.out
 create mode 100644 tests/virshtestdata/multiple-commands-8.out
 create mode 100644 tests/virshtestdata/multiple-commands-9.out
 create mode 100644 tests/virshtestdata/numeric-parsing-event.in
 create mode 100644 tests/virshtestdata/numeric-parsing-event.out
 create mode 100644 tests/virshtestdata/numeric-parsing.in
 create mode 100644 tests/virshtestdata/numeric-parsing.out
 create mode 100644 tests/virshtestdata/pool-define-as.out
 create mode 100644 tests/virshtestdata/read-big-pipe.out
 create mode 100644 tests/virshtestdata/schedinfo-invalid-argument.out
 create mode 100644 tests/virshtestdata/snapshot-create-args.in
 create mode 100644 tests/virshtestdata/snapshot-create-args.out
 create mode 100644 tests/virshtestdata/snapshot-redefine.out
 create mode 100644 tests/virshtestdata/snapshot-s2.xml
 create mode 100644 tests/virshtestdata/snapshot-s3.xml
 create mode 100755 tests/virshtestdata/snapshot.in
 create mode 100644 tests/virshtestdata/snapshot.out
 create mode 100644 tests/virshtestdata/uriprecedence-LIBVIRT_DEFAULT_URI.out
 create mode 100644 tests/virshtestdata/uriprecedence-VIRSH_DEFAULT_CONNECT_URI.out
 create mode 100644 tests/virshtestdata/uriprecedence-param.out
 create mode 100644 tests/virshtestdata/uriprecedence-xdg-config.out
 create mode 100644 tests/virshtestdata/uriprecedence-xdg/bad/libvirt/libvirt.conf
 create mode 100644 tests/virshtestdata/uriprecedence-xdg/good/libvirt/libvirt.conf
 create mode 100755 tests/virshtestdata/vcpupin.in
 create mode 100644 tests/virshtestdata/vcpupin.out
 delete mode 120000 tests/virt-admin-self-test

-- 
2.44.0
_______________________________________________
Devel mailing list -- devel@lists.libvirt.org
To unsubscribe send an email to devel-leave@lists.libvirt.org

On a Thursday in 2024, Peter Krempa wrote:
>Part 3 was supposed to be the refactor of the command parser but since
>I wanted to add few tests I've noticed that there's a lot of old cruft
>and many tests are skipped.
>
>This series refactors virshtest and optimizes it to run multiple
>commands with one virsh instance. Doing this allows us to do more
>testing in the same run time allowing us to reduce the amount of
>"expensive" tests.
>
>Further down this removes all of the 'shell' test infra which was used
>for virsh.
>
>Peter Krempa (35):
>  virshtest: Prepare for testing against output files
[..]
>  tests: Drop 'test-lib.sh'
>
> build-aux/syntax-check.mk                     |   6 +-
> docs/manpages/virsh.rst                       |  17 +-
> src/meson.build                               |   5 +
[..]
> tools/virsh-snapshot.c                        |  18 +-
> tools/vsh.c                                   |   5 -
> 115 files changed, 2352 insertions(+), 2852 deletions(-)

Reviewed-by: Ján Tomko <jtomko@redhat.com>

Jano
_______________________________________________
Devel mailing list -- devel@lists.libvirt.org
To unsubscribe send an email to devel-leave@lists.libvirt.org