Bumping avocado to version 101 has two issues.  First, there are problems
where Avocado is not logging of command lines or terminal output, and not
collecting Python logs outside the avocado namespace.

Second, the recent changes to Python handling mean that there is a single
virtual environment for all the build, instead of a separate one for testing.
Requiring a too-new version of avocado causes conflicts with any avocado
plugins installed on the host:

   $ make check-venv
   make[1]: Entering directory '/home/berrange/src/virt/qemu/build'
     GIT     ui/keycodemapdb tests/fp/berkeley-testfloat-3 tests/fp/berkeley-softfloat-3 dtc
     VENVPIP install -e /home/berrange/src/virt/qemu/python/
     VENVPIP install -r /home/berrange/src/virt/qemu/tests/requirements.txt
   ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
   avocado-framework-plugin-varianter-yaml-to-mux 98.0 requires avocado-framework==98.0, but you have avocado-framework 101.0 which is incompatible.
   avocado-framework-plugin-result-html 98.0 requires avocado-framework==98.0, but you have avocado-framework 101.0 which is incompatible.
   make[1]: Leaving directory '/home/berrange/src/virt/qemu/build'

To avoid this issue, tests/requirements.txt should use a ">=" constraint
and the version of Avocado should be limited to what distros provide
in the system packages.  Only Fedora has Avocado, and more specifically
version 92.0.  For now, this series reverts to the older requirement
(version >=88.1) while leaving further version bumps to future changes.

Paolo

Paolo Bonzini (2):
  Revert "tests/requirements.txt: bump up avocado-framework version to
    101.0"
  tests: update avocado installation to use mkvenv

 python/tests/minreqs.txt |  3 ++-
 tests/Makefile.include   | 26 +++++++++++++-------------
 tests/requirements.txt   |  9 ---------
 3 files changed, 15 insertions(+), 23 deletions(-)
 delete mode 100644 tests/requirements.txt

-- 
2.40.1

On Mon, 5 Jun 2023 at 08:58, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> Bumping avocado to version 101 has two issues.  First, there are problems
> where Avocado is not logging of command lines or terminal output, and not
> collecting Python logs outside the avocado namespace.
>
> Second, the recent changes to Python handling mean that there is a single
> virtual environment for all the build, instead of a separate one for testing.
> Requiring a too-new version of avocado causes conflicts with any avocado
> plugins installed on the host:
>
>    $ make check-venv
>    make[1]: Entering directory '/home/berrange/src/virt/qemu/build'
>      GIT     ui/keycodemapdb tests/fp/berkeley-testfloat-3 tests/fp/berkeley-softfloat-3 dtc
>      VENVPIP install -e /home/berrange/src/virt/qemu/python/
>      VENVPIP install -r /home/berrange/src/virt/qemu/tests/requirements.txt
>    ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
>    avocado-framework-plugin-varianter-yaml-to-mux 98.0 requires avocado-framework==98.0, but you have avocado-framework 101.0 which is incompatible.
>    avocado-framework-plugin-result-html 98.0 requires avocado-framework==98.0, but you have avocado-framework 101.0 which is incompatible.
>    make[1]: Leaving directory '/home/berrange/src/virt/qemu/build'
>
> To avoid this issue, tests/requirements.txt should use a ">=" constraint
> and the version of Avocado should be limited to what distros provide
> in the system packages.  Only Fedora has Avocado, and more specifically
> version 92.0.  For now, this series reverts to the older requirement
> (version >=88.1) while leaving further version bumps to future changes.

If the new Avocado version is broken, don't we also need a < constraint
so we don't get it by mistake ?

In particular, for a local build tree that currently has 101 installed,
if the tree is updated to include these two patches together, will that
correctly downgrade it to 88.1?

thanks
-- PMM

On 6/5/23 11:46, Peter Maydell wrote:
>> To avoid this issue, tests/requirements.txt should use a ">=" constraint
>> and the version of Avocado should be limited to what distros provide
>> in the system packages.  Only Fedora has Avocado, and more specifically
>> version 92.0.  For now, this series reverts to the older requirement
>> (version >=88.1) while leaving further version bumps to future changes.
> If the new Avocado version is broken, don't we also need a < constraint
> so we don't get it by mistake ?

I expected those to be bugs that get fixed in 102 or 101.1, so not a 
reason to impose a strict constraint.  But you're right, the version 
that would be installed from PyPI is the latest; I didn't notice because 
I do have avocado installed outside pyvenv/.

Is the logging issue limited to the one fixed by 
https://www.mail-archive.com/qemu-devel@nongnu.org/msg962758.html?  Or 
is there something more?

> In particular, for a local build tree that currently has 101 installed,
> if the tree is updated to include these two patches together, will that
> correctly downgrade it to 88.1?

No, it won't.  What you can do is "pyvenv/bin/pip uninstall 
avocado-framework".

Paolo

On Mon, 5 Jun 2023 at 11:51, Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> On 6/5/23 11:46, Peter Maydell wrote:
> >> To avoid this issue, tests/requirements.txt should use a ">=" constraint
> >> and the version of Avocado should be limited to what distros provide
> >> in the system packages.  Only Fedora has Avocado, and more specifically
> >> version 92.0.  For now, this series reverts to the older requirement
> >> (version >=88.1) while leaving further version bumps to future changes.
> > If the new Avocado version is broken, don't we also need a < constraint
> > so we don't get it by mistake ?
>
> I expected those to be bugs that get fixed in 102 or 101.1, so not a
> reason to impose a strict constraint.  But you're right, the version
> that would be installed from PyPI is the latest; I didn't notice because
> I do have avocado installed outside pyvenv/.
>
> Is the logging issue limited to the one fixed by
> https://www.mail-archive.com/qemu-devel@nongnu.org/msg962758.html?  Or
> is there something more?

I don't know, as I haven't tested with that patch.

> > In particular, for a local build tree that currently has 101 installed,
> > if the tree is updated to include these two patches together, will that
> > correctly downgrade it to 88.1?
>
> No, it won't.  What you can do is "pyvenv/bin/pip uninstall
> avocado-framework".

I think if we're going to revert back from Avocado 101 we should
actively do that, not merely allow older versions.

-- PMM

On 6/5/23 12:58, Peter Maydell wrote:
> On Mon, 5 Jun 2023 at 11:51, Paolo Bonzini <pbonzini@redhat.com> wrote:
>>
>> On 6/5/23 11:46, Peter Maydell wrote:
>>>> To avoid this issue, tests/requirements.txt should use a ">=" constraint
>>>> and the version of Avocado should be limited to what distros provide
>>>> in the system packages.  Only Fedora has Avocado, and more specifically
>>>> version 92.0.  For now, this series reverts to the older requirement
>>>> (version >=88.1) while leaving further version bumps to future changes.
>>> If the new Avocado version is broken, don't we also need a < constraint
>>> so we don't get it by mistake ?
>>
>> I expected those to be bugs that get fixed in 102 or 101.1, so not a
>> reason to impose a strict constraint.  But you're right, the version
>> that would be installed from PyPI is the latest; I didn't notice because
>> I do have avocado installed outside pyvenv/.
>>
>> Is the logging issue limited to the one fixed by
>> https://www.mail-archive.com/qemu-devel@nongnu.org/msg962758.html?  Or
>> is there something more?
> 
> I don't know, as I haven't tested with that patch.
> 
>>> In particular, for a local build tree that currently has 101 installed,
>>> if the tree is updated to include these two patches together, will that
>>> correctly downgrade it to 88.1?
>>
>> No, it won't.  What you can do is "pyvenv/bin/pip uninstall
>> avocado-framework".
> 
> I think if we're going to revert back from Avocado 101 we should
> actively do that, not merely allow older versions.

I think for now I can revert 9c6692db550f739a84fe4b677428df09d9fafdc3 
too (which will have the side effect of reverting to the older version) 
together with the first patch.

Paolo