ci: migration: Don't run python tests in the compat job

[PATCH] ci: migration: Don't run python tests in the compat job

Posted by Fabiano Rosas 2 months, 2 weeks ago

The vmstate-checker-script test has a bug that makes it flaky. It was
also committed by mistake and will be removed.

Since the migration-compat job takes the tests from the build-previous
job instead of the current HEAD, neither a fix or a removal of the
test will take effect for this release.

Disable the faulty/undesirable test by taking advantage that it only
runs if the PYTHON environment variable is set. This also disables the
analyze-migration-script test, but this is fine because that test
doesn't have migration compatibility implications.

Signed-off-by: Fabiano Rosas <farosas@suse.de>
---
 .gitlab-ci.d/buildtest.yml | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index aa32782405..e52456c371 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -212,6 +212,12 @@ build-previous-qemu:
     # testing an old QEMU against new features/tests that it is not
     # compatible with.
     - cd build-previous
+    # Don't allow python-based tests to run. The
+    # vmstate-checker-script test has a race that causes it to fail
+    # sometimes. It cannot be fixed it because this job runs the test
+    # from the old QEMU version. The test will be removed on master,
+    # but this job will only see the change in the next release.
+    - unset PYTHON
     # old to new
     - QTEST_QEMU_BINARY_SRC=./qemu-system-${TARGET}
           QTEST_QEMU_BINARY=../build/qemu-system-${TARGET} ./tests/qtest/migration-test
-- 
2.35.3

Re: [PATCH] ci: migration: Don't run python tests in the compat job

Posted by Peter Xu 2 months, 2 weeks ago

On Thu, Sep 05, 2024 at 03:54:45PM -0300, Fabiano Rosas wrote:
> The vmstate-checker-script test has a bug that makes it flaky. It was
> also committed by mistake and will be removed.
> 
> Since the migration-compat job takes the tests from the build-previous
> job instead of the current HEAD, neither a fix or a removal of the
> test will take effect for this release.
> 
> Disable the faulty/undesirable test by taking advantage that it only
> runs if the PYTHON environment variable is set. This also disables the
> analyze-migration-script test, but this is fine because that test
> doesn't have migration compatibility implications.
> 
> Signed-off-by: Fabiano Rosas <farosas@suse.de>

Reviewed-by: Peter Xu <peterx@redhat.com>

We should still merge your previous pull, right?  Looks like that's the
easiest indeed.

But still, just to double check with both you and Peter on the merge plan.
If that's the case, I can send the 1st 9.2 pull earlier so we can have this
in.

Thanks,

-- 
Peter Xu

Re: [PATCH] ci: migration: Don't run python tests in the compat job

Posted by Fabiano Rosas 2 months, 2 weeks ago

Peter Xu <peterx@redhat.com> writes:

> On Thu, Sep 05, 2024 at 03:54:45PM -0300, Fabiano Rosas wrote:
>> The vmstate-checker-script test has a bug that makes it flaky. It was
>> also committed by mistake and will be removed.
>> 
>> Since the migration-compat job takes the tests from the build-previous
>> job instead of the current HEAD, neither a fix or a removal of the
>> test will take effect for this release.
>> 
>> Disable the faulty/undesirable test by taking advantage that it only
>> runs if the PYTHON environment variable is set. This also disables the
>> analyze-migration-script test, but this is fine because that test
>> doesn't have migration compatibility implications.
>> 
>> Signed-off-by: Fabiano Rosas <farosas@suse.de>
>
> Reviewed-by: Peter Xu <peterx@redhat.com>
>
> We should still merge your previous pull, right?  Looks like that's the
> easiest indeed.

As I mentioned there, that pull is not to blame for this situation, so
my recommendation is to merge. However, there is still the suppression
of the deprecation messages that Peter asked about. I'll send a series
for that in a moment, but it requires qtest changes and probably a lot
of discussion.

>
> But still, just to double check with both you and Peter on the merge plan.
> If that's the case, I can send the 1st 9.2 pull earlier so we can have this
> in.
>
> Thanks,

Re: [PATCH] ci: migration: Don't run python tests in the compat job

Posted by Peter Maydell 2 months, 2 weeks ago

On Thu, 5 Sept 2024 at 20:34, Fabiano Rosas <farosas@suse.de> wrote:
>
> Peter Xu <peterx@redhat.com> writes:
>
> > On Thu, Sep 05, 2024 at 03:54:45PM -0300, Fabiano Rosas wrote:
> >> The vmstate-checker-script test has a bug that makes it flaky. It was
> >> also committed by mistake and will be removed.
> >>
> >> Since the migration-compat job takes the tests from the build-previous
> >> job instead of the current HEAD, neither a fix or a removal of the
> >> test will take effect for this release.
> >>
> >> Disable the faulty/undesirable test by taking advantage that it only
> >> runs if the PYTHON environment variable is set. This also disables the
> >> analyze-migration-script test, but this is fine because that test
> >> doesn't have migration compatibility implications.
> >>
> >> Signed-off-by: Fabiano Rosas <farosas@suse.de>
> >
> > Reviewed-by: Peter Xu <peterx@redhat.com>
> >
> > We should still merge your previous pull, right?  Looks like that's the
> > easiest indeed.
>
> As I mentioned there, that pull is not to blame for this situation, so
> my recommendation is to merge. However, there is still the suppression
> of the deprecation messages that Peter asked about. I'll send a series
> for that in a moment, but it requires qtest changes and probably a lot
> of discussion.

Looking at logs for other things I've merged, those deprecation
messages are already upstream. So I'm OK with giving your
pullreq as it stands another attempt at a merge.

thanks
-- PMM

Re: [PATCH] ci: migration: Don't run python tests in the compat job

Posted by Thomas Huth 2 months, 2 weeks ago

On 05/09/2024 20.54, Fabiano Rosas wrote:
> The vmstate-checker-script test has a bug that makes it flaky. It was
> also committed by mistake and will be removed.
> 
> Since the migration-compat job takes the tests from the build-previous
> job instead of the current HEAD, neither a fix or a removal of the
> test will take effect for this release.
> 
> Disable the faulty/undesirable test by taking advantage that it only
> runs if the PYTHON environment variable is set. This also disables the
> analyze-migration-script test, but this is fine because that test
> doesn't have migration compatibility implications.
> 
> Signed-off-by: Fabiano Rosas <farosas@suse.de>
> ---
>   .gitlab-ci.d/buildtest.yml | 6 ++++++
>   1 file changed, 6 insertions(+)
> 
> diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
> index aa32782405..e52456c371 100644
> --- a/.gitlab-ci.d/buildtest.yml
> +++ b/.gitlab-ci.d/buildtest.yml
> @@ -212,6 +212,12 @@ build-previous-qemu:
>       # testing an old QEMU against new features/tests that it is not
>       # compatible with.
>       - cd build-previous
> +    # Don't allow python-based tests to run. The
> +    # vmstate-checker-script test has a race that causes it to fail
> +    # sometimes. It cannot be fixed it because this job runs the test
> +    # from the old QEMU version. The test will be removed on master,
> +    # but this job will only see the change in the next release.

Maybe explicitly say that this can be removed once 9.2 has been released?

> +    - unset PYTHON
>       # old to new
>       - QTEST_QEMU_BINARY_SRC=./qemu-system-${TARGET}
>             QTEST_QEMU_BINARY=../build/qemu-system-${TARGET} ./tests/qtest/migration-test

Reviewed-by: Thomas Huth <thuth@redhat.com>

Re: [PATCH] ci: migration: Don't run python tests in the compat job

Posted by Peter Xu 2 months, 2 weeks ago

On Thu, Sep 05, 2024 at 09:05:51PM +0200, Thomas Huth wrote:
> On 05/09/2024 20.54, Fabiano Rosas wrote:
> > The vmstate-checker-script test has a bug that makes it flaky. It was
> > also committed by mistake and will be removed.
> > 
> > Since the migration-compat job takes the tests from the build-previous
> > job instead of the current HEAD, neither a fix or a removal of the
> > test will take effect for this release.
> > 
> > Disable the faulty/undesirable test by taking advantage that it only
> > runs if the PYTHON environment variable is set. This also disables the
> > analyze-migration-script test, but this is fine because that test
> > doesn't have migration compatibility implications.
> > 
> > Signed-off-by: Fabiano Rosas <farosas@suse.de>
> > ---
> >   .gitlab-ci.d/buildtest.yml | 6 ++++++
> >   1 file changed, 6 insertions(+)
> > 
> > diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
> > index aa32782405..e52456c371 100644
> > --- a/.gitlab-ci.d/buildtest.yml
> > +++ b/.gitlab-ci.d/buildtest.yml
> > @@ -212,6 +212,12 @@ build-previous-qemu:
> >       # testing an old QEMU against new features/tests that it is not
> >       # compatible with.
> >       - cd build-previous
> > +    # Don't allow python-based tests to run. The
> > +    # vmstate-checker-script test has a race that causes it to fail
> > +    # sometimes. It cannot be fixed it because this job runs the test
> > +    # from the old QEMU version. The test will be removed on master,
> > +    # but this job will only see the change in the next release.
> 
> Maybe explicitly say that this can be removed once 9.2 has been released?

True.  I can touch that up if I'm queuing this.

> 
> > +    - unset PYTHON
> >       # old to new
> >       - QTEST_QEMU_BINARY_SRC=./qemu-system-${TARGET}
> >             QTEST_QEMU_BINARY=../build/qemu-system-${TARGET} ./tests/qtest/migration-test
> 
> Reviewed-by: Thomas Huth <thuth@redhat.com>
> 

-- 
Peter Xu