[libvirt PATCH 0/3] tests: virtimetest: Skip more tests near year's end

Andrea Bolognani posted 3 patches 2 years, 2 months ago
Test syntax-check failed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/libvirt tags/patchew/20220101190726.30178-1-abologna@redhat.com
tests/virtimetest.c | 58 ++++++++++++++++++++++-----------------------
1 file changed, 28 insertions(+), 30 deletions(-)
[libvirt PATCH 0/3] tests: virtimetest: Skip more tests near year's end
Posted by Andrea Bolognani 2 years, 2 months ago
Failed pipeline: https://gitlab.com/libvirt/libvirt/-/pipelines/439750804
Fixed pipeline: https://gitlab.com/abologna/libvirt/-/pipelines/439847251

Andrea Bolognani (3):
  tests: virtimetest: Move comment
  tests: virtimetest: Skip more tests near year's end
  tests: virtimetest: Mention GLib version containing fix

 tests/virtimetest.c | 58 ++++++++++++++++++++++-----------------------
 1 file changed, 28 insertions(+), 30 deletions(-)

-- 
2.31.1


Re: [libvirt PATCH 0/3] tests: virtimetest: Skip more tests near year's end
Posted by Laine Stump 2 years, 2 months ago
On 1/1/22 2:07 PM, Andrea Bolognani wrote:
> Failed pipeline: https://gitlab.com/libvirt/libvirt/-/pipelines/439750804
> Fixed pipeline: https://gitlab.com/abologna/libvirt/-/pipelines/439847251
> 
> Andrea Bolognani (3):
>    tests: virtimetest: Move comment
>    tests: virtimetest: Skip more tests near year's end

Ugh, I had managed to repress the memories of this, and now it's come 
back :-O

>    tests: virtimetest: Mention GLib version containing fix
> 
>   tests/virtimetest.c | 58 ++++++++++++++++++++++-----------------------
>   1 file changed, 28 insertions(+), 30 deletions(-)
> 

Reviewed-by: Laine Stump <laine@redhat.com>


*if* you can explain to me why (at 11:55PM local time on Jan 1) all 28 
of virtimetest cases were run by "ninja -C build test", with test 27 & 
28 failing, *but* when I just ran build/tests/virtimetest directly, it 
succeeded - running it under gdb I found that isNearYearEnd() properly 
returned true, so the final 8 tests aren't run (as you would expect).

Not only that, but when I ran "ninja -C build test" at 12:01AM Jan 2, 
test 28 failed; it also failed when I directly ran virtimetest.

For the first problem - is it possible that ninja is setting the 
timezone to something different during the build? My machine is in EST 
(UTC -5).

For the 2nd problem - maybe we need to widen the definition of 
"isNearYearEnd()"? If I make the following change then the final tests 
are skipped by both ninja -C build test and virtimetest directly when 
the date is 12;14AM Jan 2. Oh, wait. That solves the 1st problem too :-)

diff --git a/tests/virtimetest.c b/tests/virtimetest.c
index 5d0c0717ca..383a4cfa59 100644
--- a/tests/virtimetest.c
+++ b/tests/virtimetest.c
@@ -104,7 +104,7 @@ isNearYearEnd(void)
      g_autoptr(GDateTime) now = g_date_time_new_now_local();

      return ((g_date_time_get_month(now) == 1 &&
-             g_date_time_get_day_of_month(now) == 1) ||
+             g_date_time_get_day_of_month(now) <= 2) ||
              (g_date_time_get_month(now) == 12 &&
               g_date_time_get_day_of_month(now) == 31));
  }

(I was just kidding about the "*if*" above - you have my Reviewed-by: in 
any case. But unless you want to spend the time figuring out why my 
tests failed on Jan 1/early Jan 2, I'm going to send the above diff as a 
patch)

Re: [libvirt PATCH 0/3] tests: virtimetest: Skip more tests near year's end
Posted by Andrea Bolognani 2 years, 2 months ago
On Sun, Jan 02, 2022 at 12:17:31AM -0500, Laine Stump wrote:
> On 1/1/22 2:07 PM, Andrea Bolognani wrote:
> >    tests: virtimetest: Skip more tests near year's end
>
> Ugh, I had managed to repress the memories of this, and now it's come back
> :-O

Same :)

> *if* you can explain to me why (at 11:55PM local time on Jan 1) all 28 of
> virtimetest cases were run by "ninja -C build test", with test 27 & 28
> failing, *but* when I just ran build/tests/virtimetest directly, it
> succeeded - running it under gdb I found that isNearYearEnd() properly
> returned true, so the final 8 tests aren't run (as you would expect).
>
> Not only that, but when I ran "ninja -C build test" at 12:01AM Jan 2, test
> 28 failed; it also failed when I directly ran virtimetest.
>
> For the first problem - is it possible that ninja is setting the timezone to
> something different during the build? My machine is in EST (UTC -5).

I don't think that's the case: I hacked virtimetest to print out the
timezone, and I get the same output whether I run it through 'meson
test' (or 'ninja test') or as a standalone program.

While doing so, I also realized something that should probably have
been obvious: since each call to TEST_LOCALOFFSET() sets the TZ
environment variable and invokes tzset(), depending on where in the
test program it is called isNearYearEnd() might return a different
result.

So perhaps we should call isNearYearEnd() once, before the first call
to TEST_LOCALOFFSET(), and use that result to decide which tests to
skip?

Honestly I'm still unclear on the interactions between the data we're
testing and the local timezone of the machine we're running tests on.
I basically shamelessly cargo-culted the existing approach :)

-- 
Andrea Bolognani / Red Hat / Virtualization