tests/functional: ppc update and download fail fix

[PATCH 1/4] tests/functional: Don't fail the whole test if a pre-cache fetch fails

Posted by Nicholas Piggin 1 year, 1 month ago

If any pre-cache downloads fail, the entire functional test run
is failed.

Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
---
 tests/functional/qemu_test/asset.py | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_test/asset.py
index f126cd5863a..f82b07e7027 100644
--- a/tests/functional/qemu_test/asset.py
+++ b/tests/functional/qemu_test/asset.py
@@ -102,7 +102,8 @@ def fetch(self):
             return str(self.cache_file)
 
         if os.environ.get("QEMU_TEST_NO_DOWNLOAD", False):
-            raise Exception("Asset cache is invalid and downloads disabled")
+            raise Exception("Failed to fetch asset %s, not found in cache "
+                            "and downloads are disabled", self.url)
 
         self.log.info("Downloading %s to %s...", self.url, self.cache_file)
         tmp_cache_file = self.cache_file.with_suffix(".download")
@@ -162,7 +163,13 @@ def precache_test(test):
         for name, asset in vars(test.__class__).items():
             if name.startswith("ASSET_") and type(asset) == Asset:
                 log.info("Attempting to cache '%s'" % asset)
-                asset.fetch()
+                try:
+                    asset.fetch()
+                except:
+                    # Asset pre-cache should not be fatal. An error is printed
+                    # and the test itself will fail if it is unable to get the
+                    # assert.
+                    pass
         log.removeHandler(handler)
 
     def precache_suite(suite):
-- 
2.45.2

Re: [PATCH 1/4] tests/functional: Don't fail the whole test if a pre-cache fetch fails

Posted by Daniel P. Berrangé 1 year, 1 month ago

On Fri, Dec 20, 2024 at 12:46:14PM +1000, Nicholas Piggin wrote:
> If any pre-cache downloads fail, the entire functional test run
> is failed.
> 
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
>  tests/functional/qemu_test/asset.py | 11 +++++++++--
>  1 file changed, 9 insertions(+), 2 deletions(-)
> 
> diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_test/asset.py
> index f126cd5863a..f82b07e7027 100644
> --- a/tests/functional/qemu_test/asset.py
> +++ b/tests/functional/qemu_test/asset.py
> @@ -102,7 +102,8 @@ def fetch(self):
>              return str(self.cache_file)
>  
>          if os.environ.get("QEMU_TEST_NO_DOWNLOAD", False):
> -            raise Exception("Asset cache is invalid and downloads disabled")
> +            raise Exception("Failed to fetch asset %s, not found in cache "
> +                            "and downloads are disabled", self.url)
>  
>          self.log.info("Downloading %s to %s...", self.url, self.cache_file)
>          tmp_cache_file = self.cache_file.with_suffix(".download")
> @@ -162,7 +163,13 @@ def precache_test(test):
>          for name, asset in vars(test.__class__).items():
>              if name.startswith("ASSET_") and type(asset) == Asset:
>                  log.info("Attempting to cache '%s'" % asset)
> -                asset.fetch()
> +                try:
> +                    asset.fetch()
> +                except:
> +                    # Asset pre-cache should not be fatal. An error is printed
> +                    # and the test itself will fail if it is unable to get the
> +                    # assert.
> +                    pass
>          log.removeHandler(handler)
>  
>      def precache_suite(suite):

Asset fetching errors & skipping tests is being handled by a pending
pull request:

  https://lists.nongnu.org/archive/html/qemu-devel/2024-12/msg04334.html
  https://lists.nongnu.org/archive/html/qemu-devel/2024-12/msg04333.html

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

Re: [PATCH 1/4] tests/functional: Don't fail the whole test if a pre-cache fetch fails

Posted by Nicholas Piggin 1 year, 1 month ago

On Fri Dec 20, 2024 at 8:38 PM AEST, Daniel P. Berrangé wrote:
> On Fri, Dec 20, 2024 at 12:46:14PM +1000, Nicholas Piggin wrote:
> > If any pre-cache downloads fail, the entire functional test run
> > is failed.
> > 
> > Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> > ---
> >  tests/functional/qemu_test/asset.py | 11 +++++++++--
> >  1 file changed, 9 insertions(+), 2 deletions(-)
> > 
> > diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_test/asset.py
> > index f126cd5863a..f82b07e7027 100644
> > --- a/tests/functional/qemu_test/asset.py
> > +++ b/tests/functional/qemu_test/asset.py
> > @@ -102,7 +102,8 @@ def fetch(self):
> >              return str(self.cache_file)
> >  
> >          if os.environ.get("QEMU_TEST_NO_DOWNLOAD", False):
> > -            raise Exception("Asset cache is invalid and downloads disabled")
> > +            raise Exception("Failed to fetch asset %s, not found in cache "
> > +                            "and downloads are disabled", self.url)
> >  
> >          self.log.info("Downloading %s to %s...", self.url, self.cache_file)
> >          tmp_cache_file = self.cache_file.with_suffix(".download")
> > @@ -162,7 +163,13 @@ def precache_test(test):
> >          for name, asset in vars(test.__class__).items():
> >              if name.startswith("ASSET_") and type(asset) == Asset:
> >                  log.info("Attempting to cache '%s'" % asset)
> > -                asset.fetch()
> > +                try:
> > +                    asset.fetch()
> > +                except:
> > +                    # Asset pre-cache should not be fatal. An error is printed
> > +                    # and the test itself will fail if it is unable to get the
> > +                    # assert.
> > +                    pass
> >          log.removeHandler(handler)
> >  
> >      def precache_suite(suite):
>
> Asset fetching errors & skipping tests is being handled by a pending
> pull request:
>
>   https://lists.nongnu.org/archive/html/qemu-devel/2024-12/msg04334.html
>   https://lists.nongnu.org/archive/html/qemu-devel/2024-12/msg04333.html

Oh thanks, it's merged now I should have looked more carefully. That
looks good thank you.

Thanks,
Nick

Re: [PATCH 1/4] tests/functional: Don't fail the whole test if a pre-cache fetch fails

Posted by Philippe Mathieu-Daudé 1 year, 1 month ago

+danpb

On 20/12/24 03:46, Nicholas Piggin wrote:
> If any pre-cache downloads fail, the entire functional test run
> is failed.
> 
> Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> ---
>   tests/functional/qemu_test/asset.py | 11 +++++++++--
>   1 file changed, 9 insertions(+), 2 deletions(-)
> 
> diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_test/asset.py
> index f126cd5863a..f82b07e7027 100644
> --- a/tests/functional/qemu_test/asset.py
> +++ b/tests/functional/qemu_test/asset.py
> @@ -102,7 +102,8 @@ def fetch(self):
>               return str(self.cache_file)
>   
>           if os.environ.get("QEMU_TEST_NO_DOWNLOAD", False):
> -            raise Exception("Asset cache is invalid and downloads disabled")
> +            raise Exception("Failed to fetch asset %s, not found in cache "
> +                            "and downloads are disabled", self.url)
>   
>           self.log.info("Downloading %s to %s...", self.url, self.cache_file)
>           tmp_cache_file = self.cache_file.with_suffix(".download")
> @@ -162,7 +163,13 @@ def precache_test(test):
>           for name, asset in vars(test.__class__).items():
>               if name.startswith("ASSET_") and type(asset) == Asset:
>                   log.info("Attempting to cache '%s'" % asset)
> -                asset.fetch()
> +                try:
> +                    asset.fetch()
> +                except:
> +                    # Asset pre-cache should not be fatal. An error is printed
> +                    # and the test itself will fail if it is unable to get the
> +                    # assert.
> +                    pass
>           log.removeHandler(handler)
>   
>       def precache_suite(suite):