[PATCH v2 1/2] tests/functional: Set current time stamp of assets when using them

Thomas Huth posted 2 patches 1 month ago
Maintainers: John Snow <jsnow@redhat.com>, Cleber Rosa <crosa@redhat.com>, Thomas Huth <thuth@redhat.com>, "Philippe Mathieu-Daudé" <philmd@linaro.org>, "Daniel P. Berrangé" <berrange@redhat.com>, Paolo Bonzini <pbonzini@redhat.com>, "Alex Bennée" <alex.bennee@linaro.org>
There is a newer version of this series
[PATCH v2 1/2] tests/functional: Set current time stamp of assets when using them
Posted by Thomas Huth 1 month ago
From: Thomas Huth <thuth@redhat.com>

We are going to remove obsolete assets from the cache, so keep
the time stamps of the assets that we use up-to-date to have a way
to detect stale assets later.

Signed-off-by: Thomas Huth <thuth@redhat.com>
---
 tests/functional/qemu_test/asset.py | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_test/asset.py
index f666125bfaf..d5c4ad04bb5 100644
--- a/tests/functional/qemu_test/asset.py
+++ b/tests/functional/qemu_test/asset.py
@@ -10,6 +10,7 @@
 import os
 import stat
 import sys
+import time
 import unittest
 import urllib.request
 from time import sleep
@@ -113,6 +114,18 @@ def _wait_for_other_download(self, tmp_cache_file):
         self.log.debug("Time out while waiting for %s!", tmp_cache_file)
         raise
 
+    def _save_time_stamp(self):
+        '''
+        Update the time stamp of the asset in the cache. Unfortunately, we
+        cannot use the modification or access time of the asset file itself,
+        since e.g. the functional jobs in the gitlab CI reload the files
+        from the gitlab cache and thus always have recent file time stamps,
+        so we have to save our asset time stamp to a separate file instead.
+        '''
+        with open(self.cache_file.with_suffix(".stamp"), 'w',
+                  encoding='utf-8') as fh:
+            fh.write(f"{int(time.time())}")
+
     def fetch(self):
         if not self.cache_dir.exists():
             self.cache_dir.mkdir(parents=True, exist_ok=True)
@@ -120,6 +133,7 @@ def fetch(self):
         if self.valid():
             self.log.debug("Using cached asset %s for %s",
                            self.cache_file, self.url)
+            self._save_time_stamp()
             return str(self.cache_file)
 
         if not self.fetchable():
@@ -208,6 +222,7 @@ def fetch(self):
             tmp_cache_file.unlink()
             raise AssetError(self, "Hash does not match %s" % self.hash)
         tmp_cache_file.replace(self.cache_file)
+        self._save_time_stamp()
         # Remove write perms to stop tests accidentally modifying them
         os.chmod(self.cache_file, stat.S_IRUSR | stat.S_IRGRP)
 
-- 
2.51.0
Re: [PATCH v2 1/2] tests/functional: Set current time stamp of assets when using them
Posted by Daniel P. Berrangé 1 month ago
On Mon, Oct 13, 2025 at 02:17:18PM +0200, Thomas Huth wrote:
> From: Thomas Huth <thuth@redhat.com>
> 
> We are going to remove obsolete assets from the cache, so keep
> the time stamps of the assets that we use up-to-date to have a way
> to detect stale assets later.
> 
> Signed-off-by: Thomas Huth <thuth@redhat.com>
> ---
>  tests/functional/qemu_test/asset.py | 15 +++++++++++++++
>  1 file changed, 15 insertions(+)
> 
> diff --git a/tests/functional/qemu_test/asset.py b/tests/functional/qemu_test/asset.py
> index f666125bfaf..d5c4ad04bb5 100644
> --- a/tests/functional/qemu_test/asset.py
> +++ b/tests/functional/qemu_test/asset.py
> @@ -10,6 +10,7 @@
>  import os
>  import stat
>  import sys
> +import time
>  import unittest
>  import urllib.request
>  from time import sleep
> @@ -113,6 +114,18 @@ def _wait_for_other_download(self, tmp_cache_file):
>          self.log.debug("Time out while waiting for %s!", tmp_cache_file)
>          raise
>  
> +    def _save_time_stamp(self):
> +        '''
> +        Update the time stamp of the asset in the cache. Unfortunately, we
> +        cannot use the modification or access time of the asset file itself,
> +        since e.g. the functional jobs in the gitlab CI reload the files
> +        from the gitlab cache and thus always have recent file time stamps,
> +        so we have to save our asset time stamp to a separate file instead.
> +        '''
> +        with open(self.cache_file.with_suffix(".stamp"), 'w',
> +                  encoding='utf-8') as fh:
> +            fh.write(f"{int(time.time())}")

Realized we can simplify that too

   self.cache_file.with_suffix(".stamp").write_text(f"{int(time.time())}")

The 'encoding' arg is redundant because we're only writing an integer
time value which is identical in every encoding.

With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|