Other projects have long switched to xz and/or lzip.

Tidy things some as well: With the removal of qemu from the tarball,
intermediately extracting the tarball again has become wasteful. Drop
that. Invoke compressors using asynchronous lists, to reduce overall
latency. Drop the -v option from the (previously implicit) gzip
invocation.

Signed-off-by: Jan Beulich <jbeulich@suse.com>
---
v3: List tarballs using ls at end of script. (Don't alter the rm-s, due
    to lack of feedback there.)
v2: Don't expand intermediate uncompressed tarball. Don't check for
    commands' availablity. Don't request statistics. Use async lists.

--- a/docs/process/release-technician-checklist.txt
+++ b/docs/process/release-technician-checklist.txt
@@ -119,7 +119,7 @@ RELEASE TARBALL
        make src-tarball           # uses git-describe (best for RCs)
         # ^find some way to add git-cache-proxy to this (done in ~iwj/.gitconfig)
        mkdir /volatile/iwj/website-thing/xen.org/oss-xen/release/$v
-       mv dist/xen-$v.tar.gz /volatile/iwj/website-thing/xen.org/oss-xen/release/$v/.
+       mv dist/xen-$v.tar.[glx]z /volatile/iwj/website-thing/xen.org/oss-xen/release/$v/.
 
        # website-thing/xen.org is cvs -d mail.xenproject.org:/home/downloads-cvs/cvs-repos co xen.org
 	cd /volatile/iwj/website-thing/xen.org
@@ -139,9 +139,12 @@ RELEASE TARBALL
 	cvs add -kb oss-xen/release/$v/
 
         cd oss-xen/release/$v
-        gpg --digest-algo=SHA256 --detach-sign -u 'xen tree' xen-$v.tar.gz
-	cvs add -kb xen-$v.tar.gz
-        cvs add -kb xen-$v.tar.gz.sig
+        for t in xen-$v.tar.[glx]z
+        do
+            gpg --digest-algo=SHA256 --detach-sign -u 'xen tree' $t
+            cvs add -kb $t
+            cvs add -kb $t.sig
+        done
         cd ../../..
 
 	cvs ci -m $v
@@ -152,6 +155,10 @@ RELEASE TARBALL
 	# should show something like
 	#   U oss-xen/release/4.8.0-rc2/xen-4.8.0-rc2.tar.gz
 	#   U oss-xen/release/4.8.0-rc2/xen-4.8.0-rc2.tar.gz.sig
+	#   U oss-xen/release/4.8.0-rc2/xen-4.8.0-rc2.tar.lz
+	#   U oss-xen/release/4.8.0-rc2/xen-4.8.0-rc2.tar.lz.sig
+	#   U oss-xen/release/4.8.0-rc2/xen-4.8.0-rc2.tar.xz
+	#   U oss-xen/release/4.8.0-rc2/xen-4.8.0-rc2.tar.xz.sig
 
 After a .0 release, update XEN_EXTRAVERSION again (to .1-pre, see above).
 
--- a/docs/process/xen-release-management.pandoc
+++ b/docs/process/xen-release-management.pandoc
@@ -274,10 +274,10 @@ Xen X.Y rcZ is tagged. You can check tha
 https://xenbits.xen.org/git-http/xen.git X.Y.0-rcZ
 
 For your convenience there is also a tarball at:
-https://downloads.xenproject.org/release/xen/X.Y.0-rcZ/xen-X.Y.0-rcZ.tar.gz
+https://downloads.xenproject.org/release/xen/X.Y.0-rcZ/xen-X.Y.0-rcZ.tar.[glx]z
 
 And the signature is at:
-https://downloads.xenproject.org/release/xen/X.Y.0-rcZ/xen-X.Y.0-rcZ.tar.gz.sig
+https://downloads.xenproject.org/release/xen/X.Y.0-rcZ/xen-X.Y.0-rcZ.tar.[glx]z.sig
 
 Please send bug reports and test reports to xen-devel@lists.xenproject.org.
 When sending bug reports, please CC relevant maintainers and me
--- a/tools/misc/mktarball
+++ b/tools/misc/mktarball
@@ -5,14 +5,6 @@
 # Takes 2 arguments, the path to the dist directory and the version
 set -ex
 
-function git_archive_into {
-    mkdir -p "$2"
-
-    git --git-dir="$1"/.git \
-	archive --format=tar HEAD | \
-	tar Cxf "$2" -
-}
-
 if [[ -z "$1" || -z "$2" ]] ; then
   echo "usage: $0 path-to-XEN_ROOT xen-version"
   exit 1
@@ -21,14 +13,21 @@ fi
 xen_root="$1"
 desc="$2"
 
-tdir="$xen_root/dist/tmp.src-tarball"
+tdir="$xen_root/dist"
 
-rm -rf $tdir
+rm -f $tdir/xen-$desc.tar.[glx]z
 
 mkdir -p $tdir
 
-git_archive_into $xen_root $tdir/xen-$desc
+git --git-dir="$xen_root/.git" archive --format=tar HEAD --prefix=xen-$desc/ \
+    >"$tdir/xen-$desc.tar"
+
+gzip -9k "$tdir/xen-$desc.tar" &
+xz -9k "$tdir/xen-$desc.tar" &
+lzip -9k "$tdir/xen-$desc.tar" &
+wait
 
-GZIP=-9v tar cz -f $xen_root/dist/xen-$desc.tar.gz -C $tdir xen-$desc
+rm -f $tdir/xen-$desc.tar
 
-echo "Source tarball in $xen_root/dist/xen-$desc.tar.gz"
+echo "Source tarball(s):"
+ls -lh $tdir/xen-$desc.tar*

Hi Jan,

On 11/09/2025 09:14, Jan Beulich wrote:
> Other projects have long switched to xz and/or lzip.
> 
> Tidy things some as well: With the removal of qemu from the tarball,
> intermediately extracting the tarball again has become wasteful. Drop
> that. Invoke compressors using asynchronous lists, to reduce overall
> latency. Drop the -v option from the (previously implicit) gzip
> invocation.
> 
> Signed-off-by: Jan Beulich <jbeulich@suse.com>

I have tested manually the steps and the correct tarballs have been 
produced. I will update my scripts to copy & sign all the tarballs once 
this is merged.

Acked-by: Julien Grall <jgrall@amazon.com>
Tested-by: Julien Grall <jgrall@amazon.com>

Is this intended for Xen 4.21?

Cheers,

-- 
Julien Grall

On 12.09.2025 23:23, Julien Grall wrote:
> On 11/09/2025 09:14, Jan Beulich wrote:
>> Other projects have long switched to xz and/or lzip.
>>
>> Tidy things some as well: With the removal of qemu from the tarball,
>> intermediately extracting the tarball again has become wasteful. Drop
>> that. Invoke compressors using asynchronous lists, to reduce overall
>> latency. Drop the -v option from the (previously implicit) gzip
>> invocation.
>>
>> Signed-off-by: Jan Beulich <jbeulich@suse.com>
> 
> I have tested manually the steps and the correct tarballs have been 
> produced. I will update my scripts to copy & sign all the tarballs once 
> this is merged.
> 
> Acked-by: Julien Grall <jgrall@amazon.com>
> Tested-by: Julien Grall <jgrall@amazon.com>

Thanks.

> Is this intended for Xen 4.21?

So far it was, but I'm increasingly unsure, seeing that it still hasn't
gone in. Cc-ing Oleksii too now. Andrew had voiced concern towards the
rm use, but hasn't come back as to his argument towards the uncompressed
tarball previously not having been removed (when I can't see that one
would have been created in the first place), hence why I couldn't make
use of his (conditional) R-b.

Jan

On 9/14/25 3:43 PM, Jan Beulich wrote:
> On 12.09.2025 23:23, Julien Grall wrote:
>> On 11/09/2025 09:14, Jan Beulich wrote:
>>> Other projects have long switched to xz and/or lzip.
>>>
>>> Tidy things some as well: With the removal of qemu from the tarball,
>>> intermediately extracting the tarball again has become wasteful. Drop
>>> that. Invoke compressors using asynchronous lists, to reduce overall
>>> latency. Drop the -v option from the (previously implicit) gzip
>>> invocation.
>>>
>>> Signed-off-by: Jan Beulich<jbeulich@suse.com>
>> I have tested manually the steps and the correct tarballs have been
>> produced. I will update my scripts to copy & sign all the tarballs once
>> this is merged.
>>
>> Acked-by: Julien Grall<jgrall@amazon.com>
>> Tested-by: Julien Grall<jgrall@amazon.com>
> Thanks.
>
>> Is this intended for Xen 4.21?

IMO, it would be nice to have that in Xen 4.21.

> So far it was, but I'm increasingly unsure, seeing that it still hasn't
> gone in. Cc-ing Oleksii too now. Andrew had voiced concern towards the
> rm use, but hasn't come back as to his argument towards the uncompressed
> tarball previously not having been removed (when I can't see that one
> would have been created in the first place), hence why I couldn't make
> use of his (conditional) R-b.

There is not too much sense in the uncompressed tarball. I prefer to not
generate it at all.

Also I have regarding .gz. If other projects switched to xz and/or
lzip (as it is mentioned in the commit message) what is the purpose of
having .gz tarball then?

~ Oleksii

On 14.09.2025 18:59, Oleksii Kurochko wrote:
> 
> On 9/14/25 3:43 PM, Jan Beulich wrote:
>> On 12.09.2025 23:23, Julien Grall wrote:
>>> On 11/09/2025 09:14, Jan Beulich wrote:
>>>> Other projects have long switched to xz and/or lzip.
>>>>
>>>> Tidy things some as well: With the removal of qemu from the tarball,
>>>> intermediately extracting the tarball again has become wasteful. Drop
>>>> that. Invoke compressors using asynchronous lists, to reduce overall
>>>> latency. Drop the -v option from the (previously implicit) gzip
>>>> invocation.
>>>>
>>>> Signed-off-by: Jan Beulich<jbeulich@suse.com>
>>> I have tested manually the steps and the correct tarballs have been
>>> produced. I will update my scripts to copy & sign all the tarballs once
>>> this is merged.
>>>
>>> Acked-by: Julien Grall<jgrall@amazon.com>
>>> Tested-by: Julien Grall<jgrall@amazon.com>
>> Thanks.
>>
>>> Is this intended for Xen 4.21?
> 
> IMO, it would be nice to have that in Xen 4.21.

May I translate this to a release-ack then?

>> So far it was, but I'm increasingly unsure, seeing that it still hasn't
>> gone in. Cc-ing Oleksii too now. Andrew had voiced concern towards the
>> rm use, but hasn't come back as to his argument towards the uncompressed
>> tarball previously not having been removed (when I can't see that one
>> would have been created in the first place), hence why I couldn't make
>> use of his (conditional) R-b.
> 
> There is not too much sense in the uncompressed tarball. I prefer to not
> generate it at all.

Generating is helpful, to do it only once (instead of once per compressed
tarball).

> Also I have regarding .gz. If other projects switched to xz and/or
> lzip (as it is mentioned in the commit message) what is the purpose of
> having .gz tarball then?

At the very least to help people presently assuming they'll find a .gz.

Jan

> On 14 Sep 2025, at 10:57, Jan Beulich <jbeulich@suse.com> wrote:
> 
> On 14.09.2025 18:59, Oleksii Kurochko wrote:
>> 
>>> On 9/14/25 3:43 PM, Jan Beulich wrote:
>>> On 12.09.2025 23:23, Julien Grall wrote:
>>>> On 11/09/2025 09:14, Jan Beulich wrote:
>>>>> Other projects have long switched to xz and/or lzip.
>>>>> 
>>>>> Tidy things some as well: With the removal of qemu from the tarball,
>>>>> intermediately extracting the tarball again has become wasteful. Drop
>>>>> that. Invoke compressors using asynchronous lists, to reduce overall
>>>>> latency. Drop the -v option from the (previously implicit) gzip
>>>>> invocation.
>>>>> 
>>>>> Signed-off-by: Jan Beulich<jbeulich@suse.com>
>>>> I have tested manually the steps and the correct tarballs have been
>>>> produced. I will update my scripts to copy & sign all the tarballs once
>>>> this is merged.
>>>> 
>>>> Acked-by: Julien Grall<jgrall@amazon.com>
>>>> Tested-by: Julien Grall<jgrall@amazon.com>
>>> Thanks.
>>> 
>>>> Is this intended for Xen 4.21?
>> 
>> IMO, it would be nice to have that in Xen 4.21.
> 
> May I translate this to a release-ack then?
Yes.

Release-Acked-by: Oleksii Kurochko <oleksii.kurochko@gmail.com>

~ Oleksii

> 
>>> So far it was, but I'm increasingly unsure, seeing that it still hasn't
>>> gone in. Cc-ing Oleksii too now. Andrew had voiced concern towards the
>>> rm use, but hasn't come back as to his argument towards the uncompressed
>>> tarball previously not having been removed (when I can't see that one
>>> would have been created in the first place), hence why I couldn't make
>>> use of his (conditional) R-b.
>> 
>> There is not too much sense in the uncompressed tarball. I prefer to not
>> generate it at all.
> 
> Generating is helpful, to do it only once (instead of once per compressed
> tarball).
> 
>> Also I have regarding .gz. If other projects switched to xz and/or
>> lzip (as it is mentioned in the commit message) what is the purpose of
>> having .gz tarball then?
> 
> At the very least to help people presently assuming they'll find a .gz.
> 
> Jan