Build initrd format directly digestable by the kernel. Additionally,
include it itself inside as boot/initrd-domU file, for domU booting in
tests.
This should avoid the need to repack tar -> cpio as part of the test
job.
Keep generating initrd.tar.gz as that's still used by older branches.
Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
---
scripts/x86_64-rootfs-alpine.sh | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/scripts/x86_64-rootfs-alpine.sh b/scripts/x86_64-rootfs-alpine.sh
index b70b3a5..13e85fa 100755
--- a/scripts/x86_64-rootfs-alpine.sh
+++ b/scripts/x86_64-rootfs-alpine.sh
@@ -58,3 +58,8 @@ passwd -d "root" root
cd /
tar cvzf "${WORKDIR}/binaries/initrd.tar.gz" \
bin dev etc home init lib mnt opt root sbin usr var
+mkdir boot
+find bin dev etc home init lib mnt opt root sbin usr var |\
+ cpio -o -H newc | gzip > boot/initrd-domU
+find bin boot dev etc home init lib mnt opt root sbin usr var |\
+ cpio -o -H newc | gzip > "${WORKDIR}/binaries/initrd.cpio.gz" \
--
2.48.1
On 07/04/2025 1:31 pm, Marek Marczykowski-Górecki wrote:
> Build initrd format directly digestable by the kernel. Additionally,
> include it itself inside as boot/initrd-domU file, for domU booting in
> tests.
> This should avoid the need to repack tar -> cpio as part of the test
> job.
>
> Keep generating initrd.tar.gz as that's still used by older branches.
>
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> ---
> scripts/x86_64-rootfs-alpine.sh | 5 +++++
> 1 file changed, 5 insertions(+)
>
> diff --git a/scripts/x86_64-rootfs-alpine.sh b/scripts/x86_64-rootfs-alpine.sh
> index b70b3a5..13e85fa 100755
> --- a/scripts/x86_64-rootfs-alpine.sh
> +++ b/scripts/x86_64-rootfs-alpine.sh
> @@ -58,3 +58,8 @@ passwd -d "root" root
> cd /
> tar cvzf "${WORKDIR}/binaries/initrd.tar.gz" \
> bin dev etc home init lib mnt opt root sbin usr var
> +mkdir boot
> +find bin dev etc home init lib mnt opt root sbin usr var |\
> + cpio -o -H newc | gzip > boot/initrd-domU
> +find bin boot dev etc home init lib mnt opt root sbin usr var |\
> + cpio -o -H newc | gzip > "${WORKDIR}/binaries/initrd.cpio.gz" \
So, after looking at this a bit more, a few notes.
Trailing \ needs dropping.
initrd-domU probably ought to be named initrd-domU-base.cpio.gz so it's
clear what it is. These are easy to fix up.
Looking through the current initrd.tar.gz, we've got:
$ ls -lah ./var/cache/apk/
total 1.9M
drwxr-xr-x 2 andrew andrew 4.0K Nov 8 15:12 .
drwxr-xr-x 4 andrew andrew 4.0K Sep 6 2024 ..
-rw-r--r-- 1 andrew andrew 425K Nov 8 15:12 APKINDEX.73fc500b.tar.gz
-rw-r--r-- 1 andrew andrew 1.4M Nov 8 15:12 APKINDEX.eb46142e.tar.gz
which (alone) is 5% of the size of the initrd. I'll submit a patch
separately to drop this.
But doesn't this result in initrd.tar.gz being included in initrd-domU,
and then (doubly) in initrd.cpio.gz ?
~Andrew
On Mon, Apr 07, 2025 at 06:11:29PM +0100, Andrew Cooper wrote:
> On 07/04/2025 1:31 pm, Marek Marczykowski-Górecki wrote:
> > Build initrd format directly digestable by the kernel. Additionally,
> > include it itself inside as boot/initrd-domU file, for domU booting in
> > tests.
> > This should avoid the need to repack tar -> cpio as part of the test
> > job.
> >
> > Keep generating initrd.tar.gz as that's still used by older branches.
> >
> > Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> > ---
> > scripts/x86_64-rootfs-alpine.sh | 5 +++++
> > 1 file changed, 5 insertions(+)
> >
> > diff --git a/scripts/x86_64-rootfs-alpine.sh b/scripts/x86_64-rootfs-alpine.sh
> > index b70b3a5..13e85fa 100755
> > --- a/scripts/x86_64-rootfs-alpine.sh
> > +++ b/scripts/x86_64-rootfs-alpine.sh
> > @@ -58,3 +58,8 @@ passwd -d "root" root
> > cd /
> > tar cvzf "${WORKDIR}/binaries/initrd.tar.gz" \
> > bin dev etc home init lib mnt opt root sbin usr var
> > +mkdir boot
> > +find bin dev etc home init lib mnt opt root sbin usr var |\
> > + cpio -o -H newc | gzip > boot/initrd-domU
> > +find bin boot dev etc home init lib mnt opt root sbin usr var |\
> > + cpio -o -H newc | gzip > "${WORKDIR}/binaries/initrd.cpio.gz" \
>
> So, after looking at this a bit more, a few notes.
>
> Trailing \ needs dropping.
>
> initrd-domU probably ought to be named initrd-domU-base.cpio.gz so it's
> clear what it is. These are easy to fix up.
Technically, it's useful as is already if you just want to check if domU
starts.
> Looking through the current initrd.tar.gz, we've got:
>
> $ ls -lah ./var/cache/apk/
> total 1.9M
> drwxr-xr-x 2 andrew andrew 4.0K Nov 8 15:12 .
> drwxr-xr-x 4 andrew andrew 4.0K Sep 6 2024 ..
> -rw-r--r-- 1 andrew andrew 425K Nov 8 15:12 APKINDEX.73fc500b.tar.gz
> -rw-r--r-- 1 andrew andrew 1.4M Nov 8 15:12 APKINDEX.eb46142e.tar.gz
>
> which (alone) is 5% of the size of the initrd. I'll submit a patch
> separately to drop this.
Makes sense.
> But doesn't this result in initrd.tar.gz being included in initrd-domU,
> and then (doubly) in initrd.cpio.gz ?
No, find (same as tar earlier) has explicit list of dirs to look at,
which don't include initrd.* nor "binaries".
--
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
On 07/04/2025 6:11 pm, Andrew Cooper wrote:
> On 07/04/2025 1:31 pm, Marek Marczykowski-Górecki wrote:
>> Build initrd format directly digestable by the kernel. Additionally,
>> include it itself inside as boot/initrd-domU file, for domU booting in
>> tests.
>> This should avoid the need to repack tar -> cpio as part of the test
>> job.
>>
>> Keep generating initrd.tar.gz as that's still used by older branches.
>>
>> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
>> ---
>> scripts/x86_64-rootfs-alpine.sh | 5 +++++
>> 1 file changed, 5 insertions(+)
>>
>> diff --git a/scripts/x86_64-rootfs-alpine.sh b/scripts/x86_64-rootfs-alpine.sh
>> index b70b3a5..13e85fa 100755
>> --- a/scripts/x86_64-rootfs-alpine.sh
>> +++ b/scripts/x86_64-rootfs-alpine.sh
>> @@ -58,3 +58,8 @@ passwd -d "root" root
>> cd /
>> tar cvzf "${WORKDIR}/binaries/initrd.tar.gz" \
>> bin dev etc home init lib mnt opt root sbin usr var
>> +mkdir boot
>> +find bin dev etc home init lib mnt opt root sbin usr var |\
>> + cpio -o -H newc | gzip > boot/initrd-domU
>> +find bin boot dev etc home init lib mnt opt root sbin usr var |\
>> + cpio -o -H newc | gzip > "${WORKDIR}/binaries/initrd.cpio.gz" \
> So, after looking at this a bit more, a few notes.
>
> Trailing \ needs dropping.
>
> initrd-domU probably ought to be named initrd-domU-base.cpio.gz so it's
> clear what it is. These are easy to fix up.
>
> Looking through the current initrd.tar.gz, we've got:
>
> $ ls -lah ./var/cache/apk/
> total 1.9M
> drwxr-xr-x 2 andrew andrew 4.0K Nov 8 15:12 .
> drwxr-xr-x 4 andrew andrew 4.0K Sep 6 2024 ..
> -rw-r--r-- 1 andrew andrew 425K Nov 8 15:12 APKINDEX.73fc500b.tar.gz
> -rw-r--r-- 1 andrew andrew 1.4M Nov 8 15:12 APKINDEX.eb46142e.tar.gz
>
> which (alone) is 5% of the size of the initrd. I'll submit a patch
> separately to drop this.
>
> But doesn't this result in initrd.tar.gz being included in initrd-domU,
> and then (doubly) in initrd.cpio.gz ?
No, because $WORKDIR is nothing, and we put it in /binaries at the top
level which is excluded.
~Andrew
On 07/04/2025 1:31 pm, Marek Marczykowski-Górecki wrote:
> Build initrd format directly digestable by the kernel. Additionally,
> include it itself inside as boot/initrd-domU file, for domU booting in
> tests.
> This should avoid the need to repack tar -> cpio as part of the test
> job.
>
> Keep generating initrd.tar.gz as that's still used by older branches.
>
> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> scripts/x86_64-rootfs-alpine.sh | 5 +++++
> 1 file changed, 5 insertions(+)
>
> diff --git a/scripts/x86_64-rootfs-alpine.sh b/scripts/x86_64-rootfs-alpine.sh
> index b70b3a5..13e85fa 100755
> --- a/scripts/x86_64-rootfs-alpine.sh
> +++ b/scripts/x86_64-rootfs-alpine.sh
> @@ -58,3 +58,8 @@ passwd -d "root" root
> cd /
> tar cvzf "${WORKDIR}/binaries/initrd.tar.gz" \
> bin dev etc home init lib mnt opt root sbin usr var
What's required to drop the tar version? Presumably merging the rest of
your series?
> +mkdir boot
> +find bin dev etc home init lib mnt opt root sbin usr var |\
> + cpio -o -H newc | gzip > boot/initrd-domU
> +find bin boot dev etc home init lib mnt opt root sbin usr var |\
> + cpio -o -H newc | gzip > "${WORKDIR}/binaries/initrd.cpio.gz" \
On 07/04/2025 5:17 pm, Andrew Cooper wrote:
> On 07/04/2025 1:31 pm, Marek Marczykowski-Górecki wrote:
>> Build initrd format directly digestable by the kernel. Additionally,
>> include it itself inside as boot/initrd-domU file, for domU booting in
>> tests.
>> This should avoid the need to repack tar -> cpio as part of the test
>> job.
>>
>> Keep generating initrd.tar.gz as that's still used by older branches.
>>
>> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
>
>> ---
>> scripts/x86_64-rootfs-alpine.sh | 5 +++++
>> 1 file changed, 5 insertions(+)
>>
>> diff --git a/scripts/x86_64-rootfs-alpine.sh b/scripts/x86_64-rootfs-alpine.sh
>> index b70b3a5..13e85fa 100755
>> --- a/scripts/x86_64-rootfs-alpine.sh
>> +++ b/scripts/x86_64-rootfs-alpine.sh
>> @@ -58,3 +58,8 @@ passwd -d "root" root
>> cd /
>> tar cvzf "${WORKDIR}/binaries/initrd.tar.gz" \
>> bin dev etc home init lib mnt opt root sbin usr var
> What's required to drop the tar version? Presumably merging the rest of
> your series?
NVM, I can't read. I'm tempted to fold in:
#TODO, drop .tar.gz when Xen 4.20 drops out of testing
That said, I expect we'll be backporting these changes.
~Andrew
On Mon, Apr 07, 2025 at 05:22:05PM +0100, Andrew Cooper wrote:
> On 07/04/2025 5:17 pm, Andrew Cooper wrote:
> > On 07/04/2025 1:31 pm, Marek Marczykowski-Górecki wrote:
> >> Build initrd format directly digestable by the kernel. Additionally,
> >> include it itself inside as boot/initrd-domU file, for domU booting in
> >> tests.
> >> This should avoid the need to repack tar -> cpio as part of the test
> >> job.
> >>
> >> Keep generating initrd.tar.gz as that's still used by older branches.
> >>
> >> Signed-off-by: Marek Marczykowski-Górecki <marmarek@invisiblethingslab.com>
> > Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
> >
> >> ---
> >> scripts/x86_64-rootfs-alpine.sh | 5 +++++
> >> 1 file changed, 5 insertions(+)
> >>
> >> diff --git a/scripts/x86_64-rootfs-alpine.sh b/scripts/x86_64-rootfs-alpine.sh
> >> index b70b3a5..13e85fa 100755
> >> --- a/scripts/x86_64-rootfs-alpine.sh
> >> +++ b/scripts/x86_64-rootfs-alpine.sh
> >> @@ -58,3 +58,8 @@ passwd -d "root" root
> >> cd /
> >> tar cvzf "${WORKDIR}/binaries/initrd.tar.gz" \
> >> bin dev etc home init lib mnt opt root sbin usr var
> > What's required to drop the tar version? Presumably merging the rest of
> > your series?
>
> NVM, I can't read. I'm tempted to fold in:
>
> #TODO, drop .tar.gz when Xen 4.20 drops out of testing
>
> That said, I expect we'll be backporting these changes.
There is one more difference: the cpio.gz contains the whole thing
twice. Once as rootfs for dom0 and then another as boot/initrd-domU.
Dropping .tar.gz is probably a good idea at some point, so I'm okay with
such comment added. But I imagine some future tests may benefit from
just one layer, which may want introducing another cpio.gz without
boot/initrd-domU included. In fact, even right now that might be useful
- for example dom0less arm64 test uses just busybox for domU rootfs, not
the whole archive (so with my changes dom0 rootfs has unused
boot/initrd-domU included).
--
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
On Mon, Apr 07, 2025 at 06:31:06PM +0200, Marek Marczykowski-Górecki wrote:
> There is one more difference: the cpio.gz contains the whole thing
> twice. Once as rootfs for dom0 and then another as boot/initrd-domU.
> Dropping .tar.gz is probably a good idea at some point, so I'm okay with
> such comment added. But I imagine some future tests may benefit from
> just one layer, which may want introducing another cpio.gz without
> boot/initrd-domU included. In fact, even right now that might be useful
> - for example dom0less arm64 test uses just busybox for domU rootfs, not
> the whole archive (so with my changes dom0 rootfs has unused
> boot/initrd-domU included).
In such case, would it make sense to prepare several initrd.cpio.gz? A
common one, then having overlays of files we want to add or replace to
the initrd of spefic test. I think osstest used to do something like
that, with command that would be:
cat common.cpio.gz overlay.cpio.gz > initrd.cpio.gz
If it matter, is seem that the "file.cpio" used to be made with
`cpio -Hnewc -o`
Cheers,
--
Anthony Perard | Vates XCP-ng Developer
XCP-ng & Xen Orchestra - Vates solutions
web: https://vates.tech
On Tue, Apr 08, 2025 at 12:06:51PM +0000, Anthony PERARD wrote: > On Mon, Apr 07, 2025 at 06:31:06PM +0200, Marek Marczykowski-Górecki wrote: > > There is one more difference: the cpio.gz contains the whole thing > > twice. Once as rootfs for dom0 and then another as boot/initrd-domU. > > Dropping .tar.gz is probably a good idea at some point, so I'm okay with > > such comment added. But I imagine some future tests may benefit from > > just one layer, which may want introducing another cpio.gz without > > boot/initrd-domU included. In fact, even right now that might be useful > > - for example dom0less arm64 test uses just busybox for domU rootfs, not > > the whole archive (so with my changes dom0 rootfs has unused > > boot/initrd-domU included). > > In such case, would it make sense to prepare several initrd.cpio.gz? A > common one, then having overlays of files we want to add or replace to > the initrd of spefic test. I think osstest used to do something like > that, with command that would be: > cat common.cpio.gz overlay.cpio.gz > initrd.cpio.gz > > If it matter, is seem that the "file.cpio" used to be made with > `cpio -Hnewc -o` This is already how final initrd is constructed with my changes. Having said that, maybe test-artifacts should not include boot/initrd-domU in initrd.cpio.gz initially, but have a separate cpio that has just boot/initrd-domU there. Most tests do use this domU initrd, but since they need to concatenate something anyway, maybe starting with two files instead of one doesn't make much difference? So, it would be: - initrd.cpio.gz - plain rootfs, for dom0 (or domU in dom0less tests) - initrd-in-boot.cpio.gz - the above initrd.cpio.gz packed again as boot/initrd-domU What do you think? -- Best Regards, Marek Marczykowski-Górecki Invisible Things Lab
© 2016 - 2025 Red Hat, Inc.