Patch series available in this git branch:
https://xenbits.xen.org/git-http/people/aperard/osstest.git br.bookworm-v1

Hi,

I intend to push this series in two waves.

First, push up to commit "Temporally switch "qemu-mainline" branch to
Bookworm". This is to test that osstest still works fine if we need to use
"buster" for a branch. Also upstream QEMU doesn't build on buster anymore, so
I've included a commit to use bookworm for it.

Second, push the remaning two patches, at least a week later, which will switch
the default debian suite.

I did run many tests, but hopefully nothing breaks.

Cheers,

Anthony PERARD (36):
  production-config: Add bookworm debian install media filename
  ts-xen-build-prep: Only force git protocol v2 on buster
  mgi-common: Fix fetch_debian_package error message
  mg-debian-installer-update: Download non-free firmware from new repo.
  ts-host-install: fix ntp.conf path on bookworm
  ts-host-install: fix ntp server setting
  ts-host-install: Restart ntp service
  preseed_create: Use new "d-i grub-installer/update-nvram" for UEFI
    installation
  preseed_create: osstest-erase-other-disks: workaround creating
    /dev/sdXD files
  preseed_create: Workaround fail grub-install on arndale
  ts-host-install,preseed_create: Do lvm vgextend at install time
  di_installcmdline_core: Add link_wait_timeout to install cmdline
  Disable persistent net generator on Bookworm
  preseed_base, ts-host-install: Change NIC NamePolicy to "mac"
  ts-xen-build-prep: Change package selection for Bookworm
  bl_getmenu_open: Read grub.cfg as root
  target_editfile: Use the same user to retrieve and send
  ts-xen-install: remove "libc6-xen" package installation
  overlay-bookworm: Import grub's 20_linux_xen from Debian Bookworm
  overlay-bookworm: 20_linux_xen: Fix XSM entries generation
  ts-xtf-install: Install python symlink
  setupboot_grub2: Parse arm64 uefi grub verbes
  bookworm: Extend ARM clock workaround
  ts-nested-setup, setup l1 lvm volume groupe in guest
  ts-leak-check: add new name for udevd workers
  ts-debian-hvm-install: Allow udev failure in install media
  ts-debian-fixup: Fix nic names for bookworm
  ts-debian-install: keep avoiding to use pygrub
  ts-debian-hvm-install: Increase min guest ram size
  bookworm: Extend guest bootloader workaround
  ts-debian-*-install: Replace dots in hostnames by dashs
  ts-xen-install: Fix bridge setup, ask to copy MAC addr
  make-flight: Keep using buster for L2 guest in nested tests
  Temporally switch "qemu-mainline" branch to Bookworm
  Switch to Debian Bookworm as default suite
  make-hosts-flight: default to bookworm

 Osstest.pm                               |   2 +-
 Osstest/Debian.pm                        |  92 +++++-
 Osstest/TestSupport.pm                   |   4 +-
 make-flight                              |  16 +-
 make-hosts-flight                        |   2 +-
 mfi-common                               |   5 +-
 mg-debian-installer-update               |   9 +-
 mgi-common                               |   2 +-
 overlay-bookworm/etc/grub.d/20_linux_xen | 379 +++++++++++++++++++++++
 production-config                        |   2 +
 ts-debian-di-install                     |   4 +-
 ts-debian-fixup                          |  15 +
 ts-debian-hvm-install                    |  21 +-
 ts-debian-install                        |  17 +-
 ts-host-install                          |  42 ++-
 ts-leak-check                            |   1 +
 ts-nested-setup                          |  10 +-
 ts-xen-build-prep                        |  10 +-
 ts-xen-install                           |   5 +-
 ts-xtf-install                           |   6 +
 20 files changed, 591 insertions(+), 53 deletions(-)
 create mode 100755 overlay-bookworm/etc/grub.d/20_linux_xen

-- 
Anthony PERARD

On Mon, Mar 18, 2024 at 04:55:09PM +0000, Anthony PERARD wrote:
> I intend to push this series in two waves.
> 
> First, push up to commit "Temporally switch "qemu-mainline" branch to
> Bookworm". This is to test that osstest still works fine if we need to use
> "buster" for a branch. Also upstream QEMU doesn't build on buster anymore, so
> I've included a commit to use bookworm for it.
> 
> Second, push the remaning two patches, at least a week later, which will switch
> the default debian suite.
> 
> Anthony PERARD (36):
>   production-config: Add bookworm debian install media filename
>   ts-xen-build-prep: Only force git protocol v2 on buster
>   mgi-common: Fix fetch_debian_package error message
>   mg-debian-installer-update: Download non-free firmware from new repo.
>   ts-host-install: fix ntp.conf path on bookworm
>   ts-host-install: fix ntp server setting
>   ts-host-install: Restart ntp service
>   preseed_create: Use new "d-i grub-installer/update-nvram" for UEFI
>     installation
>   preseed_create: osstest-erase-other-disks: workaround creating
>     /dev/sdXD files
>   preseed_create: Workaround fail grub-install on arndale
>   ts-host-install,preseed_create: Do lvm vgextend at install time
>   di_installcmdline_core: Add link_wait_timeout to install cmdline
>   Disable persistent net generator on Bookworm
>   preseed_base, ts-host-install: Change NIC NamePolicy to "mac"
>   ts-xen-build-prep: Change package selection for Bookworm
>   bl_getmenu_open: Read grub.cfg as root
>   target_editfile: Use the same user to retrieve and send
>   ts-xen-install: remove "libc6-xen" package installation
>   overlay-bookworm: Import grub's 20_linux_xen from Debian Bookworm
>   overlay-bookworm: 20_linux_xen: Fix XSM entries generation
>   ts-xtf-install: Install python symlink
>   setupboot_grub2: Parse arm64 uefi grub verbes
>   bookworm: Extend ARM clock workaround
>   ts-nested-setup, setup l1 lvm volume groupe in guest
>   ts-leak-check: add new name for udevd workers
>   ts-debian-hvm-install: Allow udev failure in install media
>   ts-debian-fixup: Fix nic names for bookworm
>   ts-debian-install: keep avoiding to use pygrub
>   ts-debian-hvm-install: Increase min guest ram size
>   bookworm: Extend guest bootloader workaround
>   ts-debian-*-install: Replace dots in hostnames by dashs
>   ts-xen-install: Fix bridge setup, ask to copy MAC addr
>   make-flight: Keep using buster for L2 guest in nested tests

I've pushed up to here. I'll push the last two patches later.

>   Temporally switch "qemu-mainline" branch to Bookworm

And I'll drop this one.

>   Switch to Debian Bookworm as default suite
>   make-hosts-flight: default to bookworm


Cheers,

-- 
Anthony PERARD

On Thu, Mar 28, 2024 at 05:54:04PM +0000, Anthony PERARD wrote:
> On Mon, Mar 18, 2024 at 04:55:09PM +0000, Anthony PERARD wrote:

I've now pushed
"production-config: Set Bookworm's debian-installer version"

and these two:

> >   Switch to Debian Bookworm as default suite
> >   make-hosts-flight: default to bookworm

osstest should start to use Debian Bookworm soon.

Cheers,

-- 
Anthony PERARD

On Fri, Apr 05, 2024 at 04:54:30PM +0100, Anthony PERARD wrote:
> On Thu, Mar 28, 2024 at 05:54:04PM +0000, Anthony PERARD wrote:
> > On Mon, Mar 18, 2024 at 04:55:09PM +0000, Anthony PERARD wrote:
> 
> I've now pushed
> "production-config: Set Bookworm's debian-installer version"
> 
> and these two:
> 
> > >   Switch to Debian Bookworm as default suite
> > >   make-hosts-flight: default to bookworm
> 
> osstest should start to use Debian Bookworm soon.

osstest failed it's own push gate, I didn't notice that the Arm* builds
was still using an old kernel (from our linux-arm-xen branch) instead of
6.1. So I've rewind this push and pushed an other fix instead:
https://lore.kernel.org/xen-devel/20240408092542.36711-1-anthony.perard@citrix.com/

Cheers,

-- 
Anthony PERARD

On Mon, Apr 08, 2024 at 10:29:08AM +0100, Anthony PERARD wrote:
> On Fri, Apr 05, 2024 at 04:54:30PM +0100, Anthony PERARD wrote:
> > On Thu, Mar 28, 2024 at 05:54:04PM +0000, Anthony PERARD wrote:
> > > On Mon, Mar 18, 2024 at 04:55:09PM +0000, Anthony PERARD wrote:
> > 
> > I've now pushed
> > "production-config: Set Bookworm's debian-installer version"
> > 
> > and these two:
> > 
> > > >   Switch to Debian Bookworm as default suite
> > > >   make-hosts-flight: default to bookworm
> > 
> > osstest should start to use Debian Bookworm soon.
> 
> osstest failed it's own push gate, I didn't notice that the Arm* builds
> was still using an old kernel (from our linux-arm-xen branch) instead of
> 6.1. So I've rewind this push and pushed an other fix instead:
> https://lore.kernel.org/xen-devel/20240408092542.36711-1-anthony.perard@citrix.com/

I've pushed the switch to Debian Bookworm again, and this time it when
through.

osstest have started to use Bookworm for testing.

Cheers,

-- 
Anthony PERARD

On Fri, Apr 19, 2024 at 09:34:21AM +0100, Anthony PERARD wrote:
> On Mon, Apr 08, 2024 at 10:29:08AM +0100, Anthony PERARD wrote:
> > On Fri, Apr 05, 2024 at 04:54:30PM +0100, Anthony PERARD wrote:
> > > On Thu, Mar 28, 2024 at 05:54:04PM +0000, Anthony PERARD wrote:
> > > > On Mon, Mar 18, 2024 at 04:55:09PM +0000, Anthony PERARD wrote:
> > > 
> > > I've now pushed
> > > "production-config: Set Bookworm's debian-installer version"
> > > 
> > > and these two:
> > > 
> > > > >   Switch to Debian Bookworm as default suite
> > > > >   make-hosts-flight: default to bookworm
> > > 
> > > osstest should start to use Debian Bookworm soon.
> > 
> > osstest failed it's own push gate, I didn't notice that the Arm* builds
> > was still using an old kernel (from our linux-arm-xen branch) instead of
> > 6.1. So I've rewind this push and pushed an other fix instead:
> > https://lore.kernel.org/xen-devel/20240408092542.36711-1-anthony.perard@citrix.com/
> 
> I've pushed the switch to Debian Bookworm again, and this time it when
> through.
> 
> osstest have started to use Bookworm for testing.

Thank you very much for doing this.

Roger.

On Mon, Mar 18, 2024 at 04:55:09PM +0000, Anthony PERARD wrote:
> Patch series available in this git branch:
> https://xenbits.xen.org/git-http/people/aperard/osstest.git br.bookworm-v1
> 
> Hi,
> 
> I intend to push this series in two waves.
> 
> First, push up to commit "Temporally switch "qemu-mainline" branch to
> Bookworm". This is to test that osstest still works fine if we need to use
> "buster" for a branch. Also upstream QEMU doesn't build on buster anymore, so
> I've included a commit to use bookworm for it.
> 
> Second, push the remaning two patches, at least a week later, which will switch
> the default debian suite.
> 
> I did run many tests, but hopefully nothing breaks.
> 
> Cheers,
> 
> Anthony PERARD (36):
>   production-config: Add bookworm debian install media filename
>   ts-xen-build-prep: Only force git protocol v2 on buster
>   mgi-common: Fix fetch_debian_package error message
>   mg-debian-installer-update: Download non-free firmware from new repo.
>   ts-host-install: fix ntp.conf path on bookworm
>   ts-host-install: fix ntp server setting
>   ts-host-install: Restart ntp service
>   preseed_create: Use new "d-i grub-installer/update-nvram" for UEFI
>     installation
>   preseed_create: osstest-erase-other-disks: workaround creating
>     /dev/sdXD files
>   preseed_create: Workaround fail grub-install on arndale
>   ts-host-install,preseed_create: Do lvm vgextend at install time
>   di_installcmdline_core: Add link_wait_timeout to install cmdline
>   Disable persistent net generator on Bookworm
>   preseed_base, ts-host-install: Change NIC NamePolicy to "mac"
>   ts-xen-build-prep: Change package selection for Bookworm
>   bl_getmenu_open: Read grub.cfg as root
>   target_editfile: Use the same user to retrieve and send
>   ts-xen-install: remove "libc6-xen" package installation
>   overlay-bookworm: Import grub's 20_linux_xen from Debian Bookworm
>   overlay-bookworm: 20_linux_xen: Fix XSM entries generation
>   ts-xtf-install: Install python symlink
>   setupboot_grub2: Parse arm64 uefi grub verbes
>   bookworm: Extend ARM clock workaround
>   ts-nested-setup, setup l1 lvm volume groupe in guest
>   ts-leak-check: add new name for udevd workers
>   ts-debian-hvm-install: Allow udev failure in install media
>   ts-debian-fixup: Fix nic names for bookworm
>   ts-debian-install: keep avoiding to use pygrub
>   ts-debian-hvm-install: Increase min guest ram size
>   bookworm: Extend guest bootloader workaround
>   ts-debian-*-install: Replace dots in hostnames by dashs
>   ts-xen-install: Fix bridge setup, ask to copy MAC addr
>   make-flight: Keep using buster for L2 guest in nested tests
>   Temporally switch "qemu-mainline" branch to Bookworm
>   Switch to Debian Bookworm as default suite
>   make-hosts-flight: default to bookworm

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

I just had a look over the commit messages mostly, as I'm not very
much into perl as to provide more accurate reviews, sorry.  LGTM, just
some comments/questions on a few patches.

Thanks, Roger.