[RFC PATCH 0/4] Update CentOS and OpenSUSE CI to Python >=3.7

Paolo Bonzini posted 4 patches 1 year, 3 months ago
Failed in applying to current master (apply log)
.gitlab-ci.d/cirrus/freebsd-12.vars           |  8 +-
.gitlab-ci.d/cirrus/freebsd-13.vars           |  8 +-
.gitlab-ci.d/cirrus/macos-12.vars             |  8 +-
scripts/ci/setup/build-environment.yml        |  2 +-
tests/docker/dockerfiles/alpine.docker        |  7 +-
tests/docker/dockerfiles/centos8.docker       | 29 +++---
.../dockerfiles/debian-amd64-cross.docker     |  7 +-
tests/docker/dockerfiles/debian-amd64.docker  |  7 +-
.../dockerfiles/debian-arm64-cross.docker     |  7 +-
.../dockerfiles/debian-armel-cross.docker     |  7 +-
.../dockerfiles/debian-armhf-cross.docker     |  7 +-
.../dockerfiles/debian-mips64el-cross.docker  |  7 +-
.../dockerfiles/debian-mipsel-cross.docker    |  7 +-
.../dockerfiles/debian-ppc64el-cross.docker   |  7 +-
.../dockerfiles/debian-s390x-cross.docker     |  7 +-
.../dockerfiles/fedora-win32-cross.docker     |  7 +-
.../dockerfiles/fedora-win64-cross.docker     |  7 +-
tests/docker/dockerfiles/fedora.docker        |  7 +-
tests/docker/dockerfiles/opensuse-leap.docker | 29 +++---
tests/docker/dockerfiles/ubuntu2004.docker    |  9 +-
tests/lcitool/libvirt-ci                      |  2 +-
tests/lcitool/mappings.yml                    | 80 ++++++++++++++++
tests/lcitool/projects/qemu.yml               |  1 +
tests/lcitool/refresh                         | 96 ++++++++++++-------
tests/lcitool/targets/centos-stream-8.yml     |  3 +
tests/lcitool/targets/opensuse-leap-153.yml   |  3 +
26 files changed, 215 insertions(+), 154 deletions(-)
create mode 100644 tests/lcitool/mappings.yml
create mode 100644 tests/lcitool/targets/centos-stream-8.yml
create mode 100644 tests/lcitool/targets/opensuse-leap-153.yml
[RFC PATCH 0/4] Update CentOS and OpenSUSE CI to Python >=3.7
Posted by Paolo Bonzini 1 year, 3 months ago
QEMU build and test scripts would like to use some new features that
were added to Python 3.7.  Useful additions include removing the need
for OrderedDict, improved handling of default text encoding, postponed
evaluation of type annotations, and improvements to asyncio.

Using new features that were added to Meson 0.63 is also being blocked
by usage of EOL'd versions of Python; Meson 0.63 does not work with the
3.6 version of Python that is provided by CentOS Stream 8 and OpenSUSE
LEAP 15.  Therefore, this series upgrades the CentOS Stream 8 and OpenSUSE
LEAP 15 dockerfiles to use the alternative Python runtimes provided by
the distros.

The main obstacle here is updating the lcitool-generated Dockerfiles;
lcitool is (by design) quite opinionated on the mapping between
"generic" package names and target-specific package names, and it
caters to Libvirt's needs in terms of package versions.

There are two possibilities to change this.

First, lately I have been working on allowing lcitool to be used as
a library.  As of this morning, it is possible for QEMU to use
lcitool's databases and formatters directly from Python, and only
replace the search mechanism to allow mappings and target paths to
be overridden.  This is the approach that is implemented in this RFC:
the first two patches in this series use libvirt-ci as library, while
the third implements the custom search path.

However, I have also proposed the custom search mechanism directly
upstream (https://gitlab.com/libvirt/libvirt-ci/-/merge_requests/341).
If this is accepted, patch 3 will become much smaller and will not
need to know any lcitool internals; in addition, using lcitool as
a library could be evaluated on its own merits because command-line
operation would be a possibility as well.

With the lcitool changes in place, the final patch shows how to
install custom package mappings and update CentOS and OpenSUSE Python
respectively to 3.8 and 3.9.

Thanks,

Paolo

Paolo Bonzini (4):
  lcitool: update submodule
  lcitool: use libvirt-ci as library
  lcitool: allow overriding package mappings and target facts
  ci, docker: update CentOS and OpenSUSE Python to non-EOL versions

 .gitlab-ci.d/cirrus/freebsd-12.vars           |  8 +-
 .gitlab-ci.d/cirrus/freebsd-13.vars           |  8 +-
 .gitlab-ci.d/cirrus/macos-12.vars             |  8 +-
 scripts/ci/setup/build-environment.yml        |  2 +-
 tests/docker/dockerfiles/alpine.docker        |  7 +-
 tests/docker/dockerfiles/centos8.docker       | 29 +++---
 .../dockerfiles/debian-amd64-cross.docker     |  7 +-
 tests/docker/dockerfiles/debian-amd64.docker  |  7 +-
 .../dockerfiles/debian-arm64-cross.docker     |  7 +-
 .../dockerfiles/debian-armel-cross.docker     |  7 +-
 .../dockerfiles/debian-armhf-cross.docker     |  7 +-
 .../dockerfiles/debian-mips64el-cross.docker  |  7 +-
 .../dockerfiles/debian-mipsel-cross.docker    |  7 +-
 .../dockerfiles/debian-ppc64el-cross.docker   |  7 +-
 .../dockerfiles/debian-s390x-cross.docker     |  7 +-
 .../dockerfiles/fedora-win32-cross.docker     |  7 +-
 .../dockerfiles/fedora-win64-cross.docker     |  7 +-
 tests/docker/dockerfiles/fedora.docker        |  7 +-
 tests/docker/dockerfiles/opensuse-leap.docker | 29 +++---
 tests/docker/dockerfiles/ubuntu2004.docker    |  9 +-
 tests/lcitool/libvirt-ci                      |  2 +-
 tests/lcitool/mappings.yml                    | 80 ++++++++++++++++
 tests/lcitool/projects/qemu.yml               |  1 +
 tests/lcitool/refresh                         | 96 ++++++++++++-------
 tests/lcitool/targets/centos-stream-8.yml     |  3 +
 tests/lcitool/targets/opensuse-leap-153.yml   |  3 +
 26 files changed, 215 insertions(+), 154 deletions(-)
 create mode 100644 tests/lcitool/mappings.yml
 create mode 100644 tests/lcitool/targets/centos-stream-8.yml
 create mode 100644 tests/lcitool/targets/opensuse-leap-153.yml

-- 
2.38.1
Re: [RFC PATCH 0/4] Update CentOS and OpenSUSE CI to Python >=3.7
Posted by Paolo Bonzini 1 year, 2 months ago
Ping.

Paolo

On Tue, Jan 17, 2023 at 10:16 AM Paolo Bonzini <pbonzini@redhat.com> wrote:
>
> QEMU build and test scripts would like to use some new features that
> were added to Python 3.7.  Useful additions include removing the need
> for OrderedDict, improved handling of default text encoding, postponed
> evaluation of type annotations, and improvements to asyncio.
>
> Using new features that were added to Meson 0.63 is also being blocked
> by usage of EOL'd versions of Python; Meson 0.63 does not work with the
> 3.6 version of Python that is provided by CentOS Stream 8 and OpenSUSE
> LEAP 15.  Therefore, this series upgrades the CentOS Stream 8 and OpenSUSE
> LEAP 15 dockerfiles to use the alternative Python runtimes provided by
> the distros.
>
> The main obstacle here is updating the lcitool-generated Dockerfiles;
> lcitool is (by design) quite opinionated on the mapping between
> "generic" package names and target-specific package names, and it
> caters to Libvirt's needs in terms of package versions.
>
> There are two possibilities to change this.
>
> First, lately I have been working on allowing lcitool to be used as
> a library.  As of this morning, it is possible for QEMU to use
> lcitool's databases and formatters directly from Python, and only
> replace the search mechanism to allow mappings and target paths to
> be overridden.  This is the approach that is implemented in this RFC:
> the first two patches in this series use libvirt-ci as library, while
> the third implements the custom search path.
>
> However, I have also proposed the custom search mechanism directly
> upstream (https://gitlab.com/libvirt/libvirt-ci/-/merge_requests/341).
> If this is accepted, patch 3 will become much smaller and will not
> need to know any lcitool internals; in addition, using lcitool as
> a library could be evaluated on its own merits because command-line
> operation would be a possibility as well.
>
> With the lcitool changes in place, the final patch shows how to
> install custom package mappings and update CentOS and OpenSUSE Python
> respectively to 3.8 and 3.9.
>
> Thanks,
>
> Paolo
>
> Paolo Bonzini (4):
>   lcitool: update submodule
>   lcitool: use libvirt-ci as library
>   lcitool: allow overriding package mappings and target facts
>   ci, docker: update CentOS and OpenSUSE Python to non-EOL versions
>
>  .gitlab-ci.d/cirrus/freebsd-12.vars           |  8 +-
>  .gitlab-ci.d/cirrus/freebsd-13.vars           |  8 +-
>  .gitlab-ci.d/cirrus/macos-12.vars             |  8 +-
>  scripts/ci/setup/build-environment.yml        |  2 +-
>  tests/docker/dockerfiles/alpine.docker        |  7 +-
>  tests/docker/dockerfiles/centos8.docker       | 29 +++---
>  .../dockerfiles/debian-amd64-cross.docker     |  7 +-
>  tests/docker/dockerfiles/debian-amd64.docker  |  7 +-
>  .../dockerfiles/debian-arm64-cross.docker     |  7 +-
>  .../dockerfiles/debian-armel-cross.docker     |  7 +-
>  .../dockerfiles/debian-armhf-cross.docker     |  7 +-
>  .../dockerfiles/debian-mips64el-cross.docker  |  7 +-
>  .../dockerfiles/debian-mipsel-cross.docker    |  7 +-
>  .../dockerfiles/debian-ppc64el-cross.docker   |  7 +-
>  .../dockerfiles/debian-s390x-cross.docker     |  7 +-
>  .../dockerfiles/fedora-win32-cross.docker     |  7 +-
>  .../dockerfiles/fedora-win64-cross.docker     |  7 +-
>  tests/docker/dockerfiles/fedora.docker        |  7 +-
>  tests/docker/dockerfiles/opensuse-leap.docker | 29 +++---
>  tests/docker/dockerfiles/ubuntu2004.docker    |  9 +-
>  tests/lcitool/libvirt-ci                      |  2 +-
>  tests/lcitool/mappings.yml                    | 80 ++++++++++++++++
>  tests/lcitool/projects/qemu.yml               |  1 +
>  tests/lcitool/refresh                         | 96 ++++++++++++-------
>  tests/lcitool/targets/centos-stream-8.yml     |  3 +
>  tests/lcitool/targets/opensuse-leap-153.yml   |  3 +
>  26 files changed, 215 insertions(+), 154 deletions(-)
>  create mode 100644 tests/lcitool/mappings.yml
>  create mode 100644 tests/lcitool/targets/centos-stream-8.yml
>  create mode 100644 tests/lcitool/targets/opensuse-leap-153.yml
>
> --
> 2.38.1