[libvirt] [PATCH v2 00/24] scripts: convert most perl scripts to python

Daniel P. Berrangé posted 24 patches 4 years, 7 months ago
Test syntax-check passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/libvirt tags/patchew/20190911162333.8668-1-berrange@redhat.com
There is a newer version of this series
Makefile.am                            |  14 +-
build-aux/augeas-gentest.pl            |  60 ----
build-aux/augeas-gentest.py            |  72 ++++
build-aux/check-spacing.pl             | 198 ----------
build-aux/check-spacing.py             | 204 +++++++++++
build-aux/header-ifdef.pl              | 182 ----------
build-aux/header-ifdef.py              | 206 +++++++++++
build-aux/minimize-po.pl               |  37 --
build-aux/minimize-po.py               |  60 ++++
build-aux/mock-noinline.pl             |  75 ----
build-aux/mock-noinline.py             |  88 +++++
build-aux/prohibit-duplicate-header.pl |  26 --
build-aux/prohibit-duplicate-header.py |  54 +++
cfg.mk                                 |  32 +-
configure.ac                           |   8 +
docs/Makefile.am                       |  17 +-
docs/genaclperms.pl                    | 125 -------
docs/genaclperms.py                    | 122 +++++++
docs/hacking.html.in                   |  30 ++
docs/hvsupport.pl                      | 458 -----------------------
docs/hvsupport.py                      | 479 +++++++++++++++++++++++++
po/Makefile.am                         |   2 +-
src/Makefile.am                        | 156 +++-----
src/access/Makefile.inc.am             |   6 +-
src/access/genpolkit.pl                | 119 ------
src/access/genpolkit.py                | 119 ++++++
src/bhyve/Makefile.inc.am              |   4 +-
src/check-aclperms.pl                  |  73 ----
src/check-aclperms.py                  |  77 ++++
src/check-aclrules.pl                  | 252 -------------
src/check-aclrules.py                  | 244 +++++++++++++
src/check-driverimpls.pl               |  80 -----
src/check-driverimpls.py               |  96 +++++
src/check-drivername.pl                |  83 -----
src/check-drivername.py                | 112 ++++++
src/check-remote-protocol.py           | 131 +++++++
src/check-symfile.pl                   |  70 ----
src/check-symfile.py                   |  80 +++++
src/check-symsorting.pl                | 106 ------
src/check-symsorting.py                | 112 ++++++
src/dtrace2systemtap.pl                | 130 -------
src/dtrace2systemtap.py                | 140 ++++++++
src/esx/Makefile.inc.am                |   2 +-
src/hyperv/Makefile.inc.am             |   2 +-
src/interface/Makefile.inc.am          |   2 +-
src/libxl/Makefile.inc.am              |   4 +-
src/locking/Makefile.inc.am            |   6 +-
src/logging/Makefile.inc.am            |   2 +-
src/lxc/Makefile.inc.am                |   4 +-
src/network/Makefile.inc.am            |   2 +-
src/node_device/Makefile.inc.am        |   2 +-
src/nwfilter/Makefile.inc.am           |   2 +-
src/qemu/Makefile.inc.am               |   4 +-
src/remote/Makefile.inc.am             |   4 +-
src/rpc/Makefile.inc.am                |   2 +-
src/rpc/gensystemtap.pl                | 193 ----------
src/rpc/gensystemtap.py                | 174 +++++++++
src/secret/Makefile.inc.am             |   2 +-
src/storage/Makefile.inc.am            |   2 +-
src/util/Makefile.inc.am               |   8 +-
src/vbox/Makefile.inc.am               |   2 +-
src/vz/Makefile.inc.am                 |   2 +-
tests/Makefile.am                      |   4 +-
tests/check-file-access.pl             | 126 -------
tests/check-file-access.py             | 121 +++++++
tests/file_access_whitelist.txt        |   2 +-
tests/group-qemu-caps.pl               | 124 -------
tests/group-qemu-caps.py               | 115 ++++++
tests/test-wrap-argv.pl                | 174 ---------
tests/test-wrap-argv.py                | 162 +++++++++
tests/testutils.c                      |  16 +-
71 files changed, 3132 insertions(+), 2872 deletions(-)
delete mode 100755 build-aux/augeas-gentest.pl
create mode 100755 build-aux/augeas-gentest.py
delete mode 100755 build-aux/check-spacing.pl
create mode 100755 build-aux/check-spacing.py
delete mode 100644 build-aux/header-ifdef.pl
create mode 100644 build-aux/header-ifdef.py
delete mode 100755 build-aux/minimize-po.pl
create mode 100755 build-aux/minimize-po.py
delete mode 100644 build-aux/mock-noinline.pl
create mode 100644 build-aux/mock-noinline.py
delete mode 100644 build-aux/prohibit-duplicate-header.pl
create mode 100644 build-aux/prohibit-duplicate-header.py
delete mode 100755 docs/genaclperms.pl
create mode 100755 docs/genaclperms.py
delete mode 100755 docs/hvsupport.pl
create mode 100755 docs/hvsupport.py
delete mode 100755 src/access/genpolkit.pl
create mode 100755 src/access/genpolkit.py
delete mode 100755 src/check-aclperms.pl
create mode 100755 src/check-aclperms.py
delete mode 100755 src/check-aclrules.pl
create mode 100755 src/check-aclrules.py
delete mode 100755 src/check-driverimpls.pl
create mode 100755 src/check-driverimpls.py
delete mode 100755 src/check-drivername.pl
create mode 100644 src/check-drivername.py
create mode 100644 src/check-remote-protocol.py
delete mode 100755 src/check-symfile.pl
create mode 100755 src/check-symfile.py
delete mode 100755 src/check-symsorting.pl
create mode 100755 src/check-symsorting.py
delete mode 100755 src/dtrace2systemtap.pl
create mode 100755 src/dtrace2systemtap.py
delete mode 100755 src/rpc/gensystemtap.pl
create mode 100755 src/rpc/gensystemtap.py
delete mode 100755 tests/check-file-access.pl
create mode 100755 tests/check-file-access.py
delete mode 100755 tests/group-qemu-caps.pl
create mode 100755 tests/group-qemu-caps.py
delete mode 100755 tests/test-wrap-argv.pl
create mode 100755 tests/test-wrap-argv.py
[libvirt] [PATCH v2 00/24] scripts: convert most perl scripts to python
Posted by Daniel P. Berrangé 4 years, 7 months ago
This series is an effort to reduce the number of different
languages we use by eliminating most use of perl in favour
of python.

This aligns with fact that the likely future build system
we'll use (meson) is written in python, and that python
is much more commonly used/understood by developers these
days than perl.

With this applied we use perl in a handful of places only:

 - src/rpc/gendispatch.pl - this is a horrendously large
   script and very hard to understand/follow. A straight
   syntax conversion to Python would still leave a hgue
   and hard to understand/follow script. It really needs
   a clean room rewrite from scratch, with better structure.

 - src/rpc/genprotocol.pl - fairly easy to convert, but
   might be obsolete depending on approach for rewriting
   gendispatch.pl, so ignored for now

 - tests/oomtrace.pl - will be purge by the patches that
   drop OOM handling anyway

 - tools/wireshark/util/genxdrstub.pl - a very large
   script, which I haven't got the courage to tackle
   yet.

 - cfg.mk/maint.mk - many syntax rules involve regexes
   which are fed to perl. Decision on what to do
   with syntax-check rules punted to another time.

 - build-aux/gitlog-to-changelog
 - build-aux/useless-if-before-free - Both pulled in
   from gnulib. Could be rewritten quite easily if
   desired, but given that we aren't maintainers of
   them right now, they're ignored as they don't
   really impact our developers.

In v2:

 - Pulled in patch to hacking file
 - Converted many more scripts
 - Forced UTF-8 character set to avoid ascii codec
   on py3 < 3.7

Daniel P. Berrangé (24):
  docs: document that C & Python are the preferred languages
  cfg.mk: fix comment detection for python semicolon check
  build: force a UTF-8 locale for python
  build-aux: rewrite augest test generator in Python
  build-aux: rewrite po file minimizer in Python
  build-aux: rewrite duplicate header checker in Python
  build-aux: rewrite whitespace checker in Python
  build-aux: rewrite mock inline checker in Python
  build-aux: rewrite header ifdef checker in Python
  src: rewrite ACL permissions checker in Python
  src: rewrite symfile sorting checker in Python
  src: rewrite symfile library checker in Python
  src: rewrite systemtap probe generator in Python
  src: rewrite systemtap function generator in Python
  src: rewrite driver name checker in Python
  src: rewrite driver impl checker in Python
  src: rewrite ACL rule checker in Python
  src: rewrite polkit ACL generator in Python
  src: rewrite remote protocol checker in Python
  tests: rewrite test argv line wrapper in Python
  tests: rewrite qemu capability grouper in Python
  tests: rewrite file access checker in Python
  docs: rewrite hvsupport.html page generator in python
  docs: rewrite polkit docs generator in Python

 Makefile.am                            |  14 +-
 build-aux/augeas-gentest.pl            |  60 ----
 build-aux/augeas-gentest.py            |  72 ++++
 build-aux/check-spacing.pl             | 198 ----------
 build-aux/check-spacing.py             | 204 +++++++++++
 build-aux/header-ifdef.pl              | 182 ----------
 build-aux/header-ifdef.py              | 206 +++++++++++
 build-aux/minimize-po.pl               |  37 --
 build-aux/minimize-po.py               |  60 ++++
 build-aux/mock-noinline.pl             |  75 ----
 build-aux/mock-noinline.py             |  88 +++++
 build-aux/prohibit-duplicate-header.pl |  26 --
 build-aux/prohibit-duplicate-header.py |  54 +++
 cfg.mk                                 |  32 +-
 configure.ac                           |   8 +
 docs/Makefile.am                       |  17 +-
 docs/genaclperms.pl                    | 125 -------
 docs/genaclperms.py                    | 122 +++++++
 docs/hacking.html.in                   |  30 ++
 docs/hvsupport.pl                      | 458 -----------------------
 docs/hvsupport.py                      | 479 +++++++++++++++++++++++++
 po/Makefile.am                         |   2 +-
 src/Makefile.am                        | 156 +++-----
 src/access/Makefile.inc.am             |   6 +-
 src/access/genpolkit.pl                | 119 ------
 src/access/genpolkit.py                | 119 ++++++
 src/bhyve/Makefile.inc.am              |   4 +-
 src/check-aclperms.pl                  |  73 ----
 src/check-aclperms.py                  |  77 ++++
 src/check-aclrules.pl                  | 252 -------------
 src/check-aclrules.py                  | 244 +++++++++++++
 src/check-driverimpls.pl               |  80 -----
 src/check-driverimpls.py               |  96 +++++
 src/check-drivername.pl                |  83 -----
 src/check-drivername.py                | 112 ++++++
 src/check-remote-protocol.py           | 131 +++++++
 src/check-symfile.pl                   |  70 ----
 src/check-symfile.py                   |  80 +++++
 src/check-symsorting.pl                | 106 ------
 src/check-symsorting.py                | 112 ++++++
 src/dtrace2systemtap.pl                | 130 -------
 src/dtrace2systemtap.py                | 140 ++++++++
 src/esx/Makefile.inc.am                |   2 +-
 src/hyperv/Makefile.inc.am             |   2 +-
 src/interface/Makefile.inc.am          |   2 +-
 src/libxl/Makefile.inc.am              |   4 +-
 src/locking/Makefile.inc.am            |   6 +-
 src/logging/Makefile.inc.am            |   2 +-
 src/lxc/Makefile.inc.am                |   4 +-
 src/network/Makefile.inc.am            |   2 +-
 src/node_device/Makefile.inc.am        |   2 +-
 src/nwfilter/Makefile.inc.am           |   2 +-
 src/qemu/Makefile.inc.am               |   4 +-
 src/remote/Makefile.inc.am             |   4 +-
 src/rpc/Makefile.inc.am                |   2 +-
 src/rpc/gensystemtap.pl                | 193 ----------
 src/rpc/gensystemtap.py                | 174 +++++++++
 src/secret/Makefile.inc.am             |   2 +-
 src/storage/Makefile.inc.am            |   2 +-
 src/util/Makefile.inc.am               |   8 +-
 src/vbox/Makefile.inc.am               |   2 +-
 src/vz/Makefile.inc.am                 |   2 +-
 tests/Makefile.am                      |   4 +-
 tests/check-file-access.pl             | 126 -------
 tests/check-file-access.py             | 121 +++++++
 tests/file_access_whitelist.txt        |   2 +-
 tests/group-qemu-caps.pl               | 124 -------
 tests/group-qemu-caps.py               | 115 ++++++
 tests/test-wrap-argv.pl                | 174 ---------
 tests/test-wrap-argv.py                | 162 +++++++++
 tests/testutils.c                      |  16 +-
 71 files changed, 3132 insertions(+), 2872 deletions(-)
 delete mode 100755 build-aux/augeas-gentest.pl
 create mode 100755 build-aux/augeas-gentest.py
 delete mode 100755 build-aux/check-spacing.pl
 create mode 100755 build-aux/check-spacing.py
 delete mode 100644 build-aux/header-ifdef.pl
 create mode 100644 build-aux/header-ifdef.py
 delete mode 100755 build-aux/minimize-po.pl
 create mode 100755 build-aux/minimize-po.py
 delete mode 100644 build-aux/mock-noinline.pl
 create mode 100644 build-aux/mock-noinline.py
 delete mode 100644 build-aux/prohibit-duplicate-header.pl
 create mode 100644 build-aux/prohibit-duplicate-header.py
 delete mode 100755 docs/genaclperms.pl
 create mode 100755 docs/genaclperms.py
 delete mode 100755 docs/hvsupport.pl
 create mode 100755 docs/hvsupport.py
 delete mode 100755 src/access/genpolkit.pl
 create mode 100755 src/access/genpolkit.py
 delete mode 100755 src/check-aclperms.pl
 create mode 100755 src/check-aclperms.py
 delete mode 100755 src/check-aclrules.pl
 create mode 100755 src/check-aclrules.py
 delete mode 100755 src/check-driverimpls.pl
 create mode 100755 src/check-driverimpls.py
 delete mode 100755 src/check-drivername.pl
 create mode 100644 src/check-drivername.py
 create mode 100644 src/check-remote-protocol.py
 delete mode 100755 src/check-symfile.pl
 create mode 100755 src/check-symfile.py
 delete mode 100755 src/check-symsorting.pl
 create mode 100755 src/check-symsorting.py
 delete mode 100755 src/dtrace2systemtap.pl
 create mode 100755 src/dtrace2systemtap.py
 delete mode 100755 src/rpc/gensystemtap.pl
 create mode 100755 src/rpc/gensystemtap.py
 delete mode 100755 tests/check-file-access.pl
 create mode 100755 tests/check-file-access.py
 delete mode 100755 tests/group-qemu-caps.pl
 create mode 100755 tests/group-qemu-caps.py
 delete mode 100755 tests/test-wrap-argv.pl
 create mode 100755 tests/test-wrap-argv.py

-- 
2.21.0

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v2 00/24] scripts: convert most perl scripts to python
Posted by Andrea Bolognani 4 years, 7 months ago
On Wed, 2019-09-11 at 17:23 +0100, Daniel P. Berrangé wrote:
> This series is an effort to reduce the number of different
> languages we use by eliminating most use of perl in favour
> of python.

Just today I found out about

  https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/7.7_release_notes/new_features#enhancement_compiler-and-tools

which means that if we interpret "supporting RHEL 7" as "supporting
the most recent RHEL 7 point release", which I believe we do, then
that's one less platform where we are forced to use Python 2! \o/

It might even be the last one, but I'm not entirely sure what the
situation is like for SLES and OpenSUSE... Jim, does SLES 12 have
Python 3?

And, as a side note: do you think you could find the time to add
OpenSUSE support to the libvirt-jenkins-ci project? That'd be very
useful, because it makes grepping for this kind of information
trivial, and also would open the door to running actual CI jobs on
the OS :)

> This aligns with fact that the likely future build system
> we'll use (meson) is written in python, and that python
> is much more commonly used/understood by developers these
> days than perl.

I believe Meson is itself implemented in Python 3, so platform
availability will have to be taken into consideration.

My hope is that we can finally ditch Python 2 for good. The
https://pythonclock.org/ keeps ticking...

-- 
Andrea Bolognani / Red Hat / Virtualization

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v2 00/24] scripts: convert most perl scripts to python
Posted by Daniel P. Berrangé 4 years, 7 months ago
On Fri, Sep 13, 2019 at 10:56:45AM +0200, Andrea Bolognani wrote:
> On Wed, 2019-09-11 at 17:23 +0100, Daniel P. Berrangé wrote:
> > This series is an effort to reduce the number of different
> > languages we use by eliminating most use of perl in favour
> > of python.
> 
> Just today I found out about
> 
>   https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/7.7_release_notes/new_features#enhancement_compiler-and-tools
> 
> which means that if we interpret "supporting RHEL 7" as "supporting
> the most recent RHEL 7 point release", which I believe we do, then
> that's one less platform where we are forced to use Python 2! \o/

I should have said that I did in fact test this series with Python 2.7,
Python 3.5 and Python 3.7 - all using the Fedora 31 RPMS of these
respective versions.

> > This aligns with fact that the likely future build system
> > we'll use (meson) is written in python, and that python
> > is much more commonly used/understood by developers these
> > days than perl.
> 
> I believe Meson is itself implemented in Python 3, so platform
> availability will have to be taken into consideration.

Yes, Meson needs py 3.4 I believe

> My hope is that we can finally ditch Python 2 for good. The
> https://pythonclock.org/ keeps ticking...

My intention was to submit patches to purge Py2 support in
December, so that we have it gone for the Jan 15 2020 release
which matches the pythonclock timeout nicely.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v2 00/24] scripts: convert most perl scripts to python
Posted by Andrea Bolognani 4 years, 7 months ago
On Fri, 2019-09-13 at 09:59 +0100, Daniel P. Berrangé wrote:
> On Fri, Sep 13, 2019 at 10:56:45AM +0200, Andrea Bolognani wrote:
> > On Wed, 2019-09-11 at 17:23 +0100, Daniel P. Berrangé wrote:
> > > This aligns with fact that the likely future build system
> > > we'll use (meson) is written in python, and that python
> > > is much more commonly used/understood by developers these
> > > days than perl.
> > 
> > I believe Meson is itself implemented in Python 3, so platform
> > availability will have to be taken into consideration.
> 
> Yes, Meson needs py 3.4 I believe

If I understand the SLES / OpenSUSE situation correctly, then SLES
12.3 should be comparable to OpenSUSE 42.3, which according to
Repology has Python 3.4.6. All other platforms have newer Python 3
versions, so we should be good.

> > My hope is that we can finally ditch Python 2 for good. The
> > https://pythonclock.org/ keeps ticking...
> 
> My intention was to submit patches to purge Py2 support in
> December, so that we have it gone for the Jan 15 2020 release
> which matches the pythonclock timeout nicely.

The reason I brought it up is because I don't think the situation
is going to change significantly in the next three months, so it
would probably make sense for these replacement Python scripts to
be Python 3 only from day one.

On the other hand, you've already gone through the trouble of
making them Python 2.7 compatible...

-- 
Andrea Bolognani / Red Hat / Virtualization

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v2 00/24] scripts: convert most perl scripts to python
Posted by Daniel P. Berrangé 4 years, 7 months ago
On Fri, Sep 13, 2019 at 11:09:16AM +0200, Andrea Bolognani wrote:
> On Fri, 2019-09-13 at 09:59 +0100, Daniel P. Berrangé wrote:
> > On Fri, Sep 13, 2019 at 10:56:45AM +0200, Andrea Bolognani wrote:
> > > On Wed, 2019-09-11 at 17:23 +0100, Daniel P. Berrangé wrote:
> > > > This aligns with fact that the likely future build system
> > > > we'll use (meson) is written in python, and that python
> > > > is much more commonly used/understood by developers these
> > > > days than perl.
> > > 
> > > I believe Meson is itself implemented in Python 3, so platform
> > > availability will have to be taken into consideration.
> > 
> > Yes, Meson needs py 3.4 I believe
> 
> If I understand the SLES / OpenSUSE situation correctly, then SLES
> 12.3 should be comparable to OpenSUSE 42.3, which according to
> Repology has Python 3.4.6. All other platforms have newer Python 3
> versions, so we should be good.
> 
> > > My hope is that we can finally ditch Python 2 for good. The
> > > https://pythonclock.org/ keeps ticking...
> > 
> > My intention was to submit patches to purge Py2 support in
> > December, so that we have it gone for the Jan 15 2020 release
> > which matches the pythonclock timeout nicely.
> 
> The reason I brought it up is because I don't think the situation
> is going to change significantly in the next three months, so it
> would probably make sense for these replacement Python scripts to
> be Python 3 only from day one.
> 
> On the other hand, you've already gone through the trouble of
> making them Python 2.7 compatible...

Honestly the only "trouble" I took was adding

  from __future__ import print_function

everything else I just wrote in the normal way & I got lucky that nothing
I did was py3 specific. Most of these scripts are using very boring python
code, so its not that surprising.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v2 00/24] scripts: convert most perl scripts to python
Posted by Andrea Bolognani 4 years, 7 months ago
On Fri, 2019-09-13 at 10:12 +0100, Daniel P. Berrangé wrote:
> On Fri, Sep 13, 2019 at 11:09:16AM +0200, Andrea Bolognani wrote:
> > On Fri, 2019-09-13 at 09:59 +0100, Daniel P. Berrangé wrote:
> > > On Fri, Sep 13, 2019 at 10:56:45AM +0200, Andrea Bolognani wrote:
> > > > My hope is that we can finally ditch Python 2 for good. The
> > > > https://pythonclock.org/ keeps ticking...
> > > 
> > > My intention was to submit patches to purge Py2 support in
> > > December, so that we have it gone for the Jan 15 2020 release
> > > which matches the pythonclock timeout nicely.
> > 
> > The reason I brought it up is because I don't think the situation
> > is going to change significantly in the next three months, so it
> > would probably make sense for these replacement Python scripts to
> > be Python 3 only from day one.
> > 
> > On the other hand, you've already gone through the trouble of
> > making them Python 2.7 compatible...
> 
> Honestly the only "trouble" I took was adding
> 
>   from __future__ import print_function
> 
> everything else I just wrote in the normal way & I got lucky that nothing
> I did was py3 specific. Most of these scripts are using very boring python
> code, so its not that surprising.

Good to know! Still, no need to wait until December before dropping
Python 2 support in my opinion: we should either be able to do it
right now, or we're going to find ourselves in an interesting
position three and a half short months from now... Let's see what
Jim has to say about SUSE :)

-- 
Andrea Bolognani / Red Hat / Virtualization

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v2 00/24] scripts: convert most perl scripts to python
Posted by Daniel P. Berrangé 4 years, 7 months ago
On Fri, Sep 13, 2019 at 11:31:34AM +0200, Andrea Bolognani wrote:
> On Fri, 2019-09-13 at 10:12 +0100, Daniel P. Berrangé wrote:
> > On Fri, Sep 13, 2019 at 11:09:16AM +0200, Andrea Bolognani wrote:
> > > On Fri, 2019-09-13 at 09:59 +0100, Daniel P. Berrangé wrote:
> > > > On Fri, Sep 13, 2019 at 10:56:45AM +0200, Andrea Bolognani wrote:
> > > > > My hope is that we can finally ditch Python 2 for good. The
> > > > > https://pythonclock.org/ keeps ticking...
> > > > 
> > > > My intention was to submit patches to purge Py2 support in
> > > > December, so that we have it gone for the Jan 15 2020 release
> > > > which matches the pythonclock timeout nicely.
> > > 
> > > The reason I brought it up is because I don't think the situation
> > > is going to change significantly in the next three months, so it
> > > would probably make sense for these replacement Python scripts to
> > > be Python 3 only from day one.
> > > 
> > > On the other hand, you've already gone through the trouble of
> > > making them Python 2.7 compatible...
> > 
> > Honestly the only "trouble" I took was adding
> > 
> >   from __future__ import print_function
> > 
> > everything else I just wrote in the normal way & I got lucky that nothing
> > I did was py3 specific. Most of these scripts are using very boring python
> > code, so its not that surprising.
> 
> Good to know! Still, no need to wait until December before dropping
> Python 2 support in my opinion: we should either be able to do it
> right now, or we're going to find ourselves in an interesting
> position three and a half short months from now... Let's see what
> Jim has to say about SUSE :)

I don't see a need to rush into it right now. Synchronizing with end of
life of python upstream, makes sense as a time point and matches what
QEMU will doo too, with its first release of 2020 dropping py2.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v2 00/24] scripts: convert most perl scripts to python
Posted by Andrea Bolognani 4 years, 7 months ago
On Fri, 2019-09-13 at 10:40 +0100, Daniel P. Berrangé wrote:
> On Fri, Sep 13, 2019 at 11:31:34AM +0200, Andrea Bolognani wrote:
> > Good to know! Still, no need to wait until December before dropping
> > Python 2 support in my opinion: we should either be able to do it
> > right now, or we're going to find ourselves in an interesting
> > position three and a half short months from now... Let's see what
> > Jim has to say about SUSE :)
> 
> I don't see a need to rush into it right now. Synchronizing with end of
> life of python upstream, makes sense as a time point and matches what
> QEMU will doo too, with its first release of 2020 dropping py2.

Sure, that sounds fine as long as we've made sure we're ready for
the switch when the time comes.

-- 
Andrea Bolognani / Red Hat / Virtualization

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v2 00/24] scripts: convert most perl scripts to python
Posted by Jim Fehlig 4 years, 7 months ago
On 9/13/19 3:09 AM, Andrea Bolognani wrote:
> If I understand the SLES / OpenSUSE situation correctly, then SLES
> 12.3 should be comparable to OpenSUSE 42.3,

Nice job! Even SUSE folks are known to get confused by the numbering :-).

openSUSE 42.3 ~= SLES12 SP3
SLES12 SP4 and SP5 have no openSUSE equivalent
openSUSE 15 ~= SLES15
openSUSE 15.1 ~= SLES15 SP1
openSUSE 15.2 ~= SLES15 SP2
...
openSUSE 16 ~= SLES16 (assumed, since they don't exist)

Regards,
Jim

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v2 00/24] scripts: convert most perl scripts to python
Posted by Jim Fehlig 4 years, 7 months ago
On 9/13/19 2:56 AM, Andrea Bolognani wrote:
> On Wed, 2019-09-11 at 17:23 +0100, Daniel P. Berrangé wrote:
>> This series is an effort to reduce the number of different
>> languages we use by eliminating most use of perl in favour
>> of python.
> 
> Just today I found out about
> 
>    https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/7.7_release_notes/new_features#enhancement_compiler-and-tools
> 
> which means that if we interpret "supporting RHEL 7" as "supporting
> the most recent RHEL 7 point release", which I believe we do, then
> that's one less platform where we are forced to use Python 2! \o/
> 
> It might even be the last one, but I'm not entirely sure what the
> situation is like for SLES and OpenSUSE... Jim, does SLES 12 have
> Python 3?

Yes, python 3.4.6. And python 2.7.13.

> And, as a side note: do you think you could find the time to add
> OpenSUSE support to the libvirt-jenkins-ci project? That'd be very
> useful, because it makes grepping for this kind of information
> trivial, and also would open the door to running actual CI jobs on
> the OS :)

I have internal jobs but agreed it would be nice to have openSUSE included in 
upstream CI on vanilla upstream :-). Any pointers on how to do that?

Regards,
Jim

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v2 00/24] scripts: convert most perl scripts to python
Posted by Andrea Bolognani 4 years, 7 months ago
On Fri, 2019-09-13 at 13:58 +0000, Jim Fehlig wrote:
> On 9/13/19 2:56 AM, Andrea Bolognani wrote:
> > Jim, does SLES 12 have
> > Python 3?
> 
> Yes, python 3.4.6. And python 2.7.13.

That's *amazing* news! \o/

> > And, as a side note: do you think you could find the time to add
> > OpenSUSE support to the libvirt-jenkins-ci project? That'd be very
> > useful, because it makes grepping for this kind of information
> > trivial, and also would open the door to running actual CI jobs on
> > the OS :)
> 
> I have internal jobs but agreed it would be nice to have openSUSE included in 
> upstream CI on vanilla upstream :-). Any pointers on how to do that?

Good to hear you're willing to help with this effort! I've been
wanting to introduce OpenSUSE support myself for a very long time,
but unfortunately I've never quite managed to scrap together the
necessary time.

The repository is

  https://libvirt.org/git/?p=libvirt-jenkins-ci.git

and most of the stuff you'd have to touch is inside the guests/
subdirectory; more specifically, you're definitely going to have to:

  * add OpenSUSE 12 and 15 to inventory and host_vars/;

  * write an AutoYaST configuration file that can be used to install
    a minimal OpenSUSE guest without user interaction and add it to
    the configs/ directory along with the existing preseed and
    kickstart configurations;

  * add mappings from abstract package names, such as 'dtrace', to
    the corresponding OpenSUSE concrete package names, such as
    'systemtap-sdt-devel', to vars/mappings.yml;

  * test installation and see what breaks, fix it, rinse and
    repeat :D

There are more steps necessary to reach full integration after the
above, such as ensuring Dockerfiles can be generated, figuring out
which projects can be build on the new target platforms and tweaking
the Jenkins / Ansible configuration, but the above is a good start
and also the part that I'm less comfortable doing myself since I'm
not at all familiar with SUSE, so if you could take care of the
above that'd be a big help indeed!

The guests/README.md file provides some more information, and the
lcitool script is hopefully not too difficult to understand. Last
but not least, if you have any questions you already know how to
get in touch with me ;)

-- 
Andrea Bolognani / Red Hat / Virtualization

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v2 00/24] scripts: convert most perl scripts to python
Posted by Jim Fehlig 4 years, 7 months ago
On 9/13/19 9:28 AM, Andrea Bolognani wrote:
> On Fri, 2019-09-13 at 13:58 +0000, Jim Fehlig wrote:
>> On 9/13/19 2:56 AM, Andrea Bolognani wrote:
>>> Jim, does SLES 12 have
>>> Python 3?
>>
>> Yes, python 3.4.6. And python 2.7.13.
> 
> That's *amazing* news! \o/
> 
>>> And, as a side note: do you think you could find the time to add
>>> OpenSUSE support to the libvirt-jenkins-ci project? That'd be very
>>> useful, because it makes grepping for this kind of information
>>> trivial, and also would open the door to running actual CI jobs on
>>> the OS :)
>>
>> I have internal jobs but agreed it would be nice to have openSUSE included in
>> upstream CI on vanilla upstream :-). Any pointers on how to do that?
> 
> Good to hear you're willing to help with this effort! I've been
> wanting to introduce OpenSUSE support myself for a very long time,
> but unfortunately I've never quite managed to scrap together the
> necessary time.

Thanks for the detailed info! I have a busy month ahead, including traveling the 
next two weeks, so I likely wont get to it until Oct. But I will get to it :-).

Regards,
Jim

> 
> The repository is
> 
>    https://libvirt.org/git/?p=libvirt-jenkins-ci.git
> 
> and most of the stuff you'd have to touch is inside the guests/
> subdirectory; more specifically, you're definitely going to have to:
> 
>    * add OpenSUSE 12 and 15 to inventory and host_vars/;
> 
>    * write an AutoYaST configuration file that can be used to install
>      a minimal OpenSUSE guest without user interaction and add it to
>      the configs/ directory along with the existing preseed and
>      kickstart configurations;
> 
>    * add mappings from abstract package names, such as 'dtrace', to
>      the corresponding OpenSUSE concrete package names, such as
>      'systemtap-sdt-devel', to vars/mappings.yml;
> 
>    * test installation and see what breaks, fix it, rinse and
>      repeat :D
> 
> There are more steps necessary to reach full integration after the
> above, such as ensuring Dockerfiles can be generated, figuring out
> which projects can be build on the new target platforms and tweaking
> the Jenkins / Ansible configuration, but the above is a good start
> and also the part that I'm less comfortable doing myself since I'm
> not at all familiar with SUSE, so if you could take care of the
> above that'd be a big help indeed!
> 
> The guests/README.md file provides some more information, and the
> lcitool script is hopefully not too difficult to understand. Last
> but not least, if you have any questions you already know how to
> get in touch with me ;)
> 


--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v2 00/24] scripts: convert most perl scripts to python
Posted by Andrea Bolognani 4 years, 7 months ago
On Fri, 2019-09-13 at 18:24 +0000, Jim Fehlig wrote:
> On 9/13/19 9:28 AM, Andrea Bolognani wrote:
> > Good to hear you're willing to help with this effort! I've been
> > wanting to introduce OpenSUSE support myself for a very long time,
> > but unfortunately I've never quite managed to scrap together the
> > necessary time.
> 
> Thanks for the detailed info! I have a busy month ahead, including traveling the 
> next two weeks, so I likely wont get to it until Oct. But I will get to it :-).

No need to rush! :)

-- 
Andrea Bolognani / Red Hat / Virtualization

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v2 00/24] scripts: convert most perl scripts to python
Posted by Jim Fehlig 4 years, 5 months ago
On 9/13/19 9:28 AM, Andrea Bolognani wrote:
> On Fri, 2019-09-13 at 13:58 +0000, Jim Fehlig wrote:
>> On 9/13/19 2:56 AM, Andrea Bolognani wrote:
>>> Jim, does SLES 12 have
>>> Python 3?
>>
>> Yes, python 3.4.6. And python 2.7.13.
> 
> That's *amazing* news! \o/
> 
>>> And, as a side note: do you think you could find the time to add
>>> OpenSUSE support to the libvirt-jenkins-ci project? That'd be very
>>> useful, because it makes grepping for this kind of information
>>> trivial, and also would open the door to running actual CI jobs on
>>> the OS :)
>>
>> I have internal jobs but agreed it would be nice to have openSUSE included in
>> upstream CI on vanilla upstream :-). Any pointers on how to do that?

Sorry for the delay. Thanks again for the below info!

> Good to hear you're willing to help with this effort! I've been
> wanting to introduce OpenSUSE support myself for a very long time,
> but unfortunately I've never quite managed to scrap together the
> necessary time.
> 
> The repository is
> 
>    https://libvirt.org/git/?p=libvirt-jenkins-ci.git
> 
> and most of the stuff you'd have to touch is inside the guests/
> subdirectory; more specifically, you're definitely going to have to:
> 
>    * add OpenSUSE 12 and 15 to inventory and host_vars/;
> 
>    * write an AutoYaST configuration file that can be used to install
>      a minimal OpenSUSE guest without user interaction and add it to
>      the configs/ directory along with the existing preseed and
>      kickstart configurations;
> 
>    * add mappings from abstract package names, such as 'dtrace', to
>      the corresponding OpenSUSE concrete package names, such as
>      'systemtap-sdt-devel', to vars/mappings.yml;

I've completed these steps.

>    * test installation and see what breaks, fix it, rinse and
>      repeat :D

But I'm still in the wash, rinse, repeat cycle :-). I'm able to 'lcitool 
install' an openSUSE Leap 15.1 guest, still working on 'lcitool update'.

Nonetheless I thought I'd send out my WIP before calling it a week. Any comments 
on what I have thus far is appreciated.

Regards,
Jim

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v2 00/24] scripts: convert most perl scripts to python
Posted by Andrea Bolognani 4 years, 5 months ago
On Sat, 2019-11-16 at 00:35 +0000, Jim Fehlig wrote:
> On 9/13/19 9:28 AM, Andrea Bolognani wrote:
> > [...] you're definitely going to have to:
> > 
> >    * add OpenSUSE 12 and 15 to inventory and host_vars/;
> > 
> >    * write an AutoYaST configuration file that can be used to install
> >      a minimal OpenSUSE guest without user interaction and add it to
> >      the configs/ directory along with the existing preseed and
> >      kickstart configurations;
> > 
> >    * add mappings from abstract package names, such as 'dtrace', to
> >      the corresponding OpenSUSE concrete package names, such as
> >      'systemtap-sdt-devel', to vars/mappings.yml;
> 
> I've completed these steps.
> 
> >    * test installation and see what breaks, fix it, rinse and
> >      repeat :D
> 
> But I'm still in the wash, rinse, repeat cycle :-). I'm able to 'lcitool 
> install' an openSUSE Leap 15.1 guest, still working on 'lcitool update'.
> 
> Nonetheless I thought I'd send out my WIP before calling it a week. Any comments 
> on what I have thus far is appreciated.

Do you mind sendind the patch out as an RFC against the
libvirt-jenkins-ci project? That way I can conveniently add comments
there instead of hijacking this unrelated thread further :)

-- 
Andrea Bolognani / Red Hat / Virtualization

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list

Re: [libvirt] [PATCH v2 00/24] scripts: convert most perl scripts to python
Posted by Daniel P. Berrangé 4 years, 7 months ago
On Wed, Sep 11, 2019 at 05:23:09PM +0100, Daniel P. Berrangé wrote:
> This series is an effort to reduce the number of different
> languages we use by eliminating most use of perl in favour
> of python.
> 
> This aligns with fact that the likely future build system
> we'll use (meson) is written in python, and that python
> is much more commonly used/understood by developers these
> days than perl.
> 
> With this applied we use perl in a handful of places only:

BTW, after the first 3, these patches are all independant
so can be applied in any order at any time, so no need to
review it all for a big bang merge.


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list