From nobody Thu May 2 16:33:49 2024
Delivered-To: importer@patchew.org
Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28
as permitted sender) client-ip=209.132.183.28;
envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com;
Authentication-Results: mx.zohomail.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass(p=none dis=none) header.from=redhat.com
ARC-Seal: i=1; a=rsa-sha256; t=1569337451; cv=none;
d=zoho.com; s=zohoarc;
b=fPh2ESF7/9Q89o+SRbKGqEdbid44NfJwQxdppKhd9gdxPWrmPC3dUrRY0+prMJ0W4OVr2NUWjLiKYWmV/AQSkmh6o3C+HgPXKeOohA0188rxQjUGD1gT+20gIs/HpN2OBN5M6SICBFiCUTNF72VII6XHuB/WB43dLeOq51zbkv0=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com;
s=zohoarc;
t=1569337451;
h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results;
bh=hIckg8CjR6dVNYHvfctweijgLqSG2m96qLQOP8FBWQs=;
b=QLS2eIjUv88iuQ+is+VLnumooYrhFoZQ5pSZDhMtCc6vGCAw/DHCjV/tyrFKEW4TlAzZfxV3gX2r9E0bfTMDO/2o7i443+gwkgYWRPii0VBJyFA90nfqpmUvLevtO1MlcPjqLbmQ1fqkQ+4Gb7EyhLAryObjRZT5LLrDur2h62M=
ARC-Authentication-Results: i=1; mx.zoho.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass header.from= (p=none dis=none)
header.from=
Return-Path:
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by
mx.zohomail.com
with SMTPS id 1569337451077541.6925470093768;
Tue, 24 Sep 2019 08:04:11 -0700 (PDT)
Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com
[10.5.11.22])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by mx1.redhat.com (Postfix) with ESMTPS id 2E30B10C093F;
Tue, 24 Sep 2019 15:04:09 +0000 (UTC)
Received: from colo-mx.corp.redhat.com
(colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21])
by smtp.corp.redhat.com (Postfix) with ESMTPS id ED34F10013D9;
Tue, 24 Sep 2019 15:04:08 +0000 (UTC)
Received: from lists01.pubmisc.prod.ext.phx2.redhat.com
(lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33])
by colo-mx.corp.redhat.com (Postfix) with ESMTP id 35EA24ED21;
Tue, 24 Sep 2019 15:04:06 +0000 (UTC)
Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com
[10.5.11.14])
by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
id x8OF43MH027800 for ;
Tue, 24 Sep 2019 11:04:03 -0400
Received: by smtp.corp.redhat.com (Postfix)
id 563EF5D9DC; Tue, 24 Sep 2019 15:04:03 +0000 (UTC)
Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com
[10.33.9.32])
by smtp.corp.redhat.com (Postfix) with ESMTP id BB6465D9D5;
Tue, 24 Sep 2019 15:04:02 +0000 (UTC)
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?=
To: libvir-list@redhat.com
Date: Tue, 24 Sep 2019 15:58:42 +0100
Message-Id: <20190924145903.17123-2-berrange@redhat.com>
In-Reply-To: <20190924145903.17123-1-berrange@redhat.com>
References: <20190924145903.17123-1-berrange@redhat.com>
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14
X-loop: libvir-list@redhat.com
Subject: [libvirt] [PATCH v3 01/22] docs: document that C & Python are the
preferred languages
X-BeenThere: libvir-list@redhat.com
X-Mailman-Version: 2.1.12
Precedence: junk
List-Id: Development discussions about the libvirt library & tools
List-Unsubscribe: ,
List-Archive:
List-Post:
List-Help:
List-Subscribe: ,
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Sender: libvir-list-bounces@redhat.com
Errors-To: libvir-list-bounces@redhat.com
X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22
X-Greylist: Sender IP whitelisted,
not delayed by milter-greylist-4.6.2 (mx1.redhat.com [10.5.110.66]);
Tue, 24 Sep 2019 15:04:09 +0000 (UTC)
Blacklist Perl and Shell code in favour of Python for
sake of readability and portability.
Signed-off-by: Daniel P. Berrang=C3=A9
Reviewed-by: J=C3=A1n Tomko
---
docs/hacking.html.in | 30 ++++++++++++++++++++++++++++++
1 file changed, 30 insertions(+)
diff --git a/docs/hacking.html.in b/docs/hacking.html.in
index 90376968c3..edf2f54ce3 100644
--- a/docs/hacking.html.in
+++ b/docs/hacking.html.in
@@ -343,6 +343,36 @@
Richard Jones' guide to working with open source projects.
=20
+
+
+
+ The libvirt repository makes use of a large number of programming
+ languages. It is anticipated that in the future libvirt will adopt
+ use of other new languages. To reduce the overall burden on develope=
rs,
+ there is thus a general desire to phase out usage of some of the
+ existing languages.
+
+
+
+ The preferred languages at this time are:
+
+
+
+ - C - for the main libvirt codebase. Dialect supported by
+ GCC/CLang only.
+ - Python - for supporting build scripts / tools. Code must
+ run with both version 2.7 and 3.x at this time.
+
+
+
+ Languages that should not be used for any new contributions:
+
+
+
+ - Perl - build scripts must be written in Python instead.
+ - Shell - build scripts must be written in Python instead.
+
+
=20
--=20
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
From nobody Thu May 2 16:33:49 2024
Delivered-To: importer@patchew.org
Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28
as permitted sender) client-ip=209.132.183.28;
envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com;
Authentication-Results: mx.zohomail.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass(p=none dis=none) header.from=redhat.com
ARC-Seal: i=1; a=rsa-sha256; t=1569337471; cv=none;
d=zoho.com; s=zohoarc;
b=DhYMzfRj87Meq4cCakwSYod4F1qsbIHHLy9tW2v38ON1pq7LmIyCIN09uTV/MeBMfx59jLXD6X6dKKz85uAZpDugc8v/Au6mQNCzOjOEUC2Dzs3jMCSKmYnGLXAycmm4o0loFRGTC1KLGtYM6UXpz5nA/SLaYyGvC18waQW2xQ0=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com;
s=zohoarc;
t=1569337471;
h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results;
bh=IPLnBr7tZz+LlMRgvSKg2CGP3hq2c8y4wL0XYpxlHEA=;
b=YEzemJ9aGvHtFztmVfw7bw2jxBacBqxcL9mTV8tXqdtUbSDPjJcvX0A8n19b7MaX4M7bVbjtqJq/Y8V6Nm3txuYR+vNkZPJBZDGU8gax4LSlHEpY7bFv0yYZyAvJjTyrRa3OKTL1soFqtrJxJND0k71SVn+VPMOzEWRCA3AamNo=
ARC-Authentication-Results: i=1; mx.zoho.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass header.from= (p=none dis=none)
header.from=
Return-Path:
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by
mx.zohomail.com
with SMTPS id 1569337471824788.7442733257443;
Tue, 24 Sep 2019 08:04:31 -0700 (PDT)
Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com
[10.5.11.11])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by mx1.redhat.com (Postfix) with ESMTPS id F152E7F752;
Tue, 24 Sep 2019 15:04:29 +0000 (UTC)
Received: from colo-mx.corp.redhat.com
(colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20])
by smtp.corp.redhat.com (Postfix) with ESMTPS id CB3C6600C8;
Tue, 24 Sep 2019 15:04:29 +0000 (UTC)
Received: from lists01.pubmisc.prod.ext.phx2.redhat.com
(lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33])
by colo-mx.corp.redhat.com (Postfix) with ESMTP id 8C402180B76F;
Tue, 24 Sep 2019 15:04:29 +0000 (UTC)
Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com
[10.5.11.14])
by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
id x8OF46XS027815 for ;
Tue, 24 Sep 2019 11:04:06 -0400
Received: by smtp.corp.redhat.com (Postfix)
id 276D75D9E1; Tue, 24 Sep 2019 15:04:06 +0000 (UTC)
Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com
[10.33.9.32])
by smtp.corp.redhat.com (Postfix) with ESMTP id 6A2F15D9D5;
Tue, 24 Sep 2019 15:04:03 +0000 (UTC)
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?=
To: libvir-list@redhat.com
Date: Tue, 24 Sep 2019 15:58:43 +0100
Message-Id: <20190924145903.17123-3-berrange@redhat.com>
In-Reply-To: <20190924145903.17123-1-berrange@redhat.com>
References: <20190924145903.17123-1-berrange@redhat.com>
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14
X-loop: libvir-list@redhat.com
Subject: [libvirt] [PATCH v3 02/22] build-aux: rewrite augest test generator
in Python
X-BeenThere: libvir-list@redhat.com
X-Mailman-Version: 2.1.12
Precedence: junk
List-Id: Development discussions about the libvirt library & tools
List-Unsubscribe: ,
List-Archive:
List-Post:
List-Help:
List-Subscribe: ,
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Sender: libvir-list-bounces@redhat.com
Errors-To: libvir-list-bounces@redhat.com
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11
X-Greylist: Sender IP whitelisted,
not delayed by milter-greylist-4.6.2 (mx1.redhat.com [10.5.110.71]);
Tue, 24 Sep 2019 15:04:30 +0000 (UTC)
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the augeas-gentest.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
The use of $(AUG_GENTEST) as a dependancy in the makefiles needed
to be fixed, because this was assumed to be the filename of the
script, but is in fact a full shell command line.
Signed-off-by: Daniel P. Berrang=C3=A9
Reviewed-by: J=C3=A1n Tomko
---
Makefile.am | 2 +-
build-aux/augeas-gentest.pl | 60 ---------------------------
build-aux/augeas-gentest.py | 72 +++++++++++++++++++++++++++++++++
src/Makefile.am | 3 +-
src/bhyve/Makefile.inc.am | 4 +-
src/interface/Makefile.inc.am | 2 +-
src/libxl/Makefile.inc.am | 4 +-
src/locking/Makefile.inc.am | 6 +--
src/logging/Makefile.inc.am | 2 +-
src/lxc/Makefile.inc.am | 4 +-
src/network/Makefile.inc.am | 2 +-
src/node_device/Makefile.inc.am | 2 +-
src/nwfilter/Makefile.inc.am | 2 +-
src/qemu/Makefile.inc.am | 4 +-
src/remote/Makefile.inc.am | 4 +-
src/secret/Makefile.inc.am | 2 +-
src/storage/Makefile.inc.am | 2 +-
src/vbox/Makefile.inc.am | 2 +-
src/vz/Makefile.inc.am | 2 +-
19 files changed, 97 insertions(+), 84 deletions(-)
delete mode 100755 build-aux/augeas-gentest.pl
create mode 100755 build-aux/augeas-gentest.py
diff --git a/Makefile.am b/Makefile.am
index 711f365504..17448a914e 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -41,7 +41,7 @@ EXTRA_DIST =3D \
run.in \
README.md \
AUTHORS.in \
- build-aux/augeas-gentest.pl \
+ build-aux/augeas-gentest.py \
build-aux/check-spacing.pl \
build-aux/gitlog-to-changelog \
build-aux/header-ifdef.pl \
diff --git a/build-aux/augeas-gentest.pl b/build-aux/augeas-gentest.pl
deleted file mode 100755
index 65834b533b..0000000000
--- a/build-aux/augeas-gentest.pl
+++ /dev/null
@@ -1,60 +0,0 @@
-#!/usr/bin/env perl
-#
-# augeas-gentest.pl: Generate an augeas test file, from an
-# example config file + test file template
-#
-# This library is free software; you can redistribute it and/or
-# modify it under the terms of the GNU Lesser General Public
-# License as published by the Free Software Foundation; either
-# version 2.1 of the License, or (at your option) any later version.
-#
-# This library is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-# Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public
-# License along with this library. If not, see
-# .
-
-use strict;
-use warnings;
-
-die "syntax: $0 CONFIG TEMPLATE\n" unless @ARGV =3D=3D 2;
-
-my $config =3D shift @ARGV;
-my $template =3D shift @ARGV;
-
-open CONFIG, "<", $config or die "cannot read $config: $!";
-open TEMPLATE, "<", $template or die "cannot read $template: $!";
-
-my $group =3D 0;
-while () {
- if (/\@CONFIG\@/) {
- my $group =3D 0;
- print " let conf =3D \"";
- while () {
- if (/^#\w/) {
- s/^#//;
- s/\"/\\\"/g;
- print $_;
- $group =3D /\[\s$/;
- } elsif ($group) {
- s/\"/\\\"/g;
- if (/#\s*\]/) {
- $group =3D 0;
- }
- if (/^#/) {
- s/^#//;
- print $_;
- }
- }
- }
- print "\"\n";
- } else {
- print $_;
- }
-}
-
-close TEMPLATE;
-close CONFIG;
diff --git a/build-aux/augeas-gentest.py b/build-aux/augeas-gentest.py
new file mode 100755
index 0000000000..18c6490a08
--- /dev/null
+++ b/build-aux/augeas-gentest.py
@@ -0,0 +1,72 @@
+#!/usr/bin/env python
+#
+# Copyright (C) 2012-2019 Red Hat, Inc.
+#
+# augeas-gentest.py: Generate an augeas test file, from an
+# example config file + test file template
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library. If not, see
+# .
+
+from __future__ import print_function
+
+import re
+import sys
+
+if len(sys.argv) !=3D 3:
+ print("syntax: %s CONFIG TEMPLATE" % sys.argv[0], file=3Dsys.stderr)
+ sys.exit(1)
+
+config =3D sys.argv[1]
+template =3D sys.argv[2]
+
+
+def expand_config(config):
+ with open(config) as fh:
+ optprog =3D re.compile(r'''^#\w.*''')
+ groupstartprog =3D re.compile(r'''.*\[\s$''')
+ groupendprog =3D re.compile(r'''#\s*\].*$''')
+
+ group =3D False
+ for line in fh:
+ if optprog.match(line) is not None:
+ line =3D line[1:]
+ line =3D line.replace('"', '\\"')
+ print(line, end=3D'')
+ if groupstartprog.match(line):
+ group =3D True
+ elif group:
+ line =3D line.replace('"', '\\"')
+
+ if groupendprog.match(line):
+ group =3D False
+
+ if line[0] =3D=3D '#':
+ line =3D line[1:]
+ print(line, end=3D'')
+
+
+def expand_template(template, config):
+ with open(template) as fh:
+ markerprog =3D re.compile(r'''\s*@CONFIG@\s*''')
+ for line in fh:
+ if markerprog.match(line) is not None:
+ print(' let conf =3D "', end=3D'')
+ expand_config(config)
+ print('"')
+ else:
+ print(line, end=3D'')
+
+
+expand_template(template, config)
diff --git a/src/Makefile.am b/src/Makefile.am
index cd955ee552..2956e4bf35 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -421,7 +421,8 @@ check-augeas: $(augeas_DATA) $(augeastest_DATA)
fi
.PHONY: check-augeas
=20
-AUG_GENTEST =3D $(PERL) $(top_srcdir)/build-aux/augeas-gentest.pl
+AUG_GENTEST_SCRIPT =3D $(top_srcdir)/build-aux/augeas-gentest.py
+AUG_GENTEST =3D $(RUNUTF8) $(PYTHON) $(AUG_GENTEST_SCRIPT)
=20
=20
#
diff --git a/src/bhyve/Makefile.inc.am b/src/bhyve/Makefile.inc.am
index 195069872a..7e49a8bb5c 100644
--- a/src/bhyve/Makefile.inc.am
+++ b/src/bhyve/Makefile.inc.am
@@ -77,7 +77,7 @@ bhyve/virtbhyved.aug: remote/libvirtd.aug.in
$< > $@
=20
bhyve/test_virtbhyved.aug: remote/test_libvirtd.aug.in \
- bhyve/virtbhyved.conf $(AUG_GENTEST)
+ bhyve/virtbhyved.conf $(AUG_GENTEST_SCRIPT)
$(AM_V_GEN)$(AUG_GENTEST) bhyve/virtbhyved.conf \
$(srcdir)/remote/test_libvirtd.aug.in | \
$(SED) \
@@ -91,7 +91,7 @@ augeas_DATA +=3D bhyve/libvirtd_bhyve.aug
augeastest_DATA +=3D bhyve/test_libvirtd_bhyve.aug
=20
bhyve/test_libvirtd_bhyve.aug: bhyve/test_libvirtd_bhyve.aug.in \
- $(srcdir)/bhyve/bhyve.conf $(AUG_GENTEST)
+ $(srcdir)/bhyve/bhyve.conf $(AUG_GENTEST_SCRIPT)
$(AM_V_GEN)$(AUG_GENTEST) $(srcdir)/bhyve/bhyve.conf $< > $@
=20
endif WITH_BHYVE
diff --git a/src/interface/Makefile.inc.am b/src/interface/Makefile.inc.am
index baa85b4ba9..7f161c487f 100644
--- a/src/interface/Makefile.inc.am
+++ b/src/interface/Makefile.inc.am
@@ -97,7 +97,7 @@ interface/virtinterfaced.aug: remote/libvirtd.aug.in
$< > $@
=20
interface/test_virtinterfaced.aug: remote/test_libvirtd.aug.in \
- interface/virtinterfaced.conf $(AUG_GENTEST)
+ interface/virtinterfaced.conf $(AUG_GENTEST_SCRIPT)
$(AM_V_GEN)$(AUG_GENTEST) interface/virtinterfaced.conf \
$(srcdir)/remote/test_libvirtd.aug.in | \
$(SED) \
diff --git a/src/libxl/Makefile.inc.am b/src/libxl/Makefile.inc.am
index c53396b7f3..3f35d1f2eb 100644
--- a/src/libxl/Makefile.inc.am
+++ b/src/libxl/Makefile.inc.am
@@ -111,7 +111,7 @@ libxl/virtxend.aug: remote/libvirtd.aug.in
$< > $@
=20
libxl/test_virtxend.aug: remote/test_libvirtd.aug.in \
- libxl/virtxend.conf $(AUG_GENTEST)
+ libxl/virtxend.conf $(AUG_GENTEST_SCRIPT)
$(AM_V_GEN)$(AUG_GENTEST) libxl/virtxend.conf \
$(srcdir)/remote/test_libvirtd.aug.in | \
$(SED) \
@@ -125,7 +125,7 @@ augeas_DATA +=3D libxl/libvirtd_libxl.aug
augeastest_DATA +=3D libxl/test_libvirtd_libxl.aug
=20
libxl/test_libvirtd_libxl.aug: libxl/test_libvirtd_libxl.aug.in \
- $(srcdir)/libxl/libxl.conf $(AUG_GENTEST)
+ $(srcdir)/libxl/libxl.conf $(AUG_GENTEST_SCRIPT)
$(AM_V_GEN)$(AUG_GENTEST) $(srcdir)/libxl/libxl.conf $< > $@
=20
INSTALL_DATA_DIRS +=3D libxl
diff --git a/src/locking/Makefile.inc.am b/src/locking/Makefile.inc.am
index fae92a6e45..df0c51aaf6 100644
--- a/src/locking/Makefile.inc.am
+++ b/src/locking/Makefile.inc.am
@@ -219,7 +219,7 @@ endif WITH_SANLOCK
if WITH_SANLOCK
if WITH_QEMU
locking/test_libvirt_sanlock.aug: locking/test_libvirt_sanlock.aug.in \
- locking/qemu-sanlock.conf $(AUG_GENTEST)
+ locking/qemu-sanlock.conf $(AUG_GENTEST_SCRIPT)
$(AM_V_GEN)$(AUG_GENTEST) locking/qemu-sanlock.conf $< > $@
=20
endif WITH_QEMU
@@ -227,12 +227,12 @@ endif WITH_SANLOCK
=20
if WITH_QEMU
locking/test_libvirt_lockd.aug: locking/test_libvirt_lockd.aug.in \
- locking/qemu-lockd.conf $(AUG_GENTEST)
+ locking/qemu-lockd.conf $(AUG_GENTEST_SCRIPT)
$(AM_V_GEN)$(AUG_GENTEST) locking/qemu-lockd.conf $< > $@
endif WITH_QEMU
=20
locking/test_virtlockd.aug: locking/test_virtlockd.aug.in \
- locking/virtlockd.conf $(AUG_GENTEST)
+ locking/virtlockd.conf $(AUG_GENTEST_SCRIPT)
$(AM_V_GEN)$(AUG_GENTEST) $(srcdir)/locking/virtlockd.conf $< > $@
=20
endif WITH_LIBVIRTD
diff --git a/src/logging/Makefile.inc.am b/src/logging/Makefile.inc.am
index 7e441dbffb..ce777e278a 100644
--- a/src/logging/Makefile.inc.am
+++ b/src/logging/Makefile.inc.am
@@ -101,7 +101,7 @@ augeas_DATA +=3D logging/virtlogd.aug
augeastest_DATA +=3D logging/test_virtlogd.aug
=20
logging/test_virtlogd.aug: logging/test_virtlogd.aug.in \
- logging/virtlogd.conf $(AUG_GENTEST)
+ logging/virtlogd.conf $(AUG_GENTEST_SCRIPT)
$(AM_V_GEN)$(AUG_GENTEST) $(srcdir)/logging/virtlogd.conf $< > $@
=20
endif WITH_LIBVIRTD
diff --git a/src/lxc/Makefile.inc.am b/src/lxc/Makefile.inc.am
index b4d560702c..0e12800886 100644
--- a/src/lxc/Makefile.inc.am
+++ b/src/lxc/Makefile.inc.am
@@ -164,7 +164,7 @@ lxc/virtlxcd.aug: remote/libvirtd.aug.in
$< > $@
=20
lxc/test_virtlxcd.aug: remote/test_libvirtd.aug.in \
- lxc/virtlxcd.conf $(AUG_GENTEST)
+ lxc/virtlxcd.conf $(AUG_GENTEST_SCRIPT)
$(AM_V_GEN)$(AUG_GENTEST) lxc/virtlxcd.conf \
$(srcdir)/remote/test_libvirtd.aug.in | \
$(SED) \
@@ -222,7 +222,7 @@ augeas_DATA +=3D lxc/libvirtd_lxc.aug
augeastest_DATA +=3D lxc/test_libvirtd_lxc.aug
=20
lxc/test_libvirtd_lxc.aug: lxc/test_libvirtd_lxc.aug.in \
- $(srcdir)/lxc/lxc.conf $(AUG_GENTEST)
+ $(srcdir)/lxc/lxc.conf $(AUG_GENTEST_SCRIPT)
$(AM_V_GEN)$(AUG_GENTEST) $(srcdir)/lxc/lxc.conf $< > $@
=20
INSTALL_DATA_DIRS +=3D lxc
diff --git a/src/network/Makefile.inc.am b/src/network/Makefile.inc.am
index 17467a65ad..0ac0e03957 100644
--- a/src/network/Makefile.inc.am
+++ b/src/network/Makefile.inc.am
@@ -104,7 +104,7 @@ network/virtnetworkd.aug: remote/libvirtd.aug.in
$< > $@
=20
network/test_virtnetworkd.aug: remote/test_libvirtd.aug.in \
- network/virtnetworkd.conf $(AUG_GENTEST)
+ network/virtnetworkd.conf $(AUG_GENTEST_SCRIPT)
$(AM_V_GEN)$(AUG_GENTEST) network/virtnetworkd.conf \
$(srcdir)/remote/test_libvirtd.aug.in | \
$(SED) \
diff --git a/src/node_device/Makefile.inc.am b/src/node_device/Makefile.inc=
.am
index eac7f92e88..a42cf8bc6a 100644
--- a/src/node_device/Makefile.inc.am
+++ b/src/node_device/Makefile.inc.am
@@ -120,7 +120,7 @@ node_device/virtnodedevd.aug: remote/libvirtd.aug.in
$< > $@
=20
node_device/test_virtnodedevd.aug: remote/test_libvirtd.aug.in \
- node_device/virtnodedevd.conf $(AUG_GENTEST)
+ node_device/virtnodedevd.conf $(AUG_GENTEST_SCRIPT)
$(AM_V_GEN)$(AUG_GENTEST) node_device/virtnodedevd.conf \
$(srcdir)/remote/test_libvirtd.aug.in | \
$(SED) \
diff --git a/src/nwfilter/Makefile.inc.am b/src/nwfilter/Makefile.inc.am
index 6acb45705c..45be065f7c 100644
--- a/src/nwfilter/Makefile.inc.am
+++ b/src/nwfilter/Makefile.inc.am
@@ -105,7 +105,7 @@ nwfilter/virtnwfilterd.aug: remote/libvirtd.aug.in
$< > $@
=20
nwfilter/test_virtnwfilterd.aug: remote/test_libvirtd.aug.in \
- nwfilter/virtnwfilterd.conf $(AUG_GENTEST)
+ nwfilter/virtnwfilterd.conf $(AUG_GENTEST_SCRIPT)
$(AM_V_GEN)$(AUG_GENTEST) nwfilter/virtnwfilterd.conf \
$(srcdir)/remote/test_libvirtd.aug.in | \
$(SED) \
diff --git a/src/qemu/Makefile.inc.am b/src/qemu/Makefile.inc.am
index 48fd0332ec..e9794520cf 100644
--- a/src/qemu/Makefile.inc.am
+++ b/src/qemu/Makefile.inc.am
@@ -169,7 +169,7 @@ qemu/virtqemud.aug: remote/libvirtd.aug.in
$< > $@
=20
qemu/test_virtqemud.aug: remote/test_libvirtd.aug.in \
- qemu/virtqemud.conf $(AUG_GENTEST)
+ qemu/virtqemud.conf $(AUG_GENTEST_SCRIPT)
$(AM_V_GEN)$(AUG_GENTEST) qemu/virtqemud.conf \
$(srcdir)/remote/test_libvirtd.aug.in | \
$(SED) \
@@ -184,7 +184,7 @@ augeas_DATA +=3D qemu/libvirtd_qemu.aug
augeastest_DATA +=3D qemu/test_libvirtd_qemu.aug
=20
qemu/test_libvirtd_qemu.aug: qemu/test_libvirtd_qemu.aug.in \
- $(srcdir)/qemu/qemu.conf $(AUG_GENTEST)
+ $(srcdir)/qemu/qemu.conf $(AUG_GENTEST_SCRIPT)
$(AM_V_GEN)$(AUG_GENTEST) $(srcdir)/qemu/qemu.conf $< > $@
=20
INSTALL_DATA_DIRS +=3D qemu
diff --git a/src/remote/Makefile.inc.am b/src/remote/Makefile.inc.am
index 5a5c90a922..b42b700488 100644
--- a/src/remote/Makefile.inc.am
+++ b/src/remote/Makefile.inc.am
@@ -285,7 +285,7 @@ remote/virtproxyd.aug: remote/libvirtd.aug.in
$< > $@
=20
remote/test_libvirtd.aug: remote/test_libvirtd.aug.in \
- remote/libvirtd.conf $(AUG_GENTEST)
+ remote/libvirtd.conf $(AUG_GENTEST_SCRIPT)
$(AM_V_GEN)$(AUG_GENTEST) remote/libvirtd.conf \
$(srcdir)/remote/test_libvirtd.aug.in | \
$(SED) \
@@ -298,7 +298,7 @@ remote/test_libvirtd.aug: remote/test_libvirtd.aug.in \
> $@ || rm -f $@
=20
remote/test_virtproxyd.aug: remote/test_libvirtd.aug.in \
- remote/virtproxyd.conf $(AUG_GENTEST)
+ remote/virtproxyd.conf $(AUG_GENTEST_SCRIPT)
$(AM_V_GEN)$(AUG_GENTEST) remote/virtproxyd.conf \
$(srcdir)/remote/test_libvirtd.aug.in | \
$(SED) \
diff --git a/src/secret/Makefile.inc.am b/src/secret/Makefile.inc.am
index 76bc67418c..1020fb64c8 100644
--- a/src/secret/Makefile.inc.am
+++ b/src/secret/Makefile.inc.am
@@ -93,7 +93,7 @@ secret/virtsecretd.aug: remote/libvirtd.aug.in
$< > $@
=20
secret/test_virtsecretd.aug: remote/test_libvirtd.aug.in \
- secret/virtsecretd.conf $(AUG_GENTEST)
+ secret/virtsecretd.conf $(AUG_GENTEST_SCRIPT)
$(AM_V_GEN)$(AUG_GENTEST) secret/virtsecretd.conf \
$(srcdir)/remote/test_libvirtd.aug.in | \
$(SED) \
diff --git a/src/storage/Makefile.inc.am b/src/storage/Makefile.inc.am
index 4dccb14ac1..28ddfeec72 100644
--- a/src/storage/Makefile.inc.am
+++ b/src/storage/Makefile.inc.am
@@ -197,7 +197,7 @@ storage/virtstoraged.aug: remote/libvirtd.aug.in
$< > $@
=20
storage/test_virtstoraged.aug: remote/test_libvirtd.aug.in \
- storage/virtstoraged.conf $(AUG_GENTEST)
+ storage/virtstoraged.conf $(AUG_GENTEST_SCRIPT)
$(AM_V_GEN)$(AUG_GENTEST) storage/virtstoraged.conf \
$(srcdir)/remote/test_libvirtd.aug.in | \
$(SED) \
diff --git a/src/vbox/Makefile.inc.am b/src/vbox/Makefile.inc.am
index 178c360b99..1f2bcb5488 100644
--- a/src/vbox/Makefile.inc.am
+++ b/src/vbox/Makefile.inc.am
@@ -117,7 +117,7 @@ vbox/virtvboxd.aug: remote/libvirtd.aug.in
$< > $@
=20
vbox/test_virtvboxd.aug: remote/test_libvirtd.aug.in \
- vbox/virtvboxd.conf $(AUG_GENTEST)
+ vbox/virtvboxd.conf $(AUG_GENTEST_SCRIPT)
$(AM_V_GEN)$(AUG_GENTEST) vbox/virtvboxd.conf \
$(srcdir)/remote/test_libvirtd.aug.in | \
$(SED) \
diff --git a/src/vz/Makefile.inc.am b/src/vz/Makefile.inc.am
index f56fceb8f7..fa7b8b79dd 100644
--- a/src/vz/Makefile.inc.am
+++ b/src/vz/Makefile.inc.am
@@ -91,7 +91,7 @@ vz/virtvzd.aug: remote/libvirtd.aug.in
$< > $@
=20
vz/test_virtvzd.aug: remote/test_libvirtd.aug.in \
- vz/virtvzd.conf $(AUG_GENTEST)
+ vz/virtvzd.conf $(AUG_GENTEST_SCRIPT)
$(AM_V_GEN)$(AUG_GENTEST) vz/virtvzd.conf \
$(srcdir)/remote/test_libvirtd.aug.in | \
$(SED) \
--=20
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
From nobody Thu May 2 16:33:49 2024
Delivered-To: importer@patchew.org
Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28
as permitted sender) client-ip=209.132.183.28;
envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com;
Authentication-Results: mx.zohomail.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass(p=none dis=none) header.from=redhat.com
ARC-Seal: i=1; a=rsa-sha256; t=1569337475; cv=none;
d=zoho.com; s=zohoarc;
b=Jp421vHRI1W0AvegKNJmopGC22y4A/7pbW3u3bYo4/VmdjNNFbVZSWH+UmMkI4nCgSRDNc/7giBW+3QHXBihwO5uN1wTPscPdpxrUIsTovTiscWYfgNDjUJUPGtFEe63eMaAhdzlBrJKP7wkvDSlszA3aZ0EG3uKJD8rvaaArho=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com;
s=zohoarc;
t=1569337475;
h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results;
bh=ouf3bcfkE0m/YGWVLf3mpJkp01s3d8tv7SVdaoyYPKE=;
b=ER6d81NZ5RqEWmMyLkiFOsayxBUDuaPx8G8XkuOENg15ra+qO7tOlQ7pR52lS1mQPTf/1Ot18UzSF6rQmW0rzgL1f/gHVwz0sdNRjozJpwIfKlGS5bozNX4TO7rlkpntDqa411TWaK2UJR6aTebt/6LCK7GCDjWM80W9N13UPFY=
ARC-Authentication-Results: i=1; mx.zoho.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass header.from= (p=none dis=none)
header.from=
Return-Path:
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by
mx.zohomail.com
with SMTPS id 156933747562331.26717819008195;
Tue, 24 Sep 2019 08:04:35 -0700 (PDT)
Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com
[10.5.11.16])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by mx1.redhat.com (Postfix) with ESMTPS id 044F910C0947;
Tue, 24 Sep 2019 15:04:34 +0000 (UTC)
Received: from colo-mx.corp.redhat.com
(colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21])
by smtp.corp.redhat.com (Postfix) with ESMTPS id C93CE5C21F;
Tue, 24 Sep 2019 15:04:33 +0000 (UTC)
Received: from lists01.pubmisc.prod.ext.phx2.redhat.com
(lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33])
by colo-mx.corp.redhat.com (Postfix) with ESMTP id 74F774EE6D;
Tue, 24 Sep 2019 15:04:33 +0000 (UTC)
Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com
[10.5.11.14])
by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
id x8OF47oL027825 for ;
Tue, 24 Sep 2019 11:04:07 -0400
Received: by smtp.corp.redhat.com (Postfix)
id 084AC5D9E1; Tue, 24 Sep 2019 15:04:07 +0000 (UTC)
Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com
[10.33.9.32])
by smtp.corp.redhat.com (Postfix) with ESMTP id 6D5D75D9D5;
Tue, 24 Sep 2019 15:04:06 +0000 (UTC)
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?=
To: libvir-list@redhat.com
Date: Tue, 24 Sep 2019 15:58:44 +0100
Message-Id: <20190924145903.17123-4-berrange@redhat.com>
In-Reply-To: <20190924145903.17123-1-berrange@redhat.com>
References: <20190924145903.17123-1-berrange@redhat.com>
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14
X-loop: libvir-list@redhat.com
Subject: [libvirt] [PATCH v3 03/22] build-aux: rewrite po file minimizer in
Python
X-BeenThere: libvir-list@redhat.com
X-Mailman-Version: 2.1.12
Precedence: junk
List-Id: Development discussions about the libvirt library & tools
List-Unsubscribe: ,
List-Archive:
List-Post:
List-Help:
List-Subscribe: ,
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Sender: libvir-list-bounces@redhat.com
Errors-To: libvir-list-bounces@redhat.com
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16
X-Greylist: Sender IP whitelisted,
not delayed by milter-greylist-4.6.2 (mx1.redhat.com [10.5.110.66]);
Tue, 24 Sep 2019 15:04:34 +0000 (UTC)
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the minimize-po.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Signed-off-by: Daniel P. Berrang=C3=A9
---
Makefile.am | 2 +-
build-aux/minimize-po.pl | 37 -------------------------
build-aux/minimize-po.py | 60 ++++++++++++++++++++++++++++++++++++++++
po/Makefile.am | 2 +-
4 files changed, 62 insertions(+), 39 deletions(-)
delete mode 100755 build-aux/minimize-po.pl
create mode 100755 build-aux/minimize-po.py
diff --git a/Makefile.am b/Makefile.am
index 17448a914e..8f688d40d0 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -45,7 +45,7 @@ EXTRA_DIST =3D \
build-aux/check-spacing.pl \
build-aux/gitlog-to-changelog \
build-aux/header-ifdef.pl \
- build-aux/minimize-po.pl \
+ build-aux/minimize-po.py \
build-aux/mock-noinline.pl \
build-aux/prohibit-duplicate-header.pl \
build-aux/useless-if-before-free \
diff --git a/build-aux/minimize-po.pl b/build-aux/minimize-po.pl
deleted file mode 100755
index 497533a836..0000000000
--- a/build-aux/minimize-po.pl
+++ /dev/null
@@ -1,37 +0,0 @@
-#!/usr/bin/perl
-
-my @block;
-my $msgstr =3D 0;
-my $empty =3D 0;
-my $unused =3D 0;
-my $fuzzy =3D 0;
-while (<>) {
- if (/^$/) {
- if (!$empty && !$unused && !$fuzzy) {
- print @block;
- }
- @block =3D ();
- $msgstr =3D 0;
- $fuzzy =3D 0;
- push @block, $_;
- } else {
- if (/^msgstr/) {
- $msgstr =3D 1;
- $empty =3D 1;
- }
- if (/^#.*fuzzy/) {
- $fuzzy =3D 1;
- }
- if (/^#~ msgstr/) {
- $unused =3D 1;
- }
- if ($msgstr && /".+"/) {
- $empty =3D 0;
- }
- push @block, $_;
- }
-}
-
-if (@block && !$empty && !$unused) {
- print @block;
-}
diff --git a/build-aux/minimize-po.py b/build-aux/minimize-po.py
new file mode 100755
index 0000000000..5046bacede
--- /dev/null
+++ b/build-aux/minimize-po.py
@@ -0,0 +1,60 @@
+#!/usr/bin/env python
+#
+# Copyright (C) 2018-2019 Red Hat, Inc.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library. If not, see
+# .
+
+from __future__ import print_function
+
+import re
+import sys
+
+block =3D []
+msgstr =3D False
+empty =3D False
+unused =3D False
+fuzzy =3D False
+
+strprog =3D re.compile(r'''.*".+".*''')
+
+for line in sys.stdin:
+ line =3D line.rstrip("\n")
+
+ if line =3D=3D "":
+ if not empty and not unused and not fuzzy:
+ for b in block:
+ print(b)
+
+ block =3D []
+ msgstr =3D False
+ fuzzy =3D False
+ block.append(line)
+ else:
+ if line.startswith("msgstr"):
+ msgstr =3D True
+ empty =3D True
+
+ if line[0] =3D=3D '#' and line.find("fuzzy") !=3D -1:
+ fuzzy =3D True
+ if line.startswith("#~ msgstr"):
+ unused =3D True
+ if msgstr and strprog.match(line):
+ empty =3D False
+
+ block.append(line)
+
+if not empty and not unused and not fuzzy:
+ for b in block:
+ print(b)
diff --git a/po/Makefile.am b/po/Makefile.am
index da117edbd5..90025b15dd 100644
--- a/po/Makefile.am
+++ b/po/Makefile.am
@@ -58,7 +58,7 @@ update-mini-po: $(POTFILE)
$(MSGMERGE) --no-location --no-fuzzy-matching --sort-output \
$$lang.po $(POTFILE) | \
$(SED) $(SED_PO_FIXUP_ARGS) | \
- $(PERL) $(top_srcdir)/build-aux/minimize-po.pl > \
+ $(RUNUTF8) $(PYTHON) $(top_srcdir)/build-aux/minimize-po.py > \
$(srcdir)/$$lang.mini.po ; \
done
=20
--=20
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
From nobody Thu May 2 16:33:49 2024
Delivered-To: importer@patchew.org
Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28
as permitted sender) client-ip=209.132.183.28;
envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com;
Authentication-Results: mx.zohomail.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass(p=none dis=none) header.from=redhat.com
ARC-Seal: i=1; a=rsa-sha256; t=1569337453; cv=none;
d=zoho.com; s=zohoarc;
b=dc+AXhchjLQTJF1+m2ySDd3M5nCbwA06fFVBLIG9sw7dnH+HFeuCZ/TG4d7veedhwqnCCn5f3IoXvSeEKZa2bulb1bP1N+FzrQm3wtOr+O70D9e01OAdCjMb1jOauX3T9RZh7bMBU4Hp3FlpN4fbcODDYgy626KYOHnwAYzwajU=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com;
s=zohoarc;
t=1569337453;
h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results;
bh=hCjNu0C7Pcwp3I41BZkBq5X5Et53EasgscbbDSHcea4=;
b=dVUEYdOh79EdH82isYflVZ26OT1RJd/VBuxn9mAzJTDq2T5Z2xgjV+AlDZl3/7yINIBdxNU6mS/sPhtU7L930hIbcezuIW3v5OEztLS2LfW5Q7CYcOu3Wj/MnSNNAwqXPQzvbqCI38IHVWVoBlq9kvtpGXi6QLTnvJhTzHGkTvI=
ARC-Authentication-Results: i=1; mx.zoho.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass header.from= (p=none dis=none)
header.from=
Return-Path:
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by
mx.zohomail.com
with SMTPS id 1569337453181140.94776536948768;
Tue, 24 Sep 2019 08:04:13 -0700 (PDT)
Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com
[10.5.11.15])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by mx1.redhat.com (Postfix) with ESMTPS id 5B244C01DDF8;
Tue, 24 Sep 2019 15:04:11 +0000 (UTC)
Received: from colo-mx.corp.redhat.com
(colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21])
by smtp.corp.redhat.com (Postfix) with ESMTPS id 304FC5B6A5;
Tue, 24 Sep 2019 15:04:11 +0000 (UTC)
Received: from lists01.pubmisc.prod.ext.phx2.redhat.com
(lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33])
by colo-mx.corp.redhat.com (Postfix) with ESMTP id E1C6E4EE69;
Tue, 24 Sep 2019 15:04:10 +0000 (UTC)
Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com
[10.5.11.14])
by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
id x8OF4AdA027844 for ;
Tue, 24 Sep 2019 11:04:10 -0400
Received: by smtp.corp.redhat.com (Postfix)
id 20F085D9E2; Tue, 24 Sep 2019 15:04:10 +0000 (UTC)
Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com
[10.33.9.32])
by smtp.corp.redhat.com (Postfix) with ESMTP id 4D1A85D9D5;
Tue, 24 Sep 2019 15:04:07 +0000 (UTC)
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?=
To: libvir-list@redhat.com
Date: Tue, 24 Sep 2019 15:58:45 +0100
Message-Id: <20190924145903.17123-5-berrange@redhat.com>
In-Reply-To: <20190924145903.17123-1-berrange@redhat.com>
References: <20190924145903.17123-1-berrange@redhat.com>
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14
X-loop: libvir-list@redhat.com
Subject: [libvirt] [PATCH v3 04/22] build-aux: rewrite duplicate header
checker in Python
X-BeenThere: libvir-list@redhat.com
X-Mailman-Version: 2.1.12
Precedence: junk
List-Id: Development discussions about the libvirt library & tools
List-Unsubscribe: ,
List-Archive:
List-Post:
List-Help:
List-Subscribe: ,
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Sender: libvir-list-bounces@redhat.com
Errors-To: libvir-list-bounces@redhat.com
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15
X-Greylist: Sender IP whitelisted,
not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.31]);
Tue, 24 Sep 2019 15:04:11 +0000 (UTC)
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the prohibit-duplicate-header.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Signed-off-by: Daniel P. Berrang=C3=A9
Reviewed-by: Erik Skultety
---
Makefile.am | 2 +-
build-aux/prohibit-duplicate-header.pl | 26 ------------
build-aux/prohibit-duplicate-header.py | 57 ++++++++++++++++++++++++++
cfg.mk | 4 +-
4 files changed, 60 insertions(+), 29 deletions(-)
delete mode 100644 build-aux/prohibit-duplicate-header.pl
create mode 100644 build-aux/prohibit-duplicate-header.py
diff --git a/Makefile.am b/Makefile.am
index 8f688d40d0..a7d8da7146 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -47,7 +47,7 @@ EXTRA_DIST =3D \
build-aux/header-ifdef.pl \
build-aux/minimize-po.py \
build-aux/mock-noinline.pl \
- build-aux/prohibit-duplicate-header.pl \
+ build-aux/prohibit-duplicate-header.py \
build-aux/useless-if-before-free \
build-aux/vc-list-files \
ci/Makefile \
diff --git a/build-aux/prohibit-duplicate-header.pl b/build-aux/prohibit-du=
plicate-header.pl
deleted file mode 100644
index 4a2ea65665..0000000000
--- a/build-aux/prohibit-duplicate-header.pl
+++ /dev/null
@@ -1,26 +0,0 @@
-#!/usr/bin/env perl
-
-use strict;
-
-my $file =3D " ";
-my $ret =3D 0;
-my %includes =3D ( );
-my $lineno =3D 0;
-
-while (<>) {
- if (not $file eq $ARGV) {
- %includes =3D ( );
- $file =3D $ARGV;
- $lineno =3D 0;
- }
- $lineno++;
- if (/^# *include *[<"]([^>"]*\.h)[">]/) {
- $includes{$1}++;
- if ($includes{$1} =3D=3D 2) {
- $ret =3D 1;
- print STDERR "$ARGV:$lineno: $_";
- print STDERR "Do not include a header more than once per file\=
n";
- }
- }
-}
-exit $ret;
diff --git a/build-aux/prohibit-duplicate-header.py b/build-aux/prohibit-du=
plicate-header.py
new file mode 100644
index 0000000000..5ed3f82ba7
--- /dev/null
+++ b/build-aux/prohibit-duplicate-header.py
@@ -0,0 +1,57 @@
+#!/usr/bin/env python
+#
+# Copyright (C) 2016-2019 Red Hat, Inc.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library. If not, see
+# .
+
+from __future__ import print_function
+
+import re
+import sys
+
+
+def check_file(filename):
+ includes =3D {}
+ lineno =3D 0
+ errs =3D False
+ with open(filename, "r") as fh:
+ headerprog =3D re.compile(r'''^# *include *[<"]([^>"]*\.h)[">]\s*$=
''')
+ for line in fh:
+ lineno =3D lineno + 1
+
+ headermatch =3D headerprog.match(line)
+ if headermatch is not None:
+ inc =3D headermatch.group(1)
+
+ if inc in includes:
+ print("%s:%d: %s" % (filename, lineno, inc),
+ file=3Dsys.stderr)
+ errs =3D True
+ else:
+ includes[inc] =3D True
+
+ return errs
+
+
+ret =3D 0
+
+for filename in sys.argv[1:]:
+ if check_file(filename):
+ ret =3D 1
+
+if ret =3D=3D 1:
+ print("Do not include a header more than once per file", file=3Dsys.st=
derr)
+
+sys.exit(ret)
diff --git a/cfg.mk b/cfg.mk
index 3731a2830b..bf4a8270d8 100644
--- a/cfg.mk
+++ b/cfg.mk
@@ -1152,8 +1152,8 @@ endif
=20
# Don't include duplicate header in the source (either *.c or *.h)
prohibit-duplicate-header:
- $(AM_V_GEN)$(VC_LIST_EXCEPT) | $(GREP) '\.[chx]$$' | xargs \
- $(PERL) -W $(top_srcdir)/build-aux/prohibit-duplicate-header.pl
+ $(AM_V_GEN)$(VC_LIST_EXCEPT) | $(GREP) '\.[chx]$$' | $(RUNUTF8) xargs \
+ $(PYTHON) $(top_srcdir)/build-aux/prohibit-duplicate-header.py
=20
spacing-check:
$(AM_V_GEN)$(VC_LIST) | $(GREP) '\.c$$' | xargs \
--=20
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
From nobody Thu May 2 16:33:49 2024
Delivered-To: importer@patchew.org
Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28
as permitted sender) client-ip=209.132.183.28;
envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com;
Authentication-Results: mx.zohomail.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass(p=none dis=none) header.from=redhat.com
ARC-Seal: i=1; a=rsa-sha256; t=1569337470; cv=none;
d=zoho.com; s=zohoarc;
b=PJK6m+kJOh+mHq4wAHHN80n8ABQJxv9YMskm9BXHyrJb/7P15IaEgqCOhkzkgWubw891O9T6UTz+qxzh3T9L9t1LDeR7h5FwlLME66ZALc4AB1fp0p55d3Z0KNBW/ulo/sv25sNep8dFAxu1WsoRVDBEzhUbCBdROPbqscFo1E4=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com;
s=zohoarc;
t=1569337470;
h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results;
bh=pY7OUDdHy1QVeDtKJ5CB59Lq7xcirCWax76YvNlrg08=;
b=WJPJf4a399bXzL+KM87nCbJbo43aODekkJ6dWI74oYQZbcxAVYCC9or92KkSCrbJ94BQ9hICoM3WNbhRqUvHQkrW9vvbz0BEwFy4uoNZzrwOyZgRq7qH7rIPntRS/4Q9/YPiqOhrQwHWkp49YeEy3E1GdPjLcyTptAdsE4hWgEg=
ARC-Authentication-Results: i=1; mx.zoho.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass header.from= (p=none dis=none)
header.from=
Return-Path:
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by
mx.zohomail.com
with SMTPS id 1569337470706831.6117514209964;
Tue, 24 Sep 2019 08:04:30 -0700 (PDT)
Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com
[10.5.11.14])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by mx1.redhat.com (Postfix) with ESMTPS id F180A3091D69;
Tue, 24 Sep 2019 15:04:28 +0000 (UTC)
Received: from colo-mx.corp.redhat.com
(colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20])
by smtp.corp.redhat.com (Postfix) with ESMTPS id C7AB25D9DC;
Tue, 24 Sep 2019 15:04:28 +0000 (UTC)
Received: from lists01.pubmisc.prod.ext.phx2.redhat.com
(lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33])
by colo-mx.corp.redhat.com (Postfix) with ESMTP id 8F7EC1800B74;
Tue, 24 Sep 2019 15:04:28 +0000 (UTC)
Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com
[10.5.11.14])
by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
id x8OF4BCO027850 for ;
Tue, 24 Sep 2019 11:04:11 -0400
Received: by smtp.corp.redhat.com (Postfix)
id 2601D5DA21; Tue, 24 Sep 2019 15:04:11 +0000 (UTC)
Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com
[10.33.9.32])
by smtp.corp.redhat.com (Postfix) with ESMTP id 66EC75D9E2;
Tue, 24 Sep 2019 15:04:10 +0000 (UTC)
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?=
To: libvir-list@redhat.com
Date: Tue, 24 Sep 2019 15:58:46 +0100
Message-Id: <20190924145903.17123-6-berrange@redhat.com>
In-Reply-To: <20190924145903.17123-1-berrange@redhat.com>
References: <20190924145903.17123-1-berrange@redhat.com>
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14
X-loop: libvir-list@redhat.com
Subject: [libvirt] [PATCH v3 05/22] build-aux: rewrite whitespace checker in
Python
X-BeenThere: libvir-list@redhat.com
X-Mailman-Version: 2.1.12
Precedence: junk
List-Id: Development discussions about the libvirt library & tools
List-Unsubscribe: ,
List-Archive:
List-Post:
List-Help:
List-Subscribe: ,
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Sender: libvir-list-bounces@redhat.com
Errors-To: libvir-list-bounces@redhat.com
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14
X-Greylist: Sender IP whitelisted,
not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.42]);
Tue, 24 Sep 2019 15:04:29 +0000 (UTC)
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the check-spacing.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Signed-off-by: Daniel P. Berrang=C3=A9
---
Makefile.am | 2 +-
build-aux/check-spacing.pl | 198 --------------------------------
build-aux/check-spacing.py | 229 +++++++++++++++++++++++++++++++++++++
cfg.mk | 4 +-
4 files changed, 232 insertions(+), 201 deletions(-)
delete mode 100755 build-aux/check-spacing.pl
create mode 100755 build-aux/check-spacing.py
diff --git a/Makefile.am b/Makefile.am
index a7d8da7146..03bf1beb78 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -42,7 +42,7 @@ EXTRA_DIST =3D \
README.md \
AUTHORS.in \
build-aux/augeas-gentest.py \
- build-aux/check-spacing.pl \
+ build-aux/check-spacing.py \
build-aux/gitlog-to-changelog \
build-aux/header-ifdef.pl \
build-aux/minimize-po.py \
diff --git a/build-aux/check-spacing.pl b/build-aux/check-spacing.pl
deleted file mode 100755
index 33377f3dd3..0000000000
--- a/build-aux/check-spacing.pl
+++ /dev/null
@@ -1,198 +0,0 @@
-#!/usr/bin/env perl
-#
-# check-spacing.pl: Report any usage of 'function (..args..)'
-# Also check for other syntax issues, such as correct use of ';'
-#
-# This library is free software; you can redistribute it and/or
-# modify it under the terms of the GNU Lesser General Public
-# License as published by the Free Software Foundation; either
-# version 2.1 of the License, or (at your option) any later version.
-#
-# This library is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-# Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public
-# License along with this library. If not, see
-# .
-
-use strict;
-use warnings;
-
-my $ret =3D 0;
-my $incomment =3D 0;
-
-foreach my $file (@ARGV) {
- # Per-file variables for multiline Curly Bracket (cb_) check
- my $cb_linenum =3D 0;
- my $cb_code =3D "";
- my $cb_scolon =3D 0;
-
- open FILE, $file;
-
- while (defined (my $line =3D )) {
- my $data =3D $line;
- # For temporary modifications
- my $tmpdata;
-
- # Kill any quoted , ; =3D or "
- $data =3D~ s/'[";,=3D]'/'X'/g;
-
- # Kill any quoted strings
- $data =3D~ s,"(?:[^\\\"]|\\.)*","XXX",g;
-
- next if $data =3D~ /^#/;
-
- # Kill contents of multi-line comments
- # and detect end of multi-line comments
- if ($incomment) {
- if ($data =3D~ m,\*/,) {
- $incomment =3D 0;
- $data =3D~ s,^.*\*/,*/,;
- } else {
- $data =3D "";
- }
- }
-
- # Kill single line comments, and detect
- # start of multi-line comments
- if ($data =3D~ m,/\*.*\*/,) {
- $data =3D~ s,/\*.*\*/,/* */,;
- } elsif ($data =3D~ m,/\*,) {
- $incomment =3D 1;
- $data =3D~ s,/\*.*,/*,;
- }
-
- # We need to match things like
- #
- # int foo (int bar, bool wizz);
- # foo (bar, wizz);
- #
- # but not match things like:
- #
- # typedef int (*foo)(bar wizz)
- #
- # we can't do this (efficiently) without
- # missing things like
- #
- # foo (*bar, wizz);
- #
- # We also don't want to spoil the $data so it can be used
- # later on.
- $tmpdata =3D $data;
- while ($tmpdata =3D~ /(\w+)\s\((?!\*)/) {
- my $kw =3D $1;
-
- # Allow space after keywords only
- if ($kw =3D~ /^(?:if|for|while|switch|return)$/) {
- $tmpdata =3D~ s/(?:$kw\s\()/XXX(/;
- } else {
- print "Whitespace after non-keyword:\n";
- print "$file:$.: $line";
- $ret =3D 1;
- last;
- }
- }
-
- # Require whitespace immediately after keywords
- if ($data =3D~ /\b(?:if|for|while|switch|return)\(/) {
- print "No whitespace after keyword:\n";
- print "$file:$.: $line";
- $ret =3D 1;
- }
-
- # Forbid whitespace between )( of a function typedef
- if ($data =3D~ /\(\*\w+\)\s+\(/) {
- print "Whitespace between ')' and '(':\n";
- print "$file:$.: $line";
- $ret =3D 1;
- }
-
- # Forbid whitespace following ( or prior to )
- # but allow whitespace before ) on a single line
- # (optionally followed by a semicolon)
- if (($data =3D~ /\s\)/ && not $data =3D~ /^\s+\);?$/) ||
- $data =3D~ /\((?!$)\s/) {
- print "Whitespace after '(' or before ')':\n";
- print "$file:$.: $line";
- $ret =3D 1;
- }
-
- # Forbid whitespace before ";" or ",". Things like below are allow=
ed:
- #
- # 1) The expression is empty for "for" loop. E.g.
- # for (i =3D 0; ; i++)
- #
- # 2) An empty statement. E.g.
- # while (write(statuswrite, &status, 1) =3D=3D -1 &&
- # errno =3D=3D EINTR)
- # ;
- #
- if ($data =3D~ /\s[;,]/) {
- unless ($data =3D~ /\S; ; / ||
- $data =3D~ /^\s+;/) {
- print "Whitespace before semicolon or comma:\n";
- print "$file:$.: $line";
- $ret =3D 1;
- }
- }
-
- # Require EOL, macro line continuation, or whitespace after ";".
- # Allow "for (;;)" as an exception.
- if ($data =3D~ /;[^ \\\n;)]/) {
- print "Invalid character after semicolon:\n";
- print "$file:$.: $line";
- $ret =3D 1;
- }
-
- # Require EOL, space, or enum/struct end after comma.
- if ($data =3D~ /,[^ \\\n)}]/) {
- print "Invalid character after comma:\n";
- print "$file:$.: $line";
- $ret =3D 1;
- }
-
- # Require spaces around assignment '=3D', compounds and '=3D=3D'
- if ($data =3D~ /[^ ]\b[!<>&|\-+*\/%\^=3D]?=3D/ ||
- $data =3D~ /=3D[^=3D \\\n]/) {
- print "Spacing around '=3D' or '=3D=3D':\n";
- print "$file:$.: $line";
- $ret =3D 1;
- }
-
- # One line conditional statements with one line bodies should
- # not use curly brackets.
- if ($data =3D~ /^\s*(if|while|for)\b.*\{$/) {
- $cb_linenum =3D $.;
- $cb_code =3D $line;
- $cb_scolon =3D 0;
- }
-
- # We need to check for exactly one semicolon inside the body,
- # because empty statements (e.g. with comment only) are
- # allowed
- if ($cb_linenum =3D=3D $. - 1 && $data =3D~ /^[^;]*;[^;]*$/) {
- $cb_code .=3D $line;
- $cb_scolon =3D 1;
- }
-
- if ($data =3D~ /^\s*}\s*$/ &&
- $cb_linenum =3D=3D $. - 2 &&
- $cb_scolon) {
-
- print "Curly brackets around single-line body:\n";
- print "$file:$cb_linenum-$.:\n$cb_code$line";
- $ret =3D 1;
-
- # There _should_ be no need to reset the values; but to
- # keep my inner peace...
- $cb_linenum =3D 0;
- $cb_scolon =3D 0;
- $cb_code =3D "";
- }
- }
- close FILE;
-}
-
-exit $ret;
diff --git a/build-aux/check-spacing.py b/build-aux/check-spacing.py
new file mode 100755
index 0000000000..6b9f3ec1ba
--- /dev/null
+++ b/build-aux/check-spacing.py
@@ -0,0 +1,229 @@
+#!/usr/bin/env python
+#
+# Copyright (C) 2012-2019 Red Hat, Inc.
+#
+# check-spacing.pl: Report any usage of 'function (..args..)'
+# Also check for other syntax issues, such as correct use of ';'
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library. If not, see
+# .
+
+from __future__ import print_function
+
+import re
+import sys
+
+
+def check_whitespace(filename):
+ errs =3D False
+ with open(filename, 'r') as fh:
+ quotedmetaprog =3D re.compile(r"""'[";,=3D]'""")
+ quotedstringprog =3D re.compile(r'''"(?:[^\\\"]|\\.)*"''')
+ commentstartprog =3D re.compile(r'''^(.*)/\*.*$''')
+ commentendprog =3D re.compile(r'''^.*\*/(.*)$''')
+ commentprog =3D re.compile(r'''^(.*)/\*.*\*/(.*)''')
+ funcprog =3D re.compile(r'''(\w+)\s\((?!\*)''')
+ keywordprog =3D re.compile(
+ r'''^.*\b(?:if|for|while|switch|return)\(.*$''')
+ functypedefprog =3D re.compile(r'''^.*\(\*\w+\)\s+\(.*$''')
+ whitespaceprog1 =3D re.compile(r'''^.*\s\).*$''')
+ whitespaceprog2 =3D re.compile(r'''^\s+\);?$''')
+ whitespaceprog3 =3D re.compile(r'''^.*\((?!$)\s.*''')
+ commasemiprog1 =3D re.compile(r'''.*\s[;,].*''')
+ commasemiprog2 =3D re.compile(r'''.*\S; ; .*''')
+ commasemiprog3 =3D re.compile(r'''^\s+;''')
+ semicolonprog =3D re.compile(r'''.*;[^ \\\n;)].*''')
+ commaprog =3D re.compile(r'''.*,[^ \\\n)}].*''')
+ assignprog1 =3D re.compile(r'''[^ ]\b[!<>&|\-+*\/%\^=3D]?=3D''')
+ assignprog2 =3D re.compile(r'''=3D[^=3D \\\n]''')
+ condstartprog =3D re.compile(r'''^\s*(if|while|for)\b.*\{$''')
+ statementprog =3D re.compile(r'''^[^;]*;[^;]*$''')
+ condendprog =3D re.compile(r'''^\s*}\s*$''')
+
+ incomment =3D False
+ # Per-file variables for multiline Curly Bracket (cb_) check
+ cb_lineno =3D 0
+ cb_code =3D ""
+ cb_scolon =3D False
+
+ lineno =3D 0
+ for line in fh:
+ lineno =3D lineno + 1
+ data =3D line
+ # For temporary modifications
+
+ # Kill any quoted , ; =3D or "
+ data =3D quotedmetaprog.sub("'X'", data)
+
+ # Kill any quoted strings
+ data =3D quotedstringprog.sub('"XXX"', data)
+
+ if data[0] =3D=3D '#':
+ continue
+
+ # Kill contents of multi-line comments
+ # and detect end of multi-line comments
+ if incomment:
+ if commentendprog.match(data):
+ data =3D commentendprog.sub('*/\2', data)
+ incomment =3D False
+ else:
+ data =3D ""
+
+ # Kill single line comments, and detect
+ # start of multi-line comments
+ if commentprog.match(data):
+ data =3D commentprog.sub(r'''\1/* */\2''', data)
+ elif commentstartprog.match(data):
+ data =3D commentstartprog.sub(r'''\1/*''', data)
+ incomment =3D True
+
+ # We need to match things like
+ #
+ # int foo (int bar, bool wizz);
+ # foo (bar, wizz);
+ #
+ # but not match things like:
+ #
+ # typedef int (*foo)(bar wizz)
+ #
+ # we can't do this (efficiently) without
+ # missing things like
+ #
+ # foo (*bar, wizz);
+ #
+ for match in funcprog.finditer(data):
+ kw =3D match.group(1)
+
+ # Allow space after keywords only
+ if kw not in ["if", "for", "while", "switch", "return"]:
+ print("Whitespace after non-keyword:",
+ file=3Dsys.stderr)
+ print("%s:%d: %s" % (filename, lineno, line),
+ file=3Dsys.stderr)
+ errs =3D True
+ break
+
+ # Require whitespace immediately after keywords
+ if keywordprog.match(data):
+ print("No whitespace after keyword:",
+ file=3Dsys.stderr)
+ print("%s:%d: %s" % (filename, lineno, line),
+ file=3Dsys.stderr)
+ errs =3D True
+
+ # Forbid whitespace between )( of a function typedef
+ if functypedefprog.match(data):
+ print("Whitespace between ')' and '(':",
+ file=3Dsys.stderr)
+ print("%s:%d: %s" % (filename, lineno, line),
+ file=3Dsys.stderr)
+ errs =3D True
+
+ # Forbid whitespace following ( or prior to )
+ # but allow whitespace before ) on a single line
+ # (optionally followed by a semicolon)
+ if ((whitespaceprog1.match(data) and
+ not whitespaceprog2.match(data))
+ or whitespaceprog3.match(data)):
+ print("Whitespace after '(' or before ')':",
+ file=3Dsys.stderr)
+ print("%s:%d: %s" % (filename, lineno, line),
+ file=3Dsys.stderr)
+ errs =3D True
+
+ # Forbid whitespace before ";" or ",". Things like
+ # below are allowed:
+ #
+ # 1) The expression is empty for "for" loop. E.g.
+ # for (i =3D 0; ; i++)
+ #
+ # 2) An empty statement. E.g.
+ # while (write(statuswrite, &status, 1) =3D=3D -1 &&
+ # errno =3D=3D EINTR)
+ # ;
+ #
+ if commasemiprog1.match(data) and not (
+ commasemiprog2.match(data) or
+ commasemiprog3.match(data)):
+ print("Whitespace before semicolon or comma:",
+ file=3Dsys.stderr)
+ print("%s:%d: %s" % (filename, lineno, line),
+ file=3Dsys.stderr)
+ errs =3D True
+
+ # Require EOL, macro line continuation, or whitespace after ";=
".
+ # Allow "for (;;)" as an exception.
+ if semicolonprog.match(data):
+ print("Invalid character after semicolon:",
+ file=3Dsys.stderr)
+ print("%s:%d: %s" % (filename, lineno, line),
+ file=3Dsys.stderr)
+ errs =3D True
+
+ # Require EOL, space, or enum/struct end after comma.
+ if commaprog.match(data):
+ print("Invalid character after comma:",
+ file=3Dsys.stderr)
+ print("%s:%d: %s" % (filename, lineno, line),
+ file=3Dsys.stderr)
+ errs =3D True
+
+ # Require spaces around assignment '=3D', compounds and '=3D=
=3D'
+ if assignprog1.match(data) or assignprog2.match(data):
+ print("Spacing around '=3D' or '=3D=3D':",
+ file=3Dsys.stderr)
+ print("%s:%d: %s" % (filename, lineno, line),
+ file=3Dsys.stderr)
+ errs =3D True
+
+ # One line conditional statements with one line bodies should
+ # not use curly brackets.
+ if condstartprog.match(data):
+ cb_lineno =3D lineno
+ cb_code =3D line
+ cb_scolon =3D False
+
+ # We need to check for exactly one semicolon inside the body,
+ # because empty statements (e.g. with comment only) are
+ # allowed
+ if (cb_lineno =3D=3D lineno - 1) and statementprog.match(data):
+ cb_code =3D cb_code + line
+ cb_scolon =3D True
+
+ if (condendprog.match(data) and
+ (cb_lineno =3D=3D lineno - 2) and
+ cb_scolon):
+ print("Curly brackets around single-line body:",
+ file=3Dsys.stderr)
+ print("%s:%d:\n%s%s" % (filename, cb_lineno - lineno,
+ cb_code, line),
+ file=3Dsys.stderr)
+ errs =3D True
+
+ # There _should_ be no need to reset the values; but to
+ # keep my inner peace...
+ cb_lineno =3D 0
+ cb_scolon =3D False
+ cb_code =3D ""
+
+ return errs
+
+
+ret =3D 0
+for filename in sys.argv[1:]:
+ if check_whitespace(filename):
+ ret =3D 1
+
+sys.exit(ret)
diff --git a/cfg.mk b/cfg.mk
index bf4a8270d8..d52d42c06c 100644
--- a/cfg.mk
+++ b/cfg.mk
@@ -1156,8 +1156,8 @@ prohibit-duplicate-header:
$(PYTHON) $(top_srcdir)/build-aux/prohibit-duplicate-header.py
=20
spacing-check:
- $(AM_V_GEN)$(VC_LIST) | $(GREP) '\.c$$' | xargs \
- $(PERL) $(top_srcdir)/build-aux/check-spacing.pl || \
+ $(AM_V_GEN)$(VC_LIST) | $(GREP) '\.c$$' | $(RUNUTF8) xargs \
+ $(PYTHON) $(top_srcdir)/build-aux/check-spacing.py || \
{ echo '$(ME): incorrect formatting' 1>&2; exit 1; }
=20
mock-noinline:
--=20
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
From nobody Thu May 2 16:33:49 2024
Delivered-To: importer@patchew.org
Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28
as permitted sender) client-ip=209.132.183.28;
envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com;
Authentication-Results: mx.zohomail.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass(p=none dis=none) header.from=redhat.com
ARC-Seal: i=1; a=rsa-sha256; t=1569337474; cv=none;
d=zoho.com; s=zohoarc;
b=OTwAYrDHM9rUKJBoDjsaPx8RK8GkZFGpFEPnbn5nAmBVYc+cUkqq2ABIy2T78fgQMSs9QhpgAt0mj6dJaxNudQDY18EyGeIienfWVb1CIAIL0Fqez48zymXcldGSulhhMWzrsJI6Q+ZPKSbJE2a0OVOMouZuaAlYWeGrTCOh4XU=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com;
s=zohoarc;
t=1569337474;
h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results;
bh=sjt3HTbEA8kyWLrcMkiS9mWJLsTXySYqLVvQOKAxo8w=;
b=B8XyuiY/7jD7ASbjRz0L5x7iXn4CAceVE8bWF4J991w8Cfjoa7fIrytr1trrK+7ZiSMFV7lDA4TIt4Rc/ss2FfvlrGnf1pv/LhN83NGTkFItwMRh60u3Sp5hG32kJLvYj+AQkw1hBOW1eWDYgkTYPbvfDBFk7GRmZio/zn3WJ5E=
ARC-Authentication-Results: i=1; mx.zoho.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass header.from= (p=none dis=none)
header.from=
Return-Path:
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by
mx.zohomail.com
with SMTPS id 1569337474683508.20128246169213;
Tue, 24 Sep 2019 08:04:34 -0700 (PDT)
Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com
[10.5.11.15])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by mx1.redhat.com (Postfix) with ESMTPS id F35E030603AD;
Tue, 24 Sep 2019 15:04:32 +0000 (UTC)
Received: from colo-mx.corp.redhat.com
(colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21])
by smtp.corp.redhat.com (Postfix) with ESMTPS id C6DDF61559;
Tue, 24 Sep 2019 15:04:32 +0000 (UTC)
Received: from lists01.pubmisc.prod.ext.phx2.redhat.com
(lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33])
by colo-mx.corp.redhat.com (Postfix) with ESMTP id 9168C4EE6A;
Tue, 24 Sep 2019 15:04:32 +0000 (UTC)
Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com
[10.5.11.14])
by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
id x8OF4CuT027859 for ;
Tue, 24 Sep 2019 11:04:12 -0400
Received: by smtp.corp.redhat.com (Postfix)
id 05A025D9E2; Tue, 24 Sep 2019 15:04:12 +0000 (UTC)
Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com
[10.33.9.32])
by smtp.corp.redhat.com (Postfix) with ESMTP id 6C0105D9D5;
Tue, 24 Sep 2019 15:04:11 +0000 (UTC)
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?=
To: libvir-list@redhat.com
Date: Tue, 24 Sep 2019 15:58:47 +0100
Message-Id: <20190924145903.17123-7-berrange@redhat.com>
In-Reply-To: <20190924145903.17123-1-berrange@redhat.com>
References: <20190924145903.17123-1-berrange@redhat.com>
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14
X-loop: libvir-list@redhat.com
Subject: [libvirt] [PATCH v3 06/22] build-aux: rewrite mock inline checker
in Python
X-BeenThere: libvir-list@redhat.com
X-Mailman-Version: 2.1.12
Precedence: junk
List-Id: Development discussions about the libvirt library & tools
List-Unsubscribe: ,
List-Archive:
List-Post:
List-Help:
List-Subscribe: ,
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Sender: libvir-list-bounces@redhat.com
Errors-To: libvir-list-bounces@redhat.com
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15
X-Greylist: Sender IP whitelisted,
not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.41]);
Tue, 24 Sep 2019 15:04:33 +0000 (UTC)
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the mock-noinline.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Signed-off-by: Daniel P. Berrang=C3=A9
---
Makefile.am | 2 +-
build-aux/mock-noinline.pl | 75 --------------------------------
build-aux/mock-noinline.py | 89 ++++++++++++++++++++++++++++++++++++++
cfg.mk | 4 +-
4 files changed, 92 insertions(+), 78 deletions(-)
delete mode 100644 build-aux/mock-noinline.pl
create mode 100644 build-aux/mock-noinline.py
diff --git a/Makefile.am b/Makefile.am
index 03bf1beb78..db77da890c 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -46,7 +46,7 @@ EXTRA_DIST =3D \
build-aux/gitlog-to-changelog \
build-aux/header-ifdef.pl \
build-aux/minimize-po.py \
- build-aux/mock-noinline.pl \
+ build-aux/mock-noinline.py \
build-aux/prohibit-duplicate-header.py \
build-aux/useless-if-before-free \
build-aux/vc-list-files \
diff --git a/build-aux/mock-noinline.pl b/build-aux/mock-noinline.pl
deleted file mode 100644
index 958e133885..0000000000
--- a/build-aux/mock-noinline.pl
+++ /dev/null
@@ -1,75 +0,0 @@
-#!/usr/bin/env perl
-
-my %noninlined;
-my %mocked;
-
-# Functions in public header don't get the noinline annotation
-# so whitelist them here
-$noninlined{"virEventAddTimeout"} =3D 1;
-# This one confuses the script as its defined in the mock file
-# but is actually just a local helper
-$noninlined{"virMockStatRedirect"} =3D 1;
-
-foreach my $arg (@ARGV) {
- if ($arg =3D~ /\.h$/) {
- #print "Scan header $arg\n";
- &scan_annotations($arg);
- } elsif ($arg =3D~ /mock\.c$/) {
- #print "Scan mock $arg\n";
- &scan_overrides($arg);
- }
-}
-
-my $warned =3D 0;
-foreach my $func (keys %mocked) {
- next if exists $noninlined{$func};
-
- $warned++;
- print STDERR "$func is mocked at $mocked{$func} but missing noinline a=
nnotation\n";
-}
-
-exit $warned ? 1 : 0;
-
-
-sub scan_annotations {
- my $file =3D shift;
-
- open FH, $file or die "cannot read $file: $!";
-
- my $func;
- while () {
- if (/^\s*(\w+)\(/ || /^(?:\w+\*?\s+)+(?:\*\s*)?(\w+)\(/) {
- my $name =3D $1;
- if ($name !~ /ATTRIBUTE/) {
- $func =3D $name;
- }
- } elsif (/^\s*$/) {
- $func =3D undef;
- }
- if (/ATTRIBUTE_NOINLINE/) {
- if (defined $func) {
- $noninlined{$func} =3D 1;
- }
- }
- }
-
- close FH
-}
-
-sub scan_overrides {
- my $file =3D shift;
-
- open FH, $file or die "cannot read $file: $!";
-
- my $func;
- while () {
- if (/^(\w+)\(/ || /^\w+\s*(?:\*\s*)?(\w+)\(/) {
- my $name =3D $1;
- if ($name =3D~ /^vir/) {
- $mocked{$name} =3D "$file:$.";
- }
- }
- }
-
- close FH
-}
diff --git a/build-aux/mock-noinline.py b/build-aux/mock-noinline.py
new file mode 100644
index 0000000000..53aeeb429c
--- /dev/null
+++ b/build-aux/mock-noinline.py
@@ -0,0 +1,89 @@
+#!/usr/bin/env python
+#
+# Copyright (C) 2017-2019 Red Hat, Inc.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library. If not, see
+# .
+
+from __future__ import print_function
+
+import re
+import sys
+
+noninlined =3D {}
+mocked =3D {}
+
+# Functions in public header don't get the noinline annotation
+# so whitelist them here
+noninlined["virEventAddTimeout"] =3D True
+# This one confuses the script as its defined in the mock file
+# but is actually just a local helper
+noninlined["virMockStatRedirect"] =3D True
+
+
+def scan_annotations(filename):
+ funcprog1 =3D re.compile(r'''^\s*(\w+)\(.*''')
+ funcprog2 =3D re.compile(r'''^(?:\w+\*?\s+)+(?:\*\s*)?(\w+)\(.*''')
+ with open(filename, "r") as fh:
+ func =3D None
+ for line in fh:
+ line =3D line.strip()
+ m =3D funcprog1.match(line)
+ if m is None:
+ m =3D funcprog2.match(line)
+ if m is not None:
+ name =3D m.group(1)
+ if name.find("ATTRIBUTE") =3D=3D -1:
+ func =3D name
+ elif line =3D=3D "":
+ func =3D None
+
+ if line.find("ATTRIBUTE_NOINLINE") !=3D -1:
+ if func is not None:
+ noninlined[func] =3D True
+
+
+def scan_overrides(filename):
+ funcprog1 =3D re.compile(r'''^(\w+)\(.*''')
+ funcprog2 =3D re.compile(r'''^\w+\s*(?:\*\s*)?(\w+)\(.*''')
+ with open(filename, "r") as fh:
+ lineno =3D 0
+ for line in fh:
+ lineno =3D lineno + 1
+
+ m =3D funcprog1.match(line)
+ if m is None:
+ m =3D funcprog2.match(line)
+ if m is not None:
+ name =3D m.group(1)
+ if name.startswith("vir"):
+ mocked[name] =3D "%s:%d" % (filename, lineno)
+
+
+for filename in sys.argv[1:]:
+ if filename.endswith(".h"):
+ scan_annotations(filename)
+ elif filename.endswith("mock.c"):
+ scan_overrides(filename)
+
+warned =3D False
+for func in mocked.keys():
+ if func not in noninlined:
+ warned =3D True
+ print("%s is mocked at %s but missing noinline annotation" %
+ (func, mocked[func]), file=3Dsys.stderr)
+
+if warned:
+ sys.exit(1)
+sys.exit(0)
diff --git a/cfg.mk b/cfg.mk
index d52d42c06c..b73d3ae1bf 100644
--- a/cfg.mk
+++ b/cfg.mk
@@ -1161,8 +1161,8 @@ spacing-check:
{ echo '$(ME): incorrect formatting' 1>&2; exit 1; }
=20
mock-noinline:
- $(AM_V_GEN)$(VC_LIST) | $(GREP) '\.[ch]$$' | xargs \
- $(PERL) $(top_srcdir)/build-aux/mock-noinline.pl
+ $(AM_V_GEN)$(VC_LIST) | $(GREP) '\.[ch]$$' | $(RUNUTF8) xargs \
+ $(PYTHON) $(top_srcdir)/build-aux/mock-noinline.py
=20
header-ifdef:
$(AM_V_GEN)$(VC_LIST) | $(GREP) '\.[h]$$' | xargs \
--=20
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
From nobody Thu May 2 16:33:49 2024
Delivered-To: importer@patchew.org
Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28
as permitted sender) client-ip=209.132.183.28;
envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com;
Authentication-Results: mx.zohomail.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass(p=none dis=none) header.from=redhat.com
ARC-Seal: i=1; a=rsa-sha256; t=1569337479; cv=none;
d=zoho.com; s=zohoarc;
b=jw4Zowgrcsk/cMg/Ly7Lk38qSnLqDSS122ZFvOINDF6k1JLJ9dxVE7G8NNwp2zZBS16yLPMX6pXHNnB3Pg9fOwmaz4trqcpcKsp8fmHfRS7wuq0NOZ13DkuacplxdYeXfjMH6PUC8GaeUg+k48Lmar3jVi45dNJvKZypvYf3gVI=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com;
s=zohoarc;
t=1569337479;
h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results;
bh=Kp6nr3hk7o644zoUwj8wsJNn4aJspiXEbtXW2nie7KU=;
b=FPWWFnEE06bIpeiE0Ti4Tku/CIEPyQaxGUZyto/xYfSK3RlnTvZPdX6tHcPZGZWtITCDyhf4TerfZ2/4i5E6MqDym+t0U/Q14VPt5g2S6ewD+4liSCOPBUoqWanra6XKcr7lbQBE5vexyWZ/w1owqs2BhlqtuBOUPZhMIJH+fo4=
ARC-Authentication-Results: i=1; mx.zoho.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass header.from= (p=none dis=none)
header.from=
Return-Path:
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by
mx.zohomail.com
with SMTPS id 1569337479745445.07581765683926;
Tue, 24 Sep 2019 08:04:39 -0700 (PDT)
Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com
[10.5.11.12])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by mx1.redhat.com (Postfix) with ESMTPS id E96063D955;
Tue, 24 Sep 2019 15:04:36 +0000 (UTC)
Received: from colo-mx.corp.redhat.com
(colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21])
by smtp.corp.redhat.com (Postfix) with ESMTPS id C084760C80;
Tue, 24 Sep 2019 15:04:36 +0000 (UTC)
Received: from lists01.pubmisc.prod.ext.phx2.redhat.com
(lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33])
by colo-mx.corp.redhat.com (Postfix) with ESMTP id 7F0CE4EE70;
Tue, 24 Sep 2019 15:04:36 +0000 (UTC)
Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com
[10.5.11.14])
by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
id x8OF4FLF027873 for ;
Tue, 24 Sep 2019 11:04:15 -0400
Received: by smtp.corp.redhat.com (Postfix)
id 14EB35D9E2; Tue, 24 Sep 2019 15:04:15 +0000 (UTC)
Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com
[10.33.9.32])
by smtp.corp.redhat.com (Postfix) with ESMTP id 502EE5D9D5;
Tue, 24 Sep 2019 15:04:12 +0000 (UTC)
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?=
To: libvir-list@redhat.com
Date: Tue, 24 Sep 2019 15:58:48 +0100
Message-Id: <20190924145903.17123-8-berrange@redhat.com>
In-Reply-To: <20190924145903.17123-1-berrange@redhat.com>
References: <20190924145903.17123-1-berrange@redhat.com>
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14
X-loop: libvir-list@redhat.com
Subject: [libvirt] [PATCH v3 07/22] build-aux: rewrite header ifdef checker
in Python
X-BeenThere: libvir-list@redhat.com
X-Mailman-Version: 2.1.12
Precedence: junk
List-Id: Development discussions about the libvirt library & tools
List-Unsubscribe: ,
List-Archive:
List-Post:
List-Help:
List-Subscribe: ,
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Sender: libvir-list-bounces@redhat.com
Errors-To: libvir-list-bounces@redhat.com
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12
X-Greylist: Sender IP whitelisted,
not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.30]);
Tue, 24 Sep 2019 15:04:37 +0000 (UTC)
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the header-ifdef.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Signed-off-by: Daniel P. Berrang=C3=A9
---
Makefile.am | 2 +-
build-aux/header-ifdef.pl | 182 ------------------------------
build-aux/header-ifdef.py | 231 ++++++++++++++++++++++++++++++++++++++
cfg.mk | 4 +-
4 files changed, 234 insertions(+), 185 deletions(-)
delete mode 100644 build-aux/header-ifdef.pl
create mode 100644 build-aux/header-ifdef.py
diff --git a/Makefile.am b/Makefile.am
index db77da890c..4ba729d3f7 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -44,7 +44,7 @@ EXTRA_DIST =3D \
build-aux/augeas-gentest.py \
build-aux/check-spacing.py \
build-aux/gitlog-to-changelog \
- build-aux/header-ifdef.pl \
+ build-aux/header-ifdef.py \
build-aux/minimize-po.py \
build-aux/mock-noinline.py \
build-aux/prohibit-duplicate-header.py \
diff --git a/build-aux/header-ifdef.pl b/build-aux/header-ifdef.pl
deleted file mode 100644
index dba3dbcbdc..0000000000
--- a/build-aux/header-ifdef.pl
+++ /dev/null
@@ -1,182 +0,0 @@
-#!/usr/bin/perl
-#
-# Validate that header files follow a standard layout:
-#
-# /*
-# ...copyright header...
-# */
-#
-# #pragma once
-# ....content....
-#
-#---
-#
-# For any file ending priv.h, before the #pragma once
-# We will have a further section
-#
-# #ifndef SYMBOL_ALLOW
-# # error ....
-# #endif /* SYMBOL_ALLOW */
-#
-#
-#---
-#
-# For public headers (files in include/), use the standard header guard in=
stead of #pragma once:
-# #ifndef SYMBOL
-# # define SYMBOL
-# ....content....
-# #endif /* SYMBOL */
-
-use strict;
-use warnings;
-
-my $STATE_COPYRIGHT_COMMENT =3D 0;
-my $STATE_COPYRIGHT_BLANK =3D 1;
-my $STATE_PRIV_START =3D 2;
-my $STATE_PRIV_ERROR =3D 3;
-my $STATE_PRIV_END =3D 4;
-my $STATE_PRIV_BLANK =3D 5;
-my $STATE_GUARD_START =3D 6;
-my $STATE_GUARD_DEFINE =3D 7;
-my $STATE_GUARD_END =3D 8;
-my $STATE_EOF =3D 9;
-my $STATE_PRAGMA =3D 10;
-
-my $file =3D " ";
-my $ret =3D 0;
-my $ifdef =3D "";
-my $ifdefpriv =3D "";
-my $publicheader =3D 0;
-
-my $state =3D $STATE_EOF;
-my $mistake =3D 0;
-
-sub mistake {
- my $msg =3D shift;
- warn $msg;
- $mistake =3D 1;
- $ret =3D 1;
-}
-
-while (<>) {
- if (not $file eq $ARGV) {
- if ($state =3D=3D $STATE_COPYRIGHT_COMMENT) {
- &mistake("$file: missing copyright comment");
- } elsif ($state =3D=3D $STATE_COPYRIGHT_BLANK) {
- &mistake("$file: missing blank line after copyright header");
- } elsif ($state =3D=3D $STATE_PRIV_START) {
- &mistake("$file: missing '#ifndef $ifdefpriv'");
- } elsif ($state =3D=3D $STATE_PRIV_ERROR) {
- &mistake("$file: missing '# error ...priv allow...'");
- } elsif ($state =3D=3D $STATE_PRIV_END) {
- &mistake("$file: missing '#endif /* $ifdefpriv */'");
- } elsif ($state =3D=3D $STATE_PRIV_BLANK) {
- &mistake("$file: missing blank line after priv header check");
- } elsif ($state =3D=3D $STATE_GUARD_START) {
- if ($publicheader) {
- &mistake("$file: missing '#ifndef $ifdef'");
- } else {
- &mistake("$file: missing '#pragma once' header guard");
- }
- } elsif ($state =3D=3D $STATE_GUARD_DEFINE) {
- &mistake("$file: missing '# define $ifdef'");
- } elsif ($state =3D=3D $STATE_GUARD_END) {
- &mistake("$file: missing '#endif /* $ifdef */'");
- }
-
- $ifdef =3D uc $ARGV;
- $ifdef =3D~ s,.*/,,;
- $ifdef =3D~ s,[^A-Z0-9],_,g;
- $ifdef =3D~ s,__+,_,g;
- unless ($ifdef =3D~ /^LIBVIRT_/ && $ARGV !~ /libvirt_internal.h/) {
- $ifdef =3D "LIBVIRT_" . $ifdef;
- }
- $ifdefpriv =3D $ifdef . "_ALLOW";
-
- $file =3D $ARGV;
- $state =3D $STATE_COPYRIGHT_COMMENT;
- $mistake =3D 0;
- $publicheader =3D ($ARGV =3D~ /include\//);
- }
-
- if ($mistake ||
- $ARGV =3D~ /config-post\.h$/ ||
- $ARGV =3D~ /vbox_(CAPI|XPCOM)/) {
- $state =3D $STATE_EOF;
- next;
- }
-
- if ($state =3D=3D $STATE_COPYRIGHT_COMMENT) {
- if (m,\*/,) {
- $state =3D $STATE_COPYRIGHT_BLANK;
- }
- } elsif ($state =3D=3D $STATE_COPYRIGHT_BLANK) {
- if (! /^$/) {
- &mistake("$file: missing blank line after copyright header");
- }
- if ($ARGV =3D~ /priv\.h$/) {
- $state =3D $STATE_PRIV_START;
- } else {
- $state =3D $STATE_GUARD_START;
- }
- } elsif ($state =3D=3D $STATE_PRIV_START) {
- if (/^$/) {
- &mistake("$file: too many blank lines after copyright header");
- } elsif (/#ifndef $ifdefpriv$/) {
- $state =3D $STATE_PRIV_ERROR;
- } else {
- &mistake("$file: missing '#ifndef $ifdefpriv'");
- }
- } elsif ($state =3D=3D $STATE_PRIV_ERROR) {
- if (/# error ".*"$/) {
- $state =3D $STATE_PRIV_END;
- } else {
- &mistake("$file: missing '# error ...priv allow...'");
- }
- } elsif ($state =3D=3D $STATE_PRIV_END) {
- if (m,#endif /\* $ifdefpriv \*/,) {
- $state =3D $STATE_PRIV_BLANK;
- } else {
- &mistake("$file: missing '#endif /* $ifdefpriv */'");
- }
- } elsif ($state =3D=3D $STATE_PRIV_BLANK) {
- if (! /^$/) {
- &mistake("$file: missing blank line after priv guard");
- }
- $state =3D $STATE_GUARD_START;
- } elsif ($state =3D=3D $STATE_GUARD_START) {
- if (/^$/) {
- &mistake("$file: too many blank lines after copyright header");
- }
- if ($publicheader) {
- if (/#ifndef $ifdef$/) {
- $state =3D $STATE_GUARD_DEFINE;
- } else {
- &mistake("$file: missing '#ifndef $ifdef'");
- }
- } else {
- if (/#pragma once/) {
- $state =3D $STATE_PRAGMA;
- } else {
- &mistake("$file: missing '#pragma once' header guard");
- }
- }
- } elsif ($state =3D=3D $STATE_GUARD_DEFINE) {
- if (/# define $ifdef$/) {
- $state =3D $STATE_GUARD_END;
- } else {
- &mistake("$file: missing '# define $ifdef'");
- }
- } elsif ($state =3D=3D $STATE_GUARD_END) {
- if (m,#endif /\* $ifdef \*/$,) {
- $state =3D $STATE_EOF;
- }
- } elsif ($state =3D=3D $STATE_PRAGMA) {
- next;
- } elsif ($state =3D=3D $STATE_EOF) {
- die "$file: unexpected content after '#endif /* $ifdef */'";
- } else {
- die "$file: unexpected state $state";
- }
-}
-exit $ret;
diff --git a/build-aux/header-ifdef.py b/build-aux/header-ifdef.py
new file mode 100644
index 0000000000..a192e01bc4
--- /dev/null
+++ b/build-aux/header-ifdef.py
@@ -0,0 +1,231 @@
+#!/usr/bin/env python
+#
+# Copyright (C) 2018-2019 Red Hat, Inc.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library. If not, see
+# .
+#
+# Validate that header files follow a standard layout:
+#
+# /*
+# ...copyright header...
+# */
+#
+# #pragma once
+# ....content....
+#
+# ---
+#
+# For any file ending priv.h, before the #pragma once
+# We will have a further section
+#
+# #ifndef SYMBOL_ALLOW
+# # error ....
+# #endif /* SYMBOL_ALLOW */
+#
+#
+# ---
+#
+# For public headers (files in include/), use the standard
+# header guard instead of #pragma once:
+# #ifndef SYMBOL
+# # define SYMBOL
+# ....content....
+# #endif /* SYMBOL */
+
+from __future__ import print_function
+
+import os.path
+import re
+import sys
+
+STATE_COPYRIGHT_COMMENT =3D 0
+STATE_COPYRIGHT_BLANK =3D 1
+STATE_PRIV_START =3D 2
+STATE_PRIV_ERROR =3D 3
+STATE_PRIV_END =3D 4
+STATE_PRIV_BLANK =3D 5
+STATE_GUARD_START =3D 6
+STATE_GUARD_DEFINE =3D 7
+STATE_GUARD_END =3D 8
+STATE_EOF =3D 9
+STATE_PRAGMA =3D 10
+
+
+def check_header(filename):
+ ifdef =3D ""
+ ifdefpriv =3D ""
+
+ state =3D STATE_EOF
+
+ ifdef =3D os.path.basename(filename).upper()
+ ifdef =3D re.sub(r"""[^A-Z0-9]""", "_", ifdef)
+ ifdef =3D re.sub(r"""__+""", "_", ifdef)
+
+ if (not ifdef.startswith("LIBVIRT_") or
+ filename.find("libvirt_internal.h") !=3D -1):
+ ifdef =3D "LIBVIRT_" + ifdef
+
+ ifdefpriv =3D ifdef + "_ALLOW"
+
+ state =3D STATE_COPYRIGHT_COMMENT
+ publicheader =3D False
+ if filename.find("include/") !=3D -1:
+ publicheader =3D True
+
+ with open(filename, "r") as fh:
+ for line in fh:
+ line =3D line.rstrip("\n")
+ if state =3D=3D STATE_COPYRIGHT_COMMENT:
+ if line.find("*/") !=3D -1:
+ state =3D STATE_COPYRIGHT_BLANK
+ elif state =3D=3D STATE_COPYRIGHT_BLANK:
+ if line !=3D "":
+ print("%s: missing blank line after copyright header" %
+ filename, file=3Dsys.stderr)
+ return True
+
+ if filename.endswith("priv.h"):
+ state =3D STATE_PRIV_START
+ else:
+ state =3D STATE_GUARD_START
+ elif state =3D=3D STATE_PRIV_START:
+ if line =3D=3D "":
+ print("%s: too many blank lines after copyright header=
" %
+ filename, file=3Dsys.stderr)
+ return True
+ elif re.match(r"""#ifndef %s$""" % ifdefpriv, line):
+ state =3D STATE_PRIV_ERROR
+ else:
+ print("%s: missing '#ifndef %s'" % (filename, ifdefpri=
v),
+ file=3Dsys.stderr)
+ return True
+ elif state =3D=3D STATE_PRIV_ERROR:
+ if re.match(r"""# error ".*"$""", line):
+ state =3D STATE_PRIV_END
+ else:
+ print("%s: missing '# error ...priv allow...'" %
+ filename, file=3Dsys.stderr)
+ return True
+ elif state =3D=3D STATE_PRIV_END:
+ if re.match(r"""#endif /\* %s \*/""" % ifdefpriv, line):
+ state =3D STATE_PRIV_BLANK
+ else:
+ print("%s: missing '#endif /* %s */'" %
+ (filename, ifdefpriv), file=3Dsys.stderr)
+ return True
+ elif state =3D=3D STATE_PRIV_BLANK:
+ if line !=3D "":
+ print("%s: missing blank line after priv guard" %
+ filename, file=3Dsys.stderr)
+ return True
+ state =3D STATE_GUARD_START
+ elif state =3D=3D STATE_GUARD_START:
+ if line =3D=3D "":
+ print("%s: too many blank lines after copyright header=
" %
+ filename, file=3Dsys.stderr)
+ return True
+ if publicheader:
+ if re.match(r"""#ifndef %s$""" % ifdef, line):
+ state =3D STATE_GUARD_DEFINE
+ else:
+ print("%s: missing '#ifndef %s'" %
+ (filename, ifdef), file=3Dsys.stderr)
+ return True
+ else:
+ if re.match(r"""#pragma once""", line):
+ state =3D STATE_PRAGMA
+ else:
+ print("%s: missing '#pragma once' header guard" %
+ filename, file=3Dsys.stderr)
+ return True
+ elif state =3D=3D STATE_GUARD_DEFINE:
+ if re.match(r"""# define %s$""" % ifdef, line):
+ state =3D STATE_GUARD_END
+ else:
+ print("%s: missing '# define %s'" %
+ (filename, ifdef), file=3Dsys.stderr)
+ return True
+ elif state =3D=3D STATE_GUARD_END:
+ if re.match(r"""#endif /\* %s \*/$""" % ifdef, line):
+ state =3D STATE_EOF
+ elif state =3D=3D STATE_PRAGMA:
+ next
+ elif state =3D=3D STATE_EOF:
+ print("%s: unexpected content after '#endif /* %s */'" %
+ (filename, ifdef), file=3Dsys.stderr)
+ return True
+ else:
+ print("%s: unexpected state $state" %
+ filename, file=3Dsys.stderr)
+ return True
+
+ if state =3D=3D STATE_COPYRIGHT_COMMENT:
+ print("%s: missing copyright comment" %
+ filename, file=3Dsys.stderr)
+ return True
+ elif state =3D=3D STATE_COPYRIGHT_BLANK:
+ print("%s: missing blank line after copyright header" %
+ filename, file=3Dsys.stderr)
+ return True
+ elif state =3D=3D STATE_PRIV_START:
+ print("%s: missing '#ifndef %s'" %
+ (filename, ifdefpriv), file=3Dsys.stderr)
+ return True
+ elif state =3D=3D STATE_PRIV_ERROR:
+ print("%s: missing '# error ...priv allow...'" %
+ filename, file=3Dsys.stderr)
+ return True
+ elif state =3D=3D STATE_PRIV_END:
+ print("%s: missing '#endif /* %s */'" %
+ (filename, ifdefpriv), file=3Dsys.stderr)
+ return True
+ elif state =3D=3D STATE_PRIV_BLANK:
+ print("%s: missing blank line after priv header check" %
+ filename, file=3Dsys.stderr)
+ return True
+ elif state =3D=3D STATE_GUARD_START:
+ if publicheader:
+ print("%s: missing '#ifndef %s'" %
+ (filename, ifdef), file=3Dsys.stderr)
+ return True
+ else:
+ print("%s: missing '#pragma once' header guard" %
+ filename, file=3Dsys.stderr)
+ return True
+ elif state =3D=3D STATE_GUARD_DEFINE:
+ print("%s: missing '# define %s'" %
+ (filename, ifdef), file=3Dsys.stderr)
+ return True
+ elif state =3D=3D STATE_GUARD_END:
+ print("%s: missing '#endif /* %s */'" %
+ (filename, ifdef), file=3Dsys.stderr)
+ return True
+
+ return False
+
+
+ret =3D 0
+
+for filename in sys.argv[1:]:
+ if filename.find("config-post.h") !=3D -1:
+ continue
+ if filename.find("vbox_CAPI") !=3D -1:
+ continue
+ if filename.find("vbox_XPCOM") !=3D -1:
+ continue
+ if check_header(filename):
+ ret =3D 1
+
+sys.exit(ret)
diff --git a/cfg.mk b/cfg.mk
index b73d3ae1bf..f0aed0365a 100644
--- a/cfg.mk
+++ b/cfg.mk
@@ -1165,8 +1165,8 @@ mock-noinline:
$(PYTHON) $(top_srcdir)/build-aux/mock-noinline.py
=20
header-ifdef:
- $(AM_V_GEN)$(VC_LIST) | $(GREP) '\.[h]$$' | xargs \
- $(PERL) $(top_srcdir)/build-aux/header-ifdef.pl
+ $(AM_V_GEN)$(VC_LIST) | $(GREP) '\.[h]$$' | $(RUNUTF8) xargs \
+ $(PYTHON) $(top_srcdir)/build-aux/header-ifdef.py
=20
test-wrap-argv:
$(AM_V_GEN)$(VC_LIST) | $(GREP) -E '\.(ldargs|args)' | xargs \
--=20
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
From nobody Thu May 2 16:33:49 2024
Delivered-To: importer@patchew.org
Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28
as permitted sender) client-ip=209.132.183.28;
envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com;
Authentication-Results: mx.zohomail.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass(p=none dis=none) header.from=redhat.com
ARC-Seal: i=1; a=rsa-sha256; t=1569337480; cv=none;
d=zoho.com; s=zohoarc;
b=G3s3KRgdFcNlFG0U3t6lpS53PRChJGkukMLeLtkC8S+gdlI+s9BTz+G3Mkdwlt/Xc8g7Zq884p7FeulOCpLPFiAzq+0klB0pQm3eXRP8czOD/EoqC74w8T2e26bs9n7Xx8vOxsZz609v/+6nTg3aGiyqiaeCYTTQtQtHN0YL5lw=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com;
s=zohoarc;
t=1569337480;
h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results;
bh=/yaYMTuaJeBm44cRm1vJL+a/kYU/SRPXokKwqM/10RE=;
b=bayQDXGVZng0DNpLIHCvRNKKVJn66TXD9jeJyMt0EyLwZ+b1jrKASrl8AL99JkUXF2uqdjBOgR805sf/bvepkD76hB7x6cvNngnFL65QV/ZcBZlgWATpowMvw+AAESkOYDL+XjhnOZBfBfx+xnLBM/PrjbWk9oygVv4g8giltLY=
ARC-Authentication-Results: i=1; mx.zoho.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass header.from= (p=none dis=none)
header.from=
Return-Path:
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by
mx.zohomail.com
with SMTPS id 1569337480771400.0317366752755;
Tue, 24 Sep 2019 08:04:40 -0700 (PDT)
Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com
[10.5.11.11])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by mx1.redhat.com (Postfix) with ESMTPS id 7F97110C094A;
Tue, 24 Sep 2019 15:04:38 +0000 (UTC)
Received: from colo-mx.corp.redhat.com
(colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20])
by smtp.corp.redhat.com (Postfix) with ESMTPS id 4EC2A6012D;
Tue, 24 Sep 2019 15:04:38 +0000 (UTC)
Received: from lists01.pubmisc.prod.ext.phx2.redhat.com
(lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33])
by colo-mx.corp.redhat.com (Postfix) with ESMTP id 0CCF9180BA9A;
Tue, 24 Sep 2019 15:04:38 +0000 (UTC)
Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com
[10.5.11.14])
by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
id x8OF4M33027898 for ;
Tue, 24 Sep 2019 11:04:22 -0400
Received: by smtp.corp.redhat.com (Postfix)
id 5794E5D9DC; Tue, 24 Sep 2019 15:04:22 +0000 (UTC)
Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com
[10.33.9.32])
by smtp.corp.redhat.com (Postfix) with ESMTP id 75B9C5D9D5;
Tue, 24 Sep 2019 15:04:15 +0000 (UTC)
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?=
To: libvir-list@redhat.com
Date: Tue, 24 Sep 2019 15:58:49 +0100
Message-Id: <20190924145903.17123-9-berrange@redhat.com>
In-Reply-To: <20190924145903.17123-1-berrange@redhat.com>
References: <20190924145903.17123-1-berrange@redhat.com>
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14
X-loop: libvir-list@redhat.com
Subject: [libvirt] [PATCH v3 08/22] src: rewrite ACL permissions checker in
Python
X-BeenThere: libvir-list@redhat.com
X-Mailman-Version: 2.1.12
Precedence: junk
List-Id: Development discussions about the libvirt library & tools
List-Unsubscribe: ,
List-Archive:
List-Post:
List-Help:
List-Subscribe: ,
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Sender: libvir-list-bounces@redhat.com
Errors-To: libvir-list-bounces@redhat.com
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11
X-Greylist: Sender IP whitelisted,
not delayed by milter-greylist-4.6.2 (mx1.redhat.com [10.5.110.66]);
Tue, 24 Sep 2019 15:04:39 +0000 (UTC)
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the check-aclperms.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Signed-off-by: Daniel P. Berrang=C3=A9
---
src/Makefile.am | 4 +--
src/check-aclperms.pl | 73 ----------------------------------------
src/check-aclperms.py | 78 +++++++++++++++++++++++++++++++++++++++++++
3 files changed, 80 insertions(+), 75 deletions(-)
delete mode 100755 src/check-aclperms.pl
create mode 100755 src/check-aclperms.py
diff --git a/src/Makefile.am b/src/Makefile.am
index 2956e4bf35..71858ea07f 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -358,11 +358,11 @@ check-aclrules:
$(addprefix $(srcdir)/,$(filter-out /%,$(STATEFUL_DRIVER_SOURCE_FILES)))
=20
check-aclperms:
- $(AM_V_GEN)$(PERL) $(srcdir)/check-aclperms.pl \
+ $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(srcdir)/check-aclperms.py \
$(srcdir)/access/viraccessperm.h \
$(srcdir)/access/viraccessperm.c
=20
-EXTRA_DIST +=3D check-driverimpls.pl check-aclrules.pl check-aclperms.pl
+EXTRA_DIST +=3D check-driverimpls.pl check-aclrules.pl check-aclperms.py
=20
check-local: check-protocol check-symfile check-symsorting \
check-drivername check-driverimpls check-aclrules \
diff --git a/src/check-aclperms.pl b/src/check-aclperms.pl
deleted file mode 100755
index 55b6598313..0000000000
--- a/src/check-aclperms.pl
+++ /dev/null
@@ -1,73 +0,0 @@
-#!/usr/bin/env perl
-#
-# Copyright (C) 2013 Red Hat, Inc.
-#
-# This library is free software; you can redistribute it and/or
-# modify it under the terms of the GNU Lesser General Public
-# License as published by the Free Software Foundation; either
-# version 2.1 of the License, or (at your option) any later version.
-#
-# This library is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-# Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public
-# License along with this library. If not, see
-# .
-#
-# This script just validates that the stringified version of
-# a virAccessPerm enum matches the enum constant name. We do
-# a lot of auto-generation of code, so when these don't match
-# problems occur, preventing auth from succeeding at all.
-
-my $hdr =3D shift;
-my $impl =3D shift;
-
-my %perms;
-
-my @perms;
-
-open HDR, $hdr or die "cannot read $hdr: $!";
-
-while () {
- if (/^\s+VIR_ACCESS_PERM_([_A-Z]+)(,?|\s|$)/) {
- my $perm =3D $1;
-
- $perms{$perm} =3D 1 unless ($perm =3D~ /_LAST$/);
- }
-}
-
-close HDR;
-
-
-open IMPL, $impl or die "cannot read $impl: $!";
-
-my $group;
-my $warned =3D 0;
-
-while (defined (my $line =3D )) {
- if ($line =3D~ /VIR_ACCESS_PERM_([_A-Z]+)_LAST/) {
- $group =3D $1;
- } elsif ($line =3D~ /"[_a-z]+"/) {
- my @bits =3D split /,/, $line;
- foreach my $bit (@bits) {
- if ($bit =3D~ /"([_a-z]+)"/) {
- my $perm =3D uc($group . "_" . $1);
- if (!exists $perms{$perm}) {
- print STDERR "Unknown perm string $1 for group $group\=
n";
- $warned =3D 1;
- }
- delete $perms{$perm};
- }
- }
- }
-}
-close IMPL;
-
-foreach my $perm (keys %perms) {
- print STDERR "Perm $perm had not string form\n";
- $warned =3D 1;
-}
-
-exit $warned;
diff --git a/src/check-aclperms.py b/src/check-aclperms.py
new file mode 100755
index 0000000000..035c0f6999
--- /dev/null
+++ b/src/check-aclperms.py
@@ -0,0 +1,78 @@
+#!/usr/bin/env python
+#
+# Copyright (C) 2013-2019 Red Hat, Inc.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library. If not, see
+# .
+#
+# This script just validates that the stringified version of
+# a virAccessPerm enum matches the enum constant name. We do
+# a lot of auto-generation of code, so when these don't match
+# problems occur, preventing auth from succeeding at all.
+
+from __future__ import print_function
+
+import re
+import sys
+
+if len(sys.argv) !=3D 3:
+ print("syntax: %s HEADER IMPL" % (sys.argv[0]), file=3Dsys.stderr)
+ sys.exit(1)
+
+hdr =3D sys.argv[1]
+impl =3D sys.argv[2]
+
+perms =3D {}
+
+with open(hdr) as fh:
+ symprog =3D re.compile(r"^\s+VIR_ACCESS_PERM_([_A-Z]+)(,?|\s|$).*")
+ for line in fh:
+ symmatch =3D symprog.match(line)
+ if symmatch is not None:
+ perm =3D symmatch.group(1)
+
+ if not perm.endswith("_LAST"):
+ perms[perm] =3D 1
+
+warned =3D False
+
+with open(impl) as fh:
+ group =3D None
+ symlastprog =3D re.compile(r".*VIR_ACCESS_PERM_([_A-Z]+)_LAST.*")
+ alnumprog =3D re.compile(r'''.*"([_a-z]+)".*''')
+
+ for line in fh:
+ symlastmatch =3D symlastprog.match(line)
+ if symlastmatch is not None:
+ group =3D symlastmatch.group(1)
+ elif alnumprog.match(line) is not None:
+ bits =3D line.split(",")
+ for bit in bits:
+ m =3D alnumprog.match(bit)
+ if m is not None:
+ perm =3D (group + "_" + m.group(1)).upper()
+ if perm not in perms:
+ print("Unknown perm string %s for group %s" %
+ (m.group(1), group), file=3Dsys.stderr)
+ warned =3D True
+
+ del perms[perm]
+
+for perm in perms.keys():
+ print("Perm %s had not string form" % perm, file=3Dsys.stderr)
+ warned =3D True
+
+if warned:
+ sys.exit(1)
+sys.exit(0)
--=20
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
From nobody Thu May 2 16:33:49 2024
Delivered-To: importer@patchew.org
Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28
as permitted sender) client-ip=209.132.183.28;
envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com;
Authentication-Results: mx.zohomail.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass(p=none dis=none) header.from=redhat.com
ARC-Seal: i=1; a=rsa-sha256; t=1569337484; cv=none;
d=zoho.com; s=zohoarc;
b=VaCv8zS0smr5R272Vt8a2WVGZCXGecAQe+0CvffJwz1gVgqCJNGyiFTwU+/uM/0rQNlrlwOpZ3mFxEeUwKMIfjG7MEa2QkXx5P83GeS5Hhk1odwtumO4Urm518LNhJqW9sqoo5ttCEHVXrU7oyUpeMmjsrTjQyL4xkIZxFs6ATg=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com;
s=zohoarc;
t=1569337484;
h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results;
bh=LopgD0BCmKhlH34czt2sfpDO42TVCbPa/4HO1hZYYNc=;
b=oOnjzKIq7f9lUXyLb4n+fLczGnoQbMIL4uo2MUcHCNfeGNt7V/SmmfBFO/VXML++LxH+t1SJbqYukWfT3gsdLm4qcNqRK7PpkweTD+VvvSc5kqii4K7K+Y79hARA8Dnl5PTQW6x2dfqAFsXYz7nTCm7csHG3gVxqK//Rx5wAvU4=
ARC-Authentication-Results: i=1; mx.zoho.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass header.from= (p=none dis=none)
header.from=
Return-Path:
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by
mx.zohomail.com
with SMTPS id 1569337484668486.35182365457547;
Tue, 24 Sep 2019 08:04:44 -0700 (PDT)
Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com
[10.5.11.13])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by mx1.redhat.com (Postfix) with ESMTPS id D5C4D30860DF;
Tue, 24 Sep 2019 15:04:42 +0000 (UTC)
Received: from colo-mx.corp.redhat.com
(colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20])
by smtp.corp.redhat.com (Postfix) with ESMTPS id B0A9860603;
Tue, 24 Sep 2019 15:04:42 +0000 (UTC)
Received: from lists01.pubmisc.prod.ext.phx2.redhat.com
(lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33])
by colo-mx.corp.redhat.com (Postfix) with ESMTP id 71BAF180BA9F;
Tue, 24 Sep 2019 15:04:42 +0000 (UTC)
Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com
[10.5.11.14])
by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
id x8OF4NXH027906 for ;
Tue, 24 Sep 2019 11:04:23 -0400
Received: by smtp.corp.redhat.com (Postfix)
id 37C145D9DC; Tue, 24 Sep 2019 15:04:23 +0000 (UTC)
Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com
[10.33.9.32])
by smtp.corp.redhat.com (Postfix) with ESMTP id 9D30F5D9D5;
Tue, 24 Sep 2019 15:04:22 +0000 (UTC)
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?=
To: libvir-list@redhat.com
Date: Tue, 24 Sep 2019 15:58:50 +0100
Message-Id: <20190924145903.17123-10-berrange@redhat.com>
In-Reply-To: <20190924145903.17123-1-berrange@redhat.com>
References: <20190924145903.17123-1-berrange@redhat.com>
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14
X-loop: libvir-list@redhat.com
Subject: [libvirt] [PATCH v3 09/22] src: rewrite symfile sorting checker in
Python
X-BeenThere: libvir-list@redhat.com
X-Mailman-Version: 2.1.12
Precedence: junk
List-Id: Development discussions about the libvirt library & tools
List-Unsubscribe: ,
List-Archive:
List-Post:
List-Help:
List-Subscribe: ,
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Sender: libvir-list-bounces@redhat.com
Errors-To: libvir-list-bounces@redhat.com
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13
X-Greylist: Sender IP whitelisted,
not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.44]);
Tue, 24 Sep 2019 15:04:43 +0000 (UTC)
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the check-symsorting.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Signed-off-by: Daniel P. Berrang=C3=A9
---
src/Makefile.am | 6 +-
src/check-symsorting.pl | 106 -----------------------------------
src/check-symsorting.py | 119 ++++++++++++++++++++++++++++++++++++++++
3 files changed, 122 insertions(+), 109 deletions(-)
delete mode 100755 src/check-symsorting.pl
create mode 100755 src/check-symsorting.py
diff --git a/src/Makefile.am b/src/Makefile.am
index 71858ea07f..7c3e9ab981 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -284,12 +284,12 @@ check-symfile:
check-admin-symfile:
endif ! WITH_LINUX
check-symsorting:
- $(AM_V_GEN)$(PERL) $(srcdir)/check-symsorting.pl \
+ $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(srcdir)/check-symsorting.py \
$(srcdir) $(SYM_FILES)
check-admin-symsorting:
- $(AM_V_GEN)$(PERL) $(srcdir)/check-symsorting.pl \
+ $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(srcdir)/check-symsorting.py \
$(srcdir) $(ADMIN_SYM_FILES)
-EXTRA_DIST +=3D check-symfile.pl check-symsorting.pl
+EXTRA_DIST +=3D check-symfile.pl check-symsorting.py
=20
# Keep this list synced with RPC_PROBE_FILES
PROTOCOL_STRUCTS =3D \
diff --git a/src/check-symsorting.pl b/src/check-symsorting.pl
deleted file mode 100755
index 51e38bdedb..0000000000
--- a/src/check-symsorting.pl
+++ /dev/null
@@ -1,106 +0,0 @@
-#!/usr/bin/env perl
-
-# Copyright (C) 2012-2013 Red Hat, Inc.
-#
-# This library is free software; you can redistribute it and/or
-# modify it under the terms of the GNU Lesser General Public
-# License as published by the Free Software Foundation; either
-# version 2.1 of the License, or (at your option) any later version.
-#
-# This library is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-# Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public
-# License along with this library. If not, see
-# .
-
-use strict;
-use warnings;
-
-die "syntax: $0 SRCDIR SYMFILE..." unless int(@ARGV) >=3D 2;
-
-my $ret =3D 0;
-my $srcdir =3D shift;
-my $lastgroup =3D undef;
-foreach my $symfile (@ARGV) {
- open SYMFILE, $symfile or die "cannot read $symfile: $!";
-
- my $line =3D 0;
- my $groupfile =3D "";
- my @group;
-
- while () {
- chomp;
-
- if (/^#\s*((\w+\/)*(\w+\.h))\s*$/) {
- $groupfile =3D $1;
- } elsif (/^#/) {
- # Ignore comments
- } elsif (/^\s*$/) {
- if (@group) {
- &check_sorting(\@group, $symfile, $line, $groupfile);
- }
- @group =3D ();
- $line =3D $.;
- } else {
- $_ =3D~ s/;//;
- push @group, $_;
- }
- }
-
- close SYMFILE;
- if (@group) {
- &check_sorting(\@group, $symfile, $line, $groupfile);
- }
- $lastgroup =3D undef;
-}
-
-sub check_sorting {
- my $group =3D shift;
- my $symfile =3D shift;
- my $line =3D shift;
- my $groupfile =3D shift;
-
- my @group =3D @{$group};
- my @sorted =3D sort { lc $a cmp lc $b } @group;
- my $sorted =3D 1;
- my $first;
- my $last;
-
- # Check that groups are in order and groupfile exists
- if (defined $lastgroup && lc $lastgroup ge lc $groupfile) {
- print "Symbol block at $symfile:$line: block not sorted\n";
- print "Move $groupfile block before $lastgroup block\n";
- print "\n";
- $ret =3D 1;
- }
- if (! -e "$srcdir/$groupfile") {
- print "Symbol block at $symfile:$line: $groupfile not found\n";
- print "\n";
- $ret =3D 1;
- }
- $lastgroup =3D $groupfile;
-
- # Check that symbols within a group are in order
- for (my $i =3D 0 ; $i <=3D $#sorted ; $i++) {
- if ($sorted[$i] ne $group[$i]) {
- $first =3D $i unless defined $first;
- $last =3D $i;
- $sorted =3D 0;
- }
- }
- if (!$sorted) {
- @group =3D splice @group, $first, ($last-$first+1);
- @sorted =3D splice @sorted, $first, ($last-$first+1);
- print "Symbol block at $symfile:$line: symbols not sorted\n";
- print map { " " . $_ . "\n" } @group;
- print "Correct ordering\n";
- print map { " " . $_ . "\n" } @sorted;
- print "\n";
- $ret =3D 1;
- }
-}
-
-exit $ret;
diff --git a/src/check-symsorting.py b/src/check-symsorting.py
new file mode 100755
index 0000000000..a25175cac8
--- /dev/null
+++ b/src/check-symsorting.py
@@ -0,0 +1,119 @@
+#!/usr/bin/env python
+
+# Copyright (C) 2012-2019 Red Hat, Inc.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library. If not, see
+# .
+
+from __future__ import print_function
+
+import os.path
+import re
+import sys
+
+if len(sys.argv) < 3:
+ print("syntax: %s SRCDIR SYMFILE..." % sys.argv[0], file=3Dsys.stderr)
+ sys.exit(1)
+
+
+def check_sorting(group, symfile, line, groupfile, lastgroup):
+ sortedgroup =3D sorted(group, key=3Dstr.lower)
+ issorted =3D True
+ first =3D None
+ last =3D None
+
+ err =3D False
+ # Check that groups are in order and groupfile exists
+ if lastgroup is not None and lastgroup.lower() > groupfile.lower():
+ print("Symbol block at %s:%s: block not sorted" %
+ (symfile, line), file=3Dsys.stderr)
+ print("Move %s block before %s block" %
+ (groupfile, lastgroup), file=3Dsys.stderr)
+ print("", file=3Dsys.stderr)
+ err =3D True
+
+ if not os.path.exists(os.path.join(srcdir, groupfile)):
+ print("Symbol block at %s:%s: %s not found" %
+ (symfile, line, groupfile), file=3Dsys.stderr)
+ print("", file=3Dsys.stderr)
+ err =3D True
+
+ # Check that symbols within a group are in order
+ for i in range(len(group)):
+ if sortedgroup[i] !=3D group[i]:
+ if first is None:
+ first =3D i
+
+ last =3D i
+ issorted =3D False
+
+ if not issorted:
+ actual =3D group[first:(last-first+1)]
+ expect =3D sortedgroup[first:(last-first+1)]
+ print("Symbol block at %s:%s: symbols not sorted" %
+ (symfile, line), file=3Dsys.stderr)
+ for g in actual:
+ print(" %s" % g, file=3Dsys.stderr)
+ print("Correct ordering", file=3Dsys.stderr)
+ for g in expect:
+ print(" %s" % g, file=3Dsys.stderr)
+ print("", file=3Dsys.stderr)
+ err =3D True
+
+ return err
+
+
+ret =3D 0
+srcdir =3D sys.argv[1]
+lastgroup =3D None
+for symfile in sys.argv[2:]:
+ with open(symfile, "r") as fh:
+ lineno =3D 0
+ groupfile =3D ""
+ group =3D []
+ thisline =3D 0
+
+ filenameprog =3D re.compile(r'''^#\s*((\w+\/)*(\w+\.h))\s*$''')
+
+ for line in fh:
+ thisline =3D thisline + 1
+ line =3D line.strip()
+
+ filenamematch =3D filenameprog.match(line)
+ if filenamematch is not None:
+ groupfile =3D filenamematch.group(1)
+ elif line =3D=3D "":
+ if len(group) > 0:
+ if check_sorting(group, symfile, lineno,
+ groupfile, lastgroup):
+ ret =3D 1
+
+ group =3D []
+ lineno =3D thisline
+ lastgroup =3D groupfile
+ elif line[0] =3D=3D '#':
+ # Ignore comments
+ pass
+ else:
+ line =3D line.strip(";")
+ group.append(line)
+
+ if len(group) > 0:
+ if check_sorting(group, symfile, lineno,
+ groupfile, lastgroup):
+ ret =3D 1
+
+ lastgroup =3D None
+
+sys.exit(ret)
--=20
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
From nobody Thu May 2 16:33:49 2024
Delivered-To: importer@patchew.org
Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28
as permitted sender) client-ip=209.132.183.28;
envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com;
Authentication-Results: mx.zohomail.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass(p=none dis=none) header.from=redhat.com
ARC-Seal: i=1; a=rsa-sha256; t=1569337483; cv=none;
d=zoho.com; s=zohoarc;
b=VgtFijcdEoM5QgRyo5BPmsnUjRQ6nUYPK+/m0KZ4n45Ym+8KcrF2fcYsGRRX4Nr+5ovXTYyhPRYOG0gqEyATrw2kIE5bpUGXDB5fy0bAT2gogRof3S0NAux5M7octzDpFqVtjhWKygtwFOXvT+ckma7jVirGovJPtWoU803xTO4=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com;
s=zohoarc;
t=1569337483;
h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results;
bh=7cy+V2M1nIkGZcFXRph1ZMu0gwTsfDM5iSLW9QjVam0=;
b=h1oy88yHpv8+uEA/M8XyB9lfMOdaVbo/sD/COM4Zh0kbS9XJsfu//141LQnO4zprcXfzgKc8YcVtPMs6EgMsm63JF9flQaHFWEIgLa7/U693Z7dVPXxBhNsQA77apAKSf66qqJN7K4suMdF8R3muHyEvqdGquujzEbmmSv77U5k=
ARC-Authentication-Results: i=1; mx.zoho.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass header.from= (p=none dis=none)
header.from=
Return-Path:
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by
mx.zohomail.com
with SMTPS id 1569337483412922.0321401656854;
Tue, 24 Sep 2019 08:04:43 -0700 (PDT)
Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com
[10.5.11.13])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by mx1.redhat.com (Postfix) with ESMTPS id 0FA25A26675;
Tue, 24 Sep 2019 15:04:41 +0000 (UTC)
Received: from colo-mx.corp.redhat.com
(colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21])
by smtp.corp.redhat.com (Postfix) with ESMTPS id DF08360624;
Tue, 24 Sep 2019 15:04:40 +0000 (UTC)
Received: from lists01.pubmisc.prod.ext.phx2.redhat.com
(lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33])
by colo-mx.corp.redhat.com (Postfix) with ESMTP id A2B954EE72;
Tue, 24 Sep 2019 15:04:40 +0000 (UTC)
Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com
[10.5.11.14])
by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
id x8OF4SGK027927 for ;
Tue, 24 Sep 2019 11:04:28 -0400
Received: by smtp.corp.redhat.com (Postfix)
id 729D25D9E1; Tue, 24 Sep 2019 15:04:28 +0000 (UTC)
Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com
[10.33.9.32])
by smtp.corp.redhat.com (Postfix) with ESMTP id 786035D9D5;
Tue, 24 Sep 2019 15:04:23 +0000 (UTC)
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?=
To: libvir-list@redhat.com
Date: Tue, 24 Sep 2019 15:58:51 +0100
Message-Id: <20190924145903.17123-11-berrange@redhat.com>
In-Reply-To: <20190924145903.17123-1-berrange@redhat.com>
References: <20190924145903.17123-1-berrange@redhat.com>
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14
X-loop: libvir-list@redhat.com
Subject: [libvirt] [PATCH v3 10/22] src: rewrite symfile library checker in
Python
X-BeenThere: libvir-list@redhat.com
X-Mailman-Version: 2.1.12
Precedence: junk
List-Id: Development discussions about the libvirt library & tools
List-Unsubscribe: ,
List-Archive:
List-Post:
List-Help:
List-Subscribe: ,
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Sender: libvir-list-bounces@redhat.com
Errors-To: libvir-list-bounces@redhat.com
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13
X-Greylist: Sender IP whitelisted,
not delayed by milter-greylist-4.6.2 (mx1.redhat.com [10.5.110.68]);
Tue, 24 Sep 2019 15:04:42 +0000 (UTC)
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the check-symfile.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Signed-off-by: Daniel P. Berrang=C3=A9
---
src/Makefile.am | 10 +++---
src/check-symfile.pl | 70 --------------------------------------
src/check-symfile.py | 81 ++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 86 insertions(+), 75 deletions(-)
delete mode 100755 src/check-symfile.pl
create mode 100755 src/check-symfile.py
diff --git a/src/Makefile.am b/src/Makefile.am
index 7c3e9ab981..d10bfdf448 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -274,11 +274,11 @@ PDWTAGS =3D \
# rule for libvirt.la. However, checking symbols relies on Linux ELF layo=
ut
if WITH_LINUX
check-symfile: libvirt.syms libvirt.la
- $(AM_V_GEN)$(PERL) $(srcdir)/check-symfile.pl libvirt.syms \
- .libs/libvirt.so
+ $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(srcdir)/check-symfile.py \
+ libvirt.syms .libs/libvirt.so
check-admin-symfile: libvirt_admin.syms libvirt-admin.la
- $(AM_V_GEN)$(PERL) $(srcdir)/check-symfile.pl libvirt_admin.syms \
- .libs/libvirt-admin.so
+ $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(srcdir)/check-symfile.py \
+ libvirt_admin.syms .libs/libvirt-admin.so
else ! WITH_LINUX
check-symfile:
check-admin-symfile:
@@ -289,7 +289,7 @@ check-symsorting:
check-admin-symsorting:
$(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(srcdir)/check-symsorting.py \
$(srcdir) $(ADMIN_SYM_FILES)
-EXTRA_DIST +=3D check-symfile.pl check-symsorting.py
+EXTRA_DIST +=3D check-symfile.py check-symsorting.py
=20
# Keep this list synced with RPC_PROBE_FILES
PROTOCOL_STRUCTS =3D \
diff --git a/src/check-symfile.pl b/src/check-symfile.pl
deleted file mode 100755
index 4f88300864..0000000000
--- a/src/check-symfile.pl
+++ /dev/null
@@ -1,70 +0,0 @@
-#!/usr/bin/env perl
-
-# Copyright (C) 2012-2013 Red Hat, Inc.
-#
-# This library is free software; you can redistribute it and/or
-# modify it under the terms of the GNU Lesser General Public
-# License as published by the Free Software Foundation; either
-# version 2.1 of the License, or (at your option) any later version.
-#
-# This library is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-# Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public
-# License along with this library. If not, see
-# .
-
-die "syntax: $0 SYMFILE ELFLIB(S)" unless int(@ARGV) >=3D 2;
-
-my $symfile =3D shift @ARGV;
-my @elflibs =3D @ARGV;
-
-my %wantsyms;
-my %gotsyms;
-
-my $ret =3D 0;
-
-open SYMFILE, $symfile or die "cannot read $symfile: $!";
-
-while () {
- next if /{/;
- next if /}/;
- next if /global:/;
- next if /local:/;
- next if /^\s*$/;
- next if /^\s*#/;
- next if /\*/;
-
- die "malformed line $_" unless /^\s*(\S+);$/;
-
- if (exists $wantsyms{$1}) {
- print STDERR "Symbol $1 is listed twice\n";
- $ret =3D 1;
- } else {
- $wantsyms{$1} =3D 1;
- }
-}
-close SYMFILE;
-
-foreach my $elflib (@elflibs) {
- open NM, "-|", "nm", $elflib or die "cannot run 'nm $elflib': $!";
-
- while () {
- next unless /^\S+\s(?:[TBD])\s(\S+)\s*$/;
-
- $gotsyms{$1} =3D 1;
- }
-
- close NM;
-}
-
-foreach my $sym (keys(%wantsyms)) {
- next if exists $gotsyms{$sym};
-
- print STDERR "Expected symbol $sym is not in ELF library\n";
- $ret =3D 1;
-}
-
-exit($ret);
diff --git a/src/check-symfile.py b/src/check-symfile.py
new file mode 100755
index 0000000000..0bb1c1e00d
--- /dev/null
+++ b/src/check-symfile.py
@@ -0,0 +1,81 @@
+#!/usr/bin/env python
+#
+# Copyright (C) 2012-2019 Red Hat, Inc.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library. If not, see
+# .
+
+from __future__ import print_function
+
+import re
+import subprocess
+import sys
+
+if len(sys.argv) < 3:
+ print("syntax: %s SYMFILE ELFLIB(S)" % sys.argv[0], file=3Dsys.stderr)
+
+symfile =3D sys.argv[1]
+elflibs =3D sys.argv[2:]
+
+wantsyms =3D {}
+gotsyms =3D {}
+
+ret =3D 0
+
+with open(symfile, "r") as fh:
+ for line in fh:
+ line =3D line.strip()
+ if line.find("{") !=3D -1:
+ continue
+ if line.find("}") !=3D -1:
+ continue
+ if line in ["global:", "local:"]:
+ continue
+ if line =3D=3D "":
+ continue
+ if line[0] =3D=3D '#':
+ continue
+ if line.find("*") !=3D -1:
+ continue
+
+ line =3D line.strip(";")
+
+ if line in wantsyms:
+ print("Symbol $1 is listed twice", file=3Dsys.stderr)
+ ret =3D 1
+ else:
+ wantsyms[line] =3D True
+
+for elflib in elflibs:
+ nm =3D subprocess.Popen(["nm", elflib], shell=3DFalse,
+ stdout=3Dsubprocess.PIPE).stdout
+
+ symprog =3D re.compile(r'''^\S+\s(?:[TBD])\s(\S+)\s*$''')
+ for line in nm:
+ line =3D line.decode("utf-8")
+ symmatch =3D symprog.match(line)
+ if symmatch is None:
+ continue
+
+ gotsyms[symmatch.group(1)] =3D True
+
+
+for sym in wantsyms.keys():
+ if sym in gotsyms:
+ continue
+
+ print("Expected symbol '%s' is not in ELF library" % sym, file=3Dsys.s=
tderr)
+ ret =3D 1
+
+sys.exit(ret)
--=20
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
From nobody Thu May 2 16:33:49 2024
Delivered-To: importer@patchew.org
Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28
as permitted sender) client-ip=209.132.183.28;
envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com;
Authentication-Results: mx.zohomail.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass(p=none dis=none) header.from=redhat.com
ARC-Seal: i=1; a=rsa-sha256; t=1569337473; cv=none;
d=zoho.com; s=zohoarc;
b=HqwY//5R50EegPfufljKhySorFvqoWESuT3ZNxJdfrIT8Q/axEN1ApxVuZ+9sk8gwQPhFMr+QYfbhQGhyiAe1mmEAe3UEW1MXuIlC7BcRjNN1vUO5+yEt54iARtxkfkVw0h5JhkEe9iMSMnVuhaQS/FvLqtB+cJUImrcNVMoKTo=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com;
s=zohoarc;
t=1569337473;
h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results;
bh=UAzLd1D00HSNQT8FxmD0GtjyyjVxJjgLaHU8aDQYsjE=;
b=Q+G7NzC97wBzuMca0TEdhX0hR8K4NW/kXoDdQs1AK4Uo4nZguJHLpY5lQHQhPOqHBDa+Ha/Xuk0RWfXtU71wvUBgOjlpGbctpgy7DBtlaksADirtSUUNhluGXZDa+mva8GmeBwNoTp4ehR86REN8vY3rrDBBSqkD5s9o3CJEygs=
ARC-Authentication-Results: i=1; mx.zoho.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass header.from= (p=none dis=none)
header.from=
Return-Path:
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by
mx.zohomail.com
with SMTPS id 1569337473179163.86738819376603;
Tue, 24 Sep 2019 08:04:33 -0700 (PDT)
Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com
[10.5.11.16])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by mx1.redhat.com (Postfix) with ESMTPS id A77A3300CB2B;
Tue, 24 Sep 2019 15:04:31 +0000 (UTC)
Received: from colo-mx.corp.redhat.com
(colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21])
by smtp.corp.redhat.com (Postfix) with ESMTPS id 83B465C21F;
Tue, 24 Sep 2019 15:04:31 +0000 (UTC)
Received: from lists01.pubmisc.prod.ext.phx2.redhat.com
(lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33])
by colo-mx.corp.redhat.com (Postfix) with ESMTP id 484684EE50;
Tue, 24 Sep 2019 15:04:31 +0000 (UTC)
Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com
[10.5.11.14])
by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
id x8OF4Tts027932 for ;
Tue, 24 Sep 2019 11:04:29 -0400
Received: by smtp.corp.redhat.com (Postfix)
id 51DB55DA21; Tue, 24 Sep 2019 15:04:29 +0000 (UTC)
Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com
[10.33.9.32])
by smtp.corp.redhat.com (Postfix) with ESMTP id B7A895D9D5;
Tue, 24 Sep 2019 15:04:28 +0000 (UTC)
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?=
To: libvir-list@redhat.com
Date: Tue, 24 Sep 2019 15:58:52 +0100
Message-Id: <20190924145903.17123-12-berrange@redhat.com>
In-Reply-To: <20190924145903.17123-1-berrange@redhat.com>
References: <20190924145903.17123-1-berrange@redhat.com>
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14
X-loop: libvir-list@redhat.com
Subject: [libvirt] [PATCH v3 11/22] src: rewrite systemtap probe generator
in Python
X-BeenThere: libvir-list@redhat.com
X-Mailman-Version: 2.1.12
Precedence: junk
List-Id: Development discussions about the libvirt library & tools
List-Unsubscribe: ,
List-Archive:
List-Post:
List-Help:
List-Subscribe: ,
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Sender: libvir-list-bounces@redhat.com
Errors-To: libvir-list-bounces@redhat.com
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16
X-Greylist: Sender IP whitelisted,
not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.46]);
Tue, 24 Sep 2019 15:04:32 +0000 (UTC)
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the dtrace2systemtap.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Signed-off-by: Daniel P. Berrang=C3=A9
---
src/Makefile.am | 9 ++-
src/dtrace2systemtap.pl | 130 ------------------------------------
src/dtrace2systemtap.py | 143 ++++++++++++++++++++++++++++++++++++++++
3 files changed, 147 insertions(+), 135 deletions(-)
delete mode 100755 src/dtrace2systemtap.pl
create mode 100755 src/dtrace2systemtap.py
diff --git a/src/Makefile.am b/src/Makefile.am
index d10bfdf448..5a1df7a93c 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -604,7 +604,6 @@ nodist_libvirt_la_SOURCES =3D libvirt_probes.h
if WITH_REMOTE
nodist_libvirt_driver_remote_la_SOURCES =3D libvirt_probes.h
endif WITH_REMOTE
-DTRACE2SYSTEMTAP_FLAGS =3D --with-modules
=20
BUILT_SOURCES +=3D libvirt_probes.h libvirt_probes.stp libvirt_functions.s=
tp
=20
@@ -639,10 +638,10 @@ RPC_PROBE_FILES +=3D $(srcdir)/rpc/virnetprotocol.x \
libvirt_functions.stp: $(RPC_PROBE_FILES) $(srcdir)/rpc/gensystemtap.pl
$(AM_V_GEN)$(PERL) -w $(srcdir)/rpc/gensystemtap.pl $(RPC_PROBE_FILES) > =
$@
=20
-%_probes.stp: %_probes.d $(srcdir)/dtrace2systemtap.pl \
+%_probes.stp: %_probes.d $(srcdir)/dtrace2systemtap.py \
$(top_builddir)/config.status
- $(AM_V_GEN)$(PERL) -w $(srcdir)/dtrace2systemtap.pl \
- $(DTRACE2SYSTEMTAP_FLAGS) $(bindir) $(sbindir) $(libdir) $< > $@
+ $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(srcdir)/dtrace2systemtap.py \
+ $(bindir) $(sbindir) $(libdir) $< > $@
=20
CLEANFILES +=3D libvirt_probes.h libvirt_probes.o libvirt_probes.lo \
libvirt_functions.stp libvirt_probes.stp
@@ -763,7 +762,7 @@ endif LIBVIRT_INIT_SCRIPT_SYSTEMD
endif WITH_LIBVIRTD
=20
=20
-EXTRA_DIST +=3D dtrace2systemtap.pl
+EXTRA_DIST +=3D dtrace2systemtap.py
=20
=20
if WITH_LIBVIRTD
diff --git a/src/dtrace2systemtap.pl b/src/dtrace2systemtap.pl
deleted file mode 100755
index c5fce248b4..0000000000
--- a/src/dtrace2systemtap.pl
+++ /dev/null
@@ -1,130 +0,0 @@
-#!/usr/bin/env perl
-#
-# Copyright (C) 2011-2012 Red Hat, Inc.
-#
-# This library is free software; you can redistribute it and/or
-# modify it under the terms of the GNU Lesser General Public
-# License as published by the Free Software Foundation; either
-# version 2.1 of the License, or (at your option) any later version.
-#
-# This library is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-# Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public
-# License along with this library. If not, see
-# .
-#
-#
-# Generate a set of systemtap probe definitions corresponding to
-# DTrace probe markers in libvirt.so
-#
-# perl dtrace2systemtap.pl probes.d > libvirt_probes.stp
-#
-
-use strict;
-use warnings;
-
-my $file;
-my @files;
-my %files;
-
-my $with_modules =3D 0;
-if ($ARGV[0] eq "--with-modules") {
- # set if we want to honor the "module" setting in the .d file
- $with_modules =3D 1;
- shift @ARGV;
-}
-
-my $bindir =3D shift @ARGV;
-my $sbindir =3D shift @ARGV;
-my $libdir =3D shift @ARGV;
-
-my $probe;
-my $args;
-
-# Read the DTraceprobes definition
-while (<>) {
- next if m,^\s*$,;
-
- next if /^\s*provider\s+\w+\s*{\s*$/;
- next if /^\s*};\s*$/;
-
- if (m,^\s*\#,) {
- if (m,^\s*\#\s*file:\s*(\S+)\s*$,) {
- $file =3D $1;
- push @files, $file;
- $files{$file} =3D { prefix =3D> undef, probes =3D> [] };
- } elsif (m,^\s*\#\s*prefix:\s*(\S+)\s*$,) {
- $files{$file}->{prefix} =3D $1;
- } elsif (m,^\s*\#\s*binary:\s*(\S+)\s*$,) {
- $files{$file}->{binary} =3D $1;
- } elsif (m,^\s*\#\s*module:\s*(\S+)\s*$,) {
- $files{$file}->{module} =3D $1;
- } else {
- # ignore unknown comments
- }
- } else {
- if (m,\s*probe\s+([a-zA-Z0-9_]+)\((.*?)(\);)?$,) {
- $probe =3D $1;
- $args =3D $2;
- if ($3) {
- push @{$files{$file}->{probes}}, [$probe, $args];
- $probe =3D $args =3D undef;
- }
- } elsif ($probe) {
- if (m,^(.*?)(\);)?$,) {
- $args .=3D $1;
- if ($2) {
- push @{$files{$file}->{probes}}, [$probe, $args];
- $probe =3D $args =3D undef;
- }
- } else {
- die "unexpected data $_ on line $.";
- }
- } else {
- die "unexpected data $_ on line $.";
- }
- }
-}
-
-# Write out the SystemTap probes
-foreach my $file (@files) {
- my $prefix =3D $files{$file}->{prefix};
- my @probes =3D @{$files{$file}->{probes}};
-
- print "# $file\n\n";
- foreach my $probe (@probes) {
- my $name =3D $probe->[0];
- my $args =3D $probe->[1];
-
- my $pname =3D $name;
- $pname =3D~ s/${prefix}_/libvirt.$prefix./;
-
- my $binary =3D "$libdir/libvirt.so";
- if (exists $files{$file}->{binary}) {
- $binary =3D $sbindir . "/" . $files{$file}->{binary};
- }
- if ($with_modules && exists $files{$file}->{module}) {
- $binary =3D $libdir . "/" . $files{$file}->{module};
- }
-
- print "probe $pname =3D process(\"$binary\").mark(\"$name\") {\n";
-
- my @args =3D split /,/, $args;
- for (my $i =3D 0 ; $i <=3D $#args ; $i++) {
- my $arg =3D $args[$i];
- my $isstr =3D $arg =3D~ /char\s+\*/;
- $arg =3D~ s/^.*\s\*?(\S+)$/$1/;
-
- if ($isstr) {
- print " $arg =3D user_string(\$arg", $i + 1, ");\n";
- } else {
- print " $arg =3D \$arg", $i + 1, ";\n";
- }
- }
- print "}\n\n";
- }
- print "\n";
-}
diff --git a/src/dtrace2systemtap.py b/src/dtrace2systemtap.py
new file mode 100755
index 0000000000..8ee5632d6b
--- /dev/null
+++ b/src/dtrace2systemtap.py
@@ -0,0 +1,143 @@
+#!/usr/bin/env python
+#
+# Copyright (C) 2011-2019 Red Hat, Inc.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library. If not, see
+# .
+#
+#
+# Generate a set of systemtap probe definitions corresponding to
+# DTrace probe markers in libvirt.so
+#
+# python dtrace2systemtap.py probes.d > libvirt_probes.stp
+#
+
+from __future__ import print_function
+
+import re
+import sys
+
+file =3D None
+filelist =3D []
+files =3D {}
+
+bindir =3D sys.argv[1]
+sbindir =3D sys.argv[2]
+libdir =3D sys.argv[3]
+dtrace =3D sys.argv[4]
+
+probe =3D None
+args =3D None
+
+# Read the DTraceprobes definition
+
+with open(dtrace, "r") as fh:
+ lineno =3D 0
+ for line in fh:
+ lineno =3D lineno + 1
+ line =3D line.strip()
+ if line =3D=3D "":
+ continue
+ if line.find("provider ") !=3D -1 and line.find("{") !=3D -1:
+ continue
+ if line.find("};") !=3D -1:
+ continue
+
+ if line.startswith("#"):
+ m =3D re.match(r'''^\#\s*file:\s*(\S+)$''', line)
+ if m is not None:
+ file =3D m.group(1)
+ filelist.append(file)
+ files[file] =3D {"prefix": None, "probes": []}
+ continue
+
+ m =3D re.match(r'''^\#\s*prefix:\s*(\S+)$''', line)
+ if m is not None:
+ files[file]["prefix"] =3D m.group(1)
+ continue
+
+ m =3D re.match(r'''^\#\s*binary:\s*(\S+)$''', line)
+ if m is not None:
+ files[file]["binary"] =3D m.group(1)
+ continue
+
+ m =3D re.match(r'''^\#\s*module:\s*(\S+)$''', line)
+ if m is not None:
+ files[file]["module"] =3D m.group(1)
+
+ # ignore unknown comments
+ else:
+ m =3D re.match(r'''probe\s+([a-zA-Z0-9_]+)\((.*?)(\);)?$''', l=
ine)
+ if m is not None:
+ probe =3D m.group(1)
+ args =3D m.group(2)
+ if m.group(3) is not None:
+ files[file]["probes"].append([probe, args])
+ probe =3D None
+ args =3D None
+ elif probe is not None:
+ m =3D re.match(r'''^(.*?)(\);)?$''', line)
+ if m is not None:
+ args =3D args + m.group(1)
+ if m.group(2) is not None:
+ files[file]["probes"].append([probe, args])
+ probe =3D None
+ args =3D None
+ else:
+ raise Exception("unexpected data %s on line %d" %
+ (line, lineno))
+ else:
+ raise Exception("unexpected data %s on line %d" %
+ (line, lineno))
+
+# Write out the SystemTap probes
+for file in filelist:
+ prefix =3D files[file]["prefix"]
+ probes =3D files[file]["probes"]
+
+ print("# %s\n" % file)
+ for probe in probes:
+ name =3D probe[0]
+ args =3D probe[1]
+
+ pname =3D name.replace(prefix + "_", "libvirt." + prefix + ".")
+
+ binary =3D libdir + "/libvirt.so"
+ if "binary" in files[file]:
+ binary =3D sbindir + "/" + files[file]["binary"]
+ if "module" in files[file]:
+ binary =3D libdir + "/" + files[file]["module"]
+
+ print("probe %s =3D process(\"%s\").mark(\"%s\") {" %
+ (pname, binary, name))
+
+ argbits =3D args.split(",")
+ for idx in range(len(argbits)):
+ arg =3D argbits[idx]
+ isstr =3D False
+ if arg.find("char *") !=3D -1:
+ isstr =3D True
+
+ m =3D re.match(r'''^.*\s\*?(\S+)$''', arg)
+ if m is not None:
+ arg =3D m.group(1)
+ else:
+ raise Exception("Malformed arg %s" % arg)
+
+ if isstr:
+ print(" %s =3D user_string($arg%d);" % (arg, idx + 1))
+ else:
+ print(" %s =3D $arg%d;" % (arg, idx + 1))
+ print("}\n")
+ print("")
--=20
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
From nobody Thu May 2 16:33:49 2024
Delivered-To: importer@patchew.org
Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28
as permitted sender) client-ip=209.132.183.28;
envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com;
Authentication-Results: mx.zohomail.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass(p=none dis=none) header.from=redhat.com
ARC-Seal: i=1; a=rsa-sha256; t=1569337486; cv=none;
d=zoho.com; s=zohoarc;
b=E29bsniXDPfrc3Hgvok1KKH6wiSOTdm6CWsvg7YovNR8QjG4P6CiQzp0+OQQ6kSKELhdKBzgoFA5Jt9jqdQ1BhcD2ZbG5fWrNEn12qTQdnlR64krST9PCRXT2rySU1v7a0xId64vCS/xSVEo2JsaJPLxoQOi9ALOk7Y+CdJ6qLY=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com;
s=zohoarc;
t=1569337486;
h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results;
bh=E/uOIIsiywEdLhmLKgTSuvTQuxTk+M1/k88w5vsinnY=;
b=DaqvNQoJo9K07akvzOHEpryxXV5+HT9dBKk3fskX4tWKQalbI+QOKNpFdrIEc88Rzpqcqw53M7NTQjSyrQbfQmKGpgnndaespNv1WArk5YVFS/IYMJk09amd699ydUc3shom5+IkbtGrY0ehBSJHaYnWFd4e18pIN/whUOGfjq8=
ARC-Authentication-Results: i=1; mx.zoho.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass header.from= (p=none dis=none)
header.from=
Return-Path:
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by
mx.zohomail.com
with SMTPS id 1569337486315975.7514913101105;
Tue, 24 Sep 2019 08:04:46 -0700 (PDT)
Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com
[10.5.11.13])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by mx1.redhat.com (Postfix) with ESMTPS id BF18320EE;
Tue, 24 Sep 2019 15:04:44 +0000 (UTC)
Received: from colo-mx.corp.redhat.com
(colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20])
by smtp.corp.redhat.com (Postfix) with ESMTPS id 9AF53608C2;
Tue, 24 Sep 2019 15:04:44 +0000 (UTC)
Received: from lists01.pubmisc.prod.ext.phx2.redhat.com
(lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33])
by colo-mx.corp.redhat.com (Postfix) with ESMTP id 5CF69180B761;
Tue, 24 Sep 2019 15:04:44 +0000 (UTC)
Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com
[10.5.11.14])
by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
id x8OF4WOh027948 for ;
Tue, 24 Sep 2019 11:04:32 -0400
Received: by smtp.corp.redhat.com (Postfix)
id 383435DAAC; Tue, 24 Sep 2019 15:04:32 +0000 (UTC)
Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com
[10.33.9.32])
by smtp.corp.redhat.com (Postfix) with ESMTP id 9D54E5D9D5;
Tue, 24 Sep 2019 15:04:29 +0000 (UTC)
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?=
To: libvir-list@redhat.com
Date: Tue, 24 Sep 2019 15:58:53 +0100
Message-Id: <20190924145903.17123-13-berrange@redhat.com>
In-Reply-To: <20190924145903.17123-1-berrange@redhat.com>
References: <20190924145903.17123-1-berrange@redhat.com>
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14
X-loop: libvir-list@redhat.com
Subject: [libvirt] [PATCH v3 12/22] src: rewrite systemtap function
generator in Python
X-BeenThere: libvir-list@redhat.com
X-Mailman-Version: 2.1.12
Precedence: junk
List-Id: Development discussions about the libvirt library & tools
List-Unsubscribe: ,
List-Archive:
List-Post:
List-Help:
List-Subscribe: ,
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Sender: libvir-list-bounces@redhat.com
Errors-To: libvir-list-bounces@redhat.com
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13
X-Greylist: Sender IP whitelisted,
not delayed by milter-greylist-4.6.2 (mx1.redhat.com [10.5.110.71]);
Tue, 24 Sep 2019 15:04:45 +0000 (UTC)
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the gensystemtap.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Signed-off-by: Daniel P. Berrang=C3=A9
---
cfg.mk | 4 +-
src/Makefile.am | 5 +-
src/rpc/Makefile.inc.am | 2 +-
src/rpc/gensystemtap.pl | 193 ----------------------------------------
src/rpc/gensystemtap.py | 184 ++++++++++++++++++++++++++++++++++++++
5 files changed, 189 insertions(+), 199 deletions(-)
delete mode 100755 src/rpc/gensystemtap.pl
create mode 100755 src/rpc/gensystemtap.py
diff --git a/cfg.mk b/cfg.mk
index f0aed0365a..18bab6e9a4 100644
--- a/cfg.mk
+++ b/cfg.mk
@@ -407,6 +407,7 @@ sc_prohibit_risky_id_promotion:
# since gnulib has more guarantees for snprintf portability
sc_prohibit_sprintf:
@prohibit=3D'\<[s]printf\>' \
+ in_vc_files=3D'\.[ch]$$' \
halt=3D'use snprintf, not sprintf' \
$(_sc_search_regexp)
=20
@@ -1272,9 +1273,6 @@ exclude_file_name_regexp--sc_prohibit_readlink =3D \
=20
exclude_file_name_regexp--sc_prohibit_setuid =3D ^src/util/virutil\.c|tool=
s/virt-login-shell\.c$$
=20
-exclude_file_name_regexp--sc_prohibit_sprintf =3D \
- ^(cfg\.mk|docs/hacking\.html\.in|.*\.stp|.*\.pl)$$
-
exclude_file_name_regexp--sc_prohibit_strncpy =3D ^src/util/virstring\.c$$
=20
exclude_file_name_regexp--sc_prohibit_strtol =3D ^examples/.*$$
diff --git a/src/Makefile.am b/src/Makefile.am
index 5a1df7a93c..5226871a73 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -635,8 +635,9 @@ RPC_PROBE_FILES +=3D $(srcdir)/rpc/virnetprotocol.x \
$(srcdir)/remote/qemu_protocol.x \
$(srcdir)/admin/admin_protocol.x
=20
-libvirt_functions.stp: $(RPC_PROBE_FILES) $(srcdir)/rpc/gensystemtap.pl
- $(AM_V_GEN)$(PERL) -w $(srcdir)/rpc/gensystemtap.pl $(RPC_PROBE_FILES) > =
$@
+libvirt_functions.stp: $(RPC_PROBE_FILES) $(srcdir)/rpc/gensystemtap.py
+ $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(srcdir)/rpc/gensystemtap.py \
+ $(RPC_PROBE_FILES) > $@
=20
%_probes.stp: %_probes.d $(srcdir)/dtrace2systemtap.py \
$(top_builddir)/config.status
diff --git a/src/rpc/Makefile.inc.am b/src/rpc/Makefile.inc.am
index b8ca53c69a..b43ee4ab7a 100644
--- a/src/rpc/Makefile.inc.am
+++ b/src/rpc/Makefile.inc.am
@@ -3,7 +3,7 @@
EXTRA_DIST +=3D \
rpc/gendispatch.pl \
rpc/genprotocol.pl \
- rpc/gensystemtap.pl \
+ rpc/gensystemtap.py \
rpc/virnetprotocol.x \
rpc/virkeepaliveprotocol.x \
$(NULL)
diff --git a/src/rpc/gensystemtap.pl b/src/rpc/gensystemtap.pl
deleted file mode 100755
index 6693d4d6f5..0000000000
--- a/src/rpc/gensystemtap.pl
+++ /dev/null
@@ -1,193 +0,0 @@
-#!/usr/bin/env perl
-#
-# Copyright (C) 2011-2012 Red Hat, Inc.
-#
-# This library is free software; you can redistribute it and/or
-# modify it under the terms of the GNU Lesser General Public
-# License as published by the Free Software Foundation; either
-# version 2.1 of the License, or (at your option) any later version.
-#
-# This library is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-# Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public
-# License along with this library. If not, see
-# .
-#
-# Generate a set of systemtap functions for translating various
-# RPC enum values into strings
-#
-# perl gensystemtap.pl */*.x > libvirt_functions.stp
-#
-
-use strict;
-
-my %funcs;
-
-my %type;
-my %status;
-my %auth;
-
-my $instatus =3D 0;
-my $intype =3D 0;
-my $inauth =3D 0;
-while (<>) {
- if (/enum\s+virNetMessageType/) {
- $intype =3D 1;
- } elsif (/enum\s+virNetMessageStatus/) {
- $instatus =3D 1;
- } elsif (/enum remote_auth_type/) {
- $inauth =3D 1;
- } elsif (/}/) {
- $instatus =3D $intype =3D $inauth =3D 0;
- } elsif ($instatus) {
- if (/^\s+VIR_NET_(\w+)\s*=3D\s*(\d+),?$/) {
- $status{$2} =3D lc $1;
- }
- } elsif ($intype) {
- if (/^\s+VIR_NET_(\w+)\s*=3D\s*(\d+),?$/) {
- $type{$2} =3D lc $1;
- }
- } elsif ($inauth) {
- if (/^\s+REMOTE_AUTH_(\w+)\s*=3D\s*(\d+),?$/) {
- $auth{$2} =3D lc $1;
- }
- } else {
- if (/(?:VIR_)?(\w+?)(?:_PROTOCOL)?_PROGRAM\s*=3D\s*0x([a-fA-F0-9]+=
)\s*;/) {
- $funcs{lc $1} =3D { id =3D> hex($2), version =3D> undef, progs=
=3D> [] };
- } elsif (/(?:VIR_)?(\w+?)(?:_PROTOCOL)?_(?:PROGRAM|PROTOCOL)_VERSI=
ON\s*=3D\s*(\d+)\s*;/) {
- $funcs{lc $1}->{version} =3D $2;
- } elsif (/(?:VIR_)?(\w+?)(?:_PROTOCOL)?_PROC_(.*?)\s+=3D\s+(\d+)/)=
{
- $funcs{lc $1}->{progs}->[$3] =3D lc $2;
- }
- }
-}
-
-print <{id}, ") {\n";
- print " programstr =3D \"", $prog, "\"\n";
-}
-print <{id}, " && version =
=3D=3D ", $funcs{$prog}->{version}, ") {\n";
-
- my $pfirst =3D 1;
- for (my $id =3D 1 ; $id <=3D $#{$funcs{$prog}->{progs}} ; $id++) {
- my $cond =3D $pfirst ? "if" : "} else if";
- $pfirst =3D 0;
- print " $cond (proc =3D=3D $id) {\n";
- print " procstr =3D \"", $funcs{$prog}->{progs}->[$id], "=
\";\n";
- }
- print " } else {\n";
- print " procstr =3D \"unknown\";\n";
- print " verbose =3D 1;\n";
- print " }\n";
-}
-print <.
+#
+# Generate a set of systemtap functions for translating various
+# RPC enum values into strings
+#
+# python gensystemtap.py */*.x > libvirt_functions.stp
+#
+
+from __future__ import print_function
+
+import re
+import sys
+
+funcs =3D {}
+
+types =3D {}
+status =3D {}
+auth =3D {}
+
+
+def load_file(fh):
+ instatus =3D False
+ intype =3D False
+ inauth =3D False
+
+ for line in fh:
+ if re.match(r'''\s*enum\s+virNetMessageType\s*''', line):
+ intype =3D True
+ elif re.match(r'''\s*enum\s+virNetMessageStatus\s*''', line):
+ instatus =3D True
+ elif re.match(r'''\s*enum remote_auth_type.*''', line):
+ inauth =3D True
+ elif line.find("}") !=3D -1:
+ intype =3D False
+ instatus =3D False
+ inauth =3D False
+ elif instatus:
+ m =3D re.match(r'''^\s+VIR_NET_(\w+)\s*=3D\s*(\d+),?$''', line)
+ if m is not None:
+ status[m.group(2)] =3D m.group(1).lower()
+ elif intype:
+ m =3D re.match(r'''^\s+VIR_NET_(\w+)\s*=3D\s*(\d+),?$''', line)
+ if m is not None:
+ types[m.group(2)] =3D m.group(1).lower()
+ elif inauth:
+ m =3D re.match(r'''^\s+REMOTE_AUTH_(\w+)\s*=3D\s*(\d+),?$''', =
line)
+ if m is not None:
+ auth[m.group(2)] =3D m.group(1).lower()
+ else:
+ m =3D re.match(r'''^.*?(?:VIR_)?(\w+?)(?:_PROTOCOL)?''' +
+ r'''_PROGRAM\s*=3D\s*0x([a-fA-F0-9]+)\s*;''', lin=
e)
+ if m is not None:
+ funcs[m.group(1).lower()] =3D {
+ "id": int(m.group(2), 16),
+ "version": None,
+ "progs": []
+ }
+ continue
+
+ m =3D re.match(r'''^.*?(?:VIR_)?(\w+?)(?:_PROTOCOL)?_''' +
+ r'''(?:PROGRAM|PROTOCOL)_VERSION\s*=3D\s*(\d+)\s*=
;''',
+ line)
+ if m is not None:
+ funcs[m.group(1).lower()]["version"] =3D m.group(2)
+ continue
+
+ m =3D re.match(r'''^.*?(?:VIR_)?(\w+?)(?:_PROTOCOL)?''' +
+ r'''_PROC_(.*?)\s+=3D\s+(\d+)''', line)
+ if m is not None:
+ funcs[m.group(1).lower()]["progs"].insert(
+ int(m.group(3)), m.group(2).lower())
+
+
+for file in sys.argv[1:]:
+ with open(file, "r") as fh:
+ load_file(fh)
+
+
+def genfunc(name, varname, types):
+ print("function %s(%s, verbose)" % (name, varname))
+ print("{")
+
+ first =3D True
+ for typename in sorted(types.keys()):
+ cond =3D "} else if"
+ if first:
+ cond =3D "if"
+ first =3D False
+
+ print(" %s (%s =3D=3D %s) {" % (cond, varname, typename))
+ print(" %sstr =3D \"%s\"" % (varname, types[typename]))
+
+ print(" } else {")
+ print(" %sstr =3D \"unknown\";" % varname)
+ print(" verbose =3D 1;")
+ print(" }")
+ print(" if (verbose) {")
+ print(" %sstr =3D %sstr . sprintf(\":%%d\", %s)" %
+ (varname, varname, varname))
+ print(" }")
+ print(" return %sstr;" % varname)
+ print("}")
+
+
+genfunc("libvirt_rpc_auth_name", "type", auth)
+genfunc("libvirt_rpc_type_name", "type", types)
+genfunc("libvirt_rpc_status_name", "status", status)
+
+print("function libvirt_rpc_program_name(program, verbose)")
+print("{")
+
+first =3D True
+for funcname in sorted(funcs.keys()):
+ cond =3D "} else if"
+ if first:
+ cond =3D "if"
+ first =3D False
+
+ print(" %s (program =3D=3D %s) {" % (cond, funcs[funcname]["id"]))
+ print(" programstr =3D \"%s\"" % funcname)
+
+print(" } else {")
+print(" programstr =3D \"unknown\";")
+print(" verbose =3D 1;")
+print(" }")
+print(" if (verbose) {")
+print(" programstr =3D programstr . sprintf(\":%d\", program)")
+print(" }")
+print(" return programstr;")
+print("}")
+
+print("function libvirt_rpc_procedure_name(program, version, proc, verbose=
)")
+print("{")
+
+first =3D True
+for prog in sorted(funcs.keys()):
+ cond =3D "} else if"
+ if first:
+ cond =3D "if"
+ first =3D False
+
+ print(" %s (program =3D=3D %s && version =3D=3D %s) {" %
+ (cond, funcs[prog]["id"], funcs[prog]["version"]))
+
+ pfirst =3D True
+ for id in range(len(funcs[prog]["progs"])):
+ pcond =3D "} else if"
+ if pfirst:
+ pcond =3D "if"
+ pfirst =3D False
+
+ print(" %s (proc =3D=3D %s) {" % (pcond, id + 1))
+ print(" procstr =3D \"%s\";" % funcs[prog]["progs"][id])
+
+ print(" } else {")
+ print(" procstr =3D \"unknown\";")
+ print(" verbose =3D 1;")
+ print(" }")
+
+print(" } else {")
+print(" procstr =3D \"unknown\";")
+print(" verbose =3D 1;")
+print(" }")
+print(" if (verbose) {")
+print(" procstr =3D procstr . sprintf(\":%d\", proc)")
+print(" }")
+print(" return procstr;")
+print("}")
--=20
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
From nobody Thu May 2 16:33:49 2024
Delivered-To: importer@patchew.org
Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28
as permitted sender) client-ip=209.132.183.28;
envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com;
Authentication-Results: mx.zohomail.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass(p=none dis=none) header.from=redhat.com
ARC-Seal: i=1; a=rsa-sha256; t=1569337489; cv=none;
d=zoho.com; s=zohoarc;
b=AhegSVIecLwU6BHg9hffyOuAEgz+PNfw593WvrmrS4xwwVRT4zl4yUbmxYkdXoAxV6KnMtqKwcppomhLvijKX/vnLmeMXVkR6Lg5e9QysnP1fCZrsBdqqBMAYaTYVV/RP/AQDuOv+KvfnsV+mVQ2lNEjSQaU75f4OEheO39HILw=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com;
s=zohoarc;
t=1569337489;
h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results;
bh=yEH0AMYMJZQtmnAaf9GSWwAh9NpHQ+OyRT9QVgkcJsU=;
b=a00rGA9NUSZpZAa1CPzbJvXM3nxIwmk9hbFOrM7bQbMHuTs3QMpE17IqvfsOuQey7GqfYAHx93PNOM7xr9Qqpfx+C8t+jotcYC0SZTBJjHgcgNxo74HA2TjknDfQtiOTpIRuA17r/mwlel8D19hhH0jHZzAtE+pSLvHupeyie3E=
ARC-Authentication-Results: i=1; mx.zoho.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass header.from= (p=none dis=none)
header.from=
Return-Path:
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by
mx.zohomail.com
with SMTPS id 1569337489055804.4776951300349;
Tue, 24 Sep 2019 08:04:49 -0700 (PDT)
Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com
[10.5.11.11])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by mx1.redhat.com (Postfix) with ESMTPS id 0B8AE898112;
Tue, 24 Sep 2019 15:04:47 +0000 (UTC)
Received: from colo-mx.corp.redhat.com
(colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20])
by smtp.corp.redhat.com (Postfix) with ESMTPS id CB6036012D;
Tue, 24 Sep 2019 15:04:46 +0000 (UTC)
Received: from lists01.pubmisc.prod.ext.phx2.redhat.com
(lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33])
by colo-mx.corp.redhat.com (Postfix) with ESMTP id 843A41803B49;
Tue, 24 Sep 2019 15:04:46 +0000 (UTC)
Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com
[10.5.11.14])
by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
id x8OF4X3q027953 for ;
Tue, 24 Sep 2019 11:04:33 -0400
Received: by smtp.corp.redhat.com (Postfix)
id 178CA5DD6D; Tue, 24 Sep 2019 15:04:33 +0000 (UTC)
Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com
[10.33.9.32])
by smtp.corp.redhat.com (Postfix) with ESMTP id 7DA755D9E1;
Tue, 24 Sep 2019 15:04:32 +0000 (UTC)
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?=
To: libvir-list@redhat.com
Date: Tue, 24 Sep 2019 15:58:54 +0100
Message-Id: <20190924145903.17123-14-berrange@redhat.com>
In-Reply-To: <20190924145903.17123-1-berrange@redhat.com>
References: <20190924145903.17123-1-berrange@redhat.com>
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14
X-loop: libvir-list@redhat.com
Subject: [libvirt] [PATCH v3 13/22] src: rewrite driver name checker in
Python
X-BeenThere: libvir-list@redhat.com
X-Mailman-Version: 2.1.12
Precedence: junk
List-Id: Development discussions about the libvirt library & tools
List-Unsubscribe: ,
List-Archive:
List-Post:
List-Help:
List-Subscribe: ,
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Sender: libvir-list-bounces@redhat.com
Errors-To: libvir-list-bounces@redhat.com
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11
X-Greylist: Sender IP whitelisted,
not delayed by milter-greylist-4.6.2 (mx1.redhat.com [10.5.110.67]);
Tue, 24 Sep 2019 15:04:48 +0000 (UTC)
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the check-drivername.pl tool in Python.
This was mostly a straight conversion, manually going line-by-line
to change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
In testing though it was discovered the existing code was broken
since it hadn't been updated after driver.h was split into many
files. Since the old code is being thrown away, the fix was done
as part of the rewrite rather than split into a separate commit.
Signed-off-by: Daniel P. Berrang=C3=A9
---
src/Makefile.am | 18 +++++--
src/check-drivername.pl | 83 -----------------------------
src/check-drivername.py | 114 ++++++++++++++++++++++++++++++++++++++++
3 files changed, 129 insertions(+), 86 deletions(-)
delete mode 100755 src/check-drivername.pl
create mode 100644 src/check-drivername.py
diff --git a/src/Makefile.am b/src/Makefile.am
index 5226871a73..0d387a1335 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -335,15 +335,27 @@ check-protocol:
endif !WITH_REMOTE
EXTRA_DIST +=3D $(PROTOCOL_STRUCTS)
=20
+DRIVERS =3D \
+ $(srcdir)/driver-hypervisor.h \
+ $(srcdir)/driver-interface.h \
+ $(srcdir)/driver-network.h \
+ $(srcdir)/driver-nodedev.h \
+ $(srcdir)/driver-nwfilter.h \
+ $(srcdir)/driver-secret.h \
+ $(srcdir)/driver-state.h \
+ $(srcdir)/driver-storage.h \
+ $(srcdir)/driver-stream.h \
+ $(NULL)
+
check-drivername:
- $(AM_V_GEN)$(PERL) $(srcdir)/check-drivername.pl \
- $(srcdir)/driver.h \
+ $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(srcdir)/check-drivername.py \
+ $(DRIVERS) \
$(srcdir)/libvirt_public.syms \
$(srcdir)/libvirt_admin_public.syms \
$(srcdir)/libvirt_qemu.syms \
$(srcdir)/libvirt_lxc.syms
=20
-EXTRA_DIST +=3D check-drivername.pl
+EXTRA_DIST +=3D check-drivername.py
=20
check-driverimpls:
$(AM_V_GEN)$(PERL) $(srcdir)/check-driverimpls.pl \
diff --git a/src/check-drivername.pl b/src/check-drivername.pl
deleted file mode 100755
index 3a62193e33..0000000000
--- a/src/check-drivername.pl
+++ /dev/null
@@ -1,83 +0,0 @@
-#!/usr/bin/env perl
-#
-# Copyright (C) 2013 Red Hat, Inc.
-#
-# This library is free software; you can redistribute it and/or
-# modify it under the terms of the GNU Lesser General Public
-# License as published by the Free Software Foundation; either
-# version 2.1 of the License, or (at your option) any later version.
-#
-# This library is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-# Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public
-# License along with this library. If not, see
-# .
-#
-
-use strict;
-use warnings;
-
-my $drvfile =3D shift;
-my @symfiles =3D @ARGV;
-
-my %symbols;
-
-foreach my $symfile (@symfiles) {
- open SYMFILE, "<", $symfile
- or die "cannot read $symfile: $!";
- while () {
- if (/^\s*(vir\w+)\s*;\s*$/) {
- $symbols{$1} =3D 1;
- }
- }
-
- close SYMFILE;
-}
-
-open DRVFILE, "<", $drvfile
- or die "cannot read $drvfile: $!";
-
-my $status =3D 0;
-
-while () {
- next if /virDrvConnectSupportsFeature/;
- if (/\*(virDrv\w+)\s*\)/) {
-
- my $drv =3D $1;
-
- next if $drv =3D~ /virDrvState/;
- next if $drv =3D~ /virDrvDomainMigrate(Prepare|Perform|Confirm|Beg=
in|Finish)/;
-
- my $sym =3D $drv;
- $sym =3D~ s/virDrv/vir/;
-
- unless (exists $symbols{$sym}) {
- print "Driver method name $drv doesn't match public API name\n=
";
- $status =3D 1;
- }
- } elsif (/^\*(vir\w+)\s*\)/) {
- my $name =3D $1;
- print "Bogus name $1\n";
- $status =3D 1;
- } elsif (/^\s*(virDrv\w+)\s+(\w+);\s*/) {
- my $drv =3D $1;
- my $field =3D $2;
-
- my $tmp =3D $drv;
- $tmp =3D~ s/virDrv//;
- $tmp =3D~ s/^NWFilter/nwfilter/;
- $tmp =3D~ s/^(\w)/lc $1/e;
-
- unless ($tmp eq $field) {
- print "Driver struct field $field should be named $tmp\n";
- $status =3D 1;
- }
- }
-}
-
-close DRVFILE;
-
-exit $status;
diff --git a/src/check-drivername.py b/src/check-drivername.py
new file mode 100644
index 0000000000..2f024f5dee
--- /dev/null
+++ b/src/check-drivername.py
@@ -0,0 +1,114 @@
+#!/usr/bin/env python
+#
+# Copyright (C) 2013-2019 Red Hat, Inc.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library. If not, see
+# .
+#
+
+from __future__ import print_function
+
+import re
+import sys
+
+drvfiles =3D []
+symfiles =3D []
+for arg in sys.argv:
+ if arg.endswith(".h"):
+ drvfiles.append(arg)
+ else:
+ symfiles.append(arg)
+
+symbols =3D {}
+
+for symfile in symfiles:
+ with open(symfile, "r") as fh:
+ for line in fh:
+ m =3D re.match(r'''^\s*(vir\w+)\s*;\s*$''', line)
+ if m is not None:
+ symbols[m.group(1)] =3D True
+
+status =3D 0
+for drvfile in drvfiles:
+ with open(drvfile, "r") as fh:
+ for line in fh:
+ if line.find("virDrvConnectSupportsFeature") !=3D -1:
+ continue
+
+ m =3D re.match(r'''.*\*(virDrv\w+)\s*\).*''', line)
+ if m is not None:
+ drv =3D m.group(1)
+
+ skip =3D [
+ "virDrvStateInitialize",
+ "virDrvStateCleanup",
+ "virDrvStateReload",
+ "virDrvStateStop",
+ "virDrvConnectURIProbe",
+ "virDrvDomainMigratePrepare",
+ "virDrvDomainMigratePrepare2",
+ "virDrvDomainMigratePrepare3",
+ "virDrvDomainMigratePrepare3Params",
+ "virDrvDomainMigratePrepareTunnel",
+ "virDrvDomainMigratePrepareTunnelParams",
+ "virDrvDomainMigratePrepareTunnel3",
+ "virDrvDomainMigratePrepareTunnel3Params",
+ "virDrvDomainMigratePerform",
+ "virDrvDomainMigratePerform3",
+ "virDrvDomainMigratePerform3Params",
+ "virDrvDomainMigrateConfirm",
+ "virDrvDomainMigrateConfirm3",
+ "virDrvDomainMigrateConfirm3Params",
+ "virDrvDomainMigrateBegin",
+ "virDrvDomainMigrateBegin3",
+ "virDrvDomainMigrateBegin3Params",
+ "virDrvDomainMigrateFinish",
+ "virDrvDomainMigrateFinish2",
+ "virDrvDomainMigrateFinish3",
+ "virDrvDomainMigrateFinish3Params",
+ "virDrvStreamInData",
+ ]
+ if drv in skip:
+ continue
+
+ sym =3D drv.replace("virDrv", "vir")
+
+ if sym not in symbols:
+ print("Driver method name %s doesn't match public API"=
%
+ drv)
+ continue
+
+ m =3D re.match(r'''^\*(vir\w+)\s*\)''', line)
+ if m is not None:
+ name =3D m.group(1)
+ print("Bogus name %s" % name)
+ status =3D 1
+ continue
+
+ m =3D re.match(r'''^\s*(virDrv\w+)\s+(\w+);\s*''', line)
+ if m is not None:
+ drv =3D m.group(1)
+ field =3D m.group(2)
+
+ tmp =3D drv.replace("virDrv", "")
+ if tmp.startswith("NWFilter"):
+ tmp =3D "nwfilter" + tmp[8:]
+ tmp =3D tmp[0:1].lower() + tmp[1:]
+
+ if tmp !=3D field:
+ print("Driver struct field %s should be named %s" %
+ (field, tmp))
+ status =3D 1
+
+sys.exit(status)
--=20
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
From nobody Thu May 2 16:33:49 2024
Delivered-To: importer@patchew.org
Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28
as permitted sender) client-ip=209.132.183.28;
envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com;
Authentication-Results: mx.zohomail.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass(p=none dis=none) header.from=redhat.com
ARC-Seal: i=1; a=rsa-sha256; t=1569337490; cv=none;
d=zoho.com; s=zohoarc;
b=lmk+V++XPFiuHqdsqZMdIeMwwEJmFUbkR3fzEEN9+8YQLLVG+r6Sl3KDMgh8edEnx0i1VJPjflLwm1QoesjEWzzQggR7rM6kQi/qd1ANnNaaz7QLz3hKIKtIrwBXqeLY+yj/fYbX/Zt8Q98Ikee82O5sxdVvIp7LHGRI4X1qLp0=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com;
s=zohoarc;
t=1569337490;
h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results;
bh=i+ruSwte2uVAgVsiTApq5nlqyfCkA4EIM7AdEWlJeDo=;
b=gfNp+0bGqZ0bScHdMh5u7xaMf4TDfhRQnj4QTQaAi0LaNChHQWTRZgUQV6afnPVGwfCB9mup+y5rDO1ZSeRZyoHJIluQe5wU+htn8WvUmR/eV1lrEbZI9mnIC2dzkuar1DcyS7f0JlrXB4xNU/E1C8A6cCm5IlJfFs0c5nJTLxI=
ARC-Authentication-Results: i=1; mx.zoho.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass header.from= (p=none dis=none)
header.from=
Return-Path:
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by
mx.zohomail.com
with SMTPS id 156933749070169.72797055902993;
Tue, 24 Sep 2019 08:04:50 -0700 (PDT)
Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com
[10.5.11.13])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by mx1.redhat.com (Postfix) with ESMTPS id 867A863704;
Tue, 24 Sep 2019 15:04:48 +0000 (UTC)
Received: from colo-mx.corp.redhat.com
(colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21])
by smtp.corp.redhat.com (Postfix) with ESMTPS id 59E25608C0;
Tue, 24 Sep 2019 15:04:48 +0000 (UTC)
Received: from lists01.pubmisc.prod.ext.phx2.redhat.com
(lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33])
by colo-mx.corp.redhat.com (Postfix) with ESMTP id 088AD4EE96;
Tue, 24 Sep 2019 15:04:48 +0000 (UTC)
Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com
[10.5.11.14])
by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
id x8OF4XbG027961 for ;
Tue, 24 Sep 2019 11:04:34 -0400
Received: by smtp.corp.redhat.com (Postfix)
id EF78A5D9DC; Tue, 24 Sep 2019 15:04:33 +0000 (UTC)
Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com
[10.33.9.32])
by smtp.corp.redhat.com (Postfix) with ESMTP id 5DA9A5DAAC;
Tue, 24 Sep 2019 15:04:33 +0000 (UTC)
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?=
To: libvir-list@redhat.com
Date: Tue, 24 Sep 2019 15:58:55 +0100
Message-Id: <20190924145903.17123-15-berrange@redhat.com>
In-Reply-To: <20190924145903.17123-1-berrange@redhat.com>
References: <20190924145903.17123-1-berrange@redhat.com>
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14
X-loop: libvir-list@redhat.com
Subject: [libvirt] [PATCH v3 14/22] src: rewrite driver impl checker in
Python
X-BeenThere: libvir-list@redhat.com
X-Mailman-Version: 2.1.12
Precedence: junk
List-Id: Development discussions about the libvirt library & tools
List-Unsubscribe: ,
List-Archive:
List-Post:
List-Help:
List-Subscribe: ,
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Sender: libvir-list-bounces@redhat.com
Errors-To: libvir-list-bounces@redhat.com
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13
X-Greylist: Sender IP whitelisted,
not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.39]);
Tue, 24 Sep 2019 15:04:49 +0000 (UTC)
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the check-driverimpls.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Signed-off-by: Daniel P. Berrang=C3=A9
---
src/Makefile.am | 4 +-
src/check-driverimpls.pl | 80 ------------------------------
src/check-driverimpls.py | 102 +++++++++++++++++++++++++++++++++++++++
3 files changed, 104 insertions(+), 82 deletions(-)
delete mode 100755 src/check-driverimpls.pl
create mode 100755 src/check-driverimpls.py
diff --git a/src/Makefile.am b/src/Makefile.am
index 0d387a1335..1066f7f7cc 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -358,7 +358,7 @@ check-drivername:
EXTRA_DIST +=3D check-drivername.py
=20
check-driverimpls:
- $(AM_V_GEN)$(PERL) $(srcdir)/check-driverimpls.pl \
+ $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(srcdir)/check-driverimpls.py \
$(filter /%,$(DRIVER_SOURCE_FILES)) \
$(filter $(srcdir)/%,$(DRIVER_SOURCE_FILES)) \
$(addprefix $(srcdir)/,$(filter-out $(srcdir)/%, \
@@ -374,7 +374,7 @@ check-aclperms:
$(srcdir)/access/viraccessperm.h \
$(srcdir)/access/viraccessperm.c
=20
-EXTRA_DIST +=3D check-driverimpls.pl check-aclrules.pl check-aclperms.py
+EXTRA_DIST +=3D check-driverimpls.py check-aclrules.pl check-aclperms.py
=20
check-local: check-protocol check-symfile check-symsorting \
check-drivername check-driverimpls check-aclrules \
diff --git a/src/check-driverimpls.pl b/src/check-driverimpls.pl
deleted file mode 100755
index 3c0d54724c..0000000000
--- a/src/check-driverimpls.pl
+++ /dev/null
@@ -1,80 +0,0 @@
-#!/usr/bin/env perl
-#
-# Copyright (C) 2013 Red Hat, Inc.
-#
-# This library is free software; you can redistribute it and/or
-# modify it under the terms of the GNU Lesser General Public
-# License as published by the Free Software Foundation; either
-# version 2.1 of the License, or (at your option) any later version.
-#
-# This library is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-# Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public
-# License along with this library. If not, see
-# .
-#
-
-use strict;
-use warnings;
-
-my $intable =3D 0;
-my $table;
-my $mainprefix;
-
-my $status =3D 0;
-while (<>) {
- if ($intable) {
- if (/}/) {
- $intable =3D 0;
- $table =3D undef;
- $mainprefix =3D undef;
- } elsif (/\.(\w+)\s*=3D\s*(\w+),?/) {
- my $api =3D $1;
- my $impl =3D $2;
-
- next if $api eq "no";
- next if $api eq "name";
- next if $impl eq "NULL";
-
- my $suffix =3D $impl;
- my $prefix =3D $impl;
- $prefix =3D~ s/^([a-z]+)(.*?)$/$1/;
-
- if (defined $mainprefix) {
- if ($mainprefix ne $prefix) {
- print "$ARGV:$. Bad prefix '$prefix' for API '$api', e=
xpecting '$mainprefix'\n";
- $status =3D 1;
- }
- } else {
- $mainprefix =3D $prefix;
- }
-
- if ($api !~ /^$mainprefix/) {
- $suffix =3D~ s/^[a-z]+//;
- $suffix =3D~ s/^([A-Z]+)/lc $1/e;
- }
-
- if ($api ne $suffix) {
- my $want =3D $api;
- $want =3D~ s/^nwf/NWF/;
- if ($api !~ /^$mainprefix/) {
- $want =3D~ s/^([a-z])/uc $1/e;
- $want =3D $mainprefix . $want;
- }
- print "$ARGV:$. Bad impl name '$impl' for API '$api', expe=
cting '$want'\n";
- $status =3D 1;
- }
- }
- } elsif (/^(?:static\s+)?(vir(?:\w+)?Driver)\s+(?!.*;)/) {
- next if $1 eq "virNWFilterCallbackDriver" ||
- $1 eq "virNWFilterTechDriver" ||
- $1 eq "virConnectDriver";
- $intable =3D 1;
- $table =3D $1;
- }
-}
-
-exit $status;
diff --git a/src/check-driverimpls.py b/src/check-driverimpls.py
new file mode 100755
index 0000000000..30c92698b3
--- /dev/null
+++ b/src/check-driverimpls.py
@@ -0,0 +1,102 @@
+#!/usr/bin/env python
+#
+# Copyright (C) 2013-2019 Red Hat, Inc.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library. If not, see
+# .
+#
+
+from __future__ import print_function
+
+import re
+import sys
+
+
+def checkdriverimpls(filename):
+ intable =3D False
+ mainprefix =3D None
+
+ errs =3D False
+ with open(filename, "r") as fh:
+ lineno =3D 0
+ for line in fh:
+ lineno =3D lineno + 1
+ if intable:
+ if line.find("}") !=3D -1:
+ intable =3D False
+ mainprefix =3D None
+ continue
+
+ m =3D re.match(r'''\s*\.(\w+)\s*=3D\s*(\w+),?\s*/''', line)
+ if m is not None:
+ api =3D m.group(1)
+ impl =3D m.group(2)
+
+ if api in ["no", "name"]:
+ continue
+ if impl in ["NULL"]:
+ continue
+
+ suffix =3D impl
+ prefix =3D re.sub(r'''^([a-z]+)(.*?)$''', r'''\1''', i=
mpl)
+
+ if mainprefix is not None:
+ if mainprefix !=3D prefix:
+ print("%s:%d Bad prefix '%s' for API '%s', " +
+ "expecting '%s'" %
+ (filename, lineno, prefix, api, mainpref=
ix),
+ file=3Dsys.stderr)
+ errs =3D True
+ else:
+ mainprefix =3D prefix
+
+ if not api.startswith(mainprefix):
+ suffix =3D re.sub(r'''^[a-z]+''', "", suffix)
+ suffix =3D re.sub(r'''^([A-Z]+)''',
+ lambda m: m.group(1).lower(), suff=
ix)
+
+ if api !=3D suffix:
+ want =3D api
+ if want.startswith("nwf"):
+ want =3D "NWF" + want[3:]
+
+ if not api.startswith(mainprefix):
+ want =3D re.sub(r'''^([a-z])''',
+ lambda m: m.group(1).upper(), wa=
nt)
+ want =3D mainprefix + want
+
+ print("%s:%d Bad impl name '%s' for API " +
+ "'%s', expecting '%s'" %
+ (filename, lineno, impl, api, want),
+ file=3Dsys.stderr)
+ errs =3D True
+ else:
+ m =3D re.match(r'''^(?:static\s+)?(vir(?:\w+)?Driver)''' +
+ r'''\s+(?!.*;)''', line)
+ if m is not None:
+ drv =3D m.group(1)
+ if drv in ["virNWFilterCallbackDriver",
+ "virNWFilterTechDriver",
+ "virConnectDriver"]:
+ continue
+ intable =3D True
+
+ return errs
+
+
+status =3D 0
+for filename in sys.argv[1:]:
+ if checkdriverimpls(filename):
+ status =3D 1
+sys.exit(status)
--=20
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
From nobody Thu May 2 16:33:49 2024
Delivered-To: importer@patchew.org
Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28
as permitted sender) client-ip=209.132.183.28;
envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com;
Authentication-Results: mx.zohomail.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass(p=none dis=none) header.from=redhat.com
ARC-Seal: i=1; a=rsa-sha256; t=1569337489; cv=none;
d=zoho.com; s=zohoarc;
b=hmPgLFauPnbFd+/I6rBlzS9fuDdXNGwFvD/NgQDBIS7+0B+UaZGCZ9WyBRIzh4bzm1a0XFLS0YlaP4Jqfg3pjRgwDWKehqxmJbueC26FoK3ONdaQgeNVXIeULcUC6wvISctsG7EtftvNN3wV1JUEV8gKwjTyD1SA+q9JRqfKkwU=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com;
s=zohoarc;
t=1569337489;
h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results;
bh=MEDir+9Hv75AGvnuGJfWbxtOS9gvSdENNf8sIz4FzJs=;
b=ZK513odE+r0IxsGcEJQfvkCVcdKwR7B2faS0bkqQZE8kD1eEA4MFTjDjiU3jWAT0wqeaWiV0LO003c4EYWJBXB3n8zIqC0mY2yuOUNiQX6HAUHYV/fJJ6KgEcNRFaEiCyF3+oYXOUhj1l4RWLzJ5WPNUUx3mq10qPQXKo+0pxCY=
ARC-Authentication-Results: i=1; mx.zoho.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass header.from= (p=none dis=none)
header.from=
Return-Path:
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by
mx.zohomail.com
with SMTPS id 1569337489136812.4073128770021;
Tue, 24 Sep 2019 08:04:49 -0700 (PDT)
Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com
[10.5.11.22])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by mx1.redhat.com (Postfix) with ESMTPS id 808023090FC7;
Tue, 24 Sep 2019 15:04:47 +0000 (UTC)
Received: from colo-mx.corp.redhat.com
(colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21])
by smtp.corp.redhat.com (Postfix) with ESMTPS id 5B1521001956;
Tue, 24 Sep 2019 15:04:47 +0000 (UTC)
Received: from lists01.pubmisc.prod.ext.phx2.redhat.com
(lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33])
by colo-mx.corp.redhat.com (Postfix) with ESMTP id 1B1464EE75;
Tue, 24 Sep 2019 15:04:47 +0000 (UTC)
Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com
[10.5.11.14])
by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
id x8OF4jFV027989 for ;
Tue, 24 Sep 2019 11:04:45 -0400
Received: by smtp.corp.redhat.com (Postfix)
id 7EAB35DD78; Tue, 24 Sep 2019 15:04:45 +0000 (UTC)
Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com
[10.33.9.32])
by smtp.corp.redhat.com (Postfix) with ESMTP id DBF995DE5B;
Tue, 24 Sep 2019 15:04:34 +0000 (UTC)
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?=
To: libvir-list@redhat.com
Date: Tue, 24 Sep 2019 15:58:56 +0100
Message-Id: <20190924145903.17123-16-berrange@redhat.com>
In-Reply-To: <20190924145903.17123-1-berrange@redhat.com>
References: <20190924145903.17123-1-berrange@redhat.com>
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14
X-loop: libvir-list@redhat.com
Subject: [libvirt] [PATCH v3 15/22] src: rewrite ACL rule checker in Python
X-BeenThere: libvir-list@redhat.com
X-Mailman-Version: 2.1.12
Precedence: junk
List-Id: Development discussions about the libvirt library & tools
List-Unsubscribe: ,
List-Archive:
List-Post:
List-Help:
List-Subscribe: ,
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Sender: libvir-list-bounces@redhat.com
Errors-To: libvir-list-bounces@redhat.com
X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22
X-Greylist: Sender IP whitelisted,
not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.43]);
Tue, 24 Sep 2019 15:04:48 +0000 (UTC)
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the check-aclrules.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Signed-off-by: Daniel P. Berrang=C3=A9
---
src/Makefile.am | 4 +-
src/check-aclrules.pl | 252 ----------------------------------------
src/check-aclrules.py | 263 ++++++++++++++++++++++++++++++++++++++++++
3 files changed, 265 insertions(+), 254 deletions(-)
delete mode 100755 src/check-aclrules.pl
create mode 100755 src/check-aclrules.py
diff --git a/src/Makefile.am b/src/Makefile.am
index 1066f7f7cc..ed1e1ca32b 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -365,7 +365,7 @@ check-driverimpls:
$(filter-out /%,$(DRIVER_SOURCE_FILES))))
=20
check-aclrules:
- $(AM_V_GEN)$(PERL) $(srcdir)/check-aclrules.pl \
+ $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(srcdir)/check-aclrules.py \
$(REMOTE_PROTOCOL) \
$(addprefix $(srcdir)/,$(filter-out /%,$(STATEFUL_DRIVER_SOURCE_FILES)))
=20
@@ -374,7 +374,7 @@ check-aclperms:
$(srcdir)/access/viraccessperm.h \
$(srcdir)/access/viraccessperm.c
=20
-EXTRA_DIST +=3D check-driverimpls.py check-aclrules.pl check-aclperms.py
+EXTRA_DIST +=3D check-driverimpls.py check-aclrules.py check-aclperms.py
=20
check-local: check-protocol check-symfile check-symsorting \
check-drivername check-driverimpls check-aclrules \
diff --git a/src/check-aclrules.pl b/src/check-aclrules.pl
deleted file mode 100755
index 0d4cac17ca..0000000000
--- a/src/check-aclrules.pl
+++ /dev/null
@@ -1,252 +0,0 @@
-#!/usr/bin/env perl
-#
-# Copyright (C) 2013-2014 Red Hat, Inc.
-#
-# This library is free software; you can redistribute it and/or
-# modify it under the terms of the GNU Lesser General Public
-# License as published by the Free Software Foundation; either
-# version 2.1 of the License, or (at your option) any later version.
-#
-# This library is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-# Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public
-# License along with this library. If not, see
-# .
-#
-# This script validates that the driver implementation of any
-# public APIs contain ACL checks.
-#
-# As the script reads each source file, it attempts to identify
-# top level function names.
-#
-# When reading the body of the functions, it looks for anything
-# that looks like an API called named XXXEnsureACL. It will
-# validate that the XXX prefix matches the name of the function
-# it occurs in.
-#
-# When it later finds the virDriverPtr table, for each entry
-# point listed, it will validate if there was a previously
-# detected EnsureACL call recorded.
-#
-use strict;
-use warnings;
-
-my $status =3D 0;
-
-my $brace =3D 0;
-my $maybefunc;
-my $intable =3D 0;
-my $table;
-
-my %acls;
-my %aclfilters;
-
-my %whitelist =3D (
- "connectClose" =3D> 1,
- "connectIsEncrypted" =3D> 1,
- "connectIsSecure" =3D> 1,
- "connectIsAlive" =3D> 1,
- "networkOpen" =3D> 1,
- "networkClose" =3D> 1,
- "nwfilterOpen" =3D> 1,
- "nwfilterClose" =3D> 1,
- "secretOpen" =3D> 1,
- "secretClose" =3D> 1,
- "storageOpen" =3D> 1,
- "storageClose" =3D> 1,
- "interfaceOpen" =3D> 1,
- "interfaceClose" =3D> 1,
- "connectURIProbe" =3D> 1,
- "localOnly" =3D> 1,
- "domainQemuAttach" =3D> 1,
- );
-
-# XXX this vzDomainMigrateConfirm3Params looks
-# bogus - determine why it doesn't have a valid
-# ACL check.
-my %implwhitelist =3D (
- "vzDomainMigrateConfirm3Params" =3D> 1,
- );
-
-my $lastfile;
-
-sub fixup_name {
- my $name =3D shift;
-
- $name =3D~ s/Nwfilter/NWFilter/;
- $name =3D~ s/Xml$/XML/;
- $name =3D~ s/Uri$/URI/;
- $name =3D~ s/Uuid$/UUID/;
- $name =3D~ s/Id$/ID/;
- $name =3D~ s/Mac$/MAC/;
- $name =3D~ s/Cpu$/CPU/;
- $name =3D~ s/Os$/OS/;
- $name =3D~ s/Nmi$/NMI/;
- $name =3D~ s/Pm/PM/;
- $name =3D~ s/Fstrim$/FSTrim/;
- $name =3D~ s/Scsi/SCSI/;
- $name =3D~ s/Wwn$/WWN/;
-
- return $name;
-}
-
-sub name_to_ProcName {
- my $name =3D shift;
-
- my @elems;
- if ($name =3D~ /_/ || (lc $name) eq "open" || (lc $name) eq "close") {
- @elems =3D split /_/, $name;
- @elems =3D map lc, @elems;
- @elems =3D map ucfirst, @elems;
- } else {
- @elems =3D $name;
- }
- @elems =3D map { fixup_name($_) } @elems;
- my $procname =3D join "", @elems;
-
- $procname =3D~ s/^([A-Z])/lc $1/e;
-
- return $procname;
-}
-
-
-my $proto =3D shift @ARGV;
-
-open PROTO, "<$proto" or die "cannot read $proto";
-
-my %filtered;
-my $incomment =3D 0;
-my $filtered =3D 0;
-while () {
- if (m,/\*\*,) {
- $incomment =3D 1;
- $filtered =3D 0;
- } elsif ($incomment) {
- if (m,\*\s\@aclfilter,) {
- $filtered =3D 1;
- } elsif ($filtered &&
- m,REMOTE_PROC_(.*)\s+=3D\s*\d+,) {
- my $api =3D name_to_ProcName($1);
- # Event filtering is handled in daemon/remote.c instead of dri=
vers
- if (! m,_EVENT_REGISTER,) {
- $filtered{$api} =3D 1;
- }
- $incomment =3D 0;
- }
- }
-}
-
-close PROTO;
-
-while (<>) {
- if (!defined $lastfile ||
- $lastfile ne $ARGV) {
- %acls =3D ();
- $brace =3D 0;
- $maybefunc =3D undef;
- $lastfile =3D $ARGV;
- }
- if ($brace =3D=3D 0) {
- # Looks for anything which appears to be a function
- # body name. Doesn't matter if we pick up bogus stuff
- # here, as long as we don't miss valid stuff
- if (m,\b(\w+)\(,) {
- $maybefunc =3D $1;
- }
- } elsif ($brace > 0) {
- if (m,(\w+)EnsureACL,) {
- # Record the fact that maybefunc contains an
- # ACL call, and make sure it is the right call!
- my $func =3D $1;
- $func =3D~ s/^vir//;
- if (!defined $maybefunc) {
- print "$ARGV:$. Unexpected check '$func' outside function\=
n";
- $status =3D 1;
- } else {
- unless ($maybefunc =3D~ /$func$/i) {
- print "$ARGV:$. Mismatch check 'vir${func}EnsureACL' f=
or function '$maybefunc'\n";
- $status =3D 1;
- }
- }
- $acls{$maybefunc} =3D 1;
- } elsif (m,(\w+)CheckACL,) {
- # Record the fact that maybefunc contains an
- # ACL filter call, and make sure it is the right call!
- my $func =3D $1;
- $func =3D~ s/^vir//;
- if (!defined $maybefunc) {
- print "$ARGV:$. Unexpected check '$func' outside function\=
n";
- $status =3D 1;
- } else {
- unless ($maybefunc =3D~ /$func$/i) {
- print "$ARGV:$. Mismatch check 'vir${func}CheckACL' fo=
r function '$maybefunc'\n";
- $status =3D 1;
- }
- }
- $aclfilters{$maybefunc} =3D 1;
- } elsif (m,\b(\w+)\(,) {
- # Handles case where we replaced an API with a new
- # one which adds new parameters, and we're left with
- # a simple stub calling the new API.
- my $callfunc =3D $1;
- if (exists $acls{$callfunc}) {
- $acls{$maybefunc} =3D 1;
- }
- if (exists $aclfilters{$callfunc}) {
- $aclfilters{$maybefunc} =3D 1;
- }
- }
- }
-
- # Pass the vir*DriverPtr tables and make sure that
- # every func listed there, has an impl which calls
- # an ACL function
- if ($intable) {
- if (/\}/) {
- $intable =3D 0;
- $table =3D undef;
- } elsif (/\.(\w+)\s*=3D\s*(\w+),?/) {
- my $api =3D $1;
- my $impl =3D $2;
-
- next if $impl eq "NULL";
-
- if ($api ne "no" &&
- $api ne "name" &&
- $table ne "virStateDriver" &&
- !exists $acls{$impl} &&
- !exists $whitelist{$api} &&
- !exists $implwhitelist{$impl}) {
- print "$ARGV:$. Missing ACL check in function '$impl' for =
'$api'\n";
- $status =3D 1;
- }
-
- if (exists $filtered{$api} &&
- !exists $aclfilters{$impl}) {
- print "$ARGV:$. Missing ACL filter in function '$impl' for=
'$api'\n";
- $status =3D 1;
- }
- }
- } elsif (/^(?:static\s+)?(vir(?:\w+)?Driver)\s+/) {
- if ($1 ne "virNWFilterCallbackDriver" &&
- $1 ne "virNWFilterTechDriver" &&
- $1 ne "virDomainConfNWFilterDriver") {
- $intable =3D 1;
- $table =3D $1;
- }
- }
-
-
- my $count;
- $count =3D s/{//g;
- $brace +=3D $count;
- $count =3D s/}//g;
- $brace -=3D $count;
-} continue {
- close ARGV if eof;
-}
-
-exit $status;
diff --git a/src/check-aclrules.py b/src/check-aclrules.py
new file mode 100755
index 0000000000..a7ccc794be
--- /dev/null
+++ b/src/check-aclrules.py
@@ -0,0 +1,263 @@
+#!/usr/bin/env python
+#
+# Copyright (C) 2013-2019 Red Hat, Inc.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library. If not, see
+# .
+#
+# This script validates that the driver implementation of any
+# public APIs contain ACL checks.
+#
+# As the script reads each source file, it attempts to identify
+# top level function names.
+#
+# When reading the body of the functions, it looks for anything
+# that looks like an API called named XXXEnsureACL. It will
+# validate that the XXX prefix matches the name of the function
+# it occurs in.
+#
+# When it later finds the virDriverPtr table, for each entry
+# point listed, it will validate if there was a previously
+# detected EnsureACL call recorded.
+#
+
+from __future__ import print_function
+
+import re
+import sys
+
+whitelist =3D {
+ "connectClose": True,
+ "connectIsEncrypted": True,
+ "connectIsSecure": True,
+ "connectIsAlive": True,
+ "networkOpen": True,
+ "networkClose": True,
+ "nwfilterOpen": True,
+ "nwfilterClose": True,
+ "secretOpen": True,
+ "secretClose": True,
+ "storageOpen": True,
+ "storageClose": True,
+ "interfaceOpen": True,
+ "interfaceClose": True,
+ "connectURIProbe": True,
+ "localOnly": True,
+ "domainQemuAttach": True,
+}
+
+# XXX this vzDomainMigrateConfirm3Params looks
+# bogus - determine why it doesn't have a valid
+# ACL check.
+implwhitelist =3D {
+ "vzDomainMigrateConfirm3Params": True,
+}
+
+lastfile =3D None
+
+
+def fixup_name(name):
+ name.replace("Nwfilter", "NWFilter")
+ name.replace("Pm", "PM")
+ name.replace("Scsi", "SCSI")
+ if name.endswith("Xml"):
+ name =3D name[:-3] + "XML"
+ elif name.endswith("Uri"):
+ name =3D name[:-3] + "URI"
+ elif name.endswith("Uuid"):
+ name =3D name[:-4] + "UUID"
+ elif name.endswith("Id"):
+ name =3D name[:-2] + "ID"
+ elif name.endswith("Mac"):
+ name =3D name[:-3] + "MAC"
+ elif name.endswith("Cpu"):
+ name =3D name[:-3] + "MAC"
+ elif name.endswith("Os"):
+ name =3D name[:-2] + "OS"
+ elif name.endswith("Nmi"):
+ name =3D name[:-3] + "NMI"
+ elif name.endswith("Fstrim"):
+ name =3D name[:-6] + "FSTrim"
+ elif name.endswith("Wwn"):
+ name =3D name[:-3] + "WWN"
+
+ return name
+
+
+def name_to_ProcName(name):
+ elems =3D []
+ if name.find("_") !=3D -1 or name.lower() in ["open", "close"]:
+ elems =3D [n.lower().capitalize() for n in name.split("_")]
+ else:
+ elems =3D [name]
+
+ elems =3D [fixup_name(n) for n in elems]
+ procname =3D "".join(elems)
+
+ return procname[0:1].lower() + procname[1:]
+
+
+proto =3D sys.argv[1]
+
+filteredmap =3D {}
+with open(proto, "r") as fh:
+ incomment =3D False
+ filtered =3D False
+
+ for line in fh:
+ if line.find("/**") !=3D -1:
+ incomment =3D True
+ filtered =3D False
+ elif incomment:
+ if line.find("* @aclfilter") !=3D -1:
+ filtered =3D True
+ elif filtered:
+ m =3D re.match(r'''.*REMOTE_PROC_(.*)\s+=3D\s*\d+.*''', li=
ne)
+ if m is not None:
+ api =3D name_to_ProcName(m.group(1))
+ # Event filtering is handled in daemon/remote.c
+ # instead of drivers
+ if line.find("_EVENT_REGISTER") =3D=3D -1:
+ filteredmap[api] =3D True
+ incomment =3D False
+
+
+def process_file(filename):
+ brace =3D 0
+ maybefunc =3D None
+ intable =3D False
+ table =3D None
+
+ acls =3D {}
+ aclfilters =3D {}
+ errs =3D False
+ with open(filename, "r") as fh:
+ lineno =3D 0
+ for line in fh:
+ lineno =3D lineno + 1
+ if brace =3D=3D 0:
+ # Looks for anything which appears to be a function
+ # body name. Doesn't matter if we pick up bogus stuff
+ # here, as long as we don't miss valid stuff
+ m =3D re.match(r'''.*\b(\w+)\(.*''', line)
+ if m is not None:
+ maybefunc =3D m.group(1)
+ elif brace > 0:
+ ensureacl =3D re.match(r'''.*?(\w+)EnsureACL.*''', line)
+ checkacl =3D re.match(r'''.*?(\w+)CheckACL.*''', line)
+ stub =3D re.match(r'''.*?\b(\w+)\(.*''', line)
+ if ensureacl is not None:
+ # Record the fact that maybefunc contains an
+ # ACL call, and make sure it is the right call!
+ func =3D ensureacl.group(1)
+ if func.startswith("vir"):
+ func =3D func[3:]
+
+ if maybefunc is None:
+ print("%s:%d Unexpected check '%s' outside functio=
n" %
+ (filename, lineno, func), file=3Dsys.stderr)
+ errs =3D True
+ else:
+ if not maybefunc.lower().endswith(func.lower()):
+ print("%s:%d Mismatch check 'vir%sEnsureACL'" +
+ "for function '%s'" %
+ (filename, lineno, func, maybefunc),
+ file=3Dsys.stderr)
+ errs =3D True
+ acls[maybefunc] =3D True
+ elif checkacl:
+ # Record the fact that maybefunc contains an
+ # ACL filter call, and make sure it is the right call!
+ func =3D checkacl.group(1)
+ if func.startswith("vir"):
+ func =3D func[3:]
+
+ if maybefunc is None:
+ print("%s:%d Unexpected check '%s' outside functio=
n" %
+ (filename, lineno, func), file=3Dsys.stderr)
+ errs =3D True
+ else:
+ if not maybefunc.lower().endswith(func.lower()):
+ print("%s:%d Mismatch check 'vir%sEnsureACL' "=
+
+ "for function '%s'" %
+ (filename, lineno, func, maybefunc),
+ file=3Dsys.stderr)
+ errs =3D True
+ aclfilters[maybefunc] =3D True
+ elif stub:
+ # Handles case where we replaced an API with a new
+ # one which adds new parameters, and we're left with
+ # a simple stub calling the new API.
+ callfunc =3D stub.group(1)
+ if callfunc in acls:
+ acls[maybefunc] =3D True
+
+ if callfunc in aclfilters:
+ aclfilters[maybefunc] =3D True
+
+ # Pass the vir*DriverPtr tables and make sure that
+ # every func listed there, has an impl which calls
+ # an ACL function
+ if intable:
+ assign =3D re.match(r'''.*?\.(\w+)\s*=3D\s*(\w+),?.*''', l=
ine)
+ if line.find("}") !=3D -1:
+ intable =3D False
+ table =3D None
+ elif assign is not None:
+ api =3D assign.group(1)
+ impl =3D assign.group(2)
+
+ if (impl !=3D "NULL" and
+ api not in ["no", "name"] and
+ table !=3D "virStateDriver"):
+ if (impl not in acls and
+ api not in whitelist and
+ impl not in implwhitelist):
+ print("%s:%d Missing ACL check in " +
+ "function '%s' for '%s'" %
+ (filename, lineno, impl, api),
+ file=3Dsys.stderr)
+ errs =3D True
+
+ if api in filteredmap and impl not in aclfilters:
+ print("%s:%d Missing ACL filter in " +
+ "function '%s' for '%s'" %
+ (filename, lineno, impl, api),
+ file=3Dsys.stderr)
+ errs =3D True
+ else:
+ m =3D re.match(r'''^(?:static\s+)?(vir(?:\w+)?Driver)\s+''=
',
+ line)
+ if m is not None:
+ name =3D m.group(1)
+ if name not in ["virNWFilterCallbackDriver",
+ "virNWFilterTechDriver",
+ "virDomainConfNWFilterDriver"]:
+ intable =3D True
+ table =3D name
+
+ if line.find("{") !=3D -1:
+ brace =3D brace + 1
+ if line.find("}") !=3D -1:
+ brace =3D brace - 1
+
+ return errs
+
+
+status =3D 0
+for filename in sys.argv[2:]:
+ if process_file(filename):
+ status =3D 1
+
+sys.exit(status)
--=20
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
From nobody Thu May 2 16:33:49 2024
Delivered-To: importer@patchew.org
Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28
as permitted sender) client-ip=209.132.183.28;
envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com;
Authentication-Results: mx.zohomail.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass(p=none dis=none) header.from=redhat.com
ARC-Seal: i=1; a=rsa-sha256; t=1569337493; cv=none;
d=zoho.com; s=zohoarc;
b=OjKP2v0RRfGkZkdAjNJc3PC3n4/7IhicK2O7wdX67E9jyx2oU3/4BSgMpTPC0InhqcaUFUrgqKyfBVNrK2LyUYdDy2kaiueGgZGosl4YBxQSaeacft+vAuEyqbLNQlKYozXddi6ep++MZIVKu3MkKxgkxEUBKHAUuxu5aINlAWc=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com;
s=zohoarc;
t=1569337493;
h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results;
bh=Yr4yzEcr9O+5ak1wGiCO4/LbcSZ3uWCnQU1eRHMuFQI=;
b=MxkB0tbqkLhmjc184fMBOBtbOeMsVRUXZKV6CgICQJdAlUOch5SJnbRSdjI+zYeRHU9NFyfWil0EtkyKAM/52zQqPbiuvBW8HRdQDWVP9Io56d/A4M2gJM0i2kLS4Tyrlc0on9i9lxP6JSTBCZyi6DXRd9Ubq4xustMGCzbiSyY=
ARC-Authentication-Results: i=1; mx.zoho.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass header.from= (p=none dis=none)
header.from=
Return-Path:
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by
mx.zohomail.com
with SMTPS id 1569337493178432.60759270756034;
Tue, 24 Sep 2019 08:04:53 -0700 (PDT)
Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com
[10.5.11.13])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by mx1.redhat.com (Postfix) with ESMTPS id 7B7E83086E25;
Tue, 24 Sep 2019 15:04:51 +0000 (UTC)
Received: from colo-mx.corp.redhat.com
(colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20])
by smtp.corp.redhat.com (Postfix) with ESMTPS id 564A0608C0;
Tue, 24 Sep 2019 15:04:51 +0000 (UTC)
Received: from lists01.pubmisc.prod.ext.phx2.redhat.com
(lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33])
by colo-mx.corp.redhat.com (Postfix) with ESMTP id 11F56180BAA5;
Tue, 24 Sep 2019 15:04:51 +0000 (UTC)
Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com
[10.5.11.14])
by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
id x8OF4ktx028002 for ;
Tue, 24 Sep 2019 11:04:46 -0400
Received: by smtp.corp.redhat.com (Postfix)
id 608FD5DD78; Tue, 24 Sep 2019 15:04:46 +0000 (UTC)
Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com
[10.33.9.32])
by smtp.corp.redhat.com (Postfix) with ESMTP id C393D5DC1B;
Tue, 24 Sep 2019 15:04:45 +0000 (UTC)
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?=
To: libvir-list@redhat.com
Date: Tue, 24 Sep 2019 15:58:57 +0100
Message-Id: <20190924145903.17123-17-berrange@redhat.com>
In-Reply-To: <20190924145903.17123-1-berrange@redhat.com>
References: <20190924145903.17123-1-berrange@redhat.com>
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14
X-loop: libvir-list@redhat.com
Subject: [libvirt] [PATCH v3 16/22] src: rewrite polkit ACL generator in
Python
X-BeenThere: libvir-list@redhat.com
X-Mailman-Version: 2.1.12
Precedence: junk
List-Id: Development discussions about the libvirt library & tools
List-Unsubscribe: ,
List-Archive:
List-Post:
List-Help:
List-Subscribe: ,
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Sender: libvir-list-bounces@redhat.com
Errors-To: libvir-list-bounces@redhat.com
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13
X-Greylist: Sender IP whitelisted,
not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.44]);
Tue, 24 Sep 2019 15:04:51 +0000 (UTC)
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the genpolkit.pl tool in Python.
This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.
Signed-off-by: Daniel P. Berrang=C3=A9
---
src/access/Makefile.inc.am | 7 ++-
src/access/genpolkit.pl | 119 ------------------------------------
src/access/genpolkit.py | 122 +++++++++++++++++++++++++++++++++++++
3 files changed, 126 insertions(+), 122 deletions(-)
delete mode 100755 src/access/genpolkit.pl
create mode 100755 src/access/genpolkit.py
diff --git a/src/access/Makefile.inc.am b/src/access/Makefile.inc.am
index 4dc742f4e5..1550cce720 100644
--- a/src/access/Makefile.inc.am
+++ b/src/access/Makefile.inc.am
@@ -44,7 +44,7 @@ GENERATED_SYM_FILES +=3D $(ACCESS_DRIVER_SYM_FILES)
=20
EXTRA_DIST +=3D \
$(ACCESS_DRIVER_POLKIT_POLICY) \
- access/genpolkit.pl \
+ access/genpolkit.py \
$(NULL)
=20
=20
@@ -63,8 +63,9 @@ libvirt_driver_access_la_LIBADD =3D
=20
=20
$(ACCESS_DRIVER_POLKIT_POLICY): $(srcdir)/access/viraccessperm.h \
- $(srcdir)/access/genpolkit.pl Makefile.am
- $(AM_V_GEN)$(PERL) $(srcdir)/access/genpolkit.pl < $< > $@ || rm -f $@
+ $(srcdir)/access/genpolkit.py Makefile.am
+ $(AM_V_GEN)$(RUNUTF8) $(PYTHON) \
+ $(srcdir)/access/genpolkit.py $< > $@ || rm -f $@
=20
if WITH_POLKIT
libvirt_driver_access_la_SOURCES +=3D $(ACCESS_DRIVER_POLKIT_SOURCES)
diff --git a/src/access/genpolkit.pl b/src/access/genpolkit.pl
deleted file mode 100755
index f8f20caf65..0000000000
--- a/src/access/genpolkit.pl
+++ /dev/null
@@ -1,119 +0,0 @@
-#!/usr/bin/env perl
-#
-# Copyright (C) 2012-2013 Red Hat, Inc.
-#
-# This library is free software; you can redistribute it and/or
-# modify it under the terms of the GNU Lesser General Public
-# License as published by the Free Software Foundation; either
-# version 2.1 of the License, or (at your option) any later version.
-#
-# This library is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-# Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public
-# License along with this library. If not, see
-# .
-#
-
-use strict;
-use warnings;
-
-my @objects =3D (
- "CONNECT", "DOMAIN", "INTERFACE", "NETWORK_PORT",
- "NETWORK","NODE_DEVICE", "NWFILTER_BINDING", "NWFILTER",
- "SECRET", "STORAGE_POOL", "STORAGE_VOL",
- );
-
-my $objects =3D join ("|", @objects);
-
-# Data we're going to be generating looks like this
-#
-#
-#
-# Monitor local virtualized systems
-# System policy prevents monitoring of local virtualized syst=
ems
-#
-# yes
-# yes
-# yes
-#
-#
-# ...more rules...
-#
-
-my %opts;
-my $in_opts =3D 0;
-
-my %perms;
-
-while (<>) {
- if ($in_opts) {
- if (m,\*/,) {
- $in_opts =3D 0;
- } elsif (/\*\s*\@(\w+):\s*(.*?)\s*$/) {
- $opts{$1} =3D $2;
- }
- } elsif (m,/\*\*,) {
- $in_opts =3D 1;
- } elsif (/VIR_ACCESS_PERM_($objects)_((?:\w|_)+),/) {
- my $object =3D lc $1;
- my $perm =3D lc $2;
- next if $perm eq "last";
-
- $object =3D~ s/_/-/g;
- $perm =3D~ s/_/-/g;
-
- $perms{$object} =3D {} unless exists $perms{$object};
- $perms{$object}->{$perm} =3D {
- desc =3D> $opts{desc},
- message =3D> $opts{message},
- anonymous =3D> $opts{anonymous}
- };
- %opts =3D ();
- }
-}
-
-print <
-
-
- Libvirt Project
- https://libvirt.org
-EOF
-
-foreach my $object (sort { $a cmp $b } keys %perms) {
- foreach my $perm (sort { $a cmp $b } keys %{$perms{$object}}) {
- my $description =3D $perms{$object}->{$perm}->{desc};
- my $message =3D $perms{$object}->{$perm}->{message};
- my $anonymous =3D $perms{$object}->{$perm}->{anonymous};
-
- die "missing description for $object.$perm" unless
- defined $description;
- die "missing message for $object.$perm" unless
- defined $message;
-
- my $allow_any =3D $anonymous ? "yes" : "no";
- my $allow_inactive =3D $allow_any;
- my $allow_active =3D $allow_any;
-
- print <
- $description
- $message
-
- $allow_any
- $allow_inactive
- $allow_active
-
-
-EOF
-
- }
-}
-
-print <
-EOF
diff --git a/src/access/genpolkit.py b/src/access/genpolkit.py
new file mode 100755
index 0000000000..04628d2d6a
--- /dev/null
+++ b/src/access/genpolkit.py
@@ -0,0 +1,122 @@
+#!/usr/bin/env python
+#
+# Copyright (C) 2012-2019 Red Hat, Inc.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library. If not, see
+# .
+#
+
+from __future__ import print_function
+
+import re
+import sys
+
+objects =3D [
+ "CONNECT", "DOMAIN", "INTERFACE", "NETWORK_PORT",
+ "NETWORK", "NODE_DEVICE", "NWFILTER_BINDING",
+ "NWFILTER", "SECRET", "STORAGE_POOL", "STORAGE_VOL",
+]
+
+objectstr =3D "|".join(objects)
+
+# Data we're going to be generating looks like this
+#
+#
+#
+# Monitor local virtualized systems
+# System policy prevents monitoring of
+# local virtualized systems
+#
+# yes
+# yes
+# yes
+#
+#
+# ...more rules...
+#
+
+opts =3D {}
+in_opts =3D False
+
+perms =3D {}
+
+aclfile =3D sys.argv[1]
+with open(aclfile, "r") as fh:
+ for line in fh:
+ if in_opts:
+ if line.find("*/") !=3D -1:
+ in_opts =3D False
+ else:
+ m =3D re.match(r'''.*\*\s*\@(\w+):\s*(.*?)\s*$''', line)
+ if m is not None:
+ opts[m.group(1)] =3D m.group(2)
+ elif line.find("**") !=3D -1:
+ in_opts =3D True
+ else:
+ m =3D re.match(r'''.*VIR_ACCESS_PERM_(%s)_((?:\w|_)+),''' %
+ objectstr, line)
+ if m is not None:
+ obj =3D m.group(1).lower()
+ perm =3D m.group(2).lower()
+ if perm =3D=3D "last":
+ continue
+
+ obj =3D obj.replace("_", "-")
+ perm =3D perm.replace("_", "-")
+
+ if obj not in perms:
+ perms[obj] =3D {}
+ perms[obj][perm] =3D {
+ "desc": opts.get("desc", None),
+ "message": opts.get("message", None),
+ "anonymous": opts.get("anonymous", None),
+ }
+ opts =3D {}
+
+print('')
+print('')
+print('')
+print(' Libvirt Project')
+print(' https://libvirt.org')
+
+for obj in sorted(perms.keys()):
+ for perm in sorted(perms[obj].keys()):
+ description =3D perms[obj][perm]["desc"]
+ message =3D perms[obj][perm]["message"]
+ anonymous =3D perms[obj][perm]["anonymous"]
+
+ if description is None:
+ raise Exception("missing description for %s.%s" % (obj, perm))
+ if message is None:
+ raise Exception("missing message for %s.%s" % (obj, perm))
+
+ allow_any =3D "no"
+ if anonymous:
+ allow_any =3D "yes"
+ allow_inactive =3D allow_any
+ allow_active =3D allow_any
+
+ print(' ' % (obj, perm))
+ print(' %s' % description)
+ print(' %s' % message)
+ print(' ')
+ print(' %s' % allow_any)
+ print(' %s' % allow_inactive)
+ print(' %s' % allow_active)
+ print(' ')
+ print(' ')
+
+print('')
--=20
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
From nobody Thu May 2 16:33:49 2024
Delivered-To: importer@patchew.org
Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28
as permitted sender) client-ip=209.132.183.28;
envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com;
Authentication-Results: mx.zohomail.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass(p=none dis=none) header.from=redhat.com
ARC-Seal: i=1; a=rsa-sha256; t=1569337497; cv=none;
d=zoho.com; s=zohoarc;
b=EjhWqXjIZKJE7NGMpuJVsp3T1tjauF6R7fqV1xZ5ILToN3Nk7l8I+Mz1Ptzxp+YidGGkrsyfcmkYEEjSTNsfDyBLMF/Gq4/r/nIev/1y1cAc036fOd81xKMvKWw0P4S3l2Mby699PFAPPzJ7mJFWGDmlIijOsrbB1KJau9yCl5Y=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com;
s=zohoarc;
t=1569337497;
h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results;
bh=d4GuZomtSDFpHgvlEAZvkq+S31KEDaCog7M7WbTQMyg=;
b=n0/g+RnIPcVWK/Z8HGCf73lDP1UqGcvAZiYBr7cveVV4bE1QYT2rTnIExO6rqFQc/sCHcAaafS/kjRzXqUcT9pmGX+bZEwhVOAPGlgerD0+INpSGdpuovNt+st6UwuNxeB2HVypml05W83ohbVDGYDcJcjha9UCQ8fGe/oKf1Fk=
ARC-Authentication-Results: i=1; mx.zoho.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass header.from= (p=none dis=none)
header.from=
Return-Path:
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by
mx.zohomail.com
with SMTPS id 1569337497197262.27110012180026;
Tue, 24 Sep 2019 08:04:57 -0700 (PDT)
Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com
[10.5.11.13])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by mx1.redhat.com (Postfix) with ESMTPS id 908A22A09D3;
Tue, 24 Sep 2019 15:04:55 +0000 (UTC)
Received: from colo-mx.corp.redhat.com
(colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20])
by smtp.corp.redhat.com (Postfix) with ESMTPS id 70D0A608C2;
Tue, 24 Sep 2019 15:04:55 +0000 (UTC)
Received: from lists01.pubmisc.prod.ext.phx2.redhat.com
(lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33])
by colo-mx.corp.redhat.com (Postfix) with ESMTP id 32077180BAA9;
Tue, 24 Sep 2019 15:04:55 +0000 (UTC)
Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com
[10.5.11.14])
by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
id x8OF4lIB028009 for ;
Tue, 24 Sep 2019 11:04:47 -0400
Received: by smtp.corp.redhat.com (Postfix)
id 3EA675D9DC; Tue, 24 Sep 2019 15:04:47 +0000 (UTC)
Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com
[10.33.9.32])
by smtp.corp.redhat.com (Postfix) with ESMTP id A3D73425F;
Tue, 24 Sep 2019 15:04:46 +0000 (UTC)
From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?=
To: libvir-list@redhat.com
Date: Tue, 24 Sep 2019 15:58:58 +0100
Message-Id: <20190924145903.17123-18-berrange@redhat.com>
In-Reply-To: <20190924145903.17123-1-berrange@redhat.com>
References: <20190924145903.17123-1-berrange@redhat.com>
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14
X-loop: libvir-list@redhat.com
Subject: [libvirt] [PATCH v3 17/22] src: rewrite remote protocol checker in
Python
X-BeenThere: libvir-list@redhat.com
X-Mailman-Version: 2.1.12
Precedence: junk
List-Id: Development discussions about the libvirt library & tools
List-Unsubscribe: ,
List-Archive:
List-Post:
List-Help:
List-Subscribe: ,
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Sender: libvir-list-bounces@redhat.com
Errors-To: libvir-list-bounces@redhat.com
X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13
X-Greylist: Sender IP whitelisted,
not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]);
Tue, 24 Sep 2019 15:04:56 +0000 (UTC)
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the pdwtags processing script in Python.
The original inline shell and perl code was completely
unintelligible. The new python code is a manual conversion
that attempts todo basically the same thing.
Signed-off-by: Daniel P. Berrang=C3=A9
---
cfg.mk | 3 +-
src/Makefile.am | 99 ++++---------------------
src/check-remote-protocol.py | 137 +++++++++++++++++++++++++++++++++++
3 files changed, 154 insertions(+), 85 deletions(-)
create mode 100644 src/check-remote-protocol.py
diff --git a/cfg.mk b/cfg.mk
index 18bab6e9a4..66df1448b4 100644
--- a/cfg.mk
+++ b/cfg.mk
@@ -353,6 +353,7 @@ sc_prohibit_mkstemp:
# access with F_OK or R_OK is okay, though.
sc_prohibit_access_xok:
@prohibit=3D'access(at)? *\(.*X_OK' \
+ in_vc_files=3D'\.[ch]$$' \
halt=3D'use virFileIsExecutable instead of access(,X_OK)' \
$(_sc_search_regexp)
=20
@@ -1227,7 +1228,7 @@ exclude_file_name_regexp--sc_prohibit_PATH_MAX =3D \
^cfg\.mk$$
=20
exclude_file_name_regexp--sc_prohibit_access_xok =3D \
- ^(cfg\.mk|src/util/virutil\.c)$$
+ ^(src/util/virutil\.c)$$
=20
exclude_file_name_regexp--sc_prohibit_asprintf =3D \
^(cfg\.mk|bootstrap.conf$$|examples/|src/util/virstring\.[ch]$$|tests/vi=
rcgroupmock\.c|tools/virt-login-shell\.c|tools/nss/libvirt_nss\.c$$)
diff --git a/src/Makefile.am b/src/Makefile.am
index ed1e1ca32b..05ad832668 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -192,84 +192,6 @@ DRIVER_SOURCES +=3D \
=20
=20
=20
-# Ensure that we don't change the struct or member names or member ordering
-# in remote_protocol.x The embedded perl below needs a few comments, and
-# presumes you know what pdwtags output looks like:
-# * use -0777 -n to slurp the entire file into $_.
-# * the "split" splits on the /* DD */ comments, so that $p iterates
-# through the struct definitions.
-# * process only "struct remote_..." entries
-# * remove comments and preceding TAB throughout
-# * remove empty lines throughout
-# * remove white space at end of buffer
-
-# With pdwtags 1.8, --verbose output includes separators like these:
-# /* 93 */
-# /* <0> (null):0 */
-# with the second line omitted for intrinsic types.
-# Whereas with pdwtags 1.3, they look like this:
-# /* <2d2> /usr/include/libio.h:180 */
-# The alternation of the following regexps matches both cases.
-r1 =3D /\* \d+ \*/
-r2 =3D /\* <[[:xdigit:]]+> \S+:\d+ \*/
-libs_prefix =3D remote_|qemu_|lxc_|admin_
-other_prefix =3D keepalive|vir(Net|LockSpace|LXCMonitor)
-struct_prefix =3D ($(libs_prefix)|$(other_prefix))
-
-# Depending on configure options, libtool creates one or both of
-# remote/{,.libs/}libvirt_driver_remote_la-remote_protocol.o. We want
-# the newest of the two, in case configure options changed and a stale
-# file is left around from an earlier build.
-# The pdwtags output is completely different when building with clang
-# which causes the comparison against expected output to fail, so skip
-# if using clang as CC.
-PDWTAGS =3D \
- $(AM_V_GEN)if $(CC) -v 2>&1 | grep -q clang; then \
- echo 'WARNING: skipping pdwtags test with Clang' >&2; \
- exit 0; \
- fi; \
- if (pdwtags --help) > /dev/null 2>&1; then \
- o=3D`ls -t $(<:.lo=3D.$(OBJEXT)) \
- $(subst /,/.libs/,$(<:.lo=3D.$(OBJEXT))) \
- 2>/dev/null | sed -n 1p`; \
- test -f "$$o" || { echo ".o for $< not found" >&2; exit 1; }; \
- pdwtags --verbose $$o > $(@F)-t1 2> $(@F)-t2; \
- if test ! -s $(@F)-t1 && test -s $(@F)-t2; then \
- rm -rf $(@F)-t?; \
- echo 'WARNING: pdwtags appears broken; skipping the $@ test' >&2;\
- else \
- $(PERL) -0777 -n \
- -e 'foreach my $$p (split m!\n*(?:$(r1)|$(r2))\n!) {' \
- -e ' if ($$p =3D~ /^(struct|enum) $(struct_prefix)/ ||' \
- -e ' $$p =3D~ /^enum \{/) {' \
- -e ' $$p =3D~ s!\t*/\*.*?\*/!!sg;' \
- -e ' $$p =3D~ s!\s+\n!\n!sg;' \
- -e ' $$p =3D~ s!\s+$$!!;' \
- -e ' $$p =3D~ s!\t! !g;' \
- -e ' print "$$p\n";' \
- -e ' $$n++;' \
- -e ' }' \
- -e '}' \
- -e 'BEGIN {' \
- -e ' print "/* -*- c -*- */\n";' \
- -e '}' \
- -e 'END {' \
- -e ' if ($$n < 1) {' \
- -e ' warn "WARNING: your pdwtags program is too old\n";' \
- -e ' warn "WARNING: skipping the $@ test\n";' \
- -e ' warn "WARNING: install dwarves-1.3 or newer\n";' \
- -e ' exit 8;' \
- -e ' }' \
- -e '}' \
- < $(@F)-t1 > $(@F)-t3; \
- case $$? in 8) rm -f $(@F)-t?; exit 0;; 0) ;; *) exit 1;; esac;\
- diff -u $(@)s $(@F)-t3; st=3D$$?; rm -f $(@F)-t?; exit $$st; \
- fi; \
- else \
- echo 'WARNING: you lack pdwtags; skipping the $@ test' >&2; \
- echo 'WARNING: install the dwarves package to get pdwtags' >&2; \
- fi
-
# .libs/libvirt.so is built by libtool as a side-effect of the Makefile
# rule for libvirt.la. However, checking symbols relies on Linux ELF layo=
ut
if WITH_LINUX
@@ -306,34 +228,43 @@ PROTOCOL_STRUCTS =3D \
if WITH_REMOTE
check-protocol: $(PROTOCOL_STRUCTS) $(PROTOCOL_STRUCTS:structs=3Dstruct)
=20
+# Ensure that we don't change the struct or member names or member ordering
+# in remote_protocol.x The process-pdwtags.py post-processes output to
+# extract the bits we want.
+
# The .o file that pdwtags parses is created as a side effect of running
# libtool; but from make's perspective we depend on the .lo file.
$(srcdir)/remote_protocol-struct \
$(srcdir)/qemu_protocol-struct \
$(srcdir)/lxc_protocol-struct: \
$(srcdir)/%-struct: remote/libvirt_driver_remote_la-%.lo
- $(PDWTAGS)
+ $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(srcdir)/check-remote-protocol.py \
+ "$(CC)" "$(OBJEXT)" $< $(@)s
$(srcdir)/virnetprotocol-struct $(srcdir)/virkeepaliveprotocol-struct: \
$(srcdir)/%-struct: rpc/libvirt_net_rpc_la-%.lo
- $(PDWTAGS)
+ $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(srcdir)/check-remote-protocol.py \
+ "$(CC)" "$(OBJEXT)" $< $(@)s
if WITH_LXC
$(srcdir)/lxc_monitor_protocol-struct: \
$(srcdir)/%-struct: lxc/libvirt_driver_lxc_impl_la-%.lo
- $(PDWTAGS)
+ $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(srcdir)/check-remote-protocol.py \
+ "$(CC)" "$(OBJEXT)" $< $(@)s
endif WITH_LXC
$(srcdir)/lock_protocol-struct: \
$(srcdir)/%-struct: locking/lockd_la-%.lo
- $(PDWTAGS)
+ $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(srcdir)/check-remote-protocol.py \
+ "$(CC)" "$(OBJEXT)" $< $(@)s
$(srcdir)/admin_protocol-struct: \
$(srcdir)/%-struct: admin/libvirt_admin_la-%.lo
- $(PDWTAGS)
+ $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(srcdir)/check-remote-protocol.py \
+ "$(CC)" "$(OBJEXT)" $< $(@)s
=20
else !WITH_REMOTE
# The $(PROTOCOL_STRUCTS) files must live in git, because they cannot be
# re-generated when configured --without-remote.
check-protocol:
endif !WITH_REMOTE
-EXTRA_DIST +=3D $(PROTOCOL_STRUCTS)
+EXTRA_DIST +=3D $(PROTOCOL_STRUCTS) check-remote-protocol.py
=20
DRIVERS =3D \
$(srcdir)/driver-hypervisor.h \
diff --git a/src/check-remote-protocol.py b/src/check-remote-protocol.py
new file mode 100644
index 0000000000..3b7b915e88
--- /dev/null
+++ b/src/check-remote-protocol.py
@@ -0,0 +1,137 @@
+#!/usr/bin/env python
+#
+# Copyright (C) 2019 Red Hat, Inc.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library. If not, see
+# .
+#
+# This uses pdwtags to check remote protocol defs
+#
+# * the "split" splits on the /* DD */ comments, so that $p iterates
+# through the struct definitions.
+# * process only "struct remote_..." entries
+# * remove comments and preceding TAB throughout
+# * remove empty lines throughout
+# * remove white space at end of buffer
+
+from __future__ import print_function
+
+import os
+import os.path
+import re
+import subprocess
+import sys
+
+cc =3D sys.argv[1]
+objext =3D sys.argv[2]
+proto_lo =3D sys.argv[3]
+expected =3D sys.argv[4]
+
+proto_lo =3D proto_lo.replace("/", "/.libs/")
+
+ccargv =3D cc.split(" ")
+ccargv.append("-v")
+ccproc =3D subprocess.Popen(ccargv, stdout=3Dsubprocess.PIPE,
+ stderr=3Dsubprocess.STDOUT)
+out, err =3D ccproc.communicate()
+out =3D out.decode("utf-8")
+if out.find("clang") !=3D -1:
+ print("WARNING: skipping pdwtags test with Clang", file=3Dsys.stderr)
+ sys.exit(0)
+
+
+def which(program):
+ def is_exe(fpath):
+ return (os.path.isfile(fpath) and
+ os.access(fpath, os.X_OK))
+
+ fpath, fname =3D os.path.split(program)
+ if fpath:
+ if is_exe(program):
+ return program
+ else:
+ for path in os.environ["PATH"].split(os.pathsep):
+ exe_file =3D os.path.join(path, program)
+ if is_exe(exe_file):
+ return exe_file
+
+ return None
+
+
+pdwtags =3D which("pdwtags")
+if pdwtags is None:
+ print("WARNING: you lack pdwtags; skipping the protocol test",
+ file=3Dsys.stderr)
+ print("WARNING: install the dwarves package to get pdwtags",
+ file=3Dsys.stderr)
+ sys.exit(0)
+
+proto_o =3D proto_lo.replace(".lo", ".o")
+
+if not os.path.exists(proto_o):
+ raise Exception("Missing %s", proto_o)
+
+pdwtagsproc =3D subprocess.Popen(["pdwtags", "--verbose", proto_o],
+ stdout=3Dsubprocess.PIPE, stderr=3Dsubproce=
ss.PIPE)
+out, err =3D pdwtagsproc.communicate()
+out =3D out.decode("utf-8")
+err =3D err.decode("utf-8")
+
+if out =3D=3D "" and err !=3D "":
+ print("WARNING: no output, pdwtags appears broken:", file=3Dsys.stderr)
+ for l in err.strip().split("\n"):
+ print("WARNING: %s" % l, file=3Dsys.stderr)
+ print("WARNING: skipping the remote protocol test", file=3Dsys.stderr)
+ sys.exit(0)
+
+# With pdwtags 1.8, --verbose output includes separators like these:
+# /* 93 */
+# /* <0> (null):0 */
+# with the second line omitted for intrinsic types.
+# Whereas with pdwtags 1.3, they look like this:
+# /* <2d2> /usr/include/libio.h:180 */
+# The alternation of the following regexps matches both cases.
+r1 =3D r'''/\* \d+ \*/'''
+r2 =3D r'''/\* <[0-9a-fA-F]+> \S+:\d+ \*/'''
+
+libs_prefix =3D "remote_|qemu_|lxc_|admin_"
+other_prefix =3D "keepalive|vir(Net|LockSpace|LXCMonitor)"
+struct_prefix =3D "(" + libs_prefix + "|" + other_prefix + ")"
+
+n =3D 0
+bits =3D re.split(r'''\n*(?:%s|%s)\n''' % (r1, r2), out)
+actual =3D ["/* -*- c -*- */"]
+
+for bit in bits:
+ if (re.match(r'''^(struct|enum)\s+''' + struct_prefix, bit) or
+ bit.startswith("enum {")):
+ bit =3D re.sub(r'''\t*/\*.*?\*/''', "", bit)
+ bit =3D re.sub(r'''\s+\n''', '''\n''', bit)
+ bit =3D re.sub(r'''\s+$''', "", bit)
+ bit =3D re.sub(r'''\t''', " ", bit)
+ actual.append(bit)
+ n =3D n + 1
+
+if n < 1:
+ print("WARNING: No structs/enums matched. Your", file=3Dsys.stderr)
+ print("WARNING: pdwtags program is probably too old", file=3Dsys.stder=
r)
+ print("WARNING: skipping the remote protocol test", file=3Dsys.stderr)
+ print("WARNING: install dwarves-1.3 or newer", file=3Dsys.stderr)
+ sys.exit(8)
+
+diff =3D subprocess.Popen(["diff", "-u", expected, "-"], stdin=3Dsubproces=
s.PIPE)
+actualstr =3D "\n".join(actual) + "\n"
+diff.communicate(input=3Dactualstr.encode("utf-8"))
+
+sys.exit(diff.returncode)
--=20
2.21.0
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
From nobody Thu May 2 16:33:49 2024
Delivered-To: importer@patchew.org
Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28
as permitted sender) client-ip=209.132.183.28;
envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com;
Authentication-Results: mx.zohomail.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass(p=none dis=none) header.from=redhat.com
ARC-Seal: i=1; a=rsa-sha256; t=1569337493; cv=none;
d=zoho.com; s=zohoarc;
b=my1/96fB1SMWIDxoL7YFIR12dvxjzmXZQFHG8UUST8sW/Mw6EFLlLeLlzZYpn38WxWFQyOpMkwrH2gpR98bnxX+yMGv5mL+EZ3sujJEnDAI8zE7pfpGoK66a8CU3Whn+wUuzRMkNP2gGBO9VKGfDQbbljFga55NMt1MXy8nB3NI=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com;
s=zohoarc;
t=1569337493;
h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results;
bh=MmpC/3oC+/+QqijGWBppoq8EeZ9b2EJ4dzhGSYgmd9Y=;
b=N9EEbYIkXhA8E6AFYKJt1w5JljtwrFDiJloDJcJuSnIp6QDX5oqHhJtGKeqCUpVIPMNQII5AK2EWKz8GcsyiKmzvkHeRODvcaw/02U+vDiUhoNrnPc3taHkKXTeIsVWDHMfskh5sqUVWX2VvjV4JKqYGTwUc+GHYQzcF9hkSoNc=
ARC-Authentication-Results: i=1; mx.zoho.com;
spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as
permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com;
dmarc=pass header.from= (p=none dis=none)
header.from=
Return-Path:
Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by
mx.zohomail.com
with SMTPS id 1569337493200312.11924048035473;
Tue, 24 Sep 2019 08:04:53 -0700 (PDT)
Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com
[10.5.11.11])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by mx1.redhat.com (Postfix) with ESMTPS id ACE121918640;
Tue, 24 Sep 2019 15:04:51 +0000 (UTC)
Received: from colo-mx.corp.redhat.com
(colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21])
by smtp.corp.redhat.com (Postfix) with ESMTPS id 83D5F600C8;
Tue, 24 Sep 2019 15:04:51 +0000 (UTC)
Received: from lists01.pubmisc.prod.ext.phx2.redhat.com
(lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33])
by colo-mx.corp.redhat.com (Postfix) with ESMTP id 404684EEB8;
Tue, 24 Sep 2019 15:04:51 +0000 (UTC)
Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com
[10.5.11.14])
by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP
id x8OF4muh028051 for