[libvirt] [PATCH v5 09/23] src: rewrite systemtap function generator in Python

Daniel P. Berrangé posted 23 patches 5 years ago
There is a newer version of this series
[libvirt] [PATCH v5 09/23] src: rewrite systemtap function generator in Python
Posted by Daniel P. Berrangé 5 years ago
As part of an goal to eliminate Perl from libvirt build tools,
rewrite the gensystemtap.pl tool in Python.

This was a straight conversion, manually going line-by-line to
change the syntax from Perl to Python. Thus the overall structure
of the file and approach is the same.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
---
 Makefile.am               |   1 +
 build-aux/syntax-check.mk |   4 +-
 scripts/gensystemtap.py   | 184 ++++++++++++++++++++++++++++++++++++
 src/Makefile.am           |   5 +-
 src/rpc/Makefile.inc.am   |   1 -
 src/rpc/gensystemtap.pl   | 193 --------------------------------------
 6 files changed, 189 insertions(+), 199 deletions(-)
 create mode 100755 scripts/gensystemtap.py
 delete mode 100755 src/rpc/gensystemtap.pl

diff --git a/Makefile.am b/Makefile.am
index e50a29635b..afed409c94 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -51,6 +51,7 @@ EXTRA_DIST = \
   scripts/check-symfile.py \
   scripts/check-symsorting.py \
   scripts/dtrace2systemtap.py \
+  scripts/gensystemtap.py \
   scripts/header-ifdef.py \
   scripts/minimize-po.py \
   scripts/mock-noinline.py \
diff --git a/build-aux/syntax-check.mk b/build-aux/syntax-check.mk
index 016e05e7a2..26eb5b94d9 100644
--- a/build-aux/syntax-check.mk
+++ b/build-aux/syntax-check.mk
@@ -466,6 +466,7 @@ sc_prohibit_risky_id_promotion:
 # since gnulib has more guarantees for snprintf portability
 sc_prohibit_sprintf:
 	@prohibit='\<[s]printf\>' \
+	in_vc_files='\.[ch]$$' \
 	halt='use snprintf, not sprintf' \
 	  $(_sc_search_regexp)
 
@@ -2261,9 +2262,6 @@ exclude_file_name_regexp--sc_prohibit_readlink = \
 
 exclude_file_name_regexp--sc_prohibit_setuid = ^src/util/virutil\.c|tools/virt-login-shell\.c$$
 
-exclude_file_name_regexp--sc_prohibit_sprintf = \
-  ^(build-aux/syntax-check\.mk|docs/hacking\.html\.in|.*\.stp|.*\.pl)$$
-
 exclude_file_name_regexp--sc_prohibit_strncpy = ^src/util/virstring\.c$$
 
 exclude_file_name_regexp--sc_prohibit_strtol = ^examples/.*$$
diff --git a/scripts/gensystemtap.py b/scripts/gensystemtap.py
new file mode 100755
index 0000000000..c4bc1620d0
--- /dev/null
+++ b/scripts/gensystemtap.py
@@ -0,0 +1,184 @@
+#!/usr/bin/env python
+#
+# Copyright (C) 2011-2019 Red Hat, Inc.
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library.  If not, see
+# <http://www.gnu.org/licenses/>.
+#
+# Generate a set of systemtap functions for translating various
+# RPC enum values into strings
+#
+#   python gensystemtap.py */*.x > libvirt_functions.stp
+#
+
+from __future__ import print_function
+
+import re
+import sys
+
+funcs = {}
+
+types = {}
+status = {}
+auth = {}
+
+
+def load_file(fh):
+    instatus = False
+    intype = False
+    inauth = False
+
+    for line in fh:
+        if re.search(r'''enum\s+virNetMessageType''', line):
+            intype = True
+        elif re.search(r'''enum\s+virNetMessageStatus''', line):
+            instatus = True
+        elif re.search(r'''enum remote_auth_type''', line):
+            inauth = True
+        elif line.find("}") != -1:
+            intype = False
+            instatus = False
+            inauth = False
+        elif instatus:
+            m = re.search(r'''^\s+VIR_NET_(\w+)\s*=\s*(\d+),?$''', line)
+            if m is not None:
+                status[m.group(2)] = m.group(1).lower()
+        elif intype:
+            m = re.search(r'''^\s+VIR_NET_(\w+)\s*=\s*(\d+),?$''', line)
+            if m is not None:
+                types[m.group(2)] = m.group(1).lower()
+        elif inauth:
+            m = re.search(r'''^\s+REMOTE_AUTH_(\w+)\s*=\s*(\d+),?$''', line)
+            if m is not None:
+                auth[m.group(2)] = m.group(1).lower()
+        else:
+            m = re.search(r'''(?:VIR_)?(\w+?)(?:_PROTOCOL)?''' +
+                          r'''_PROGRAM\s*=\s*0x([a-fA-F0-9]+)\s*;''', line)
+            if m is not None:
+                funcs[m.group(1).lower()] = {
+                    "id": int(m.group(2), 16),
+                    "version": None,
+                    "progs": []
+                }
+                continue
+
+            m = re.search(r'''(?:VIR_)?(\w+?)(?:_PROTOCOL)?_''' +
+                          r'''(?:PROGRAM|PROTOCOL)_VERSION\s*=\s*(\d+)\s*;''',
+                          line)
+            if m is not None:
+                funcs[m.group(1).lower()]["version"] = m.group(2)
+                continue
+
+            m = re.search(r'''(?:VIR_)?(\w+?)(?:_PROTOCOL)?''' +
+                          r'''_PROC_(.*?)\s+=\s+(\d+)''', line)
+            if m is not None:
+                funcs[m.group(1).lower()]["progs"].insert(
+                    int(m.group(3)), m.group(2).lower())
+
+
+for file in sys.argv[1:]:
+    with open(file, "r") as fh:
+        load_file(fh)
+
+
+def genfunc(name, varname, types):
+    print("function %s(%s, verbose)" % (name, varname))
+    print("{")
+
+    first = True
+    for typename in sorted(types.keys()):
+        cond = "} else if"
+        if first:
+            cond = "if"
+            first = False
+
+        print("  %s (%s == %s) {" % (cond, varname, typename))
+        print("      %sstr = \"%s\"" % (varname, types[typename]))
+
+    print("  } else {")
+    print("      %sstr = \"unknown\";" % varname)
+    print("      verbose = 1;")
+    print("  }")
+    print("  if (verbose) {")
+    print("      %sstr = %sstr . sprintf(\":%%d\", %s)" %
+          (varname, varname, varname))
+    print("  }")
+    print("  return %sstr;" % varname)
+    print("}")
+
+
+genfunc("libvirt_rpc_auth_name", "type", auth)
+genfunc("libvirt_rpc_type_name", "type", types)
+genfunc("libvirt_rpc_status_name", "status", status)
+
+print("function libvirt_rpc_program_name(program, verbose)")
+print("{")
+
+first = True
+for funcname in sorted(funcs.keys()):
+    cond = "} else if"
+    if first:
+        cond = "if"
+        first = False
+
+    print("  %s (program == %s) {" % (cond, funcs[funcname]["id"]))
+    print("      programstr = \"%s\"" % funcname)
+
+print("  } else {")
+print("      programstr = \"unknown\";")
+print("      verbose = 1;")
+print("  }")
+print("  if (verbose) {")
+print("      programstr = programstr . sprintf(\":%d\", program)")
+print("  }")
+print("  return programstr;")
+print("}")
+
+print("function libvirt_rpc_procedure_name(program, version, proc, verbose)")
+print("{")
+
+first = True
+for prog in sorted(funcs.keys()):
+    cond = "} else if"
+    if first:
+        cond = "if"
+        first = False
+
+    print("  %s (program == %s && version == %s) {" %
+          (cond, funcs[prog]["id"], funcs[prog]["version"]))
+
+    pfirst = True
+    for id in range(len(funcs[prog]["progs"])):
+        pcond = "} else if"
+        if pfirst:
+            pcond = "if"
+            pfirst = False
+
+        print("      %s (proc == %s) {" % (pcond, id + 1))
+        print("          procstr = \"%s\";" % funcs[prog]["progs"][id])
+
+    print("      } else {")
+    print("          procstr = \"unknown\";")
+    print("          verbose = 1;")
+    print("      }")
+
+print("  } else {")
+print("      procstr = \"unknown\";")
+print("      verbose = 1;")
+print("  }")
+print("  if (verbose) {")
+print("      procstr = procstr . sprintf(\":%d\", proc)")
+print("  }")
+print("  return procstr;")
+print("}")
diff --git a/src/Makefile.am b/src/Makefile.am
index 085c193013..b0ca9d3442 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -561,8 +561,9 @@ RPC_PROBE_FILES += $(srcdir)/rpc/virnetprotocol.x \
 		  $(srcdir)/remote/qemu_protocol.x \
 		  $(srcdir)/admin/admin_protocol.x
 
-libvirt_functions.stp: $(RPC_PROBE_FILES) $(srcdir)/rpc/gensystemtap.pl
-	$(AM_V_GEN)$(PERL) -w $(srcdir)/rpc/gensystemtap.pl $(RPC_PROBE_FILES) > $@
+libvirt_functions.stp: $(RPC_PROBE_FILES) $(top_srcdir)/scripts/gensystemtap.py
+	$(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(top_srcdir)/scripts/gensystemtap.py \
+	  $(RPC_PROBE_FILES) > $@
 
 %_probes.stp: %_probes.d $(top_srcdir)/scripts/dtrace2systemtap.py \
 		$(top_builddir)/config.status
diff --git a/src/rpc/Makefile.inc.am b/src/rpc/Makefile.inc.am
index a5bde92c9f..11920b3270 100644
--- a/src/rpc/Makefile.inc.am
+++ b/src/rpc/Makefile.inc.am
@@ -3,7 +3,6 @@
 EXTRA_DIST += \
 	rpc/gendispatch.pl \
 	rpc/genprotocol.pl \
-	rpc/gensystemtap.pl \
 	rpc/virnetprotocol.x \
 	rpc/virkeepaliveprotocol.x \
 	$(NULL)
diff --git a/src/rpc/gensystemtap.pl b/src/rpc/gensystemtap.pl
deleted file mode 100755
index 6693d4d6f5..0000000000
--- a/src/rpc/gensystemtap.pl
+++ /dev/null
@@ -1,193 +0,0 @@
-#!/usr/bin/env perl
-#
-# Copyright (C) 2011-2012 Red Hat, Inc.
-#
-# This library is free software; you can redistribute it and/or
-# modify it under the terms of the GNU Lesser General Public
-# License as published by the Free Software Foundation; either
-# version 2.1 of the License, or (at your option) any later version.
-#
-# This library is distributed in the hope that it will be useful,
-# but WITHOUT ANY WARRANTY; without even the implied warranty of
-# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
-# Lesser General Public License for more details.
-#
-# You should have received a copy of the GNU Lesser General Public
-# License along with this library.  If not, see
-# <http://www.gnu.org/licenses/>.
-#
-# Generate a set of systemtap functions for translating various
-# RPC enum values into strings
-#
-#   perl gensystemtap.pl */*.x > libvirt_functions.stp
-#
-
-use strict;
-
-my %funcs;
-
-my %type;
-my %status;
-my %auth;
-
-my $instatus = 0;
-my $intype = 0;
-my $inauth = 0;
-while (<>) {
-    if (/enum\s+virNetMessageType/) {
-        $intype = 1;
-    } elsif (/enum\s+virNetMessageStatus/) {
-        $instatus = 1;
-    } elsif (/enum remote_auth_type/) {
-        $inauth = 1;
-    } elsif (/}/) {
-        $instatus = $intype = $inauth = 0;
-    } elsif ($instatus) {
-        if (/^\s+VIR_NET_(\w+)\s*=\s*(\d+),?$/) {
-            $status{$2} = lc $1;
-        }
-    } elsif ($intype) {
-        if (/^\s+VIR_NET_(\w+)\s*=\s*(\d+),?$/) {
-            $type{$2} = lc $1;
-        }
-    } elsif ($inauth) {
-        if (/^\s+REMOTE_AUTH_(\w+)\s*=\s*(\d+),?$/) {
-            $auth{$2} = lc $1;
-        }
-    } else {
-        if (/(?:VIR_)?(\w+?)(?:_PROTOCOL)?_PROGRAM\s*=\s*0x([a-fA-F0-9]+)\s*;/) {
-            $funcs{lc $1} = { id => hex($2), version => undef, progs => [] };
-        } elsif (/(?:VIR_)?(\w+?)(?:_PROTOCOL)?_(?:PROGRAM|PROTOCOL)_VERSION\s*=\s*(\d+)\s*;/) {
-            $funcs{lc $1}->{version} = $2;
-        } elsif (/(?:VIR_)?(\w+?)(?:_PROTOCOL)?_PROC_(.*?)\s+=\s+(\d+)/) {
-            $funcs{lc $1}->{progs}->[$3] = lc $2;
-        }
-    }
-}
-
-print <<EOF;
-function libvirt_rpc_auth_name(type, verbose)
-{
-EOF
-my $first = 1;
-foreach my $type (sort(keys %auth)) {
-    my $cond = $first ? "if" : "} else if";
-    $first = 0;
-    print "  $cond (type == ", $type, ") {\n";
-    print "      typestr = \"", $auth{$type}, "\"\n";
-}
-print <<EOF;
-  } else {
-      typestr = "unknown";
-      verbose = 1;
-  }
-  if (verbose) {
-      typestr = typestr . sprintf(":%d", type)
-  }
-  return typestr;
-}
-EOF
-
-print <<EOF;
-function libvirt_rpc_type_name(type, verbose)
-{
-EOF
-$first = 1;
-foreach my $type (sort(keys %type)) {
-    my $cond = $first ? "if" : "} else if";
-    $first = 0;
-    print "  $cond (type == ", $type, ") {\n";
-    print "      typestr = \"", $type{$type}, "\"\n";
-}
-print <<EOF;
-  } else {
-      typestr = "unknown";
-      verbose = 1;
-  }
-  if (verbose) {
-      typestr = typestr . sprintf(":%d", type)
-  }
-  return typestr;
-}
-EOF
-
-print <<EOF;
-function libvirt_rpc_status_name(status, verbose)
-{
-EOF
-$first = 1;
-foreach my $status (sort(keys %status)) {
-    my $cond = $first ? "if" : "} else if";
-    $first = 0;
-    print "  $cond (status == ", $status, ") {\n";
-    print "      statusstr = \"", $status{$status}, "\"\n";
-}
-print <<EOF;
-  } else {
-      statusstr = "unknown";
-      verbose = 1;
-  }
-  if (verbose) {
-      statusstr = statusstr . sprintf(":%d", status)
-  }
-  return statusstr;
-}
-EOF
-
-print <<EOF;
-function libvirt_rpc_program_name(program, verbose)
-{
-EOF
-$first = 1;
-foreach my $prog (sort(keys %funcs)) {
-    my $cond = $first ? "if" : "} else if";
-    $first = 0;
-    print "  $cond (program == ", $funcs{$prog}->{id}, ") {\n";
-    print "      programstr = \"", $prog, "\"\n";
-}
-print <<EOF;
-  } else {
-      programstr = "unknown";
-      verbose = 1;
-  }
-  if (verbose) {
-      programstr = programstr . sprintf(":%d", program)
-  }
-  return programstr;
-}
-EOF
-
-
-print <<EOF;
-function libvirt_rpc_procedure_name(program, version, proc, verbose)
-{
-EOF
-$first = 1;
-foreach my $prog (sort(keys %funcs)) {
-    my $cond = $first ? "if" : "} else if";
-    $first = 0;
-    print "  $cond (program == ", $funcs{$prog}->{id}, " && version == ", $funcs{$prog}->{version}, ") {\n";
-
-    my $pfirst = 1;
-    for (my $id = 1 ; $id <= $#{$funcs{$prog}->{progs}} ; $id++) {
-        my $cond = $pfirst ? "if" : "} else if";
-        $pfirst = 0;
-        print "      $cond (proc == $id) {\n";
-        print "          procstr = \"", $funcs{$prog}->{progs}->[$id], "\";\n";
-    }
-    print "      } else {\n";
-    print "          procstr = \"unknown\";\n";
-    print "          verbose = 1;\n";
-    print "      }\n";
-}
-print <<EOF;
-  } else {
-      procstr = "unknown";
-      verbose = 1;
-  }
-  if (verbose) {
-      procstr = procstr . sprintf(":%d", proc)
-  }
-  return procstr;
-}
-EOF
-- 
2.21.0

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v5 09/23] src: rewrite systemtap function generator in Python
Posted by Ján Tomko 5 years ago
On Mon, Nov 11, 2019 at 02:38:12PM +0000, Daniel P. Berrangé wrote:
>As part of an goal to eliminate Perl from libvirt build tools,
>rewrite the gensystemtap.pl tool in Python.
>
>This was a straight conversion, manually going line-by-line to
>change the syntax from Perl to Python. Thus the overall structure
>of the file and approach is the same.
>
>Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
>---
> Makefile.am               |   1 +
> build-aux/syntax-check.mk |   4 +-
> scripts/gensystemtap.py   | 184 ++++++++++++++++++++++++++++++++++++
> src/Makefile.am           |   5 +-
> src/rpc/Makefile.inc.am   |   1 -
> src/rpc/gensystemtap.pl   | 193 --------------------------------------
> 6 files changed, 189 insertions(+), 199 deletions(-)
> create mode 100755 scripts/gensystemtap.py
> delete mode 100755 src/rpc/gensystemtap.pl
>

Reviewed-by: Ján Tomko <jtomko@redhat.com>

Jano
--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list
Re: [libvirt] [PATCH v5 09/23] src: rewrite systemtap function generator in Python
Posted by Cole Robinson 5 years ago
On 11/11/19 9:38 AM, Daniel P. Berrangé wrote:
> As part of an goal to eliminate Perl from libvirt build tools,
> rewrite the gensystemtap.pl tool in Python.
> 
> This was a straight conversion, manually going line-by-line to
> change the syntax from Perl to Python. Thus the overall structure
> of the file and approach is the same.
> 
> Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

libvirt_functions.stp looks the same before and after

Tested-by: Cole Robinson <crobinso@redhat.com>

- Cole

--
libvir-list mailing list
libvir-list@redhat.com
https://www.redhat.com/mailman/listinfo/libvir-list