From nobody Sat Apr 27 03:20:53 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1567679273; cv=none; d=zoho.com; s=zohoarc; b=RULu+SzvQyJhMrgZCmavhj/Zrqkrg2KtddpYMdMT1XnDewFAJDQeEI37ebGaiT4VWMNIsp7th+8vZUiPywMK9C6UdGRVrOgaehoJTBTEhRRwpFTQTm0oyBkLk5v19OA5vyYyWYjRUm5Aqm2tvfnQUDV2iQf9Wc6qIJfTEmhPB4o= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1567679273; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=5LV7G7PqzbGDtIvPojh0lrcj1z7HifJwECQDbkdkz1c=; b=e8Ggm1AhJ+885u7aadIYjGEtgKOj/Rug7wliPm9kUHgidzbancCIOgKBFO8fVOKKM1l55EXHxDz/j5p/5cj84Qm84fAs6gnNOo31GWhRp5bqlDImsYzY+x+AeL3wmAHjnrdx/JI+M3CtQNB11j7vVSsDmXhM5IKwdTkJdlDQw1k= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1567679273208360.3545436090726; Thu, 5 Sep 2019 03:27:53 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id D6F808831C; Thu, 5 Sep 2019 10:27:51 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B4D3F3DA6; Thu, 5 Sep 2019 10:27:51 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 713681806B09; Thu, 5 Sep 2019 10:27:51 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x85AQajD027972 for ; Thu, 5 Sep 2019 06:26:36 -0400 Received: by smtp.corp.redhat.com (Postfix) id 291AD5D9E2; Thu, 5 Sep 2019 10:26:36 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8C8D55D9E5; Thu, 5 Sep 2019 10:26:33 +0000 (UTC) From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Thu, 5 Sep 2019 11:21:56 +0100 Message-Id: <20190905102206.18753-2-berrange@redhat.com> In-Reply-To: <20190905102206.18753-1-berrange@redhat.com> References: <20190905102206.18753-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH 01/11] cfg.mk: fix comment detection for python semicolon check X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Thu, 05 Sep 2019 10:27:52 +0000 (UTC) The pattern ^[^#].*\;$$ Was attempting to detect any trailing ';' in python code which was not in a comment. This does not allow for the comment '#' character to be indented with whitespace. Signed-off-by: Daniel P. Berrang=C3=A9 --- cfg.mk | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cfg.mk b/cfg.mk index 1f29729949..d10dfa16a1 100644 --- a/cfg.mk +++ b/cfg.mk @@ -815,7 +815,7 @@ sc_require_enum_last_marker: =20 # In Python files we don't want to end lines with a semicolon like in C sc_prohibit_semicolon_at_eol_in_python: - @prohibit=3D'^[^#].*\;$$' \ + @prohibit=3D'^[^#]*\;$$' \ in_vc_files=3D'\.py$$' \ halt=3D'python does not require to end lines with a semicolon' \ $(_sc_search_regexp) --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sat Apr 27 03:20:53 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1567679252; cv=none; d=zoho.com; s=zohoarc; b=oDvpB0ekBk9X1jddjHPonwuDo9rNRi8BR7KlYfoIqRZWf4W2GJt8uGCJ/+4dcpKKmaKcr6eygnxvExAxgg2e76MvkvYdmpFIxxDcYbioO+WN69cWMbheBunwbXYTqe4whZPR1aIOOj/SbZpC4bJ/hSjIPg6Nv6htuAvVFwnTTv0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1567679252; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=ipXYrrqlLb5GWxSCM2VoYiXo0bcCg0+BtE+M94eSIFo=; b=NClrxz+k6hG/IueUGHcchHmYsE+d9HB5gXSBxz7ASUXpSuBEOkaRiTnF/hbUJUiJdcjfEZb83lkhExvemwbKEeQp5XpYqPC29P673Uur6sovLMd1lzMYPAZyqeFVh5d4zpEOoJiRhdLHDdJc4fEoL2K1er7+urzgPVQUT4IdCCo= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1567679252596243.64000638718016; Thu, 5 Sep 2019 03:27:32 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 8433130821EC; Thu, 5 Sep 2019 10:27:30 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id D18D55D9E1; Thu, 5 Sep 2019 10:27:28 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 84C0624FDE; Thu, 5 Sep 2019 10:27:25 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x85AQbU9027987 for ; Thu, 5 Sep 2019 06:26:37 -0400 Received: by smtp.corp.redhat.com (Postfix) id 5316B5D9E5; Thu, 5 Sep 2019 10:26:37 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 706185D9E2; Thu, 5 Sep 2019 10:26:36 +0000 (UTC) From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Thu, 5 Sep 2019 11:21:57 +0100 Message-Id: <20190905102206.18753-3-berrange@redhat.com> In-Reply-To: <20190905102206.18753-1-berrange@redhat.com> References: <20190905102206.18753-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH 02/11] docs: rewrite hvsupport.html page generator in python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.47]); Thu, 05 Sep 2019 10:27:31 +0000 (UTC) As part of an goal to eliminate Perl from libvirt build tools, rewrite the hvsupport.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. The new impl generates byte-for-byte identical output to the old impl. Signed-off-by: Daniel P. Berrang=C3=A9 --- docs/Makefile.am | 6 +- docs/hvsupport.pl | 458 -------------------------------------------- docs/hvsupport.py | 479 ++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 482 insertions(+), 461 deletions(-) delete mode 100755 docs/hvsupport.pl create mode 100755 docs/hvsupport.py diff --git a/docs/Makefile.am b/docs/Makefile.am index 1cdb584b0b..f7aba5499d 100644 --- a/docs/Makefile.am +++ b/docs/Makefile.am @@ -194,7 +194,7 @@ EXTRA_DIST=3D \ $(internals_html_in) $(internals_html) $(fonts) \ $(kbase_html_in) $(kbase_html) \ aclperms.htmlinc \ - hvsupport.pl \ + hvsupport.py \ $(schema_DATA) =20 acl_generated =3D aclperms.htmlinc @@ -230,11 +230,11 @@ web: $(dot_html) $(internals_html) $(kbase_html) \ =20 hvsupport.html: $(srcdir)/hvsupport.html.in =20 -$(srcdir)/hvsupport.html.in: $(srcdir)/hvsupport.pl $(api_DATA) \ +$(srcdir)/hvsupport.html.in: $(srcdir)/hvsupport.py $(api_DATA) \ $(top_srcdir)/src/libvirt_public.syms \ $(top_srcdir)/src/libvirt_qemu.syms $(top_srcdir)/src/libvirt_lxc.syms \ $(top_srcdir)/src/driver.h - $(AM_V_GEN)$(PERL) $(srcdir)/hvsupport.pl $(top_srcdir)/src > $@ \ + $(AM_V_GEN)$(PYTHON) $(srcdir)/hvsupport.py $(top_srcdir)/src > $@ \ || { rm $@ && exit 1; } =20 news.html.in: \ diff --git a/docs/hvsupport.pl b/docs/hvsupport.pl deleted file mode 100755 index 494b8a27ec..0000000000 --- a/docs/hvsupport.pl +++ /dev/null @@ -1,458 +0,0 @@ -#!/usr/bin/env perl - -use strict; -use warnings; - -use File::Find; - -die "syntax: $0 SRCDIR\n" unless int(@ARGV) =3D=3D 1; - -my $srcdir =3D shift @ARGV; - -my $symslibvirt =3D "$srcdir/libvirt_public.syms"; -my $symsqemu =3D "$srcdir/libvirt_qemu.syms"; -my $symslxc =3D "$srcdir/libvirt_lxc.syms"; -my @drivertable =3D ( - "$srcdir/driver-hypervisor.h", - "$srcdir/driver-interface.h", - "$srcdir/driver-network.h", - "$srcdir/driver-nodedev.h", - "$srcdir/driver-nwfilter.h", - "$srcdir/driver-secret.h", - "$srcdir/driver-state.h", - "$srcdir/driver-storage.h", - "$srcdir/driver-stream.h", - ); - -my %groupheaders =3D ( - "virHypervisorDriver" =3D> "Hypervisor APIs", - "virNetworkDriver" =3D> "Virtual Network APIs", - "virInterfaceDriver" =3D> "Host Interface APIs", - "virNodeDeviceDriver" =3D> "Host Device APIs", - "virStorageDriver" =3D> "Storage Pool APIs", - "virSecretDriver" =3D> "Secret APIs", - "virNWFilterDriver" =3D> "Network Filter APIs", - ); - - -my @srcs; -find({ - wanted =3D> sub { - if (m!$srcdir/.*/\w+_(driver|common|tmpl|monitor|hal|udev)\.c$!) { - push @srcs, $_ if $_ !~ /vbox_driver\.c/; - } - }, no_chdir =3D> 1}, $srcdir); - -# Map API=C2=A0functions to the header and documentation files they're in -# so that we can generate proper hyperlinks to their documentation. -# -# The function names are grep'd from the XML output of apibuild.py. -sub getAPIFilenames { - my $filename =3D shift; - - my %files; - my $line; - - open FILE, "<", $filename or die "cannot read $filename: $!"; - - while (defined($line =3D )) { - if ($line =3D~ /function name=3D'([^']+)' file=3D'([^']+)'/) { - $files{$1} =3D $2; - } - } - - close FILE; - - if (keys %files =3D=3D 0) { - die "No functions found in $filename. Has the apibuild.py output c= hanged?"; - } - return \%files; -} - -sub parseSymsFile { - my $apisref =3D shift; - my $prefix =3D shift; - my $filename =3D shift; - my $xmlfilename =3D shift; - - my $line; - my $vers; - my $prevvers; - - my $filenames =3D getAPIFilenames($xmlfilename); - - open FILE, "<$filename" - or die "cannot read $filename: $!"; - - while (defined($line =3D )) { - chomp $line; - next if $line =3D~ /^\s*#/; - next if $line =3D~ /^\s*$/; - next if $line =3D~ /^\s*(global|local):/; - if ($line =3D~ /^\s*${prefix}_(\d+\.\d+\.\d+)\s*{\s*$/) { - if (defined $vers) { - die "malformed syms file"; - } - $vers =3D $1; - } elsif ($line =3D~ /\s*}\s*;\s*$/) { - if (defined $prevvers) { - die "malformed syms file"; - } - $prevvers =3D $vers; - $vers =3D undef; - } elsif ($line =3D~ /\s*}\s*${prefix}_(\d+\.\d+\.\d+)\s*;\s*$/) { - if ($1 ne $prevvers) { - die "malformed syms file $1 !=3D $vers"; - } - $prevvers =3D $vers; - $vers =3D undef; - } elsif ($line =3D~ /\s*(\w+)\s*;\s*$/) { - $$apisref{$1} =3D {}; - $$apisref{$1}->{vers} =3D $vers; - $$apisref{$1}->{file} =3D $$filenames{$1}; - } else { - die "unexpected data $line\n"; - } - } - - close FILE; -} - -my %apis; -# Get the list of all public APIs and their corresponding version -parseSymsFile(\%apis, "LIBVIRT", $symslibvirt, "$srcdir/../docs/libvirt-ap= i.xml"); - -# And the same for the QEMU specific APIs -parseSymsFile(\%apis, "LIBVIRT_QEMU", $symsqemu, "$srcdir/../docs/libvirt-= qemu-api.xml"); - -# And the same for the LXC specific APIs -parseSymsFile(\%apis, "LIBVIRT_LXC", $symslxc, "$srcdir/../docs/libvirt-lx= c-api.xml"); - - -# Some special things which aren't public APIs, -# but we want to report -$apis{virConnectSupportsFeature}->{vers} =3D "0.3.2"; -$apis{virDomainMigratePrepare}->{vers} =3D "0.3.2"; -$apis{virDomainMigratePerform}->{vers} =3D "0.3.2"; -$apis{virDomainMigrateFinish}->{vers} =3D "0.3.2"; -$apis{virDomainMigratePrepare2}->{vers} =3D "0.5.0"; -$apis{virDomainMigrateFinish2}->{vers} =3D "0.5.0"; -$apis{virDomainMigratePrepareTunnel}->{vers} =3D "0.7.2"; - -$apis{virDomainMigrateBegin3}->{vers} =3D "0.9.2"; -$apis{virDomainMigratePrepare3}->{vers} =3D "0.9.2"; -$apis{virDomainMigratePrepareTunnel3}->{vers} =3D "0.9.2"; -$apis{virDomainMigratePerform3}->{vers} =3D "0.9.2"; -$apis{virDomainMigrateFinish3}->{vers} =3D "0.9.2"; -$apis{virDomainMigrateConfirm3}->{vers} =3D "0.9.2"; - -$apis{virDomainMigrateBegin3Params}->{vers} =3D "1.1.0"; -$apis{virDomainMigratePrepare3Params}->{vers} =3D "1.1.0"; -$apis{virDomainMigratePrepareTunnel3Params}->{vers} =3D "1.1.0"; -$apis{virDomainMigratePerform3Params}->{vers} =3D "1.1.0"; -$apis{virDomainMigrateFinish3Params}->{vers} =3D "1.1.0"; -$apis{virDomainMigrateConfirm3Params}->{vers} =3D "1.1.0"; - - - -# Now we want to get the mapping between public APIs -# and driver struct fields. This lets us later match -# update the driver impls with the public APis. - -my $line; - -# Group name -> hash of APIs { fields -> api name } -my %groups; -my $ingrp; -foreach my $drivertable (@drivertable) { - open FILE, "<$drivertable" - or die "cannot read $drivertable: $!"; - - while (defined($line =3D )) { - if ($line =3D~ /struct _(vir\w*Driver)/) { - my $grp =3D $1; - if ($grp ne "virStateDriver" && - $grp ne "virStreamDriver") { - $ingrp =3D $grp; - $groups{$ingrp} =3D { apis =3D> {}, drivers =3D> {} }; - } - } elsif ($ingrp) { - if ($line =3D~ /^\s*vir(?:Drv)(\w+)\s+(\w+);\s*$/) { - my $field =3D $2; - my $name =3D $1; - - my $api; - if (exists $apis{"vir$name"}) { - $api =3D "vir$name"; - } elsif ($name =3D~ /\w+(Open|Close|URIProbe)/) { - next; - } else { - die "driver $name does not have a public API"; - } - $groups{$ingrp}->{apis}->{$field} =3D $api; - } elsif ($line =3D~ /};/) { - $ingrp =3D undef; - } - } - } - - close FILE; -} - - -# Finally, we read all the primary driver files and extract -# the driver API tables from each one. - -foreach my $src (@srcs) { - open FILE, "<$src" or - die "cannot read $src: $!"; - - my $groups_regex =3D join("|", keys %groups); - $ingrp =3D undef; - my $impl; - while (defined($line =3D )) { - if (!$ingrp) { - #=C2=A0skip non-matching lines early to save time - next if not $line =3D~ /$groups_regex/; - - if ($line =3D~ /^\s*(?:static\s+)?($groups_regex)\s+(\w+)\s*= =3D\s*{/ || - $line =3D~ /^\s*(?:static\s+)?($groups_regex)\s+NAME\(\w+\= )\s*=3D\s*{/) { - $ingrp =3D $1; - $impl =3D $src; - - if ($impl =3D~ m,.*/node_device_(\w+)\.c,) { - $impl =3D $1; - } else { - $impl =3D~ s,.*/(\w+?)_((\w+)_)?(\w+)\.c,$1,; - } - - if ($groups{$ingrp}->{drivers}->{$impl}) { - die "Group $ingrp already contains $impl"; - } - - $groups{$ingrp}->{drivers}->{$impl} =3D {}; - } - - } else { - if ($line =3D~ m!\s*\.(\w+)\s*=3D\s*(\w+)\s*,?\s*(?:/\*\s*(\d+= \.\d+\.\d+)\s*(?:-\s*(\d+\.\d+\.\d+))?\s*\*/\s*)?$!) { - my $api =3D $1; - my $meth =3D $2; - my $vers =3D $3; - my $deleted =3D $4; - - next if $api eq "no" || $api eq "name"; - - if ($meth eq "NULL" && !defined $deleted) { - die "Method impl for $api is NULL, but no deleted vers= ion is provided"; - } - if ($meth ne "NULL" && defined $deleted) { - die "Method impl for $api is non-NULL, but deleted ver= sion is provided"; - } - - die "Method $meth in $src is missing version" unless defin= ed $vers || $api eq "connectURIProbe"; - - if (!exists($groups{$ingrp}->{apis}->{$api})) { - next if $api =3D~ /\w(Open|Close|URIProbe)/; - - die "Found unexpected method $api in $ingrp\n"; - } - - $groups{$ingrp}->{drivers}->{$impl}->{$api} =3D {}; - $groups{$ingrp}->{drivers}->{$impl}->{$api}->{vers} =3D $v= ers; - $groups{$ingrp}->{drivers}->{$impl}->{$api}->{deleted} = =3D $deleted; - if ($api eq "domainMigratePrepare" || - $api eq "domainMigratePrepare2" || - $api eq "domainMigratePrepare3") { - if (!$groups{$ingrp}->{drivers}->{$impl}->{"domainMigr= ate"}) { - $groups{$ingrp}->{drivers}->{$impl}->{"domainMigra= te"} =3D {}; - $groups{$ingrp}->{drivers}->{$impl}->{"domainMigra= te"}->{vers} =3D $vers; - } - } - - } elsif ($line =3D~ /}/) { - $ingrp =3D undef; - } - } - } - - close FILE; -} - - -# The '.open' driver method is used for 3 public APIs, so we -# have a bit of manual fixup todo with the per-driver versioning -# and support matrix - -$groups{virHypervisorDriver}->{apis}->{"openAuth"} =3D "virConnectOpenAuth= "; -$groups{virHypervisorDriver}->{apis}->{"openReadOnly"} =3D "virConnectOpen= ReadOnly"; -$groups{virHypervisorDriver}->{apis}->{"domainMigrate"} =3D "virDomainMigr= ate"; - -my $openAuthVers =3D (0 * 1000 * 1000) + (4 * 1000) + 0; - -foreach my $drv (keys %{$groups{"virHypervisorDriver"}->{drivers}}) { - my $openVersStr =3D $groups{"virHypervisorDriver"}->{drivers}->{$drv}-= >{"connectOpen"}->{vers}; - my $openVers; - if ($openVersStr =3D~ /(\d+)\.(\d+)\.(\d+)/) { - $openVers =3D ($1 * 1000 * 1000) + ($2 * 1000) + $3; - } - - # virConnectOpenReadOnly always matches virConnectOpen version - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"connectOpenReadOn= ly"} =3D - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"connectOpen"}; - - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"connectOpenAuth"}= =3D {}; - - # virConnectOpenAuth is always 0.4.0 if the driver existed - # before this time, otherwise it matches the version of - # the driver's virConnectOpen entry - if ($openVersStr eq "Y" || - $openVers >=3D $openAuthVers) { - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"connectOpenAu= th"}->{vers} =3D $openVersStr; - } else { - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"connectOpenAu= th"}->{vers} =3D "0.4.0"; - } -} - - -# Another special case for the virDomainCreateLinux which was replaced -# with virDomainCreateXML -$groups{virHypervisorDriver}->{apis}->{"domainCreateLinux"} =3D "virDomain= CreateLinux"; - -my $createAPIVers =3D (0 * 1000 * 1000) + (0 * 1000) + 3; - -foreach my $drv (keys %{$groups{"virHypervisorDriver"}->{drivers}}) { - my $createVersStr =3D $groups{"virHypervisorDriver"}->{drivers}->{$drv= }->{"domainCreateXML"}->{vers}; - next unless defined $createVersStr; - my $createVers; - if ($createVersStr =3D~ /(\d+)\.(\d+)\.(\d+)/) { - $createVers =3D ($1 * 1000 * 1000) + ($2 * 1000) + $3; - } - - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"domainCreateLinux= "} =3D {}; - - # virCreateLinux is always 0.0.3 if the driver existed - # before this time, otherwise it matches the version of - # the driver's virCreateXML entry - if ($createVersStr eq "Y" || - $createVers >=3D $createAPIVers) { - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"domainCreateL= inux"}->{vers} =3D $createVersStr; - } else { - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"domainCreateL= inux"}->{vers} =3D "0.0.3"; - } -} - - -# Finally we generate the HTML file with the tables - -print < - - - -

libvirt API support matrix

- -
    - -

    -This page documents which libvirt calls work on -which libvirt drivers / hypervisors, and which version the API appeared -in. If a hypervisor driver later dropped support for the API, the version -when it was removed is also mentioned (highlighted in -dark red). -

    - -EOF - - foreach my $grp (sort { $a cmp $b } keys %groups) { - print "

    ", $groupheaders{$grp}, "

    \n"; - print < - - -API -Version -EOF - - foreach my $drv (sort { $a cmp $b } keys %{$groups{$grp}->{drivers}}) { - print " $drv\n"; - } - - print < - - -EOF - - my $row =3D 0; - foreach my $field (sort { - $groups{$grp}->{apis}->{$a} - cmp - $groups{$grp}->{apis}->{$b} - } keys %{$groups{$grp}->{apis}}) { - my $api =3D $groups{$grp}->{apis}->{$field}; - my $vers =3D $apis{$api}->{vers}; - my $htmlgrp =3D $apis{$api}->{file}; - print < - -EOF - - if (defined $htmlgrp) { - print <$api -EOF - - } else { - print $api; - } - print < -$vers -EOF - - foreach my $drv (sort {$a cmp $b } keys %{$groups{$grp}->{drivers}= }) { - print ""; - if (exists $groups{$grp}->{drivers}->{$drv}->{$field}) { - if ($groups{$grp}->{drivers}->{$drv}->{$field}->{vers}) { - print $groups{$grp}->{drivers}->{$drv}->{$field}->{ver= s}; - } - if ($groups{$grp}->{drivers}->{$drv}->{$field}->{deleted})= { - print " - ", $groups{$grp}= ->{drivers}->{$drv}->{$field}->{deleted}, ""; - } - } - print "\n"; - } - - print < -EOF - - $row++; - if (($row % 15) =3D=3D 0) { - print < -API -Version -EOF - - foreach my $drv (sort { $a cmp $b } keys %{$groups{$grp}->{dri= vers}}) { - print " $drv\n"; - } - - print < -EOF - } - - } - - print < - -EOF -} - -print < - -EOF diff --git a/docs/hvsupport.py b/docs/hvsupport.py new file mode 100755 index 0000000000..cfbecd82ae --- /dev/null +++ b/docs/hvsupport.py @@ -0,0 +1,479 @@ +#!/usr/bin/env python +# +# Copyright (C) 2011-2019 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . + +from __future__ import print_function + +import sys +import os.path +import re + +if len(sys.argv) !=3D 2: + print("syntax: %s SRCDIR\n" % sys.argv[0], file=3Dsys.stderr) + +srcdir =3D sys.argv[1] + +symslibvirt =3D os.path.join(srcdir, "libvirt_public.syms") +symsqemu =3D os.path.join(srcdir, "libvirt_qemu.syms") +symslxc =3D os.path.join(srcdir, "libvirt_lxc.syms") +drivertablefiles =3D [ + os.path.join(srcdir, "driver-hypervisor.h"), + os.path.join(srcdir, "driver-interface.h"), + os.path.join(srcdir, "driver-network.h"), + os.path.join(srcdir, "driver-nodedev.h"), + os.path.join(srcdir, "driver-nwfilter.h"), + os.path.join(srcdir, "driver-secret.h"), + os.path.join(srcdir, "driver-state.h"), + os.path.join(srcdir, "driver-storage.h"), + os.path.join(srcdir, "driver-stream.h"), +] + +groupheaders =3D { + "virHypervisorDriver": "Hypervisor APIs", + "virNetworkDriver": "Virtual Network APIs", + "virInterfaceDriver": "Host Interface APIs", + "virNodeDeviceDriver": "Host Device APIs", + "virStorageDriver": "Storage Pool APIs", + "virSecretDriver": "Secret APIs", + "virNWFilterDriver": "Network Filter APIs", +} + + +srcs =3D [] +for root, dirs, files in os.walk(srcdir): + for file in files: + if ((file.endswith("driver.c") and + not file.endswith("vbox_driver.c")) or + file.endswith("common.c") or + file.endswith("tmpl.c") or + file.endswith("monitor.c") or + file.endswith("hal.c") or + file.endswith("udev.c")): + srcs.append(os.path.join(root, file)) + + +# Map API functions to the header and documentation files they're in +# so that we can generate proper hyperlinks to their documentation. +# +# The function names are grep'd from the XML output of apibuild.py. +def getAPIFilenames(filename): + files =3D {} + + with open(filename) as fh: + prog =3D re.compile(r"\s* hash of APIs { fields -> api name } +groups =3D {} +ingrp =3D None +for drivertablefile in drivertablefiles: + with open(drivertablefile) as fh: + starttableprog =3D re.compile(r"struct _(vir\w*Driver)") + endtableprog =3D re.compile(r"};") + callbackprog =3D re.compile(r"^\s*vir(?:Drv)(\w+)\s+(\w+);\s*$") + ignoreapiprog =3D re.compile(r"\w+(Open|Close|URIProbe)") + for line in fh: + starttablematch =3D starttableprog.match(line) + if starttablematch is not None: + grp =3D starttablematch.group(1) + if grp !=3D "virStateDriver" and grp !=3D "virStreamDriver= ": + ingrp =3D grp + groups[ingrp] =3D { + "apis": {}, + "drivers": {} + } + elif ingrp !=3D None: + callbackmatch =3D callbackprog.match(line) + if callbackmatch is not None: + name =3D callbackmatch.group(1) + field =3D callbackmatch.group(2) + + api =3D "vir" + name + if api in apis: + groups[ingrp]["apis"][field] =3D api + elif ignoreapiprog.match(api) !=3D None: + continue + else: + raise Exception("driver %s does not have a public = API" % name) + elif endtableprog.match(line): + ingrp =3D None + + +# Finally, we read all the primary driver files and extract +# the driver API tables from each one. + +for src in srcs: + with open(src) as fh: + groupsre =3D "|".join(groups.keys()) + groupsprog2 =3D re.compile(r"^\s*(static\s+)?(" + groupsre + r")\s= +(\w+)\s*=3D\s*{") + groupsprog3 =3D re.compile(r"^\s*(static\s+)?(" + groupsre + r")\s= +NAME\(\w+\)\s*=3D\s*{") + nodedevimplprog =3D re.compile(r".*/node_device_(\w+)\.c") + miscimplprog =3D re.compile(r".*/(\w+?)_((\w+)_)?(\w+)\.c") + callbackprog =3D re.compile(r"\s*\.(\w+)\s*=3D\s*(\w+)\s*,?\s*(?:/= \*\s*(\d+\.\d+\.\d+)\s*(?:-\s*(\d+\.\d+\.\d+))?\s*\*/\s*)?$") + skipapiprog =3D re.compile(r"\w+(Open|Close|URIProbe)") + + ingrp =3D None + impl =3D None + for line in fh: + if ingrp is None: + m =3D groupsprog2.match(line) + if m is None: + m =3D groupsprog3.match(line) + if m is not None: + ingrp =3D m.group(2) + impl =3D src + + implmatch =3D nodedevimplprog.match(impl) + if implmatch is None: + implmatch =3D miscimplprog.match(impl) + if implmatch is None: + raise Exception("Unexpected impl format '%s'" % im= pl) + impl =3D implmatch.group(1) + + if impl in groups[ingrp]["drivers"]: + raise Exception("Group %s already contains %s" % (= ingrp, impl)) + + groups[ingrp]["drivers"][impl] =3D {} + else: + callbackmatch =3D callbackprog.match(line) + if callbackmatch is not None: + api =3D callbackmatch.group(1) + meth =3D callbackmatch.group(2) + vers =3D callbackmatch.group(3) + deleted =3D callbackmatch.group(4) + + if api =3D=3D "no" or api =3D=3D "name": + continue + + if meth =3D=3D "NULL" and deleted is None: + raise Exception("Method impl for %s is NULL, but n= o deleted version is provided" % api) + + if meth !=3D "NULL" and deleted is not None: + raise Exception("Method impl for %s is non-NULL, b= ut deleted version is provided" % api) + + if vers is None and api !=3D "connectURIProbe": + raise Exception("Method %s in %s is missing versio= n" % (meth, src)) + + if api not in groups[ingrp]["apis"]: + if skipapiprog.match(api): + continue + + raise Exception("Found unexpected method %s in %s"= % (api, ingrp)) + + groups[ingrp]["drivers"][impl][api] =3D { + "vers": vers, + "deleted": deleted, + } + + if (api =3D=3D "domainMigratePrepare" or + api =3D=3D "domainMigratePrepare2" or + api =3D=3D "domainMigratePrepare3"): + if "domainMigrate" not in groups[ingrp]["drivers"]= [impl]: + groups[ingrp]["drivers"][impl]["domainMigrate"= ] =3D { + "vers": vers, + } + elif line.find("}") !=3D -1: + ingrp =3D None + + +# The '.open' driver method is used for 3 public APIs, so we +# have a bit of manual fixup todo with the per-driver versioning +# and support matrix + +groups["virHypervisorDriver"]["apis"]["openAuth"] =3D "virConnectOpenAuth" +groups["virHypervisorDriver"]["apis"]["openReadOnly"] =3D "virConnectOpenR= eadOnly" +groups["virHypervisorDriver"]["apis"]["domainMigrate"] =3D "virDomainMigra= te" + +openAuthVers =3D (0 * 1000 * 1000) + (4 * 1000) + 0 + +for drv in groups["virHypervisorDriver"]["drivers"].keys(): + openVersStr =3D groups["virHypervisorDriver"]["drivers"][drv]["connect= Open"]["vers"] + openVers =3D 0 + if openVersStr !=3D "Y": + openVersBits =3D openVersStr.split(".") + if len(openVersBits) !=3D 3: + raise Exception("Expected 3 digit version for %s" % openVersSt= r) + openVers =3D (int(openVersBits[0]) * 1000 * 1000) + (int(openVersB= its[1]) * 1000) + int(openVersBits[2]) + + # virConnectOpenReadOnly always matches virConnectOpen version + groups["virHypervisorDriver"]["drivers"][drv]["connectOpenReadOnly"] = =3D \ + groups["virHypervisorDriver"]["drivers"][drv]["connectOpen"] + + # virConnectOpenAuth is always 0.4.0 if the driver existed + # before this time, otherwise it matches the version of + # the driver's virConnectOpen entry + if openVersStr =3D=3D "Y" or openVers >=3D openAuthVers: + vers =3D openVersStr + else: + vers =3D "0.4.0" + groups["virHypervisorDriver"]["drivers"][drv]["connectOpenAuth"] =3D { + "vers": vers, + } + + +# Another special case for the virDomainCreateLinux which was replaced +# with virDomainCreateXML +groups["virHypervisorDriver"]["apis"]["domainCreateLinux"] =3D "virDomainC= reateLinux" + +createAPIVers =3D (0 * 1000 * 1000) + (0 * 1000) + 3 + +for drv in groups["virHypervisorDriver"]["drivers"].keys(): + if "domainCreateXML" not in groups["virHypervisorDriver"]["drivers"][d= rv]: + continue + createVersStr =3D groups["virHypervisorDriver"]["drivers"][drv]["domai= nCreateXML"]["vers"] + createVers =3D 0 + if createVersStr !=3D "Y": + createVersBits =3D createVersStr.split(".") + if len(createVersBits) !=3D 3: + raise Exception("Expected 3 digit version for %s" % createVers= Str) + createVers =3D (int(createVersBits[0]) * 1000 * 1000) + (int(creat= eVersBits[1]) * 1000) + int(createVersBits[2]) + + # virCreateLinux is always 0.0.3 if the driver existed + # before this time, otherwise it matches the version of + # the driver's virCreateXML entry + if createVersStr =3D=3D "Y" or createVers >=3D createAPIVers: + vers =3D createVersStr + else: + vers =3D "0.0.3" + + groups["virHypervisorDriver"]["drivers"][drv]["domainCreateLinux"] =3D= { + "vers": vers, + } + + +# Finally we generate the HTML file with the tables + +print(''' + + + +

    libvirt API support matrix

    + +
      + +

      +This page documents which libvirt calls work on +which libvirt drivers / hypervisors, and which version the API appeared +in. If a hypervisor driver later dropped support for the API, the version +when it was removed is also mentioned (highlighted in +dark red). +

      +''') + +for grp in sorted(groups.keys()): + print("

      %s

      " % (grp, groupheaders[grp])) + print(''' + + + +''') + + for drv in sorted(groups[grp]["drivers"].keys()): + print(" " % drv) + + + print(''' + +''') + + row =3D 0 + def sortkey(field): + return groups[grp]["apis"][field] + for field in sorted(groups[grp]["apis"].keys(), key=3Dsortkey): + api =3D groups[grp]["apis"][field] + vers =3D apis[api]["vers"] + htmlgrp =3D apis[api].get("file") + print("") + + if htmlgrp is not None: + print('''" % api) + + print("" % vers) + + for drv in sorted(groups[grp]["drivers"].keys()): + info =3D "" + if field in groups[grp]["drivers"][drv]: + vers =3D groups[grp]["drivers"][drv][field]["vers"] + if vers is not None: + info =3D info + vers + + deleted =3D groups[grp]["drivers"][drv][field].get("delete= d") + if deleted is not None: + info =3D info + (''' - %s''' % deleted) + + print("" % info) + + print("") + + row =3D row + 1 + if (row % 15) =3D=3D 0: + print(''' + +''') + + for drv in sorted(groups[grp]["drivers"].keys()): + print(" " % drv) + + print("") + + print("\n
      APIVersion%s
      \n%s\n<= /td>''' % (htmlgrp, api, api)) + else: + print("\n%s%s%s
      APIVersion%s
      ") + +print("\n") --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sat Apr 27 03:20:53 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1567679263; cv=none; d=zoho.com; s=zohoarc; b=Ph/dRekByvuburJtnyg+icMDKxvehm7XFulg+K32z8fbJm+Qx3OP7bDOSfjBxL6nWT47KcnfDSFjSltbyhQ6X1IkYh/Y18AJoblZxLnj566Olo3FysM4Bc3fw76NRam97y4zWgwfbMajaxJ3RdPiAL9lENMurufWbSUf6yYuJKM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1567679263; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=LCczb4NxnPZlEMhIL7OEMhIfmnL2qCKHYDZ/FP4/5/o=; b=Jg5heCXLynM/DfI2UUsKBnMCKCZ8LNCdwPMmbmdQ6fftxl7kxs7VDSxFBxH8Eil5/tc4OxwVNAqNosQXfjvUpUOEL0Cx5ECGvSaVv0kDVrl4WKjm5vqoeDni9/UAZTaThnl1vVqh66OKGLGEa9Wcaf7bNkkkxKxwNzFkYQbqnhQ= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1567679263890880.5867968089394; Thu, 5 Sep 2019 03:27:43 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id ABAAF1DA5; Thu, 5 Sep 2019 10:27:42 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 66BEE5D713; Thu, 5 Sep 2019 10:27:42 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 0B77424F32; Thu, 5 Sep 2019 10:27:42 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x85AQctI027995 for ; Thu, 5 Sep 2019 06:26:38 -0400 Received: by smtp.corp.redhat.com (Postfix) id 377985DA60; Thu, 5 Sep 2019 10:26:38 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 99F105D9E2; Thu, 5 Sep 2019 10:26:37 +0000 (UTC) From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Thu, 5 Sep 2019 11:21:58 +0100 Message-Id: <20190905102206.18753-4-berrange@redhat.com> In-Reply-To: <20190905102206.18753-1-berrange@redhat.com> References: <20190905102206.18753-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH 03/11] docs: rewrite ACL permissions checker in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx1.redhat.com [10.5.110.71]); Thu, 05 Sep 2019 10:27:43 +0000 (UTC) As part of an goal to eliminate Perl from libvirt build tools, rewrite the check-aclperms.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. Signed-off-by: Daniel P. Berrang=C3=A9 --- src/Makefile.am | 4 +-- src/check-aclperms.pl | 73 ---------------------------------------- src/check-aclperms.py | 77 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 79 insertions(+), 75 deletions(-) delete mode 100755 src/check-aclperms.pl create mode 100755 src/check-aclperms.py diff --git a/src/Makefile.am b/src/Makefile.am index f5093b9c90..2eb4387231 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -359,11 +359,11 @@ check-aclrules: $(addprefix $(srcdir)/,$(filter-out /%,$(STATEFUL_DRIVER_SOURCE_FILES))) =20 check-aclperms: - $(AM_V_GEN)$(PERL) $(srcdir)/check-aclperms.pl \ + $(AM_V_GEN)$(PYTHON) $(srcdir)/check-aclperms.py \ $(srcdir)/access/viraccessperm.h \ $(srcdir)/access/viraccessperm.c =20 -EXTRA_DIST +=3D check-driverimpls.pl check-aclrules.pl check-aclperms.pl +EXTRA_DIST +=3D check-driverimpls.pl check-aclrules.pl check-aclperms.py =20 check-local: check-protocol check-symfile check-symsorting \ check-drivername check-driverimpls check-aclrules \ diff --git a/src/check-aclperms.pl b/src/check-aclperms.pl deleted file mode 100755 index 55b6598313..0000000000 --- a/src/check-aclperms.pl +++ /dev/null @@ -1,73 +0,0 @@ -#!/usr/bin/env perl -# -# Copyright (C) 2013 Red Hat, Inc. -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library. If not, see -# . -# -# This script just validates that the stringified version of -# a virAccessPerm enum matches the enum constant name. We do -# a lot of auto-generation of code, so when these don't match -# problems occur, preventing auth from succeeding at all. - -my $hdr =3D shift; -my $impl =3D shift; - -my %perms; - -my @perms; - -open HDR, $hdr or die "cannot read $hdr: $!"; - -while () { - if (/^\s+VIR_ACCESS_PERM_([_A-Z]+)(,?|\s|$)/) { - my $perm =3D $1; - - $perms{$perm} =3D 1 unless ($perm =3D~ /_LAST$/); - } -} - -close HDR; - - -open IMPL, $impl or die "cannot read $impl: $!"; - -my $group; -my $warned =3D 0; - -while (defined (my $line =3D )) { - if ($line =3D~ /VIR_ACCESS_PERM_([_A-Z]+)_LAST/) { - $group =3D $1; - } elsif ($line =3D~ /"[_a-z]+"/) { - my @bits =3D split /,/, $line; - foreach my $bit (@bits) { - if ($bit =3D~ /"([_a-z]+)"/) { - my $perm =3D uc($group . "_" . $1); - if (!exists $perms{$perm}) { - print STDERR "Unknown perm string $1 for group $group\= n"; - $warned =3D 1; - } - delete $perms{$perm}; - } - } - } -} -close IMPL; - -foreach my $perm (keys %perms) { - print STDERR "Perm $perm had not string form\n"; - $warned =3D 1; -} - -exit $warned; diff --git a/src/check-aclperms.py b/src/check-aclperms.py new file mode 100755 index 0000000000..ace6ab6757 --- /dev/null +++ b/src/check-aclperms.py @@ -0,0 +1,77 @@ +#!/usr/bin/env python +# +# Copyright (C) 2013-2019 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . +# +# This script just validates that the stringified version of +# a virAccessPerm enum matches the enum constant name. We do +# a lot of auto-generation of code, so when these don't match +# problems occur, preventing auth from succeeding at all. + +from __future__ import print_function + +import re +import sys + +if len(sys.argv) !=3D 3: + print("syntax: %s HEADER IMPL" % (sys.argv[0]), file=3Dsys.stderr) + sys.exit(1) + +hdr =3D sys.argv[1] +impl =3D sys.argv[2] + +perms =3D {} + +with open(hdr) as fh: + symprog =3D re.compile(r"^\s+VIR_ACCESS_PERM_([_A-Z]+)(,?|\s|$).*") + for line in fh: + symmatch =3D symprog.match(line) + if symmatch is not None: + perm =3D symmatch.group(1) + + if not perm.endswith("_LAST"): + perms[perm] =3D 1 + +warned =3D False + +with open(impl) as fh: + group =3D None + symlastprog =3D re.compile(r".*VIR_ACCESS_PERM_([_A-Z]+)_LAST.*") + alnumprog =3D re.compile(r'''.*"([_a-z]+)".*''') + + for line in fh: + symlastmatch =3D symlastprog.match(line) + if symlastmatch is not None: + group =3D symlastmatch.group(1) + elif alnumprog.match(line) is not None: + bits =3D line.split(",") + for bit in bits: + m =3D alnumprog.match(bit) + if m is not None: + perm =3D (group + "_" + m.group(1)).upper() + if perm not in perms: + print("Unknown perm string %s for group %s" % (m.g= roup(1), group), file=3Dsys.stderr) + warned =3D True + + del perms[perm] + +for perm in perms.keys(): + print("Perm %s had not string form" % perm, file=3Dsys.stderr) + warned =3D True + +if warned: + sys.exit(1) +sys.exit(0) --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sat Apr 27 03:20:53 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1567679276; cv=none; d=zoho.com; s=zohoarc; b=Vw3Ap3pZ/IsuIBf7u1E2WhJYFHpvtgwN6QqM8WCVU2G0y0r5o8CjiwvXYl0bOg/pTcTRVJBZtx33f7ZneNRPldoaMcYv6zQYxaU9cM8JaYzW67OIqbEBVqkljAo69ZS8IL4UhuQNtTLLOjOExIYbRBaF5W5imiu1na/XRhWi3Yc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1567679276; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=oIgnQsG9SNWqY8lVmLit8WDp0DokvuYYqP5DQl4LeKU=; b=Uac+4l/NuKAURopXnlz2D17HYXs5Cd9y9yv/uzOVcg1BmTWmxsakPdBZz3wvsxI7d7v1huUOicxIk824GtaAx5vj3vruPfo1J5ZLEtgGcEqx2ID3zoiLWvAawSliWB7f5ppC5YMYStgETLivt/OHJwAFzH4Hs5FYDdKOlbeQe5Y= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1567679276744523.9029566866241; Thu, 5 Sep 2019 03:27:56 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9A30A30603AA; Thu, 5 Sep 2019 10:27:55 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7275F5D704; Thu, 5 Sep 2019 10:27:55 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 3233A24F36; Thu, 5 Sep 2019 10:27:55 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x85AQd3J028002 for ; Thu, 5 Sep 2019 06:26:39 -0400 Received: by smtp.corp.redhat.com (Postfix) id 1A78B5DA60; Thu, 5 Sep 2019 10:26:39 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7CD165D9E2; Thu, 5 Sep 2019 10:26:38 +0000 (UTC) From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Thu, 5 Sep 2019 11:21:59 +0100 Message-Id: <20190905102206.18753-5-berrange@redhat.com> In-Reply-To: <20190905102206.18753-1-berrange@redhat.com> References: <20190905102206.18753-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH 04/11] docs: rewrite symfile sorting checker in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.41]); Thu, 05 Sep 2019 10:27:56 +0000 (UTC) As part of an goal to eliminate Perl from libvirt build tools, rewrite the check-symsorting.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. Signed-off-by: Daniel P. Berrang=C3=A9 --- src/Makefile.am | 6 +-- src/check-symsorting.pl | 106 ------------------------------------- src/check-symsorting.py | 112 ++++++++++++++++++++++++++++++++++++++++ 3 files changed, 115 insertions(+), 109 deletions(-) delete mode 100755 src/check-symsorting.pl create mode 100755 src/check-symsorting.py diff --git a/src/Makefile.am b/src/Makefile.am index 2eb4387231..f30b085d07 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -285,12 +285,12 @@ check-symfile: check-admin-symfile: endif ! WITH_LINUX check-symsorting: - $(AM_V_GEN)$(PERL) $(srcdir)/check-symsorting.pl \ + $(AM_V_GEN)$(PYTHON) $(srcdir)/check-symsorting.py \ $(srcdir) $(SYM_FILES) check-admin-symsorting: - $(AM_V_GEN)$(PERL) $(srcdir)/check-symsorting.pl \ + $(AM_V_GEN)$(PYTHON) $(srcdir)/check-symsorting.py \ $(srcdir) $(ADMIN_SYM_FILES) -EXTRA_DIST +=3D check-symfile.pl check-symsorting.pl +EXTRA_DIST +=3D check-symfile.pl check-symsorting.py =20 # Keep this list synced with RPC_PROBE_FILES PROTOCOL_STRUCTS =3D \ diff --git a/src/check-symsorting.pl b/src/check-symsorting.pl deleted file mode 100755 index 51e38bdedb..0000000000 --- a/src/check-symsorting.pl +++ /dev/null @@ -1,106 +0,0 @@ -#!/usr/bin/env perl - -# Copyright (C) 2012-2013 Red Hat, Inc. -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library. If not, see -# . - -use strict; -use warnings; - -die "syntax: $0 SRCDIR SYMFILE..." unless int(@ARGV) >=3D 2; - -my $ret =3D 0; -my $srcdir =3D shift; -my $lastgroup =3D undef; -foreach my $symfile (@ARGV) { - open SYMFILE, $symfile or die "cannot read $symfile: $!"; - - my $line =3D 0; - my $groupfile =3D ""; - my @group; - - while () { - chomp; - - if (/^#\s*((\w+\/)*(\w+\.h))\s*$/) { - $groupfile =3D $1; - } elsif (/^#/) { - # Ignore comments - } elsif (/^\s*$/) { - if (@group) { - &check_sorting(\@group, $symfile, $line, $groupfile); - } - @group =3D (); - $line =3D $.; - } else { - $_ =3D~ s/;//; - push @group, $_; - } - } - - close SYMFILE; - if (@group) { - &check_sorting(\@group, $symfile, $line, $groupfile); - } - $lastgroup =3D undef; -} - -sub check_sorting { - my $group =3D shift; - my $symfile =3D shift; - my $line =3D shift; - my $groupfile =3D shift; - - my @group =3D @{$group}; - my @sorted =3D sort { lc $a cmp lc $b } @group; - my $sorted =3D 1; - my $first; - my $last; - - # Check that groups are in order and groupfile exists - if (defined $lastgroup && lc $lastgroup ge lc $groupfile) { - print "Symbol block at $symfile:$line: block not sorted\n"; - print "Move $groupfile block before $lastgroup block\n"; - print "\n"; - $ret =3D 1; - } - if (! -e "$srcdir/$groupfile") { - print "Symbol block at $symfile:$line: $groupfile not found\n"; - print "\n"; - $ret =3D 1; - } - $lastgroup =3D $groupfile; - - # Check that symbols within a group are in order - for (my $i =3D 0 ; $i <=3D $#sorted ; $i++) { - if ($sorted[$i] ne $group[$i]) { - $first =3D $i unless defined $first; - $last =3D $i; - $sorted =3D 0; - } - } - if (!$sorted) { - @group =3D splice @group, $first, ($last-$first+1); - @sorted =3D splice @sorted, $first, ($last-$first+1); - print "Symbol block at $symfile:$line: symbols not sorted\n"; - print map { " " . $_ . "\n" } @group; - print "Correct ordering\n"; - print map { " " . $_ . "\n" } @sorted; - print "\n"; - $ret =3D 1; - } -} - -exit $ret; diff --git a/src/check-symsorting.py b/src/check-symsorting.py new file mode 100755 index 0000000000..a1d16a4c43 --- /dev/null +++ b/src/check-symsorting.py @@ -0,0 +1,112 @@ +#!/usr/bin/env python + +# Copyright (C) 2012-2019 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . + +from __future__ import print_function + +import os.path +import re +import sys + +if len(sys.argv) < 3: + print("syntax: %s SRCDIR SYMFILE..." % sys.argv[0], file=3Dsys.stderr) + sys.exit(1) + +def check_sorting(group, symfile, line, groupfile, lastgroup): + sortedgroup =3D sorted(group, key=3Dstr.lower) + issorted =3D True + first =3D None + last =3D None + + err =3D False + # Check that groups are in order and groupfile exists + if lastgroup is not None and lastgroup.lower() > groupfile.lower(): + print("Symbol block at %s:%s: block not sorted" % (symfile, line),= file=3Dsys.stderr) + print("Move %s block before %s block" % (groupfile, lastgroup), fi= le=3Dsys.stderr) + print("", file=3Dsys.stderr) + err =3D True + + if not os.path.exists(os.path.join(srcdir, groupfile)): + print("Symbol block at %s:%s: %s not found" % (symfile, line, grou= pfile), file=3Dsys.stderr) + print("", file=3Dsys.stderr) + err =3D True + + # Check that symbols within a group are in order + for i in range(len(group)): + if sortedgroup[i] !=3D group[i]: + if first is None: + first =3D i + + last =3D i + issorted =3D False + + if not issorted: + actual =3D group[first:(last-first+1)] + expect =3D sortedgroup[first:(last-first+1)] + print("Symbol block at %s:%s: symbols not sorted" % (symfile, line= ), file=3Dsys.stderr) + for g in actual: + print(" %s" % g, file=3Dsys.stderr) + print("Correct ordering", file=3Dsys.stderr) + for g in expect: + print(" %s" % g, file=3Dsys.stderr) + print("", file=3Dsys.stderr) + err =3D True + + return err + + +ret =3D 0 +srcdir =3D sys.argv[1] +lastgroup =3D None +for symfile in sys.argv[2:]: + with open(symfile, "r") as fh: + lineno =3D 0 + groupfile =3D "" + group =3D [] + thisline =3D 0 + + filenameprog =3D re.compile(r'''^#\s*((\w+\/)*(\w+\.h))\s*$''') + + for line in fh: + thisline =3D thisline + 1 + line =3D line.strip() + + filenamematch =3D filenameprog.match(line) + if filenamematch is not None: + groupfile =3D filenamematch.group(1) + elif line =3D=3D "": + if len(group) > 0: + if check_sorting(group, symfile, lineno, groupfile, la= stgroup): + ret =3D 1 + + group =3D [] + lineno =3D thisline + lastgroup =3D groupfile + elif line[0] =3D=3D '#': + # Ignore comments + pass + else: + line =3D line.strip(";") + group.append(line) + + if len(group) > 0: + if check_sorting(group, symfile, lineno, groupfile, lastgroup): + ret =3D 1 + + lastgroup =3D None + +sys.exit(ret) --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sat Apr 27 03:20:53 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1567679280; cv=none; d=zoho.com; s=zohoarc; b=SpMatQRRI5CkosCZ0zXXhN7135ZZvSsUSdcfNAWELJZyzgN7l8HFhE5Q9h6z6OCJZ94fJx0ka3OFZQeNEubkF5FQYUPDDOUJKUY9kVBdWM00hFbZWWzBn/5R7Qy82LOKc09KBm7ycDMLdOils/JNvlIxulrn8crhsI56Phd/SgM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1567679280; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=RAC94O131Pju3eY/gnlqQcxZu9a9TKSvlYdC9ne//wY=; b=korl8SzbRtTj0fuXFtCuLvnDE0uyEa67AS6Yh9a/1cfvWKgGiKtLSXX58R93dQoaKE7eUy0nAz4PteSY6WGoTRJd6gnjTc+9fAL87CkJnDwKNnL6FalFyoqXzv3sjcU0Qod3oCUjuEEmmyB1WHCyGxtmQp/eej5bZhti8/FtelU= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1567679280334714.0833118421806; Thu, 5 Sep 2019 03:28:00 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id EBAC518B3D84; Thu, 5 Sep 2019 10:27:58 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id C104A5D704; Thu, 5 Sep 2019 10:27:58 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 79478180BA9A; Thu, 5 Sep 2019 10:27:58 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x85AQeN8028012 for ; Thu, 5 Sep 2019 06:26:40 -0400 Received: by smtp.corp.redhat.com (Postfix) id F230C5DA60; Thu, 5 Sep 2019 10:26:39 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 60AE55D9E2; Thu, 5 Sep 2019 10:26:39 +0000 (UTC) From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Thu, 5 Sep 2019 11:22:00 +0100 Message-Id: <20190905102206.18753-6-berrange@redhat.com> In-Reply-To: <20190905102206.18753-1-berrange@redhat.com> References: <20190905102206.18753-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH 05/11] docs: rewrite symfile library checker in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx1.redhat.com [10.5.110.63]); Thu, 05 Sep 2019 10:27:59 +0000 (UTC) As part of an goal to eliminate Perl from libvirt build tools, rewrite the check-symfile.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. Signed-off-by: Daniel P. Berrang=C3=A9 --- src/Makefile.am | 6 ++-- src/check-symfile.pl | 70 -------------------------------------- src/check-symfile.py | 80 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 83 insertions(+), 73 deletions(-) delete mode 100755 src/check-symfile.pl create mode 100755 src/check-symfile.py diff --git a/src/Makefile.am b/src/Makefile.am index f30b085d07..1724b3b809 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -275,10 +275,10 @@ PDWTAGS =3D \ # rule for libvirt.la. However, checking symbols relies on Linux ELF layo= ut if WITH_LINUX check-symfile: libvirt.syms libvirt.la - $(AM_V_GEN)$(PERL) $(srcdir)/check-symfile.pl libvirt.syms \ + $(AM_V_GEN)$(PYTHON) $(srcdir)/check-symfile.py libvirt.syms \ .libs/libvirt.so check-admin-symfile: libvirt_admin.syms libvirt-admin.la - $(AM_V_GEN)$(PERL) $(srcdir)/check-symfile.pl libvirt_admin.syms \ + $(AM_V_GEN)$(PYTHON) $(srcdir)/check-symfile.py libvirt_admin.syms \ .libs/libvirt-admin.so else ! WITH_LINUX check-symfile: @@ -290,7 +290,7 @@ check-symsorting: check-admin-symsorting: $(AM_V_GEN)$(PYTHON) $(srcdir)/check-symsorting.py \ $(srcdir) $(ADMIN_SYM_FILES) -EXTRA_DIST +=3D check-symfile.pl check-symsorting.py +EXTRA_DIST +=3D check-symfile.py check-symsorting.py =20 # Keep this list synced with RPC_PROBE_FILES PROTOCOL_STRUCTS =3D \ diff --git a/src/check-symfile.pl b/src/check-symfile.pl deleted file mode 100755 index 4f88300864..0000000000 --- a/src/check-symfile.pl +++ /dev/null @@ -1,70 +0,0 @@ -#!/usr/bin/env perl - -# Copyright (C) 2012-2013 Red Hat, Inc. -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library. If not, see -# . - -die "syntax: $0 SYMFILE ELFLIB(S)" unless int(@ARGV) >=3D 2; - -my $symfile =3D shift @ARGV; -my @elflibs =3D @ARGV; - -my %wantsyms; -my %gotsyms; - -my $ret =3D 0; - -open SYMFILE, $symfile or die "cannot read $symfile: $!"; - -while () { - next if /{/; - next if /}/; - next if /global:/; - next if /local:/; - next if /^\s*$/; - next if /^\s*#/; - next if /\*/; - - die "malformed line $_" unless /^\s*(\S+);$/; - - if (exists $wantsyms{$1}) { - print STDERR "Symbol $1 is listed twice\n"; - $ret =3D 1; - } else { - $wantsyms{$1} =3D 1; - } -} -close SYMFILE; - -foreach my $elflib (@elflibs) { - open NM, "-|", "nm", $elflib or die "cannot run 'nm $elflib': $!"; - - while () { - next unless /^\S+\s(?:[TBD])\s(\S+)\s*$/; - - $gotsyms{$1} =3D 1; - } - - close NM; -} - -foreach my $sym (keys(%wantsyms)) { - next if exists $gotsyms{$sym}; - - print STDERR "Expected symbol $sym is not in ELF library\n"; - $ret =3D 1; -} - -exit($ret); diff --git a/src/check-symfile.py b/src/check-symfile.py new file mode 100755 index 0000000000..3154619bfe --- /dev/null +++ b/src/check-symfile.py @@ -0,0 +1,80 @@ +#!/usr/bin/env python +# +# Copyright (C) 2012-2019 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . + +from __future__ import print_function + +import re +import subprocess +import sys + +if len(sys.argv) < 3: + print("syntax: %s SYMFILE ELFLIB(S)" % sys.argv[0], file=3Dsys.stderr) + +symfile =3D sys.argv[1] +elflibs =3D sys.argv[2:] + +wantsyms =3D {} +gotsyms =3D {} + +ret =3D 0 + +with open(symfile, "r") as fh: + for line in fh: + line =3D line.strip() + if line.find("{") !=3D -1: + continue + if line.find("}") !=3D -1: + continue + if line in ["global:", "local:"]: + continue + if line =3D=3D "": + continue + if line[0] =3D=3D '#': + continue + if line.find("*") !=3D -1: + continue + + line =3D line.strip(";") + + if line in wantsyms: + print("Symbol $1 is listed twice", file=3Dsys.stderr) + ret =3D 1 + else: + wantsyms[line] =3D True + +for elflib in elflibs: + nm =3D subprocess.Popen(["nm", elflib], shell=3DFalse, stdout=3Dsubpro= cess.PIPE).stdout + + symprog =3D re.compile(r'''^\S+\s(?:[TBD])\s(\S+)\s*$''') + for line in nm: + line =3D line.decode("utf-8") + symmatch =3D symprog.match(line) + if symmatch is None: + continue + + gotsyms[symmatch.group(1)] =3D True + + +for sym in wantsyms.keys(): + if sym in gotsyms: + continue + + print("Expected symbol '%s' is not in ELF library" % sym, file=3Dsys.s= tderr) + ret =3D 1 + +sys.exit(ret) --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sat Apr 27 03:20:53 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1567679325; cv=none; d=zoho.com; s=zohoarc; b=kvzCm96SrC4+mdEwz87cb1tO4dXldvjDc0vSoGTFxzFn8CLgwFosFSCbI70r1aQylvGgh8/L9u1YoO2Q9c2+lXIOiFpZL+XYPVH001PtOePlg7iW1wLrQVZrRH9pXv7U+EvVj95FUztsbOjn6c6qBLcE0EwzJKfZ4mX3T4f4ZJs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1567679325; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=9+fkCGpq4higBZy2vAlX0Zql1X4LFFIK+DnbntQSOvk=; b=BJp/1BPqyYlBsemeRM9bgVkanahJhitAjNS/jmf8rlrzImP4UA/LzxQ/o6+x4/ROcxEXcY1mhfq+gswMIo5PRK0gu3pOtLXj7pz9YhwCYK+CMnnYfJlYkB2eFE5Zr7kC7Z8OyjP1XnsCK1OnNS7YlFuF6HrZN8bDLXB7BsQlcVk= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1567679325796877.3738151316682; Thu, 5 Sep 2019 03:28:45 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 27F0E8535C; Thu, 5 Sep 2019 10:28:44 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 00C885C231; Thu, 5 Sep 2019 10:28:44 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id A0CD61806B01; Thu, 5 Sep 2019 10:28:43 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x85AQeBE028021 for ; Thu, 5 Sep 2019 06:26:40 -0400 Received: by smtp.corp.redhat.com (Postfix) id D5B3C5DA60; Thu, 5 Sep 2019 10:26:40 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4443A5D9E2; Thu, 5 Sep 2019 10:26:40 +0000 (UTC) From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Thu, 5 Sep 2019 11:22:01 +0100 Message-Id: <20190905102206.18753-7-berrange@redhat.com> In-Reply-To: <20190905102206.18753-1-berrange@redhat.com> References: <20190905102206.18753-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH 06/11] docs: rewrite augest test generator in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.25]); Thu, 05 Sep 2019 10:28:45 +0000 (UTC) As part of an goal to eliminate Perl from libvirt build tools, rewrite the augeas-gentest.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. Signed-off-by: Daniel P. Berrang=C3=A9 --- Makefile.am | 2 +- build-aux/augeas-gentest.pl | 60 ------------------------------- build-aux/augeas-gentest.py | 72 +++++++++++++++++++++++++++++++++++++ src/Makefile.am | 2 +- 4 files changed, 74 insertions(+), 62 deletions(-) delete mode 100755 build-aux/augeas-gentest.pl create mode 100755 build-aux/augeas-gentest.py diff --git a/Makefile.am b/Makefile.am index cf9ff94f4f..e909d29d10 100644 --- a/Makefile.am +++ b/Makefile.am @@ -41,7 +41,7 @@ EXTRA_DIST =3D \ run.in \ README.md \ AUTHORS.in \ - build-aux/augeas-gentest.pl \ + build-aux/augeas-gentest.py \ build-aux/check-spacing.pl \ build-aux/gitlog-to-changelog \ build-aux/header-ifdef.pl \ diff --git a/build-aux/augeas-gentest.pl b/build-aux/augeas-gentest.pl deleted file mode 100755 index 65834b533b..0000000000 --- a/build-aux/augeas-gentest.pl +++ /dev/null @@ -1,60 +0,0 @@ -#!/usr/bin/env perl -# -# augeas-gentest.pl: Generate an augeas test file, from an -# example config file + test file template -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library. If not, see -# . - -use strict; -use warnings; - -die "syntax: $0 CONFIG TEMPLATE\n" unless @ARGV =3D=3D 2; - -my $config =3D shift @ARGV; -my $template =3D shift @ARGV; - -open CONFIG, "<", $config or die "cannot read $config: $!"; -open TEMPLATE, "<", $template or die "cannot read $template: $!"; - -my $group =3D 0; -while (