From nobody Sun May 5 10:21:35 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 207.211.31.120 as permitted sender) client-ip=207.211.31.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1575912139; cv=none; d=zohomail.com; s=zohoarc; b=Kst3iUQyV3UGzC5dzv73bQl4Mmmy0J1DDU0dzgpG64LNPJOEUiOOUn9HrBEjM7HP+JvdcLPZw+SxZrBLZ+onNrdPSk1x+Q8GWZ4uvXMNp5iI71bdpZzU2rlIdAmg8e0hxkbd8CL1fzArkMTumQqoDyA5nIi/ssZ0sGM2RJbf8zk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1575912139; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=vEcNXMYXiJHLhzyt4GReg2kGKU9slkxFQGsdMdXKqzM=; b=aNf7+vjKJ/V1XhVHElZcl721ZaFDYCtldVss7DFqmnM8BeTxt2iJ5GIxIL4nDurI1w4YRAKTkyoZAZEd9PD560L/JYZNCP2OhK1YBjt2oX4yK0jpBC0GrTeJLRn3wpQGoHe0/b90orqN7esdbmVVJDhMZlsJrCXM58662Mu1drg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by mx.zohomail.com with SMTPS id 1575912139570605.7222764048968; Mon, 9 Dec 2019 09:22:19 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-145-ai4Whma_OcieXAf4ZtaCNQ-1; Mon, 09 Dec 2019 12:22:15 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E36FE100A158; Mon, 9 Dec 2019 17:22:09 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id AFCD41001B09; Mon, 9 Dec 2019 17:22:09 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 0CCA283554; Mon, 9 Dec 2019 17:22:09 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xB9HLYbN015119 for ; Mon, 9 Dec 2019 12:21:34 -0500 Received: by smtp.corp.redhat.com (Postfix) id 769795D6C8; Mon, 9 Dec 2019 17:21:34 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id B53995D6B7; Mon, 9 Dec 2019 17:21:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1575912138; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=vEcNXMYXiJHLhzyt4GReg2kGKU9slkxFQGsdMdXKqzM=; b=BEj++l3vq6eA0ZqNayg/A/EkfxZNGH3BPKWx4E984IjStIX235sEFgm8uZu4+SuptYNN4+ Q7PA/y+nCF/aYoWCeJ5TECGp/m50hLxkNA8ZCC+XaFWA58JjvbYUubC8MUMOiGhJmcV79K Sf6lfGhQfisdMRUy2ppiTMd+OFLVYCo= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 9 Dec 2019 17:14:19 +0000 Message-Id: <20191209171422.31091-2-berrange@redhat.com> In-Reply-To: <20191209171422.31091-1-berrange@redhat.com> References: <20191209171422.31091-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Cc: Cole Robinson Subject: [libvirt] [PATCH v6 1/4] tests: rewrite qemu capability grouper in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: ai4Whma_OcieXAf4ZtaCNQ-1 X-Mimecast-Spam-Score: 0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) As part of a goal to eliminate Perl from libvirt build tools, rewrite the group-qemu-caps.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. Tested-by: Cole Robinson Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: Cole Robinson --- Makefile.am | 1 + build-aux/syntax-check.mk | 3 +- scripts/group-qemu-caps.py | 121 ++++++++++++++++++++++++++++++++++++ tests/group-qemu-caps.pl | 124 ------------------------------------- 4 files changed, 124 insertions(+), 125 deletions(-) create mode 100755 scripts/group-qemu-caps.py delete mode 100755 tests/group-qemu-caps.pl diff --git a/Makefile.am b/Makefile.am index 15df6ed68e..545f2bdebe 100644 --- a/Makefile.am +++ b/Makefile.am @@ -59,6 +59,7 @@ EXTRA_DIST =3D \ scripts/esx_vi_generator.py \ scripts/genpolkit.py \ scripts/gensystemtap.py \ + scripts/group-qemu-caps.py \ scripts/header-ifdef.py \ scripts/hyperv_wmi_generator.py \ scripts/minimize-po.py \ diff --git a/build-aux/syntax-check.mk b/build-aux/syntax-check.mk index b29def5a30..5cc1e206c2 100644 --- a/build-aux/syntax-check.mk +++ b/build-aux/syntax-check.mk @@ -2194,7 +2194,8 @@ test-wrap-argv: $(PYTHON) $(top_srcdir)/scripts/test-wrap-argv.py --check =20 group-qemu-caps: - $(AM_V_GEN)$(PERL) $(top_srcdir)/tests/group-qemu-caps.pl --check $(top_s= rcdir)/ + $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(top_srcdir)/scripts/group-qemu-caps.py \ + --check --prefix $(top_srcdir)/ =20 # List all syntax-check exemptions: exclude_file_name_regexp--sc_avoid_strcase =3D ^tools/vsh\.h$$ diff --git a/scripts/group-qemu-caps.py b/scripts/group-qemu-caps.py new file mode 100755 index 0000000000..c53ff3ff6f --- /dev/null +++ b/scripts/group-qemu-caps.py @@ -0,0 +1,121 @@ +#!/usr/bin/env python3 +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . +# +# +# Regroup array values into smaller groups separated by numbered comments. +# +# If --check is the first parameter, the script will return +# a non-zero value if a file is not grouped correctly. +# Otherwise the files are regrouped in place. + +import argparse +import re +import subprocess +import sys + + +def regroup_caps(check, filename, start_regex, end_regex, + trailing_newline, counter_prefix): + step =3D 5 + + original =3D [] + with open(filename, "r") as fh: + for line in fh: + original.append(line) + + fixed =3D [] + game_on =3D False + counter =3D 0 + for line in original: + line =3D line.rstrip("\n") + if game_on: + if re.search(r'''.*/\* [0-9]+ \*/.*''', line): + continue + if re.search(r'''^\s*$''', line): + continue + if counter % step =3D=3D 0: + if counter !=3D 0: + fixed.append("\n") + fixed.append("%s/* %d */\n" % (counter_prefix, counter)) + + if not (line.find("/*") !=3D -1 and line.find("*/") =3D=3D -1): + # count two-line comments as one line + counter =3D counter + 1 + + if re.search(start_regex, line): + game_on =3D True + elif game_on and re.search(end_regex, line): + if (counter - 1) % step =3D=3D 0: + fixed =3D fixed[:-1] # /* $counter */ + if counter !=3D 1: + fixed =3D fixed[:-1] # \n + + if trailing_newline: + fixed.append("\n") + + game_on =3D False + + fixed.append(line + "\n") + + if check: + orig =3D "".join(original) + new =3D "".join(fixed) + if new !=3D orig: + diff =3D subprocess.Popen(["diff", "-u", filename, "-"], + stdin=3Dsubprocess.PIPE) + diff.communicate(input=3Dnew.encode('utf-8')) + + print("Incorrect line wrapping in $file", + file=3Dsys.stderr) + print("Use group-qemu-caps.py to generate data files", + file=3Dsys.stderr) + return False + else: + with open(filename, "w") as fh: + for line in fixed: + print(line, file=3Dfh, end=3D'') + + return True + + +parser =3D argparse.ArgumentParser(description=3D'QEMU capabilities group = formatter') +parser.add_argument('--check', action=3D"store_true", + help=3D'check existing files only') +parser.add_argument('--prefix', default=3D'', + help=3D'source code tree prefix') +args =3D parser.parse_args() + +errs =3D False + +if not regroup_caps(args.check, + args.prefix + 'src/qemu/qemu_capabilities.c', + r'virQEMUCaps grouping marker', + r'\);', + 0, + " "): + errs =3D True + +if not regroup_caps(args.check, + args.prefix + 'src/qemu/qemu_capabilities.h', + r'virQEMUCapsFlags grouping marker', + r'QEMU_CAPS_LAST \/\* this must', + 1, + " "): + errs =3D True + +if errs: + sys.exit(1) +sys.exit(0) diff --git a/tests/group-qemu-caps.pl b/tests/group-qemu-caps.pl deleted file mode 100755 index 829e63a562..0000000000 --- a/tests/group-qemu-caps.pl +++ /dev/null @@ -1,124 +0,0 @@ -#!/usr/bin/env perl -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library. If not, see -# . -# -# -# Regroup array values into smaller groups separated by numbered comments. -# -# If --check is the first parameter, the script will return -# a non-zero value if a file is not grouped correctly. -# Otherwise the files are regrouped in place. - -use strict; -use warnings; - -my $check =3D 0; - -if (defined $ARGV[0] && $ARGV[0] eq "--check") { - $check =3D 1; - shift @ARGV; -} - -my $prefix =3D ''; -if (defined $ARGV[0]) { - $prefix =3D $ARGV[0]; - shift @ARGV; -} - -my $ret =3D 0; -if (®roup_caps($prefix . 'src/qemu/qemu_capabilities.c', - 'virQEMUCaps grouping marker', - '\);', - 0, - " ") < 0) { - $ret =3D 1; -} -if (®roup_caps($prefix . 'src/qemu/qemu_capabilities.h', - 'virQEMUCapsFlags grouping marker', - 'QEMU_CAPS_LAST \/\* this must', - 1, - " ") < 0) { - $ret =3D 1; -} - -exit $ret; - -sub regroup_caps { - my $filename =3D shift; - my $start_regex =3D shift; - my $end_regex =3D shift; - my $trailing_newline =3D shift; - my $counter_prefix =3D shift; - my $step =3D 5; - - open FILE, '<', $filename or die "cannot open $filename: $!"; - my @original =3D ; - close FILE; - - my @fixed; - my $game_on =3D 0; - my $counter =3D 0; - foreach (@original) { - if ($game_on) { - next if ($_ =3D~ '/\* [0-9]+ \*/'); - next if (/^\s+$/); - if ($counter % $step =3D=3D 0) { - if ($counter !=3D 0) { - push @fixed, "\n"; - } - push @fixed, "$counter_prefix/* $counter */\n"; - } - if (!($_ =3D~ '/\*' && !($_ =3D~ '\*/'))) { - # count two-line comments as one line - $counter++; - } - } - if (/$start_regex/) { - $game_on =3D 1; - } elsif ($game_on && $_ =3D~ /$end_regex/) { - if (($counter -1) % $step =3D=3D 0) { - pop @fixed; # /* $counter */ - if ($counter !=3D 1) { - pop @fixed; # \n - } - } - if ($trailing_newline) { - push @fixed, "\n"; - } - $game_on =3D 0; - } - push @fixed, $_; - } - - if ($check) { - my $nl =3D join('', @fixed); - my $ol =3D join('', @original); - unless ($nl eq $ol) { - open DIFF, "| diff -u $filename -" or die "cannot run diff: $!= "; - print DIFF $nl; - close DIFF; - - print STDERR "Incorrect array grouping in $filename\n"; - print STDERR "Use group-qemu-caps.pl to group long array membe= rs\n"; - return -1; - } - } else { - open FILE, '>', $filename or die "cannot open $filename: $!"; - foreach my $line (@fixed) { - print FILE $line; - } - close FILE; - } -} --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sun May 5 10:21:35 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1575912184; cv=none; d=zohomail.com; s=zohoarc; b=NwqRWIn5aTFQplr4kVWpcX+37gt4MiHReU9AztAc6YPIE3PjTrb+aafAA0CYV0y9Jfez6KxqBdZr5t//xqWg/whx3+fFzEb5y5T6IqcajgnbszevdC0KaYFwuXU9laNMCzkxXIpM+H91sVrtRT1zE1Zg13wrQQyE9uScxCFLaK4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1575912184; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=nvFjlhVZ1LwvZY+zFZtrJUWwCtp1xXrW3IXj58lXiwg=; b=ElOOd8hFoSHSjeyARIEamOCwZfdMuF87tSlO5k2wOgG44aOr5mkcWF+4BIRqjkDqo5UtC1HnIcS7iX15epgI3s+sKccfoMClnWsW6rkSEgZiTyvWViC0P5YQ6Gi/JNu+XZIApgCCAbgJFA4mMB5rqhFAg98Sikc9E2PjkSG10b4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 1575912184909659.2211342430397; Mon, 9 Dec 2019 09:23:04 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-145-lv1dB2tnMLKy9hiR8WsTkw-1; Mon, 09 Dec 2019 12:22:15 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0E2B485B6F3; Mon, 9 Dec 2019 17:22:10 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id CF59F1001B07; Mon, 9 Dec 2019 17:22:09 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 56C7383555; Mon, 9 Dec 2019 17:22:09 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xB9HLZVE015125 for ; Mon, 9 Dec 2019 12:21:35 -0500 Received: by smtp.corp.redhat.com (Postfix) id 5B1BF5D6C8; Mon, 9 Dec 2019 17:21:35 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id BCC735D6B7; Mon, 9 Dec 2019 17:21:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1575912182; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=nvFjlhVZ1LwvZY+zFZtrJUWwCtp1xXrW3IXj58lXiwg=; b=HNbzWtHtx8baGRj9wExbAZ6N2wykdLW4XMQayW6b8JV4MW90DnSVaBoGew1IW670jPE8f8 rg4C2QUrPoCqp6No2ZNIaErlKP5EidIr1B8SQTbLxupYtGZJvxzQ8YdU9SWheFkU27xdZX tt8hDVSMJmFY52PghLqsYjmR9VCZCTE= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 9 Dec 2019 17:14:20 +0000 Message-Id: <20191209171422.31091-3-berrange@redhat.com> In-Reply-To: <20191209171422.31091-1-berrange@redhat.com> References: <20191209171422.31091-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v6 2/4] tests: rewrite file access checker in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: lv1dB2tnMLKy9hiR8WsTkw-1 X-Mimecast-Spam-Score: 0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) As part of a goal to eliminate Perl from libvirt build tools, rewrite the check-file-access.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: Cole Robinson --- Makefile.am | 1 + scripts/check-file-access.py | 125 ++++++++++++++++++++++++++++++ tests/Makefile.am | 3 +- tests/check-file-access.pl | 130 -------------------------------- tests/file_access_whitelist.txt | 2 +- 5 files changed, 128 insertions(+), 133 deletions(-) create mode 100755 scripts/check-file-access.py delete mode 100755 tests/check-file-access.pl diff --git a/Makefile.am b/Makefile.am index 545f2bdebe..b9d33f5cf8 100644 --- a/Makefile.am +++ b/Makefile.am @@ -52,6 +52,7 @@ EXTRA_DIST =3D \ scripts/check-aclrules.py \ scripts/check-drivername.py \ scripts/check-driverimpls.py \ + scripts/check-file-access.py \ scripts/check-remote-protocol.py \ scripts/check-symfile.py \ scripts/check-symsorting.py \ diff --git a/scripts/check-file-access.py b/scripts/check-file-access.py new file mode 100755 index 0000000000..dd39de2d79 --- /dev/null +++ b/scripts/check-file-access.py @@ -0,0 +1,125 @@ +#!/usr/bin/env python3 +# +# Copyright (C) 2016-2019 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . +# +# This script is supposed to check test_file_access.txt file and +# warn about file accesses outside our working tree. +# +# + +import re +import sys + +if len(sys.argv) !=3D 3: + print("syntax: %s ACCESS-FILE ACCESS-WHITELIST") + sys.exit(1) + +access_file =3D sys.argv[1] +whitelist_file =3D sys.argv[2] + +known_actions =3D ["open", "fopen", "access", "stat", "lstat", "connect"] + +files =3D [] +whitelist =3D [] + +with open(access_file, "r") as fh: + for line in fh: + line =3D line.rstrip("\n") + + m =3D re.search(r'''^(\S*):\s*(\S*):\s*(\S*)(\s*:\s*(.*))?$''', li= ne) + if m is not None: + rec =3D { + "path": m.group(1), + "action": m.group(2), + "progname": m.group(3), + "testname": m.group(5), + } + files.append(rec) + else: + raise Exception("Malformed line %s" % line) + +with open(whitelist_file, "r") as fh: + for line in fh: + line =3D line.rstrip("\n") + + if re.search(r'''^\s*#.*$''', line): + continue # comment + if line =3D=3D "": + continue + + m =3D re.search(r'''^(\S*):\s*(\S*)(:\s*(\S*)(\s*:\s*(.*))?)?$''',= line) + if m is not None and m.group(2) in known_actions: + # $path: $action: $progname: $testname + rec =3D { + "path": m.group(1), + "action": m.group(3), + "progname": m.group(4), + "testname": m.group(6), + } + whitelist.append(rec) + else: + m =3D re.search(r'''^(\S*)(:\s*(\S*)(\s*:\s*(.*))?)?$''', line) + if m is not None: + # $path: $progname: $testname + rec =3D { + "path": m.group(1), + "action": None, + "progname": m.group(3), + "testname": m.group(5), + } + whitelist.append(rec) + else: + raise Exception("Malformed line %s" % line) + + +# Now we should check if %traces is included in $whitelist. For +# now checking just keys is sufficient +err =3D False +for file in files: + match =3D False + + for rule in whitelist: + if not re.match("^" + rule["path"] + "$", file["path"]): + continue + + if (rule["action"] is not None and + not re.match("^" + rule["action"] + "$", file["action"])): + continue + + if (rule["progname"] is not None and + not re.match("^" + rule["progname"] + "$", file["progname"= ])): + continue + + if (rule["testname"] is not None and + file["testname"] is not None and + not re.match("^" + rule["testname"] + "$", file["testname"= ])): + continue + + match =3D True + + if not match: + err =3D True + print("%s: %s: %s" % + (file["path"], file["action"], file["progname"]), + end=3D"") + if file["testname"] is not None: + print(": %s" % file["testname"], end=3D"") + print("") + +if err: + sys.exit(1) +sys.exit(0) diff --git a/tests/Makefile.am b/tests/Makefile.am index 9716d9d2be..1e364dad70 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -454,7 +454,7 @@ EXTRA_DIST +=3D $(test_scripts) if WITH_LINUX check-access: file-access-clean VIR_TEST_FILE_ACCESS=3D1 $(MAKE) $(AM_MAKEFLAGS) check - $(PERL) $(abs_srcdir)/check-file-access.pl \ + $(RUNUTF8) $(PYTHON) $(top_srcdir)/scripts/check-file-access.py \ $(abs_builddir)/test_file_access.txt \ $(abs_srcdir)/file_access_whitelist.txt | sort -u =20 @@ -463,7 +463,6 @@ file-access-clean: endif WITH_LINUX =20 EXTRA_DIST +=3D \ - check-file-access.pl \ file_access_whitelist.txt =20 if WITH_TESTS diff --git a/tests/check-file-access.pl b/tests/check-file-access.pl deleted file mode 100755 index 2926126b14..0000000000 --- a/tests/check-file-access.pl +++ /dev/null @@ -1,130 +0,0 @@ -#!/usr/bin/env perl -# -# Copyright (C) 2016 Red Hat, Inc. -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library. If not, see -# . -# -# This script is supposed to check test_file_access.txt file and -# warn about file accesses outside our working tree. -# -# - -use strict; -use warnings; - -sub usage { - die "$0 access_file file_access_whitelist\n"; -} - -my $access_file =3D shift or usage(); -my $whitelist_file =3D shift or usage(); - -my @known_actions =3D ("open", "fopen", "access", "stat", "lstat", "connec= t"); - -my @files; -my @whitelist; - -open FILE, "<", $access_file or die "Unable to open $access_file: $!"; -while () { - chomp; - if (/^(\S*):\s*(\S*):\s*(\S*)(\s*:\s*(.*))?$/) { - my %rec; - ${rec}{path} =3D $1; - ${rec}{action} =3D $2; - ${rec}{progname} =3D $3; - if (defined $5) { - ${rec}{testname} =3D $5; - } - push (@files, \%rec); - } else { - die "Malformed line $_"; - } -} -close FILE; - -open FILE, "<", $whitelist_file or die "Unable to open $whitelist_file: $!= "; -while () { - chomp; - if (/^\s*#.*$/) { - # comment - } elsif (/^(\S*):\s*(\S*)(:\s*(\S*)(\s*:\s*(.*))?)?$/ and - grep /^$2$/, @known_actions) { - # $path: $action: $progname: $testname - my %rec; - ${rec}{path} =3D $1; - ${rec}{action} =3D $3; - if (defined $4) { - ${rec}{progname} =3D $4; - } - if (defined $6) { - ${rec}{testname} =3D $6; - } - push (@whitelist, \%rec); - } elsif (/^(\S*)(:\s*(\S*)(\s*:\s*(.*))?)?$/) { - # $path: $progname: $testname - my %rec; - ${rec}{path} =3D $1; - if (defined $3) { - ${rec}{progname} =3D $3; - } - if (defined $5) { - ${rec}{testname} =3D $5; - } - push (@whitelist, \%rec); - } else { - die "Malformed line $_"; - } -} -close FILE; - -# Now we should check if %traces is included in $whitelist. For -# now checking just keys is sufficient -my $error =3D 0; -for my $file (@files) { - my $match =3D 0; - - for my $rule (@whitelist) { - if (not %${file}{path} =3D~ m/^$rule->{path}$/) { - next; - } - - if (defined %${rule}{action} and - not %${file}{action} =3D~ m/^$rule->{action}$/) { - next; - } - - if (defined %${rule}{progname} and - not %${file}{progname} =3D~ m/^$rule->{progname}$/) { - next; - } - - if (defined %${rule}{testname} and - defined %${file}{testname} and - not %${file}{testname} =3D~ m/^$rule->{testname}$/) { - next; - } - - $match =3D 1; - } - - if (not $match) { - $error =3D 1; - print "$file->{path}: $file->{action}: $file->{progname}"; - print ": $file->{testname}" if defined %${file}{testname}; - print "\n"; - } -} - -exit $error; diff --git a/tests/file_access_whitelist.txt b/tests/file_access_whitelist.= txt index 3fb318cbab..5ec7ee63bb 100644 --- a/tests/file_access_whitelist.txt +++ b/tests/file_access_whitelist.txt @@ -5,7 +5,7 @@ # $path: $progname: $testname # $path: $action: $progname: $testname # -# All these variables are evaluated as perl RE. So to allow +# All these variables are evaluated as python RE. So to allow # /dev/sda and /dev/sdb, you can just '/dev/sd[a-b]', or to allow # /proc/$pid/status you can '/proc/\d+/status' and so on. # Moreover, $action, $progname and $testname can be empty, in which --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sun May 5 10:21:35 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) client-ip=205.139.110.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1575912149; cv=none; d=zohomail.com; s=zohoarc; b=MRkoOfeM7QcfmOzyTHk6ihgH+rUhMwE/slTcRLuOpQf8Tg3oi18rrb+KmwOTYpu2bukBbruhEsbl/3e2iweZneXHAkpp3m9hYrxjMGBdCoOAAw4sisl0PdWJddHUOTuqbVZptoTlrKtQhkGpAi5FrmdjVgOAD0Dslv+1Wn6VFfE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1575912149; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=CZPVV53cn9kh/4TMEb+HS0cOPyHRjj7R5+HjcTzaZRk=; b=Frv4iWsB2xEHdfF2tJsPhVd63V9Ub0M4ryvcKRBrij16ZJWnOUtBfYuE+03Pha5nQLcyqZZJ6c/BZksV8blpTJFeuwJD7U4jdJukHjDDuIdtxbx1gBtCAoMO7IGHWkyHcei25ImQ+e0s27duPS7dFzYJJDhN2RgAqSmFDKRfkZM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by mx.zohomail.com with SMTPS id 1575912149871854.0452211868845; Mon, 9 Dec 2019 09:22:29 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-154-EyxHisOlO8S4f85i6SodLA-1; Mon, 09 Dec 2019 12:22:20 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A292C85B6FA; Mon, 9 Dec 2019 17:22:13 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7F4F15C219; Mon, 9 Dec 2019 17:22:13 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 3FEB318034EC; Mon, 9 Dec 2019 17:22:13 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xB9HLaJB015141 for ; Mon, 9 Dec 2019 12:21:36 -0500 Received: by smtp.corp.redhat.com (Postfix) id 862095D6C8; Mon, 9 Dec 2019 17:21:36 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id A19EB5D6B7; Mon, 9 Dec 2019 17:21:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1575912148; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=CZPVV53cn9kh/4TMEb+HS0cOPyHRjj7R5+HjcTzaZRk=; b=U0Gadvy3r622dkIY5k70JFi7ryQtzuZJDO5oVL/k/n3+skKq1OG0l/EwgbztBYx5uFFsua d9mh4XsSO56SwQZPcD62T5kFio+HyrE/n+k0/3JEG6CcCzUbEbVAjgeArAOFm5LLOUqWkQ N73Iudtu65Mx4EKK1BjzPUfxJ3xNKHk= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 9 Dec 2019 17:14:21 +0000 Message-Id: <20191209171422.31091-4-berrange@redhat.com> In-Reply-To: <20191209171422.31091-1-berrange@redhat.com> References: <20191209171422.31091-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Cc: Cole Robinson Subject: [libvirt] [PATCH v6 3/4] docs: rewrite hvsupport.html page generator in python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-MC-Unique: EyxHisOlO8S4f85i6SodLA-1 X-Mimecast-Spam-Score: 0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) As part of a goal to eliminate Perl from libvirt build tools, rewrite the hvsupport.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. The new impl generates byte-for-byte identical output to the old impl. Tested-by: Cole Robinson Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: Cole Robinson --- Makefile.am | 1 + docs/Makefile.am | 7 +- docs/hvsupport.pl | 459 --------------------------------------- scripts/hvsupport.py | 502 +++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 506 insertions(+), 463 deletions(-) delete mode 100755 docs/hvsupport.pl create mode 100755 scripts/hvsupport.py diff --git a/Makefile.am b/Makefile.am index b9d33f5cf8..51ef567d3b 100644 --- a/Makefile.am +++ b/Makefile.am @@ -62,6 +62,7 @@ EXTRA_DIST =3D \ scripts/gensystemtap.py \ scripts/group-qemu-caps.py \ scripts/header-ifdef.py \ + scripts/hvsupport.py \ scripts/hyperv_wmi_generator.py \ scripts/minimize-po.py \ scripts/mock-noinline.py \ diff --git a/docs/Makefile.am b/docs/Makefile.am index 5f5dce28eb..18d17b3727 100644 --- a/docs/Makefile.am +++ b/docs/Makefile.am @@ -245,7 +245,6 @@ EXTRA_DIST=3D \ $(internals_html_in) $(internals_rst) $(fonts) \ $(kbase_html_in) $(kbase_rst) \ aclperms.htmlinc \ - hvsupport.pl \ $(schema_DATA) =20 acl_generated =3D aclperms.htmlinc @@ -275,12 +274,12 @@ timestamp=3D"$(shell if test -n "$$SOURCE_DATE_EPOCH"= ; \ =20 hvsupport.html: hvsupport.html.in =20 -hvsupport.html.in: $(srcdir)/hvsupport.pl $(api_DATA) \ +hvsupport.html.in: $(top_srcdir)/scripts/hvsupport.py $(api_DATA) \ $(top_srcdir)/src/libvirt_public.syms \ $(top_srcdir)/src/libvirt_qemu.syms $(top_srcdir)/src/libvirt_lxc.syms \ $(top_srcdir)/src/driver.h - $(AM_V_GEN)$(PERL) $(srcdir)/hvsupport.pl $(top_srcdir) $(top_builddir) >= $@ \ - || { rm $@ && exit 1; } + $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(top_srcdir)/scripts/hvsupport.py \ + $(top_srcdir) $(top_builddir) > $@ || { rm $@ && exit 1; } =20 news.html.in: \ $(srcdir)/news.xml \ diff --git a/docs/hvsupport.pl b/docs/hvsupport.pl deleted file mode 100755 index 0977098eac..0000000000 --- a/docs/hvsupport.pl +++ /dev/null @@ -1,459 +0,0 @@ -#!/usr/bin/env perl - -use strict; -use warnings; - -use File::Find; - -die "syntax: $0 SRCDIR BUILDDIR\n" unless int(@ARGV) =3D=3D 2; - -my $srcdir =3D shift @ARGV; -my $builddir =3D shift @ARGV; - -my $symslibvirt =3D "$srcdir/src/libvirt_public.syms"; -my $symsqemu =3D "$srcdir/src/libvirt_qemu.syms"; -my $symslxc =3D "$srcdir/src/libvirt_lxc.syms"; -my @drivertable =3D ( - "$srcdir/src/driver-hypervisor.h", - "$srcdir/src/driver-interface.h", - "$srcdir/src/driver-network.h", - "$srcdir/src/driver-nodedev.h", - "$srcdir/src/driver-nwfilter.h", - "$srcdir/src/driver-secret.h", - "$srcdir/src/driver-state.h", - "$srcdir/src/driver-storage.h", - "$srcdir/src/driver-stream.h", - ); - -my %groupheaders =3D ( - "virHypervisorDriver" =3D> "Hypervisor APIs", - "virNetworkDriver" =3D> "Virtual Network APIs", - "virInterfaceDriver" =3D> "Host Interface APIs", - "virNodeDeviceDriver" =3D> "Host Device APIs", - "virStorageDriver" =3D> "Storage Pool APIs", - "virSecretDriver" =3D> "Secret APIs", - "virNWFilterDriver" =3D> "Network Filter APIs", - ); - - -my @srcs; -find({ - wanted =3D> sub { - if (m!$srcdir/src/.*/\w+_(driver|common|tmpl|monitor|hal|udev)\.c$= !) { - push @srcs, $_ if $_ !~ /vbox_driver\.c/; - } - }, no_chdir =3D> 1}, "$srcdir/src"); - -# Map API=C2=A0functions to the header and documentation files they're in -# so that we can generate proper hyperlinks to their documentation. -# -# The function names are grep'd from the XML output of apibuild.py. -sub getAPIFilenames { - my $filename =3D shift; - - my %files; - my $line; - - open FILE, "<", $filename or die "cannot read $filename: $!"; - - while (defined($line =3D )) { - if ($line =3D~ /function name=3D'([^']+)' file=3D'([^']+)'/) { - $files{$1} =3D $2; - } - } - - close FILE; - - if (keys %files =3D=3D 0) { - die "No functions found in $filename. Has the apibuild.py output c= hanged?"; - } - return \%files; -} - -sub parseSymsFile { - my $apisref =3D shift; - my $prefix =3D shift; - my $filename =3D shift; - my $xmlfilename =3D shift; - - my $line; - my $vers; - my $prevvers; - - my $filenames =3D getAPIFilenames($xmlfilename); - - open FILE, "<$filename" - or die "cannot read $filename: $!"; - - while (defined($line =3D )) { - chomp $line; - next if $line =3D~ /^\s*#/; - next if $line =3D~ /^\s*$/; - next if $line =3D~ /^\s*(global|local):/; - if ($line =3D~ /^\s*${prefix}_(\d+\.\d+\.\d+)\s*{\s*$/) { - if (defined $vers) { - die "malformed syms file"; - } - $vers =3D $1; - } elsif ($line =3D~ /\s*}\s*;\s*$/) { - if (defined $prevvers) { - die "malformed syms file"; - } - $prevvers =3D $vers; - $vers =3D undef; - } elsif ($line =3D~ /\s*}\s*${prefix}_(\d+\.\d+\.\d+)\s*;\s*$/) { - if ($1 ne $prevvers) { - die "malformed syms file $1 !=3D $vers"; - } - $prevvers =3D $vers; - $vers =3D undef; - } elsif ($line =3D~ /\s*(\w+)\s*;\s*$/) { - $$apisref{$1} =3D {}; - $$apisref{$1}->{vers} =3D $vers; - $$apisref{$1}->{file} =3D $$filenames{$1}; - } else { - die "unexpected data $line\n"; - } - } - - close FILE; -} - -my %apis; -# Get the list of all public APIs and their corresponding version -parseSymsFile(\%apis, "LIBVIRT", $symslibvirt, "$builddir/docs/libvirt-api= .xml"); - -# And the same for the QEMU specific APIs -parseSymsFile(\%apis, "LIBVIRT_QEMU", $symsqemu, "$builddir/docs/libvirt-q= emu-api.xml"); - -# And the same for the LXC specific APIs -parseSymsFile(\%apis, "LIBVIRT_LXC", $symslxc, "$builddir/docs/libvirt-lxc= -api.xml"); - - -# Some special things which aren't public APIs, -# but we want to report -$apis{virConnectSupportsFeature}->{vers} =3D "0.3.2"; -$apis{virDomainMigratePrepare}->{vers} =3D "0.3.2"; -$apis{virDomainMigratePerform}->{vers} =3D "0.3.2"; -$apis{virDomainMigrateFinish}->{vers} =3D "0.3.2"; -$apis{virDomainMigratePrepare2}->{vers} =3D "0.5.0"; -$apis{virDomainMigrateFinish2}->{vers} =3D "0.5.0"; -$apis{virDomainMigratePrepareTunnel}->{vers} =3D "0.7.2"; - -$apis{virDomainMigrateBegin3}->{vers} =3D "0.9.2"; -$apis{virDomainMigratePrepare3}->{vers} =3D "0.9.2"; -$apis{virDomainMigratePrepareTunnel3}->{vers} =3D "0.9.2"; -$apis{virDomainMigratePerform3}->{vers} =3D "0.9.2"; -$apis{virDomainMigrateFinish3}->{vers} =3D "0.9.2"; -$apis{virDomainMigrateConfirm3}->{vers} =3D "0.9.2"; - -$apis{virDomainMigrateBegin3Params}->{vers} =3D "1.1.0"; -$apis{virDomainMigratePrepare3Params}->{vers} =3D "1.1.0"; -$apis{virDomainMigratePrepareTunnel3Params}->{vers} =3D "1.1.0"; -$apis{virDomainMigratePerform3Params}->{vers} =3D "1.1.0"; -$apis{virDomainMigrateFinish3Params}->{vers} =3D "1.1.0"; -$apis{virDomainMigrateConfirm3Params}->{vers} =3D "1.1.0"; - - - -# Now we want to get the mapping between public APIs -# and driver struct fields. This lets us later match -# update the driver impls with the public APis. - -my $line; - -# Group name -> hash of APIs { fields -> api name } -my %groups; -my $ingrp; -foreach my $drivertable (@drivertable) { - open FILE, "<$drivertable" - or die "cannot read $drivertable: $!"; - - while (defined($line =3D )) { - if ($line =3D~ /struct _(vir\w*Driver)/) { - my $grp =3D $1; - if ($grp ne "virStateDriver" && - $grp ne "virStreamDriver") { - $ingrp =3D $grp; - $groups{$ingrp} =3D { apis =3D> {}, drivers =3D> {} }; - } - } elsif ($ingrp) { - if ($line =3D~ /^\s*vir(?:Drv)(\w+)\s+(\w+);\s*$/) { - my $field =3D $2; - my $name =3D $1; - - my $api; - if (exists $apis{"vir$name"}) { - $api =3D "vir$name"; - } elsif ($name =3D~ /\w+(Open|Close|URIProbe)/) { - next; - } else { - die "driver $name does not have a public API"; - } - $groups{$ingrp}->{apis}->{$field} =3D $api; - } elsif ($line =3D~ /};/) { - $ingrp =3D undef; - } - } - } - - close FILE; -} - - -# Finally, we read all the primary driver files and extract -# the driver API tables from each one. - -foreach my $src (@srcs) { - open FILE, "<$src" or - die "cannot read $src: $!"; - - my $groups_regex =3D join("|", keys %groups); - $ingrp =3D undef; - my $impl; - while (defined($line =3D )) { - if (!$ingrp) { - #=C2=A0skip non-matching lines early to save time - next if not $line =3D~ /$groups_regex/; - - if ($line =3D~ /^\s*(?:static\s+)?($groups_regex)\s+(\w+)\s*= =3D\s*{/ || - $line =3D~ /^\s*(?:static\s+)?($groups_regex)\s+NAME\(\w+\= )\s*=3D\s*{/) { - $ingrp =3D $1; - $impl =3D $src; - - if ($impl =3D~ m,.*/node_device_(\w+)\.c,) { - $impl =3D $1; - } else { - $impl =3D~ s,.*/(\w+?)_((\w+)_)?(\w+)\.c,$1,; - } - - if ($groups{$ingrp}->{drivers}->{$impl}) { - die "Group $ingrp already contains $impl"; - } - - $groups{$ingrp}->{drivers}->{$impl} =3D {}; - } - - } else { - if ($line =3D~ m!\s*\.(\w+)\s*=3D\s*(\w+)\s*,?\s*(?:/\*\s*(\d+= \.\d+\.\d+)\s*(?:-\s*(\d+\.\d+\.\d+))?\s*\*/\s*)?$!) { - my $api =3D $1; - my $meth =3D $2; - my $vers =3D $3; - my $deleted =3D $4; - - next if $api eq "no" || $api eq "name"; - - if ($meth eq "NULL" && !defined $deleted) { - die "Method impl for $api is NULL, but no deleted vers= ion is provided"; - } - if ($meth ne "NULL" && defined $deleted) { - die "Method impl for $api is non-NULL, but deleted ver= sion is provided"; - } - - die "Method $meth in $src is missing version" unless defin= ed $vers || $api eq "connectURIProbe"; - - if (!exists($groups{$ingrp}->{apis}->{$api})) { - next if $api =3D~ /\w(Open|Close|URIProbe)/; - - die "Found unexpected method $api in $ingrp\n"; - } - - $groups{$ingrp}->{drivers}->{$impl}->{$api} =3D {}; - $groups{$ingrp}->{drivers}->{$impl}->{$api}->{vers} =3D $v= ers; - $groups{$ingrp}->{drivers}->{$impl}->{$api}->{deleted} = =3D $deleted; - if ($api eq "domainMigratePrepare" || - $api eq "domainMigratePrepare2" || - $api eq "domainMigratePrepare3") { - if (!$groups{$ingrp}->{drivers}->{$impl}->{"domainMigr= ate"}) { - $groups{$ingrp}->{drivers}->{$impl}->{"domainMigra= te"} =3D {}; - $groups{$ingrp}->{drivers}->{$impl}->{"domainMigra= te"}->{vers} =3D $vers; - } - } - - } elsif ($line =3D~ /}/) { - $ingrp =3D undef; - } - } - } - - close FILE; -} - - -# The '.open' driver method is used for 3 public APIs, so we -# have a bit of manual fixup todo with the per-driver versioning -# and support matrix - -$groups{virHypervisorDriver}->{apis}->{"openAuth"} =3D "virConnectOpenAuth= "; -$groups{virHypervisorDriver}->{apis}->{"openReadOnly"} =3D "virConnectOpen= ReadOnly"; -$groups{virHypervisorDriver}->{apis}->{"domainMigrate"} =3D "virDomainMigr= ate"; - -my $openAuthVers =3D (0 * 1000 * 1000) + (4 * 1000) + 0; - -foreach my $drv (keys %{$groups{"virHypervisorDriver"}->{drivers}}) { - my $openVersStr =3D $groups{"virHypervisorDriver"}->{drivers}->{$drv}-= >{"connectOpen"}->{vers}; - my $openVers; - if ($openVersStr =3D~ /(\d+)\.(\d+)\.(\d+)/) { - $openVers =3D ($1 * 1000 * 1000) + ($2 * 1000) + $3; - } - - # virConnectOpenReadOnly always matches virConnectOpen version - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"connectOpenReadOn= ly"} =3D - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"connectOpen"}; - - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"connectOpenAuth"}= =3D {}; - - # virConnectOpenAuth is always 0.4.0 if the driver existed - # before this time, otherwise it matches the version of - # the driver's virConnectOpen entry - if ($openVersStr eq "Y" || - $openVers >=3D $openAuthVers) { - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"connectOpenAu= th"}->{vers} =3D $openVersStr; - } else { - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"connectOpenAu= th"}->{vers} =3D "0.4.0"; - } -} - - -# Another special case for the virDomainCreateLinux which was replaced -# with virDomainCreateXML -$groups{virHypervisorDriver}->{apis}->{"domainCreateLinux"} =3D "virDomain= CreateLinux"; - -my $createAPIVers =3D (0 * 1000 * 1000) + (0 * 1000) + 3; - -foreach my $drv (keys %{$groups{"virHypervisorDriver"}->{drivers}}) { - my $createVersStr =3D $groups{"virHypervisorDriver"}->{drivers}->{$drv= }->{"domainCreateXML"}->{vers}; - next unless defined $createVersStr; - my $createVers; - if ($createVersStr =3D~ /(\d+)\.(\d+)\.(\d+)/) { - $createVers =3D ($1 * 1000 * 1000) + ($2 * 1000) + $3; - } - - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"domainCreateLinux= "} =3D {}; - - # virCreateLinux is always 0.0.3 if the driver existed - # before this time, otherwise it matches the version of - # the driver's virCreateXML entry - if ($createVersStr eq "Y" || - $createVers >=3D $createAPIVers) { - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"domainCreateL= inux"}->{vers} =3D $createVersStr; - } else { - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"domainCreateL= inux"}->{vers} =3D "0.0.3"; - } -} - - -# Finally we generate the HTML file with the tables - -print < - - - -

libvirt API support matrix

- -
    - -

    -This page documents which libvirt calls work on -which libvirt drivers / hypervisors, and which version the API appeared -in. If a hypervisor driver later dropped support for the API, the version -when it was removed is also mentioned (highlighted in -dark red). -

    - -EOF - - foreach my $grp (sort { $a cmp $b } keys %groups) { - print "

    ", $groupheaders{$grp}, "

    \n"; - print < - - -API -Version -EOF - - foreach my $drv (sort { $a cmp $b } keys %{$groups{$grp}->{drivers}}) { - print " $drv\n"; - } - - print < - - -EOF - - my $row =3D 0; - foreach my $field (sort { - $groups{$grp}->{apis}->{$a} - cmp - $groups{$grp}->{apis}->{$b} - } keys %{$groups{$grp}->{apis}}) { - my $api =3D $groups{$grp}->{apis}->{$field}; - my $vers =3D $apis{$api}->{vers}; - my $htmlgrp =3D $apis{$api}->{file}; - print < - -EOF - - if (defined $htmlgrp) { - print <$api -EOF - - } else { - print $api; - } - print < -$vers -EOF - - foreach my $drv (sort {$a cmp $b } keys %{$groups{$grp}->{drivers}= }) { - print ""; - if (exists $groups{$grp}->{drivers}->{$drv}->{$field}) { - if ($groups{$grp}->{drivers}->{$drv}->{$field}->{vers}) { - print $groups{$grp}->{drivers}->{$drv}->{$field}->{ver= s}; - } - if ($groups{$grp}->{drivers}->{$drv}->{$field}->{deleted})= { - print " - ", $groups{$grp}= ->{drivers}->{$drv}->{$field}->{deleted}, ""; - } - } - print "\n"; - } - - print < -EOF - - $row++; - if (($row % 15) =3D=3D 0) { - print < -API -Version -EOF - - foreach my $drv (sort { $a cmp $b } keys %{$groups{$grp}->{dri= vers}}) { - print " $drv\n"; - } - - print < -EOF - } - - } - - print < - -EOF -} - -print < - -EOF diff --git a/scripts/hvsupport.py b/scripts/hvsupport.py new file mode 100755 index 0000000000..14c41da348 --- /dev/null +++ b/scripts/hvsupport.py @@ -0,0 +1,502 @@ +#!/usr/bin/env python3 +# +# Copyright (C) 2011-2019 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . + +import sys +import os.path +import re + +if len(sys.argv) !=3D 3: + print("syntax: %s TOP-SRCDIR TOP-BUILDDIR\n" % sys.argv[0], file=3Dsys= .stderr) + +srcdir =3D sys.argv[1] +builddir =3D sys.argv[2] + +symslibvirt =3D os.path.join(srcdir, "src", "libvirt_public.syms") +symsqemu =3D os.path.join(srcdir, "src", "libvirt_qemu.syms") +symslxc =3D os.path.join(srcdir, "src", "libvirt_lxc.syms") +drivertablefiles =3D [ + os.path.join(srcdir, "src", "driver-hypervisor.h"), + os.path.join(srcdir, "src", "driver-interface.h"), + os.path.join(srcdir, "src", "driver-network.h"), + os.path.join(srcdir, "src", "driver-nodedev.h"), + os.path.join(srcdir, "src", "driver-nwfilter.h"), + os.path.join(srcdir, "src", "driver-secret.h"), + os.path.join(srcdir, "src", "driver-state.h"), + os.path.join(srcdir, "src", "driver-storage.h"), + os.path.join(srcdir, "src", "driver-stream.h"), +] + +groupheaders =3D { + "virHypervisorDriver": "Hypervisor APIs", + "virNetworkDriver": "Virtual Network APIs", + "virInterfaceDriver": "Host Interface APIs", + "virNodeDeviceDriver": "Host Device APIs", + "virStorageDriver": "Storage Pool APIs", + "virSecretDriver": "Secret APIs", + "virNWFilterDriver": "Network Filter APIs", +} + + +srcs =3D [] +for root, dirs, files in os.walk(os.path.join(srcdir, "src")): + for file in files: + if ((file.endswith("driver.c") and + not file.endswith("vbox_driver.c")) or + file.endswith("common.c") or + file.endswith("tmpl.c") or + file.endswith("monitor.c") or + file.endswith("hal.c") or + file.endswith("udev.c")): + srcs.append(os.path.join(root, file)) + + +# Map API functions to the header and documentation files they're in +# so that we can generate proper hyperlinks to their documentation. +# +# The function names are grep'd from the XML output of apibuild.py. +def getAPIFilenames(filename): + files =3D {} + + with open(filename) as fh: + for line in fh: + res =3D re.search(r"function name=3D'([^']+)' file=3D'([^']+)'= ", line) + if res is not None: + files[res.group(1)] =3D res.group(2) + + if len(files) =3D=3D 0: + raise Exception(("No functions found in %s. " + + "Has the apibuild.py output changed?") % + filename) + + return files + + +def parseSymsFile(apisref, prefix, filename, xmlfilename): + vers =3D None + prevvers =3D None + + filenames =3D getAPIFilenames(xmlfilename) + + with open(filename) as fh: + for line in fh: + line =3D line.strip() + + if line =3D=3D "": + continue + if line[0] =3D=3D '#': + continue + if line.startswith("global:"): + continue + if line.startswith("local:"): + continue + + groupstartmatch =3D re.search(r"^\s*%s_(\d+\.\d+\.\d+)\s*{\s*$= " % + prefix, line) + groupendmatch1 =3D re.search(r"^\s*}\s*;\s*$", line) + groupendmatch2 =3D re.search(r"^\s*}\s*%s_(\d+\.\d+\.\d+)\s*;\= s*$" % + prefix, line) + symbolmatch =3D re.search(r"^\s*(\w+)\s*;\s*$", line) + if groupstartmatch is not None: + if vers is not None: + raise Exception("malformed syms file when starting gro= up") + + vers =3D groupstartmatch.group(1) + elif groupendmatch1 is not None: + if prevvers is not None: + raise Exception("malformed syms file when ending group= ") + + prevvers =3D vers + vers =3D None + elif groupendmatch2 is not None: + if groupendmatch2.group(1) !=3D prevvers: + raise Exception(("malformed syms file %s !=3D %s " + + "when ending group") % + (groupendmatch2.group(1), prevvers)) + + prevvers =3D vers + vers =3D None + elif symbolmatch is not None: + name =3D symbolmatch.group(1) + apisref[name] =3D { + "vers": vers, + "file": filenames.get(name), + } + else: + raise Exception("unexpected data %s" % line) + + +apis =3D {} +# Get the list of all public APIs and their corresponding version +parseSymsFile(apis, "LIBVIRT", symslibvirt, + os.path.join(builddir, "docs", "libvirt-api.xml")) + +# And the same for the QEMU specific APIs +parseSymsFile(apis, "LIBVIRT_QEMU", symsqemu, + os.path.join(builddir, "docs", "libvirt-qemu-api.xml")) + +# And the same for the LXC specific APIs +parseSymsFile(apis, "LIBVIRT_LXC", symslxc, + os.path.join(builddir, "docs", "libvirt-lxc-api.xml")) + + +# Some special things which aren't public APIs, +# but we want to report +apis["virConnectSupportsFeature"] =3D { + "vers": "0.3.2" +} +apis["virDomainMigratePrepare"] =3D { + "vers": "0.3.2" +} +apis["virDomainMigratePerform"] =3D { + "vers": "0.3.2" +} +apis["virDomainMigrateFinish"] =3D { + "vers": "0.3.2" +} +apis["virDomainMigratePrepare2"] =3D { + "vers": "0.5.0" +} +apis["virDomainMigrateFinish2"] =3D { + "vers": "0.5.0" +} +apis["virDomainMigratePrepareTunnel"] =3D { + "vers": "0.7.2" +} + +apis["virDomainMigrateBegin3"] =3D { + "vers": "0.9.2" +} +apis["virDomainMigratePrepare3"] =3D { + "vers": "0.9.2" +} +apis["virDomainMigratePrepareTunnel3"] =3D { + "vers": "0.9.2" +} +apis["virDomainMigratePerform3"] =3D { + "vers": "0.9.2" +} +apis["virDomainMigrateFinish3"] =3D { + "vers": "0.9.2" +} +apis["virDomainMigrateConfirm3"] =3D { + "vers": "0.9.2" +} + +apis["virDomainMigrateBegin3Params"] =3D { + "vers": "1.1.0" +} +apis["virDomainMigratePrepare3Params"] =3D { + "vers": "1.1.0" +} +apis["virDomainMigratePrepareTunnel3Params"] =3D { + "vers": "1.1.0" +} +apis["virDomainMigratePerform3Params"] =3D { + "vers": "1.1.0" +} +apis["virDomainMigrateFinish3Params"] =3D { + "vers": "1.1.0" +} +apis["virDomainMigrateConfirm3Params"] =3D { + "vers": "1.1.0" +} + + +# Now we want to get the mapping between public APIs +# and driver struct fields. This lets us later match +# update the driver impls with the public APis. + +# Group name -> hash of APIs { fields -> api name } +groups =3D {} +ingrp =3D None +for drivertablefile in drivertablefiles: + with open(drivertablefile) as fh: + for line in fh: + starttablematch =3D re.search(r"struct _(vir\w*Driver)", line) + if starttablematch is not None: + grp =3D starttablematch.group(1) + if grp !=3D "virStateDriver" and grp !=3D "virStreamDriver= ": + ingrp =3D grp + groups[ingrp] =3D { + "apis": {}, + "drivers": {} + } + elif ingrp is not None: + callbackmatch =3D re.search(r"^\s*vir(?:Drv)(\w+)\s+(\w+);= \s*$", + line) + if callbackmatch is not None: + name =3D callbackmatch.group(1) + field =3D callbackmatch.group(2) + + api =3D "vir" + name + if api in apis: + groups[ingrp]["apis"][field] =3D api + elif re.search(r"\w+(Open|Close|URIProbe)", api) is no= t None: + continue + else: + raise Exception(("driver %s does not have " + + "a public API") % name) + elif re.search(r"};", line): + ingrp =3D None + + +# Finally, we read all the primary driver files and extract +# the driver API tables from each one. + +for src in srcs: + with open(src) as fh: + groupsre =3D "|".join(groups.keys()) + + ingrp =3D None + impl =3D None + for line in fh: + if ingrp is None: + m =3D re.search(r"^\s*(static\s+)?(" + + groupsre + + r")\s+(\w+)\s*=3D\s*{", line) + if m is None: + m =3D re.search(r"^\s*(static\s+)?(" + + groupsre + + r")\s+NAME\(\w+\)\s*=3D\s*{", line) + if m is not None: + ingrp =3D m.group(2) + impl =3D src + + implmatch =3D re.search(r".*/node_device_(\w+)\.c", im= pl) + if implmatch is None: + implmatch =3D re.search(r".*/(\w+?)_((\w+)_)?(\w+)= \.c", impl) + if implmatch is None: + raise Exception("Unexpected impl format '%s'" % im= pl) + impl =3D implmatch.group(1) + + if impl in groups[ingrp]["drivers"]: + raise Exception( + "Group %s already contains %s" % (ingrp, impl)) + + groups[ingrp]["drivers"][impl] =3D {} + else: + callbackmatch =3D re.search(r"\s*\.(\w+)\s*=3D\s*(\w+)\s*,= ?\s*" + + r"(?:/\*\s*(\d+\.\d+\.\d+)\s*" + + r"(?:-\s*(\d+\.\d+\.\d+))?\s*\*/= \s*)?$", + line) + if callbackmatch is not None: + api =3D callbackmatch.group(1) + meth =3D callbackmatch.group(2) + vers =3D callbackmatch.group(3) + deleted =3D callbackmatch.group(4) + + if api =3D=3D "no" or api =3D=3D "name": + continue + + if meth =3D=3D "NULL" and deleted is None: + raise Exception( + ("Method impl for %s is NULL, but " + + "no deleted version is provided") % api) + + if meth !=3D "NULL" and deleted is not None: + raise Exception( + ("Method impl for %s is non-NULL, but " + + "deleted version is provided") % api) + + if vers is None and api !=3D "connectURIProbe": + raise Exception( + "Method %s in %s is missing version" % + (meth, src)) + + if api not in groups[ingrp]["apis"]: + if re.search(r"\w+(Open|Close|URIProbe)", api): + continue + + raise Exception("Found unexpected method " + + "%s in %s" % (api, ingrp)) + + groups[ingrp]["drivers"][impl][api] =3D { + "vers": vers, + "deleted": deleted, + } + + if (api =3D=3D "domainMigratePrepare" or + api =3D=3D "domainMigratePrepare2" or + api =3D=3D "domainMigratePrepare3"): + if ("domainMigrate" not in + groups[ingrp]["drivers"][impl]): + groups[ingrp]["drivers"][impl]["domainMigrate"= ] =3D { + "vers": vers, + } + elif line.find("}") !=3D -1: + ingrp =3D None + + +# The '.open' driver method is used for 3 public APIs, so we +# have a bit of manual fixup todo with the per-driver versioning +# and support matrix + +groups["virHypervisorDriver"]["apis"]["openAuth"] =3D \ + "virConnectOpenAuth" +groups["virHypervisorDriver"]["apis"]["openReadOnly"] =3D \ + "virConnectOpenReadOnly" +groups["virHypervisorDriver"]["apis"]["domainMigrate"] =3D \ + "virDomainMigrate" + +openAuthVers =3D (0 * 1000 * 1000) + (4 * 1000) + 0 + +drivers =3D groups["virHypervisorDriver"]["drivers"] +for drv in drivers.keys(): + openVersStr =3D drivers[drv]["connectOpen"]["vers"] + openVers =3D 0 + if openVersStr !=3D "Y": + openVersBits =3D openVersStr.split(".") + if len(openVersBits) !=3D 3: + raise Exception("Expected 3 digit version for %s" % openVersSt= r) + openVers =3D ((int(openVersBits[0]) * 1000 * 1000) + + (int(openVersBits[1]) * 1000) + + int(openVersBits[2])) + + # virConnectOpenReadOnly always matches virConnectOpen version + drivers[drv]["connectOpenReadOnly"] =3D \ + drivers[drv]["connectOpen"] + + # virConnectOpenAuth is always 0.4.0 if the driver existed + # before this time, otherwise it matches the version of + # the driver's virConnectOpen entry + if openVersStr =3D=3D "Y" or openVers >=3D openAuthVers: + vers =3D openVersStr + else: + vers =3D "0.4.0" + drivers[drv]["connectOpenAuth"] =3D { + "vers": vers, + } + + +# Another special case for the virDomainCreateLinux which was replaced +# with virDomainCreateXML +groups["virHypervisorDriver"]["apis"]["domainCreateLinux"] =3D \ + "virDomainCreateLinux" + +createAPIVers =3D (0 * 1000 * 1000) + (0 * 1000) + 3 + +for drv in drivers.keys(): + if "domainCreateXML" not in drivers[drv]: + continue + createVersStr =3D drivers[drv]["domainCreateXML"]["vers"] + createVers =3D 0 + if createVersStr !=3D "Y": + createVersBits =3D createVersStr.split(".") + if len(createVersBits) !=3D 3: + raise Exception("Expected 3 digit version for %s" % createVers= Str) + createVers =3D ((int(createVersBits[0]) * 1000 * 1000) + + (int(createVersBits[1]) * 1000) + + int(createVersBits[2])) + + # virCreateLinux is always 0.0.3 if the driver existed + # before this time, otherwise it matches the version of + # the driver's virCreateXML entry + if createVersStr =3D=3D "Y" or createVers >=3D createAPIVers: + vers =3D createVersStr + else: + vers =3D "0.0.3" + + drivers[drv]["domainCreateLinux"] =3D { + "vers": vers, + } + + +# Finally we generate the HTML file with the tables + +print(''' + + + +

    libvirt API support matrix

    + +
      + +

      +This page documents which libvirt calls work on +which libvirt drivers / hypervisors, and which version the API appeared +in. If a hypervisor driver later dropped support for the API, the version +when it was removed is also mentioned (highlighted in +dark red). +

      +''') + +for grp in sorted(groups.keys()): + print("

      %s

      " % (grp, groupheaders[grp])) + print(''' + + + +''') + + for drv in sorted(groups[grp]["drivers"].keys()): + print(" " % drv) + + print(''' + +''') + + row =3D 0 + + def sortkey(field): + return groups[grp]["apis"][field] + + for field in sorted(groups[grp]["apis"].keys(), key=3Dsortkey): + api =3D groups[grp]["apis"][field] + vers =3D apis[api]["vers"] + htmlgrp =3D apis[api].get("file") + print("") + + if htmlgrp is not None: + print(('''''') % + (htmlgrp, api, api)) + else: + print("" % api) + + print("" % vers) + + for drv in sorted(groups[grp]["drivers"].keys()): + info =3D "" + if field in groups[grp]["drivers"][drv]: + vers =3D groups[grp]["drivers"][drv][field]["vers"] + if vers is not None: + info =3D info + vers + + deleted =3D groups[grp]["drivers"][drv][field].get("delete= d") + if deleted is not None: + info =3D info + (''' - ''' + + '''%s''' % deleted) + + print("" % info) + + print("") + + row =3D row + 1 + if (row % 15) =3D=3D 0: + print(''' + +''') + + for drv in sorted(groups[grp]["drivers"].keys()): + print(" " % drv) + + print("") + + print("\n
      APIVersion%s
      \n''' + + '''%s\n\n%s%s%s
      APIVersion%s
      ") + +print("\n") --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sun May 5 10:21:35 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1575912150; cv=none; d=zohomail.com; s=zohoarc; b=EbJFhurMaudR2ujoN04H0P+o6A+rI7i9IppOXl9XZ70MtZHNusDPM55L0CtwdfaYypcTYK/bswzEfaBtgKriUgir6zAyRqXE35ihpIL6f1R1QpRC0EkVOFS2mu4ZOGmN0CUQqj95N7uApqKzRl6FVpwAz6s9zvVNx6phSigkHU8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1575912150; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=6D8Ynlbr7o3OtABGM32LTszM8aRRPstfCHhewFtCmuQ=; b=Yhu8vzVkFPjsREAHNSn3AQqiBNMZmaSixcfWO1aY5bNX4t1a/Cyv/WkXpptn03tOQ3GEi6Tota3LY+5jwN5dh0FBwbDopDXz+8yVGghwliqvLdvJdoeIdItl4GhvUuu2Oc8QmNBRd8CfM61DY7vL0aPnPdSO/09EBhta4gBlTls= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 1575912150194634.6372960378169; Mon, 9 Dec 2019 09:22:30 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-37-tmmTyKk3OJWS-nBj40yHpQ-1; Mon, 09 Dec 2019 12:22:23 -0500 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3D2D4100A169; Mon, 9 Dec 2019 17:22:18 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9A82160BE0; Mon, 9 Dec 2019 17:22:17 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 4AE7E83558; Mon, 9 Dec 2019 17:22:17 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx05.intmail.prod.int.phx2.redhat.com [10.5.11.15]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xB9HLbRm015146 for ; Mon, 9 Dec 2019 12:21:37 -0500 Received: by smtp.corp.redhat.com (Postfix) id 8D1135D6C5; Mon, 9 Dec 2019 17:21:37 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id CCDDA5D6B7; Mon, 9 Dec 2019 17:21:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1575912149; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=6D8Ynlbr7o3OtABGM32LTszM8aRRPstfCHhewFtCmuQ=; b=AwrP7PtDm1PnqgT/N+K91IRDtCwNeticjd1eHZKmfxsGYQ7V7HjoSOK5amo2ep6Tzd1het 953fWBHOJaG8mIHDl7YZqLIvs8MOVKQW231hnOLo7/6kI4p9+jqrM74FTB1Hu7hD4w5ECw gKFM9BR6zwDuLw2TUIMLZnQ7gmaJ95Q= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 9 Dec 2019 17:14:22 +0000 Message-Id: <20191209171422.31091-5-berrange@redhat.com> In-Reply-To: <20191209171422.31091-1-berrange@redhat.com> References: <20191209171422.31091-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.15 X-loop: libvir-list@redhat.com Cc: Cole Robinson Subject: [libvirt] [PATCH v6 4/4] docs: rewrite polkit docs generator in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-MC-Unique: tmmTyKk3OJWS-nBj40yHpQ-1 X-Mimecast-Spam-Score: 0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) As part of a goal to eliminate Perl from libvirt build tools, rewrite the genaclperms.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. Tested-by: Cole Robinson Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: Cole Robinson --- Makefile.am | 1 + docs/Makefile.am | 5 +- docs/genaclperms.pl | 125 ----------------------------------------- scripts/genaclperms.py | 121 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 124 insertions(+), 128 deletions(-) delete mode 100755 docs/genaclperms.pl create mode 100755 scripts/genaclperms.py diff --git a/Makefile.am b/Makefile.am index 51ef567d3b..632281d9b6 100644 --- a/Makefile.am +++ b/Makefile.am @@ -58,6 +58,7 @@ EXTRA_DIST =3D \ scripts/check-symsorting.py \ scripts/dtrace2systemtap.py \ scripts/esx_vi_generator.py \ + scripts/genaclperms.py \ scripts/genpolkit.py \ scripts/gensystemtap.py \ scripts/group-qemu-caps.py \ diff --git a/docs/Makefile.am b/docs/Makefile.am index 18d17b3727..eb17da4299 100644 --- a/docs/Makefile.am +++ b/docs/Makefile.am @@ -236,7 +236,6 @@ schemadir =3D $(pkgdatadir)/schemas schema_DATA =3D $(wildcard $(srcdir)/schemas/*.rng) =20 EXTRA_DIST=3D \ - genaclperms.pl \ site.xsl subsite.xsl newapi.xsl page.xsl \ wrapstring.xsl \ $(dot_html_in) $(dot_rst) $(gif) $(apipng) \ @@ -250,8 +249,8 @@ EXTRA_DIST=3D \ acl_generated =3D aclperms.htmlinc =20 aclperms.htmlinc: $(top_srcdir)/src/access/viraccessperm.h \ - $(srcdir)/genaclperms.pl Makefile.am - $(AM_V_GEN)$(PERL) $(srcdir)/genaclperms.pl $< > $@ + $(top_srcdir)/scripts/genaclperms.py Makefile.am + $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(top_srcdir)/scripts/genaclperms.py $< >= $@ =20 CLEANFILES =3D \ $(dot_html) \ diff --git a/docs/genaclperms.pl b/docs/genaclperms.pl deleted file mode 100755 index 0de2cfad4d..0000000000 --- a/docs/genaclperms.pl +++ /dev/null @@ -1,125 +0,0 @@ -#!/usr/bin/env perl -# -# Copyright (C) 2013 Red Hat, Inc. -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library. If not, see -# . -# - -use strict; -use warnings; - -my @objects =3D ( - "CONNECT", "DOMAIN", "INTERFACE", - "NETWORK_PORT", "NETWORK", "NODE_DEVICE", - "NWFILTER_BINDING", "NWFILTER", - "SECRET", "STORAGE_POOL", "STORAGE_VOL", - ); - -my %class; - -foreach my $object (@objects) { - my $class =3D lc $object; - - $class =3D~ s/(^\w|_\w)/uc $1/eg; - $class =3D~ s/_//g; - $class =3D~ s/Nwfilter/NWFilter/; - $class =3D "vir" . $class . "Ptr"; - - $class{$object} =3D $class; -} - -my $objects =3D join ("|", @objects); - -my %opts; -my $in_opts =3D 0; - -my %perms; - -while (<>) { - if ($in_opts) { - if (m,\*/,) { - $in_opts =3D 0; - } elsif (/\*\s*\@(\w+):\s*(.*?)\s*$/) { - $opts{$1} =3D $2; - } - } elsif (m,/\*\*,) { - $in_opts =3D 1; - } elsif (/VIR_ACCESS_PERM_($objects)_((?:\w|_)+),/) { - my $object =3D $1; - my $perm =3D lc $2; - next if $perm eq "last"; - - $perm =3D~ s/_/-/g; - - $perms{$object} =3D {} unless exists $perms{$object}; - $perms{$object}->{$perm} =3D { - desc =3D> $opts{desc}, - message =3D> $opts{message}, - anonymous =3D> $opts{anonymous} - }; - %opts =3D (); - } -} - -print < - - - -EOF - -foreach my $object (sort { $a cmp $b } keys %perms) { - my $class =3D $class{$object}; - my $olink =3D lc "object_" . $object; - print <$class - - - - - - - - -EOF - - foreach my $perm (sort { $a cmp $b } keys %{$perms{$object}}) { - my $description =3D $perms{$object}->{$perm}->{desc}; - - die "missing description for $object.$perm" unless - defined $description; - - my $plink =3D lc "perm_" . $object . "_" . $perm; - $plink =3D~ s/-/_/g; - - print < - - - -EOF - - } - - print < -
      PermissionDescription
      $perm$description
      -EOF -} - -print < - -EOF diff --git a/scripts/genaclperms.py b/scripts/genaclperms.py new file mode 100755 index 0000000000..e228b3ef60 --- /dev/null +++ b/scripts/genaclperms.py @@ -0,0 +1,121 @@ +#!/usr/bin/env python3 +# +# Copyright (C) 2013-2019 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . +# + +import re +import sys + +objects =3D [ + "CONNECT", "DOMAIN", "INTERFACE", + "NETWORK_PORT", "NETWORK", "NODE_DEVICE", + "NWFILTER_BINDING", "NWFILTER", + "SECRET", "STORAGE_POOL", "STORAGE_VOL", +] + + +classes =3D {} + +for obj in objects: + klass =3D obj.lower() + + klass =3D re.sub(r'''(^\w|_\w)''', lambda a: a.group(1).upper(), klass) + klass =3D klass.replace("_", "") + klass =3D klass.replace("Nwfilter", "NWFilter") + klass =3D "vir" + klass + "Ptr" + + classes[obj] =3D klass + + +objectstr =3D "|".join(objects) + +opts =3D {} +in_opts =3D {} + +perms =3D {} + +aclfile =3D sys.argv[1] +with open(aclfile, "r") as fh: + for line in fh: + if in_opts: + if line.find("*/") !=3D -1: + in_opts =3D False + else: + m =3D re.search(r'''\*\s*\@(\w+):\s*(.*?)\s*$''', line) + if m is not None: + opts[m.group(1)] =3D m.group(2) + elif line.find("**") !=3D -1: + in_opts =3D True + else: + m =3D re.search(r'''VIR_ACCESS_PERM_(%s)_((?:\w|_)+),''' % + objectstr, line) + if m is not None: + obj =3D m.group(1) + perm =3D m.group(2).lower() + if perm =3D=3D "last": + continue + + perm =3D perm.replace("_", "-") + + if obj not in perms: + perms[obj] =3D {} + perms[obj][perm] =3D { + "desc": opts.get("desc", None), + "message": opts.get("message", None), + "anonymous": opts.get("anonymous", None), + } + opts =3D {} + +print('') +print('') +print('') +print(' ') + +for obj in sorted(perms.keys()): + klass =3D classes[obj] + + olink =3D "object_" + obj.lower() + + print('

      %s

      ' % (olink, klass)) + print(' ') + print(' ') + print(' ') + print(' ') + print(' ') + print(' ') + print(' ') + print(' ') + + for perm in sorted(perms[obj].keys()): + description =3D perms[obj][perm]["desc"] + + if description is None: + raise Exception("missing description for %s.%s" % (obj, perm)) + + plink =3D "perm_" + obj.lower() + "_" + perm.lower() + plink =3D plink.replace("-", "_") + + print(' ') + print(' ' % (plink, perm)) + print(' ' % description) + print(' ') + + print(' ') + print('
      PermissionDescription
      %s%s
      ') + +print(' ') +print('') --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list