From nobody Sun May 5 18:51:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 207.211.31.120 as permitted sender) client-ip=207.211.31.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1573483501; cv=none; d=zoho.com; s=zohoarc; b=RubaNU1g4ivjz8myYa1BJwEYsT3mdz4GIX35aV6LZeNgs59rGwJu5yWiKp6F2ms6ArDdzBFywtOmELR1xAnDBzoyRXDGNIoJw+8rxA50nVo33pSkcECZJoTgoLeFS8JvUoLmE70fOuPoalDiExBVBkGOppji3b43jtBMdpV6NtA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573483501; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=LhCTcip87gsYFjdWas/l0qFX4wjUtGlRMGLrv8Zooh8=; b=IZemnJe0ZERI5NRKItN1dm5ofWxEW8rgVvKJ17Ip7pUBp5j2C5cQ7Sm7lyCm0MRY28Kqsnpwa1I1amLFrx3embt1MVdKd8e9zwGJMDy0/SBf29p7j2K865X7OQZyPNKqU5BekaLKWrm0mxB8gi1EcirqSFRZ4+Wrd2Sb8nd3Hfg= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by mx.zohomail.com with SMTPS id 1573483501593595.293180778161; Mon, 11 Nov 2019 06:45:01 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-259-7_lMeCqhNFKVwcbSHq4wWQ-1; Mon, 11 Nov 2019 09:44:58 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 4D67B18B5F96; Mon, 11 Nov 2019 14:44:53 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id BD1AB100EBA7; Mon, 11 Nov 2019 14:44:52 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 7B6F018034EA; Mon, 11 Nov 2019 14:44:51 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xABEiopC020333 for ; Mon, 11 Nov 2019 09:44:50 -0500 Received: by smtp.corp.redhat.com (Postfix) id 53AC75D9C9; Mon, 11 Nov 2019 14:44:50 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id B90F919023; Mon, 11 Nov 2019 14:44:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573483500; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=LhCTcip87gsYFjdWas/l0qFX4wjUtGlRMGLrv8Zooh8=; b=PPA/5f8y6oOGkzmHWzwb4wC53/VMd/McK3pDsZVEWsJpAloSmjbq0tdfHKDQ3tgjpX0sw7 S8s3JROpFKICjKc8okDRC6CxjZLtHmimtNLSqSxfv5o0l9QFpfV6z4gmIrG5PnrxhIAYwO lB9BtIckd2zYfkH9aWMMsk++1NdeiMw= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 11 Nov 2019 14:38:04 +0000 Message-Id: <20191111143826.16050-2-berrange@redhat.com> In-Reply-To: <20191111143826.16050-1-berrange@redhat.com> References: <20191111143826.16050-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v5 01/23] build-aux: rewrite duplicate header checker in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: 7_lMeCqhNFKVwcbSHq4wWQ-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" As part of an goal to eliminate Perl from libvirt build tools, rewrite the prohibit-duplicate-header.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: J=C3=A1n Tomko --- Makefile.am | 2 +- build-aux/prohibit-duplicate-header.pl | 26 ------------ build-aux/syntax-check.mk | 4 +- scripts/prohibit-duplicate-header.py | 56 ++++++++++++++++++++++++++ 4 files changed, 59 insertions(+), 29 deletions(-) delete mode 100644 build-aux/prohibit-duplicate-header.pl create mode 100644 scripts/prohibit-duplicate-header.py diff --git a/Makefile.am b/Makefile.am index f758745d91..9471cf7117 100644 --- a/Makefile.am +++ b/Makefile.am @@ -50,7 +50,7 @@ EXTRA_DIST =3D \ build-aux/header-ifdef.pl \ scripts/minimize-po.py \ build-aux/mock-noinline.pl \ - build-aux/prohibit-duplicate-header.pl \ + scripts/prohibit-duplicate-header.py \ build-aux/syntax-check.mk \ build-aux/useless-if-before-free \ build-aux/vc-list-files \ diff --git a/build-aux/prohibit-duplicate-header.pl b/build-aux/prohibit-du= plicate-header.pl deleted file mode 100644 index 4a2ea65665..0000000000 --- a/build-aux/prohibit-duplicate-header.pl +++ /dev/null @@ -1,26 +0,0 @@ -#!/usr/bin/env perl - -use strict; - -my $file =3D " "; -my $ret =3D 0; -my %includes =3D ( ); -my $lineno =3D 0; - -while (<>) { - if (not $file eq $ARGV) { - %includes =3D ( ); - $file =3D $ARGV; - $lineno =3D 0; - } - $lineno++; - if (/^# *include *[<"]([^>"]*\.h)[">]/) { - $includes{$1}++; - if ($includes{$1} =3D=3D 2) { - $ret =3D 1; - print STDERR "$ARGV:$lineno: $_"; - print STDERR "Do not include a header more than once per file\= n"; - } - } -} -exit $ret; diff --git a/build-aux/syntax-check.mk b/build-aux/syntax-check.mk index 427f12fd24..9b6c157029 100644 --- a/build-aux/syntax-check.mk +++ b/build-aux/syntax-check.mk @@ -2153,8 +2153,8 @@ endif =20 # Don't include duplicate header in the source (either *.c or *.h) prohibit-duplicate-header: - $(AM_V_GEN)$(VC_LIST_EXCEPT) | $(GREP) '\.[chx]$$' | xargs \ - $(PERL) -W $(top_srcdir)/build-aux/prohibit-duplicate-header.pl + $(AM_V_GEN)$(VC_LIST_EXCEPT) | $(GREP) '\.[chx]$$' | $(RUNUTF8) xargs \ + $(PYTHON) $(top_srcdir)/scripts/prohibit-duplicate-header.py =20 spacing-check: $(AM_V_GEN)$(VC_LIST) | $(GREP) '\.c$$' | xargs \ diff --git a/scripts/prohibit-duplicate-header.py b/scripts/prohibit-duplic= ate-header.py new file mode 100644 index 0000000000..dfdfa0bf0b --- /dev/null +++ b/scripts/prohibit-duplicate-header.py @@ -0,0 +1,56 @@ +#!/usr/bin/env python +# +# Copyright (C) 2016-2019 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . + +from __future__ import print_function + +import re +import sys + + +def check_file(filename): + includes =3D {} + lineno =3D 0 + errs =3D False + with open(filename, "r") as fh: + for line in fh: + lineno =3D lineno + 1 + + headermatch =3D re.search(r'''^# *include *[<"]([^>"]*\.h)[">]= ''', line) + if headermatch is not None: + inc =3D headermatch.group(1) + + if inc in includes: + print("%s:%d: %s" % (filename, lineno, inc), + file=3Dsys.stderr) + errs =3D True + else: + includes[inc] =3D True + + return errs + + +ret =3D 0 + +for filename in sys.argv[1:]: + if check_file(filename): + ret =3D 1 + +if ret =3D=3D 1: + print("Do not include a header more than once per file", file=3Dsys.st= derr) + +sys.exit(ret) --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sun May 5 18:51:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1573483502; cv=none; d=zoho.com; s=zohoarc; b=bbQNfsZM0L6MGeRURESuSL5xxz1iHPlcINEgrKthdXzVz0i5woO8HgpJsxf7TrJRGLmZQp7T7ZUgJGCGUD7JLp2mX92RfQSDAox3hEWpBUwNGC2dPhE7/mbZ+puWbh/+sSDso4mZ3+6iY+7dsaMhjoYALd5ARkej+EqcLIPt+Jk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573483502; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=F++An/cEk5/qluGGFPFy84SPBL6PLYpKqn8eCptdGno=; b=K+15WqAyti1CuH9KGYPmwE86Sx6fCuT0MXTWEvmZuWnTfLSk9eOeMgLPJE0sldoiWmBoS7h13MsLy736ToAQa+WI8oqiKOjzA2OZZkBIQTnazR8J6BCC1iIG27bKtAeZuNYVxBShsDtmEB2BJJ/N0LQaq6SKpzjVGBmudfvptQU= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 1573483502273496.20232436520735; Mon, 11 Nov 2019 06:45:02 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-299-xIYRuykGPTuBYexwQKbZhw-1; Mon, 11 Nov 2019 09:44:58 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9352D18B5F97; Mon, 11 Nov 2019 14:44:53 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6DFB41001B03; Mon, 11 Nov 2019 14:44:53 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id DF8144E562; Mon, 11 Nov 2019 14:44:52 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xABEipax020343 for ; Mon, 11 Nov 2019 09:44:51 -0500 Received: by smtp.corp.redhat.com (Postfix) id 56E4517AD4; Mon, 11 Nov 2019 14:44:51 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9A2295D9C9; Mon, 11 Nov 2019 14:44:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573483501; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=F++An/cEk5/qluGGFPFy84SPBL6PLYpKqn8eCptdGno=; b=DjWDx9lzjkBwMzZMqOvfHGDerdyNeAwcX4btaL1oKL1c9VyT2C53pzppH1HsFBBQenypfq R3JZurixAol0MCTVITyiBJK42W3Eda6vCm8VntoeruOZDe908whXURLIjaLuw9JdUbnAv6 JTzzFS6AYvOrgNVMePr6bIMQJVNOFLQ= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 11 Nov 2019 14:38:05 +0000 Message-Id: <20191111143826.16050-3-berrange@redhat.com> In-Reply-To: <20191111143826.16050-1-berrange@redhat.com> References: <20191111143826.16050-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v5 02/23] build-aux: rewrite whitespace checker in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: xIYRuykGPTuBYexwQKbZhw-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" As part of an goal to eliminate Perl from libvirt build tools, rewrite the check-spacing.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. Signed-off-by: Daniel P. Berrang=C3=A9 --- Makefile.am | 2 +- build-aux/check-spacing.pl | 198 -------------------------------- build-aux/syntax-check.mk | 4 +- scripts/check-spacing.py | 229 +++++++++++++++++++++++++++++++++++++ 4 files changed, 232 insertions(+), 201 deletions(-) delete mode 100755 build-aux/check-spacing.pl create mode 100755 scripts/check-spacing.py diff --git a/Makefile.am b/Makefile.am index 9471cf7117..5187ca6cc2 100644 --- a/Makefile.am +++ b/Makefile.am @@ -46,7 +46,7 @@ EXTRA_DIST =3D \ README.md \ AUTHORS.in \ scripts/augeas-gentest.py \ - build-aux/check-spacing.pl \ + scripts/check-spacing.py \ build-aux/header-ifdef.pl \ scripts/minimize-po.py \ build-aux/mock-noinline.pl \ diff --git a/build-aux/check-spacing.pl b/build-aux/check-spacing.pl deleted file mode 100755 index 33377f3dd3..0000000000 --- a/build-aux/check-spacing.pl +++ /dev/null @@ -1,198 +0,0 @@ -#!/usr/bin/env perl -# -# check-spacing.pl: Report any usage of 'function (..args..)' -# Also check for other syntax issues, such as correct use of ';' -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library. If not, see -# . - -use strict; -use warnings; - -my $ret =3D 0; -my $incomment =3D 0; - -foreach my $file (@ARGV) { - # Per-file variables for multiline Curly Bracket (cb_) check - my $cb_linenum =3D 0; - my $cb_code =3D ""; - my $cb_scolon =3D 0; - - open FILE, $file; - - while (defined (my $line =3D )) { - my $data =3D $line; - # For temporary modifications - my $tmpdata; - - # Kill any quoted , ; =3D or " - $data =3D~ s/'[";,=3D]'/'X'/g; - - # Kill any quoted strings - $data =3D~ s,"(?:[^\\\"]|\\.)*","XXX",g; - - next if $data =3D~ /^#/; - - # Kill contents of multi-line comments - # and detect end of multi-line comments - if ($incomment) { - if ($data =3D~ m,\*/,) { - $incomment =3D 0; - $data =3D~ s,^.*\*/,*/,; - } else { - $data =3D ""; - } - } - - # Kill single line comments, and detect - # start of multi-line comments - if ($data =3D~ m,/\*.*\*/,) { - $data =3D~ s,/\*.*\*/,/* */,; - } elsif ($data =3D~ m,/\*,) { - $incomment =3D 1; - $data =3D~ s,/\*.*,/*,; - } - - # We need to match things like - # - # int foo (int bar, bool wizz); - # foo (bar, wizz); - # - # but not match things like: - # - # typedef int (*foo)(bar wizz) - # - # we can't do this (efficiently) without - # missing things like - # - # foo (*bar, wizz); - # - # We also don't want to spoil the $data so it can be used - # later on. - $tmpdata =3D $data; - while ($tmpdata =3D~ /(\w+)\s\((?!\*)/) { - my $kw =3D $1; - - # Allow space after keywords only - if ($kw =3D~ /^(?:if|for|while|switch|return)$/) { - $tmpdata =3D~ s/(?:$kw\s\()/XXX(/; - } else { - print "Whitespace after non-keyword:\n"; - print "$file:$.: $line"; - $ret =3D 1; - last; - } - } - - # Require whitespace immediately after keywords - if ($data =3D~ /\b(?:if|for|while|switch|return)\(/) { - print "No whitespace after keyword:\n"; - print "$file:$.: $line"; - $ret =3D 1; - } - - # Forbid whitespace between )( of a function typedef - if ($data =3D~ /\(\*\w+\)\s+\(/) { - print "Whitespace between ')' and '(':\n"; - print "$file:$.: $line"; - $ret =3D 1; - } - - # Forbid whitespace following ( or prior to ) - # but allow whitespace before ) on a single line - # (optionally followed by a semicolon) - if (($data =3D~ /\s\)/ && not $data =3D~ /^\s+\);?$/) || - $data =3D~ /\((?!$)\s/) { - print "Whitespace after '(' or before ')':\n"; - print "$file:$.: $line"; - $ret =3D 1; - } - - # Forbid whitespace before ";" or ",". Things like below are allow= ed: - # - # 1) The expression is empty for "for" loop. E.g. - # for (i =3D 0; ; i++) - # - # 2) An empty statement. E.g. - # while (write(statuswrite, &status, 1) =3D=3D -1 && - # errno =3D=3D EINTR) - # ; - # - if ($data =3D~ /\s[;,]/) { - unless ($data =3D~ /\S; ; / || - $data =3D~ /^\s+;/) { - print "Whitespace before semicolon or comma:\n"; - print "$file:$.: $line"; - $ret =3D 1; - } - } - - # Require EOL, macro line continuation, or whitespace after ";". - # Allow "for (;;)" as an exception. - if ($data =3D~ /;[^ \\\n;)]/) { - print "Invalid character after semicolon:\n"; - print "$file:$.: $line"; - $ret =3D 1; - } - - # Require EOL, space, or enum/struct end after comma. - if ($data =3D~ /,[^ \\\n)}]/) { - print "Invalid character after comma:\n"; - print "$file:$.: $line"; - $ret =3D 1; - } - - # Require spaces around assignment '=3D', compounds and '=3D=3D' - if ($data =3D~ /[^ ]\b[!<>&|\-+*\/%\^=3D]?=3D/ || - $data =3D~ /=3D[^=3D \\\n]/) { - print "Spacing around '=3D' or '=3D=3D':\n"; - print "$file:$.: $line"; - $ret =3D 1; - } - - # One line conditional statements with one line bodies should - # not use curly brackets. - if ($data =3D~ /^\s*(if|while|for)\b.*\{$/) { - $cb_linenum =3D $.; - $cb_code =3D $line; - $cb_scolon =3D 0; - } - - # We need to check for exactly one semicolon inside the body, - # because empty statements (e.g. with comment only) are - # allowed - if ($cb_linenum =3D=3D $. - 1 && $data =3D~ /^[^;]*;[^;]*$/) { - $cb_code .=3D $line; - $cb_scolon =3D 1; - } - - if ($data =3D~ /^\s*}\s*$/ && - $cb_linenum =3D=3D $. - 2 && - $cb_scolon) { - - print "Curly brackets around single-line body:\n"; - print "$file:$cb_linenum-$.:\n$cb_code$line"; - $ret =3D 1; - - # There _should_ be no need to reset the values; but to - # keep my inner peace... - $cb_linenum =3D 0; - $cb_scolon =3D 0; - $cb_code =3D ""; - } - } - close FILE; -} - -exit $ret; diff --git a/build-aux/syntax-check.mk b/build-aux/syntax-check.mk index 9b6c157029..d308896b26 100644 --- a/build-aux/syntax-check.mk +++ b/build-aux/syntax-check.mk @@ -2157,8 +2157,8 @@ prohibit-duplicate-header: $(PYTHON) $(top_srcdir)/scripts/prohibit-duplicate-header.py =20 spacing-check: - $(AM_V_GEN)$(VC_LIST) | $(GREP) '\.c$$' | xargs \ - $(PERL) $(top_srcdir)/build-aux/check-spacing.pl || \ + $(AM_V_GEN)$(VC_LIST) | $(GREP) '\.c$$' | $(RUNUTF8) xargs \ + $(PYTHON) $(top_srcdir)/scripts/check-spacing.py || \ { echo '$(ME): incorrect formatting' 1>&2; exit 1; } =20 mock-noinline: diff --git a/scripts/check-spacing.py b/scripts/check-spacing.py new file mode 100755 index 0000000000..6b9f3ec1ba --- /dev/null +++ b/scripts/check-spacing.py @@ -0,0 +1,229 @@ +#!/usr/bin/env python +# +# Copyright (C) 2012-2019 Red Hat, Inc. +# +# check-spacing.pl: Report any usage of 'function (..args..)' +# Also check for other syntax issues, such as correct use of ';' +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . + +from __future__ import print_function + +import re +import sys + + +def check_whitespace(filename): + errs =3D False + with open(filename, 'r') as fh: + quotedmetaprog =3D re.compile(r"""'[";,=3D]'""") + quotedstringprog =3D re.compile(r'''"(?:[^\\\"]|\\.)*"''') + commentstartprog =3D re.compile(r'''^(.*)/\*.*$''') + commentendprog =3D re.compile(r'''^.*\*/(.*)$''') + commentprog =3D re.compile(r'''^(.*)/\*.*\*/(.*)''') + funcprog =3D re.compile(r'''(\w+)\s\((?!\*)''') + keywordprog =3D re.compile( + r'''^.*\b(?:if|for|while|switch|return)\(.*$''') + functypedefprog =3D re.compile(r'''^.*\(\*\w+\)\s+\(.*$''') + whitespaceprog1 =3D re.compile(r'''^.*\s\).*$''') + whitespaceprog2 =3D re.compile(r'''^\s+\);?$''') + whitespaceprog3 =3D re.compile(r'''^.*\((?!$)\s.*''') + commasemiprog1 =3D re.compile(r'''.*\s[;,].*''') + commasemiprog2 =3D re.compile(r'''.*\S; ; .*''') + commasemiprog3 =3D re.compile(r'''^\s+;''') + semicolonprog =3D re.compile(r'''.*;[^ \\\n;)].*''') + commaprog =3D re.compile(r'''.*,[^ \\\n)}].*''') + assignprog1 =3D re.compile(r'''[^ ]\b[!<>&|\-+*\/%\^=3D]?=3D''') + assignprog2 =3D re.compile(r'''=3D[^=3D \\\n]''') + condstartprog =3D re.compile(r'''^\s*(if|while|for)\b.*\{$''') + statementprog =3D re.compile(r'''^[^;]*;[^;]*$''') + condendprog =3D re.compile(r'''^\s*}\s*$''') + + incomment =3D False + # Per-file variables for multiline Curly Bracket (cb_) check + cb_lineno =3D 0 + cb_code =3D "" + cb_scolon =3D False + + lineno =3D 0 + for line in fh: + lineno =3D lineno + 1 + data =3D line + # For temporary modifications + + # Kill any quoted , ; =3D or " + data =3D quotedmetaprog.sub("'X'", data) + + # Kill any quoted strings + data =3D quotedstringprog.sub('"XXX"', data) + + if data[0] =3D=3D '#': + continue + + # Kill contents of multi-line comments + # and detect end of multi-line comments + if incomment: + if commentendprog.match(data): + data =3D commentendprog.sub('*/\2', data) + incomment =3D False + else: + data =3D "" + + # Kill single line comments, and detect + # start of multi-line comments + if commentprog.match(data): + data =3D commentprog.sub(r'''\1/* */\2''', data) + elif commentstartprog.match(data): + data =3D commentstartprog.sub(r'''\1/*''', data) + incomment =3D True + + # We need to match things like + # + # int foo (int bar, bool wizz); + # foo (bar, wizz); + # + # but not match things like: + # + # typedef int (*foo)(bar wizz) + # + # we can't do this (efficiently) without + # missing things like + # + # foo (*bar, wizz); + # + for match in funcprog.finditer(data): + kw =3D match.group(1) + + # Allow space after keywords only + if kw not in ["if", "for", "while", "switch", "return"]: + print("Whitespace after non-keyword:", + file=3Dsys.stderr) + print("%s:%d: %s" % (filename, lineno, line), + file=3Dsys.stderr) + errs =3D True + break + + # Require whitespace immediately after keywords + if keywordprog.match(data): + print("No whitespace after keyword:", + file=3Dsys.stderr) + print("%s:%d: %s" % (filename, lineno, line), + file=3Dsys.stderr) + errs =3D True + + # Forbid whitespace between )( of a function typedef + if functypedefprog.match(data): + print("Whitespace between ')' and '(':", + file=3Dsys.stderr) + print("%s:%d: %s" % (filename, lineno, line), + file=3Dsys.stderr) + errs =3D True + + # Forbid whitespace following ( or prior to ) + # but allow whitespace before ) on a single line + # (optionally followed by a semicolon) + if ((whitespaceprog1.match(data) and + not whitespaceprog2.match(data)) + or whitespaceprog3.match(data)): + print("Whitespace after '(' or before ')':", + file=3Dsys.stderr) + print("%s:%d: %s" % (filename, lineno, line), + file=3Dsys.stderr) + errs =3D True + + # Forbid whitespace before ";" or ",". Things like + # below are allowed: + # + # 1) The expression is empty for "for" loop. E.g. + # for (i =3D 0; ; i++) + # + # 2) An empty statement. E.g. + # while (write(statuswrite, &status, 1) =3D=3D -1 && + # errno =3D=3D EINTR) + # ; + # + if commasemiprog1.match(data) and not ( + commasemiprog2.match(data) or + commasemiprog3.match(data)): + print("Whitespace before semicolon or comma:", + file=3Dsys.stderr) + print("%s:%d: %s" % (filename, lineno, line), + file=3Dsys.stderr) + errs =3D True + + # Require EOL, macro line continuation, or whitespace after ";= ". + # Allow "for (;;)" as an exception. + if semicolonprog.match(data): + print("Invalid character after semicolon:", + file=3Dsys.stderr) + print("%s:%d: %s" % (filename, lineno, line), + file=3Dsys.stderr) + errs =3D True + + # Require EOL, space, or enum/struct end after comma. + if commaprog.match(data): + print("Invalid character after comma:", + file=3Dsys.stderr) + print("%s:%d: %s" % (filename, lineno, line), + file=3Dsys.stderr) + errs =3D True + + # Require spaces around assignment '=3D', compounds and '=3D= =3D' + if assignprog1.match(data) or assignprog2.match(data): + print("Spacing around '=3D' or '=3D=3D':", + file=3Dsys.stderr) + print("%s:%d: %s" % (filename, lineno, line), + file=3Dsys.stderr) + errs =3D True + + # One line conditional statements with one line bodies should + # not use curly brackets. + if condstartprog.match(data): + cb_lineno =3D lineno + cb_code =3D line + cb_scolon =3D False + + # We need to check for exactly one semicolon inside the body, + # because empty statements (e.g. with comment only) are + # allowed + if (cb_lineno =3D=3D lineno - 1) and statementprog.match(data): + cb_code =3D cb_code + line + cb_scolon =3D True + + if (condendprog.match(data) and + (cb_lineno =3D=3D lineno - 2) and + cb_scolon): + print("Curly brackets around single-line body:", + file=3Dsys.stderr) + print("%s:%d:\n%s%s" % (filename, cb_lineno - lineno, + cb_code, line), + file=3Dsys.stderr) + errs =3D True + + # There _should_ be no need to reset the values; but to + # keep my inner peace... + cb_lineno =3D 0 + cb_scolon =3D False + cb_code =3D "" + + return errs + + +ret =3D 0 +for filename in sys.argv[1:]: + if check_whitespace(filename): + ret =3D 1 + +sys.exit(ret) --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sun May 5 18:51:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1573483513; cv=none; d=zoho.com; s=zohoarc; b=Roin4Vu96BFZQ6o2q2QNlcspkUNwvmPTmwPSnA52Fl0iS6scaR81HVQubcUDJ2z5fykuM9MNV6DtGQFO0fbWqPEv8qrNhZpwUh8FRACb0+701uoBKN6QTt7ZoxuZeb3MriN0W98wVEXkT5eDZFsFN/XdTlQhClSkNHMOBvjNy1I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573483513; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=kLnn9mNoJfxDPFlpzjHIxDUpSMWix0qa1ih6HIVdVW0=; b=TYral9mslKqryRqw0OmBG7U3ZLmXGnYYvn21iBwzqhodiGnUYr8F28pTJxtJPQyKxaTn9Bdowh8c98144YkmYdy0CgOXT5o7rQWrcuVL480mr1x+r9I//bjodM+vZkMjVRwYAldXQtcfy2mnKk1djXCUbDq51KibJtigoICFJbE= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 1573483513396806.7295225340433; Mon, 11 Nov 2019 06:45:13 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-205-PiaXwbzMPj-3tlUeAz2XPA-1; Mon, 11 Nov 2019 09:45:08 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2AC838A2423; Mon, 11 Nov 2019 14:44:59 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 07B584B4; Mon, 11 Nov 2019 14:44:59 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id C16CB18034EE; Mon, 11 Nov 2019 14:44:58 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xABEiqNE020351 for ; Mon, 11 Nov 2019 09:44:52 -0500 Received: by smtp.corp.redhat.com (Postfix) id 39C595DD73; Mon, 11 Nov 2019 14:44:52 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9CCC85D9C9; Mon, 11 Nov 2019 14:44:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573483512; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=kLnn9mNoJfxDPFlpzjHIxDUpSMWix0qa1ih6HIVdVW0=; b=B87qr1XNiceypzsishtGb25a4rVunoUvxP7mf/KGU6aKiJ0NBU+C9UbpYtvztpdELTzTgn YHJ0WZO7FvWNF7P/HkxWTpvFbhxhoE9GRHiRBj6t2tv7NYrUlbzSRRmsQt/owbJ125x14O oMYurel21+0swJFYxVrdApx45MKQMB0= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 11 Nov 2019 14:38:06 +0000 Message-Id: <20191111143826.16050-4-berrange@redhat.com> In-Reply-To: <20191111143826.16050-1-berrange@redhat.com> References: <20191111143826.16050-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v5 03/23] build-aux: rewrite mock inline checker in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-MC-Unique: PiaXwbzMPj-3tlUeAz2XPA-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" As part of an goal to eliminate Perl from libvirt build tools, rewrite the mock-noinline.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: J=C3=A1n Tomko Tested-by: Cole Robinson --- Makefile.am | 2 +- build-aux/mock-noinline.pl | 75 --------------------------------- build-aux/syntax-check.mk | 4 +- scripts/mock-noinline.py | 85 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 88 insertions(+), 78 deletions(-) delete mode 100644 build-aux/mock-noinline.pl create mode 100644 scripts/mock-noinline.py diff --git a/Makefile.am b/Makefile.am index 5187ca6cc2..d9369c9197 100644 --- a/Makefile.am +++ b/Makefile.am @@ -49,7 +49,7 @@ EXTRA_DIST =3D \ scripts/check-spacing.py \ build-aux/header-ifdef.pl \ scripts/minimize-po.py \ - build-aux/mock-noinline.pl \ + scripts/mock-noinline.py \ scripts/prohibit-duplicate-header.py \ build-aux/syntax-check.mk \ build-aux/useless-if-before-free \ diff --git a/build-aux/mock-noinline.pl b/build-aux/mock-noinline.pl deleted file mode 100644 index b005b8d95e..0000000000 --- a/build-aux/mock-noinline.pl +++ /dev/null @@ -1,75 +0,0 @@ -#!/usr/bin/env perl - -my %noninlined; -my %mocked; - -# Functions in public header don't get the noinline annotation -# so whitelist them here -$noninlined{"virEventAddTimeout"} =3D 1; -# This one confuses the script as its defined in the mock file -# but is actually just a local helper -$noninlined{"virMockStatRedirect"} =3D 1; - -foreach my $arg (@ARGV) { - if ($arg =3D~ /\.h$/) { - #print "Scan header $arg\n"; - &scan_annotations($arg); - } elsif ($arg =3D~ /mock\.c$/) { - #print "Scan mock $arg\n"; - &scan_overrides($arg); - } -} - -my $warned =3D 0; -foreach my $func (keys %mocked) { - next if exists $noninlined{$func}; - - $warned++; - print STDERR "$func is mocked at $mocked{$func} but missing noinline a= nnotation\n"; -} - -exit $warned ? 1 : 0; - - -sub scan_annotations { - my $file =3D shift; - - open FH, $file or die "cannot read $file: $!"; - - my $func; - while () { - if (/^\s*(\w+)\(/ || /^(?:\w+\*?\s+)+(?:\*\s*)?(\w+)\(/) { - my $name =3D $1; - if ($name !~ /(?:G_GNUC|ATTRIBUTE)/) { - $func =3D $name; - } - } elsif (/^\s*$/) { - $func =3D undef; - } - if (/G_GNUC_NO_INLINE/) { - if (defined $func) { - $noninlined{$func} =3D 1; - } - } - } - - close FH -} - -sub scan_overrides { - my $file =3D shift; - - open FH, $file or die "cannot read $file: $!"; - - my $func; - while () { - if (/^(\w+)\(/ || /^\w+\s*(?:\*\s*)?(\w+)\(/) { - my $name =3D $1; - if ($name =3D~ /^vir/) { - $mocked{$name} =3D "$file:$."; - } - } - } - - close FH -} diff --git a/build-aux/syntax-check.mk b/build-aux/syntax-check.mk index d308896b26..0cfd181224 100644 --- a/build-aux/syntax-check.mk +++ b/build-aux/syntax-check.mk @@ -2162,8 +2162,8 @@ spacing-check: { echo '$(ME): incorrect formatting' 1>&2; exit 1; } =20 mock-noinline: - $(AM_V_GEN)$(VC_LIST) | $(GREP) '\.[ch]$$' | xargs \ - $(PERL) $(top_srcdir)/build-aux/mock-noinline.pl + $(AM_V_GEN)$(VC_LIST) | $(GREP) '\.[ch]$$' | $(RUNUTF8) xargs \ + $(PYTHON) $(top_srcdir)/scripts/mock-noinline.py =20 header-ifdef: $(AM_V_GEN)$(VC_LIST) | $(GREP) '\.[h]$$' | xargs \ diff --git a/scripts/mock-noinline.py b/scripts/mock-noinline.py new file mode 100644 index 0000000000..2770ea1238 --- /dev/null +++ b/scripts/mock-noinline.py @@ -0,0 +1,85 @@ +#!/usr/bin/env python +# +# Copyright (C) 2017-2019 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . + +from __future__ import print_function + +import re +import sys + +noninlined =3D {} +mocked =3D {} + +# Functions in public header don't get the noinline annotation +# so whitelist them here +noninlined["virEventAddTimeout"] =3D True +# This one confuses the script as its defined in the mock file +# but is actually just a local helper +noninlined["virMockStatRedirect"] =3D True + + +def scan_annotations(filename): + with open(filename, "r") as fh: + func =3D None + for line in fh: + line =3D line.strip() + m =3D re.search(r'''^\s*(\w+)\(''', line) + if m is None: + m =3D re.search(r'''^(?:\w+\*?\s+)+(?:\*\s*)?(\w+)\(''', l= ine) + if m is not None: + name =3D m.group(1) + if name.find("ATTRIBUTE") =3D=3D -1 and name.find("G_GNUC_= ") =3D=3D -1: + func =3D name + elif line =3D=3D "": + func =3D None + + if line.find("G_GNUC_NO_INLINE") !=3D -1: + if func is not None: + noninlined[func] =3D True + + +def scan_overrides(filename): + with open(filename, "r") as fh: + lineno =3D 0 + for line in fh: + lineno =3D lineno + 1 + + m =3D re.search(r'''^(\w+)\(''', line) + if m is None: + m =3D re.search(r'''^\w+\s*(?:\*\s*)?(\w+)\(''', line) + if m is not None: + name =3D m.group(1) + if name.startswith("vir"): + mocked[name] =3D "%s:%d" % (filename, lineno) + + +for filename in sys.argv[1:]: + if filename.endswith(".h"): + scan_annotations(filename) + elif filename.endswith("mock.c"): + scan_overrides(filename) + +warned =3D False +for func in mocked.keys(): + if func not in noninlined: + warned =3D True + print("%s is mocked at %s but missing noinline annotation" % + (func, mocked[func]), file=3Dsys.stderr) + +if warned: + sys.exit(1) +sys.exit(0) --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sun May 5 18:51:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 207.211.31.120 as permitted sender) client-ip=207.211.31.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1573483513; cv=none; d=zoho.com; s=zohoarc; b=cDikeZ+6YRSI+jXSvseg+2vc8D/xfLAWlaRjJ+3QyQ0qGD/93aCT2Yh6To2WKdWRw9fHHugtfX4tTLIdbUFJLgU7ip8b3pxy0uqNSjE4bjCOu5dxCiNE0LburjREiaJPDPRM7oAD1waCclW2jN3gjfO5UhzPvR4uLd038aI+D6k= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573483513; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=k/Zar0IWWwmQ6IBQt+aLK1DL4zzwdLCNe8gp/Py1IYE=; b=jA/ulGIWrKKpqOV/sty0oniTfuw2LtsZZqeVapbJ5lHkGnLZ9mqy2qGsRHjR4oPV4GphStpGw5CpfUZpF18pZbn6ocgo2wuD9xyGQ7G2dgCtpOUiYWMOn7vyBWqdqMavBZ/IUwxU90JQ2P+Q+p5X5/1qMVOvBSTh8kQh6G2xdl4= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by mx.zohomail.com with SMTPS id 1573483513819691.5192981219024; Mon, 11 Nov 2019 06:45:13 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-119-0lJVFTYhNHOaEONxbBVOKA-1; Mon, 11 Nov 2019 09:45:10 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 8493C890F0A; Mon, 11 Nov 2019 14:45:01 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 607BD1001B03; Mon, 11 Nov 2019 14:45:01 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 197D54E56D; Mon, 11 Nov 2019 14:45:01 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xABEirVc020358 for ; Mon, 11 Nov 2019 09:44:53 -0500 Received: by smtp.corp.redhat.com (Postfix) id 3B44F17AD4; Mon, 11 Nov 2019 14:44:53 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7EA7B5D9C9; Mon, 11 Nov 2019 14:44:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573483512; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=k/Zar0IWWwmQ6IBQt+aLK1DL4zzwdLCNe8gp/Py1IYE=; b=hARpBOwCuQulObJ4y5i7MaRY9Ew/lOE2zuTMn3GjG7fKgUDwcA6yRVcjRTf9BVZ8rVOStM lP4hY/to0Ei5O/C1rBFK6zAu7eolZCofWrjOsdiRsNItfUHtJRbXPCcEA+3RCJViQ7ipyx uT/AWn5T4UHtAQrw1JYSkETY9XqbTUA= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 11 Nov 2019 14:38:07 +0000 Message-Id: <20191111143826.16050-5-berrange@redhat.com> In-Reply-To: <20191111143826.16050-1-berrange@redhat.com> References: <20191111143826.16050-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v5 04/23] build-aux: rewrite header ifdef checker in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: 0lJVFTYhNHOaEONxbBVOKA-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" As part of an goal to eliminate Perl from libvirt build tools, rewrite the header-ifdef.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: J=C3=A1n Tomko Tested-by: Cole Robinson --- Makefile.am | 2 +- build-aux/header-ifdef.pl | 182 ------------------------------ build-aux/syntax-check.mk | 4 +- scripts/header-ifdef.py | 231 ++++++++++++++++++++++++++++++++++++++ 4 files changed, 234 insertions(+), 185 deletions(-) delete mode 100644 build-aux/header-ifdef.pl create mode 100644 scripts/header-ifdef.py diff --git a/Makefile.am b/Makefile.am index d9369c9197..6cccbf38da 100644 --- a/Makefile.am +++ b/Makefile.am @@ -47,7 +47,7 @@ EXTRA_DIST =3D \ AUTHORS.in \ scripts/augeas-gentest.py \ scripts/check-spacing.py \ - build-aux/header-ifdef.pl \ + scripts/header-ifdef.py \ scripts/minimize-po.py \ scripts/mock-noinline.py \ scripts/prohibit-duplicate-header.py \ diff --git a/build-aux/header-ifdef.pl b/build-aux/header-ifdef.pl deleted file mode 100644 index dba3dbcbdc..0000000000 --- a/build-aux/header-ifdef.pl +++ /dev/null @@ -1,182 +0,0 @@ -#!/usr/bin/perl -# -# Validate that header files follow a standard layout: -# -# /* -# ...copyright header... -# */ -# -# #pragma once -# ....content.... -# -#--- -# -# For any file ending priv.h, before the #pragma once -# We will have a further section -# -# #ifndef SYMBOL_ALLOW -# # error .... -# #endif /* SYMBOL_ALLOW */ -# -# -#--- -# -# For public headers (files in include/), use the standard header guard in= stead of #pragma once: -# #ifndef SYMBOL -# # define SYMBOL -# ....content.... -# #endif /* SYMBOL */ - -use strict; -use warnings; - -my $STATE_COPYRIGHT_COMMENT =3D 0; -my $STATE_COPYRIGHT_BLANK =3D 1; -my $STATE_PRIV_START =3D 2; -my $STATE_PRIV_ERROR =3D 3; -my $STATE_PRIV_END =3D 4; -my $STATE_PRIV_BLANK =3D 5; -my $STATE_GUARD_START =3D 6; -my $STATE_GUARD_DEFINE =3D 7; -my $STATE_GUARD_END =3D 8; -my $STATE_EOF =3D 9; -my $STATE_PRAGMA =3D 10; - -my $file =3D " "; -my $ret =3D 0; -my $ifdef =3D ""; -my $ifdefpriv =3D ""; -my $publicheader =3D 0; - -my $state =3D $STATE_EOF; -my $mistake =3D 0; - -sub mistake { - my $msg =3D shift; - warn $msg; - $mistake =3D 1; - $ret =3D 1; -} - -while (<>) { - if (not $file eq $ARGV) { - if ($state =3D=3D $STATE_COPYRIGHT_COMMENT) { - &mistake("$file: missing copyright comment"); - } elsif ($state =3D=3D $STATE_COPYRIGHT_BLANK) { - &mistake("$file: missing blank line after copyright header"); - } elsif ($state =3D=3D $STATE_PRIV_START) { - &mistake("$file: missing '#ifndef $ifdefpriv'"); - } elsif ($state =3D=3D $STATE_PRIV_ERROR) { - &mistake("$file: missing '# error ...priv allow...'"); - } elsif ($state =3D=3D $STATE_PRIV_END) { - &mistake("$file: missing '#endif /* $ifdefpriv */'"); - } elsif ($state =3D=3D $STATE_PRIV_BLANK) { - &mistake("$file: missing blank line after priv header check"); - } elsif ($state =3D=3D $STATE_GUARD_START) { - if ($publicheader) { - &mistake("$file: missing '#ifndef $ifdef'"); - } else { - &mistake("$file: missing '#pragma once' header guard"); - } - } elsif ($state =3D=3D $STATE_GUARD_DEFINE) { - &mistake("$file: missing '# define $ifdef'"); - } elsif ($state =3D=3D $STATE_GUARD_END) { - &mistake("$file: missing '#endif /* $ifdef */'"); - } - - $ifdef =3D uc $ARGV; - $ifdef =3D~ s,.*/,,; - $ifdef =3D~ s,[^A-Z0-9],_,g; - $ifdef =3D~ s,__+,_,g; - unless ($ifdef =3D~ /^LIBVIRT_/ && $ARGV !~ /libvirt_internal.h/) { - $ifdef =3D "LIBVIRT_" . $ifdef; - } - $ifdefpriv =3D $ifdef . "_ALLOW"; - - $file =3D $ARGV; - $state =3D $STATE_COPYRIGHT_COMMENT; - $mistake =3D 0; - $publicheader =3D ($ARGV =3D~ /include\//); - } - - if ($mistake || - $ARGV =3D~ /config-post\.h$/ || - $ARGV =3D~ /vbox_(CAPI|XPCOM)/) { - $state =3D $STATE_EOF; - next; - } - - if ($state =3D=3D $STATE_COPYRIGHT_COMMENT) { - if (m,\*/,) { - $state =3D $STATE_COPYRIGHT_BLANK; - } - } elsif ($state =3D=3D $STATE_COPYRIGHT_BLANK) { - if (! /^$/) { - &mistake("$file: missing blank line after copyright header"); - } - if ($ARGV =3D~ /priv\.h$/) { - $state =3D $STATE_PRIV_START; - } else { - $state =3D $STATE_GUARD_START; - } - } elsif ($state =3D=3D $STATE_PRIV_START) { - if (/^$/) { - &mistake("$file: too many blank lines after copyright header"); - } elsif (/#ifndef $ifdefpriv$/) { - $state =3D $STATE_PRIV_ERROR; - } else { - &mistake("$file: missing '#ifndef $ifdefpriv'"); - } - } elsif ($state =3D=3D $STATE_PRIV_ERROR) { - if (/# error ".*"$/) { - $state =3D $STATE_PRIV_END; - } else { - &mistake("$file: missing '# error ...priv allow...'"); - } - } elsif ($state =3D=3D $STATE_PRIV_END) { - if (m,#endif /\* $ifdefpriv \*/,) { - $state =3D $STATE_PRIV_BLANK; - } else { - &mistake("$file: missing '#endif /* $ifdefpriv */'"); - } - } elsif ($state =3D=3D $STATE_PRIV_BLANK) { - if (! /^$/) { - &mistake("$file: missing blank line after priv guard"); - } - $state =3D $STATE_GUARD_START; - } elsif ($state =3D=3D $STATE_GUARD_START) { - if (/^$/) { - &mistake("$file: too many blank lines after copyright header"); - } - if ($publicheader) { - if (/#ifndef $ifdef$/) { - $state =3D $STATE_GUARD_DEFINE; - } else { - &mistake("$file: missing '#ifndef $ifdef'"); - } - } else { - if (/#pragma once/) { - $state =3D $STATE_PRAGMA; - } else { - &mistake("$file: missing '#pragma once' header guard"); - } - } - } elsif ($state =3D=3D $STATE_GUARD_DEFINE) { - if (/# define $ifdef$/) { - $state =3D $STATE_GUARD_END; - } else { - &mistake("$file: missing '# define $ifdef'"); - } - } elsif ($state =3D=3D $STATE_GUARD_END) { - if (m,#endif /\* $ifdef \*/$,) { - $state =3D $STATE_EOF; - } - } elsif ($state =3D=3D $STATE_PRAGMA) { - next; - } elsif ($state =3D=3D $STATE_EOF) { - die "$file: unexpected content after '#endif /* $ifdef */'"; - } else { - die "$file: unexpected state $state"; - } -} -exit $ret; diff --git a/build-aux/syntax-check.mk b/build-aux/syntax-check.mk index 0cfd181224..016e05e7a2 100644 --- a/build-aux/syntax-check.mk +++ b/build-aux/syntax-check.mk @@ -2166,8 +2166,8 @@ mock-noinline: $(PYTHON) $(top_srcdir)/scripts/mock-noinline.py =20 header-ifdef: - $(AM_V_GEN)$(VC_LIST) | $(GREP) '\.[h]$$' | xargs \ - $(PERL) $(top_srcdir)/build-aux/header-ifdef.pl + $(AM_V_GEN)$(VC_LIST) | $(GREP) '\.[h]$$' | $(RUNUTF8) xargs \ + $(PYTHON) $(top_srcdir)/scripts/header-ifdef.py =20 test-wrap-argv: $(AM_V_GEN)$(VC_LIST) | $(GREP) -E '\.(ldargs|args)' | xargs \ diff --git a/scripts/header-ifdef.py b/scripts/header-ifdef.py new file mode 100644 index 0000000000..94c2882225 --- /dev/null +++ b/scripts/header-ifdef.py @@ -0,0 +1,231 @@ +#!/usr/bin/env python +# +# Copyright (C) 2018-2019 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . +# +# Validate that header files follow a standard layout: +# +# /* +# ...copyright header... +# */ +# +# #pragma once +# ....content.... +# +# --- +# +# For any file ending priv.h, before the #pragma once +# We will have a further section +# +# #ifndef SYMBOL_ALLOW +# # error .... +# #endif /* SYMBOL_ALLOW */ +# +# +# --- +# +# For public headers (files in include/), use the standard +# header guard instead of #pragma once: +# #ifndef SYMBOL +# # define SYMBOL +# ....content.... +# #endif /* SYMBOL */ + +from __future__ import print_function + +import os.path +import re +import sys + +STATE_COPYRIGHT_COMMENT =3D 0 +STATE_COPYRIGHT_BLANK =3D 1 +STATE_PRIV_START =3D 2 +STATE_PRIV_ERROR =3D 3 +STATE_PRIV_END =3D 4 +STATE_PRIV_BLANK =3D 5 +STATE_GUARD_START =3D 6 +STATE_GUARD_DEFINE =3D 7 +STATE_GUARD_END =3D 8 +STATE_EOF =3D 9 +STATE_PRAGMA =3D 10 + + +def check_header(filename): + ifdef =3D "" + ifdefpriv =3D "" + + state =3D STATE_EOF + + ifdef =3D os.path.basename(filename).upper() + ifdef =3D re.sub(r"""[^A-Z0-9]""", "_", ifdef) + ifdef =3D re.sub(r"""__+""", "_", ifdef) + + if (not ifdef.startswith("LIBVIRT_") or + filename.find("libvirt_internal.h") !=3D -1): + ifdef =3D "LIBVIRT_" + ifdef + + ifdefpriv =3D ifdef + "_ALLOW" + + state =3D STATE_COPYRIGHT_COMMENT + publicheader =3D False + if filename.find("include/") !=3D -1: + publicheader =3D True + + with open(filename, "r") as fh: + for line in fh: + line =3D line.rstrip("\n") + if state =3D=3D STATE_COPYRIGHT_COMMENT: + if line.find("*/") !=3D -1: + state =3D STATE_COPYRIGHT_BLANK + elif state =3D=3D STATE_COPYRIGHT_BLANK: + if line !=3D "": + print("%s: missing blank line after copyright header" % + filename, file=3Dsys.stderr) + return True + + if filename.endswith("priv.h"): + state =3D STATE_PRIV_START + else: + state =3D STATE_GUARD_START + elif state =3D=3D STATE_PRIV_START: + if line =3D=3D "": + print("%s: too many blank lines after copyright header= " % + filename, file=3Dsys.stderr) + return True + elif re.search(r"""#ifndef %s$""" % ifdefpriv, line): + state =3D STATE_PRIV_ERROR + else: + print("%s: missing '#ifndef %s'" % (filename, ifdefpri= v), + file=3Dsys.stderr) + return True + elif state =3D=3D STATE_PRIV_ERROR: + if re.search(r"""# error ".*"$""", line): + state =3D STATE_PRIV_END + else: + print("%s: missing '# error ...priv allow...'" % + filename, file=3Dsys.stderr) + return True + elif state =3D=3D STATE_PRIV_END: + if re.search(r"""#endif /\* %s \*/""" % ifdefpriv, line): + state =3D STATE_PRIV_BLANK + else: + print("%s: missing '#endif /* %s */'" % + (filename, ifdefpriv), file=3Dsys.stderr) + return True + elif state =3D=3D STATE_PRIV_BLANK: + if line !=3D "": + print("%s: missing blank line after priv guard" % + filename, file=3Dsys.stderr) + return True + state =3D STATE_GUARD_START + elif state =3D=3D STATE_GUARD_START: + if line =3D=3D "": + print("%s: too many blank lines after copyright header= " % + filename, file=3Dsys.stderr) + return True + if publicheader: + if re.search(r"""#ifndef %s$""" % ifdef, line): + state =3D STATE_GUARD_DEFINE + else: + print("%s: missing '#ifndef %s'" % + (filename, ifdef), file=3Dsys.stderr) + return True + else: + if re.search(r"""#pragma once""", line): + state =3D STATE_PRAGMA + else: + print("%s: missing '#pragma once' header guard" % + filename, file=3Dsys.stderr) + return True + elif state =3D=3D STATE_GUARD_DEFINE: + if re.search(r"""# define %s$""" % ifdef, line): + state =3D STATE_GUARD_END + else: + print("%s: missing '# define %s'" % + (filename, ifdef), file=3Dsys.stderr) + return True + elif state =3D=3D STATE_GUARD_END: + if re.search(r"""#endif /\* %s \*/$""" % ifdef, line): + state =3D STATE_EOF + elif state =3D=3D STATE_PRAGMA: + next + elif state =3D=3D STATE_EOF: + print("%s: unexpected content after '#endif /* %s */'" % + (filename, ifdef), file=3Dsys.stderr) + return True + else: + print("%s: unexpected state $state" % + filename, file=3Dsys.stderr) + return True + + if state =3D=3D STATE_COPYRIGHT_COMMENT: + print("%s: missing copyright comment" % + filename, file=3Dsys.stderr) + return True + elif state =3D=3D STATE_COPYRIGHT_BLANK: + print("%s: missing blank line after copyright header" % + filename, file=3Dsys.stderr) + return True + elif state =3D=3D STATE_PRIV_START: + print("%s: missing '#ifndef %s'" % + (filename, ifdefpriv), file=3Dsys.stderr) + return True + elif state =3D=3D STATE_PRIV_ERROR: + print("%s: missing '# error ...priv allow...'" % + filename, file=3Dsys.stderr) + return True + elif state =3D=3D STATE_PRIV_END: + print("%s: missing '#endif /* %s */'" % + (filename, ifdefpriv), file=3Dsys.stderr) + return True + elif state =3D=3D STATE_PRIV_BLANK: + print("%s: missing blank line after priv header check" % + filename, file=3Dsys.stderr) + return True + elif state =3D=3D STATE_GUARD_START: + if publicheader: + print("%s: missing '#ifndef %s'" % + (filename, ifdef), file=3Dsys.stderr) + return True + else: + print("%s: missing '#pragma once' header guard" % + filename, file=3Dsys.stderr) + return True + elif state =3D=3D STATE_GUARD_DEFINE: + print("%s: missing '# define %s'" % + (filename, ifdef), file=3Dsys.stderr) + return True + elif state =3D=3D STATE_GUARD_END: + print("%s: missing '#endif /* %s */'" % + (filename, ifdef), file=3Dsys.stderr) + return True + + return False + + +ret =3D 0 + +for filename in sys.argv[1:]: + if filename.find("config-post.h") !=3D -1: + continue + if filename.find("vbox_CAPI") !=3D -1: + continue + if filename.find("vbox_XPCOM") !=3D -1: + continue + if check_header(filename): + ret =3D 1 + +sys.exit(ret) --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sun May 5 18:51:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 207.211.31.120 as permitted sender) client-ip=207.211.31.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1573483513; cv=none; d=zoho.com; s=zohoarc; b=gxe0cWo/SvANGMiLbWcc9fVJOVhPswShF6jAxh9Sf2ZL6+FZAe/cBaiuNits6gQVdg+JyXaEmc/OX9nPGGY1H807PQJi3LfDzS+yIW+pB/gsCYCfdnaYpEFTDDRx1AenOmnogmuCDLb2mOIU0VzMxGx+09g6RmLC4myt6B2dV8I= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573483513; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=1AB4y2pqo4URgIchFFb7V5CWnMhEvCvcwNdmiRePSSA=; b=faL9jj+JSfm07GLjAf1Gs6AD4cGFNqE7MlthYNo619SIZt2KJBYQkEn6QRpmTNpEvW4rHSb3z4y9KhaHxcomcbnk4Ydo9z2A8XQPGS0RmoCR4n+XTGIG4b6wmJjHQOQQvO2Tqr4ukzm77EVlsJ9mF7/kJ0VcFvzFD/HeAQblcKo= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by mx.zohomail.com with SMTPS id 1573483513345659.6757302061385; Mon, 11 Nov 2019 06:45:13 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-87-1ga6FySEMmWEsG7q-d-j6g-1; Mon, 11 Nov 2019 09:45:06 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 98B1E18B9FBE; Mon, 11 Nov 2019 14:45:01 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6E20F5DA7D; Mon, 11 Nov 2019 14:45:01 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 31CD918034EA; Mon, 11 Nov 2019 14:45:01 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xABEisM2020371 for ; Mon, 11 Nov 2019 09:44:54 -0500 Received: by smtp.corp.redhat.com (Postfix) id 1C0815DD73; Mon, 11 Nov 2019 14:44:54 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8167A5D9C9; Mon, 11 Nov 2019 14:44:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573483512; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=1AB4y2pqo4URgIchFFb7V5CWnMhEvCvcwNdmiRePSSA=; b=A2yBOCLoP0vVTXCOqqABTRhPava0A95+5RUAy2D9iXElP7ZdGI0qRmat7j/7w5s3vUiPZQ +zJzn60UUUZmVR9l1SGNnp6uqG25ri1wJLayPDN3FdeQkLo5fpvpJv7MTYzsqKhu4G9WPP gVPzqitRYV8n8rYH7uJ+x1B9XRLBijs= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 11 Nov 2019 14:38:08 +0000 Message-Id: <20191111143826.16050-6-berrange@redhat.com> In-Reply-To: <20191111143826.16050-1-berrange@redhat.com> References: <20191111143826.16050-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v5 05/23] src: rewrite ACL permissions checker in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-MC-Unique: 1ga6FySEMmWEsG7q-d-j6g-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" As part of an goal to eliminate Perl from libvirt build tools, rewrite the check-aclperms.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: J=C3=A1n Tomko Tested-by: Cole Robinson --- Makefile.am | 1 + scripts/check-aclperms.py | 75 +++++++++++++++++++++++++++++++++++++++ src/Makefile.am | 4 +-- src/check-aclperms.pl | 73 ------------------------------------- 4 files changed, 78 insertions(+), 75 deletions(-) create mode 100755 scripts/check-aclperms.py delete mode 100755 src/check-aclperms.pl diff --git a/Makefile.am b/Makefile.am index 6cccbf38da..ab9d09fcd4 100644 --- a/Makefile.am +++ b/Makefile.am @@ -46,6 +46,7 @@ EXTRA_DIST =3D \ README.md \ AUTHORS.in \ scripts/augeas-gentest.py \ + scripts/check-aclperms.py \ scripts/check-spacing.py \ scripts/header-ifdef.py \ scripts/minimize-po.py \ diff --git a/scripts/check-aclperms.py b/scripts/check-aclperms.py new file mode 100755 index 0000000000..b1084a3758 --- /dev/null +++ b/scripts/check-aclperms.py @@ -0,0 +1,75 @@ +#!/usr/bin/env python +# +# Copyright (C) 2013-2019 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . +# +# This script just validates that the stringified version of +# a virAccessPerm enum matches the enum constant name. We do +# a lot of auto-generation of code, so when these don't match +# problems occur, preventing auth from succeeding at all. + +from __future__ import print_function + +import re +import sys + +if len(sys.argv) !=3D 3: + print("syntax: %s HEADER IMPL" % (sys.argv[0]), file=3Dsys.stderr) + sys.exit(1) + +hdr =3D sys.argv[1] +impl =3D sys.argv[2] + +perms =3D {} + +with open(hdr) as fh: + for line in fh: + symmatch =3D re.search(r"^\s+VIR_ACCESS_PERM_([_A-Z]+)(,?|\s|$)", = line) + if symmatch is not None: + perm =3D symmatch.group(1) + + if not perm.endswith("_LAST"): + perms[perm] =3D 1 + +warned =3D False + +with open(impl) as fh: + group =3D None + + for line in fh: + symlastmatch =3D re.search(r"VIR_ACCESS_PERM_([_A-Z]+)_LAST", line) + if symlastmatch is not None: + group =3D symlastmatch.group(1) + elif re.search(r'''"[_a-z]+"''', line) is not None: + bits =3D line.split(",") + for bit in bits: + m =3D re.search(r'''"([_a-z]+)"''', bit) + if m is not None: + perm =3D (group + "_" + m.group(1)).upper() + if perm not in perms: + print("Unknown perm string %s for group %s" % + (m.group(1), group), file=3Dsys.stderr) + warned =3D True + + del perms[perm] + +for perm in perms.keys(): + print("Perm %s had not string form" % perm, file=3Dsys.stderr) + warned =3D True + +if warned: + sys.exit(1) +sys.exit(0) diff --git a/src/Makefile.am b/src/Makefile.am index 9b0a46702b..318dd6c20f 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -353,11 +353,11 @@ check-aclrules: $(addprefix $(srcdir)/,$(filter-out /%,$(STATEFUL_DRIVER_SOURCE_FILES))) =20 check-aclperms: - $(AM_V_GEN)$(PERL) $(srcdir)/check-aclperms.pl \ + $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(top_srcdir)/scripts/check-aclperms.py \ $(srcdir)/access/viraccessperm.h \ $(srcdir)/access/viraccessperm.c =20 -EXTRA_DIST +=3D check-driverimpls.pl check-aclrules.pl check-aclperms.pl +EXTRA_DIST +=3D check-driverimpls.pl check-aclrules.pl =20 check-local: check-protocol check-symfile check-symsorting \ check-drivername check-driverimpls check-aclrules \ diff --git a/src/check-aclperms.pl b/src/check-aclperms.pl deleted file mode 100755 index 55b6598313..0000000000 --- a/src/check-aclperms.pl +++ /dev/null @@ -1,73 +0,0 @@ -#!/usr/bin/env perl -# -# Copyright (C) 2013 Red Hat, Inc. -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library. If not, see -# . -# -# This script just validates that the stringified version of -# a virAccessPerm enum matches the enum constant name. We do -# a lot of auto-generation of code, so when these don't match -# problems occur, preventing auth from succeeding at all. - -my $hdr =3D shift; -my $impl =3D shift; - -my %perms; - -my @perms; - -open HDR, $hdr or die "cannot read $hdr: $!"; - -while () { - if (/^\s+VIR_ACCESS_PERM_([_A-Z]+)(,?|\s|$)/) { - my $perm =3D $1; - - $perms{$perm} =3D 1 unless ($perm =3D~ /_LAST$/); - } -} - -close HDR; - - -open IMPL, $impl or die "cannot read $impl: $!"; - -my $group; -my $warned =3D 0; - -while (defined (my $line =3D )) { - if ($line =3D~ /VIR_ACCESS_PERM_([_A-Z]+)_LAST/) { - $group =3D $1; - } elsif ($line =3D~ /"[_a-z]+"/) { - my @bits =3D split /,/, $line; - foreach my $bit (@bits) { - if ($bit =3D~ /"([_a-z]+)"/) { - my $perm =3D uc($group . "_" . $1); - if (!exists $perms{$perm}) { - print STDERR "Unknown perm string $1 for group $group\= n"; - $warned =3D 1; - } - delete $perms{$perm}; - } - } - } -} -close IMPL; - -foreach my $perm (keys %perms) { - print STDERR "Perm $perm had not string form\n"; - $warned =3D 1; -} - -exit $warned; --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sun May 5 18:51:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 205.139.110.120 as permitted sender) client-ip=205.139.110.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1573483519; cv=none; d=zoho.com; s=zohoarc; b=CKT4IlgKGXrXmnFkIcEfe+M9mSeBeco8ddFU5KCB79i2iKlWk3ir+fkWlxwDrL8jTFCgmfqyVUQxDbetzxXqOZBXoxHNCjBQc5aB3rn/99hNpGN80pfZAYTgwclYy/gtxFMl3cA47Jeula7al4V4TQly7WQaLHZTf8NjUwL3F1s= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573483519; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=fzxORFFNALx+Z5iDCZW8uwOJG0dR94LH9dEnH9D01CU=; b=F9ee8RLDB8LSOKQBSaUd6vy1GLFeoC8PjPawNnj+frZTw1mpmt+GA/vJAHeya4iygRi+e1gJJsYnVg06TbeDeHjidmq5wO8UzqVHXt9Q5ZrmfAB91DsMCYcnHRRCP372TMduEPeafiUHEbCTO1CF7+2bAq7wozX+FF0FDFHCoAE= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by mx.zohomail.com with SMTPS id 1573483519674376.26990707953644; Mon, 11 Nov 2019 06:45:19 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-412-lsaloS60NpKDcCkGNuuWdQ-1; Mon, 11 Nov 2019 09:45:15 -0500 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C3ED6DB30; Mon, 11 Nov 2019 14:45:06 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 894916015A; Mon, 11 Nov 2019 14:45:06 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 124A318034FE; Mon, 11 Nov 2019 14:45:06 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xABEit2X020379 for ; Mon, 11 Nov 2019 09:44:55 -0500 Received: by smtp.corp.redhat.com (Postfix) id F2A0E17AD4; Mon, 11 Nov 2019 14:44:54 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 611465D9C9; Mon, 11 Nov 2019 14:44:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573483518; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=fzxORFFNALx+Z5iDCZW8uwOJG0dR94LH9dEnH9D01CU=; b=Aucq3PZ+x/lLDPWv57hzPytL5XwpKQ34DLENrMNASWXp2oPY1nQGS3Z0Mhe7wWTixmTXS2 tOUv7ts4dIrnRJ78qV9VZMgalaFRe/Tlp4p8EXZPZfVA5nfJiSgibI1xtLJ9Xgbi0uQ2ml UvgJJu5WOun/pWhYmePl2I5Ty060Jes= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 11 Nov 2019 14:38:09 +0000 Message-Id: <20191111143826.16050-7-berrange@redhat.com> In-Reply-To: <20191111143826.16050-1-berrange@redhat.com> References: <20191111143826.16050-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v5 06/23] src: rewrite symfile sorting checker in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-MC-Unique: lsaloS60NpKDcCkGNuuWdQ-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" As part of an goal to eliminate Perl from libvirt build tools, rewrite the check-symsorting.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: J=C3=A1n Tomko Tested-by: Cole Robinson --- Makefile.am | 1 + scripts/check-symsorting.py | 117 ++++++++++++++++++++++++++++++++++++ src/Makefile.am | 4 +- src/admin/Makefile.inc.am | 2 +- src/check-symsorting.pl | 106 -------------------------------- 5 files changed, 121 insertions(+), 109 deletions(-) create mode 100755 scripts/check-symsorting.py delete mode 100755 src/check-symsorting.pl diff --git a/Makefile.am b/Makefile.am index ab9d09fcd4..b98c932ff7 100644 --- a/Makefile.am +++ b/Makefile.am @@ -48,6 +48,7 @@ EXTRA_DIST =3D \ scripts/augeas-gentest.py \ scripts/check-aclperms.py \ scripts/check-spacing.py \ + scripts/check-symsorting.py \ scripts/header-ifdef.py \ scripts/minimize-po.py \ scripts/mock-noinline.py \ diff --git a/scripts/check-symsorting.py b/scripts/check-symsorting.py new file mode 100755 index 0000000000..a6de8df9ec --- /dev/null +++ b/scripts/check-symsorting.py @@ -0,0 +1,117 @@ +#!/usr/bin/env python + +# Copyright (C) 2012-2019 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . + +from __future__ import print_function + +import os.path +import re +import sys + +if len(sys.argv) < 3: + print("syntax: %s SRCDIR SYMFILE..." % sys.argv[0], file=3Dsys.stderr) + sys.exit(1) + + +def check_sorting(group, symfile, line, groupfile, lastgroup): + sortedgroup =3D sorted(group, key=3Dstr.lower) + issorted =3D True + first =3D None + last =3D None + + err =3D False + # Check that groups are in order and groupfile exists + if lastgroup is not None and lastgroup.lower() > groupfile.lower(): + print("Symbol block at %s:%s: block not sorted" % + (symfile, line), file=3Dsys.stderr) + print("Move %s block before %s block" % + (groupfile, lastgroup), file=3Dsys.stderr) + print("", file=3Dsys.stderr) + err =3D True + + if not os.path.exists(os.path.join(srcdir, groupfile)): + print("Symbol block at %s:%s: %s not found" % + (symfile, line, groupfile), file=3Dsys.stderr) + print("", file=3Dsys.stderr) + err =3D True + + # Check that symbols within a group are in order + for i in range(len(group)): + if sortedgroup[i] !=3D group[i]: + if first is None: + first =3D i + + last =3D i + issorted =3D False + + if not issorted: + actual =3D group[first:(last-first+1)] + expect =3D sortedgroup[first:(last-first+1)] + print("Symbol block at %s:%s: symbols not sorted" % + (symfile, line), file=3Dsys.stderr) + for g in actual: + print(" %s" % g, file=3Dsys.stderr) + print("Correct ordering", file=3Dsys.stderr) + for g in expect: + print(" %s" % g, file=3Dsys.stderr) + print("", file=3Dsys.stderr) + err =3D True + + return err + + +ret =3D 0 +srcdir =3D sys.argv[1] +lastgroup =3D None +for symfile in sys.argv[2:]: + with open(symfile, "r") as fh: + lineno =3D 0 + groupfile =3D "" + group =3D [] + thisline =3D 0 + + for line in fh: + thisline =3D thisline + 1 + line =3D line.strip() + + filenamematch =3D re.search(r'''^#\s*((\w+\/)*(\w+\.h))\s*$'''= , line) + if filenamematch is not None: + groupfile =3D filenamematch.group(1) + elif line =3D=3D "": + if len(group) > 0: + if check_sorting(group, symfile, lineno, + groupfile, lastgroup): + ret =3D 1 + + group =3D [] + lineno =3D thisline + lastgroup =3D groupfile + elif line[0] =3D=3D '#': + # Ignore comments + pass + else: + line =3D line.strip(";") + group.append(line) + + if len(group) > 0: + if check_sorting(group, symfile, lineno, + groupfile, lastgroup): + ret =3D 1 + + lastgroup =3D None + +sys.exit(ret) diff --git a/src/Makefile.am b/src/Makefile.am index 318dd6c20f..41cfd79ad3 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -283,9 +283,9 @@ else ! WITH_LINUX check-symfile: endif ! WITH_LINUX check-symsorting: - $(AM_V_GEN)$(PERL) $(srcdir)/check-symsorting.pl \ + $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(top_srcdir)/scripts/check-symsorting.py= \ $(srcdir) $(SYM_FILES) -EXTRA_DIST +=3D check-symfile.pl check-symsorting.pl +EXTRA_DIST +=3D check-symfile.pl =20 # Keep this list synced with RPC_PROBE_FILES PROTOCOL_STRUCTS =3D \ diff --git a/src/admin/Makefile.inc.am b/src/admin/Makefile.inc.am index 3feb23aa20..97048df6f3 100644 --- a/src/admin/Makefile.inc.am +++ b/src/admin/Makefile.inc.am @@ -112,7 +112,7 @@ check-admin-symfile: endif ! WITH_LINUX =20 check-admin-symsorting: - $(AM_V_GEN)$(PERL) $(srcdir)/check-symsorting.pl \ + $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(top_srcdir)/scripts/check-symsorting.py= \ $(srcdir) $(ADMIN_SYM_FILES) =20 check-admin-drivername: diff --git a/src/check-symsorting.pl b/src/check-symsorting.pl deleted file mode 100755 index 51e38bdedb..0000000000 --- a/src/check-symsorting.pl +++ /dev/null @@ -1,106 +0,0 @@ -#!/usr/bin/env perl - -# Copyright (C) 2012-2013 Red Hat, Inc. -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library. If not, see -# . - -use strict; -use warnings; - -die "syntax: $0 SRCDIR SYMFILE..." unless int(@ARGV) >=3D 2; - -my $ret =3D 0; -my $srcdir =3D shift; -my $lastgroup =3D undef; -foreach my $symfile (@ARGV) { - open SYMFILE, $symfile or die "cannot read $symfile: $!"; - - my $line =3D 0; - my $groupfile =3D ""; - my @group; - - while () { - chomp; - - if (/^#\s*((\w+\/)*(\w+\.h))\s*$/) { - $groupfile =3D $1; - } elsif (/^#/) { - # Ignore comments - } elsif (/^\s*$/) { - if (@group) { - &check_sorting(\@group, $symfile, $line, $groupfile); - } - @group =3D (); - $line =3D $.; - } else { - $_ =3D~ s/;//; - push @group, $_; - } - } - - close SYMFILE; - if (@group) { - &check_sorting(\@group, $symfile, $line, $groupfile); - } - $lastgroup =3D undef; -} - -sub check_sorting { - my $group =3D shift; - my $symfile =3D shift; - my $line =3D shift; - my $groupfile =3D shift; - - my @group =3D @{$group}; - my @sorted =3D sort { lc $a cmp lc $b } @group; - my $sorted =3D 1; - my $first; - my $last; - - # Check that groups are in order and groupfile exists - if (defined $lastgroup && lc $lastgroup ge lc $groupfile) { - print "Symbol block at $symfile:$line: block not sorted\n"; - print "Move $groupfile block before $lastgroup block\n"; - print "\n"; - $ret =3D 1; - } - if (! -e "$srcdir/$groupfile") { - print "Symbol block at $symfile:$line: $groupfile not found\n"; - print "\n"; - $ret =3D 1; - } - $lastgroup =3D $groupfile; - - # Check that symbols within a group are in order - for (my $i =3D 0 ; $i <=3D $#sorted ; $i++) { - if ($sorted[$i] ne $group[$i]) { - $first =3D $i unless defined $first; - $last =3D $i; - $sorted =3D 0; - } - } - if (!$sorted) { - @group =3D splice @group, $first, ($last-$first+1); - @sorted =3D splice @sorted, $first, ($last-$first+1); - print "Symbol block at $symfile:$line: symbols not sorted\n"; - print map { " " . $_ . "\n" } @group; - print "Correct ordering\n"; - print map { " " . $_ . "\n" } @sorted; - print "\n"; - $ret =3D 1; - } -} - -exit $ret; --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sun May 5 18:51:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 207.211.31.120 as permitted sender) client-ip=207.211.31.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1573483513; cv=none; d=zoho.com; s=zohoarc; b=ZDTSrltz82SHjroExkINy78s4Zb8TSzP2ET7VfPnMRb3djeKoRRaodCZyJQ+RITwcJX6oyTg5a3ab53Ga8/ifYJVXorbM9xviAuto0siIKx7w9sqjbO2D4ICuuKfAKTuIbLCU+L3GTf9ZuQFtpkJEnzbyHrL+vNmR5jGF3b+VuM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573483513; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=V5ZUMmVWKYoNrNzXdRGVKG+2MdM6ZUFZNsZLhxjLgUY=; b=N7Xs6v7a/Wb+fzIjufCMqrf4KeptVbOp7fT/yYSp5SS5h1U8Ut1kK7Gpkb/zh6zke6vyUK6WKaN7fbuP7vXgsNUPTDRh3YhOVrK3ZKhzN+5lekNDM0PY+kNTwOhef9siXj1OzWWbvJ/A0f5aHQTNhXxy/NbevsDz2I6RZblg49Q= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by mx.zohomail.com with SMTPS id 1573483513626554.090889072312; Mon, 11 Nov 2019 06:45:13 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-81-0owh_V3DOFGrOJCfPy38DA-1; Mon, 11 Nov 2019 09:45:08 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 3160B18B9FC2; Mon, 11 Nov 2019 14:45:03 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id EE68C5DA7D; Mon, 11 Nov 2019 14:45:02 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 7712218034FA; Mon, 11 Nov 2019 14:45:02 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xABEittY020393 for ; Mon, 11 Nov 2019 09:44:55 -0500 Received: by smtp.corp.redhat.com (Postfix) id D49105DD73; Mon, 11 Nov 2019 14:44:55 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 43A0C5D9C9; Mon, 11 Nov 2019 14:44:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573483512; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=V5ZUMmVWKYoNrNzXdRGVKG+2MdM6ZUFZNsZLhxjLgUY=; b=BwoosdVqH9VeOR+DDUyXGns91SU0nTrjGEepLUpAgUcQo+yIN8vKLwbCurAHRsaN55eenK dtE0iYgvEKMquRGwVQUoZqt4a82ytAVQEMVAz3zA7V4o51vez+nVGvI8luB3GW+cW5emw2 zD9b5BBvc+Qtg2N7EK65/fG6l6NBXG8= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 11 Nov 2019 14:38:10 +0000 Message-Id: <20191111143826.16050-8-berrange@redhat.com> In-Reply-To: <20191111143826.16050-1-berrange@redhat.com> References: <20191111143826.16050-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v5 07/23] src: rewrite symfile library checker in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-MC-Unique: 0owh_V3DOFGrOJCfPy38DA-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" As part of an goal to eliminate Perl from libvirt build tools, rewrite the check-symfile.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: J=C3=A1n Tomko Tested-by: Cole Robinson --- Makefile.am | 1 + scripts/check-symfile.py | 80 +++++++++++++++++++++++++++++++++++++++ src/Makefile.am | 5 +-- src/admin/Makefile.inc.am | 4 +- src/check-symfile.pl | 70 ---------------------------------- 5 files changed, 85 insertions(+), 75 deletions(-) create mode 100755 scripts/check-symfile.py delete mode 100755 src/check-symfile.pl diff --git a/Makefile.am b/Makefile.am index b98c932ff7..1c7b8045c2 100644 --- a/Makefile.am +++ b/Makefile.am @@ -48,6 +48,7 @@ EXTRA_DIST =3D \ scripts/augeas-gentest.py \ scripts/check-aclperms.py \ scripts/check-spacing.py \ + scripts/check-symfile.py \ scripts/check-symsorting.py \ scripts/header-ifdef.py \ scripts/minimize-po.py \ diff --git a/scripts/check-symfile.py b/scripts/check-symfile.py new file mode 100755 index 0000000000..a543a0fbcd --- /dev/null +++ b/scripts/check-symfile.py @@ -0,0 +1,80 @@ +#!/usr/bin/env python +# +# Copyright (C) 2012-2019 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . + +from __future__ import print_function + +import re +import subprocess +import sys + +if len(sys.argv) < 3: + print("syntax: %s SYMFILE ELFLIB(S)" % sys.argv[0], file=3Dsys.stderr) + +symfile =3D sys.argv[1] +elflibs =3D sys.argv[2:] + +wantsyms =3D {} +gotsyms =3D {} + +ret =3D 0 + +with open(symfile, "r") as fh: + for line in fh: + line =3D line.strip() + if line.find("{") !=3D -1: + continue + if line.find("}") !=3D -1: + continue + if line in ["global:", "local:"]: + continue + if line =3D=3D "": + continue + if line[0] =3D=3D '#': + continue + if line.find("*") !=3D -1: + continue + + line =3D line.strip(";") + + if line in wantsyms: + print("Symbol $1 is listed twice", file=3Dsys.stderr) + ret =3D 1 + else: + wantsyms[line] =3D True + +for elflib in elflibs: + nm =3D subprocess.Popen(["nm", elflib], shell=3DFalse, + stdout=3Dsubprocess.PIPE).stdout + + for line in nm: + line =3D line.decode("utf-8") + symmatch =3D re.search(r'''^\S+\s(?:[TBD])\s(\S+)\s*$''', line) + if symmatch is None: + continue + + gotsyms[symmatch.group(1)] =3D True + + +for sym in wantsyms.keys(): + if sym in gotsyms: + continue + + print("Expected symbol '%s' is not in ELF library" % sym, file=3Dsys.s= tderr) + ret =3D 1 + +sys.exit(ret) diff --git a/src/Makefile.am b/src/Makefile.am index 41cfd79ad3..929eef784c 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -277,15 +277,14 @@ PDWTAGS =3D \ # rule for libvirt.la. However, checking symbols relies on Linux ELF layo= ut if WITH_LINUX check-symfile: libvirt.syms libvirt.la - $(AM_V_GEN)$(PERL) $(srcdir)/check-symfile.pl libvirt.syms \ - .libs/libvirt.so + $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(top_srcdir)/scripts/check-symfile.py \ + libvirt.syms .libs/libvirt.so else ! WITH_LINUX check-symfile: endif ! WITH_LINUX check-symsorting: $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(top_srcdir)/scripts/check-symsorting.py= \ $(srcdir) $(SYM_FILES) -EXTRA_DIST +=3D check-symfile.pl =20 # Keep this list synced with RPC_PROBE_FILES PROTOCOL_STRUCTS =3D \ diff --git a/src/admin/Makefile.inc.am b/src/admin/Makefile.inc.am index 97048df6f3..a02991df68 100644 --- a/src/admin/Makefile.inc.am +++ b/src/admin/Makefile.inc.am @@ -105,8 +105,8 @@ libvirt_admin_la_CFLAGS =3D \ =20 if WITH_LINUX check-admin-symfile: admin/libvirt_admin.syms libvirt-admin.la - $(AM_V_GEN)$(PERL) $(srcdir)/check-symfile.pl admin/libvirt_admin.syms \ - .libs/libvirt-admin.so + $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(top_srcdir)/scripts/check-symfile.py \ + admin/libvirt_admin.syms .libs/libvirt-admin.so else ! WITH_LINUX check-admin-symfile: endif ! WITH_LINUX diff --git a/src/check-symfile.pl b/src/check-symfile.pl deleted file mode 100755 index 4f88300864..0000000000 --- a/src/check-symfile.pl +++ /dev/null @@ -1,70 +0,0 @@ -#!/usr/bin/env perl - -# Copyright (C) 2012-2013 Red Hat, Inc. -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library. If not, see -# . - -die "syntax: $0 SYMFILE ELFLIB(S)" unless int(@ARGV) >=3D 2; - -my $symfile =3D shift @ARGV; -my @elflibs =3D @ARGV; - -my %wantsyms; -my %gotsyms; - -my $ret =3D 0; - -open SYMFILE, $symfile or die "cannot read $symfile: $!"; - -while () { - next if /{/; - next if /}/; - next if /global:/; - next if /local:/; - next if /^\s*$/; - next if /^\s*#/; - next if /\*/; - - die "malformed line $_" unless /^\s*(\S+);$/; - - if (exists $wantsyms{$1}) { - print STDERR "Symbol $1 is listed twice\n"; - $ret =3D 1; - } else { - $wantsyms{$1} =3D 1; - } -} -close SYMFILE; - -foreach my $elflib (@elflibs) { - open NM, "-|", "nm", $elflib or die "cannot run 'nm $elflib': $!"; - - while () { - next unless /^\S+\s(?:[TBD])\s(\S+)\s*$/; - - $gotsyms{$1} =3D 1; - } - - close NM; -} - -foreach my $sym (keys(%wantsyms)) { - next if exists $gotsyms{$sym}; - - print STDERR "Expected symbol $sym is not in ELF library\n"; - $ret =3D 1; -} - -exit($ret); --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sun May 5 18:51:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1573483566; cv=none; d=zoho.com; s=zohoarc; b=m09G2M55PfLQtmIW7X3nElb2NbJfvHQo3Tx7Q9I3vDDYnNafhe0/9T97hiymp8Az+hWNHa0QoWaowG/0gYJeEFer0lK++hm9wQna306PE4NHPcxwdlPLLhOiQdh7VjBsRLvc38kZB20GN7XR29AQAAE+Mc3wM7XX83GlqoOs4rQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573483566; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=9xx9t91Pc34b3dXUUSRFXWxMDqwi2bsM8Qz3pPOU1Es=; b=ToLalnrLpjX+5UyU7TMjEB3slzpcAPbTv6Wd0N82Kb0EJeGpPHolCUiuCXhd//HQ3+BOpeJkkaTRkdbOT5h0m3gQx0NokUeD55gUuLVeK7F7PZVUz1rJM62sVnIUoRyvOw3nJ6Oe4orteWtytpwNhny8cZKkfl+9Coh1oRSdWV4= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 1573483566390135.12072237097698; Mon, 11 Nov 2019 06:46:06 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-73-u9ADSBE5PqaFfG9L_RkL_w-1; Mon, 11 Nov 2019 09:45:20 -0500 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B9B70100551C; Mon, 11 Nov 2019 14:45:13 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7C4FE4145; Mon, 11 Nov 2019 14:45:13 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 0C32A1803509; Mon, 11 Nov 2019 14:45:13 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xABEiuxq020400 for ; Mon, 11 Nov 2019 09:44:56 -0500 Received: by smtp.corp.redhat.com (Postfix) id B547E5D9C9; Mon, 11 Nov 2019 14:44:56 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 252A918340; Mon, 11 Nov 2019 14:44:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573483565; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=9xx9t91Pc34b3dXUUSRFXWxMDqwi2bsM8Qz3pPOU1Es=; b=DomlqMPtmvxTCFAGDk18/eMruSpZY/jtGen2YnfJNY0DSxTNLz3JDCOETzI6xH3IodczYX XnL501LNq0GPKpGgdiWEkwZagftsk9bFb+uyQ5TSdBcQzJGoGQ138SFaoTrAgZH2/T1R7p ThM549yYU+nyoY668yQ0laf24/ywspo= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 11 Nov 2019 14:38:11 +0000 Message-Id: <20191111143826.16050-9-berrange@redhat.com> In-Reply-To: <20191111143826.16050-1-berrange@redhat.com> References: <20191111143826.16050-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v5 08/23] src: rewrite systemtap probe generator in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-MC-Unique: u9ADSBE5PqaFfG9L_RkL_w-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" As part of an goal to eliminate Perl from libvirt build tools, rewrite the dtrace2systemtap.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: J=C3=A1n Tomko Tested-by: Cole Robinson --- Makefile.am | 1 + scripts/dtrace2systemtap.py | 143 ++++++++++++++++++++++++++++++++++++ src/Makefile.am | 10 +-- src/dtrace2systemtap.pl | 130 -------------------------------- 4 files changed, 147 insertions(+), 137 deletions(-) create mode 100755 scripts/dtrace2systemtap.py delete mode 100755 src/dtrace2systemtap.pl diff --git a/Makefile.am b/Makefile.am index 1c7b8045c2..e50a29635b 100644 --- a/Makefile.am +++ b/Makefile.am @@ -50,6 +50,7 @@ EXTRA_DIST =3D \ scripts/check-spacing.py \ scripts/check-symfile.py \ scripts/check-symsorting.py \ + scripts/dtrace2systemtap.py \ scripts/header-ifdef.py \ scripts/minimize-po.py \ scripts/mock-noinline.py \ diff --git a/scripts/dtrace2systemtap.py b/scripts/dtrace2systemtap.py new file mode 100755 index 0000000000..922713e599 --- /dev/null +++ b/scripts/dtrace2systemtap.py @@ -0,0 +1,143 @@ +#!/usr/bin/env python +# +# Copyright (C) 2011-2019 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . +# +# +# Generate a set of systemtap probe definitions corresponding to +# DTrace probe markers in libvirt.so +# +# python dtrace2systemtap.py probes.d > libvirt_probes.stp +# + +from __future__ import print_function + +import re +import sys + +file =3D None +filelist =3D [] +files =3D {} + +bindir =3D sys.argv[1] +sbindir =3D sys.argv[2] +libdir =3D sys.argv[3] +dtrace =3D sys.argv[4] + +probe =3D None +args =3D None + +# Read the DTraceprobes definition + +with open(dtrace, "r") as fh: + lineno =3D 0 + for line in fh: + lineno =3D lineno + 1 + line =3D line.strip() + if line =3D=3D "": + continue + if line.find("provider ") !=3D -1 and line.find("{") !=3D -1: + continue + if line.find("};") !=3D -1: + continue + + if line.startswith("#"): + m =3D re.search(r'''^\#\s*file:\s*(\S+)$''', line) + if m is not None: + file =3D m.group(1) + filelist.append(file) + files[file] =3D {"prefix": None, "probes": []} + continue + + m =3D re.search(r'''^\#\s*prefix:\s*(\S+)$''', line) + if m is not None: + files[file]["prefix"] =3D m.group(1) + continue + + m =3D re.search(r'''^\#\s*binary:\s*(\S+)$''', line) + if m is not None: + files[file]["binary"] =3D m.group(1) + continue + + m =3D re.search(r'''^\#\s*module:\s*(\S+)$''', line) + if m is not None: + files[file]["module"] =3D m.group(1) + + # ignore unknown comments + else: + m =3D re.search(r'''probe\s+([a-zA-Z0-9_]+)\((.*?)(\);)?$''', = line) + if m is not None: + probe =3D m.group(1) + args =3D m.group(2) + if m.group(3) is not None: + files[file]["probes"].append([probe, args]) + probe =3D None + args =3D None + elif probe is not None: + m =3D re.search(r'''^(.*?)(\);)?$''', line) + if m is not None: + args =3D args + m.group(1) + if m.group(2) is not None: + files[file]["probes"].append([probe, args]) + probe =3D None + args =3D None + else: + raise Exception("unexpected data %s on line %d" % + (line, lineno)) + else: + raise Exception("unexpected data %s on line %d" % + (line, lineno)) + +# Write out the SystemTap probes +for file in filelist: + prefix =3D files[file]["prefix"] + probes =3D files[file]["probes"] + + print("# %s\n" % file) + for probe in probes: + name =3D probe[0] + args =3D probe[1] + + pname =3D name.replace(prefix + "_", "libvirt." + prefix + ".") + + binary =3D libdir + "/libvirt.so" + if "binary" in files[file]: + binary =3D sbindir + "/" + files[file]["binary"] + if "module" in files[file]: + binary =3D libdir + "/" + files[file]["module"] + + print("probe %s =3D process(\"%s\").mark(\"%s\") {" % + (pname, binary, name)) + + argbits =3D args.split(",") + for idx in range(len(argbits)): + arg =3D argbits[idx] + isstr =3D False + if arg.find("char *") !=3D -1: + isstr =3D True + + m =3D re.search(r'''^.*\s\*?(\S+)$''', arg) + if m is not None: + arg =3D m.group(1) + else: + raise Exception("Malformed arg %s" % arg) + + if isstr: + print(" %s =3D user_string($arg%d);" % (arg, idx + 1)) + else: + print(" %s =3D $arg%d;" % (arg, idx + 1)) + print("}\n") + print("") diff --git a/src/Makefile.am b/src/Makefile.am index 929eef784c..085c193013 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -530,7 +530,6 @@ if WITH_DTRACE_PROBES libvirt_la_BUILT_LIBADD +=3D libvirt_probes.lo libvirt_la_DEPENDENCIES +=3D libvirt_probes.lo libvirt_probes.o nodist_libvirt_la_SOURCES =3D libvirt_probes.h -DTRACE2SYSTEMTAP_FLAGS =3D --with-modules =20 BUILT_SOURCES +=3D libvirt_probes.h libvirt_probes.stp libvirt_functions.s= tp =20 @@ -565,10 +564,10 @@ RPC_PROBE_FILES +=3D $(srcdir)/rpc/virnetprotocol.x \ libvirt_functions.stp: $(RPC_PROBE_FILES) $(srcdir)/rpc/gensystemtap.pl $(AM_V_GEN)$(PERL) -w $(srcdir)/rpc/gensystemtap.pl $(RPC_PROBE_FILES) > = $@ =20 -%_probes.stp: %_probes.d $(srcdir)/dtrace2systemtap.pl \ +%_probes.stp: %_probes.d $(top_srcdir)/scripts/dtrace2systemtap.py \ $(top_builddir)/config.status - $(AM_V_GEN)$(PERL) -w $(srcdir)/dtrace2systemtap.pl \ - $(DTRACE2SYSTEMTAP_FLAGS) $(bindir) $(sbindir) $(libdir) $< > $@ + $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(top_srcdir)/scripts/dtrace2systemtap.py= \ + $(bindir) $(sbindir) $(libdir) $< > $@ =20 CLEANFILES +=3D libvirt_probes.h libvirt_probes.o libvirt_probes.lo \ libvirt_functions.stp libvirt_probes.stp @@ -689,9 +688,6 @@ endif LIBVIRT_INIT_SCRIPT_SYSTEMD endif WITH_LIBVIRTD =20 =20 -EXTRA_DIST +=3D dtrace2systemtap.pl - - if WITH_LIBVIRTD libexec_PROGRAMS +=3D libvirt_iohelper libvirt_iohelper_SOURCES =3D $(UTIL_IO_HELPER_SOURCES) diff --git a/src/dtrace2systemtap.pl b/src/dtrace2systemtap.pl deleted file mode 100755 index c5fce248b4..0000000000 --- a/src/dtrace2systemtap.pl +++ /dev/null @@ -1,130 +0,0 @@ -#!/usr/bin/env perl -# -# Copyright (C) 2011-2012 Red Hat, Inc. -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library. If not, see -# . -# -# -# Generate a set of systemtap probe definitions corresponding to -# DTrace probe markers in libvirt.so -# -# perl dtrace2systemtap.pl probes.d > libvirt_probes.stp -# - -use strict; -use warnings; - -my $file; -my @files; -my %files; - -my $with_modules =3D 0; -if ($ARGV[0] eq "--with-modules") { - # set if we want to honor the "module" setting in the .d file - $with_modules =3D 1; - shift @ARGV; -} - -my $bindir =3D shift @ARGV; -my $sbindir =3D shift @ARGV; -my $libdir =3D shift @ARGV; - -my $probe; -my $args; - -# Read the DTraceprobes definition -while (<>) { - next if m,^\s*$,; - - next if /^\s*provider\s+\w+\s*{\s*$/; - next if /^\s*};\s*$/; - - if (m,^\s*\#,) { - if (m,^\s*\#\s*file:\s*(\S+)\s*$,) { - $file =3D $1; - push @files, $file; - $files{$file} =3D { prefix =3D> undef, probes =3D> [] }; - } elsif (m,^\s*\#\s*prefix:\s*(\S+)\s*$,) { - $files{$file}->{prefix} =3D $1; - } elsif (m,^\s*\#\s*binary:\s*(\S+)\s*$,) { - $files{$file}->{binary} =3D $1; - } elsif (m,^\s*\#\s*module:\s*(\S+)\s*$,) { - $files{$file}->{module} =3D $1; - } else { - # ignore unknown comments - } - } else { - if (m,\s*probe\s+([a-zA-Z0-9_]+)\((.*?)(\);)?$,) { - $probe =3D $1; - $args =3D $2; - if ($3) { - push @{$files{$file}->{probes}}, [$probe, $args]; - $probe =3D $args =3D undef; - } - } elsif ($probe) { - if (m,^(.*?)(\);)?$,) { - $args .=3D $1; - if ($2) { - push @{$files{$file}->{probes}}, [$probe, $args]; - $probe =3D $args =3D undef; - } - } else { - die "unexpected data $_ on line $."; - } - } else { - die "unexpected data $_ on line $."; - } - } -} - -# Write out the SystemTap probes -foreach my $file (@files) { - my $prefix =3D $files{$file}->{prefix}; - my @probes =3D @{$files{$file}->{probes}}; - - print "# $file\n\n"; - foreach my $probe (@probes) { - my $name =3D $probe->[0]; - my $args =3D $probe->[1]; - - my $pname =3D $name; - $pname =3D~ s/${prefix}_/libvirt.$prefix./; - - my $binary =3D "$libdir/libvirt.so"; - if (exists $files{$file}->{binary}) { - $binary =3D $sbindir . "/" . $files{$file}->{binary}; - } - if ($with_modules && exists $files{$file}->{module}) { - $binary =3D $libdir . "/" . $files{$file}->{module}; - } - - print "probe $pname =3D process(\"$binary\").mark(\"$name\") {\n"; - - my @args =3D split /,/, $args; - for (my $i =3D 0 ; $i <=3D $#args ; $i++) { - my $arg =3D $args[$i]; - my $isstr =3D $arg =3D~ /char\s+\*/; - $arg =3D~ s/^.*\s\*?(\S+)$/$1/; - - if ($isstr) { - print " $arg =3D user_string(\$arg", $i + 1, ");\n"; - } else { - print " $arg =3D \$arg", $i + 1, ";\n"; - } - } - print "}\n\n"; - } - print "\n"; -} --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sun May 5 18:51:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 205.139.110.61 as permitted sender) client-ip=205.139.110.61; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1573483567; cv=none; d=zoho.com; s=zohoarc; b=at55jwYYVAP5Fr8a2EzD5/EiX/V6jQphVWi4C6OvLZ40PIGsyuAIono8FivE7HbiU55gM3icdmnN7Nb/iGjbEjXUA+NZfEtnpuUDo6haQILODtM+2ImaprQSYCGqIPcTLUuLu3PXFS2wsVIFbYCRvr07WLTNEhwed5kAIEcvxQ8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573483567; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=lyq2/blq39jNc9wVFFN7LBUNxOpRxUnX0rcr9xRSuq4=; b=BYgjoEFM+PuGuLcoGRdpzCVxrBzmrL+Cn5O1SNpT/42YdojprLXVr8trbnB+ahJoxYf+/jFbt49t9eOXIad26HX9HEik7GGfP57oDMD+aqE2GJSzmR4Nbd9FuzfbwK1WQptSPiVbjcG+dwUvrMwTNi5CW8Pdrlr9lYvp0LMdM8o= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) by mx.zohomail.com with SMTPS id 1573483567490284.4211407592377; Mon, 11 Nov 2019 06:46:07 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-219-UxU43hYUMpqk62okoMN2BA-1; Mon, 11 Nov 2019 09:45:15 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.phx2.redhat.com [10.5.11.23]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BF22C86A06F; Mon, 11 Nov 2019 14:45:06 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7D512798E; Mon, 11 Nov 2019 14:45:06 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 078F94E571; Mon, 11 Nov 2019 14:45:06 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xABEivLk020413 for ; Mon, 11 Nov 2019 09:44:57 -0500 Received: by smtp.corp.redhat.com (Postfix) id 96AE417AD4; Mon, 11 Nov 2019 14:44:57 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 070C95D9C9; Mon, 11 Nov 2019 14:44:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573483566; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=lyq2/blq39jNc9wVFFN7LBUNxOpRxUnX0rcr9xRSuq4=; b=Fqrt5CZf+yuxH5q2JqyQMphawjsS/y3/IhqPBk0EKRyFi7E9R4gi2w3I2hfqlcrPetkih2 saAsDTtduqAwB/HDvahI2bYnT7jZ33u/7fl8TXNeBSJEioOvRseNCBRB2h0m8lQJPvoyOX RCfYTu3yhCqRNA4sVQt0Ie9IkkwhLpg= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 11 Nov 2019 14:38:12 +0000 Message-Id: <20191111143826.16050-10-berrange@redhat.com> In-Reply-To: <20191111143826.16050-1-berrange@redhat.com> References: <20191111143826.16050-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v5 09/23] src: rewrite systemtap function generator in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.23 X-MC-Unique: UxU43hYUMpqk62okoMN2BA-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" As part of an goal to eliminate Perl from libvirt build tools, rewrite the gensystemtap.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: J=C3=A1n Tomko Tested-by: Cole Robinson --- Makefile.am | 1 + build-aux/syntax-check.mk | 4 +- scripts/gensystemtap.py | 184 ++++++++++++++++++++++++++++++++++++ src/Makefile.am | 5 +- src/rpc/Makefile.inc.am | 1 - src/rpc/gensystemtap.pl | 193 -------------------------------------- 6 files changed, 189 insertions(+), 199 deletions(-) create mode 100755 scripts/gensystemtap.py delete mode 100755 src/rpc/gensystemtap.pl diff --git a/Makefile.am b/Makefile.am index e50a29635b..afed409c94 100644 --- a/Makefile.am +++ b/Makefile.am @@ -51,6 +51,7 @@ EXTRA_DIST =3D \ scripts/check-symfile.py \ scripts/check-symsorting.py \ scripts/dtrace2systemtap.py \ + scripts/gensystemtap.py \ scripts/header-ifdef.py \ scripts/minimize-po.py \ scripts/mock-noinline.py \ diff --git a/build-aux/syntax-check.mk b/build-aux/syntax-check.mk index 016e05e7a2..26eb5b94d9 100644 --- a/build-aux/syntax-check.mk +++ b/build-aux/syntax-check.mk @@ -466,6 +466,7 @@ sc_prohibit_risky_id_promotion: # since gnulib has more guarantees for snprintf portability sc_prohibit_sprintf: @prohibit=3D'\<[s]printf\>' \ + in_vc_files=3D'\.[ch]$$' \ halt=3D'use snprintf, not sprintf' \ $(_sc_search_regexp) =20 @@ -2261,9 +2262,6 @@ exclude_file_name_regexp--sc_prohibit_readlink =3D \ =20 exclude_file_name_regexp--sc_prohibit_setuid =3D ^src/util/virutil\.c|tool= s/virt-login-shell\.c$$ =20 -exclude_file_name_regexp--sc_prohibit_sprintf =3D \ - ^(build-aux/syntax-check\.mk|docs/hacking\.html\.in|.*\.stp|.*\.pl)$$ - exclude_file_name_regexp--sc_prohibit_strncpy =3D ^src/util/virstring\.c$$ =20 exclude_file_name_regexp--sc_prohibit_strtol =3D ^examples/.*$$ diff --git a/scripts/gensystemtap.py b/scripts/gensystemtap.py new file mode 100755 index 0000000000..c4bc1620d0 --- /dev/null +++ b/scripts/gensystemtap.py @@ -0,0 +1,184 @@ +#!/usr/bin/env python +# +# Copyright (C) 2011-2019 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . +# +# Generate a set of systemtap functions for translating various +# RPC enum values into strings +# +# python gensystemtap.py */*.x > libvirt_functions.stp +# + +from __future__ import print_function + +import re +import sys + +funcs =3D {} + +types =3D {} +status =3D {} +auth =3D {} + + +def load_file(fh): + instatus =3D False + intype =3D False + inauth =3D False + + for line in fh: + if re.search(r'''enum\s+virNetMessageType''', line): + intype =3D True + elif re.search(r'''enum\s+virNetMessageStatus''', line): + instatus =3D True + elif re.search(r'''enum remote_auth_type''', line): + inauth =3D True + elif line.find("}") !=3D -1: + intype =3D False + instatus =3D False + inauth =3D False + elif instatus: + m =3D re.search(r'''^\s+VIR_NET_(\w+)\s*=3D\s*(\d+),?$''', lin= e) + if m is not None: + status[m.group(2)] =3D m.group(1).lower() + elif intype: + m =3D re.search(r'''^\s+VIR_NET_(\w+)\s*=3D\s*(\d+),?$''', lin= e) + if m is not None: + types[m.group(2)] =3D m.group(1).lower() + elif inauth: + m =3D re.search(r'''^\s+REMOTE_AUTH_(\w+)\s*=3D\s*(\d+),?$''',= line) + if m is not None: + auth[m.group(2)] =3D m.group(1).lower() + else: + m =3D re.search(r'''(?:VIR_)?(\w+?)(?:_PROTOCOL)?''' + + r'''_PROGRAM\s*=3D\s*0x([a-fA-F0-9]+)\s*;''', li= ne) + if m is not None: + funcs[m.group(1).lower()] =3D { + "id": int(m.group(2), 16), + "version": None, + "progs": [] + } + continue + + m =3D re.search(r'''(?:VIR_)?(\w+?)(?:_PROTOCOL)?_''' + + r'''(?:PROGRAM|PROTOCOL)_VERSION\s*=3D\s*(\d+)\s= *;''', + line) + if m is not None: + funcs[m.group(1).lower()]["version"] =3D m.group(2) + continue + + m =3D re.search(r'''(?:VIR_)?(\w+?)(?:_PROTOCOL)?''' + + r'''_PROC_(.*?)\s+=3D\s+(\d+)''', line) + if m is not None: + funcs[m.group(1).lower()]["progs"].insert( + int(m.group(3)), m.group(2).lower()) + + +for file in sys.argv[1:]: + with open(file, "r") as fh: + load_file(fh) + + +def genfunc(name, varname, types): + print("function %s(%s, verbose)" % (name, varname)) + print("{") + + first =3D True + for typename in sorted(types.keys()): + cond =3D "} else if" + if first: + cond =3D "if" + first =3D False + + print(" %s (%s =3D=3D %s) {" % (cond, varname, typename)) + print(" %sstr =3D \"%s\"" % (varname, types[typename])) + + print(" } else {") + print(" %sstr =3D \"unknown\";" % varname) + print(" verbose =3D 1;") + print(" }") + print(" if (verbose) {") + print(" %sstr =3D %sstr . sprintf(\":%%d\", %s)" % + (varname, varname, varname)) + print(" }") + print(" return %sstr;" % varname) + print("}") + + +genfunc("libvirt_rpc_auth_name", "type", auth) +genfunc("libvirt_rpc_type_name", "type", types) +genfunc("libvirt_rpc_status_name", "status", status) + +print("function libvirt_rpc_program_name(program, verbose)") +print("{") + +first =3D True +for funcname in sorted(funcs.keys()): + cond =3D "} else if" + if first: + cond =3D "if" + first =3D False + + print(" %s (program =3D=3D %s) {" % (cond, funcs[funcname]["id"])) + print(" programstr =3D \"%s\"" % funcname) + +print(" } else {") +print(" programstr =3D \"unknown\";") +print(" verbose =3D 1;") +print(" }") +print(" if (verbose) {") +print(" programstr =3D programstr . sprintf(\":%d\", program)") +print(" }") +print(" return programstr;") +print("}") + +print("function libvirt_rpc_procedure_name(program, version, proc, verbose= )") +print("{") + +first =3D True +for prog in sorted(funcs.keys()): + cond =3D "} else if" + if first: + cond =3D "if" + first =3D False + + print(" %s (program =3D=3D %s && version =3D=3D %s) {" % + (cond, funcs[prog]["id"], funcs[prog]["version"])) + + pfirst =3D True + for id in range(len(funcs[prog]["progs"])): + pcond =3D "} else if" + if pfirst: + pcond =3D "if" + pfirst =3D False + + print(" %s (proc =3D=3D %s) {" % (pcond, id + 1)) + print(" procstr =3D \"%s\";" % funcs[prog]["progs"][id]) + + print(" } else {") + print(" procstr =3D \"unknown\";") + print(" verbose =3D 1;") + print(" }") + +print(" } else {") +print(" procstr =3D \"unknown\";") +print(" verbose =3D 1;") +print(" }") +print(" if (verbose) {") +print(" procstr =3D procstr . sprintf(\":%d\", proc)") +print(" }") +print(" return procstr;") +print("}") diff --git a/src/Makefile.am b/src/Makefile.am index 085c193013..b0ca9d3442 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -561,8 +561,9 @@ RPC_PROBE_FILES +=3D $(srcdir)/rpc/virnetprotocol.x \ $(srcdir)/remote/qemu_protocol.x \ $(srcdir)/admin/admin_protocol.x =20 -libvirt_functions.stp: $(RPC_PROBE_FILES) $(srcdir)/rpc/gensystemtap.pl - $(AM_V_GEN)$(PERL) -w $(srcdir)/rpc/gensystemtap.pl $(RPC_PROBE_FILES) > = $@ +libvirt_functions.stp: $(RPC_PROBE_FILES) $(top_srcdir)/scripts/gensystemt= ap.py + $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(top_srcdir)/scripts/gensystemtap.py \ + $(RPC_PROBE_FILES) > $@ =20 %_probes.stp: %_probes.d $(top_srcdir)/scripts/dtrace2systemtap.py \ $(top_builddir)/config.status diff --git a/src/rpc/Makefile.inc.am b/src/rpc/Makefile.inc.am index a5bde92c9f..11920b3270 100644 --- a/src/rpc/Makefile.inc.am +++ b/src/rpc/Makefile.inc.am @@ -3,7 +3,6 @@ EXTRA_DIST +=3D \ rpc/gendispatch.pl \ rpc/genprotocol.pl \ - rpc/gensystemtap.pl \ rpc/virnetprotocol.x \ rpc/virkeepaliveprotocol.x \ $(NULL) diff --git a/src/rpc/gensystemtap.pl b/src/rpc/gensystemtap.pl deleted file mode 100755 index 6693d4d6f5..0000000000 --- a/src/rpc/gensystemtap.pl +++ /dev/null @@ -1,193 +0,0 @@ -#!/usr/bin/env perl -# -# Copyright (C) 2011-2012 Red Hat, Inc. -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library. If not, see -# . -# -# Generate a set of systemtap functions for translating various -# RPC enum values into strings -# -# perl gensystemtap.pl */*.x > libvirt_functions.stp -# - -use strict; - -my %funcs; - -my %type; -my %status; -my %auth; - -my $instatus =3D 0; -my $intype =3D 0; -my $inauth =3D 0; -while (<>) { - if (/enum\s+virNetMessageType/) { - $intype =3D 1; - } elsif (/enum\s+virNetMessageStatus/) { - $instatus =3D 1; - } elsif (/enum remote_auth_type/) { - $inauth =3D 1; - } elsif (/}/) { - $instatus =3D $intype =3D $inauth =3D 0; - } elsif ($instatus) { - if (/^\s+VIR_NET_(\w+)\s*=3D\s*(\d+),?$/) { - $status{$2} =3D lc $1; - } - } elsif ($intype) { - if (/^\s+VIR_NET_(\w+)\s*=3D\s*(\d+),?$/) { - $type{$2} =3D lc $1; - } - } elsif ($inauth) { - if (/^\s+REMOTE_AUTH_(\w+)\s*=3D\s*(\d+),?$/) { - $auth{$2} =3D lc $1; - } - } else { - if (/(?:VIR_)?(\w+?)(?:_PROTOCOL)?_PROGRAM\s*=3D\s*0x([a-fA-F0-9]+= )\s*;/) { - $funcs{lc $1} =3D { id =3D> hex($2), version =3D> undef, progs= =3D> [] }; - } elsif (/(?:VIR_)?(\w+?)(?:_PROTOCOL)?_(?:PROGRAM|PROTOCOL)_VERSI= ON\s*=3D\s*(\d+)\s*;/) { - $funcs{lc $1}->{version} =3D $2; - } elsif (/(?:VIR_)?(\w+?)(?:_PROTOCOL)?_PROC_(.*?)\s+=3D\s+(\d+)/)= { - $funcs{lc $1}->{progs}->[$3] =3D lc $2; - } - } -} - -print <{id}, ") {\n"; - print " programstr =3D \"", $prog, "\"\n"; -} -print <{id}, " && version = =3D=3D ", $funcs{$prog}->{version}, ") {\n"; - - my $pfirst =3D 1; - for (my $id =3D 1 ; $id <=3D $#{$funcs{$prog}->{progs}} ; $id++) { - my $cond =3D $pfirst ? "if" : "} else if"; - $pfirst =3D 0; - print " $cond (proc =3D=3D $id) {\n"; - print " procstr =3D \"", $funcs{$prog}->{progs}->[$id], "= \";\n"; - } - print " } else {\n"; - print " procstr =3D \"unknown\";\n"; - print " verbose =3D 1;\n"; - print " }\n"; -} -print < (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by mx.zohomail.com with SMTPS id 1573483566132728.2511329137852; Mon, 11 Nov 2019 06:46:06 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-408-6w2w1zemO7qHrKYxII_8IQ-1; Mon, 11 Nov 2019 09:45:19 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A0AA38A2432; Mon, 11 Nov 2019 14:45:12 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 68B6B4B4; Mon, 11 Nov 2019 14:45:12 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 0BD8818034FF; Mon, 11 Nov 2019 14:45:12 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xABEiwdj020421 for ; Mon, 11 Nov 2019 09:44:58 -0500 Received: by smtp.corp.redhat.com (Postfix) id 7689A5DD73; Mon, 11 Nov 2019 14:44:58 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id DB7E55D9C9; Mon, 11 Nov 2019 14:44:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573483564; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=BrHmO8wAkZhGm6Q6M7Amh1U6xWTGJ/hd3vsu7fxpNf0=; b=DOq4fE/THCWM41xCeXkbj/OnuafsY35qZa6DyPCSBDhV37czoY4uBeH/fbeWIVGB8mm0Of PpduAqqN+ugKOfB3x7BlTJd4aUpiM8+QUDf5IwlkKvAs4nNxY82aJB2+fuRqVIWQplaUSJ q0Cb9P2MTBE2hs7jIM31JghQ9jP4mVY= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 11 Nov 2019 14:38:13 +0000 Message-Id: <20191111143826.16050-11-berrange@redhat.com> In-Reply-To: <20191111143826.16050-1-berrange@redhat.com> References: <20191111143826.16050-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v5 10/23] src: rewrite driver name checker in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-MC-Unique: 6w2w1zemO7qHrKYxII_8IQ-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" As part of an goal to eliminate Perl from libvirt build tools, rewrite the check-drivername.pl tool in Python. This was mostly a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. In testing though it was discovered the existing code was broken since it hadn't been updated after driver.h was split into many files. Since the old code is being thrown away, the fix was done as part of the rewrite rather than split into a separate commit. Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: J=C3=A1n Tomko Tested-by: Cole Robinson --- Makefile.am | 1 + scripts/check-drivername.py | 114 ++++++++++++++++++++++++++++++++++++ src/Makefile.am | 18 ++++-- src/check-drivername.pl | 83 -------------------------- 4 files changed, 129 insertions(+), 87 deletions(-) create mode 100644 scripts/check-drivername.py delete mode 100755 src/check-drivername.pl diff --git a/Makefile.am b/Makefile.am index afed409c94..7155ab6ce9 100644 --- a/Makefile.am +++ b/Makefile.am @@ -47,6 +47,7 @@ EXTRA_DIST =3D \ AUTHORS.in \ scripts/augeas-gentest.py \ scripts/check-aclperms.py \ + scripts/check-drivername.py \ scripts/check-spacing.py \ scripts/check-symfile.py \ scripts/check-symsorting.py \ diff --git a/scripts/check-drivername.py b/scripts/check-drivername.py new file mode 100644 index 0000000000..3226ee7962 --- /dev/null +++ b/scripts/check-drivername.py @@ -0,0 +1,114 @@ +#!/usr/bin/env python +# +# Copyright (C) 2013-2019 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . +# + +from __future__ import print_function + +import re +import sys + +drvfiles =3D [] +symfiles =3D [] +for arg in sys.argv: + if arg.endswith(".h"): + drvfiles.append(arg) + else: + symfiles.append(arg) + +symbols =3D {} + +for symfile in symfiles: + with open(symfile, "r") as fh: + for line in fh: + m =3D re.search(r'''^\s*(vir\w+)\s*;\s*$''', line) + if m is not None: + symbols[m.group(1)] =3D True + +status =3D 0 +for drvfile in drvfiles: + with open(drvfile, "r") as fh: + for line in fh: + if line.find("virDrvConnectSupportsFeature") !=3D -1: + continue + + m =3D re.search(r'''\*(virDrv\w+)\s*\)''', line) + if m is not None: + drv =3D m.group(1) + + skip =3D [ + "virDrvStateInitialize", + "virDrvStateCleanup", + "virDrvStateReload", + "virDrvStateStop", + "virDrvConnectURIProbe", + "virDrvDomainMigratePrepare", + "virDrvDomainMigratePrepare2", + "virDrvDomainMigratePrepare3", + "virDrvDomainMigratePrepare3Params", + "virDrvDomainMigratePrepareTunnel", + "virDrvDomainMigratePrepareTunnelParams", + "virDrvDomainMigratePrepareTunnel3", + "virDrvDomainMigratePrepareTunnel3Params", + "virDrvDomainMigratePerform", + "virDrvDomainMigratePerform3", + "virDrvDomainMigratePerform3Params", + "virDrvDomainMigrateConfirm", + "virDrvDomainMigrateConfirm3", + "virDrvDomainMigrateConfirm3Params", + "virDrvDomainMigrateBegin", + "virDrvDomainMigrateBegin3", + "virDrvDomainMigrateBegin3Params", + "virDrvDomainMigrateFinish", + "virDrvDomainMigrateFinish2", + "virDrvDomainMigrateFinish3", + "virDrvDomainMigrateFinish3Params", + "virDrvStreamInData", + ] + if drv in skip: + continue + + sym =3D drv.replace("virDrv", "vir") + + if sym not in symbols: + print("Driver method name %s doesn't match public API"= % + drv) + continue + + m =3D re.search(r'''^\*(vir\w+)\s*\)''', line) + if m is not None: + name =3D m.group(1) + print("Bogus name %s" % name) + status =3D 1 + continue + + m =3D re.search(r'''^\s*(virDrv\w+)\s+(\w+);\s*''', line) + if m is not None: + drv =3D m.group(1) + field =3D m.group(2) + + tmp =3D drv.replace("virDrv", "") + if tmp.startswith("NWFilter"): + tmp =3D "nwfilter" + tmp[8:] + tmp =3D tmp[0:1].lower() + tmp[1:] + + if tmp !=3D field: + print("Driver struct field %s should be named %s" % + (field, tmp)) + status =3D 1 + +sys.exit(status) diff --git a/src/Makefile.am b/src/Makefile.am index b0ca9d3442..55ad51abf1 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -330,15 +330,25 @@ check-protocol: endif !WITH_REMOTE EXTRA_DIST +=3D $(PROTOCOL_STRUCTS) =20 +DRIVERS =3D \ + $(srcdir)/driver-hypervisor.h \ + $(srcdir)/driver-interface.h \ + $(srcdir)/driver-network.h \ + $(srcdir)/driver-nodedev.h \ + $(srcdir)/driver-nwfilter.h \ + $(srcdir)/driver-secret.h \ + $(srcdir)/driver-state.h \ + $(srcdir)/driver-storage.h \ + $(srcdir)/driver-stream.h \ + $(NULL) + check-drivername: - $(AM_V_GEN)$(PERL) $(srcdir)/check-drivername.pl \ - $(srcdir)/driver.h \ + $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(top_srcdir)/scripts/check-drivername.py= \ + $(DRIVERS) \ $(srcdir)/libvirt_public.syms \ $(srcdir)/libvirt_qemu.syms \ $(srcdir)/libvirt_lxc.syms =20 -EXTRA_DIST +=3D check-drivername.pl - check-driverimpls: $(AM_V_GEN)$(PERL) $(srcdir)/check-driverimpls.pl \ $(filter /%,$(DRIVER_SOURCE_FILES)) \ diff --git a/src/check-drivername.pl b/src/check-drivername.pl deleted file mode 100755 index 3a62193e33..0000000000 --- a/src/check-drivername.pl +++ /dev/null @@ -1,83 +0,0 @@ -#!/usr/bin/env perl -# -# Copyright (C) 2013 Red Hat, Inc. -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library. If not, see -# . -# - -use strict; -use warnings; - -my $drvfile =3D shift; -my @symfiles =3D @ARGV; - -my %symbols; - -foreach my $symfile (@symfiles) { - open SYMFILE, "<", $symfile - or die "cannot read $symfile: $!"; - while () { - if (/^\s*(vir\w+)\s*;\s*$/) { - $symbols{$1} =3D 1; - } - } - - close SYMFILE; -} - -open DRVFILE, "<", $drvfile - or die "cannot read $drvfile: $!"; - -my $status =3D 0; - -while () { - next if /virDrvConnectSupportsFeature/; - if (/\*(virDrv\w+)\s*\)/) { - - my $drv =3D $1; - - next if $drv =3D~ /virDrvState/; - next if $drv =3D~ /virDrvDomainMigrate(Prepare|Perform|Confirm|Beg= in|Finish)/; - - my $sym =3D $drv; - $sym =3D~ s/virDrv/vir/; - - unless (exists $symbols{$sym}) { - print "Driver method name $drv doesn't match public API name\n= "; - $status =3D 1; - } - } elsif (/^\*(vir\w+)\s*\)/) { - my $name =3D $1; - print "Bogus name $1\n"; - $status =3D 1; - } elsif (/^\s*(virDrv\w+)\s+(\w+);\s*/) { - my $drv =3D $1; - my $field =3D $2; - - my $tmp =3D $drv; - $tmp =3D~ s/virDrv//; - $tmp =3D~ s/^NWFilter/nwfilter/; - $tmp =3D~ s/^(\w)/lc $1/e; - - unless ($tmp eq $field) { - print "Driver struct field $field should be named $tmp\n"; - $status =3D 1; - } - } -} - -close DRVFILE; - -exit $status; --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sun May 5 18:51:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1573483524; cv=none; d=zoho.com; s=zohoarc; b=L5f1jdwC4MFzDNLE3BJLZtvJ8Q3zNkoaCCZRWpN+q2TRLrFpDh9B5Cq9M2X7wPbiSuRhMBxJY/EqQe6opAY+snyxGqoLSeE6pnIEqdcO1nWdk03ISmUpgMkqqmy8+B4FRbXkV6eBNOGeKlvMqtVhk9aVqR9VzTtu9LiLHXZnOXs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573483524; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ZKjYDIo+KGyEClqJI317b/hd7waZ6QhXFJJ/QIFmBq4=; b=jzU6AWR7O9ady7LIakxykNfDpI905n2Zlu+tO7Af2A3lPTDw2fSgIritZUDJnxNMIsYU73Dh0kTSrgmokKdynXKsVPeTgTIo5ZCqiT/2bGQOtzbEdqdbRm38Y+41aooP/WYKuDPuqAeRjB1WuKx0pPdjIj666T3gMaKS0u6C7Ks= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 1573483524330587.0352184692581; Mon, 11 Nov 2019 06:45:24 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-57-aGeTu-oVPEK8CRRLb0ijAQ-1; Mon, 11 Nov 2019 09:45:18 -0500 Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 651D28A243E; Mon, 11 Nov 2019 14:45:09 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 262131001B03; Mon, 11 Nov 2019 14:45:09 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id A3C5E4BB65; Mon, 11 Nov 2019 14:45:08 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xABEixvQ020435 for ; Mon, 11 Nov 2019 09:44:59 -0500 Received: by smtp.corp.redhat.com (Postfix) id 56CEE5DD73; Mon, 11 Nov 2019 14:44:59 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id BBB985D9C9; Mon, 11 Nov 2019 14:44:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573483523; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=ZKjYDIo+KGyEClqJI317b/hd7waZ6QhXFJJ/QIFmBq4=; b=Tzz6+QR0ym+KrbCcRRfFNUj7xx4bTlIkYqUBUDBNMx8BiLdpHS6r6W7vowMzwd1M0uEqxc od9ZQztR2tzKvDLcS4Rc79Su6KLdWHJ/MGGzYptXOhjdNoccLelfhB8x15P0NtnZ+tThjn jB1W9SdqtlMPTcHfWXTc7a7d48tDd/g= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 11 Nov 2019 14:38:14 +0000 Message-Id: <20191111143826.16050-12-berrange@redhat.com> In-Reply-To: <20191111143826.16050-1-berrange@redhat.com> References: <20191111143826.16050-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v5 11/23] src: rewrite driver impl checker in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-MC-Unique: aGeTu-oVPEK8CRRLb0ijAQ-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" As part of an goal to eliminate Perl from libvirt build tools, rewrite the check-driverimpls.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: J=C3=A1n Tomko Tested-by: Cole Robinson --- Makefile.am | 1 + scripts/check-driverimpls.py | 102 +++++++++++++++++++++++++++++++++++ src/Makefile.am | 4 +- src/check-driverimpls.pl | 80 --------------------------- 4 files changed, 105 insertions(+), 82 deletions(-) create mode 100755 scripts/check-driverimpls.py delete mode 100755 src/check-driverimpls.pl diff --git a/Makefile.am b/Makefile.am index 7155ab6ce9..407a664626 100644 --- a/Makefile.am +++ b/Makefile.am @@ -48,6 +48,7 @@ EXTRA_DIST =3D \ scripts/augeas-gentest.py \ scripts/check-aclperms.py \ scripts/check-drivername.py \ + scripts/check-driverimpls.py \ scripts/check-spacing.py \ scripts/check-symfile.py \ scripts/check-symsorting.py \ diff --git a/scripts/check-driverimpls.py b/scripts/check-driverimpls.py new file mode 100755 index 0000000000..78d53e75a4 --- /dev/null +++ b/scripts/check-driverimpls.py @@ -0,0 +1,102 @@ +#!/usr/bin/env python +# +# Copyright (C) 2013-2019 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . +# + +from __future__ import print_function + +import re +import sys + + +def checkdriverimpls(filename): + intable =3D False + mainprefix =3D None + + errs =3D False + with open(filename, "r") as fh: + lineno =3D 0 + for line in fh: + lineno =3D lineno + 1 + if intable: + if line.find("}") !=3D -1: + intable =3D False + mainprefix =3D None + continue + + m =3D re.search(r'''\.(\w+)\s*=3D\s*(\w+),?/''', line) + if m is not None: + api =3D m.group(1) + impl =3D m.group(2) + + if api in ["no", "name"]: + continue + if impl in ["NULL"]: + continue + + suffix =3D impl + prefix =3D re.sub(r'''^([a-z]+)(.*?)$''', r'''\1''', i= mpl) + + if mainprefix is not None: + if mainprefix !=3D prefix: + print("%s:%d Bad prefix '%s' for API '%s', " + + "expecting '%s'" % + (filename, lineno, prefix, api, mainpref= ix), + file=3Dsys.stderr) + errs =3D True + else: + mainprefix =3D prefix + + if not api.startswith(mainprefix): + suffix =3D re.sub(r'''^[a-z]+''', "", suffix) + suffix =3D re.sub(r'''^([A-Z]+)''', + lambda m: m.group(1).lower(), suff= ix) + + if api !=3D suffix: + want =3D api + if want.startswith("nwf"): + want =3D "NWF" + want[3:] + + if not api.startswith(mainprefix): + want =3D re.sub(r'''^([a-z])''', + lambda m: m.group(1).upper(), wa= nt) + want =3D mainprefix + want + + print("%s:%d Bad impl name '%s' for API " + + "'%s', expecting '%s'" % + (filename, lineno, impl, api, want), + file=3Dsys.stderr) + errs =3D True + else: + m =3D re.search(r'''^(?:static\s+)?(vir(?:\w+)?Driver)''' + + r'''\s+(?!.*;)''', line) + if m is not None: + drv =3D m.group(1) + if drv in ["virNWFilterCallbackDriver", + "virNWFilterTechDriver", + "virConnectDriver"]: + continue + intable =3D True + + return errs + + +status =3D 0 +for filename in sys.argv[1:]: + if checkdriverimpls(filename): + status =3D 1 +sys.exit(status) diff --git a/src/Makefile.am b/src/Makefile.am index 55ad51abf1..62f1d55402 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -350,7 +350,7 @@ check-drivername: $(srcdir)/libvirt_lxc.syms =20 check-driverimpls: - $(AM_V_GEN)$(PERL) $(srcdir)/check-driverimpls.pl \ + $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(top_srcdir)/scripts/check-driverimpls.p= y \ $(filter /%,$(DRIVER_SOURCE_FILES)) \ $(filter $(srcdir)/%,$(DRIVER_SOURCE_FILES)) \ $(addprefix $(srcdir)/,$(filter-out $(srcdir)/%, \ @@ -366,7 +366,7 @@ check-aclperms: $(srcdir)/access/viraccessperm.h \ $(srcdir)/access/viraccessperm.c =20 -EXTRA_DIST +=3D check-driverimpls.pl check-aclrules.pl +EXTRA_DIST +=3D check-aclrules.pl =20 check-local: check-protocol check-symfile check-symsorting \ check-drivername check-driverimpls check-aclrules \ diff --git a/src/check-driverimpls.pl b/src/check-driverimpls.pl deleted file mode 100755 index 3c0d54724c..0000000000 --- a/src/check-driverimpls.pl +++ /dev/null @@ -1,80 +0,0 @@ -#!/usr/bin/env perl -# -# Copyright (C) 2013 Red Hat, Inc. -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library. If not, see -# . -# - -use strict; -use warnings; - -my $intable =3D 0; -my $table; -my $mainprefix; - -my $status =3D 0; -while (<>) { - if ($intable) { - if (/}/) { - $intable =3D 0; - $table =3D undef; - $mainprefix =3D undef; - } elsif (/\.(\w+)\s*=3D\s*(\w+),?/) { - my $api =3D $1; - my $impl =3D $2; - - next if $api eq "no"; - next if $api eq "name"; - next if $impl eq "NULL"; - - my $suffix =3D $impl; - my $prefix =3D $impl; - $prefix =3D~ s/^([a-z]+)(.*?)$/$1/; - - if (defined $mainprefix) { - if ($mainprefix ne $prefix) { - print "$ARGV:$. Bad prefix '$prefix' for API '$api', e= xpecting '$mainprefix'\n"; - $status =3D 1; - } - } else { - $mainprefix =3D $prefix; - } - - if ($api !~ /^$mainprefix/) { - $suffix =3D~ s/^[a-z]+//; - $suffix =3D~ s/^([A-Z]+)/lc $1/e; - } - - if ($api ne $suffix) { - my $want =3D $api; - $want =3D~ s/^nwf/NWF/; - if ($api !~ /^$mainprefix/) { - $want =3D~ s/^([a-z])/uc $1/e; - $want =3D $mainprefix . $want; - } - print "$ARGV:$. Bad impl name '$impl' for API '$api', expe= cting '$want'\n"; - $status =3D 1; - } - } - } elsif (/^(?:static\s+)?(vir(?:\w+)?Driver)\s+(?!.*;)/) { - next if $1 eq "virNWFilterCallbackDriver" || - $1 eq "virNWFilterTechDriver" || - $1 eq "virConnectDriver"; - $intable =3D 1; - $table =3D $1; - } -} - -exit $status; --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sun May 5 18:51:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 207.211.31.120 as permitted sender) client-ip=207.211.31.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1573483569; cv=none; d=zoho.com; s=zohoarc; b=dZJam1Hs995CWlJCguEH3UKFhNMB3dVEYMuawFRLsAShrOqFw3IFAVa1s3ZEpNvwfK6Y8k2OWotNs69dlscOyZquTlfQl3sw6mEUv6Cqjxpn+2FxV52v0FjU4YECVnIeA4VIv3Df9SwkEWwj+0eo9GMBY3f9hqzSlBtt/13GBkU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573483569; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=PYdiNluJiVDzbNjkZFZ8O9kBexoHJd+4N7yesZbqSos=; b=IykwQKGUDJBG948/aM58Ye9Rdh7zVBVW5J+/jOgqq5hbuikkDQrk4djTPjw0eT60krVOE4pSlw/E5aYlxszsSgiV9MY5PxjxlDniMF58g5Wm5nd8/Cm9Iip2DhsMSh8rSTD9MivZxnpGvL9rvUlCnZ5K/I0+gyH5bYf94gscEw8= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by mx.zohomail.com with SMTPS id 1573483569209168.7416225310327; Mon, 11 Nov 2019 06:46:09 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-148-e3UlWQHVOLuVD6FsJCArzA-1; Mon, 11 Nov 2019 09:45:21 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 289AD85EE8B; Mon, 11 Nov 2019 14:45:15 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id EE75C4B4; Mon, 11 Nov 2019 14:45:14 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 912E24E57C; Mon, 11 Nov 2019 14:45:14 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xABEj0DK020443 for ; Mon, 11 Nov 2019 09:45:00 -0500 Received: by smtp.corp.redhat.com (Postfix) id 3BF3C5DD73; Mon, 11 Nov 2019 14:45:00 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9C4595D9C9; Mon, 11 Nov 2019 14:44:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573483566; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=PYdiNluJiVDzbNjkZFZ8O9kBexoHJd+4N7yesZbqSos=; b=HYYn4Qr+iOtYrjMgykrnVm/fi9Zts3vsQ8an/yNqXwRebyuVcY54IjgkitcHYR86wl+T1m h1uJuq7nAX28iit0zXGjxVLIy/vsHnkSiRnhl6KGsDxuWPeMWAWPwS9pJ4fFBfayaHTHbh 5YnsTx5TYWOtqFcNMgFcrHe3m5RJpc0= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 11 Nov 2019 14:38:15 +0000 Message-Id: <20191111143826.16050-13-berrange@redhat.com> In-Reply-To: <20191111143826.16050-1-berrange@redhat.com> References: <20191111143826.16050-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v5 12/23] src: rewrite ACL rule checker in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-MC-Unique: e3UlWQHVOLuVD6FsJCArzA-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" As part of an goal to eliminate Perl from libvirt build tools, rewrite the check-aclrules.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: J=C3=A1n Tomko Tested-by: Cole Robinson --- Makefile.am | 1 + scripts/check-aclrules.py | 263 ++++++++++++++++++++++++++++++++++++++ src/Makefile.am | 4 +- src/check-aclrules.pl | 252 ------------------------------------ 4 files changed, 265 insertions(+), 255 deletions(-) create mode 100755 scripts/check-aclrules.py delete mode 100755 src/check-aclrules.pl diff --git a/Makefile.am b/Makefile.am index 407a664626..f28b07d814 100644 --- a/Makefile.am +++ b/Makefile.am @@ -47,6 +47,7 @@ EXTRA_DIST =3D \ AUTHORS.in \ scripts/augeas-gentest.py \ scripts/check-aclperms.py \ + scripts/check-aclrules.py \ scripts/check-drivername.py \ scripts/check-driverimpls.py \ scripts/check-spacing.py \ diff --git a/scripts/check-aclrules.py b/scripts/check-aclrules.py new file mode 100755 index 0000000000..3fab126a4a --- /dev/null +++ b/scripts/check-aclrules.py @@ -0,0 +1,263 @@ +#!/usr/bin/env python +# +# Copyright (C) 2013-2019 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . +# +# This script validates that the driver implementation of any +# public APIs contain ACL checks. +# +# As the script reads each source file, it attempts to identify +# top level function names. +# +# When reading the body of the functions, it looks for anything +# that looks like an API called named XXXEnsureACL. It will +# validate that the XXX prefix matches the name of the function +# it occurs in. +# +# When it later finds the virDriverPtr table, for each entry +# point listed, it will validate if there was a previously +# detected EnsureACL call recorded. +# + +from __future__ import print_function + +import re +import sys + +whitelist =3D { + "connectClose": True, + "connectIsEncrypted": True, + "connectIsSecure": True, + "connectIsAlive": True, + "networkOpen": True, + "networkClose": True, + "nwfilterOpen": True, + "nwfilterClose": True, + "secretOpen": True, + "secretClose": True, + "storageOpen": True, + "storageClose": True, + "interfaceOpen": True, + "interfaceClose": True, + "connectURIProbe": True, + "localOnly": True, + "domainQemuAttach": True, +} + +# XXX this vzDomainMigrateConfirm3Params looks +# bogus - determine why it doesn't have a valid +# ACL check. +implwhitelist =3D { + "vzDomainMigrateConfirm3Params": True, +} + +lastfile =3D None + + +def fixup_name(name): + name.replace("Nwfilter", "NWFilter") + name.replace("Pm", "PM") + name.replace("Scsi", "SCSI") + if name.endswith("Xml"): + name =3D name[:-3] + "XML" + elif name.endswith("Uri"): + name =3D name[:-3] + "URI" + elif name.endswith("Uuid"): + name =3D name[:-4] + "UUID" + elif name.endswith("Id"): + name =3D name[:-2] + "ID" + elif name.endswith("Mac"): + name =3D name[:-3] + "MAC" + elif name.endswith("Cpu"): + name =3D name[:-3] + "MAC" + elif name.endswith("Os"): + name =3D name[:-2] + "OS" + elif name.endswith("Nmi"): + name =3D name[:-3] + "NMI" + elif name.endswith("Fstrim"): + name =3D name[:-6] + "FSTrim" + elif name.endswith("Wwn"): + name =3D name[:-3] + "WWN" + + return name + + +def name_to_ProcName(name): + elems =3D [] + if name.find("_") !=3D -1 or name.lower() in ["open", "close"]: + elems =3D [n.lower().capitalize() for n in name.split("_")] + else: + elems =3D [name] + + elems =3D [fixup_name(n) for n in elems] + procname =3D "".join(elems) + + return procname[0:1].lower() + procname[1:] + + +proto =3D sys.argv[1] + +filteredmap =3D {} +with open(proto, "r") as fh: + incomment =3D False + filtered =3D False + + for line in fh: + if line.find("/**") !=3D -1: + incomment =3D True + filtered =3D False + elif incomment: + if line.find("* @aclfilter") !=3D -1: + filtered =3D True + elif filtered: + m =3D re.search(r'''REMOTE_PROC_(.*)\s+=3D\s*\d+''', line) + if m is not None: + api =3D name_to_ProcName(m.group(1)) + # Event filtering is handled in daemon/remote.c + # instead of drivers + if line.find("_EVENT_REGISTER") =3D=3D -1: + filteredmap[api] =3D True + incomment =3D False + + +def process_file(filename): + brace =3D 0 + maybefunc =3D None + intable =3D False + table =3D None + + acls =3D {} + aclfilters =3D {} + errs =3D False + with open(filename, "r") as fh: + lineno =3D 0 + for line in fh: + lineno =3D lineno + 1 + if brace =3D=3D 0: + # Looks for anything which appears to be a function + # body name. Doesn't matter if we pick up bogus stuff + # here, as long as we don't miss valid stuff + m =3D re.search(r'''\b(\w+)\(''', line) + if m is not None: + maybefunc =3D m.group(1) + elif brace > 0: + ensureacl =3D re.search(r'''(\w+)EnsureACL''', line) + checkacl =3D re.search(r'''(\w+)CheckACL''', line) + stub =3D re.search(r'''\b(\w+)\(''', line) + if ensureacl is not None: + # Record the fact that maybefunc contains an + # ACL call, and make sure it is the right call! + func =3D ensureacl.group(1) + if func.startswith("vir"): + func =3D func[3:] + + if maybefunc is None: + print("%s:%d Unexpected check '%s' outside functio= n" % + (filename, lineno, func), file=3Dsys.stderr) + errs =3D True + else: + if not maybefunc.lower().endswith(func.lower()): + print("%s:%d Mismatch check 'vir%sEnsureACL'" + + "for function '%s'" % + (filename, lineno, func, maybefunc), + file=3Dsys.stderr) + errs =3D True + acls[maybefunc] =3D True + elif checkacl: + # Record the fact that maybefunc contains an + # ACL filter call, and make sure it is the right call! + func =3D checkacl.group(1) + if func.startswith("vir"): + func =3D func[3:] + + if maybefunc is None: + print("%s:%d Unexpected check '%s' outside functio= n" % + (filename, lineno, func), file=3Dsys.stderr) + errs =3D True + else: + if not maybefunc.lower().endswith(func.lower()): + print("%s:%d Mismatch check 'vir%sEnsureACL' "= + + "for function '%s'" % + (filename, lineno, func, maybefunc), + file=3Dsys.stderr) + errs =3D True + aclfilters[maybefunc] =3D True + elif stub: + # Handles case where we replaced an API with a new + # one which adds new parameters, and we're left with + # a simple stub calling the new API. + callfunc =3D stub.group(1) + if callfunc in acls: + acls[maybefunc] =3D True + + if callfunc in aclfilters: + aclfilters[maybefunc] =3D True + + # Pass the vir*DriverPtr tables and make sure that + # every func listed there, has an impl which calls + # an ACL function + if intable: + assign =3D re.search(r'''\.(\w+)\s*=3D\s*(\w+),?''', line) + if line.find("}") !=3D -1: + intable =3D False + table =3D None + elif assign is not None: + api =3D assign.group(1) + impl =3D assign.group(2) + + if (impl !=3D "NULL" and + api not in ["no", "name"] and + table !=3D "virStateDriver"): + if (impl not in acls and + api not in whitelist and + impl not in implwhitelist): + print("%s:%d Missing ACL check in " + + "function '%s' for '%s'" % + (filename, lineno, impl, api), + file=3Dsys.stderr) + errs =3D True + + if api in filteredmap and impl not in aclfilters: + print("%s:%d Missing ACL filter in " + + "function '%s' for '%s'" % + (filename, lineno, impl, api), + file=3Dsys.stderr) + errs =3D True + else: + m =3D re.search(r'''^(?:static\s+)?(vir(?:\w+)?Driver)\s+'= '', + line) + if m is not None: + name =3D m.group(1) + if name not in ["virNWFilterCallbackDriver", + "virNWFilterTechDriver", + "virDomainConfNWFilterDriver"]: + intable =3D True + table =3D name + + if line.find("{") !=3D -1: + brace =3D brace + 1 + if line.find("}") !=3D -1: + brace =3D brace - 1 + + return errs + + +status =3D 0 +for filename in sys.argv[2:]: + if process_file(filename): + status =3D 1 + +sys.exit(status) diff --git a/src/Makefile.am b/src/Makefile.am index 62f1d55402..bb63e2486c 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -357,7 +357,7 @@ check-driverimpls: $(filter-out /%,$(DRIVER_SOURCE_FILES)))) =20 check-aclrules: - $(AM_V_GEN)$(PERL) $(srcdir)/check-aclrules.pl \ + $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(top_srcdir)/scripts/check-aclrules.py \ $(REMOTE_PROTOCOL) \ $(addprefix $(srcdir)/,$(filter-out /%,$(STATEFUL_DRIVER_SOURCE_FILES))) =20 @@ -366,8 +366,6 @@ check-aclperms: $(srcdir)/access/viraccessperm.h \ $(srcdir)/access/viraccessperm.c =20 -EXTRA_DIST +=3D check-aclrules.pl - check-local: check-protocol check-symfile check-symsorting \ check-drivername check-driverimpls check-aclrules \ check-aclperms check-admin diff --git a/src/check-aclrules.pl b/src/check-aclrules.pl deleted file mode 100755 index 0d4cac17ca..0000000000 --- a/src/check-aclrules.pl +++ /dev/null @@ -1,252 +0,0 @@ -#!/usr/bin/env perl -# -# Copyright (C) 2013-2014 Red Hat, Inc. -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library. If not, see -# . -# -# This script validates that the driver implementation of any -# public APIs contain ACL checks. -# -# As the script reads each source file, it attempts to identify -# top level function names. -# -# When reading the body of the functions, it looks for anything -# that looks like an API called named XXXEnsureACL. It will -# validate that the XXX prefix matches the name of the function -# it occurs in. -# -# When it later finds the virDriverPtr table, for each entry -# point listed, it will validate if there was a previously -# detected EnsureACL call recorded. -# -use strict; -use warnings; - -my $status =3D 0; - -my $brace =3D 0; -my $maybefunc; -my $intable =3D 0; -my $table; - -my %acls; -my %aclfilters; - -my %whitelist =3D ( - "connectClose" =3D> 1, - "connectIsEncrypted" =3D> 1, - "connectIsSecure" =3D> 1, - "connectIsAlive" =3D> 1, - "networkOpen" =3D> 1, - "networkClose" =3D> 1, - "nwfilterOpen" =3D> 1, - "nwfilterClose" =3D> 1, - "secretOpen" =3D> 1, - "secretClose" =3D> 1, - "storageOpen" =3D> 1, - "storageClose" =3D> 1, - "interfaceOpen" =3D> 1, - "interfaceClose" =3D> 1, - "connectURIProbe" =3D> 1, - "localOnly" =3D> 1, - "domainQemuAttach" =3D> 1, - ); - -# XXX this vzDomainMigrateConfirm3Params looks -# bogus - determine why it doesn't have a valid -# ACL check. -my %implwhitelist =3D ( - "vzDomainMigrateConfirm3Params" =3D> 1, - ); - -my $lastfile; - -sub fixup_name { - my $name =3D shift; - - $name =3D~ s/Nwfilter/NWFilter/; - $name =3D~ s/Xml$/XML/; - $name =3D~ s/Uri$/URI/; - $name =3D~ s/Uuid$/UUID/; - $name =3D~ s/Id$/ID/; - $name =3D~ s/Mac$/MAC/; - $name =3D~ s/Cpu$/CPU/; - $name =3D~ s/Os$/OS/; - $name =3D~ s/Nmi$/NMI/; - $name =3D~ s/Pm/PM/; - $name =3D~ s/Fstrim$/FSTrim/; - $name =3D~ s/Scsi/SCSI/; - $name =3D~ s/Wwn$/WWN/; - - return $name; -} - -sub name_to_ProcName { - my $name =3D shift; - - my @elems; - if ($name =3D~ /_/ || (lc $name) eq "open" || (lc $name) eq "close") { - @elems =3D split /_/, $name; - @elems =3D map lc, @elems; - @elems =3D map ucfirst, @elems; - } else { - @elems =3D $name; - } - @elems =3D map { fixup_name($_) } @elems; - my $procname =3D join "", @elems; - - $procname =3D~ s/^([A-Z])/lc $1/e; - - return $procname; -} - - -my $proto =3D shift @ARGV; - -open PROTO, "<$proto" or die "cannot read $proto"; - -my %filtered; -my $incomment =3D 0; -my $filtered =3D 0; -while () { - if (m,/\*\*,) { - $incomment =3D 1; - $filtered =3D 0; - } elsif ($incomment) { - if (m,\*\s\@aclfilter,) { - $filtered =3D 1; - } elsif ($filtered && - m,REMOTE_PROC_(.*)\s+=3D\s*\d+,) { - my $api =3D name_to_ProcName($1); - # Event filtering is handled in daemon/remote.c instead of dri= vers - if (! m,_EVENT_REGISTER,) { - $filtered{$api} =3D 1; - } - $incomment =3D 0; - } - } -} - -close PROTO; - -while (<>) { - if (!defined $lastfile || - $lastfile ne $ARGV) { - %acls =3D (); - $brace =3D 0; - $maybefunc =3D undef; - $lastfile =3D $ARGV; - } - if ($brace =3D=3D 0) { - # Looks for anything which appears to be a function - # body name. Doesn't matter if we pick up bogus stuff - # here, as long as we don't miss valid stuff - if (m,\b(\w+)\(,) { - $maybefunc =3D $1; - } - } elsif ($brace > 0) { - if (m,(\w+)EnsureACL,) { - # Record the fact that maybefunc contains an - # ACL call, and make sure it is the right call! - my $func =3D $1; - $func =3D~ s/^vir//; - if (!defined $maybefunc) { - print "$ARGV:$. Unexpected check '$func' outside function\= n"; - $status =3D 1; - } else { - unless ($maybefunc =3D~ /$func$/i) { - print "$ARGV:$. Mismatch check 'vir${func}EnsureACL' f= or function '$maybefunc'\n"; - $status =3D 1; - } - } - $acls{$maybefunc} =3D 1; - } elsif (m,(\w+)CheckACL,) { - # Record the fact that maybefunc contains an - # ACL filter call, and make sure it is the right call! - my $func =3D $1; - $func =3D~ s/^vir//; - if (!defined $maybefunc) { - print "$ARGV:$. Unexpected check '$func' outside function\= n"; - $status =3D 1; - } else { - unless ($maybefunc =3D~ /$func$/i) { - print "$ARGV:$. Mismatch check 'vir${func}CheckACL' fo= r function '$maybefunc'\n"; - $status =3D 1; - } - } - $aclfilters{$maybefunc} =3D 1; - } elsif (m,\b(\w+)\(,) { - # Handles case where we replaced an API with a new - # one which adds new parameters, and we're left with - # a simple stub calling the new API. - my $callfunc =3D $1; - if (exists $acls{$callfunc}) { - $acls{$maybefunc} =3D 1; - } - if (exists $aclfilters{$callfunc}) { - $aclfilters{$maybefunc} =3D 1; - } - } - } - - # Pass the vir*DriverPtr tables and make sure that - # every func listed there, has an impl which calls - # an ACL function - if ($intable) { - if (/\}/) { - $intable =3D 0; - $table =3D undef; - } elsif (/\.(\w+)\s*=3D\s*(\w+),?/) { - my $api =3D $1; - my $impl =3D $2; - - next if $impl eq "NULL"; - - if ($api ne "no" && - $api ne "name" && - $table ne "virStateDriver" && - !exists $acls{$impl} && - !exists $whitelist{$api} && - !exists $implwhitelist{$impl}) { - print "$ARGV:$. Missing ACL check in function '$impl' for = '$api'\n"; - $status =3D 1; - } - - if (exists $filtered{$api} && - !exists $aclfilters{$impl}) { - print "$ARGV:$. Missing ACL filter in function '$impl' for= '$api'\n"; - $status =3D 1; - } - } - } elsif (/^(?:static\s+)?(vir(?:\w+)?Driver)\s+/) { - if ($1 ne "virNWFilterCallbackDriver" && - $1 ne "virNWFilterTechDriver" && - $1 ne "virDomainConfNWFilterDriver") { - $intable =3D 1; - $table =3D $1; - } - } - - - my $count; - $count =3D s/{//g; - $brace +=3D $count; - $count =3D s/}//g; - $brace -=3D $count; -} continue { - close ARGV if eof; -} - -exit $status; --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sun May 5 18:51:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 205.139.110.120 as permitted sender) client-ip=205.139.110.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1573483569; cv=none; d=zoho.com; s=zohoarc; b=HtTy5QhkzQjgU3GUg/aC6btIEqAKtgQU4cRVHXDlwmL00v6Hp0bz54RFMmpAdnk7wjlYD9M3/+kuArkUzpqX08zm7DgB/royVXAytC3czIqyffgKA5+8WaNSX/BJnF1W/zUsw8sUIhFOdIJFVCu7+8y4eSQQmCqmve4OwtpzawU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573483569; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=iA8jVdombxEzaExUV2jFuWOjITfbOb9SGBeboD5zGHQ=; b=hvUQ9OTOdWQH5qJry7n9gaNLIEchDBqGLMRuU3IU47ob+YxT71+Z15lzzaqJambOIPrdWD0TXYg3bCAIwnwz07K+n1Yo2m2SIdWt8IRes1HWwo8x0apSVCbR9SYd3ju1etsFh/dgYTqCCYlhnPULdLoA3DqbxOT/s3bf0kQyeRE= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by mx.zohomail.com with SMTPS id 1573483569035928.7182914023036; Mon, 11 Nov 2019 06:46:09 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-429-gGLMVjwvNmq2SP74U5liJw-1; Mon, 11 Nov 2019 09:45:25 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id CD1A985EE72; Mon, 11 Nov 2019 14:45:17 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 9D9155DA7D; Mon, 11 Nov 2019 14:45:17 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 452AC18033C0; Mon, 11 Nov 2019 14:45:17 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xABEj1XL020458 for ; Mon, 11 Nov 2019 09:45:01 -0500 Received: by smtp.corp.redhat.com (Postfix) id 1B7C35DA7D; Mon, 11 Nov 2019 14:45:01 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7F2C25D9C9; Mon, 11 Nov 2019 14:45:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573483567; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=iA8jVdombxEzaExUV2jFuWOjITfbOb9SGBeboD5zGHQ=; b=NO0s/au/kk5tKaaSiTJjHq9LzU8ORKKtJ4wa5xLlKfutlduKl+oBfRdLr24cai+u8DVtLO 3D7J+kCYiXAAc/YHs/9nKrrWQgasjWoirjY1Vaka5s6E195iYdeG3BAl/+0sW4NFNkEytj LY5KyN6lQZUBqk5UwGoEhsfViCKqlN4= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 11 Nov 2019 14:38:16 +0000 Message-Id: <20191111143826.16050-14-berrange@redhat.com> In-Reply-To: <20191111143826.16050-1-berrange@redhat.com> References: <20191111143826.16050-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v5 13/23] src: rewrite polkit ACL generator in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-MC-Unique: gGLMVjwvNmq2SP74U5liJw-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" As part of an goal to eliminate Perl from libvirt build tools, rewrite the genpolkit.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: J=C3=A1n Tomko Tested-by: Cole Robinson --- Makefile.am | 1 + scripts/genpolkit.py | 122 +++++++++++++++++++++++++++++++++++++ src/access/Makefile.inc.am | 6 +- src/access/genpolkit.pl | 119 ------------------------------------ 4 files changed, 126 insertions(+), 122 deletions(-) create mode 100755 scripts/genpolkit.py delete mode 100755 src/access/genpolkit.pl diff --git a/Makefile.am b/Makefile.am index f28b07d814..e7ebe7281a 100644 --- a/Makefile.am +++ b/Makefile.am @@ -54,6 +54,7 @@ EXTRA_DIST =3D \ scripts/check-symfile.py \ scripts/check-symsorting.py \ scripts/dtrace2systemtap.py \ + scripts/genpolkit.py \ scripts/gensystemtap.py \ scripts/header-ifdef.py \ scripts/minimize-po.py \ diff --git a/scripts/genpolkit.py b/scripts/genpolkit.py new file mode 100755 index 0000000000..0cdba2bd3c --- /dev/null +++ b/scripts/genpolkit.py @@ -0,0 +1,122 @@ +#!/usr/bin/env python +# +# Copyright (C) 2012-2019 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . +# + +from __future__ import print_function + +import re +import sys + +objects =3D [ + "CONNECT", "DOMAIN", "INTERFACE", "NETWORK_PORT", + "NETWORK", "NODE_DEVICE", "NWFILTER_BINDING", + "NWFILTER", "SECRET", "STORAGE_POOL", "STORAGE_VOL", +] + +objectstr =3D "|".join(objects) + +# Data we're going to be generating looks like this +# +# +# +# Monitor local virtualized systems +# System policy prevents monitoring of +# local virtualized systems +# +# yes +# yes +# yes +# +# +# ...more rules... +# + +opts =3D {} +in_opts =3D False + +perms =3D {} + +aclfile =3D sys.argv[1] +with open(aclfile, "r") as fh: + for line in fh: + if in_opts: + if line.find("*/") !=3D -1: + in_opts =3D False + else: + m =3D re.search(r'''\*\s*\@(\w+):\s*(.*?)\s*$''', line) + if m is not None: + opts[m.group(1)] =3D m.group(2) + elif line.find("**") !=3D -1: + in_opts =3D True + else: + m =3D re.search(r'''VIR_ACCESS_PERM_(%s)_((?:\w|_)+),''' % + objectstr, line) + if m is not None: + obj =3D m.group(1).lower() + perm =3D m.group(2).lower() + if perm =3D=3D "last": + continue + + obj =3D obj.replace("_", "-") + perm =3D perm.replace("_", "-") + + if obj not in perms: + perms[obj] =3D {} + perms[obj][perm] =3D { + "desc": opts.get("desc", None), + "message": opts.get("message", None), + "anonymous": opts.get("anonymous", None), + } + opts =3D {} + +print('') +print('') +print('') +print(' Libvirt Project') +print(' https://libvirt.org') + +for obj in sorted(perms.keys()): + for perm in sorted(perms[obj].keys()): + description =3D perms[obj][perm]["desc"] + message =3D perms[obj][perm]["message"] + anonymous =3D perms[obj][perm]["anonymous"] + + if description is None: + raise Exception("missing description for %s.%s" % (obj, perm)) + if message is None: + raise Exception("missing message for %s.%s" % (obj, perm)) + + allow_any =3D "no" + if anonymous: + allow_any =3D "yes" + allow_inactive =3D allow_any + allow_active =3D allow_any + + print(' ' % (obj, perm)) + print(' %s' % description) + print(' %s' % message) + print(' ') + print(' %s' % allow_any) + print(' %s' % allow_inactive) + print(' %s' % allow_active) + print(' ') + print(' ') + +print('') diff --git a/src/access/Makefile.inc.am b/src/access/Makefile.inc.am index fd0a5d8098..11f87c6aa7 100644 --- a/src/access/Makefile.inc.am +++ b/src/access/Makefile.inc.am @@ -43,7 +43,6 @@ ACCESS_DRIVER_POLKIT_POLICY =3D access/org.libvirt.api.po= licy GENERATED_SYM_FILES +=3D $(ACCESS_DRIVER_SYM_FILES) =20 EXTRA_DIST +=3D \ - access/genpolkit.pl \ $(NULL) =20 =20 @@ -66,8 +65,9 @@ libvirt_driver_access_la_LIBADD =3D \ =20 =20 $(ACCESS_DRIVER_POLKIT_POLICY): $(srcdir)/access/viraccessperm.h \ - $(srcdir)/access/genpolkit.pl Makefile.am - $(AM_V_GEN)$(PERL) $(srcdir)/access/genpolkit.pl < $< > $@ || rm -f $@ + $(top_srcdir)/scripts/genpolkit.py Makefile.am + $(AM_V_GEN)$(RUNUTF8) $(PYTHON) \ + $(top_srcdir)/scripts/genpolkit.py $< > $@ || rm -f $@ =20 if WITH_POLKIT libvirt_driver_access_la_SOURCES +=3D $(ACCESS_DRIVER_POLKIT_SOURCES) diff --git a/src/access/genpolkit.pl b/src/access/genpolkit.pl deleted file mode 100755 index f8f20caf65..0000000000 --- a/src/access/genpolkit.pl +++ /dev/null @@ -1,119 +0,0 @@ -#!/usr/bin/env perl -# -# Copyright (C) 2012-2013 Red Hat, Inc. -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library. If not, see -# . -# - -use strict; -use warnings; - -my @objects =3D ( - "CONNECT", "DOMAIN", "INTERFACE", "NETWORK_PORT", - "NETWORK","NODE_DEVICE", "NWFILTER_BINDING", "NWFILTER", - "SECRET", "STORAGE_POOL", "STORAGE_VOL", - ); - -my $objects =3D join ("|", @objects); - -# Data we're going to be generating looks like this -# -# -# -# Monitor local virtualized systems -# System policy prevents monitoring of local virtualized syst= ems -# -# yes -# yes -# yes -# -# -# ...more rules... -# - -my %opts; -my $in_opts =3D 0; - -my %perms; - -while (<>) { - if ($in_opts) { - if (m,\*/,) { - $in_opts =3D 0; - } elsif (/\*\s*\@(\w+):\s*(.*?)\s*$/) { - $opts{$1} =3D $2; - } - } elsif (m,/\*\*,) { - $in_opts =3D 1; - } elsif (/VIR_ACCESS_PERM_($objects)_((?:\w|_)+),/) { - my $object =3D lc $1; - my $perm =3D lc $2; - next if $perm eq "last"; - - $object =3D~ s/_/-/g; - $perm =3D~ s/_/-/g; - - $perms{$object} =3D {} unless exists $perms{$object}; - $perms{$object}->{$perm} =3D { - desc =3D> $opts{desc}, - message =3D> $opts{message}, - anonymous =3D> $opts{anonymous} - }; - %opts =3D (); - } -} - -print < - - - Libvirt Project - https://libvirt.org -EOF - -foreach my $object (sort { $a cmp $b } keys %perms) { - foreach my $perm (sort { $a cmp $b } keys %{$perms{$object}}) { - my $description =3D $perms{$object}->{$perm}->{desc}; - my $message =3D $perms{$object}->{$perm}->{message}; - my $anonymous =3D $perms{$object}->{$perm}->{anonymous}; - - die "missing description for $object.$perm" unless - defined $description; - die "missing message for $object.$perm" unless - defined $message; - - my $allow_any =3D $anonymous ? "yes" : "no"; - my $allow_inactive =3D $allow_any; - my $allow_active =3D $allow_any; - - print < - $description - $message - - $allow_any - $allow_inactive - $allow_active - - -EOF - - } -} - -print < -EOF --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sun May 5 18:51:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1573483566; cv=none; d=zoho.com; s=zohoarc; b=J4f7jtAST0LmAxwX+U6gdMU00hNycVsDp7HJdEefluMmshG6yr+wq6cRqmsPeogXVmtvJOg/PZi5fFPMRniEd1zRiF/SIMhC7Fw4Rq6MLBqUJlM/2JtS95msoBwVBC6BaxyO5MLI9LaTxGMVN1+xPolspM6L2HeBxDnSit76NYc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573483566; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=8n2cF/Ge9owStxSXUtmBcHnGEcjmQcXvOZNW2oSXvek=; b=SCvjxw+lKd0CZYDmrmBHIwhgJ1GsnuCJV2dKAkmMN5N4JhSIDgvX2zw5Lp2+OLCUgEJX5hyxi7L1HTq92F0SlOBdbqK8AJa+bSAiVNLqnGo/o+K+UNIU0p6h+tTU3lhaVcGVnLL/8Z0vDRzzd+h4Z0myRkOZ6JGbaQLBz6nsHGI= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 1573483566905115.03546009661636; Mon, 11 Nov 2019 06:46:06 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-65-myI5Gw5mMq2R3inB-nmHqg-1; Mon, 11 Nov 2019 09:45:24 -0500 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B0CBD1852E21; Mon, 11 Nov 2019 14:45:18 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 7FC6084F7; Mon, 11 Nov 2019 14:45:18 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 30F904E57D; Mon, 11 Nov 2019 14:45:18 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xABEj155020486 for ; Mon, 11 Nov 2019 09:45:02 -0500 Received: by smtp.corp.redhat.com (Postfix) id F14695DD73; Mon, 11 Nov 2019 14:45:01 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 60EC55D9C9; Mon, 11 Nov 2019 14:45:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573483565; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=8n2cF/Ge9owStxSXUtmBcHnGEcjmQcXvOZNW2oSXvek=; b=fIFLcCXllAqcOaWnbtW584TCAbXBFoNfkd7tNwMVAt/6zcTS/Pjs5Cyg1GFpArUPfbSsRY IKbe1DVrT+DRf6oJ4XESeODjcBKCPMjwdVhjRPm3lI9gY6pMFtCNRdvsgtP2wHAwcnSYa/ HMRmPH0LJPrXYYhGm0du64X0EWTdLh8= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 11 Nov 2019 14:38:17 +0000 Message-Id: <20191111143826.16050-15-berrange@redhat.com> In-Reply-To: <20191111143826.16050-1-berrange@redhat.com> References: <20191111143826.16050-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v5 14/23] src: rewrite remote protocol checker in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-MC-Unique: myI5Gw5mMq2R3inB-nmHqg-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" As part of an goal to eliminate Perl from libvirt build tools, rewrite the pdwtags processing script in Python. The original inline shell and perl code was completely unintelligible. The new python code is a manual conversion that attempts todo basically the same thing. Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: J=C3=A1n Tomko Tested-by: Cole Robinson --- Makefile.am | 1 + build-aux/syntax-check.mk | 3 +- scripts/check-remote-protocol.py | 136 +++++++++++++++++++++++++++++++ src/Makefile.am | 98 ++++------------------ 4 files changed, 155 insertions(+), 83 deletions(-) create mode 100644 scripts/check-remote-protocol.py diff --git a/Makefile.am b/Makefile.am index e7ebe7281a..8c9c73c715 100644 --- a/Makefile.am +++ b/Makefile.am @@ -50,6 +50,7 @@ EXTRA_DIST =3D \ scripts/check-aclrules.py \ scripts/check-drivername.py \ scripts/check-driverimpls.py \ + scripts/check-remote-protocol.py \ scripts/check-spacing.py \ scripts/check-symfile.py \ scripts/check-symsorting.py \ diff --git a/build-aux/syntax-check.mk b/build-aux/syntax-check.mk index 26eb5b94d9..a61818855c 100644 --- a/build-aux/syntax-check.mk +++ b/build-aux/syntax-check.mk @@ -412,6 +412,7 @@ sc_prohibit_mkstemp: # access with F_OK or R_OK is okay, though. sc_prohibit_access_xok: @prohibit=3D'access(at)? *\(.*X_OK' \ + in_vc_files=3D'\.[ch]$$' \ halt=3D'use virFileIsExecutable instead of access(,X_OK)' \ $(_sc_search_regexp) =20 @@ -2216,7 +2217,7 @@ exclude_file_name_regexp--sc_prohibit_PATH_MAX =3D \ ^build-aux/syntax-check\.mk$$ =20 exclude_file_name_regexp--sc_prohibit_access_xok =3D \ - ^(build-aux/syntax-check\.mk|src/util/virutil\.c)$$ + ^(src/util/virutil\.c)$$ =20 exclude_file_name_regexp--sc_prohibit_asprintf =3D \ ^(build-aux/syntax-check\.mk|bootstrap.conf$$|examples/|src/util/virstri= ng\.[ch]$$|tests/vircgroupmock\.c|tools/virt-login-shell\.c|tools/nss/libvi= rt_nss\.c$$) diff --git a/scripts/check-remote-protocol.py b/scripts/check-remote-protoc= ol.py new file mode 100644 index 0000000000..074f8c0ae1 --- /dev/null +++ b/scripts/check-remote-protocol.py @@ -0,0 +1,136 @@ +#!/usr/bin/env python +# +# Copyright (C) 2019 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . +# +# This uses pdwtags to check remote protocol defs +# +# * the "split" splits on the /* DD */ comments, so that $p iterates +# through the struct definitions. +# * process only "struct remote_..." entries +# * remove comments and preceding TAB throughout +# * remove empty lines throughout +# * remove white space at end of buffer + +from __future__ import print_function + +import os +import os.path +import re +import subprocess +import sys + +cc =3D sys.argv[1] +objext =3D sys.argv[2] +proto_lo =3D sys.argv[3] +expected =3D sys.argv[4] + +proto_lo =3D proto_lo.replace("/", "/.libs/") + +ccargv =3D cc.split(" ") +ccargv.append("-v") +ccproc =3D subprocess.Popen(ccargv, stdout=3Dsubprocess.PIPE, + stderr=3Dsubprocess.STDOUT) +out, err =3D ccproc.communicate() +out =3D out.decode("utf-8") +if out.find("clang") !=3D -1: + print("WARNING: skipping pdwtags test with Clang", file=3Dsys.stderr) + sys.exit(0) + + +def which(program): + def is_exe(fpath): + return (os.path.isfile(fpath) and + os.access(fpath, os.X_OK)) + + fpath, fname =3D os.path.split(program) + if fpath: + if is_exe(program): + return program + else: + for path in os.environ["PATH"].split(os.pathsep): + exe_file =3D os.path.join(path, program) + if is_exe(exe_file): + return exe_file + + return None + + +pdwtags =3D which("pdwtags") +if pdwtags is None: + print("WARNING: you lack pdwtags; skipping the protocol test", + file=3Dsys.stderr) + print("WARNING: install the dwarves package to get pdwtags", + file=3Dsys.stderr) + sys.exit(0) + +proto_o =3D proto_lo.replace(".lo", ".o") + +if not os.path.exists(proto_o): + raise Exception("Missing %s", proto_o) + +pdwtagsproc =3D subprocess.Popen(["pdwtags", "--verbose", proto_o], + stdout=3Dsubprocess.PIPE, stderr=3Dsubproce= ss.PIPE) +out, err =3D pdwtagsproc.communicate() +out =3D out.decode("utf-8") +err =3D err.decode("utf-8") + +if out =3D=3D "" and err !=3D "": + print("WARNING: no output, pdwtags appears broken:", file=3Dsys.stderr) + for l in err.strip().split("\n"): + print("WARNING: %s" % l, file=3Dsys.stderr) + print("WARNING: skipping the remote protocol test", file=3Dsys.stderr) + sys.exit(0) + +# With pdwtags 1.8, --verbose output includes separators like these: +# /* 93 */ +# /* <0> (null):0 */ +# with the second line omitted for intrinsic types. +# Whereas with pdwtags 1.3, they look like this: +# /* <2d2> /usr/include/libio.h:180 */ +# The alternation of the following regexps matches both cases. +r1 =3D r'''/\* \d+ \*/''' +r2 =3D r'''/\* <[0-9a-fA-F]+> \S+:\d+ \*/''' + +libs_prefix =3D "remote_|qemu_|lxc_|admin_" +other_prefix =3D "keepalive|vir(Net|LockSpace|LXCMonitor)" +struct_prefix =3D "(" + libs_prefix + "|" + other_prefix + ")" + +n =3D 0 +bits =3D re.split(r'''\n*(?:%s|%s)\n''' % (r1, r2), out) +actual =3D ["/* -*- c -*- */"] + +for bit in bits: + if re.search(r'''^(struct|enum)\s+''' + struct_prefix, bit): + bit =3D re.sub(r'''\t*/\*.*?\*/''', "", bit) + bit =3D re.sub(r'''\s+\n''', '''\n''', bit) + bit =3D re.sub(r'''\s+$''', "", bit) + bit =3D re.sub(r'''\t''', " ", bit) + actual.append(bit) + n =3D n + 1 + +if n < 1: + print("WARNING: No structs/enums matched. Your", file=3Dsys.stderr) + print("WARNING: pdwtags program is probably too old", file=3Dsys.stder= r) + print("WARNING: skipping the remote protocol test", file=3Dsys.stderr) + print("WARNING: install dwarves-1.3 or newer", file=3Dsys.stderr) + sys.exit(8) + +diff =3D subprocess.Popen(["diff", "-u", expected, "-"], stdin=3Dsubproces= s.PIPE) +actualstr =3D "\n".join(actual) + "\n" +diff.communicate(input=3Dactualstr.encode("utf-8")) + +sys.exit(diff.returncode) diff --git a/src/Makefile.am b/src/Makefile.am index bb63e2486c..c40e61e7ee 100644 --- a/src/Makefile.am +++ b/src/Makefile.am @@ -196,83 +196,6 @@ DRIVER_SOURCES +=3D \ =20 =20 =20 -# Ensure that we don't change the struct or member names or member ordering -# in remote_protocol.x The embedded perl below needs a few comments, and -# presumes you know what pdwtags output looks like: -# * use -0777 -n to slurp the entire file into $_. -# * the "split" splits on the /* DD */ comments, so that $p iterates -# through the struct definitions. -# * process only "struct remote_..." entries -# * remove comments and preceding TAB throughout -# * remove empty lines throughout -# * remove white space at end of buffer - -# With pdwtags 1.8, --verbose output includes separators like these: -# /* 93 */ -# /* <0> (null):0 */ -# with the second line omitted for intrinsic types. -# Whereas with pdwtags 1.3, they look like this: -# /* <2d2> /usr/include/libio.h:180 */ -# The alternation of the following regexps matches both cases. -r1 =3D /\* \d+ \*/ -r2 =3D /\* <[[:xdigit:]]+> \S+:\d+ \*/ -libs_prefix =3D remote_|qemu_|lxc_|admin_ -other_prefix =3D keepalive|vir(Net|LockSpace|LXCMonitor) -struct_prefix =3D ($(libs_prefix)|$(other_prefix)) - -# Depending on configure options, libtool creates one or both of -# remote/{,.libs/}libvirt_driver_remote_la-remote_protocol.o. We want -# the newest of the two, in case configure options changed and a stale -# file is left around from an earlier build. -# The pdwtags output is completely different when building with clang -# which causes the comparison against expected output to fail, so skip -# if using clang as CC. -PDWTAGS =3D \ - $(AM_V_GEN)if $(CC) -v 2>&1 | grep -q clang; then \ - echo 'WARNING: skipping pdwtags test with Clang' >&2; \ - exit 0; \ - fi; \ - if (pdwtags --help) > /dev/null 2>&1; then \ - o=3D`ls -t $(<:.lo=3D.$(OBJEXT)) \ - $(subst /,/.libs/,$(<:.lo=3D.$(OBJEXT))) \ - 2>/dev/null | sed -n 1p`; \ - test -f "$$o" || { echo ".o for $< not found" >&2; exit 1; }; \ - pdwtags --verbose $$o > $(@F)-t1 2> $(@F)-t2; \ - if test ! -s $(@F)-t1 && test -s $(@F)-t2; then \ - rm -rf $(@F)-t?; \ - echo 'WARNING: pdwtags appears broken; skipping the $@ test' >&2;\ - else \ - $(PERL) -0777 -n \ - -e 'foreach my $$p (split m!\n*(?:$(r1)|$(r2))\n!) {' \ - -e ' if ($$p =3D~ /^(struct|enum) $(struct_prefix)/) {' \ - -e ' $$p =3D~ s!\t*/\*.*?\*/!!sg;' \ - -e ' $$p =3D~ s!\s+\n!\n!sg;' \ - -e ' $$p =3D~ s!\s+$$!!;' \ - -e ' $$p =3D~ s!\t! !g;' \ - -e ' print "$$p\n";' \ - -e ' $$n++;' \ - -e ' }' \ - -e '}' \ - -e 'BEGIN {' \ - -e ' print "/* -*- c -*- */\n";' \ - -e '}' \ - -e 'END {' \ - -e ' if ($$n < 1) {' \ - -e ' warn "WARNING: your pdwtags program is too old\n";' \ - -e ' warn "WARNING: skipping the $@ test\n";' \ - -e ' warn "WARNING: install dwarves-1.3 or newer\n";' \ - -e ' exit 8;' \ - -e ' }' \ - -e '}' \ - < $(@F)-t1 > $(@F)-t3; \ - case $$? in 8) rm -f $(@F)-t?; exit 0;; 0) ;; *) exit 1;; esac;\ - diff -u $(@)s $(@F)-t3; st=3D$$?; rm -f $(@F)-t?; exit $$st; \ - fi; \ - else \ - echo 'WARNING: you lack pdwtags; skipping the $@ test' >&2; \ - echo 'WARNING: install the dwarves package to get pdwtags' >&2; \ - fi - # .libs/libvirt.so is built by libtool as a side-effect of the Makefile # rule for libvirt.la. However, checking symbols relies on Linux ELF layo= ut if WITH_LINUX @@ -301,27 +224,38 @@ PROTOCOL_STRUCTS =3D \ if WITH_REMOTE check-protocol: $(PROTOCOL_STRUCTS) $(PROTOCOL_STRUCTS:structs=3Dstruct) =20 +# Ensure that we don't change the struct or member names or member ordering +# in remote_protocol.x The process-pdwtags.py post-processes output to +# extract the bits we want. + +CHECK_REMOTE_PROTOCOL =3D $(top_srcdir)/scripts/check-remote-protocol.py + # The .o file that pdwtags parses is created as a side effect of running # libtool; but from make's perspective we depend on the .lo file. $(srcdir)/remote_protocol-struct \ $(srcdir)/qemu_protocol-struct \ $(srcdir)/lxc_protocol-struct: \ $(srcdir)/%-struct: remote/libvirt_driver_remote_la-%.lo - $(PDWTAGS) + $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(CHECK_REMOTE_PROTOCOL) \ + "$(CC)" "$(OBJEXT)" $< $(@)s $(srcdir)/virnetprotocol-struct $(srcdir)/virkeepaliveprotocol-struct: \ $(srcdir)/%-struct: rpc/libvirt_net_rpc_la-%.lo - $(PDWTAGS) + $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(CHECK_REMOTE_PROTOCOL) \ + "$(CC)" "$(OBJEXT)" $< $(@)s if WITH_LXC $(srcdir)/lxc_monitor_protocol-struct: \ $(srcdir)/%-struct: lxc/libvirt_driver_lxc_impl_la-%.lo - $(PDWTAGS) + $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(CHECK_REMOTE_PROTOCOL) \ + "$(CC)" "$(OBJEXT)" $< $(@)s endif WITH_LXC $(srcdir)/lock_protocol-struct: \ $(srcdir)/%-struct: locking/lockd_la-%.lo - $(PDWTAGS) + $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(CHECK_REMOTE_PROTOCOL) \ + "$(CC)" "$(OBJEXT)" $< $(@)s $(srcdir)/admin_protocol-struct: \ $(srcdir)/%-struct: admin/libvirt_admin_la-%.lo - $(PDWTAGS) + $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(CHECK_REMOTE_PROTOCOL) \ + "$(CC)" "$(OBJEXT)" $< $(@)s =20 else !WITH_REMOTE # The $(PROTOCOL_STRUCTS) files must live in git, because they cannot be --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sun May 5 18:51:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1573483529; cv=none; d=zoho.com; s=zohoarc; b=KV7UtITq9nGUVFdxlRXkvxqkNRBF7fd59cqgKwCzlM+1SLY8iJt9yiZ7jdjwtpblCbpIzqwJoPzbd43fN+GRG9Re8dIRFpeMTxsuucJPJaTQEbjtWiC5wOX01WLUtX90ITTp+VawyoKOYuwmXOBtRIrlQtI2xb++/bsy2Tz1iZc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573483529; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=bpGMD0Hr0bLAIfPbD/aRJHAUNYFqdzsGzQLl/epwzx8=; b=OQOjOWhhbowgIegjN8MsWm6uymec4HqOc18Qg2SEBv9axKSutE9VfZM0XPhk0Q6kfkxYbjb93XomUmW2YhsfMvyMAwPVuK+inPxxZc13uYrYJ5Rnxp+SHyfVcwDjhemX/5IUj3JfCpl6dy2+gbjBPZlbZq9UWTWN1o/DCFP4zoM= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 1573483529330974.9561523451071; Mon, 11 Nov 2019 06:45:29 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-299-cMtG87UWNi2aD0iW7_QdLA-1; Mon, 11 Nov 2019 09:45:26 -0500 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E317D107ACE5; Mon, 11 Nov 2019 14:45:19 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B6F0B851A; Mon, 11 Nov 2019 14:45:19 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 6A9304E597; Mon, 11 Nov 2019 14:45:19 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xABEj3jG020500 for ; Mon, 11 Nov 2019 09:45:03 -0500 Received: by smtp.corp.redhat.com (Postfix) id 4835618B91; Mon, 11 Nov 2019 14:45:03 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 421DE5D9C9; Mon, 11 Nov 2019 14:45:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573483528; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=bpGMD0Hr0bLAIfPbD/aRJHAUNYFqdzsGzQLl/epwzx8=; b=f7j+eAZ7TWNIktLWScB8ltNiwGR77EVWyryZPNs/c+yMT/GMB8ngYjn+QoFNz2hmB9+p/4 Gw/UNQXv+qdajslDbEELiy2WRn+QfHFKyWUPjF+0Z4sy52RvZMAXZy0Mka+96dg/d5+fhs P2Xwwofy2N5fOeeZwXBQyznNghhaFwQ= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 11 Nov 2019 14:38:18 +0000 Message-Id: <20191111143826.16050-16-berrange@redhat.com> In-Reply-To: <20191111143826.16050-1-berrange@redhat.com> References: <20191111143826.16050-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v5 15/23] tests: rewrite test argv line wrapper in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-MC-Unique: cMtG87UWNi2aD0iW7_QdLA-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" As part of an goal to eliminate Perl from libvirt build tools, rewrite the test-wrap-argv.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: J=C3=A1n Tomko --- Makefile.am | 1 + build-aux/syntax-check.mk | 4 +- scripts/test-wrap-argv.py | 170 +++++++++++++++++++++++++++++++++++++ tests/test-wrap-argv.pl | 174 -------------------------------------- tests/testutils.c | 16 ++-- 5 files changed, 181 insertions(+), 184 deletions(-) create mode 100755 scripts/test-wrap-argv.py delete mode 100755 tests/test-wrap-argv.pl diff --git a/Makefile.am b/Makefile.am index 8c9c73c715..6f6cead526 100644 --- a/Makefile.am +++ b/Makefile.am @@ -61,6 +61,7 @@ EXTRA_DIST =3D \ scripts/minimize-po.py \ scripts/mock-noinline.py \ scripts/prohibit-duplicate-header.py \ + scripts/test-wrap-argv.py \ build-aux/syntax-check.mk \ build-aux/useless-if-before-free \ build-aux/vc-list-files \ diff --git a/build-aux/syntax-check.mk b/build-aux/syntax-check.mk index a61818855c..7d54df182a 100644 --- a/build-aux/syntax-check.mk +++ b/build-aux/syntax-check.mk @@ -2172,8 +2172,8 @@ header-ifdef: $(PYTHON) $(top_srcdir)/scripts/header-ifdef.py =20 test-wrap-argv: - $(AM_V_GEN)$(VC_LIST) | $(GREP) -E '\.(ldargs|args)' | xargs \ - $(PERL) $(top_srcdir)/tests/test-wrap-argv.pl --check + $(AM_V_GEN)$(VC_LIST) | $(GREP) -E '\.(ldargs|args)' | $(RUNUTF8) xargs \ + $(PYTHON) $(top_srcdir)/scripts/test-wrap-argv.py --check =20 group-qemu-caps: $(AM_V_GEN)$(PERL) $(top_srcdir)/tests/group-qemu-caps.pl --check $(top_s= rcdir)/ diff --git a/scripts/test-wrap-argv.py b/scripts/test-wrap-argv.py new file mode 100755 index 0000000000..dd9e7de824 --- /dev/null +++ b/scripts/test-wrap-argv.py @@ -0,0 +1,170 @@ +#!/usr/bin/env python +# +# Copyright (C) 2019 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . +# +# This script is intended to be passed a list of .args files, used +# to store command line ARGV for the test suites. It will reformat +# them such that there is at most one '-param value' on each line +# of the file. Parameter values that are longer than 80 chars will +# also be split. +# +# If --in-place is supplied as the first parameter of this script, +# the files will be changed in place. +# If --check is the first parameter, the script will return +# a non-zero value if a file is not wrapped correctly. +# Otherwise the rewrapped files are printed to the standard output. + +from __future__ import print_function + +import argparse +import subprocess +import sys + + +def rewrap_line(line): + bits =3D line.split(" ") + + # bits contains env vars, then the command line + # and then the arguments + env =3D [] + cmd =3D None + args =3D [] + + if bits[0].find("=3D") =3D=3D -1: + cmd =3D bits[0] + bits =3D bits[1:] + + for bit in bits: + # If no command is defined yet, we must still + # have env vars + if cmd is None: + # Look for leading / to indicate command name + if bit.startswith("/"): + cmd =3D bit + else: + env.append(bit) + else: + # If there's a leading '-' then this is a new + # parameter, otherwise its a value for the prev + # parameter. + if bit.startswith("-"): + args.append(bit) + else: + args[-1] =3D args[-1] + " " + bit + + # We might have to split line argument values... + args =3D [rewrap_arg(arg) for arg in args] + + # Print env + command first + return " \\\n".join(env + [cmd] + args) + "\n" + + +def rewrap_arg(arg): + ret =3D [] + max_len =3D 78 + + while len(arg) > max_len: + split =3D arg.rfind(",", 0, max_len + 1) + if split =3D=3D -1: + split =3D arg.rfind(":", 0, max_len + 1) + if split =3D=3D -1: + split =3D arg.rfind(" ", 0, max_len + 1) + if split =3D=3D -1: + print("cannot find nice place to split '%s' below 80 chars" % + arg, file=3Dsys.stderr) + split =3D max_len - 1 + + split =3D split + 1 + + ret.append(arg[0:split]) + arg =3D arg[split:] + + ret.append(arg) + return "\\\n".join(ret) + + +def rewrap(filename, in_place, check): + # Read the original file + with open(filename, 'r') as fh: + orig_lines =3D [] + for line in fh: + orig_lines.append(line) + + if len(orig_lines) =3D=3D 0: + return + + lines =3D [] + for line in orig_lines: + if line.endswith("\\\n"): + line =3D line[:-2] + lines.append(line) + + # Kill the last new line in the file + lines[-1] =3D lines[-1].rstrip("\n") + + # Reconstruct the master data by joining all lines + # and then split again based on the real desired + # newlines + lines =3D "".join(lines).split("\n") + + # Now each 'lines' entry represents a single command, we + # can process them + new_lines =3D [] + for line in lines: + new_lines.append(rewrap_line(line)) + + if in_place: + with open(filename, "w") as fh: + for line in new_lines: + print(line, file=3Dfh) + elif check: + orig =3D "".join(orig_lines) + new =3D "".join(new_lines) + if new !=3D orig: + diff =3D subprocess.Popen(["diff", "-u", filename, "-"], + stdin=3Dsubprocess.PIPE) + diff.communicate(input=3Dnew.encode('utf-8')) + + print("Incorrect line wrapping in $file", + file=3Dsys.stderr) + print("Use test-wrap-argv.py to wrap test data files", + file=3Dsys.stderr) + return False + else: + for line in new_lines: + print(line) + + return True + + +parser =3D argparse.ArgumentParser(description=3D'Test arg line wrapper') +parser.add_argument('--in-place', '-i', action=3D"store_true", + help=3D'modify files in-place') +parser.add_argument('--check', action=3D"store_true", + help=3D'check existing files only') +parser.add_argument('files', nargs=3D"+", + help=3D"filenames to check") +args =3D parser.parse_args() + +errs =3D False +for filename in args.files: + if not rewrap(filename, args.in_place, args.check): + errs =3D True + +if errs: + sys.exit(1) +sys.exit(0) diff --git a/tests/test-wrap-argv.pl b/tests/test-wrap-argv.pl deleted file mode 100755 index 7867e9d719..0000000000 --- a/tests/test-wrap-argv.pl +++ /dev/null @@ -1,174 +0,0 @@ -#!/usr/bin/env perl -# -# Copyright (C) 2015 Red Hat, Inc. -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library. If not, see -# . -# -# This script is intended to be passed a list of .args files, used -# to store command line ARGV for the test suites. It will reformat -# them such that there is at most one '-param value' on each line -# of the file. Parameter values that are longer than 80 chars will -# also be split. -# -# If --in-place is supplied as the first parameter of this script, -# the files will be changed in place. -# If --check is the first parameter, the script will return -# a non-zero value if a file is not wrapped correctly. -# Otherwise the rewrapped files are printed to the standard output. - -$in_place =3D 0; -$check =3D 0; - -if (@ARGV[0] eq "--in-place" or @ARGV[0] eq "-i") { - $in_place =3D 1; - shift @ARGV; -} elsif (@ARGV[0] eq "--check") { - $check =3D 1; - shift @ARGV; -} - -$ret =3D 0; -foreach my $file (@ARGV) { - if (&rewrap($file) < 0) { - $ret =3D 1; - } -} - -exit $ret; - -sub rewrap { - my $file =3D shift; - - # Read the original file - open FILE, "<", $file or die "cannot read $file: $!"; - my @orig_lines =3D ; - close FILE; - my @lines =3D @orig_lines; - foreach (@lines) { - # If there is a trailing '\' then kill the new line - if (/\\$/) { - chomp; - $_ =3D~ s/\\$//; - } - } - - # Skip empty files - return unless @lines; - - # Kill the last new line in the file - chomp @lines[$#lines]; - - # Reconstruct the master data by joining all lines - # and then split again based on the real desired - # newlines - @lines =3D split /\n/, join('', @lines); - - # Now each @lines represents a single command, we - # can process them - @lines =3D map { &rewrap_line($_) } @lines; - - if ($in_place) { - open FILE, ">", $file or die "cannot write $file: $!"; - foreach my $line (@lines) { - print FILE $line; - } - close FILE; - } elsif ($check) { - my $nl =3D join('', @lines); - my $ol =3D join('', @orig_lines); - unless ($nl eq $ol) { - open DIFF, "| diff -u $file -" or die "cannot run diff: $!"; - print DIFF $nl; - close DIFF; - - print STDERR "Incorrect line wrapping in $file\n"; - print STDERR "Use test-wrap-argv.pl to wrap test data files\n"; - return -1; - } - } else { - foreach my $line (@lines) { - print $line; - } - } - return 0; -} - -sub rewrap_line { - my $line =3D shift; - my @bits =3D split / /, join('', $line); - - # @bits contains env vars, then the command line - # and then the arguments - my @env; - my $cmd; - my @args; - - if ($bits[0] !~ /=3D/) { - $cmd =3D shift @bits; - } - - foreach my $bit (@bits) { - # If no command is defined yet, we must still - # have env vars - if (!defined $cmd) { - # Look for leading / to indicate command name - if ($bit =3D~ m,^/,) { - $cmd =3D $bit; - } else { - push @env, $bit; - } - } else { - # If there's a leading '-' then this is a new - # parameter, otherwise its a value for the prev - # parameter. - if ($bit =3D~ m,^-,) { - push @args, $bit; - } else { - $args[$#args] .=3D " " . $bit; - } - } - } - - # We might have to split line argument values... - @args =3D map { &rewrap_arg($_) } @args; - # Print env + command first - return join(" \\\n", @env, $cmd, @args), "\n"; -} - -sub rewrap_arg { - my $arg =3D shift; - my @ret; - my $max_len =3D 78; - - while (length($arg) > $max_len) { - my $split =3D rindex $arg, ",", $max_len; - if ($split =3D=3D -1) { - $split =3D rindex $arg, ":", $max_len; - } - if ($split =3D=3D -1) { - $split =3D rindex $arg, " ", $max_len; - } - if ($split =3D=3D -1) { - warn "cannot find nice place to split '$arg' below 80 chars\n"; - $split =3D $max_len - 1; - } - $split++; - - push @ret, substr $arg, 0, $split; - $arg =3D substr $arg, $split; - } - push @ret, $arg; - return join("\\\n", @ret); -} diff --git a/tests/testutils.c b/tests/testutils.c index da236c74a1..1acdc71fca 100644 --- a/tests/testutils.c +++ b/tests/testutils.c @@ -59,7 +59,7 @@ static size_t testCounter; static virBitmapPtr testBitmap; =20 char *progname; -static char *perl; +static char *python; =20 static int virTestUseTerminalColors(void) { @@ -399,15 +399,15 @@ virTestRewrapFile(const char *filename) virStringHasSuffix(filename, ".ldargs"))) return 0; =20 - if (!perl) { - fprintf(stderr, "cannot rewrap %s: unable to find perl in path", f= ilename); + if (!python) { + fprintf(stderr, "cannot rewrap %s: unable to find python in path",= filename); return -1; } =20 - if (virAsprintf(&script, "%s/test-wrap-argv.pl", abs_srcdir) < 0) + if (virAsprintf(&script, "%s/scripts/test-wrap-argv.py", abs_top_srcdi= r) < 0) goto cleanup; =20 - cmd =3D virCommandNewArgList(perl, script, "--in-place", filename, NUL= L); + cmd =3D virCommandNewArgList(python, script, "--in-place", filename, N= ULL); if (virCommandRun(cmd, NULL) < 0) goto cleanup; =20 @@ -895,8 +895,8 @@ int virTestMain(int argc, } } =20 - /* Find perl early because some tests override PATH */ - perl =3D virFindFileInPath("perl"); + /* Find python early because some tests override PATH */ + python =3D virFindFileInPath("python"); =20 ret =3D (func)(); =20 @@ -907,7 +907,7 @@ int virTestMain(int argc, fprintf(stderr, " %-3zu %s\n", testCounter, ret =3D=3D 0 ? "OK" : = "FAIL"); } virLogReset(); - VIR_FREE(perl); + VIR_FREE(python); return ret; } =20 --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sun May 5 18:51:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 205.139.110.120 as permitted sender) client-ip=205.139.110.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1573483533; cv=none; d=zoho.com; s=zohoarc; b=fNBY02APVZQD0tqVfVLMBB5CECYyS1TXCQlYYzy7NDnHdz1su04aEbg4kBdr26PNEc/he08Gv5PqR+kPYOUKt8Svas1Q6ihXUe/SMXoV2MEgeQ7F7nsgMPY2saysCcmBJg9dVgUu6RmBh0PxStDCDjkLkBWJa5yWmL/oDCo2kbU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573483533; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=9YbELyDUG90M4vLIhp5Vj4/jVfVt7SaOSBxPNma/Lcs=; b=mMh7+EZ1whnkM7Gcgpvi+s3wO/8fo5C/t+u43nZ73E8CtCwCzba2eip8s8xc48UBL89sMV+Nyy/+4NJ+6U51sW6XH8efxYg0CKWM1YhcfM1XnMY4Am+2rcBWiwePJPsGfwX6/b074MMX2Qc4ErB0GzWySXZ6bhBrmd9PGBbpvg4= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by mx.zohomail.com with SMTPS id 1573483533150174.90540112526242; Mon, 11 Nov 2019 06:45:33 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-288-zuudlnZGPKCYkSmU_x8sgg-1; Mon, 11 Nov 2019 09:45:29 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id EA9908A2422; Mon, 11 Nov 2019 14:45:22 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id BD35D4B5; Mon, 11 Nov 2019 14:45:22 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 72BA6180221C; Mon, 11 Nov 2019 14:45:22 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xABEj8kC020534 for ; Mon, 11 Nov 2019 09:45:08 -0500 Received: by smtp.corp.redhat.com (Postfix) id A9D575DD73; Mon, 11 Nov 2019 14:45:08 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id E0DEF1B41F; Mon, 11 Nov 2019 14:45:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573483531; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=9YbELyDUG90M4vLIhp5Vj4/jVfVt7SaOSBxPNma/Lcs=; b=TiPXX+ofrU7p5GaSgh7EzdAuAQXY28jiOx+yc3E/gS9er2yBpo2qA64p5sQ+afIqqJWkq7 pwETFlWa7Ea1vg9MeZHA4vrLUoWswdbjv5arOevyILwa3i4cdK02DW7O9LRurwY1QA+R6e 4cSxLq76nNA6sqQccnhYYfoDaW4CPoI= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 11 Nov 2019 14:38:19 +0000 Message-Id: <20191111143826.16050-17-berrange@redhat.com> In-Reply-To: <20191111143826.16050-1-berrange@redhat.com> References: <20191111143826.16050-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v5 16/23] tests: rewrite qemu capability grouper in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-MC-Unique: zuudlnZGPKCYkSmU_x8sgg-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" As part of an goal to eliminate Perl from libvirt build tools, rewrite the group-qemu-caps.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. Signed-off-by: Daniel P. Berrang=C3=A9 Tested-by: Cole Robinson --- Makefile.am | 1 + build-aux/syntax-check.mk | 3 +- scripts/group-qemu-caps.py | 123 ++++++++++++++++++++++++++++++++++++ tests/group-qemu-caps.pl | 124 ------------------------------------- 4 files changed, 126 insertions(+), 125 deletions(-) create mode 100755 scripts/group-qemu-caps.py delete mode 100755 tests/group-qemu-caps.pl diff --git a/Makefile.am b/Makefile.am index 6f6cead526..769cd4ce64 100644 --- a/Makefile.am +++ b/Makefile.am @@ -57,6 +57,7 @@ EXTRA_DIST =3D \ scripts/dtrace2systemtap.py \ scripts/genpolkit.py \ scripts/gensystemtap.py \ + scripts/group-qemu-caps.py \ scripts/header-ifdef.py \ scripts/minimize-po.py \ scripts/mock-noinline.py \ diff --git a/build-aux/syntax-check.mk b/build-aux/syntax-check.mk index 7d54df182a..44639f499e 100644 --- a/build-aux/syntax-check.mk +++ b/build-aux/syntax-check.mk @@ -2176,7 +2176,8 @@ test-wrap-argv: $(PYTHON) $(top_srcdir)/scripts/test-wrap-argv.py --check =20 group-qemu-caps: - $(AM_V_GEN)$(PERL) $(top_srcdir)/tests/group-qemu-caps.pl --check $(top_s= rcdir)/ + $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(top_srcdir)/scripts/group-qemu-caps.py \ + --check --prefix $(top_srcdir)/ =20 # List all syntax-check exemptions: exclude_file_name_regexp--sc_avoid_strcase =3D ^tools/vsh\.h$$ diff --git a/scripts/group-qemu-caps.py b/scripts/group-qemu-caps.py new file mode 100755 index 0000000000..3edaf5d09f --- /dev/null +++ b/scripts/group-qemu-caps.py @@ -0,0 +1,123 @@ +#!/usr/bin/env python +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . +# +# +# Regroup array values into smaller groups separated by numbered comments. +# +# If --check is the first parameter, the script will return +# a non-zero value if a file is not grouped correctly. +# Otherwise the files are regrouped in place. + +from __future__ import print_function + +import argparse +import re +import subprocess +import sys + + +def regroup_caps(check, filename, start_regex, end_regex, + trailing_newline, counter_prefix): + step =3D 5 + + original =3D [] + with open(filename, "r") as fh: + for line in fh: + original.append(line) + + fixed =3D [] + game_on =3D False + counter =3D 0 + for line in original: + line =3D line.rstrip("\n") + if game_on: + if re.search(r'''.*/\* [0-9]+ \*/.*''', line): + continue + if re.search(r'''^\s*$''', line): + continue + if counter % step =3D=3D 0: + if counter !=3D 0: + fixed.append("\n") + fixed.append("%s/* %d */\n" % (counter_prefix, counter)) + + if not (line.find("/*") !=3D -1 and line.find("*/") =3D=3D -1): + # count two-line comments as one line + counter =3D counter + 1 + + if re.search(start_regex, line): + game_on =3D True + elif game_on and re.search(end_regex, line): + if (counter - 1) % step =3D=3D 0: + fixed =3D fixed[:-1] # /* $counter */ + if counter !=3D 1: + fixed =3D fixed[:-1] # \n + + if trailing_newline: + fixed.append("\n") + + game_on =3D False + + fixed.append(line + "\n") + + if check: + orig =3D "".join(original) + new =3D "".join(fixed) + if new !=3D orig: + diff =3D subprocess.Popen(["diff", "-u", filename, "-"], + stdin=3Dsubprocess.PIPE) + diff.communicate(input=3Dnew.encode('utf-8')) + + print("Incorrect line wrapping in $file", + file=3Dsys.stderr) + print("Use test-wrap-argv.py to wrap test data files", + file=3Dsys.stderr) + return False + else: + with open(filename, "w") as fh: + for line in fixed: + print(line, file=3Dfh, end=3D'') + + return True + + +parser =3D argparse.ArgumentParser(description=3D'Test arg line wrapper') +parser.add_argument('--check', action=3D"store_true", + help=3D'check existing files only') +parser.add_argument('--prefix', default=3D'', + help=3D'source code tree prefix') +args =3D parser.parse_args() + +errs =3D False + +if not regroup_caps(args.check, + args.prefix + 'src/qemu/qemu_capabilities.c', + r'virQEMUCaps grouping marker', + r'\);', + 0, + " "): + errs =3D True + +if not regroup_caps(args.check, + args.prefix + 'src/qemu/qemu_capabilities.h', + r'virQEMUCapsFlags grouping marker', + r'QEMU_CAPS_LAST \/\* this must', + 1, + " "): + errs =3D True + +if errs: + sys.exit(1) +sys.exit(0) diff --git a/tests/group-qemu-caps.pl b/tests/group-qemu-caps.pl deleted file mode 100755 index 829e63a562..0000000000 --- a/tests/group-qemu-caps.pl +++ /dev/null @@ -1,124 +0,0 @@ -#!/usr/bin/env perl -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library. If not, see -# . -# -# -# Regroup array values into smaller groups separated by numbered comments. -# -# If --check is the first parameter, the script will return -# a non-zero value if a file is not grouped correctly. -# Otherwise the files are regrouped in place. - -use strict; -use warnings; - -my $check =3D 0; - -if (defined $ARGV[0] && $ARGV[0] eq "--check") { - $check =3D 1; - shift @ARGV; -} - -my $prefix =3D ''; -if (defined $ARGV[0]) { - $prefix =3D $ARGV[0]; - shift @ARGV; -} - -my $ret =3D 0; -if (®roup_caps($prefix . 'src/qemu/qemu_capabilities.c', - 'virQEMUCaps grouping marker', - '\);', - 0, - " ") < 0) { - $ret =3D 1; -} -if (®roup_caps($prefix . 'src/qemu/qemu_capabilities.h', - 'virQEMUCapsFlags grouping marker', - 'QEMU_CAPS_LAST \/\* this must', - 1, - " ") < 0) { - $ret =3D 1; -} - -exit $ret; - -sub regroup_caps { - my $filename =3D shift; - my $start_regex =3D shift; - my $end_regex =3D shift; - my $trailing_newline =3D shift; - my $counter_prefix =3D shift; - my $step =3D 5; - - open FILE, '<', $filename or die "cannot open $filename: $!"; - my @original =3D ; - close FILE; - - my @fixed; - my $game_on =3D 0; - my $counter =3D 0; - foreach (@original) { - if ($game_on) { - next if ($_ =3D~ '/\* [0-9]+ \*/'); - next if (/^\s+$/); - if ($counter % $step =3D=3D 0) { - if ($counter !=3D 0) { - push @fixed, "\n"; - } - push @fixed, "$counter_prefix/* $counter */\n"; - } - if (!($_ =3D~ '/\*' && !($_ =3D~ '\*/'))) { - # count two-line comments as one line - $counter++; - } - } - if (/$start_regex/) { - $game_on =3D 1; - } elsif ($game_on && $_ =3D~ /$end_regex/) { - if (($counter -1) % $step =3D=3D 0) { - pop @fixed; # /* $counter */ - if ($counter !=3D 1) { - pop @fixed; # \n - } - } - if ($trailing_newline) { - push @fixed, "\n"; - } - $game_on =3D 0; - } - push @fixed, $_; - } - - if ($check) { - my $nl =3D join('', @fixed); - my $ol =3D join('', @original); - unless ($nl eq $ol) { - open DIFF, "| diff -u $filename -" or die "cannot run diff: $!= "; - print DIFF $nl; - close DIFF; - - print STDERR "Incorrect array grouping in $filename\n"; - print STDERR "Use group-qemu-caps.pl to group long array membe= rs\n"; - return -1; - } - } else { - open FILE, '>', $filename or die "cannot open $filename: $!"; - foreach my $line (@fixed) { - print FILE $line; - } - close FILE; - } -} --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sun May 5 18:51:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1573483534; cv=none; d=zoho.com; s=zohoarc; b=TIJ+rm0b6BWbY/9StADZliwgj3GrbGEo4/pmTzfIBgcmwbrIN5e/M19A9xCOYfU3WWp9eeD82mhwEBB7Ld2ZoRCa+q4x9l/irO+h7wNADpydM5SzzC4kKWTmCuM1aBn53H9QHMsTA2bJvuZbaz6TBPFud7i5WqKSI0fEagRqU0U= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573483534; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=AZizGoOw/5+vZNcCVIv+HnHefdRObZOdgRLQi5lqbWw=; b=Io/4bT6VK+lcAUt1NV7bxzK+f9kTtI9PTXMRCl6U1XDnvbmPY9q+625JwPrxhe30Pk/kbZ1E3GjUYLJgOzIPulAtBmYPVtOuQH2tNEtKyWFAFp/mmCrdpxrwuxnv6hfbldg+ve/JCpqFcm5sDMS9x8KJgMR+v/a4Ov53mlxL7Lc= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 1573483534522135.37681485294513; Mon, 11 Nov 2019 06:45:34 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-277-qedqnHraO-ew_3yJETTP9A-1; Mon, 11 Nov 2019 09:45:29 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DF8BC85EE95; Mon, 11 Nov 2019 14:45:23 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A02FC60904; Mon, 11 Nov 2019 14:45:23 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 4EEEC1802225; Mon, 11 Nov 2019 14:45:23 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xABEj9qX020545 for ; Mon, 11 Nov 2019 09:45:09 -0500 Received: by smtp.corp.redhat.com (Postfix) id AB6B55DD73; Mon, 11 Nov 2019 14:45:09 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id EF0661B41F; Mon, 11 Nov 2019 14:45:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573483533; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=AZizGoOw/5+vZNcCVIv+HnHefdRObZOdgRLQi5lqbWw=; b=PaeM7EqAR8tKd5ajqY8JRZkaDUWN+95W//RNhWsOtzjc6vtjg5zcWsJ7qPH8AIil5jlzMD /ba7Ormw5gnOwf0WrjuhqYP2K75hNGPDg4AEa/ByclSOdOIxi717n6kuLB4azMptqrMEiv pIuHXX67DOWAe7VrFdAV4dkXHZYgSM0= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 11 Nov 2019 14:38:20 +0000 Message-Id: <20191111143826.16050-18-berrange@redhat.com> In-Reply-To: <20191111143826.16050-1-berrange@redhat.com> References: <20191111143826.16050-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v5 17/23] tests: rewrite file access checker in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-MC-Unique: qedqnHraO-ew_3yJETTP9A-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" As part of an goal to eliminate Perl from libvirt build tools, rewrite the check-file-access.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. Signed-off-by: Daniel P. Berrang=C3=A9 --- Makefile.am | 1 + scripts/check-file-access.py | 123 +++++++++++++++++++++++++++++++ tests/Makefile.am | 3 +- tests/check-file-access.pl | 126 -------------------------------- tests/file_access_whitelist.txt | 2 +- 5 files changed, 126 insertions(+), 129 deletions(-) create mode 100755 scripts/check-file-access.py delete mode 100755 tests/check-file-access.pl diff --git a/Makefile.am b/Makefile.am index 769cd4ce64..19114069e3 100644 --- a/Makefile.am +++ b/Makefile.am @@ -50,6 +50,7 @@ EXTRA_DIST =3D \ scripts/check-aclrules.py \ scripts/check-drivername.py \ scripts/check-driverimpls.py \ + scripts/check-file-access.py \ scripts/check-remote-protocol.py \ scripts/check-spacing.py \ scripts/check-symfile.py \ diff --git a/scripts/check-file-access.py b/scripts/check-file-access.py new file mode 100755 index 0000000000..cdcbf2666f --- /dev/null +++ b/scripts/check-file-access.py @@ -0,0 +1,123 @@ +#!/usr/bin/env python +# +# Copyright (C) 2016-2019 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . +# +# This script is supposed to check test_file_access.txt file and +# warn about file accesses outside our working tree. +# +# + +from __future__ import print_function + +import re +import sys + +access_file =3D "test_file_access.txt" +whitelist_file =3D "file_access_whitelist.txt" + +known_actions =3D ["open", "fopen", "access", "stat", "lstat", "connect"] + +files =3D [] +whitelist =3D [] + +with open(access_file, "r") as fh: + for line in fh: + line =3D line.rstrip("\n") + + m =3D re.search(r'''^(\S*):\s*(\S*):\s*(\S*)(\s*:\s*(.*))?$''', li= ne) + if m is not None: + rec =3D { + "path": m.group(1), + "action": m.group(2), + "progname": m.group(3), + "testname": m.group(5), + } + files.append(rec) + else: + raise Exception("Malformed line %s" % line) + +with open(whitelist_file, "r") as fh: + for line in fh: + line =3D line.rstrip("\n") + + if re.search(r'''^\s*#.*$''', line): + continue # comment + if line =3D=3D "": + continue + + m =3D re.search(r'''^(\S*):\s*(\S*)(:\s*(\S*)(\s*:\s*(.*))?)?$''',= line) + if m is not None and m.group(2) in known_actions: + # $path: $action: $progname: $testname + rec =3D { + "path": m.group(1), + "action": m.group(3), + "progname": m.group(4), + "testname": m.group(6), + } + whitelist.append(rec) + else: + m =3D re.search(r'''^(\S*)(:\s*(\S*)(\s*:\s*(.*))?)?$''', line) + if m is not None: + # $path: $progname: $testname + rec =3D { + "path": m.group(1), + "action": None, + "progname": m.group(3), + "testname": m.group(5), + } + whitelist.append(rec) + else: + raise Exception("Malformed line %s" % line) + + +# Now we should check if %traces is included in $whitelist. For +# now checking just keys is sufficient +err =3D False +for file in files: + match =3D False + + for rule in whitelist: + if not re.search("^" + rule["path"], file["path"]): + continue + + if (rule["action"] is not None and + not re.search("^" + rule["action"], file["action"])): + continue + + if (rule["progname"] is not None and + not re.search("^" + rule["progname"], file["progname"])): + continue + + if (rule["testname"] is not None and + file["testname"] is not None and + not re.search("^" + rule["testname"], file["testname"])): + continue + + match =3D True + + if not match: + err =3D True + print("%s: %s: %s" % + (file["path"], file["action"], file["progname"]), + file=3Dsys.stderr, end=3D"") + if file["testname"] is not None: + print(": %s" % file["testname"], file=3Dsys.stderr, end=3D"") + print("", file=3Dsys.stderr) + +if err: + sys.exit(1) +sys.exit(0) diff --git a/tests/Makefile.am b/tests/Makefile.am index 9d9c582e42..c3bca26019 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -453,14 +453,13 @@ EXTRA_DIST +=3D $(test_scripts) if WITH_LINUX check-access: file-access-clean VIR_TEST_FILE_ACCESS=3D1 $(MAKE) $(AM_MAKEFLAGS) check - $(PERL) check-file-access.pl | sort -u + $(RUNUTF8) $(PYTHON) $(top_srcdir)/scripts/check-file-access.py | sort -u =20 file-access-clean: > test_file_access.txt endif WITH_LINUX =20 EXTRA_DIST +=3D \ - check-file-access.pl \ file_access_whitelist.txt =20 if WITH_TESTS diff --git a/tests/check-file-access.pl b/tests/check-file-access.pl deleted file mode 100755 index ea0b7a18a2..0000000000 --- a/tests/check-file-access.pl +++ /dev/null @@ -1,126 +0,0 @@ -#!/usr/bin/env perl -# -# Copyright (C) 2016 Red Hat, Inc. -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library. If not, see -# . -# -# This script is supposed to check test_file_access.txt file and -# warn about file accesses outside our working tree. -# -# - -use strict; -use warnings; - -my $access_file =3D "test_file_access.txt"; -my $whitelist_file =3D "file_access_whitelist.txt"; - -my @known_actions =3D ("open", "fopen", "access", "stat", "lstat", "connec= t"); - -my @files; -my @whitelist; - -open FILE, "<", $access_file or die "Unable to open $access_file: $!"; -while () { - chomp; - if (/^(\S*):\s*(\S*):\s*(\S*)(\s*:\s*(.*))?$/) { - my %rec; - ${rec}{path} =3D $1; - ${rec}{action} =3D $2; - ${rec}{progname} =3D $3; - if (defined $5) { - ${rec}{testname} =3D $5; - } - push (@files, \%rec); - } else { - die "Malformed line $_"; - } -} -close FILE; - -open FILE, "<", $whitelist_file or die "Unable to open $whitelist_file: $!= "; -while () { - chomp; - if (/^\s*#.*$/) { - # comment - } elsif (/^(\S*):\s*(\S*)(:\s*(\S*)(\s*:\s*(.*))?)?$/ and - grep /^$2$/, @known_actions) { - # $path: $action: $progname: $testname - my %rec; - ${rec}{path} =3D $1; - ${rec}{action} =3D $3; - if (defined $4) { - ${rec}{progname} =3D $4; - } - if (defined $6) { - ${rec}{testname} =3D $6; - } - push (@whitelist, \%rec); - } elsif (/^(\S*)(:\s*(\S*)(\s*:\s*(.*))?)?$/) { - # $path: $progname: $testname - my %rec; - ${rec}{path} =3D $1; - if (defined $3) { - ${rec}{progname} =3D $3; - } - if (defined $5) { - ${rec}{testname} =3D $5; - } - push (@whitelist, \%rec); - } else { - die "Malformed line $_"; - } -} -close FILE; - -# Now we should check if %traces is included in $whitelist. For -# now checking just keys is sufficient -my $error =3D 0; -for my $file (@files) { - my $match =3D 0; - - for my $rule (@whitelist) { - if (not %${file}{path} =3D~ m/^$rule->{path}$/) { - next; - } - - if (defined %${rule}{action} and - not %${file}{action} =3D~ m/^$rule->{action}$/) { - next; - } - - if (defined %${rule}{progname} and - not %${file}{progname} =3D~ m/^$rule->{progname}$/) { - next; - } - - if (defined %${rule}{testname} and - defined %${file}{testname} and - not %${file}{testname} =3D~ m/^$rule->{testname}$/) { - next; - } - - $match =3D 1; - } - - if (not $match) { - $error =3D 1; - print "$file->{path}: $file->{action}: $file->{progname}"; - print ": $file->{testname}" if defined %${file}{testname}; - print "\n"; - } -} - -exit $error; diff --git a/tests/file_access_whitelist.txt b/tests/file_access_whitelist.= txt index 3fb318cbab..5ec7ee63bb 100644 --- a/tests/file_access_whitelist.txt +++ b/tests/file_access_whitelist.txt @@ -5,7 +5,7 @@ # $path: $progname: $testname # $path: $action: $progname: $testname # -# All these variables are evaluated as perl RE. So to allow +# All these variables are evaluated as python RE. So to allow # /dev/sda and /dev/sdb, you can just '/dev/sd[a-b]', or to allow # /proc/$pid/status you can '/proc/\d+/status' and so on. # Moreover, $action, $progname and $testname can be empty, in which --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sun May 5 18:51:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 207.211.31.120 as permitted sender) client-ip=207.211.31.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1573483537; cv=none; d=zoho.com; s=zohoarc; b=YPojWhX//Efjfdraal7NpKat9lUNwy8jwG/9VaS0I3L9TixkwJq3e95bN+pCMYGbdrEDeoh1KP33N/qDZBX9KV0EDUyBMPD8iSWlKsBcafrIcEolJ6j97J5NSEDFYZaP9/DhkPiRyFIt4ORXCtEIPbsxZl4XuHn9Pxq9Hq0vEKw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573483537; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=+vSEQ8Gk4iKAARqJGRIlSYdqe3MZXja6mrgaFbvticc=; b=G4k/0voCLtgqTQkFIM2unae39UHnXqudrLWzi0Guufbhk+DdiM0akPc571uoj4IFOW3HsR2NWUFRmC2Dj+3s+7sbtJL4c1m5lufZM5jNqcGySDaHcqf929eO+14ypyGKtQW95TQtzBUPjRFca1VIUW4SRkYSc9j6zwfmlsAMhjQ= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [207.211.31.120]) by mx.zohomail.com with SMTPS id 157348353724365.37617993562333; Mon, 11 Nov 2019 06:45:37 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-413-OT9MUpCrMcGWuC7F33vkIg-1; Mon, 11 Nov 2019 09:45:33 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DA4B9107ACE4; Mon, 11 Nov 2019 14:45:27 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B42604B5; Mon, 11 Nov 2019 14:45:27 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 66C3C4EA4A; Mon, 11 Nov 2019 14:45:27 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xABEjBng020559 for ; Mon, 11 Nov 2019 09:45:11 -0500 Received: by smtp.corp.redhat.com (Postfix) id 6BA7717AD4; Mon, 11 Nov 2019 14:45:11 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id F13885DD73; Mon, 11 Nov 2019 14:45:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573483536; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=+vSEQ8Gk4iKAARqJGRIlSYdqe3MZXja6mrgaFbvticc=; b=V2SzXsaLvZ8SnTu/hxyVxKI95EHqmO56zsVWjcSvIj5axPSfxbsROQximauqESQbZ1iBpT xrrr5zC2H1InuSqOVsbjAaZa28TdKnWqoWExf0LuVnYsdMKYS9wd+t8DqbrKOhw3foLwv4 fccYfL639PRAhK4b7ttSG0MEbpQrptU= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 11 Nov 2019 14:38:21 +0000 Message-Id: <20191111143826.16050-19-berrange@redhat.com> In-Reply-To: <20191111143826.16050-1-berrange@redhat.com> References: <20191111143826.16050-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v5 18/23] docs: rewrite hvsupport.html page generator in python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-MC-Unique: OT9MUpCrMcGWuC7F33vkIg-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" As part of an goal to eliminate Perl from libvirt build tools, rewrite the hvsupport.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. The new impl generates byte-for-byte identical output to the old impl. Signed-off-by: Daniel P. Berrang=C3=A9 Tested-by: Cole Robinson --- Makefile.am | 1 + docs/Makefile.am | 7 +- docs/hvsupport.pl | 459 --------------------------------------- scripts/hvsupport.py | 504 +++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 508 insertions(+), 463 deletions(-) delete mode 100755 docs/hvsupport.pl create mode 100755 scripts/hvsupport.py diff --git a/Makefile.am b/Makefile.am index 19114069e3..bc8aecc4a2 100644 --- a/Makefile.am +++ b/Makefile.am @@ -60,6 +60,7 @@ EXTRA_DIST =3D \ scripts/gensystemtap.py \ scripts/group-qemu-caps.py \ scripts/header-ifdef.py \ + scripts/hvsupport.py \ scripts/minimize-po.py \ scripts/mock-noinline.py \ scripts/prohibit-duplicate-header.py \ diff --git a/docs/Makefile.am b/docs/Makefile.am index 0311bbedd8..cf83949168 100644 --- a/docs/Makefile.am +++ b/docs/Makefile.am @@ -172,7 +172,6 @@ EXTRA_DIST=3D \ $(internals_html_in) $(fonts) \ $(kbase_html_in) \ aclperms.htmlinc \ - hvsupport.pl \ $(schema_DATA) =20 acl_generated =3D aclperms.htmlinc @@ -212,12 +211,12 @@ web: $(dot_html) $(internals_html) $(kbase_html) \ =20 hvsupport.html: hvsupport.html.in =20 -hvsupport.html.in: $(srcdir)/hvsupport.pl $(api_DATA) \ +hvsupport.html.in: $(top_srcdir)/scripts/hvsupport.py $(api_DATA) \ $(top_srcdir)/src/libvirt_public.syms \ $(top_srcdir)/src/libvirt_qemu.syms $(top_srcdir)/src/libvirt_lxc.syms \ $(top_srcdir)/src/driver.h - $(AM_V_GEN)$(PERL) $(srcdir)/hvsupport.pl $(top_srcdir) $(top_builddir) >= $@ \ - || { rm $@ && exit 1; } + $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(top_srcdir)/scripts/hvsupport.py \ + $(top_srcdir) $(top_builddir) > $@ || { rm $@ && exit 1; } =20 news.html.in: \ $(srcdir)/news.xml \ diff --git a/docs/hvsupport.pl b/docs/hvsupport.pl deleted file mode 100755 index 0977098eac..0000000000 --- a/docs/hvsupport.pl +++ /dev/null @@ -1,459 +0,0 @@ -#!/usr/bin/env perl - -use strict; -use warnings; - -use File::Find; - -die "syntax: $0 SRCDIR BUILDDIR\n" unless int(@ARGV) =3D=3D 2; - -my $srcdir =3D shift @ARGV; -my $builddir =3D shift @ARGV; - -my $symslibvirt =3D "$srcdir/src/libvirt_public.syms"; -my $symsqemu =3D "$srcdir/src/libvirt_qemu.syms"; -my $symslxc =3D "$srcdir/src/libvirt_lxc.syms"; -my @drivertable =3D ( - "$srcdir/src/driver-hypervisor.h", - "$srcdir/src/driver-interface.h", - "$srcdir/src/driver-network.h", - "$srcdir/src/driver-nodedev.h", - "$srcdir/src/driver-nwfilter.h", - "$srcdir/src/driver-secret.h", - "$srcdir/src/driver-state.h", - "$srcdir/src/driver-storage.h", - "$srcdir/src/driver-stream.h", - ); - -my %groupheaders =3D ( - "virHypervisorDriver" =3D> "Hypervisor APIs", - "virNetworkDriver" =3D> "Virtual Network APIs", - "virInterfaceDriver" =3D> "Host Interface APIs", - "virNodeDeviceDriver" =3D> "Host Device APIs", - "virStorageDriver" =3D> "Storage Pool APIs", - "virSecretDriver" =3D> "Secret APIs", - "virNWFilterDriver" =3D> "Network Filter APIs", - ); - - -my @srcs; -find({ - wanted =3D> sub { - if (m!$srcdir/src/.*/\w+_(driver|common|tmpl|monitor|hal|udev)\.c$= !) { - push @srcs, $_ if $_ !~ /vbox_driver\.c/; - } - }, no_chdir =3D> 1}, "$srcdir/src"); - -# Map API=C2=A0functions to the header and documentation files they're in -# so that we can generate proper hyperlinks to their documentation. -# -# The function names are grep'd from the XML output of apibuild.py. -sub getAPIFilenames { - my $filename =3D shift; - - my %files; - my $line; - - open FILE, "<", $filename or die "cannot read $filename: $!"; - - while (defined($line =3D )) { - if ($line =3D~ /function name=3D'([^']+)' file=3D'([^']+)'/) { - $files{$1} =3D $2; - } - } - - close FILE; - - if (keys %files =3D=3D 0) { - die "No functions found in $filename. Has the apibuild.py output c= hanged?"; - } - return \%files; -} - -sub parseSymsFile { - my $apisref =3D shift; - my $prefix =3D shift; - my $filename =3D shift; - my $xmlfilename =3D shift; - - my $line; - my $vers; - my $prevvers; - - my $filenames =3D getAPIFilenames($xmlfilename); - - open FILE, "<$filename" - or die "cannot read $filename: $!"; - - while (defined($line =3D )) { - chomp $line; - next if $line =3D~ /^\s*#/; - next if $line =3D~ /^\s*$/; - next if $line =3D~ /^\s*(global|local):/; - if ($line =3D~ /^\s*${prefix}_(\d+\.\d+\.\d+)\s*{\s*$/) { - if (defined $vers) { - die "malformed syms file"; - } - $vers =3D $1; - } elsif ($line =3D~ /\s*}\s*;\s*$/) { - if (defined $prevvers) { - die "malformed syms file"; - } - $prevvers =3D $vers; - $vers =3D undef; - } elsif ($line =3D~ /\s*}\s*${prefix}_(\d+\.\d+\.\d+)\s*;\s*$/) { - if ($1 ne $prevvers) { - die "malformed syms file $1 !=3D $vers"; - } - $prevvers =3D $vers; - $vers =3D undef; - } elsif ($line =3D~ /\s*(\w+)\s*;\s*$/) { - $$apisref{$1} =3D {}; - $$apisref{$1}->{vers} =3D $vers; - $$apisref{$1}->{file} =3D $$filenames{$1}; - } else { - die "unexpected data $line\n"; - } - } - - close FILE; -} - -my %apis; -# Get the list of all public APIs and their corresponding version -parseSymsFile(\%apis, "LIBVIRT", $symslibvirt, "$builddir/docs/libvirt-api= .xml"); - -# And the same for the QEMU specific APIs -parseSymsFile(\%apis, "LIBVIRT_QEMU", $symsqemu, "$builddir/docs/libvirt-q= emu-api.xml"); - -# And the same for the LXC specific APIs -parseSymsFile(\%apis, "LIBVIRT_LXC", $symslxc, "$builddir/docs/libvirt-lxc= -api.xml"); - - -# Some special things which aren't public APIs, -# but we want to report -$apis{virConnectSupportsFeature}->{vers} =3D "0.3.2"; -$apis{virDomainMigratePrepare}->{vers} =3D "0.3.2"; -$apis{virDomainMigratePerform}->{vers} =3D "0.3.2"; -$apis{virDomainMigrateFinish}->{vers} =3D "0.3.2"; -$apis{virDomainMigratePrepare2}->{vers} =3D "0.5.0"; -$apis{virDomainMigrateFinish2}->{vers} =3D "0.5.0"; -$apis{virDomainMigratePrepareTunnel}->{vers} =3D "0.7.2"; - -$apis{virDomainMigrateBegin3}->{vers} =3D "0.9.2"; -$apis{virDomainMigratePrepare3}->{vers} =3D "0.9.2"; -$apis{virDomainMigratePrepareTunnel3}->{vers} =3D "0.9.2"; -$apis{virDomainMigratePerform3}->{vers} =3D "0.9.2"; -$apis{virDomainMigrateFinish3}->{vers} =3D "0.9.2"; -$apis{virDomainMigrateConfirm3}->{vers} =3D "0.9.2"; - -$apis{virDomainMigrateBegin3Params}->{vers} =3D "1.1.0"; -$apis{virDomainMigratePrepare3Params}->{vers} =3D "1.1.0"; -$apis{virDomainMigratePrepareTunnel3Params}->{vers} =3D "1.1.0"; -$apis{virDomainMigratePerform3Params}->{vers} =3D "1.1.0"; -$apis{virDomainMigrateFinish3Params}->{vers} =3D "1.1.0"; -$apis{virDomainMigrateConfirm3Params}->{vers} =3D "1.1.0"; - - - -# Now we want to get the mapping between public APIs -# and driver struct fields. This lets us later match -# update the driver impls with the public APis. - -my $line; - -# Group name -> hash of APIs { fields -> api name } -my %groups; -my $ingrp; -foreach my $drivertable (@drivertable) { - open FILE, "<$drivertable" - or die "cannot read $drivertable: $!"; - - while (defined($line =3D )) { - if ($line =3D~ /struct _(vir\w*Driver)/) { - my $grp =3D $1; - if ($grp ne "virStateDriver" && - $grp ne "virStreamDriver") { - $ingrp =3D $grp; - $groups{$ingrp} =3D { apis =3D> {}, drivers =3D> {} }; - } - } elsif ($ingrp) { - if ($line =3D~ /^\s*vir(?:Drv)(\w+)\s+(\w+);\s*$/) { - my $field =3D $2; - my $name =3D $1; - - my $api; - if (exists $apis{"vir$name"}) { - $api =3D "vir$name"; - } elsif ($name =3D~ /\w+(Open|Close|URIProbe)/) { - next; - } else { - die "driver $name does not have a public API"; - } - $groups{$ingrp}->{apis}->{$field} =3D $api; - } elsif ($line =3D~ /};/) { - $ingrp =3D undef; - } - } - } - - close FILE; -} - - -# Finally, we read all the primary driver files and extract -# the driver API tables from each one. - -foreach my $src (@srcs) { - open FILE, "<$src" or - die "cannot read $src: $!"; - - my $groups_regex =3D join("|", keys %groups); - $ingrp =3D undef; - my $impl; - while (defined($line =3D )) { - if (!$ingrp) { - #=C2=A0skip non-matching lines early to save time - next if not $line =3D~ /$groups_regex/; - - if ($line =3D~ /^\s*(?:static\s+)?($groups_regex)\s+(\w+)\s*= =3D\s*{/ || - $line =3D~ /^\s*(?:static\s+)?($groups_regex)\s+NAME\(\w+\= )\s*=3D\s*{/) { - $ingrp =3D $1; - $impl =3D $src; - - if ($impl =3D~ m,.*/node_device_(\w+)\.c,) { - $impl =3D $1; - } else { - $impl =3D~ s,.*/(\w+?)_((\w+)_)?(\w+)\.c,$1,; - } - - if ($groups{$ingrp}->{drivers}->{$impl}) { - die "Group $ingrp already contains $impl"; - } - - $groups{$ingrp}->{drivers}->{$impl} =3D {}; - } - - } else { - if ($line =3D~ m!\s*\.(\w+)\s*=3D\s*(\w+)\s*,?\s*(?:/\*\s*(\d+= \.\d+\.\d+)\s*(?:-\s*(\d+\.\d+\.\d+))?\s*\*/\s*)?$!) { - my $api =3D $1; - my $meth =3D $2; - my $vers =3D $3; - my $deleted =3D $4; - - next if $api eq "no" || $api eq "name"; - - if ($meth eq "NULL" && !defined $deleted) { - die "Method impl for $api is NULL, but no deleted vers= ion is provided"; - } - if ($meth ne "NULL" && defined $deleted) { - die "Method impl for $api is non-NULL, but deleted ver= sion is provided"; - } - - die "Method $meth in $src is missing version" unless defin= ed $vers || $api eq "connectURIProbe"; - - if (!exists($groups{$ingrp}->{apis}->{$api})) { - next if $api =3D~ /\w(Open|Close|URIProbe)/; - - die "Found unexpected method $api in $ingrp\n"; - } - - $groups{$ingrp}->{drivers}->{$impl}->{$api} =3D {}; - $groups{$ingrp}->{drivers}->{$impl}->{$api}->{vers} =3D $v= ers; - $groups{$ingrp}->{drivers}->{$impl}->{$api}->{deleted} = =3D $deleted; - if ($api eq "domainMigratePrepare" || - $api eq "domainMigratePrepare2" || - $api eq "domainMigratePrepare3") { - if (!$groups{$ingrp}->{drivers}->{$impl}->{"domainMigr= ate"}) { - $groups{$ingrp}->{drivers}->{$impl}->{"domainMigra= te"} =3D {}; - $groups{$ingrp}->{drivers}->{$impl}->{"domainMigra= te"}->{vers} =3D $vers; - } - } - - } elsif ($line =3D~ /}/) { - $ingrp =3D undef; - } - } - } - - close FILE; -} - - -# The '.open' driver method is used for 3 public APIs, so we -# have a bit of manual fixup todo with the per-driver versioning -# and support matrix - -$groups{virHypervisorDriver}->{apis}->{"openAuth"} =3D "virConnectOpenAuth= "; -$groups{virHypervisorDriver}->{apis}->{"openReadOnly"} =3D "virConnectOpen= ReadOnly"; -$groups{virHypervisorDriver}->{apis}->{"domainMigrate"} =3D "virDomainMigr= ate"; - -my $openAuthVers =3D (0 * 1000 * 1000) + (4 * 1000) + 0; - -foreach my $drv (keys %{$groups{"virHypervisorDriver"}->{drivers}}) { - my $openVersStr =3D $groups{"virHypervisorDriver"}->{drivers}->{$drv}-= >{"connectOpen"}->{vers}; - my $openVers; - if ($openVersStr =3D~ /(\d+)\.(\d+)\.(\d+)/) { - $openVers =3D ($1 * 1000 * 1000) + ($2 * 1000) + $3; - } - - # virConnectOpenReadOnly always matches virConnectOpen version - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"connectOpenReadOn= ly"} =3D - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"connectOpen"}; - - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"connectOpenAuth"}= =3D {}; - - # virConnectOpenAuth is always 0.4.0 if the driver existed - # before this time, otherwise it matches the version of - # the driver's virConnectOpen entry - if ($openVersStr eq "Y" || - $openVers >=3D $openAuthVers) { - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"connectOpenAu= th"}->{vers} =3D $openVersStr; - } else { - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"connectOpenAu= th"}->{vers} =3D "0.4.0"; - } -} - - -# Another special case for the virDomainCreateLinux which was replaced -# with virDomainCreateXML -$groups{virHypervisorDriver}->{apis}->{"domainCreateLinux"} =3D "virDomain= CreateLinux"; - -my $createAPIVers =3D (0 * 1000 * 1000) + (0 * 1000) + 3; - -foreach my $drv (keys %{$groups{"virHypervisorDriver"}->{drivers}}) { - my $createVersStr =3D $groups{"virHypervisorDriver"}->{drivers}->{$drv= }->{"domainCreateXML"}->{vers}; - next unless defined $createVersStr; - my $createVers; - if ($createVersStr =3D~ /(\d+)\.(\d+)\.(\d+)/) { - $createVers =3D ($1 * 1000 * 1000) + ($2 * 1000) + $3; - } - - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"domainCreateLinux= "} =3D {}; - - # virCreateLinux is always 0.0.3 if the driver existed - # before this time, otherwise it matches the version of - # the driver's virCreateXML entry - if ($createVersStr eq "Y" || - $createVers >=3D $createAPIVers) { - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"domainCreateL= inux"}->{vers} =3D $createVersStr; - } else { - $groups{"virHypervisorDriver"}->{drivers}->{$drv}->{"domainCreateL= inux"}->{vers} =3D "0.0.3"; - } -} - - -# Finally we generate the HTML file with the tables - -print < - - - -

libvirt API support matrix

- -
    - -

    -This page documents which libvirt calls work on -which libvirt drivers / hypervisors, and which version the API appeared -in. If a hypervisor driver later dropped support for the API, the version -when it was removed is also mentioned (highlighted in -dark red). -

    - -EOF - - foreach my $grp (sort { $a cmp $b } keys %groups) { - print "

    ", $groupheaders{$grp}, "

    \n"; - print < - - -API -Version -EOF - - foreach my $drv (sort { $a cmp $b } keys %{$groups{$grp}->{drivers}}) { - print " $drv\n"; - } - - print < - - -EOF - - my $row =3D 0; - foreach my $field (sort { - $groups{$grp}->{apis}->{$a} - cmp - $groups{$grp}->{apis}->{$b} - } keys %{$groups{$grp}->{apis}}) { - my $api =3D $groups{$grp}->{apis}->{$field}; - my $vers =3D $apis{$api}->{vers}; - my $htmlgrp =3D $apis{$api}->{file}; - print < - -EOF - - if (defined $htmlgrp) { - print <$api -EOF - - } else { - print $api; - } - print < -$vers -EOF - - foreach my $drv (sort {$a cmp $b } keys %{$groups{$grp}->{drivers}= }) { - print ""; - if (exists $groups{$grp}->{drivers}->{$drv}->{$field}) { - if ($groups{$grp}->{drivers}->{$drv}->{$field}->{vers}) { - print $groups{$grp}->{drivers}->{$drv}->{$field}->{ver= s}; - } - if ($groups{$grp}->{drivers}->{$drv}->{$field}->{deleted})= { - print " - ", $groups{$grp}= ->{drivers}->{$drv}->{$field}->{deleted}, ""; - } - } - print "\n"; - } - - print < -EOF - - $row++; - if (($row % 15) =3D=3D 0) { - print < -API -Version -EOF - - foreach my $drv (sort { $a cmp $b } keys %{$groups{$grp}->{dri= vers}}) { - print " $drv\n"; - } - - print < -EOF - } - - } - - print < - -EOF -} - -print < - -EOF diff --git a/scripts/hvsupport.py b/scripts/hvsupport.py new file mode 100755 index 0000000000..7900f86f0f --- /dev/null +++ b/scripts/hvsupport.py @@ -0,0 +1,504 @@ +#!/usr/bin/env python +# +# Copyright (C) 2011-2019 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . + +from __future__ import print_function + +import sys +import os.path +import re + +if len(sys.argv) !=3D 3: + print("syntax: %s TOP-SRCDIR TOP-BUILDDIR\n" % sys.argv[0], file=3Dsys= .stderr) + +srcdir =3D sys.argv[1] +builddir =3D sys.argv[2] + +symslibvirt =3D os.path.join(srcdir, "src", "libvirt_public.syms") +symsqemu =3D os.path.join(srcdir, "src", "libvirt_qemu.syms") +symslxc =3D os.path.join(srcdir, "src", "libvirt_lxc.syms") +drivertablefiles =3D [ + os.path.join(srcdir, "src", "driver-hypervisor.h"), + os.path.join(srcdir, "src", "driver-interface.h"), + os.path.join(srcdir, "src", "driver-network.h"), + os.path.join(srcdir, "src", "driver-nodedev.h"), + os.path.join(srcdir, "src", "driver-nwfilter.h"), + os.path.join(srcdir, "src", "driver-secret.h"), + os.path.join(srcdir, "src", "driver-state.h"), + os.path.join(srcdir, "src", "driver-storage.h"), + os.path.join(srcdir, "src", "driver-stream.h"), +] + +groupheaders =3D { + "virHypervisorDriver": "Hypervisor APIs", + "virNetworkDriver": "Virtual Network APIs", + "virInterfaceDriver": "Host Interface APIs", + "virNodeDeviceDriver": "Host Device APIs", + "virStorageDriver": "Storage Pool APIs", + "virSecretDriver": "Secret APIs", + "virNWFilterDriver": "Network Filter APIs", +} + + +srcs =3D [] +for root, dirs, files in os.walk(os.path.join(srcdir, "src")): + for file in files: + if ((file.endswith("driver.c") and + not file.endswith("vbox_driver.c")) or + file.endswith("common.c") or + file.endswith("tmpl.c") or + file.endswith("monitor.c") or + file.endswith("hal.c") or + file.endswith("udev.c")): + srcs.append(os.path.join(root, file)) + + +# Map API functions to the header and documentation files they're in +# so that we can generate proper hyperlinks to their documentation. +# +# The function names are grep'd from the XML output of apibuild.py. +def getAPIFilenames(filename): + files =3D {} + + with open(filename) as fh: + for line in fh: + res =3D re.search(r"function name=3D'([^']+)' file=3D'([^']+)'= ", line) + if res is not None: + files[res.group(1)] =3D res.group(2) + + if len(files) =3D=3D 0: + raise Exception(("No functions found in %s. " + + "Has the apibuild.py output changed?") % + filename) + + return files + + +def parseSymsFile(apisref, prefix, filename, xmlfilename): + vers =3D None + prevvers =3D None + + filenames =3D getAPIFilenames(xmlfilename) + + with open(filename) as fh: + for line in fh: + line =3D line.strip() + + if line =3D=3D "": + continue + if line[0] =3D=3D '#': + continue + if line.startswith("global:"): + continue + if line.startswith("local:"): + continue + + groupstartmatch =3D re.search(r"^\s*%s_(\d+\.\d+\.\d+)\s*{\s*$= " % + prefix, line) + groupendmatch1 =3D re.search(r"^\s*}\s*;\s*$", line) + groupendmatch2 =3D re.search(r"^\s*}\s*%s_(\d+\.\d+\.\d+)\s*;\= s*$" % + prefix, line) + symbolmatch =3D re.search(r"^\s*(\w+)\s*;\s*$", line) + if groupstartmatch is not None: + if vers is not None: + raise Exception("malformed syms file when starting gro= up") + + vers =3D groupstartmatch.group(1) + elif groupendmatch1 is not None: + if prevvers is not None: + raise Exception("malformed syms file when ending group= ") + + prevvers =3D vers + vers =3D None + elif groupendmatch2 is not None: + if groupendmatch2.group(1) !=3D prevvers: + raise Exception("malformed syms file %s !=3D %s " + + "when ending group" % + (groupendmatch2.group(1), prevvers)) + + prevvers =3D vers + vers =3D None + elif symbolmatch is not None: + name =3D symbolmatch.group(1) + apisref[name] =3D { + "vers": vers, + "file": filenames.get(name), + } + else: + raise Exception("unexpected data %s" % line) + + +apis =3D {} +# Get the list of all public APIs and their corresponding version +parseSymsFile(apis, "LIBVIRT", symslibvirt, + os.path.join(builddir, "docs", "libvirt-api.xml")) + +# And the same for the QEMU specific APIs +parseSymsFile(apis, "LIBVIRT_QEMU", symsqemu, + os.path.join(builddir, "docs", "libvirt-qemu-api.xml")) + +# And the same for the LXC specific APIs +parseSymsFile(apis, "LIBVIRT_LXC", symslxc, + os.path.join(builddir, "docs", "libvirt-lxc-api.xml")) + + +# Some special things which aren't public APIs, +# but we want to report +apis["virConnectSupportsFeature"] =3D { + "vers": "0.3.2" +} +apis["virDomainMigratePrepare"] =3D { + "vers": "0.3.2" +} +apis["virDomainMigratePerform"] =3D { + "vers": "0.3.2" +} +apis["virDomainMigrateFinish"] =3D { + "vers": "0.3.2" +} +apis["virDomainMigratePrepare2"] =3D { + "vers": "0.5.0" +} +apis["virDomainMigrateFinish2"] =3D { + "vers": "0.5.0" +} +apis["virDomainMigratePrepareTunnel"] =3D { + "vers": "0.7.2" +} + +apis["virDomainMigrateBegin3"] =3D { + "vers": "0.9.2" +} +apis["virDomainMigratePrepare3"] =3D { + "vers": "0.9.2" +} +apis["virDomainMigratePrepareTunnel3"] =3D { + "vers": "0.9.2" +} +apis["virDomainMigratePerform3"] =3D { + "vers": "0.9.2" +} +apis["virDomainMigrateFinish3"] =3D { + "vers": "0.9.2" +} +apis["virDomainMigrateConfirm3"] =3D { + "vers": "0.9.2" +} + +apis["virDomainMigrateBegin3Params"] =3D { + "vers": "1.1.0" +} +apis["virDomainMigratePrepare3Params"] =3D { + "vers": "1.1.0" +} +apis["virDomainMigratePrepareTunnel3Params"] =3D { + "vers": "1.1.0" +} +apis["virDomainMigratePerform3Params"] =3D { + "vers": "1.1.0" +} +apis["virDomainMigrateFinish3Params"] =3D { + "vers": "1.1.0" +} +apis["virDomainMigrateConfirm3Params"] =3D { + "vers": "1.1.0" +} + + +# Now we want to get the mapping between public APIs +# and driver struct fields. This lets us later match +# update the driver impls with the public APis. + +# Group name -> hash of APIs { fields -> api name } +groups =3D {} +ingrp =3D None +for drivertablefile in drivertablefiles: + with open(drivertablefile) as fh: + for line in fh: + starttablematch =3D re.search(r"struct _(vir\w*Driver)", line) + if starttablematch is not None: + grp =3D starttablematch.group(1) + if grp !=3D "virStateDriver" and grp !=3D "virStreamDriver= ": + ingrp =3D grp + groups[ingrp] =3D { + "apis": {}, + "drivers": {} + } + elif ingrp is not None: + callbackmatch =3D re.search(r"^\s*vir(?:Drv)(\w+)\s+(\w+);= \s*$", + line) + if callbackmatch is not None: + name =3D callbackmatch.group(1) + field =3D callbackmatch.group(2) + + api =3D "vir" + name + if api in apis: + groups[ingrp]["apis"][field] =3D api + elif re.search(r"\w+(Open|Close|URIProbe)", api) is no= t None: + continue + else: + raise Exception(("driver %s does not have " + + "a public API") % name) + elif re.search(r"};", line): + ingrp =3D None + + +# Finally, we read all the primary driver files and extract +# the driver API tables from each one. + +for src in srcs: + with open(src) as fh: + groupsre =3D "|".join(groups.keys()) + + ingrp =3D None + impl =3D None + for line in fh: + if ingrp is None: + m =3D re.search(r"^\s*(static\s+)?(" + + groupsre + + r")\s+(\w+)\s*=3D\s*{", line) + if m is None: + m =3D re.search(r"^\s*(static\s+)?(" + + groupsre + + r")\s+NAME\(\w+\)\s*=3D\s*{", line) + if m is not None: + ingrp =3D m.group(2) + impl =3D src + + implmatch =3D re.search(r".*/node_device_(\w+)\.c", im= pl) + if implmatch is None: + implmatch =3D re.search(r".*/(\w+?)_((\w+)_)?(\w+)= \.c", impl) + if implmatch is None: + raise Exception("Unexpected impl format '%s'" % im= pl) + impl =3D implmatch.group(1) + + if impl in groups[ingrp]["drivers"]: + raise Exception( + "Group %s already contains %s" % (ingrp, impl)) + + groups[ingrp]["drivers"][impl] =3D {} + else: + callbackmatch =3D re.search(r"\s*\.(\w+)\s*=3D\s*(\w+)\s*,= ?\s*" + + r"(?:/\*\s*(\d+\.\d+\.\d+)\s*" + + r"(?:-\s*(\d+\.\d+\.\d+))?\s*\*/= \s*)?$", + line) + if callbackmatch is not None: + api =3D callbackmatch.group(1) + meth =3D callbackmatch.group(2) + vers =3D callbackmatch.group(3) + deleted =3D callbackmatch.group(4) + + if api =3D=3D "no" or api =3D=3D "name": + continue + + if meth =3D=3D "NULL" and deleted is None: + raise Exception( + ("Method impl for %s is NULL, but " + + "no deleted version is provided") % api) + + if meth !=3D "NULL" and deleted is not None: + raise Exception( + ("Method impl for %s is non-NULL, but " + + "deleted version is provided") % api) + + if vers is None and api !=3D "connectURIProbe": + raise Exception( + "Method %s in %s is missing version" % + (meth, src)) + + if api not in groups[ingrp]["apis"]: + if re.search(r"\w+(Open|Close|URIProbe)", api): + continue + + raise Exception("Found unexpected method " + + "%s in %s" % (api, ingrp)) + + groups[ingrp]["drivers"][impl][api] =3D { + "vers": vers, + "deleted": deleted, + } + + if (api =3D=3D "domainMigratePrepare" or + api =3D=3D "domainMigratePrepare2" or + api =3D=3D "domainMigratePrepare3"): + if ("domainMigrate" not in + groups[ingrp]["drivers"][impl]): + groups[ingrp]["drivers"][impl]["domainMigrate"= ] =3D { + "vers": vers, + } + elif line.find("}") !=3D -1: + ingrp =3D None + + +# The '.open' driver method is used for 3 public APIs, so we +# have a bit of manual fixup todo with the per-driver versioning +# and support matrix + +groups["virHypervisorDriver"]["apis"]["openAuth"] =3D \ + "virConnectOpenAuth" +groups["virHypervisorDriver"]["apis"]["openReadOnly"] =3D \ + "virConnectOpenReadOnly" +groups["virHypervisorDriver"]["apis"]["domainMigrate"] =3D \ + "virDomainMigrate" + +openAuthVers =3D (0 * 1000 * 1000) + (4 * 1000) + 0 + +drivers =3D groups["virHypervisorDriver"]["drivers"] +for drv in drivers.keys(): + openVersStr =3D drivers[drv]["connectOpen"]["vers"] + openVers =3D 0 + if openVersStr !=3D "Y": + openVersBits =3D openVersStr.split(".") + if len(openVersBits) !=3D 3: + raise Exception("Expected 3 digit version for %s" % openVersSt= r) + openVers =3D ((int(openVersBits[0]) * 1000 * 1000) + + (int(openVersBits[1]) * 1000) + + int(openVersBits[2])) + + # virConnectOpenReadOnly always matches virConnectOpen version + drivers[drv]["connectOpenReadOnly"] =3D \ + drivers[drv]["connectOpen"] + + # virConnectOpenAuth is always 0.4.0 if the driver existed + # before this time, otherwise it matches the version of + # the driver's virConnectOpen entry + if openVersStr =3D=3D "Y" or openVers >=3D openAuthVers: + vers =3D openVersStr + else: + vers =3D "0.4.0" + drivers[drv]["connectOpenAuth"] =3D { + "vers": vers, + } + + +# Another special case for the virDomainCreateLinux which was replaced +# with virDomainCreateXML +groups["virHypervisorDriver"]["apis"]["domainCreateLinux"] =3D \ + "virDomainCreateLinux" + +createAPIVers =3D (0 * 1000 * 1000) + (0 * 1000) + 3 + +for drv in drivers.keys(): + if "domainCreateXML" not in drivers[drv]: + continue + createVersStr =3D drivers[drv]["domainCreateXML"]["vers"] + createVers =3D 0 + if createVersStr !=3D "Y": + createVersBits =3D createVersStr.split(".") + if len(createVersBits) !=3D 3: + raise Exception("Expected 3 digit version for %s" % createVers= Str) + createVers =3D ((int(createVersBits[0]) * 1000 * 1000) + + (int(createVersBits[1]) * 1000) + + int(createVersBits[2])) + + # virCreateLinux is always 0.0.3 if the driver existed + # before this time, otherwise it matches the version of + # the driver's virCreateXML entry + if createVersStr =3D=3D "Y" or createVers >=3D createAPIVers: + vers =3D createVersStr + else: + vers =3D "0.0.3" + + drivers[drv]["domainCreateLinux"] =3D { + "vers": vers, + } + + +# Finally we generate the HTML file with the tables + +print(''' + + + +

    libvirt API support matrix

    + +
      + +

      +This page documents which libvirt calls work on +which libvirt drivers / hypervisors, and which version the API appeared +in. If a hypervisor driver later dropped support for the API, the version +when it was removed is also mentioned (highlighted in +dark red). +

      +''') + +for grp in sorted(groups.keys()): + print("

      %s

      " % (grp, groupheaders[grp])) + print(''' + + + +''') + + for drv in sorted(groups[grp]["drivers"].keys()): + print(" " % drv) + + print(''' + +''') + + row =3D 0 + + def sortkey(field): + return groups[grp]["apis"][field] + + for field in sorted(groups[grp]["apis"].keys(), key=3Dsortkey): + api =3D groups[grp]["apis"][field] + vers =3D apis[api]["vers"] + htmlgrp =3D apis[api].get("file") + print("") + + if htmlgrp is not None: + print(('''''') % + (htmlgrp, api, api)) + else: + print("" % api) + + print("" % vers) + + for drv in sorted(groups[grp]["drivers"].keys()): + info =3D "" + if field in groups[grp]["drivers"][drv]: + vers =3D groups[grp]["drivers"][drv][field]["vers"] + if vers is not None: + info =3D info + vers + + deleted =3D groups[grp]["drivers"][drv][field].get("delete= d") + if deleted is not None: + info =3D info + (''' - ''' + + '''%s''' % deleted) + + print("" % info) + + print("") + + row =3D row + 1 + if (row % 15) =3D=3D 0: + print(''' + +''') + + for drv in sorted(groups[grp]["drivers"].keys()): + print(" " % drv) + + print("") + + print("\n
      APIVersion%s
      \n''' + + '''%s\n\n%s%s%s
      APIVersion%s
      ") + +print("\n") --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sun May 5 18:51:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 205.139.110.120 as permitted sender) client-ip=205.139.110.120; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1573483581; cv=none; d=zoho.com; s=zohoarc; b=U60HgCbJNz5lR0UITIZ1wEmEMN4sXIJdomkooHqdwO/6bmAhzihlSr/TBdof7oRRGDcuay2Dwp9dfq1tyxRoHGNwJfEj5jl8yQPv9efh8mWe//SqbBrQmTMX5bZAYZlPzS23shXpJLUU5SeM25pLtI3KAuId+LCalo4dYrUA+xI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573483581; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=w2jxsF6sba4PpwiMfsxWx7qKv4fPadKsbbwxtOGig8Q=; b=onhCa5cBFx7Z0Tn3DyqpFGMtpTytkgZPckUKbrXR2KDbL1UqCiVh00WfI85vNual6WRIQJm00atMXDvyLgpUMGmflGupZmbISsd2mfgsLOgkXTWfzGAI4z8csqCR1IQhtxgMhsMXcoMNMFFRrpWF1Bt7/08rwTPxnUoIxqiQAl8= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 205.139.110.120 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-1.mimecast.com (us-smtp-delivery-1.mimecast.com [205.139.110.120]) by mx.zohomail.com with SMTPS id 1573483581233745.8285359737957; Mon, 11 Nov 2019 06:46:21 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-304-zeZRR-qPN_KhfesspdY4qA-1; Mon, 11 Nov 2019 09:45:27 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BAD9A8A2431; Mon, 11 Nov 2019 14:45:22 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 8C95F4B4; Mon, 11 Nov 2019 14:45:22 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 39CF4180221B; Mon, 11 Nov 2019 14:45:22 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xABEjCUg020566 for ; Mon, 11 Nov 2019 09:45:12 -0500 Received: by smtp.corp.redhat.com (Postfix) id 4D1085DD73; Mon, 11 Nov 2019 14:45:12 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id AFD6F5E240; Mon, 11 Nov 2019 14:45:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573483579; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=w2jxsF6sba4PpwiMfsxWx7qKv4fPadKsbbwxtOGig8Q=; b=hEb/BpQKNqWaouUUXQzN7T7021KoN/2vZ067RGVoN6Fp5ktH6tgM//tW/Wmkw+8Ad7D0br tqnFHAc002bIlK+SfcMYqiTNJQzuyzkE7CGLJbivni7vHfNNyeFkonSaxMYqBzdtVVjrKS EPtGZy4WE2t/toaJC7OaOAaGirDZTbw= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 11 Nov 2019 14:38:22 +0000 Message-Id: <20191111143826.16050-20-berrange@redhat.com> In-Reply-To: <20191111143826.16050-1-berrange@redhat.com> References: <20191111143826.16050-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v5 19/23] docs: rewrite polkit docs generator in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-MC-Unique: zeZRR-qPN_KhfesspdY4qA-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" As part of an goal to eliminate Perl from libvirt build tools, rewrite the genaclperms.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. Signed-off-by: Daniel P. Berrang=C3=A9 Tested-by: Cole Robinson --- Makefile.am | 1 + docs/Makefile.am | 6 +- docs/genaclperms.pl | 125 ----------------------------------------- scripts/genaclperms.py | 123 ++++++++++++++++++++++++++++++++++++++++ 4 files changed, 127 insertions(+), 128 deletions(-) delete mode 100755 docs/genaclperms.pl create mode 100755 scripts/genaclperms.py diff --git a/Makefile.am b/Makefile.am index bc8aecc4a2..a14474f535 100644 --- a/Makefile.am +++ b/Makefile.am @@ -56,6 +56,7 @@ EXTRA_DIST =3D \ scripts/check-symfile.py \ scripts/check-symsorting.py \ scripts/dtrace2systemtap.py \ + scripts/genaclperms.py \ scripts/genpolkit.py \ scripts/gensystemtap.py \ scripts/group-qemu-caps.py \ diff --git a/docs/Makefile.am b/docs/Makefile.am index cf83949168..ca7a2ec260 100644 --- a/docs/Makefile.am +++ b/docs/Makefile.am @@ -163,7 +163,7 @@ schemadir =3D $(pkgdatadir)/schemas schema_DATA =3D $(wildcard $(srcdir)/schemas/*.rng) =20 EXTRA_DIST=3D \ - apibuild.py genaclperms.pl \ + apibuild.py \ site.xsl subsite.xsl newapi.xsl page.xsl \ wrapstring.xsl \ $(dot_html_in) $(gif) $(apipng) \ @@ -177,8 +177,8 @@ EXTRA_DIST=3D \ acl_generated =3D aclperms.htmlinc =20 aclperms.htmlinc: $(top_srcdir)/src/access/viraccessperm.h \ - $(srcdir)/genaclperms.pl Makefile.am - $(AM_V_GEN)$(PERL) $(srcdir)/genaclperms.pl $< > $@ + $(top_srcdir)/scripts/genaclperms.py Makefile.am + $(AM_V_GEN)$(RUNUTF8) $(PYTHON) $(top_srcdir)/scripts/genaclperms.py $< >= $@ =20 CLEANFILES =3D \ $(dot_html) \ diff --git a/docs/genaclperms.pl b/docs/genaclperms.pl deleted file mode 100755 index e20b4c11c3..0000000000 --- a/docs/genaclperms.pl +++ /dev/null @@ -1,125 +0,0 @@ -#!/usr/bin/env perl -# -# Copyright (C) 2013 Red Hat, Inc. -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library. If not, see -# . -# - -use strict; -use warnings; - -my @objects =3D ( - "CONNECT", "DOMAIN", "INTERFACE", - "NETWORK_PORT", "NETWORK", "NODE_DEVICE", - "NWFILTER_BINDING", "NWFILTER", - "SECRET", "STORAGE_POOL", "STORAGE_VOL", - ); - -my %class; - -foreach my $object (@objects) { - my $class =3D lc $object; - - $class =3D~ s/(^\w|_\w)/uc $1/eg; - $class =3D~ s/_//g; - $class =3D~ s/Nwfilter/NWFilter/; - $class =3D "vir" . $class . "Ptr"; - - $class{$object} =3D $class; -} - -my $objects =3D join ("|", @objects); - -my %opts; -my $in_opts =3D 0; - -my %perms; - -while (<>) { - if ($in_opts) { - if (m,\*/,) { - $in_opts =3D 0; - } elsif (/\*\s*\@(\w+):\s*(.*?)\s*$/) { - $opts{$1} =3D $2; - } - } elsif (m,/\*\*,) { - $in_opts =3D 1; - } elsif (/VIR_ACCESS_PERM_($objects)_((?:\w|_)+),/) { - my $object =3D $1; - my $perm =3D lc $2; - next if $perm eq "last"; - - $perm =3D~ s/_/-/g; - - $perms{$object} =3D {} unless exists $perms{$object}; - $perms{$object}->{$perm} =3D { - desc =3D> $opts{desc}, - message =3D> $opts{message}, - anonymous =3D> $opts{anonymous} - }; - %opts =3D (); - } -} - -print < - - - -EOF - -foreach my $object (sort { $a cmp $b } keys %perms) { - my $class =3D $class{$object}; - my $olink =3D lc "object_" . $object; - print <$class - - - - - - - - -EOF - - foreach my $perm (sort { $a cmp $b } keys %{$perms{$object}}) { - my $description =3D $perms{$object}->{$perm}->{desc}; - - die "missing description for $object.$perm" unless - defined $description; - - my $plink =3D lc "perm_" . $object . "_" . $perm; - $plink =3D~ s/-/_/g; - - print < - - - -EOF - - } - - print < -
      PermissionDescription
      $perm$description
      -EOF -} - -print < - -EOF diff --git a/scripts/genaclperms.py b/scripts/genaclperms.py new file mode 100755 index 0000000000..cd92346449 --- /dev/null +++ b/scripts/genaclperms.py @@ -0,0 +1,123 @@ +#!/usr/bin/env python +# +# Copyright (C) 2013-2019 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . +# + +from __future__ import print_function + +import re +import sys + +objects =3D [ + "CONNECT", "DOMAIN", "INTERFACE", + "NETWORK_PORT", "NETWORK", "NODE_DEVICE", + "NWFILTER_BINDING", "NWFILTER", + "SECRET", "STORAGE_POOL", "STORAGE_VOL", +] + + +classes =3D {} + +for obj in objects: + klass =3D obj.lower() + + klass =3D re.sub(r'''(^\w|_\w)''', lambda a: a.group(1).upper(), klass) + klass =3D klass.replace("_", "") + klass =3D klass.replace("Nwfilter", "NWFilter") + klass =3D "vir" + klass + "Ptr" + + classes[obj] =3D klass + + +objectstr =3D "|".join(objects) + +opts =3D {} +in_opts =3D {} + +perms =3D {} + +aclfile =3D sys.argv[1] +with open(aclfile, "r") as fh: + for line in fh: + if in_opts: + if line.find("*/") !=3D -1: + in_opts =3D False + else: + m =3D re.search(r'''\*\s*\@(\w+):\s*(.*?)\s*$''', line) + if m is not None: + opts[m.group(1)] =3D m.group(2) + elif line.find("**") !=3D -1: + in_opts =3D True + else: + m =3D re.search(r'''VIR_ACCESS_PERM_(%s)_((?:\w|_)+),''' % + objectstr, line) + if m is not None: + obj =3D m.group(1) + perm =3D m.group(2).lower() + if perm =3D=3D "last": + continue + + perm =3D perm.replace("_", "-") + + if obj not in perms: + perms[obj] =3D {} + perms[obj][perm] =3D { + "desc": opts.get("desc", None), + "message": opts.get("message", None), + "anonymous": opts.get("anonymous", None), + } + opts =3D {} + +print('') +print('') +print('') +print(' ') + +for obj in sorted(perms.keys()): + klass =3D classes[obj] + + olink =3D "object_" + obj.lower() + + print('

      %s

      ' % (olink, klass)) + print(' ') + print(' ') + print(' ') + print(' ') + print(' ') + print(' ') + print(' ') + print(' ') + + for perm in sorted(perms[obj].keys()): + description =3D perms[obj][perm]["desc"] + + if description is None: + raise Exception("missing description for %s.%s" % (obj, perm)) + + plink =3D "perm_" + obj.lower() + "_" + perm.lower() + plink =3D plink.replace("-", "_") + + print(' ') + print(' ' % (plink, perm)) + print(' ' % description) + print(' ') + + print(' ') + print('
      PermissionDescription
      %s%s
      ') + +print(' ') +print('') --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sun May 5 18:51:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 205.139.110.61 as permitted sender) client-ip=205.139.110.61; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1573483563; cv=none; d=zoho.com; s=zohoarc; b=j8mKgtnFfGA56yU4MxqF4zHyx6aufdQwBFGpXFbwukjjO8YHEJoWEyKv4j6x/3Dyzmoi8dwIywHz7tGY4InW8NKswUdDi3l5sSB1Gte/CBrlYtUdhLqhdw/VxyQYqL46uQwZXyT5m/kKfxqJrFQ8cGNqIV/flDGgZ+4tz3nHNTk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573483563; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ZWx5aXw5AnJwep5tInRl0bmNyqVW28YTSHpgBY7+AYc=; b=LNnq59vxyNxjTuN32hzxI6ucGGOl6fskdollPwyQTwKy9sv/1f9msoqWE8Vl1PbXGhU0UJhRtfxgIK0/BUJIlP1UkKqvSPnOhDyik0PTLeWjLE9Nk5jPErdhIh4UOceHqszOFOMz04QjY25mD5pYBg/2h04nWS9fiZFfVnntclE= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) by mx.zohomail.com with SMTPS id 15734835631991018.1480630558252; Mon, 11 Nov 2019 06:46:03 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-422-SV4B82B_PhSkbA-lq_9CnA-1; Mon, 11 Nov 2019 09:45:32 -0500 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BC0A5DBE7; Mon, 11 Nov 2019 14:45:26 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 92F9E63147; Mon, 11 Nov 2019 14:45:26 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 2E7D31802158; Mon, 11 Nov 2019 14:45:26 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xABEjD3d020579 for ; Mon, 11 Nov 2019 09:45:13 -0500 Received: by smtp.corp.redhat.com (Postfix) id 2D1C15DEC0; Mon, 11 Nov 2019 14:45:13 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 907295DA7D; Mon, 11 Nov 2019 14:45:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573483562; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=ZWx5aXw5AnJwep5tInRl0bmNyqVW28YTSHpgBY7+AYc=; b=EwDtyz+cgXYpVKT+dXuidYTckPZenyFZyQ9Kq7ROR0UAucxoLi6hEhUVuKn6oeClkAycBx aObzgCutOKCjOSnfugjbk2weQW081nXk9KVygUsw4PbBozqN8Ce/4Wi6Pru5aveyZCl7OA 1WnEbfjWyZKgWTdi7gzcIspUqQuFYyU= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 11 Nov 2019 14:38:23 +0000 Message-Id: <20191111143826.16050-21-berrange@redhat.com> In-Reply-To: <20191111143826.16050-1-berrange@redhat.com> References: <20191111143826.16050-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v5 20/23] docs: move apibuild.py to the scripts/ directory X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-MC-Unique: SV4B82B_PhSkbA-lq_9CnA-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Signed-off-by: Daniel P. Berrang=C3=A9 --- Makefile.am | 1 + build-aux/syntax-check.mk | 2 +- docs/Makefile.am | 5 ++--- {docs =3D> scripts}/apibuild.py | 0 4 files changed, 4 insertions(+), 4 deletions(-) rename {docs =3D> scripts}/apibuild.py (100%) diff --git a/Makefile.am b/Makefile.am index a14474f535..67d07eff41 100644 --- a/Makefile.am +++ b/Makefile.am @@ -45,6 +45,7 @@ EXTRA_DIST =3D \ run.in \ README.md \ AUTHORS.in \ + scripts/apibuild.py \ scripts/augeas-gentest.py \ scripts/check-aclperms.py \ scripts/check-aclrules.py \ diff --git a/build-aux/syntax-check.mk b/build-aux/syntax-check.mk index 44639f499e..ee8a496efa 100644 --- a/build-aux/syntax-check.mk +++ b/build-aux/syntax-check.mk @@ -2284,7 +2284,7 @@ exclude_file_name_regexp--sc_trailing_blank =3D \ /sysinfodata/.*\.data|/virhostcpudata/.*\.cpuinfo|^gnulib/local/.*/.*dif= f$$ =20 exclude_file_name_regexp--sc_unmarked_diagnostics =3D \ - ^(docs/apibuild.py|tests/virt-aa-helper-test|docs/js/.*\.js)$$ + ^(scripts/apibuild.py|tests/virt-aa-helper-test|docs/js/.*\.js)$$ =20 exclude_file_name_regexp--sc_size_of_brackets =3D build-aux/syntax-check\.= mk =20 diff --git a/docs/Makefile.am b/docs/Makefile.am index ca7a2ec260..f37417cafb 100644 --- a/docs/Makefile.am +++ b/docs/Makefile.am @@ -163,7 +163,6 @@ schemadir =3D $(pkgdatadir)/schemas schema_DATA =3D $(wildcard $(srcdir)/schemas/*.rng) =20 EXTRA_DIST=3D \ - apibuild.py \ site.xsl subsite.xsl newapi.xsl page.xsl \ wrapstring.xsl \ $(dot_html_in) $(gif) $(apipng) \ @@ -278,13 +277,13 @@ python_generated_files =3D \ libvirt-admin-refs.xml \ $(NULL) =20 -APIBUILD=3D$(srcdir)/apibuild.py +APIBUILD=3D$(top_srcdir)/scripts/apibuild.py APIBUILD_STAMP=3Dapibuild.py.stamp CLEANFILES +=3D $(APIBUILD_STAMP) =20 $(python_generated_files): $(APIBUILD_STAMP) =20 -$(APIBUILD_STAMP): $(srcdir)/apibuild.py \ +$(APIBUILD_STAMP): $(top_srcdir)/scripts/apibuild.py \ $(top_srcdir)/include/libvirt/libvirt.h \ $(top_srcdir)/include/libvirt/libvirt-common.h.in \ $(top_srcdir)/include/libvirt/libvirt-domain-checkpoint.h \ diff --git a/docs/apibuild.py b/scripts/apibuild.py similarity index 100% rename from docs/apibuild.py rename to scripts/apibuild.py --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sun May 5 18:51:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1573483538; cv=none; d=zoho.com; s=zohoarc; b=W0T2O+G5GvN6I4Q1zbzF4Aq4p7OLKE4M9YeBHWOuuxi0G9Siao9gaUClOEEJrE1SHKGCRIq0df+OW7I3DqCgffrn6+AGC8PwPJU2JLPKdHWJ4Wk1a/XWE8xMsigQU5BMPdpl2Ka2TDkfuqWOQHGIaR2SVPWgLU8+rsDRo1zhJ8U= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573483538; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Gkbysj2zmn+ownC2p88CjpLxkSaQiWtHbcgQvohpUjY=; b=WywC0Un96N5VxT8iXQAtRzFFJkPEjNcPWtxqCP8eVGOz4f6UEhsJETvT4563W2AYVzM0P/Qf/U8v5OyTex502+MWsKNzbgnBjR057WqnmdljS0HTxDo/COUlqp8wMj0DKyfggM7WQ/SaYh70ZI2Rk73nsVL3dst2pH6DRIVMUuY= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 1573483538449108.43024812141243; Mon, 11 Nov 2019 06:45:38 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-425-y_vJ6-M2OiyTHWV2qwVXYw-1; Mon, 11 Nov 2019 09:45:35 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B99F185EE6D; Mon, 11 Nov 2019 14:45:30 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 87B2E5DA7D; Mon, 11 Nov 2019 14:45:30 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 3A0644EA4D; Mon, 11 Nov 2019 14:45:30 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xABEjEBO020586 for ; Mon, 11 Nov 2019 09:45:14 -0500 Received: by smtp.corp.redhat.com (Postfix) id 4147B5DEC0; Mon, 11 Nov 2019 14:45:14 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 731E45DA7D; Mon, 11 Nov 2019 14:45:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573483537; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=Gkbysj2zmn+ownC2p88CjpLxkSaQiWtHbcgQvohpUjY=; b=WntURhnPddZtxE61qKtcmdvT63nmP6kB3Ud1VisY7B4zRWRj7dEGd4y89U4tG/dOM6sESf ZqhrqYwPPBjmVn4pPMGnboTpcGkfTMrmjjI9vZHh9uu7v6y2O6l0ErWVQo/3QrwU6IJz3b WOEXZPJ3E82LENXKvNq6CLwpAjf6Ghk= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 11 Nov 2019 14:38:24 +0000 Message-Id: <20191111143826.16050-22-berrange@redhat.com> In-Reply-To: <20191111143826.16050-1-berrange@redhat.com> References: <20191111143826.16050-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v5 21/23] docs: move reformat-news.py to the scripts/ directory X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-MC-Unique: y_vJ6-M2OiyTHWV2qwVXYw-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Signed-off-by: Daniel P. Berrang=C3=A9 --- Makefile.am | 7 ++++--- {docs =3D> scripts}/reformat-news.py | 0 2 files changed, 4 insertions(+), 3 deletions(-) rename {docs =3D> scripts}/reformat-news.py (100%) diff --git a/Makefile.am b/Makefile.am index 67d07eff41..8359ebc6ad 100644 --- a/Makefile.am +++ b/Makefile.am @@ -66,6 +66,7 @@ EXTRA_DIST =3D \ scripts/minimize-po.py \ scripts/mock-noinline.py \ scripts/prohibit-duplicate-header.py \ + scripts/reformat-news.py \ scripts/test-wrap-argv.py \ build-aux/syntax-check.mk \ build-aux/useless-if-before-free \ @@ -81,7 +82,7 @@ pkgconfig_DATA =3D libvirt.pc libvirt-qemu.pc libvirt-lxc= .pc libvirt-admin.pc NEWS: \ $(srcdir)/docs/news.xml \ $(srcdir)/docs/news-ascii.xsl \ - $(srcdir)/docs/reformat-news.py + $(top_srcdir)/scripts/reformat-news.py $(AM_V_GEN) \ if [ -x $(XSLTPROC) ]; then \ $(XSLTPROC) --nonet \ @@ -89,14 +90,14 @@ NEWS: \ $(srcdir)/docs/news.xml \ >$@-tmp \ || { rm -f $@-tmp; exit 1; }; \ - $(RUNUTF8) $(PYTHON) $(srcdir)/docs/reformat-news.py $@-tmp >$@ \ + $(RUNUTF8) $(PYTHON) $(top_srcdir)/scripts/reformat-news.py $@-tmp >$@ \ || { rm -f $@-tmp; exit 1; }; \ rm -f $@-tmp; \ fi EXTRA_DIST +=3D \ $(srcdir)/docs/news.xml \ $(srcdir)/docs/news-ascii.xsl \ - $(srcdir)/docs/reformat-news.py + $(NULL) =20 rpm: clean @(unset CDPATH ; $(MAKE) dist && rpmbuild -ta $(distdir).tar.xz) diff --git a/docs/reformat-news.py b/scripts/reformat-news.py similarity index 100% rename from docs/reformat-news.py rename to scripts/reformat-news.py --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sun May 5 18:51:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 205.139.110.61 as permitted sender) client-ip=205.139.110.61; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1573483543; cv=none; d=zoho.com; s=zohoarc; b=iP0qHisMAThLjPvVH8PoZRb17o/TQ+sxyvhsbVwKwKJV4oo34tYb11tI/K2dIGR/Iie8/Ot3LZfXiwiFK42iF+Eo1BrmanUD5KKNcwdHKTT4jqCcjtmulJSY6xJy7itnv5XzzKQ+pajvg1qCI60qpLvl6Fywn2mxJxmQ5cL0sgg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573483543; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=j1MqcT8RAmpGqCMBQQzJzvptxM1aj62Zr5g8sMfBQXU=; b=Thu3N/oeGYfs0oxSnFadCL+i5dMzZsUFHXPTeop35u4BIMSqX4GJSK+ZRsW+BuN4codRLMllnZ5D1CTe7NyyNIxWwpDfwvQj0eXXiJKV7svdnlrjKA3jv5wTv+WDHK7TEtcBkl/LWGrxLceU8jEdB/Zkhi+dktFn8R7foY9fSmY= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 205.139.110.61 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.61]) by mx.zohomail.com with SMTPS id 1573483543150749.4029442621476; Mon, 11 Nov 2019 06:45:43 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-78-sNeYG-YNO1aOAfUYfO5ATw-1; Mon, 11 Nov 2019 09:45:39 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1B7DFDBEE; Mon, 11 Nov 2019 14:45:34 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id EC107608E6; Mon, 11 Nov 2019 14:45:33 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id AF25F4ED21; Mon, 11 Nov 2019 14:45:33 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xABEjFJp020600 for ; Mon, 11 Nov 2019 09:45:15 -0500 Received: by smtp.corp.redhat.com (Postfix) id 2358E18B91; Mon, 11 Nov 2019 14:45:15 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 879905DA7D; Mon, 11 Nov 2019 14:45:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573483542; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=j1MqcT8RAmpGqCMBQQzJzvptxM1aj62Zr5g8sMfBQXU=; b=iv21K1UfoLg1FBNtSW9GJGdMJjMahf1XZ8p/vUqvb/p5YUsgFUyXEBOc3sxMYDSIT1SV/d fD/2UzxzjSYt03IOBpmBWMQQWKI/nOnxQcLM1V8XXK6dtrbat+QDmqyDmBbE1Y9DefkH5c BSB8pb/bGOnq4Z3Aa9BuMDoOcByfXwM= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 11 Nov 2019 14:38:25 +0000 Message-Id: <20191111143826.16050-23-berrange@redhat.com> In-Reply-To: <20191111143826.16050-1-berrange@redhat.com> References: <20191111143826.16050-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v5 22/23] docs: move esx_vi_generator.py to the scripts/ directory X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-MC-Unique: sNeYG-YNO1aOAfUYfO5ATw-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Signed-off-by: Daniel P. Berrang=C3=A9 --- Makefile.am | 1 + {src/esx =3D> scripts}/esx_vi_generator.py | 0 src/esx/Makefile.inc.am | 6 +++--- 3 files changed, 4 insertions(+), 3 deletions(-) rename {src/esx =3D> scripts}/esx_vi_generator.py (100%) diff --git a/Makefile.am b/Makefile.am index 8359ebc6ad..14456161c3 100644 --- a/Makefile.am +++ b/Makefile.am @@ -57,6 +57,7 @@ EXTRA_DIST =3D \ scripts/check-symfile.py \ scripts/check-symsorting.py \ scripts/dtrace2systemtap.py \ + scripts/esx_vi_generator.py \ scripts/genaclperms.py \ scripts/genpolkit.py \ scripts/gensystemtap.py \ diff --git a/src/esx/esx_vi_generator.py b/scripts/esx_vi_generator.py similarity index 100% rename from src/esx/esx_vi_generator.py rename to scripts/esx_vi_generator.py diff --git a/src/esx/Makefile.inc.am b/src/esx/Makefile.inc.am index 6b10755b7e..aa2d17c791 100644 --- a/src/esx/Makefile.inc.am +++ b/src/esx/Makefile.inc.am @@ -43,7 +43,6 @@ ESX_DRIVER_GENERATED =3D \ ESX_DRIVER_EXTRA_DIST =3D \ esx/README \ esx/esx_vi_generator.input \ - esx/esx_vi_generator.py \ $(NULL) =20 ESX_GENERATED_STAMP =3D .esx_vi_generator.stamp @@ -60,9 +59,10 @@ BUILT_SOURCES +=3D $(ESX_DRIVER_GENERATED) $(ESX_DRIVER_GENERATED): $(ESX_GENERATED_STAMP) =20 $(ESX_GENERATED_STAMP): $(srcdir)/esx/esx_vi_generator.input \ - $(srcdir)/esx/esx_vi_generator.py + $(top_srcdir)/scripts/esx_vi_generator.py $(AM_V_GEN) $(RUNUTF8) $(PYTHON) \ - $(srcdir)/esx/esx_vi_generator.py $(srcdir) $(builddir) && touch $@ + $(top_srcdir)/scripts/esx_vi_generator.py \ + $(srcdir) $(builddir) && touch $@ =20 CLEANFILES +=3D $(ESX_DRIVER_GENERATED) $(ESX_GENERATED_STAMP) =20 --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list From nobody Sun May 5 18:51:42 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1573483707; cv=none; d=zoho.com; s=zohoarc; b=N7JeX9FEyR3pZLxVgjsTc5uTjbKSXmdalMQvrSOBt39EjKPQDDjhF71x+2CPSh1T2CinI5qu2trTan+RT66JxZLo+C8OY8rDYgjtSUlR/XBTpTFVL7u3Ls5uQGYxYDbqisnFwhUKegy6RvK7BZqO0xRzm6nE94qYoxwzZvuAank= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573483707; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=mIfj22wjf2WF6Zi/UOztzMCoQ1/C/+DxFEXiAuon3s0=; b=DMFwBC3pGHTGN3FGSrDU1fhlGEynIBOcCZtz1ihMQkizsZMTodzB4E1pWYwjVdB7Ml7jZD7Rkqvb+uMlThrRNj215OBu8N3M2wDMfK4Dr1a8SdewR3+JHdxvA8a7oFKehZ0V834vUjXYJ/Yfv6ODS6o9NUehLJctgKv6UyAK+IY= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 1573483707720957.3737706994724; Mon, 11 Nov 2019 06:48:27 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-32-CAslzrfsNvOAE6YR7wxebQ-1; Mon, 11 Nov 2019 09:45:43 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 9E516800D49; Mon, 11 Nov 2019 14:45:37 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 784705DA7D; Mon, 11 Nov 2019 14:45:37 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 359584EDAF; Mon, 11 Nov 2019 14:45:37 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xABEjGmD020607 for ; Mon, 11 Nov 2019 09:45:16 -0500 Received: by smtp.corp.redhat.com (Postfix) id 06F735DA7D; Mon, 11 Nov 2019 14:45:16 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6B1B15DEC0; Mon, 11 Nov 2019 14:45:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573483703; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=mIfj22wjf2WF6Zi/UOztzMCoQ1/C/+DxFEXiAuon3s0=; b=dzj/Aa4WQaVTZVrtr/LW9PIcN9J+6QKiEadQFROgxFkLGi6Ywsbgm4lxrA/Xwds2CdRKAu 9u17f9nLAaggUHtm9+B5vP81LgyGjgVI+JbORkDCACzLrcSABP/I68BHFdsa4lk/sEyIvj YcbEEj+uhmtaLrlXv9KXr4cR469kd+w= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 11 Nov 2019 14:38:26 +0000 Message-Id: <20191111143826.16050-24-berrange@redhat.com> In-Reply-To: <20191111143826.16050-1-berrange@redhat.com> References: <20191111143826.16050-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v5 23/23] docs: move hyperv_wmi_generator.py to the scripts/ directory X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-MC-Unique: CAslzrfsNvOAE6YR7wxebQ-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: Cole Robinson --- Makefile.am | 1 + {src/hyperv =3D> scripts}/hyperv_wmi_generator.py | 0 src/hyperv/Makefile.inc.am | 5 ++--- 3 files changed, 3 insertions(+), 3 deletions(-) rename {src/hyperv =3D> scripts}/hyperv_wmi_generator.py (100%) diff --git a/Makefile.am b/Makefile.am index 14456161c3..adad46f1f0 100644 --- a/Makefile.am +++ b/Makefile.am @@ -64,6 +64,7 @@ EXTRA_DIST =3D \ scripts/group-qemu-caps.py \ scripts/header-ifdef.py \ scripts/hvsupport.py \ + scripts/hyperv_wmi_generator.py \ scripts/minimize-po.py \ scripts/mock-noinline.py \ scripts/prohibit-duplicate-header.py \ diff --git a/src/hyperv/hyperv_wmi_generator.py b/scripts/hyperv_wmi_genera= tor.py similarity index 100% rename from src/hyperv/hyperv_wmi_generator.py rename to scripts/hyperv_wmi_generator.py diff --git a/src/hyperv/Makefile.inc.am b/src/hyperv/Makefile.inc.am index b71602c971..5d801b28b6 100644 --- a/src/hyperv/Makefile.inc.am +++ b/src/hyperv/Makefile.inc.am @@ -23,7 +23,6 @@ HYPERV_GENERATED_STAMP =3D .hyperv_wmi_generator.stamp =20 HYPERV_DRIVER_EXTRA_DIST =3D \ hyperv/hyperv_wmi_generator.input \ - hyperv/hyperv_wmi_generator.py \ $(NULL) =20 DRIVER_SOURCE_FILES +=3D $(HYPERV_DRIVER_SOURCES) @@ -38,9 +37,9 @@ BUILT_SOURCES +=3D $(HYPERV_DRIVER_GENERATED) $(HYPERV_DRIVER_GENERATED): $(HYPERV_GENERATED_STAMP) =20 $(HYPERV_GENERATED_STAMP): $(srcdir)/hyperv/hyperv_wmi_generator.input \ - $(srcdir)/hyperv/hyperv_wmi_generator.py + $(top_srcdir)/scripts/hyperv_wmi_generator.py $(AM_V_GEN) $(RUNUTF8) $(PYTHON) \ - $(srcdir)/hyperv/hyperv_wmi_generator.py $(srcdir) $(builddir) \ + $(top_srcdir)/scripts/hyperv_wmi_generator.py $(srcdir) $(builddir) \ && touch $@ =20 CLEANFILES +=3D $(HYPERV_DRIVER_GENERATED) $(HYPERV_GENERATED_STAMP) --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list