From nobody Sun Feb 8 03:57:57 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; envelope-from=libvir-list-bounces@redhat.com; helo=mx1.redhat.com; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1568219344; cv=none; d=zoho.com; s=zohoarc; b=VvoWnOgU7StR886OrMEYH+J68mKBE0rHm0k54s6OhblvgxUwGwgQlEbJkwY7vd+8N2geWRaEjlHAnibel0qzfZv4qO5g9i5BUm4uWK66w1t7l4Gzhge01V/i1nhqYe87EP/bVANFyLTdemm+dHYLQllIs+aGkRjsLNy6fpkii/o= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1568219344; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To:ARC-Authentication-Results; bh=wIFSVdGFCCQ1Wl4ooTmuv8OqUysUG9ye6drp0WXRvPg=; b=G2cteWC3IfoW5CCtWYgOKek9BXCipr5tczmnIlAPCEiGo9wWTvlB+X7BDKi9ouQii3O5wBsjMV2DrodNFi0iQ+KNnYnGur/1oHOchueOmhOSx9joOaHurmjZ+enAsFKEbhb6n5rJEdKQ4l12KPiaONFZqawDdNgoIbchd/dQv4U= ARC-Authentication-Results: i=1; mx.zoho.com; spf=pass (zoho.com: domain of redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from mx1.redhat.com (mx1.redhat.com [209.132.183.28]) by mx.zohomail.com with SMTPS id 1568219344832210.54588634384538; Wed, 11 Sep 2019 09:29:04 -0700 (PDT) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 2652D8980FF; Wed, 11 Sep 2019 16:29:02 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id F257060C5E; Wed, 11 Sep 2019 16:29:01 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 9E71E1803515; Wed, 11 Sep 2019 16:29:01 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id x8BGSZcS013295 for ; Wed, 11 Sep 2019 12:28:35 -0400 Received: by smtp.corp.redhat.com (Postfix) id 2386F6017E; Wed, 11 Sep 2019 16:28:35 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 878CE6012A; Wed, 11 Sep 2019 16:28:34 +0000 (UTC) From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Wed, 11 Sep 2019 17:23:31 +0100 Message-Id: <20190911162333.8668-23-berrange@redhat.com> In-Reply-To: <20190911162333.8668-1-berrange@redhat.com> References: <20190911162333.8668-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v2 22/24] tests: rewrite file access checker in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.6.2 (mx1.redhat.com [10.5.110.67]); Wed, 11 Sep 2019 16:29:02 +0000 (UTC) As part of an goal to eliminate Perl from libvirt build tools, rewrite the check-file-access.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. Signed-off-by: Daniel P. Berrang=C3=A9 --- tests/Makefile.am | 4 +- tests/check-file-access.pl | 126 -------------------------------- tests/check-file-access.py | 121 ++++++++++++++++++++++++++++++ tests/file_access_whitelist.txt | 2 +- 4 files changed, 124 insertions(+), 129 deletions(-) delete mode 100755 tests/check-file-access.pl create mode 100755 tests/check-file-access.py diff --git a/tests/Makefile.am b/tests/Makefile.am index f92710db43..48d1924707 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -450,14 +450,14 @@ EXTRA_DIST +=3D $(test_scripts) if WITH_LINUX check-access: file-access-clean VIR_TEST_FILE_ACCESS=3D1 $(MAKE) $(AM_MAKEFLAGS) check - $(PERL) check-file-access.pl | sort -u + $(RUNUTF8) $(PYTHON) check-file-access.py | sort -u =20 file-access-clean: > test_file_access.txt endif WITH_LINUX =20 EXTRA_DIST +=3D \ - check-file-access.pl \ + check-file-access.py \ file_access_whitelist.txt =20 if WITH_TESTS diff --git a/tests/check-file-access.pl b/tests/check-file-access.pl deleted file mode 100755 index ea0b7a18a2..0000000000 --- a/tests/check-file-access.pl +++ /dev/null @@ -1,126 +0,0 @@ -#!/usr/bin/env perl -# -# Copyright (C) 2016 Red Hat, Inc. -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library. If not, see -# . -# -# This script is supposed to check test_file_access.txt file and -# warn about file accesses outside our working tree. -# -# - -use strict; -use warnings; - -my $access_file =3D "test_file_access.txt"; -my $whitelist_file =3D "file_access_whitelist.txt"; - -my @known_actions =3D ("open", "fopen", "access", "stat", "lstat", "connec= t"); - -my @files; -my @whitelist; - -open FILE, "<", $access_file or die "Unable to open $access_file: $!"; -while () { - chomp; - if (/^(\S*):\s*(\S*):\s*(\S*)(\s*:\s*(.*))?$/) { - my %rec; - ${rec}{path} =3D $1; - ${rec}{action} =3D $2; - ${rec}{progname} =3D $3; - if (defined $5) { - ${rec}{testname} =3D $5; - } - push (@files, \%rec); - } else { - die "Malformed line $_"; - } -} -close FILE; - -open FILE, "<", $whitelist_file or die "Unable to open $whitelist_file: $!= "; -while () { - chomp; - if (/^\s*#.*$/) { - # comment - } elsif (/^(\S*):\s*(\S*)(:\s*(\S*)(\s*:\s*(.*))?)?$/ and - grep /^$2$/, @known_actions) { - # $path: $action: $progname: $testname - my %rec; - ${rec}{path} =3D $1; - ${rec}{action} =3D $3; - if (defined $4) { - ${rec}{progname} =3D $4; - } - if (defined $6) { - ${rec}{testname} =3D $6; - } - push (@whitelist, \%rec); - } elsif (/^(\S*)(:\s*(\S*)(\s*:\s*(.*))?)?$/) { - # $path: $progname: $testname - my %rec; - ${rec}{path} =3D $1; - if (defined $3) { - ${rec}{progname} =3D $3; - } - if (defined $5) { - ${rec}{testname} =3D $5; - } - push (@whitelist, \%rec); - } else { - die "Malformed line $_"; - } -} -close FILE; - -# Now we should check if %traces is included in $whitelist. For -# now checking just keys is sufficient -my $error =3D 0; -for my $file (@files) { - my $match =3D 0; - - for my $rule (@whitelist) { - if (not %${file}{path} =3D~ m/^$rule->{path}$/) { - next; - } - - if (defined %${rule}{action} and - not %${file}{action} =3D~ m/^$rule->{action}$/) { - next; - } - - if (defined %${rule}{progname} and - not %${file}{progname} =3D~ m/^$rule->{progname}$/) { - next; - } - - if (defined %${rule}{testname} and - defined %${file}{testname} and - not %${file}{testname} =3D~ m/^$rule->{testname}$/) { - next; - } - - $match =3D 1; - } - - if (not $match) { - $error =3D 1; - print "$file->{path}: $file->{action}: $file->{progname}"; - print ": $file->{testname}" if defined %${file}{testname}; - print "\n"; - } -} - -exit $error; diff --git a/tests/check-file-access.py b/tests/check-file-access.py new file mode 100755 index 0000000000..33583a5b52 --- /dev/null +++ b/tests/check-file-access.py @@ -0,0 +1,121 @@ +#!/usr/bin/env python +# +# Copyright (C) 2016-2019 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . +# +# This script is supposed to check test_file_access.txt file and +# warn about file accesses outside our working tree. +# +# + +from __future__ import print_function + +import re +import sys + +access_file =3D "test_file_access.txt" +whitelist_file =3D "file_access_whitelist.txt" + +known_actions =3D ["open", "fopen", "access", "stat", "lstat", "connect"] + +files =3D [] +whitelist =3D [] + +with open(access_file, "r") as fh: + for line in fh: + line =3D line.rstrip("\n") + + m =3D re.match(r'''^(\S*):\s*(\S*):\s*(\S*)(\s*:\s*(.*))?$''', lin= e) + if m is not None: + rec =3D { + "path": m.group(1), + "action": m.group(2), + "progname": m.group(3), + "testname": m.group(5), + } + files.append(rec) + else: + raise Exception("Malformed line %s" % line) + +with open(whitelist_file, "r") as fh: + for line in fh: + line =3D line.rstrip("\n") + + if re.match(r'''^\s*#.*$''', line): + continue # comment + if line =3D=3D "": + continue + + m =3D re.match(r'''^(\S*):\s*(\S*)(:\s*(\S*)(\s*:\s*(.*))?)?$''', = line) + if m is not None and m.group(2) in known_actions: + # $path: $action: $progname: $testname + rec =3D { + "path": m.group(1), + "action": m.group(3), + "progname": m.group(4), + "testname": m.group(6), + } + whitelist.append(rec) + else: + m =3D re.match(r'''^(\S*)(:\s*(\S*)(\s*:\s*(.*))?)?$''', line) + if m is not None: + # $path: $progname: $testname + rec =3D { + "path": m.group(1), + "action": None, + "progname": m.group(3), + "testname": m.group(5), + } + whitelist.append(rec) + else: + raise Exception("Malformed line %s" % line) + + +# Now we should check if %traces is included in $whitelist. For +# now checking just keys is sufficient +err =3D False +for file in files: + match =3D False + + for rule in whitelist: + if not re.match("^" + rule["path"], file["path"]): + continue + + if (rule["action"] is not None and + not re.match("^" + rule["action"], file["action"])): + continue + + if (rule["progname"] is not None and + not re.match("^" + rule["progname"], file["progname"])): + continue + + if (rule["testname"] is not None and + file["testname"] is not None and + not re.match("^" + rule["testname"], file["testname"])): + continue + + match =3D True + + if not match: + err =3D True + print("%s: %s: %s" % (file["path"], file["action"], file["progname= "]), file=3Dsys.stderr, end=3D"") + if file["testname"] is not None: + print(": %s" % file["testname"], file=3Dsys.stderr, end=3D"") + print("", file=3Dsys.stderr) + +if err: + sys.exit(1) +sys.exit(0) diff --git a/tests/file_access_whitelist.txt b/tests/file_access_whitelist.= txt index 3fb318cbab..5ec7ee63bb 100644 --- a/tests/file_access_whitelist.txt +++ b/tests/file_access_whitelist.txt @@ -5,7 +5,7 @@ # $path: $progname: $testname # $path: $action: $progname: $testname # -# All these variables are evaluated as perl RE. So to allow +# All these variables are evaluated as python RE. So to allow # /dev/sda and /dev/sdb, you can just '/dev/sd[a-b]', or to allow # /proc/$pid/status you can '/proc/\d+/status' and so on. # Moreover, $action, $progname and $testname can be empty, in which --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list