From nobody Tue Nov 26 13:31:09 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1573483534; cv=none; d=zoho.com; s=zohoarc; b=TIJ+rm0b6BWbY/9StADZliwgj3GrbGEo4/pmTzfIBgcmwbrIN5e/M19A9xCOYfU3WWp9eeD82mhwEBB7Ld2ZoRCa+q4x9l/irO+h7wNADpydM5SzzC4kKWTmCuM1aBn53H9QHMsTA2bJvuZbaz6TBPFud7i5WqKSI0fEagRqU0U= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573483534; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=AZizGoOw/5+vZNcCVIv+HnHefdRObZOdgRLQi5lqbWw=; b=Io/4bT6VK+lcAUt1NV7bxzK+f9kTtI9PTXMRCl6U1XDnvbmPY9q+625JwPrxhe30Pk/kbZ1E3GjUYLJgOzIPulAtBmYPVtOuQH2tNEtKyWFAFp/mmCrdpxrwuxnv6hfbldg+ve/JCpqFcm5sDMS9x8KJgMR+v/a4Ov53mlxL7Lc= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-2.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 1573483534522135.37681485294513; Mon, 11 Nov 2019 06:45:34 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-277-qedqnHraO-ew_3yJETTP9A-1; Mon, 11 Nov 2019 09:45:29 -0500 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.phx2.redhat.com [10.5.11.13]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id DF8BC85EE95; Mon, 11 Nov 2019 14:45:23 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.20]) by smtp.corp.redhat.com (Postfix) with ESMTPS id A02FC60904; Mon, 11 Nov 2019 14:45:23 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 4EEEC1802225; Mon, 11 Nov 2019 14:45:23 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xABEj9qX020545 for ; Mon, 11 Nov 2019 09:45:09 -0500 Received: by smtp.corp.redhat.com (Postfix) id AB6B55DD73; Mon, 11 Nov 2019 14:45:09 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id EF0661B41F; Mon, 11 Nov 2019 14:45:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573483533; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=AZizGoOw/5+vZNcCVIv+HnHefdRObZOdgRLQi5lqbWw=; b=PaeM7EqAR8tKd5ajqY8JRZkaDUWN+95W//RNhWsOtzjc6vtjg5zcWsJ7qPH8AIil5jlzMD /ba7Ormw5gnOwf0WrjuhqYP2K75hNGPDg4AEa/ByclSOdOIxi717n6kuLB4azMptqrMEiv pIuHXX67DOWAe7VrFdAV4dkXHZYgSM0= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 11 Nov 2019 14:38:20 +0000 Message-Id: <20191111143826.16050-18-berrange@redhat.com> In-Reply-To: <20191111143826.16050-1-berrange@redhat.com> References: <20191111143826.16050-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v5 17/23] tests: rewrite file access checker in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.13 X-MC-Unique: qedqnHraO-ew_3yJETTP9A-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" As part of an goal to eliminate Perl from libvirt build tools, rewrite the check-file-access.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. Signed-off-by: Daniel P. Berrang=C3=A9 --- Makefile.am | 1 + scripts/check-file-access.py | 123 +++++++++++++++++++++++++++++++ tests/Makefile.am | 3 +- tests/check-file-access.pl | 126 -------------------------------- tests/file_access_whitelist.txt | 2 +- 5 files changed, 126 insertions(+), 129 deletions(-) create mode 100755 scripts/check-file-access.py delete mode 100755 tests/check-file-access.pl diff --git a/Makefile.am b/Makefile.am index 769cd4ce64..19114069e3 100644 --- a/Makefile.am +++ b/Makefile.am @@ -50,6 +50,7 @@ EXTRA_DIST =3D \ scripts/check-aclrules.py \ scripts/check-drivername.py \ scripts/check-driverimpls.py \ + scripts/check-file-access.py \ scripts/check-remote-protocol.py \ scripts/check-spacing.py \ scripts/check-symfile.py \ diff --git a/scripts/check-file-access.py b/scripts/check-file-access.py new file mode 100755 index 0000000000..cdcbf2666f --- /dev/null +++ b/scripts/check-file-access.py @@ -0,0 +1,123 @@ +#!/usr/bin/env python +# +# Copyright (C) 2016-2019 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . +# +# This script is supposed to check test_file_access.txt file and +# warn about file accesses outside our working tree. +# +# + +from __future__ import print_function + +import re +import sys + +access_file =3D "test_file_access.txt" +whitelist_file =3D "file_access_whitelist.txt" + +known_actions =3D ["open", "fopen", "access", "stat", "lstat", "connect"] + +files =3D [] +whitelist =3D [] + +with open(access_file, "r") as fh: + for line in fh: + line =3D line.rstrip("\n") + + m =3D re.search(r'''^(\S*):\s*(\S*):\s*(\S*)(\s*:\s*(.*))?$''', li= ne) + if m is not None: + rec =3D { + "path": m.group(1), + "action": m.group(2), + "progname": m.group(3), + "testname": m.group(5), + } + files.append(rec) + else: + raise Exception("Malformed line %s" % line) + +with open(whitelist_file, "r") as fh: + for line in fh: + line =3D line.rstrip("\n") + + if re.search(r'''^\s*#.*$''', line): + continue # comment + if line =3D=3D "": + continue + + m =3D re.search(r'''^(\S*):\s*(\S*)(:\s*(\S*)(\s*:\s*(.*))?)?$''',= line) + if m is not None and m.group(2) in known_actions: + # $path: $action: $progname: $testname + rec =3D { + "path": m.group(1), + "action": m.group(3), + "progname": m.group(4), + "testname": m.group(6), + } + whitelist.append(rec) + else: + m =3D re.search(r'''^(\S*)(:\s*(\S*)(\s*:\s*(.*))?)?$''', line) + if m is not None: + # $path: $progname: $testname + rec =3D { + "path": m.group(1), + "action": None, + "progname": m.group(3), + "testname": m.group(5), + } + whitelist.append(rec) + else: + raise Exception("Malformed line %s" % line) + + +# Now we should check if %traces is included in $whitelist. For +# now checking just keys is sufficient +err =3D False +for file in files: + match =3D False + + for rule in whitelist: + if not re.search("^" + rule["path"], file["path"]): + continue + + if (rule["action"] is not None and + not re.search("^" + rule["action"], file["action"])): + continue + + if (rule["progname"] is not None and + not re.search("^" + rule["progname"], file["progname"])): + continue + + if (rule["testname"] is not None and + file["testname"] is not None and + not re.search("^" + rule["testname"], file["testname"])): + continue + + match =3D True + + if not match: + err =3D True + print("%s: %s: %s" % + (file["path"], file["action"], file["progname"]), + file=3Dsys.stderr, end=3D"") + if file["testname"] is not None: + print(": %s" % file["testname"], file=3Dsys.stderr, end=3D"") + print("", file=3Dsys.stderr) + +if err: + sys.exit(1) +sys.exit(0) diff --git a/tests/Makefile.am b/tests/Makefile.am index 9d9c582e42..c3bca26019 100644 --- a/tests/Makefile.am +++ b/tests/Makefile.am @@ -453,14 +453,13 @@ EXTRA_DIST +=3D $(test_scripts) if WITH_LINUX check-access: file-access-clean VIR_TEST_FILE_ACCESS=3D1 $(MAKE) $(AM_MAKEFLAGS) check - $(PERL) check-file-access.pl | sort -u + $(RUNUTF8) $(PYTHON) $(top_srcdir)/scripts/check-file-access.py | sort -u =20 file-access-clean: > test_file_access.txt endif WITH_LINUX =20 EXTRA_DIST +=3D \ - check-file-access.pl \ file_access_whitelist.txt =20 if WITH_TESTS diff --git a/tests/check-file-access.pl b/tests/check-file-access.pl deleted file mode 100755 index ea0b7a18a2..0000000000 --- a/tests/check-file-access.pl +++ /dev/null @@ -1,126 +0,0 @@ -#!/usr/bin/env perl -# -# Copyright (C) 2016 Red Hat, Inc. -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library. If not, see -# . -# -# This script is supposed to check test_file_access.txt file and -# warn about file accesses outside our working tree. -# -# - -use strict; -use warnings; - -my $access_file =3D "test_file_access.txt"; -my $whitelist_file =3D "file_access_whitelist.txt"; - -my @known_actions =3D ("open", "fopen", "access", "stat", "lstat", "connec= t"); - -my @files; -my @whitelist; - -open FILE, "<", $access_file or die "Unable to open $access_file: $!"; -while () { - chomp; - if (/^(\S*):\s*(\S*):\s*(\S*)(\s*:\s*(.*))?$/) { - my %rec; - ${rec}{path} =3D $1; - ${rec}{action} =3D $2; - ${rec}{progname} =3D $3; - if (defined $5) { - ${rec}{testname} =3D $5; - } - push (@files, \%rec); - } else { - die "Malformed line $_"; - } -} -close FILE; - -open FILE, "<", $whitelist_file or die "Unable to open $whitelist_file: $!= "; -while () { - chomp; - if (/^\s*#.*$/) { - # comment - } elsif (/^(\S*):\s*(\S*)(:\s*(\S*)(\s*:\s*(.*))?)?$/ and - grep /^$2$/, @known_actions) { - # $path: $action: $progname: $testname - my %rec; - ${rec}{path} =3D $1; - ${rec}{action} =3D $3; - if (defined $4) { - ${rec}{progname} =3D $4; - } - if (defined $6) { - ${rec}{testname} =3D $6; - } - push (@whitelist, \%rec); - } elsif (/^(\S*)(:\s*(\S*)(\s*:\s*(.*))?)?$/) { - # $path: $progname: $testname - my %rec; - ${rec}{path} =3D $1; - if (defined $3) { - ${rec}{progname} =3D $3; - } - if (defined $5) { - ${rec}{testname} =3D $5; - } - push (@whitelist, \%rec); - } else { - die "Malformed line $_"; - } -} -close FILE; - -# Now we should check if %traces is included in $whitelist. For -# now checking just keys is sufficient -my $error =3D 0; -for my $file (@files) { - my $match =3D 0; - - for my $rule (@whitelist) { - if (not %${file}{path} =3D~ m/^$rule->{path}$/) { - next; - } - - if (defined %${rule}{action} and - not %${file}{action} =3D~ m/^$rule->{action}$/) { - next; - } - - if (defined %${rule}{progname} and - not %${file}{progname} =3D~ m/^$rule->{progname}$/) { - next; - } - - if (defined %${rule}{testname} and - defined %${file}{testname} and - not %${file}{testname} =3D~ m/^$rule->{testname}$/) { - next; - } - - $match =3D 1; - } - - if (not $match) { - $error =3D 1; - print "$file->{path}: $file->{action}: $file->{progname}"; - print ": $file->{testname}" if defined %${file}{testname}; - print "\n"; - } -} - -exit $error; diff --git a/tests/file_access_whitelist.txt b/tests/file_access_whitelist.= txt index 3fb318cbab..5ec7ee63bb 100644 --- a/tests/file_access_whitelist.txt +++ b/tests/file_access_whitelist.txt @@ -5,7 +5,7 @@ # $path: $progname: $testname # $path: $action: $progname: $testname # -# All these variables are evaluated as perl RE. So to allow +# All these variables are evaluated as python RE. So to allow # /dev/sda and /dev/sdb, you can just '/dev/sd[a-b]', or to allow # /proc/$pid/status you can '/proc/\d+/status' and so on. # Moreover, $action, $progname and $testname can be empty, in which --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list