From nobody Tue Nov 26 13:46:12 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) client-ip=207.211.31.81; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-1.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1573483529; cv=none; d=zoho.com; s=zohoarc; b=KV7UtITq9nGUVFdxlRXkvxqkNRBF7fd59cqgKwCzlM+1SLY8iJt9yiZ7jdjwtpblCbpIzqwJoPzbd43fN+GRG9Re8dIRFpeMTxsuucJPJaTQEbjtWiC5wOX01WLUtX90ITTp+VawyoKOYuwmXOBtRIrlQtI2xb++/bsy2Tz1iZc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zoho.com; s=zohoarc; t=1573483529; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=bpGMD0Hr0bLAIfPbD/aRJHAUNYFqdzsGzQLl/epwzx8=; b=OQOjOWhhbowgIegjN8MsWm6uymec4HqOc18Qg2SEBv9axKSutE9VfZM0XPhk0Q6kfkxYbjb93XomUmW2YhsfMvyMAwPVuK+inPxxZc13uYrYJ5Rnxp+SHyfVcwDjhemX/5IUj3JfCpl6dy2+gbjBPZlbZq9UWTWN1o/DCFP4zoM= ARC-Authentication-Results: i=1; mx.zoho.com; dkim=pass; spf=pass (zoho.com: domain of redhat.com designates 207.211.31.81 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) header.from= Return-Path: Received: from us-smtp-delivery-1.mimecast.com (us-smtp-1.mimecast.com [207.211.31.81]) by mx.zohomail.com with SMTPS id 1573483529330974.9561523451071; Mon, 11 Nov 2019 06:45:29 -0800 (PST) Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-299-cMtG87UWNi2aD0iW7_QdLA-1; Mon, 11 Nov 2019 09:45:26 -0500 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E317D107ACE5; Mon, 11 Nov 2019 14:45:19 +0000 (UTC) Received: from colo-mx.corp.redhat.com (colo-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.21]) by smtp.corp.redhat.com (Postfix) with ESMTPS id B6F0B851A; Mon, 11 Nov 2019 14:45:19 +0000 (UTC) Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by colo-mx.corp.redhat.com (Postfix) with ESMTP id 6A9304E597; Mon, 11 Nov 2019 14:45:19 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id xABEj3jG020500 for ; Mon, 11 Nov 2019 09:45:03 -0500 Received: by smtp.corp.redhat.com (Postfix) id 4835618B91; Mon, 11 Nov 2019 14:45:03 +0000 (UTC) Received: from catbus.gsslab.fab.redhat.com (dhcp-32.gsslab.fab.redhat.com [10.33.9.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 421DE5D9C9; Mon, 11 Nov 2019 14:45:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1573483528; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=bpGMD0Hr0bLAIfPbD/aRJHAUNYFqdzsGzQLl/epwzx8=; b=f7j+eAZ7TWNIktLWScB8ltNiwGR77EVWyryZPNs/c+yMT/GMB8ngYjn+QoFNz2hmB9+p/4 Gw/UNQXv+qdajslDbEELiy2WRn+QfHFKyWUPjF+0Z4sy52RvZMAXZy0Mka+96dg/d5+fhs P2Xwwofy2N5fOeeZwXBQyznNghhaFwQ= From: =?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= To: libvir-list@redhat.com Date: Mon, 11 Nov 2019 14:38:18 +0000 Message-Id: <20191111143826.16050-16-berrange@redhat.com> In-Reply-To: <20191111143826.16050-1-berrange@redhat.com> References: <20191111143826.16050-1-berrange@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH v5 15/23] tests: rewrite test argv line wrapper in Python X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-MC-Unique: cMtG87UWNi2aD0iW7_QdLA-1 X-Mimecast-Spam-Score: 0 Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) Content-Type: text/plain; charset="utf-8" As part of an goal to eliminate Perl from libvirt build tools, rewrite the test-wrap-argv.pl tool in Python. This was a straight conversion, manually going line-by-line to change the syntax from Perl to Python. Thus the overall structure of the file and approach is the same. Signed-off-by: Daniel P. Berrang=C3=A9 Reviewed-by: J=C3=A1n Tomko --- Makefile.am | 1 + build-aux/syntax-check.mk | 4 +- scripts/test-wrap-argv.py | 170 +++++++++++++++++++++++++++++++++++++ tests/test-wrap-argv.pl | 174 -------------------------------------- tests/testutils.c | 16 ++-- 5 files changed, 181 insertions(+), 184 deletions(-) create mode 100755 scripts/test-wrap-argv.py delete mode 100755 tests/test-wrap-argv.pl diff --git a/Makefile.am b/Makefile.am index 8c9c73c715..6f6cead526 100644 --- a/Makefile.am +++ b/Makefile.am @@ -61,6 +61,7 @@ EXTRA_DIST =3D \ scripts/minimize-po.py \ scripts/mock-noinline.py \ scripts/prohibit-duplicate-header.py \ + scripts/test-wrap-argv.py \ build-aux/syntax-check.mk \ build-aux/useless-if-before-free \ build-aux/vc-list-files \ diff --git a/build-aux/syntax-check.mk b/build-aux/syntax-check.mk index a61818855c..7d54df182a 100644 --- a/build-aux/syntax-check.mk +++ b/build-aux/syntax-check.mk @@ -2172,8 +2172,8 @@ header-ifdef: $(PYTHON) $(top_srcdir)/scripts/header-ifdef.py =20 test-wrap-argv: - $(AM_V_GEN)$(VC_LIST) | $(GREP) -E '\.(ldargs|args)' | xargs \ - $(PERL) $(top_srcdir)/tests/test-wrap-argv.pl --check + $(AM_V_GEN)$(VC_LIST) | $(GREP) -E '\.(ldargs|args)' | $(RUNUTF8) xargs \ + $(PYTHON) $(top_srcdir)/scripts/test-wrap-argv.py --check =20 group-qemu-caps: $(AM_V_GEN)$(PERL) $(top_srcdir)/tests/group-qemu-caps.pl --check $(top_s= rcdir)/ diff --git a/scripts/test-wrap-argv.py b/scripts/test-wrap-argv.py new file mode 100755 index 0000000000..dd9e7de824 --- /dev/null +++ b/scripts/test-wrap-argv.py @@ -0,0 +1,170 @@ +#!/usr/bin/env python +# +# Copyright (C) 2019 Red Hat, Inc. +# +# This library is free software; you can redistribute it and/or +# modify it under the terms of the GNU Lesser General Public +# License as published by the Free Software Foundation; either +# version 2.1 of the License, or (at your option) any later version. +# +# This library is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +# Lesser General Public License for more details. +# +# You should have received a copy of the GNU Lesser General Public +# License along with this library. If not, see +# . +# +# This script is intended to be passed a list of .args files, used +# to store command line ARGV for the test suites. It will reformat +# them such that there is at most one '-param value' on each line +# of the file. Parameter values that are longer than 80 chars will +# also be split. +# +# If --in-place is supplied as the first parameter of this script, +# the files will be changed in place. +# If --check is the first parameter, the script will return +# a non-zero value if a file is not wrapped correctly. +# Otherwise the rewrapped files are printed to the standard output. + +from __future__ import print_function + +import argparse +import subprocess +import sys + + +def rewrap_line(line): + bits =3D line.split(" ") + + # bits contains env vars, then the command line + # and then the arguments + env =3D [] + cmd =3D None + args =3D [] + + if bits[0].find("=3D") =3D=3D -1: + cmd =3D bits[0] + bits =3D bits[1:] + + for bit in bits: + # If no command is defined yet, we must still + # have env vars + if cmd is None: + # Look for leading / to indicate command name + if bit.startswith("/"): + cmd =3D bit + else: + env.append(bit) + else: + # If there's a leading '-' then this is a new + # parameter, otherwise its a value for the prev + # parameter. + if bit.startswith("-"): + args.append(bit) + else: + args[-1] =3D args[-1] + " " + bit + + # We might have to split line argument values... + args =3D [rewrap_arg(arg) for arg in args] + + # Print env + command first + return " \\\n".join(env + [cmd] + args) + "\n" + + +def rewrap_arg(arg): + ret =3D [] + max_len =3D 78 + + while len(arg) > max_len: + split =3D arg.rfind(",", 0, max_len + 1) + if split =3D=3D -1: + split =3D arg.rfind(":", 0, max_len + 1) + if split =3D=3D -1: + split =3D arg.rfind(" ", 0, max_len + 1) + if split =3D=3D -1: + print("cannot find nice place to split '%s' below 80 chars" % + arg, file=3Dsys.stderr) + split =3D max_len - 1 + + split =3D split + 1 + + ret.append(arg[0:split]) + arg =3D arg[split:] + + ret.append(arg) + return "\\\n".join(ret) + + +def rewrap(filename, in_place, check): + # Read the original file + with open(filename, 'r') as fh: + orig_lines =3D [] + for line in fh: + orig_lines.append(line) + + if len(orig_lines) =3D=3D 0: + return + + lines =3D [] + for line in orig_lines: + if line.endswith("\\\n"): + line =3D line[:-2] + lines.append(line) + + # Kill the last new line in the file + lines[-1] =3D lines[-1].rstrip("\n") + + # Reconstruct the master data by joining all lines + # and then split again based on the real desired + # newlines + lines =3D "".join(lines).split("\n") + + # Now each 'lines' entry represents a single command, we + # can process them + new_lines =3D [] + for line in lines: + new_lines.append(rewrap_line(line)) + + if in_place: + with open(filename, "w") as fh: + for line in new_lines: + print(line, file=3Dfh) + elif check: + orig =3D "".join(orig_lines) + new =3D "".join(new_lines) + if new !=3D orig: + diff =3D subprocess.Popen(["diff", "-u", filename, "-"], + stdin=3Dsubprocess.PIPE) + diff.communicate(input=3Dnew.encode('utf-8')) + + print("Incorrect line wrapping in $file", + file=3Dsys.stderr) + print("Use test-wrap-argv.py to wrap test data files", + file=3Dsys.stderr) + return False + else: + for line in new_lines: + print(line) + + return True + + +parser =3D argparse.ArgumentParser(description=3D'Test arg line wrapper') +parser.add_argument('--in-place', '-i', action=3D"store_true", + help=3D'modify files in-place') +parser.add_argument('--check', action=3D"store_true", + help=3D'check existing files only') +parser.add_argument('files', nargs=3D"+", + help=3D"filenames to check") +args =3D parser.parse_args() + +errs =3D False +for filename in args.files: + if not rewrap(filename, args.in_place, args.check): + errs =3D True + +if errs: + sys.exit(1) +sys.exit(0) diff --git a/tests/test-wrap-argv.pl b/tests/test-wrap-argv.pl deleted file mode 100755 index 7867e9d719..0000000000 --- a/tests/test-wrap-argv.pl +++ /dev/null @@ -1,174 +0,0 @@ -#!/usr/bin/env perl -# -# Copyright (C) 2015 Red Hat, Inc. -# -# This library is free software; you can redistribute it and/or -# modify it under the terms of the GNU Lesser General Public -# License as published by the Free Software Foundation; either -# version 2.1 of the License, or (at your option) any later version. -# -# This library is distributed in the hope that it will be useful, -# but WITHOUT ANY WARRANTY; without even the implied warranty of -# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU -# Lesser General Public License for more details. -# -# You should have received a copy of the GNU Lesser General Public -# License along with this library. If not, see -# . -# -# This script is intended to be passed a list of .args files, used -# to store command line ARGV for the test suites. It will reformat -# them such that there is at most one '-param value' on each line -# of the file. Parameter values that are longer than 80 chars will -# also be split. -# -# If --in-place is supplied as the first parameter of this script, -# the files will be changed in place. -# If --check is the first parameter, the script will return -# a non-zero value if a file is not wrapped correctly. -# Otherwise the rewrapped files are printed to the standard output. - -$in_place =3D 0; -$check =3D 0; - -if (@ARGV[0] eq "--in-place" or @ARGV[0] eq "-i") { - $in_place =3D 1; - shift @ARGV; -} elsif (@ARGV[0] eq "--check") { - $check =3D 1; - shift @ARGV; -} - -$ret =3D 0; -foreach my $file (@ARGV) { - if (&rewrap($file) < 0) { - $ret =3D 1; - } -} - -exit $ret; - -sub rewrap { - my $file =3D shift; - - # Read the original file - open FILE, "<", $file or die "cannot read $file: $!"; - my @orig_lines =3D ; - close FILE; - my @lines =3D @orig_lines; - foreach (@lines) { - # If there is a trailing '\' then kill the new line - if (/\\$/) { - chomp; - $_ =3D~ s/\\$//; - } - } - - # Skip empty files - return unless @lines; - - # Kill the last new line in the file - chomp @lines[$#lines]; - - # Reconstruct the master data by joining all lines - # and then split again based on the real desired - # newlines - @lines =3D split /\n/, join('', @lines); - - # Now each @lines represents a single command, we - # can process them - @lines =3D map { &rewrap_line($_) } @lines; - - if ($in_place) { - open FILE, ">", $file or die "cannot write $file: $!"; - foreach my $line (@lines) { - print FILE $line; - } - close FILE; - } elsif ($check) { - my $nl =3D join('', @lines); - my $ol =3D join('', @orig_lines); - unless ($nl eq $ol) { - open DIFF, "| diff -u $file -" or die "cannot run diff: $!"; - print DIFF $nl; - close DIFF; - - print STDERR "Incorrect line wrapping in $file\n"; - print STDERR "Use test-wrap-argv.pl to wrap test data files\n"; - return -1; - } - } else { - foreach my $line (@lines) { - print $line; - } - } - return 0; -} - -sub rewrap_line { - my $line =3D shift; - my @bits =3D split / /, join('', $line); - - # @bits contains env vars, then the command line - # and then the arguments - my @env; - my $cmd; - my @args; - - if ($bits[0] !~ /=3D/) { - $cmd =3D shift @bits; - } - - foreach my $bit (@bits) { - # If no command is defined yet, we must still - # have env vars - if (!defined $cmd) { - # Look for leading / to indicate command name - if ($bit =3D~ m,^/,) { - $cmd =3D $bit; - } else { - push @env, $bit; - } - } else { - # If there's a leading '-' then this is a new - # parameter, otherwise its a value for the prev - # parameter. - if ($bit =3D~ m,^-,) { - push @args, $bit; - } else { - $args[$#args] .=3D " " . $bit; - } - } - } - - # We might have to split line argument values... - @args =3D map { &rewrap_arg($_) } @args; - # Print env + command first - return join(" \\\n", @env, $cmd, @args), "\n"; -} - -sub rewrap_arg { - my $arg =3D shift; - my @ret; - my $max_len =3D 78; - - while (length($arg) > $max_len) { - my $split =3D rindex $arg, ",", $max_len; - if ($split =3D=3D -1) { - $split =3D rindex $arg, ":", $max_len; - } - if ($split =3D=3D -1) { - $split =3D rindex $arg, " ", $max_len; - } - if ($split =3D=3D -1) { - warn "cannot find nice place to split '$arg' below 80 chars\n"; - $split =3D $max_len - 1; - } - $split++; - - push @ret, substr $arg, 0, $split; - $arg =3D substr $arg, $split; - } - push @ret, $arg; - return join("\\\n", @ret); -} diff --git a/tests/testutils.c b/tests/testutils.c index da236c74a1..1acdc71fca 100644 --- a/tests/testutils.c +++ b/tests/testutils.c @@ -59,7 +59,7 @@ static size_t testCounter; static virBitmapPtr testBitmap; =20 char *progname; -static char *perl; +static char *python; =20 static int virTestUseTerminalColors(void) { @@ -399,15 +399,15 @@ virTestRewrapFile(const char *filename) virStringHasSuffix(filename, ".ldargs"))) return 0; =20 - if (!perl) { - fprintf(stderr, "cannot rewrap %s: unable to find perl in path", f= ilename); + if (!python) { + fprintf(stderr, "cannot rewrap %s: unable to find python in path",= filename); return -1; } =20 - if (virAsprintf(&script, "%s/test-wrap-argv.pl", abs_srcdir) < 0) + if (virAsprintf(&script, "%s/scripts/test-wrap-argv.py", abs_top_srcdi= r) < 0) goto cleanup; =20 - cmd =3D virCommandNewArgList(perl, script, "--in-place", filename, NUL= L); + cmd =3D virCommandNewArgList(python, script, "--in-place", filename, N= ULL); if (virCommandRun(cmd, NULL) < 0) goto cleanup; =20 @@ -895,8 +895,8 @@ int virTestMain(int argc, } } =20 - /* Find perl early because some tests override PATH */ - perl =3D virFindFileInPath("perl"); + /* Find python early because some tests override PATH */ + python =3D virFindFileInPath("python"); =20 ret =3D (func)(); =20 @@ -907,7 +907,7 @@ int virTestMain(int argc, fprintf(stderr, " %-3zu %s\n", testCounter, ret =3D=3D 0 ? "OK" : = "FAIL"); } virLogReset(); - VIR_FREE(perl); + VIR_FREE(python); return ret; } =20 --=20 2.21.0 -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list