From nobody Mon Sep 16 19:56:28 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=arm.com Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1684489624713924.5616237133653; Fri, 19 May 2023 02:47:04 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.536874.835600 (Exim 4.92) (envelope-from ) id 1pzwgu-0006k7-13; Fri, 19 May 2023 09:46:28 +0000 Received: by outflank-mailman (output) from mailman id 536874.835600; Fri, 19 May 2023 09:46:28 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pzwgt-0006k0-Uj; Fri, 19 May 2023 09:46:27 +0000 Received: by outflank-mailman (input) for mailman id 536874; Fri, 19 May 2023 09:46:27 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pzwgt-0006V3-84 for xen-devel@lists.xenproject.org; Fri, 19 May 2023 09:46:27 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-sth1.inumbo.com (Halon) with ESMTP id 057abb1a-f62a-11ed-b22d-6b7b168915f2; Fri, 19 May 2023 11:46:26 +0200 (CEST) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 4D94B2F4; Fri, 19 May 2023 02:47:10 -0700 (PDT) Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.199.1]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 1E6843F73F; Fri, 19 May 2023 02:46:24 -0700 (PDT) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 057abb1a-f62a-11ed-b22d-6b7b168915f2 From: Luca Fancellu To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, wei.chen@arm.com, Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu Subject: [PATCH 1/2] xen/misra: add diff-report.py tool Date: Fri, 19 May 2023 10:46:12 +0100 Message-Id: <20230519094613.2134153-2-luca.fancellu@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230519094613.2134153-1-luca.fancellu@arm.com> References: <20230519094613.2134153-1-luca.fancellu@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1684489626824100001 Content-Type: text/plain; charset="utf-8" Add a new tool, diff-report.py that can be used to make diff between reports generated by xen-analysis.py tool. Currently this tool supports the Xen cppcheck text report format in its operations. The tool prints every finding that is in the report passed with -r (check report) which is not in the report passed with -b (baseline). Signed-off-by: Luca Fancellu Acked-by: Stefano Stabellini Tested-by: Stefano Stabellini --- Changes from v1: - removed 2 method from class ReportEntry that landed there by a mistake on rebase. - Made the script compatible also with python2 (Stefano) --- xen/scripts/diff-report.py | 80 ++++++++++++++ .../xen_analysis/diff_tool/__init__.py | 0 .../xen_analysis/diff_tool/cppcheck_report.py | 44 ++++++++ xen/scripts/xen_analysis/diff_tool/debug.py | 40 +++++++ xen/scripts/xen_analysis/diff_tool/report.py | 100 ++++++++++++++++++ 5 files changed, 264 insertions(+) create mode 100755 xen/scripts/diff-report.py create mode 100644 xen/scripts/xen_analysis/diff_tool/__init__.py create mode 100644 xen/scripts/xen_analysis/diff_tool/cppcheck_report.py create mode 100644 xen/scripts/xen_analysis/diff_tool/debug.py create mode 100644 xen/scripts/xen_analysis/diff_tool/report.py diff --git a/xen/scripts/diff-report.py b/xen/scripts/diff-report.py new file mode 100755 index 000000000000..f97cb2355cc3 --- /dev/null +++ b/xen/scripts/diff-report.py @@ -0,0 +1,80 @@ +#!/usr/bin/env python3 + +from __future__ import print_function +import os +import sys +from argparse import ArgumentParser +from xen_analysis.diff_tool.cppcheck_report import CppcheckReport +from xen_analysis.diff_tool.debug import Debug +from xen_analysis.diff_tool.report import ReportError + + +def log_info(text, end=3D'\n'): + # type: (str, str) -> None + global args + global file_out + + if (args.verbose): + print(text, end=3Dend, file=3Dfile_out) + + +def main(argv): + # type: (list) -> None + global args + global file_out + + parser =3D ArgumentParser(prog=3D"diff-report.py") + parser.add_argument("-b", "--baseline", required=3DTrue, type=3Dstr, + help=3D"Path to the baseline report.") + parser.add_argument("--debug", action=3D'store_true', + help=3D"Produce intermediate reports during operat= ions.") + parser.add_argument("-o", "--out", default=3D"stdout", type=3Dstr, + help=3D"Where to print the tool output. Default is= " + "stdout") + parser.add_argument("-r", "--report", required=3DTrue, type=3Dstr, + help=3D"Path to the 'check report', the one checke= d " + "against the baseline.") + parser.add_argument("-v", "--verbose", action=3D'store_true', + help=3D"Print more informations during the run.") + + args =3D parser.parse_args() + + if args.out =3D=3D "stdout": + file_out =3D sys.stdout + else: + try: + file_out =3D open(args.out, "wt") + except OSError as e: + print("ERROR: Issue opening file {}: {}".format(args.out, e)) + sys.exit(1) + + debug =3D Debug(args) + + try: + baseline_path =3D os.path.realpath(args.baseline) + log_info("Loading baseline report {}".format(baseline_path), "") + baseline =3D CppcheckReport(baseline_path) + baseline.parse() + debug.debug_print_parsed_report(baseline) + log_info(" [OK]") + new_rep_path =3D os.path.realpath(args.report) + log_info("Loading check report {}".format(new_rep_path), "") + new_rep =3D CppcheckReport(new_rep_path) + new_rep.parse() + debug.debug_print_parsed_report(new_rep) + log_info(" [OK]") + except ReportError as e: + print("ERROR: {}".format(e)) + sys.exit(1) + + output =3D new_rep - baseline + print(output, end=3D"", file=3Dfile_out) + + if len(output) > 0: + sys.exit(1) + + sys.exit(0) + + +if __name__ =3D=3D "__main__": + main(sys.argv[1:]) diff --git a/xen/scripts/xen_analysis/diff_tool/__init__.py b/xen/scripts/x= en_analysis/diff_tool/__init__.py new file mode 100644 index 000000000000..e69de29bb2d1 diff --git a/xen/scripts/xen_analysis/diff_tool/cppcheck_report.py b/xen/sc= ripts/xen_analysis/diff_tool/cppcheck_report.py new file mode 100644 index 000000000000..e7e80a9dde84 --- /dev/null +++ b/xen/scripts/xen_analysis/diff_tool/cppcheck_report.py @@ -0,0 +1,44 @@ +#!/usr/bin/env python3 + +import re +from .report import Report, ReportError + + +class CppcheckReport(Report): + def __init__(self, report_path): + # type: (str) -> None + super(CppcheckReport, self).__init__(report_path) + # This matches a string like: + # path/to/file.c(,): + # and captures file name path and line number + # the last capture group is used for text substitution in __str__ + self.__report_entry_regex =3D re.compile(r'^(.*)\((\d+)(,\d+\):.*)= $') + + def parse(self): + # type: () -> None + report_path =3D self.get_report_path() + try: + with open(report_path, "rt") as infile: + report_lines =3D infile.readlines() + except OSError as e: + raise ReportError("Issue with reading file {}: {}" + .format(report_path, e)) + for line in report_lines: + entry =3D self.__report_entry_regex.match(line) + if entry and entry.group(1) and entry.group(2): + file_path =3D entry.group(1) + line_number =3D int(entry.group(2)) + self.add_entry(file_path, line_number, line) + else: + raise ReportError("Malformed report entry in file {}:\n{}" + .format(report_path, line)) + + def __str__(self): + # type: () -> str + ret =3D "" + for entry in self.to_list(): + ret +=3D re.sub(self.__report_entry_regex, + r'{}({}\3'.format(entry.file_path, + entry.line_number), + entry.text) + return ret diff --git a/xen/scripts/xen_analysis/diff_tool/debug.py b/xen/scripts/xen_= analysis/diff_tool/debug.py new file mode 100644 index 000000000000..65cca2464110 --- /dev/null +++ b/xen/scripts/xen_analysis/diff_tool/debug.py @@ -0,0 +1,40 @@ +#!/usr/bin/env python3 + +from __future__ import print_function +import os +from .report import Report + + +class Debug: + def __init__(self, args): + self.args =3D args + + def __get_debug_out_filename(self, path, type): + # type: (str, str) -> str + # Take basename + file_name =3D os.path.basename(path) + # Split in name and extension + file_name =3D os.path.splitext(file_name) + if self.args.out !=3D "stdout": + out_folder =3D os.path.dirname(self.args.out) + else: + out_folder =3D "./" + dbg_report_path =3D out_folder + file_name[0] + type + file_name[1] + + return dbg_report_path + + def __debug_print_report(self, report, type): + # type: (Report, str) -> None + report_name =3D self.__get_debug_out_filename(report.get_report_pa= th(), + type) + try: + with open(report_name, "wt") as outfile: + print(report, end=3D"", file=3Doutfile) + except OSError as e: + print("ERROR: Issue opening file {}: {}".format(report_name, e= )) + + def debug_print_parsed_report(self, report): + # type: (Report) -> None + if not self.args.debug: + return + self.__debug_print_report(report, ".parsed") diff --git a/xen/scripts/xen_analysis/diff_tool/report.py b/xen/scripts/xen= _analysis/diff_tool/report.py new file mode 100644 index 000000000000..4a303d61b3ea --- /dev/null +++ b/xen/scripts/xen_analysis/diff_tool/report.py @@ -0,0 +1,100 @@ +#!/usr/bin/env python3 + +import os + + +class ReportError(Exception): + pass + + +class Report(object): + class ReportEntry: + def __init__(self, file_path, line_number, entry_text, line_id): + # type: (str, int, list, int) -> None + if not isinstance(line_number, int) or \ + not isinstance(line_id, int): + raise ReportError("ReportEntry constructor wrong type args= ") + self.file_path =3D file_path + self.line_number =3D line_number + self.text =3D entry_text + self.line_id =3D line_id + + def __init__(self, report_path): + # type: (str) -> None + self.__entries =3D {} + self.__path =3D report_path + self.__last_line_order =3D 0 + + def parse(self): + # type: () -> None + raise ReportError("Please create a specialised class from 'Report'= .") + + def get_report_path(self): + # type: () -> str + return self.__path + + def get_report_entries(self): + # type: () -> dict + return self.__entries + + def add_entry(self, entry_path, entry_line_number, entry_text): + # type: (str, int, str) -> None + entry =3D Report.ReportEntry(entry_path, entry_line_number, entry_= text, + self.__last_line_order) + if entry_path in self.__entries.keys(): + self.__entries[entry_path].append(entry) + else: + self.__entries[entry_path] =3D [entry] + self.__last_line_order +=3D 1 + + def to_list(self): + # type: () -> list + report_list =3D [] + for _, entries in self.__entries.items(): + for entry in entries: + report_list.append(entry) + + report_list.sort(key=3Dlambda x: x.line_id) + return report_list + + def __str__(self): + # type: () -> str + ret =3D "" + for entry in self.to_list(): + ret +=3D entry.file_path + ":" + entry.line_number + ":" + ent= ry.text + + return ret + + def __len__(self): + # type: () -> int + return len(self.to_list()) + + def __sub__(self, report_b): + # type: (Report) -> Report + if self.__class__ !=3D report_b.__class__: + raise ReportError("Diff of different type of report!") + + filename, file_extension =3D os.path.splitext(self.__path) + diff_report =3D self.__class__(filename + ".diff" + file_extension) + # Put in the diff report only records of this report that are not + # present in the report_b. + for file_path, entries in self.__entries.items(): + rep_b_entries =3D report_b.get_report_entries() + if file_path in rep_b_entries.keys(): + # File path exists in report_b, so check what entries of t= hat + # file path doesn't exist in report_b and add them to the = diff + rep_b_entries_num =3D [ + x.line_number for x in rep_b_entries[file_path] + ] + for entry in entries: + if entry.line_number not in rep_b_entries_num: + diff_report.add_entry(file_path, entry.line_number, + entry.text) + else: + # File path doesn't exist in report_b, so add every entry + # of that file path to the diff + for entry in entries: + diff_report.add_entry(file_path, entry.line_number, + entry.text) + + return diff_report --=20 2.34.1