From nobody Wed May 15 11:18:51 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=arm.com Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1677664446916236.81933175844028; Wed, 1 Mar 2023 01:54:06 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.503937.776364 (Exim 4.92) (envelope-from ) id 1pXJ9V-0007cQ-4c; Wed, 01 Mar 2023 09:53:37 +0000 Received: by outflank-mailman (output) from mailman id 503937.776364; Wed, 01 Mar 2023 09:53:37 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pXJ9U-0007c6-Vc; Wed, 01 Mar 2023 09:53:36 +0000 Received: by outflank-mailman (input) for mailman id 503937; Wed, 01 Mar 2023 09:53:35 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pXJ9T-0007ZS-KZ for xen-devel@lists.xenproject.org; Wed, 01 Mar 2023 09:53:35 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-sth1.inumbo.com (Halon) with ESMTP id ed71c45c-b816-11ed-96a0-2f268f93b82a; Wed, 01 Mar 2023 10:53:33 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 925E94B3; Wed, 1 Mar 2023 01:54:15 -0800 (PST) Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.195.25]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2DABD3F881; Wed, 1 Mar 2023 01:53:31 -0800 (PST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ed71c45c-b816-11ed-96a0-2f268f93b82a From: Luca Fancellu To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, wei.chen@arm.com, Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu Subject: [PATCH v2 1/2] xen/cppcheck: add a way to exclude files from the scan Date: Wed, 1 Mar 2023 09:53:19 +0000 Message-Id: <20230301095320.264301-2-luca.fancellu@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230301095320.264301-1-luca.fancellu@arm.com> References: <20230301095320.264301-1-luca.fancellu@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1677664448577100003 Content-Type: text/plain; charset="utf-8" Add a way to exclude files from the scan, in this way we can skip some findings from the report on those files that Xen doesn't own. To do that, introduce the exclude-list.json file under docs/misra, this file will be populated with relative path to the files/folder to be excluded. Introduce a new module, exclusion_file_list.py, to deal with the exclusion list file and use the new module in cppcheck_analysis.py to take a list of excluded paths to update the suppression list of cppcheck. Modified --suppress flag for cppcheck tool to remove unmatchedSuppression findings for those external file that are listed but for example not built for the current architecture. Add documentation for the file. Signed-off-by: Luca Fancellu --- Changes from v1: - Indentation fixed (Jan) --- --- docs/misra/exclude-list.json | 4 + docs/misra/exclude-list.rst | 44 +++++++++++ xen/scripts/xen_analysis/cppcheck_analysis.py | 20 ++++- .../xen_analysis/exclusion_file_list.py | 79 +++++++++++++++++++ 4 files changed, 145 insertions(+), 2 deletions(-) create mode 100644 docs/misra/exclude-list.json create mode 100644 docs/misra/exclude-list.rst create mode 100644 xen/scripts/xen_analysis/exclusion_file_list.py diff --git a/docs/misra/exclude-list.json b/docs/misra/exclude-list.json new file mode 100644 index 000000000000..1fb0ac67747b --- /dev/null +++ b/docs/misra/exclude-list.json @@ -0,0 +1,4 @@ +{ + "version": "1.0", + "content": [] +} diff --git a/docs/misra/exclude-list.rst b/docs/misra/exclude-list.rst new file mode 100644 index 000000000000..969539c46beb --- /dev/null +++ b/docs/misra/exclude-list.rst @@ -0,0 +1,44 @@ +.. SPDX-License-Identifier: CC-BY-4.0 + +Exclude file list for xen-analysis script +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The code analysis is performed on the Xen codebase for both MISRA checkers= and +static analysis checkers, there are some files however that needs to be re= moved +from the findings report because they are not owned by Xen and they must b= e kept +in sync with their origin (completely or even partially), hence we can't e= asily +fix findings or deviate from them. + +For this reason the file docs/misra/exclude-list.json is used to exclude e= very +entry listed in that file from the final report. +Currently only the cppcheck analysis will use this file. + +Here is an example of the exclude-list.json file:: + +|{ +| "version": "1.0", +| "content": [ +| { +| "rel_path": "relative/path/from/xen/file", +| "comment": "This file is originated from ..." +| }, +| { +| "rel_path": "relative/path/from/xen/folder/*", +| "comment": "This folder is a library" +| }, +| { +| "rel_path": "relative/path/from/xen/mem*.c", +| "comment": "memcpy.c, memory.c and memcmp.c are from the outs= ide" +| } +| ] +|} + +Here is an explanation of the fields inside an object of the "content" arr= ay: + - rel_path: it is the relative path from the Xen folder to the file/folde= r that + needs to be excluded from the analysis report, it can contain a wildcar= d to + match more than one file/folder at the time. This field is mandatory. + - comment: an optional comment to explain why the file is removed from the + analysis. + +To ease the review and the modifications of the entries, they shall be lis= ted in +alphabetical order referring to the rel_path field. diff --git a/xen/scripts/xen_analysis/cppcheck_analysis.py b/xen/scripts/xe= n_analysis/cppcheck_analysis.py index cc1f403d315e..e385e2c8f54a 100644 --- a/xen/scripts/xen_analysis/cppcheck_analysis.py +++ b/xen/scripts/xen_analysis/cppcheck_analysis.py @@ -1,7 +1,8 @@ #!/usr/bin/env python3 =20 import os, re, shutil -from . import settings, utils, cppcheck_report_utils +from . import settings, utils, cppcheck_report_utils, exclusion_file_list +from .exclusion_file_list import ExclusionFileListError =20 class GetMakeVarsPhaseError(Exception): pass @@ -50,6 +51,21 @@ def __generate_suppression_list(out_file): # header for cppcheck supplist_file.write("*:*generated/compiler-def.h\n") =20 + try: + exclusion_file =3D \ + "{}/docs/misra/exclude-list.json".format(settings.repo= _dir) + exclusion_list =3D \ + exclusion_file_list.load_exclusion_file_list(exclusion= _file) + except ExclusionFileListError as e: + raise CppcheckDepsPhaseError( + "Issue with reading file {}: {}".format(exclusion_file= , e) + ) + + # Append excluded files to the suppression list, using * as er= ror id + # to suppress every findings on that file + for path in exclusion_list: + supplist_file.write("*:{}\n".format(path)) + for entry in headers: filename =3D entry["file"] try: @@ -128,7 +144,7 @@ def generate_cppcheck_deps(): --relative-paths=3D{} --inline-suppr --suppressions-list=3D{}/suppression-list.txt - --suppress=3D'unmatchedSuppression:*generated/compiler-def.h' + --suppress=3D'unmatchedSuppression:*' --include=3D{}/include/xen/config.h -DCPPCHECK """.format(settings.outdir, CPPCHECK_BUILD_DIR, settings.xen_dir, diff --git a/xen/scripts/xen_analysis/exclusion_file_list.py b/xen/scripts/= xen_analysis/exclusion_file_list.py new file mode 100644 index 000000000000..4a47a90f5944 --- /dev/null +++ b/xen/scripts/xen_analysis/exclusion_file_list.py @@ -0,0 +1,79 @@ +#!/usr/bin/env python3 + +import os, glob, json +from . import settings + +class ExclusionFileListError(Exception): + pass + + +def __cppcheck_path_exclude_syntax(path): + # Prepending * to the relative path to match every path where the Xen + # codebase could be + path =3D "*" + path + + # Check if the path is to a folder without the wildcard at the end + if not (path.endswith(".c") or path.endswith(".h") or path.endswith("*= ")): + # The path is to a folder, if it doesn't have the final /, add it + if not path.endswith("/"): + path =3D path + "/" + # Since the path is a folder, add a wildcard to the end so that + # cppcheck will remove every issue related with this path + path =3D path + "*" + + return path + + +# Reads the exclusion file list and returns a list of relative path to be +# excluded. +def load_exclusion_file_list(input_file): + ret =3D [] + try: + with open(input_file, "rt") as handle: + content =3D json.load(handle) + entries =3D content['content'] + except json.JSONDecodeError as e: + raise ExclusionFileListError( + "JSON decoding error in file {}: {}".format(input_file, e) + ) + except KeyError: + raise ExclusionFileListError( + "Malformed JSON file: content field not found!" + ) + except Exception as e: + raise ExclusionFileListError( + "Can't open file {}: {}".format(input_file, e) + ) + + for entry in entries: + try: + path =3D entry['rel_path'] + except KeyError: + raise ExclusionFileListError( + "Malformed JSON entry: rel_path field not found!" + ) + abs_path =3D settings.xen_dir + "/" + path + check_path =3D [abs_path] + + # If the path contains wildcards, solve them + if '*' in abs_path: + check_path =3D glob.glob(abs_path) + + # Check that the path exists + for filepath_object in check_path: + if not os.path.exists(filepath_object): + raise ExclusionFileListError( + "Malformed path: {} refers to {} that does not exists" + .format(path, filepath_object) + ) + + if settings.analysis_tool =3D=3D "cppcheck": + path =3D __cppcheck_path_exclude_syntax(path) + else: + raise ExclusionFileListError( + "Unimplemented for {}!".format(settings.analysis_tool) + ) + + ret.append(path) + + return ret --=20 2.34.1 From nobody Wed May 15 11:18:51 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=arm.com Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1677664449643143.16661297525275; Wed, 1 Mar 2023 01:54:09 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.503938.776371 (Exim 4.92) (envelope-from ) id 1pXJ9V-0007nj-IZ; Wed, 01 Mar 2023 09:53:37 +0000 Received: by outflank-mailman (output) from mailman id 503938.776371; Wed, 01 Mar 2023 09:53:37 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pXJ9V-0007mV-DX; Wed, 01 Mar 2023 09:53:37 +0000 Received: by outflank-mailman (input) for mailman id 503938; Wed, 01 Mar 2023 09:53:36 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1pXJ9U-0007ZN-GF for xen-devel@lists.xenproject.org; Wed, 01 Mar 2023 09:53:36 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-flk1.inumbo.com (Halon) with ESMTP id ee00b70f-b816-11ed-a550-8520e6686977; Wed, 01 Mar 2023 10:53:34 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 21648152B; Wed, 1 Mar 2023 01:54:17 -0800 (PST) Received: from e125770.cambridge.arm.com (e125770.arm.com [10.1.195.25]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 96D4C3F881; Wed, 1 Mar 2023 01:53:32 -0800 (PST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: ee00b70f-b816-11ed-a550-8520e6686977 From: Luca Fancellu To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, wei.chen@arm.com, Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu , Michal Orzel Subject: [PATCH v2 2/2] xen/misra: add entries to exclude-list.json Date: Wed, 1 Mar 2023 09:53:20 +0000 Message-Id: <20230301095320.264301-3-luca.fancellu@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230301095320.264301-1-luca.fancellu@arm.com> References: <20230301095320.264301-1-luca.fancellu@arm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1677664450767100001 Content-Type: text/plain; charset="utf-8" Add entries to the exclude-list.json for those files that need to be excluded from the analysis scan. Signed-off-by: Luca Fancellu Signed-off-by: Michal Orzel Reviewed-by: Stefano Stabellini --- This list is originated from Michal's work here: https://patchwork.kernel.org/project/xen-devel/patch/20221116092032.4423-1-= michal.orzel@amd.com/#25123099 and changed to adapt to this task. Changes from v1: - updated list with new files from Stefano - add comment field for every entry (Jan) --- --- docs/misra/exclude-list.json | 199 ++++++++++++++++++++++++++++++++++- 1 file changed, 198 insertions(+), 1 deletion(-) diff --git a/docs/misra/exclude-list.json b/docs/misra/exclude-list.json index 1fb0ac67747b..ca1e2dd678ff 100644 --- a/docs/misra/exclude-list.json +++ b/docs/misra/exclude-list.json @@ -1,4 +1,201 @@ { "version": "1.0", - "content": [] + "content": [ + { + "rel_path": "arch/arm/arm64/cpufeature.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "arch/arm/arm64/insn.c", + "comment": "Imported on Linux, ignore for now" + }, + { + "rel_path": "arch/arm/arm64/lib/find_next_bit.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "arch/x86/acpi/boot.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "arch/x86/acpi/cpu_idle.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "arch/x86/acpi/cpufreq/cpufreq.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "arch/x86/acpi/cpuidle_menu.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "arch/x86/acpi/lib.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "arch/x86/cpu/amd.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "arch/x86/cpu/centaur.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "arch/x86/cpu/common.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "arch/x86/cpu/hygon.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "arch/x86/cpu/intel.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "arch/x86/cpu/intel_cacheinfo.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "arch/x86/cpu/mcheck/non-fatal.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "arch/x86/cpu/mtrr/*", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "arch/x86/cpu/mwait-idle.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "arch/x86/delay.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "arch/x86/dmi_scan.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "arch/x86/mpparse.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "arch/x86/srat.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "arch/x86/time.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "arch/x86/x86_64/mmconf-fam10h.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "common/bitmap.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "common/bunzip2.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "common/earlycpio.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "common/inflate.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "common/libfdt/*", + "comment": "External library" + }, + { + "rel_path": "common/livepatch_elf.c", + "comment": "Not in scope initially as it generates many violat= ions and it is not enabled in safety configurations" + }, + { + "rel_path": "common/lzo.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "common/lz4/decompress.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "common/radix-tree.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "common/ubsan/ubsan.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "common/un*.c", + "comment": "unlz4.c implementation by Yann Collet, the others = un* are from Linux, ignore for now" + }, + { + "rel_path": "common/xz/*", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "common/zstd/*", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "crypto/*", + "comment": "Origin is external and documented in crypto/README= .source" + }, + { + "rel_path": "drivers/acpi/apei/*", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "drivers/acpi/hwregs.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "drivers/acpi/numa.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "drivers/acpi/osl.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "drivers/acpi/tables.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "drivers/acpi/tables/*", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "drivers/acpi/utilities/*", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "drivers/video/font_*", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "lib/list-sort.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "lib/rbtree.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "lib/xxhash*.c", + "comment": "Imported from Linux, ignore for now" + }, + { + "rel_path": "xsm/flask/*", + "comment": "Not in scope initially as it generates many violat= ions and it is not enabled in safety configurations" + } + ] } --=20 2.34.1