From nobody Fri May 17 01:43:50 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=arm.com Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1670418487025562.749808829015; Wed, 7 Dec 2022 05:08:07 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.456299.714070 (Exim 4.92) (envelope-from ) id 1p2u8t-0004Rz-O1; Wed, 07 Dec 2022 13:07:19 +0000 Received: by outflank-mailman (output) from mailman id 456299.714070; Wed, 07 Dec 2022 13:07:19 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p2u8t-0004Rs-LB; Wed, 07 Dec 2022 13:07:19 +0000 Received: by outflank-mailman (input) for mailman id 456299; Wed, 07 Dec 2022 13:07:18 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p2u8s-0004EJ-1u for xen-devel@lists.xenproject.org; Wed, 07 Dec 2022 13:07:18 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-sth1.inumbo.com (Halon) with ESMTP id 11f907cb-7630-11ed-91b6-6bf2151ebd3b; Wed, 07 Dec 2022 14:07:15 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6394AD6E; Wed, 7 Dec 2022 05:07:21 -0800 (PST) Received: from e125770.cambridge.arm.com (e125770.cambridge.arm.com [10.1.195.16]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3CCB63F73D; Wed, 7 Dec 2022 05:07:13 -0800 (PST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 11f907cb-7630-11ed-91b6-6bf2151ebd3b From: Luca Fancellu To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, wei.chen@arm.com, Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu Subject: [PATCH v3 1/5] xen/scripts: add xen-analysis.py for coverity and eclair analysis Date: Wed, 7 Dec 2022 13:07:00 +0000 Message-Id: <20221207130704.20573-2-luca.fancellu@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221207130704.20573-1-luca.fancellu@arm.com> References: <20221207130704.20573-1-luca.fancellu@arm.com> X-ZM-MESSAGEID: 1670418487385100003 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Add new script for coverity/eclair analysis tool that will enable the procedure to suppress findings when these tool are used. The procedure is documented in docs/misra/documenting-violations.rst and the script is documented in docs/misra/xen-static-analysis.rst. Add in docs/misra/ the files safe.json and false-positive-{coverity,eclair}.json that are JSON files containing the data structures for the justifications, they are used by the analysis script to link the Xen tags to the proprietary tool comment. Add docs/misra/documenting-violations.rst to explain how to add justifications. Add docs/misra/xen-static-analysis.rst to explain how to use the script to analyse Xen. Add analysis artifacts files to .gitignore. Signed-off-by: Luca Fancellu Reviewed-by: Stefano Stabellini --- v3 changes: - improved help content in the script (Stefano) - used utils.invoke_command from this patch (Stefano) v2 changes: - no change --- --- .gitignore | 1 + docs/misra/documenting-violations.rst | 191 +++++++++++++++++++ docs/misra/false-positive-coverity.json | 12 ++ docs/misra/false-positive-eclair.json | 12 ++ docs/misra/safe.json | 11 ++ docs/misra/xen-static-analysis.rst | 54 ++++++ xen/scripts/xen-analysis.py | 31 +++ xen/scripts/xen_analysis/__init__.py | 0 xen/scripts/xen_analysis/generic_analysis.py | 87 +++++++++ xen/scripts/xen_analysis/settings.py | 111 +++++++++++ xen/scripts/xen_analysis/tag_database.py | 109 +++++++++++ xen/scripts/xen_analysis/utils.py | 56 ++++++ 12 files changed, 675 insertions(+) create mode 100644 docs/misra/documenting-violations.rst create mode 100644 docs/misra/false-positive-coverity.json create mode 100644 docs/misra/false-positive-eclair.json create mode 100644 docs/misra/safe.json create mode 100644 docs/misra/xen-static-analysis.rst create mode 100755 xen/scripts/xen-analysis.py create mode 100644 xen/scripts/xen_analysis/__init__.py create mode 100644 xen/scripts/xen_analysis/generic_analysis.py create mode 100644 xen/scripts/xen_analysis/settings.py create mode 100644 xen/scripts/xen_analysis/tag_database.py create mode 100644 xen/scripts/xen_analysis/utils.py diff --git a/.gitignore b/.gitignore index ea3243af9dde..f5a66f6194dd 100644 --- a/.gitignore +++ b/.gitignore @@ -10,6 +10,7 @@ *.c.cppcheck *.opic *.a +*.safparse *.so *.so.[0-9]* *.bin diff --git a/docs/misra/documenting-violations.rst b/docs/misra/documenting= -violations.rst new file mode 100644 index 000000000000..1d23447556d2 --- /dev/null +++ b/docs/misra/documenting-violations.rst @@ -0,0 +1,191 @@ +.. SPDX-License-Identifier: CC-BY-4.0 + +Documenting violations +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Static analysers are used on the Xen codebase for both static analysis and= MISRA +compliance. +There might be the need to suppress some findings instead of fixing them a= nd +many tools permit the usage of in-code comments that suppress findings so = that +they are not shown in the final report. + +Xen includes a tool capable of translating a specific comment used in its +codebase to the right proprietary in-code comment understandable by the se= lected +analyser that suppress its finding. + +In the Xen codebase, these tags will be used to document and suppress find= ings: + + - SAF-X-safe: This tag means that the next line of code contains a findin= g, but + the non compliance to the checker is analysed and demonstrated to be sa= fe. + - SAF-X-false-positive-: This tag means that the next line of code + contains a finding, but the finding is a bug of the tool. + +SAF stands for Static Analyser Finding, the X is a placeholder for a posit= ive +number that starts from zero, the number after SAF- shall be incremental a= nd +unique, base ten notation and without leading zeros. + +Entries in the database shall never be removed, even if they are not used +anymore in the code (if a patch is removing or modifying the faulty line). +This is to make sure that numbers are not reused which could lead to confl= icts +with old branches or misleading justifications. + +An entry can be reused in multiple places in the code to suppress a findin= g if +and only if the justification holds for the same non-compliance to the cod= ing +standard. + +An orphan entry, that is an entry who was justifying a finding in the code= , but +later that code was removed and there is no other use of that entry in the= code, +can be reused as long as the justification for the finding holds. This is = done +to avoid the allocation of a new entry with exactly the same justification= , that +would lead to waste of space and maintenance issues of the database. + +The files where to store all the justifications are in xen/docs/misra/ and= are +named as safe.json and false-positive-.json, they have JSON format, = each +one has a different justification schema which shares some fields. + +Here is an example to add a new justification in safe.json:: + +|{ +| "version": "1.0", +| "content": [ +| { +| "id": "SAF-0-safe", +| "analyser": { +| "coverity": "misra_c_2012_rule_20_7_violation", +| "eclair": "MC3R1.R20.7" +| }, +| "name": "R20.7 C macro parameters not used as expression", +| "text": "The macro parameters used in this [...]" +| }, +| { +| "id": "SAF-1-safe", +| "analyser": {}, +| "name": "Sentinel", +| "text": "Next ID to be used" +| } +| ] +|} + +To document a finding in safe.json, just add another block {[...]} before = the +sentinel block, using the id contained in the sentinel block and increment= by +one the number contained in the id of the sentinel block. + +Here is an explanation of the fields inside an object of the "content" arr= ay: + - id: it is a unique string that is used to refer to the finding, many fi= nding + can be tagged with the same id, if the justification holds for any appl= ied + case. + It tells the tool to substitute a Xen in-code comment having this struc= ture: + /* SAF-0-safe [...] \*/ + - analyser: it is an object containing pair of key-value strings, the key= is + the analyser, so it can be coverity or eclair, the value is the proprie= tary + id corresponding on the finding, for example when coverity is used as + analyser, the tool will translate the Xen in-code coment in this way: + /* SAF-0-safe [...] \*/ -> /* coverity[misra_c_2012_rule_20_7_violation= ] \*/ + if the object doesn't have a key-value, then the corresponding in-code + comment won't be translated. + - name: a simple name for the finding + - text: a proper justification to turn off the finding. + + +Here is an example to add a new justification in false-positive-.jso= n:: + +|{ +| "version": "1.0", +| "content": [ +| { +| "id": "SAF-0-false-positive-", +| "violation-id": "", +| "tool-version": "", +| "name": "R20.7 [...]", +| "text": "[...]" +| }, +| { +| "id": "SAF-1-false-positive-", +| "violation-id": "", +| "tool-version": "", +| "name": "Sentinel", +| "text": "Next ID to be used" +| } +| ] +|} + +To document a finding in false-positive-.json, just add another block +{[...]} before the sentinel block, using the id contained in the sentinel = block +and increment by one the number contained in the id of the sentinel block. + +Here is an explanation of the fields inside an object of the "content" arr= ay: + - id: it has the same meaning as in the "safe" justification schema. + It tells the tool to substitute a Xen in-code comment having this struc= ture: + /* SAF-0-false-positive- [...] \*/ + - violation-id: its value is a string containing the proprietary id + corresponding to the finding, for example when is coverity, the = Xen + tool will translate the Xen in-code coment in this way: + /* SAF-0-false-positive-coverity [...] \*/ -> /* coverity[misra_c_2012_= rule_20_7_violation] \*/ + if the object doesn't have a value, then the corresponding in-code comm= ent + won't be translated. + - tool-version: the version of the tool affected by the false positive, i= f it + is discovered in more than one version, this string can be a range + (eg. 2.7 - 3.0) + - name, text: they have the same meaning as in the "safe" justification s= chema. + + +Justification example +--------------------- + +Here an example of the usage of the in-code comment tags to suppress a fin= ding +for the Rule 8.6: + +Eclair reports it in its web report, file xen/include/xen/kernel.h, line 6= 8: + +| MC3R1.R8.6 for program 'xen/xen-syms', variable '_start' has no definiti= on + +Also coverity reports it, here is an extract of the finding: + +| xen/include/xen/kernel.h:68: +| 1. misra_c_2012_rule_8_6_violation: Function "_start" is declared but ne= ver + defined. + +The analysers are complaining because we have this in xen/include/xen/kern= el.h +at line 68:: + +| extern char _start[], _end[], start[]; + +Those are symbols exported by the linker, hence we will need to have a pro= per +deviation for this finding. + +We will prepare our entry in the safe.json database:: + +|{ +| "version": "1.0", +| "content": [ +| { +| [...] +| }, +| { +| "id": "SAF-1-safe", +| "analyser": { +| "eclair": "MC3R1.R8.6", +| "coverity": "misra_c_2012_rule_8_6_violation" +| }, +| "name": "Rule 8.6: linker script defined symbols", +| "text": "It is safe to declare this symbol because it is defi= ned in the linker script." +| }, +| { +| "id": "SAF-2-safe", +| "analyser": {}, +| "name": "Sentinel", +| "text": "Next ID to be used" +| } +| ] +|} + +And we will use the proper tag above the violation line:: + +| /* SAF-1-safe R8.6 linker defined symbols */ +| extern char _start[], _end[], start[]; + +This entry will fix also the violation on _end and start, because they are= on +the same line and the same "violation ID". + +Also, the same tag can be used on other symbols from the linker that are +declared in the codebase, because the justification holds for them too. diff --git a/docs/misra/false-positive-coverity.json b/docs/misra/false-pos= itive-coverity.json new file mode 100644 index 000000000000..462448414f80 --- /dev/null +++ b/docs/misra/false-positive-coverity.json @@ -0,0 +1,12 @@ +{ + "version": "1.0", + "content": [ + { + "id": "SAF-0-false-positive-coverity", + "violation-id": "", + "tool-version": "", + "name": "Sentinel", + "text": "Next ID to be used" + } + ] +} diff --git a/docs/misra/false-positive-eclair.json b/docs/misra/false-posit= ive-eclair.json new file mode 100644 index 000000000000..1d6ea5d7f045 --- /dev/null +++ b/docs/misra/false-positive-eclair.json @@ -0,0 +1,12 @@ +{ + "version": "1.0", + "content": [ + { + "id": "SAF-0-false-positive-eclair", + "violation-id": "", + "tool-version": "", + "name": "Sentinel", + "text": "Next ID to be used" + } + ] +} diff --git a/docs/misra/safe.json b/docs/misra/safe.json new file mode 100644 index 000000000000..e079d3038120 --- /dev/null +++ b/docs/misra/safe.json @@ -0,0 +1,11 @@ +{ + "version": "1.0", + "content": [ + { + "id": "SAF-0-safe", + "analyser": {}, + "name": "Sentinel", + "text": "Next ID to be used" + } + ] +} diff --git a/docs/misra/xen-static-analysis.rst b/docs/misra/xen-static-ana= lysis.rst new file mode 100644 index 000000000000..5b886474d4a0 --- /dev/null +++ b/docs/misra/xen-static-analysis.rst @@ -0,0 +1,54 @@ +.. SPDX-License-Identifier: CC-BY-4.0 + +Xen static analysis +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The Xen codebase integrates some scripts and tools that helps the develope= r to +perform static analysis of the code, currently Xen supports three analysis= tool +that are eclair, coverity and cppcheck. +The Xen tree has a script (xen-analysis.py) available to ease the analysis +process and it integrates a way to suppress findings on these tools (only = Eclair +and Coverity are currently supported by the script), please check the +documenting-violation.rst document to know more about it. + +Analyse Xen with Coverity or Eclair +----------------------------------- + +The xen-analysis.py script has two arguments to select which tool is used = for +the analysis: + + - xen-analysis.py --run-coverity -- [optional make arguments] + - xen-analysis.py --run-eclair -- [optional make arguments] + +For example when using Coverity to analyse a Xen build obtained by passing= these +arguments to the make system: XEN_TARGET_ARCH=3Darm64 +CROSS_COMPILE=3Daarch64-linux-gnu-, the optional make arguments passed to +xen-analysis.py must be the same and the command below should be passed to +Coverity in its build phase: + + - xen-analysis.py --run-coverity -- XEN_TARGET_ARCH=3Darm64 + CROSS_COMPILE=3Daarch64-linux-gnu- + +Which tells to the script to prepare the codebase for an analysis by Cover= ity +and forwards the make arguments to the make build invocation. + +When invoking the script, the procedure below will be followed: + + 1. Find which files among \*.c and \*.h has any in-code comment as + /* SAF-X-[...] \*/, the meaning of these comments is explained in + documenting-violation.rst. + Save the files obtained as .safparse and generate files w= here + the special in-code comments above are substituted with the proprietary + in-code comment used by the selected analysis tool. The safe.json and + false-positive-.json text file database are used to link each Xe= n tag + to the right proprietary in-code comment. + 2. Now Xen compilation starts using every su= pplied + at the script invocation. Coverity and Eclair are capable of intercept= ing + the compiler running from make to perform their analysis without + instrumenting the makefile. + 3. As final step every .safparse file are reverted back as a= nd + every artifact related to the analysis will be cleaned. + This step is performed even in case any of the previous step fail, to = skip + this step, call the script adding the --no-clean argument, but before + running again the script, call it with the --clean-only argument, that= will + execute only this cleaning step. diff --git a/xen/scripts/xen-analysis.py b/xen/scripts/xen-analysis.py new file mode 100755 index 000000000000..b5d9ef1862c9 --- /dev/null +++ b/xen/scripts/xen-analysis.py @@ -0,0 +1,31 @@ +#!/usr/bin/env python3 + +import sys +from xen_analysis import settings, generic_analysis +from xen_analysis.generic_analysis import * + + +def main(argv): + ret_code =3D 0 + settings.parse_commandline(argv) + try: + if settings.step_parse_tags: + generic_analysis.parse_xen_tags() + if settings.step_build_xen: + generic_analysis.build_xen() + except (ParseTagPhaseError, BuildPhaseError) as e: + print("ERROR: {}".format(e)) + if hasattr(e, "errorcode"): + ret_code =3D e.errorcode + finally: + if settings.step_clean_analysis: + e =3D generic_analysis.clean_analysis_artifacts() + if e: + print("ERROR: {}".format(e)) + ret_code =3D 1 + + sys.exit(ret_code) + + +if __name__ =3D=3D "__main__": + main(sys.argv[1:]) diff --git a/xen/scripts/xen_analysis/__init__.py b/xen/scripts/xen_analysi= s/__init__.py new file mode 100644 index 000000000000..e69de29bb2d1 diff --git a/xen/scripts/xen_analysis/generic_analysis.py b/xen/scripts/xen= _analysis/generic_analysis.py new file mode 100644 index 000000000000..372ff3566442 --- /dev/null +++ b/xen/scripts/xen_analysis/generic_analysis.py @@ -0,0 +1,87 @@ +#!/usr/bin/env python3 + +import os, subprocess +from . import settings, utils, tag_database + +class ParseTagPhaseError(Exception): + pass + +class BuildPhaseError(Exception): + pass + +class CleanPhaseError(Exception): + pass + + +def parse_xen_tags(): + # Load the database for the Xen tags + subs_list =3D tag_database.load_tag_database( + settings.analysis_tool, + [settings.repo_dir + "/docs/misra/safe.json"] + ) + subs_list =3D tag_database.load_tag_database( + settings.analysis_tool, + [settings.repo_dir + "/docs/misra/false-positive-{}.json" + .format(settings.analysis_tool)], + subs_list, + "false-positive" + ) + + # Create outdir if it doesn't exists + os.makedirs(settings.outdir, exist_ok=3DTrue) + + # The following lambda function will return a file if it contains line= s with + # a comment containing "SAF--{safe|false-positive-}" on a + # single line. + grep_action =3D lambda x: utils.grep(x, + tag_database.get_xen_tag_comment_regex( + settings.analysis_= tool) + ) + # Look for a list of .h/.c files that matches the condition above + parse_file_list =3D utils.recursive_find_file(settings.xen_dir, r'.*\.= [ch]$', + grep_action) + + for entry in parse_file_list: + file =3D entry["file"] + bkp_file =3D file + ".safparse" + if os.path.isfile(bkp_file): + raise ParseTagPhaseError( + "Found {}, please check the integrity of {}" + .format(bkp_file,file) + ) + os.rename(file, bkp_file) + time_bkp_file =3D os.stat(bkp_file) + # Create from .safparse but with the Xen tag parsed + tag_database.substitute_tags(settings.analysis_tool, bkp_file, ent= ry, + subs_list) + # Set timestamp for file equal to bkp_file, so that if the file is + # modified during the process by the user, we can catch it + os.utime(file, (time_bkp_file.st_atime, time_bkp_file.st_mtime)) + + +def build_xen(): + utils.invoke_command( + "make -C {} {} build" + .format(settings.xen_dir, settings.make_forward_args), + False, BuildPhaseError, + "Build error occured when running:\n{}" + ) + + +def clean_analysis_artifacts(): + safparse_files =3D utils.recursive_find_file(settings.xen_dir, + r'.*.safparse$') + for original_file in safparse_files: + # This commands strips the .safparse extension, leaving + parsed_file_path =3D os.path.splitext(original_file)[0] + mtime_original_file =3D os.stat(original_file).st_mtime + mtime_parsed_file =3D os.stat(parsed_file_path).st_mtime + if mtime_original_file !=3D mtime_parsed_file: + return CleanPhaseError( + "The file {} was modified during the analysis " + "procedure, it is impossible now to restore from the " + "content of {}, please handle it manually" + .format(parsed_file_path, original_file) + ) + # Replace .safparse to + os.replace(original_file, parsed_file_path) diff --git a/xen/scripts/xen_analysis/settings.py b/xen/scripts/xen_analysi= s/settings.py new file mode 100644 index 000000000000..e8d474eaf378 --- /dev/null +++ b/xen/scripts/xen_analysis/settings.py @@ -0,0 +1,111 @@ +#!/usr/bin/env python3 + +import sys, re, os + +module_dir =3D os.path.dirname(os.path.realpath(__file__)) +xen_dir =3D os.path.realpath(module_dir + "/../..") +repo_dir =3D os.path.realpath(xen_dir + "/..") +tools_dir =3D os.path.realpath(xen_dir + "/tools") + +step_parse_tags =3D True +step_build_xen =3D True +step_clean_analysis =3D True + +target_build =3D False +target_clean =3D False + +analysis_tool =3D "" +make_forward_args =3D "" +outdir =3D xen_dir + + +def help(): + msg=3D""" +Usage: {} [OPTION] ... [-- [make arguments]] + +This script runs the analysis on the Xen codebase. + +The phases for the analysis are , , + +Depending on the options below, only some phases will run: + +: tags, build, clean +--build-only: build +--clean-only: clean +--no-build: tags, clean +--no-clean: tags, build + +--no-build/--no-clean can be passed together to avoid both clean and build +phases. +Tags and build phases need to specify --run-coverity or --run-eclair. + +Options: + --build-only Run only the commands to build Xen with the optional make + arguments passed to the script + --clean-only Run only the commands to clean the analysis artifacts + -h, --help Print this help + --no-build Skip the build Xen phase + --no-clean Don\'t clean the analysis artifacts on exit + --run-coverity Run the analysis for the Coverity tool + --run-eclair Run the analysis for the Eclair tool +""" + print(msg.format(sys.argv[0])) + + +def parse_commandline(argv): + global analysis_tool + global make_forward_args + global outdir + global step_parse_tags + global step_build_xen + global step_clean_analysis + global target_build + global target_clean + forward_to_make =3D False + for option in argv: + if forward_to_make: + # Intercept outdir + outdir_regex =3D re.match("^O=3D(.*)$", option) + if outdir_regex: + outdir =3D outdir_regex.group(1) + # Forward any make arguments + make_forward_args =3D make_forward_args + " " + option + elif option =3D=3D "--build-only": + target_build =3D True + elif option =3D=3D "--clean-only": + target_clean =3D True + elif (option =3D=3D "--help") or (option =3D=3D "-h"): + help() + sys.exit(0) + elif option =3D=3D "--no-build": + step_build_xen =3D False + elif option =3D=3D "--no-clean": + step_clean_analysis =3D False + elif (option =3D=3D "--run-coverity") or (option =3D=3D "--run-ecl= air"): + analysis_tool =3D option[6:] + elif option =3D=3D "--": + forward_to_make =3D True + else: + print("Invalid option: {}".format(option)) + help() + sys.exit(1) + + if target_build and target_clean: + print("--build-only is not compatible with --clean-only argument.") + sys.exit(1) + + if target_clean: + step_parse_tags =3D False + step_build_xen =3D False + step_clean_analysis =3D True + return + + if analysis_tool =3D=3D "": + print("Please specify one analysis tool.") + help() + sys.exit(1) + + if target_build: + step_parse_tags =3D False + step_build_xen =3D True + step_clean_analysis =3D False diff --git a/xen/scripts/xen_analysis/tag_database.py b/xen/scripts/xen_ana= lysis/tag_database.py new file mode 100644 index 000000000000..ca374bbb62dd --- /dev/null +++ b/xen/scripts/xen_analysis/tag_database.py @@ -0,0 +1,109 @@ +#!/usr/bin/env python3 + +import re, json + +class TagDatabaseError(Exception): + pass + +# This is the dictionary for the rules that translates to proprietary comm= ents: +# - cppcheck: /* cppcheck-suppress[id] */ +# - coverity: /* coverity[id] */ +# - eclair: /* -E> hide id 1 "" */ +# Add entries to support more analyzers +tool_syntax =3D { + "cppcheck":"cppcheck-suppress[VID]", + "coverity":"coverity[VID]", + "eclair":"-E> hide VID 1 \"\"" +} + + +def get_xen_tag_index_type_regex(tool): + return r'^SAF-(\d+)-(safe|false-positive-' + tool + ')$' + + +def get_xen_tag_comment_regex(tool): + return r'^[ \t]*/\* +(SAF-\d+-(?:safe|false-positive-' + tool + ')).*\= */$' + + +# Returns a data structure containing dictionaries for safe and false-posi= tive-* +# Xen tags, the key is the unique index of the tag and the content is the +# proprietary in-code comment to be used when the tag is found in the code= base +def load_tag_database(tool, input_files, data_struct =3D None, schema =3D = "safe"): + ret =3D data_struct if data_struct is not None else { + "safe": {}, + "false-positive-" + tool: {} + } + database =3D [] + + # Open all input files + for file in input_files: + try: + with open(file, "rt") as handle: + content =3D json.load(handle) + database =3D database + content['content'] + except json.JSONDecodeError as e: + raise TagDatabaseError("JSON decoding error in file {}: {}" + .format(file, e)) + except Exception as e: + raise TagDatabaseError("Can't open file {}: {}" + .format(file, e)) + + for entry in database: + # If the false-positive schema is used, check the proprietary id i= n the + # 'violation-id' field, otherwise rely on the "safe" schema. + if schema =3D=3D "false-positive": + proprietary_id =3D entry['violation-id'] + elif tool in entry['analyser']: + proprietary_id =3D entry['analyser'][tool] + else: + proprietary_id =3D "" + if proprietary_id !=3D "": + comment=3Dtool_syntax[tool].replace("VID",proprietary_id) + # Regex to capture the index of the Xen tag and the schema + xen_tag =3D re.compile(get_xen_tag_index_type_regex(tool))\ + .match(entry["id"]) + if xen_tag and xen_tag.group(1) and xen_tag.group(2): + # Save in safe or false-positive-* the key {#id: "comment"} + id_number =3D int(xen_tag.group(1)) + key =3D xen_tag.group(2) + ret[key][id_number] =3D "/* {} */\n".format(comment) + else: + raise TagDatabaseError( + "Error in database file, entry {} has unexpected " + "format.".format(entry["id"]) + ) + + return ret + + +def substitute_tags(tool, input_file, grep_struct, subs_rules): + try: + with open(grep_struct["file"], "wt") as outfile: + + try: + with open(input_file, "rt") as infile: + parsed_content =3D infile.readlines() + except Exception as e: + raise TagDatabaseError("Issue with reading file {}: {}" + .format(input_file, e)) + + # grep_struct contains the line number where the comments are,= the + # line number starts from 1 but in the array the first line is= zero. + # For every line where there is a Xen tag comment, get the Xen= tag + # that is in the capture group zero, extract from the Xen tag = the + # unique index and the type (safe, false-positive-*) and with = those + # information access the subs_rules dictionary to see if there= is + # a match + for line_number in grep_struct["matches"]: + xen_tag =3D grep_struct["matches"][line_number][0] + xen_tag_regex_obj =3D re.compile( + get_xen_tag_index_type_regex(tool)).match(xen_= tag) + id_number =3D int(xen_tag_regex_obj.group(1)) + key =3D xen_tag_regex_obj.group(2) + if id_number in subs_rules[key]: + parsed_content[line_number-1] =3D subs_rules[key][id_n= umber] + + outfile.writelines(parsed_content) + except Exception as e: + raise TagDatabaseError("Issue with writing file {}: {}" + .format(grep_struct["file"], e)) diff --git a/xen/scripts/xen_analysis/utils.py b/xen/scripts/xen_analysis/u= tils.py new file mode 100644 index 000000000000..1193e3f4631e --- /dev/null +++ b/xen/scripts/xen_analysis/utils.py @@ -0,0 +1,56 @@ +#!/usr/bin/env python3 + +import os, re, subprocess + + +def grep(filepath, regex): + regObj =3D re.compile(regex) + res =3D { "file": filepath, "matches": {} } + try: + with open(filepath, "rt") as f: + line_number =3D 1 + for line in f: + match =3D regObj.match(line) + if match: + res["matches"][line_number] =3D match.groups() + line_number =3D line_number + 1 + except Exception as e: + print("WARNING: Can't open {}: {}".format(filepath, e)) + + # Return filename and line matches if there are + return res if res["matches"] else {} + + +def recursive_find_file(path, filename_regex, action =3D None): + filename_reg_obj =3D re.compile(filename_regex) + res =3D [] + for root, dirs, fnames in os.walk(path): + for fname in fnames: + if filename_reg_obj.match(fname): + if action is None: + res.append(os.path.join(root, fname)) + else: + out =3D action(os.path.join(root, fname)) + if out: + res.append(out) + + return res + + +def invoke_command(command, needs_output, exeption_type =3D Exception, + exeption_msg =3D ""): + try: + pipe_stdout =3D subprocess.PIPE if (needs_output =3D=3D True) else= None + output =3D subprocess.run(command, shell=3DTrue, check=3DTrue, + stdout=3Dpipe_stdout, stderr=3Dsubprocess.= STDOUT, + encoding=3D'utf8') + except (subprocess.CalledProcessError, subprocess.SubprocessError) as = e: + if needs_output =3D=3D True: + exeption_msg =3D exeption_msg.format(e.cmd, output.stdout) + else: + exeption_msg =3D exeption_msg.format(e.cmd) + excp =3D exeption_type(exeption_msg) + excp.errorcode =3D e.returncode if hasattr(e, 'returncode') else 1 + raise excp + + return output.stdout --=20 2.17.1 From nobody Fri May 17 01:43:50 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=arm.com Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1670418486269882.0217878757526; Wed, 7 Dec 2022 05:08:06 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.456300.714081 (Exim 4.92) (envelope-from ) id 1p2u8w-0004ik-4t; Wed, 07 Dec 2022 13:07:22 +0000 Received: by outflank-mailman (output) from mailman id 456300.714081; Wed, 07 Dec 2022 13:07:22 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p2u8w-0004ia-0c; Wed, 07 Dec 2022 13:07:22 +0000 Received: by outflank-mailman (input) for mailman id 456300; Wed, 07 Dec 2022 13:07:20 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p2u8t-0004EJ-MP for xen-devel@lists.xenproject.org; Wed, 07 Dec 2022 13:07:20 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-sth1.inumbo.com (Halon) with ESMTP id 12de07bd-7630-11ed-91b6-6bf2151ebd3b; Wed, 07 Dec 2022 14:07:16 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id E84E6ED1; Wed, 7 Dec 2022 05:07:22 -0800 (PST) Received: from e125770.cambridge.arm.com (e125770.cambridge.arm.com [10.1.195.16]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id DB9CD3F73D; Wed, 7 Dec 2022 05:07:14 -0800 (PST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 12de07bd-7630-11ed-91b6-6bf2151ebd3b From: Luca Fancellu To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, wei.chen@arm.com, Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu Subject: [PATCH v3 2/5] xen/scripts: add cppcheck tool to the xen-analysis.py script Date: Wed, 7 Dec 2022 13:07:01 +0000 Message-Id: <20221207130704.20573-3-luca.fancellu@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221207130704.20573-1-luca.fancellu@arm.com> References: <20221207130704.20573-1-luca.fancellu@arm.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZM-MESSAGEID: 1670418487403100004 Add Cppcheck analysis to the xen-analysis.py script using the arguments --run-cppcheck. Now cppcheck analysis will build Xen while the analysis is performed on the source files, it will produce a text report and an additional html output when the script is called with --cppcheck-html. With this patch cppcheck will benefit of platform configuration files that will help it to understand the target of the compilation and improve the analysis. These are XML files placed in xen/tools/cppcheck-plat/, describing to cppcheck the length of basic types to help it in the analysis. To do so: - Update xen-analysis.py with the code to integrate cppcheck. - add platform configuration files for cppcheck.. - add cppcheck-cc.sh script that is a wrapper for cppcheck and it's used as Xen compiler, it will intercept all flags given from the make build system and it will execute cppcheck on the compiled file together with the file compilation. - guarded hypercall-defs.c with CPPCHECK define because cppcheck gets confused as the file does not contain c code. - add false-positive-cppcheck.json file - update documentation. - update .gitignore Signed-off-by: Luca Fancellu Reviewed-by: Stefano Stabellini --- v3 changes: - used right argument on echo command in cppcheck-cc.sh (Stefano) - improved help in the script, now --build-only will generate also cppcheck report when cppcheck is called (Stefano) - add a comment in documenting-violation.rst to show how to locate the proprietary violation id of cppcheck in its reports (Stefano) v2 changes: - changes to commit message (Jan) - changes to the cppcheck-cc.sh script and on platform XML files, fixed version handling in cppcheck_analysis.py (Stefano) - Move revert of Makefile and xen/tools/merge_cppcheck_reports.py to the following patch, modified .gitignore to handle the presence of both Makefile invoked cppcheck and this new method (Stefano) --- --- .gitignore | 6 + docs/misra/cppcheck.txt | 27 +- docs/misra/documenting-violations.rst | 27 +- docs/misra/false-positive-cppcheck.json | 12 + docs/misra/xen-static-analysis.rst | 42 ++- xen/include/hypercall-defs.c | 9 + xen/scripts/xen-analysis.py | 18 +- xen/scripts/xen_analysis/cppcheck_analysis.py | 273 ++++++++++++++++++ .../xen_analysis/cppcheck_report_utils.py | 130 +++++++++ xen/scripts/xen_analysis/generic_analysis.py | 9 +- xen/scripts/xen_analysis/settings.py | 88 +++++- xen/tools/cppcheck-cc.sh | 223 ++++++++++++++ xen/tools/cppcheck-plat/arm32-wchar_t4.xml | 17 ++ xen/tools/cppcheck-plat/arm64-wchar_t2.xml | 17 ++ xen/tools/cppcheck-plat/x86_64-wchar_t2.xml | 17 ++ 15 files changed, 864 insertions(+), 51 deletions(-) create mode 100644 docs/misra/false-positive-cppcheck.json create mode 100644 xen/scripts/xen_analysis/cppcheck_analysis.py create mode 100644 xen/scripts/xen_analysis/cppcheck_report_utils.py create mode 100755 xen/tools/cppcheck-cc.sh create mode 100644 xen/tools/cppcheck-plat/arm32-wchar_t4.xml create mode 100644 xen/tools/cppcheck-plat/arm64-wchar_t2.xml create mode 100644 xen/tools/cppcheck-plat/x86_64-wchar_t2.xml diff --git a/.gitignore b/.gitignore index f5a66f6194dd..39efe2933a33 100644 --- a/.gitignore +++ b/.gitignore @@ -8,8 +8,11 @@ *.d *.d2 *.c.cppcheck +*.cppcheck.txt +*.cppcheck.xml *.opic *.a +*.c.json *.safparse *.so *.so.[0-9]* @@ -282,9 +285,11 @@ xen/arch/*/efi/efi.h xen/arch/*/efi/pe.c xen/arch/*/efi/runtime.c xen/arch/*/include/asm/asm-offsets.h +xen/build-dir-cppcheck/ xen/common/config_data.S xen/common/config.gz xen/cppcheck-htmlreport/ +xen/cppcheck-report/ xen/cppcheck-misra.* xen/include/headers*.chk xen/include/compat/* @@ -315,6 +320,7 @@ xen/xsm/flask/xenpolicy-* tools/flask/policy/policy.conf tools/flask/policy/xenpolicy-* xen/xen +xen/suppression-list.txt xen/xen-cppcheck.xml xen/xen-syms xen/xen-syms.map diff --git a/docs/misra/cppcheck.txt b/docs/misra/cppcheck.txt index 25d8c3050b72..f7b9f678b4d5 100644 --- a/docs/misra/cppcheck.txt +++ b/docs/misra/cppcheck.txt @@ -3,8 +3,7 @@ Cppcheck for Xen static and MISRA analysis =20 Xen can be analysed for both static analysis problems and MISRA violation = using cppcheck, the open source tool allows the creation of a report with all the -findings. Xen has introduced the support in the Makefile so it's very easy= to -use and in this document we can see how. +findings. =20 The minimum version required for cppcheck is 2.7. Note that at the time of writing (June 2022), the version 2.8 is known to be broken [1]. @@ -38,27 +37,7 @@ Dependencies are listed in the readme.md of the project = repository. Use cppcheck to analyse Xen =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D =20 -Using cppcheck integration is very simple, it requires few steps: - - 1) Compile Xen - 2) call the cppcheck make target to generate a report in xml format: - make CPPCHECK_MISRA=3Dy cppcheck - 3) call the cppcheck-html make target to generate a report in xml and html - format: - make CPPCHECK_MISRA=3Dy cppcheck-html - - In case the cppcheck binaries are not in the PATH, CPPCHECK and - CPPCHECK_HTMLREPORT variables can be overridden with the full path to = the - binaries: - - make -C xen \ - CPPCHECK=3D/path/to/cppcheck \ - CPPCHECK_HTMLREPORT=3D/path/to/cppcheck-htmlreport \ - CPPCHECK_MISRA=3Dy \ - cppcheck-html - -The output is by default in a folder named cppcheck-htmlreport, but the na= me -can be changed by passing it in the CPPCHECK_HTMLREPORT_OUTDIR variable. - +To analyse Xen using cppcheck, please refer to the document +xen-static-analysis.rst, section "Analyse Xen with Cppcheck". =20 [1] https://sourceforge.net/p/cppcheck/discussion/general/thread/bfc3ab6c4= 1/?limit=3D25 diff --git a/docs/misra/documenting-violations.rst b/docs/misra/documenting= -violations.rst index 1d23447556d2..0d02a8e6f905 100644 --- a/docs/misra/documenting-violations.rst +++ b/docs/misra/documenting-violations.rst @@ -51,6 +51,7 @@ Here is an example to add a new justification in safe.jso= n:: | { | "id": "SAF-0-safe", | "analyser": { +| "cppcheck": "misra-c2012-20.7", | "coverity": "misra_c_2012_rule_20_7_violation", | "eclair": "MC3R1.R20.7" | }, @@ -77,9 +78,9 @@ Here is an explanation of the fields inside an object of = the "content" array: It tells the tool to substitute a Xen in-code comment having this struc= ture: /* SAF-0-safe [...] \*/ - analyser: it is an object containing pair of key-value strings, the key= is - the analyser, so it can be coverity or eclair, the value is the proprie= tary - id corresponding on the finding, for example when coverity is used as - analyser, the tool will translate the Xen in-code coment in this way: + the analyser, so it can be cppcheck, coverity or eclair, the value is t= he + proprietary id corresponding on the finding, for example when coverity = is + used as analyser, the tool will translate the Xen in-code coment in thi= s way: /* SAF-0-safe [...] \*/ -> /* coverity[misra_c_2012_rule_20_7_violation= ] \*/ if the object doesn't have a key-value, then the corresponding in-code comment won't be translated. @@ -189,3 +190,23 @@ the same line and the same "violation ID". =20 Also, the same tag can be used on other symbols from the linker that are declared in the codebase, because the justification holds for them too. + +A possible violation found by Cppcheck can be handled in the same way, fro= m the +cppcheck text report it is possible to identify the violation id: + +| include/public/arch-arm.h(226,0):misra-c2012-20.7:style:Expressions resu= lting from the expansion of macro parameters shall be enclosed in parenthes= es (Misra rule 20.7) + +The violation id can be located also in the HTML report, opening index.htm= l from +the browser, the violations can be filtered by id in the left side panel, = under +the column "Defect ID". On the right there will be a list of files with th= e type +of violation and the violation line number, for the same violation above, = there +will be an entry like the following and the violation id will be in the co= lumn +"Id": + +| include/public/arch-arm.h +| [...] +| 226 misra-c2012-20.7 style Expressions resulting from the expansion of = macro parameters shall be enclosed in parentheses (Misra rule 20.7) +| [...] + +Given the violation id "misra-c2012-20.7", the procedure above can be foll= owed +to justify this finding. diff --git a/docs/misra/false-positive-cppcheck.json b/docs/misra/false-pos= itive-cppcheck.json new file mode 100644 index 000000000000..5d4da2ce6170 --- /dev/null +++ b/docs/misra/false-positive-cppcheck.json @@ -0,0 +1,12 @@ +{ + "version": "1.0", + "content": [ + { + "id": "SAF-0-false-positive-cppcheck", + "violation-id": "", + "tool-version": "", + "name": "Sentinel", + "text": "Next ID to be used" + } + ] +} diff --git a/docs/misra/xen-static-analysis.rst b/docs/misra/xen-static-ana= lysis.rst index 5b886474d4a0..2712255db1b0 100644 --- a/docs/misra/xen-static-analysis.rst +++ b/docs/misra/xen-static-analysis.rst @@ -7,9 +7,8 @@ The Xen codebase integrates some scripts and tools that hel= ps the developer to perform static analysis of the code, currently Xen supports three analysis= tool that are eclair, coverity and cppcheck. The Xen tree has a script (xen-analysis.py) available to ease the analysis -process and it integrates a way to suppress findings on these tools (only = Eclair -and Coverity are currently supported by the script), please check the -documenting-violation.rst document to know more about it. +process and it integrates a way to suppress findings on these tools, please +check the documenting-violation.rst document to know more about it. =20 Analyse Xen with Coverity or Eclair ----------------------------------- @@ -52,3 +51,40 @@ When invoking the script, the procedure below will be fo= llowed: this step, call the script adding the --no-clean argument, but before running again the script, call it with the --clean-only argument, that= will execute only this cleaning step. + + +Analyse Xen with Cppcheck +------------------------- + +Cppcheck tool is integrated in xen-analysis.py script, when using the scri= pt, +the tool will be called on every source file compiled by the make build sy= stem. +Here how to start the analysis with Cppcheck: + + - xen-analysis.py --run-cppcheck [--cppcheck-misra] [--cppcheck-html] -- + [optional make arguments] + +The command above tells the script to prepare the codebase and use Cppchec= k tool +for the analysis. +The optional argument --cppcheck-misra activates the analysis also for MIS= RA +compliance. +The optional argument --cppcheck-html instruct cppcheck to produce an addi= tional +HTML report. + +When invoking the script for Cppcheck analysis, the followed procedure is +similar to the one above for Coverity or Eclair, but it has some additional +steps: + + 1. This step is the same as step 1 for Coverity/Eclair. + 2. The cppcheck dependency are created, build directory for cppcheck anal= ysis + and an header file containing internal compiler macro + (include/generated/compiler-def.h) are generated + 3. Xen compilation starts using every suppli= ed + at the script invocation, but because cppcheck is not able to intercep= t the + compiled files and flags on compiler invocation, a script (cppcheck-cc= .sh) + is passed as CC to the make system, it is a wrapper for the compiler t= hat + will also execute cppcheck on every compiled file. + 4. After the compilation and analysis, the cppcheck report will be created + putting together all the cppcheck report fragments for every analysed = file. + Cppcheck will produce a text fragment and an additional XML report fra= gment + if the script is configured to produce the HTML output. + 5. This step is the same as step 3 for Coverity/Eclair. diff --git a/xen/include/hypercall-defs.c b/xen/include/hypercall-defs.c index 189612107402..41e1af01f6b2 100644 --- a/xen/include/hypercall-defs.c +++ b/xen/include/hypercall-defs.c @@ -60,6 +60,13 @@ * are possible. */ =20 +/* + * Cppcheck thinks this file needs to be analysed because it is preprocess= ed by + * the compiler, but it gets confused because this file does not contains C + * code. Hence protect the code when CPPCHECK is used. + */ +#ifndef CPPCHECK + #ifdef CONFIG_HVM #define PREFIX_hvm hvm #else @@ -288,3 +295,5 @@ mca do do - = - - #ifndef CONFIG_PV_SHIM_EXCLUSIVE paging_domctl_cont do do do do - #endif + +#endif /* !CPPCHECK */ diff --git a/xen/scripts/xen-analysis.py b/xen/scripts/xen-analysis.py index b5d9ef1862c9..8e50c27cd898 100755 --- a/xen/scripts/xen-analysis.py +++ b/xen/scripts/xen-analysis.py @@ -1,28 +1,42 @@ #!/usr/bin/env python3 =20 import sys -from xen_analysis import settings, generic_analysis +from xen_analysis import settings, generic_analysis, cppcheck_analysis from xen_analysis.generic_analysis import * +from xen_analysis.cppcheck_analysis import * + +PhaseExceptions =3D (GetMakeVarsPhaseError, ParseTagPhaseError, + CppcheckDepsPhaseError, BuildPhaseError, + CppcheckReportPhaseError) =20 =20 def main(argv): ret_code =3D 0 settings.parse_commandline(argv) try: + if settings.step_get_make_vars: + cppcheck_analysis.get_make_vars() if settings.step_parse_tags: generic_analysis.parse_xen_tags() + if settings.step_cppcheck_deps: + cppcheck_analysis.generate_cppcheck_deps() if settings.step_build_xen: generic_analysis.build_xen() - except (ParseTagPhaseError, BuildPhaseError) as e: + if settings.step_cppcheck_report: + cppcheck_analysis.generate_cppcheck_report() + except PhaseExceptions as e: print("ERROR: {}".format(e)) if hasattr(e, "errorcode"): ret_code =3D e.errorcode finally: if settings.step_clean_analysis: + cppcheck_analysis.clean_analysis_artifacts() e =3D generic_analysis.clean_analysis_artifacts() if e: print("ERROR: {}".format(e)) ret_code =3D 1 + if settings.step_distclean_analysis: + cppcheck_analysis.clean_reports() =20 sys.exit(ret_code) =20 diff --git a/xen/scripts/xen_analysis/cppcheck_analysis.py b/xen/scripts/xe= n_analysis/cppcheck_analysis.py new file mode 100644 index 000000000000..0e952a169641 --- /dev/null +++ b/xen/scripts/xen_analysis/cppcheck_analysis.py @@ -0,0 +1,273 @@ +#!/usr/bin/env python3 + +import os, re, shutil +from . import settings, utils, cppcheck_report_utils + +class GetMakeVarsPhaseError(Exception): + pass + +class CppcheckDepsPhaseError(Exception): + pass + +class CppcheckReportPhaseError(Exception): + pass + +CPPCHECK_BUILD_DIR =3D "build-dir-cppcheck" +CPPCHECK_HTMLREPORT_OUTDIR =3D "cppcheck-htmlreport" +CPPCHECK_REPORT_OUTDIR =3D "cppcheck-report" +cppcheck_extra_make_args =3D "" +xen_cc =3D "" + +def get_make_vars(): + global xen_cc + invoke_make =3D utils.invoke_command( + "make -C {} {} export-variable-CC" + .format(settings.xen_dir, settings.make_forward_args), + True, GetMakeVarsPhaseError, + "Error occured retrieving make vars:\n{}" + ) + + cc_var_regex =3D re.search('^CC=3D(.*)$', invoke_make, flags=3Dre.M) + if cc_var_regex: + xen_cc =3D cc_var_regex.group(1) + + if xen_cc =3D=3D "": + raise GetMakeVarsPhaseError("CC variable not found in Xen make out= put") + + +def __generate_suppression_list(out_file): + # The following lambda function will return a file if it contains line= s with + # a comment containing "cppcheck-suppress[*]" on a single line. + grep_action =3D lambda x: utils.grep(x, + r'^[ \t]*/\* cppcheck-suppress\[(.*)\] \*/$') + # Look for a list of .h files that matches the condition above + headers =3D utils.recursive_find_file(settings.xen_dir, r'.*\.h$', + grep_action) + + try: + with open(out_file, "wt") as supplist_file: + # Add this rule to skip every finding in the autogenerated + # header for cppcheck + supplist_file.write("*:*generated/compiler-def.h\n") + + for entry in headers: + filename =3D entry["file"] + try: + with open(filename, "rt") as infile: + header_content =3D infile.readlines() + except OSError as e: + raise CppcheckDepsPhaseError( + "Issue with reading file {}: {}" + .format(filename, e) + ) + header_lines_len =3D len(header_content) + # line_num in entry will be header_content[line_num-1], he= re we + # are going to search the first line after line_num that h= ave + # anything different from comments or empty line, because = the + # in-code comment suppression is related to that line then. + for line_num in entry["matches"]: + cppcheck_violation_id =3D "" + tmp_line =3D line_num + # look up to which line is referring the comment at + # line_num (which would be header_content[tmp_line-1]) + comment_section =3D False + while tmp_line < header_lines_len: + line =3D header_content[tmp_line] + # Matches a line with just optional spaces/tabs an= d the + # start of a comment '/*' + comment_line_starts =3D re.match('^[ \t]*/\*.*$', = line) + # Matches a line with text and the end of a commen= t '*/' + comment_line_stops =3D re.match('^.*\*/$', line) + if (not comment_section) and comment_line_starts: + comment_section =3D True + if (len(line.strip()) !=3D 0) and (not comment_sec= tion): + cppcheck_violation_id =3D entry["matches"][lin= e_num][0] + break + if comment_section and comment_line_stops: + comment_section =3D False + tmp_line =3D tmp_line + 1 + + if cppcheck_violation_id =3D=3D "": + raise CppcheckDepsPhaseError( + "Error matching cppcheck comment in {} at line= {}." + .format(filename, line_num) + ) + # Write [error id]:[filename]:[line] + # tmp_line refers to the array index, so translated to= the + # file line (that begins with 1) it is tmp_line+1 + supplist_file.write( + "{}:{}:{}\n".format(cppcheck_violation_id, fil= ename, + (tmp_line + 1)) + ) + except OSError as e: + raise CppcheckDepsPhaseError("Issue with writing file {}: {}" + .format(out_file, e)) + + +def generate_cppcheck_deps(): + global cppcheck_extra_make_args + + # Compile flags to pass to cppcheck: + # - include config.h as this is passed directly to the compiler. + # - define CPPCHECK as we use it to disable or enable some specific pa= rt of + # the code to solve some cppcheck issues. + # - explicitely enable some cppcheck checks as we do not want to use "= all" + # which includes unusedFunction which gives wrong positives as we ch= eck + # file per file. + # - Explicitly suppress warnings on compiler-def.h because cppcheck th= rows + # an unmatchedSuppression due to the rule we put in suppression-list= .txt + # to skip every finding in the file. + # + # Compiler defines are in compiler-def.h which is included in config.h + # + cppcheck_flags=3D""" +--cppcheck-build-dir=3D{}/{} + --max-ctu-depth=3D10 + --enable=3Dstyle,information,missingInclude + --template=3D\'{{file}}({{line}},{{column}}):{{id}}:{{severity}}:{{messag= e}}\' + --relative-paths=3D{} + --inline-suppr + --suppressions-list=3D{}/suppression-list.txt + --suppress=3D'unmatchedSuppression:*generated/compiler-def.h' + --include=3D{}/include/xen/config.h + -DCPPCHECK +""".format(settings.outdir, CPPCHECK_BUILD_DIR, settings.xen_dir, + settings.outdir, settings.xen_dir) + + invoke_cppcheck =3D utils.invoke_command( + "{} --version".format(settings.cppcheck_binpath), + True, CppcheckDepsPhaseError, + "Error occured retrieving cppcheck version:\n{}\n\n{}" + ) + + version_regex =3D re.search('^Cppcheck (.*)$', invoke_cppcheck, flags= =3Dre.M) + # Currently, only cppcheck version >=3D 2.7 is supported, but version = 2.8 is + # known to be broken, please refer to docs/misra/cppcheck.txt + if (not version_regex) or (not version_regex.group(1).startswith("2.7"= )): + raise CppcheckDepsPhaseError( + "Can't find cppcheck version or version is not 2.7" + ) + + # If misra option is selected, append misra addon and generate cppcheck + # files for misra analysis + if settings.cppcheck_misra: + cppcheck_flags =3D cppcheck_flags + " --addon=3Dcppcheck-misra.jso= n" + + utils.invoke_command( + "{}/convert_misra_doc.py -i {}/docs/misra/rules.rst" + " -o {}/cppcheck-misra.txt -j {}/cppcheck-misra.json" + .format(settings.tools_dir, settings.repo_dir, + settings.outdir, settings.outdir), + False, CppcheckDepsPhaseError, + "An error occured when running:\n{}" + ) + + # Generate compiler macros + os.makedirs("{}/include/generated".format(settings.outdir), exist_ok= =3DTrue) + utils.invoke_command( + "{} -dM -E -o \"{}/include/generated/compiler-def.h\" - < /dev= /null" + .format(xen_cc, settings.outdir), + False, CppcheckDepsPhaseError, + "An error occured when running:\n{}" + ) + + # Generate cppcheck suppression list + __generate_suppression_list( + "{}/suppression-list.txt".format(settings.outdir)) + + # Generate cppcheck build folder + os.makedirs("{}/{}".format(settings.outdir, CPPCHECK_BUILD_DIR), + exist_ok=3DTrue) + + cppcheck_cc_flags =3D """--compiler=3D{} --cppcheck-cmd=3D{} {} + --cppcheck-plat=3D{}/cppcheck-plat --ignore-path=3Dtools/ + --ignore-path=3Darch/x86/efi/check.c +""".format(xen_cc, settings.cppcheck_binpath, cppcheck_flags, + settings.tools_dir) + + if settings.cppcheck_html: + cppcheck_cc_flags =3D cppcheck_cc_flags + " --cppcheck-html" + + # Generate the extra make argument to pass the cppcheck-cc.sh wrapper = as CC + cppcheck_extra_make_args =3D "CC=3D\"{}/cppcheck-cc.sh {} --\"".format( + settings.tools_dir, + cppcheck_cc_flags + ).replace("\n", "") + + +def generate_cppcheck_report(): + # Prepare text report + # Look for a list of .cppcheck.txt files, those are the txt report + # fragments + fragments =3D utils.recursive_find_file(settings.outdir, r'.*\.cppchec= k.txt$') + text_report_dir =3D "{}/{}".format(settings.outdir, + CPPCHECK_REPORT_OUTDIR) + report_filename =3D "{}/xen-cppcheck.txt".format(text_report_dir) + os.makedirs(text_report_dir, exist_ok=3DTrue) + try: + cppcheck_report_utils.cppcheck_merge_txt_fragments(fragments, + report_filename, + [settings.xen_d= ir]) + except cppcheck_report_utils.CppcheckTXTReportError as e: + raise CppcheckReportPhaseError(e) + + # If HTML output is requested + if settings.cppcheck_html: + # Look for a list of .cppcheck.xml files, those are the XML report + # fragments + fragments =3D utils.recursive_find_file(settings.outdir, + r'.*\.cppcheck.xml$') + html_report_dir =3D "{}/{}".format(settings.outdir, + CPPCHECK_HTMLREPORT_OUTDIR) + xml_filename =3D "{}/xen-cppcheck.xml".format(html_report_dir) + os.makedirs(html_report_dir, exist_ok=3DTrue) + try: + cppcheck_report_utils.cppcheck_merge_xml_fragments(fragments, + xml_filenam= e, + settings.xe= n_dir, + settings.ou= tdir) + except cppcheck_report_utils.CppcheckHTMLReportError as e: + raise CppcheckReportPhaseError(e) + # Call cppcheck-htmlreport utility to generate the HTML output + utils.invoke_command( + "{} --file=3D{} --source-dir=3D{} --report-dir=3D{}/html --tit= le=3DXen" + .format(settings.cppcheck_htmlreport_binpath, xml_filename, + settings.xen_dir, html_report_dir), + False, CppcheckReportPhaseError, + "Error occured generating Cppcheck HTML report:\n{}" + ) + # Strip src and obj path from *.html files + html_files =3D utils.recursive_find_file(html_report_dir, r'.*\.ht= ml$') + try: + cppcheck_report_utils.cppcheck_strip_path_html(html_files, + (settings.xen_d= ir, + settings.outdi= r)) + except cppcheck_report_utils.CppcheckHTMLReportError as e: + raise CppcheckReportPhaseError(e) + + +def clean_analysis_artifacts(): + clean_files =3D ("suppression-list.txt", "cppcheck-misra.txt", + "cppcheck-misra.json") + cppcheck_build_dir =3D "{}/{}".format(settings.outdir, CPPCHECK_BUILD_= DIR) + if os.path.isdir(cppcheck_build_dir): + shutil.rmtree(cppcheck_build_dir) + artifact_files =3D utils.recursive_find_file(settings.outdir, + r'.*\.(?:c\.json|cppcheck\.txt|cppcheck\.x= ml)$') + for file in clean_files: + file =3D "{}/{}".format(settings.outdir, file) + if os.path.isfile(file): + artifact_files.append(file) + for delfile in artifact_files: + os.remove(delfile) + + +def clean_reports(): + text_report_dir =3D "{}/{}".format(settings.outdir, + CPPCHECK_REPORT_OUTDIR) + html_report_dir =3D "{}/{}".format(settings.outdir, + CPPCHECK_HTMLREPORT_OUTDIR) + if os.path.isdir(text_report_dir): + shutil.rmtree(text_report_dir) + if os.path.isdir(html_report_dir): + shutil.rmtree(html_report_dir) diff --git a/xen/scripts/xen_analysis/cppcheck_report_utils.py b/xen/script= s/xen_analysis/cppcheck_report_utils.py new file mode 100644 index 000000000000..02440aefdfec --- /dev/null +++ b/xen/scripts/xen_analysis/cppcheck_report_utils.py @@ -0,0 +1,130 @@ +#!/usr/bin/env python3 + +import os +from xml.etree import ElementTree + +class CppcheckHTMLReportError(Exception): + pass + +class CppcheckTXTReportError(Exception): + pass + + +def __elements_equal(el1, el2): + if type(el1) !=3D type(el2): return False + + if el1.find('location') is None: return False + if el2.find('location') is None: return False + + el1_location =3D str(el1.find('location').attrib) + el2_location =3D str(el2.find('location').attrib) + + if el1_location !=3D el2_location: return False + + return True + + +def __contain_element(new, lst): + for elem in lst: + if __elements_equal(new, elem): + return True + return False + + +def __get_xml_root_file(filename): + try: + result_xml_root =3D ElementTree.parse(filename).getroot() + except ElementTree.ParseError as e: + raise CppcheckHTMLReportError( + "XML parsing error in {}: {}".format(filename, e) + ) + return result_xml_root + + +def __sanitize_cppcheck_xml_path(xml_tree, src_path, obj_path): + # Some path are relative to the source tree but some others are genera= ted + # in the obj tree, for cppcheck when using cppcheck-htmlreport we can = pass + # only one source tree where the files will be fetched if relative pat= h are + # found. So for every path that does not exists in src tree, we guess = it + # comes from obj tree and we put explicit absolute path to it + error_item_root =3D xml_tree.findall("errors")[0] + for error_item in error_item_root: + for location_item in error_item.findall("location"): + path =3D location_item.attrib["file"] + new_obj_path =3D obj_path + "/" + path + new_src_path =3D src_path + "/" + path + if (path[0] !=3D "/") and (not os.path.isfile(new_src_path)) \ + and os.path.isfile(new_obj_path): + location_item.attrib["file"] =3D new_obj_path + + +def cppcheck_merge_xml_fragments(fragments_list, out_xml_file, src_path, + obj_path): + + result_xml =3D __get_xml_root_file(fragments_list[0]) + insert_point =3D result_xml.findall("errors")[0] + for xml_file in fragments_list[1:]: + xml_root =3D __get_xml_root_file(xml_file) + curr_elem_list =3D list(insert_point) + new_elem_list =3D list(xml_root.findall("errors")[0]) + for xml_error_elem in new_elem_list: + if not __contain_element(xml_error_elem, curr_elem_list): + insert_point.insert(1, xml_error_elem) + + if result_xml is None: + return False + + __sanitize_cppcheck_xml_path(result_xml, src_path, obj_path) + + ElementTree.ElementTree(result_xml).write(out_xml_file) + + return True + + +def cppcheck_merge_txt_fragments(fragments_list, out_txt_file, strip_paths= ): + try: + with open(out_txt_file, "wt") as outfile: + # Using a set will remove automatically the duplicate lines + text_report_content =3D set() + for file in fragments_list: + try: + with open(file, "rt") as infile: + frag_lines =3D infile.readlines() + except OSError as e: + raise CppcheckTXTReportError( + "Issue with reading file {}: {}" + .format(file, e) + ) + text_report_content.update(frag_lines) + + # Back to modifiable list + text_report_content =3D list(text_report_content) + # Strip path from report lines + for i in list(range(0, len(text_report_content))): + for path in strip_paths: + text_report_content[i] =3D text_report_content[i].repl= ace( + path + "/"= , "") + #=C2=A0Write the final text report + outfile.writelines(text_report_content) + except OSError as e: + raise CppcheckTXTReportError("Issue with writing file {}: {}" + .format(out_txt_file, e)) + + +def cppcheck_strip_path_html(html_files, strip_paths): + for file in html_files: + try: + with open(file, "rt") as infile: + html_lines =3D infile.readlines() + except OSError as e: + raise CppcheckHTMLReportError("Issue with reading file {}: {}" + .format(file, = e)) + for i in list(range(0, len(html_lines))): + for path in strip_paths: + html_lines[i] =3D html_lines[i].replace(path + "/", "") + try: + with open(file, "wt") as outfile: + outfile.writelines(html_lines) + except OSError as e: + raise CppcheckHTMLReportError("Issue with writing file {}: {}" + .format(file, = e)) diff --git a/xen/scripts/xen_analysis/generic_analysis.py b/xen/scripts/xen= _analysis/generic_analysis.py index 372ff3566442..94122aebace0 100644 --- a/xen/scripts/xen_analysis/generic_analysis.py +++ b/xen/scripts/xen_analysis/generic_analysis.py @@ -1,7 +1,7 @@ #!/usr/bin/env python3 =20 -import os, subprocess -from . import settings, utils, tag_database +import os +from . import settings, utils, tag_database, cppcheck_analysis =20 class ParseTagPhaseError(Exception): pass @@ -61,8 +61,9 @@ def parse_xen_tags(): =20 def build_xen(): utils.invoke_command( - "make -C {} {} build" - .format(settings.xen_dir, settings.make_forward_args), + "make -C {} {} {} build" + .format(settings.xen_dir, settings.make_forward_args, + cppcheck_analysis.cppcheck_extra_make_args), False, BuildPhaseError, "Build error occured when running:\n{}" ) diff --git a/xen/scripts/xen_analysis/settings.py b/xen/scripts/xen_analysi= s/settings.py index e8d474eaf378..a8502e554e95 100644 --- a/xen/scripts/xen_analysis/settings.py +++ b/xen/scripts/xen_analysis/settings.py @@ -7,14 +7,23 @@ xen_dir =3D os.path.realpath(module_dir + "/../..") repo_dir =3D os.path.realpath(xen_dir + "/..") tools_dir =3D os.path.realpath(xen_dir + "/tools") =20 +step_get_make_vars =3D False step_parse_tags =3D True +step_cppcheck_deps =3D False step_build_xen =3D True +step_cppcheck_report =3D False step_clean_analysis =3D True +step_distclean_analysis =3D False =20 target_build =3D False target_clean =3D False +target_distclean =3D False =20 analysis_tool =3D "" +cppcheck_binpath =3D "cppcheck" +cppcheck_html =3D False +cppcheck_htmlreport_binpath =3D "cppcheck-htmlreport" +cppcheck_misra =3D False make_forward_args =3D "" outdir =3D xen_dir =20 @@ -25,44 +34,66 @@ Usage: {} [OPTION] ... [-- [make arguments]] =20 This script runs the analysis on the Xen codebase. =20 -The phases for the analysis are , , +The phases for the analysis are , , , =20 Depending on the options below, only some phases will run: =20 -: tags, build, clean ---build-only: build +: tags, build, [cppcheck report], clean +--build-only: build, [cppcheck report] --clean-only: clean +--distclean: clean, [destroy cppcheck report] --no-build: tags, clean --no-clean: tags, build =20 --no-build/--no-clean can be passed together to avoid both clean and build phases. -Tags and build phases need to specify --run-coverity or --run-eclair. +Tags and build phases need to specify --run-coverity, --run-eclair or +--run-cppcheck. +Cppcheck report creation phase runs only when --run-cppcheck is passed to = the +script. =20 Options: - --build-only Run only the commands to build Xen with the optional make - arguments passed to the script - --clean-only Run only the commands to clean the analysis artifacts - -h, --help Print this help - --no-build Skip the build Xen phase - --no-clean Don\'t clean the analysis artifacts on exit - --run-coverity Run the analysis for the Coverity tool - --run-eclair Run the analysis for the Eclair tool + --build-only Run only the commands to build Xen with the option= al + make arguments passed to the script + --clean-only Run only the commands to clean the analysis artifa= cts + --cppcheck-bin=3D Path to the cppcheck binary (Default: {}) + --cppcheck-html Produce an additional HTML output report for Cppch= eck + --cppcheck-html-bin=3D Path to the cppcheck-html binary (Default: {}) + --cppcheck-misra Activate the Cppcheck MISRA analysis + --distclean Clean analysis artifacts and reports + -h, --help Print this help + --no-build Skip the build Xen phase + --no-clean Don\'t clean the analysis artifacts on exit + --run-coverity Run the analysis for the Coverity tool + --run-cppcheck Run the Cppcheck analysis tool on Xen + --run-eclair Run the analysis for the Eclair tool """ - print(msg.format(sys.argv[0])) + print(msg.format(sys.argv[0], cppcheck_binpath, + cppcheck_htmlreport_binpath)) =20 =20 def parse_commandline(argv): global analysis_tool + global cppcheck_binpath + global cppcheck_html + global cppcheck_htmlreport_binpath + global cppcheck_misra global make_forward_args global outdir + global step_get_make_vars global step_parse_tags + global step_cppcheck_deps global step_build_xen + global step_cppcheck_report global step_clean_analysis + global step_distclean_analysis global target_build global target_clean + global target_distclean forward_to_make =3D False for option in argv: + args_with_content_regex =3D re.match(r'^(--[a-z]+[a-z-]*)=3D(.*)$'= , option) + if forward_to_make: # Intercept outdir outdir_regex =3D re.match("^O=3D(.*)$", option) @@ -74,6 +105,18 @@ def parse_commandline(argv): target_build =3D True elif option =3D=3D "--clean-only": target_clean =3D True + elif args_with_content_regex and \ + args_with_content_regex.group(1) =3D=3D "--cppcheck-bin": + cppcheck_binpath =3D args_with_content_regex.group(2) + elif option =3D=3D "--cppcheck-html": + cppcheck_html =3D True + elif args_with_content_regex and \ + args_with_content_regex.group(1) =3D=3D "--cppcheck-html-bin": + cppcheck_htmlreport_binpath =3D args_with_content_regex.group(= 2) + elif option =3D=3D "--cppcheck-misra": + cppcheck_misra =3D True + elif option =3D=3D "--distclean": + target_distclean =3D True elif (option =3D=3D "--help") or (option =3D=3D "-h"): help() sys.exit(0) @@ -83,6 +126,11 @@ def parse_commandline(argv): step_clean_analysis =3D False elif (option =3D=3D "--run-coverity") or (option =3D=3D "--run-ecl= air"): analysis_tool =3D option[6:] + elif (option =3D=3D "--run-cppcheck"): + analysis_tool =3D "cppcheck" + step_get_make_vars =3D True + step_cppcheck_deps =3D True + step_cppcheck_report =3D True elif option =3D=3D "--": forward_to_make =3D True else: @@ -90,13 +138,23 @@ def parse_commandline(argv): help() sys.exit(1) =20 - if target_build and target_clean: - print("--build-only is not compatible with --clean-only argument.") + if target_build and (target_clean or target_distclean): + print("--build-only is not compatible with --clean-only/--distclea= n " + "argument.") sys.exit(1) =20 + if target_distclean: + # Implicit activation of clean target + target_clean =3D True + + step_distclean_analysis =3D True + if target_clean: + step_get_make_vars =3D False step_parse_tags =3D False + step_cppcheck_deps =3D False step_build_xen =3D False + step_cppcheck_report =3D False step_clean_analysis =3D True return =20 diff --git a/xen/tools/cppcheck-cc.sh b/xen/tools/cppcheck-cc.sh new file mode 100755 index 000000000000..f6728e4c1084 --- /dev/null +++ b/xen/tools/cppcheck-cc.sh @@ -0,0 +1,223 @@ +#!/usr/bin/env bash + +set -e + +function help() { + cat < + +This script is a wrapper for cppcheck that enables it to analyse the files= that +are the target for the build, it is used in place of a selected compiler a= nd the +make process will run it on every file that needs to be built. +All the arguments passed to the original compiler are forwarded to it with= out +modification, furthermore, they are used to improve the cppcheck analysis. + +Options: + --compiler=3D Use this compiler for the build + --cppcheck-cmd=3D Command line for the cppcheck analysis. + --cppcheck-html Prepare for cppcheck HTML output + --cppcheck-plat=3D Path to the cppcheck platform folder + --ignore-path=3D This script won't run cppcheck on the files having t= his + path, the compiler will run anyway on them. This argum= ent + can be specified multiple times. + -h, --help Print this help +EOF +} + +CC_FILE=3D"" +COMPILER=3D"" +CPPCHECK_HTML=3D"n" +CPPCHECK_PLAT_PATH=3D"" +CPPCHECK_TOOL=3D"" +CPPCHECK_TOOL_ARGS=3D"" +FORWARD_FLAGS=3D"" +IGNORE_PATH=3D"n" +IGNORE_PATH_LIST=3D"" +JDB_FILE=3D"" +OBJTREE_PATH=3D"" + +# Variable used for arg parsing +forward_to_cc=3D"n" +sm_tool_args=3D"n" +obj_arg_content=3D"n" + +for OPTION in "$@" +do + if [ "${forward_to_cc}" =3D "y" ]; then + if [[ ${OPTION} =3D=3D *.c ]] + then + CC_FILE=3D"${OPTION}" + elif [ "${OPTION}" =3D "-o" ] + then + # After -o there is the path to the obj file, flag it + obj_arg_content=3D"y" + elif [ "${obj_arg_content}" =3D "y" ] + then + # This must be the path to the obj file, turn off flag and sav= e path + OBJTREE_PATH=3D"$(dirname "${OPTION}")" + obj_arg_content=3D"n" + fi + # Forward any argument to the compiler + FORWARD_FLAGS=3D"${FORWARD_FLAGS} ${OPTION}" + continue + fi + case ${OPTION} in + -h|--help) + help + exit 0 + ;; + --compiler=3D*) + COMPILER=3D"${OPTION#*=3D}" + sm_tool_args=3D"n" + ;; + --cppcheck-cmd=3D*) + CPPCHECK_TOOL=3D"${OPTION#*=3D}" + sm_tool_args=3D"y" + ;; + --cppcheck-html) + CPPCHECK_HTML=3D"y" + sm_tool_args=3D"n" + ;; + --cppcheck-plat=3D*) + CPPCHECK_PLAT_PATH=3D"${OPTION#*=3D}" + sm_tool_args=3D"n" + ;; + --ignore-path=3D*) + IGNORE_PATH_LIST=3D"${IGNORE_PATH_LIST} ${OPTION#*=3D}" + sm_tool_args=3D"n" + ;; + --) + forward_to_cc=3D"y" + sm_tool_args=3D"n" + ;; + *) + if [ "${sm_tool_args}" =3D "y" ]; then + CPPCHECK_TOOL_ARGS=3D"${CPPCHECK_TOOL_ARGS} ${OPTION}" + else + echo "Invalid option ${OPTION}" + exit 1 + fi + ;; + esac +done + +if [ "${COMPILER}" =3D "" ] +then + echo "--compiler arg is mandatory." + exit 1 +fi + +function create_jcd() { + local line=3D"${1}" + local arg_num=3D0 + local same_line=3D0 + + { + echo "[" + echo " {" + echo " \"arguments\": [" + + for arg in ${line}; do + # This code prevents to put comma in the last element of the l= ist or + # on sequential lines that are going to be merged + if [ "${arg_num}" -ne 0 ] && [ "${same_line}" -eq 0 ] + then + echo "," + fi + if [ "${same_line}" -ne 0 ] + then + echo -n "${arg}\"" + same_line=3D0 + elif [ "${arg}" =3D "-iquote" ] || [ "${arg}" =3D "-I" ] + then + # cppcheck doesn't understand -iquote, substitute with -I + echo -n " \"-I" + same_line=3D1 + else + echo -n " \"${arg}\"" + fi + arg_num=3D$(( arg_num + 1 )) + done + echo "" + echo " ]," + echo " \"directory\": \"$(pwd -P)\"," + echo " \"file\": \"${CC_FILE}\"" + echo " }" + echo "]" + } > "${JDB_FILE}" +} + + +# Execute compiler with forwarded flags +# Shellcheck complains about missing quotes on FORWARD_FLAGS, but they can= 't be +# used here +# shellcheck disable=3DSC2086 +${COMPILER} ${FORWARD_FLAGS} + +if [ -n "${CC_FILE}" ]; +then + for path in ${IGNORE_PATH_LIST} + do + if [[ ${CC_FILE} =3D=3D *${path}* ]] + then + IGNORE_PATH=3D"y" + echo "${0}: ${CC_FILE} ignored by --ignore-path matching *${pa= th}*" + fi + done + if [ "${IGNORE_PATH}" =3D "n" ] + then + JDB_FILE=3D"${OBJTREE_PATH}/$(basename "${CC_FILE}".json)" + + # Prepare the Json Compilation Database for the file + create_jcd "${COMPILER} ${FORWARD_FLAGS}" + + out_file=3D"${OBJTREE_PATH}/$(basename "${CC_FILE%.c}".cppcheck.tx= t)" + + # Select the right target platform, ARCH is generated from Xen Mak= efile + case ${ARCH} in + arm64) + # arm64 has efi code compiled with -fshort-wchar + platform=3D"${CPPCHECK_PLAT_PATH}/arm64-wchar_t2.xml" + ;; + arm32) + # arm32 has no efi code + platform=3D"${CPPCHECK_PLAT_PATH}/arm32-wchar_t4.xml" + ;; + x86_64) + # x86_64 has efi code compiled with -fshort-wchar + platform=3D"${CPPCHECK_PLAT_PATH}/x86_64-wchar_t2.xml" + ;; + *) + echo "ARCH: ${ARCH} not expected!" + exit 1 + ;; + esac + + if [ ! -f "${platform}" ] + then + echo "${platform} not found!" + exit 1 + fi + + # Shellcheck complains about missing quotes on CPPCHECK_TOOL_ARGS,= but + # they can't be used here + # shellcheck disable=3DSC2086 + ${CPPCHECK_TOOL} ${CPPCHECK_TOOL_ARGS} \ + --project=3D"${JDB_FILE}" \ + --output-file=3D"${out_file}" \ + --platform=3D"${platform}" + + if [ "${CPPCHECK_HTML}" =3D "y" ] + then + # Shellcheck complains about missing quotes on CPPCHECK_TOOL_A= RGS, + # but they can't be used here + # shellcheck disable=3DSC2086 + ${CPPCHECK_TOOL} ${CPPCHECK_TOOL_ARGS} \ + --project=3D"${JDB_FILE}" \ + --output-file=3D"${out_file%.txt}.xml" \ + --platform=3D"${platform}" \ + -q \ + --xml + fi + fi +fi diff --git a/xen/tools/cppcheck-plat/arm32-wchar_t4.xml b/xen/tools/cppchec= k-plat/arm32-wchar_t4.xml new file mode 100644 index 000000000000..3aefa7ba5c98 --- /dev/null +++ b/xen/tools/cppcheck-plat/arm32-wchar_t4.xml @@ -0,0 +1,17 @@ + + + 8 + unsigned + + 2 + 4 + 4 + 8 + 4 + 8 + 8 + 4 + 4 + 4 + + diff --git a/xen/tools/cppcheck-plat/arm64-wchar_t2.xml b/xen/tools/cppchec= k-plat/arm64-wchar_t2.xml new file mode 100644 index 000000000000..b2dc2fb2cc50 --- /dev/null +++ b/xen/tools/cppcheck-plat/arm64-wchar_t2.xml @@ -0,0 +1,17 @@ + + + 8 + unsigned + + 2 + 4 + 8 + 8 + 4 + 8 + 16 + 8 + 8 + 2 + + diff --git a/xen/tools/cppcheck-plat/x86_64-wchar_t2.xml b/xen/tools/cppche= ck-plat/x86_64-wchar_t2.xml new file mode 100644 index 000000000000..0b3bb6e0fb8e --- /dev/null +++ b/xen/tools/cppcheck-plat/x86_64-wchar_t2.xml @@ -0,0 +1,17 @@ + + + 8 + signed + + 2 + 4 + 8 + 8 + 4 + 8 + 16 + 8 + 8 + 2 + + --=20 2.17.1 From nobody Fri May 17 01:43:50 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=arm.com Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1670418485957353.03068549228374; Wed, 7 Dec 2022 05:08:05 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.456302.714095 (Exim 4.92) (envelope-from ) id 1p2u8w-0004vB-UY; Wed, 07 Dec 2022 13:07:22 +0000 Received: by outflank-mailman (output) from mailman id 456302.714095; Wed, 07 Dec 2022 13:07:22 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p2u8w-0004uE-Nh; Wed, 07 Dec 2022 13:07:22 +0000 Received: by outflank-mailman (input) for mailman id 456302; Wed, 07 Dec 2022 13:07:20 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p2u8u-0004EJ-EO for xen-devel@lists.xenproject.org; Wed, 07 Dec 2022 13:07:20 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-sth1.inumbo.com (Halon) with ESMTP id 13ca6521-7630-11ed-91b6-6bf2151ebd3b; Wed, 07 Dec 2022 14:07:18 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5C34823A; Wed, 7 Dec 2022 05:07:24 -0800 (PST) Received: from e125770.cambridge.arm.com (e125770.cambridge.arm.com [10.1.195.16]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 6ACA63F73D; Wed, 7 Dec 2022 05:07:16 -0800 (PST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 13ca6521-7630-11ed-91b6-6bf2151ebd3b From: Luca Fancellu To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, wei.chen@arm.com, Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu Subject: [PATCH v3 3/5] xen/Makefile: remove Cppcheck invocation from the Makefile Date: Wed, 7 Dec 2022 13:07:02 +0000 Message-Id: <20221207130704.20573-4-luca.fancellu@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221207130704.20573-1-luca.fancellu@arm.com> References: <20221207130704.20573-1-luca.fancellu@arm.com> X-ZM-MESSAGEID: 1670418487362100001 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The script xen-analysis.py is going to be used for the analysis with cppcheck, so remove the rules from the Makefile. The python script xen/tools/merge_cppcheck_reports.py was used by the makefile rules, but its functionality is integrated in the xen-analysis.py script now, so it can be removed. Remove some entry from the .gitignore related to Cppcheck invocation from Makefile. This is a partial revert of commit 43aa3f6e72d34 where the entry xen/xen-cppcheck.xml from .gitignore is reverted because its path has changed, *.c.cppcheck entry is not needed anymore either, merge_cppcheck_reports.py script is reverted for the reason above, all makefile changes are reverted. Every other change to processor.h, config.h, kconfig.h is kept because they are used with the new process. This is a partial revert of commit 57caa53753212 where Makefile changes are reverted, convert_misra_doc.py script is kept as it is used with the new process. Signed-off-by: Luca Fancellu Acked-by: Jan Beulich --- v3 changes: - add information on reverted commits (Jan) v2 changes: - new patch --- --- .gitignore | 2 - xen/Makefile | 116 ++-------------------------- xen/tools/merge_cppcheck_reports.py | 86 --------------------- 3 files changed, 6 insertions(+), 198 deletions(-) delete mode 100755 xen/tools/merge_cppcheck_reports.py diff --git a/.gitignore b/.gitignore index 39efe2933a33..68566d0c2587 100644 --- a/.gitignore +++ b/.gitignore @@ -7,7 +7,6 @@ *.o *.d *.d2 -*.c.cppcheck *.cppcheck.txt *.cppcheck.xml *.opic @@ -321,7 +320,6 @@ tools/flask/policy/policy.conf tools/flask/policy/xenpolicy-* xen/xen xen/suppression-list.txt -xen/xen-cppcheck.xml xen/xen-syms xen/xen-syms.map xen/xen.* diff --git a/xen/Makefile b/xen/Makefile index e55fb98e11ee..2d55bb9401f4 100644 --- a/xen/Makefile +++ b/xen/Makefile @@ -457,7 +457,7 @@ endif # need-config =20 __all: build =20 -main-targets :=3D build install uninstall clean distclean MAP cppcheck cpp= check-html +main-targets :=3D build install uninstall clean distclean MAP .PHONY: $(main-targets) ifneq ($(XEN_TARGET_ARCH),x86_32) $(main-targets): %: _% ; @@ -566,18 +566,16 @@ _clean: $(Q)$(MAKE) $(clean)=3Dtools/kconfig find . \( -name "*.o" -o -name ".*.d" -o -name ".*.d2" \ -o -name ".*.o.tmp" -o -name "*~" -o -name "core" \ - -o -name '*.lex.c' -o -name '*.tab.[ch]' -o -name '*.c.cppcheck' \ - -o -name "*.gcno" -o -name ".*.cmd" -o -name "lib.a" \) -exec rm -f {} \; + -o -name '*.lex.c' -o -name '*.tab.[ch]' -o -name "*.gcno" \ + -o -name ".*.cmd" -o -name "lib.a" \) -exec rm -f {} \; rm -f include/asm $(TARGET) $(TARGET).gz $(TARGET)-syms $(TARGET)-syms.map rm -f $(TARGET).efi $(TARGET).efi.map $(TARGET).efi.stripped rm -f asm-offsets.s arch/*/include/asm/asm-offsets.h rm -f .banner .allconfig.tmp include/xen/compile.h - rm -f cppcheck-misra.* xen-cppcheck.xml =20 .PHONY: _distclean _distclean: clean rm -f tags TAGS cscope.files cscope.in.out cscope.out cscope.po.out GTAGS= GPATH GRTAGS GSYMS .config source - rm -rf $(CPPCHECK_HTMLREPORT_OUTDIR) =20 $(TARGET).gz: $(TARGET) gzip -n -f -9 < $< > $@.new @@ -651,111 +649,9 @@ cloc: done; \ done | cloc --list-file=3D- =20 -# What cppcheck command to use. -# To get proper results, it is recommended to build cppcheck manually from= the -# latest source and use CPPCHECK to give the full path to the built versio= n. -CPPCHECK ?=3D cppcheck - -# What cppcheck-htmlreport to use. -# If you give the full path to a self compiled cppcheck, this should be set -# to the full path to cppcheck-html in the htmlreport directory of cppchec= k. -# On recent distribution, this is available in the standard path. -CPPCHECK_HTMLREPORT ?=3D cppcheck-htmlreport - -# By default we generate the report in cppcheck-htmlreport directory in the -# build directory. This can be changed by giving a directory in this varia= ble. -CPPCHECK_HTMLREPORT_OUTDIR ?=3D cppcheck-htmlreport - -# By default we do not check misra rules, to enable pass "CPPCHECK_MISRA= =3Dy" to -# make command line. -CPPCHECK_MISRA ?=3D n - -# Compile flags to pass to cppcheck: -# - include directories and defines Xen Makefile is passing (from CFLAGS) -# - include config.h as this is passed directly to the compiler. -# - define CPPCHECK as we use to disable or enable some specific part of t= he -# code to solve some cppcheck issues. -# - explicitely enable some cppcheck checks as we do not want to use "all" -# which includes unusedFunction which gives wrong positives as we check = file -# per file. -# -# Compiler defines are in compiler-def.h which is included in config.h -# -CPPCHECKFLAGS :=3D -DCPPCHECK --max-ctu-depth=3D10 \ - --enable=3Dstyle,information,missingInclude \ - --include=3D$(srctree)/include/xen/config.h \ - -I $(srctree)/xsm/flask/include \ - -I $(srctree)/include/xen/libfdt \ - $(filter -D% -I%,$(CFLAGS)) - -# We need to find all C files (as we are not checking assembly files) so -# we find all generated .o files which have a .c corresponding file. -CPPCHECKFILES :=3D $(wildcard $(patsubst $(objtree)/%.o,$(srctree)/%.c, \ - $(filter-out $(objtree)/tools/%, \ - $(shell find $(objtree) -name "*.o")))) - -# Headers and files required to run cppcheck on a file -CPPCHECKDEPS :=3D $(objtree)/include/generated/autoconf.h \ - $(objtree)/include/generated/compiler-def.h - -ifeq ($(CPPCHECK_MISRA),y) - CPPCHECKFLAGS +=3D --addon=3Dcppcheck-misra.json - CPPCHECKDEPS +=3D cppcheck-misra.json -endif - -quiet_cmd_cppcheck_xml =3D CPPCHECK $(patsubst $(srctree)/%,%,$<) -cmd_cppcheck_xml =3D $(CPPCHECK) -v -q --xml $(CPPCHECKFLAGS) \ - --output-file=3D$@ $< - -quiet_cmd_merge_cppcheck_reports =3D CPPCHECK-MERGE $@ -cmd_merge_cppcheck_reports =3D $(PYTHON) $(srctree)/tools/merge_cppcheck_r= eports.py $^ $@ - -quiet_cmd_cppcheck_html =3D CPPCHECK-HTML $< -cmd_cppcheck_html =3D $(CPPCHECK_HTMLREPORT) --file=3D$< --source-dir=3D$(= srctree) \ - --report-dir=3D$(CPPCHECK_HTMLREPORT_OUTDIR) --title= =3DXen - -PHONY +=3D _cppcheck _cppcheck-html cppcheck-version - -_cppcheck-html: xen-cppcheck.xml - $(call if_changed,cppcheck_html) - -_cppcheck: xen-cppcheck.xml - -xen-cppcheck.xml: $(patsubst $(srctree)/%.c,$(objtree)/%.c.cppcheck,$(CPPC= HECKFILES)) -ifeq ($(CPPCHECKFILES),) - $(error Please build Xen before running cppcheck) -endif - $(call if_changed,merge_cppcheck_reports) - -$(objtree)/%.c.cppcheck: $(srctree)/%.c $(CPPCHECKDEPS) | cppcheck-version - $(call if_changed,cppcheck_xml) - -cppcheck-version: - $(Q)if ! which $(CPPCHECK) > /dev/null 2>&1; then \ - echo "Cannot find cppcheck executable: $(CPPCHECK)"; \ - exit 1; \ - fi - $(Q)if [ "$$($(CPPCHECK) --version | awk '{print ($$2 < 2.7)}')" -eq 1 ];= then \ - echo "Please upgrade your cppcheck to version 2.7 or greater"; \ - exit 1; \ - fi - -# List of Misra rules to respect is written inside a doc. -# In order to have some helpful text in the cppcheck output, generate a te= xt -# file containing the rules identifier, classification and text from the X= en -# documentation file. Also generate a json file with the right arguments f= or -# cppcheck in json format including the list of rules to ignore. -# -# convert_misra_doc.py, producing both targets at the same time, should be -# executed only once. Utilize a pattern rule to achieve this effect, with = the -# stem kind of arbitrarily chosen to be "cppcheck". -.PRECIOUS: %-misra.json -%-misra.txt %-misra.json: $(XEN_ROOT)/docs/misra/rules.rst $(srctree)/tool= s/convert_misra_doc.py - $(Q)$(PYTHON) $(srctree)/tools/convert_misra_doc.py -i $< -o $*-misra.txt= -j $*-misra.json - -# Put this in generated headers this way it is cleaned by include/Makefile -$(objtree)/include/generated/compiler-def.h: - $(Q)$(CC) -dM -E -o $@ - < /dev/null +# Target used by xen-analysis.sh script to retrieve Xen build system varia= bles +export-variable-%: + $(info $*=3D$($*)) =20 endif #config-build endif # need-sub-make diff --git a/xen/tools/merge_cppcheck_reports.py b/xen/tools/merge_cppcheck= _reports.py deleted file mode 100755 index 1c1b63ba56b8..000000000000 --- a/xen/tools/merge_cppcheck_reports.py +++ /dev/null @@ -1,86 +0,0 @@ -#!/usr/bin/env python - -""" -This script acts as a tool to merge XML files created by cppcheck. -Usage: - merge_cppcheck_reports.py [FILES] [OUTPUT] - - FILES - list of XML files with extension .cppcheck - OUTPUT - file to store results (with .xml extension). - If not specified, the script will print results to stdout. -""" - -import sys -from xml.etree import ElementTree - -def elements_equal(el1, el2): - if type(el1) !=3D type(el2): return False - - if el1.find('location') is None: return False - if el2.find('location') is None: return False - - el1_location =3D str(el1.find('location').attrib) - el2_location =3D str(el2.find('location').attrib) - - if el1_location !=3D el2_location: return False - - return True - -def contain_element(new, lst): - for elem in lst: - if elements_equal(new, elem): - return True - return False - -def merge(files): - try: - result_xml_root =3D ElementTree.parse(files[0]).getroot() - except: - print("Xml parsing error in %s\n" % (files[0])) - print("Please upgrade your cppcheck to version 2.7 or greater") - sys.exit(1) - insert_point =3D result_xml_root.findall("errors")[0] - curr =3D 1 - total =3D len(files) - numelem =3D len(insert_point) - for xml_file in files[1:]: - try: - xml_root =3D ElementTree.parse(xml_file).getroot() - except: - print("Xml parsing error in %s\n" % (xml_file)) - print("Please upgrade your cppcheck to version 2.7 or greater") - sys.exit(1) - curr_elem_list =3D list(insert_point) - new_elem_list =3D list(xml_root.findall("errors")[0]) - for xml_error_elem in new_elem_list: - if not contain_element(xml_error_elem, curr_elem_list): - insert_point.insert(1,xml_error_elem) - numelem =3D numelem + 1 - curr =3D curr + 1 - sys.stdout.write('\r') - sys.stdout.write(" %d / %d" % (curr,total)) - sys.stdout.flush() - - sys.stdout.write('\r\n') - print("Done: %d elements" % (numelem)) - return result_xml_root - -def run(): - files =3D [] - output =3D None - for i in sys.argv[1:]: - output =3D i if '.xml' in i else None - files.append(i) if '.cppcheck' in i else None - - result =3D merge(files) - - if result is None: - return - - if output is not None: - ElementTree.ElementTree(result).write(output) - else: - print(ElementTree.tostring(result).decode('utf-8')) - -if __name__ =3D=3D '__main__': - run() --=20 2.17.1 From nobody Fri May 17 01:43:50 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=arm.com Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1670418485868471.477646526399; Wed, 7 Dec 2022 05:08:05 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.456301.714087 (Exim 4.92) (envelope-from ) id 1p2u8w-0004mS-HR; Wed, 07 Dec 2022 13:07:22 +0000 Received: by outflank-mailman (output) from mailman id 456301.714087; Wed, 07 Dec 2022 13:07:22 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p2u8w-0004mH-CL; Wed, 07 Dec 2022 13:07:22 +0000 Received: by outflank-mailman (input) for mailman id 456301; Wed, 07 Dec 2022 13:07:20 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p2u8u-0004CY-6F for xen-devel@lists.xenproject.org; Wed, 07 Dec 2022 13:07:20 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-flk1.inumbo.com (Halon) with ESMTP id 1485c9e6-7630-11ed-8fd2-01056ac49cbb; Wed, 07 Dec 2022 14:07:19 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C48A013D5; Wed, 7 Dec 2022 05:07:25 -0800 (PST) Received: from e125770.cambridge.arm.com (e125770.cambridge.arm.com [10.1.195.16]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id D2F173F73D; Wed, 7 Dec 2022 05:07:17 -0800 (PST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 1485c9e6-7630-11ed-8fd2-01056ac49cbb From: Luca Fancellu To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, wei.chen@arm.com, Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu Subject: [PATCH v3 4/5] tools/misra: fix skipped rule numbers Date: Wed, 7 Dec 2022 13:07:03 +0000 Message-Id: <20221207130704.20573-5-luca.fancellu@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221207130704.20573-1-luca.fancellu@arm.com> References: <20221207130704.20573-1-luca.fancellu@arm.com> X-ZM-MESSAGEID: 1670418487471100007 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" MISRA rules are in the format Rule X.Y, currently the script convert_misra_doc.py is using two nested loop through range(1,22) to enumerate rules that needs to be skipped, using combination of X.Y in that range, however there are two issues in the code: - rule 22 is never included because the range(1,22) produces a range in [1..21] - the second issue is that the code is producing invalid MISRA C 2012 rules, for example 1.21 and so on Fix the issue using a dictionary that list the rules in misra c2012. Fixes: 57caa5375321 ("xen: Add MISRA support to cppcheck make rule") Signed-off-by: Luca Fancellu Reviewed-by: Stefano Stabellini --- v3 changes: - none Changes in v2: - modified commit message, add R-by (Stefano) --- --- xen/tools/convert_misra_doc.py | 32 ++++++++++++++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-) diff --git a/xen/tools/convert_misra_doc.py b/xen/tools/convert_misra_doc.py index caa4487f645f..13074d8a2e91 100755 --- a/xen/tools/convert_misra_doc.py +++ b/xen/tools/convert_misra_doc.py @@ -14,6 +14,34 @@ Usage: =20 import sys, getopt, re =20 +# MISRA rule are identified by two numbers, e.g. Rule 1.2, the main rule n= umber +# and a sub-number. This dictionary contains the number of the MISRA rule = as key +# and the maximum sub-number for that rule as value. +misra_c2012_rules =3D { + 1:4, + 2:7, + 3:2, + 4:2, + 5:9, + 6:2, + 7:4, + 8:14, + 9:5, + 10:8, + 11:9, + 12:5, + 13:6, + 14:4, + 15:7, + 16:7, + 17:8, + 18:8, + 19:2, + 20:14, + 21:21, + 22:10 +} + def main(argv): infile =3D '' outfile =3D '' @@ -142,8 +170,8 @@ def main(argv): skip_list =3D [] =20 # Search for missing rules and add a dummy text with the rule number - for i in list(range(1,22)): - for j in list(range(1,22)): + for i in misra_c2012_rules: + for j in list(range(1,misra_c2012_rules[i]+1)): if str(i) + '.' + str(j) not in rule_list: outstr.write('Rule ' + str(i) + '.' + str(j) + '\n') outstr.write('No description for rule ' + str(i) + '.' + s= tr(j) --=20 2.17.1 From nobody Fri May 17 01:43:50 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=arm.com Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 167041848822943.755000413187645; Wed, 7 Dec 2022 05:08:08 -0800 (PST) Received: from list by lists.xenproject.org with outflank-mailman.456303.714103 (Exim 4.92) (envelope-from ) id 1p2u8x-000593-Lj; Wed, 07 Dec 2022 13:07:23 +0000 Received: by outflank-mailman (output) from mailman id 456303.714103; Wed, 07 Dec 2022 13:07:23 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p2u8x-00055z-CI; Wed, 07 Dec 2022 13:07:23 +0000 Received: by outflank-mailman (input) for mailman id 456303; Wed, 07 Dec 2022 13:07:21 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1p2u8v-0004CY-Rf for xen-devel@lists.xenproject.org; Wed, 07 Dec 2022 13:07:21 +0000 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by se1-gles-flk1.inumbo.com (Halon) with ESMTP id 15678740-7630-11ed-8fd2-01056ac49cbb; Wed, 07 Dec 2022 14:07:21 +0100 (CET) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 38A87D6E; Wed, 7 Dec 2022 05:07:27 -0800 (PST) Received: from e125770.cambridge.arm.com (e125770.cambridge.arm.com [10.1.195.16]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 46FA73F73D; Wed, 7 Dec 2022 05:07:19 -0800 (PST) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 15678740-7630-11ed-8fd2-01056ac49cbb From: Luca Fancellu To: xen-devel@lists.xenproject.org Cc: bertrand.marquis@arm.com, wei.chen@arm.com, Andrew Cooper , George Dunlap , Jan Beulich , Julien Grall , Stefano Stabellini , Wei Liu Subject: [PATCH v3 5/5] xen: Justify linker script defined symbols in include/xen/kernel.h Date: Wed, 7 Dec 2022 13:07:04 +0000 Message-Id: <20221207130704.20573-6-luca.fancellu@arm.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20221207130704.20573-1-luca.fancellu@arm.com> References: <20221207130704.20573-1-luca.fancellu@arm.com> X-ZM-MESSAGEID: 1670418489249100011 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Eclair and Coverity found violation of the MISRA rule 8.6 for the symbols _start, _end, start, _stext, _etext, _srodata, _erodata, _sinittext, _einittext which are declared in xen/include/xen/kernel.h. All those symbols are defined by the liker script so we can deviate from the rule 8.6 for these cases. Signed-off-by: Luca Fancellu Reviewed-by: Stefano Stabellini Acked-by: Jan Beulich --- v3 changes: - none v2 changes: - add R-by and Ack-by (Stefano, Jan) --- --- docs/misra/safe.json | 9 +++++++++ xen/include/xen/kernel.h | 4 ++++ 2 files changed, 13 insertions(+) diff --git a/docs/misra/safe.json b/docs/misra/safe.json index e079d3038120..e3c8a1d8eb36 100644 --- a/docs/misra/safe.json +++ b/docs/misra/safe.json @@ -3,6 +3,15 @@ "content": [ { "id": "SAF-0-safe", + "analyser": { + "eclair": "MC3R1.R8.6", + "coverity": "misra_c_2012_rule_8_6_violation" + }, + "name": "Rule 8.6: linker script defined symbols", + "text": "It is safe to declare this symbol because it is defin= ed in the linker script." + }, + { + "id": "SAF-1-safe", "analyser": {}, "name": "Sentinel", "text": "Next ID to be used" diff --git a/xen/include/xen/kernel.h b/xen/include/xen/kernel.h index 8cd142032d3b..f1a7713784fc 100644 --- a/xen/include/xen/kernel.h +++ b/xen/include/xen/kernel.h @@ -65,24 +65,28 @@ 1; \ }) =20 +/* SAF-0-safe */ extern char _start[], _end[], start[]; #define is_kernel(p) ({ \ char *__p =3D (char *)(unsigned long)(p); \ (__p >=3D _start) && (__p < _end); \ }) =20 +/* SAF-0-safe */ extern char _stext[], _etext[]; #define is_kernel_text(p) ({ \ char *__p =3D (char *)(unsigned long)(p); \ (__p >=3D _stext) && (__p < _etext); \ }) =20 +/* SAF-0-safe */ extern const char _srodata[], _erodata[]; #define is_kernel_rodata(p) ({ \ const char *__p =3D (const char *)(unsigned long)(p); \ (__p >=3D _srodata) && (__p < _erodata); \ }) =20 +/* SAF-0-safe */ extern char _sinittext[], _einittext[]; #define is_kernel_inittext(p) ({ \ char *__p =3D (char *)(unsigned long)(p); \ --=20 2.17.1