From nobody Sun Feb 8 17:47:07 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A9DE535B638; Sat, 31 Jan 2026 14:25:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769869522; cv=none; b=FuWEdT441tzclI/Mq/NQPYVrhS/X6u8EciedPxffAfTqG2Rq6tQqNFvXTeB+VULI+GfGFK70aH47kwVHmnQt6w3LXce6zf2+bsCx0L4KuJpF4+O0WasbDB3xvZhKGVLl1PgV5nc5EHEtZvLN34rpcuuomI8X9OHd35SVCpojCSE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1769869522; c=relaxed/simple; bh=JgAESjeyMdGBTP/DT74Q19UkIkpbBQsD4O6RYrpw4MQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ZuTGCed9RDlz2hO8e/NSNrdP8hHIIzwTlTnaaRtzHe8cbKoPRqV+CWYCeo11hf13KdZU1W1wyxf/4tjeHsHjMp4R6B5bj3VNVXIiR6+E8Ryp1nW3Ae0WjsGGmMX5AS+zAp8ACOvpkohA0/30eYs7UCuQ+SyYDvczl7eRpaUtSR4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=FWjwGdDD; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="FWjwGdDD" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2BFFCC2BCAF; Sat, 31 Jan 2026 14:25:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1769869522; bh=JgAESjeyMdGBTP/DT74Q19UkIkpbBQsD4O6RYrpw4MQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=FWjwGdDDFTueSCv2dGBxIxcKTvz6ucwoQyWv2j0Enk5xwhFFzMr/Ai1X7nhp7J0Zn KUZWXHrUjbulh7pnOD45vi2mjdH9u1F+3y55NKPPkrYzNy4Svo+LXOhE95ucYTCkqs zm4i6CzQop1iYv6z8oZ8TnQOhSk/9BLdRXK06mDyPMJnjoIWWrE8MBP43XiAHGqFlT sVOK8sBbmRg991tJEBDsw1JPHerSy4TVVL3/W1Lm52j+xgKTOMWxuYO30ST/Bj8CoK tIyYF3hyUU0JvsJ+CCR7VWRAzO/+o1rg53sCKQ57wQ0PxRSyOm/afznVCo5F8higx+ OCD2IcsBZSeGA== Received: from mchehab by mail.kernel.org with local (Exim 4.99.1) (envelope-from ) id 1vmBua-0000000AVmN-1hvH; Sat, 31 Jan 2026 15:25:20 +0100 From: Mauro Carvalho Chehab To: Jonathan Corbet , Linux Doc Mailing List Cc: Mauro Carvalho Chehab , intel-wired-lan@lists.osuosl.org, linux-hardening@vger.kernel.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, Mauro Carvalho Chehab , Randy Dunlap , Shuah Khan Subject: [PATCH v4 40/41] docs: python: add helpers to run unit tests Date: Sat, 31 Jan 2026 15:25:14 +0100 Message-ID: X-Mailer: git-send-email 2.52.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Sender: Mauro Carvalho Chehab While python internal libraries have support for unit tests, its output is not nice. Add a helper module to improve its output. I wrote this module last year while testing some scripts I used internally. The initial skeleton was generated with the help of LLM tools, but it was higly modified to ensure that it will work as I would expect. Signed-off-by: Mauro Carvalho Chehab --- Documentation/tools/python.rst | 2 + Documentation/tools/unittest.rst | 24 ++ tools/lib/python/unittest_helper.py | 348 ++++++++++++++++++++++++++++ 3 files changed, 374 insertions(+) create mode 100644 Documentation/tools/unittest.rst create mode 100755 tools/lib/python/unittest_helper.py diff --git a/Documentation/tools/python.rst b/Documentation/tools/python.rst index 1444c1816735..3b7299161f20 100644 --- a/Documentation/tools/python.rst +++ b/Documentation/tools/python.rst @@ -11,3 +11,5 @@ Python libraries feat kdoc kabi + + unittest diff --git a/Documentation/tools/unittest.rst b/Documentation/tools/unittes= t.rst new file mode 100644 index 000000000000..14a2b2a65236 --- /dev/null +++ b/Documentation/tools/unittest.rst @@ -0,0 +1,24 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +Python unittest +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +Checking consistency of python modules can be complex. Sometimes, it is +useful to define a set of unit tests to help checking them. + +While the actual test implementation is usecase dependent, Python already +provides a standard way to add unit tests by using ``import unittest``. + +Using such class, requires setting up a test suite. Also, the default form= at +is a little bit ackward. To improve it and provide a more uniform way to +report errors, some unittest classes and functions are defined. + + +Unittest helper module +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +.. automodule:: lib.python.unittest_helper + :members: + :show-inheritance: + :undoc-members: diff --git a/tools/lib/python/unittest_helper.py b/tools/lib/python/unittes= t_helper.py new file mode 100755 index 000000000000..d2efb78d8561 --- /dev/null +++ b/tools/lib/python/unittest_helper.py @@ -0,0 +1,348 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: GPL-2.0 +# Copyright(c) 2025-2026: Mauro Carvalho Chehab . +# +# pylint: disable=3DC0103,R0912,R0914,E1101 + +""" +Provides helper functions and classes execute python unit tests. + +Those help functions provide a nice colored output summary of each +executed test and, when a test fails, it shows the different in diff +format when running in verbose mode, like:: + + $ tools/unittests/nested_match.py -v + ... + Traceback (most recent call last): + File "/new_devel/docs/tools/unittests/nested_match.py", line 69, in te= st_count_limit + self.assertEqual(replaced, "bar(a); bar(b); foo(c)") + ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + AssertionError: 'bar(a) foo(b); foo(c)' !=3D 'bar(a); bar(b); foo(c)' + - bar(a) foo(b); foo(c) + ? ^^^^ + + bar(a); bar(b); foo(c) + ? ^^^^^ + ... + +It also allows filtering what tests will be executed via ``-k`` parameter. + +Typical usage is to do:: + + from unittest_helper import run_unittest + ... + + if __name__ =3D=3D "__main__": + run_unittest(__file__) + +If passing arguments is needed, on a more complex scenario, it can be +used like on this example:: + + from unittest_helper import TestUnits, run_unittest + ... + env =3D {'sudo': ""} + ... + if __name__ =3D=3D "__main__": + runner =3D TestUnits() + base_parser =3D runner.parse_args() + base_parser.add_argument('--sudo', action=3D'store_true', + help=3D'Enable tests requiring sudo privil= eges') + + args =3D base_parser.parse_args() + + # Update module-level flag + if args.sudo: + env['sudo'] =3D "1" + + # Run tests with customized arguments + runner.run(__file__, parser=3Dbase_parser, args=3Dargs, env=3Denv) +""" + +import argparse +import atexit +import os +import re +import unittest +import sys + +from unittest.mock import patch + + +class Summary(unittest.TestResult): + """ + Overrides ``unittest.TestResult`` class to provide a nice colored + summary. When in verbose mode, displays actual/expected difference in + unified diff format. + """ + def __init__(self, *args, **kwargs): + super().__init__(*args, **kwargs) + + #: Dictionary to store organized test results. + self.test_results =3D {} + + #: max length of the test names. + self.max_name_length =3D 0 + + def startTest(self, test): + super().startTest(test) + test_id =3D test.id() + parts =3D test_id.split(".") + + # Extract module, class, and method names + if len(parts) >=3D 3: + module_name =3D parts[-3] + else: + module_name =3D "" + if len(parts) >=3D 2: + class_name =3D parts[-2] + else: + class_name =3D "" + + method_name =3D parts[-1] + + # Build the hierarchical structure + if module_name not in self.test_results: + self.test_results[module_name] =3D {} + + if class_name not in self.test_results[module_name]: + self.test_results[module_name][class_name] =3D [] + + # Track maximum test name length for alignment + display_name =3D f"{method_name}:" + + self.max_name_length =3D max(len(display_name), self.max_name_leng= th) + + def _record_test(self, test, status): + test_id =3D test.id() + parts =3D test_id.split(".") + if len(parts) >=3D 3: + module_name =3D parts[-3] + else: + module_name =3D "" + if len(parts) >=3D 2: + class_name =3D parts[-2] + else: + class_name =3D "" + method_name =3D parts[-1] + self.test_results[module_name][class_name].append((method_name, st= atus)) + + def addSuccess(self, test): + super().addSuccess(test) + self._record_test(test, "OK") + + def addFailure(self, test, err): + super().addFailure(test, err) + self._record_test(test, "FAIL") + + def addError(self, test, err): + super().addError(test, err) + self._record_test(test, "ERROR") + + def addSkip(self, test, reason): + super().addSkip(test, reason) + self._record_test(test, f"SKIP ({reason})") + + def printResults(self): + """ + Print results using colors if tty. + """ + # Check for ANSI color support + use_color =3D sys.stdout.isatty() + COLORS =3D { + "OK": "\033[32m", # Green + "FAIL": "\033[31m", # Red + "SKIP": "\033[1;33m", # Yellow + "PARTIAL": "\033[33m", # Orange + "EXPECTED_FAIL": "\033[36m", # Cyan + "reset": "\033[0m", # Reset to default terminal col= or + } + if not use_color: + for c in COLORS: + COLORS[c] =3D "" + + # Calculate maximum test name length + if not self.test_results: + return + try: + lengths =3D [] + for module in self.test_results.values(): + for tests in module.values(): + for test_name, _ in tests: + lengths.append(len(test_name) + 1) # +1 for colon + max_length =3D max(lengths) + 2 # Additional padding + except ValueError: + sys.exit("Test list is empty") + + # Print results + for module_name, classes in self.test_results.items(): + print(f"{module_name}:") + for class_name, tests in classes.items(): + print(f" {class_name}:") + for test_name, status in tests: + # Get base status without reason for SKIP + if status.startswith("SKIP"): + status_code =3D status.split()[0] + else: + status_code =3D status + color =3D COLORS.get(status_code, "") + print( + f" {test_name + ':':<{max_length}}{color}{s= tatus}{COLORS['reset']}" + ) + print() + + # Print summary + print(f"\nRan {self.testsRun} tests", end=3D"") + if hasattr(self, "timeTaken"): + print(f" in {self.timeTaken:.3f}s", end=3D"") + print() + + if not self.wasSuccessful(): + print(f"\n{COLORS['FAIL']}FAILED (", end=3D"") + failures =3D getattr(self, "failures", []) + errors =3D getattr(self, "errors", []) + if failures: + print(f"failures=3D{len(failures)}", end=3D"") + if errors: + if failures: + print(", ", end=3D"") + print(f"errors=3D{len(errors)}", end=3D"") + print(f"){COLORS['reset']}") + + +def flatten_suite(suite): + """Flatten test suite hierarchy.""" + tests =3D [] + for item in suite: + if isinstance(item, unittest.TestSuite): + tests.extend(flatten_suite(item)) + else: + tests.append(item) + return tests + + +class TestUnits: + """ + Helper class to set verbosity level. + + This class discover test files, import its unittest classes and + executes the test on it. + """ + def parse_args(self): + """Returns a parser for command line arguments.""" + parser =3D argparse.ArgumentParser(description=3D"Test runner with= regex filtering") + parser.add_argument("-v", "--verbose", action=3D"count", default= =3D1) + parser.add_argument("-f", "--failfast", action=3D"store_true") + parser.add_argument("-k", "--keyword", + help=3D"Regex pattern to filter test methods") + return parser + + def run(self, caller_file=3DNone, pattern=3DNone, + suite=3DNone, parser=3DNone, args=3DNone, env=3DNone): + """ + Execute all tests from the unity test file. + + It contains several optional parameters: + + ``caller_file``: + - name of the file that contains test. + + typical usage is to place __file__ at the caller test, e.g.= :: + + if __name__ =3D=3D "__main__": + TestUnits().run(__file__) + + ``pattern``: + - optional pattern to match multiple file names. Defaults + to basename of ``caller_file``. + + ``suite``: + - an unittest suite initialized by the caller using + ``unittest.TestLoader().discover()``. + + ``parser``: + - an argparse parser. If not defined, this helper will create + one. + + ``args``: + - an ``argparse.Namespace`` data filled by the caller. + + ``env``: + - environment variables that will be passed to the test suite + + At least ``caller_file`` or ``suite`` must be used, otherwise a + ``TypeError`` will be raised. + """ + if not args: + if not parser: + parser =3D self.parse_args() + args =3D parser.parse_args() + + if not caller_file and not suite: + raise TypeError("Either caller_file or suite is needed at Test= Units") + + if env: + patcher =3D patch.dict(os.environ, env) + patcher.start() + # ensure it gets stopped after + atexit.register(patcher.stop) + + verbose =3D args.verbose + + if verbose >=3D 2: + unittest.TextTestRunner(verbosity=3Dverbose).run =3D lambda su= ite: suite + + # Load ONLY tests from the calling file + if not suite: + if not pattern: + pattern =3D caller_file + + loader =3D unittest.TestLoader() + suite =3D loader.discover(start_dir=3Dos.path.dirname(caller_f= ile), + pattern=3Dos.path.basename(caller_file= )) + + # Flatten the suite for environment injection + tests_to_inject =3D flatten_suite(suite) + + # Filter tests by method name if -k specified + if args.keyword: + try: + pattern =3D re.compile(args.keyword) + filtered_suite =3D unittest.TestSuite() + for test in tests_to_inject: # Use the pre-flattened list + method_name =3D test.id().split(".")[-1] + if pattern.search(method_name): + filtered_suite.addTest(test) + suite =3D filtered_suite + except re.error as e: + sys.stderr.write(f"Invalid regex pattern: {e}\n") + sys.exit(1) + else: + # Maintain original suite structure if no keyword filtering + suite =3D unittest.TestSuite(tests_to_inject) + + if verbose >=3D 2: + resultclass =3D None + else: + resultclass =3D Summary + + runner =3D unittest.TextTestRunner(verbosity=3Dargs.verbose, + resultclass=3Dresultclass, + failfast=3Dargs.failfast) + result =3D runner.run(suite) + if resultclass: + result.printResults() + + sys.exit(not result.wasSuccessful()) + + +def run_unittest(fname): + """ + Basic usage of TestUnits class. + + Use it when there's no need to pass any extra argument to the tests + with. The recommended way is to place this at the end of each + unittest module:: + + if __name__ =3D=3D "__main__": + run_unittest(__file__) + """ + TestUnits().run(fname) --=20 2.52.0