From nobody Wed Jul 1 18:37:12 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2A836C433F5 for ; Fri, 17 Dec 2021 04:37:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232727AbhLQEhx (ORCPT ); Thu, 16 Dec 2021 23:37:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48660 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232634AbhLQEhv (ORCPT ); Thu, 16 Dec 2021 23:37:51 -0500 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9B123C06173E for ; Thu, 16 Dec 2021 20:37:51 -0800 (PST) Received: by mail-yb1-xb4a.google.com with SMTP id q15-20020a252a0f000000b006092c637e21so2549865ybq.5 for ; Thu, 16 Dec 2021 20:37:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc:content-transfer-encoding; bh=FgvhjAy2U9ZMb/cMiyQmvKherI8tilVgiwLz3YqJS2k=; b=lVT4Wwt1od8mrUmPfzNAKQD1TrLG3RYn5+kD+ZYwwYoPZmGapa+mDiGDUiLR5O5Iff 8VSBLiJtaXF8HPnvb+2xI2sBn/s7FIS3kzN+qKWaXfag14tThIvJDiqH4yU/Ee1PJ7Jf FInqbLjpBYnD1gvokiGwmdIr4w5ReKch8azr832qzLIIZGpfhNQQYDiRobDylPusD7t2 DTMv2cUzh7z8qF+SpaargZ1l96f3QsLg1/xvQH05cgEtHkiXXbkiMcj/Vm767c9G3H4t eQj6l2FPRDQff7Vvd5eGlIyFeQrmYmf3m5UK0jOoQcOagKleq5tHycKjmZlmKDmadLYn gjJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc:content-transfer-encoding; bh=FgvhjAy2U9ZMb/cMiyQmvKherI8tilVgiwLz3YqJS2k=; b=d1MjvR0veV6uyomB5aTmsPB+jGPzBhFCKMiJag1BXChZchta7JOrgsO6V7xXXC6O8s MQpFwDlujvfFohQExgHBZ0/VzKhIOC4jiaAoWGzAMHFTknEppM7qi9pa+wiz4/RkJVQq PC/SkNN2K47Zu3vVvSek7vvxCIW1FiMHgEyDKiKSBisExbnu+343zYtRCruif2LHLPaG 4FvpqFAqLmZ/xoRohNYO8UhA3AAk3e8NZ0uHTN5IFAVdiwaStNT+OU5zFNTSntEKkmmL uabDTRwDM1szTiBvlfQ2M9QJlcBXqPdar4l0JLBM+sDaGwghkrHc6tYmoNw27QBSNcr3 y16g== X-Gm-Message-State: AOAM531MgctyeKEAcWDvDqeKWDq4UnzMV5w3UGDweratlFzTj4w9xM7f g/EajnmFGjq2jCZcvYc/+mvTPiOSMxQn2ZE= X-Google-Smtp-Source: ABdhPJxHs2f/yJGlFIXsfUb4V699/lbzH0qg35gOTk2BYKrMYiNq46buXjFPYM3qvfo8C5Zi75QEG96+9wTJ7fc= X-Received: from sharinder.c.googlers.com ([fda3:e722:ac3:cc00:4f:4b78:c0a8:c73]) (user=sharinder job=sendgmr) by 2002:a05:6902:1007:: with SMTP id w7mr2101192ybt.718.1639715870851; Thu, 16 Dec 2021 20:37:50 -0800 (PST) Date: Fri, 17 Dec 2021 04:37:10 +0000 In-Reply-To: <20211217043716.794289-1-sharinder@google.com> Message-Id: <20211217043716.794289-2-sharinder@google.com> Mime-Version: 1.0 References: <20211217043716.794289-1-sharinder@google.com> X-Mailer: git-send-email 2.34.1.173.g76aa8bc2d0-goog Subject: [PATCH v5 1/7] Documentation: KUnit: Rewrite main page From: Harinder Singh To: davidgow@google.com, brendanhiggins@google.com, shuah@kernel.org, corbet@lwn.net Cc: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, tim.bird@sony.com, elver@google.com, Harinder Singh Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Add a section on advantages of unit testing, how to write unit tests, KUnit features and Prerequisites. Signed-off-by: Harinder Singh --- Documentation/dev-tools/kunit/index.rst | 166 +++++++++++++----------- 1 file changed, 88 insertions(+), 78 deletions(-) diff --git a/Documentation/dev-tools/kunit/index.rst b/Documentation/dev-to= ols/kunit/index.rst index cacb35ec658d..55d2444b0745 100644 --- a/Documentation/dev-tools/kunit/index.rst +++ b/Documentation/dev-tools/kunit/index.rst @@ -1,11 +1,12 @@ .. SPDX-License-Identifier: GPL-2.0 =20 -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -KUnit - Unit Testing for the Linux Kernel -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D +KUnit - Linux Kernel Unit Testing +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D =20 .. toctree:: :maxdepth: 2 + :caption: Contents: =20 start usage @@ -16,82 +17,91 @@ KUnit - Unit Testing for the Linux Kernel tips running_tips =20 -What is KUnit? -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D - -KUnit is a lightweight unit testing and mocking framework for the Linux ke= rnel. - -KUnit is heavily inspired by JUnit, Python's unittest.mock, and -Googletest/Googlemock for C++. KUnit provides facilities for defining unit= test -cases, grouping related test cases into test suites, providing common -infrastructure for running tests, and much more. - -KUnit consists of a kernel component, which provides a set of macros for e= asily -writing unit tests. Tests written against KUnit will run on kernel boot if -built-in, or when loaded if built as a module. These tests write out resul= ts to -the kernel log in `TAP `_ format. - -To make running these tests (and reading the results) easier, KUnit offers -:doc:`kunit_tool `, which builds a `User Mode Linux -`_ kernel, runs it, and parses the= test -results. This provides a quick way of running KUnit tests during developme= nt, -without requiring a virtual machine or separate hardware. - -Get started now: Documentation/dev-tools/kunit/start.rst - -Why KUnit? -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D - -A unit test is supposed to test a single unit of code in isolation, hence = the -name. A unit test should be the finest granularity of testing and as such = should -allow all possible code paths to be tested in the code under test; this is= only -possible if the code under test is very small and does not have any extern= al -dependencies outside of the test's control like hardware. - -KUnit provides a common framework for unit tests within the kernel. - -KUnit tests can be run on most architectures, and most tests are architect= ure -independent. All built-in KUnit tests run on kernel startup. Alternativel= y, -KUnit and KUnit tests can be built as modules and tests will run when the = test -module is loaded. - -.. note:: - - KUnit can also run tests without needing a virtual machine or actu= al - hardware under User Mode Linux. User Mode Linux is a Linux archite= cture, - like ARM or x86, which compiles the kernel as a Linux executable. = KUnit - can be used with UML either by building with ``ARCH=3Dum`` (like a= ny other - architecture), or by using :doc:`kunit_tool `. - -KUnit is fast. Excluding build time, from invocation to completion KUnit c= an run -several dozen tests in only 10 to 20 seconds; this might not sound like a = big -deal to some people, but having such fast and easy to run tests fundamenta= lly -changes the way you go about testing and even writing code in the first pl= ace. -Linus himself said in his `git talk at Google -`_: - - "... a lot of people seem to think that performance is about doing the - same thing, just doing it faster, and that is not true. That is not what - performance is all about. If you can do something really fast, really - well, people will start using it differently." - -In this context Linus was talking about branching and merging, -but this point also applies to testing. If your tests are slow, unreliable= , are -difficult to write, and require a special setup or special hardware to run, -then you wait a lot longer to write tests, and you wait a lot longer to run -tests; this means that tests are likely to break, unlikely to test a lot of -things, and are unlikely to be rerun once they pass. If your tests are rea= lly -fast, you run them all the time, every time you make a change, and every t= ime -someone sends you some code. Why trust that someone ran all their tests -correctly on every change when you can just run them yourself in less time= than -it takes to read their test log? +This section details the kernel unit testing framework. + +Introduction +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +KUnit (Kernel unit testing framework) provides a common framework for +unit tests within the Linux kernel. Using KUnit, you can define groups +of test cases called test suites. The tests either run on kernel boot +if built-in, or load as a module. KUnit automatically flags and reports +failed test cases in the kernel log. The test results appear in `TAP +(Test Anything Protocol) format `_. It is inspi= red by +JUnit, Python=E2=80=99s unittest.mock, and GoogleTest/GoogleMock (C++ unit= testing +framework). + +KUnit tests are part of the kernel, written in the C (programming) +language, and test parts of the Kernel implementation (example: a C +language function). Excluding build time, from invocation to +completion, KUnit can run around 100 tests in less than 10 seconds. +KUnit can test any kernel component, for example: file system, system +calls, memory management, device drivers and so on. + +KUnit follows the white-box testing approach. The test has access to +internal system functionality. KUnit runs in kernel space and is not +restricted to things exposed to user-space. + +In addition, KUnit has kunit_tool, a script (``tools/testing/kunit/kunit.p= y``) +that configures the Linux kernel, runs KUnit tests under QEMU or UML (`Use= r Mode +Linux `_), parses the test result= s and +displays them in a user friendly manner. + +Features +-------- + +- Provides a framework for writing unit tests. +- Runs tests on any kernel architecture. +- Runs a test in milliseconds. + +Prerequisites +------------- + +- Any Linux kernel compatible hardware. +- For Kernel under test, Linux kernel version 5.5 or greater. + +Unit Testing +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +A unit test tests a single unit of code in isolation. A unit test is the f= inest +granularity of testing and allows all possible code paths to be tested in = the +code under test. This is possible if the code under test is small and does= not +have any external dependencies outside of the test's control like hardware. + + +Write Unit Tests +---------------- + +To write good unit tests, there is a simple but powerful pattern: +Arrange-Act-Assert. This is a great way to structure test cases and +defines an order of operations. + +- Arrange inputs and targets: At the start of the test, arrange the data + that allows a function to work. Example: initialize a statement or + object. +- Act on the target behavior: Call your function/code under test. +- Assert expected outcome: Verify that the result (or resulting state) is = as + expected. + +Unit Testing Advantages +----------------------- + +- Increases testing speed and development in the long run. +- Detects bugs at initial stage and therefore decreases bug fix cost + compared to acceptance testing. +- Improves code quality. +- Encourages writing testable code. =20 How do I use it? =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 -* Documentation/dev-tools/kunit/start.rst - for new users of KUnit -* Documentation/dev-tools/kunit/tips.rst - for short examples of best pr= actices -* Documentation/dev-tools/kunit/usage.rst - for a more detailed explanat= ion of KUnit features -* Documentation/dev-tools/kunit/api/index.rst - for the list of KUnit AP= Is used for testing -* Documentation/dev-tools/kunit/kunit-tool.rst - for more information on= the kunit_tool helper script -* Documentation/dev-tools/kunit/faq.rst - for answers to some common que= stions about KUnit +* Documentation/dev-tools/kunit/start.rst - for KUnit new users. +* Documentation/dev-tools/kunit/usage.rst - KUnit features. +* Documentation/dev-tools/kunit/tips.rst - best practices with + examples. +* Documentation/dev-tools/kunit/api/index.rst - KUnit APIs + used for testing. +* Documentation/dev-tools/kunit/kunit-tool.rst - kunit_tool helper + script. +* Documentation/dev-tools/kunit/faq.rst - KUnit common questions and + answers. --=20 2.34.1.173.g76aa8bc2d0-goog From nobody Wed Jul 1 18:37:12 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 76369C433FE for ; Fri, 17 Dec 2021 04:38:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232757AbhLQEh6 (ORCPT ); Thu, 16 Dec 2021 23:37:58 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48694 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232728AbhLQEh4 (ORCPT ); Thu, 16 Dec 2021 23:37:56 -0500 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ABB6CC06173E for ; Thu, 16 Dec 2021 20:37:56 -0800 (PST) Received: by mail-yb1-xb49.google.com with SMTP id v20-20020a25fc14000000b005c2109e5ad1so2497451ybd.9 for ; Thu, 16 Dec 2021 20:37:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Zr32vRf61UZFH6hZpGjG1QCJKVeFmDuobe731qgvIyQ=; b=OYYGBcmVMIMB5a0rQA9rNoKnYoqunv+KEdCj56VbJuN/t6BzNS/lsYozW+4/RIV0wr I6MG0hwXQs1m1DstzwyiQ7OJttrLrAh5vAD5kjnxs2qsvM8MXqeqQWfBZ7TWkS97nGyY 1nH0jNXdTAvHhv4vzitQBX3xwxxtJ92N9ESXkEJR1lhQit6E8gwsiUNBJlDJxf5gHwSr Ims1VGo0Fw0IG+bUo3eIhJKRkfYn3QPdOOfvbSckG2t5zVFMLzBMLiMG+hTq05H28cPz W/xNr9/NqWNZjIu5goy6ehCYDLQdCRBqX+nRkTv6v2WwEOwmZMS6PRB6/JojTJ4Cukcn +BZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Zr32vRf61UZFH6hZpGjG1QCJKVeFmDuobe731qgvIyQ=; b=Lhi1Zjs0k6ieo/5BeefQhwaSGtH25ZGrZLw7scghfTJ+sHHYa19IswRArlc43X+CK/ FfmW6jW9n/WDq5UOtEY6hOQnEzwST6k3+pf2hnVXKN8/Wb68PtGasHeIqsbvmKUZYtGo cY0dajlKofmBy65GYZMVzl3YGZ+3JLUf4J5WmYdCYnbfis2HJ8UAcSjPo8tXHPeE3e4r M1vXBMiE5UahmheDU1JHW44kk7NLBUJy6yJdszpQOI91u2cClYij6RmqfhHcIbKX/Hq+ UUe1H/daQphCYZAGdZDLdYLyKEBWbOOEJism9wwoA5ZXWIiCG/ngP14nZ+yp8IH892+l yNJQ== X-Gm-Message-State: AOAM530bFofQtjiL2nkznbH4E9MarkGzSZUSoSrd22D8Y/jv/oyfyEkU FvYBgvNSUnJzISKFXjOt8ecyKO09CxPoqmw= X-Google-Smtp-Source: ABdhPJw/ZRrYxrGznUu58JvQL9LX5blG9T7YNBBtxSVp60C0jbP6SQt4hkdSEGHzBJqs9WG52CfCK5Voag2Pgq8= X-Received: from sharinder.c.googlers.com ([fda3:e722:ac3:cc00:4f:4b78:c0a8:c73]) (user=sharinder job=sendgmr) by 2002:a25:ad45:: with SMTP id l5mr2066696ybe.198.1639715875996; Thu, 16 Dec 2021 20:37:55 -0800 (PST) Date: Fri, 17 Dec 2021 04:37:11 +0000 In-Reply-To: <20211217043716.794289-1-sharinder@google.com> Message-Id: <20211217043716.794289-3-sharinder@google.com> Mime-Version: 1.0 References: <20211217043716.794289-1-sharinder@google.com> X-Mailer: git-send-email 2.34.1.173.g76aa8bc2d0-goog Subject: [PATCH v5 2/7] Documentation: KUnit: Rewrite getting started From: Harinder Singh To: davidgow@google.com, brendanhiggins@google.com, shuah@kernel.org, corbet@lwn.net Cc: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, tim.bird@sony.com, elver@google.com, Harinder Singh Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Clarify the purpose of kunit_tool and fixed consistency issues Signed-off-by: Harinder Singh --- Documentation/dev-tools/kunit/start.rst | 195 +++++++++++++----------- 1 file changed, 102 insertions(+), 93 deletions(-) diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-to= ols/kunit/start.rst index 1e00f9226f74..55f8df1abd40 100644 --- a/Documentation/dev-tools/kunit/start.rst +++ b/Documentation/dev-tools/kunit/start.rst @@ -4,132 +4,136 @@ Getting Started =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 -Installing dependencies +Installing Dependencies =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -KUnit has the same dependencies as the Linux kernel. As long as you can bu= ild -the kernel, you can run KUnit. +KUnit has the same dependencies as the Linux kernel. As long as you can +build the kernel, you can run KUnit. =20 -Running tests with the KUnit Wrapper -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -Included with KUnit is a simple Python wrapper which runs tests under User= Mode -Linux, and formats the test results. - -The wrapper can be run with: +Running tests with kunit_tool +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D +kunit_tool is a Python script, which configures and builds a kernel, runs +tests, and formats the test results. From the kernel repository, you +can run kunit_tool: =20 .. code-block:: bash =20 ./tools/testing/kunit/kunit.py run =20 -For more information on this wrapper (also called kunit_tool) check out the -Documentation/dev-tools/kunit/kunit-tool.rst page. +For more information on this wrapper, see: +Documentation/dev-tools/kunit/kunit-tool.rst. + +Creating a ``.kunitconfig`` +--------------------------- + +By default, kunit_tool runs a selection of tests. However, you can specify= which +unit tests to run by creating a ``.kunitconfig`` file with kernel config o= ptions +that enable only a specific set of tests and their dependencies. +The ``.kunitconfig`` file contains a list of kconfig options which are req= uired +to run the desired targets. The ``.kunitconfig`` also contains any other t= est +specific config options, such as test dependencies. For example: the +``FAT_FS`` tests - ``FAT_KUNIT_TEST``, depends on +``FAT_FS``. ``FAT_FS`` can be enabled by selecting either ``MSDOS_FS`` +or ``VFAT_FS``. To run ``FAT_KUNIT_TEST``, the ``.kunitconfig`` has: =20 -Creating a .kunitconfig ------------------------ -If you want to run a specific set of tests (rather than those listed in the -KUnit defconfig), you can provide Kconfig options in the ``.kunitconfig`` = file. -This file essentially contains the regular Kernel config, with the specific -test targets as well. The ``.kunitconfig`` should also contain any other c= onfig -options required by the tests. +.. code-block:: none + + CONFIG_KUNIT=3Dy + CONFIG_MSDOS_FS=3Dy + CONFIG_FAT_KUNIT_TEST=3Dy =20 -A good starting point for a ``.kunitconfig`` is the KUnit defconfig: +1. A good starting point for the ``.kunitconfig``, is the KUnit default + config. Run the command: =20 .. code-block:: bash =20 cd $PATH_TO_LINUX_REPO cp tools/testing/kunit/configs/default.config .kunitconfig =20 -You can then add any other Kconfig options you wish, e.g.: +.. note :: + You may want to remove CONFIG_KUNIT_ALL_TESTS from the ``.kunitconfig``= as + it will enable a number of additional tests that you may not want. + +2. You can then add any other Kconfig options, for example: =20 .. code-block:: none =20 CONFIG_LIST_KUNIT_TEST=3Dy =20 -:doc:`kunit_tool ` will ensure that all config options set in -``.kunitconfig`` are set in the kernel ``.config`` before running the test= s. -It'll warn you if you haven't included the dependencies of the options you= 're -using. +Before running the tests, kunit_tool ensures that all config options +set in ``.kunitconfig`` are set in the kernel ``.config``. It will warn +you if you have not included dependencies for the options used. =20 -.. note:: - Note that removing something from the ``.kunitconfig`` will not trigger= a - rebuild of the ``.config`` file: the configuration is only updated if t= he - ``.kunitconfig`` is not a subset of ``.config``. This means that you ca= n use - other tools (such as make menuconfig) to adjust other config options. +.. note :: + The configuration is only updated if the ``.kunitconfig`` is not a + subset of ``.config``. You can use tools (for example: + make menuconfig) to adjust other config options. =20 - -Running the tests (KUnit Wrapper) ---------------------------------- - -To make sure that everything is set up correctly, simply invoke the Python -wrapper from your kernel repo: +Running Tests (KUnit Wrapper) +----------------------------- +1. To make sure that everything is set up correctly, invoke the Python + wrapper from your kernel repository: =20 .. code-block:: bash =20 ./tools/testing/kunit/kunit.py run =20 -.. note:: - You may want to run ``make mrproper`` first. - If everything worked correctly, you should see the following: =20 -.. code-block:: bash +.. code-block:: =20 Generating .config ... Building KUnit Kernel ... Starting KUnit Kernel ... =20 -followed by a list of tests that are run. All of them should be passing. +The tests will pass or fail. =20 -.. note:: - Because it is building a lot of sources for the first time, the - ``Building KUnit kernel`` step may take a while. +.. note :: + Because it is building a lot of sources for the first time, the + ``Building KUnit kernel`` may take a while. =20 -Running tests without the KUnit Wrapper +Running Tests without the KUnit Wrapper =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D - -If you'd rather not use the KUnit Wrapper (if, for example, you need to -integrate with other systems, or use an architecture other than UML), KUni= t can -be included in any kernel, and the results read out and parsed manually. - -.. note:: - KUnit is not designed for use in a production system, and it's possible= that - tests may reduce the stability or security of the system. - - - -Configuring the kernel +If you do not want to use the KUnit Wrapper (for example: you want code +under test to integrate with other systems, or use a different/ +unsupported architecture or configuration), KUnit can be included in +any kernel, and the results are read out and parsed manually. + +.. note :: + ``CONFIG_KUNIT`` should not be enabled in a production environment. + Enabling KUnit disables Kernel Address-Space Layout Randomization + (KASLR), and tests may affect the state of the kernel in ways not + suitable for production. + +Configuring the Kernel ---------------------- +To enable KUnit itself, you need to enable the ``CONFIG_KUNIT`` Kconfig +option (under Kernel Hacking/Kernel Testing and Coverage in +``menuconfig``). From there, you can enable any KUnit tests. They +usually have config options ending in ``_KUNIT_TEST``. =20 -In order to enable KUnit itself, you simply need to enable the ``CONFIG_KU= NIT`` -Kconfig option (it's under Kernel Hacking/Kernel Testing and Coverage in -menuconfig). From there, you can enable any KUnit tests you want: they usu= ally -have config options ending in ``_KUNIT_TEST``. - -KUnit and KUnit tests can be compiled as modules: in this case the tests i= n a -module will be run when the module is loaded. +KUnit and KUnit tests can be compiled as modules. The tests in a module +will run when the module is loaded. =20 - -Running the tests (w/o KUnit Wrapper) +Running Tests (without KUnit Wrapper) ------------------------------------- +Build and run your kernel. In the kernel log, the test output is printed +out in the TAP format. This will only happen by default if KUnit/tests +are built-in. Otherwise the module will need to be loaded. =20 -Build and run your kernel as usual. Test output will be written to the ker= nel -log in `TAP `_ format. - -.. note:: - It's possible that there will be other lines and/or data interspersed i= n the - TAP output. - +.. note :: + Some lines and/or data may get interspersed in the TAP output. =20 -Writing your first test +Writing Your First Test =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +In your kernel repository, let's add some code that we can test. =20 -In your kernel repo let's add some code that we can test. Create a file -``drivers/misc/example.h`` with the contents: +1. Create a file ``drivers/misc/example.h``, which includes: =20 .. code-block:: c =20 int misc_example_add(int left, int right); =20 -create a file ``drivers/misc/example.c``: +2. Create a file ``drivers/misc/example.c``, which includes: =20 .. code-block:: c =20 @@ -142,21 +146,22 @@ create a file ``drivers/misc/example.c``: return left + right; } =20 -Now add the following lines to ``drivers/misc/Kconfig``: +3. Add the following lines to ``drivers/misc/Kconfig``: =20 .. code-block:: kconfig =20 config MISC_EXAMPLE bool "My example" =20 -and the following lines to ``drivers/misc/Makefile``: +4. Add the following lines to ``drivers/misc/Makefile``: =20 .. code-block:: make =20 obj-$(CONFIG_MISC_EXAMPLE) +=3D example.o =20 -Now we are ready to write the test. The test will be in -``drivers/misc/example-test.c``: +Now we are ready to write the test cases. + +1. Add the below test case in ``drivers/misc/example_test.c``: =20 .. code-block:: c =20 @@ -191,7 +196,7 @@ Now we are ready to write the test. The test will be in }; kunit_test_suite(misc_example_test_suite); =20 -Now add the following to ``drivers/misc/Kconfig``: +2. Add the following lines to ``drivers/misc/Kconfig``: =20 .. code-block:: kconfig =20 @@ -200,20 +205,20 @@ Now add the following to ``drivers/misc/Kconfig``: depends on MISC_EXAMPLE && KUNIT=3Dy default KUNIT_ALL_TESTS =20 -and the following to ``drivers/misc/Makefile``: +3. Add the following lines to ``drivers/misc/Makefile``: =20 .. code-block:: make =20 - obj-$(CONFIG_MISC_EXAMPLE_TEST) +=3D example-test.o + obj-$(CONFIG_MISC_EXAMPLE_TEST) +=3D example_test.o =20 -Now add it to your ``.kunitconfig``: +4. Add the following lines to ``.kunitconfig``: =20 .. code-block:: none =20 CONFIG_MISC_EXAMPLE=3Dy CONFIG_MISC_EXAMPLE_TEST=3Dy =20 -Now you can run the test: +5. Run the test: =20 .. code-block:: bash =20 @@ -227,16 +232,20 @@ You should see the following failure: [16:08:57] [PASSED] misc-example:misc_example_add_test_basic [16:08:57] [FAILED] misc-example:misc_example_test_failure [16:08:57] EXPECTATION FAILED at drivers/misc/example-test.c:17 - [16:08:57] This test never passes. + [16:08:57] This test never passes. ... =20 -Congrats! You just wrote your first KUnit test! +Congrats! You just wrote your first KUnit test. =20 Next Steps =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -* Check out the Documentation/dev-tools/kunit/tips.rst page for tips on - writing idiomatic KUnit tests. -* Check out the :doc:`running_tips` page for tips on - how to make running KUnit tests easier. -* Optional: see the :doc:`usage` page for a more - in-depth explanation of KUnit. + +* Documentation/dev-tools/kunit/usage.rst - KUnit features. +* Documentation/dev-tools/kunit/tips.rst - best practices with + examples. +* Documentation/dev-tools/kunit/api/index.rst - KUnit APIs + used for testing. +* Documentation/dev-tools/kunit/kunit-tool.rst - kunit_tool helper + script. +* Documentation/dev-tools/kunit/faq.rst - KUnit common questions and + answers. --=20 2.34.1.173.g76aa8bc2d0-goog From nobody Wed Jul 1 18:37:12 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1917FC4332F for ; Fri, 17 Dec 2021 04:38:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232770AbhLQEiI (ORCPT ); Thu, 16 Dec 2021 23:38:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48724 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232761AbhLQEiC (ORCPT ); Thu, 16 Dec 2021 23:38:02 -0500 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C4F38C06173F for ; Thu, 16 Dec 2021 20:38:01 -0800 (PST) Received: by mail-yb1-xb4a.google.com with SMTP id w5-20020a25ac05000000b005c55592df4dso2495723ybi.12 for ; Thu, 16 Dec 2021 20:38:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc:content-transfer-encoding; bh=K24+5mXsl0tqDSvX4ae3NiIhxWkMNrm1J2fCengDhvg=; b=V0w9l5S/XUm9lh3XahP/yUKAfho+70crDg/SbtQAsQZ3X176LBgkZyLkO2yvFzjUvU wZu+ixfocHpPY4bOW2gLVDwQT8JrKfSgy7wjujqd33ZSJVpAnEZJDaaOWgwOIS8vg+V9 k1GccRV27efOaJkUSbNv8gEMYf5OTg6/7F1U2/uWijvS68GXhHTeU3NxDY9ZOcusz28v 4Dz3R4cxQC7A06co/+mKtEXg0GggqKvCxuxwpWXL6KbODOkXZhR3z7DLI/td3aU+7nm1 AnbBxewLLJGq2ptUuzdldvzVwFLrN4ICKQrLLtflcQzGpx4ovRQB9Qg4P1PJTdEL3fLi ZPUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc:content-transfer-encoding; bh=K24+5mXsl0tqDSvX4ae3NiIhxWkMNrm1J2fCengDhvg=; b=avb5I5Hhuu/MR9zcVZnuqtc1nfxqe9XWHY/bhEfFS1WlLjpbTKfr1+czbFSINdCTqT BJx+Uo4RG/SCJFM6G0C0bxJgO8hdaOYmMDGZ73Uh2185sKuZ23duLtBVNp0K6Y0ygDL/ WQKo5gL/khE7WzQWgWAqguipusVQPbyQQl959omBS5I5Oy3my9D00p5myQrcwFou7K4E UVkZnuD19D32sy7aSobqe5aFqOI/P1QwhhqMuE2kfssib12/oPSp9HtFbBSi9/fppcpZ sBw8OjFcx0LEKupXkYv02BVcj64WccWUH2YqHnhVpwk6gSLhxA67hhGGd+zPxUzQkZQi /RmA== X-Gm-Message-State: AOAM531qpp/m8lko4smNCyN4mVYsTtuGhIFmIkKhusFSvmtaxJJSgIBQ bPRX7ICo8q7k6+DzXC3v4KWljvNnIxcHAVI= X-Google-Smtp-Source: ABdhPJzqb//QMYNzU3s7+A1AheVZ9d9MNeiHYI3pxRtbg5y1pKJfXtJdinP3EdbvXpG8i54PHGKibmXfxMeTbAA= X-Received: from sharinder.c.googlers.com ([fda3:e722:ac3:cc00:4f:4b78:c0a8:c73]) (user=sharinder job=sendgmr) by 2002:a25:7ec1:: with SMTP id z184mr1919789ybc.103.1639715881010; Thu, 16 Dec 2021 20:38:01 -0800 (PST) Date: Fri, 17 Dec 2021 04:37:12 +0000 In-Reply-To: <20211217043716.794289-1-sharinder@google.com> Message-Id: <20211217043716.794289-4-sharinder@google.com> Mime-Version: 1.0 References: <20211217043716.794289-1-sharinder@google.com> X-Mailer: git-send-email 2.34.1.173.g76aa8bc2d0-goog Subject: [PATCH v5 3/7] Documentation: KUnit: Added KUnit Architecture From: Harinder Singh To: davidgow@google.com, brendanhiggins@google.com, shuah@kernel.org, corbet@lwn.net Cc: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, tim.bird@sony.com, elver@google.com, Harinder Singh Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Describe the components of KUnit and how the kernel mode parts interact with kunit_tool. Signed-off-by: Harinder Singh --- .../dev-tools/kunit/architecture.rst | 204 ++++++++++++++++++ Documentation/dev-tools/kunit/index.rst | 2 + Documentation/dev-tools/kunit/start.rst | 1 + 3 files changed, 207 insertions(+) create mode 100644 Documentation/dev-tools/kunit/architecture.rst diff --git a/Documentation/dev-tools/kunit/architecture.rst b/Documentation= /dev-tools/kunit/architecture.rst new file mode 100644 index 000000000000..aa2cea821e25 --- /dev/null +++ b/Documentation/dev-tools/kunit/architecture.rst @@ -0,0 +1,204 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +KUnit Architecture +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +The KUnit architecture can be divided into two parts: + +- Kernel testing library +- kunit_tool (Command line test harness) + +In-Kernel Testing Framework +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D + +The kernel testing library supports KUnit tests written in C using +KUnit. KUnit tests are kernel code. KUnit does several things: + +- Organizes tests +- Reports test results +- Provides test utilities + +Test Cases +---------- + +The fundamental unit in KUnit is the test case. The KUnit test cases are +grouped into KUnit suites. A KUnit test case is a function with type +signature ``void (*)(struct kunit *test)``. +These test case functions are wrapped in a struct called +``struct kunit_case``. For code, see: + +.. kernel-doc:: include/kunit/test.h + :identifiers: kunit_case + +.. note: + ``generate_params`` is optional for non-parameterized tests. + +Each KUnit test case gets a ``struct kunit`` context +object passed to it that tracks a running test. The KUnit assertion +macros and other KUnit utilities use the ``struct kunit`` context +object. As an exception, there are two fields: + +- ``->priv``: The setup functions can use it to store arbitrary test + user data. + +- ``->param_value``: It contains the parameter value which can be + retrieved in the parameterized tests. + +Test Suites +----------- + +A KUnit suite includes a collection of test cases. The KUnit suites +are represented by the ``struct kunit_suite``. For example: + +.. code-block:: c + + static struct kunit_case example_test_cases[] =3D { + KUNIT_CASE(example_test_foo), + KUNIT_CASE(example_test_bar), + KUNIT_CASE(example_test_baz), + {} + }; + + static struct kunit_suite example_test_suite =3D { + .name =3D "example", + .init =3D example_test_init, + .exit =3D example_test_exit, + .test_cases =3D example_test_cases, + }; + kunit_test_suite(example_test_suite); + +In the above example, the test suite ``example_test_suite``, runs the +test cases ``example_test_foo``, ``example_test_bar``, and +``example_test_baz``. Before running the test, the ``example_test_init`` +is called and after running the test, ``example_test_exit`` is called. +The ``kunit_test_suite(example_test_suite)`` registers the test suite +with the KUnit test framework. + +Executor +-------- + +The KUnit executor can list and run built-in KUnit tests on boot. +The Test suites are stored in a linker section +called ``.kunit_test_suites``. For code, see: +https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/in= clude/asm-generic/vmlinux.lds.h?h=3Dv5.15#n945. +The linker section consists of an array of pointers to +``struct kunit_suite``, and is populated by the ``kunit_test_suites()`` +macro. To run all tests compiled into the kernel, the KUnit executor +iterates over the linker section array. + +.. kernel-figure:: kunit_suitememorydiagram.svg + :alt: KUnit Suite Memory + + KUnit Suite Memory Diagram + +On the kernel boot, the KUnit executor uses the start and end addresses +of this section to iterate over and run all tests. For code, see: +https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/li= b/kunit/executor.c + +When built as a module, the ``kunit_test_suites()`` macro defines a +``module_init()`` function, which runs all the tests in the compilation +unit instead of utilizing the executor. + +In KUnit tests, some error classes do not affect other tests +or parts of the kernel, each KUnit case executes in a separate thread +context. For code, see: +https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/li= b/kunit/try-catch.c?h=3Dv5.15#n58 + +Assertion Macros +---------------- + +KUnit tests verify state using expectations/assertions. +All expectations/assertions are formatted as: +``KUNIT_{EXPECT|ASSERT}_[_MSG](kunit, property[, message])`` + +- ``{EXPECT|ASSERT}`` determines whether the check is an assertion or an + expectation. + + - For an expectation, if the check fails, marks the test as failed + and logs the failure. + + - An assertion, on failure, causes the test case to terminate + immediately. + + - Assertions call function: + ``void __noreturn kunit_abort(struct kunit *)``. + + - ``kunit_abort`` calls function: + ``void __noreturn kunit_try_catch_throw(struct kunit_try_catch *try_ca= tch)``. + + - ``kunit_try_catch_throw`` calls function: + ``void complete_and_exit(struct completion *, long) __noreturn;`` + and terminates the special thread context. + +- ```` denotes a check with options: ``TRUE`` (supplied property + has the boolean value =E2=80=9Ctrue=E2=80=9D), ``EQ`` (two supplied prop= erties are + equal), ``NOT_ERR_OR_NULL`` (supplied pointer is not null and does not + contain an =E2=80=9Cerr=E2=80=9D value). + +- ``[_MSG]`` prints a custom message on failure. + +Test Result Reporting +--------------------- +KUnit prints test results in KTAP format. KTAP is based on TAP14, see: +https://github.com/isaacs/testanything.github.io/blob/tap14/tap-version-14= -specification.md. +KTAP (yet to be standardized format) works with KUnit and Kselftest. +The KUnit executor prints KTAP results to dmesg, and debugfs +(if configured). + +Parameterized Tests +------------------- + +Each KUnit parameterized test is associated with a collection of +parameters. The test is invoked multiple times, once for each parameter +value and the parameter is stored in the ``param_value`` field. +The test case includes a ``KUNIT_CASE_PARAM()`` macro that accepts a +generator function. +The generator function is passed the previous parameter and returns the ne= xt +parameter. It also provides a macro to generate common-case generators bas= ed on +arrays. + +For code, see: + +.. kernel-doc:: include/kunit/test.h + :identifiers: KUNIT_ARRAY_PARAM + + +kunit_tool (Command Line Test Harness) +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +kunit_tool is a Python script ``(tools/testing/kunit/kunit.py)`` +that can be used to configure, build, exec, parse and run (runs other +commands in order) test results. You can either run KUnit tests using +kunit_tool or can include KUnit in kernel and parse manually. + +- ``configure`` command generates the kernel ``.config`` from a + ``.kunitconfig`` file (and any architecture-specific options). + For some architectures, additional config options are specified in the + ``qemu_config`` Python script + (For example: ``tools/testing/kunit/qemu_configs/powerpc.py``). + It parses both the existing ``.config`` and the ``.kunitconfig`` files + and ensures that ``.config`` is a superset of ``.kunitconfig``. + If this is not the case, it will combine the two and run + ``make olddefconfig`` to regenerate the ``.config`` file. It then + verifies that ``.config`` is now a superset. This checks if all + Kconfig dependencies are correctly specified in ``.kunitconfig``. + ``kunit_config.py`` includes the parsing Kconfigs code. The code which + runs ``make olddefconfig`` is a part of ``kunit_kernel.py``. You can + invoke this command via: ``./tools/testing/kunit/kunit.py config`` and + generate a ``.config`` file. +- ``build`` runs ``make`` on the kernel tree with required options + (depends on the architecture and some options, for example: build_dir) + and reports any errors. + To build a KUnit kernel from the current ``.config``, you can use the + ``build`` argument: ``./tools/testing/kunit/kunit.py build``. +- ``exec`` command executes kernel results either directly (using + User-mode Linux configuration), or via an emulator such + as QEMU. It reads results from the log via standard + output (stdout), and passes them to ``parse`` to be parsed. + If you already have built a kernel with built-in KUnit tests, + you can run the kernel and display the test results with the ``exec`` + argument: ``./tools/testing/kunit/kunit.py exec``. +- ``parse`` extracts the KTAP output from a kernel log, parses + the test results, and prints a summary. For failed tests, any + diagnostic output will be included. diff --git a/Documentation/dev-tools/kunit/index.rst b/Documentation/dev-to= ols/kunit/index.rst index 55d2444b0745..50d3ef9359dd 100644 --- a/Documentation/dev-tools/kunit/index.rst +++ b/Documentation/dev-tools/kunit/index.rst @@ -9,6 +9,7 @@ KUnit - Linux Kernel Unit Testing :caption: Contents: =20 start + architecture usage kunit-tool api/index @@ -96,6 +97,7 @@ How do I use it? =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 * Documentation/dev-tools/kunit/start.rst - for KUnit new users. +* Documentation/dev-tools/kunit/architecture.rst - KUnit architecture. * Documentation/dev-tools/kunit/usage.rst - KUnit features. * Documentation/dev-tools/kunit/tips.rst - best practices with examples. diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-to= ols/kunit/start.rst index 55f8df1abd40..5dd2c88fa2bd 100644 --- a/Documentation/dev-tools/kunit/start.rst +++ b/Documentation/dev-tools/kunit/start.rst @@ -240,6 +240,7 @@ Congrats! You just wrote your first KUnit test. Next Steps =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 +* Documentation/dev-tools/kunit/architecture.rst - KUnit architecture. * Documentation/dev-tools/kunit/usage.rst - KUnit features. * Documentation/dev-tools/kunit/tips.rst - best practices with examples. --=20 2.34.1.173.g76aa8bc2d0-goog From nobody Wed Jul 1 18:37:12 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E31BC433F5 for ; Fri, 17 Dec 2021 04:38:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232792AbhLQEiK (ORCPT ); Thu, 16 Dec 2021 23:38:10 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48756 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232819AbhLQEiH (ORCPT ); Thu, 16 Dec 2021 23:38:07 -0500 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1FCD5C061751 for ; Thu, 16 Dec 2021 20:38:07 -0800 (PST) Received: by mail-yb1-xb49.google.com with SMTP id j18-20020a258152000000b006000d292a42so2661997ybm.0 for ; Thu, 16 Dec 2021 20:38:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=14H6xhOpDNBLVv1Dapd5bhsBv/80ue6Xu3pTRU70RaY=; b=bJQQg00ZUFYb9zmlvF1MbTJ8+ZkokomUC56XoJVFk8L06x5BF8432X/1lyyGZBJN8B eUNHKA1MGFVZ9+w0ADQUYhcKPAggq3cWogf6jM6zsdlobdGsFcIsJ5Hqn9k1sfGIQJiA KwqhhIzajFL5ksUD/4EPXIQlvgGSK5wsCdVfs+3ywlOMrJIy8Idw0YcXHhgAQmG5kG/a o9ka6XDQd8nOhfeOt3jzn5yhAJWyEuu1o/yTgFLk7ObhdKJaGZaO2GEI1Ft0w5rMdKPj MlSpnb/L+Ae/d2ICfIf/VWKOl/G2hiFhOAFkpjA7snp7rWlZvob2rVDWdYu8ipSj7zGB 6tjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=14H6xhOpDNBLVv1Dapd5bhsBv/80ue6Xu3pTRU70RaY=; b=BBcgCYUM0hfJOaw4VCDkd5Txm1oainKyPxQ3hIU83xvU4vBP47+mNe/Zfnymbm7BK0 wg+hdp307ZLX3Qxz/eycvUcKu8Yhl5Z8cpR5ytCYK4B/TShbYLLv7UwlibML3M36eHlR lRNMLCePLISNbBsI2i738pfjUPSua22vH3I2WfE2T43t868yZJpXrJ/+HQr9nndxY4a7 8jr/IR3vtVeZWsrqFT7UuAfgVs4/D6lPsW6Uc0Ye8edC4DOXTtLtjWAGJXxjRbaBqhVH +vLYmNdzPhDuaFGcA2zG0xa5MX3LrDgmXtjJlopYAmpjoa8m54ooljrJ07gpYOP7hLun O4Jg== X-Gm-Message-State: AOAM53159jwsYfLkC4/d+8OiIPMOMeduCg/MiaOnXuLuKfUBzeaT558r 5O3S26dCOYroUN7cRbvfnioWLcoH4kHMje8= X-Google-Smtp-Source: ABdhPJxc0DLD6RFjOWqqWrvOnVAMfTPYPxOyieNtCNMxGwDqx7JvRbpxzH/hbkLZT1U9HlNNMXY3zTgY5UrGhTI= X-Received: from sharinder.c.googlers.com ([fda3:e722:ac3:cc00:4f:4b78:c0a8:c73]) (user=sharinder job=sendgmr) by 2002:a25:cd47:: with SMTP id d68mr1995108ybf.537.1639715886312; Thu, 16 Dec 2021 20:38:06 -0800 (PST) Date: Fri, 17 Dec 2021 04:37:13 +0000 In-Reply-To: <20211217043716.794289-1-sharinder@google.com> Message-Id: <20211217043716.794289-5-sharinder@google.com> Mime-Version: 1.0 References: <20211217043716.794289-1-sharinder@google.com> X-Mailer: git-send-email 2.34.1.173.g76aa8bc2d0-goog Subject: [PATCH v5 4/7] Documentation: kunit: Reorganize documentation related to running tests From: Harinder Singh To: davidgow@google.com, brendanhiggins@google.com, shuah@kernel.org, corbet@lwn.net Cc: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, tim.bird@sony.com, elver@google.com, Harinder Singh Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Consolidate documentation running tests into two pages: "run tests with kunit_tool" and "run tests without kunit_tool". Signed-off-by: Harinder Singh Reviewed-by: Tim Bird --- Documentation/dev-tools/kunit/index.rst | 4 + Documentation/dev-tools/kunit/run_manual.rst | 57 ++++ Documentation/dev-tools/kunit/run_wrapper.rst | 247 ++++++++++++++++++ Documentation/dev-tools/kunit/start.rst | 4 +- 4 files changed, 311 insertions(+), 1 deletion(-) create mode 100644 Documentation/dev-tools/kunit/run_manual.rst create mode 100644 Documentation/dev-tools/kunit/run_wrapper.rst diff --git a/Documentation/dev-tools/kunit/index.rst b/Documentation/dev-to= ols/kunit/index.rst index 50d3ef9359dd..f9f37997b58c 100644 --- a/Documentation/dev-tools/kunit/index.rst +++ b/Documentation/dev-tools/kunit/index.rst @@ -10,6 +10,8 @@ KUnit - Linux Kernel Unit Testing =20 start architecture + run_wrapper + run_manual usage kunit-tool api/index @@ -98,6 +100,8 @@ How do I use it? =20 * Documentation/dev-tools/kunit/start.rst - for KUnit new users. * Documentation/dev-tools/kunit/architecture.rst - KUnit architecture. +* Documentation/dev-tools/kunit/run_wrapper.rst - run kunit_tool. +* Documentation/dev-tools/kunit/run_manual.rst - run tests without kunit= _tool. * Documentation/dev-tools/kunit/usage.rst - KUnit features. * Documentation/dev-tools/kunit/tips.rst - best practices with examples. diff --git a/Documentation/dev-tools/kunit/run_manual.rst b/Documentation/d= ev-tools/kunit/run_manual.rst new file mode 100644 index 000000000000..e7b46421f247 --- /dev/null +++ b/Documentation/dev-tools/kunit/run_manual.rst @@ -0,0 +1,57 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D +Run Tests without kunit_tool +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D + +If we do not want to use kunit_tool (For example: we want to integrate +with other systems, or run tests on real hardware), we can +include KUnit in any kernel, read out results, and parse manually. + +.. note:: KUnit is not designed for use in a production system. It is + possible that tests may reduce the stability or security of + the system. + +Configure the Kernel +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +KUnit tests can run without kunit_tool. This can be useful, if: + +- We have an existing kernel configuration to test. +- Need to run on real hardware (or using an emulator/VM kunit_tool + does not support). +- Wish to integrate with some existing testing systems. + +KUnit is configured with the ``CONFIG_KUNIT`` option, and individual +tests can also be built by enabling their config options in our +``.config``. KUnit tests usually (but don't always) have config options +ending in ``_KUNIT_TEST``. Most tests can either be built as a module, +or be built into the kernel. + +.. note :: + + We can enable the ``KUNIT_ALL_TESTS`` config option to + automatically enable all tests with satisfied dependencies. This is + a good way of quickly testing everything applicable to the current + config. + +Once we have built our kernel (and/or modules), it is simple to run +the tests. If the tests are built-in, they will run automatically on the +kernel boot. The results will be written to the kernel log (``dmesg``) +in TAP format. + +If the tests are built as modules, they will run when the module is +loaded. + +.. code-block :: bash + + # modprobe example-test + +The results will appear in TAP format in ``dmesg``. + +.. note :: + + If ``CONFIG_KUNIT_DEBUGFS`` is enabled, KUnit test results will + be accessible from the ``debugfs`` filesystem (if mounted). + They will be in ``/sys/kernel/debug/kunit//results``, in + TAP format. diff --git a/Documentation/dev-tools/kunit/run_wrapper.rst b/Documentation/= dev-tools/kunit/run_wrapper.rst new file mode 100644 index 000000000000..653985ce9cae --- /dev/null +++ b/Documentation/dev-tools/kunit/run_wrapper.rst @@ -0,0 +1,247 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D +Run Tests with kunit_tool +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D + +We can either run KUnit tests using kunit_tool or can run tests +manually, and then use kunit_tool to parse the results. To run tests +manually, see: Documentation/dev-tools/kunit/run_manual.rst. +As long as we can build the kernel, we can run KUnit. + +kunit_tool is a Python script which configures and builds a kernel, runs +tests, and formats the test results. + +Run command: + +.. code-block:: + + ./tools/testing/kunit/kunit.py run + +We should see the following: + +.. code-block:: + + Generating .config... + Building KUnit kernel... + Starting KUnit kernel... + +We may want to use the following options: + +.. code-block:: + + ./tools/testing/kunit/kunit.py run --timeout=3D30 --jobs=3D`nproc --all + +- ``--timeout`` sets a maximum amount of time for tests to run. +- ``--jobs`` sets the number of threads to build the kernel. + +kunit_tool will generate a ``.kunitconfig`` with a default +configuration, if no other ``.kunitconfig`` file exists +(in the build directory). In addition, it verifies that the +generated ``.config`` file contains the ``CONFIG`` options in the +``.kunitconfig``. +It is also possible to pass a separate ``.kunitconfig`` fragment to +kunit_tool. This is useful if we have several different groups of +tests we want to run independently, or if we want to use pre-defined +test configs for certain subsystems. + +To use a different ``.kunitconfig`` file (such as one +provided to test a particular subsystem), pass it as an option: + +.. code-block:: + + ./tools/testing/kunit/kunit.py run --kunitconfig=3Dfs/ext4/.kunitconfig + +To view kunit_tool flags (optional command-line arguments), run: + +.. code-block:: + + ./tools/testing/kunit/kunit.py run --help + +Create a ``.kunitconfig`` File +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D + +If we want to run a specific set of tests (rather than those listed +in the KUnit ``defconfig``), we can provide Kconfig options in the +``.kunitconfig`` file. For default .kunitconfig, see: +https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/to= ols/testing/kunit/configs/default.config. +A ``.kunitconfig`` is a ``minconfig`` (a .config +generated by running ``make savedefconfig``), used for running a +specific set of tests. This file contains the regular Kernel configs +with specific test targets. The ``.kunitconfig`` also +contains any other config options required by the tests (For example: +dependencies for features under tests, configs that enable/disable +certain code blocks, arch configs and so on). + +To create a ``.kunitconfig``, using the KUnit ``defconfig``: + +.. code-block:: + + cd $PATH_TO_LINUX_REPO + cp tools/testing/kunit/configs/default.config .kunit/.kunitconfig + +We can then add any other Kconfig options. For example: + +.. code-block:: + + CONFIG_LIST_KUNIT_TEST=3Dy + +kunit_tool ensures that all config options in ``.kunitconfig`` are +set in the kernel ``.config`` before running the tests. It warns if we +have not included the options dependencies. + +.. note:: Removing something from the ``.kunitconfig`` will + not rebuild the ``.config file``. The configuration is only + updated if the ``.kunitconfig`` is not a subset of ``.config``. + This means that we can use other tools + (For example: ``make menuconfig``) to adjust other config options. + The build dir needs to be set for ``make menuconfig`` to + work, therefore by default use ``make O=3D.kunit menuconfig``. + +Configure, Build, and Run Tests +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D + +If we want to make manual changes to the KUnit build process, we +can run part of the KUnit build process independently. +When running kunit_tool, from a ``.kunitconfig``, we can generate a +``.config`` by using the ``config`` argument: + +.. code-block:: + + ./tools/testing/kunit/kunit.py config + +To build a KUnit kernel from the current ``.config``, we can use the +``build`` argument: + +.. code-block:: + + ./tools/testing/kunit/kunit.py build + +If we already have built UML kernel with built-in KUnit tests, we +can run the kernel, and display the test results with the ``exec`` +argument: + +.. code-block:: + + ./tools/testing/kunit/kunit.py exec + +The ``run`` command discussed in section: **Run Tests with kunit_tool**, +is equivalent to running the above three commands in sequence. + +Parse Test Results +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +KUnit tests output displays results in TAP (Test Anything Protocol) +format. When running tests, kunit_tool parses this output and prints +a summary. To see the raw test results in TAP format, we can pass the +``--raw_output`` argument: + +.. code-block:: + + ./tools/testing/kunit/kunit.py run --raw_output + +If we have KUnit results in the raw TAP format, we can parse them and +print the human-readable summary with the ``parse`` command for +kunit_tool. This accepts a filename for an argument, or will read from +standard input. + +.. code-block:: bash + + # Reading from a file + ./tools/testing/kunit/kunit.py parse /var/log/dmesg + # Reading from stdin + dmesg | ./tools/testing/kunit/kunit.py parse + +Run Selected Test Suites +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +By passing a bash style glob filter to the ``exec`` or ``run`` +commands, we can run a subset of the tests built into a kernel . For +example: if we only want to run KUnit resource tests, use: + +.. code-block:: + + ./tools/testing/kunit/kunit.py run 'kunit-resource*' + +This uses the standard glob format with wildcard characters. + +Run Tests on qemu +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +kunit_tool supports running tests on qemu as well as +via UML. To run tests on qemu, by default it requires two flags: + +- ``--arch``: Selects a configs collection (Kconfig, qemu config options + and so on), that allow KUnit tests to be run on the specified + architecture in a minimal way. The architecture argument is same as + the option name passed to the ``ARCH`` variable used by Kbuild. + Not all architectures currently support this flag, but we can use + ``--qemu_config`` to handle it. If ``um`` is passed (or this flag + is ignored), the tests will run via UML. Non-UML architectures, + for example: i386, x86_64, arm and so on; run on qemu. + +- ``--cross_compile``: Specifies the Kbuild toolchain. It passes the + same argument as passed to the ``CROSS_COMPILE`` variable used by + Kbuild. As a reminder, this will be the prefix for the toolchain + binaries such as GCC. For example: + + - ``sparc64-linux-gnu`` if we have the sparc toolchain installed on + our system. + + - ``$HOME/toolchains/microblaze/gcc-9.2.0-nolibc/microblaze-linux/bin/mi= croblaze-linux`` + if we have downloaded the microblaze toolchain from the 0-day + website to a directory in our home directory called toolchains. + +If we want to run KUnit tests on an architecture not supported by +the ``--arch`` flag, or want to run KUnit tests on qemu using a +non-default configuration; then we can write our own``QemuConfig``. +These ``QemuConfigs`` are written in Python. They have an import line +``from..qemu_config import QemuArchParams`` at the top of the file. +The file must contain a variable called ``QEMU_ARCH`` that has an +instance of ``QemuArchParams`` assigned to it. See example in: +``tools/testing/kunit/qemu_configs/x86_64.py``. + +Once we have a ``QemuConfig``, we can pass it into kunit_tool, +using the ``--qemu_config`` flag. When used, this flag replaces the +``--arch`` flag. For example: using +``tools/testing/kunit/qemu_configs/x86_64.py``, the invocation appear +as + +.. code-block:: bash + + ./tools/testing/kunit/kunit.py run \ + --timeout=3D60 \ + --jobs=3D12 \ + --qemu_config=3D./tools/testing/kunit/qemu_configs/x86_64.py + +To run existing KUnit tests on non-UML architectures, see: +Documentation/dev-tools/kunit/non_uml.rst. + +Command-Line Arguments +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + +kunit_tool has a number of other command-line arguments which can +be useful for our test environment. Below the most commonly used +command line arguments: + +- ``--help``: Lists all available options. To list common options, + place ``--help`` before the command. To list options specific to that + command, place ``--help`` after the command. + + .. note:: Different commands (``config``, ``build``, ``run``, etc) + have different supported options. +- ``--build_dir``: Specifies kunit_tool build directory. It includes + the ``.kunitconfig``, ``.config`` files and compiled kernel. + +- ``--make_options``: Specifies additional options to pass to make, when + compiling a kernel (using ``build`` or ``run`` commands). For example: + to enable compiler warnings, we can pass ``--make_options W=3D1``. + +- ``--alltests``: Builds a UML kernel with all config options enabled + using ``make allyesconfig``. This allows us to run as many tests as + possible. + + .. note:: It is slow and prone to breakage as new options are + added or modified. Instead, enable all tests + which have satisfied dependencies by adding + ``CONFIG_KUNIT_ALL_TESTS=3Dy`` to your ``.kunitconfig``. diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-to= ols/kunit/start.rst index 5dd2c88fa2bd..af13f443c976 100644 --- a/Documentation/dev-tools/kunit/start.rst +++ b/Documentation/dev-tools/kunit/start.rst @@ -20,7 +20,7 @@ can run kunit_tool: ./tools/testing/kunit/kunit.py run =20 For more information on this wrapper, see: -Documentation/dev-tools/kunit/kunit-tool.rst. +Documentation/dev-tools/kunit/run_wrapper.rst. =20 Creating a ``.kunitconfig`` --------------------------- @@ -241,6 +241,8 @@ Next Steps =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 * Documentation/dev-tools/kunit/architecture.rst - KUnit architecture. +* Documentation/dev-tools/kunit/run_wrapper.rst - run kunit_tool. +* Documentation/dev-tools/kunit/run_manual.rst - run tests without kunit= _tool. * Documentation/dev-tools/kunit/usage.rst - KUnit features. * Documentation/dev-tools/kunit/tips.rst - best practices with examples. --=20 2.34.1.173.g76aa8bc2d0-goog From nobody Wed Jul 1 18:37:12 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 22DD8C4332F for ; Fri, 17 Dec 2021 04:38:20 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232786AbhLQEiR (ORCPT ); Thu, 16 Dec 2021 23:38:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48786 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232793AbhLQEiM (ORCPT ); Thu, 16 Dec 2021 23:38:12 -0500 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 22D1EC06173F for ; Thu, 16 Dec 2021 20:38:12 -0800 (PST) Received: by mail-yb1-xb49.google.com with SMTP id b15-20020a25ae8f000000b005c20f367790so2575576ybj.2 for ; Thu, 16 Dec 2021 20:38:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=2z0CUzAdRey0nX54Uun34Y9z1Xvd8EY9dE9XMgLPAQ0=; b=ceyZHABEIk+tjBb4Apz5E0DkD5xwJ04e2dUBa4fdArJzf9JEWODDIXXdQIYJz7opi1 X/e4BpTQSUNdVWqpShPLumnz2dNn2JL4oaAAqmcU2oYjlTU+H4BCH22sgX3zKbw+IKx5 fJtGotl73kaluMuQfVa6cSJBR/d0Iz7q0Qbi4M/rZ4HBVd5SVywr7TTgAHmop2gKGbjL /b9XYM2024tcCxIAUDMZiOLWubDuWmpX3aMDVjTJK9nRhagrh1xUOc4rK8OQHHzWAMN3 GWDg7f/lgFrxULGiS98L6Z30/JAODG/QKS7pWh1AttOLgzF2bRDNfxalImf7wGnsxzwQ DwRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=2z0CUzAdRey0nX54Uun34Y9z1Xvd8EY9dE9XMgLPAQ0=; b=DilDjskMUlKbwKsdsL4erFIi9lY3HGNrr8yoMe0CU6SP2tIPP3oFAviqJNyIB8/edj EbP0Dmq1XuTyNk4J30Swe5dCFItb1dAlim0AgES825phxi30StpzwTfYU8IjEQd5mynC Q4pjC/A6ousU0V+QulDZtZsO9RMyaun0anoIHio1kTISPTRv7Q7YS7V8UX3pYQryqVBz E/aAGSN7hEEwlqModcCwL+EnNJeNcPO4UgMIO8eFqnC2ZeCB6BFLcHhF3r8sQ19non/J Wxm5Iywz0wlh56fHCh3eZYF+u+P3Iy3dYJ5y1o5gahGZcoKXv68dvc1dI6zc1nvfGuOY jI4g== X-Gm-Message-State: AOAM531+hAI/IC9oZphOR30gItZohps7Z1QqPFeRQfHkEd4E7EcSlQ2G 2eW+wVl8WhgQo22BnC1yZyGGTy9iXJYUcD8= X-Google-Smtp-Source: ABdhPJzyLCIhglw1zmdXY/jOd9qzxBHcdtpv6TOQNS4xRzsLybe8wH8pyzTdjr/iCb1+EoAeobEyP3sI9XE0MA0= X-Received: from sharinder.c.googlers.com ([fda3:e722:ac3:cc00:4f:4b78:c0a8:c73]) (user=sharinder job=sendgmr) by 2002:a25:3626:: with SMTP id d38mr2083774yba.451.1639715891393; Thu, 16 Dec 2021 20:38:11 -0800 (PST) Date: Fri, 17 Dec 2021 04:37:14 +0000 In-Reply-To: <20211217043716.794289-1-sharinder@google.com> Message-Id: <20211217043716.794289-6-sharinder@google.com> Mime-Version: 1.0 References: <20211217043716.794289-1-sharinder@google.com> X-Mailer: git-send-email 2.34.1.173.g76aa8bc2d0-goog Subject: [PATCH v5 5/7] Documentation: KUnit: Rework writing page to focus on writing tests From: Harinder Singh To: davidgow@google.com, brendanhiggins@google.com, shuah@kernel.org, corbet@lwn.net Cc: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, tim.bird@sony.com, elver@google.com, Harinder Singh Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" We now have dedicated pages on running tests. Therefore refocus the usage page on writing tests and add content from tips page and information on other architectures. Signed-off-by: Harinder Singh Reviewed-by: Tim Bird --- Documentation/dev-tools/kunit/index.rst | 2 +- Documentation/dev-tools/kunit/start.rst | 2 +- Documentation/dev-tools/kunit/usage.rst | 578 +++++++++++------------- 3 files changed, 255 insertions(+), 327 deletions(-) diff --git a/Documentation/dev-tools/kunit/index.rst b/Documentation/dev-to= ols/kunit/index.rst index f9f37997b58c..595205348d2d 100644 --- a/Documentation/dev-tools/kunit/index.rst +++ b/Documentation/dev-tools/kunit/index.rst @@ -102,7 +102,7 @@ How do I use it? * Documentation/dev-tools/kunit/architecture.rst - KUnit architecture. * Documentation/dev-tools/kunit/run_wrapper.rst - run kunit_tool. * Documentation/dev-tools/kunit/run_manual.rst - run tests without kunit= _tool. -* Documentation/dev-tools/kunit/usage.rst - KUnit features. +* Documentation/dev-tools/kunit/usage.rst - write tests. * Documentation/dev-tools/kunit/tips.rst - best practices with examples. * Documentation/dev-tools/kunit/api/index.rst - KUnit APIs diff --git a/Documentation/dev-tools/kunit/start.rst b/Documentation/dev-to= ols/kunit/start.rst index af13f443c976..a858ab009944 100644 --- a/Documentation/dev-tools/kunit/start.rst +++ b/Documentation/dev-tools/kunit/start.rst @@ -243,7 +243,7 @@ Next Steps * Documentation/dev-tools/kunit/architecture.rst - KUnit architecture. * Documentation/dev-tools/kunit/run_wrapper.rst - run kunit_tool. * Documentation/dev-tools/kunit/run_manual.rst - run tests without kunit= _tool. -* Documentation/dev-tools/kunit/usage.rst - KUnit features. +* Documentation/dev-tools/kunit/usage.rst - write tests. * Documentation/dev-tools/kunit/tips.rst - best practices with examples. * Documentation/dev-tools/kunit/api/index.rst - KUnit APIs diff --git a/Documentation/dev-tools/kunit/usage.rst b/Documentation/dev-to= ols/kunit/usage.rst index 63f1bb89ebf5..76af931a332c 100644 --- a/Documentation/dev-tools/kunit/usage.rst +++ b/Documentation/dev-tools/kunit/usage.rst @@ -1,57 +1,13 @@ .. SPDX-License-Identifier: GPL-2.0 =20 -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D -Using KUnit -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D - -The purpose of this document is to describe what KUnit is, how it works, h= ow it -is intended to be used, and all the concepts and terminology that are need= ed to -understand it. This guide assumes a working knowledge of the Linux kernel = and -some basic knowledge of testing. - -For a high level introduction to KUnit, including setting up KUnit for your -project, see Documentation/dev-tools/kunit/start.rst. - -Organization of this document -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D - -This document is organized into two main sections: Testing and Common Patt= erns. -The first covers what unit tests are and how to use KUnit to write them. T= he -second covers common testing patterns, e.g. how to isolate code and make it -possible to unit test code that was otherwise un-unit-testable. - -Testing -=3D=3D=3D=3D=3D=3D=3D - -What is KUnit? --------------- - -"K" is short for "kernel" so "KUnit" is the "(Linux) Kernel Unit Testing -Framework." KUnit is intended first and foremost for writing unit tests; i= t is -general enough that it can be used to write integration tests; however, th= is is -a secondary goal. KUnit has no ambition of being the only testing framewor= k for -the kernel; for example, it does not intend to be an end-to-end testing -framework. - -What is Unit Testing? ---------------------- - -A `unit test `_ is a test th= at -tests code at the smallest possible scope, a *unit* of code. In the C -programming language that's a function. - -Unit tests should be written for all the publicly exposed functions in a -compilation unit; so that is all the functions that are exported in either= a -*class* (defined below) or all functions which are **not** static. - Writing Tests -------------- +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 Test Cases -~~~~~~~~~~ +---------- =20 The fundamental unit in KUnit is the test case. A test case is a function = with -the signature ``void (*)(struct kunit *test)``. It calls a function to be = tested +the signature ``void (*)(struct kunit *test)``. It calls the function unde= r test and then sets *expectations* for what should happen. For example: =20 .. code-block:: c @@ -65,18 +21,19 @@ and then sets *expectations* for what should happen. Fo= r example: KUNIT_FAIL(test, "This test never passes."); } =20 -In the above example ``example_test_success`` always passes because it does -nothing; no expectations are set, so all expectations pass. On the other h= and -``example_test_failure`` always fails because it calls ``KUNIT_FAIL``, whi= ch is -a special expectation that logs a message and causes the test case to fail. +In the above example, ``example_test_success`` always passes because it do= es +nothing; no expectations are set, and therefore all expectations pass. On = the +other hand ``example_test_failure`` always fails because it calls ``KUNIT_= FAIL``, +which is a special expectation that logs a message and causes the test cas= e to +fail. =20 Expectations ~~~~~~~~~~~~ -An *expectation* is a way to specify that you expect a piece of code to do -something in a test. An expectation is called like a function. A test is m= ade -by setting expectations about the behavior of a piece of code under test; = when -one or more of the expectations fail, the test case fails and information = about -the failure is logged. For example: +An *expectation* specifies that we expect a piece of code to do something = in a +test. An expectation is called like a function. A test is made by setting +expectations about the behavior of a piece of code under test. When one or= more +expectations fail, the test case fails and information about the failure is +logged. For example: =20 .. code-block:: c =20 @@ -86,29 +43,28 @@ the failure is logged. For example: KUNIT_EXPECT_EQ(test, 2, add(1, 1)); } =20 -In the above example ``add_test_basic`` makes a number of assertions about= the -behavior of a function called ``add``; the first parameter is always of ty= pe -``struct kunit *``, which contains information about the current test cont= ext; -the second parameter, in this case, is what the value is expected to be; t= he +In the above example, ``add_test_basic`` makes a number of assertions abou= t the +behavior of a function called ``add``. The first parameter is always of ty= pe +``struct kunit *``, which contains information about the current test cont= ext. +The second parameter, in this case, is what the value is expected to be. T= he last value is what the value actually is. If ``add`` passes all of these expectations, the test case, ``add_test_basic`` will pass; if any one of t= hese expectations fails, the test case will fail. =20 -It is important to understand that a test case *fails* when any expectatio= n is -violated; however, the test will continue running, potentially trying other -expectations until the test case ends or is otherwise terminated. This is = as -opposed to *assertions* which are discussed later. +A test case *fails* when any expectation is violated; however, the test wi= ll +continue to run, and try other expectations until the test case ends or is +otherwise terminated. This is as opposed to *assertions* which are discuss= ed +later. =20 -To learn about more expectations supported by KUnit, see -Documentation/dev-tools/kunit/api/test.rst. +To learn about more KUnit expectations, see Documentation/dev-tools/kunit/= api/test.rst. =20 .. note:: - A single test case should be pretty short, pretty easy to understand, - focused on a single behavior. + A single test case should be short, easy to understand, and focused on a + single behavior. =20 -For example, if we wanted to properly test the add function above, we would -create additional tests cases which would each test a different property t= hat an -add function should have like this: +For example, if we want to rigorously test the ``add`` function above, cre= ate +additional tests cases which would test each property that an ``add`` func= tion +should have as shown below: =20 .. code-block:: c =20 @@ -134,56 +90,43 @@ add function should have like this: KUNIT_EXPECT_EQ(test, INT_MIN, add(INT_MAX, 1)); } =20 -Notice how it is immediately obvious what all the properties that we are t= esting -for are. - Assertions ~~~~~~~~~~ =20 -KUnit also has the concept of an *assertion*. An assertion is just like an -expectation except the assertion immediately terminates the test case if i= t is -not satisfied. - -For example: +An assertion is like an expectation, except that the assertion immediately +terminates the test case if the condition is not satisfied. For example: =20 .. code-block:: c =20 - static void mock_test_do_expect_default_return(struct kunit *test) + static void test_sort(struct kunit *test) { - struct mock_test_context *ctx =3D test->priv; - struct mock *mock =3D ctx->mock; - int param0 =3D 5, param1 =3D -5; - const char *two_param_types[] =3D {"int", "int"}; - const void *two_params[] =3D {¶m0, ¶m1}; - const void *ret; - - ret =3D mock->do_expect(mock, - "test_printk", test_printk, - two_param_types, two_params, - ARRAY_SIZE(two_params)); - KUNIT_ASSERT_NOT_ERR_OR_NULL(test, ret); - KUNIT_EXPECT_EQ(test, -4, *((int *) ret)); + int *a, i, r =3D 1; + a =3D kunit_kmalloc_array(test, TEST_LEN, sizeof(*a), GFP_KERNEL); + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, a); + for (i =3D 0; i < TEST_LEN; i++) { + r =3D (r * 725861) % 6599; + a[i] =3D r; + } + sort(a, TEST_LEN, sizeof(*a), cmpint, NULL); + for (i =3D 0; i < TEST_LEN-1; i++) + KUNIT_EXPECT_LE(test, a[i], a[i + 1]); } =20 -In this example, the method under test should return a pointer to a value,= so -if the pointer returned by the method is null or an errno, we don't want to -bother continuing the test since the following expectation could crash the= test -case. `ASSERT_NOT_ERR_OR_NULL(...)` allows us to bail out of the test case= if -the appropriate conditions have not been satisfied to complete the test. +In this example, the method under test should return pointer to a value. I= f the +pointer returns null or an errno, we want to stop the test since the follo= wing +expectation could crash the test case. `ASSERT_NOT_ERR_OR_NULL(...)` allow= s us +to bail out of the test case if the appropriate conditions are not satisfi= ed to +complete the test. =20 Test Suites ~~~~~~~~~~~ =20 -Now obviously one unit test isn't very helpful; the power comes from having -many test cases covering all of a unit's behaviors. Consequently it is com= mon -to have many *similar* tests; in order to reduce duplication in these clos= ely -related tests most unit testing frameworks - including KUnit - provide the -concept of a *test suite*. A *test suite* is just a collection of test cas= es -for a unit of code with a set up function that gets invoked before every t= est -case and then a tear down function that gets invoked after every test case -completes. - -Example: +We need many test cases covering all the unit's behaviors. It is common to= have +many similar tests. In order to reduce duplication in these closely related +tests, most unit testing frameworks (including KUnit) provide the concept = of a +*test suite*. A test suite is a collection of test cases for a unit of code +with a setup function that gets invoked before every test case and then a = tear +down function that gets invoked after every test case completes. For examp= le: =20 .. code-block:: c =20 @@ -202,23 +145,48 @@ Example: }; kunit_test_suite(example_test_suite); =20 -In the above example the test suite, ``example_test_suite``, would run the= test -cases ``example_test_foo``, ``example_test_bar``, and ``example_test_baz``; -each would have ``example_test_init`` called immediately before it and wou= ld -have ``example_test_exit`` called immediately after it. +In the above example, the test suite ``example_test_suite`` would run the = test +cases ``example_test_foo``, ``example_test_bar``, and ``example_test_baz``= . Each +would have ``example_test_init`` called immediately before it and +``example_test_exit`` called immediately after it. ``kunit_test_suite(example_test_suite)`` registers the test suite with the KUnit test framework. =20 .. note:: - A test case will only be run if it is associated with a test suite. + A test case will only run if it is associated with a test suite. + +``kunit_test_suite(...)`` is a macro which tells the linker to put the +specified test suite in a special linker section so that it can be run by = KUnit +either after ``late_init``, or when the test module is loaded (if the test= was +built as a module). + +For more information, see Documentation/dev-tools/kunit/api/test.rst. + +Writing Tests For Other Architectures +------------------------------------- + +It is better to write tests that run on UML to tests that only run under a +particular architecture. It is better to write tests that run under QEMU or +another easy to obtain (and monetarily free) software environment to a spe= cific +piece of hardware. =20 -``kunit_test_suite(...)`` is a macro which tells the linker to put the spe= cified -test suite in a special linker section so that it can be run by KUnit eith= er -after late_init, or when the test module is loaded (depending on whether t= he -test was built in or not). +Nevertheless, there are still valid reasons to write a test that is archit= ecture +or hardware specific. For example, we might want to test code that really +belongs in ``arch/some-arch/*``. Even so, try to write the test so that it= does +not depend on physical hardware. Some of our test cases may not need hardw= are, +only few tests actually require the hardware to test it. When hardware is = not +available, instead of disabling tests, we can skip them. =20 -For more information on these types of things see the -Documentation/dev-tools/kunit/api/test.rst. +Now that we have narrowed down exactly what bits are hardware specific, the +actual procedure for writing and running the tests is same as writing norm= al +KUnit tests. + +.. important:: + We may have to reset hardware state. If this is not possible, we may on= ly + be able to run one test case per invocation. + +.. TODO(brendanhiggins@google.com): Add an actual example of an architectu= re- + dependent KUnit test. =20 Common Patterns =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D @@ -226,43 +194,39 @@ Common Patterns Isolating Behavior ------------------ =20 -The most important aspect of unit testing that other forms of testing do n= ot -provide is the ability to limit the amount of code under test to a single = unit. -In practice, this is only possible by being able to control what code gets= run -when the unit under test calls a function and this is usually accomplished -through some sort of indirection where a function is exposed as part of an= API -such that the definition of that function can be changed without affecting= the -rest of the code base. In the kernel this primarily comes from two constru= cts, -classes, structs that contain function pointers that are provided by the -implementer, and architecture-specific functions which have definitions se= lected -at compile time. +Unit testing limits the amount of code under test to a single unit. It con= trols +what code gets run when the unit under test calls a function. Where a func= tion +is exposed as part of an API such that the definition of that function can= be +changed without affecting the rest of the code base. In the kernel, this c= omes +from two constructs: classes, which are structs that contain function poin= ters +provided by the implementer, and architecture-specific functions, which ha= ve +definitions selected at compile time. =20 Classes ~~~~~~~ =20 Classes are not a construct that is built into the C programming language; -however, it is an easily derived concept. Accordingly, pretty much every p= roject -that does not use a standardized object oriented library (like GNOME's GOb= ject) -has their own slightly different way of doing object oriented programming;= the -Linux kernel is no exception. +however, it is an easily derived concept. Accordingly, in most cases, every +project that does not use a standardized object oriented library (like GNO= ME's +GObject) has their own slightly different way of doing object oriented +programming; the Linux kernel is no exception. =20 The central concept in kernel object oriented programming is the class. In= the kernel, a *class* is a struct that contains function pointers. This create= s a contract between *implementers* and *users* since it forces them to use the -same function signature without having to call the function directly. In o= rder -for it to truly be a class, the function pointers must specify that a poin= ter -to the class, known as a *class handle*, be one of the parameters; this ma= kes -it possible for the member functions (also known as *methods*) to have acc= ess -to member variables (more commonly known as *fields*) allowing the same -implementation to have multiple *instances*. - -Typically a class can be *overridden* by *child classes* by embedding the -*parent class* in the child class. Then when a method provided by the child -class is called, the child implementation knows that the pointer passed to= it is -of a parent contained within the child; because of this, the child can com= pute -the pointer to itself because the pointer to the parent is always a fixed = offset -from the pointer to the child; this offset is the offset of the parent con= tained -in the child struct. For example: +same function signature without having to call the function directly. To b= e a +class, the function pointers must specify that a pointer to the class, kno= wn as +a *class handle*, be one of the parameters. Thus the member functions (also +known as *methods*) have access to member variables (also known as *fields= *) +allowing the same implementation to have multiple *instances*. + +A class can be *overridden* by *child classes* by embedding the *parent cl= ass* +in the child class. Then when the child class *method* is called, the child +implementation knows that the pointer passed to it is of a parent contained +within the child. Thus, the child can compute the pointer to itself becaus= e the +pointer to the parent is always a fixed offset from the pointer to the chi= ld. +This offset is the offset of the parent contained in the child struct. For +example: =20 .. code-block:: c =20 @@ -290,8 +254,8 @@ in the child struct. For example: self->width =3D width; } =20 -In this example (as in most kernel code) the operation of computing the po= inter -to the child from the pointer to the parent is done by ``container_of``. +In this example, computing the pointer to the child from the pointer to the +parent is done by ``container_of``. =20 Faking Classes ~~~~~~~~~~~~~~ @@ -300,14 +264,11 @@ In order to unit test a piece of code that calls a me= thod in a class, the behavior of the method must be controllable, otherwise the test ceases to = be a unit test and becomes an integration test. =20 -A fake just provides an implementation of a piece of code that is differen= t than -what runs in a production instance, but behaves identically from the stand= point -of the callers; this is usually done to replace a dependency that is hard = to -deal with, or is slow. - -A good example for this might be implementing a fake EEPROM that just stor= es the -"contents" in an internal buffer. For example, let's assume we have a clas= s that -represents an EEPROM: +A fake class implements a piece of code that is different than what runs i= n a +production instance, but behaves identical from the standpoint of the call= ers. +This is done to replace a dependency that is hard to deal with, or is slow= . For +example, implementing a fake EEPROM that stores the "contents" in an +internal buffer. Assume we have a class that represents an EEPROM: =20 .. code-block:: c =20 @@ -316,7 +277,7 @@ represents an EEPROM: ssize_t (*write)(struct eeprom *this, size_t offset, const char *buffer,= size_t count); }; =20 -And we want to test some code that buffers writes to the EEPROM: +And we want to test code that buffers writes to the EEPROM: =20 .. code-block:: c =20 @@ -329,7 +290,7 @@ And we want to test some code that buffers writes to th= e EEPROM: struct eeprom_buffer *new_eeprom_buffer(struct eeprom *eeprom); void destroy_eeprom_buffer(struct eeprom *eeprom); =20 -We can easily test this code by *faking out* the underlying EEPROM: +We can test this code by *faking out* the underlying EEPROM: =20 .. code-block:: c =20 @@ -456,14 +417,14 @@ We can now use it to test ``struct eeprom_buffer``: destroy_eeprom_buffer(ctx->eeprom_buffer); } =20 -Testing against multiple inputs +Testing Against Multiple Inputs ------------------------------- =20 -Testing just a few inputs might not be enough to have confidence that the = code -works correctly, e.g. for a hash function. +Testing just a few inputs is not enough to ensure that the code works corr= ectly, +for example: testing a hash function. =20 -In such cases, it can be helpful to have a helper macro or function, e.g. = this -fictitious example for ``sha1sum(1)`` +We can write a helper macro or function. The function is called for each i= nput. +For example, to test ``sha1sum(1)``, we can write: =20 .. code-block:: c =20 @@ -475,16 +436,15 @@ fictitious example for ``sha1sum(1)`` TEST_SHA1("hello world", "2aae6c35c94fcfb415dbe95f408b9ce91ee846ed"); TEST_SHA1("hello world!", "430ce34d020724ed75a196dfc2ad67c77772d169"); =20 +Note the use of the ``_MSG`` version of ``KUNIT_EXPECT_STREQ`` to print a = more +detailed error and make the assertions clearer within the helper macros. =20 -Note the use of ``KUNIT_EXPECT_STREQ_MSG`` to give more context when it fa= ils -and make it easier to track down. (Yes, in this example, ``want`` is likely -going to be unique enough on its own). +The ``_MSG`` variants are useful when the same expectation is called multi= ple +times (in a loop or helper function) and thus the line number is not enoug= h to +identify what failed, as shown below. =20 -The ``_MSG`` variants are even more useful when the same expectation is ca= lled -multiple times (in a loop or helper function) and thus the line number isn= 't -enough to identify what failed, like below. - -In some cases, it can be helpful to write a *table-driven test* instead, e= .g. +In complicated cases, we recommend using a *table-driven test* compared to= the +helper macro variation, for example: =20 .. code-block:: c =20 @@ -513,17 +473,18 @@ In some cases, it can be helpful to write a *table-dr= iven test* instead, e.g. } =20 =20 -There's more boilerplate involved, but it can: +There is more boilerplate code involved, but it can: + +* be more readable when there are multiple inputs/outputs (due to field na= mes). =20 -* be more readable when there are multiple inputs/outputs thanks to field = names, + * For example, see ``fs/ext4/inode-test.c``. =20 - * E.g. see ``fs/ext4/inode-test.c`` for an example of both. -* reduce duplication if test cases can be shared across multiple tests. +* reduce duplication if test cases are shared across multiple tests. =20 - * E.g. if we wanted to also test ``sha256sum``, we could add a ``sha256`` + * For example: if we want to test ``sha256sum``, we could add a ``sha256= `` field and reuse ``cases``. =20 -* be converted to a "parameterized test", see below. +* be converted to a "parameterized test". =20 Parameterized Testing ~~~~~~~~~~~~~~~~~~~~~ @@ -531,7 +492,7 @@ Parameterized Testing The table-driven testing pattern is common enough that KUnit has special support for it. =20 -Reusing the same ``cases`` array from above, we can write the test as a +By reusing the same ``cases`` array from above, we can write the test as a "parameterized test" with the following. =20 .. code-block:: c @@ -582,193 +543,160 @@ Reusing the same ``cases`` array from above, we can= write the test as a =20 .. _kunit-on-non-uml: =20 -KUnit on non-UML architectures -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D - -By default KUnit uses UML as a way to provide dependencies for code under = test. -Under most circumstances KUnit's usage of UML should be treated as an -implementation detail of how KUnit works under the hood. Nevertheless, the= re -are instances where being able to run architecture-specific code or test -against real hardware is desirable. For these reasons KUnit supports runni= ng on -other architectures. - -Running existing KUnit tests on non-UML architectures ------------------------------------------------------ +Exiting Early on Failed Expectations +------------------------------------ =20 -There are some special considerations when running existing KUnit tests on -non-UML architectures: +We can use ``KUNIT_EXPECT_EQ`` to mark the test as failed and continue +execution. In some cases, it is unsafe to continue. We can use the +``KUNIT_ASSERT`` variant to exit on failure. =20 -* Hardware may not be deterministic, so a test that always passes or fai= ls - when run under UML may not always do so on real hardware. -* Hardware and VM environments may not be hermetic. KUnit tries its best= to - provide a hermetic environment to run tests; however, it cannot manage= state - that it doesn't know about outside of the kernel. Consequently, tests = that - may be hermetic on UML may not be hermetic on other architectures. -* Some features and tooling may not be supported outside of UML. -* Hardware and VMs are slower than UML. +.. code-block:: c =20 -None of these are reasons not to run your KUnit tests on real hardware; th= ey are -only things to be aware of when doing so. + void example_test_user_alloc_function(struct kunit *test) + { + void *object =3D alloc_some_object_for_me(); =20 -Currently, the KUnit Wrapper (``tools/testing/kunit/kunit.py``) (aka -kunit_tool) only fully supports running tests inside of UML and QEMU; howe= ver, -this is only due to our own time limitations as humans working on KUnit. I= t is -entirely possible to support other emulators and even actual hardware, but= for -now QEMU and UML is what is fully supported within the KUnit Wrapper. Agai= n, to -be clear, this is just the Wrapper. The actualy KUnit tests and the KUnit -library they are written in is fully architecture agnostic and can be used= in -virtually any setup, you just won't have the benefit of typing a single co= mmand -out of the box and having everything magically work perfectly. + /* Make sure we got a valid pointer back. */ + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, object); + do_something_with_object(object); + } =20 -Again, all core KUnit framework features are fully supported on all -architectures, and using them is straightforward: Most popular architectur= es -are supported directly in the KUnit Wrapper via QEMU. Currently, supported -architectures on QEMU include: +Allocating Memory +----------------- =20 -* i386 -* x86_64 -* arm -* arm64 -* alpha -* powerpc -* riscv -* s390 -* sparc +Where you might use ``kzalloc``, you can instead use ``kunit_kzalloc`` as = KUnit +will then ensure that the memory is freed once the test completes. =20 -In order to run KUnit tests on one of these architectures via QEMU with the -KUnit wrapper, all you need to do is specify the flags ``--arch`` and -``--cross_compile`` when invoking the KUnit Wrapper. For example, we could= run -the default KUnit tests on ARM in the following manner (assuming we have a= n ARM -toolchain installed): +This is useful because it lets us use the ``KUNIT_ASSERT_EQ`` macros to ex= it +early from a test without having to worry about remembering to call ``kfre= e``. +For example: =20 -.. code-block:: bash +.. code-block:: c =20 - tools/testing/kunit/kunit.py run --timeout=3D60 --jobs=3D12 --arch=3Darm = --cross_compile=3Darm-linux-gnueabihf- + void example_test_allocation(struct kunit *test) + { + char *buffer =3D kunit_kzalloc(test, 16, GFP_KERNEL); + /* Ensure allocation succeeded. */ + KUNIT_ASSERT_NOT_ERR_OR_NULL(test, buffer); =20 -Alternatively, if you want to run your tests on real hardware or in some o= ther -emulation environment, all you need to do is to take your kunitconfig, your -Kconfig options for the tests you would like to run, and merge them into -whatever config your are using for your platform. That's it! + KUNIT_ASSERT_STREQ(test, buffer, ""); + } =20 -For example, let's say you have the following kunitconfig: =20 -.. code-block:: none +Testing Static Functions +------------------------ =20 - CONFIG_KUNIT=3Dy - CONFIG_KUNIT_EXAMPLE_TEST=3Dy +If we do not want to expose functions or variables for testing, one option= is to +conditionally ``#include`` the test file at the end of your .c file. For +example: =20 -If you wanted to run this test on an x86 VM, you might add the following c= onfig -options to your ``.config``: +.. code-block:: c =20 -.. code-block:: none + /* In my_file.c */ =20 - CONFIG_KUNIT=3Dy - CONFIG_KUNIT_EXAMPLE_TEST=3Dy - CONFIG_SERIAL_8250=3Dy - CONFIG_SERIAL_8250_CONSOLE=3Dy + static int do_interesting_thing(); =20 -All these new options do is enable support for a common serial console nee= ded -for logging. + #ifdef CONFIG_MY_KUNIT_TEST + #include "my_kunit_test.c" + #endif =20 -Next, you could build a kernel with these tests as follows: +Injecting Test-Only Code +------------------------ =20 +Similar to as shown above, we can add test-specific logic. For example: =20 -.. code-block:: bash +.. code-block:: c =20 - make ARCH=3Dx86 olddefconfig - make ARCH=3Dx86 + /* In my_file.h */ =20 -Once you have built a kernel, you could run it on QEMU as follows: + #ifdef CONFIG_MY_KUNIT_TEST + /* Defined in my_kunit_test.c */ + void test_only_hook(void); + #else + void test_only_hook(void) { } + #endif =20 -.. code-block:: bash +This test-only code can be made more useful by accessing the current ``kun= it_test`` +as shown in next section: *Accessing The Current Test*. =20 - qemu-system-x86_64 -enable-kvm \ - -m 1024 \ - -kernel arch/x86_64/boot/bzImage \ - -append 'console=3DttyS0' \ - --nographic +Accessing The Current Test +-------------------------- =20 -Interspersed in the kernel logs you might see the following: +In some cases, we need to call test-only code from outside the test file. +For example, see example in section *Injecting Test-Only Code* or if +we are providing a fake implementation of an ops struct. Using +``kunit_test`` field in ``task_struct``, we can access it via +``current->kunit_test``. =20 -.. code-block:: none +The example below includes how to implement "mocking": =20 - TAP version 14 - # Subtest: example - 1..1 - # example_simple_test: initializing - ok 1 - example_simple_test - ok 1 - example +.. code-block:: c =20 -Congratulations, you just ran a KUnit test on the x86 architecture! + #include /* for current */ =20 -In a similar manner, kunit and kunit tests can also be built as modules, -so if you wanted to run tests in this way you might add the following conf= ig -options to your ``.config``: + struct test_data { + int foo_result; + int want_foo_called_with; + }; =20 -.. code-block:: none + static int fake_foo(int arg) + { + struct kunit *test =3D current->kunit_test; + struct test_data *test_data =3D test->priv; =20 - CONFIG_KUNIT=3Dm - CONFIG_KUNIT_EXAMPLE_TEST=3Dm + KUNIT_EXPECT_EQ(test, test_data->want_foo_called_with, arg); + return test_data->foo_result; + } =20 -Once the kernel is built and installed, a simple + static void example_simple_test(struct kunit *test) + { + /* Assume priv (private, a member used to pass test data from + * the init function) is allocated in the suite's .init */ + struct test_data *test_data =3D test->priv; =20 -.. code-block:: bash + test_data->foo_result =3D 42; + test_data->want_foo_called_with =3D 1; =20 - modprobe example-test + /* In a real test, we'd probably pass a pointer to fake_foo somewhere + * like an ops struct, etc. instead of calling it directly. */ + KUNIT_EXPECT_EQ(test, fake_foo(1), 42); + } =20 -...will run the tests. +In this example, we are using the ``priv`` member of ``struct kunit`` as a= way +of passing data to the test from the init function. In general ``priv`` is +pointer that can be used for any user data. This is preferred over static +variables, as it avoids concurrency issues. =20 -.. note:: - Note that you should make sure your test depends on ``KUNIT=3Dy`` in Kc= onfig - if the test does not support module build. Otherwise, it will trigger - compile errors if ``CONFIG_KUNIT`` is ``m``. +Had we wanted something more flexible, we could have used a named ``kunit_= resource``. +Each test can have multiple resources which have string names providing th= e same +flexibility as a ``priv`` member, but also, for example, allowing helper +functions to create resources without conflicting with each other. It is a= lso +possible to define a clean up function for each resource, making it easy to +avoid resource leaks. For more information, see Documentation/dev-tools/ku= nit/api/test.rst. =20 -Writing new tests for other architectures ------------------------------------------ +Failing The Current Test +------------------------ =20 -The first thing you must do is ask yourself whether it is necessary to wri= te a -KUnit test for a specific architecture, and then whether it is necessary to -write that test for a particular piece of hardware. In general, writing a = test -that depends on having access to a particular piece of hardware or softwar= e (not -included in the Linux source repo) should be avoided at all costs. +If we want to fail the current test, we can use ``kunit_fail_current_test(= fmt, args...)`` +which is defined in ```` and does not require pulling in= ````. +For example, we have an option to enable some extra debug checks on some d= ata +structures as shown below: =20 -Even if you only ever plan on running your KUnit test on your hardware -configuration, other people may want to run your tests and may not have ac= cess -to your hardware. If you write your test to run on UML, then anyone can ru= n your -tests without knowing anything about your particular setup, and you can st= ill -run your tests on your hardware setup just by compiling for your architect= ure. +.. code-block:: c =20 -.. important:: - Always prefer tests that run on UML to tests that only run under a part= icular - architecture, and always prefer tests that run under QEMU or another ea= sy - (and monetarily free) to obtain software environment to a specific piec= e of - hardware. - -Nevertheless, there are still valid reasons to write an architecture or ha= rdware -specific test: for example, you might want to test some code that really b= elongs -in ``arch/some-arch/*``. Even so, try your best to write the test so that = it -does not depend on physical hardware: if some of your test cases don't nee= d the -hardware, only require the hardware for tests that actually need it. - -Now that you have narrowed down exactly what bits are hardware specific, t= he -actual procedure for writing and running the tests is pretty much the same= as -writing normal KUnit tests. One special caveat is that you have to reset -hardware state in between test cases; if this is not possible, you may onl= y be -able to run one test case per invocation. + #include =20 -.. TODO(brendanhiggins@google.com): Add an actual example of an architectu= re- - dependent KUnit test. + #ifdef CONFIG_EXTRA_DEBUG_CHECKS + static void validate_my_data(struct data *data) + { + if (is_valid(data)) + return; =20 -KUnit debugfs representation -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D -When kunit test suites are initialized, they create an associated directory -in ``/sys/kernel/debug/kunit/``. The directory contains one f= ile + kunit_fail_current_test("data %p is invalid", data); =20 -- results: "cat results" displays results of each test case and the results - of the entire suite for the last test run. + /* Normal, non-KUnit, error reporting code here. */ + } + #else + static void my_debug_function(void) { } + #endif =20 -The debugfs representation is primarily of use when kunit test suites are -run in a native environment, either as modules or builtin. Having a way -to display results like this is valuable as otherwise results can be -intermixed with other events in dmesg output. The maximum size of each -results file is KUNIT_LOG_SIZE bytes (defined in ``include/kunit/test.h``). --=20 2.34.1.173.g76aa8bc2d0-goog From nobody Wed Jul 1 18:37:12 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 94FE5C433FE for ; Fri, 17 Dec 2021 04:38:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232799AbhLQEiU (ORCPT ); Thu, 16 Dec 2021 23:38:20 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48808 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232813AbhLQEiR (ORCPT ); Thu, 16 Dec 2021 23:38:17 -0500 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4F0F9C061574 for ; Thu, 16 Dec 2021 20:38:17 -0800 (PST) Received: by mail-yb1-xb4a.google.com with SMTP id w9-20020a25c709000000b00608a9489fc1so2402848ybe.20 for ; Thu, 16 Dec 2021 20:38:17 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc:content-transfer-encoding; bh=4R5iddFr+IUod/lbvTdO8uVUTfoKwT100oJMRYRCd3c=; b=GiL0ljqNfrIO5ppeZmn2UjENlxeLrATwmqMeFtKn1F55Pw4FXdtHFtAweYvo5vLoJA +yQuab55OIWuxr2T/DjgQ20MK3mE4meGgjl0+EqUv4Wobh7C90SUBeeAprIzGzUDYP3n aGSy/VYdV8x/19UmuNSnpbqYyYdrrEtZ5P+zOmtUoGV8rS7C5GyXmDyMnnZnccDneY3y yQRQsfqlrLAK7TtyCgz7xzjMvXDXZtdeOvPcYXG/rxXjx5FHlxcdSIb7hVcm/6RJ9VA1 GyHJn/6OKz0cClJfzPAbtrMN9D+Jt5WYZDirdGd1OhydhtQYqDfSGDQ4cqKMuYKISNdR 33EQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc:content-transfer-encoding; bh=4R5iddFr+IUod/lbvTdO8uVUTfoKwT100oJMRYRCd3c=; b=va/IDZHHm4B+cHKGJQrUwFlIF6639ETARaoTcRUsgaJ4RTSzjh39J+CLkNeASP7pZm jvAS98WDu/DFNSwLd+Pm46W7C778w36MSv29WbFycA6xxMY3/alJ+gDPc+Fu0USOENga Xq/oEiNqe6PeRQ7tmhRsrSWLshTVSvqfYkuX8N7pTi7xozt5jVBj7PdxRBab6gTKvlya 3B6OChGMAXEQWolb/lc/OWbbrxxe+8WTt3ngAeeXjexoKt1n9aUkPmFrgwl4+euYaYON rUMRfWYMkmKx1o57aOdjuxOrqwiMP3MRwxhjzO0FHqaGs8zoW3ghKl8/ecy9lPG2oFCv Zb1A== X-Gm-Message-State: AOAM532D6B1/MiYQfuMTPkqPEjuSx2/33PBn6ui5znEjHyL/Rz1Mez1w tHhgSliSMWglCTmDxhwFDQdbzkkS80CxYz0= X-Google-Smtp-Source: ABdhPJyOdH7mdhGzcWp8jfRAnigRXzODr5GFFtMi2d3/iLvuNRXMZCIBmLGnmNtGqpQNbVjMliim5wJ98ludZmc= X-Received: from sharinder.c.googlers.com ([fda3:e722:ac3:cc00:4f:4b78:c0a8:c73]) (user=sharinder job=sendgmr) by 2002:a25:7209:: with SMTP id n9mr1970786ybc.692.1639715896583; Thu, 16 Dec 2021 20:38:16 -0800 (PST) Date: Fri, 17 Dec 2021 04:37:15 +0000 In-Reply-To: <20211217043716.794289-1-sharinder@google.com> Message-Id: <20211217043716.794289-7-sharinder@google.com> Mime-Version: 1.0 References: <20211217043716.794289-1-sharinder@google.com> X-Mailer: git-send-email 2.34.1.173.g76aa8bc2d0-goog Subject: [PATCH v5 6/7] Documentation: KUnit: Restyle Test Style and Nomenclature page From: Harinder Singh To: davidgow@google.com, brendanhiggins@google.com, shuah@kernel.org, corbet@lwn.net Cc: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, tim.bird@sony.com, elver@google.com, Harinder Singh Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Rewrite page to enhance content consistency. Signed-off-by: Harinder Singh --- Documentation/dev-tools/kunit/style.rst | 105 ++++++++++++------------ 1 file changed, 51 insertions(+), 54 deletions(-) diff --git a/Documentation/dev-tools/kunit/style.rst b/Documentation/dev-to= ols/kunit/style.rst index 8dbcdc552606..b6d0d7359f00 100644 --- a/Documentation/dev-tools/kunit/style.rst +++ b/Documentation/dev-tools/kunit/style.rst @@ -4,37 +4,36 @@ Test Style and Nomenclature =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D =20 -To make finding, writing, and using KUnit tests as simple as possible, it's +To make finding, writing, and using KUnit tests as simple as possible, it = is strongly encouraged that they are named and written according to the guide= lines -below. While it's possible to write KUnit tests which do not follow these = rules, +below. While it is possible to write KUnit tests which do not follow these= rules, they may break some tooling, may conflict with other tests, and may not be= run automatically by testing systems. =20 -It's recommended that you only deviate from these guidelines when: +It is recommended that you only deviate from these guidelines when: =20 -1. Porting tests to KUnit which are already known with an existing name, or -2. Writing tests which would cause serious problems if automatically run (= e.g., - non-deterministically producing false positives or negatives, or taking= an - extremely long time to run). +1. Porting tests to KUnit which are already known with an existing name. +2. Writing tests which would cause serious problems if automatically run. = For + example, non-deterministically producing false positives or negatives, = or + taking a long time to run. =20 Subsystems, Suites, and Tests =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D =20 -In order to make tests as easy to find as possible, they're grouped into s= uites -and subsystems. A test suite is a group of tests which test a related area= of -the kernel, and a subsystem is a set of test suites which test different p= arts -of the same kernel subsystem or driver. +To make tests easy to find, they are grouped into suites and subsystems. A= test +suite is a group of tests which test a related area of the kernel. A subsy= stem +is a set of test suites which test different parts of a kernel subsystem +or a driver. =20 Subsystems ---------- =20 Every test suite must belong to a subsystem. A subsystem is a collection o= f one or more KUnit test suites which test the same driver or part of the kernel= . A -rule of thumb is that a test subsystem should match a single kernel module= . If -the code being tested can't be compiled as a module, in many cases the sub= system -should correspond to a directory in the source tree or an entry in the -MAINTAINERS file. If unsure, follow the conventions set by tests in similar -areas. +test subsystem should match a single kernel module. If the code being test= ed +cannot be compiled as a module, in many cases the subsystem should corresp= ond to +a directory in the source tree or an entry in the ``MAINTAINERS`` file. If +unsure, follow the conventions set by tests in similar areas. =20 Test subsystems should be named after the code being tested, either after = the module (wherever possible), or after the directory or files being tested. = Test @@ -42,9 +41,8 @@ subsystems should be named to avoid ambiguity where neces= sary. =20 If a test subsystem name has multiple components, they should be separated= by underscores. *Do not* include "test" or "kunit" directly in the subsystem = name -unless you are actually testing other tests or the kunit framework itself. - -Example subsystems could be: +unless we are actually testing other tests or the kunit framework itself. = For +example, subsystems could be called: =20 ``ext4`` Matches the module and filesystem name. @@ -56,48 +54,46 @@ Example subsystems could be: Has several components (``snd``, ``hda``, ``codec``, ``hdmi``) separated= by underscores. Matches the module name. =20 -Avoid names like these: +Avoid names as shown in examples below: =20 ``linear-ranges`` Names should use underscores, not dashes, to separate words. Prefer ``linear_ranges``. ``qos-kunit-test`` - As well as using underscores, this name should not have "kunit-test" as a - suffix, and ``qos`` is ambiguous as a subsystem name. ``power_qos`` woul= d be a - better name. + This name should use underscores, and not have "kunit-test" as a + suffix. ``qos`` is also ambiguous as a subsystem name, because several p= arts + of the kernel have a ``qos`` subsystem. ``power_qos`` would be a better = name. ``pc_parallel_port`` The corresponding module name is ``parport_pc``, so this subsystem shoul= d also be named ``parport_pc``. =20 .. note:: - The KUnit API and tools do not explicitly know about subsystems. T= hey're - simply a way of categorising test suites and naming modules which - provides a simple, consistent way for humans to find and run tests= . This - may change in the future, though. + The KUnit API and tools do not explicitly know about subsystems. T= hey are + a way of categorizing test suites and naming modules which provide= s a + simple, consistent way for humans to find and run tests. This may = change + in the future. =20 Suites ------ =20 KUnit tests are grouped into test suites, which cover a specific area of -functionality being tested. Test suites can have shared initialisation and -shutdown code which is run for all tests in the suite. -Not all subsystems will need to be split into multiple test suites (e.g. s= imple drivers). +functionality being tested. Test suites can have shared initialization and +shutdown code which is run for all tests in the suite. Not all subsystems = need +to be split into multiple test suites (for example, simple drivers). =20 Test suites are named after the subsystem they are part of. If a subsystem contains several suites, the specific area under test should be appended t= o the subsystem name, separated by an underscore. =20 In the event that there are multiple types of test using KUnit within a -subsystem (e.g., both unit tests and integration tests), they should be pu= t into -separate suites, with the type of test as the last element in the suite na= me. -Unless these tests are actually present, avoid using ``_test``, ``_unittes= t`` or -similar in the suite name. +subsystem (for example, both unit tests and integration tests), they shoul= d be +put into separate suites, with the type of test as the last element in the= suite +name. Unless these tests are actually present, avoid using ``_test``, ``_u= nittest`` +or similar in the suite name. =20 The full test suite name (including the subsystem name) should be specifie= d as the ``.name`` member of the ``kunit_suite`` struct, and forms the base for= the -module name (see below). - -Example test suites could include: +module name. For example, test suites could include: =20 ``ext4_inode`` Part of the ``ext4`` subsystem, testing the ``inode`` area. @@ -109,26 +105,27 @@ Example test suites could include: The ``kasan`` subsystem has only one suite, so the suite name is the sam= e as the subsystem name. =20 -Avoid names like: +Avoid names, for example: =20 ``ext4_ext4_inode`` - There's no reason to state the subsystem twice. + There is no reason to state the subsystem twice. ``property_entry`` The suite name is ambiguous without the subsystem name. ``kasan_integration_test`` Because there is only one suite in the ``kasan`` subsystem, the suite sh= ould - just be called ``kasan``. There's no need to redundantly add - ``integration_test``. Should a separate test suite with, for example, un= it - tests be added, then that suite could be named ``kasan_unittest`` or sim= ilar. + just be called as ``kasan``. Do not redundantly add + ``integration_test``. It should be a separate test suite. For example, i= f the + unit tests are added, then that suite could be named as ``kasan_unittest= `` or + similar. =20 Test Cases ---------- =20 Individual tests consist of a single function which tests a constrained -codepath, property, or function. In the test output, individual tests' res= ults -will show up as subtests of the suite's results. +codepath, property, or function. In the test output, an individual test's +results will show up as subtests of the suite's results. =20 -Tests should be named after what they're testing. This is often the name o= f the +Tests should be named after what they are testing. This is often the name = of the function being tested, with a description of the input or codepath being t= ested. As tests are C functions, they should be named and written in accordance w= ith the kernel coding style. @@ -136,7 +133,7 @@ the kernel coding style. .. note:: As tests are themselves functions, their names cannot conflict with other C identifiers in the kernel. This may require some creative - naming. It's a good idea to make your test functions `static` to a= void + naming. It is a good idea to make your test functions `static` to = avoid polluting the global namespace. =20 Example test names include: @@ -162,16 +159,16 @@ This Kconfig entry must: * be named ``CONFIG__KUNIT_TEST``: where is the name of the t= est suite. * be listed either alongside the config entries for the driver/subsystem b= eing - tested, or be under [Kernel Hacking]=E2=86=92[Kernel Testing and Coverag= e] -* depend on ``CONFIG_KUNIT`` + tested, or be under [Kernel Hacking]->[Kernel Testing and Coverage] +* depend on ``CONFIG_KUNIT``. * be visible only if ``CONFIG_KUNIT_ALL_TESTS`` is not enabled. * have a default value of ``CONFIG_KUNIT_ALL_TESTS``. -* have a brief description of KUnit in the help text +* have a brief description of KUnit in the help text. =20 -Unless there's a specific reason not to (e.g. the test is unable to be bui= lt as -a module), Kconfig entries for tests should be tristate. +If we are not able to meet above conditions (for example, the test is unab= le to +be built as a module), Kconfig entries for tests should be tristate. =20 -An example Kconfig entry: +For example, a Kconfig entry might look like: =20 .. code-block:: none =20 @@ -182,8 +179,8 @@ An example Kconfig entry: help This builds unit tests for foo. =20 - For more information on KUnit and unit tests in general, please refer - to the KUnit documentation in Documentation/dev-tools/kunit/. + For more information on KUnit and unit tests in general, + please refer to the KUnit documentation in Documentation/dev-tools/kun= it/. =20 If unsure, say N. =20 --=20 2.34.1.173.g76aa8bc2d0-goog From nobody Wed Jul 1 18:37:12 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D9B88C433EF for ; Fri, 17 Dec 2021 04:38:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232814AbhLQEi0 (ORCPT ); Thu, 16 Dec 2021 23:38:26 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48852 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232864AbhLQEiX (ORCPT ); Thu, 16 Dec 2021 23:38:23 -0500 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A88F9C061751 for ; Thu, 16 Dec 2021 20:38:22 -0800 (PST) Received: by mail-yb1-xb4a.google.com with SMTP id w9-20020a25c709000000b00608a9489fc1so2403133ybe.20 for ; Thu, 16 Dec 2021 20:38:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=Fkg1mI+Db1oVO87JCisJGaRTIFR9sODJhCV59aBxEMg=; b=Tj3meYekaMc40xRg2gOOa2R1K6Z7UK/MBmbN3EnS9Mp8DnlQlPCjXF34eAqShGVK+P 59mFBmk9o3oOmmNPmBNQTzhZ25PLT9cI18lFD3STzbVIJz/N47hJIo5oeivLioFgYbnF 1UUn1XS4fOmuL5OxZY1YonTe7Gj4QiBXZlniqemvl9uP7PaZKRL3LzhtjO+fnK7TyDPn AZC1lRJDSlqMAivyF+hKKSBKdFc4T9NmNUTzut3dEXb9AgVJCuSp9HX8wFAMinNZbF8Y Q4bU/ilg75qhD61271BiUxZTRmUeaqkPWbFsoQuwkybvXwjlngOD02+DReLUjKOmJNMP /G0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=Fkg1mI+Db1oVO87JCisJGaRTIFR9sODJhCV59aBxEMg=; b=ALpMycWyTc7nKmItuWES5gVqGKX0QY56/2MxAYwqz6tVwEm6+EW+WFPCFFqb8zscHF lEwTZ5RmLZ1ivOzFanc/qoofLkcIb1sxvfEHw8WYDHuwhJTaT1/21GRxUSKN8QIvUAa4 V6mUhcXPfjQzPrKyS8e6Mjslj+RvL0vf0Y+8t4Q52GuLKVme8FAb1M0bByX33EONHJAb v1bqd4dtcEcyLDXSEjwhOJwFzW80x7mG5JpPkk7tPhRKInmLe6FDpEfCXqFShlcABbsg pGqKHLrd9IA6rvJKltC/imuJSKkTDhcftdHM7nQLv3ISdLnr0RNVpj94g6LV2UQ1R8aF Nniw== X-Gm-Message-State: AOAM5315rweQ56c3aGp7D4yuLXAipraLdX0tAXP3QT/RTVgjiPpo6uLW QHZjYWG1sWNwbf8RMa19SPKraZKor9nLN/o= X-Google-Smtp-Source: ABdhPJxXxK6aWyjr6j7IicczuiAbEf0DU7IIPOvC9+NBN96/LXbwnuIIzXbpJ3osiiRKmjeJvacad2NaLz3aOKk= X-Received: from sharinder.c.googlers.com ([fda3:e722:ac3:cc00:4f:4b78:c0a8:c73]) (user=sharinder job=sendgmr) by 2002:a25:acc3:: with SMTP id x3mr2292079ybd.332.1639715901479; Thu, 16 Dec 2021 20:38:21 -0800 (PST) Date: Fri, 17 Dec 2021 04:37:16 +0000 In-Reply-To: <20211217043716.794289-1-sharinder@google.com> Message-Id: <20211217043716.794289-8-sharinder@google.com> Mime-Version: 1.0 References: <20211217043716.794289-1-sharinder@google.com> X-Mailer: git-send-email 2.34.1.173.g76aa8bc2d0-goog Subject: [PATCH v5 7/7] Documentation: KUnit: Restyled Frequently Asked Questions From: Harinder Singh To: davidgow@google.com, brendanhiggins@google.com, shuah@kernel.org, corbet@lwn.net Cc: linux-kselftest@vger.kernel.org, kunit-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, tim.bird@sony.com, elver@google.com, Harinder Singh Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Reword to align with other chapters. Signed-off-by: Harinder Singh --- Documentation/dev-tools/kunit/faq.rst | 73 +++++++++++++-------------- 1 file changed, 36 insertions(+), 37 deletions(-) diff --git a/Documentation/dev-tools/kunit/faq.rst b/Documentation/dev-tool= s/kunit/faq.rst index 5c6555d020f3..172e239791a8 100644 --- a/Documentation/dev-tools/kunit/faq.rst +++ b/Documentation/dev-tools/kunit/faq.rst @@ -4,56 +4,55 @@ Frequently Asked Questions =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D =20 -How is this different from Autotest, kselftest, etc? -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D +How is this different from Autotest, kselftest, and so on? +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D KUnit is a unit testing framework. Autotest, kselftest (and some others) a= re not. =20 A `unit test `_ is supposed = to -test a single unit of code in isolation, hence the name. A unit test shoul= d be -the finest granularity of testing and as such should allow all possible co= de -paths to be tested in the code under test; this is only possible if the co= de -under test is very small and does not have any external dependencies outsi= de of +test a single unit of code in isolation and hence the name *unit test*. A = unit +test should be the finest granularity of testing and should allow all poss= ible +code paths to be tested in the code under test. This is only possible if t= he +code under test is small and does not have any external dependencies outsi= de of the test's control like hardware. =20 There are no testing frameworks currently available for the kernel that do= not -require installing the kernel on a test machine or in a VM and all require -tests to be written in userspace and run on the kernel under test; this is= true -for Autotest, kselftest, and some others, disqualifying any of them from b= eing -considered unit testing frameworks. +require installing the kernel on a test machine or in a virtual machine. A= ll +testing frameworks require tests to be written in userspace and run on the +kernel under test. This is true for Autotest, kselftest, and some others, +disqualifying any of them from being considered unit testing frameworks. =20 Does KUnit support running on architectures other than UML? =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 -Yes, well, mostly. +Yes, mostly. =20 -For the most part, the KUnit core framework (what you use to write the tes= ts) -can compile to any architecture; it compiles like just another part of the +For the most part, the KUnit core framework (what we use to write the test= s) +can compile to any architecture. It compiles like just another part of the kernel and runs when the kernel boots, or when built as a module, when the -module is loaded. However, there is some infrastructure, -like the KUnit Wrapper (``tools/testing/kunit/kunit.py``) that does not su= pport -other architectures. +module is loaded. However, there is infrastructure, like the KUnit Wrapper +(``tools/testing/kunit/kunit.py``) that does not support other architectur= es. =20 -In short, this means that, yes, you can run KUnit on other architectures, = but -it might require more work than using KUnit on UML. +In short, yes, you can run KUnit on other architectures, but it might requ= ire +more work than using KUnit on UML. =20 For more information, see :ref:`kunit-on-non-uml`. =20 -What is the difference between a unit test and these other kinds of tests? -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +What is the difference between a unit test and other kinds of tests? +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Most existing tests for the Linux kernel would be categorized as an integr= ation test, or an end-to-end test. =20 -- A unit test is supposed to test a single unit of code in isolation, henc= e the - name. A unit test should be the finest granularity of testing and as such - should allow all possible code paths to be tested in the code under test= ; this - is only possible if the code under test is very small and does not have = any - external dependencies outside of the test's control like hardware. +- A unit test is supposed to test a single unit of code in isolation. A un= it + test should be the finest granularity of testing and, as such, allows all + possible code paths to be tested in the code under test. This is only po= ssible + if the code under test is small and does not have any external dependenc= ies + outside of the test's control like hardware. - An integration test tests the interaction between a minimal set of compo= nents, usually just two or three. For example, someone might write an integrati= on test to test the interaction between a driver and a piece of hardware, o= r to test the interaction between the userspace libraries the kernel provides= and - the kernel itself; however, one of these tests would probably not test t= he + the kernel itself. However, one of these tests would probably not test t= he entire kernel along with hardware interactions and interactions with the userspace. - An end-to-end test usually tests the entire system from the perspective = of the @@ -62,26 +61,26 @@ test, or an end-to-end test. hardware with a production userspace and then trying to exercise some be= havior that depends on interactions between the hardware, the kernel, and users= pace. =20 -KUnit isn't working, what should I do? -=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D +KUnit is not working, what should I do? +=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =20 Unfortunately, there are a number of things which can break, but here are = some things to try. =20 -1. Try running ``./tools/testing/kunit/kunit.py run`` with the ``--raw_out= put`` +1. Run ``./tools/testing/kunit/kunit.py run`` with the ``--raw_output`` parameter. This might show details or error messages hidden by the kuni= t_tool parser. 2. Instead of running ``kunit.py run``, try running ``kunit.py config``, ``kunit.py build``, and ``kunit.py exec`` independently. This can help = track down where an issue is occurring. (If you think the parser is at fault,= you - can run it manually against stdin or a file with ``kunit.py parse``.) -3. Running the UML kernel directly can often reveal issues or error messag= es - kunit_tool ignores. This should be as simple as running ``./vmlinux`` a= fter - building the UML kernel (e.g., by using ``kunit.py build``). Note that = UML - has some unusual requirements (such as the host having a tmpfs filesyst= em - mounted), and has had issues in the past when built statically and the = host - has KASLR enabled. (On older host kernels, you may need to run ``setarch - `uname -m` -R ./vmlinux`` to disable KASLR.) + can run it manually against ``stdin`` or a file with ``kunit.py parse``= .) +3. Running the UML kernel directly can often reveal issues or error messag= es, + ``kunit_tool`` ignores. This should be as simple as running ``./vmlinux= `` + after building the UML kernel (for example, by using ``kunit.py build``= ). + Note that UML has some unusual requirements (such as the host having a = tmpfs + filesystem mounted), and has had issues in the past when built statical= ly and + the host has KASLR enabled. (On older host kernels, you may need to run + ``setarch `uname -m` -R ./vmlinux`` to disable KASLR.) 4. Make sure the kernel .config has ``CONFIG_KUNIT=3Dy`` and at least one = test (e.g. ``CONFIG_KUNIT_EXAMPLE_TEST=3Dy``). kunit_tool will keep its .con= fig around, so you can see what config was used after running ``kunit.py ru= n``. --=20 2.34.1.173.g76aa8bc2d0-goog