[PATCH 00/10] drm: selftest: Convert to KUnit

Maíra Canal posted 10 patches 3 years, 10 months ago
There is a newer version of this series
drivers/gpu/drm/Kconfig                       |   20 +-
drivers/gpu/drm/Makefile                      |    2 +-
drivers/gpu/drm/selftests/Makefile            |    8 -
.../gpu/drm/selftests/drm_buddy_selftests.h   |   15 -
.../gpu/drm/selftests/drm_cmdline_selftests.h |   68 -
drivers/gpu/drm/selftests/drm_mm_selftests.h  |   28 -
.../gpu/drm/selftests/drm_modeset_selftests.h |   40 -
drivers/gpu/drm/selftests/drm_selftest.c      |  109 --
drivers/gpu/drm/selftests/drm_selftest.h      |   41 -
drivers/gpu/drm/selftests/test-drm_buddy.c    |  994 --------------
.../drm/selftests/test-drm_cmdline_parser.c   | 1141 -----------------
.../drm/selftests/test-drm_damage_helper.c    |  667 ----------
drivers/gpu/drm/selftests/test-drm_format.c   |  280 ----
.../drm/selftests/test-drm_modeset_common.c   |   32 -
.../drm/selftests/test-drm_modeset_common.h   |   52 -
drivers/gpu/drm/tests/.kunitconfig            |    3 +
drivers/gpu/drm/tests/Kconfig                 |  130 ++
drivers/gpu/drm/tests/Makefile                |   10 +
drivers/gpu/drm/tests/test-drm_buddy.c        |  748 +++++++++++
.../gpu/drm/tests/test-drm_cmdline_parser.c   |  799 ++++++++++++
.../gpu/drm/tests/test-drm_damage_helper.c    |  633 +++++++++
.../test-drm_dp_mst_helper.c                  |   82 +-
drivers/gpu/drm/tests/test-drm_format.c       |  284 ++++
.../test-drm_framebuffer.c                    |   25 +-
.../drm/{selftests => tests}/test-drm_mm.c    | 1135 +++++++---------
.../test-drm_plane_helper.c                   |  101 +-
.../drm/{selftests => tests}/test-drm_rect.c  |  124 +-
27 files changed, 3240 insertions(+), 4331 deletions(-)
delete mode 100644 drivers/gpu/drm/selftests/Makefile
delete mode 100644 drivers/gpu/drm/selftests/drm_buddy_selftests.h
delete mode 100644 drivers/gpu/drm/selftests/drm_cmdline_selftests.h
delete mode 100644 drivers/gpu/drm/selftests/drm_mm_selftests.h
delete mode 100644 drivers/gpu/drm/selftests/drm_modeset_selftests.h
delete mode 100644 drivers/gpu/drm/selftests/drm_selftest.c
delete mode 100644 drivers/gpu/drm/selftests/drm_selftest.h
delete mode 100644 drivers/gpu/drm/selftests/test-drm_buddy.c
delete mode 100644 drivers/gpu/drm/selftests/test-drm_cmdline_parser.c
delete mode 100644 drivers/gpu/drm/selftests/test-drm_damage_helper.c
delete mode 100644 drivers/gpu/drm/selftests/test-drm_format.c
delete mode 100644 drivers/gpu/drm/selftests/test-drm_modeset_common.c
delete mode 100644 drivers/gpu/drm/selftests/test-drm_modeset_common.h
create mode 100644 drivers/gpu/drm/tests/.kunitconfig
create mode 100644 drivers/gpu/drm/tests/Kconfig
create mode 100644 drivers/gpu/drm/tests/Makefile
create mode 100644 drivers/gpu/drm/tests/test-drm_buddy.c
create mode 100644 drivers/gpu/drm/tests/test-drm_cmdline_parser.c
create mode 100644 drivers/gpu/drm/tests/test-drm_damage_helper.c
rename drivers/gpu/drm/{selftests => tests}/test-drm_dp_mst_helper.c (73%)
create mode 100644 drivers/gpu/drm/tests/test-drm_format.c
rename drivers/gpu/drm/{selftests => tests}/test-drm_framebuffer.c (96%)
rename drivers/gpu/drm/{selftests => tests}/test-drm_mm.c (58%)
rename drivers/gpu/drm/{selftests => tests}/test-drm_plane_helper.c (62%)
rename drivers/gpu/drm/{selftests => tests}/test-drm_rect.c (53%)
[PATCH 00/10] drm: selftest: Convert to KUnit
Posted by Maíra Canal 3 years, 10 months ago
KUnit unifies the test structure and provides helper tools that simplify
the development of tests. The basic use case allows running tests as regular
processes, which makes it easier to run unit tests on a development machine
and to integrate the tests into a CI system.

That said, the conversion of selftests for DRM to KUnit tests is beneficial
as it unifies the testing API by using the KUnit API.

KUnit is beneficial for developers as it eases the process to run unit tests.
It is possible to run the tests by using the kunit-tool on userspace with the
following command:

./tools/testing/kunit/kunit.py run --kunitconfig=drivers/gpu/drm/tests --arch=x86_64

For CI system, it is possible to execute during the build. But, we also think
about IGT: we are developing a patch to introduce KUnit to IGT.

These patches were developed during a KUnit hackathon [0] last October. Now,
we believe that both the IGT side and the Kernel side are in good shape for
submission.

If you are willing to check the output, here is the Pastebin with the output
and execution times [1].

[0] https://groups.google.com/g/kunit-dev/c/YqFR1q2uZvk/m/IbvItSfHBAAJ
[1] https://pastebin.com/FJjLPKsC

- Arthur Grillo, Isabella Basso, and Maíra Canal

Arthur Grillo (2):
  drm: selftest: refactor drm_cmdline_parser
  drm: selftest: convert drm_mm selftest to KUnit

Maíra Canal (8):
  drm: selftest: convert drm_damage_helper selftest to KUnit
  drm: selftest: convert drm_cmdline_parser selftest to KUnit
  drm: selftest: convert drm_rect selftest to KUnit
  drm: selftest: convert drm_format selftest to KUnit
  drm: selftest: convert drm_plane_helper selftest to KUnit
  drm: selftest: convert drm_dp_mst_helper selftest to KUnit
  drm: selftest: convert drm_framebuffer selftest to KUnit
  drm: selftest: convert drm_buddy selftest to KUnit

 drivers/gpu/drm/Kconfig                       |   20 +-
 drivers/gpu/drm/Makefile                      |    2 +-
 drivers/gpu/drm/selftests/Makefile            |    8 -
 .../gpu/drm/selftests/drm_buddy_selftests.h   |   15 -
 .../gpu/drm/selftests/drm_cmdline_selftests.h |   68 -
 drivers/gpu/drm/selftests/drm_mm_selftests.h  |   28 -
 .../gpu/drm/selftests/drm_modeset_selftests.h |   40 -
 drivers/gpu/drm/selftests/drm_selftest.c      |  109 --
 drivers/gpu/drm/selftests/drm_selftest.h      |   41 -
 drivers/gpu/drm/selftests/test-drm_buddy.c    |  994 --------------
 .../drm/selftests/test-drm_cmdline_parser.c   | 1141 -----------------
 .../drm/selftests/test-drm_damage_helper.c    |  667 ----------
 drivers/gpu/drm/selftests/test-drm_format.c   |  280 ----
 .../drm/selftests/test-drm_modeset_common.c   |   32 -
 .../drm/selftests/test-drm_modeset_common.h   |   52 -
 drivers/gpu/drm/tests/.kunitconfig            |    3 +
 drivers/gpu/drm/tests/Kconfig                 |  130 ++
 drivers/gpu/drm/tests/Makefile                |   10 +
 drivers/gpu/drm/tests/test-drm_buddy.c        |  748 +++++++++++
 .../gpu/drm/tests/test-drm_cmdline_parser.c   |  799 ++++++++++++
 .../gpu/drm/tests/test-drm_damage_helper.c    |  633 +++++++++
 .../test-drm_dp_mst_helper.c                  |   82 +-
 drivers/gpu/drm/tests/test-drm_format.c       |  284 ++++
 .../test-drm_framebuffer.c                    |   25 +-
 .../drm/{selftests => tests}/test-drm_mm.c    | 1135 +++++++---------
 .../test-drm_plane_helper.c                   |  101 +-
 .../drm/{selftests => tests}/test-drm_rect.c  |  124 +-
 27 files changed, 3240 insertions(+), 4331 deletions(-)
 delete mode 100644 drivers/gpu/drm/selftests/Makefile
 delete mode 100644 drivers/gpu/drm/selftests/drm_buddy_selftests.h
 delete mode 100644 drivers/gpu/drm/selftests/drm_cmdline_selftests.h
 delete mode 100644 drivers/gpu/drm/selftests/drm_mm_selftests.h
 delete mode 100644 drivers/gpu/drm/selftests/drm_modeset_selftests.h
 delete mode 100644 drivers/gpu/drm/selftests/drm_selftest.c
 delete mode 100644 drivers/gpu/drm/selftests/drm_selftest.h
 delete mode 100644 drivers/gpu/drm/selftests/test-drm_buddy.c
 delete mode 100644 drivers/gpu/drm/selftests/test-drm_cmdline_parser.c
 delete mode 100644 drivers/gpu/drm/selftests/test-drm_damage_helper.c
 delete mode 100644 drivers/gpu/drm/selftests/test-drm_format.c
 delete mode 100644 drivers/gpu/drm/selftests/test-drm_modeset_common.c
 delete mode 100644 drivers/gpu/drm/selftests/test-drm_modeset_common.h
 create mode 100644 drivers/gpu/drm/tests/.kunitconfig
 create mode 100644 drivers/gpu/drm/tests/Kconfig
 create mode 100644 drivers/gpu/drm/tests/Makefile
 create mode 100644 drivers/gpu/drm/tests/test-drm_buddy.c
 create mode 100644 drivers/gpu/drm/tests/test-drm_cmdline_parser.c
 create mode 100644 drivers/gpu/drm/tests/test-drm_damage_helper.c
 rename drivers/gpu/drm/{selftests => tests}/test-drm_dp_mst_helper.c (73%)
 create mode 100644 drivers/gpu/drm/tests/test-drm_format.c
 rename drivers/gpu/drm/{selftests => tests}/test-drm_framebuffer.c (96%)
 rename drivers/gpu/drm/{selftests => tests}/test-drm_mm.c (58%)
 rename drivers/gpu/drm/{selftests => tests}/test-drm_plane_helper.c (62%)
 rename drivers/gpu/drm/{selftests => tests}/test-drm_rect.c (53%)

-- 
2.36.1

Re: [PATCH 00/10] drm: selftest: Convert to KUnit
Posted by David Gow 3 years, 10 months ago
On Wed, Jun 15, 2022 at 9:59 PM Maíra Canal <maira.canal@usp.br> wrote:
>
> KUnit unifies the test structure and provides helper tools that simplify
> the development of tests. The basic use case allows running tests as regular
> processes, which makes it easier to run unit tests on a development machine
> and to integrate the tests into a CI system.
>
> That said, the conversion of selftests for DRM to KUnit tests is beneficial
> as it unifies the testing API by using the KUnit API.
>
> KUnit is beneficial for developers as it eases the process to run unit tests.
> It is possible to run the tests by using the kunit-tool on userspace with the
> following command:
>
> ./tools/testing/kunit/kunit.py run --kunitconfig=drivers/gpu/drm/tests --arch=x86_64
>
> For CI system, it is possible to execute during the build. But, we also think
> about IGT: we are developing a patch to introduce KUnit to IGT.
>
> These patches were developed during a KUnit hackathon [0] last October. Now,
> we believe that both the IGT side and the Kernel side are in good shape for
> submission.
>
> If you are willing to check the output, here is the Pastebin with the output
> and execution times [1].
>
> [0] https://groups.google.com/g/kunit-dev/c/YqFR1q2uZvk/m/IbvItSfHBAAJ
> [1] https://pastebin.com/FJjLPKsC
>
> - Arthur Grillo, Isabella Basso, and Maíra Canal

Great to see these going upstream!

I've tested them on my machine, both with x86_64 qemu and with UML using:
./tools/testing/kunit/kunit.py run
--kunitconfig=drivers/gpu/drm/tests/.kunitconfig \
--kconfig_add CONFIG_UML_PCI_OVER_VIRTIO=y \
--kconfig_add CONFIG_VIRTIO_UML=y

And all 114 tests pass, and everything looks good. My only minor notes
(from a quick look at the results, rather than a detailed review of
the code) are that the test names have a few small oddities:
- The suites all end in _tests (or _test, in the case of
drm_plane_helper_test). This is a bit redundant (and while there is
only one drm_plane_helper_test, the inconsistency with the others is a
bit awkward), so removing the suffix may be cleaner. (Or at least
being optimistic, and making drm_plane_helper_test plural.)
- The drm_cmdline_parser_tests suite's tests have some inconsistencies
name-wise: they're the only ones to start with drm_, not igt_, and
they have a few capital letters in some of the
'drm_cmdline_test_force_D_' tests. (It's also technically redundant to
start all of the test names with drm_cmdline_test, given the suite
name.)

Of course, if you're trying to keep compatibility with existing tests
or tooling, or there's some deeper reason they're named like this,
it's definitely not a dealbreaker.

Either way, this whole series is:

Tested-by: David Gow <davidgow@google.com>

Cheers,
-- David
Re: [PATCH 00/10] drm: selftest: Convert to KUnit
Posted by Javier Martinez Canillas 3 years, 10 months ago
On 6/16/22 16:55, David Gow wrote:
> On Wed, Jun 15, 2022 at 9:59 PM Maíra Canal <maira.canal@usp.br> wrote:
>>
>> KUnit unifies the test structure and provides helper tools that simplify
>> the development of tests. The basic use case allows running tests as regular
>> processes, which makes it easier to run unit tests on a development machine
>> and to integrate the tests into a CI system.
>>
>> That said, the conversion of selftests for DRM to KUnit tests is beneficial
>> as it unifies the testing API by using the KUnit API.
>>
>> KUnit is beneficial for developers as it eases the process to run unit tests.
>> It is possible to run the tests by using the kunit-tool on userspace with the
>> following command:
>>
>> ./tools/testing/kunit/kunit.py run --kunitconfig=drivers/gpu/drm/tests --arch=x86_64
>>
>> For CI system, it is possible to execute during the build. But, we also think
>> about IGT: we are developing a patch to introduce KUnit to IGT.
>>
>> These patches were developed during a KUnit hackathon [0] last October. Now,
>> we believe that both the IGT side and the Kernel side are in good shape for
>> submission.
>>
>> If you are willing to check the output, here is the Pastebin with the output
>> and execution times [1].
>>
>> [0] https://groups.google.com/g/kunit-dev/c/YqFR1q2uZvk/m/IbvItSfHBAAJ
>> [1] https://pastebin.com/FJjLPKsC
>>
>> - Arthur Grillo, Isabella Basso, and Maíra Canal
> 
> Great to see these going upstream!
>

Indeed, this is pretty awesome!
 
I haven't reviewed the patches yet but just have a meta comment. There's a TODO
entry for this [0] in Documentation/gpu/todo.rst, so I think that you could add
a patch removing that as a part of this series.

[0]: https://cgit.freedesktop.org/drm/drm/tree/Documentation/gpu/todo.rst#n620

-- 
Best regards,

Javier Martinez Canillas
Linux Engineering
Red Hat