Hello,
this series is part of the larger effort aiming to convert all
standalone tests to the CI runners so that they are properly executed on
patches submission.

Some of those tests are validating bpftool behavior(test_bpftool_map.sh,
test_bpftool_metadata.sh, test_bpftool_synctypes.py, test_bpftool.py...)
and so they do not integrate well in test_progs. This series proposes to
introduce a new runner to support those tests. This new runner is
heavily inspired from test_progs, with slightly less features.

- First commit move the assert macros exposed by test_progs into a
  shared header so that they can be used by the new runner
- Second commit introduces the new runner, as well as a first test
- Third commit adds a second test
- Fourth commit drops the legacy scripts corresponding to those new
  tests

The series only adds the runner, but it is not enough to have it running
in CI. The github actions need to be updated both in [1] and [2], and I
am not sure how it si supposed to be contributed (the part needed in [1]
must likely be integrated in the commit that is systematically brought
on top of the bpf-next_base branch). Still, an example of test_bpftool
run can be found in [3], in which I have forked and updated the needed
Github actions ([4])

[1] https://github.com/kernel-patches/bpf.git
[2] https://github.com/libbpf/ci
[3] https://github.com/kernel-patches/bpf/pull/10711
[4] https://github.com/Tropicao/libbpf-ci

Signed-off-by: Alexis Lothoré (eBPF Foundation) <alexis.lothore@bootlin.com>
---
Alexis Lothoré (eBPF Foundation) (4):
      bpf/selftests: move assert macros into a dedicated header
      bpf/selftests: introduce bptool test runner and a first test
      selftests/bpf: add bpftool map manipulations tests
      selftests/bpf: remove converted bpftool test scripts

 tools/testing/selftests/bpf/.gitignore             |   1 +
 tools/testing/selftests/bpf/Makefile               |  16 +-
 tools/testing/selftests/bpf/assert_helpers.h       | 231 ++++++++++++
 tools/testing/selftests/bpf/bpftool_helpers.c      | 114 ++++++
 tools/testing/selftests/bpf/bpftool_helpers.h      |  19 +
 .../testing/selftests/bpf/bpftool_tests/.gitignore |   2 +
 .../bpf/bpftool_tests/bpftool_maps_access.c        | 370 +++++++++++++++++++
 .../selftests/bpf/bpftool_tests/bpftool_metadata.c | 128 +++++++
 tools/testing/selftests/bpf/test_bpftool.c         | 126 +++++++
 tools/testing/selftests/bpf/test_bpftool.h         |  36 ++
 tools/testing/selftests/bpf/test_bpftool_map.sh    | 398 ---------------------
 .../testing/selftests/bpf/test_bpftool_metadata.sh |  85 -----
 tools/testing/selftests/bpf/test_progs.h           | 226 +-----------
 13 files changed, 1041 insertions(+), 711 deletions(-)
---
base-commit: 807cd0dc688b0e7314f53dc3a594f247bb3b665b
change-id: 20251212-bpftool-tests-bb165c4cceb8

Best regards,
-- 
Alexis Lothoré (eBPF Foundation) <alexis.lothore@bootlin.com>

On Wed, Jan 14, 2026 at 12:59 AM Alexis Lothoré (eBPF Foundation)
<alexis.lothore@bootlin.com> wrote:
>
> Hello,
> this series is part of the larger effort aiming to convert all
> standalone tests to the CI runners so that they are properly executed on
> patches submission.
>
> Some of those tests are validating bpftool behavior(test_bpftool_map.sh,
> test_bpftool_metadata.sh, test_bpftool_synctypes.py, test_bpftool.py...)
> and so they do not integrate well in test_progs. This series proposes to

Can you elaborate why they do not integrate well? In my mind,
test_progs should be the only runner into which we invest effort
(parallel tests, all the different filtering, etc; why would we have
to reimplement subsets of this). The fact that we have test_maps and
test_verifier is historical and if we had enough time we'd merge all
of them into test_progs.

What exactly in test_progs would prevent us from implementing bpftool
test runner?

> introduce a new runner to support those tests. This new runner is
> heavily inspired from test_progs, with slightly less features.
>
> - First commit move the assert macros exposed by test_progs into a
>   shared header so that they can be used by the new runner
> - Second commit introduces the new runner, as well as a first test
> - Third commit adds a second test
> - Fourth commit drops the legacy scripts corresponding to those new
>   tests
>
> The series only adds the runner, but it is not enough to have it running
> in CI. The github actions need to be updated both in [1] and [2], and I
> am not sure how it si supposed to be contributed (the part needed in [1]
> must likely be integrated in the commit that is systematically brought
> on top of the bpf-next_base branch). Still, an example of test_bpftool
> run can be found in [3], in which I have forked and updated the needed
> Github actions ([4])
>
> [1] https://github.com/kernel-patches/bpf.git
> [2] https://github.com/libbpf/ci
> [3] https://github.com/kernel-patches/bpf/pull/10711
> [4] https://github.com/Tropicao/libbpf-ci
>
> Signed-off-by: Alexis Lothoré (eBPF Foundation) <alexis.lothore@bootlin.com>
> ---
> Alexis Lothoré (eBPF Foundation) (4):
>       bpf/selftests: move assert macros into a dedicated header
>       bpf/selftests: introduce bptool test runner and a first test
>       selftests/bpf: add bpftool map manipulations tests
>       selftests/bpf: remove converted bpftool test scripts
>
>  tools/testing/selftests/bpf/.gitignore             |   1 +
>  tools/testing/selftests/bpf/Makefile               |  16 +-
>  tools/testing/selftests/bpf/assert_helpers.h       | 231 ++++++++++++
>  tools/testing/selftests/bpf/bpftool_helpers.c      | 114 ++++++
>  tools/testing/selftests/bpf/bpftool_helpers.h      |  19 +
>  .../testing/selftests/bpf/bpftool_tests/.gitignore |   2 +
>  .../bpf/bpftool_tests/bpftool_maps_access.c        | 370 +++++++++++++++++++
>  .../selftests/bpf/bpftool_tests/bpftool_metadata.c | 128 +++++++
>  tools/testing/selftests/bpf/test_bpftool.c         | 126 +++++++
>  tools/testing/selftests/bpf/test_bpftool.h         |  36 ++
>  tools/testing/selftests/bpf/test_bpftool_map.sh    | 398 ---------------------
>  .../testing/selftests/bpf/test_bpftool_metadata.sh |  85 -----
>  tools/testing/selftests/bpf/test_progs.h           | 226 +-----------
>  13 files changed, 1041 insertions(+), 711 deletions(-)
> ---
> base-commit: 807cd0dc688b0e7314f53dc3a594f247bb3b665b
> change-id: 20251212-bpftool-tests-bb165c4cceb8
>
> Best regards,
> --
> Alexis Lothoré (eBPF Foundation) <alexis.lothore@bootlin.com>
>

Hi Andrii,

On Thu Jan 15, 2026 at 6:58 PM CET, Andrii Nakryiko wrote:
> On Wed, Jan 14, 2026 at 12:59 AM Alexis Lothoré (eBPF Foundation)
> <alexis.lothore@bootlin.com> wrote:
>>
>> Hello,
>> this series is part of the larger effort aiming to convert all
>> standalone tests to the CI runners so that they are properly executed on
>> patches submission.
>>
>> Some of those tests are validating bpftool behavior(test_bpftool_map.sh,
>> test_bpftool_metadata.sh, test_bpftool_synctypes.py, test_bpftool.py...)
>> and so they do not integrate well in test_progs. This series proposes to
>
> Can you elaborate why they do not integrate well? In my mind,
> test_progs should be the only runner into which we invest effort
> (parallel tests, all the different filtering, etc; why would we have
> to reimplement subsets of this). The fact that we have test_maps and
> test_verifier is historical and if we had enough time we'd merge all
> of them into test_progs.
>
> What exactly in test_progs would prevent us from implementing bpftool
> test runner?

I don't think there is any strong technical blocker preventing from
integrating those tests directly into test_progs. That's rather about
the fact that test_progs tests depends (almost) exclusively on
libbpf/skeletons. Those bpftool tests rather need to directly execute
bpftool and parse its stdout output, so I thought that it made sense to
have a dedicated runner for this. If I'm wrong and so if those tests
should rather be moved in the test_progs runner (eg to avoid duplicating
the runner features), I'm fine with it. Any additional opinion on this
is welcome.

Thanks,

Alexis
-- 
Alexis Lothoré, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

On Thu, Jan 15, 2026 at 11:57 PM Alexis Lothoré
<alexis.lothore@bootlin.com> wrote:
>
> Hi Andrii,
>
> On Thu Jan 15, 2026 at 6:58 PM CET, Andrii Nakryiko wrote:
> > On Wed, Jan 14, 2026 at 12:59 AM Alexis Lothoré (eBPF Foundation)
> > <alexis.lothore@bootlin.com> wrote:
> >>
> >> Hello,
> >> this series is part of the larger effort aiming to convert all
> >> standalone tests to the CI runners so that they are properly executed on
> >> patches submission.
> >>
> >> Some of those tests are validating bpftool behavior(test_bpftool_map.sh,
> >> test_bpftool_metadata.sh, test_bpftool_synctypes.py, test_bpftool.py...)
> >> and so they do not integrate well in test_progs. This series proposes to
> >
> > Can you elaborate why they do not integrate well? In my mind,
> > test_progs should be the only runner into which we invest effort
> > (parallel tests, all the different filtering, etc; why would we have
> > to reimplement subsets of this). The fact that we have test_maps and
> > test_verifier is historical and if we had enough time we'd merge all
> > of them into test_progs.
> >
> > What exactly in test_progs would prevent us from implementing bpftool
> > test runner?
>
> I don't think there is any strong technical blocker preventing from
> integrating those tests directly into test_progs. That's rather about
> the fact that test_progs tests depends (almost) exclusively on
> libbpf/skeletons. Those bpftool tests rather need to directly execute

There are actually plenty of test in test_progs that do networking
setups, calling system() to launch various binaries, etc. So it never
was purely for libbpf/skeletons/whatnot.

So yeah, I think bpftool testing should still be implemented as one
(or many) test_progs tests (and maybe subtests), utilizing
test_progs's generic multi-process testing setup, filtering,
reporting, etc infrastructure. No need to add extra runners.

> bpftool and parse its stdout output, so I thought that it made sense to
> have a dedicated runner for this. If I'm wrong and so if those tests
> should rather be moved in the test_progs runner (eg to avoid duplicating
> the runner features), I'm fine with it. Any additional opinion on this
> is welcome.
>
> Thanks,
>
> Alexis
> --
> Alexis Lothoré, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>