[PATCH mptcp-next 0/7] selftests: mptcp: improve results in slow CI envs

Matthieu Baerts (NGI0) posted 7 patches 3 months, 1 week ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/multipath-tcp/mptcp_net-next tags/patchew/20240118-selftests-mptcp-simult-slack-qemu-no-kvm-v1-0-f3a4db41c3be@kernel.org
tools/testing/selftests/net/mptcp/config          |  3 +++
tools/testing/selftests/net/mptcp/mptcp_join.sh   | 27 +++++++++--------------
tools/testing/selftests/net/mptcp/mptcp_lib.sh    |  2 +-
tools/testing/selftests/net/mptcp/settings        |  2 +-
tools/testing/selftests/net/mptcp/simult_flows.sh |  8 +++----
5 files changed, 20 insertions(+), 22 deletions(-)
[PATCH mptcp-next 0/7] selftests: mptcp: improve results in slow CI envs
Posted by Matthieu Baerts (NGI0) 3 months, 1 week ago
The initial goal of this series is to improve the results when slow test
environments, e.g. when using QEmu without KVM.

I looked at the results from LKFT [1] and noticed that some mptcp_join
subtests were skipped because it was not possible to add IPTables rules.
It looks like some KConfig were missing. The 3 first patches should fix
that.

I also noticed that mptcp_join selftest stopped after a timeout of 20
minutes. We keep adding tests there, it is normal to increase this
timeout at some points, especially because we forced many subtests to
have a longer connections, to have enough time for some actions to
happen. Patch 4 fixes that.

Patch 5 decreases the BW in simult flows, hopefully reducing the load on
the CPU, and improving the stability.

Patch 6 is needed because our CI run MPTCP Connect twice: the second
time with '-m mmap'. When parsing the results, the tests from the two
runs have the same title, hence the same "fingerprint". Instead of
having a post-processing step modifying the results for this specific
case, it is now possible to set a prefix.

When looking at the results of mptcp_join, I also realised that the
'userspace' tests were blocking after having displayed the results.
That's because they wait for the end of the transfer, while they don't
need to: tests are the end are ignored because this is done in the
background (but that's fine, other tests check that). Patch 7 improves
the situation there.

Link: https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20240108/testrun/21966973/suite/kselftest-net-mptcp/tests/ [1]
Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
---
Matthieu Baerts (NGI0) (7):
      selftests: mptcp: add missing kconfig for NF Filter
      selftests: mptcp: add missing kconfig for NF Filter in v6
      selftests: mptcp: add missing kconfig for NF Mangle
      selftests: mptcp: increase timeout to 30 min
      selftests: mptcp: decrease BW in simult flows
      selftests: mptcp: allow changing subtests prefix
      selftests: mptcp: join: stop transfer when check is done

 tools/testing/selftests/net/mptcp/config          |  3 +++
 tools/testing/selftests/net/mptcp/mptcp_join.sh   | 27 +++++++++--------------
 tools/testing/selftests/net/mptcp/mptcp_lib.sh    |  2 +-
 tools/testing/selftests/net/mptcp/settings        |  2 +-
 tools/testing/selftests/net/mptcp/simult_flows.sh |  8 +++----
 5 files changed, 20 insertions(+), 22 deletions(-)
---
base-commit: 387cadd951df0629cc1098546acb3da13e8f6ebf
change-id: 20240110-selftests-mptcp-simult-slack-qemu-no-kvm-da67d9055c04

Best regards,
-- 
Matthieu Baerts (NGI0) <matttbe@kernel.org>
Re: [PATCH mptcp-next 0/7] selftests: mptcp: improve results in slow CI envs
Posted by Matthieu Baerts 2 months, 4 weeks ago
Hi Geliang, Paolo,

On 18/01/2024 19:27, Matthieu Baerts (NGI0) wrote:
> The initial goal of this series is to improve the results when slow test
> environments, e.g. when using QEmu without KVM.
> 
> I looked at the results from LKFT [1] and noticed that some mptcp_join
> subtests were skipped because it was not possible to add IPTables rules.
> It looks like some KConfig were missing. The 3 first patches should fix
> that.
> 
> I also noticed that mptcp_join selftest stopped after a timeout of 20
> minutes. We keep adding tests there, it is normal to increase this
> timeout at some points, especially because we forced many subtests to
> have a longer connections, to have enough time for some actions to
> happen. Patch 4 fixes that.
> 
> Patch 5 decreases the BW in simult flows, hopefully reducing the load on
> the CPU, and improving the stability.
> 
> Patch 6 is needed because our CI run MPTCP Connect twice: the second
> time with '-m mmap'. When parsing the results, the tests from the two
> runs have the same title, hence the same "fingerprint". Instead of
> having a post-processing step modifying the results for this specific
> case, it is now possible to set a prefix.
> 
> When looking at the results of mptcp_join, I also realised that the
> 'userspace' tests were blocking after having displayed the results.
> That's because they wait for the end of the transfer, while they don't
> need to: tests are the end are ignored because this is done in the
> background (but that's fine, other tests check that). Patch 7 improves
> the situation there.

Thank you for the reviews!

Now in our tree (fix for -net) with the modification demanded by Geliang
on patch 3/7:

New patches for t/upstream-net and t/upstream:
- a773245de129: selftests: mptcp: add missing kconfig for NF Filter
- 616af1a08349: selftests: mptcp: add missing kconfig for NF Filter in v6
- 2d4ccf16a1ac: selftests: mptcp: add missing kconfig for NF Mangle
- ac8595b0b31c: selftests: mptcp: increase timeout to 30 min
- ba7e09d80ffe: selftests: mptcp: decrease BW in simult flows
- e8a61a14a58f: selftests: mptcp: allow changing subtests prefix
- eb5f4f5f0a80: selftests: mptcp: join: stop transfer when check is done
- Results: 6d56aa24e733..b4671e59d316 (export-net)
- Results: cdd4fe00f4c5..81c0f68b0316 (export)

Tests are now in progress:

https://cirrus-ci.com/github/multipath-tcp/mptcp_net-next/export-net/20240130T121223
https://cirrus-ci.com/github/multipath-tcp/mptcp_net-next/export/20240130T121223

Cheers,
Matt
-- 
Sponsored by the NGI0 Core fund.