[v2] selftests: mptcp: mark unstable subtests as flaky

[PATCH mptcp-next v2 0/4] selftests: mptcp: mark unstable subtests as flaky

Matthieu Baerts (NGI0) posted 4 patches 8 months ago

Diff against v1 v1 v3
Download series mbox

Patches applied successfully (tree, apply log)
git fetch https://github.com/multipath-tcp/mptcp_net-next tags/patchew/20240521-selftests-mptcp-disable-flaky-v2-0-eba143bcbad0@kernel.org

There is a newer version of this series

tools/testing/selftests/net/mptcp/mptcp_join.sh   | 10 +++++-
tools/testing/selftests/net/mptcp/mptcp_lib.sh    | 38 +++++++++++++++++------
tools/testing/selftests/net/mptcp/simult_flows.sh |  6 ++--
3 files changed, 41 insertions(+), 13 deletions(-)

Expand all Fold all

[PATCH mptcp-next v2 0/4] selftests: mptcp: mark unstable subtests as flaky

Posted by Matthieu Baerts (NGI0) 8 months ago

Some subtests can be unstable, failing once every X runs. Fixing them
can take time: there could be an issue in the kernel or in the subtest,
and it is then important to do a proper analysis, not to hide real bugs.

To avoid creating noises on the different CIs, some subtests have been
marked as flaky. As a result, errors with these subtests (if any) are
ignored.

Note that the MPTCP CI will continue to track these flaky subtests. All
these unstable subtests are also tracked by our bug tracker.

Signed-off-by: Matthieu Baerts (NGI0) <matttbe@kernel.org>
---
Changes in v2:
- Patch 2/4: only mark "unbalanced bwidth" subtest as flaky (Paolo)
- Link to v1: https://lore.kernel.org/r/20240520-selftests-mptcp-disable-flaky-v1-0-02e23ba0bc3b@kernel.org

---
Matthieu Baerts (NGI0) (4):
      selftests: mptcp: lib: support flaky subtests
      selftests: mptcp: simult flows: mark 'unbalanced' tests as flaky
      selftests: mptcp: join: mark 'fastclose' tests as flaky
      selftests: mptcp: join: mark 'fail' tests as flaky

 tools/testing/selftests/net/mptcp/mptcp_join.sh   | 10 +++++-
 tools/testing/selftests/net/mptcp/mptcp_lib.sh    | 38 +++++++++++++++++------
 tools/testing/selftests/net/mptcp/simult_flows.sh |  6 ++--
 3 files changed, 41 insertions(+), 13 deletions(-)
---
base-commit: 6816c73b882ab90adec3eb18ff296b30bea570b8
change-id: 20240520-selftests-mptcp-disable-flaky-380759d6503c

Best regards,
-- 
Matthieu Baerts (NGI0) <matttbe@kernel.org>