From nobody Mon Sep 16 19:36:56 2024 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0ECFB182DF for ; Mon, 20 May 2024 14:38:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716215884; cv=none; b=b3qhUycbjEx0uKkLTTbMvscLRVUxkpGoNXFfIek2BQoSEJSlSshjGtK+EYuyzjn/uetdj+DOdl0S/FRHQcg2wQpSFS29UYsa6v4rcYFA2WxvWXp9GCFC1+fYFQYIvipsu9X77x3JzoqCQf7YqEqtimCF5GS9/tea8C013AlhoT8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716215884; c=relaxed/simple; bh=7cO4+g901/OvGtpUux7VPcBFyjT/JLEPvy0BAA2THy4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=W2T9kJUY6CyWnmOSz6Mq8Yn6XxmLL1yi92oRFrQ7RFgp/CrreQPlzvJbKrph/cBLi+ncrQ+wJfg8o8XVqufy2T/XzTCuVD6NtWod11ma9k5arJRVmo4w16pUBSR9beQHN5h5cPQZjQgw16dyHB8pu0Q3QnE6IvMhPelft/+O5Lo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=BjakA6ya; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="BjakA6ya" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 53215C2BD11; Mon, 20 May 2024 14:38:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1716215883; bh=7cO4+g901/OvGtpUux7VPcBFyjT/JLEPvy0BAA2THy4=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=BjakA6yaXyoxAQJlLI9H6XPcSKMeFREr2/B+ClhR72h/9wc8+EQLNX51OOryZ77Yk dY94kfT2VjcEn7JWkJ3j86WgBXogLhuqP1nujdsotpF9ZG67OrgyX/46BeEkZCXg4r GfpPTUD9K1kR7aJC9YI84quFFHGCnPqwtirHjQbmyEvBZtIHCfCQUK0L6JAuiwpjlt UbViYKcCdtedvLewj8cRAgLhpzzoYfHBe5FvMzVa5IKIlUb2iyarO9fiGKXGLvjlSo wsPAWLbom+1lzFjZTqJCXfhQAcPiA4ELgjkyJSilufpvPEsSEhAXh18DRNLXt7PnUe 1CispUvHfkWzw== From: "Matthieu Baerts (NGI0)" Date: Mon, 20 May 2024 16:37:55 +0200 Subject: [PATCH mptcp-next 1/4] selftests: mptcp: lib: support flaky subtests Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20240520-selftests-mptcp-disable-flaky-v1-1-02e23ba0bc3b@kernel.org> References: <20240520-selftests-mptcp-disable-flaky-v1-0-02e23ba0bc3b@kernel.org> In-Reply-To: <20240520-selftests-mptcp-disable-flaky-v1-0-02e23ba0bc3b@kernel.org> To: mptcp@lists.linux.dev Cc: "Matthieu Baerts (NGI0)" X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3450; i=matttbe@kernel.org; h=from:subject:message-id; bh=7cO4+g901/OvGtpUux7VPcBFyjT/JLEPvy0BAA2THy4=; b=owEBbQKS/ZANAwAIAfa3gk9CaaBzAcsmYgBmS2BJ/del5qlDW3YtwdQwAmkS3Ie1W8ju+unPC J0fMqxN3QyJAjMEAAEIAB0WIQToy4X3aHcFem4n93r2t4JPQmmgcwUCZktgSQAKCRD2t4JPQmmg c40UEAC3ZW0pU1UgqoNmT2vE+F4ZKnHy/7QwOlTZX90sVwK/goDi8zlz+prBqsl2VUC2YVRu6Eu KsxW5yNGKoReUHRQDQrff+HQqOHGvSriIKglKihkTCzRhdO0ColO3q0szTuv2D6qkwBIbiLwZ9s DuOMxbAQA7LgBIQ1TLP/MImlHNMbDVTe9E0C3HGM3I66elpUYtKF4soAkzmXVxPRbOKQlf2gbRx GdukmY183f83uNQwExr/msevuThFW5mI7pfEmt83hqbQpaOiAjPtuu8w0w6Tc9qORMq8Jz1w2C9 FLR1W0HtfnJWsCOywPQtRrMkElDmBHQAK3MrIJH5pQqvW+OGMLmDsTJUYsa4bqc5UjUTJWQdAeO XnEELSNuL8lLWHg7k4DFgA34quBnLdJIzTJwGKVCxDxBIgbYX6QcR82DNKvWMNmFn1E0DDr094g Jd7gRoS+uUHkW4xgu4FKLHeT3bSJv4PChX6mEAdGYe7zK20Gatv0HNDlcngI02aH8fEqBpnu/Jh U4oaKw3mZesiy463dqtuyiZOQWPKw/G7dPLOOhcP2Jb27ee3pkugCpIJwg2J4VThtTRIvq6wRfp PvvFebuNt5CBj1cqvMX4jr20eN3PJd/yr8o3snIsf02AlDa6Ui7yzhqS/Xc2L+Zb6DDB2Oi/yRZ sL11eT0xzlwDM0g== X-Developer-Key: i=matttbe@kernel.org; a=openpgp; fpr=E8CB85F76877057A6E27F77AF6B7824F4269A073 Some subtests can be unstable, failing once every X runs. Fixing them can take time: there could be an issue in the kernel or in the subtest, and it is then important to do a proper analysis, not to hide real bugs. To avoid creating noises on the different CIs, it is important to have a simple way to mark subtests as flaky, and ignore the errors. This is what this patch introduces: subtests can be marked as flaky by setting MPTCP_LIB_SUBTEST_FLAKY env var to 1, e.g. MPTCP_LIB_SUBTEST_FLAKY=3D1 The subtest will be executed, and errors (if any) will be ignored. It is still good to run these subtests, as it exercises code, and the results can still be useful for the on-going investigations. Note that the MPTCP CI will continue to track these flaky subtests, and a ticket should be created before marking a subtest as flaky. Signed-off-by: Matthieu Baerts (NGI0) --- tools/testing/selftests/net/mptcp/mptcp_lib.sh | 38 ++++++++++++++++++++--= ---- 1 file changed, 29 insertions(+), 9 deletions(-) diff --git a/tools/testing/selftests/net/mptcp/mptcp_lib.sh b/tools/testing= /selftests/net/mptcp/mptcp_lib.sh index ad2ebda5cb64..44febbc11131 100644 --- a/tools/testing/selftests/net/mptcp/mptcp_lib.sh +++ b/tools/testing/selftests/net/mptcp/mptcp_lib.sh @@ -21,6 +21,7 @@ declare -rx MPTCP_LIB_AF_INET6=3D10 =20 MPTCP_LIB_SUBTESTS=3D() MPTCP_LIB_SUBTESTS_DUPLICATED=3D0 +MPTCP_LIB_SUBTEST_FLAKY=3D0 MPTCP_LIB_TEST_COUNTER=3D0 MPTCP_LIB_TEST_FORMAT=3D"%02u %-50s" MPTCP_LIB_IP_MPTCP=3D0 @@ -41,6 +42,18 @@ else readonly MPTCP_LIB_COLOR_RESET=3D fi =20 +# SELFTESTS_MPTCP_LIB_EXPECT_ALL_FEATURES env var can be set when validati= ng all +# features using the last version of the kernel and the selftests to make = sure +# a test is not being skipped by mistake. +mptcp_lib_expect_all_features() { + [ "${SELFTESTS_MPTCP_LIB_EXPECT_ALL_FEATURES:-}" =3D "1" ] +} + +mptcp_lib_subtest_is_flaky() { + # If all features are expected, do not threat them as flaky + [ "${MPTCP_LIB_SUBTEST_FLAKY}" =3D 1 ] && ! mptcp_lib_expect_all_features +} + # $1: color, $2: text mptcp_lib_print_color() { echo -e "${MPTCP_LIB_START_PRINT:-}${*}${MPTCP_LIB_COLOR_RESET}" @@ -72,20 +85,21 @@ mptcp_lib_pr_skip() { } =20 mptcp_lib_pr_fail() { - mptcp_lib_print_err "[FAIL]${1:+ ${*}}" + local title + + if mptcp_lib_subtest_is_flaky; then + title=3D"IGNO" + else + title=3D"FAIL" + fi + + mptcp_lib_print_err "[${title}]${1:+ ${*}}" } =20 mptcp_lib_pr_info() { mptcp_lib_print_info "INFO: ${*}" } =20 -# SELFTESTS_MPTCP_LIB_EXPECT_ALL_FEATURES env var can be set when validati= ng all -# features using the last version of the kernel and the selftests to make = sure -# a test is not being skipped by mistake. -mptcp_lib_expect_all_features() { - [ "${SELFTESTS_MPTCP_LIB_EXPECT_ALL_FEATURES:-}" =3D "1" ] -} - # $1: msg mptcp_lib_fail_if_expected_feature() { if mptcp_lib_expect_all_features; then @@ -208,7 +222,13 @@ mptcp_lib_result_pass() { =20 # $1: test name mptcp_lib_result_fail() { - __mptcp_lib_result_add "not ok" "${1}" + if mptcp_lib_subtest_is_flaky; then + # It might sound better to use 'not ok # TODO' or 'ok # SKIP', + # but some CIs don't understand 'TODO' and treat SKIP as errors. + __mptcp_lib_result_add "ok" "${1} # IGNORE" + else + __mptcp_lib_result_add "not ok" "${1}" + fi } =20 # $1: test name --=20 2.43.0 From nobody Mon Sep 16 19:36:56 2024 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5E285182DF for ; Mon, 20 May 2024 14:38:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716215885; cv=none; b=IhvyrQwnFGitB3WDgJFJMuOc42s2hgxzBGGISJzNNY6+MKseKg/EN6Niv/P6uj7KW6X54GPhwj0Ilu+OuEK/m4Hozr3LFtVjj8tdNfZFPMMrzzIap90Jrf59X22wkm5E1SLKQ7MI27wX8N7Kj9W+X/zuPam7NN4fP5K//f3xE40= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716215885; c=relaxed/simple; bh=2HxGpZwVK2rOGGZre5cdP6cDadrO8+aT7XaABZk/cEM=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=auUd2eAbTGW/RlrGZW5pTNkGMLX/t074ZP3isO9zCaOYj1zYZDaX2EncSAufzACiM1CPwjvBS8gonubI0bqqQlG3rC5fVz2i3zTY8g1oOvPXaglaI48nyQGtHhqwEU7FC8GnqzXZt4PmmWvwvJpZnQ6Or1z+sLknlQ7UEHAwETc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=RJBTRA/T; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="RJBTRA/T" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5907FC2BD10; Mon, 20 May 2024 14:38:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1716215884; bh=2HxGpZwVK2rOGGZre5cdP6cDadrO8+aT7XaABZk/cEM=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=RJBTRA/TUi4rDRDvi0FYcKRIR6ffFrGAuDMmODC7rpJ7ljlP2sEwOFlLjIhTUgHLw vjR7MLMffwimp4jbAYyluitSYgxWUBVoK8XXJfP12Xggc3N/g5lMmbgrqMo2fLPDX0 yWJD5trU6zHBL0h+jzrutFpqAOVBMREs3RtS4t1g31//EtlF0apVLjHi/M/Gd1RjI+ W0NLxWj0PAQeY684VSMyCNvhqfzkLkGLNKR4Bmz7v4Ff5fLRa1pa778Twaqbc63qe1 ZO2O4NpOOEy99HSavwc1DrVvOxoWWXvF8n8/vgvs0Hdl+RcmirvNWduc5u41EnkcC6 GTuQqhVEYI2dg== From: "Matthieu Baerts (NGI0)" Date: Mon, 20 May 2024 16:37:56 +0200 Subject: [PATCH mptcp-next 2/4] selftests: mptcp: simult flows: mark 'unbalanced' tests as flaky Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20240520-selftests-mptcp-disable-flaky-v1-2-02e23ba0bc3b@kernel.org> References: <20240520-selftests-mptcp-disable-flaky-v1-0-02e23ba0bc3b@kernel.org> In-Reply-To: <20240520-selftests-mptcp-disable-flaky-v1-0-02e23ba0bc3b@kernel.org> To: mptcp@lists.linux.dev Cc: "Matthieu Baerts (NGI0)" X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2179; i=matttbe@kernel.org; h=from:subject:message-id; bh=2HxGpZwVK2rOGGZre5cdP6cDadrO8+aT7XaABZk/cEM=; b=owEBbQKS/ZANAwAIAfa3gk9CaaBzAcsmYgBmS2BJH2QYB8sIvDRpKrIhbpl3Yqhes+Y/PW7A7 DnABxnwh8SJAjMEAAEIAB0WIQToy4X3aHcFem4n93r2t4JPQmmgcwUCZktgSQAKCRD2t4JPQmmg c3muD/9dSYKGeMs2eW9sfxl33xoVzoAUl8X0/5JpqA+1lTp3AiVspu+Ti/ptW42slxnRbve3xcX wfLSLDjMN3cjmDHr9d/bDJoLciBg1VY+j9IdGqXmg8P0TYVaARwOnzxfQtz4og8zApv7cYc/s5I 36e+QYqXGHJ7B2Kk27uxr4howln8H0u+o1qagh65aPUDtJe6N+NYLzcZOZGkpfsk1QfLSBQVlpN B7fvABJh79m3Ppk7M6SMRXu3gIvfxRpjyeLA6WzMi/62Vxqdpc6FO4BLaCAeCY6oL3PHjRFHmMf LtcUxGcKEjhYLbe/CnJqd/6nT6bOwcYr19qsXDeMUTJ8J4FsVfvfQrz6pXGbG2HFxVMWigS8KId MpRSYC2EY9NExbzXNBRwGjv8i4NOfxA5iHTxakw6IAYPQ1NP0FQ2IPZZsSpJ1jnDwaELgWH65Iw 702zbhyuxtNPPKZRMXcUQfHnqqBmkbkyRl5Hi3C0ry4vjc3eQUvI1lohAWqqstBFLICSGvhcMZ3 5aTP2xfutPyF6A9FArAK9vqTB5YZ23AkHgCeYDsZrJfWTcgZUZsb4cYASAiXMHgIuQeZVWjMWHf Z+k9S7l13trh12Ml8hP6IllugJ2O19Z1QMUKYeCSk6Xw9Bqv8DfP0EnxTLXHPvXErk3PZ3mdMjb QL7vwrLx5ZrgLRQ== X-Developer-Key: i=matttbe@kernel.org; a=openpgp; fpr=E8CB85F76877057A6E27F77AF6B7824F4269A073 These tests are flaky since their introduction. This might be less or not visible depending on the CI running the tests, especially if it is also busy doing other tasks in parallel. A first analysis shown that the transfer can be slowed down when there are some re-injections at the MPTCP level. Such re-injections can of course happen, and disturb the transfer, but it looks strange to have them in this lab. That could be caused by the kernel having access to less CPU cycles -- e.g. when other activities are executed in parallel -- or by a misinterpretation on the MPTCP packet scheduler side. While this is being investigated, the tests are marked as flaky not to create noises in other CIs. Link: https://github.com/multipath-tcp/mptcp_net-next/issues/475 Signed-off-by: Matthieu Baerts (NGI0) --- tools/testing/selftests/net/mptcp/simult_flows.sh | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/net/mptcp/simult_flows.sh b/tools/test= ing/selftests/net/mptcp/simult_flows.sh index d0b39c2e38a3..b37829e0cb07 100755 --- a/tools/testing/selftests/net/mptcp/simult_flows.sh +++ b/tools/testing/selftests/net/mptcp/simult_flows.sh @@ -244,7 +244,7 @@ run_test() do_transfer $small $large $time lret=3D$? mptcp_lib_result_code "${lret}" "${msg}" - if [ $lret -ne 0 ]; then + if [ $lret -ne 0 ] && ! mptcp_lib_subtest_is_flaky; then ret=3D$lret [ $bail -eq 0 ] || exit $ret fi @@ -254,7 +254,7 @@ run_test() do_transfer $large $small $time lret=3D$? mptcp_lib_result_code "${lret}" "${msg}" - if [ $lret -ne 0 ]; then + if [ $lret -ne 0 ] && ! mptcp_lib_subtest_is_flaky; then ret=3D$lret [ $bail -eq 0 ] || exit $ret fi @@ -290,6 +290,7 @@ run_test 10 10 0 0 "balanced bwidth" run_test 10 10 1 25 "balanced bwidth with unbalanced delay" =20 # we still need some additional infrastructure to pass the following test-= cases +MPTCP_LIB_SUBTEST_FLAKY=3D1 run_test 15 5 0 0 "unbalanced bwidth" run_test 15 5 1 25 "unbalanced bwidth with unbalanced delay" run_test 15 5 25 1 "unbalanced bwidth with opposed, unbalanced delay" --=20 2.43.0 From nobody Mon Sep 16 19:36:56 2024 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5B23B182DF for ; Mon, 20 May 2024 14:38:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716215886; cv=none; b=qDtF6fnqMUz3/Y285myPX9pGdd/6GWBBow6QY2e785/VJLzN9n/AE8ozJmOP9h8PzNte8piEqZg24AwRdpkP7AJrmbIuuzN8ux2jhV2ZTX7SPV5uuxDmvYeAlXLkdDqRHAvBQxUcTj+Y+ASIjeW24OQVEj502vXObhD7/3NmKps= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716215886; c=relaxed/simple; bh=NNwISTmCUhySHdqJqAx/XtaM/YAyCnKegpzkvXP4K8I=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=GKOxAgCWbJGaxgtlOMDDXogNELvlMbkku6LDT7OAtiC1bMxpjZ80fXPHdaDxCAk1ALzqhaSUEA8md0ojHIaf0BGJje5wHRpna4/5rnupqze9Bm6M3+MZP5VzTCvlOR4zIYdqwpWkE1zSyGQ0lxnP5JvlyC2oNgrM+WjEuUhysaI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=hJMqkHTT; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="hJMqkHTT" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5E953C4AF0C; Mon, 20 May 2024 14:38:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1716215885; bh=NNwISTmCUhySHdqJqAx/XtaM/YAyCnKegpzkvXP4K8I=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=hJMqkHTT4+qiaUkIZ2wBomlnNT2l4Fh/99QturqtrDijEWDk81awGY0DvnH02dtak PU4Essaa5aO5DdBgQwebXTW7aZSIsBmMKdDsP83KJUsfQJ8eU4sqA+8oQR8CQ3I7PJ wXee1pQlqiXIu0axCsUA/sLkXY/TguEYYbVnYVsT/VGY2JzeAltlH/V6V17Nawkhh7 DSVWpAuq9jZVCjQ4fv0TKIJ1EofbT7zuVAscONUCxVHt95zcEEsbzGsIe37llfgedt moQYGMwMe6CRk4MWfusedUSbfaOcPtQHOKmiL/onWVHlRua87drzqGht7rmM4rtfWH fiMAyUV7bW7/w== From: "Matthieu Baerts (NGI0)" Date: Mon, 20 May 2024 16:37:57 +0200 Subject: [PATCH mptcp-next 3/4] selftests: mptcp: join: mark 'fastclose' tests as flaky Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20240520-selftests-mptcp-disable-flaky-v1-3-02e23ba0bc3b@kernel.org> References: <20240520-selftests-mptcp-disable-flaky-v1-0-02e23ba0bc3b@kernel.org> In-Reply-To: <20240520-selftests-mptcp-disable-flaky-v1-0-02e23ba0bc3b@kernel.org> To: mptcp@lists.linux.dev Cc: "Matthieu Baerts (NGI0)" X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1880; i=matttbe@kernel.org; h=from:subject:message-id; bh=NNwISTmCUhySHdqJqAx/XtaM/YAyCnKegpzkvXP4K8I=; b=owEBbQKS/ZANAwAIAfa3gk9CaaBzAcsmYgBmS2BJO7F8hEBKU57xZR/FqDqIwAvDpH7VNGSo8 T/KaUCTNvmJAjMEAAEIAB0WIQToy4X3aHcFem4n93r2t4JPQmmgcwUCZktgSQAKCRD2t4JPQmmg c/bHD/9sCNXhwo6ffmFnqT7DPkn5FucWDOzob7GkSXPxyEHUKJUSh6Eb+mCDw//uWXrP+SsBR68 wKco7poWeVRpN26xwpbu/iDwfSgic0OMvgEVYs84ri+PukBnbGKbHelc6sxdf52uaWiY1H6MzSj C49Vhe+O7/Q7+GhHMWw8w4KiSFuCKcQWMS1NqeidZ0o0QmggdVEfP2nOtuIHuocu3bLAvvdUtam 54D5t7HMuA05apCBYXT10UQk7srMhP0V9gPB+Cpndz2wMgy3zz+1jsEtxah3Mne+zM7y8KHOyTV yjU0UTMyBXahvTFeNP34gNKp5nqbTY1RuRYukuLHexnQLr3sUnD5og2sm+RnEpdfpO7wfG6PSXq PkYqxqTtgOAciv7JqLb4VhMqClLf57KGaChRw4TKug93/n9TAc5/+0xX5TibLOb/Cvf5WX61Kvr whlX8QgmT7AuUzJrdXAZQObiL/Wt7PE5FYUXb2Utk0/k9f5OhJfgDMuuHUlb21vFtY34xm05Rp2 rWs74L3Gtf8RIlclbgEEzwviVMNEC4yJSrpGriu53Uj/zCaP79M6JxKBggwCbEnX6Ys5id717NV jrZb5vLvpAGmPC+63KYFV4HkfExgNVl0igYGNZtFiRGl5RI0m8YEx6azGxDudK4zl73mgo8drFL 4hjcg7ki3j9ijgg== X-Developer-Key: i=matttbe@kernel.org; a=openpgp; fpr=E8CB85F76877057A6E27F77AF6B7824F4269A073 These tests are flaky since their introduction. This might be less or not visible depending on the CI running the tests, especially if it is also busy doing other tasks in parallel, and if a debug kernel config is being used. It looks like this issue is often present with the NetDev CI. While this is being investigated, the tests are marked as flaky not to create noises on such CIs. Link: https://github.com/multipath-tcp/mptcp_net-next/issues/324 Signed-off-by: Matthieu Baerts (NGI0) --- tools/testing/selftests/net/mptcp/mptcp_join.sh | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/net/mptcp/mptcp_join.sh b/tools/testin= g/selftests/net/mptcp/mptcp_join.sh index fefa9173bdaa..b869b46823d7 100755 --- a/tools/testing/selftests/net/mptcp/mptcp_join.sh +++ b/tools/testing/selftests/net/mptcp/mptcp_join.sh @@ -261,6 +261,8 @@ reset() =20 TEST_NAME=3D"${1}" =20 + MPTCP_LIB_SUBTEST_FLAKY=3D0 # reset if modified + if skip_test; then MPTCP_LIB_TEST_COUNTER=3D$((MPTCP_LIB_TEST_COUNTER+1)) last_test_ignored=3D1 @@ -448,7 +450,9 @@ reset_with_tcp_filter() # $1: err msg fail_test() { - ret=3D${KSFT_FAIL} + if ! mptcp_lib_subtest_is_flaky; then + ret=3D${KSFT_FAIL} + fi =20 if [ ${#} -gt 0 ]; then print_fail "${@}" @@ -3069,6 +3073,7 @@ fullmesh_tests() fastclose_tests() { if reset_check_counter "fastclose test" "MPTcpExtMPFastcloseTx"; then + MPTCP_LIB_SUBTEST_FLAKY=3D1 test_linkfail=3D1024 fastclose=3Dclient \ run_tests $ns1 $ns2 10.0.1.1 chk_join_nr 0 0 0 @@ -3077,6 +3082,7 @@ fastclose_tests() fi =20 if reset_check_counter "fastclose server test" "MPTcpExtMPFastcloseRx"; t= hen + MPTCP_LIB_SUBTEST_FLAKY=3D1 test_linkfail=3D1024 fastclose=3Dserver \ run_tests $ns1 $ns2 10.0.1.1 chk_join_nr 0 0 0 0 0 0 1 --=20 2.43.0 From nobody Mon Sep 16 19:36:56 2024 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 14152182DF for ; Mon, 20 May 2024 14:38:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716215887; cv=none; b=OkzuCtsTlbuPtSV/FgtDKtzjDMSMTpoCu4PM2ncwCOy4VnPmPiY0me7fpTZamV5HXc0KFz52fmLe1laiRIjGNBLiHZxSBcp7RVBonesmIJDyLPg+Yv4jefgdLKXdWHZG6ZQ2yAdQkxtQvoAIQUMtCNvm6Qo30tCCUiG3ND6ZCQs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716215887; c=relaxed/simple; bh=8MBb3YGLk1JPv2fvJpk6cJVJlQcDE2+JIiyeeg0Lk68=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Tt1gLWaQ/qHmgZJRMSw+SMOIJTyXl3BBOrOuZ24/sBXT9LGRGbZqp3i8jhOWobPB/X9ShPypSpC3IzzZNQJ01RVEFK692rcy2QyMw6wLlaSxTQmMg9xdcc6N0jysHmf1A+8+P2jVtbuP84IRuLbWist311X68KyWdy1jeWl+zxA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=hBe6hWrL; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="hBe6hWrL" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 63338C4AF10; Mon, 20 May 2024 14:38:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1716215886; bh=8MBb3YGLk1JPv2fvJpk6cJVJlQcDE2+JIiyeeg0Lk68=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=hBe6hWrLUzUX5BhwPfrUnj755GLSfQyVqCK9/+SaOMVyjscdHf4lgZbWRbQGC/ZG8 VMt9yHTSZrXVvnCGxu9VZ/8O6W7YGiUAUCCe2PUK0XGmwm6NMFPt97yg5vQuEgl6CV gPOavMndQr1OhF/YSm6OPYny5uEGJEoOpDx4srh+SDQjcLTWQ7BvgOxfPzDd+i2xJJ QH3SZnc5N1T0iHehPQGkLnQFyNL79oBiq26lT1ACnrVBtBfajbFDG1Cj7XalvY7F9o pKNlIS4aVRkcBb1GRDwvRo66CSrFqJYoT21sijBF3MOGHj6Gd3b9tAt+igH+FJ5MSE dB2KtPmmqPfNA== From: "Matthieu Baerts (NGI0)" Date: Mon, 20 May 2024 16:37:58 +0200 Subject: [PATCH mptcp-next 4/4] selftests: mptcp: join: mark 'fail' tests as flaky Precedence: bulk X-Mailing-List: mptcp@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20240520-selftests-mptcp-disable-flaky-v1-4-02e23ba0bc3b@kernel.org> References: <20240520-selftests-mptcp-disable-flaky-v1-0-02e23ba0bc3b@kernel.org> In-Reply-To: <20240520-selftests-mptcp-disable-flaky-v1-0-02e23ba0bc3b@kernel.org> To: mptcp@lists.linux.dev Cc: "Matthieu Baerts (NGI0)" X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1372; i=matttbe@kernel.org; h=from:subject:message-id; bh=8MBb3YGLk1JPv2fvJpk6cJVJlQcDE2+JIiyeeg0Lk68=; b=owEBbQKS/ZANAwAIAfa3gk9CaaBzAcsmYgBmS2BJ64PICjLfSqZKZdM53UkdGdbzPgprxR5Q3 4Y8oTTKPsmJAjMEAAEIAB0WIQToy4X3aHcFem4n93r2t4JPQmmgcwUCZktgSQAKCRD2t4JPQmmg cyASEACDDpeWsX7Zv0C+fLnjOR+D9+6sgj3PN/tOL5Z1RoFbsfgL7Z9nMyJ2BKnrKrTtFYmxnWQ sUzDNXnu4m6UiOIzkewS5cgcZ0nj5aiowdH3WIa+VHy5mUAv6n5cNUNychmPUgCNH9FKIOTXkxY AoW3j55Vjf1sYPgPwX69PDI50J00ew+YunlFzoWmmQ35ZkfAJDG+l7XrG8fV9ZO98lqJSQEbUYf IhnxX+0CEalZJJIty5TwxZkC6bTXXakAmD5qBXfAqRQ7z7p/m2kBKeNGCNSjhwVVA9wAtPIaK4/ mAvCa1VBfVR2BwnGy4Jm6B1KUcepkrJF6DsalR6lNuu2n35SNXTyVW1g+MOLtNWGj5kRYg3p/Ne OB7O4U6tkXd8opOdj4oLPhp/oXHM7Q/Y0YjOvEdqZyUlzh5nbJP7emsTVKlqvi4a5XqTS59FrI+ Be8qyka2XyVVFspBqs375j1c2IBcYNHFnq+3wr3Ai10QMQ7rMxbg/bYQOp88NwWJv5nXLYmzh07 2XoEa+N20TvWUnlUN5KtxkQdH74dewBX6nhqVdvKapMg6+gjZEhmZQAnxm7eXpW5mSRa0EF4Bu8 hEzBNVmhJ5fvSVyNAdESQHidzSnJh5IhkzwOlA9k54PwCguORk0ZP6HwvZmkNd3jYFWUEVPYYff x72xF84NZnEwfQg== X-Developer-Key: i=matttbe@kernel.org; a=openpgp; fpr=E8CB85F76877057A6E27F77AF6B7824F4269A073 These tests are rarely unstable. It depends on the CI running the tests, especially if it is also busy doing other tasks in parallel, and if a debug kernel config is being used. It looks like this issue is sometimes present with the NetDev CI. While this is being investigated, the tests are marked as flaky not to create noises on such CIs. Link: https://github.com/multipath-tcp/mptcp_net-next/issues/491 Signed-off-by: Matthieu Baerts (NGI0) --- tools/testing/selftests/net/mptcp/mptcp_join.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tools/testing/selftests/net/mptcp/mptcp_join.sh b/tools/testin= g/selftests/net/mptcp/mptcp_join.sh index b869b46823d7..2b66c5fa71eb 100755 --- a/tools/testing/selftests/net/mptcp/mptcp_join.sh +++ b/tools/testing/selftests/net/mptcp/mptcp_join.sh @@ -3101,6 +3101,7 @@ fail_tests() { # single subflow if reset_with_fail "Infinite map" 1; then + MPTCP_LIB_SUBTEST_FLAKY=3D1 test_linkfail=3D128 \ run_tests $ns1 $ns2 10.0.1.1 chk_join_nr 0 0 0 +1 +0 1 0 1 "$(pedit_action_pkts)" @@ -3109,6 +3110,7 @@ fail_tests() =20 # multiple subflows if reset_with_fail "MP_FAIL MP_RST" 2; then + MPTCP_LIB_SUBTEST_FLAKY=3D1 tc -n $ns2 qdisc add dev ns2eth1 root netem rate 1mbit delay 5ms pm_nl_set_limits $ns1 0 1 pm_nl_set_limits $ns2 0 1 --=20 2.43.0