From nobody Mon Sep 16 19:40:56 2024 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EC25C12F376; Fri, 24 May 2024 16:31:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716568271; cv=none; b=A4TDHokGyWQYuTg4HlyzAznQ9vj0M+ijldFivCBLkZUkNjT5P7nHL1vI5KW83WgSm5vuNvS2wQ9+A6hTNfmjCdlMtpLa5OjtsShjKtXHgcAOji46JTQjDk11dcuTg13xbhMfq8DGc+eVXJxVg/iWfsO9Vms718dnBwAO+SpIAy8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716568271; c=relaxed/simple; bh=yPbJdqedAncEfT9IzQBXiQKofuqC24Di2UVNAhdVr8g=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=D4jSP0xnE9JEPMAqLLDv8qC/4L09Pb8U1xak0RatNQxoEN7rNxm34qREQuKSXWlNFIrifooPh+hP+p+3uiL24VXlntA7JQ4FzMcA4K+zgFVxoQTUZ0nuulsyGQGvkE+KGmjoNw2C9WA04EXiho8gbUBUCYasPdKsj5W6VzVHx7A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=sLZzraza; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="sLZzraza" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0BFBEC2BD11; Fri, 24 May 2024 16:31:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1716568270; bh=yPbJdqedAncEfT9IzQBXiQKofuqC24Di2UVNAhdVr8g=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=sLZzrazaIibQ5T5WyD9u/2TYer8rLSQfZxwV9vQKz44HLRBJnngS9/m5//VAg0bQG KNBrt9ypGtCENRomuWlJ09aYc6SukFFJNjDDWP/++Vf0XDvPdn3FOtNs6ni1MTLeWm AbqlMMED9FFoEoXsgC3QvoiK/VXVkmZYhShqhIcWQ6b3Z3kQDTf0pFJQ4wDNnie6AL z7iotGnZ9NN4UrVZH1MtD/rsBIQouVB9eT+IQ68msdngHFIOfVAilQ3p1P5Mp+GPmM jwpMk0V6a1G1Ykqm0qptrMfn81c1s0XhREjbc/n6FaBxZynMGOr3x6vmvbrE0Ztwy2 p5O/YndBgbbOQ== From: "Matthieu Baerts (NGI0)" Date: Fri, 24 May 2024 18:30:56 +0200 Subject: [PATCH net 1/4] selftests: mptcp: lib: support flaky subtests Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20240524-upstream-net-20240524-selftests-mptcp-flaky-v1-1-a352362f3f8e@kernel.org> References: <20240524-upstream-net-20240524-selftests-mptcp-flaky-v1-0-a352362f3f8e@kernel.org> In-Reply-To: <20240524-upstream-net-20240524-selftests-mptcp-flaky-v1-0-a352362f3f8e@kernel.org> To: mptcp@lists.linux.dev, Mat Martineau , Geliang Tang , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Shuah Khan Cc: netdev@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, "Matthieu Baerts (NGI0)" X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2969; i=matttbe@kernel.org; h=from:subject:message-id; bh=yPbJdqedAncEfT9IzQBXiQKofuqC24Di2UVNAhdVr8g=; b=owEBbQKS/ZANAwAIAfa3gk9CaaBzAcsmYgBmUMDItdM/SWeKpNfMnuffNtE+vb+fB1JmoEoz4 zoVGlb1VyWJAjMEAAEIAB0WIQToy4X3aHcFem4n93r2t4JPQmmgcwUCZlDAyAAKCRD2t4JPQmmg c35LD/4lqaM6MJarjFRt6PaKbiGLcgwwsS7B1M2hMZXcOlZdwzHTgdAVykgT4tAg706JjnB1RCE ZVnOp5wLspFf5Orj9nc4TZ9POGeOQSc4MLV2NMyYoD2J2F1GiNMW+DchObTWjGeFF2KwR6Ip8UC QPhTQ/3Rhn+eVAdChopjbUTs3GduncC1lZ5m0DSejvhJiGZpTjaF5HS5TdHzI4uoGSAjuFpEme9 SerOuOHJzxMKAJIoFoD4Ool3huj5TQv770zkKKAvVpc8ZxYlIexft6GuckUdcMqHSXSBRUwPYXf hUjeHqj+1c5buki7zG9trATieSl9uNZoplxoXEwm7t5fjZlANJieDPYRlUwkbnv8DdHXw7AeMiI 0XNuyShnH9CaLiML3Sgi90YJUBWzb5GJC0ORFRTeZ8BL4mGIwBN4Z15bwaRsk7lM4KMW/kX8QOm wlX5Ke7dDNvE0aPGulrLYPEBvO+uEK1aj+2G8JNMKtixsIXqW3FiKc2fvbq8cbOLe6r/hQ5iiKw 6GrqgJXSoy+t6XwqrS7dRq52loe6Dtl+HQycHKFrJwgQUvL7vBGT87zFrP2dLoD2KNlM0sqmBx+ F9CpAsuexfEe+XDhjqNf7H/g2byi95hwxhY7f/1aaCSG5PTri0XDPqaZWHabh1z0rwccc02+VgR Wb2sEsZIPPTBJ8A== X-Developer-Key: i=matttbe@kernel.org; a=openpgp; fpr=E8CB85F76877057A6E27F77AF6B7824F4269A073 Some subtests can be unstable, failing once every X runs. Fixing them can take time: there could be an issue in the kernel or in the subtest, and it is then important to do a proper analysis, not to hide real bugs. To avoid creating noises on the different CIs, it is important to have a simple way to mark subtests as flaky, and ignore the errors. This is what this patch introduces: subtests can be marked as flaky by setting MPTCP_LIB_SUBTEST_FLAKY env var to 1, e.g. MPTCP_LIB_SUBTEST_FLAKY=3D1 The subtest will be executed, and errors (if any) will be ignored. It is still good to run these subtests, as it exercises code, and the results can still be useful for the on-going investigations. Note that the MPTCP CI will continue to track these flaky subtests by setting SELFTESTS_MPTCP_LIB_OVERRIDE_FLAKY env var to 1, and a ticket has to be created before marking subtests as flaky. Reviewed-by: Mat Martineau Signed-off-by: Matthieu Baerts (NGI0) --- tools/testing/selftests/net/mptcp/mptcp_lib.sh | 30 ++++++++++++++++++++++= ++-- 1 file changed, 28 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/net/mptcp/mptcp_lib.sh b/tools/testing= /selftests/net/mptcp/mptcp_lib.sh index ad2ebda5cb64..6ffa9b7a3260 100644 --- a/tools/testing/selftests/net/mptcp/mptcp_lib.sh +++ b/tools/testing/selftests/net/mptcp/mptcp_lib.sh @@ -21,6 +21,7 @@ declare -rx MPTCP_LIB_AF_INET6=3D10 =20 MPTCP_LIB_SUBTESTS=3D() MPTCP_LIB_SUBTESTS_DUPLICATED=3D0 +MPTCP_LIB_SUBTEST_FLAKY=3D0 MPTCP_LIB_TEST_COUNTER=3D0 MPTCP_LIB_TEST_FORMAT=3D"%02u %-50s" MPTCP_LIB_IP_MPTCP=3D0 @@ -41,6 +42,16 @@ else readonly MPTCP_LIB_COLOR_RESET=3D fi =20 +# SELFTESTS_MPTCP_LIB_OVERRIDE_FLAKY env var can be set not to ignore erro= rs +# from subtests marked as flaky +mptcp_lib_override_flaky() { + [ "${SELFTESTS_MPTCP_LIB_OVERRIDE_FLAKY:-}" =3D 1 ] +} + +mptcp_lib_subtest_is_flaky() { + [ "${MPTCP_LIB_SUBTEST_FLAKY}" =3D 1 ] && ! mptcp_lib_override_flaky +} + # $1: color, $2: text mptcp_lib_print_color() { echo -e "${MPTCP_LIB_START_PRINT:-}${*}${MPTCP_LIB_COLOR_RESET}" @@ -72,7 +83,16 @@ mptcp_lib_pr_skip() { } =20 mptcp_lib_pr_fail() { - mptcp_lib_print_err "[FAIL]${1:+ ${*}}" + local title cmt + + if mptcp_lib_subtest_is_flaky; then + title=3D"IGNO" + cmt=3D" (flaky)" + else + title=3D"FAIL" + fi + + mptcp_lib_print_err "[${title}]${cmt}${1:+ ${*}}" } =20 mptcp_lib_pr_info() { @@ -208,7 +228,13 @@ mptcp_lib_result_pass() { =20 # $1: test name mptcp_lib_result_fail() { - __mptcp_lib_result_add "not ok" "${1}" + if mptcp_lib_subtest_is_flaky; then + # It might sound better to use 'not ok # TODO' or 'ok # SKIP', + # but some CIs don't understand 'TODO' and treat SKIP as errors. + __mptcp_lib_result_add "ok" "${1} # IGNORE Flaky" + else + __mptcp_lib_result_add "not ok" "${1}" + fi } =20 # $1: test name --=20 2.43.0 From nobody Mon Sep 16 19:40:56 2024 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D4AF912C7F8; Fri, 24 May 2024 16:31:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716568273; cv=none; b=tW31iCZeMvPdy41KpjUNpW2pMEBsJ3qdDH9YLshKXXPzpilJPNNw6D6nJWSfjiC9jmicTATwZ3SWPT6zgEKbJEF24/3SFCMAxEuBzJAHhV3XEdIvN+ZnCQc5j6hfJ4Hoqy5igqsJQ/iMA+CVcUVc8pEIi8UENSMS5ZaufiKjdDA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716568273; c=relaxed/simple; bh=lmxCtQTHyu0kw+aS06uj4cbDABnxfvXo+bEFeYsiE5o=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=lp/7NzfdBxJ4cNLfZ0B2GKmii0x5naNzS5J6V6qpaQScB2xZ96Y+0pYSHi2hjUqUNV4IjaUewFe7MI/MN+D6qjRHlt+YLjYYI/oVPxY98gD9C9OgBn22D2GSTR2Ffvk7P1qcKISUgjO0Qk9wOu9VpQEudSs/uFv2ZMZfzh4r4jo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ZyHC5U0l; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZyHC5U0l" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E81E1C32782; Fri, 24 May 2024 16:31:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1716568273; bh=lmxCtQTHyu0kw+aS06uj4cbDABnxfvXo+bEFeYsiE5o=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=ZyHC5U0lWULT1nZ+KyfNDGb7NCNC2RkpnvXkmv8VwGvJi7FlT4EWc/fznAZ6P4fAU SOoyvu4/mLzfwN19zSvsHKeQtEeA9HOAU6jz41H3RgcUjFq7b9k3zwXiPLEvdKAsba 14XibfJmvwFQMlWAZUolXMG21B8a6Gwo+Clw4wd7dyiLO5IJYiyHTilzy4nZg8+K5A DZkPFpaKTNj0vdMEBvJni+KCG2cMupNyFX4YeXvTk/SCwl4Xs1A4+BDcWahO6RgyEp eZ88jWvHrKIYlBxftIpa5tVAUyO8NTkeQqaprXE/GTb2dXWHDlDA1e+NOfhaTwUUj9 68ITkw0X4UpGA== From: "Matthieu Baerts (NGI0)" Date: Fri, 24 May 2024 18:30:57 +0200 Subject: [PATCH net 2/4] selftests: mptcp: simult flows: mark 'unbalanced' tests as flaky Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20240524-upstream-net-20240524-selftests-mptcp-flaky-v1-2-a352362f3f8e@kernel.org> References: <20240524-upstream-net-20240524-selftests-mptcp-flaky-v1-0-a352362f3f8e@kernel.org> In-Reply-To: <20240524-upstream-net-20240524-selftests-mptcp-flaky-v1-0-a352362f3f8e@kernel.org> To: mptcp@lists.linux.dev, Mat Martineau , Geliang Tang , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Shuah Khan Cc: netdev@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, "Matthieu Baerts (NGI0)" X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=2352; i=matttbe@kernel.org; h=from:subject:message-id; bh=lmxCtQTHyu0kw+aS06uj4cbDABnxfvXo+bEFeYsiE5o=; b=owEBbQKS/ZANAwAIAfa3gk9CaaBzAcsmYgBmUMDIGq202cDziEiiivGSTcaxZRm5rmgu8t6TG unZguP60HiJAjMEAAEIAB0WIQToy4X3aHcFem4n93r2t4JPQmmgcwUCZlDAyAAKCRD2t4JPQmmg c7cEEADkYpxFzx84a5Yut1is7xie+THY2wRF+PZBenv6fscupaep30FLcJj5jFoWD0/CvqYe3lE 2VjxHCLQUWiiuNm5apn1jYqboO2lGOtVHbDRF7bUYa7jhpBFwWor/pt29etcKQj9cTT4wbQu8H7 cjo7tIzn+w3m06UU5K27WikXKvJ/Oz/0u4mGyAupSbQzULmfw1gXj7pGyXLtDSgDQXF+uyCyV1k idBTjxJehGf87jgKmQTJv+6OUk83I0VBTgiHfCzsQxxIG5BCugj8iltDXgdl8DL3NDxKaIvjBqZ Fa2GUSUEKfP2EtTD2TGqaFsUpeK87ASpql2PyEc0ZTrdAQTy38pMKV8cJo+lTRCeFV3N2MDJGdD BD13MM+QoDMmHgSLWlKkhqb0+CPwTWK/SIc/gTZmCkWc586D441mT+7p8fMo6+l8dHAf6GUvPYm gHrbbFqYC8P/UjVOdRd1O7u6N+mOqR4+z9Yvj5j5PlUBeOcQsfUXF+Dy+YsWdz5ytQviArwnuzq LuV/I6rcE9vZSlQJhRe03RbMrNJPHig4YIDUrGjhh6jQxBPDfLWaMklH3iezr9y0lt9RQVeYERr FTWqCjrtsLw8XUzSQsXa6786L0q/5CLkEVVDKYQeev5U4FkUsK0uXlJC5m6JYdwoovEn7Ehqddz qHTXTGs1rN8Mtvw== X-Developer-Key: i=matttbe@kernel.org; a=openpgp; fpr=E8CB85F76877057A6E27F77AF6B7824F4269A073 These tests are flaky since their introduction. This might be less or not visible depending on the CI running the tests, especially if it is also busy doing other tasks in parallel. A first analysis shown that the transfer can be slowed down when there are some re-injections at the MPTCP level. Such re-injections can of course happen, and disturb the transfer, but it looks strange to have them in this lab. That could be caused by the kernel having access to less CPU cycles -- e.g. when other activities are executed in parallel -- or by a misinterpretation on the MPTCP packet scheduler side. While this is being investigated, the tests are marked as flaky not to create noises in other CIs. Fixes: 219d04992b68 ("mptcp: push pending frames when subflow has free spac= e") Link: https://github.com/multipath-tcp/mptcp_net-next/issues/475 Reviewed-by: Mat Martineau Signed-off-by: Matthieu Baerts (NGI0) --- tools/testing/selftests/net/mptcp/simult_flows.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tools/testing/selftests/net/mptcp/simult_flows.sh b/tools/test= ing/selftests/net/mptcp/simult_flows.sh index 4b14b4412166..f74e1c3c126d 100755 --- a/tools/testing/selftests/net/mptcp/simult_flows.sh +++ b/tools/testing/selftests/net/mptcp/simult_flows.sh @@ -244,7 +244,7 @@ run_test() do_transfer $small $large $time lret=3D$? mptcp_lib_result_code "${lret}" "${msg}" - if [ $lret -ne 0 ]; then + if [ $lret -ne 0 ] && ! mptcp_lib_subtest_is_flaky; then ret=3D$lret [ $bail -eq 0 ] || exit $ret fi @@ -254,7 +254,7 @@ run_test() do_transfer $large $small $time lret=3D$? mptcp_lib_result_code "${lret}" "${msg}" - if [ $lret -ne 0 ]; then + if [ $lret -ne 0 ] && ! mptcp_lib_subtest_is_flaky; then ret=3D$lret [ $bail -eq 0 ] || exit $ret fi @@ -290,7 +290,7 @@ run_test 10 10 0 0 "balanced bwidth" run_test 10 10 1 25 "balanced bwidth with unbalanced delay" =20 # we still need some additional infrastructure to pass the following test-= cases -run_test 10 3 0 0 "unbalanced bwidth" +MPTCP_LIB_SUBTEST_FLAKY=3D1 run_test 10 3 0 0 "unbalanced bwidth" run_test 10 3 1 25 "unbalanced bwidth with unbalanced delay" run_test 10 3 25 1 "unbalanced bwidth with opposed, unbalanced delay" =20 --=20 2.43.0 From nobody Mon Sep 16 19:40:56 2024 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A6D8E130A42; Fri, 24 May 2024 16:31:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716568276; cv=none; b=sJUNqiY1MBtFvCqTPlweRQxPYvlKOK/jx045Crpt6G9+xGIFpwuD48aLKBSir5/q4DBMHkhFU6FSLf8uj58WkR+dJqkEQPbVuBJaWIiKKK44AdU7fTun4xZ5VDjiUNfS0Ggz3QuYCg9agd4j4gEYWF/w0EgtPKX6NaKmqRjehd8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716568276; c=relaxed/simple; bh=LBOws/Zh/7AplUcMrmYpeecU94L6qJWYzHbTqUIchzI=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=LgmXWQUyT9bm7Othu1ncLA1PxBzEHvPRHVUarLZauOfEDyGIB4Ket0CZ2xZ/m0TxJkXZFg7rD5RZC69uKjSqcLh1Yag2dPWBzJGHzLF8N+q0FokyqmMnTVjfi5HHCzIn5lSexfXlhU06h/oojb0wl9ztp0a8r9rVj4qUv1tV+xc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ZWwk141V; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZWwk141V" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D3F20C2BBFC; Fri, 24 May 2024 16:31:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1716568276; bh=LBOws/Zh/7AplUcMrmYpeecU94L6qJWYzHbTqUIchzI=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=ZWwk141VvoP4mem2p1m5PNaygHVLUojwVLLcFsrdrXrOgC3JaKyRKYQ2Jy18RVSYk +Kc+MGL4mJB5KltwU3wnhFaHNEo87pbLi836KArX8fc+BZy0A/vgls4kUtwfsE0ZAL LyrQ4ooTH8do2Et+QdZMbW2LanAApThC+rjx6tkrRMK4fxWp4HA+JV167vw1/zjGlq OgMX/7LkAV8DfbR6SD55HGKG8zYk7A7IMA3xBekDsIGFjQV6nX2rwVybasXBjuxZnI MVOR9gW+i2Eal0LGU20R6/RONLBPdj28lHcdf01kAta8Ph57tJ3EA+96uF9jYzRm+V 64c50GcsTs05Q== From: "Matthieu Baerts (NGI0)" Date: Fri, 24 May 2024 18:30:58 +0200 Subject: [PATCH net 3/4] selftests: mptcp: join: mark 'fastclose' tests as flaky Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20240524-upstream-net-20240524-selftests-mptcp-flaky-v1-3-a352362f3f8e@kernel.org> References: <20240524-upstream-net-20240524-selftests-mptcp-flaky-v1-0-a352362f3f8e@kernel.org> In-Reply-To: <20240524-upstream-net-20240524-selftests-mptcp-flaky-v1-0-a352362f3f8e@kernel.org> To: mptcp@lists.linux.dev, Mat Martineau , Geliang Tang , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Shuah Khan Cc: netdev@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, "Matthieu Baerts (NGI0)" X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1997; i=matttbe@kernel.org; h=from:subject:message-id; bh=LBOws/Zh/7AplUcMrmYpeecU94L6qJWYzHbTqUIchzI=; b=owEBbQKS/ZANAwAIAfa3gk9CaaBzAcsmYgBmUMDI70nhBvwho5Lq8I0fOfOneqcQCtLhPxk5t VYLXdAX0xWJAjMEAAEIAB0WIQToy4X3aHcFem4n93r2t4JPQmmgcwUCZlDAyAAKCRD2t4JPQmmg c2ImEADY6EfqvQRqsMbHdFV+ubnAHc+fDGTjNTbDodMoJ/rZfy6RiALAlF6v4/U8ZZ/eIDOpFuq xeKFBT2Jkqh0AUo7K1rhINhh/A50gZ7REC0VCEiTORVVJaK9Hf72ZWE9wqUh+jDSxC544SE7tKb H9yZKNe5J0rs0NA2ijxAzolYT41aspy27mfvwV+Fa+U7lvFBqfzoXpOro44fTSRHz3PY+6IDl+z cEPDcvtfroMwdmC6+/ukEpwsBWfr4/aTMHbY8utKnurO25+jLa2nqYs95jQNV2G9rFO5mQwYL9l LI4vU4BenOhWB8LUTpu55K5QyAFJOyLS9tKh6ldHjCimoAxx83mpQBuLxD4UGiOicEJ8Ke+c9uB O2WttQmJuOP511urjYXzCYg+S4hwVfxW741Qib9gZU4NjpD+3njwTMI/CUUdK3Enxi1QJOmqYjz QStGghQ/HY3IwiP62Chs7+OgOzupOmZ8phSxlCX6z0aKXyxEl0gPIi7elt+2JZJ0EUf60IPpTs3 WkP0DRrxBxXav9w2w0ViOnnprjwGinPDd4oMVGRsTiHOGEcB8Sr+rmuEUUmCT+e2yb3Sm/83QHz b5YBZ/hVVgDogxLLyDS2WxG8TEn7JF2AM7aB+N2Mxf2Gd0n4MA7L3E2rBrzsv0UoaEwV2qyEic7 jkltEFTiup1anCw== X-Developer-Key: i=matttbe@kernel.org; a=openpgp; fpr=E8CB85F76877057A6E27F77AF6B7824F4269A073 These tests are flaky since their introduction. This might be less or not visible depending on the CI running the tests, especially if it is also busy doing other tasks in parallel, and if a debug kernel config is being used. It looks like this issue is often present with the NetDev CI. While this is being investigated, the tests are marked as flaky not to create noises on such CIs. Fixes: 01542c9bf9ab ("selftests: mptcp: add fastclose testcase") Link: https://github.com/multipath-tcp/mptcp_net-next/issues/324 Reviewed-by: Mat Martineau Signed-off-by: Matthieu Baerts (NGI0) --- tools/testing/selftests/net/mptcp/mptcp_join.sh | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/tools/testing/selftests/net/mptcp/mptcp_join.sh b/tools/testin= g/selftests/net/mptcp/mptcp_join.sh index fefa9173bdaa..b869b46823d7 100755 --- a/tools/testing/selftests/net/mptcp/mptcp_join.sh +++ b/tools/testing/selftests/net/mptcp/mptcp_join.sh @@ -261,6 +261,8 @@ reset() =20 TEST_NAME=3D"${1}" =20 + MPTCP_LIB_SUBTEST_FLAKY=3D0 # reset if modified + if skip_test; then MPTCP_LIB_TEST_COUNTER=3D$((MPTCP_LIB_TEST_COUNTER+1)) last_test_ignored=3D1 @@ -448,7 +450,9 @@ reset_with_tcp_filter() # $1: err msg fail_test() { - ret=3D${KSFT_FAIL} + if ! mptcp_lib_subtest_is_flaky; then + ret=3D${KSFT_FAIL} + fi =20 if [ ${#} -gt 0 ]; then print_fail "${@}" @@ -3069,6 +3073,7 @@ fullmesh_tests() fastclose_tests() { if reset_check_counter "fastclose test" "MPTcpExtMPFastcloseTx"; then + MPTCP_LIB_SUBTEST_FLAKY=3D1 test_linkfail=3D1024 fastclose=3Dclient \ run_tests $ns1 $ns2 10.0.1.1 chk_join_nr 0 0 0 @@ -3077,6 +3082,7 @@ fastclose_tests() fi =20 if reset_check_counter "fastclose server test" "MPTcpExtMPFastcloseRx"; t= hen + MPTCP_LIB_SUBTEST_FLAKY=3D1 test_linkfail=3D1024 fastclose=3Dserver \ run_tests $ns1 $ns2 10.0.1.1 chk_join_nr 0 0 0 0 0 0 1 --=20 2.43.0 From nobody Mon Sep 16 19:40:56 2024 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9C12D130A7A; Fri, 24 May 2024 16:31:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716568279; cv=none; b=WhU0lm9s8OazYKj+xX9OmRaYLY/TJhLeVRUc2FTNAaPTL+9G5wlf6xiOrX7sxLrhwxHiXiGGFY6lirCouhhSZFan24ryUl6b9IhwaYqJBxsXnJF19HrOohMedED0dStXIC95pmjj5oC5QH3doOjJTwosFyrvl3AJP6dXq6i/pYY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716568279; c=relaxed/simple; bh=Qqr2iBCRG6XI9ufgkFxNFbP4uEkYYEK3OUdo+52EU5k=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=sggQlA9YpatIh8I+UU6jnMe2lf3X82v5MEvOaihX9cMWo2Dkp2Et5rcm/fZNBtEpgATtkhnl4RMqf6hYvyq16/FTiSoDRAsxWCMQC9o35BWIALA7pwBGoA1QB3v09pZpZtwOJ6qJQBnQHrY4/0UBoW0FKVhessMYW/p2zfVYybE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ZkLHv8Pk; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZkLHv8Pk" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BCF83C32782; Fri, 24 May 2024 16:31:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1716568279; bh=Qqr2iBCRG6XI9ufgkFxNFbP4uEkYYEK3OUdo+52EU5k=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=ZkLHv8Pkxn9mUY1HnezEPGJrpbRVXwMsJKsLmhps7nNgbLnZlHPJ2COZpcYc9DmOy My3NvyHfjvsXK7AIjTX0QaRzlVeMebK7oKI/+Cjx2imHnCJSN8rzalz/+RbJ2m708v COTEHrmlM0UVpnXInU7aqauuglaUhUHHdT+evF5awhF82hHpNu83hCrsqE0gFIRLqX N3/WsfxQaDZTs0nTsVtpWx/7IcsKZQ/R0m0rYfLoMG6/oQ5I/I674IxDIjoXLIAFSw pTnJ2J21xPQp2eRmyqx0JYzVxSZxSYFZVryxUL0IGs8IySkScXLuZJyTcg1TYFXn+O eU81odHgpO5JQ== From: "Matthieu Baerts (NGI0)" Date: Fri, 24 May 2024 18:30:59 +0200 Subject: [PATCH net 4/4] selftests: mptcp: join: mark 'fail' tests as flaky Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20240524-upstream-net-20240524-selftests-mptcp-flaky-v1-4-a352362f3f8e@kernel.org> References: <20240524-upstream-net-20240524-selftests-mptcp-flaky-v1-0-a352362f3f8e@kernel.org> In-Reply-To: <20240524-upstream-net-20240524-selftests-mptcp-flaky-v1-0-a352362f3f8e@kernel.org> To: mptcp@lists.linux.dev, Mat Martineau , Geliang Tang , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Shuah Khan Cc: netdev@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, "Matthieu Baerts (NGI0)" X-Mailer: b4 0.13.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=1492; i=matttbe@kernel.org; h=from:subject:message-id; bh=Qqr2iBCRG6XI9ufgkFxNFbP4uEkYYEK3OUdo+52EU5k=; b=owEBbQKS/ZANAwAIAfa3gk9CaaBzAcsmYgBmUMDIBXYVVB2frHAK+KWc1s+dBlNhOBQI503dG bU6GUd31Z2JAjMEAAEIAB0WIQToy4X3aHcFem4n93r2t4JPQmmgcwUCZlDAyAAKCRD2t4JPQmmg c8PxEACbtIqm+60nrixYuPj6dYq4NRlkY8bPpYQrEXeHdCrQ6Vk2slfEFCEnDvou0cJ1usAFtWq MGeyoOvTeECoRrkZagFQEoMqrnFu+HeAR28qMbifCBuQCYhZS52vb8tD/xL8SiRrkN/jrDNg0JM bXQc5gUH04tNcfOzfzt9Vu5AeinUN38pmcgb7ajIMnBVTuh0Hpz15imOX9yAWThfDeJ4q9zJBh1 CSDQNLAoEdxQdpQogCgFWKApNu5t+E9PzSXVkhfJrs25+Ls3fOzyinVa4gFx0YHZHOQW4UWYJIb YBd+CO9Yyv/k29CRo3jZIzBk8egSrXYFvbvW82Fr1lSoLOqtVZS78jsViYS6xuKCPyyBYc9pqnm pfz9q7XEb1bZzebJyPaf3NNJi8zR0MntQFOFNRvDWWeP6pjAlNFeIG3G0HB2rS0UvHezsdMKvut aD2cy1UUKz4Kcdy2NpMEceslq+rPMNrWiq59FR/3kBODVhtIpZ8bOaQDYgyrJ6j8JG0bh87Jq+a 9AaQeZX4XC8lcxh/rcRVsnnJO91g9YTiGKiGHSt/4CZWkLVMRY0Ny5eDPKGdouLv1PKJrpiWO3w 9AN6s3koBl/TMVHqERKd4t5btpFremb0YjLE5hB+zq+IK20uEKymobrYKN7rl0bTGqkc2ozkMTd AfWPlHr+CsBemXQ== X-Developer-Key: i=matttbe@kernel.org; a=openpgp; fpr=E8CB85F76877057A6E27F77AF6B7824F4269A073 These tests are rarely unstable. It depends on the CI running the tests, especially if it is also busy doing other tasks in parallel, and if a debug kernel config is being used. It looks like this issue is sometimes present with the NetDev CI. While this is being investigated, the tests are marked as flaky not to create noises on such CIs. Fixes: b6e074e171bc ("selftests: mptcp: add infinite map testcase") Link: https://github.com/multipath-tcp/mptcp_net-next/issues/491 Reviewed-by: Mat Martineau Signed-off-by: Matthieu Baerts (NGI0) --- tools/testing/selftests/net/mptcp/mptcp_join.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tools/testing/selftests/net/mptcp/mptcp_join.sh b/tools/testin= g/selftests/net/mptcp/mptcp_join.sh index b869b46823d7..2b66c5fa71eb 100755 --- a/tools/testing/selftests/net/mptcp/mptcp_join.sh +++ b/tools/testing/selftests/net/mptcp/mptcp_join.sh @@ -3101,6 +3101,7 @@ fail_tests() { # single subflow if reset_with_fail "Infinite map" 1; then + MPTCP_LIB_SUBTEST_FLAKY=3D1 test_linkfail=3D128 \ run_tests $ns1 $ns2 10.0.1.1 chk_join_nr 0 0 0 +1 +0 1 0 1 "$(pedit_action_pkts)" @@ -3109,6 +3110,7 @@ fail_tests() =20 # multiple subflows if reset_with_fail "MP_FAIL MP_RST" 2; then + MPTCP_LIB_SUBTEST_FLAKY=3D1 tc -n $ns2 qdisc add dev ns2eth1 root netem rate 1mbit delay 5ms pm_nl_set_limits $ns1 0 1 pm_nl_set_limits $ns2 0 1 --=20 2.43.0