[PATCH mptcp-net 00/13] selftests: mptcp: skip tests when features are not supported

Matthieu Baerts posted 13 patches 1 year, 1 month ago
Failed in applying to current master (apply log)
Maintainers: Matthieu Baerts <matthieu.baerts@tessares.net>, "David S. Miller" <davem@davemloft.net>, Eric Dumazet <edumazet@google.com>, Jakub Kicinski <kuba@kernel.org>, Paolo Abeni <pabeni@redhat.com>, Shuah Khan <shuah@kernel.org>, Florian Westphal <fw@strlen.de>, Christoph Paasch <cpaasch@apple.com>, Mat Martineau <mathew.j.martineau@linux.intel.com>, Kishen Maloor <kishen.maloor@intel.com>, Dmytro Shytyi <dmytro@shytyi.net>, Geliang Tang <geliang.tang@suse.com>
There is a newer version of this series
tools/testing/selftests/net/mptcp/Makefile         |  2 +-
tools/testing/selftests/net/mptcp/config           |  1 +
tools/testing/selftests/net/mptcp/diag.sh          |  4 +++
tools/testing/selftests/net/mptcp/mptcp_connect.sh | 10 +++++++
tools/testing/selftests/net/mptcp/mptcp_join.sh    | 18 ++++++++++--
tools/testing/selftests/net/mptcp/mptcp_lib.sh     | 34 ++++++++++++++++++++++
tools/testing/selftests/net/mptcp/mptcp_sockopt.sh |  4 +++
tools/testing/selftests/net/mptcp/pm_netlink.sh    |  4 +++
tools/testing/selftests/net/mptcp/simult_flows.sh  |  4 +++
tools/testing/selftests/net/mptcp/userspace_pm.sh  | 17 ++++++++++-
10 files changed, 94 insertions(+), 4 deletions(-)
[PATCH mptcp-net 00/13] selftests: mptcp: skip tests when features are not supported
Posted by Matthieu Baerts 1 year, 1 month ago
Selftests are supposed to run on any kernels, including the old ones not
supporting all MPTCP features.

These different patches add support for older kernels. More similar
checks will be needed but here is a first batch not to have too many
patches (because each patch fixes different commits).

I tried to minimise the modifications and do the checks from the bash
script not to have to detect the feature when being used in a .c file,
then pass the info to the bash script, skip the test, clean the
environment, etc.

Link: https://github.com/multipath-tcp/mptcp_net-next/issues/368
Signed-off-by: Matthieu Baerts <matthieu.baerts@tessares.net>
---
Matthieu Baerts (13):
      selftests: mptcp: connect: skip if MPTCP is not supported
      selftests: mptcp: pm nl: skip if MPTCP is not supported
      selftests: mptcp: join: skip if MPTCP is not supported
      selftests: mptcp: diag: skip if MPTCP is not supported
      selftests: mptcp: simult flows: skip if MPTCP is not supported
      selftests: mptcp: sockopt: skip if MPTCP is not supported
      selftests: mptcp: userspace pm: skip if MPTCP is not supported
      selftests: mptcp: lib: skip if missing symbol
      selftests: mptcp: connect: skip TFO tests if not supported
      selftests: mptcp: userspace pm: skip if 'ip' tool is unavailable
      selftests: mptcp: userspace pm: skip if not supported
      selftests: mptcp: userspace pm: skip PM listener events tests if unavailable
      selftests: mptcp: join: skip PM listener events tests if unavailable

 tools/testing/selftests/net/mptcp/Makefile         |  2 +-
 tools/testing/selftests/net/mptcp/config           |  1 +
 tools/testing/selftests/net/mptcp/diag.sh          |  4 +++
 tools/testing/selftests/net/mptcp/mptcp_connect.sh | 10 +++++++
 tools/testing/selftests/net/mptcp/mptcp_join.sh    | 18 ++++++++++--
 tools/testing/selftests/net/mptcp/mptcp_lib.sh     | 34 ++++++++++++++++++++++
 tools/testing/selftests/net/mptcp/mptcp_sockopt.sh |  4 +++
 tools/testing/selftests/net/mptcp/pm_netlink.sh    |  4 +++
 tools/testing/selftests/net/mptcp/simult_flows.sh  |  4 +++
 tools/testing/selftests/net/mptcp/userspace_pm.sh  | 17 ++++++++++-
 10 files changed, 94 insertions(+), 4 deletions(-)
---
base-commit: 016e68a1441264585c15defedc5ccf883b3b4211
change-id: 20230406-mptcp-issue-368-selftests-old-kernels-fffd69d67d91

Best regards,
-- 
Matthieu Baerts <matthieu.baerts@tessares.net>
Re: [PATCH mptcp-net 00/13] selftests: mptcp: skip tests when features are not supported
Posted by Mat Martineau 1 year ago
On Fri, 7 Apr 2023, Matthieu Baerts wrote:

> Selftests are supposed to run on any kernels, including the old ones not
> supporting all MPTCP features.
>
> These different patches add support for older kernels. More similar
> checks will be needed but here is a first batch not to have too many
> patches (because each patch fixes different commits).
>
> I tried to minimise the modifications and do the checks from the bash
> script not to have to detect the feature when being used in a .c file,
> then pass the info to the bash script, skip the test, clean the
> environment, etc.
>
> Link: https://github.com/multipath-tcp/mptcp_net-next/issues/368
> Signed-off-by: Matthieu Baerts <matthieu.baerts@tessares.net>

Hi Matthieu -

The "skip if MPTCP is not supported" patches (1-7) and patch 10 (missing 
ip tool) look good to me:

Reviewed-by: Mat Martineau <martineau@kernel.org>


I have some comments on patch 8 and related, I'll reply separately on 
those.

Thanks!

Mat


> ---
> Matthieu Baerts (13):
>      selftests: mptcp: connect: skip if MPTCP is not supported
>      selftests: mptcp: pm nl: skip if MPTCP is not supported
>      selftests: mptcp: join: skip if MPTCP is not supported
>      selftests: mptcp: diag: skip if MPTCP is not supported
>      selftests: mptcp: simult flows: skip if MPTCP is not supported
>      selftests: mptcp: sockopt: skip if MPTCP is not supported
>      selftests: mptcp: userspace pm: skip if MPTCP is not supported
>      selftests: mptcp: lib: skip if missing symbol
>      selftests: mptcp: connect: skip TFO tests if not supported
>      selftests: mptcp: userspace pm: skip if 'ip' tool is unavailable
>      selftests: mptcp: userspace pm: skip if not supported
>      selftests: mptcp: userspace pm: skip PM listener events tests if unavailable
>      selftests: mptcp: join: skip PM listener events tests if unavailable
>
> tools/testing/selftests/net/mptcp/Makefile         |  2 +-
> tools/testing/selftests/net/mptcp/config           |  1 +
> tools/testing/selftests/net/mptcp/diag.sh          |  4 +++
> tools/testing/selftests/net/mptcp/mptcp_connect.sh | 10 +++++++
> tools/testing/selftests/net/mptcp/mptcp_join.sh    | 18 ++++++++++--
> tools/testing/selftests/net/mptcp/mptcp_lib.sh     | 34 ++++++++++++++++++++++
> tools/testing/selftests/net/mptcp/mptcp_sockopt.sh |  4 +++
> tools/testing/selftests/net/mptcp/pm_netlink.sh    |  4 +++
> tools/testing/selftests/net/mptcp/simult_flows.sh  |  4 +++
> tools/testing/selftests/net/mptcp/userspace_pm.sh  | 17 ++++++++++-
> 10 files changed, 94 insertions(+), 4 deletions(-)
> ---
> base-commit: 016e68a1441264585c15defedc5ccf883b3b4211
> change-id: 20230406-mptcp-issue-368-selftests-old-kernels-fffd69d67d91
>
> Best regards,
> -- 
> Matthieu Baerts <matthieu.baerts@tessares.net>
>
>
>
Re: [PATCH mptcp-net 00/13] selftests: mptcp: skip tests when features are not supported
Posted by Matthieu Baerts 1 year ago
Hi Mat,

On 25/04/2023 01:47, Mat Martineau wrote:
> On Fri, 7 Apr 2023, Matthieu Baerts wrote:
> 
>> Selftests are supposed to run on any kernels, including the old ones not
>> supporting all MPTCP features.
>>
>> These different patches add support for older kernels. More similar
>> checks will be needed but here is a first batch not to have too many
>> patches (because each patch fixes different commits).
>>
>> I tried to minimise the modifications and do the checks from the bash
>> script not to have to detect the feature when being used in a .c file,
>> then pass the info to the bash script, skip the test, clean the
>> environment, etc.
>>
>> Link: https://github.com/multipath-tcp/mptcp_net-next/issues/368
>> Signed-off-by: Matthieu Baerts <matthieu.baerts@tessares.net>
> 
> Hi Matthieu -
> 
> The "skip if MPTCP is not supported" patches (1-7) and patch 10 (missing
> ip tool) look good to me:
> 
> Reviewed-by: Mat Martineau <martineau@kernel.org>

Thank you for having looked at these patches!

I was already working on a v2 and these patches will be slightly
modified. I will not add your RvB tag right now if that's OK.

> I have some comments on patch 8 and related, I'll reply separately on
> those.

Thank you! I only saw one comment on patch 11/13. Did I miss something?

Cheers,
Matt
-- 
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net
Re: [PATCH mptcp-net 00/13] selftests: mptcp: skip tests when features are not supported
Posted by Mat Martineau 1 year ago
On Tue, 25 Apr 2023, Matthieu Baerts wrote:

> Hi Mat,
>
> On 25/04/2023 01:47, Mat Martineau wrote:
>> On Fri, 7 Apr 2023, Matthieu Baerts wrote:
>>
>>> Selftests are supposed to run on any kernels, including the old ones not
>>> supporting all MPTCP features.
>>>
>>> These different patches add support for older kernels. More similar
>>> checks will be needed but here is a first batch not to have too many
>>> patches (because each patch fixes different commits).
>>>
>>> I tried to minimise the modifications and do the checks from the bash
>>> script not to have to detect the feature when being used in a .c file,
>>> then pass the info to the bash script, skip the test, clean the
>>> environment, etc.
>>>
>>> Link: https://github.com/multipath-tcp/mptcp_net-next/issues/368
>>> Signed-off-by: Matthieu Baerts <matthieu.baerts@tessares.net>
>>
>> Hi Matthieu -
>>
>> The "skip if MPTCP is not supported" patches (1-7) and patch 10 (missing
>> ip tool) look good to me:
>>
>> Reviewed-by: Mat Martineau <martineau@kernel.org>
>
> Thank you for having looked at these patches!
>
> I was already working on a v2 and these patches will be slightly
> modified. I will not add your RvB tag right now if that's OK.
>

Sure, no problem.

>> I have some comments on patch 8 and related, I'll reply separately on
>> those.
>
> Thank you! I only saw one comment on patch 11/13. Did I miss something?
>

You didn't miss anything - I should have been clearer that I intended for 
the patch 11 reply to address the common issues of the remaining patches 
(8-9, 11-13).


- Mat
Re: [PATCH mptcp-net 00/13] selftests: mptcp: skip tests when features are not supported
Posted by Mat Martineau 1 year ago
On Mon, 24 Apr 2023, Mat Martineau wrote:

> On Fri, 7 Apr 2023, Matthieu Baerts wrote:
>
>> Selftests are supposed to run on any kernels, including the old ones not
>> supporting all MPTCP features.
>> 
>> These different patches add support for older kernels. More similar
>> checks will be needed but here is a first batch not to have too many
>> patches (because each patch fixes different commits).
>> 
>> I tried to minimise the modifications and do the checks from the bash
>> script not to have to detect the feature when being used in a .c file,
>> then pass the info to the bash script, skip the test, clean the
>> environment, etc.
>> 
>> Link: https://github.com/multipath-tcp/mptcp_net-next/issues/368
>> Signed-off-by: Matthieu Baerts <matthieu.baerts@tessares.net>
>
> Hi Matthieu -
>
> The "skip if MPTCP is not supported" patches (1-7) and patch 10 (missing ip 
> tool) look good to me:
>
> Reviewed-by: Mat Martineau <martineau@kernel.org>
>
>
> I have some comments on patch 8 and related, I'll reply separately on those.
>
> Thanks!
>

One more question:

Did you check some older kernels to see how the tests behaved? So far I 
only had the chance to try 5.4 (because it was easily available). If I 
recall, we may have had some behavior changes along the way that could 
cause some test failures even if features are present.


- Mat

>
>> ---
>> Matthieu Baerts (13):
>>      selftests: mptcp: connect: skip if MPTCP is not supported
>>      selftests: mptcp: pm nl: skip if MPTCP is not supported
>>      selftests: mptcp: join: skip if MPTCP is not supported
>>      selftests: mptcp: diag: skip if MPTCP is not supported
>>      selftests: mptcp: simult flows: skip if MPTCP is not supported
>>      selftests: mptcp: sockopt: skip if MPTCP is not supported
>>      selftests: mptcp: userspace pm: skip if MPTCP is not supported
>>      selftests: mptcp: lib: skip if missing symbol
>>      selftests: mptcp: connect: skip TFO tests if not supported
>>      selftests: mptcp: userspace pm: skip if 'ip' tool is unavailable
>>      selftests: mptcp: userspace pm: skip if not supported
>>      selftests: mptcp: userspace pm: skip PM listener events tests if 
>> unavailable
>>      selftests: mptcp: join: skip PM listener events tests if unavailable
>> 
>> tools/testing/selftests/net/mptcp/Makefile         |  2 +-
>> tools/testing/selftests/net/mptcp/config           |  1 +
>> tools/testing/selftests/net/mptcp/diag.sh          |  4 +++
>> tools/testing/selftests/net/mptcp/mptcp_connect.sh | 10 +++++++
>> tools/testing/selftests/net/mptcp/mptcp_join.sh    | 18 ++++++++++--
>> tools/testing/selftests/net/mptcp/mptcp_lib.sh     | 34 
>> ++++++++++++++++++++++
>> tools/testing/selftests/net/mptcp/mptcp_sockopt.sh |  4 +++
>> tools/testing/selftests/net/mptcp/pm_netlink.sh    |  4 +++
>> tools/testing/selftests/net/mptcp/simult_flows.sh  |  4 +++
>> tools/testing/selftests/net/mptcp/userspace_pm.sh  | 17 ++++++++++-
>> 10 files changed, 94 insertions(+), 4 deletions(-)
>> ---
>> base-commit: 016e68a1441264585c15defedc5ccf883b3b4211
>> change-id: 20230406-mptcp-issue-368-selftests-old-kernels-fffd69d67d91
>> 
>> Best regards,
>> -- 
>> Matthieu Baerts <matthieu.baerts@tessares.net>
>> 
>> 
>> 
>
Re: [PATCH mptcp-net 00/13] selftests: mptcp: skip tests when features are not supported
Posted by Matthieu Baerts 1 year ago
On 25/04/2023 02:21, Mat Martineau wrote:
> On Mon, 24 Apr 2023, Mat Martineau wrote:
> 
>> On Fri, 7 Apr 2023, Matthieu Baerts wrote:
>>
>>> Selftests are supposed to run on any kernels, including the old ones not
>>> supporting all MPTCP features.
>>>
>>> These different patches add support for older kernels. More similar
>>> checks will be needed but here is a first batch not to have too many
>>> patches (because each patch fixes different commits).
>>>
>>> I tried to minimise the modifications and do the checks from the bash
>>> script not to have to detect the feature when being used in a .c file,
>>> then pass the info to the bash script, skip the test, clean the
>>> environment, etc.
>>>
>>> Link: https://github.com/multipath-tcp/mptcp_net-next/issues/368
>>> Signed-off-by: Matthieu Baerts <matthieu.baerts@tessares.net>
>>
>> Hi Matthieu -
>>
>> The "skip if MPTCP is not supported" patches (1-7) and patch 10
>> (missing ip tool) look good to me:
>>
>> Reviewed-by: Mat Martineau <martineau@kernel.org>
>>
>>
>> I have some comments on patch 8 and related, I'll reply separately on
>> those.
>>
>> Thanks!
>>
> 
> One more question:
> 
> Did you check some older kernels to see how the tests behaved? So far I
> only had the chance to try 5.4 (because it was easily available). If I
> recall, we may have had some behavior changes along the way that could
> cause some test failures even if features are present.

Yes, I previously checked on v6.1 and in between, I also checked on
v5.15. That's why a v2 is coming :)

I didn't check on v5.10 but I don't think it is worth trying to support
it, mptcp_join.sh is validating a lot of unsupported features in v5.10.
But if someone else wants to look at it, why not :)

Cheers,
Matt
-- 
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net
Re: [PATCH mptcp-net 00/13] selftests: mptcp: skip tests when features are not supported
Posted by Geliang Tang 1 year, 1 month ago
Matthieu Baerts <matthieu.baerts@tessares.net> 于2023年4月8日周六 01:06写道:
>
> Selftests are supposed to run on any kernels, including the old ones not
> supporting all MPTCP features.
>
> These different patches add support for older kernels. More similar
> checks will be needed but here is a first batch not to have too many
> patches (because each patch fixes different commits).
>
> I tried to minimise the modifications and do the checks from the bash
> script not to have to detect the feature when being used in a .c file,
> then pass the info to the bash script, skip the test, clean the
> environment, etc.
>
> Link: https://github.com/multipath-tcp/mptcp_net-next/issues/368
> Signed-off-by: Matthieu Baerts <matthieu.baerts@tessares.net>
> ---
> Matthieu Baerts (13):
>       selftests: mptcp: connect: skip if MPTCP is not supported
>       selftests: mptcp: pm nl: skip if MPTCP is not supported
>       selftests: mptcp: join: skip if MPTCP is not supported
>       selftests: mptcp: diag: skip if MPTCP is not supported
>       selftests: mptcp: simult flows: skip if MPTCP is not supported
>       selftests: mptcp: sockopt: skip if MPTCP is not supported
>       selftests: mptcp: userspace pm: skip if MPTCP is not supported
>       selftests: mptcp: lib: skip if missing symbol
>       selftests: mptcp: connect: skip TFO tests if not supported
>       selftests: mptcp: userspace pm: skip if 'ip' tool is unavailable
>       selftests: mptcp: userspace pm: skip if not supported
>       selftests: mptcp: userspace pm: skip PM listener events tests if unavailable

Hi Matt,

How about merging these three commits and "selftests: mptcp: userspace
pm: skip if MPTCP is not supported" into a single commit?

Thanks,
-Geliang

>       selftests: mptcp: join: skip PM listener events tests if unavailable
>
>  tools/testing/selftests/net/mptcp/Makefile         |  2 +-
>  tools/testing/selftests/net/mptcp/config           |  1 +
>  tools/testing/selftests/net/mptcp/diag.sh          |  4 +++
>  tools/testing/selftests/net/mptcp/mptcp_connect.sh | 10 +++++++
>  tools/testing/selftests/net/mptcp/mptcp_join.sh    | 18 ++++++++++--
>  tools/testing/selftests/net/mptcp/mptcp_lib.sh     | 34 ++++++++++++++++++++++
>  tools/testing/selftests/net/mptcp/mptcp_sockopt.sh |  4 +++
>  tools/testing/selftests/net/mptcp/pm_netlink.sh    |  4 +++
>  tools/testing/selftests/net/mptcp/simult_flows.sh  |  4 +++
>  tools/testing/selftests/net/mptcp/userspace_pm.sh  | 17 ++++++++++-
>  10 files changed, 94 insertions(+), 4 deletions(-)
> ---
> base-commit: 016e68a1441264585c15defedc5ccf883b3b4211
> change-id: 20230406-mptcp-issue-368-selftests-old-kernels-fffd69d67d91
>
> Best regards,
> --
> Matthieu Baerts <matthieu.baerts@tessares.net>
>
>
Re: [PATCH mptcp-net 00/13] selftests: mptcp: skip tests when features are not supported
Posted by Matthieu Baerts 1 year, 1 month ago
Hi Geliang,

Thank you for having looked at these patches.

On 20/04/2023 05:49, Geliang Tang wrote:
> Matthieu Baerts <matthieu.baerts@tessares.net> 于2023年4月8日周六 01:06写道:
>>
>> Selftests are supposed to run on any kernels, including the old ones not
>> supporting all MPTCP features.
>>
>> These different patches add support for older kernels. More similar
>> checks will be needed but here is a first batch not to have too many
>> patches (because each patch fixes different commits).
>>
>> I tried to minimise the modifications and do the checks from the bash
>> script not to have to detect the feature when being used in a .c file,
>> then pass the info to the bash script, skip the test, clean the
>> environment, etc.
>>
>> Link: https://github.com/multipath-tcp/mptcp_net-next/issues/368
>> Signed-off-by: Matthieu Baerts <matthieu.baerts@tessares.net>
>> ---
>> Matthieu Baerts (13):
>>       selftests: mptcp: connect: skip if MPTCP is not supported
>>       selftests: mptcp: pm nl: skip if MPTCP is not supported
>>       selftests: mptcp: join: skip if MPTCP is not supported
>>       selftests: mptcp: diag: skip if MPTCP is not supported
>>       selftests: mptcp: simult flows: skip if MPTCP is not supported
>>       selftests: mptcp: sockopt: skip if MPTCP is not supported
>>       selftests: mptcp: userspace pm: skip if MPTCP is not supported
>>       selftests: mptcp: lib: skip if missing symbol
>>       selftests: mptcp: connect: skip TFO tests if not supported
>>       selftests: mptcp: userspace pm: skip if 'ip' tool is unavailable
>>       selftests: mptcp: userspace pm: skip if not supported
>>       selftests: mptcp: userspace pm: skip PM listener events tests if unavailable
> 
> Hi Matt,
> 
> How about merging these three commits and "selftests: mptcp: userspace
> pm: skip if MPTCP is not supported" into a single commit?

I prefer not to because they fix different issues, to be backported to
different stable versions. If I mix all of them together, it will be
harder to have them backported.

Maybe the patches 7/13 and 11/13 could be squashed together but I didn't
because the 7 first patches are doing the same for all our different
tests we have (and different "Fixes" tags).

Cheers,
Matt
-- 
Tessares | Belgium | Hybrid Access Solutions
www.tessares.net