selftests: kselftest: fix wrong test number in ksft_exit_skip

[PATCH] selftests: kselftest: fix wrong test number in ksft_exit_skip

Posted by Sarthak Sharma 1 month, 2 weeks ago

ksft_exit_skip() increments ksft_xskip before printing the KTAP
result. As a result, ksft_test_num() already includes the skipped
test.

Adding 1 to ksft_test_num() increments the printed test number
again, producing an incorrect test number and wrong KTAP output.

Drop the extra increment and print ksft_test_num() directly.

Fixes: b85d387c9b09 ("kselftest: fix TAP output for skipped tests")
Signed-off-by: Sarthak Sharma <sarthak.sharma@arm.com>
---
 tools/testing/selftests/kselftest.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/kselftest.h b/tools/testing/selftests/kselftest.h
index afbcf8412ae5..827b47cf4df8 100644
--- a/tools/testing/selftests/kselftest.h
+++ b/tools/testing/selftests/kselftest.h
@@ -449,7 +449,7 @@ static inline __noreturn __printf(1, 2) void ksft_exit_skip(const char *msg, ...
 	 */
 	if (ksft_plan || ksft_test_num()) {
 		ksft_cnt.ksft_xskip++;
-		printf("ok %u # SKIP ", 1 + ksft_test_num());
+		printf("ok %u # SKIP ", ksft_test_num());
 	} else {
 		printf("1..0 # SKIP ");
 	}
-- 
2.39.5

Re: [PATCH] selftests: kselftest: fix wrong test number in ksft_exit_skip

Posted by Shuah Khan 1 month, 2 weeks ago

On 4/27/26 05:24, Sarthak Sharma wrote:
> ksft_exit_skip() increments ksft_xskip before printing the KTAP
> result. As a result, ksft_test_num() already includes the skipped
> test.
> 
> Adding 1 to ksft_test_num() increments the printed test number
> again, producing an incorrect test number and wrong KTAP output.
> 
> Drop the extra increment and print ksft_test_num() directly.

I applied this patch to linux-kselftest fixes branch.

I ran a few tests and couldn't really see the problem this patch
fixes with and without the patch.

Can you send me before and after for mm or timers test runs?

> 
> Fixes: b85d387c9b09 ("kselftest: fix TAP output for skipped tests")
> Signed-off-by: Sarthak Sharma <sarthak.sharma@arm.com>
> ---
>   tools/testing/selftests/kselftest.h | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/kselftest.h b/tools/testing/selftests/kselftest.h
> index afbcf8412ae5..827b47cf4df8 100644
> --- a/tools/testing/selftests/kselftest.h
> +++ b/tools/testing/selftests/kselftest.h
> @@ -449,7 +449,7 @@ static inline __noreturn __printf(1, 2) void ksft_exit_skip(const char *msg, ...
>   	 */
>   	if (ksft_plan || ksft_test_num()) {
>   		ksft_cnt.ksft_xskip++;
> -		printf("ok %u # SKIP ", 1 + ksft_test_num());
> +		printf("ok %u # SKIP ", ksft_test_num());
>   	} else {
>   		printf("1..0 # SKIP ");
>   	}

thanks,
-- Shuah

Re: [PATCH] selftests: kselftest: fix wrong test number in ksft_exit_skip

Posted by Sarthak Sharma 1 month, 2 weeks ago

Hi Shuah!

On 4/27/26 11:14 PM, Shuah Khan wrote:
> On 4/27/26 05:24, Sarthak Sharma wrote:
>> ksft_exit_skip() increments ksft_xskip before printing the KTAP
>> result. As a result, ksft_test_num() already includes the skipped
>> test.
>>
>> Adding 1 to ksft_test_num() increments the printed test number
>> again, producing an incorrect test number and wrong KTAP output.
>>
>> Drop the extra increment and print ksft_test_num() directly.
> 
> I applied this patch to linux-kselftest fixes branch.
> 
> I ran a few tests and couldn't really see the problem this patch
> fixes with and without the patch.
> 
> Can you send me before and after for mm or timers test runs?

Sure, this will affect the output whenever a ksft_exit_skip() is called
after a ksft plan has been printed.

Let us look at acct_syscall test. When I run the test without root, I
get the following output before applying the patch:

TAP version 13
1..1
ok 2 # SKIP This test needs root to run!
# 1 skipped test(s) detected. Consider enabling relevant config options
to improve coverage.
# Totals: pass:0 fail:0 xfail:0 xpass:0 skip:1 error:0

where we should have observed ok 1 instead of ok 2.

When I run the test without root, after applying the patch, I get the
corrected output:

TAP version 13
1..1
ok 1 # SKIP This test needs root to run!
# 1 skipped test(s) detected. Consider enabling relevant config options
to improve coverage.
# Totals: pass:0 fail:0 xfail:0 xpass:0 skip:1 error:0

Since a lot of selftests are being formatted to give KTAP output right
now, this patch is expected to fix a larger number of tests once that
gets merged.

> 
>>
>> Fixes: b85d387c9b09 ("kselftest: fix TAP output for skipped tests")
>> Signed-off-by: Sarthak Sharma <sarthak.sharma@arm.com>
>> ---
>>   tools/testing/selftests/kselftest.h | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/tools/testing/selftests/kselftest.h b/tools/testing/
>> selftests/kselftest.h
>> index afbcf8412ae5..827b47cf4df8 100644
>> --- a/tools/testing/selftests/kselftest.h
>> +++ b/tools/testing/selftests/kselftest.h
>> @@ -449,7 +449,7 @@ static inline __noreturn __printf(1, 2) void
>> ksft_exit_skip(const char *msg, ...
>>        */
>>       if (ksft_plan || ksft_test_num()) {
>>           ksft_cnt.ksft_xskip++;
>> -        printf("ok %u # SKIP ", 1 + ksft_test_num());
>> +        printf("ok %u # SKIP ", ksft_test_num());
>>       } else {
>>           printf("1..0 # SKIP ");
>>       }
> 
> thanks,
> -- Shuah

Re: [PATCH] selftests: kselftest: fix wrong test number in ksft_exit_skip

Posted by Shuah Khan 1 month, 1 week ago

On 4/27/26 23:10, Sarthak Sharma wrote:
> Hi Shuah!
> 
> On 4/27/26 11:14 PM, Shuah Khan wrote:
>> On 4/27/26 05:24, Sarthak Sharma wrote:
>>> ksft_exit_skip() increments ksft_xskip before printing the KTAP
>>> result. As a result, ksft_test_num() already includes the skipped
>>> test.
>>>
>>> Adding 1 to ksft_test_num() increments the printed test number
>>> again, producing an incorrect test number and wrong KTAP output.
>>>
>>> Drop the extra increment and print ksft_test_num() directly.
>>
>> I applied this patch to linux-kselftest fixes branch.
>>
>> I ran a few tests and couldn't really see the problem this patch
>> fixes with and without the patch.
>>
>> Can you send me before and after for mm or timers test runs?
> 
> Sure, this will affect the output whenever a ksft_exit_skip() is called
> after a ksft plan has been printed.
> 
> Let us look at acct_syscall test. When I run the test without root, I
> get the following output before applying the patch:
> 
> TAP version 13
> 1..1
> ok 2 # SKIP This test needs root to run!
> # 1 skipped test(s) detected. Consider enabling relevant config options
> to improve coverage.
> # Totals: pass:0 fail:0 xfail:0 xpass:0 skip:1 error:0
> 
> where we should have observed ok 1 instead of ok 2.
> 
> When I run the test without root, after applying the patch, I get the
> corrected output:
> 
> TAP version 13
> 1..1
> ok 1 # SKIP This test needs root to run!
> # 1 skipped test(s) detected. Consider enabling relevant config options
> to improve coverage.
> # Totals: pass:0 fail:0 xfail:0 xpass:0 skip:1 error:0
> 
> Since a lot of selftests are being formatted to give KTAP output right
> now, this patch is expected to fix a larger number of tests once that
> gets merged.
> 

Sounds good.

Thanks.
-- Shuah