[v3] Allow preemption during IPI completion waiting to improve real-time performance

[PATCH v3 02/12] smp: Enable preemption early in smp_call_function_single

Posted by Chuyi Zhou 2 weeks, 5 days ago

Now smp_call_function_single() disables preemption mainly for the following
reasons:

- To protect the per-cpu csd_data from concurrent modification by other
tasks on the current CPU in the !wait case. For the wait case,
synchronization is not a concern as on-stack csd is used.

- To prevent the remote online CPU from being offlined. Specifically, we
want to ensure that no new IPIs are queued after smpcfd_dying_cpu() has
finished.

Disabling preemption for the entire execution is unnecessary, especially
csd_lock_wait() part does not require preemption protection. This patch
enables preemption before csd_lock_wait() to reduce the preemption-disabled
critical section.

Signed-off-by: Chuyi Zhou <zhouchuyi@bytedance.com>
Reviewed-by: Muchun Song <muchun.song@linux.dev>
---
 kernel/smp.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/kernel/smp.c b/kernel/smp.c
index fc1f7a964616..b603d4229f95 100644
--- a/kernel/smp.c
+++ b/kernel/smp.c
@@ -685,11 +685,16 @@ int smp_call_function_single(int cpu, smp_call_func_t func, void *info,
 
 	err = generic_exec_single(cpu, csd);
 
+	/*
+	 * @csd is stack-allocated when @wait is true. No concurrent access
+	 * except from the IPI completion path, so we can re-enable preemption
+	 * early to reduce latency.
+	 */
+	put_cpu();
+
 	if (wait)
 		csd_lock_wait(csd);
 
-	put_cpu();
-
 	return err;
 }
 EXPORT_SYMBOL(smp_call_function_single);
-- 
2.20.1

Re: [PATCH v3 02/12] smp: Enable preemption early in smp_call_function_single

Posted by Steven Rostedt 2 weeks, 5 days ago

On Wed, 18 Mar 2026 12:56:28 +0800
"Chuyi Zhou" <zhouchuyi@bytedance.com> wrote:

> diff --git a/kernel/smp.c b/kernel/smp.c
> index fc1f7a964616..b603d4229f95 100644
> --- a/kernel/smp.c
> +++ b/kernel/smp.c
> @@ -685,11 +685,16 @@ int smp_call_function_single(int cpu, smp_call_func_t func, void *info,
>  
>  	err = generic_exec_single(cpu, csd);
>  
> +	/*
> +	 * @csd is stack-allocated when @wait is true. No concurrent access
> +	 * except from the IPI completion path, so we can re-enable preemption
> +	 * early to reduce latency.
> +	 */

Thanks for the comment. I walked through the code and this looks fine to me.

> +	put_cpu();
> +
>  	if (wait)
>  		csd_lock_wait(csd);
>  
> -	put_cpu();
> -
>  	return err;

Reviewed-by: Steven Rostedt (Google) <rostedt@goodmis.org>

-- Steve

Re: [PATCH v3 02/12] smp: Enable preemption early in smp_call_function_single

Posted by Chuyi Zhou 2 weeks, 4 days ago

Hi Steve,

在 2026/3/18 22:14, Steven Rostedt 写道:
> On Wed, 18 Mar 2026 12:56:28 +0800
> "Chuyi Zhou" <zhouchuyi@bytedance.com> wrote:
> 
>> diff --git a/kernel/smp.c b/kernel/smp.c
>> index fc1f7a964616..b603d4229f95 100644
>> --- a/kernel/smp.c
>> +++ b/kernel/smp.c
>> @@ -685,11 +685,16 @@ int smp_call_function_single(int cpu, smp_call_func_t func, void *info,
>>   
>>   	err = generic_exec_single(cpu, csd);
>>   
>> +	/*
>> +	 * @csd is stack-allocated when @wait is true. No concurrent access
>> +	 * except from the IPI completion path, so we can re-enable preemption
>> +	 * early to reduce latency.
>> +	 */
> 
> Thanks for the comment. I walked through the code and this looks fine to me.
> 
>> +	put_cpu();
>> +
>>   	if (wait)
>>   		csd_lock_wait(csd);
>>   
>> -	put_cpu();
>> -
>>   	return err;
> 
> Reviewed-by: Steven Rostedt (Google) <rostedt@goodmis.org>
> 

Thanks!

> -- Steve