sched: avoid unnecessary atomic_read when sync_core_before_usermode() is empty

[RFC PATCH] sched: avoid unnecessary atomic_read when sync_core_before_usermode() is empty

Posted by Tianchen Ding 4 years, 2 months ago

On archs except x86, CONFIG_ARCH_HAS_SYNC_CORE_BEFORE_USERMODE is not
defined. We found membarrier_mm_sync_core_before_usermode() looks like
this when compiled by gcc10:

	if (current->mm != mm)
		return;
	atomic_read(&mm->membarrier_state);

This memory access is unnecessary. Remove it to improve performance.

Signed-off-by: Tianchen Ding <dtcccc@linux.alibaba.com>
---
 include/linux/sched/mm.h | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/include/linux/sched/mm.h b/include/linux/sched/mm.h
index a80356e9dc69..3ded68d9f913 100644
--- a/include/linux/sched/mm.h
+++ b/include/linux/sched/mm.h
@@ -401,6 +401,7 @@ enum {
 #include <asm/membarrier.h>
 #endif
 
+#ifdef CONFIG_ARCH_HAS_SYNC_CORE_BEFORE_USERMODE
 static inline void membarrier_mm_sync_core_before_usermode(struct mm_struct *mm)
 {
 	if (current->mm != mm)
@@ -410,6 +411,11 @@ static inline void membarrier_mm_sync_core_before_usermode(struct mm_struct *mm)
 		return;
 	sync_core_before_usermode();
 }
+#else
+static inline void membarrier_mm_sync_core_before_usermode(struct mm_struct *mm)
+{
+}
+#endif
 
 extern void membarrier_exec_mmap(struct mm_struct *mm);
 
-- 
2.33.0

Re: [RFC PATCH] sched: avoid unnecessary atomic_read when sync_core_before_usermode() is empty

Posted by Tianchen Ding 4 years, 2 months ago

We've run schbench and found wakeup latency on some arm64 machines worse 
than others. Perf shows there's a hotspot on 
atomic_read(&mm->membarrier_state);

We're still working for the real reason behind it (maybe cache or sth 
hardware related), and we do see remove this function can help improve 
performance.

Thanks.

On 2022/4/2 11:08, Tianchen Ding wrote:
> On archs except x86, CONFIG_ARCH_HAS_SYNC_CORE_BEFORE_USERMODE is not
> defined. We found membarrier_mm_sync_core_before_usermode() looks like
> this when compiled by gcc10:
> 
> 	if (current->mm != mm)
> 		return;
> 	atomic_read(&mm->membarrier_state);
> 
> This memory access is unnecessary. Remove it to improve performance.
> 
> Signed-off-by: Tianchen Ding <dtcccc@linux.alibaba.com>
> ---
>   include/linux/sched/mm.h | 6 ++++++
>   1 file changed, 6 insertions(+)
> 
> diff --git a/include/linux/sched/mm.h b/include/linux/sched/mm.h
> index a80356e9dc69..3ded68d9f913 100644
> --- a/include/linux/sched/mm.h
> +++ b/include/linux/sched/mm.h
> @@ -401,6 +401,7 @@ enum {
>   #include <asm/membarrier.h>
>   #endif
>   
> +#ifdef CONFIG_ARCH_HAS_SYNC_CORE_BEFORE_USERMODE
>   static inline void membarrier_mm_sync_core_before_usermode(struct mm_struct *mm)
>   {
>   	if (current->mm != mm)
> @@ -410,6 +411,11 @@ static inline void membarrier_mm_sync_core_before_usermode(struct mm_struct *mm)
>   		return;
>   	sync_core_before_usermode();
>   }
> +#else
> +static inline void membarrier_mm_sync_core_before_usermode(struct mm_struct *mm)
> +{
> +}
> +#endif
>   
>   extern void membarrier_exec_mmap(struct mm_struct *mm);
>