[PATCH 3/6] mm: always inline __mk_vma_flags() and invoked functions

Lorenzo Stoakes (Oracle) posted 6 patches 1 month ago
[PATCH 3/6] mm: always inline __mk_vma_flags() and invoked functions
Posted by Lorenzo Stoakes (Oracle) 1 month ago
Be explicit about __mk_vma_flags() (which is used by the mk_vma_flags()
macro) always being inline, as we rely on the compiler converting this
function into meaningful.

Also update all of the functions __mk_vma_flags() ultimately invokes to be
always inline too.

Note that test_bitmap_const_eval() asserts that the relevant bitmap
functions result in build time constant values.

Additionally, vma_flag_set() operates on a vma_flags_t type, so it is
inconsistently named versus other VMA flags functions.

We only use vma_flag_set() in __mk_vma_flags() so we don't need to worry
about its new name being rather cumbersome, so rename it to
vma_flags_set_flag() to disambiguate it from vma_flags_set().

Also update the VMA test headers to reflect the changes.

Signed-off-by: Lorenzo Stoakes (Oracle) <ljs@kernel.org>
---
 include/linux/mm.h                 | 8 +++++---
 include/linux/mm_types.h           | 2 +-
 tools/testing/vma/include/custom.h | 5 +++--
 tools/testing/vma/include/dup.h    | 5 +++--
 4 files changed, 12 insertions(+), 8 deletions(-)

diff --git a/include/linux/mm.h b/include/linux/mm.h
index 9a052eedcdf4..66b90de30bf6 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -1031,21 +1031,23 @@ static inline bool vma_test_atomic_flag(struct vm_area_struct *vma, vma_flag_t b
 }
 
 /* Set an individual VMA flag in flags, non-atomically. */
-static inline void vma_flag_set(vma_flags_t *flags, vma_flag_t bit)
+static __always_inline void vma_flags_set_flag(vma_flags_t *flags,
+		vma_flag_t bit)
 {
 	unsigned long *bitmap = flags->__vma_flags;
 
 	__set_bit((__force int)bit, bitmap);
 }
 
-static inline vma_flags_t __mk_vma_flags(size_t count, const vma_flag_t *bits)
+static __always_inline vma_flags_t __mk_vma_flags(size_t count,
+		const vma_flag_t *bits)
 {
 	vma_flags_t flags;
 	int i;
 
 	vma_flags_clear_all(&flags);
 	for (i = 0; i < count; i++)
-		vma_flag_set(&flags, bits[i]);
+		vma_flags_set_flag(&flags, bits[i]);
 	return flags;
 }
 
diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h
index 1a808d78245d..294efc22b2a4 100644
--- a/include/linux/mm_types.h
+++ b/include/linux/mm_types.h
@@ -1056,7 +1056,7 @@ struct vm_area_struct {
 } __randomize_layout;
 
 /* Clears all bits in the VMA flags bitmap, non-atomically. */
-static inline void vma_flags_clear_all(vma_flags_t *flags)
+static __always_inline void vma_flags_clear_all(vma_flags_t *flags)
 {
 	bitmap_zero(flags->__vma_flags, NUM_VMA_FLAG_BITS);
 }
diff --git a/tools/testing/vma/include/custom.h b/tools/testing/vma/include/custom.h
index 802a76317245..833ff4d7f799 100644
--- a/tools/testing/vma/include/custom.h
+++ b/tools/testing/vma/include/custom.h
@@ -102,7 +102,8 @@ static inline void vma_lock_init(struct vm_area_struct *vma, bool reset_refcnt)
 		refcount_set(&vma->vm_refcnt, 0);
 }
 
-static inline vma_flags_t __mk_vma_flags(size_t count, const vma_flag_t *bits)
+static __always_inline vma_flags_t __mk_vma_flags(size_t count,
+		const vma_flag_t *bits)
 {
 	vma_flags_t flags;
 	int i;
@@ -114,6 +115,6 @@ static inline vma_flags_t __mk_vma_flags(size_t count, const vma_flag_t *bits)
 	vma_flags_clear_all(&flags);
 	for (i = 0; i < count; i++)
 		if (bits[i] < NUM_VMA_FLAG_BITS)
-			vma_flag_set(&flags, bits[i]);
+			vma_flags_set_flag(&flags, bits[i]);
 	return flags;
 }
diff --git a/tools/testing/vma/include/dup.h b/tools/testing/vma/include/dup.h
index 59788bc14d75..ef6b9d963acc 100644
--- a/tools/testing/vma/include/dup.h
+++ b/tools/testing/vma/include/dup.h
@@ -780,12 +780,13 @@ static inline void vma_flags_clear_word(vma_flags_t *flags, unsigned long value)
 	*bitmap &= ~value;
 }
 
-static inline void vma_flags_clear_all(vma_flags_t *flags)
+static __always_inline void vma_flags_clear_all(vma_flags_t *flags)
 {
 	bitmap_zero(ACCESS_PRIVATE(flags, __vma_flags), NUM_VMA_FLAG_BITS);
 }
 
-static inline void vma_flag_set(vma_flags_t *flags, vma_flag_t bit)
+static __always_inline void vma_flags_set_flag(vma_flags_t *flags,
+		vma_flag_t bit)
 {
 	unsigned long *bitmap = ACCESS_PRIVATE(flags, __vma_flags);
 
-- 
2.53.0
Re: [PATCH 3/6] mm: always inline __mk_vma_flags() and invoked functions
Posted by Pedro Falcato 2 weeks, 1 day ago
On Thu, Mar 05, 2026 at 10:50:16AM +0000, Lorenzo Stoakes (Oracle) wrote:
> Be explicit about __mk_vma_flags() (which is used by the mk_vma_flags()
> macro) always being inline, as we rely on the compiler converting this
> function into meaningful.
		meaningful what?

> 
> Also update all of the functions __mk_vma_flags() ultimately invokes to be
> always inline too.
> 
> Note that test_bitmap_const_eval() asserts that the relevant bitmap
> functions result in build time constant values.
> 
> Additionally, vma_flag_set() operates on a vma_flags_t type, so it is
> inconsistently named versus other VMA flags functions.
> 
> We only use vma_flag_set() in __mk_vma_flags() so we don't need to worry
> about its new name being rather cumbersome, so rename it to
> vma_flags_set_flag() to disambiguate it from vma_flags_set().
> 
> Also update the VMA test headers to reflect the changes.
> 
> Signed-off-by: Lorenzo Stoakes (Oracle) <ljs@kernel.org>

Reviewed-by: Pedro Falcato <pfalcato@suse.de>

Is there an actual difference in codegen here? On -O2.

-- 
Pedro
Re: [PATCH 3/6] mm: always inline __mk_vma_flags() and invoked functions
Posted by Lorenzo Stoakes (Oracle) 2 weeks, 1 day ago
On Wed, Mar 25, 2026 at 02:54:50PM +0000, Pedro Falcato wrote:
> On Thu, Mar 05, 2026 at 10:50:16AM +0000, Lorenzo Stoakes (Oracle) wrote:
> > Be explicit about __mk_vma_flags() (which is used by the mk_vma_flags()
> > macro) always being inline, as we rely on the compiler converting this
> > function into meaningful.
> 		meaningful what?

'into the equivalent compile-time constant code' probably fine.

Andrew - could you update that if there's time?

>
> >
> > Also update all of the functions __mk_vma_flags() ultimately invokes to be
> > always inline too.
> >
> > Note that test_bitmap_const_eval() asserts that the relevant bitmap
> > functions result in build time constant values.
> >
> > Additionally, vma_flag_set() operates on a vma_flags_t type, so it is
> > inconsistently named versus other VMA flags functions.
> >
> > We only use vma_flag_set() in __mk_vma_flags() so we don't need to worry
> > about its new name being rather cumbersome, so rename it to
> > vma_flags_set_flag() to disambiguate it from vma_flags_set().
> >
> > Also update the VMA test headers to reflect the changes.
> >
> > Signed-off-by: Lorenzo Stoakes (Oracle) <ljs@kernel.org>
>
> Reviewed-by: Pedro Falcato <pfalcato@suse.de>
>
> Is there an actual difference in codegen here? On -O2.

No, this is more about consistency.

>
> --
> Pedro

Thanks, Lorenzo
Re: [PATCH 3/6] mm: always inline __mk_vma_flags() and invoked functions
Posted by Andrew Morton 2 weeks, 1 day ago
On Wed, 25 Mar 2026 14:58:14 +0000 "Lorenzo Stoakes (Oracle)" <ljs@kernel.org> wrote:

> On Wed, Mar 25, 2026 at 02:54:50PM +0000, Pedro Falcato wrote:
> > On Thu, Mar 05, 2026 at 10:50:16AM +0000, Lorenzo Stoakes (Oracle) wrote:
> > > Be explicit about __mk_vma_flags() (which is used by the mk_vma_flags()
> > > macro) always being inline, as we rely on the compiler converting this
> > > function into meaningful.
> > 		meaningful what?
> 
> 'into the equivalent compile-time constant code' probably fine.
> 
> Andrew - could you update that if there's time?


np


: Be explicit about __mk_vma_flags() (which is used by the mk_vma_flags()
: macro) always being inline, as we rely on the compiler converting this
: function into the equivalent compile-time constant code.

what does "compile-time constant code" actually mean?  That constants
within the code are evaluated at compile-time?
Re: [PATCH 3/6] mm: always inline __mk_vma_flags() and invoked functions
Posted by Lorenzo Stoakes (Oracle) 2 weeks, 1 day ago
On Wed, Mar 25, 2026 at 09:09:49AM -0700, Andrew Morton wrote:
> On Wed, 25 Mar 2026 14:58:14 +0000 "Lorenzo Stoakes (Oracle)" <ljs@kernel.org> wrote:
>
> > On Wed, Mar 25, 2026 at 02:54:50PM +0000, Pedro Falcato wrote:
> > > On Thu, Mar 05, 2026 at 10:50:16AM +0000, Lorenzo Stoakes (Oracle) wrote:
> > > > Be explicit about __mk_vma_flags() (which is used by the mk_vma_flags()
> > > > macro) always being inline, as we rely on the compiler converting this
> > > > function into meaningful.
> > > 		meaningful what?
> >
> > 'into the equivalent compile-time constant code' probably fine.
> >
> > Andrew - could you update that if there's time?
>
>
> np
>
>
> : Be explicit about __mk_vma_flags() (which is used by the mk_vma_flags()
> : macro) always being inline, as we rely on the compiler converting this
> : function into the equivalent compile-time constant code.
>
> what does "compile-time constant code" actually mean?  That constants
> within the code are evaluated at compile-time?

Yeah, so effectively the compiler rewrites:

x = mk_vma_flags(VMA_READ_BIT, VMA_WRITE_BIT);

To:

x = (1UL << VMA_READ_BIT) | (1UL << VMA_WRITE_BIT);

And then:

x = 3;

Various efforts at checking generated assembly has confirmed this.


Maybe 'into an inline constant value' is better?

Thanks, Lorenzo
Re: [PATCH 3/6] mm: always inline __mk_vma_flags() and invoked functions
Posted by Andrew Morton 2 weeks, 1 day ago
On Wed, 25 Mar 2026 16:23:53 +0000 "Lorenzo Stoakes (Oracle)" <ljs@kernel.org> wrote:

> Maybe 'into an inline constant value' is better?

<bikeshedbikeshed>

How about

: Be explicit about __mk_vma_flags() (which is used by the mk_vma_flags()
: macro) always being inline, as we rely on the compiler turning all
: constants into compile-time ones.
Re: [PATCH 3/6] mm: always inline __mk_vma_flags() and invoked functions
Posted by Lorenzo Stoakes (Oracle) 2 weeks, 1 day ago
On Wed, Mar 25, 2026 at 11:27:55AM -0700, Andrew Morton wrote:
> On Wed, 25 Mar 2026 16:23:53 +0000 "Lorenzo Stoakes (Oracle)" <ljs@kernel.org> wrote:
>
> > Maybe 'into an inline constant value' is better?
>
> <bikeshedbikeshed>
>
> How about
>
> : Be explicit about __mk_vma_flags() (which is used by the mk_vma_flags()
> : macro) always being inline, as we rely on the compiler turning all
> : constants into compile-time ones.
>

Well I think that loses the meaning a bit.

Something like:

Be explicit about __mk_vma_flags() (which is used by the mk_vma_flags()
-macro) always being inline, as we rely on the compiler converting this
-function into meaningful.
+macro) always being inline, as we rely on the compiler to evaluate the
+loop in this function and determine that it can replace the code with the
+an equivalent constant value, e.g. that:
+
+__mk_vma_flags(2, (const vma_flag_t []){ VMA_WRITE_BIT, VMA_EXEC_BIT });
+
+Can be replaced with:
+
+(1UL << VMA_WRITE_BIT) | (1UL << VMA_EXEC_BIT)
+
+= (1UL << 1) | (1UL << 2) = 6
+
+Most likely an 'inline' will suffice for this, but be explicit as we can
+be.

Should verbosely cover that off.

Thanks, Lorenzo
Re: [PATCH 3/6] mm: always inline __mk_vma_flags() and invoked functions
Posted by Andrew Morton 2 weeks, 1 day ago
On Wed, 25 Mar 2026 18:44:13 +0000 "Lorenzo Stoakes (Oracle)" <ljs@kernel.org> wrote:

> > : Be explicit about __mk_vma_flags() (which is used by the mk_vma_flags()
> > : macro) always being inline, as we rely on the compiler turning all
> > : constants into compile-time ones.
> >
> 
> Well I think that loses the meaning a bit.
> 
> Something like:
> 
> Be explicit about __mk_vma_flags() (which is used by the mk_vma_flags()
> -macro) always being inline, as we rely on the compiler converting this
> -function into meaningful.
> +macro) always being inline, as we rely on the compiler to evaluate the
> +loop in this function and determine that it can replace the code with the
> +an equivalent constant value, e.g. that:
> +
> +__mk_vma_flags(2, (const vma_flag_t []){ VMA_WRITE_BIT, VMA_EXEC_BIT });
> +
> +Can be replaced with:
> +
> +(1UL << VMA_WRITE_BIT) | (1UL << VMA_EXEC_BIT)
> +
> += (1UL << 1) | (1UL << 2) = 6
> +
> +Most likely an 'inline' will suffice for this, but be explicit as we can
> +be.
> 
> Should verbosely cover that off.

ok ;)
Re: [PATCH 3/6] mm: always inline __mk_vma_flags() and invoked functions
Posted by David Hildenbrand (Arm) 1 month ago
On 3/5/26 11:50, Lorenzo Stoakes (Oracle) wrote:
> Be explicit about __mk_vma_flags() (which is used by the mk_vma_flags()
> macro) always being inline, as we rely on the compiler converting this
> function into meaningful.
> 
> Also update all of the functions __mk_vma_flags() ultimately invokes to be
> always inline too.
> 
> Note that test_bitmap_const_eval() asserts that the relevant bitmap
> functions result in build time constant values.
> 
> Additionally, vma_flag_set() operates on a vma_flags_t type, so it is
> inconsistently named versus other VMA flags functions.
> 
> We only use vma_flag_set() in __mk_vma_flags() so we don't need to worry
> about its new name being rather cumbersome, so rename it to
> vma_flags_set_flag() to disambiguate it from vma_flags_set().
> 
> Also update the VMA test headers to reflect the changes.

Acked-by: David Hildenbrand (Arm) <david@kernel.org>

-- 
Cheers,

David