include/linux/cgroup-defs.h | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
Commit d23b5c577715 ("cgroup: Make operations on the cgroup root_list RCU
safe") adds a new rcu_head to the cgroup_root structure and kvfree_rcu()
for freeing the cgroup_root.
The current implementation of kvfree_rcu(), however, has the limitation
that the offset of the rcu_head structure within the larger data
structure must be less than 4096 or the compilation will fail. See the
macro definition of __is_kvfree_rcu_offset() in include/linux/rcupdate.h
for more information.
By putting rcu_head below the large cgroup structure, any change to the
cgroup structure that makes it larger run the risk of causing build
failure under certain configurations. Commit 77070eeb8821 ("cgroup:
Avoid false cacheline sharing of read mostly rstat_cpu") happens to be
the last straw that breaks it. Fix this problem by moving the rcu_head
structure up before the cgroup structure.
Fixes: d23b5c577715 ("cgroup: Make operations on the cgroup root_list RCU safe")
Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
Closes: https://lore.kernel.org/lkml/20231207143806.114e0a74@canb.auug.org.au/
Signed-off-by: Waiman Long <longman@redhat.com>
Acked-by: Yafang Shao <laoar.shao@gmail.com>
---
include/linux/cgroup-defs.h | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/include/linux/cgroup-defs.h b/include/linux/cgroup-defs.h
index 5a97ea95b564..ea48c861cd36 100644
--- a/include/linux/cgroup-defs.h
+++ b/include/linux/cgroup-defs.h
@@ -562,6 +562,10 @@ struct cgroup_root {
/* Unique id for this hierarchy. */
int hierarchy_id;
+ /* A list running through the active hierarchies */
+ struct list_head root_list;
+ struct rcu_head rcu; /* Must be near the top */
+
/*
* The root cgroup. The containing cgroup_root will be destroyed on its
* release. cgrp->ancestors[0] will be used overflowing into the
@@ -575,10 +579,6 @@ struct cgroup_root {
/* Number of cgroups in the hierarchy, used only for /proc/cgroups */
atomic_t nr_cgrps;
- /* A list running through the active hierarchies */
- struct list_head root_list;
- struct rcu_head rcu;
-
/* Hierarchy-specific flags */
unsigned int flags;
--
2.39.3On Thu, Dec 07, 2023 at 08:46:14AM -0500, Waiman Long wrote:
> Commit d23b5c577715 ("cgroup: Make operations on the cgroup root_list RCU
> safe") adds a new rcu_head to the cgroup_root structure and kvfree_rcu()
> for freeing the cgroup_root.
>
> The current implementation of kvfree_rcu(), however, has the limitation
> that the offset of the rcu_head structure within the larger data
> structure must be less than 4096 or the compilation will fail. See the
> macro definition of __is_kvfree_rcu_offset() in include/linux/rcupdate.h
> for more information.
>
> By putting rcu_head below the large cgroup structure, any change to the
> cgroup structure that makes it larger run the risk of causing build
> failure under certain configurations. Commit 77070eeb8821 ("cgroup:
> Avoid false cacheline sharing of read mostly rstat_cpu") happens to be
> the last straw that breaks it. Fix this problem by moving the rcu_head
> structure up before the cgroup structure.
>
> Fixes: d23b5c577715 ("cgroup: Make operations on the cgroup root_list RCU safe")
> Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
> Closes: https://lore.kernel.org/lkml/20231207143806.114e0a74@canb.auug.org.au/
> Signed-off-by: Waiman Long <longman@redhat.com>
> Acked-by: Yafang Shao <laoar.shao@gmail.com>
Applied to cgroup/for-6.8.
Thanks.
--
tejun
On Thu, Dec 7, 2023 at 5:46 AM Waiman Long <longman@redhat.com> wrote:
>
> Commit d23b5c577715 ("cgroup: Make operations on the cgroup root_list RCU
> safe") adds a new rcu_head to the cgroup_root structure and kvfree_rcu()
> for freeing the cgroup_root.
>
> The current implementation of kvfree_rcu(), however, has the limitation
> that the offset of the rcu_head structure within the larger data
> structure must be less than 4096 or the compilation will fail. See the
> macro definition of __is_kvfree_rcu_offset() in include/linux/rcupdate.h
> for more information.
>
> By putting rcu_head below the large cgroup structure, any change to the
> cgroup structure that makes it larger run the risk of causing build
> failure under certain configurations. Commit 77070eeb8821 ("cgroup:
> Avoid false cacheline sharing of read mostly rstat_cpu") happens to be
> the last straw that breaks it. Fix this problem by moving the rcu_head
> structure up before the cgroup structure.
>
> Fixes: d23b5c577715 ("cgroup: Make operations on the cgroup root_list RCU safe")
> Reported-by: Stephen Rothwell <sfr@canb.auug.org.au>
> Closes: https://lore.kernel.org/lkml/20231207143806.114e0a74@canb.auug.org.au/
> Signed-off-by: Waiman Long <longman@redhat.com>
> Acked-by: Yafang Shao <laoar.shao@gmail.com>
Reviewed-by: Yosry Ahmed <yosryahmed@google.com>
On Thu, Dec 07, 2023 at 08:46:14AM -0500, Waiman Long <longman@redhat.com> wrote:
> Commit 77070eeb8821 ("cgroup: Avoid false cacheline sharing of read
> mostly rstat_cpu") happens to be the last straw that breaks it.
FTR, when I build kernel from that commit, I can see
> struct cgroup_root {
> struct kernfs_root * kf_root; /* 0 8 */
> unsigned int subsys_mask; /* 8 4 */
> int hierarchy_id; /* 12 4 */
>
> /* XXX 48 bytes hole, try to pack */
>
> /* --- cacheline 1 boundary (64 bytes) --- */
> struct cgroup cgrp __attribute__((__aligned__(64))); /* 64 2368 */
>
> /* XXX last struct has 8 bytes of padding */
>
> /* --- cacheline 38 boundary (2432 bytes) --- */
> struct cgroup * cgrp_ancestor_storage; /* 2432 8 */
> atomic_t nr_cgrps; /* 2440 4 */
>
> /* XXX 4 bytes hole, try to pack */
>
> struct list_head root_list; /* 2448 16 */
> struct callback_head rcu __attribute__((__aligned__(8))); /* 2464 16 */
> unsigned int flags; /* 2480 4 */
> char release_agent_path[4096]; /* 2484 4096 */
> /* --- cacheline 102 boundary (6528 bytes) was 52 bytes ago --- */
> char name[64]; /* 6580 64 */
>
> /* size: 6656, cachelines: 104, members: 11 */
> /* sum members: 6592, holes: 2, sum holes: 52 */
> /* padding: 12 */
> /* paddings: 1, sum paddings: 8 */
> /* forced alignments: 2, forced holes: 1, sum forced holes: 48 */
> } __attribute__((__aligned__(64)));
2480 has still quite a reserve below 4096. (I can't see an CONFIG_*
affecting this.)
Perhaps, I missed something from the linux-next merging thread?
Michal
On 12/7/23 11:46, Michal Koutný wrote:
> On Thu, Dec 07, 2023 at 08:46:14AM -0500, Waiman Long <longman@redhat.com> wrote:
>> Commit 77070eeb8821 ("cgroup: Avoid false cacheline sharing of read
>> mostly rstat_cpu") happens to be the last straw that breaks it.
> FTR, when I build kernel from that commit, I can see
>
>> struct cgroup_root {
>> struct kernfs_root * kf_root; /* 0 8 */
>> unsigned int subsys_mask; /* 8 4 */
>> int hierarchy_id; /* 12 4 */
>>
>> /* XXX 48 bytes hole, try to pack */
>>
>> /* --- cacheline 1 boundary (64 bytes) --- */
>> struct cgroup cgrp __attribute__((__aligned__(64))); /* 64 2368 */
>>
>> /* XXX last struct has 8 bytes of padding */
>>
>> /* --- cacheline 38 boundary (2432 bytes) --- */
>> struct cgroup * cgrp_ancestor_storage; /* 2432 8 */
>> atomic_t nr_cgrps; /* 2440 4 */
>>
>> /* XXX 4 bytes hole, try to pack */
>>
>> struct list_head root_list; /* 2448 16 */
>> struct callback_head rcu __attribute__((__aligned__(8))); /* 2464 16 */
>> unsigned int flags; /* 2480 4 */
>> char release_agent_path[4096]; /* 2484 4096 */
>> /* --- cacheline 102 boundary (6528 bytes) was 52 bytes ago --- */
>> char name[64]; /* 6580 64 */
>>
>> /* size: 6656, cachelines: 104, members: 11 */
>> /* sum members: 6592, holes: 2, sum holes: 52 */
>> /* padding: 12 */
>> /* paddings: 1, sum paddings: 8 */
>> /* forced alignments: 2, forced holes: 1, sum forced holes: 48 */
>> } __attribute__((__aligned__(64)));
> 2480 has still quite a reserve below 4096. (I can't see an CONFIG_*
> affecting this.)
>
> Perhaps, I missed something from the linux-next merging thread?
CONFIG_LOCKDEP and some other debug configs are enabled with
allmodconfig. This can greatly increase the size of some of the
structures. I am not able to use pahole due to missing BTF info so I
don't the exact size. However, I can reproduce the build failure and the
patch is able to fix it.
Cheers,
Longman
On Thu, Dec 07, 2023 at 12:40:33PM -0500, Waiman Long <longman@redhat.com> wrote: > CONFIG_LOCKDEP and some other debug configs are enabled with allmodconfig. Interesting, I had CONFIG_LOCKDEP=y. I shared the numbers to rule out you're after something unrelated > However, I can reproduce the build failure and the patch is able to > fix it. That is a good proof :-) The patch makes sense for future robustness, Reviewed-by: Michal Koutný <mkoutny@suse.com>
© 2016 - 2025 Red Hat, Inc.