From nobody Fri Oct 3 19:17:24 2025 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E37AE3375AF for ; Wed, 27 Aug 2025 08:26:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756283211; cv=none; b=VzMcJ7yly4pypAFfUU/hPjMWhClOM1KHNBE9b+L+ka+dXM/0OrkCLHkrRlew4SGHXs0VQeILL9TOZjKNrIoAhGT7jCGz6cewB4Zc5XkaVXPfjVKERsxaeJGu+c3b6V1I7roC9vXpjWOub/8AzIQRc1fvqjprd1iNF1Iz4dSMwLk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756283211; c=relaxed/simple; bh=3jzoc1tVl/nqut2ZAqI9yIzhP/0lFTqRMmN7y7IZvIQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ryvRkUz3OyplkFHm3FBm3YRhqBbd5TUL4LsLP9cDRTSIsPQC0g0EbUrMppHnMLnN3mEJHnQ5Boqto1jk44DmkAuokEa5oljsWUwLDEhA8sUobt3VDNNBZX/C+El3FaMJKke2j1o6E576Rb2U8tlL8E1tbknhdIYp1wowE4WCmQQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz; spf=pass smtp.mailfrom=suse.cz; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=Wn11fDoW; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=CHq4BEOJ; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=Wn11fDoW; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=CHq4BEOJ; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.cz Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="Wn11fDoW"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="CHq4BEOJ"; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="Wn11fDoW"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="CHq4BEOJ" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 4B1DB2209A; Wed, 27 Aug 2025 08:26:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1756283195; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yZDXjBw35P2yFQeVQZgP7f941t6WUXjXrrLFu/SNxls=; b=Wn11fDoWyb5lNgkV9KCo0k/lslEl7z1mRoVeMr7SVRMWnRoP8kfzmGCYoctfAPsmB/pMKX EFlgakrM82ktRujIWMwr7x+9ubZJ1bHU7C0zb3/4efuRQ4wcdmBiMKywNCkXZlij9qRKfW iYdV+oY4ArF4Ha0wnXonYZ0thch3tas= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1756283195; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yZDXjBw35P2yFQeVQZgP7f941t6WUXjXrrLFu/SNxls=; b=CHq4BEOJE41StmRy/f/PbhVq59Wt5PcKV00urFT65UM8fhNEpktnM/kmbX95SRh2PA1sqj 4KYHqOVmHEbfMgCA== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1756283195; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yZDXjBw35P2yFQeVQZgP7f941t6WUXjXrrLFu/SNxls=; b=Wn11fDoWyb5lNgkV9KCo0k/lslEl7z1mRoVeMr7SVRMWnRoP8kfzmGCYoctfAPsmB/pMKX EFlgakrM82ktRujIWMwr7x+9ubZJ1bHU7C0zb3/4efuRQ4wcdmBiMKywNCkXZlij9qRKfW iYdV+oY4ArF4Ha0wnXonYZ0thch3tas= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1756283195; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yZDXjBw35P2yFQeVQZgP7f941t6WUXjXrrLFu/SNxls=; b=CHq4BEOJE41StmRy/f/PbhVq59Wt5PcKV00urFT65UM8fhNEpktnM/kmbX95SRh2PA1sqj 4KYHqOVmHEbfMgCA== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 31E6313AA3; Wed, 27 Aug 2025 08:26:35 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id wK7pCzvBrmhNfgAAD6G6ig (envelope-from ); Wed, 27 Aug 2025 08:26:35 +0000 From: Vlastimil Babka Date: Wed, 27 Aug 2025 10:26:35 +0200 Subject: [PATCH v6 03/10] slab: add sheaf support for batching kfree_rcu() operations Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250827-slub-percpu-caches-v6-3-f0f775a3f73f@suse.cz> References: <20250827-slub-percpu-caches-v6-0-f0f775a3f73f@suse.cz> In-Reply-To: <20250827-slub-percpu-caches-v6-0-f0f775a3f73f@suse.cz> To: Suren Baghdasaryan , "Liam R. Howlett" , Christoph Lameter , David Rientjes Cc: Roman Gushchin , Harry Yoo , Uladzislau Rezki , linux-mm@kvack.org, linux-kernel@vger.kernel.org, rcu@vger.kernel.org, maple-tree@lists.infradead.org, vbabka@suse.cz X-Mailer: b4 0.14.2 X-Spam-Level: X-Spamd-Result: default: False [-8.30 / 50.00]; REPLY(-4.00)[]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RCVD_VIA_SMTP_AUTH(0.00)[]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RCPT_COUNT_TWELVE(0.00)[12]; ARC_NA(0.00)[]; RCVD_TLS_ALL(0.00)[]; FREEMAIL_ENVRCPT(0.00)[gmail.com]; R_RATELIMIT(0.00)[to_ip_from(RLwn5r54y1cp81no5tmbbew5oc)]; FROM_HAS_DN(0.00)[]; FREEMAIL_CC(0.00)[linux.dev,oracle.com,gmail.com,kvack.org,vger.kernel.org,lists.infradead.org,suse.cz]; MID_RHS_MATCH_FROM(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DKIM_SIGNED(0.00)[suse.cz:s=susede2_rsa,suse.cz:s=susede2_ed25519]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.cz:email,suse.cz:mid] X-Spam-Flag: NO X-Spam-Score: -8.30 Extend the sheaf infrastructure for more efficient kfree_rcu() handling. For caches with sheaves, on each cpu maintain a rcu_free sheaf in addition to main and spare sheaves. kfree_rcu() operations will try to put objects on this sheaf. Once full, the sheaf is detached and submitted to call_rcu() with a handler that will try to put it in the barn, or flush to slab pages using bulk free, when the barn is full. Then a new empty sheaf must be obtained to put more objects there. It's possible that no free sheaves are available to use for a new rcu_free sheaf, and the allocation in kfree_rcu() context can only use GFP_NOWAIT and thus may fail. In that case, fall back to the existing kfree_rcu() implementation. Expected advantages: - batching the kfree_rcu() operations, that could eventually replace the existing batching - sheaves can be reused for allocations via barn instead of being flushed to slabs, which is more efficient - this includes cases where only some cpus are allowed to process rcu callbacks (Android) Possible disadvantage: - objects might be waiting for more than their grace period (it is determined by the last object freed into the sheaf), increasing memory usage - but the existing batching does that too. Only implement this for CONFIG_KVFREE_RCU_BATCHED as the tiny implementation favors smaller memory footprint over performance. Add CONFIG_SLUB_STATS counters free_rcu_sheaf and free_rcu_sheaf_fail to count how many kfree_rcu() used the rcu_free sheaf successfully and how many had to fall back to the existing implementation. Reviewed-by: Harry Yoo Reviewed-by: Suren Baghdasaryan Signed-off-by: Vlastimil Babka --- mm/slab.h | 2 + mm/slab_common.c | 24 +++++++ mm/slub.c | 193 +++++++++++++++++++++++++++++++++++++++++++++++++++= ++-- 3 files changed, 214 insertions(+), 5 deletions(-) diff --git a/mm/slab.h b/mm/slab.h index 206987ce44a4d053ebe3b5e50784d2dd23822cd1..f1866f2d9b211bb0d7f24644b80= ef4b50a7c3d24 100644 --- a/mm/slab.h +++ b/mm/slab.h @@ -435,6 +435,8 @@ static inline bool is_kmalloc_normal(struct kmem_cache = *s) return !(s->flags & (SLAB_CACHE_DMA|SLAB_ACCOUNT|SLAB_RECLAIM_ACCOUNT)); } =20 +bool __kfree_rcu_sheaf(struct kmem_cache *s, void *obj); + #define SLAB_CORE_FLAGS (SLAB_HWCACHE_ALIGN | SLAB_CACHE_DMA | \ SLAB_CACHE_DMA32 | SLAB_PANIC | \ SLAB_TYPESAFE_BY_RCU | SLAB_DEBUG_OBJECTS | \ diff --git a/mm/slab_common.c b/mm/slab_common.c index e2b197e47866c30acdbd1fee4159f262a751c5a7..2d806e02568532a1000fd3912db= 6978e945dcfa8 100644 --- a/mm/slab_common.c +++ b/mm/slab_common.c @@ -1608,6 +1608,27 @@ static void kfree_rcu_work(struct work_struct *work) kvfree_rcu_list(head); } =20 +static bool kfree_rcu_sheaf(void *obj) +{ + struct kmem_cache *s; + struct folio *folio; + struct slab *slab; + + if (is_vmalloc_addr(obj)) + return false; + + folio =3D virt_to_folio(obj); + if (unlikely(!folio_test_slab(folio))) + return false; + + slab =3D folio_slab(folio); + s =3D slab->slab_cache; + if (s->cpu_sheaves) + return __kfree_rcu_sheaf(s, obj); + + return false; +} + static bool need_offload_krc(struct kfree_rcu_cpu *krcp) { @@ -1952,6 +1973,9 @@ void kvfree_call_rcu(struct rcu_head *head, void *ptr) if (!head) might_sleep(); =20 + if (kfree_rcu_sheaf(ptr)) + return; + // Queue the object but don't yet schedule the batch. if (debug_rcu_head_queue(ptr)) { // Probable double kfree_rcu(), just leak. diff --git a/mm/slub.c b/mm/slub.c index 0822a817c28c2c4666e853ef0f433842c64f607a..7492076cf8c388793c09a64496a= 3b8850ef0d8ec 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -367,6 +367,8 @@ enum stat_item { ALLOC_FASTPATH, /* Allocation from cpu slab */ ALLOC_SLOWPATH, /* Allocation by getting a new cpu slab */ FREE_PCS, /* Free to percpu sheaf */ + FREE_RCU_SHEAF, /* Free to rcu_free sheaf */ + FREE_RCU_SHEAF_FAIL, /* Failed to free to a rcu_free sheaf */ FREE_FASTPATH, /* Free to cpu slab */ FREE_SLOWPATH, /* Freeing not to cpu slab */ FREE_FROZEN, /* Freeing to frozen slab */ @@ -461,6 +463,7 @@ struct slab_sheaf { struct rcu_head rcu_head; struct list_head barn_list; }; + struct kmem_cache *cache; unsigned int size; void *objects[]; }; @@ -469,6 +472,7 @@ struct slub_percpu_sheaves { local_trylock_t lock; struct slab_sheaf *main; /* never NULL when unlocked */ struct slab_sheaf *spare; /* empty or full, may be NULL */ + struct slab_sheaf *rcu_free; /* for batching kfree_rcu() */ struct node_barn *barn; }; =20 @@ -2526,6 +2530,8 @@ static struct slab_sheaf *alloc_empty_sheaf(struct km= em_cache *s, gfp_t gfp) if (unlikely(!sheaf)) return NULL; =20 + sheaf->cache =3D s; + stat(s, SHEAF_ALLOC); =20 return sheaf; @@ -2650,6 +2656,43 @@ static void sheaf_flush_unused(struct kmem_cache *s,= struct slab_sheaf *sheaf) sheaf->size =3D 0; } =20 +static void __rcu_free_sheaf_prepare(struct kmem_cache *s, + struct slab_sheaf *sheaf) +{ + bool init =3D slab_want_init_on_free(s); + void **p =3D &sheaf->objects[0]; + unsigned int i =3D 0; + + while (i < sheaf->size) { + struct slab *slab =3D virt_to_slab(p[i]); + + memcg_slab_free_hook(s, slab, p + i, 1); + alloc_tagging_slab_free_hook(s, slab, p + i, 1); + + if (unlikely(!slab_free_hook(s, p[i], init, true))) { + p[i] =3D p[--sheaf->size]; + continue; + } + + i++; + } +} + +static void rcu_free_sheaf_nobarn(struct rcu_head *head) +{ + struct slab_sheaf *sheaf; + struct kmem_cache *s; + + sheaf =3D container_of(head, struct slab_sheaf, rcu_head); + s =3D sheaf->cache; + + __rcu_free_sheaf_prepare(s, sheaf); + + sheaf_flush_unused(s, sheaf); + + free_empty_sheaf(s, sheaf); +} + /* * Caller needs to make sure migration is disabled in order to fully flush * single cpu's sheaves @@ -2662,7 +2705,7 @@ static void sheaf_flush_unused(struct kmem_cache *s, = struct slab_sheaf *sheaf) static void pcs_flush_all(struct kmem_cache *s) { struct slub_percpu_sheaves *pcs; - struct slab_sheaf *spare; + struct slab_sheaf *spare, *rcu_free; =20 local_lock(&s->cpu_sheaves->lock); pcs =3D this_cpu_ptr(s->cpu_sheaves); @@ -2670,6 +2713,9 @@ static void pcs_flush_all(struct kmem_cache *s) spare =3D pcs->spare; pcs->spare =3D NULL; =20 + rcu_free =3D pcs->rcu_free; + pcs->rcu_free =3D NULL; + local_unlock(&s->cpu_sheaves->lock); =20 if (spare) { @@ -2677,6 +2723,9 @@ static void pcs_flush_all(struct kmem_cache *s) free_empty_sheaf(s, spare); } =20 + if (rcu_free) + call_rcu(&rcu_free->rcu_head, rcu_free_sheaf_nobarn); + sheaf_flush_main(s); } =20 @@ -2693,6 +2742,11 @@ static void __pcs_flush_all_cpu(struct kmem_cache *s= , unsigned int cpu) free_empty_sheaf(s, pcs->spare); pcs->spare =3D NULL; } + + if (pcs->rcu_free) { + call_rcu(&pcs->rcu_free->rcu_head, rcu_free_sheaf_nobarn); + pcs->rcu_free =3D NULL; + } } =20 static void pcs_destroy(struct kmem_cache *s) @@ -2718,6 +2772,7 @@ static void pcs_destroy(struct kmem_cache *s) */ =20 WARN_ON(pcs->spare); + WARN_ON(pcs->rcu_free); =20 if (!WARN_ON(pcs->main->size)) { free_empty_sheaf(s, pcs->main); @@ -3775,7 +3830,7 @@ static bool has_pcs_used(int cpu, struct kmem_cache *= s) =20 pcs =3D per_cpu_ptr(s->cpu_sheaves, cpu); =20 - return (pcs->spare || pcs->main->size); + return (pcs->spare || pcs->rcu_free || pcs->main->size); } =20 static void pcs_flush_all(struct kmem_cache *s); @@ -5401,6 +5456,127 @@ bool free_to_pcs(struct kmem_cache *s, void *object) return true; } =20 +static void rcu_free_sheaf(struct rcu_head *head) +{ + struct slab_sheaf *sheaf; + struct node_barn *barn; + struct kmem_cache *s; + + sheaf =3D container_of(head, struct slab_sheaf, rcu_head); + + s =3D sheaf->cache; + + /* + * This may remove some objects due to slab_free_hook() returning false, + * so that the sheaf might no longer be completely full. But it's easier + * to handle it as full (unless it became completely empty), as the code + * handles it fine. The only downside is that sheaf will serve fewer + * allocations when reused. It only happens due to debugging, which is a + * performance hit anyway. + */ + __rcu_free_sheaf_prepare(s, sheaf); + + barn =3D get_node(s, numa_mem_id())->barn; + + /* due to slab_free_hook() */ + if (unlikely(sheaf->size =3D=3D 0)) + goto empty; + + /* + * Checking nr_full/nr_empty outside lock avoids contention in case the + * barn is at the respective limit. Due to the race we might go over the + * limit but that should be rare and harmless. + */ + + if (data_race(barn->nr_full) < MAX_FULL_SHEAVES) { + stat(s, BARN_PUT); + barn_put_full_sheaf(barn, sheaf); + return; + } + + stat(s, BARN_PUT_FAIL); + sheaf_flush_unused(s, sheaf); + +empty: + if (data_race(barn->nr_empty) < MAX_EMPTY_SHEAVES) { + barn_put_empty_sheaf(barn, sheaf); + return; + } + + free_empty_sheaf(s, sheaf); +} + +bool __kfree_rcu_sheaf(struct kmem_cache *s, void *obj) +{ + struct slub_percpu_sheaves *pcs; + struct slab_sheaf *rcu_sheaf; + + if (!local_trylock(&s->cpu_sheaves->lock)) + goto fail; + + pcs =3D this_cpu_ptr(s->cpu_sheaves); + + if (unlikely(!pcs->rcu_free)) { + + struct slab_sheaf *empty; + + if (pcs->spare && pcs->spare->size =3D=3D 0) { + pcs->rcu_free =3D pcs->spare; + pcs->spare =3D NULL; + goto do_free; + } + + empty =3D barn_get_empty_sheaf(pcs->barn); + + if (empty) { + pcs->rcu_free =3D empty; + goto do_free; + } + + local_unlock(&s->cpu_sheaves->lock); + + empty =3D alloc_empty_sheaf(s, GFP_NOWAIT); + + if (!empty) + goto fail; + + if (!local_trylock(&s->cpu_sheaves->lock)) { + barn_put_empty_sheaf(pcs->barn, empty); + goto fail; + } + + pcs =3D this_cpu_ptr(s->cpu_sheaves); + + if (unlikely(pcs->rcu_free)) + barn_put_empty_sheaf(pcs->barn, empty); + else + pcs->rcu_free =3D empty; + } + +do_free: + + rcu_sheaf =3D pcs->rcu_free; + + rcu_sheaf->objects[rcu_sheaf->size++] =3D obj; + + if (likely(rcu_sheaf->size < s->sheaf_capacity)) + rcu_sheaf =3D NULL; + else + pcs->rcu_free =3D NULL; + + local_unlock(&s->cpu_sheaves->lock); + + if (rcu_sheaf) + call_rcu(&rcu_sheaf->rcu_head, rcu_free_sheaf); + + stat(s, FREE_RCU_SHEAF); + return true; + +fail: + stat(s, FREE_RCU_SHEAF_FAIL); + return false; +} + /* * Bulk free objects to the percpu sheaves. * Unlike free_to_pcs() this includes the calls to all necessary hooks @@ -5410,10 +5586,8 @@ static void free_to_pcs_bulk(struct kmem_cache *s, s= ize_t size, void **p) { struct slub_percpu_sheaves *pcs; struct slab_sheaf *main, *empty; + bool init =3D slab_want_init_on_free(s); unsigned int batch, i =3D 0; - bool init; - - init =3D slab_want_init_on_free(s); =20 while (i < size) { struct slab *slab =3D virt_to_slab(p[i]); @@ -6900,6 +7074,11 @@ int __kmem_cache_shutdown(struct kmem_cache *s) struct kmem_cache_node *n; =20 flush_all_cpus_locked(s); + + /* we might have rcu sheaves in flight */ + if (s->cpu_sheaves) + rcu_barrier(); + /* Attempt to free all objects */ for_each_kmem_cache_node(s, node, n) { if (n->barn) @@ -8275,6 +8454,8 @@ STAT_ATTR(ALLOC_PCS, alloc_cpu_sheaf); STAT_ATTR(ALLOC_FASTPATH, alloc_fastpath); STAT_ATTR(ALLOC_SLOWPATH, alloc_slowpath); STAT_ATTR(FREE_PCS, free_cpu_sheaf); +STAT_ATTR(FREE_RCU_SHEAF, free_rcu_sheaf); +STAT_ATTR(FREE_RCU_SHEAF_FAIL, free_rcu_sheaf_fail); STAT_ATTR(FREE_FASTPATH, free_fastpath); STAT_ATTR(FREE_SLOWPATH, free_slowpath); STAT_ATTR(FREE_FROZEN, free_frozen); @@ -8373,6 +8554,8 @@ static struct attribute *slab_attrs[] =3D { &alloc_fastpath_attr.attr, &alloc_slowpath_attr.attr, &free_cpu_sheaf_attr.attr, + &free_rcu_sheaf_attr.attr, + &free_rcu_sheaf_fail_attr.attr, &free_fastpath_attr.attr, &free_slowpath_attr.attr, &free_frozen_attr.attr, --=20 2.51.0