From nobody Sat Apr 18 03:32:14 2026 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8495E3590B3 for ; Wed, 11 Feb 2026 09:42:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770802963; cv=none; b=Yixqii1TNrlp6JtMS0Ajl0n4Ft0ZFvyOKPi5iTyc4cCZbxeoyo+uy4cIArXBE9Dub7txyMVpSX9slMNcta6WEc94xc7RcijvdEING/ddQdIKPg6Y8l8T+pVfdxP+MS1i0tQegqVySQEe2QzlYN5UCuVb80nU0hprz+KRIvZiPuM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1770802963; c=relaxed/simple; bh=HnOPKhUbGfasn9ycdnsjV/XKeYiQWLOzg9jjMCIDkOA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:To:Cc; b=O/bWLdeWWFY73OZtL4FgfyPpUKtX/62KWLZ1EYA7gZhYj1BudpjKBlrKaFEy0V38yujO0ffDmsXI6hop4yDQauC2i4JG4a08A7nvZsxXDsB9VqPENfufmr6TuMugpileYWTy6AiC/2adVgv0u9CqjhPtDj1ZWeGLfV4FtMfbAsk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz; spf=pass smtp.mailfrom=suse.cz; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=SVoaGoCC; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=ZetMEFKj; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=SVoaGoCC; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=ZetMEFKj; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.cz Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="SVoaGoCC"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="ZetMEFKj"; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="SVoaGoCC"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="ZetMEFKj" Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 170A25BCC6; Wed, 11 Feb 2026 09:42:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1770802960; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=K5Un/BwBCJb/Lo7qf21XKs3tKGaJwngf0MHuo+OeL0A=; b=SVoaGoCC5gDmQFRyWjA3PCKyu+t/Ois681D4Z9KrsVvsO92E7/NlXIvOlJVWQwApirMC7D m52UpX/ubppqr2vha2Y/gt++mHlKdVcSRxzatYjIPhRPvhuknUF4N47l1oFe7qi5PJCQLR UpBiD1z3cb7idw4z3FpmQYGx6iDR9sU= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1770802960; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=K5Un/BwBCJb/Lo7qf21XKs3tKGaJwngf0MHuo+OeL0A=; b=ZetMEFKjbfbgF/T7ZZvZ/oJ3QQ45vUMJ1CCDkAy7QJxlMCjmN03nOwXXPnsFDJMxyVO29l 4XT+7EaMHBoqVOBg== Authentication-Results: smtp-out2.suse.de; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=SVoaGoCC; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=ZetMEFKj DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1770802960; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=K5Un/BwBCJb/Lo7qf21XKs3tKGaJwngf0MHuo+OeL0A=; b=SVoaGoCC5gDmQFRyWjA3PCKyu+t/Ois681D4Z9KrsVvsO92E7/NlXIvOlJVWQwApirMC7D m52UpX/ubppqr2vha2Y/gt++mHlKdVcSRxzatYjIPhRPvhuknUF4N47l1oFe7qi5PJCQLR UpBiD1z3cb7idw4z3FpmQYGx6iDR9sU= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1770802960; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=K5Un/BwBCJb/Lo7qf21XKs3tKGaJwngf0MHuo+OeL0A=; b=ZetMEFKjbfbgF/T7ZZvZ/oJ3QQ45vUMJ1CCDkAy7QJxlMCjmN03nOwXXPnsFDJMxyVO29l 4XT+7EaMHBoqVOBg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id EC6BF3EA62; Wed, 11 Feb 2026 09:42:39 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 9OdiOQ9PjGn+GwAAD6G6ig (envelope-from ); Wed, 11 Feb 2026 09:42:39 +0000 From: Vlastimil Babka Date: Wed, 11 Feb 2026 10:42:30 +0100 Subject: [PATCH] slab: distinguish lock and trylock for sheaf_flush_main() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260211-b4-sheaf-flush-v1-1-4e7f492f0055@suse.cz> X-B4-Tracking: v=1; b=H4sIAAVPjGkC/x3MQQqAIBBA0avErBtQiciuEi3UxhyICocikO6et HyL/wsIZSaBsSmQ6WbhY6/QbQMhuX0l5KUajDK9Mlqj71ASuYhxuyShIW+ts2Hw0UONzkyRn38 4ze/7AXBfFEtgAAAA X-Change-ID: 20260211-b4-sheaf-flush-2eb99a9c8bfb To: Harry Yoo Cc: Hao Li , Marcelo Tosatti , Andrew Morton , Christoph Lameter , David Rientjes , Roman Gushchin , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Vlastimil Babka X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=2971; i=vbabka@suse.cz; h=from:subject:message-id; bh=HnOPKhUbGfasn9ycdnsjV/XKeYiQWLOzg9jjMCIDkOA=; b=owEBiQF2/pANAwAIAbvgsHXSRYiaAcsmYgBpjE8LmXCShpS1gNp1Z+8ihRo1+pG2xXlG70nQ1 7Gc/jjIj+uJAU8EAAEIADkWIQR7u8hBFZkjSJZITfG74LB10kWImgUCaYxPCxsUgAAAAAAEAA5t YW51MiwyLjUrMS4xMSwyLDIACgkQu+CwddJFiJoKYQf+LtgmATGmvJam7nTWPDf+z2jFqbzAmby /5qT5f7+Qv9OiURW1/5Lcrq8If5GpouLNDdgwGLnbzG9wRJJRKNbQOSGrOWdD8ie4sajKWo7cKZ 6jOYSwY47GS3QS7a0QOqcsmfE6mI51gBa/p2v7a7f7vGpBy+2QDHtyruBMvnRfEZYJVxTOi0+NV +mYaZUSlpLlxBKsX96GKD+0HG3FEKL47c8iBRXcIvxponMjTA7dHHT5w2dxh7F5Od1fybSdwBx+ ZD/OwCbt5ViI01MDd2ZJ0DsMlX0WiS1SWP2Ta20pOpprXkzucLd2sAtCV8VPMIQdZNv8vUKLVB3 +EvnyUnk1RQ== X-Developer-Key: i=vbabka@suse.cz; a=openpgp; fpr=A940D434992C2E8E99103D50224FA7E7CC82A664 X-Spam-Score: -4.51 X-Spamd-Result: default: False [-4.51 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[suse.cz:s=susede2_rsa,suse.cz:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RCVD_VIA_SMTP_AUTH(0.00)[]; ARC_NA(0.00)[]; RCPT_COUNT_SEVEN(0.00)[10]; MIME_TRACE(0.00)[0:+]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; DKIM_SIGNED(0.00)[suse.cz:s=susede2_rsa,suse.cz:s=susede2_ed25519]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.cz:dkim,suse.cz:mid,suse.cz:email,imap1.dmz-prg2.suse.org:rdns,imap1.dmz-prg2.suse.org:helo]; DKIM_TRACE(0.00)[suse.cz:+] X-Spam-Level: X-Rspamd-Action: no action X-Rspamd-Queue-Id: 170A25BCC6 X-Rspamd-Server: rspamd1.dmz-prg2.suse.org X-Spam-Flag: NO sheaf_flush_main() can be called from __pcs_replace_full_main() where the trylock can in theory fail, and pcs_flush_all() where it's not expected to and it would be actually a problem if it failed and left the main sheaf not flushed. To make this explicit, split the function into sheaf_flush_main() (using local_lock()) and sheaf_try_flush_main() (using local_trylock()) where both call __sheaf_flush_main_batch() to flush a single batch of objects. This will allow lockdep to verify our assumptions. Signed-off-by: Vlastimil Babka Reviewed-by: Hao Li Reviewed-by: Harry Yoo --- mm/slub.c | 47 +++++++++++++++++++++++++++++++++++++---------- 1 file changed, 37 insertions(+), 10 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index 18c30872d196..12912b29f5bb 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -2844,19 +2844,19 @@ static void __kmem_cache_free_bulk(struct kmem_cach= e *s, size_t size, void **p); * object pointers are moved to a on-stack array under the lock. To bound = the * stack usage, limit each batch to PCS_BATCH_MAX. * - * returns true if at least partially flushed + * Must be called with s->cpu_sheaves->lock locked, returns with the lock + * unlocked. + * + * Returns how many objects are remaining to be flushed */ -static bool sheaf_flush_main(struct kmem_cache *s) +static unsigned int __sheaf_flush_main_batch(struct kmem_cache *s) { struct slub_percpu_sheaves *pcs; unsigned int batch, remaining; void *objects[PCS_BATCH_MAX]; struct slab_sheaf *sheaf; - bool ret =3D false; =20 -next_batch: - if (!local_trylock(&s->cpu_sheaves->lock)) - return ret; + lockdep_assert_held(this_cpu_ptr(&s->cpu_sheaves->lock)); =20 pcs =3D this_cpu_ptr(s->cpu_sheaves); sheaf =3D pcs->main; @@ -2874,10 +2874,37 @@ static bool sheaf_flush_main(struct kmem_cache *s) =20 stat_add(s, SHEAF_FLUSH, batch); =20 - ret =3D true; + return remaining; +} =20 - if (remaining) - goto next_batch; +static void sheaf_flush_main(struct kmem_cache *s) +{ + unsigned int remaining; + + do { + local_lock(&s->cpu_sheaves->lock); + + remaining =3D __sheaf_flush_main_batch(s); + + } while (remaining); +} + +/* + * Returns true if the main sheaf was at least partially flushed. + */ +static bool sheaf_try_flush_main(struct kmem_cache *s) +{ + unsigned int remaining; + bool ret =3D false; + + do { + if (!local_trylock(&s->cpu_sheaves->lock)) + return ret; + + ret =3D true; + remaining =3D __sheaf_flush_main_batch(s); + + } while (remaining); =20 return ret; } @@ -5685,7 +5712,7 @@ __pcs_replace_full_main(struct kmem_cache *s, struct = slub_percpu_sheaves *pcs, if (put_fail) stat(s, BARN_PUT_FAIL); =20 - if (!sheaf_flush_main(s)) + if (!sheaf_try_flush_main(s)) return NULL; =20 if (!local_trylock(&s->cpu_sheaves->lock)) --- base-commit: 27125df9a5d3b4cfd03bce3a8ec405a368cc9aae change-id: 20260211-b4-sheaf-flush-2eb99a9c8bfb Best regards, --=20 Vlastimil Babka