From nobody Sun Dec 14 06:23:25 2025 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 17AFC2FABF9 for ; Mon, 1 Sep 2025 11:08:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756724937; cv=none; b=F1Sd29wXcH3GZOeHDjJwu1p2HPuUW3scoBXDQNj5vOMIwtxG4yVKwOk64GaM0qFYrtHDGURMp68GaI4H1wByeS/DW6H+R9GHv3frqR2qq5mv+gy/YoQRpUwDgxFCH6vBGKTwHe9KLHreR+oCoU5RJ2+HrX4EiD1z7VZyywHewK4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756724937; c=relaxed/simple; bh=k7o7Y+ypSzp3fzPsE3Nbit72mtC/H91CKcAZSwdt9Sc=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=oNysYNgZmdkLv/mEmvrZWolSyPQixL2Ysz1747MeS+1gRw1r5CYTBBqBmEhxWwp7n4c5r4kLOdZnzEB7JbNDhILxubjhxFQ0F7y+Wm5jKluyFqc3bMZ0LeFzwf+D8vGMevm0LwmKSHj56Y+DZA4ZxUorANIDB9UGECGp15tDb5s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz; spf=pass smtp.mailfrom=suse.cz; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=I+plZ7C/; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=J8/V7o8t; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=I+plZ7C/; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=J8/V7o8t; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.cz Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="I+plZ7C/"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="J8/V7o8t"; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="I+plZ7C/"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="J8/V7o8t" Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 383BC2116F; Mon, 1 Sep 2025 11:08:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=feUOntQoitUe7mPlmKZms9YiTw8kD3bBDWarz0bQ+B8=; b=I+plZ7C/i7j6SSn1V9IG+u3eTFEI5fgncxBLiuSBgfdbPBCnbjQakhASgZmVp8xFeOyarH llVba0qnmEKgXbsyKEGI5gM5xAqIPvyXpeatWEj8xf7d46rgiu0pi+i+GduQfNcd24JWqG dCnvXrVGi8AwAG0SwMa2LF3u3rIbPdE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=feUOntQoitUe7mPlmKZms9YiTw8kD3bBDWarz0bQ+B8=; b=J8/V7o8t0IKnVXn2OPDaVq7LyzGdfd6XpoMhRzdwYsDgYMgSe00do2GgU6OP8MHnrMUlq5 5ZnHlczdBI4bT9AQ== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b="I+plZ7C/"; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b="J8/V7o8t" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=feUOntQoitUe7mPlmKZms9YiTw8kD3bBDWarz0bQ+B8=; b=I+plZ7C/i7j6SSn1V9IG+u3eTFEI5fgncxBLiuSBgfdbPBCnbjQakhASgZmVp8xFeOyarH llVba0qnmEKgXbsyKEGI5gM5xAqIPvyXpeatWEj8xf7d46rgiu0pi+i+GduQfNcd24JWqG dCnvXrVGi8AwAG0SwMa2LF3u3rIbPdE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=feUOntQoitUe7mPlmKZms9YiTw8kD3bBDWarz0bQ+B8=; b=J8/V7o8t0IKnVXn2OPDaVq7LyzGdfd6XpoMhRzdwYsDgYMgSe00do2GgU6OP8MHnrMUlq5 5ZnHlczdBI4bT9AQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 1CCCD13A31; Mon, 1 Sep 2025 11:08:53 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id kHi+BsV+tWjtDgAAD6G6ig (envelope-from ); Mon, 01 Sep 2025 11:08:53 +0000 From: Vlastimil Babka Date: Mon, 01 Sep 2025 13:08:51 +0200 Subject: [PATCH 01/12] maple_tree: Fix check_bulk_rebalance() test locks Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250901-maple-sheaves-v1-1-d6a1166b53f2@suse.cz> References: <20250901-maple-sheaves-v1-0-d6a1166b53f2@suse.cz> In-Reply-To: <20250901-maple-sheaves-v1-0-d6a1166b53f2@suse.cz> To: "Liam R. Howlett" , Matthew Wilcox , Lorenzo Stoakes , Jann Horn , Pedro Falcato , Suren Baghdasaryan Cc: Harry Yoo , Andrew Morton , maple-tree@lists.infradead.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, Vlastimil Babka X-Mailer: b4 0.14.2 X-Spam-Level: X-Spam-Flag: NO X-Rspamd-Queue-Id: 383BC2116F X-Rspamd-Action: no action X-Rspamd-Server: rspamd1.dmz-prg2.suse.org X-Spamd-Result: default: False [-4.51 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[suse.cz:s=susede2_rsa,suse.cz:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; RCPT_COUNT_TWELVE(0.00)[13]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_SIGNED(0.00)[suse.cz:s=susede2_rsa,suse.cz:s=susede2_ed25519]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; R_RATELIMIT(0.00)[to_ip_from(RL5jz3zk9nm44ai14dcppf93zb)]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.cz:dkim,suse.cz:mid,suse.cz:email,oracle.com:email,imap1.dmz-prg2.suse.org:rdns,imap1.dmz-prg2.suse.org:helo]; DKIM_TRACE(0.00)[suse.cz:+] X-Spam-Score: -4.51 From: "Liam R. Howlett" The check_bulk_rebalance() test was not correctly locking the tree which caused issues with the sheaves testing in later patches. Adding the missing locks fixed the issue. Signed-off-by: Liam R. Howlett Signed-off-by: Vlastimil Babka Reviewed-by: Sidhartha Kumar --- tools/testing/radix-tree/maple.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tools/testing/radix-tree/maple.c b/tools/testing/radix-tree/ma= ple.c index 172700fb7784d29f9403003b4484a5ebd7aa316b..159d5307b30a4b37e6cf2941848= b8718e1b891d9 100644 --- a/tools/testing/radix-tree/maple.c +++ b/tools/testing/radix-tree/maple.c @@ -36465,6 +36465,7 @@ static inline void check_bulk_rebalance(struct mapl= e_tree *mt) =20 build_full_tree(mt, 0, 2); =20 + mas_lock(&mas); /* erase every entry in the tree */ do { /* set up bulk store mode */ @@ -36474,6 +36475,7 @@ static inline void check_bulk_rebalance(struct mapl= e_tree *mt) } while (mas_prev(&mas, 0) !=3D NULL); =20 mas_destroy(&mas); + mas_unlock(&mas); } =20 void farmer_tests(void) --=20 2.51.0 From nobody Sun Dec 14 06:23:25 2025 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CCFD3310652 for ; Mon, 1 Sep 2025 11:09:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756724947; cv=none; b=fW3WWIzflipG8YFAEZoCsAP8Toio5gwrr/9jAtbMsy6fsXBw/yuQ+p7o+irSGen4tHSOYIPBp/ejzuaFxFYddGmvj/a+hU0aDPKrci91JKhzChZsyqoF/VV6h9OWGtQRRCBWPsbacA6+6mBe8iLM5W5P5tkkxNp9V9dLCFQzGvs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756724947; c=relaxed/simple; bh=XbOoID+Ep0MW58l++b+mu3jOxsgzwD6mNlO9jVsuOMU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=qf23UTZIb46HHmbESHENe9Jak51WANiKBXuBEIvTk5P3xLqmT5nG/EroZYccVFcVDTrb7YUM0uCQJfm+s+tmF4DWEoIfNh2Dtk1q4eYHasfPqJdT8on6ZsLtjXMl19PJO+aKo7Xfc5KDA7/sGXdWpOpTDcXfNhXgwCbyYLdk5nI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz; spf=pass smtp.mailfrom=suse.cz; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=PVZniRDU; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=mNw4yh6E; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=PVZniRDU; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=mNw4yh6E; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.cz Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="PVZniRDU"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="mNw4yh6E"; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="PVZniRDU"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="mNw4yh6E" Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 513AE1F388; Mon, 1 Sep 2025 11:08:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5j5fQlGn0LPWPBdKT0pIiyIhFFb2vklTM3tMRG9lg9Q=; b=PVZniRDU9YIsk3d628yJbZaolIEDlI0jypXglbbnmSBtZjPlSL5XPliyTp5GOh/N90t9rj my1TMACjoFK3dxKTPUO72yo3ulVeWw0AuBXuPcXOqT8RrIzFi0jpBbFB5cDjBVSIkdY0lh Kmg25I+7cDxrUWhd7+3RoFzvvVUyir8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5j5fQlGn0LPWPBdKT0pIiyIhFFb2vklTM3tMRG9lg9Q=; b=mNw4yh6EqE8Pv3Svnai5eEjVI3q6e4GghUize9gqB3v2thZp+HqgWCO7j3tzUDpxHBidlO jgQGcOtvGdHfn4Bw== Authentication-Results: smtp-out2.suse.de; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=PVZniRDU; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=mNw4yh6E DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5j5fQlGn0LPWPBdKT0pIiyIhFFb2vklTM3tMRG9lg9Q=; b=PVZniRDU9YIsk3d628yJbZaolIEDlI0jypXglbbnmSBtZjPlSL5XPliyTp5GOh/N90t9rj my1TMACjoFK3dxKTPUO72yo3ulVeWw0AuBXuPcXOqT8RrIzFi0jpBbFB5cDjBVSIkdY0lh Kmg25I+7cDxrUWhd7+3RoFzvvVUyir8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5j5fQlGn0LPWPBdKT0pIiyIhFFb2vklTM3tMRG9lg9Q=; b=mNw4yh6EqE8Pv3Svnai5eEjVI3q6e4GghUize9gqB3v2thZp+HqgWCO7j3tzUDpxHBidlO jgQGcOtvGdHfn4Bw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 334E113A3E; Mon, 1 Sep 2025 11:08:53 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id mC43DMV+tWjtDgAAD6G6ig (envelope-from ); Mon, 01 Sep 2025 11:08:53 +0000 From: Vlastimil Babka Date: Mon, 01 Sep 2025 13:08:52 +0200 Subject: [PATCH 02/12] tools: Add testing support for changes to slab for sheaves Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250901-maple-sheaves-v1-2-d6a1166b53f2@suse.cz> References: <20250901-maple-sheaves-v1-0-d6a1166b53f2@suse.cz> In-Reply-To: <20250901-maple-sheaves-v1-0-d6a1166b53f2@suse.cz> To: "Liam R. Howlett" , Matthew Wilcox , Lorenzo Stoakes , Jann Horn , Pedro Falcato , Suren Baghdasaryan Cc: Harry Yoo , Andrew Morton , maple-tree@lists.infradead.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, Vlastimil Babka , "Liam R. Howlett" X-Mailer: b4 0.14.2 X-Spamd-Result: default: False [-4.51 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[suse.cz:s=susede2_rsa,suse.cz:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; RCPT_COUNT_TWELVE(0.00)[14]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_SIGNED(0.00)[suse.cz:s=susede2_rsa,suse.cz:s=susede2_ed25519]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; R_RATELIMIT(0.00)[to_ip_from(RL5jz3zk9nm44ai14dcppf93zb)]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[oracle.com:email,imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,suse.cz:mid,suse.cz:dkim,suse.cz:email]; DKIM_TRACE(0.00)[suse.cz:+] X-Spam-Flag: NO X-Spam-Level: X-Rspamd-Queue-Id: 513AE1F388 X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Action: no action X-Spam-Score: -4.51 From: "Liam R. Howlett" The slab changes for sheaves requires more effort in the testing code. Unite all the kmem_cache work into the tools/include slab header for both the vma and maple tree testing. The vma test code also requires importing more #defines to allow for seamless use of the shared kmem_cache code. This adds the pthread header to the slab header in the tools directory to allow for the pthread_mutex in linux.c. Signed-off-by: Liam R. Howlett Signed-off-by: Vlastimil Babka --- tools/include/linux/slab.h | 137 ++++++++++++++++++++++++++++++++++= ++-- tools/testing/shared/linux.c | 26 ++------ tools/testing/shared/maple-shim.c | 1 + tools/testing/vma/vma_internal.h | 94 +------------------------- 4 files changed, 142 insertions(+), 116 deletions(-) diff --git a/tools/include/linux/slab.h b/tools/include/linux/slab.h index c87051e2b26f5a7fee0362697fae067076b8e84d..c5c5cc6db5668be2cc94c29065c= cfa7ca7b4bb08 100644 --- a/tools/include/linux/slab.h +++ b/tools/include/linux/slab.h @@ -4,11 +4,31 @@ =20 #include #include +#include =20 -#define SLAB_PANIC 2 #define SLAB_RECLAIM_ACCOUNT 0x00020000UL /* Objects are rec= laimable */ =20 #define kzalloc_node(size, flags, node) kmalloc(size, flags) +enum _slab_flag_bits { + _SLAB_KMALLOC, + _SLAB_HWCACHE_ALIGN, + _SLAB_PANIC, + _SLAB_TYPESAFE_BY_RCU, + _SLAB_ACCOUNT, + _SLAB_FLAGS_LAST_BIT +}; + +#define __SLAB_FLAG_BIT(nr) ((unsigned int __force)(1U << (nr))) +#define __SLAB_FLAG_UNUSED ((unsigned int __force)(0U)) + +#define SLAB_HWCACHE_ALIGN __SLAB_FLAG_BIT(_SLAB_HWCACHE_ALIGN) +#define SLAB_PANIC __SLAB_FLAG_BIT(_SLAB_PANIC) +#define SLAB_TYPESAFE_BY_RCU __SLAB_FLAG_BIT(_SLAB_TYPESAFE_BY_RCU) +#ifdef CONFIG_MEMCG +# define SLAB_ACCOUNT __SLAB_FLAG_BIT(_SLAB_ACCOUNT) +#else +# define SLAB_ACCOUNT __SLAB_FLAG_UNUSED +#endif =20 void *kmalloc(size_t size, gfp_t gfp); void kfree(void *p); @@ -23,6 +43,86 @@ enum slab_state { FULL }; =20 +struct kmem_cache { + pthread_mutex_t lock; + unsigned int size; + unsigned int align; + unsigned int sheaf_capacity; + int nr_objs; + void *objs; + void (*ctor)(void *); + bool non_kernel_enabled; + unsigned int non_kernel; + unsigned long nr_allocated; + unsigned long nr_tallocated; + bool exec_callback; + void (*callback)(void *); + void *private; +}; + +struct kmem_cache_args { + /** + * @align: The required alignment for the objects. + * + * %0 means no specific alignment is requested. + */ + unsigned int align; + /** + * @sheaf_capacity: The maximum size of the sheaf. + */ + unsigned int sheaf_capacity; + /** + * @useroffset: Usercopy region offset. + * + * %0 is a valid offset, when @usersize is non-%0 + */ + unsigned int useroffset; + /** + * @usersize: Usercopy region size. + * + * %0 means no usercopy region is specified. + */ + unsigned int usersize; + /** + * @freeptr_offset: Custom offset for the free pointer + * in &SLAB_TYPESAFE_BY_RCU caches + * + * By default &SLAB_TYPESAFE_BY_RCU caches place the free pointer + * outside of the object. This might cause the object to grow in size. + * Cache creators that have a reason to avoid this can specify a custom + * free pointer offset in their struct where the free pointer will be + * placed. + * + * Note that placing the free pointer inside the object requires the + * caller to ensure that no fields are invalidated that are required to + * guard against object recycling (See &SLAB_TYPESAFE_BY_RCU for + * details). + * + * Using %0 as a value for @freeptr_offset is valid. If @freeptr_offset + * is specified, %use_freeptr_offset must be set %true. + * + * Note that @ctor currently isn't supported with custom free pointers + * as a @ctor requires an external free pointer. + */ + unsigned int freeptr_offset; + /** + * @use_freeptr_offset: Whether a @freeptr_offset is used. + */ + bool use_freeptr_offset; + /** + * @ctor: A constructor for the objects. + * + * The constructor is invoked for each object in a newly allocated slab + * page. It is the cache user's responsibility to free object in the + * same state as after calling the constructor, or deal appropriately + * with any differences between a freshly constructed and a reallocated + * object. + * + * %NULL means no constructor. + */ + void (*ctor)(void *); +}; + static inline void *kzalloc(size_t size, gfp_t gfp) { return kmalloc(size, gfp | __GFP_ZERO); @@ -37,9 +137,38 @@ static inline void *kmem_cache_alloc(struct kmem_cache = *cachep, int flags) } void kmem_cache_free(struct kmem_cache *cachep, void *objp); =20 -struct kmem_cache *kmem_cache_create(const char *name, unsigned int size, - unsigned int align, unsigned int flags, - void (*ctor)(void *)); + +struct kmem_cache * +__kmem_cache_create_args(const char *name, unsigned int size, + struct kmem_cache_args *args, unsigned int flags); + +/* If NULL is passed for @args, use this variant with default arguments. */ +static inline struct kmem_cache * +__kmem_cache_default_args(const char *name, unsigned int size, + struct kmem_cache_args *args, unsigned int flags) +{ + struct kmem_cache_args kmem_default_args =3D {}; + + return __kmem_cache_create_args(name, size, &kmem_default_args, flags); +} + +static inline struct kmem_cache * +__kmem_cache_create(const char *name, unsigned int size, unsigned int alig= n, + unsigned int flags, void (*ctor)(void *)) +{ + struct kmem_cache_args kmem_args =3D { + .align =3D align, + .ctor =3D ctor, + }; + + return __kmem_cache_create_args(name, size, &kmem_args, flags); +} + +#define kmem_cache_create(__name, __object_size, __args, ...) \ + _Generic((__args), \ + struct kmem_cache_args *: __kmem_cache_create_args, \ + void *: __kmem_cache_default_args, \ + default: __kmem_cache_create)(__name, __object_size, __args, __VA_ARGS__) =20 void kmem_cache_free_bulk(struct kmem_cache *cachep, size_t size, void **l= ist); int kmem_cache_alloc_bulk(struct kmem_cache *cachep, gfp_t gfp, size_t siz= e, diff --git a/tools/testing/shared/linux.c b/tools/testing/shared/linux.c index 0f97fb0d19e19c327aa4843a35b45cc086f4f366..97b8412ccbb6d222604c7b397c5= 3c65618d8d51b 100644 --- a/tools/testing/shared/linux.c +++ b/tools/testing/shared/linux.c @@ -16,21 +16,6 @@ int nr_allocated; int preempt_count; int test_verbose; =20 -struct kmem_cache { - pthread_mutex_t lock; - unsigned int size; - unsigned int align; - int nr_objs; - void *objs; - void (*ctor)(void *); - unsigned int non_kernel; - unsigned long nr_allocated; - unsigned long nr_tallocated; - bool exec_callback; - void (*callback)(void *); - void *private; -}; - void kmem_cache_set_callback(struct kmem_cache *cachep, void (*callback)(v= oid *)) { cachep->callback =3D callback; @@ -234,23 +219,26 @@ int kmem_cache_alloc_bulk(struct kmem_cache *cachep, = gfp_t gfp, size_t size, } =20 struct kmem_cache * -kmem_cache_create(const char *name, unsigned int size, unsigned int align, - unsigned int flags, void (*ctor)(void *)) +__kmem_cache_create_args(const char *name, unsigned int size, + struct kmem_cache_args *args, + unsigned int flags) { struct kmem_cache *ret =3D malloc(sizeof(*ret)); =20 pthread_mutex_init(&ret->lock, NULL); ret->size =3D size; - ret->align =3D align; + ret->align =3D args->align; + ret->sheaf_capacity =3D args->sheaf_capacity; ret->nr_objs =3D 0; ret->nr_allocated =3D 0; ret->nr_tallocated =3D 0; ret->objs =3D NULL; - ret->ctor =3D ctor; + ret->ctor =3D args->ctor; ret->non_kernel =3D 0; ret->exec_callback =3D false; ret->callback =3D NULL; ret->private =3D NULL; + return ret; } =20 diff --git a/tools/testing/shared/maple-shim.c b/tools/testing/shared/maple= -shim.c index 640df76f483e09f3b6f85612786060dd273e2362..9d7b743415660305416e972fa75= b56824211b0eb 100644 --- a/tools/testing/shared/maple-shim.c +++ b/tools/testing/shared/maple-shim.c @@ -3,5 +3,6 @@ /* Very simple shim around the maple tree. */ =20 #include "maple-shared.h" +#include =20 #include "../../../lib/maple_tree.c" diff --git a/tools/testing/vma/vma_internal.h b/tools/testing/vma/vma_inter= nal.h index 3639aa8dd2b06ebe5b9cfcfe6669994fd38c482d..09732eff8dd84555563b3d48580= 5ebab7b204584 100644 --- a/tools/testing/vma/vma_internal.h +++ b/tools/testing/vma/vma_internal.h @@ -26,6 +26,7 @@ #include #include #include +#include =20 extern unsigned long stack_guard_gap; #ifdef CONFIG_MMU @@ -509,65 +510,6 @@ struct pagetable_move_control { .len_in =3D len_, \ } =20 -struct kmem_cache_args { - /** - * @align: The required alignment for the objects. - * - * %0 means no specific alignment is requested. - */ - unsigned int align; - /** - * @useroffset: Usercopy region offset. - * - * %0 is a valid offset, when @usersize is non-%0 - */ - unsigned int useroffset; - /** - * @usersize: Usercopy region size. - * - * %0 means no usercopy region is specified. - */ - unsigned int usersize; - /** - * @freeptr_offset: Custom offset for the free pointer - * in &SLAB_TYPESAFE_BY_RCU caches - * - * By default &SLAB_TYPESAFE_BY_RCU caches place the free pointer - * outside of the object. This might cause the object to grow in size. - * Cache creators that have a reason to avoid this can specify a custom - * free pointer offset in their struct where the free pointer will be - * placed. - * - * Note that placing the free pointer inside the object requires the - * caller to ensure that no fields are invalidated that are required to - * guard against object recycling (See &SLAB_TYPESAFE_BY_RCU for - * details). - * - * Using %0 as a value for @freeptr_offset is valid. If @freeptr_offset - * is specified, %use_freeptr_offset must be set %true. - * - * Note that @ctor currently isn't supported with custom free pointers - * as a @ctor requires an external free pointer. - */ - unsigned int freeptr_offset; - /** - * @use_freeptr_offset: Whether a @freeptr_offset is used. - */ - bool use_freeptr_offset; - /** - * @ctor: A constructor for the objects. - * - * The constructor is invoked for each object in a newly allocated slab - * page. It is the cache user's responsibility to free object in the - * same state as after calling the constructor, or deal appropriately - * with any differences between a freshly constructed and a reallocated - * object. - * - * %NULL means no constructor. - */ - void (*ctor)(void *); -}; - static inline void vma_iter_invalidate(struct vma_iterator *vmi) { mas_pause(&vmi->mas); @@ -652,40 +594,6 @@ static inline void vma_init(struct vm_area_struct *vma= , struct mm_struct *mm) vma->vm_lock_seq =3D UINT_MAX; } =20 -struct kmem_cache { - const char *name; - size_t object_size; - struct kmem_cache_args *args; -}; - -static inline struct kmem_cache *__kmem_cache_create(const char *name, - size_t object_size, - struct kmem_cache_args *args) -{ - struct kmem_cache *ret =3D malloc(sizeof(struct kmem_cache)); - - ret->name =3D name; - ret->object_size =3D object_size; - ret->args =3D args; - - return ret; -} - -#define kmem_cache_create(__name, __object_size, __args, ...) \ - __kmem_cache_create((__name), (__object_size), (__args)) - -static inline void *kmem_cache_alloc(struct kmem_cache *s, gfp_t gfpflags) -{ - (void)gfpflags; - - return calloc(s->object_size, 1); -} - -static inline void kmem_cache_free(struct kmem_cache *s, void *x) -{ - free(x); -} - /* * These are defined in vma.h, but sadly vm_stat_account() is referenced by * kernel/fork.c, so we have to these broadly available there, and tempora= rily --=20 2.51.0 From nobody Sun Dec 14 06:23:25 2025 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 71FC83126C5 for ; Mon, 1 Sep 2025 11:09:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756724943; cv=none; b=N0JunEI7UN5ISACRR3FeGKsYasc+VrDuzp+DC4yvmlZ39Yqx8EyjDrACi8dGF7c3GOzpxf3NUcnQkj0jyfxBifVIKwr+UQsTybMh9QrVzYF91i9JLhuyv+T25LceRniyYcP3635kjMftjwJpPafGLFZu3eJto2IVCteYusg3Huc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756724943; c=relaxed/simple; bh=UlGxvACdv155XrFFrWk0ecsws481oUU4TIOFXFk1vIg=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=dUZFRdSe7HIpuwBrW82I0C5Q/eXcYG6fFMGdYBH4mAhxv7S4Nyhj6zzAUgE6dBloLWkXE1gg3qeFASrwz1qLoH6ZwbiN5Opu+dudFNYLSdE66bN1t4x0cvyIgLgtU+v7kTs+WGbo6wzNn1SbTMwDPZySs2ZOiuYc+K4eVEfn47s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz; spf=pass smtp.mailfrom=suse.cz; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=JXEC2JSP; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=tLWW5Zdr; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=JXEC2JSP; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=tLWW5Zdr; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.cz Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="JXEC2JSP"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="tLWW5Zdr"; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="JXEC2JSP"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="tLWW5Zdr" Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 6468321170; Mon, 1 Sep 2025 11:08:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0KRBIhNrsFaoYAcvZ0YP3W0XTTeCHuYd9glX8HrySn4=; b=JXEC2JSPbAhghivVpT57oMvEpSklUDeXYRyjnFw0yUL2Adv/0jqVqXd2LWyEmhUOcI4tih ZMQbFVXtHX0pF+8GRqc3eqZxtij3l+T7pVjwLQz/NMQHpjXgV+cCY03UrogQ1HnMuCbGT5 /5h+ITU1KJDdjzwLJBK7VPpbCCmOYSE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0KRBIhNrsFaoYAcvZ0YP3W0XTTeCHuYd9glX8HrySn4=; b=tLWW5ZdrYhhfSA1+rqSGfXbeA1LfUrViBljw5vxy5N1FPEKGTU2x0WFIZNtd1pwF6R0Vti I5BJONZ3T2iTqDBQ== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=JXEC2JSP; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=tLWW5Zdr DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0KRBIhNrsFaoYAcvZ0YP3W0XTTeCHuYd9glX8HrySn4=; b=JXEC2JSPbAhghivVpT57oMvEpSklUDeXYRyjnFw0yUL2Adv/0jqVqXd2LWyEmhUOcI4tih ZMQbFVXtHX0pF+8GRqc3eqZxtij3l+T7pVjwLQz/NMQHpjXgV+cCY03UrogQ1HnMuCbGT5 /5h+ITU1KJDdjzwLJBK7VPpbCCmOYSE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0KRBIhNrsFaoYAcvZ0YP3W0XTTeCHuYd9glX8HrySn4=; b=tLWW5ZdrYhhfSA1+rqSGfXbeA1LfUrViBljw5vxy5N1FPEKGTU2x0WFIZNtd1pwF6R0Vti I5BJONZ3T2iTqDBQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 4A2C013A7C; Mon, 1 Sep 2025 11:08:53 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id uMHJEcV+tWjtDgAAD6G6ig (envelope-from ); Mon, 01 Sep 2025 11:08:53 +0000 From: Vlastimil Babka Date: Mon, 01 Sep 2025 13:08:53 +0200 Subject: [PATCH 03/12] maple_tree: use percpu sheaves for maple_node_cache Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250901-maple-sheaves-v1-3-d6a1166b53f2@suse.cz> References: <20250901-maple-sheaves-v1-0-d6a1166b53f2@suse.cz> In-Reply-To: <20250901-maple-sheaves-v1-0-d6a1166b53f2@suse.cz> To: "Liam R. Howlett" , Matthew Wilcox , Lorenzo Stoakes , Jann Horn , Pedro Falcato , Suren Baghdasaryan Cc: Harry Yoo , Andrew Morton , maple-tree@lists.infradead.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, Vlastimil Babka X-Mailer: b4 0.14.2 X-Spam-Level: X-Spam-Flag: NO X-Rspamd-Queue-Id: 6468321170 X-Rspamd-Action: no action X-Rspamd-Server: rspamd1.dmz-prg2.suse.org X-Spamd-Result: default: False [-4.51 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[suse.cz:s=susede2_rsa,suse.cz:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; RCPT_COUNT_TWELVE(0.00)[13]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_SIGNED(0.00)[suse.cz:s=susede2_rsa,suse.cz:s=susede2_ed25519]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; R_RATELIMIT(0.00)[to_ip_from(RL5jz3zk9nm44ai14dcppf93zb)]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.cz:dkim,suse.cz:mid,suse.cz:email,imap1.dmz-prg2.suse.org:rdns,imap1.dmz-prg2.suse.org:helo]; DKIM_TRACE(0.00)[suse.cz:+] X-Spam-Score: -4.51 Setup the maple_node_cache with percpu sheaves of size 32 to hopefully improve its performance. Note this will not immediately take advantage of sheaf batching of kfree_rcu() operations due to the maple tree using call_rcu with custom callbacks. The followup changes to maple tree will change that and also make use of the prefilled sheaves functionality. Signed-off-by: Vlastimil Babka Reviewed-by: Suren Baghdasaryan Reviewed-by: Sidhartha Kumar --- lib/maple_tree.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/lib/maple_tree.c b/lib/maple_tree.c index b4ee2d29d7a962ca374467d0533185f2db3d35ff..a0db6bdc63793b8bbd544e24639= 1d99e880dede3 100644 --- a/lib/maple_tree.c +++ b/lib/maple_tree.c @@ -6302,9 +6302,14 @@ bool mas_nomem(struct ma_state *mas, gfp_t gfp) =20 void __init maple_tree_init(void) { + struct kmem_cache_args args =3D { + .align =3D sizeof(struct maple_node), + .sheaf_capacity =3D 32, + }; + maple_node_cache =3D kmem_cache_create("maple_node", - sizeof(struct maple_node), sizeof(struct maple_node), - SLAB_PANIC, NULL); + sizeof(struct maple_node), &args, + SLAB_PANIC); } =20 /** --=20 2.51.0 From nobody Sun Dec 14 06:23:25 2025 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B642430F549 for ; Mon, 1 Sep 2025 11:09:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756724949; cv=none; b=kTCl93XrEzigq10g2Ce6FSynGYGWTE4GmXI8QJUpUfe21VyDyKpK8t4viOZQ2BHhOI4b2c7Ikqe2xYpcKYzuauomMvBHcQ1GjS+lxlgp46gzYCVwKBB1RMOxntbuFeG14WbmC7U7EhRU22g6WstqV55GlrHOVF4DdA+N0TNAfkA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756724949; c=relaxed/simple; bh=atq+CHH4zhDPyoWu6Y++rUXCbmBzjqspFCyejAZKp4c=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=XmxMvqbUmiENPOXThxw+9oYOhmPryF0Yv3M4/F6eIdEyq33rNSxAYeA89r6UXDEIRQme0+oT93dxNiUttFS6U/l739Shw6vDdZb5t/snPsG2EQopa5tQ81g+EnBkGT951llqSyISWs2ZQlGuTLdFrTRdJ/iySYM/U+7DCQqmtng= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz; spf=pass smtp.mailfrom=suse.cz; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=AaRMkOJh; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=F5uB/8aw; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=AaRMkOJh; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=F5uB/8aw; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.cz Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="AaRMkOJh"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="F5uB/8aw"; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="AaRMkOJh"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="F5uB/8aw" Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 82CC821172; Mon, 1 Sep 2025 11:08:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tJhUfBeZeLxh7W8CevHKP2oQFWa3IdvgSf/9qDFU160=; b=AaRMkOJhu2G5Ls8tX/3MWoxgZt1PtPRcguBY8N//JqYZ4b5ssCrQ7CkrPGE8M2fnw3cYIE AcogPqeQ9eZ93prsKeyYgkmeNJ0ObBj4A401051acjh2Be5ftir19Mas7UUVzUxkrBW5zY EGA6Uc7Lu4NsmcTI7+JnpTP/EhpfR10= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tJhUfBeZeLxh7W8CevHKP2oQFWa3IdvgSf/9qDFU160=; b=F5uB/8aw/r1QgzBsKcDoocYUNgdj+8Tn+5/lLVpwTQ7Cz4iLT3gcTLOiW25VF5wn8P+QFp 9Ta5H7PcmlpRDZAg== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=AaRMkOJh; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b="F5uB/8aw" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tJhUfBeZeLxh7W8CevHKP2oQFWa3IdvgSf/9qDFU160=; b=AaRMkOJhu2G5Ls8tX/3MWoxgZt1PtPRcguBY8N//JqYZ4b5ssCrQ7CkrPGE8M2fnw3cYIE AcogPqeQ9eZ93prsKeyYgkmeNJ0ObBj4A401051acjh2Be5ftir19Mas7UUVzUxkrBW5zY EGA6Uc7Lu4NsmcTI7+JnpTP/EhpfR10= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=tJhUfBeZeLxh7W8CevHKP2oQFWa3IdvgSf/9qDFU160=; b=F5uB/8aw/r1QgzBsKcDoocYUNgdj+8Tn+5/lLVpwTQ7Cz4iLT3gcTLOiW25VF5wn8P+QFp 9Ta5H7PcmlpRDZAg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 5DDDC13ABA; Mon, 1 Sep 2025 11:08:53 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id aOeZFsV+tWjtDgAAD6G6ig (envelope-from ); Mon, 01 Sep 2025 11:08:53 +0000 From: Vlastimil Babka Date: Mon, 01 Sep 2025 13:08:54 +0200 Subject: [PATCH 04/12] tools/testing: include maple-shared.h in maple.c Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250901-maple-sheaves-v1-4-d6a1166b53f2@suse.cz> References: <20250901-maple-sheaves-v1-0-d6a1166b53f2@suse.cz> In-Reply-To: <20250901-maple-sheaves-v1-0-d6a1166b53f2@suse.cz> To: "Liam R. Howlett" , Matthew Wilcox , Lorenzo Stoakes , Jann Horn , Pedro Falcato , Suren Baghdasaryan Cc: Harry Yoo , Andrew Morton , maple-tree@lists.infradead.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, Vlastimil Babka X-Mailer: b4 0.14.2 X-Spamd-Result: default: False [-4.51 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[suse.cz:s=susede2_rsa,suse.cz:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; RCPT_COUNT_TWELVE(0.00)[13]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_SIGNED(0.00)[suse.cz:s=susede2_rsa,suse.cz:s=susede2_ed25519]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; R_RATELIMIT(0.00)[to_ip_from(RL5jz3zk9nm44ai14dcppf93zb)]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,suse.cz:mid,suse.cz:dkim,suse.cz:email]; DKIM_TRACE(0.00)[suse.cz:+] X-Spam-Flag: NO X-Spam-Level: X-Rspamd-Queue-Id: 82CC821172 X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Action: no action X-Spam-Score: -4.51 There's code duplication and we are about to add more functionality in maple-shared.h that we will need in userspace maple test to be available. Signed-off-by: Vlastimil Babka --- tools/testing/radix-tree/maple.c | 11 +++-------- 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/tools/testing/radix-tree/maple.c b/tools/testing/radix-tree/ma= ple.c index 159d5307b30a4b37e6cf2941848b8718e1b891d9..18db97a916f039bf72046c3ec3e= 7faffaeb5b755 100644 --- a/tools/testing/radix-tree/maple.c +++ b/tools/testing/radix-tree/maple.c @@ -8,14 +8,6 @@ * difficult to handle in kernel tests. */ =20 -#define CONFIG_DEBUG_MAPLE_TREE -#define CONFIG_MAPLE_SEARCH -#define MAPLE_32BIT (MAPLE_NODE_SLOTS > 31) -#include "test.h" -#include -#include -#include - #define module_init(x) #define module_exit(x) #define MODULE_AUTHOR(x) @@ -23,6 +15,9 @@ #define MODULE_LICENSE(x) #define dump_stack() assert(0) =20 +#include "maple-shared.h" +#include "test.h" + #include "../../../lib/maple_tree.c" #include "../../../lib/test_maple_tree.c" =20 --=20 2.51.0 From nobody Sun Dec 14 06:23:25 2025 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B43B1313E1F for ; Mon, 1 Sep 2025 11:09:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756724956; cv=none; b=sk1YLFrAGN6jRc8MmAk5qzbXE01NEryQQtqqK+428/5igpEWS/SqnpoX0OYYPd9bkswkKOkiff0d9k3s1vhKqeMyz2Y8SLg5bNlHE2KE3KeGebhd/7pGdUXQthjXl30+7w8yZ6wPB3ZmUOe1lpjYpAvfZraVD6NWGUMKarGB5A8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756724956; c=relaxed/simple; bh=HkdnpY/4D0YR705Z4sr6/18J6w2LIasvhwkCf88MmGw=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=dm5DfeYX94kuY5hW81zVwuB2C+eAvv+yZBQ+DQieNhnd+1kqadVW+H6dlL4mN1HRwVeiSG2hYaODZgWC/FKwMm5bplaNAS4wnBTQV7E51gtBzcVY3IGkrvCB7qrnl6n+/CK9DPSiskHAUDM3CHhcXawrX7YU267zIHDDWAlDoKs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz; spf=pass smtp.mailfrom=suse.cz; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=Z5c+ySOp; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=90sQNf+E; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=Z5c+ySOp; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=90sQNf+E; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.cz Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="Z5c+ySOp"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="90sQNf+E"; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="Z5c+ySOp"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="90sQNf+E" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 91D421F38A; Mon, 1 Sep 2025 11:08:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jt4x7IV34QbOESa72I6EsO7Rh0wlySwqCcuJjlcUiJ0=; b=Z5c+ySOpAZmhexw/+8cxAtn6gqTZddJRBvPDlb9TZPBBLg0KYg3kCb9HKFMI0KboUKPxIg YceR05iAiJD5o8y0wKlxvKnpZtb95s833EhLWKzJXSqMUSiAJacgfoMWb/5forB+JHGNXe iUGGsgJCI4ydnAzni+0OfsSccBtz/ao= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jt4x7IV34QbOESa72I6EsO7Rh0wlySwqCcuJjlcUiJ0=; b=90sQNf+EB0SvmEXrjKaFlLcDyjlJQIeqr3bT4vCRuyB12ePlNmbp1s7GH2e7QPpFKPT2/Q iwZmvuD61HWZ1TBw== Authentication-Results: smtp-out2.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jt4x7IV34QbOESa72I6EsO7Rh0wlySwqCcuJjlcUiJ0=; b=Z5c+ySOpAZmhexw/+8cxAtn6gqTZddJRBvPDlb9TZPBBLg0KYg3kCb9HKFMI0KboUKPxIg YceR05iAiJD5o8y0wKlxvKnpZtb95s833EhLWKzJXSqMUSiAJacgfoMWb/5forB+JHGNXe iUGGsgJCI4ydnAzni+0OfsSccBtz/ao= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=jt4x7IV34QbOESa72I6EsO7Rh0wlySwqCcuJjlcUiJ0=; b=90sQNf+EB0SvmEXrjKaFlLcDyjlJQIeqr3bT4vCRuyB12ePlNmbp1s7GH2e7QPpFKPT2/Q iwZmvuD61HWZ1TBw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 710861378C; Mon, 1 Sep 2025 11:08:53 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id eJlQG8V+tWjtDgAAD6G6ig (envelope-from ); Mon, 01 Sep 2025 11:08:53 +0000 From: Vlastimil Babka Date: Mon, 01 Sep 2025 13:08:55 +0200 Subject: [PATCH 05/12] tools/testing/vma: Implement vm_refcnt reset Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250901-maple-sheaves-v1-5-d6a1166b53f2@suse.cz> References: <20250901-maple-sheaves-v1-0-d6a1166b53f2@suse.cz> In-Reply-To: <20250901-maple-sheaves-v1-0-d6a1166b53f2@suse.cz> To: "Liam R. Howlett" , Matthew Wilcox , Lorenzo Stoakes , Jann Horn , Pedro Falcato , Suren Baghdasaryan Cc: Harry Yoo , Andrew Morton , maple-tree@lists.infradead.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, Vlastimil Babka X-Mailer: b4 0.14.2 X-Spam-Level: X-Spamd-Result: default: False [-4.30 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.20)[-0.997]; MIME_GOOD(-0.10)[text/plain]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RCVD_VIA_SMTP_AUTH(0.00)[]; MIME_TRACE(0.00)[0:+]; ARC_NA(0.00)[]; RCPT_COUNT_TWELVE(0.00)[13]; MID_RHS_MATCH_FROM(0.00)[]; RCVD_TLS_ALL(0.00)[]; DKIM_SIGNED(0.00)[suse.cz:s=susede2_rsa,suse.cz:s=susede2_ed25519]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; DBL_BLOCKED_OPENRESOLVER(0.00)[oracle.com:email,imap1.dmz-prg2.suse.org:helo,suse.cz:email,suse.cz:mid] X-Spam-Flag: NO X-Spam-Score: -4.30 From: "Liam R. Howlett" Add the reset of the ref count in vma_lock_init(). This is needed if the vma memory is not zeroed on allocation. Signed-off-by: Liam R. Howlett Signed-off-by: Vlastimil Babka --- tools/testing/vma/vma_internal.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tools/testing/vma/vma_internal.h b/tools/testing/vma/vma_inter= nal.h index 09732eff8dd84555563b3d485805ebab7b204584..972ab2686e0a3654cef611ce9f3= 409bc0c38dc80 100644 --- a/tools/testing/vma/vma_internal.h +++ b/tools/testing/vma/vma_internal.h @@ -1326,8 +1326,8 @@ static inline void ksm_exit(struct mm_struct *mm) =20 static inline void vma_lock_init(struct vm_area_struct *vma, bool reset_re= fcnt) { - (void)vma; - (void)reset_refcnt; + if (reset_refcnt) + refcount_set(&vma->vm_refcnt, 0); } =20 static inline void vma_numab_state_init(struct vm_area_struct *vma) --=20 2.51.0 From nobody Sun Dec 14 06:23:25 2025 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 16AE731194A for ; Mon, 1 Sep 2025 11:09:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756724955; cv=none; b=hHSIyOD1t7CRIdbfxo2V/Zzus8ILmGGjXqxcbQ3uwfm+3kSSQLh2jk15kqI+h0yRI1vZTDCuW/YkxYW9PmewMnAf/4/1fNnpg4E+SQ+t0QwAPap2OlE1Oct+98RfnGkqUMgxUhNds4BfROrOdihaUrLzAOyqgGrh6YvaigumRVM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756724955; c=relaxed/simple; bh=vqYRfDHV1134/hwscyIGvmfOy94+T2+FLB70CnCxZNA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=srDqpi9xcI/y/0lN/abuJ6shmC37c3UvxSS/u1Utbq4zWw3ye++vOyMCg0GgWK2ZDGU9X0q9T2VLlfcn3AVf+t3SVjeJa/TwpaR/ltJBBgleZ9J5SR0vwHrIknG5IDXKKFW5nUO5NVSgBhMgsajafrGZb2lJad5tgDFPqsirkww= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz; spf=pass smtp.mailfrom=suse.cz; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=SY0138kq; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=KhLG1iam; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=SY0138kq; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=KhLG1iam; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.cz Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="SY0138kq"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="KhLG1iam"; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="SY0138kq"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="KhLG1iam" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 9B2F121175; Mon, 1 Sep 2025 11:08:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=m8oRFcilmtndVzdHD+w9Znk922NjHNSPX4NFgV/7dew=; b=SY0138kqKl2jJSZVy/q1nixHUQLdDCivifvAbvYeRaYbrQxn9fMuANMGGktM37lJJpVMAX q0dtx//6JNCvIN+U16+DdWFD0kfvUydWcWy85FGg4AEQboPWw1mDhXx3iXj9dEOpzeceYR wTO12jqPSb+CdQlRAnGv+Yqu2SaLVY8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=m8oRFcilmtndVzdHD+w9Znk922NjHNSPX4NFgV/7dew=; b=KhLG1iamkBax+Zxkkl4j8GGek1stlQa1ZBXazOBWDh5BQGF5JwoVHrhdVjyxKnx6OnX6OE quAxyiHgfslZxhDQ== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=m8oRFcilmtndVzdHD+w9Znk922NjHNSPX4NFgV/7dew=; b=SY0138kqKl2jJSZVy/q1nixHUQLdDCivifvAbvYeRaYbrQxn9fMuANMGGktM37lJJpVMAX q0dtx//6JNCvIN+U16+DdWFD0kfvUydWcWy85FGg4AEQboPWw1mDhXx3iXj9dEOpzeceYR wTO12jqPSb+CdQlRAnGv+Yqu2SaLVY8= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=m8oRFcilmtndVzdHD+w9Znk922NjHNSPX4NFgV/7dew=; b=KhLG1iamkBax+Zxkkl4j8GGek1stlQa1ZBXazOBWDh5BQGF5JwoVHrhdVjyxKnx6OnX6OE quAxyiHgfslZxhDQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 8559013A31; Mon, 1 Sep 2025 11:08:53 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 4OBGIMV+tWjtDgAAD6G6ig (envelope-from ); Mon, 01 Sep 2025 11:08:53 +0000 From: Vlastimil Babka Date: Mon, 01 Sep 2025 13:08:56 +0200 Subject: [PATCH 06/12] testing/radix-tree/maple: Hack around kfree_rcu not existing Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250901-maple-sheaves-v1-6-d6a1166b53f2@suse.cz> References: <20250901-maple-sheaves-v1-0-d6a1166b53f2@suse.cz> In-Reply-To: <20250901-maple-sheaves-v1-0-d6a1166b53f2@suse.cz> To: "Liam R. Howlett" , Matthew Wilcox , Lorenzo Stoakes , Jann Horn , Pedro Falcato , Suren Baghdasaryan Cc: Harry Yoo , Andrew Morton , maple-tree@lists.infradead.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, Vlastimil Babka X-Mailer: b4 0.14.2 X-Spamd-Result: default: False [-4.30 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.20)[-0.999]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCPT_COUNT_TWELVE(0.00)[13]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_SIGNED(0.00)[suse.cz:s=susede2_rsa,suse.cz:s=susede2_ed25519]; FROM_HAS_DN(0.00)[]; R_RATELIMIT(0.00)[to_ip_from(RLzz977skxseo48jckwk35q6sc)]; FROM_EQ_ENVFROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; DBL_BLOCKED_OPENRESOLVER(0.00)[oracle.com:email,suse.cz:mid,suse.cz:email,suse.de:email,imap1.dmz-prg2.suse.org:helo] X-Spam-Flag: NO X-Spam-Level: X-Spam-Score: -4.30 From: "Liam R. Howlett" liburcu doesn't have kfree_rcu (or anything similar). Despite that, we can hack around it in a trivial fashion, by adding a wrapper. The wrapper only works for maple_nodes because we cannot get the kmem_cache pointer any other way in the test code. Link: https://lore.kernel.org/all/20250812162124.59417-1-pfalcato@suse.de/ Suggested-by: Pedro Falcato Signed-off-by: Liam R. Howlett Signed-off-by: Vlastimil Babka --- tools/testing/radix-tree/maple.c | 3 +-- tools/testing/shared/maple-shared.h | 11 +++++++++++ tools/testing/shared/maple-shim.c | 6 ++++++ 3 files changed, 18 insertions(+), 2 deletions(-) diff --git a/tools/testing/radix-tree/maple.c b/tools/testing/radix-tree/ma= ple.c index 18db97a916f039bf72046c3ec3e7faffaeb5b755..7fe91f24849b35723ec6aadbe45= ec7d2abedcc11 100644 --- a/tools/testing/radix-tree/maple.c +++ b/tools/testing/radix-tree/maple.c @@ -15,10 +15,9 @@ #define MODULE_LICENSE(x) #define dump_stack() assert(0) =20 -#include "maple-shared.h" #include "test.h" =20 -#include "../../../lib/maple_tree.c" +#include "../shared/maple-shim.c" #include "../../../lib/test_maple_tree.c" =20 #define RCU_RANGE_COUNT 1000 diff --git a/tools/testing/shared/maple-shared.h b/tools/testing/shared/map= le-shared.h index dc4d30f3860b9bd23b4177c7d7926ac686887815..2a1e9a8594a2834326cd9374738= b2a2c7c3f9f7c 100644 --- a/tools/testing/shared/maple-shared.h +++ b/tools/testing/shared/maple-shared.h @@ -10,4 +10,15 @@ #include #include "linux/init.h" =20 +void maple_rcu_cb(struct rcu_head *head); +#define rcu_cb maple_rcu_cb + +#define kfree_rcu(_struct, _memb) \ +do { \ + typeof(_struct) _p_struct =3D (_struct); \ + \ + call_rcu(&((_p_struct)->_memb), rcu_cb); \ +} while(0); + + #endif /* __MAPLE_SHARED_H__ */ diff --git a/tools/testing/shared/maple-shim.c b/tools/testing/shared/maple= -shim.c index 9d7b743415660305416e972fa75b56824211b0eb..16252ee616c0489c80490ff25b8= d255427bf9fdc 100644 --- a/tools/testing/shared/maple-shim.c +++ b/tools/testing/shared/maple-shim.c @@ -6,3 +6,9 @@ #include =20 #include "../../../lib/maple_tree.c" + +void maple_rcu_cb(struct rcu_head *head) { + struct maple_node *node =3D container_of(head, struct maple_node, rcu); + + kmem_cache_free(maple_node_cache, node); +} --=20 2.51.0 From nobody Sun Dec 14 06:23:25 2025 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B3369311C13 for ; Mon, 1 Sep 2025 11:09:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756724962; cv=none; b=CiXD0kwHah6bccwxIS04DEE8FfgQraH26RTFcevVfMVkItPvnIdeg2UlMSYXGERtFhUzW1hZMibHRDMOd+0y6HM0wJ/7e3vO737a0/tU3H9PfsINliDETJKpMFy4XQJ9RVgJQgBzUTGPOV8bOPQcGAVXXzSZItdKDsYVI5qCFbo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756724962; c=relaxed/simple; bh=NEnPKJoh4PSRWp8CZDKpOmH3tB/16GPOtmyVTP2ZnoU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=cg2hckUucTICpso6yACI03miUg5k6W86LrXNFT8+jwyhJ7/N4Fq11pANtKhE3C8joNvaEEgZRSPMG8Sb/OK0jzY1wDZeHFFLyRHg4eKcb3+i887GThvV7fz3Q8RcHpr1xaHFFxcCGCxGN7C811e9TKDoCTeMJfNSWh5seB+qJAM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz; spf=pass smtp.mailfrom=suse.cz; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=S0BrW2dO; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=80l0mWms; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=S0BrW2dO; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=80l0mWms; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.cz Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="S0BrW2dO"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="80l0mWms"; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="S0BrW2dO"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="80l0mWms" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id AE99821179; Mon, 1 Sep 2025 11:08:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4DoczhIJm/QIxVWZ7D37eQBiOD25npSD9wVr63TFoHc=; b=S0BrW2dOwi0z04Wj3WboKaP8vSfpyJifHoyHj866eCywFn5ijYNZ0vaeOwjBE2J3qLBbw+ qz+BOxH5ouaguFBsCkcr6fVR4S7ROOOXNMteW/Xvgzh1f0BMVaMgNbcTU3OuKfwceRRVT0 69pdz8U4KZtb+qf8IHL06oF6EuwcRM0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4DoczhIJm/QIxVWZ7D37eQBiOD25npSD9wVr63TFoHc=; b=80l0mWmsnyMbEJ1/ElIfu3/n/y/ayVlDq6CjlfcCEDHZ6PPp9HlzPKAOV1w65VxnWdihjM j+7OpTNfqp/viPCQ== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4DoczhIJm/QIxVWZ7D37eQBiOD25npSD9wVr63TFoHc=; b=S0BrW2dOwi0z04Wj3WboKaP8vSfpyJifHoyHj866eCywFn5ijYNZ0vaeOwjBE2J3qLBbw+ qz+BOxH5ouaguFBsCkcr6fVR4S7ROOOXNMteW/Xvgzh1f0BMVaMgNbcTU3OuKfwceRRVT0 69pdz8U4KZtb+qf8IHL06oF6EuwcRM0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4DoczhIJm/QIxVWZ7D37eQBiOD25npSD9wVr63TFoHc=; b=80l0mWmsnyMbEJ1/ElIfu3/n/y/ayVlDq6CjlfcCEDHZ6PPp9HlzPKAOV1w65VxnWdihjM j+7OpTNfqp/viPCQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 98E0413A3E; Mon, 1 Sep 2025 11:08:53 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id EBALJcV+tWjtDgAAD6G6ig (envelope-from ); Mon, 01 Sep 2025 11:08:53 +0000 From: Vlastimil Babka Date: Mon, 01 Sep 2025 13:08:57 +0200 Subject: [PATCH 07/12] maple_tree: Use kfree_rcu in ma_free_rcu Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250901-maple-sheaves-v1-7-d6a1166b53f2@suse.cz> References: <20250901-maple-sheaves-v1-0-d6a1166b53f2@suse.cz> In-Reply-To: <20250901-maple-sheaves-v1-0-d6a1166b53f2@suse.cz> To: "Liam R. Howlett" , Matthew Wilcox , Lorenzo Stoakes , Jann Horn , Pedro Falcato , Suren Baghdasaryan Cc: Harry Yoo , Andrew Morton , maple-tree@lists.infradead.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, Vlastimil Babka X-Mailer: b4 0.14.2 X-Spam-Level: X-Spamd-Result: default: False [-4.30 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.20)[-0.997]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCPT_COUNT_TWELVE(0.00)[13]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_SIGNED(0.00)[suse.cz:s=susede2_rsa,suse.cz:s=susede2_ed25519]; FROM_HAS_DN(0.00)[]; R_RATELIMIT(0.00)[to_ip_from(RLzz977skxseo48jckwk35q6sc)]; FROM_EQ_ENVFROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.cz:email,suse.cz:mid,imap1.dmz-prg2.suse.org:helo,suse.de:email] X-Spam-Flag: NO X-Spam-Score: -4.30 From: Pedro Falcato kfree_rcu is an optimized version of call_rcu + kfree. It used to not be possible to call it on non-kmalloc objects, but this restriction was lifted ever since SLOB was dropped from the kernel, and since commit 6c6c47b063b5 ("mm, slab: call kvfree_rcu_barrier() from kmem_cache_destroy(= )"). Thus, replace call_rcu + mt_free_rcu with kfree_rcu. Signed-off-by: Pedro Falcato Signed-off-by: Vlastimil Babka --- lib/maple_tree.c | 13 +++---------- 1 file changed, 3 insertions(+), 10 deletions(-) diff --git a/lib/maple_tree.c b/lib/maple_tree.c index a0db6bdc63793b8bbd544e246391d99e880dede3..d77e82362f03905040ac61630f9= 2fe9af1e59f98 100644 --- a/lib/maple_tree.c +++ b/lib/maple_tree.c @@ -191,13 +191,6 @@ static inline void mt_free_bulk(size_t size, void __rc= u **nodes) kmem_cache_free_bulk(maple_node_cache, size, (void **)nodes); } =20 -static void mt_free_rcu(struct rcu_head *head) -{ - struct maple_node *node =3D container_of(head, struct maple_node, rcu); - - kmem_cache_free(maple_node_cache, node); -} - /* * ma_free_rcu() - Use rcu callback to free a maple node * @node: The node to free @@ -208,7 +201,7 @@ static void mt_free_rcu(struct rcu_head *head) static void ma_free_rcu(struct maple_node *node) { WARN_ON(node->parent !=3D ma_parent_ptr(node)); - call_rcu(&node->rcu, mt_free_rcu); + kfree_rcu(node, rcu); } =20 static void mt_set_height(struct maple_tree *mt, unsigned char height) @@ -5281,7 +5274,7 @@ static void mt_free_walk(struct rcu_head *head) mt_free_bulk(node->slot_len, slots); =20 free_leaf: - mt_free_rcu(&node->rcu); + mt_free_one(node); } =20 static inline void __rcu **mte_destroy_descend(struct maple_enode **enode, @@ -5365,7 +5358,7 @@ static void mt_destroy_walk(struct maple_enode *enode= , struct maple_tree *mt, =20 free_leaf: if (free) - mt_free_rcu(&node->rcu); + mt_free_one(node); else mt_clear_meta(mt, node, node->type); } --=20 2.51.0 From nobody Sun Dec 14 06:23:25 2025 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8CD5C31AF04 for ; Mon, 1 Sep 2025 11:09:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756724969; cv=none; b=oRA9CLuvkTyIfWfpJvP4Fx36o5BdDvUisZqHEsrOmUB8P7PHE90E13lKjzvED9sMSGUpWttTzmEhkSZfepu1/aZJsq2jhF9N5Lngd7fs+BYLSVQR7kY24ckB/H/G2m6WWHoerr5qQ1Ztip2CFSbrbrdUBO0Q2+ctuBQFEPEtTEE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756724969; c=relaxed/simple; bh=cbYZ4uYu2Ew4je+ZVdv6l5r3Tds6HvQmitLofhje3ds=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=u2pBAm35jhsllENerIN7mBrQ4ZmP8X70lrVIsfcSI/cL6IspvnetxxyGXQGVvdBy5Yv4HWguzXiwUKN9A99Kn3e9mYD/VMleyDee4/l5BLvnW31YiHDa2Ubzz9Ka0c1076KenW5Pq1awy9/XHGd0NHEpmZ5y37/j/XYC3RoDMaw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz; spf=pass smtp.mailfrom=suse.cz; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=zKTz6Lov; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=TTbyuHqB; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=zKTz6Lov; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=TTbyuHqB; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.cz Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="zKTz6Lov"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="TTbyuHqB"; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="zKTz6Lov"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="TTbyuHqB" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id C29D62117A; Mon, 1 Sep 2025 11:08:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Orc9L8mxDfBM2P4Cd87ufTnmwdvv431SxJfeIHzvdf8=; b=zKTz6Lovr5q/xB/OgJ4lFP4VG791nn9WUZ4LQIxwfrfi3FJ5eKFvtpckPEMVHyiJlozJJ0 NEr7AAQbfKzzLmTRGT91+4KNRXV/iIVPYlKGqBGBgC6Qy0G2bPvAYN06KOI45+UTIBCGjb 2O2qZkYl4FlRzIW0t0dVuzSaZGCdzYQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Orc9L8mxDfBM2P4Cd87ufTnmwdvv431SxJfeIHzvdf8=; b=TTbyuHqBOtS89J5z/MDvzGrGoZgGM8awDt34B0yPMv3zPVL5OtplAWc6UYFAmuFtfwS7OV jGC0Ri+QQEJul+CQ== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Orc9L8mxDfBM2P4Cd87ufTnmwdvv431SxJfeIHzvdf8=; b=zKTz6Lovr5q/xB/OgJ4lFP4VG791nn9WUZ4LQIxwfrfi3FJ5eKFvtpckPEMVHyiJlozJJ0 NEr7AAQbfKzzLmTRGT91+4KNRXV/iIVPYlKGqBGBgC6Qy0G2bPvAYN06KOI45+UTIBCGjb 2O2qZkYl4FlRzIW0t0dVuzSaZGCdzYQ= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Orc9L8mxDfBM2P4Cd87ufTnmwdvv431SxJfeIHzvdf8=; b=TTbyuHqBOtS89J5z/MDvzGrGoZgGM8awDt34B0yPMv3zPVL5OtplAWc6UYFAmuFtfwS7OV jGC0Ri+QQEJul+CQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id ACCE113A7C; Mon, 1 Sep 2025 11:08:53 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id 2JzeKcV+tWjtDgAAD6G6ig (envelope-from ); Mon, 01 Sep 2025 11:08:53 +0000 From: Vlastimil Babka Date: Mon, 01 Sep 2025 13:08:58 +0200 Subject: [PATCH 08/12] maple_tree: Replace mt_free_one() with kfree() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250901-maple-sheaves-v1-8-d6a1166b53f2@suse.cz> References: <20250901-maple-sheaves-v1-0-d6a1166b53f2@suse.cz> In-Reply-To: <20250901-maple-sheaves-v1-0-d6a1166b53f2@suse.cz> To: "Liam R. Howlett" , Matthew Wilcox , Lorenzo Stoakes , Jann Horn , Pedro Falcato , Suren Baghdasaryan Cc: Harry Yoo , Andrew Morton , maple-tree@lists.infradead.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, Vlastimil Babka X-Mailer: b4 0.14.2 X-Spamd-Result: default: False [-4.30 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.20)[-0.997]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCPT_COUNT_TWELVE(0.00)[13]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_SIGNED(0.00)[suse.cz:s=susede2_rsa,suse.cz:s=susede2_ed25519]; FROM_HAS_DN(0.00)[]; R_RATELIMIT(0.00)[to_ip_from(RLzz977skxseo48jckwk35q6sc)]; FROM_EQ_ENVFROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:helo,suse.cz:mid,suse.cz:email,suse.de:email] X-Spam-Flag: NO X-Spam-Level: X-Spam-Score: -4.30 From: Pedro Falcato kfree() is a little shorter and works with kmem_cache_alloc'd pointers too. Also lets us remove one more helper. Signed-off-by: Pedro Falcato Signed-off-by: Vlastimil Babka --- lib/maple_tree.c | 13 ++++--------- 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/lib/maple_tree.c b/lib/maple_tree.c index d77e82362f03905040ac61630f92fe9af1e59f98..b361b484cfcaacd99472dd4c2b8= de9260b307425 100644 --- a/lib/maple_tree.c +++ b/lib/maple_tree.c @@ -181,11 +181,6 @@ static inline int mt_alloc_bulk(gfp_t gfp, size_t size= , void **nodes) return kmem_cache_alloc_bulk(maple_node_cache, gfp, size, nodes); } =20 -static inline void mt_free_one(struct maple_node *node) -{ - kmem_cache_free(maple_node_cache, node); -} - static inline void mt_free_bulk(size_t size, void __rcu **nodes) { kmem_cache_free_bulk(maple_node_cache, size, (void **)nodes); @@ -5274,7 +5269,7 @@ static void mt_free_walk(struct rcu_head *head) mt_free_bulk(node->slot_len, slots); =20 free_leaf: - mt_free_one(node); + kfree(node); } =20 static inline void __rcu **mte_destroy_descend(struct maple_enode **enode, @@ -5358,7 +5353,7 @@ static void mt_destroy_walk(struct maple_enode *enode= , struct maple_tree *mt, =20 free_leaf: if (free) - mt_free_one(node); + kfree(node); else mt_clear_meta(mt, node, node->type); } @@ -5585,7 +5580,7 @@ void mas_destroy(struct ma_state *mas) mt_free_bulk(count, (void __rcu **)&node->slot[1]); total -=3D count; } - mt_free_one(ma_mnode_ptr(node)); + kfree(ma_mnode_ptr(node)); total--; } =20 @@ -6635,7 +6630,7 @@ static void mas_dup_free(struct ma_state *mas) } =20 node =3D mte_to_node(mas->node); - mt_free_one(node); + kfree(node); } =20 /* --=20 2.51.0 From nobody Sun Dec 14 06:23:25 2025 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.223.131]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 33A5531A06A for ; Mon, 1 Sep 2025 11:09:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.131 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756724966; cv=none; b=RrYxVxgG54u1LvUj90JGBRXxLWV8p+ozV33TZ9F1sYppubOvNRVUtugCgKBhEInszc0p8dD62LLKYVi53KMEEVvOtf8YzAbed5hZaGtlrS3H5WAdBIzlvTQ77aFgFoyHskAaJrDt3UDD2OIDllcq2LtZNIIy2M/D4XVyLGf/tsU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756724966; c=relaxed/simple; bh=9DVF5YlHhzvZ3QdPg/Fne3pqcsKvAT7L7x/2E24PN/Y=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=kmVBEzWAYwjJszZ/+yVX5fbJO8rm9FkG1Wv0eIhzSaG15tIvCufIkMWgvyRLl4QhadzlpA6alFjsAFYyEYjECSKjQMBYmnLaEcvvw0xy6ZDicjiQ0S96QOqXVwHKdRFU0/wK+O0qtWCr+xNz8h0SFBKhn8yzADzqSdu6XqxzDnE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz; spf=pass smtp.mailfrom=suse.cz; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=fOSewe5A; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=BWOLqaNY; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=fOSewe5A; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=BWOLqaNY; arc=none smtp.client-ip=195.135.223.131 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.cz Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="fOSewe5A"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="BWOLqaNY"; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="fOSewe5A"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="BWOLqaNY" Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id D641F1F38C; Mon, 1 Sep 2025 11:08:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=htQ+/TA5SKDWSG5dnn2v4RIOWxXiBF6DNi9KcdkVFvU=; b=fOSewe5ApnD56gxuszMv1ZxWgYnHilfM5JvSQ57oZnziFZrZC2Cr0kGjzNbx4C598jb+RO XMazmhNa9fZfCpKrq1LNP2fKaPjv7tpHfZXZIVp00ITgoAZcbCJ19V2nnvgrEutWSGkkq1 jb91GDUJlRoXd8ZBSEj6G1wlJjqtPB4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=htQ+/TA5SKDWSG5dnn2v4RIOWxXiBF6DNi9KcdkVFvU=; b=BWOLqaNYD0gV3me+lqMPb2sV/a5rtUVLASqnMxiAvcHVsCv4HPe7F/brqal5/xn99eBSa0 usVtg+TSxavBBBCg== Authentication-Results: smtp-out2.suse.de; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=fOSewe5A; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=BWOLqaNY DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=htQ+/TA5SKDWSG5dnn2v4RIOWxXiBF6DNi9KcdkVFvU=; b=fOSewe5ApnD56gxuszMv1ZxWgYnHilfM5JvSQ57oZnziFZrZC2Cr0kGjzNbx4C598jb+RO XMazmhNa9fZfCpKrq1LNP2fKaPjv7tpHfZXZIVp00ITgoAZcbCJ19V2nnvgrEutWSGkkq1 jb91GDUJlRoXd8ZBSEj6G1wlJjqtPB4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=htQ+/TA5SKDWSG5dnn2v4RIOWxXiBF6DNi9KcdkVFvU=; b=BWOLqaNYD0gV3me+lqMPb2sV/a5rtUVLASqnMxiAvcHVsCv4HPe7F/brqal5/xn99eBSa0 usVtg+TSxavBBBCg== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id BFA781378C; Mon, 1 Sep 2025 11:08:53 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id UPGFLsV+tWjtDgAAD6G6ig (envelope-from ); Mon, 01 Sep 2025 11:08:53 +0000 From: Vlastimil Babka Date: Mon, 01 Sep 2025 13:08:59 +0200 Subject: [PATCH 09/12] tools: Add sheaf to slab testing Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250901-maple-sheaves-v1-9-d6a1166b53f2@suse.cz> References: <20250901-maple-sheaves-v1-0-d6a1166b53f2@suse.cz> In-Reply-To: <20250901-maple-sheaves-v1-0-d6a1166b53f2@suse.cz> To: "Liam R. Howlett" , Matthew Wilcox , Lorenzo Stoakes , Jann Horn , Pedro Falcato , Suren Baghdasaryan Cc: Harry Yoo , Andrew Morton , maple-tree@lists.infradead.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, Vlastimil Babka X-Mailer: b4 0.14.2 X-Spam-Level: X-Spam-Flag: NO X-Rspamd-Queue-Id: D641F1F38C X-Rspamd-Action: no action X-Rspamd-Server: rspamd1.dmz-prg2.suse.org X-Spamd-Result: default: False [-4.51 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[suse.cz:s=susede2_rsa,suse.cz:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; RCPT_COUNT_TWELVE(0.00)[13]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_SIGNED(0.00)[suse.cz:s=susede2_rsa,suse.cz:s=susede2_ed25519]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; R_RATELIMIT(0.00)[to_ip_from(RL5jz3zk9nm44ai14dcppf93zb)]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[imap1.dmz-prg2.suse.org:rdns,imap1.dmz-prg2.suse.org:helo,suse.cz:dkim,suse.cz:mid,suse.cz:email,oracle.com:email]; DKIM_TRACE(0.00)[suse.cz:+] X-Spam-Score: -4.51 From: "Liam R. Howlett" Add the sheaf structs to the slab header and the functions to the testing/shared/linux.c file. Signed-off-by: Liam R. Howlett Signed-off-by: Vlastimil Babka --- tools/include/linux/slab.h | 28 ++++++++++++++ tools/testing/shared/linux.c | 89 ++++++++++++++++++++++++++++++++++++++++= ++++ 2 files changed, 117 insertions(+) diff --git a/tools/include/linux/slab.h b/tools/include/linux/slab.h index c5c5cc6db5668be2cc94c29065ccfa7ca7b4bb08..94937a699402bd1f31887dfb52b= 6fd0a3c986f43 100644 --- a/tools/include/linux/slab.h +++ b/tools/include/linux/slab.h @@ -123,6 +123,18 @@ struct kmem_cache_args { void (*ctor)(void *); }; =20 +struct slab_sheaf { + union { + struct list_head barn_list; + /* only used for prefilled sheafs */ + unsigned int capacity; + }; + struct kmem_cache *cache; + unsigned int size; + int node; /* only used for rcu_sheaf */ + void *objects[]; +}; + static inline void *kzalloc(size_t size, gfp_t gfp) { return kmalloc(size, gfp | __GFP_ZERO); @@ -173,5 +185,21 @@ __kmem_cache_create(const char *name, unsigned int siz= e, unsigned int align, void kmem_cache_free_bulk(struct kmem_cache *cachep, size_t size, void **l= ist); int kmem_cache_alloc_bulk(struct kmem_cache *cachep, gfp_t gfp, size_t siz= e, void **list); +struct slab_sheaf * +kmem_cache_prefill_sheaf(struct kmem_cache *s, gfp_t gfp, unsigned int siz= e); + +void * +kmem_cache_alloc_from_sheaf(struct kmem_cache *s, gfp_t gfp, + struct slab_sheaf *sheaf); + +void kmem_cache_return_sheaf(struct kmem_cache *s, gfp_t gfp, + struct slab_sheaf *sheaf); +int kmem_cache_refill_sheaf(struct kmem_cache *s, gfp_t gfp, + struct slab_sheaf **sheafp, unsigned int size); + +static inline unsigned int kmem_cache_sheaf_size(struct slab_sheaf *sheaf) +{ + return sheaf->size; +} =20 #endif /* _TOOLS_SLAB_H */ diff --git a/tools/testing/shared/linux.c b/tools/testing/shared/linux.c index 97b8412ccbb6d222604c7b397c53c65618d8d51b..4ceff7969b78cf8e33cd1e021c6= 8bc9f8a02a7a1 100644 --- a/tools/testing/shared/linux.c +++ b/tools/testing/shared/linux.c @@ -137,6 +137,12 @@ void kmem_cache_free_bulk(struct kmem_cache *cachep, s= ize_t size, void **list) if (kmalloc_verbose) pr_debug("Bulk free %p[0-%zu]\n", list, size - 1); =20 + if (cachep->exec_callback) { + if (cachep->callback) + cachep->callback(cachep->private); + cachep->exec_callback =3D false; + } + pthread_mutex_lock(&cachep->lock); for (int i =3D 0; i < size; i++) kmem_cache_free_locked(cachep, list[i]); @@ -242,6 +248,89 @@ __kmem_cache_create_args(const char *name, unsigned in= t size, return ret; } =20 +struct slab_sheaf * +kmem_cache_prefill_sheaf(struct kmem_cache *s, gfp_t gfp, unsigned int siz= e) +{ + struct slab_sheaf *sheaf; + unsigned int capacity; + + if (s->exec_callback) { + if (s->callback) + s->callback(s->private); + s->exec_callback =3D false; + } + + capacity =3D max(size, s->sheaf_capacity); + + sheaf =3D calloc(1, sizeof(*sheaf) + sizeof(void *) * capacity); + if (!sheaf) + return NULL; + + sheaf->cache =3D s; + sheaf->capacity =3D capacity; + sheaf->size =3D kmem_cache_alloc_bulk(s, gfp, size, sheaf->objects); + if (!sheaf->size) { + free(sheaf); + return NULL; + } + + return sheaf; +} + +int kmem_cache_refill_sheaf(struct kmem_cache *s, gfp_t gfp, + struct slab_sheaf **sheafp, unsigned int size) +{ + struct slab_sheaf *sheaf =3D *sheafp; + int refill; + + if (sheaf->size >=3D size) + return 0; + + if (size > sheaf->capacity) { + sheaf =3D kmem_cache_prefill_sheaf(s, gfp, size); + if (!sheaf) + return -ENOMEM; + + kmem_cache_return_sheaf(s, gfp, *sheafp); + *sheafp =3D sheaf; + return 0; + } + + refill =3D kmem_cache_alloc_bulk(s, gfp, size - sheaf->size, + &sheaf->objects[sheaf->size]); + if (!refill) + return -ENOMEM; + + sheaf->size +=3D refill; + return 0; +} + +void kmem_cache_return_sheaf(struct kmem_cache *s, gfp_t gfp, + struct slab_sheaf *sheaf) +{ + if (sheaf->size) + kmem_cache_free_bulk(s, sheaf->size, &sheaf->objects[0]); + + free(sheaf); +} + +void * +kmem_cache_alloc_from_sheaf(struct kmem_cache *s, gfp_t gfp, + struct slab_sheaf *sheaf) +{ + void *obj; + + if (sheaf->size =3D=3D 0) { + printf("Nothing left in sheaf!\n"); + return NULL; + } + + obj =3D sheaf->objects[--sheaf->size]; + sheaf->objects[sheaf->size] =3D NULL; + + return obj; +} + /* * Test the test infrastructure for kem_cache_alloc/free and bulk counterp= arts. */ --=20 2.51.0 From nobody Sun Dec 14 06:23:25 2025 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CCE6831B108 for ; Mon, 1 Sep 2025 11:09:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756724976; cv=none; b=Vl1VzLHbgZ5eC2dp9ZthmzWggtBSEbtd24xL3MmB2s11putIxjVxToKX9BjGjfQ7fPNK3gcm5m+egoR9yUhBmrcdkSnYru+dTKHjblM/Kli4WCUj6bMdIAc/LSl879EYcSpJ60ZEy1114lSbbII8HFq4r0FJvOjq0C96hupJmIc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756724976; c=relaxed/simple; bh=rcDxtjdz4Ghoo1ymRr7DS+ySy65r5C3QYO2Bl8V9/G4=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Cw8wQGvklKX3J+yfH9RfqACyWWcUGwSp8AChv60yXF4REJt/GhBlBHB/ZKFTE33dwfDJEOLnDKy3liHNBWYtkq8Sq6EY6eHLGrvB9zXCbEj9wrSyi9gcbNnn6dgF6CpehOsuy1a/iN5mXEARemSVfhnfQrsm9xJiHjtKkzsh89M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz; spf=pass smtp.mailfrom=suse.cz; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=i4Vj+JP2; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=uehvyEa+; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=j7RHCXFy; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=99AyC0vR; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.cz Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="i4Vj+JP2"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="uehvyEa+"; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="j7RHCXFy"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="99AyC0vR" Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id F03862117C; Mon, 1 Sep 2025 11:08:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1756724934; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=240g5gOvCSTSQtFdroGNL/n4PS7v1EePvChrfoLWaKs=; b=i4Vj+JP2PCBpH0KMX3lXYaPstnXUFWkc9J1aiaBW7yhVGZZpIfGQXmMwzBSGv7ADKmJjz/ KSr4YEHadxfCM2xYPh+sqKMrOBCwTUjMTTPBIKed6emchi3XHxEXnjI7ylmO+GhGesgub0 UWnFibUll++lziv1FFyWAYSXhx1gN+o= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1756724934; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=240g5gOvCSTSQtFdroGNL/n4PS7v1EePvChrfoLWaKs=; b=uehvyEa+LJKHl5A8ZS8Csn2aIOq4a3jvkD0Mps6i6zkMiVoBrkKJZUO5Wdf2UhG8lh0z+v ABO6QQK99khZOYAQ== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b=j7RHCXFy; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b=99AyC0vR DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=240g5gOvCSTSQtFdroGNL/n4PS7v1EePvChrfoLWaKs=; b=j7RHCXFyq+VM9Is0qcDDQimAV9o+rj9F9p5psPg5QKhILw7mg288pu00DvqyN+qEuCec6A JsbXaiRlwGZclfL3igmXrmHAj9zTBjiQ05Z3fZoh7u6n/H71A2MtoH2+Pdd/5lg+9N0v7U ZvNjGbg75V6t0exrG85NFemDZ7X/1X4= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1756724933; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=240g5gOvCSTSQtFdroGNL/n4PS7v1EePvChrfoLWaKs=; b=99AyC0vRpSTT51zWCBDYBH1Uj0rF88H3Dk9K7W5hpxLtUOk5MeUybh7ShH9PDIP2dVze2f ZJc0Tj3BGVlOS2Aw== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id D46CD13A31; Mon, 1 Sep 2025 11:08:53 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id iBuXM8V+tWjtDgAAD6G6ig (envelope-from ); Mon, 01 Sep 2025 11:08:53 +0000 From: Vlastimil Babka Date: Mon, 01 Sep 2025 13:09:00 +0200 Subject: [PATCH 10/12] maple_tree: Sheaf conversion and testing Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250901-maple-sheaves-v1-10-d6a1166b53f2@suse.cz> References: <20250901-maple-sheaves-v1-0-d6a1166b53f2@suse.cz> In-Reply-To: <20250901-maple-sheaves-v1-0-d6a1166b53f2@suse.cz> To: "Liam R. Howlett" , Matthew Wilcox , Lorenzo Stoakes , Jann Horn , Pedro Falcato , Suren Baghdasaryan Cc: Harry Yoo , Andrew Morton , maple-tree@lists.infradead.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, Vlastimil Babka X-Mailer: b4 0.14.2 X-Spamd-Result: default: False [-4.51 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[suse.cz:s=susede2_rsa,suse.cz:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; RCPT_COUNT_TWELVE(0.00)[13]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_SIGNED(0.00)[suse.cz:s=susede2_rsa,suse.cz:s=susede2_ed25519]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; R_RATELIMIT(0.00)[to_ip_from(RL5jz3zk9nm44ai14dcppf93zb)]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.cz:mid,suse.cz:dkim,suse.cz:email,imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,oracle.com:email]; DKIM_TRACE(0.00)[suse.cz:+] X-Spam-Flag: NO X-Spam-Level: X-Rspamd-Queue-Id: F03862117C X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Action: no action X-Spam-Score: -4.51 From: "Liam R. Howlett" Use sheaves instead of bulk allocations. This should speed up the allocations and the return path of unused allocations. Remove the push and pop of nodes from the maple state as this is now handled by the slab layer with sheaves. Testing has been removed as necessary since the features of the tree have been reduced. Signed-off-by: Liam R. Howlett Signed-off-by: Vlastimil Babka --- include/linux/maple_tree.h | 6 +- lib/maple_tree.c | 329 ++++++--------------------- lib/test_maple_tree.c | 8 + tools/testing/radix-tree/maple.c | 464 ++---------------------------------= ---- tools/testing/shared/linux.c | 5 +- 5 files changed, 99 insertions(+), 713 deletions(-) diff --git a/include/linux/maple_tree.h b/include/linux/maple_tree.h index bafe143b1f783202e27b32567fffee4149e8e266..166fd67e00d882b1e6de1f80c1b= 590bba7497cd3 100644 --- a/include/linux/maple_tree.h +++ b/include/linux/maple_tree.h @@ -442,7 +442,8 @@ struct ma_state { struct maple_enode *node; /* The node containing this entry */ unsigned long min; /* The minimum index of this node - implied pivot min= */ unsigned long max; /* The maximum index of this node - implied pivot max= */ - struct maple_alloc *alloc; /* Allocated nodes for this operation */ + struct slab_sheaf *sheaf; /* Allocated nodes for this operation */ + unsigned long node_request; enum maple_status status; /* The status of the state (active, start, none= , etc) */ unsigned char depth; /* depth of tree descent during write */ unsigned char offset; @@ -490,7 +491,8 @@ struct ma_wr_state { .status =3D ma_start, \ .min =3D 0, \ .max =3D ULONG_MAX, \ - .alloc =3D NULL, \ + .node_request=3D 0, \ + .sheaf =3D NULL, \ .mas_flags =3D 0, \ .store_type =3D wr_invalid, \ } diff --git a/lib/maple_tree.c b/lib/maple_tree.c index b361b484cfcaacd99472dd4c2b8de9260b307425..cfe80c50a97e3118eefc24275fb= cd2eec5e6e6e8 100644 --- a/lib/maple_tree.c +++ b/lib/maple_tree.c @@ -186,6 +186,22 @@ static inline void mt_free_bulk(size_t size, void __rc= u **nodes) kmem_cache_free_bulk(maple_node_cache, size, (void **)nodes); } =20 +static void mt_return_sheaf(struct slab_sheaf *sheaf) +{ + kmem_cache_return_sheaf(maple_node_cache, GFP_KERNEL, sheaf); +} + +static struct slab_sheaf *mt_get_sheaf(gfp_t gfp, int count) +{ + return kmem_cache_prefill_sheaf(maple_node_cache, gfp, count); +} + +static int mt_refill_sheaf(gfp_t gfp, struct slab_sheaf **sheaf, + unsigned int size) +{ + return kmem_cache_refill_sheaf(maple_node_cache, gfp, sheaf, size); +} + /* * ma_free_rcu() - Use rcu callback to free a maple node * @node: The node to free @@ -578,67 +594,6 @@ static __always_inline bool mte_dead_node(const struct= maple_enode *enode) return ma_dead_node(node); } =20 -/* - * mas_allocated() - Get the number of nodes allocated in a maple state. - * @mas: The maple state - * - * The ma_state alloc member is overloaded to hold a pointer to the first - * allocated node or to the number of requested nodes to allocate. If bit= 0 is - * set, then the alloc contains the number of requested nodes. If there i= s an - * allocated node, then the total allocated nodes is in that node. - * - * Return: The total number of nodes allocated - */ -static inline unsigned long mas_allocated(const struct ma_state *mas) -{ - if (!mas->alloc || ((unsigned long)mas->alloc & 0x1)) - return 0; - - return mas->alloc->total; -} - -/* - * mas_set_alloc_req() - Set the requested number of allocations. - * @mas: the maple state - * @count: the number of allocations. - * - * The requested number of allocations is either in the first allocated no= de, - * located in @mas->alloc->request_count, or directly in @mas->alloc if th= ere is - * no allocated node. Set the request either in the node or do the necess= ary - * encoding to store in @mas->alloc directly. - */ -static inline void mas_set_alloc_req(struct ma_state *mas, unsigned long c= ount) -{ - if (!mas->alloc || ((unsigned long)mas->alloc & 0x1)) { - if (!count) - mas->alloc =3D NULL; - else - mas->alloc =3D (struct maple_alloc *)(((count) << 1U) | 1U); - return; - } - - mas->alloc->request_count =3D count; -} - -/* - * mas_alloc_req() - get the requested number of allocations. - * @mas: The maple state - * - * The alloc count is either stored directly in @mas, or in - * @mas->alloc->request_count if there is at least one node allocated. De= code - * the request count if it's stored directly in @mas->alloc. - * - * Return: The allocation request count. - */ -static inline unsigned int mas_alloc_req(const struct ma_state *mas) -{ - if ((unsigned long)mas->alloc & 0x1) - return (unsigned long)(mas->alloc) >> 1; - else if (mas->alloc) - return mas->alloc->request_count; - return 0; -} - /* * ma_pivots() - Get a pointer to the maple node pivots. * @node: the maple node @@ -1142,77 +1097,15 @@ static int mas_ascend(struct ma_state *mas) */ static inline struct maple_node *mas_pop_node(struct ma_state *mas) { - struct maple_alloc *ret, *node =3D mas->alloc; - unsigned long total =3D mas_allocated(mas); - unsigned int req =3D mas_alloc_req(mas); + struct maple_node *ret; =20 - /* nothing or a request pending. */ - if (WARN_ON(!total)) + if (WARN_ON_ONCE(!mas->sheaf)) return NULL; =20 - if (total =3D=3D 1) { - /* single allocation in this ma_state */ - mas->alloc =3D NULL; - ret =3D node; - goto single_node; - } - - if (node->node_count =3D=3D 1) { - /* Single allocation in this node. */ - mas->alloc =3D node->slot[0]; - mas->alloc->total =3D node->total - 1; - ret =3D node; - goto new_head; - } - node->total--; - ret =3D node->slot[--node->node_count]; - node->slot[node->node_count] =3D NULL; - -single_node: -new_head: - if (req) { - req++; - mas_set_alloc_req(mas, req); - } - + ret =3D kmem_cache_alloc_from_sheaf(maple_node_cache, GFP_NOWAIT, mas->sh= eaf); memset(ret, 0, sizeof(*ret)); - return (struct maple_node *)ret; -} - -/* - * mas_push_node() - Push a node back on the maple state allocation. - * @mas: The maple state - * @used: The used maple node - * - * Stores the maple node back into @mas->alloc for reuse. Updates allocat= ed and - * requested node count as necessary. - */ -static inline void mas_push_node(struct ma_state *mas, struct maple_node *= used) -{ - struct maple_alloc *reuse =3D (struct maple_alloc *)used; - struct maple_alloc *head =3D mas->alloc; - unsigned long count; - unsigned int requested =3D mas_alloc_req(mas); =20 - count =3D mas_allocated(mas); - - reuse->request_count =3D 0; - reuse->node_count =3D 0; - if (count) { - if (head->node_count < MAPLE_ALLOC_SLOTS) { - head->slot[head->node_count++] =3D reuse; - head->total++; - goto done; - } - reuse->slot[0] =3D head; - reuse->node_count =3D 1; - } - - reuse->total =3D count + 1; - mas->alloc =3D reuse; -done: - if (requested > 1) - mas_set_alloc_req(mas, requested - 1); + return ret; } =20 /* @@ -1222,75 +1115,32 @@ static inline void mas_push_node(struct ma_state *m= as, struct maple_node *used) */ static inline void mas_alloc_nodes(struct ma_state *mas, gfp_t gfp) { - struct maple_alloc *node; - unsigned long allocated =3D mas_allocated(mas); - unsigned int requested =3D mas_alloc_req(mas); - unsigned int count; - void **slots =3D NULL; - unsigned int max_req =3D 0; - - if (!requested) - return; + if (unlikely(mas->sheaf)) { + unsigned long refill =3D mas->node_request; =20 - mas_set_alloc_req(mas, 0); - if (mas->mas_flags & MA_STATE_PREALLOC) { - if (allocated) + if(kmem_cache_sheaf_size(mas->sheaf) >=3D refill) { + mas->node_request =3D 0; return; - WARN_ON(!allocated); - } - - if (!allocated || mas->alloc->node_count =3D=3D MAPLE_ALLOC_SLOTS) { - node =3D (struct maple_alloc *)mt_alloc_one(gfp); - if (!node) - goto nomem_one; - - if (allocated) { - node->slot[0] =3D mas->alloc; - node->node_count =3D 1; - } else { - node->node_count =3D 0; } =20 - mas->alloc =3D node; - node->total =3D ++allocated; - node->request_count =3D 0; - requested--; - } + if (mt_refill_sheaf(gfp, &mas->sheaf, refill)) + goto error; =20 - node =3D mas->alloc; - while (requested) { - max_req =3D MAPLE_ALLOC_SLOTS - node->node_count; - slots =3D (void **)&node->slot[node->node_count]; - max_req =3D min(requested, max_req); - count =3D mt_alloc_bulk(gfp, max_req, slots); - if (!count) - goto nomem_bulk; - - if (node->node_count =3D=3D 0) { - node->slot[0]->node_count =3D 0; - node->slot[0]->request_count =3D 0; - } + mas->node_request =3D 0; + return; + } =20 - node->node_count +=3D count; - allocated +=3D count; - /* find a non-full node*/ - do { - node =3D node->slot[0]; - } while (unlikely(node->node_count =3D=3D MAPLE_ALLOC_SLOTS)); - requested -=3D count; + mas->sheaf =3D mt_get_sheaf(gfp, mas->node_request); + if (likely(mas->sheaf)) { + mas->node_request =3D 0; + return; } - mas->alloc->total =3D allocated; - return; =20 -nomem_bulk: - /* Clean up potential freed allocations on bulk failure */ - memset(slots, 0, max_req * sizeof(unsigned long)); - mas->alloc->total =3D allocated; -nomem_one: - mas_set_alloc_req(mas, requested); +error: =20 mas_set_err(mas, -ENOMEM); } =20 + /* * mas_free() - Free an encoded maple node * @mas: The maple state @@ -1301,42 +1151,7 @@ static inline void mas_alloc_nodes(struct ma_state *= mas, gfp_t gfp) */ static inline void mas_free(struct ma_state *mas, struct maple_enode *used) { - struct maple_node *tmp =3D mte_to_node(used); - - if (mt_in_rcu(mas->tree)) - ma_free_rcu(tmp); - else - mas_push_node(mas, tmp); -} - -/* - * mas_node_count_gfp() - Check if enough nodes are allocated and request = more - * if there is not enough nodes. - * @mas: The maple state - * @count: The number of nodes needed - * @gfp: the gfp flags - */ -static void mas_node_count_gfp(struct ma_state *mas, int count, gfp_t gfp) -{ - unsigned long allocated =3D mas_allocated(mas); - - if (allocated < count) { - mas_set_alloc_req(mas, count - allocated); - mas_alloc_nodes(mas, gfp); - } -} - -/* - * mas_node_count() - Check if enough nodes are allocated and request more= if - * there is not enough nodes. - * @mas: The maple state - * @count: The number of nodes needed - * - * Note: Uses GFP_NOWAIT | __GFP_NOWARN for gfp flags. - */ -static void mas_node_count(struct ma_state *mas, int count) -{ - return mas_node_count_gfp(mas, count, GFP_NOWAIT | __GFP_NOWARN); + ma_free_rcu(mte_to_node(used)); } =20 /* @@ -2511,10 +2326,7 @@ static inline void mas_topiary_node(struct ma_state = *mas, enode =3D tmp_mas->node; tmp =3D mte_to_node(enode); mte_set_node_dead(enode); - if (in_rcu) - ma_free_rcu(tmp); - else - mas_push_node(mas, tmp); + ma_free_rcu(tmp); } =20 /* @@ -4162,7 +3974,7 @@ static inline void mas_wr_prealloc_setup(struct ma_wr= _state *wr_mas) * * Return: Number of nodes required for preallocation. */ -static inline int mas_prealloc_calc(struct ma_wr_state *wr_mas, void *entr= y) +static inline void mas_prealloc_calc(struct ma_wr_state *wr_mas, void *ent= ry) { struct ma_state *mas =3D wr_mas->mas; unsigned char height =3D mas_mt_height(mas); @@ -4208,7 +4020,7 @@ static inline int mas_prealloc_calc(struct ma_wr_stat= e *wr_mas, void *entry) WARN_ON_ONCE(1); } =20 - return ret; + mas->node_request =3D ret; } =20 /* @@ -4269,15 +4081,15 @@ static inline enum store_type mas_wr_store_type(str= uct ma_wr_state *wr_mas) */ static inline void mas_wr_preallocate(struct ma_wr_state *wr_mas, void *en= try) { - int request; + struct ma_state *mas =3D wr_mas->mas; =20 mas_wr_prealloc_setup(wr_mas); - wr_mas->mas->store_type =3D mas_wr_store_type(wr_mas); - request =3D mas_prealloc_calc(wr_mas, entry); - if (!request) + mas->store_type =3D mas_wr_store_type(wr_mas); + mas_prealloc_calc(wr_mas, entry); + if (!mas->node_request) return; =20 - mas_node_count(wr_mas->mas, request); + mas_alloc_nodes(mas, GFP_NOWAIT | __GFP_NOWARN); } =20 /** @@ -5390,7 +5202,6 @@ static inline void mte_destroy_walk(struct maple_enod= e *enode, */ void *mas_store(struct ma_state *mas, void *entry) { - int request; MA_WR_STATE(wr_mas, mas, entry); =20 trace_ma_write(__func__, mas, 0, entry); @@ -5420,11 +5231,11 @@ void *mas_store(struct ma_state *mas, void *entry) return wr_mas.content; } =20 - request =3D mas_prealloc_calc(&wr_mas, entry); - if (!request) + mas_prealloc_calc(&wr_mas, entry); + if (!mas->node_request) goto store; =20 - mas_node_count(mas, request); + mas_alloc_nodes(mas, GFP_NOWAIT | __GFP_NOWARN); if (mas_is_err(mas)) return NULL; =20 @@ -5512,20 +5323,19 @@ EXPORT_SYMBOL_GPL(mas_store_prealloc); int mas_preallocate(struct ma_state *mas, void *entry, gfp_t gfp) { MA_WR_STATE(wr_mas, mas, entry); - int ret =3D 0; - int request; =20 mas_wr_prealloc_setup(&wr_mas); mas->store_type =3D mas_wr_store_type(&wr_mas); - request =3D mas_prealloc_calc(&wr_mas, entry); - if (!request) + mas_prealloc_calc(&wr_mas, entry); + if (!mas->node_request) goto set_flag; =20 mas->mas_flags &=3D ~MA_STATE_PREALLOC; - mas_node_count_gfp(mas, request, gfp); + mas_alloc_nodes(mas, gfp); if (mas_is_err(mas)) { - mas_set_alloc_req(mas, 0); - ret =3D xa_err(mas->node); + int ret =3D xa_err(mas->node); + + mas->node_request =3D 0; mas_destroy(mas); mas_reset(mas); return ret; @@ -5533,7 +5343,7 @@ int mas_preallocate(struct ma_state *mas, void *entry= , gfp_t gfp) =20 set_flag: mas->mas_flags |=3D MA_STATE_PREALLOC; - return ret; + return 0; } EXPORT_SYMBOL_GPL(mas_preallocate); =20 @@ -5547,9 +5357,6 @@ EXPORT_SYMBOL_GPL(mas_preallocate); */ void mas_destroy(struct ma_state *mas) { - struct maple_alloc *node; - unsigned long total; - /* * When using mas_for_each() to insert an expected number of elements, * it is possible that the number inserted is less than the expected @@ -5570,21 +5377,11 @@ void mas_destroy(struct ma_state *mas) } mas->mas_flags &=3D ~(MA_STATE_BULK|MA_STATE_PREALLOC); =20 - total =3D mas_allocated(mas); - while (total) { - node =3D mas->alloc; - mas->alloc =3D node->slot[0]; - if (node->node_count > 1) { - size_t count =3D node->node_count - 1; - - mt_free_bulk(count, (void __rcu **)&node->slot[1]); - total -=3D count; - } - kfree(ma_mnode_ptr(node)); - total--; - } + mas->node_request =3D 0; + if (mas->sheaf) + mt_return_sheaf(mas->sheaf); =20 - mas->alloc =3D NULL; + mas->sheaf =3D NULL; } EXPORT_SYMBOL_GPL(mas_destroy); =20 @@ -5634,7 +5431,8 @@ int mas_expected_entries(struct ma_state *mas, unsign= ed long nr_entries) /* Internal nodes */ nr_nodes +=3D DIV_ROUND_UP(nr_nodes, nonleaf_cap); /* Add working room for split (2 nodes) + new parents */ - mas_node_count_gfp(mas, nr_nodes + 3, GFP_KERNEL); + mas->node_request =3D nr_nodes + 3; + mas_alloc_nodes(mas, GFP_KERNEL); =20 /* Detect if allocations run out */ mas->mas_flags |=3D MA_STATE_PREALLOC; @@ -6281,7 +6079,7 @@ bool mas_nomem(struct ma_state *mas, gfp_t gfp) mas_alloc_nodes(mas, gfp); } =20 - if (!mas_allocated(mas)) + if (!mas->sheaf) return false; =20 mas->status =3D ma_start; @@ -7676,8 +7474,9 @@ void mas_dump(const struct ma_state *mas) =20 pr_err("[%u/%u] index=3D%lx last=3D%lx\n", mas->offset, mas->end, mas->index, mas->last); - pr_err(" min=3D%lx max=3D%lx alloc=3D" PTR_FMT ", depth=3D%u, flags= =3D%x\n", - mas->min, mas->max, mas->alloc, mas->depth, mas->mas_flags); + pr_err(" min=3D%lx max=3D%lx sheaf=3D" PTR_FMT ", request %lu depth= =3D%u, flags=3D%x\n", + mas->min, mas->max, mas->sheaf, mas->node_request, mas->depth, + mas->mas_flags); if (mas->index > mas->last) pr_err("Check index & last\n"); } diff --git a/lib/test_maple_tree.c b/lib/test_maple_tree.c index cb3936595b0d56a9682ff100eba54693a1427829..1848d127eb50650e7cc2b9dfbb1= 5ed93aa889f01 100644 --- a/lib/test_maple_tree.c +++ b/lib/test_maple_tree.c @@ -2746,6 +2746,7 @@ static noinline void __init check_fuzzer(struct maple= _tree *mt) mtree_test_erase(mt, ULONG_MAX - 10); } =20 +#if 0 /* duplicate the tree with a specific gap */ static noinline void __init check_dup_gaps(struct maple_tree *mt, unsigned long nr_entries, bool zero_start, @@ -2770,6 +2771,7 @@ static noinline void __init check_dup_gaps(struct map= le_tree *mt, mtree_store_range(mt, i*10, (i+1)*10 - gap, xa_mk_value(i), GFP_KERNEL); =20 + mt_dump(mt, mt_dump_dec); mt_init_flags(&newmt, MT_FLAGS_ALLOC_RANGE | MT_FLAGS_LOCK_EXTERN); mt_set_non_kernel(99999); down_write(&newmt_lock); @@ -2779,9 +2781,12 @@ static noinline void __init check_dup_gaps(struct ma= ple_tree *mt, =20 rcu_read_lock(); mas_for_each(&mas, tmp, ULONG_MAX) { + printk("%lu nodes %lu\n", mas.index, + kmem_cache_sheaf_count(newmas.sheaf)); newmas.index =3D mas.index; newmas.last =3D mas.last; mas_store(&newmas, tmp); + mt_dump(&newmt, mt_dump_dec); } rcu_read_unlock(); mas_destroy(&newmas); @@ -2878,6 +2883,7 @@ static noinline void __init check_dup(struct maple_tr= ee *mt) cond_resched(); } } +#endif =20 static noinline void __init check_bnode_min_spanning(struct maple_tree *mt) { @@ -4077,9 +4083,11 @@ static int __init maple_tree_seed(void) check_fuzzer(&tree); mtree_destroy(&tree); =20 +#if 0 mt_init_flags(&tree, MT_FLAGS_ALLOC_RANGE); check_dup(&tree); mtree_destroy(&tree); +#endif =20 mt_init_flags(&tree, MT_FLAGS_ALLOC_RANGE); check_bnode_min_spanning(&tree); diff --git a/tools/testing/radix-tree/maple.c b/tools/testing/radix-tree/ma= ple.c index 7fe91f24849b35723ec6aadbe45ec7d2abedcc11..da3e03d73b52162dab6fa5c368a= d7b71b9e58521 100644 --- a/tools/testing/radix-tree/maple.c +++ b/tools/testing/radix-tree/maple.c @@ -57,430 +57,6 @@ struct rcu_reader_struct { struct rcu_test_struct2 *test; }; =20 -static int get_alloc_node_count(struct ma_state *mas) -{ - int count =3D 1; - struct maple_alloc *node =3D mas->alloc; - - if (!node || ((unsigned long)node & 0x1)) - return 0; - while (node->node_count) { - count +=3D node->node_count; - node =3D node->slot[0]; - } - return count; -} - -static void check_mas_alloc_node_count(struct ma_state *mas) -{ - mas_node_count_gfp(mas, MAPLE_ALLOC_SLOTS + 1, GFP_KERNEL); - mas_node_count_gfp(mas, MAPLE_ALLOC_SLOTS + 3, GFP_KERNEL); - MT_BUG_ON(mas->tree, get_alloc_node_count(mas) !=3D mas->alloc->total); - mas_destroy(mas); -} - -/* - * check_new_node() - Check the creation of new nodes and error path - * verification. - */ -static noinline void __init check_new_node(struct maple_tree *mt) -{ - - struct maple_node *mn, *mn2, *mn3; - struct maple_alloc *smn; - struct maple_node *nodes[100]; - int i, j, total; - - MA_STATE(mas, mt, 0, 0); - - check_mas_alloc_node_count(&mas); - - /* Try allocating 3 nodes */ - mtree_lock(mt); - mt_set_non_kernel(0); - /* request 3 nodes to be allocated. */ - mas_node_count(&mas, 3); - /* Allocation request of 3. */ - MT_BUG_ON(mt, mas_alloc_req(&mas) !=3D 3); - /* Allocate failed. */ - MT_BUG_ON(mt, mas.node !=3D MA_ERROR(-ENOMEM)); - MT_BUG_ON(mt, !mas_nomem(&mas, GFP_KERNEL)); - - MT_BUG_ON(mt, mas_allocated(&mas) !=3D 3); - mn =3D mas_pop_node(&mas); - MT_BUG_ON(mt, not_empty(mn)); - MT_BUG_ON(mt, mn =3D=3D NULL); - MT_BUG_ON(mt, mas.alloc =3D=3D NULL); - MT_BUG_ON(mt, mas.alloc->slot[0] =3D=3D NULL); - mas_push_node(&mas, mn); - mas_reset(&mas); - mas_destroy(&mas); - mtree_unlock(mt); - - - /* Try allocating 1 node, then 2 more */ - mtree_lock(mt); - /* Set allocation request to 1. */ - mas_set_alloc_req(&mas, 1); - /* Check Allocation request of 1. */ - MT_BUG_ON(mt, mas_alloc_req(&mas) !=3D 1); - mas_set_err(&mas, -ENOMEM); - /* Validate allocation request. */ - MT_BUG_ON(mt, !mas_nomem(&mas, GFP_KERNEL)); - /* Eat the requested node. */ - mn =3D mas_pop_node(&mas); - MT_BUG_ON(mt, not_empty(mn)); - MT_BUG_ON(mt, mn =3D=3D NULL); - MT_BUG_ON(mt, mn->slot[0] !=3D NULL); - MT_BUG_ON(mt, mn->slot[1] !=3D NULL); - MT_BUG_ON(mt, mas_allocated(&mas) !=3D 0); - - mn->parent =3D ma_parent_ptr(mn); - ma_free_rcu(mn); - mas.status =3D ma_start; - mas_destroy(&mas); - /* Allocate 3 nodes, will fail. */ - mas_node_count(&mas, 3); - /* Drop the lock and allocate 3 nodes. */ - mas_nomem(&mas, GFP_KERNEL); - /* Ensure 3 are allocated. */ - MT_BUG_ON(mt, mas_allocated(&mas) !=3D 3); - /* Allocation request of 0. */ - MT_BUG_ON(mt, mas_alloc_req(&mas) !=3D 0); - - MT_BUG_ON(mt, mas.alloc =3D=3D NULL); - MT_BUG_ON(mt, mas.alloc->slot[0] =3D=3D NULL); - MT_BUG_ON(mt, mas.alloc->slot[1] =3D=3D NULL); - /* Ensure we counted 3. */ - MT_BUG_ON(mt, mas_allocated(&mas) !=3D 3); - /* Free. */ - mas_reset(&mas); - mas_destroy(&mas); - - /* Set allocation request to 1. */ - mas_set_alloc_req(&mas, 1); - MT_BUG_ON(mt, mas_alloc_req(&mas) !=3D 1); - mas_set_err(&mas, -ENOMEM); - /* Validate allocation request. */ - MT_BUG_ON(mt, !mas_nomem(&mas, GFP_KERNEL)); - MT_BUG_ON(mt, mas_allocated(&mas) !=3D 1); - /* Check the node is only one node. */ - mn =3D mas_pop_node(&mas); - MT_BUG_ON(mt, not_empty(mn)); - MT_BUG_ON(mt, mas_allocated(&mas) !=3D 0); - MT_BUG_ON(mt, mn =3D=3D NULL); - MT_BUG_ON(mt, mn->slot[0] !=3D NULL); - MT_BUG_ON(mt, mn->slot[1] !=3D NULL); - MT_BUG_ON(mt, mas_allocated(&mas) !=3D 0); - mas_push_node(&mas, mn); - MT_BUG_ON(mt, mas_allocated(&mas) !=3D 1); - MT_BUG_ON(mt, mas.alloc->node_count); - - mas_set_alloc_req(&mas, 2); /* request 2 more. */ - MT_BUG_ON(mt, mas_alloc_req(&mas) !=3D 2); - mas_set_err(&mas, -ENOMEM); - MT_BUG_ON(mt, !mas_nomem(&mas, GFP_KERNEL)); - MT_BUG_ON(mt, mas_allocated(&mas) !=3D 3); - MT_BUG_ON(mt, mas.alloc =3D=3D NULL); - MT_BUG_ON(mt, mas.alloc->slot[0] =3D=3D NULL); - MT_BUG_ON(mt, mas.alloc->slot[1] =3D=3D NULL); - for (i =3D 2; i >=3D 0; i--) { - mn =3D mas_pop_node(&mas); - MT_BUG_ON(mt, mas_allocated(&mas) !=3D i); - MT_BUG_ON(mt, !mn); - MT_BUG_ON(mt, not_empty(mn)); - mn->parent =3D ma_parent_ptr(mn); - ma_free_rcu(mn); - } - - total =3D 64; - mas_set_alloc_req(&mas, total); /* request 2 more. */ - MT_BUG_ON(mt, mas_alloc_req(&mas) !=3D total); - mas_set_err(&mas, -ENOMEM); - MT_BUG_ON(mt, !mas_nomem(&mas, GFP_KERNEL)); - for (i =3D total; i > 0; i--) { - unsigned int e =3D 0; /* expected node_count */ - - if (!MAPLE_32BIT) { - if (i >=3D 35) - e =3D i - 34; - else if (i >=3D 5) - e =3D i - 4; - else if (i >=3D 2) - e =3D i - 1; - } else { - if (i >=3D 4) - e =3D i - 3; - else if (i >=3D 1) - e =3D i - 1; - else - e =3D 0; - } - - MT_BUG_ON(mt, mas.alloc->node_count !=3D e); - mn =3D mas_pop_node(&mas); - MT_BUG_ON(mt, not_empty(mn)); - MT_BUG_ON(mt, mas_allocated(&mas) !=3D i - 1); - MT_BUG_ON(mt, !mn); - mn->parent =3D ma_parent_ptr(mn); - ma_free_rcu(mn); - } - - total =3D 100; - for (i =3D 1; i < total; i++) { - mas_set_alloc_req(&mas, i); - mas_set_err(&mas, -ENOMEM); - MT_BUG_ON(mt, !mas_nomem(&mas, GFP_KERNEL)); - for (j =3D i; j > 0; j--) { - mn =3D mas_pop_node(&mas); - MT_BUG_ON(mt, mas_allocated(&mas) !=3D j - 1); - MT_BUG_ON(mt, !mn); - MT_BUG_ON(mt, not_empty(mn)); - mas_push_node(&mas, mn); - MT_BUG_ON(mt, mas_allocated(&mas) !=3D j); - mn =3D mas_pop_node(&mas); - MT_BUG_ON(mt, not_empty(mn)); - MT_BUG_ON(mt, mas_allocated(&mas) !=3D j - 1); - mn->parent =3D ma_parent_ptr(mn); - ma_free_rcu(mn); - } - MT_BUG_ON(mt, mas_allocated(&mas) !=3D 0); - - mas_set_alloc_req(&mas, i); - mas_set_err(&mas, -ENOMEM); - MT_BUG_ON(mt, !mas_nomem(&mas, GFP_KERNEL)); - for (j =3D 0; j <=3D i/2; j++) { - MT_BUG_ON(mt, mas_allocated(&mas) !=3D i - j); - nodes[j] =3D mas_pop_node(&mas); - MT_BUG_ON(mt, mas_allocated(&mas) !=3D i - j - 1); - } - - while (j) { - j--; - mas_push_node(&mas, nodes[j]); - MT_BUG_ON(mt, mas_allocated(&mas) !=3D i - j); - } - MT_BUG_ON(mt, mas_allocated(&mas) !=3D i); - for (j =3D 0; j <=3D i/2; j++) { - MT_BUG_ON(mt, mas_allocated(&mas) !=3D i - j); - mn =3D mas_pop_node(&mas); - MT_BUG_ON(mt, not_empty(mn)); - mn->parent =3D ma_parent_ptr(mn); - ma_free_rcu(mn); - MT_BUG_ON(mt, mas_allocated(&mas) !=3D i - j - 1); - } - mas_reset(&mas); - MT_BUG_ON(mt, mas_nomem(&mas, GFP_KERNEL)); - mas_destroy(&mas); - - } - - /* Set allocation request. */ - total =3D 500; - mas_node_count(&mas, total); - /* Drop the lock and allocate the nodes. */ - mas_nomem(&mas, GFP_KERNEL); - MT_BUG_ON(mt, !mas.alloc); - i =3D 1; - smn =3D mas.alloc; - while (i < total) { - for (j =3D 0; j < MAPLE_ALLOC_SLOTS; j++) { - i++; - MT_BUG_ON(mt, !smn->slot[j]); - if (i =3D=3D total) - break; - } - smn =3D smn->slot[0]; /* next. */ - } - MT_BUG_ON(mt, mas_allocated(&mas) !=3D total); - mas_reset(&mas); - mas_destroy(&mas); /* Free. */ - - MT_BUG_ON(mt, mas_allocated(&mas) !=3D 0); - for (i =3D 1; i < 128; i++) { - mas_node_count(&mas, i); /* Request */ - mas_nomem(&mas, GFP_KERNEL); /* Fill request */ - MT_BUG_ON(mt, mas_allocated(&mas) !=3D i); /* check request filled */ - for (j =3D i; j > 0; j--) { /*Free the requests */ - mn =3D mas_pop_node(&mas); /* get the next node. */ - MT_BUG_ON(mt, mn =3D=3D NULL); - MT_BUG_ON(mt, not_empty(mn)); - mn->parent =3D ma_parent_ptr(mn); - ma_free_rcu(mn); - } - MT_BUG_ON(mt, mas_allocated(&mas) !=3D 0); - } - - for (i =3D 1; i < MAPLE_NODE_MASK + 1; i++) { - MA_STATE(mas2, mt, 0, 0); - mas_node_count(&mas, i); /* Request */ - mas_nomem(&mas, GFP_KERNEL); /* Fill request */ - MT_BUG_ON(mt, mas_allocated(&mas) !=3D i); /* check request filled */ - for (j =3D 1; j <=3D i; j++) { /* Move the allocations to mas2 */ - mn =3D mas_pop_node(&mas); /* get the next node. */ - MT_BUG_ON(mt, mn =3D=3D NULL); - MT_BUG_ON(mt, not_empty(mn)); - mas_push_node(&mas2, mn); - MT_BUG_ON(mt, mas_allocated(&mas2) !=3D j); - } - MT_BUG_ON(mt, mas_allocated(&mas) !=3D 0); - MT_BUG_ON(mt, mas_allocated(&mas2) !=3D i); - - for (j =3D i; j > 0; j--) { /*Free the requests */ - MT_BUG_ON(mt, mas_allocated(&mas2) !=3D j); - mn =3D mas_pop_node(&mas2); /* get the next node. */ - MT_BUG_ON(mt, mn =3D=3D NULL); - MT_BUG_ON(mt, not_empty(mn)); - mn->parent =3D ma_parent_ptr(mn); - ma_free_rcu(mn); - } - MT_BUG_ON(mt, mas_allocated(&mas2) !=3D 0); - } - - - MT_BUG_ON(mt, mas_allocated(&mas) !=3D 0); - mas_node_count(&mas, MAPLE_ALLOC_SLOTS + 1); /* Request */ - MT_BUG_ON(mt, mas.node !=3D MA_ERROR(-ENOMEM)); - MT_BUG_ON(mt, !mas_nomem(&mas, GFP_KERNEL)); - MT_BUG_ON(mt, mas_allocated(&mas) !=3D MAPLE_ALLOC_SLOTS + 1); - MT_BUG_ON(mt, mas.alloc->node_count !=3D MAPLE_ALLOC_SLOTS); - - mn =3D mas_pop_node(&mas); /* get the next node. */ - MT_BUG_ON(mt, mn =3D=3D NULL); - MT_BUG_ON(mt, not_empty(mn)); - MT_BUG_ON(mt, mas_allocated(&mas) !=3D MAPLE_ALLOC_SLOTS); - MT_BUG_ON(mt, mas.alloc->node_count !=3D MAPLE_ALLOC_SLOTS - 1); - - mas_push_node(&mas, mn); - MT_BUG_ON(mt, mas_allocated(&mas) !=3D MAPLE_ALLOC_SLOTS + 1); - MT_BUG_ON(mt, mas.alloc->node_count !=3D MAPLE_ALLOC_SLOTS); - - /* Check the limit of pop/push/pop */ - mas_node_count(&mas, MAPLE_ALLOC_SLOTS + 2); /* Request */ - MT_BUG_ON(mt, mas_alloc_req(&mas) !=3D 1); - MT_BUG_ON(mt, mas.node !=3D MA_ERROR(-ENOMEM)); - MT_BUG_ON(mt, !mas_nomem(&mas, GFP_KERNEL)); - MT_BUG_ON(mt, mas_alloc_req(&mas)); - MT_BUG_ON(mt, mas.alloc->node_count !=3D 1); - MT_BUG_ON(mt, mas_allocated(&mas) !=3D MAPLE_ALLOC_SLOTS + 2); - mn =3D mas_pop_node(&mas); - MT_BUG_ON(mt, not_empty(mn)); - MT_BUG_ON(mt, mas_allocated(&mas) !=3D MAPLE_ALLOC_SLOTS + 1); - MT_BUG_ON(mt, mas.alloc->node_count !=3D MAPLE_ALLOC_SLOTS); - mas_push_node(&mas, mn); - MT_BUG_ON(mt, mas.alloc->node_count !=3D 1); - MT_BUG_ON(mt, mas_allocated(&mas) !=3D MAPLE_ALLOC_SLOTS + 2); - mn =3D mas_pop_node(&mas); - MT_BUG_ON(mt, not_empty(mn)); - mn->parent =3D ma_parent_ptr(mn); - ma_free_rcu(mn); - for (i =3D 1; i <=3D MAPLE_ALLOC_SLOTS + 1; i++) { - mn =3D mas_pop_node(&mas); - MT_BUG_ON(mt, not_empty(mn)); - mn->parent =3D ma_parent_ptr(mn); - ma_free_rcu(mn); - } - MT_BUG_ON(mt, mas_allocated(&mas) !=3D 0); - - - for (i =3D 3; i < MAPLE_NODE_MASK * 3; i++) { - mas.node =3D MA_ERROR(-ENOMEM); - mas_node_count(&mas, i); /* Request */ - mas_nomem(&mas, GFP_KERNEL); /* Fill request */ - mn =3D mas_pop_node(&mas); /* get the next node. */ - mas_push_node(&mas, mn); /* put it back */ - mas_destroy(&mas); - - mas.node =3D MA_ERROR(-ENOMEM); - mas_node_count(&mas, i); /* Request */ - mas_nomem(&mas, GFP_KERNEL); /* Fill request */ - mn =3D mas_pop_node(&mas); /* get the next node. */ - mn2 =3D mas_pop_node(&mas); /* get the next node. */ - mas_push_node(&mas, mn); /* put them back */ - mas_push_node(&mas, mn2); - mas_destroy(&mas); - - mas.node =3D MA_ERROR(-ENOMEM); - mas_node_count(&mas, i); /* Request */ - mas_nomem(&mas, GFP_KERNEL); /* Fill request */ - mn =3D mas_pop_node(&mas); /* get the next node. */ - mn2 =3D mas_pop_node(&mas); /* get the next node. */ - mn3 =3D mas_pop_node(&mas); /* get the next node. */ - mas_push_node(&mas, mn); /* put them back */ - mas_push_node(&mas, mn2); - mas_push_node(&mas, mn3); - mas_destroy(&mas); - - mas.node =3D MA_ERROR(-ENOMEM); - mas_node_count(&mas, i); /* Request */ - mas_nomem(&mas, GFP_KERNEL); /* Fill request */ - mn =3D mas_pop_node(&mas); /* get the next node. */ - mn->parent =3D ma_parent_ptr(mn); - ma_free_rcu(mn); - mas_destroy(&mas); - - mas.node =3D MA_ERROR(-ENOMEM); - mas_node_count(&mas, i); /* Request */ - mas_nomem(&mas, GFP_KERNEL); /* Fill request */ - mn =3D mas_pop_node(&mas); /* get the next node. */ - mn->parent =3D ma_parent_ptr(mn); - ma_free_rcu(mn); - mn =3D mas_pop_node(&mas); /* get the next node. */ - mn->parent =3D ma_parent_ptr(mn); - ma_free_rcu(mn); - mn =3D mas_pop_node(&mas); /* get the next node. */ - mn->parent =3D ma_parent_ptr(mn); - ma_free_rcu(mn); - mas_destroy(&mas); - } - - mas.node =3D MA_ERROR(-ENOMEM); - mas_node_count(&mas, 5); /* Request */ - mas_nomem(&mas, GFP_KERNEL); /* Fill request */ - MT_BUG_ON(mt, mas_allocated(&mas) !=3D 5); - mas.node =3D MA_ERROR(-ENOMEM); - mas_node_count(&mas, 10); /* Request */ - mas_nomem(&mas, GFP_KERNEL); /* Fill request */ - mas.status =3D ma_start; - MT_BUG_ON(mt, mas_allocated(&mas) !=3D 10); - mas_destroy(&mas); - - mas.node =3D MA_ERROR(-ENOMEM); - mas_node_count(&mas, MAPLE_ALLOC_SLOTS - 1); /* Request */ - mas_nomem(&mas, GFP_KERNEL); /* Fill request */ - MT_BUG_ON(mt, mas_allocated(&mas) !=3D MAPLE_ALLOC_SLOTS - 1); - mas.node =3D MA_ERROR(-ENOMEM); - mas_node_count(&mas, 10 + MAPLE_ALLOC_SLOTS - 1); /* Request */ - mas_nomem(&mas, GFP_KERNEL); /* Fill request */ - mas.status =3D ma_start; - MT_BUG_ON(mt, mas_allocated(&mas) !=3D 10 + MAPLE_ALLOC_SLOTS - 1); - mas_destroy(&mas); - - mas.node =3D MA_ERROR(-ENOMEM); - mas_node_count(&mas, MAPLE_ALLOC_SLOTS + 1); /* Request */ - mas_nomem(&mas, GFP_KERNEL); /* Fill request */ - MT_BUG_ON(mt, mas_allocated(&mas) !=3D MAPLE_ALLOC_SLOTS + 1); - mas.node =3D MA_ERROR(-ENOMEM); - mas_node_count(&mas, MAPLE_ALLOC_SLOTS * 2 + 2); /* Request */ - mas_nomem(&mas, GFP_KERNEL); /* Fill request */ - mas.status =3D ma_start; - MT_BUG_ON(mt, mas_allocated(&mas) !=3D MAPLE_ALLOC_SLOTS * 2 + 2); - mas_destroy(&mas); - - mas.node =3D MA_ERROR(-ENOMEM); - mas_node_count(&mas, MAPLE_ALLOC_SLOTS * 2 + 1); /* Request */ - mas_nomem(&mas, GFP_KERNEL); /* Fill request */ - MT_BUG_ON(mt, mas_allocated(&mas) !=3D MAPLE_ALLOC_SLOTS * 2 + 1); - mas.node =3D MA_ERROR(-ENOMEM); - mas_node_count(&mas, MAPLE_ALLOC_SLOTS * 3 + 2); /* Request */ - mas_nomem(&mas, GFP_KERNEL); /* Fill request */ - mas.status =3D ma_start; - MT_BUG_ON(mt, mas_allocated(&mas) !=3D MAPLE_ALLOC_SLOTS * 3 + 2); - mas_destroy(&mas); - - mtree_unlock(mt); -} - /* * Check erasing including RCU. */ @@ -35452,8 +35028,7 @@ static void check_dfs_preorder(struct maple_tree *m= t) mt_init_flags(mt, MT_FLAGS_ALLOC_RANGE); mas_reset(&mas); mt_zero_nr_tallocated(); - mt_set_non_kernel(200); - mas_expected_entries(&mas, max); + mt_set_non_kernel(1000); for (count =3D 0; count <=3D max; count++) { mas.index =3D mas.last =3D count; mas_store(&mas, xa_mk_value(count)); @@ -35518,6 +35093,13 @@ static unsigned char get_vacant_height(struct ma_w= r_state *wr_mas, void *entry) return vacant_height; } =20 +static int mas_allocated(struct ma_state *mas) +{ + if (mas->sheaf) + return kmem_cache_sheaf_size(mas->sheaf); + + return 0; +} /* Preallocation testing */ static noinline void __init check_prealloc(struct maple_tree *mt) { @@ -35536,7 +35118,10 @@ static noinline void __init check_prealloc(struct = maple_tree *mt) =20 /* Spanning store */ mas_set_range(&mas, 470, 500); - MT_BUG_ON(mt, mas_preallocate(&mas, ptr, GFP_KERNEL) !=3D 0); + + mas_wr_preallocate(&wr_mas, ptr); + MT_BUG_ON(mt, mas.store_type !=3D wr_spanning_store); + MT_BUG_ON(mt, mas_is_err(&mas)); allocated =3D mas_allocated(&mas); height =3D mas_mt_height(&mas); vacant_height =3D get_vacant_height(&wr_mas, ptr); @@ -35546,6 +35131,7 @@ static noinline void __init check_prealloc(struct m= aple_tree *mt) allocated =3D mas_allocated(&mas); MT_BUG_ON(mt, allocated !=3D 0); =20 + mas_wr_preallocate(&wr_mas, ptr); MT_BUG_ON(mt, mas_preallocate(&mas, ptr, GFP_KERNEL) !=3D 0); allocated =3D mas_allocated(&mas); height =3D mas_mt_height(&mas); @@ -35586,20 +35172,6 @@ static noinline void __init check_prealloc(struct = maple_tree *mt) mn->parent =3D ma_parent_ptr(mn); ma_free_rcu(mn); =20 - MT_BUG_ON(mt, mas_preallocate(&mas, ptr, GFP_KERNEL) !=3D 0); - allocated =3D mas_allocated(&mas); - height =3D mas_mt_height(&mas); - vacant_height =3D get_vacant_height(&wr_mas, ptr); - MT_BUG_ON(mt, allocated !=3D 1 + (height - vacant_height) * 3); - mn =3D mas_pop_node(&mas); - MT_BUG_ON(mt, mas_allocated(&mas) !=3D allocated - 1); - mas_push_node(&mas, mn); - MT_BUG_ON(mt, mas_allocated(&mas) !=3D allocated); - MT_BUG_ON(mt, mas_preallocate(&mas, ptr, GFP_KERNEL) !=3D 0); - mas_destroy(&mas); - allocated =3D mas_allocated(&mas); - MT_BUG_ON(mt, allocated !=3D 0); - MT_BUG_ON(mt, mas_preallocate(&mas, ptr, GFP_KERNEL) !=3D 0); allocated =3D mas_allocated(&mas); height =3D mas_mt_height(&mas); @@ -36400,11 +35972,17 @@ static void check_nomem_writer_race(struct maple_= tree *mt) check_load(mt, 6, xa_mk_value(0xC)); mtree_unlock(mt); =20 + mt_set_non_kernel(0); /* test for the same race but with mas_store_gfp() */ mtree_store_range(mt, 0, 5, xa_mk_value(0xA), GFP_KERNEL); mtree_store_range(mt, 6, 10, NULL, GFP_KERNEL); =20 mas_set_range(&mas, 0, 5); + + /* setup writer 2 that will trigger the race condition */ + mt_set_private(mt); + mt_set_callback(writer2); + mtree_lock(mt); mas_store_gfp(&mas, NULL, GFP_KERNEL); =20 @@ -36546,10 +36124,6 @@ void farmer_tests(void) check_erase_testset(&tree); mtree_destroy(&tree); =20 - mt_init_flags(&tree, 0); - check_new_node(&tree); - mtree_destroy(&tree); - if (!MAPLE_32BIT) { mt_init_flags(&tree, MT_FLAGS_ALLOC_RANGE); check_rcu_simulated(&tree); diff --git a/tools/testing/shared/linux.c b/tools/testing/shared/linux.c index 4ceff7969b78cf8e33cd1e021c68bc9f8a02a7a1..8c72571559583759456c2b469a2= abc2611117c13 100644 --- a/tools/testing/shared/linux.c +++ b/tools/testing/shared/linux.c @@ -64,7 +64,8 @@ void *kmem_cache_alloc_lru(struct kmem_cache *cachep, str= uct list_lru *lru, =20 if (!(gfp & __GFP_DIRECT_RECLAIM)) { if (!cachep->non_kernel) { - cachep->exec_callback =3D true; + if (cachep->callback) + cachep->exec_callback =3D true; return NULL; } =20 @@ -210,6 +211,8 @@ int kmem_cache_alloc_bulk(struct kmem_cache *cachep, gf= p_t gfp, size_t size, for (i =3D 0; i < size; i++) __kmem_cache_free_locked(cachep, p[i]); pthread_mutex_unlock(&cachep->lock); + if (cachep->callback) + cachep->exec_callback =3D true; return 0; } =20 --=20 2.51.0 From nobody Sun Dec 14 06:23:25 2025 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A00BA30FF2A for ; Mon, 1 Sep 2025 11:09:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756724982; cv=none; b=ZvJLUHRE92mtm0ts/GK4bGTupaJsi//GtI/yoPbQrdeKYMDQDSq7rY2z3zl07l1E87Avkrz3G/V+yjzcWh6/oWpVAuVwUN8RgyqoV5aRLDHtzgynUMc7TLEw3CuVGqmu16xhwGGRqOyhQyZnJzkMumsXze20RdTCUrnDezpqP98= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756724982; c=relaxed/simple; bh=qYYD8QB3+ZJl58441sYK9bECFC/e2CHKoe9h07tdYcE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=dnXpMot7E9XOEi8Jj540XUDEixeBPAg8MXpz78n3c/Zu4dErSQE/KZW2q8zdISYThcgR8wU/SYl1gQDUpWWqK9Kx0qN7DQHpnxoe0OfZTBQErJLBtdvTzySDzbKuD1zcrqE5hUz2Ctk769NfeDVaMmGouOl5oV9h2UEJCSD9c+s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz; spf=pass smtp.mailfrom=suse.cz; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=xBn0jzYN; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=L0+y4jgm; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=xBn0jzYN; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=L0+y4jgm; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.cz Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="xBn0jzYN"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="L0+y4jgm"; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="xBn0jzYN"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="L0+y4jgm" Received: from imap1.dmz-prg2.suse.org (unknown [10.150.64.97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 102C82117D; Mon, 1 Sep 2025 11:08:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1756724934; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/Yl47dy68ehTR4F5YItfKWugcgsVIsFegWRdmZS6Qxw=; b=xBn0jzYNn+cWvKcxEb5yLy9T7DeTegxMy0R03jKU+MfoNbZl71MKg1m7TEDKA/fyS2+ewb 3qw9mwf1avZJU/Uwf7afiMvcrYAk7K8TUbDaJMkrzWNZZj0Sg9vqZUFanYxMcn11qU4ItF lBmQIO/JaIJnTz3KV59IzOgusHOcvgM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1756724934; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/Yl47dy68ehTR4F5YItfKWugcgsVIsFegWRdmZS6Qxw=; b=L0+y4jgmKzcw6eFw93KihWmpq/qgGb6cdLN5yAiFn9CpX0iwJ97Bqa0iZ1w621WqRu86qA sXOYCLl6NGb93dCQ== Authentication-Results: smtp-out1.suse.de; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1756724934; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/Yl47dy68ehTR4F5YItfKWugcgsVIsFegWRdmZS6Qxw=; b=xBn0jzYNn+cWvKcxEb5yLy9T7DeTegxMy0R03jKU+MfoNbZl71MKg1m7TEDKA/fyS2+ewb 3qw9mwf1avZJU/Uwf7afiMvcrYAk7K8TUbDaJMkrzWNZZj0Sg9vqZUFanYxMcn11qU4ItF lBmQIO/JaIJnTz3KV59IzOgusHOcvgM= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1756724934; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=/Yl47dy68ehTR4F5YItfKWugcgsVIsFegWRdmZS6Qxw=; b=L0+y4jgmKzcw6eFw93KihWmpq/qgGb6cdLN5yAiFn9CpX0iwJ97Bqa0iZ1w621WqRu86qA sXOYCLl6NGb93dCQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id ED9E613A3E; Mon, 1 Sep 2025 11:08:53 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id AM+6OcV+tWjtDgAAD6G6ig (envelope-from ); Mon, 01 Sep 2025 11:08:53 +0000 From: Vlastimil Babka Date: Mon, 01 Sep 2025 13:09:01 +0200 Subject: [PATCH 11/12] maple_tree: Add single node allocation support to maple state Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250901-maple-sheaves-v1-11-d6a1166b53f2@suse.cz> References: <20250901-maple-sheaves-v1-0-d6a1166b53f2@suse.cz> In-Reply-To: <20250901-maple-sheaves-v1-0-d6a1166b53f2@suse.cz> To: "Liam R. Howlett" , Matthew Wilcox , Lorenzo Stoakes , Jann Horn , Pedro Falcato , Suren Baghdasaryan Cc: Harry Yoo , Andrew Morton , maple-tree@lists.infradead.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, Vlastimil Babka , "Liam R. Howlett" X-Mailer: b4 0.14.2 X-Spam-Level: X-Spamd-Result: default: False [-4.30 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_SHORT(-0.20)[-0.997]; MIME_GOOD(-0.10)[text/plain]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_TLS_ALL(0.00)[]; RCPT_COUNT_TWELVE(0.00)[14]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_SIGNED(0.00)[suse.cz:s=susede2_rsa,suse.cz:s=susede2_ed25519]; FROM_HAS_DN(0.00)[]; R_RATELIMIT(0.00)[to_ip_from(RLzz977skxseo48jckwk35q6sc)]; FROM_EQ_ENVFROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; RCVD_COUNT_TWO(0.00)[2]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.cz:email,suse.cz:mid,oracle.com:email,imap1.dmz-prg2.suse.org:helo] X-Spam-Flag: NO X-Spam-Score: -4.30 From: "Liam R. Howlett" The fast path through a write will require replacing a single node in the tree. Using a sheaf (32 nodes) is too heavy for the fast path, so special case the node store operation by just allocating one node in the maple state. Signed-off-by: Liam R. Howlett Signed-off-by: Vlastimil Babka --- include/linux/maple_tree.h | 4 +++- lib/maple_tree.c | 47 +++++++++++++++++++++++++++++++++++-= ---- tools/testing/radix-tree/maple.c | 9 ++++++-- 3 files changed, 51 insertions(+), 9 deletions(-) diff --git a/include/linux/maple_tree.h b/include/linux/maple_tree.h index 166fd67e00d882b1e6de1f80c1b590bba7497cd3..562a1e9e5132b5b1fa8f8402a7c= add8abb65e323 100644 --- a/include/linux/maple_tree.h +++ b/include/linux/maple_tree.h @@ -443,6 +443,7 @@ struct ma_state { unsigned long min; /* The minimum index of this node - implied pivot min= */ unsigned long max; /* The maximum index of this node - implied pivot max= */ struct slab_sheaf *sheaf; /* Allocated nodes for this operation */ + struct maple_node *alloc; /* allocated nodes */ unsigned long node_request; enum maple_status status; /* The status of the state (active, start, none= , etc) */ unsigned char depth; /* depth of tree descent during write */ @@ -491,8 +492,9 @@ struct ma_wr_state { .status =3D ma_start, \ .min =3D 0, \ .max =3D ULONG_MAX, \ - .node_request=3D 0, \ .sheaf =3D NULL, \ + .alloc =3D NULL, \ + .node_request=3D 0, \ .mas_flags =3D 0, \ .store_type =3D wr_invalid, \ } diff --git a/lib/maple_tree.c b/lib/maple_tree.c index cfe80c50a97e3118eefc24275fbcd2eec5e6e6e8..61a322f945c28f5c3297c506923= f00bcce5c7bca 100644 --- a/lib/maple_tree.c +++ b/lib/maple_tree.c @@ -1095,16 +1095,23 @@ static int mas_ascend(struct ma_state *mas) * * Return: A pointer to a maple node. */ -static inline struct maple_node *mas_pop_node(struct ma_state *mas) +static __always_inline struct maple_node *mas_pop_node(struct ma_state *ma= s) { struct maple_node *ret; =20 + if (mas->alloc) { + ret =3D mas->alloc; + mas->alloc =3D NULL; + goto out; + } + if (WARN_ON_ONCE(!mas->sheaf)) return NULL; =20 ret =3D kmem_cache_alloc_from_sheaf(maple_node_cache, GFP_NOWAIT, mas->sh= eaf); - memset(ret, 0, sizeof(*ret)); =20 +out: + memset(ret, 0, sizeof(*ret)); return ret; } =20 @@ -1115,9 +1122,34 @@ static inline struct maple_node *mas_pop_node(struct= ma_state *mas) */ static inline void mas_alloc_nodes(struct ma_state *mas, gfp_t gfp) { - if (unlikely(mas->sheaf)) { - unsigned long refill =3D mas->node_request; + if (!mas->node_request) + return; + + if (mas->node_request =3D=3D 1) { + if (mas->sheaf) + goto use_sheaf; + + if (mas->alloc) + return; =20 + mas->alloc =3D mt_alloc_one(gfp); + if (!mas->alloc) + goto error; + + mas->node_request =3D 0; + return; + } + +use_sheaf: + if (unlikely(mas->alloc)) { + kfree(mas->alloc); + mas->alloc =3D NULL; + } + + if (mas->sheaf) { + unsigned long refill; + + refill =3D mas->node_request; if(kmem_cache_sheaf_size(mas->sheaf) >=3D refill) { mas->node_request =3D 0; return; @@ -5380,8 +5412,11 @@ void mas_destroy(struct ma_state *mas) mas->node_request =3D 0; if (mas->sheaf) mt_return_sheaf(mas->sheaf); - mas->sheaf =3D NULL; + + if (mas->alloc) + kfree(mas->alloc); + mas->alloc =3D NULL; } EXPORT_SYMBOL_GPL(mas_destroy); =20 @@ -6079,7 +6114,7 @@ bool mas_nomem(struct ma_state *mas, gfp_t gfp) mas_alloc_nodes(mas, gfp); } =20 - if (!mas->sheaf) + if (!mas->sheaf && !mas->alloc) return false; =20 mas->status =3D ma_start; diff --git a/tools/testing/radix-tree/maple.c b/tools/testing/radix-tree/ma= ple.c index da3e03d73b52162dab6fa5c368ad7b71b9e58521..89da991e12cd97e44971757ddc1= 05ef46c68ea4c 100644 --- a/tools/testing/radix-tree/maple.c +++ b/tools/testing/radix-tree/maple.c @@ -35095,10 +35095,15 @@ static unsigned char get_vacant_height(struct ma_= wr_state *wr_mas, void *entry) =20 static int mas_allocated(struct ma_state *mas) { + int total =3D 0; + + if (mas->alloc) + total++; + if (mas->sheaf) - return kmem_cache_sheaf_size(mas->sheaf); + total +=3D kmem_cache_sheaf_size(mas->sheaf); =20 - return 0; + return total; } /* Preallocation testing */ static noinline void __init check_prealloc(struct maple_tree *mt) --=20 2.51.0 From nobody Sun Dec 14 06:23:25 2025 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.223.130]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 552AF30FF2A for ; Mon, 1 Sep 2025 11:09:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=195.135.223.130 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756724988; cv=none; b=eFBOakN76Q1/EVx6C8y+nDF+nMJx28zv1HK9Vf6gQ4u+kaWMwu5EciIsZsBrQBRi85Jt802re38GJev/3aHdHAw/TDdbGcG3iYdDJu6LExATwvmPUEpDm8QjgG9RW40IwRB+aaKncJD13lSNdthZH9DbaI1XgICSkkZvb+Gb20M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756724988; c=relaxed/simple; bh=dkkOcEsD7xn5yD67IHEifvMlDweLyNkUukV+EgT6MdE=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=OHGHI7x3lSOk5Fccv5k8cmlPyxqrUTvucDtF24W1NSfmcpaotaeSE8DHFeJod8P5YhfBU5W6YIqNnduW4OD2qkG5EBaGt7L1YsK8EXwpdsaC6LZaXdjX3rJ/GsLSFPndoJxSoF+uaxFV5kWlMzoLQ4kzeRyeysKdH7/4MQubOmE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz; spf=pass smtp.mailfrom=suse.cz; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=nlPUE/e7; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=wxl/KWzB; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b=nlPUE/e7; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b=wxl/KWzB; arc=none smtp.client-ip=195.135.223.130 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=suse.cz Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=suse.cz Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="nlPUE/e7"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="wxl/KWzB"; dkim=pass (1024-bit key) header.d=suse.cz header.i=@suse.cz header.b="nlPUE/e7"; dkim=permerror (0-bit key) header.d=suse.cz header.i=@suse.cz header.b="wxl/KWzB" Received: from imap1.dmz-prg2.suse.org (imap1.dmz-prg2.suse.org [IPv6:2a07:de40:b281:104:10:150:64:97]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 2495F2117F; Mon, 1 Sep 2025 11:08:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1756724934; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1gsZxO7UwDiPyyfTVlyF4n/52yTA0EMnZqzW234FbC8=; b=nlPUE/e7F+X8WrUw6J4qfcaLHa1uKKS4XsuggwRVtF7gTcz6nFif/WwVknP27fkOgF8s+Y c+EmrfkGo3vXsFVUVjzCG1vpuxoslzDrD/1Puf/BAybyr8hg2XeZZ68zHLelYE+w82cAC9 XaT0ZxkP1W+ocEUdOALgCcVJZY1mdRc= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1756724934; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1gsZxO7UwDiPyyfTVlyF4n/52yTA0EMnZqzW234FbC8=; b=wxl/KWzB0/GJpacisYIzHBgqZs9iqq7+mJn12vMwX9zQiiIWqooa/mHXaGDOtTjlqsCTbv FZdwNt8oIPrStaCQ== Authentication-Results: smtp-out1.suse.de; dkim=pass header.d=suse.cz header.s=susede2_rsa header.b="nlPUE/e7"; dkim=pass header.d=suse.cz header.s=susede2_ed25519 header.b="wxl/KWzB" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1756724934; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1gsZxO7UwDiPyyfTVlyF4n/52yTA0EMnZqzW234FbC8=; b=nlPUE/e7F+X8WrUw6J4qfcaLHa1uKKS4XsuggwRVtF7gTcz6nFif/WwVknP27fkOgF8s+Y c+EmrfkGo3vXsFVUVjzCG1vpuxoslzDrD/1Puf/BAybyr8hg2XeZZ68zHLelYE+w82cAC9 XaT0ZxkP1W+ocEUdOALgCcVJZY1mdRc= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1756724934; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1gsZxO7UwDiPyyfTVlyF4n/52yTA0EMnZqzW234FbC8=; b=wxl/KWzB0/GJpacisYIzHBgqZs9iqq7+mJn12vMwX9zQiiIWqooa/mHXaGDOtTjlqsCTbv FZdwNt8oIPrStaCQ== Received: from imap1.dmz-prg2.suse.org (localhost [127.0.0.1]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by imap1.dmz-prg2.suse.org (Postfix) with ESMTPS id 0E11A1378C; Mon, 1 Sep 2025 11:08:54 +0000 (UTC) Received: from dovecot-director2.suse.de ([2a07:de40:b281:106:10:150:64:167]) by imap1.dmz-prg2.suse.org with ESMTPSA id oAwtA8Z+tWjtDgAAD6G6ig (envelope-from ); Mon, 01 Sep 2025 11:08:54 +0000 From: Vlastimil Babka Date: Mon, 01 Sep 2025 13:09:02 +0200 Subject: [PATCH 12/12] maple_tree: Convert forking to use the sheaf interface Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250901-maple-sheaves-v1-12-d6a1166b53f2@suse.cz> References: <20250901-maple-sheaves-v1-0-d6a1166b53f2@suse.cz> In-Reply-To: <20250901-maple-sheaves-v1-0-d6a1166b53f2@suse.cz> To: "Liam R. Howlett" , Matthew Wilcox , Lorenzo Stoakes , Jann Horn , Pedro Falcato , Suren Baghdasaryan Cc: Harry Yoo , Andrew Morton , maple-tree@lists.infradead.org, linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, Vlastimil Babka , "Liam R. Howlett" X-Mailer: b4 0.14.2 X-Spamd-Result: default: False [-4.51 / 50.00]; BAYES_HAM(-3.00)[100.00%]; NEURAL_HAM_LONG(-1.00)[-1.000]; R_DKIM_ALLOW(-0.20)[suse.cz:s=susede2_rsa,suse.cz:s=susede2_ed25519]; NEURAL_HAM_SHORT(-0.20)[-1.000]; MIME_GOOD(-0.10)[text/plain]; MX_GOOD(-0.01)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FUZZY_RATELIMITED(0.00)[rspamd.com]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; RCPT_COUNT_TWELVE(0.00)[14]; MIME_TRACE(0.00)[0:+]; TO_DN_SOME(0.00)[]; RECEIVED_SPAMHAUS_BLOCKED_OPENRESOLVER(0.00)[2a07:de40:b281:106:10:150:64:167:received]; MID_RHS_MATCH_FROM(0.00)[]; DKIM_SIGNED(0.00)[suse.cz:s=susede2_rsa,suse.cz:s=susede2_ed25519]; FROM_EQ_ENVFROM(0.00)[]; FROM_HAS_DN(0.00)[]; R_RATELIMIT(0.00)[to_ip_from(RL5jz3zk9nm44ai14dcppf93zb)]; RCVD_COUNT_TWO(0.00)[2]; TO_MATCH_ENVRCPT_ALL(0.00)[]; DBL_BLOCKED_OPENRESOLVER(0.00)[suse.cz:mid,suse.cz:dkim,suse.cz:email,imap1.dmz-prg2.suse.org:helo,imap1.dmz-prg2.suse.org:rdns,oracle.com:email]; DKIM_TRACE(0.00)[suse.cz:+] X-Spam-Flag: NO X-Spam-Level: X-Rspamd-Queue-Id: 2495F2117F X-Rspamd-Server: rspamd2.dmz-prg2.suse.org X-Rspamd-Action: no action X-Spam-Score: -4.51 From: "Liam R. Howlett" Use the generic interface which should result in less bulk allocations during a forking. A part of this is to abstract the freeing of the sheaf or maple state allocations into its own function so mas_destroy() and the tree duplication code can use the same functionality to return any unused resources. Signed-off-by: Liam R. Howlett Reviewed-by: Suren Baghdasaryan Signed-off-by: Vlastimil Babka --- lib/maple_tree.c | 42 +++++++++++++++++++++++------------------- 1 file changed, 23 insertions(+), 19 deletions(-) diff --git a/lib/maple_tree.c b/lib/maple_tree.c index 61a322f945c28f5c3297c506923f00bcce5c7bca..5ef15e39fda8c7c65035fb7ed12= 5b82dfa52ca69 100644 --- a/lib/maple_tree.c +++ b/lib/maple_tree.c @@ -1172,6 +1172,19 @@ static inline void mas_alloc_nodes(struct ma_state *= mas, gfp_t gfp) mas_set_err(mas, -ENOMEM); } =20 +static inline void mas_empty_nodes(struct ma_state *mas) +{ + mas->node_request =3D 0; + if (mas->sheaf) { + mt_return_sheaf(mas->sheaf); + mas->sheaf =3D NULL; + } + + if (mas->alloc) { + kfree(mas->alloc); + mas->alloc =3D NULL; + } +} =20 /* * mas_free() - Free an encoded maple node @@ -5408,15 +5421,7 @@ void mas_destroy(struct ma_state *mas) mas->mas_flags &=3D ~MA_STATE_REBALANCE; } mas->mas_flags &=3D ~(MA_STATE_BULK|MA_STATE_PREALLOC); - - mas->node_request =3D 0; - if (mas->sheaf) - mt_return_sheaf(mas->sheaf); - mas->sheaf =3D NULL; - - if (mas->alloc) - kfree(mas->alloc); - mas->alloc =3D NULL; + mas_empty_nodes(mas); } EXPORT_SYMBOL_GPL(mas_destroy); =20 @@ -6504,7 +6509,7 @@ static inline void mas_dup_alloc(struct ma_state *mas= , struct ma_state *new_mas, struct maple_node *node =3D mte_to_node(mas->node); struct maple_node *new_node =3D mte_to_node(new_mas->node); enum maple_type type; - unsigned char request, count, i; + unsigned char count, i; void __rcu **slots; void __rcu **new_slots; unsigned long val; @@ -6512,20 +6517,17 @@ static inline void mas_dup_alloc(struct ma_state *m= as, struct ma_state *new_mas, /* Allocate memory for child nodes. */ type =3D mte_node_type(mas->node); new_slots =3D ma_slots(new_node, type); - request =3D mas_data_end(mas) + 1; - count =3D mt_alloc_bulk(gfp, request, (void **)new_slots); - if (unlikely(count < request)) { - memset(new_slots, 0, request * sizeof(void *)); - mas_set_err(mas, -ENOMEM); + count =3D mas->node_request =3D mas_data_end(mas) + 1; + mas_alloc_nodes(mas, gfp); + if (unlikely(mas_is_err(mas))) return; - } =20 - /* Restore node type information in slots. */ slots =3D ma_slots(node, type); for (i =3D 0; i < count; i++) { val =3D (unsigned long)mt_slot_locked(mas->tree, slots, i); val &=3D MAPLE_NODE_MASK; - ((unsigned long *)new_slots)[i] |=3D val; + new_slots[i] =3D ma_mnode_ptr((unsigned long)mas_pop_node(mas) | + val); } } =20 @@ -6579,7 +6581,7 @@ static inline void mas_dup_build(struct ma_state *mas= , struct ma_state *new_mas, /* Only allocate child nodes for non-leaf nodes. */ mas_dup_alloc(mas, new_mas, gfp); if (unlikely(mas_is_err(mas))) - return; + goto empty_mas; } else { /* * This is the last leaf node and duplication is @@ -6612,6 +6614,8 @@ static inline void mas_dup_build(struct ma_state *mas= , struct ma_state *new_mas, /* Make them the same height */ new_mas->tree->ma_flags =3D mas->tree->ma_flags; rcu_assign_pointer(new_mas->tree->ma_root, root); +empty_mas: + mas_empty_nodes(mas); } =20 /** --=20 2.51.0