From nobody Thu Dec 18 20:23:28 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6033EE8015 for ; Fri, 8 Sep 2023 14:53:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244152AbjIHOx1 (ORCPT ); Fri, 8 Sep 2023 10:53:27 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56634 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244134AbjIHOxZ (ORCPT ); Fri, 8 Sep 2023 10:53:25 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 66AB91BFA for ; Fri, 8 Sep 2023 07:53:20 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 065781FEFE; Fri, 8 Sep 2023 14:53:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1694184799; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9/ffVZ2jgkNFPV8v9GjIklo4R9arDXW9nEkTxNB0K6c=; b=KUyJUvRQS4gA2ke4GVINk61D6KwZrbVETPHCWg2keeQHFwJDBjRWoICJcRSmEkd1dxkjj5 TweCBVVTjorbVIzvDPI61iMA74jg4u+dwy5WGvFONU/ZC9FZihSGpc9w87jcACXu3wiaDg SwKeICwK+C9hN+SMhocIUAFapCddKZ0= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1694184799; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9/ffVZ2jgkNFPV8v9GjIklo4R9arDXW9nEkTxNB0K6c=; b=fZHG/B1YFdRe43J1fIyQGYzL7Ak+D5TYIdHhViuuB1fMP+TMnern1sOdtMTiv+4XRfZu7K CUoHOR832W1ZyoDQ== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id D27E713357; Fri, 8 Sep 2023 14:53:18 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id SDC+Ml41+2QaBQAAMHmgww (envelope-from ); Fri, 08 Sep 2023 14:53:18 +0000 From: Vlastimil Babka To: David Rientjes , Christoph Lameter , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Jay Patel Cc: Roman Gushchin , Pekka Enberg , Joonsoo Kim , linux-mm@kvack.org, patches@lists.linux.dev, linux-kernel@vger.kernel.org, Vlastimil Babka Subject: [PATCH 1/4] mm/slub: simplify the last resort slab order calculation Date: Fri, 8 Sep 2023 16:53:04 +0200 Message-ID: <20230908145302.30320-7-vbabka@suse.cz> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230908145302.30320-6-vbabka@suse.cz> References: <20230908145302.30320-6-vbabka@suse.cz> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" If calculate_order() can't fit even a single large object within slub_max_order, it will try using the smallest necessary order that may exceed slub_max_order but not MAX_ORDER. Currently this is done with a call to calc_slab_order() which is unecessary. We can simply use get_order(size). No functional change. Signed-off-by: Vlastimil Babka Reviewed-by: Jay Patel Tested-by: Jay Patel --- mm/slub.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/slub.c b/mm/slub.c index f7940048138c..c6e694cb17b9 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -4193,7 +4193,7 @@ static inline int calculate_order(unsigned int size) /* * Doh this slab cannot be placed using slub_max_order. */ - order =3D calc_slab_order(size, 1, MAX_ORDER, 1); + order =3D get_order(size); if (order <=3D MAX_ORDER) return order; return -ENOSYS; --=20 2.42.0 From nobody Thu Dec 18 20:23:28 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A3C8EE8013 for ; Fri, 8 Sep 2023 14:53:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244188AbjIHOxd (ORCPT ); Fri, 8 Sep 2023 10:53:33 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56672 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244143AbjIHOx0 (ORCPT ); Fri, 8 Sep 2023 10:53:26 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [IPv6:2001:67c:2178:6::1c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 35DFD1BF1 for ; Fri, 8 Sep 2023 07:53:21 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 2FCAB21D25; Fri, 8 Sep 2023 14:53:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1694184799; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1Z1sKJpFTH+GkYm0khClKXyg32JfDaf6DR+O1U32B1U=; b=YuP4yGAgh8IJXJqsJ73xpH1c8mWzwQKxk19hlkWByx+xqNYN38WeOb48/N6bxU0xCx8maW W1jKRpnjfatbKB8YELQlCeuLU2hRZzscE/fnXEfChJl//ljQGtUhCDbG9stCFPzEOpYgEa m9vCuphfho6ykKwxQy2FsDwCQfrC7nE= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1694184799; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1Z1sKJpFTH+GkYm0khClKXyg32JfDaf6DR+O1U32B1U=; b=zdKRoEqR5QeAuWFifH3Asdg2OEmp22MdGD1d+kXPIOxuQfreIS8fFSyp4hCLbK4CVYH8py dB2y1nrKP4MuEvDg== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 0857E131FD; Fri, 8 Sep 2023 14:53:19 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id AMVrAV81+2QaBQAAMHmgww (envelope-from ); Fri, 08 Sep 2023 14:53:19 +0000 From: Vlastimil Babka To: David Rientjes , Christoph Lameter , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Jay Patel Cc: Roman Gushchin , Pekka Enberg , Joonsoo Kim , linux-mm@kvack.org, patches@lists.linux.dev, linux-kernel@vger.kernel.org, Vlastimil Babka Subject: [PATCH 2/4] mm/slub: remove min_objects loop from calculate_order() Date: Fri, 8 Sep 2023 16:53:05 +0200 Message-ID: <20230908145302.30320-8-vbabka@suse.cz> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230908145302.30320-6-vbabka@suse.cz> References: <20230908145302.30320-6-vbabka@suse.cz> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" calculate_order() currently has two nested loops. The inner one that gradually modifies the acceptable waste from 1/16 up to 1/4, and the outer one that decreases min_objects down to 2. Upon closer inspection, the outer loop is unnecessary. Decreasing min_objects could have in theory two effects to make the inner loop and its call to calc_slab_order() succeed where a previous iteration with higher min_objects would not: - it could cause the min_objects-derived min_order to fit within slub_max_order. But min_objects is already pre-capped to max_objects that's derived from slub_max_order above the loops, so every iteration tries at least slub_max_order in calc_slab_order() - it could cause calc_slab_order() to be called with lower min_objects thus potentially lower min_order in its loop. This would make a difference if the lower order could cause the fractional waste test to succeed where a higher order has already failed with same fract_leftover in the previous iteration with a higher min_order. But that's not possible, because increasing the order can only result in lower (or same) fractional waste. If we increase the slab size 2 times, we will fit at least 2 times the number of objects (thus same fraction of waste), or it will allow us to fit one more object (lower fraction of waste). For more confidence I have tried adding a printk to notify when decreasing min_objects resulted in a success, and simulated calculations for a range of object sizes, nr_cpus and page_sizes. As expected, the printk never triggered. Thus remove the outer loop and adjust comments accordingly. There's almost no functional change except a weird corner case when slub_min_objects=3D1 on boot command line would cause the whole two nested loops to be skipped before this patch. Now it would try to find the best layout as usual, resulting in potentially higher orderthat minimizes waste. This is not wrong and will be further expanded by the next patch. Signed-off-by: Vlastimil Babka Reviewed-by: Jay Patel Tested-by: Jay Patel --- mm/slub.c | 38 ++++++++++++++++++-------------------- 1 file changed, 18 insertions(+), 20 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index c6e694cb17b9..5c287d96b212 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -4141,14 +4141,6 @@ static inline int calculate_order(unsigned int size) unsigned int max_objects; unsigned int nr_cpus; =20 - /* - * Attempt to find best configuration for a slab. This - * works by first attempting to generate a layout with - * the best configuration and backing off gradually. - * - * First we increase the acceptable waste in a slab. Then - * we reduce the minimum objects required in a slab. - */ min_objects =3D slub_min_objects; if (!min_objects) { /* @@ -4168,18 +4160,24 @@ static inline int calculate_order(unsigned int size) max_objects =3D order_objects(slub_max_order, size); min_objects =3D min(min_objects, max_objects); =20 - while (min_objects > 1) { - unsigned int fraction; - - fraction =3D 16; - while (fraction >=3D 4) { - order =3D calc_slab_order(size, min_objects, - slub_max_order, fraction); - if (order <=3D slub_max_order) - return order; - fraction /=3D 2; - } - min_objects--; + /* + * Attempt to find best configuration for a slab. This works by first + * attempting to generate a layout with the best possible configuration a= nd + * backing off gradually. + * + * We start with accepting at most 1/16 waste and try to find the + * smallest order from min_objects-derived/slub_min_order up to + * slub_max_order that will satisfy the constraint. Note that increasing + * the order can only result in same or less fractional waste, not more. + * + * If that fails, we increase the acceptable fraction of waste and try + * again. + */ + for (unsigned int fraction =3D 16; fraction >=3D 4; fraction /=3D 2) { + order =3D calc_slab_order(size, min_objects, slub_max_order, + fraction); + if (order <=3D slub_max_order) + return order; } =20 /* --=20 2.42.0 From nobody Thu Dec 18 20:23:28 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DFFD4EE8012 for ; Fri, 8 Sep 2023 14:53:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244177AbjIHOxb (ORCPT ); Fri, 8 Sep 2023 10:53:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56666 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244135AbjIHOxZ (ORCPT ); Fri, 8 Sep 2023 10:53:25 -0400 Received: from smtp-out1.suse.de (smtp-out1.suse.de [195.135.220.28]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C96011FC0 for ; Fri, 8 Sep 2023 07:53:20 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 57A6A21D26; Fri, 8 Sep 2023 14:53:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1694184799; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yGsbXXqcf4ORLaF4fFwqqZYAQFs76ztvW90bKyvUaBg=; b=yaoSVUcftdz9Z0pvCocl71s6a42GoguCtjRHXa0NVRtJyYOG6A+9KfluK9Nx3m9FsnxsoH nQhFs/wVHJNXuBKUYjlgy0YSvjKAfZz5BhBOE/d/6MtSEdQP0EBpF8i9JPxbt4GAX6uJVi y0ZD68CA0KFlChqiBb0LB0e5BbPQwtk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1694184799; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yGsbXXqcf4ORLaF4fFwqqZYAQFs76ztvW90bKyvUaBg=; b=1F8uln1OGPghmt/eKsFza2ILUJbA9ByJ69bS4pA/H55d4XBFwWCspcGJ6dS96kWwjv3Hpo 6+lv3dA372hQ5VCA== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 3169A13357; Fri, 8 Sep 2023 14:53:19 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id wGFkC181+2QaBQAAMHmgww (envelope-from ); Fri, 08 Sep 2023 14:53:19 +0000 From: Vlastimil Babka To: David Rientjes , Christoph Lameter , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Jay Patel Cc: Roman Gushchin , Pekka Enberg , Joonsoo Kim , linux-mm@kvack.org, patches@lists.linux.dev, linux-kernel@vger.kernel.org, Vlastimil Babka Subject: [PATCH 3/4] mm/slub: attempt to find layouts up to 1/2 waste in calculate_order() Date: Fri, 8 Sep 2023 16:53:06 +0200 Message-ID: <20230908145302.30320-9-vbabka@suse.cz> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230908145302.30320-6-vbabka@suse.cz> References: <20230908145302.30320-6-vbabka@suse.cz> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The main loop in calculate_order() currently tries to find an order with at most 1/4 waste. If that's impossible (for particular large object sizes), there's a fallback that will try to place one object within slab_max_order. If we expand the loop boundary to also allow up to 1/2 waste as the last resort, we can remove the fallback and simplify the code, as the loop will find an order for such sizes as well. Note we don't need to allow more than 1/2 waste as that will never happen - calc_slab_order() would calculate more objects to fit, reducing waste below 1/2. Sucessfully finding an order in the loop (compared to the fallback) will also have the benefit in trying to satisfy min_objects, because the fallback was passing 1. Thus the resulting slab orders might be larger (not because it would improve waste, but to reduce pressure on shared locks), which is one of the goals of calculate_order(). For example, with nr_cpus=3D1 and 4kB PAGE_SIZE, slub_max_order=3D3, before the patch we would get the following orders for these object sizes: 2056 to 10920 - order-3 as selected by the loop 10928 to 12280 - order-2 due to fallback, as <1/4 waste is not possible 12288 to 32768 - order-3 as <1/4 waste is again possible After the patch: 2056 to 32768 - order-3, because even in the range of 10928 to 12280 we try to satisfy the calculated min_objects. As a result the code is simpler and gives more consistent results. Signed-off-by: Vlastimil Babka Reviewed-by: Feng Tang Reviewed-by: Jay Patel Tested-by: Jay Patel --- mm/slub.c | 14 ++++---------- 1 file changed, 4 insertions(+), 10 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index 5c287d96b212..f04eb029d85a 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -4171,23 +4171,17 @@ static inline int calculate_order(unsigned int size) * the order can only result in same or less fractional waste, not more. * * If that fails, we increase the acceptable fraction of waste and try - * again. + * again. The last iteration with fraction of 1/2 would effectively + * accept any waste and give us the order determined by min_objects, as + * long as at least single object fits within slub_max_order. */ - for (unsigned int fraction =3D 16; fraction >=3D 4; fraction /=3D 2) { + for (unsigned int fraction =3D 16; fraction > 1; fraction /=3D 2) { order =3D calc_slab_order(size, min_objects, slub_max_order, fraction); if (order <=3D slub_max_order) return order; } =20 - /* - * We were unable to place multiple objects in a slab. Now - * lets see if we can place a single object there. - */ - order =3D calc_slab_order(size, 1, slub_max_order, 1); - if (order <=3D slub_max_order) - return order; - /* * Doh this slab cannot be placed using slub_max_order. */ --=20 2.42.0 From nobody Thu Dec 18 20:23:28 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3C344EE8011 for ; Fri, 8 Sep 2023 14:53:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244134AbjIHOxa (ORCPT ); Fri, 8 Sep 2023 10:53:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56672 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244103AbjIHOxZ (ORCPT ); Fri, 8 Sep 2023 10:53:25 -0400 Received: from smtp-out2.suse.de (smtp-out2.suse.de [195.135.220.29]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CA35F1FC1 for ; Fri, 8 Sep 2023 07:53:20 -0700 (PDT) Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out2.suse.de (Postfix) with ESMTPS id 825EA1FEFF; Fri, 8 Sep 2023 14:53:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_rsa; t=1694184799; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TBS/rfQAa9CuGAOXLs/AG66+kZ+35/pxxXJ2H3zd1io=; b=AErap95nCfprazdQmp9OXffnXycQVwIbGZpHDVA0rP1sASRm6I/cAKuQEAWyQwn3yPrIhj n2us9Yv71zG8n60xarSDoURo97o0UJr6tPpUw1eQl5dNdrsDvGofv7aMgtveGRu7mN2InU ErsEOQVo1bhO94LeTmRRs1dVrdIzL/Y= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.cz; s=susede2_ed25519; t=1694184799; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=TBS/rfQAa9CuGAOXLs/AG66+kZ+35/pxxXJ2H3zd1io=; b=PBQchTCPYcBZU1rMZMkGvd3jPhMJtIy0CS2wp+p1ullQX6NqzbDhx7UDZYJDCdQdtJKh3v ubVT/MasEOcFelDA== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 59E68131FD; Fri, 8 Sep 2023 14:53:19 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id iIFAFV81+2QaBQAAMHmgww (envelope-from ); Fri, 08 Sep 2023 14:53:19 +0000 From: Vlastimil Babka To: David Rientjes , Christoph Lameter , Hyeonggon Yoo <42.hyeyoo@gmail.com>, Jay Patel Cc: Roman Gushchin , Pekka Enberg , Joonsoo Kim , linux-mm@kvack.org, patches@lists.linux.dev, linux-kernel@vger.kernel.org, Vlastimil Babka Subject: [PATCH 4/4] mm/slub: refactor calculate_order() and calc_slab_order() Date: Fri, 8 Sep 2023 16:53:07 +0200 Message-ID: <20230908145302.30320-10-vbabka@suse.cz> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230908145302.30320-6-vbabka@suse.cz> References: <20230908145302.30320-6-vbabka@suse.cz> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" After the previous cleanups, we can now move some code from calc_slab_order() to calculate_order() so it's executed just once, and do some more cleanups. - move the min_order and MAX_OBJS_PER_PAGE evaluation to calc_slab_order(). - change calc_slab_order() parameter min_objects to min_order Also make MAX_OBJS_PER_PAGE check more robust by considering also min_objects in addition to slub_min_order. Otherwise this is not a functional change. Signed-off-by: Vlastimil Babka Reviewed-by: Feng Tang Reviewed-by: Jay Patel Tested-by: Jay Patel --- mm/slub.c | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/mm/slub.c b/mm/slub.c index f04eb029d85a..1c91f72c7239 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -4110,17 +4110,12 @@ static unsigned int slub_min_objects; * the smallest order which will fit the object. */ static inline unsigned int calc_slab_order(unsigned int size, - unsigned int min_objects, unsigned int max_order, + unsigned int min_order, unsigned int max_order, unsigned int fract_leftover) { - unsigned int min_order =3D slub_min_order; unsigned int order; =20 - if (order_objects(min_order, size) > MAX_OBJS_PER_PAGE) - return get_order(size * MAX_OBJS_PER_PAGE) - 1; - - for (order =3D max(min_order, (unsigned int)get_order(min_objects * size)= ); - order <=3D max_order; order++) { + for (order =3D min_order; order <=3D max_order; order++) { =20 unsigned int slab_size =3D (unsigned int)PAGE_SIZE << order; unsigned int rem; @@ -4139,7 +4134,7 @@ static inline int calculate_order(unsigned int size) unsigned int order; unsigned int min_objects; unsigned int max_objects; - unsigned int nr_cpus; + unsigned int min_order; =20 min_objects =3D slub_min_objects; if (!min_objects) { @@ -4152,7 +4147,7 @@ static inline int calculate_order(unsigned int size) * order on systems that appear larger than they are, and too * low order on systems that appear smaller than they are. */ - nr_cpus =3D num_present_cpus(); + unsigned int nr_cpus =3D num_present_cpus(); if (nr_cpus <=3D 1) nr_cpus =3D nr_cpu_ids; min_objects =3D 4 * (fls(nr_cpus) + 1); @@ -4160,6 +4155,10 @@ static inline int calculate_order(unsigned int size) max_objects =3D order_objects(slub_max_order, size); min_objects =3D min(min_objects, max_objects); =20 + min_order =3D max(slub_min_order, (unsigned int)get_order(min_objects * s= ize)); + if (order_objects(min_order, size) > MAX_OBJS_PER_PAGE) + return get_order(size * MAX_OBJS_PER_PAGE) - 1; + /* * Attempt to find best configuration for a slab. This works by first * attempting to generate a layout with the best possible configuration a= nd @@ -4176,7 +4175,7 @@ static inline int calculate_order(unsigned int size) * long as at least single object fits within slub_max_order. */ for (unsigned int fraction =3D 16; fraction > 1; fraction /=3D 2) { - order =3D calc_slab_order(size, min_objects, slub_max_order, + order =3D calc_slab_order(size, min_order, slub_max_order, fraction); if (order <=3D slub_max_order) return order; --=20 2.42.0