From nobody Thu Sep 24 14:27:09 2026 Received: from mail-yx2-f43.google.com (mail-yx2-f43.google.com [74.125.224.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 761763CAE7F for ; Wed, 23 Sep 2026 21:10:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.224.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790197851; cv=none; b=i4Cdjt/duEnIdWekVsdcvhBF38Mxdk1iVY/e7tcxjeA8Da7pN0k9piDNs9OkS69Izl51QCOH3p74APWP6zB5H+5q2Tk685J5LsmqYVJhTmtJsOHMuhumWGqeyiyG+5kFVPZ1VSOP2UQbJR0v872OeclnQEe07Al4MjN5SpD0duY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790197851; c=relaxed/simple; bh=Tvs161wK/s6K/mulS/cEIE9S02EbKDqru6nU3kHzh6k=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gBZrPchwZPUMtw4FlTu8PBzr0UklrqhGMndinaLjnah8AP6aLakXc6pd+EjsV4rt1L0ckKsHZBwEMPJJjvpTaLvXR2ZZdvPH6fuc2ifzwvxr4Vtt2A5O6a/I9UzPPIqxKxYVSdzf/f/JWc+ySDi1b4nItHyKmtIrVJzE52lVB8g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=Dc8mP499; arc=none smtp.client-ip=74.125.224.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="Dc8mP499" Received: by mail-yx2-f43.google.com with SMTP id 00721157ae682-85d43db0c29so20634227b3.2 for ; Wed, 23 Sep 2026 14:10:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1790197845; x=1790802645; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=zePS8k8rZkvT//tsMfFKaNJkinJJhgZrZUtUQPvClt0=; b=Dc8mP499sNbmnrwOL4gT4CpT2VbCFbNCphPOI87/uJX5psAthH/0OoenOq4HkR3pnl /4vuPmJK1h2EUZWLzzkCYj7dtsIMwDQgAbkbO6MwZHl+J7csg8Yms2UkY/MurchPwKeW lX3KHT0jXNoWe1gCwXWhxSe82pFGtdLfTU50mQSXK4tKAbXavJtptfA2o6B+V60/Lp3z 7qaDDrw19T4yDZqdQS9D9AVgGxG0PIXnbtMgAokmNTOATdUCoO3iuAq9L/7Wk6DJ96oC sYiSwzfiCbM5iwBLx1x+EOZhdkabnK46sBPexTtJ+PAaJePHG44m4v3iD/rx6vfmSAMk lLtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790197845; x=1790802645; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=zePS8k8rZkvT//tsMfFKaNJkinJJhgZrZUtUQPvClt0=; b=AjpUH4ClySE3DJPysad5Fyyn4KSYqia0W7rUJyXBbn5/tle/US78JB3MAS3165qG4x PDAEn2b4Rii6tGAFR37TlBa8KouwN/IcK+h5OdEp1bGN5JrBk8z3hDAJPw46SKBDjloY /afEZxzyyIrpRp2cBgFpjutp5gXvA0vzuyVqvS9qGeD8TTT3D7JmtHhM4xn1beO35X3/ HlRMeZhdjXd7wPOFD/wGdrA4Wy/PVrc/X0KF/NuaSMUHJKuLsIGj1zuQd3jLFVj0nPMj PFUyzsArtXfq8PbA+Z2juEjnLweM/b8zCNq2vKaV/LVkjK/sZ71B2q4AXmzz5+GdzKGB dBzg== X-Gm-Message-State: AFuF++mONppQ7k3n3stlMOzTuZo+XlXa0AT07U2V5osolWne7Koh5aoH 3FqEdT+Dm0JSeFGFK4UgTwjkO8vWTjaSn8bS8udKal22ph2nhUcbQOdc9nJr6JELXYI= X-Gm-Gg: AYBFou3DzUD+O5tAd9vO1WGMDGbD7fJkJYVmqNTKNwcgeKKSdHMZOnXwUm7urp6xfqf p9X0OKhFkIBBNkkjfdujx1NXGCP+my/9iRYF/pwcan8L2ZYmuIHP4HLQuhVGmqi0rOg1XcT9EBU PLwQKXtAh0BVqZfAmIZB0XVMxocFWwIElRG8SagpGKa+rCZkZ7fSIXklnU6SSPjY2NIrgj6ENvz o821US5woXnk0xKT9Vbn8Y9lRuuKFdBzZ9bk5djfv29QnCfweOLeBnf6JltMqxJ0mhvDmbQBlxn IFPXA+0KLaoApo2IwOc6OI+pzRjRkKJzgaP7OtmJEB5/VsP0TmXyV7fljH8W163/ZD2Ho2YwQns fBM16/NtSH3/bBBOv0KmTSs6ZpX8XwZKnKZ9LO5XVKIHpMXDtssknEfV+ZGDKRo2a8QH5f7BDNI f0etfvM7i41c+PNpXaywNWWkGJ/R4W8m/9rK1WYNjuyXh9gizT5RcKq3DB7ZxTeYvQV9nnEPx3L m7JmL+uY7B35IATCJWewBfIC9Vlg26QnukPFYV4ed70V5DhNEPQg+FgEtRZ X-Received: by 2002:a05:690c:6610:b0:873:17c9:b054 with SMTP id 00721157ae682-8a648505a0dmr2839327b3.21.1790197845286; Wed, 23 Sep 2026 14:10:45 -0700 (PDT) Received: from gourry-fedora-PF4VCD3F.lan (pool-173-79-60-52.washdc.fios.verizon.net. [173.79.60.52]) by smtp.gmail.com with ESMTPSA id af79cd13be357-93c2485926esm305893085a.19.2026.09.23.14.10.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Sep 2026 14:10:44 -0700 (PDT) From: Gregory Price To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, kernel-team@meta.com, willy@infradead.org, jack@suse.cz, akpm@linux-foundation.org, david@kernel.org, gourry@gourry.net, ziy@nvidia.com, joshua.hahnjy@gmail.com, rakie.kim@sk.com, ying.huang@linux.alibaba.com, vbabka@kernel.org, surenb@google.com, mhocko@suse.com, brendan.jackman@linux.dev, hannes@cmpxchg.org Subject: [RFC PATCH 1/6] mm/page_alloc: clarify bulk allocator flag scope Date: Wed, 23 Sep 2026 17:10:35 -0400 Message-ID: <20260923211041.3127588-2-gourry@gourry.net> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260923211041.3127588-1-gourry@gourry.net> References: <20260923211041.3127588-1-gourry@gourry.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" alloc_pages_bulk_noprof() uses its local alloc_flags for fast-path watermark, cpuset, and CMA decisions. These are distinct from the allocator behavior flags stored in alloc_context. Rename the local variable to fastpath_alloc_flags. No change in functionality. Signed-off-by: Gregory Price --- mm/page_alloc.c | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 07e03c7c7771..728dd1a36f0f 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5169,7 +5169,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, struct per_cpu_pages *pcp; struct list_head *pcp_list; struct alloc_context ac; - unsigned int alloc_flags =3D ALLOC_WMARK_LOW; + unsigned int fastpath_alloc_flags =3D ALLOC_WMARK_LOW; int nr_populated =3D 0, nr_account =3D 0; =20 /* @@ -5210,7 +5210,8 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, /* May set ALLOC_NOFRAGMENT, fragmentation will return 1 page. */ gfp &=3D gfp_allowed_mask; gfp =3D current_gfp_context(gfp); - if (!prepare_alloc_pages(gfp, 0, preferred_nid, nodemask, &ac, &gfp, &all= oc_flags)) + if (!prepare_alloc_pages(gfp, 0, preferred_nid, nodemask, &ac, &gfp, + &fastpath_alloc_flags)) goto out; =20 /* Find an allowed local zone that meets the low watermark. */ @@ -5218,7 +5219,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, for_next_zone_zonelist_nodemask(zone, z, ac.highest_zoneidx, ac.nodemask)= { unsigned long mark; =20 - if (cpusets_enabled() && (alloc_flags & ALLOC_CPUSET) && + if (cpusets_enabled() && (fastpath_alloc_flags & ALLOC_CPUSET) && !__cpuset_zone_allowed(zone, gfp)) { continue; } @@ -5228,16 +5229,17 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, in= t preferred_nid, goto failed; } =20 - cond_accept_memory(zone, 0, alloc_flags); + cond_accept_memory(zone, 0, fastpath_alloc_flags); retry_this_zone: - mark =3D wmark_pages(zone, alloc_flags & ALLOC_WMARK_MASK) + nr_pages - = nr_populated; + mark =3D wmark_pages(zone, fastpath_alloc_flags & ALLOC_WMARK_MASK) + + nr_pages - nr_populated; if (zone_watermark_fast(zone, 0, mark, zonelist_zone_idx(ac.preferred_zoneref), - alloc_flags, gfp)) { + fastpath_alloc_flags, gfp)) { break; } =20 - if (cond_accept_memory(zone, 0, alloc_flags)) + if (cond_accept_memory(zone, 0, fastpath_alloc_flags)) goto retry_this_zone; =20 /* Try again if zone has deferred pages */ @@ -5269,8 +5271,9 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, continue; } =20 - page =3D __rmqueue_pcplist(zone, 0, ac.migratetype, alloc_flags, - pcp, pcp_list); + page =3D __rmqueue_pcplist(zone, 0, ac.migratetype, + fastpath_alloc_flags, pcp, + pcp_list); if (unlikely(!page)) { /* Try and allocate at least one page */ if (!nr_account) { --=20 2.55.0 From nobody Thu Sep 24 14:27:09 2026 Received: from mail-qk2-f43.google.com (mail-qk2-f43.google.com [74.125.230.235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BC0323B71D3 for ; Wed, 23 Sep 2026 21:10:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.230.235 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790197852; cv=none; b=OG5QGwimrFkn3gW4LwdiY6uk0KchTW0syMBpin4+vmNylpgdG6XfJyDDXcxtznl4pljPwlb5Y21nVu31YgVowHJgWpUErJ2PoSZxqEVn1q9A+XgXqdC9OVakbJ9f3lFS6z+d9ZtO0oNMRdjuiROvMbCpN3ss8BWz6D0Zkp+mops= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790197852; c=relaxed/simple; bh=XRO48Bgw48FqtNFnwxoO2t8s1KdDx0HkbyEfUCKJjsU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=R3CA7nAtaZZ/Jz41PLOR0JjmT4axsurcHuNiVn2+qPWtIfeFxVssNTmZIQNGDVWaM6FraA9fA/0sxZeyYSAyuOxCCMVxn0mLIFZaS81vQVtj8kQx5uJVZJpAmcPqemY0ndvNS5TlxS9Hx8BnvlJtt+tZzoJsL/JsmWg1uT0q0M0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=iMQewwTX; arc=none smtp.client-ip=74.125.230.235 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="iMQewwTX" Received: by mail-qk2-f43.google.com with SMTP id af79cd13be357-93910cc46c4so134833385a.2 for ; Wed, 23 Sep 2026 14:10:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1790197847; x=1790802647; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=vPVSrxtSx9kDTub89+jb9IguuxzhfykwRy7ihRNBm8Q=; b=iMQewwTXR44v5uEfqfL3yUAYonKfI5suYEY3eSOPcf+9HZK0FtexNM1/DfrPs+mx7E gtXnvWbJyAfS07L5x4Q+idvHPGXXwnurk1dt+CaxeoUMERbA9YD3VrSTD6AcVagzMolX bn1g5OGd6QTOZJqk99MjF8hihbHD45gPffkjmLBhl1zTzu26ZjBoAjb4J4Ex3mSJKdnP 7UwMqWWGV4a/W9Cwd4Fd69HgdQjiuFLnu8tSrTKQC9rwIKEDDspvCLxx1HGZOLC+7bhP 0VU9+Dw8F5AzlBsYVFhTl/+vCyxOtniYUbyvpim/5qYYIU8KAk7IET4VwptUHSeaxbln T+fw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790197847; x=1790802647; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=vPVSrxtSx9kDTub89+jb9IguuxzhfykwRy7ihRNBm8Q=; b=UNtcVjIPlFUdg6xl9CIbb4WGt+ZyX1qEtbm6vx1Tqn1ULhAA8Fq6SoUYuYUv6OKqja mAKBwrHGxiJxCDJ+6WaaJqJY+zRi7W0eumYHT3TUBDkmQ2wIJmL5Hv9wJ9jiTR2BAxlu k9nP9+Z6JOhiKl0Yvf5YBsGx+zgNMJ0iTLrE//9Y8i10buiBxV9kOfc894lbhGTzP8oQ g1jIprGpmS1yBspEE0td61OoN0k+nlkAFRcXssIsD01bVWkFY+wk+2Ze744jOXg0C2Bn Z3ktJrucglwSVpUIq69512i7qbTmj/enho0cPUmN/wt6svrKb1uxTMS1gUbUmKnuIdwr h2Sw== X-Gm-Message-State: AFuF++nCEH0wyX0ZoMrJ3xO+/H+MYcGoNrKpjEnWPPcKdCkzo531Znv2 0iAs3HbOHAtYlUnPRIEEluwjHPScAJP9TPk+T7iBMjPmNQpHrOY7fZHKcVmjh0zm8lM= X-Gm-Gg: AYBFou3U4OPZubiTg/9h+p33KzPDNgXFIi27F+rDo9qoKv3/mkTiI+wYwd2mrpaRbRG qCrIyC7JZHpyBiTZMrZXM4xrf51hgAXE4WUbGukmxZMtNZaJSmN11+D+GYL2B+9xqpoAUgWikWr G7m+spQvdvmuVmf//hk2gVKWLsomj4CRC2aKfldLMBz+Ayja9dKdQqcyFwfk+cC+v0akU4H0yHH 9k0iU03EFlTpfA4SV9oXpd9x0xEVS6Vm8JHdSdxDNbCD5gIYnZlHmkeQ+0GRT2sdmfnj79boEdA 9CCycrOysSe1lg4p4ijy2xaG6pcI86MEh0Zv5AYuWkTnLyd4KZrQBiHuL5KE6RWGYFdDnVuzrOx Czjlzle2zbKaKwHUAUgnkEtRkGzl7onLr895kNuJAV6niWc6D8NFREXjKaqijYwBBP1WneQ8nHW 9J2keepxdG5QtsN2My1A4eubOEgf8fsOcJXgTgfNlXk1CNdSI9ZRlIATJK9cgEI3HGe/q78UtqP q9EJP19OWvZZoa2rEDWxoItNuPcLkH1dPH3dNGUGlsW5FBk8tt86N1fgraX X-Received: by 2002:a05:620a:2586:b0:939:9bcf:8d35 with SMTP id af79cd13be357-93c348aee16mr64595285a.35.1790197847255; Wed, 23 Sep 2026 14:10:47 -0700 (PDT) Received: from gourry-fedora-PF4VCD3F.lan (pool-173-79-60-52.washdc.fios.verizon.net. [173.79.60.52]) by smtp.gmail.com with ESMTPSA id af79cd13be357-93c2485926esm305893085a.19.2026.09.23.14.10.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Sep 2026 14:10:46 -0700 (PDT) From: Gregory Price To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, kernel-team@meta.com, willy@infradead.org, jack@suse.cz, akpm@linux-foundation.org, david@kernel.org, gourry@gourry.net, ziy@nvidia.com, joshua.hahnjy@gmail.com, rakie.kim@sk.com, ying.huang@linux.alibaba.com, vbabka@kernel.org, surenb@google.com, mhocko@suse.com, brendan.jackman@linux.dev, hannes@cmpxchg.org Subject: [RFC PATCH 2/6] mm/page_alloc: refactor alloc_flags preparation Date: Wed, 23 Sep 2026 17:10:36 -0400 Message-ID: <20260923211041.3127588-3-gourry@gourry.net> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260923211041.3127588-1-gourry@gourry.net> References: <20260923211041.3127588-1-gourry@gourry.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" __alloc_frozen_pages_noprof() validates allocation flags, checks the order, prepares ALLOC_NOLOCK, and selects its initial watermark. alloc_pages_bulk_noprof() separately initializes its allocation context and low-watermark flags. Move this setup into prepare_alloc_flags() and call it from both paths. The bulk allocator continues to pass ALLOC_DEFAULT. No change in functionality. Signed-off-by: Gregory Price --- mm/page_alloc.c | 158 ++++++++++++++++++++++++++---------------------- 1 file changed, 87 insertions(+), 71 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 728dd1a36f0f..b4a021fad183 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5138,6 +5138,84 @@ static inline bool prepare_alloc_pages(gfp_t gfp_mas= k, unsigned int order, return true; } =20 +static inline bool alloc_order_allowed(gfp_t gfp, unsigned int order, + unsigned int alloc_flags) +{ + if (alloc_flags & ALLOC_NOLOCK) + return pcp_allowed_order(order); + + /* + * There are several places where we assume that the order value is sane + * so bail out early if the request is out of bound. + */ + return !(WARN_ON_ONCE_GFP(order > MAX_PAGE_ORDER, gfp)); +} + +static inline bool alloc_nolock_allowed(void) +{ + if (!can_spin_trylock()) + return false; + + /* Bailout, since _deferred_grow_zone() needs to take a lock */ + if (deferred_pages_enabled()) + return false; + + return true; +} + +/* + * GFP flags to set for ALLOC_NOLOCK i.e. alloc_pages_nolock(). + * + * Do not specify __GFP_DIRECT_RECLAIM, since direct claim is not allowed. + * Do not specify __GFP_KSWAPD_RECLAIM either, since wake up of kswapd + * is not safe in arbitrary context. + * + * These two are the conditions for gfpflags_allow_spinning() being true. + * + * Specify __GFP_NOWARN since failing alloc_pages_nolock() is not a reason + * to warn. Also warn would trigger printk() which is unsafe from + * various contexts. We cannot use printk_deferred_enter() to mitigate, + * since the running context is unknown. + * + * Specify __GFP_ZERO to make sure that call to kmsan_alloc_page() below + * is safe in any context. Also zeroing the page is mandatory for + * BPF use cases. + * + * Though __GFP_NOMEMALLOC is not checked in the code path below, + * specify it here to highlight that alloc_pages_nolock() + * doesn't want to deplete reserves. + */ +static const gfp_t gfp_nolock =3D __GFP_NOWARN | __GFP_ZERO | __GFP_NOMEMA= LLOC | + __GFP_COMP; + +static __always_inline bool prepare_alloc_flags(gfp_t *gfp, unsigned int o= rder, + unsigned int alloc_flags, unsigned int *prepared_alloc_flags, + unsigned int *fastpath_alloc_flags) +{ + /* Other flags could be supported later if needed. */ + if (WARN_ON(alloc_flags & ~(ALLOC_NOLOCK | ALLOC_NO_CODETAG))) + return false; + + if (!alloc_order_allowed(*gfp, order, alloc_flags)) + return false; + + *prepared_alloc_flags =3D alloc_flags; + *fastpath_alloc_flags =3D alloc_flags; + + if (alloc_flags & ALLOC_NOLOCK) { + /* Certain other flags could be supported later if needed. */ + VM_WARN_ON_ONCE(*gfp & ~(__GFP_ACCOUNT | gfp_nolock)); + if (!alloc_nolock_allowed()) + return false; + *gfp |=3D gfp_nolock; + *fastpath_alloc_flags |=3D ALLOC_WMARK_MIN; + } else { + *fastpath_alloc_flags |=3D ALLOC_WMARK_LOW; + } + + return true; +} + /* * __alloc_pages_bulk - Allocate a number of order-0 pages to an array * @gfp: GFP flags for the allocation @@ -5169,7 +5247,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, struct per_cpu_pages *pcp; struct list_head *pcp_list; struct alloc_context ac; - unsigned int fastpath_alloc_flags =3D ALLOC_WMARK_LOW; + unsigned int fastpath_alloc_flags; int nr_populated =3D 0, nr_account =3D 0; =20 /* @@ -5207,6 +5285,10 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int= preferred_nid, goto failed; #endif =20 + if (!prepare_alloc_flags(&gfp, 0, ALLOC_DEFAULT, &ac.alloc_flags, + &fastpath_alloc_flags)) + goto out; + /* May set ALLOC_NOFRAGMENT, fragmentation will return 1 page. */ gfp &=3D gfp_allowed_mask; gfp =3D current_gfp_context(gfp); @@ -5335,56 +5417,6 @@ void free_pages_bulk(struct page **page_array, unsig= ned long nr_pages) } } =20 -static inline bool alloc_order_allowed(gfp_t gfp, unsigned int order, - unsigned int alloc_flags) -{ - if (alloc_flags & ALLOC_NOLOCK) - return pcp_allowed_order(order); - - /* - * There are several places where we assume that the order value is sane - * so bail out early if the request is out of bound. - */ - return !(WARN_ON_ONCE_GFP(order > MAX_PAGE_ORDER, gfp)); -} - -static inline bool alloc_nolock_allowed(void) -{ - if (!can_spin_trylock()) - return false; - - /* Bailout, since _deferred_grow_zone() needs to take a lock */ - if (deferred_pages_enabled()) - return false; - - return true; -} - -/* - * GFP flags to set for ALLOC_NOLOCK i.e. alloc_pages_nolock(). - * - * Do not specify __GFP_DIRECT_RECLAIM, since direct claim is not allowed. - * Do not specify __GFP_KSWAPD_RECLAIM either, since wake up of kswapd - * is not safe in arbitrary context. - * - * These two are the conditions for gfpflags_allow_spinning() being true. - * - * Specify __GFP_NOWARN since failing alloc_pages_nolock() is not a reason - * to warn. Also warn would trigger printk() which is unsafe from - * various contexts. We cannot use printk_deferred_enter() to mitigate, - * since the running context is unknown. - * - * Specify __GFP_ZERO to make sure that call to kmsan_alloc_page() below - * is safe in any context. Also zeroing the page is mandatory for - * BPF use cases. - * - * Though __GFP_NOMEMALLOC is not checked in the code path below, - * specify it here to highlight that alloc_pages_nolock() - * doesn't want to deplete reserves. - */ -static const gfp_t gfp_nolock =3D __GFP_NOWARN | __GFP_ZERO | __GFP_NOMEMA= LLOC | - __GFP_COMP; - /* * This is the 'heart' of the zoned buddy allocator. */ @@ -5393,29 +5425,13 @@ struct page *__alloc_frozen_pages_noprof(gfp_t gfp,= unsigned int order, { struct page *page; gfp_t alloc_gfp; /* The gfp_t that was actually used for allocation */ - struct alloc_context ac =3D { - .alloc_flags =3D alloc_flags, - }; - unsigned int fastpath_alloc_flags =3D alloc_flags; - - /* Other flags could be supported later if needed. */ - if (WARN_ON(alloc_flags & ~(ALLOC_NOLOCK | ALLOC_NO_CODETAG))) - return NULL; + struct alloc_context ac; + unsigned int fastpath_alloc_flags; =20 - if (!alloc_order_allowed(gfp, order, alloc_flags)) + if (!prepare_alloc_flags(&gfp, order, alloc_flags, &ac.alloc_flags, + &fastpath_alloc_flags)) return NULL; =20 - if (alloc_flags & ALLOC_NOLOCK) { - /* Certain other flags could be supported later if needed. */ - VM_WARN_ON_ONCE(gfp & ~(__GFP_ACCOUNT | gfp_nolock)); - if (!alloc_nolock_allowed()) - return NULL; - gfp |=3D gfp_nolock; - fastpath_alloc_flags |=3D ALLOC_WMARK_MIN; - } else { - fastpath_alloc_flags |=3D ALLOC_WMARK_LOW; - } - gfp &=3D gfp_allowed_mask; /* * Apply scoped allocation constraints. This is mainly about GFP_NOFS --=20 2.55.0 From nobody Thu Sep 24 14:27:09 2026 Received: from mail-qk2-f43.google.com (mail-qk2-f43.google.com [74.125.230.235]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 02CAC3C2BB9 for ; Wed, 23 Sep 2026 21:10:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.230.235 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790197853; cv=none; b=uFUopGrmNh1WlQPUGR9gskUJLSGYRA46aN/pMMb0WmXhKyYwI9oHvr1sck9UpZ496c7DEH3tp7fbKopbpHnCIJv9zJM/avRnsuqfHvVIj4CNlTIGTIGJqD0bUJ3MdjZ9kadOS78dS3enDmER4g3GLSkY1+vuwDzcURBRRvgUCpA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790197853; c=relaxed/simple; bh=6+p9c16JaWvd1hm6n4rhzqJqBx/gTTw/01UKuroqhmA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=o4iG4/EZNs8B7hhYYB0nYm7zZuDQcYvstZtnQzChdKzGt87hgCRPAMDTHfO8mgQHCjR6AOnult6kfC21nrHB07Zk1jXqFbGsWjNaM7WNeQgnam8LxjjymrJd1gnCP0jDeFudI8FyJQxWcsJkr5QXWn3e4moObd+lR/6wpXb+r0g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=uz8sfil1; arc=none smtp.client-ip=74.125.230.235 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="uz8sfil1" Received: by mail-qk2-f43.google.com with SMTP id af79cd13be357-93910cc46c4so134834785a.2 for ; Wed, 23 Sep 2026 14:10:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1790197849; x=1790802649; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=xVr7KuSWyr8CUJERgbEEA0iI2fqWk3sHPjbcAbZFfj4=; b=uz8sfil1N48qs4SpM6831TG710nH4VNPJXmYySjrhJ7Cy6EkA2UyinpfsnKx78dT3D vJE+NdFypAXQw/W9ej1lKkKJo34nrU4y/Jk9sKIL6gAEpothK17WdBx9qk9fcHGwXAin 0eKaooFB6VYEq5Ar8IZ+pism6w7GZkEQwtR5Uzk/XjM3FycNSzqYlcNQdu0dFEtTVa9K 8z7v4NxGjnAHVx3Ul5l229ndO6zO4cCe41VmzXVRhfVDA4rpnx13W5A5txqwvjQsPvV2 CbMG5KqL8NOptIPnKp6pYuiHvmLZC5h7kCfr3F0dXgf2x7kZG3Ho/NjzCZm5NM8fDTIX KDZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790197849; x=1790802649; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=xVr7KuSWyr8CUJERgbEEA0iI2fqWk3sHPjbcAbZFfj4=; b=StKi5P0vaDUOHMGNHJFQJCku/0cTLgL9avzl2hcWTdiGqKMprRyFSluGvSDuGQ4aN3 jedePc75DV4GOj9YO5KLmd/w4Qs3EaVr69Dj325E+ZK+vtwTSs4OalhfMuUD3rz6TDgj Vd8R/ACbIwukAKYZ4PhMxW4pkcQfk+di6xZVl2Q95RXXCXx2GHbp96SZXvrI+0zjtXgk jD1IyZ++P5iszinkpk8FQSmVD/aJQ74FntX13iF2Fuspam1JlzvOtPRvCVAOJsqt4YSA HxssikMzn6JO9SubFvKAt6KP9rI8Uqf6xh2OMDjRKNVG8guMtaF/pFePfG4THn+PQkSl 9t0Q== X-Gm-Message-State: AFuF++le10XeIFj5C58ucc0yB7YDENsUsraQWc1REr80JptqaFoUO6jH /ACWCJ+gaRK0mnIZkqgMfsMUNAqCdj5jHhhgWTr/jnyUcQO+ZAgsUtOeO/UzMIShboI= X-Gm-Gg: AYBFou1DD/UqB3pHk1G+LntKv6GRBcookxBjBpSfXgGD1Oj987AhWH6Fnna7ztUhY5f XoTlfEZme60LRGOT/KDNkn6zc6CIDSZnC1GwCn1yKhMNb9+QSU6m8kOx1nI71BhWrwBigcopdj5 Lr0zHXeGJkZpxA5KsPe3iPDDowlcTbQQzS8MN6YN3etzojWMiXe1RBxm3YAnqAizlKS8HRWlrY2 Ckz1F+RD5RbYmG2J7BEer4UOdodlJH6ny947Ss7YnH1L7ES0zv3woYAbObWfopOURpUYQ9IYneh JuA9MqdmddcentYMeg9+UeO7aOjNdzr2IGRjRuEz5ggxlduYPi/tmX3KSGDiDIgWiYAi81Kdg+/ eT5425OBQhf9eXgQlE3pzs+9VYcrMC7X6DEQ3cO5G1z+ggvQRx3FZ+en0rvexL0DfIwFASg1tcQ 3Z7jMpHNMvrDyg/ikjR3o8ooGHmZHhsSoUMZJeyQcb46feNJ43mvR9RKJZWfyt1zws/hP2YcelQ xU4hppSCEISTgS3RHLU+PIa9oCr/ZVpBkrO/yuGIV1IpEr/tCxmWbtXYHn3 X-Received: by 2002:a05:620a:880a:b0:934:ab73:ac53 with SMTP id af79cd13be357-93c34573b4fmr67685585a.0.1790197848646; Wed, 23 Sep 2026 14:10:48 -0700 (PDT) Received: from gourry-fedora-PF4VCD3F.lan (pool-173-79-60-52.washdc.fios.verizon.net. [173.79.60.52]) by smtp.gmail.com with ESMTPSA id af79cd13be357-93c2485926esm305893085a.19.2026.09.23.14.10.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Sep 2026 14:10:48 -0700 (PDT) From: Gregory Price To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, kernel-team@meta.com, willy@infradead.org, jack@suse.cz, akpm@linux-foundation.org, david@kernel.org, gourry@gourry.net, ziy@nvidia.com, joshua.hahnjy@gmail.com, rakie.kim@sk.com, ying.huang@linux.alibaba.com, vbabka@kernel.org, surenb@google.com, mhocko@suse.com, brendan.jackman@linux.dev, hannes@cmpxchg.org, Brendan Jackman Subject: [RFC PATCH 3/6] mm/page_alloc: add an alloc_flags-aware folio allocator Date: Wed, 23 Sep 2026 17:10:37 -0400 Message-ID: <20260923211041.3127588-4-gourry@gourry.net> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260923211041.3127588-1-gourry@gourry.net> References: <20260923211041.3127588-1-gourry@gourry.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Brendan Jackman The low-level page allocator accepts explicit allocator behavior flags, but the folio helper hard-codes ALLOC_DEFAULT. Add an MM-internal folio allocator that accepts alloc_flags. Keep the existing exported folio and node interfaces unchanged as ALLOC_DEFAULT wrappers. No functional change intended. Signed-off-by: Brendan Jackman Signed-off-by: Gregory Price --- mm/page_alloc.c | 14 +++++++++++--- mm/page_alloc.h | 6 ++++++ 2 files changed, 17 insertions(+), 3 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index b4a021fad183..feb9c64efd84 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5510,13 +5510,21 @@ struct page *alloc_pages_node_noprof(int nid, gfp_t= gfp_mask, unsigned int order } EXPORT_SYMBOL(alloc_pages_node_noprof); =20 -struct folio *__folio_alloc_noprof(gfp_t gfp, unsigned int order, int pref= erred_nid, - nodemask_t *nodemask) +struct folio *__folio_alloc_flags_noprof(gfp_t gfp, unsigned int order, + int preferred_nid, nodemask_t *nodemask, + unsigned int alloc_flags) { struct page *page =3D __alloc_pages_noprof(gfp | __GFP_COMP, order, - preferred_nid, nodemask, ALLOC_DEFAULT); + preferred_nid, nodemask, alloc_flags); return page_rmappable_folio(page); } + +struct folio *__folio_alloc_noprof(gfp_t gfp, unsigned int order, + int preferred_nid, nodemask_t *nodemask) +{ + return __folio_alloc_flags_noprof(gfp, order, preferred_nid, nodemask, + ALLOC_DEFAULT); +} EXPORT_SYMBOL(__folio_alloc_noprof); =20 /* diff --git a/mm/page_alloc.h b/mm/page_alloc.h index b9259deddb59..986e733b586e 100644 --- a/mm/page_alloc.h +++ b/mm/page_alloc.h @@ -258,6 +258,12 @@ struct page *__alloc_pages_noprof(gfp_t gfp, unsigned = int order, int preferred_n nodemask_t *nodemask, unsigned int alloc_flags); #define __alloc_pages(...) alloc_hooks(__alloc_pages_noprof(__VA_ARGS__)) =20 +struct folio *__folio_alloc_flags_noprof(gfp_t gfp, unsigned int order, + int preferred_nid, nodemask_t *nodemask, + unsigned int alloc_flags); +#define __folio_alloc_flags(...) \ + alloc_hooks(__folio_alloc_flags_noprof(__VA_ARGS__)) + extern void zone_pcp_reset(struct zone *zone); extern void zone_pcp_disable(struct zone *zone); extern void zone_pcp_enable(struct zone *zone); --=20 2.55.0 From nobody Thu Sep 24 14:27:09 2026 Received: from mail-qv2-f20.google.com (mail-qv2-f20.google.com [74.125.230.148]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 437FA3FD969 for ; Wed, 23 Sep 2026 21:10:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.230.148 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790197856; cv=none; b=pUFs2mMawdZ5cyNvaFJrsywOC44lWYkHtRl8Vor1n7JLdtpWMdRtrGX+R0FwmNXLJ0ZYalk6BV0tYnt13Q2pswZT9QCZhwe9P9atL5LgozLyLoi8Pr0YZ+pGOouhwhvwXlH6HslN6gxzcrTEEgGebAk5FrB7TiA5idYSquxGJXo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790197856; c=relaxed/simple; bh=f7jLp3wHPMKJ2uLRlkyduFLNcqp1N3nkDtlgrZmrOxA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ldEgI+HcpNdXoIY0DJk0XsdB5uHedF3+RExJ6qCdu6foLZthIuQVrl6WQlqQ99W3KL6zl/KfxWQdcGt+asVzR7J8MQsW8qSMMzM3lhs1ikHmIv+x8thGATbf2Hb7wrl05yi1SpC7H4+Wm/1LnmHWtPvGxudnkz/OmG/sHtetXW4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=BHqR8XBl; arc=none smtp.client-ip=74.125.230.148 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="BHqR8XBl" Received: by mail-qv2-f20.google.com with SMTP id 6a1803df08f44-90cdfc9b6e4so13124986d6.3 for ; Wed, 23 Sep 2026 14:10:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1790197850; x=1790802650; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=C/Ioh0E0wp4TmybDNX/mkYZyZ5cUYapYs7dOai2tJpk=; b=BHqR8XBlbOtqtefjkV+fJT9SncSAw/sxdhOfw+7rGGaU41TwxJnLH3+2XV/i6CP8jR b46a4XbDeN6LOLro5NCa2eIE5WYkGSnkHnj1Dlk4yjYAa1XY9fIofA/gosw9NLVIY9x1 cawnmAganxt31/QJb0rDtPVvB9oESgZ5u3qKZrkPxGyX17v34wKtOSNqHX5Olve7wtZA 3TMIfSDsuujPly4Ht/d/eEax6E1M27bZVUZD2moFbwconVDTpo2IIIcmbJ8K7Nl6Be/e TW3LdHgG9CRG/Ek6MEhRiHMZyBB0/oaVnUaT4/71ukzaSBJBGlxO4WI3gOwq8b4i/eF3 ZrHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790197850; x=1790802650; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=C/Ioh0E0wp4TmybDNX/mkYZyZ5cUYapYs7dOai2tJpk=; b=oFRSt1UUVW0900pxpsM9GKKU/L0WR+ICIvJirksBrhLZZnljRkIe05aBytJmjRtnbQ z7bEYqq0mDsW2AyEbSKTpMBcM2rYK6F+P9INgBF7FdN8bo2JyG8EcF5kq5oTEX8Mc4h6 uWIjCysjNQeECAXQTXQIZKD5iJbjp/fmoTzNGUtiK1CcFATYDNuMYK8E7vcMWanxA08K UM2BcMD4d15cco2QEo1hckS9So8/q/0RZc35p2UkDplPROQ9U7r+Sl2IHnFK7aq0QqK4 AfKhacRivdR9H72QJXlo7etD1P6SUKUbIkU5sYbdLKNh43H98g+gda+r68jkxp01o6l0 Jfhw== X-Gm-Message-State: AFuF++nSE0HzJfxH96p4dzTzvM1Q+fLbKbwtYhk6G7XLT7Q7KSFdsISo tDJV/fOFjLYitDmTr3/ipjPRe/ETfXTHrw627sjz2c8VsmfnLyiq721jYlYleZA4iZ0= X-Gm-Gg: AYBFou10jL25RVGcYB7XpcoxxXhXJMHpgra8CE1kHzAwBTfRBjIfyjyo1JMgWR6yT5u vD3zFhs1+5MCscEhFachZwN/U3v0zVwYs9rTFwlFV9sB7g7SazdgeyA5cdfm5TmMxVREelKdSqJ FAMV4keWyEY2t2zesJHPMZDuBCKTPZnvUVF4V5MddL33+eQ1QwGjtIOA/7/Xd5UgSUrxKo6V9Id A+pFHX9JNtNyEJ5vk0zGerVvMnqdef0QxvGkaYMoTnFhr2Na+ToCS0MgtOJWkR+3BZOvS5/YAB5 zBx1Jk4N7Ud8vou1kcRX84bONecY0zqIn773nLHRKXuzzGoP8wghnGKAEQE3/MjzalZNhARkTFV ymXUI3Lpb/Gc9XiwU1ilZ8d7tKHJKes1c952AR1nQ8ivBKNfJkNbqpf0ZZ/BpwW0srcZWoLZ730 cDrNfLA0Azh5Vxn0UdqHeWOxrhOE2D4iADUXKEnvd/HI4duRlCQoBRIm61Xqz94wSHb4ODJL584 AJ2uMWn2OfwW90R3kmWEjlE3fGIuwy5FTfpT/uSFAZu9wLN3v5IoEqYqCO2tg== X-Received: by 2002:a05:620a:2690:b0:939:b55a:3c7 with SMTP id af79cd13be357-93c344ef4famr76423385a.41.1790197850184; Wed, 23 Sep 2026 14:10:50 -0700 (PDT) Received: from gourry-fedora-PF4VCD3F.lan (pool-173-79-60-52.washdc.fios.verizon.net. [173.79.60.52]) by smtp.gmail.com with ESMTPSA id af79cd13be357-93c2485926esm305893085a.19.2026.09.23.14.10.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Sep 2026 14:10:49 -0700 (PDT) From: Gregory Price To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, kernel-team@meta.com, willy@infradead.org, jack@suse.cz, akpm@linux-foundation.org, david@kernel.org, gourry@gourry.net, ziy@nvidia.com, joshua.hahnjy@gmail.com, rakie.kim@sk.com, ying.huang@linux.alibaba.com, vbabka@kernel.org, surenb@google.com, mhocko@suse.com, brendan.jackman@linux.dev, hannes@cmpxchg.org, Brendan Jackman Subject: [RFC PATCH 4/6] mm/mempolicy: plumb alloc_flags through folio allocation Date: Wed, 23 Sep 2026 17:10:38 -0400 Message-ID: <20260923211041.3127588-5-gourry@gourry.net> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260923211041.3127588-1-gourry@gourry.net> References: <20260923211041.3127588-1-gourry@gourry.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Brendan Jackman The NUMA policy allocation paths hard-code ALLOC_DEFAULT before reaching the page allocator. Add an alloc_flags argument to the MM-internal policy allocator and carry it through preferred-many, huge-page, and general policy allocation paths. Keep the public mempolicy folio interface as an ALLOC_DEFAULT wrapper. Task policy selection is not safe in every interrupt context, so non-task ALLOC_NOLOCK requests bypass it and use the supplied preferred node directly. No existing caller behavior is changed. Link: https://lore.kernel.org/r/20260726-page_alloc-unmapped-v3-25-6f5729aa= 9832@google.com Signed-off-by: Brendan Jackman Signed-off-by: Gregory Price --- mm/mempolicy.c | 44 ++++++++++++++++++++++++++++++++------------ mm/mempolicy.h | 28 ++++++++++++++++++++++++++++ 2 files changed, 60 insertions(+), 12 deletions(-) create mode 100644 mm/mempolicy.h diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 4c0b8ff1a7e6..2c01699081b7 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -120,6 +120,7 @@ #include =20 #include "internal.h" +#include "mempolicy.h" #include "page_alloc.h" =20 /* Internal flags */ @@ -2437,7 +2438,8 @@ bool mempolicy_in_oom_domain(struct task_struct *tsk, } =20 static struct page *alloc_pages_preferred_many(gfp_t gfp, unsigned int ord= er, - int nid, nodemask_t *nodemask) + int nid, nodemask_t *nodemask, + unsigned int alloc_flags) { struct page *page; gfp_t preferred_gfp; @@ -2451,10 +2453,10 @@ static struct page *alloc_pages_preferred_many(gfp_= t gfp, unsigned int order, preferred_gfp =3D gfp | __GFP_NOWARN; preferred_gfp &=3D ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL); page =3D __alloc_frozen_pages_noprof(preferred_gfp, order, nid, nodemask, - ALLOC_DEFAULT); + alloc_flags); if (!page) page =3D __alloc_frozen_pages_noprof(gfp, order, nid, NULL, - ALLOC_DEFAULT); + alloc_flags); =20 return page; } @@ -2466,11 +2468,13 @@ static struct page *alloc_pages_preferred_many(gfp_= t gfp, unsigned int order, * @pol: Pointer to the NUMA mempolicy. * @ilx: Index for interleave mempolicy (also distinguishes alloc_pages()). * @nid: Preferred node (usually numa_node_id() but @mpol may override it). + * @alloc_flags: ALLOC_* flags. * * Return: The page on success or NULL if allocation fails. */ static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order, - struct mempolicy *pol, pgoff_t ilx, int nid) + struct mempolicy *pol, pgoff_t ilx, int nid, + unsigned int alloc_flags) { nodemask_t *nodemask; struct page *page; @@ -2478,7 +2482,8 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsig= ned int order, nodemask =3D policy_nodemask(gfp, pol, ilx, &nid); =20 if (pol->mode =3D=3D MPOL_PREFERRED_MANY) - return alloc_pages_preferred_many(gfp, order, nid, nodemask); + return alloc_pages_preferred_many(gfp, order, nid, nodemask, + alloc_flags); =20 if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && /* filter "hugepage" allocation, unless from alloc_pages() */ @@ -2502,7 +2507,7 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsig= ned int order, */ page =3D __alloc_frozen_pages_noprof( gfp | __GFP_THISNODE | __GFP_NORETRY, order, - nid, NULL, ALLOC_DEFAULT); + nid, NULL, alloc_flags); if (page || !(gfp & __GFP_DIRECT_RECLAIM)) return page; /* @@ -2514,7 +2519,8 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsig= ned int order, } } =20 - page =3D __alloc_frozen_pages_noprof(gfp, order, nid, nodemask, ALLOC_DEF= AULT); + page =3D __alloc_frozen_pages_noprof(gfp, order, nid, nodemask, + alloc_flags); =20 if (unlikely(pol->mode =3D=3D MPOL_INTERLEAVE || pol->mode =3D=3D MPOL_WEIGHTED_INTERLEAVE) && page) { @@ -2530,11 +2536,18 @@ static struct page *alloc_pages_mpol(gfp_t gfp, uns= igned int order, return page; } =20 -struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order, - struct mempolicy *pol, pgoff_t ilx, int nid) +struct folio *__folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order, + struct mempolicy *pol, pgoff_t ilx, int nid, + unsigned int alloc_flags) { - struct page *page =3D alloc_pages_mpol(gfp | __GFP_COMP, order, pol, - ilx, nid); + struct page *page; + + if (unlikely((alloc_flags & ALLOC_NOLOCK) && !in_task())) + return __folio_alloc_flags_noprof(gfp, order, nid, NULL, + alloc_flags); + + page =3D alloc_pages_mpol(gfp | __GFP_COMP, order, pol, ilx, nid, + alloc_flags); if (!page) return NULL; =20 @@ -2542,6 +2555,13 @@ struct folio *folio_alloc_mpol_noprof(gfp_t gfp, uns= igned int order, return page_rmappable_folio(page); } =20 +struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order, + struct mempolicy *pol, pgoff_t ilx, int nid) +{ + return __folio_alloc_mpol_noprof(gfp, order, pol, ilx, nid, + ALLOC_DEFAULT); +} + /** * vma_alloc_folio - Allocate a folio for a VMA. * @gfp: GFP flags. @@ -2586,7 +2606,7 @@ struct page *alloc_frozen_pages_noprof(gfp_t gfp, uns= igned order) pol =3D get_task_policy(current); =20 return alloc_pages_mpol(gfp, order, pol, NO_INTERLEAVE_INDEX, - numa_node_id()); + numa_node_id(), ALLOC_DEFAULT); } =20 /** diff --git a/mm/mempolicy.h b/mm/mempolicy.h new file mode 100644 index 000000000000..8d2c07868f6e --- /dev/null +++ b/mm/mempolicy.h @@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * mm-internal API for mempolicy.c. Public API lives in + * include/linux/mempolicy.h. + */ +#ifndef __MM_MEMPOLICY_H +#define __MM_MEMPOLICY_H + +#include +#include + +#include "page_alloc.h" + +#ifdef CONFIG_NUMA +struct folio *__folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order, + struct mempolicy *pol, pgoff_t ilx, int nid, + unsigned int alloc_flags); +#else +static inline struct folio *__folio_alloc_mpol_noprof(gfp_t gfp, + unsigned int order, struct mempolicy *pol, pgoff_t ilx, int nid, + unsigned int alloc_flags) +{ + return __folio_alloc_flags_noprof(gfp, order, numa_node_id(), NULL, + alloc_flags); +} +#endif + +#endif --=20 2.55.0 From nobody Thu Sep 24 14:27:09 2026 Received: from mail-qk2-f13.google.com (mail-qk2-f13.google.com [74.125.230.205]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 47F8C418365 for ; Wed, 23 Sep 2026 21:10:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.230.205 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790197857; cv=none; b=FIiMRy4kdk8b1+GMRXApQZ8pK1xh+y7W+jYfLEbUXtKzjh9rDoy9FxhPs/k0PH6UXXjDtk9qPTR5N66WWplyg3hvSyziIxjLttfrnCVAinSU419jD0iTe2KNBTK+U+jfmndRt3jHxN/D/wRB0YyVYsq60PzscCDahT5ktVwRfpU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790197857; c=relaxed/simple; bh=8PRj6qx0RbwILj6Hf9BDGD4tq16yZjFSnmzbs9ElPF4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SlG/fSnP7Lx7Bh6KR+iJBejrdLjAAnMCwUssvWV5rLzcoaS2LCi0x61vwj8p/KI2Gsr4wdif8Q2BvUcoRrz5p8JlCJ4S96m6vQ/XEycp9SFYeB1WvkhBJNz+oWEhfAXH5XOME/M1CP2wgrnePAWilFAhDUwu2bkWqvarf0DXBSA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=XqWnKD2l; arc=none smtp.client-ip=74.125.230.205 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="XqWnKD2l" Received: by mail-qk2-f13.google.com with SMTP id af79cd13be357-939109fafddso93042085a.0 for ; Wed, 23 Sep 2026 14:10:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1790197852; x=1790802652; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=Ri8dUqYh2ciqE73sjjK4gAgS5aPTyFxGGtfGAJFHdJc=; b=XqWnKD2lCOBydJVl5hLCNoWD4kSNYPxz5yg0q4nMFB+Fs2UJldGhArZfkUqsgRPMNX pj7NLD4SuRj7MxDS668ZIbHFThAByt0K2d5NT9cUVaxqKsOddhIoVVXfUKzqWAHIHSAV yjA8CTA3f27H/ZtduPko9Mom4mm9sIDvu6Rxuhnkk5JWlwXNL2oDKHhyx8HecZT6nxu/ fF0+Otq5tSFnAa/T3pRWsE94yrgmk04WmoliBdsNm7BeLJIQ6AZC2fyBBQApejS49zWZ Msg7WAhzoYhiCrPO+h02gVL9PqMxrxRHBW6sKXvKXVPbb3M9EgfctjkWenA24pelT1qf fT6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790197852; x=1790802652; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=Ri8dUqYh2ciqE73sjjK4gAgS5aPTyFxGGtfGAJFHdJc=; b=tCwAsYHE5LVFqKWs83qr/NVwarBQZkk+oh4ibTy2xitXJNBd0rUVaBCsCJOVwc+8dA B2ROrLAbaiXh4Thi8AsTAOKoAX6IqjzOJczuTXnQWeLfMNthI9Zd9istXrKLVZGA/NSY kVIXTPl09r0QIwpXTNfaAok6nPkTk3+X6Brm7/Etcn8oY5gx3SBGJe1Jevmg5nuAB/d0 ZFGHn8Cm9mAtHEFLf7amfFAYp0dzroY956ydclw3LuLNB9JgAsvf1Pw4VqWnWL7EoiMZ p1yWGc/QN4Bovm7stG133uLKQXwDokz0aXQzWdyXDvWS7rp5r6t87uyvZRP6M+WX8uIr fVOA== X-Gm-Message-State: AFuF++kBDxXKnV5IEIJg16UGfCCQkcBaPEGoH05/Zd2NdBh9DbmKpsRS +PprH4I/OsMUu1WMfhW0MFCPqLRyr+mZ9GTo4BYrdX4mJEzYRXcJnbcxlPVjyxuWevQ= X-Gm-Gg: AYBFou2Y5C7K5Ai36U7wbXr+OR6pAzG1ScZZmzceajw3RDuzSQqAPI3JnMeAtpJBlqC V5geOuf8dBFST9nW4HfkRm5gud5Hj+Ba+lhfez4QKMomBsosh/OJrRIlHYJCOIcCqNOs9VkSvOS +exHwa3IYpTuFi8vfzHpiYfeqHLFVYKq4+nJwo2UhfDOTR3wqZZMATODD28JS9miWISl1nkx86n CLAfKVy18Ewkvvj2SgSIaog7gy6sga49mkueuNqvqrjhTjIJjh76G566ilRW6um2LMCQMStYRLs XKEETIzGB1AM0AOdAUd6OWYWW6okspHRssyMsshQhy9F1ZxG5VJgWNcaF8WZZ28+yE7YomuDP5c YkTeoZZvPywXU/70iORNcQBREeNMXZuOI29+aNRrIV73tTOyOMOn1pxUTAEk5mPe91yz83bDwB8 l/lnrtfkyMQkghNN/mpdUA62Xp/qEKPLnVIv3wEnK76eA6fraYS4D6KwIeNJA+7wuWSxd/fnw54 51PTz5FU68CgZrjUsrmcN63f7u7lE5M+snyruxDcs59Muk/9MNct6hlLuItRQ7dyh0OUbM= X-Received: by 2002:a05:620a:31a4:b0:93b:d79e:18f3 with SMTP id af79cd13be357-93c34540d1bmr71767185a.54.1790197852086; Wed, 23 Sep 2026 14:10:52 -0700 (PDT) Received: from gourry-fedora-PF4VCD3F.lan (pool-173-79-60-52.washdc.fios.verizon.net. [173.79.60.52]) by smtp.gmail.com with ESMTPSA id af79cd13be357-93c2485926esm305893085a.19.2026.09.23.14.10.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Sep 2026 14:10:51 -0700 (PDT) From: Gregory Price To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, kernel-team@meta.com, willy@infradead.org, jack@suse.cz, akpm@linux-foundation.org, david@kernel.org, gourry@gourry.net, ziy@nvidia.com, joshua.hahnjy@gmail.com, rakie.kim@sk.com, ying.huang@linux.alibaba.com, vbabka@kernel.org, surenb@google.com, mhocko@suse.com, brendan.jackman@linux.dev, hannes@cmpxchg.org, Brendan Jackman Subject: [RFC PATCH 5/6] mm/filemap: plumb alloc_flags through folio allocation Date: Wed, 23 Sep 2026 17:10:39 -0400 Message-ID: <20260923211041.3127588-6-gourry@gourry.net> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260923211041.3127588-1-gourry@gourry.net> References: <20260923211041.3127588-1-gourry@gourry.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Brendan Jackman The filemap folio allocator applies NUMA policy and node selection before reaching the low-level folio allocator. It currently provides no way for an MM-internal caller to preserve explicit allocator behavior flags across that selection. Add an internal filemap folio allocator that accepts alloc_flags and carries them through every allocation path. Keep filemap_alloc_folio_noprof() as an ALLOC_DEFAULT wrapper so existing callers and the exported interface remain unchanged. No functional change intended. Link: https://lore.kernel.org/r/20260726-page_alloc-unmapped-v3-26-6f5729aa= 9832@google.com Signed-off-by: Brendan Jackman Signed-off-by: Gregory Price --- mm/filemap.c | 45 +++++++++++++++++++++++++++++++++++++++------ 1 file changed, 39 insertions(+), 6 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index b74bc1e5015c..3b33cc873b73 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -52,6 +52,8 @@ =20 #include #include "internal.h" +#include "mempolicy.h" +#include "page_alloc.h" =20 #define CREATE_TRACE_POINTS #include @@ -995,31 +997,62 @@ int filemap_add_folio(struct address_space *mapping, = struct folio *folio, EXPORT_SYMBOL_GPL(filemap_add_folio); =20 #ifdef CONFIG_NUMA -struct folio *filemap_alloc_folio_noprof(gfp_t gfp, unsigned int order, - struct mempolicy *policy) +static inline struct folio *__filemap_alloc_folio_noprof(gfp_t gfp, + unsigned int order, struct mempolicy *policy, + unsigned int alloc_flags) { int n; struct folio *folio; =20 + /* Task policy and cpuset spreading are not safe in all IRQ contexts. */ + if (unlikely((alloc_flags & ALLOC_NOLOCK) && !in_task())) + return __folio_alloc_flags_noprof(gfp, order, numa_node_id(), + NULL, alloc_flags); + if (policy) - return folio_alloc_mpol_noprof(gfp, order, policy, - NO_INTERLEAVE_INDEX, numa_node_id()); + return __folio_alloc_mpol_noprof(gfp, order, policy, + NO_INTERLEAVE_INDEX, numa_node_id(), alloc_flags); =20 if (cpuset_do_page_mem_spread()) { unsigned int cpuset_mems_cookie; + do { cpuset_mems_cookie =3D read_mems_allowed_begin(); n =3D cpuset_mem_spread_node(); - folio =3D __folio_alloc_node_noprof(gfp, order, n); + folio =3D __folio_alloc_flags_noprof(gfp, order, n, NULL, + alloc_flags); } while (!folio && read_mems_allowed_retry(cpuset_mems_cookie)); =20 return folio; } - return folio_alloc_noprof(gfp, order); + + if (in_interrupt() || (gfp & __GFP_THISNODE)) + return __folio_alloc_flags_noprof(gfp, order, numa_node_id(), + NULL, alloc_flags); + + return __folio_alloc_mpol_noprof(gfp, order, get_task_policy(current), + NO_INTERLEAVE_INDEX, numa_node_id(), alloc_flags); +} + +struct folio *filemap_alloc_folio_noprof(gfp_t gfp, unsigned int order, + struct mempolicy *policy) +{ + return __filemap_alloc_folio_noprof(gfp, order, policy, ALLOC_DEFAULT); } EXPORT_SYMBOL(filemap_alloc_folio_noprof); +#else +static inline struct folio *__filemap_alloc_folio_noprof(gfp_t gfp, + unsigned int order, struct mempolicy *policy, + unsigned int alloc_flags) +{ + return __folio_alloc_flags_noprof(gfp, order, numa_node_id(), NULL, + alloc_flags); +} #endif =20 +#define __filemap_alloc_folio(...) \ + alloc_hooks(__filemap_alloc_folio_noprof(__VA_ARGS__)) + /* * filemap_invalidate_lock_two - lock invalidate_lock for two mappings * --=20 2.55.0 From nobody Thu Sep 24 14:27:09 2026 Received: from mail-qk2-f13.google.com (mail-qk2-f13.google.com [74.125.230.205]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D1645415F0B for ; Wed, 23 Sep 2026 21:10:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.230.205 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790197862; cv=none; b=tsgB/khg8tKCrikdpUxKXwReWworZdQsKHUkntsZpAobSgoTmjC76MXrL36DFcPNBAYt8l6dXcnBsFZeVeKMdIbnpQVMkMqp3cf3xriun8tA03+6r2iqWmOoKNZL6msP57GPt8kMzIq/wRgJwTRU0opxhky8t5O8a82ICqYbKrw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790197862; c=relaxed/simple; bh=vJCC8feguoeyY1Gu7GMMIAn/XZEg2QFDZSQSJ2H/ge0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Axi6TDjEPJaSUldTlAoQsR/ZfMm5eKTubslUlnh/HK9z0FtO3pGvwvphZARChfStNw3d2kyJOtAuHz59cR+rlN7+61ge7hAC/s5CxEcsI7W65utI3XHnjdTUm0dGIsvBUsckSmFB9raAQIhCjHcDZM/yJy+MyXUBLeFBLMVUX08= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net; spf=pass smtp.mailfrom=gourry.net; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b=G1AB8eJx; arc=none smtp.client-ip=74.125.230.205 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=gourry.net Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gourry.net Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gourry.net header.i=@gourry.net header.b="G1AB8eJx" Received: by mail-qk2-f13.google.com with SMTP id af79cd13be357-93910cadeb0so97989285a.3 for ; Wed, 23 Sep 2026 14:10:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gourry.net; s=google; t=1790197854; x=1790802654; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=z68SIWaMimWKPBbcPzrXpDhBMBPzNt9VtEr/ioTExD4=; b=G1AB8eJx1/1uzS2vlnhekJbGmQ+IKUs5MR/YxJ9FCbCU0E7vc1aP4jEbZoTEBw+Ki2 Drq9Cybukp3V7SzKwgv2+47JN9G+woIMI5mf0vuTE6ddsB+IEby796P2wrwIjl+lSCzX lVl3RIkRNwVoXKTdQqI3GixRZNxAS9+npffxU5w/Lm62rmqHyM7s75aiJGjhRPghm4ZO iOljVVYcZN3MT4Vz460RXSv2YcNEoIhfWxNVlffhQKzIWzOM4DGB3sFKR46ekaMkA/Yt +ZFXNvVIVNh6yIQLTVd3hyxUYUbV6iFsL54VgRgdJqM3xRk5djrSqAl8b9jcP/Hhowej UPKw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790197854; x=1790802654; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=z68SIWaMimWKPBbcPzrXpDhBMBPzNt9VtEr/ioTExD4=; b=nUqdRKggBLUOH6HtULo6xobfRKipn0VOmqdpzxKJ1bsqAXYUE8D874JlixNNaBPDxP OC0FFVa2LAc4XkPCXOptx9Nlrf+NkXqSm6LoEbyL0JPcPWbGGqMnjGtN10DdDLQ7g1jJ rBa1C9a0AqP0KmigE0TKo+ElltDnGqV9dsrcEQjbqpDYcOWKQlPSfce4AlaidSNLWi4D yIcGOEs3l3eCv+3r8YQbKrn1UF7plq7eCR88aOiy3L1cskpfcC/9n+pwYfy6aTpXogwz +REsl4jXHq70V+kWkYHHOfc9R0/9CpYbbr6PWGHcmNpABU1Z40tIH2YuMFsL7xwuUofK 3HQw== X-Gm-Message-State: AFuF++kRwAGqKCawG+JQPOw4JIFlBzUb19T2GubLHmFuSO3TSXf3DYYV xr62S0F7/VohxNkHn5tnZeRl5xEj8l1AiYZ+CkuZcrl4KbC1FMOuV8AaSZieh8Rm9x0= X-Gm-Gg: AYBFou3w1h3LVjALlNzIpLcBQBYEJ505mvtDoOuBKPwoMOYm+xeT1QPZVRTez7jJDN+ qlK/e01K7ydl5aFXDeeNEACgiocGl49Sh5Kb6strVWr+tl2nUyTrx94U4iqoVSr76Qqcw8YP0mL K+YEMfehyFe96DiDRfms4gnfeh15eOycagInZnAAU12FXtvawZJgpG+HAqfim8XkmgLAM2vIy9C jDZ6w6xKA3xVgDCZx/eGfJh9A9Bd7Qio+Rn35es05WUZ0VWCV73kEEshhKGepRkdpsXGXj3XXt7 /zuN1MmhtvARDnTUHPoeHFZ8eRHR7t3o0cvc+zoIVnKRFA+acXgpucPbRMcsELb8xR6UGvU5Rl4 nJZgZrNoI6fCQMrzhj18U5ij7k76AVgQ+YymQR8JbM3Xij4qR8ojhTZ4HmjStlMaGgULB6W7xmQ Jzk6SmqypfZDOtWhUXxSSfVCQJAiyU+cxVccvesyHi43HP0tEJHtmYsKqgbRJ9FRmM2Of0Y6M/I z4CncqVc0Kgg/hS1qgHktDtSMna0L0oJPQz49UExSHrl0q2X8ZZiD7XGOPI X-Received: by 2002:a05:620a:25ce:b0:93b:d79a:c464 with SMTP id af79cd13be357-93c3480c8famr61505485a.60.1790197853636; Wed, 23 Sep 2026 14:10:53 -0700 (PDT) Received: from gourry-fedora-PF4VCD3F.lan (pool-173-79-60-52.washdc.fios.verizon.net. [173.79.60.52]) by smtp.gmail.com with ESMTPSA id af79cd13be357-93c2485926esm305893085a.19.2026.09.23.14.10.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Sep 2026 14:10:53 -0700 (PDT) From: Gregory Price To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, kernel-team@meta.com, willy@infradead.org, jack@suse.cz, akpm@linux-foundation.org, david@kernel.org, gourry@gourry.net, ziy@nvidia.com, joshua.hahnjy@gmail.com, rakie.kim@sk.com, ying.huang@linux.alibaba.com, vbabka@kernel.org, surenb@google.com, mhocko@suse.com, brendan.jackman@linux.dev, hannes@cmpxchg.org Subject: [RFC PATCH 6/6] mm/page_alloc: let the bulk allocator carry alloc_flags Date: Wed, 23 Sep 2026 17:10:40 -0400 Message-ID: <20260923211041.3127588-7-gourry@gourry.net> X-Mailer: git-send-email 2.55.0 In-Reply-To: <20260923211041.3127588-1-gourry@gourry.net> References: <20260923211041.3127588-1-gourry@gourry.net> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" __alloc_pages_noprof() accepts explicit allocator behavior flags, but the bulk allocator does not. Internal callers therefore cannot select those behaviors through the bulk interface. Add __alloc_pages_bulk_noprof() for flag-carrying MM callers. Keep alloc_pages_bulk_noprof() as a flag-free wrapper passing ALLOC_DEFAULT, so existing callers and the exported interface remain unchanged. Preserve the original GFP mask for the single-page fallback so it repeats flag preparation from the caller inputs, and pass the prepared fast-path flags to prep_new_page(). Signed-off-by: Gregory Price --- mm/page_alloc.c | 28 +++++++++++++++++++--------- mm/page_alloc.h | 4 ++++ 2 files changed, 23 insertions(+), 9 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index feb9c64efd84..8fa2522ef418 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5219,6 +5219,7 @@ static __always_inline bool prepare_alloc_flags(gfp_t= *gfp, unsigned int order, /* * __alloc_pages_bulk - Allocate a number of order-0 pages to an array * @gfp: GFP flags for the allocation + * @alloc_flags: Allocation behavior flags * @preferred_nid: The preferred NUMA node ID to allocate from * @nodemask: Set of nodes to allocate from, may be NULL * @nr_pages: The number of pages desired in the array @@ -5237,15 +5238,16 @@ static __always_inline bool prepare_alloc_flags(gfp= _t *gfp, unsigned int order, * @page_array were set to %NULL on entry, the slots from 0 to the return = value * - 1 will be filled. */ -unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid, - nodemask_t *nodemask, int nr_pages, - struct page **page_array) +unsigned long __alloc_pages_bulk_noprof(gfp_t gfp, unsigned int alloc_flag= s, + int preferred_nid, nodemask_t *nodemask, int nr_pages, + struct page **page_array) { struct page *page; struct zone *zone; struct zoneref *z; struct per_cpu_pages *pcp; struct list_head *pcp_list; + gfp_t orig_gfp =3D gfp; struct alloc_context ac; unsigned int fastpath_alloc_flags; int nr_populated =3D 0, nr_account =3D 0; @@ -5285,7 +5287,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, goto failed; #endif =20 - if (!prepare_alloc_flags(&gfp, 0, ALLOC_DEFAULT, &ac.alloc_flags, + if (!prepare_alloc_flags(&gfp, 0, alloc_flags, &ac.alloc_flags, &fastpath_alloc_flags)) goto out; =20 @@ -5296,7 +5298,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, &fastpath_alloc_flags)) goto out; =20 - /* Find an allowed local zone that meets the low watermark. */ + /* Find an allowed local zone that meets the required watermark. */ z =3D ac.preferred_zoneref; for_next_zone_zonelist_nodemask(zone, z, ac.highest_zoneidx, ac.nodemask)= { unsigned long mark; @@ -5332,8 +5334,8 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, } =20 /* - * If there are no allowed local zones that meets the watermarks then - * try to allocate a single page and reclaim if necessary. + * If there are no allowed local zones that meet the watermarks, try the + * single-page allocator. */ if (unlikely(!zone)) goto failed; @@ -5366,7 +5368,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, } nr_account++; =20 - prep_new_page(page, 0, gfp, ALLOC_DEFAULT); + prep_new_page(page, 0, gfp, fastpath_alloc_flags); trace_mm_page_alloc(page, 0, gfp, ac.migratetype); kmsan_alloc_page(page, 0, gfp & ~__GFP_RECLAIM); set_page_refcounted(page); @@ -5382,11 +5384,19 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, in= t preferred_nid, return nr_populated; =20 failed: - page =3D __alloc_pages_noprof(gfp, 0, preferred_nid, nodemask, ALLOC_DEFA= ULT); + page =3D __alloc_pages_noprof(orig_gfp, 0, preferred_nid, nodemask, + alloc_flags); if (page) page_array[nr_populated++] =3D page; goto out; } + +unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid, + nodemask_t *nodemask, int nr_pages, struct page **page_array) +{ + return __alloc_pages_bulk_noprof(gfp, ALLOC_DEFAULT, preferred_nid, + nodemask, nr_pages, page_array); +} EXPORT_SYMBOL_GPL(alloc_pages_bulk_noprof); =20 /* diff --git a/mm/page_alloc.h b/mm/page_alloc.h index 986e733b586e..a09bfd201c3b 100644 --- a/mm/page_alloc.h +++ b/mm/page_alloc.h @@ -264,6 +264,10 @@ struct folio *__folio_alloc_flags_noprof(gfp_t gfp, un= signed int order, #define __folio_alloc_flags(...) \ alloc_hooks(__folio_alloc_flags_noprof(__VA_ARGS__)) =20 +unsigned long __alloc_pages_bulk_noprof(gfp_t gfp, unsigned int alloc_flag= s, + int preferred_nid, nodemask_t *nodemask, int nr_pages, + struct page **page_array); + extern void zone_pcp_reset(struct zone *zone); extern void zone_pcp_disable(struct zone *zone); extern void zone_pcp_enable(struct zone *zone); --=20 2.55.0