From nobody Sat Apr 4 06:08:15 2026 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 49AA73EE1E3 for ; Fri, 20 Mar 2026 18:24:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774031049; cv=none; b=B20mY6+hOPptFD9tq6rflEVGcKWHU8q0mYAj4eNCGEvtceCWbdCty7d4a2UXsiBJhgtAVKq6jrQGZ3RuzvSKlpc9G8n+r0Hm0L3Hf0VNZq9j+0kEIXThKMI/B4jH9y1iFo0bNgslr25Iho1VZVHyZnhB/PrIeKoNmZ0ac5fIBFY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774031049; c=relaxed/simple; bh=p5V9+rxCoQlpv7ntVns5iiNQyRSTevBFEOoboYChBeI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=anjV807M2OIKn9rTN+p7YHDDFnazd1veFkgJou8ja6juEOZQUVrm9dIrFIGq312eiQFGSYudaiCjspwXFEpf21TbDpYuKTvEvmy9ZPqMF0llA14ZmJwtdMKUI+xeb5cCzOk2h3B3hLCv2XwAqNWCCak/tocyFIf3G1bQC4X2Hvc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--jackmanb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=lDn4DkB9; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--jackmanb.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="lDn4DkB9" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-486fa35b005so21872335e9.2 for ; Fri, 20 Mar 2026 11:24:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1774031047; x=1774635847; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=A4tHzLv2BIce7TOZ68EmuhqmY6Sg+by5v9VmsY48yBI=; b=lDn4DkB9DC9XwwQXXAKo/nf5u1HWv2SvdqkpSi1h8sRe+nl+H/QQg27BSTw2PCIDDA Te5QFitSWFPH1KMOrVgw7QVhx2SadyT6l8jLYaK3jwWqXEowcZYz3bzxjSaKT+BeZesX N5ghznoRZj9sInRRN5DzuanIX4JhDihLEYm5sN9H0BJ6npHM5Qe8hLg5u/hGTvSsKqAK AOunJsKq0xL6FSFh3/5kW7jK59bm/PFbHVvxnmJqb7t/v59ohTNRrq4CqosZHHA/GsEM CP8zJaFA54sK+3LTspV2yv6XdJNNp5RAhSyV2daV7pTIHLRBSY6fVSEqIbzFBuoaZDLN 9v0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1774031047; x=1774635847; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=A4tHzLv2BIce7TOZ68EmuhqmY6Sg+by5v9VmsY48yBI=; b=ImJdtogxPZ1WNmtyCvWglHF2SAfChIS15AAOM2ciMuMsq8LNF6AWrwVFKnAD/9L2C0 T3UkOWS1I45I225cN7oM5PWQY/YMuEArJ+gAcyn+MIhZIWe4Irp76imJoNMcHL8CUI48 +kldOOGBU5sashaAeOMRSouVjqeirBNGU1NobwtwNp9p1Tvup8AZHZuoi/FEZLp3DoP9 a6geSkuuOjOHGMamiUmxdP2b4iRYr6ap9l4UvD3eyZ21ChZpxSfRyTXT6dNJFtA6zjPP 2Qz1pbDz5AProKYa1dWai8AZo7Tw7T2H5C/zgHRQ8NS0jGbt8bR7BMPSNxCOnAP8KDAC +GXQ== X-Forwarded-Encrypted: i=1; AJvYcCUDDtbbiImyBoMk/t7CzloHSZdUFyPSTJlbNhdDRKQT/wFe8OsdjUexZRTciFrDJ1qSBraIfjhrUCClYSU=@vger.kernel.org X-Gm-Message-State: AOJu0Yw1hGvOKpPKFbTlIKXHam+KJeJqZdoaFCvHchauhNZYW1iKasEJ u7e2TbLLeLrRay+nvdHVD0ZxyxDVJkHWzTDfmK94AwuyADWDcAqqPxdzkmeBFzI2TF4XF3e5uKO hfq7gvZIb9tuPxQ== X-Received: from wmi11.prod.google.com ([2002:a05:600c:20b:b0:477:5a4b:d57f]) (user=jackmanb job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:8508:b0:485:3b00:f92e with SMTP id 5b1f17b1804b1-486fedab7dbmr58230605e9.2.1774031046800; Fri, 20 Mar 2026 11:24:06 -0700 (PDT) Date: Fri, 20 Mar 2026 18:23:42 +0000 In-Reply-To: <20260320-page_alloc-unmapped-v2-0-28bf1bd54f41@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260320-page_alloc-unmapped-v2-0-28bf1bd54f41@google.com> X-Mailer: b4 0.14.3 Message-ID: <20260320-page_alloc-unmapped-v2-18-28bf1bd54f41@google.com> Subject: [PATCH v2 18/22] mm/page_alloc: introduce ALLOC_NOBLOCK From: Brendan Jackman To: Borislav Petkov , Dave Hansen , Peter Zijlstra , Andrew Morton , David Hildenbrand , Vlastimil Babka , Wei Xu , Johannes Weiner , Zi Yan , Lorenzo Stoakes Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, x86@kernel.org, rppt@kernel.org, Sumit Garg , derkling@google.com, reijiw@google.com, Will Deacon , rientjes@google.com, "Kalyazin, Nikita" , patrick.roy@linux.dev, "Itazuri, Takahiro" , Andy Lutomirski , David Kaplan , Thomas Gleixner , Brendan Jackman , Yosry Ahmed Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable This flag is set unless we can be sure the caller isn't in an atomic context. The allocator will soon start needing to call set_direct_map_* APIs which cannot be called with IRQs off. It will need to do this even before direct reclaim is possible. Despite the fact that, in principle, ALLOC_NOBLOCK is distinct from __GFP_DIRECT_RECLAIM, in order to avoid introducing a GFP flag, just infer the former based on whether the caller set the latter. This means that, in practice, ALLOC_NOBLOCK is just !__GFP_DIRECT_RECLAIM, except that it is not influenced by gfp_allowed_mask. This could change later, though. Call it ALLOC_NOBLOCK in order to try and mitigate confusion vs the recently-removed ALLOC_NON_BLOCK, which meant something different. Signed-off-by: Brendan Jackman --- mm/internal.h | 1 + mm/page_alloc.c | 29 ++++++++++++++++++++++------- 2 files changed, 23 insertions(+), 7 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index cc19a90a7933f..865991aca06ea 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -1431,6 +1431,7 @@ unsigned int reclaim_clean_pages_from_list(struct zon= e *zone, #define ALLOC_HIGHATOMIC 0x200 /* Allows access to MIGRATE_HIGHATOMIC */ #define ALLOC_TRYLOCK 0x400 /* Only use spin_trylock in allocation path */ #define ALLOC_KSWAPD 0x800 /* allow waking of kswapd, __GFP_KSWAPD_RECLAI= M set */ +#define ALLOC_NOBLOCK 0x1000 /* Caller may be atomic */ =20 /* Flags that allow allocations below the min watermark. */ #define ALLOC_RESERVES (ALLOC_HARDER|ALLOC_MIN_RESERVE|ALLOC_HIGHATOMIC|AL= LOC_OOM) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 9a07c552a1f8a..83d06a6db6433 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -4608,6 +4608,8 @@ gfp_to_alloc_flags(gfp_t gfp_mask, unsigned int order) (gfp_mask & (__GFP_HIGH | __GFP_KSWAPD_RECLAIM)); =20 if (!(gfp_mask & __GFP_DIRECT_RECLAIM)) { + alloc_flags |=3D ALLOC_NOBLOCK; + /* * Not worth trying to allocate harder for __GFP_NOMEMALLOC even * if it can't schedule. @@ -4801,14 +4803,13 @@ check_retry_cpuset(int cpuset_mems_cookie, struct a= lloc_context *ac) =20 static inline struct page * __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order, - struct alloc_context *ac) + struct alloc_context *ac, unsigned int alloc_flags) { bool can_direct_reclaim =3D gfp_mask & __GFP_DIRECT_RECLAIM; bool can_compact =3D can_direct_reclaim && gfp_compaction_allowed(gfp_mas= k); bool nofail =3D gfp_mask & __GFP_NOFAIL; const bool costly_order =3D order > PAGE_ALLOC_COSTLY_ORDER; struct page *page =3D NULL; - unsigned int alloc_flags; unsigned long did_some_progress; enum compact_priority compact_priority; enum compact_result compact_result; @@ -4860,7 +4861,7 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int o= rder, * kswapd needs to be woken up, and to avoid the cost of setting up * alloc_flags precisely. So we do that now. */ - alloc_flags =3D gfp_to_alloc_flags(gfp_mask, order); + alloc_flags |=3D gfp_to_alloc_flags(gfp_mask, order); =20 /* * We need to recalculate the starting point for the zonelist iterator @@ -5086,6 +5087,18 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int = order, return page; } =20 +static inline unsigned int init_alloc_flags(gfp_t gfp_mask, unsigned int f= lags) +{ + /* + * If the caller allowed __GFP_DIRECT_RECLAIM, they can't be atomic. + * Note this is a separate determination from whether direct reclaim is + * actually allowed, it must happen before applying gfp_allowed_mask. + */ + if (!(gfp_mask & __GFP_DIRECT_RECLAIM)) + flags |=3D ALLOC_NOBLOCK; + return flags; +} + static inline bool prepare_alloc_pages(gfp_t gfp_mask, unsigned int order, int preferred_nid, nodemask_t *nodemask, struct alloc_context *ac, gfp_t *alloc_gfp, @@ -5166,7 +5179,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, struct list_head *pcp_list; struct alloc_context ac; gfp_t alloc_gfp; - unsigned int alloc_flags =3D ALLOC_WMARK_LOW; + unsigned int alloc_flags =3D init_alloc_flags(gfp, ALLOC_WMARK_LOW); int nr_populated =3D 0, nr_account =3D 0; =20 /* @@ -5307,7 +5320,7 @@ struct page *__alloc_frozen_pages_noprof(gfp_t gfp, u= nsigned int order, int preferred_nid, nodemask_t *nodemask) { struct page *page; - unsigned int alloc_flags =3D ALLOC_WMARK_LOW; + unsigned int alloc_flags =3D init_alloc_flags(gfp, ALLOC_WMARK_LOW); gfp_t alloc_gfp; /* The gfp_t that was actually used for allocation */ struct alloc_context ac =3D { }; =20 @@ -5352,7 +5365,7 @@ struct page *__alloc_frozen_pages_noprof(gfp_t gfp, u= nsigned int order, */ ac.nodemask =3D nodemask; =20 - page =3D __alloc_pages_slowpath(alloc_gfp, order, &ac); + page =3D __alloc_pages_slowpath(alloc_gfp, order, &ac, alloc_flags); =20 out: if (memcg_kmem_online() && (gfp & __GFP_ACCOUNT) && page && @@ -7872,11 +7885,13 @@ struct page *alloc_frozen_pages_nolock_noprof(gfp_t= gfp_flags, int nid, unsigned */ gfp_t alloc_gfp =3D __GFP_NOWARN | __GFP_ZERO | __GFP_NOMEMALLOC | __GFP_= COMP | gfp_flags; - unsigned int alloc_flags =3D ALLOC_TRYLOCK; + unsigned int alloc_flags =3D init_alloc_flags(alloc_gfp, ALLOC_TRYLOCK); struct alloc_context ac =3D { }; struct page *page; =20 VM_WARN_ON_ONCE(gfp_flags & ~__GFP_ACCOUNT); + VM_WARN_ON_ONCE(!(alloc_flags & ALLOC_NOBLOCK)); + /* * In PREEMPT_RT spin_trylock() will call raw_spin_lock() which is * unsafe in NMI. If spin_trylock() is called from hard IRQ the current --=20 2.51.2