From nobody Wed Jun 17 02:49:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 50AF830E0D6 for ; Tue, 21 Apr 2026 22:01:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808883; cv=none; b=Zp9G/rAgcfnmTrvaudaI3VB0EBc8RV4Hv/nOUbBgsgPCDGND9DnTxNTKGkhSMkcM00ZuXZBRysXRavMJ6K4f9t1GDXNsVLWXinuSbyMSvHYbwi6qb2WZCGLGRUG7SBfY+IbReNDtW5QE5JbQkuEtg5lc/+Zx9mW4W1P01QJB0J8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808883; c=relaxed/simple; bh=9EKMu0q4T5CXWStF6YI05YDQp8pVyMFXjsLQILWW/H0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=UcTN0IdMVGCCMEs51hiwOW3n5AZ841iqDEBdkN/0LlnPlhxp8dWJ3sLJdl6WaDvgw3/vzUHX9qoKM9byqruLSZ7CPx958BCZB3rzEZ5OugXarni4krl7B220yXWjtKk0BLq2FxzSAPp148utQxLb5kWjCqEa4CUTy1lw2Lu8U9A= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Ipqywo9A; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=I2KOMYzo; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Ipqywo9A"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="I2KOMYzo" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1776808880; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=n/RSJYR+H+bpWdktahJfxrAiG8FsUf0fzB70AZgkr2Y=; b=Ipqywo9AxhwSBlmRY2/bParGyUq68MozoRVNDwCzds/b3Zs5xRWiMWalRGDIG3tNQMUXYv gwI6BG1/MLWAQHGPRkyxd3aKRp4CMJpKxbm+QNY9k0udDMFG+FfptzxG2mkXuIYCUJY8Ld va/HEWwg+XkN972kkavEKln4RA7oRJY= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-473-JRq6yRzJOLySlW1Y8qttIw-1; Tue, 21 Apr 2026 18:01:18 -0400 X-MC-Unique: JRq6yRzJOLySlW1Y8qttIw-1 X-Mimecast-MFC-AGG-ID: JRq6yRzJOLySlW1Y8qttIw_1776808877 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-43d03065782so3208271f8f.0 for ; Tue, 21 Apr 2026 15:01:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1776808877; x=1777413677; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=n/RSJYR+H+bpWdktahJfxrAiG8FsUf0fzB70AZgkr2Y=; b=I2KOMYzoBza7QLqkDRml6qnw06tIkjBmTRm1QEDYiegBJpSXaX7h39G9G+jFw3SLeu 64OAYNM1yMeex6Id7CSOPXhEc75mN7EIzTsjkitK9io/e5rOModwT/aHk5bRSewzivXY d2d2wVp31GYpuHvmEbEOYbwCjAk5MK9/yE439ulLuEAlotDEPQHtNa8AC26CNfAjYXep 8AZHQa52Y4iF3PIKCvDjqvIpv1s/Uc1+Tu/6PaxaOMsa8bvSYOJl+SyRDQY6oCSYObSf n+hGiNUNRK2qoM+sJ70E7rMji4LFVQgPfQAsgfYH0HVQbtfwzVgtXWDDM3xYw/bgQosO AdbA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776808877; x=1777413677; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=n/RSJYR+H+bpWdktahJfxrAiG8FsUf0fzB70AZgkr2Y=; b=UhPTsTG++r3RcvfO0FPSggodoCVME0Nw/VCaM6jjKoXp061/U/b0RnfrpKTlCZM5c8 iu9ME9wJ2SEqDk83TFoOjve4jcU2DVTvp5U2KZMrfeARfXA/1DV+5lAwoFG/wzbYoX8O PCCh4Jzz0oetcXhuKVxEWJhZqcoPs2Qta04btE+A26Xc1Z3kxfFRUnfJpHAqvn9Qulr0 5vmXiPhDYW3iyH2oJOc7GysmpD47x6XTydvmG/+YCro/kqSFNtA1VAQtis50/Bd4u+Z3 P35OVX2Gu/82IPfXxz2z9qn48MN/JF0KyWt9W3qhmuv35R428Kidd5NxyeTKmg+yhCNr +NpQ== X-Gm-Message-State: AOJu0Yz2kVBgf9dSvUPy+vBwDjZfsErcu2CIWeosX1tEpmPjPumchPzB iulYWSl8bNnUaKA4ULNx4qrP9Vhddf5GYNYzHNgG94/tAq3LVenLEmdquI70b8LVQFa1w6kzrYD B6yEzSG3/HnNUS4699y3tYKbrCVrqid/fwFKDK9KqsZeTfnUhP2vuN/P6ox9ByosGW8J4GgXVvq mkWh+ph0C+c0YPSNOdwmejCLkIQrDbo807m4d7Ny9kZsw= X-Gm-Gg: AeBDies8CND0b/HDLSOayEcjQZnGVG9gG4wusx9swgVuDL+ngdgsjv2sf0qk+iFA2nU xrTtmIExrW81oik9oCKMJbvV+KMQ3i0xSC3E/NmWgrf0H65N+36vQvGXFpoYaEiaGCk6/rPeqaJ C44LhylaexsYeE9tIlNFEG9LOeVKvqFSvZSGoI/JfL+u+mFWHVVtAjtmisQfFHxmq16Q9V2fkDv WOWaN30PwRo7oCgOap5s46iYmTmhBzg8PY1BxEpl8zUIukplYnGjokWfD+UZ+vyeQqRrr4tjqPL wKdJsopYV1zvxd5C93t7TGc9W+R38yZ/LCa67wl8UKlpzNJLCS8bcL7PkyB7XIeUb9B173ElJvi wj/7eSRGYNcpUEfw+8BAX0PrgVe1zi9qVm3asuX/A3CrHkiH6vym4eQ== X-Received: by 2002:a05:6000:40ca:b0:43d:70de:1c62 with SMTP id ffacd0b85a97d-43fe3e120c6mr31540753f8f.45.1776808876756; Tue, 21 Apr 2026 15:01:16 -0700 (PDT) X-Received: by 2002:a05:6000:40ca:b0:43d:70de:1c62 with SMTP id ffacd0b85a97d-43fe3e120c6mr31540636f8f.45.1776808875976; Tue, 21 Apr 2026 15:01:15 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-21.inter.net.il. [80.230.25.21]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-43fe4cc0f31sm40811860f8f.12.2026.04.21.15.01.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Apr 2026 15:01:15 -0700 (PDT) Date: Tue, 21 Apr 2026 18:01:10 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Johannes Weiner , Zi Yan , Lorenzo Stoakes , "Liam R. Howlett" , Mike Rapoport , "Matthew Wilcox (Oracle)" , Muchun Song , Oscar Salvador , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Ying Huang , Alistair Popple , Hugh Dickins , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , linux-fsdevel@vger.kernel.org Subject: [PATCH RFC v3 01/19] mm: thread user_addr through page allocator for cache-friendly zeroing Message-ID: <9dd9deabd42801f3c344326991d1431c3d8db39d.1776808210.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Thread a user virtual address from vma_alloc_folio() down through the page allocator to post_alloc_hook(). This is plumbing preparation for a subsequent patch that will use user_addr to call folio_zero_user() for cache-friendly zeroing of user pages. The user_addr is stored in struct alloc_context and flows through: vma_alloc_folio -> folio_alloc_mpol -> __alloc_pages_mpol -> __alloc_frozen_pages -> get_page_from_freelist -> prep_new_page -> post_alloc_hook Public APIs (__alloc_pages, __folio_alloc, folio_alloc_mpol) gain a user_addr parameter directly. Callers that do not need user_addr pass USER_ADDR_NONE ((unsigned long)-1), since address 0 is a valid user mapping. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- include/linux/gfp.h | 25 +++++++++++++++++-------- mm/compaction.c | 6 ++---- mm/filemap.c | 3 ++- mm/hugetlb.c | 36 ++++++++++++++++++++---------------- mm/internal.h | 9 ++++++--- mm/khugepaged.c | 2 +- mm/mempolicy.c | 39 ++++++++++++++++++++++++++------------- mm/migrate.c | 2 +- mm/page_alloc.c | 38 ++++++++++++++++++++++---------------- mm/page_frag_cache.c | 4 ++-- mm/shmem.c | 2 +- mm/slub.c | 4 ++-- mm/swap_state.c | 2 +- 13 files changed, 103 insertions(+), 69 deletions(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index 51ef13ed756e..10f653338042 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -226,12 +226,18 @@ static inline void arch_free_page(struct page *page, = int order) { } static inline void arch_alloc_page(struct page *page, int order) { } #endif =20 +/* + * Sentinel for user_addr: indicates a non-user allocation. + * Cannot use 0 because address 0 is a valid userspace mapping. + */ +#define USER_ADDR_NONE ((unsigned long)-1) + struct page *__alloc_pages_noprof(gfp_t gfp, unsigned int order, int prefe= rred_nid, - nodemask_t *nodemask); + nodemask_t *nodemask, unsigned long user_addr); #define __alloc_pages(...) alloc_hooks(__alloc_pages_noprof(__VA_ARGS__)) =20 struct folio *__folio_alloc_noprof(gfp_t gfp, unsigned int order, int pref= erred_nid, - nodemask_t *nodemask); + nodemask_t *nodemask, unsigned long user_addr); #define __folio_alloc(...) alloc_hooks(__folio_alloc_noprof(__VA_ARGS__)) =20 unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int preferred_nid, @@ -286,7 +292,7 @@ __alloc_pages_node_noprof(int nid, gfp_t gfp_mask, unsi= gned int order) VM_BUG_ON(nid < 0 || nid >=3D MAX_NUMNODES); warn_if_node_offline(nid, gfp_mask); =20 - return __alloc_pages_noprof(gfp_mask, order, nid, NULL); + return __alloc_pages_noprof(gfp_mask, order, nid, NULL, USER_ADDR_NONE); } =20 #define __alloc_pages_node(...) alloc_hooks(__alloc_pages_node_noprof(__= VA_ARGS__)) @@ -297,7 +303,7 @@ struct folio *__folio_alloc_node_noprof(gfp_t gfp, unsi= gned int order, int nid) VM_BUG_ON(nid < 0 || nid >=3D MAX_NUMNODES); warn_if_node_offline(nid, gfp); =20 - return __folio_alloc_noprof(gfp, order, nid, NULL); + return __folio_alloc_noprof(gfp, order, nid, NULL, USER_ADDR_NONE); } =20 #define __folio_alloc_node(...) alloc_hooks(__folio_alloc_node_noprof(__= VA_ARGS__)) @@ -322,7 +328,8 @@ static inline struct page *alloc_pages_node_noprof(int = nid, gfp_t gfp_mask, struct page *alloc_pages_noprof(gfp_t gfp, unsigned int order); struct folio *folio_alloc_noprof(gfp_t gfp, unsigned int order); struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order, - struct mempolicy *mpol, pgoff_t ilx, int nid); + struct mempolicy *mpol, pgoff_t ilx, int nid, + unsigned long user_addr); struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, struct vm_area_= struct *vma, unsigned long addr); #else @@ -335,14 +342,16 @@ static inline struct folio *folio_alloc_noprof(gfp_t = gfp, unsigned int order) return __folio_alloc_node_noprof(gfp, order, numa_node_id()); } static inline struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned in= t order, - struct mempolicy *mpol, pgoff_t ilx, int nid) + struct mempolicy *mpol, pgoff_t ilx, int nid, + unsigned long user_addr) { - return folio_alloc_noprof(gfp, order); + return __folio_alloc_noprof(gfp, order, numa_node_id(), NULL, user_addr); } static inline struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, struct vm_area_struct *vma, unsigned long addr) { - return folio_alloc_noprof(gfp, order); + return folio_alloc_mpol_noprof(gfp, order, NULL, 0, numa_node_id(), + addr); } #endif =20 diff --git a/mm/compaction.c b/mm/compaction.c index 1e8f8eca318c..82f2914962f5 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -82,8 +82,7 @@ static inline bool is_via_compact_memory(int order) { ret= urn false; } =20 static struct page *mark_allocated_noprof(struct page *page, unsigned int = order, gfp_t gfp_flags) { - post_alloc_hook(page, order, __GFP_MOVABLE); - set_page_refcounted(page); + post_alloc_hook(page, order, __GFP_MOVABLE, USER_ADDR_NONE) return page; } #define mark_allocated(...) alloc_hooks(mark_allocated_noprof(__VA_ARGS__)) @@ -1832,8 +1831,7 @@ static struct folio *compaction_alloc_noprof(struct f= olio *src, unsigned long da set_page_private(&freepage[size], start_order); } dst =3D (struct folio *)freepage; - - post_alloc_hook(&dst->page, order, __GFP_MOVABLE); + post_alloc_hook(&dst->page, order, __GFP_MOVABLE, USER_ADDR_NONE); set_page_refcounted(&dst->page); if (order) prep_compound_page(&dst->page, order); diff --git a/mm/filemap.c b/mm/filemap.c index 6cd7974d4ada..bfc6554b993d 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -998,7 +998,8 @@ struct folio *filemap_alloc_folio_noprof(gfp_t gfp, uns= igned int order, =20 if (policy) return folio_alloc_mpol_noprof(gfp, order, policy, - NO_INTERLEAVE_INDEX, numa_node_id()); + NO_INTERLEAVE_INDEX, numa_node_id(), + USER_ADDR_NONE); =20 if (cpuset_do_page_mem_spread()) { unsigned int cpuset_mems_cookie; diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 0beb6e22bc26..de8361b503d2 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1842,7 +1842,8 @@ struct address_space *hugetlb_folio_mapping_lock_writ= e(struct folio *folio) } =20 static struct folio *alloc_buddy_frozen_folio(int order, gfp_t gfp_mask, - int nid, nodemask_t *nmask, nodemask_t *node_alloc_noretry) + int nid, nodemask_t *nmask, nodemask_t *node_alloc_noretry, + unsigned long addr) { struct folio *folio; bool alloc_try_hard =3D true; @@ -1859,7 +1860,7 @@ static struct folio *alloc_buddy_frozen_folio(int ord= er, gfp_t gfp_mask, if (alloc_try_hard) gfp_mask |=3D __GFP_RETRY_MAYFAIL; =20 - folio =3D (struct folio *)__alloc_frozen_pages(gfp_mask, order, nid, nmas= k); + folio =3D (struct folio *)__alloc_frozen_pages(gfp_mask, order, nid, nmas= k, addr); =20 /* * If we did not specify __GFP_RETRY_MAYFAIL, but still got a @@ -1888,7 +1889,7 @@ static struct folio *alloc_buddy_frozen_folio(int ord= er, gfp_t gfp_mask, =20 static struct folio *only_alloc_fresh_hugetlb_folio(struct hstate *h, gfp_t gfp_mask, int nid, nodemask_t *nmask, - nodemask_t *node_alloc_noretry) + nodemask_t *node_alloc_noretry, unsigned long addr) { struct folio *folio; int order =3D huge_page_order(h); @@ -1900,7 +1901,7 @@ static struct folio *only_alloc_fresh_hugetlb_folio(s= truct hstate *h, folio =3D alloc_gigantic_frozen_folio(order, gfp_mask, nid, nmask); else folio =3D alloc_buddy_frozen_folio(order, gfp_mask, nid, nmask, - node_alloc_noretry); + node_alloc_noretry, addr); if (folio) init_new_hugetlb_folio(folio); return folio; @@ -1914,11 +1915,12 @@ static struct folio *only_alloc_fresh_hugetlb_folio= (struct hstate *h, * pages is zero, and the accounting must be done in the caller. */ static struct folio *alloc_fresh_hugetlb_folio(struct hstate *h, - gfp_t gfp_mask, int nid, nodemask_t *nmask) + gfp_t gfp_mask, int nid, nodemask_t *nmask, + unsigned long addr) { struct folio *folio; =20 - folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, NULL); + folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, NULL, a= ddr); if (folio) hugetlb_vmemmap_optimize_folio(h, folio); return folio; @@ -1958,7 +1960,7 @@ static struct folio *alloc_pool_huge_folio(struct hst= ate *h, struct folio *folio; =20 folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, node, - nodes_allowed, node_alloc_noretry); + nodes_allowed, node_alloc_noretry, USER_ADDR_NONE); if (folio) return folio; } @@ -2127,7 +2129,8 @@ int dissolve_free_hugetlb_folios(unsigned long start_= pfn, unsigned long end_pfn) * Allocates a fresh surplus page from the page allocator. */ static struct folio *alloc_surplus_hugetlb_folio(struct hstate *h, - gfp_t gfp_mask, int nid, nodemask_t *nmask) + gfp_t gfp_mask, int nid, nodemask_t *nmask, + unsigned long addr) { struct folio *folio =3D NULL; =20 @@ -2139,7 +2142,7 @@ static struct folio *alloc_surplus_hugetlb_folio(stru= ct hstate *h, goto out_unlock; spin_unlock_irq(&hugetlb_lock); =20 - folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask); + folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, addr); if (!folio) return NULL; =20 @@ -2182,7 +2185,7 @@ static struct folio *alloc_migrate_hugetlb_folio(stru= ct hstate *h, gfp_t gfp_mas if (hstate_is_gigantic(h)) return NULL; =20 - folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask); + folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, USER_ADDR_NO= NE); if (!folio) return NULL; =20 @@ -2218,14 +2221,14 @@ struct folio *alloc_buddy_hugetlb_folio_with_mpol(s= truct hstate *h, if (mpol_is_preferred_many(mpol)) { gfp_t gfp =3D gfp_mask & ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL); =20 - folio =3D alloc_surplus_hugetlb_folio(h, gfp, nid, nodemask); + folio =3D alloc_surplus_hugetlb_folio(h, gfp, nid, nodemask, addr); =20 /* Fallback to all nodes if page=3D=3DNULL */ nodemask =3D NULL; } =20 if (!folio) - folio =3D alloc_surplus_hugetlb_folio(h, gfp_mask, nid, nodemask); + folio =3D alloc_surplus_hugetlb_folio(h, gfp_mask, nid, nodemask, addr); mpol_cond_put(mpol); return folio; } @@ -2332,7 +2335,8 @@ static int gather_surplus_pages(struct hstate *h, lon= g delta) * down the road to pick the current node if that is the case. */ folio =3D alloc_surplus_hugetlb_folio(h, htlb_alloc_mask(h), - NUMA_NO_NODE, &alloc_nodemask); + NUMA_NO_NODE, &alloc_nodemask, + USER_ADDR_NONE); if (!folio) { alloc_ok =3D false; break; @@ -2738,7 +2742,7 @@ static int alloc_and_dissolve_hugetlb_folio(struct fo= lio *old_folio, spin_unlock_irq(&hugetlb_lock); gfp_mask =3D htlb_alloc_mask(h) | __GFP_THISNODE; new_folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, - nid, NULL); + nid, NULL, USER_ADDR_NONE); if (!new_folio) return -ENOMEM; goto retry; @@ -3434,13 +3438,13 @@ static void __init hugetlb_hstate_alloc_pages_oneno= de(struct hstate *h, int nid) gfp_t gfp_mask =3D htlb_alloc_mask(h) | __GFP_THISNODE; =20 folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, - &node_states[N_MEMORY], NULL); + &node_states[N_MEMORY], NULL, USER_ADDR_NONE); if (!folio && !list_empty(&folio_list) && hugetlb_vmemmap_optimizable_size(h)) { prep_and_add_allocated_folios(h, &folio_list); INIT_LIST_HEAD(&folio_list); folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, - &node_states[N_MEMORY], NULL); + &node_states[N_MEMORY], NULL, USER_ADDR_NONE); } if (!folio) break; diff --git a/mm/internal.h b/mm/internal.h index cb0af847d7d9..0b9c0bd133d3 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -672,6 +672,7 @@ struct alloc_context { */ enum zone_type highest_zoneidx; bool spread_dirty_pages; + unsigned long user_addr; }; =20 /* @@ -887,16 +888,18 @@ static inline void prep_compound_tail(struct page *he= ad, int tail_idx) set_page_private(p, 0); } =20 -void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flag= s); +void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flag= s, + unsigned long user_addr); extern bool free_pages_prepare(struct page *page, unsigned int order); =20 extern int user_min_free_kbytes; =20 struct page *__alloc_frozen_pages_noprof(gfp_t, unsigned int order, int ni= d, - nodemask_t *); + nodemask_t *, unsigned long user_addr); #define __alloc_frozen_pages(...) \ alloc_hooks(__alloc_frozen_pages_noprof(__VA_ARGS__)) void free_frozen_pages(struct page *page, unsigned int order); +void free_frozen_pages_zeroed(struct page *page, unsigned int order); void free_unref_folios(struct folio_batch *fbatch); =20 #ifdef CONFIG_NUMA @@ -904,7 +907,7 @@ struct page *alloc_frozen_pages_noprof(gfp_t, unsigned = int order); #else static inline struct page *alloc_frozen_pages_noprof(gfp_t gfp, unsigned i= nt order) { - return __alloc_frozen_pages_noprof(gfp, order, numa_node_id(), NULL); + return __alloc_frozen_pages_noprof(gfp, order, numa_node_id(), NULL, USER= _ADDR_NONE); } #endif =20 diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 1dd3cfca610d..f7e0f37f0632 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -1055,7 +1055,7 @@ static enum scan_result alloc_charge_folio(struct fol= io **foliop, struct mm_stru int node =3D hpage_collapse_find_target_node(cc); struct folio *folio; =20 - folio =3D __folio_alloc(gfp, HPAGE_PMD_ORDER, node, &cc->alloc_nmask); + folio =3D __folio_alloc(gfp, HPAGE_PMD_ORDER, node, &cc->alloc_nmask, USE= R_ADDR_NONE); if (!folio) { *foliop =3D NULL; count_vm_event(THP_COLLAPSE_ALLOC_FAILED); diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 0e5175f1c767..ca2f430a7ffd 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -1454,7 +1454,7 @@ static struct folio *alloc_migration_target_by_mpol(s= truct folio *src, else gfp =3D GFP_HIGHUSER_MOVABLE | __GFP_RETRY_MAYFAIL | __GFP_COMP; =20 - return folio_alloc_mpol(gfp, order, pol, ilx, nid); + return folio_alloc_mpol(gfp, order, pol, ilx, nid, USER_ADDR_NONE); } #else =20 @@ -2406,7 +2406,8 @@ bool mempolicy_in_oom_domain(struct task_struct *tsk, } =20 static struct page *alloc_pages_preferred_many(gfp_t gfp, unsigned int ord= er, - int nid, nodemask_t *nodemask) + int nid, nodemask_t *nodemask, + unsigned long user_addr) { struct page *page; gfp_t preferred_gfp; @@ -2419,9 +2420,11 @@ static struct page *alloc_pages_preferred_many(gfp_t= gfp, unsigned int order, */ preferred_gfp =3D gfp | __GFP_NOWARN; preferred_gfp &=3D ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL); - page =3D __alloc_frozen_pages_noprof(preferred_gfp, order, nid, nodemask); + page =3D __alloc_frozen_pages_noprof(preferred_gfp, order, nid, + nodemask, user_addr); if (!page) - page =3D __alloc_frozen_pages_noprof(gfp, order, nid, NULL); + page =3D __alloc_frozen_pages_noprof(gfp, order, nid, NULL, + user_addr); =20 return page; } @@ -2436,8 +2439,9 @@ static struct page *alloc_pages_preferred_many(gfp_t = gfp, unsigned int order, * * Return: The page on success or NULL if allocation fails. */ -static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order, - struct mempolicy *pol, pgoff_t ilx, int nid) +static struct page *__alloc_pages_mpol(gfp_t gfp, unsigned int order, + struct mempolicy *pol, pgoff_t ilx, int nid, + unsigned long user_addr) { nodemask_t *nodemask; struct page *page; @@ -2445,7 +2449,8 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsig= ned int order, nodemask =3D policy_nodemask(gfp, pol, ilx, &nid); =20 if (pol->mode =3D=3D MPOL_PREFERRED_MANY) - return alloc_pages_preferred_many(gfp, order, nid, nodemask); + return alloc_pages_preferred_many(gfp, order, nid, nodemask, + user_addr); =20 if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && /* filter "hugepage" allocation, unless from alloc_pages() */ @@ -2469,7 +2474,7 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsig= ned int order, */ page =3D __alloc_frozen_pages_noprof( gfp | __GFP_THISNODE | __GFP_NORETRY, order, - nid, NULL); + nid, NULL, user_addr); if (page || !(gfp & __GFP_DIRECT_RECLAIM)) return page; /* @@ -2481,7 +2486,7 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsig= ned int order, } } =20 - page =3D __alloc_frozen_pages_noprof(gfp, order, nid, nodemask); + page =3D __alloc_frozen_pages_noprof(gfp, order, nid, nodemask, user_addr= ); =20 if (unlikely(pol->mode =3D=3D MPOL_INTERLEAVE || pol->mode =3D=3D MPOL_WEIGHTED_INTERLEAVE) && page) { @@ -2497,17 +2502,25 @@ static struct page *alloc_pages_mpol(gfp_t gfp, uns= igned int order, return page; } =20 -struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order, +static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order, struct mempolicy *pol, pgoff_t ilx, int nid) { - struct page *page =3D alloc_pages_mpol(gfp | __GFP_COMP, order, pol, - ilx, nid); + return __alloc_pages_mpol(gfp, order, pol, ilx, nid, USER_ADDR_NONE); +} + +struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order, + struct mempolicy *pol, pgoff_t ilx, int nid, + unsigned long user_addr) +{ + struct page *page =3D __alloc_pages_mpol(gfp | __GFP_COMP, order, pol, + ilx, nid, user_addr); if (!page) return NULL; =20 set_page_refcounted(page); return page_rmappable_folio(page); } +EXPORT_SYMBOL(folio_alloc_mpol_noprof); =20 /** * vma_alloc_folio - Allocate a folio for a VMA. @@ -2535,7 +2548,7 @@ struct folio *vma_alloc_folio_noprof(gfp_t gfp, int o= rder, struct vm_area_struct gfp |=3D __GFP_NOWARN; =20 pol =3D get_vma_policy(vma, addr, order, &ilx); - folio =3D folio_alloc_mpol_noprof(gfp, order, pol, ilx, numa_node_id()); + folio =3D folio_alloc_mpol_noprof(gfp, order, pol, ilx, numa_node_id(), a= ddr); mpol_cond_put(pol); return folio; } diff --git a/mm/migrate.c b/mm/migrate.c index 1bf2cf8c44dd..df805a763991 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -2202,7 +2202,7 @@ struct folio *alloc_migration_target(struct folio *sr= c, unsigned long private) if (is_highmem_idx(zidx) || zidx =3D=3D ZONE_MOVABLE) gfp_mask |=3D __GFP_HIGHMEM; =20 - return __folio_alloc(gfp_mask, order, nid, mtc->nmask); + return __folio_alloc(gfp_mask, order, nid, mtc->nmask, USER_ADDR_NONE); } =20 #ifdef CONFIG_NUMA diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 2d4b6f1a554e..1cf5551849fe 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1837,7 +1837,7 @@ static inline bool should_skip_init(gfp_t flags) } =20 inline void post_alloc_hook(struct page *page, unsigned int order, - gfp_t gfp_flags) + gfp_t gfp_flags, unsigned long user_addr) { bool init =3D !want_init_on_free() && want_init_on_alloc(gfp_flags) && !should_skip_init(gfp_flags); @@ -1892,9 +1892,10 @@ inline void post_alloc_hook(struct page *page, unsig= ned int order, } =20 static void prep_new_page(struct page *page, unsigned int order, gfp_t gfp= _flags, - unsigned int alloc_flags) + unsigned int alloc_flags, + unsigned long user_addr) { - post_alloc_hook(page, order, gfp_flags); + post_alloc_hook(page, order, gfp_flags, user_addr); =20 if (order && (gfp_flags & __GFP_COMP)) prep_compound_page(page, order); @@ -3959,7 +3960,8 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int o= rder, int alloc_flags, page =3D rmqueue(zonelist_zone(ac->preferred_zoneref), zone, order, gfp_mask, alloc_flags, ac->migratetype); if (page) { - prep_new_page(page, order, gfp_mask, alloc_flags); + prep_new_page(page, order, gfp_mask, alloc_flags, + ac->user_addr); =20 /* * If this is a high-order atomic allocation then check @@ -4194,7 +4196,8 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned= int order, =20 /* Prep a captured page if available */ if (page) - prep_new_page(page, order, gfp_mask, alloc_flags); + prep_new_page(page, order, gfp_mask, alloc_flags, + ac->user_addr); =20 /* Try get a page from the freelist if available */ if (!page) @@ -5187,7 +5190,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, } nr_account++; =20 - prep_new_page(page, 0, gfp, 0); + prep_new_page(page, 0, gfp, 0, USER_ADDR_NONE); set_page_refcounted(page); page_array[nr_populated++] =3D page; } @@ -5201,7 +5204,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, return nr_populated; =20 failed: - page =3D __alloc_pages_noprof(gfp, 0, preferred_nid, nodemask); + page =3D __alloc_pages_noprof(gfp, 0, preferred_nid, nodemask, USER_ADDR_= NONE); if (page) page_array[nr_populated++] =3D page; goto out; @@ -5212,12 +5215,13 @@ EXPORT_SYMBOL_GPL(alloc_pages_bulk_noprof); * This is the 'heart' of the zoned buddy allocator. */ struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order, - int preferred_nid, nodemask_t *nodemask) + int preferred_nid, nodemask_t *nodemask, + unsigned long user_addr) { struct page *page; unsigned int alloc_flags =3D ALLOC_WMARK_LOW; gfp_t alloc_gfp; /* The gfp_t that was actually used for allocation */ - struct alloc_context ac =3D { }; + struct alloc_context ac =3D { .user_addr =3D user_addr }; =20 /* * There are several places where we assume that the order value is sane @@ -5277,11 +5281,13 @@ struct page *__alloc_frozen_pages_noprof(gfp_t gfp,= unsigned int order, EXPORT_SYMBOL(__alloc_frozen_pages_noprof); =20 struct page *__alloc_pages_noprof(gfp_t gfp, unsigned int order, - int preferred_nid, nodemask_t *nodemask) + int preferred_nid, nodemask_t *nodemask, + unsigned long user_addr) { struct page *page; =20 - page =3D __alloc_frozen_pages_noprof(gfp, order, preferred_nid, nodemask); + page =3D __alloc_frozen_pages_noprof(gfp, order, preferred_nid, + nodemask, user_addr); if (page) set_page_refcounted(page); return page; @@ -5289,10 +5295,10 @@ struct page *__alloc_pages_noprof(gfp_t gfp, unsign= ed int order, EXPORT_SYMBOL(__alloc_pages_noprof); =20 struct folio *__folio_alloc_noprof(gfp_t gfp, unsigned int order, int pref= erred_nid, - nodemask_t *nodemask) + nodemask_t *nodemask, unsigned long user_addr) { struct page *page =3D __alloc_pages_noprof(gfp | __GFP_COMP, order, - preferred_nid, nodemask); + preferred_nid, nodemask, user_addr); return page_rmappable_folio(page); } EXPORT_SYMBOL(__folio_alloc_noprof); @@ -6910,7 +6916,7 @@ static void split_free_frozen_pages(struct list_head = *list, gfp_t gfp_mask) list_for_each_entry_safe(page, next, &list[order], lru) { int i; =20 - post_alloc_hook(page, order, gfp_mask); + post_alloc_hook(page, order, gfp_mask, USER_ADDR_NONE); if (!order) continue; =20 @@ -7116,7 +7122,7 @@ int alloc_contig_frozen_range_noprof(unsigned long st= art, unsigned long end, struct page *head =3D pfn_to_page(start); =20 check_new_pages(head, order); - prep_new_page(head, order, gfp_mask, 0); + prep_new_page(head, order, gfp_mask, 0, USER_ADDR_NONE); } else { ret =3D -EINVAL; WARN(true, "PFN range: requested [%lu, %lu), allocated [%lu, %lu)\n", @@ -7781,7 +7787,7 @@ struct page *alloc_frozen_pages_nolock_noprof(gfp_t g= fp_flags, int nid, unsigned gfp_t alloc_gfp =3D __GFP_NOWARN | __GFP_ZERO | __GFP_NOMEMALLOC | __GFP_= COMP | gfp_flags; unsigned int alloc_flags =3D ALLOC_TRYLOCK; - struct alloc_context ac =3D { }; + struct alloc_context ac =3D { .user_addr =3D USER_ADDR_NONE }; struct page *page; =20 VM_WARN_ON_ONCE(gfp_flags & ~__GFP_ACCOUNT); diff --git a/mm/page_frag_cache.c b/mm/page_frag_cache.c index d2423f30577e..bcd3d1aa8589 100644 --- a/mm/page_frag_cache.c +++ b/mm/page_frag_cache.c @@ -57,10 +57,10 @@ static struct page *__page_frag_cache_refill(struct pag= e_frag_cache *nc, gfp_mask =3D (gfp_mask & ~__GFP_DIRECT_RECLAIM) | __GFP_COMP | __GFP_NOWARN | __GFP_NORETRY | __GFP_NOMEMALLOC; page =3D __alloc_pages(gfp_mask, PAGE_FRAG_CACHE_MAX_ORDER, - numa_mem_id(), NULL); + numa_mem_id(), NULL, USER_ADDR_NONE); #endif if (unlikely(!page)) { - page =3D __alloc_pages(gfp, 0, numa_mem_id(), NULL); + page =3D __alloc_pages(gfp, 0, numa_mem_id(), NULL, USER_ADDR_NONE); order =3D 0; } =20 diff --git a/mm/shmem.c b/mm/shmem.c index b40f3cd48961..896cef466b0c 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -1927,7 +1927,7 @@ static struct folio *shmem_alloc_folio(gfp_t gfp, int= order, struct folio *folio; =20 mpol =3D shmem_get_pgoff_policy(info, index, order, &ilx); - folio =3D folio_alloc_mpol(gfp, order, mpol, ilx, numa_node_id()); + folio =3D folio_alloc_mpol(gfp, order, mpol, ilx, numa_node_id(), USER_AD= DR_NONE); mpol_cond_put(mpol); =20 return folio; diff --git a/mm/slub.c b/mm/slub.c index 0c906fefc31b..fc8f998a0fe1 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -3266,7 +3266,7 @@ static inline struct slab *alloc_slab_page(gfp_t flag= s, int node, else if (node =3D=3D NUMA_NO_NODE) page =3D alloc_frozen_pages(flags, order); else - page =3D __alloc_frozen_pages(flags, order, node, NULL); + page =3D __alloc_frozen_pages(flags, order, node, NULL, USER_ADDR_NONE); =20 if (!page) return NULL; @@ -5178,7 +5178,7 @@ static void *___kmalloc_large_node(size_t size, gfp_t= flags, int node) if (node =3D=3D NUMA_NO_NODE) page =3D alloc_frozen_pages_noprof(flags, order); else - page =3D __alloc_frozen_pages_noprof(flags, order, node, NULL); + page =3D __alloc_frozen_pages_noprof(flags, order, node, NULL, USER_ADDR= _NONE); =20 if (page) { ptr =3D page_address(page); diff --git a/mm/swap_state.c b/mm/swap_state.c index 6d0eef7470be..12ac29ae818c 100644 --- a/mm/swap_state.c +++ b/mm/swap_state.c @@ -568,7 +568,7 @@ struct folio *swap_cache_alloc_folio(swp_entry_t entry,= gfp_t gfp_mask, return NULL; =20 /* Allocate a new folio to be added into the swap cache. */ - folio =3D folio_alloc_mpol(gfp_mask, 0, mpol, ilx, numa_node_id()); + folio =3D folio_alloc_mpol(gfp_mask, 0, mpol, ilx, numa_node_id(), USER_A= DDR_NONE); if (!folio) return NULL; /* Try add the new folio, returns existing folio or NULL on failure. */ --=20 MST From nobody Wed Jun 17 02:49:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2FF092C1788 for ; Tue, 21 Apr 2026 22:01:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808885; cv=none; b=cnocy8xK37+DYOUbHiuAwJlzL0j4MYS+OKX9K4HSXOZ2Io98T4z3ZrUUhUnGe7X68NWnHhkXx9zGW9Xb9nTzZfRTHwZPLp5hKejm3If40Iia3D1lJmBm7EmMD0NPKH31+GxK500HUacvro3wdYBkNPu8O/wFBa97r1W8fZl/Yfc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808885; c=relaxed/simple; bh=+zNBlTlQkFqQMbEC4hj3ywqszA8Y1ImDjehUW9ILJAs=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Uow/YLb4/0uc7xC1W63u39jZupmOQ1+vZCBokH5kRiqPPON2PSRmTpCpbhwO2vAmcmHcUmMgdSfBHyYgfa0HYL2ft9xTSNd5PJ3SoFvrFfc8c61a6MkfsC7CQmZLu3AsC6J234emkHuKPXu53phRT4YNeovHjpPDPQeAXp0Wjg8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=BnEaIBwH; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=oolJTOcl; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="BnEaIBwH"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="oolJTOcl" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1776808882; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Dn5xupL5V4qF15edJs01tgV6QOrsZ1JWzK8CN9u8yrs=; b=BnEaIBwHg0DmQBiMMo7rNi07jDmfjgA0gwy4Ztu2XQpKt5LfGFeZnDiQ+9bOTr4NLgMaAm ydxmDAjmlkdMPW7HH2o4ubcy8WYHQE5IeydINVsH19Uh6pAggBSUEzbNtJWK5NMe3dr927 eMaBj81dtRReCTRwpYxxbQ0fXQh0p4I= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-210-jO76yfbVOsmwn9Bs662dzg-1; Tue, 21 Apr 2026 18:01:21 -0400 X-MC-Unique: jO76yfbVOsmwn9Bs662dzg-1 X-Mimecast-MFC-AGG-ID: jO76yfbVOsmwn9Bs662dzg_1776808880 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-4837bfcfe0dso53362715e9.1 for ; Tue, 21 Apr 2026 15:01:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1776808880; x=1777413680; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Dn5xupL5V4qF15edJs01tgV6QOrsZ1JWzK8CN9u8yrs=; b=oolJTOclmbBVkwYsj78uD4rVrZE3yJmNTtoN/bZA/2N3ee00y3go8RwJET3vxFo4ok LCBuF66E+0XKA38quz2vxtWvYhdcbcHh7QqIPDUG9fps9o08tEchuO9LCwPUzOFoWrwX XqsyWGHdZq9hCTm81rucu4WwSWj3z3JEWONNW/V5I5sNOPfezj7wTSFZesa6Sl6SIxGH 3jOzUS5nQonPQxbTx667vcbrJWxfzI7HBzJ+A7RRDIPOs2m10iA3XR9AbgFff/KYU8n2 tHeWrHHUcDZ9RvYp3qMsoeWxUkfmqe8LYwH11YIh+wFiyAS/ndPGSVS3uRDBcGl+fcyM NESQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776808880; x=1777413680; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Dn5xupL5V4qF15edJs01tgV6QOrsZ1JWzK8CN9u8yrs=; b=dRkFpyrjpGiDMgJ+XTp6x1AOvK3QI7Q1NMFcdof3q0P549x/CsmFQGPkjloyrSxZ+C 1l/E3qSFWU5CDL0E6g6suXrLjFOpGE1FuYItnP+QjnZGVDIAZHt8u9A/tQwngP/wbFHG MTQudMchnu9C3Jft/NFE9Y0qWf2STuJHljGKcIjPHgo4eS16T68MIZtCPLLSRUITyU+V rI1tBDV/dQxg7MxUzTEpCF58Swd+5RijtJBXSZj6VFVWgsYTd0n1R7Ig8IXDP9+9+3iV ULTva6gB5O9DFJrY/5HMXow8rikDBpq0WBObyYc46fzYEsZBpA2TChPQeQIj/XiCdImq mehg== X-Gm-Message-State: AOJu0YxtW3SG6CG0djIXA39LM9ZeO8caC/5123RINmDqvMivpN7G5LB7 RS/POmwbprpQ/P6NbIBQMpBF1Wqev+rHi5vbsJbrbJDjYNrvPiZ5eyp98PJI0pa/NFHScA5vdKB n378X2IQi0h/lK689RiO5hzwrvh+EAJ54WZPLlcOryeyAl7ihRuZ3VDbdJmo6OHNJyVMR5gvhqP Oe8pubdz3q9lGCZ4fuFIwB2+JjNVIKLNbXB8H4foHE1is= X-Gm-Gg: AeBDievRnCHyPmYUUu/xbCuoMkMjF1fO1y1ra5OuIvrmE/JqZrftyD2WZV7pZwbSvqh QfGmQ86VwAMwQy0zNXpWf0H7Za7kUBpIfzeIsXhgtg+v0PmWrJ46WlksbMsjdQb71bzc/VQeLc6 qI93o3jVJx5c7ksJ4lp3orhF76fdJlxvuuKPFZVUJp0j/YYaT6b+4q+9OCyGIsjJbnAPVLJJ+Qu LdnaiGnOozWCmJ+xNkh6qBAoPFB3YNE786Rqep6radw4AJHOy1ENh91reiY0pQPb1EhvKPAZCIY WJ2foVTz2swA8yNtSppTy224x7vM67i8UPOm77xoXsHK7mCWFIQvXeVsuRb5uRuqvDeAi+w7Mzi PHVL7SCrvQMYY5t1ZTBXdcncaNV8qEPohaaqMOoZN1tBafV6ajVLg0A== X-Received: by 2002:a05:600c:1e09:b0:48a:592c:e655 with SMTP id 5b1f17b1804b1-48a592ce7c2mr22291675e9.17.1776808879659; Tue, 21 Apr 2026 15:01:19 -0700 (PDT) X-Received: by 2002:a05:600c:1e09:b0:48a:592c:e655 with SMTP id 5b1f17b1804b1-48a592ce7c2mr22291235e9.17.1776808879144; Tue, 21 Apr 2026 15:01:19 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-21.inter.net.il. [80.230.25.21]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4891df9e50asm153245945e9.0.2026.04.21.15.01.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Apr 2026 15:01:18 -0700 (PDT) Date: Tue, 21 Apr 2026 18:01:16 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Lorenzo Stoakes , "Liam R. Howlett" , Mike Rapoport Subject: [PATCH RFC v3 02/19] mm: add folio_zero_user stub for configs without THP/HUGETLBFS Message-ID: <2c8e630afa8b68232d648c08eec4b2fc7d933dc1.1776808210.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" folio_zero_user() is defined in mm/memory.c under CONFIG_TRANSPARENT_HUGEPAGE || CONFIG_HUGETLBFS. A subsequent patch will call it from post_alloc_hook() for all user page zeroing, so configs without THP or HUGETLBFS will need a stub. Add a macro in the #else branch that falls back to clear_user_highpages(), which handles cache aliasing correctly on VIPT architectures and is always available via highmem.h. Without THP/HUGETLBFS, only order-0 user pages are allocated, so the locality optimization in the real folio_zero_user() (zero near the faulting address last) is not needed. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- include/linux/mm.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 5be3d8a8f806..541d36e5e420 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -4718,6 +4718,9 @@ static inline bool vma_is_special_huge(const struct v= m_area_struct *vma) (vma->vm_flags & (VM_PFNMAP | VM_MIXEDMAP))); } =20 +#else /* !CONFIG_TRANSPARENT_HUGEPAGE && !CONFIG_HUGETLBFS */ +#define folio_zero_user(folio, addr_hint) \ + clear_user_highpages(&(folio)->page, (addr_hint), folio_nr_pages(folio)) #endif /* CONFIG_TRANSPARENT_HUGEPAGE || CONFIG_HUGETLBFS */ =20 #if MAX_NUMNODES > 1 --=20 MST From nobody Wed Jun 17 02:49:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E7A2934F479 for ; Tue, 21 Apr 2026 22:01:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808888; cv=none; b=BiJJHSezdRbMH0lcF9bqeNL0urCB6QsxRbO9dS3bj+m9pm82oazU/kiH8QI8cuNQ7GJMtG5g/TfiKo/MEz3F1AtnNHGJkOGF1exw7VjvTOwa6SBLerpHY0dPAy1lz5gXNBUEy4reHBxNMaGW3Bs7pkC2ns3KJ7xttqqhfR7rYTQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808888; c=relaxed/simple; bh=wKXQYY6FfDI9a/16Lq7irXZnvvpx8cOvs36gzMXBZhQ=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=TV4Y3kV2QmVAICNDN5Qkexsb6IvN8cct9CneYeh+thcRpQ34K7zjFiKisdH7hLoY6gwqdElYcC5U3q+BWf1RHlin35VcJydZ7IiH8eDDkIFEADl7KGi7YE+b4sgwcmFeRb/7i8unKmzWRo0KGjuijmb6MZR9ZT8hrLguq8t/gH8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=YGSQSO22; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=o9UuzKDZ; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="YGSQSO22"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="o9UuzKDZ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1776808886; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=/Et+lyeWTt5FeJ2gsFu/8TUEAo6EEauFnoYjswzfOPc=; b=YGSQSO22MLh+CKJRfi6q6Tnez94aPOA0kXscVNJmIsBveiShjK2qne4mbrcguqtmsC0zfx TuLXNUYbSsJZQxr6fGPOZ2R1IufuLyH0CG3fb1lgyWxSRnnneat7hLr1lpCr0AR4YEwODA rW57q9YP1nWgdv7MMvL3c9x50sRjBCM= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-231-UIHvN-n8O4KdldBpeeYKzA-1; Tue, 21 Apr 2026 18:01:24 -0400 X-MC-Unique: UIHvN-n8O4KdldBpeeYKzA-1 X-Mimecast-MFC-AGG-ID: UIHvN-n8O4KdldBpeeYKzA_1776808883 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-488d56f87e8so41965065e9.0 for ; Tue, 21 Apr 2026 15:01:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1776808883; x=1777413683; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=/Et+lyeWTt5FeJ2gsFu/8TUEAo6EEauFnoYjswzfOPc=; b=o9UuzKDZ0lnyjfKbeVgIaPXSLFtjF2FAGNZbH4dEykiI5H7NfKEm8d4IbHuhxX15hA e94HUKBVhS+5rDNXI9PEsHO78f1y16vlxbLn72AcZOjOo9He3XerVz3YllFA0dd3d5UG 0DrKrkcXXHO0DsZDFrc1ANDJuXTTaTUMimyB7m1B7PosXLkbmbS+Px/WLC5EorRnEg2K S9Lce/UqLmjXU2My/i7sSJxqyQPTbSK0XigRBHuD8eYiXjmXc1uz1HbUoF/sp8XbcJhs OEFy3KeBODfxJSE/HeSNiRMioCNvmZg2vb/JkQZNkpSfdIOv6uoUtrVcnVVrlULqBmjI aZTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776808883; x=1777413683; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/Et+lyeWTt5FeJ2gsFu/8TUEAo6EEauFnoYjswzfOPc=; b=T6WHs0UnfnGbCrhKyLlnB3Xwfg/zMROiL+uDSC12vFVKY5izUfUA/J/6bvN7fLe4et BtnRCaa3znxoHZUTIb/cFTTAhM/wZY9fvEmiyS1lK2Mjtlt46DPUAUfuF1l8DdgJgKCD wv+/DTBgQqPVCgvT2j4u+vxixJB4uS6Dk7vUwdidl7ygR7zQynBxe5lKV+K7wGl5fnX1 +fKEcD6jSlZOJtLbuQK1MLKDQFa9+ftQ21UJoBS1UnPcQ0SFpmYdt9E756roo5WtmJzi ZCK1iKWNPAzr9mMc7mW9UpDqtRb23AoE8ok8FUGXKlUSmRwYJ3IgJAzWu529wVbzrDbQ vMTw== X-Gm-Message-State: AOJu0Yw39IocgPYesHQBrb5QgTMls9tzlygNcCcnHhOds+i83T6qji+8 +O2OmDpD2hNXdXvp0jhqkNoUuQkjxAglqF69wePftlSu5wFgQJxpJpuEY3707CnDnu2CZhoIy/V C6LLAxR26OZIX6i5VUabAZUW2/HMI8RE79MULDxBy82160Tfut/PbtQAhSy0a6DglPKdB7jr53R fhZOaZdh2rz9EzVksPbvQF7o5qEeLc39tDJZ5j18Iw//4= X-Gm-Gg: AeBDievXop/CjdQEvQ5rzDE82M9v9HSlx8HGZAHjcnmA0WG7Sqjr2rsWZd2pqI9T7lv MnV0a6NsfLP+w9w7Kn+1CM/H3zqjDapUhyLYIH9Xe2K03SUfeOiHFZWipheCz9PP9Z/znp8ROh9 1tb762GoH9OGKWwj27ToWEDOjVwPpwWSmRmCupyRztJGbhF9oj87MCDlYoZHJdKitkkiBlDrFbi CSj/wleXHk36X9RXtq4+nAG4cmNnduzbl6o48EVlKPSTjyDVikTAHMlCGPqOSHsH02o//IEGcxM +Z+jR3LxCyOlSC0JPG3xQefoEmJKas8FgQizFhDyLDDC9iAM+95TbB3q965wE0MtAyagixpxZ4Z jWNk4Gbtbf+olcuJcGQoUoaMcROFSmIIPYBgNlRpVME/jjWvu8qJemA== X-Received: by 2002:a05:600c:468c:b0:48a:56d5:16f2 with SMTP id 5b1f17b1804b1-48a56d5179amr56679045e9.7.1776808883048; Tue, 21 Apr 2026 15:01:23 -0700 (PDT) X-Received: by 2002:a05:600c:468c:b0:48a:56d5:16f2 with SMTP id 5b1f17b1804b1-48a56d5179amr56678425e9.7.1776808882298; Tue, 21 Apr 2026 15:01:22 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-21.inter.net.il. [80.230.25.21]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48a5549f582sm28902765e9.33.2026.04.21.15.01.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Apr 2026 15:01:21 -0700 (PDT) Date: Tue, 21 Apr 2026 18:01:19 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Johannes Weiner , Zi Yan Subject: [PATCH RFC v3 03/19] mm: page_alloc: move prep_compound_page before post_alloc_hook Message-ID: <6e574eb533603db74fd03244399df760a174bdea.1776808210.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move prep_compound_page() before post_alloc_hook() in prep_new_page(). The next patch adds a folio_zero_user() call to post_alloc_hook(), which uses folio_nr_pages() to determine how many pages to zero. Without compound metadata set up first, folio_nr_pages() returns 1 for higher-order allocations, so only the first page would be zeroed. All other operations in post_alloc_hook() (arch_alloc_page, KASAN, debug, page owner, etc.) use raw page pointers with explicit order counts and are unaffected by this reordering. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/page_alloc.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 1cf5551849fe..99c01eb2d59e 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1895,11 +1895,11 @@ static void prep_new_page(struct page *page, unsign= ed int order, gfp_t gfp_flags unsigned int alloc_flags, unsigned long user_addr) { - post_alloc_hook(page, order, gfp_flags, user_addr); - if (order && (gfp_flags & __GFP_COMP)) prep_compound_page(page, order); =20 + post_alloc_hook(page, order, gfp_flags, user_addr); + /* * page is set pfmemalloc when ALLOC_NO_WATERMARKS was necessary to * allocate the page. The expectation is that the caller is taking --=20 MST From nobody Wed Jun 17 02:49:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 72D3840DFBB for ; Tue, 21 Apr 2026 22:01:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808891; cv=none; b=enFvsoiyFd/gPzbpAnmLXhgD0PQr8qoc/UG690tu1DKQEgGma3XgAuLGUyb65wWKrlmmqIrpjT6zwAz5kPy0Kd7AaA/KqMpeS6SZEthRlx1jX6+edRVRNEXNY1CjXiT6sDihCiWhWgbudxhsxLBzZ9EDQIY4crhzGUiMezI0Itk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808891; c=relaxed/simple; bh=HzTLV61VjnP6fPkrfX0cTZjMU7mjfErb3xYIOpy+wWg=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=lq4kp7hIWWtCq/YdglAzXps4QIvWhGJgOcUsfedVzB6aBqDf0NDfRQIkWx/UfJiSRxUEplfjzjg/fk0BIfblANwlXiEcBu+4mcUh8wbsDqEUv55NWvrqSJ8OXf7AC8lBZIHI+QgV5dVUgFjPIPae13VRpHAoQ3Yc1yvEHNrFshA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=ENLvh7jv; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=i9WCkpo2; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ENLvh7jv"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="i9WCkpo2" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1776808889; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=XpE7HJIK5tc8NP4T14qa/I9qbq0lyvaV0C1nU6qjpMk=; b=ENLvh7jvf7bFXs+/ETIfxvPWdMFnan2RPW9qtQujm3xmySoEvzvqW4yCNZNDVe576WpGFP DdM7Y9oV4YNz10dyw0Pj41FqC3hiZunGuw/gxI2i1iw7Tg0NLs+AVhrxtb+9Sx3QXjzvOW oDIIW9yNilCjA9seeQvPFkxPbONSXII= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-522-7gKW1UjaPVWvBe61MIEtFw-1; Tue, 21 Apr 2026 18:01:28 -0400 X-MC-Unique: 7gKW1UjaPVWvBe61MIEtFw-1 X-Mimecast-MFC-AGG-ID: 7gKW1UjaPVWvBe61MIEtFw_1776808887 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-43d121c4271so3415321f8f.3 for ; Tue, 21 Apr 2026 15:01:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1776808886; x=1777413686; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=XpE7HJIK5tc8NP4T14qa/I9qbq0lyvaV0C1nU6qjpMk=; b=i9WCkpo2xZnt9DxAN7yXjf7/XkYQCn8ZWYocQZJwnod26FIC3weQn85KquFgDc3DDQ Y/MoL25iqZUPereT3GWONy7W1JGIJrJ1zYF6dB4Yr6unk3S+Be+tPrB0IWx3L7c69asY uv9xItJHhecF/6n2rRMF6PrZV7aesdGXctvmm7esA0FKK1tqhO7FthG1ueywu8riqj5N ltL4bYYXrdClhkDlnTLWfW/GgsBhbo+QuNx00m+XASdP5VoP2cmbCeYEKeW8A9NXQAPi y3xEXRWHQdrFCeJRsvJ6ODSgaafOzryiKb1/L90wTdL1ZQpNQihMPcbzvJmQbvz52mBa 92Ww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776808886; x=1777413686; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XpE7HJIK5tc8NP4T14qa/I9qbq0lyvaV0C1nU6qjpMk=; b=leo+VZL06BLGIcAflrNxpMZ/GRfR2LbSpQQy2/CQDzR2NPLl1sIzkzxE7XRNGvJw78 uFAw8g3L/5/jATmwlIFC39G0E6gBO1ATbGTf7a4H/iSkatwwX22G8VpBcX22zdxmo6kh tYMMlX3FBOqhNGD53E74HHa2DA/TNrODoUDgW+jglyuL2Q2P6STpIWdiK5acdKvbsaUk JbyhPI7i3pBHxmNSyQmWhMN04UqcXwrtU2abFAHZefd9IQmoJy23inHW531yT8w+wR4K Fney4u4tJgTqFFt7ebYnBlbCkF3ICCsLxWz4AXHouKQf5C13hpKh66pzgHkh8Z7icOJN 9SqQ== X-Gm-Message-State: AOJu0Yz3caAqSDbPChx4QWH9Qi5oj86p5ftgxmTNjMuKOdD6IJh7PLVc uI2AAfnxBmEIIKRmukJRf6RMS75exzhJfEpYPHwxmILprGImd/pC1/z87TLFNBBuKHU+z47cU7f e+orL7wCMr7CSQQXIt3PONeHJGhj7rlfhG4Kyd4ofzOHCtjvh6sEJQj2Ks4oOHsvk0mfHYon3Zo G0z5OYpcLRyJlWnBI7VmR65OBK/BLsSH/mg7QXIrH5ubE= X-Gm-Gg: AeBDievpR6GXAIhTldM9JITEQKA1BvBrGJ0FWW5dhEageZueaLdC/HRll24La2vr32P MHxF9trT3jtGYzFm25eMZpSp2NgjYwMEg2+RIQ4YA4zV5qBgljZvR+VdML53KQN+X/E5ykcHKbr K1aPRfrzZApNF0tz9xji/B0U8Fau/a+Xuo+ZOcjlJEbMNuEIxpQB7BVSACmrzau5QQdJRvq1T6f xuBfWcxFOhv9IdnML5Hl23Ci2+1ACpB5kujihKItEtk8nKP9QK8wCN9xty7zJZN0TxKBH+f7OeC VonTWFV4yxj0Iy03EhPv0oAXW1Txit7fp5hZ2nBGnWpOMXzT9LaHCGJmRvAWSUCRBWt0AzDjbQJ dtsg0IPyTtA6BrxCABLGulq0udkTxTAl3o5cAr7Xv9JpR+5HLFjQ/mQ== X-Received: by 2002:a05:600c:8115:b0:488:904b:f31 with SMTP id 5b1f17b1804b1-488fb77e27cmr242401725e9.22.1776808886553; Tue, 21 Apr 2026 15:01:26 -0700 (PDT) X-Received: by 2002:a05:600c:8115:b0:488:904b:f31 with SMTP id 5b1f17b1804b1-488fb77e27cmr242401185e9.22.1776808886074; Tue, 21 Apr 2026 15:01:26 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-21.inter.net.il. [80.230.25.21]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48a4b329542sm227417095e9.3.2026.04.21.15.01.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Apr 2026 15:01:24 -0700 (PDT) Date: Tue, 21 Apr 2026 18:01:22 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Johannes Weiner , Zi Yan Subject: [PATCH RFC v3 04/19] mm: use folio_zero_user for user pages in post_alloc_hook Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When post_alloc_hook() needs to zero a page for an explicit __GFP_ZERO allocation and user_addr is set, use folio_zero_user() instead of kernel_init_pages(). This zeros near the faulting address last, keeping those cachelines hot for the impending user access. folio_zero_user() is only used for explicit __GFP_ZERO, not for init_on_alloc. On architectures with virtually-indexed caches (e.g., ARM), clear_user_highpage() performs per-line cache operations; using it for init_on_alloc would add overhead that kernel_init_pages() avoids (the page fault path flushes the cache at PTE installation time regardless). No functional change yet: current callers do not pass __GFP_ZERO for user pages (they zero at the callsite instead). Subsequent patches will convert them. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/page_alloc.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 99c01eb2d59e..db2192ffc27c 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1882,9 +1882,20 @@ inline void post_alloc_hook(struct page *page, unsig= ned int order, for (i =3D 0; i !=3D 1 << order; ++i) page_kasan_tag_reset(page + i); } - /* If memory is still not initialized, initialize it now. */ - if (init) - kernel_init_pages(page, 1 << order); + /* + * If memory is still not initialized, initialize it now. + * When __GFP_ZERO was explicitly requested and user_addr is set, + * use folio_zero_user() which zeros near the faulting address + * last, keeping those cachelines hot. For init_on_alloc, use + * kernel_init_pages() to avoid unnecessary cache flush overhead + * on architectures with virtually-indexed caches. + */ + if (init) { + if ((gfp_flags & __GFP_ZERO) && user_addr !=3D USER_ADDR_NONE) + folio_zero_user(page_folio(page), user_addr); + else + kernel_init_pages(page, 1 << order); + } =20 set_page_owner(page, order, gfp_flags); page_table_check_alloc(page, order); --=20 MST From nobody Wed Jun 17 02:49:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3745329ACF7 for ; Tue, 21 Apr 2026 22:01:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808894; cv=none; b=F5NuDd6XtNOvfxD+jKIX3HRnX/t7r3wCwVbOSR4XdbALJTrRhzj2D0d7J7K8G468xq5GWZttiqr2t6Uuh50vqjBVRpQhZT6nFQVJfz8IBEgboYhyHt4Z7GWTMNyXoHl6EhOBbt/yAimSbikPaD7tqjTQiMPgoE8itYV1kvPkvEg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808894; c=relaxed/simple; bh=B4v7SD6SJ84KapLgBy+iXreANiiaF75HE81bu1RBt3E=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=L+xd5+Z6Q5nYxfwyaV6pl/aeR325FoHIM7nfXBJmjUkWzBprX5h7f26GplBcAykG7GjQ+5yBMBQzaQ7Zrdx1TkuQpNg1eneB1p8vDolQWlAyKBsc5vW6lImgGMqF3xm7CK/d/OGPzEINinuTVubJM+XopjMsrV2639H3N6mPne8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Aq7IMNGd; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=OYz3UKvd; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Aq7IMNGd"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="OYz3UKvd" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1776808892; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=jpAQ9O1oMgUNlo5ukg9OEno1SM1k+70VK2/jH7/8vVM=; b=Aq7IMNGdy+vcFdysvnN7cwn345HFShyusYwTAF8UzGdv0/BkSTUI0cwxDI9BIWWOiU8kOF aCEILSvOlmzq5NgHrGt6kpNGZzCdA2dfw+/Fn/mTEbA1f2Q3t3n0OOBiXXBJSE8Agvd5hf Y37AqHmFMI3O5yJtZStxAAUl+rxt6vM= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-635-UsxnR7VkMFWEh8Obe5btyA-1; Tue, 21 Apr 2026 18:01:31 -0400 X-MC-Unique: UsxnR7VkMFWEh8Obe5btyA-1 X-Mimecast-MFC-AGG-ID: UsxnR7VkMFWEh8Obe5btyA_1776808890 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-488f973ddfeso37113765e9.3 for ; Tue, 21 Apr 2026 15:01:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1776808889; x=1777413689; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=jpAQ9O1oMgUNlo5ukg9OEno1SM1k+70VK2/jH7/8vVM=; b=OYz3UKvddTv4wdWWfZHksWdWXivUKma9irF6YyNe1dsMSJKEZ+aj73NSxEMLthSyNq QYMCyy8nQQA2dk6owdrDjWjRJaHygKfQgRpEvv9rynswBfVnydqbHPQVr3W77LVeYocr HkUDb3scxz2nUrb84A/8Sz4z5U/ujAc62a14ySoTBXsnzGNlMyM4c9Mg5gqP2f1tVxrg iyahw3Yd575yu6nzEeWlnmdO7nJMc9jUoMKo16i24b9b4D6pD+uO/X8oD1wrIUi+TEE7 pWtgYia1G4TqwRPHsBafNGpzudKaxoBpYAKUU1PJ2XH1Xe79dcAKZXiKbp5ElkHajBgZ pEdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776808889; x=1777413689; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jpAQ9O1oMgUNlo5ukg9OEno1SM1k+70VK2/jH7/8vVM=; b=Qt5j5WTq7VW3xBYxb6uGOowqF0AjPII9pP3E+YgQAdQRYmwAF2TC+SO0SGbDPUnkhB DL3lOIjyvrdpDgyRYVLc+6a6rHbShn1SWngq7PbH+GIDebbJ4NuD86sw2g+iFtoUcgPI DCDhj+bKhwZzdoTRhrFldy9NXlhqFlgATYIxSTjk6eUSmAqo7OWPgx3WKHCc+8hWpgyC QoHXbxyMi726xQlR0I57bTPRklBV+WqrPSR/OjwAsoXMuG1eeITExCbJkIPY+hh0nJ+6 EJeTOPhWsJMU4+R5mp1GbWRNEGAfybczTwumH16Xo6JJ7GhxFxl4lroknJFvZQA5iShW GEng== X-Gm-Message-State: AOJu0Yxpw1QxKvk9eF4uieG3shu/FSZbiTuWaDvIqC5/d+GGDZIJ6pGy N6kEF5goGnDTzUicxF/XEmrd/WMMkuKOGOV8D2Tsnb6aXOgXC2HKjgAA+KIqYqKAsPu0lyreFmf CKT8rzv2gCb9bla2ddZRIvMKTevpkSBCMmbpaFj7pPJki27/jhVkgLIUUUTODIzKhn7IZF/ZNx1 mQdiStOPAAm8X9sUPvtXaBo5Ga9pIfz6L7HmiVJzfP0XU= X-Gm-Gg: AeBDieuMNZfUj7177IgYa3B5Fl+f46f2DV7o6SJ1xExG6khQ+972HFTWR2fLj1NQEQI NFyVGUdwPAuPQLonFMhJ3cc5KRufmoVFdPPRMW7FB3OnOoW0cIR36v4LM9oqpQe4jRgz/nK1cls bXrpEaMynXkkaRnPQmzrgF2TkI8qm3gf3GBF2AnujmbYHwPNNGhoobAINlsCb3JFI9aKuqA+Zoo iw1FrJcv39LB1y3UcrqgLScbbKddjQ9d0q2CFFfEp91eGotp8vQOrS9Ug/y6c17MyWopIbY8cim CtijgqDamCfGlUUSp80V7Lt7pruGOTdLGq3wOfnaGfjf3ZGUrzFIN4xFokSUIKw5qncuzBxe+Kb cpXYXK/e+c+h8h2Jgz6xl6OpxcY5KWDR8lNcCV7UEza94DPmPWSQyjg== X-Received: by 2002:a05:600c:c10e:b0:488:8185:e672 with SMTP id 5b1f17b1804b1-488fb792e84mr214398345e9.30.1776808889566; Tue, 21 Apr 2026 15:01:29 -0700 (PDT) X-Received: by 2002:a05:600c:c10e:b0:488:8185:e672 with SMTP id 5b1f17b1804b1-488fb792e84mr214397765e9.30.1776808889047; Tue, 21 Apr 2026 15:01:29 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-21.inter.net.il. [80.230.25.21]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-488fc0f8193sm361461265e9.1.2026.04.21.15.01.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Apr 2026 15:01:28 -0700 (PDT) Date: Tue, 21 Apr 2026 18:01:26 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Lorenzo Stoakes , "Liam R. Howlett" , Mike Rapoport Subject: [PATCH RFC v3 05/19] mm: use __GFP_ZERO in vma_alloc_zeroed_movable_folio Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that post_alloc_hook() handles cache-friendly user page zeroing via folio_zero_user(), convert vma_alloc_zeroed_movable_folio() to pass __GFP_ZERO instead of zeroing at the callsite. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/highmem.h | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/include/linux/highmem.h b/include/linux/highmem.h index af03db851a1d..ffa683f64f1d 100644 --- a/include/linux/highmem.h +++ b/include/linux/highmem.h @@ -320,13 +320,8 @@ static inline struct folio *vma_alloc_zeroed_movable_folio(struct vm_area_struct *vma, unsigned long vaddr) { - struct folio *folio; - - folio =3D vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, vma, vaddr); - if (folio && user_alloc_needs_zeroing()) - clear_user_highpage(&folio->page, vaddr); - - return folio; + return vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, + 0, vma, vaddr); } #endif =20 --=20 MST From nobody Wed Jun 17 02:49:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 04FFF25392A for ; Tue, 21 Apr 2026 22:01:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808898; cv=none; b=o1RfJNJ9bCwdZPrnekLVJ+NygjNVlrCyuwvJ+KXKUroo6CVLKF4wFeZSFBmuMdeTQnhnwUgwSZgKxv4q7qXI4vNjWUaSz/RILwCjQQqoxkrfWn/7+Gi6td7MHxAsrfX1wSz0QeCdyiDzGzc8uVppVOX6V+xTZuPfibPbYCxNHgk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808898; c=relaxed/simple; bh=vQft5FFy4m26TJsCTwjR9OY8cV1I9SP5mZO5ylhP/Rk=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=okLVOaIaxwCAiSTz1DLuzw5WYqOfDWiewqQCBlWi0oT9yGTAnGo6zGh6XpU8E9vwp7azGGjwNb0hkupaxP5mdnWQTn4gEeO/dhhTOqbDbceOLutQ4m7s1J9CmRWV3nZiS1EoOsioVTF0FLkTotql4tAQHuYUfGbBc8tDgpJO/CA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=GnhFVZbP; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=hyFnfw+Y; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="GnhFVZbP"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="hyFnfw+Y" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1776808896; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ogU7GmeRIQqtLBfKG+aC0EEQurcY6mw6YDIsyPfIVI4=; b=GnhFVZbPhKpu4Phi4PxeG11ZG1EqvKpLVV47uoyrJLhClnPyRLMkxP0OynwoDIhToEmr0i dEFiaOHO68W+cdBOaUq/4o3oT1g3VqffBPrR/LR9yAHluJC/hA5/7dZUp/rUuyl+nc5+kI LlQbWUSAFwc023FqbWNaTnegFWqv7cE= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-647-6ZUpsJrZMuuayFGmWf7ztA-1; Tue, 21 Apr 2026 18:01:34 -0400 X-MC-Unique: 6ZUpsJrZMuuayFGmWf7ztA-1 X-Mimecast-MFC-AGG-ID: 6ZUpsJrZMuuayFGmWf7ztA_1776808893 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-4891f2eecb7so16044745e9.0 for ; Tue, 21 Apr 2026 15:01:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1776808893; x=1777413693; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=ogU7GmeRIQqtLBfKG+aC0EEQurcY6mw6YDIsyPfIVI4=; b=hyFnfw+YJIOsizOdJfSnsOeWhLfKebHk/OsSL4rTUW6xBWp0ZjMEgplhKnvv6EYufG EEjpmqpLp0dLZnCpUGMqG8A7GdlGf0oqF5/pTPmoemvGdwl2YtS6kDYINX/JAcEIiAnL YxtrTV+FCTCZXxeN/MvZfopkU4Bl49pRs/pdYBJS79drja2jBhmYbeE+Y33VKO6hnPah rm6Lirq6Y2v8EmzI1JqJd0KRmGLY74OnUIlZtGA55LLq2yboyKlZn/PLzeLQwZaHSlbb q97QkgRj1jCK8d6FrV80kZ9wsXshT3klBEhwHCWz6hVFVv8o2d9FIOb+8bbD6ujrDkFI gILw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776808893; x=1777413693; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ogU7GmeRIQqtLBfKG+aC0EEQurcY6mw6YDIsyPfIVI4=; b=npGRB5ZRG8UT4hURPckWIt0MH6JHQH7it7TSjcUbWlBDGDrbWqt4Fv6Uxwt5rdLoOi Wo2GHAMHNfeDKQ4iXiP/C9TjO2wiW2J1kfvUVDaxgpjcpwuRN2JT9jUpFnRbWTbUQleO VfvoGjQAH7DqPA/fM4D+A42it2vRJLjYvzBv3ZKthwLzSDOEJL2l+GIyuWWeR0/FuAlt /+gfSLyOsGUwmf04pPMs/qvfY/Q1FXAweoenvPNId0oxaoo60SXtU9iw9lnZCyt+QY0Q 0ApIXetzosBCAQjaIMtAsy9NqJWe8khprqMANMrj4wNaqmPTJELj2G+wetwY9XBmq5VD sTfw== X-Gm-Message-State: AOJu0Yw6qrM5B3QwWQGK+P1bqcj0GXWOodSU61rBnHtRxC7eZUBIQam7 STKbrWVGPu61MNPNEeOY9zTpBAIiwDNxBlS4vuMpDjChXGuBeKs9MKGmJn92VmdxVjx9Z9Xyo0/ fzh43ozhqhCaFp7va/W3PFl2wQ2H/X2J2DnQcgJuYf/q+8V39eKSA1MoLlYsCYMErBQ7KbyZLbM hUpmECjW10Nfu2ucFtD9Yo8/f9jwi1Jyrb0ukwDhE2UD8= X-Gm-Gg: AeBDieu08Hwd1IDV4ELVLDoUHsl//cCvVasCyy8dOH3xEmN01WrV2B4NJjAh8ADFy41 PElzAm5PzlShpu07kIKnSs9uPLjbq18OSYZHf3RX/IC1JfQwdznDvfBKPhMBpTbmLWnXln2I3TZ v7pYKx+otOYu/BHjDIu3f3P/UwJWPpZkdlRRxRIPDL+Fvt54UfrOSJJTOssRsmFc7lFPWoTKgP4 LHxH29p1mo1M1cZ58YZ9Jem//xSuHrPX5IwgtPp78/Rm+jMihL8GxmDkFyuhIWU65uX4HaBKYWB 36H5nlx5fyk5PbcvVppto5Qv1D9RUr1J8WuyuCr4JY3Z9jICdhDoinz4jaGdK1fXT5ktqgPsJMY IH5XF8BIEIARgda1wlvt/UPegYRzoMpixorCFvdYKuUVUKtXt7j3lvw== X-Received: by 2002:a05:600c:6085:b0:488:936a:6220 with SMTP id 5b1f17b1804b1-488fb782618mr270881715e9.21.1776808893068; Tue, 21 Apr 2026 15:01:33 -0700 (PDT) X-Received: by 2002:a05:600c:6085:b0:488:936a:6220 with SMTP id 5b1f17b1804b1-488fb782618mr270881225e9.21.1776808892535; Tue, 21 Apr 2026 15:01:32 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-21.inter.net.il. [80.230.25.21]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-489201cde98sm158378665e9.7.2026.04.21.15.01.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Apr 2026 15:01:31 -0700 (PDT) Date: Tue, 21 Apr 2026 18:01:29 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Lorenzo Stoakes , "Liam R. Howlett" , Mike Rapoport Subject: [PATCH RFC v3 06/19] mm: use __GFP_ZERO in alloc_anon_folio Message-ID: <41d4e2acba16a35d8572e39a8cf73e33ed9fc30d.1776808210.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Convert alloc_anon_folio() to pass __GFP_ZERO instead of zeroing at the callsite. The allocator now handles cache-friendly zeroing via folio_zero_user() in post_alloc_hook(). Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/memory.c | 15 +++------------ 1 file changed, 3 insertions(+), 12 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 07778814b4a8..ed3797a6e121 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4662,7 +4662,7 @@ static struct folio *alloc_swap_folio(struct vm_fault= *vmf) gfp =3D vma_thp_gfp_mask(vma); while (orders) { addr =3D ALIGN_DOWN(vmf->address, PAGE_SIZE << order); - folio =3D vma_alloc_folio(gfp, order, vma, addr); + folio =3D vma_alloc_folio(gfp, order, vma, vmf->address); if (folio) { if (!mem_cgroup_swapin_charge_folio(folio, vma->vm_mm, gfp, entry)) @@ -5176,10 +5176,10 @@ static struct folio *alloc_anon_folio(struct vm_fau= lt *vmf) goto fallback; =20 /* Try allocating the highest of the remaining orders. */ - gfp =3D vma_thp_gfp_mask(vma); + gfp =3D vma_thp_gfp_mask(vma) | __GFP_ZERO; while (orders) { addr =3D ALIGN_DOWN(vmf->address, PAGE_SIZE << order); - folio =3D vma_alloc_folio(gfp, order, vma, addr); + folio =3D vma_alloc_folio(gfp, order, vma, vmf->address); if (folio) { if (mem_cgroup_charge(folio, vma->vm_mm, gfp)) { count_mthp_stat(order, MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE); @@ -5187,15 +5187,6 @@ static struct folio *alloc_anon_folio(struct vm_faul= t *vmf) goto next; } folio_throttle_swaprate(folio, gfp); - /* - * When a folio is not zeroed during allocation - * (__GFP_ZERO not used) or user folios require special - * handling, folio_zero_user() is used to make sure - * that the page corresponding to the faulting address - * will be hot in the cache after zeroing. - */ - if (user_alloc_needs_zeroing()) - folio_zero_user(folio, vmf->address); return folio; } next: --=20 MST From nobody Wed Jun 17 02:49:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E3E43352C35 for ; Tue, 21 Apr 2026 22:01:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808908; cv=none; b=ScJghoWblIQhV4RQL64CmSaqg7gfq7+316tFwZ02g2mJwrOD22kDieIKGn4wYk0U4aHr3QBCWjYw+UZWHr7/x/3yzp1akMteR16ejUHt1iwmu0CWAG95gn5cT7kx/MkBN+cbCgRDCIYRG0AlbBlUq92QL+jENJs5dIfST30I2vw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808908; c=relaxed/simple; bh=Ni2emDZHUT3ZFPskAmph4HwdB+pXcvb9empy5VSuXX4=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=giUp6CqNdslf8ySZX1nxDuEQn1sDnAcQqfidBSB3HU0++d7aNVOUR8kpgD+v/kAT/B+ebsbCXkslsEY1x52Dudw9HX46z5UywBJLoRThYc4otIomayxXM45HYk8R2zDOGeBMoHhno0LuqnhPi48F2RjWS+nbWOiiJTPn4nrGbCQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=H6cSD5sZ; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=GWd4gynj; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="H6cSD5sZ"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="GWd4gynj" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1776808903; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=PFV0AQVcu3YAl0tYQcglaFwWX0QLOSqW8EZn+GtNSTo=; b=H6cSD5sZQfQJxHDYfPOS29Dfyo0nqIGdNids/gXBb9oMAAZTARRj7dLMd7m4zzASPokDmU QGaWkupisqN76HWAeh1ee8ToBfqHglCv2PDnzMwwm5AyhaKTbHLcMQINKTGDxNd0JaACGI NIVuKHRbJBGtjB+kTTPYtehY+d+GEn4= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-586-qtQXdsE7Ma6OsGXmJXw4cw-1; Tue, 21 Apr 2026 18:01:42 -0400 X-MC-Unique: qtQXdsE7Ma6OsGXmJXw4cw-1 X-Mimecast-MFC-AGG-ID: qtQXdsE7Ma6OsGXmJXw4cw_1776808897 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-43ea7a5da57so3712674f8f.1 for ; Tue, 21 Apr 2026 15:01:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1776808897; x=1777413697; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=PFV0AQVcu3YAl0tYQcglaFwWX0QLOSqW8EZn+GtNSTo=; b=GWd4gynjULrMkqNnvCvTt9XRynpYGHJhLCA+cQAFkT5ZTdELH1vraNUUXc8fHjrWVy RqT7TEm6wByELwEKZqHi1YdfdRuv9bOK/Gn5zCGO10g67Cj18dyrbnBIrIb83alxwAti C/JqfzBaH89SsjyOp8K75KHued/r3Yo6pFEHwe6G/bTUwnYvONZjF55njmDbeD9gupvE hiE8Si2SOKPGu51Ltmlacon+9xwSwkD+oHqH7NqGsUlkj4U0g14RRiFwNL3ryJxnHBYW C+gjOqs58/gZefEQ3eifJuj4UHRTPIGRnj831U9LBLN9SyJWgnCPHb3lMuNjDSlITuVJ sPMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776808897; x=1777413697; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PFV0AQVcu3YAl0tYQcglaFwWX0QLOSqW8EZn+GtNSTo=; b=shlegBdVmPmYrk2n1gmkXcuEHxJS8SFNlIjHWXCuDGu1rA+MSLiqeM9BBV8274EjuO Sb9Owi2AgfZLXeYVmkWew0I++P1iMjFYCASIHkePP7rb1WH1YYqzrWXvh9iQEuad1AiW T0v5veEA1vXAlsbTfGscjTCpazOO3bpbJ5zgSU1ymH5rObsMTJvqaZBE+67cBOT8ZsQG mjREXxZtMgPFlFyRMfWVuRS6EB2PSqJhzi5kXgrLZYxmd8KMVCrJqLqhc1DbjsImyIV3 1LLjSQhy46Z91iDak9cH74LQ5+uBGRKU5V7F2638f3cpvcWxuTUvbJJnyyr+sIFWmxF9 ceBQ== X-Gm-Message-State: AOJu0Yx1+6S4+XYewZbnHcE6zkKQwxm4A27PHM0zQexT/x2pg7jDWEtg fRdQUEsB/QyPGoe9cmJf0pMnaAkHKbkMgAeCTSPC+i0k2pndfMHtx2nG5mV3LcAQoFCFBruMecU 7JypuC8IfmzI2rKv2BMUfvg8hmm+uh/ap3d6f+O1szvOOiuOw1O9/TVbKQg35ZZv9nsw5MmlJTM 07uHMmtVMeNhOM0sOfQsYhxSP2GMtp+rlkCpwj4EQhhrM= X-Gm-Gg: AeBDievYaoLR3Ir5h3NkHLRRws3cjDWOm9qQbIBV269A7qBtiStGe5nzwgCobLdn1VS JAj43P364flVSLJZZI8UR8j/WLlorY6hbyaepx5QC33yRz7Z/TNB/nSehYHliMCritYuuZcX3Bv m/gXceBWL7OeXZ0rfFE07S390yMuV3WzPZ1lTLdoYMwZtxiv8vj//+up9cMw/CawGjxqTnnuUdc oSvI/f1NyuIS5X2jWoyJ5wkouR0zs/+I46jKV3PqtDGjwDXLCY1XFLJ/yM3//fyR8LZAL51fMaE QDeIt0AL4a5vSwRu5UYGppDczGnr1xWE2WM/ia3znxVRxfPqN4Zxw0P+fEaXaDk5l1K9KXc+/mN J6vlztvZAY2qZ/Ot8LkhMThjgRgGBUnSmKkOEvoMJ+JhH09nrQvuBgg== X-Received: by 2002:a05:6000:18a5:b0:43c:ff58:35c3 with SMTP id ffacd0b85a97d-43fe3dc818fmr31329467f8f.10.1776808896382; Tue, 21 Apr 2026 15:01:36 -0700 (PDT) X-Received: by 2002:a05:6000:18a5:b0:43c:ff58:35c3 with SMTP id ffacd0b85a97d-43fe3dc818fmr31329402f8f.10.1776808895818; Tue, 21 Apr 2026 15:01:35 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-21.inter.net.il. [80.230.25.21]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-43fe4e3a174sm44697673f8f.18.2026.04.21.15.01.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Apr 2026 15:01:35 -0700 (PDT) Date: Tue, 21 Apr 2026 18:01:32 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Lorenzo Stoakes , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang Subject: [PATCH RFC v3 07/19] mm: use __GFP_ZERO in vma_alloc_anon_folio_pmd Message-ID: <028a9602232ce76e2b207f730982ad3135897bc3.1776808210.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Convert vma_alloc_anon_folio_pmd() to pass __GFP_ZERO instead of zeroing at the callsite. Pass the exact fault address (not PMD-aligned) to vma_alloc_folio() to ensure the cache locality optimization in folio_zero_user() works correctly. The NUMA interleave index computation already shifts by PAGE_SHIFT + order, so the unmasked address gives the same result. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/huge_memory.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 8e2746ea74ad..3f2a868cf9e9 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1256,11 +1256,11 @@ EXPORT_SYMBOL_GPL(thp_get_unmapped_area); static struct folio *vma_alloc_anon_folio_pmd(struct vm_area_struct *vma, unsigned long addr) { - gfp_t gfp =3D vma_thp_gfp_mask(vma); + gfp_t gfp =3D vma_thp_gfp_mask(vma) | __GFP_ZERO; const int order =3D HPAGE_PMD_ORDER; struct folio *folio; =20 - folio =3D vma_alloc_folio(gfp, order, vma, addr & HPAGE_PMD_MASK); + folio =3D vma_alloc_folio(gfp, order, vma, addr); =20 if (unlikely(!folio)) { count_vm_event(THP_FAULT_FALLBACK); @@ -1279,14 +1279,6 @@ static struct folio *vma_alloc_anon_folio_pmd(struct= vm_area_struct *vma, } folio_throttle_swaprate(folio, gfp); =20 - /* - * When a folio is not zeroed during allocation (__GFP_ZERO not used) - * or user folios require special handling, folio_zero_user() is used to - * make sure that the page corresponding to the faulting address will be - * hot in the cache after zeroing. - */ - if (user_alloc_needs_zeroing()) - folio_zero_user(folio, addr); /* * The memory barrier inside __folio_mark_uptodate makes sure that * folio_zero_user writes become visible before the set_pmd_at() --=20 MST From nobody Wed Jun 17 02:49:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E04F640DFBB for ; Tue, 21 Apr 2026 22:01:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808905; cv=none; b=TikhSHYlxjzm1nZY4zJrMGl5Kp4OyF3h4koEn8e+Vj0i7XziRtM3EkCV92wQTHRvaHKR0m0fhamWtJ0CM/G+cun55//JH/ER8flqrF3gkh/yI+NGumJOrcfTwaWI/w9t7mTJRSDJlI8kIgBdaSIfZ4+Enu6RpMS0ab0qnuYnv5Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808905; c=relaxed/simple; bh=EWFOuheoC1DvTBgPAHezGTmeJdyms6YPvVMBqd1xoVo=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=cTHzaI1r2zMgsd8vSf2IxCvDdnt1tbA8fj6wFBuxhyhnKPlybN6+VUdHNSPQ0WRMkTeevFoc3BGm3k0v9WCcHDo5BilKFCNKjY56cDKjrTpPiZ4loxhm95dcgU9MPXRdzC9WWzxoZf9F87XL2sc0keD3fg+e0AZ57oGJdkZdgII= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=VLTXgLIR; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=N0M9ssAq; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="VLTXgLIR"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="N0M9ssAq" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1776808903; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=d5XFq6evYUr+s+QSCea6SglT5emqQpzsUniObvQ6PpU=; b=VLTXgLIRYKxhn9XUHo/+ttVvYoK0s3ePy+pnGO8ARmH8tSnIkhfHqtcJuEIHoQEAfgES+p tN6PeSkndpnjpEB0HvfE+b93TLrduOVUVVghJNAiUkK/7jQB+8gro5ZVwpTbicO32f3/2w Jw8oz14VSRWMdqovF7nmJwNYUsChqxk= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-187-QDyE-tJoPDO7tq6mya4OiQ-1; Tue, 21 Apr 2026 18:01:41 -0400 X-MC-Unique: QDyE-tJoPDO7tq6mya4OiQ-1 X-Mimecast-MFC-AGG-ID: QDyE-tJoPDO7tq6mya4OiQ_1776808901 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-48378df3469so31088255e9.1 for ; Tue, 21 Apr 2026 15:01:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1776808900; x=1777413700; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=d5XFq6evYUr+s+QSCea6SglT5emqQpzsUniObvQ6PpU=; b=N0M9ssAqURM6CtMxjVm2OmYfRHCdorvWKUYlCQceYV+36FwU7tOVI/EaqQf1hJTxrg xNNlyTTh7Sz7eyagRU4R6am5VhmYb+AkNAt3SNFk8qk0fvTk9zrPfPXJINSLKRotFe3z kwkmKjBGQvi5OEURd0shpLHkNgbJ7i5DarT9tZh8qxSaxroD8FPeYGvXtVf7iFcg+Zaw YisGkJ9Kc2tUebT88LjQTe7ovr6Q2vGqI2wLeutl5mMFPGv5sedI1uGFOHWhrMzMex/m IqGUF+8Rb1fB4XsEwe8AS0pwDeV9OOBzudDN519wCeJlUwYmukvFLB/51ngKLpUYiKXf 7W1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776808900; x=1777413700; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=d5XFq6evYUr+s+QSCea6SglT5emqQpzsUniObvQ6PpU=; b=MBSHNz0o5XvYU59+4Xo5+nT9gDaFqotlmRBecXj+gpWeH1JGt++ItnN31cbkXJzNxF vU4+0/PeDs8zz/ODH7Of4dmM7awdqyMGaOmlzs5kgsVhuTognqBelQseKQvpYeEaqI5l qaHujwXyAJy9PlLkEb1osxd+rD+WnU13Z3dMEx3vxGdHXJw6/E1iiQTLUzBtlraSW1j9 6og9JeWS+nJozgTIy5Q/1yCfbUsyfldcXY4ItiPijIgzHQMTvPsWhOtHjOhNrKNvf8gJ Dv2M38HUDWNL63Re7rMpeobYdbZH85MepJvzLD5Nx1xwunTn0T3s2aq30YBT9OMHAs16 UZyw== X-Gm-Message-State: AOJu0YxPc1fUl+fEVzs0d8QXil6TPbK7n8J3gkTl+1zrTBSgbRwDLZfL +7GuiiuefMohCUl1mC0WDnuE5RiP/XQML8tjrrWtNbXPtR7QuVYL6SNVRCWkBZhz0eZ3+0Km4O/ MwTk7WCehRWg/RpfNYUF0jOtFk+5sWTSfGgnJ//EsxNqyrX70jsVlHis6aRKmjKA5iK5MO79ltm TZLvJgHKupQ6YIS0Fid5ev9htP5vb7jd+K2NTZqf+TrWg= X-Gm-Gg: AeBDies+2LM897NPjj70tsvi6Ek1GUKXyHBOAXD0Ftca8L01K4qgKxye97kQ2ONTPst mv7fTNhm7Rdwlaiq+W+OuZ+Ijwm6UYlMkhSuoY6wpA0abFDQsdJv+em04x5oKMreNqJEWkMvu7m IjsfJirNPZdTl2EgB6Q743B7Qyj3JGVFID135txCqDuWiQByiyysWED11GDnRnbUhXd/VnmhQP2 A6FliqFgDXZTIU18py/kXK9ohySdPtllWMy4m+gLmtOSaKly7teE44Fkn9YyFwiIHpG3VvRSLLU PyD2PWsRgyiKYsEfEebtX7NwyQQU5Ueu2pDcLeOukhn9Vsruc4jlUGYbB9AfZ955h0pPdvlgj06 qiQtN7rt9z/kzlOdNPpLna3ILrINDQFZD10lB3rCm5AKj28zF+eDThA== X-Received: by 2002:a05:600c:3ba0:b0:486:faa8:9e4 with SMTP id 5b1f17b1804b1-488fb8b91a7mr256829035e9.12.1776808900263; Tue, 21 Apr 2026 15:01:40 -0700 (PDT) X-Received: by 2002:a05:600c:3ba0:b0:486:faa8:9e4 with SMTP id 5b1f17b1804b1-488fb8b91a7mr256828685e9.12.1776808899635; Tue, 21 Apr 2026 15:01:39 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-21.inter.net.il. [80.230.25.21]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4891bba6276sm93953895e9.0.2026.04.21.15.01.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Apr 2026 15:01:39 -0700 (PDT) Date: Tue, 21 Apr 2026 18:01:36 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Muchun Song , Oscar Salvador Subject: [PATCH RFC v3 08/19] mm: hugetlb: use __GFP_ZERO and skip zeroing for zeroed pages Message-ID: <6897aec7727120849077661a33248fa2d58b4fe5.1776808210.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Convert the hugetlb fault and fallocate paths to use __GFP_ZERO. For pages allocated from the buddy allocator, post_alloc_hook() handles zeroing (with zeroed skip when the host already zeroed the page). Hugetlb surplus pages need special handling because they can be pre-allocated into the pool during mmap (by hugetlb_acct_memory) before any page fault. Pool pages are kept around and may need zeroing long after buddy allocation, so PG_zeroed (consumed at allocation time) cannot track their state. Add a bool *zeroed output parameter to alloc_hugetlb_folio() so callers know whether the page needs zeroing. Buddy-allocated pages are always zeroed (zeroed by post_alloc_hook). Pool pages use a new HPG_zeroed flag to track whether the page is known-zero (freshly buddy-allocated, never mapped to userspace). The flag is set in alloc_surplus_hugetlb_folio() after buddy allocation and cleared in free_huge_folio() when a user-mapped page returns to the pool. Callers that do not need zeroing (CoW, migration) pass NULL for zeroed and 0 for gfp. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- fs/hugetlbfs/inode.c | 10 ++++++-- include/linux/hugetlb.h | 8 ++++-- mm/hugetlb.c | 54 ++++++++++++++++++++++++++++++++--------- 3 files changed, 56 insertions(+), 16 deletions(-) diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c index 3f70c47981de..d5d570d6eff4 100644 --- a/fs/hugetlbfs/inode.c +++ b/fs/hugetlbfs/inode.c @@ -822,14 +822,20 @@ static long hugetlbfs_fallocate(struct file *file, in= t mode, loff_t offset, * folios in these areas, we need to consume the reserves * to keep reservation accounting consistent. */ - folio =3D alloc_hugetlb_folio(&pseudo_vma, addr, false); + { + bool zeroed; + + folio =3D alloc_hugetlb_folio(&pseudo_vma, addr, false, + __GFP_ZERO, &zeroed); if (IS_ERR(folio)) { mutex_unlock(&hugetlb_fault_mutex_table[hash]); error =3D PTR_ERR(folio); goto out; } - folio_zero_user(folio, addr); + if (!zeroed) + folio_zero_user(folio, addr); __folio_mark_uptodate(folio); + } error =3D hugetlb_add_to_page_cache(folio, mapping, index); if (unlikely(error)) { restore_reserve_on_error(h, &pseudo_vma, addr, folio); diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 65910437be1c..094714c607f9 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -598,6 +598,7 @@ enum hugetlb_page_flags { HPG_vmemmap_optimized, HPG_raw_hwp_unreliable, HPG_cma, + HPG_zeroed, __NR_HPAGEFLAGS, }; =20 @@ -658,6 +659,7 @@ HPAGEFLAG(Freed, freed) HPAGEFLAG(VmemmapOptimized, vmemmap_optimized) HPAGEFLAG(RawHwpUnreliable, raw_hwp_unreliable) HPAGEFLAG(Cma, cma) +HPAGEFLAG(Zeroed, zeroed) =20 #ifdef CONFIG_HUGETLB_PAGE =20 @@ -705,7 +707,8 @@ int isolate_or_dissolve_huge_folio(struct folio *folio,= struct list_head *list); int replace_free_hugepage_folios(unsigned long start_pfn, unsigned long en= d_pfn); void wait_for_freed_hugetlb_folios(void); struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, - unsigned long addr, bool cow_from_owner); + unsigned long addr, bool cow_from_owner, + gfp_t gfp, bool *zeroed); struct folio *alloc_hugetlb_folio_nodemask(struct hstate *h, int preferred= _nid, nodemask_t *nmask, gfp_t gfp_mask, bool allow_alloc_fallback); @@ -1117,7 +1120,8 @@ static inline void wait_for_freed_hugetlb_folios(void) =20 static inline struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, unsigned long addr, - bool cow_from_owner) + bool cow_from_owner, + gfp_t gfp, bool *zeroed) { return NULL; } diff --git a/mm/hugetlb.c b/mm/hugetlb.c index de8361b503d2..4f0ed01f5b13 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1744,6 +1744,9 @@ void free_huge_folio(struct folio *folio) int nid =3D folio_nid(folio); struct hugepage_subpool *spool =3D hugetlb_folio_subpool(folio); bool restore_reserve; + + /* Page was mapped to userspace; no longer known-zero */ + folio_clear_hugetlb_zeroed(folio); unsigned long flags; =20 VM_BUG_ON_FOLIO(folio_ref_count(folio), folio); @@ -2146,6 +2149,10 @@ static struct folio *alloc_surplus_hugetlb_folio(str= uct hstate *h, if (!folio) return NULL; =20 + /* Mark as known-zero only if __GFP_ZERO was requested */ + if (gfp_mask & __GFP_ZERO) + folio_set_hugetlb_zeroed(folio); + spin_lock_irq(&hugetlb_lock); /* * nr_huge_pages needs to be adjusted within the same lock cycle @@ -2209,11 +2216,11 @@ static struct folio *alloc_migrate_hugetlb_folio(st= ruct hstate *h, gfp_t gfp_mas */ static struct folio *alloc_buddy_hugetlb_folio_with_mpol(struct hstate *h, - struct vm_area_struct *vma, unsigned long addr) + struct vm_area_struct *vma, unsigned long addr, gfp_t gfp) { struct folio *folio =3D NULL; struct mempolicy *mpol; - gfp_t gfp_mask =3D htlb_alloc_mask(h); + gfp_t gfp_mask =3D htlb_alloc_mask(h) | gfp; int nid; nodemask_t *nodemask; =20 @@ -2910,7 +2917,8 @@ typedef enum { * When it's set, the allocation will bypass all vma level reservations. */ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, - unsigned long addr, bool cow_from_owner) + unsigned long addr, bool cow_from_owner, + gfp_t gfp, bool *zeroed) { struct hugepage_subpool *spool =3D subpool_vma(vma); struct hstate *h =3D hstate_vma(vma); @@ -2919,7 +2927,9 @@ struct folio *alloc_hugetlb_folio(struct vm_area_stru= ct *vma, map_chg_state map_chg; int ret, idx; struct hugetlb_cgroup *h_cg =3D NULL; - gfp_t gfp =3D htlb_alloc_mask(h) | __GFP_RETRY_MAYFAIL; + bool from_pool; + + gfp |=3D htlb_alloc_mask(h) | __GFP_RETRY_MAYFAIL; =20 idx =3D hstate_index(h); =20 @@ -2987,13 +2997,15 @@ struct folio *alloc_hugetlb_folio(struct vm_area_st= ruct *vma, folio =3D dequeue_hugetlb_folio_vma(h, vma, addr, gbl_chg); if (!folio) { spin_unlock_irq(&hugetlb_lock); - folio =3D alloc_buddy_hugetlb_folio_with_mpol(h, vma, addr); + folio =3D alloc_buddy_hugetlb_folio_with_mpol(h, vma, addr, gfp); if (!folio) goto out_uncharge_cgroup; spin_lock_irq(&hugetlb_lock); list_add(&folio->lru, &h->hugepage_activelist); folio_ref_unfreeze(folio, 1); - /* Fall through */ + from_pool =3D false; + } else { + from_pool =3D true; } =20 /* @@ -3016,6 +3028,14 @@ struct folio *alloc_hugetlb_folio(struct vm_area_str= uct *vma, =20 spin_unlock_irq(&hugetlb_lock); =20 + if (zeroed) { + if (from_pool) + *zeroed =3D folio_test_hugetlb_zeroed(folio); + else + *zeroed =3D true; /* buddy-allocated, zeroed by post_alloc_hook */ + folio_clear_hugetlb_zeroed(folio); + } + hugetlb_set_folio_subpool(folio, spool); =20 if (map_chg !=3D MAP_CHG_ENFORCED) { @@ -5004,7 +5024,7 @@ int copy_hugetlb_page_range(struct mm_struct *dst, st= ruct mm_struct *src, spin_unlock(src_ptl); spin_unlock(dst_ptl); /* Do not use reserve as it's private owned */ - new_folio =3D alloc_hugetlb_folio(dst_vma, addr, false); + new_folio =3D alloc_hugetlb_folio(dst_vma, addr, false, 0, NULL); if (IS_ERR(new_folio)) { folio_put(pte_folio); ret =3D PTR_ERR(new_folio); @@ -5533,7 +5553,7 @@ static vm_fault_t hugetlb_wp(struct vm_fault *vmf) * be acquired again before returning to the caller, as expected. */ spin_unlock(vmf->ptl); - new_folio =3D alloc_hugetlb_folio(vma, vmf->address, cow_from_owner); + new_folio =3D alloc_hugetlb_folio(vma, vmf->address, cow_from_owner, 0, N= ULL); =20 if (IS_ERR(new_folio)) { /* @@ -5793,7 +5813,11 @@ static vm_fault_t hugetlb_no_page(struct address_spa= ce *mapping, goto out; } =20 - folio =3D alloc_hugetlb_folio(vma, vmf->address, false); + { + bool zeroed; + + folio =3D alloc_hugetlb_folio(vma, vmf->address, false, + __GFP_ZERO, &zeroed); if (IS_ERR(folio)) { /* * Returning error will result in faulting task being @@ -5813,9 +5837,15 @@ static vm_fault_t hugetlb_no_page(struct address_spa= ce *mapping, ret =3D 0; goto out; } - folio_zero_user(folio, vmf->real_address); + /* + * Buddy-allocated pages are zeroed in post_alloc_hook(). + * Pool pages bypass the allocator, zero them here. + */ + if (!zeroed) + folio_zero_user(folio, vmf->real_address); __folio_mark_uptodate(folio); new_folio =3D true; + } =20 if (vma->vm_flags & VM_MAYSHARE) { int err =3D hugetlb_add_to_page_cache(folio, mapping, @@ -6252,7 +6282,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, goto out; } =20 - folio =3D alloc_hugetlb_folio(dst_vma, dst_addr, false); + folio =3D alloc_hugetlb_folio(dst_vma, dst_addr, false, 0, NULL); if (IS_ERR(folio)) { pte_t *actual_pte =3D hugetlb_walk(dst_vma, dst_addr, PMD_SIZE); if (actual_pte) { @@ -6299,7 +6329,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, goto out; } =20 - folio =3D alloc_hugetlb_folio(dst_vma, dst_addr, false); + folio =3D alloc_hugetlb_folio(dst_vma, dst_addr, false, 0, NULL); if (IS_ERR(folio)) { folio_put(*foliop); ret =3D -ENOMEM; --=20 MST From nobody Wed Jun 17 02:49:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D77EA3502B8 for ; Tue, 21 Apr 2026 22:01:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808908; cv=none; b=MHD1GS/+3puf72ISrtUAzpko53GB+9kE+MKFez40QHcnUM4lhZE0XpcZRHRcHYFfcZuaY0xGZkxG8FCgHj2wLMT8g0vHfGfAfo0v1sTFJbv6xXYXMQkSBfq9RW/kZSuJWrksX5uT6wE0RQ7cohyw8/NYZmqMUmIOEsFlp7yIB4Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808908; c=relaxed/simple; bh=1ejL+Z1wL5a9wUXBRiBrtQNmVFq1XngLbhmkTIDh9l0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=famyBf9G4eowMAivYuSFJBn/a8M7s7M1WseolPZ2Y5gfUkFqy33FvZUD5CorDkCTVDWxOsFzpxxVINe/vpyaoKjAeFpaVhYxuimT3qeX0piZEP7mva/epx3OC2naDYU51lDO5bDQMr0BAweb81hqxU8UpJkkmHbzwQAq8qtZI7o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=LqIMIUTE; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=qeJZKX+l; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="LqIMIUTE"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="qeJZKX+l" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1776808906; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=MQ6M8fMmd15EsU5jHzG0CofBcVEbXAB28EkZbWFSSyk=; b=LqIMIUTEKTtiMH+DskFf9mlc8sou+kceV1fVHpzIfMy1kt803DDNwE+wmutGGlIH5nsCWp 1rm/YKZeFI6YV9oSPq7TuUvVU7cEdsWy5QyiC9boEgyt9n4D9yEZ+T406ZnYRGYwoEx1IK r104U6B6kznGfMr1HfjL9gQQlGpnaAs= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-21-opVl3cUEOFu_i_yKTGPo_Q-1; Tue, 21 Apr 2026 18:01:44 -0400 X-MC-Unique: opVl3cUEOFu_i_yKTGPo_Q-1 X-Mimecast-MFC-AGG-ID: opVl3cUEOFu_i_yKTGPo_Q_1776808903 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-48a55ecc249so8064405e9.1 for ; Tue, 21 Apr 2026 15:01:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1776808903; x=1777413703; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=MQ6M8fMmd15EsU5jHzG0CofBcVEbXAB28EkZbWFSSyk=; b=qeJZKX+ltdv8jQjv8mBZioAmNZ/IjVuEXxyBjDQdV0ck831aMsjUpQP5WWJaDJfwVB yToOsq1k7ThOpkradQiQ8f3cmRDxlyIPCviZtisdVU8g0EH5fLPjuORMYGPXw1g8dHwa K+qWNnRJaAuf7k/VSp/Vyq4P+XejrdwdpXA1rk1modwCiETKGDJvzjWztTD53MWWPxn3 G8rST9Mil+NzhiP6sOIr/KabqLbV2M3ihYsZcpOFtc51iulkds/Dj6ZP6DvPsGIEw/QO Yo699OzFE8CFCoDN8qRE06huSRAE38AFlHtNnRxkP51GhKUvf/zQ7PsdrfcoLMLzJH2C UvAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776808903; x=1777413703; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MQ6M8fMmd15EsU5jHzG0CofBcVEbXAB28EkZbWFSSyk=; b=jMVCjSZQ4q7vQbzJUEyyNFMNKpyqHF3rKQzsDVfz4cc+gXzXyZtt0H3tqxQLk8Hka3 d9Nq3VFoYklqzh4GS80p6cKSH8ZRSptyeXU95IDmWgcaKn1PJurtUtI4bWuPcODwaKZa wj6SMtDGD9YlgiwZW6t3KWIqf94DSBuglZlRoBwFYMSJIxvpAfQbsEF+lNkP7sRZFvun hnzzTHP4i2obhl9+Lt+Si8XjePMuEXyhUA5mN9ts+XYtk6pIVm2j4NZRJfA8byF/Sh8M J8y/lH3rm+MtmaSVXES0YNESbSPpUaG/WVWovPNC66dtq3GUtGeMA72iEw8XwnpseQF4 +aGQ== X-Gm-Message-State: AOJu0Yw2IWkga2BTS8HQecM1+JJoIGgVz2YoKNXkzvjA42C7PDIMjXxn D0xHfR5YsFWofB84R8r4OVyJDYUH0nInTDFQZWxtLxpale88OWA8j/jDu7Iv+ev7dTZ7DJ0rm6j a4BZDQ8vdpH+PLywWFuQOByJx2vfoWxjPuIeIhL0vysS0bIIvn5RuLHm+YLEqR+uQJa+vAgzbVZ X49YQD2ZqdQWP6Ou4YeZN/zl9nmXyJa15C8buB4eMjpc0= X-Gm-Gg: AeBDietKXSzbC0A5llVMLevb3SHr/vvnowbPzWCq65bS5GxBRj9wGLDPUsVbyvYsrzx CTRxZdIw2UUBrhdGvEPiRfOnOvEP5hQKePVsTffLeHf34yn1n/9HaRHgNyeUS1zusBSsByEBJaW FnO8QjW3Mo/5aqOpKQgmhy1Zsleoh+rru3CR/zOUNCeJUNHeHX3F92CqfMqXzhEnoFscvq4sPxP 3EVE4gkwcN++2d53OFmaP0WaLw47+ReYKVs2JAn9A1HMauxgv2Sk6NDUbAkCYN6lWAPG2IAE715 k3cOTaVQrciTExYUz4l6y/SAXTQE8UiMl9mGv/Y5nTAFGLrwmrbqXy7yRkLcsiTq6GbDF03gxRH odQjDw0ybgusDjJIkslQVCnlkSZABLH3PacNmCR08MsloSzONfIQsDg== X-Received: by 2002:a05:600c:5294:b0:48a:563c:c8d6 with SMTP id 5b1f17b1804b1-48a563cd0eemr57313405e9.7.1776808903036; Tue, 21 Apr 2026 15:01:43 -0700 (PDT) X-Received: by 2002:a05:600c:5294:b0:48a:563c:c8d6 with SMTP id 5b1f17b1804b1-48a563cd0eemr57312985e9.7.1776808902531; Tue, 21 Apr 2026 15:01:42 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-21.inter.net.il. [80.230.25.21]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-43fe4cb1365sm42637228f8f.7.2026.04.21.15.01.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Apr 2026 15:01:42 -0700 (PDT) Date: Tue, 21 Apr 2026 18:01:39 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Muchun Song , Oscar Salvador , Hugh Dickins , Baolin Wang Subject: [PATCH RFC v3 09/19] mm: memfd: skip zeroing for zeroed hugetlb pool pages Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" gather_surplus_pages() pre-allocates hugetlb pages into the pool during mmap. Pass __GFP_ZERO so these pages are zeroed by the buddy allocator, and HPG_zeroed is set by alloc_surplus_hugetlb_folio. Add bool *zeroed output to alloc_hugetlb_folio_reserve() so callers can check whether the pool page is known-zero. memfd's memfd_alloc_folio() uses this to skip the explicit folio_zero_user() when the page is already zero. This avoids redundant zeroing for memfd hugetlb pages that were pre-allocated into the pool and never mapped to userspace. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/hugetlb.h | 6 ++++-- mm/hugetlb.c | 11 +++++++++-- mm/memfd.c | 17 +++++++++++------ 3 files changed, 24 insertions(+), 10 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 094714c607f9..93bb06a33f57 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -713,7 +713,8 @@ struct folio *alloc_hugetlb_folio_nodemask(struct hstat= e *h, int preferred_nid, nodemask_t *nmask, gfp_t gfp_mask, bool allow_alloc_fallback); struct folio *alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_= nid, - nodemask_t *nmask, gfp_t gfp_mask); + nodemask_t *nmask, gfp_t gfp_mask, + bool *zeroed); =20 int hugetlb_add_to_page_cache(struct folio *folio, struct address_space *m= apping, pgoff_t idx); @@ -1128,7 +1129,8 @@ static inline struct folio *alloc_hugetlb_folio(struc= t vm_area_struct *vma, =20 static inline struct folio * alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_nid, - nodemask_t *nmask, gfp_t gfp_mask) + nodemask_t *nmask, gfp_t gfp_mask, + bool *zeroed) { return NULL; } diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 4f0ed01f5b13..f02583b9faab 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -2241,7 +2241,7 @@ struct folio *alloc_buddy_hugetlb_folio_with_mpol(str= uct hstate *h, } =20 struct folio *alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_= nid, - nodemask_t *nmask, gfp_t gfp_mask) + nodemask_t *nmask, gfp_t gfp_mask, bool *zeroed) { struct folio *folio; =20 @@ -2257,6 +2257,12 @@ struct folio *alloc_hugetlb_folio_reserve(struct hst= ate *h, int preferred_nid, h->resv_huge_pages--; =20 spin_unlock_irq(&hugetlb_lock); + + if (zeroed && folio) { + *zeroed =3D folio_test_hugetlb_zeroed(folio); + folio_clear_hugetlb_zeroed(folio); + } + return folio; } =20 @@ -2341,7 +2347,8 @@ static int gather_surplus_pages(struct hstate *h, lon= g delta) * It is okay to use NUMA_NO_NODE because we use numa_mem_id() * down the road to pick the current node if that is the case. */ - folio =3D alloc_surplus_hugetlb_folio(h, htlb_alloc_mask(h), + folio =3D alloc_surplus_hugetlb_folio(h, + htlb_alloc_mask(h) | __GFP_ZERO, NUMA_NO_NODE, &alloc_nodemask, USER_ADDR_NONE); if (!folio) { diff --git a/mm/memfd.c b/mm/memfd.c index 919c2a53eb96..b9b44ed54db5 100644 --- a/mm/memfd.c +++ b/mm/memfd.c @@ -90,20 +90,24 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgo= ff_t idx) if (nr_resv < 0) return ERR_PTR(nr_resv); =20 + { + bool zeroed; + folio =3D alloc_hugetlb_folio_reserve(h, numa_node_id(), NULL, - gfp_mask); + gfp_mask, + &zeroed); if (folio) { u32 hash; =20 /* - * Zero the folio to prevent information leaks to userspace. - * Use folio_zero_user() which is optimized for huge/gigantic - * pages. Pass 0 as addr_hint since this is not a faulting path - * and we don't have a user virtual address yet. + * Zero the folio to prevent information leaks to + * userspace. Skip if the pool page is known-zero + * (HPG_zeroed set during pool pre-allocation). */ - folio_zero_user(folio, 0); + if (!zeroed) + folio_zero_user(folio, 0); =20 /* * Mark the folio uptodate before adding to page cache, @@ -139,6 +143,7 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgo= ff_t idx) hugetlb_unreserve_pages(inode, idx, idx + 1, 0); return ERR_PTR(err); } + } #endif return shmem_read_folio(memfd->f_mapping, idx); } --=20 MST From nobody Wed Jun 17 02:49:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A18DB26B2DA for ; Tue, 21 Apr 2026 22:01:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808916; cv=none; b=AQl7GBrMikPGiFMKybuJXZ102YpPjkoVDhfBIC3b9Xi2p8HnpM2Bes+ae+SvzQvfVdEZG19oNadoRTWNi87AuKmv7CarkezQV5GIYeQECmuLG5oQLwNA7IgwTXlm8mdZEMGCIuQbCesu+PN9v5R/pGwGhL/06xrgCz/1I/1D3aw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808916; c=relaxed/simple; bh=xyYx6EKfSNYgy8umDBVjFDGC9ANxj/5Tbt7GkFR38jI=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=kSbu7j+jQfUy+LwgussevICrPMtGjodhAwgFSAD4U5K8m6CkW7ZEq9zKfP/p4UUwLg64XV6XS8QY68gpVd6V6cf7hJ05LauAZGfCTAaebWTu1cLm7kq0H9/aNuHBFrA67AA9p8aZilamTswq6y5ZxT46MxaJeQv5bkN62XVItvs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=TVVSrmPU; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=VM8sk+fz; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="TVVSrmPU"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="VM8sk+fz" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1776808910; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=JVwG8S2po1iduREwk8io7PMDE8wG9YqvUAWHOi+D4qs=; b=TVVSrmPUWpOgnz9hK2o5obxxtYVua2LTSj6RnPBOFOK2CGGVYuCbkWMW2fKVhTsHUTRDqk jlQK5AdAA/7BPMFJPKzFkEUs0Igbr44M9TdFVvOk2LHBqcgaVpfLZHDQ1fRQMIvVo/Ik48 yQj4z767Qp/x77+2fLAmbBlfisg5GQw= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-225-AVB03p7JNN-KujiFg-ui4w-1; Tue, 21 Apr 2026 18:01:48 -0400 X-MC-Unique: AVB03p7JNN-KujiFg-ui4w-1 X-Mimecast-MFC-AGG-ID: AVB03p7JNN-KujiFg-ui4w_1776808907 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-488c2cc0cbaso31080285e9.3 for ; Tue, 21 Apr 2026 15:01:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1776808907; x=1777413707; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=JVwG8S2po1iduREwk8io7PMDE8wG9YqvUAWHOi+D4qs=; b=VM8sk+fzYFB9wZrk6YKEd38F5lC5fNsgZhgtGGi85s8KRi/uWx0lareBfmfdcVrWGe j8MjQrL+jyJkLq3CIaj5r/VNoHysPPxSnw+5eo1nUWCjGFl/fkfQA8xBQVbiNYuOaDA5 WzaVZoyY9ZNo5af5Z2SIrwOQvawrD/37UqhTY2z/UpI9pQGyE+269yGyrgYunGPO4MuW crRnYP9u/B3uSTAMyJeLi668q5D6UgEAJSAS4pIFi7An2eyI1ETJ5EKB0KS+PAY4105F wR1qdk5SBBRvgMFUqjHoRid+JdalxzXd+peb5SwkwzmNJ3Y0qQydN8V57HhRMonG/xuV 7Cpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776808907; x=1777413707; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JVwG8S2po1iduREwk8io7PMDE8wG9YqvUAWHOi+D4qs=; b=WkfwlCbq5hg5ZcaQPKInir2qvjSMD1iNex3wg2H1K5xoP3EoNcGC01O7ftIbaghxty YTH4wWanMydsgvBKuz8uAra/HBops7lT1EnBL7rGDf79mAxnM6MQn0Vpua/nqb/yWdr/ 2hydVDd3dVBFEvgMNjXkhCvwaPvd0gvISOqLZa0k6GvFOLvgiGq4AODB2NXWZRqQtjXc LTlbpcLoYV74KvHqqgzniEC9LrxDm/ES5YMeFtwR7F1Z/IZm4kXC256aU++h7QlgWF7c xPqJ0z9ZSYF0Hp2TZjLEXHCY29YboFfKk2HZY/HnRfKn4ZaLZvFeZ/uT5aZcoI5/5tIE KfeQ== X-Gm-Message-State: AOJu0Yy1CLU2lbvAZ+klgocqZ5f5pGupXkSYeTfUscmlNI5E0Nf3VJWG W7tBb7lZSgNdBlSCjuMph5jL2aDQkOo9ud+Z5AxwK6KX3AZMwnUBLdXjlX+hDZRIeXIzJ2d8rMJ AN5ZM1+tYJ85UTjIRmQu/LAHQE3+xsF4onlUq2W1otJURu1AaIvOnK96crIbYeReFzKNJtkFL2X EZtItKXYRduzGIuLAnKmTdLmsid9WOZzKgRRyWE+2Vpuk= X-Gm-Gg: AeBDietUnCn7jSDq/mib3vf1AAkGxzKQRMyAYx+tSFTayfWj5NCgEt1QhoIC214UkGU fsf7cCng8p2KHqwJM1H1c663we9K03qEMzZrpOPQmg3WF+MC/N0rZ+Spj+wxq2tl+vrk7tpkH7s qpTu3UXtLtMb4FM/96YLoFRK35lBmlqq1ni+NqW+IpZVQFB20SD36UecdGsYLss47cZcw6nG3UE jrRXAag1lh8/GV78uBJow1DHWvf4+2n1diIPYiS9D9LB71J3117IicG/TUfgMUzTmUCXv3m1KDu /IBTiYQJTTzUTTFJlu3uLI0t4oJi556lrSXUXXd8/TkX+Wz1Aef3hA2SdLlAQijxBf7BzD/zRF/ /7Ob2a98B8OsvrVgJnb+h7vEQtCwVNdulLJCvbL9+zYzHXBrjrLA+IQ== X-Received: by 2002:a05:600c:5294:b0:48a:563c:c8d6 with SMTP id 5b1f17b1804b1-48a563cd0eemr57316175e9.7.1776808906943; Tue, 21 Apr 2026 15:01:46 -0700 (PDT) X-Received: by 2002:a05:600c:5294:b0:48a:563c:c8d6 with SMTP id 5b1f17b1804b1-48a563cd0eemr57315565e9.7.1776808906442; Tue, 21 Apr 2026 15:01:46 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-21.inter.net.il. [80.230.25.21]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-488fc0f8193sm361473545e9.1.2026.04.21.15.01.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Apr 2026 15:01:45 -0700 (PDT) Date: Tue, 21 Apr 2026 18:01:42 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Richard Henderson , Matt Turner , Magnus Lindholm , Greg Ungerer , Geert Uytterhoeven , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , linux-alpha@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-s390@vger.kernel.org Subject: [PATCH RFC v3 10/19] mm: remove arch vma_alloc_zeroed_movable_folio overrides Message-ID: <006f9142e591ba8c340c3b354aee76aec5c285b9.1776808210.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that the generic vma_alloc_zeroed_movable_folio() uses __GFP_ZERO, the arch-specific macros on alpha, m68k, s390, and x86 that did the same thing are redundant. Remove them. arm64 is not affected: it has a real function override that handles MTE tag zeroing, not just __GFP_ZERO. Suggested-by: David Hildenbrand Signed-off-by: Michael S. Tsirkin Acked-by: Geert Uytterhoeven # m68k Acked-by: Greg Ungerer Acked-by: Magnus Lindholm --- arch/alpha/include/asm/page.h | 3 --- arch/m68k/include/asm/page_no.h | 3 --- arch/s390/include/asm/page.h | 3 --- arch/x86/include/asm/page.h | 3 --- 4 files changed, 12 deletions(-) diff --git a/arch/alpha/include/asm/page.h b/arch/alpha/include/asm/page.h index 59d01f9b77f6..4327029cd660 100644 --- a/arch/alpha/include/asm/page.h +++ b/arch/alpha/include/asm/page.h @@ -12,9 +12,6 @@ =20 extern void clear_page(void *page); =20 -#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ - vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr) - extern void copy_page(void * _to, void * _from); #define copy_user_page(to, from, vaddr, pg) copy_page(to, from) =20 diff --git a/arch/m68k/include/asm/page_no.h b/arch/m68k/include/asm/page_n= o.h index d2532bc407ef..f511b763a235 100644 --- a/arch/m68k/include/asm/page_no.h +++ b/arch/m68k/include/asm/page_no.h @@ -12,9 +12,6 @@ extern unsigned long memory_end; =20 #define copy_user_page(to, from, vaddr, pg) copy_page(to, from) =20 -#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ - vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr) - #define __pa(vaddr) ((unsigned long)(vaddr)) #define __va(paddr) ((void *)((unsigned long)(paddr))) =20 diff --git a/arch/s390/include/asm/page.h b/arch/s390/include/asm/page.h index f339258135f7..04020a19a5cf 100644 --- a/arch/s390/include/asm/page.h +++ b/arch/s390/include/asm/page.h @@ -67,9 +67,6 @@ static inline void copy_page(void *to, void *from) =20 #define copy_user_page(to, from, vaddr, pg) copy_page(to, from) =20 -#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ - vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr) - #ifdef CONFIG_STRICT_MM_TYPECHECKS #define STRICT_MM_TYPECHECKS #endif diff --git a/arch/x86/include/asm/page.h b/arch/x86/include/asm/page.h index 416dc88e35c1..92fa975b46f3 100644 --- a/arch/x86/include/asm/page.h +++ b/arch/x86/include/asm/page.h @@ -28,9 +28,6 @@ static inline void copy_user_page(void *to, void *from, u= nsigned long vaddr, copy_page(to, from); } =20 -#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ - vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr) - #ifndef __pa #define __pa(x) __phys_addr((unsigned long)(x)) #endif --=20 MST From nobody Wed Jun 17 02:49:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4B3A3280CD5 for ; Tue, 21 Apr 2026 22:01:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808916; cv=none; b=XtwcekTtbdyRagldBAy57vZmL6d5PMzf43kMtCSPwT+cwVjeJ6ce7w+iGFeUsHvCQsCERIX3/Hyfd5WsEkoRy+/uZfMood98m5qIxc8kuHWKhklmz4g48H1TrtE9YwkKwxlXch+02NEP6glUpWG2zrwAA3sdYvLm8+Ka9YwR6Ds= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808916; c=relaxed/simple; bh=sDo0b8xddYVEGpaxJMvmIOo2W7GKA24+V8mFvrgK7XQ=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=BRHttwbIBpBvdimLzQok9MjZ8RxQcwg9idEvbkPhN2Y0GwkI+Q5yZhMXNrGOA5cYdPVPZ7NgNNmwvfidULCEsyWD+lbK/feX1iDOLl6Y5JusDCRBgt4lNaw4u1ygbqzDDEQlgEN5wiqqmJ6K9pbElyxXDmPZJOTte15Gto2x3/k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=H/bqEOVf; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=Noq0WcU1; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="H/bqEOVf"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="Noq0WcU1" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1776808914; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=iXNCfNPQyocyWkSCOxjnq/WqPhuX3fxPK4FCFhhbcx8=; b=H/bqEOVfaL1WAPA7faZTBr+Yrrq6SAPRCT6tF89++b6wrodyJnsiD79I7AN/jLlqq2c/jH HicscYZfhJHpLtPxCe3jqRPVXRHQkxvInzDOaq4g9p0BOKOy0S1KPMV2cHuCkuQIA/2ZG+ cARpXVBBG1XGPylwNspqxGYxX3vZyTA= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-395--WewoLBgPBOe1Km6Gz3eQw-1; Tue, 21 Apr 2026 18:01:52 -0400 X-MC-Unique: -WewoLBgPBOe1Km6Gz3eQw-1 X-Mimecast-MFC-AGG-ID: -WewoLBgPBOe1Km6Gz3eQw_1776808911 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-440d0d7872aso2251345f8f.2 for ; Tue, 21 Apr 2026 15:01:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1776808911; x=1777413711; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=iXNCfNPQyocyWkSCOxjnq/WqPhuX3fxPK4FCFhhbcx8=; b=Noq0WcU1rIwmZGSzOTEphpBQHauka5o1YK0bfW2eJncn0c9ustOUmRMJfeZ3Utp8xa 4Q1F9ygvvkuT7yhbYKfTTh7exhSNAFgCWCrKrHZyFzdX3fip4O2dM0EQFDT1jLXOqMu5 MJHkRrDeKyM75jFwHb8zD6tvAPA2gLN2msGIF096xY47h9OOsOGx5VUagIogQQj1uiQ1 MDqNyvGtB/s7cUV83T5C9NIRGOhRo3Bhy820WeJykK7O70Cjg7bme9hyQ2wjHrHjirf1 Te3jxwSfIEmgSPinAZCC5yUf3Qt7JFc4TyVdiCO7Fi+XoNFgig+QtCcRQKqqaGLxeyrM hKEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776808911; x=1777413711; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iXNCfNPQyocyWkSCOxjnq/WqPhuX3fxPK4FCFhhbcx8=; b=kkPXSwWmUXWkJcTBbSgQALEy+ZJ3ugPQsbm2EbQYkJtEd99OEQ0XWJed6VBl1diHyc f6IraLTdmpstOqIe7ncnMEb24RzExAOfKrcD/ATSPXwiuRpfGvdL9N+pyebPxOi0pfHo uggCUFsEQ5wuAZVO01OQT1988C6G76oUFb2/CnUITN86w3RjJ3cvzxMa0Eep/3ixdgr/ jnYJyr3O8vkXqHjmSUAe33PktJctAMeQEfBD/nxOMLrWO4nZOHaCC3dnhD96+7y8uzLL xDetaFdytv6E8obksZZD8V+arsngtPmmf6r+cFrzEYkbGntGm1oJR8b6a8dqXZkIL9Cz pm1w== X-Gm-Message-State: AOJu0YzRLYGIerYciLIMezIa5qhPN0zHxTLtvXLz6M474DK4UNUvj1LZ A6s1kEptITvp9d9iYZ32TTPH4Lq0em3bHKQO8PgBrgIR6chsBr4z/OA46SWoNixnfNN6C1C4oML 3JR9jZFrI4CjUPED3wJOunu4tbV3+H+xdxNB3YkPzNKSysiLzGk1wabquQ/6xWPN3VPb62+WSAT xvwbkIQPc/f0KJwvftaOCcExnvGtXBBjvmyI40VT/vZFM= X-Gm-Gg: AeBDies9AOu3M5rqyZeQCGwn0MgTxgUD8ACzBy1n89j3gXRkq4iiMln6JRlnejVOQl0 Mu+QUKfhMCIBCZ84bD5VzfWQFQ7T20fFetyhD/w5+E2OfXQ5vhTy1Cxw4qUwtByNfIaBNOkMipB xlfpwyOyhUrzWKMOzZqz5YQs8WrVWYS6aogvJLArrmtohnxooAeDq3zBROptsEEIQPnX0x1Jb3e hHFcJlZUk4PwIR/t3nqT46XMVVdbsLwDngWlJnwLTKEdcI2hjhRlk7Yhv0MTHUjCpuHBHZqAcs6 C/2dWGc5ZgJMGYhFDR996bkMA3StJ2nV396Uu0GNu6IXA9RZAN+prVmKq259Q38sVneGmOWZa17 CCmwF9H5v869TcnnMJ9G6X/xiyyXfGiHsmc3bZDHk86n5riIZ1jTEPA== X-Received: by 2002:a05:6000:2481:b0:43d:77a8:3bb6 with SMTP id ffacd0b85a97d-43fe3e239d9mr31452792f8f.47.1776808910753; Tue, 21 Apr 2026 15:01:50 -0700 (PDT) X-Received: by 2002:a05:6000:2481:b0:43d:77a8:3bb6 with SMTP id ffacd0b85a97d-43fe3e239d9mr31452682f8f.47.1776808909583; Tue, 21 Apr 2026 15:01:49 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-21.inter.net.il. [80.230.25.21]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-43fe4cc375dsm36898455f8f.14.2026.04.21.15.01.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Apr 2026 15:01:48 -0700 (PDT) Date: Tue, 21 Apr 2026 18:01:46 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Johannes Weiner , Zi Yan Subject: [PATCH RFC v3 11/19] mm: page_alloc: propagate PageReported flag across buddy splits Message-ID: <7d1337e364980896be012ab34fd26bb3bf82c7af.1776808210.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When a reported free page is split via expand() to satisfy a smaller allocation, the sub-pages placed back on the free lists lose the PageReported flag. This means they will be unnecessarily re-reported to the hypervisor in the next reporting cycle, wasting work. Propagate the PageReported flag to sub-pages during expand() so that they are recognized as already-reported. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/page_alloc.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index db2192ffc27c..211e9e32b91d 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1730,7 +1730,7 @@ struct page *__pageblock_pfn_to_page(unsigned long st= art_pfn, * -- nyc */ static inline unsigned int expand(struct zone *zone, struct page *page, in= t low, - int high, int migratetype) + int high, int migratetype, bool reported) { unsigned int size =3D 1 << high; unsigned int nr_added =3D 0; @@ -1752,6 +1752,15 @@ static inline unsigned int expand(struct zone *zone,= struct page *page, int low, __add_to_free_list(&page[size], zone, high, migratetype, false); set_buddy_order(&page[size], high); nr_added +=3D size; + + /* + * The parent page has been reported to the host. The + * sub-pages are part of the same reported block, so mark + * them reported too. This avoids re-reporting pages that + * the host already knows about. + */ + if (reported) + __SetPageReported(&page[size]); } =20 return nr_added; @@ -1762,9 +1771,10 @@ static __always_inline void page_del_and_expand(stru= ct zone *zone, int high, int migratetype) { int nr_pages =3D 1 << high; + bool was_reported =3D page_reported(page); =20 __del_page_from_free_list(page, zone, high, migratetype); - nr_pages -=3D expand(zone, page, low, high, migratetype); + nr_pages -=3D expand(zone, page, low, high, migratetype, was_reported); account_freepages(zone, -nr_pages, migratetype); } =20 @@ -2334,7 +2344,8 @@ try_to_claim_block(struct zone *zone, struct page *pa= ge, =20 del_page_from_free_list(page, zone, current_order, block_type); change_pageblock_range(page, current_order, start_type); - nr_added =3D expand(zone, page, order, current_order, start_type); + nr_added =3D expand(zone, page, order, current_order, start_type, + false); account_freepages(zone, nr_added, start_type); return page; } --=20 MST From nobody Wed Jun 17 02:49:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8C2463876AF for ; Tue, 21 Apr 2026 22:01:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808920; cv=none; b=eXW9esFtOv8n/ROQ5puZ/4waXkO5pYBk1eDT3htSSKaqgebyl5Zp1im8Ive+t7t7AtlgYaCA38iX/Y7jG9iY+nFVykLn/qajunUsyfmf48myyl5Xvnh9aZDSlxXMednm6r9uRdhlcsdxx/yqK0Za4MT0sbHo1LszKyaJoHy817o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808920; c=relaxed/simple; bh=LJJEeh3Kra/C2pMpQ4Gn9UIYlxvxiINCp1ujoGQTHGQ=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Fqyd0yo1zT0QshfRII+Bg9TWiFeFJUI3EVcjrzuFbhBWG8IQGo8t6KvKzFUjfgaK8CthNf4DoWcyo0izewWFpOworgKTyQncurpy2BnO3AEuYsKALJ5Sgcv9YecpZuvWCMR8eZ8fgJrz9Zwh9O/Y69JFQfbqTU4DmsXDFbNh47k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Oe/5ecou; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=haIzvpWL; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Oe/5ecou"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="haIzvpWL" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1776808917; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=kkE7kXfqxv7y8uiwZ1OnT5X0E9fFqwrxFGyVrifZzZU=; b=Oe/5ecoulQYXx7GnMaDQNSzTfcXsTKIJFjKwxqsDBaQcsHDbJUA+av0CqF+Q9GtXBqwR/s 9rf5s8UwUBjC3JNFQVlQMaD+7MZKNeAFDUlPGca3ucAl6tGEj4MC1TTs+8GMT6NXrEUCu2 Dh1D0OclsYy7QJdIbgCzyRn79/RgPS8= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-12-8cAoOsQ7OCKp2tUkePyIww-1; Tue, 21 Apr 2026 18:01:56 -0400 X-MC-Unique: 8cAoOsQ7OCKp2tUkePyIww-1 X-Mimecast-MFC-AGG-ID: 8cAoOsQ7OCKp2tUkePyIww_1776808915 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-486fa07f2bbso35931485e9.2 for ; Tue, 21 Apr 2026 15:01:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1776808914; x=1777413714; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=kkE7kXfqxv7y8uiwZ1OnT5X0E9fFqwrxFGyVrifZzZU=; b=haIzvpWLILrsZfxNJi8jUvUOAysZRFIGuOD9sIypp7FQpPvqmV8ZHw84k88/4hmM6N 7t6Ss3Ydxsfv/eBzyw9FTlg0g8V8r43PAX2lanpSklXVK3K7V5O/wc/OSXa6mzxJqy3I MgeHRBgj7/wRvvrP7aV92Ju8Wd2hj89ABlh5nBka+t97c7Ta+wjv4/1kAXMgvTjUC1N2 L9AQB4yRxB+TrclNu69ovNY5Ol7R4OJKmbtoo94TQtt5FSnCxoaDrPC8hIWya8riR/ab wprfPGNgdwWNc8vEvlYBTMnic6LRhXv67A0Qpm6aH353X2rbHBS1hKEpKDZq01pDnFox ZaEg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776808914; x=1777413714; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kkE7kXfqxv7y8uiwZ1OnT5X0E9fFqwrxFGyVrifZzZU=; b=nipiPAp/eeb4vzqhSVRKpESAW6T8U0D3W9NNFnCPtAgxEkLkcr3cBCiKMlHHMz8J8Q UH7GQWTdPLdYprK2z21zcQGt07wfgnGvo82gqB0wILRd/BY8cJQnZd58cezJxFXFNRcl 32dA1Z61nFHpavzWOi4ZJIYwWagBN776ZFlIj0eJerL8mKjUep16jr5XNgWLo2+ok8kq foPhrtR6Fb3yvltIkEp1Yao/zbqUrlGsvkS2lTN2iUT/MNEiwUWBbPLkO3yqxCb3J0sb EuUW8W9IPHjJPE7Xrxp56nJnEkgEfUwEPoYwT2HVqlCX8S3YptbdV/UJi4yyLrTDTO6W Pz2g== X-Gm-Message-State: AOJu0YzNPf1jSJmhmDUHhKbN+K3ghmkytMjuNnKn9B2XP0fYP9Z2f66X xeB2oK49FijUFhQRbNgTGqnQK0eutcT0wdjw4DqDAORumBxsCqWpNAbILfesk8ikdXhGqcCBVvg XozWhjB52W6zMgLsxEVRVReWDYple2gksGO+dhTkc6NtlSsteOy04VX2KcyJOpZAqnxwSt9a39x Wntha8EK3vyKvqT/IAlPPj4oPYjWTb9IT993G6FsnTuZg= X-Gm-Gg: AeBDiet1lvSz77ZKUgKQlMt3bXGDx82KpAAdOw1B3JutQEmzlgDOCjipGs1+30dhuYl UX85QM94ZF9xEZJqQNu/iILois9P9M4cdJnbFoAg/ySPMr1nc5aiCKVLjK/14/j+swPHxTwtRA8 WodcTWKnAy1uUBEP83L230jyJ3Ot3HYNPEgnaHEmMYgK0kd496c8Ca61A7czKEJ+xklbjHqS5BV 8DZ+JjSTdpbWYy34Zqde/eWYJ67lx5ilCT+VTvzQAS7IZpkVYFcpX+OWy6uEQ+8MRWqfUZ2oxAo XWAT4YvEMYyAieLfDxSLneK8y/JZfxsuRqzqzOt8cAD6Pi0tajtWyqkSZAVRUjOmZvoYNd5EcJm eYl89LC0/+OBQMdxV1Foi3A1fBD0bKstcaapkxIbBxA6asB7ZRdrvWA== X-Received: by 2002:a05:600c:8b84:b0:480:1c69:9d36 with SMTP id 5b1f17b1804b1-488fb76e4aamr310252335e9.17.1776808913962; Tue, 21 Apr 2026 15:01:53 -0700 (PDT) X-Received: by 2002:a05:600c:8b84:b0:480:1c69:9d36 with SMTP id 5b1f17b1804b1-488fb76e4aamr310251835e9.17.1776808913341; Tue, 21 Apr 2026 15:01:53 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-21.inter.net.il. [80.230.25.21]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-488fc1c01cfsm410290095e9.10.2026.04.21.15.01.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Apr 2026 15:01:52 -0700 (PDT) Date: Tue, 21 Apr 2026 18:01:50 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Lorenzo Stoakes , "Liam R. Howlett" , Mike Rapoport , Johannes Weiner , Zi Yan Subject: [PATCH RFC v3 12/19] mm: page_reporting: skip redundant zeroing of host-zeroed reported pages Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When a guest reports free pages to the hypervisor via the page reporting framework (used by virtio-balloon and hv_balloon), the host typically zeros those pages when reclaiming their backing memory. However, when those pages are later allocated in the guest, post_alloc_hook() unconditionally zeros them again if __GFP_ZERO is set. This double-zeroing is wasteful, especially for large pages. Avoid redundant zeroing: - Add a host_zeroes_pages flag to page_reporting_dev_info, allowing drivers to declare that their host zeros reported pages on reclaim. A static key (page_reporting_host_zeroes) gates the fast path. - Add PG_zeroed page flag (sharing PG_private bit) to mark pages that have been zeroed by the host. Set it on reported pages during allocation from the buddy in page_del_and_expand(). - Thread the zeroed bool through rmqueue -> prep_new_page -> post_alloc_hook, where it skips redundant zeroing for __GFP_ZERO allocations. No driver sets host_zeroes_pages yet; a follow-up patch to virtio_balloon is needed to opt in. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- include/linux/mm.h | 28 +++++++++++++++++ include/linux/page-flags.h | 12 ++++++- include/linux/page_reporting.h | 3 ++ mm/compaction.c | 5 +-- mm/internal.h | 2 +- mm/page_alloc.c | 57 ++++++++++++++++++++++------------ mm/page_reporting.c | 14 ++++++++- mm/page_reporting.h | 12 +++++++ 8 files changed, 108 insertions(+), 25 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 541d36e5e420..821034dd33d1 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -4817,6 +4817,34 @@ static inline bool user_alloc_needs_zeroing(void) &init_on_alloc); } =20 +/** + * __page_test_clear_zeroed - test and clear the zeroed marker. + * @page: the page to test. + * + * Returns true if the page was zeroed by the host, and clears + * the marker. Caller must have exclusive access to @page. + */ +static inline bool __page_test_clear_zeroed(struct page *page) +{ + if (PageZeroed(page)) { + __ClearPageZeroed(page); + return true; + } + return false; +} + +/** + * folio_test_clear_zeroed - test and clear the zeroed marker. + * @folio: the folio to test. + * + * Returns true if the folio was zeroed by the host, and clears + * the marker. Callers can skip their own zeroing. + */ +static inline bool folio_test_clear_zeroed(struct folio *folio) +{ + return __page_test_clear_zeroed(&folio->page); +} + int arch_get_shadow_stack_status(struct task_struct *t, unsigned long __us= er *status); int arch_set_shadow_stack_status(struct task_struct *t, unsigned long stat= us); int arch_lock_shadow_stack_status(struct task_struct *t, unsigned long sta= tus); diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index f7a0e4af0c73..aa0de99247d4 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -135,6 +135,8 @@ enum pageflags { PG_swapcache =3D PG_owner_priv_1, /* Swap page: swp_entry_t in private */ /* Some filesystems */ PG_checked =3D PG_owner_priv_1, + /* Page contents are known to be zero */ + PG_zeroed =3D PG_private, =20 /* * Depending on the way an anonymous folio can be mapped into a page @@ -679,6 +681,13 @@ FOLIO_TEST_CLEAR_FLAG_FALSE(young) FOLIO_FLAG_FALSE(idle) #endif =20 +/* + * PageZeroed() tracks pages known to be zero. The allocator + * uses this to skip redundant zeroing in post_alloc_hook(). + */ +__PAGEFLAG(Zeroed, zeroed, PF_NO_COMPOUND) +#define __PG_ZEROED (1UL << PG_zeroed) + /* * PageReported() is used to track reported free pages within the Buddy * allocator. We can use the non-atomic version of the test and set @@ -1207,9 +1216,10 @@ static __always_inline void __ClearPageAnonExclusive= (struct page *page) * * __PG_HWPOISON is exceptional because it needs to be kept beyond page's * alloc-free cycle to prevent from reusing the page. + * __PG_ZEROED survives alloc-free cycles to track known-zero pages. */ #define PAGE_FLAGS_CHECK_AT_PREP \ - ((PAGEFLAGS_MASK & ~__PG_HWPOISON) | LRU_GEN_MASK | LRU_REFS_MASK) + ((PAGEFLAGS_MASK & ~(__PG_HWPOISON | __PG_ZEROED)) | LRU_GEN_MASK | LRU_R= EFS_MASK) =20 /* * Flags stored in the second page of a compound page. They may overlap diff --git a/include/linux/page_reporting.h b/include/linux/page_reporting.h index fe648dfa3a7c..10faadfeb4fb 100644 --- a/include/linux/page_reporting.h +++ b/include/linux/page_reporting.h @@ -13,6 +13,9 @@ struct page_reporting_dev_info { int (*report)(struct page_reporting_dev_info *prdev, struct scatterlist *sg, unsigned int nents); =20 + /* If true, host zeros reported pages on reclaim */ + bool host_zeroes_pages; + /* work struct for processing reports */ struct delayed_work work; =20 diff --git a/mm/compaction.c b/mm/compaction.c index 82f2914962f5..3d9ae727a98a 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -82,7 +82,8 @@ static inline bool is_via_compact_memory(int order) { ret= urn false; } =20 static struct page *mark_allocated_noprof(struct page *page, unsigned int = order, gfp_t gfp_flags) { - post_alloc_hook(page, order, __GFP_MOVABLE, USER_ADDR_NONE) + post_alloc_hook(page, order, __GFP_MOVABLE, false, USER_ADDR_NONE); + set_page_refcounted(page); return page; } #define mark_allocated(...) alloc_hooks(mark_allocated_noprof(__VA_ARGS__)) @@ -1831,7 +1832,7 @@ static struct folio *compaction_alloc_noprof(struct f= olio *src, unsigned long da set_page_private(&freepage[size], start_order); } dst =3D (struct folio *)freepage; - post_alloc_hook(&dst->page, order, __GFP_MOVABLE, USER_ADDR_NONE); + post_alloc_hook(&dst->page, order, __GFP_MOVABLE, false, USER_ADDR_NONE); set_page_refcounted(&dst->page); if (order) prep_compound_page(&dst->page, order); diff --git a/mm/internal.h b/mm/internal.h index 0b9c0bd133d3..4c33249e03f0 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -889,7 +889,7 @@ static inline void prep_compound_tail(struct page *head= , int tail_idx) } =20 void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flag= s, - unsigned long user_addr); + bool zeroed, unsigned long user_addr); extern bool free_pages_prepare(struct page *page, unsigned int order); =20 extern int user_min_free_kbytes; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 211e9e32b91d..2098d569d80c 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1774,6 +1774,7 @@ static __always_inline void page_del_and_expand(struc= t zone *zone, bool was_reported =3D page_reported(page); =20 __del_page_from_free_list(page, zone, high, migratetype); + nr_pages -=3D expand(zone, page, low, high, migratetype, was_reported); account_freepages(zone, -nr_pages, migratetype); } @@ -1846,8 +1847,10 @@ static inline bool should_skip_init(gfp_t flags) return (flags & __GFP_SKIP_ZERO); } =20 + inline void post_alloc_hook(struct page *page, unsigned int order, - gfp_t gfp_flags, unsigned long user_addr) + gfp_t gfp_flags, bool zeroed, + unsigned long user_addr) { bool init =3D !want_init_on_free() && want_init_on_alloc(gfp_flags) && !should_skip_init(gfp_flags); @@ -1856,6 +1859,14 @@ inline void post_alloc_hook(struct page *page, unsig= ned int order, =20 set_page_private(page, 0); =20 + /* + * If the page is zeroed, skip memory initialization. + * We still need to handle tag zeroing separately since the host + * does not know about memory tags. + */ + if (zeroed && init && !zero_tags) + init =3D false; + arch_alloc_page(page, order); debug_pagealloc_map_pages(page, 1 << order); =20 @@ -1913,13 +1924,13 @@ inline void post_alloc_hook(struct page *page, unsi= gned int order, } =20 static void prep_new_page(struct page *page, unsigned int order, gfp_t gfp= _flags, - unsigned int alloc_flags, - unsigned long user_addr) + unsigned int alloc_flags, bool zeroed, + unsigned long user_addr) { if (order && (gfp_flags & __GFP_COMP)) prep_compound_page(page, order); =20 - post_alloc_hook(page, order, gfp_flags, user_addr); + post_alloc_hook(page, order, gfp_flags, zeroed, user_addr); =20 /* * page is set pfmemalloc when ALLOC_NO_WATERMARKS was necessary to @@ -3261,7 +3272,7 @@ static inline void zone_statistics(struct zone *prefe= rred_zone, struct zone *z, static __always_inline struct page *rmqueue_buddy(struct zone *preferred_zone, struct zone *zone, unsigned int order, unsigned int alloc_flags, - int migratetype) + int migratetype, bool *zeroed) { struct page *page; unsigned long flags; @@ -3296,6 +3307,7 @@ struct page *rmqueue_buddy(struct zone *preferred_zon= e, struct zone *zone, } } spin_unlock_irqrestore(&zone->lock, flags); + *zeroed =3D __page_test_clear_zeroed(page); } while (check_new_pages(page, order)); =20 __count_zid_vm_events(PGALLOC, page_zonenum(page), 1 << order); @@ -3357,10 +3369,9 @@ static int nr_pcp_alloc(struct per_cpu_pages *pcp, s= truct zone *zone, int order) /* Remove page from the per-cpu list, caller must protect the list */ static inline struct page *__rmqueue_pcplist(struct zone *zone, unsigned int order, - int migratetype, - unsigned int alloc_flags, + int migratetype, unsigned int alloc_flags, struct per_cpu_pages *pcp, - struct list_head *list) + struct list_head *list, bool *zeroed) { struct page *page; =20 @@ -3381,6 +3392,7 @@ struct page *__rmqueue_pcplist(struct zone *zone, uns= igned int order, page =3D list_first_entry(list, struct page, pcp_list); list_del(&page->pcp_list); pcp->count -=3D 1 << order; + *zeroed =3D __page_test_clear_zeroed(page); } while (check_new_pages(page, order)); =20 return page; @@ -3389,7 +3401,8 @@ struct page *__rmqueue_pcplist(struct zone *zone, uns= igned int order, /* Lock and remove page from the per-cpu list */ static struct page *rmqueue_pcplist(struct zone *preferred_zone, struct zone *zone, unsigned int order, - int migratetype, unsigned int alloc_flags) + int migratetype, unsigned int alloc_flags, + bool *zeroed) { struct per_cpu_pages *pcp; struct list_head *list; @@ -3408,7 +3421,8 @@ static struct page *rmqueue_pcplist(struct zone *pref= erred_zone, */ pcp->free_count >>=3D 1; list =3D &pcp->lists[order_to_pindex(migratetype, order)]; - page =3D __rmqueue_pcplist(zone, order, migratetype, alloc_flags, pcp, li= st); + page =3D __rmqueue_pcplist(zone, order, migratetype, alloc_flags, + pcp, list, zeroed); pcp_spin_unlock(pcp, UP_flags); if (page) { __count_zid_vm_events(PGALLOC, page_zonenum(page), 1 << order); @@ -3433,19 +3447,19 @@ static inline struct page *rmqueue(struct zone *preferred_zone, struct zone *zone, unsigned int order, gfp_t gfp_flags, unsigned int alloc_flags, - int migratetype) + int migratetype, bool *zeroed) { struct page *page; =20 if (likely(pcp_allowed_order(order))) { page =3D rmqueue_pcplist(preferred_zone, zone, order, - migratetype, alloc_flags); + migratetype, alloc_flags, zeroed); if (likely(page)) goto out; } =20 page =3D rmqueue_buddy(preferred_zone, zone, order, alloc_flags, - migratetype); + migratetype, zeroed); =20 out: /* Separate test+clear to avoid unnecessary atomics */ @@ -3836,6 +3850,7 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int o= rder, int alloc_flags, struct pglist_data *last_pgdat =3D NULL; bool last_pgdat_dirty_ok =3D false; bool no_fallback; + bool zeroed; bool skip_kswapd_nodes =3D nr_online_nodes > 1; bool skipped_kswapd_nodes =3D false; =20 @@ -3980,10 +3995,11 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int= order, int alloc_flags, =20 try_this_zone: page =3D rmqueue(zonelist_zone(ac->preferred_zoneref), zone, order, - gfp_mask, alloc_flags, ac->migratetype); + gfp_mask, alloc_flags, ac->migratetype, + &zeroed); if (page) { prep_new_page(page, order, gfp_mask, alloc_flags, - ac->user_addr); + zeroed, ac->user_addr); =20 /* * If this is a high-order atomic allocation then check @@ -4218,7 +4234,7 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned= int order, =20 /* Prep a captured page if available */ if (page) - prep_new_page(page, order, gfp_mask, alloc_flags, + prep_new_page(page, order, gfp_mask, alloc_flags, false, ac->user_addr); =20 /* Try get a page from the freelist if available */ @@ -5193,6 +5209,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, /* Attempt the batch allocation */ pcp_list =3D &pcp->lists[order_to_pindex(ac.migratetype, 0)]; while (nr_populated < nr_pages) { + bool zeroed =3D false; =20 /* Skip existing pages */ if (page_array[nr_populated]) { @@ -5201,7 +5218,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, } =20 page =3D __rmqueue_pcplist(zone, 0, ac.migratetype, alloc_flags, - pcp, pcp_list); + pcp, pcp_list, &zeroed); if (unlikely(!page)) { /* Try and allocate at least one page */ if (!nr_account) { @@ -5212,7 +5229,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, } nr_account++; =20 - prep_new_page(page, 0, gfp, 0, USER_ADDR_NONE); + prep_new_page(page, 0, gfp, 0, zeroed, USER_ADDR_NONE); set_page_refcounted(page); page_array[nr_populated++] =3D page; } @@ -6938,7 +6955,7 @@ static void split_free_frozen_pages(struct list_head = *list, gfp_t gfp_mask) list_for_each_entry_safe(page, next, &list[order], lru) { int i; =20 - post_alloc_hook(page, order, gfp_mask, USER_ADDR_NONE); + post_alloc_hook(page, order, gfp_mask, false, USER_ADDR_NONE); if (!order) continue; =20 @@ -7144,7 +7161,7 @@ int alloc_contig_frozen_range_noprof(unsigned long st= art, unsigned long end, struct page *head =3D pfn_to_page(start); =20 check_new_pages(head, order); - prep_new_page(head, order, gfp_mask, 0, USER_ADDR_NONE); + prep_new_page(head, order, gfp_mask, 0, false, USER_ADDR_NONE); } else { ret =3D -EINVAL; WARN(true, "PFN range: requested [%lu, %lu), allocated [%lu, %lu)\n", diff --git a/mm/page_reporting.c b/mm/page_reporting.c index f0042d5743af..6177d2413743 100644 --- a/mm/page_reporting.c +++ b/mm/page_reporting.c @@ -50,6 +50,8 @@ EXPORT_SYMBOL_GPL(page_reporting_order); #define PAGE_REPORTING_DELAY (2 * HZ) static struct page_reporting_dev_info __rcu *pr_dev_info __read_mostly; =20 +DEFINE_STATIC_KEY_FALSE(page_reporting_host_zeroes); + enum { PAGE_REPORTING_IDLE =3D 0, PAGE_REPORTING_REQUESTED, @@ -129,8 +131,11 @@ page_reporting_drain(struct page_reporting_dev_info *p= rdev, * report on the new larger page when we make our way * up to that higher order. */ - if (PageBuddy(page) && buddy_order(page) =3D=3D order) + if (PageBuddy(page) && buddy_order(page) =3D=3D order) { __SetPageReported(page); + if (page_reporting_host_zeroes_pages()) + __SetPageZeroed(page); + } } while ((sg =3D sg_next(sg))); =20 /* reinitialize scatterlist now that it is empty */ @@ -386,6 +391,10 @@ int page_reporting_register(struct page_reporting_dev_= info *prdev) /* Assign device to allow notifications */ rcu_assign_pointer(pr_dev_info, prdev); =20 + /* enable zeroed page optimization if host zeroes reported pages */ + if (prdev->host_zeroes_pages) + static_branch_enable(&page_reporting_host_zeroes); + /* enable page reporting notification */ if (!static_key_enabled(&page_reporting_enabled)) { static_branch_enable(&page_reporting_enabled); @@ -410,6 +419,9 @@ void page_reporting_unregister(struct page_reporting_de= v_info *prdev) =20 /* Flush any existing work, and lock it out */ cancel_delayed_work_sync(&prdev->work); + + if (prdev->host_zeroes_pages) + static_branch_disable(&page_reporting_host_zeroes); } =20 mutex_unlock(&page_reporting_mutex); diff --git a/mm/page_reporting.h b/mm/page_reporting.h index c51dbc228b94..736ea7b37e9e 100644 --- a/mm/page_reporting.h +++ b/mm/page_reporting.h @@ -15,6 +15,13 @@ DECLARE_STATIC_KEY_FALSE(page_reporting_enabled); extern unsigned int page_reporting_order; void __page_reporting_notify(void); =20 +DECLARE_STATIC_KEY_FALSE(page_reporting_host_zeroes); + +static inline bool page_reporting_host_zeroes_pages(void) +{ + return static_branch_unlikely(&page_reporting_host_zeroes); +} + static inline bool page_reported(struct page *page) { return static_branch_unlikely(&page_reporting_enabled) && @@ -46,6 +53,11 @@ static inline void page_reporting_notify_free(unsigned i= nt order) #else /* CONFIG_PAGE_REPORTING */ #define page_reported(_page) false =20 +static inline bool page_reporting_host_zeroes_pages(void) +{ + return false; +} + static inline void page_reporting_notify_free(unsigned int order) { } --=20 MST From nobody Wed Jun 17 02:49:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A2C334F479 for ; Tue, 21 Apr 2026 22:02:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808922; cv=none; b=NAAfNSWcc4a70CxHOPWoKJZKO361LAg2g5oBOFGGWDHX7qI5w0Pc3pFFeOyj55fPko86zwPnUxwgTu/pvGXAmTEK7haPRcrbdcOxvlhrGyvEACJmqbmD/gvELrSDHzir9y6xj0GgJTCXqLMup73/fbuh1O7rR5ifDR+vKFGkPXw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808922; c=relaxed/simple; bh=arS3wwEKF7EIFSX54DNKYimVm0l+bQRNcNnFnnvhoV0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=p7QQ1ic1dDtHLu7HEJLEOyyR13S7TqEYv3n4qX+wxXJfgNRB2rUsqEULzo+zW5Z58ZzSAoE/8vtNmM1hAQCMsbV3EvMdsYRy5De7p0MYtYrFC2M5oKnu0nZOViRtwBoCia1KlOLWxW2N0ysHoKaxDLLkz3927ybYmVrPpf2DRc0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=cqZsnL4e; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=kA6mT5El; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="cqZsnL4e"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="kA6mT5El" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1776808919; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=NAiu8nAu6jsdbD59MaLaRp9DyOH1KPADlHXUZUyeJ14=; b=cqZsnL4e1NcgQZkmPgabzzOhP+wxzQMePvBr4P5fi3o+Uf811Qsjwc4RSrM7LsOqwN0gW8 a5K23Ihy5Drjj+Aya+dKVbkIjSiptNqcaJbBBwLCgyekjmEKlFGHEdvsbbIYYpy/48Qy3B bqbcODI1WK4/vHatwvOW0fm9XVz/Q9o= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-1-EjwPctsyP3i3siBZ10X-TQ-1; Tue, 21 Apr 2026 18:01:58 -0400 X-MC-Unique: EjwPctsyP3i3siBZ10X-TQ-1 X-Mimecast-MFC-AGG-ID: EjwPctsyP3i3siBZ10X-TQ_1776808917 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-4362197d1easo3526070f8f.2 for ; Tue, 21 Apr 2026 15:01:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1776808917; x=1777413717; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=NAiu8nAu6jsdbD59MaLaRp9DyOH1KPADlHXUZUyeJ14=; b=kA6mT5ElNKqIo+AgB2ZEckxfwH1giVhemA9MSOQ+ozaA1eqKVj7GuH9jzykNDSxWjq Jxt31ToEmJccEumSZTocnvODCk9Ii4Oyi7I+1qdnGf+6pRs0/pdmNHyYyf3S+5JK2tjj +mqWNDuSPGwEiEpwTLTuHFiOfXULHOvDHaCVGGhZNManbBbtnqTmvP3Hqjx7zsbS0GVD ajwSgLeWhiCXBKd5V19+r+LzqSt7MMNxZZOIweRYA4bMgpFQHWT0lNDixIC3Vc13SqbA 8J5RK6Zh5fdeKUPgWLg7RyZ6JF7JOoCa0kHRpnE+3BcLvfm3dpZMGM7tgtbfqsbpPlgp JUnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776808917; x=1777413717; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NAiu8nAu6jsdbD59MaLaRp9DyOH1KPADlHXUZUyeJ14=; b=A2vC8s3wjOgjWaQk+a+Okx6G1MOotvr2N1FpWzcukgavayd8MFIRDUrQ9LiE2G0apY sMJ0xn1AeKQP1My3CimJcDS0ktrAQqDJe+vOheySAsi9xmiZMfDbMZePofj0Eln7uAU1 gupw8Ye3doPmPMxUTb/05lQThSjOi5xyr55p24JwjSP+ABqSjyA9m6RR24+zxCLZkomw ux/zdBLxlXfywSFVGVD18aqIl2z2BNmmcM8tQ5IX26cX+cGdNJCn96YhmxCuIsoqfgMi ar90MSaBg+AgphExQdLIOwQLVSKbwhq2F7MWLmHh6YbYmURKGdFbzUKU4ithKViIrqOF fVEg== X-Gm-Message-State: AOJu0YziRcnwsxDhbusub/z6khl6dKSaHnXgYVKiOrPylXHHp3W0FFd2 TMr0ObaABp9piE3Xhan+OalMyUccrJH+FGrc3Ay0JZyYfovH9uxtcZ8VKr0A5REFefYxzZmNnOb 3CPIbNjVdZr+T5dsiSK0vNeq5LYtmqxFu6rpHy/w67X5U+HTAfV0aNm3pd/9JltIgUtQdrib48u R9qgCj1e2pIjM0pUm63AYgeL0J7iSdikm4sO9WeMpdjJA= X-Gm-Gg: AeBDiesCl32GZDJCw93JpyZqlG/3V47Dt+N3g9nj3VpmPnHXRwfLsmN/ReWbsqpH954 MOPyMawv1N0gmurUDg1dOVp6Dn2emsTo6+fZTjOWDNmZvi+EJUFPJ1Edy/rK1hrPwe/FOTgxZuO JVQ1SxENlseZCWX2+Jw1X6AHxTQCaJHDGYkxVZA2zkCCX06PfWCvTRaQnu0o8T1FI3VilS5thvs Lhq8wQLFTbJjZRjA6j+5Q0eEkQo1cJ84t1l4O/WtW5V3hQND5U00W1m1RX1E2emP3oe4aUfWvha ryTIAm8HIlQaUcnl5w+uPm0zFOZZ2eKiJRyvIihA8+89OKw4qVHvGFxIqN8W4dHaOS32rb/GQrP IzU2KzGDKpiy6ZEqUllq92SX4hB3jKC7DyGO9eOQMN+gdw7HH24eaTQ== X-Received: by 2002:a05:6000:184b:b0:43f:e231:6721 with SMTP id ffacd0b85a97d-43fe3db2c4emr30836537f8f.2.1776808916948; Tue, 21 Apr 2026 15:01:56 -0700 (PDT) X-Received: by 2002:a05:6000:184b:b0:43f:e231:6721 with SMTP id ffacd0b85a97d-43fe3db2c4emr30836484f8f.2.1776808916351; Tue, 21 Apr 2026 15:01:56 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-21.inter.net.il. [80.230.25.21]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-43fe4e46471sm39731110f8f.28.2026.04.21.15.01.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Apr 2026 15:01:55 -0700 (PDT) Date: Tue, 21 Apr 2026 18:01:53 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= Subject: [PATCH RFC v3 13/19] virtio_balloon: a hack to enable host-zeroed page optimization Message-ID: <585a71dace8705d604da7d3093c70896eef5b2bc.1776808210.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a module parameter host_zeroes_pages to opt in to the zeroed page optimization. A proper virtio feature flag is needed before this can be merged. insmod virtio_balloon.ko host_zeroes_pages=3D1 Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- drivers/virtio/virtio_balloon.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index d1fbc8fe8470..165b123caa64 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -19,6 +19,11 @@ #include #include =20 +static bool host_zeroes_pages; +module_param(host_zeroes_pages, bool, 0444); +MODULE_PARM_DESC(host_zeroes_pages, + "Host zeroes reported pages, skip guest re-zeroing"); + /* * Balloon device works in 4K page units. So each page is pointed to by * multiple balloon pages. All memory counters in this driver are in ball= oon @@ -1039,6 +1044,8 @@ static int virtballoon_probe(struct virtio_device *vd= ev) vb->pr_dev_info.order =3D 5; #endif =20 + /* TODO: needs a virtio feature flag */ + vb->pr_dev_info.host_zeroes_pages =3D host_zeroes_pages; err =3D page_reporting_register(&vb->pr_dev_info); if (err) goto out_unregister_oom; --=20 MST From nobody Wed Jun 17 02:49:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A6E383822AA for ; Tue, 21 Apr 2026 22:02:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808924; cv=none; b=l66BU2iVJFpGXkhgr+2qWvf2bKvx0ocLf88IaL/RNvxOkBPdw7ssQEqqsjUZq8cTAxU6LAwnV+AFIaU9DlUcyWSaAcEdNBuymjdqKR3RnUH64p4cylTNL57Ih7LLvuI2DK5sa2jsPbjnsFgu6wk3zlBAYibIqSvS8l+2tln1ajo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808924; c=relaxed/simple; bh=DFAESwwsDTSQc7OD6QuzKMEO6K5LP9EvdbUmNXhsvmE=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=eaXlrNKMwxQPJE14b1anpfYy/Pd4TEzJFl/7YM6rGHfgrIUN4dfgQy135JEZPTEIIVBvtRQ3TVC2S9VnuM7gSMfm13ib55mmoiQaSbnznzPP7eAcwnH5/399tvTwrMHDqRaW+9sDNQPs8AT3Lhlk4a0CNyVh1ENijJeZUho4+GQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=JFNXlWyX; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=PKUpCrpk; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="JFNXlWyX"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="PKUpCrpk" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1776808922; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=cryXnIzewbC4JFE37j7BqLz76I8OG7HbHQFiiqeqBgM=; b=JFNXlWyXmOkYjjpYO/R1iC47mfCqj6IADqqSSqkda6Q9QNg0EEYjx7UvBI9DfJg0zaJ1c+ 6cyWrxehuG6McS5tGK1BH+ltjiEJ/b+P/REEYgs+jq5S1KUerPvtUYhT3SYFpY59Z7Uslw cGS/87w/YSWWlaYspvIAFTk8CwlvNjw= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-369-AiZ9qKd8P1K3mmsv1zxgkg-1; Tue, 21 Apr 2026 18:02:01 -0400 X-MC-Unique: AiZ9qKd8P1K3mmsv1zxgkg-1 X-Mimecast-MFC-AGG-ID: AiZ9qKd8P1K3mmsv1zxgkg_1776808920 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-48a5775d647so9446285e9.2 for ; Tue, 21 Apr 2026 15:02:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1776808920; x=1777413720; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=cryXnIzewbC4JFE37j7BqLz76I8OG7HbHQFiiqeqBgM=; b=PKUpCrpk5WZMXnL+UhXXRFywfKrCq4noahJbTOZcnh8hQoaOhrpeH/wE/j1hozxie7 qE32fE2CljgQ/pcusGXLlQwTcn9xHaftI47RAMnhCOPg/6TgHETQrbux44pBcIt3pSnm 31WBSK+g+6eoyPGt4DykhMVTnNZjaTATeHll3nhRRTxLb/M1wF10yJF5t4lTbmU8D4PL dC3W7izM4FS/3bG033OXAbyKlZFTNAOZ7VlykbtXeA6RslYFlvK9tKChFJCTyazro2v9 sl3kMZR5Haiw7KB5VnKKtB9xE8h2B3nbxh75kT5CG8ZMI0LHC1uOaK29JroTIVW9viXP Av7Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776808920; x=1777413720; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cryXnIzewbC4JFE37j7BqLz76I8OG7HbHQFiiqeqBgM=; b=GSYBAy/bxM4MeunDqvM7N68yT0jAJOal/vSk7+k6r83jCejFV6D7HrpueX/tOc2C9B DMH2ZqPUO3C/OoPd4Eyq9Q75bp4VxJju9C8R8Etk50/x+pDFU/rk1Xp9/7wDhF7aT7Gv 2khnHYUU9ADfTJs5b18pLZMSPEmVeoeGpvZMx6ZyLTZVmGM9sP0DUj3+9yeQh2bWDXD/ 76e+RZX+Hg+sh+7E92tLDgiciNNKv1s5Oz+W6R52TnCHZLiv1xxVCqs9S0v/wObmo2WT suflY9JUmR5sBvB1vgy35+Y60cXTSwKfpH+vr2u1dWFsq0tUCSnJT5LIgN05kwaVMa/W b0QQ== X-Gm-Message-State: AOJu0YzXyGVDj7coBHIkfQM7kPR0LShOYVKQJgY289Z73LIPlt+bV9/b 6/MJIcPOp6stYkDOEpq/WNntSdNr8F+2nGEoLwMFod79DbzjdiAgpnbVKtIge0PqRbPLxb9QnQg tduMZmdg5lFnKYW3mkm0KNko1eBzYgmPwXVacvE44MO6pBpAQb9JpK/jNNr/jOl7/rMVNkWz+O5 g0SSnmOXq2A5G6I4rGBUbXtzOLsCCAyNJaVNA2UDaOdMM= X-Gm-Gg: AeBDietHerX+EWQRmBwwbvrxXQ4HTzTK7d1XZWFMldI2bzATuTkl2NH+j6pRTUrgqq6 tGjzDfMqZ3S2rZySMJsyVurrg8y8CiGu6yTXmNmFDV6J4/NdccR1y0P+PG3SwXYdBpPIfSmT2RI kK7BbqVBn7C8swcD3lGLwSF7pi6i0cIx3Cu4IrHKshSovvjLrdXOFRVr/NqgALbD0BZYurXm44L s5TTUUEKqKexcEt6WCoURejqXnO0ViXZ2UsGRAwnq2u1N3bL6QaG55bVNGJuRhPZh44xtGYTzQm fExlAk8Et0HUEI6E8by1QpD3VfW3pUOQhiwvtgwS/T30cb+kmehXi3cmUcyXr0UVRHWKRAeL/+0 5yS9PJ4XN/ZwxmBr3bhJ3PtXymE/TBNQ40h9+vxVASWNDORnvLEmPcw== X-Received: by 2002:a05:600c:8483:b0:488:7ff6:1f75 with SMTP id 5b1f17b1804b1-488fb782d91mr292618925e9.21.1776808919889; Tue, 21 Apr 2026 15:01:59 -0700 (PDT) X-Received: by 2002:a05:600c:8483:b0:488:7ff6:1f75 with SMTP id 5b1f17b1804b1-488fb782d91mr292618445e9.21.1776808919403; Tue, 21 Apr 2026 15:01:59 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-21.inter.net.il. [80.230.25.21]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-488ffc558f2sm265846935e9.1.2026.04.21.15.01.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Apr 2026 15:01:58 -0700 (PDT) Date: Tue, 21 Apr 2026 18:01:56 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Johannes Weiner , Zi Yan Subject: [PATCH RFC v3 14/19] mm: page_reporting: add flush parameter with page budget Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a write-only module parameter 'flush' that triggers immediate page reporting. The value specifies approximately how many pages (at page_reporting_order) to report. The flush loops through reporting cycles, each processing up to PAGE_REPORTING_CAPACITY pages, until the budget is exhausted, all pages are reported, or a signal is pending. This is helpful when there is a lot of memory freed quickly, and a single cycle may not process all free pages due to internal budget limits. echo 512 > /sys/module/page_reporting/parameters/flush Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/page_reporting.c | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/mm/page_reporting.c b/mm/page_reporting.c index 6177d2413743..c09a8ac754dc 100644 --- a/mm/page_reporting.c +++ b/mm/page_reporting.c @@ -354,6 +354,48 @@ static void page_reporting_process(struct work_struct = *work) static DEFINE_MUTEX(page_reporting_mutex); DEFINE_STATIC_KEY_FALSE(page_reporting_enabled); =20 +static int page_reporting_flush_set(const char *val, + const struct kernel_param *kp) +{ + struct page_reporting_dev_info *prdev; + unsigned int budget; + int err; + + err =3D kstrtouint(val, 0, &budget); + if (err) + return err; + if (!budget) + return 0; + + mutex_lock(&page_reporting_mutex); + prdev =3D rcu_dereference_protected(pr_dev_info, + lockdep_is_held(&page_reporting_mutex)); + if (prdev) { + unsigned int reported; + + for (reported =3D 0; reported < budget; + reported +=3D PAGE_REPORTING_CAPACITY) { + flush_delayed_work(&prdev->work); + __page_reporting_request(prdev); + flush_delayed_work(&prdev->work); + if (atomic_read(&prdev->state) =3D=3D PAGE_REPORTING_IDLE) + break; + if (signal_pending(current)) + break; + } + } + mutex_unlock(&page_reporting_mutex); + return 0; +} + +static const struct kernel_param_ops flush_ops =3D { + .set =3D page_reporting_flush_set, + .get =3D param_get_uint, +}; +static unsigned int page_reporting_flush; +module_param_cb(flush, &flush_ops, &page_reporting_flush, 0200); +MODULE_PARM_DESC(flush, "Report up to N pages at page_reporting_order"); + int page_reporting_register(struct page_reporting_dev_info *prdev) { int err =3D 0; --=20 MST From nobody Wed Jun 17 02:49:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C39FA34D90F for ; Tue, 21 Apr 2026 22:02:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808931; cv=none; b=rzcoNfzIjs/3Ic3ydIw0qXGd8l21Zk7mEdQRLMPQRlkAf06j70TcPcQWq27IH0mfRUkWcXJNZfJNDD/WpCrwb2nFIlxFh5Eddpd+03IA6wg5Wz+fZrgQKNf1EwutbbZnXITNISqLokYnF7C6WIJQhLASNRsnszatKLsajj/gOFE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808931; c=relaxed/simple; bh=I+6H8EURLdFuL8zgQfH6lh8KyVwVos3w3eBqKRjNL0E=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Ct25e4z1kJ4aKgeVyJW+Yu5LNp+q6aOz8cEuO49oVaSLyVUzZkRmuEg+H51EDYur4zJVnNgUERqw2QjZwSqsoRqnvRGm7LgrWlyYd0qfNNzFoyYMpmkYdE6vHB5duVDjNnxaJHYWPyq0v1OV8XLOv60KXgLgUjb7oSYyFR9zoNU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=EsWkuM1h; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=KCs/v9UE; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="EsWkuM1h"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="KCs/v9UE" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1776808927; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=DqibEu51w521jmMlLo0nOiOBZCjTDi4b9M623UPDLCI=; b=EsWkuM1h2sKQoPuOC+H9Y1XOuqI6MRz5WFdTmGvgMlo3JjwZvPnZ9WVnf/V8wE79vlK3tU KQTKO9nRC5s8Pa9+wB4Vxv9mnrFLbqQ6LvdCMHaCevjif/AD1HlQq8MGNOgLqTrA+ESXNd 0IlJb8wfZgnvOwYTexnFoUoxuSp3qDQ= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-462-9AmY-2l2Mw62nc5Y3zrbQg-1; Tue, 21 Apr 2026 18:02:06 -0400 X-MC-Unique: 9AmY-2l2Mw62nc5Y3zrbQg-1 X-Mimecast-MFC-AGG-ID: 9AmY-2l2Mw62nc5Y3zrbQg_1776808925 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-488d3eec9bcso30645915e9.3 for ; Tue, 21 Apr 2026 15:02:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1776808925; x=1777413725; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=DqibEu51w521jmMlLo0nOiOBZCjTDi4b9M623UPDLCI=; b=KCs/v9UETIPXMgu/jGF+NSTVZIVnJAi/JH1VskoaS5DYKzkNr8oP2/hV5cv5gyqdIA QqYbswqciM9nNyiexe2bQptyOcPLYusCTvnof3Cw6pAU7PINerk9MHmQLwXYPTTAJdYZ b/ENp79OCRpEBvenlfuYPMAZJT8fQ4EiGbJGrnnkm/AlO2Tk6j6Qft3gnYIVEAKh171L THOnW0axveofPYtMAT3w4WrLqODA9MqSMJ5iDhNf8cyfewn97epXqq93JozYctJSWG0d 8n5taE75U5TKaf/xlQOYEcxnGSyIkm3H5VPFvyvhOpfbYYpOQjNY+yMnpMUcSke1y06I 5niA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776808925; x=1777413725; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DqibEu51w521jmMlLo0nOiOBZCjTDi4b9M623UPDLCI=; b=lxWbqAighll8HTLk6HLs7l/C2dH0TT9G6kvd1ux9+LLUwbPpQ82Wxd7hzYzav2xNcH syVRL/uc+YjJJSqLDrsi+y4jgYFrT72VZykG0g54RAwxenokXdUdSZrJrwA9zyE/xQaO 5TCprgQo7dtspjc2m6lgkywhJEPKvT5HKUxEmI++wqJnA7DCr5zVGJHvnTMuR82JXhE/ WEHZe9YyHM5yfeANuaCz/YOCFcaUERKBJv0F+krVM4najYKhPpowAaiD61opG0K1+0uO I+RQWhRRlDIPbqJY48ZeIc+a2fDpdJNTj8lIvyPj3wvDJSqhspOCH+c5PqvgPHFQ1ovO 3bwA== X-Gm-Message-State: AOJu0Yxgudpw8EIu19Z7Jt2MLh4c9PoP0Pux5+OAb2BH0vfy2i5UWqTg LT4JproRgaPOlg3L/OuFtP6GA2TYl+/FqkFLmCpwlqItBJ4aK9UZCHeZdYFSTWpeXc+mYWzzYFS Ww6SHHM3Pi0r/C0m/3/41jn+8gGHvKy48AMJmCeolkxoNqwsxpTvYPzJqEyZWKTfEWwq0Z/7Uba 0GzeDMLcsO1JtP0JBhhT7b2RHOmH8TV1Swfr/672K5oFc= X-Gm-Gg: AeBDievvtPn2QXqsXIjSqeNaIKkRqb4e9dd2WEzBAXwxJAjRNIHJlHHn3G12I0Aff7Y KInm6SxiJMs4C/exgY+fHOFXu4dp6IagglnhnsOI1atf1EWB58bXILfYKnidskDvJwzGRzVHpue 4gZbC12qrCSfDKzNaQaMvededfFWXeuy3TiJr1qkXuNLpZWlB3foPnQclMpg6sgtXhaTduF8Vnl 6dK06OzYU2RwFUECaxn3ISOeQGLALK6erpLHCwNoMlMqLDVStiJ5Js0xml1vbKv+Ur0FS7EDgbi ejZUCY2J4D+Ig2XXFBpjBleZc8Attt44wHho90J1c6dgR7fQFhDT63I7Ld7wxra8hLES7biRIkS TcQAGHElMz18Klg/Nu/jjA3Syvcedcc0g4mfMKu0kmtpZjWw1vVWzgw== X-Received: by 2002:a05:600c:8483:b0:488:7ff6:1f75 with SMTP id 5b1f17b1804b1-488fb782d91mr292621385e9.21.1776808923986; Tue, 21 Apr 2026 15:02:03 -0700 (PDT) X-Received: by 2002:a05:600c:8483:b0:488:7ff6:1f75 with SMTP id 5b1f17b1804b1-488fb782d91mr292620925e9.21.1776808923421; Tue, 21 Apr 2026 15:02:03 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-21.inter.net.il. [80.230.25.21]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-488fc17f642sm364997035e9.5.2026.04.21.15.02.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Apr 2026 15:02:02 -0700 (PDT) Date: Tue, 21 Apr 2026 18:01:59 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Johannes Weiner , Zi Yan , Lorenzo Stoakes , "Liam R. Howlett" , Mike Rapoport Subject: [PATCH RFC v3 15/19] mm: add free_frozen_pages_zeroed Message-ID: <86755bb7ab63e2374a242ce24a9a3f1981f10583.1776808210.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add free_frozen_pages_zeroed(page, order) to free a frozen page while marking it as zeroed, so the next allocation can skip redundant zeroing. An FPI_ZEROED internal flag carries the hint through the free path. PageZeroed is set after __free_pages_prepare() clears all flags, so the hint survives on the free list. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/gfp.h | 1 + mm/internal.h | 1 - mm/page_alloc.c | 21 ++++++++++++++++++++- 3 files changed, 21 insertions(+), 2 deletions(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index 10f653338042..12ab91e2ed57 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -398,6 +398,7 @@ __meminit void *alloc_pages_exact_nid_noprof(int nid, s= ize_t size, gfp_t gfp_mas extern void __free_pages(struct page *page, unsigned int order); extern void free_pages_nolock(struct page *page, unsigned int order); extern void free_pages(unsigned long addr, unsigned int order); +void free_frozen_pages_zeroed(struct page *page, unsigned int order); =20 #define __free_page(page) __free_pages((page), 0) #define free_page(addr) free_pages((addr), 0) diff --git a/mm/internal.h b/mm/internal.h index 4c33249e03f0..e655385b269c 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -899,7 +899,6 @@ struct page *__alloc_frozen_pages_noprof(gfp_t, unsigne= d int order, int nid, #define __alloc_frozen_pages(...) \ alloc_hooks(__alloc_frozen_pages_noprof(__VA_ARGS__)) void free_frozen_pages(struct page *page, unsigned int order); -void free_frozen_pages_zeroed(struct page *page, unsigned int order); void free_unref_folios(struct folio_batch *fbatch); =20 #ifdef CONFIG_NUMA diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 2098d569d80c..9311374bbd2d 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -90,6 +90,13 @@ typedef int __bitwise fpi_t; /* Free the page without taking locks. Rely on trylock only. */ #define FPI_TRYLOCK ((__force fpi_t)BIT(2)) =20 +/* + * The page contents are known to be zero (e.g., the host zeroed them + * during balloon deflate). Set PageZeroed after free so the next + * allocation can skip redundant zeroing. + */ +#define FPI_ZEROED ((__force fpi_t)BIT(3)) + /* prevent >1 _updater_ of zone percpu pageset ->high and ->batch fields */ static DEFINE_MUTEX(pcp_batch_high_lock); #define MIN_PERCPU_PAGELIST_HIGH_FRACTION (8) @@ -1611,8 +1618,11 @@ static void __free_pages_ok(struct page *page, unsig= ned int order, unsigned long pfn =3D page_to_pfn(page); struct zone *zone =3D page_zone(page); =20 - if (__free_pages_prepare(page, order, fpi_flags)) + if (__free_pages_prepare(page, order, fpi_flags)) { + if (fpi_flags & FPI_ZEROED) + __SetPageZeroed(page); free_one_page(zone, page, pfn, order, fpi_flags); + } } =20 void __meminit __free_pages_core(struct page *page, unsigned int order, @@ -3012,6 +3022,9 @@ static void __free_frozen_pages(struct page *page, un= signed int order, if (!__free_pages_prepare(page, order, fpi_flags)) return; =20 + if (fpi_flags & FPI_ZEROED) + __SetPageZeroed(page); + /* * We only track unmovable, reclaimable and movable on pcp lists. * Place ISOLATE pages on the isolated list because they are being @@ -3050,6 +3063,12 @@ void free_frozen_pages(struct page *page, unsigned i= nt order) __free_frozen_pages(page, order, FPI_NONE); } =20 +void free_frozen_pages_zeroed(struct page *page, unsigned int order) +{ + __free_frozen_pages(page, order, FPI_ZEROED); +} +EXPORT_SYMBOL(free_frozen_pages_zeroed); + void free_frozen_pages_nolock(struct page *page, unsigned int order) { __free_frozen_pages(page, order, FPI_TRYLOCK); --=20 MST From nobody Wed Jun 17 02:49:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 25E7E3A1E70 for ; Tue, 21 Apr 2026 22:02:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808933; cv=none; b=LEC+UMnJncS5QXN9DDiMk31cUuqxV7iOPiheni/6aNG5wmKTqmBKft2Wol6G0Z08OKi0Ioh2wSPArvsEaK8hMoVJhOaBgBpt/YDLEIht5XSyEZUfKVOM/6lEj/u+VxoE5kCvaCnnLMhlxWX1r9YC4HLrw/P+5UPVg59d1kKH/54= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808933; c=relaxed/simple; bh=ArOZv9uCSES6WoLoP2qfLy3zOOTQa3tVnsSTEE/uUjI=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=RgzhHYylLC6RrC2D21jUnymrfvn6tmQUzT04iT6hXpQw6uGHCob4/UgkZI4fIoLUKQoQtaMvHCPGzzxPGZUx56dteO3wQPdVXpLCbJuN3WxOKZ9dwjVoQoLrTn6Wq7pu+4yfcwUG5JCQNiQGskaMNgGNMkzuB4tYwvDleGN8ShE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=N30hWXjj; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=V+VpvEZj; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="N30hWXjj"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="V+VpvEZj" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1776808931; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=b5ITR8zgbYah+ZxSMqe409RlUP6fQP6IKxEaDIol6Sg=; b=N30hWXjj9HibWjKM69cMF8kr3LAjS82uEgkkVsjdQpBPaTF1TVvKhbfhv9Z4sLrNZVTCD9 6p42Olou/P8B7AcrtzpLZDrk2t2VYi4An/ogFLZ0K5bnv9Wmla8NPtuevgH/W68Evaz6W9 Fn/i4Zo9Cu/JYGK5gIxFRzuH3wwb3RM= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-661-loCFxwP6Oc2gnhGfXCScKg-1; Tue, 21 Apr 2026 18:02:09 -0400 X-MC-Unique: loCFxwP6Oc2gnhGfXCScKg-1 X-Mimecast-MFC-AGG-ID: loCFxwP6Oc2gnhGfXCScKg_1776808929 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-43d77286244so3118433f8f.1 for ; Tue, 21 Apr 2026 15:02:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1776808928; x=1777413728; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=b5ITR8zgbYah+ZxSMqe409RlUP6fQP6IKxEaDIol6Sg=; b=V+VpvEZjWrjvm5ewdDurUWRGyET8QZqoRIEi1nwjj6ypQsTMSFE5EHkabAxi/3UnF8 yKTlknEhGGkk6cBHhpwvmnCmimkt8b3D1OjP2r0Y0DufWSAqT5egowyC5Af388wQdGB0 KxXhPDeqmP5L0Cozcy/WxoKNLgVD+Y6OqUelbEfQukJ7i4nW4sPeCx7kvkQb1M8ozXVG giSCXJH+atGw3X1ZAV8QyG2bYHHmHhwZAK/cra6iJfpRKE1xT5XEOQyJGnWRf0kTLs6y A71BUgWe8281DVtE52ecz46e+oinUyPAfil2s29+r9n2jxnfhweOr7QlSeGeJBKHccbL 5d3g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776808928; x=1777413728; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=b5ITR8zgbYah+ZxSMqe409RlUP6fQP6IKxEaDIol6Sg=; b=BiUjmpfO0a82tQfIGymp8HbM80hIac+1FXa2QlERtECjBud2w7NPNbWISAQ88ZGCbY 10DJLgv3vtzPddvivSmoLY4bVRnOBAH1zfaFjo4o260P33UExfUVlwKxqG00b5pob3RG pjzRYT+YrIwLMXfior6FCbdSYDwCZqZli/V6l4K74/MNUfp5fxeurOHNV2Jm3pqaKG89 OdJl/0Sst7+C7u+nO60FFrijKr/KEXvc4YCgaDh7f/JdGbWnAGbSzrg+nfoow1tV+Pst eTh7IGI4MnLdyyOqgZ8ObiWOdaBPtV3F+Ym5clWhbCJfzyzIb73o+vBuQoh/VWdW555c 5axQ== X-Gm-Message-State: AOJu0Yw4DsihZ+6819DE86iVoR8GOctvZ9S+hoyTnj8n7Mf5SXqw2Gr/ wAdZN9mi4lXMZd/AnuHU/Z17hTna5UzEQawPLr7AuGL2LB6orz6G55Oy1UsH4hLyWrdV6x8duBb l2kdZPxi7PymrzhxwuF9qLR6nd9iBeDUssNpJ0betSj3LwIHrPChR9zmhm1yTlZBI7bX/OYHsHD w0YBFhmAsVhOpCypbfRuO6ipKKFXYWizCSyVuj4Be+YVQ= X-Gm-Gg: AeBDietO0mm9gkXbYWsSzWgnlmbkCCHKRI/G85amWyUS4/g6veOzC/bx4wgmKmAn2Il 6VEHblKPa6RyrJQ1ZhO0GLN561EY7O/hsDm2IUBgvi0x2YVbqH6whZhqIdzSRbgkfb0Ds99BXia ff52YU1cIdJAXxky6nX16Gy37M+kOeho4JCczipN6AGhrmpLzwAZn4+ZRBevEAs9/K/JhU2f0/N CYH7Ty4jCmjdf+H0Z/VXNFcic5HaIdYljNbakm2Rn+bVLLcChqyKjpT7aKI7C4b3ByWkdmUpJzq YCx8N7rfJMerX/udT5eqd9gDXm88ZGl0eJG/a+bLpUie4E37RQ1y2olQcFLeTjAxYjLDZKmPJ1u HbB/6qGos7+ITUU0vG5YLctSe7YmabNni/lXEL9eDjqgZTM9CJ9FwUg== X-Received: by 2002:a05:600c:1da1:b0:488:79a3:f04c with SMTP id 5b1f17b1804b1-488fb7863femr300220525e9.27.1776808928337; Tue, 21 Apr 2026 15:02:08 -0700 (PDT) X-Received: by 2002:a05:600c:1da1:b0:488:79a3:f04c with SMTP id 5b1f17b1804b1-488fb7863femr300219935e9.27.1776808927735; Tue, 21 Apr 2026 15:02:07 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-21.inter.net.il. [80.230.25.21]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48a5549f582sm28914685e9.33.2026.04.21.15.02.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Apr 2026 15:02:06 -0700 (PDT) Date: Tue, 21 Apr 2026 18:02:03 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Lorenzo Stoakes , "Liam R. Howlett" , Mike Rapoport , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , Barry Song Subject: [PATCH RFC v3 16/19] mm: add put_page_zeroed and folio_put_zeroed Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add put_page_zeroed() / folio_put_zeroed() for callers that hold a reference to a page known to be zeroed. If this drops the last reference, the page goes through __folio_put_zeroed() which calls free_frozen_pages_zeroed() so the zeroed hint is preserved. If someone else still holds a reference, the hint is simply lost -- this is best-effort. This is useful for balloon drivers during deflation: the host has already zeroed the pages, and the balloon is typically the sole owner. But if the page happens to be shared, silently dropping the hint is safe and avoids the need for callers to check the refcount. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/mm.h | 12 ++++++++++++ mm/swap.c | 18 ++++++++++++++++-- 2 files changed, 28 insertions(+), 2 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 821034dd33d1..878544830369 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1640,6 +1640,7 @@ static inline struct folio *virt_to_folio(const void = *x) } =20 void __folio_put(struct folio *folio); +void __folio_put_zeroed(struct folio *folio); =20 void split_page(struct page *page, unsigned int order); void folio_copy(struct folio *dst, struct folio *src); @@ -1817,6 +1818,17 @@ static inline void folio_put(struct folio *folio) __folio_put(folio); } =20 +static inline void folio_put_zeroed(struct folio *folio) +{ + if (folio_put_testzero(folio)) + __folio_put_zeroed(folio); +} + +static inline void put_page_zeroed(struct page *page) +{ + folio_put_zeroed(page_folio(page)); +} + /** * folio_put_refs - Reduce the reference count on a folio. * @folio: The folio. diff --git a/mm/swap.c b/mm/swap.c index bb19ccbece46..5d05a463b46a 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -94,7 +94,7 @@ static void page_cache_release(struct folio *folio) unlock_page_lruvec_irqrestore(lruvec, flags); } =20 -void __folio_put(struct folio *folio) +static void ___folio_put(struct folio *folio, bool zeroed) { if (unlikely(folio_is_zone_device(folio))) { free_zone_device_folio(folio); @@ -109,10 +109,24 @@ void __folio_put(struct folio *folio) page_cache_release(folio); folio_unqueue_deferred_split(folio); mem_cgroup_uncharge(folio); - free_frozen_pages(&folio->page, folio_order(folio)); + if (zeroed) + free_frozen_pages_zeroed(&folio->page, folio_order(folio)); + else + free_frozen_pages(&folio->page, folio_order(folio)); +} + +void __folio_put(struct folio *folio) +{ + ___folio_put(folio, false); } EXPORT_SYMBOL(__folio_put); =20 +void __folio_put_zeroed(struct folio *folio) +{ + ___folio_put(folio, true); +} +EXPORT_SYMBOL(__folio_put_zeroed); + typedef void (*move_fn_t)(struct lruvec *lruvec, struct folio *folio); =20 static void lru_add(struct lruvec *lruvec, struct folio *folio) --=20 MST From nobody Wed Jun 17 02:49:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C247D3A641A for ; Tue, 21 Apr 2026 22:02:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808936; cv=none; b=Ds95vDXi7KRGgIJYzmgXsDD6csHXNE8algM9BmgVfIRjxUUpZaI7B80vRPvvcZAjiROF7AhFA5e+Nskxvgq5bbUFDjova2ZROMUBGcUkc1or6adZYlG+y5FpIlQPe1XNZHEdVhVW5eVOgzqRfJFBItKOLOPk6sVOG1CDcf7ufpk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808936; c=relaxed/simple; bh=qSEJ1Kijx5ZivGiuuv5dmsRR2ofsgHwqXpUDJzbMbi0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=r0QmstpX3p2n105tic3Oo8uFjBShJzCJNJQmUtIFgJRKQryA5y0FP/D5f0kUosLl+naZnpcNg6Q1t7TlP7Hqs1n/cJoelyDy6AedoMPv5bRjt3KkHkA2KLad78sF3G16DhittueLCzigCHf1P2rrfTkRBJUF7CmCm+mWK86iqPQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=HSdB8Yve; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=uHSaOflW; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="HSdB8Yve"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="uHSaOflW" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1776808934; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=B71z8WI6bGkbALDtbCGicFiZB6WFkZ9HcNzSfAqdU04=; b=HSdB8Yvew7jUWPR+BNNxK/HpbNndT5kRPUkSJsLP8WaygL/nSkokgrReyDwF82L8GdpTcK WMQk7b0/MiFvcf0rbtEt9MMi8NyUh/OwcZ4LCdkh2p+jhY3ZAtk5rfWHSHeFsygfzzaPUn j2T55VwIzMWRUgm5JpMvmeJj57GRkdU= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-671-XkYtm4nwM4WV0Q0-G5kgKA-1; Tue, 21 Apr 2026 18:02:12 -0400 X-MC-Unique: XkYtm4nwM4WV0Q0-G5kgKA-1 X-Mimecast-MFC-AGG-ID: XkYtm4nwM4WV0Q0-G5kgKA_1776808932 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-48a5775d647so9448215e9.2 for ; Tue, 21 Apr 2026 15:02:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1776808931; x=1777413731; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=B71z8WI6bGkbALDtbCGicFiZB6WFkZ9HcNzSfAqdU04=; b=uHSaOflWSVZqCPD3BaW5c2MB6L+EMCH+PFHIEn/HEFApxJIvC7xFzbBaopYNpbKhto hP7dzyD5wRjtwV5i4gaL9rZg9xLU9gqNj/5i/Yx0exbMmQnj8B+zxcBox76cE6pwSfMv TlT7b6NvnHOce+UKvOXZl+nxLKf5VyLY76ifqHs0F9VgWwUHyRYrsFxVGqu04zd6q3QF 90xW1wKy8prxAgGm4DPt9SLnHFlxR9foFBl2SJay340rBP3UMm4kiv06ol2ZeljYfuVx 3/FzkKQqcIgSinVaecajBl0KPSwtVWShU4sdGp9Fqf5/pSnYS0cGRXblGsaHtafD4pNw Nz0A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776808931; x=1777413731; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=B71z8WI6bGkbALDtbCGicFiZB6WFkZ9HcNzSfAqdU04=; b=ONGOaz1acuZ3MgH5kP7YRy5WamBoK+PRa6Uv7FSH8AxXuNIyyl8RXwBacuK+iVkHyg gc8PqMG/2noxRzXO8ijCX3gZFjyMOjnjj1mkaaMuD1b8N5JD3fL3QiniOiUQs83uWoTX UM4sOcJXHeFesB3jCiPS1nuoZ8HLoorIWh2NX+aIL7V+lLpWHTAXRsp4DiK142sjYFNU g2UzupX3vo/sbhCBo9stxI91f7+9NRs2WoP5w7zrw+XCoalHMfxHZwPyPe1Usx72i/YZ ShnZJLHbEK8hl41iwBGLgH3cIkKabyeMfVDtfsg7LXAg1Uwo8bFwtQNYdVfKNDF0IyAg 5grw== X-Gm-Message-State: AOJu0YwR+/UFaIVZ4G296z6pXUaRln4h6B+ML9brzw9rhiaWo2Z3ZY+I XQx3WRemvJbDTsHypg6gPxecc6mmcZtUbH56VMrCHhDCD2G5SonQ+o8WYPvD9dNA3TL3/UPl4zf R+vMpMBPCqRj8JOMols5fIkvwb0firZSDYXnlGQ+w4bU8oVZ28d9K7rhdjWo8+UharULxCDA2W8 JaX3QfDzLOkWJrZhyoyNSNbxq1lhYnvf1WUMdjK+kNjfg= X-Gm-Gg: AeBDietWcEmHsI3dXlFKjJFFG14Et41k8JiM3aMxY0CnsiSbEdMdNxDKQ5jqfCNGF9H i1CwblZj3UFp+UdEG0xvXo0+ySbTz+baAdY19WCTXSgDpsoNxna+QglNFYyGs+dNRrCXEwt6zYL Bq15LJmHqgkqyZe6khM76Bud+oq53ET3mJnv7CEnfw60Af7CMDrA95VoCP8IEnufdAT33WZfnyc 1Lhsb5J3O6BzdkX+OSjSGdN1UkrTsZUVoRu7KfeJxEzXvRIj2u8g1743FR0HOXwXr6UGKkzccQN ebchPWiy+Hv5gwt8xDquOKaa/sDzsg//wZ8c4E3jb36eGUbEJC0yFVOYogs3uID0mGMuvPdxwWH P0BFFzo2sQqR/yGHS8tB+xqXwvTeBTEs3qCqEdHMT8qR8cQMaCu6VCA== X-Received: by 2002:a05:600c:4ec6:b0:489:1f98:71e3 with SMTP id 5b1f17b1804b1-4891f987487mr132175435e9.28.1776808931382; Tue, 21 Apr 2026 15:02:11 -0700 (PDT) X-Received: by 2002:a05:600c:4ec6:b0:489:1f98:71e3 with SMTP id 5b1f17b1804b1-4891f987487mr132175005e9.28.1776808930932; Tue, 21 Apr 2026 15:02:10 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-21.inter.net.il. [80.230.25.21]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-43fe4e4d525sm38750206f8f.31.2026.04.21.15.02.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Apr 2026 15:02:10 -0700 (PDT) Date: Tue, 21 Apr 2026 18:02:07 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Johannes Weiner , Zi Yan Subject: [PATCH RFC v3 17/19] mm: page_alloc: clear PG_zeroed on buddy merge if not both zero Message-ID: <6992d257f953733586275aa766add3195aa93770.1776808210.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When two buddy pages merge in __free_one_page(), preserve PG_zeroed on the merged page only if both buddies have the flag set. Otherwise clear it. Without this, a zeroed page (freed via free_frozen_pages_zeroed from balloon deflate) could merge with a non-zero buddy. The merged page would inherit PG_zeroed, and a later __GFP_ZERO allocation would skip zeroing stale data in the non-zero half. The page reporting path is not affected: it sets PG_zeroed during allocation (page_del_and_expand), not on free list pages. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/page_alloc.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 9311374bbd2d..122b49a6d435 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -991,6 +991,8 @@ static inline void __free_one_page(struct page *page, unsigned long buddy_pfn =3D 0; unsigned long combined_pfn; struct page *buddy; + bool buddy_zeroed; + bool page_zeroed; bool to_tail; =20 VM_BUG_ON(!zone_is_initialized(zone)); @@ -1029,6 +1031,8 @@ static inline void __free_one_page(struct page *page, goto done_merging; } =20 + buddy_zeroed =3D PageZeroed(buddy); + /* * Our buddy is free or it is CONFIG_DEBUG_PAGEALLOC guard page, * merge with it and move up one order. @@ -1047,10 +1051,17 @@ static inline void __free_one_page(struct page *pag= e, change_pageblock_range(buddy, order, migratetype); } =20 + page_zeroed =3D PageZeroed(page); + __ClearPageZeroed(page); + __ClearPageZeroed(buddy); + combined_pfn =3D buddy_pfn & pfn; page =3D page + (combined_pfn - pfn); pfn =3D combined_pfn; order++; + + if (page_zeroed && buddy_zeroed) + __SetPageZeroed(page); } =20 done_merging: --=20 MST From nobody Wed Jun 17 02:49:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 064E73A6B6D for ; Tue, 21 Apr 2026 22:02:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808939; cv=none; b=EsxY1LBP4xDkdqZFPhKnjC8nRJvUuR399hr5Mw9SM0H3Pt4l8qlb8etUx9UMwLGSWU6R/SeltdUFLPHiOuCWEzubZSZDrilA4DptFSnwM6ZH1MnNPVNlM/Dd2qu6qY3dogEXxwTb1B4P7YA5WBAzo4o3zzUDgn2hn2YN7K23Mcw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808939; c=relaxed/simple; bh=t6QdLjw1QOnLG7lxO09z0R2n3QTY09HOSu8w9aFfJ3w=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=pajRurhcnvrokSXrvHouvgZB3FxdXbIpNnwA6qass9TtZE+PvSP6/kf8XUT71Zd93m+EpIifvR7a55I0myEdNqLj2M5zjR4TxsiuSP1YwkI3xc4aIEGj7pm5Jp8qygfzqF925XlpcRNrUPQZKA/ynMKKlZr21wCnNCXzGis3WdM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=fDv61e8p; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=n0rwOPsp; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="fDv61e8p"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="n0rwOPsp" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1776808937; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=83Or3bPRvcXeLqEVqfcTGNmNcJ2rHHbe/Ydh3aV3puc=; b=fDv61e8p+C+rHNLwKwFrhAjcVbHZ0rTZMXvVCeHkTBp6d0hVv+lFNhzAEgbCopJLfhPl+a GTRh1cccuAOFCzZH6dMdSTg3lmJ2em45Nf5NZsSUeBA6O8ccV/XGlmT0YpES2f0/Umg7qw QzrQ/tPJytxpCmzuVk6EpKUBaNV3clo= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-111-SkksooMhOJeohrkJOa35Zw-1; Tue, 21 Apr 2026 18:02:16 -0400 X-MC-Unique: SkksooMhOJeohrkJOa35Zw-1 X-Mimecast-MFC-AGG-ID: SkksooMhOJeohrkJOa35Zw_1776808935 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-4836abfc742so50806935e9.0 for ; Tue, 21 Apr 2026 15:02:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1776808934; x=1777413734; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=83Or3bPRvcXeLqEVqfcTGNmNcJ2rHHbe/Ydh3aV3puc=; b=n0rwOPspP6blJPJFwE1nelM+DfaaVvQRHBINP5Q+iDh7ltrbUW5uRKZ9m3Ld1zBGa+ ioKYi90KrM1RqTZhKamzThYL7+OWvaT8LX5U7tt5V//xD1BMlG5PcoGUmmU59Fl+q55q bk7Emkuz+BIESgKkHS7ARhN4744jw4o+7EoC6HP3AjgQ0nvBc/mpr9A82kg4v6sptJTV endLASUdexZj1q5EUoGOFKzCr9K9uXiCMhvBMth6cwqW+WVMa8VjkDc4PLlxG/69FdED FF/FhP/1iXQtLpRm++4PTehOHzbQ/S/TohZhv6OuIo3gemGcoFt2JywRpAjXw3VWVNI4 szDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776808934; x=1777413734; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=83Or3bPRvcXeLqEVqfcTGNmNcJ2rHHbe/Ydh3aV3puc=; b=YuOWohYmW2UGZ/9XevaXyQvKqIO1lSSd/w+eSZtk0L+vrLnAUtnBXFbFw52CGU31Dx cIip/28fZ1f1VD5UGmJTJkC+UT+8qFgWjav+K83DnUgQS3AZlIMcOrRT7pW/h5Zbnvyh uNvNODrIhqTOR+6moWRtx7Lq02H813z063nqYTXIK/mAGglQ3y9G/DoDcspqf7yLAG44 pxGVKv0zh6EfgGJUtyUbI/YWuyyMyzvG5Psi53TcZ0LSjlwdlTQaVoUPKwQmpsX/hb3m L9HZtcfPRkpWJgFq0Pq1lobBukdXgs7l0AxKqgDsgfHc4scfywHOPdbxPAWJpgAMoQ8n H3Yw== X-Gm-Message-State: AOJu0Yyv1EOs+O7Pbl9YOXmsGeMhyYBwk2CGwWzStrqmfhUaCo2IdT2f YXUWTG8+DEJUZCXLqzRfOJKmwjGwkgX+m2L/9BAHnH1925LusHXnh7szKdbHjq6sIz54n42Tw/W ftFV4/jWIDaAAZadlmC7peg0zTw6l8MOkOdT4YkDBLeJUjg7cy7+XHFH31sQ8ViaMRvcTg2eQeb O+lQnW+lUgUpCNx4VtLJOJ351hxh3T0iedtD5lVjMLdag= X-Gm-Gg: AeBDieuLo1z2XUfi9eKgOeImF4BKpE/R6pq1duUpqEBdckb7Zluh1CmJcgJTJgHWUWN j4Eu92moRYrkIi5xMlWj+/lRMGD1Yf182KsSTjTEg/XJ580Hbru1ZV6LxtilsgdZIF/JhbWGJ9t SzJoHH+avKjMTzY7PEvO2r4H2gkl3lA1expzhko25LfqEsGTF8yGmz3hIs8GKwpvsxBxfk+s/0i Xr4gZ4FnqhqrTfgbQjU+x5NSwxyqokb9tvF+8fdNWJVgVAkHfvleg/zgQNP1g8fQMdjAjH4JyEI 2ej7CSp0GaspDdJkuQ5B1+5hISCSKwVvJlLzXaMJIy+jJsBHFVSNBumNgFcMIEe4QuTcLuwDmcg F5MQh8/s7uVXFFJmD9St8lymawQk8Km1IL+CLDINBNXxXMlOLAG4DBQ== X-Received: by 2002:a05:600c:a105:b0:486:fb69:4960 with SMTP id 5b1f17b1804b1-488fb7786e4mr201642765e9.19.1776808934602; Tue, 21 Apr 2026 15:02:14 -0700 (PDT) X-Received: by 2002:a05:600c:a105:b0:486:fb69:4960 with SMTP id 5b1f17b1804b1-488fb7786e4mr201642415e9.19.1776808934123; Tue, 21 Apr 2026 15:02:14 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-21.inter.net.il. [80.230.25.21]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48a5aa3ae83sm884845e9.12.2026.04.21.15.02.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Apr 2026 15:02:13 -0700 (PDT) Date: Tue, 21 Apr 2026 18:02:11 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Johannes Weiner , Zi Yan Subject: [PATCH RFC v3 18/19] mm: page_alloc: preserve PG_zeroed in page_del_and_expand Message-ID: <23f51ec61026f598a30a2a031fcf88100afc1f98.1776808210.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Don't unconditionally clear PG_zeroed for non-reported pages in page_del_and_expand(). Pages freed via free_frozen_pages_zeroed (balloon deflate) already have the flag set and should keep it through buddy allocation, not just PCP reuse. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/page_alloc.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 122b49a6d435..3f5ed022cb9c 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1751,7 +1751,8 @@ struct page *__pageblock_pfn_to_page(unsigned long st= art_pfn, * -- nyc */ static inline unsigned int expand(struct zone *zone, struct page *page, in= t low, - int high, int migratetype, bool reported) + int high, int migratetype, bool reported, + bool zeroed) { unsigned int size =3D 1 << high; unsigned int nr_added =3D 0; @@ -1782,6 +1783,8 @@ static inline unsigned int expand(struct zone *zone, = struct page *page, int low, */ if (reported) __SetPageReported(&page[size]); + if (zeroed) + __SetPageZeroed(&page[size]); } =20 return nr_added; @@ -1793,10 +1796,12 @@ static __always_inline void page_del_and_expand(str= uct zone *zone, { int nr_pages =3D 1 << high; bool was_reported =3D page_reported(page); + bool was_zeroed =3D PageZeroed(page); =20 __del_page_from_free_list(page, zone, high, migratetype); =20 - nr_pages -=3D expand(zone, page, low, high, migratetype, was_reported); + nr_pages -=3D expand(zone, page, low, high, migratetype, was_reported, + was_zeroed); account_freepages(zone, -nr_pages, migratetype); } =20 @@ -2373,11 +2378,13 @@ try_to_claim_block(struct zone *zone, struct page *= page, /* Take ownership for orders >=3D pageblock_order */ if (current_order >=3D pageblock_order) { unsigned int nr_added; + bool was_reported =3D page_reported(page); + bool was_zeroed =3D PageZeroed(page); =20 del_page_from_free_list(page, zone, current_order, block_type); change_pageblock_range(page, current_order, start_type); nr_added =3D expand(zone, page, order, current_order, start_type, - false); + was_reported, was_zeroed); account_freepages(zone, nr_added, start_type); return page; } --=20 MST From nobody Wed Jun 17 02:49:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 59C4C3A6EFF for ; Tue, 21 Apr 2026 22:02:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808942; cv=none; b=UheTO8khjg1N3TgXBFy525gl/aApQUyV4Cwu01EUwoQr1+Q6vyah2mQkzLlWyTvGH0eeWc2vguHNuuvqSC0lPaTZU+3bBjFSLDlGOVGcKK6BiFS0JzYcRxx7DDwoQzUh6hZTohwWzuRJ8Dw2RXp28Qd1YrQEikv/Xk2ykOA80ko= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1776808942; c=relaxed/simple; bh=w7yku+W4GfSSG96SUrLMIOzLwQD5fzXNoQPFKEr+ev4=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=a+6iq14lLNaIdAbQbfWh7iTkpxi3BWMuBy2bZwcEfCmqDIvNuz6/HQe3pVgIMXehZncBuKIsyc7rfOTETCFKwUTop+46jcUo1rTJ8a0q+VIFdgNqw0ULg3JzmpDn7NQ00k2lx1iLo6ogL/WtSZrusBv0hfKyp3MrjiSqPEdlFlc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Zqiczn97; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=RQfSvFlP; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Zqiczn97"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="RQfSvFlP" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1776808940; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=WzL0/LIharUBH04VP2SJLM2VLBD1UEvntqvv4vY+Hyg=; b=Zqiczn971++LrWtiCmygieeZ/SUPSlLXlPpJFuP4Uu8yO2Ed6+TeLdxe4dRMAIMaA6MawU q8NRef4ZRzbV3p5Pv96N4XT8HUy96YtAdXUKYkQ7Iwbe7o0nKPgSgKUHfep+zLaz0g0end ffxvbg7U6Op3FX9ggqWuzpeg5Vxl4Qc= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-593-sAKF-XWdPhGaGufFUlwipg-1; Tue, 21 Apr 2026 18:02:19 -0400 X-MC-Unique: sAKF-XWdPhGaGufFUlwipg-1 X-Mimecast-MFC-AGG-ID: sAKF-XWdPhGaGufFUlwipg_1776808938 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-48378df3469so31092765e9.1 for ; Tue, 21 Apr 2026 15:02:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1776808938; x=1777413738; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=WzL0/LIharUBH04VP2SJLM2VLBD1UEvntqvv4vY+Hyg=; b=RQfSvFlP3feCj0yrooCJAzrwyjfK/FUw/BFoNyfcdYIL7pvqvPaFgQVFIozsazaprh lgFJ9lSceUNrG+LT4SAVM9UcwVxqgQYnN54tOygxHuIPt60oPLZoLNd3ftu2QITUy9pN oAyZvzf7ZcBblhDbnvfzwXnt1M+xOd1Bt+aUlU2bLLoiObDPDCKiClnyG9a/aruWYqnj 0pvWUuvBir2C+7RSggTCKAUqOIODFN/T0SbK1UgG0XB0XUloAXt17hdxlvkAYYU3ucSf gJ91YtpEvIbK83Pfzb9CfyqT6f8K605/1SkRx1EAQqKG/xR1LWXHJmmVQDhg5u1nczB3 Qffw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1776808938; x=1777413738; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WzL0/LIharUBH04VP2SJLM2VLBD1UEvntqvv4vY+Hyg=; b=rID2YD7geuffGSEPvVEVrpo7B6mO3SWhhS/yBRI23OP6A/0TLdx1Jx9Gg8sUoEoATs aLbimkUllY/YoW80jhzW2GrA9aC4d/WBkNWoXaKTWGAkG83ajGA+kRFR2k5ucyJTVvaf bzIQ+SLmx7Aenj+DW6/lrCWbS6ZtkFG4tvTuZ0hFNIHBSNVIJ3ZWoHYaOtokwRbZVHke EgGCIB7DURMz5QlxLpOzmRzHnvgwC07+ZpUcvmgEcEx58R9U9Y7eB9nHYBDYOKRab648 O00jJfZf1woo7yKzxMcq9BRwL8yvvQQA2b/1a7mvI2jPH0z3kWHKqC8CGPpvrAYvmmXB vCvg== X-Gm-Message-State: AOJu0YwtmCGI3BrR5+5pvjbI2fELH0Sz78j6w8P/c6jSYqukctIJsp3s blyZS93gzjojpVSxY1cAIf09r7Jivh+6wlEbeLqpRuHjH4nRiQe5ML6Y3z6YhL5JR4p1UN9+Cpi aUhqeuCuZ+jfiHhurVbYSJkrFdmjWq4UEmLjKSCyxCRqHjqDqywSJ1QA1KMKVX3GJXwawQ1/OY6 8x4tK8CfOxXXVjw72Xgf0JTexChrREUGzCjH+GubsyQV0= X-Gm-Gg: AeBDieu+yLT3bfBDAVt0z+nyoiancUQdVsKABUF5DZEk7EXk9h6Y2c/+yWUQXPEbpj6 14w9W0C+nuT9NfHBHFVEYnBcjBcz8O6TPgcp4+ks7cPQo4d+KWthwh/Mo7I3SU7hD84nwA4Iq0z Z/994twYsES+IjX/gXej+ylz53dAUXQ+X6AOR2D2d/+1O89qewkI0TcvhEyIavfm5idNmMn12Je jliEH8lOlVFrCwTJvjfK57bD2tIbqKSNg6FAuMPKoOGf5zCQxfLI8jq/c8kWWVea0HkllcLnDQq fM0iFfKNHrc4Iv7y2uC/Cs9Hw1w6kT5uWbAhrwA4BMexoRFw9fwJ5zG5rtRoGLqwE1eX7LgTdIy yqHZo39+UeHWg/rRNRTbWQcnlyi+GGnoKqOQwOgXcJ5c/mJaIttUFVw== X-Received: by 2002:a05:600c:b96:b0:48a:5339:a46 with SMTP id 5b1f17b1804b1-48a53390f6cmr90666205e9.9.1776808937705; Tue, 21 Apr 2026 15:02:17 -0700 (PDT) X-Received: by 2002:a05:600c:b96:b0:48a:5339:a46 with SMTP id 5b1f17b1804b1-48a53390f6cmr90665755e9.9.1776808937145; Tue, 21 Apr 2026 15:02:17 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-21.inter.net.il. [80.230.25.21]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-488fc0b4c85sm383193925e9.0.2026.04.21.15.02.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 21 Apr 2026 15:02:16 -0700 (PDT) Date: Tue, 21 Apr 2026 18:02:14 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= Subject: [PATCH RFC v3 19/19] virtio_balloon: mark deflated pages as zeroed Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When host_zeroes_pages is set, the host has zeroed the balloon pages on reclaim. Use put_page_zeroed() during deflation so the freed pages are marked as zeroed in the buddy allocator, allowing the next allocation to skip redundant zeroing. put_page_zeroed() is best-effort: if the balloon is the sole holder (the common case), the zeroed hint reaches the buddy allocator via free_frozen_pages_zeroed(). If someone else holds a reference, the hint is silently lost. Once balloon pages are converted to frozen pages (no refcount), this can switch to free_frozen_pages_zeroed() directly. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- drivers/virtio/virtio_balloon.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index 165b123caa64..3058d48fc8de 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -299,7 +299,10 @@ static void release_pages_balloon(struct virtio_balloo= n *vb, =20 list_for_each_entry_safe(page, next, pages, lru) { list_del(&page->lru); - put_page(page); /* balloon reference */ + if (host_zeroes_pages && !page_poisoning_enabled_static()) + put_page_zeroed(page); + else + put_page(page); /* balloon reference */ } } =20 --=20 MST