From nobody Mon Jun 15 18:00:34 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C4C021DDC35 for ; Sun, 26 Apr 2026 21:47:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240062; cv=none; b=faRgmUgm+P1iFU4b5UuXrYvbyeplRVEPGX3EiB+h5rMqkZ5Q71pTuTsWvjqUDLNauep2e72uBaMRwsDpK57srm/vICvGRp/rV5npzg2w2WFkVKXQuXjD5khQXBKiBksuoAYoP6nRz3ZBnqpZZg21q6KcxwHbHC23i4Z8h23LpFE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240062; c=relaxed/simple; bh=3yv7g0qz2r8KeF7Q/4i8eq2vYhnDxLUrftD/JPsDNIo=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=pFRLyIynuJsSUNrLVuNEzBTFyRna7HJgFYIs3CM6yP95EvuQSpQSL0A7Agjb+l6GicrIT51jmGm+mpLYkhxMSKQD61dpiwA6t5hXR6idOTGL0abFnt14qphwo/8Es8EwWGnkyjdtLQq4dRxyjQjA+/2XIbHSz7B7hrs2m1XN1hw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=RXYaGlOg; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=G8WT0/XT; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="RXYaGlOg"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="G8WT0/XT" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777240059; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=yXr1u1aEvptlqN9vuxArrnVOz3uXJxzDs4ChyPf8Auw=; b=RXYaGlOgPfzRmZgzkBBEu2j+sDfnlHYDCmE9j6mhMAwgkGfChrn7rgZ6HAdg74ehBxaFFZ roOyL2rFodsCOdRW5To8N/2v5BePHvhthwadSNco9+/uxwphQQSZRPZBGUEuCzc1N+SKp2 +JMdkPheRRbkgiCYF2CZwUb7wk5EGok= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-259-VYuc8FxsMAuzGzhBGbKkYw-1; Sun, 26 Apr 2026 17:47:38 -0400 X-MC-Unique: VYuc8FxsMAuzGzhBGbKkYw-1 X-Mimecast-MFC-AGG-ID: VYuc8FxsMAuzGzhBGbKkYw_1777240057 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-43d1fec59c9so6059473f8f.0 for ; Sun, 26 Apr 2026 14:47:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1777240057; x=1777844857; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=yXr1u1aEvptlqN9vuxArrnVOz3uXJxzDs4ChyPf8Auw=; b=G8WT0/XTQ5ESRkPWYBgPF8qH0sfKQ722nffGAZMEr7BiKDzdOrjQvlgvz63Xou8YlU 1ZpKZcqoqL8GlZLDYylNP1ZCrj0QxTfRgDXy7gJUYqrkDf1mTkE53D6fNndB2Gh7Fw8p UScdP3vAENwPjXypIv3+EiaRRRrkU1jYgBCNIAlQs0qt0HZBqcznJZ3h3b3eKcK4Rm14 t7Mlbe5BoUouU0/DgX1zNwPnxMrV9Nzp+G3KzUaETgUIVP3R6vKcBKLF033AQWeIdLju jXcKlnlnBnaHzOFIZ8idFBLQ4JBUPqUcllLT5T4qT6sTADgBwev/i/iP3QBpBD8+Ju+k AeSA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777240057; x=1777844857; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yXr1u1aEvptlqN9vuxArrnVOz3uXJxzDs4ChyPf8Auw=; b=Ba8lARR+nJhGkHsTjq8IOXoI8hg3Ugvi+RicWS3je9KkBEke0puy3VWQXICJlMMdpX ZXZ3OxaCzVMuLOsSkYnCPptimYPQPILExs2A/lXu7XD3ZKDkQ18B9XRPxgfZIVYsiH45 fQCqG6mqmxd5ix6Ov+JRC35igawlBxP+K72aXCkdBdkR5p3WpoDV4cW1zUZzFNBdK9dK IvowB0ZiAVhTYPO/LmjZ2gtV/UzuBRGep516iomdM2Othp2MbA28Xv0MzIyMcCotLnQy fEtwVBzTGVVNJQkGWpq8tQfvbxHfEOidqPaYwbDstyDwCH46dFMDELY91tgByPy06f7B 4niQ== X-Gm-Message-State: AOJu0Yywsn7m3sjsokm25EwOsiffr1Gh4Rxxb+FmDfXya4t8qHrA2cDu UlwX+UYSX9FCCmAGWFQRyijWxE56/6g+IeLsCRdz4qzljXPPimFgT4tMVNeZ2L9lyXRUB9ZP5al FHWluBn2g3TnpkvJg/RO7PwKUaT2MiDgc8EAzQXcHeGvWsjYWEzDSldUj/ropMKH68Qpvy9JEzZ 0oQwUYKeK218xfEQbYn02Sh7QcENm/yNWr0GPghW0b1L4= X-Gm-Gg: AeBDiesZKOZVo/3D+Q0DMXUH1zHoIKw9JHrTEz0Wi0VHzHmS7kN+jrk8qySdXDCbn3b EclXbjEnuxPbctTtuY0VHhvB88iLNEbD2Ge4F7dSef8Mu2u6Pa8uFtF3rEPmfprwQ4pfw1wvuAt QOG6aw4ptSF+72UHcXinj702XsqxMW/vXqUsDDNVViF9IJW3SlNimYzImOExL/iNn5RKJqV9bXZ mZgTFNEjz576UVZs+B9KFYKCPbhXhbItNCjoolw06T6MKvwfWpHa47uG+sjx5ajhq1r9BmLz1Rm ImV4Kmoo7EGkVQauQzspPZFb5ru2Frqjd3xUc/yEZzb3+kgtTk8UAhoCVOztCznxx/x3YxMvFt0 QKgq4rRPjf4zHp8JahBgakfAqX2q7CySFrLF7mEl9U03zNAfYRoFYL+In X-Received: by 2002:a05:6000:4709:b0:441:2397:f40f with SMTP id ffacd0b85a97d-4412397f431mr33247012f8f.4.1777240057163; Sun, 26 Apr 2026 14:47:37 -0700 (PDT) X-Received: by 2002:a05:6000:4709:b0:441:2397:f40f with SMTP id ffacd0b85a97d-4412397f431mr33246952f8f.4.1777240056601; Sun, 26 Apr 2026 14:47:36 -0700 (PDT) Received: from redhat.com (IGLD-80-230-47-179.inter.net.il. [80.230.47.179]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-43fe4e4daf2sm73823721f8f.33.2026.04.26.14.47.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 26 Apr 2026 14:47:36 -0700 (PDT) Date: Sun, 26 Apr 2026 17:47:33 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Johannes Weiner , Zi Yan , Lorenzo Stoakes , "Liam R. Howlett" , Mike Rapoport , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Ying Huang , Alistair Popple Subject: [PATCH RFC v4 01/22] mm: move vma_alloc_folio to page_alloc.c Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move vma_alloc_folio_noprof() from an inline in gfp.h (for !NUMA) and mempolicy.c (for NUMA) to page_alloc.c. The declaration is moved outside the #ifdef CONFIG_NUMA block so both configs use the same real function. On NUMA, it calls the mempolicy allocation path as before. On !NUMA, it calls folio_alloc_noprof() directly. This prepares for a subsequent patch that will thread user_addr through the allocator: having vma_alloc_folio in page_alloc.c means user_addr can be passed to the internal allocation path without changing public API signatures or duplicating plumbing in both gfp.h and mempolicy.c. No functional change. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/gfp.h | 9 ++------- mm/mempolicy.c | 17 ----------------- mm/page_alloc.c | 28 ++++++++++++++++++++++++++++ 3 files changed, 30 insertions(+), 24 deletions(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index 51ef13ed756e..7ccbda35b9ad 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -318,13 +318,13 @@ static inline struct page *alloc_pages_node_noprof(in= t nid, gfp_t gfp_mask, =20 #define alloc_pages_node(...) alloc_hooks(alloc_pages_node_noprof(__VA_= ARGS__)) =20 +struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, + struct vm_area_struct *vma, unsigned long addr); #ifdef CONFIG_NUMA struct page *alloc_pages_noprof(gfp_t gfp, unsigned int order); struct folio *folio_alloc_noprof(gfp_t gfp, unsigned int order); struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order, struct mempolicy *mpol, pgoff_t ilx, int nid); -struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, struct vm_area_= struct *vma, - unsigned long addr); #else static inline struct page *alloc_pages_noprof(gfp_t gfp_mask, unsigned int= order) { @@ -339,11 +339,6 @@ static inline struct folio *folio_alloc_mpol_noprof(gf= p_t gfp, unsigned int orde { return folio_alloc_noprof(gfp, order); } -static inline struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, - struct vm_area_struct *vma, unsigned long addr) -{ - return folio_alloc_noprof(gfp, order); -} #endif =20 #define alloc_pages(...) alloc_hooks(alloc_pages_noprof(__VA_ARGS__)) diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 0e5175f1c767..f0f85c89da82 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -2524,23 +2524,6 @@ struct folio *folio_alloc_mpol_noprof(gfp_t gfp, uns= igned int order, * * Return: The folio on success or NULL if allocation fails. */ -struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, struct vm_area_= struct *vma, - unsigned long addr) -{ - struct mempolicy *pol; - pgoff_t ilx; - struct folio *folio; - - if (vma->vm_flags & VM_DROPPABLE) - gfp |=3D __GFP_NOWARN; - - pol =3D get_vma_policy(vma, addr, order, &ilx); - folio =3D folio_alloc_mpol_noprof(gfp, order, pol, ilx, numa_node_id()); - mpol_cond_put(pol); - return folio; -} -EXPORT_SYMBOL(vma_alloc_folio_noprof); - struct page *alloc_frozen_pages_noprof(gfp_t gfp, unsigned order) { struct mempolicy *pol =3D &default_policy; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 2d4b6f1a554e..0e6ec7310087 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5297,6 +5297,34 @@ struct folio *__folio_alloc_noprof(gfp_t gfp, unsign= ed int order, int preferred_ } EXPORT_SYMBOL(__folio_alloc_noprof); =20 +#ifdef CONFIG_NUMA +struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, + struct vm_area_struct *vma, unsigned long addr) +{ + struct mempolicy *pol; + pgoff_t ilx; + struct folio *folio; + + if (vma->vm_flags & VM_DROPPABLE) + gfp |=3D __GFP_NOWARN; + + pol =3D get_vma_policy(vma, addr, order, &ilx); + folio =3D folio_alloc_mpol_noprof(gfp, order, pol, ilx, numa_node_id()); + mpol_cond_put(pol); + return folio; +} +#else +struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, + struct vm_area_struct *vma, unsigned long addr) +{ + if (vma->vm_flags & VM_DROPPABLE) + gfp |=3D __GFP_NOWARN; + + return folio_alloc_noprof(gfp, order); +} +#endif +EXPORT_SYMBOL(vma_alloc_folio_noprof); + /* * Common helper functions. Never use with __GFP_HIGHMEM because the retur= ned * address cannot represent highmem pages. Use alloc_pages and then kmap if --=20 MST From nobody Mon Jun 15 18:00:34 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C03ED261B70 for ; Sun, 26 Apr 2026 21:47:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240065; cv=none; b=PhEtkfRCV9G/iMlZ5JvU92dJzqj3GcNZeZNxAbc50oks8Hxe3ic+AMRfggqZTBbhWQf9aNwkgqKSv60bfSIC96/Qqa9j/YAholjT4J4wVc58eVvKD/ulMxZks803SagUbwec0df9Pp0EqLQd879/FeA6IoJouBn9wNwnvT1f7wo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240065; c=relaxed/simple; bh=BxAAIgTOcFIabrBhXgct8q9SxWHk4OL7KtA+YehjDgA=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=O4HUw5xPOEN47zysItCx7wDz/Qk7zwzzpuT5cHLM/ucOJ2+ZdWo6qnLPXvDDY6VJSY02OLhh+3peYQvHP2SFQ925c95NuM+IFBe3WzqeqsrijK/SxvhXgBFv00nKuLM5euTBhBJQy/L5Pi+W58CvAJf6nF5Ub9oyAWiZrSUPsZE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Llcte4yr; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=G/Faw3gB; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Llcte4yr"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="G/Faw3gB" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777240062; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=xFat/tQU1LWbkfRr9XYlFioABzGDJGEMsD6O36sEfN8=; b=Llcte4yrHhUETUmPSla/1q4JOzSZROtoKsQoeD5g/X4W3SzBbvRMlqdpAdYzizYaeNEx0k dWWmsSFuXsMWDrG0eSLzZIOmQMaF7U2yrHt2mE9cHHse4ldxZk7dVE2vICG7A2HN3Gak4M JJf3munJIJZrf4DryCD5zxkSHOeYNsE= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-387-XRaKYWngM9G96IrKVr-4Ag-1; Sun, 26 Apr 2026 17:47:41 -0400 X-MC-Unique: XRaKYWngM9G96IrKVr-4Ag-1 X-Mimecast-MFC-AGG-ID: XRaKYWngM9G96IrKVr-4Ag_1777240060 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-4440c5944fbso448680f8f.2 for ; Sun, 26 Apr 2026 14:47:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1777240060; x=1777844860; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=xFat/tQU1LWbkfRr9XYlFioABzGDJGEMsD6O36sEfN8=; b=G/Faw3gBaxWN3U8PN8B9Qo5NamF4Ofj80kVE4wi5zgsRp4MYwxIhn1H45NaZr5j0bX KYSOMHuZO/LtpDbxPgwmjUAqQvnBwuebGenRyequKWVtZtbB2g4e5SfDNkQxiY2Jgfhu 3KUs1+jSKuZY5fipEkUhbHmWE30KnetSaRYpDi29ehaTO0qNFnaDbNvU6lHXe9NEOtn7 ZwAO9FbX/6shcDTp1qHOBSgdIHdVb34c8ZDPqnidq2dSVCZNLoTTnk3tA4vdJ05WoHE7 bB78wVTeb9aXzRHURpaGmSa4Pw2l7NHD9MZtLehNuwodcHSjfwC6080E6WwAM2sedGoi E09w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777240060; x=1777844860; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xFat/tQU1LWbkfRr9XYlFioABzGDJGEMsD6O36sEfN8=; b=XVOC7iCC7xhIY6C9ZlsFlf4X0DKe5FOl0Yzq2KTQqA1iAAwGD0IxVuXfR89Uo3dVFC B6HYCkLCU2MP1fqydGhd6rXC2nbljl5Ob6Ehlm14ZcJk1M8AN/gtlQoIUlDFZMgaazvW C5tfQQ1+2hQ1f0P1CKxsPugmUqsL5qGf6emib8MZW3d0yse5kKiHwvvMTEs1vMhmjNGi v/yG6JNRo6jMRvmSUenz+1UNCxsqNMZlAjfSXysPp2jVhskQlxj5xKn9ZTyOAPDLWlf4 SUtdl5A94DcdV24Z61IJeXO7eiXhPT1SG9LV1c0ecqygzcbt1xfupvguFrt4sUrvmj1v nh8g== X-Gm-Message-State: AOJu0YxcQ/mgGMoBfQDLuV73BkF/fDgVJVhScqDiwQLcsQsp9BvXMvN8 EBucV74W1FKPj1MkOCUmgP8CjkXAwC4qRWO+ZrVCVmf36bFwoV8h0lJTDexrF97RLpphVyRmXJy zfrEupGxkWbQgA7QBP/1m4NavssBK6mrby7Z4NXAe5DINDDadichjKGJDqSQwjoISoXRSVr6DTP tZY14HEIDWtPxqR3QJrzO+j/yI2eZr/AA+dAa1CAzu7JE= X-Gm-Gg: AeBDiesGvGkhZLr7KfiPJ+K9fWoPvMj7wr3wU3ezCKFmiektF+mkbh3W1rtmDa6fUG2 XJrb4GedjWy8s1G90gGmMqD1lJNpTU0F+fB53UyMlZ8RbyU7ZTr9QDzN8ZcpVNne21DclTJG+d5 YTGqrbeQfT+hD2s4paOKauhSLTEpk0+6PmVxAX6RJQi3BnjcIfARjg9B+ovYrD6XfVm9dVVp1dg /lmsXiEFM5ZyGxa7XLL7RyeqwKJJa8F2OhbD5W8H1WUbYFWT6SSkZH6DfRql25yqjrA2klhdhGp 4pU8P0JOZPusJkaF1YWta7pwGeBFDK3lV3LoRv8Ko54wtpXycjzytYHjdC0xqRoOfGm6VQskJ2G ZXWhYHTx0TTshkEXYTwErorCHeU0MuKX4zKDF0ff0Xe0WBKS4Q2oh4j1D X-Received: by 2002:a05:6000:2309:b0:43f:e211:6e6 with SMTP id ffacd0b85a97d-43fe3db9b6fmr62507629f8f.1.1777240060097; Sun, 26 Apr 2026 14:47:40 -0700 (PDT) X-Received: by 2002:a05:6000:2309:b0:43f:e211:6e6 with SMTP id ffacd0b85a97d-43fe3db9b6fmr62507584f8f.1.1777240059595; Sun, 26 Apr 2026 14:47:39 -0700 (PDT) Received: from redhat.com (IGLD-80-230-47-179.inter.net.il. [80.230.47.179]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-444eebec868sm795291f8f.31.2026.04.26.14.47.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 26 Apr 2026 14:47:39 -0700 (PDT) Date: Sun, 26 Apr 2026 17:47:36 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Johannes Weiner , Zi Yan , Lorenzo Stoakes , "Liam R. Howlett" , Mike Rapoport Subject: [PATCH RFC v4 02/22] mm: add vma_alloc_folio_user_addr Message-ID: <0652fa7c0ed9be9794f44cc1f10ad19ed5f3990a.1777223007.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add vma_alloc_folio_user_addr() which will be used in follow-up patches. It= takes a separate user_addr parameter for the cache-friendly zeroing hint, independent of the addr used for NUMA policy lookup. The NUMA interleave index is computed from (addr - vma->vm_start) >> (PAGE_SHIFT + order), so addr must be folio-aligned for correct NUMA placement. But the zeroing hint wants the exact fault address for cache locality. vma_alloc_folio() becomes a thin wrapper that passes addr for both. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/gfp.h | 4 ++++ mm/page_alloc.c | 17 +++++++++++++---- 2 files changed, 17 insertions(+), 4 deletions(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index 7ccbda35b9ad..7069b810f171 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -320,6 +320,9 @@ static inline struct page *alloc_pages_node_noprof(int = nid, gfp_t gfp_mask, =20 struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, struct vm_area_struct *vma, unsigned long addr); +struct folio *vma_alloc_folio_user_addr_noprof(gfp_t gfp, int order, + struct vm_area_struct *vma, unsigned long addr, + unsigned long user_addr); #ifdef CONFIG_NUMA struct page *alloc_pages_noprof(gfp_t gfp, unsigned int order); struct folio *folio_alloc_noprof(gfp_t gfp, unsigned int order); @@ -345,6 +348,7 @@ static inline struct folio *folio_alloc_mpol_noprof(gfp= _t gfp, unsigned int orde #define folio_alloc(...) alloc_hooks(folio_alloc_noprof(__VA_ARGS__)) #define folio_alloc_mpol(...) alloc_hooks(folio_alloc_mpol_noprof(__VA_A= RGS__)) #define vma_alloc_folio(...) alloc_hooks(vma_alloc_folio_noprof(__VA_ARG= S__)) +#define vma_alloc_folio_user_addr(...) alloc_hooks(vma_alloc_folio_user_a= ddr_noprof(__VA_ARGS__)) =20 #define alloc_page(gfp_mask) alloc_pages(gfp_mask, 0) =20 diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 0e6ec7310087..6d31a5c99e93 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5298,8 +5298,9 @@ struct folio *__folio_alloc_noprof(gfp_t gfp, unsigne= d int order, int preferred_ EXPORT_SYMBOL(__folio_alloc_noprof); =20 #ifdef CONFIG_NUMA -struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, - struct vm_area_struct *vma, unsigned long addr) +struct folio *vma_alloc_folio_user_addr_noprof(gfp_t gfp, int order, + struct vm_area_struct *vma, unsigned long addr, + unsigned long user_addr) { struct mempolicy *pol; pgoff_t ilx; @@ -5314,8 +5315,9 @@ struct folio *vma_alloc_folio_noprof(gfp_t gfp, int o= rder, return folio; } #else -struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, - struct vm_area_struct *vma, unsigned long addr) +struct folio *vma_alloc_folio_user_addr_noprof(gfp_t gfp, int order, + struct vm_area_struct *vma, unsigned long addr, + unsigned long user_addr) { if (vma->vm_flags & VM_DROPPABLE) gfp |=3D __GFP_NOWARN; @@ -5323,6 +5325,13 @@ struct folio *vma_alloc_folio_noprof(gfp_t gfp, int = order, return folio_alloc_noprof(gfp, order); } #endif +EXPORT_SYMBOL(vma_alloc_folio_user_addr_noprof); + +struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, + struct vm_area_struct *vma, unsigned long addr) +{ + return vma_alloc_folio_user_addr_noprof(gfp, order, vma, addr, addr); +} EXPORT_SYMBOL(vma_alloc_folio_noprof); =20 /* --=20 MST From nobody Mon Jun 15 18:00:34 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 18ABC19ADA4 for ; Sun, 26 Apr 2026 21:47:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240070; cv=none; b=BHWYNyv3PXmBsFAHJLNeyHiiLX4gjmG8NCTd9LQpTPVG407XsLwnjMsj/taBzP9zfGZKQG5fWnk0C0fv0sPI927y/MjAi1kiij0GvmIlkv52tkghvECIyKGCzwzBfaQsgK5fOh/V5Z/n4soSnSOypaQliR1SXy4L92K5qhu4UEo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240070; c=relaxed/simple; bh=hx0yTnAGjed+ZiGYNdOhGK/LkqBjrTZzlSaJNIMB3vo=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Z8S0V3m7KwsNl3qk1YbsSac+T2vnag9aM/SMPtKJlvy+nlJn8bXWWnfQMpSZZuwB5kW57WZaCrcn0bTyRiBr/YMdFswnInm6Ggf7HQ/bWXFXzlmfNpG88JOlb86TQEHZzO9TaD2r0vFbsoXGrGZdJ4DMkY60PDflalgZNY7iRMU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=c/OM9ECZ; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=roKQuDMg; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="c/OM9ECZ"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="roKQuDMg" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777240067; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=huPFgAdwBnP/c9GYBPeC0qLQn/5dXNlXXwvv1VfNs5g=; b=c/OM9ECZ14X8QSTIqK4bABIB0krhic5BfbE+8hzzD1ti2TmUzlRWwYQMLwEh4IAM5mBOpl 5P7cw6OTzGuA2EzmqOiFAgexV6lrsYrjy57Ns+kFvpBXH9VHou+l2LQsws6BCe3PmJcEhr KS6M+ydmUMCkMTnrtAW4vNYuOvBof4w= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-439-_WEXHoX-PuWTTYk-oyQdUA-1; Sun, 26 Apr 2026 17:47:45 -0400 X-MC-Unique: _WEXHoX-PuWTTYk-oyQdUA-1 X-Mimecast-MFC-AGG-ID: _WEXHoX-PuWTTYk-oyQdUA_1777240065 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-43d1dea12aaso15719f8f.1 for ; Sun, 26 Apr 2026 14:47:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1777240064; x=1777844864; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=huPFgAdwBnP/c9GYBPeC0qLQn/5dXNlXXwvv1VfNs5g=; b=roKQuDMg4PERnjcA0bJt1vo7CuK5G5PKt/Oq57d3bTRe66q+UBAT0MNSOjcOdH7uMS BdYoGkJxr2I5nQ/ZE4Q9YD8zhJkg/vn6FIv2KlJaySUYz3qTCcfCsL5pOZSr6uZJilL9 g42c3mAiML2YAWpTKru9Ew8I3CJ/YPTaC3d1z80ovtsbUMmchV25TQjwQnnAqSdbeIo2 uoWE+D6WTosGr31NNThR2RsMm7/ukYITzm2kpGMYvLiD8FAjJtnIOXPDJTBdqwBxn65G pDdFM1zZfbG0m2Mr9xJu3Np3zJBT4wvMZK/PANe8SetE5gdB5/N95jb+KhLNdv5kn74R /wfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777240064; x=1777844864; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=huPFgAdwBnP/c9GYBPeC0qLQn/5dXNlXXwvv1VfNs5g=; b=m2qrnF26QNJX0exA8Vsr00TKyMqil7wAIqL6EKUt41evc+mcMFn88Yfp3HXV3EFsg/ 10fUnuzs4gM1WLQUeSwklEF7q5yIQJSUGWKgi2cwoXPd3WXsAtPiFlmNoStCjYNo0YcG vdx+zlF6Np/T8pFnXsdrFQsj1CaAJ9VaPmiJmu79dhoEN96M7B6LWvcVFkf0769D9dg5 A6UbJcBGxBHEU4DjygLvTgQj9+CuGiGIxhrBO9E5qnuixZTm3AmdPN0gBybRVtLgri7d L3un/63laxSQWb5GSajdFiwfvUSFwVHxO1LtFjlKEv0bgdJMykyR1dlu0GGeNsSG/Uwu OcLw== X-Gm-Message-State: AOJu0Yy/OWm6E0GEZWzxbgm4dqB3AfJB07qKL59TXIXWymKNqVCdqQDt c29oDc04Vz+f/HOvROaKtxy6I3nW1c2A04hQOUYBWo4n71XmeMhWt9kbmJ6xY032nnSo38390r+ r/NTky83Yf3x0u1LSO8WUt81Ge6I4Swltt1P3BxFK/95YMlcdvPtOeUGeuVm46A4XNPXLDcXv7f YSKcCb3yKTlZdvFelMY3Vpfatric49Mr7F/EzlwcpO9Kw= X-Gm-Gg: AeBDievDOxZEXkq895WfXSqwGJA+16qTOiSmplw2dz4uTtnaKjeSDLkiFTqzvikr17H wRJLEH96mXnS5Wp9J6vY8/Bb7TxJExgticchQqcv4dY623UUhK14SJpIxa/ZLtykV7GaOQ/BF13 A0PQ9hG3vOLNs08kvbg7FaQ0FgY6gw+NODD3slh/RKRkBeDGtuvH/l1TVpIwT4BaTs3RvWG5hkz c7pYMxiPuTP27qo21e2hCY+dw5SSxDNqthKO+VZ7eH3Ec0flFUHPFiAVeIiWT+4suc5aFbP3iVi +X3q3eXAh/JAgpYm2ERN302Gxa/IOK0E9+hZNfoAecIQa3THV3aX7uAvaq5NaNmVu1dquJNTh6r 6kRocHo03qeEJdQwh99LJDTuxzQlyApgYi4opu/RVdzjdSnJxK0z59JGz X-Received: by 2002:a05:6000:240e:b0:439:ae2a:755e with SMTP id ffacd0b85a97d-43fe3e0b606mr61745937f8f.23.1777240064214; Sun, 26 Apr 2026 14:47:44 -0700 (PDT) X-Received: by 2002:a05:6000:240e:b0:439:ae2a:755e with SMTP id ffacd0b85a97d-43fe3e0b606mr61745852f8f.23.1777240063489; Sun, 26 Apr 2026 14:47:43 -0700 (PDT) Received: from redhat.com (IGLD-80-230-47-179.inter.net.il. [80.230.47.179]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-43fe4e46898sm81017698f8f.27.2026.04.26.14.47.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 26 Apr 2026 14:47:43 -0700 (PDT) Date: Sun, 26 Apr 2026 17:47:39 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Johannes Weiner , Zi Yan , Lorenzo Stoakes , "Liam R. Howlett" , Mike Rapoport , Muchun Song , Oscar Salvador , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo Subject: [PATCH RFC v4 03/22] mm: thread user_addr through page allocator for cache-friendly zeroing Message-ID: <83d63423ea5575de831afad200af32524df5958b.1777223007.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Thread a user virtual address from vma_alloc_folio() down through the page allocator to post_alloc_hook(). This is plumbing preparation for a subsequent patch that will use user_addr to call folio_zero_user() for cache-friendly zeroing of user pages. The user_addr is stored in struct alloc_context and flows through: vma_alloc_folio -> folio_alloc_mpol -> __alloc_pages_mpol -> __alloc_frozen_pages -> get_page_from_freelist -> prep_new_page -> post_alloc_hook user_addr is threaded through internal APIs only (__alloc_frozen_pages_noprof, __alloc_pages_mpol). Public APIs (__alloc_pages, __folio_alloc, folio_alloc_mpol) are unchanged. USER_ADDR_NONE ((unsigned long)-1) is used for non-user allocations, since address 0 is a valid userspace mapping. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- include/linux/gfp.h | 8 +++++++- mm/compaction.c | 5 ++--- mm/hugetlb.c | 36 ++++++++++++++++++++---------------- mm/internal.h | 12 +++++++++--- mm/mempolicy.c | 42 +++++++++++++++++++++++++++++++----------- mm/page_alloc.c | 44 +++++++++++++++++++++++++++++--------------- mm/slub.c | 4 ++-- 7 files changed, 100 insertions(+), 51 deletions(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index 7069b810f171..e275cc80e19e 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -226,6 +226,12 @@ static inline void arch_free_page(struct page *page, i= nt order) { } static inline void arch_alloc_page(struct page *page, int order) { } #endif =20 +/* + * Sentinel for user_addr: indicates a non-user allocation. + * Cannot use 0 because address 0 is a valid userspace mapping. + */ +#define USER_ADDR_NONE ((unsigned long)-1) + struct page *__alloc_pages_noprof(gfp_t gfp, unsigned int order, int prefe= rred_nid, nodemask_t *nodemask); #define __alloc_pages(...) alloc_hooks(__alloc_pages_noprof(__VA_ARGS__)) @@ -340,7 +346,7 @@ static inline struct folio *folio_alloc_noprof(gfp_t gf= p, unsigned int order) static inline struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned in= t order, struct mempolicy *mpol, pgoff_t ilx, int nid) { - return folio_alloc_noprof(gfp, order); + return __folio_alloc_noprof(gfp, order, numa_node_id(), NULL); } #endif =20 diff --git a/mm/compaction.c b/mm/compaction.c index 1e8f8eca318c..c1039a9373e5 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -82,7 +82,7 @@ static inline bool is_via_compact_memory(int order) { ret= urn false; } =20 static struct page *mark_allocated_noprof(struct page *page, unsigned int = order, gfp_t gfp_flags) { - post_alloc_hook(page, order, __GFP_MOVABLE); + post_alloc_hook(page, order, __GFP_MOVABLE, USER_ADDR_NONE); set_page_refcounted(page); return page; } @@ -1832,8 +1832,7 @@ static struct folio *compaction_alloc_noprof(struct f= olio *src, unsigned long da set_page_private(&freepage[size], start_order); } dst =3D (struct folio *)freepage; - - post_alloc_hook(&dst->page, order, __GFP_MOVABLE); + post_alloc_hook(&dst->page, order, __GFP_MOVABLE, USER_ADDR_NONE); set_page_refcounted(&dst->page); if (order) prep_compound_page(&dst->page, order); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 0beb6e22bc26..de8361b503d2 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1842,7 +1842,8 @@ struct address_space *hugetlb_folio_mapping_lock_writ= e(struct folio *folio) } =20 static struct folio *alloc_buddy_frozen_folio(int order, gfp_t gfp_mask, - int nid, nodemask_t *nmask, nodemask_t *node_alloc_noretry) + int nid, nodemask_t *nmask, nodemask_t *node_alloc_noretry, + unsigned long addr) { struct folio *folio; bool alloc_try_hard =3D true; @@ -1859,7 +1860,7 @@ static struct folio *alloc_buddy_frozen_folio(int ord= er, gfp_t gfp_mask, if (alloc_try_hard) gfp_mask |=3D __GFP_RETRY_MAYFAIL; =20 - folio =3D (struct folio *)__alloc_frozen_pages(gfp_mask, order, nid, nmas= k); + folio =3D (struct folio *)__alloc_frozen_pages(gfp_mask, order, nid, nmas= k, addr); =20 /* * If we did not specify __GFP_RETRY_MAYFAIL, but still got a @@ -1888,7 +1889,7 @@ static struct folio *alloc_buddy_frozen_folio(int ord= er, gfp_t gfp_mask, =20 static struct folio *only_alloc_fresh_hugetlb_folio(struct hstate *h, gfp_t gfp_mask, int nid, nodemask_t *nmask, - nodemask_t *node_alloc_noretry) + nodemask_t *node_alloc_noretry, unsigned long addr) { struct folio *folio; int order =3D huge_page_order(h); @@ -1900,7 +1901,7 @@ static struct folio *only_alloc_fresh_hugetlb_folio(s= truct hstate *h, folio =3D alloc_gigantic_frozen_folio(order, gfp_mask, nid, nmask); else folio =3D alloc_buddy_frozen_folio(order, gfp_mask, nid, nmask, - node_alloc_noretry); + node_alloc_noretry, addr); if (folio) init_new_hugetlb_folio(folio); return folio; @@ -1914,11 +1915,12 @@ static struct folio *only_alloc_fresh_hugetlb_folio= (struct hstate *h, * pages is zero, and the accounting must be done in the caller. */ static struct folio *alloc_fresh_hugetlb_folio(struct hstate *h, - gfp_t gfp_mask, int nid, nodemask_t *nmask) + gfp_t gfp_mask, int nid, nodemask_t *nmask, + unsigned long addr) { struct folio *folio; =20 - folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, NULL); + folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, NULL, a= ddr); if (folio) hugetlb_vmemmap_optimize_folio(h, folio); return folio; @@ -1958,7 +1960,7 @@ static struct folio *alloc_pool_huge_folio(struct hst= ate *h, struct folio *folio; =20 folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, node, - nodes_allowed, node_alloc_noretry); + nodes_allowed, node_alloc_noretry, USER_ADDR_NONE); if (folio) return folio; } @@ -2127,7 +2129,8 @@ int dissolve_free_hugetlb_folios(unsigned long start_= pfn, unsigned long end_pfn) * Allocates a fresh surplus page from the page allocator. */ static struct folio *alloc_surplus_hugetlb_folio(struct hstate *h, - gfp_t gfp_mask, int nid, nodemask_t *nmask) + gfp_t gfp_mask, int nid, nodemask_t *nmask, + unsigned long addr) { struct folio *folio =3D NULL; =20 @@ -2139,7 +2142,7 @@ static struct folio *alloc_surplus_hugetlb_folio(stru= ct hstate *h, goto out_unlock; spin_unlock_irq(&hugetlb_lock); =20 - folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask); + folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, addr); if (!folio) return NULL; =20 @@ -2182,7 +2185,7 @@ static struct folio *alloc_migrate_hugetlb_folio(stru= ct hstate *h, gfp_t gfp_mas if (hstate_is_gigantic(h)) return NULL; =20 - folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask); + folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, USER_ADDR_NO= NE); if (!folio) return NULL; =20 @@ -2218,14 +2221,14 @@ struct folio *alloc_buddy_hugetlb_folio_with_mpol(s= truct hstate *h, if (mpol_is_preferred_many(mpol)) { gfp_t gfp =3D gfp_mask & ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL); =20 - folio =3D alloc_surplus_hugetlb_folio(h, gfp, nid, nodemask); + folio =3D alloc_surplus_hugetlb_folio(h, gfp, nid, nodemask, addr); =20 /* Fallback to all nodes if page=3D=3DNULL */ nodemask =3D NULL; } =20 if (!folio) - folio =3D alloc_surplus_hugetlb_folio(h, gfp_mask, nid, nodemask); + folio =3D alloc_surplus_hugetlb_folio(h, gfp_mask, nid, nodemask, addr); mpol_cond_put(mpol); return folio; } @@ -2332,7 +2335,8 @@ static int gather_surplus_pages(struct hstate *h, lon= g delta) * down the road to pick the current node if that is the case. */ folio =3D alloc_surplus_hugetlb_folio(h, htlb_alloc_mask(h), - NUMA_NO_NODE, &alloc_nodemask); + NUMA_NO_NODE, &alloc_nodemask, + USER_ADDR_NONE); if (!folio) { alloc_ok =3D false; break; @@ -2738,7 +2742,7 @@ static int alloc_and_dissolve_hugetlb_folio(struct fo= lio *old_folio, spin_unlock_irq(&hugetlb_lock); gfp_mask =3D htlb_alloc_mask(h) | __GFP_THISNODE; new_folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, - nid, NULL); + nid, NULL, USER_ADDR_NONE); if (!new_folio) return -ENOMEM; goto retry; @@ -3434,13 +3438,13 @@ static void __init hugetlb_hstate_alloc_pages_oneno= de(struct hstate *h, int nid) gfp_t gfp_mask =3D htlb_alloc_mask(h) | __GFP_THISNODE; =20 folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, - &node_states[N_MEMORY], NULL); + &node_states[N_MEMORY], NULL, USER_ADDR_NONE); if (!folio && !list_empty(&folio_list) && hugetlb_vmemmap_optimizable_size(h)) { prep_and_add_allocated_folios(h, &folio_list); INIT_LIST_HEAD(&folio_list); folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, - &node_states[N_MEMORY], NULL); + &node_states[N_MEMORY], NULL, USER_ADDR_NONE); } if (!folio) break; diff --git a/mm/internal.h b/mm/internal.h index cb0af847d7d9..8e4616e42b4a 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -672,6 +672,7 @@ struct alloc_context { */ enum zone_type highest_zoneidx; bool spread_dirty_pages; + unsigned long user_addr; }; =20 /* @@ -887,24 +888,29 @@ static inline void prep_compound_tail(struct page *he= ad, int tail_idx) set_page_private(p, 0); } =20 -void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flag= s); +void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flag= s, + unsigned long user_addr); extern bool free_pages_prepare(struct page *page, unsigned int order); =20 extern int user_min_free_kbytes; =20 struct page *__alloc_frozen_pages_noprof(gfp_t, unsigned int order, int ni= d, - nodemask_t *); + nodemask_t *, unsigned long user_addr); #define __alloc_frozen_pages(...) \ alloc_hooks(__alloc_frozen_pages_noprof(__VA_ARGS__)) void free_frozen_pages(struct page *page, unsigned int order); +void free_frozen_pages_zeroed(struct page *page, unsigned int order); void free_unref_folios(struct folio_batch *fbatch); =20 #ifdef CONFIG_NUMA struct page *alloc_frozen_pages_noprof(gfp_t, unsigned int order); +struct folio *folio_alloc_mpol_user_noprof(gfp_t gfp, unsigned int order, + struct mempolicy *pol, pgoff_t ilx, int nid, + unsigned long user_addr); #else static inline struct page *alloc_frozen_pages_noprof(gfp_t gfp, unsigned i= nt order) { - return __alloc_frozen_pages_noprof(gfp, order, numa_node_id(), NULL); + return __alloc_frozen_pages_noprof(gfp, order, numa_node_id(), NULL, USER= _ADDR_NONE); } #endif =20 diff --git a/mm/mempolicy.c b/mm/mempolicy.c index f0f85c89da82..06403a3812b4 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -2406,7 +2406,8 @@ bool mempolicy_in_oom_domain(struct task_struct *tsk, } =20 static struct page *alloc_pages_preferred_many(gfp_t gfp, unsigned int ord= er, - int nid, nodemask_t *nodemask) + int nid, nodemask_t *nodemask, + unsigned long user_addr) { struct page *page; gfp_t preferred_gfp; @@ -2419,9 +2420,11 @@ static struct page *alloc_pages_preferred_many(gfp_t= gfp, unsigned int order, */ preferred_gfp =3D gfp | __GFP_NOWARN; preferred_gfp &=3D ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL); - page =3D __alloc_frozen_pages_noprof(preferred_gfp, order, nid, nodemask); + page =3D __alloc_frozen_pages_noprof(preferred_gfp, order, nid, + nodemask, user_addr); if (!page) - page =3D __alloc_frozen_pages_noprof(gfp, order, nid, NULL); + page =3D __alloc_frozen_pages_noprof(gfp, order, nid, NULL, + user_addr); =20 return page; } @@ -2436,8 +2439,9 @@ static struct page *alloc_pages_preferred_many(gfp_t = gfp, unsigned int order, * * Return: The page on success or NULL if allocation fails. */ -static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order, - struct mempolicy *pol, pgoff_t ilx, int nid) +static struct page *__alloc_pages_mpol(gfp_t gfp, unsigned int order, + struct mempolicy *pol, pgoff_t ilx, int nid, + unsigned long user_addr) { nodemask_t *nodemask; struct page *page; @@ -2445,7 +2449,8 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsig= ned int order, nodemask =3D policy_nodemask(gfp, pol, ilx, &nid); =20 if (pol->mode =3D=3D MPOL_PREFERRED_MANY) - return alloc_pages_preferred_many(gfp, order, nid, nodemask); + return alloc_pages_preferred_many(gfp, order, nid, nodemask, + user_addr); =20 if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && /* filter "hugepage" allocation, unless from alloc_pages() */ @@ -2469,7 +2474,7 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsig= ned int order, */ page =3D __alloc_frozen_pages_noprof( gfp | __GFP_THISNODE | __GFP_NORETRY, order, - nid, NULL); + nid, NULL, user_addr); if (page || !(gfp & __GFP_DIRECT_RECLAIM)) return page; /* @@ -2481,7 +2486,7 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsig= ned int order, } } =20 - page =3D __alloc_frozen_pages_noprof(gfp, order, nid, nodemask); + page =3D __alloc_frozen_pages_noprof(gfp, order, nid, nodemask, user_addr= ); =20 if (unlikely(pol->mode =3D=3D MPOL_INTERLEAVE || pol->mode =3D=3D MPOL_WEIGHTED_INTERLEAVE) && page) { @@ -2497,11 +2502,18 @@ static struct page *alloc_pages_mpol(gfp_t gfp, uns= igned int order, return page; } =20 -struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order, +static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order, struct mempolicy *pol, pgoff_t ilx, int nid) { - struct page *page =3D alloc_pages_mpol(gfp | __GFP_COMP, order, pol, - ilx, nid); + return __alloc_pages_mpol(gfp, order, pol, ilx, nid, USER_ADDR_NONE); +} + +struct folio *folio_alloc_mpol_user_noprof(gfp_t gfp, unsigned int order, + struct mempolicy *pol, pgoff_t ilx, int nid, + unsigned long user_addr) +{ + struct page *page =3D __alloc_pages_mpol(gfp | __GFP_COMP, order, pol, + ilx, nid, user_addr); if (!page) return NULL; =20 @@ -2509,6 +2521,14 @@ struct folio *folio_alloc_mpol_noprof(gfp_t gfp, uns= igned int order, return page_rmappable_folio(page); } =20 +struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order, + struct mempolicy *pol, pgoff_t ilx, int nid) +{ + return folio_alloc_mpol_user_noprof(gfp, order, pol, ilx, nid, + USER_ADDR_NONE); +} +EXPORT_SYMBOL(folio_alloc_mpol_noprof); + /** * vma_alloc_folio - Allocate a folio for a VMA. * @gfp: GFP flags. diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 6d31a5c99e93..c19eaf76607c 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1837,7 +1837,7 @@ static inline bool should_skip_init(gfp_t flags) } =20 inline void post_alloc_hook(struct page *page, unsigned int order, - gfp_t gfp_flags) + gfp_t gfp_flags, unsigned long user_addr) { bool init =3D !want_init_on_free() && want_init_on_alloc(gfp_flags) && !should_skip_init(gfp_flags); @@ -1892,9 +1892,10 @@ inline void post_alloc_hook(struct page *page, unsig= ned int order, } =20 static void prep_new_page(struct page *page, unsigned int order, gfp_t gfp= _flags, - unsigned int alloc_flags) + unsigned int alloc_flags, + unsigned long user_addr) { - post_alloc_hook(page, order, gfp_flags); + post_alloc_hook(page, order, gfp_flags, user_addr); =20 if (order && (gfp_flags & __GFP_COMP)) prep_compound_page(page, order); @@ -3959,7 +3960,8 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int o= rder, int alloc_flags, page =3D rmqueue(zonelist_zone(ac->preferred_zoneref), zone, order, gfp_mask, alloc_flags, ac->migratetype); if (page) { - prep_new_page(page, order, gfp_mask, alloc_flags); + prep_new_page(page, order, gfp_mask, alloc_flags, + ac->user_addr); =20 /* * If this is a high-order atomic allocation then check @@ -4194,7 +4196,8 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned= int order, =20 /* Prep a captured page if available */ if (page) - prep_new_page(page, order, gfp_mask, alloc_flags); + prep_new_page(page, order, gfp_mask, alloc_flags, + ac->user_addr); =20 /* Try get a page from the freelist if available */ if (!page) @@ -5072,7 +5075,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, struct zoneref *z; struct per_cpu_pages *pcp; struct list_head *pcp_list; - struct alloc_context ac; + struct alloc_context ac =3D { .user_addr =3D USER_ADDR_NONE }; gfp_t alloc_gfp; unsigned int alloc_flags =3D ALLOC_WMARK_LOW; int nr_populated =3D 0, nr_account =3D 0; @@ -5187,7 +5190,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, } nr_account++; =20 - prep_new_page(page, 0, gfp, 0); + prep_new_page(page, 0, gfp, 0, USER_ADDR_NONE); set_page_refcounted(page); page_array[nr_populated++] =3D page; } @@ -5212,12 +5215,13 @@ EXPORT_SYMBOL_GPL(alloc_pages_bulk_noprof); * This is the 'heart' of the zoned buddy allocator. */ struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order, - int preferred_nid, nodemask_t *nodemask) + int preferred_nid, nodemask_t *nodemask, + unsigned long user_addr) { struct page *page; unsigned int alloc_flags =3D ALLOC_WMARK_LOW; gfp_t alloc_gfp; /* The gfp_t that was actually used for allocation */ - struct alloc_context ac =3D { }; + struct alloc_context ac =3D { .user_addr =3D user_addr }; =20 /* * There are several places where we assume that the order value is sane @@ -5278,10 +5282,12 @@ EXPORT_SYMBOL(__alloc_frozen_pages_noprof); =20 struct page *__alloc_pages_noprof(gfp_t gfp, unsigned int order, int preferred_nid, nodemask_t *nodemask) + { struct page *page; =20 - page =3D __alloc_frozen_pages_noprof(gfp, order, preferred_nid, nodemask); + page =3D __alloc_frozen_pages_noprof(gfp, order, preferred_nid, + nodemask, USER_ADDR_NONE); if (page) set_page_refcounted(page); return page; @@ -5310,7 +5316,8 @@ struct folio *vma_alloc_folio_user_addr_noprof(gfp_t = gfp, int order, gfp |=3D __GFP_NOWARN; =20 pol =3D get_vma_policy(vma, addr, order, &ilx); - folio =3D folio_alloc_mpol_noprof(gfp, order, pol, ilx, numa_node_id()); + folio =3D folio_alloc_mpol_user_noprof(gfp, order, pol, ilx, + numa_node_id(), user_addr); mpol_cond_put(pol); return folio; } @@ -5319,10 +5326,17 @@ struct folio *vma_alloc_folio_user_addr_noprof(gfp_= t gfp, int order, struct vm_area_struct *vma, unsigned long addr, unsigned long user_addr) { + struct page *page; + if (vma->vm_flags & VM_DROPPABLE) gfp |=3D __GFP_NOWARN; =20 - return folio_alloc_noprof(gfp, order); + page =3D __alloc_frozen_pages_noprof(gfp | __GFP_COMP, order, + numa_node_id(), NULL, user_addr); + if (!page) + return NULL; + set_page_refcounted(page); + return page_rmappable_folio(page); } #endif EXPORT_SYMBOL(vma_alloc_folio_user_addr_noprof); @@ -6947,7 +6961,7 @@ static void split_free_frozen_pages(struct list_head = *list, gfp_t gfp_mask) list_for_each_entry_safe(page, next, &list[order], lru) { int i; =20 - post_alloc_hook(page, order, gfp_mask); + post_alloc_hook(page, order, gfp_mask, USER_ADDR_NONE); if (!order) continue; =20 @@ -7153,7 +7167,7 @@ int alloc_contig_frozen_range_noprof(unsigned long st= art, unsigned long end, struct page *head =3D pfn_to_page(start); =20 check_new_pages(head, order); - prep_new_page(head, order, gfp_mask, 0); + prep_new_page(head, order, gfp_mask, 0, USER_ADDR_NONE); } else { ret =3D -EINVAL; WARN(true, "PFN range: requested [%lu, %lu), allocated [%lu, %lu)\n", @@ -7818,7 +7832,7 @@ struct page *alloc_frozen_pages_nolock_noprof(gfp_t g= fp_flags, int nid, unsigned gfp_t alloc_gfp =3D __GFP_NOWARN | __GFP_ZERO | __GFP_NOMEMALLOC | __GFP_= COMP | gfp_flags; unsigned int alloc_flags =3D ALLOC_TRYLOCK; - struct alloc_context ac =3D { }; + struct alloc_context ac =3D { .user_addr =3D USER_ADDR_NONE }; struct page *page; =20 VM_WARN_ON_ONCE(gfp_flags & ~__GFP_ACCOUNT); diff --git a/mm/slub.c b/mm/slub.c index 0c906fefc31b..fc8f998a0fe1 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -3266,7 +3266,7 @@ static inline struct slab *alloc_slab_page(gfp_t flag= s, int node, else if (node =3D=3D NUMA_NO_NODE) page =3D alloc_frozen_pages(flags, order); else - page =3D __alloc_frozen_pages(flags, order, node, NULL); + page =3D __alloc_frozen_pages(flags, order, node, NULL, USER_ADDR_NONE); =20 if (!page) return NULL; @@ -5178,7 +5178,7 @@ static void *___kmalloc_large_node(size_t size, gfp_t= flags, int node) if (node =3D=3D NUMA_NO_NODE) page =3D alloc_frozen_pages_noprof(flags, order); else - page =3D __alloc_frozen_pages_noprof(flags, order, node, NULL); + page =3D __alloc_frozen_pages_noprof(flags, order, node, NULL, USER_ADDR= _NONE); =20 if (page) { ptr =3D page_address(page); --=20 MST From nobody Mon Jun 15 18:00:34 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 616F22505B2 for ; Sun, 26 Apr 2026 21:47:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240071; cv=none; b=E8k1vhXDqeETHtJSrJ0JoGao7jSRt/T2+v4CsX3aJUri1VY5pIPde9z46Q+5ox6hYcuLQls7OsS20hZSfKU55k4qcv65xgOtSVFG7LRcYRfz2A66tUlQSHdvMEe7nMOFvW2icz+VUAsTmsefSffbunZ04uKXotv9+IPd3StDalg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240071; c=relaxed/simple; bh=+zNBlTlQkFqQMbEC4hj3ywqszA8Y1ImDjehUW9ILJAs=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=hQMYkVK2qe8NGDZ22mRk1351vLoCycOfMFIT7SD2YlTJgxpgguy4nj9DwgA6E7H6azLWH38KPJMegkYPfVrHazLtQoZcTBb6pBHtx0mu4gVRYWhccMOiFWKgiOy94WxbyFrX1dLcHFJwH0GY0vOQipXj+93XlKzF8r/QnoE2Y68= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=UvKDtvnL; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=Bw8AeJPs; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="UvKDtvnL"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="Bw8AeJPs" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777240069; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Dn5xupL5V4qF15edJs01tgV6QOrsZ1JWzK8CN9u8yrs=; b=UvKDtvnL11JLzEMFQC6f0LRovIRu69QMmhq5he1lU410lFA9LUt0TquPfG1C+Cno8Mt6Ma d5SVTzXdP3ZnFbQBCHNlH3cid9FCLvhQdNwRBD/Bcc1EdxHzygftaOyT5VTyhLL5fPxTOk WSbp1t3sZ30nkC8EgNwIKJcjE46nmvQ= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-549-iQGdzugeNaqqKsiOsjrcyQ-1; Sun, 26 Apr 2026 17:47:48 -0400 X-MC-Unique: iQGdzugeNaqqKsiOsjrcyQ-1 X-Mimecast-MFC-AGG-ID: iQGdzugeNaqqKsiOsjrcyQ_1777240067 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-488ba2f4094so57572025e9.1 for ; Sun, 26 Apr 2026 14:47:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1777240067; x=1777844867; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Dn5xupL5V4qF15edJs01tgV6QOrsZ1JWzK8CN9u8yrs=; b=Bw8AeJPs/EXO/UdUp24JnkPfo101ccz+UThZIuQim1PT/zOYgRSup1emJFrgbQCrAH YGtrZQ5QQewnmwHNyhHSuVYjM6441gWMEliCkzuMl5YzvR+CGxMNeCO/ukpVug3yKko6 O9aS5RuE0rW1u9qi5w1MacpRfDs2dXofHc7Ou7tCD/OSZ9G5a5F7JxmL92IlCuY2dvOy 14RcVTmGHV4a7C1N9Y5ytlvtFzG+GIxWZRh5VpwiUvVMgCmxctyy2sI7yzbs7qYJno+F v19+JzH7l08Q6UjPV1dsH+4vxaf6FOp8eFtDUXMh3+EI2fL8WwViZawNMLvM8tq8p9MT obpg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777240067; x=1777844867; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Dn5xupL5V4qF15edJs01tgV6QOrsZ1JWzK8CN9u8yrs=; b=VAqN0o9gBQdp8lgjJ7YDoKGbppwspqjhDgWSWVOwDKYlPHyuygEU4CcRPg5pc8X2l1 m0tf8q4iJtU4PwWZi4FCTFMN/qHKSJoVoJ/4fLYfkwTC3Pu84P7AZ0stNdygwabybiMv NRiRu8hEIWaXomFXLsVEEqkPCWpzmAUOIMsB1Y5rlHWMtsgm/eR6NoBcaP2m7Z/i1I3M 4lByJbBx+lXl/QrobYtJA/502IgvTHBszRvHQedShWM9j9/tslKJ2y23e8o8JbwFeADw wA8+t0FVHuaJwvbpyFaspHlbWiyU7j1x0FotfX4B3FYKNKS5VdWsGIQD2rBiko6ljfN2 58ww== X-Gm-Message-State: AOJu0YwXoeqKdQ+CsJN03gQSioElDwbXSAeRWXEPwMnNZaHzfezVTLlJ sLIYBqBHp0I3oh9mrYCRWvDElDJx8ZlqDL0MUDojv2pyCuZTeykcjVYNzvi7ZKGogPk+vq+fMEn YOJk4ErZmn1XNS1bIItNckfebvtNaxgnV2MSD574F5uzrfYSP3toGYeTttcO1oZKB6Zpxj/IyOd 8KLw80gK/OvOuy98S6U/HdRLN+A4AcAAbp+rySeC6Etzs= X-Gm-Gg: AeBDiesh2MFoptXs0+4f3naHYGKCudA/EaD23cpQa8dsY11q6pqPpYrQD/xjVHb2m2U 56bW99Gsi8qVNTKL79+OoQVTAQgmYGGTOdwgp7NtZ/HmaZipLR+s7ZU1sx91huKj5WMGHrUBVlz /oFLAzD8jHWfrB/7bH/+Fbw9b7ySHx6KtZ0CMxy83YILVt/RfBNMjwqJrlFCqKxqfPI6jqcWzEt tCMey5kr9al4cMy77C3IZlTKvdNY8hHdBodhnH2tLYqI/i2GtxRjA/nC2dACDpoWkk+iETIAdO9 qflmCwkYfgzQAqNHgWdWD9jLaregk19UWCS4i3YVV4JlXS/J95OVavuTpHrgLhHQCXTTngEBmNy Wp7e3cALBs19bXF7PaUAhpwfhBA94iAnrK5JX4SmXvog8BpzHNFM1FdAZ X-Received: by 2002:a05:600c:348d:b0:48a:674d:e756 with SMTP id 5b1f17b1804b1-48a677401c7mr138299805e9.20.1777240066863; Sun, 26 Apr 2026 14:47:46 -0700 (PDT) X-Received: by 2002:a05:600c:348d:b0:48a:674d:e756 with SMTP id 5b1f17b1804b1-48a677401c7mr138299475e9.20.1777240066327; Sun, 26 Apr 2026 14:47:46 -0700 (PDT) Received: from redhat.com (IGLD-80-230-47-179.inter.net.il. [80.230.47.179]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-488fc10019bsm765742785e9.4.2026.04.26.14.47.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 26 Apr 2026 14:47:45 -0700 (PDT) Date: Sun, 26 Apr 2026 17:47:43 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Lorenzo Stoakes , "Liam R. Howlett" , Mike Rapoport Subject: [PATCH RFC v4 04/22] mm: add folio_zero_user stub for configs without THP/HUGETLBFS Message-ID: <1c5deda275078ffb432af62ae293af1f696d29a7.1777223007.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" folio_zero_user() is defined in mm/memory.c under CONFIG_TRANSPARENT_HUGEPAGE || CONFIG_HUGETLBFS. A subsequent patch will call it from post_alloc_hook() for all user page zeroing, so configs without THP or HUGETLBFS will need a stub. Add a macro in the #else branch that falls back to clear_user_highpages(), which handles cache aliasing correctly on VIPT architectures and is always available via highmem.h. Without THP/HUGETLBFS, only order-0 user pages are allocated, so the locality optimization in the real folio_zero_user() (zero near the faulting address last) is not needed. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- include/linux/mm.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 5be3d8a8f806..541d36e5e420 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -4718,6 +4718,9 @@ static inline bool vma_is_special_huge(const struct v= m_area_struct *vma) (vma->vm_flags & (VM_PFNMAP | VM_MIXEDMAP))); } =20 +#else /* !CONFIG_TRANSPARENT_HUGEPAGE && !CONFIG_HUGETLBFS */ +#define folio_zero_user(folio, addr_hint) \ + clear_user_highpages(&(folio)->page, (addr_hint), folio_nr_pages(folio)) #endif /* CONFIG_TRANSPARENT_HUGEPAGE || CONFIG_HUGETLBFS */ =20 #if MAX_NUMNODES > 1 --=20 MST From nobody Mon Jun 15 18:00:34 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7AA25219EB for ; Sun, 26 Apr 2026 21:47:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240074; cv=none; b=dmtcQTH2ByvT3sdtYHlaT0V8B6xDcChTSk6RDtsqWoc1mOt2YtzZ7E5UpDMsR9ohrNQdZ7obzYXtihNUZrm6HLvkEf3+VfUzWF6AWEB0RJtTqJzYsANwHQJtx+7TmK+Mmh5tnk2u4Rm9zRuckj7lTIvDWDEQlbhQSO1agkSRJzU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240074; c=relaxed/simple; bh=JlDejs9gYAIkoE2XJP/rDDJpTLoWc0jBBYztgwi8/CI=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=eQihJbN8O5EfLD5KIiy4MOVcEikpYhPhrCI1w8piEzrmvxecsFuSAUkhahHa9OUmK2PKKe7x3/pD3COzGoNSXzNeVt5JuNq5A9N92Q/ZGiLDyzcq06oXWHh9pFK8autpZI/ne3exS+Zd0V5haDi0bxDGOjRsq1Z8ORe7mbbOdag= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=a3uh4aj4; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=eTXpa9kq; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="a3uh4aj4"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="eTXpa9kq" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777240072; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Ri4XPy9sNyekDQ8kcO16tmq8sU4MreY6RcvDvpK2Y+0=; b=a3uh4aj4se+XzHVAbNm8N+S2YrIKEnIYTfxYeZlnWCN1fO10EfrNyQLu5zYNSZwiOEu6tk 4ruTktYQLL5IwVa/nwM5o1AysDkGfB8Df5KatynyW7BeRLIUQXQpz9CYL26qsHl3Z0m8jl INQpuh+V0I9hz+gn1O1L8XE+gNL6vvo= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-278-UeIGb3VpODG-YccR63vsKg-1; Sun, 26 Apr 2026 17:47:51 -0400 X-MC-Unique: UeIGb3VpODG-YccR63vsKg-1 X-Mimecast-MFC-AGG-ID: UeIGb3VpODG-YccR63vsKg_1777240070 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-440d0c4401aso6888464f8f.0 for ; Sun, 26 Apr 2026 14:47:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1777240070; x=1777844870; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Ri4XPy9sNyekDQ8kcO16tmq8sU4MreY6RcvDvpK2Y+0=; b=eTXpa9kqZoiCPqKQ+oKt7/OdryeIMjGaIdi1Sjil/0m6ALV8K8QCAAkVg+OK0BymFN iwnWTVxEXuDO972E8JAZtS+501XEIcUTnN+p9sbepqvZ7F9juKYGFLfG86NOb9hcvzA6 9hOO+fdRuqdgSioaUB0ds82ntJ6Z6wRoNkU23Z8kOsqMl5U95H4lehpo2mfAWFAX4eA9 SHRetLAN7rRDNbQfEdbVnJerJ60lYnz3Xc9ZmTeYm8HP13zKUM0mZPu/nNYoOjRbUg5e NTqVSObsitN7TO+jx/G/GNZjkwa/rgj/cYkumjMGQjVfm/zV0/x9ahPoaZnI8JvhKpQm R06g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777240070; x=1777844870; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ri4XPy9sNyekDQ8kcO16tmq8sU4MreY6RcvDvpK2Y+0=; b=p4RIqawS/5MwNvQYrRe3VgVH0e8Rnke0TZRmqLKtuVtqZZKcWKKd59AAPJ4uYdryYM hLWiYhmJ5xT586x59eHZDW009XDjI6Ck8JST+un/92wZUOa/rnKff4xWTVM2vpYLlCR+ 9MXDyDtBbarFA4CQOmtE7jlOaSynYNptSw6nt7YTa5K0XGH2p+S9IEQflaezyFhPMgbh Fd7AvmTBzqchFZuv5nDrsWpYPUpvmuSmMWxh59prkBnZeYHomvxP0B7gktzMlgy4zsJy wJbt5aio5zbf+i72WXwDjtm5nGyg7zN//RPLVM7JC0XN4BE3dI9lUtpoIRAknBrhGOLV F6Xw== X-Gm-Message-State: AOJu0YyNYcv6+VnMWky6cCbFz73ydqOTe3IiOjtOF+ODO30dqVjnOUd7 wQnSVPdwjwbBxxd71wd9bidRRt9R1tCXYOKjnvn8NzUq4Rn6H+1Wki1AXUOXXzPNS5biYuRIDMJ YLIYYZ5/ASTAvvVV2AcRN7Py020dgInjBqNqBZQm7eXec8i0GEgTz0Qj58seU7I2kLmllk3AUB/ q8Dp0NijZyAeT5qPHcAzFGmGePZHdgPv840Bxdzu7AiPc= X-Gm-Gg: AeBDiet5Vdaf24tIxwMTQM+qyB6XwxToBxJ3FrvaHK6s2QUicisoEEtv+n1/Ox7o2IF LixCXQdykbRiDtzFQMRo4XyrlWqVO+aZDcXJRhmF9wZ9tBVXVPZGGO3vl9+bwnx0q03BTMaIudv DL+G0DpoZXVAR7jCsaQbTue+dcUR+X6Edo2P8dkzA7nP/9O/6/lYJQ/EaVg5LV8nPVkHArt6Fi0 pMFLKgKylExuQ2KnFmrR/UvOuDO5IYRbHBmefUcKKNDt1xXVFKriedU5ZQVE5E3ngfxcsor69sV 7qFU57xCn+B2FiBJ2K2QwlBZmmEZjxn27OeubeMcMXAOUm9vFvw02NZhcBJK1PZzyD31CmPbqH2 ikRY4NCwTR//GdtYBQnKYr26PWyU/6oTZU7UdYpcp+CzagXTgLhgFDUBk X-Received: by 2002:a05:600c:4714:b0:48a:5574:3a5b with SMTP id 5b1f17b1804b1-48a55743d05mr350691915e9.27.1777240069647; Sun, 26 Apr 2026 14:47:49 -0700 (PDT) X-Received: by 2002:a05:600c:4714:b0:48a:5574:3a5b with SMTP id 5b1f17b1804b1-48a55743d05mr350691515e9.27.1777240069081; Sun, 26 Apr 2026 14:47:49 -0700 (PDT) Received: from redhat.com (IGLD-80-230-47-179.inter.net.il. [80.230.47.179]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-488fc10019bsm765744485e9.4.2026.04.26.14.47.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 26 Apr 2026 14:47:48 -0700 (PDT) Date: Sun, 26 Apr 2026 17:47:46 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Johannes Weiner , Zi Yan Subject: [PATCH RFC v4 05/22] mm: page_alloc: move prep_compound_page before post_alloc_hook Message-ID: <9c4c4cacb2cd5d1fd32219ab00f27e70600c3900.1777223007.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move prep_compound_page() before post_alloc_hook() in prep_new_page(). The next patch adds a folio_zero_user() call to post_alloc_hook(), which uses folio_nr_pages() to determine how many pages to zero. Without compound metadata set up first, folio_nr_pages() returns 1 for higher-order allocations, so only the first page would be zeroed. All other operations in post_alloc_hook() (arch_alloc_page, KASAN, debug, page owner, etc.) use raw page pointers with explicit order counts and are unaffected by this reordering. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/page_alloc.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index c19eaf76607c..04c03c56abec 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1895,11 +1895,11 @@ static void prep_new_page(struct page *page, unsign= ed int order, gfp_t gfp_flags unsigned int alloc_flags, unsigned long user_addr) { - post_alloc_hook(page, order, gfp_flags, user_addr); - if (order && (gfp_flags & __GFP_COMP)) prep_compound_page(page, order); =20 + post_alloc_hook(page, order, gfp_flags, user_addr); + /* * page is set pfmemalloc when ALLOC_NO_WATERMARKS was necessary to * allocate the page. The expectation is that the caller is taking --=20 MST From nobody Mon Jun 15 18:00:34 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 409DA24679C for ; Sun, 26 Apr 2026 21:47:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240079; cv=none; b=ueL2eJjLW/EEVpAXxxdmOXDl1DZzeSSpOKslRwAc2GFPOgMBoAs7dnxpICwJ5NRZkXliTOeYlEXSfA26RYHtMb4GdN79VTXqqBN48I0mme3pU+zv4m1REnjN9m0jpKOp5lLgHvypAdUBS7gHSbYBFTRu4ShRMBOkeHIyyrgzJe8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240079; c=relaxed/simple; bh=eoqyj7rvtCF30mdMOUUqj/fkCg1yPgf0J85njMHy46U=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=nnA+0isOfzk61mEJVR4AvjrmNFjd9DQSz78oai0qnEY16l+rASUiB3CwIhb8SEreY4L/Rs4gArcTj/MsWurEurQMH1Ayhcv6mVL7AtVAwFElPG0JmcZoIADwuiOiHkh4c89G99sBl8FpTEMrIhlQt4D1S/Pi84Iz9KMaaEHZ25I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=IKUGJIuo; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=oDyyP0R1; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="IKUGJIuo"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="oDyyP0R1" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777240075; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=XUA6w1XPigv17O5YNh9KAjtaq44ytBpbajfhTmLgsNY=; b=IKUGJIuo0B5nvL5/QhHuO+Khi0bd3LZrg93w4rGT9/kGRXKLtIHqciQhJ8PA21fZXdhojz 87+0FhNads8mZRPf7Bi0HHYrAGOB+Y8+Pwfp657Z6/D5LR6BnSo3M7l6yNX6IMBcjvRqrd CLM6BTRSqv364DS1C3S+hXukR/OHpsw= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-158-BzpSMdZqMpas-EnLsKCpgQ-1; Sun, 26 Apr 2026 17:47:54 -0400 X-MC-Unique: BzpSMdZqMpas-EnLsKCpgQ-1 X-Mimecast-MFC-AGG-ID: BzpSMdZqMpas-EnLsKCpgQ_1777240073 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-488bd1ee9e7so84268645e9.1 for ; Sun, 26 Apr 2026 14:47:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1777240072; x=1777844872; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=XUA6w1XPigv17O5YNh9KAjtaq44ytBpbajfhTmLgsNY=; b=oDyyP0R1ZgN8m+YxqMEJ/IMawN0XcVay0FZBi/W2aeuyRelJ6+whXKize0W+SBrBKb Uh1N9KTkTBEDLvqOo4f1Q5qXNdFnrN/dZEAFr88+6zZ8UoR03uKEtIYysD9w5HrZRBrc f+GEqMc63YuFmgsVRr9xm37xo12z24Y4IfSEbyWQVQHrgHOUwL8D4f6G9eAUksOcswkP P8dnJtNrHrMLYOrIta/2nfluKZVE8un8RokHHEYcMr+M6Apd9YdGzJIpkhBPufz+6re+ YbOiL01y63FIMz1EwpfRkAomY01XU0L6LlMMrUO9VCZXQUptRQYsTUI6t0HEtx927Z3F nx5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777240072; x=1777844872; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XUA6w1XPigv17O5YNh9KAjtaq44ytBpbajfhTmLgsNY=; b=eCVDFJqrBDi0AWXu7ZviKt3Wghw477HmlBURsgWdySf4KW1Cf2NV53uFb2IXiIoUS6 YmEaQL0qheAgcf6uqcn3uAZobnFI/xgpFG6umSG2efGgBODIWmypparaQZ915sP7rJW6 Dt4oRW/VAVMdtcMGHXx8wNxVzkWmBH+f4oX9ZRgrfwIMTh1ZA9HI1fRXPHOuBmbBxVV1 wCvzomp2bwJEHl9FenCR9wvqPxi9MNxxLejpNrDMxYw8cLzareRVXiz0ZAsMWO5ozY+l zyIeaUc4Bqxl7qhiQ6KEaBZXyaF7wRL2hF02GHd0CLLHvUV71OplJrX6wTYuLLZ/eOQG gYVg== X-Gm-Message-State: AOJu0Yw6RsJ29ajyzkRQTnS+l1oV8rojQ/LcC/NyiMRUHklXn/WkH0Ad 80a8Gdeb6VscTW+Os9wW3T7YKIuqXhc2b+dY2rX9IJJbk5F4Kh4E8QpB3up4bmAsiNYc9CKjLMj c9URCpdUNjCDalJctqHVD4S0LsH8nfTmJH5hBrW5YkvOCkeGCinOG+wwN1+MD/DRls9eENEquyb U33uaUUjESoGAlkPyPaaO64LdJgTtWI0XMKYABGUoly/0= X-Gm-Gg: AeBDiespkMF5Pbi95TGHnghU8/n07vcxDdkhnqFkru3Y6DEywlDgDHNW7jVHpJAY7OT MgA4DLOsL5puIHLjm6kYlV4yHbR/ufF3d8cRcs9/H+VhFo9Lx/BHlJJMDVbD8gdFwWToc8wwupq ItydFxIXpvaCSfWzQuibX8rGLcZ1P90GwmoDfErZF9YOh5HZ93eUPAWc0Hwkez36UHoqcDxEvKZ U1QljzYIXCuZQzI6+NrvcLMsZrStPIs9RyWzFLped/1ut8RQhbStE7Nln5Kw3YUyKTU4sblPVwi 17+1Ity4E+BtFv95w9Ikjj+6rC1MbKqbagkbUqb/P0/y3E+pZulv2vP/jXQw5C9YVzeapQVzqb4 8xBXl7xzFlGvrC1Oo6ZmNJyWlItDwOmt4CDE8J303+TMteqqzLk/DR18j X-Received: by 2002:a05:600c:2d4b:b0:489:c57:75da with SMTP id 5b1f17b1804b1-4890c57760emr286456175e9.31.1777240072497; Sun, 26 Apr 2026 14:47:52 -0700 (PDT) X-Received: by 2002:a05:600c:2d4b:b0:489:c57:75da with SMTP id 5b1f17b1804b1-4890c57760emr286455905e9.31.1777240072037; Sun, 26 Apr 2026 14:47:52 -0700 (PDT) Received: from redhat.com (IGLD-80-230-47-179.inter.net.il. [80.230.47.179]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-488fb75c695sm225077675e9.14.2026.04.26.14.47.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 26 Apr 2026 14:47:51 -0700 (PDT) Date: Sun, 26 Apr 2026 17:47:49 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Johannes Weiner , Zi Yan Subject: [PATCH RFC v4 06/22] mm: use folio_zero_user for user pages in post_alloc_hook Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When post_alloc_hook() needs to zero a page for an explicit __GFP_ZERO allocation and user_addr is set, use folio_zero_user() instead of kernel_init_pages(). This zeros near the faulting address last, keeping those cachelines hot for the impending user access. folio_zero_user() is only used for explicit __GFP_ZERO, not for init_on_alloc. On architectures with virtually-indexed caches (e.g., ARM), clear_user_highpage() performs per-line cache operations; using it for init_on_alloc would add overhead that kernel_init_pages() avoids (the page fault path flushes the cache at PTE installation time regardless). No functional change yet: current callers do not pass __GFP_ZERO for user pages (they zero at the callsite instead). Subsequent patches will convert them. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/page_alloc.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 04c03c56abec..7791bc1eeefa 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1882,9 +1882,20 @@ inline void post_alloc_hook(struct page *page, unsig= ned int order, for (i =3D 0; i !=3D 1 << order; ++i) page_kasan_tag_reset(page + i); } - /* If memory is still not initialized, initialize it now. */ - if (init) - kernel_init_pages(page, 1 << order); + /* + * If memory is still not initialized, initialize it now. + * When __GFP_ZERO was explicitly requested and user_addr is set, + * use folio_zero_user() which zeros near the faulting address + * last, keeping those cachelines hot. For init_on_alloc, use + * kernel_init_pages() to avoid unnecessary cache flush overhead + * on architectures with virtually-indexed caches. + */ + if (init) { + if ((gfp_flags & __GFP_ZERO) && user_addr !=3D USER_ADDR_NONE) + folio_zero_user(page_folio(page), user_addr); + else + kernel_init_pages(page, 1 << order); + } =20 set_page_owner(page, order, gfp_flags); page_table_check_alloc(page, order); --=20 MST From nobody Mon Jun 15 18:00:34 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A831C2F25F4 for ; Sun, 26 Apr 2026 21:48:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240082; cv=none; b=S8KFAQhLYtn7wRm8xNaBzqgzUxrnlzhAfTaFPntoJ6zQW7wmDuO/nRjnZyJ2OBowNg7a+fEIWl6vwU7oBvzC5TRE0zp2pihO1mk3OgVL83NHYsoBwDuIq21CRzHRmiRYWgny1pVVmJEV48mRxh6rI8hho0wWJcdpunmwLwy89RI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240082; c=relaxed/simple; bh=B4v7SD6SJ84KapLgBy+iXreANiiaF75HE81bu1RBt3E=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=BzFgAxc+0o62/sBr1SFQ+KF2WDp7wip9SYSBA651I6jSUpiN3/ph+5bom9qVYArAsEykoUaD22CqMYYA4u2Zc7pFOVSnAZNOVML41TQ9/jaln8miOj3oXjtrIEJI39MlUS/ulfaxLPkbd6dJeca8PhWJ6j9deVlcQyq6JjRGfe0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Qqbsm0MX; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=abbXjOrL; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Qqbsm0MX"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="abbXjOrL" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777240079; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=jpAQ9O1oMgUNlo5ukg9OEno1SM1k+70VK2/jH7/8vVM=; b=Qqbsm0MXf1mnZ/kjXu3Ky/u1WujklA1DeCnV78Z/eb7ze2TfVAY/egQxw6P5/Kvo4Rf9rC fraZ3fldaSb4mekhoAP7/kZHWnS+PXu+XNf4vWeMC2AsjYSFF6qSiHopkZgdmGXn/Rx9Za ghyETr/qF2oPlE3wxgwxhqeP+jU5CEM= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-640-PxfrDqRbNSO2NRRdxSY9bg-1; Sun, 26 Apr 2026 17:47:58 -0400 X-MC-Unique: PxfrDqRbNSO2NRRdxSY9bg-1 X-Mimecast-MFC-AGG-ID: PxfrDqRbNSO2NRRdxSY9bg_1777240077 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-48a5523fd53so31258665e9.3 for ; Sun, 26 Apr 2026 14:47:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1777240077; x=1777844877; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=jpAQ9O1oMgUNlo5ukg9OEno1SM1k+70VK2/jH7/8vVM=; b=abbXjOrLWo5I9bXU+FqxjYAtv64OdLLvR+G08NxougZv3yea2IK2H10MgIHCc+kA9B 5Mv1vlviGemq1wsDfEFsWCyhRfCib+Nb82S7zZmiVWuJfebuIccihqMGXyJNaUXHSPg2 a2+flaqNVpeSQ//EyEHGrUITVG2EjEm2PY7WomAIUhKVaz1Nzr5/v+4GvuirD8mnc9r5 3cD7UepI+mkx2WZtWN8ag3Lozx1PMc6t1YUdLLYjNL5EwN4Wt14awAaet47IUoF5bHMw x+k2mAhpjbGkQvn9mT1Y5H45tIhRp1B6aW7Co2aTSnh+cL98i8CMlhDLnnR0YZlCz9Lt xi9g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777240077; x=1777844877; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jpAQ9O1oMgUNlo5ukg9OEno1SM1k+70VK2/jH7/8vVM=; b=Nj88t9fWKOAPmW7L4BaVscNCqebJ4peAlHbBDCBqrHsfK7/44qo1nTUBjA64Plaz0C 19jHLwF8P5MTAFTYhw7SBOt0ZZrtH3O/J2mez3C4u6i4xVx5QG6tnx50OaNoR0futab9 eck+jzpsoEGCwjEB4YO1haIFUsrodqZkQQycGdcEXI9lPY9xo/vODUqreK8Obk0DKEI6 gv1LWc7Jf31o9bHTgFafLWSD4RGMUYm9Y74PzNgaqhUp37lhrTwnsytsy12rq2INJoYP VFnkC1kmTxA8VHuxD76E365bZGv+TdaCquiBplLq78mpLG8BKIGWN6aJHu+zXMOAOLF5 wKdA== X-Gm-Message-State: AOJu0YyhS+60uNxX6+jHpHJZ++h932hXy9d/7u5JShcCX0G/UtD6JpDV 7AJrza+ZmA+RnErdZr0MQdNMoM7sL3ih5k+T1hIGY9g4AtC1KU6Z4VSxu2firlyy7LfcF09XcUf xhDxRxCW7+FTwIr6vRNSU16MO8PDSZsnjFQzVCl3JNiS+D4YjiSn9R67Xb1usPgtX/xcm/jNft6 pOAGyX1H7F7xhrzoR2L/EU4BkTXuX4X9T3Wih1zK3aNnI= X-Gm-Gg: AeBDiesocDDeZ6J5H6Cyl52rZt1hT+xwlcYehvqk1R731Xv3SUd22oJdh4+lBE+WNqj OSHCjPl8uIucwtj6fV5d8K/D3hy2TqECqtIrE2sMOu0EGrujG4AFxuCy08HK9KC8VVcf/DwqzCo XccMNQ4DR1auoMrtYRIaUbPAfDD2SSSAypbwCoqWRI75oWqdQv0an+C45tF+aGkkI8tbbdu5QgT OzjkM95CdpkQjbVB+vEQq1hcbG48+w0X9HSICUaeNz7og78pSyspxpf58IOAbHnwyzd9LA9R4BF aV47ycFPWp2QB1a11ft3UKHQFc5jeTUgjkR9syC4iZezT7obxKqpHPXebXcfM5Ep0FtpQi7kfjc mMywjal9HPtEbigoDD+1FKm3riNLHtgGpjFR4KUYMfUhV3qZSNQB/GiH7 X-Received: by 2002:a05:600c:348d:b0:48a:674d:e756 with SMTP id 5b1f17b1804b1-48a677401c7mr138304075e9.20.1777240076895; Sun, 26 Apr 2026 14:47:56 -0700 (PDT) X-Received: by 2002:a05:600c:348d:b0:48a:674d:e756 with SMTP id 5b1f17b1804b1-48a677401c7mr138303755e9.20.1777240076353; Sun, 26 Apr 2026 14:47:56 -0700 (PDT) Received: from redhat.com (IGLD-80-230-47-179.inter.net.il. [80.230.47.179]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48a525a0b1asm218200665e9.2.2026.04.26.14.47.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 26 Apr 2026 14:47:55 -0700 (PDT) Date: Sun, 26 Apr 2026 17:47:52 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Lorenzo Stoakes , "Liam R. Howlett" , Mike Rapoport Subject: [PATCH RFC v4 07/22] mm: use __GFP_ZERO in vma_alloc_zeroed_movable_folio Message-ID: <5cfba90528b7abc1ebee68ac17efb6e3813fd916.1777223007.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that post_alloc_hook() handles cache-friendly user page zeroing via folio_zero_user(), convert vma_alloc_zeroed_movable_folio() to pass __GFP_ZERO instead of zeroing at the callsite. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/highmem.h | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/include/linux/highmem.h b/include/linux/highmem.h index af03db851a1d..ffa683f64f1d 100644 --- a/include/linux/highmem.h +++ b/include/linux/highmem.h @@ -320,13 +320,8 @@ static inline struct folio *vma_alloc_zeroed_movable_folio(struct vm_area_struct *vma, unsigned long vaddr) { - struct folio *folio; - - folio =3D vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, vma, vaddr); - if (folio && user_alloc_needs_zeroing()) - clear_user_highpage(&folio->page, vaddr); - - return folio; + return vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, + 0, vma, vaddr); } #endif =20 --=20 MST From nobody Mon Jun 15 18:00:34 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7506221B9F6 for ; Sun, 26 Apr 2026 21:48:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240085; cv=none; b=RUanHRs/ikSLkSpPqyeSS3PWEA23Y+wUiEY6jZOK9V1BYf21bLxPUQNsY+tQscpwFNBHWYS7vIh9s4oxQgbDixiOuh9y/3/78AmkphuOv4ZzucVOp6Jnl64oKUYejsaoTlqUEiuGWELzIpXuSwAdV68rqvY5l7kGle1HI1yMDag= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240085; c=relaxed/simple; bh=NvzxDfyhBY8HhpKlujoOhD2X7P7j0RP/UcBg8w+dASc=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=sm4nbWhRKnmXt1EJl7xCtoNRKbRk9NPvhKyrzi/UjkzqgeCTwc6eq7QPEO1w/LWc01K0x6fsMOCbcoWIjFn3gsUAfBz4zowca3ToVHLQ4YaTgK1HY7bu6UaG/3JBFGy2eRD5BULSBFKGEGa5qTyHGGwa6iUfCgOeXmc2yqyuOiI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=YCfr5suO; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=IkKLt60a; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="YCfr5suO"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="IkKLt60a" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777240082; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=lya+idGqxu4DlV/pLONwdjLPocorK4fxVlb4E+6fg/I=; b=YCfr5suOEKP7qfopaJIw10JGnMAPzUp40TQk3hKE8HSvSjSPR7uk0ulo19R3AxqDOMtMj4 GE34EJ6MWLJt4RGhNu4TayF+h6aFEx4wDtrWN4nZaA2Idb/zSxdzPiuAa9DwpVA0Dlx9Ww t+B3nZXqMh4UllMhkl/QGKt5fStF50k= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-15-AW2CRkq4OCmshpJLgKPXWg-1; Sun, 26 Apr 2026 17:48:01 -0400 X-MC-Unique: AW2CRkq4OCmshpJLgKPXWg-1 X-Mimecast-MFC-AGG-ID: AW2CRkq4OCmshpJLgKPXWg_1777240080 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-43d1dea12aaso15818f8f.1 for ; Sun, 26 Apr 2026 14:48:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1777240080; x=1777844880; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=lya+idGqxu4DlV/pLONwdjLPocorK4fxVlb4E+6fg/I=; b=IkKLt60at9chVdJNk30U/f8SbSscrkRIpVRxLgLd6eETY+Q3zuYVTGzi21RiVjKq1a TS0eGG7sK6UXc9HYPV3hy5RN5B3nu312ui+b9ZOPmtsBMGPOGDBWv4uGOFLRnW6qTesS BrUvDQqK5LlM0MPmyP27aifXK+jP6Lcd4wsqj2exV1EKCKbUnLoON7zh3Hk8tSw42qsG Z/sN9Qzb2bBGyYMleAdcNtYR3Ysibef02JGeIu60yaSfdJfGg6eonhXIxDgCCHRItf3x DWRduwTtsezX3R4dVRBlJsuZFt73S42Yqbw96rzV8H9WWPIsiUdYtcwT0JnPXEeWjO4D CG9w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777240080; x=1777844880; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lya+idGqxu4DlV/pLONwdjLPocorK4fxVlb4E+6fg/I=; b=bDnu+YUPejEE+cI+JK0oQhtDMG5iNUBPSLY3SpbZgZf8SW7Hs9tBBdAV5TlBR8shCF TVUIM6L8mtYD2iJV4Ra+RN5zcl+j5hm99FrF3VYEzcXComNNlRzc/ouSm3+jO9b8AExV C6KQ/WuWBiBhSwp7zAjg5BP7/xu8gKlf4Vfkm3Nqwq2CDdWw+ikVsteaAGriC5+A/Mzj Kw3sQZTUs5GqGMVJuvqToXgHaFM1gFDOBSMczRklPp4L/NEpKwUHME1EfcVXTj9gxJdB 8X3Qs33TTx8tS4bVM9qDLTgw+5IpTmRUtmrJVvgxjmimXUG0x2B08SY44+w2ffpgAnol mBPQ== X-Gm-Message-State: AOJu0Ywzi/2y18dQ+biujjNP0idRiWaLkh55ADPCiYM98pfAYP3Y4NDx DbxWPCiok4L82TzkFtNc/+ekQDc3PeoVuqDdsCMMmz+tXA6AlRO3ULMWasMrjhyXL2rJiDOt1Vv 5Y6cH1KbHhfFQLzqIAt3GKQSqqbv+Cwu94k62qZMh9D6qW0e3CVnyDJmhy/MR6kXWdpRX2D9a19 cBYFFvWSMdIc9QyBIDD+8BbjVUNBwlNxk2y+XrymeK3Rc= X-Gm-Gg: AeBDieteLw26RvsVlR9RN5PiJlLGqKwWcYuqXNMq52uW6zjLs30YDtfqj4jflC1cwha PE4d7cxgowQFoMUvYrI49qi0V+5zassBHwVFRsyRtEe2ZewIYPLgrpZr6t9Z1DicGGp5j0wW0bv aPEHrNubgJ66DDn6N4WimG3WQ6ORs0ve573X4LzxBm7K/jG8yrg3yt9Mg8QJNpoTpR3ywF4eRI/ B8qZBuJO+M9oRbJbIViHnYTPViK/dkkXBKyaB1f0581CCa1OKEXcFGc1mTTiYna3JqfdNxlg0PT 2LhHKV6C6jMvCUfaJXwGA0mWYksf8Eczrfl8fKBJredNQY/3z2xRejAEeWf+V5A94qG43miAV3c 4Yt4YwfiNq/xiSvhZhi10RG1ffCeim4ggk1XvVwPfiIlE1jgpT9SWMxUK X-Received: by 2002:a05:6000:420a:b0:43d:773d:78ff with SMTP id ffacd0b85a97d-43fe3e0b61bmr62764575f8f.27.1777240079920; Sun, 26 Apr 2026 14:47:59 -0700 (PDT) X-Received: by 2002:a05:6000:420a:b0:43d:773d:78ff with SMTP id ffacd0b85a97d-43fe3e0b61bmr62764530f8f.27.1777240079334; Sun, 26 Apr 2026 14:47:59 -0700 (PDT) Received: from redhat.com (IGLD-80-230-47-179.inter.net.il. [80.230.47.179]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-43fe4e46898sm81019233f8f.27.2026.04.26.14.47.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 26 Apr 2026 14:47:58 -0700 (PDT) Date: Sun, 26 Apr 2026 17:47:56 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Lorenzo Stoakes , "Liam R. Howlett" , Mike Rapoport Subject: [PATCH RFC v4 08/22] mm: use __GFP_ZERO in alloc_anon_folio Message-ID: <28642fd392362fff6d84d7d16acf9653c43cbcb3.1777223007.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Convert alloc_anon_folio() to pass __GFP_ZERO instead of zeroing at the cal= lsite. Use vma_alloc_folio_user_addr() to pass the folio-aligned address for NUMA policy and the raw vmf->address for cache-friendly zeroing. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/memory.c | 17 +++++------------ 1 file changed, 5 insertions(+), 12 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 07778814b4a8..83ec73791fae 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4662,7 +4662,8 @@ static struct folio *alloc_swap_folio(struct vm_fault= *vmf) gfp =3D vma_thp_gfp_mask(vma); while (orders) { addr =3D ALIGN_DOWN(vmf->address, PAGE_SIZE << order); - folio =3D vma_alloc_folio(gfp, order, vma, addr); + folio =3D vma_alloc_folio_user_addr(gfp, order, vma, addr, + vmf->address); if (folio) { if (!mem_cgroup_swapin_charge_folio(folio, vma->vm_mm, gfp, entry)) @@ -5176,10 +5177,11 @@ static struct folio *alloc_anon_folio(struct vm_fau= lt *vmf) goto fallback; =20 /* Try allocating the highest of the remaining orders. */ - gfp =3D vma_thp_gfp_mask(vma); + gfp =3D vma_thp_gfp_mask(vma) | __GFP_ZERO; while (orders) { addr =3D ALIGN_DOWN(vmf->address, PAGE_SIZE << order); - folio =3D vma_alloc_folio(gfp, order, vma, addr); + folio =3D vma_alloc_folio_user_addr(gfp, order, vma, addr, + vmf->address); if (folio) { if (mem_cgroup_charge(folio, vma->vm_mm, gfp)) { count_mthp_stat(order, MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE); @@ -5187,15 +5189,6 @@ static struct folio *alloc_anon_folio(struct vm_faul= t *vmf) goto next; } folio_throttle_swaprate(folio, gfp); - /* - * When a folio is not zeroed during allocation - * (__GFP_ZERO not used) or user folios require special - * handling, folio_zero_user() is used to make sure - * that the page corresponding to the faulting address - * will be hot in the cache after zeroing. - */ - if (user_alloc_needs_zeroing()) - folio_zero_user(folio, vmf->address); return folio; } next: --=20 MST From nobody Mon Jun 15 18:00:34 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A9F2E261B70 for ; Sun, 26 Apr 2026 21:48:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240091; cv=none; b=GCrTol9z6k+Qqqg+eSrbGuK+G8GjX+EOwbWRJZ9J0J9fwwpKDHH8v6E4lDJ2YunClJd017sisMtLEJanvXLbpx0vbvTA0gtq+YtMzrQEMyAIipXPfCZNIbQLs3sbUumC5Iuku0a+XxNZkpjIpJ3MLS7nC7MJYguIcpAIsB5+ZKk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240091; c=relaxed/simple; bh=ojtPzu7q3ge/eAe4Ro2eoAuMOwfc296i2Q9/KCCbkM4=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=gSt2YAniXHvzvLIioyRXW3SgPa+ehegiXLoBCc2/jh3vfn1ahC3dl/DQ30rPOc4EqhhxGKo1lfEy3F07YuVWlwSc/Hn9QDobGeGtL5PNznh5RNe04cjC/cXBtbVam8V1anzvZWHT9MQr+5HHs2SECqc4oYtoRmUL1jmp/YiZdG0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=QnA/nKN2; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=dZDfmAJ5; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="QnA/nKN2"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="dZDfmAJ5" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777240086; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=CI0eARfgql4LoyAd3/aNb/WcXh5TjMiJ3J1/62uF7XY=; b=QnA/nKN2504jfoxv0bGhwImtgqQaIKkJdHtIlPiyM1jSeL+OyDCL/XuIULu80+KnL7juaH eq7nkCzrT7op6lRVQbmF4vHHfykkDCdpKfAschtlDTMea+pPqiJEdxtAk9UJXC+l/Yt1Wu zY2s10KX/H5Z0QggO5+45i9uBQDCNlg= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-204-d4seAa4ZP2eh3vHSHxoJKg-1; Sun, 26 Apr 2026 17:48:05 -0400 X-MC-Unique: d4seAa4ZP2eh3vHSHxoJKg-1 X-Mimecast-MFC-AGG-ID: d4seAa4ZP2eh3vHSHxoJKg_1777240084 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-48a5952c635so51082965e9.2 for ; Sun, 26 Apr 2026 14:48:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1777240084; x=1777844884; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=CI0eARfgql4LoyAd3/aNb/WcXh5TjMiJ3J1/62uF7XY=; b=dZDfmAJ5oJ5kbvMOXByVKlqzkJfN3iQ6YU/kRBcyMRvNeVwx6kAX47fFG7Q+FWOaFh ngGCbwvCYxh4ZZHmdjCAiwW4QTV+gDDAoEJgBo4uZyq1ytDK2Ho+HV970SqLXx9u0Aw+ AN2vFHWcWLo73izAp2t8kC/0tVqhayWGnoM8sUiBUl2mihAlIjFF8T/nhx83qhYrXTJz mobdpM+OdJ45l+SuLi0vQcOYeHi1kQ18WCHXejBGlAPP/AhdNCUlmvxIzxfikmtTxm3m X5i2agiWyuV7UBWygYs7Sf/4LAhYuxKAoZ/CX6vEFpQJOyqA/sPXAhdSEItxetrkGlY7 O+Gg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777240084; x=1777844884; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CI0eARfgql4LoyAd3/aNb/WcXh5TjMiJ3J1/62uF7XY=; b=YXq6MyI3pxq2RFckuiJlvnRdv0akcZRaumpIX15PQwn45r3dURiOAvqCbYpqHnADxB iRa0Jc8Wastn3WF4c8/5io5ClLZS4TDa2F0BVCrDA/fWuAXP/UZrzrjNkTeMQi9KGvvW /nhbwYxPeCaYjzlp4QVCOsiCOXQ03Re0mZJ9YYk3evoR7yMp+nVnVlI7zTn9pn+5RyO+ sEokVjLdnfK6GZOm554cw+wd/2d+rGt8DPV9+oWq/gzGTuT9VyEp2lDYKORUVPTwY0HI 1pgsjf1/WDdtvsZqC49zBlJbMVgyz4oAJXIpk/VjlsaZ5VR/HviL72hLB4Wx5d6mjVmi yaYQ== X-Gm-Message-State: AOJu0Yw7Hbjbj0TMSH7lKbooskmDO4L39Q4XWsHhUbMcLVax7M4iYRI7 Y6FLq36Ooluoq4EdPfuj6lyMAH9WXl3XyEa5W3wKHj/k+nu4Cac/5Bz2WPqDi42gR7OIveR+0G5 ga04SCiwlzYpiSOiPaArxmHKIj+S3TbxUmS4DuJBkqEifti0m7/W6trRPx/ezyWaAzHgHqJ8cp4 LjAfG3bCxswmtj5WHxYYpvB7eJS81SduPF5xDbqSTm178= X-Gm-Gg: AeBDievOTonNRWQzwT4cOB6v8zpeSky87kbSXTPYKHhRrCVgCjAYd8dt0nu3tvAspKs I0wDRLPjzveu+SHyE1F/LNc0ERRNTzQvBK4bLTJNXM9vUNXgk7YhfkBHvxWiEdTvQ0XwuuED8QT SnvNoOOejJk8FQandSYPV+T4Be0cZWeYGj9wO2pSWVeugYf6zdVhdGA3Z5a7g8+W9TYJneLn5sW LIMtS9fx4zdYfHnS2HXrMhSNWPdZlGsdPKZQZ+2BPXBZV5NI9mwMtQLudSHUCC1MW+U626mGbRW dhGodC6ymsWgC79uljrWseMW9lrHQB/2TfBInZFVwF0OroM7kjNkFxaRNW4Tyf13d9pmhfnIShx 55fAB+LApDUFIPQGGS2HwAGra4WZ9OW2kWOHJTje37eY3WQoHzxfucycu X-Received: by 2002:a05:600c:8587:b0:485:3f30:6250 with SMTP id 5b1f17b1804b1-488fb7856b9mr439937125e9.20.1777240083884; Sun, 26 Apr 2026 14:48:03 -0700 (PDT) X-Received: by 2002:a05:600c:8587:b0:485:3f30:6250 with SMTP id 5b1f17b1804b1-488fb7856b9mr439936725e9.20.1777240083430; Sun, 26 Apr 2026 14:48:03 -0700 (PDT) Received: from redhat.com (IGLD-80-230-47-179.inter.net.il. [80.230.47.179]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4899dc95920sm158861425e9.6.2026.04.26.14.48.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 26 Apr 2026 14:48:02 -0700 (PDT) Date: Sun, 26 Apr 2026 17:47:59 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Lorenzo Stoakes , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang Subject: [PATCH RFC v4 09/22] mm: use __GFP_ZERO in vma_alloc_anon_folio_pmd Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Convert vma_alloc_anon_folio_pmd() to pass __GFP_ZERO instead of zeroing at the callsite. Use vma_alloc_folio_user_addr() to pass the PMD-aligned haddr for NUMA policy and the raw vmf->address for cache-friendly zeroing. Signed-off-by: Michael S. Tsirkin --- mm/huge_memory.c | 18 +++++------------- 1 file changed, 5 insertions(+), 13 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 8e2746ea74ad..752f0b2e5bac 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1254,13 +1254,13 @@ unsigned long thp_get_unmapped_area(struct file *fi= lp, unsigned long addr, EXPORT_SYMBOL_GPL(thp_get_unmapped_area); =20 static struct folio *vma_alloc_anon_folio_pmd(struct vm_area_struct *vma, - unsigned long addr) + unsigned long haddr, unsigned long user_addr) { - gfp_t gfp =3D vma_thp_gfp_mask(vma); + gfp_t gfp =3D vma_thp_gfp_mask(vma) | __GFP_ZERO; const int order =3D HPAGE_PMD_ORDER; struct folio *folio; =20 - folio =3D vma_alloc_folio(gfp, order, vma, addr & HPAGE_PMD_MASK); + folio =3D vma_alloc_folio_user_addr(gfp, order, vma, haddr, user_addr); =20 if (unlikely(!folio)) { count_vm_event(THP_FAULT_FALLBACK); @@ -1279,14 +1279,6 @@ static struct folio *vma_alloc_anon_folio_pmd(struct= vm_area_struct *vma, } folio_throttle_swaprate(folio, gfp); =20 - /* - * When a folio is not zeroed during allocation (__GFP_ZERO not used) - * or user folios require special handling, folio_zero_user() is used to - * make sure that the page corresponding to the faulting address will be - * hot in the cache after zeroing. - */ - if (user_alloc_needs_zeroing()) - folio_zero_user(folio, addr); /* * The memory barrier inside __folio_mark_uptodate makes sure that * folio_zero_user writes become visible before the set_pmd_at() @@ -1328,7 +1320,7 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct= vm_fault *vmf) pgtable_t pgtable; vm_fault_t ret =3D 0; =20 - folio =3D vma_alloc_anon_folio_pmd(vma, vmf->address); + folio =3D vma_alloc_anon_folio_pmd(vma, haddr, vmf->address); if (unlikely(!folio)) return VM_FAULT_FALLBACK; =20 @@ -2033,7 +2025,7 @@ static vm_fault_t do_huge_zero_wp_pmd(struct vm_fault= *vmf) struct folio *folio; vm_fault_t ret =3D 0; =20 - folio =3D vma_alloc_anon_folio_pmd(vma, vmf->address); + folio =3D vma_alloc_anon_folio_pmd(vma, haddr, vmf->address); if (unlikely(!folio)) return VM_FAULT_FALLBACK; =20 --=20 MST From nobody Mon Jun 15 18:00:34 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 44B4C1DDC35 for ; Sun, 26 Apr 2026 21:48:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240091; cv=none; b=splGd4GaeBtYRBVS8Eivix9n/3NRGr4SjrVMRGh7sd3oV2aop53f4mUa8aU6DdxbfnNqvGsrZ3DOKGv8R+y2x6Iik3ul+1RDxwfmZePnEuoHQjnbX/QoeyYo0Po6NgIuasb++s+67QppTY/Kh4DD0Az7/gLD9ksncxp3PQvVmMU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240091; c=relaxed/simple; bh=EWFOuheoC1DvTBgPAHezGTmeJdyms6YPvVMBqd1xoVo=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=sxc6awuu7KC8o22gh2xej25BNVpXVrnVePblx8Bcrkx9aB2q1RQd5sm5GulFSbEUdx/F8wlXR9vtdbQdFnOSdXIlbYSmDFaO6Q+9m+SFgwNydFOWDciKibo7CIPMwupNVArb+noXjzsZ6SXG6cPoWnq5u9LuAOzru1ZmecQGrJ4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=KaY0WGEJ; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=r//s0APx; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="KaY0WGEJ"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="r//s0APx" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777240089; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=d5XFq6evYUr+s+QSCea6SglT5emqQpzsUniObvQ6PpU=; b=KaY0WGEJFljGuHRBYU9eDutqJJgff7cMyGeCiVXimEKsFTdHKh6EYB4gSk0FS13ot398rI I3YSkop2vqt/yhQUfFuXX8sCLr6u3CwCKKO+gRHG4xy8f7PvjHWuFJKXYD4VnhLgHBjYDk aGJ3rt3sC4c18FchbUEqr8YUZp51c1s= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-127-zxrwBfa_N4mpuNQ5DHltEQ-1; Sun, 26 Apr 2026 17:48:08 -0400 X-MC-Unique: zxrwBfa_N4mpuNQ5DHltEQ-1 X-Mimecast-MFC-AGG-ID: zxrwBfa_N4mpuNQ5DHltEQ_1777240087 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-4891f97aef0so42145565e9.2 for ; Sun, 26 Apr 2026 14:48:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1777240087; x=1777844887; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=d5XFq6evYUr+s+QSCea6SglT5emqQpzsUniObvQ6PpU=; b=r//s0APxB7kER5D+DvkS/ZuZH4n6SrohXYFe/2vVCWj/K2LeMPiyFItdd4N/wEMYEJ HwcOfrpRfrVrFSvygrKqVJe1+yhjIgtUadokqJSc8iffXhjz9fYi3ZyHCiOWdsXe9TzZ Mm8+vWJWD+tJuKVv5U3tTWNIryxRMNInGexC2BuddpVcz79QqHuiuJqob+XvdgyDcOrV wZ9JznOu5I1m/sPwuFSiQddWqDGX8NVScOV4sWcmGhWwy0Re+F52cQJNMKVojqZ2RwvH 0958RL6arN+mubAFDt5xTAv9QsZLQLOr7r31bNGEJnSt1RAUjv7p1xaRSDcubKpHnbi/ 35MQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777240087; x=1777844887; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=d5XFq6evYUr+s+QSCea6SglT5emqQpzsUniObvQ6PpU=; b=rviVYLL5woQaqK9piIEaNTBbcyUL0IXNhyLdUvT8j/6eMkrDbl2zW8dFsKVF2AQ2rg 6sxfk6I2S1nXR1i9IPXZWdchyKxhHLWHAbc1nAp0JWa5HN3w4gdWuJRcqxXVg8D449Vo L2wHWD22ivT53jKizcuF7n5gdnl67BG8han4s/6kk3Q9WMFgcT5gXKEtHCxyzDNTqYRN bnZaNa7dflrlfPvNpliS7/52y2FM8O5Gj0h6KkF11pjP056pkrncx0ufj9y5Vr+JSfoG 8ZlNft4JVDmXfVC89C4yidGcGvLYl7qM1fwwgsW6QgBOibvEsjEF4fLIlBDNGdG/CuF0 R6SA== X-Gm-Message-State: AOJu0YyMJQEvteSM6rmqYZVL7o40jcn5Cn0txQXE1owqrV9Gji/T6Dv7 dZ83Hu9fVob23791+sKYF+hHutEe6SiReN6TYWocWhi1O5Zbx8+vxzd/itr7rBLfwAj6w1UuWNO uxqjrvKIgewQ8Ep4Vdu/PGY11+tA9b4GRWjUxz/AGNW5cg/x9bUniwY4xqK42K5FRP0E8hr9fve A3M/Qnz9ACmyMABm4INUjyRRus4gflX9Zj18WuOSASdWk= X-Gm-Gg: AeBDiev7oviqE0LjQKPmYE82AM5Hc0RtDkM43Y9jvTuRLwWFqf642K5myn6oPIGqgq0 FnFRtMeJZfI5ZFtHdygy+881HUnbRCd+kE3MevAtB1pwh5RYlXUlreQyIa2UIqQW91bDxvWJobv 1yH8kJjtkxYzSrA5pDoDwZ4mMb8A95NsAW+4BEJOeDT9HRkhSZitpb3cPqpFkBPTXftMJ6zCsAl bchMHFxJSqL/hktnlFOGbJp7DM8EkBzpu1yxBWE7tdIQRqVvr+TdKwSCr1yUipBb1h2qJ+BwsRt /3909U4bAobfDBHOmIdPtpSR7TlFEfD36WHPY3EBjlGuyhTqxPpY6GK0iJmFS73PnGGp1wh+84z H9M5eUVlx3qYPC5IR1NpO2yxOReBFpLIRqdJ4GDaJFQ5WZv6c496LqvJ5 X-Received: by 2002:a05:600c:1f94:b0:489:1c2d:211e with SMTP id 5b1f17b1804b1-4891c2d2213mr444173155e9.5.1777240086797; Sun, 26 Apr 2026 14:48:06 -0700 (PDT) X-Received: by 2002:a05:600c:1f94:b0:489:1c2d:211e with SMTP id 5b1f17b1804b1-4891c2d2213mr444172595e9.5.1777240086146; Sun, 26 Apr 2026 14:48:06 -0700 (PDT) Received: from redhat.com (IGLD-80-230-47-179.inter.net.il. [80.230.47.179]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-43fe4e3a341sm81500119f8f.24.2026.04.26.14.48.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 26 Apr 2026 14:48:05 -0700 (PDT) Date: Sun, 26 Apr 2026 17:48:03 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Muchun Song , Oscar Salvador Subject: [PATCH RFC v4 10/22] mm: hugetlb: use __GFP_ZERO and skip zeroing for zeroed pages Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Convert the hugetlb fault and fallocate paths to use __GFP_ZERO. For pages allocated from the buddy allocator, post_alloc_hook() handles zeroing (with zeroed skip when the host already zeroed the page). Hugetlb surplus pages need special handling because they can be pre-allocated into the pool during mmap (by hugetlb_acct_memory) before any page fault. Pool pages are kept around and may need zeroing long after buddy allocation, so PG_zeroed (consumed at allocation time) cannot track their state. Add a bool *zeroed output parameter to alloc_hugetlb_folio() so callers know whether the page needs zeroing. Buddy-allocated pages are always zeroed (zeroed by post_alloc_hook). Pool pages use a new HPG_zeroed flag to track whether the page is known-zero (freshly buddy-allocated, never mapped to userspace). The flag is set in alloc_surplus_hugetlb_folio() after buddy allocation and cleared in free_huge_folio() when a user-mapped page returns to the pool. Callers that do not need zeroing (CoW, migration) pass NULL for zeroed and 0 for gfp. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- fs/hugetlbfs/inode.c | 10 ++++++-- include/linux/hugetlb.h | 8 ++++-- mm/hugetlb.c | 54 ++++++++++++++++++++++++++++++++--------- 3 files changed, 56 insertions(+), 16 deletions(-) diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c index 3f70c47981de..d5d570d6eff4 100644 --- a/fs/hugetlbfs/inode.c +++ b/fs/hugetlbfs/inode.c @@ -822,14 +822,20 @@ static long hugetlbfs_fallocate(struct file *file, in= t mode, loff_t offset, * folios in these areas, we need to consume the reserves * to keep reservation accounting consistent. */ - folio =3D alloc_hugetlb_folio(&pseudo_vma, addr, false); + { + bool zeroed; + + folio =3D alloc_hugetlb_folio(&pseudo_vma, addr, false, + __GFP_ZERO, &zeroed); if (IS_ERR(folio)) { mutex_unlock(&hugetlb_fault_mutex_table[hash]); error =3D PTR_ERR(folio); goto out; } - folio_zero_user(folio, addr); + if (!zeroed) + folio_zero_user(folio, addr); __folio_mark_uptodate(folio); + } error =3D hugetlb_add_to_page_cache(folio, mapping, index); if (unlikely(error)) { restore_reserve_on_error(h, &pseudo_vma, addr, folio); diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 65910437be1c..094714c607f9 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -598,6 +598,7 @@ enum hugetlb_page_flags { HPG_vmemmap_optimized, HPG_raw_hwp_unreliable, HPG_cma, + HPG_zeroed, __NR_HPAGEFLAGS, }; =20 @@ -658,6 +659,7 @@ HPAGEFLAG(Freed, freed) HPAGEFLAG(VmemmapOptimized, vmemmap_optimized) HPAGEFLAG(RawHwpUnreliable, raw_hwp_unreliable) HPAGEFLAG(Cma, cma) +HPAGEFLAG(Zeroed, zeroed) =20 #ifdef CONFIG_HUGETLB_PAGE =20 @@ -705,7 +707,8 @@ int isolate_or_dissolve_huge_folio(struct folio *folio,= struct list_head *list); int replace_free_hugepage_folios(unsigned long start_pfn, unsigned long en= d_pfn); void wait_for_freed_hugetlb_folios(void); struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, - unsigned long addr, bool cow_from_owner); + unsigned long addr, bool cow_from_owner, + gfp_t gfp, bool *zeroed); struct folio *alloc_hugetlb_folio_nodemask(struct hstate *h, int preferred= _nid, nodemask_t *nmask, gfp_t gfp_mask, bool allow_alloc_fallback); @@ -1117,7 +1120,8 @@ static inline void wait_for_freed_hugetlb_folios(void) =20 static inline struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, unsigned long addr, - bool cow_from_owner) + bool cow_from_owner, + gfp_t gfp, bool *zeroed) { return NULL; } diff --git a/mm/hugetlb.c b/mm/hugetlb.c index de8361b503d2..4f0ed01f5b13 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1744,6 +1744,9 @@ void free_huge_folio(struct folio *folio) int nid =3D folio_nid(folio); struct hugepage_subpool *spool =3D hugetlb_folio_subpool(folio); bool restore_reserve; + + /* Page was mapped to userspace; no longer known-zero */ + folio_clear_hugetlb_zeroed(folio); unsigned long flags; =20 VM_BUG_ON_FOLIO(folio_ref_count(folio), folio); @@ -2146,6 +2149,10 @@ static struct folio *alloc_surplus_hugetlb_folio(str= uct hstate *h, if (!folio) return NULL; =20 + /* Mark as known-zero only if __GFP_ZERO was requested */ + if (gfp_mask & __GFP_ZERO) + folio_set_hugetlb_zeroed(folio); + spin_lock_irq(&hugetlb_lock); /* * nr_huge_pages needs to be adjusted within the same lock cycle @@ -2209,11 +2216,11 @@ static struct folio *alloc_migrate_hugetlb_folio(st= ruct hstate *h, gfp_t gfp_mas */ static struct folio *alloc_buddy_hugetlb_folio_with_mpol(struct hstate *h, - struct vm_area_struct *vma, unsigned long addr) + struct vm_area_struct *vma, unsigned long addr, gfp_t gfp) { struct folio *folio =3D NULL; struct mempolicy *mpol; - gfp_t gfp_mask =3D htlb_alloc_mask(h); + gfp_t gfp_mask =3D htlb_alloc_mask(h) | gfp; int nid; nodemask_t *nodemask; =20 @@ -2910,7 +2917,8 @@ typedef enum { * When it's set, the allocation will bypass all vma level reservations. */ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, - unsigned long addr, bool cow_from_owner) + unsigned long addr, bool cow_from_owner, + gfp_t gfp, bool *zeroed) { struct hugepage_subpool *spool =3D subpool_vma(vma); struct hstate *h =3D hstate_vma(vma); @@ -2919,7 +2927,9 @@ struct folio *alloc_hugetlb_folio(struct vm_area_stru= ct *vma, map_chg_state map_chg; int ret, idx; struct hugetlb_cgroup *h_cg =3D NULL; - gfp_t gfp =3D htlb_alloc_mask(h) | __GFP_RETRY_MAYFAIL; + bool from_pool; + + gfp |=3D htlb_alloc_mask(h) | __GFP_RETRY_MAYFAIL; =20 idx =3D hstate_index(h); =20 @@ -2987,13 +2997,15 @@ struct folio *alloc_hugetlb_folio(struct vm_area_st= ruct *vma, folio =3D dequeue_hugetlb_folio_vma(h, vma, addr, gbl_chg); if (!folio) { spin_unlock_irq(&hugetlb_lock); - folio =3D alloc_buddy_hugetlb_folio_with_mpol(h, vma, addr); + folio =3D alloc_buddy_hugetlb_folio_with_mpol(h, vma, addr, gfp); if (!folio) goto out_uncharge_cgroup; spin_lock_irq(&hugetlb_lock); list_add(&folio->lru, &h->hugepage_activelist); folio_ref_unfreeze(folio, 1); - /* Fall through */ + from_pool =3D false; + } else { + from_pool =3D true; } =20 /* @@ -3016,6 +3028,14 @@ struct folio *alloc_hugetlb_folio(struct vm_area_str= uct *vma, =20 spin_unlock_irq(&hugetlb_lock); =20 + if (zeroed) { + if (from_pool) + *zeroed =3D folio_test_hugetlb_zeroed(folio); + else + *zeroed =3D true; /* buddy-allocated, zeroed by post_alloc_hook */ + folio_clear_hugetlb_zeroed(folio); + } + hugetlb_set_folio_subpool(folio, spool); =20 if (map_chg !=3D MAP_CHG_ENFORCED) { @@ -5004,7 +5024,7 @@ int copy_hugetlb_page_range(struct mm_struct *dst, st= ruct mm_struct *src, spin_unlock(src_ptl); spin_unlock(dst_ptl); /* Do not use reserve as it's private owned */ - new_folio =3D alloc_hugetlb_folio(dst_vma, addr, false); + new_folio =3D alloc_hugetlb_folio(dst_vma, addr, false, 0, NULL); if (IS_ERR(new_folio)) { folio_put(pte_folio); ret =3D PTR_ERR(new_folio); @@ -5533,7 +5553,7 @@ static vm_fault_t hugetlb_wp(struct vm_fault *vmf) * be acquired again before returning to the caller, as expected. */ spin_unlock(vmf->ptl); - new_folio =3D alloc_hugetlb_folio(vma, vmf->address, cow_from_owner); + new_folio =3D alloc_hugetlb_folio(vma, vmf->address, cow_from_owner, 0, N= ULL); =20 if (IS_ERR(new_folio)) { /* @@ -5793,7 +5813,11 @@ static vm_fault_t hugetlb_no_page(struct address_spa= ce *mapping, goto out; } =20 - folio =3D alloc_hugetlb_folio(vma, vmf->address, false); + { + bool zeroed; + + folio =3D alloc_hugetlb_folio(vma, vmf->address, false, + __GFP_ZERO, &zeroed); if (IS_ERR(folio)) { /* * Returning error will result in faulting task being @@ -5813,9 +5837,15 @@ static vm_fault_t hugetlb_no_page(struct address_spa= ce *mapping, ret =3D 0; goto out; } - folio_zero_user(folio, vmf->real_address); + /* + * Buddy-allocated pages are zeroed in post_alloc_hook(). + * Pool pages bypass the allocator, zero them here. + */ + if (!zeroed) + folio_zero_user(folio, vmf->real_address); __folio_mark_uptodate(folio); new_folio =3D true; + } =20 if (vma->vm_flags & VM_MAYSHARE) { int err =3D hugetlb_add_to_page_cache(folio, mapping, @@ -6252,7 +6282,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, goto out; } =20 - folio =3D alloc_hugetlb_folio(dst_vma, dst_addr, false); + folio =3D alloc_hugetlb_folio(dst_vma, dst_addr, false, 0, NULL); if (IS_ERR(folio)) { pte_t *actual_pte =3D hugetlb_walk(dst_vma, dst_addr, PMD_SIZE); if (actual_pte) { @@ -6299,7 +6329,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, goto out; } =20 - folio =3D alloc_hugetlb_folio(dst_vma, dst_addr, false); + folio =3D alloc_hugetlb_folio(dst_vma, dst_addr, false, 0, NULL); if (IS_ERR(folio)) { folio_put(*foliop); ret =3D -ENOMEM; --=20 MST From nobody Mon Jun 15 18:00:34 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5687F24A044 for ; Sun, 26 Apr 2026 21:48:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240094; cv=none; b=gJx80jB24v7xXiZP5HnzUQUz0nRaO1zpso5T3w28UGQV18vl4fINis950Z3VmxOtzy5l/MLsCgIP0/mZpUOKRPy/dD0jAwRxu5ui0h1a2e2nQVkGw4XzL585wxqfcylo9+s8QbvJ4/XAJHJgG+HImbBiW1/m6F0puqJVTBJNeug= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240094; c=relaxed/simple; bh=1ejL+Z1wL5a9wUXBRiBrtQNmVFq1XngLbhmkTIDh9l0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=oasOk9sOZ+0MyUh4CManbyQDBaw2NyhaHdE3iAYl5Ny0MXhxg+At5ggHaY0ik7zQNxnl9+duUpC3byBjdLd9pkVeeKp9D7TCwOKFgp+qJzjhdhnQsScuYyKLiL1GxtTG09CKIRFeV+TZXf0tX4ZfWudrTEMO4UpZCJ4Wz1fJZzE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=KHuSpHsP; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=GXmeX6gv; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="KHuSpHsP"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="GXmeX6gv" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777240092; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=MQ6M8fMmd15EsU5jHzG0CofBcVEbXAB28EkZbWFSSyk=; b=KHuSpHsPoYA6DfHg3tVHcdvuH7TlR17TuA2VosOtbJ6uyCtNhDbKtEmrke4DDL7nXCMdzn 9zS6HWp8Y5Rp/3rfGGz66AocEH28eyOVVZh5RpwJxiW7NYnQMKdGTttFbPEAJ9JFiMQEGA OHQox3asDBxGkaD4PSlMQdZN0PZJBtc= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-572-dA0ECkKNN-SAmHVcdABiPA-1; Sun, 26 Apr 2026 17:48:11 -0400 X-MC-Unique: dA0ECkKNN-SAmHVcdABiPA-1 X-Mimecast-MFC-AGG-ID: dA0ECkKNN-SAmHVcdABiPA_1777240090 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-43d780757eeso6032790f8f.1 for ; Sun, 26 Apr 2026 14:48:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1777240090; x=1777844890; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=MQ6M8fMmd15EsU5jHzG0CofBcVEbXAB28EkZbWFSSyk=; b=GXmeX6gvK3D/EjV/QSu9awglM4+1qFcULvcueqjwrudpIaLbMuw5noy8l1N1aRqavO N+YFgaMzxzlsm1OukYod2PlYNu6lfWR4mJMWPd+2fHxbVgdxQLVUs8Fv4InDco1niYKh WuDLM0aS47+k4pyXtqXKubg4BS6gnRpT1UGRjsAKuidrFHiovXrSm1YnF+qeNEAz7vag dsp7BteErofrsnQ2RPvg0JX3Nf8xMC2xeINJj7VoCtuJgRrdvWYvlri7o23ixHZiLJeu BE8UH4Clqo4IJNWcJuxk9FVPp3mFk60Cr6AN91opZw5GbwCDM0O3U+ETpJprBjwcCmrS IsPw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777240090; x=1777844890; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MQ6M8fMmd15EsU5jHzG0CofBcVEbXAB28EkZbWFSSyk=; b=Eep3TP9rMNNHSiJzVz9HyrMnq0kigZxeyt5EN7b2WZpBlT2VzKQGFcG+ouitzlnDRu i9qVSUnw+tKw5E6f0QFC186rE0p+lrAQceMSgXA5OIzjX9IooF0/pa169LPGLO+r1kwX f2i1QDOY2kFdTYvbsHUfaFhwPeHSUuQIbFQIVPUou0oNlTVbFbHam7MzbW90wN8of618 1A7YRnXYrZfXbdo2p8zIy+Aa7no1OjAB04S47Ewrxv9+jEvNgPDso2fUzTUQkYphndC3 7ZYMWLh2zmvPWDdi9NFNLbXTuakM7kdO+3O4uVWWmt+D51CCF0VQnHacq4k4tcZyBPVO ckMg== X-Gm-Message-State: AOJu0Yz2NPvAGmM7fGPqvTZV8GRcx4/si3uHMgekxN4gHXYSS5qJm26V r0x9fa8kfI4U4girqgqS8eOHROu8SUurRTq9LuX13XN++4JYC8MxKSFnGtJtL8izifVV8Rw0OpT QNKGUcIUdJM++IQ+VeH/YU+gMEygsiHf4whNK1fu7slDHnQtaujrtMxCT3TrFAlA4jnd+kBTbab V3x2U1nNPdenpddqvKNtwORS5iqObhtSSpma8ZuBF5qdw= X-Gm-Gg: AeBDievjRJOpES3SsjXx28xrYK4HIwQiUSsGuzmNymdfbEvD442i8GEZmuLL98jFs24 /CqNhGQT8lzBq2QzY79aocCCTgDzgBIVQ93M60QhsKMz0WVMRdJltbWIJu/jbbentQpZsbcKBjv 4PrkG4ZHxWmVEdEcc6f7nmGhMG+fESXUb3mUp9WNC58NXJWr/XFg48c5/LaJXO7xpDaNsTiCXxt q+3lyEPVtwi5bXwCHhBsb6gZd8LYNkmG8aJadJ2p5w4bruc+VXgK3WP1F08TjFQUEiwPSZ/oks1 G/t7WybF6oJFpHB3pNSEtDO0dM8z8iw+o7mTdvNv+DJmJbqDFA3s5h0a0TNQDkAear1MRI4zEzO T/bWV8UglYmaXNxFDNryCd6veujdaChiJDgX8R8oCmWBDigRXbx3nX+Ph X-Received: by 2002:a5d:5848:0:b0:439:bee4:8a93 with SMTP id ffacd0b85a97d-43fe4073758mr63737409f8f.12.1777240089622; Sun, 26 Apr 2026 14:48:09 -0700 (PDT) X-Received: by 2002:a5d:5848:0:b0:439:bee4:8a93 with SMTP id ffacd0b85a97d-43fe4073758mr63737360f8f.12.1777240089089; Sun, 26 Apr 2026 14:48:09 -0700 (PDT) Received: from redhat.com (IGLD-80-230-47-179.inter.net.il. [80.230.47.179]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-43fe4cc0f31sm76655583f8f.12.2026.04.26.14.48.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 26 Apr 2026 14:48:08 -0700 (PDT) Date: Sun, 26 Apr 2026 17:48:06 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Muchun Song , Oscar Salvador , Hugh Dickins , Baolin Wang Subject: [PATCH RFC v4 11/22] mm: memfd: skip zeroing for zeroed hugetlb pool pages Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" gather_surplus_pages() pre-allocates hugetlb pages into the pool during mmap. Pass __GFP_ZERO so these pages are zeroed by the buddy allocator, and HPG_zeroed is set by alloc_surplus_hugetlb_folio. Add bool *zeroed output to alloc_hugetlb_folio_reserve() so callers can check whether the pool page is known-zero. memfd's memfd_alloc_folio() uses this to skip the explicit folio_zero_user() when the page is already zero. This avoids redundant zeroing for memfd hugetlb pages that were pre-allocated into the pool and never mapped to userspace. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/hugetlb.h | 6 ++++-- mm/hugetlb.c | 11 +++++++++-- mm/memfd.c | 17 +++++++++++------ 3 files changed, 24 insertions(+), 10 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 094714c607f9..93bb06a33f57 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -713,7 +713,8 @@ struct folio *alloc_hugetlb_folio_nodemask(struct hstat= e *h, int preferred_nid, nodemask_t *nmask, gfp_t gfp_mask, bool allow_alloc_fallback); struct folio *alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_= nid, - nodemask_t *nmask, gfp_t gfp_mask); + nodemask_t *nmask, gfp_t gfp_mask, + bool *zeroed); =20 int hugetlb_add_to_page_cache(struct folio *folio, struct address_space *m= apping, pgoff_t idx); @@ -1128,7 +1129,8 @@ static inline struct folio *alloc_hugetlb_folio(struc= t vm_area_struct *vma, =20 static inline struct folio * alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_nid, - nodemask_t *nmask, gfp_t gfp_mask) + nodemask_t *nmask, gfp_t gfp_mask, + bool *zeroed) { return NULL; } diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 4f0ed01f5b13..f02583b9faab 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -2241,7 +2241,7 @@ struct folio *alloc_buddy_hugetlb_folio_with_mpol(str= uct hstate *h, } =20 struct folio *alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_= nid, - nodemask_t *nmask, gfp_t gfp_mask) + nodemask_t *nmask, gfp_t gfp_mask, bool *zeroed) { struct folio *folio; =20 @@ -2257,6 +2257,12 @@ struct folio *alloc_hugetlb_folio_reserve(struct hst= ate *h, int preferred_nid, h->resv_huge_pages--; =20 spin_unlock_irq(&hugetlb_lock); + + if (zeroed && folio) { + *zeroed =3D folio_test_hugetlb_zeroed(folio); + folio_clear_hugetlb_zeroed(folio); + } + return folio; } =20 @@ -2341,7 +2347,8 @@ static int gather_surplus_pages(struct hstate *h, lon= g delta) * It is okay to use NUMA_NO_NODE because we use numa_mem_id() * down the road to pick the current node if that is the case. */ - folio =3D alloc_surplus_hugetlb_folio(h, htlb_alloc_mask(h), + folio =3D alloc_surplus_hugetlb_folio(h, + htlb_alloc_mask(h) | __GFP_ZERO, NUMA_NO_NODE, &alloc_nodemask, USER_ADDR_NONE); if (!folio) { diff --git a/mm/memfd.c b/mm/memfd.c index 919c2a53eb96..b9b44ed54db5 100644 --- a/mm/memfd.c +++ b/mm/memfd.c @@ -90,20 +90,24 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgo= ff_t idx) if (nr_resv < 0) return ERR_PTR(nr_resv); =20 + { + bool zeroed; + folio =3D alloc_hugetlb_folio_reserve(h, numa_node_id(), NULL, - gfp_mask); + gfp_mask, + &zeroed); if (folio) { u32 hash; =20 /* - * Zero the folio to prevent information leaks to userspace. - * Use folio_zero_user() which is optimized for huge/gigantic - * pages. Pass 0 as addr_hint since this is not a faulting path - * and we don't have a user virtual address yet. + * Zero the folio to prevent information leaks to + * userspace. Skip if the pool page is known-zero + * (HPG_zeroed set during pool pre-allocation). */ - folio_zero_user(folio, 0); + if (!zeroed) + folio_zero_user(folio, 0); =20 /* * Mark the folio uptodate before adding to page cache, @@ -139,6 +143,7 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgo= ff_t idx) hugetlb_unreserve_pages(inode, idx, idx + 1, 0); return ERR_PTR(err); } + } #endif return shmem_read_folio(memfd->f_mapping, idx); } --=20 MST From nobody Mon Jun 15 18:00:34 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5BCD5219EB for ; Sun, 26 Apr 2026 21:48:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240098; cv=none; b=dANKMClYyphGWFTiEEm6CmXPHNywW0ETgJlfyEoznCcgQ+6YgF87N/RbiVoXuORZ+9lyqTNDkV201rzbEXZWgS3cFoBsqSsms6lRHW9xEydMvLq6djwiWbefmSzK4gEGR9h3sqmZsphpcZTGQQtsHdqzVWfowFLvTxaXHoJjLs4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240098; c=relaxed/simple; bh=o+vhOTjLhvS47fv9tom+Yt3TM9vgXV0mo+tAnh6qa+g=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=WrR+4+ZNBx2Krm+qc+l+fSVOc0ghCJ9fRDZDgwR+Zcb8bfvrjQovcQQ5GQ10owVBEcX24ePoyC45Y4G8YOt9zHFXeEtYnbShmy/wf7VgKg+AXr/A48qbWDrbM3DSTx8kd+oWmgB/wHv4+8yIlp3R4F51K00Sq2I0DfbggzXZpI8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=TgPxFQY1; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=rezq0HIx; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="TgPxFQY1"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="rezq0HIx" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777240096; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=KxxPeFo8b5YQQgq6okT6k4ODeRwXIuGSW0SUh4EUnR0=; b=TgPxFQY1VvQDzV5yyrlydLtRzB0pW9KQSYCoi3c5I3q9+mjOTnqUGV7Sbo290w9wNb7En0 wOvmRbdDw3eyQISgm7xbK020jiC8xXzTWp8XLzJQHSviJU0d9pPwYaHrmI7BIw4MBer5Ot kMWo/hjB8cpOBkH+uh0qztTA1ro2xWc= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-526-vn442kAoNpqqYlla_Bl4OQ-1; Sun, 26 Apr 2026 17:48:15 -0400 X-MC-Unique: vn442kAoNpqqYlla_Bl4OQ-1 X-Mimecast-MFC-AGG-ID: vn442kAoNpqqYlla_Bl4OQ_1777240094 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-4837b6f6b93so102942315e9.3 for ; Sun, 26 Apr 2026 14:48:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1777240093; x=1777844893; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=KxxPeFo8b5YQQgq6okT6k4ODeRwXIuGSW0SUh4EUnR0=; b=rezq0HIx5sTnzlA6s6dxqTv+0IQOo9Z2pEqzW6tGfRCFJYVKubIuFa441hLh0WV3vw diZUcsi5VcsDVjsDnWCSM7C0+/zl8ocXL4WXK/Qm4W8WDjwebbKg5iqKfq4kg0NxW649 iJEjknvzFUx5M/Hc8XX5ZMKcInkNQxm9KSKS5rUUxwxwIAHEy3fEMORIib8HsETFvwAF Dag+vaTxlitpVxNNh9WcgI1ihClnwDbuZwpoGg8V4UuSd8lBcNK1dxGx82WGnue9q1t5 wFcuIBq3cuOjBUPNkG7tOy/PEGHLbWsRUcijkzFYiHltASuSmpYPtPeRZzyNItpGctQt RvwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777240093; x=1777844893; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KxxPeFo8b5YQQgq6okT6k4ODeRwXIuGSW0SUh4EUnR0=; b=XOE9pEV/mBndjdZB0VjQ7EyUo+zW86O5vaZwMdspVD3gudCVgYSMj/dLYVqLHm09CG 60Kz8Y3xSwHaq3yyvJ8w/BPG9YICz8R98Tm7rl1EQmiD4OWSLNP//hGc/2LCKyhpOkyp AdSvhe2LSyW40GCaOD66G2fl+I00KLnHbYjVi/4FrWaxw5YepdpHij0lKc40MSr7aGUi N4Wac8gx0JcKmnIfXO/LuHSQ9s16wFUSSidgqYtOUL6RYuHInXLIwrwRjsFsFhx662jd T8YDMY6De7XhNWLAAXf2XiAbzjUfOE9dOBzITLjoJVloZBq6n0Y0Q93bAKlVF1l9OHce fxjg== X-Gm-Message-State: AOJu0YykY+9aeJK2D2ihoZBO4XdVHqjUOi7uRFGszTuOjl5irM0U9dZ0 d3RtV67dHhdIWbCm9DyBj2AJdH49LYOupnmNHv7+LHBc1oODwt7jLp0fn4L7CgDahMpbA6VLcgU 9KbLMTRFbaSk1+pZj7EfiplJvwrOtPJR7vMrEfRlDZv8fLB53pAb5FdW8QzJy+E0ygEhAa4rSdB 5KdBKXfReklukpXXR2sRNlONURxI+n6xsASwySX6PKa0A= X-Gm-Gg: AeBDieuM4PttrbvKdA1Zf1YpnO2kQEj3zw5KP2iBG13pHGUsvTLDdiNu8A3aKrKvfqo fYKQDl9wIJ2/IHUZqr5aysn38JdpIW+U1O5TYok6lgo02OpC/RGsGPtdQOPE8HsOhtnUqIeC7h+ eDt5LbmGxRvM7ybjZFf6/599bxWqTxgIQSjaKKGwyPHUm67P0sVgGBx3lbPQTstz/2SZhUek7XW peLTGkPbuO3MB/MPeM6p3mh4q2hLbaD2BawydJe9MHPlMLhz16g/Ys9PkwGkIbln+1XwMrb5Hiv spkLunUJkLL0diRuTOz8GWbxva/26DHT4/WpzLDU5I5eqXF7X7f/If3IXN/cMsidSuhSc+5eJYH GS+SaRZMOAAQrO9YHy9JxcIDMJYaH1hV75p0ThnQFteoH/AkGy3Ssfjb7 X-Received: by 2002:a05:600c:6296:b0:488:c40b:c8a4 with SMTP id 5b1f17b1804b1-488fb73d764mr551943115e9.1.1777240093482; Sun, 26 Apr 2026 14:48:13 -0700 (PDT) X-Received: by 2002:a05:600c:6296:b0:488:c40b:c8a4 with SMTP id 5b1f17b1804b1-488fb73d764mr551942475e9.1.1777240092952; Sun, 26 Apr 2026 14:48:12 -0700 (PDT) Received: from redhat.com (IGLD-80-230-47-179.inter.net.il. [80.230.47.179]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-489fec8f7cbsm454414365e9.11.2026.04.26.14.48.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 26 Apr 2026 14:48:12 -0700 (PDT) Date: Sun, 26 Apr 2026 17:48:09 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Magnus Lindholm , Greg Ungerer , Geert Uytterhoeven , Richard Henderson , Matt Turner , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , linux-alpha@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-s390@vger.kernel.org Subject: [PATCH RFC v4 12/22] mm: remove arch vma_alloc_zeroed_movable_folio overrides Message-ID: <424f14373b3339c93cab7a080614b7e0dbe7596f.1777223007.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that the generic vma_alloc_zeroed_movable_folio() uses __GFP_ZERO, the arch-specific macros on alpha, m68k, s390, and x86 that did the same thing are redundant. Remove them. arm64 is not affected: it has a real function override that handles MTE tag zeroing, not just __GFP_ZERO. Suggested-by: David Hildenbrand Acked-by: Magnus Lindholm Acked-by: Greg Ungerer Acked-by: Geert Uytterhoeven # m68k Signed-off-by: Michael S. Tsirkin --- arch/alpha/include/asm/page.h | 3 --- arch/m68k/include/asm/page_no.h | 3 --- arch/s390/include/asm/page.h | 3 --- arch/x86/include/asm/page.h | 3 --- 4 files changed, 12 deletions(-) diff --git a/arch/alpha/include/asm/page.h b/arch/alpha/include/asm/page.h index 59d01f9b77f6..4327029cd660 100644 --- a/arch/alpha/include/asm/page.h +++ b/arch/alpha/include/asm/page.h @@ -12,9 +12,6 @@ =20 extern void clear_page(void *page); =20 -#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ - vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr) - extern void copy_page(void * _to, void * _from); #define copy_user_page(to, from, vaddr, pg) copy_page(to, from) =20 diff --git a/arch/m68k/include/asm/page_no.h b/arch/m68k/include/asm/page_n= o.h index d2532bc407ef..f511b763a235 100644 --- a/arch/m68k/include/asm/page_no.h +++ b/arch/m68k/include/asm/page_no.h @@ -12,9 +12,6 @@ extern unsigned long memory_end; =20 #define copy_user_page(to, from, vaddr, pg) copy_page(to, from) =20 -#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ - vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr) - #define __pa(vaddr) ((unsigned long)(vaddr)) #define __va(paddr) ((void *)((unsigned long)(paddr))) =20 diff --git a/arch/s390/include/asm/page.h b/arch/s390/include/asm/page.h index f339258135f7..04020a19a5cf 100644 --- a/arch/s390/include/asm/page.h +++ b/arch/s390/include/asm/page.h @@ -67,9 +67,6 @@ static inline void copy_page(void *to, void *from) =20 #define copy_user_page(to, from, vaddr, pg) copy_page(to, from) =20 -#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ - vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr) - #ifdef CONFIG_STRICT_MM_TYPECHECKS #define STRICT_MM_TYPECHECKS #endif diff --git a/arch/x86/include/asm/page.h b/arch/x86/include/asm/page.h index 416dc88e35c1..92fa975b46f3 100644 --- a/arch/x86/include/asm/page.h +++ b/arch/x86/include/asm/page.h @@ -28,9 +28,6 @@ static inline void copy_user_page(void *to, void *from, u= nsigned long vaddr, copy_page(to, from); } =20 -#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ - vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr) - #ifndef __pa #define __pa(x) __phys_addr((unsigned long)(x)) #endif --=20 MST From nobody Mon Jun 15 18:00:34 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B122C21B9F6 for ; Sun, 26 Apr 2026 21:48:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240101; cv=none; b=peyRmSkaiIxYDYncu3lQgnkbE+IKP9B2tAiOFZwoq3rUvN+Cy2U/zDMplQBGS+jT0/tNuB3o1OXEzYbMcWAX1/J1GieazByhnxipbkIj08i51wvwDK7g3mG2Ul2pugaULUfWg0WmfMbB3heb5rIWQSaEvSboO5SyRkM9IbmKUVU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240101; c=relaxed/simple; bh=IxckQwCgbZJbxxjvhlOEmINi1CgwfFxzwO5yadjBe7c=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=aPTOtNq21eHKQXmD6MyJLSN50pG8hRFHmAnA7S/x0sY9F09Z7BmjANPPiAm2+VSYsDU2EtqKPtDkPB5rQk3+i9QLHrCRefaKLez6l5HF2Pm7HMI+ZIo1LSU0kly/ENz+Rd6Mi8pKB4TFt540eYeh9kNfkpue9idBgSkbjfc2iy4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=a6T2JzsQ; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=HPqYSX8l; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="a6T2JzsQ"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="HPqYSX8l" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777240098; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=8jzD5+aaEj/qpAoN4ceRyyQr8AzYBOgh6GvhLOGpHCU=; b=a6T2JzsQibgyHcY7ss1SGU/lOrwYhrPOB6cjBv//n4xr6dfSNUS1+rDM4zCqS7czA+NNpQ UtLoR3n9DJCo5pSjqH3mf/5FwkudY1QkdEVoZfqS8zJMBrbZXVpDVPF5HWmNHhD0qXfgsL vRgE3HzlkXTq1jFFDBPfoj4FVK0gulU= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-235-ebPKbZMbO5ed-yzs-6OStg-1; Sun, 26 Apr 2026 17:48:17 -0400 X-MC-Unique: ebPKbZMbO5ed-yzs-6OStg-1 X-Mimecast-MFC-AGG-ID: ebPKbZMbO5ed-yzs-6OStg_1777240097 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-4837b6f6b93so102942475e9.3 for ; Sun, 26 Apr 2026 14:48:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1777240096; x=1777844896; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=8jzD5+aaEj/qpAoN4ceRyyQr8AzYBOgh6GvhLOGpHCU=; b=HPqYSX8lJQCsJRIQ3mpWauY/BDwlHDAIZm2jVQg64s82lptb7rtu1n7NFy1nyzLY9d SR/FQPoOinq8i7un+i5wLIw94KM3w6tcfloK7Zi0kTQcwlR8ZghnSsKpSsERtQBqh+nV dzegN3RrwjJz6ostLbvRwq63hxQbkeY+yVTVo8Epi9clKQJzsG5Xj+BxIffbK88XCYIV LogTo/NHNBr9y5NnmCe59IwpAsQzXy7DuUnt9NIHP2Qs8M9i6du0XJEFx8wIcj9X3E4Y XfDJFbZ8Y4KpGrFdh21SLnZUTCDMGDMs8cnQiuGqM/p7Xf2j1PHiTn0s4vSI/66DCgIq BLog== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777240096; x=1777844896; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8jzD5+aaEj/qpAoN4ceRyyQr8AzYBOgh6GvhLOGpHCU=; b=Aqk56SrbgHhzVz42fZHQZf65vB+54x5jR4gHPjrldxR03tH2rENRGZryhyV7pue6Tj AAdwaDdVOjLJypUj9mnPtPA7Y4LwagFpaOGuIsyVqPBtqAsyPxnxbikeXwOeDx0jCnJB h+TIvEOrBcuTAebmaILXS1w8yaeg3uoZPflDG9+pZbxgJG90D69ltjkXOYmoW4a41bE6 BTXiuLPQzYqN4NHIQmb04hqfCkA4z4MGiXn8kPK/P5ogP/miCnhNx+xRyWa2NVSKwvQT A0B8jjy9COAMavzb52ko0D+H+WM35IKAr1tK3tGLatdMh2xyRSmbchnk0qeHk4ywMo1a bj1Q== X-Gm-Message-State: AOJu0Yx4xJzL5wOcfjMXfJ78CP5VJUDjH0SoIvwQNsUob0ei0ToCgVl0 cbXEKJdYSTI8UYegEhR75cGhP7AgcmI7Xb/QJNF3FJeDLBIYS38ezN5sUOEQKiRiSHXhIKw1AAR fOjCys5AC6lrckmvvcEDIECc3rvpUSjKHuzuXHZj4uUxSFqD7TEncRMB/++pEGdYm9jzyMUDaJc Wc48fW/BGcne7/MreRkCh62GJ1/QMXTKrePgahg7a38Cg= X-Gm-Gg: AeBDiev5OEdt+/WljyMyKarGRNPv+TXZ6q/jhpKa9aD5XFeQ3646jyY0hPShZpJwlPC RMRE9HCPoIzqy+N7MOV/J9PZFne8HlTWKiYMLe876GWU5WMAn7gJDtthNwQqPZb4F7thc3fVBrQ uCmm3yDrpxnvavAd/JFdQQd4Q3I87eOEpiFKnO1d+Y6ZZikE7mdjJDuBfybd1gzdArZs6I9enEj 9XVRaQJUmT6ETeR4JRw2y4itsu3WonY56xr8aU4A0W/zVYbwUDzDiVP86g8vkmHaH5tYVbOk9nl 4JsKMV0WKYADd/5YX0OJlcNhnqGfZ7gAHGqbFEl4oZ3H50ux/8RFxn2wIBC/iwj8oEQhv97zNT7 ruOtFYlzyuUZwABVhqeTTBHSr/GIW4eUtFqTK1wnwSsVCnH6cEWwBkZDR X-Received: by 2002:a05:600c:820d:b0:48a:569c:abab with SMTP id 5b1f17b1804b1-48a569cadacmr341610075e9.14.1777240096225; Sun, 26 Apr 2026 14:48:16 -0700 (PDT) X-Received: by 2002:a05:600c:820d:b0:48a:569c:abab with SMTP id 5b1f17b1804b1-48a569cadacmr341609585e9.14.1777240095687; Sun, 26 Apr 2026 14:48:15 -0700 (PDT) Received: from redhat.com (IGLD-80-230-47-179.inter.net.il. [80.230.47.179]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4412150a071sm39398823f8f.21.2026.04.26.14.48.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 26 Apr 2026 14:48:15 -0700 (PDT) Date: Sun, 26 Apr 2026 17:48:13 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Johannes Weiner , Zi Yan Subject: [PATCH RFC v4 13/22] mm: page_alloc: propagate PageReported flag across buddy splits Message-ID: <68154113e50456eb8adab9bc5419c30effc59144.1777223007.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When a reported free page is split via expand() to satisfy a smaller allocation, the sub-pages placed back on the free lists lose the PageReported flag. This means they will be unnecessarily re-reported to the hypervisor in the next reporting cycle, wasting work. Propagate the PageReported flag to sub-pages during expand() so that they are recognized as already-reported. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/page_alloc.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 7791bc1eeefa..ca4f9c0948af 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1730,7 +1730,7 @@ struct page *__pageblock_pfn_to_page(unsigned long st= art_pfn, * -- nyc */ static inline unsigned int expand(struct zone *zone, struct page *page, in= t low, - int high, int migratetype) + int high, int migratetype, bool reported) { unsigned int size =3D 1 << high; unsigned int nr_added =3D 0; @@ -1752,6 +1752,15 @@ static inline unsigned int expand(struct zone *zone,= struct page *page, int low, __add_to_free_list(&page[size], zone, high, migratetype, false); set_buddy_order(&page[size], high); nr_added +=3D size; + + /* + * The parent page has been reported to the host. The + * sub-pages are part of the same reported block, so mark + * them reported too. This avoids re-reporting pages that + * the host already knows about. + */ + if (reported) + __SetPageReported(&page[size]); } =20 return nr_added; @@ -1762,9 +1771,10 @@ static __always_inline void page_del_and_expand(stru= ct zone *zone, int high, int migratetype) { int nr_pages =3D 1 << high; + bool was_reported =3D page_reported(page); =20 __del_page_from_free_list(page, zone, high, migratetype); - nr_pages -=3D expand(zone, page, low, high, migratetype); + nr_pages -=3D expand(zone, page, low, high, migratetype, was_reported); account_freepages(zone, -nr_pages, migratetype); } =20 @@ -2334,7 +2344,8 @@ try_to_claim_block(struct zone *zone, struct page *pa= ge, =20 del_page_from_free_list(page, zone, current_order, block_type); change_pageblock_range(page, current_order, start_type); - nr_added =3D expand(zone, page, order, current_order, start_type); + nr_added =3D expand(zone, page, order, current_order, start_type, + false); account_freepages(zone, nr_added, start_type); return page; } --=20 MST From nobody Mon Jun 15 18:00:34 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 14237305E3B for ; Sun, 26 Apr 2026 21:48:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240105; cv=none; b=soK6cgwavTPRYe0vMjVwyKWglSPl/Gt7USxxvc7Z0BK0mfy1QjAnFSWJ3kQyomESIW3azUxg+JpTHgsA38E/RlanfEZQnoM2Tyav+4Z0/uN3DKLzweE+pKEnNyIQ5nniuLBKpvb/Oo3qIZjY9BTne8M0CleEiwagktIBn+cAo3k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240105; c=relaxed/simple; bh=iHi2fAUskCoevI2UUGMqLzIRuLJ/lgJ9ZjKWpSzu6pI=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Ihn2bBAB9O7Tc8CGGmOwCpuyzaYv7eHQ3JcaYfOly+3RhAUEy/G1FFoQzIHeHuybbzVKX7BAfmFrK2st901dojSIc0lNq2Nn/uZLyRJpV2U3qWz0CWdbEfzcn+jOsKm3B6+P8cM+AD1CXFiMTJPnR4PhGTrFqLYxKqlheuQaRJU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=JGSCo28A; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=Jf6AcZIK; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="JGSCo28A"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="Jf6AcZIK" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777240102; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=dr6ZqFIA5Mzb1p3xA6qD39ykvEmtI/Ko65Z86qQV6KI=; b=JGSCo28AfJ1KYnijS1k2he9LgnLsJqkCAqF+qbhPwZxitqCfbuCp2OsXoV5zpr5cNCEoSN Kb7UaDpuOHuSjX+m0lJdexkHCRDkUIw7H+8DfpyXgaM9Oy7bVoLFTxHpHrqNlQP2JeJfZm STNZhkUt7122/RwjESab2pB2W/3Tw2w= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-523-CAV6rHVWMjerOmyBI9EdiA-1; Sun, 26 Apr 2026 17:48:21 -0400 X-MC-Unique: CAV6rHVWMjerOmyBI9EdiA-1 X-Mimecast-MFC-AGG-ID: CAV6rHVWMjerOmyBI9EdiA_1777240100 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-488d8deb75fso78404045e9.3 for ; Sun, 26 Apr 2026 14:48:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1777240100; x=1777844900; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=dr6ZqFIA5Mzb1p3xA6qD39ykvEmtI/Ko65Z86qQV6KI=; b=Jf6AcZIKzg598aJV9b9LcDSbx897YI9XnJ6/eYp+ALu9NuJ2Bpa3OrQ6xqw3fNMb8l HojnQkxXXVSctPq1AhhSgqgRYeoZxqCwtMntx7zWoAnWf8xThBTjPzoPSoN1vEv7d9go GhdPpl0W7WHgf6HY8dYeIVW4T6MVhSLMw3/brrPxNTD85WK0I7uqvR90je+vm6HjUOf/ 93+6GFIOWUfVqNBvmujkYgmdb81y33oLs0jIdyP8xzGa5sjoKmRdl6bMSGpUUlIbJYli PVW1AphJXNU8fkbvAkp2b4UySomXqQGYRywwJu2H/Sg6FIoRXiNMSx5rcI2dSjh5LLLE +YdA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777240100; x=1777844900; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dr6ZqFIA5Mzb1p3xA6qD39ykvEmtI/Ko65Z86qQV6KI=; b=l4wmEIWMNTzi6x8kYUc6FGSbjvLc/PFqObinZo79encQNM4ipVXiA+Uhg1q3cbedEr 0q88UTpDHQTpw3Ou6BFwIVNaeZxvIW/5+y4egygQVC8hADzxipBUWxL3lF7IeqpHvfyW 7hTy8zmuJlYhdgJ3Pi0BvJCxXVw/XGF0Vo6akuBP4K3G+DisVfOGROfZT/rwfp9s/I83 zoLzBMGbmVcVh95f/XOTUEbE2s43giQH027iNo0csjTs6LWB4IH8SIdosE4aYYcOsJJ0 K+fbjklVF6q3BKTcRKL7EXbuUILv6jS6imyYImyBVfOs0nXsZxhy5E9UK/4e5Dfr+jVh tHwQ== X-Gm-Message-State: AOJu0Yz76Y/iRMv1LhGZq06AmWiZXy9qRlBbhHNFrBccUlq+kh19iCj8 6wdslAUrKzYRbFMx5qJDzXeNrOXP71rmUDvPJI0mtWiM1IiFHGad+EZIKM/fRxYXjxGwsYJA/C1 Mq+2TdfYc+FER6wqv0RgUYXJOxXT9rWQagX+eyFVybD/28t0cGufk2o6ScjmczSW70tg0d2u2Bj PpgxcLnZU9M2U3h7Bfn9+oJldvgP8kbHTf10JQ9p6fQTE= X-Gm-Gg: AeBDietFN+htBf5jnOsotJGKo5mxgVElKSHaMuBw4k6XmlEw57Tj5FUcMKkstdZQd5v LqRoTMyAMMsIN3F37Tyg4RCjy6ZsHwgo8H/uLkfmAa3h5zLCnXK8hiafjBT7LfC0q7aEqq+d3za vsemlOLzzXhnYES0MacsqZP+a1y4/4HWclfVHV5r+XohDah1PhGdGP9/E2EcgyhvBorqyJVXs9h /YWqEsLw5iAO4K0YaWdCEPzPiTGue6SJ9OSW6KptVV2GtyZ1nST0z9Jfq51w1iuacXnr+7sOOyd 29fpx5Ci1OGTvbIe1/4CKd3M/KqRXt26iqqskbQxbqUVkxuEx1bDL1+HkE5KgxUSpdLGcSecHfR Kfjj/CLXcY8n7yHDDvHOZ1LQurpiArnTrSKUWOJE246hS2J/Ssjq/BERo X-Received: by 2002:a05:600c:8909:b0:489:1b10:d896 with SMTP id 5b1f17b1804b1-4891b10dd45mr420917245e9.0.1777240099577; Sun, 26 Apr 2026 14:48:19 -0700 (PDT) X-Received: by 2002:a05:600c:8909:b0:489:1b10:d896 with SMTP id 5b1f17b1804b1-4891b10dd45mr420916875e9.0.1777240099012; Sun, 26 Apr 2026 14:48:19 -0700 (PDT) Received: from redhat.com (IGLD-80-230-47-179.inter.net.il. [80.230.47.179]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-488fb7aa593sm236693945e9.24.2026.04.26.14.48.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 26 Apr 2026 14:48:18 -0700 (PDT) Date: Sun, 26 Apr 2026 17:48:15 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Lorenzo Stoakes , "Liam R. Howlett" , Mike Rapoport , Johannes Weiner , Zi Yan Subject: [PATCH RFC v4 14/22] mm: page_reporting: skip redundant zeroing of host-zeroed reported pages Message-ID: <788d5a09394ed1456beaab23f75d91846ceeb611.1777223007.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When a guest reports free pages to the hypervisor via the page reporting framework (used by virtio-balloon and hv_balloon), the host typically zeros those pages when reclaiming their backing memory. However, when those pages are later allocated in the guest, post_alloc_hook() unconditionally zeros them again if __GFP_ZERO is set. This double-zeroing is wasteful, especially for large pages. Avoid redundant zeroing: - Add a host_zeroes_pages flag to page_reporting_dev_info, allowing drivers to declare that their host zeros reported pages on reclaim. A static key (page_reporting_host_zeroes) gates the fast path. - Add PG_zeroed page flag (sharing PG_private bit) to mark pages that have been zeroed by the host. Set it on reported pages during allocation from the buddy in page_del_and_expand(). - Thread the zeroed bool through rmqueue -> prep_new_page -> post_alloc_hook, where it skips redundant zeroing for __GFP_ZERO allocations. No driver sets host_zeroes_pages yet; a follow-up patch to virtio_balloon is needed to opt in. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- include/linux/page-flags.h | 9 +++++ include/linux/page_reporting.h | 3 ++ mm/compaction.c | 6 ++-- mm/internal.h | 2 +- mm/page_alloc.c | 66 +++++++++++++++++++++++----------- mm/page_reporting.c | 14 +++++++- mm/page_reporting.h | 12 +++++++ 7 files changed, 87 insertions(+), 25 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index f7a0e4af0c73..eef2499cba8b 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -135,6 +135,8 @@ enum pageflags { PG_swapcache =3D PG_owner_priv_1, /* Swap page: swp_entry_t in private */ /* Some filesystems */ PG_checked =3D PG_owner_priv_1, + /* Page contents are known to be zero */ + PG_zeroed =3D PG_private, =20 /* * Depending on the way an anonymous folio can be mapped into a page @@ -679,6 +681,13 @@ FOLIO_TEST_CLEAR_FLAG_FALSE(young) FOLIO_FLAG_FALSE(idle) #endif =20 +/* + * PageZeroed() tracks pages known to be zero. The allocator + * uses this to skip redundant zeroing in post_alloc_hook(). + */ +__PAGEFLAG(Zeroed, zeroed, PF_NO_COMPOUND) +#define __PG_ZEROED (1UL << PG_zeroed) + /* * PageReported() is used to track reported free pages within the Buddy * allocator. We can use the non-atomic version of the test and set diff --git a/include/linux/page_reporting.h b/include/linux/page_reporting.h index fe648dfa3a7c..10faadfeb4fb 100644 --- a/include/linux/page_reporting.h +++ b/include/linux/page_reporting.h @@ -13,6 +13,9 @@ struct page_reporting_dev_info { int (*report)(struct page_reporting_dev_info *prdev, struct scatterlist *sg, unsigned int nents); =20 + /* If true, host zeros reported pages on reclaim */ + bool host_zeroes_pages; + /* work struct for processing reports */ struct delayed_work work; =20 diff --git a/mm/compaction.c b/mm/compaction.c index c1039a9373e5..61209cd408ea 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -82,7 +82,8 @@ static inline bool is_via_compact_memory(int order) { ret= urn false; } =20 static struct page *mark_allocated_noprof(struct page *page, unsigned int = order, gfp_t gfp_flags) { - post_alloc_hook(page, order, __GFP_MOVABLE, USER_ADDR_NONE); + __ClearPageZeroed(page); + post_alloc_hook(page, order, __GFP_MOVABLE, false, USER_ADDR_NONE); set_page_refcounted(page); return page; } @@ -1832,7 +1833,8 @@ static struct folio *compaction_alloc_noprof(struct f= olio *src, unsigned long da set_page_private(&freepage[size], start_order); } dst =3D (struct folio *)freepage; - post_alloc_hook(&dst->page, order, __GFP_MOVABLE, USER_ADDR_NONE); + __ClearPageZeroed(&dst->page); + post_alloc_hook(&dst->page, order, __GFP_MOVABLE, false, USER_ADDR_NONE); set_page_refcounted(&dst->page); if (order) prep_compound_page(&dst->page, order); diff --git a/mm/internal.h b/mm/internal.h index 8e4616e42b4a..0600d824ba03 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -889,7 +889,7 @@ static inline void prep_compound_tail(struct page *head= , int tail_idx) } =20 void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flag= s, - unsigned long user_addr); + bool zeroed, unsigned long user_addr); extern bool free_pages_prepare(struct page *page, unsigned int order); =20 extern int user_min_free_kbytes; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index ca4f9c0948af..eff01a819744 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1774,6 +1774,7 @@ static __always_inline void page_del_and_expand(struc= t zone *zone, bool was_reported =3D page_reported(page); =20 __del_page_from_free_list(page, zone, high, migratetype); + nr_pages -=3D expand(zone, page, low, high, migratetype, was_reported); account_freepages(zone, -nr_pages, migratetype); } @@ -1846,8 +1847,10 @@ static inline bool should_skip_init(gfp_t flags) return (flags & __GFP_SKIP_ZERO); } =20 + inline void post_alloc_hook(struct page *page, unsigned int order, - gfp_t gfp_flags, unsigned long user_addr) + gfp_t gfp_flags, bool zeroed, + unsigned long user_addr) { bool init =3D !want_init_on_free() && want_init_on_alloc(gfp_flags) && !should_skip_init(gfp_flags); @@ -1856,6 +1859,14 @@ inline void post_alloc_hook(struct page *page, unsig= ned int order, =20 set_page_private(page, 0); =20 + /* + * If the page is zeroed, skip memory initialization. + * We still need to handle tag zeroing separately since the host + * does not know about memory tags. + */ + if (zeroed && init && !zero_tags) + init =3D false; + arch_alloc_page(page, order); debug_pagealloc_map_pages(page, 1 << order); =20 @@ -1913,13 +1924,13 @@ inline void post_alloc_hook(struct page *page, unsi= gned int order, } =20 static void prep_new_page(struct page *page, unsigned int order, gfp_t gfp= _flags, - unsigned int alloc_flags, - unsigned long user_addr) + unsigned int alloc_flags, bool zeroed, + unsigned long user_addr) { if (order && (gfp_flags & __GFP_COMP)) prep_compound_page(page, order); =20 - post_alloc_hook(page, order, gfp_flags, user_addr); + post_alloc_hook(page, order, gfp_flags, zeroed, user_addr); =20 /* * page is set pfmemalloc when ALLOC_NO_WATERMARKS was necessary to @@ -3189,6 +3200,7 @@ int __isolate_free_page(struct page *page, unsigned i= nt order) } =20 del_page_from_free_list(page, zone, order, mt); + __ClearPageZeroed(page); =20 /* * Set the pageblock if the isolated page is at least half of a @@ -3261,7 +3273,7 @@ static inline void zone_statistics(struct zone *prefe= rred_zone, struct zone *z, static __always_inline struct page *rmqueue_buddy(struct zone *preferred_zone, struct zone *zone, unsigned int order, unsigned int alloc_flags, - int migratetype) + int migratetype, bool *zeroed) { struct page *page; unsigned long flags; @@ -3296,6 +3308,8 @@ struct page *rmqueue_buddy(struct zone *preferred_zon= e, struct zone *zone, } } spin_unlock_irqrestore(&zone->lock, flags); + *zeroed =3D PageZeroed(page); + __ClearPageZeroed(page); } while (check_new_pages(page, order)); =20 __count_zid_vm_events(PGALLOC, page_zonenum(page), 1 << order); @@ -3357,10 +3371,9 @@ static int nr_pcp_alloc(struct per_cpu_pages *pcp, s= truct zone *zone, int order) /* Remove page from the per-cpu list, caller must protect the list */ static inline struct page *__rmqueue_pcplist(struct zone *zone, unsigned int order, - int migratetype, - unsigned int alloc_flags, + int migratetype, unsigned int alloc_flags, struct per_cpu_pages *pcp, - struct list_head *list) + struct list_head *list, bool *zeroed) { struct page *page; =20 @@ -3381,6 +3394,8 @@ struct page *__rmqueue_pcplist(struct zone *zone, uns= igned int order, page =3D list_first_entry(list, struct page, pcp_list); list_del(&page->pcp_list); pcp->count -=3D 1 << order; + *zeroed =3D PageZeroed(page); + __ClearPageZeroed(page); } while (check_new_pages(page, order)); =20 return page; @@ -3389,7 +3404,8 @@ struct page *__rmqueue_pcplist(struct zone *zone, uns= igned int order, /* Lock and remove page from the per-cpu list */ static struct page *rmqueue_pcplist(struct zone *preferred_zone, struct zone *zone, unsigned int order, - int migratetype, unsigned int alloc_flags) + int migratetype, unsigned int alloc_flags, + bool *zeroed) { struct per_cpu_pages *pcp; struct list_head *list; @@ -3408,7 +3424,8 @@ static struct page *rmqueue_pcplist(struct zone *pref= erred_zone, */ pcp->free_count >>=3D 1; list =3D &pcp->lists[order_to_pindex(migratetype, order)]; - page =3D __rmqueue_pcplist(zone, order, migratetype, alloc_flags, pcp, li= st); + page =3D __rmqueue_pcplist(zone, order, migratetype, alloc_flags, + pcp, list, zeroed); pcp_spin_unlock(pcp, UP_flags); if (page) { __count_zid_vm_events(PGALLOC, page_zonenum(page), 1 << order); @@ -3433,19 +3450,19 @@ static inline struct page *rmqueue(struct zone *preferred_zone, struct zone *zone, unsigned int order, gfp_t gfp_flags, unsigned int alloc_flags, - int migratetype) + int migratetype, bool *zeroed) { struct page *page; =20 if (likely(pcp_allowed_order(order))) { page =3D rmqueue_pcplist(preferred_zone, zone, order, - migratetype, alloc_flags); + migratetype, alloc_flags, zeroed); if (likely(page)) goto out; } =20 page =3D rmqueue_buddy(preferred_zone, zone, order, alloc_flags, - migratetype); + migratetype, zeroed); =20 out: /* Separate test+clear to avoid unnecessary atomics */ @@ -3836,6 +3853,7 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int o= rder, int alloc_flags, struct pglist_data *last_pgdat =3D NULL; bool last_pgdat_dirty_ok =3D false; bool no_fallback; + bool zeroed; bool skip_kswapd_nodes =3D nr_online_nodes > 1; bool skipped_kswapd_nodes =3D false; =20 @@ -3980,10 +3998,11 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int= order, int alloc_flags, =20 try_this_zone: page =3D rmqueue(zonelist_zone(ac->preferred_zoneref), zone, order, - gfp_mask, alloc_flags, ac->migratetype); + gfp_mask, alloc_flags, ac->migratetype, + &zeroed); if (page) { prep_new_page(page, order, gfp_mask, alloc_flags, - ac->user_addr); + zeroed, ac->user_addr); =20 /* * If this is a high-order atomic allocation then check @@ -4217,9 +4236,11 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigne= d int order, count_vm_event(COMPACTSTALL); =20 /* Prep a captured page if available */ - if (page) - prep_new_page(page, order, gfp_mask, alloc_flags, + if (page) { + __ClearPageZeroed(page); + prep_new_page(page, order, gfp_mask, alloc_flags, false, ac->user_addr); + } =20 /* Try get a page from the freelist if available */ if (!page) @@ -5193,6 +5214,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, /* Attempt the batch allocation */ pcp_list =3D &pcp->lists[order_to_pindex(ac.migratetype, 0)]; while (nr_populated < nr_pages) { + bool zeroed =3D false; =20 /* Skip existing pages */ if (page_array[nr_populated]) { @@ -5201,7 +5223,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, } =20 page =3D __rmqueue_pcplist(zone, 0, ac.migratetype, alloc_flags, - pcp, pcp_list); + pcp, pcp_list, &zeroed); if (unlikely(!page)) { /* Try and allocate at least one page */ if (!nr_account) { @@ -5212,7 +5234,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, } nr_account++; =20 - prep_new_page(page, 0, gfp, 0, USER_ADDR_NONE); + prep_new_page(page, 0, gfp, 0, zeroed, USER_ADDR_NONE); set_page_refcounted(page); page_array[nr_populated++] =3D page; } @@ -6983,7 +7005,8 @@ static void split_free_frozen_pages(struct list_head = *list, gfp_t gfp_mask) list_for_each_entry_safe(page, next, &list[order], lru) { int i; =20 - post_alloc_hook(page, order, gfp_mask, USER_ADDR_NONE); + __ClearPageZeroed(page); + post_alloc_hook(page, order, gfp_mask, false, USER_ADDR_NONE); if (!order) continue; =20 @@ -7188,8 +7211,9 @@ int alloc_contig_frozen_range_noprof(unsigned long st= art, unsigned long end, } else if (start =3D=3D outer_start && end =3D=3D outer_end && is_power_o= f_2(end - start)) { struct page *head =3D pfn_to_page(start); =20 + __ClearPageZeroed(head); check_new_pages(head, order); - prep_new_page(head, order, gfp_mask, 0, USER_ADDR_NONE); + prep_new_page(head, order, gfp_mask, 0, false, USER_ADDR_NONE); } else { ret =3D -EINVAL; WARN(true, "PFN range: requested [%lu, %lu), allocated [%lu, %lu)\n", diff --git a/mm/page_reporting.c b/mm/page_reporting.c index f0042d5743af..6177d2413743 100644 --- a/mm/page_reporting.c +++ b/mm/page_reporting.c @@ -50,6 +50,8 @@ EXPORT_SYMBOL_GPL(page_reporting_order); #define PAGE_REPORTING_DELAY (2 * HZ) static struct page_reporting_dev_info __rcu *pr_dev_info __read_mostly; =20 +DEFINE_STATIC_KEY_FALSE(page_reporting_host_zeroes); + enum { PAGE_REPORTING_IDLE =3D 0, PAGE_REPORTING_REQUESTED, @@ -129,8 +131,11 @@ page_reporting_drain(struct page_reporting_dev_info *p= rdev, * report on the new larger page when we make our way * up to that higher order. */ - if (PageBuddy(page) && buddy_order(page) =3D=3D order) + if (PageBuddy(page) && buddy_order(page) =3D=3D order) { __SetPageReported(page); + if (page_reporting_host_zeroes_pages()) + __SetPageZeroed(page); + } } while ((sg =3D sg_next(sg))); =20 /* reinitialize scatterlist now that it is empty */ @@ -386,6 +391,10 @@ int page_reporting_register(struct page_reporting_dev_= info *prdev) /* Assign device to allow notifications */ rcu_assign_pointer(pr_dev_info, prdev); =20 + /* enable zeroed page optimization if host zeroes reported pages */ + if (prdev->host_zeroes_pages) + static_branch_enable(&page_reporting_host_zeroes); + /* enable page reporting notification */ if (!static_key_enabled(&page_reporting_enabled)) { static_branch_enable(&page_reporting_enabled); @@ -410,6 +419,9 @@ void page_reporting_unregister(struct page_reporting_de= v_info *prdev) =20 /* Flush any existing work, and lock it out */ cancel_delayed_work_sync(&prdev->work); + + if (prdev->host_zeroes_pages) + static_branch_disable(&page_reporting_host_zeroes); } =20 mutex_unlock(&page_reporting_mutex); diff --git a/mm/page_reporting.h b/mm/page_reporting.h index c51dbc228b94..736ea7b37e9e 100644 --- a/mm/page_reporting.h +++ b/mm/page_reporting.h @@ -15,6 +15,13 @@ DECLARE_STATIC_KEY_FALSE(page_reporting_enabled); extern unsigned int page_reporting_order; void __page_reporting_notify(void); =20 +DECLARE_STATIC_KEY_FALSE(page_reporting_host_zeroes); + +static inline bool page_reporting_host_zeroes_pages(void) +{ + return static_branch_unlikely(&page_reporting_host_zeroes); +} + static inline bool page_reported(struct page *page) { return static_branch_unlikely(&page_reporting_enabled) && @@ -46,6 +53,11 @@ static inline void page_reporting_notify_free(unsigned i= nt order) #else /* CONFIG_PAGE_REPORTING */ #define page_reported(_page) false =20 +static inline bool page_reporting_host_zeroes_pages(void) +{ + return false; +} + static inline void page_reporting_notify_free(unsigned int order) { } --=20 MST From nobody Mon Jun 15 18:00:34 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B7292261B70 for ; Sun, 26 Apr 2026 21:48:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240107; cv=none; b=R/vZmZbzJvF/hvUKozWhPc9PyJQVjdbCbESM+c7UWZl7l5Zcb2y5ivzhKTDxTBhdX/tMrY8BpaqE7rbhxh6NXkg6XS1r86dPhtbedFcdQ6cfmlhDEQlxcDde/vqgONr9uXdKMhkA60woqlw8FaCXQ1sCpnYMml2AdzQQzwnsUxE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240107; c=relaxed/simple; bh=GQIxt3426GlDeqT1vnTp0KvfVXzR4vpnMpFfUxs113g=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=hfTkx4gy9kKn4g9Ze8jqYTomZrPQ//zcUHxZsXLRnXeqmYHAjNBpDMUpCnAyRaBAUTMq/V+Tx0qKiQTajad2wOBF8yWKFst07HTK/Jn6JaBqG6QQy2kRUxsqrRoDC5iMzKo129iGflriDoHRnlhUS0TZHVbZ7P1ocGElLfJanb4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=WC3Cw5uB; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=dEL1bdtK; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="WC3Cw5uB"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="dEL1bdtK" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777240104; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=iX7QlDEIRvMq/lhdobB75J6k8cJUQBTaq/dVJzKRSYI=; b=WC3Cw5uBo3HFGVPImiwsIktgHhwArlKdGK5QT+D6y7ogEu7XY+OwZJK/lsesFnTVqSyOHL mqGFpwlfehXy8qEMwKvT+A7oZZmqL32hlKmB6TBTLvGPKExpfbrpAV+t2r8V9PhcxIFO7H HUiTr8oYx/65LSAzybHNbysxO7lCefc= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-391-jc7XGKDaNLe1SNGigepoJA-1; Sun, 26 Apr 2026 17:48:23 -0400 X-MC-Unique: jc7XGKDaNLe1SNGigepoJA-1 X-Mimecast-MFC-AGG-ID: jc7XGKDaNLe1SNGigepoJA_1777240103 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-4411a215f17so5934004f8f.1 for ; Sun, 26 Apr 2026 14:48:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1777240102; x=1777844902; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=iX7QlDEIRvMq/lhdobB75J6k8cJUQBTaq/dVJzKRSYI=; b=dEL1bdtKYVAuagDpkX9IzYvbwk3W1NeKPbSakSQvDJsoutMVmVWazSPdjCTr36/c2P qOuigJkTvcjV8XQ/ZpkLDkjxx601fcZNTmg5258oZTrePdaAv9wz4DAain4ba8mfQXY0 iehw07Ot/PDCeX6ZXWSAw1m2hF2k0vKCEDbOqZu/IbU4e5HKuxVe6gl8M7Pd2Bi+YcUX z0gVJc5jNrNXx7BeBeHxkfarN5LGfBauMyrJ7MB5/A7WducZwD/UecTxMUkfX7t3uZA+ fvGgeT3PtlaefP0nV7blvBu2OI3RUNhY+QRqFv2UwwRjABbunlO7y2VMOTm1y5v3tNB7 x2mQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777240102; x=1777844902; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iX7QlDEIRvMq/lhdobB75J6k8cJUQBTaq/dVJzKRSYI=; b=POtkoA8CIMMtRdRrsgYdzwQdsVKB+xtW6yAQdyRhgRXXsJbGHFZ1CSW88RIwN4R/Fo k5BRyGURHVhcyTyNhwTPq9JD8UVCm0aZZbU701RSwEkQPgbXcn3e+6GVSlJ76EQy/sdU EWNqTPxGjWg+QVOfS2HXIxhjSqtdAy3tt+YHZnDXu5Lh/1gc1ymWj+BL96QinIXYT+Uq VJYiHXpH5Dbq6BPuRSoQ7ksk5nIHHZ/V8HcY8Tp/qOgjypkwzYw/7JigRMSst7rd4mMO riZRFFc7Rg4WFLMaWktzyid3sdKGl4lCZbgrNKib0PTP1r8qGEMfTBcIKyco+wfxVZTL LE8A== X-Gm-Message-State: AOJu0Yw6Ahu9LnMaV2k5Tpvo6+ArAwj3R4OgiCJnDhrAnEtZlgttTkVb oZN0M9+9cOwWToc0PY4sKP6vTkTm/BIpFKS+YqqDC2ac1dzZpQaXFoI+olhxoiLiEavY5t8ELSm ZL5KZVj5+s9ULAr/rDRzK+c32n8RQt6jnHykHQkt4mThkDayBjZR+pIOR2qL0JziQTys/73200N jZQHZcRAT1KTpvT9QTbeolVtHOpt/VpvZpvLKG0dbxc6k= X-Gm-Gg: AeBDiesVpF3GRy5cTZG8MC5diRNRg9xPHsET/Y/HToUDssDgAAZyFivxzzzu0airrgY EBM2Fd0w6slg1n/3TQ2Gs+nVU9bPAjT3Vwv7SKuE5RT/9pl1P91rGN0T0bgJPrlcClsP7bj793W VNkn7MKZIOIVyH/PFMWqSgpx9Wm5Hl/VVMJg23R2dfVhTUNMPJkeL5Kufp1YjTjXNYqG8aJwWMf hyvD8UuKhFjvgbQRmosxWFlJvCgiUbUCPs6oJoe2X8h3pz8povkSo4BHMC3ipWt4X+dJ+KC4luO QE57A6/QuP2IjJ519Mjf6Z19oDPSUoZ+eMboYtn4l1QhCjOznHQEp0KIVG0J+zMqOC5RWKxUo19 fNBL8ehOhO1fun7cLH5VjWyPFaSpxWtMN+bBOrqb+oE0R50dbhYylZ8UB X-Received: by 2002:a05:6000:420a:b0:43c:4810:dc4a with SMTP id ffacd0b85a97d-43fe3e14f81mr62740571f8f.38.1777240102289; Sun, 26 Apr 2026 14:48:22 -0700 (PDT) X-Received: by 2002:a05:6000:420a:b0:43c:4810:dc4a with SMTP id ffacd0b85a97d-43fe3e14f81mr62740532f8f.38.1777240101794; Sun, 26 Apr 2026 14:48:21 -0700 (PDT) Received: from redhat.com (IGLD-80-230-47-179.inter.net.il. [80.230.47.179]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-43fe4cc0f31sm76656472f8f.12.2026.04.26.14.48.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 26 Apr 2026 14:48:21 -0700 (PDT) Date: Sun, 26 Apr 2026 17:48:19 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Johannes Weiner , Zi Yan Subject: [PATCH RFC v4 15/22] mm: page_alloc: clear PG_zeroed on buddy merge if not both zero Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When two buddy pages merge in __free_one_page(), preserve PG_zeroed on the merged page only if both buddies have the flag set. Otherwise clear it. Without this, a zeroed page (freed via free_frozen_pages_zeroed from balloon deflate) could merge with a non-zero buddy. The merged page would inherit PG_zeroed, and a later __GFP_ZERO allocation would skip zeroing stale data in the non-zero half. The page reporting path is not affected: it sets PG_zeroed during allocation (page_del_and_expand), not on free list pages. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/page_alloc.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index eff01a819744..1183ef3e91c9 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -984,10 +984,14 @@ static inline void __free_one_page(struct page *page, unsigned long buddy_pfn =3D 0; unsigned long combined_pfn; struct page *buddy; + bool buddy_zeroed; + bool page_zeroed; bool to_tail; =20 VM_BUG_ON(!zone_is_initialized(zone)); - VM_BUG_ON_PAGE(page->flags.f & PAGE_FLAGS_CHECK_AT_PREP, page); + /* PG_zeroed (aliased to PG_private) is valid on free-list pages */ + VM_BUG_ON_PAGE(page->flags.f & + (PAGE_FLAGS_CHECK_AT_PREP & ~__PG_ZEROED), page); =20 VM_BUG_ON(migratetype =3D=3D -1); VM_BUG_ON_PAGE(pfn & ((1 << order) - 1), page); @@ -1022,6 +1026,8 @@ static inline void __free_one_page(struct page *page, goto done_merging; } =20 + buddy_zeroed =3D PageZeroed(buddy); + /* * Our buddy is free or it is CONFIG_DEBUG_PAGEALLOC guard page, * merge with it and move up one order. @@ -1040,10 +1046,17 @@ static inline void __free_one_page(struct page *pag= e, change_pageblock_range(buddy, order, migratetype); } =20 + page_zeroed =3D PageZeroed(page); + __ClearPageZeroed(page); + __ClearPageZeroed(buddy); + combined_pfn =3D buddy_pfn & pfn; page =3D page + (combined_pfn - pfn); pfn =3D combined_pfn; order++; + + if (page_zeroed && buddy_zeroed) + __SetPageZeroed(page); } =20 done_merging: --=20 MST From nobody Mon Jun 15 18:00:34 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E4C8E2F0661 for ; Sun, 26 Apr 2026 21:48:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240110; cv=none; b=YfNAG69/lG2rnNWBsypMwXEOn/LKPvWHxDX7JHOSkxmCgM9PFc74+5HQK8KiKC59IDAQZ1zeFJypwgsH2WC8gttupcSAublvywGwcBD+SATiRtDOP8ngoLFjShBpMRueT/DFrHcUk/20gINfWWtBjGIMxbp6QWSX6YYO7UjSVpA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240110; c=relaxed/simple; bh=ddif2Keg7MsypRPUTjBR227ZRe8X6d5tcSgDIOVelv4=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=eQ4bx520eApdx6z4dJ89v88kcICNCxznFhpHQMUfJju55DHaXZeQcIHx42MgUVdbf5CbJc8MoN082Ie/h4nbM1fQcZ0TwJquyeW3GsVbE2DzR+S36Gs35oEzbOLwoWGrIOQxp3ryoRKUfwUmP123nqVKbzTJG4ik8yRj3d678w0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=aRhNMpdU; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=mUzj/irs; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="aRhNMpdU"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="mUzj/irs" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777240107; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=AlVg98QjuOmMUhZ+nzy6BehsEB+gtg7rKVvx2cLdIro=; b=aRhNMpdUfzeK5d68kgPLAPYJMi5OcNDPmhj/HvXJ1I3/KpcvOer07NBeALNGYyzptYskHi dZacY6LDuvyx2yneNP3Fl4/jYfqnazpkjlFOiqkVtPH0vQ4R0GF2yodLurM96KxTmHzRb7 gsrrtIRdZ5BSfB/PGErXr2ac/uuTxG4= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-660-k9NdOVf6O1ONTNnMu9bVbg-1; Sun, 26 Apr 2026 17:48:26 -0400 X-MC-Unique: k9NdOVf6O1ONTNnMu9bVbg-1 X-Mimecast-MFC-AGG-ID: k9NdOVf6O1ONTNnMu9bVbg_1777240105 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-43fe791a398so7399446f8f.0 for ; Sun, 26 Apr 2026 14:48:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1777240105; x=1777844905; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=AlVg98QjuOmMUhZ+nzy6BehsEB+gtg7rKVvx2cLdIro=; b=mUzj/irs2hTh/JpZq3BlZVkL4zZ1u5TI78jCUu0cZ8fOqi777qbwulJ86Ann4L2Dmk e6XnLkjlc5PoIJOMIRp6zj8HqHdyEX2sc4n65JlWKA+k/LDNaCTSx1lUnU7Fnf/bv0Xu oYa0rLnnJBC9pWMJUFDNt+byC80eFWHvsxDaLFMtg1/tPEyAk0c5j7Kkr+J8xQ3h2PL5 MfYxrwFDhItHNfa0+GS2d1TMFK2QIRz5ncfdJknXzZtFqlDAiWxwze6OwkZ7BnpXQzvV vSVsHLFnBU4eEBqX8+r72EG6s6cuhc3h8AFZ7s2sg2E0zowHV9O++fNiSc2GmDtNvsq8 VXmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777240105; x=1777844905; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AlVg98QjuOmMUhZ+nzy6BehsEB+gtg7rKVvx2cLdIro=; b=Pr4B/woKpApOEkVV8BgCi6lGzgenO8wqJ7yjT4E0PY/+bQHfya7TDQjup1Ep1DRL6B 7AE/8iovLHCOvn0P8BrWd17F5wfHDQ5Fq1OKRWdafauX0HdvlaUnuu31VW8KIA+mgWVO 6zaMmKdf7qjKzYdfumhuH66u7qEGXfNDK8ti+iMWF6otzZXWvOnCVtm8ao6r5Oj0akQ8 BhHiW9FqLQNQS4F808B67dXiSQLs38/hse2P5/HR17mA4IYwqqFD9TXabqgOZ/cqOaDz Hdf8Va0hVNhLDd+tmvjjLZscJ5c8r9w1QzFFiLg1xoHIWzSpz3Ex5VTA+hQVgANhlyET D5Ng== X-Gm-Message-State: AOJu0Yxx9+lzddBq307wGtUGNmHQUG1WiunIC1KIAQYMUk3Cu9JfY0K0 ZQ1mNVz8c6imlwq4mP5Qzw/kpB9ng0b+Xo4mEboM1pVLtaBRo4WzZQnzbYCEMvO85g4jlMLJcOB aaWxHJNNmcudfhACS1Jr96qQ/OloeLJJyNMioQNYcanDue4cyYzyMPkYmYTNRbOfi9ShC2GTJvW 7NbtYxOOjPWK7I6cHcvY7r1mKW4oiqnjCo1DwCAqVNWTg= X-Gm-Gg: AeBDieuG1DeGaNNZEOD89Hy5XXfWTHQFcuncC3zy6BtI2mWZGj7h574T5f8X9KAH+e1 0lCJ29CQ+bH/uy+Pgz5hKHcztq3WH3qq3Nt62imguevIMFYCw9sYivTn+eiGZwP0b7usXGbqTyO hUFaOMKpuJXWfB+kNjTAcjfjefzo9YVYFhSvw0JIOBrXBUvuBehJ7yU+4bJ6Kb6etsO3DxbmPTn 4m4N8VVkzR+9dyvWKkTg13+qQBh3ol669Iut7JwUqr8qcZfCLIxfb+g28FucgTCsmKpuZ+CNc3V 209VK0biFEfRLWqi+O/d1Q0Kxe6LddeXuU6eIO3n+sHvI1/G8yKIpOVDtIMQnh8PGUeRw4MeU+3 W8nrGNwEV2Zm9b/1e1hzhFo42Wb2ugFMKqIw9VBOXHIAx+v/HxsGnoUIE X-Received: by 2002:a05:6000:24c7:b0:43c:fe0e:5bb9 with SMTP id ffacd0b85a97d-43fe3dd4ca7mr63036152f8f.12.1777240105138; Sun, 26 Apr 2026 14:48:25 -0700 (PDT) X-Received: by 2002:a05:6000:24c7:b0:43c:fe0e:5bb9 with SMTP id ffacd0b85a97d-43fe3dd4ca7mr63036104f8f.12.1777240104579; Sun, 26 Apr 2026 14:48:24 -0700 (PDT) Received: from redhat.com (IGLD-80-230-47-179.inter.net.il. [80.230.47.179]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-43fe4e4ffa8sm78084778f8f.35.2026.04.26.14.48.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 26 Apr 2026 14:48:24 -0700 (PDT) Date: Sun, 26 Apr 2026 17:48:21 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Johannes Weiner , Zi Yan Subject: [PATCH RFC v4 16/22] mm: page_alloc: preserve PG_zeroed in page_del_and_expand Message-ID: <1db0cfd570fff4b93688619eba55c655be589b35.1777223007.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Don't unconditionally clear PG_zeroed for non-reported pages in page_del_and_expand(). Pages freed via free_frozen_pages_zeroed (balloon deflate) already have the flag set and should keep it through buddy allocation, not just PCP reuse. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/page_alloc.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 1183ef3e91c9..1169714406e7 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1743,7 +1743,8 @@ struct page *__pageblock_pfn_to_page(unsigned long st= art_pfn, * -- nyc */ static inline unsigned int expand(struct zone *zone, struct page *page, in= t low, - int high, int migratetype, bool reported) + int high, int migratetype, bool reported, + bool zeroed) { unsigned int size =3D 1 << high; unsigned int nr_added =3D 0; @@ -1774,6 +1775,8 @@ static inline unsigned int expand(struct zone *zone, = struct page *page, int low, */ if (reported) __SetPageReported(&page[size]); + if (zeroed) + __SetPageZeroed(&page[size]); } =20 return nr_added; @@ -1785,10 +1788,12 @@ static __always_inline void page_del_and_expand(str= uct zone *zone, { int nr_pages =3D 1 << high; bool was_reported =3D page_reported(page); + bool was_zeroed =3D PageZeroed(page); =20 __del_page_from_free_list(page, zone, high, migratetype); =20 - nr_pages -=3D expand(zone, page, low, high, migratetype, was_reported); + nr_pages -=3D expand(zone, page, low, high, migratetype, was_reported, + was_zeroed); account_freepages(zone, -nr_pages, migratetype); } =20 @@ -2365,11 +2370,13 @@ try_to_claim_block(struct zone *zone, struct page *= page, /* Take ownership for orders >=3D pageblock_order */ if (current_order >=3D pageblock_order) { unsigned int nr_added; + bool was_reported =3D page_reported(page); + bool was_zeroed =3D PageZeroed(page); =20 del_page_from_free_list(page, zone, current_order, block_type); change_pageblock_range(page, current_order, start_type); nr_added =3D expand(zone, page, order, current_order, start_type, - false); + was_reported, was_zeroed); account_freepages(zone, nr_added, start_type); return page; } --=20 MST From nobody Mon Jun 15 18:00:34 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BBEEA35898 for ; Sun, 26 Apr 2026 21:48:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240114; cv=none; b=osKTsiuKVnCru6Bhozm8Ol80E/QfPxSUqvP+jUicXMZ7xfcS+ZvX4fG1rSupVeYAcoSyBJb4UJsab4Wi7BcOjxszSdxo51LtTNCvSUlWMJnyCQEAeG148V3AIoc4MuW7Ej22UnY8SHsWZu8g0j/vxHOkuUsF+hGoAe/27flJj3M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240114; c=relaxed/simple; bh=/H/b9HRDS/FqkVfSUr24VvwENCpz3EuK5eZWes4C9AA=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=aBtYch+jwgMKZijbuUzGGf9XImv7J6UxwI8+sA+pr3CLCHZMswj0XFXh9QmvE+Gs2AJcVqNlmrTOHYglfg0alAR2M3JqiDeITbpw8H4F+dRr83Un8e6eCAgnPk92x7yinwyyD9FcA3fZYPxuEFQGeY7ukYYix9l5QVXjWpQ2co8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=SwcsLbti; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=Olmy6e6U; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="SwcsLbti"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="Olmy6e6U" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777240110; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=fYAXl7F7h4aq1MzyiFunWzYtUjLEe/+xUEddBLdqVEQ=; b=SwcsLbtitcVK8jbszpUgM3IryzkOa1gNUYYMushC/1zT9kgtGN6nN6cGsqnJLeC7HS74zb 6A2OavUx2pj2sP964qLQZitSLAeAgnJSuysnXscGMyTXI0NYgGRPc+xsv905j/N+UP+xzL +R9v0tPcdF5zB0dcEgpw8c7nRUodC28= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-367-OVAC0wu0ObG6ugIUpzcKLw-1; Sun, 26 Apr 2026 17:48:29 -0400 X-MC-Unique: OVAC0wu0ObG6ugIUpzcKLw-1 X-Mimecast-MFC-AGG-ID: OVAC0wu0ObG6ugIUpzcKLw_1777240108 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-43d1fec59c9so6059654f8f.0 for ; Sun, 26 Apr 2026 14:48:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1777240108; x=1777844908; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=fYAXl7F7h4aq1MzyiFunWzYtUjLEe/+xUEddBLdqVEQ=; b=Olmy6e6UIf+Bh1unV4FKEwxFSvxID3rwgmdhTp4I+6SiR6L7ABQbUHVWPbJ2Gw4lys JcvNzeq5Cqw3oxTbXML2gRiCAM7FCcrETJEpTMG0YxalFQOkbu1tnXkmgbL+688A51la t6sme+ocPA4Upou2ZZKzYtMpG+9oNaql7VR/kJ7BCGMUkbOvfYeztsAi8i7JwcO9CL4T PmPpV6bKIdWFnPopKZr6k0AiD9zGp+N4U7UYJrhkgROT4h1JPfID3MnFopTLg9awUlbO UFeWHTcXsTSNOzipo4luuP6xoFYKftSDAs7PlXayAQgyR7q+0R8AGfw6uo+7arvucKQ/ 8+7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777240108; x=1777844908; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fYAXl7F7h4aq1MzyiFunWzYtUjLEe/+xUEddBLdqVEQ=; b=M+tcG9YGhEFNnPSv9fEiqObFYIYzZlmiknzHBLfHUowt3wXDnKqYed0lUu6ZT9I/9u KHVizDlKV+IkF56+onBgoV1xJjnUvdtPOhjRi6r8BHh3BejVQRtB5s+WEtRMDRSQFSdA 9DSit7iDX+Rnjlhoskp8PWrnvsRyyURYsR3Hpq4n3MLD6UhlUoMxdNNgtS0T62RWhD61 yAsp3LSKurmlhc4wks40Vuxs6Ia2i1VeNp04VSMimskUcUpkpB2jPZJjeUxLVa+K+FYQ jBG5nxULuhBBQ6hUsRs8EbZaJLGWSuSfQqR/Jk7ITw5goQ1HG00rc2Tq/BmSfUMSTdOx BQ5g== X-Gm-Message-State: AOJu0Yzq084+1bWZv/riRbxWCTSI1hYAviBL7ClVyZQvVbK4pRPaf6Gi IWncuW9UHVCLW/vM6mkRpj1P74AHu/A3UA83vG2mRnqcA+bXnaFgKC8srd9fzDiNqPupjZoJuhY ZtJTL/8J1DYEhmdWExvemd/WJ5SyL+dbzKb2kwqI2Rkl/vGp6ayqs0GLS3ks40yG1LoJP1lKr8O uyj0ZRZD3ttSQg6c5HqfuBjHOXJWs6SN7g1nC5TmRbsRs= X-Gm-Gg: AeBDiesYuw7etspcREVtfRoqp8s24LHA4/FHfJfj8XlKFp2EFRtmVU0WKgcIT9dniu1 p8j+2xrW/5Lg4jjAimGQfZ0WGXZBswxS25HTZ9bRSq7vleuEL1pVJhn2gDR6FklZ6f+iQ92wjOE a/OrF0xUCx1YPMixYaZ10Vf7EmB6PgH/rSNvf6n2mJPJwPAra4vYWifNEsjZQ219H/gaexdwWKq 3p39Nau80hUaFDbGX9+U0NnFRDJsB4jl8pEP+CQ6aaC24F6m7rY20wQRoH5llciS2UdO/wuzesW xXkBVXhfHO7Uu+XGv553qcFHf+ntMRTU2kFqSow6j8B5ZQ16RbIV40mmhHnHpeIdim69rTO0BMR hELN7Y9ayrB9WnBmGDFUIC9y6R2664Nw2oFVuBhZtCZBVg0eVdf0L3pnh X-Received: by 2002:a05:6000:290c:b0:43d:772d:2b61 with SMTP id ffacd0b85a97d-43fe4073831mr59751637f8f.15.1777240108122; Sun, 26 Apr 2026 14:48:28 -0700 (PDT) X-Received: by 2002:a05:6000:290c:b0:43d:772d:2b61 with SMTP id ffacd0b85a97d-43fe4073831mr59751578f8f.15.1777240107558; Sun, 26 Apr 2026 14:48:27 -0700 (PDT) Received: from redhat.com (IGLD-80-230-47-179.inter.net.il. [80.230.47.179]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-43fe4e3a79esm79138134f8f.17.2026.04.26.14.48.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 26 Apr 2026 14:48:27 -0700 (PDT) Date: Sun, 26 Apr 2026 17:48:24 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Lorenzo Stoakes , "Liam R. Howlett" , Mike Rapoport , Johannes Weiner , Zi Yan Subject: [PATCH RFC v4 17/22] mm: page_reporting: add per-page zeroed bitmap for host feedback Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The host may skip zeroing some reported pages (e.g., due to alignment constraints or bounce buffer fallback in QEMU). Currently, when host_zeroes_pages is set, all reported pages are unconditionally marked PG_zeroed -- even ones the host did not actually zero. Add a zeroed_bitmap to page_reporting_dev_info that the report() callback can use to indicate which pages were actually zeroed. Before calling report(), the framework initializes the bitmap: all bits set if host_zeroes_pages (optimistic default), all clear otherwise. The driver's report() can then clear bits for pages that were not zeroed. page_reporting_drain() checks the bitmap per-page instead of the global host_zeroes_pages flag. No driver uses per-page feedback yet; the bitmap is pre-filled based on host_zeroes_pages, so behavior is unchanged. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/page_reporting.h | 7 +++++++ mm/page_reporting.c | 8 ++++++-- 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/include/linux/page_reporting.h b/include/linux/page_reporting.h index 10faadfeb4fb..5b63de21949c 100644 --- a/include/linux/page_reporting.h +++ b/include/linux/page_reporting.h @@ -16,6 +16,13 @@ struct page_reporting_dev_info { /* If true, host zeros reported pages on reclaim */ bool host_zeroes_pages; =20 + /* + * Per-page zeroed status, indexed by scatterlist position. + * The driver's report() callback must clear the bitmap, + * then set bits for pages that were actually zeroed. + */ + DECLARE_BITMAP(zeroed_bitmap, PAGE_REPORTING_CAPACITY); + /* work struct for processing reports */ struct delayed_work work; =20 diff --git a/mm/page_reporting.c b/mm/page_reporting.c index 6177d2413743..61f7f08d02d4 100644 --- a/mm/page_reporting.c +++ b/mm/page_reporting.c @@ -108,6 +108,7 @@ page_reporting_drain(struct page_reporting_dev_info *pr= dev, struct scatterlist *sgl, unsigned int nents, bool reported) { struct scatterlist *sg =3D sgl; + unsigned int i =3D 0; =20 /* * Drain the now reported pages back into their respective @@ -122,7 +123,7 @@ page_reporting_drain(struct page_reporting_dev_info *pr= dev, =20 /* If the pages were not reported due to error skip flagging */ if (!reported) - continue; + goto next; =20 /* * If page was not commingled with another page we can @@ -133,9 +134,12 @@ page_reporting_drain(struct page_reporting_dev_info *p= rdev, */ if (PageBuddy(page) && buddy_order(page) =3D=3D order) { __SetPageReported(page); - if (page_reporting_host_zeroes_pages()) + if (page_reporting_host_zeroes_pages() && + test_bit(i, prdev->zeroed_bitmap)) __SetPageZeroed(page); } +next: + i++; } while ((sg =3D sg_next(sg))); =20 /* reinitialize scatterlist now that it is empty */ --=20 MST From nobody Mon Jun 15 18:00:34 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2E3E630AAB8 for ; Sun, 26 Apr 2026 21:48:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240115; cv=none; b=M3JT/qIDQnYi1HKPGk2bu+HIGuGu+Mna3tqnBsgySNqan0UzHcaG8qJ0FjAZwU8tOOJdZAivzzMPMsm4ei+JzcfrEB3ba+1AkjUI7jEToy3+gBcXAobXe6v7oaTCvbLMq/MNQ9h+UFHvHb7FMEgcLkIndsS3uMsFqLRRtPKooUA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240115; c=relaxed/simple; bh=qQW9TjyXt/G4RVZeOZhv7EWw5q3drusbpwikbnE9UQk=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=eogw/7mxrmJpE19sNAM05IcVuenciqfMsBth5imnq2DOxN3bkx+jwmrgBv5lUM4F9e0LEPVgNL28QvA/nfFmrXMT2c5ov78OS9FkViXJ3QQBnIJwsFw6+j/Q+58acEzJzrtKSGhoB1horeA6qiHMnxbHkUI6PcK7om4rZQbYmsU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=DCdWsxYR; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=Pp3lNv3V; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="DCdWsxYR"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="Pp3lNv3V" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777240113; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=mdZwqDH0Vgol6kXYX8uKZUPg/RwJvIkyVtJHSwLpmoU=; b=DCdWsxYR1qoWTm+xLybCzZEQaT7ENvceNU24zKbpHs7ogwH3hygfvT4zHQQjvmF6aEibPW +GJVenaidQq685ywlrt1FaALW5Ke8xLta2+lfhVoX1+xCuyKyJWRYSMndvXm66Qcbde4pl Q9BFRSmeCmO6NcHj1LQ5rTN3gfu59eQ= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-193-vfm4lWBSORuPXuf5faS-IQ-1; Sun, 26 Apr 2026 17:48:32 -0400 X-MC-Unique: vfm4lWBSORuPXuf5faS-IQ-1 X-Mimecast-MFC-AGG-ID: vfm4lWBSORuPXuf5faS-IQ_1777240111 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-48a548b585cso40731785e9.2 for ; Sun, 26 Apr 2026 14:48:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1777240111; x=1777844911; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=mdZwqDH0Vgol6kXYX8uKZUPg/RwJvIkyVtJHSwLpmoU=; b=Pp3lNv3VM+CLUScJgXpLmtWBT600TUZMkQpo4t5JWM+hBq9RS2GYpW9MHSH4JT0hUP jVNqgyMx3+ZZ70cMSiBuL0S6zdL1VH+v/eeo2Zfa3V+w3fvsWhMhAnEdmChkOzamu10m 4DyEGOsPBdlw9c9DgJKhMHfP40ZgCRjy1XieVutu0LqXe1Yy3HlPcO8LEQyvJL1fEKPa 6CE+DVS/JwzGXVUTBS3JdurrRhe9yL47FIFuOCpC+UJvOGMrIACVi6OATip5HHfAQPHQ PYw9gCNoIqLPvic4fkmyTM3icg13cUOF/mw5m1P1MoZnqkDkNiY0dceHFxp+8ZHQKqfd b+gg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777240111; x=1777844911; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mdZwqDH0Vgol6kXYX8uKZUPg/RwJvIkyVtJHSwLpmoU=; b=VrBM+J0x/kcbGe2ImAUw1YwTXAgdu9pNJOabivREyO/liKZGoqKiB0lCg9QNs9EMIk 1k6WPAlcvWaeGpA33956fybotGRuRmjH4PN+OCBaecMZyjS0OycdE/EbcV+i5A/aGJD1 PDUkfyBFUWsMDBzDsI92I9vfOOAVC9IYOeX8BHXCAgrSTBwqOfb7S+CRkbUlfjSatN4m ikZ8cpl1YLOSXVE3L+v/BdDkS+M7LEuz+VVN9C3cVTtUZh+PprqimqzjDSki53z4ytyd FZ63u4VktMlzT02nlk9k/OY9gzIs9NcrBgBjp+YaTZbWmNdx5pq0nRyAUaqFCNO8CJDx jkSw== X-Gm-Message-State: AOJu0YxolZPdIobTebaI4RjKCjTIQme/oagT4ueheJedKAkIdp9IDXoL GHQWFuys2/szaanpa6nPQZK/s7D7996ezJKwXnTek15CLwSxtEJ4lrVdLrk2b622cnn+twsorse FbprRdhRt+HadeVolBD7fFjcrFeXhOUr+/VWIaGhUVAFFczPbVRhkEOvpOArCRGyApCER6Ap/oF yMz2W5KQuU7hMfjS3paPQs/mW1gKOWevxRjY6MvG4jvgw= X-Gm-Gg: AeBDievyVgq+T2cK7Pdzae8vQ48nv6nVrohvZy212LhzH3aSRvN8q4Xliw3YJb2GCgk I1+fSeP4bPDT49+hVN8Cz7AFqs7ZmDBpl0ZmKYM61Wl8Kg6lB4nihMcuFtbV3AIafxqDW5jcWz0 XlCEZkabXE3oWBLUn6ruCA3TTRRFZWm734U0FUr2v879OgPE7UI9uIPAjlizynac7zZz5Vi1NO9 5bVHnjc6j/3bzLsi5j8llqJQclSexpsspCK4PT0XuXh5xdFZeLGTDNUrlMwx6ko3MXHhOPUIANb 7YExF6cTmrpXm+6JQaxt3+cFDWTbE6G2aKs14FLUN1XcTmvR3li/PcZXWgkGERjHlHQyTSiTUX+ q0PCrM8jbOVVaBzPODI7zU6h5m5vSNwCVVIbRe/E5SSA062cIv6Jxo++i X-Received: by 2002:a05:600c:37c6:b0:48a:65a5:750f with SMTP id 5b1f17b1804b1-48a65a57563mr161705205e9.21.1777240110906; Sun, 26 Apr 2026 14:48:30 -0700 (PDT) X-Received: by 2002:a05:600c:37c6:b0:48a:65a5:750f with SMTP id 5b1f17b1804b1-48a65a57563mr161704745e9.21.1777240110388; Sun, 26 Apr 2026 14:48:30 -0700 (PDT) Received: from redhat.com (IGLD-80-230-47-179.inter.net.il. [80.230.47.179]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-489207ebd3asm254796095e9.6.2026.04.26.14.48.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 26 Apr 2026 14:48:29 -0700 (PDT) Date: Sun, 26 Apr 2026 17:48:27 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= Subject: [PATCH RFC v4 18/22] virtio_balloon: a hack to enable host-zeroed page optimization Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a module parameter host_zeroes_pages to opt in to the zeroed page optimization. A proper virtio feature flag is needed before this can be merged. insmod virtio_balloon.ko host_zeroes_pages=3D1 Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- drivers/virtio/virtio_balloon.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index d1fbc8fe8470..8c15530b51b3 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -19,6 +19,11 @@ #include #include =20 +static bool host_zeroes_pages; +module_param(host_zeroes_pages, bool, 0444); +MODULE_PARM_DESC(host_zeroes_pages, + "Host zeroes reported pages, skip guest re-zeroing"); + /* * Balloon device works in 4K page units. So each page is pointed to by * multiple balloon pages. All memory counters in this driver are in ball= oon @@ -204,6 +209,8 @@ static int virtballoon_free_page_report(struct page_rep= orting_dev_info *pr_dev_i struct virtqueue *vq =3D vb->reporting_vq; unsigned int unused, err; =20 + bitmap_zero(pr_dev_info->zeroed_bitmap, nents); + /* We should always be able to add these buffers to an empty queue. */ err =3D virtqueue_add_inbuf(vq, sg, nents, vb, GFP_NOWAIT); =20 @@ -220,6 +227,9 @@ static int virtballoon_free_page_report(struct page_rep= orting_dev_info *pr_dev_i /* When host has read buffer, this completes via balloon_ack */ wait_event(vb->acked, virtqueue_get_buf(vq, &unused)); =20 + if (host_zeroes_pages) + bitmap_fill(pr_dev_info->zeroed_bitmap, nents); + return 0; } =20 @@ -1039,6 +1049,8 @@ static int virtballoon_probe(struct virtio_device *vd= ev) vb->pr_dev_info.order =3D 5; #endif =20 + /* TODO: needs a virtio feature flag */ + vb->pr_dev_info.host_zeroes_pages =3D host_zeroes_pages; err =3D page_reporting_register(&vb->pr_dev_info); if (err) goto out_unregister_oom; --=20 MST From nobody Mon Jun 15 18:00:34 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 67C0630DD2F for ; Sun, 26 Apr 2026 21:48:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240118; cv=none; b=HMB6ew7qpvO9hIrm0OJGZ2qeL/GYBWvMtbtFuP11A86eT8ay9Kci6W0RTFsIJ1Yso01Wd5pcfyL+nCQ6Q2VKlsutBSlFOc4etvGzitmoIiTz1cYhmBKLYUH2AXGJw6NQPsrgrwtRstiza9iWawvd4e9krOMp0Mf7jlAJlEgFEOc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240118; c=relaxed/simple; bh=d/DtwOXUxWzwJlhiqF5ZSiNoDuNbGpQpz89YwsrRjLQ=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=AnYCucyOxd64BT0/zhPJ8sCattuw7eiGZaKemsb7r4X4+NOmnO2UxOdAMKhbr28cE6f7qCnVl++5U+qNo3mBuR8qsGFHTC/tBKMJegDIZHHbYArJixzlrzlVmPWTin3mbt+nG//irX7G3BScYuEuzPwQu15h6hndHjJFY4QOlWk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Y8gpPyHY; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=n1qAZ0jF; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Y8gpPyHY"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="n1qAZ0jF" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777240116; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=BqSjL1olH8+AffRmYI03HiN9kMkx6LQqrsSuHYVoaLI=; b=Y8gpPyHYKWJryL5g33r8xHukaPrLKrH6+mDewz0w/G681VRi/UUXWLJbk129Cz1OYHRyz8 MV27l1HwXXIS9kTJvmrbrEgOitTN3ywa6Ralbq/Q7IBzw+FLcGh/kq+jMtA5V3fQ99rwav 4r6JA6yHq5hHyl+87ZSWUjXu3B+AqSo= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-656-CXRsti2kPm68Bn-vWGWU2A-1; Sun, 26 Apr 2026 17:48:35 -0400 X-MC-Unique: CXRsti2kPm68Bn-vWGWU2A-1 X-Mimecast-MFC-AGG-ID: CXRsti2kPm68Bn-vWGWU2A_1777240114 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-48a548b585cso40731895e9.2 for ; Sun, 26 Apr 2026 14:48:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1777240114; x=1777844914; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=BqSjL1olH8+AffRmYI03HiN9kMkx6LQqrsSuHYVoaLI=; b=n1qAZ0jFy/KK2sWQ44OFyemLdvHfipj9fugVSIZjqXmfgiwOuZDQwfT/o4vx+mMatk e2phiUshDQXCEtecl1n5N3hZyVnPXpUhzpb/qPrCk6QiOgsvzb6aubPjghIuBv0F2TcL zUmT5XCpU9LWu44GnZPruzJ6wAuWmEU9sc010hMfJeyIliULmnLCO6NvbvAZYT2JJ7mL 3u2zM/uIf5YKkqw3Tm2DUqM77HZF63f8ffjaEKIuqFTB/DlxsQESNhA09TXXMw8tiH6q uAFSYB+pCPm8Ej16KNAr+9mdgOLo1hLO3r7JbOL2QpKY0xBN7ixe6g5Kg62VTHkwDxsc 2xTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777240114; x=1777844914; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BqSjL1olH8+AffRmYI03HiN9kMkx6LQqrsSuHYVoaLI=; b=I5Nfa6NUiCGAvc9IMSupgYQWIyGxkZSOR277eMxIdrGz0qbrsnJS1d2LVlCBsGGGqv tJhRU5aaMD+5k+MJBgN0V7VGDI6S3FjcCXsm+K5E0R7QIx4WDg1SFllviel5L/I99OUm ZcrC356/nzHPn5ceBXs76oF2xTFL06i++A/bIUEEzK5XrieW5Jl9UtCOAcUblusmYlVq WFePTCcskbvk8fM11GVmSbBpu3+zVIJbK3Bext1TN8bUtuFANuchGcMufAoLxNK9sXpg 6IAySv6T6Uesf3ymNVEu8+k4OZxX77dkkxYqubwvoXcRtLThh+SQkzRMPBeVJhFCMD1L BdLQ== X-Gm-Message-State: AOJu0YxjXsoMOH55sFQJ1fvhzLqHPtQVsfC6Ubs1AbLoc4wCipnTHsbD CHhHmRlKsCV1maDdNM7gVKEHUfaBPXhac/mF+zy2uBuEfXUGbEctNWrH3h4Cw7F/D0phImAWgrt dkr9Nkna5wxEYLb8zsz6rD8GgTFp48uFMCB0NhMm3a2s1KhXlMAhcWDhSUs15meoq2IRa52MaEs KhrPE4RH6Jx7AETnVM2WA+XsSiiiRvReYR4E32AyizFzw= X-Gm-Gg: AeBDieuDRfIAGGQT1sW0jCld0HeZOqn1KjO/7Bodk04CAJscHPxcbj/2V1IDZzNwV3q Etj1/ENA70PfzvU898H3gKolAHQywPHJ/+Wp9/Ra2Gl9GG/YASiiUJbzeFOGjYqMdKNHS4Kgub6 PTyKNBpPpEt4w5+mE4J4kgEl1My+j17SRSXTPBJSw0bXVaoxYXOBhW4lCQWVfjtCjzvr+gXkB4W PEaEutXi2HvvtBc+UuzhNBoxzV9koUvC8846tidJVnRRhyQyT72aP2OaHcGdR4+qo+H7uOPsVWj WoH72QeMTCgwB1Y5aGtuOKkOeKT5IA6q5Og3OD6zUjP0gP9ixBUeaC5EDrqN9wN1ny7b9c2M4/S O89G9VwVvHtB/BmG31qyBh17HykxTmMHEpvGkX0ca1L9ummZeKsE2Bv8h X-Received: by 2002:a05:600c:620d:b0:489:1d23:4524 with SMTP id 5b1f17b1804b1-4891d23468bmr440690795e9.5.1777240113654; Sun, 26 Apr 2026 14:48:33 -0700 (PDT) X-Received: by 2002:a05:600c:620d:b0:489:1d23:4524 with SMTP id 5b1f17b1804b1-4891d23468bmr440690285e9.5.1777240113138; Sun, 26 Apr 2026 14:48:33 -0700 (PDT) Received: from redhat.com (IGLD-80-230-47-179.inter.net.il. [80.230.47.179]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-488fc0f8188sm806361295e9.2.2026.04.26.14.48.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 26 Apr 2026 14:48:32 -0700 (PDT) Date: Sun, 26 Apr 2026 17:48:30 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Johannes Weiner , Zi Yan Subject: [PATCH RFC v4 19/22] mm: page_reporting: add flush parameter with page budget Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a write-only module parameter 'flush' that triggers immediate page reporting. The value specifies approximately how many pages (at page_reporting_order) to report. The flush loops through reporting cycles, each processing up to PAGE_REPORTING_CAPACITY pages, until the budget is exhausted, all pages are reported, or a signal is pending. This is helpful when there is a lot of memory freed quickly, and a single cycle may not process all free pages due to internal budget limits. echo 512 > /sys/module/page_reporting/parameters/flush Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/page_reporting.c | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/mm/page_reporting.c b/mm/page_reporting.c index 61f7f08d02d4..05bdcf89f3e3 100644 --- a/mm/page_reporting.c +++ b/mm/page_reporting.c @@ -358,6 +358,48 @@ static void page_reporting_process(struct work_struct = *work) static DEFINE_MUTEX(page_reporting_mutex); DEFINE_STATIC_KEY_FALSE(page_reporting_enabled); =20 +static int page_reporting_flush_set(const char *val, + const struct kernel_param *kp) +{ + struct page_reporting_dev_info *prdev; + unsigned int budget; + int err; + + err =3D kstrtouint(val, 0, &budget); + if (err) + return err; + if (!budget) + return 0; + + mutex_lock(&page_reporting_mutex); + prdev =3D rcu_dereference_protected(pr_dev_info, + lockdep_is_held(&page_reporting_mutex)); + if (prdev) { + unsigned int reported; + + for (reported =3D 0; reported < budget; + reported +=3D PAGE_REPORTING_CAPACITY) { + flush_delayed_work(&prdev->work); + __page_reporting_request(prdev); + flush_delayed_work(&prdev->work); + if (atomic_read(&prdev->state) =3D=3D PAGE_REPORTING_IDLE) + break; + if (signal_pending(current)) + break; + } + } + mutex_unlock(&page_reporting_mutex); + return 0; +} + +static const struct kernel_param_ops flush_ops =3D { + .set =3D page_reporting_flush_set, + .get =3D param_get_uint, +}; +static unsigned int page_reporting_flush; +module_param_cb(flush, &flush_ops, &page_reporting_flush, 0200); +MODULE_PARM_DESC(flush, "Report up to N pages at page_reporting_order"); + int page_reporting_register(struct page_reporting_dev_info *prdev) { int err =3D 0; --=20 MST From nobody Mon Jun 15 18:00:34 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 11CAE307494 for ; Sun, 26 Apr 2026 21:48:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240121; cv=none; b=aRzfxGd+9p66a2/RxxYTF8OLRYlziMxkNLNX9hxD/qaUEbfRUN3Pzgg32yplwQ3hDjvsTbYUG/xXdz4R5VyjQiuiyKe0HdYGiz735eRnePoiWiQkdpVwsHxrwFhh9RM4pl9mDmxSW+j0B4HRCOffx1Q/G9cL2/MqnMyfJcSydk0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240121; c=relaxed/simple; bh=QLA2RGwCxgyfX1bS6CbrL8hgVv2A/s1slaGKecTu3qE=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=KKdKTiqTRt0ij75TKNP+/6XW/re5y4g4bC5OOTwTu7gic4WybebhITkZh88WnAztzbVbWQ+hk8qsr8Pbm0IGSU0+ZtZ5Qiv9C5oJTEVUCI+jaO+ef0e0gLnDx4MReOFrxxgekcTvSxF/O69wFiRuxqDBWmcj3G8IeFZcyqHMXkM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=LLKhO/74; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=QEuk6L29; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="LLKhO/74"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="QEuk6L29" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777240119; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ym5bn+CWMcoek2qBVqBOry4vxURkIsMQd8yhmA+BaCk=; b=LLKhO/74o7jPDiLRI4pUlkAp8ImF9MEeho5nrcDaSsegDnkHWrxbci++NN/0AEJ3KNJzuY 7OWnGgvePQgJlw8BC/LOhJztUUt16ZQkD1TRAVru9Lw8Bn/FHb45vqxnFQqcFPsAqemnEG arrV3B6ZKX7wHIB+JTDrSsQFlSbO+pI= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-668-527_0QK0OcCTqYxcLJMOaQ-1; Sun, 26 Apr 2026 17:48:38 -0400 X-MC-Unique: 527_0QK0OcCTqYxcLJMOaQ-1 X-Mimecast-MFC-AGG-ID: 527_0QK0OcCTqYxcLJMOaQ_1777240117 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-488ba2f4094so57573505e9.1 for ; Sun, 26 Apr 2026 14:48:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1777240116; x=1777844916; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=ym5bn+CWMcoek2qBVqBOry4vxURkIsMQd8yhmA+BaCk=; b=QEuk6L291av7zMgazUMYK0n1VT17Gc2frbG9gfqamlvKhhAYIIgq3bYRml7JB/OQNc fqI3fS8NGy0R17XPRMx/FRgG4WBH6cnhLMeoodf420IcpcsHx/HmiBSZMGbTHo+RlfFk rfkuzQRLIPNytVUWVEEUa0yhgmpjbb+ztLmHoQIT0a3Ep2qOxInpRVbCtmOPNX1BV2l6 NMZBSA1OfwtLIY0EnC3rzrAcztivKgR6y+awH12HcOoWrIzRoifVWvrFa6sum4c9t8cS vQ+O5FHfnlmhu6GkaQ3dod+zPYYk/ASWYzzuwr+SKMmAuSvEjWpwK07YfKQ2Uy1T1mDn xlCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777240116; x=1777844916; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ym5bn+CWMcoek2qBVqBOry4vxURkIsMQd8yhmA+BaCk=; b=BBpuzEeP52DuDSA1z1xkh5pEE1OIAfuHfOZDo3ysyaeog0yah+iyBRT4L08coDQxt8 Eeb+0aW4c2slqZrqFvzfogzEijhw3bJVHmKLwjBltk3VQKZhz3LrhyCdtAckwbY445eM EPC4FWc68osw5U+tNq/gLpmxuhpd2ZEztB+XDHeKdK9v3zagsMEXpDfdAydhBa1WJERb F1tBsyz4wwYi141pMbVfINd06RMw+Wc4j1LrtlyVnjwFqaOxCgEoh9xUxONgDw9eltKO RFme+bQcrJNEgvoMTuNneGRaOWNDgogUS/x8VnCTB2td3OHt4JPZKPkuquTMcwLUnBHa 6qSQ== X-Gm-Message-State: AOJu0YwIrRiLtS/bQ5cxUhq7NC4Qe0cFUWDq2+bToySxEwybTLZtE+bb V5d2HWdBnbY0zH0FREZ6w9+SARxmNGPUMAu/ovA+8clI236r+6cJeTl+/0/5/bk4DOQlo3Oy8Vw p0wJVyw+0hWHwx3UU7sqTmuPUSl0EXKOQ8ECNWNoY1ORR72tUP6hK8lLO/U5EndS8ozkGSULkDw tvVo61Rl10gnUtRXsYPwcfxmIG/EOut9Z5faTnLniIie4= X-Gm-Gg: AeBDies4j2+uRQRS60D/gxye3w2KruQr29wHSVhjCcBH3kB7nj90jK0CiOZmF513Okg UPqIV6mw0W1xQlj5Q8qxoeilMItJEh3Ka8/NU6BKWFHdmCNb6s/RSjq2LXMeLCRsi0AlNXIxOku YVVbvwrd2H2H5Bs9VNmy2q3YcSAm8FVw/1omdarIJCToxNXXZSf8gxS0OrAM9FuzpKo0psi0Rbx CUnwCS5IbHoqzQjQ3CI7oJN/48NIN7LyHTrUKj3THW9ORPa00+VUygI3CKiBWWxXVAw0TVNrVmt nqkbmRWSCAur4HO8j1WB3f+xJQg31Qni3e5uUR4ruoj4i4CW0TMh9PjU6AeYxXqTqdxVRmk7kwN xk9r4YOC0ZdLjIP9+EzaItSCHkBPs+NTQ6b1x0zkC5tV38brsSP1IZUuI X-Received: by 2002:a05:600c:c085:b0:488:c530:48a0 with SMTP id 5b1f17b1804b1-488fb784843mr454260285e9.24.1777240116501; Sun, 26 Apr 2026 14:48:36 -0700 (PDT) X-Received: by 2002:a05:600c:c085:b0:488:c530:48a0 with SMTP id 5b1f17b1804b1-488fb784843mr454259925e9.24.1777240116042; Sun, 26 Apr 2026 14:48:36 -0700 (PDT) Received: from redhat.com (IGLD-80-230-47-179.inter.net.il. [80.230.47.179]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-489201cde98sm505689585e9.7.2026.04.26.14.48.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 26 Apr 2026 14:48:35 -0700 (PDT) Date: Sun, 26 Apr 2026 17:48:33 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Lorenzo Stoakes , "Liam R. Howlett" , Mike Rapoport , Johannes Weiner , Zi Yan Subject: [PATCH RFC v4 20/22] mm: add free_frozen_pages_zeroed Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add free_frozen_pages_zeroed(page, order) to free a frozen page while marking it as zeroed, so the next allocation can skip redundant zeroing. An FPI_ZEROED internal flag carries the hint through the free path. PageZeroed is set after __free_pages_prepare() clears all flags, so the hint survives on the free list. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/gfp.h | 1 + mm/internal.h | 1 - mm/page_alloc.c | 21 ++++++++++++++++++++- 3 files changed, 21 insertions(+), 2 deletions(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index e275cc80e19e..766b1c7f0731 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -394,6 +394,7 @@ __meminit void *alloc_pages_exact_nid_noprof(int nid, s= ize_t size, gfp_t gfp_mas extern void __free_pages(struct page *page, unsigned int order); extern void free_pages_nolock(struct page *page, unsigned int order); extern void free_pages(unsigned long addr, unsigned int order); +void free_frozen_pages_zeroed(struct page *page, unsigned int order); =20 #define __free_page(page) __free_pages((page), 0) #define free_page(addr) free_pages((addr), 0) diff --git a/mm/internal.h b/mm/internal.h index 0600d824ba03..60b983872d51 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -899,7 +899,6 @@ struct page *__alloc_frozen_pages_noprof(gfp_t, unsigne= d int order, int nid, #define __alloc_frozen_pages(...) \ alloc_hooks(__alloc_frozen_pages_noprof(__VA_ARGS__)) void free_frozen_pages(struct page *page, unsigned int order); -void free_frozen_pages_zeroed(struct page *page, unsigned int order); void free_unref_folios(struct folio_batch *fbatch); =20 #ifdef CONFIG_NUMA diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 1169714406e7..981ddf3566b5 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -90,6 +90,13 @@ typedef int __bitwise fpi_t; /* Free the page without taking locks. Rely on trylock only. */ #define FPI_TRYLOCK ((__force fpi_t)BIT(2)) =20 +/* + * The page contents are known to be zero (e.g., the host zeroed them + * during balloon deflate). Set PageZeroed after free so the next + * allocation can skip redundant zeroing. + */ +#define FPI_ZEROED ((__force fpi_t)BIT(3)) + /* prevent >1 _updater_ of zone percpu pageset ->high and ->batch fields */ static DEFINE_MUTEX(pcp_batch_high_lock); #define MIN_PERCPU_PAGELIST_HIGH_FRACTION (8) @@ -1624,8 +1631,11 @@ static void __free_pages_ok(struct page *page, unsig= ned int order, unsigned long pfn =3D page_to_pfn(page); struct zone *zone =3D page_zone(page); =20 - if (__free_pages_prepare(page, order, fpi_flags)) + if (__free_pages_prepare(page, order, fpi_flags)) { + if (fpi_flags & FPI_ZEROED) + __SetPageZeroed(page); free_one_page(zone, page, pfn, order, fpi_flags); + } } =20 void __meminit __free_pages_core(struct page *page, unsigned int order, @@ -3032,6 +3042,9 @@ static void __free_frozen_pages(struct page *page, un= signed int order, if (!__free_pages_prepare(page, order, fpi_flags)) return; =20 + if (fpi_flags & FPI_ZEROED) + __SetPageZeroed(page); + /* * We only track unmovable, reclaimable and movable on pcp lists. * Place ISOLATE pages on the isolated list because they are being @@ -3070,6 +3083,12 @@ void free_frozen_pages(struct page *page, unsigned i= nt order) __free_frozen_pages(page, order, FPI_NONE); } =20 +void free_frozen_pages_zeroed(struct page *page, unsigned int order) +{ + __free_frozen_pages(page, order, FPI_ZEROED); +} +EXPORT_SYMBOL(free_frozen_pages_zeroed); + void free_frozen_pages_nolock(struct page *page, unsigned int order) { __free_frozen_pages(page, order, FPI_TRYLOCK); --=20 MST From nobody Mon Jun 15 18:00:34 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3512530AAD8 for ; Sun, 26 Apr 2026 21:48:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240125; cv=none; b=dU90BnIP2ON638MGfbk1vGkf/HZXtbKrIN76MYd/4d6SyvMIYhLn/DkrLAWxzr4/r8JXISi85vPbgGZ77jidQxnla+NUrut2Nd/oWCkKF3BCz5HBSVUQdeeEb1xXd6GRBxXF0lP6SPuiDc5qhUG2vUhBDNLfPpkZup7F9az0tqM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240125; c=relaxed/simple; bh=UOL/ZsOQuxZxCdyyXjZBk1f9WnmSmNTDh0iqg53ebK0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=OwIbjgyfyjx0lJZ8UmtKUkTvT13QHW3KHyat9qBMEbvFge0L/OAfpZ/dNf+j+IxQoOWGxWTtPifKPNqc0itwMkpNME1sPJYpENbxSY0yKE9+jp/GcyP/71UXKFInZNK1tu/IiGKsNfwdMC/ms0X8NoSTC8Aamz8UFBvbh+DMRZA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=J+sbDDem; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=j5STN2aI; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="J+sbDDem"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="j5STN2aI" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777240123; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=yr2uOCi7TxHPhFnCx38/iKRYLu7TO1I9ea6WeFSAtGo=; b=J+sbDDemBD3MheA7xfu2vBzvL87YBNPnvZXSV+xw9M+/TF4UqjFTJ0flkUW+VYG7/8ckCx Lgga8DiptalSWJcOIIdXEKbsPmyV9Dg3NxAI+RDcvfqEpaY5b2sqBXjLVHk4KmDCWE41dn x0qRRMJnl0EGVPjYaCwFLCs5ylBeG5Y= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-408-HNVpJhstMQWMUARHw8q_nA-1; Sun, 26 Apr 2026 17:48:42 -0400 X-MC-Unique: HNVpJhstMQWMUARHw8q_nA-1 X-Mimecast-MFC-AGG-ID: HNVpJhstMQWMUARHw8q_nA_1777240121 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-488d2cd2674so74605995e9.0 for ; Sun, 26 Apr 2026 14:48:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1777240120; x=1777844920; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=yr2uOCi7TxHPhFnCx38/iKRYLu7TO1I9ea6WeFSAtGo=; b=j5STN2aI/nd7ryzC/ocbfZDzSxv+WIxa+yYaeQa+yf/Ax2+mgBwfq5/QNP1eqz9/C8 ZWM9zD+aLh9YojwybLzKI6AUsWUrvEs1WO3hPAvpVfn9JLTUWu56bbTBa8jRcsVGGBZs 01UAmDo7VXSAB6EIBgF9aAfk5ChnW5dbpKX2ZHf1d4RP7laXxYbg70PTP/KI97oWpH6X m8SnecP13HKxOm5HZi4JaIqkTYteXV/d/CaUVqpUrqwO6Npij38q7k4VD6ZJ98Qzumyq YIb/WGClWfcqwqZZ84MTwaiHURhxC1RuqUEsiTnWpjsDfQvTy1baqxNpM9fy6uA2hDyR ncRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777240120; x=1777844920; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yr2uOCi7TxHPhFnCx38/iKRYLu7TO1I9ea6WeFSAtGo=; b=JTXgI/8zFI1DeSa1vf18kH167uq+KUkSmqp/alb/Zj272/pFcJKVy3KI69kvxwqQjh dJ171v5wJJalUZexspay0B5DNJnaQ4lwDhwAeAlX+V5M/0IU3ydj8tQE2bbHyy0nPpk4 L3fty9sS0yZP2QOHVFj5kEZDu2dlWAdix4PrCjDqZ/pJaeA0BgDegcsNG/h+32Pcj8DL 0ZYTDsWRxU3ClAuN0vQT4M0Pxd3f4y3Zl/Zx4UxjPVFZ5SJpx4fBmsJcskU6Iq9CSgds 0o/X+A2rCihfD3KjVJy21V7NSAb5CG/Apj2jcPweCMPmLcZO5VyFiONkLsxC65Frq9vZ nzTw== X-Gm-Message-State: AOJu0Yz/zWr4mncR4vMJxlvll1WGMch3dnHc970dY1AbP2jC2J7tCamc WIGRxvksDhnHDVUek0fdBqCza56MKNyl3DQh+IjQwHg9dJ83+8sQ5p+7nC4GiseduzIrdGvdlGV 1kF9us7A1+AuIvKJ3uq2Yd24wz8AevqKa3N0dmFeT2sDv01B/H8dfy20A0cOlPtNARFybPcUArv W0WdF6bXM1Yoh3aPqP23qCEmnBfLg/F1EOHiP9QJ30VCI= X-Gm-Gg: AeBDietb9iUdrs6Q+LID7WqwM/sAbRR9CwSuJ1Mex8hsWyg74oLKe7sye8nVPEK4gAD YhM0HchcK0yDBpP9XWUZF8X94GBWdvaFK6fWD0mzkw9mw/kDNiSaD+XjBLcoD4ZXwcXCNG9CsM2 ChH7b1EJjahHHKWpoPJOYm52ansqYs9Dpx7PT/JbVXso0uhgWZ5YbUoNdBYlPzvWIJR+SWNxwP/ xogw28NEAnq7lBPr9muJ3bDKF4BZTymLw/iwF5U2t6HXbz29jUVhbjyD9wYOqOCWgPUp3VmV4YK 9QRy/YkIrf9mZGXjnfwC6KNoCGTSVPKm6TGZ/ETDn3kZ6gK2md0Siho8wpBXDxDY+FN3MuAle5k 3n8LVHCqIkFPgoyb+X4eWs4kpaTAhuwEi/4wAXDc9yrsn8l2oPWXoXRll X-Received: by 2002:a05:600c:4746:b0:488:9439:881a with SMTP id 5b1f17b1804b1-488fb738412mr555456555e9.2.1777240120470; Sun, 26 Apr 2026 14:48:40 -0700 (PDT) X-Received: by 2002:a05:600c:4746:b0:488:9439:881a with SMTP id 5b1f17b1804b1-488fb738412mr555455875e9.2.1777240119660; Sun, 26 Apr 2026 14:48:39 -0700 (PDT) Received: from redhat.com (IGLD-80-230-47-179.inter.net.il. [80.230.47.179]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-488fb7bf7besm270287995e9.34.2026.04.26.14.48.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 26 Apr 2026 14:48:39 -0700 (PDT) Date: Sun, 26 Apr 2026 17:48:36 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Lorenzo Stoakes , "Liam R. Howlett" , Mike Rapoport , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu Subject: [PATCH RFC v4 21/22] mm: add put_page_zeroed and folio_put_zeroed Message-ID: <111219af35c4bb357e30cf1a77265fb63eecb21c.1777223007.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add put_page_zeroed() / folio_put_zeroed() for callers that hold a reference to a page known to be zeroed. If this drops the last reference, the page goes through __folio_put_zeroed() which calls free_frozen_pages_zeroed() so the zeroed hint is preserved. If someone else still holds a reference, the hint is simply lost -- this is best-effort. This is useful for balloon drivers during deflation: the host has already zeroed the pages, and the balloon is typically the sole owner. But if the page happens to be shared, silently dropping the hint is safe and avoids the need for callers to check the refcount. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/mm.h | 12 ++++++++++++ mm/swap.c | 18 ++++++++++++++++-- 2 files changed, 28 insertions(+), 2 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 541d36e5e420..1c6bf82a967a 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1640,6 +1640,7 @@ static inline struct folio *virt_to_folio(const void = *x) } =20 void __folio_put(struct folio *folio); +void __folio_put_zeroed(struct folio *folio); =20 void split_page(struct page *page, unsigned int order); void folio_copy(struct folio *dst, struct folio *src); @@ -1817,6 +1818,17 @@ static inline void folio_put(struct folio *folio) __folio_put(folio); } =20 +static inline void folio_put_zeroed(struct folio *folio) +{ + if (folio_put_testzero(folio)) + __folio_put_zeroed(folio); +} + +static inline void put_page_zeroed(struct page *page) +{ + folio_put_zeroed(page_folio(page)); +} + /** * folio_put_refs - Reduce the reference count on a folio. * @folio: The folio. diff --git a/mm/swap.c b/mm/swap.c index bb19ccbece46..5d05a463b46a 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -94,7 +94,7 @@ static void page_cache_release(struct folio *folio) unlock_page_lruvec_irqrestore(lruvec, flags); } =20 -void __folio_put(struct folio *folio) +static void ___folio_put(struct folio *folio, bool zeroed) { if (unlikely(folio_is_zone_device(folio))) { free_zone_device_folio(folio); @@ -109,10 +109,24 @@ void __folio_put(struct folio *folio) page_cache_release(folio); folio_unqueue_deferred_split(folio); mem_cgroup_uncharge(folio); - free_frozen_pages(&folio->page, folio_order(folio)); + if (zeroed) + free_frozen_pages_zeroed(&folio->page, folio_order(folio)); + else + free_frozen_pages(&folio->page, folio_order(folio)); +} + +void __folio_put(struct folio *folio) +{ + ___folio_put(folio, false); } EXPORT_SYMBOL(__folio_put); =20 +void __folio_put_zeroed(struct folio *folio) +{ + ___folio_put(folio, true); +} +EXPORT_SYMBOL(__folio_put_zeroed); + typedef void (*move_fn_t)(struct lruvec *lruvec, struct folio *folio); =20 static void lru_add(struct lruvec *lruvec, struct folio *folio) --=20 MST From nobody Mon Jun 15 18:00:34 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 564E630EF8B for ; Sun, 26 Apr 2026 21:48:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240127; cv=none; b=sNuHwmkyOE04TmNjrufKCtn8S1NwmRa364nq42I0wzxkZgereIR9C7RyBj6qpJ1nkqSrCrzOFhZoPmU9s/7w7QauhoA1bRDVWA+pdGQPNRnrNt0otpD4nqZGm24zKPz0Ztr/9HYxEwnO00H7TZgfTcnz/GZMBWiQ5fVvK69YJO8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1777240127; c=relaxed/simple; bh=AwIv3WhJazU+sFjaROXcIit57FR98tq+do8n4h80ZYs=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=MVPGwsclLx0nAv3VGpDrzE5wVyH0TtJdKrlzg0k/MR15FGUAgn6CjOS50VxYCVlL+nD3SMv6hxGzwwdMsGxM3lHhVxTh96FxuDRQf05cYlLhB7qbZc9cFSOlO1VgmV2LlaoY4+Qx21QBCVoo3h/aZwfpxjkgytT34ewHb/ob5I4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=QiyDtpd/; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=WAEYIuYD; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="QiyDtpd/"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="WAEYIuYD" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1777240125; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Sf8WIccv4sYUoC01taSJ6nrrobzratXFM4dveYAPw/U=; b=QiyDtpd/lzCDcYBK0IJ3GQAGYZSTmtotUwgW1lSaxQ6atJrCqjQ2vJDfF6GKEwlEqpdkc0 gqe7NYXxReEqBe7ql3FkphfhBkakSGn6Qi11PVA6LW/Ko3+Vco9+M3FT1nK/2wR6MlCRR4 9KsFcG3XC2q1SQIt5kQGxeirhFb5vX0= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-413-GPYXEBYhNQK4A_qwTMohTg-1; Sun, 26 Apr 2026 17:48:44 -0400 X-MC-Unique: GPYXEBYhNQK4A_qwTMohTg-1 X-Mimecast-MFC-AGG-ID: GPYXEBYhNQK4A_qwTMohTg_1777240123 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-43d1fec59c9so6059748f8f.0 for ; Sun, 26 Apr 2026 14:48:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1777240123; x=1777844923; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Sf8WIccv4sYUoC01taSJ6nrrobzratXFM4dveYAPw/U=; b=WAEYIuYDsV4c2T/mrbQc+C0Y5/2z9awRD1TK0aMMgITxnksgn6XVKJ/uLfKfO42bIm O7NkXs7gUqbg+gnAN437ic0vKwuQzKFGHBd8iGLvykkOyf66bDNTen1XgWr0mfwMYnm5 tuWIT1SRWqlngAqKMrtXRJ1siqQWx1wuk2R+uEHpJUy6YOgljidQZSHDqUa3ksDUqWjJ /yRASf7odweTgpgXMiFmszYSXCZ9T9Vg3xFTYQExVUuW2A1wgG3DRotvzU0ak7/hSHcd aUH6DWILJOt9pu+YKvR8EMlEgE5wbUzzvZnEBsgVrD4Cg0mZNlzpbmI05f8hOsnX+nPr yvBA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1777240123; x=1777844923; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Sf8WIccv4sYUoC01taSJ6nrrobzratXFM4dveYAPw/U=; b=dbIK+AXX/O6dQr/uuzYyYqCR22WBOFUau1ze0NDNF7IaUiWufw715EKj4AqQjWB1tp kzfqvId4ORNC5kGdDvCtg75JWUKfYmrlYdGpG9i2lq2jCezMKNAdeb1Ov3blGECWFcHl AkxjH99OXtaDYdoEIwwDHM26dn4+DKr1jdCuT7Xx4qRbAoBKt7+o234M8/Hm/N53DfDD mTdac3bQC/RoBI6qaa+H20nC5UIzkXFlsF2tahFCJNP0lMiiA3qBbXTYseT1aue3+C+T C7MPrNGWM6trrsywrZunPVnL7QAmHmp06x+Ef6ggIh4AKb5Sbi4oeyVryxaBNMwvNOSE J7dA== X-Gm-Message-State: AOJu0YxHq57Nn4HXui27iehR6xPzk7/hFKB5f30dWfNA/IJ09QYHw/VH ZzG5CGN7wkspKyD9Ug3O0MDC5bPQMrqDtBXI1O1HofbVSfTo/NJdiWxwOw2gOEh2Len0vT7UIKG mBRzgHqVpQdEZT66P1i3tSer3JJEy8RDzD7bNjCgjLU4RraJA+qAyK73wh5YjQBJaFAxvFhowam OyJBSCCY8npIUaAhIEESzK5TwRiIQ+Mu+JdRm9zBYPPkc= X-Gm-Gg: AeBDies+Vjdp8THczIIgQ89mUaTiBa4rGsHVR4a43ZkxjpcfHkB8Y9Rhxqoa2cU3uOx mEQI/6KpUc0SSZ4ch3CR4Ef6dF6eyGxNIH9zjCwHGhAdu53z9FSsg5vi7ll09v4RSc84QYrB2kU cYEbzO78PFu9h6iPTou82BTOUcuy+0cPOECyLf3P2jnhBM4RfPPaWawgCM/iGlzqGk19lxljnc4 MkOteY+94QTQazQGn9241qrwWL1WIGG5YqOYm44yL+fKO18B1uFCu0Kwhq26rMPzloFi+uO8gR5 UM5FAbm3Gi9sNt7daEg/BwDNWdZO7WjyUupxk0Ab9t/2w/rR+/YR+POLVSuAAjds/ZpLem5jDmt aEN7gto+cK85QRKYkZ6BQta2nR5cBehFcJsHReoujlEZX5mcLbnlPOiF8 X-Received: by 2002:a5d:6f13:0:b0:43e:a703:3675 with SMTP id ffacd0b85a97d-43fe40329ecmr53892363f8f.5.1777240122815; Sun, 26 Apr 2026 14:48:42 -0700 (PDT) X-Received: by 2002:a5d:6f13:0:b0:43e:a703:3675 with SMTP id ffacd0b85a97d-43fe40329ecmr53892312f8f.5.1777240122338; Sun, 26 Apr 2026 14:48:42 -0700 (PDT) Received: from redhat.com (IGLD-80-230-47-179.inter.net.il. [80.230.47.179]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-43fe4e3a341sm81502808f8f.24.2026.04.26.14.48.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 26 Apr 2026 14:48:41 -0700 (PDT) Date: Sun, 26 Apr 2026 17:48:39 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Vlastimil Babka , Brendan Jackman , Michal Hocko , Suren Baghdasaryan , Jason Wang , Andrea Arcangeli , Gregory Price , linux-mm@kvack.org, virtualization@lists.linux.dev, Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= Subject: [PATCH RFC v4 22/22] virtio_balloon: mark deflated pages as zeroed Message-ID: <966969d970f95ee2a573e83ac26162e15f758261.1777223007.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When host_zeroes_pages is set, the host has zeroed the balloon pages on reclaim. Use put_page_zeroed() during deflation so the freed pages are marked as zeroed in the buddy allocator, allowing the next allocation to skip redundant zeroing. put_page_zeroed() is best-effort: if the balloon is the sole holder (the common case), the zeroed hint reaches the buddy allocator via free_frozen_pages_zeroed(). If someone else holds a reference, the hint is silently lost. Once balloon pages are converted to frozen pages (no refcount), this can switch to free_frozen_pages_zeroed() directly. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- drivers/virtio/virtio_balloon.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index 8c15530b51b3..cdc3c960397d 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -304,7 +304,10 @@ static void release_pages_balloon(struct virtio_balloo= n *vb, =20 list_for_each_entry_safe(page, next, pages, lru) { list_del(&page->lru); - put_page(page); /* balloon reference */ + if (host_zeroes_pages && !page_poisoning_enabled_static()) + put_page_zeroed(page); + else + put_page(page); /* balloon reference */ } } =20 --=20 MST