From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0A7DA351C3D for ; Thu, 7 May 2026 22:22:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192556; cv=none; b=WkkRN2t9O+yzIOWAfzTIeHOy/vfThENSYbBVIYVAbGiefxRNXz9oNgH+lm7KXgtAssHZLljGtRCnd2r0j11DTjT3k/4yqzntSG6+X4R+bIhiDEpQWnd4epxyjWjsZLJryFTk0hPyFuQJIMEmR5Ap8HvbOumpSzT3macBZZAgi+E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192556; c=relaxed/simple; bh=s/4V9KfrxV4ySmIGDyM5yp8UGJQjsodO+qkWhdfAvd0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=LUi5Uni74PKZdY0CH2stZpjH9uyN2IiJY9uo61cZHe6IGR3IfXsngbaoHQmthrmqSyqStcj0Z2hMQj60IznV6JSSWAuSNHL1w3iaQPhVzNXNQB28YSt3+DZycS7f42rjwaOZBqNymGmESx9AmObmV0AU4VQ9Z3sdu6Z7yvZJTSc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=MHOc6abl; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=At0lfbpv; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="MHOc6abl"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="At0lfbpv" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192554; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ka3oylQNziq94hqi5anbLpSAA5gWdZ5Hm2jMq8WmIng=; b=MHOc6ablwBdTRvojNn+jvFeGcG2uRDfjoM09ZPVbUdJw0KRMzvbAPoqerXMIVV5VAWJukD N/imxmQgo2Gy59B/4LItgcWDob7PmX0QrAkbUsCIv827f4xoo0BLtPEi/GeYjRA7ksHz9a mzG1oVp+LiSfx8f0GjqihVuh3MjqYTg= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-658-O8Ez6F14NwqSqMye1ldDSw-1; Thu, 07 May 2026 18:22:33 -0400 X-MC-Unique: O8Ez6F14NwqSqMye1ldDSw-1 X-Mimecast-MFC-AGG-ID: O8Ez6F14NwqSqMye1ldDSw_1778192552 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-48e6260968fso6999135e9.1 for ; Thu, 07 May 2026 15:22:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192552; x=1778797352; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=ka3oylQNziq94hqi5anbLpSAA5gWdZ5Hm2jMq8WmIng=; b=At0lfbpvQ7qtwzaBkfJ0mrvrfwqFuGYKr5+o2daAkN+HhOc+jIl9Lb5Ee+2qIFbDgj nH3tvimM6rTIE2Rns9MX9wK7CPVBvpqzo5epPs4JCtm0k/64wXF8drgHmPcefHnctbt2 SBspX618KD3Usb+Wmd2ZBoIsZmFa6xpayfLtH6NdfDhWs5FaZ9wZKaJzsl0BxMdMYCoK k0zFbqk9q1J5VeECXgv/iy350S1uCXzdEd3bmckvwxzFV2bwsH0sPYABq/qwj6+64Lw2 0SO9ElS99D5hMusGHFCUKe8RsIZxMhFjJXEEydOE4R/dmMJyn5nVqhSypvn1HNf3u7E/ gsFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192552; x=1778797352; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ka3oylQNziq94hqi5anbLpSAA5gWdZ5Hm2jMq8WmIng=; b=CDQzueTHUJAjEnFmxI2MiFg/ZJQL7m2TKGrf6aInldYtUsKcKV5dB14BIyFFz8FxWL p2DdlkGOaRtjEFZEVs9ySVrZTbknwSHHttCN2RPYPaXjKRRTs/gp4CVtCc3NHf9WLJ9f OLIwhgw5WA1L7tGjMYman2BybD22VN8zcxWahAhepbg9xRGf/x47PoyMCJKxqJ4ELAVi SGnc13no+p5cc9oVr9G40E6kjBU22rwyqgFzrurAeoRCb5lQI57XyqTG32jt65mIEZmv 1hbTRJJWN6Dg7eaFdrxh6KOZfcepbN1IklFLXQoI+bKJ6CLoKx4E6qvHSMMvGluL4NkV +sOw== X-Gm-Message-State: AOJu0YwMm89f8WWoADVeARAUIoNtq9MePGbPre7yGXVpzo+Y7RTtltx0 y3bw47g29ALYRFKMgD7BNdpYmr/gZrsAFdJ0pJoubhw5re9u5k1PniFSf6e9Gjvo7/QXbOkK2yB L8OjKyBbt26Cvry62vWHoVno9c6H6C+ncVl7yE0lF7e81a1UIF2CIYNsJnJxvw0/Tess2ooY5ES JbDt4IYhzgeYBP9ds9Tpa5OnkT39ZuZ3nlgmqIKnitxuc= X-Gm-Gg: AeBDieuXrPyhhVncLEUQaNdlKbEc+f1aJjTO2ts2aHkzhgMBXYyQpqyk78YbgTu6V+n M2rVbQpQzhXNBoHQjrST/FnghMru4fw5YKGxpU1ZJendF4Gi5rdjvt93zGXcCNQFhvqEdNTcvSQ K7nBoW4OLROrP/bPDzpP5ZaQlPrXwgrotpmxnvmHQK0Qg27KkLNxfVfQPJqfzqchh7vlrk6CK1i hqDD1t7UX7nUdsuNL09/TtVl0Ad2GFGKz66qeXcyKVklUPD01T+0hw3jpgULchYHmOSiIdpUiVj gcbx7PkKZ+Msfeh9dz7SDafXAfMVN2+xKC31h0y0OO2rCds3leVxUEt2afJQ/qyNqrLMSomUCYw x2yQsrqQik9xk49YWEZzuelnE+60SvxCcrrUSmHuZ X-Received: by 2002:a05:600c:4e43:b0:485:30d4:6b9e with SMTP id 5b1f17b1804b1-48e51f4c1e5mr172323165e9.21.1778192551439; Thu, 07 May 2026 15:22:31 -0700 (PDT) X-Received: by 2002:a05:600c:4e43:b0:485:30d4:6b9e with SMTP id 5b1f17b1804b1-48e51f4c1e5mr172322665e9.21.1778192550845; Thu, 07 May 2026 15:22:30 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48e65607127sm10239565e9.1.2026.05.07.15.22.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:22:30 -0700 (PDT) Date: Thu, 7 May 2026 18:22:28 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Zi Yan , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , linux-mm@kvack.org Subject: [PATCH v5 01/28] mm: mempolicy: fix interleave index for unaligned VMA start Message-ID: <401fa82d946cd96ed200be5be4f911371e3f6f3c.1778192416.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The NUMA interleave index formula (addr - vm_start) >> shift gives wrong results when vm_start is not aligned to the folio size: the subtraction before the shift allows low bits to affect the result via borrows. Use (addr >> shift) - (vm_start >> shift) instead, which independently aligns both values before computing the difference. No functional change for current callers: the fix only affects NUMA interleave and weighted-interleave policies. The only current large-order caller is drm_pagemap which does not use NUMA interleave. All other callers use order 0 where the old and new formulas are equivalent. However subsequent patches in this series add large-order callers that pass unaligned fault addresses, making this fix necessary. Signed-off-by: Michael S. Tsirkin --- mm/mempolicy.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/mempolicy.c b/mm/mempolicy.c index f0f85c89da82..583b64f2b4d3 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -2043,7 +2043,8 @@ struct mempolicy *get_vma_policy(struct vm_area_struc= t *vma, if (pol->mode =3D=3D MPOL_INTERLEAVE || pol->mode =3D=3D MPOL_WEIGHTED_INTERLEAVE) { *ilx +=3D vma->vm_pgoff >> order; - *ilx +=3D (addr - vma->vm_start) >> (PAGE_SHIFT + order); + *ilx +=3D (addr >> (PAGE_SHIFT + order)) - + (vma->vm_start >> (PAGE_SHIFT + order)); } return pol; } --=20 MST From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A9A7397E9A for ; Thu, 7 May 2026 22:22:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192561; cv=none; b=eZfw0YL8D5kSNqLyBrvXohpY1k5Q+XOMtp9Ygq1Cn+xAgzdtYfS38qnuSVXn7J/+XgZBvEk/0ngx4D4Ms1dOOvIa0t7SMsvoQdFc75kVpuKdie0SAqanP6i1GuzautD8RqpzRbXWuRm++IfCMNXeryFM7xJXnV77PVl3DOUcRsk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192561; c=relaxed/simple; bh=4jO1k2TrAkHUsVj6FyxohCUVeNSZ7GMM/TuuS2hUOUg=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=dFXBk4pMBKc5NrK3eedQUwza/KaZwTHXqdYKl15SEevpuD88UlDoG01/7P1ytrw2AAjHY+vjOy+p7Oo8nwPEtG4ZCx+PbcmlTXPU71oDimZF0lJj8y2G7qJ68pE/E7ICL828XiHU2PT1W8ze5Eeu634DYo1Z7KX7+eAq/Ce+Vto= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Pl5y4B3U; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=RSUBddrs; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Pl5y4B3U"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="RSUBddrs" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192558; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=eGyUdojsR0b/AAxiGg7aPnV5Y+huy5DwlenqjNbFa0M=; b=Pl5y4B3UYuvjQ5IKKO+u4g5c48C4R+t3Dau8G6rj1mn40Dr+W1qLQ+uq6d0QAUwsWoFzck rlXN66bo/uKmDqiH/RNXkM23fen+q5vNmrooh4f6JDUgbtYOkRQSXcB6HttS5ImOmLRyL4 u3gN6JM8tWlCPQQObxw7+NhEg0U0/SA= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-360-QSkBMZxpNEuxG-sRHCkaYQ-1; Thu, 07 May 2026 18:22:37 -0400 X-MC-Unique: QSkBMZxpNEuxG-sRHCkaYQ-1 X-Mimecast-MFC-AGG-ID: QSkBMZxpNEuxG-sRHCkaYQ_1778192556 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-48e60b55de2so7978645e9.3 for ; Thu, 07 May 2026 15:22:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192556; x=1778797356; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=eGyUdojsR0b/AAxiGg7aPnV5Y+huy5DwlenqjNbFa0M=; b=RSUBddrsGYZGn/3au+3/LZPHhCOdO8adAqfyoDWFiztAtU1O7gNk08WHFgQ6cJkMul HIKTJtHwLOvTe9tZNV/Chj05hTbRjZLBQ2xozEiWeGM6p5ZMqJUsrFuClUWgWFpF+yg9 TRS3WXVbb8zyUuuhD0A91os6a983r4POdc4flpiK9IxpYL6qvxw6D43Ja5Ccwo0DevyD xrZhPR7gjJRQW2MIF8d9hLKSQZBaWyNN9Bm8T715wXetYP2icKhdAWtiGZLdKTVdl4qW KR1U3NpQBicnCU7T4I99SN5hItCBCX+M1JtDQXTKM0ROu5udyq1hgRVf33ohbab6TcTM xCZQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192556; x=1778797356; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eGyUdojsR0b/AAxiGg7aPnV5Y+huy5DwlenqjNbFa0M=; b=Geou+7RI2AOG6HZ+llcPNqaHb+1WFbdy69IcgZqxpEFx5DNYdOGvsz63WhmIdRmFo5 JW+UOkOQ7buC3Xxu2zzY78Gf7wyV4rnbggoNg6zIo8NSVPnmAvCGhJdHgyChejV+q3Xi i9DTXi35xAU+4Jb21Y3SV1mOtK5uAdHgZ3Sqazsko0rxvfVuTQWgkGNttAYksbZF7eij Oo08RArmcPggKrTJ234VXJHhWm72u9KYYjn1IpV20fvxcb+50BKozV0SoTe7tWKzBUPw WafGbasrBeG52ypJlGhUqs3GtPxTWlElPi2Ykbm/qLlGNBPy2D7lrogkHTk2bItMe8Vf IW/g== X-Gm-Message-State: AOJu0YzIKMMcM7EMilONr5xKm3uYJwTNs1i5ei0nneHDHpGsQZhwGYjl DzDxoDB9QJaApLLfuX49vlRXHdUvV2eZTF9XOkdYX08VnssIz5JsvYfd2Az+HyjyBY+mJPk7JyK hchxp1i0dyNBHsPRRsX+tXMlMb7VrXvrvs+e32ufO/YzcsZZ3G/eDSPUmXuirzu2gA/6+aLK7or y1PQLejVtV5bY1Ah26mnZGW9RMIJar6wJgDjXA6RBgtRE= X-Gm-Gg: AeBDieu1Pb3RT1pCasl1JCXdXRvC9AK8WYCywPPvsgcQ8V1NLiv57um185KRMOL9/Gc 29hIk26WMe0r/p540H8aJWvWs3Qp9gGpGqO2H2TovvmWfmS5FdngQi4MWNbfcPRCXOhLLNTWsGG 0x8PmqnZsrrri3ODd2OnvF4abz2SlrV+kYsSZfKHBvtb1wkGCIOo00khFK1TOXT0P89FofQq1rU d0b8AWSBIy3T7wnwVP4Y2uZgq4e6bYfAVMhiX+rZUsugXREBkcBmZqiUXRuYLfbtkeVSfdHhTZn j74LmB0idhNfW7yw0+raZygkiPmFCsv8coqg1WVi8HWqJEs6iB9YfLrH9oqxShPI8SzYjcCuVie ncVk4CMXPIkSExHuKXrAcY+lZeFhpTV6UVYjAgtqb X-Received: by 2002:a05:600c:4449:b0:48a:89d9:a419 with SMTP id 5b1f17b1804b1-48e51f2e67fmr155695325e9.11.1778192555406; Thu, 07 May 2026 15:22:35 -0700 (PDT) X-Received: by 2002:a05:600c:4449:b0:48a:89d9:a419 with SMTP id 5b1f17b1804b1-48e51f2e67fmr155694655e9.11.1778192554712; Thu, 07 May 2026 15:22:34 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48e642eb9e8sm5315695e9.9.2026.05.07.15.22.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:22:34 -0700 (PDT) Date: Thu, 7 May 2026 18:22:31 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Muchun Song , Oscar Salvador , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , linux-mm@kvack.org Subject: [PATCH v5 02/28] mm: thread user_addr through page allocator for cache-friendly zeroing Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Thread a user virtual address from vma_alloc_folio() down through the page allocator to post_alloc_hook(). This is plumbing preparation for a subsequent patch that will use user_addr to call folio_zero_user() for cache-friendly zeroing of user pages. The user_addr is stored in struct alloc_context and flows through: vma_alloc_folio -> folio_alloc_mpol -> __alloc_pages_mpol -> __alloc_frozen_pages -> get_page_from_freelist -> prep_new_page -> post_alloc_hook USER_ADDR_NONE ((unsigned long)-1) is used for non-user allocations, since address 0 is a valid userspace mapping. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh mm: vma_alloc_folio: accept unaligned address Align addr to PAGE_SIZE << order inside vma_alloc_folio before using it for NUMA policy lookup. All current callers pass order 0 where any page-aligned address is already aligned, so no functional change. This will allow higher-order callers to pass the raw fault address without pre-aligning. Fold vma_alloc_folio_user_addr into vma_alloc_folio since it is no longer needed as a separate API. Signed-off-by: Michael S. Tsirkin --- include/linux/gfp.h | 2 +- mm/compaction.c | 5 ++--- mm/hugetlb.c | 36 ++++++++++++++++++++---------------- mm/internal.h | 18 +++++++++++++++--- mm/mempolicy.c | 42 +++++++++++++++++++++++++++++++----------- mm/page_alloc.c | 44 +++++++++++++++++++++++++++++--------------- mm/slub.c | 4 ++-- 7 files changed, 100 insertions(+), 51 deletions(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index 7ccbda35b9ad..ee35c5367abc 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -337,7 +337,7 @@ static inline struct folio *folio_alloc_noprof(gfp_t gf= p, unsigned int order) static inline struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned in= t order, struct mempolicy *mpol, pgoff_t ilx, int nid) { - return folio_alloc_noprof(gfp, order); + return __folio_alloc_noprof(gfp, order, numa_node_id(), NULL); } #endif =20 diff --git a/mm/compaction.c b/mm/compaction.c index 1e8f8eca318c..c1039a9373e5 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -82,7 +82,7 @@ static inline bool is_via_compact_memory(int order) { ret= urn false; } =20 static struct page *mark_allocated_noprof(struct page *page, unsigned int = order, gfp_t gfp_flags) { - post_alloc_hook(page, order, __GFP_MOVABLE); + post_alloc_hook(page, order, __GFP_MOVABLE, USER_ADDR_NONE); set_page_refcounted(page); return page; } @@ -1832,8 +1832,7 @@ static struct folio *compaction_alloc_noprof(struct f= olio *src, unsigned long da set_page_private(&freepage[size], start_order); } dst =3D (struct folio *)freepage; - - post_alloc_hook(&dst->page, order, __GFP_MOVABLE); + post_alloc_hook(&dst->page, order, __GFP_MOVABLE, USER_ADDR_NONE); set_page_refcounted(&dst->page); if (order) prep_compound_page(&dst->page, order); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 0beb6e22bc26..de8361b503d2 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1842,7 +1842,8 @@ struct address_space *hugetlb_folio_mapping_lock_writ= e(struct folio *folio) } =20 static struct folio *alloc_buddy_frozen_folio(int order, gfp_t gfp_mask, - int nid, nodemask_t *nmask, nodemask_t *node_alloc_noretry) + int nid, nodemask_t *nmask, nodemask_t *node_alloc_noretry, + unsigned long addr) { struct folio *folio; bool alloc_try_hard =3D true; @@ -1859,7 +1860,7 @@ static struct folio *alloc_buddy_frozen_folio(int ord= er, gfp_t gfp_mask, if (alloc_try_hard) gfp_mask |=3D __GFP_RETRY_MAYFAIL; =20 - folio =3D (struct folio *)__alloc_frozen_pages(gfp_mask, order, nid, nmas= k); + folio =3D (struct folio *)__alloc_frozen_pages(gfp_mask, order, nid, nmas= k, addr); =20 /* * If we did not specify __GFP_RETRY_MAYFAIL, but still got a @@ -1888,7 +1889,7 @@ static struct folio *alloc_buddy_frozen_folio(int ord= er, gfp_t gfp_mask, =20 static struct folio *only_alloc_fresh_hugetlb_folio(struct hstate *h, gfp_t gfp_mask, int nid, nodemask_t *nmask, - nodemask_t *node_alloc_noretry) + nodemask_t *node_alloc_noretry, unsigned long addr) { struct folio *folio; int order =3D huge_page_order(h); @@ -1900,7 +1901,7 @@ static struct folio *only_alloc_fresh_hugetlb_folio(s= truct hstate *h, folio =3D alloc_gigantic_frozen_folio(order, gfp_mask, nid, nmask); else folio =3D alloc_buddy_frozen_folio(order, gfp_mask, nid, nmask, - node_alloc_noretry); + node_alloc_noretry, addr); if (folio) init_new_hugetlb_folio(folio); return folio; @@ -1914,11 +1915,12 @@ static struct folio *only_alloc_fresh_hugetlb_folio= (struct hstate *h, * pages is zero, and the accounting must be done in the caller. */ static struct folio *alloc_fresh_hugetlb_folio(struct hstate *h, - gfp_t gfp_mask, int nid, nodemask_t *nmask) + gfp_t gfp_mask, int nid, nodemask_t *nmask, + unsigned long addr) { struct folio *folio; =20 - folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, NULL); + folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, NULL, a= ddr); if (folio) hugetlb_vmemmap_optimize_folio(h, folio); return folio; @@ -1958,7 +1960,7 @@ static struct folio *alloc_pool_huge_folio(struct hst= ate *h, struct folio *folio; =20 folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, node, - nodes_allowed, node_alloc_noretry); + nodes_allowed, node_alloc_noretry, USER_ADDR_NONE); if (folio) return folio; } @@ -2127,7 +2129,8 @@ int dissolve_free_hugetlb_folios(unsigned long start_= pfn, unsigned long end_pfn) * Allocates a fresh surplus page from the page allocator. */ static struct folio *alloc_surplus_hugetlb_folio(struct hstate *h, - gfp_t gfp_mask, int nid, nodemask_t *nmask) + gfp_t gfp_mask, int nid, nodemask_t *nmask, + unsigned long addr) { struct folio *folio =3D NULL; =20 @@ -2139,7 +2142,7 @@ static struct folio *alloc_surplus_hugetlb_folio(stru= ct hstate *h, goto out_unlock; spin_unlock_irq(&hugetlb_lock); =20 - folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask); + folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, addr); if (!folio) return NULL; =20 @@ -2182,7 +2185,7 @@ static struct folio *alloc_migrate_hugetlb_folio(stru= ct hstate *h, gfp_t gfp_mas if (hstate_is_gigantic(h)) return NULL; =20 - folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask); + folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, USER_ADDR_NO= NE); if (!folio) return NULL; =20 @@ -2218,14 +2221,14 @@ struct folio *alloc_buddy_hugetlb_folio_with_mpol(s= truct hstate *h, if (mpol_is_preferred_many(mpol)) { gfp_t gfp =3D gfp_mask & ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL); =20 - folio =3D alloc_surplus_hugetlb_folio(h, gfp, nid, nodemask); + folio =3D alloc_surplus_hugetlb_folio(h, gfp, nid, nodemask, addr); =20 /* Fallback to all nodes if page=3D=3DNULL */ nodemask =3D NULL; } =20 if (!folio) - folio =3D alloc_surplus_hugetlb_folio(h, gfp_mask, nid, nodemask); + folio =3D alloc_surplus_hugetlb_folio(h, gfp_mask, nid, nodemask, addr); mpol_cond_put(mpol); return folio; } @@ -2332,7 +2335,8 @@ static int gather_surplus_pages(struct hstate *h, lon= g delta) * down the road to pick the current node if that is the case. */ folio =3D alloc_surplus_hugetlb_folio(h, htlb_alloc_mask(h), - NUMA_NO_NODE, &alloc_nodemask); + NUMA_NO_NODE, &alloc_nodemask, + USER_ADDR_NONE); if (!folio) { alloc_ok =3D false; break; @@ -2738,7 +2742,7 @@ static int alloc_and_dissolve_hugetlb_folio(struct fo= lio *old_folio, spin_unlock_irq(&hugetlb_lock); gfp_mask =3D htlb_alloc_mask(h) | __GFP_THISNODE; new_folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, - nid, NULL); + nid, NULL, USER_ADDR_NONE); if (!new_folio) return -ENOMEM; goto retry; @@ -3434,13 +3438,13 @@ static void __init hugetlb_hstate_alloc_pages_oneno= de(struct hstate *h, int nid) gfp_t gfp_mask =3D htlb_alloc_mask(h) | __GFP_THISNODE; =20 folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, - &node_states[N_MEMORY], NULL); + &node_states[N_MEMORY], NULL, USER_ADDR_NONE); if (!folio && !list_empty(&folio_list) && hugetlb_vmemmap_optimizable_size(h)) { prep_and_add_allocated_folios(h, &folio_list); INIT_LIST_HEAD(&folio_list); folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, - &node_states[N_MEMORY], NULL); + &node_states[N_MEMORY], NULL, USER_ADDR_NONE); } if (!folio) break; diff --git a/mm/internal.h b/mm/internal.h index cb0af847d7d9..e39abab956e7 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -641,6 +641,12 @@ void calculate_min_free_kbytes(void); int __meminit init_per_zone_wmark_min(void); void page_alloc_sysctl_init(void); =20 +/* + * Sentinel for user_addr: indicates a non-user allocation. + * Cannot use 0 because address 0 is a valid userspace mapping. + */ +#define USER_ADDR_NONE ((unsigned long)-1) + /* * Structure for holding the mostly immutable allocation parameters passed * between functions involved in allocations, including the alloc_pages* @@ -672,6 +678,7 @@ struct alloc_context { */ enum zone_type highest_zoneidx; bool spread_dirty_pages; + unsigned long user_addr; }; =20 /* @@ -887,24 +894,29 @@ static inline void prep_compound_tail(struct page *he= ad, int tail_idx) set_page_private(p, 0); } =20 -void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flag= s); +void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flag= s, + unsigned long user_addr); extern bool free_pages_prepare(struct page *page, unsigned int order); =20 extern int user_min_free_kbytes; =20 struct page *__alloc_frozen_pages_noprof(gfp_t, unsigned int order, int ni= d, - nodemask_t *); + nodemask_t *, unsigned long user_addr); #define __alloc_frozen_pages(...) \ alloc_hooks(__alloc_frozen_pages_noprof(__VA_ARGS__)) void free_frozen_pages(struct page *page, unsigned int order); +void free_frozen_pages_zeroed(struct page *page, unsigned int order); void free_unref_folios(struct folio_batch *fbatch); =20 #ifdef CONFIG_NUMA struct page *alloc_frozen_pages_noprof(gfp_t, unsigned int order); +struct folio *folio_alloc_mpol_user_noprof(gfp_t gfp, unsigned int order, + struct mempolicy *pol, pgoff_t ilx, int nid, + unsigned long user_addr); #else static inline struct page *alloc_frozen_pages_noprof(gfp_t gfp, unsigned i= nt order) { - return __alloc_frozen_pages_noprof(gfp, order, numa_node_id(), NULL); + return __alloc_frozen_pages_noprof(gfp, order, numa_node_id(), NULL, USER= _ADDR_NONE); } #endif =20 diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 583b64f2b4d3..fc4c2198da01 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -2407,7 +2407,8 @@ bool mempolicy_in_oom_domain(struct task_struct *tsk, } =20 static struct page *alloc_pages_preferred_many(gfp_t gfp, unsigned int ord= er, - int nid, nodemask_t *nodemask) + int nid, nodemask_t *nodemask, + unsigned long user_addr) { struct page *page; gfp_t preferred_gfp; @@ -2420,9 +2421,11 @@ static struct page *alloc_pages_preferred_many(gfp_t= gfp, unsigned int order, */ preferred_gfp =3D gfp | __GFP_NOWARN; preferred_gfp &=3D ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL); - page =3D __alloc_frozen_pages_noprof(preferred_gfp, order, nid, nodemask); + page =3D __alloc_frozen_pages_noprof(preferred_gfp, order, nid, + nodemask, user_addr); if (!page) - page =3D __alloc_frozen_pages_noprof(gfp, order, nid, NULL); + page =3D __alloc_frozen_pages_noprof(gfp, order, nid, NULL, + user_addr); =20 return page; } @@ -2437,8 +2440,9 @@ static struct page *alloc_pages_preferred_many(gfp_t = gfp, unsigned int order, * * Return: The page on success or NULL if allocation fails. */ -static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order, - struct mempolicy *pol, pgoff_t ilx, int nid) +static struct page *__alloc_pages_mpol(gfp_t gfp, unsigned int order, + struct mempolicy *pol, pgoff_t ilx, int nid, + unsigned long user_addr) { nodemask_t *nodemask; struct page *page; @@ -2446,7 +2450,8 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsig= ned int order, nodemask =3D policy_nodemask(gfp, pol, ilx, &nid); =20 if (pol->mode =3D=3D MPOL_PREFERRED_MANY) - return alloc_pages_preferred_many(gfp, order, nid, nodemask); + return alloc_pages_preferred_many(gfp, order, nid, nodemask, + user_addr); =20 if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && /* filter "hugepage" allocation, unless from alloc_pages() */ @@ -2470,7 +2475,7 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsig= ned int order, */ page =3D __alloc_frozen_pages_noprof( gfp | __GFP_THISNODE | __GFP_NORETRY, order, - nid, NULL); + nid, NULL, user_addr); if (page || !(gfp & __GFP_DIRECT_RECLAIM)) return page; /* @@ -2482,7 +2487,7 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsig= ned int order, } } =20 - page =3D __alloc_frozen_pages_noprof(gfp, order, nid, nodemask); + page =3D __alloc_frozen_pages_noprof(gfp, order, nid, nodemask, user_addr= ); =20 if (unlikely(pol->mode =3D=3D MPOL_INTERLEAVE || pol->mode =3D=3D MPOL_WEIGHTED_INTERLEAVE) && page) { @@ -2498,11 +2503,18 @@ static struct page *alloc_pages_mpol(gfp_t gfp, uns= igned int order, return page; } =20 -struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order, +static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order, struct mempolicy *pol, pgoff_t ilx, int nid) { - struct page *page =3D alloc_pages_mpol(gfp | __GFP_COMP, order, pol, - ilx, nid); + return __alloc_pages_mpol(gfp, order, pol, ilx, nid, USER_ADDR_NONE); +} + +struct folio *folio_alloc_mpol_user_noprof(gfp_t gfp, unsigned int order, + struct mempolicy *pol, pgoff_t ilx, int nid, + unsigned long user_addr) +{ + struct page *page =3D __alloc_pages_mpol(gfp | __GFP_COMP, order, pol, + ilx, nid, user_addr); if (!page) return NULL; =20 @@ -2510,6 +2522,14 @@ struct folio *folio_alloc_mpol_noprof(gfp_t gfp, uns= igned int order, return page_rmappable_folio(page); } =20 +struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order, + struct mempolicy *pol, pgoff_t ilx, int nid) +{ + return folio_alloc_mpol_user_noprof(gfp, order, pol, ilx, nid, + USER_ADDR_NONE); +} +EXPORT_SYMBOL(folio_alloc_mpol_noprof); + /** * vma_alloc_folio - Allocate a folio for a VMA. * @gfp: GFP flags. diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 0e6ec7310087..c9efc07741b9 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1837,7 +1837,7 @@ static inline bool should_skip_init(gfp_t flags) } =20 inline void post_alloc_hook(struct page *page, unsigned int order, - gfp_t gfp_flags) + gfp_t gfp_flags, unsigned long user_addr) { bool init =3D !want_init_on_free() && want_init_on_alloc(gfp_flags) && !should_skip_init(gfp_flags); @@ -1892,9 +1892,10 @@ inline void post_alloc_hook(struct page *page, unsig= ned int order, } =20 static void prep_new_page(struct page *page, unsigned int order, gfp_t gfp= _flags, - unsigned int alloc_flags) + unsigned int alloc_flags, + unsigned long user_addr) { - post_alloc_hook(page, order, gfp_flags); + post_alloc_hook(page, order, gfp_flags, user_addr); =20 if (order && (gfp_flags & __GFP_COMP)) prep_compound_page(page, order); @@ -3959,7 +3960,8 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int o= rder, int alloc_flags, page =3D rmqueue(zonelist_zone(ac->preferred_zoneref), zone, order, gfp_mask, alloc_flags, ac->migratetype); if (page) { - prep_new_page(page, order, gfp_mask, alloc_flags); + prep_new_page(page, order, gfp_mask, alloc_flags, + ac->user_addr); =20 /* * If this is a high-order atomic allocation then check @@ -4194,7 +4196,8 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned= int order, =20 /* Prep a captured page if available */ if (page) - prep_new_page(page, order, gfp_mask, alloc_flags); + prep_new_page(page, order, gfp_mask, alloc_flags, + ac->user_addr); =20 /* Try get a page from the freelist if available */ if (!page) @@ -5072,7 +5075,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, struct zoneref *z; struct per_cpu_pages *pcp; struct list_head *pcp_list; - struct alloc_context ac; + struct alloc_context ac =3D { .user_addr =3D USER_ADDR_NONE }; gfp_t alloc_gfp; unsigned int alloc_flags =3D ALLOC_WMARK_LOW; int nr_populated =3D 0, nr_account =3D 0; @@ -5187,7 +5190,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, } nr_account++; =20 - prep_new_page(page, 0, gfp, 0); + prep_new_page(page, 0, gfp, 0, USER_ADDR_NONE); set_page_refcounted(page); page_array[nr_populated++] =3D page; } @@ -5212,12 +5215,13 @@ EXPORT_SYMBOL_GPL(alloc_pages_bulk_noprof); * This is the 'heart' of the zoned buddy allocator. */ struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order, - int preferred_nid, nodemask_t *nodemask) + int preferred_nid, nodemask_t *nodemask, + unsigned long user_addr) { struct page *page; unsigned int alloc_flags =3D ALLOC_WMARK_LOW; gfp_t alloc_gfp; /* The gfp_t that was actually used for allocation */ - struct alloc_context ac =3D { }; + struct alloc_context ac =3D { .user_addr =3D user_addr }; =20 /* * There are several places where we assume that the order value is sane @@ -5278,10 +5282,12 @@ EXPORT_SYMBOL(__alloc_frozen_pages_noprof); =20 struct page *__alloc_pages_noprof(gfp_t gfp, unsigned int order, int preferred_nid, nodemask_t *nodemask) + { struct page *page; =20 - page =3D __alloc_frozen_pages_noprof(gfp, order, preferred_nid, nodemask); + page =3D __alloc_frozen_pages_noprof(gfp, order, preferred_nid, + nodemask, USER_ADDR_NONE); if (page) set_page_refcounted(page); return page; @@ -5309,7 +5315,8 @@ struct folio *vma_alloc_folio_noprof(gfp_t gfp, int o= rder, gfp |=3D __GFP_NOWARN; =20 pol =3D get_vma_policy(vma, addr, order, &ilx); - folio =3D folio_alloc_mpol_noprof(gfp, order, pol, ilx, numa_node_id()); + folio =3D folio_alloc_mpol_user_noprof(gfp, order, pol, ilx, + numa_node_id(), addr); mpol_cond_put(pol); return folio; } @@ -5317,10 +5324,17 @@ struct folio *vma_alloc_folio_noprof(gfp_t gfp, int= order, struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, struct vm_area_struct *vma, unsigned long addr) { + struct page *page; + if (vma->vm_flags & VM_DROPPABLE) gfp |=3D __GFP_NOWARN; =20 - return folio_alloc_noprof(gfp, order); + page =3D __alloc_frozen_pages_noprof(gfp | __GFP_COMP, order, + numa_node_id(), NULL, addr); + if (!page) + return NULL; + set_page_refcounted(page); + return page_rmappable_folio(page); } #endif EXPORT_SYMBOL(vma_alloc_folio_noprof); @@ -6938,7 +6952,7 @@ static void split_free_frozen_pages(struct list_head = *list, gfp_t gfp_mask) list_for_each_entry_safe(page, next, &list[order], lru) { int i; =20 - post_alloc_hook(page, order, gfp_mask); + post_alloc_hook(page, order, gfp_mask, USER_ADDR_NONE); if (!order) continue; =20 @@ -7144,7 +7158,7 @@ int alloc_contig_frozen_range_noprof(unsigned long st= art, unsigned long end, struct page *head =3D pfn_to_page(start); =20 check_new_pages(head, order); - prep_new_page(head, order, gfp_mask, 0); + prep_new_page(head, order, gfp_mask, 0, USER_ADDR_NONE); } else { ret =3D -EINVAL; WARN(true, "PFN range: requested [%lu, %lu), allocated [%lu, %lu)\n", @@ -7809,7 +7823,7 @@ struct page *alloc_frozen_pages_nolock_noprof(gfp_t g= fp_flags, int nid, unsigned gfp_t alloc_gfp =3D __GFP_NOWARN | __GFP_ZERO | __GFP_NOMEMALLOC | __GFP_= COMP | gfp_flags; unsigned int alloc_flags =3D ALLOC_TRYLOCK; - struct alloc_context ac =3D { }; + struct alloc_context ac =3D { .user_addr =3D USER_ADDR_NONE }; struct page *page; =20 VM_WARN_ON_ONCE(gfp_flags & ~__GFP_ACCOUNT); diff --git a/mm/slub.c b/mm/slub.c index 0c906fefc31b..fc8f998a0fe1 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -3266,7 +3266,7 @@ static inline struct slab *alloc_slab_page(gfp_t flag= s, int node, else if (node =3D=3D NUMA_NO_NODE) page =3D alloc_frozen_pages(flags, order); else - page =3D __alloc_frozen_pages(flags, order, node, NULL); + page =3D __alloc_frozen_pages(flags, order, node, NULL, USER_ADDR_NONE); =20 if (!page) return NULL; @@ -5178,7 +5178,7 @@ static void *___kmalloc_large_node(size_t size, gfp_t= flags, int node) if (node =3D=3D NUMA_NO_NODE) page =3D alloc_frozen_pages_noprof(flags, order); else - page =3D __alloc_frozen_pages_noprof(flags, order, node, NULL); + page =3D __alloc_frozen_pages_noprof(flags, order, node, NULL, USER_ADDR= _NONE); =20 if (page) { ptr =3D page_address(page); --=20 MST From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8D9FA395DA9 for ; Thu, 7 May 2026 22:22:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192562; cv=none; b=m8AiFSmxsv5sG2aBcb8uNHw9kKfPQceaEuYjbDny6GZf6ZtZYAv9TAfD96gCYzGu4iEMAfhwOk677xc8ptgHdNeZd4ouVxPdWNQtGGvh1++0U714IXrtn8dD8OudVmsk+Cq+oz3HfObJAxG7Hy3roqhY6bEuBI1Y/rEC8pwD/vU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192562; c=relaxed/simple; bh=SyvCDA/S1MBPKWjxV8tbMqgXQnFhRvuNE+RO4Eohe+Q=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=IT+OA9ZRyrWOY74Y/r7kFRHau/jYcu1r7xP1BUHVHOQeQiS2LXspiqw8L2amTOhfuiBcmmZCdWHbXrFuQGkIQwVCSx39+JJQ0uW4j5RZuv+GDNGiD8J6yEVr/yTMphWzFds4vQZD3ncCd/ZKOSvNPdvxfGUfwbxRE+wuU+MfPNw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=eehWK8HU; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=noaO0Vgc; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="eehWK8HU"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="noaO0Vgc" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192560; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=QJWhlvrr2Y6sYROe+xcrAWinMhD4jCh5Mi8G3Y5SCz4=; b=eehWK8HUo7ogJlyczJqKXa+CZ2oKMQaTBvJGDLnMKfoKivP7LO56k4vgdEIodI0vCwurEM i+WWies1TVnh1orRAnqcad+lFgSw+37KbSGvmXlD2DUnO49AGoO/QybgLVcZuFoLke+zxE Q4GSj0E5w4GCptHEXmgwG8Pfdk+lJPo= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-467-V7j0pCmmOzejvpSE0H78jw-1; Thu, 07 May 2026 18:22:39 -0400 X-MC-Unique: V7j0pCmmOzejvpSE0H78jw-1 X-Mimecast-MFC-AGG-ID: V7j0pCmmOzejvpSE0H78jw_1778192558 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-48d0889c1ecso8934215e9.0 for ; Thu, 07 May 2026 15:22:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192558; x=1778797358; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=QJWhlvrr2Y6sYROe+xcrAWinMhD4jCh5Mi8G3Y5SCz4=; b=noaO0VgcihSQSPh9pzW/S3sryaIA0NzFWaUobCQazNb1xGarytJUEE8Nsh2XXewYFE sHam5SmIp6TUEdsmDygQbjLjQuXOEBsafm3JXTGEtU3r84TcjIGcY8lRqKH5zmoDc4fv NB+2QO6/FgM8nAiTh4cBvPNC8vQqYpcJz+K6OwrgfGFt50nOsCDu4M8ThdGQ5doy8pKM YjPWB6+JYGdcqJNJNV8BZUXzUN9f36qbBU595Sk+NUhJ2jQ7rV6onjTKO4mi0VD52c3Z 4yj2bt2N5HaERq4rrGiChkZJEO2MEoyro910kDzUKJIHv2R3p1biqD7lUlycZ8/1w0zE BaRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192558; x=1778797358; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QJWhlvrr2Y6sYROe+xcrAWinMhD4jCh5Mi8G3Y5SCz4=; b=AfhNM/690d+OouKp0JG12zsiAfMtlX4HSIAnQEjUBAbTLbUC+gsH2nqwi5t7k0eIed 0Zkm2/8uJp6WGLRsZuhtkH8tPmft0/Af0LmgRcPiSvut0F1iIOHgeiVg+XmiYOKgw/wm TEpfNS0ef1jadVQDFks/J+hb0BSSxeOaZlVDbrFOgPVfg+/Mp9DoHqY4CKHDbatn+/+E sfn1d7RY0LG7mdj3/i8qXm2EPVY9ccpnP/ZJ/6hrNqC3buAdZYohW9OCINMYjU3umUtH GaShNCc5+JPbLNxytinh2ZXYozedLY4DLeCqID9jrtIbmY9QeGQV+GQtJjl7VLNrf80r H63A== X-Gm-Message-State: AOJu0Yx082bL2gHauqSZPU3HJ7I/cu6O00FjDHmj4/EB8vhdA+r8VObh 2+TRLv85j7XUeFpvBXMFF2fAYagBGDlZ2RjxuC+7bMfpU8naUxveyuUD/FMDbapY+XuPyOTHB9M QWM4DGL8WnBY0qeJKRkI/GRECTvD9FNJd0OpPD4ROcnmC3KjYJqwTUyA3i8oqKkBfSR5R6oAuCk 7abeVgAfReB9IMihWqxNsCsXVftWN/7Hm+4Yf+6wCiFkA= X-Gm-Gg: AeBDieuCpDsE2mmj2IOAIC2mfyswLfoeO8xgvs6UYJsud81hBSdH/FoGmFDtbjhnhPX G6D9BjNONX/ALI4QLyuPDy7MIajWHcw7UF/CrTMMFB+zQFrtDo1nOsbCOKC5E4y/hAh+7ihkVp7 c7ZUf1t/UdPcwix79aW8H2bIvsrXeVfVT7gQf+1PInnGLN8+Ay6weERdByYEbl+GlyucysDHEFL zSKfYiO5q/6zJEvFNzDN3s7zWllnPsQJjW8Q3OPSs0SOnB0JsYEvrgJGH9MRHRB1E322DfVcHxA 80EgNsVz9yTnnvqbfso4TUoJKfJjZ24rVwegAfv51QPgTWZwFgbfb/VjgDV+xZNqOLd71WwC8no NSUfO8Hx9BCHuh5yr6wc04GaoS1jTVyafosApc5Q/ X-Received: by 2002:a05:600c:a111:b0:471:700:f281 with SMTP id 5b1f17b1804b1-48e51f46e24mr127891675e9.25.1778192557848; Thu, 07 May 2026 15:22:37 -0700 (PDT) X-Received: by 2002:a05:600c:a111:b0:471:700:f281 with SMTP id 5b1f17b1804b1-48e51f46e24mr127891365e9.25.1778192557283; Thu, 07 May 2026 15:22:37 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45416b5e3fesm1973208f8f.19.2026.05.07.15.22.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:22:36 -0700 (PDT) Date: Thu, 7 May 2026 18:22:34 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , linux-mm@kvack.org Subject: [PATCH v5 03/28] mm: add folio_zero_user stub for configs without THP/HUGETLBFS Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" folio_zero_user() is defined in mm/memory.c under CONFIG_TRANSPARENT_HUGEPAGE || CONFIG_HUGETLBFS. A subsequent patch will call it from post_alloc_hook() for all user page zeroing, so configs without THP or HUGETLBFS will need a stub. Add a macro in the #else branch that falls back to clear_user_highpages(), which handles cache aliasing correctly on VIPT architectures and is always available via highmem.h. Without THP/HUGETLBFS, only order-0 user pages are allocated, so the locality optimization in the real folio_zero_user() (zero near the faulting address last) is not needed. This also matches what vma_alloc_zeroed_movable_folio currently does. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- include/linux/mm.h | 3 +++ 1 file changed, 3 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 5be3d8a8f806..541d36e5e420 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -4718,6 +4718,9 @@ static inline bool vma_is_special_huge(const struct v= m_area_struct *vma) (vma->vm_flags & (VM_PFNMAP | VM_MIXEDMAP))); } =20 +#else /* !CONFIG_TRANSPARENT_HUGEPAGE && !CONFIG_HUGETLBFS */ +#define folio_zero_user(folio, addr_hint) \ + clear_user_highpages(&(folio)->page, (addr_hint), folio_nr_pages(folio)) #endif /* CONFIG_TRANSPARENT_HUGEPAGE || CONFIG_HUGETLBFS */ =20 #if MAX_NUMNODES > 1 --=20 MST From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DA2983976BB for ; Thu, 7 May 2026 22:22:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192567; cv=none; b=ky8DYyuFft2NZj9fxmMCSsCBpJRGwF2PulBr9qfyXc4fQj/hp4V9RvBYMP3dVkn1TMZmxarjrQRtvajNqHHi8Ds+f9Ov06GFk3kBxIH6SZDudMN4banKCaQn/hIXKGqZuAu7G9F94scqxa+BuHD1O9F2SPpmGAxQZ2CZCkLFX+Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192567; c=relaxed/simple; bh=cYnvOSKzt3Z5LKY0YwQVyrVs2NSaHnVvr9KHUQESnbE=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=A91d+YFqYvKuDev4Iqkw4l8eG7rORftJFWyqYsfiEuPrfTQpYcORX1k97a+vH1hoanO6ue7MEz2MHlaWv7D+3eSZZ8W1Ll5V5wpE2Xih0uh7Pc4YS/5oFexUh7Sp1lM7KUmX4R3+p71iWvEmtfft9IlrXpSH/wUJPuc4Gn7gKRs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=AG+h8oWr; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=b8A0HZal; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="AG+h8oWr"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="b8A0HZal" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192563; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=NawwnsjUX7qccisIUGIt5emibxZpAoP7nEpyB49cuUs=; b=AG+h8oWrwCGTaIb4/8rJjTbit3fAMyMwI6jd1OVlkRudVvcq7QYGJ0E/D8hZTAcSal+hGM fvK9yrmdhZdGPELH7pIPMOexTblgUJ19C304HMbfluAMpo0Q189pI985aKf4UBKFAipwcf fnzK+J0f+glexS9gY0xuyLWx2EquGCE= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-623-oJPb8afqO6eSEGGsinJj6A-1; Thu, 07 May 2026 18:22:42 -0400 X-MC-Unique: oJPb8afqO6eSEGGsinJj6A-1 X-Mimecast-MFC-AGG-ID: oJPb8afqO6eSEGGsinJj6A_1778192561 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-43d103e46c3so840143f8f.3 for ; Thu, 07 May 2026 15:22:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192561; x=1778797361; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=NawwnsjUX7qccisIUGIt5emibxZpAoP7nEpyB49cuUs=; b=b8A0HZalxYPlVNf0XFV/DAt4Of+MR3ADeYTHcH5iwaJ5TgvA9RXv7qBqzcl0U/WOf9 BX2qVWLRdPaxKTUUcJRHpe+2gFuSX2nbM4oGP8vdMDYTkTbMQDYOjL53ViiCdAEoL73q bkiGR4aV4bZyZBmhIxazyl77Rd6QM8VZZ2hXincKRmxX0mSpOUS18wsGNHP4VcYB/5W9 UKa28to386wVmiedeGrRo2dUHEpEcMzfUGhC8VU3gGwPEBmkUeoa4FNbdQC9WvXcCDmg t01ev9FgXnljrvqx/ko9E2tnLzUarXqskenMCwH4zTP9Ij7Uzhm9P2AN1lVR5gUiv1af s8VQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192561; x=1778797361; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NawwnsjUX7qccisIUGIt5emibxZpAoP7nEpyB49cuUs=; b=ES+WBmT8vueaYGbs+pi+AmjrbAJ1591u4vGJB9HEpJ1H9IjyIuyoGuOnDsS/3aNi1T +bLvETvhEPBwZyUZnFOIm6M1QOLWes9JpAPZy7FpUxGWln4fWzCJyvXh6V0Zrvx8hsxN tWO3mE1r12pT84sxqisPSgu7HpTOLZ0+dgA7siIHSRpzPhThAOaMmiSl6CKJAtRtKlhk WDqIxFNETpQ0tKI4rR4kIdq/0LpMasR19+OS/+jAwnS5Wmk55YFE6xWy4pxkQicHj6wG 0yutV1C38EIEYRusR2h3+rR5lQY0gpbQJoPJDN9P4eOIDRJSlhm/lBb5puwo5ycvWLqn 60OQ== X-Gm-Message-State: AOJu0YwSSqXb7xqsqWanV+k9Kc4m3hzCCFuWniwqdrGIb5rg+hr4cqnA h70SFsLwouODhGz8JlXf9Yla0vi7ZmECEatQH57Lo8uyHMAHwJKksADTZFSKdykEASKuj6WMPG4 F9EFkY6mPdo7+vRipYIsMEP3kH6zNrlvNDGMEoY//U8e9MvRv0+UfrfJv34/hMv13eRBqAot4VW Hi9ql7wKpEFStlvoJVn1PwosGzrfGTIRLGmaHyA/gigR8= X-Gm-Gg: AeBDieswAxNOU9HgViTm/vXGh+Q5gu7T/Ix7+CcvbKqBb3iqLKGDfY3AbRf7DaCM+2A Mp5IoYQkMcGm4ILV7qBnZGfxuqhC2X5DZ/0g6O97WjxG2AVpZYQ3u6Lkkb8I74rPSBqX7ZDa02T 6vv1yxDERgMv5loQ1q7iXa1yxM0J1uTw48uGU8pbWUdm7PmmxQ9nmvpEV2euwpeasnP0cMx+wgo R572U6aa6C7be7oXGTdhqesVuoL5kY86r9Sz8DWKrWHKFKYVYeme7QseoZtoeKmb+GFjW0v/KB8 e5Zx+kkG8bt127+54cahfcj6NT0JdA+8Gun6iyySsHBFmYnlkNuVaoJcv73U3nJ+7Z9dg6gmI3z E/6LRIdAQHqarPFtP3eb99hbeODzSvOrlhssLH/2H X-Received: by 2002:a05:600c:45c5:b0:48e:526e:1040 with SMTP id 5b1f17b1804b1-48e676b45eamr1526155e9.23.1778192561029; Thu, 07 May 2026 15:22:41 -0700 (PDT) X-Received: by 2002:a05:600c:45c5:b0:48e:526e:1040 with SMTP id 5b1f17b1804b1-48e676b45eamr1525695e9.23.1778192560371; Thu, 07 May 2026 15:22:40 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48e65c01d7bsm4569775e9.8.2026.05.07.15.22.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:22:39 -0700 (PDT) Date: Thu, 7 May 2026 18:22:37 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , Vlastimil Babka , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , linux-mm@kvack.org Subject: [PATCH v5 04/28] mm: page_alloc: move prep_compound_page before post_alloc_hook Message-ID: <249b45850d1c864563ba4415d55eeca9ef12dab9.1778192416.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move prep_compound_page() before post_alloc_hook() in prep_new_page(). The next patch adds a folio_zero_user() call to post_alloc_hook(), which uses folio_nr_pages() to determine how many pages to zero. Without compound metadata set up first, folio_nr_pages() returns 1 for higher-order allocations, so only the first page would be zeroed. All other operations in post_alloc_hook() (arch_alloc_page, KASAN, debug, page owner, etc.) use raw page pointers with explicit order counts and are unaffected by this reordering. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/page_alloc.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index c9efc07741b9..92640ddb0b7b 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1895,11 +1895,11 @@ static void prep_new_page(struct page *page, unsign= ed int order, gfp_t gfp_flags unsigned int alloc_flags, unsigned long user_addr) { - post_alloc_hook(page, order, gfp_flags, user_addr); - if (order && (gfp_flags & __GFP_COMP)) prep_compound_page(page, order); =20 + post_alloc_hook(page, order, gfp_flags, user_addr); + /* * page is set pfmemalloc when ALLOC_NO_WATERMARKS was necessary to * allocate the page. The expectation is that the caller is taking --=20 MST From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DF47439A056 for ; Thu, 7 May 2026 22:22:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192570; cv=none; b=R23Bce6n7wKKKsud5Xu4obPBpkmMxNAhYJAzQEt4GspddRnTw4DMoroHM53izZetlvwNy13Cw1Olp7pe0ZPf7XK14/orTWke4qmJyr+jgl3sccM8i2ks4E+F29cBuQ+8s2KG537r8qDO3aLln15SRzA/1NyptfCNSr8i9aBefOY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192570; c=relaxed/simple; bh=PR7dIULo6oVGd71HWH/+M3MZeJj687W04Wl35HO9EdY=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=F83xuQxbU5baQUe1cVr86vckpfWMe5iD/+qXLF4UFX7Ha+Ax4neQ206h1CQ3qCs8rGl1ZRA+VIlaaEjMzUIaWaWSSSyF+9PstvkjFw7KjoAQkObrVgtIRE2io+CX3QrFGuGjPPbR5bHLyMKKzFacmBSIhcA8nYDZj00k7DNyAcY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=cWXHrN+G; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=uOuIhPlO; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="cWXHrN+G"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="uOuIhPlO" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192567; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=6XZVwPk/FHchRW7F1q982DWChdDEL7ZRxXSzf1eFvwA=; b=cWXHrN+Gry8ZKxK5fJctv8Zl3s+MvWh0zt73Wpi3ReeMJrBpVgdmo0Xco9zLd7Hiv+glzn RjZRIsxe89QEtRgO39Y/r8hy9tfCpPH0HKPaBTgKvAF4Rjmu+9GMjWttUcAS/8kNrI9qld iVwPaJraVCxCVbnAu/o45xW3dOm9OIg= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-59-ltb2Hh7rNOeIhSv_fRRE4g-1; Thu, 07 May 2026 18:22:46 -0400 X-MC-Unique: ltb2Hh7rNOeIhSv_fRRE4g-1 X-Mimecast-MFC-AGG-ID: ltb2Hh7rNOeIhSv_fRRE4g_1778192566 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-43d789cebcfso1679561f8f.1 for ; Thu, 07 May 2026 15:22:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192565; x=1778797365; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=6XZVwPk/FHchRW7F1q982DWChdDEL7ZRxXSzf1eFvwA=; b=uOuIhPlOgctZJeqLu3T77YRzSZ2b6OfoanmkZusWVlFkK0PZ04HMmZ53N2iwcvOKKt D196L/PSnF1g7xPiJFHwUxiuzJwgSRuwNAhCxufXQ1svueM9Wf5ogAVihDUAXlVZUpqu gAVGRPw1DFw0PP88YWr+Y3f/egt+aMerOorhHZ5FBctCUY3wFMlsGU7ktnKMIt/+yKuB mmS4Jw3bGkUV74mMoyomyq2C/MroJnGPadtsmbCdH0BXKBxepQLpoPLEDv+2GIzcXeYG ZSVQU9IFXU8WJgfjrccK3TIovTHx39y+SudMZ4c51RQ6lTlUvt06m53DOizbzmnlw5+U A2wQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192565; x=1778797365; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6XZVwPk/FHchRW7F1q982DWChdDEL7ZRxXSzf1eFvwA=; b=tH1H1A7ubjAxAWkmfYuWUq8gpjIhPvHj4tL5bv5Y2sxaaLRlUoOMaShTRufPjpkuSU SWTyAugeKzH/Q/AhhYSIZ/lOpxM93RKWYugD/XzLTBe0j4qTX+PdP2pOTnuvRJ/4hMhR oI95SGxtywa1TJe/U+5cOG1QN7v5SMnn3Vc4JKr1TdZVEWO9fv0WrwJnaanXkaBpIzJM pMOiOOgUB9rDNgYtuKatr4IO9IPxc8laFx//BPdF9/kCS4s+mSJoZjx2hyZuKkQFr6DZ d0BbVUClm6JSwx2fcQrfxC6c/dXBLCD5C9ULGGzEZFarcGc3S4UcaOkpJ+FFV29omvF1 bgqA== X-Gm-Message-State: AOJu0Yzb76Jhy7nmBiK5g69842YEvuNYqywCIVUsEOH4pBD6QkcsTpoP 20GmoIEUFkRHpkL6B34G2SNsx8YzczLbmL0LujqbdJRjEK7OCTAaR2lGcUz0j9LmiG/3TA3k+aD UnWA1+IBL96tdCWQgsGoNMJ1ol+ljtJbvvszvU8WBgKHRci5BP4yMYnfo7M8rocxjndrhTmdZIr ymatedUbsV68oVAXvxe1o9ELV15Xxuhoxy5WmGRDg21ug= X-Gm-Gg: AeBDietwbSsHXQYnFpWt7L1AAC96qEC0RrL7z8zh1r5dKXeoqdxybyJHY7Laupr0Tcy ns77C875C8JeVBDiK3zHk85C1zWMDpOJZfcgppU0U0TNkLGDl9WZ/z4us/r5b9wR1OkdTRxJlke 601kkrDvRoZJODhGkwz0UX95s6gfvT0WegGXd/rIG/vrVyYtqskuyUetOyd94DBBdz2bQA6kL+S +nABpEUVQME0PgcV/zlKLTGlJFUEuoVUx74Gy3Vwp5frNupNrJfJoBV3uvx2GS7PSdO31NL8WeW Ud04UBavkESCLZCeMEZouJ1qJK48qY92Tb69vyMQ11niuUd7yWGYspcGdUEsGPN4YDYdL7GFbsC +LLQpEEVJjLgxBOu/oKOY2Sqcpk0iI1I93AfH+sJC X-Received: by 2002:a05:600c:3b8f:b0:48a:79d8:a8d6 with SMTP id 5b1f17b1804b1-48e642deefamr16106415e9.7.1778192565123; Thu, 07 May 2026 15:22:45 -0700 (PDT) X-Received: by 2002:a05:600c:3b8f:b0:48a:79d8:a8d6 with SMTP id 5b1f17b1804b1-48e642deefamr16105995e9.7.1778192564484; Thu, 07 May 2026 15:22:44 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48e65c0da87sm3543195e9.21.2026.05.07.15.22.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:22:43 -0700 (PDT) Date: Thu, 7 May 2026 18:22:40 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , Vlastimil Babka , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , linux-mm@kvack.org Subject: [PATCH v5 05/28] mm: use folio_zero_user for user pages in post_alloc_hook Message-ID: <4b89470a07497a55ce2a8d31e4b746e734df1f4e.1778192416.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When post_alloc_hook() needs to zero a page for an explicit __GFP_ZERO allocation for a user page (user_addr is set), use folio_zero_us= er() instead of kernel_init_pages(). This zeros near the faulting address last, keeping those cachelines hot for the impending user access. folio_zero_user() is only used for explicit __GFP_ZERO, not for init_on_alloc. On architectures with virtually-indexed caches (e.g., ARM), clear_user_highpage() performs per-line cache operations; using it for init_on_alloc would add overhead that kernel_init_pages() avoids (the page fault path flushes the cache at PTE installation time regardless). No functional change yet: current callers do not pass __GFP_ZERO for user pages (they zero at the callsite instead). Subsequent patches will convert them. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/page_alloc.c | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 92640ddb0b7b..2bfa9ab60976 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1882,9 +1882,20 @@ inline void post_alloc_hook(struct page *page, unsig= ned int order, for (i =3D 0; i !=3D 1 << order; ++i) page_kasan_tag_reset(page + i); } - /* If memory is still not initialized, initialize it now. */ - if (init) - kernel_init_pages(page, 1 << order); + /* + * If memory is still not initialized, initialize it now. + * When __GFP_ZERO was explicitly requested and user_addr is set, + * use folio_zero_user() which zeros near the faulting address + * last, keeping those cachelines hot. For init_on_alloc, use + * kernel_init_pages() to avoid unnecessary cache flush overhead + * on architectures with virtually-indexed caches. + */ + if (init) { + if ((gfp_flags & __GFP_ZERO) && user_addr !=3D USER_ADDR_NONE) + folio_zero_user(page_folio(page), user_addr); + else + kernel_init_pages(page, 1 << order); + } =20 set_page_owner(page, order, gfp_flags); page_table_check_alloc(page, order); --=20 MST From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2F2663563D4 for ; Thu, 7 May 2026 22:22:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192573; cv=none; b=todVbtq0VLmk6DX6/nhvpGbgZvl5OLLnr7iehbqJ7FKD7lXLg7qshwoRALNyP0M7m/F7Arj5r3WyTxysvd9Ckz7qTV/ArMpFbNH5hREEU7gHSj7/BUEb9vey3JF9Ma621ArJUUMpyYm5sb3p/ZnocKeXoowPG8onNhNHQixzylg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192573; c=relaxed/simple; bh=B4v7SD6SJ84KapLgBy+iXreANiiaF75HE81bu1RBt3E=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Zib+lexnvVWw/AFuvBuqUdpnnoPPuCjhO9jLpDo9rvidg3gqlPaV1c9PBn92bBTdGZjpFwY9RrzhNdqcRbHzQ3O6ucTxp8OOL63//5LwX7sitiNXuKt7AlVjvisscgDAe7kQIwzwfJBFXZiwtLnkPDR802Yxf+2ICLPOvaeOsBU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=ZDiAenX5; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=EQzmLZ8e; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ZDiAenX5"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="EQzmLZ8e" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192571; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=jpAQ9O1oMgUNlo5ukg9OEno1SM1k+70VK2/jH7/8vVM=; b=ZDiAenX5k557crH1UwzRjlvCyJSs2594qy6e4lhe/C6G5thGn2aesp8X0EiwgOu2Tj8dEh hGRiXTA0aqWpPAfcRl8OVrqEXFOZKYLUqv0BQ93h7ckSHGGMTBle1T0/wqkZ4CAUtbL744 3o9OMlTIG7MVPVAsnxSGEfTApErBPgE= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-156-yoFPyhD2OZiReDfDWJpAMQ-1; Thu, 07 May 2026 18:22:50 -0400 X-MC-Unique: yoFPyhD2OZiReDfDWJpAMQ-1 X-Mimecast-MFC-AGG-ID: yoFPyhD2OZiReDfDWJpAMQ_1778192569 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-44f1b4d0fb0so922138f8f.1 for ; Thu, 07 May 2026 15:22:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192569; x=1778797369; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=jpAQ9O1oMgUNlo5ukg9OEno1SM1k+70VK2/jH7/8vVM=; b=EQzmLZ8ezIIQ2AQc1YuTH6ZV+DNZqq+EsUGqI4iabHPyhqIQhkuOhoXEV5vdw8R5c5 QG6pPjt9acJ3YThnCy/DslC3JIhx6oPvrEIcIsXWLoChfxxKLGA35er/VFAownJBKrz1 1F8RFJNWAeyPn7jCD4ZQ4gp3HdM0Qsciyh8QK4m6q/APKkSMigS7jDyHekJNKPTtfmKL 0ctH5rgCDUpvQc5pSnR6K1qGEnNuEWdcZCXkDGwQJSYJSX5CmXMIGRN01G7h+r2Ha8JD i91RDBqvqv54ASl2liOjI+xtmxLVG2Hq2f0XvVLT5dMvC6Bchug3KwgIFYrFFO8P0pVX r7Mg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192569; x=1778797369; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jpAQ9O1oMgUNlo5ukg9OEno1SM1k+70VK2/jH7/8vVM=; b=iecX+AY//5IqP73220UZCEsFhMXmAtDJH9Sa/bCdLQ5oYO69VXcM3uhoxfXQBrLmlG egCFQNCnBwlD0irVU7l4ZRjzKhoe3BN3SPtG6PD5IKD4mBaFrTxPipGcr1/EqTty39CN +QBnXJhmimj4MMQdntAgTcN7a/DeO+QsmuaI0VbrX3BwxKhkhLXWT8L0dgV7lYo3VM/e oW9C8oBCcKVEkq+qMj0svhh7YstXZBzArjBTATUCsKg54jzdAyKRxyBigp9Xs4+UDsnw IkGYMmNL2iaoqJtmvK6CApaS0Q3oqXwnvGd7gdx7p7+x2jTQadSP79Jbqw0ZSYfsLdgf obAg== X-Gm-Message-State: AOJu0YzCr5693me3/7i4xIGIEz5d/oX9Bj1EyPFwa/2sR/VZMrqzpdcY 3fmGe69hlCH3TO6Wt+JfFdrVbRJ+P87jaeki0m5QJFM1MCsnBRGaR1wrwVozxAf3/P8GmwJYyv8 UFRTVl4q1bnpyNLl1ZFLTzKGQvyG0Nx449j3bCnPyzcAQcEZz8SSryQbEdjjJzqFWd75oGCevFa 8AlKJo65gaPDF4vySq1jtG8VDi0fZlEeqLkHnYCnx+Cfo= X-Gm-Gg: AeBDieuan7JkfOhoh8ZaaqRdDqfziEZFQDeedrr8v1UevF1cerO9Z/vWTPlPkEA3+Yn Ngzl1rL8SybFhX4IHpNNYH8idxDx3i7Y9B+gmPQFdPy8tpsrc2bZgN9GxfEF7uPBwCP8hQKCuOI qc3VfQingIJO1KOzk+jpnrNo03GKI5Z5z5qZ0rE7lMNGyvGuY0Wj/AM6ibMXOY2y7gdyyrd4KOA uDRl6eQyWyjOihQwkxcyNnZpLhFusdGLo4HknHEPVB3v0QctaCp4pgTbyb4OHLSjK6aspi6fhxB Q5MbOOno1XxyIYMo6ROPBX6A3sFvQPX3eWzCZol9c8ciLD45XYEUQl1iWofA7oDNYaTMY0OC2xQ 0Uu40F34J1ISnNR8RtQLaWqPVicGntpzFJm51E+AU X-Received: by 2002:a05:600c:c11c:b0:488:a723:ea53 with SMTP id 5b1f17b1804b1-48e51f5398emr112335485e9.7.1778192568568; Thu, 07 May 2026 15:22:48 -0700 (PDT) X-Received: by 2002:a05:600c:c11c:b0:488:a723:ea53 with SMTP id 5b1f17b1804b1-48e51f5398emr112335225e9.7.1778192567996; Thu, 07 May 2026 15:22:47 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48e642e9d76sm5598515e9.8.2026.05.07.15.22.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:22:47 -0700 (PDT) Date: Thu, 7 May 2026 18:22:44 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , linux-mm@kvack.org Subject: [PATCH v5 06/28] mm: use __GFP_ZERO in vma_alloc_zeroed_movable_folio Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that post_alloc_hook() handles cache-friendly user page zeroing via folio_zero_user(), convert vma_alloc_zeroed_movable_folio() to pass __GFP_ZERO instead of zeroing at the callsite. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/highmem.h | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/include/linux/highmem.h b/include/linux/highmem.h index af03db851a1d..ffa683f64f1d 100644 --- a/include/linux/highmem.h +++ b/include/linux/highmem.h @@ -320,13 +320,8 @@ static inline struct folio *vma_alloc_zeroed_movable_folio(struct vm_area_struct *vma, unsigned long vaddr) { - struct folio *folio; - - folio =3D vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, vma, vaddr); - if (folio && user_alloc_needs_zeroing()) - clear_user_highpage(&folio->page, vaddr); - - return folio; + return vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, + 0, vma, vaddr); } #endif =20 --=20 MST From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C73EE39A056 for ; Thu, 7 May 2026 22:22:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192576; cv=none; b=kxzbF6Ivap5ZvrT2Q6mZwBT0mgwflIfPo8SxMADspyhh4aCD2VVUScLvoQ2WMp54pK/0GM1p+tGSm5FIaoNi5I4SeRA8Z1fa+GP4ygWR7YcH3lhsUExCy7Pz4q2Fpkl4lR9NUP+0AgLHW/OlzTfmRt6dBHDDrgNazsREl6HcBjk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192576; c=relaxed/simple; bh=qZKFSaT31Wgo7mt5Xa//ol6/mvGRJjSk2moWE0Nv1Ow=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=SUxuMi78WSY+osoJJQsF30NrWjLp1N9InLU7SglAQie6oYHXDKppbZxmIUZNEDYY4M9JRJQ5pZWp5qTLXhgOePqfNlZmjgmgUL58fK5yVmfbsKLkfg1ReVmRU7YUhtg0H24LV23VdBIbSCxUT/IIgbzdraKGaw/9SleM8qcU22g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=EKvHNyVI; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=k59R+1Mm; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="EKvHNyVI"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="k59R+1Mm" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192573; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=0cHnxMdJPiaftqs6T9PdnHyPobnKwxYX/QlPeFWMLsg=; b=EKvHNyVIKhaEFPPt6yl9oUkhkALBpIkXriAyZTnVRgrcXguc4M0AoFtslUkocmTEJbBXb3 FuS3FurujbgFzQrNeNtlb96eFltQeBrFbVoQMmiT3TRUtN2P4Jb4+CIoXSsgdmu+K2ailR t2M2rK2PmzA5oB8QvOuFTiEvxO21DtM= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-201-_kJDUFZ-P7-nNKpUpwc2cA-1; Thu, 07 May 2026 18:22:52 -0400 X-MC-Unique: _kJDUFZ-P7-nNKpUpwc2cA-1 X-Mimecast-MFC-AGG-ID: _kJDUFZ-P7-nNKpUpwc2cA_1778192571 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-48d127eb013so7831665e9.1 for ; Thu, 07 May 2026 15:22:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192571; x=1778797371; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=0cHnxMdJPiaftqs6T9PdnHyPobnKwxYX/QlPeFWMLsg=; b=k59R+1MmBfCdmp36as/LOXnLU0ZIl5eHX8AVckZ9t9rr1ouO5cxRB+KMDXcCxz0zRO 5AkYkuL4i8KvVQxKlqCDDhyxnURq6u6nS33YN1/R5xjhkWRO0wsQ1KBWIAgbU7qThidh iyQv8UBN3rZbpq6J8olL2POegeIL07GuzC2lHSDlt6QOBa1dLtCfL2k/R6Bm4KYaGTt0 UpQszgqrJStC57cY4+CcRe0875a5FU1eXIHP6fFqUpKHw+d0sRONuo3VKa4HMrmnO75m X0GhxeM3zVMsCvbi97voHhAgyJ45fP/DM6Lgjqplzn5IkZEg+VkL/x9/St9sOdNeOOiA JcdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192571; x=1778797371; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0cHnxMdJPiaftqs6T9PdnHyPobnKwxYX/QlPeFWMLsg=; b=XubrxeG/nOJ/tcsy+eexbgpTXCdkZPMkJ5Ci6NytMEnYZGpx7ve7rew41bUUwXXYij ntVlCq04jMvDUGQRRwQaM6lkK7ZGhVIUr0wxopeKiCVqLuP5/UzPu0x8Occnj0/05GmG mjrUI/BedoevRdjvLiRq1a8vvFig8S7DBnhbT+05tmKP/lMw6jeNgQg1S6dG0rTpTLwU /HABWfaYYDDs8ByF4kNq544DWSa7lfaD3ES3QBg8YBvxOrgfESXs34X0hG1zeKWauf2t ONn++rEkMLY68JZTcsgDsp7YCB1TOsbAmPlg3LxqbQgmpmEx0BTt4991Io1kyENokFzB E5Vw== X-Gm-Message-State: AOJu0Yy8mzGWN4w1Q+JmdUEDBE0QKM0BGeibXMPo7zDbMVph8LrtgTRb 0I2rRzUuIHtLfS/gy2jHeaOVRIkbgpqNuVDT7Gyl+hET3touK3IMWLeTAanvQsor+IdpB0X3kVt 6zVfiu9I4bLvTn46IG7TXeIjBTSk2yw+7oc8gYOaiuQiAffQjTOuUBe3cFwUCKAjcPNog2xLc+W VkOAuXl1znSLLp7dL74I9TymiS7c+9C0M+6FN4rAZHpa0= X-Gm-Gg: AeBDiev1zkHnghQMJjFRHA48LCazu2b7k/FV86P/x/P3g/DmruSZpJfOL4ec+4gUkfm xlXOtHY08bw5VbdEzytRZWW9HQ5Y59ErDiiCdvZOvx7Bnw0fsfcaS4g4dHRBCPH7w6rhLrz2DKj 78bO155zjaZBmX2OTTICS553bcDB1lKTA2m81etoAKSKC5jNkDVQ5IZbl6VOg/lchCLWE8N3kor AW6gfIFzezqFDSL1zqD9slC1b1oi1njhZfQPzXCUSET/SOFt1oNBUd/dOgcjfLilIntHd5D+DZ3 VvfsiAuGeXMUs0sku1y4VGJ53sOhMFQc5Z4hGHeHM2JzYLFuJn1IVUwZxc6JJD3wcZUbNG1V0VF aZZhSx+u96uMdmiVKl3I3eWXrvnhrmtSNVBPq0QWD X-Received: by 2002:a05:600c:45d3:b0:48a:563c:c8d6 with SMTP id 5b1f17b1804b1-48e6767dd6bmr1897885e9.7.1778192571020; Thu, 07 May 2026 15:22:51 -0700 (PDT) X-Received: by 2002:a05:600c:45d3:b0:48a:563c:c8d6 with SMTP id 5b1f17b1804b1-48e6767dd6bmr1897445e9.7.1778192570328; Thu, 07 May 2026 15:22:50 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45416b5c300sm2083644f8f.18.2026.05.07.15.22.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:22:49 -0700 (PDT) Date: Thu, 7 May 2026 18:22:48 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , linux-mm@kvack.org Subject: [PATCH v5 07/28] mm: alloc_anon_folio: pass raw fault address to vma_alloc_folio Message-ID: <6d63fef8150c92fcffde0742877fe18ccc3da59a.1778192416.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that vma_alloc_folio aligns the address internally, callers no longer need to pre-align. Pass vmf->address directly. Signed-off-by: Michael S. Tsirkin --- mm/memory.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 07778814b4a8..70ab8b3e3a29 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4661,8 +4661,7 @@ static struct folio *alloc_swap_folio(struct vm_fault= *vmf) /* Try allocating the highest of the remaining orders. */ gfp =3D vma_thp_gfp_mask(vma); while (orders) { - addr =3D ALIGN_DOWN(vmf->address, PAGE_SIZE << order); - folio =3D vma_alloc_folio(gfp, order, vma, addr); + folio =3D vma_alloc_folio(gfp, order, vma, vmf->address); if (folio) { if (!mem_cgroup_swapin_charge_folio(folio, vma->vm_mm, gfp, entry)) @@ -5178,8 +5177,7 @@ static struct folio *alloc_anon_folio(struct vm_fault= *vmf) /* Try allocating the highest of the remaining orders. */ gfp =3D vma_thp_gfp_mask(vma); while (orders) { - addr =3D ALIGN_DOWN(vmf->address, PAGE_SIZE << order); - folio =3D vma_alloc_folio(gfp, order, vma, addr); + folio =3D vma_alloc_folio(gfp, order, vma, vmf->address); if (folio) { if (mem_cgroup_charge(folio, vma->vm_mm, gfp)) { count_mthp_stat(order, MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE); --=20 MST From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D3A939D6D6 for ; Thu, 7 May 2026 22:22:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192578; cv=none; b=q93/TeiYrfR++CdukTLTQRuOnRHg9jzLJOWbUxtl7tiDXtbNq90YlE2uag0PVzd74x31qRLkbWQeOXRy13cIfK6VmUNjzTGzaMV+aFSHqR/bEMqZP6+wz0oZxgxj5QCF83/UJzhvsyNqmLwzkLbm1g9Rlrz8ajDVdrfKK5NxeaI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192578; c=relaxed/simple; bh=fYSxmyqlJoJGwiBnqgQinqO7ohbTB5EihcsAv6YlT1k=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=ifjl/jyZsG/Psg6OublsakoQSUaoNdTMr1ARSwH/q+GmOZWLRSkLwSYSYbgGjntkwO09koz2kqGPgvahJ+8AWXloSbHWPw4xdRQY/jQac4tNiWXbkuPOv8b661JHZDqL31xJlott819i3UQKm9NvqYGvDJhtjqigIG09WtLiVHU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=FnU9HIEm; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=Gofj2PH0; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="FnU9HIEm"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="Gofj2PH0" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192576; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=kz/3QoaXGDemZuo9qs/jsQY7mExo5lZjuRcNC3I0rZE=; b=FnU9HIEm0Sr+ykFRBv+KUQl/jcTU8LDDUe7vhKIIcRUygKzkSnlXavAGyV6RaldDPFYxBj vW22YWN3OmomGUAoBcuFinF3lCQY7mPKolm9y47a60XjTp+8vTm8izVU0cIN+y3F69jmY4 CD2lB5dRsTBHuGN1HfiAm9PukHm1Sug= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-278-WoctFcy8MWCCII9g1jm3uw-1; Thu, 07 May 2026 18:22:55 -0400 X-MC-Unique: WoctFcy8MWCCII9g1jm3uw-1 X-Mimecast-MFC-AGG-ID: WoctFcy8MWCCII9g1jm3uw_1778192574 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-44bf1ac8893so1115486f8f.0 for ; Thu, 07 May 2026 15:22:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192574; x=1778797374; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=kz/3QoaXGDemZuo9qs/jsQY7mExo5lZjuRcNC3I0rZE=; b=Gofj2PH0S2ii0AlgD4B3/tJrEucD7f5saEHG1CosKfpQxzjvHI32Yx0YwJLS0R53OI 0vP7G6VPu1aKpuysE1Ri6xYuC4luTHEuwKtGEYxOhGKNwi1+kxvogehryHZWRzsbW3Jt 8WqORuq4pexrpQLWXhknQgN/iNq11EuFivPgo+h2yh+EvptZJrQELOQYiOA2OdyosWLE JUJlmb4d8rwGg9qwiSpNKTt3ZtfvpWit+VD4LRYJ9wdcSRoMYzp9+0MuMwHLsB/Qdius 933yN8ZNI59dWxdRzO+pu44jL12/OImy2nQs3bOwd+DCSLGOM/LZxbATxRNBOAw6CtgW nSjA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192574; x=1778797374; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kz/3QoaXGDemZuo9qs/jsQY7mExo5lZjuRcNC3I0rZE=; b=Ii3i9kKApTLDDviazEQgC74W9ksknOI6cO30VJrBvgbuIfWJXyZeElzPI//KTUUN0G c7tv4vpoPlOjU/u8t3ngxAyqiQqCVf2pJ2dfDQh9pW+AIrqZTAoAW+bx/d70DSmiubVp 2/+xCIfwwxdEYqeThsMWR4CFBqzmIWrdFh8/cVx0BAT8pFlwZZbDJXou95UsuxbqnJDO ZCp9pFPStOI/d+CeTdg3jg1YlVlSdluLO+LrZFXrY5QVrzcgpkwSTDfUFCMciY9lIlp5 AaxBoyfgUVEnG/BuKATXKhwPR5iMKo8ry4OXfGspadVx4MjkxXxotGDpe3syxQ6jwUVO gzgA== X-Gm-Message-State: AOJu0Yztrsn9dQmBwTImzYDQ1ICLyuH1vgUAp/QDqv6KKK3ZtKtFHzYR pbCEIWTf9pACSg5VkN/ShJoFR+Kb1ZpW/lPc9AoPsO/RS3BuxmLm8PAL6vkntzj+5E8Vju7/KnP +n2Ud+N+LUIdiRP6c92CKhUt2f/40dA3ScR9hM5U5eQ0GCFVgU5vIYEOf2PSC2rTyyntsg09N3J rQka5rv87xmqc4GinE3qslaJVnIIvkbJcr6FYYgp/nwNQ= X-Gm-Gg: AeBDieuwy6z8eqRCE0D+evrPYk5ffI9nwUNLzmRh+OnTNIE1IZJ5G0bICPbieRH1Ppb 6E/NTIR2igvaRsPCN7upC+0jn8fOPlrk4Wldo+mxB6r7pogsFXi9d6YGG3U6bOkweFsLYUyJQpC 6nB75qi563mjI6W1aBlMobj+XVEXuWTRlyKHGKsCBHgBCCYqvdHrX67EGBxOnc3GP/35qXEKRHy uM/lo710z3ldts4x0mb6xPEc8XPD9mDXfpyAbhjPYyejtaTGQnZ9huDuEI2TmcnyLTeJ89HjNqy tKfFGq8tw/SAtutO5R9O4cJ9s06iTkBZGUEhx/BRX9CazKbLjb2m1fj6rqm2FbAr07G1QQHkz9D Ng24p0e+Yoa8I9jvMSNBu4CwHqqiuJ8/1rG0y9Vwf X-Received: by 2002:a05:600c:c094:b0:488:b14f:b8ed with SMTP id 5b1f17b1804b1-48e51dd89ffmr116909415e9.0.1778192573758; Thu, 07 May 2026 15:22:53 -0700 (PDT) X-Received: by 2002:a05:600c:c094:b0:488:b14f:b8ed with SMTP id 5b1f17b1804b1-48e51dd89ffmr116908985e9.0.1778192573187; Thu, 07 May 2026 15:22:53 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-454128207ebsm1463589f8f.7.2026.05.07.15.22.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:22:52 -0700 (PDT) Date: Thu, 7 May 2026 18:22:50 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , linux-mm@kvack.org Subject: [PATCH v5 08/28] mm: use __GFP_ZERO in alloc_anon_folio Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Convert alloc_anon_folio() to pass __GFP_ZERO instead of zeroing at the callsite. post_alloc_hook uses the fault address passed through vma_alloc_folio for cache-friendly zeroing. Also convert alloc_swap_folio() to pass the raw fault address for the same cache-friendly zeroing benefit on swap-in. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/memory.c | 11 +---------- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 70ab8b3e3a29..bb24a14d97c2 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5175,7 +5175,7 @@ static struct folio *alloc_anon_folio(struct vm_fault= *vmf) goto fallback; =20 /* Try allocating the highest of the remaining orders. */ - gfp =3D vma_thp_gfp_mask(vma); + gfp =3D vma_thp_gfp_mask(vma) | __GFP_ZERO; while (orders) { folio =3D vma_alloc_folio(gfp, order, vma, vmf->address); if (folio) { @@ -5185,15 +5185,6 @@ static struct folio *alloc_anon_folio(struct vm_faul= t *vmf) goto next; } folio_throttle_swaprate(folio, gfp); - /* - * When a folio is not zeroed during allocation - * (__GFP_ZERO not used) or user folios require special - * handling, folio_zero_user() is used to make sure - * that the page corresponding to the faulting address - * will be hot in the cache after zeroing. - */ - if (user_alloc_needs_zeroing()) - folio_zero_user(folio, vmf->address); return folio; } next: --=20 MST From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6B90939EF25 for ; Thu, 7 May 2026 22:23:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192581; cv=none; b=RbabioclzAtVIkmPNSLcZX0ow3zndpGjMq0umoH/CWlkZfSCD4mSZXuTeIAUTSkaX4d/jVPbbRLqJV9ltZI56J27AHNB2R9e+sn68Axlh7bs1YRgjIs50dV5FUNmwVt2LlFl2z0WuRABP/170zaAgLAGTLNlv+8XQ2znts6FRME= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192581; c=relaxed/simple; bh=ta8h6WWIgHM6wr6R6uVViBDGa15PaaSe6wmcskt5PZM=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=G9u4SxSvHXMQrBwRIKgWKO6JDMysSv9z3CQIg8hzwRN4BMSeY0i3M5G8pATBIe1tm8UvuhjZFijmhC2RjkqR+zAEXpqXySryFUlfNI6PERVdjLYfHJC4FOw9B8i/8OefZMKqkA4a+8XvLbZT+utB5Xi35bF0H92LYs2vb8Ejmd0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=NiBdX4IM; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=jjyescdR; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="NiBdX4IM"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="jjyescdR" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192579; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=C0iJX4IreXsyXpLqNVxKVCvdOAtYgnPhF7cVJs4Qyr8=; b=NiBdX4IM5Pf+GYm1E6wtIr/xQ54JJB21TjycvLddy3x5Xkh0GC70GS99SBfprVRqB6mtOH P8ExTvQOVsZOFlP9iMMdbZoujvKFiC7XzkG/ylozAiDWmxYFGbeLtmApY8Irutk1e9jbCV szZoL32UkbZ0KdpbSeWypMs5p6h5Vgc= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-574--gd-N5tAOdycYQmWbDOw9w-1; Thu, 07 May 2026 18:22:58 -0400 X-MC-Unique: -gd-N5tAOdycYQmWbDOw9w-1 X-Mimecast-MFC-AGG-ID: -gd-N5tAOdycYQmWbDOw9w_1778192577 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-488f973ddfeso9352285e9.3 for ; Thu, 07 May 2026 15:22:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192576; x=1778797376; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=C0iJX4IreXsyXpLqNVxKVCvdOAtYgnPhF7cVJs4Qyr8=; b=jjyescdRNUcvOppoixWLHtRIXh9EZVQY+IpTpgR9M3Nfkm4FyqitOq/wgydMKyre2+ OQc48pzStoKFYcw9ubs3ssLjOKIwxw4v3HjpFdB3+OKSKdePwn0LSKZnbjPn6Pb1UssB OipNJshyXByQm2RgXggOyCnZJ+RKLSMyOg/X+M+fZRBX23rvr5avSQyDPWjMddCuIXls D04QU3XCg0jTvSd0BK0fcEN/qTpJoVVUXDt63YtAxsA2D/taHRNlTs9TVoZoHwopEWvb qcserUx4TOYy80TuitziNxSLv5bF5gpIapdQjCihI9YKiide7IB5yXfTXy23m3KXEQX+ n9zA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192576; x=1778797376; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=C0iJX4IreXsyXpLqNVxKVCvdOAtYgnPhF7cVJs4Qyr8=; b=BHrl+sVUtSlnBzi+SldjPLvOTj7xGunVgbOWm4yJFpdXUG9eGdR+r2KW+uB8yVy7LE vlm0yfrjmvbK1sYP8AjoESRXcMSPs7tjC5VrynjdvT+M7e7YLqidmQRaTzisK/T4IhLo htAAclf6vnE+Oys5fLyFvmtTsfrXUMHqQKhgb25A3Yn7I32ReJU4SulV/E4Wg5kSUF0t IfaGVyXIi12hXavJ9INChBwOTc/I4Eg3Z38Ikvj5gqg0f24MFraCkCGaeYsqoTpCWbIV JRN50sKviQCRnMe1cuHJv8aBsoRbQlvS7SRTDlbqYCTQp5nGpLjOb2FVJp4UOeBzyHeH hf/g== X-Gm-Message-State: AOJu0YwLZZGLEV3oCfM9O7WAqFYsnxNeJf7s8pN0TV6bDLeM6D/cX4Tf 5eS3+B9SlTdfeFo6bkubwcJnz+qu5En3mLuqdifRv9shcSFwp8W55XmgTSV/DbRi98PqFmXAV/q LMWinpqPai6l6U53CFAT+QtZhka4e3owqcBZCMMnQOqKjR3/roxG7agaaIcFtjtNQ00fybu6j/s gFnLipS1tJ6DJARZCz1QlU/WbQ125zcBiEW4zrcfBC3dw= X-Gm-Gg: AeBDietle3/E+WNn2WwCjeM3PkWoam9QLe0xurJVGbggNuYrXxKHqE1R73Or1KLTjQX V+0sKKXSKSw+rdb9iYjd7opzXMixeROV4m5IvSeEDSbTIs5EmDX5yZWCqS7JyMMsASvSt8XPDCQ OxAcQSpKQm1gX4TWeMIK1QFE7kR/JRFUna1Kp4MJ+m04LYQG1YECVt+4mqphjYwRzzo8hMIecAR vuTYKsCOvn7IIeC4VY47oTosMyCHMUQDdplfH7ljpwD0NQVAe74S1JO8806iJ26hS84kRGXMi/z 44FPS6r7VKA73TVyx5XZ61BYZ+T3nVeKFE23P7ZON3LHxhOFMDtIf7zw/1ZLg+MJhEtWW9ubdAx vzmK7k6bMEK0b9s7giMYR5Ka5FeC8BYnbR9gRSvE+ X-Received: by 2002:a05:600c:35d2:b0:487:4eb:d125 with SMTP id 5b1f17b1804b1-48e51e15589mr167969405e9.9.1778192576314; Thu, 07 May 2026 15:22:56 -0700 (PDT) X-Received: by 2002:a05:600c:35d2:b0:487:4eb:d125 with SMTP id 5b1f17b1804b1-48e51e15589mr167968925e9.9.1778192575750; Thu, 07 May 2026 15:22:55 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-454131bcee3sm2024100f8f.15.2026.05.07.15.22.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:22:55 -0700 (PDT) Date: Thu, 7 May 2026 18:22:53 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , linux-mm@kvack.org Subject: [PATCH v5 09/28] mm: vma_alloc_anon_folio_pmd: pass raw fault address to vma_alloc_folio Message-ID: <96a2f5a938bf65315ac95e75b4e7741bae018cdd.1778192416.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that vma_alloc_folio aligns the address internally, drop the redundant HPAGE_PMD_MASK alignment at the callsite. Signed-off-by: Michael S. Tsirkin --- mm/huge_memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 8e2746ea74ad..f51c0841ce91 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1260,7 +1260,7 @@ static struct folio *vma_alloc_anon_folio_pmd(struct = vm_area_struct *vma, const int order =3D HPAGE_PMD_ORDER; struct folio *folio; =20 - folio =3D vma_alloc_folio(gfp, order, vma, addr & HPAGE_PMD_MASK); + folio =3D vma_alloc_folio(gfp, order, vma, addr); =20 if (unlikely(!folio)) { count_vm_event(THP_FAULT_FALLBACK); --=20 MST From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B9A4D39F172 for ; Thu, 7 May 2026 22:23:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192584; cv=none; b=N4tu+5rCwgv5Fw50PEEvkY0UgfcmKIzlQ6Ov1YK4g/LA1K3gkOvRqKfFoSjiryh8cCIc1yT4Uim/cEEa/JwZTB5Dav5ctvGS8dOeqEtVPPOs6RHqQzrxsvOCkn8MiwAY8fBetTGq/d4yEi23UDbzWVP7jpa/S6jhUSI616lWLPk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192584; c=relaxed/simple; bh=unMz317XBjr12JhqBnnd7akMDs264PznGD/sGQDqeSQ=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=VHYjTT60wTv1mYSY4TdmPEr0GUdyLl5szv+3m+A+ENeijriW0vF6O66Sr8VJJ6o04h3TeU0X5heQfBYUeb2SnZpnmnmMZJPvyf8iZIFjSJGkYUvRCrFG6Zn3TYR73VrqDEfj9bgwNU5bfV0mJh8gabozkKbq89PfsC31AdhLcK4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=AxEl6DEg; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=TD/PRgIZ; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="AxEl6DEg"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="TD/PRgIZ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192581; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=vuT7UcDWAinUs+BER1tBruP3Sa2tRxGHJfBaIl3a77Q=; b=AxEl6DEgw/gXwCAXZOJWl913ynQqo0/3iRtADYpukUaG3e68/+4zEahCqvJfOBMdjsE5q2 FjBPV68fd6gDxXVugEBuaPdz1ilUrZrmijhJxxK/stDYx1hzjFBIVS/xed/GwGvUxQGvqJ RQdaIhr1UsGMfjrZvQi2e5nO4TfeHc0= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-423-T49rQbobNCuh4JCOCKvWqA-1; Thu, 07 May 2026 18:23:00 -0400 X-MC-Unique: T49rQbobNCuh4JCOCKvWqA-1 X-Mimecast-MFC-AGG-ID: T49rQbobNCuh4JCOCKvWqA_1778192579 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-44ffa15dc73so1176539f8f.1 for ; Thu, 07 May 2026 15:23:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192579; x=1778797379; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=vuT7UcDWAinUs+BER1tBruP3Sa2tRxGHJfBaIl3a77Q=; b=TD/PRgIZzJiP6B4subwnqW3HfawB1Hb7o5Xoj9FORrtJ4izab9s3SA72Cky+ftusB3 drnGVJrwVDZWG956K4zQhdOZSD6Hf9BEHnBJqCZX7CInf77fWWvm3ENmHRBdYd18NcQL uXX5jqYV/6OhPF0VXSHcas8Xe6AN/WgrwshWPrYnPzSR3C+iqfoaZZvBxNBDlvx3WbPN YY1xdNiAIOpC3hc9Z/UWJJ0CsRgd5l7WyV1CGYI8KjtswGRoBrdECVYpCvQnfP+KucIm eI8uxqdnQG0zewH+DIHtEyvAH14F5unQdXTq/7gY2EW0zthTMlpDdi0zzllamIb5E2XH otgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192579; x=1778797379; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vuT7UcDWAinUs+BER1tBruP3Sa2tRxGHJfBaIl3a77Q=; b=UjcWSAI/RCxurUyoy+YrL3U7MivDSYKpvwIBQlPkclwwSvtdGIOZUMs+c1jQNznLgI noRMHjz5iApRa0AUlHlGRXv4B/u2gm9Vv1sSw0NDbayVvPg3QcITsdboIJ8NdINT0Jiv +UvChEiTK3fww8LjwJJ2djv8XsebuwNpsKV8kNGLb3XB/BI/ihBCLxLM/WQrpec8rD+3 JD76IYA0VPjovsk27FFIqTD7nG1kiLUsR/XiXi7Sf0uj9tiT6i1ffzhosr1yV7ebfmzp lnyBgHoQl7/lqBSp+SNHmsx6xw77C4pcLe4Ar3OJ2rMXi2VP4wBt6f3WzP8j+g3bwRMa oqQA== X-Gm-Message-State: AOJu0Yyxy1J5qjHLk0k01N5XkYhGA8VJ3ayxKg3WYJQHNLeipftOxDKo 81T8+2TbkNOs2n4tCtI4+hvQSmox/uiVRZsqmdmz7bjeDWNofApz5LkIIhMwhfioel+vIHNUWC0 oxR9+F1myeviql7Cn1gozWAVAwuDp32EaiAYL7QO7tGoR91cO6PCieZkgoAK35z/XLxVaciI5VT uDnbacuWPZ9PAGOHfMbksZvnQoDKgN/QMpBKq5yLZTLHA= X-Gm-Gg: AeBDievHaxDVXfxsR99OUnPTUHVEzk4kg1ZNLlCmcQZjlXY06jr3nnPO0XMFpK2bjsP bcEak+k3ju9mZ7zIeaslIbBAhNyanZsh5VIV9xzmYyS302rU9sioHPygwNNlcjig/7AXHaIvo22 yML0g7NQbhlL/oh1A8mRxSpDiGJMLM+sWqYCDT0AV/OOjr9BG0sTmCWjBLj5H4sF2qUDigvGfzF UbOOsrJI6BXJQ6gk+iyCSOtXXunjMaVn/joSLSOrjInLkIuI08n3ogD1WqY7MD689f5ILM7J+fa 4M8n+Otf8Mndw1wzIa4obeA28h4juwmmUdffpETmSxVtaNkuGI/bX0B6ALPjjfa7sviAR6b4oQj VEmrRo7xnhkmMb1PegufDHGU6yXG1AXKJfFivQrtd X-Received: by 2002:a05:600c:3ba7:b0:48a:7772:c26b with SMTP id 5b1f17b1804b1-48e51f40884mr168625925e9.26.1778192578963; Thu, 07 May 2026 15:22:58 -0700 (PDT) X-Received: by 2002:a05:600c:3ba7:b0:48a:7772:c26b with SMTP id 5b1f17b1804b1-48e51f40884mr168625385e9.26.1778192578402; Thu, 07 May 2026 15:22:58 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48e67349df0sm2936025e9.10.2026.05.07.15.22.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:22:58 -0700 (PDT) Date: Thu, 7 May 2026 18:22:55 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Lorenzo Stoakes , Zi Yan , Baolin Wang , "Liam R. Howlett" , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , linux-mm@kvack.org Subject: [PATCH v5 10/28] mm: use __GFP_ZERO in vma_alloc_anon_folio_pmd Message-ID: <1644850b5868bdeacb02c753aa1d7fb20aca199a.1778192416.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Convert vma_alloc_anon_folio_pmd() to pass __GFP_ZERO instead of zeroing at the callsite. post_alloc_hook uses the fault address passed through vma_alloc_folio for cache-friendly zeroing. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/huge_memory.c | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index f51c0841ce91..3f2a868cf9e9 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1256,7 +1256,7 @@ EXPORT_SYMBOL_GPL(thp_get_unmapped_area); static struct folio *vma_alloc_anon_folio_pmd(struct vm_area_struct *vma, unsigned long addr) { - gfp_t gfp =3D vma_thp_gfp_mask(vma); + gfp_t gfp =3D vma_thp_gfp_mask(vma) | __GFP_ZERO; const int order =3D HPAGE_PMD_ORDER; struct folio *folio; =20 @@ -1279,14 +1279,6 @@ static struct folio *vma_alloc_anon_folio_pmd(struct= vm_area_struct *vma, } folio_throttle_swaprate(folio, gfp); =20 - /* - * When a folio is not zeroed during allocation (__GFP_ZERO not used) - * or user folios require special handling, folio_zero_user() is used to - * make sure that the page corresponding to the faulting address will be - * hot in the cache after zeroing. - */ - if (user_alloc_needs_zeroing()) - folio_zero_user(folio, addr); /* * The memory barrier inside __folio_mark_uptodate makes sure that * folio_zero_user writes become visible before the set_pmd_at() --=20 MST From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1905D3A0EB3 for ; Thu, 7 May 2026 22:23:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192587; cv=none; b=jjdcSGNM24sOWD89KPqdVaI7WnAiDYGh/Nip5MQu72XhibMXLLqbyhRPOApSBJCCt0zd0UbQ3c1BtkLTMrlnR7uDxRRoyZqtYxiDnUUBs711gCP0/zPlWXP98zzYePdgFROx+zHN7m/lV++cWhBsqMZJfIWC4NImQ8cSc/t0m2o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192587; c=relaxed/simple; bh=waG2D4APFI5SI7DiSsGF8Rwi5b0iO3BMaWBrJYKabcQ=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=rUyxB/t74ZAeMr6YGhPA8+H/2sfRNJO1zLWVKLeRkze81FrsBc/Ub1fX5fOZdTLWEvot2VVA1t3qzeSilQG8U9PmPmdO6pMsuSb47RaBlGpgwGE8XuA5H7ifDwZtoMWRzo2g+X0R54IDJqCq9Pr9RGOLVwVfsuWUdxR3SLIDTbE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=BX3dSV+A; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=jqad8bwp; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="BX3dSV+A"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="jqad8bwp" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192585; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=cqwX2bUTal2ZUsYHM753A+5S3P8U5GEJZpwze5+wO1I=; b=BX3dSV+AE0DW5KW8OZpEKwzcR0PuMxLN1CjcypKAPBkkMHVOU77HMZ/oDAN4JJ4WatL51S 3HuNLsAsmLoq7UwGfiIxP0/FTIKdKhqgO4QfSb7DpjPSZd3FIMWM3gF0TnAUA7UOJgxOb8 3aHU064UkFEIoNveyKjC1J01xbVkhZA= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-664-DMV7uuJnOjKS-ENJLD8SuA-1; Thu, 07 May 2026 18:23:03 -0400 X-MC-Unique: DMV7uuJnOjKS-ENJLD8SuA-1 X-Mimecast-MFC-AGG-ID: DMV7uuJnOjKS-ENJLD8SuA_1778192582 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-48d0889c1ecso8934915e9.0 for ; Thu, 07 May 2026 15:23:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192581; x=1778797381; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=cqwX2bUTal2ZUsYHM753A+5S3P8U5GEJZpwze5+wO1I=; b=jqad8bwpNoTJ2ddM2yh+24ATI6AGwFEb17MJwLq7b8VXC84NKzARFbd3PQ0tPyMSXU fa/Z/jGa+lXngIV7IFofBu02YP1ONMJIv1iOQz3Ln0bfYLzLnDwE8DJ6VberAE5oGH1x FW0TTbJ+erTyNXIJdepXz/eFOH8A6rojdOq07PTktIT/joII27GWKjGcmUYbepjWhcaL n4bYNKPSNdjxhPdB238ww97V8qPM67PHTGP5O40+mVLcLCm+hPc/HNvZ9n8DptPiRPdG k3AyXa+aO/i+/QSjYDDEM+KL7dQpidLeSXQI5wl4aWR7jMAKIZo2u/3gXVAOQjI0UOnF 4VSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192581; x=1778797381; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cqwX2bUTal2ZUsYHM753A+5S3P8U5GEJZpwze5+wO1I=; b=eQVHiUq6/tHynMO0yJktoZ55dwSqSs3ApDvt+o1PWpDDqE5IGEj6/jmtbq772v+jdf Qeoq0G781IUi+VCrUPm/UW9lFbkNGJDiiNWBaM22qJu8IWaw4Bsc1JkhcddbrQbLkIRI fa+AANACvDCDkWNvZKPJVWyF0iUUot7pNCUvSJa9CqFYCUtwaH1MXlGoBbDJX/UzRhjm EGprLpHqo1SWDdUSDalaDEWZ5E/arpnXxM6TbOTor9XDVRCGpwsTEeL1gKlayAn4B1Mx osOtgLEAeprg9tdMKRqklQesQGWyvdBglfhuiYkLVyo6TbrFh0CMvYAoaD44zE5B0RxM ECWg== X-Gm-Message-State: AOJu0YxtMFkFesXDcnS8Xdg5y2AIXmOWFah1+91d7oN4izvYDrH202V7 Tfus0SU4EvcJgTgP/o+A4km2RAk6dZR8Wwd3hozGtkicIzbHeL2kDt8gJkCrj3qL5B008Dg4S9R 3wTY17a8qC7ZA1ZAyGPvv2AAuI/yQyy1VaHhfOwsqJfN4qkQV4BGgjb6DZV1cNP6hTWJJ6vZc7H x5n/AMYuR1mrE8pOlNsdDdwRX0cZwGjuIUgk5EFRiKopo= X-Gm-Gg: AeBDievtIKiAs8u69qrEtHgvsdJPYsLc1EZDTVgqRAmNZ4CVP0Rn4WeuEbMIHhjG0mH xjrSjfgpi1VRmxbNMLGHQt+WD+5NHf7eMoJYARCLx9FapPYENYUVYh2gwPZyCvmUGTJb0orJjkD LSJKYJU53mWUBjzRQ67+gNaR3rzXR4LJgeqwXK3hPj1JuLcR4zUucSX25kH7Pmej5BUHJcV7Y2j IoflKHfSiAUV7dhMgv8Uo6i7lEvqFGGe9b1UWvRIi4KnLU9adY3eF71o3+Uamqb8TYKl+yXy7FO 3O4YLnQdwfomu6IZO0jiHg74z6C9q/qw+1Pz4v2i9mNv3jD6w03dFk3igGKuNiMPOIqTWfPa7Ir azSlR1uBEhJQ/06M7Y+N1eYQltwfVNGXNNJq79jxM X-Received: by 2002:a05:600c:1e8a:b0:488:fd7e:1063 with SMTP id 5b1f17b1804b1-48e51f4cdcdmr152031115e9.29.1778192581322; Thu, 07 May 2026 15:23:01 -0700 (PDT) X-Received: by 2002:a05:600c:1e8a:b0:488:fd7e:1063 with SMTP id 5b1f17b1804b1-48e51f4cdcdmr152030715e9.29.1778192580730; Thu, 07 May 2026 15:23:00 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48e67349ddbsm3318515e9.9.2026.05.07.15.22.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:23:00 -0700 (PDT) Date: Thu, 7 May 2026 18:22:58 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Muchun Song , Oscar Salvador , David Hildenbrand , Andrew Morton , linux-mm@kvack.org Subject: [PATCH v5 11/28] mm: hugetlb: use __GFP_ZERO and skip zeroing for zeroed pages Message-ID: <23cda0a1cc4103c3f5f187749d5b5b2934d85821.1778192416.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Convert the hugetlb fault and fallocate paths to use __GFP_ZERO. For pages allocated from the buddy allocator, post_alloc_hook() handles zeroing. Hugetlb surplus pages need special handling because they can be pre-allocated into the pool during mmap (by hugetlb_acct_memory) before any page fault. Pool pages are kept around and may need zeroing long after buddy allocation, so a buddy-level zeroed hint (consumed at allocation time) cannot track their state. Add a bool *zeroed output parameter to alloc_hugetlb_folio() so callers know whether the page needs zeroing. Buddy-allocated pages are always zeroed (zeroed by post_alloc_hook). Pool pages use a new HPG_zeroed flag to track whether the page is known-zero (freshly buddy-allocated, never mapped to userspace). The flag is set in alloc_surplus_hugetlb_folio() after buddy allocation and cleared in free_huge_folio() when a user-mapped page returns to the pool. Callers that do not need zeroing (CoW, migration) pass NULL for zeroed and 0 for gfp. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- fs/hugetlbfs/inode.c | 10 ++++++-- include/linux/hugetlb.h | 8 +++++-- mm/hugetlb.c | 52 ++++++++++++++++++++++++++++++----------- 3 files changed, 53 insertions(+), 17 deletions(-) diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c index 3f70c47981de..d5d570d6eff4 100644 --- a/fs/hugetlbfs/inode.c +++ b/fs/hugetlbfs/inode.c @@ -822,14 +822,20 @@ static long hugetlbfs_fallocate(struct file *file, in= t mode, loff_t offset, * folios in these areas, we need to consume the reserves * to keep reservation accounting consistent. */ - folio =3D alloc_hugetlb_folio(&pseudo_vma, addr, false); + { + bool zeroed; + + folio =3D alloc_hugetlb_folio(&pseudo_vma, addr, false, + __GFP_ZERO, &zeroed); if (IS_ERR(folio)) { mutex_unlock(&hugetlb_fault_mutex_table[hash]); error =3D PTR_ERR(folio); goto out; } - folio_zero_user(folio, addr); + if (!zeroed) + folio_zero_user(folio, addr); __folio_mark_uptodate(folio); + } error =3D hugetlb_add_to_page_cache(folio, mapping, index); if (unlikely(error)) { restore_reserve_on_error(h, &pseudo_vma, addr, folio); diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 65910437be1c..094714c607f9 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -598,6 +598,7 @@ enum hugetlb_page_flags { HPG_vmemmap_optimized, HPG_raw_hwp_unreliable, HPG_cma, + HPG_zeroed, __NR_HPAGEFLAGS, }; =20 @@ -658,6 +659,7 @@ HPAGEFLAG(Freed, freed) HPAGEFLAG(VmemmapOptimized, vmemmap_optimized) HPAGEFLAG(RawHwpUnreliable, raw_hwp_unreliable) HPAGEFLAG(Cma, cma) +HPAGEFLAG(Zeroed, zeroed) =20 #ifdef CONFIG_HUGETLB_PAGE =20 @@ -705,7 +707,8 @@ int isolate_or_dissolve_huge_folio(struct folio *folio,= struct list_head *list); int replace_free_hugepage_folios(unsigned long start_pfn, unsigned long en= d_pfn); void wait_for_freed_hugetlb_folios(void); struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, - unsigned long addr, bool cow_from_owner); + unsigned long addr, bool cow_from_owner, + gfp_t gfp, bool *zeroed); struct folio *alloc_hugetlb_folio_nodemask(struct hstate *h, int preferred= _nid, nodemask_t *nmask, gfp_t gfp_mask, bool allow_alloc_fallback); @@ -1117,7 +1120,8 @@ static inline void wait_for_freed_hugetlb_folios(void) =20 static inline struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, unsigned long addr, - bool cow_from_owner) + bool cow_from_owner, + gfp_t gfp, bool *zeroed) { return NULL; } diff --git a/mm/hugetlb.c b/mm/hugetlb.c index de8361b503d2..b5bc2a9f5022 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1744,6 +1744,9 @@ void free_huge_folio(struct folio *folio) int nid =3D folio_nid(folio); struct hugepage_subpool *spool =3D hugetlb_folio_subpool(folio); bool restore_reserve; + + /* Page was mapped to userspace; no longer known-zero */ + folio_clear_hugetlb_zeroed(folio); unsigned long flags; =20 VM_BUG_ON_FOLIO(folio_ref_count(folio), folio); @@ -2146,6 +2149,10 @@ static struct folio *alloc_surplus_hugetlb_folio(str= uct hstate *h, if (!folio) return NULL; =20 + /* Mark as known-zero only if __GFP_ZERO was requested */ + if (gfp_mask & __GFP_ZERO) + folio_set_hugetlb_zeroed(folio); + spin_lock_irq(&hugetlb_lock); /* * nr_huge_pages needs to be adjusted within the same lock cycle @@ -2209,11 +2216,11 @@ static struct folio *alloc_migrate_hugetlb_folio(st= ruct hstate *h, gfp_t gfp_mas */ static struct folio *alloc_buddy_hugetlb_folio_with_mpol(struct hstate *h, - struct vm_area_struct *vma, unsigned long addr) + struct vm_area_struct *vma, unsigned long addr, gfp_t gfp) { struct folio *folio =3D NULL; struct mempolicy *mpol; - gfp_t gfp_mask =3D htlb_alloc_mask(h); + gfp_t gfp_mask =3D htlb_alloc_mask(h) | gfp; int nid; nodemask_t *nodemask; =20 @@ -2910,7 +2917,8 @@ typedef enum { * When it's set, the allocation will bypass all vma level reservations. */ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, - unsigned long addr, bool cow_from_owner) + unsigned long addr, bool cow_from_owner, + gfp_t gfp, bool *zeroed) { struct hugepage_subpool *spool =3D subpool_vma(vma); struct hstate *h =3D hstate_vma(vma); @@ -2919,7 +2927,9 @@ struct folio *alloc_hugetlb_folio(struct vm_area_stru= ct *vma, map_chg_state map_chg; int ret, idx; struct hugetlb_cgroup *h_cg =3D NULL; - gfp_t gfp =3D htlb_alloc_mask(h) | __GFP_RETRY_MAYFAIL; + bool from_pool; + + gfp |=3D htlb_alloc_mask(h) | __GFP_RETRY_MAYFAIL; =20 idx =3D hstate_index(h); =20 @@ -2987,13 +2997,15 @@ struct folio *alloc_hugetlb_folio(struct vm_area_st= ruct *vma, folio =3D dequeue_hugetlb_folio_vma(h, vma, addr, gbl_chg); if (!folio) { spin_unlock_irq(&hugetlb_lock); - folio =3D alloc_buddy_hugetlb_folio_with_mpol(h, vma, addr); + folio =3D alloc_buddy_hugetlb_folio_with_mpol(h, vma, addr, gfp); if (!folio) goto out_uncharge_cgroup; spin_lock_irq(&hugetlb_lock); list_add(&folio->lru, &h->hugepage_activelist); folio_ref_unfreeze(folio, 1); - /* Fall through */ + from_pool =3D false; + } else { + from_pool =3D true; } =20 /* @@ -3016,6 +3028,14 @@ struct folio *alloc_hugetlb_folio(struct vm_area_str= uct *vma, =20 spin_unlock_irq(&hugetlb_lock); =20 + if (zeroed) { + if (from_pool) + *zeroed =3D folio_test_hugetlb_zeroed(folio); + else + *zeroed =3D true; /* buddy-allocated, zeroed by post_alloc_hook */ + folio_clear_hugetlb_zeroed(folio); + } + hugetlb_set_folio_subpool(folio, spool); =20 if (map_chg !=3D MAP_CHG_ENFORCED) { @@ -5004,7 +5024,7 @@ int copy_hugetlb_page_range(struct mm_struct *dst, st= ruct mm_struct *src, spin_unlock(src_ptl); spin_unlock(dst_ptl); /* Do not use reserve as it's private owned */ - new_folio =3D alloc_hugetlb_folio(dst_vma, addr, false); + new_folio =3D alloc_hugetlb_folio(dst_vma, addr, false, 0, NULL); if (IS_ERR(new_folio)) { folio_put(pte_folio); ret =3D PTR_ERR(new_folio); @@ -5533,7 +5553,7 @@ static vm_fault_t hugetlb_wp(struct vm_fault *vmf) * be acquired again before returning to the caller, as expected. */ spin_unlock(vmf->ptl); - new_folio =3D alloc_hugetlb_folio(vma, vmf->address, cow_from_owner); + new_folio =3D alloc_hugetlb_folio(vma, vmf->address, cow_from_owner, 0, N= ULL); =20 if (IS_ERR(new_folio)) { /* @@ -5727,7 +5747,7 @@ static vm_fault_t hugetlb_no_page(struct address_spac= e *mapping, struct vm_fault *vmf) { u32 hash =3D hugetlb_fault_mutex_hash(mapping, vmf->pgoff); - bool new_folio, new_anon_folio =3D false; + bool new_folio, new_anon_folio =3D false, zeroed; struct vm_area_struct *vma =3D vmf->vma; struct mm_struct *mm =3D vma->vm_mm; struct hstate *h =3D hstate_vma(vma); @@ -5793,7 +5813,8 @@ static vm_fault_t hugetlb_no_page(struct address_spac= e *mapping, goto out; } =20 - folio =3D alloc_hugetlb_folio(vma, vmf->address, false); + folio =3D alloc_hugetlb_folio(vma, vmf->address, false, + __GFP_ZERO, &zeroed); if (IS_ERR(folio)) { /* * Returning error will result in faulting task being @@ -5813,7 +5834,12 @@ static vm_fault_t hugetlb_no_page(struct address_spa= ce *mapping, ret =3D 0; goto out; } - folio_zero_user(folio, vmf->real_address); + /* + * Buddy-allocated pages are zeroed in post_alloc_hook(). + * Pool pages bypass the allocator, zero them here. + */ + if (!zeroed) + folio_zero_user(folio, vmf->real_address); __folio_mark_uptodate(folio); new_folio =3D true; =20 @@ -6252,7 +6278,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, goto out; } =20 - folio =3D alloc_hugetlb_folio(dst_vma, dst_addr, false); + folio =3D alloc_hugetlb_folio(dst_vma, dst_addr, false, 0, NULL); if (IS_ERR(folio)) { pte_t *actual_pte =3D hugetlb_walk(dst_vma, dst_addr, PMD_SIZE); if (actual_pte) { @@ -6299,7 +6325,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, goto out; } =20 - folio =3D alloc_hugetlb_folio(dst_vma, dst_addr, false); + folio =3D alloc_hugetlb_folio(dst_vma, dst_addr, false, 0, NULL); if (IS_ERR(folio)) { folio_put(*foliop); ret =3D -ENOMEM; --=20 MST From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C84B039B481 for ; Thu, 7 May 2026 22:23:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192595; cv=none; b=aBiEdGIg/LDtBkBcuXlv5QwpDDfRkd0bdGbtFHIgJZ1pU+B0CHm1REzGpQhTHzeooolMQ7pkG/qYQ63dq71vkSeSBG1s4LjzcGYUFk3TI93qNNB2lIK6P8Tgm+cuPUfQRXIoW37j/mI7qup6GDFljlY17ghnNig/I6hkB5Hi13Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192595; c=relaxed/simple; bh=VEpka7eQ+qjoP3Y8krH9qirwfLCY6yv7XpuwQILKeLY=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=RFRjXm/ilU4iR5mt/i6CVVGN4BZ9/jEldJlkH29DOc0xJp5APuZ4BJ45drjjkNZs09NN20+e5Waa+npfWRnHf/NnRAj00Pud/2lr95Flh/RQs23FpbkUW6Lq9lObnd4HaOqpE96diMvuH7DKoYBWvZ72ZKrlmCLJ4EtXWQfi54Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=c8ySwf06; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=nEJ3rc81; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="c8ySwf06"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="nEJ3rc81" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192591; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ZzA2kE2SxyD+isdRkzcnXWm71tDaAigOtuMMAbID/Ow=; b=c8ySwf06LSTelCYmicfB/HpWzG+Y5SWh5ljqrKcKa/pQ1GNGqxfndej9wSRdnxrqbsIqyB 8F8Dh0L99GnoM9tfqXfgaFyH5Tfc9gi2MP7s2O4QY7XCk+tKtOCt/x3bnM3GObRM2om/g5 wfhIj37u+zMGi61qW7R3nf744zvwXjc= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-568-z60fbatVOkSfhmj0p7C4iA-1; Thu, 07 May 2026 18:23:06 -0400 X-MC-Unique: z60fbatVOkSfhmj0p7C4iA-1 X-Mimecast-MFC-AGG-ID: z60fbatVOkSfhmj0p7C4iA_1778192584 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-44d68ed8f95so912741f8f.1 for ; Thu, 07 May 2026 15:23:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192584; x=1778797384; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=ZzA2kE2SxyD+isdRkzcnXWm71tDaAigOtuMMAbID/Ow=; b=nEJ3rc81wZu4Cyol7nr/1UZroRJHDQevxRROxzXXa3sAe+RVe4bQ1Zsm93ZrOcq7xy oO/rQmLsxu0ANXNW56w1sHyAOest7YyUVwW2ngpRYOnjcc+EQ15o4DUDrL+JuREz3CXF UFI3EgAl+hW7O4ZhZzP1DGn/CYIG9NF1j7Xd2OCc31UIPoxsVwW07Cw0LoGWbuIlMjRc JjHWVKO8sZHApREmd6CZDPrtTMRG+Zt/Gre+xM1vAWlKIobqQBHtXCrsQXbqB1Gnxijb 5rPk1+EkHyjpkMl/wQJWe5k5GBh4mcClzOapB0/jgWiUKmbyQBE9KhcKPY2rbFMKTJdt W3Hw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192584; x=1778797384; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZzA2kE2SxyD+isdRkzcnXWm71tDaAigOtuMMAbID/Ow=; b=G65hPO6Bsg12C1IUIp/t4gFwEM7qNzy7GSCqSrvq9I6SCA1M38Z/vn1L2TC+bh4XtB eHL/PCdKpN8i5GxmGNO6qWmTC0xgsSUc3x76IrL3GIeH2cIUhjHJGxHmxCPsGVVm+YK6 JoUyMq3Le6F9fQjbVfL4/wGpkI8jl28IDMB7hh30aZlQD1nJMnyFrBBONbpgdwPEQ3Ot OQJ8aXrqCL8tf1I0bKQMdEQzzk7x16k5ONkQZahcScV6vG+yZaqgfUrszmxDyunhjdew /sF1K5Q2M9HnNGKIEXv7YdSxSrCL7X0nLW46nK9n4jSLtr51xJep9JOmGFhY2ws+BCKx hk0g== X-Gm-Message-State: AOJu0Yw9SyLYloKvBGbtu6/4fR1s9WDmxIpVm/QBImeOU4ULHfEzVK6s EHKQYZ8mVpxcGanJg1hKalIlVL2aIySXe2Pl5tbcNc3TyyKQLRioP2ZWNtkeaI94OQYVBTTG5II 6nRUnPup1Z59bO78Uxv5OKqODMVyw23jRAe/lunT/CgLLATLkJkyOLs5zY/bFwABvU/m3lGAixO YoP9aOZ4mlsWv2NbZlkXYJWcBI8akCH76ukbWqBnt7jK0= X-Gm-Gg: Acq92OFN6ZK1IhGuFnoyjg5lSJEGvssHAD7CZvPeCqgpx8DfnW+hopWLAijGhKUaw02 ytH83vKWoUcfwxMojYXKBLyYGmS/70eMPwmBmWkUEocMstuRzRMa2GTc0aK+RFD70oP3Or9V9Y8 Me3E84YCrJyfne2KQBhbNwpth1Q9//WhK5Ee4D/X2bDvvbuUAcpwCB8BG2LxbQ2qkCIPQwWP5xu X4UVVSl1Av1j/RksZ08wiqKSn04+QJtJJaWcbyJa/kOaNLh94pv6UfESMGG83SJUNqny6JnYqWh eIpoLvxP3p+WBeb1YwX4iPoLqFEal0JO5xsRiPJmT93XhPOlSrpiE5fpc1suPvgENyCfxGIawty eoQoYbnZeglc2U1XjR+Qt0x79mG5GtewxDPkgWjMv X-Received: by 2002:a5d:5e01:0:b0:43e:531f:720c with SMTP id ffacd0b85a97d-452e96d51bbmr7220463f8f.11.1778192583675; Thu, 07 May 2026 15:23:03 -0700 (PDT) X-Received: by 2002:a5d:5e01:0:b0:43e:531f:720c with SMTP id ffacd0b85a97d-452e96d51bbmr7220413f8f.11.1778192582995; Thu, 07 May 2026 15:23:02 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4541721bab8sm1995822f8f.27.2026.05.07.15.23.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:23:02 -0700 (PDT) Date: Thu, 7 May 2026 18:23:00 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Muchun Song , Oscar Salvador , David Hildenbrand , Andrew Morton , Hugh Dickins , Baolin Wang , linux-mm@kvack.org Subject: [PATCH v5 12/28] mm: memfd: skip zeroing for zeroed hugetlb pool pages Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" gather_surplus_pages() pre-allocates hugetlb pages into the pool during mmap. Pass __GFP_ZERO so these pages are zeroed by the buddy allocator, and HPG_zeroed is set by alloc_surplus_hugetlb_folio. Add bool *zeroed output to alloc_hugetlb_folio_reserve() so callers can check whether the pool page is known-zero. memfd's memfd_alloc_folio() uses this to skip the explicit folio_zero_user() when the page is already zero. This avoids redundant zeroing for memfd hugetlb pages that were pre-allocated into the pool and never mapped to userspace. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/hugetlb.h | 6 ++++-- mm/hugetlb.c | 11 +++++++++-- mm/memfd.c | 14 ++++++++------ 3 files changed, 21 insertions(+), 10 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 094714c607f9..93bb06a33f57 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -713,7 +713,8 @@ struct folio *alloc_hugetlb_folio_nodemask(struct hstat= e *h, int preferred_nid, nodemask_t *nmask, gfp_t gfp_mask, bool allow_alloc_fallback); struct folio *alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_= nid, - nodemask_t *nmask, gfp_t gfp_mask); + nodemask_t *nmask, gfp_t gfp_mask, + bool *zeroed); =20 int hugetlb_add_to_page_cache(struct folio *folio, struct address_space *m= apping, pgoff_t idx); @@ -1128,7 +1129,8 @@ static inline struct folio *alloc_hugetlb_folio(struc= t vm_area_struct *vma, =20 static inline struct folio * alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_nid, - nodemask_t *nmask, gfp_t gfp_mask) + nodemask_t *nmask, gfp_t gfp_mask, + bool *zeroed) { return NULL; } diff --git a/mm/hugetlb.c b/mm/hugetlb.c index b5bc2a9f5022..ad536d7aee59 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -2241,7 +2241,7 @@ struct folio *alloc_buddy_hugetlb_folio_with_mpol(str= uct hstate *h, } =20 struct folio *alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_= nid, - nodemask_t *nmask, gfp_t gfp_mask) + nodemask_t *nmask, gfp_t gfp_mask, bool *zeroed) { struct folio *folio; =20 @@ -2257,6 +2257,12 @@ struct folio *alloc_hugetlb_folio_reserve(struct hst= ate *h, int preferred_nid, h->resv_huge_pages--; =20 spin_unlock_irq(&hugetlb_lock); + + if (zeroed && folio) { + *zeroed =3D folio_test_hugetlb_zeroed(folio); + folio_clear_hugetlb_zeroed(folio); + } + return folio; } =20 @@ -2341,7 +2347,8 @@ static int gather_surplus_pages(struct hstate *h, lon= g delta) * It is okay to use NUMA_NO_NODE because we use numa_mem_id() * down the road to pick the current node if that is the case. */ - folio =3D alloc_surplus_hugetlb_folio(h, htlb_alloc_mask(h), + folio =3D alloc_surplus_hugetlb_folio(h, + htlb_alloc_mask(h) | __GFP_ZERO, NUMA_NO_NODE, &alloc_nodemask, USER_ADDR_NONE); if (!folio) { diff --git a/mm/memfd.c b/mm/memfd.c index 919c2a53eb96..4026fda71762 100644 --- a/mm/memfd.c +++ b/mm/memfd.c @@ -69,6 +69,7 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgoff= _t idx) #ifdef CONFIG_HUGETLB_PAGE struct folio *folio; gfp_t gfp_mask; + bool zeroed; =20 if (is_file_hugepages(memfd)) { /* @@ -93,17 +94,18 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgo= ff_t idx) folio =3D alloc_hugetlb_folio_reserve(h, numa_node_id(), NULL, - gfp_mask); + gfp_mask, + &zeroed); if (folio) { u32 hash; =20 /* - * Zero the folio to prevent information leaks to userspace. - * Use folio_zero_user() which is optimized for huge/gigantic - * pages. Pass 0 as addr_hint since this is not a faulting path - * and we don't have a user virtual address yet. + * Zero the folio to prevent information leaks to + * userspace. Skip if the pool page is known-zero + * (HPG_zeroed set during pool pre-allocation). */ - folio_zero_user(folio, 0); + if (!zeroed) + folio_zero_user(folio, 0); =20 /* * Mark the folio uptodate before adding to page cache, --=20 MST From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E008C39A071 for ; Thu, 7 May 2026 22:23:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192592; cv=none; b=Cdrxcy/Jw5mbF0cJAWU6vrTu4CiYRNUnSTClNRZZ+fzN01fevbNvwycgwmZ1Ekii6k04adOW1NyvtbcpQphSHpa62wU6fT3L4AGzxtxnV5V18XwaR3fUQGHVGTK64OYISGAZ0cz3P+o/jAizSCv4U78HXfEfc0yygEeUT7MMv4w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192592; c=relaxed/simple; bh=o+vhOTjLhvS47fv9tom+Yt3TM9vgXV0mo+tAnh6qa+g=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=dwwOe18090XlMG0/laa11N/mexY+HafouvgBBZgGptEKq0ZK/IDAOHji+FkettjVgDLctED3E6wGqOfl3kJLi3+qg6a74YjK5iZD5BzNt/f1iSi543RXgeh++27Q3OYUnfgrcqgSUVGjMAeigjw5EsinxEfUgkyNoEmK8FyhMqA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Kxn/7SRT; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=K+cB4XwQ; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Kxn/7SRT"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="K+cB4XwQ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192589; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=KxxPeFo8b5YQQgq6okT6k4ODeRwXIuGSW0SUh4EUnR0=; b=Kxn/7SRTVg+1y11F3weI900bT53eSmNN/IZfbKom9F+w5LBFqwnxAZPo/9elyIOm0uLSHt l5UsvJQOFSWe3Lvc2eP45z6M5dMvMVHukBk449fL93SJlH+IfI3MQXZ87/qtW8ve4lRj6X Vs8j6xQ9hCtOdYbzFjbcDUAuH3WKYcM= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-638-0M7-v7H8Psui1KY78wyhPA-1; Thu, 07 May 2026 18:23:08 -0400 X-MC-Unique: 0M7-v7H8Psui1KY78wyhPA-1 X-Mimecast-MFC-AGG-ID: 0M7-v7H8Psui1KY78wyhPA_1778192587 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-441243ba35fso1209856f8f.0 for ; Thu, 07 May 2026 15:23:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192587; x=1778797387; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=KxxPeFo8b5YQQgq6okT6k4ODeRwXIuGSW0SUh4EUnR0=; b=K+cB4XwQKShvaqBdKcldswwmuDkaf431z7P2oI5KuH9wWoQw7UVnkM4nmfcRQNE69P xAIVztalsE7+yaC2+YoHpT7wM7dInlVJWIx6VN3SUz7tly8plrFGTfz78Bn6UuUKEckT 05tWgrgdoNqrwkO8nTgkkbGPpKSwLkQHpZbPCYb5RIxim5FjXilDYT5G7fKHFdE6g9fW etuixKtwd6ZUJAu6MmUYJrIpHm10FtQAyqXqK+Px0/Pc25wzmxZ/6AYP8QzgHn4EN783 HKUg+ocLt318X5k88aEAFhm9CxXj1oCpT/AvgmcyoC6ayHxl1vRHodMOibhGrehcN157 p5Wg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192587; x=1778797387; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KxxPeFo8b5YQQgq6okT6k4ODeRwXIuGSW0SUh4EUnR0=; b=HiyPDu4NL5jCDeAJ9fXqw+q84wHjCmAZr01YRDGTagfj80toP5C1aKGEyRCWLmcBxz JwllZusM473adV68YogauzKLsLr6/KYs1YpPT/lY7NvYxhLt6pnq+kXp3+Yk6NYv/9nu tZguari48pqfXuj8a+XlBSFUJpLNclywjd4q1WiDa+nHyYOrMZGTLG/N8WiogaZ8Feka 5Nt1dUzKpLyT3d4gswB/aIuCBjPD9pgI5kuv6QIr6zHPPPuVjtSbeEMxB48+Tg5rsFYe gSv0CNrF98u81tv1c8+CCZqHEU+LM8o2Zis6nJ0qVdurT82MicIGl880h13Q2M/XUiBs A0eQ== X-Gm-Message-State: AOJu0Yx0z3BMc94/6ayFyf2yEMHIWBj/nhN5cKgswvQwDbTn1z+P4ytD qOAIiOn/3zzgFI8YadzT+Pkn2rIzyh9CipxesIg6tS59yy9340FRjyd+ncwLNoCME9XrInuxJZW PQGLJ3Au9eexPJqm0+7uy3rU5P09Ec/MP1r33WpopoKpNK+ABd7HyQHykqBJd6eU9/9S/B2DAVT qqoRyTm/sDhnUKe9oersNuJaTKJ1FbvaikqrWY/MRNYLw= X-Gm-Gg: AeBDieuAu7JazqoY+W3QBILeg1ojXN+x6qCvdbh5OspHEGThBeG32EVqY/RzYsNiCDV G+NxyaGNdhOisRPu8DmWUSwN9R1M2lpiB4J1I4XI972PrfXB6pUVo7mL+tUYHnBzGdPv1BK6bAX wT/8lLqEY9MJdUME0OYOSWsgwcvzqAyEpyqpc0KJau2gbgsJGIhUJLlkn1upMaQt10jVWol+a8U 9nxQGW+UgpTk0T2uM5BudoIhkiUtgQ9hyyeVzsflDXZxsFHdjIX2wVLyeeoLrZHBVnvQnrYn4r8 zsW82wsJy6gxhflEKI5o28rE2cw/wPqv20tOXhRFdhppxgzPamRpeWQCdLaQsB017hElpDBD3IY qom+40rWK0qFemn++gywI3cQnEmM/n7dMP1CIiQF8 X-Received: by 2002:a05:600c:1c0f:b0:48a:5342:36b5 with SMTP id 5b1f17b1804b1-48e51f3deddmr177389185e9.21.1778192586834; Thu, 07 May 2026 15:23:06 -0700 (PDT) X-Received: by 2002:a05:600c:1c0f:b0:48a:5342:36b5 with SMTP id 5b1f17b1804b1-48e51f3deddmr177388435e9.21.1778192586184; Thu, 07 May 2026 15:23:06 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45412820303sm2289369f8f.4.2026.05.07.15.23.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:23:05 -0700 (PDT) Date: Thu, 7 May 2026 18:23:03 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: David Hildenbrand , Magnus Lindholm , Greg Ungerer , Geert Uytterhoeven , Richard Henderson , Matt Turner , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , linux-alpha@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-s390@vger.kernel.org Subject: [PATCH v5 13/28] mm: remove arch vma_alloc_zeroed_movable_folio overrides Message-ID: <44d46ba982b39e53a2b5f978dd7239836e535196.1778192416.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that the generic vma_alloc_zeroed_movable_folio() uses __GFP_ZERO, the arch-specific macros on alpha, m68k, s390, and x86 that did the same thing are redundant. Remove them. arm64 is not affected: it has a real function override that handles MTE tag zeroing, not just __GFP_ZERO. Suggested-by: David Hildenbrand Acked-by: Magnus Lindholm Acked-by: Greg Ungerer Acked-by: Geert Uytterhoeven # m68k Signed-off-by: Michael S. Tsirkin --- arch/alpha/include/asm/page.h | 3 --- arch/m68k/include/asm/page_no.h | 3 --- arch/s390/include/asm/page.h | 3 --- arch/x86/include/asm/page.h | 3 --- 4 files changed, 12 deletions(-) diff --git a/arch/alpha/include/asm/page.h b/arch/alpha/include/asm/page.h index 59d01f9b77f6..4327029cd660 100644 --- a/arch/alpha/include/asm/page.h +++ b/arch/alpha/include/asm/page.h @@ -12,9 +12,6 @@ =20 extern void clear_page(void *page); =20 -#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ - vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr) - extern void copy_page(void * _to, void * _from); #define copy_user_page(to, from, vaddr, pg) copy_page(to, from) =20 diff --git a/arch/m68k/include/asm/page_no.h b/arch/m68k/include/asm/page_n= o.h index d2532bc407ef..f511b763a235 100644 --- a/arch/m68k/include/asm/page_no.h +++ b/arch/m68k/include/asm/page_no.h @@ -12,9 +12,6 @@ extern unsigned long memory_end; =20 #define copy_user_page(to, from, vaddr, pg) copy_page(to, from) =20 -#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ - vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr) - #define __pa(vaddr) ((unsigned long)(vaddr)) #define __va(paddr) ((void *)((unsigned long)(paddr))) =20 diff --git a/arch/s390/include/asm/page.h b/arch/s390/include/asm/page.h index f339258135f7..04020a19a5cf 100644 --- a/arch/s390/include/asm/page.h +++ b/arch/s390/include/asm/page.h @@ -67,9 +67,6 @@ static inline void copy_page(void *to, void *from) =20 #define copy_user_page(to, from, vaddr, pg) copy_page(to, from) =20 -#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ - vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr) - #ifdef CONFIG_STRICT_MM_TYPECHECKS #define STRICT_MM_TYPECHECKS #endif diff --git a/arch/x86/include/asm/page.h b/arch/x86/include/asm/page.h index 416dc88e35c1..92fa975b46f3 100644 --- a/arch/x86/include/asm/page.h +++ b/arch/x86/include/asm/page.h @@ -28,9 +28,6 @@ static inline void copy_user_page(void *to, void *from, u= nsigned long vaddr, copy_page(to, from); } =20 -#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ - vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr) - #ifndef __pa #define __pa(x) __phys_addr((unsigned long)(x)) #endif --=20 MST From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 86A4039A06F for ; Thu, 7 May 2026 22:23:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192595; cv=none; b=Bfr548F4+zJHKaSD00ktXkTtxRfZ7LKwjKTSclr1ZrdtcHK68tA9AHcvwzwqc9e8QQubTKTou2JUXF8rb5RoVJ2FxAzecupofm3NjcHYG8sU3OJEg4Q7MzeqwimkQS+5hNn8/oWD4DUNidgX1rgSVddXeuwvNTjpqfKFHB9VaBg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192595; c=relaxed/simple; bh=IqcJvUNNbK1XhZiTXLCyFfQD4uhQUIj8/4+m+H9660U=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Abwb4PgMMRZ7Ms4thMN2xLB6oJNpvzmYNcYEpPvl7y5h7tJ6B0eAiR/D0UO2vrs86Puz96vHZYUL39DlxwsCAAhAB4IUF6ritmLVhQRHdc9KGnQcnqqOff8GDEeUwRdWYjLWclXWrkRUb3Cs2SZJOtX33UymtqoaLw03OjGr8yk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=CCmOx5b3; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=o8PlubGy; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="CCmOx5b3"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="o8PlubGy" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192592; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=bXud0h6YxK7twmSfS1n2EW2FIV5pqeOXrOEyMytphT4=; b=CCmOx5b3bI/euSJQE3PqlSbu/AiI1B3q6nmWLpZuIyG/glt/6R+hd1MGMwk99Nn11n5DLx VqCnz0Fr8xU8jn7v5zAlc//urLhzH/WS4izDDHR6fnudZF7bBnfRuw8WlNV2NqjUdLZGMo 0vkDi+JzekxomkZ1bit+ZUsrGA6ttHY= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-338-OgZ5KgWXPDujR2tpFofysA-1; Thu, 07 May 2026 18:23:11 -0400 X-MC-Unique: OgZ5KgWXPDujR2tpFofysA-1 X-Mimecast-MFC-AGG-ID: OgZ5KgWXPDujR2tpFofysA_1778192590 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-4470d6d2a4fso1370050f8f.1 for ; Thu, 07 May 2026 15:23:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192590; x=1778797390; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=bXud0h6YxK7twmSfS1n2EW2FIV5pqeOXrOEyMytphT4=; b=o8PlubGyUfakY0ju+ktrlfVGEg82ShKP01PBS5neoWuMNAG6elDfVbS1eNbwZyeknA +ymzYdHRiz5U3jjgd1KojsGeNGTUdg348RidRSWXHOwoICrInqC1sF+c+JtUoUNSmEfy 8GrkZXECmvx1LnLNLjHkS+bkIq35NLfuD+d8h/MQvelYIT+J/nJHXjO8b/ikk98i7Ac9 TbyrdQ3pcD+bASA4R17FbKTGimD0Y5qwcAIZePf8n7EMkiiBlfTvutRxNPOZteQ/3q2Q 0jDmuoVeog3zFErzry0dQ2pRqjHD7Ysyq3NjI6pzSbfpIPGatiJpv3+qQ/c6eofuV/b0 0CBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192590; x=1778797390; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bXud0h6YxK7twmSfS1n2EW2FIV5pqeOXrOEyMytphT4=; b=jAbAWlHJbDonQVB2GOesaSWLxoTgRHbYhuicnMnUasVOsGRlma9KDplsW9rYVgKyoO 86JUXD57MDJg08Cob/4HvHR+ZTi4MdOuLNbbj2b4KUTNjPQYteSL3U9Qu+l2LBWkfT7g lOevyQNJ4K0Rj8y0ciT5NGN/DPuLgJ1CZOT/aG8hlG0m8js5m4Uo4BuTBPyKbQC6byTo HWeX7AfKNsEEN1I/aAZ6Wf9bVuJuF76wcakjCNjm60XAkLyt4QiIm2asJ11p7/APtLaB z0L/opeGpVfB+im1JUwSIosp+vqfmTusyS04wrIL4kR2euK0r/DfwfBXJiYpJLPnOU/r N6Zw== X-Gm-Message-State: AOJu0Yylwr0qi4gxE+3V/a7FQDSjIgOX2DL+LbgFxT9wGJHbvMVOWqg6 OLk5iUwzIN/MGpreiJ9JyTovUYGPEiqTP/9KiGkU8ZCl9m/IW+LMNnaA4+iWoKU+mh1YaqfaE7Z xAR0DPeO98mxsW1/ZeKuRGE+W8NvejUavDDxUNj7xK9k/QSK2ZO9DnuBBbSCN+BoObWBiO97T8w 5bGSGyfxfqKLXkiPNZm5Wndq40r3sD8TJOdtI04OPlWAE= X-Gm-Gg: AeBDiessyGGSFM9VLwclCnIGzFuhDuQa6rNXYkJZ8OJ2V3h3dq3t/MtvmqWbfgt+fS1 z4FNxhU0SRKpYgAaFYnsZkw4zOYE6bgMw5i7jBqZ27Kl+lVXGo1g1st/V5Kdc4QeAajwrcQJXNB xttYWfBdNBrxBBqXxa//WCqaAN/FKh9HTBQ94SdjifeYMILmnOk3IuFtwDLCnkz/bJtPuZEqjKe Z4Q1roo6VQ85Ssq0VkwQdIJxUQvl5HSr3I6x4RktK13+n9OvvfNXo4NWwzpSWrIukUl74j5t48E jNdYX2XT09F6CHr96ilKMa95/Et1vl3a9EnpdmQjQI1PRRJs+PpcL1KcgcopNWSE+8NbpnYYCgo fxnLkPz0ABi1xJp/rB1OPfB1n5VqthELMqyo57Mbo X-Received: by 2002:a05:600c:4c25:b0:48a:65a5:750f with SMTP id 5b1f17b1804b1-48e51f46ddamr94972265e9.21.1778192589839; Thu, 07 May 2026 15:23:09 -0700 (PDT) X-Received: by 2002:a05:600c:4c25:b0:48a:65a5:750f with SMTP id 5b1f17b1804b1-48e51f46ddamr94971895e9.21.1778192589203; Thu, 07 May 2026 15:23:09 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48e53895effsm150472825e9.3.2026.05.07.15.23.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:23:08 -0700 (PDT) Date: Thu, 7 May 2026 18:23:06 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: David Hildenbrand , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , virtualization@lists.linux.dev, linux-mm@kvack.org Subject: [PATCH v5 14/28] mm: page_reporting: allow driver to set batch capacity Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a capacity field to page_reporting_dev_info so drivers can control the maximum number of pages per report batch. This is useful when the driver needs to reserve virtqueue descriptors for metadata (e.g., a bitmap buffer) alongside the page buffers. The value is capped at PAGE_REPORTING_CAPACITY and rounded down to a power of 2. If unset (0), defaults to PAGE_REPORTING_CAPACITY. The virtio_balloon driver sets capacity to the reporting virtqueue size, letting page_reporting adapt to whatever the device provides. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- drivers/virtio/virtio_balloon.c | 5 +---- include/linux/page_reporting.h | 3 +++ mm/page_reporting.c | 26 +++++++++++++++----------- 3 files changed, 19 insertions(+), 15 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index d1fbc8fe8470..7ed024315539 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -1017,10 +1017,6 @@ static int virtballoon_probe(struct virtio_device *v= dev) unsigned int capacity; =20 capacity =3D virtqueue_get_vring_size(vb->reporting_vq); - if (capacity < PAGE_REPORTING_CAPACITY) { - err =3D -ENOSPC; - goto out_unregister_oom; - } =20 /* * The default page reporting order is @pageblock_order, which @@ -1039,6 +1035,7 @@ static int virtballoon_probe(struct virtio_device *vd= ev) vb->pr_dev_info.order =3D 5; #endif =20 + vb->pr_dev_info.capacity =3D capacity; err =3D page_reporting_register(&vb->pr_dev_info); if (err) goto out_unregister_oom; diff --git a/include/linux/page_reporting.h b/include/linux/page_reporting.h index fe648dfa3a7c..306468b6c7d8 100644 --- a/include/linux/page_reporting.h +++ b/include/linux/page_reporting.h @@ -21,6 +21,9 @@ struct page_reporting_dev_info { =20 /* Minimal order of page reporting */ unsigned int order; + + /* Max pages per report batch (default PAGE_REPORTING_CAPACITY) */ + unsigned int capacity; }; =20 /* Tear-down and bring-up for page reporting devices */ diff --git a/mm/page_reporting.c b/mm/page_reporting.c index f0042d5743af..247cda44e9de 100644 --- a/mm/page_reporting.c +++ b/mm/page_reporting.c @@ -174,10 +174,10 @@ page_reporting_cycle(struct page_reporting_dev_info *= prdev, struct zone *zone, * list processed. This should result in us reporting all pages on * an idle system in about 30 seconds. * - * The division here should be cheap since PAGE_REPORTING_CAPACITY - * should always be a power of 2. + * The division here should be cheap since capacity should + * always be a power of 2. */ - budget =3D DIV_ROUND_UP(area->nr_free, PAGE_REPORTING_CAPACITY * 16); + budget =3D DIV_ROUND_UP(area->nr_free, prdev->capacity * 16); =20 /* loop through free list adding unreported pages to sg list */ list_for_each_entry_safe(page, next, list, lru) { @@ -222,10 +222,10 @@ page_reporting_cycle(struct page_reporting_dev_info *= prdev, struct zone *zone, spin_unlock_irq(&zone->lock); =20 /* begin processing pages in local list */ - err =3D prdev->report(prdev, sgl, PAGE_REPORTING_CAPACITY); + err =3D prdev->report(prdev, sgl, prdev->capacity); =20 /* reset offset since the full list was reported */ - *offset =3D PAGE_REPORTING_CAPACITY; + *offset =3D prdev->capacity; =20 /* update budget to reflect call to report function */ budget--; @@ -234,7 +234,7 @@ page_reporting_cycle(struct page_reporting_dev_info *pr= dev, struct zone *zone, spin_lock_irq(&zone->lock); =20 /* flush reported pages from the sg list */ - page_reporting_drain(prdev, sgl, PAGE_REPORTING_CAPACITY, !err); + page_reporting_drain(prdev, sgl, prdev->capacity, !err); =20 /* * Reset next to first entry, the old next isn't valid @@ -260,13 +260,13 @@ static int page_reporting_process_zone(struct page_reporting_dev_info *prdev, struct scatterlist *sgl, struct zone *zone) { - unsigned int order, mt, leftover, offset =3D PAGE_REPORTING_CAPACITY; + unsigned int order, mt, leftover, offset =3D prdev->capacity; unsigned long watermark; int err =3D 0; =20 /* Generate minimum watermark to be able to guarantee progress */ watermark =3D low_wmark_pages(zone) + - (PAGE_REPORTING_CAPACITY << page_reporting_order); + (prdev->capacity << page_reporting_order); =20 /* * Cancel request if insufficient free memory or if we failed @@ -290,7 +290,7 @@ page_reporting_process_zone(struct page_reporting_dev_i= nfo *prdev, } =20 /* report the leftover pages before going idle */ - leftover =3D PAGE_REPORTING_CAPACITY - offset; + leftover =3D prdev->capacity - offset; if (leftover) { sgl =3D &sgl[offset]; err =3D prdev->report(prdev, sgl, leftover); @@ -322,11 +322,11 @@ static void page_reporting_process(struct work_struct= *work) atomic_set(&prdev->state, state); =20 /* allocate scatterlist to store pages being reported on */ - sgl =3D kmalloc_objs(*sgl, PAGE_REPORTING_CAPACITY); + sgl =3D kmalloc_objs(*sgl, prdev->capacity); if (!sgl) goto err_out; =20 - sg_init_table(sgl, PAGE_REPORTING_CAPACITY); + sg_init_table(sgl, prdev->capacity); =20 for_each_zone(zone) { err =3D page_reporting_process_zone(prdev, sgl, zone); @@ -376,6 +376,10 @@ int page_reporting_register(struct page_reporting_dev_= info *prdev) page_reporting_order =3D pageblock_order; } =20 + if (!prdev->capacity || prdev->capacity > PAGE_REPORTING_CAPACITY) + prdev->capacity =3D PAGE_REPORTING_CAPACITY; + prdev->capacity =3D rounddown_pow_of_two(prdev->capacity); + /* initialize state and work structures */ atomic_set(&prdev->state, PAGE_REPORTING_IDLE); INIT_DELAYED_WORK(&prdev->work, &page_reporting_process); --=20 MST From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4B9D93A1A50 for ; Thu, 7 May 2026 22:23:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192598; cv=none; b=qxg/GTAsHM6vRrln1DbnlJoTTcTSAdGSMzkCIDI56Ky2BsxX38GymytTas7kG4BNfbL1NCoHKhyDR5vTgZM4UvNRC2y/Kzix9aRfLN+Q5eoIiZvRsfTb9+MtIqsXCc/Jj5QX1tjHWCFO8rvHRhw7B5afnxdQPPU4bkXuoRBFCxE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192598; c=relaxed/simple; bh=fd6TQPZxmdeWg/qPlbuU+1NCV53OBUv0/D8R5vSHGhU=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=ZGaAP+22cyT3J0LfZzOn7E/2v6z7e3IqcSlO6gQ/Iq6xZQZ3s7Rz0YUO0QOt6Q83pA0aRhXJyKNTBZr9VTPKEsus9gnxtcKkGKB1cy7qaVOv5d/q2QTjHQbYWHvwM0XA77FrXSgOD6MsipqwH5TSODymfrFAA+ARVLFRTXubIBM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=SXHlxCsD; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=B5jNRe2u; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="SXHlxCsD"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="B5jNRe2u" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192596; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=eTIW0e9wG6pgXoJqtpclYC5TuSr4GLdLxR6Dc7Uapl0=; b=SXHlxCsDNgrJJwp7CTyv7cFhm64ar2TMFyprvtFUOvgSEA3PMcGhikoT8uqLGQg/wZ32pn yKIUxdDOGZvGM5ce/CtKmoAjJ9FJETK4i5skMf9R5Z1alq/K1WjPhlhqAW/r+qaS9VEzLx iBdHFlBNY3gagh0VRecSKqI1aOVedIY= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-616-8DDcRMa8NVe9TfunHgbITQ-1; Thu, 07 May 2026 18:23:14 -0400 X-MC-Unique: 8DDcRMa8NVe9TfunHgbITQ-1 X-Mimecast-MFC-AGG-ID: 8DDcRMa8NVe9TfunHgbITQ_1778192593 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-48a5952c635so13461505e9.2 for ; Thu, 07 May 2026 15:23:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192592; x=1778797392; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=eTIW0e9wG6pgXoJqtpclYC5TuSr4GLdLxR6Dc7Uapl0=; b=B5jNRe2uQ0Lqcmrh5ZBG38muIsTu95lVejojd542SwLm9yUBifEb+uMgAHObGrE77q /DJqaWOkSKEkfG5sDJAlnSLxKvu14fyKqU6MwA3kjMO8OxQQzywgBvBx+lSc/gtpDbvO 8YWpawLc4xONNTzATCwDy2m8g4kiUguDZdnXXQu+aL69+Me/0ymhKm2YX/WQA4Xz/xEB ZuKxLVm1Nt4RYf6onomyUCIpAeFqC4V2WKqjBH38PmMdjakERdGb1O/Aup6MrMT6o4Eb +dK1dYCkOadXBDrUNfBt8Y8Cwm8PiH6ZsZz+FqtuV4W3nQ1vwsgmfvamMfPWB3kr5MfS /i1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192592; x=1778797392; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eTIW0e9wG6pgXoJqtpclYC5TuSr4GLdLxR6Dc7Uapl0=; b=EU+18g7ZR+I4U9gvMeyqhnCkZGef+E+VBGoTM1Y/VActKx2tr3G2aZfS82hH1gUiiC KxWbakZSkZXOO4Nodqvj/6yLse+9V+NnN9pdagTFhvHYJXxiDAcepDPr+jGWCy0wQdYA MmYPny9+tPUv2KhFHZ5716vK6dyZJypKPW+8Ldrx9xwVa8dy5W+SxNl0gMaAaoEsPm8Z jnxivuMlc4i7vDUhURqCeuXuwtbuouGF2l4rCQ6rD0UxwS+bZ2+mVNTZrMwB4FgPi03K zoaq/45w7TETX47BIpIEIG5fwHhQMHNMwF79yjSND3K6AH3sr1FW0mXx1caPF7rD/XFH SEVA== X-Gm-Message-State: AOJu0Yw6iUpa0C7D7uD+CCT6E27+fpnCA8fzWYPkNPDlY4XpIYAb0w9G pE3d0YKz1aMQyhkuxZugURKgVKPy/d2+mDX9pUDBim6NBdAX9pb2rZO67Bym4AQDgV+0sk6I1xJ kBtXHYIenOHzmaqwf8a3BiPSpnTjO0DYjqRnvIEJ7ayUWg0E5Wy7JesPiHEJwI/9nzJvcUKHDeC zXK1z4po73j9z4CoTgS11Y6U4yRTY0VNUObnOACW0obMA= X-Gm-Gg: AeBDiet8z05fX+o6U60oQvenDsl/HbzbUDId6e6euqXjkT2MBgAgUGvpgffpNmbha1Q uQdbsVxCBsi+UZHjOczDF8ua9w90ts90aXSt0q6/XBGjrwpzAu5xTlMNYbA2caj5w8qMy8VdtVV kHHhHghKdbJ+rnv49ChZ9Gj1e6GDeN2UhMO+nd0K/8+E7RmGImddgol6kmbLUMbjMiNRHGiKmq1 NS7zJTlULYKCZ651vG4mzTtUEHwysqgBdVKTUf9V4zAJ97vGgLTMLdT2yfJGoBvXhcxkSOvnD8/ CyxBpMjR1D5Lx7Q8mXxJuxs1UDCZ0Ox5yL/XlsULx9AqrmwsPfzrz3kbnIXtwolrVP7oiC8PLDn YWDxlKLHIu52zjZmwapJFcJskQefPrPUT8rrePb3a X-Received: by 2002:a05:600c:b8a:b0:48a:56de:d620 with SMTP id 5b1f17b1804b1-48e51f32ac3mr161298795e9.14.1778192592378; Thu, 07 May 2026 15:23:12 -0700 (PDT) X-Received: by 2002:a05:600c:b8a:b0:48a:56de:d620 with SMTP id 5b1f17b1804b1-48e51f32ac3mr161298425e9.14.1778192591788; Thu, 07 May 2026 15:23:11 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48e67444baesm2942875e9.8.2026.05.07.15.23.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:23:11 -0700 (PDT) Date: Thu, 7 May 2026 18:23:09 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , Vlastimil Babka , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , linux-mm@kvack.org Subject: [PATCH v5 15/28] mm: page_alloc: propagate PageReported flag across buddy splits Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When a reported free page is split via expand() to satisfy a smaller allocation, the sub-pages placed back on the free lists lose the PageReported flag. This means they will be unnecessarily re-reported to the hypervisor in the next reporting cycle, wasting work. Propagate the PageReported flag to sub-pages during expand(), both in page_del_and_expand() and try_to_claim_block(), so that they are recognized as already-reported. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/page_alloc.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 2bfa9ab60976..127b343d3783 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1730,7 +1730,7 @@ struct page *__pageblock_pfn_to_page(unsigned long st= art_pfn, * -- nyc */ static inline unsigned int expand(struct zone *zone, struct page *page, in= t low, - int high, int migratetype) + int high, int migratetype, bool reported) { unsigned int size =3D 1 << high; unsigned int nr_added =3D 0; @@ -1752,6 +1752,15 @@ static inline unsigned int expand(struct zone *zone,= struct page *page, int low, __add_to_free_list(&page[size], zone, high, migratetype, false); set_buddy_order(&page[size], high); nr_added +=3D size; + + /* + * The parent page has been reported to the host. The + * sub-pages are part of the same reported block, so mark + * them reported too. This avoids re-reporting pages that + * the host already knows about. + */ + if (reported) + __SetPageReported(&page[size]); } =20 return nr_added; @@ -1762,9 +1771,10 @@ static __always_inline void page_del_and_expand(stru= ct zone *zone, int high, int migratetype) { int nr_pages =3D 1 << high; + bool was_reported =3D page_reported(page); =20 __del_page_from_free_list(page, zone, high, migratetype); - nr_pages -=3D expand(zone, page, low, high, migratetype); + nr_pages -=3D expand(zone, page, low, high, migratetype, was_reported); account_freepages(zone, -nr_pages, migratetype); } =20 @@ -2331,10 +2341,12 @@ try_to_claim_block(struct zone *zone, struct page *= page, /* Take ownership for orders >=3D pageblock_order */ if (current_order >=3D pageblock_order) { unsigned int nr_added; + bool was_reported =3D page_reported(page); =20 del_page_from_free_list(page, zone, current_order, block_type); change_pageblock_range(page, current_order, start_type); - nr_added =3D expand(zone, page, order, current_order, start_type); + nr_added =3D expand(zone, page, order, current_order, start_type, + was_reported); account_freepages(zone, nr_added, start_type); return page; } --=20 MST From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1AEA13A6B81 for ; Thu, 7 May 2026 22:23:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192606; cv=none; b=CqzkqArpNnnOuBv+CMEiF8gPMlCN2FJbNMvhejIjoP7tdQpXAgTJns5wYexa/dyWrg3kWJU1MKtpPNqGnqKrql9O4a5dao5FZpKYESA/wX/ZGMa2j1qhoAuFWBmGgdHf2XTWQfZ3M7I1tXuOrE09v3LSdm9QXqF85SO6bIatlHU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192606; c=relaxed/simple; bh=WMCiVGj3DVL83UDto774y56RQOw5UuMPBOKdlF+WGtA=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=cnrsrOZqLdjBfmKRhOuD5e4iNWn0061FcnwPfaFLrZoSnh+4ssyHIVzs4gbWnm497O/c1m4whPRi8iRDUerc+ZGhmFO2gRG8l4v6YB9tft9li2AqmI+4AKKryWwAJATf0HUo9Jg/gQKJPJGRe7iXDt+ymVuGWS/lDiFLZGnI8n0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=F1RIcnus; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=nqcfGakL; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="F1RIcnus"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="nqcfGakL" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192600; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=t1oE2m6ZJQCRMesu6P3aMUgnmuqGZRVXH3PwwJINZLk=; b=F1RIcnusUVSw5dBNa+QHwOa0K/9/lvlqbsWC3aHf2F/qL9vG2dXhXrCQdzv61FsLPp0Mtk 5GiFJstQ3qbuBv+W3hHaSQTTtvWHdSphrgPRaD3N8iIGf19s/CTi/8DJh1RS4N8F2iubF1 RLeps5OLD7pIitgHHUBR2NuGzj8g528= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-148-rMoogZHyM-eMboXYYDksgQ-1; Thu, 07 May 2026 18:23:18 -0400 X-MC-Unique: rMoogZHyM-eMboXYYDksgQ-1 X-Mimecast-MFC-AGG-ID: rMoogZHyM-eMboXYYDksgQ_1778192596 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-4518f777225so871939f8f.1 for ; Thu, 07 May 2026 15:23:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192595; x=1778797395; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=t1oE2m6ZJQCRMesu6P3aMUgnmuqGZRVXH3PwwJINZLk=; b=nqcfGakLVsRsNtQRy/1aZud2X2rSxNxEjiBIK5lSvCZ88Ip46fu6PrRlb9DrkhMu6A sfZ9rWXoDR5Dq9DFcQpJBCVm9GzAJ3r3wsq9UmQTgaAp5UScB15GV+swCMI8GyNiUqG+ 2c9k6+AIIwb7A0V5i5SS8vz+jicJozdvPTozOrQd5kkHzMZ11JjDjoHSFADFisYafMtf HPH3cZywZa8gHR4onLT443wnk5Ej6B6V5PDJgrBRcPk5zkv5hdWzZeURPhML6Bq7UpHU UYO/5m+xWsD3JSHmVlV5/aZmU3rSLflmz6eEg8t4kMVYC6oelT5EMaSlDjJsXeCMErrV /Vtw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192595; x=1778797395; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=t1oE2m6ZJQCRMesu6P3aMUgnmuqGZRVXH3PwwJINZLk=; b=aWalv2Zfrgy1vQKoH2U8XZyhTQWWIlDQI6ErbKuSb4MvFeuiytLqmarQf8UyQSOi0/ SBVg1QnPItxbb7GSNQv9L9tLCDbU7YD/GJwIhzXN5u78T6glfAOZnEVCoSQhsv9JIUp2 iYFlPEdmvPiN4/UCCM9WX27VOWIpMiyafihnJeI+5OpavMVybV0oJYw+OhO96kwjgD8a 0iCEDgD6eNLpLAwQJLIk42MPfbt6Q995Aa1C6CUtObR9vpKnPzzsE7K1HIFXxq47m1QR nKVaBz/i5SbRt9xhEhWtmJN0FHRByprEdUX02CPjC3Weh/6VMysFTshY9C37vjvwW4tL yk6g== X-Gm-Message-State: AOJu0YxAGUOxFaXvBf/Zd8nJyHeP+uJ8JQ14Dj3golUMGLwY0X412oog /4cfOgllrQsKUf2eHUhbgnCBgLA2Q8LcrwnYITr/uDfg1vB72kADR56sf6Mqgl9JJV7ynOUBxbJ YbXE0rZauYP5nRLuRE5+/XiKUMVsI7zAWL2f7/FFAe8M4QizM3k9fSKUTCan8mTgtGqyU+1/7/K dZioj5A0DSDcG20HGlo9zyG9psay3BDxTXTyGEpMvfeiU= X-Gm-Gg: Acq92OE4UgTlq5No6gjbKFhp7CD93OYQd5Et8FvRwsv11RZfkZ4dxvDdCDBQqDxNfBG jdcqyDNhRe5D87QlXX3oE83fYO3ZH5GLVTuVIpllyrv5AZWus7x0t86hbSKanOU4Bwft3t94ISQ lWOAdwI+yf+Ts/5Q9jkx9sPCqFM0SEWPtv7E/6iNQCu2kOxYfEoZmmRp3eSKXfBhttJB7LUYyl6 9N2DepASLIQ4uWYHktmDPE94j/jyFcWmYPbXdKv7vOngFLwuHDOAVq0OFMusVHa3R7jVxZfLDM0 +BZYBPlkoBz62zBlDobRs+6obu8oeL6H65jsTVE5evJ67kJOJbOZs+speAxmrpb5QxCJZLbOR10 wlnjzFUov+CSjuJYWuk3wBHdJMN2f3Vh1wqER5FWe X-Received: by 2002:a05:6000:1ac6:b0:43e:a9ba:b194 with SMTP id ffacd0b85a97d-4515d5c6b69mr16657083f8f.34.1778192595160; Thu, 07 May 2026 15:23:15 -0700 (PDT) X-Received: by 2002:a05:6000:1ac6:b0:43e:a9ba:b194 with SMTP id ffacd0b85a97d-4515d5c6b69mr16657042f8f.34.1778192594539; Thu, 07 May 2026 15:23:14 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-454177de66esm1815153f8f.33.2026.05.07.15.23.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:23:14 -0700 (PDT) Date: Thu, 7 May 2026 18:23:12 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , linux-mm@kvack.org Subject: [PATCH v5 16/28] mm: page_reporting: skip redundant zeroing of host-zeroed reported pages Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When a guest reports free pages to the hypervisor via the page reporting framework (used by virtio-balloon and hv_balloon), the host typically zeros those pages when reclaiming their backing memory. However, when those pages are later allocated in the guest, post_alloc_hook() unconditionally zeros them again if __GFP_ZERO is set. This double-zeroing is wasteful, especially for large pages. Avoid redundant zeroing: - Add a host_zeroes_pages flag to page_reporting_dev_info, allowing drivers to declare that their host zeros reported pages on reclaim. A static key (page_reporting_host_zeroes) gates the fast path. - Add PG_zeroed page flag (sharing PG_private bit) to mark pages that have been zeroed by the host. Set it in page_reporting_drain() after the host reports them. - Thread the zeroed bool through rmqueue -> prep_new_page -> post_alloc_hook, where it skips redundant zeroing for __GFP_ZERO allocations. No driver sets host_zeroes_pages yet; a follow-up patch to virtio_balloon is needed to opt in. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- include/linux/page-flags.h | 9 +++++ include/linux/page_reporting.h | 3 ++ mm/compaction.c | 6 ++-- mm/internal.h | 2 +- mm/page_alloc.c | 66 +++++++++++++++++++++++----------- mm/page_reporting.c | 14 +++++++- mm/page_reporting.h | 12 +++++++ 7 files changed, 87 insertions(+), 25 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index f7a0e4af0c73..eef2499cba8b 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -135,6 +135,8 @@ enum pageflags { PG_swapcache =3D PG_owner_priv_1, /* Swap page: swp_entry_t in private */ /* Some filesystems */ PG_checked =3D PG_owner_priv_1, + /* Page contents are known to be zero */ + PG_zeroed =3D PG_private, =20 /* * Depending on the way an anonymous folio can be mapped into a page @@ -679,6 +681,13 @@ FOLIO_TEST_CLEAR_FLAG_FALSE(young) FOLIO_FLAG_FALSE(idle) #endif =20 +/* + * PageZeroed() tracks pages known to be zero. The allocator + * uses this to skip redundant zeroing in post_alloc_hook(). + */ +__PAGEFLAG(Zeroed, zeroed, PF_NO_COMPOUND) +#define __PG_ZEROED (1UL << PG_zeroed) + /* * PageReported() is used to track reported free pages within the Buddy * allocator. We can use the non-atomic version of the test and set diff --git a/include/linux/page_reporting.h b/include/linux/page_reporting.h index 306468b6c7d8..81e5a2819b3c 100644 --- a/include/linux/page_reporting.h +++ b/include/linux/page_reporting.h @@ -13,6 +13,9 @@ struct page_reporting_dev_info { int (*report)(struct page_reporting_dev_info *prdev, struct scatterlist *sg, unsigned int nents); =20 + /* If true, host zeros reported pages on reclaim */ + bool host_zeroes_pages; + /* work struct for processing reports */ struct delayed_work work; =20 diff --git a/mm/compaction.c b/mm/compaction.c index c1039a9373e5..61209cd408ea 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -82,7 +82,8 @@ static inline bool is_via_compact_memory(int order) { ret= urn false; } =20 static struct page *mark_allocated_noprof(struct page *page, unsigned int = order, gfp_t gfp_flags) { - post_alloc_hook(page, order, __GFP_MOVABLE, USER_ADDR_NONE); + __ClearPageZeroed(page); + post_alloc_hook(page, order, __GFP_MOVABLE, false, USER_ADDR_NONE); set_page_refcounted(page); return page; } @@ -1832,7 +1833,8 @@ static struct folio *compaction_alloc_noprof(struct f= olio *src, unsigned long da set_page_private(&freepage[size], start_order); } dst =3D (struct folio *)freepage; - post_alloc_hook(&dst->page, order, __GFP_MOVABLE, USER_ADDR_NONE); + __ClearPageZeroed(&dst->page); + post_alloc_hook(&dst->page, order, __GFP_MOVABLE, false, USER_ADDR_NONE); set_page_refcounted(&dst->page); if (order) prep_compound_page(&dst->page, order); diff --git a/mm/internal.h b/mm/internal.h index e39abab956e7..a01bc2c85cf2 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -895,7 +895,7 @@ static inline void prep_compound_tail(struct page *head= , int tail_idx) } =20 void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flag= s, - unsigned long user_addr); + bool zeroed, unsigned long user_addr); extern bool free_pages_prepare(struct page *page, unsigned int order); =20 extern int user_min_free_kbytes; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 127b343d3783..e5db2601d673 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1774,6 +1774,7 @@ static __always_inline void page_del_and_expand(struc= t zone *zone, bool was_reported =3D page_reported(page); =20 __del_page_from_free_list(page, zone, high, migratetype); + nr_pages -=3D expand(zone, page, low, high, migratetype, was_reported); account_freepages(zone, -nr_pages, migratetype); } @@ -1846,8 +1847,10 @@ static inline bool should_skip_init(gfp_t flags) return (flags & __GFP_SKIP_ZERO); } =20 + inline void post_alloc_hook(struct page *page, unsigned int order, - gfp_t gfp_flags, unsigned long user_addr) + gfp_t gfp_flags, bool zeroed, + unsigned long user_addr) { bool init =3D !want_init_on_free() && want_init_on_alloc(gfp_flags) && !should_skip_init(gfp_flags); @@ -1856,6 +1859,14 @@ inline void post_alloc_hook(struct page *page, unsig= ned int order, =20 set_page_private(page, 0); =20 + /* + * If the page is zeroed, skip memory initialization. + * We still need to handle tag zeroing separately since the host + * does not know about memory tags. + */ + if (zeroed && init && !zero_tags) + init =3D false; + arch_alloc_page(page, order); debug_pagealloc_map_pages(page, 1 << order); =20 @@ -1913,13 +1924,13 @@ inline void post_alloc_hook(struct page *page, unsi= gned int order, } =20 static void prep_new_page(struct page *page, unsigned int order, gfp_t gfp= _flags, - unsigned int alloc_flags, - unsigned long user_addr) + unsigned int alloc_flags, bool zeroed, + unsigned long user_addr) { if (order && (gfp_flags & __GFP_COMP)) prep_compound_page(page, order); =20 - post_alloc_hook(page, order, gfp_flags, user_addr); + post_alloc_hook(page, order, gfp_flags, zeroed, user_addr); =20 /* * page is set pfmemalloc when ALLOC_NO_WATERMARKS was necessary to @@ -3190,6 +3201,7 @@ int __isolate_free_page(struct page *page, unsigned i= nt order) } =20 del_page_from_free_list(page, zone, order, mt); + __ClearPageZeroed(page); =20 /* * Set the pageblock if the isolated page is at least half of a @@ -3262,7 +3274,7 @@ static inline void zone_statistics(struct zone *prefe= rred_zone, struct zone *z, static __always_inline struct page *rmqueue_buddy(struct zone *preferred_zone, struct zone *zone, unsigned int order, unsigned int alloc_flags, - int migratetype) + int migratetype, bool *zeroed) { struct page *page; unsigned long flags; @@ -3297,6 +3309,8 @@ struct page *rmqueue_buddy(struct zone *preferred_zon= e, struct zone *zone, } } spin_unlock_irqrestore(&zone->lock, flags); + *zeroed =3D PageZeroed(page); + __ClearPageZeroed(page); } while (check_new_pages(page, order)); =20 __count_zid_vm_events(PGALLOC, page_zonenum(page), 1 << order); @@ -3358,10 +3372,9 @@ static int nr_pcp_alloc(struct per_cpu_pages *pcp, s= truct zone *zone, int order) /* Remove page from the per-cpu list, caller must protect the list */ static inline struct page *__rmqueue_pcplist(struct zone *zone, unsigned int order, - int migratetype, - unsigned int alloc_flags, + int migratetype, unsigned int alloc_flags, struct per_cpu_pages *pcp, - struct list_head *list) + struct list_head *list, bool *zeroed) { struct page *page; =20 @@ -3382,6 +3395,8 @@ struct page *__rmqueue_pcplist(struct zone *zone, uns= igned int order, page =3D list_first_entry(list, struct page, pcp_list); list_del(&page->pcp_list); pcp->count -=3D 1 << order; + *zeroed =3D PageZeroed(page); + __ClearPageZeroed(page); } while (check_new_pages(page, order)); =20 return page; @@ -3390,7 +3405,8 @@ struct page *__rmqueue_pcplist(struct zone *zone, uns= igned int order, /* Lock and remove page from the per-cpu list */ static struct page *rmqueue_pcplist(struct zone *preferred_zone, struct zone *zone, unsigned int order, - int migratetype, unsigned int alloc_flags) + int migratetype, unsigned int alloc_flags, + bool *zeroed) { struct per_cpu_pages *pcp; struct list_head *list; @@ -3409,7 +3425,8 @@ static struct page *rmqueue_pcplist(struct zone *pref= erred_zone, */ pcp->free_count >>=3D 1; list =3D &pcp->lists[order_to_pindex(migratetype, order)]; - page =3D __rmqueue_pcplist(zone, order, migratetype, alloc_flags, pcp, li= st); + page =3D __rmqueue_pcplist(zone, order, migratetype, alloc_flags, + pcp, list, zeroed); pcp_spin_unlock(pcp, UP_flags); if (page) { __count_zid_vm_events(PGALLOC, page_zonenum(page), 1 << order); @@ -3434,19 +3451,19 @@ static inline struct page *rmqueue(struct zone *preferred_zone, struct zone *zone, unsigned int order, gfp_t gfp_flags, unsigned int alloc_flags, - int migratetype) + int migratetype, bool *zeroed) { struct page *page; =20 if (likely(pcp_allowed_order(order))) { page =3D rmqueue_pcplist(preferred_zone, zone, order, - migratetype, alloc_flags); + migratetype, alloc_flags, zeroed); if (likely(page)) goto out; } =20 page =3D rmqueue_buddy(preferred_zone, zone, order, alloc_flags, - migratetype); + migratetype, zeroed); =20 out: /* Separate test+clear to avoid unnecessary atomics */ @@ -3837,6 +3854,7 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int o= rder, int alloc_flags, struct pglist_data *last_pgdat =3D NULL; bool last_pgdat_dirty_ok =3D false; bool no_fallback; + bool zeroed; bool skip_kswapd_nodes =3D nr_online_nodes > 1; bool skipped_kswapd_nodes =3D false; =20 @@ -3981,10 +3999,11 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int= order, int alloc_flags, =20 try_this_zone: page =3D rmqueue(zonelist_zone(ac->preferred_zoneref), zone, order, - gfp_mask, alloc_flags, ac->migratetype); + gfp_mask, alloc_flags, ac->migratetype, + &zeroed); if (page) { prep_new_page(page, order, gfp_mask, alloc_flags, - ac->user_addr); + zeroed, ac->user_addr); =20 /* * If this is a high-order atomic allocation then check @@ -4218,9 +4237,11 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigne= d int order, count_vm_event(COMPACTSTALL); =20 /* Prep a captured page if available */ - if (page) - prep_new_page(page, order, gfp_mask, alloc_flags, + if (page) { + __ClearPageZeroed(page); + prep_new_page(page, order, gfp_mask, alloc_flags, false, ac->user_addr); + } =20 /* Try get a page from the freelist if available */ if (!page) @@ -5194,6 +5215,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, /* Attempt the batch allocation */ pcp_list =3D &pcp->lists[order_to_pindex(ac.migratetype, 0)]; while (nr_populated < nr_pages) { + bool zeroed =3D false; =20 /* Skip existing pages */ if (page_array[nr_populated]) { @@ -5202,7 +5224,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, } =20 page =3D __rmqueue_pcplist(zone, 0, ac.migratetype, alloc_flags, - pcp, pcp_list); + pcp, pcp_list, &zeroed); if (unlikely(!page)) { /* Try and allocate at least one page */ if (!nr_account) { @@ -5213,7 +5235,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, } nr_account++; =20 - prep_new_page(page, 0, gfp, 0, USER_ADDR_NONE); + prep_new_page(page, 0, gfp, 0, zeroed, USER_ADDR_NONE); set_page_refcounted(page); page_array[nr_populated++] =3D page; } @@ -6975,7 +6997,8 @@ static void split_free_frozen_pages(struct list_head = *list, gfp_t gfp_mask) list_for_each_entry_safe(page, next, &list[order], lru) { int i; =20 - post_alloc_hook(page, order, gfp_mask, USER_ADDR_NONE); + __ClearPageZeroed(page); + post_alloc_hook(page, order, gfp_mask, false, USER_ADDR_NONE); if (!order) continue; =20 @@ -7180,8 +7203,9 @@ int alloc_contig_frozen_range_noprof(unsigned long st= art, unsigned long end, } else if (start =3D=3D outer_start && end =3D=3D outer_end && is_power_o= f_2(end - start)) { struct page *head =3D pfn_to_page(start); =20 + __ClearPageZeroed(head); check_new_pages(head, order); - prep_new_page(head, order, gfp_mask, 0, USER_ADDR_NONE); + prep_new_page(head, order, gfp_mask, 0, false, USER_ADDR_NONE); } else { ret =3D -EINVAL; WARN(true, "PFN range: requested [%lu, %lu), allocated [%lu, %lu)\n", diff --git a/mm/page_reporting.c b/mm/page_reporting.c index 247cda44e9de..1f48fcd7c042 100644 --- a/mm/page_reporting.c +++ b/mm/page_reporting.c @@ -50,6 +50,8 @@ EXPORT_SYMBOL_GPL(page_reporting_order); #define PAGE_REPORTING_DELAY (2 * HZ) static struct page_reporting_dev_info __rcu *pr_dev_info __read_mostly; =20 +DEFINE_STATIC_KEY_FALSE(page_reporting_host_zeroes); + enum { PAGE_REPORTING_IDLE =3D 0, PAGE_REPORTING_REQUESTED, @@ -129,8 +131,11 @@ page_reporting_drain(struct page_reporting_dev_info *p= rdev, * report on the new larger page when we make our way * up to that higher order. */ - if (PageBuddy(page) && buddy_order(page) =3D=3D order) + if (PageBuddy(page) && buddy_order(page) =3D=3D order) { __SetPageReported(page); + if (page_reporting_host_zeroes_pages()) + __SetPageZeroed(page); + } } while ((sg =3D sg_next(sg))); =20 /* reinitialize scatterlist now that it is empty */ @@ -390,6 +395,10 @@ int page_reporting_register(struct page_reporting_dev_= info *prdev) /* Assign device to allow notifications */ rcu_assign_pointer(pr_dev_info, prdev); =20 + /* enable zeroed page optimization if host zeroes reported pages */ + if (prdev->host_zeroes_pages) + static_branch_enable(&page_reporting_host_zeroes); + /* enable page reporting notification */ if (!static_key_enabled(&page_reporting_enabled)) { static_branch_enable(&page_reporting_enabled); @@ -414,6 +423,9 @@ void page_reporting_unregister(struct page_reporting_de= v_info *prdev) =20 /* Flush any existing work, and lock it out */ cancel_delayed_work_sync(&prdev->work); + + if (prdev->host_zeroes_pages) + static_branch_disable(&page_reporting_host_zeroes); } =20 mutex_unlock(&page_reporting_mutex); diff --git a/mm/page_reporting.h b/mm/page_reporting.h index c51dbc228b94..736ea7b37e9e 100644 --- a/mm/page_reporting.h +++ b/mm/page_reporting.h @@ -15,6 +15,13 @@ DECLARE_STATIC_KEY_FALSE(page_reporting_enabled); extern unsigned int page_reporting_order; void __page_reporting_notify(void); =20 +DECLARE_STATIC_KEY_FALSE(page_reporting_host_zeroes); + +static inline bool page_reporting_host_zeroes_pages(void) +{ + return static_branch_unlikely(&page_reporting_host_zeroes); +} + static inline bool page_reported(struct page *page) { return static_branch_unlikely(&page_reporting_enabled) && @@ -46,6 +53,11 @@ static inline void page_reporting_notify_free(unsigned i= nt order) #else /* CONFIG_PAGE_REPORTING */ #define page_reported(_page) false =20 +static inline bool page_reporting_host_zeroes_pages(void) +{ + return false; +} + static inline void page_reporting_notify_free(unsigned int order) { } --=20 MST From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 90F1939BFEF for ; Thu, 7 May 2026 22:23:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192609; cv=none; b=lKtr5pw7AzfO39SOyiXtd0M+IeTeUP4rIIPzasiL1n2TkLlFuMX4/vtXKGnZUhel8bJ9kJ7CVpwPH+PxoVzDw979LLKtbhs36UKQuiib23HFHKReZpReMBtkhcRqIA+/bSAY5YB6hFKzqJcK8Z2mCdGITfoG6J/hOjbP/rF1yDo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192609; c=relaxed/simple; bh=h33C6BLNHiocTz5KC1arzWXU6+/Zulu6S8cQDI8rRLY=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=hmim4foJOKi6MRPzQsULr+snNn9516wOmecV5JWBvpt7ntfUd1zfWJagBOzaWvQlxlA+T/yvpHPB2kl4Hpfbvrnv7qKdTOpSsl0NwiZAkKdS0STO5OubsYiraO3UlJnNj2M8NrSTiiT3ueGrd4Q/b+d8p4+EgZ8G84wpWVXL7Mo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=eboJ9nyN; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=aZ3+Oik+; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="eboJ9nyN"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="aZ3+Oik+" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192603; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=b/zNXUvir974pwEF1ci0jhyssPrmgKAWIncWIRRvDt4=; b=eboJ9nyN0J6Xz1uPqmF0zIYREMUnd0WqRIpmf5D/ymBv98xSxNRenfH4edBSTxYjLzUy4i yrIxkVfdWSFHUB6g/ZVl9qcaVgXMvqHcc4+Hpuh2rrFGyUOUO4CN+/qUhX/J24O5dWX3cM ch+UmGI6YECEFl0LcngxHFKnqTsd15w= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-694-_Lv16i2xO3uEsf2mCldeKw-1; Thu, 07 May 2026 18:23:21 -0400 X-MC-Unique: _Lv16i2xO3uEsf2mCldeKw-1 X-Mimecast-MFC-AGG-ID: _Lv16i2xO3uEsf2mCldeKw_1778192600 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-48a548b585cso8249655e9.2 for ; Thu, 07 May 2026 15:23:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192600; x=1778797400; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=b/zNXUvir974pwEF1ci0jhyssPrmgKAWIncWIRRvDt4=; b=aZ3+Oik+za4ZWLU29Nk8pRsV8288EZVlSryr868VBjYSoRNg4diFPZzVfGeNlBeqSD UDktZXpBx2uPRCj8uxANKufwZBKiR24BtfeYkkt3iZz9YGisClf7Sggv+rq6YMQGsTtD BOuY1wAhhmvrfiim0g0bkvjl4S7GBh451T5glAuYblftRdvtFg7b/8Mc5RpUUl5oNYYl ZGV0zE7apzaAEcty13m6jSeAe76Lh+Ult0ixM8nQhT8L0+RZV3qOGXzubwtz+VhWceHU 3jRqw/jimxPbf/mcinKJ/M5W/MOQQurwXQsFAtUA5uhGRlXusTdcI0DeAlDof8giB8dj FMwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192600; x=1778797400; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=b/zNXUvir974pwEF1ci0jhyssPrmgKAWIncWIRRvDt4=; b=oyJPzIas1RG/q7uEeALf7vYtkHMVB3fLU4FEo2nqo6kS4HJId/a8dPogtoHXqfjbmu lxlpPJE7cRNXQ4weC6/qX0UJCYhgCsd1v/B3jlpv5a0+u3AkvbbJZRzxuq+pZE9g3bEK 6DplGLb2A7C7vFtOwKilEQ0SuDlbjQkCRu2u1aZ0EDkOUgxOS5Rl5DuZMSn3euMKB5EX fjdbKRz5YzMbxUKfyq7mGkyM5AF0NjKpD7/EEqZpgaMGf0vo+cLOoJfhkPmyGwqI9qqA yz54+cZyL5DMwpoCij0M4WSDv8eXzOYjL7shSpuWdysGJDwrZ1LfrSpXFVbhiTh88joj A7+Q== X-Gm-Message-State: AOJu0YxjtTPQ+QQRP1meR/b15S3PwNovVxnFZDTXDZM64tSOBtyV2B0S RRenZqDgnNRJCsqwaFwLkZvmA9IvnhOuA+6ANeoaUknx1bXyXZm39QsHNNZ9pJwe9+pjVJYKKo7 mvDYBgUCwLENG3qH5eqVpPNwToN+3dhuZfoV7xqEYf5SageUiS51CP1PL/dSXqb025f3kGeFZei coZZR1jBcLBTKIVEfYSuWPQH+riTEdcqykEYS10WLaAq0= X-Gm-Gg: AeBDieuZE1k4l9EoRvPon6ydb3AEI/S+o8UE2GwolUEat2m963CjxVgbTC1pmxf9/Kw n9qtbm5E8CJ1Pcr2SsDjoTJRIwQvooKsZIHu+R72QSEOhtfBcuyvKrbGmV5ic4fOi1uicj1xByS sXZvN3+begySaIMfQfYNPPZdLbU7EiIFkVsMK7mCzNZanrKFkoDkEknK1CSMqVw6/ecfA+UcP5Y 2yuWVeE/5PoycspkYes0K8ktG941s2537PSEOQ0ZkKqzO4qbfuKAzMCsA9wU3w5lte8dKpvxZUT H2upw4JKy5i4udfwjoejcfdCmUSfBjKGPJTiDlh36snoYxK2Nsib7mSILBaa8aJiuNOM6qrhcwC HjtKE8FVX5wFCMQwjMCWO5EDN++eI+aIdfU5hzJVN X-Received: by 2002:a05:600c:c094:b0:487:5c0:671f with SMTP id 5b1f17b1804b1-48e51e1a7c4mr115500095e9.9.1778192599496; Thu, 07 May 2026 15:23:19 -0700 (PDT) X-Received: by 2002:a05:600c:c094:b0:487:5c0:671f with SMTP id 5b1f17b1804b1-48e51e1a7c4mr115499725e9.9.1778192598907; Thu, 07 May 2026 15:23:18 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48e539109b7sm142179255e9.15.2026.05.07.15.23.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:23:18 -0700 (PDT) Date: Thu, 7 May 2026 18:23:14 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , linux-mm@kvack.org Subject: [PATCH v5 17/28] mm: page_reporting: add per-page zeroed bitmap for host feedback Message-ID: <8f7e00cf0ab76d2d393e043bf82cca32546e9e4f.1778192416.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The host may skip zeroing some reported pages (e.g., due to alignment constraints or bounce buffer fallback in QEMU). Currently, when host_zeroes_pages is set, all reported pages are unconditionally marked PG_zeroed - even ones the host did not actually zero. Add a zeroed_bitmap to page_reporting_dev_info that the report() callback can use to indicate which pages were actually zeroed. The driver's report() callback is responsible for managing the bitmap: zeroing it before sending pages to the host, then setting bits for pages the host actually zeroed. page_reporting_drain() checks the bitmap per-page in addition to the global host_zeroes_pages flag. No driver sets host_zeroes_pages yet, so the static key is off and the bitmap is never read. Behavior is unchanged. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- include/linux/page_reporting.h | 7 +++++++ mm/page_reporting.c | 8 ++++++-- 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/include/linux/page_reporting.h b/include/linux/page_reporting.h index 81e5a2819b3c..df929f682901 100644 --- a/include/linux/page_reporting.h +++ b/include/linux/page_reporting.h @@ -16,6 +16,13 @@ struct page_reporting_dev_info { /* If true, host zeros reported pages on reclaim */ bool host_zeroes_pages; =20 + /* + * Per-page zeroed status, indexed by scatterlist position. + * The driver's report() callback must clear the bitmap, + * then set bits for pages that were actually zeroed. + */ + DECLARE_BITMAP(zeroed_bitmap, PAGE_REPORTING_CAPACITY); + /* work struct for processing reports */ struct delayed_work work; =20 diff --git a/mm/page_reporting.c b/mm/page_reporting.c index 1f48fcd7c042..8137ff50ce1c 100644 --- a/mm/page_reporting.c +++ b/mm/page_reporting.c @@ -108,6 +108,7 @@ page_reporting_drain(struct page_reporting_dev_info *pr= dev, struct scatterlist *sgl, unsigned int nents, bool reported) { struct scatterlist *sg =3D sgl; + unsigned int i =3D 0; =20 /* * Drain the now reported pages back into their respective @@ -122,7 +123,7 @@ page_reporting_drain(struct page_reporting_dev_info *pr= dev, =20 /* If the pages were not reported due to error skip flagging */ if (!reported) - continue; + goto next; =20 /* * If page was not commingled with another page we can @@ -133,9 +134,12 @@ page_reporting_drain(struct page_reporting_dev_info *p= rdev, */ if (PageBuddy(page) && buddy_order(page) =3D=3D order) { __SetPageReported(page); - if (page_reporting_host_zeroes_pages()) + if (page_reporting_host_zeroes_pages() && + test_bit(i, prdev->zeroed_bitmap)) __SetPageZeroed(page); } +next: + i++; } while ((sg =3D sg_next(sg))); =20 /* reinitialize scatterlist now that it is empty */ --=20 MST From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A462F399011 for ; Thu, 7 May 2026 22:23:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192611; cv=none; b=a8RZckPRtrGwygMTSUev3FxLRjntoDDAwZZU51bVNLjxk1mJQuBFhmsUN6QxZXhuPUZwVHxjH12v9ZEuWN8KdzMCK6/piABqN71dPFCxUSe2vPFDSx43MynfKdBJCaL+PJ2/twAvjqgiUeHL/YVQkaWGQRX3FKK71/GHbRE6dh8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192611; c=relaxed/simple; bh=3blNm03p0sEINzdQ5ippK+lU1Y9A0zqQCf4DUU5ix7c=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=jdFYxGYGKQ5IiBPoy91m9FQ28Ht2t+sX+pzRyrbPLwQHQsYfjJnXHW0LPY4jQZfj7tqoPFPSb1vHnUNC5lAqbwqBP5g01YH6GzReU9QW5U1p7h12IKICdvrmDBIJC5NpigXNd4Jag0w4njeXwQdwmDc+nin7gsuCRcw6o5HfcwE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=TBACxlXz; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=EGr2SMh/; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="TBACxlXz"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="EGr2SMh/" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192604; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=/cazEjjhaO5BxPm4JyLG8Gc9L6oe2zaOenErLTI/evM=; b=TBACxlXzaD2DmduvpS/H/KcZgHuUgt9Irgho+YOyoJsIrST4KRcodIoRG1h9BvYerAqGTa Hh3usivjXeMw4xJfVCoidZaixwWlBAwZLpbDQK9sYCYky/NxE+dLd5dG1puShmxN71aa8l /TNSo9fx94Zx58o4D0oym/XnQD8svSM= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-154-hNMc_Ta1M8eFN_13DyX3oQ-1; Thu, 07 May 2026 18:23:23 -0400 X-MC-Unique: hNMc_Ta1M8eFN_13DyX3oQ-1 X-Mimecast-MFC-AGG-ID: hNMc_Ta1M8eFN_13DyX3oQ_1778192602 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-4470d6d2a4fso1370154f8f.1 for ; Thu, 07 May 2026 15:23:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192602; x=1778797402; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=/cazEjjhaO5BxPm4JyLG8Gc9L6oe2zaOenErLTI/evM=; b=EGr2SMh/C9n5omQPj+qH0OINWZvK8weYKhzq/TRQvD5wAc7BkkcVYMFR5VhaYXJM4z XfZmcAPD9fPPB7vqV9WhpqfgMqP4EDU7wItZRl1EYHm6ClcnSo/szSrDK7D6o6R6ew2R Aqwyq+k7og4ltSwBxFTV3vj2PWCs9CaCs70eeZtmDa/wPFllKfSXqi6UD9ANzNQWTiAg fm/alHDJTONXVOKhYUjk6xkP/No7uEoyMYnlpdE5/LvkcD5IYlRyaYALCvOidWAjr5Os Q/YdUAMHlRa47zAFlDJs7iUpmyQewIXdjuGV3fzldOa6/knDQyzoDvlCFrb/mp8V2Pta 5pCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192602; x=1778797402; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/cazEjjhaO5BxPm4JyLG8Gc9L6oe2zaOenErLTI/evM=; b=Q8bun1xxP0ZCdJSvzcjSUJVavgVC9qaSJuZ8r+nRYylDuw0+v27lY0iFkzqub/arLg OsBVVGvDzUqVhbricbt5IMMbARVY68NMCkJ2Yzqu3f4gSLh8VoMhXbGlL9f52SP10/vu TQTdT4DwKkF4Ub1uzVklyypXIwXGk0nJpeBpFlAa8L6qJdCyiu31qtdf7a+BV2Y3xh/b +2QqpvhGCT2Sjsh7zLWTRObB0U+s4TIQnEZhLiWHpKiXfODO2E7PuQRnqMxrUM6WkziV 9azgQL1DjlNjB+80feCzAVvNXRan41CnoGCPBzLiWCKfx5xBxJd+ADfbtcZYiXUB1F5f PMoA== X-Gm-Message-State: AOJu0Yysayd1UAKYZABNaOoVV9MBxsGJz7A6CC45xHpXrZ+9PznmY6ti Zjt3S/qDf/2Mj9mjKb+pywbq8FLT1OgHktvPHnwq1kr7CF5sDTOBsVBw9G902Jyuk/SrKmkWiH8 gSOFv+iZZejQqoDh86l7EMn2BQOTM3Z5nLt3NZ2cEAqvDayqnSHe4gmUo5ZIrfSxkC41SqxHR5/ LQcsisAKSOix7+61KctGrXN28j1V8GdZ8DFihhXJ7WUcg= X-Gm-Gg: AeBDiev+giWlnXPP2IrQHzC8TiEvKvEYsrbguBOB+F1K2j6TWju+c6AwZAH3/jwB136 JsoGKBK/2rIB0iwt2oDlMbAZgnN9QdK11Yzl97PRcTwnqOc1or8X+6BJPSnCy83TKf9WdRR22bA 9HaWHgQC0UqXNjH/SWQjCkpjH4OXSoQfrN2YsenpbGPrBtmaledwdxOBDLUONs7jqRi5MGZJ/ep FhvCtvKJ9eIcx5PbJbWrvX2CIFFg9mjHaghXr3Wg3LeNO9F4rIwGyfBSoj+oLSjFDnguELz254y MY0QOwShNqUjHy1u4YI4ZapL3gciqgodpHpisBRa0AEiyH0KZsed9V/u1V2K2sMNbO91w3gtn/v 9hPPVW3bXArtuVzv0xa6M6A8Dz5fNNaZ/uWmcRb7g X-Received: by 2002:a05:600c:4e0c:b0:48d:46a:6e5b with SMTP id 5b1f17b1804b1-48e51e19110mr170280245e9.7.1778192601944; Thu, 07 May 2026 15:23:21 -0700 (PDT) X-Received: by 2002:a05:600c:4e0c:b0:48d:46a:6e5b with SMTP id 5b1f17b1804b1-48e51e19110mr170279835e9.7.1778192601462; Thu, 07 May 2026 15:23:21 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48e53907e8asm148617135e9.13.2026.05.07.15.23.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:23:21 -0700 (PDT) Date: Thu, 7 May 2026 18:23:19 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , Vlastimil Babka , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , linux-mm@kvack.org Subject: [PATCH v5 18/28] mm: page_alloc: clear PG_zeroed on buddy merge if not both zero Message-ID: <9928bb52244928b53172e7d95f770bc63d982b4e.1778192416.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When two buddy pages merge in __free_one_page(), preserve PG_zeroed on the merged page only if both buddies have the flag set. Otherwise clear it. The merged page would inherit PG_zeroed, and a later __GFP_ZERO allocation would skip zeroing stale data in the non-zero half. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/page_alloc.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index e5db2601d673..63b7f396ff30 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -984,10 +984,14 @@ static inline void __free_one_page(struct page *page, unsigned long buddy_pfn =3D 0; unsigned long combined_pfn; struct page *buddy; + bool buddy_zeroed; + bool page_zeroed; bool to_tail; =20 VM_BUG_ON(!zone_is_initialized(zone)); - VM_BUG_ON_PAGE(page->flags.f & PAGE_FLAGS_CHECK_AT_PREP, page); + /* PG_zeroed (aliased to PG_private) is valid on free-list pages */ + VM_BUG_ON_PAGE(page->flags.f & + (PAGE_FLAGS_CHECK_AT_PREP & ~__PG_ZEROED), page); =20 VM_BUG_ON(migratetype =3D=3D -1); VM_BUG_ON_PAGE(pfn & ((1 << order) - 1), page); @@ -1022,6 +1026,8 @@ static inline void __free_one_page(struct page *page, goto done_merging; } =20 + buddy_zeroed =3D PageZeroed(buddy); + /* * Our buddy is free or it is CONFIG_DEBUG_PAGEALLOC guard page, * merge with it and move up one order. @@ -1040,10 +1046,17 @@ static inline void __free_one_page(struct page *pag= e, change_pageblock_range(buddy, order, migratetype); } =20 + page_zeroed =3D PageZeroed(page); + __ClearPageZeroed(page); + __ClearPageZeroed(buddy); + combined_pfn =3D buddy_pfn & pfn; page =3D page + (combined_pfn - pfn); pfn =3D combined_pfn; order++; + + if (page_zeroed && buddy_zeroed) + __SetPageZeroed(page); } =20 done_merging: --=20 MST From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4D0F03A6B6E for ; Thu, 7 May 2026 22:23:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192613; cv=none; b=QkIMR+mUQ7q3lBJrQvqln9y1BlKZfZE1HnXKfWVff+kRmzqWYZ1wJF2zc407FIrp3PSG4PfhJUcbcdiK1GJh+vjfM24xlytUfv9EUDmkjqMSBe94hzWQjSqqMRAyHbQ41BKK725CnICC2wBLntyUzCOCe10LJLEogLU0mES6PDA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192613; c=relaxed/simple; bh=oRpxEzeFk7zQ2qjVzXOgrFHr86o70/YLRN7Wn9pfibs=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=ftUBjEJo1YyUfXwxqtvAIyD9Ll8Mo+zULi5yrZUF6jbuFVywsfCAZBtoUoC638UGIwc7yDPzjqRUqoZAlt2HUzAgE/5qesHO42M9leTmdM1hx1VYkqD2kcx6u4Z4Pwx7VlUG1xCnkgBTXzq5E0hA6mNajCyoFIQ3rF7b6PMMWpU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=LVHp8Eig; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=WrvmNMCp; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="LVHp8Eig"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="WrvmNMCp" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192607; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=QlFr+m2Pk5vg0O1/5MoawoNrwUSv9ERusu/cYrbRsds=; b=LVHp8EigCS88aaXzTIhTWiBL7m1beI2chFdABileJMASsnfQqdriczI68J9e5MxBWc6i3r GNwOh8QfBm+HirrM7SxiexSl/CoJeh/4JoQ5dKgVe43swaRrNRp0mOl1sSgF/T4YRZjYhi gLJ3x6EWsE5Z5+tSN2lcCBlUDj4gu6g= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-536-l1EmIOaANGKyHJDvCgSEKA-1; Thu, 07 May 2026 18:23:26 -0400 X-MC-Unique: l1EmIOaANGKyHJDvCgSEKA-1 X-Mimecast-MFC-AGG-ID: l1EmIOaANGKyHJDvCgSEKA_1778192605 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-43ff19e54beso960512f8f.2 for ; Thu, 07 May 2026 15:23:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192605; x=1778797405; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=QlFr+m2Pk5vg0O1/5MoawoNrwUSv9ERusu/cYrbRsds=; b=WrvmNMCpp5ynxbz5XKRS6msygZ3cl/XmW/nigrE/8sdpGdNjF2TPykmBQMkPKITUeQ oigQgcC9gAMO9+MSABZhBGaxyl3i/T4bQJdzlAci2dHIca+2VNRyISet8Ig9N/6rwz8R a6FQY02KLVYJRZ3cdBqxB1TMQzfXl6jieUxd7b6Tx+P2vVrBy5e8mWttTj5pURw09xPx deB3orw+fXMn2gFarzegPfZ7VNulNl2rUt8grRyGfzbRr6F/0OE9qtUm1hNCeKWPvNbf Pzl/LjB5kdIV92udqqc9fTOo9t+vhDsKnHaZfySJBHt8Iv6uwTRRgZ8hjGY4e9HAJ9jR 7BCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192605; x=1778797405; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QlFr+m2Pk5vg0O1/5MoawoNrwUSv9ERusu/cYrbRsds=; b=rIw4OuoT6yrTqG8bn9/AOX2sRsb89nbt+mrZPHTHjmFMTG92eBuQ/BQ5H8GfNmZXBj FgoNp1wzNL/5yyiKr0qkFbISEmDNUBYv3eaorX9mJyG/D4zQC7qHvPovZ2NbevztoMWC DW3BBapttnlQeZwqcjGKUgxfqhxx2R7h2dc/qA5k2ox9tV0XE/nBZ4Q76nZ3aQ8Qp08f bNps3kTk/y389BkKz7RJDJeqeT+iswtzcFhHRoKwm5NNOEyWmVJ2E8+M622hRAZrbJC9 6XquNwh65h3ghr++N3STtBKdwaSfPSwqFhY0ia0X+zmiC+4osc7KElMAAVEXWhpOJUhQ 9iHA== X-Gm-Message-State: AOJu0YyKheEM0VTRAp2kFn43N94HD/CdYCoqdP9szxQgrmVnEmrR1jIE UXNyIOVQkA1nt2Iwi/Jm1UlEBeWUOt1TW7+YDElFh3uOcsOOWa5PK139nLz2rDZOpL6n202aWT7 Vxu30UEQ2cI1oMdc534+4lySFK8mFqjtNwSAaEisjLn8gqi2w0L/vCB3d2TIXhdd3orhyDdjKcV kMXP9a7pX/BCpTkPVbSA+p0NjRKLrRBXQ8fBMGG/jOJbc= X-Gm-Gg: Acq92OFmuoV0wpHnN6jDV5xVa4uKXIpmCSAk4yGdgW8lDKpPU2r0QG7st1VKhgj91JZ 7Aih68+iOr5x76ZlZ1z42T7RtojiUacmmUF8AmjPJ0bSkOm1n0TmVsH4WAaERyAi2RteJKlkRbu /J5sK08hKLzwMrc/2WA7f4gW/LqXx03cqj61GIUCTdjtWP3/J2LoSWyGZrj+DTIqS7hZvI4ILHH 0wvL7IL+NYwTH3EN1WZjbzmpDl6iunMdbEinWOZcX5euSSIfNr7CJWNQ9aPmbrSTUwdx7sNSGnh E14oko0rTuKsc4e88sNt9gwo7gEnAUVmw9vIlbXwG/U1dDdc7IfnSGP47J4Sr/435GJSwpBhf+2 xtEXd2cQCH7hgILrHMVvmKxjjgauVjKztUn/5qmjA X-Received: by 2002:a05:6000:24c3:b0:441:3144:efc5 with SMTP id ffacd0b85a97d-45463d8a9a9mr271790f8f.42.1778192604611; Thu, 07 May 2026 15:23:24 -0700 (PDT) X-Received: by 2002:a05:6000:24c3:b0:441:3144:efc5 with SMTP id ffacd0b85a97d-45463d8a9a9mr271750f8f.42.1778192604026; Thu, 07 May 2026 15:23:24 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45412cefe8bsm2089364f8f.11.2026.05.07.15.23.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:23:23 -0700 (PDT) Date: Thu, 7 May 2026 18:23:21 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , Vlastimil Babka , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , linux-mm@kvack.org Subject: [PATCH v5 19/28] mm: page_alloc: preserve PG_zeroed in page_del_and_expand Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Propagate PG_zeroed through buddy splits in page_del_and_expand() and try_to_claim_block(). When a zeroed high-order page is split to satisfy a smaller allocation, the sub-pages placed back on the free lists keep PG_zeroed. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/page_alloc.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 63b7f396ff30..5c6ab949855a 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1743,7 +1743,8 @@ struct page *__pageblock_pfn_to_page(unsigned long st= art_pfn, * -- nyc */ static inline unsigned int expand(struct zone *zone, struct page *page, in= t low, - int high, int migratetype, bool reported) + int high, int migratetype, bool reported, + bool zeroed) { unsigned int size =3D 1 << high; unsigned int nr_added =3D 0; @@ -1774,6 +1775,8 @@ static inline unsigned int expand(struct zone *zone, = struct page *page, int low, */ if (reported) __SetPageReported(&page[size]); + if (zeroed) + __SetPageZeroed(&page[size]); } =20 return nr_added; @@ -1785,10 +1788,12 @@ static __always_inline void page_del_and_expand(str= uct zone *zone, { int nr_pages =3D 1 << high; bool was_reported =3D page_reported(page); + bool was_zeroed =3D PageZeroed(page); =20 __del_page_from_free_list(page, zone, high, migratetype); =20 - nr_pages -=3D expand(zone, page, low, high, migratetype, was_reported); + nr_pages -=3D expand(zone, page, low, high, migratetype, was_reported, + was_zeroed); account_freepages(zone, -nr_pages, migratetype); } =20 @@ -2366,11 +2371,12 @@ try_to_claim_block(struct zone *zone, struct page *= page, if (current_order >=3D pageblock_order) { unsigned int nr_added; bool was_reported =3D page_reported(page); + bool was_zeroed =3D PageZeroed(page); =20 del_page_from_free_list(page, zone, current_order, block_type); change_pageblock_range(page, current_order, start_type); nr_added =3D expand(zone, page, order, current_order, start_type, - was_reported); + was_reported, was_zeroed); account_freepages(zone, nr_added, start_type); return page; } --=20 MST From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A82E33A6F05 for ; Thu, 7 May 2026 22:23:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192616; cv=none; b=n8VoxXaZRMPI0VbLLXSxUqD5Mm8mpho679ttqUlk7eD4eLf1h+MFUwdMU2+i8OiIHyqb4wublSb32xiHEg0SZTrCwsLnDMfjFJvBGJBtDlC9IGdKFJtodrqo+jJ/ZxoxM2QkXTsUeao526YIRXvhFVvFM63wDI/XnRyTdG7lYfs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192616; c=relaxed/simple; bh=XxUQw3tZx0AkRKlM9jSBDC3gsiVvkne3m0EtV0t+S60=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=ff6Dko8B2V6tQx4xIBdQlJdfUSnauJinvHKzinjPlhHWHEAacs8RAqGOha8+TJJoqBxx0dHieNXNCkNUH2NXzGM+LNR+pDljp7W4Ef6ASZNpOiecsU9XNfFufYsKUvdQZ1hUVREZ8Y9MTbEoH/RWR78/EU9VmB6sx7P6jXNbE2o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=emNJIG0e; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=uI1Jmyee; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="emNJIG0e"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="uI1Jmyee" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192610; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=XhmJzOluo2YbyCNWY9nO19Tj7FrnfhHS9n0nNp+gIQc=; b=emNJIG0erfkvmtd9snyQhBNCCDcLf/XnsUY7YMPejno0vvx+VhBQ+F+i7uSeMPKbPLE1Iz 03RhX9pSQQh5CnBWK5UB1AeSHk1EBquz07lRIb2/Y+7f+CPBNCR8eT9UHSfmQYscOI+Qf6 AK6MDtcYOAaXuhp7uN7IpXwLHyfD23I= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-169-L710aIC9NZOY8TJe9AbREA-1; Thu, 07 May 2026 18:23:28 -0400 X-MC-Unique: L710aIC9NZOY8TJe9AbREA-1 X-Mimecast-MFC-AGG-ID: L710aIC9NZOY8TJe9AbREA_1778192607 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-44f65835b77so796755f8f.2 for ; Thu, 07 May 2026 15:23:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192607; x=1778797407; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=XhmJzOluo2YbyCNWY9nO19Tj7FrnfhHS9n0nNp+gIQc=; b=uI1Jmyeef68H+o4GhArn2DG5e2N/BWe9jXvsLPmIRCRXciIMu7HgemPVD+WGQHyq/v WIpndtYNo/nEAwX+pQgu882vW4ajY6MlfvvmrEDe5XlxClYI2MP1nRO1exJe/J7mlEY3 S3yQpnzAT61tamRxSbSJQcva6GmbRznkQwTLouYqFCDw21bz4cRkTb8cwMWMOW8rO45o VQgEzuODYeFUfgQb9+I5WvXUrYOCLjge2mhGiwrdy++KcxXYdYxsYcPiwgg8n4B/OsRY gvvnaxwv8lLqjZxCoexxB/58m9KwtXxXX26RQaQFQ9iPqxISDZjsEQ9AlJUwSfwkhL77 9D3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192607; x=1778797407; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=XhmJzOluo2YbyCNWY9nO19Tj7FrnfhHS9n0nNp+gIQc=; b=sVCXWOI39OI9TGAhNyaRURYBIKHuJVQ8/eKkI5fxBeKPf2MftUSQI16bF/syuUckmq S8eXxsrhAU4Roi6BuOIxrQQsLcbHe/Jji/2csSOTj2a7QdfZAegeBnqo6E3MEPuqBArk oEOq0rI0kD3cRaX9i7nmI2+q7uVMfdcpC/DZ8CZs1ABvs5XV/zD7o8cXqO39iRJTs2FG R53eOo+ufn6lbqacVqGBlOXcxV1TibiGD8K6QWZITJhhPKimDfUtu3AtZc3WUx33cs4+ 25PhBKScWkdhuI8OtafCP/rT6f8rJMtMmp07OJJEcWLUZJk/YSMVFqDBSFE7sO+yUuf9 Tr5g== X-Gm-Message-State: AOJu0YwdSGUhk3Sp1LhXQN73ZnF5rhDSgskCDHoRzE2mjpJHwSiXL1ih 6dVrsk6LhWS5Rk8DLzt87kohJLezXigBBffL8CLpwqvWLHuTUlhpcs4i/9hp9sqTn8NDESQ9TbN PhcAsscyMrj4ekKgQFCLN9NjjWogKIkA9xiMEmKMn84rvJkeRONqAKkrQUSQvS5+8eUpaH/CUk5 TgyXPa7N7MkelbYev8v8KKbzBWEWISlt7yl8JhC/oAaNY= X-Gm-Gg: Acq92OEnuq4nIB9LAQUx5SGNnfap5qyNxE1oRl6MBr3OSSBIEl90pFsiy4JSXSr7vq5 tSgzGIt0Jy4Ug+GSzCeJnNo+tiLtoWimtnVb8XMp+cmWLsE6N7K0bP/g03k4wTk+zYxEMogx48v kj2P3ubYwQ6ZNBOeU/yrPFmtQPavu5XqK2BCCxZXNeAEuWDBGIva4CMzrJgcLV9P63vmJhRwR2V +MAO2IER76w5aU6kBKbHFlIdMM8JoAffn7KxKPXHb60kRj49zMc35577tYwitjivyU78psL88vI K6dnfG2Hucp4IDig/hdbzubYyy8tqkmpA71hCs9hJd026UHrohD1npBVN96GnH9soMQTtbx/Em8 bKVePxEkVvsWVHVkQM95RdOOMep3U9laRRrqz6NRN X-Received: by 2002:a05:6000:4211:b0:43f:e99a:ff91 with SMTP id ffacd0b85a97d-4515b057368mr16166105f8f.4.1778192607047; Thu, 07 May 2026 15:23:27 -0700 (PDT) X-Received: by 2002:a05:6000:4211:b0:43f:e99a:ff91 with SMTP id ffacd0b85a97d-4515b057368mr16166067f8f.4.1778192606385; Thu, 07 May 2026 15:23:26 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45412cefd38sm2251262f8f.9.2026.05.07.15.23.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:23:26 -0700 (PDT) Date: Thu, 7 May 2026 18:23:24 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: David Hildenbrand , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , virtualization@lists.linux.dev Subject: [PATCH v5 20/28] virtio_balloon: submit reported pages as individual buffers Message-ID: <84a052f6bb0b0e286e9bd0f640e4f7269b1921c4.1778192416.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Submit each reported page as a separate virtqueue buffer instead of one buffer with an sg list of all pages. This avoids indirect descriptor allocation (kmalloc in the reporting path) and gives per-page used length feedback from the device. On error, the already-queued pages are kicked and drained before the error is returned. The caller (page_reporting_drain) then marks the batch as unreported, which is conservative but correct. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- drivers/virtio/virtio_balloon.c | 36 +++++++++++++++++++++------------ 1 file changed, 23 insertions(+), 13 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index 7ed024315539..e99ffbbdd2bd 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -202,25 +202,35 @@ static int virtballoon_free_page_report(struct page_r= eporting_dev_info *pr_dev_i struct virtio_balloon *vb =3D container_of(pr_dev_info, struct virtio_balloon, pr_dev_info); struct virtqueue *vq =3D vb->reporting_vq; - unsigned int unused, err; + unsigned int i, err; =20 /* We should always be able to add these buffers to an empty queue. */ - err =3D virtqueue_add_inbuf(vq, sg, nents, vb, GFP_NOWAIT); + for (i =3D 0; i < nents; i++) { + struct scatterlist one; =20 - /* - * In the extremely unlikely case that something has occurred and we - * are able to trigger an error we will simply display a warning - * and exit without actually processing the pages. - */ - if (WARN_ON_ONCE(err)) - return err; + sg_init_table(&one, 1); + sg_set_page(&one, sg_page(&sg[i]), sg[i].length, + sg[i].offset); + err =3D virtqueue_add_inbuf(vq, &one, 1, &sg[i], GFP_NOWAIT); + if (WARN_ON_ONCE(err)) { + nents =3D i; + break; + } + } =20 - virtqueue_kick(vq); + if (nents) { + virtqueue_kick(vq); =20 - /* When host has read buffer, this completes via balloon_ack */ - wait_event(vb->acked, virtqueue_get_buf(vq, &unused)); + /* When host has read buffer, this completes via balloon_ack */ + for (i =3D 0; i < nents; i++) { + unsigned int unused; =20 - return 0; + wait_event(vb->acked, + virtqueue_get_buf(vq, &unused)); + } + } + + return err; } =20 static void set_page_pfns(struct virtio_balloon *vb, --=20 MST From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 599C13A7584 for ; Thu, 7 May 2026 22:23:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192618; cv=none; b=tjcFq/0vB61Vx4xa7+Srg3YkIxvdkrjGGsMzqUE7h32omccEALJx8Py5m4qXt1hA8SJQBmXpF40qhCzZ8oPpK5/d0gjiFAIjNAs/e7hRfX7D45bOTJkw3FiqgUYp7SGh/XiQ2x+l9fArMf/pGn4LiSxfH2U3EXheN1AN/uy1E+A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192618; c=relaxed/simple; bh=EL2Reby3Sb0JeLNnkgrxvJ5sCkfhGgIj5uWxUglejhE=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=OA6jf4qQa9LRQjSkuG5IOSI4wnVYqOj0zkQUFJskwHEZtfTG59SY42O1jnlNtkmkc78E5u+a9Bj/hSrKqEnCoNDAPKHAN9WBHcGMji5jGM73PgalH7iDDLdZaWyqTC3HIQdOvotinLWaCbEcIUIyBAz9KF4pBdMH1V2USA11VYs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=TACqLUvc; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=JidxKXab; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="TACqLUvc"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="JidxKXab" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192613; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=1YDr6AZkMpuuq2CYnJYP4SwOU9jLN3mgGUSThqOO84A=; b=TACqLUvczed+oCaVfUI2Gxl09QvDeaQOrng8FoFmTJWKwCKJAp1yZnHRikfqUDYYdamcLN 6QPWbhx681/sANLu4Qm3mQAS5xzNGKGce9EodsM1r1IbyMWS44OWdXy/MV+R8G4GxvB1+n 6NNz+w+huLvKBeDQp+WxVO6MMYQkQwc= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-17-AH8Z-zzoOQ23pX40xhwBtA-1; Thu, 07 May 2026 18:23:31 -0400 X-MC-Unique: AH8Z-zzoOQ23pX40xhwBtA-1 X-Mimecast-MFC-AGG-ID: AH8Z-zzoOQ23pX40xhwBtA_1778192610 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-48a5952c635so13462625e9.2 for ; Thu, 07 May 2026 15:23:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192609; x=1778797409; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=1YDr6AZkMpuuq2CYnJYP4SwOU9jLN3mgGUSThqOO84A=; b=JidxKXabipW5ffW5AgZRgJEb0abLKDqBQtzOdVg/OJAH5ifNaTNl+JbxtzZIbSNrgb ySVitLgLKiUaHOVSO8lmK81gR4kujzaYv+QK8B3Pt0DtpSujvXbBXK2BDtFQcJDs+YXw 5EFUtViY1EpWX+YVIuRxaxd0v8JN+QhxZP66F2DzBOU9ZSpHUdlIe6qs42OI9sbR0j0q Z4bz/Ct4QQxJejk9IQVUy87pTQrRO5l+pFfLQT/vxjgLIA6z/AJDtel11etOSiADUZg2 BJpv3RSi3d+ak+oIhtWm3s3gw23LOBbSFeb0zAvG//uIK7Tney0hSZHjq1kcZMxVWx0Z oWbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192609; x=1778797409; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1YDr6AZkMpuuq2CYnJYP4SwOU9jLN3mgGUSThqOO84A=; b=klPoaXI3STN/vq115rmDl+H7ijFi58yRU+DudkMgxBiXRjch4bRrYnJpALXGp0nZkM 09X+iNtv+F7wwh7kqEoy5sT/S8D01eNTsOpDfefZSAevCkD058quSh0xVDzFGKrj2bbM D0FnBatq/J1MHQqIQPhYr0SsSPkLMb1uWHb9x5XDfxwN78JQkATvsId6C/sZHOE4dXLe j+/r5Rb/vxUt2pHYxkJgakkv6g5y+F3Wni9WWPfhLhpKKuw3V3lZbyuU7VyP9X/jZmlZ iVq6wkqEUF7ukZ8ZDFiIQbBe0+dU9HsY0Jci3KbXKu3GKGRAc4LCBj7xUNHt7tnKYYxy 8U6A== X-Gm-Message-State: AOJu0YwezoAOVdAHTDzExKghbzNex9vRjlwUH+WenT6JuBgvAOQ/zP/d fHWEiCRi6OPZzYgoDm53FW59b7Sxjeg5Cl8FrjBk1b15Bi2K85PzvIsq6JlHzBuaniqXghDFonA BaRCrKhKSCAYkpDm1OllQDtHYD3e7YHc8ruVB4HtUxHBNpoZXb8CwbWx+UvY7mRxJejggJCt7oV 1bneU2MVorI2tCkPSpSC77d2n+eemyHwVW5pRTQgw0TYQ= X-Gm-Gg: AeBDiesGSeEfJBotQ7IGjyMjCkVyfe5/W0gGWEE7r1/ZNaFRAExip9mNgFnZn0rGQ0F MB1VJ4lo2lXgjwDVQL4iQBeFjWO1BC6mWcpQUZtmYWH4+1f8C1tqoT+idZ5GIoXbW7flt4eaT5c mcUAmATlPbfc/ASz9wFeS9BVyL8It8bW7H02xsDq1UGQuVN07Vz3wfb6P79hT2D7j1N8l+2CrGj q4myMp3Qs4ar4LKiaiFq/AOfa943Ft2TleSNI1lfCXsR0qZhOgtLjRBMsme2KWbKd0B9rUq7ZZa evv23gIPq5+wlTzOSkBZxcakluMJhEP8GavYNMRAqTRMBUCvpPyrAVKALzP8XLvk6oI5Z86X7Qh bFGGG40IxSFFctsZ0N+c309sKGLB7Vve5W/SyByIu X-Received: by 2002:a05:600c:1f0d:b0:488:b8bc:6a32 with SMTP id 5b1f17b1804b1-48e51f45874mr152541255e9.23.1778192609392; Thu, 07 May 2026 15:23:29 -0700 (PDT) X-Received: by 2002:a05:600c:1f0d:b0:488:b8bc:6a32 with SMTP id 5b1f17b1804b1-48e51f45874mr152540905e9.23.1778192608850; Thu, 07 May 2026 15:23:28 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48e67455581sm2909895e9.4.2026.05.07.15.23.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:23:28 -0700 (PDT) Date: Thu, 7 May 2026 18:23:26 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , Vlastimil Babka , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , linux-mm@kvack.org Subject: [PATCH v5 21/28] mm: page_reporting: add flush parameter with page budget Message-ID: <26688a24bd4c2dc461859b47e887e730b2577f1c.1778192416.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a write-only module parameter 'flush' that triggers immediate page reporting. The value specifies a page budget: at least this many pages (at page_reporting_order) will be reported, or all unreported pages if fewer remain. The actual number reported may exceed the budget since each reporting pass processes a full cycle across all zones. This is helpful when there is a lot of memory freed quickly, and a single cycle may not process all free pages due to internal budget limits. echo 512 > /sys/module/page_reporting/parameters/flush Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/page_reporting.c | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/mm/page_reporting.c b/mm/page_reporting.c index 8137ff50ce1c..5590645acaa9 100644 --- a/mm/page_reporting.c +++ b/mm/page_reporting.c @@ -358,6 +358,48 @@ static void page_reporting_process(struct work_struct = *work) static DEFINE_MUTEX(page_reporting_mutex); DEFINE_STATIC_KEY_FALSE(page_reporting_enabled); =20 +static int page_reporting_flush_set(const char *val, + const struct kernel_param *kp) +{ + struct page_reporting_dev_info *prdev; + unsigned int budget; + int err; + + err =3D kstrtouint(val, 0, &budget); + if (err) + return err; + if (!budget) + return 0; + + mutex_lock(&page_reporting_mutex); + prdev =3D rcu_dereference_protected(pr_dev_info, + lockdep_is_held(&page_reporting_mutex)); + if (prdev) { + unsigned int reported; + + for (reported =3D 0; reported < budget; + reported +=3D prdev->capacity) { + flush_delayed_work(&prdev->work); + __page_reporting_request(prdev); + flush_delayed_work(&prdev->work); + if (atomic_read(&prdev->state) =3D=3D PAGE_REPORTING_IDLE) + break; + if (signal_pending(current)) + break; + } + } + mutex_unlock(&page_reporting_mutex); + return 0; +} + +static const struct kernel_param_ops flush_ops =3D { + .set =3D page_reporting_flush_set, + .get =3D param_get_uint, +}; +static unsigned int page_reporting_flush; +module_param_cb(flush, &flush_ops, &page_reporting_flush, 0200); +MODULE_PARM_DESC(flush, "Report at least N pages at page_reporting_order, = or until all reported"); + int page_reporting_register(struct page_reporting_dev_info *prdev) { int err =3D 0; --=20 MST From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6F24039934E for ; Thu, 7 May 2026 22:23:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192623; cv=none; b=Xd1DrziZI7ti4HQN6+3clMQpGiaaTCM0CUpMPzLQNlmy2H+7JI4YDf8NAA0QbANN3ZnXGipTFlXmFo/zdPc2C70HuYqcp8liSrdhJNATlHuDFWwj1bVcUVV4N/eAGSoqx4Lct+bQveQfEJ1aTym5cSOX/uvqQQjISHjS3TD7e+Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192623; c=relaxed/simple; bh=R5JoOSUpnCNXT3wTHs1n8P71YjvIX/5NmZh4Srb0TOA=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=IRl8wfqbgXWwBBC9o/dx7ecMXFfNv9SJokozbIgKhSYSNhV7bWMcPR2PR1aYSkE/vFHy6XWzoz5EZ0IK9ZNUVMehr27wzYc/fJXcQi4S3jQsHBC5s9ye1xtxXO9/RffeSnjMHyX9Y/wl2UVl1ga6N4stCT9Drf6qmyuMyLWEvP0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Bpiaqz1d; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=ejXw6HNc; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Bpiaqz1d"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="ejXw6HNc" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192618; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=vW+jGDNFEDzTL0ZYP8xjaWjT8cnTW8x9jCORFRBulSw=; b=Bpiaqz1dGVYIe23lCaXemsPzAjzzmewzeYWMI6Y7LorvPxiis1W/ofjasOVRt5RxLG5SXv VQnEjl6aACThkE0Nirl7TM54RBYQoZmowuj6b0+88yJ+tKG5kACyi/Suwk/egrLVtwV7av oXrVQJyfNtdkmSGuqriWcdR72g12AVI= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-440-CqoxG5J1Mb-JDcikzRK01w-1; Thu, 07 May 2026 18:23:33 -0400 X-MC-Unique: CqoxG5J1Mb-JDcikzRK01w-1 X-Mimecast-MFC-AGG-ID: CqoxG5J1Mb-JDcikzRK01w_1778192612 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-4411a2c034fso1131132f8f.3 for ; Thu, 07 May 2026 15:23:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192612; x=1778797412; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=vW+jGDNFEDzTL0ZYP8xjaWjT8cnTW8x9jCORFRBulSw=; b=ejXw6HNcH4qyayam7/DSNMHiPHsEvSWrF7edyY+KNnc22vYlnkzfzhqyQUlizEb8xj 5t0/rJfDqnvZ5XByk+ErT7f30UJGJIpIZy0W/6OGSVxtOCk63Fhm0R1xntQgM+DAOOFS 05eG8Zuhwwau7Ru1PRCug/vHDMhv11KNwTEQNZjkY8VPXVqiM/oHasjDfGeOHy6smhzp SkAd5cFXI19Z2T3xoo5lGLt77fNKQ9EeGy8WpR0I9keXNc04gYwjXzRuMpOZQJHa9he1 l3mqxSv6P7SPMITvzOfnIT51qQt1E6ze1xdvs2hEnJUZHlZDRLWxd4lVSukzkkm1cv80 DjHQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192612; x=1778797412; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vW+jGDNFEDzTL0ZYP8xjaWjT8cnTW8x9jCORFRBulSw=; b=fGPZSAHw/IiPuMwJecs7DWcGEmSh5MT/g8B9Z0HlWZJuCj1VeY81sohNxRlQzbrGkV KQAhXhEVmGXQHaKviuVZkpd01OmnAMPV6csyxMCVVlOnJiVSdP4EhoMxn4mtUOyLNhWf JqlSm5/pLyHXB+r5h8kLCJtKL/4qDfnh2RU+d7GGKWW3ZStJpYEEUQb0VqOZYEjJVyfG FqTGzkFYZqkYYocL/QjsMEnJPrNlRZbhfPpKFRxKEdJw4YiSWTXWZf8sWXgUhqrbPsU+ wKpv3FcyHiGmqhMpasCaJm7zNkVdmnZR2iJOXTmB61jj3jIs8C2awdrlVo1RRXRVVUkJ mPOw== X-Gm-Message-State: AOJu0YwP719xm0JvMDoFQ60A249zzA/X8N0+anGMXl4QTYCtO5Yp30qI MCHaIh7qjQdbTcT8h4Y8egW5saeSGfn2YNdIdH3cvX3lhzKrS+BtwR6sktSjzQ4+5nf6BJBE0Ao BCX8G2tMMSBIHvmQ+YQL2Qus37OUFpM8957M4xQajPV3NE/LdnSUkTxiiAWyIdUCKEKt6FvEMod KdW1zI3gsTbCsSxQOkJdlDMcTAu7tXPbrp78M2DFTY2FA= X-Gm-Gg: AeBDiestz0H6lXnZngkry1d1kmIh9FE4sVmfy2PK18zWP2ANQvQrPok7lsrcR+Myh+t 3KRGiLRTbbk3Jea3bWlRWR+8jOqdoFk+ekx6eAm92ahITVx+IQZEcbrFFgJkYvVF0CcdUo15f4a sggFrezL5fHuuKtBoGHZIcSR5cnqw142+UFzxJhenxs7IkrzDVe5y4XWb6UiGUqUJhU7M7xmvje BL+ownSdD//yR+VjF9+li6V25GgxrKZoFWXCCpEjq0Sb05Lb88Oe7X17n8f8sH8LvkfhWMjbiqa KeH9tS9rqcWwpxHbP1QwRVujq0zM1p2vq02OO6HN1peGpqk24A4bvhob06wEd9GSI/k0q8VbIO/ o3ljZn0IkIUut1vaP0pcfveMlOYImoqpnlcsxCGkd X-Received: by 2002:a05:600c:3e05:b0:488:9ed3:1492 with SMTP id 5b1f17b1804b1-48e51f2a997mr175518475e9.10.1778192611953; Thu, 07 May 2026 15:23:31 -0700 (PDT) X-Received: by 2002:a05:600c:3e05:b0:488:9ed3:1492 with SMTP id 5b1f17b1804b1-48e51f2a997mr175518165e9.10.1778192611407; Thu, 07 May 2026 15:23:31 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48e65d1367bsm3470575e9.31.2026.05.07.15.23.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:23:31 -0700 (PDT) Date: Thu, 7 May 2026 18:23:29 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , Vlastimil Babka , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , linux-mm@kvack.org Subject: [PATCH v5 22/28] mm: page_alloc: propagate PG_zeroed in split_large_buddy Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When a large zeroed page is freed with order > pageblock_order, split_large_buddy splits it into pageblock-sized pieces. Propagate PG_zeroed from the head page to each sub-block head so the zeroed hint is not lost. The propagation reads PageZeroed from the head page (set by __free_pages_ok before calling free_one_page) rather than from FPI_ZEROED, to avoid incorrectly marking deferred pages that share the same code path but may not be zeroed. For order-0 frees this is a single-iteration no-op (the head already has the flag). For higher-order frees it ensures all sub-block heads inherit the zeroed hint. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/page_alloc.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 5c6ab949855a..954d86655f77 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1554,6 +1554,7 @@ static void split_large_buddy(struct zone *zone, stru= ct page *page, unsigned long pfn, int order, fpi_t fpi) { unsigned long end =3D pfn + (1 << order); + bool zeroed =3D PageZeroed(page); =20 VM_WARN_ON_ONCE(!IS_ALIGNED(pfn, 1 << order)); /* Caller removed page from freelist, buddy info cleared! */ @@ -1565,6 +1566,8 @@ static void split_large_buddy(struct zone *zone, stru= ct page *page, do { int mt =3D get_pfnblock_migratetype(page, pfn); =20 + if (zeroed) + __SetPageZeroed(page); __free_one_page(page, pfn, zone, order, mt, fpi); pfn +=3D 1 << order; if (pfn =3D=3D end) @@ -1624,8 +1627,11 @@ static void __free_pages_ok(struct page *page, unsig= ned int order, unsigned long pfn =3D page_to_pfn(page); struct zone *zone =3D page_zone(page); =20 - if (__free_pages_prepare(page, order, fpi_flags)) + if (__free_pages_prepare(page, order, fpi_flags)) { + if (fpi_flags & FPI_ZEROED) + __SetPageZeroed(page); free_one_page(zone, page, pfn, order, fpi_flags); + } } =20 void __meminit __free_pages_core(struct page *page, unsigned int order, --=20 MST From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 53F9D3563D4 for ; Thu, 7 May 2026 22:23:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192625; cv=none; b=HZo5g0XPHZypEPplRd0GVAW+jFouHf5FL+1ncugGVxUtw4wUWJFpTWzzMn6mBL/e4tjZPgIhqquyXcFdkKLPatooK1FPvJljvXDLqs26QZ1yCWO9LOZODf8EOSUl2lpqbqvDOk29X6jjgSLCW9/4vofsri7S2ElXzGn9kVzuEFg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192625; c=relaxed/simple; bh=1RD9d/+/lqDsWY9u1FvWqjpjd693PTYsgjDeYz93Bbw=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=gJzqVvVfoi2NXcLq3eNcc3ntOTnCziyKjzrCPUPJtJE1d5XzYzoDcQ9qkWalCHD7V8fkhHog2G9yHjuOVEpAhdNunI/Mcj6VrHCIAC3SolV2zgEqmwf2R07SN5IcVm3awPPJbk+Whf99eSOEJulcckRfmQKsMlU7iWFgSna4myY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=IcKhln1S; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=FXSrkiDB; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="IcKhln1S"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="FXSrkiDB" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192619; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=warOUQQwxUg2qGQS9g7Xsjf/DX7jQncHlRwRmsTPpz4=; b=IcKhln1SaE26P03o9f9x3ohJ91kf7Kd5yYtPuUUfZMpo66I/0PPl3rKnkCA9usVad5taCN R10IWVivNlVnNWUErlpjL6PI0hHSkJyn3z25w4zVH3rgNUs6NRwYnFz0ZCLZXYqJ9USQ6J K+rhZwWGnrjvhPdc9KL/LYIXPalsCFk= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-357-eeMNzzr3OmWrp27qwNbUcQ-1; Thu, 07 May 2026 18:23:37 -0400 X-MC-Unique: eeMNzzr3OmWrp27qwNbUcQ-1 X-Mimecast-MFC-AGG-ID: eeMNzzr3OmWrp27qwNbUcQ_1778192615 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-449b2a183d3so880128f8f.0 for ; Thu, 07 May 2026 15:23:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192615; x=1778797415; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=warOUQQwxUg2qGQS9g7Xsjf/DX7jQncHlRwRmsTPpz4=; b=FXSrkiDBvsrFvhqm4fyi1pduh7iH7ri09F/YLVYE0Q0UZhHhez/33XhWAzuYHH5cXO iHYt0dXGgFse7gAVbU2I1mfWzAujItG0b6ejF49SuW35b5mi3qZ3Ez1nKCgYDyDBJQbt mUfFK6KGcmA+zSwDxDedov0Yity8V/f90L9h7ge23wngBOhg6KHuY4VzXVlRTb7ffc0i gg7J/AtazRTZu9rjVd+Yr3OupNJeBMgAk4tgksZ/m+HVXiPBG7brajUZXR1NePEHs9m1 yoGbMOrN03o3UHIZhT2x9WPFYXQNrwd0LRFHsjZjYrWL1B8V2BHS6WHg9ss45YKBMlAO X1zQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192615; x=1778797415; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=warOUQQwxUg2qGQS9g7Xsjf/DX7jQncHlRwRmsTPpz4=; b=hpN6mF6l+JDRKYe4n+8+J9zNrofcdKbPkO6LI9UXI5Jdg9i8BfEj4ul0fpy4Qhrqq8 DImxspOqC1IX8wQXCJZijM5iwWa3pB8dr68/YvYlznec7Ie124ywjFROlERPGH0AeK4E I9D40DHaJMhycYUN8Xw59hHQL6aUphrjoTR2AiNDD5+hiLheMEAcNAK8i7bG0NDghvM7 EY/2YRRcz1IagXGs7WHyt5MNuNFndBW/TWiq3Q8U6YvEfykdzIG6HnFAVSGCtkmmLvOL iKrkuCKn1ypra6X7iqeCe3m8Go1q9QOlbLvVlOrK5jybgG0MUcZXeio23NYhoMrINQCw Jy9g== X-Gm-Message-State: AOJu0YxIE/HJ/Urr2oeTPXggG11FeQrqGKRsH/fJ7RqzTVp84tNEqju8 TzO+3cOMTs1Nq5i1AjFudvMCI9PD6rOQ/UArWNL4Aq4WkO67aASHJQR+mhKUSg6Ux/CXQRI9Drn VBPmwQPIcCj8+2DGudwu8Rqs7FuAJf+qDvq78rLvk92vqw9uBe1VlDeaSBBGN/nKGiwedpqvo+O YCGJOuirPLjWVcE9qMPZY8u/6m+OZ5LyWlTo2OK3nRFyw= X-Gm-Gg: Acq92OF/mxLHnGEjqqJK8NxorVw1pGycC9SW2HIZQs9FzxkKGEJ8jTS2v+BMMdIh4VA x70hTjMHLVK9ZnPuu2y9fYJ9mtMEK8fY77yM7XgJ7AAlEfjr9yz3ARnl0gu7ib5eps07e74qYRp XhcVLHAMXbdcj8oulMUEug7+rxVT1GTWgc8Mn0Eqk8Vx1xgLDPshIkWPSsj/ZcLrgm/5jcjiGEl zNmt3bvgGpZUe578+as7md4xn8f+eWD9lPxPwnwPKnOXXSil7dEjxngzq1NOukGtFWnubfN8GcI DWjeo+q2QKTf7naRI72m/OBFNPqlj6nhEqblAxhnLXx4oxKQKuE2yXBz4EJfLgVey38zEzqeGU5 QPCnu6BfEOxIcheW/gDYyc1mo7OiyplfQBT6pZnXJ/blsc1Xn8O0= X-Received: by 2002:a05:6000:2383:b0:43e:a75e:352 with SMTP id ffacd0b85a97d-452e78b35a2mr6956797f8f.4.1778192614743; Thu, 07 May 2026 15:23:34 -0700 (PDT) X-Received: by 2002:a05:6000:2383:b0:43e:a75e:352 with SMTP id ffacd0b85a97d-452e78b35a2mr6956758f8f.4.1778192614089; Thu, 07 May 2026 15:23:34 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45417124ee4sm2243093f8f.28.2026.05.07.15.23.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:23:33 -0700 (PDT) Date: Thu, 7 May 2026 18:23:31 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , linux-mm@kvack.org Subject: [PATCH v5 23/28] mm: add free_frozen_pages_zeroed Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add free_frozen_pages_zeroed(page, order) to free a frozen page while marking it as zeroed, so the next allocation can skip redundant zeroing. An FPI_ZEROED internal flag carries the hint through the free path. PageZeroed is set after __free_pages_prepare() clears all flags, so the hint survives on the free list. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/gfp.h | 1 + mm/internal.h | 1 - mm/page_alloc.c | 16 ++++++++++++++++ 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index ee35c5367abc..e0d5743de68d 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -384,6 +384,7 @@ __meminit void *alloc_pages_exact_nid_noprof(int nid, s= ize_t size, gfp_t gfp_mas extern void __free_pages(struct page *page, unsigned int order); extern void free_pages_nolock(struct page *page, unsigned int order); extern void free_pages(unsigned long addr, unsigned int order); +void free_frozen_pages_zeroed(struct page *page, unsigned int order); =20 #define __free_page(page) __free_pages((page), 0) #define free_page(addr) free_pages((addr), 0) diff --git a/mm/internal.h b/mm/internal.h index a01bc2c85cf2..4b9ced7fdca2 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -905,7 +905,6 @@ struct page *__alloc_frozen_pages_noprof(gfp_t, unsigne= d int order, int nid, #define __alloc_frozen_pages(...) \ alloc_hooks(__alloc_frozen_pages_noprof(__VA_ARGS__)) void free_frozen_pages(struct page *page, unsigned int order); -void free_frozen_pages_zeroed(struct page *page, unsigned int order); void free_unref_folios(struct folio_batch *fbatch); =20 #ifdef CONFIG_NUMA diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 954d86655f77..ca3b2072be21 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -90,6 +90,13 @@ typedef int __bitwise fpi_t; /* Free the page without taking locks. Rely on trylock only. */ #define FPI_TRYLOCK ((__force fpi_t)BIT(2)) =20 +/* + * The page contents are known to be zero (e.g., the host zeroed them + * during balloon deflate). Set PageZeroed after free so the next + * allocation can skip redundant zeroing. + */ +#define FPI_ZEROED ((__force fpi_t)BIT(3)) + /* prevent >1 _updater_ of zone percpu pageset ->high and ->batch fields */ static DEFINE_MUTEX(pcp_batch_high_lock); #define MIN_PERCPU_PAGELIST_HIGH_FRACTION (8) @@ -3038,6 +3045,9 @@ static void __free_frozen_pages(struct page *page, un= signed int order, if (!__free_pages_prepare(page, order, fpi_flags)) return; =20 + if (fpi_flags & FPI_ZEROED) + __SetPageZeroed(page); + /* * We only track unmovable, reclaimable and movable on pcp lists. * Place ISOLATE pages on the isolated list because they are being @@ -3076,6 +3086,12 @@ void free_frozen_pages(struct page *page, unsigned i= nt order) __free_frozen_pages(page, order, FPI_NONE); } =20 +void free_frozen_pages_zeroed(struct page *page, unsigned int order) +{ + __free_frozen_pages(page, order, FPI_ZEROED); +} +EXPORT_SYMBOL(free_frozen_pages_zeroed); + void free_frozen_pages_nolock(struct page *page, unsigned int order) { __free_frozen_pages(page, order, FPI_TRYLOCK); --=20 MST From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E21C639B495 for ; Thu, 7 May 2026 22:23:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192627; cv=none; b=AS2rAzoHDFqalsVEwlwNgvtyondLkbbP20JnP9uR92yeoZttKeeDrmgzyfuSWhhg9e61J5Q0Bh5e/05tEkn1f4H6R6tHcJ1/IwP54eKGSub+CWxS7dYxc5R92W1bn42DJL1Yjmu8uyIZeb24N2q8nsOXx0cdc56iPk4Tl2EYOjc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192627; c=relaxed/simple; bh=33jBXpQG8iQmZVU25bClBpFXphW8Qc7XbIRgidzWh2g=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=ZglflYsV4mub93IigO7925cQOKxrr92lIJr5XqnOiQB5d9VYnLa3+mGDbxs3012gX6kMTLJD1swei2JEAgxzS0wDyrkjc3YTm2iUW8H8OaISAR8OMZZg/BpPrA570UOtVw+t+xFv5z9gXnt7VpBYjjBgyZpfgVn3ma6dZGuUf7U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=ChBP/YhM; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=Rnxg18DC; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ChBP/YhM"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="Rnxg18DC" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192621; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=c6IynAVVs9vsrhP+5T0GREErRceOWiVny1P8BCgsPGY=; b=ChBP/YhM/7god6PztgD9eEIEoYp4lXWYuXors9eKfzZbAAYtqobwx4hs1VMxC0NRfPvrM8 /mw3gQDy11IrWUQnkZpXTV9bW98V5khjzXpiH828OQw5R1zzP51oIjK3z+5+mIkcP1FD7+ t8XMRfxcFXrV7M9zelfv+ayo503icdk= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-210-5Q1PxuZYMyGEAhGF2ca81Q-1; Thu, 07 May 2026 18:23:39 -0400 X-MC-Unique: 5Q1PxuZYMyGEAhGF2ca81Q-1 X-Mimecast-MFC-AGG-ID: 5Q1PxuZYMyGEAhGF2ca81Q_1778192618 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-488bd1ee9e7so11861785e9.1 for ; Thu, 07 May 2026 15:23:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192618; x=1778797418; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=c6IynAVVs9vsrhP+5T0GREErRceOWiVny1P8BCgsPGY=; b=Rnxg18DCu8A0dUTMEXoH+0erZNxH5OyisSSpegn2XUt1i0cPXUYmpLeln8XMV2x9m/ x97cOzskJ6FIh0mxzDoGDboZu+AvXdJ/PjbMVTh32PVQg6x6QGvk3nDTzMQ9r4xlvn9o iunLtCDSijJSwCht6R8FtD1WOFzGP2TLgNFXJIZxrjCy6At+LJKjbbghiqu0xF3ygJz3 Af+RCoExsOtrIwLfhP24yF8GGZi9KR4L3OlAcYcr8Ouft+y1e9/ffpKrG0nbzqaTTsg/ 7A5r1b/7Y+ZA5RiHjoDEguqTqEn0OzcfUC/PuyikZrymRS6R0/goHO84HG1fgkY9kZop LB6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192618; x=1778797418; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=c6IynAVVs9vsrhP+5T0GREErRceOWiVny1P8BCgsPGY=; b=CDpX6RuMdhcKlxADc/BzKqzsuA1/Jk6rMB5vC2sW1oXYlIq4dOB/PC2yooS6vrdYk3 jJKWN+ksJVEhjFN2JIOUNcTblabbjp+NEND38rRvCTFK7kbyJSdc+uxxJjXFRnN4F2Ag i1Yb4g/X/B+CpeJtYa7nAe+iBtLLKQUZ8dTCSsNLjDktrYCfJm53A209FWQSDBq2+jEl HL9cHz3zhfkoTd8xzEO/g9DsBLHwfIjI27A2ajyJWNmwXMqD1e5TsCyvM3Wba/MGtUB+ GRrsL9O6PsupNd8kHZh1l5opQvV1Ma2Bb63fRdxVfZNPoh40cFe2nKtgzer0q+ZKmZbq wM+w== X-Gm-Message-State: AOJu0YwATWtu666+Lob8F67G5vbuPbMb8mqZOY1aKoNigUcjP9wzCP8u re8JuigJxCa5NIwI8jqPYRQr7Aj1e8cT5XrTAnvR6lKRl+o42+Zy71lcISsqonUnNBI3UolUCaZ COtQ2tPbEk6kj4fOVopjm0uhD9EnSYkd8dXRaRlFQ9EXfCj7JgNFRefvM+bwG7atNWLysWwqNAh GhMS0XSD4np/sMpkx9wSVfB5HMZrDLJS32ztpfR6P0frg= X-Gm-Gg: AeBDieuzFxWBy7R3Hx0rfN7HQlF0d81UJylM3aN1GM/bbyIcRkVWxfs61RQPfvKNzEJ in+VWa5/2KK6clYbKwvoV86z8AVbisP0GlVnWVp2mBHn/aXyHu+ZGgeOHKAFYvMOrUgxhkcTKvM kfghNnQ7l1UYlhujs9OHICkh9pe/U7zoxj6L5WH8uNU5Z38SoLClIK8hJI+kcJZSjgvzjaUiNRs eDRIB6lxhAVW6Uiz2VIyBanN+gVOJcCd3QxAMr6cKcIdNKsd13w9MKkQaQYIiAzb4MjwTQAFjJ2 cLg2wDcy1x/LoPaREljOeHdmH3N8OGE+uz03s2RZazK2REW9rZDENEfhDjXC+xiJLvnuX0otVav uHEbDdCtY+YGXitF57xcQqr2Zg+RSzfqyEocr2cy/ X-Received: by 2002:a05:600c:4592:b0:489:1ff1:74d3 with SMTP id 5b1f17b1804b1-48e51f3c4cemr161117875e9.20.1778192618036; Thu, 07 May 2026 15:23:38 -0700 (PDT) X-Received: by 2002:a05:600c:4592:b0:489:1ff1:74d3 with SMTP id 5b1f17b1804b1-48e51f3c4cemr161117185e9.20.1778192617352; Thu, 07 May 2026 15:23:37 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48e53895effsm150494415e9.3.2026.05.07.15.23.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:23:36 -0700 (PDT) Date: Thu, 7 May 2026 18:23:34 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , Barry Song , Axel Rasmussen , Yuanchu Xie , Wei Xu , linux-mm@kvack.org Subject: [PATCH v5 24/28] mm: add put_page_zeroed and folio_put_zeroed Message-ID: <83ca7d34d23392d2c830e0417b1d38afbd23fb56.1778192416.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add put_page_zeroed() / folio_put_zeroed() for callers that hold a reference to a page known to be zeroed. If this drops the last reference, the page goes through __folio_put_zeroed() which calls free_frozen_pages_zeroed() so the zeroed hint is preserved. If someone else still holds a reference, the hint is simply lost - this is best-effort. This is useful for balloon drivers during deflation: the host has already zeroed the pages, and the balloon is typically the sole owner. But if the page happens to be shared, silently dropping the hint is safe and avoids the need for callers to check the refcount. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/mm.h | 12 ++++++++++++ mm/swap.c | 18 ++++++++++++++++-- 2 files changed, 28 insertions(+), 2 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 541d36e5e420..1c6bf82a967a 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1640,6 +1640,7 @@ static inline struct folio *virt_to_folio(const void = *x) } =20 void __folio_put(struct folio *folio); +void __folio_put_zeroed(struct folio *folio); =20 void split_page(struct page *page, unsigned int order); void folio_copy(struct folio *dst, struct folio *src); @@ -1817,6 +1818,17 @@ static inline void folio_put(struct folio *folio) __folio_put(folio); } =20 +static inline void folio_put_zeroed(struct folio *folio) +{ + if (folio_put_testzero(folio)) + __folio_put_zeroed(folio); +} + +static inline void put_page_zeroed(struct page *page) +{ + folio_put_zeroed(page_folio(page)); +} + /** * folio_put_refs - Reduce the reference count on a folio. * @folio: The folio. diff --git a/mm/swap.c b/mm/swap.c index bb19ccbece46..5d05a463b46a 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -94,7 +94,7 @@ static void page_cache_release(struct folio *folio) unlock_page_lruvec_irqrestore(lruvec, flags); } =20 -void __folio_put(struct folio *folio) +static void ___folio_put(struct folio *folio, bool zeroed) { if (unlikely(folio_is_zone_device(folio))) { free_zone_device_folio(folio); @@ -109,10 +109,24 @@ void __folio_put(struct folio *folio) page_cache_release(folio); folio_unqueue_deferred_split(folio); mem_cgroup_uncharge(folio); - free_frozen_pages(&folio->page, folio_order(folio)); + if (zeroed) + free_frozen_pages_zeroed(&folio->page, folio_order(folio)); + else + free_frozen_pages(&folio->page, folio_order(folio)); +} + +void __folio_put(struct folio *folio) +{ + ___folio_put(folio, false); } EXPORT_SYMBOL(__folio_put); =20 +void __folio_put_zeroed(struct folio *folio) +{ + ___folio_put(folio, true); +} +EXPORT_SYMBOL(__folio_put_zeroed); + typedef void (*move_fn_t)(struct lruvec *lruvec, struct folio *folio); =20 static void lru_add(struct lruvec *lruvec, struct folio *folio) --=20 MST From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3EDD8397E8E for ; Thu, 7 May 2026 22:23:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192630; cv=none; b=fW7plGqFqt9QeBEgFz6uLtSs2hIGnwoQOk0CFg1mytCBbtdgQ/J5Td+krE+ACo9QDF6MnE+hxQdDhLEjwxD2hVOk5pXAT842Mkqa0JdRXPwctwzW7twXMP9yr9RWVNQZg6D+tCV9LtpMpCyeN9fs5/dVjGUsNMAxN86CME4QX/I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192630; c=relaxed/simple; bh=qVMVxmVny/cl+4bAkcimdBPo6tV/J+GAvcnleNWWJIw=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=kdGDsbq35NGQ6uHXIHRM/gGt67tBCPdEcWhMIheaVT9qTqmjhb24jlQbbsOGWFvc2kBEZuRXh18SI3wLLkLvQWLK25FBpTtNuR+xfijJFrl9WvSO6ly/8nq7sLFE+YAkqBP4TiPq5tvKg3I9v9NlYpdn7MgR1BC09wPa2NK5LYw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=hbsWd+Qn; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=YkkBlMzy; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="hbsWd+Qn"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="YkkBlMzy" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192623; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=2Si0dv8bSUlUiRXwl4HtcwIXYc0lxxBk3yMuAVuTy5o=; b=hbsWd+QnYk9B1ifZ7OMhion9DmeocEWEP0zrrYIXWpO0fUC3aUW6gWMHNzlqeajzDG2RGL yxMD1zwP6uUiQtwZLXm9e4hbnukAwzbLJW7yNQc9tehyRXpbYig+c4+DSyJ/8bJuVfl1RA Rx+BKazG8eZk2rEWO7MuVDpe9XWLXXI= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-520-HUOPDvLiMjeQbAerrnkyqA-1; Thu, 07 May 2026 18:23:42 -0400 X-MC-Unique: HUOPDvLiMjeQbAerrnkyqA-1 X-Mimecast-MFC-AGG-ID: HUOPDvLiMjeQbAerrnkyqA_1778192621 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-48d046fac74so7248435e9.3 for ; Thu, 07 May 2026 15:23:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192621; x=1778797421; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=2Si0dv8bSUlUiRXwl4HtcwIXYc0lxxBk3yMuAVuTy5o=; b=YkkBlMzyv6PCO+yNd9OsHZi4PpcbnoTwwGJFGi1BXVizpHCXFARlhCt/hQiaHlu6c5 se3PbZ2E3wpTdqw5m69I6oBWVqmFWD1L2vNR5qnM4CBm6Bl83bpPCGp2FuUew1lC65SJ aVa+B0Yl9lBP/cqothnuhTYHykrsIoFe2BSe3cQJK33rorvpjTXo+yB+O25r718HFc7J tKHZF7rUq+2xX2LsS0/hslzcKuIFONk6GaCrxEM/+dWZeGxwsXF71R7KVoOn+Sdd6Xha s28LOTz/cXnMhzI0MuQ9z08fAbZrhTNR33cBTcMK9Yf0fSWnoQ58ULDFCWeyF8ffjk8I qgqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192621; x=1778797421; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=2Si0dv8bSUlUiRXwl4HtcwIXYc0lxxBk3yMuAVuTy5o=; b=qzdYN4a7eTMZ/jBE8HaXiwTa0P/Zyu2u4zrC6k/P0aYgqF1VReERJwG7I8ZLk74wvy D8Os3EKGDVo7FUyNxwiBzcPOkI4BQtzv4XQkXay15PkovfD1gzyvRXc/mfZaTI09dDu7 56qcc0PdFFKsgmX60ojKVE/O6J/itSNViY9siUAfEVcE6tUhSDRBQqAifcZ++V0IdjNl GCUTiR7sB3/RK/f/TFIl32hvU0YNtHDbsR29jymDtnqBRr3sTbczxVn+1Mnoci7a3RLx oaZWZ3NMwZN3JFcW9la6vPZ6SRtlnSqTyi0xUQTDpdhTUSfVB6942K4bVTAoOE4pworz P1Cw== X-Gm-Message-State: AOJu0YwkN1qwRwJblujgN1CUayBa/94mpIJFWOsR3hZykelK/BcaWCBk MIgupK3ZBCy3xJ31BfyAO5dAHWKv6WVko2NZUSdiUhfBBlBz6wiG7ff4YJkDUww2Nf3PsNm3Vcf cAgt4rwJwjTIMSk7BqvAjG9DTKxAsUt7FOQQ+g3MlQ73L8sQU3U6VMc6OwjgwB3ai04LrHcEN+X zQVthvFIOiQD8LlGwlMdyIvK5oakekBubjAVBP1rcL92s= X-Gm-Gg: AeBDietT5ttBdYgvxEfA+zxuXCjm4yKpurdfRFXezfQQMQYkeoB8kINO8V4SeMAmyT9 +53GrDwGM8yBRBWSfJ4bKF9lAGx5zexAt/xw6kZQbWWa1KGIFXy3C+SYx+nJ0Ree3rQurJ4kg4W xMzUEYwDRfYYklTesGLIVEosp0i4Dpy/0iwkEqQL4UJNboknwF1LKI5sUOPZsdG6p8m4WJfww7M Bxk6jl9WpclttbhCKcYWiFbe4l4t++xZ6tpXCjQClAelv3YrurAULkqLdm3x1+hUilClqQGKrF7 vYsUlUQQf3syQ7kZUrPyrK0swPSOCVJa6pF/B9l46g/qNSTT+g6ASrASEd96oTXac4Zg31FISqh sX3Q8balwJKdfgdSs4aCMgSw+43nxUE32B/4ONlJSjkCSlZUiZJQ= X-Received: by 2002:a05:600c:4881:b0:485:3ff1:d5ed with SMTP id 5b1f17b1804b1-48e51e0a697mr100906435e9.1.1778192620411; Thu, 07 May 2026 15:23:40 -0700 (PDT) X-Received: by 2002:a05:600c:4881:b0:485:3ff1:d5ed with SMTP id 5b1f17b1804b1-48e51e0a697mr100906125e9.1.1778192619693; Thu, 07 May 2026 15:23:39 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48e67455581sm2917475e9.4.2026.05.07.15.23.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:23:39 -0700 (PDT) Date: Thu, 7 May 2026 18:23:37 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: David Hildenbrand , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , virtualization@lists.linux.dev Subject: [PATCH v5 25/28] virtio_balloon: implement VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When the device offers DEVICE_INIT_ON_INFLATE (bit 7), the device initializes inflated pages and returns a per-page bitmap indicating which pages were successfully initialized. The driver appends a device-writable bitmap buffer to each inflate descriptor chain via virtqueue_add_sgs. After the host acknowledges, the driver checks bitmap bits (bounded by used_len) and marks pages with SetPageZeroed. tell_host() returns used_len from virtqueue_get_buf(). Bitmap reads are bounded: fill_balloon() and virtballoon_migratepage() only trust bits within the used_len range. On deflate, release_pages_balloon checks PageZeroed per page and uses put_page_zeroed for pages the host initialized, propagating the zeroed hint to the buddy allocator. If inflate_vq has fewer than 2 descriptors, the feature is cleared at probe time. If PAGE_POISON is negotiated with non-zero poison_val, the feature is cleared in validate(). See the virtio spec change: https://lore.kernel.org/all/9c69b992c3dd83dfef3db92cd86b2fd8a0730d48.177773= 1396.git.mst@redhat.com Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- drivers/virtio/virtio_balloon.c | 97 +++++++++++++++++++++++++---- include/uapi/linux/virtio_balloon.h | 1 + 2 files changed, 87 insertions(+), 11 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index e99ffbbdd2bd..708b0c344ae9 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -120,6 +120,9 @@ struct virtio_balloon { struct virtqueue *reporting_vq; struct page_reporting_dev_info pr_dev_info; =20 + /* Bitmap returned by host for DEVICE_INIT_ON_INFLATE */ + DECLARE_BITMAP(inflate_bitmap, VIRTIO_BALLOON_ARRAY_PFNS_MAX); + /* State for keeping the wakeup_source active while adjusting the balloon= */ spinlock_t wakeup_lock; bool processing_wakeup_event; @@ -180,20 +183,30 @@ static void balloon_ack(struct virtqueue *vq) wake_up(&vb->acked); } =20 -static void tell_host(struct virtio_balloon *vb, struct virtqueue *vq) +static unsigned int tell_host(struct virtio_balloon *vb, struct virtqueue = *vq) { - struct scatterlist sg; + struct scatterlist sg_out, sg_in; + struct scatterlist *sgs[] =3D { &sg_out, &sg_in }; unsigned int len; =20 - sg_init_one(&sg, vb->pfns, sizeof(vb->pfns[0]) * vb->num_pfns); + sg_init_one(&sg_out, vb->pfns, sizeof(vb->pfns[0]) * vb->num_pfns); =20 - /* We should always be able to add one buffer to an empty queue. */ - virtqueue_add_outbuf(vq, &sg, 1, vb, GFP_KERNEL); + if (vq =3D=3D vb->inflate_vq && + virtio_has_feature(vb->vdev, + VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE)) { + unsigned int bitmap_bytes; + + bitmap_bytes =3D DIV_ROUND_UP(vb->num_pfns, 8); + bitmap_zero(vb->inflate_bitmap, vb->num_pfns); + sg_init_one(&sg_in, vb->inflate_bitmap, bitmap_bytes); + virtqueue_add_sgs(vq, sgs, 1, 1, vb, GFP_KERNEL); + } else { + virtqueue_add_outbuf(vq, &sg_out, 1, vb, GFP_KERNEL); + } virtqueue_kick(vq); =20 - /* When host has read buffer, this completes via balloon_ack */ wait_event(vb->acked, virtqueue_get_buf(vq, &len)); - + return len; } =20 static int virtballoon_free_page_report(struct page_reporting_dev_info *pr= _dev_info, @@ -290,8 +303,37 @@ static unsigned int fill_balloon(struct virtio_balloon= *vb, size_t num) =20 num_allocated_pages =3D vb->num_pfns; /* Did we get any? */ - if (vb->num_pfns !=3D 0) - tell_host(vb, vb->inflate_vq); + if (vb->num_pfns !=3D 0) { + unsigned int used_len =3D tell_host(vb, vb->inflate_vq); + + if (virtio_has_feature(vb->vdev, + VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE)) { + unsigned int i; + unsigned int valid_bits =3D used_len * 8; + + for (i =3D 0; i < vb->num_pfns; + i +=3D VIRTIO_BALLOON_PAGES_PER_PAGE) { + unsigned int pfn, j; + bool zeroed =3D true; + + if (i + VIRTIO_BALLOON_PAGES_PER_PAGE > valid_bits) + break; + for (j =3D 0; j < VIRTIO_BALLOON_PAGES_PER_PAGE; j++) { + if (!test_bit(i + j, vb->inflate_bitmap)) { + zeroed =3D false; + break; + } + } + if (zeroed) { + pfn =3D virtio32_to_cpu(vb->vdev, + vb->pfns[i]); + __SetPageZeroed(pfn_to_page(pfn >> + (PAGE_SHIFT - + VIRTIO_BALLOON_PFN_SHIFT))); + } + } + } + } mutex_unlock(&vb->balloon_lock); =20 return num_allocated_pages; @@ -304,7 +346,12 @@ static void release_pages_balloon(struct virtio_balloo= n *vb, =20 list_for_each_entry_safe(page, next, pages, lru) { list_del(&page->lru); - put_page(page); /* balloon reference */ + if (PageZeroed(page)) { + __ClearPageZeroed(page); + put_page_zeroed(page); + } else { + put_page(page); + } } } =20 @@ -851,7 +898,25 @@ static int virtballoon_migratepage(struct balloon_dev_= info *vb_dev_info, /* balloon's page migration 1st step -- inflate "newpage" */ vb->num_pfns =3D VIRTIO_BALLOON_PAGES_PER_PAGE; set_page_pfns(vb, vb->pfns, newpage); - tell_host(vb, vb->inflate_vq); + { + unsigned int used_len =3D tell_host(vb, vb->inflate_vq); + + if (virtio_has_feature(vb->vdev, + VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE) && + used_len >=3D DIV_ROUND_UP(VIRTIO_BALLOON_PAGES_PER_PAGE, 8)) { + unsigned int j; + bool zeroed =3D true; + + for (j =3D 0; j < VIRTIO_BALLOON_PAGES_PER_PAGE; j++) { + if (!test_bit(j, vb->inflate_bitmap)) { + zeroed =3D false; + break; + } + } + if (zeroed) + __SetPageZeroed(newpage); + } + } =20 /* balloon's page migration 2nd step -- deflate "page" */ vb->num_pfns =3D VIRTIO_BALLOON_PAGES_PER_PAGE; @@ -956,6 +1021,12 @@ static int virtballoon_probe(struct virtio_device *vd= ev) if (err) goto out_free_vb; =20 + if (virtio_has_feature(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE) && + virtqueue_get_vring_size(vb->inflate_vq) < 2) { + err =3D -ENOSPC; + goto out_del_vqs; + } + if (!virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_DEFLATE_ON_OOM)) vb->vb_dev_info.adjust_managed_page_count =3D true; #ifdef CONFIG_BALLOON_MIGRATION @@ -1171,6 +1242,9 @@ static int virtballoon_validate(struct virtio_device = *vdev) else if (!virtio_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_POISON)) __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_REPORTING); =20 + if (virtio_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_POISON) && + !want_init_on_free()) + __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE); __virtio_clear_bit(vdev, VIRTIO_F_ACCESS_PLATFORM); return 0; } @@ -1182,6 +1256,7 @@ static unsigned int features[] =3D { VIRTIO_BALLOON_F_FREE_PAGE_HINT, VIRTIO_BALLOON_F_PAGE_POISON, VIRTIO_BALLOON_F_REPORTING, + VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE, }; =20 static struct virtio_driver virtio_balloon_driver =3D { diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virti= o_balloon.h index ee35a372805d..d129736cc3a8 100644 --- a/include/uapi/linux/virtio_balloon.h +++ b/include/uapi/linux/virtio_balloon.h @@ -37,6 +37,7 @@ #define VIRTIO_BALLOON_F_FREE_PAGE_HINT 3 /* VQ to report free pages */ #define VIRTIO_BALLOON_F_PAGE_POISON 4 /* Guest is using page poisoning */ #define VIRTIO_BALLOON_F_REPORTING 5 /* Page reporting virtqueue */ +#define VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE 6 /* Device initializes pa= ges on inflate */ =20 /* Size of a PFN in the balloon interface. */ #define VIRTIO_BALLOON_PFN_SHIFT 12 --=20 MST From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 462A23A9625 for ; Thu, 7 May 2026 22:23:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192632; cv=none; b=XWdfDzcqBLYqLSehjNxJa4abNIfD5tj9S/5LbW8dqDz0Cfu+pFXGXiw6Q3QXZZ88WZOxUAZB6Au/dkTVh75Ac6ZurGxM/PnNedgcHkqswMlbyropSUpCa7QdCtHmwV9IKb1DgIR3wRSjR+QyR4wuw1QBDVPIjn3iaYmpj7ZhQUY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192632; c=relaxed/simple; bh=mQqx6N7f4HKujpbDEae/MRGG23U+Ado82zzAxmBrUmI=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=iZ97llygY6Qw98Vo03tbgZtKo86GsR4KzJczXSw3AoHtnauzkB1uLC++m5bUsMZz5Xi+Q1XV9sLXwbRdyFro39YSwlcudWQO51QOHcVL/xolDH0gwljSqctqfNct4q69kX1HgrFHfMl+sIOruEgcys4AzCKB2dY3Suj2F668kws= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=ikEesiH4; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=IIg4ENo1; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ikEesiH4"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="IIg4ENo1" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192625; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=MpBn8Jg+yemSfT/rKcYB8AJ8VmedmgkSc9fIviOsi3s=; b=ikEesiH4aJ/nIOpNNaMGgO8oRGbgVzSiAeCLMh5tXxVlfJGPGksEWeFRuCBbIkawl5prIq NBX2ofqdqbfCAlTYPqwj1FGzY5wTqG+pht07DtBlisDSSCZx/6C4E9wRLi5rpvONtJaKUB H1cFSjQ+QtYwGUBaL3zskz/Ti7sE1Ag= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-641-5UA7EUNUNB-4_ftyXDy0Tg-1; Thu, 07 May 2026 18:23:44 -0400 X-MC-Unique: 5UA7EUNUNB-4_ftyXDy0Tg-1 X-Mimecast-MFC-AGG-ID: 5UA7EUNUNB-4_ftyXDy0Tg_1778192623 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-43d789cebcfso1679853f8f.1 for ; Thu, 07 May 2026 15:23:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192623; x=1778797423; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=MpBn8Jg+yemSfT/rKcYB8AJ8VmedmgkSc9fIviOsi3s=; b=IIg4ENo1FSvvS8jpFolifWd8thcHGhV5q+ZpjTHdhCCjAfkHFpmQyI9UDgd572nT++ NOL666pXt435sfNsAhfwGmLFJBFZK5Chl88zh0TYpbh+aD2MOKH6PQy3EWwr4H+BdNnC 1xynTQSj8P7cCMZ542CyVDrp+I6uiEm2BfUqcxccIK4WgUseLWP8Pc+OcxzMY2ZIUI3b Hh2kY3sMhFdTbhzqH/NSHuEdZ3o9xNgiHNbnsBoFkEOAPWTxXOQWjdmVQZYiGfGEkTw7 GJJu61xSHwDuQpGZEfHtJlng/zKs8bGIwu+aKALLsAKrBFBP0bgSvGxW3CI8Jb9PsQxz lStg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192623; x=1778797423; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MpBn8Jg+yemSfT/rKcYB8AJ8VmedmgkSc9fIviOsi3s=; b=HYqV8U34mIDRhGik0DAJb0Oe/+CpdLUkP8SXmPVstePpnwduMevQ5Ig8DOnIfCNpAW MCPeUsA9P8KC3M18K12hDrqbXd17dkzh53wFir/ib8fDarbgy7a7aNkpYvCRlM66g6// /1mpFa6TLLk00IBttsVQpOSiQoJHKvUZzWiyAjDxDCRBQG43YvLvDTOB2IdUfZ7SE624 /2v5Kip+1KvWwu06JG0NoUsuyL5yOL+h+5awijbc1qM4Tce5Jk81TGdEHNY+W85z7kud JWzKWCgeoeTAbUlHK9PoqXcLYbtiaMcMfW34CVtS8gKnGUHAVgrYRGijLJuoCI01rxe4 Y/Fw== X-Gm-Message-State: AOJu0Yw/3axxPJPcrJVZ8ltvaN1KeRijPPCXmssPat9ZBqYUYGqTD9/F fri/rTAOjqdHLsAmQI3xagUD5axCmDM66MK9wDxP+HhJUgASgQc6w6Y2yCm73xbOrERa9SwKqHl dcU5zA//BDdBWYnh3nRWttgcoaBMBp6EJvkDpCMnDxEUzh4x70YVCV+MjkzELHnf2gg/2hW3eHq TUmTM1snEKBJV1oKyBK0cMY4o4aaAeNFEHaEyXzgEuEgU= X-Gm-Gg: AeBDiesZDVqXQLOT4wvbF4evFhHL+RXNixV/nDUGtqaIpOuWKSi6anOlaKu3IccGg3y oipukn9+qFoRQTxYLzefm4qN3CxKZOZ5oQyl1rAp7gO38zhjdhVjNKhCYSvn3lL9mFQ3JEpUJA1 9cYssFCHERkIHzF4n3qrVid0O/O/MebEyuEC1PcnI92R1rs/A8EU/gk9C3CWAEEkY5hbRQSp9Ho CG1DplbaPv93YIPagkHp1HqYzqrV6/HGLRCoANygKkRxRVleFiZtwpez5ghdDTEWHvIA63sWOxw KK75thK0rplWEKQWANpNuKIXNLS4EHURr7zBxeu3Xh2EJnT/ZtQ33DuR8pSNi1sEGlhmH8e6g52 ywxNfX0mNkSCytwhpy3h/y6RcYGcXZv23Yxs7QyNP X-Received: by 2002:a05:600c:a10a:b0:48d:50a:6ef4 with SMTP id 5b1f17b1804b1-48e5dffab24mr53046545e9.11.1778192622634; Thu, 07 May 2026 15:23:42 -0700 (PDT) X-Received: by 2002:a05:600c:a10a:b0:48d:50a:6ef4 with SMTP id 5b1f17b1804b1-48e5dffab24mr53046285e9.11.1778192622056; Thu, 07 May 2026 15:23:42 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48e538a517bsm166228835e9.4.2026.05.07.15.23.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:23:41 -0700 (PDT) Date: Thu, 7 May 2026 18:23:40 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: Andrew Morton , David Hildenbrand , linux-mm@kvack.org, virtualization@lists.linux.dev Subject: [PATCH v5 26/28] mm: balloon: use put_page_zeroed for zeroed balloon pages Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When a balloon page marked PageZeroed is freed during migration, use put_page_zeroed() to propagate the zeroed hint to the buddy allocator. Previously the hint was silently lost via plain put_page(). Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/balloon.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/mm/balloon.c b/mm/balloon.c index 96a8f1e20bc6..1bf7eb2642a9 100644 --- a/mm/balloon.c +++ b/mm/balloon.c @@ -324,7 +324,12 @@ static int balloon_page_migrate(struct page *newpage, = struct page *page, balloon_page_finalize(page); spin_unlock_irqrestore(&balloon_pages_lock, flags); =20 - put_page(page); + if (PageZeroed(page)) { + __ClearPageZeroed(page); + put_page_zeroed(page); + } else { + put_page(page); + } =20 return 0; } --=20 MST From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8A15E3A6B6E for ; Thu, 7 May 2026 22:23:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192633; cv=none; b=kfgPi33naFbuq5frmUq4fEQHIvKBzWGrTqA1VZJKTUUEec56TKhthF2yDvzc0YvJGEiGEc2cECw5IkT9/N6dkzbAPRB/TNCi2MJLPuOv0omKpQR7k7fU4sW/DmLEPmOZ5GUxGUX93RgKhHoOm7a+63mFuJhGnY7lXC0YvI0+lDM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192633; c=relaxed/simple; bh=AK9TO94T0KU+04u+dGjmV3b60mJRmYIjdCC4LEd/MMo=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=DDH+nTyxZd/AyL4/IqxbgVWgZDSwdujVKWhAcyoZg+as5J45y0Fxplm4wRZDX3+DFuVlbYuI/mItVRHeS/ibmXGZ6pIts2eNjoq3zaC0NXtkg8TH2yu8S8D0k0vuSFkNo4r1b340pE5t1ppyDfpP+EBNIG9RgmKynWyCrvDe7KQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=IX+vUPjS; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=kaBhtjWT; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="IX+vUPjS"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="kaBhtjWT" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192628; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=NRctwQPPX8xkJe678Z/hFoNL2OQQ834j5nPYy6NU4ac=; b=IX+vUPjSPpT3ns/5chTU5AJYlcpw4ydRk/vjtCrqQjg44aDOGh+4cObozOiFvEStaXFjgc gAzqS/dtj7gWEf5LY+Mloa0s9lrnfrZCV9B+BwHsUx6m3zNFEN5iP2Zd705vNokmA13lRS 0Hevwbt7uG5axVkcARusx0lDXLVXlyQ= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-634-kCNeRZ_9ObyQ4rc_dKmqqA-1; Thu, 07 May 2026 18:23:47 -0400 X-MC-Unique: kCNeRZ_9ObyQ4rc_dKmqqA-1 X-Mimecast-MFC-AGG-ID: kCNeRZ_9ObyQ4rc_dKmqqA_1778192625 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-488ba2f4094so9818975e9.1 for ; Thu, 07 May 2026 15:23:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192625; x=1778797425; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=NRctwQPPX8xkJe678Z/hFoNL2OQQ834j5nPYy6NU4ac=; b=kaBhtjWT1/Zk8C+i4z4pgzpMZTRNvc95xZioR9djRvEM1kvwJkintzLYjyVIgWHtm5 2gTXofTzJREUD0pA2LWTGLDfyBhchWHhMRELa1S0g/rN/lLxHMavbEXRDZAEr2KI18GJ UlkGzd4Xnoz+xI7++bCC2fjOl+K5OdZwSXMuvdvDv+YZYfPlplkfj4IJhvD0PpKsZ6Mw 11VH71LSmG+qz1AM7mfDiaSnCQgMztvKVKFBq0J5Qs+XeXfU17D5+JHHcCYGJuHYyNXN c5qmiG0GsjaB+NRKxMxSBoVQ27xxgxyobLCJGaB62HWrKKDnpdRkPvvSAAGfRvmcgkpU sOGA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192625; x=1778797425; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NRctwQPPX8xkJe678Z/hFoNL2OQQ834j5nPYy6NU4ac=; b=bXBqeRfk9AvuPFLghZkKY+KAiagri+7RcHANxyjUnuguLZGlVadQ5q1wnO/6SqEOVp syjvov2ZcTyqwSYk9nhRV0LgSz9IiYFdfj5hp7SX66tgomEqGt6BLoWmqDk+OBJuaBh8 C7H9S1/5WeIiM+y92LF7IF42EYU9gHlSAyFZhn0dVxEW1WBG2wSuEhswOx8YzyRTUPam EkHr0XV8EAuc4TQve31T7Rxz6EQaQWlwk3A5Zp0H2lQLm3Dt78RxKTv1hLBrgSZEIKPd wvPoRYA8yu2Rr4PCiq1PS/v/h/hmjCBmx+hPSUJGevjaCczMhZ/XJN2eS1dT4Zc9/Ctg 7Vqg== X-Gm-Message-State: AOJu0Yy0AJoBfjqfr23wtA+C9pOpmZU4CRsyd+rm5t3Ido0dYFjGrjlQ VygM3alklzZeEnDLzs2YdX0b0hJD835FnueOCFU8wXL1VuavA4IL8D9iaINAkz7pP0KA3itO+yk BBLJ82F3WmNyTjuoFx8e8vaX6ifLNex7Nd/Pdgu+woqQmriKyyqKJJTKpOdOaKj5+M7f98Xg2xV O066NBalkaWIGyEjLbxZfYXn9tjUpVeZiNqwKCW47MzBw= X-Gm-Gg: AeBDievOUI5IEBUFVF0A7S7u6GPEIvb9WXe9yGd3nC6RmMFHz0TZ8Ug+ibJY6POOCrg c1t0424CKreqMogYwz5wvDQoC4cRFlzaJ9jpEscu+bLKVxsPJC4T25/2Hk1bZXX7ZW5yUtAvrhz TPwBvVZnVWxQgRllUcvvYzQzFmbFf0nybWhetgOiyADMt+1CvWY2q0/D338sH5NZaQHmVjtou0f t2vyXlMl7Q0O0BFPn7wZ0noLZ0oEe0UlJIQmGsyG93uLxRSAgaWSepYsFfSGnQCsnT89KMyxz6Z ZKK3PHNROj4/v/RJekfT1d4yWRzV92mZkw/Y0YOPgeaBtib7V+xd00rceD9RN8Rii3rM8dauXqe JIlNuFgixEcq3j6xhZdzfxHos9H5Ehh5x36tYpcIY X-Received: by 2002:a05:600c:2e0c:b0:48e:526e:1011 with SMTP id 5b1f17b1804b1-48e526e106bmr81739885e9.25.1778192624894; Thu, 07 May 2026 15:23:44 -0700 (PDT) X-Received: by 2002:a05:600c:2e0c:b0:48e:526e:1011 with SMTP id 5b1f17b1804b1-48e526e106bmr81739685e9.25.1778192624331; Thu, 07 May 2026 15:23:44 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-48e65c01d7bsm4592885e9.8.2026.05.07.15.23.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:23:43 -0700 (PDT) Date: Thu, 7 May 2026 18:23:42 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: David Hildenbrand , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , virtualization@lists.linux.dev Subject: [PATCH v5 27/28] virtio_balloon: disable reporting zeroed optimization for confidential guests Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In confidential computing environments (TDX, SEV-SNP), the host is untrusted and may lie about zeroing reported pages. Disable host_zeroes_pages so the guest does not skip re-zeroing based on the used_len hint from an untrusted device. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- drivers/virtio/virtio_balloon.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index 708b0c344ae9..1eb9a6376038 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -18,6 +18,7 @@ #include #include #include +#include =20 /* * Balloon device works in 4K page units. So each page is pointed to by @@ -1117,6 +1118,8 @@ static int virtballoon_probe(struct virtio_device *vd= ev) #endif =20 vb->pr_dev_info.capacity =3D capacity; + vb->pr_dev_info.host_zeroes_pages =3D + !cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT); err =3D page_reporting_register(&vb->pr_dev_info); if (err) goto out_unregister_oom; --=20 MST From nobody Sat Jun 13 10:56:32 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AB8FD3AC0EE for ; Thu, 7 May 2026 22:23:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192634; cv=none; b=KKQf46CbdjRIO6pCyRxD8UcE0CKO0+KgNp7aQ/iMfTEChvNIN5sGts53yHDfWhMrhu2PBJT4MjVk/EOGK6jkZWenl8u47px2iHFr5OQhBjVLvXnuBFZxonSyRogjaji7JTWvrwnIgB46hz4Pwfuj0PdCxZ3BV3W30unmPMs6gnY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778192634; c=relaxed/simple; bh=LdhWu2S21/nEYuhXIUl1UmI3udDTmzhwlO4S09ZxBvw=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=NwQv8J0P/rlbqe1urAdWmuVuB84AwQbE5WM0QYVvMlLNpv9i0pxDf7flnxSBW4hm/hYMRIWb6jQlOoJuSnzN6/O47ZBSsfPrCtoXlt1sUKWb1ItEj/z28iIYfEmOp1ilLklY6l1N0DGWUTkvDzhyim3FWiKuVSzp0FeSKG3Jyus= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=PXlOTwrd; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=LQoJRP1k; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="PXlOTwrd"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="LQoJRP1k" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1778192631; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=dugSKqa2HC5Rex+/VdUGb27yVezCs4DCWJev6IAeQaE=; b=PXlOTwrdp1EqlFjncrOi+Tn5AHW3OFZ08RQiYnjIKjiYf/Pw1pMvZPyRMj2HryerRegmTc Csm+1USQExFZNvdLERccPQ3m7ndEI3QInV2OfFu5w03OUlmsEWBDKYT4HgchmaoaMlkSI5 OVBJtzkj2mGaaOpRUYkNYba7qofrzqs= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-658-0IOBL-DyOdWFpi60f8MhdA-1; Thu, 07 May 2026 18:23:50 -0400 X-MC-Unique: 0IOBL-DyOdWFpi60f8MhdA-1 X-Mimecast-MFC-AGG-ID: 0IOBL-DyOdWFpi60f8MhdA_1778192628 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-43ff19e54beso960657f8f.2 for ; Thu, 07 May 2026 15:23:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1778192627; x=1778797427; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=dugSKqa2HC5Rex+/VdUGb27yVezCs4DCWJev6IAeQaE=; b=LQoJRP1kKaYpNKAjZEPVZuCZ+xDVyCHJjtuk8nbNkpwmRAYv19tK1Rat9q9d6d0bbF sIhf+m84rEnoiKm17/5wGm1+jgBoO+3RdcX7E2yMDJXJkP+we2X/K63iy5s/vWyMw6Bj 2Jddfo6RoJfyA1syUpCu/QX/Z5R14BZtHGKYJatCbzCjDKY8nrgeBsdH3H94H9bhTYpl gbDRXIncQyk68VHrUWkaBMbMebgmvkENAHN9DYVNUfssLnPW5zQVuiVrL8GsV25GNSrF CHwBJM/wWPgxdAjw9wW6gaXM+6YWKx0G1gI5jk6U9AJXvxjgMh8F/ZB4YmUEwFMnI+wp BnLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778192627; x=1778797427; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dugSKqa2HC5Rex+/VdUGb27yVezCs4DCWJev6IAeQaE=; b=qy84G4nRPgrrhFxFeFXZTm+XOzFmDkwt9FswZXBy7taRijzESyq05WKig189x2FFwK urv7P4e7QbF67FMdXSUczvaPjmb5QZxRZR31JbaxZKbQ5BZqB8AAAoCgUsJVtIBbJbJI D/iDXWZZ5XA7ncS3KNQ+yPoZ9e6OS9qsnkAtAr6mKIH01HN9ZonZhAHdKxEUvUg/VaCM Oz8Nx8amrRY3D6cgzdq3+Xzb49S4zFUXAHHmNuZ8VNMngBnfAJJJGEwDWkxSqWo915a0 wZk6Zhe8JmsfCRQLTCOqDZmbpZnBAx32tejqdjuhzTt0ItmaOqjnIU3Dq5G7EPu11K9D Ks4w== X-Gm-Message-State: AOJu0YxI6P2o0DV/NqyvWV5e5IChRN/InW/nCTJgerEtzBk9ohbcwuzM ESfvJQSFY/PpRS4g6GseW+G8ZvpeDpifqYPVeGxeij1lJk74Bj2xLDhuG76YftpyIQ4W+g3ksCJ gVdzM99KBZGgA1RDXoZ48JGQlKoKkdHlj8b11TUJ/DGIiaP224/Db8o2VRlhQevDaZv4kJwiYK0 54NC2kunadBGkC+eATAw7BILKujeUbTY2m4hgyjX8l7dE= X-Gm-Gg: AeBDieuGUQ9IQxEfwINpxrbZvo8DGpQZBgls9eWRRoK/y67sYsVrG1rjbi8piUt1aHL l8j1WRT8tK4DZSkekYKG0eZUEw+MYVskxrW92KUP0wOKoTLb91bDkq0QdDLXWxA1N+IIRoXtG8F XmFotOPn5QUFMhJUoqaHOx+SI4X7wxOl8XMAgPZpU9shCwC+TMscx+yBnR57HedU8QzakqVyjuE m+KzkPLIp3iOtdVGzJ5K7k2+50pQuOIbER1/YtHXtjzIBY9yvGtffQc5soNFJp/vYoaidAG3znj kkqyPO7YUU+iqMEDhrVjlui2cYKEq5+v4hl/zcR0t2BpF9qk4R3EurgUwgynBgpw/Apbrb6WBKl btg7/uIFvBcgvqOotgoVoNVD6Pw2jqOR0uHGjkT+v X-Received: by 2002:a05:600c:a408:b0:48e:62ec:f264 with SMTP id 5b1f17b1804b1-48e676a5251mr1267065e9.17.1778192627249; Thu, 07 May 2026 15:23:47 -0700 (PDT) X-Received: by 2002:a05:600c:a408:b0:48e:62ec:f264 with SMTP id 5b1f17b1804b1-48e676a5251mr1266735e9.17.1778192626614; Thu, 07 May 2026 15:23:46 -0700 (PDT) Received: from redhat.com (IGLD-80-230-48-7.inter.net.il. [80.230.48.7]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4541721a05fsm1976259f8f.30.2026.05.07.15.23.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 07 May 2026 15:23:46 -0700 (PDT) Date: Thu, 7 May 2026 18:23:44 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: David Hildenbrand , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , virtualization@lists.linux.dev Subject: [PATCH v5 28/28] virtio_balloon: implement VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED (bit 6): when negotiated, the device guarantees it initializes reported pages (zeros, or poison_val if PAGE_POISON). The device signals success via the used length of each reporting_vq element. Gate host_zeroes_pages on both the feature bit and the actual page content: when PAGE_POISON is negotiated with poison_val !=3D 0, the device fills with poison bytes, not zeros. Clear the feature in validate() if REPORTING is not present or if running i= n a confidential computing environment (untrusted host). Renumber DEVICE_INIT_ON_INFLATE from bit 6 to bit 7 to make room. See the virtio spec change: https://github.com/oasis-tcs/virtio-spec/issues/244 Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh virtio_balloon: skip zeroing for host-zeroed reported pages Check per-page used length returned by the device to determine which reported pages were zeroed. If used_len matches the page size, the device successfully initialized the page (e.g. via MADV_DONTNEED), and we set the corresponding zeroed_bitmap bit. This requires no feature negotiation: existing devices return used_len=3D0 (the conservative "not zeroed" case), while updated devices return the page size on successful discard. host_zeroes_pages is set unconditionally so the page_reporting drain path checks the bitmap and marks matching pages as PG_zeroed in the buddy allocator. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- drivers/virtio/virtio_balloon.c | 30 ++++++++++++++++++++++++++--- include/uapi/linux/virtio_balloon.h | 3 ++- 2 files changed, 29 insertions(+), 4 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index 1eb9a6376038..f1ad842eb3d6 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -218,6 +218,8 @@ static int virtballoon_free_page_report(struct page_rep= orting_dev_info *pr_dev_i struct virtqueue *vq =3D vb->reporting_vq; unsigned int i, err; =20 + bitmap_zero(pr_dev_info->zeroed_bitmap, nents); + /* We should always be able to add these buffers to an empty queue. */ for (i =3D 0; i < nents; i++) { struct scatterlist one; @@ -237,10 +239,14 @@ static int virtballoon_free_page_report(struct page_r= eporting_dev_info *pr_dev_i =20 /* When host has read buffer, this completes via balloon_ack */ for (i =3D 0; i < nents; i++) { - unsigned int unused; + struct scatterlist *entry; + unsigned int used_len; =20 wait_event(vb->acked, - virtqueue_get_buf(vq, &unused)); + (entry =3D virtqueue_get_buf(vq, &used_len))); + if (used_len =3D=3D entry->length) + set_bit(entry - sg, + pr_dev_info->zeroed_bitmap); } } =20 @@ -1118,8 +1124,16 @@ static int virtballoon_probe(struct virtio_device *v= dev) #endif =20 vb->pr_dev_info.capacity =3D capacity; + /* + * With PAGE_POISON, device fills with poison_val not + * zeros; only treat as zeroed when poison_val is 0. + */ vb->pr_dev_info.host_zeroes_pages =3D - !cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT); + virtio_has_feature(vdev, + VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED) && + (!virtio_has_feature(vdev, + VIRTIO_BALLOON_F_PAGE_POISON) || + want_init_on_free()); err =3D page_reporting_register(&vb->pr_dev_info); if (err) goto out_unregister_oom; @@ -1245,9 +1259,18 @@ static int virtballoon_validate(struct virtio_device= *vdev) else if (!virtio_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_POISON)) __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_REPORTING); =20 + if (!virtio_has_feature(vdev, VIRTIO_BALLOON_F_REPORTING)) + __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED); + + /* Device fills with poison_val, not zeros; disable zeroed hint */ if (virtio_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_POISON) && !want_init_on_free()) __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE); + + if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) { + __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED); + __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE); + } __virtio_clear_bit(vdev, VIRTIO_F_ACCESS_PLATFORM); return 0; } @@ -1259,6 +1282,7 @@ static unsigned int features[] =3D { VIRTIO_BALLOON_F_FREE_PAGE_HINT, VIRTIO_BALLOON_F_PAGE_POISON, VIRTIO_BALLOON_F_REPORTING, + VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED, VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE, }; =20 diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virti= o_balloon.h index d129736cc3a8..cbaf18e0b17c 100644 --- a/include/uapi/linux/virtio_balloon.h +++ b/include/uapi/linux/virtio_balloon.h @@ -37,7 +37,8 @@ #define VIRTIO_BALLOON_F_FREE_PAGE_HINT 3 /* VQ to report free pages */ #define VIRTIO_BALLOON_F_PAGE_POISON 4 /* Guest is using page poisoning */ #define VIRTIO_BALLOON_F_REPORTING 5 /* Page reporting virtqueue */ -#define VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE 6 /* Device initializes pa= ges on inflate */ +#define VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED 6 /* Device initializes repo= rted pages */ +#define VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE 7 /* Device initializes pa= ges on inflate */ =20 /* Size of a PFN in the balloon interface. */ #define VIRTIO_BALLOON_PFN_SHIFT 12 --=20 MST