From nobody Mon Jun 8 10:56:13 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5159B35C197 for ; Fri, 29 May 2026 15:22:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068137; cv=none; b=KthVR2yRLaT/X2LxUAk5++HMQU0+qve9NdRvBcUoHDFffbPDbtcURi+3sySukGpNoDOcfjZueDXcY9cPC1zmXc6no2soyZhSlJi3syokHxOQaNN7ZZmuxsnnGUnOvdmuFW2iD6XxXhgUzBbdDRi4DZesYBwNN85vBSJPBt6/uxI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068137; c=relaxed/simple; bh=yI1iGkmiSzMFJtnw1lJVP5u1P/QKkpQHXLglYhfy84A=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=bhpqDpNfl2lva4f1xO6tHc/1E6M0uQc7/3u+xVnSRDG3Vkvr4CDGot3KO4yGq+0iLEbY/M2ZWkfD1C9jPyzpduf06KoLplvsuv05io04hECcf2F+wSLlL2hKX7Su9vBg69gMeBrEJpEcm3HRj30PWMtt2vLMOWvZ5v9YsYT9h20= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=bun+YZLm; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=ErFML1Dp; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="bun+YZLm"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="ErFML1Dp" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068135; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=315e7bSessxyz662R8s4IU7FTBDlj+mVbEZvVNtQrTU=; b=bun+YZLmH1ZBFsj6W0Rmk1LgtrZA2lLTD/L8h2mMfHXTxHL1BXK9xRhtXMQsUOzNUoW5/n ShQOWKDAkQelFsFz0R9E4jIEo+sOs27xWXJl+3z7ccU821qBqTm2RGm8Q2gOOjQslbbi7I bM4Pp9dTjy+zIAg1gae/q3JdWwrcc7c= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-91-kSo6d3MhPkOXNmdE6zXpRA-1; Fri, 29 May 2026 11:22:14 -0400 X-MC-Unique: kSo6d3MhPkOXNmdE6zXpRA-1 X-Mimecast-MFC-AGG-ID: kSo6d3MhPkOXNmdE6zXpRA_1780068133 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-48fd3dbd16aso115314155e9.0 for ; Fri, 29 May 2026 08:22:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068133; x=1780672933; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=315e7bSessxyz662R8s4IU7FTBDlj+mVbEZvVNtQrTU=; b=ErFML1DpHf3HqbIEY2F4v7/pz6xnERf0I7Wf+f30so2xHb7rz1FRJBRNsUzLNXnFzA 1LiMKQsj01bouizfzF4m6kySSezMkgzxJMEt6AAZ5QcUyvKGpNwnnyrPHVFojuPKcdD5 NysE5pbo0LjWJHu2elVVrZlb7jSUU1DZy/g72ladWLcoPd4fsl4YE1mUj/O59vvWBTNP V3aNmdCPVNorXEs9H8tU1sJhucLRaIay+4LXT4poSqzk9nQluIg8Nks441Z6Hi7YmPwV etkZQlhLvf3ano+DFq7DX2IVNsmYTa+6IaFSoCO7H93eriEo4KFXjI3/BU05246wN4s3 KMGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068133; x=1780672933; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=315e7bSessxyz662R8s4IU7FTBDlj+mVbEZvVNtQrTU=; b=kw1sT9KIgqBqV/Z0yC7kwutut70fcrfPXhsGnaoKi1hFEWdzN9GL4DvRzMI1jw7PVF lHYV6i+PbBVISqxGltXBvYR+LNC6yniuDyYo0McU5BUacvUCBSI24k5TaUGC4xAbHz8K Q+cOm6YM/snUvO/0BHChbU3dyEuykvmMJsLdburw+/WyGtDJ+cfApdWPKFJXpyqwbXsR R+3Yj8PsvQWFiPDEpCVUr5n06YuiHJyhA24TFzeF7S8W7Bp1AgKQGIgXwiF0TRpRfirE t2M7JL4iqdw9XEZ2QKL5DBPVuts2TlyrYJoTaYhTl1Xo5cixKV4W9XJXFmJGlSAGEE/S 4UjQ== X-Gm-Message-State: AOJu0YxsDWiS0BrWWboBQTWsegoxUMDw0vjX8OZ32sjxDb0axSDw7D36 MesxjRPRYJ/vFFF5HWJlEFFzQkasd58N4r93nEm/+//eO3BvtZYQizknW4yHxAkXZgDrHQqr8pk MlDjnw1B3KQwdCHQeUUIUER2OIPpJYE/lps9imMcsKz8qQrnPijJaVo+XwDEODI+gz6SyCsB8zW 35qR7WwQGGqOTVlDnq6Pm1Gi725LcGTGi1q2SrN//HL/g= X-Gm-Gg: Acq92OGC97zmH+45w1cce4gpmFS/+x2eK2Wyr/ZPKI05jBo1sNkDJDRTq0kHmZ11yxz SjsjBIYNdmywzVayAhHzEC8UwHTlCesWGJQBWbUKTPjAWKzkodDHA89o2V/Q32uPAhd4DP+RLv/ MlTxCHsYt8lmcbhHnho3CVvc6qyByIfxDP/apiZkeIDkgiqsSYXuyPJLylg0QLhkTLBtFPFmbxe Cwk3b36R8cVGpsj4zEM4EwagWDxAno15JzG4oemI+jycptCrr7BwYeKZ+Gn6/EN7DMw4SdiPL7G cLyyoVseot7rT+k303gsb+nLEEgocTaSFdeVNi6i62Tt5PHaFscwbuvHE2Mziv7HV+NbzIym/fB JgGLAEEuWRsbE4OutCTOa1epJFsVhgkRZKelnXHZas0KhaHuqUe1AEA== X-Received: by 2002:a05:600c:8b12:b0:490:2238:4021 with SMTP id 5b1f17b1804b1-490a29079a0mr2313485e9.8.1780068132933; Fri, 29 May 2026 08:22:12 -0700 (PDT) X-Received: by 2002:a05:600c:8b12:b0:490:2238:4021 with SMTP id 5b1f17b1804b1-490a29079a0mr2311035e9.8.1780068130983; Fri, 29 May 2026 08:22:10 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45ef34b47eesm4805302f8f.9.2026.05.29.08.22.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:22:10 -0700 (PDT) Date: Fri, 29 May 2026 11:22:05 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 01/37] mm: mempolicy: fix interleave index calculation Message-ID: <56ada5b2073a7e307e03fcb3f5327d0cf3dbeb8a.1780067977.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The NUMA interleave index was computed as two separate terms: *ilx +=3D vma->vm_pgoff >> order; *ilx +=3D (addr - vma->vm_start) >> (PAGE_SHIFT + order); This has two problems: 1. When vm_start is not aligned to the folio size, the subtraction before the shift lets low bits affect the result via borrows. 2. For file-backed VMAs, shifting vm_pgoff and the VMA offset independently loses carries between them, giving wrong chunk indices when vm_pgoff is not aligned to order. Combine into a single expression that adds vm_pgoff and the page-granularity VMA offset first, then shifts once: *ilx +=3D (vma->vm_pgoff + (addr >> PAGE_SHIFT) - (vma->vm_start >> PAGE_SHIFT)) >> order; For anonymous VMAs, vm_pgoff equals vm_start >> PAGE_SHIFT, so the vm_pgoff and vm_start terms cancel and the result reduces to addr >> (PAGE_SHIFT + order), same as before. For file-backed VMAs, the sum vm_pgoff + (addr >> PAGE_SHIFT) - (vm_start >> PAGE_SHIFT) gives the file page offset of addr. Shifting by order gives the correct file chunk index. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Reviewed-by: Gregory Price --- mm/mempolicy.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 4e4421b22b59..d139b074a599 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -2048,8 +2048,9 @@ struct mempolicy *get_vma_policy(struct vm_area_struc= t *vma, pol =3D get_task_policy(current); if (pol->mode =3D=3D MPOL_INTERLEAVE || pol->mode =3D=3D MPOL_WEIGHTED_INTERLEAVE) { - *ilx +=3D vma->vm_pgoff >> order; - *ilx +=3D (addr - vma->vm_start) >> (PAGE_SHIFT + order); + *ilx +=3D (vma->vm_pgoff + + (addr >> PAGE_SHIFT) - + (vma->vm_start >> PAGE_SHIFT)) >> order; } return pol; } --=20 MST From nobody Mon Jun 8 10:56:13 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4E82A367B70 for ; Fri, 29 May 2026 15:22:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068141; cv=none; b=FgP3/CppAEtP0/3kfqql/6+0eGzgd9hCBCXEDaQWm+FivYjOjFMzbaRFwuwcL0H8mcNjJ5Iw3U03xVYYub1A1k4vC3bDgk3QyfsKJmRkyhBYdAYwI5Y03TrWhzbFi3MsPtmVnFVwwm8KPJWUuuDv/kdQORArW8adX+SgUkwUZ4k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068141; c=relaxed/simple; bh=B9hCuvGWJTakJ4KIERi0AyOg+0/ybGKp5cfFr2jdtWo=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=OGXmEaLw9WW8IjypLVJ71vVPNSxs7MLif4V3IliFrwS2yGdvKtZWdo1wEjY96aqfJSK22ZU5FtPe8ueKQItvJhcoaKOfGwGjL5eKF6n6v1oOZLqCEfMoDYdJUedfA//aMDKBAegXPPE5L2m6PcZ+1HFa/xUujrfLpyP+xhHjgEs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=H9olcQq8; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=UBHsnKVu; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="H9olcQq8"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="UBHsnKVu" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068139; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=apkJ3vhwafShLYfDd+RnV0F+C4AxcxVXpW06K2eXGn8=; b=H9olcQq8GwknshUs0w5jZqcG6v8WeskuNJn2exfiI6ep0ps3cn9Hxftv/Yo0cqiTayajIC UsUeGloOYYhsRsqt0Cz5CwbIGExiX8qS+3a0ZByHXvQ+bxDwEVMLf5knUXoah4AgC3P+0S 7lfJB6x6gRoUOafidMqsjX7ZmHnIA2s= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-563-4jzV1ggwNjWhAxrYHbElLg-1; Fri, 29 May 2026 11:22:18 -0400 X-MC-Unique: 4jzV1ggwNjWhAxrYHbElLg-1 X-Mimecast-MFC-AGG-ID: 4jzV1ggwNjWhAxrYHbElLg_1780068137 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-44a3aee3813so6281386f8f.3 for ; Fri, 29 May 2026 08:22:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068137; x=1780672937; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=apkJ3vhwafShLYfDd+RnV0F+C4AxcxVXpW06K2eXGn8=; b=UBHsnKVueUqZIqC8svt2XHr80/+5jPYPceKcnFsTZQyizUrHUuJv/hM1tdUfBu1Ybv Oha/YZV4qfdUucXE4buzlR14h7N5PZC/1y+INcUTiKS+HMsDcptJ6NMiCq6Rn9HhqqrQ 2tUM4L2Blkn39j3sEy74FnbL7hBdcC06oLnnFJtaHk38by6SYrtCGWBkt9o3MI2q9jbp VGBSmr/qR/s8MTt0PBprq1LotyH6NS+WtvAGOlSkfDbOHQl9fVmEAArcWEePctklYe7i lHQNpxDSfdpHSveFf3BAzlEcxrriEZfdZnMAw+90Kbl9ZsxriN0VWAoO6eTT4X+WGpUa e3WQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068137; x=1780672937; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=apkJ3vhwafShLYfDd+RnV0F+C4AxcxVXpW06K2eXGn8=; b=Pb+Plh8rfpiP5ILiwSuDFAPPu5JPKWNgOunhXaEHNrJ4eZcHKDOw4IVaWPVGUfeNUC 2VeqiySt0pgV6JwmdLtPNKK1bhgKjuwwD42bBc7267w2VHIZz2MXlz45cUQwdPEqT6Xz DPgYd4s3XrwBZpRgvXFM8uPW/coVY8y9UE4k0+vE/v/+BfRS7i1U6IFYKn+38mzlS/YC 1jCKc8zJa/5ZVw5ygDEvpC5DE/WHRZYE2cFBGR5C6e3EgZP+QzREcH03OeMUR30Eofes iXN8cYUpEq32XRhvVNBBp4AqhtFFV6cpelvG1KXvpuXJjEPucJq6q2V5t/Q3wuUJcco7 PVug== X-Gm-Message-State: AOJu0YzznP0kpoFpdKOx4KC7mLp4O76l3kANcgBLJy9UFQ6bFJhTcJJU 1QLd/sgFAnokNefDTckD6x3ezyhA8/NDUXAl0rBkee5VVBm03UCXN+JSHm3/vzYlDGOFwyjAzYD UbJEMb9oszlDyTZ20YlO6U1m5h0qpOww8Ldm+5Ui/r1aQUU6AO76IoVe7UmJ49xz56YKBNQU/cg bPXvppcBJpKdpE1B298Rfpn4FopGfaKD7RtKVMlBT446M= X-Gm-Gg: Acq92OHE8uJHUb3EtKYjOFxn84NPsb/ByWI4/2VwG+nMPLT2DeVvXf4w411X+78u70+ Qq+MLs3Z0jflAI+qqMUEPaWT90+HFes6p/UVp1tSx/knjPEry5hTjphbnNnTn1sTclGtTO8DsTt bX/4789mOE3h5DWJlSPQBb7NEhw9RzoSBl8siffDIdyqW46nUUSN7sBNBggVmObQeCp+lw9QTUn EsJNtpBkV3Ij3cD0RKZVmrwxTrY/NoZvxlus9V8OxozWhljg9x7xANaxtCYkcTopjQGCISIZWn/ ehgwk9RiVA1xU9BWQ/Z7hUYp5Ecbs0WgHbDxdG384ZE9HPGztV0igiTM1mk81x8hossQXLuBpFP DgS27lHoUYMoHjtIlCDV8K2KG3fa/vMCtGxpiHZ6FMfvJwhJs8TCI6w== X-Received: by 2002:adf:e684:0:b0:45e:f31a:7ae6 with SMTP id ffacd0b85a97d-45ef6b8aadbmr345907f8f.30.1780068136717; Fri, 29 May 2026 08:22:16 -0700 (PDT) X-Received: by 2002:adf:e684:0:b0:45e:f31a:7ae6 with SMTP id ffacd0b85a97d-45ef6b8aadbmr345803f8f.30.1780068136163; Fri, 29 May 2026 08:22:16 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45ef34b7d6bsm4708531f8f.10.2026.05.29.08.22.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:22:15 -0700 (PDT) Date: Fri, 29 May 2026 11:22:11 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli , Miaohe Lin , Naoya Horiguchi Subject: [PATCH v9 02/37] mm: memory-failure: serialize TestSetPageHWPoison with zone->lock Message-ID: <2c527d20c99cdfe64a77bcf8da75f742d4c6991e.1780067977.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" TestSetPageHWPoison() is called without zone->lock, so its atomic update to page->flags can race with non-atomic flag operations that run under zone->lock in the buddy allocator. In particular, __free_pages_prepare() does: page->flags.f &=3D ~PAGE_FLAGS_CHECK_AT_PREP; This non-atomic read-modify-write, while correctly excluding __PG_HWPOISON from the mask, can still lose a concurrent TestSetPageHWPoison if the read happens before the poison bit is set and the write happens after. Follow-up patches in this series add similar non-atomic flag operations as well. Fix by acquiring zone->lock around TestSetPageHWPoison and around ClearPageHWPoison in the retry path. This serializes with all buddy flag manipulation. The cost is negligible: one lock/unlock in an extremely rare path (hardware memory errors). Note: SetPageHWPoison and TestClearPageHWPoison calls elsewhere in this file operate on pages already removed from the buddy allocator or on non-buddy pages (DAX, hugetlb), so they do not need zone->lock protection. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Acked-by: Miaohe Lin --- mm/memory-failure.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index ee42d4361309..d106f2c135c7 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -2348,6 +2348,8 @@ int memory_failure(unsigned long pfn, int flags) unsigned long page_flags; bool retry =3D true; int hugetlb =3D 0; + struct zone *zone; + unsigned long mf_flags; =20 if (!sysctl_memory_failure_recovery) panic("Memory failure on page %lx", pfn); @@ -2390,7 +2392,10 @@ int memory_failure(unsigned long pfn, int flags) if (hugetlb) goto unlock_mutex; =20 + zone =3D page_zone(p); + spin_lock_irqsave(&zone->lock, mf_flags); if (TestSetPageHWPoison(p)) { + spin_unlock_irqrestore(&zone->lock, mf_flags); res =3D -EHWPOISON; if (flags & MF_ACTION_REQUIRED) res =3D kill_accessing_process(current, pfn, flags); @@ -2399,6 +2404,7 @@ int memory_failure(unsigned long pfn, int flags) action_result(pfn, MF_MSG_ALREADY_POISONED, MF_FAILED); goto unlock_mutex; } + spin_unlock_irqrestore(&zone->lock, mf_flags); =20 /* * We need/can do nothing about count=3D0 pages. @@ -2420,7 +2426,9 @@ int memory_failure(unsigned long pfn, int flags) } else { /* We lost the race, try again */ if (retry) { + spin_lock_irqsave(&zone->lock, mf_flags); ClearPageHWPoison(p); + spin_unlock_irqrestore(&zone->lock, mf_flags); retry =3D false; goto try_again; } --=20 MST From nobody Mon Jun 8 10:56:13 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A15FD3BCD3A for ; Fri, 29 May 2026 15:22:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068146; cv=none; b=G6ubP4yjdzMvzdaGxErm9Ap8gPfejudVSO3Q4bwPYMXCRU3wEP7V+Sp75fKmAEJ12xzdnOFR1rPIgdMDgWvk0CGZbgh/xsx+/9iKyx/qMiJHaaSWspni58qWU89xgvmPrTZNylC4BQm12vmE7DsznXOsLD5t3r3IlZ7Vv6pCW/w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068146; c=relaxed/simple; bh=2SCjG0p+gTn8qm5MHLYDPQz8gTh6+lwWx6Kgl/LoFJQ=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=s3n8VnF4685TV7K51x6d6mrxqkEVh7M5N7tbhvfz1jtZtqWR92cXcabZcgzjLvboosBpWOtmU90W4IEE7yqGBE8NLxy2TawECxEt0aAl6/d851VZH6iZz2bjLwpg78briXUZQSkpmzlalC7JTHjHrJ7Spmh7nDfecWcNwDhlMag= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=bvdAz3Yy; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=LhYPA+6e; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="bvdAz3Yy"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="LhYPA+6e" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068144; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=6a9U/grsiUVqQvDGNE/rE2t/3gE4VHHRcC3EwM49fXU=; b=bvdAz3Yy2b+2L7ZCUqVryV/aByeZ7IZimErgqW0dBKXHkJcdPTG+ol/RWC6HFRb/v/XeZX 0ZzAYlYPhL0qgbMCDNtbDT2wPYb4nh3FaRtOb/vT7maP60ztWN7oHIP0xs9QiB8ZshEnbX Wj84L/lLz6co04H9wCro/wtpFCkfHKo= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-206-d47kFWgrN8Kv_bgjwjxDTQ-1; Fri, 29 May 2026 11:22:23 -0400 X-MC-Unique: d47kFWgrN8Kv_bgjwjxDTQ-1 X-Mimecast-MFC-AGG-ID: d47kFWgrN8Kv_bgjwjxDTQ_1780068142 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-45eec27f5a5so1055655f8f.2 for ; Fri, 29 May 2026 08:22:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068142; x=1780672942; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=6a9U/grsiUVqQvDGNE/rE2t/3gE4VHHRcC3EwM49fXU=; b=LhYPA+6elEb87i1rdq/nNEOOY64EF0CFABbcl6oRfk74IBxbrmIRx5cMc+gAc5AvnY LZveU2tKcoMf4fXZrNTxsbxrlqXVy8j63kWTmN8D87Nra+3f+ahfAuEKgeKfpyuwTQsd nqMMYRHqU2w+Gdt/+gMM/5IsWCc490ajluBsFQtlcjfHFoDrp/eqoisVXFKP8xdsC+Od A3ecvjS5b15vvPx+347hriAu6i8OpttwvW/15wTjK4MxSKML9rX8eeG35a62tP63ZqX3 7jje2PkGVP6ZWLb+I2HbvjMBl/tFxGFmT6c6PmWLm9oJoipBn9AzTmMS26BKEt78jTWU Ztcw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068142; x=1780672942; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6a9U/grsiUVqQvDGNE/rE2t/3gE4VHHRcC3EwM49fXU=; b=UdWL15G1yo0veDIcaKayfQtQMQbJeaedSLjfX/XJk8KzMRalsTd+nQVNDrIuQn/v/S URgn5DpKKSb03PFAA4F+EYwUVaL0BEf+XOtFtgkm0oRGr3++Obp1UdJvwGUe23V6RVEw ynF9hlMA/Vd5mmHnN8sT059sVzqa0LpE4eYzFRmLgHYBTFwguy+wEOxLqL7F1i5y4DNi zTTcjO+byoVVO08L6VyuboV/KWFGGeWNZZrafvy2YB+eIcfQggWu3jSvp9TGf6IR2uhh 3LgW+P7peB4hZKmkWfse44UXS0EyM4LJ11JfWzIb9PY8Cv9KrnIVZaCmFQ5nrYw3b1FC qhqw== X-Gm-Message-State: AOJu0YxfBTgcaBt5pvjjzfjiNkPmu5FcZpyjsah5noSyVTsLWVlPsUGM PTUadPqmuJHWcnJ1Wku+4d95DlZZe0CM07vuLh1T7Jc2Edl++Jl5BfsU2yyzViZ2oQM0YtyI5CQ /KR4c8MJjgWfe2dH8XStv9x0V0tcRSfCzaGMsSame3XkDbK2S/VNOUFWiBePb4b9ojt7oqG0zbG 0A4csbiSXeT7N4qiB6ykC9zbcRfHwPGyXPJ9OiCfPu/jA= X-Gm-Gg: Acq92OErlrMaS2j9HMLJjZAhGlyaE9YH1ZefucjOwqMGwUtX+ehP/v2RqNZxWjVE2ex UBnUyoxXzKRuk8lADZa3ySjQeTV1W5ZjyO5s8eJstFSMdAbmIQvQAwEUup+TDSoGbdJu1RsAeHM Tc8/2c04/V1rHi3jaEPihl3QZ7CqtI1zhQLpaNZDdk/Clzr+BrvY4M+5bpQQNsas/PTKR1G+vi5 5ttqBJQ744TgX4RA7dbqQSCukyK59iSLJTuZnx1DrJUV0fG4oqxvPeLFAZ3oUMDrRF6jdjzuYNi HupenkODDMNlcR0gcWxa2Y20Z41w0L+I8/Vudh/G/A/4hiCLpA1pY2Q5CUIog2hCawzVYIuIiOD I5cKDhrDVh447rvruVIv2zlY9k82Qzy2QCNxp1Js3HnwPIQr3ppK/EA== X-Received: by 2002:adf:f2cf:0:b0:45e:f672:40fb with SMTP id ffacd0b85a97d-45ef6b4f72bmr394897f8f.22.1780068141795; Fri, 29 May 2026 08:22:21 -0700 (PDT) X-Received: by 2002:adf:f2cf:0:b0:45e:f672:40fb with SMTP id ffacd0b85a97d-45ef6b4f72bmr394796f8f.22.1780068141211; Fri, 29 May 2026 08:22:21 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45ef354cd87sm3989750f8f.24.2026.05.29.08.22.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:22:20 -0700 (PDT) Date: Fri, 29 May 2026 11:22:16 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 03/37] mm: page_alloc: propagate PageReported flag across buddy splits Message-ID: <35539ca539f76bec575e8a6abc0cf42add858c0f.1780067977.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When a reported free page is split via expand() to satisfy a smaller allocation, the sub-pages placed back on the free lists lose the PageReported flag. This means they will be unnecessarily re-reported to the hypervisor in the next reporting cycle, wasting work. While I was unable to quantify the performance difference, it is an obvious waste, even if small. Propagate the PageReported flag to sub-pages during expand(), both in page_del_and_expand() and try_to_claim_block(), so that they are recognized as already-reported. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/page_alloc.c | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 227d58dc3de6..1192c4e47da4 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1510,6 +1510,7 @@ static void split_large_buddy(struct zone *zone, stru= ct page *page, unsigned long pfn, int order, fpi_t fpi) { unsigned long end =3D pfn + (1 << order); + bool reported =3D PageReported(page); =20 VM_WARN_ON_ONCE(!IS_ALIGNED(pfn, 1 << order)); /* Caller removed page from freelist, buddy info cleared! */ @@ -1521,6 +1522,8 @@ static void split_large_buddy(struct zone *zone, stru= ct page *page, do { int mt =3D get_pfnblock_migratetype(page, pfn); =20 + if (reported) + __SetPageReported(page); __free_one_page(page, pfn, zone, order, mt, fpi); pfn +=3D 1 << order; if (pfn =3D=3D end) @@ -1699,7 +1702,7 @@ struct page *__pageblock_pfn_to_page(unsigned long st= art_pfn, * -- nyc */ static inline unsigned int expand(struct zone *zone, struct page *page, in= t low, - int high, int migratetype) + int high, int migratetype, bool reported) { unsigned int size =3D 1 << high; unsigned int nr_added =3D 0; @@ -1721,6 +1724,15 @@ static inline unsigned int expand(struct zone *zone,= struct page *page, int low, __add_to_free_list(&page[size], zone, high, migratetype, false); set_buddy_order(&page[size], high); nr_added +=3D size; + + /* + * The parent page has been reported to the host. The + * sub-pages are part of the same reported block, so mark + * them reported too. This avoids re-reporting pages that + * the host already knows about. + */ + if (reported) + __SetPageReported(&page[size]); } =20 return nr_added; @@ -1731,9 +1743,10 @@ static __always_inline void page_del_and_expand(stru= ct zone *zone, int high, int migratetype) { int nr_pages =3D 1 << high; + bool was_reported =3D page_reported(page); =20 __del_page_from_free_list(page, zone, high, migratetype); - nr_pages -=3D expand(zone, page, low, high, migratetype); + nr_pages -=3D expand(zone, page, low, high, migratetype, was_reported); account_freepages(zone, -nr_pages, migratetype); } =20 @@ -2288,10 +2301,12 @@ try_to_claim_block(struct zone *zone, struct page *= page, /* Take ownership for orders >=3D pageblock_order */ if (current_order >=3D pageblock_order) { unsigned int nr_added; + bool was_reported =3D page_reported(page); =20 del_page_from_free_list(page, zone, current_order, block_type); change_pageblock_range(page, current_order, start_type); - nr_added =3D expand(zone, page, order, current_order, start_type); + nr_added =3D expand(zone, page, order, current_order, start_type, + was_reported); account_freepages(zone, nr_added, start_type); return page; } --=20 MST From nobody Mon Jun 8 10:56:13 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 913643CEBA7 for ; Fri, 29 May 2026 15:22:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068152; cv=none; b=Oi/KZRjdkM8Ww/iisGvhSw8dHRUoS5dj0Bwob0OieIx7WU/qOb7Zu8YQ6iaUKPKJ9tazT/C2XOV4yEjZVSlAXDpkEpq1srCa9sXbIoLxBTcwCK6Gls/l5dSXTHD+UcuMb8axQHMmQB3Ka6OSx+PiBR7gsqz9WH5Ih8FP1Hb1dWw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068152; c=relaxed/simple; bh=sa7tjyyH5w82gQsWdnwXLh3+WpyPoKY2RLFi26AWDX4=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=bEx84kr98Ygit8l9sIfOCrdYLPA9S9u0llgrQYLBwsOTIbeM3b76oxNJSGqyFt8ywK2F5Wl0nxI1MxpCFpUd6vUThmSrk0tQVIfQNmWAb3rUYpSVj664nulrhqa6PQxgsKDmh1y61AjhxsIw1vHMwQ070XHxN0xgoeekenw43TI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=JrbCeweW; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=L6bGD92Q; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="JrbCeweW"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="L6bGD92Q" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068149; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=CGpf7NEGXo6YXa4YKC+LNXdeGyms+G/W0uzlFnpQv6k=; b=JrbCeweWk5sp1syjirB0rb6zL7vlzHhm1xqrmT+Wtuh1grsRlYtAYiLOWeSc5gbWZ22DF8 0IKukvclsZeKD0EyV5YoxG6i3WgWEqAxLK2nqOWjmq5eGBjVdAHHFT+8jJ2CpU4zYNUvpJ bd4g0LyZ53ZFQEs99TYgw1qIcjB+ULo= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-12-SCybJitAMmakhQQlsAyAWw-1; Fri, 29 May 2026 11:22:28 -0400 X-MC-Unique: SCybJitAMmakhQQlsAyAWw-1 X-Mimecast-MFC-AGG-ID: SCybJitAMmakhQQlsAyAWw_1780068147 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-49081106196so27064655e9.1 for ; Fri, 29 May 2026 08:22:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068147; x=1780672947; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=CGpf7NEGXo6YXa4YKC+LNXdeGyms+G/W0uzlFnpQv6k=; b=L6bGD92QKORPF/0LBOdHglcp50+VxP3VMjjJyjzab5Avh2OL7PUubFaadFHDVBksVF ehi4c8Xv4C8qsnLbzRHW6W9SB8wai+brN7hf6QpZWJhz7CPdbcfq6ZhEYUhhautI9ps/ m/ddXVaXoe6jEHpfX1vCrBKrf2gi4MU6nSZMAbO6HQXDWIZh8Ge7m+cCyw7uWHEOI00R pxRY+nIVGFSYbdg2mCSFHqnA+5UAkCHA3ubHW7F0aoNErDnb2whLYB3W0e4BjPoe9q8k n/sJ/e99D1I96hmzWrccGE821LoIRh5rW2YGNtaLUQq/I8WLDMb6+ZGfTuwHrqANh04Q 0iCQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068147; x=1780672947; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CGpf7NEGXo6YXa4YKC+LNXdeGyms+G/W0uzlFnpQv6k=; b=OPSaT5OsMf2Oir4FL2F2b/PZJBrgLji5socTutX/il4zycGktDIhoqx+VJVOpRjKvt AzKBNUt618BrhYp7cwABTu1FQgMl7K/yO6/3kf54f5gwLvOEqXzbzK2/qz7HhBjJKn2e YT3TjreYJC2DV05gh2sJIGjiOV4I3/sQIaaVbLQ1ps1G65OI4aS/K24LvlkhCjnXrnMP KmFzKBGLbNzKU6yImWFCjhp3L4DjZCkxyqHI5x7Jr80BlUc//LkOKmylcQdEGGZuAnqa sS/CLNjCoW2JLkbN5bZkJoVXWDr26Oezou+5u7zuMPG2xUFpV7vlJ/Yos8B9yTqcIkuG Tmiw== X-Gm-Message-State: AOJu0Yze+2MqcnF7haMN1UILWxvHDDcjrX8G0IqHAsairfH3G79wjXSP yMJQu64CzTJOOcRBL8PIYqb+wnks0qyHivukzta341IQOsEHbbA85msESQ4THczgubCNiU2sVPd IE6cibfdyxvnB9phZGoBJwTFxvfcHcAjtwUFSU7ARo0yxysbRAs1ew5KXyzL1ZlcW5hmySkc4v9 9+ML8C5k+lDbEkW1IpVazZplt1JQCwldFhy73yeds8Ikw= X-Gm-Gg: Acq92OGDSg1Efk6b3lZJfJDNbSBxn+L/3JBKxt9AHCKrimHNtMS2iZGeUP/vh/VMkN1 Z+KtR0/aKaaKOt/9kxohxJxTleiEYPwMdZuO/vB0dARqO/hmVWJVohHoisFipd9GgYcdssEMGs0 MBI8RgEv/liwGjXoj5zEUtfxF+Bo3NBGswVrPrj7sycA4rCrAAGueQqU8JUh5yJjU42oLkTePxT es4IKP2XTPT6QPTmylXnk+ercIboKwACMX3oCHHJb1NXiGiuF8rBHdld3Eo9lRXbZhd+p3JvHBE NYg+l4L6DRnn9W+tBf/tPgOGdGKgr1fUcuhQJ6rN9otMz7za2klf6UJHr2W6cHNH4Y64RMm4TcE xKfsZh04vz4rj8uBCXQN4Egsy4CxUXA+VxM5+omY/D6NHW3flu1tP+g== X-Received: by 2002:a05:600c:34cf:b0:490:5e2a:f924 with SMTP id 5b1f17b1804b1-490a2900e81mr2221545e9.7.1780068146896; Fri, 29 May 2026 08:22:26 -0700 (PDT) X-Received: by 2002:a05:600c:34cf:b0:490:5e2a:f924 with SMTP id 5b1f17b1804b1-490a2900e81mr2220045e9.7.1780068146235; Fri, 29 May 2026 08:22:26 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4909caa7e17sm56125215e9.9.2026.05.29.08.22.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:22:25 -0700 (PDT) Date: Fri, 29 May 2026 11:22:21 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 04/37] mm: page_reporting: allow driver to set batch capacity Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a capacity field to page_reporting_dev_info so drivers can control the maximum number of pages per report batch. This is useful when the driver needs to reserve virtqueue descriptors for metadata (e.g., a bitmap buffer) alongside the page buffers. The value is capped at PAGE_REPORTING_CAPACITY and rounded down to a power of 2. If unset (0), defaults to PAGE_REPORTING_CAPACITY. The virtio_balloon driver sets capacity to the reporting virtqueue size, letting page_reporting adapt to whatever the device provides. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- drivers/virtio/virtio_balloon.c | 5 +---- include/linux/page_reporting.h | 3 +++ mm/page_reporting.c | 25 ++++++++++++++----------- 3 files changed, 18 insertions(+), 15 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index f6c2dff33f8a..6a1a610c2cb1 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -1017,10 +1017,6 @@ static int virtballoon_probe(struct virtio_device *v= dev) unsigned int capacity; =20 capacity =3D virtqueue_get_vring_size(vb->reporting_vq); - if (capacity < PAGE_REPORTING_CAPACITY) { - err =3D -ENOSPC; - goto out_unregister_oom; - } =20 vb->pr_dev_info.order =3D PAGE_REPORTING_ORDER_UNSPECIFIED; =20 @@ -1041,6 +1037,7 @@ static int virtballoon_probe(struct virtio_device *vd= ev) vb->pr_dev_info.order =3D 5; #endif =20 + vb->pr_dev_info.capacity =3D capacity; err =3D page_reporting_register(&vb->pr_dev_info); if (err) goto out_unregister_oom; diff --git a/include/linux/page_reporting.h b/include/linux/page_reporting.h index 9d4ca5c218a0..5ab5be02fa15 100644 --- a/include/linux/page_reporting.h +++ b/include/linux/page_reporting.h @@ -22,6 +22,9 @@ struct page_reporting_dev_info { =20 /* Minimal order of page reporting */ unsigned int order; + + /* Max pages per report batch (default PAGE_REPORTING_CAPACITY) */ + unsigned int capacity; }; =20 /* Tear-down and bring-up for page reporting devices */ diff --git a/mm/page_reporting.c b/mm/page_reporting.c index 7418f2e500bb..a6ddf6fafc20 100644 --- a/mm/page_reporting.c +++ b/mm/page_reporting.c @@ -174,10 +174,10 @@ page_reporting_cycle(struct page_reporting_dev_info *= prdev, struct zone *zone, * list processed. This should result in us reporting all pages on * an idle system in about 30 seconds. * - * The division here should be cheap since PAGE_REPORTING_CAPACITY - * should always be a power of 2. + * The division here should be cheap since capacity should + * always be a power of 2. */ - budget =3D DIV_ROUND_UP(area->nr_free, PAGE_REPORTING_CAPACITY * 16); + budget =3D DIV_ROUND_UP(area->nr_free, prdev->capacity * 16); =20 /* loop through free list adding unreported pages to sg list */ list_for_each_entry_safe(page, next, list, lru) { @@ -222,10 +222,10 @@ page_reporting_cycle(struct page_reporting_dev_info *= prdev, struct zone *zone, spin_unlock_irq(&zone->lock); =20 /* begin processing pages in local list */ - err =3D prdev->report(prdev, sgl, PAGE_REPORTING_CAPACITY); + err =3D prdev->report(prdev, sgl, prdev->capacity); =20 /* reset offset since the full list was reported */ - *offset =3D PAGE_REPORTING_CAPACITY; + *offset =3D prdev->capacity; =20 /* update budget to reflect call to report function */ budget--; @@ -234,7 +234,7 @@ page_reporting_cycle(struct page_reporting_dev_info *pr= dev, struct zone *zone, spin_lock_irq(&zone->lock); =20 /* flush reported pages from the sg list */ - page_reporting_drain(prdev, sgl, PAGE_REPORTING_CAPACITY, !err); + page_reporting_drain(prdev, sgl, prdev->capacity, !err); =20 /* * Reset next to first entry, the old next isn't valid @@ -260,13 +260,13 @@ static int page_reporting_process_zone(struct page_reporting_dev_info *prdev, struct scatterlist *sgl, struct zone *zone) { - unsigned int order, mt, leftover, offset =3D PAGE_REPORTING_CAPACITY; + unsigned int order, mt, leftover, offset =3D prdev->capacity; unsigned long watermark; int err =3D 0; =20 /* Generate minimum watermark to be able to guarantee progress */ watermark =3D low_wmark_pages(zone) + - (PAGE_REPORTING_CAPACITY << page_reporting_order); + (prdev->capacity << page_reporting_order); =20 /* * Cancel request if insufficient free memory or if we failed @@ -290,7 +290,7 @@ page_reporting_process_zone(struct page_reporting_dev_i= nfo *prdev, } =20 /* report the leftover pages before going idle */ - leftover =3D PAGE_REPORTING_CAPACITY - offset; + leftover =3D prdev->capacity - offset; if (leftover) { sgl =3D &sgl[offset]; err =3D prdev->report(prdev, sgl, leftover); @@ -322,11 +322,11 @@ static void page_reporting_process(struct work_struct= *work) atomic_set(&prdev->state, state); =20 /* allocate scatterlist to store pages being reported on */ - sgl =3D kmalloc_objs(*sgl, PAGE_REPORTING_CAPACITY); + sgl =3D kmalloc_objs(*sgl, prdev->capacity); if (!sgl) goto err_out; =20 - sg_init_table(sgl, PAGE_REPORTING_CAPACITY); + sg_init_table(sgl, prdev->capacity); =20 for_each_zone(zone) { err =3D page_reporting_process_zone(prdev, sgl, zone); @@ -377,6 +377,9 @@ int page_reporting_register(struct page_reporting_dev_i= nfo *prdev) page_reporting_order =3D pageblock_order; } =20 + if (!prdev->capacity || prdev->capacity > PAGE_REPORTING_CAPACITY) + prdev->capacity =3D PAGE_REPORTING_CAPACITY; + /* initialize state and work structures */ atomic_set(&prdev->state, PAGE_REPORTING_IDLE); INIT_DELAYED_WORK(&prdev->work, &page_reporting_process); --=20 MST From nobody Mon Jun 8 10:56:13 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 648223E1D19 for ; Fri, 29 May 2026 15:22:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068156; cv=none; b=ZmzjKXx5AKil92BjcYC71e9m/PN9yDnVbzwh/ib6WHPEvDuZQMaNv58qKQMm6Dpq8XQF5BzqN1C3ViyM+XZbsiSftX2brcqzHDE+VufCZAm8cOXofV0oBZ1rKdmtkzKyjjnEZb69zb29ITOx9Q37vWWEFGzejrnqdyBaZi5o+Sk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068156; c=relaxed/simple; bh=zfTSssqCCnno4sdUyiW/sc4lt3FSQTZh1jfv0HEfTik=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=FSJcjJl+woQDJBd2CmR5VZ3h5DceeWVUuCcTdoMuSfYBZrqMq2Hrv334xb7z4krVUGOBQtIaje/59rDdy+A5e1QCLYESB59AJAXzWbTWmEeUPK+NoFp61jQlHEF+MB/MF+fChBE+/NEFwEz230F/czgnnPrziMFHPKIEhIEZYT0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Q3f4U2ha; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=Y+SOk1Wf; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Q3f4U2ha"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="Y+SOk1Wf" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068154; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=3RnCcviTYcs0bJrQCPa7/Ta1tOKxh1kkpoqwRDnBft0=; b=Q3f4U2haBuvuzxfAXnu4OSPJ96PbWImWFQMEJKbOfbETDPSPX8rEmkaFOqFWi9x0z7HjRl 00Z6T4Dbq1qx0as8PalWGaFDcAqMYsKNJLWKkW9z0mKdemAFXwf278lAzwQWsRT5w8G6Qg 5t2k8kQXki91cGhfSKLCYSV7FjXHG7E= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-657-bYxwXsd9P6SxwATXBc4Uyg-1; Fri, 29 May 2026 11:22:33 -0400 X-MC-Unique: bYxwXsd9P6SxwATXBc4Uyg-1 X-Mimecast-MFC-AGG-ID: bYxwXsd9P6SxwATXBc4Uyg_1780068152 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-48fd3dbd16aso115315595e9.0 for ; Fri, 29 May 2026 08:22:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068152; x=1780672952; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=3RnCcviTYcs0bJrQCPa7/Ta1tOKxh1kkpoqwRDnBft0=; b=Y+SOk1WfAQGkn5Mn3ufqySIYf9gPcRMtA/lX8nSlSX2S7wuoEUGjpD7XyANA6xSlj1 S8Canka4zR+t3GvhHuadp87hILQmPithDg1WReaQ+Nglj5aOtDlw+4iDs+I04bc3gwi2 0xC93giK8Uf6nkbCSHfDJS/wVRmd3Gyi/1Ni+yj53j0XGCkiH0aOignP7S2j1jdC/ex3 FYMANL8Wss34UbxXizwZuNaJrNnjnF3mZnUGu+YUcH4CT6hMmgLymHuLIQfmPFoP9io1 aJpVtgVMsC9X+pgE5lyZfhhsnd43M94idyeBciO7OobWmpmkWKNNA9G0xJvOdyNeKgi7 xAoQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068152; x=1780672952; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3RnCcviTYcs0bJrQCPa7/Ta1tOKxh1kkpoqwRDnBft0=; b=hSCW4l3ozluJyC5TAUh+2SxQreRAE/Izf1m8b3LQxK8e9lcIYYICBJYH0kqEFUFvOq MMwNDt3/XoCimRo2Gvv5FqZEDcXwrns4FdQLeBtpapiTgdRrBVlFGqchomO/l6V3rGsU lMSS5U8HoSR8I8ZDyfy5gNfaFU7Ba8ZE86iw7jtGeS376dpqqRWfXMgZdlNEeY05Cs21 2dyMvqb0c1I+LkKfmIDIHnXGePE8oTgv6uQSYhnmThCuBXW5zJSFMRgJo2wT32MahiLE k/1TJq2AhkMUzUz+2WzeXdLYY29nFkVDGSAUkUlfsVsUT9lGCnhR70PpO9IsKW/aYArL +jWQ== X-Gm-Message-State: AOJu0YyF1wqTGxI8uFtj7hyUoCoSIyQ+aPiNqwlrzv1mLzR6NbBjJ+RA J/R5nTqtyoqr4rDhgrgNgmdgQAQYGX63R6QdvSbMlLmRoH05nb2i9jnJBvG7Oi5npAleYpOkG5G Dz9sR4TSIgWt37AcJ31WJCV71noYDy816H2HUSC/yzDVa6SJSpn9Kef7Ltnt+Sdqw7wNOBIgH+r Z7O8xI6Lfu9DCinMC6mPWzKXtQHYkSWoUzAtNdfY8j22A= X-Gm-Gg: Acq92OEHeoLw/5JQFSB+vANBdetbT4UxpYiplSb9C108V5KArz3iuVA4LRt6Pzq3oHD PN0E78d5pz3UnXI3hAOBUdnlZQWEwZoOTmsNyKBopvC4bk/l4QKRQyZSE3rGOfowl2OMRdsZrfn PCdAArgC6qDDEXSc8vjlwF2o2d/yS4AW2GgxlPxVg9yi/aOpIXHLySEpv10uDeVWt8z5C8cVHkW N691pALq7ccf7Q3efecIwmstAZfumNMX0FdkBIK+cNSxo/wMJIn74QGDMlJTty9lEPNbZ0I80WF 6XxD9vWut+c2IKNu4FeDj2IPXcpkNysJfhzpDhM6vgrZmCEaq9FptG+/LGthqmWiu2prgPnBl8+ EiMrD//9pocqnqW4XCThRwgZd1erMBhsKXB1gtlC2rMblYWI4kqB+EQ== X-Received: by 2002:a05:600c:34cf:b0:490:5e2a:f924 with SMTP id 5b1f17b1804b1-490a2900e81mr2228145e9.7.1780068151888; Fri, 29 May 2026 08:22:31 -0700 (PDT) X-Received: by 2002:a05:600c:34cf:b0:490:5e2a:f924 with SMTP id 5b1f17b1804b1-490a2900e81mr2226845e9.7.1780068151254; Fri, 29 May 2026 08:22:31 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4909d6eb3adsm46766005e9.11.2026.05.29.08.22.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:22:30 -0700 (PDT) Date: Fri, 29 May 2026 11:22:26 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 05/37] mm: hugetlb: remove dead alloc_hugetlb_folio stub Message-ID: <8e2d9431b0e324100d550c765024c883560a4367.1780067977.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Remove the !CONFIG_HUGETLB_PAGE stub for alloc_hugetlb_folio(). The stub is dead code: all callers are in mm/hugetlb.c (CONFIG_HUGETLB_PAGE) or fs/hugetlbfs/inode.c (CONFIG_HUGETLBFS), and CONFIG_HUGETLB_PAGE is def_bool HUGETLBFS with nothing selecting it independently. The stub is also broken: it returns NULL, but all callers check IS_ERR(folio), so a NULL return would not be caught and would crash on the subsequent folio dereference. Remove it now since follow-up patches change the signature of alloc_hugetlb_folio and would otherwise need to update the broken stub too. Signed-off-by: Michael S. Tsirkin Reviewed-by: Gregory Price Assisted-by: Claude:claude-opus-4-6 Reviewed-by: Dev Jain --- include/linux/hugetlb.h | 7 ------- 1 file changed, 7 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 93418625d3c5..f016bc2e8936 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -1129,13 +1129,6 @@ static inline void wait_for_freed_hugetlb_folios(voi= d) { } =20 -static inline struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, - unsigned long addr, - bool cow_from_owner) -{ - return NULL; -} - static inline struct folio * alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_nid, nodemask_t *nmask, gfp_t gfp_mask) --=20 MST From nobody Mon Jun 8 10:56:13 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 47F863C1985 for ; Fri, 29 May 2026 15:22:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068162; cv=none; b=WAeoPppzNv553Ssi6rJmWis+XYayYGGRNyQ1mA8QkBPQiWFKx1yYbABOkSKflQocS01wDqihnoBwUOU7/DrpZqNQ/jFl7In1V/nZRihhabci9jpu9AUmnM4rRT1u1bK/nVsBF+4cPuMyk7QUABfMU2S+NLoNuTUt4UsgH6CFsbw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068162; c=relaxed/simple; bh=UliapvFzBZgGZQz4lzdPjxY0rdM8nGKqjKGuh8rusnI=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=eAlscTmevbMPjH8H974ekkfdQKnCCN5Q/gKJMfj982AWKGLvHArZqaE2K49nzdqkywIjFGpJfi7v8JNeZbDaPR21sc+FmmQzyfUaiL6qLIYZxzwHcIIjevqDka5z8iMMxEK/JoSUO+IWMr+TEzOQuWvIRFYYei2Y5icOsmh80ag= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=B9rbl8N5; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=UVczpk9P; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="B9rbl8N5"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="UVczpk9P" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068160; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=1d6/7ocvnM9x5v92z8cdenZMlPT5Fb+FBGOKjHhSxWM=; b=B9rbl8N5HN3M8rYrLTO8GdWfLYDIQcUyX3voGdL+XaJBgT9S9659Jp8eto2lG2mO0iYOgB XU3nVWHxIlfn+WPYqZeA4mEKDOwOAq56H++mXIvv8r1XcQBg8RsALnabAUhzAWeEKSunmQ u4dHejHn4eJRqV0w2Sgis5wFtCvnFGE= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-582--8Kap3CnPgGYvZrpG4putQ-1; Fri, 29 May 2026 11:22:38 -0400 X-MC-Unique: -8Kap3CnPgGYvZrpG4putQ-1 X-Mimecast-MFC-AGG-ID: -8Kap3CnPgGYvZrpG4putQ_1780068157 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-43d7a5b9678so8348747f8f.2 for ; Fri, 29 May 2026 08:22:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068157; x=1780672957; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=1d6/7ocvnM9x5v92z8cdenZMlPT5Fb+FBGOKjHhSxWM=; b=UVczpk9P9Z//ma+vyJlGPNn5qDuZ0ZfzZ/JIw/vijnos+OjkOp+JFFOwumr2kym4tD lRxhz4NSH65oFeDFUclOmKEp6XCo8C/GzWFEWXe3Qw1hiGxH26xpRJrhxP+Pll2riCYr Lo6vfZUF4g5Iandq/LJ/Gz4yVB2l8r8ws+ALGttj0p1mdjEiNb7OuFSCL+gN3fD0qtyD h1++xHPKf8ubZ4wjbatYvenho/b+kiTQ2i86Ku1HhVqybC8hPoLbWbXIb48rYngYbys8 L31EibKkd163Jq5bXNF+OblWC6/RDRLtvjzJZ/qnsZRGbJGuFJxMzAOkYEbZWatVK2AQ SJaQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068157; x=1780672957; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1d6/7ocvnM9x5v92z8cdenZMlPT5Fb+FBGOKjHhSxWM=; b=Msi8Fl+JFsF/kbymzMiaK7nHOXZCdKuhJ51zk1azCvV2a1s5mKAGnWPFdYtORXexmy o70ly6gEwYTYFpJYTLhxW5eNXj+oIRhzUH/TNDvcsGb9gJ6rkOIG8gUD0h4Wt2xoLiam C5e2gfgApkhT4CKRlZIXKACdW0LOd0ofSAQLvc8tT7s54TzpSQWReKZQSjEia4HESb18 zp/r/VJTjbWG/8FufQXfznCUrotvaRByhiHFgiXQhS+SitOMdHXxjBbbXMWPgfk7Wuxo 9aUzZh6nh7tjNWewgIhJUuAQ++h3Ydw4YKZ1zPQvm07K7X1VyCuAEpME0UIE/gGS/xhQ qUEQ== X-Gm-Message-State: AOJu0YzA1PqfkVD7/9rt5wa0T3qgKTLYAA54LAU3mhubbZMvBWKNdFGZ PGlt2A5EmlNGkP2379BD1eWdBsHIAkz23gakXYaaxqZzTUaU/MEVrOUG6iuVcG7oE45BKY/2AIJ BIchiMYOPPyCLo6M5ObZ21bnbQ2RJMNpZ8P4TO/5PYZhuIzyUllqaJjXXkIKZTPD2J+cJwp82Nw h6sx/j0TEsxC6oGGhr5ov3Kk1JPLgqNSoU7tZ1qnYe+lM= X-Gm-Gg: Acq92OEDR+TLmVqurB3KRbhM0IAcGjvRvPy2HihFhl2FMPasshMm4NOIRK8mFmlj36m QPrxo4W7As+LjspjwcyWqpa1fe3Y6RgEX5mLMt9yNqE2Adxy1pNwjhznka5gKZZpis8yHc7BzKN la0TbR/6CyugA0YedmEdPKrR+hCMXtwyFoFxF4NzZzssWo4R94zCWKZTQfcZSYLgX+lsDvkU/z7 350bM0Nz2a4XhY7HNCZjRdVqkJ2dLbbQgGdXvuI7VCzEdZA5d8tsdTkMKRpM9r2683tIgLE+QC8 VEq5gyVRIsJ/ox7cHR+duw/ETBoOP3ENkaMzcyIgnl4/jfM0tWcBS0U7FXHMG7lPmWFW+HFNHeV sp8wprQf2UcJZe/3hM6mg6hqHBqP1RK01bwe4gVxqUnXRSNbLFJgcSw== X-Received: by 2002:a05:600d:8444:20b0:490:44eb:c1d9 with SMTP id 5b1f17b1804b1-490a2947c86mr1027405e9.28.1780068157078; Fri, 29 May 2026 08:22:37 -0700 (PDT) X-Received: by 2002:a05:600d:8444:20b0:490:44eb:c1d9 with SMTP id 5b1f17b1804b1-490a2947c86mr1026475e9.28.1780068156493; Fri, 29 May 2026 08:22:36 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4909c0e8c1bsm22645615e9.3.2026.05.29.08.22.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:22:35 -0700 (PDT) Date: Fri, 29 May 2026 11:22:31 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 06/37] mm: move vma_alloc_folio_noprof to page_alloc.c Message-ID: <926990355eadc624f8ef12b30b50f8aaf67118f0.1780067977.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move vma_alloc_folio_noprof() from an inline in gfp.h (for !NUMA) and mempolicy.c (for NUMA) to page_alloc.c. This prepares for a subsequent patch that will thread user_addr through the allocator: having vma_alloc_folio_noprof in page_alloc.c means user_addr can be passed to the internal allocation path without changing public API signatures or duplicating plumbing in both gfp.h and mempolicy.c. The !NUMA path gains the VM_DROPPABLE -> __GFP_NOWARN check that the NUMA path already had. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- include/linux/gfp.h | 9 ++------- mm/mempolicy.c | 32 -------------------------------- mm/page_alloc.c | 43 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 45 insertions(+), 39 deletions(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index 51ef13ed756e..7ccbda35b9ad 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -318,13 +318,13 @@ static inline struct page *alloc_pages_node_noprof(in= t nid, gfp_t gfp_mask, =20 #define alloc_pages_node(...) alloc_hooks(alloc_pages_node_noprof(__VA_= ARGS__)) =20 +struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, + struct vm_area_struct *vma, unsigned long addr); #ifdef CONFIG_NUMA struct page *alloc_pages_noprof(gfp_t gfp, unsigned int order); struct folio *folio_alloc_noprof(gfp_t gfp, unsigned int order); struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order, struct mempolicy *mpol, pgoff_t ilx, int nid); -struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, struct vm_area_= struct *vma, - unsigned long addr); #else static inline struct page *alloc_pages_noprof(gfp_t gfp_mask, unsigned int= order) { @@ -339,11 +339,6 @@ static inline struct folio *folio_alloc_mpol_noprof(gf= p_t gfp, unsigned int orde { return folio_alloc_noprof(gfp, order); } -static inline struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, - struct vm_area_struct *vma, unsigned long addr) -{ - return folio_alloc_noprof(gfp, order); -} #endif =20 #define alloc_pages(...) alloc_hooks(alloc_pages_noprof(__VA_ARGS__)) diff --git a/mm/mempolicy.c b/mm/mempolicy.c index d139b074a599..a1707ad498a8 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -2516,38 +2516,6 @@ struct folio *folio_alloc_mpol_noprof(gfp_t gfp, uns= igned int order, return page_rmappable_folio(page); } =20 -/** - * vma_alloc_folio - Allocate a folio for a VMA. - * @gfp: GFP flags. - * @order: Order of the folio. - * @vma: Pointer to VMA. - * @addr: Virtual address of the allocation. Must be inside @vma. - * - * Allocate a folio for a specific address in @vma, using the appropriate - * NUMA policy. The caller must hold the mmap_lock of the mm_struct of the - * VMA to prevent it from going away. Should be used for all allocations - * for folios that will be mapped into user space, excepting hugetlbfs, and - * excepting where direct use of folio_alloc_mpol() is more appropriate. - * - * Return: The folio on success or NULL if allocation fails. - */ -struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, struct vm_area_= struct *vma, - unsigned long addr) -{ - struct mempolicy *pol; - pgoff_t ilx; - struct folio *folio; - - if (vma->vm_flags & VM_DROPPABLE) - gfp |=3D __GFP_NOWARN; - - pol =3D get_vma_policy(vma, addr, order, &ilx); - folio =3D folio_alloc_mpol_noprof(gfp, order, pol, ilx, numa_node_id()); - mpol_cond_put(pol); - return folio; -} -EXPORT_SYMBOL(vma_alloc_folio_noprof); - struct page *alloc_frozen_pages_noprof(gfp_t gfp, unsigned order) { struct mempolicy *pol =3D &default_policy; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 1192c4e47da4..0c4f4c678233 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5288,6 +5288,49 @@ struct folio *__folio_alloc_noprof(gfp_t gfp, unsign= ed int order, int preferred_ } EXPORT_SYMBOL(__folio_alloc_noprof); =20 +#ifdef CONFIG_NUMA +/** + * vma_alloc_folio - Allocate a folio for a VMA. + * @gfp: GFP flags. + * @order: Order of the folio. + * @vma: Pointer to VMA. + * @addr: Virtual address of the allocation. Must be inside @vma. + * + * Allocate a folio for a specific address in @vma, using the appropriate + * NUMA policy. The caller must hold the mmap_lock of the mm_struct of the + * VMA to prevent it from going away. Should be used for all allocations + * for folios that will be mapped into user space, excepting hugetlbfs, and + * excepting where direct use of folio_alloc_mpol() is more appropriate. + * + * Return: The folio on success or NULL if allocation fails. + */ +struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, + struct vm_area_struct *vma, unsigned long addr) +{ + struct mempolicy *pol; + pgoff_t ilx; + struct folio *folio; + + if (vma->vm_flags & VM_DROPPABLE) + gfp |=3D __GFP_NOWARN; + + pol =3D get_vma_policy(vma, addr, order, &ilx); + folio =3D folio_alloc_mpol_noprof(gfp, order, pol, ilx, numa_node_id()); + mpol_cond_put(pol); + return folio; +} +#else +struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, + struct vm_area_struct *vma, unsigned long addr) +{ + if (vma->vm_flags & VM_DROPPABLE) + gfp |=3D __GFP_NOWARN; + + return folio_alloc_noprof(gfp, order); +} +#endif +EXPORT_SYMBOL(vma_alloc_folio_noprof); + /* * Common helper functions. Never use with __GFP_HIGHMEM because the retur= ned * address cannot represent highmem pages. Use alloc_pages and then kmap if --=20 MST From nobody Mon Jun 8 10:56:13 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BBD533612EE for ; Fri, 29 May 2026 15:22:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068169; cv=none; b=MVDMCd3PuTodBzCdy5cfOgR8yXuWJAk52Kx1l42nUsiWdCVBn+ihUNX6j15mngvIjxHdp9oS0b05gqze4casrdJSloH2//49h1QJeT+OUEAQj3pdPAZE4SDi7N9+aqI3mG14WqnDHeoSl3H8es1zHcVExnM55gTUkgFwkVqA0to= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068169; c=relaxed/simple; bh=KqYWBRZRf/mrcqCV3qPhinTk6/Z3VkXq0rHmQ+IepUw=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Ev49EYs+jW9Q8JGYZzk+oravCyxQR7Uu3BV2/ftjEvaN5nG5tbmUpZivfo38hFKKHdicY6ky6rpmax9zxOE7x/Z6p7v7ak2pzNIDmQyoGZ3WY7KfhTqoDOiKeT8pR6e8q5P5pOeDaPmnJ8aw8TzwQd4kHfT1/RHyosUHVGQenPA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=FQrQQ6Rt; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=sGZUr7od; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="FQrQQ6Rt"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="sGZUr7od" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068165; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=8pf2t+ayX4ZbJBLq7vZzKsLH4wzafJDxlutICFZ1OB4=; b=FQrQQ6RtUflJG8GcfXr5kkB7SPBh9Qu02CGtnFZtN0LqGVB68Y0zNBqZ67qhyG8mX0ScxB Gf4EVIzefdgLdxBhSEUBbGefJmlaOEJinKzuNmnnWekqLma6fBTcLSqvHJAjhUumq5yhoi roCbzr/E+gDyzLjlLvThMShQzRAn3uk= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-217-38rXxtsYPaWW4PvipXEMyw-1; Fri, 29 May 2026 11:22:44 -0400 X-MC-Unique: 38rXxtsYPaWW4PvipXEMyw-1 X-Mimecast-MFC-AGG-ID: 38rXxtsYPaWW4PvipXEMyw_1780068163 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-45eec2badc4so1067860f8f.2 for ; Fri, 29 May 2026 08:22:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068163; x=1780672963; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=8pf2t+ayX4ZbJBLq7vZzKsLH4wzafJDxlutICFZ1OB4=; b=sGZUr7odqgv/4/2ByXcHH7G4g1lVVYq8wt1gTmevL5jt2bJdbX9NuQgnmR3h6GKJs4 TP7OwEWYZewciTSMoK/vGHpQnv/JMx3G0RWXkq+I5K+Le/TGenwu4vvZhCyZE3ink+QT 50qY8bFKXwY8MDvlI5pgCkKjlr7/VBq35lFFCPKxDDZYfrSZDAfN3AWg2nt7AVGdf1yM aIBKyy6ibdU+kdrBwfPr1kA+0Ao1psulc2ZUO4E9sVVunaHnc8mB1n6IKzQHXt9gb3uJ m063UgmXrlaPAwkHTT4Jh2bU/WygjFC4k+M6prqGA69rPWY7HLMhrcz9Kqbx8knyiXyy Gx1A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068163; x=1780672963; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8pf2t+ayX4ZbJBLq7vZzKsLH4wzafJDxlutICFZ1OB4=; b=kPE06mMRBUXow/rN5rnoP+Np/UlM8lwsi9tKDmebLhYnj5hJ1hpI0EIoKnKz2K+rcq 6hQuXm0lkjABHiTc+8/3hGF5KkT75aIRbHd/7uUjaty3VKjy+LsMl9+CSnaBKLEnWQig 7VvAkdDpFxj6JYYl7Gk5MuDkV+BR4AHO2yl24dgtHrbzU2AszXXwSOUYrh6LvH0g881k fNt2a7jxAxQJ3k45mqHAmTE/Rtgnu/9xhV0EbLSPwhKxZVLlzeqbwIZh0vRMtzrgf0Ej vEp+6dURvTT4yKOVTYsIbyav0ArB5mhGDIqcMOUO6fnmwedmQz4263BG43jWRgg+txRM 63GQ== X-Gm-Message-State: AOJu0Yy0wstJHBb9aTna+rkQRjLpr4expFolHy5OdL9aRI79jYkY0hxK imZRkQ1IIr0kObCXPBrjOlrTC5+HWPzkm4kK5vsf3WiIANcdOrBRMzjwQbWQQyUT1aqzglCoYYq oETQ/zKsJUVSUr3KQYm39Ps51tKSpoXwWqRpYNJS8Awtn6x0N5olUlrtEJq6oBQahm5OOAQx2i8 Vnv9OCIL62cu5C0J7dR5vvQUMFip/y7fR39Tt/xcl7Tis= X-Gm-Gg: Acq92OHUZoTbwxmWwfB771/imAfM4Vmel23Se3nNWjlo19CUESrVDy854LTMeYfjg8q VY1xIU0PWK29pALTr0TzDvUQOERN0yYb0L0ReOQRy+KGH9yWe8NQn3lbMxvt8PPdoahCfJ8Noy2 FgkilJ1Jt/3sqM2xngeCGXdr1WZC2wrm8UZ7faxAj43PMpCYd9nGvZ7vKDhjGOmdvwyuqM5uiOu q6nWsOUXj7Hkdq+wJc8VmtSU+wQMb5I0sAAleFTLPPI5QNp18xbr754ZAz5I36CIHehJf4O2jgk 1ohfGUMAfhiR6uAg8PUOZNH0R3yzTdqNJBOeVGU3WDDgkRWgbepTrNb4WuTwJyKl2xjSg4URUzG +G62ksgg8mLhIHIG0BqKODYKgGzCTMZVqCzjIBxKIuyyqQwvJowSW6A== X-Received: by 2002:a5d:6191:0:b0:452:75ca:3fab with SMTP id ffacd0b85a97d-45ef6baa518mr354739f8f.38.1780068162629; Fri, 29 May 2026 08:22:42 -0700 (PDT) X-Received: by 2002:a5d:6191:0:b0:452:75ca:3fab with SMTP id ffacd0b85a97d-45ef6baa518mr354633f8f.38.1780068161887; Fri, 29 May 2026 08:22:41 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45ef34a065esm4523815f8f.5.2026.05.29.08.22.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:22:41 -0700 (PDT) Date: Fri, 29 May 2026 11:22:36 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli , Jann Horn , Pedro Falcato , Harry Yoo , Hao Li Subject: [PATCH v9 07/37] mm: thread user_addr through page allocator for cache-friendly zeroing Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Thread a user virtual address from vma_alloc_folio() down through the page allocator to post_alloc_hook(). This is plumbing preparation for a subsequent patch that will use user_addr to call folio_zero_user() for cache-friendly zeroing of user pages. The user_addr is stored in struct alloc_context and flows through: vma_alloc_folio -> folio_alloc_mpol -> __alloc_pages_mpol -> __alloc_frozen_pages -> get_page_from_freelist -> prep_new_page -> post_alloc_hook USER_ADDR_NONE ((unsigned long)-1) is used for non-user allocations, since address 0 is a valid userspace mapping. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- include/linux/gfp.h | 2 +- mm/compaction.c | 5 ++--- mm/hugetlb.c | 36 ++++++++++++++++++++---------------- mm/internal.h | 22 +++++++++++++++++++--- mm/mempolicy.c | 44 ++++++++++++++++++++++++++++++++------------ mm/mmap.c | 6 ++++++ mm/page_alloc.c | 44 +++++++++++++++++++++++++++++--------------- mm/slub.c | 4 ++-- 8 files changed, 111 insertions(+), 52 deletions(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index 7ccbda35b9ad..ee35c5367abc 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -337,7 +337,7 @@ static inline struct folio *folio_alloc_noprof(gfp_t gf= p, unsigned int order) static inline struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned in= t order, struct mempolicy *mpol, pgoff_t ilx, int nid) { - return folio_alloc_noprof(gfp, order); + return __folio_alloc_noprof(gfp, order, numa_node_id(), NULL); } #endif =20 diff --git a/mm/compaction.c b/mm/compaction.c index 3648ce22c807..72684fe81e83 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -82,7 +82,7 @@ static inline bool is_via_compact_memory(int order) { ret= urn false; } =20 static struct page *mark_allocated_noprof(struct page *page, unsigned int = order, gfp_t gfp_flags) { - post_alloc_hook(page, order, __GFP_MOVABLE); + post_alloc_hook(page, order, __GFP_MOVABLE, USER_ADDR_NONE); set_page_refcounted(page); return page; } @@ -1849,8 +1849,7 @@ static struct folio *compaction_alloc_noprof(struct f= olio *src, unsigned long da set_page_private(&freepage[size], start_order); } dst =3D (struct folio *)freepage; - - post_alloc_hook(&dst->page, order, __GFP_MOVABLE); + post_alloc_hook(&dst->page, order, __GFP_MOVABLE, USER_ADDR_NONE); set_page_refcounted(&dst->page); if (order) prep_compound_page(&dst->page, order); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index f24bf49be047..a999f3ead852 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1806,7 +1806,8 @@ struct address_space *hugetlb_folio_mapping_lock_writ= e(struct folio *folio) } =20 static struct folio *alloc_buddy_frozen_folio(int order, gfp_t gfp_mask, - int nid, nodemask_t *nmask, nodemask_t *node_alloc_noretry) + int nid, nodemask_t *nmask, nodemask_t *node_alloc_noretry, + unsigned long addr) { struct folio *folio; bool alloc_try_hard =3D true; @@ -1823,7 +1824,7 @@ static struct folio *alloc_buddy_frozen_folio(int ord= er, gfp_t gfp_mask, if (alloc_try_hard) gfp_mask |=3D __GFP_RETRY_MAYFAIL; =20 - folio =3D (struct folio *)__alloc_frozen_pages(gfp_mask, order, nid, nmas= k); + folio =3D (struct folio *)__alloc_frozen_pages(gfp_mask, order, nid, nmas= k, addr); =20 /* * If we did not specify __GFP_RETRY_MAYFAIL, but still got a @@ -1852,7 +1853,7 @@ static struct folio *alloc_buddy_frozen_folio(int ord= er, gfp_t gfp_mask, =20 static struct folio *only_alloc_fresh_hugetlb_folio(struct hstate *h, gfp_t gfp_mask, int nid, nodemask_t *nmask, - nodemask_t *node_alloc_noretry) + nodemask_t *node_alloc_noretry, unsigned long addr) { struct folio *folio; int order =3D huge_page_order(h); @@ -1864,7 +1865,7 @@ static struct folio *only_alloc_fresh_hugetlb_folio(s= truct hstate *h, folio =3D alloc_gigantic_frozen_folio(order, gfp_mask, nid, nmask); else folio =3D alloc_buddy_frozen_folio(order, gfp_mask, nid, nmask, - node_alloc_noretry); + node_alloc_noretry, addr); if (folio) init_new_hugetlb_folio(folio); return folio; @@ -1878,11 +1879,12 @@ static struct folio *only_alloc_fresh_hugetlb_folio= (struct hstate *h, * pages is zero, and the accounting must be done in the caller. */ static struct folio *alloc_fresh_hugetlb_folio(struct hstate *h, - gfp_t gfp_mask, int nid, nodemask_t *nmask) + gfp_t gfp_mask, int nid, nodemask_t *nmask, + unsigned long addr) { struct folio *folio; =20 - folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, NULL); + folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, NULL, a= ddr); if (folio) hugetlb_vmemmap_optimize_folio(h, folio); return folio; @@ -1922,7 +1924,7 @@ static struct folio *alloc_pool_huge_folio(struct hst= ate *h, struct folio *folio; =20 folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, node, - nodes_allowed, node_alloc_noretry); + nodes_allowed, node_alloc_noretry, USER_ADDR_NONE); if (folio) return folio; } @@ -2091,7 +2093,8 @@ int dissolve_free_hugetlb_folios(unsigned long start_= pfn, unsigned long end_pfn) * Allocates a fresh surplus page from the page allocator. */ static struct folio *alloc_surplus_hugetlb_folio(struct hstate *h, - gfp_t gfp_mask, int nid, nodemask_t *nmask) + gfp_t gfp_mask, int nid, nodemask_t *nmask, + unsigned long addr) { struct folio *folio =3D NULL; =20 @@ -2103,7 +2106,7 @@ static struct folio *alloc_surplus_hugetlb_folio(stru= ct hstate *h, goto out_unlock; spin_unlock_irq(&hugetlb_lock); =20 - folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask); + folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, addr); if (!folio) return NULL; =20 @@ -2146,7 +2149,7 @@ static struct folio *alloc_migrate_hugetlb_folio(stru= ct hstate *h, gfp_t gfp_mas if (hstate_is_gigantic(h)) return NULL; =20 - folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask); + folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, USER_ADDR_NO= NE); if (!folio) return NULL; =20 @@ -2182,14 +2185,14 @@ struct folio *alloc_buddy_hugetlb_folio_with_mpol(s= truct hstate *h, if (mpol_is_preferred_many(mpol)) { gfp_t gfp =3D gfp_mask & ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL); =20 - folio =3D alloc_surplus_hugetlb_folio(h, gfp, nid, nodemask); + folio =3D alloc_surplus_hugetlb_folio(h, gfp, nid, nodemask, addr); =20 /* Fallback to all nodes if page=3D=3DNULL */ nodemask =3D NULL; } =20 if (!folio) - folio =3D alloc_surplus_hugetlb_folio(h, gfp_mask, nid, nodemask); + folio =3D alloc_surplus_hugetlb_folio(h, gfp_mask, nid, nodemask, addr); mpol_cond_put(mpol); return folio; } @@ -2296,7 +2299,8 @@ static int gather_surplus_pages(struct hstate *h, lon= g delta) * down the road to pick the current node if that is the case. */ folio =3D alloc_surplus_hugetlb_folio(h, htlb_alloc_mask(h), - NUMA_NO_NODE, &alloc_nodemask); + NUMA_NO_NODE, &alloc_nodemask, + USER_ADDR_NONE); if (!folio) { alloc_ok =3D false; break; @@ -2702,7 +2706,7 @@ static int alloc_and_dissolve_hugetlb_folio(struct fo= lio *old_folio, spin_unlock_irq(&hugetlb_lock); gfp_mask =3D htlb_alloc_mask(h) | __GFP_THISNODE; new_folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, - nid, NULL); + nid, NULL, USER_ADDR_NONE); if (!new_folio) return -ENOMEM; goto retry; @@ -3400,13 +3404,13 @@ static void __init hugetlb_hstate_alloc_pages_oneno= de(struct hstate *h, int nid) gfp_t gfp_mask =3D htlb_alloc_mask(h) | __GFP_THISNODE; =20 folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, - &node_states[N_MEMORY], NULL); + &node_states[N_MEMORY], NULL, USER_ADDR_NONE); if (!folio && !list_empty(&folio_list) && hugetlb_vmemmap_optimizable_size(h)) { prep_and_add_allocated_folios(h, &folio_list); INIT_LIST_HEAD(&folio_list); folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, - &node_states[N_MEMORY], NULL); + &node_states[N_MEMORY], NULL, USER_ADDR_NONE); } if (!folio) break; diff --git a/mm/internal.h b/mm/internal.h index 5a2ddcf68e0b..389098200aa6 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -662,6 +662,16 @@ void calculate_min_free_kbytes(void); int __meminit init_per_zone_wmark_min(void); void page_alloc_sysctl_init(void); =20 +/* + * Sentinel for user_addr: indicates a non-user allocation. + * Cannot use 0 because address 0 is a valid userspace mapping. + * (unsigned long)-1 is safe because: + * 1. vm_end =3D addr + len <=3D TASK_SIZE, and vm_end is exclusive, + * so -1 is never inside any VMA. + * 2. It will only be compared to page-aligned addresses. + */ +#define USER_ADDR_NONE ((unsigned long)-1) + /* * Structure for holding the mostly immutable allocation parameters passed * between functions involved in allocations, including the alloc_pages* @@ -693,6 +703,7 @@ struct alloc_context { */ enum zone_type highest_zoneidx; bool spread_dirty_pages; + unsigned long user_addr; }; =20 /* @@ -916,24 +927,29 @@ static inline void init_compound_tail(struct page *ta= il, prep_compound_tail(tail, head, order); } =20 -void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flag= s); +void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flag= s, + unsigned long user_addr); extern bool free_pages_prepare(struct page *page, unsigned int order); =20 extern int user_min_free_kbytes; =20 struct page *__alloc_frozen_pages_noprof(gfp_t, unsigned int order, int ni= d, - nodemask_t *); + nodemask_t *, unsigned long user_addr); #define __alloc_frozen_pages(...) \ alloc_hooks(__alloc_frozen_pages_noprof(__VA_ARGS__)) void free_frozen_pages(struct page *page, unsigned int order); +void free_frozen_pages_zeroed(struct page *page, unsigned int order); void free_unref_folios(struct folio_batch *fbatch); =20 #ifdef CONFIG_NUMA struct page *alloc_frozen_pages_noprof(gfp_t, unsigned int order); +struct folio *folio_alloc_mpol_user_noprof(gfp_t gfp, unsigned int order, + struct mempolicy *pol, pgoff_t ilx, int nid, + unsigned long user_addr); #else static inline struct page *alloc_frozen_pages_noprof(gfp_t gfp, unsigned i= nt order) { - return __alloc_frozen_pages_noprof(gfp, order, numa_node_id(), NULL); + return __alloc_frozen_pages_noprof(gfp, order, numa_node_id(), NULL, USER= _ADDR_NONE); } #endif =20 diff --git a/mm/mempolicy.c b/mm/mempolicy.c index a1707ad498a8..f573ff32e94d 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -2413,7 +2413,8 @@ bool mempolicy_in_oom_domain(struct task_struct *tsk, } =20 static struct page *alloc_pages_preferred_many(gfp_t gfp, unsigned int ord= er, - int nid, nodemask_t *nodemask) + int nid, nodemask_t *nodemask, + unsigned long user_addr) { struct page *page; gfp_t preferred_gfp; @@ -2426,25 +2427,29 @@ static struct page *alloc_pages_preferred_many(gfp_= t gfp, unsigned int order, */ preferred_gfp =3D gfp | __GFP_NOWARN; preferred_gfp &=3D ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL); - page =3D __alloc_frozen_pages_noprof(preferred_gfp, order, nid, nodemask); + page =3D __alloc_frozen_pages_noprof(preferred_gfp, order, nid, + nodemask, user_addr); if (!page) - page =3D __alloc_frozen_pages_noprof(gfp, order, nid, NULL); + page =3D __alloc_frozen_pages_noprof(gfp, order, nid, NULL, + user_addr); =20 return page; } =20 /** - * alloc_pages_mpol - Allocate pages according to NUMA mempolicy. + * __alloc_pages_mpol - Allocate pages according to NUMA mempolicy. * @gfp: GFP flags. * @order: Order of the page allocation. * @pol: Pointer to the NUMA mempolicy. * @ilx: Index for interleave mempolicy (also distinguishes alloc_pages()). * @nid: Preferred node (usually numa_node_id() but @mpol may override it). + * @user_addr: User fault address for cache-friendly zeroing, or USER_ADDR= _NONE. * * Return: The page on success or NULL if allocation fails. */ -static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order, - struct mempolicy *pol, pgoff_t ilx, int nid) +static struct page *__alloc_pages_mpol(gfp_t gfp, unsigned int order, + struct mempolicy *pol, pgoff_t ilx, int nid, + unsigned long user_addr) { nodemask_t *nodemask; struct page *page; @@ -2452,7 +2457,8 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsig= ned int order, nodemask =3D policy_nodemask(gfp, pol, ilx, &nid); =20 if (pol->mode =3D=3D MPOL_PREFERRED_MANY) - return alloc_pages_preferred_many(gfp, order, nid, nodemask); + return alloc_pages_preferred_many(gfp, order, nid, nodemask, + user_addr); =20 if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && /* filter "hugepage" allocation, unless from alloc_pages() */ @@ -2476,7 +2482,7 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsig= ned int order, */ page =3D __alloc_frozen_pages_noprof( gfp | __GFP_THISNODE | __GFP_NORETRY, order, - nid, NULL); + nid, NULL, user_addr); if (page || !(gfp & __GFP_DIRECT_RECLAIM)) return page; /* @@ -2488,7 +2494,7 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsig= ned int order, } } =20 - page =3D __alloc_frozen_pages_noprof(gfp, order, nid, nodemask); + page =3D __alloc_frozen_pages_noprof(gfp, order, nid, nodemask, user_addr= ); =20 if (unlikely(pol->mode =3D=3D MPOL_INTERLEAVE || pol->mode =3D=3D MPOL_WEIGHTED_INTERLEAVE) && page) { @@ -2504,11 +2510,18 @@ static struct page *alloc_pages_mpol(gfp_t gfp, uns= igned int order, return page; } =20 -struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order, +static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order, struct mempolicy *pol, pgoff_t ilx, int nid) { - struct page *page =3D alloc_pages_mpol(gfp | __GFP_COMP, order, pol, - ilx, nid); + return __alloc_pages_mpol(gfp, order, pol, ilx, nid, USER_ADDR_NONE); +} + +struct folio *folio_alloc_mpol_user_noprof(gfp_t gfp, unsigned int order, + struct mempolicy *pol, pgoff_t ilx, int nid, + unsigned long user_addr) +{ + struct page *page =3D __alloc_pages_mpol(gfp | __GFP_COMP, order, pol, + ilx, nid, user_addr); if (!page) return NULL; =20 @@ -2516,6 +2529,13 @@ struct folio *folio_alloc_mpol_noprof(gfp_t gfp, uns= igned int order, return page_rmappable_folio(page); } =20 +struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order, + struct mempolicy *pol, pgoff_t ilx, int nid) +{ + return folio_alloc_mpol_user_noprof(gfp, order, pol, ilx, nid, + USER_ADDR_NONE); +} + struct page *alloc_frozen_pages_noprof(gfp_t gfp, unsigned order) { struct mempolicy *pol =3D &default_policy; diff --git a/mm/mmap.c b/mm/mmap.c index 5754d1c36462..73413cebc418 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -855,6 +855,12 @@ __get_unmapped_area(struct file *file, unsigned long a= ddr, unsigned long len, if (IS_ERR_VALUE(addr)) return addr; =20 + /* + * The check below ensures vm_end =3D addr + len <=3D TASK_SIZE. + * Since (unsigned long)-1 (USER_ADDR_NONE) >=3D TASK_SIZE and + * vm_end is exclusive, USER_ADDR_NONE is thus never a valid + * userspace address. + */ if (addr > TASK_SIZE - len) return -ENOMEM; if (offset_in_page(addr)) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 0c4f4c678233..b96c9892f6c6 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1819,7 +1819,7 @@ static inline bool should_skip_init(gfp_t flags) } =20 inline void post_alloc_hook(struct page *page, unsigned int order, - gfp_t gfp_flags) + gfp_t gfp_flags, unsigned long user_addr) { bool init =3D !want_init_on_free() && want_init_on_alloc(gfp_flags) && !should_skip_init(gfp_flags); @@ -1874,9 +1874,10 @@ inline void post_alloc_hook(struct page *page, unsig= ned int order, } =20 static void prep_new_page(struct page *page, unsigned int order, gfp_t gfp= _flags, - unsigned int alloc_flags) + unsigned int alloc_flags, + unsigned long user_addr) { - post_alloc_hook(page, order, gfp_flags); + post_alloc_hook(page, order, gfp_flags, user_addr); =20 if (order && (gfp_flags & __GFP_COMP)) prep_compound_page(page, order); @@ -3958,7 +3959,8 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int o= rder, int alloc_flags, page =3D rmqueue(zonelist_zone(ac->preferred_zoneref), zone, order, gfp_mask, alloc_flags, ac->migratetype); if (page) { - prep_new_page(page, order, gfp_mask, alloc_flags); + prep_new_page(page, order, gfp_mask, alloc_flags, + ac->user_addr); =20 return page; } else { @@ -4186,7 +4188,8 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned= int order, =20 /* Prep a captured page if available */ if (page) - prep_new_page(page, order, gfp_mask, alloc_flags); + prep_new_page(page, order, gfp_mask, alloc_flags, + ac->user_addr); =20 /* Try get a page from the freelist if available */ if (!page) @@ -5063,7 +5066,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, struct zoneref *z; struct per_cpu_pages *pcp; struct list_head *pcp_list; - struct alloc_context ac; + struct alloc_context ac =3D { .user_addr =3D USER_ADDR_NONE }; gfp_t alloc_gfp; unsigned int alloc_flags =3D ALLOC_WMARK_LOW; int nr_populated =3D 0, nr_account =3D 0; @@ -5178,7 +5181,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, } nr_account++; =20 - prep_new_page(page, 0, gfp, 0); + prep_new_page(page, 0, gfp, 0, USER_ADDR_NONE); set_page_refcounted(page); page_array[nr_populated++] =3D page; } @@ -5203,12 +5206,13 @@ EXPORT_SYMBOL_GPL(alloc_pages_bulk_noprof); * This is the 'heart' of the zoned buddy allocator. */ struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order, - int preferred_nid, nodemask_t *nodemask) + int preferred_nid, nodemask_t *nodemask, + unsigned long user_addr) { struct page *page; unsigned int alloc_flags =3D ALLOC_WMARK_LOW; gfp_t alloc_gfp; /* The gfp_t that was actually used for allocation */ - struct alloc_context ac =3D { }; + struct alloc_context ac =3D { .user_addr =3D user_addr }; =20 /* * There are several places where we assume that the order value is sane @@ -5269,10 +5273,12 @@ EXPORT_SYMBOL(__alloc_frozen_pages_noprof); =20 struct page *__alloc_pages_noprof(gfp_t gfp, unsigned int order, int preferred_nid, nodemask_t *nodemask) + { struct page *page; =20 - page =3D __alloc_frozen_pages_noprof(gfp, order, preferred_nid, nodemask); + page =3D __alloc_frozen_pages_noprof(gfp, order, preferred_nid, + nodemask, USER_ADDR_NONE); if (page) set_page_refcounted(page); return page; @@ -5315,7 +5321,8 @@ struct folio *vma_alloc_folio_noprof(gfp_t gfp, int o= rder, gfp |=3D __GFP_NOWARN; =20 pol =3D get_vma_policy(vma, addr, order, &ilx); - folio =3D folio_alloc_mpol_noprof(gfp, order, pol, ilx, numa_node_id()); + folio =3D folio_alloc_mpol_user_noprof(gfp, order, pol, ilx, + numa_node_id(), addr); mpol_cond_put(pol); return folio; } @@ -5323,10 +5330,17 @@ struct folio *vma_alloc_folio_noprof(gfp_t gfp, int= order, struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, struct vm_area_struct *vma, unsigned long addr) { + struct page *page; + if (vma->vm_flags & VM_DROPPABLE) gfp |=3D __GFP_NOWARN; =20 - return folio_alloc_noprof(gfp, order); + page =3D __alloc_frozen_pages_noprof(gfp | __GFP_COMP, order, + numa_node_id(), NULL, addr); + if (!page) + return NULL; + set_page_refcounted(page); + return page_rmappable_folio(page); } #endif EXPORT_SYMBOL(vma_alloc_folio_noprof); @@ -6907,7 +6921,7 @@ static void split_free_frozen_pages(struct list_head = *list, gfp_t gfp_mask) list_for_each_entry_safe(page, next, &list[order], lru) { int i; =20 - post_alloc_hook(page, order, gfp_mask); + post_alloc_hook(page, order, gfp_mask, USER_ADDR_NONE); if (!order) continue; =20 @@ -7113,7 +7127,7 @@ int alloc_contig_frozen_range_noprof(unsigned long st= art, unsigned long end, struct page *head =3D pfn_to_page(start); =20 check_new_pages(head, order); - prep_new_page(head, order, gfp_mask, 0); + prep_new_page(head, order, gfp_mask, 0, USER_ADDR_NONE); } else { ret =3D -EINVAL; WARN(true, "PFN range: requested [%lu, %lu), allocated [%lu, %lu)\n", @@ -7778,7 +7792,7 @@ struct page *alloc_frozen_pages_nolock_noprof(gfp_t g= fp_flags, int nid, unsigned gfp_t alloc_gfp =3D __GFP_NOWARN | __GFP_ZERO | __GFP_NOMEMALLOC | __GFP_= COMP | gfp_flags; unsigned int alloc_flags =3D ALLOC_TRYLOCK; - struct alloc_context ac =3D { }; + struct alloc_context ac =3D { .user_addr =3D USER_ADDR_NONE }; struct page *page; =20 VM_WARN_ON_ONCE(gfp_flags & ~__GFP_ACCOUNT); diff --git a/mm/slub.c b/mm/slub.c index 0baa906f39ab..74dd2d96941b 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -3275,7 +3275,7 @@ static inline struct slab *alloc_slab_page(gfp_t flag= s, int node, else if (node =3D=3D NUMA_NO_NODE) page =3D alloc_frozen_pages(flags, order); else - page =3D __alloc_frozen_pages(flags, order, node, NULL); + page =3D __alloc_frozen_pages(flags, order, node, NULL, USER_ADDR_NONE); =20 if (!page) return NULL; @@ -5235,7 +5235,7 @@ static void *___kmalloc_large_node(size_t size, gfp_t= flags, int node) if (node =3D=3D NUMA_NO_NODE) page =3D alloc_frozen_pages_noprof(flags, order); else - page =3D __alloc_frozen_pages_noprof(flags, order, node, NULL); + page =3D __alloc_frozen_pages_noprof(flags, order, node, NULL, USER_ADDR= _NONE); =20 if (page) { ptr =3D page_address(page); --=20 MST From nobody Mon Jun 8 10:56:13 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 88AA43F0AA8 for ; Fri, 29 May 2026 15:22:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068173; cv=none; b=Hom7Xg/1CZ2Jaj7gErjcTH9oy7f2ztykKG5Xi0p3813NFqzadumuip/GBJH43zq+F/rPHGUcJZpErx7roP+1OoqFcDXvE320fZ7NFXAJNnFmbF5XR5Vh4b68h/Fl4A6VFlBcWQI+BN3F4ncJ13sNXJa9Fllp/VB126zgdjCp+dw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068173; c=relaxed/simple; bh=8w0BAdcYEx1DYP7LpA2admc4ftuLWgtvGXP7cauVxpA=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=uONGCKfvcaQmju6SoIT6rlh2WLQUwZHh0FGwJHdxdSN5xSlvixfLDOt4RVYmDc6e83GNUIN/+i/LY8ysmWxWUIa7vzlG+1YgOt5pf6904zO8B7XqjjBvaSmbKZod/8eNPB09RdCDf6NlevKMTkvkQqqQRRhBpRLG/vke4WmDufE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=ErqOQrWV; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=Q04WtqBd; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ErqOQrWV"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="Q04WtqBd" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068170; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=VBSoYVls/goaUOaeb0ArVPkRixJg6uACMT8ZdsicbY4=; b=ErqOQrWVLJzuVZwWYDc0q1rLvwdOrOxMrbMMDsuZVy+QMsW0NTjdQ1v+VkV09ffsuEUY3D 6JqlWmVKzNK5cfUbM8lTtbetDIalQnVFkUoXqMuWdWYncClZGA5af73vuL10GPSP48X53g uhgcmVs+EbjbtzgWu6T1rqdLQmKFSKo= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-371-kesy4s9SMfe7tLUIqN7rKw-1; Fri, 29 May 2026 11:22:49 -0400 X-MC-Unique: kesy4s9SMfe7tLUIqN7rKw-1 X-Mimecast-MFC-AGG-ID: kesy4s9SMfe7tLUIqN7rKw_1780068168 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-490261c79fcso55614025e9.0 for ; Fri, 29 May 2026 08:22:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068168; x=1780672968; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=VBSoYVls/goaUOaeb0ArVPkRixJg6uACMT8ZdsicbY4=; b=Q04WtqBdVcZQD4UJjwiOsDFmDIgEzOy4SmEDM0RJLilto3Pni1gNPTo8YLR6F4KNmO KustQBvZkmve0gCnoBvIheNXPvvamVDEuAHk7akrXhmfoQANCzfg5G28KmTcfmisxaYT mAkUbH1p52ke7H+6BqIGWiVT1VMUDfFj39ZHgRCeqbG/WxpHzP5NKe6lIKIwtgzQs+rV dQoBTgC63JGYkH0m033GbxpSrKdlAAY4z//n6ZcOnc0L/jn3riLG0xIvrBJzz65KSUAu DnwbagaiDsrcdCjlrQCUiGkTdOXXM6rRnVOK4MB/Jm6oVR9U9SOFY0LimDFOgYtP8qgx g4Hw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068168; x=1780672968; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VBSoYVls/goaUOaeb0ArVPkRixJg6uACMT8ZdsicbY4=; b=blQx7qulvieZ+XbPDctXzJhLHNVfKG66pEhdNXyc6whMxUAvEARPM+fGArjQo2jBVM m9MahqhnFT4hmuxrC3ubMHQ+tjWj7HIShWc4UQ5N98iNgbvArDqtWDtws/aXduipQhJP 7q8A3So9z9yNEea31qsjDg8EkwQDz259ZKvE5jFOi3ku7+YrH0byjVgscb/enwckvl/o Ymar9HFL8zH+QNU4ZrD3cmmjhfy54wqf0tLFcUhEqMIpnHCt/QFi+Mxu6XceJS955CAR frBT3Hp0sGdT94Q0voQgbSuDHTbUKCOq2yKelszEuhAlEkRZXzIFStkj9TcbXGFCBCFx VyNw== X-Gm-Message-State: AOJu0YwAzPhtt1GoPhImwjRYc2CJ3MV4JPdI/ePAlU9GOSNCj1aJSXks YpZyiK14giVvKm/OAjUL1x4z/V9cFB2jx07+p2987hi6K2Gk5goxejXUiuV0qYkDGtJsfRATSxv sFyOeBdI0kQgLIVSyltxrRxEq2y5tZO9Hx5ubcmCAhEBVd5JpQkWcDfeVE1iv3tbPIaPTVOekDQ xbwblocls7vJ5b6L7YP1MUxD8K1ra3NwdjifDm3LLx4wY= X-Gm-Gg: Acq92OEYEOmRhky3jfir8SvQlAugePTRr8fOfGs0+BFSL7iWw8XsED6SYiJlq7Ty30E oVEbiTyPO/U9rzO+SWTWjRNREywogrBmUFqbu19KloFHzJu4VqrLOU7CwBYjA4SL9kXZx4ffqbr k4QAFsV/gmxKNGe4SXm9oSN0yy1TuZxyWd83IepLTDEK0oOVtcjo1jsi2vVx5pvlh/Wi4r1wold L8sdKy5ETADF8Sf0O75gXsctfCSej4v/sgY/Zl35p+qOJow2xG9qtnYGkg/zSVHmwrYTbh9i29B U1UJLnAnPgFs1gfJn2usGF1pBHyIiaKN4GJOqH6LA+f9lQ0h7bjbP4cBhtvd0zCwPIADov9uOem OkCo6lNYub/RetCQi6F1O0RDwTzSYhmmZ+8+sDvu/Nubt2j35NFp2tQ== X-Received: by 2002:a05:600c:8207:b0:48f:e044:927d with SMTP id 5b1f17b1804b1-4909c62336fmr43495415e9.10.1780068167609; Fri, 29 May 2026 08:22:47 -0700 (PDT) X-Received: by 2002:a05:600c:8207:b0:48f:e044:927d with SMTP id 5b1f17b1804b1-4909c62336fmr43494295e9.10.1780068166844; Fri, 29 May 2026 08:22:46 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4909caa7faasm50679705e9.11.2026.05.29.08.22.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:22:46 -0700 (PDT) Date: Fri, 29 May 2026 11:22:42 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 08/37] mm: add alloc_contig_frozen_pages_user for cache-friendly zeroing Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a _user variant of alloc_contig_frozen_pages that accepts a user_addr parameter for cache-friendly zeroing of contiguous allocations. No functional change; all existing callers continue to pass USER_ADDR_NONE. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/gfp.h | 6 ++++++ mm/page_alloc.c | 32 +++++++++++++++++++++++++------- 2 files changed, 31 insertions(+), 7 deletions(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index ee35c5367abc..73109d4e31a4 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -453,6 +453,12 @@ struct page *alloc_contig_frozen_pages_noprof(unsigned= long nr_pages, #define alloc_contig_frozen_pages(...) \ alloc_hooks(alloc_contig_frozen_pages_noprof(__VA_ARGS__)) =20 +struct page *alloc_contig_frozen_pages_user_noprof(unsigned long nr_pages, + gfp_t gfp_mask, int nid, nodemask_t *nodemask, + unsigned long user_addr); +#define alloc_contig_frozen_pages_user(...) \ + alloc_hooks(alloc_contig_frozen_pages_user_noprof(__VA_ARGS__)) + struct page *alloc_contig_pages_noprof(unsigned long nr_pages, gfp_t gfp_m= ask, int nid, nodemask_t *nodemask); #define alloc_contig_pages(...) \ diff --git a/mm/page_alloc.c b/mm/page_alloc.c index b96c9892f6c6..b725a3b59835 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -6999,8 +6999,9 @@ static void __free_contig_frozen_range(unsigned long = pfn, unsigned long nr_pages * * Return: zero on success or negative error code. */ -int alloc_contig_frozen_range_noprof(unsigned long start, unsigned long en= d, - acr_flags_t alloc_flags, gfp_t gfp_mask) +static int __alloc_contig_frozen_range(unsigned long start, unsigned long = end, + acr_flags_t alloc_flags, gfp_t gfp_mask, + unsigned long user_addr) { const unsigned int order =3D ilog2(end - start); unsigned long outer_start, outer_end; @@ -7127,7 +7128,7 @@ int alloc_contig_frozen_range_noprof(unsigned long st= art, unsigned long end, struct page *head =3D pfn_to_page(start); =20 check_new_pages(head, order); - prep_new_page(head, order, gfp_mask, 0, USER_ADDR_NONE); + prep_new_page(head, order, gfp_mask, 0, user_addr); } else { ret =3D -EINVAL; WARN(true, "PFN range: requested [%lu, %lu), allocated [%lu, %lu)\n", @@ -7137,6 +7138,13 @@ int alloc_contig_frozen_range_noprof(unsigned long s= tart, unsigned long end, undo_isolate_page_range(start, end); return ret; } + +int alloc_contig_frozen_range_noprof(unsigned long start, unsigned long en= d, + acr_flags_t alloc_flags, gfp_t gfp_mask) +{ + return __alloc_contig_frozen_range(start, end, alloc_flags, gfp_mask, + USER_ADDR_NONE); +} EXPORT_SYMBOL(alloc_contig_frozen_range_noprof); =20 /** @@ -7255,8 +7263,9 @@ static bool zone_spans_last_pfn(const struct zone *zo= ne, * * Return: pointer to contiguous frozen pages on success, or NULL if not s= uccessful. */ -struct page *alloc_contig_frozen_pages_noprof(unsigned long nr_pages, - gfp_t gfp_mask, int nid, nodemask_t *nodemask) +struct page *alloc_contig_frozen_pages_user_noprof(unsigned long nr_pages, + gfp_t gfp_mask, int nid, nodemask_t *nodemask, + unsigned long user_addr) { unsigned long ret, pfn, flags; struct zonelist *zonelist; @@ -7284,10 +7293,11 @@ struct page *alloc_contig_frozen_pages_noprof(unsig= ned long nr_pages, * win the race and cause allocation to fail. */ spin_unlock_irqrestore(&zone->lock, flags); - ret =3D alloc_contig_frozen_range_noprof(pfn, + ret =3D __alloc_contig_frozen_range(pfn, pfn + nr_pages, ACR_FLAGS_NONE, - gfp_mask); + gfp_mask, + user_addr); if (!ret) return pfn_to_page(pfn); spin_lock_irqsave(&zone->lock, flags); @@ -7309,6 +7319,14 @@ struct page *alloc_contig_frozen_pages_noprof(unsign= ed long nr_pages, } return NULL; } +EXPORT_SYMBOL(alloc_contig_frozen_pages_user_noprof); + +struct page *alloc_contig_frozen_pages_noprof(unsigned long nr_pages, + gfp_t gfp_mask, int nid, nodemask_t *nodemask) +{ + return alloc_contig_frozen_pages_user_noprof(nr_pages, gfp_mask, nid, + nodemask, USER_ADDR_NONE); +} EXPORT_SYMBOL(alloc_contig_frozen_pages_noprof); =20 /** --=20 MST From nobody Mon Jun 8 10:56:13 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A46213F4DF1 for ; Fri, 29 May 2026 15:22:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068178; cv=none; b=ruaropZ5J85eyQyHxPO5xR5tOLA79nAWdVImFZOA+Hk3/VJ9zjoBSW7hEbdIZSqoNjDoLvsi7neGLQshUznJTf9MaYoaLmU7/JtXBI0Z9lx2AepR9jUcj4CWntWk9bBxFOOxb7haMlhzbKSDwAtI/WNY5SRopQcaf30BOfJIf2s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068178; c=relaxed/simple; bh=nrsmIXcyZYzWzlqPkl9t9uUpVmTvW8/nYuSQvt237DY=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=YHfwF7tnU6c8pLeSVKRXFqhmX2c+NqGl1oQItAdTU/gyW+t0bWwdV26GCUrwHdmoRL50DvMFDwWyoBaViP6cdp2cP/6JsutLnmOvEYivClsVj68nqXEgOIh7GxNTzsC4RjtD8laz3hNX00Pcytast5hxh6+X0s2IxaPZqgSgE7Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=NrMS/+Av; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=rNe7n0+Z; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="NrMS/+Av"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="rNe7n0+Z" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068175; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Gi0K9k7DoK/Ig5tkAzEbdYCqiHQSfFH4LjBzL6zaPFI=; b=NrMS/+AvGK8G2UU45iMquP7C7KhQCiXeZlRAbzH6ec3m4VlbdMXu6H8NmNPqBK5yH6ZHWU gbofqMKvv//7dfSO1a4q/dhDAggAn0SezuQpAY5jKgOu52Vx5qWEvikizj9hMPYzidSvEb spvV39dSneHl8hiLzVrGJmqvevkxL8E= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-340-Vaw-hgc5Ndu4Au4vWxGohQ-1; Fri, 29 May 2026 11:22:54 -0400 X-MC-Unique: Vaw-hgc5Ndu4Au4vWxGohQ-1 X-Mimecast-MFC-AGG-ID: Vaw-hgc5Ndu4Au4vWxGohQ_1780068173 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-4909deb82d0so5868575e9.1 for ; Fri, 29 May 2026 08:22:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068173; x=1780672973; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Gi0K9k7DoK/Ig5tkAzEbdYCqiHQSfFH4LjBzL6zaPFI=; b=rNe7n0+Z9IMuaKCzU0QUbzI8OwbZwtTr1Mk81bgkbLIsYPsLwrzjZLvxNcu0RJH3pu 2eDn5kP2Beo8vWekDzi18wtyuBat+nVjxTh+pXvyVfkot8RpfK7ksH0WqVdVq2XbSZA8 tDJbC7lVu6RVIUMrDiH6LGb1c7Ds7JvAtqHrH+htduI5gEVIdJPKPNF8ZGWSbuGbBPhL QPe+mbMb+/X2Ip4doUKt11pb/r5fCIWeF+67RYMPUA/hlDUTEaAuLIDcxjsymB93jLgg MSAe18FWoAeuuxcZCFNwIeirN37FGyo5V8/QNLX5InK2m2V46d1f7B5WWYuMar38Kxq2 jB5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068173; x=1780672973; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Gi0K9k7DoK/Ig5tkAzEbdYCqiHQSfFH4LjBzL6zaPFI=; b=WaVAMR6YNxeEJ/z6dd/VLUEVhnOzIqfPpYWrgjkUjv3SsZC0q2c603bz+6M+2MQtjN vps8p47J9w78X3pGBLpkyLwbrL2qHkLv8D+lcuJ9J/2d/TJzP7Vu2r/pbNLABaFR6uMS FqRqprhU1Qi3UGIdWR1MFz9VgJmo9XhAkbSmu/+Kt4mLIf6sIWB5UQYLtP4b1ZWe+tsV BhxOuidRA6KOSd/TBmkDax5vsmu61IC3plsJaRofF4tyEDDc6o9HMO0g0mAD0grQ2PNx NzSzMtFD0g43br2jG+db9HFIdwCSQBauBi6nFt7DptYM2LMxlnzGuKy/CvFiAWeaLAdC R+Zg== X-Gm-Message-State: AOJu0YyqDno7iaIfU57AgaTAwv07eUP805epVLqlb7FaPsQtK2gFDpha aNU9I5sYr58j5ODSKetIFFS3V3GBVM8Xk49zVZQ5+lmhKMhY0r4tMkl7J4RCXYiWUWgSV8dYGBS 9dzk7YsrbvpFHWXTzkVFCHy1Znlxu23bWRiri3FpR+12VUTM14CATilsBM3LEsO8W+zSqncbk77 iTKOVTQH1JdhjVobhqi88l7FDwIak+prBPcU2BqsYQk2A= X-Gm-Gg: Acq92OE3vJF1/0zjmi6u78VlxgXH5i6O0je0tYy2gw4xl3IhHIDREpe+xV19LEwnXvw 1eoNfwvJBPfV11OPqj2bCf3BdDg4IZltGGLQ7+pqe403zlh0l/UVVeAjTShpZPNnxS5KkKLblQy rTsV9seOn9ETNjogItmkbdhNQ0ru/0IyykVO4FV1ortRkDLMwZIqJd5KYmGbp4M/B1xrP6RJ5cz NM/3vFoLQLC28XIrRr54Y1oYK2vdlJIm4EO80FH8r8iaflDDEpyMFVDEO4Z5ZbStdDGM37xM11O 00RpvEaz6Jr5ALDzWPGqlfCk5t2RyDkcwrqMOM6lV+QaVlStyGNIldZMxHzWs4F0cfsk9WtTS/R dKLj8WR7rW11taFv+mxRne/C+s1dRV08cxFpdxuW0QlEHiL3bUaVi+w== X-Received: by 2002:a05:600c:4715:b0:48f:be94:d82c with SMTP id 5b1f17b1804b1-490a293bcaamr1010735e9.19.1780068172710; Fri, 29 May 2026 08:22:52 -0700 (PDT) X-Received: by 2002:a05:600c:4715:b0:48f:be94:d82c with SMTP id 5b1f17b1804b1-490a293bcaamr1009735e9.19.1780068172085; Fri, 29 May 2026 08:22:52 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4909d6975e2sm75634355e9.6.2026.05.29.08.22.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:22:51 -0700 (PDT) Date: Fri, 29 May 2026 11:22:47 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 09/37] mm: hugetlb: thread user_addr through gigantic page allocation Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Thread the user_addr parameter through alloc_gigantic_frozen_folio so that gigantic page allocations can benefit from cache-friendly zeroing. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/hugetlb.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index a999f3ead852..3a6afbe99116 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1375,7 +1375,7 @@ static struct folio *dequeue_hugetlb_folio_vma(struct= hstate *h, =20 #if defined(CONFIG_ARCH_HAS_GIGANTIC_PAGE) && defined(CONFIG_CONTIG_ALLOC) static struct folio *alloc_gigantic_frozen_folio(int order, gfp_t gfp_mask, - int nid, nodemask_t *nodemask) + int nid, nodemask_t *nodemask, unsigned long addr) { struct folio *folio; =20 @@ -1386,13 +1386,15 @@ static struct folio *alloc_gigantic_frozen_folio(in= t order, gfp_t gfp_mask, if (hugetlb_cma_exclusive_alloc()) return NULL; =20 - folio =3D (struct folio *)alloc_contig_frozen_pages(1 << order, gfp_mask, - nid, nodemask); + folio =3D (struct folio *)alloc_contig_frozen_pages_user(1 << order, + gfp_mask, + nid, nodemask, + addr); return folio; } #else /* !CONFIG_ARCH_HAS_GIGANTIC_PAGE || !CONFIG_CONTIG_ALLOC */ static struct folio *alloc_gigantic_frozen_folio(int order, gfp_t gfp_mask= , int nid, - nodemask_t *nodemask) + nodemask_t *nodemask, unsigned long addr) { return NULL; } @@ -1862,7 +1864,8 @@ static struct folio *only_alloc_fresh_hugetlb_folio(s= truct hstate *h, nid =3D numa_mem_id(); =20 if (order_is_gigantic(order)) - folio =3D alloc_gigantic_frozen_folio(order, gfp_mask, nid, nmask); + folio =3D alloc_gigantic_frozen_folio(order, gfp_mask, nid, nmask, + addr); else folio =3D alloc_buddy_frozen_folio(order, gfp_mask, nid, nmask, node_alloc_noretry, addr); --=20 MST From nobody Mon Jun 8 10:56:13 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3E17E3F9F2A for ; Fri, 29 May 2026 15:23:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068188; cv=none; b=f2DZb83fzLgBrkUuInfaLVeaWvE1LeTr+Ydt0ZvIBU2EaLpjBW3v+vAGHgHaT+9X7YK84zpAam8lRBCuhNYOOpPT/t33lkXMjcjtiAsrhxGxiJ5GLqCXGiUSNEN4IXtvMviaDfF0YDPvQY+BI5oSDvOKYDtKx0mYkVKHcAxyu6E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068188; c=relaxed/simple; bh=nrMNVIUtRT6KxUfPFXjAD5jr466RgmTT9RZMroXpfzs=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=k1lzCbjtLoZzBM/+qgOzG+Oiq0LbdDXfW5taNEKDmpEXAdhdWvxY2WZvSFdhh05xgDcwLmKkPKbEZxy9Tbrd3NBK4BxF4f6wUaDk1SS+N2ytOTzWxi5OD16DtEVJrEsmE6FGS0xTa81TInPcPlcZ7ZHYWDcLuLj2xfdk6IpJY9g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=A2CuRXpP; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=d1ws1Uar; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="A2CuRXpP"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="d1ws1Uar" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068185; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=phYVyxy4NXZDWgDwVjZEeX0+DVaGt3sk04puvAHm6ak=; b=A2CuRXpPa/+UPCP7hbfBYRUgDrsLGToNC0KabQI4voB24NNSfPM4MKjd/1Aiv/c57Q+oLs L+mkYzuXfGPHT1K8vy2lqerfdw0Gi/GXtFwcHMHU++zlWsrfXP85VDOA3ok5mIFeMN10Hr 3PqueKj3cNpZMW0fDE3kWC2hLOOeS3Q= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-352-ScjkhzLDPK-ospSHyIA6Jg-1; Fri, 29 May 2026 11:23:01 -0400 X-MC-Unique: ScjkhzLDPK-ospSHyIA6Jg-1 X-Mimecast-MFC-AGG-ID: ScjkhzLDPK-ospSHyIA6Jg_1780068179 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-4904ee02e72so60960875e9.1 for ; Fri, 29 May 2026 08:23:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068179; x=1780672979; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=phYVyxy4NXZDWgDwVjZEeX0+DVaGt3sk04puvAHm6ak=; b=d1ws1UarbbQiXbHGrlb85cZ1iOo3Bzs/fJhhVg7W6iE4/So2x3AE3mMBnNZbxMcP+7 UednORu+wpQ7XB9ZvMKhgG4AdI/xxA7A5kTSsNYE9Vuo8Zexlm8yH1UrADWrg96Y74j+ LtNU/csj6ULmPpfJZZnOAPAio+MsHaA88FRppUB3KFdnpSXna9oB1VQFY7KN3lFG/dVS HKmynsOrL6Ea/3036QtNjHNtnY1L8pm8sWc9f3xaKG+ELQwmHbRoq+fhKA+NyI2m7RTA pBL2BsOY4c92VrsRfuvfT/9cevNrSi2PqUA388Gc3Htzxyy4XMf1j45EncJVl1Oq6vj7 LC9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068179; x=1780672979; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=phYVyxy4NXZDWgDwVjZEeX0+DVaGt3sk04puvAHm6ak=; b=jWFMveQ7No51FL8qe6hp85UC9ltew2MXXwYIIPyXhaWT98ftIxzZevkHZNOxpqL9L1 I9iNQh10228TSo5ExeVZB1yymMbDpOq6S8AP6TtQRtWQgZ/7zSDe8vDWk3h0wPHtloE8 rkBiJXfW67odX3WniEOq+16qRJlBUj9L7VdW18bXqUPf8k+ZPkCqj/COVRpM0NBjMMYR Vn6WuX2GZyItlJE8pfVqkaZH3z6LDuJKQ4zYeenM8bfT1HHnQW1HqAekxwnpTWLOIMNr 8/gGWDZndhJKyYi9YqGkPWBGvyQRF3U4a1hhzHB7WiFI6MCG6xLAral3X3JRFF3fUY8z KWRg== X-Gm-Message-State: AOJu0Yyh/Ch9D6k/+dPmE/Z3WBddCnMcZcFzm7HYdY639jKYs6HB3Kn0 vbohkzFvgfXEgwOl6f391sVJFeoymsfTFmVg/ZX1wTh9Kb69F7zaekmpHCPGdilaqR5/EIBcQok kBzO2zSl4C9PftvsveQLVYTWOeej2LrTah0YJCMucgr4kzNf+7UO15fO8Tjl3wxbesKNlaqBoDX 2TqwAV/fcHOGT4bXK106cuUCgv9tyYG6nQA22xFXeClVA= X-Gm-Gg: Acq92OGBPu/BdIuowpNUIa6i3v4F9JRU02vwZdrbP73x/YhFam4wZN2yKR+C0SozNkX UTVHjr0zZpoMUx1udKE1/mTEO9Ry7EqasYAdJSiucxXyqXHROk6V1DaTi0+aQEmGlrjMUa9aiYk h1wsH7ztJ4130Siyf11OZlKrSAuCC4FrONVXP6pV8rpy8BChoRPaQHjamJAV3OeQ2vn7DOr1T+E 8p6Zf9Y/mO7+SjeHpymFAPv0QrOUZiud+VyZCziAPIcE6cPdLZObe9vDVVedDMIhgU1P6vsDK9H Hcp+LZJmK9Oqnx1csmm8+fHUIHKmC6zqcVd9oJWTnfoql71XTrkusOKRzGENbd9BxOugxIIXjqm sZMZOE4aB19NgHhmMl1V8FUVF51O34Wy8clRx8XF58okXi2rD+oJ+7A== X-Received: by 2002:a05:600c:4686:b0:489:1b10:d896 with SMTP id 5b1f17b1804b1-490a28d3f2emr3053655e9.0.1780068178685; Fri, 29 May 2026 08:22:58 -0700 (PDT) X-Received: by 2002:a05:600c:4686:b0:489:1b10:d896 with SMTP id 5b1f17b1804b1-490a28d3f2emr3052135e9.0.1780068177967; Fri, 29 May 2026 08:22:57 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4909c0498e0sm23826835e9.0.2026.05.29.08.22.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:22:57 -0700 (PDT) Date: Fri, 29 May 2026 11:22:52 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 10/37] mm: add folio_zero_user stub for configs without THP/HUGETLBFS Message-ID: <806b21ab94d4e92e78a092ac6dfbc2564fc6d0ca.1780067977.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" folio_zero_user() is defined in mm/memory.c under CONFIG_TRANSPARENT_HUGEPAGE || CONFIG_HUGETLBFS. A subsequent patch will call it from post_alloc_hook() for all user page zeroing, so configs without THP or HUGETLBFS will need a stub. Add a stub that uses clear_user_highpages() with aligned addr_hint. Without THP/HUGETLBFS, only order-0 user pages are allocated, so the locality optimization in the real folio_zero_user() (zero near the faulting address last) is not needed. This also matches what vma_alloc_zeroed_movable_folio currently does. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- include/linux/mm.h | 2 ++ mm/memory.c | 9 +++++++++ 2 files changed, 11 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index af23453e9dbd..d1e768dcda13 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -5070,6 +5070,8 @@ long copy_folio_from_user(struct folio *dst_folio, const void __user *usr_src, bool allow_pagefault); =20 +#else /* !CONFIG_TRANSPARENT_HUGEPAGE && !CONFIG_HUGETLBFS */ +void folio_zero_user(struct folio *folio, unsigned long addr_hint); #endif /* CONFIG_TRANSPARENT_HUGEPAGE || CONFIG_HUGETLBFS */ =20 #if MAX_NUMNODES > 1 diff --git a/mm/memory.c b/mm/memory.c index ea6568571131..02d3e53fc91b 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -7546,6 +7546,15 @@ long copy_folio_from_user(struct folio *dst_folio, } #endif /* CONFIG_TRANSPARENT_HUGEPAGE || CONFIG_HUGETLBFS */ =20 +#if !defined(CONFIG_TRANSPARENT_HUGEPAGE) && !defined(CONFIG_HUGETLBFS) +void folio_zero_user(struct folio *folio, unsigned long addr_hint) +{ + unsigned long base =3D ALIGN_DOWN(addr_hint, folio_size(folio)); + + clear_user_highpages(&folio->page, base, folio_nr_pages(folio)); +} +#endif + #if defined(CONFIG_SPLIT_PTE_PTLOCKS) && ALLOC_SPLIT_PTLOCKS =20 static struct kmem_cache *page_ptl_cachep; --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3B2123FB07F for ; Fri, 29 May 2026 15:23:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068190; cv=none; b=qNgapuIa0xL+gk3djkZU9ZC0rwklv7q6DX0jsQDeUEqp3LyaN1Dy8ZWWa1fb+7d516s6pccTJn1NBGuPPc5OZsTEY9hWbGduw4j5VyerQa6FoFEJ2dlDYVmCFk3YrOpeOvQAUoFJe1VeacwLp3RCAYaRWctgsBk7ww73CoQxxqk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068190; c=relaxed/simple; bh=W8/RgV8YlHoI/N38IUKKZXJPftBVftzLE1J25cjwbi0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=H1RtvFcJTZ7ss7XE76bai9jLeQLC5CeGsqH1D0h8kSxu6Ce+D3pGEiDbkEPhCQdRvnzr6mg5lyst1FZnglkvn2iFxA76t7+qJW8oKV8zt8BeJ+7SJIMx//TTdiEW0wWus4M3HQrhKSQKTcoCDpyltMvQ9+nrNIg3HoC+dH2vJ88= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=QltAKcxg; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=oE6+avOD; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="QltAKcxg"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="oE6+avOD" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068187; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=kjfty6vgLrS4T7uumYbV6HhVn6mEWZk8nuiVgC9VboM=; b=QltAKcxgNt533q1q+Z51wHw2Ewwn/ACeC98pd2X3JaM5jIGf6GruON3A3QEiBq95maRkb5 DAG9hzluSO3vN4i+zo9i57WdOqIWS6vH1z/ce3DBEGnLpPbJWlm1XY2v7uQbbXRzR6fU3J Nn4eMq5j10XqaeB1MP2Mjf6WGXCs4jQ= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-76-NLgZVBzINNid8R3D81hWHQ-1; Fri, 29 May 2026 11:23:05 -0400 X-MC-Unique: NLgZVBzINNid8R3D81hWHQ-1 X-Mimecast-MFC-AGG-ID: NLgZVBzINNid8R3D81hWHQ_1780068185 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-45aeac88af4so11159252f8f.3 for ; Fri, 29 May 2026 08:23:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068184; x=1780672984; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=kjfty6vgLrS4T7uumYbV6HhVn6mEWZk8nuiVgC9VboM=; b=oE6+avOD4zOmVrEnQoMvxllZv62Y+qqtLiTfCM9FznkaBnRcgzOgzGSUgpWAD/jdfb c5IavWvu2stzxSsOplIu9kYcjhphJJz+qO0X0GDDkwpwvTA0MlsEL5nu8E+oWAVw41Zj llTZdqFfr7jyck3yR8jyzQLXwY71/kDZIcI4PSgNGcZwhcauhMizL29Bi5/9XjCcxXmT wfnpXc5uHCw4Kez2lkYAYtzA1Lk4Vaf4hCmjRytSKmlDj461t85CgywHfLTolckMuauf Nd7sTcFFO94ePjE/2C9L0TAJZHiFGYW+JUk9Xx/CiafnttNR8e3Jxhd/qulI35Tlhu8F bYKg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068184; x=1780672984; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kjfty6vgLrS4T7uumYbV6HhVn6mEWZk8nuiVgC9VboM=; b=kybjyjCJaa4NMUrxpsWeRhLJrT+qM9Ny1tiLmkKYNP292MC7dH2WIY62/luAunbHy+ CRhhbMTF8nlVhF6/7czWFPCg64Cm1gZJUBjA87YGgBUmvkdcEuN0fzkwVaBAwsikU/YH etB+I9/tiWr3PKwQu4NAPuRMY0aRc0PpHgFh1sHSuwZNnyEnH5iEX1y44x5ZQk0B1Eh2 3LaOl/fpRk/f1ljAzgPG90A0dBqeEWzp7O6DNz/xvJopHJPKAdWurhkJ41Gqs3izLLyA QIc/eb3uRRHQSHmVZUER62aBcowJBAU0E63DKOkrcMRtqxE9EngHh/bgpcXfA1o0vG5v pglQ== X-Gm-Message-State: AOJu0YxVqB4JDsuCtQ2/3o7Fdo3g644Qd9pkqwiz5fjHDmMThY49K/qD tzqBJxns5/8cGi3Vk1bPaGe4DgrKQZ3Y5j1TIYiDbSpS3ZCtQb2yOs1ZRWKR3TGrj+YRK6VPfWc VIW6tycpfTKs7oMACA/w7TLL41GLRRjMrHF1qheSnmbHW9RhhVrFM9FClJixYELrWSCv4BAjO7G tCFXAlB+znapa3taZWRGnOdpK+BZPsz3eiR8SmKtsva3g= X-Gm-Gg: Acq92OEu69/CfmAmnOTohbQJSURlYtXEI8XygCu3+fzmEC8vhPijp8pwFwWhi7qkkFF 6cqdmawnVuNNU6Vu8b//j3qGF/pWwoSEY8iV/RtyiGg3y62Aa/KEmTJYuHFNyFI9Cw4f/ukuKa9 tsOEedhD/TCjcyYn6hvg0/kSBSGyHelECySw1uwkGwPtxYLzYChvs7wfPDZyuZd18nhGWF6oByS mkDfbwmjb8sxQT7hpsiLTtJmKFcQgcEm0UTmVniBFvJjq8PFeSgPPsGZzl+h2G9hejifqWaoTZy Svy7Soy6woHHRDasRDEV2+OE/H0TfK9LiNAKONVSbuxWMxQy96pF5STjK8XN73wSJztRBTcdoz/ M3DUr5rDLhJjb8slNh10kQIVw0SxN/Fqn05u5uHw0hlv/i1lXpv9GaA== X-Received: by 2002:a05:6000:1603:b0:45e:82b7:c260 with SMTP id ffacd0b85a97d-45ef6af7a60mr366103f8f.2.1780068184378; Fri, 29 May 2026 08:23:04 -0700 (PDT) X-Received: by 2002:a05:6000:1603:b0:45e:82b7:c260 with SMTP id ffacd0b85a97d-45ef6af7a60mr365980f8f.2.1780068183483; Fri, 29 May 2026 08:23:03 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45ef354c682sm3952557f8f.23.2026.05.29.08.22.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:23:02 -0700 (PDT) Date: Fri, 29 May 2026 11:22:58 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 11/37] mm: page_alloc: move prep_compound_page before post_alloc_hook Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move prep_compound_page() before post_alloc_hook() in prep_new_page(). The next patch adds a folio_zero_user() call to post_alloc_hook(), which uses folio_nr_pages() to determine how many pages to zero. Without compound metadata set up first, folio_nr_pages() returns 1 for higher-order allocations, so only the first page would be zeroed. All other operations in post_alloc_hook() (arch_alloc_page, KASAN, debug, page owner, etc.) use raw page pointers with explicit order counts and are unaffected by this reordering. Also reorder compaction_alloc_noprof() for consistency. Compaction currently passes USER_ADDR_NONE so folio_zero_user() is not called there, but keeping the same ordering avoids a future tripping hazard. Reviewed-by: Gregory Price Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/compaction.c | 4 ++-- mm/page_alloc.c | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index 72684fe81e83..4336e433c99b 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -1849,10 +1849,10 @@ static struct folio *compaction_alloc_noprof(struct= folio *src, unsigned long da set_page_private(&freepage[size], start_order); } dst =3D (struct folio *)freepage; - post_alloc_hook(&dst->page, order, __GFP_MOVABLE, USER_ADDR_NONE); - set_page_refcounted(&dst->page); if (order) prep_compound_page(&dst->page, order); + post_alloc_hook(&dst->page, order, __GFP_MOVABLE, USER_ADDR_NONE); + set_page_refcounted(&dst->page); cc->nr_freepages -=3D 1 << order; cc->nr_migratepages -=3D 1 << order; return page_rmappable_folio(&dst->page); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index b725a3b59835..3108ac9061ce 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1877,11 +1877,11 @@ static void prep_new_page(struct page *page, unsign= ed int order, gfp_t gfp_flags unsigned int alloc_flags, unsigned long user_addr) { - post_alloc_hook(page, order, gfp_flags, user_addr); - if (order && (gfp_flags & __GFP_COMP)) prep_compound_page(page, order); =20 + post_alloc_hook(page, order, gfp_flags, user_addr); + /* * page is set pfmemalloc when ALLOC_NO_WATERMARKS was necessary to * allocate the page. The expectation is that the caller is taking --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E6BA93FE640 for ; Fri, 29 May 2026 15:23:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068195; cv=none; b=mOhi7BI4i8Y1QVBdmbfQPQG5WScIJ8XLN8ndiJZUgLZqgIpRO22dIsXU6LWF6gXroNviV8ROnO3c4PuLiXeCaiqm5fy90Bk4RGxRwn37CdAxxvlu2sF98ex+LyMXnl+8MXtX16p63pkJ1j4TkTNkXt0MrU9WhHVTMpiC3cLWpzk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068195; c=relaxed/simple; bh=Y8nJwk5bnDwHUaiRuu38UiZrbZgOyq6/yOrVYrwIdY8=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=L7puezxNHt3HgmscG5Snifs+io71m67WlLV5D9eVXyh3cM5fgaw5KUSuwNUjtrq6ggKpZEvfxKizFuZtZ+hHe7ntobP4PuCZTCeLbfMMbNqCBmjKRjsXrhHDFpC7HseO1+IFA/xPMk1yv+pP1yxNIBgIzueDRzA1BWq70NLtBVI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=APirCoaT; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=lvbi7PUV; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="APirCoaT"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="lvbi7PUV" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068193; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Xr0Wq/tgum3c1wWvimcTvl9O1enHA5UssViDpRUdduM=; b=APirCoaTRGpugczVmqZjXm8eOmvDF4MOK0vXhs0CisI6HivuXVCo2F1q5DzF7KhAWTmjyO vMGknFOTkbE8mK+2266t8N6F4OhWA2UShpBLpSOh9pObd2hU/rrser+qtva+RLUxh/QAar ObddwoeUqRRP82SKidiGijPOOOA1xaU= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-594-iLHg0es-M0uOTEUoYTUOBA-1; Fri, 29 May 2026 11:23:11 -0400 X-MC-Unique: iLHg0es-M0uOTEUoYTUOBA-1 X-Mimecast-MFC-AGG-ID: iLHg0es-M0uOTEUoYTUOBA_1780068189 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-44d9ace59efso8073174f8f.1 for ; Fri, 29 May 2026 08:23:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068189; x=1780672989; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Xr0Wq/tgum3c1wWvimcTvl9O1enHA5UssViDpRUdduM=; b=lvbi7PUV4shPHLtGCiVIvyLieeUaa4cR3/c03ruMENZOwg+4n7PlJAEyGgwcyWBNSS QYvpnuR7cfHiks+JAGtgI1vm7dzlS6b1BAEu29MlrMw6CvEdSPGQcOkCMsIWsrvIcxt6 0BqviX90e3L0spLfecpIpfcwo0R29QQ8KB3LnTzAEXR8G1YzQDTSOdAPKE1oSTdS9MOe rEREeuAY3WF6+6+qswOP3mnV7341ydMKsfvIID87ROWZIPil4BjSm6CMVZeJ2xAEMDxd LIKGd0C8f9/VKDdEFuY/rCZ/xpC0Edz/9rZ1FHyARJxv9uUyFHJapOcIw7MiPo8G0/x1 exng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068189; x=1780672989; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Xr0Wq/tgum3c1wWvimcTvl9O1enHA5UssViDpRUdduM=; b=j68dAX+V5qBdIMANvTFddE6EGG1IYcgx1KwYMM3Qe2Ri9D6xhlpo/4zB2xKPOEKCQS XeFaxgjy87orpBRHaFbBkrX+ASYAimXUsmx5nj8uQ40kOULpVD3BNXDtYvx4GHZyWdJi k6V0mhJsyO9YuewPIPaDdjMFDe4sG5C7RgpogCH0pS6cPZ9H/4CtgInqYxB6qHp8ZmDw taidBqpIGW+Yll2RID0mzhi3p6SwP3LQCLmU7ij5ps/RSw/BIMNi95YeyWiwvpixqRJX efkzXYft7/OL9og2aknNxpuykP7qIea9db/aSJmvCg+ylkuBq8EMIhDSonBUQCrpVlgK C7Pw== X-Gm-Message-State: AOJu0YzsvoOGMGLU5z72x7vUsIJ1jMCL6A/M7fZZR0HnNwFS+XE0L6oI VtwzF+ATP1AD/rcAUVCh1CY4X+1yyVri0eIkl2dU3RYsWc4Owh4S4Otvql1XHX3gL04pQeDg8F5 MIeHHVbaxlnS1xPfV18RljfKChrDXIlhbSTGB43Rw4YHsMeTboNy6KNZuGC+Ot0FBnNKbBhnPr4 M7FEUa2us9DowBfX75E3/4TGLKhxKulHUKh4qWecqpXEA= X-Gm-Gg: Acq92OH4VKdHsgNU4i2YE/R3EbTllcjsv1pZUvQqSjDUNEwfvD6dQxL9b4NoD+cyY5c MkAkVEg8iHb1J/stj3Inwm1qUHCJxIfEdL33XaOx4HdkrqMB1s04LHdwP3cnm9Vs1AsNgj48nQU +Rc2t5ZSdWD8/zEvRVhpldTuBlHaXLZeiU1nF5g9Pgv5Fe5VoaSqJAa7HCo1lABcGcVRXZ7FA/V LqZqH7KkUDZvT60n3FbZbh2rmhJ9oehEl1/NuPyt6iKfGzq3lj0Ad5X2c0eRIIwWNylGZuFTQxk TVGJnYdO9Zi6GHhDoVF9gbvVMwFmFwn0FJPbNPnMtLmubyvFupzd5VeI/36r602vuy4zwPtMMws zWVUdnKm4i+kTTX07OQyIseYKwiC++IUdGWZ0VQ9UQurowhY2mqmPVQ== X-Received: by 2002:a05:6000:41f9:b0:45e:93d6:359b with SMTP id ffacd0b85a97d-45ef6b2254amr513242f8f.9.1780068189059; Fri, 29 May 2026 08:23:09 -0700 (PDT) X-Received: by 2002:a05:6000:41f9:b0:45e:93d6:359b with SMTP id ffacd0b85a97d-45ef6b2254amr513116f8f.9.1780068188420; Fri, 29 May 2026 08:23:08 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45ef35874b7sm4319354f8f.35.2026.05.29.08.23.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:23:07 -0700 (PDT) Date: Fri, 29 May 2026 11:23:03 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 12/37] mm: use folio_zero_user for user pages in post_alloc_hook Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When post_alloc_hook() needs to zero a page for an explicit __GFP_ZERO allocation for a user page (user_addr is set), use folio_zero_us= er() instead of kernel_init_pages(). This zeros near the faulting address last, keeping those cachelines hot for the impending user access. folio_zero_user() is only used for explicit __GFP_ZERO, not for init_on_alloc. On architectures with virtually-indexed caches (e.g., ARM), clear_user_highpage() performs per-line cache operations; using it for init_on_alloc would add overhead that kernel_init_pages() avoids (the page fault path flushes the cache at PTE installation time regardless). No functional change yet: current callers do not pass __GFP_ZERO for user pages (they zero at the callsite instead). Subsequent patches will convert them. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/page_alloc.c | 35 ++++++++++++++++++++++++++++++++--- 1 file changed, 32 insertions(+), 3 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 3108ac9061ce..ad0655387e9d 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1864,9 +1864,38 @@ inline void post_alloc_hook(struct page *page, unsig= ned int order, for (i =3D 0; i !=3D 1 << order; ++i) page_kasan_tag_reset(page + i); } - /* If memory is still not initialized, initialize it now. */ - if (init) - kernel_init_pages(page, 1 << order); + /* + * On architectures with cache aliasing, pages zeroed via the + * kernel direct map (e.g. init_on_free) must be re-zeroed + * through a user-congruent mapping. Host-zeroed pages + * (zeroed flag) don't need this: physical RAM is clean. + */ + if (!init && (gfp_flags & __GFP_ZERO) && + user_addr !=3D USER_ADDR_NONE && + user_alloc_needs_zeroing()) + init =3D true; + /* + * If memory is still not initialized, initialize it now. + * When __GFP_ZERO was explicitly requested and user_addr is set, + * use folio_zero_user() which zeros near the faulting address + * last, keeping those cachelines hot. For init_on_alloc, use + * kernel_init_pages() to avoid unnecessary cache flush overhead + * on architectures with virtually-indexed caches. + */ + if (init) { + if ((gfp_flags & __GFP_ZERO) && user_addr !=3D USER_ADDR_NONE) { + /* + * folio_zero_user relies on folio_nr_pages which + * requires __GFP_COMP for order > 0. All user folio + * allocations set __GFP_COMP via __folio_alloc. + * user_addr !=3D USER_ADDR_NONE implies sleepable + * context (user page fault). + */ + VM_WARN_ON_ONCE(order && !(gfp_flags & __GFP_COMP)); + folio_zero_user(page_folio(page), user_addr); + } else + kernel_init_pages(page, 1 << order); + } =20 set_page_owner(page, order, gfp_flags); page_table_check_alloc(page, order); --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A399F400DEB for ; Fri, 29 May 2026 15:23:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068198; cv=none; b=dgBhwCZAlxBXLVkA/FSlmXHBb8NRCxA0i35tCzlBAXMdCSiywvk0PoUKFc7uPmAQwpfv5Rz978JxGvTFAf6jkUa6f3pA3TAuTe9vcy2Kh2fw64uooOXdTfWQC/yM1kMHhRSrxm2DyWY8FvVpjiuxoyZACcq0wrWD5/AZKrfRVUg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068198; c=relaxed/simple; bh=B4v7SD6SJ84KapLgBy+iXreANiiaF75HE81bu1RBt3E=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=ihhIk47co3XroM8m4UH3g96u0TOI+GTuQPU0iMcf5gYgd6niJo0cSQ2yopLiguMAN8JvZUxc69xYO1UqpFZmMdJ6yrUyr6/TtXeigkM02+uq8A4YOp6BYXNKMCLSqS9eB+34At30DbkRJf1Nqrp7AhFCp9LYS3iqQ414eVGxLGQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=YTyJCtn4; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=sOIw0zgZ; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="YTyJCtn4"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="sOIw0zgZ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068196; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=jpAQ9O1oMgUNlo5ukg9OEno1SM1k+70VK2/jH7/8vVM=; b=YTyJCtn4mPExPTO0zgeD2h79mhz9HnfK/oDGkYhb2NG3n7WFbpnADmS1vxNy2sN2HJIt8r hB2nSOdfwHSH7VU22xiqOQw4THcvgm7In2txs+MZoiixB2OABT3hOaz/dNV3E3wgWUTnuU P+gyHHvbMYM01VvP4UXt0YbrdwV0J5I= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-253-RZssYalbNjmcXyK722TZgw-1; Fri, 29 May 2026 11:23:15 -0400 X-MC-Unique: RZssYalbNjmcXyK722TZgw-1 X-Mimecast-MFC-AGG-ID: RZssYalbNjmcXyK722TZgw_1780068194 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-490479c2911so83513325e9.2 for ; Fri, 29 May 2026 08:23:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068194; x=1780672994; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=jpAQ9O1oMgUNlo5ukg9OEno1SM1k+70VK2/jH7/8vVM=; b=sOIw0zgZorjzAnpuutVytEsjU9YDIkHJVyUDKcj/FEn3ZWcvSDwtAyFj2+6+C9VuXf HLwxgbg40Uk1/7UDA35qXiisAGhG7SOHmeVsfNKNIg/yq9h5G8LkQKAxLElZf9STR+Zi 7nem3+vn5yNA20zanrgqh6SDlVu3JhuoOXKDj8aCZCCCAHBEfZo2aHuSsu/vwQuqk0IK li0Zt/0Rac68Gj8uszCx1t8BNp5T0kDY7vxBfFQXoW79ZhxxUKjLbCYsYmcI1wyT3AAn e3/dIuvpxyi1FA8h6QlCHVaMy7fhLCgvYdsB6H5GlgSjIGzqP/ra0b4lW7TifPEBzmtu fvIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068194; x=1780672994; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jpAQ9O1oMgUNlo5ukg9OEno1SM1k+70VK2/jH7/8vVM=; b=YISmcuMuJyAe28d3ILVoNkikGKoDaELnXG+ULgVyVJssh+rkEWA7baU+LphB0P9BlZ kc1k7s98GeZiZJmaspNfQ3edOz3ZOdDzIxCABIo2rOodFK9BYWUuT9jx/ToTwQIobEwy YpcYgjNbKa9Js8JSXmmt22BsO0L1kmNHAH6tMLQEHk16LvEZOXDgWqwhk9IheEHUgfFh wHkFMrWwslDK6m3W/odsQ+LQsmAaukSZ9gkjuYJi5Vx5YXAitUOOZk9w2Mm0I/4BeoxL 0M2sQv+MecZOltRJGpRGV/wjSbg0jyrU0hYYkUjl2nmtD96zDzLZfLWUp2HkkB5RHBb5 GNCQ== X-Gm-Message-State: AOJu0YwjG2qRIFVOu2u0BK211D4lPTJv1//kYY5AYXNgK0fPbEAwYPLH 7zvUjBB3O8GQgDOwhr/aD6ND/sYt+VQVonpjD3oGdaQvVmIKEf2jxSzqNVNEKEfJjwagw7zXZ1/ 1dFKPLlowqB/h3Hi8+frDYP6b1ar1Xqs537bQF+nGcXfeUD2c8polfMHL9DMJU6qfgLCHcaq7gW niP60w1wZdzvJyZCjNPuIIe62DEOhSrKAF4J/ivX4qyAg= X-Gm-Gg: Acq92OEPUoiJMs59QvoMI052vRmkUb/C/CHW8LmYuAmmwzUmrNfbK6gvACiud7MQtnT Y585b8yB+0PIF7W8dHzIadEk8iUfjE3g8eMDwHkiADbCvkn/IUBOEjLD68M3jnMCn3Y1FKywtcT uhnb4L3JbdeP8vtpe0BmhFFuEe8jMI85RCmlJi7KdWlqSMlNzgNO2C9poXtgvEMBIi7VZ9l6krb CZ3vg0CV5uW6fgTHAx951ZCdSIy1OPcKlFjVwYj9d+1RfGNHYXbH0kd/rzXUZKHWo2fzSg3uEMU +E+RpyX8O3TAHDVr3BdmtK+z7HBnYncVLQWEDGF0Y4SaaF0sPN76SLbE9wxCZfk8R63esCMMKG3 BtXRlOlTpw3awgNFI6HPGFJy781aXJu129qA3MZaUiylxa7ZWb6n0zw== X-Received: by 2002:a05:600c:3587:b0:490:958f:2a5e with SMTP id 5b1f17b1804b1-490a291d967mr2192105e9.13.1780068194131; Fri, 29 May 2026 08:23:14 -0700 (PDT) X-Received: by 2002:a05:600c:3587:b0:490:958f:2a5e with SMTP id 5b1f17b1804b1-490a291d967mr2190915e9.13.1780068193465; Fri, 29 May 2026 08:23:13 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45ef35601d7sm5881021f8f.28.2026.05.29.08.23.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:23:13 -0700 (PDT) Date: Fri, 29 May 2026 11:23:08 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 13/37] mm: use __GFP_ZERO in vma_alloc_zeroed_movable_folio Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that post_alloc_hook() handles cache-friendly user page zeroing via folio_zero_user(), convert vma_alloc_zeroed_movable_folio() to pass __GFP_ZERO instead of zeroing at the callsite. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/highmem.h | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/include/linux/highmem.h b/include/linux/highmem.h index af03db851a1d..ffa683f64f1d 100644 --- a/include/linux/highmem.h +++ b/include/linux/highmem.h @@ -320,13 +320,8 @@ static inline struct folio *vma_alloc_zeroed_movable_folio(struct vm_area_struct *vma, unsigned long vaddr) { - struct folio *folio; - - folio =3D vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, vma, vaddr); - if (folio && user_alloc_needs_zeroing()) - clear_user_highpage(&folio->page, vaddr); - - return folio; + return vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, + 0, vma, vaddr); } #endif =20 --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1EB14405C33 for ; Fri, 29 May 2026 15:23:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068206; cv=none; b=FddooqwYOj1UcSjF/CihuF+ltvjWw5gzsGjeJnqjgotRyUhnLNzZi6m6H5jmFjjU7x6QJNN3XdEunj264TLJXPntQQxpHH37bSKV96HVTO80UXCyadeUJxKfN2i60R3nLFL3WaS/OJd02M6L+DLS2irsZENHefFn7qhjZj3iZv8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068206; c=relaxed/simple; bh=+lLqKB25GJ8Nk/N/QA5/TP30kbyhPVpqAySiqciCa4w=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Hn6qXO4XyLJIRA6c3ccMI80ti2Cxb/apViDbdArmJ1c1KYOsor5N7npuC16MA8y4zCnq0fwqLYLomlWaHoj6dj7i59gMK7r0FWm5AnntFXTTdjtZqxpnANHzt2SFQMCyoq/wzWXaxnqO+Dhaky1pAtF0irVQvo6ki+aHCG80krE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=dK1aQaXX; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=OlpWIUV4; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="dK1aQaXX"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="OlpWIUV4" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068204; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=yTMC7Vi3qz2eQrpu1XXxCoOuK0TKvwPfinMf53HH2Ps=; b=dK1aQaXXY0JHN4RWNfdQ7XY4BkKGITOeYoyRCWADUc9ezTd7k69HCpRBziJ2yQutO8zFKb tnYq9L+hrAU3tbOEl6KlaHlku4L7uP41oAkZ/7lQdM0mDoiQdHO7a1nGc+50+KQaO3/lGe 9JTjgVfOS6zLKKaB7htJRbukbu9eJiA= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-630-tP0DxZgcO1yPNWfKyMliyg-1; Fri, 29 May 2026 11:23:22 -0400 X-MC-Unique: tP0DxZgcO1yPNWfKyMliyg-1 X-Mimecast-MFC-AGG-ID: tP0DxZgcO1yPNWfKyMliyg_1780068201 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-48eb0da933fso93822945e9.0 for ; Fri, 29 May 2026 08:23:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068201; x=1780673001; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=yTMC7Vi3qz2eQrpu1XXxCoOuK0TKvwPfinMf53HH2Ps=; b=OlpWIUV4FBBPwNsZ8qu2qqvQdMJThXFnzKBbUjl2YYo2oIEEYHnOsaLdXgAAlPqVR4 GKS75CfNj+cenXZL+x15szsXRcDIkwX3LercyhdytT+R62VEEqutvamNuhemslzo5Lh2 h/u0ahzxw2bZD32vXs+uXuHm5bM3aVXHZskesPM4ztyaMTnYa7Llx4EE/9dj5GVfd+w6 ovL166BG6fZThuKkTxi8SUZ4vF+CitxBveclW779oNWgKt3SU90rwfd1f7ujkVSypGyX BfC3XDvcViZlt8IuXtrSIICY6LMTf1wQxDIpp5B3i9FKutEZc4JmF2dp/qEYv5tCq03X g5Ow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068201; x=1780673001; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yTMC7Vi3qz2eQrpu1XXxCoOuK0TKvwPfinMf53HH2Ps=; b=s7NGJDGRLtPw5Ax1PzEp8JcJaZq/SfIJ4udUBhLlu3IUsQ/mYZgORlsO2Y9lRFMQRv IjsCyKkNqhQTPhr7sqoydJUgy6Beayzd+ev7eLVV6HkpRHIA+6dPn0tPJTwVoPe7vqH2 /4NoHyGeXXQau0QBgUipkp3hnXBfdQ8tBaIjnRuzM0CoSHjZT1raccg9T7bEQrCbLj72 bqUmfdwogtRLMpcpTcWKPzqNGO+Kai8FJshAnt+gyUuFW/wqp0j8b27NkBhSVx2Rr9jK ld2JhDuFdRGp6Hel1rXlvylgHjsfkDbYsHvs539IHl+m/vK6m9P+qLj0ykwTuIxUAhHF C+4w== X-Gm-Message-State: AOJu0YwkbLGZ62oqGhicex/IBkdWeAGVy1YHhiGbJ1KQueRoYkjuSz45 GSoihkGJq1GP1zWWIPmeFFMYocbFEs1ohqM1bUl1ZUM9LpZjopmkOnnNRMEcXACnJCgorz1Tc+z 7m9PcxGCTPgqctK/wwTlV7VK1FtMZ4L0UxZTwpZTOmhVbYcJKUdR6GBz/o388khXZVuUsIPsWD7 NYqTjZYMoncVPQQuCiQDm5rwMdZoWTIHNEVlZFxTc1LR4= X-Gm-Gg: Acq92OF5xwMlD2GENsCuY0vSmFn5j+IluJmxX35P4hrEucqteKvXJE7DGlBREuBjhz5 DvVQmqZJbOGNbkUD2AO0me0E2NJzN3uKtK8Y6vESYPsOMGSNB9TknutyxeLe70zlNXkuO6sJ8oV WxWz2vmbLEnE+RtC7gBVEBLP9z8o0XczTrCyDD9Oer8jvWpdAE1LPrNfjxsXJEHHLwYvjPNW8UZ +7RuCGQrBx9MXDxxabZHzsuv1ohz0oxx7+U9C+IWxCqAZ4y8N7nzm4VzyjYefjHNL54DURi1GF+ Jk3hKeMFOHzVVpOBzUkVuFMTTs9DdkfwYftLHYr7hVzW88jvN/6mX+2nj55eIFf4+q6pjKuj/e5 7Hg8rYHaiDpuUooVY75ewxL3yOlyUoh2iLsIQAsvwM6TMyY1QeNNqCQ== X-Received: by 2002:a05:600c:4fc4:b0:490:6889:202 with SMTP id 5b1f17b1804b1-490a29639e6mr548255e9.29.1780068200978; Fri, 29 May 2026 08:23:20 -0700 (PDT) X-Received: by 2002:a05:600c:4fc4:b0:490:6889:202 with SMTP id 5b1f17b1804b1-490a29639e6mr546645e9.29.1780068200009; Fri, 29 May 2026 08:23:20 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4909d6f35f8sm40956085e9.13.2026.05.29.08.23.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:23:19 -0700 (PDT) Date: Fri, 29 May 2026 11:23:13 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli , Magnus Lindholm , Greg Ungerer , Geert Uytterhoeven , Richard Henderson , Matt Turner , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , linux-alpha@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-s390@vger.kernel.org Subject: [PATCH v9 14/37] mm: remove arch vma_alloc_zeroed_movable_folio overrides Message-ID: <17905f527ee63479bd406efc228acf481a54b2d0.1780067977.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that the generic vma_alloc_zeroed_movable_folio() uses __GFP_ZERO, the arch-specific macros on alpha, m68k, s390, and x86 that did the same thing are redundant. Remove them. arm64 is not affected: it has a real function override that handles MTE tag zeroing, not just __GFP_ZERO. Suggested-by: David Hildenbrand Acked-by: Magnus Lindholm Acked-by: Greg Ungerer Acked-by: Geert Uytterhoeven # m68k Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Reviewed-by: Gregory Price --- arch/alpha/include/asm/page.h | 3 --- arch/m68k/include/asm/page_no.h | 3 --- arch/s390/include/asm/page.h | 3 --- arch/x86/include/asm/page.h | 3 --- include/linux/highmem.h | 8 +++++--- 5 files changed, 5 insertions(+), 15 deletions(-) diff --git a/arch/alpha/include/asm/page.h b/arch/alpha/include/asm/page.h index 59d01f9b77f6..4327029cd660 100644 --- a/arch/alpha/include/asm/page.h +++ b/arch/alpha/include/asm/page.h @@ -12,9 +12,6 @@ =20 extern void clear_page(void *page); =20 -#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ - vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr) - extern void copy_page(void * _to, void * _from); #define copy_user_page(to, from, vaddr, pg) copy_page(to, from) =20 diff --git a/arch/m68k/include/asm/page_no.h b/arch/m68k/include/asm/page_n= o.h index d2532bc407ef..f511b763a235 100644 --- a/arch/m68k/include/asm/page_no.h +++ b/arch/m68k/include/asm/page_no.h @@ -12,9 +12,6 @@ extern unsigned long memory_end; =20 #define copy_user_page(to, from, vaddr, pg) copy_page(to, from) =20 -#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ - vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr) - #define __pa(vaddr) ((unsigned long)(vaddr)) #define __va(paddr) ((void *)((unsigned long)(paddr))) =20 diff --git a/arch/s390/include/asm/page.h b/arch/s390/include/asm/page.h index 56da819a79e6..e995d2a413f9 100644 --- a/arch/s390/include/asm/page.h +++ b/arch/s390/include/asm/page.h @@ -67,9 +67,6 @@ static inline void copy_page(void *to, void *from) =20 #define copy_user_page(to, from, vaddr, pg) copy_page(to, from) =20 -#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ - vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr) - #ifdef CONFIG_STRICT_MM_TYPECHECKS #define STRICT_MM_TYPECHECKS #endif diff --git a/arch/x86/include/asm/page.h b/arch/x86/include/asm/page.h index 416dc88e35c1..92fa975b46f3 100644 --- a/arch/x86/include/asm/page.h +++ b/arch/x86/include/asm/page.h @@ -28,9 +28,6 @@ static inline void copy_user_page(void *to, void *from, u= nsigned long vaddr, copy_page(to, from); } =20 -#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ - vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr) - #ifndef __pa #define __pa(x) __phys_addr((unsigned long)(x)) #endif diff --git a/include/linux/highmem.h b/include/linux/highmem.h index ffa683f64f1d..7b5955bf9336 100644 --- a/include/linux/highmem.h +++ b/include/linux/highmem.h @@ -303,7 +303,6 @@ static inline void clear_user_highpages(struct page *pa= ge, unsigned long vaddr, #endif } =20 -#ifndef vma_alloc_zeroed_movable_folio /** * vma_alloc_zeroed_movable_folio - Allocate a zeroed page for a VMA. * @vma: The VMA the page is to be allocated for. @@ -317,12 +316,15 @@ static inline void clear_user_highpages(struct page *= page, unsigned long vaddr, * we are out of memory. */ static inline -struct folio *vma_alloc_zeroed_movable_folio(struct vm_area_struct *vma, +struct folio *vma_alloc_zeroed_movable_folio_noprof(struct vm_area_struct = *vma, unsigned long vaddr) { - return vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, + return vma_alloc_folio_noprof(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr); } +#ifndef vma_alloc_zeroed_movable_folio +#define vma_alloc_zeroed_movable_folio(...) \ + alloc_hooks(vma_alloc_zeroed_movable_folio_noprof(__VA_ARGS__)) #endif =20 static inline void clear_highpage(struct page *page) --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 931DE40910D for ; Fri, 29 May 2026 15:23:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068211; cv=none; b=B1+RJTSli72rjM6vizTrxLJYZk/2pnkCDL/G+6ww5AUAGDoGegfvelslPcmkAGxGdDlC17Ma6yAcF5NuOyaHLQGWYlQnAMqnaeQ+Qq3Q6k1Tvl9My0sDmvHwx+LtIarW/uIQai//IENvndU7ezaklMpqB8+Mne+b9TwO+0uVA0M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068211; c=relaxed/simple; bh=iDiOdN3wLJj9wOdkZsdqwIAEdg+4DXjaS99pBUtFYbk=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Qa9yTgUD5oFFSnKWK6iBEyb3KQj5lvMXbSSUAFrLEkN5uVhhC/Bex4sJ5RuJsQFzw7dA0FggURyN6GsaPBO3l0R0H6n2g1sDtrH/uji9gji4vQ91PA/kqZzkFiikxaa2EX4ZDEoUjHmSHe/qw9RCH9euU8m5eHDJIs90699CdyU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=O9cNzULJ; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=WJikXfpI; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="O9cNzULJ"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="WJikXfpI" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068208; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=vK+J6BapUtjwGwdYlifbIYhLZtsZgN3kLB9idS28qTQ=; b=O9cNzULJXKPlx8BwUmumFFGc/Ryk7Af+j6dVRq7yZhpIK06v3o4gssRZu7Bqm0Q13Uxux6 9nhkk2l0bT7gZoIs6rTfAU9SyD5va63UhczcrPA8FbD+i023IpcnZaG4blvEtqNQV/WcoW tfNehoSszV7A4nQZpPEPAemNGAb6WGM= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-304-PK3WEH_7PeCBBDJ2NOtW2w-1; Fri, 29 May 2026 11:23:27 -0400 X-MC-Unique: PK3WEH_7PeCBBDJ2NOtW2w-1 X-Mimecast-MFC-AGG-ID: PK3WEH_7PeCBBDJ2NOtW2w_1780068206 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-45ea38c03a1so7513337f8f.1 for ; Fri, 29 May 2026 08:23:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068206; x=1780673006; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=vK+J6BapUtjwGwdYlifbIYhLZtsZgN3kLB9idS28qTQ=; b=WJikXfpIdLr5tTzyAD83aNcH2pbW8v7UwZhliQDE2a3L5HEs2eSsegVEcg1/xRds9u 0VBrYrmkNpusW/IHrGTuOikpHsBN2PFlxULfNJWwO7xxRsIsaFghgJba3H1uIIZ608X4 fmZBJY4hRj1HbmnuRFvkalEFxPew5SxTsLZWHQnuu8E8IiIJMunjC2FOw/L7psUxEqpf QVgb9gV3wjVHZ0+dzqGTSeZsRhP/Ek4lnKg4hNglB9wIsJSQalhu7DXzYcuJhx/1r+Rp k1HMwJvJV72bJqyXUGQTgfjkwoY77rmXjfsrZYiJHKNPn5pngleKybRiTod02Qu5jNq1 nN6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068206; x=1780673006; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vK+J6BapUtjwGwdYlifbIYhLZtsZgN3kLB9idS28qTQ=; b=ZSt5tmGjqawfyAG93HzyrjI/Zk8bIsu4K1oKX2AxYTyJSQoZmctALR1h8IrOZf7tCk huaKlFGKKlvC0DD7FugD3g0MCNv6Qizboul1f7yYmAprHUzwqPZLyTrkoR1iQjeKnVzQ iiENZp8cZZxRsA8Joz7cbJR2nj4ABBGD2BzjTcLbV9UIdF6IZnxCQcRIufBlj3967HsO Mpse6jEnztXSbBUahNBUXSP6mGfK9o9C+C+YgDLDZu5HZvQRVcs2DX/jesRhMRJaKbT/ vzL7eKFNROibTBsMvlhni1WiaERwFXkd6GO2Sn8mtCL1B2+1k9HfB9zEsoU3THoVvC6K UXOw== X-Gm-Message-State: AOJu0YwEy1HFtohywDxgt0cihaly5AfriPzIxZDsBhAoaV+WMvu5kvS/ hpUlo2PtLqngt1WQKH2o9lY5Bn1QPoBKl/ytdno7A6F/DIe8peATQ5VFdNAP76zbx8RKtFVQ2TD lMmJFOCiLRKLICQI4ImbdR1iC3NOdcA+fbMa0fXkrfva8ilM2UEWJ864xUM/XWHpVshADDJo9vo FEvu6hGsm5qLYG02jdMUc5ZmTDf4xeGZNk6bELLW0nzyI= X-Gm-Gg: Acq92OFQDV5tT3ceIKzMwrmMilukHvV+BIXW+NMCUsTuGie9PepS6i5GheYf9Zng71n 2gZ2TcMPc3K3vhTA6nM1NryJqaT9Nbwalkmj3hc9oIh4WYKV6vJ4XQY3oX4zCoPJZNRtxRbbQ7f oMEk2fD3RSRDDSxG/EXqF0Wj5EABhR3JxlKg4iNYgAXtMz+bBpWcwCTAiLabiy3gf4sRb/i9Msp 7E/lAr7VjeJ/m4PkZ9KZc2LzCx833ZhYnL5WCMkZrj+/SRZSnzagO4+XCDgml93OO6b7QxkPlY7 JtjdYHHQndQ5HB29jpEML79/I8cdJ8iRttBoHjzcx153xaxsDOZtUB+BtRIE3QaAtec0hvP9+fq PcIx9YNpNxXe49X0wnYZU/JEgg9XlgonCn6RdKS7mcLrg0SnQ083fug== X-Received: by 2002:a05:600c:1592:b0:490:58e0:f466 with SMTP id 5b1f17b1804b1-490a2977dd7mr372105e9.33.1780068205694; Fri, 29 May 2026 08:23:25 -0700 (PDT) X-Received: by 2002:a05:600c:1592:b0:490:58e0:f466 with SMTP id 5b1f17b1804b1-490a2977dd7mr370945e9.33.1780068205067; Fri, 29 May 2026 08:23:25 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45ef34b47eesm4815815f8f.9.2026.05.29.08.23.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:23:24 -0700 (PDT) Date: Fri, 29 May 2026 11:23:20 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 15/37] mm: alloc_anon_folio: pass raw fault address to vma_alloc_folio Message-ID: <552604621ff4b05b9ebb69ca75abb83ac8c0912e.1780067977.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Pass vmf->address directly instead of ALIGN_DOWN(vmf->address, ...). vma_alloc_folio_noprof now aligns internally for NUMA interleave, and post_alloc_hook will use the raw address for cache-friendly zeroing via folio_zero_user(). Signed-off-by: Michael S. Tsirkin Reviewed-by: Gregory Price Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/memory.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 02d3e53fc91b..75bee9501666 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5252,8 +5252,7 @@ static struct folio *alloc_anon_folio(struct vm_fault= *vmf) /* Try allocating the highest of the remaining orders. */ gfp =3D vma_thp_gfp_mask(vma); while (orders) { - addr =3D ALIGN_DOWN(vmf->address, PAGE_SIZE << order); - folio =3D vma_alloc_folio(gfp, order, vma, addr); + folio =3D vma_alloc_folio(gfp, order, vma, vmf->address); if (folio) { if (mem_cgroup_charge(folio, vma->vm_mm, gfp)) { count_mthp_stat(order, MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE); --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0F9AB41B369 for ; Fri, 29 May 2026 15:23:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068215; cv=none; b=Rb5i9xQMxxpeZFmSplhkYGmUp5F6JpLTQ9oZVDYwQdgK764Mm2qVcJ9n8VwR8g60bnijrevHNpTy8Lwi2IwIZyDFXkgGDgAPwvcgp/1ZL2NiYBB9GwgbZiJ2SxCfT8D1IiwsEm6Ky8tdAbFWZWbmSUMOuOBao+kfK2jGxsW1Z9I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068215; c=relaxed/simple; bh=NCrpBEyb9BtjGhGhnTsHPNPFOIuuUXGjOyr7FM5hCbE=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=NSCEIut7Y1kX18xkKCKXUS6svpsPls2XMMpdgSVBofcm0D4+bksTziseAzkcbvMG8lugFbpJrsvM+yy1J12x01dLi6l2BCcFx1f1t1s9d+w94xGvT2q9xrtNX8/PAGDn9O/uXU8wcw3cHDnOpK0q8AVnKCYJeYfo3o6R+H5l81I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Z6hsLO3B; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=MxEyEPzu; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Z6hsLO3B"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="MxEyEPzu" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068213; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=YBx6SgN438Kse3DYiPhfWEcJAMkqbJwsQPerWAK4QDs=; b=Z6hsLO3BMe8klH2VVDjVJXVVnee73LsklJ2YPGaMPx7LZjO+oVOV9QchRHM9NoIZZO2tm0 6pfRslmQzACOIWUUEyBTP6XOQg3dsedstUGSlmizy9N+5vTv1V/JssFYxR08sQ7ITj/6lZ vnqWBwDjtqvpjra26wa1L7NIRn+NG5E= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-411-SZ8MVMt5ODe8VjOq_usSVw-1; Fri, 29 May 2026 11:23:32 -0400 X-MC-Unique: SZ8MVMt5ODe8VjOq_usSVw-1 X-Mimecast-MFC-AGG-ID: SZ8MVMt5ODe8VjOq_usSVw_1780068211 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-45ef3c7073eso483524f8f.2 for ; Fri, 29 May 2026 08:23:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068211; x=1780673011; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=YBx6SgN438Kse3DYiPhfWEcJAMkqbJwsQPerWAK4QDs=; b=MxEyEPzuM+qs8/MXYUx/mxmEsUCzYB+AFuxWl84A8XwCGeqdDdwDe7Zu/pYKZLNS54 WALgRhYUAyXKtf2oLZu98WYQDoX3rnLsFozqnIgwcMMP17p0WkeLNOHmx63dE3EEaKFH PM3QYOzG/njTlooAK6+jvZjuR61a+ff6pRJXKJ7+B4SZ08OiulKCwKhljRQxwfALcgsx tynzwxPIhOdPJdh6VvBFUV6bDUJnsdofLn7zxDtwFip5HA3bd1NbFDNuMjD5LX+aPh6t GrQsEWsSABpvxuIGclWs4zXaE9MjnU0hCH8Py93lpHDHZX8v32UPmPB0IA/azXd4hkKe B+NQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068211; x=1780673011; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YBx6SgN438Kse3DYiPhfWEcJAMkqbJwsQPerWAK4QDs=; b=GSbPYOqMPLrq7+g7CZbazG2evZsTVBmv/RDCo2r6jMXRceYNYv3XZsHE8FGcmGWPm/ q01yZ62RtWvmRJXZlhCVhl4IUNTftCgBtCgVL9LyvhtQqgEqDsAmZDh8Zzn7HG6jvqhU JqCtdSG0Ddpze6zHu7cTATUkRTIyWO95CU9lN1QcAWARK8zS9ds8M/DgroVibrBoNlBY qmjCeS1hqZXowCoOyDJbT/hn8TZj/V0dU7HMeFnmwa3BKwo+wdy3GdbGsy6rQsnUEuV/ q5Rv2ttwUpk14K9BM2QPj4QsGMwTuufncqxao1IoXSvoPfFiAk/UbxPFNY0CdL59oIUM G0AA== X-Gm-Message-State: AOJu0YxpY1mS9gi6p33AXNh/uolQd84ZSkoqdckFepgkbszwsCyJc5lx M3WEo182CUItBXtc1C+U4ODIS7pfdeD/WDj/uon9b53NtI0KUroZkN+iHv//1u1tXB8Mdqb8dNH tkAtUuPNSVsJspbo2eGYEfdENK26TqHJIplyWdKze4LQYO5RmfoR6GKXHyLNpPnT79nLa5tPIRg EXnl5yhi6F2anqVtDdW2S9iEN7Yz8TAU25WMdlltdFUmw= X-Gm-Gg: Acq92OGZpxQiUURo+21z5kaR32ccMOEEFxnePtvb+1xuJa3dhVzCP24qbjXYajOwyx/ 1KiPcuaoZbVSlkKatS8p5GVMG6pu+MyKq4dbVB6WImHhQJhsj1zQjvk45yBZM3EzxUQ3DO1plaN rgAtXK+qeCpwlE4vmMFZyfk2OJ9JDPBAxpeK4Qy0OsbvVgZO0qcm9ZPV7rOnepZS4VL0jCJoiRE VLcJxZyqhZC2JhEbE1kB1kvkQlEjSPpo4Sm9EZGnJAV3zC6hePh5O9gUm6TcBFbsQPeHZAVsedt LM37KoJFNeJlTecVT2jjXWVs7efnAiD0v1VgAdOecHDiGbWvu9nNaZKTnMCyrIXF33zArFbwPny l3JQhO13uLSI0Xnjnk+5MjkguIi+X6MsqbW6/FJHSZPXD+XzZpdNQWg== X-Received: by 2002:a05:6000:4812:b0:45e:f6d5:7db5 with SMTP id ffacd0b85a97d-45ef6d57f21mr93726f8f.18.1780068210718; Fri, 29 May 2026 08:23:30 -0700 (PDT) X-Received: by 2002:a05:6000:4812:b0:45e:f6d5:7db5 with SMTP id ffacd0b85a97d-45ef6d57f21mr93615f8f.18.1780068210090; Fri, 29 May 2026 08:23:30 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45ef35601d7sm5883909f8f.28.2026.05.29.08.23.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:23:29 -0700 (PDT) Date: Fri, 29 May 2026 11:23:25 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 16/37] mm: alloc_swap_folio: pass raw fault address to vma_alloc_folio Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Same change as the previous patch but for alloc_swap_folio: pass vmf->address directly instead of ALIGN_DOWN(vmf->address, ...). Signed-off-by: Michael S. Tsirkin Reviewed-by: Gregory Price Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/memory.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 75bee9501666..adc228fe3578 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4734,8 +4734,7 @@ static struct folio *alloc_swap_folio(struct vm_fault= *vmf) /* Try allocating the highest of the remaining orders. */ gfp =3D vma_thp_gfp_mask(vma); while (orders) { - addr =3D ALIGN_DOWN(vmf->address, PAGE_SIZE << order); - folio =3D vma_alloc_folio(gfp, order, vma, addr); + folio =3D vma_alloc_folio(gfp, order, vma, vmf->address); if (folio) { if (!mem_cgroup_swapin_charge_folio(folio, vma->vm_mm, gfp, entry)) --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5C4833D1A97 for ; Fri, 29 May 2026 15:23:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068221; cv=none; b=UCSs7TU/maDYSzP4/nGS5lMV5xOR38e48Rq2I5g0VKpqBzQ3TDwO1A3noaAp0wXFKZqGFIpqWV3EuR25rDebztrBEd4THlSoKnYIAcSk3ii2RLir366nN2JBvYL+UoF+BsA2jS/6XZftgVZwPi14HzxnCD6MHH/TrSV2it9ROVs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068221; c=relaxed/simple; bh=OWZLv3RcZNMXBK++VDzgdX5Eo6g4iZLM958nD4m3nTc=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=d9roXctnF29ZqfWfwjZGpG9RIo3VQ+rtpJUiEOHeDGLRMQBfsaFJvkyLaBMqecD0OwbDMFYC9hIU+aBmq7UKwUtXmdgp2i8LU8g6FuCmEWkRjZbcGiScebGy6AeKOSO+/Rr6cjE834E413qOp5b6Tqv+NoDHJOpUNZdli2s4tME= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=HbT+FGQ4; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=g/SVBFc7; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="HbT+FGQ4"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="g/SVBFc7" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068219; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=JxMX/pzMUOtdSR/DFO8X8z+nr68xKXZK5hfRbiRHySU=; b=HbT+FGQ4KMBLy2PEPv2GWqQobGU5w9cBBQeczN270Tz8jE3GXAfk4W79zXx9z/WqPCacee 1j1om3OjMiGUYaetELa4/3QXSxCQCIhViE+coLyenSfy0t1eBPr5Qk5CHKAhB3IMLkf6/y WTKPEV36F5LLeNbXGjjGsCMaNE/Auq0= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-313-UvBltZ3TNu68eFNVflNO6Q-1; Fri, 29 May 2026 11:23:37 -0400 X-MC-Unique: UvBltZ3TNu68eFNVflNO6Q-1 X-Mimecast-MFC-AGG-ID: UvBltZ3TNu68eFNVflNO6Q_1780068217 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-49045f93baeso55189945e9.1 for ; Fri, 29 May 2026 08:23:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068216; x=1780673016; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=JxMX/pzMUOtdSR/DFO8X8z+nr68xKXZK5hfRbiRHySU=; b=g/SVBFc7wEtpQ2Fouzv6zaTJLzfgZBsxSDgzcX/NHtoDNq/pIYXJnfREapgfzmYppf bmHu7bw9/Dq5eGjVc59wu7/nW7RTvY2CMhDIP9Mrhq3bhJWxjtfTO+jpihxLzZTnvRJc Qc6DcpBCERWbYmRVp61zIH/vo8U/9X95i6xn6hK7fbETGUznLMedxXxcTW7FoeDSmksK didJ24iB00b74T5GR1Xk+Nhe33jeJKiocwckoCx7qkNS9E/0j3brpoP3/ZupHRnXKUUZ FpLDIcsxmGcSa2KtEdeosZwyZB7UwJYNmgzJjGHT7hB11C3+28cPz9qx1J1AxlKBZC06 8uGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068216; x=1780673016; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JxMX/pzMUOtdSR/DFO8X8z+nr68xKXZK5hfRbiRHySU=; b=aZ9HW857hYcotA1dGL2BXywJpIG07uaSb+m/6LV1iXIO6y0Wr+gDvzyTJCef0J9uDL 3yN4GZa/v04CgI/bsLRcLNURrxavXiMRJRPlGfxsPZrKR1YTQWXVrOg5wZ78SnTVx5XZ BXJH2gGTZgUBcUFMu0PITfhuRSUXtlOrV/bqxJR0S8cBkxQBcPM7WaCcrF++LuLkSt63 23sQH2FtnUyw26YpO9jcIZ3Fzxhc4N351dtMXEjTbc8M17Xp1NmgIivNYRkwgqGmT1bF nIMF8F2oEcDwu9xU7vmbcbmArrFu/EnofcZzhdGPGzPsEVF3GiJMummenheIBuRa3viw Jxow== X-Gm-Message-State: AOJu0YxoZv3mtJsaG2SQ3nuSiKXHc1BtkpjkQWwZg8jw3RKgwHEtaSB5 GuDffLpF2UokGSm0uP4XQwNKNd+y9rbeylZIOHuphtdO0KNRf5mIKWddvOQoCck1l3Dy0Q7Kkhx 2K2OP33AqHePa9oKTV2rsdL+O88kxftH14Ay6vKDmCLuL+b2WB+gr6q44KezYXNsDeJh1sCKRIT jA0vF+FVN6OFOlxaJhh+YXiYszqnsINCVrGqjgXWW9rZg= X-Gm-Gg: Acq92OG3yJP0qT1mzl6TDEZuo4DUu/RB+RDnGBJG4paYted8RGKwqjtZJ5J8OT44ByH xapFc69OPs3wjISenINaasR2Nc0Dlk/S0o36Vgxh6kIQ7qpAv1wMMCNdfdmrMhT3O4G1qkpKgSD FD2pbf8sxw3PfwB8xPc1vunuUdM6hCwhrQvlsd2MWLn/HcPaUeAhRWwAi+Yra9Mes/tcGENI3Xr RscIFOTMGR9QKujdoWgRrvydGmLCqjEV/R/RGW9lRVyrOPWqDmTE1pah2IhkQ/F1tRncHKi2Hvk TSv+E1vGPr1pKX98aDbEyeN7RCtehF2+4w8OFWvy103kP+LBs3UqRWOKPmARfSfeaaVoioNRhi8 fc9wEEuDNp7fxWQo0joIXgktIANpebWCGtOTS+ZCh/Z8bV0oiQ/mfsg== X-Received: by 2002:a05:600c:3587:b0:48a:75b9:b0bc with SMTP id 5b1f17b1804b1-490a2941425mr1584945e9.29.1780068216424; Fri, 29 May 2026 08:23:36 -0700 (PDT) X-Received: by 2002:a05:600c:3587:b0:48a:75b9:b0bc with SMTP id 5b1f17b1804b1-490a2941425mr1583165e9.29.1780068215553; Fri, 29 May 2026 08:23:35 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4909dff2a80sm52414665e9.3.2026.05.29.08.23.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:23:35 -0700 (PDT) Date: Fri, 29 May 2026 11:23:30 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 17/37] mm: use __GFP_ZERO in alloc_anon_folio Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Convert alloc_anon_folio() to pass __GFP_ZERO instead of zeroing at the callsite. post_alloc_hook uses the fault address passed through vma_alloc_folio for cache-friendly zeroing. Note: with __GFP_ZERO, the folio is zeroed before mem_cgroup_charge(). If the charge fails, the zeroing work is wasted. Previously zeroing was done after a successful charge. This is inherent to moving zeroing into the allocator. Signed-off-by: Michael S. Tsirkin Reviewed-by: Gregory Price Assisted-by: Claude:claude-opus-4-6 --- mm/memory.c | 11 +---------- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index adc228fe3578..895ac72121fe 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5249,7 +5249,7 @@ static struct folio *alloc_anon_folio(struct vm_fault= *vmf) goto fallback; =20 /* Try allocating the highest of the remaining orders. */ - gfp =3D vma_thp_gfp_mask(vma); + gfp =3D vma_thp_gfp_mask(vma) | __GFP_ZERO; while (orders) { folio =3D vma_alloc_folio(gfp, order, vma, vmf->address); if (folio) { @@ -5259,15 +5259,6 @@ static struct folio *alloc_anon_folio(struct vm_faul= t *vmf) goto next; } folio_throttle_swaprate(folio, gfp); - /* - * When a folio is not zeroed during allocation - * (__GFP_ZERO not used) or user folios require special - * handling, folio_zero_user() is used to make sure - * that the page corresponding to the faulting address - * will be hot in the cache after zeroing. - */ - if (user_alloc_needs_zeroing()) - folio_zero_user(folio, vmf->address); return folio; } next: --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EE9EF3EA946 for ; Fri, 29 May 2026 15:23:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068241; cv=none; b=fbXhddTckz18/hoKO62t+tEZZ0KHvC/9DV/wy47/IZrSvuXDCSVxSAkfKnQAK3eCBMxJrxj+6Y/GirNaApNZepAiabfd6W9aG2TsxRpYN8qOrT4E3e2LojrMOxfeD03aGbYdBC5icBZqVNrc63nco23LIF7otWBjfr2YNxcuUs0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068241; c=relaxed/simple; bh=3uGUVJNy6kGsME3V8gAHY8oH+alSOzgrnDm2DGDTDks=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=qhPvkGUzpEkLwlY1ihEFvgPSDOC3+owF8TxeZrWLppF3WJnOUtb3TtZr6J/EKyI1lCY8B+eyM6mD87gQ2rrY6Cbj/sIbkX7Z0Zj6+X0joyshDqnFpBzJdzmdxeIN+qxh0mipfIX4asHMmRTI8CpvXzIEtd9m12a6qoBzCf9WUKs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Xkp2QP5K; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=iyW3EhVh; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Xkp2QP5K"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="iyW3EhVh" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068239; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=yH/Aq3jRD/J0BW4aKC2BcM/dVQFsNlzLQi2tKUktXO0=; b=Xkp2QP5KeOpHtud5b+LDenRU83+GFkeIIIFxHGdPJEu0rKNoWcspth1hNEhi+FbamZkr1v dJsqJyh2FFiyu5Bcrw6OacGrP6yFuek07rYPBu8LVBq109MKuWEV0hq/0P4bivuubO0JGP GICumbVC/a5UAPXTDOlM/UNSGfphvDo= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-439-2zKwDYXXM7OqOT_E011abQ-1; Fri, 29 May 2026 11:23:57 -0400 X-MC-Unique: 2zKwDYXXM7OqOT_E011abQ-1 X-Mimecast-MFC-AGG-ID: 2zKwDYXXM7OqOT_E011abQ_1780068237 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-4904ee02e72so60967855e9.1 for ; Fri, 29 May 2026 08:23:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068236; x=1780673036; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=yH/Aq3jRD/J0BW4aKC2BcM/dVQFsNlzLQi2tKUktXO0=; b=iyW3EhVhS10n/lZSX0noYzBt/hsqGp36BT/SmOEOolFuuJjsfARpc9BAN8QrmwMrIo viIDtFMqBX36dICEkXggF5QrZecMMxuCBMW78BPBAVm+5g5R+DRsqvooHh+hpnEYvwbN 2QCCqym2EN+dHb/WoHNxjbSsv64WK+eCDEvI74a/cfa6PRYL9nNcwYW2ZFBZol5d5Bj0 81b6LpKEmfvxMdzMmD1GrMasOv0tnIP0/OMITSMelI3leF7m3WhzQty6TkuMAce7OIFZ P2D97vWRYExjLmBVPhyKuzeKEvuU8VvwNMdkQhoLLdIlcdwK0DHZ8Mu3X+52ptRmPAsu Zzjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068236; x=1780673036; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yH/Aq3jRD/J0BW4aKC2BcM/dVQFsNlzLQi2tKUktXO0=; b=FP7WdJrPg0fFpXe8HqQZQqeqM0WnahGHjj6MyIX1b/8Qb9712pyHBboiij0HIbXYE4 GmMJ5HNDaTQ2uGeGY5uVokMcNXHNX9Rv+l92m6DvAgnHbn8JDacNsllFFYmKiUeeloCn KR6C3WQZp6Sxfi91D+oQQqByJjhDAA09feP0IOqeb1RLIhotSDBw3JTNl0Mq4fxbSNxD ZGswTAxJbhwyBUoe0JMZK+GGuixAw9mdo0nihgtKZZ8PTAgCMhhQ583NQBcx8q7gHZwq Uz0Yisr7yoXlkaek2jCC2hKJeg9Ul+TP9mSTM1m90qNw+oCGryCDldy7lYl64WXTo+hV t1mA== X-Gm-Message-State: AOJu0YxFWsbfQbstnFb8sF4c5S0SNTUEyyID+DSbIA+YK6Ca6cOOswA/ MX+stg1Z7GCyrnAMxJQqmhSsiXoznxoH138fUM31A2PeS3BKPSRmg2F6hbMX6B5D5O0IcD88B80 oKnnFtYhMDBs0McDKTKaS3tFLOXv1AoUhV3vifoyEoinST8oG/oblH8V4wNLW7g+6/eyrUEKmf8 Ce6hB7jn36mQzpOgiIDtnhEgPtyTE9bGjxKZAsFushSgo= X-Gm-Gg: Acq92OEs3IMiWuR26IjNrncaHhvr+VTuZcMv/pqzpPG7xFFfX8iBjSgtxEVYVmFDheA HewXn9m0lYVWFWaqW7y3otwbqyMZ+aBu0WCUDe8UMKzFe1+0akRPmp2lP3Q7+Kb8pd2cILxoIb0 uFnl5J41fVL80s2IEr9YMJagssBfVyYfIJMpnxdnQ/1S94thzUE/9RpzlTtzjEurSJBNNvDey7H qxJr7ygmTHlbPlOCUsFKrNlbJgKUexh/Jg61rOFYOs6E4tMPk2ifRP74DxFJH/2MDnC65K7rVhJ r8LkAmT72j3A1iku+lYouis93B3YW5K98Ywzb2gpNsB+q3I5XF/vkigKFXOWQmtlGXxQtmv5Azj Kc/sT2WhOtTLseTVCnvr/wka73GU= X-Received: by 2002:a05:600d:6413:20b0:490:890a:da46 with SMTP id 5b1f17b1804b1-490a292a4b7mr1450855e9.2.1780068236480; Fri, 29 May 2026 08:23:56 -0700 (PDT) X-Received: by 2002:a05:600d:6413:20b0:490:890a:da46 with SMTP id 5b1f17b1804b1-490a292a4b7mr1449715e9.2.1780068235845; Fri, 29 May 2026 08:23:55 -0700 (PDT) Received: from redhat.com ([2a0d:6fc0:151d:1200:7829:d653:3902:e833]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45ef356a4c4sm4498359f8f.31.2026.05.29.08.23.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:23:55 -0700 (PDT) Date: Fri, 29 May 2026 11:23:35 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 18/37] mm: vma_alloc_anon_folio_pmd: pass raw fault address to vma_alloc_folio Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that vma_alloc_folio aligns the address internally, drop the redundant HPAGE_PMD_MASK alignment at the callsite. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Reviewed-by: Gregory Price --- mm/huge_memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 970e077019b7..d689e6491ddb 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1337,7 +1337,7 @@ static struct folio *vma_alloc_anon_folio_pmd(struct = vm_area_struct *vma, const int order =3D HPAGE_PMD_ORDER; struct folio *folio; =20 - folio =3D vma_alloc_folio(gfp, order, vma, addr & HPAGE_PMD_MASK); + folio =3D vma_alloc_folio(gfp, order, vma, addr); =20 if (unlikely(!folio)) { count_vm_event(THP_FAULT_FALLBACK); --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 60F363F1651 for ; Fri, 29 May 2026 15:24:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068246; cv=none; b=W81/7VntUPq/d/7rQ9lldFxVca9/KzsgmNlanKDIx/6Go2defbUB6VgpRgUvSC68rhu55L698AmgBDtRHsPbzcnHrKZluVuylR1swCXeosFxNmXsdkwNHHw6jFJjPE41NZun27t33nwOhBMq8n5AwGYZLDK6/IeqOI011sb3rh8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068246; c=relaxed/simple; bh=xLQXzmmGjvEYiJuwU2B0ROV9KzzjJEnYf6CxLCTRo/c=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=a8Dw9dFslQ6mJt9EH+AN8ldGPNk8YO5mnb9oIJw33UDq8Gcd40tnbPXCXl7wITgP4OUg5goRhx6BDFy1bZ4VadEwbKGMZzaXbtNAX/VSjVHe+WsuemGCU4IhkYL5qLRYK7Y/AcUVm9/6GS8vaMNP3ibxm6kMt33PtNhp+m5bECk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=ivKOvE5I; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=GuSCZK4z; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ivKOvE5I"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="GuSCZK4z" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068244; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=p8FgYlTim+1qafXpBvmXgDRFgmjZT3BzBxl1fqmzl20=; b=ivKOvE5IumAu6zBp2sy/LwrAw4gvCQ7dDc1mIPzV8A8GFzGlclvZYXm+OarE5s6plf+nLG a0ShWOYD0vdIdBis7kbLm51PUUXnfbYeZeXS6PCaOTpq7jejX7egLrEiuRuub4EYYK89Io dXmrjSjWCkIlaXYbbDHyPSCBymBMzzs= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-541-I2qHAMZkOAmkDFy_Vyhzig-1; Fri, 29 May 2026 11:24:03 -0400 X-MC-Unique: I2qHAMZkOAmkDFy_Vyhzig-1 X-Mimecast-MFC-AGG-ID: I2qHAMZkOAmkDFy_Vyhzig_1780068242 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-45ef63d1214so173135f8f.3 for ; Fri, 29 May 2026 08:24:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068242; x=1780673042; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=p8FgYlTim+1qafXpBvmXgDRFgmjZT3BzBxl1fqmzl20=; b=GuSCZK4zaO6C+iaWzVWCNPJYjKVRx0/6Cc/jHlJ9faJPbRUGGLLUuPNewX7MnLeDjD xO4keU3sWVJZpTFVXarTZ1fB/hsqtDFkW5E5To5Wq6Ykp2b17nWIkHhEKXGetk3hl75S bsWZT4qZAEdJd5VG9ICstiFGQe0d0a+JoNm/WMqDR2K9keCWnIX/TC97ag3rj4J7sghN QnUOqZF4Ng8eoY9TljY6wN/g5FH75At6JUnYq/hHt4NQpHpA3fzPl4z3C5yNzdzYtdgl r1nIeKRlAr9QQJw35YAKY3VNfD6LCFQan3R3xVzQaGh4Tn5U+EVdqltPQhP0+z9dWwrh kixQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068242; x=1780673042; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=p8FgYlTim+1qafXpBvmXgDRFgmjZT3BzBxl1fqmzl20=; b=G7lJqU/KBeNAVAe0dEtavd4W2kZ3NxoLiXgaH2Xn4klwckrP5R5nWJ9lJREwK2fXyp uLsZTZAWfx8+5awoQDQcVI+1BtxHIeMOI4xya0csZiJArt8+HWNslERhWHLUQHo5gEw8 2XnhHhgA3YBtEZ6oRNJvgu7GKNbjmZN8ZsLTr/H1eOiywfanr0op3Y05WrnOO7eZakoz OU6IsTvpLiit99H0y9/5DqR/vCXbVnrXUjy4lQLljXtdjRSUclghN1zOACjDPxRSjCfl ZFMDAESa4BUn3+NdizDB5U5Hapnzhvc3LNLWXfuSviJnAprUd08oU2JBr60s/KIyGP9f 8zDA== X-Gm-Message-State: AOJu0YwF1Otp2xJH/S/F5SEyXAtSPHlq8QqLV7tP8eOLFE8lxiDATTBX pYvW0jaxvQrb7w92C0jLxinSvzTYuLB+ju9SlI4N6xN0G/F8YI+cskXtsIxtrpH3pZzKHEpCtJR dJbzfdLLauiBEosmMenAXV8espCNbLcwDyhY+hTJbsKr/PkyOMHDJPITZ2n+gn+OyoZrAYS1bPr zymCbkQXZBJzxiQBMU1QaPW5l3+mEznrFTJkhYJkUMP0k= X-Gm-Gg: Acq92OHZYZJ46vEc2AacqwY7cyTekjB6gceOza4bPkCpoynXYRhWGDGW1+qQAI5WPaW 9WwZKPpweYDsopT6SVr72+9d7+bMVCjBPT3Atmor8h/4ohX6bkprN6ZuU6FUdts54qLM9IWcafR aLCWwRC9wI4K3B7iqmQlmiCHymsN7f/r08JsQEqtjYoeb9UbJit8BPq0f5xpjSwmrrESX+hWxLM RwyrroSbPIl6PRFy2OGhGYbUZwo9yYxJeJjbVQvCQTWBh90AQ34k0k+nLBVQevpzUTPfxcNeQJo mbmGEKAyRRS3IOvNOEmETRh/3DOEwQIBLKkVuql9G1sgbLaPKQzv/VZd1IRbgA1jqnYRQOpC8+K AkwcWNdIkT72OKXAdU1emQPZ59ztE49GnKHb9SRbDS90b8HuxN5lGmA== X-Received: by 2002:a05:600d:6445:20b0:490:4f91:5519 with SMTP id 5b1f17b1804b1-490a2958db9mr849535e9.18.1780068241850; Fri, 29 May 2026 08:24:01 -0700 (PDT) X-Received: by 2002:a05:600d:6445:20b0:490:4f91:5519 with SMTP id 5b1f17b1804b1-490a2958db9mr848535e9.18.1780068241185; Fri, 29 May 2026 08:24:01 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4909c104920sm16482535e9.21.2026.05.29.08.23.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:24:00 -0700 (PDT) Date: Fri, 29 May 2026 11:23:56 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 19/37] mm: use __GFP_ZERO in vma_alloc_anon_folio_pmd Message-ID: <6dcf0045e069812ef06a71ebcf67ef0c819eb51d.1780067977.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Convert vma_alloc_anon_folio_pmd() to pass __GFP_ZERO instead of zeroing at the callsite. post_alloc_hook uses the fault address passed through vma_alloc_folio for cache-friendly zeroing. Note: with __GFP_ZERO, the folio is zeroed before mem_cgroup_charge(). If the charge fails, the zeroing work is wasted. Previously zeroing was done after a successful charge. This is inherent to moving zeroing into the allocator. Signed-off-by: Michael S. Tsirkin Reviewed-by: Gregory Price Assisted-by: Claude:claude-opus-4-6 --- mm/huge_memory.c | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index d689e6491ddb..4978d34532ea 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1333,7 +1333,7 @@ EXPORT_SYMBOL_GPL(thp_get_unmapped_area); static struct folio *vma_alloc_anon_folio_pmd(struct vm_area_struct *vma, unsigned long addr) { - gfp_t gfp =3D vma_thp_gfp_mask(vma); + gfp_t gfp =3D vma_thp_gfp_mask(vma) | __GFP_ZERO; const int order =3D HPAGE_PMD_ORDER; struct folio *folio; =20 @@ -1356,17 +1356,9 @@ static struct folio *vma_alloc_anon_folio_pmd(struct= vm_area_struct *vma, } folio_throttle_swaprate(folio, gfp); =20 - /* - * When a folio is not zeroed during allocation (__GFP_ZERO not used) - * or user folios require special handling, folio_zero_user() is used to - * make sure that the page corresponding to the faulting address will be - * hot in the cache after zeroing. - */ - if (user_alloc_needs_zeroing()) - folio_zero_user(folio, addr); /* * The memory barrier inside __folio_mark_uptodate makes sure that - * folio_zero_user writes become visible before the set_pmd_at() + * page zeroing becomes visible before the set_pmd_at() * write. */ __folio_mark_uptodate(folio); --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 744193F1AC6 for ; Fri, 29 May 2026 15:24:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068253; cv=none; b=sFEhyYB37khbqZmKRUrpcyDMD5yS7Ae2CLF+gy/Gbsw7xgdSPpm6lfVg3z79EiV34JGQ2CIv1iqoojcOki8CwOCYHkPZPP7T9I7CxcpTJjGVXNEP6c7ZOtXwKDDFKcEcw6nTr2gWDNwgIiKOG9I06ZDBwOhtyJhZECr0pslULYc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068253; c=relaxed/simple; bh=MOLGrRKYtAzXHsjQPObl4upRAuBjzhEEowJkGu/kpHs=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=JTLs8EFNrbo1BdsAi7VKC8xwTatuiUi443aC6nfx6lDecWAJA7qvoSyr3736lkbqFMHAkJ+qrKNrNkn2n9t0qpSRqXhroS1SogL8SfWPkZ7I7cwsCFw/vq99LcADxegsPptS99jeG5crXVUt6HrQYWcsQoboOVHQoCJX3i20QjI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Y1rDkGx+; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=NxQJgS7x; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Y1rDkGx+"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="NxQJgS7x" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068250; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=YFnBVDBDzVeV9p9dYXmJ4JPUR3YMu/6T3H2GlWvoQjs=; b=Y1rDkGx+vgybkYSA0eHhqScEYfbjq18Eq4ln1deiNEyAPONZEixy18jVjqfT/cTf1HO3kk 4hL4yJL2v6riSxiIebrW0uPvLzWMTTvKeF+DjvotMkozORVeUx4cO9qibTJpSxv8cZ9sGM 8ZIU55mHaYg9GNDKVWJN5GKOaqaG+3w= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-647-7SsUxWKDOH2QvgRoIY1EZA-1; Fri, 29 May 2026 11:24:09 -0400 X-MC-Unique: 7SsUxWKDOH2QvgRoIY1EZA-1 X-Mimecast-MFC-AGG-ID: 7SsUxWKDOH2QvgRoIY1EZA_1780068248 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-4909e55c480so9358375e9.3 for ; Fri, 29 May 2026 08:24:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068248; x=1780673048; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=YFnBVDBDzVeV9p9dYXmJ4JPUR3YMu/6T3H2GlWvoQjs=; b=NxQJgS7xbOel4A65M5NDQPcbdAuoMzKOGY4B3PHbTyXX7oA8Mm6BPqBZ8MXz0SuIL2 t+G6B9+GK4ryYcdWkmkGmtFJpfYKUZ4s0z3PRaq+kHNd8BqN7fbkmQBVmQu4RtBv3WmV /TvBLR320GuqxDkbZBoiJMRqg0GECY9mO6XvxQVZbvpiw9r3YL/oueP6e+73wAZBqumj SODnekqRfzLqjJu8KN9Tf5CaxZ5zBqaf81X/xmv6XmA+BTm/hsmYsQ5UOD3wGdOhyCyk w48pE4ISMvcnZIQM90gHg2dh3hjZEriMFOcA+t/BBR1LXI9AWcGs6hi/6KYAh5QyOVk4 RIDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068248; x=1780673048; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YFnBVDBDzVeV9p9dYXmJ4JPUR3YMu/6T3H2GlWvoQjs=; b=M9F62lJiSQjJXUQaynIXu7tVPDEFHhvToOqcOdvP0qM4lSW+76rsUVKPykoTM3t5Y7 eIQDlEImnUcAiGy6+rD0DEKbZPHLOX9qMvkkjUDflPhRX4i2+LMoG5ZlskHZs8kN05RF zbGe11QMuk0ScArmG57fkUaxPG+Q93LW78D8+1I/5LsxFQ1kd4FSCBE1tjHDScBeKngi oq9QdBv0CqQOX9An8Y1yMs2wjmSt7DiZMgREW+AninvVb6qwuVkEa9XVpOrIJteKRDti V9gKmpz7z+pFWQM6ZPClKu70cCFnD35nR+c7WOthfjWikOfbwRIQFWDUIHGIQXEU1CM/ Md+Q== X-Gm-Message-State: AOJu0YwdaAGGPL+kB+HdBJTjgujbVL3bXAkca/N7ZrHGvsjNgaanN43F peeoV6IS73+sImgkHE6wYn/iS6MeO9fbXWWPvVg/G0gI9ldS2wZpBgkwtiNr+nIjEwH3XoUoFj9 N54jahAgoRZ3GbQDLODmYxhbQzKQfMDYiuGECSyi2dkePRyegfXscWNR3MIxXtWsh731K482EKJ N195xgDIu9V3VVaaVie7HD6pR0Vp2iOP9mDCqOyJ15VDc= X-Gm-Gg: Acq92OGeD/cNT7eVsqvNsOv5Dfg2cPvojmIcdgZ4YDorC14ChfeLg6pX83nrop2VBGt r1KtCvOuX6qddWjsZrOR39vcAA/oiT7kQkpcnapKQ0U7d0Lq+GwFPzUApOKOinc+XffCdZrHKEX XPXJkUXOTsj11ikKPgGX08cuJwntAAXUi6gMEJY6rA8UaZnEkjG94Ig7g9T1KVwZ//NjrpWqfge ImGZHvrFzlBAFbKxwecMUjza7fuU/2IzyK63h3V7ZOh/ogmfqyQQPNpXESuM6dGkDfHMqjwxpXb oMWXGbyTQy4YJ8+x9gBAmrKUnPR4eKs8gRD4sUllLYe4/sXM0uhz5yuzu22DRg4qdbzWig8K9Qd +8VWGZCKzGvkB9LwHWHptAPhwc0LO1IczqxHxwV2ObzHqIfYwr+DtRg== X-Received: by 2002:a05:600c:6385:b0:490:40f1:5314 with SMTP id 5b1f17b1804b1-490a290ec08mr2725455e9.1.1780068247642; Fri, 29 May 2026 08:24:07 -0700 (PDT) X-Received: by 2002:a05:600c:6385:b0:490:40f1:5314 with SMTP id 5b1f17b1804b1-490a290ec08mr2723495e9.1.1780068246818; Fri, 29 May 2026 08:24:06 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4909c09abadsm16343845e9.1.2026.05.29.08.24.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:24:06 -0700 (PDT) Date: Fri, 29 May 2026 11:24:01 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 20/37] mm: hugetlb: add gfp parameter and skip zeroing for zeroed pages Message-ID: <783540fe912fe437c98385c1b288a7695ef45ab9.1780067977.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a gfp_t parameter to alloc_hugetlb_folio(). When __GFP_ZERO is set, the function guarantees the returned folio is zeroed: - Fresh allocations (buddy or gigantic): zeroed by post_alloc_hook via __GFP_ZERO, HPG_zeroed set by alloc_surplus_hugetlb_folio. - Pool pages with HPG_zeroed set: already zeroed, skip. - Pool pages without HPG_zeroed: zeroed via folio_zero_user(). The address parameter is renamed to user_addr; the function aligns it internally for reservation and NUMA policy lookups. For pages that need zeroing, user_addr is passed to folio_zero_user() for cache-friendly zeroing near the faulting subpage. All callers pass a page-aligned address; the hugetlb_no_page caller passes vmf->real_address & PAGE_MASK for consistency. HPG_zeroed (stored in hugetlb folio->private bits) tracks known-zero pool pages. It is set when alloc_surplus_hugetlb_folio allocates with __GFP_ZERO, and cleared in free_huge_folio when the page returns to the pool after userspace use. Suggested-by: Gregory Price Reviewed-by: Gregory Price Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- fs/hugetlbfs/inode.c | 3 +-- include/linux/hugetlb.h | 5 ++++- mm/hugetlb.c | 42 +++++++++++++++++++++++++++++------------ 3 files changed, 35 insertions(+), 15 deletions(-) diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c index 8b05bec08e04..5856a3530c7b 100644 --- a/fs/hugetlbfs/inode.c +++ b/fs/hugetlbfs/inode.c @@ -810,13 +810,12 @@ static long hugetlbfs_fallocate(struct file *file, in= t mode, loff_t offset, * folios in these areas, we need to consume the reserves * to keep reservation accounting consistent. */ - folio =3D alloc_hugetlb_folio(&pseudo_vma, addr, false); + folio =3D alloc_hugetlb_folio(&pseudo_vma, addr, false, __GFP_ZERO); if (IS_ERR(folio)) { mutex_unlock(&hugetlb_fault_mutex_table[hash]); error =3D PTR_ERR(folio); goto out; } - folio_zero_user(folio, addr); __folio_mark_uptodate(folio); error =3D hugetlb_add_to_page_cache(folio, mapping, index); if (unlikely(error)) { diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index f016bc2e8936..49e5557d6cc0 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -599,6 +599,7 @@ enum hugetlb_page_flags { HPG_vmemmap_optimized, HPG_raw_hwp_unreliable, HPG_cma, + HPG_zeroed, __NR_HPAGEFLAGS, }; =20 @@ -659,6 +660,7 @@ HPAGEFLAG(Freed, freed) HPAGEFLAG(VmemmapOptimized, vmemmap_optimized) HPAGEFLAG(RawHwpUnreliable, raw_hwp_unreliable) HPAGEFLAG(Cma, cma) +HPAGEFLAG(Zeroed, zeroed) =20 #ifdef CONFIG_HUGETLB_PAGE =20 @@ -706,7 +708,8 @@ int isolate_or_dissolve_huge_folio(struct folio *folio,= struct list_head *list); int replace_free_hugepage_folios(unsigned long start_pfn, unsigned long en= d_pfn); void wait_for_freed_hugetlb_folios(void); struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, - unsigned long addr, bool cow_from_owner); + unsigned long user_addr, bool cow_from_owner, + gfp_t gfp); struct folio *alloc_hugetlb_folio_nodemask(struct hstate *h, int preferred= _nid, nodemask_t *nmask, gfp_t gfp_mask, bool allow_alloc_fallback); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 3a6afbe99116..4ccf4eb91d92 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1712,6 +1712,9 @@ void free_huge_folio(struct folio *folio) bool restore_reserve; unsigned long flags; =20 + /* Page was mapped to userspace; no longer known-zero */ + folio_clear_hugetlb_zeroed(folio); + VM_BUG_ON_FOLIO(folio_ref_count(folio), folio); VM_BUG_ON_FOLIO(folio_mapcount(folio), folio); =20 @@ -2113,6 +2116,10 @@ static struct folio *alloc_surplus_hugetlb_folio(str= uct hstate *h, if (!folio) return NULL; =20 + /* Mark as known-zero only if __GFP_ZERO was requested */ + if (gfp_mask & __GFP_ZERO) + folio_set_hugetlb_zeroed(folio); + spin_lock_irq(&hugetlb_lock); /* * nr_huge_pages needs to be adjusted within the same lock cycle @@ -2176,11 +2183,11 @@ static struct folio *alloc_migrate_hugetlb_folio(st= ruct hstate *h, gfp_t gfp_mas */ static struct folio *alloc_buddy_hugetlb_folio_with_mpol(struct hstate *h, - struct vm_area_struct *vma, unsigned long addr) + struct vm_area_struct *vma, unsigned long addr, gfp_t gfp) { struct folio *folio =3D NULL; struct mempolicy *mpol; - gfp_t gfp_mask =3D htlb_alloc_mask(h); + gfp_t gfp_mask =3D htlb_alloc_mask(h) | gfp; int nid; nodemask_t *nodemask; =20 @@ -2877,16 +2884,19 @@ typedef enum { * When it's set, the allocation will bypass all vma level reservations. */ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, - unsigned long addr, bool cow_from_owner) + unsigned long user_addr, bool cow_from_owner, + gfp_t gfp) { struct hugepage_subpool *spool =3D subpool_vma(vma); struct hstate *h =3D hstate_vma(vma); + unsigned long addr =3D user_addr & huge_page_mask(h); struct folio *folio; long retval, gbl_chg, gbl_reserve; map_chg_state map_chg; int ret, idx; struct hugetlb_cgroup *h_cg =3D NULL; - gfp_t gfp =3D htlb_alloc_mask(h) | __GFP_RETRY_MAYFAIL; + + gfp |=3D htlb_alloc_mask(h) | __GFP_RETRY_MAYFAIL; =20 idx =3D hstate_index(h); =20 @@ -2954,13 +2964,12 @@ struct folio *alloc_hugetlb_folio(struct vm_area_st= ruct *vma, folio =3D dequeue_hugetlb_folio_vma(h, vma, addr, gbl_chg); if (!folio) { spin_unlock_irq(&hugetlb_lock); - folio =3D alloc_buddy_hugetlb_folio_with_mpol(h, vma, addr); + folio =3D alloc_buddy_hugetlb_folio_with_mpol(h, vma, user_addr, gfp); if (!folio) goto out_uncharge_cgroup; spin_lock_irq(&hugetlb_lock); list_add(&folio->lru, &h->hugepage_activelist); folio_ref_unfreeze(folio, 1); - /* Fall through */ } =20 /* @@ -2983,6 +2992,10 @@ struct folio *alloc_hugetlb_folio(struct vm_area_str= uct *vma, =20 spin_unlock_irq(&hugetlb_lock); =20 + if ((gfp & __GFP_ZERO) && !folio_test_hugetlb_zeroed(folio)) + folio_zero_user(folio, user_addr); + folio_clear_hugetlb_zeroed(folio); + hugetlb_set_folio_subpool(folio, spool); =20 if (map_chg !=3D MAP_CHG_ENFORCED) { @@ -4991,7 +5004,7 @@ int copy_hugetlb_page_range(struct mm_struct *dst, st= ruct mm_struct *src, spin_unlock(src_ptl); spin_unlock(dst_ptl); /* Do not use reserve as it's private owned */ - new_folio =3D alloc_hugetlb_folio(dst_vma, addr, false); + new_folio =3D alloc_hugetlb_folio(dst_vma, addr, false, 0); if (IS_ERR(new_folio)) { folio_put(pte_folio); ret =3D PTR_ERR(new_folio); @@ -5520,7 +5533,7 @@ static vm_fault_t hugetlb_wp(struct vm_fault *vmf) * be acquired again before returning to the caller, as expected. */ spin_unlock(vmf->ptl); - new_folio =3D alloc_hugetlb_folio(vma, vmf->address, cow_from_owner); + new_folio =3D alloc_hugetlb_folio(vma, vmf->address, cow_from_owner, 0); =20 if (IS_ERR(new_folio)) { /* @@ -5780,7 +5793,13 @@ static vm_fault_t hugetlb_no_page(struct address_spa= ce *mapping, goto out; } =20 - folio =3D alloc_hugetlb_folio(vma, vmf->address, false); + /* + * Passing vmf->real_address would work just as well, + * but PAGE_MASK helps make sure we never pass + * USER_ADDR_NONE by mistake. + */ + folio =3D alloc_hugetlb_folio(vma, vmf->real_address & PAGE_MASK, + false, __GFP_ZERO); if (IS_ERR(folio)) { /* * Returning error will result in faulting task being @@ -5800,7 +5819,6 @@ static vm_fault_t hugetlb_no_page(struct address_spac= e *mapping, ret =3D 0; goto out; } - folio_zero_user(folio, vmf->real_address); __folio_mark_uptodate(folio); new_folio =3D true; =20 @@ -6239,7 +6257,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, goto out; } =20 - folio =3D alloc_hugetlb_folio(dst_vma, dst_addr, false); + folio =3D alloc_hugetlb_folio(dst_vma, dst_addr, false, 0); if (IS_ERR(folio)) { pte_t *actual_pte =3D hugetlb_walk(dst_vma, dst_addr, PMD_SIZE); if (actual_pte) { @@ -6286,7 +6304,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, goto out; } =20 - folio =3D alloc_hugetlb_folio(dst_vma, dst_addr, false); + folio =3D alloc_hugetlb_folio(dst_vma, dst_addr, false, 0); if (IS_ERR(folio)) { folio_put(*foliop); ret =3D -ENOMEM; --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 89A69421F14 for ; Fri, 29 May 2026 15:24:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068259; cv=none; b=LBrdbMSpoCDiYSLZDKoUvhHBan6TL9sSvbPrrwg5hd/gfRGUpyteHHbSvLlFn2LzP0bqSo3VdWvuG5AvqsoCLyit9/3c2eZq6DfsF0RgcxRFAPCPMHE5XjwNg6tiGkd9CP5uufqprgn8WieR/QughN03VEFMAYov6/ByC3vM10U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068259; c=relaxed/simple; bh=Axczvl8C15WZZkzyVXb9w4AL4G4ah5FFkNusKG6v0bk=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=gYtZhbxqEpruqbq2jb7ak0o0FkFW2D0bV1sXLjI6kmStlEd8JUV2DwTh1gHeMlAOsL0uVdIx2bQVvC2DADCXTvXnmqHZStF1tvQ//5K6pXMJ42EnqHES6Q3f8XtjzyOedElp80PMZREymwF8Vct07G0kLGYWQ793vrnc5g1sbAU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=hmjHS/xz; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=CxejAr9l; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="hmjHS/xz"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="CxejAr9l" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068256; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Ha0MKvEjPpqKYjU+TNR9cOwfxNwOYM3BFVBNkyNiIfo=; b=hmjHS/xz6CWhW6iIu6yN2Rj+aC6ePhv1TnPgBFi2JTpyqu0lb0JeLiP82KBRifb3qI/72m lAiHZuNk42XbfB67HCHlmW8iS0KLE40f8i6U84rvNO3QeIjGUZp8r3YHr/vWaZj5HvFpol iaoRJUHx/4Gy76tmL1+SgeC8/mTYrS8= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-218-f2l9qu0MPE2GLYFK3rPcHA-1; Fri, 29 May 2026 11:24:15 -0400 X-MC-Unique: f2l9qu0MPE2GLYFK3rPcHA-1 X-Mimecast-MFC-AGG-ID: f2l9qu0MPE2GLYFK3rPcHA_1780068254 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-4903dcb32f8so69896415e9.0 for ; Fri, 29 May 2026 08:24:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068254; x=1780673054; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Ha0MKvEjPpqKYjU+TNR9cOwfxNwOYM3BFVBNkyNiIfo=; b=CxejAr9lWDksrvKmsTrsjjJq3XQZMgX8fCSdk7Dl53tHCAFEYkfqOS3soRQ4YcVwbY YkZZxunQI5ccwDhbqZasF9UJME20r7QdlZooK15fz5n+xA5uvPlr4S0cdQg6y64jp/s+ nyoCVK9xJV0YM3V28GYV0HP9DuBC7rN6FpslzhV7zaOPAwbC0sz63tgOU5/v0+9WHxrc E/ccCJEgZvdoCUHU1jgNOYyyadBGl5RCqqI0C0l5kurNIG7MSxIlNZwqlHrhd43zzoDq ai0NNBPkpVElasPJ93tFeae9/qCmT/61YlvPqO1i/BFE7ONY7nqSJV2r7ZgRD7mrYryN 8uJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068254; x=1780673054; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ha0MKvEjPpqKYjU+TNR9cOwfxNwOYM3BFVBNkyNiIfo=; b=Ln9kXDlnaQ23LQ72JLyTKZjAaYf6a44WSF7IPxMuWYDBR/3g5aKLtKSbVjYyHzZbc0 hGvqN7SVy6O98QPk9vckG7LPXfp3D//bbsdMnyOAZ36hpy22SNviEV8aLAcBh5U6EB1y pjMSFq+bBiPg27wwmM9GA3MEAUwz4znjZXNxjzolcNPTUi3atGyFHypGGYe8+WDt2ICh JxxeY0fhveOQ52y+Y2v1AHPImlhlZ3KXdb/59wZ5ueqCG3WoJ5I6Vzsrg0tYVTTzFiRC DK0+VoevOYtrybhVSIyB8KO3ZD6IDyeV3g/aVdpND4/REUuYezIAv/aE2T437s3p9Spc Rtgg== X-Gm-Message-State: AOJu0YwijtzWkfz/5JSmwa9ZwtkIld6DR5sxioznJdq9GucpLJPZdvWL H46HJ0Ranhz7iJi23U4Uf2xOrU0YAGqita8O73QxYtHkNbQId4fq7Z/Q1kM11CqIMfR2etYoAif A+TMchcabzuXAiF65ncWL1/EswqnzNUfj/aR77N4lQOKzfUmf8NZAIOJtsVxXu2rNRdk10CaK+8 sL0ZMwmAC/opUEO7DX14GQszfyue93ll6L0+9zGa6dycw= X-Gm-Gg: Acq92OG+MkfTMWXuF/vouaO3bf2cVSm9NzRZAoT0W+2qlf+xInTXawr8Ssag0D6txNA Rts4gTN847eT2H/3lg7oaGBevp038fKHz6+KOGVG3IC4Dq3Tfgl6gNJw8j9GxfYfDn0OI/Esc5O s35CraR+GyH3+wK3+HcQvfXou+GPTv96NRXg9Y0qt45PeUlp82nd0s2rVPnS182QQfaUoo49Dae qMxHryVI0CR0bDIH020h6NzQN5pwqOqvWU8A7ysDhlGASHcRLVkxIQmyYI3/8SI3He2Z8QeEjxp V5huE5N37ksiyQvga7hKGPakXwyA8qEQ7XBT/EjDLi9nNmG+TeGT66vIlnsi9F/qE4O3IxjrRXj qehrrehLrZMxX4R7KX73yuDTaRXKtgYv9+Yc8//RsZivN3yKXVZsq3Q== X-Received: by 2002:a05:600d:640f:10b0:489:2005:b36e with SMTP id 5b1f17b1804b1-490a2941d4emr1179115e9.19.1780068253562; Fri, 29 May 2026 08:24:13 -0700 (PDT) X-Received: by 2002:a05:600d:640f:10b0:489:2005:b36e with SMTP id 5b1f17b1804b1-490a2941d4emr1178125e9.19.1780068252950; Fri, 29 May 2026 08:24:12 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4909c103a89sm16422195e9.20.2026.05.29.08.24.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:24:12 -0700 (PDT) Date: Fri, 29 May 2026 11:24:07 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 21/37] mm: memfd: skip zeroing for zeroed hugetlb pool pages Message-ID: <43b19542a07b568279d1a6e734d6729d347b78d5.1780067977.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add bool *zeroed output to alloc_hugetlb_folio_reserve() so callers can check whether the pool page is known-zero. memfd's memfd_alloc_folio() uses this to skip the explicit folio_zero_user() when the page is already zero. This avoids redundant zeroing for memfd hugetlb pages that were pre-allocated into the pool and never mapped to userspace. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/cma.h | 3 ++- include/linux/hugetlb.h | 6 ++++-- mm/cma.c | 6 ++++-- mm/hugetlb.c | 11 +++++++++-- mm/hugetlb_cma.c | 4 ++-- mm/memfd.c | 14 ++++++++------ 6 files changed, 29 insertions(+), 15 deletions(-) diff --git a/include/linux/cma.h b/include/linux/cma.h index 8555d38a97b1..dee88909cf5d 100644 --- a/include/linux/cma.h +++ b/include/linux/cma.h @@ -53,7 +53,8 @@ extern bool cma_release(struct cma *cma, const struct pag= e *pages, unsigned long =20 struct page *cma_alloc_frozen(struct cma *cma, unsigned long count, unsigned int align, bool no_warn); -struct page *cma_alloc_frozen_compound(struct cma *cma, unsigned int order= ); +struct page *cma_alloc_frozen_compound(struct cma *cma, unsigned int order, + gfp_t caller_gfp); bool cma_release_frozen(struct cma *cma, const struct page *pages, unsigned long count); =20 diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 49e5557d6cc0..3d23267014e6 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -714,7 +714,8 @@ struct folio *alloc_hugetlb_folio_nodemask(struct hstat= e *h, int preferred_nid, nodemask_t *nmask, gfp_t gfp_mask, bool allow_alloc_fallback); struct folio *alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_= nid, - nodemask_t *nmask, gfp_t gfp_mask); + nodemask_t *nmask, gfp_t gfp_mask, + bool *zeroed); =20 int hugetlb_add_to_page_cache(struct folio *folio, struct address_space *m= apping, pgoff_t idx); @@ -1134,7 +1135,8 @@ static inline void wait_for_freed_hugetlb_folios(void) =20 static inline struct folio * alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_nid, - nodemask_t *nmask, gfp_t gfp_mask) + nodemask_t *nmask, gfp_t gfp_mask, + bool *zeroed) { return NULL; } diff --git a/mm/cma.c b/mm/cma.c index c7ca567f4c5c..27971f6264ab 100644 --- a/mm/cma.c +++ b/mm/cma.c @@ -924,9 +924,11 @@ struct page *cma_alloc_frozen(struct cma *cma, unsigne= d long count, return __cma_alloc_frozen(cma, count, align, gfp); } =20 -struct page *cma_alloc_frozen_compound(struct cma *cma, unsigned int order) +struct page *cma_alloc_frozen_compound(struct cma *cma, unsigned int order, + gfp_t caller_gfp) { - gfp_t gfp =3D GFP_KERNEL | __GFP_COMP | __GFP_NOWARN; + gfp_t gfp =3D GFP_KERNEL | __GFP_COMP | __GFP_NOWARN | + (caller_gfp & __GFP_ZERO); =20 return __cma_alloc_frozen(cma, 1 << order, order, gfp); } diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 4ccf4eb91d92..649972101ace 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -2208,7 +2208,7 @@ struct folio *alloc_buddy_hugetlb_folio_with_mpol(str= uct hstate *h, } =20 struct folio *alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_= nid, - nodemask_t *nmask, gfp_t gfp_mask) + nodemask_t *nmask, gfp_t gfp_mask, bool *zeroed) { struct folio *folio; =20 @@ -2224,6 +2224,12 @@ struct folio *alloc_hugetlb_folio_reserve(struct hst= ate *h, int preferred_nid, h->resv_huge_pages--; =20 spin_unlock_irq(&hugetlb_lock); + + if (zeroed && folio) { + *zeroed =3D folio_test_hugetlb_zeroed(folio); + folio_clear_hugetlb_zeroed(folio); + } + return folio; } =20 @@ -2308,7 +2314,8 @@ static int gather_surplus_pages(struct hstate *h, lon= g delta) * It is okay to use NUMA_NO_NODE because we use numa_mem_id() * down the road to pick the current node if that is the case. */ - folio =3D alloc_surplus_hugetlb_folio(h, htlb_alloc_mask(h), + folio =3D alloc_surplus_hugetlb_folio(h, + htlb_alloc_mask(h), NUMA_NO_NODE, &alloc_nodemask, USER_ADDR_NONE); if (!folio) { diff --git a/mm/hugetlb_cma.c b/mm/hugetlb_cma.c index 7693ccefd0c6..c9266b25be3d 100644 --- a/mm/hugetlb_cma.c +++ b/mm/hugetlb_cma.c @@ -35,14 +35,14 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order,= gfp_t gfp_mask, return NULL; =20 if (hugetlb_cma[nid]) - page =3D cma_alloc_frozen_compound(hugetlb_cma[nid], order); + page =3D cma_alloc_frozen_compound(hugetlb_cma[nid], order, gfp_mask); =20 if (!page && !(gfp_mask & __GFP_THISNODE)) { for_each_node_mask(node, *nodemask) { if (node =3D=3D nid || !hugetlb_cma[node]) continue; =20 - page =3D cma_alloc_frozen_compound(hugetlb_cma[node], order); + page =3D cma_alloc_frozen_compound(hugetlb_cma[node], order, gfp_mask); if (page) break; } diff --git a/mm/memfd.c b/mm/memfd.c index fb425f4e315f..5518f7d2d91f 100644 --- a/mm/memfd.c +++ b/mm/memfd.c @@ -69,6 +69,7 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgoff= _t idx) #ifdef CONFIG_HUGETLB_PAGE struct folio *folio; gfp_t gfp_mask; + bool zeroed; =20 if (is_file_hugepages(memfd)) { /* @@ -93,17 +94,18 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgo= ff_t idx) folio =3D alloc_hugetlb_folio_reserve(h, numa_node_id(), NULL, - gfp_mask); + gfp_mask, + &zeroed); if (folio) { u32 hash; =20 /* - * Zero the folio to prevent information leaks to userspace. - * Use folio_zero_user() which is optimized for huge/gigantic - * pages. Pass 0 as addr_hint since this is not a faulting path - * and we don't have a user virtual address yet. + * Zero the folio to prevent information leaks to + * userspace. Skip if the pool page is known-zero + * (HPG_zeroed set during pool pre-allocation). */ - folio_zero_user(folio, 0); + if (!zeroed) + folio_zero_user(folio, 0); =20 /* * Mark the folio uptodate before adding to page cache, --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C4F953F23B1 for ; Fri, 29 May 2026 15:24:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068264; cv=none; b=NYPzlZMStDZoPAX/7oeHeCRx+PifQ9vD97tjUQ0vKk4Gzba6setp5eHfrOTcMKUFAYcO9rlwYETusHY4pN2B19rPCgW+vBLfXbc5H+MwccTc2yCWyTFmP+WtYen2qfGsep7AIHfLAFe1/uTiNGRsNWExDpyzhNVZ0EbE1UIlhyE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068264; c=relaxed/simple; bh=I7Z+iWN2H2agsUhhcouVK/58cmhoc5qiXmXbq5vVzSs=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=GZ6BKbLUYMmv88pWEuC2/tFnimgcb/4fRio8hi8wk+WsNlw3PXXp7R+lEb5w2/S0ahuqLa0+Dcn4ZTFNjumZ8Y8riab77oR4JWgDE/pTSUXHoyEu47zXS0kGgmjb3E3zcYXJQW+MpXUAwrLv1yjW6dmNim33gPHOgQWQSRUdwq0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=MrGi4OlM; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=dlYaQuQ7; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="MrGi4OlM"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="dlYaQuQ7" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068261; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=9A9ndauF873xtwqkChKCCeJTzSdyA513zveCjMygaEY=; b=MrGi4OlMb69JuM5tOzOvW0wIQdN2IneoUXpVw7bvrwj1QXanS79MMrmYCrrGEGaF5VN5fn 2brVTk9vc4Dq0Ahw2uzYqzheKyzWF9gGprsPkPrvi8XtYySqniIOA3e6gA8gMo6EN3odVV 7fZM5nL/ETLLTkTJkN3Gjcb4sprLebI= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-225-O4-UPMTANGOPMBvMjG5jWA-1; Fri, 29 May 2026 11:24:20 -0400 X-MC-Unique: O4-UPMTANGOPMBvMjG5jWA-1 X-Mimecast-MFC-AGG-ID: O4-UPMTANGOPMBvMjG5jWA_1780068259 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-45ed830a0eeso2797754f8f.3 for ; Fri, 29 May 2026 08:24:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068259; x=1780673059; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=9A9ndauF873xtwqkChKCCeJTzSdyA513zveCjMygaEY=; b=dlYaQuQ7E4E3h9rer2VDxh7XDMhPpI4jo/ycET9Bi+iMw9lQXUPpWKoJL2XcPTJLfA jMh8wJ0KszCrtDSaF0MhmNAsaEqVqOxxo+jPHVSar6XgLI0E3Eg+CzCzbePjiEE8/mX9 HIuEzFpE2lpoj/zG9JV4G9TDWUYMPctlZNof/nJAfR8FFQPv4d5k2LpHvQhWd+791Rxc SOB+0uAoQJDpU4n4zHrZ8FG4gj8oxFCtyV61Hv5wqjvFXtZH9Ijt+IXTFbJi7SNDP8/7 OrpvToNwuT4VbRn+OA3cannYkFV/J3ouHKqywV7lrk5iB+SA0LCS3OLIU/+nCPSfQ5MU eOfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068259; x=1780673059; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9A9ndauF873xtwqkChKCCeJTzSdyA513zveCjMygaEY=; b=nCpVdIeMgirt5Qh68yA6H2Nfb2rRiX+cXG4gBh8VfbYWoB8Xb3V4j05njeHoQaSBVe WqpZ4zh9Opx0EfDCRouRdI6W4O7JTdS1oSkN+Mg7RxQ3FOfI+bj0rOqZBc3Y87CUeowh 1gt47ZNplDGi58M8l9sqQ/te/M9D8MKOhr2NBPCwzTFgtTxfYcJiO0T9Gq3TV6ynE8Ql 1F+it8z6QRUCZMhsF9VhtkoGkQujAyxHY5JVs/AzhqFQ4ThnKppUcb59Pk1nzte+zBnU GKKa7EYanEiKXq6LWJrD91ziH7gLuuF8GC9gorSYZzhxxk7Dyq6ThybU6NmxJPgc4ZFQ POOQ== X-Gm-Message-State: AOJu0YwdMDGL531nR0oDRMlsaKh5MYqmvBH+X38LQKbtZBaG7erGDRSq CtDyhdlVorcSQ3vmCLopy7g3bvroCS7BXeaK/ttB3sypjI5oxd9oSaz+lD1CJaYM/MBAMFyqd8L vU6TiitiLdFpURryLqZMV5qysi1DliBJcDZkNiuZkaECPzcpBEnxgn5cCDwF0bZDzd+EWOURkIk 0ZsJiebQZAnd595BE1jhuokbYmqmiwU5BRRGi9kE1zJg0= X-Gm-Gg: Acq92OGjszntvtT2RJSwBcwLyEP/+90b5z4E6wROGwe+NUqq/Zg+pQs8do4JlVNUk3A YDJ+I19XHVIB1z3zmbZDJm+zMrWqfZreU/jqTrPExVsDfVUt7Vdwy1uIsXlvflvfzeTdS/xVhJO B5CaPsozgs7dPHnARIauDt0JfA0XtQbnwhTZOxtFjwRuIG7Fn1jSGHjYdt5dfTxIwkWTl5c8sGS ohxUcohjmeWU7R7OS8NGW63AQrCISySNhAw3Jd0tANJzZhxens/VhBZRJfE+VLORE7F8vdQ2uyO fXvJjrEMby0Pb3qv9tJ/qh4CbiKEjcow4KRsTHwsx2bVMVKlqh5sauEwC8AjWTxCxoNvQgDS7yc IXuTSMkN/L4oFegjOvr3bemiCWzHB+zUswdxILlQUztQgZCkEX+tRvQ== X-Received: by 2002:a05:600c:8582:b0:48f:d1c0:721e with SMTP id 5b1f17b1804b1-490a2921e1fmr1319085e9.12.1780068258733; Fri, 29 May 2026 08:24:18 -0700 (PDT) X-Received: by 2002:a05:600c:8582:b0:48f:d1c0:721e with SMTP id 5b1f17b1804b1-490a2921e1fmr1318075e9.12.1780068258076; Fri, 29 May 2026 08:24:18 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4909cab0e94sm51175135e9.12.2026.05.29.08.24.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:24:17 -0700 (PDT) Date: Fri, 29 May 2026 11:24:13 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 22/37] mm: page_reporting: skip redundant zeroing of host-zeroed reported pages Message-ID: <9c83fa52fe2e3cff6b14ea1819d8b4b6c6f7cb7c.1780067977.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When a guest reports free pages to the hypervisor via the page reporting framework (used by virtio-balloon and hv_balloon), the host typically zeros those pages when reclaiming their backing memory. However, when those pages are later allocated in the guest, post_alloc_hook() unconditionally zeros them again if __GFP_ZERO is set. This double-zeroing is wasteful, especially for large pages. Avoid redundant zeroing: - Add a host_zeroes_pages flag to page_reporting_dev_info, allowing drivers to declare that their host zeros reported pages on reclaim. A static key (page_reporting_host_zeroes) gates the fast path. - Add PG_zeroed page flag (sharing PG_private bit) to mark pages that have been zeroed by the host. Set it in page_reporting_drain() after the host reports them. - Thread the zeroed bool through rmqueue -> prep_new_page -> post_alloc_hook, where it skips redundant zeroing for __GFP_ZERO allocations. Currently the PG_zeroed hint can be lost when pages are split (expand) or merged in the buddy allocator. This is harmless: losing the hint just means the page gets re-zeroed, which is correct but suboptimal. Follow-up patches propagate PG_zeroed across splits and merges to preserve the hint on common paths. No driver sets host_zeroes_pages yet; a follow-up patch to virtio_balloon is needed to opt in. PG_zeroed pages may pass through PCP lists before being freed. This is safe: __free_pages_prepare clears all PAGE_FLAGS_CHECK_AT_PREP flags (including PG_zeroed/PG_private) before the page re-enters the buddy allocator. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- include/linux/page-flags.h | 9 +++++ include/linux/page_reporting.h | 3 ++ mm/compaction.c | 6 ++- mm/internal.h | 2 +- mm/page_alloc.c | 68 +++++++++++++++++++++++----------- mm/page_reporting.c | 14 ++++++- mm/page_reporting.h | 12 ++++++ 7 files changed, 88 insertions(+), 26 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 0e03d816e8b9..4ee64134acc3 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -135,6 +135,8 @@ enum pageflags { PG_swapcache =3D PG_owner_priv_1, /* Swap page: swp_entry_t in private */ /* Some filesystems */ PG_checked =3D PG_owner_priv_1, + /* Page contents are known to be zero */ + PG_zeroed =3D PG_private, =20 /* * Depending on the way an anonymous folio can be mapped into a page @@ -673,6 +675,13 @@ FOLIO_TEST_CLEAR_FLAG_FALSE(young) FOLIO_FLAG_FALSE(idle) #endif =20 +/* + * PageZeroed() tracks pages known to be zero. The allocator + * uses this to skip redundant zeroing in post_alloc_hook(). + */ +__PAGEFLAG(Zeroed, zeroed, PF_NO_COMPOUND) +#define __PG_ZEROED (1UL << PG_zeroed) + /* * PageReported() is used to track reported free pages within the Buddy * allocator. We can use the non-atomic version of the test and set diff --git a/include/linux/page_reporting.h b/include/linux/page_reporting.h index 5ab5be02fa15..c331c6b36687 100644 --- a/include/linux/page_reporting.h +++ b/include/linux/page_reporting.h @@ -14,6 +14,9 @@ struct page_reporting_dev_info { int (*report)(struct page_reporting_dev_info *prdev, struct scatterlist *sg, unsigned int nents); =20 + /* If true, host zeros reported pages on reclaim */ + bool host_zeroes_pages; + /* work struct for processing reports */ struct delayed_work work; =20 diff --git a/mm/compaction.c b/mm/compaction.c index 4336e433c99b..8000fc5e0a2e 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -82,7 +82,8 @@ static inline bool is_via_compact_memory(int order) { ret= urn false; } =20 static struct page *mark_allocated_noprof(struct page *page, unsigned int = order, gfp_t gfp_flags) { - post_alloc_hook(page, order, __GFP_MOVABLE, USER_ADDR_NONE); + __ClearPageZeroed(page); + post_alloc_hook(page, order, __GFP_MOVABLE, false, USER_ADDR_NONE); set_page_refcounted(page); return page; } @@ -1849,9 +1850,10 @@ static struct folio *compaction_alloc_noprof(struct = folio *src, unsigned long da set_page_private(&freepage[size], start_order); } dst =3D (struct folio *)freepage; + __ClearPageZeroed(&dst->page); if (order) prep_compound_page(&dst->page, order); - post_alloc_hook(&dst->page, order, __GFP_MOVABLE, USER_ADDR_NONE); + post_alloc_hook(&dst->page, order, __GFP_MOVABLE, false, USER_ADDR_NONE); set_page_refcounted(&dst->page); cc->nr_freepages -=3D 1 << order; cc->nr_migratepages -=3D 1 << order; diff --git a/mm/internal.h b/mm/internal.h index 389098200aa6..fd910743ddc3 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -928,7 +928,7 @@ static inline void init_compound_tail(struct page *tail, } =20 void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flag= s, - unsigned long user_addr); + bool zeroed, unsigned long user_addr); extern bool free_pages_prepare(struct page *page, unsigned int order); =20 extern int user_min_free_kbytes; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index ad0655387e9d..5158f7e23d18 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1746,6 +1746,7 @@ static __always_inline void page_del_and_expand(struc= t zone *zone, bool was_reported =3D page_reported(page); =20 __del_page_from_free_list(page, zone, high, migratetype); + nr_pages -=3D expand(zone, page, low, high, migratetype, was_reported); account_freepages(zone, -nr_pages, migratetype); } @@ -1818,8 +1819,10 @@ static inline bool should_skip_init(gfp_t flags) return (flags & __GFP_SKIP_ZERO); } =20 + inline void post_alloc_hook(struct page *page, unsigned int order, - gfp_t gfp_flags, unsigned long user_addr) + gfp_t gfp_flags, bool zeroed, + unsigned long user_addr) { bool init =3D !want_init_on_free() && want_init_on_alloc(gfp_flags) && !should_skip_init(gfp_flags); @@ -1828,6 +1831,14 @@ inline void post_alloc_hook(struct page *page, unsig= ned int order, =20 set_page_private(page, 0); =20 + /* + * If the page is zeroed, skip memory initialization. + * We still need to handle tag zeroing separately since the host + * does not know about memory tags. + */ + if (zeroed && init && !zero_tags) + init =3D false; + arch_alloc_page(page, order); debug_pagealloc_map_pages(page, 1 << order); =20 @@ -1870,7 +1881,7 @@ inline void post_alloc_hook(struct page *page, unsign= ed int order, * through a user-congruent mapping. Host-zeroed pages * (zeroed flag) don't need this: physical RAM is clean. */ - if (!init && (gfp_flags & __GFP_ZERO) && + if (!zeroed && !init && (gfp_flags & __GFP_ZERO) && user_addr !=3D USER_ADDR_NONE && user_alloc_needs_zeroing()) init =3D true; @@ -1903,13 +1914,13 @@ inline void post_alloc_hook(struct page *page, unsi= gned int order, } =20 static void prep_new_page(struct page *page, unsigned int order, gfp_t gfp= _flags, - unsigned int alloc_flags, - unsigned long user_addr) + unsigned int alloc_flags, bool zeroed, + unsigned long user_addr) { if (order && (gfp_flags & __GFP_COMP)) prep_compound_page(page, order); =20 - post_alloc_hook(page, order, gfp_flags, user_addr); + post_alloc_hook(page, order, gfp_flags, zeroed, user_addr); =20 /* * page is set pfmemalloc when ALLOC_NO_WATERMARKS was necessary to @@ -3175,6 +3186,7 @@ int __isolate_free_page(struct page *page, unsigned i= nt order) } =20 del_page_from_free_list(page, zone, order, mt); + __ClearPageZeroed(page); =20 /* * Set the pageblock if the isolated page is at least half of a @@ -3247,7 +3259,7 @@ static inline void zone_statistics(struct zone *prefe= rred_zone, struct zone *z, static __always_inline struct page *rmqueue_buddy(struct zone *preferred_zone, struct zone *zone, unsigned int order, unsigned int alloc_flags, - int migratetype) + int migratetype, bool *zeroed) { struct page *page; unsigned long flags; @@ -3282,6 +3294,8 @@ struct page *rmqueue_buddy(struct zone *preferred_zon= e, struct zone *zone, } } spin_unlock_irqrestore(&zone->lock, flags); + *zeroed =3D PageZeroed(page); + __ClearPageZeroed(page); } while (check_new_pages(page, order)); =20 /* @@ -3350,10 +3364,9 @@ static int nr_pcp_alloc(struct per_cpu_pages *pcp, s= truct zone *zone, int order) /* Remove page from the per-cpu list, caller must protect the list */ static inline struct page *__rmqueue_pcplist(struct zone *zone, unsigned int order, - int migratetype, - unsigned int alloc_flags, + int migratetype, unsigned int alloc_flags, struct per_cpu_pages *pcp, - struct list_head *list) + struct list_head *list, bool *zeroed) { struct page *page; =20 @@ -3388,6 +3401,8 @@ struct page *__rmqueue_pcplist(struct zone *zone, uns= igned int order, page =3D list_first_entry(list, struct page, pcp_list); list_del(&page->pcp_list); pcp->count -=3D 1 << order; + *zeroed =3D PageZeroed(page); + __ClearPageZeroed(page); } while (check_new_pages(page, order)); =20 return page; @@ -3396,7 +3411,8 @@ struct page *__rmqueue_pcplist(struct zone *zone, uns= igned int order, /* Lock and remove page from the per-cpu list */ static struct page *rmqueue_pcplist(struct zone *preferred_zone, struct zone *zone, unsigned int order, - int migratetype, unsigned int alloc_flags) + int migratetype, unsigned int alloc_flags, + bool *zeroed) { struct per_cpu_pages *pcp; struct list_head *list; @@ -3414,7 +3430,8 @@ static struct page *rmqueue_pcplist(struct zone *pref= erred_zone, */ pcp->free_count >>=3D 1; list =3D &pcp->lists[order_to_pindex(migratetype, order)]; - page =3D __rmqueue_pcplist(zone, order, migratetype, alloc_flags, pcp, li= st); + page =3D __rmqueue_pcplist(zone, order, migratetype, alloc_flags, + pcp, list, zeroed); pcp_spin_unlock(pcp); if (page) { __count_zid_vm_events(PGALLOC, page_zonenum(page), 1 << order); @@ -3439,19 +3456,19 @@ static inline struct page *rmqueue(struct zone *preferred_zone, struct zone *zone, unsigned int order, gfp_t gfp_flags, unsigned int alloc_flags, - int migratetype) + int migratetype, bool *zeroed) { struct page *page; =20 if (likely(pcp_allowed_order(order))) { page =3D rmqueue_pcplist(preferred_zone, zone, order, - migratetype, alloc_flags); + migratetype, alloc_flags, zeroed); if (likely(page)) goto out; } =20 page =3D rmqueue_buddy(preferred_zone, zone, order, alloc_flags, - migratetype); + migratetype, zeroed); =20 out: /* Separate test+clear to avoid unnecessary atomics */ @@ -3842,6 +3859,7 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int o= rder, int alloc_flags, struct pglist_data *last_pgdat =3D NULL; bool last_pgdat_dirty_ok =3D false; bool no_fallback; + bool zeroed; bool skip_kswapd_nodes =3D nr_online_nodes > 1; bool skipped_kswapd_nodes =3D false; =20 @@ -3986,10 +4004,11 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int= order, int alloc_flags, =20 try_this_zone: page =3D rmqueue(zonelist_zone(ac->preferred_zoneref), zone, order, - gfp_mask, alloc_flags, ac->migratetype); + gfp_mask, alloc_flags, ac->migratetype, + &zeroed); if (page) { prep_new_page(page, order, gfp_mask, alloc_flags, - ac->user_addr); + zeroed, ac->user_addr); =20 return page; } else { @@ -4216,9 +4235,11 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigne= d int order, count_vm_event(COMPACTSTALL); =20 /* Prep a captured page if available */ - if (page) - prep_new_page(page, order, gfp_mask, alloc_flags, + if (page) { + __ClearPageZeroed(page); + prep_new_page(page, order, gfp_mask, alloc_flags, false, ac->user_addr); + } =20 /* Try get a page from the freelist if available */ if (!page) @@ -5191,6 +5212,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, /* Attempt the batch allocation */ pcp_list =3D &pcp->lists[order_to_pindex(ac.migratetype, 0)]; while (nr_populated < nr_pages) { + bool zeroed =3D false; =20 /* Skip existing pages */ if (page_array[nr_populated]) { @@ -5199,7 +5221,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, } =20 page =3D __rmqueue_pcplist(zone, 0, ac.migratetype, alloc_flags, - pcp, pcp_list); + pcp, pcp_list, &zeroed); if (unlikely(!page)) { /* Try and allocate at least one page */ if (!nr_account) { @@ -5210,7 +5232,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, } nr_account++; =20 - prep_new_page(page, 0, gfp, 0, USER_ADDR_NONE); + prep_new_page(page, 0, gfp, 0, zeroed, USER_ADDR_NONE); set_page_refcounted(page); page_array[nr_populated++] =3D page; } @@ -6950,7 +6972,8 @@ static void split_free_frozen_pages(struct list_head = *list, gfp_t gfp_mask) list_for_each_entry_safe(page, next, &list[order], lru) { int i; =20 - post_alloc_hook(page, order, gfp_mask, USER_ADDR_NONE); + __ClearPageZeroed(page); + post_alloc_hook(page, order, gfp_mask, false, USER_ADDR_NONE); if (!order) continue; =20 @@ -7156,8 +7179,9 @@ static int __alloc_contig_frozen_range(unsigned long = start, unsigned long end, } else if (start =3D=3D outer_start && end =3D=3D outer_end && is_power_o= f_2(end - start)) { struct page *head =3D pfn_to_page(start); =20 + __ClearPageZeroed(head); check_new_pages(head, order); - prep_new_page(head, order, gfp_mask, 0, user_addr); + prep_new_page(head, order, gfp_mask, 0, false, user_addr); } else { ret =3D -EINVAL; WARN(true, "PFN range: requested [%lu, %lu), allocated [%lu, %lu)\n", diff --git a/mm/page_reporting.c b/mm/page_reporting.c index a6ddf6fafc20..8d52718be876 100644 --- a/mm/page_reporting.c +++ b/mm/page_reporting.c @@ -50,6 +50,8 @@ EXPORT_SYMBOL_GPL(page_reporting_order); #define PAGE_REPORTING_DELAY (2 * HZ) static struct page_reporting_dev_info __rcu *pr_dev_info __read_mostly; =20 +DEFINE_STATIC_KEY_FALSE(page_reporting_host_zeroes); + enum { PAGE_REPORTING_IDLE =3D 0, PAGE_REPORTING_REQUESTED, @@ -129,8 +131,11 @@ page_reporting_drain(struct page_reporting_dev_info *p= rdev, * report on the new larger page when we make our way * up to that higher order. */ - if (PageBuddy(page) && buddy_order(page) =3D=3D order) + if (PageBuddy(page) && buddy_order(page) =3D=3D order) { __SetPageReported(page); + if (page_reporting_host_zeroes_pages()) + __SetPageZeroed(page); + } } while ((sg =3D sg_next(sg))); =20 /* reinitialize scatterlist now that it is empty */ @@ -390,6 +395,10 @@ int page_reporting_register(struct page_reporting_dev_= info *prdev) /* Assign device to allow notifications */ rcu_assign_pointer(pr_dev_info, prdev); =20 + /* enable zeroed page optimization if host zeroes reported pages */ + if (prdev->host_zeroes_pages) + static_branch_enable(&page_reporting_host_zeroes); + /* enable page reporting notification */ if (!static_key_enabled(&page_reporting_enabled)) { static_branch_enable(&page_reporting_enabled); @@ -414,6 +423,9 @@ void page_reporting_unregister(struct page_reporting_de= v_info *prdev) =20 /* Flush any existing work, and lock it out */ cancel_delayed_work_sync(&prdev->work); + + if (prdev->host_zeroes_pages) + static_branch_disable(&page_reporting_host_zeroes); } =20 mutex_unlock(&page_reporting_mutex); diff --git a/mm/page_reporting.h b/mm/page_reporting.h index c51dbc228b94..736ea7b37e9e 100644 --- a/mm/page_reporting.h +++ b/mm/page_reporting.h @@ -15,6 +15,13 @@ DECLARE_STATIC_KEY_FALSE(page_reporting_enabled); extern unsigned int page_reporting_order; void __page_reporting_notify(void); =20 +DECLARE_STATIC_KEY_FALSE(page_reporting_host_zeroes); + +static inline bool page_reporting_host_zeroes_pages(void) +{ + return static_branch_unlikely(&page_reporting_host_zeroes); +} + static inline bool page_reported(struct page *page) { return static_branch_unlikely(&page_reporting_enabled) && @@ -46,6 +53,11 @@ static inline void page_reporting_notify_free(unsigned i= nt order) #else /* CONFIG_PAGE_REPORTING */ #define page_reported(_page) false =20 +static inline bool page_reporting_host_zeroes_pages(void) +{ + return false; +} + static inline void page_reporting_notify_free(unsigned int order) { } --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9F588423A7C for ; Fri, 29 May 2026 15:24:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068272; cv=none; b=MJgB5YL3RHTO8U0P5YNtvrZmB553UBJIXhO8ShLJ8a62wPbxxTBuElF/1w8Krpy4xgysTcF4vq+6Nc+TfqC03kFnffN4o25sG8h2H/RUDCTVRaaYQBPbVVtShZG4oAaPYEQAGKq6cW3YuP8mTkSZ7ZNK+fuvu1sPANKRJzei06g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068272; c=relaxed/simple; bh=9yxmheTwtEMdpXjxhcMq1i9/4nCbH4yqJLWMjAN8B0Y=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=sbWNtgqoPluYjqhBBZfxTHLpcWqeSXYP/cJvQpH0Y7q4bD1c8XRl0pJf3pnU0IXIoEXyyOzgt3z5llKdBY6L6e20ORNIMlxYOh3zOVGSxkQqtThtFtYcbJ7U6lw8HE9WEO4e4T74BqST0dWK9e5vYRUMvOQFH5HLbDR8+OFgy74= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=c+JpZ81L; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=H5gUySYk; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="c+JpZ81L"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="H5gUySYk" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068267; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=8dg7l+XMBUZMVnB4+qTLsym8fNCo7JvPxwQL0Jq5Z3w=; b=c+JpZ81LIvP0qgDnz/QYWb8+vAS4c7FijKF1xgtvSsch9zleXlo08cFxMbwtTV/wlXGgK1 +/avlr6aGHJ4f+UEtis1MFMuSBgdhXmO8cWYtd9AsyMzJCapgdLWG7TAjeFuzSSPZhRTyq P7NTryoUmajd5+XaBFRCdjHxOKysOfY= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-668-e09ult4jMS2c6sWEe7us1A-1; Fri, 29 May 2026 11:24:25 -0400 X-MC-Unique: e09ult4jMS2c6sWEe7us1A-1 X-Mimecast-MFC-AGG-ID: e09ult4jMS2c6sWEe7us1A_1780068264 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-44696b11265so16693332f8f.0 for ; Fri, 29 May 2026 08:24:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068264; x=1780673064; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=8dg7l+XMBUZMVnB4+qTLsym8fNCo7JvPxwQL0Jq5Z3w=; b=H5gUySYk+xa3k4Wm4aO51vHq8+pFx3zvh1JKWJnWM/hXDZcidGZrQv0oHcuV55r6x0 QV+jTRBBrJGr7euKaSvFT1ZohAptgk9Lj7hPKhvvfxKUntHpQazH5ktCcn8U7hVOBgwE vmw0uw4vNk4z603ekBYL0x+8JiLRI9qHJjFcZKbNd8gOKH6UCSatZUZZZggFx9gZVKKy vCXz7+lQolgpSkG20QOpPyYxFONoLSRv4e3u5okMALBkG5hjeOZ4NqFRtcZlJdWkA3kq DQk3RsVmIEu58whJwJxV6OaMO84Z1dDTdLhSH2QZq0QVUekO8J6gObdHDE8g1Gmsz9Et 3XWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068264; x=1780673064; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8dg7l+XMBUZMVnB4+qTLsym8fNCo7JvPxwQL0Jq5Z3w=; b=krU4jA6zwOW+jTBEUFGx8QhEK9a9FpaKK5zBfpivqdWAAwVgWTxH8tRJTngjbCiNNf td5mp9VisGoYd2KKT5sonFPE9BtaVQvMwAPeBbj2RG+c5fWRylbhMPUccP8KH+/sUsdc FQmVblHaRYD1WvNWT5Hdj9Vu1vLrzSO7gb/P6eHiGEQ7l52wx9xcFgFYPa7CzuJ+LmnX ejCkx9xxw8TWuTBfnRSd2ypydv66xCNei7PrrXTUiaCChjBM/pWxt+A10qVlqUIqUm92 ncFbpO/iLdEsZei7GaWbbMplYHA4hZ9B/N6jV6lu2ieomxavuIVjNzQxxMWO4RauRxhz a6uw== X-Gm-Message-State: AOJu0YxyGDBbwSFNHWe5ZqgfJ7uTG1nl3ldtY8JeITAwp/1GGyOjDdnY AGe/6pnIists0n/kPOEaFFZsEdlaeJ9DojYea+N0RCaYEi8U99TF4xhgq+DTe/kY3p8lHh186Rz GRPpMMEC8fVpbjyo+U22/1X6M7bOWMEzbmbeYTJWkb4N3/jupT0w+jOwT4fVKhZJi+j7SMhsSQu CJ6X6TXD5WxJ35IJafA72lr7t56XpQBdLZsTZK/dHkffw= X-Gm-Gg: Acq92OGLjdXuQc39M4PF6Rt5d5wlFSD89JeCVK+FaZ6f+mZRmFJ+PIIZJb19pQ+pr4x 8XI1TkFYLjW0iDVFooTSKyHgO4nB417DIHgGROAkFxE6ff0+Ub1izTKwCDIDSBDy9T6OSdUeNSP /9kNJHk8t58pEZPxgMB8cf3VHgqw4LHM7M4D1BL2lKHyltcHzWPWV61CydGmusF67uAlxJ3A03x UXGicCML2qnU9EY/FsD+6wpETty2cZzQbYJ+aBqRlRdukcfb8Xzz111zjFuEx8qWuTbEMvjJwF1 XuPHFQrHVrr5b8hmggiHEuIzP/ba82mgkLJt/BPA/82R7dkoj3quVTJX02Om4eCTXb19spCpCdE Jr2mTLkk0L5iHZ+Le2ADGqa0csjwxqRpKODKkhrksK6UpV91AvS80gA== X-Received: by 2002:adf:ea85:0:b0:45e:eec6:5023 with SMTP id ffacd0b85a97d-45ef6b8831dmr324588f8f.37.1780068263732; Fri, 29 May 2026 08:24:23 -0700 (PDT) X-Received: by 2002:adf:ea85:0:b0:45e:eec6:5023 with SMTP id ffacd0b85a97d-45ef6b8831dmr324493f8f.37.1780068262980; Fri, 29 May 2026 08:24:22 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45ef34b834esm4139592f8f.11.2026.05.29.08.24.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:24:22 -0700 (PDT) Date: Fri, 29 May 2026 11:24:18 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 23/37] mm: page_alloc: use aliasing checks instead of user_alloc_needs_zeroing Message-ID: <8229023108f900d50b47bd85b5a8f8eb5f95ef03.1780067977.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Replace user_alloc_needs_zeroing() with the direct aliasing checks (cpu_dcache_is_aliasing() || cpu_icache_is_aliasing()) in the post_alloc_hook aliasing guard. user_alloc_needs_zeroing() includes a !init_on_alloc term that means "allocator didn't zero this page." But in this guard's context (!zeroed && !init && __GFP_ZERO), we already know the page is zero; init incorporates init_on_alloc via want_init_on_alloc(). The only question left is whether the cache architecture needs the data re-zeroed through a congruent mapping, which is purely cpu_dcache_is_aliasing() || cpu_icache_is_aliasing(). On non-aliasing architectures with init_on_free=3Dtrue and init_on_alloc=3Dfalse, this avoids a redundant re-zero of an already-zero page. Note on PowerPC: PowerPC overrides clear_user_page to call flush_dcache_page after clear_page, but on freshly allocated pages PG_dcache_clean is already clear (cleared by __free_pages_prepare), so flush_dcache_page is a no-op. Skipping this here thus has no effect. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/page_alloc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 5158f7e23d18..4cb7e779a6c5 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1883,7 +1883,7 @@ inline void post_alloc_hook(struct page *page, unsign= ed int order, */ if (!zeroed && !init && (gfp_flags & __GFP_ZERO) && user_addr !=3D USER_ADDR_NONE && - user_alloc_needs_zeroing()) + (cpu_dcache_is_aliasing() || cpu_icache_is_aliasing())) init =3D true; /* * If memory is still not initialized, initialize it now. --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3F0F93F1AC0 for ; Fri, 29 May 2026 15:24:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068276; cv=none; b=NjXEQj/EXwoUbewzVK22cUn0udo9DsmNslGHlzTNlhiCpn6njEztuIHj97yAFTshTq2bN5Tz7NxJE34o+eyZuuY3lVMmPhry7Cm+AdEdHq3OPp9dXejWXeHeqADQ1YtYMCN5+3DcNnKo+BpX9hSjTfWshA9nPRElvjocG7Uq6s4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068276; c=relaxed/simple; bh=b0H6igos+bxXZv2CiE0eAIuacEUkYyhOa0XUsWe315U=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Nihv+Dr5YX5nN9C3pZPz2tCSJI7S2zMZBGE/3igFPdnr8tCwsICcHgUQLVhlkmhkaCguPSRfVoxDlW0fTkvrimhz72vIova1pnST0hzG0DUBBmbeKC5KhuN7lFbDtFEZv7+TrZA3c01ba14qGY8SybCdKHw1bocbzAFgxhdhNB4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=UWmQM206; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=jQOKvlWD; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="UWmQM206"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="jQOKvlWD" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068272; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=IWNxSxbqwby36tOoHcOEDoEnbpMh6i49D1nHoSwbp8U=; b=UWmQM206VpkB7lIEytEYfxSrALVVTYdmIY3983BLAWLuYyMf7jVfMwfCDa+MdaAkPrZFkz RYah/KpJQTlprTYszbnlECoXtUs3v1lPNMosZuWN57/tIVK689B5VOi4O2+5QPswuY5o+S i1Q8H8/fxnzLBzVqYdhQi8SyHpxnOgg= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-694-kQ6TJYcCNtanF_hIgcJ5KQ-1; Fri, 29 May 2026 11:24:30 -0400 X-MC-Unique: kQ6TJYcCNtanF_hIgcJ5KQ-1 X-Mimecast-MFC-AGG-ID: kQ6TJYcCNtanF_hIgcJ5KQ_1780068269 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-4904ee02e72so60972835e9.1 for ; Fri, 29 May 2026 08:24:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068269; x=1780673069; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=IWNxSxbqwby36tOoHcOEDoEnbpMh6i49D1nHoSwbp8U=; b=jQOKvlWDTadiWdksJLPouCQLexoAM4MnZ8B59WLap89k+qt27QKSoVfPpie4kFPU+A e1uy74CFSA6nqU8F1rGEf1xJzOkS9ovHiKyvN7suMeiB6wxoo7765q1ggNz4La4SB/fC aoIiwCmfz085av9mWq/gQwmPd+Fi0wn9A5x+ki73dzq9P+UmBz4lut3EmBVW8FjLNzsP QDKAO6m01RHWR4iR/8spF3VZSYOs5Fm4TPpk1aN6Bh3alvDUkMxwJF5ihDSECAIKcH9+ PmUng73mWboVAjdJnt1C2L7R1faYMQ4mursPYZcNbw4/5p1VA7eEf8O115KeAMsLf5OA nd0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068269; x=1780673069; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IWNxSxbqwby36tOoHcOEDoEnbpMh6i49D1nHoSwbp8U=; b=AGgMlBxtqWawxUe8qvaH2tor2E2+ur1ppxVfzaZAt13aQdaD9HTKbYiHH4gb+I9WNc kF1tTtJZHx0DsqFZ2TeEf9OU6Uvo3kKt9Z3CIsoZyOUUmNtZLcqbSk2SRRDhxngXJVrv 90XdKzuv7vF2TkWbI6LKE0eT02Y42lnOwOi8sC8NLpkHSB28D9WIfcqjggOFGgy0IGYw uxKFrSyeMro+lFPo6YgKjBcwI886Kfo5Kd+8Wg1pme60UVkqSh/pdw0iu6G3+ZEWGanf hTnZwKhPHvoFOaM1r5RTsZ+58Wqu3K9j+pNJ64sCwct1GQVoC/z4v2yLNVqqfHB34MTr KEEQ== X-Gm-Message-State: AOJu0Yz2v2C9Mmjf8m7l9khdvxg3dY1XRyMvf3Ld67KuR26jr3hcchQ/ IJN7UfzLNQcotevdhXkW2xtvrp51bqlaHXW1g5p7D8qjIftJmTJRjrR2z/zXgCHO84x/6vkmOTf NQBCkVVMVkmFL7gHMSXnS+L/m+6QByv91iu3FfpyE2Zi4nEJqc+L+14dCWIjtI3ldty3ZL2h4W6 /PNXsjGD6H/gGG/+I/BolbpWMFpyPQnKIvm2JGdnqXrAM= X-Gm-Gg: Acq92OEYPn6lWBJ3vY9nBpbsVqiawj+gJ/ptrI5QrRxq4BImrYnzv8qrNXBwNRAeqr3 UOy/xAjvDZqCTiuP4ZY26oird1UHdQ3kxehecdUFeDDdMO+kiEIjTTBYKCeGAXe5mAQm48r9KGK L9i18U3GVmVSUzFDgvtRjPBVVRuL4XLwhFXXPStLfXBNUk+d6b4y9ZsDgxTF0T2Wri8mpU9kXQX H0LJ0/fMuQYKQCfV2c8WTN94OggFh7rIxQSgsJ7qOpFJwZGDIx7GRmDfr6pNmBMkVuzm7Cc6XHp YnhXvqZmV5sG9vjlNg3m7XpPK3CRGxn89a16+IRxQkYKxQxk1P0ZeipGqu8SNflIgaNeP1SXnIN s38VsQBgNlqCFMqsGinQZazYTmhro5qJZWLVW1SOxAP4r0Ngc6MBZSQ== X-Received: by 2002:a05:600d:11:b0:490:3d62:f5e1 with SMTP id 5b1f17b1804b1-490a2963b8emr1049005e9.22.1780068268721; Fri, 29 May 2026 08:24:28 -0700 (PDT) X-Received: by 2002:a05:600d:11:b0:490:3d62:f5e1 with SMTP id 5b1f17b1804b1-490a2963b8emr1047575e9.22.1780068268015; Fri, 29 May 2026 08:24:28 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4909caa7e17sm56228165e9.9.2026.05.29.08.24.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:24:27 -0700 (PDT) Date: Fri, 29 May 2026 11:24:23 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 24/37] mm: page_reporting: add per-page zeroed bitmap for host feedback Message-ID: <217e59b7f6693a0cddd62158be80dcfb12cb879a.1780067978.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The host may skip zeroing some reported pages (e.g., due to alignment constraints or bounce buffer fallback in QEMU). Currently, when host_zeroes_pages is set, all reported pages are unconditionally marked PG_zeroed - even ones the host did not actually zero. Add a zeroed_bitmap to page_reporting_dev_info that the report() callback can use to indicate which pages were actually zeroed. The driver's report() callback is responsible for managing the bitmap: zeroing it before sending pages to the host, then setting bits for pages the host actually zeroed. page_reporting_drain() checks the bitmap per-page in addition to the global host_zeroes_pages flag. No driver sets host_zeroes_pages yet, so the static key is off and the bitmap is never read. Behavior is unchanged. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- include/linux/page_reporting.h | 7 +++++++ mm/page_reporting.c | 8 ++++++-- 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/include/linux/page_reporting.h b/include/linux/page_reporting.h index c331c6b36687..e2e6a487ddab 100644 --- a/include/linux/page_reporting.h +++ b/include/linux/page_reporting.h @@ -17,6 +17,13 @@ struct page_reporting_dev_info { /* If true, host zeros reported pages on reclaim */ bool host_zeroes_pages; =20 + /* + * Per-page zeroed status, indexed by scatterlist position. + * The driver's report() callback must clear the bitmap, + * then set bits for pages that were actually zeroed. + */ + DECLARE_BITMAP(zeroed_bitmap, PAGE_REPORTING_CAPACITY); + /* work struct for processing reports */ struct delayed_work work; =20 diff --git a/mm/page_reporting.c b/mm/page_reporting.c index 8d52718be876..f68c79e57427 100644 --- a/mm/page_reporting.c +++ b/mm/page_reporting.c @@ -108,6 +108,7 @@ page_reporting_drain(struct page_reporting_dev_info *pr= dev, struct scatterlist *sgl, unsigned int nents, bool reported) { struct scatterlist *sg =3D sgl; + unsigned int i =3D 0; =20 /* * Drain the now reported pages back into their respective @@ -122,7 +123,7 @@ page_reporting_drain(struct page_reporting_dev_info *pr= dev, =20 /* If the pages were not reported due to error skip flagging */ if (!reported) - continue; + goto next; =20 /* * If page was not commingled with another page we can @@ -133,9 +134,12 @@ page_reporting_drain(struct page_reporting_dev_info *p= rdev, */ if (PageBuddy(page) && buddy_order(page) =3D=3D order) { __SetPageReported(page); - if (page_reporting_host_zeroes_pages()) + if (page_reporting_host_zeroes_pages() && + test_bit(i, prdev->zeroed_bitmap)) __SetPageZeroed(page); } +next: + i++; } while ((sg =3D sg_next(sg))); =20 /* reinitialize scatterlist now that it is empty */ --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 224D0426ED3 for ; Fri, 29 May 2026 15:24:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068281; cv=none; b=pB30PH5UuDzyY2gJauxBYqG8HjtXMk0Numl7KylbkLMA1034enu5oIinvNZTm9fcqYM+uzY+LaTKP5lUDXIamzNaAEM+RTVPdpxtdE2ras/9+MqJj8dE42THpsCH0z3CheW637KJcL1MK+eVeJBR4vsRR1ppXBE023Bd9Nci0Yg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068281; c=relaxed/simple; bh=c6xJpyp5zpjmRc9jnwYRt4nCMF1SinUlHasxCFwTPkI=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=iuNTwidtTN5+5o8PycBWE07qubx7bUw28F74qQHbSs+1o6UUS/6IJhSANVlrMVLSYg5K4Y0Hj9wxndQTkg1hE1EMIhNtdhgSJ9NSLSkiUBUNQxhFm6DtKqhTGzwQfZOSzCWnCpjRFGLyLNIDFyYx5irApEIWjNmjBQRSjvbhUF4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=hfvNjctb; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=CJjg1iwJ; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="hfvNjctb"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="CJjg1iwJ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068276; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=qdOI4mS59hNGv2oV9kd67xyAhp7b5Wts0l8Olu5Azv0=; b=hfvNjctbYxLYHahl6/MCCHR/3J0VLEB6LHLJf8QO7BvNbifne87s10XGiupjI3Og//LIPh +3JfG1O4LlFmCPZEZXott8qWegFs3SnF0ySmH0HpMXytYgtNOmF+HMlYsAJ5G2VxzupRpK 88khCjw5LhkxxeJ+Fo1Amm3IunMy2ck= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-473-w15L5dgtOWa8KrFFBuB9_w-1; Fri, 29 May 2026 11:24:35 -0400 X-MC-Unique: w15L5dgtOWa8KrFFBuB9_w-1 X-Mimecast-MFC-AGG-ID: w15L5dgtOWa8KrFFBuB9_w_1780068274 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-4903dcb32f8so69898845e9.0 for ; Fri, 29 May 2026 08:24:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068274; x=1780673074; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=qdOI4mS59hNGv2oV9kd67xyAhp7b5Wts0l8Olu5Azv0=; b=CJjg1iwJzGxHtocFKWJfrPey4g3o44lSdJUSgvYPVUYXCk0zbiYgDSCGIpwF9hSM4r knoSoB6bdu+q8dkI7q8JP++FroKqZ3pMCaM4/1jPU3DQeB7QGq3IaLSGwyFwhEKJlL9B ivyHpi1coc2DhRQzCkgoMlfQzmyI/t9KPw6dHjZ0eOd/Z0FTx50wwDcb5BKweM617v3k rIB7vwLewUnHaE7vGsXZ8KyM3ORHJfxANNDI9BY6wiS/qN6U8E2XIF1IxI2Uixtyhv2P LICRN9518Hj4qGt4WHanPKzS74vcV1sPwTetmMR/TwaNdx+JiCnguIEgakOTq1Mdzvwq AkTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068274; x=1780673074; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qdOI4mS59hNGv2oV9kd67xyAhp7b5Wts0l8Olu5Azv0=; b=Qt/5MPONe027sGP0Tx1i8cMx+pvBtKPwBGdnHQluXdoRcJ+nPqOxKxMCA42MLVxFPr FzFVF/Wq7EOmKl7vqe9UFV4eRT4xDipVpcpkMBGZqqjyVkN65CmgeGxoqlUcnpicNuUr IkJ7w3DzVKacOhaTrBJOIraZOy78NIChUhRvONtEAgc8bOkMZoPyIa8PLxDDwq7tXdXp GQ9no1XpwxeP1alIuCk0MAmkr8lzxutqou8MSljRDSAklfPwkQa/8OozGH2g/va+Bp1C yLMtoybojWJF1ZPzYaiZyRfqFQqUGxT8QYsOHNWk6rT6Z1Ei+xe4gDn+HhBtcBIJO3Sn NGow== X-Gm-Message-State: AOJu0YxLXytEfV9CAX8aUbA6GVlutWWWSKveEJ1Zkz64uQAhX6I1fsGU bjsBgfRNlxPXS1N6T/bSyuc2LaxyJNRTLY4HcRfzUbb0t0je5LbiBNxi+2mLNSruGEwnCoLopKO Q7zb3IiZwMxuJFrBuK70SUHS+ggU6Q1DLcalvhZt8eiKBNdIUThGEASeoC0W4AyLkcKc89uXs+b /YEgKpz9+EG7uhqy1JDhv/VSsXq1CmfeAsmHF8X0kRBxY= X-Gm-Gg: Acq92OEY5TA84b05VQwA0qVZb2IVKiw6ceu5e71/kd/EUafINwTlOPtAY7czigfn7cX sqPiDBOtJAgETKPOu2o6nneb7nXdHD2tZD09oaa2paVUMX/gt0B4yJ95OZmre36+Lnr/UeCdHK3 LF3Tg33X8VbBrWuPx7CRNNwxpXMrIe0KwkfEiR3yNeLf6F0yUV1XyyEMDIBfppAaBGwAWeHecpQ D8DUK4gOzkKqPNZD0teUj4AL1El/ZwquUptxT+356f6loyul342oOeSmE0+ppE1u67VaMPYjcCf zKCGdgAq0L7jUp0XyDG4MM/XamJIENByd95z4g9YxhgtnNZ+OR/SEKy+nfv9Zvmx57H1EkQIntP wVITSPOj/GGdP4nKOOXgdtb4tdlP4iPaAlZDTifSxWBF92rctYDOkdA== X-Received: by 2002:a05:600c:1988:b0:490:845e:3a6 with SMTP id 5b1f17b1804b1-490a295013bmr1065755e9.25.1780068274096; Fri, 29 May 2026 08:24:34 -0700 (PDT) X-Received: by 2002:a05:600c:1988:b0:490:845e:3a6 with SMTP id 5b1f17b1804b1-490a295013bmr1064315e9.25.1780068273271; Fri, 29 May 2026 08:24:33 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45ef34a0674sm3959493f8f.8.2026.05.29.08.24.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:24:32 -0700 (PDT) Date: Fri, 29 May 2026 11:24:28 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 25/37] mm: page_alloc: clear PG_zeroed on buddy merge if not both zero Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When two buddy pages merge in __free_one_page(), preserve PG_zeroed on the merged page only if both buddies have the flag set. Otherwise clear it. The merged page would inherit PG_zeroed, and a later __GFP_ZERO allocation would skip zeroing stale data in the non-zero half. Signed-off-by: Michael S. Tsirkin Reviewed-by: Gregory Price Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- include/linux/page-flags.h | 1 + mm/page_alloc.c | 15 ++++++++++++++- 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 4ee64134acc3..ff0b192b38e5 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -680,6 +680,7 @@ FOLIO_FLAG_FALSE(idle) * uses this to skip redundant zeroing in post_alloc_hook(). */ __PAGEFLAG(Zeroed, zeroed, PF_NO_COMPOUND) +CLEARPAGEFLAG(Zeroed, zeroed, PF_NO_COMPOUND) #define __PG_ZEROED (1UL << PG_zeroed) =20 /* diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 4cb7e779a6c5..ff614e422eec 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -940,10 +940,14 @@ static inline void __free_one_page(struct page *page, unsigned long buddy_pfn =3D 0; unsigned long combined_pfn; struct page *buddy; + bool buddy_zeroed; + bool page_zeroed; bool to_tail; =20 VM_BUG_ON(!zone_is_initialized(zone)); - VM_BUG_ON_PAGE(page->flags.f & PAGE_FLAGS_CHECK_AT_PREP, page); + /* PG_zeroed (aliased to PG_private) is valid on free-list pages */ + VM_BUG_ON_PAGE(page->flags.f & + (PAGE_FLAGS_CHECK_AT_PREP & ~__PG_ZEROED), page); =20 VM_BUG_ON(migratetype =3D=3D -1); VM_BUG_ON_PAGE(pfn & ((1 << order) - 1), page); @@ -978,6 +982,8 @@ static inline void __free_one_page(struct page *page, goto done_merging; } =20 + buddy_zeroed =3D PageZeroed(buddy); + /* * Our buddy is free or it is CONFIG_DEBUG_PAGEALLOC guard page, * merge with it and move up one order. @@ -996,10 +1002,17 @@ static inline void __free_one_page(struct page *page, change_pageblock_range(buddy, order, migratetype); } =20 + page_zeroed =3D PageZeroed(page); + __ClearPageZeroed(page); + __ClearPageZeroed(buddy); + combined_pfn =3D buddy_pfn & pfn; page =3D page + (combined_pfn - pfn); pfn =3D combined_pfn; order++; + + if (page_zeroed && buddy_zeroed) + __SetPageZeroed(page); } =20 done_merging: --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 14A2E3F44CE for ; Fri, 29 May 2026 15:24:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068284; cv=none; b=cWktzrimrmGkQnJOyWxomx6Xgjhbdamm6LlM7xC/9K6gARP2XFosp88Cthw5MSnsHde44+ayWqMtqBVfSG/lCqJE55+vTP2Jek6TKnEsU1lH9dzN/fZd4MvdgZJKYloRhC/wgOK36LmVSBKaJE1HHxAGCnvmobNfTWou42Bz1qE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068284; c=relaxed/simple; bh=5mf25/lf1DdZ8laPTDH+Cdq3xa7RrEIj1yvI/mG2wH0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=lyunAwgGJqKJcE2tAzb2H99Qae7E8FlRR5rNVbLIEtVH+y+00crgDNQ5UEH7WuaVkwdZ2kZRgVVvBMlUktsd0poGu5Esw/rCbOlkLbZhVJSDalV81B5g/xzxEqOiL4ku2bfRtso886ZC9dxSzIZlPI7g6RnOO+qMkwSirpF6rFE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=W5VIatvg; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=h6TLrDUX; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="W5VIatvg"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="h6TLrDUX" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068282; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=19buVa4Rma7mh18uS22QrkEHb5BZSKvMOxe4gXL+M+k=; b=W5VIatvgioJIsto0T+DHq6TJzy2Yp0BW1menfvkOYq74CqMPQBIvTOLprRCd/hHBla+cI8 YtJJYDo5DtuUYJ4fE94gd/BUxmyy0oUBWpm2Bq1TytfgXYYEN8JveKT6NXFdMSOSk6DdAG 1DnxnGuYdFPhQ7SCdlaQp0Tj2TldLf0= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-122-qxF4jN-sNaiszI12X--JSg-1; Fri, 29 May 2026 11:24:40 -0400 X-MC-Unique: qxF4jN-sNaiszI12X--JSg-1 X-Mimecast-MFC-AGG-ID: qxF4jN-sNaiszI12X--JSg_1780068280 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-45eec2badc4so1069010f8f.2 for ; Fri, 29 May 2026 08:24:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068279; x=1780673079; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=19buVa4Rma7mh18uS22QrkEHb5BZSKvMOxe4gXL+M+k=; b=h6TLrDUXjmv5CWjipWHukTDfICiG0wbaP1dNZtalQJPWjsv6r3QZM9HRp+IwSGocGv 1vvaDUcIoa91thqF2AFaUg4rallXcKPXS1tTd7la6IBGHX0fTf+bAn8sTxqV50iYxRgc WXgnezoOEpyEdlTOWehYHFy+sDc74ZUps9yuXuQUjb6F8wGp0cvs91/sVltCF+Al40+a zVTTVowDf6DpyqMUZj/+BRjr1jCzbW8fsz3pD0QbOYVsNuP+5Ot7L/NpBJADWYD1mBdD yw/XLKbEEoGqbovl7pUu2mW18y5nSY+nBq32Y/87PHwHXczX4Shs7cwGmr5BKAlNNZIR rKgg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068279; x=1780673079; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=19buVa4Rma7mh18uS22QrkEHb5BZSKvMOxe4gXL+M+k=; b=kO1WLYFu1oeiuswhSKvF53Q1BDTMfYcnaUwlbBfMUifu71DcEmJ9YQsU0VDKIp4Z02 qD+E3GUVH1I8Slo2BESPeq7sg6P4wM5HvbZwE/C8cb9LhXwcu6u1xTNrByjUUPjxe+Za 7mcHRpdImbeer1JCg87wqRorAPNNOZg+e5Yp3IDfpFIdFQqEZnO9oIqqjVYwod9i19Gp 2bZeGJhN7nH7k5cV6qlWJ920U2O6Uyg1bO3m8Vx5XIglV5CxpCZiP0vvnuqVm2BE3onG DaJ2cLKmtSD3q4/vifKMb2OB3Xgc33HWcpv3X2ACImld9IUPfcFeLuAaqNETzPctYCvM Nr0A== X-Gm-Message-State: AOJu0YwgDTXwq64QW+rVQXuTqlyEVulhGmock0JOcjOmjBS3SI3cuGFT 4YEIzyIMSsqoDLkenL/aLlmqdQQVx6eSKsqCPzdeB8TTKb90Po55bOc4vKS5QjCTUcrkInqbtif RPIuOWuyhQN+M8+yK7Pc2TVYS1z8Bps0z5mfuV/Vub1yyJHIvcoxJcEKtFhv9fTdCtGl0gnToqB 05bBW7wqG7BOuoUlTYB5e47omKtUTdl/DaV8Nlx3bHkPc= X-Gm-Gg: Acq92OGuhXhGjl9UR48qvFHoXes52TspYRnHyPafoQR0KqigN29XCYuQ1AKlolZgAG8 y0paQ1yMD5CAUvrsEuedMV4nw37r4S/sik4e1ZoRlspnMcKL4kVaexMsQUUFszZzYGGS/wOuImV LcF09niqzJXlaP/vCPMnij6bkheAIu7AVxWqVR2Gyfh9qpTzYo689NarG528YIKlZmfHzSpu+94 BnA3W0B9rjh2ZVuFy7UZH0qD4fLNMLJGp5G1eimUhCtKonqohmggs1zvGLidCSTDuD4FP6zMl9T a77upoiXXgrcvv2JoKkZZGOlZSc3//M4aXs+XMeEZobWsQmjmP2A02xvKfFtFC8G/qemmcNQYX2 U13S+4SQJ0Z0F1FEDEaUC6v6FKd40W4jqR8DkpsQMLNvhewFoMKtlog== X-Received: by 2002:a05:600c:4e92:b0:490:8fd1:72a with SMTP id 5b1f17b1804b1-490a2900dacmr2027575e9.3.1780068279322; Fri, 29 May 2026 08:24:39 -0700 (PDT) X-Received: by 2002:a05:600c:4e92:b0:490:8fd1:72a with SMTP id 5b1f17b1804b1-490a2900dacmr2026175e9.3.1780068278476; Fri, 29 May 2026 08:24:38 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4909d6975e2sm75803555e9.6.2026.05.29.08.24.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:24:38 -0700 (PDT) Date: Fri, 29 May 2026 11:24:33 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 26/37] mm: page_alloc: preserve PG_zeroed in page_del_and_expand Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Propagate PG_zeroed through buddy splits in page_del_and_expand() and try_to_claim_block(). When a zeroed high-order page is split to satisfy a smaller allocation, the sub-pages placed back on the free lists keep PG_zeroed. Signed-off-by: Michael S. Tsirkin Reviewed-by: Gregory Price Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/page_alloc.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index ff614e422eec..72e52f049cf0 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1715,7 +1715,8 @@ struct page *__pageblock_pfn_to_page(unsigned long st= art_pfn, * -- nyc */ static inline unsigned int expand(struct zone *zone, struct page *page, in= t low, - int high, int migratetype, bool reported) + int high, int migratetype, bool reported, + bool zeroed) { unsigned int size =3D 1 << high; unsigned int nr_added =3D 0; @@ -1746,6 +1747,8 @@ static inline unsigned int expand(struct zone *zone, = struct page *page, int low, */ if (reported) __SetPageReported(&page[size]); + if (zeroed) + __SetPageZeroed(&page[size]); } =20 return nr_added; @@ -1757,10 +1760,12 @@ static __always_inline void page_del_and_expand(str= uct zone *zone, { int nr_pages =3D 1 << high; bool was_reported =3D page_reported(page); + bool was_zeroed =3D PageZeroed(page); =20 __del_page_from_free_list(page, zone, high, migratetype); =20 - nr_pages -=3D expand(zone, page, low, high, migratetype, was_reported); + nr_pages -=3D expand(zone, page, low, high, migratetype, was_reported, + was_zeroed); account_freepages(zone, -nr_pages, migratetype); } =20 @@ -2356,11 +2361,12 @@ try_to_claim_block(struct zone *zone, struct page *= page, if (current_order >=3D pageblock_order) { unsigned int nr_added; bool was_reported =3D page_reported(page); + bool was_zeroed =3D PageZeroed(page); =20 del_page_from_free_list(page, zone, current_order, block_type); change_pageblock_range(page, current_order, start_type); nr_added =3D expand(zone, page, order, current_order, start_type, - was_reported); + was_reported, was_zeroed); account_freepages(zone, nr_added, start_type); return page; } --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0A16B42B746 for ; Fri, 29 May 2026 15:24:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068289; cv=none; b=oorZQuKsjD5VnCKKatTbVPpRLxTd0SgQf1wVNw2JwR0zIfUfxUnM7cWgNdLQipHEVTtBiCYr0CEgBivHupSSeT4kBk+9O5qBH9zjuxvUbyvZ3baVOK9HeTxqZm/E1MQCWwY0vlY97VLrYpvfGbI+0qfO/0n/SL5B2oCt0n+8nmo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068289; c=relaxed/simple; bh=NEUFfCyJTTjkAORA91qK34iLisrtuxPbcfRlH9SPC6Q=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=XZyqdnZZLRj+sSGxiQFLKjAYmCagI/wtJCWpREgt2Up9yPDW0nlSk1EUDiwPntF2Uun+ir4LvRGfxmI4WIuzekj1U5Z37bArblreTpZGrU6jApAtA9z1v+OhzUU7aC/lamoHhG3pamALlcD/5T5HNTIWeS+quBxtDgn+eF8mr6g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=frM9KVVe; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=Lu3Xy2j7; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="frM9KVVe"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="Lu3Xy2j7" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068287; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=TrGGx5WHa8djlqQq7KjWOiK6GbnTz3UET362lIFlLIk=; b=frM9KVVeVfXxn8IG5spbZuNswEja8O5FiksbmqsRJq5d/INTQ1DfWYWtYSd6Sqzt1t+KyZ fhTyQF7LWK2dkX/UdolxG9UYiOyBINMqQF33/dmrt05ykq5svvk9wTKTrCo2CmHgOVPYCF YQhAGjtbLm4lMEVFVT68Jj9U/lfxwXY= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-623-q9q37VMQN1SzyM4tpfpsWw-1; Fri, 29 May 2026 11:24:45 -0400 X-MC-Unique: q9q37VMQN1SzyM4tpfpsWw-1 X-Mimecast-MFC-AGG-ID: q9q37VMQN1SzyM4tpfpsWw_1780068284 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-45ef3aeeb41so385149f8f.1 for ; Fri, 29 May 2026 08:24:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068284; x=1780673084; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=TrGGx5WHa8djlqQq7KjWOiK6GbnTz3UET362lIFlLIk=; b=Lu3Xy2j7SnJiEOvoT3am1A5cs8DAFghQxwjOPnI8ZvPK+cmOiOi5xmfZPxQRVICKpN 6fXVIP6obq4qcc5y1RJkbdDTvW0PSsTVfXHYW2eKMRPjdzcwwPvlQg+lDMIYoll2HkmS N2V6sMhqiOFSdMwACvcu20poAs0YR//SAR9l/4helqFpAbl2iEZvPyit+CdgsLrRijBB PHAL7uUHcNQv9vVGEzehvGcsTUi7PnmJev0rTjStkWN+N8FtwhtTm21RBtWeqxm21G4i cA6xn4t+aP3ohmIeWBuLClcK61Xq++xzWHJyLSV3jEFsmDaL8puWpZbyrjYHS9rRq7HX EClw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068284; x=1780673084; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TrGGx5WHa8djlqQq7KjWOiK6GbnTz3UET362lIFlLIk=; b=oXWzQCph03kHe3IgN58v7vMrvO6JuSqUW6WresDodjkboB4j3wXFRNSU2lIT1w5jx1 9RtlU+2+MIkJ7r0g50eqX5PfbedZzRM+fyL8PjiU2smBmmaU+5viSto0E2Edf+r99gCe OMbyUHCd0TmLPqFht/QdqTXCpaom8qCQElUswuFF1U/1lfV7F1ZKYSdWXCre8QD+Vy3P SQSjMBozZK4HXN849pKjBJwevYPTXV7h9UX1kMdf94nZS5/a1pTkMAcWtYfmfpMmzieM uu5quv5kBD5LoZcUvJBvepnI9BbiU/kbpOj3aWAtwxrBeJdBE4oG/8VitkQPeeKjw9oj eWcg== X-Gm-Message-State: AOJu0YwYshPMxgoWCzJuM7gfDziZhKXmRIhEAFbt4lvS9ikcIAiujIvw hQA3jbq+d/1zhWZRpDgQuviHcfyq/rldI+ZeaQ/9PZwYKi28ZzD4wYdXIxjCgZP7YqPQeExuVw8 NOfQjkOSUuy95z8/W5Vn/QuHuQzZ5Te8ZooFxnW3ISun2pZ8W5krVnLRTQtM/PCoDqfOBevytdj Bt3zxh+h2LrUrQXU8yI36y0vnNKiIryHhZIgiHA7VA5EE= X-Gm-Gg: Acq92OEY86I7s12mBvPHRrLuILVZS1rTj54m1haEQeNGdOR8qBjiMjpyFB5KPSA2Eqt O9VWINXs6ohPpredQJcEBUxu1A3a1J2HvOgLKhVx0SUlTC4wxLimvu5SdzorBCv4bmKWIkPib6o McDKewCk1iY3dSbWP2pLjgLrAcKdKH1VuRSd1svJHq4SL+A30OGMXHpCM4DNcEjDe2kuPLIflX/ XYzeoa6gUDRMFlpHzRjU2vj6anZuTvy1dOwTTQRotatr5g+AwzAlNgoM2VnAlWk8BfcyQgonj02 CrsyHQHcLUHzPGrEv2443X2BDigmYOmO/IQ8YqEHMRdNo5DvlVoKF+UqaZTQ0n12fOn6W8adTsE uN2Z2ku4Pj9SUso2l5clocTjRVxcYKB1TiGjadrIhhYWmrEB24aMb2w== X-Received: by 2002:a05:600c:630f:b0:490:845d:652d with SMTP id 5b1f17b1804b1-490a2948eedmr1400695e9.15.1780068284353; Fri, 29 May 2026 08:24:44 -0700 (PDT) X-Received: by 2002:a05:600c:630f:b0:490:845d:652d with SMTP id 5b1f17b1804b1-490a2948eedmr1399385e9.15.1780068283514; Fri, 29 May 2026 08:24:43 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4909d7002e8sm58813385e9.15.2026.05.29.08.24.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:24:43 -0700 (PDT) Date: Fri, 29 May 2026 11:24:38 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 27/37] virtio_balloon: submit reported pages as individual buffers Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Submit each reported page as a separate virtqueue buffer instead of one buffer with an sg list of all pages. This avoids indirect descriptor allocation (kmalloc in the reporting path) and gives per-page used length feedback from the device. On error, the already-queued pages are kicked and drained before the error is returned. The caller (page_reporting_drain) then marks the batch as unreported, which is conservative but correct. Note: if the virtqueue is broken, wait_event on virtqueue_get_buf hangs. This is a pre-existing issue: the old single-buffer code had the same hang. EVENT_IDX is not a concern: callbacks were never disabled, so the virtqueue manages used_event automatically. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- drivers/virtio/virtio_balloon.c | 40 +++++++++++++++++++++------------ 1 file changed, 26 insertions(+), 14 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index 6a1a610c2cb1..53b4a3984e7d 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -187,7 +187,9 @@ static void tell_host(struct virtio_balloon *vb, struct= virtqueue *vq) =20 sg_init_one(&sg, vb->pfns, sizeof(vb->pfns[0]) * vb->num_pfns); =20 - /* We should always be able to add one buffer to an empty queue. */ + /* We made sure the vq is large enough so we should always + * be able to add one buffer to an empty queue. + */ virtqueue_add_outbuf(vq, &sg, 1, vb, GFP_KERNEL); virtqueue_kick(vq); =20 @@ -202,25 +204,35 @@ static int virtballoon_free_page_report(struct page_r= eporting_dev_info *pr_dev_i struct virtio_balloon *vb =3D container_of(pr_dev_info, struct virtio_balloon, pr_dev_info); struct virtqueue *vq =3D vb->reporting_vq; - unsigned int unused, err; + unsigned int i, err =3D 0; =20 /* We should always be able to add these buffers to an empty queue. */ - err =3D virtqueue_add_inbuf(vq, sg, nents, vb, GFP_NOWAIT); + for (i =3D 0; i < nents; i++) { + struct scatterlist one; =20 - /* - * In the extremely unlikely case that something has occurred and we - * are able to trigger an error we will simply display a warning - * and exit without actually processing the pages. - */ - if (WARN_ON_ONCE(err)) - return err; + sg_init_table(&one, 1); + sg_set_page(&one, sg_page(&sg[i]), sg[i].length, + sg[i].offset); + err =3D virtqueue_add_inbuf(vq, &one, 1, &sg[i], GFP_NOWAIT); + if (WARN_ON_ONCE(err)) { + nents =3D i; + break; + } + } =20 - virtqueue_kick(vq); + if (nents) { + virtqueue_kick(vq); =20 - /* When host has read buffer, this completes via balloon_ack */ - wait_event(vb->acked, virtqueue_get_buf(vq, &unused)); + /* When host has read buffer, this completes via balloon_ack */ + for (i =3D 0; i < nents; i++) { + unsigned int unused; =20 - return 0; + wait_event(vb->acked, + virtqueue_get_buf(vq, &unused)); + } + } + + return err; } =20 static void set_page_pfns(struct virtio_balloon *vb, --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CFD9142B75F for ; Fri, 29 May 2026 15:24:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068294; cv=none; b=Jg2H9/e74b3w8urbiwcge1R1RDWTeRY+tfppShXUZHpWXR7xrkwXZ/gjT5b8BLnd16o2TJ5d9dLUQ3f6jY4SovR/nkRY+Xbe87VWxa8BnhGD5jafhjUP+jAlb5r6aScFBPrDxARAXufxPG4S1rlErG4KZY6RC8c+Sczevv0gzRw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068294; c=relaxed/simple; bh=CCqjoUNUurzYE4twri0CPtkeKv57rqlJVdiWTN8T65w=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=OHp7927n5tSReq9HCCQZm2TCwqYbW4bvwF5bTCnXphkLpLsB8l2i/lRjkzvKWLFey4YkdurZ7aveZ+wzoI0BAPcwTjzAS7RoZL1te0qXfRapbMCkxQigz0GReabrzZHij395Rp8Ptgkm+LE54iAFLfAJ20pcp9na+z/htH4O5Xc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=N3Y9CTfP; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=aTu/uLTt; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="N3Y9CTfP"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="aTu/uLTt" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068292; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=vnDpUYDWGGzhd8LJgVzrd2lW64WfRqjlcwd6dZFOBgQ=; b=N3Y9CTfPsVAOdeGsTPulohIFLqeB+L7KIOJcExm0i0SHSVpNabTYsRoTCY/k68v7Cz0xIb GU+ipG7mZYTbTimulzmPmOckHChw8P725ppswbNHSyTnlnU8YEFPbk3g/B1cmIlFnlqfxS C/SUhdrQaHh+4hXKwzuFOdsfDbFFsiU= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-689-vV5t9NXAPXe4lgjrXRoXjA-1; Fri, 29 May 2026 11:24:50 -0400 X-MC-Unique: vV5t9NXAPXe4lgjrXRoXjA-1 X-Mimecast-MFC-AGG-ID: vV5t9NXAPXe4lgjrXRoXjA_1780068289 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-45ef63d1214so173673f8f.3 for ; Fri, 29 May 2026 08:24:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068289; x=1780673089; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=vnDpUYDWGGzhd8LJgVzrd2lW64WfRqjlcwd6dZFOBgQ=; b=aTu/uLTt3NKJJb1OPZ/VCrfk36tFGQFiABLCGgxla3tEkwnZCxHFKDV6ezVsQMZSlW n7gX23Aky6TTBd+HrWsKRXg1H/eHK7kU/7Bap6K9H97XMVe1yrewxvxK8raMnGDUNJvS SCnKLPoAIRoSK7Zw0leNrEvaCTzOEMCKWAHfyPRwbml1ky10zxPBPuerq/yHvpvO+GwE ddfpMsiK8jiKZtkS5zIK1Osq7bn8XUAlwxMyDiBhNSQ59ONP2VBOhYNy9LMarC+QOP2+ bDjvd+8DhUBRmLyIjYuc/6Y3kLaFze4FFNqWNhSPcKCZ3QvtKF5Cq57n0Qr/Bjy5Np2l ceEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068289; x=1780673089; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vnDpUYDWGGzhd8LJgVzrd2lW64WfRqjlcwd6dZFOBgQ=; b=N47U/rMmTGWqWJZ7QFppZ4OfPEa+x6jSn4+08KkVNRKio0YrrGTUEXJ8O4EBOazeEc CYdktRTMGnRySASWzKtRNsB3PInp9/FiyAVXDw7Gk92E/qkIgQeOKOkdClG4IZreA+cI pP83e0sVwDmXZM4PKNDXBP+uENrWJ70qc92spzpLOZxRjAlBsKJRaci63SMoBjxE1Yhl +QHD3h0W7YWgNvaqHio7Jf8zRgSh0Q5WYLHSm+cwgK6AW0L6Wsc7fhfBaMJjMlv195VH WzawVpcDIy84XPk65ZqstckfwFTJfUHX1QFWkhua5Iu8nkvYIH7Ip7m1bRZ4BDHLllvc iokw== X-Gm-Message-State: AOJu0Yz81Nm7kAtuIgPKt2aZWkFD/E+HS1NsnEyWpbiJ3Z88syJqVg9V u6FaSi46KeUKBoi8oMZZS4NjOv77FFij4QgTvfRPr+gfwu8LrXhhB0IZT4B47X4bTDNJtYG8y1N X16K1JhTkOWdB3wTwvy1ZlhEwEWVZ6STkuZQofpHS8fpXM2d2Yof8G5GDYtAeuKB982UUU6ZuRk UOZcpPiYU2KKioXQHfj8BcDF0P9wSdQbmewe+2N5N/oLY= X-Gm-Gg: Acq92OHnCEWrOwBbGMEGAETpKwpg6w4N4nlGj84P9LPeFBx/yxgn6ylZTOnj0dxsA08 j8TSnkmrEyyqlAuuYOxD6C4t7KmouH60y+5LyFE7/UKS9FO+KS+uRWCFFdot6mh7IqNFY2ivu9C FXOPYiUq/YXcFHZU8N0jECeeiXiO4RuHFsDZovwBdYH5AVXkBV+8wUDnZFH4etOkbZ88QLAc8rK lLm1CWGtx8ORls2y3BHVQaSHWNscf1DVeMpYqy57vWY7Q+cmndH4lcZ37O/qIif+X0hyxYDm932 m37nu+otT7mpmdilNYW65JwAkjUuflbKnBT5G5nmVcsvp1B8GawPxmKwFoFeVG/MyAMJVYoMNu8 i/TRnFGwqoCasoyo0o/hRgmwSy8v778P4nuWZpsiPHYhSD4Hoke2rDg== X-Received: by 2002:a05:6000:549:b0:43b:4f86:e985 with SMTP id ffacd0b85a97d-45ef6b816bemr370800f8f.33.1780068289212; Fri, 29 May 2026 08:24:49 -0700 (PDT) X-Received: by 2002:a05:6000:549:b0:43b:4f86:e985 with SMTP id ffacd0b85a97d-45ef6b816bemr370703f8f.33.1780068288597; Fri, 29 May 2026 08:24:48 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45ef354bb62sm3964455f8f.19.2026.05.29.08.24.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:24:48 -0700 (PDT) Date: Fri, 29 May 2026 11:24:43 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 28/37] virtio_balloon: disable indirect descriptors Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" A follow-up patch (DEVICE_INIT_ON_INFLATE) adds an inbuf (bitmap) alongside the outbuf (pfns), making total_sg=3D2. With INDIRECT_DESC negotiated, virtqueue_use_indirect() triggers for total_sg > 1. Balloon never needs more than 2 entries and virtqueue sizes are 128+, so indirect descriptors are unnecessary. Disabling them avoids a GFP_KERNEL allocation inside virtqueue_add_sgs when VIRTIO_RING_F_INDIRECT_DESC is negotiated. This allocation could trigger OOM reclaim while balloon_lock is held, leading to a deadlock: OOM notifier -> leak_balloon -> balloon_lock. With single-buffer submissions (previous patch) and no indirect descriptors, virtqueue_add never allocates memory. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- drivers/virtio/virtio_balloon.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index 53b4a3984e7d..1fa1c7fa285f 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -7,6 +7,7 @@ */ =20 #include +#include #include #include #include @@ -1175,6 +1176,13 @@ static int virtballoon_validate(struct virtio_device= *vdev) else if (!virtio_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_POISON)) __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_REPORTING); =20 + /* + * Balloon submits 1-2 sg entries max per buffer, virtqueue + * sizes are 128+. Disable indirect descriptors to avoid + * GFP_KERNEL allocation in virtqueue_add under balloon_lock, + * which could deadlock via OOM -> oom_notify -> leak_balloon. + */ + __virtio_clear_bit(vdev, VIRTIO_RING_F_INDIRECT_DESC); __virtio_clear_bit(vdev, VIRTIO_F_ACCESS_PLATFORM); return 0; } --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D1DC73F54C3 for ; Fri, 29 May 2026 15:24:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068299; cv=none; b=b0OXG9zLMJGkFFDjX6SPOctdq7b0deQAdKniYjVS7kCXyTGIbP001GgqPg0M52vCJREkPpYYZiV5vOM/iLTCkcuFJrH7jX6Q/ojEMeK+ZTMdHKJtHHG+HjgNtnsSwBB/DkgndF7NwJnaAFyntUbKU44R4Q47wJYa0VUkKlsA+oM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068299; c=relaxed/simple; bh=6BBhnidGx3LLixACksZDtaGP5p7mfI6f8MPPVr2Gwnw=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=JKL8rOexAIJcshd2mkQe3NyNdRoBsVASf5c7AOe3nySH3Dg8he++idS+yCPTdXoIHbTu80la0C2y0JKGN+4vSZW3EjVhXvZPpKdvezhZdLgoQC5sRJA3oZFIDpzscPdJCfpAOzVHtDM48A+uDTV/5tjFrB80RhMNoeXl2eh7cVA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Eja1pvMt; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=Js1t1pmO; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Eja1pvMt"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="Js1t1pmO" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068297; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=A+sAoCfzYzafV/AuzgTCD2HmceB0XqeceJe9byk3cHc=; b=Eja1pvMtErU1Q9wFY9H7hca69JtRdcByouMS2dIuolvWWXUVdsAWfQCH82AO/ArHaT3TrP IUfApfFmwyZPj3OHr1IwmzX3nUFswou9fxOm2BKDMscpa9GdAU2Q+XTDoMNWkajc9TWdHg AOgtgdfbaI25wJSBjWye/lrLgeiwBPY= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-307-7ymrE4ryNuS8fJpnAQah1Q-1; Fri, 29 May 2026 11:24:55 -0400 X-MC-Unique: 7ymrE4ryNuS8fJpnAQah1Q-1 X-Mimecast-MFC-AGG-ID: 7ymrE4ryNuS8fJpnAQah1Q_1780068295 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-490402ae2c1so77889045e9.0 for ; Fri, 29 May 2026 08:24:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068294; x=1780673094; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=A+sAoCfzYzafV/AuzgTCD2HmceB0XqeceJe9byk3cHc=; b=Js1t1pmOU49rrmSUBrS6DAYUQPGMF5fnDxWrmFRdHcTBrY8KsBKmNJfwZ7dYZGO3/R L95aaTG3sG8H/xOYm9nAPBSTh+DrGO9IycAxrVBuF3qidbOGJPsULp0+2v1gOUCOrBes OosVHHU0EtjzdIDClGzpt8r2IFg49J5oEkom/h59MyW6LstsAf948+RrlDkxMxCLVigG luaaDFkx4ljNjTh7eG5JEooXGoVM8SXpngquLByz0EYpq8dE5Ywzo347h6RQ+/SqVgT5 JfgC3PCl36simqhTBjz0c0uXT0a9NXnc77ea2HhG1UZa+l4wkkVNY7XWtGJmLE6VNWaI IsyA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068294; x=1780673094; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=A+sAoCfzYzafV/AuzgTCD2HmceB0XqeceJe9byk3cHc=; b=n1aPBCaGfaOi5sza1Yj9JvN01TySDz+szIf3oenL/nUS2pYwWYqymfnAy5kZclJL5G tkUYRRdqt8Fkd4GRqTeACx9grRoJoLzAHBOIrtXA1hwovLv9n1wfTdkFbN4vBprATYw1 A6UMDDIbJWPWwjSXG2QpmYuYp/zyGwxLA+igz+ywiHV/+vL8g21SlYyZgjLaB1s4PuVI E6PIxzIkqUfmz9o8m8q27NCCMkaWRQ7v2C9zivGxWeZhfSxvNfpytF8sPZxTZTUoAtMY 5hVeXYccj9lzu0EpppmwpW9DF/i+tE0JQpIanBpeHDqHTwx7iG2arFN5U7tih6VXof7O Rmww== X-Gm-Message-State: AOJu0Yw1DF71C88QeouUY6HLXE20mSy9K7JsV+G4NX23+PSj/PUj9h8u xf5TPPBPxF7RKEvBqz1MuA+LO/prnyYNbPheDUf7qQ26EaUwhrMOscth//LP0BTaBCsTPn98gWO GIQeByWInup0UreZDihBsP2g5OrMzMTJzRjHveSl4PKjgT71OUC9ZjMj1jqX4udpt+m2gprCTr2 FhOItkjOAMRGLi1Ku8lFn1byIYEUVNcqHgSExFPEHE5M0= X-Gm-Gg: Acq92OH6fX61s/UZLGBxp2wtqLWaEBhYVHW7/YguvtbUCqXgVPw4dKYj81lOSLCKZdz VMhA1i5ZgOxUmw4JaR2lWZD3+cDfDHto3cuBnRjVNQJmmu9NcTEPHwo6HgncXoxZqRO5tzf749v Tjk+rv0FnN4XmaEer1DKC9c2vswJHT8/rFDsLGa77jJcLUZU2s77svev1/7XiNf43PjNuInyS9P BuR4NgXOvAAY8xM+orjM2/D6WOSKNSPy0PLWiXhlalRrtF/DNT4DA84Zgug6D+4t2TWbXECxKQA 8Fzpuh6Ib30kN1jzloqHZNx6fIwdqLIwr9CGcwpEudBJh7O9FtJKGBzGJItft2u2OYOY2OBexyM pJErUou5obgV0/UjGyPUm3iKVw9u4cy5ZjE9ZAMdqsp0p+JQNZjc0bg== X-Received: by 2002:a05:600c:1c1b:b0:48f:e230:29f4 with SMTP id 5b1f17b1804b1-4909c639d91mr48523635e9.15.1780068294392; Fri, 29 May 2026 08:24:54 -0700 (PDT) X-Received: by 2002:a05:600c:1c1b:b0:48f:e230:29f4 with SMTP id 5b1f17b1804b1-4909c639d91mr48522645e9.15.1780068293714; Fri, 29 May 2026 08:24:53 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4909ca6575csm82033265e9.4.2026.05.29.08.24.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:24:53 -0700 (PDT) Date: Fri, 29 May 2026 11:24:48 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 29/37] mm: page_reporting: add flush parameter with page budget Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a write-only module parameter 'flush' that triggers immediate page reporting. The value specifies a page budget: at least this many pages (at page_reporting_order) will be reported, or all unreported pages if fewer remain. The actual number reported may exceed the budget since each reporting pass processes a full cycle across all zones. This is helpful when there is a lot of memory freed quickly, and a single cycle may not process all free pages due to internal budget limits. echo 512 > /sys/module/page_reporting/parameters/flush Note: the set callback runs under kernel param_lock, so writing this parameter blocks other built-in parameter writes until the flush loop completes. This is acceptable for a privileged debug/test parameter. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/page_reporting.c | 54 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/mm/page_reporting.c b/mm/page_reporting.c index f68c79e57427..3f584f538c68 100644 --- a/mm/page_reporting.c +++ b/mm/page_reporting.c @@ -358,6 +358,60 @@ static void page_reporting_process(struct work_struct = *work) static DEFINE_MUTEX(page_reporting_mutex); DEFINE_STATIC_KEY_FALSE(page_reporting_enabled); =20 +static int page_reporting_flush_set(const char *val, + const struct kernel_param *kp) +{ + struct page_reporting_dev_info *prdev; + unsigned int budget; + int err; + + err =3D kstrtouint(val, 0, &budget); + if (err) + return err; + if (!budget) + return 0; + + mutex_lock(&page_reporting_mutex); + prdev =3D rcu_dereference_protected(pr_dev_info, + lockdep_is_held(&page_reporting_mutex)); + if (prdev) { + unsigned int reported; + bool interrupted =3D false; + + for (reported =3D 0; reported < budget; + reported +=3D min(prdev->capacity, budget - reported)) { + /* + * First flush completes any previously scheduled + * reporting work. Then request a new reporting + * cycle and flush again to execute it. + */ + flush_delayed_work(&prdev->work); + __page_reporting_request(prdev); + flush_delayed_work(&prdev->work); + if (atomic_read(&prdev->state) =3D=3D PAGE_REPORTING_IDLE) + break; + if (signal_pending(current)) { + interrupted =3D true; + break; + } + } + if (interrupted) { + mutex_unlock(&page_reporting_mutex); + return -EINTR; + } + } + mutex_unlock(&page_reporting_mutex); + return 0; +} + +static const struct kernel_param_ops flush_ops =3D { + .set =3D page_reporting_flush_set, + .get =3D param_get_uint, +}; +static unsigned int page_reporting_flush; +module_param_cb(flush, &flush_ops, &page_reporting_flush, 0200); +MODULE_PARM_DESC(flush, "Report at least N pages at page_reporting_order, = or until all reported"); + int page_reporting_register(struct page_reporting_dev_info *prdev) { int err =3D 0; --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 022323F58C7 for ; Fri, 29 May 2026 15:25:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068304; cv=none; b=igB63eIK4tiZoOaFhUp1qElYq3LblEPXuON3rrvTKZLtV+D4168PNiZeyuXfeUyIjvp9I7LpEc4OlGC8i5UJl9aP/8rRFxQxsE9ijkGX6A18CYUEKQIQ0xPjWUWU8OCUFIuzeCr2kmupkQMh+IBplEA8aPkjzd0F9xLWJzKL2M4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068304; c=relaxed/simple; bh=gX3aTYbHX/GyGonutM5PlW3FkeXMFkBsDPOEXbSa+qI=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=fxG0A6CnsYWhqUuQ/UAFX+bGBi1tm7xReEgvPV2WtpTeTKTGA6Cy3ods8gYoV7zuo+Vy5GD4lNoIiLcBmakKEmOHUBc16UHM5+fhdkL8ZEI+HpwkjZcr3Z+op4m2XjWAAv+8ZMwLNcYZGg+TXYKd8j/vdDFiqpn4sdbsqp0l2Hk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=BFKURluI; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=gjutIoJT; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="BFKURluI"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="gjutIoJT" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068302; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=yuucL8pRyKGlavgC043yPLfH9MArN/0/T/V0zfd6qpo=; b=BFKURluIe3y2KQWW3uCqt6yCGENYSjLGSk1Js0e9M03MnSzsdbZQwWrls90NV6jW2cFV/p nZLDE7w+1gYs6Gp4Mp1O5tKJ4566MegCT9eO1OK0A8c1AWw9QUWpd/8NsJyOv24siyB7QI KDrIOG9RwGicf5Tp0UyyUr6c6DDbA6Q= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-687-hMCU-IfaMkSK356zUntkxg-1; Fri, 29 May 2026 11:25:00 -0400 X-MC-Unique: hMCU-IfaMkSK356zUntkxg-1 X-Mimecast-MFC-AGG-ID: hMCU-IfaMkSK356zUntkxg_1780068300 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-49045f93baeso55197945e9.1 for ; Fri, 29 May 2026 08:25:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068299; x=1780673099; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=yuucL8pRyKGlavgC043yPLfH9MArN/0/T/V0zfd6qpo=; b=gjutIoJTikFVPpOB/MHQqPju3lnz6+qbKE7JJYLMmK7B97eNapwHC2x+9EtFfzJKwF YLH5I3DTHZjF+rqPuE6XI9zuveg3mPeFOTI5omXguXuZIe0m6dY7MeqfEaQyC0SMl2d4 M+RCDb3IsUc1bJSBdvUqP9feKPK97/WczkHYOYdv3nlfwrYc8KiSa+c+rktvoMH+9P8y xTgbsKQLahg5AS55W8pT1ZnnUwWjBXOwkIWtzVDdtRTtNBZgthYqJUSR+nzoeQhKHVHm zhfTm/2ncmL9iqE5ORkjqsQYZssj7sJ3htu88bAa9dbI91JCz/kIjEHYRq+WShBdu+2U LtHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068299; x=1780673099; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yuucL8pRyKGlavgC043yPLfH9MArN/0/T/V0zfd6qpo=; b=MMNQoDrLNp8OpgICi9fKE65Z12X87bH4e3ysQgmCurFVc/VBL7TxEU2z0t1n32Bd8R 7wJOJ9yKN1Os6CPOcYEZshsibuRWLPXXbFHUOEls533E2j83RlhUpLR55CwTMjXilbz3 kznv6B3sCCMacSkgCqJqMXeOLHCEOa9w+PKrsrKZf7+ChB0nPeCUnToCU/vgucaV4AQ0 eMj/zfa9wNdDRsB/DCMdojwoqL3MG9hvHyCLeSZpA67nOBEkpxqVcMKk+oSkeYVw/MJ+ cmzDb0T6WuYw43O+gwO1U/RWevsUuP2beLdQu5Owud3XrDmiINNAdikYqJu0OtIkmmeB /H6w== X-Gm-Message-State: AOJu0Yxvd79nChXwoNJSq6GDN5T7NguCX67PGAxX0jMsXLV74hrrBHMu EvjyD0VneVNoYiaC1V/zuuvvt8ALj/SNb0sNKAnfvAAUw4UEi2FMUcRe4m7BkF+ISbpiP/NcpD2 zqj3lOopNBI4gDEdLbF8DdRuXa04BdQKMg40LIMoPmSMWcB5B6HB6m8ETDsy9ba0UOpHHmJgZRU 5ka+88hyTS4xDySwDiDXpU1LpxAnek79Gi4x3Vsd7mJeQ= X-Gm-Gg: Acq92OGNermo6+x4QWhPPAZ9ruovb4RFRBMp1C6Ok/txYsW9mHchYZU6Lj+cy7C+Pc/ O7b7MaWYK5Jb3KY7dsUN2FhVaNJr+klQtYOoXCe63SS+x4q4B1Dr4LvpahJeGI+vZ9NNEPlEjbj 5bcb8AsnJbwc56E9/6g8kVAm+73fxYUXdj9dZMU4xnrqwsUNdoC+AcGudDfSMsjxucDj0c53Nmf Dbb3jgDN9UtzP937NINaOUqmhqFuPOkcRXZr4EpafYujF5kpXzaN51q8ry9RyV5JKK0ldiBcq61 rd6C1nxOBvK1v3gcH17nVNdnQI21RPad3iaCtHtjTWBt9P/IOzodhBIkp33wI239z0H/9u/Ug35 sd3ISxhkjhKSzoDfKomnj3GLyNsqGWK9FFCHpifDJ1++HC/+DkqoB9g== X-Received: by 2002:a05:600c:4f08:b0:48a:581c:ead with SMTP id 5b1f17b1804b1-490a290da24mr2201885e9.10.1780068299533; Fri, 29 May 2026 08:24:59 -0700 (PDT) X-Received: by 2002:a05:600c:4f08:b0:48a:581c:ead with SMTP id 5b1f17b1804b1-490a290da24mr2200735e9.10.1780068298871; Fri, 29 May 2026 08:24:58 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4909d68b853sm49196885e9.3.2026.05.29.08.24.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:24:58 -0700 (PDT) Date: Fri, 29 May 2026 11:24:53 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 30/37] mm: page_alloc: propagate PG_zeroed in split_large_buddy Message-ID: <64f9b0a9c5b1a3db23b4e4d0b11c3609c248708b.1780067978.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When splitting a large buddy page, propagate the PG_zeroed flag to each sub-page before freeing it. __free_pages_prepare clears all flags (including PG_zeroed), so the flag must be re-set on each fragment after the split. This ensures that the buddy merge logic can see PG_zeroed on pages that were part of a larger zeroed block. Signed-off-by: Michael S. Tsirkin Reviewed-by: Gregory Price Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/page_alloc.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 72e52f049cf0..702fa2ced1e1 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1524,6 +1524,7 @@ static void split_large_buddy(struct zone *zone, stru= ct page *page, { unsigned long end =3D pfn + (1 << order); bool reported =3D PageReported(page); + bool zeroed =3D PageZeroed(page); =20 VM_WARN_ON_ONCE(!IS_ALIGNED(pfn, 1 << order)); /* Caller removed page from freelist, buddy info cleared! */ @@ -1537,6 +1538,8 @@ static void split_large_buddy(struct zone *zone, stru= ct page *page, =20 if (reported) __SetPageReported(page); + if (zeroed) + __SetPageZeroed(page); __free_one_page(page, pfn, zone, order, mt, fpi); pfn +=3D 1 << order; if (pfn =3D=3D end) --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7E2F03F58EF for ; Fri, 29 May 2026 15:25:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068309; cv=none; b=CpcQiBYH48+2IraHat+ejBEf7MiXdDFzgw5MfxFHa3e3xbqz3mu8xGyvrSmFfDi6G+vUXwKJ1kuHK3caBjRJxzvOo1r5Y0DlYqlJ6Gc0Ub3LlJl+0BvoqmhT2q8qTiAaGYeE648rxN49In9MGVIlkdVG3BOLj9uzcPg9+uoasu4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068309; c=relaxed/simple; bh=o1q520LUtIf3WEFA+mnt5VZVOCpKdiGNtHAvZKuDwl4=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=ZRAVInN6myWza3OEwEfUGrD39PHZ2WcyR1iY4bQJbFYpnbF0xC4Jd1mnZcmXzAnLRg6xUbFvnZVAmCFMzWi8srs6kV3YQla6WKqs5T5rxFPLE8zdS7QXhiRm3G9znVXCGFgwVKojS56t7fWAR5MI22J5DRQ8t7CCxZj64zzFaLw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=fYVEgBqE; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=XKewBRfZ; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="fYVEgBqE"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="XKewBRfZ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068307; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=PhUe/N+eFNvMvlBoMUh1oQQhfmsYB8gjnR/tug0hsFs=; b=fYVEgBqEe1qSgXWuALGrKjW3mmSCrUARkkCVUtGUU40ktLfKASxU3618FN5qF5AMh5satp iIZoxo2ZrIIwrz9Ycr3j26U0aKuIIHw4ErMbJJ1zcN9KyqprNE/PDnYY8bH/gL0vbo9Epr 6wJjriPBK3oIY8Y/I+hpWt9hWZp6Zik= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-459-tfB6behaOmOAzb0Ji90dgw-1; Fri, 29 May 2026 11:25:06 -0400 X-MC-Unique: tfB6behaOmOAzb0Ji90dgw-1 X-Mimecast-MFC-AGG-ID: tfB6behaOmOAzb0Ji90dgw_1780068305 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-45e2f74da10so12750297f8f.2 for ; Fri, 29 May 2026 08:25:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068305; x=1780673105; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=PhUe/N+eFNvMvlBoMUh1oQQhfmsYB8gjnR/tug0hsFs=; b=XKewBRfZz55RITf/GuGRkjPaVLKBAGOkRdxyi+PcqlRYyTbn3nTg4pOgcytaIrnOKq hX9XCjEwye1lsMO5+WeClh0NBo2NE7ZFSYLn5GD4LFT7QXEr4SKE0cyUQ+tm3rgzmNvG tMEokeZKtXGoST7jB55xqEuovkRmZpw8ZIXKB40OygLBcPgYC+OLClM/XBHe6OYEF9rx q7Xw3RGnaH9HML7c5id7FktfIaS77its/f0rWewOF+FYVkmWw2ZI6i/KpS/PREc7u9RF yHT4tyOlSpeJRiYlM4vBx/oiR8szgZD0oC3NUvsRPEj7ejaPM6CRjCypHv5agQioMnjz 7IRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068305; x=1780673105; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PhUe/N+eFNvMvlBoMUh1oQQhfmsYB8gjnR/tug0hsFs=; b=OsJy92bhmXzwuO6FmnLR0mXyf3UNlPlNS4GDU4SeHGzom4FDpuJa2zf4/xCo2uM5Or Kx/GfJHB3LrqmBO7HDKTiIdg1+FiubgH9XMFb2MwEABnf+aBOVnaFWHuyXd8o3v4ysNY UScJiXQuJ8x2nE9IH2QulyatkdDgXBX73lnc5NOBrMDglOOcY3faKSYnYlZlJWOfzugK Ao+f3mfrxQeTI2QPfWS8uxtuOtwtvxc6V1LSCRyPgC58ZcZgTGCpgOJCiNNfCw3RFMzo e/AF4cXqZ0JqZykd98XMArim6RZmN6rx0K8XAsudH99QuncwEWdaB6BSlYNAVfp0xQt4 CuRA== X-Gm-Message-State: AOJu0YyEJ0iaAK4GHrB6Fj9C/xkk6MkLrCJHL2Vl6tMtamwp/H9LkI9y jVjye8M1Q7fEkLAJXPVh2QZRBwBtF1WTBcDnaK0utVPcXbHb7/GdFZ7bbx+RlkcLzm/8ciGRhJq qSw4r4eSBnTCtT7IMvBU+n2ypiwd5SQJfjTS5kzuGQPOjbxmGAHXZ1TX/p8QOF02pCYUK9YannQ 6dKIL6WCoDVAvV1bxT3Xuzfv+SWC9qZI3TJ52TclooD1I= X-Gm-Gg: Acq92OHFkMTyv/oMySwvsmZzj5eh39mKh4ltW0qwj4K4+pX/FegNbYEnjo8FhaRjWF8 7y02+tMhy1hdmSeLn4tI5N8l3bZhoQ0Q4DfWTD7mrZEoPK24YGsa2uuz4BNYkrmTKstne1H+OuS P8M2s0SBohWa/ovjZSgyJoPAbxcz8xnjWyq0fz+ZFuySrGuRV+b2hry/WBRKzQxAL0PPCS3G9/G 9L1oC22d4K0myjTfqzA7+4i3/B3wXHELvC3EfF2e1JhnhPQ1S36n5W1hN/2ZJLgwgXjqciIL5lx n2lB+USOp9TcawBsCvvx2mC+zm8v2YB6parvnm9vvDEGdgQBdPnS9OcLssPlSpIDRcNcHh6SPo+ XSOA7KknE+lGBWwkSXWFWJCjdbllK/wF0AsmvfwjTy8PwPY5DISPCMQ== X-Received: by 2002:adf:f690:0:b0:45e:73b4:85cc with SMTP id ffacd0b85a97d-45ef6b815f2mr388346f8f.35.1780068304727; Fri, 29 May 2026 08:25:04 -0700 (PDT) X-Received: by 2002:adf:f690:0:b0:45e:73b4:85cc with SMTP id ffacd0b85a97d-45ef6b815f2mr388225f8f.35.1780068303965; Fri, 29 May 2026 08:25:03 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45ef34b47eesm4828235f8f.9.2026.05.29.08.24.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:25:03 -0700 (PDT) Date: Fri, 29 May 2026 11:24:59 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 31/37] virtio_balloon: skip zeroing for host-zeroed reported pages Message-ID: <6bff99e8890ea91c93e3e8f0c27ddb3b5b2528eb.1780067978.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Implement VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED (per virtio spec proposal): when negotiated, the device initializes reported pages (zeros, or poison_val if PAGE_POISON). Check per-page used length returned by the device to determine which reported pages were zeroed. If used_len matches the page size, the device successfully initialized the page (e.g. via MADV_DONTNEED), and we set the corresponding zeroed_bitmap bit. Gate host_zeroes_pages on the feature bit and page content: when PAGE_POISON is negotiated with non-zero poison_val, the device fills with poison not zeros, so pages are not zeroed. Clear the feature in validate() if REPORTING is not present or if PAGE_POISON is active with non-zero poison_val. See the virtio spec change: https://github.com/oasis-tcs/virtio-spec/issues/244 Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- drivers/virtio/virtio_balloon.c | 22 ++++++++++++++++++++-- include/uapi/linux/virtio_balloon.h | 1 + 2 files changed, 21 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index 1fa1c7fa285f..e3afa6f32ba5 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -207,6 +207,8 @@ static int virtballoon_free_page_report(struct page_rep= orting_dev_info *pr_dev_i struct virtqueue *vq =3D vb->reporting_vq; unsigned int i, err =3D 0; =20 + bitmap_zero(pr_dev_info->zeroed_bitmap, nents); + /* We should always be able to add these buffers to an empty queue. */ for (i =3D 0; i < nents; i++) { struct scatterlist one; @@ -226,10 +228,14 @@ static int virtballoon_free_page_report(struct page_r= eporting_dev_info *pr_dev_i =20 /* When host has read buffer, this completes via balloon_ack */ for (i =3D 0; i < nents; i++) { - unsigned int unused; + struct scatterlist *entry; + unsigned int used_len; =20 wait_event(vb->acked, - virtqueue_get_buf(vq, &unused)); + (entry =3D virtqueue_get_buf(vq, &used_len))); + if (used_len =3D=3D entry->length) + set_bit(entry - sg, + pr_dev_info->zeroed_bitmap); } } =20 @@ -1051,6 +1057,9 @@ static int virtballoon_probe(struct virtio_device *vd= ev) #endif =20 vb->pr_dev_info.capacity =3D capacity; + vb->pr_dev_info.host_zeroes_pages =3D + virtio_has_feature(vdev, + VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED); err =3D page_reporting_register(&vb->pr_dev_info); if (err) goto out_unregister_oom; @@ -1176,6 +1185,14 @@ static int virtballoon_validate(struct virtio_device= *vdev) else if (!virtio_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_POISON)) __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_REPORTING); =20 + if (!virtio_has_feature(vdev, VIRTIO_BALLOON_F_REPORTING)) + __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED); + + /* Device fills with poison_val, not zeros; disable zeroed hint */ + if (virtio_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_POISON) && + !want_init_on_free()) + __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED); + /* * Balloon submits 1-2 sg entries max per buffer, virtqueue * sizes are 128+. Disable indirect descriptors to avoid @@ -1194,6 +1211,7 @@ static unsigned int features[] =3D { VIRTIO_BALLOON_F_FREE_PAGE_HINT, VIRTIO_BALLOON_F_PAGE_POISON, VIRTIO_BALLOON_F_REPORTING, + VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED, }; =20 static struct virtio_driver virtio_balloon_driver =3D { diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virti= o_balloon.h index ee35a372805d..13074631f300 100644 --- a/include/uapi/linux/virtio_balloon.h +++ b/include/uapi/linux/virtio_balloon.h @@ -37,6 +37,7 @@ #define VIRTIO_BALLOON_F_FREE_PAGE_HINT 3 /* VQ to report free pages */ #define VIRTIO_BALLOON_F_PAGE_POISON 4 /* Guest is using page poisoning */ #define VIRTIO_BALLOON_F_REPORTING 5 /* Page reporting virtqueue */ +#define VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED 6 /* Device initializes repo= rted pages */ =20 /* Size of a PFN in the balloon interface. */ #define VIRTIO_BALLOON_PFN_SHIFT 12 --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D1AD73E51F5 for ; Fri, 29 May 2026 15:25:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068315; cv=none; b=HzNiIaPt5B6Id/XpzxpsPYdeAv6boZjsRDT0Pc4vtokwnRoAzhOSFwARFuf9RWBjFpAO6tfhasYgRyHED74f97ftgUFjacX+vLYdNQS7mQuTMvSpOG5GAXAIHukVhTB8DSKhPixitBZVHcGAt/kOs4ESgNs9ywMv6GLuA8DtzgE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068315; c=relaxed/simple; bh=KQP828o7Oia72X2oJuTNYtM9uBAruxIc45lg9ub3w4U=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=X8/2RB1gvbtKPXBy71O9URfAdn8kScE33MANAkhCAzC5Rz+d6btYuCeH+/iKcBPHbrB3rwPRyhI5xoMKFwVst88gpPvkd4zphXFHtpgtNrICKKSRJaCGomhzR+FTy74bogD+tmEijCBgoU2Wai4oaxY5KeJHgLw2pY9kwQpADL0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=e6b0snqX; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=O7ayGMkR; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="e6b0snqX"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="O7ayGMkR" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068312; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=3kNatCmWHyttmbkexMBnMmdZ3Q708953lxRldkYnzX4=; b=e6b0snqX4PONJePy5k5SX0vOtBlB/ZL+IoRuNKCANEaMPiV7SjiTS6r/IZ6mMbYjFA0E4t 6zcHRilscTdH5VAJLajqSp7giEOXyNU8zVXOD0IvqtbHK3VQrIyoTQc+bEDXSkCmkX9nej zr7paysGx4maSo8RvL62MR5tkoiLG78= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-113-G_GXkOA_PgOxWxQuDelY7w-1; Fri, 29 May 2026 11:25:11 -0400 X-MC-Unique: G_GXkOA_PgOxWxQuDelY7w-1 X-Mimecast-MFC-AGG-ID: G_GXkOA_PgOxWxQuDelY7w_1780068310 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-45ef0af9517so739682f8f.3 for ; Fri, 29 May 2026 08:25:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068310; x=1780673110; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=3kNatCmWHyttmbkexMBnMmdZ3Q708953lxRldkYnzX4=; b=O7ayGMkRDFR4ApHYei1uG5lecgumtMZzmQsoL5atx1V9bwNXbd3L6qr9qowQNunvB5 WmtJ/W1clckLxovlkVNSBh02TR33gJue/xtKb/TPVuCOZuTZ6e67mv1lXvqjs9NqbNr5 d3+9EvRtCEbu1ti/vl4/be9Dgmj9xQohxsMeG9FyrMTImw5THE1Cm8NDyUNkPFQoUoKL ryIra+EMPZKmzBSZ8v3JYABXgy2v8u4v7h1Hyk9B8I1AMDWq9Mge0PeWW7vNaPkdK1t0 kOZQFz0/uJtxE6o/KRGT99nGSb0xlH56rdt06XBkGcp8Nd1vncCC3u+GF7eEIVIyty+f NwMg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068310; x=1780673110; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3kNatCmWHyttmbkexMBnMmdZ3Q708953lxRldkYnzX4=; b=UQ0AXPFm7rI6TEemHGy7uCVaJllL2dzggCcF2uAi9bCFLSv7cM/UB1UByfuqEE80+5 BBoKAupockglE1KfEhrT4nH06yvXRk94N4gSzK1TqV88FF+zqGShBqV6LIwjXw+cKbgv n1RLYKnXeh/r7Jl1GnIcF51OrHl0Ek/nql8cuQoRxZ6kPTDSUMj7TcQ3f4n02j3B+nCD fbU5a00ipK0Rz/20Wn2CIRu5IBcT2KK+2DUDXFbw9BQXmAE+/AOTRxqP6ABzMgkwlH57 iN3JOg03ZvZaX5pZFt/u53qj8FFdSEQBnJMqX3d/TC+rd10HlNJoCtI6AianCrdczQgK BOqQ== X-Gm-Message-State: AOJu0YzsiqBAHfhocHNi942ebpoMF5b1ePyJ8LujQAzJiH+h+5VBjPeK ASeQERMrd9Wre8XlmcyWkhlgsE3wqyAhHjbFTk1vDAfZDMMqFguhzeTN9pcZ+9UVhKXF9RCVfJ6 WpCXJXNKkyYCM4aAzpLFzAzrXSo/JFHDXpQGGzzNqSvfllrBxf3qD6BnsT5626kBHkFj3a0nsbU RUamszdjaqG24099QhEfbS5974YLRgNBz94WRVsF1HmPk= X-Gm-Gg: Acq92OHb7zgVKsFIXHHgeOXIg6UlVVf2kzZfGt8UwXpAd7udiUvLTI34YGNMoY4y8dX /oi/Lyd6UqyVZE+TQmikZhNtJcxLln1+F/ldKurTOUY63UD5xp6s5kBpa+ElCFb2tAlsFy9/FVr hWIa0irMy1wUJiswkfzfLqRxzteytfT/4Y6uvYKuMvjoRkvNvuriz8Rz4UR+G/EM4nerr9/EKXc qtiafSbUd/woqNjDwVMDFxznZmDGjxBESIntM8IxODjhJFoYn4FJJsDimBD825KYlYOcYXcp4E4 KkuM7OQlLDmBHSJiIlvShpIZCODHn0O9v+M77eYla5ui7Cl5ATg9pDeEf7PCziLvyDAzev0HrJQ 6lNpvrBhnT0mqBJeBu/Gbq3KCGAbJHQZpaM/AEQooF6e2b6NcbyqGrw== X-Received: by 2002:a05:6000:2892:b0:45e:f581:6483 with SMTP id ffacd0b85a97d-45ef6b60373mr513554f8f.30.1780068309939; Fri, 29 May 2026 08:25:09 -0700 (PDT) X-Received: by 2002:a05:6000:2892:b0:45e:f581:6483 with SMTP id ffacd0b85a97d-45ef6b60373mr513407f8f.30.1780068309118; Fri, 29 May 2026 08:25:09 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45ef3556f51sm4162514f8f.25.2026.05.29.08.25.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:25:08 -0700 (PDT) Date: Fri, 29 May 2026 11:25:04 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 32/37] virtio_balloon: disable reporting zeroed optimization for confidential guests Message-ID: <1c8bd24b901c36666c675044bfebf498740d1594.1780067978.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In confidential computing environments (TDX, SEV-SNP), the host is untrusted and may lie about zeroing reported pages. Clear DEVICE_INIT_REPORTED in validate() so the guest does not skip re-zeroing based on hints from an untrusted device. Note: currently REPORTING remains enabled and VIRTIO_F_ACCESS_PLATFORM is cleared in CC environments. This is known to leak free page physical addresses to the host. Whether that, or ballooning in general, is a security concern in CC is up to the user. This patch only disables our new zeroed-page hints where the host is untrusted. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- drivers/virtio/virtio_balloon.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index e3afa6f32ba5..bf1172ad5419 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -19,6 +19,7 @@ #include #include #include +#include =20 /* * Balloon device works in 4K page units. So each page is pointed to by @@ -1193,6 +1194,8 @@ static int virtballoon_validate(struct virtio_device = *vdev) !want_init_on_free()) __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED); =20 + if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) + __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED); /* * Balloon submits 1-2 sg entries max per buffer, virtqueue * sizes are 128+. Disable indirect descriptors to avoid --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 53B2C3F6614 for ; Fri, 29 May 2026 15:25:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068320; cv=none; b=Raj+crrsERQjIJLvnnRjtCX0aHV0gIZC6sQZbplFStby0GX2zDdfJeiQJaSJMd8GLRa8c6tekEor5O4CPczxZV7u3duVhE7QF0QY9e0z/gUw0xx7x9ntyBqOmuITAd9pu5iGgMK2btmrdE7u7bX4c1BkGBPzdiHuSeycWrCjZjM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068320; c=relaxed/simple; bh=m+2CGKGy9tP1X5jr3y7WRquw/r2rQI3y1dy5VzBRNcs=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=tnXVwTuEYvwYlfmIGf6XvGuptD+8CgOpLkxUuLStE/s3T8OdUT5hH4InG26gZKol+1TQTF38pxYAEjv9yoWCKKuGAycFtuiNCH5MXvr9hjMWBPRCy6lN5snx+5UEHc4gw04L5ybIswwr0+yOsdOpkgoke1RFOEPQ9FPq6uhxeWQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=NDY4c3oT; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=feT4VT01; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="NDY4c3oT"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="feT4VT01" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068318; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=hVyoesbwmJaC/SCrniPu7amcCOYT9PX+z/FEgZLf3FM=; b=NDY4c3oTqSN/a+bkjglsvIZOxVlmRleuZurfOK94mmCZyKH8fYJLGExTdUKExdO+9rr8n0 lOZFexdx6Str6Ip77u+f66MzMlJrSmBAw7zBRNji0yJnk1YHYWZsmMLBhgUEdSuKZ/i6NI VfbtW1OV4PJhQXlFkjqJPyrGNxtKHeg= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-53-sa8QtCLPOJC77WVLbQ5brQ-1; Fri, 29 May 2026 11:25:16 -0400 X-MC-Unique: sa8QtCLPOJC77WVLbQ5brQ-1 X-Mimecast-MFC-AGG-ID: sa8QtCLPOJC77WVLbQ5brQ_1780068315 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-45e81291d62so11858487f8f.1 for ; Fri, 29 May 2026 08:25:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068315; x=1780673115; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=hVyoesbwmJaC/SCrniPu7amcCOYT9PX+z/FEgZLf3FM=; b=feT4VT016LjaWtHefHuEbSQaLDKTtV0WdH56EKDu2WhSRJoaUtrCqTxIaYL9w2Ehp3 +f6jlD87to5FDz3eWbEdAhM5q97/tfFMmehSZJarIOxVjsKwHOrMhl9I8CreoTmMUK+7 zS84Xp5CevH+O0nN7RtcySfEbfGgCH5SF5VOlPWPKWMZkj1R/bTuC4P2Caydf5R3DoQ1 K4gkdDI1PO7iN1N94TFUxiXLSnZxlYxRz9FkN/mA9i7ejzW3UqBMkTyR40BdWEuwy+4v lG21oZtFHIEYaY44HZBgu0/6O4yZF/9iB7GKxz+kUTtI4lfj+JjWQl0RdHfd4FCI/ULe DU8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068315; x=1780673115; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hVyoesbwmJaC/SCrniPu7amcCOYT9PX+z/FEgZLf3FM=; b=QxWqsVRcvcMBCjBLwKdwkR/2UkFgYL17htvtREi4esjiSrg/oHdw/8nXmNqRW6AtFq br6JydAs9Fv+lOZ41KEDR4QGQVidNY//i/aEs+co4b3jFFermqDAvcYPVA+d3t8F4Op7 rlTPGiimBf+f6f8smp0kW6M+E8jxev/p6CwV7xKLuwzQNlINKDyQyMnB7gH5m7HXH07O erMVsnsbolWDwMNJAtRbH+olghfubH6h+HRSujT/E8iE8zkGpx83gtZaVZ/FPW7axXG6 oqLPMh++0Ei/jXz7ejVISNHocT454XVnbLXW1BbjZbRnQHl3aCt5UD8VImGiJIKEPlGG FFqQ== X-Gm-Message-State: AOJu0Yy10K0u2Ty5ZssmS/Z7evwoHP2/DeZ90Pxi5mnpVnoYQ9wQ9KS9 YCwy4LzEBcduVg4YfA9LYgUvwHs7JLBGXU1MeICchEtM0ur6qidt0KYLCg9aId/xJufxiUKIQsA HMIpKK8mkqxi9tgQuMQhEAVC6+9rNdIO6TXXdpFryv1krnFxQ096XDF8sl3P2xrj5U19Rk26t/T xAlrcIyLXUYckusbrYWv8r1jlHLnzmTe0R1ouM6Zr1XMc= X-Gm-Gg: Acq92OFiS/0DSKdmKLlI6nzdqs0QC3VyW4bqP/R3OZUbRLA9XFQFOl7IX6qGQzT2hbZ 8qaeiYlIT2+toMGE2W0l86mafJUUbC5emXM7dJSoISOFZoxpywVNRHt9o8o6T8Fkw8LCJ+nQcrm EOMH9HgMZosmgDCxh9+qkquR3EmhbRXmoC0mWKRlezrQ73dIMRKhUMuXkIAVlbJ41BD4g9si4hu BsDohah0j3AmVELwlcVjtINiF6Ez2npAn29eT1ypdWfmqaWbAzb5RptEX3VDqiQbNqcLFdf0yG5 j9/UmLusi2DRbEM7H35v0K4qMQ/0bY9+eQHd2CAe8Bj5AjexHXw0Re+Gq/pbmErsu3RvdLgdsTx gd3f94A6DnNMJUE2MYlPrr5cgpPVU9rWl0p7D1evEzHO6idB5hwiu9w== X-Received: by 2002:a5d:49c2:0:b0:455:7d77:1d25 with SMTP id ffacd0b85a97d-45ef6b75cf8mr364505f8f.27.1780068314977; Fri, 29 May 2026 08:25:14 -0700 (PDT) X-Received: by 2002:a5d:49c2:0:b0:455:7d77:1d25 with SMTP id ffacd0b85a97d-45ef6b75cf8mr364397f8f.27.1780068314292; Fri, 29 May 2026 08:25:14 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45ef357635fsm4180544f8f.33.2026.05.29.08.25.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:25:13 -0700 (PDT) Date: Fri, 29 May 2026 11:25:09 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 33/37] mm: add free_frozen_pages_zeroed Message-ID: <2869cf9cacba23da37dd8b5a8417f8ae30861724.1780067978.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add free_frozen_pages_zeroed(page, order) to free a frozen page while marking it as zeroed, so the next allocation can skip redundant zeroing. An FPI_ZEROED internal flag carries the hint through the free path. PageZeroed is set after __free_pages_prepare() clears all flags, so the hint survives on the free list. __SetPageZeroed is non-atomic but safe here: the page is frozen (refcount 0) and not yet on any free list. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/gfp.h | 1 + mm/internal.h | 1 - mm/page_alloc.c | 29 ++++++++++++++++++++++++++++- 3 files changed, 29 insertions(+), 2 deletions(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index 73109d4e31a4..d24b61e45861 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -384,6 +384,7 @@ __meminit void *alloc_pages_exact_nid_noprof(int nid, s= ize_t size, gfp_t gfp_mas extern void __free_pages(struct page *page, unsigned int order); extern void free_pages_nolock(struct page *page, unsigned int order); extern void free_pages(unsigned long addr, unsigned int order); +void free_frozen_pages_zeroed(struct page *page, unsigned int order); =20 #define __free_page(page) __free_pages((page), 0) #define free_page(addr) free_pages((addr), 0) diff --git a/mm/internal.h b/mm/internal.h index fd910743ddc3..4af5e72742ba 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -938,7 +938,6 @@ struct page *__alloc_frozen_pages_noprof(gfp_t, unsigne= d int order, int nid, #define __alloc_frozen_pages(...) \ alloc_hooks(__alloc_frozen_pages_noprof(__VA_ARGS__)) void free_frozen_pages(struct page *page, unsigned int order); -void free_frozen_pages_zeroed(struct page *page, unsigned int order); void free_unref_folios(struct folio_batch *fbatch); =20 #ifdef CONFIG_NUMA diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 702fa2ced1e1..399e2038dcd9 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -90,6 +90,13 @@ typedef int __bitwise fpi_t; /* Free the page without taking locks. Rely on trylock only. */ #define FPI_TRYLOCK ((__force fpi_t)BIT(2)) =20 +/* + * The page contents are known to be zero (e.g., the host zeroed them + * during balloon deflate). Set PageZeroed after free so the next + * allocation can skip redundant zeroing. + */ +#define FPI_ZEROED ((__force fpi_t)BIT(3)) + /* prevent >1 _updater_ of zone percpu pageset ->high and ->batch fields */ static DEFINE_MUTEX(pcp_batch_high_lock); #define MIN_PERCPU_PAGELIST_HIGH_FRACTION (8) @@ -1599,8 +1606,19 @@ static void __free_pages_ok(struct page *page, unsig= ned int order, unsigned long pfn =3D page_to_pfn(page); struct zone *zone =3D page_zone(page); =20 - if (__free_pages_prepare(page, order, fpi_flags)) + if (__free_pages_prepare(page, order, fpi_flags)) { + /* + * Only mark as zeroed if the page is actually still zero. + * kernel_poison_pages() overwrites with PAGE_POISON (0xAA) + * when CONFIG_PAGE_POISONING is enabled. With init_on_free, + * kernel_init_pages() re-zeroes after poisoning, so the + * page is still zero. + */ + if ((fpi_flags & FPI_ZEROED) && + (!page_poisoning_enabled_static() || want_init_on_free())) + __SetPageZeroed(page); free_one_page(zone, page, pfn, order, fpi_flags); + } } =20 void __meminit __free_pages_core(struct page *page, unsigned int order, @@ -3021,6 +3039,9 @@ static void __free_frozen_pages(struct page *page, un= signed int order, if (!__free_pages_prepare(page, order, fpi_flags)) return; =20 + if (fpi_flags & FPI_ZEROED) + __SetPageZeroed(page); + /* * We only track unmovable, reclaimable and movable on pcp lists. * Place ISOLATE pages on the isolated list because they are being @@ -3059,6 +3080,12 @@ void free_frozen_pages(struct page *page, unsigned i= nt order) __free_frozen_pages(page, order, FPI_NONE); } =20 +void free_frozen_pages_zeroed(struct page *page, unsigned int order) +{ + __free_frozen_pages(page, order, FPI_ZEROED); +} +EXPORT_SYMBOL(free_frozen_pages_zeroed); + void free_frozen_pages_nolock(struct page *page, unsigned int order) { __free_frozen_pages(page, order, FPI_TRYLOCK); --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9D9A5438FEF for ; Fri, 29 May 2026 15:25:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068324; cv=none; b=InmozmVF53ZAUk7ATGUwgCbwYtOtFi3tyfd07zZo016wM736gVX3BWR//kXH7MhnMQWNGnfqthPAO7F09IwkGisMt3jc3C/f/rK3rMhmWx1VYCsQmNJFRJJqcjHQmszqaEYsmI5SDxChPFpMIgP573kPrYpVwRhCGlROoyMmCfA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068324; c=relaxed/simple; bh=V5Y93H57q2xi9ZVdVLl8OLD+jeqpAWNwQzgfVZDhJj0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=iIvjptdMoogN4ZG91hsnK7uGsmN2cdyBAoRvSuef0eByFlnHWKTlSd5+OTxvull9RS5jtRAvCwwtgAepSQ3yXZl7XpTNq026kCeoMPcQN/ZanPcQff3RaK1fE9mC2ieC81Ewy1Ihhbj0hvdyZRNB1xhR2eD6EoYCTdDbsFZOnoc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=AGIKh9vf; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=q0JWnCIX; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="AGIKh9vf"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="q0JWnCIX" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068322; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=lR7ppjC8ZhT95gw1LzQhYf3JteDpTHqAJsB6tR6PXLg=; b=AGIKh9vfNI+zXMnouUkf+y0a58gOQnT3SFhUZLxdR8nuxQMUk5ODB+aNBEor1LmX3kpsbP Tz9nTaXZBpAEwnO4b8jio/tXvO2fxbJyXZAEVwsWyptMeMqdMe50ja64VqtFdSZt9nhpbi sJNuGsY+hAOiXQivQitqSw+wdF4+h70= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-382-SgwH1iOcO2maaiRVKzn3qw-1; Fri, 29 May 2026 11:25:21 -0400 X-MC-Unique: SgwH1iOcO2maaiRVKzn3qw-1 X-Mimecast-MFC-AGG-ID: SgwH1iOcO2maaiRVKzn3qw_1780068320 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-49047e9ca88so63694475e9.3 for ; Fri, 29 May 2026 08:25:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068320; x=1780673120; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=lR7ppjC8ZhT95gw1LzQhYf3JteDpTHqAJsB6tR6PXLg=; b=q0JWnCIX1jFA6K1B93VRFexRrcglnTmFZETuGPyWuocUxPbiMPhHOvf7UYiDS6fIDi dkmKV/GJ+fzn2kXpL4m6t4aWRAXPKg+1Du/+kVCThKr/U8xOYsmQpkkPCY5WruwHPrqF mpDxUa/BfdtSWTXnj6WxP3Iv9iVWkaD5Uwl9IsZxFeqzAhnKwYhh+3iSE3dMcpSgZx8i XMsH0czKUB6ZjlQTtzrIxoHnB/nRGakTI/YCBzjMzfxKanS2lA2eM6+c49L2PqfVY77V 7QliuzGzHLeTPq98CJkHropwZoGcYA8tSbLJxMDEJhnGndLUtICr0CXllhTv2C04SVNE hMiw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068320; x=1780673120; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lR7ppjC8ZhT95gw1LzQhYf3JteDpTHqAJsB6tR6PXLg=; b=VI6QARfoxx1FfuxpTHX91YSsUFCDLIeqgVUCmALCGncml1uh0wIzmOKLTXMZB4W+1W 1dPr3fGpNH/GuLVMdh8tLZp7AlbfrUpTA2aDAokAZwC4PaMdEG0UHI4xginVCnNicrk4 dLfaJrtsPoC9qI+rwhLUOz9R7Ch1MlG++f8ArmyuDWRkrRA3+g2uU7nM41xMypxzDxjJ WECDFI1LCMJ2VFOLcIQf+sr+Y/cMXm5W3Dm7LALfV3CFiJSrnkhAcTKItX9tI5RNpw7K SA6CkqJdtry3zrE6UDlbTHXwkljjVpLElexECBsGq9nLKQ9A5fiF/RYMjsnnBW2TzcLp EKzw== X-Gm-Message-State: AOJu0Ywecj2HSvJcO3nS/7S189hw9w7C0ou85u+qYYP71HIr/AohWUDu SzAGjyNg3xFl6MyxXMc0eXH2MTIxAAxmhkWmkNDPal48c83EQdqaMoURGsHCLQkc4ZfdsqMGGpU /mDvr1mVlXaZBemtQZiq6Is92nXZLkDyusZXw1ivj0z9/henEKJnDG0vz/Egof0KdclL2GH7jbQ uaRwjQwO7PozWfYiLeyGSO0C4w4NSPOvOO8FHaVboHrU0= X-Gm-Gg: Acq92OFeoNMUy7YGd35H8U9/n+UDdXtsaas7raVYrnDiOCRfK6yEMD4lEe0qXxoa1Ya DXHnKeirafYgQhJREOXIfSbi0Ma7afhmE2x2eG48Fv+b1pXaVyt8egExTHhch5eHYDsz4Z9As19 M7yD4aR8YiIXCwribzDpy8CPIu72kdwRZFyjwFapexHaqags7uUZqlnIT5CkIfxaU1twAi0oEaj nKw5ksxeMfS/KLjSJPSd045CqiEQRgYbP94LDjD1eu7/pd83yqXDqXdwO1LHiN9Np58Q4PM9HkP A4z6r6vWbWeZpnC+KdDlsMPuUwdO2UulA6Xh0JnsUA7aIzHYkae7/6axdNtRTiOaCIK70Wi2cC2 P2Sqra6TAR/PXwaV1q2vvJ9O7YdXyLI0dEKpwv+NAIZKz+LdEvgFgDA== X-Received: by 2002:a05:600c:c3c1:20b0:490:9d1b:2022 with SMTP id 5b1f17b1804b1-490a2938ad2mr1299495e9.13.1780068319886; Fri, 29 May 2026 08:25:19 -0700 (PDT) X-Received: by 2002:a05:600c:c3c1:20b0:490:9d1b:2022 with SMTP id 5b1f17b1804b1-490a2938ad2mr1298395e9.13.1780068319303; Fri, 29 May 2026 08:25:19 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-490a25e4fe7sm7967095e9.0.2026.05.29.08.25.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:25:18 -0700 (PDT) Date: Fri, 29 May 2026 11:25:14 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 34/37] mm: page_alloc: skip kernel_init_pages for FPI_ZEROED when safe Message-ID: <5c9d3afebc53d21cd13feb05b27f503f367d46b2.1780067978.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In __free_pages_prepare(), when FPI_ZEROED is set the page is already known to be zero. We can skip kernel_init_pages() if page poisoning is not enabled (because poison would overwrite the zeroes). This avoids redundant zeroing work when freeing pages that are already known to contain all zeros. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/page_alloc.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 399e2038dcd9..90a92be96ebf 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1447,7 +1447,14 @@ __always_inline bool __free_pages_prepare(struct pag= e *page, if (kasan_has_integrated_init()) init =3D false; } - if (init) + /* + * Skip redundant zeroing when the page is already known-zero + * (FPI_ZEROED) and page poisoning did not overwrite it. + * When page_poisoning is enabled, kernel_poison_pages above + * wrote PAGE_POISON (0xAA), so we must re-zero. + */ + if (init && !((fpi_flags & FPI_ZEROED) && + !page_poisoning_enabled_static())) kernel_init_pages(page, 1 << order); =20 /* --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6441F438FF5 for ; Fri, 29 May 2026 15:25:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068332; cv=none; b=EieM6WDzyUNOpvMtwf6zVJkGpoq1IgekvZTxOnDvrf9MfjwgkwgRGyxVXL8qIiHYpw5w8YRMHXL8nRQoF7i21WcwOFX6ZCLgXa+5IHwPiYysx/G1jpzFbGVFj8qkXbZwCRKb2zWVogFUD0Z3KA1lMMp6dsiNCmbsvKYWfIJbhR4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068332; c=relaxed/simple; bh=4jLzODooKNrRdgluzPWerRBde7vslo8ZvEbzyAfj6vs=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=ISVDsrpkjY4DX6gCfhBtBwbxew9bCzGkcaB5l4/Tfuor92Eczf2mLjSUaSr4UprlA4yNomyG81he+kVWddTWGSVQfCBluB4UFYT3SwdEk+w7Th5Gv7G0kd5lmjWWd+ocMrqEpPs/rMDhExOuhbNsRIW6smE7gn7XYKqlN/P/ZUc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=gUu1Hc0H; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=aUzcSMVY; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="gUu1Hc0H"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="aUzcSMVY" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068329; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=RfirkbM5TmZ61fE0cxqW40X8dDTC2oX5xU5kp1FvlUs=; b=gUu1Hc0HAnHS78irJmTo5NJLq/aeeiWVp/uuOS+xq0dD35JGzFv3cl7+f35K1ciKM+gJ0A L7vL6zQx/Oc77uLD0ubZDTdEc7SK2lpeZ/jLa9iUJk186D7qIoiWCjfcLIvFifOiEwCN8G QBC+WZ937aEffVFI/1LtShD8ioTBgp0= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-611-Eh4qUj1RORSLwAIn09h4iw-1; Fri, 29 May 2026 11:25:26 -0400 X-MC-Unique: Eh4qUj1RORSLwAIn09h4iw-1 X-Mimecast-MFC-AGG-ID: Eh4qUj1RORSLwAIn09h4iw_1780068326 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-45ef6417092so186647f8f.2 for ; Fri, 29 May 2026 08:25:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068325; x=1780673125; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=RfirkbM5TmZ61fE0cxqW40X8dDTC2oX5xU5kp1FvlUs=; b=aUzcSMVYTcG1a1AoBEOZITGRPzrodlZ6S8FCh5TOwuQxVP3MDZLE0SqYMlBlpTxsAD bUlvypdN4BclYIcVp8nQ5YjjjeZxDnC6+brRWOEBBXQyamza/AxxvG2XQYxOb0vPTfgz ysLM2lwYparz4wKPw5LiZQjb+uwU14Yb8DC3+WtbLlXKHOWA94nDv/zICbiaWQNBA5Tq DOq1O+760oIIqB+4NzjFUNmDcuOKkzmjEt2o2KvjwCip4VJDs+aQNOKhBKuNqEeuHQNo VZzwLJ2ExMgEx2tMlu0ttmx7mdzGsXbHYwyduBnMNJaI66tj9TYaL7PSKAy8VnbKAXWF pm1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068325; x=1780673125; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RfirkbM5TmZ61fE0cxqW40X8dDTC2oX5xU5kp1FvlUs=; b=OX1gZ7Zn4XA8n7AowoZxOsHCh6laNCmgJy3mDtmd5R7pw7/uhbe+/bNOzHoL95LiVj oE9BwqRDuXy82MnvO1XdDfdRlNpM8UeBirmqRKV8PtrDdeeS5D3IJR9Pqmvu8FVy4J5V Y8w2D9zWDQ/B95m3pjY6bmr5csukITpw5ls2mxh25U5wlKg921vC5JiXmSa45ZHgNRX9 gi3Z+6RvvBlBjlc90aU93T23e2RJbAhsHopyc5rRXG6xVS2fU6xGRHxljF4IBXRebzgw R77PTymKeYNEr6DqXG4xqATTZExB0pQ96IfTI0DeO1P96e2hQ2Wyf5iUE/8edGkromwz RkTQ== X-Gm-Message-State: AOJu0YxmVEXrbeNVL3f5yjqlV+OPELc5zddPt8GxyaNEBjqDm8C0lZ6+ jrQgrHQ+ogEjAaoyJCLGEdr+hwvrueOoOwc5GApgx+W2w1W3FDSdsRjxyCj0vUDQiVSndGWWiWB KqMZgsDQafFkNy/MhWbvXodZY1Z5adQ1Wo7h5qELCMAkHTyL/KMDb5ulIUHge3G8t6h6962BTVU kI59142UaE6ddYVnByKfMtov6J7SjHY95sMMNUvO4luvA= X-Gm-Gg: Acq92OEMra3GbmqxRVaE9jTcIXSIMAPW9lBWhwQoDkHqnzp75yyT1BkgiRHlSRPI7h3 oLDdw6moUmVIoXquwYx0+JNW3Mamjdgm3a098BMObgHAAn36tggfTIki7sUlKMonQTdqQkhphv4 gXfTSrB1VLvB6ChutPwJkmboZV3pYkccUjT9BzVgeFdUTu2xzk66ERDmUqx+ff1lxN3PP2nBCT/ GLstgkvYjTVtgH4C6eyEsOT+ihUefxToMfS0Clqh+oFum4xauyTj/N2Enu6r5HpoTD8ZlwrVWhq TqJ3nz4dh8MPuu7N82RKMuig58B/pWuF4DwskfSuRcsvGApkdWLYD2K2jBDvBDRxtBPwlruwKAq ikcg5XXZJASk7vxbSNqJaVxU3D6iVPv2DabvP+GFCIF84E0m42SwFzQ== X-Received: by 2002:a05:600c:450d:b0:490:778:4fe4 with SMTP id 5b1f17b1804b1-490a2952091mr1218605e9.26.1780068325379; Fri, 29 May 2026 08:25:25 -0700 (PDT) X-Received: by 2002:a05:600c:450d:b0:490:778:4fe4 with SMTP id 5b1f17b1804b1-490a2952091mr1217355e9.26.1780068324663; Fri, 29 May 2026 08:25:24 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45ef34b7d6bsm4731601f8f.10.2026.05.29.08.25.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:25:24 -0700 (PDT) Date: Fri, 29 May 2026 11:25:19 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli , Qi Zheng , Shakeel Butt , Youngjun Park Subject: [PATCH v9 35/37] mm: add put_page_zeroed and folio_put_zeroed Message-ID: <343a0d61f4b597791e3ebd16dffc5b91a0d733e0.1780067978.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add put_page_zeroed() / folio_put_zeroed() for callers that hold a reference to a page known to be zeroed. If this drops the last reference, the zeroed hint is propagated to the buddy allocator. If someone else still holds a reference, the hint is simply lost - this is best-effort. This is useful for balloon drivers during deflation: the host has already zeroed the pages, and the balloon is typically the sole owner. But if the page happens to be shared, silently dropping the hint is safe and avoids the need for callers to check the refcount. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/mm.h | 13 +++++++++++++ mm/swap.c | 20 ++++++++++++++++++-- 2 files changed, 31 insertions(+), 2 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index d1e768dcda13..86638ffaad01 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1913,6 +1913,7 @@ static inline struct folio *virt_to_folio(const void = *x) } =20 void __folio_put(struct folio *folio); +void __folio_put_zeroed(struct folio *folio); =20 void split_page(struct page *page, unsigned int order); void folio_copy(struct folio *dst, struct folio *src); @@ -2090,6 +2091,18 @@ static inline void folio_put(struct folio *folio) __folio_put(folio); } =20 +/* Caller must be sole owner to guarantee page is still zero */ +static inline void folio_put_zeroed(struct folio *folio) +{ + if (folio_put_testzero(folio)) + __folio_put_zeroed(folio); +} + +static inline void put_page_zeroed(struct page *page) +{ + folio_put_zeroed(page_folio(page)); +} + /** * folio_put_refs - Reduce the reference count on a folio. * @folio: The folio. diff --git a/mm/swap.c b/mm/swap.c index 5cc44f0de987..ecec780172ad 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -94,13 +94,15 @@ static void page_cache_release(struct folio *folio) lruvec_unlock_irqrestore(lruvec, flags); } =20 -void __folio_put(struct folio *folio) +static void ___folio_put(struct folio *folio, bool zeroed) { + /* zeroed hint ignored for now, no current user */ if (unlikely(folio_is_zone_device(folio))) { free_zone_device_folio(folio); return; } =20 + /* zeroed hint ignored for now, no current user */ if (folio_test_hugetlb(folio)) { free_huge_folio(folio); return; @@ -109,10 +111,24 @@ void __folio_put(struct folio *folio) page_cache_release(folio); folio_unqueue_deferred_split(folio); mem_cgroup_uncharge(folio); - free_frozen_pages(&folio->page, folio_order(folio)); + if (zeroed) + free_frozen_pages_zeroed(&folio->page, folio_order(folio)); + else + free_frozen_pages(&folio->page, folio_order(folio)); +} + +void __folio_put(struct folio *folio) +{ + ___folio_put(folio, false); } EXPORT_SYMBOL(__folio_put); =20 +void __folio_put_zeroed(struct folio *folio) +{ + ___folio_put(folio, true); +} +EXPORT_SYMBOL(__folio_put_zeroed); + typedef void (*move_fn_t)(struct lruvec *lruvec, struct folio *folio); =20 static void lru_add(struct lruvec *lruvec, struct folio *folio) --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 505DE3F86FB for ; Fri, 29 May 2026 15:25:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068335; cv=none; b=UOAm+7sTWiHww7AbmPndeTmH0Co6bf5XmmZpXMWdzpZpbduXt0Ir7BTMYzFI/4R6BIS8dCMq6T+zShg/twITzzF7NTs9ZhVA9o1PHYwCfoUDwu3lec/hepSEPmaXKkCewX20pSvHD48nWgEXQWQf92v4O5mAlQBEKNWNo40PZ9o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068335; c=relaxed/simple; bh=Z+HYjvvqCchjYSSoU6qbOk9Gov2pfNet7tVjjUNszJc=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=kjzdQ0aDtNUiqsquwVp2nJqihZMmYqxSTfCyFqpoQFTc56r+5qthpA2foOsSbdb6/Y3+/wCpmkOQrJgGFMtd+pOMS0QRbRnxg6TtJ74J26pffJPL+tTgIfER8aBeKXfmUlatjNVUkBSW3F1ycycc+1TYswuUzdEv1dXA/pLPlNI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=BmMhnZBc; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=ekUhgd8j; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="BmMhnZBc"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="ekUhgd8j" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068333; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=PJ5vvVKjiUlKZvlIAO4D+aYwhndM/ly8HIEHblxR0ik=; b=BmMhnZBcbnZeooEGRJWEDpBFjKaMXv7WG4JWRasxcUPOAk18QDsLlWe6R2tUeIAG9UoCsw ZtaCNLgeCjYYOy9Px8hqoxQxR6g2xmzlydPcbeaSb0lddtVB+XeAgW8KxsmbSmJvYGaHln I4xMjB/ocJif38B9oTDJ9foErIGrHX4= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-325-26sApI2SOPq9i24u0GhraQ-1; Fri, 29 May 2026 11:25:31 -0400 X-MC-Unique: 26sApI2SOPq9i24u0GhraQ-1 X-Mimecast-MFC-AGG-ID: 26sApI2SOPq9i24u0GhraQ_1780068331 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-48d127eb013so67712875e9.1 for ; Fri, 29 May 2026 08:25:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068331; x=1780673131; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=PJ5vvVKjiUlKZvlIAO4D+aYwhndM/ly8HIEHblxR0ik=; b=ekUhgd8jLS5PtptZrkDO5CNctLYHpWZ21mWlFJ/CSEFXSbjyLI77B6iAMi0776MSEv KKM2iFvBkayVpkgvLCtc5Ns7MDXBg3P12WkWZo7n9B91HRpWlHC/YVgsXwknrXKNdQ6C eKvvmPzWWhEmu1/anP5/0taJp4zNHSYso4PkUwz8gXQnkGH0k+6EOVvlIYs0H8Ru88c8 veZVZ00h6o6Sk3SfpaarYGsF3E1rbtNAc3sB/bfXx7WG4yVBgoSUFXyHkx1zzcZtfZ+P XOWJClrJTjVk3nFDsX+M4bHOmeC5PUZ0NGxkmOiJNrnOfx2kyWt1ZsNCJ5B3EaQhddvk W1Cg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068331; x=1780673131; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PJ5vvVKjiUlKZvlIAO4D+aYwhndM/ly8HIEHblxR0ik=; b=g9452kHG9NcbK96PTiUjcsTAKb+7g6PDGih/IeSz3cDe0y65Mv00Iq4mP0q+zadGkB ycC1wC/gjITq24zSv8Z7y4HN4TkmsIfOWsncs30D96NVhH6ltpQwxsg7s/ra2NcOJeTN F9+GsImdmFbJZiQyEAKaUIL705UKKvJPOyOM5wMdM2eVzqzJ0AwgRIs3v8J7yo0IBKk+ wF6G0ubmug00el8MvhELuvwZ1ntNuLfWc6QRRtHqVHbKogtZ/DdDGMBLmnbY2BL/dY5g GZbutswJxYah7WFBjyniFE+amOIjAVzJSmFumSjpb0UFmWivsBGLPZdVS/awV+SzpjC7 W32Q== X-Gm-Message-State: AOJu0YxvOseXKQYcTC/ZZVusmbRTeB/b+337cOa6yQEZwrKheJXr0t3w YpG4vpduLvcuTHvz9IORZSCyI9KAjE9kOM8ncnmqXte5WWFOERHn8Pn7IfEn61Njvf1wkOGFW0J CFD/sC51zEAjHQesbrmTpzFcV94MXeiI8vQDqykQ0GWyT0+fl+TZSdGg03x4w3RivsvHT8XWcoW mEdsNjpmyV1UUxqJP7vWH1Z8WT0yvKrf9AWIMn0x+wOmM= X-Gm-Gg: Acq92OHRIxnaa2rjJphibTLNGk+QdN6TmN5jpdk+/ClTy6+eDlhx1Q9pcQkA/Fo/mSn MjX3uKWsbM/WHkAraHSOWNFGIFY+FWpjCQXtKbBly9N1CKyxijkMeIERaoyw/jrsN7ANJa5tMsk j9nmoKW87h2MHUqAgP5INGdj6NRVuXJjW470WO0gFyRP9r/tU54bD1i1rrUaWsHn63EdUHax6yL ZLTOAGP7eN/P7Oak7ue4nut7L+t9nmxCT3DwpxJpH8KbLHWoXJWjruVA9Pz6wQOreMEfFDbRV2D OjBqvsUnkZZVyU3b1AdViqHBMfTIwAu/OsTrZiqLt4dU6yDqm2ZFbtzlnvoGb3AoDxJo3WQCIJG MbL8zncdAMUVPjwOZQs/Y8aX1c6bDWTB3kttWNk+NbaQkPOiLWT7UyA== X-Received: by 2002:a05:600c:4f89:b0:490:4717:970f with SMTP id 5b1f17b1804b1-490a292ec27mr1653895e9.14.1780068330495; Fri, 29 May 2026 08:25:30 -0700 (PDT) X-Received: by 2002:a05:600c:4f89:b0:490:4717:970f with SMTP id 5b1f17b1804b1-490a292ec27mr1652655e9.14.1780068329671; Fri, 29 May 2026 08:25:29 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4909d6a0a89sm62305115e9.7.2026.05.29.08.25.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:25:29 -0700 (PDT) Date: Fri, 29 May 2026 11:25:24 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 36/37] mm: balloon: use put_page_zeroed for zeroed balloon pages Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When a balloon page marked PageZeroed is freed during migration, use put_page_zeroed() to propagate the zeroed hint to the buddy allocator. Previously the hint was silently lost via plain put_page(). No page has PageZeroed set yet; the next patch (VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE) will set it on pages the host has zeroed during inflate. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/balloon.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/mm/balloon.c b/mm/balloon.c index 96a8f1e20bc6..6c9dd8ab0c5d 100644 --- a/mm/balloon.c +++ b/mm/balloon.c @@ -324,7 +324,15 @@ static int balloon_page_migrate(struct page *newpage, = struct page *page, balloon_page_finalize(page); spin_unlock_irqrestore(&balloon_pages_lock, flags); =20 - put_page(page); + if (PageZeroed(page)) { + /* Atomic to serialize with memory_failure's + * TestSetPageHWPoison; not under zone->lock here. + */ + ClearPageZeroed(page); + put_page_zeroed(page); + } else { + put_page(page); + } =20 return 0; } --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2ACA143C069 for ; Fri, 29 May 2026 15:25:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068342; cv=none; b=uIWuTOoVWdArjzRYod4pAWEbBuGoDQ04MzPZ2Hq6x8g63/qiUI8MStC88PESr/SNVcs8aoihAbCjVezhWnj++c2402VDWJIsKLlrs+unuAFht5sd1yNhrHZAHSwy1t2mP9lASZ/ji4kIqXnS2rSE1tcl+/hTceyTCbaCFHHfKy0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780068342; c=relaxed/simple; bh=6pCnJ6k6szHuGciBTfw9TfV267L0cdUbYVFf2bw/M5g=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=EpN9VfivK3it9I7X8ydmcIHi9MHu5gQcT8uo5HXXlcrpRKj90QzbHe0A3lTJOb8PZYKQFtNtu564B7yD6Odkd0mfKgNaYps8v9O4KwYI5qM95Hd9ENdHlHf924NbFST11J4NmpVJCnI7D/QdZ5djxHETmsqICjvqCxzKvu1nCq8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=XvYqZOx4; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=PwsXG2F2; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="XvYqZOx4"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="PwsXG2F2" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780068339; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=a5iRhiSdz01WKxZI1toNz5FYne+GoBDYJal3naikSkE=; b=XvYqZOx4yErVeXnihZN0JE8MAo7dvCvn1uwTafiYPgzZyTUc9VNyWIM+CsJWsS1Tijo9Hh comUuRAoLRKUKJwDwvKUrdHg2bXRVJFOcoZnyBsF5djzn5Pzqh8p2louTH46fbqfrU5ycH UP+udz4j7d8TOrmUsBDb3JIx3AESeEE= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-37-7eMOS2NGPiyv0vp_fxyvEQ-1; Fri, 29 May 2026 11:25:37 -0400 X-MC-Unique: 7eMOS2NGPiyv0vp_fxyvEQ-1 X-Mimecast-MFC-AGG-ID: 7eMOS2NGPiyv0vp_fxyvEQ_1780068336 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-490402ae2c1so77894155e9.0 for ; Fri, 29 May 2026 08:25:36 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780068336; x=1780673136; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=a5iRhiSdz01WKxZI1toNz5FYne+GoBDYJal3naikSkE=; b=PwsXG2F2Fwe7DrVmyScnuK1d+HMNSzVyB/+ZannH38HT0udqM1V+tmNSjDl4LBNr8H e2hAs74eRABvskcgNSLyoIY1zaYQjtlbxaFeV62sflhtfB8AQliPj6sS3gBSwPZfeVoY /Xh5O4AOHHtlzeivVa8el5eF5j+kTbm5Say94+QufJfAKdegAjHcLoencjeKuDo/iUYW m269sBpKqDLngKtPXVbnKx8b6rvBHUNvy8Wld68LhOjBqu82+pNXzbLqJZDwNg7u1vBz eGcdtI8j8vmReO/7vbyhD3D6UlImy8Y01cMvffN5BZ5f9am7dBd8TM1lXYil0DKLxK5I QtFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780068336; x=1780673136; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=a5iRhiSdz01WKxZI1toNz5FYne+GoBDYJal3naikSkE=; b=E1IR9sSxOK5V81VmnZ2KapMtTEHbOAVFOgkVF0qwIktDT9mhgVFS3Y59fiyCac3Upw fLN3Et3x5ZiucXC81fDaU3cPV+IVFkXMq4XK6pdvm4CGhJZ3UJfbvT1283UJYpdceA6b fhmwPvO7c0RXwaaDIHt157dfbozrHBLkbFgDJOBxcjNNueCbvcfwW2Q4o6bny2Rm1U6c DrvWypYYg+PPUQ2BjQUI1A07+B/oIRVWaEYqdcwpzTTfwAefh5jIpZFqnjlwjauDw4iI zZzvtaab+Ytx9RqQAqHkc8kvAu2/s83JyNy22uo0nCsnRra+Swqtj+4oI4gFp2GY0t3t oNTQ== X-Gm-Message-State: AOJu0YwZ61j+b+TZO859vQHvXX1rDVi3o/db7sIoFoXOMJ8LJ9Ju5XEL fIHBAeYYkv8C5ev+b/WUo+fAsZ8kkSVSCdrgJHyl0DoQQNdI7oRNSeeMG15RiqtRPpdwa0ixRrG kR+FRjaQxXRwT9rXeyK3z9XRLPML+IB9b3zkQ1CgwZqnGLtGRit9QQ1fqSYsIZtqHSm3EXzbCge gG6EK4GIhKaVmsubq+JLU7DOBDstYYDUDY/IqCTSK4CgU= X-Gm-Gg: Acq92OG+JWxlAiXl31W84yJqmtDSOwiQxzPoNpF3VsatL1rwfexvg0yjtjjFlIoa9nV IEH6YR0MgCGU+gFoKOP3RNx5dX++6RBqOQPUqDbIdc5pbMbKwgwFHLnmH04nGHp+ZDNFBpcuZNx ENDvAKUluQtMbPADf/Fp5DMWNh+Ze7J3PEZjeGZejDf9M8YLBag/GidQUSGZxA8kamXd9JhDFVj whMGz7TR6r0dpDCDjv5MteWUjVOdeeB3b1uXkZwXFFrV2U1SKROq6EGn5XN1Iwm57e0H7Uc/n3p 6hfhzlquQHPMc55IDxST83Rs0XnNktKh5fTX2oovRnmy7K8EBLY8bunil7fyyQQs4OnYVNLi4xL 1kZ90dxELK8i5kgBxHf608ZhPSKLeIexTuwq6Omwm+0L7Yk8XWHtCLQ== X-Received: by 2002:a05:600c:3b18:b0:490:8b0b:d3b1 with SMTP id 5b1f17b1804b1-4909c62909bmr43943805e9.12.1780068335708; Fri, 29 May 2026 08:25:35 -0700 (PDT) X-Received: by 2002:a05:600c:3b18:b0:490:8b0b:d3b1 with SMTP id 5b1f17b1804b1-4909c62909bmr43942545e9.12.1780068334819; Fri, 29 May 2026 08:25:34 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4909d6975e2sm75889295e9.6.2026.05.29.08.25.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 May 2026 08:25:34 -0700 (PDT) Date: Fri, 29 May 2026 11:25:29 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v9 37/37] virtio_balloon: implement VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE Message-ID: <37a5da77b003cfecc4e46eb1886f46b4b79fb95b.1780067978.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When the device offers DEVICE_INIT_ON_INFLATE (bit 7), the device initializes inflated pages and returns a per-page bitmap indicating which pages were successfully initialized. The driver appends a device-writable bitmap buffer to each inflate descriptor chain via virtqueue_add_sgs. After the host acknowledges, the driver checks bitmap bits (bounded by used_len) and marks pages with SetPageZeroed. tell_host() returns used_len from virtqueue_get_buf(). Bitmap reads are bounded: fill_balloon() and virtballoon_migratepage() only trust bits within the used_len range. On deflate, release_pages_balloon checks PageZeroed per page and uses put_page_zeroed for pages the host initialized, propagating the zeroed hint to the buddy allocator. If inflate_vq has fewer than 2 descriptors, probe fails with -ENOSPC. If PAGE_POISON is negotiated with non-zero poison_val, the feature is cleared in validate(). See the virtio spec change: https://lore.kernel.org/all/9c69b992c3dd83dfef3db92cd86b2fd8a0730d48.177773= 1396.git.mst@redhat.com __SetPageZeroed on inflated pages is non-atomic but safe: the balloon owns the page exclusively at this point. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- drivers/virtio/virtio_balloon.c | 107 ++++++++++++++++++++++++---- include/uapi/linux/virtio_balloon.h | 1 + 2 files changed, 96 insertions(+), 12 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index bf1172ad5419..4abc7b3b991c 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -20,6 +20,7 @@ #include #include #include +#include =20 /* * Balloon device works in 4K page units. So each page is pointed to by @@ -122,6 +123,13 @@ struct virtio_balloon { struct virtqueue *reporting_vq; struct page_reporting_dev_info pr_dev_info; =20 + /* No DMA alignment needed: ACCESS_PLATFORM is cleared, + * so virtio bypasses the DMA API. If this ever changes, + * add ____dma_from_device_aligned here. + */ + /* Bitmap returned by host for DEVICE_INIT_ON_INFLATE */ + DECLARE_BITMAP(inflate_bitmap, VIRTIO_BALLOON_ARRAY_PFNS_MAX); + /* State for keeping the wakeup_source active while adjusting the balloon= */ spinlock_t wakeup_lock; bool processing_wakeup_event; @@ -182,22 +190,33 @@ static void balloon_ack(struct virtqueue *vq) wake_up(&vb->acked); } =20 -static void tell_host(struct virtio_balloon *vb, struct virtqueue *vq) +static unsigned int tell_host(struct virtio_balloon *vb, struct virtqueue = *vq) { - struct scatterlist sg; + struct scatterlist sg_out, sg_in; + struct scatterlist *sgs[] =3D { &sg_out, &sg_in }; unsigned int len; =20 - sg_init_one(&sg, vb->pfns, sizeof(vb->pfns[0]) * vb->num_pfns); + sg_init_one(&sg_out, vb->pfns, sizeof(vb->pfns[0]) * vb->num_pfns); =20 /* We made sure the vq is large enough so we should always * be able to add one buffer to an empty queue. */ - virtqueue_add_outbuf(vq, &sg, 1, vb, GFP_KERNEL); + if (vq =3D=3D vb->inflate_vq && + virtio_has_feature(vb->vdev, + VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE)) { + unsigned int bitmap_bytes; + + bitmap_bytes =3D DIV_ROUND_UP(vb->num_pfns, 8); + bitmap_zero(vb->inflate_bitmap, vb->num_pfns); + sg_init_one(&sg_in, vb->inflate_bitmap, bitmap_bytes); + virtqueue_add_sgs(vq, sgs, 1, 1, vb, GFP_KERNEL); + } else { + virtqueue_add_outbuf(vq, &sg_out, 1, vb, GFP_KERNEL); + } virtqueue_kick(vq); =20 - /* When host has read buffer, this completes via balloon_ack */ wait_event(vb->acked, virtqueue_get_buf(vq, &len)); - + return len; } =20 static int virtballoon_free_page_report(struct page_reporting_dev_info *pr= _dev_info, @@ -300,8 +319,37 @@ static unsigned int fill_balloon(struct virtio_balloon= *vb, size_t num) =20 num_allocated_pages =3D vb->num_pfns; /* Did we get any? */ - if (vb->num_pfns !=3D 0) - tell_host(vb, vb->inflate_vq); + if (vb->num_pfns !=3D 0) { + unsigned int used_len =3D tell_host(vb, vb->inflate_vq); + + if (virtio_has_feature(vb->vdev, + VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE)) { + unsigned int i; + unsigned int valid_bits =3D used_len * 8; + + for (i =3D 0; i < vb->num_pfns; + i +=3D VIRTIO_BALLOON_PAGES_PER_PAGE) { + unsigned int pfn, j; + bool zeroed =3D true; + + if (i + VIRTIO_BALLOON_PAGES_PER_PAGE > valid_bits) + break; + for (j =3D 0; j < VIRTIO_BALLOON_PAGES_PER_PAGE; j++) { + if (!test_bit_le(i + j, vb->inflate_bitmap)) { + zeroed =3D false; + break; + } + } + if (zeroed) { + pfn =3D virtio32_to_cpu(vb->vdev, + vb->pfns[i]); + __SetPageZeroed(pfn_to_page(pfn >> + (PAGE_SHIFT - + VIRTIO_BALLOON_PFN_SHIFT))); + } + } + } + } mutex_unlock(&vb->balloon_lock); =20 return num_allocated_pages; @@ -314,7 +362,12 @@ static void release_pages_balloon(struct virtio_balloo= n *vb, =20 list_for_each_entry_safe(page, next, pages, lru) { list_del(&page->lru); - put_page(page); /* balloon reference */ + if (PageZeroed(page)) { + __ClearPageZeroed(page); + put_page_zeroed(page); + } else { + put_page(page); + } } } =20 @@ -861,8 +914,27 @@ static int virtballoon_migratepage(struct balloon_dev_= info *vb_dev_info, /* balloon's page migration 1st step -- inflate "newpage" */ vb->num_pfns =3D VIRTIO_BALLOON_PAGES_PER_PAGE; set_page_pfns(vb, vb->pfns, newpage); - tell_host(vb, vb->inflate_vq); + { + unsigned int used_len =3D tell_host(vb, vb->inflate_vq); =20 + if (virtio_has_feature(vb->vdev, + VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE) && + used_len >=3D DIV_ROUND_UP(VIRTIO_BALLOON_PAGES_PER_PAGE, 8)) { + unsigned int j; + bool zeroed =3D true; + + for (j =3D 0; j < VIRTIO_BALLOON_PAGES_PER_PAGE; j++) { + if (!test_bit_le(j, vb->inflate_bitmap)) { + zeroed =3D false; + break; + } + } + if (zeroed) + __SetPageZeroed(newpage); + } + } + + __ClearPageZeroed(page); /* balloon's page migration 2nd step -- deflate "page" */ vb->num_pfns =3D VIRTIO_BALLOON_PAGES_PER_PAGE; set_page_pfns(vb, vb->pfns, page); @@ -966,6 +1038,12 @@ static int virtballoon_probe(struct virtio_device *vd= ev) if (err) goto out_free_vb; =20 + if (virtio_has_feature(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE) && + virtqueue_get_vring_size(vb->inflate_vq) < 2) { + err =3D -ENOSPC; + goto out_del_vqs; + } + if (!virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_DEFLATE_ON_OOM)) vb->vb_dev_info.adjust_managed_page_count =3D true; #ifdef CONFIG_BALLOON_MIGRATION @@ -1191,11 +1269,15 @@ static int virtballoon_validate(struct virtio_devic= e *vdev) =20 /* Device fills with poison_val, not zeros; disable zeroed hint */ if (virtio_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_POISON) && - !want_init_on_free()) + !want_init_on_free()) { __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED); + __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE); + } =20 - if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) + if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) { __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED); + __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE); + } /* * Balloon submits 1-2 sg entries max per buffer, virtqueue * sizes are 128+. Disable indirect descriptors to avoid @@ -1215,6 +1297,7 @@ static unsigned int features[] =3D { VIRTIO_BALLOON_F_PAGE_POISON, VIRTIO_BALLOON_F_REPORTING, VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED, + VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE, }; =20 static struct virtio_driver virtio_balloon_driver =3D { diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virti= o_balloon.h index 13074631f300..cbaf18e0b17c 100644 --- a/include/uapi/linux/virtio_balloon.h +++ b/include/uapi/linux/virtio_balloon.h @@ -38,6 +38,7 @@ #define VIRTIO_BALLOON_F_PAGE_POISON 4 /* Guest is using page poisoning */ #define VIRTIO_BALLOON_F_REPORTING 5 /* Page reporting virtqueue */ #define VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED 6 /* Device initializes repo= rted pages */ +#define VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE 7 /* Device initializes pa= ges on inflate */ =20 /* Size of a PFN in the balloon interface. */ #define VIRTIO_BALLOON_PFN_SHIFT 12 --=20 MST