From nobody Sun May 24 22:35:50 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8E07D3C060B for ; Wed, 20 May 2026 22:20:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315624; cv=none; b=IH82nWcdvmXKBkmriwaSOZLb0/coXhWggiSzWbh4YelplYNS/+lq6nSsSd41G6/036+ZCHi1p4TjL3vcr5hlXJV+HWJrLAV83uUI1k3AV3c5M922aCtX4zN+0CyGGDZMeYvIguHWnVZMRu4a2C+2mZqURqK2nOJu1oRLkhWEi5Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315624; c=relaxed/simple; bh=ivSgG/97imSLXWE5yAu9GDoEGm+2i+hvCGOloJMpUro=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=L6gwCiiA2OTXO8M4uxfW6azCbnDxXEOSRtOrCkyyYrlwel5dp0GMG76lNiQKpeUn8s38N0G5yC+YUEnJCEF/+rGh2H6aHH3ysk6Pd8T2luaUKzEY1s6nwm4ftQNdDJkb9pkN99It5t57TPIP80k015UaUYPtc/iYBJfpQ53ifgk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=QjOFcpWd; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=X1UKZ6yI; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="QjOFcpWd"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="X1UKZ6yI" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315621; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=piBMwx0YNR4u+eZF7dD5f/57awj2BSKfWqeLUqtqejA=; b=QjOFcpWdQNVCSnM5ANpyENzcuvc0JmLtdWpXpftMJ22M3p/0SlFUzIium+CpizjWR2QR2B kiWs9kgdAysHqx+EBPHO+FeavTz88L+5wAE/ZMhXggjQ3i46mLti/p+R0FDHzJXNnAVHru LnM3I4by3qouVHb13JZV+u3uz03PxiA= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-471-bFX_wMnkNGGXLkoY1gRePw-1; Wed, 20 May 2026 18:20:20 -0400 X-MC-Unique: bFX_wMnkNGGXLkoY1gRePw-1 X-Mimecast-MFC-AGG-ID: bFX_wMnkNGGXLkoY1gRePw_1779315619 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-48eb0da933fso39134635e9.0 for ; Wed, 20 May 2026 15:20:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315619; x=1779920419; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=piBMwx0YNR4u+eZF7dD5f/57awj2BSKfWqeLUqtqejA=; b=X1UKZ6yIeBqFw4g2FYk2VCN9WAHPT6fEwJtuWqfhW0fd4/EdHoe8XtZf+IEZq1Hf1i GEp/YQpH2inZJsysIjtnEPo5Bp2BCYds+hsW8umcL5HneTL4Pgu6UrIIt71Jw2FINh3r 6Qoad4hXsQPwjWYbHMwd5NP2jvnAH8uo6xOL/N5RkHc9QpGzPoPDD0c+Rylkt2XJv23s RUWFHsLtiV8ojwoGi9p8Sv65XeFUex3K1P3VMqxwoygSHfMqYFe9ZxdY8kxF7w6KfIPy kfGnDP+0m1u0O3EupgmqyW7AwqhPRSpUk3UWwC03Oe7TRR/L3GMH3LsC7fnYkOZLz4Mj LCOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315619; x=1779920419; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=piBMwx0YNR4u+eZF7dD5f/57awj2BSKfWqeLUqtqejA=; b=o0z1u7Iuhq83qyqvsJCnYdX304VrXjHnzdBFKGqkcmukkteZF4axje7G+7UI9QT7Is mB+jjhZA4Ggb992/lycpEeSVEB0F4f+T/6p4/fPk0BB9uVWE48RXNsAzkv1vWBf34tB9 jMgAAQZwef0BxNRLd5S5C+wPYDSuPQiWdYb+heoOXSobEObeF5UVuBv72I1eDzy2zPOI Q39W+7l8pPAUVtZfOtwQDDhu8ZmwZqcvLXc5R4gh9wJmLtCfTpnN/GKmMldAMBEG0ySl 0MgH50oWCFxFIejAqr3SEsg5Hxt+A3d4EE0Z9jRydce56RNkxmQmo9RUyz5MTU27Drxh BM5A== X-Gm-Message-State: AOJu0Yy/8zOcZ2gntYAU8B1Ho6cJNS1scNz0IwRAgM3B5h/nMDzUNN1o +8WIKITgSrW4gVUuJR/DSomJ8x1iv3hvGKBjhjrz9Q1BhU7Y4dfatBjF0325ItIUcMUx3sFFwvj Hv//0VILCNZ8W6cI5PsjM7nJu42CGpTKHNfWkN8zPsyYvmsmP9XIjU6wJvCaAS9o+PXpdztjwoZ oEV+v8Ss31+m08gcU79j+y1wAZltAAXU1Es6uMBg9HPtE= X-Gm-Gg: Acq92OF3I+035QyKL96LHuXEAGWo+Iis/3OWygkTQ7qT6t+yEXuSyo1ZSl+6e/xK3Lt skSgpLhKA3Vp34hAm5ZwD3WXbjtLLBBwffKLbnOv3X62TbzpKg9tU1C/sGysERv9wcCOoTzutoD RKPvFY04JudDTSvbCkid9LU3p/wNz8TzTJrjwLCwhuZPPFtdj2VavEtnYgWGVsp/xoM4OhdzGBf yEBq9Uvm3zXHeeZYJ5vKNN8JWqFlOYsE89zEiwaoXwtrBleY06wL8JqwMsKZ7Eed1ftsLksEUB9 6+LY+89w8otdBqniTMqEuZzQMMRcJKT4uy+2QYfMJTKCdzhwiOA7gL5S+ebGF6l5LXFjXjzhW2j Tu3ccQp+Evykf91B6vwNRFPfPpPH8rZwtGL1ZiO+xPaU= X-Received: by 2002:a05:600c:4753:b0:48a:7a10:4f17 with SMTP id 5b1f17b1804b1-490360485f7mr2619535e9.6.1779315618733; Wed, 20 May 2026 15:20:18 -0700 (PDT) X-Received: by 2002:a05:600c:4753:b0:48a:7a10:4f17 with SMTP id 5b1f17b1804b1-490360485f7mr2618645e9.6.1779315618045; Wed, 20 May 2026 15:20:18 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49033d4ad9bsm18916095e9.8.2026.05.20.15.20.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:20:17 -0700 (PDT) Date: Wed, 20 May 2026 18:20:13 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 01/37] mm: mempolicy: fix interleave index for unaligned VMA start Message-ID: <2cc8a28608f548d582c4e5609eb036ac24f7ae6f.1779315441.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The NUMA interleave index formula (addr - vm_start) >> shift gives wrong results when vm_start is not aligned to the folio size: the subtraction before the shift allows low bits to affect the result via borrows. Use (addr >> shift) - (vm_start >> shift) instead, which independently aligns both values before computing the difference. No functional change for current callers: the fix only affects NUMA interleave and weighted-interleave policies. Current large-order callers either pre-align the address (vma_alloc_anon_folio_pmd) or do not use NUMA interleave (drm_pagemap). All other callers use order 0 where the old and new formulas are equivalent. However subsequent patches in this series add large-order callers that pass unaligned fault addresses, making this fix necessary. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Reviewed-by: Gregory Price --- mm/mempolicy.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 4e4421b22b59..b2c21ed1fd84 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -2049,7 +2049,8 @@ struct mempolicy *get_vma_policy(struct vm_area_struc= t *vma, if (pol->mode =3D=3D MPOL_INTERLEAVE || pol->mode =3D=3D MPOL_WEIGHTED_INTERLEAVE) { *ilx +=3D vma->vm_pgoff >> order; - *ilx +=3D (addr - vma->vm_start) >> (PAGE_SHIFT + order); + *ilx +=3D (addr >> (PAGE_SHIFT + order)) - + (vma->vm_start >> (PAGE_SHIFT + order)); } return pol; } --=20 MST From nobody Sun May 24 22:35:50 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 125643BF665 for ; Wed, 20 May 2026 22:20:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315628; cv=none; b=Vo+OWPb9VZ4lMSDWSIdY9W8QccFnDgSOXP5mnu1ST5V6LsPyo2UCV7vXIjqxgx5mDR0B+ukEYtI8GJcA/j37p2P/ZohrmLBUFrh5yvOXnY6L6x6AqzY93X7+NomEVQ6lkhzavkMJxYHeUO0QMLvQKlokcfc/1VDeDgpOCJe70sg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315628; c=relaxed/simple; bh=BFG0o0OJVbxM+Mfp5F1a0Qegi36i+qWUB4aV2HzNESs=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=cbFkBgtu4tDJINc4+F9xn1wfUSpw4ZggpBUQwrHb7haplNZ4qkhCNnQ9sKYRFfJ5oe/Y5svSLInEhzCIBtgFhJNl5sOjkBubdtM3FVgrSzFwe0pHvM0mJ38FnDZIewSjjtbu5IVVnhB9Cks8GRzLBDL2PQn5mVkKTMd2R7u+zjQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=iNV7drXH; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=UPZU/PzN; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="iNV7drXH"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="UPZU/PzN" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315626; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=eAu/RRdfYrA/U3um2PuIXBOQM7QYu96dWR4U/hhBwhM=; b=iNV7drXHLnEhmTBIW1QD730Ep3o9ZKkk7Gb8DQ/fbH/XB6soBuxegJuQOjgtoq34sJ7uum FwcrpkNbpGS7C9fN65bpp7PMLH7BWuCpY/MF6JSyz5GMRe6YYgDQhG98JX9xbMON1rm/d9 dDJlqQJMHFsOlrWXVu1lnRCBMqrdbHk= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-653-NvgLiyqTPtmyGqCA5dLmqw-1; Wed, 20 May 2026 18:20:24 -0400 X-MC-Unique: NvgLiyqTPtmyGqCA5dLmqw-1 X-Mimecast-MFC-AGG-ID: NvgLiyqTPtmyGqCA5dLmqw_1779315624 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-490261c79fcso11308675e9.0 for ; Wed, 20 May 2026 15:20:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315624; x=1779920424; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=eAu/RRdfYrA/U3um2PuIXBOQM7QYu96dWR4U/hhBwhM=; b=UPZU/PzNN+qvF+NvKH/SP0i2AfC05V2OquMPyv0MbGD+cLf4ixQBgtYSwAUwFkmzNe 8Dd6/q1O1ZhJNOSKY5RQqkzKTHZOscCqltSSBeWmNJK0UiZ3jZ8rUZnNpUL1u8dtQyD7 hWwmvQZg9t25iBb3jDbI2k7eXV0Obb0vqrqT8/sowlibxETuR9ptCMWUzZ5jdm4WzjAT bdtB3kFAVXpKXkL3HopCLFhPucooNYaJgyomyICy5YpQJMejUWZStD4La9d+x7NitJ/F tgCpMMGLsOllw+yabeR5slP4OvG+D5wJcmFYXt1QdG0bvQG4scXBtjRx7Dvt9xVT3yTQ ohyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315624; x=1779920424; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=eAu/RRdfYrA/U3um2PuIXBOQM7QYu96dWR4U/hhBwhM=; b=UhAWa5S5wIc7Wdcy/fqNXwjnR86NMVyLB903kBNyyg6YRqHTDTF+KJhKjQzjmwfKEB E8N3C3eDO4BpScU4aD5NcohHec6orrx6WcKFv3u3MrpugUUn6u0YNUiQ/tOhE+eazsIl uy0M2/RvVGQ06kMt1oT65tD0T1bkgmklmKkG3/XYbnWlT0wwI6ijzPDUeQcktq+Z1t74 UmZIUMBRY9E7FmbGFvDNOdiFuIOT7GUuKTGliUJq0RBwhhDZyy89Vyr0OSfbAcMlz1v9 gdIrxPHzpU6d5OTwtU4SCarr37nxgI444jLc2DjCRQsQPXTfbKkdCU/I5rN1XzYy+pkp zbiw== X-Gm-Message-State: AOJu0YzaZ46q90jSc2v8Wh/HYOgsei4yS4ovKWMR0QavRc9Q720SGQ4d PdSHclmKzISSZs39k8i6z/SczwKYuBnW6+JroMQxdtIR65zXA/FRu7yA1NOBcd+E+yTM+bYssf7 OPXA6V/O3w4VQnTyrfakYuWfnEjtu03J6qDdgXXHtOyPDHpMFk9HskF8WnqcdluEcrlRzou6t45 a71hzT9N1HINXB9o7EoUyRHaOpDDukPcDdZjRKNRSKDsM= X-Gm-Gg: Acq92OFAzwLWp1z7cTAPZJkuAF1OtXLBDU21jRRYADYzi2QrkHpQtB2S49aVsb6INMW 3oYm5BGkgn2SwbdiLKvuAUmRzkfvDGvjS1AR4kcEfDYt/RQ/ixfJ+7HaHjvD+AlxBY/0mSi0vrz Ys/q7B6y4+IG9LCRZ9gkUSvz6GwBVA9eTD9ybUN9U9UzHarsIQIlpkhglHobWcc+2S5qIel0Aw9 qqCjp8zzGvFDfaSI5edpmcQsFarNpkN5zdB0eNXxbdTE81JgNinXpBqIlGR/dgUEKoLe+b5w75x ocJZZwuQPxrFuHrQc58hjMNuOKg9Y2zjG0ToELj01K7Z7uL7ycpF2BMLykN+vbKH5c01MCYe4g7 /PgwCzqlT1GgL4NTp/cXqK5cYbWtI01alajtdthz20Lj5kXSz9inoGw== X-Received: by 2002:a05:600c:3b1f:b0:490:f7c:b19 with SMTP id 5b1f17b1804b1-49035fcbd6bmr2495615e9.0.1779315623668; Wed, 20 May 2026 15:20:23 -0700 (PDT) X-Received: by 2002:a05:600c:3b1f:b0:490:f7c:b19 with SMTP id 5b1f17b1804b1-49035fcbd6bmr2494745e9.0.1779315622997; Wed, 20 May 2026 15:20:22 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49033d9edcbsm24375095e9.13.2026.05.20.15.20.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:20:22 -0700 (PDT) Date: Wed, 20 May 2026 18:20:18 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli , Miaohe Lin , Naoya Horiguchi Subject: [PATCH v8 02/37] mm: memory-failure: serialize TestSetPageHWPoison with zone->lock Message-ID: <2140873c1955d1ddde7b20dc86313215dbbc4645.1779315441.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" TestSetPageHWPoison() is called without zone->lock, so its atomic update to page->flags can race with non-atomic flag operations that run under zone->lock in the buddy allocator. In particular, __free_pages_prepare() does: page->flags.f &=3D ~PAGE_FLAGS_CHECK_AT_PREP; This non-atomic read-modify-write, while correctly excluding __PG_HWPOISON from the mask, can still lose a concurrent TestSetPageHWPoison if the read happens before the poison bit is set and the write happens after. Follow-up patches in this series add similar non-atomic flag operations as well. Fix by acquiring zone->lock around TestSetPageHWPoison. This serializes with all buddy flag manipulation. The cost is negligible: one lock/unlock in an extremely rare path (hardware memory errors). Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/memory-failure.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index ee42d4361309..a6b61172dd13 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -2348,6 +2348,8 @@ int memory_failure(unsigned long pfn, int flags) unsigned long page_flags; bool retry =3D true; int hugetlb =3D 0; + struct zone *zone; + unsigned long mf_flags; =20 if (!sysctl_memory_failure_recovery) panic("Memory failure on page %lx", pfn); @@ -2390,7 +2392,10 @@ int memory_failure(unsigned long pfn, int flags) if (hugetlb) goto unlock_mutex; =20 + zone =3D page_zone(p); + spin_lock_irqsave(&zone->lock, mf_flags); if (TestSetPageHWPoison(p)) { + spin_unlock_irqrestore(&zone->lock, mf_flags); res =3D -EHWPOISON; if (flags & MF_ACTION_REQUIRED) res =3D kill_accessing_process(current, pfn, flags); @@ -2399,6 +2404,7 @@ int memory_failure(unsigned long pfn, int flags) action_result(pfn, MF_MSG_ALREADY_POISONED, MF_FAILED); goto unlock_mutex; } + spin_unlock_irqrestore(&zone->lock, mf_flags); =20 /* * We need/can do nothing about count=3D0 pages. --=20 MST From nobody Sun May 24 22:35:50 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3D6933BED4A for ; Wed, 20 May 2026 22:20:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315637; cv=none; b=roMCaFdJ5tfed8ONaxmdwqUmXytSFR4BsL06cZFmZQKiD1/XcEVb2S4TtSQ2R+7t6IIzAL3wbN0IZHg6nJNXZYwo/lKmqdQ+wTs1VTr49jzyIorhV/hcVwG9I1pKBc034w0vybwjppeUWG7iGLJXN0alpI1plBFA/Hc0sg/15Es= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315637; c=relaxed/simple; bh=2SCjG0p+gTn8qm5MHLYDPQz8gTh6+lwWx6Kgl/LoFJQ=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=oZkYOaWv+RleaFpQAqXYaOE5Uok1jPMD+/GO/oCgDC8S4SFjuP3aoNzUf1B1IInaK0MaSZu7yrb6Xu0DlPj43kKPx2bt/aFNOLiZM5zl5lSaySqYSA0ZprF6xvCt1M21UXdV1AHFLP+ccp/+0TuWqPOq74WthTZBVqOvMz5KYVc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=aAVqiMoN; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=DHvCj154; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="aAVqiMoN"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="DHvCj154" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315635; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=6a9U/grsiUVqQvDGNE/rE2t/3gE4VHHRcC3EwM49fXU=; b=aAVqiMoNB22r5mum0UOteSdqPBHBTKFWmqxarFemkqT50JtqE6izqWP4KPKNNox94/++to g8JAPypjhbhtUdJtYepVBxr0LIx43S0OHHBxeDCJKnPbPhKcLUvbZZAdFRlCJHXfY+j4G7 t+9P7bCMsl5FYubRhELGqpLeJt0T06k= Received: from mail-lj1-f200.google.com (mail-lj1-f200.google.com [209.85.208.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-456-4MM78GcFMFi3y4r2ZjgdgQ-1; Wed, 20 May 2026 18:20:34 -0400 X-MC-Unique: 4MM78GcFMFi3y4r2ZjgdgQ-1 X-Mimecast-MFC-AGG-ID: 4MM78GcFMFi3y4r2ZjgdgQ_1779315632 Received: by mail-lj1-f200.google.com with SMTP id 38308e7fff4ca-393e64f3d68so21437471fa.0 for ; Wed, 20 May 2026 15:20:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315632; x=1779920432; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=6a9U/grsiUVqQvDGNE/rE2t/3gE4VHHRcC3EwM49fXU=; b=DHvCj154KBaz0mdArezigE3zjpmpGHBV6ML6n06cN8DGeqqkU1GTGKGBxeJDTubGKt 3bKl6wSAqHvOvpa5XUZQuR2m3wVifqvtgebzIGTUoAn1gmsexEVK3537FeDZHQ4wtYow 7UIcnR9ijaWrS2duM76JevsQr+hvt2Hk4glmi5Oh0Z5ooGn9DuY+UX1LhfyQi/8JQ+9X bWrGIHg0cBOWsFo4E4/0T3QxpBQ6aLu0cjWNv/kjHCQvZlvw5RVhYpRFtvecUQJAhyTP wfW5Fh8x9wWOxD39Idm7R0n2PxCqTHrX7OpBFRId+gNpVangH9xLuPtgYDcZKMfM+2LR koJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315632; x=1779920432; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6a9U/grsiUVqQvDGNE/rE2t/3gE4VHHRcC3EwM49fXU=; b=aWOBh9ZEOFhpPhrlPUBvk7onrfVbaoXi8eiEwx4k3YlpyQGDZ4bfYBw4c2F+gctKQF va3dkIxkJ3XBfRz1bd675ktvTktv20yL2v8wN0REuF7ShUPsI+f9/rNrUnYQz0C4Ps5Y eP+pYVOkp+kMiIdh8SIKIbxfN5/ZZzpVaoSiLDz+RcU2DjDwbvyQMqRMk5LzBvX/lJaj 0drrRqhNGCBm10RlnRzYY3Dn9jM+tmQ5QC6b6oXhosdJlHPVtamUZFrASdfGLf0x+xuk GH8GgnVQmq8DFlZSDgsDvpOOkJ8QlXlXex9RJjIcjCq0jOknzQL3rNh95hj/zvwhiBos S3zw== X-Gm-Message-State: AOJu0Ywvi21NHyIvAZAD+hMQfUaxLA+U3SWzHXOV9P0ODHkDzVaxcy5r JpFKQjjotLb+JCPewvue7OWXANcN3441McEa20W7D/x9s+tJqrVtHVzNGhNjIfBC7XcIPc56E4w FIbjLVX5mZphQM74X2pu96pWCEBsbv5XrGqlxjdScK6Kc06V5ojRSTci6aNywfxjM09TerHv016 SzIvD7ZKMzcQFNQ5bfeH0F0VA75ZDQM8ziOREqRyV7sTM= X-Gm-Gg: Acq92OEWlmvu4Kl62u9CR+iaiLllN9Gc/O/dTLR7+8fmdhydjD6DqYvYKcmb/ccCZCC Edk8Ldmg3Lo8I1pFu2WxsCt8Dh3/eYA3dI7McbiWxRGM0O6+lEEXwzjtUM5QTiTjOSLpzgyxXA1 1SoY2CFT3r48zejfXtxYc5CV4jYky9IyKAIKk8hLdQhYaDTeWsdC6LzS1w4xBdAnkXu0C1IpgK5 n9f18EMje9CIVofCEyKgRPNqiz13ookcjelVD4BSP/rvXNfqKs50UjwoCMCG6YT5B8ReBD5D7Ss 4wIl1GCZ79nBTJErXQXZEVzKKOCYEtp5M1tIkvPmyGqHxQ9D+934AUqau1D8DbysrMFC1FD3iRA 2WZGjLsIuiN/IlPPmkgDlMvYgc9qlCHE1CKmcgn9mqUE= X-Received: by 2002:a2e:bc90:0:b0:393:fa0f:d64a with SMTP id 38308e7fff4ca-395ca6449bbmr948531fa.28.1779315632050; Wed, 20 May 2026 15:20:32 -0700 (PDT) X-Received: by 2002:a2e:bc90:0:b0:393:fa0f:d64a with SMTP id 38308e7fff4ca-395ca6449bbmr948031fa.28.1779315631503; Wed, 20 May 2026 15:20:31 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-3958828150csm32418221fa.7.2026.05.20.15.20.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:20:30 -0700 (PDT) Date: Wed, 20 May 2026 18:20:23 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 03/37] mm: page_alloc: propagate PageReported flag across buddy splits Message-ID: <20af8010c2e8ebeb736ae6754e640267b96b8c2d.1779315441.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When a reported free page is split via expand() to satisfy a smaller allocation, the sub-pages placed back on the free lists lose the PageReported flag. This means they will be unnecessarily re-reported to the hypervisor in the next reporting cycle, wasting work. While I was unable to quantify the performance difference, it is an obvious waste, even if small. Propagate the PageReported flag to sub-pages during expand(), both in page_del_and_expand() and try_to_claim_block(), so that they are recognized as already-reported. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/page_alloc.c | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 227d58dc3de6..1192c4e47da4 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1510,6 +1510,7 @@ static void split_large_buddy(struct zone *zone, stru= ct page *page, unsigned long pfn, int order, fpi_t fpi) { unsigned long end =3D pfn + (1 << order); + bool reported =3D PageReported(page); =20 VM_WARN_ON_ONCE(!IS_ALIGNED(pfn, 1 << order)); /* Caller removed page from freelist, buddy info cleared! */ @@ -1521,6 +1522,8 @@ static void split_large_buddy(struct zone *zone, stru= ct page *page, do { int mt =3D get_pfnblock_migratetype(page, pfn); =20 + if (reported) + __SetPageReported(page); __free_one_page(page, pfn, zone, order, mt, fpi); pfn +=3D 1 << order; if (pfn =3D=3D end) @@ -1699,7 +1702,7 @@ struct page *__pageblock_pfn_to_page(unsigned long st= art_pfn, * -- nyc */ static inline unsigned int expand(struct zone *zone, struct page *page, in= t low, - int high, int migratetype) + int high, int migratetype, bool reported) { unsigned int size =3D 1 << high; unsigned int nr_added =3D 0; @@ -1721,6 +1724,15 @@ static inline unsigned int expand(struct zone *zone,= struct page *page, int low, __add_to_free_list(&page[size], zone, high, migratetype, false); set_buddy_order(&page[size], high); nr_added +=3D size; + + /* + * The parent page has been reported to the host. The + * sub-pages are part of the same reported block, so mark + * them reported too. This avoids re-reporting pages that + * the host already knows about. + */ + if (reported) + __SetPageReported(&page[size]); } =20 return nr_added; @@ -1731,9 +1743,10 @@ static __always_inline void page_del_and_expand(stru= ct zone *zone, int high, int migratetype) { int nr_pages =3D 1 << high; + bool was_reported =3D page_reported(page); =20 __del_page_from_free_list(page, zone, high, migratetype); - nr_pages -=3D expand(zone, page, low, high, migratetype); + nr_pages -=3D expand(zone, page, low, high, migratetype, was_reported); account_freepages(zone, -nr_pages, migratetype); } =20 @@ -2288,10 +2301,12 @@ try_to_claim_block(struct zone *zone, struct page *= page, /* Take ownership for orders >=3D pageblock_order */ if (current_order >=3D pageblock_order) { unsigned int nr_added; + bool was_reported =3D page_reported(page); =20 del_page_from_free_list(page, zone, current_order, block_type); change_pageblock_range(page, current_order, start_type); - nr_added =3D expand(zone, page, order, current_order, start_type); + nr_added =3D expand(zone, page, order, current_order, start_type, + was_reported); account_freepages(zone, nr_added, start_type); return page; } --=20 MST From nobody Sun May 24 22:35:50 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 30A1433A9FE for ; Wed, 20 May 2026 22:20:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315651; cv=none; b=MmRYb50+u/TvspmXmqHKSNGf36/LK8jatksV//OpM9bFbTl9P27dMbIuyQSdtfp62O389/VykS/4/E2Lqi1kU0Un28PhOix24EHq+E4iPVpflpi1Ozu8bI9yjuyLQxJZsGEGG26mh5eY9PuRHXfBsSUYw8GZc0WUDP8C7656kqo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315651; c=relaxed/simple; bh=DRd7Xt1v5jz8DB3KNV92VDBlshly3yQmr6003O/RIg8=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=k8dwhNrmo1sa1qey6gQXSWXQ1OPYJMfXM3erfKN/g1l1N7+MCI+SXM34f+CtLfw1agrFiXOq3McK+MixDDWzrnFbVlKDZluCrs2Dw9aW6RwbNk+P0TAcClU0PuYyZRVq8ziMB1WqLazuhc9rNNlWRF//k+pQ6EFONoqFn06D1tM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Un5rIHDj; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=OnojuL+e; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Un5rIHDj"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="OnojuL+e" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315649; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=6KLmapuZeL0TvG+NgLwJQJBgDKEe1s3MFixcWgGLhrA=; b=Un5rIHDjKLgm93QVHQlFpzC194JVWPIrkMCOvoScTGRITGpea3jPFStT8H6zHZxZcGXaIV uRR4vlUw0AQ/pIFm191P9491fYyi4PoBM9b6ppX6jlLOdMtIgYAIDizYvkqlBV3oR9UpMy v5Bp0/ppqqC4KK3knBT6UEbU9fXgtHo= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-681-Yyv7GTVyM_2tUpV-yps3IA-1; Wed, 20 May 2026 18:20:42 -0400 X-MC-Unique: Yyv7GTVyM_2tUpV-yps3IA-1 X-Mimecast-MFC-AGG-ID: Yyv7GTVyM_2tUpV-yps3IA_1779315641 Received: by mail-ed1-f72.google.com with SMTP id 4fb4d7f45d1cf-67fd8c53ccdso691247a12.3 for ; Wed, 20 May 2026 15:20:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315641; x=1779920441; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=6KLmapuZeL0TvG+NgLwJQJBgDKEe1s3MFixcWgGLhrA=; b=OnojuL+eXht9TI6g/dRSIHjBSqqzU0S4tFKYCiq7N4vUK0dOvoy8nkkW0PmVe/0vtu Egk++0I2SPYLXLYnyJcy/TJUix4v0ogdHXHRzjnpNHB9bvY4BxyEfYc9sll9TmGflk+N 5k/3ly63qdQI4eah/4oCf9K544hwKmxHW1zeR5TvBuf7EZL4SdFqPuWO1unTgdutABnb +YUNpW58nzuyHE1nLaFnarbm4ufv3KdVbm2EtFZ8H/LeQGOQ/iEDeZ6HvlfrN1mtLO0J E7i8fXvPrIoALStI1JDqGxq0dH8mIihOkrcGSjqV5Lzzn6YsrmeGGrB2PqFx9r3s0NMa EDUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315641; x=1779920441; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=6KLmapuZeL0TvG+NgLwJQJBgDKEe1s3MFixcWgGLhrA=; b=jimBtXwvLNL83wj2QQvymPjzNW6iNdrEgfAyadWMbbwMjinR4ZLe/+A2r8nrXHHtWN 5hxx57nxmREFi5lThWxkz3FtkqEvnhYsSDuUvwijkp2ooC2Yxrrzx/BaDbZlyMv5h+a/ tryKkv80/qgZ3K5p7CnrTNnpuHRSp8CYDy0y1cW9B7m8ce2EXpf0OeSgf6Rja7n2mwr6 9EBRndHf2F+lhoMcCq+c0kFgmr/2fSB7DYqxJWz6/E2B6xUP+ECOngAmRBQf7FiWSzp7 t63/kEfJQ1dv6zHAg81rgo0inHAGmGiuoCN+ILb6hXEai2HnUk1s5Ker9hODhLpro7ev 7tQA== X-Gm-Message-State: AOJu0Yz9oSLudrJ6vlM2VrLF/joSr9qX2leGiDMZRAYg9BCEg0wOLVVK J92KlDlETjB1+aP1YVgXdE8EpBfEQDOFXAVJvsKMXI7epcNQ1B5NEmCd28RaNwcxie+e9N9nsUR UvyhjP1vo21ZzyYoBCUv+kP6MzQ8Al1rAD1Tt7VNWW6wAY6PyCeDpuKWDw5GxSspuFvUb+ewFtg 1Ob3DBWE2Gt8DMmyQYE+6Afa9mAU3va7lYYuuk5xPKgH0= X-Gm-Gg: Acq92OHplYGyX6uclaJ0vjaYgFkuLJjbX+Fui2eoEV9sP3aCnGw6f6llXpVYrMxs3sH tTTRoqLR307RW8caW4d7vCn5VTQgCxqdHvpoeCZSGpnineDtfKdYB7vk+f/Xbw2g/rZZ8e8qxfP 15N0YYq+xqworUY5H0P4QGCh1jW5u8JAfPa7OoFpoh75MbdOsi0nYwrC9SzN+7c15ek/1WqjDlI NIXT+ubwmjJ9pKEVrGOMhDYVtzRj9YMpoY/GMKcAg7oBHS9NiCBowKL6S/YYMLlmC1CIQvo46wF s/6Jwpqwfllll+o4Hymlz72EgWM5uANCRhvMHi6PV9HcazD2cKE26lMVzV+xQpUl7J4uuAyg7Ij gePrsrwPDxXxJ5LHYjUhhE6gaNMvny/WRmJrb0MA6OT8= X-Received: by 2002:a05:6402:84c:b0:683:5589:4317 with SMTP id 4fb4d7f45d1cf-688361e61a7mr34143a12.11.1779315640617; Wed, 20 May 2026 15:20:40 -0700 (PDT) X-Received: by 2002:a05:6402:84c:b0:683:5589:4317 with SMTP id 4fb4d7f45d1cf-688361e61a7mr34086a12.11.1779315639756; Wed, 20 May 2026 15:20:39 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-684744d13aesm5851373a12.18.2026.05.20.15.20.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:20:38 -0700 (PDT) Date: Wed, 20 May 2026 18:20:31 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 04/37] mm: page_reporting: allow driver to set batch capacity Message-ID: <41e4b7824bbd0fbc94a1c9a52ac1f261fb032a8a.1779315441.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a capacity field to page_reporting_dev_info so drivers can control the maximum number of pages per report batch. This is useful when the driver needs to reserve virtqueue descriptors for metadata (e.g., a bitmap buffer) alongside the page buffers. The value is capped at PAGE_REPORTING_CAPACITY and rounded down to a power of 2. If unset (0), defaults to PAGE_REPORTING_CAPACITY. The virtio_balloon driver sets capacity to the reporting virtqueue size, letting page_reporting adapt to whatever the device provides. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- drivers/virtio/virtio_balloon.c | 5 +---- include/linux/page_reporting.h | 3 +++ mm/page_reporting.c | 28 +++++++++++++++++----------- 3 files changed, 21 insertions(+), 15 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index f6c2dff33f8a..6a1a610c2cb1 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -1017,10 +1017,6 @@ static int virtballoon_probe(struct virtio_device *v= dev) unsigned int capacity; =20 capacity =3D virtqueue_get_vring_size(vb->reporting_vq); - if (capacity < PAGE_REPORTING_CAPACITY) { - err =3D -ENOSPC; - goto out_unregister_oom; - } =20 vb->pr_dev_info.order =3D PAGE_REPORTING_ORDER_UNSPECIFIED; =20 @@ -1041,6 +1037,7 @@ static int virtballoon_probe(struct virtio_device *vd= ev) vb->pr_dev_info.order =3D 5; #endif =20 + vb->pr_dev_info.capacity =3D capacity; err =3D page_reporting_register(&vb->pr_dev_info); if (err) goto out_unregister_oom; diff --git a/include/linux/page_reporting.h b/include/linux/page_reporting.h index 9d4ca5c218a0..5ab5be02fa15 100644 --- a/include/linux/page_reporting.h +++ b/include/linux/page_reporting.h @@ -22,6 +22,9 @@ struct page_reporting_dev_info { =20 /* Minimal order of page reporting */ unsigned int order; + + /* Max pages per report batch (default PAGE_REPORTING_CAPACITY) */ + unsigned int capacity; }; =20 /* Tear-down and bring-up for page reporting devices */ diff --git a/mm/page_reporting.c b/mm/page_reporting.c index 7418f2e500bb..ae8d55f89f25 100644 --- a/mm/page_reporting.c +++ b/mm/page_reporting.c @@ -174,10 +174,10 @@ page_reporting_cycle(struct page_reporting_dev_info *= prdev, struct zone *zone, * list processed. This should result in us reporting all pages on * an idle system in about 30 seconds. * - * The division here should be cheap since PAGE_REPORTING_CAPACITY - * should always be a power of 2. + * The division here should be cheap since capacity should + * always be a power of 2. */ - budget =3D DIV_ROUND_UP(area->nr_free, PAGE_REPORTING_CAPACITY * 16); + budget =3D DIV_ROUND_UP(area->nr_free, prdev->capacity * 16); =20 /* loop through free list adding unreported pages to sg list */ list_for_each_entry_safe(page, next, list, lru) { @@ -222,10 +222,10 @@ page_reporting_cycle(struct page_reporting_dev_info *= prdev, struct zone *zone, spin_unlock_irq(&zone->lock); =20 /* begin processing pages in local list */ - err =3D prdev->report(prdev, sgl, PAGE_REPORTING_CAPACITY); + err =3D prdev->report(prdev, sgl, prdev->capacity); =20 /* reset offset since the full list was reported */ - *offset =3D PAGE_REPORTING_CAPACITY; + *offset =3D prdev->capacity; =20 /* update budget to reflect call to report function */ budget--; @@ -234,7 +234,7 @@ page_reporting_cycle(struct page_reporting_dev_info *pr= dev, struct zone *zone, spin_lock_irq(&zone->lock); =20 /* flush reported pages from the sg list */ - page_reporting_drain(prdev, sgl, PAGE_REPORTING_CAPACITY, !err); + page_reporting_drain(prdev, sgl, prdev->capacity, !err); =20 /* * Reset next to first entry, the old next isn't valid @@ -260,13 +260,13 @@ static int page_reporting_process_zone(struct page_reporting_dev_info *prdev, struct scatterlist *sgl, struct zone *zone) { - unsigned int order, mt, leftover, offset =3D PAGE_REPORTING_CAPACITY; + unsigned int order, mt, leftover, offset =3D prdev->capacity; unsigned long watermark; int err =3D 0; =20 /* Generate minimum watermark to be able to guarantee progress */ watermark =3D low_wmark_pages(zone) + - (PAGE_REPORTING_CAPACITY << page_reporting_order); + (prdev->capacity << page_reporting_order); =20 /* * Cancel request if insufficient free memory or if we failed @@ -290,7 +290,7 @@ page_reporting_process_zone(struct page_reporting_dev_i= nfo *prdev, } =20 /* report the leftover pages before going idle */ - leftover =3D PAGE_REPORTING_CAPACITY - offset; + leftover =3D prdev->capacity - offset; if (leftover) { sgl =3D &sgl[offset]; err =3D prdev->report(prdev, sgl, leftover); @@ -322,11 +322,11 @@ static void page_reporting_process(struct work_struct= *work) atomic_set(&prdev->state, state); =20 /* allocate scatterlist to store pages being reported on */ - sgl =3D kmalloc_objs(*sgl, PAGE_REPORTING_CAPACITY); + sgl =3D kmalloc_objs(*sgl, prdev->capacity); if (!sgl) goto err_out; =20 - sg_init_table(sgl, PAGE_REPORTING_CAPACITY); + sg_init_table(sgl, prdev->capacity); =20 for_each_zone(zone) { err =3D page_reporting_process_zone(prdev, sgl, zone); @@ -377,6 +377,12 @@ int page_reporting_register(struct page_reporting_dev_= info *prdev) page_reporting_order =3D pageblock_order; } =20 + if (!prdev->capacity || prdev->capacity > PAGE_REPORTING_CAPACITY) + prdev->capacity =3D PAGE_REPORTING_CAPACITY; + + /* Power of 2 so division by capacity in the budget calc is cheap */ + prdev->capacity =3D rounddown_pow_of_two(prdev->capacity); + /* initialize state and work structures */ atomic_set(&prdev->state, PAGE_REPORTING_IDLE); INIT_DELAYED_WORK(&prdev->work, &page_reporting_process); --=20 MST From nobody Sun May 24 22:35:50 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A4EB83BF685 for ; Wed, 20 May 2026 22:20:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315654; cv=none; b=fmBE7xbxZpWelcsHZc2+si8hY/89OmEYppvAuqdnoxuAMVuCvWYcrC5LO+u8on6ae2uiRlll41yUfpxk5saC7e6P/EkXj+AqCAX5f4cB7irKuIZJXB1HQfIWfOA7fPHRTRko5QNCItXpSwFCHwBv6LuLhER02uQtgLbzuBS5PQc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315654; c=relaxed/simple; bh=zfTSssqCCnno4sdUyiW/sc4lt3FSQTZh1jfv0HEfTik=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=OOkvRidfjXLLzmfXSJplH/iNhs2145FoDwNbMifIVUkY84kzkFODUX8qHZEcwqia4TY28c1Xw48qGosJBfDiajOPsnkHUuWRctAzmXsI6iCa5L3smJr24Tzet27gL1GVUwyKKwIt7wb5xx9iqlzXTkQdNlD9Rrxc+LKf3oJ9PnM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=RfOkdqOW; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=O7yv41Bl; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="RfOkdqOW"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="O7yv41Bl" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315652; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=3RnCcviTYcs0bJrQCPa7/Ta1tOKxh1kkpoqwRDnBft0=; b=RfOkdqOW78OKWBvrK7xGnQDOD/p97XFbmwVitA2wATN1GUxKW96GtX0nxattzWWNH66+qG w0yFCkO2KZREHxFkb9BuYVCipcT5N1mYNtfjD2XJTUcRaDL3Qk/mB3GwkuWTGK9mpGfVkX g04srEDpHY5mg4VCwLJMaUYyAvCOOiE= Received: from mail-lj1-f198.google.com (mail-lj1-f198.google.com [209.85.208.198]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-359-gVK7HCdCPWGXYK03-X30RQ-1; Wed, 20 May 2026 18:20:51 -0400 X-MC-Unique: gVK7HCdCPWGXYK03-X30RQ-1 X-Mimecast-MFC-AGG-ID: gVK7HCdCPWGXYK03-X30RQ_1779315650 Received: by mail-lj1-f198.google.com with SMTP id 38308e7fff4ca-393d5b5ae63so20212401fa.3 for ; Wed, 20 May 2026 15:20:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315649; x=1779920449; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=3RnCcviTYcs0bJrQCPa7/Ta1tOKxh1kkpoqwRDnBft0=; b=O7yv41BlWF+A6nfer7sZli6l7i3BRDomuShWSIWZR6hYNgtKY1CDNRaFZ4WpUxB4qZ NEdjSK6m7KCu/dcimJSkwXvh5L7NRcCcpex6yNYHHvl4tVAsTJt8B8XbLvpGccA+At/e GD7LPz7wzVfrMwSNjaWB6HUvnbC2i1QUQdNLEAkwzkDNo3vvUA+MMTbXr2WCKjLZ4BMQ WNt8hc8TISHWonpPGIiglxjW5YnXLyH2m98kUBHdggWKj07kELg1mYCXhG6b3bd0hE7H njTSNs5UL9VxJH9jY6GHSK6KDDxHoWu3t3qOXDyjD3yNeq19z9WKwj4un0G7YOTC2CG9 TTww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315649; x=1779920449; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3RnCcviTYcs0bJrQCPa7/Ta1tOKxh1kkpoqwRDnBft0=; b=nHVPDPLBvjLFGOv3RehWo+53/fhe8zLAwJx2gxrxpCRsRlCSX2qgsS/yBGhD0J0Dre rmsmcp9nT9s387GiWBbmJtU8DSqW30bpYSdeyDIFd9qI5Ma4apjeZI/9UNSSho90ypor BfTzoEZWLoviC2axQAXmDGiP32CgBMjiaiGD8YFn0n+/JsuPzrUZXOerohCdwAFm37sb Z3dOwNsm+YcRGgUfBoXm3OYnzgcF4E+iqB47mxcvp+JeDX7HBaKj9mPNAXt1YM5tlO1M HfACgCLcy3rCgiXZb3ilb/dF9xGH7H4DSVz4dqw3FlipefMi3AAOsma7mFXRAldNW2LI eTxg== X-Gm-Message-State: AOJu0YzvhNzm6cG7PgVnztORKr79n4i7xqxtGoMQFfIR+/IoOVY6CUPo 9Y0hjeTc/OFE9ftP7RMPFBw3hJNYHfvC2pA928E7WvRAYVME881Yspl/NXC8bJGoRKTxHGwlxP4 LGiMD6yA0HN1aLVl/sAXgYLs++iB9GL+67IPLJc9OzmRw5My4MWwv1bizC/2aDhguG/9i+LS/kb m5wj2tv/SDuS4Qz9Q/e1vKh5dNAiebpV10sH3IQnAAtuY= X-Gm-Gg: Acq92OFRdYZK/gK1KAIO8JPvyigiZEUnAkg7vMgo7B0w4lMlDTp2RrQZxtEH2QRFoZI B2obKer15t7uCMYm9hpalg3KmyznWpHv5/NDFsfhL3Kh2ydJ6DJ1LywRXR8x682EmORlyEJbIdA dWbWjb3RTtJGRSFU/vEKPe3HWU+UtJK9VItNrl7otox5UPlxTvdgLwJ6VUNlH0sPKggGxyow+rP q9I3blOnkvJks+wvcCPe8C2qNIEeFcJUXMaBJq0vy3hd/d+O99vS13f5xw5cnqYRzwGOpdd266A S+23BavZ+nHqfL+Vy7sgVrAzbW9aqCTS7P/hMpMrGLgQC7GNezXRg9ahahES47wHTwIWCkIgbl0 8gu4dG1tGOdF9HGpGa5ducGI0+Uoo/Po640sugpmR85k= X-Received: by 2002:a2e:a593:0:b0:38e:98b0:3a7e with SMTP id 38308e7fff4ca-395ca5b175bmr844591fa.22.1779315649151; Wed, 20 May 2026 15:20:49 -0700 (PDT) X-Received: by 2002:a2e:a593:0:b0:38e:98b0:3a7e with SMTP id 38308e7fff4ca-395ca5b175bmr844081fa.22.1779315648437; Wed, 20 May 2026 15:20:48 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-395882cf497sm34824461fa.16.2026.05.20.15.20.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:20:46 -0700 (PDT) Date: Wed, 20 May 2026 18:20:40 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 05/37] mm: hugetlb: remove dead alloc_hugetlb_folio stub Message-ID: <5756eedfaae9e7b69a00da30428ab099f3941d26.1779315441.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Remove the !CONFIG_HUGETLB_PAGE stub for alloc_hugetlb_folio(). The stub is dead code: all callers are in mm/hugetlb.c (CONFIG_HUGETLB_PAGE) or fs/hugetlbfs/inode.c (CONFIG_HUGETLBFS), and CONFIG_HUGETLB_PAGE is def_bool HUGETLBFS with nothing selecting it independently. The stub is also broken: it returns NULL, but all callers check IS_ERR(folio), so a NULL return would not be caught and would crash on the subsequent folio dereference. Remove it now since follow-up patches change the signature of alloc_hugetlb_folio and would otherwise need to update the broken stub too. Signed-off-by: Michael S. Tsirkin Reviewed-by: Gregory Price Assisted-by: Claude:claude-opus-4-6 --- include/linux/hugetlb.h | 7 ------- 1 file changed, 7 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 93418625d3c5..f016bc2e8936 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -1129,13 +1129,6 @@ static inline void wait_for_freed_hugetlb_folios(voi= d) { } =20 -static inline struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, - unsigned long addr, - bool cow_from_owner) -{ - return NULL; -} - static inline struct folio * alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_nid, nodemask_t *nmask, gfp_t gfp_mask) --=20 MST From nobody Sun May 24 22:35:50 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 090853BF66C for ; Wed, 20 May 2026 22:21:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315666; cv=none; b=KuJsIOPgHxd8Z4xCql1L6/0QdOH4MLPjQYsbaB/ajQPcDjzQzA8wr7C4Gjjg0pm6rFRJveVXAQV/2xtufcnKoGgfLZ3NHSyhDrn3EcNcL8UyngSej2ysfuPGpuS5YTaYBWY3uTjIaqwqR82lwiedCqUtNLB5TJ8x/NiRReqjExk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315666; c=relaxed/simple; bh=YV4jBXFQrQQpbg0jR4Mv8OiHoL2bzi28JLwzDY6BCc0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=UN9kTdOoeFpRGy8xKpYHIbJAB+h1bZo2T3/rBZeJXILx76QD6/gMyh8hTc0fiMRujaozEJ9jIQsYvipbO/2MqfHgqHL0SC9yIUyiDaJ2hUUKc7UcAIeaM/JdNvZM5PWDzIAhjKBJthVg9mdwKSXob204i85sdFS24cyT3UwPK1g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=DkglxuUp; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=MlEOHa9s; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="DkglxuUp"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="MlEOHa9s" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315664; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Ri6YQ3HlLjUAH8mONUkbxIg9Wl46GlG+rJaCqygm0zg=; b=DkglxuUpa5QCWwADSGWgNKz1UUi9SClxRz4UALxthr2oeT9IbkwqcpokYpacE4nYu87MvC ka+7mHg9h3CT3+ThkkOyzXF6D9skRG6zwKGVmxrJ0LTG4nQO+2mwT1YxBMJBDRM5tnGgPY maLyptXMRi9YXJWB4G0ej6oaFh/mPJ0= Received: from mail-lj1-f199.google.com (mail-lj1-f199.google.com [209.85.208.199]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-352-iBIs-DH0MfaCSfWVnJdcHA-1; Wed, 20 May 2026 18:20:59 -0400 X-MC-Unique: iBIs-DH0MfaCSfWVnJdcHA-1 X-Mimecast-MFC-AGG-ID: iBIs-DH0MfaCSfWVnJdcHA_1779315658 Received: by mail-lj1-f199.google.com with SMTP id 38308e7fff4ca-393adad635dso32923331fa.3 for ; Wed, 20 May 2026 15:20:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315658; x=1779920458; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Ri6YQ3HlLjUAH8mONUkbxIg9Wl46GlG+rJaCqygm0zg=; b=MlEOHa9sAsc6B/+YqmfQpWEW4WnFSU/Yoy40oitKpB+kw5QKXfnIxOCzAozUXlY3fg Ljhi/QKCiaxz4o9L4u//SCAaitADD7V2NhUw1cR9B1nA0LhIYCDPhapS/EG+JQbNsTIw zGGWGFFxyHwSA8EcaDqTQYyFBTZJoSeNQHNnqwGPCBF5/QKOJou+x08XVepppUjSy8H2 g5WP24bGJa+RL7CvaskioWHIOyz4z8j1vBgBjFDzU13BtF/CyMijL5oZEMXHGVCHbtYF rD9+bZE5n5iurfteV5VcHQviGT9l/XGdNaBz24e1hDopaDaEypIGXgYbZl923FcvPH0+ r0yA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315658; x=1779920458; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ri6YQ3HlLjUAH8mONUkbxIg9Wl46GlG+rJaCqygm0zg=; b=cIB++tP+5vJXRVW8qPjstDI/Kj6f+MDTAg9ETm2WPFzR0BTaes+IfUNo2V97CIMo5r tUaEdlzT/3ZBZx4A6fLmwFSUAF7DlodXPXQL27zssJ6S5C4tsRjloC5QNMCrQJz4Ke2n JhZb8XI0Y946+b+7F8DMbe5l2l2wxoskiG/2Ipn4O1q0oSbIARyVVA5tNSsILXkxPxq5 BN7JDzhOI/ouWXAhEOr/L+Z5LiBLWmIpFczWNL70Mo0GC8A3yR/pZsY0j3JGWXTFFz5Z fr5L0xiuCwufNH4eB6a2Vw6axd8Go71au4LcXXwYMI7IWziUZQQf92cF+6FFfCZm60Dx bMLQ== X-Gm-Message-State: AOJu0YyVDZ2GFa6P/+JUxPS6ICxxuuioUpnLKRdr7kLYhvS3kFcyIWzb elwYCRbKMkhaeVvTpK6rHoLuYXNligmBoPX4PGt4shKW8ZnMu/FjoHKKZrJ20zDYh6zn2TWB3SO wAvMITk+Nf6bO2yCPxyetdCrk24K1AXAiGnkvdd4zhfthI6/QqeYoulkkkWhZX1rgvUZj0e2EXb zdTmcAhZfK5TJx8CoeK2LOArYfWd5prP3vmddTARxqn3g= X-Gm-Gg: Acq92OHwSgdvYTvVMZWJ0fema1bpI7n37UWI1xs+hRciizmWqk7uPPj7QU/3PT9l7fn PaAZTjXkRNzUVfrS2qDAGxy22N2AxEfdo6y28XtTI4OIPgy9gJvMxVhTyNekt9y6AAnZr6IrgS8 NPrpwOKIhoIGEUk0xySJYd/NDsHTKZZzvWyY9b9KlW3lOaElQinu7x8lNp9HYwdBm6XR4KBh642 F2zXmu7a/fhvun997qHC2UmEGMoNInOK6aMVxfhrC6dcV/SUGcmebRR7hEaWxrKWTKgJCzvUi7y HtviAJGxTB9eBH3rCmfW30kjevzwjuq0Fg5TPdhlSF3K0ZOtWmdODMdApa0bOFptwg2P6Ks/D3K C4TvRiEuAIehS5OS5fFXvLESf+dt2rEOy9Zro+spERGM= X-Received: by 2002:a05:651c:1512:b0:38a:45d6:c246 with SMTP id 38308e7fff4ca-395ca1dfb3emr1246721fa.8.1779315657849; Wed, 20 May 2026 15:20:57 -0700 (PDT) X-Received: by 2002:a05:651c:1512:b0:38a:45d6:c246 with SMTP id 38308e7fff4ca-395ca1dfb3emr1246201fa.8.1779315657241; Wed, 20 May 2026 15:20:57 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 38308e7fff4ca-39588791302sm31998751fa.32.2026.05.20.15.20.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:20:55 -0700 (PDT) Date: Wed, 20 May 2026 18:20:48 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 06/37] mm: move vma_alloc_folio_noprof to page_alloc.c Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move vma_alloc_folio_noprof() from an inline in gfp.h (for !NUMA) and mempolicy.c (for NUMA) to page_alloc.c. This prepares for a subsequent patch that will thread user_addr through the allocator: having vma_alloc_folio_noprof in page_alloc.c means user_addr can be passed to the internal allocation path without changing public API signatures or duplicating plumbing in both gfp.h and mempolicy.c. The !NUMA path gains the VM_DROPPABLE -> __GFP_NOWARN check that the NUMA path already had. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- include/linux/gfp.h | 9 ++------- mm/mempolicy.c | 32 -------------------------------- mm/page_alloc.c | 43 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 45 insertions(+), 39 deletions(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index 51ef13ed756e..7ccbda35b9ad 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -318,13 +318,13 @@ static inline struct page *alloc_pages_node_noprof(in= t nid, gfp_t gfp_mask, =20 #define alloc_pages_node(...) alloc_hooks(alloc_pages_node_noprof(__VA_= ARGS__)) =20 +struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, + struct vm_area_struct *vma, unsigned long addr); #ifdef CONFIG_NUMA struct page *alloc_pages_noprof(gfp_t gfp, unsigned int order); struct folio *folio_alloc_noprof(gfp_t gfp, unsigned int order); struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order, struct mempolicy *mpol, pgoff_t ilx, int nid); -struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, struct vm_area_= struct *vma, - unsigned long addr); #else static inline struct page *alloc_pages_noprof(gfp_t gfp_mask, unsigned int= order) { @@ -339,11 +339,6 @@ static inline struct folio *folio_alloc_mpol_noprof(gf= p_t gfp, unsigned int orde { return folio_alloc_noprof(gfp, order); } -static inline struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, - struct vm_area_struct *vma, unsigned long addr) -{ - return folio_alloc_noprof(gfp, order); -} #endif =20 #define alloc_pages(...) alloc_hooks(alloc_pages_noprof(__VA_ARGS__)) diff --git a/mm/mempolicy.c b/mm/mempolicy.c index b2c21ed1fd84..39e556e3d263 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -2516,38 +2516,6 @@ struct folio *folio_alloc_mpol_noprof(gfp_t gfp, uns= igned int order, return page_rmappable_folio(page); } =20 -/** - * vma_alloc_folio - Allocate a folio for a VMA. - * @gfp: GFP flags. - * @order: Order of the folio. - * @vma: Pointer to VMA. - * @addr: Virtual address of the allocation. Must be inside @vma. - * - * Allocate a folio for a specific address in @vma, using the appropriate - * NUMA policy. The caller must hold the mmap_lock of the mm_struct of the - * VMA to prevent it from going away. Should be used for all allocations - * for folios that will be mapped into user space, excepting hugetlbfs, and - * excepting where direct use of folio_alloc_mpol() is more appropriate. - * - * Return: The folio on success or NULL if allocation fails. - */ -struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, struct vm_area_= struct *vma, - unsigned long addr) -{ - struct mempolicy *pol; - pgoff_t ilx; - struct folio *folio; - - if (vma->vm_flags & VM_DROPPABLE) - gfp |=3D __GFP_NOWARN; - - pol =3D get_vma_policy(vma, addr, order, &ilx); - folio =3D folio_alloc_mpol_noprof(gfp, order, pol, ilx, numa_node_id()); - mpol_cond_put(pol); - return folio; -} -EXPORT_SYMBOL(vma_alloc_folio_noprof); - struct page *alloc_frozen_pages_noprof(gfp_t gfp, unsigned order) { struct mempolicy *pol =3D &default_policy; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 1192c4e47da4..0c4f4c678233 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5288,6 +5288,49 @@ struct folio *__folio_alloc_noprof(gfp_t gfp, unsign= ed int order, int preferred_ } EXPORT_SYMBOL(__folio_alloc_noprof); =20 +#ifdef CONFIG_NUMA +/** + * vma_alloc_folio - Allocate a folio for a VMA. + * @gfp: GFP flags. + * @order: Order of the folio. + * @vma: Pointer to VMA. + * @addr: Virtual address of the allocation. Must be inside @vma. + * + * Allocate a folio for a specific address in @vma, using the appropriate + * NUMA policy. The caller must hold the mmap_lock of the mm_struct of the + * VMA to prevent it from going away. Should be used for all allocations + * for folios that will be mapped into user space, excepting hugetlbfs, and + * excepting where direct use of folio_alloc_mpol() is more appropriate. + * + * Return: The folio on success or NULL if allocation fails. + */ +struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, + struct vm_area_struct *vma, unsigned long addr) +{ + struct mempolicy *pol; + pgoff_t ilx; + struct folio *folio; + + if (vma->vm_flags & VM_DROPPABLE) + gfp |=3D __GFP_NOWARN; + + pol =3D get_vma_policy(vma, addr, order, &ilx); + folio =3D folio_alloc_mpol_noprof(gfp, order, pol, ilx, numa_node_id()); + mpol_cond_put(pol); + return folio; +} +#else +struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, + struct vm_area_struct *vma, unsigned long addr) +{ + if (vma->vm_flags & VM_DROPPABLE) + gfp |=3D __GFP_NOWARN; + + return folio_alloc_noprof(gfp, order); +} +#endif +EXPORT_SYMBOL(vma_alloc_folio_noprof); + /* * Common helper functions. Never use with __GFP_HIGHMEM because the retur= ned * address cannot represent highmem pages. Use alloc_pages and then kmap if --=20 MST From nobody Sun May 24 22:35:50 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 51C493B637A for ; Wed, 20 May 2026 22:21:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315673; cv=none; b=cs2Y+XaVhR1iNxZecy9pdv3VeVPgEaWetGCDn0FPynvZcBOTD3EeLi/3PB+PFs6x8WoF/FhzYuMLsAbgnwcZJ8x/0/z+sFggp+nBJc6nI/Tj21arO87QCF0VTk/uaBFq9XTkO7ech5F3v5Svqr+RK20n4zIY07C2WIbx/x0bsEE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315673; c=relaxed/simple; bh=nm8GORv82an3t6ivQkoNLOXWYPouiYHiZDPzLs720wQ=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=FxggbTCmGqeWIC7WNl0+vxl+8ZcxlC7y67aJo7ws03PAMAUns0LdfYiehtywJ+cwDdR0ki2B8kFm4T9rQCWNY/B0pThweSNt7vSQWT8hkcXkSa/AD5GR2TGsBmyPVskg9Mo6zgx1YkJikxQ06t3F08c7VhNkj6gnQWtIiGUULsg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=VjCY3Qpf; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=jk+lQZnZ; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="VjCY3Qpf"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="jk+lQZnZ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315670; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=REzEk5mBWXWtdy7K3TGxl481jdkcTH+o6ajGP1cp+mA=; b=VjCY3Qpfb/YRCBXo4FdkdSk8gwXmg4WG2AEotYTrEeXCH40hXPzVl+HQ3afwlqC/CCskWi yA6M108pVMqcUsSB0FIRPzoIzFB+gYwDeXbWm/fqnkt0aKKj5WfUcGyZRpZJxk0ArwcO9k 9k/skFpLuKbEwhg4K0MgrAMxfOparyI= Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-39-1CA_c-9dMpu0b3v0f7f88g-1; Wed, 20 May 2026 18:21:06 -0400 X-MC-Unique: 1CA_c-9dMpu0b3v0f7f88g-1 X-Mimecast-MFC-AGG-ID: 1CA_c-9dMpu0b3v0f7f88g_1779315665 Received: by mail-ed1-f69.google.com with SMTP id 4fb4d7f45d1cf-6852c63bd1aso6557847a12.0 for ; Wed, 20 May 2026 15:21:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315665; x=1779920465; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=REzEk5mBWXWtdy7K3TGxl481jdkcTH+o6ajGP1cp+mA=; b=jk+lQZnZS1c6hV3/HSVPuFPxqc+S8OJAhLsL/iyq9LQ8Q/2w4QVkZoTd2yDscWYly3 /DbnEGDm0HEe7ExflpnNTKQuQ0QQa0GmbIDAqTGfzBMFMiEwW6ADPSmKgm4I+IdHlyJf e8hrL3frO1HpIzdH02nwsyWy79ANJiWr8qoRtwkQCXfGgMNhS4Whq/AR8MVaZ9Mxoqd/ Kcsf9VRRLIY8O+Hi3GBJEaS3W8GFaD6w/uHZsl9K9KIbcQwkfmRrJ855/dorDEwDaPXW B94QRLcc1tl9xuhdsRRkAfOpAExd3V2ozvZCmMjw/AfiAgJlUuElxG3SBeY6k11nzRTW LegQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315665; x=1779920465; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=REzEk5mBWXWtdy7K3TGxl481jdkcTH+o6ajGP1cp+mA=; b=n2pVf4zGXdHY2hHI2w3vb4KZhesa65nWPXuzUkiyI2E2OVfGhac11S3yUYK8bzeUEc BqQhltomzhowBoZPjBlVZBHyzRvd6yGVUjUIVOGHxi4kLCiM2FPBG4i4fxAWlTliOKtq vEheCuWIV5bLkghRvI86vNIPyorRHm8Ms9MrRgNSOjDBa8QqRF7x0TXVMMA/Xyb2kTjQ lbuMtzytKvhoFD+9+QzqcROp1JWSXuWqMxTvN/958hXjPZoR3R+tjZjnQYcWA4kb8Mqn tPdJVceK3p7WMe5LfJEc7hRezTDNBormh6ZsWXa1vbr6nPTTWh0Dc7GKxqjOIcKw0Tgc 1oqg== X-Gm-Message-State: AOJu0YynlqtSUe9PdQQv4jlDedPoGi5t6wBUjofMOhjXdx4KjUCG7vXx 79vFKmmyxFZejY9SwCB2ZE7G/BxjU9bnnTamzcrLYsFZPkxVz3YLRFwHSqdMb29md5qKhl9khqU iC4m351P+9kRBzovtPAyF0qxzOTxayLADCH8AOe6uW8Gszh4WuEeIZPp1hTNosci6pkM5PkQN/0 /k5DRodNWYdvKkuSyr6ol59jHtJTu1y9xiq9UIogtD/kc= X-Gm-Gg: Acq92OGARNnMDJcsEmWMyBF75RHQVW6DRaMADoHHdOzUPF1bAE2ZdK+GYgqQmqUoVcy kOG0pshkG538DM/QcPFS+TYFvzsRDcKdArp85seMq5CYP3PlPvfxLGO/JDPDvSEr1skY9djo0Y5 ihHWYkWPcEJjBIVsRK00l2O+M6LB4eWrg4JP8XjaG4ItROVynzn65Bga/5kwSukjgfb4uRjUOoQ 8HgU+0plT1RD0f3KVCI2nHO/oxP+MmVUrvInWgGrHidkEy+51t7/V3l9ZMYPYJoSIksnVET9sLY gYlLWEXs/1OGbBIXIBQFvsOKc0gquijPeAg9UMY1/q9NVZJ4agM2y1JxERM52lJDBOfPgbzOca8 UUCVHQIdTopN0E5MAG571/48a71h2hdN+UxoviAsJc6s= X-Received: by 2002:a05:6402:2b96:b0:683:c39d:4da9 with SMTP id 4fb4d7f45d1cf-68836bd0ab3mr13644a12.25.1779315664801; Wed, 20 May 2026 15:21:04 -0700 (PDT) X-Received: by 2002:a05:6402:2b96:b0:683:c39d:4da9 with SMTP id 4fb4d7f45d1cf-68836bd0ab3mr13577a12.25.1779315664085; Wed, 20 May 2026 15:21:04 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-68311660f2fsm8228213a12.16.2026.05.20.15.20.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:21:03 -0700 (PDT) Date: Wed, 20 May 2026 18:20:57 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli , Jann Horn , Pedro Falcato , Harry Yoo , Hao Li Subject: [PATCH v8 07/37] mm: thread user_addr through page allocator for cache-friendly zeroing Message-ID: <84fba4b319a02e630fe4e1f75e66efbcfa513a09.1779315441.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Thread a user virtual address from vma_alloc_folio() down through the page allocator to post_alloc_hook(). This is plumbing preparation for a subsequent patch that will use user_addr to call folio_zero_user() for cache-friendly zeroing of user pages. The user_addr is stored in struct alloc_context and flows through: vma_alloc_folio -> folio_alloc_mpol -> __alloc_pages_mpol -> __alloc_frozen_pages -> get_page_from_freelist -> prep_new_page -> post_alloc_hook USER_ADDR_NONE ((unsigned long)-1) is used for non-user allocations, since address 0 is a valid userspace mapping. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- include/linux/gfp.h | 2 +- mm/compaction.c | 5 ++--- mm/hugetlb.c | 36 ++++++++++++++++++++---------------- mm/internal.h | 22 +++++++++++++++++++--- mm/mempolicy.c | 44 ++++++++++++++++++++++++++++++++------------ mm/mmap.c | 6 ++++++ mm/page_alloc.c | 44 +++++++++++++++++++++++++++++--------------- mm/slub.c | 4 ++-- 8 files changed, 111 insertions(+), 52 deletions(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index 7ccbda35b9ad..ee35c5367abc 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -337,7 +337,7 @@ static inline struct folio *folio_alloc_noprof(gfp_t gf= p, unsigned int order) static inline struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned in= t order, struct mempolicy *mpol, pgoff_t ilx, int nid) { - return folio_alloc_noprof(gfp, order); + return __folio_alloc_noprof(gfp, order, numa_node_id(), NULL); } #endif =20 diff --git a/mm/compaction.c b/mm/compaction.c index 3648ce22c807..72684fe81e83 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -82,7 +82,7 @@ static inline bool is_via_compact_memory(int order) { ret= urn false; } =20 static struct page *mark_allocated_noprof(struct page *page, unsigned int = order, gfp_t gfp_flags) { - post_alloc_hook(page, order, __GFP_MOVABLE); + post_alloc_hook(page, order, __GFP_MOVABLE, USER_ADDR_NONE); set_page_refcounted(page); return page; } @@ -1849,8 +1849,7 @@ static struct folio *compaction_alloc_noprof(struct f= olio *src, unsigned long da set_page_private(&freepage[size], start_order); } dst =3D (struct folio *)freepage; - - post_alloc_hook(&dst->page, order, __GFP_MOVABLE); + post_alloc_hook(&dst->page, order, __GFP_MOVABLE, USER_ADDR_NONE); set_page_refcounted(&dst->page); if (order) prep_compound_page(&dst->page, order); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index f24bf49be047..a999f3ead852 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1806,7 +1806,8 @@ struct address_space *hugetlb_folio_mapping_lock_writ= e(struct folio *folio) } =20 static struct folio *alloc_buddy_frozen_folio(int order, gfp_t gfp_mask, - int nid, nodemask_t *nmask, nodemask_t *node_alloc_noretry) + int nid, nodemask_t *nmask, nodemask_t *node_alloc_noretry, + unsigned long addr) { struct folio *folio; bool alloc_try_hard =3D true; @@ -1823,7 +1824,7 @@ static struct folio *alloc_buddy_frozen_folio(int ord= er, gfp_t gfp_mask, if (alloc_try_hard) gfp_mask |=3D __GFP_RETRY_MAYFAIL; =20 - folio =3D (struct folio *)__alloc_frozen_pages(gfp_mask, order, nid, nmas= k); + folio =3D (struct folio *)__alloc_frozen_pages(gfp_mask, order, nid, nmas= k, addr); =20 /* * If we did not specify __GFP_RETRY_MAYFAIL, but still got a @@ -1852,7 +1853,7 @@ static struct folio *alloc_buddy_frozen_folio(int ord= er, gfp_t gfp_mask, =20 static struct folio *only_alloc_fresh_hugetlb_folio(struct hstate *h, gfp_t gfp_mask, int nid, nodemask_t *nmask, - nodemask_t *node_alloc_noretry) + nodemask_t *node_alloc_noretry, unsigned long addr) { struct folio *folio; int order =3D huge_page_order(h); @@ -1864,7 +1865,7 @@ static struct folio *only_alloc_fresh_hugetlb_folio(s= truct hstate *h, folio =3D alloc_gigantic_frozen_folio(order, gfp_mask, nid, nmask); else folio =3D alloc_buddy_frozen_folio(order, gfp_mask, nid, nmask, - node_alloc_noretry); + node_alloc_noretry, addr); if (folio) init_new_hugetlb_folio(folio); return folio; @@ -1878,11 +1879,12 @@ static struct folio *only_alloc_fresh_hugetlb_folio= (struct hstate *h, * pages is zero, and the accounting must be done in the caller. */ static struct folio *alloc_fresh_hugetlb_folio(struct hstate *h, - gfp_t gfp_mask, int nid, nodemask_t *nmask) + gfp_t gfp_mask, int nid, nodemask_t *nmask, + unsigned long addr) { struct folio *folio; =20 - folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, NULL); + folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, NULL, a= ddr); if (folio) hugetlb_vmemmap_optimize_folio(h, folio); return folio; @@ -1922,7 +1924,7 @@ static struct folio *alloc_pool_huge_folio(struct hst= ate *h, struct folio *folio; =20 folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, node, - nodes_allowed, node_alloc_noretry); + nodes_allowed, node_alloc_noretry, USER_ADDR_NONE); if (folio) return folio; } @@ -2091,7 +2093,8 @@ int dissolve_free_hugetlb_folios(unsigned long start_= pfn, unsigned long end_pfn) * Allocates a fresh surplus page from the page allocator. */ static struct folio *alloc_surplus_hugetlb_folio(struct hstate *h, - gfp_t gfp_mask, int nid, nodemask_t *nmask) + gfp_t gfp_mask, int nid, nodemask_t *nmask, + unsigned long addr) { struct folio *folio =3D NULL; =20 @@ -2103,7 +2106,7 @@ static struct folio *alloc_surplus_hugetlb_folio(stru= ct hstate *h, goto out_unlock; spin_unlock_irq(&hugetlb_lock); =20 - folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask); + folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, addr); if (!folio) return NULL; =20 @@ -2146,7 +2149,7 @@ static struct folio *alloc_migrate_hugetlb_folio(stru= ct hstate *h, gfp_t gfp_mas if (hstate_is_gigantic(h)) return NULL; =20 - folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask); + folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, USER_ADDR_NO= NE); if (!folio) return NULL; =20 @@ -2182,14 +2185,14 @@ struct folio *alloc_buddy_hugetlb_folio_with_mpol(s= truct hstate *h, if (mpol_is_preferred_many(mpol)) { gfp_t gfp =3D gfp_mask & ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL); =20 - folio =3D alloc_surplus_hugetlb_folio(h, gfp, nid, nodemask); + folio =3D alloc_surplus_hugetlb_folio(h, gfp, nid, nodemask, addr); =20 /* Fallback to all nodes if page=3D=3DNULL */ nodemask =3D NULL; } =20 if (!folio) - folio =3D alloc_surplus_hugetlb_folio(h, gfp_mask, nid, nodemask); + folio =3D alloc_surplus_hugetlb_folio(h, gfp_mask, nid, nodemask, addr); mpol_cond_put(mpol); return folio; } @@ -2296,7 +2299,8 @@ static int gather_surplus_pages(struct hstate *h, lon= g delta) * down the road to pick the current node if that is the case. */ folio =3D alloc_surplus_hugetlb_folio(h, htlb_alloc_mask(h), - NUMA_NO_NODE, &alloc_nodemask); + NUMA_NO_NODE, &alloc_nodemask, + USER_ADDR_NONE); if (!folio) { alloc_ok =3D false; break; @@ -2702,7 +2706,7 @@ static int alloc_and_dissolve_hugetlb_folio(struct fo= lio *old_folio, spin_unlock_irq(&hugetlb_lock); gfp_mask =3D htlb_alloc_mask(h) | __GFP_THISNODE; new_folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, - nid, NULL); + nid, NULL, USER_ADDR_NONE); if (!new_folio) return -ENOMEM; goto retry; @@ -3400,13 +3404,13 @@ static void __init hugetlb_hstate_alloc_pages_oneno= de(struct hstate *h, int nid) gfp_t gfp_mask =3D htlb_alloc_mask(h) | __GFP_THISNODE; =20 folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, - &node_states[N_MEMORY], NULL); + &node_states[N_MEMORY], NULL, USER_ADDR_NONE); if (!folio && !list_empty(&folio_list) && hugetlb_vmemmap_optimizable_size(h)) { prep_and_add_allocated_folios(h, &folio_list); INIT_LIST_HEAD(&folio_list); folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, - &node_states[N_MEMORY], NULL); + &node_states[N_MEMORY], NULL, USER_ADDR_NONE); } if (!folio) break; diff --git a/mm/internal.h b/mm/internal.h index 5a2ddcf68e0b..389098200aa6 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -662,6 +662,16 @@ void calculate_min_free_kbytes(void); int __meminit init_per_zone_wmark_min(void); void page_alloc_sysctl_init(void); =20 +/* + * Sentinel for user_addr: indicates a non-user allocation. + * Cannot use 0 because address 0 is a valid userspace mapping. + * (unsigned long)-1 is safe because: + * 1. vm_end =3D addr + len <=3D TASK_SIZE, and vm_end is exclusive, + * so -1 is never inside any VMA. + * 2. It will only be compared to page-aligned addresses. + */ +#define USER_ADDR_NONE ((unsigned long)-1) + /* * Structure for holding the mostly immutable allocation parameters passed * between functions involved in allocations, including the alloc_pages* @@ -693,6 +703,7 @@ struct alloc_context { */ enum zone_type highest_zoneidx; bool spread_dirty_pages; + unsigned long user_addr; }; =20 /* @@ -916,24 +927,29 @@ static inline void init_compound_tail(struct page *ta= il, prep_compound_tail(tail, head, order); } =20 -void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flag= s); +void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flag= s, + unsigned long user_addr); extern bool free_pages_prepare(struct page *page, unsigned int order); =20 extern int user_min_free_kbytes; =20 struct page *__alloc_frozen_pages_noprof(gfp_t, unsigned int order, int ni= d, - nodemask_t *); + nodemask_t *, unsigned long user_addr); #define __alloc_frozen_pages(...) \ alloc_hooks(__alloc_frozen_pages_noprof(__VA_ARGS__)) void free_frozen_pages(struct page *page, unsigned int order); +void free_frozen_pages_zeroed(struct page *page, unsigned int order); void free_unref_folios(struct folio_batch *fbatch); =20 #ifdef CONFIG_NUMA struct page *alloc_frozen_pages_noprof(gfp_t, unsigned int order); +struct folio *folio_alloc_mpol_user_noprof(gfp_t gfp, unsigned int order, + struct mempolicy *pol, pgoff_t ilx, int nid, + unsigned long user_addr); #else static inline struct page *alloc_frozen_pages_noprof(gfp_t gfp, unsigned i= nt order) { - return __alloc_frozen_pages_noprof(gfp, order, numa_node_id(), NULL); + return __alloc_frozen_pages_noprof(gfp, order, numa_node_id(), NULL, USER= _ADDR_NONE); } #endif =20 diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 39e556e3d263..ea3043e0075b 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -2413,7 +2413,8 @@ bool mempolicy_in_oom_domain(struct task_struct *tsk, } =20 static struct page *alloc_pages_preferred_many(gfp_t gfp, unsigned int ord= er, - int nid, nodemask_t *nodemask) + int nid, nodemask_t *nodemask, + unsigned long user_addr) { struct page *page; gfp_t preferred_gfp; @@ -2426,25 +2427,29 @@ static struct page *alloc_pages_preferred_many(gfp_= t gfp, unsigned int order, */ preferred_gfp =3D gfp | __GFP_NOWARN; preferred_gfp &=3D ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL); - page =3D __alloc_frozen_pages_noprof(preferred_gfp, order, nid, nodemask); + page =3D __alloc_frozen_pages_noprof(preferred_gfp, order, nid, + nodemask, user_addr); if (!page) - page =3D __alloc_frozen_pages_noprof(gfp, order, nid, NULL); + page =3D __alloc_frozen_pages_noprof(gfp, order, nid, NULL, + user_addr); =20 return page; } =20 /** - * alloc_pages_mpol - Allocate pages according to NUMA mempolicy. + * __alloc_pages_mpol - Allocate pages according to NUMA mempolicy. * @gfp: GFP flags. * @order: Order of the page allocation. * @pol: Pointer to the NUMA mempolicy. * @ilx: Index for interleave mempolicy (also distinguishes alloc_pages()). * @nid: Preferred node (usually numa_node_id() but @mpol may override it). + * @user_addr: User fault address for cache-friendly zeroing, or USER_ADDR= _NONE. * * Return: The page on success or NULL if allocation fails. */ -static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order, - struct mempolicy *pol, pgoff_t ilx, int nid) +static struct page *__alloc_pages_mpol(gfp_t gfp, unsigned int order, + struct mempolicy *pol, pgoff_t ilx, int nid, + unsigned long user_addr) { nodemask_t *nodemask; struct page *page; @@ -2452,7 +2457,8 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsig= ned int order, nodemask =3D policy_nodemask(gfp, pol, ilx, &nid); =20 if (pol->mode =3D=3D MPOL_PREFERRED_MANY) - return alloc_pages_preferred_many(gfp, order, nid, nodemask); + return alloc_pages_preferred_many(gfp, order, nid, nodemask, + user_addr); =20 if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && /* filter "hugepage" allocation, unless from alloc_pages() */ @@ -2476,7 +2482,7 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsig= ned int order, */ page =3D __alloc_frozen_pages_noprof( gfp | __GFP_THISNODE | __GFP_NORETRY, order, - nid, NULL); + nid, NULL, user_addr); if (page || !(gfp & __GFP_DIRECT_RECLAIM)) return page; /* @@ -2488,7 +2494,7 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsig= ned int order, } } =20 - page =3D __alloc_frozen_pages_noprof(gfp, order, nid, nodemask); + page =3D __alloc_frozen_pages_noprof(gfp, order, nid, nodemask, user_addr= ); =20 if (unlikely(pol->mode =3D=3D MPOL_INTERLEAVE || pol->mode =3D=3D MPOL_WEIGHTED_INTERLEAVE) && page) { @@ -2504,11 +2510,18 @@ static struct page *alloc_pages_mpol(gfp_t gfp, uns= igned int order, return page; } =20 -struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order, +static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order, struct mempolicy *pol, pgoff_t ilx, int nid) { - struct page *page =3D alloc_pages_mpol(gfp | __GFP_COMP, order, pol, - ilx, nid); + return __alloc_pages_mpol(gfp, order, pol, ilx, nid, USER_ADDR_NONE); +} + +struct folio *folio_alloc_mpol_user_noprof(gfp_t gfp, unsigned int order, + struct mempolicy *pol, pgoff_t ilx, int nid, + unsigned long user_addr) +{ + struct page *page =3D __alloc_pages_mpol(gfp | __GFP_COMP, order, pol, + ilx, nid, user_addr); if (!page) return NULL; =20 @@ -2516,6 +2529,13 @@ struct folio *folio_alloc_mpol_noprof(gfp_t gfp, uns= igned int order, return page_rmappable_folio(page); } =20 +struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order, + struct mempolicy *pol, pgoff_t ilx, int nid) +{ + return folio_alloc_mpol_user_noprof(gfp, order, pol, ilx, nid, + USER_ADDR_NONE); +} + struct page *alloc_frozen_pages_noprof(gfp_t gfp, unsigned order) { struct mempolicy *pol =3D &default_policy; diff --git a/mm/mmap.c b/mm/mmap.c index 5754d1c36462..73413cebc418 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -855,6 +855,12 @@ __get_unmapped_area(struct file *file, unsigned long a= ddr, unsigned long len, if (IS_ERR_VALUE(addr)) return addr; =20 + /* + * The check below ensures vm_end =3D addr + len <=3D TASK_SIZE. + * Since (unsigned long)-1 (USER_ADDR_NONE) >=3D TASK_SIZE and + * vm_end is exclusive, USER_ADDR_NONE is thus never a valid + * userspace address. + */ if (addr > TASK_SIZE - len) return -ENOMEM; if (offset_in_page(addr)) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 0c4f4c678233..b96c9892f6c6 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1819,7 +1819,7 @@ static inline bool should_skip_init(gfp_t flags) } =20 inline void post_alloc_hook(struct page *page, unsigned int order, - gfp_t gfp_flags) + gfp_t gfp_flags, unsigned long user_addr) { bool init =3D !want_init_on_free() && want_init_on_alloc(gfp_flags) && !should_skip_init(gfp_flags); @@ -1874,9 +1874,10 @@ inline void post_alloc_hook(struct page *page, unsig= ned int order, } =20 static void prep_new_page(struct page *page, unsigned int order, gfp_t gfp= _flags, - unsigned int alloc_flags) + unsigned int alloc_flags, + unsigned long user_addr) { - post_alloc_hook(page, order, gfp_flags); + post_alloc_hook(page, order, gfp_flags, user_addr); =20 if (order && (gfp_flags & __GFP_COMP)) prep_compound_page(page, order); @@ -3958,7 +3959,8 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int o= rder, int alloc_flags, page =3D rmqueue(zonelist_zone(ac->preferred_zoneref), zone, order, gfp_mask, alloc_flags, ac->migratetype); if (page) { - prep_new_page(page, order, gfp_mask, alloc_flags); + prep_new_page(page, order, gfp_mask, alloc_flags, + ac->user_addr); =20 return page; } else { @@ -4186,7 +4188,8 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned= int order, =20 /* Prep a captured page if available */ if (page) - prep_new_page(page, order, gfp_mask, alloc_flags); + prep_new_page(page, order, gfp_mask, alloc_flags, + ac->user_addr); =20 /* Try get a page from the freelist if available */ if (!page) @@ -5063,7 +5066,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, struct zoneref *z; struct per_cpu_pages *pcp; struct list_head *pcp_list; - struct alloc_context ac; + struct alloc_context ac =3D { .user_addr =3D USER_ADDR_NONE }; gfp_t alloc_gfp; unsigned int alloc_flags =3D ALLOC_WMARK_LOW; int nr_populated =3D 0, nr_account =3D 0; @@ -5178,7 +5181,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, } nr_account++; =20 - prep_new_page(page, 0, gfp, 0); + prep_new_page(page, 0, gfp, 0, USER_ADDR_NONE); set_page_refcounted(page); page_array[nr_populated++] =3D page; } @@ -5203,12 +5206,13 @@ EXPORT_SYMBOL_GPL(alloc_pages_bulk_noprof); * This is the 'heart' of the zoned buddy allocator. */ struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order, - int preferred_nid, nodemask_t *nodemask) + int preferred_nid, nodemask_t *nodemask, + unsigned long user_addr) { struct page *page; unsigned int alloc_flags =3D ALLOC_WMARK_LOW; gfp_t alloc_gfp; /* The gfp_t that was actually used for allocation */ - struct alloc_context ac =3D { }; + struct alloc_context ac =3D { .user_addr =3D user_addr }; =20 /* * There are several places where we assume that the order value is sane @@ -5269,10 +5273,12 @@ EXPORT_SYMBOL(__alloc_frozen_pages_noprof); =20 struct page *__alloc_pages_noprof(gfp_t gfp, unsigned int order, int preferred_nid, nodemask_t *nodemask) + { struct page *page; =20 - page =3D __alloc_frozen_pages_noprof(gfp, order, preferred_nid, nodemask); + page =3D __alloc_frozen_pages_noprof(gfp, order, preferred_nid, + nodemask, USER_ADDR_NONE); if (page) set_page_refcounted(page); return page; @@ -5315,7 +5321,8 @@ struct folio *vma_alloc_folio_noprof(gfp_t gfp, int o= rder, gfp |=3D __GFP_NOWARN; =20 pol =3D get_vma_policy(vma, addr, order, &ilx); - folio =3D folio_alloc_mpol_noprof(gfp, order, pol, ilx, numa_node_id()); + folio =3D folio_alloc_mpol_user_noprof(gfp, order, pol, ilx, + numa_node_id(), addr); mpol_cond_put(pol); return folio; } @@ -5323,10 +5330,17 @@ struct folio *vma_alloc_folio_noprof(gfp_t gfp, int= order, struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, struct vm_area_struct *vma, unsigned long addr) { + struct page *page; + if (vma->vm_flags & VM_DROPPABLE) gfp |=3D __GFP_NOWARN; =20 - return folio_alloc_noprof(gfp, order); + page =3D __alloc_frozen_pages_noprof(gfp | __GFP_COMP, order, + numa_node_id(), NULL, addr); + if (!page) + return NULL; + set_page_refcounted(page); + return page_rmappable_folio(page); } #endif EXPORT_SYMBOL(vma_alloc_folio_noprof); @@ -6907,7 +6921,7 @@ static void split_free_frozen_pages(struct list_head = *list, gfp_t gfp_mask) list_for_each_entry_safe(page, next, &list[order], lru) { int i; =20 - post_alloc_hook(page, order, gfp_mask); + post_alloc_hook(page, order, gfp_mask, USER_ADDR_NONE); if (!order) continue; =20 @@ -7113,7 +7127,7 @@ int alloc_contig_frozen_range_noprof(unsigned long st= art, unsigned long end, struct page *head =3D pfn_to_page(start); =20 check_new_pages(head, order); - prep_new_page(head, order, gfp_mask, 0); + prep_new_page(head, order, gfp_mask, 0, USER_ADDR_NONE); } else { ret =3D -EINVAL; WARN(true, "PFN range: requested [%lu, %lu), allocated [%lu, %lu)\n", @@ -7778,7 +7792,7 @@ struct page *alloc_frozen_pages_nolock_noprof(gfp_t g= fp_flags, int nid, unsigned gfp_t alloc_gfp =3D __GFP_NOWARN | __GFP_ZERO | __GFP_NOMEMALLOC | __GFP_= COMP | gfp_flags; unsigned int alloc_flags =3D ALLOC_TRYLOCK; - struct alloc_context ac =3D { }; + struct alloc_context ac =3D { .user_addr =3D USER_ADDR_NONE }; struct page *page; =20 VM_WARN_ON_ONCE(gfp_flags & ~__GFP_ACCOUNT); diff --git a/mm/slub.c b/mm/slub.c index 0baa906f39ab..74dd2d96941b 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -3275,7 +3275,7 @@ static inline struct slab *alloc_slab_page(gfp_t flag= s, int node, else if (node =3D=3D NUMA_NO_NODE) page =3D alloc_frozen_pages(flags, order); else - page =3D __alloc_frozen_pages(flags, order, node, NULL); + page =3D __alloc_frozen_pages(flags, order, node, NULL, USER_ADDR_NONE); =20 if (!page) return NULL; @@ -5235,7 +5235,7 @@ static void *___kmalloc_large_node(size_t size, gfp_t= flags, int node) if (node =3D=3D NUMA_NO_NODE) page =3D alloc_frozen_pages_noprof(flags, order); else - page =3D __alloc_frozen_pages_noprof(flags, order, node, NULL); + page =3D __alloc_frozen_pages_noprof(flags, order, node, NULL, USER_ADDR= _NONE); =20 if (page) { ptr =3D page_address(page); --=20 MST From nobody Sun May 24 22:35:50 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1EEF23BED5C for ; Wed, 20 May 2026 22:21:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315676; cv=none; b=enxfRgW2b5AH7AYr4H1HZMfy6BMX0XP0oQ8lKH3nF/4T5hPV+cgJTFumOlZQiaOMKte+WhK1thQ4oBf/8XbRNDbaxAgcDuNHMg4fycAiKozzjvVwJdyLiDNNtudM76UGoqMU3f8U74vkc5Gdd37GMeA3lU0Ze4G4VMuNY/7Iets= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315676; c=relaxed/simple; bh=8w0BAdcYEx1DYP7LpA2admc4ftuLWgtvGXP7cauVxpA=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=X5MFNJh68E+6U2wNIZmciqGayeE5teKEf+5p6Wg0X1MKC/UTKEVWxyUInJwpRVYSTV4besm6NBSBUncoZZXO8ZcswNpOH9OIlo+CWua4gJF55mE9zfTLQUppHH3jKNd2gUUZhj2DFBQVWt7L6UqTuZcuY5Hi+Fe4QUm1N/jd86g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=GMXnREsD; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=rTkxLCzo; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="GMXnREsD"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="rTkxLCzo" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315674; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=VBSoYVls/goaUOaeb0ArVPkRixJg6uACMT8ZdsicbY4=; b=GMXnREsDaprFpzAtZfqm8EbgOYMApfAAA4DrgM309W64Mh5JF68LyPr/6lU/7jyy4VNQK8 wQSyudThi18iPolQ978gTPaLvIvqPcBzneol8klXHlZZRYlk5FWrbOzerbqGn02jkQWR5W iSpLjuCcaMM3Vu6J3cS5R2wExZNvB9w= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-690-woH7FChOMaWyTVZaiZ6FOQ-1; Wed, 20 May 2026 18:21:13 -0400 X-MC-Unique: woH7FChOMaWyTVZaiZ6FOQ-1 X-Mimecast-MFC-AGG-ID: woH7FChOMaWyTVZaiZ6FOQ_1779315672 Received: by mail-ed1-f72.google.com with SMTP id 4fb4d7f45d1cf-67b88d32946so4831012a12.3 for ; Wed, 20 May 2026 15:21:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315672; x=1779920472; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=VBSoYVls/goaUOaeb0ArVPkRixJg6uACMT8ZdsicbY4=; b=rTkxLCzoJYWm90uLTi7A17HuUc+HHSpEXelW/d9OI9kjY4v3mNGJpSKckAYEikfcmG SOPUQEg54NaPyUHUxZGB9sBxkpEvt6N6j0drx5VIW4ZTYhBS3xGgfAyP/AI4HRaeMSZq +vFLs2+jPrp1kpfNoILj3j9kXoVlPqDafGxd8te3kn6hy44iEDvUtWhAqIsS/l4mzqNi +FNVYEEhGZ0676nK+bnFWD3jcM/Q4g+F77osxMU+LkNZHXiG7jxzJMk6SGI3jfn/zn4Y R8f9u6kpqDsDqKG77+MFscfUQRKdeEbfG7sHv3SaMWtAfs7kd9v3uLeXr1TdLJrhMD/5 TzgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315672; x=1779920472; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VBSoYVls/goaUOaeb0ArVPkRixJg6uACMT8ZdsicbY4=; b=l+7N6lJMumyA301x2avhvShk7OOTPHlGLAO6qFWF380dguixCgAvMT1jiqlRDj33GW p8/IrpCXylDPIAuDG+QJkVSLB3r7TX+IfWpe9noJOGfqRLUvp0HcPNZvQ1E43VME7ASt /2wgLH/r6w6xhiXyQlPHam/YqCV0iWPTtCUfOWKDyCLojXCAu1wQqAxH1/WaJde95wSU aCFRdjJ+pqG/8Ixq0QgnSDMzuWTm1zEdux1AEOWaDjWO/aL4f0qsusTNZBTHNCd+C8m5 kll7oPAku7ZkD23GF5TyYqAXQ83LsIl7/MEzN+xQWDMZVtM1la4+5YI2CTzJB3X9XLg1 tSfA== X-Gm-Message-State: AOJu0YwiedjanC9oHr5cyH5PxT6UTnDClExM3Qs9TB6JicL6VsxV9vvL /bVqKxec240XLefHa8Y7Ou4RhKS9oy+coo0jGv2Qn+sskBU8uQ1AH/fxYMXNVCE16PC4q+zw1Xd VxHjfloDVlqenohgn9VaR0VKsKiRhvhdMYj6H+XDzHUggqTUA54GNqE4DfflOKIdNmTn3PixW+q 6jtkoE198x2nKYtmHpMKMhk6w8sMGdLKFWPz3vM5uBvjk= X-Gm-Gg: Acq92OFOCBMCyk6G7ZS1ckmSNFNDGV5kqWimYXISNTb9pOyX3Rw1aYFOwuDBIhKpOIR Qa6rg3xyAwpRvucCJRgNkmkmCwHB8FVUvCha2nL+foUYB7jSEkyFQrR3bVrocTsasCA64XODZXI 7ldX/0zExmbrQy2jXEOepubJFXvyle1DibEgU2G4omHEIKBuRjWiBm/+OBm786iVMgqSY+z0h+5 v6zd+nP2X2P3/5X36k2aFRPJCoxj9VrbRxaF+XHlBydx+51TuKrMOOrwcdFr263FURc/0jEsK2g K0v/3H/5b2bSXe2DzNloUcB9KprnOXTqSR6oh+WwTyK6ngIeAe1YkoFyUg/bzHaTajRKCzWAi+9 bo4i6crezQpuI2XfuwurLhRm/7T1435HAGCPMqq5dMnQ= X-Received: by 2002:a05:6402:278b:b0:66a:19bd:5cb8 with SMTP id 4fb4d7f45d1cf-68836ace712mr19650a12.18.1779315671742; Wed, 20 May 2026 15:21:11 -0700 (PDT) X-Received: by 2002:a05:6402:278b:b0:66a:19bd:5cb8 with SMTP id 4fb4d7f45d1cf-68836ace712mr19586a12.18.1779315671078; Wed, 20 May 2026 15:21:11 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-68310b3e8e5sm8646938a12.4.2026.05.20.15.21.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:21:09 -0700 (PDT) Date: Wed, 20 May 2026 18:21:04 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 08/37] mm: add alloc_contig_frozen_pages_user for cache-friendly zeroing Message-ID: <001e0eaa4ea51951e0a78cea5d6a61ddbc6a099a.1779315441.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a _user variant of alloc_contig_frozen_pages that accepts a user_addr parameter for cache-friendly zeroing of contiguous allocations. No functional change; all existing callers continue to pass USER_ADDR_NONE. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/gfp.h | 6 ++++++ mm/page_alloc.c | 32 +++++++++++++++++++++++++------- 2 files changed, 31 insertions(+), 7 deletions(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index ee35c5367abc..73109d4e31a4 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -453,6 +453,12 @@ struct page *alloc_contig_frozen_pages_noprof(unsigned= long nr_pages, #define alloc_contig_frozen_pages(...) \ alloc_hooks(alloc_contig_frozen_pages_noprof(__VA_ARGS__)) =20 +struct page *alloc_contig_frozen_pages_user_noprof(unsigned long nr_pages, + gfp_t gfp_mask, int nid, nodemask_t *nodemask, + unsigned long user_addr); +#define alloc_contig_frozen_pages_user(...) \ + alloc_hooks(alloc_contig_frozen_pages_user_noprof(__VA_ARGS__)) + struct page *alloc_contig_pages_noprof(unsigned long nr_pages, gfp_t gfp_m= ask, int nid, nodemask_t *nodemask); #define alloc_contig_pages(...) \ diff --git a/mm/page_alloc.c b/mm/page_alloc.c index b96c9892f6c6..b725a3b59835 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -6999,8 +6999,9 @@ static void __free_contig_frozen_range(unsigned long = pfn, unsigned long nr_pages * * Return: zero on success or negative error code. */ -int alloc_contig_frozen_range_noprof(unsigned long start, unsigned long en= d, - acr_flags_t alloc_flags, gfp_t gfp_mask) +static int __alloc_contig_frozen_range(unsigned long start, unsigned long = end, + acr_flags_t alloc_flags, gfp_t gfp_mask, + unsigned long user_addr) { const unsigned int order =3D ilog2(end - start); unsigned long outer_start, outer_end; @@ -7127,7 +7128,7 @@ int alloc_contig_frozen_range_noprof(unsigned long st= art, unsigned long end, struct page *head =3D pfn_to_page(start); =20 check_new_pages(head, order); - prep_new_page(head, order, gfp_mask, 0, USER_ADDR_NONE); + prep_new_page(head, order, gfp_mask, 0, user_addr); } else { ret =3D -EINVAL; WARN(true, "PFN range: requested [%lu, %lu), allocated [%lu, %lu)\n", @@ -7137,6 +7138,13 @@ int alloc_contig_frozen_range_noprof(unsigned long s= tart, unsigned long end, undo_isolate_page_range(start, end); return ret; } + +int alloc_contig_frozen_range_noprof(unsigned long start, unsigned long en= d, + acr_flags_t alloc_flags, gfp_t gfp_mask) +{ + return __alloc_contig_frozen_range(start, end, alloc_flags, gfp_mask, + USER_ADDR_NONE); +} EXPORT_SYMBOL(alloc_contig_frozen_range_noprof); =20 /** @@ -7255,8 +7263,9 @@ static bool zone_spans_last_pfn(const struct zone *zo= ne, * * Return: pointer to contiguous frozen pages on success, or NULL if not s= uccessful. */ -struct page *alloc_contig_frozen_pages_noprof(unsigned long nr_pages, - gfp_t gfp_mask, int nid, nodemask_t *nodemask) +struct page *alloc_contig_frozen_pages_user_noprof(unsigned long nr_pages, + gfp_t gfp_mask, int nid, nodemask_t *nodemask, + unsigned long user_addr) { unsigned long ret, pfn, flags; struct zonelist *zonelist; @@ -7284,10 +7293,11 @@ struct page *alloc_contig_frozen_pages_noprof(unsig= ned long nr_pages, * win the race and cause allocation to fail. */ spin_unlock_irqrestore(&zone->lock, flags); - ret =3D alloc_contig_frozen_range_noprof(pfn, + ret =3D __alloc_contig_frozen_range(pfn, pfn + nr_pages, ACR_FLAGS_NONE, - gfp_mask); + gfp_mask, + user_addr); if (!ret) return pfn_to_page(pfn); spin_lock_irqsave(&zone->lock, flags); @@ -7309,6 +7319,14 @@ struct page *alloc_contig_frozen_pages_noprof(unsign= ed long nr_pages, } return NULL; } +EXPORT_SYMBOL(alloc_contig_frozen_pages_user_noprof); + +struct page *alloc_contig_frozen_pages_noprof(unsigned long nr_pages, + gfp_t gfp_mask, int nid, nodemask_t *nodemask) +{ + return alloc_contig_frozen_pages_user_noprof(nr_pages, gfp_mask, nid, + nodemask, USER_ADDR_NONE); +} EXPORT_SYMBOL(alloc_contig_frozen_pages_noprof); =20 /** --=20 MST From nobody Sun May 24 22:35:50 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 169513B637A for ; Wed, 20 May 2026 22:21:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315683; cv=none; b=RpywxuFv+gbkhUihTCsMHgrxQmHJgLB5E9YZCBnDUdPBuawatMayLvIoZD53MtqKhsA/0lWYi/zG4xrxI9LTaKFqSLWxgOhfGNHiF6nFfA9U+EafjOIPjKLWivHXgot0IzuDTvg/a6bR9f33/GXByDzBNM66gkU5lv2hzdIY8L8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315683; c=relaxed/simple; bh=nrsmIXcyZYzWzlqPkl9t9uUpVmTvW8/nYuSQvt237DY=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=HzefBOxxQn0S2QwhyVrrUe3w2hKYlHyjYUhtED1iKojpYzDJCJchPGPHuPJcXgkw+jouoUGP+xytvEr4jV20KECyS1GZCrj3fvxOh+bvGGJPGfHsDFZY0rvJaS9K8OQTglHwMRt8BIiGOqExbXMoLxJ1Khm0RUF3paBrBuPB9Ps= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=SI+GgCCM; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=pU6yKxNA; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="SI+GgCCM"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="pU6yKxNA" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315681; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Gi0K9k7DoK/Ig5tkAzEbdYCqiHQSfFH4LjBzL6zaPFI=; b=SI+GgCCMJ1ZXvoS6QBjbSbj1Zcu+oZ4Fq/GOFZvnHKlkyWGgy+ZYPrdgrKhnibLmyvFUzp ZSuQKGwjw3hF0BuFTPbfOQJHmeCpXyz2XjL0nnvQqSpgyfAbdejwUcD+S1pfYPxA5blFT2 9670lKYYFIxvn1NZRpGp2TAFC2TaEaU= Received: from mail-ed1-f71.google.com (mail-ed1-f71.google.com [209.85.208.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-642-P4cN4CljMQ25pS6_tK9s8w-1; Wed, 20 May 2026 18:21:19 -0400 X-MC-Unique: P4cN4CljMQ25pS6_tK9s8w-1 X-Mimecast-MFC-AGG-ID: P4cN4CljMQ25pS6_tK9s8w_1779315679 Received: by mail-ed1-f71.google.com with SMTP id 4fb4d7f45d1cf-680308c6c62so4882642a12.1 for ; Wed, 20 May 2026 15:21:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315678; x=1779920478; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Gi0K9k7DoK/Ig5tkAzEbdYCqiHQSfFH4LjBzL6zaPFI=; b=pU6yKxNAtHwbVRfWKPMMfqCl7yiIkiqhrA4uZQ6tKBL6yozS7LF5QhyIFf+qTjBwT7 EpOpJgfYvMyvuBaFK6yDkuZuLpsTOn1CxF6wtX3gqRQRdGKK3BBiBz4l9wc30J432B0x PqosHx+TfKVD2Pb7dQdfgZsdpwMinsuvlb1yj0H3S4TLxlKiU38r/574Pha0IkqW2c5U fbWuwrDxaffMUt6TkY/wN69PL5pbIKstELro5kzLAzMryQnBpK2F7lp9X2UxNqe0LGCc o1MGHqBM9aAsdOKIKKowychsAM+3h+9k753TuM20nt310OIT/CsJ3l35VRCmLDgtoka/ NQCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315678; x=1779920478; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Gi0K9k7DoK/Ig5tkAzEbdYCqiHQSfFH4LjBzL6zaPFI=; b=gUquYwKm1wWkfY3m6nACzAPQPOaQt0KoK2peckPfmYyLZVNiRsbvwGssn29D6JIUph 1cQwpQtdzK+ImOtZMGncpfESDtpIjvp+WbgQMiHEilalkv5r69yi09r8K7m/gGkJ+74O pbZW5ibIw7Bj8j/oP9TkFtl89pGsYS6VsKgqZbZSwAH7uFBt4itbrMl+mRw8DKPwbJKm pm1AFhbQI63b9GMqUsc/tNbes/6lzYCcPU+FKNwmNumBuz2YJ/eEW8IYrjEX3EwEzp9q TMy4qkiacx7on8nHeAUaLqtWeATZPjfSrO1892hOp5l8eSaJbpr7Qd/ME22b5W1T86N2 oggw== X-Gm-Message-State: AOJu0YyI6aqNIgiRHwhe6O3gl0Id/1L2UaY+uZytwYJlaT/9wT9+r/Qc wMuID0DhNf50mZKPW97Me42Qsx8WbcPVvpm3oSUpgBkkuFXHIaZn1uBJoZ6H1bMu0ZSSpUZdcWU UJ3ok3L8OstCGCzp/4oE1YHaathUY1o58mBmxJKYFZ2vMr8uueorS+NMJkRSws6AyV0VQFj2rnW LPIt6Qrsmz3MBoFnblnDGx7ONpd+vdFQ0aDQq0ArgSczE= X-Gm-Gg: Acq92OEVmK5RPun55JSKEheoHBfmfNqb7SCAmdtno+G0MnPnpJoUqQ9zyow8NaNfI5X Q5pQYoExN26BvXeofPPYTnWWgp9Yaz3rTwzv6JPdZ1+JAU6VRxzgrXu02mtwwz76yljyHTfAMI2 HsKWhKBBhmUmGuZPBjKxXEmel3FqiKZFwbVzeMrz6H4fWM6UyIodHoIdtakZ9x2p/YtTwVcLZYb eAiJLCk9CyUIjNFryORR/4mGfBmuVCTm/uVKZ7i36xw40dUrUwrHGLQps4yCNI7aSBvfVcDFTW6 jmQDrUhOSt4UkXEevmgWNIZyGuyDjvB/wjJEsIcup/nLrHDZ/7Iush+h0SR7St0o4WhOiaaCBxE nfNJw82XtFOF5cXWpHXXUx0Ls4AKIySTKnT8os0jF/no= X-Received: by 2002:a05:6402:388e:b0:67f:d8bc:1c1d with SMTP id 4fb4d7f45d1cf-688367d7471mr20731a12.20.1779315678512; Wed, 20 May 2026 15:21:18 -0700 (PDT) X-Received: by 2002:a05:6402:388e:b0:67f:d8bc:1c1d with SMTP id 4fb4d7f45d1cf-688367d7471mr20674a12.20.1779315678044; Wed, 20 May 2026 15:21:18 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-687393e8e89sm1281214a12.8.2026.05.20.15.21.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:21:16 -0700 (PDT) Date: Wed, 20 May 2026 18:21:11 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 09/37] mm: hugetlb: thread user_addr through gigantic page allocation Message-ID: <9778009828d334fd4262d60b20d8d25a54e0f133.1779315441.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Thread the user_addr parameter through alloc_gigantic_frozen_folio so that gigantic page allocations can benefit from cache-friendly zeroing. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/hugetlb.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index a999f3ead852..3a6afbe99116 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1375,7 +1375,7 @@ static struct folio *dequeue_hugetlb_folio_vma(struct= hstate *h, =20 #if defined(CONFIG_ARCH_HAS_GIGANTIC_PAGE) && defined(CONFIG_CONTIG_ALLOC) static struct folio *alloc_gigantic_frozen_folio(int order, gfp_t gfp_mask, - int nid, nodemask_t *nodemask) + int nid, nodemask_t *nodemask, unsigned long addr) { struct folio *folio; =20 @@ -1386,13 +1386,15 @@ static struct folio *alloc_gigantic_frozen_folio(in= t order, gfp_t gfp_mask, if (hugetlb_cma_exclusive_alloc()) return NULL; =20 - folio =3D (struct folio *)alloc_contig_frozen_pages(1 << order, gfp_mask, - nid, nodemask); + folio =3D (struct folio *)alloc_contig_frozen_pages_user(1 << order, + gfp_mask, + nid, nodemask, + addr); return folio; } #else /* !CONFIG_ARCH_HAS_GIGANTIC_PAGE || !CONFIG_CONTIG_ALLOC */ static struct folio *alloc_gigantic_frozen_folio(int order, gfp_t gfp_mask= , int nid, - nodemask_t *nodemask) + nodemask_t *nodemask, unsigned long addr) { return NULL; } @@ -1862,7 +1864,8 @@ static struct folio *only_alloc_fresh_hugetlb_folio(s= truct hstate *h, nid =3D numa_mem_id(); =20 if (order_is_gigantic(order)) - folio =3D alloc_gigantic_frozen_folio(order, gfp_mask, nid, nmask); + folio =3D alloc_gigantic_frozen_folio(order, gfp_mask, nid, nmask, + addr); else folio =3D alloc_buddy_frozen_folio(order, gfp_mask, nid, nmask, node_alloc_noretry, addr); --=20 MST From nobody Sun May 24 22:35:50 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5FEDA3B0AE5 for ; Wed, 20 May 2026 22:21:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315688; cv=none; b=JDtJ4wjCHermtfXvh4JAKuiiUbGjwHZDFyZA0kKu+zGqMB59FTeAjDzebY5Bf2CknUeIFdRuiZOPfwCDFe0BZ5aAT1pbyL1wKb1qjiBsKiP0f5TaBmfRFCUDgwRk2KtX7TZDGNJApOVnOYHX6Uibkw+fZMX1k0lqeSOvGGgp+wM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315688; c=relaxed/simple; bh=nrMNVIUtRT6KxUfPFXjAD5jr466RgmTT9RZMroXpfzs=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=GwaqM3N6iZb6LuvU85zCbho2UNwS1z5sUFHFCRUQEyP5zqRiVY0JdEoz0Zh3e48EU4yf+D1nkjdmEDCEpxwyP2kP2qAlFA1i7J28baq11nXEh2v3kB9oJ0qVkGCZftW0T0uFyf0ZXh5qz673rgPoHL/n7hG4ApmBKdZbi6DMQxo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=BLox6b2b; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=eygIwWUL; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="BLox6b2b"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="eygIwWUL" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315686; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=phYVyxy4NXZDWgDwVjZEeX0+DVaGt3sk04puvAHm6ak=; b=BLox6b2bqUO68wHW0ta08CFNoOF7XaUJBm9oGQ/UGzqEmk9urwDFmyM53USjrwbuSK9Tw9 tCrwLT6WFdIbD39FfoR3Dz8HFzQT0x0lli7NVAt+tjcnR1ApZobdA9cQ0XIwb4sofEydJ8 X+PSOsm4Dqd4xW1dVQ92stUsahEUB3s= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-378-vemcGC7aOZ6AcUtrHC8wSw-1; Wed, 20 May 2026 18:21:25 -0400 X-MC-Unique: vemcGC7aOZ6AcUtrHC8wSw-1 X-Mimecast-MFC-AGG-ID: vemcGC7aOZ6AcUtrHC8wSw_1779315684 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-45e2f74da10so5971997f8f.2 for ; Wed, 20 May 2026 15:21:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315684; x=1779920484; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=phYVyxy4NXZDWgDwVjZEeX0+DVaGt3sk04puvAHm6ak=; b=eygIwWULxV0W+Bxt3c0upEN1phdpvQTP8sJNoH5orxbfJvfxDQsGpHn1i3nviDF+Ut unISWMWYoh1N+CK0HEmmBh4MMvNqrjK7xqnXtJ7kcimjVDrAVNht85uhIfU9fueXPMvW FYvhu2fTVAdHo3qSDRuD6f9P9FL79yINj/IkzqqugyymJr9i+BkIhpUTlfimXHg43Iun GmELybMmSUooJnILt1sCst1DxI4szaZsJ81/hzXz8reS5hzUZuwtAKp8XOzFCDQc/7iU mop3Yt21iTsFV58+yiDf4oNLUp2+5orLpZeuMOL8WeQRHba6xIeRuaq/z2/bkL4xfIyu UlTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315684; x=1779920484; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=phYVyxy4NXZDWgDwVjZEeX0+DVaGt3sk04puvAHm6ak=; b=BGsE3ivBdFxMlZJl/+gPYTBnpWm5kmicOAmF5PacJLWywDzqbCE2fOOtVkL7NsfDP/ F6MWzWkjVwmlmTfR5E9Q+rEwDDogK11xvFAK7ZcGGxCNTyJa9ilKJb2WznkOdGq+zPUE eDi8AS88V3ff3OGx/ZGLqduPqxni5fS9/76xuO7snDYR2jXcRo0ntbo2OZ1m7KB2nQ8G rB6unPUkMYTQSiJp1k1wT49I6Mt5BhaW5aNDbDTQT1UL/VtHFAqRrlPsEZGEePz1Q6MU HM9jitxi6kw0g+n+jDjJWD+1Jv0liXZgtmHvcAufyqOr8YPwI9MQfpvXQHqvSo3BMfsG 83XA== X-Gm-Message-State: AOJu0YwaVSNHH9yoRr3P3OFvFr3i3QwUeNkD9pe1BZKOcD87Ngoqh8h2 XSet4sDiCzKwrMSYMovm/ECNTuXdHk/4QsfN/NAAYNhGSVR3TJpHFJ9S/4GheaAbVpSMoI3cNS1 6EeYhp6NlOFjDxZYvShsnRngQL8eQi+penr6ytd690Ap35aT2RYEhjmwMb4NLpaNltLJ0YUEq6m h7g72yJYZ6owkwPzd2WEmV+Vn7OHZQ7OcQAMmsqf4Dk/U= X-Gm-Gg: Acq92OHsasavlkSsEepa3DnmQBXlI92gLYWJt4CGhLJXLEQNdgBQBXqWNziOFJFv+Ay 7bG4jaA+tktnamjN0ynv1hc4oqVkVzO9ywHynV1Sgr1BS/4uSJav9LbiQhPkFS6Ps2r7nYYMz1N yKyLmqwhpZTSuUQnIfeSC7nbWZTLkZJ3p2BjwBm15UsibVt1JfMJUm+s05vLcDqn7G7bM+0mLdi GQIC26/7GPPDSKd04c2ESFoCS/LdE6TbyVmb6FmHF9z7YV3joXCa9CpJeIUjDKzx3niPGkoTzqH jMhezrmULG+3CVz2xBEj5LOMxd1d5S6SCbcL0wnvfNvxN1w1C4f3vqMvfMMs52PvouKuI58wfi8 JuUrT1KMKKeYDTEF7gPLM3YmtBB5KH4Jqy1Gcwgy7/ws= X-Received: by 2002:a05:6000:2c05:b0:43d:309b:9c4f with SMTP id ffacd0b85a97d-45ea38cd4d7mr309390f8f.6.1779315684018; Wed, 20 May 2026 15:21:24 -0700 (PDT) X-Received: by 2002:a05:6000:2c05:b0:43d:309b:9c4f with SMTP id ffacd0b85a97d-45ea38cd4d7mr309285f8f.6.1779315683262; Wed, 20 May 2026 15:21:23 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45da0a1aeafsm59910901f8f.23.2026.05.20.15.21.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:21:22 -0700 (PDT) Date: Wed, 20 May 2026 18:21:18 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 10/37] mm: add folio_zero_user stub for configs without THP/HUGETLBFS Message-ID: <501f8a7a582a97011a23e3051f3d0999c66c10bc.1779315441.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" folio_zero_user() is defined in mm/memory.c under CONFIG_TRANSPARENT_HUGEPAGE || CONFIG_HUGETLBFS. A subsequent patch will call it from post_alloc_hook() for all user page zeroing, so configs without THP or HUGETLBFS will need a stub. Add a stub that uses clear_user_highpages() with aligned addr_hint. Without THP/HUGETLBFS, only order-0 user pages are allocated, so the locality optimization in the real folio_zero_user() (zero near the faulting address last) is not needed. This also matches what vma_alloc_zeroed_movable_folio currently does. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- include/linux/mm.h | 2 ++ mm/memory.c | 9 +++++++++ 2 files changed, 11 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index af23453e9dbd..d1e768dcda13 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -5070,6 +5070,8 @@ long copy_folio_from_user(struct folio *dst_folio, const void __user *usr_src, bool allow_pagefault); =20 +#else /* !CONFIG_TRANSPARENT_HUGEPAGE && !CONFIG_HUGETLBFS */ +void folio_zero_user(struct folio *folio, unsigned long addr_hint); #endif /* CONFIG_TRANSPARENT_HUGEPAGE || CONFIG_HUGETLBFS */ =20 #if MAX_NUMNODES > 1 diff --git a/mm/memory.c b/mm/memory.c index ea6568571131..02d3e53fc91b 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -7546,6 +7546,15 @@ long copy_folio_from_user(struct folio *dst_folio, } #endif /* CONFIG_TRANSPARENT_HUGEPAGE || CONFIG_HUGETLBFS */ =20 +#if !defined(CONFIG_TRANSPARENT_HUGEPAGE) && !defined(CONFIG_HUGETLBFS) +void folio_zero_user(struct folio *folio, unsigned long addr_hint) +{ + unsigned long base =3D ALIGN_DOWN(addr_hint, folio_size(folio)); + + clear_user_highpages(&folio->page, base, folio_nr_pages(folio)); +} +#endif + #if defined(CONFIG_SPLIT_PTE_PTLOCKS) && ALLOC_SPLIT_PTLOCKS =20 static struct kmem_cache *page_ptl_cachep; --=20 MST From nobody Sun May 24 22:35:50 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3AB713BF66C for ; Wed, 20 May 2026 22:21:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315693; cv=none; b=H3ePQJArqYKh/3ezd0RbG08oIsOVrdob4nyecgtp6woeKqBoWavCTqhVDztCEEZ8qvXrjIQmQ2PhbnEjR4JKDte29cYPlxkO2dWaESjTBuF0BiT+dPBRq5iia3O0Xmv957lyUL+LIqu6yvzaA3PjSZMWk9psiYBmneVVR/v94F4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315693; c=relaxed/simple; bh=W8/RgV8YlHoI/N38IUKKZXJPftBVftzLE1J25cjwbi0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=XNkqnUkBDrkRKGXj8EgUskxEXFCmz2M01LcFAiUJA1DXBQyiarTmcRu3FbnkHaBOV8+EatPRzSyJpLNcM9ra8eoA5OAfhVemdWluYZefN9TSaVu6FBl9ZB13UaN2fgiO+6rRlntg26a0w0VH5iW9zE1YdwgeyuNulN0jXAok0P0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=W10NeEB9; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=iM0sKAUU; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="W10NeEB9"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="iM0sKAUU" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315691; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=kjfty6vgLrS4T7uumYbV6HhVn6mEWZk8nuiVgC9VboM=; b=W10NeEB9XVrAaYPTBmsFf2gNs4cFmHHxMlYP1aoRQ6vlGgzF6f9RLwh3LY9PzNn4M9nGEH FOOlHj1P7Livlb3wGR3DUIPAlp9hqVkjUYNpNuCBAiA3iaf6JUMvnFn3XfdamPhABN0ZQ2 ndmbUQVtQDsRYSgeANF+M++1YducH0U= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-235-D0NsMJBvO1m6L36Q2uf-5A-1; Wed, 20 May 2026 18:21:30 -0400 X-MC-Unique: D0NsMJBvO1m6L36Q2uf-5A-1 X-Mimecast-MFC-AGG-ID: D0NsMJBvO1m6L36Q2uf-5A_1779315689 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-48fde68e420so47682295e9.0 for ; Wed, 20 May 2026 15:21:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315689; x=1779920489; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=kjfty6vgLrS4T7uumYbV6HhVn6mEWZk8nuiVgC9VboM=; b=iM0sKAUUJ0bRjmel8lGjCyw9Iypua1eWbmrXVHCYCBTS9yPfn9Ep9GsY5TfdWo7sUl ThoDx0jIhG1vnM/HU4zRRWDbsw2zW87C4sT3QH4twLi8zfBe8uSgHv3ztPDKY3hqb1Wa R65eA/os8TMB84EnSHJJZYsa4vJeYkbko3CUUGwCmk9lYk4m1qc3Qar5YJ+0U2zxDsdP SnzGy1I7tsZ4VuNVgz4edzICO7qEUeAk2irdZ53ZZvDmCf6FSQGXLPjGfdeCtaqo5vm+ HWdDadOaYqDfLlpS+gc0uisK+Eps4XZkiKG6tzfSom0LlK4rAxoJyoH1en57zFL35P0W 1Y6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315689; x=1779920489; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=kjfty6vgLrS4T7uumYbV6HhVn6mEWZk8nuiVgC9VboM=; b=I5VDP0WgTOI5ptfFq1TTtDIblE/Bt0/o0ULyHNjebAPnuLXkG9/piFek2ppswkGAqZ pH4DKQbb+zdpSMnQds+WkGZ29f75QA7VSl1cHzIgN1tUzZlY8h8WGBSic1g+kl32Y8iE aqOt/wDgsxqCEv8A/ufxtQmogDg1HiTzoSFi2rvz3sYZJR2xIJ+XMtF8V6noQ1Q1RQpH Yj+1znyCkBFyJG/0LJOD40DIxD8i5+n1fJ+Nd9+8tzwv5LEoMePaK39g2CVdAuz3nWaO 6gP1DTemtCKdE4CKyQ4xDxQ01U5GnKRO918nYjh0Leml0PLxYwKp+TR1BDSw9IPpJA9g dRTg== X-Gm-Message-State: AOJu0YyKWluypIcCygIIlCVMWqyksFLGmdfy0F+d/2kmJxRIUnfSx1QJ vMEbfO0VKizjRdwTwXa2szwU0Rl+BpNthm1m0O4zk8RbqrdQ1I2rVvuuS40tq3nRhrqLCs8hpdw h67crser0TYw2i+m6n9zfM1ODEZJaNHr1jW3HCKBs90IcaqvLSh5MkZIPpRZNpDkNDwjYBjeOxU lBV/da1Ya5NsHO4ljFJQYOil+AAu0ulBMX1R76MepV/tc= X-Gm-Gg: Acq92OEFyw/e85gOVD98/A58bDsmRamxmFh0uD+y0v0lhwvFo4FYJm0u+SQC4175eSD LnwzI07VcqHUuVjI3xDjBYhqyQmcDUeXLF+Pf/JnB6YQy+mK661Qj1/LMb3FyZGe72CSXge1bNR zyi7YkaduiSwcVLK7juHZvMCKWTkr9XMj0rB94aBTqd9HUv6n2/aKVPnmGZ9C8IOg00KujyOWPm wzCoTnROJfqW85k/7s+ZVVsiZSljVmSlGZrhNFGmfTZZIe/ko+2X6H4rK8hP/UO8aTcgcBIHx21 1oBoc95vIErTUPr77NoSupQ7mwC1nsZyiBGSQhaBeaVZQPLNGXA1+KZbngS2T6Xs8zGRoVzb/a4 bm7BpZCgUjfBYr8EBDTzLa8JZg7NQHvXM8299nMhUY+4= X-Received: by 2002:a05:600c:5c8:b0:488:f453:b976 with SMTP id 5b1f17b1804b1-490360a9e24mr1510095e9.27.1779315688748; Wed, 20 May 2026 15:21:28 -0700 (PDT) X-Received: by 2002:a05:600c:5c8:b0:488:f453:b976 with SMTP id 5b1f17b1804b1-490360a9e24mr1509455e9.27.1779315688284; Wed, 20 May 2026 15:21:28 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45da0fe0f72sm61409614f8f.25.2026.05.20.15.21.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:21:27 -0700 (PDT) Date: Wed, 20 May 2026 18:21:23 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 11/37] mm: page_alloc: move prep_compound_page before post_alloc_hook Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move prep_compound_page() before post_alloc_hook() in prep_new_page(). The next patch adds a folio_zero_user() call to post_alloc_hook(), which uses folio_nr_pages() to determine how many pages to zero. Without compound metadata set up first, folio_nr_pages() returns 1 for higher-order allocations, so only the first page would be zeroed. All other operations in post_alloc_hook() (arch_alloc_page, KASAN, debug, page owner, etc.) use raw page pointers with explicit order counts and are unaffected by this reordering. Also reorder compaction_alloc_noprof() for consistency. Compaction currently passes USER_ADDR_NONE so folio_zero_user() is not called there, but keeping the same ordering avoids a future tripping hazard. Reviewed-by: Gregory Price Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/compaction.c | 4 ++-- mm/page_alloc.c | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index 72684fe81e83..4336e433c99b 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -1849,10 +1849,10 @@ static struct folio *compaction_alloc_noprof(struct= folio *src, unsigned long da set_page_private(&freepage[size], start_order); } dst =3D (struct folio *)freepage; - post_alloc_hook(&dst->page, order, __GFP_MOVABLE, USER_ADDR_NONE); - set_page_refcounted(&dst->page); if (order) prep_compound_page(&dst->page, order); + post_alloc_hook(&dst->page, order, __GFP_MOVABLE, USER_ADDR_NONE); + set_page_refcounted(&dst->page); cc->nr_freepages -=3D 1 << order; cc->nr_migratepages -=3D 1 << order; return page_rmappable_folio(&dst->page); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index b725a3b59835..3108ac9061ce 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1877,11 +1877,11 @@ static void prep_new_page(struct page *page, unsign= ed int order, gfp_t gfp_flags unsigned int alloc_flags, unsigned long user_addr) { - post_alloc_hook(page, order, gfp_flags, user_addr); - if (order && (gfp_flags & __GFP_COMP)) prep_compound_page(page, order); =20 + post_alloc_hook(page, order, gfp_flags, user_addr); + /* * page is set pfmemalloc when ALLOC_NO_WATERMARKS was necessary to * allocate the page. The expectation is that the caller is taking --=20 MST From nobody Sun May 24 22:35:51 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D3D2F3BF66C for ; Wed, 20 May 2026 22:21:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315699; cv=none; b=Ga0kgK2F/b0EjEusvDlAC1IVNyHz/oNdt8qRfIGA3jIuE6KkalaWVyDfiUvDDWI7oi1CUuJ78pmXlnjYpFZ0R9yEpPE2SmKogKO8MTF/XQYGj+fJBX3nYGkmYn2/8HN2VFW5McpUlOPjyPTzg4LQywfi1PV9yuxiiK5tnBcvSQc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315699; c=relaxed/simple; bh=Y8nJwk5bnDwHUaiRuu38UiZrbZgOyq6/yOrVYrwIdY8=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=n28bWaFDAgoljsaXBwmMbeHsaJIJK/cwh8V7Vs3JMKsQo1Bw1OltoUpatEskIe/JtRZeMr44NF/vHLhFrVBJWpuxETWSkphaIvOcGgEXg0A0eGBXkIBoREK8GnnxltIRKakbuK++eYXBfLinB5qxcPxg+evbrl/szMikYX5qViY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=LsRKgfP6; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=AdSQbS2e; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="LsRKgfP6"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="AdSQbS2e" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315697; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Xr0Wq/tgum3c1wWvimcTvl9O1enHA5UssViDpRUdduM=; b=LsRKgfP65J1Gd4QwwTjG7pngG2208o8urC1jFdiLReF48/RZviSbNWVe/xVAQUokCPmUCP yjeQ9d6taqTtCXT4F7EHfvkIBAINo43k5qSLkfMZsZ94+9/xsya6DmKA2PcDhCt6zGNhTv VJYUJ7o41iXjMjhCek81oyi2btzyobY= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-104-1crSytsMPjSg7n1u4ib8tQ-1; Wed, 20 May 2026 18:21:35 -0400 X-MC-Unique: 1crSytsMPjSg7n1u4ib8tQ-1 X-Mimecast-MFC-AGG-ID: 1crSytsMPjSg7n1u4ib8tQ_1779315694 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-48feb8b9acfso31986405e9.3 for ; Wed, 20 May 2026 15:21:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315694; x=1779920494; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Xr0Wq/tgum3c1wWvimcTvl9O1enHA5UssViDpRUdduM=; b=AdSQbS2ebErSG0lgal0pInuneXoDDsmvOsakL90P89aHvb3rqRybRwwBFCoGzmvns5 G7eIZbJByYE6BiYUVC4At+eXoaFOKxfFwXga1xzKEUn98jT58RbOqdSG3kmHArO69zG2 OH0aJ3rg5Cx6qVUwEaynR49dElvkLuURfPd/fXa+Xz/pbmKfqQM6SF5at0ercghFtLt5 QObn4L0C+uU97hVh+UG3c2Emz6Y8uhjoO4nqWClIUjTIm4WWNIbrjNgZN/2NCAY16/XR 2MwgekSepPClnNXu5LacBA/1zQKUdHD1L14kMTkkq8sR/FZj0JVNl6Y7jZ5eokRuHylD pbUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315694; x=1779920494; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Xr0Wq/tgum3c1wWvimcTvl9O1enHA5UssViDpRUdduM=; b=PJTZ19Y1ToMyApmICNFxVxi3M6ywdlwzsvQ+3DmRqQyft0FJ/XTb4FeNX88mcCzvgY ARFFPqPiCBrmzCr9e7K+l+fUzhTjT34vfZBSmVqRixlIhbIqGcW1VBiYGVhOz7438pnW wCRROAIrElnclZ7mttLdjziLjip0g3cMBXBJ3ymsHgd69AgIM80jXpcsUTQ5cLZdfrq+ Y2WK+KrSIjBjo5TJm3aWyKjIxZ/mIcz9ISv3kVYio9ztq6LTC5c9ilhEdpZXP5Fk+W0Y 5RFdfLxO63deIXRmCZ338FbvjHHPt2AU1VourbcPwnim6l3JScltVAHIoN/nn11eKQeZ SyYA== X-Gm-Message-State: AOJu0YzENZGdsUwD93eRzp9j1lLKMywJ04qgDYEUANtABM7sLjvfbhKF acDA4OXB/D+WjF8JO73am2voS2NSTjscrlviQhjFw9N5RmO7NHiWNWMYSBsSsBZZ+FalL71ps17 WpxR1MAhtc8WtmbYnYrp5ggKmspyzpSFaQcYTVLzfGDmsAntUyu7fwkhhLOkz4MvRHqLO7Tv9U/ +xUe2ifJc3D5lEvhZwjnuDEpquQ97yEgxkMN0spjkMjfs= X-Gm-Gg: Acq92OFHjVJYg1+mtV73xJQ1eTzgY/QkGz9DVXtEuMfkrE9zUzm8R4qfFkfI30jB0GL lwXyyTZOuwMcixvM7ZsP9NQ7YgijH5aZr9iB7li5S0eorgxlfECir5SSnynM5eTK0U1Eb79MCHZ /fs7+yeCQfaEcFm+QUTg6WhdCxBCHBu0yb1NIZ83kfof7PKe/zWVjkXPFbCi9Ia/3f0LCxrKsQn KjnEn3WmKyKOa6SoWS8aKTf+4Dgq/atjPtWoTfiJygwoIPzxGBzhYp0gvVaUqOC1iftn+s7Cw/4 QzmGCO/kUAFdFxevHHDMPyqHVRtyxz2tIaLIHonzo17weLmWysXiJlEwfx9Y+hwqyxZHZ2MM5gk G7zYzzqHH2CVkMl298jq2+mV1yOZ797Vv6BdyGtee3Yk= X-Received: by 2002:a05:600c:1f89:b0:489:201c:dc46 with SMTP id 5b1f17b1804b1-49036068fb7mr2635625e9.12.1779315694180; Wed, 20 May 2026 15:21:34 -0700 (PDT) X-Received: by 2002:a05:600c:1f89:b0:489:201c:dc46 with SMTP id 5b1f17b1804b1-49036068fb7mr2634915e9.12.1779315693632; Wed, 20 May 2026 15:21:33 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49035c2159esm1982445e9.4.2026.05.20.15.21.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:21:33 -0700 (PDT) Date: Wed, 20 May 2026 18:21:28 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 12/37] mm: use folio_zero_user for user pages in post_alloc_hook Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When post_alloc_hook() needs to zero a page for an explicit __GFP_ZERO allocation for a user page (user_addr is set), use folio_zero_us= er() instead of kernel_init_pages(). This zeros near the faulting address last, keeping those cachelines hot for the impending user access. folio_zero_user() is only used for explicit __GFP_ZERO, not for init_on_alloc. On architectures with virtually-indexed caches (e.g., ARM), clear_user_highpage() performs per-line cache operations; using it for init_on_alloc would add overhead that kernel_init_pages() avoids (the page fault path flushes the cache at PTE installation time regardless). No functional change yet: current callers do not pass __GFP_ZERO for user pages (they zero at the callsite instead). Subsequent patches will convert them. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/page_alloc.c | 35 ++++++++++++++++++++++++++++++++--- 1 file changed, 32 insertions(+), 3 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 3108ac9061ce..ad0655387e9d 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1864,9 +1864,38 @@ inline void post_alloc_hook(struct page *page, unsig= ned int order, for (i =3D 0; i !=3D 1 << order; ++i) page_kasan_tag_reset(page + i); } - /* If memory is still not initialized, initialize it now. */ - if (init) - kernel_init_pages(page, 1 << order); + /* + * On architectures with cache aliasing, pages zeroed via the + * kernel direct map (e.g. init_on_free) must be re-zeroed + * through a user-congruent mapping. Host-zeroed pages + * (zeroed flag) don't need this: physical RAM is clean. + */ + if (!init && (gfp_flags & __GFP_ZERO) && + user_addr !=3D USER_ADDR_NONE && + user_alloc_needs_zeroing()) + init =3D true; + /* + * If memory is still not initialized, initialize it now. + * When __GFP_ZERO was explicitly requested and user_addr is set, + * use folio_zero_user() which zeros near the faulting address + * last, keeping those cachelines hot. For init_on_alloc, use + * kernel_init_pages() to avoid unnecessary cache flush overhead + * on architectures with virtually-indexed caches. + */ + if (init) { + if ((gfp_flags & __GFP_ZERO) && user_addr !=3D USER_ADDR_NONE) { + /* + * folio_zero_user relies on folio_nr_pages which + * requires __GFP_COMP for order > 0. All user folio + * allocations set __GFP_COMP via __folio_alloc. + * user_addr !=3D USER_ADDR_NONE implies sleepable + * context (user page fault). + */ + VM_WARN_ON_ONCE(order && !(gfp_flags & __GFP_COMP)); + folio_zero_user(page_folio(page), user_addr); + } else + kernel_init_pages(page, 1 << order); + } =20 set_page_owner(page, order, gfp_flags); page_table_check_alloc(page, order); --=20 MST From nobody Sun May 24 22:35:51 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F2FDE3C0611 for ; Wed, 20 May 2026 22:21:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315704; cv=none; b=G0Onwb7krH8qJud3/PodDA2+wQTKNlFU2loh29FRuuneybk8ZP0GkSastgN5+MtpTTdwHbspA/jQQ+ekjrISP+snyM4R7wpnERWg8LR3CQ9UotmELwS7MQR3P2z4kqPBi63y1BH8OXSlPKZgoWzkmIBxTx6SeN/2IXiCHyO1XEM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315704; c=relaxed/simple; bh=B4v7SD6SJ84KapLgBy+iXreANiiaF75HE81bu1RBt3E=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=dw6rG5I+sxcCIDnolKLbgh1pXZ4ZHNTipLv2RFcsj4Hwgrx75CfH7FtK966azrgB7PoBFR7lt50s/ND7qRQaO0w0HAeY0tX6Xh/RbC+WWX3dGBGqxRyvyvbIGDKW5lCm+/xv+OSDTCgvAMvKf6ku5KymW1fo2afDzAeZkEE3UbA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=ay/iaZkH; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=hGNcq9N7; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ay/iaZkH"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="hGNcq9N7" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315702; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=jpAQ9O1oMgUNlo5ukg9OEno1SM1k+70VK2/jH7/8vVM=; b=ay/iaZkHfOAYwlSlXLbBF4/axJd0NLUBQSbgNENDC0yVyMG2BGWWukB95/c0EP9diLCv9E 6RROMokQvP+Hplo/JqtAWqP824KLI892JU50kj6VI1BUzAxni+DVAhKoiTaSsGVZawbe3w wEIMwPF6caiGaatSRqhXILZ3LskVcVU= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-194-UbrhV5qJNam-Zl_inIlaLQ-1; Wed, 20 May 2026 18:21:40 -0400 X-MC-Unique: UbrhV5qJNam-Zl_inIlaLQ-1 X-Mimecast-MFC-AGG-ID: UbrhV5qJNam-Zl_inIlaLQ_1779315699 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-44d79da8cf7so4412927f8f.2 for ; Wed, 20 May 2026 15:21:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315699; x=1779920499; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=jpAQ9O1oMgUNlo5ukg9OEno1SM1k+70VK2/jH7/8vVM=; b=hGNcq9N7pUarL6XtaFvxL+cInCOU6M3RV5ST/YgjIi1NWLSQUIzk69wt++ygqUSnTr EKPyOFxSGKSZ/fBZ7J3kqX7oad7mocwcEppRWmms5J/EYWeQA1FRxMiu58WO97nuFyoC d+frtIuHngMbaC5d0Zo0IeCf75gwQCDuxUIN+SIRw6Nzjo45zZnjL2T0GbCmNFU5U6F9 N8hXOc9c8gEdyZqg6urSWGC0FiGm0hCPNypyU4Ebn8g0U4DvSW9dJYcJJpvhrTbjMk23 nPwAaco8SCYQIP7UTC5WvomvL3MpQPvA9QRmtbIu8gHELxxjdfLOwhaswhf9VcxHN8Fq k6Bg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315699; x=1779920499; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jpAQ9O1oMgUNlo5ukg9OEno1SM1k+70VK2/jH7/8vVM=; b=dSrPQO9vqSRYlMDY0Wa91Y8C+LCNaQKePkF2ePhHEEKiu6d5p14nQG5NMPxc1ivoh4 Bz90zqij3HmkXf8oXqy6VdSlA7Rv8JMSFMhPOi5j1gwJaY3jShhjfzS9bpINzWRivybE vfmFqJW7De0o8lLcqdNJIItqO/jj3igT+QkttUaZUjrQH5VEouVgcpeZXYcneIXcpb/9 it26JdzbDnT60TwLnKsuJUse1C9pc5JOemv54HQc2tOgp0jo8nbA2bJJYsznbhby38IM CsChGHXFezwyT1YcLzSQtNSrGth3lkUsi0rCD2Wd3IHhKwpfm6GzUGvwszSfA1xeI0hu mU5Q== X-Gm-Message-State: AOJu0YwC2zXkwsRwYszZyNEsJvoB8J8kBFKLdDgiab+s7mNb0kwsUeDZ 38ea9EFRN1L1K1RoLBF04ViW/L/n8rosaxtKEqsPT+s3zSk1azFvcmzIY4voL+S6HdRTrefZ0Bi M/LlC5hXpHrXR57ZhEkw6ufaoFFY7QC8OnkBlliNEqybEvlwNCy5WigBPULRUH1B9cQ9BTb+dS9 OXXOSHtSSDK54k9UudymjMfnME7rN7gONVvJr67LUZwoU= X-Gm-Gg: Acq92OGwgxkM7pH+Ns6rwLwyKQGdmddCsP8usHahmM1BRPoRgRrjo2a1FOG+uSNVhGk ffi8mWjawj4a/dw35HVfdrAvf3gLPPsfHTFjN6uEjsP440MyHAB65lUICzmGNWN3x6JbgxJQOIw Hpg/WZeHXrTZhqQYiYlSN5j1X2pMkUBMpCcG+C0l0gFK3wsmHT+yMEu3neBiexRxN5TPrtYwYw0 RGCKdWVwFdNdF/p+CiccE+XBvmxGok1AN9nkpU+rg37O5W4VwAKGYIdSNgPIcCk7wFX9GMcdx68 Bgz38a3G8STUV3RKvgQWARwXbSfVi44ziboPgGodnB4+rsZ7/71Juskp9BuctP546xV/v537fQ+ rRpNgC3UIB23i0suwphgsdORI5j1iONCzOs+lE6yKOWk= X-Received: by 2002:a05:6000:186f:b0:45d:b14b:23fb with SMTP id ffacd0b85a97d-45ea38bae4dmr340610f8f.11.1779315699283; Wed, 20 May 2026 15:21:39 -0700 (PDT) X-Received: by 2002:a05:6000:186f:b0:45d:b14b:23fb with SMTP id ffacd0b85a97d-45ea38bae4dmr340516f8f.11.1779315698648; Wed, 20 May 2026 15:21:38 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45d9e767d0bsm55493374f8f.3.2026.05.20.15.21.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:21:38 -0700 (PDT) Date: Wed, 20 May 2026 18:21:33 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 13/37] mm: use __GFP_ZERO in vma_alloc_zeroed_movable_folio Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that post_alloc_hook() handles cache-friendly user page zeroing via folio_zero_user(), convert vma_alloc_zeroed_movable_folio() to pass __GFP_ZERO instead of zeroing at the callsite. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/highmem.h | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/include/linux/highmem.h b/include/linux/highmem.h index af03db851a1d..ffa683f64f1d 100644 --- a/include/linux/highmem.h +++ b/include/linux/highmem.h @@ -320,13 +320,8 @@ static inline struct folio *vma_alloc_zeroed_movable_folio(struct vm_area_struct *vma, unsigned long vaddr) { - struct folio *folio; - - folio =3D vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, vma, vaddr); - if (folio && user_alloc_needs_zeroing()) - clear_user_highpage(&folio->page, vaddr); - - return folio; + return vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, + 0, vma, vaddr); } #endif =20 --=20 MST From nobody Sun May 24 22:35:51 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 74CB23BED4A for ; Wed, 20 May 2026 22:21:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315711; cv=none; b=KAQ8zzHSpC1V8OQIaiYsXudpBMwUTdPy+cgyfQnPYb2lrzfQHlQSlKiCVTW3CMmTwt8pXJywsqggJ1/xnN2qEnNFBfsFC0tquvmwzvLqHbnx2bV2F9rErWLuPgyngqOEZI0nfqki9f/GLUMLmgEgZ4t39KiLzQ9YxEq1VG2z6s4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315711; c=relaxed/simple; bh=+lLqKB25GJ8Nk/N/QA5/TP30kbyhPVpqAySiqciCa4w=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=hm8iYlrkFFzChp4vFPlmEVg8H2W7DlTTkwkvuFrahs6jIt9rj/aXYvoXG/X9J8pt4AeL6ey6I9LrfYgWiWjdFG+FZ/BaDUXH1+pmhH8eegwXaGWPcZQHELc2esUiwQRM2rvz9ZuEr/K3aYH+Lj/9eysQFtQZvzMQrdvs9jC7fBo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Bap42vw2; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=GIiPUiFe; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Bap42vw2"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="GIiPUiFe" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315709; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=yTMC7Vi3qz2eQrpu1XXxCoOuK0TKvwPfinMf53HH2Ps=; b=Bap42vw2MtpEJ4+5RwJUJZTWCTx3n9+bIhYdNYh/9G+wNwQNAiGwG216inxcBXm1U5ZTUZ +m08fc+j9KiOeJDhY5NMZksg+gYp75gma8ljBP181i09CYNVBbbbkkW3GnA4ab47KK7M03 WT1eOnA5UUAbrYOhIBAy7peZkgYDHeQ= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-213-fb4tmiewO9SuQJjSYxo4CQ-1; Wed, 20 May 2026 18:21:48 -0400 X-MC-Unique: fb4tmiewO9SuQJjSYxo4CQ-1 X-Mimecast-MFC-AGG-ID: fb4tmiewO9SuQJjSYxo4CQ_1779315707 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-48ff0eb77b5so51027585e9.0 for ; Wed, 20 May 2026 15:21:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315706; x=1779920506; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=yTMC7Vi3qz2eQrpu1XXxCoOuK0TKvwPfinMf53HH2Ps=; b=GIiPUiFeyXoYfQcSzCQ8UDLKNF8eIxCGoGWYZUAP2105svD2H6KOUyYeVB/Xr6sMva 3BowiY+z5O5SrdheNyzuqgpyYM/zshTqAAXW7g04eN0C8eL1Au+kUrpa9Xiuobj1UpoT 2awR7+9l/TNjwnv5QdShBeZjYM794QlHyumKE21++S52oDAQ9cjQy7617xplq8PJxteE 7s2fgPlDrVBD7v/BxhdCdTcuExdsWcu6iOogWguLL1dwSU3xPudt+5MlOGxt9Y/qUdO3 32BAWJ+t7w/zg5DBl7xa11mSmhIG7hfAXnJGG1z8Z2kxetk015oSEB71v4lLLG/SnqC/ mPPA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315706; x=1779920506; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yTMC7Vi3qz2eQrpu1XXxCoOuK0TKvwPfinMf53HH2Ps=; b=qQm/t2rzAzXGZihxrw4dZQtzFnk+V/ZvAm265nHNlbh1cL3qIRXa5pFSOzDrNKve3d 57BeyTohTpwgYiuQ2ILnvwnwQAjdpBKvw+7tSc741cvl7gclvIlDLthZMCf5QW1rKLSW VlZIRJmLSGF6vxR9tdWyYz8QiSPUKr9vakLi/a0CalvMef+68fdGj4AeHFpippNUeqla jgKMioe1DUHhv44SLf3KaskJuHxf5KQAG2vTBVqy4ujoa2ezG61+IQurMwFGtNBs6Cjp 6NtcuV6IERArd9RR2ebF+cJNL5HFLtNVM7Frft2Y12wn7XArDnCiKMvXk5L2GlCCRKlB g3gA== X-Gm-Message-State: AOJu0YwDeZuzgX6sk3reeSxmgjOntmCZefyR/0XEVoFjMCr32XGmit+Z hDCh9R/aweMRaavNtsE19yU9G5psW9x8OmhHRlOZvGxTSrR1BllzPNQYLtUv+3zMuRQFLgzkGXF FEVC3OaMQ8S0Lm8ifOZbD8qPNZSQbiyYgOLOetXxgj0la/08BOo0s33AEtaHndu/xBpp5Bya04L dKULhdZ3bx9P/5E3UPknl57/oGR7voHWAuKEjibMoGsw8= X-Gm-Gg: Acq92OHjaJp4Gp8X6sOYOVaKndn3L5qbTe2T+f6qbhu5/gEfWgZ1iO/eLQfhLPQre8l Hv3QtVwHV9zZevVChn0bVFxqASbjHJmrIpol1c+olPbqL1gM6bOPTteVHKI3TbV433k8nNcENnA SNkftCPfxZS1YhEIga7qVdaaqJprbrwF3DAWILE1UI85GWnkV6gGOJf8IhEzoyknv8YRyKWVrnR nseIEjdMAcaDeMTQ03BZk8eY1UUjL378d2yO7mYTx/xgD86+sFNCWzkSd75nT8ACbhghr4BZXex WtXwnRm5/TqgvPXKfQSA4cnJIbbrWHVdUWkHctP7Tf/+WP43MMqkDNdrUhLmKSBLjyvaTVAM0oi PfJQi4kHxDj9AVa8bW+lkLSilD7uc6z4Okof6yzasW0cX/sulyhoNQA== X-Received: by 2002:a05:600c:492f:b0:48f:dfe3:dae3 with SMTP id 5b1f17b1804b1-490360af200mr1567485e9.17.1779315706281; Wed, 20 May 2026 15:21:46 -0700 (PDT) X-Received: by 2002:a05:600c:492f:b0:48f:dfe3:dae3 with SMTP id 5b1f17b1804b1-490360af200mr1566725e9.17.1779315705516; Wed, 20 May 2026 15:21:45 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49033d8e24bsm20846035e9.10.2026.05.20.15.21.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:21:44 -0700 (PDT) Date: Wed, 20 May 2026 18:21:38 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli , Magnus Lindholm , Greg Ungerer , Geert Uytterhoeven , Richard Henderson , Matt Turner , Heiko Carstens , Vasily Gorbik , Alexander Gordeev , Christian Borntraeger , Sven Schnelle , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , linux-alpha@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-s390@vger.kernel.org Subject: [PATCH v8 14/37] mm: remove arch vma_alloc_zeroed_movable_folio overrides Message-ID: <9c2ac9d2fc159822b9fd6cb1304bf222324d70a3.1779315441.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that the generic vma_alloc_zeroed_movable_folio() uses __GFP_ZERO, the arch-specific macros on alpha, m68k, s390, and x86 that did the same thing are redundant. Remove them. arm64 is not affected: it has a real function override that handles MTE tag zeroing, not just __GFP_ZERO. Suggested-by: David Hildenbrand Acked-by: Magnus Lindholm Acked-by: Greg Ungerer Acked-by: Geert Uytterhoeven # m68k Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Reviewed-by: Gregory Price --- arch/alpha/include/asm/page.h | 3 --- arch/m68k/include/asm/page_no.h | 3 --- arch/s390/include/asm/page.h | 3 --- arch/x86/include/asm/page.h | 3 --- include/linux/highmem.h | 8 +++++--- 5 files changed, 5 insertions(+), 15 deletions(-) diff --git a/arch/alpha/include/asm/page.h b/arch/alpha/include/asm/page.h index 59d01f9b77f6..4327029cd660 100644 --- a/arch/alpha/include/asm/page.h +++ b/arch/alpha/include/asm/page.h @@ -12,9 +12,6 @@ =20 extern void clear_page(void *page); =20 -#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ - vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr) - extern void copy_page(void * _to, void * _from); #define copy_user_page(to, from, vaddr, pg) copy_page(to, from) =20 diff --git a/arch/m68k/include/asm/page_no.h b/arch/m68k/include/asm/page_n= o.h index d2532bc407ef..f511b763a235 100644 --- a/arch/m68k/include/asm/page_no.h +++ b/arch/m68k/include/asm/page_no.h @@ -12,9 +12,6 @@ extern unsigned long memory_end; =20 #define copy_user_page(to, from, vaddr, pg) copy_page(to, from) =20 -#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ - vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr) - #define __pa(vaddr) ((unsigned long)(vaddr)) #define __va(paddr) ((void *)((unsigned long)(paddr))) =20 diff --git a/arch/s390/include/asm/page.h b/arch/s390/include/asm/page.h index 56da819a79e6..e995d2a413f9 100644 --- a/arch/s390/include/asm/page.h +++ b/arch/s390/include/asm/page.h @@ -67,9 +67,6 @@ static inline void copy_page(void *to, void *from) =20 #define copy_user_page(to, from, vaddr, pg) copy_page(to, from) =20 -#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ - vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr) - #ifdef CONFIG_STRICT_MM_TYPECHECKS #define STRICT_MM_TYPECHECKS #endif diff --git a/arch/x86/include/asm/page.h b/arch/x86/include/asm/page.h index 416dc88e35c1..92fa975b46f3 100644 --- a/arch/x86/include/asm/page.h +++ b/arch/x86/include/asm/page.h @@ -28,9 +28,6 @@ static inline void copy_user_page(void *to, void *from, u= nsigned long vaddr, copy_page(to, from); } =20 -#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ - vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr) - #ifndef __pa #define __pa(x) __phys_addr((unsigned long)(x)) #endif diff --git a/include/linux/highmem.h b/include/linux/highmem.h index ffa683f64f1d..7b5955bf9336 100644 --- a/include/linux/highmem.h +++ b/include/linux/highmem.h @@ -303,7 +303,6 @@ static inline void clear_user_highpages(struct page *pa= ge, unsigned long vaddr, #endif } =20 -#ifndef vma_alloc_zeroed_movable_folio /** * vma_alloc_zeroed_movable_folio - Allocate a zeroed page for a VMA. * @vma: The VMA the page is to be allocated for. @@ -317,12 +316,15 @@ static inline void clear_user_highpages(struct page *= page, unsigned long vaddr, * we are out of memory. */ static inline -struct folio *vma_alloc_zeroed_movable_folio(struct vm_area_struct *vma, +struct folio *vma_alloc_zeroed_movable_folio_noprof(struct vm_area_struct = *vma, unsigned long vaddr) { - return vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, + return vma_alloc_folio_noprof(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr); } +#ifndef vma_alloc_zeroed_movable_folio +#define vma_alloc_zeroed_movable_folio(...) \ + alloc_hooks(vma_alloc_zeroed_movable_folio_noprof(__VA_ARGS__)) #endif =20 static inline void clear_highpage(struct page *page) --=20 MST From nobody Sun May 24 22:35:51 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AEBB13BED5C for ; Wed, 20 May 2026 22:21:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315716; cv=none; b=nHAO34CQh8p+QDph+W/1nHyoPqASozhfV3tSvyYL/gJX7SKdYQ/zDQM6wlIrE8XqRGaY4kC4v4SuWXf+7e+GPY+XXWas6SYFuLzuKqUna/6HF98wn1rMPzNbV8KjBT1lgKFcYH8JE8sG/Nsf2uKv+PYgCoKJCRqyDtXnvWnlTyc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315716; c=relaxed/simple; bh=iDiOdN3wLJj9wOdkZsdqwIAEdg+4DXjaS99pBUtFYbk=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=En1q590h93f6oZy/omwya026tLJ+/i5SF9G9uXmHBdbJkhbinY5JiKNlG56p3bs564XgfbCV2wKdeJVS6R5sqEWbKmc8iRhovlJwUJg8ckxzh60lNy9oX8mWF46CIIBX1cJ2BLp62Dvs5AKxcr/fXtA6BJmTh1XRZmuBZpPu4pQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Q6fZsleH; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=tEg7QU1p; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Q6fZsleH"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="tEg7QU1p" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315713; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=vK+J6BapUtjwGwdYlifbIYhLZtsZgN3kLB9idS28qTQ=; b=Q6fZsleHotC07LRvWrqVVfLgEAkwvSk8j+Yw7zxj5sq4nv3fu1wQFYkXblb3NlddLEK4OU ETFqPaHhG18W0tJGkQYdFcOhJ8fKj25r+4S1JMYuqTDDXKeDevdzFrxgBP9GiNtKOkOyWA nlkCiHWTC3xSu5CJKU+BrxAWKgbvFzA= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-86-rBRTxmLOP5O-NEOulQRJYw-1; Wed, 20 May 2026 18:21:52 -0400 X-MC-Unique: rBRTxmLOP5O-NEOulQRJYw-1 X-Mimecast-MFC-AGG-ID: rBRTxmLOP5O-NEOulQRJYw_1779315712 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-45e7c7289c3so3707966f8f.1 for ; Wed, 20 May 2026 15:21:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315711; x=1779920511; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=vK+J6BapUtjwGwdYlifbIYhLZtsZgN3kLB9idS28qTQ=; b=tEg7QU1pZ7bhuguGvdYkmO3pP+Axw+SZQWDGKe/7x8Tjaw1Xp2m8UZ7NoEy6t7Hk9E dI+Pnip052gd6XrsSrZJ8euTfYUweJ9DfC9weqIM9dDzqFM8ZVfTKwJBOq7y8svAFE9p gLpUKencRhuhwByRIznXTQaZUaE9MjLrq28d8vjhtg3srcrwj5n+Kr0bSa2frQFH4ai4 99NssOluQBkzYpT15yv+AlzwEtm4aoblfPxSl1NPXauVOJyAiWvqcpwd3yKlFB7RN+x3 0RsYg2ScfNnif2kUYLvsFmXF4G8FNnTxtMfydhgNd1P5NWwA9UoFersNIiB5JMI+q+2i GcTw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315711; x=1779920511; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vK+J6BapUtjwGwdYlifbIYhLZtsZgN3kLB9idS28qTQ=; b=smc8OjO/G04ZYlyHikedKFnsB339jfOJTTs/huWTUBnScPEgJdBTQ6jNcCZziVYP/S 9IPcD1GPzUgJxQCg71TG15/onF/enB56Qioew0J6PUsdUdpl3TUL4pdvgZ0bw7nZQLZg +M53Fmo5ei3XXDhfBSrEd/RTHnY64Xf7Rz5k5+KlsNT91KM+rXyi1IH/F8oO+mkcq6XL pmianhCquWijWygnh4LuOa/ukABserfXsh5yZ/qJ+wQTt+rUrl17gm2/SQDKwFhSe0tK 4/GbSYYpylvBj+TJAUL3erAYV1JoK37yOFCc3XiLZr7nwJlT4TisomH5odpau5WgFIPx yGjw== X-Gm-Message-State: AOJu0Yw1DrS80Pr1HLdZzg/gh6CVw4vC4bXR6f6H2cFVzZ5DLU8j8TEL zVnfNdPlQ+fGxSDC+JFj1vHT9NVMsg0o6FDMmL9jjNBcQUGR7E+Uanf+EKIGzUOKL8qjFopQxDf s7/Stmd3G9nRpLK8ZMml4wil7fCKdp0BSjMv3hec7EhUPuWXNPrG7mTdqBuQWO8+DTnLWm70zMM iLIPXy9QnP8lsjLzNhtLGcJmDs/EgtKeDUYT6m61FRZxQ= X-Gm-Gg: Acq92OEK7hwVzxoPLpjtLA2CjZDFVSlKJktmAUGP76du7+Rt+Qr4Qeq1z8xbrLF1IJl m+xkNiGGnNI1XnBqJ20lStnUFMw0LeDIHVVw1IK48dLG8Qd9TwpGn4/lL39NuKHAlJ1xCfPQzvX FhyHEcJAUjwWl6uVoJ/OYj4t29pO8/Oln3QocF4a3xzaahruDn4DI8xsiV7Rb45rJ0gJlPYBgUn 9fvbESM9nWu0/jeOOdiu6i+70Ns9b5Nd5QZM8uDeQFbO+M6J+f519PK2zKHdjkZaKn3L+MknjjL gMXVYFyR7YAxwFN8KRnUGEm11JWLWIrAss2zG7GTNjWOgrs8FaYE0v2gf0Aub/RakCFbHRpmSp7 mnIbzypCp+jV9ZSVdrrEV/b+WTlLYM41AunEjWQe7Tjl0AqoimNpdvQ== X-Received: by 2002:a05:6000:29db:b0:452:bc74:b129 with SMTP id ffacd0b85a97d-45ea34accaamr165835f8f.16.1779315711290; Wed, 20 May 2026 15:21:51 -0700 (PDT) X-Received: by 2002:a05:6000:29db:b0:452:bc74:b129 with SMTP id ffacd0b85a97d-45ea34accaamr165775f8f.16.1779315710727; Wed, 20 May 2026 15:21:50 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45da0fe13a7sm59869358f8f.29.2026.05.20.15.21.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:21:50 -0700 (PDT) Date: Wed, 20 May 2026 18:21:45 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 15/37] mm: alloc_anon_folio: pass raw fault address to vma_alloc_folio Message-ID: <467ed47f63a499c8d4c6b8209f6a818fb30229b0.1779315441.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Pass vmf->address directly instead of ALIGN_DOWN(vmf->address, ...). vma_alloc_folio_noprof now aligns internally for NUMA interleave, and post_alloc_hook will use the raw address for cache-friendly zeroing via folio_zero_user(). Signed-off-by: Michael S. Tsirkin Reviewed-by: Gregory Price Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/memory.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 02d3e53fc91b..75bee9501666 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5252,8 +5252,7 @@ static struct folio *alloc_anon_folio(struct vm_fault= *vmf) /* Try allocating the highest of the remaining orders. */ gfp =3D vma_thp_gfp_mask(vma); while (orders) { - addr =3D ALIGN_DOWN(vmf->address, PAGE_SIZE << order); - folio =3D vma_alloc_folio(gfp, order, vma, addr); + folio =3D vma_alloc_folio(gfp, order, vma, vmf->address); if (folio) { if (mem_cgroup_charge(folio, vma->vm_mm, gfp)) { count_mthp_stat(order, MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE); --=20 MST From nobody Sun May 24 22:35:51 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EE5043BFE33 for ; Wed, 20 May 2026 22:21:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315721; cv=none; b=alXd5MNw4rOra1f4meOuWOu4WTQZvsY9zfUGqY1rGKEV9f+Fr8i2Z5rStNhYpTvQ0sE7J/qExXM24hfXmS+I37IFvpWLnCQIyhFaPcXVYtyfb8pmJFr1Oqx+7hzabDdJ9zvYpcxUnuhqno2sozm2FF0Z9/do0zFG9XwjZ71Of/U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315721; c=relaxed/simple; bh=NCrpBEyb9BtjGhGhnTsHPNPFOIuuUXGjOyr7FM5hCbE=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=nFxcYeCpUaCRbBXjDZpr4lopKRsMZqJq/qT3lZDNFdGnPgZb5QSme7Di42dEBjsgyhL3ZxPlFVru/ql/NtE08OK1UGQoJQ8iZHAxqc/o0qzGKB+/dVG7TCkhYz6wI7Edc1ogRUa1eepVoK5ABWGGL4ajjeYdmxJygrV8CPPT/ys= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=ASsbQhI6; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=hTXwRs0m; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ASsbQhI6"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="hTXwRs0m" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315719; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=YBx6SgN438Kse3DYiPhfWEcJAMkqbJwsQPerWAK4QDs=; b=ASsbQhI66DkMqQZGWYpgm5d0etNxDM1KTnwrBg4UY0vmitEA7JebVh1YLQhlWG+dAdxQFg WFojMQFXGH0PlY+vD1qYK4EmjX+RGExuXpl8ZfMCjhd29CDEGyF4g4XUwhcmQqYip4s+Hq 6I8TbSIPAjYCfu35qO/ZMb8ku5bMlfg= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-342-78eu8uQ-NTKpbKZY4BgMhw-1; Wed, 20 May 2026 18:21:57 -0400 X-MC-Unique: 78eu8uQ-NTKpbKZY4BgMhw-1 X-Mimecast-MFC-AGG-ID: 78eu8uQ-NTKpbKZY4BgMhw_1779315716 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-43ff19e54beso3596810f8f.2 for ; Wed, 20 May 2026 15:21:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315716; x=1779920516; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=YBx6SgN438Kse3DYiPhfWEcJAMkqbJwsQPerWAK4QDs=; b=hTXwRs0mDK8NhLRrzagP4Y6MkM8+Clc2gf39LhFJ9TB5K9xMH/08BlgExukdxOK9Gx fRruGZCwR0Dfk/PMYfqhJOgZDGu0Vucx5ykqu5sezfy/aTgdd8aw1kNND4Q6O0d66+zT 9R4VoapHwibgcwCmycsH4JKqdripBa825v6q7AFQxcOh84sr1ywWeFwkgwpanmWONWUv iiwPb5bUdwjJxH69gfeK0u1VH9RmFQw1+UezI3Ypt6UkTXuChz1LxAn/ohfPRr53j3r3 7S94O80HfAm2BDFoUi4i3ftCB+FJgQxP7nS9WZ+AhVsse50JU18mZ4PmQoLAtgcKQ/Si TWrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315716; x=1779920516; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YBx6SgN438Kse3DYiPhfWEcJAMkqbJwsQPerWAK4QDs=; b=qy2tsf4wPxp/lVB6NGKzZkbe1HzxKiSG/JdDDOGDJapLwArAUpUbnyDgWDrDvitaRq EaoFKHO72hVgexJSDxrZXkcbMsbg3/0jhFmEJhgs0oLOYzdzJxZ5VVkT24waxyWztolh ydHyLiA1nSAx8WS9IjeygQiEW2Rghig2RBjssqskTVeyhgzJQIh8QdVZiL/reLWFDdZc axLWILcUDtaZFsHRtW96ILRCyQurkVQ99EhbBCQcrEC64vhIExtkuX99DMTnWB6fYaAs q1VpDa3rpym+2Zr434Kb0KmtKvsA/8iLCtXJ2LlJ2DfJYQD7NORiyDJOmXUHfnUwXu34 7Qxw== X-Gm-Message-State: AOJu0YyxCgIs5U/mRt88mmPwvFU+amtv0iMYNr6zmN+rZAdzYW3CQMcM fLxNQn/3ZHadnP+0V6FJQ7yMPHdUGHUqpxVdrQLwTHmGyNbSwico1WbxazBDk7aQGqnBUFvVbZU LPr8zS7/uRd6O02217slNipMPzd9wQgMJMu8QIR4pfIPSYjHxmzNgF7iTo1Ty8cdwMTWbfagVYW Vy1RWlCvNDuIcjSypDaFDC66WLLdhE6KYzB6YXaMyZEr8= X-Gm-Gg: Acq92OEfvytRfttJfCfYlj/T8EFIb+yxgGVTMo8Km+iaOL5+uZvJ0RPXsU7OwFPiQJi O4cjChDbGNkvbJOPkiewK0FdFgXg2+Ma2bbOZRVEYFAyETW6MXKXKrSwI7QMLsljDNfN22ckcWr MSfwIjRP3YqrGg6SwrvyaDP+vPqN+8w+LyjCoEbrRKb+tZbNVpxIZ8RC7ursSuxj6+JOuS7tM+o ZCrXW0Yx2fdSvf1Nv430R2VpfrfDC7lQN+fmuX7sdybso1iHr5zCO64/WcpNclB5Xk7ra/Ma7pR k18Fr3dVXka3eqo4jiZb/PuyIUn0nwMQohhDbrjU83ES+5MKIhwuwLLc4CzybgARvyodQ91USUH bIp/3F89KP6KxywpNHiX55jKqltbgo0A2zwlx88EKFRMzJD8ePB7O3w== X-Received: by 2002:a05:600c:468b:b0:488:aa33:dcbd with SMTP id 5b1f17b1804b1-490360c12cbmr2778975e9.26.1779315716258; Wed, 20 May 2026 15:21:56 -0700 (PDT) X-Received: by 2002:a05:600c:468b:b0:488:aa33:dcbd with SMTP id 5b1f17b1804b1-490360c12cbmr2778335e9.26.1779315715676; Wed, 20 May 2026 15:21:55 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49035ecc679sm2541065e9.6.2026.05.20.15.21.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:21:55 -0700 (PDT) Date: Wed, 20 May 2026 18:21:50 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 16/37] mm: alloc_swap_folio: pass raw fault address to vma_alloc_folio Message-ID: <00a4571e8ae75a457353726e1ec9111aebf76e31.1779315441.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Same change as the previous patch but for alloc_swap_folio: pass vmf->address directly instead of ALIGN_DOWN(vmf->address, ...). Signed-off-by: Michael S. Tsirkin Reviewed-by: Gregory Price Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/memory.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 75bee9501666..adc228fe3578 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4734,8 +4734,7 @@ static struct folio *alloc_swap_folio(struct vm_fault= *vmf) /* Try allocating the highest of the remaining orders. */ gfp =3D vma_thp_gfp_mask(vma); while (orders) { - addr =3D ALIGN_DOWN(vmf->address, PAGE_SIZE << order); - folio =3D vma_alloc_folio(gfp, order, vma, addr); + folio =3D vma_alloc_folio(gfp, order, vma, vmf->address); if (folio) { if (!mem_cgroup_swapin_charge_folio(folio, vma->vm_mm, gfp, entry)) --=20 MST From nobody Sun May 24 22:35:51 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2B70A3C1986 for ; Wed, 20 May 2026 22:22:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315726; cv=none; b=tKHNdkJdR3Gbxh7hDMw9BEuYf7/ysMCE5/J62uxEmBM2xMcxoWqFL4rBsLOttouW/BUWXWnW65/OQAteN0OnQPLvHjkrqXw2Epsr/3Md6CS6Ab7K+Hc7+krYUgKWSFhzWx56nr86J7RVd+VFU85d9nf1RmST7FQFpF98Dh74m/4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315726; c=relaxed/simple; bh=OWZLv3RcZNMXBK++VDzgdX5Eo6g4iZLM958nD4m3nTc=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=ezwJZvaaGSINK8TAvKwa5AXOiNxvRF/CUfzxa2iXD2J0/QWBNS9/Z6cCQUkV71XjZx+HEisbcuU6YESWFucmrkI2/KTdJlB3EzLhAHaPPnB6HKu5wawjUx0MYbJWVKpv/t6sZcvMhan0P9DMTLWKNmX4ZkEY1hVdn9N2LSdUm28= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=ISU1+I4k; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=Q7tJUscg; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ISU1+I4k"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="Q7tJUscg" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315724; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=JxMX/pzMUOtdSR/DFO8X8z+nr68xKXZK5hfRbiRHySU=; b=ISU1+I4ktR8e07pgIzWHS0Xb5knafZGInWbw8q+mvlvU/BTZOk8DTHjojZ2qFD2NMyvndE VqzFuGYUxX2IYlLY9St7c5Rs418eT2PE4HmH78MFVaCmkyUESvnQTrB6K4Z1csjgUdUSMi hUm6Oywi0rFkQuIGF//DA56xzgZ5qIk= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-644-n5C57hbjPfuVoCZ1VZ4sqQ-1; Wed, 20 May 2026 18:22:02 -0400 X-MC-Unique: n5C57hbjPfuVoCZ1VZ4sqQ-1 X-Mimecast-MFC-AGG-ID: n5C57hbjPfuVoCZ1VZ4sqQ_1779315722 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-45dbbbf8d57so7163286f8f.3 for ; Wed, 20 May 2026 15:22:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315721; x=1779920521; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=JxMX/pzMUOtdSR/DFO8X8z+nr68xKXZK5hfRbiRHySU=; b=Q7tJUscg0KuxTTWquEnim3nenh2eMw+IsVw3Klb03St2gZdB72VosPfZeZOAGFVdPZ Po6w3UYOHRtGvieoa1m6zgT8eUlSiGWzAsPTeFDcOie6stjGCyPTlQ4HGG0t9G5UqYoO x0AvFPe3KtuxhnW0Gi2aSxIFs4+Y0moGZ4Qul8KkBaesP+JpX+7km0HO/nOd76+aUEvZ xv9Lx9vK3NSR1N0ua2rsPoteV2yQgpQaFpHgBy/Xtbcn2AbyLipLdUM2wGBZq76fkCLb 4wYDQDd3XyUVyXQ6ftZ1kKeX9De6ECGif7UrwpocHMEFdrM9UtKit5l00vp+/w5ta/jQ yf4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315721; x=1779920521; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=JxMX/pzMUOtdSR/DFO8X8z+nr68xKXZK5hfRbiRHySU=; b=tK1J2N4yCowX0uLMJdQ+wLJhN2IrLtbmnQB9mTfM8sj2uVIOi4zDWwc7ZNe6BWOqGg KnHO8A8FOXjbh5ixGtQRCcCPzhGtVTBqM2rtGcbSkgdAsOByeVK2UXUXKGbkF93HZ1IP 28qYUwOUxLo6R4SRBGmRiyZr0vm3ftpgO8174ToxhyeEuff/1otVB2e9L0OVdcAQz4pG dcrS6aPDvtO1gIbQAv5o5Jwo2PEPB+/cZVkz8lwW30cb3NjOo2B2mfKsvA8TVy/TEaFS gwsGjHcs8gqotxgRosHaU/TQ2cwgpQQ9sqb8LmvhTzk7+qIqzarUvpay5DLsNWB/Y6qA 1u/g== X-Gm-Message-State: AOJu0YwwNxzoaX1mC5i0OhVQqXqy3frRyo/ACnNVzIZBcpXcIfoUvv/8 m4IfT9DR80wLgd4TZc2neWRqZa8JC69qvDN4PK55onk5lXZ4yuUAPX7knHEOPixYX6xxXI2h21O SCAnDz5g0Y1k1N07iUR8PN6SD+VpEaN/FFF27Boz1iKwgC4LVj+PmooIq5UROiyZGj1D3dQfwNI t+o5rf1pCl6Z/S/MYCv/tdMTG5pYsZBJhx/rjMpnO46ks= X-Gm-Gg: Acq92OEW/l36GEnmi/nCq91ZDSkXLp2t3pBEM8qmW7GzNdg6r/QuLodqdIcCA56q6+r fCWQ0C/n3nDAWlP8acYpVQlyn6QWFQkdzK7fQhk866BjrsuUIFUYNJwi33v7bW6Cya+qKx+XGw1 oIRhb/uhH/UES8SqEJrnvG/K3DCfbMhrN0koRDBBDV+E+QaW+uuXGkga8nE6QLYKqkKQi0F4q7l RJqUZiETCeQNgQ4zfnjr5+O20nNpZU8+/PFqx3+gqgikB9h+4CIdrBdBI+55r5uwSKtbOryuwSq RJA5Ejm7oLkHq88N03qJv/N3NRjCexvv76hsnoyrdmH0nFSaNjZEHs2wzUamYKF3bZ3lRZIKOa0 79JjmaG7eZKoYxXbqGXXtN1uWtggl2Hl8eg6klcIoBdI= X-Received: by 2002:a05:6000:2f88:b0:45e:878b:6c36 with SMTP id ffacd0b85a97d-45ea3ce5a81mr251968f8f.35.1779315721384; Wed, 20 May 2026 15:22:01 -0700 (PDT) X-Received: by 2002:a05:6000:2f88:b0:45e:878b:6c36 with SMTP id ffacd0b85a97d-45ea3ce5a81mr251868f8f.35.1779315720651; Wed, 20 May 2026 15:22:00 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45da0a178adsm60809630f8f.18.2026.05.20.15.21.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:22:00 -0700 (PDT) Date: Wed, 20 May 2026 18:21:55 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 17/37] mm: use __GFP_ZERO in alloc_anon_folio Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Convert alloc_anon_folio() to pass __GFP_ZERO instead of zeroing at the callsite. post_alloc_hook uses the fault address passed through vma_alloc_folio for cache-friendly zeroing. Note: with __GFP_ZERO, the folio is zeroed before mem_cgroup_charge(). If the charge fails, the zeroing work is wasted. Previously zeroing was done after a successful charge. This is inherent to moving zeroing into the allocator. Signed-off-by: Michael S. Tsirkin Reviewed-by: Gregory Price Assisted-by: Claude:claude-opus-4-6 --- mm/memory.c | 11 +---------- 1 file changed, 1 insertion(+), 10 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index adc228fe3578..895ac72121fe 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5249,7 +5249,7 @@ static struct folio *alloc_anon_folio(struct vm_fault= *vmf) goto fallback; =20 /* Try allocating the highest of the remaining orders. */ - gfp =3D vma_thp_gfp_mask(vma); + gfp =3D vma_thp_gfp_mask(vma) | __GFP_ZERO; while (orders) { folio =3D vma_alloc_folio(gfp, order, vma, vmf->address); if (folio) { @@ -5259,15 +5259,6 @@ static struct folio *alloc_anon_folio(struct vm_faul= t *vmf) goto next; } folio_throttle_swaprate(folio, gfp); - /* - * When a folio is not zeroed during allocation - * (__GFP_ZERO not used) or user folios require special - * handling, folio_zero_user() is used to make sure - * that the page corresponding to the faulting address - * will be hot in the cache after zeroing. - */ - if (user_alloc_needs_zeroing()) - folio_zero_user(folio, vmf->address); return folio; } next: --=20 MST From nobody Sun May 24 22:35:51 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 39B7C379988 for ; Wed, 20 May 2026 22:22:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315732; cv=none; b=TDfU8yOGFg6+zVEO1WAr3B6RGGNFlgskV2Ay6S7bjckTICP1e+WwfVWz9BXKavXvYW6ICmhLORd7A4bBm16dvEZI/7DkgEWX3tnk5iBn4Rc/ko/Xg3zBDWPR7RTXsz6NfLjiDOct3Vo1pRkI/rlMtEbcsUgMs7qkpSAvAqoJ7dw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315732; c=relaxed/simple; bh=3uGUVJNy6kGsME3V8gAHY8oH+alSOzgrnDm2DGDTDks=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=aQpiXVkgB2AMiCQvQ7UmDROIRR0XBp4LcEtWyOo3vbOY+grbsA7jKdOaXouNqbClFkeb70vC3DFHP8KmKJ7m0iJJcH/R3c3MH0a1S4JQqd3+N/3MId4L9pjtgV70O78NV/hI3pKitz781ItVvcyHhwdOv8G+v+oofPdyfiqB+SY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=P9XdPD1M; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=HPRpjzxH; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="P9XdPD1M"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="HPRpjzxH" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315730; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=yH/Aq3jRD/J0BW4aKC2BcM/dVQFsNlzLQi2tKUktXO0=; b=P9XdPD1MYma22hfdj3JqzGrZhm5fSINAlUEfd+u0rYhUoCVkZjZrW6qRmEaiclkCCXH1ZL ferDOrLO/NKnOtehiwDLAwpXcg08e9JhLFh+4hQba/U/AnbglTVvQWi081xBcj2hOkxZMs Xy/3nCuryeDFkfAsaWD4UI+ptY/61Bw= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-563-8cmvjo5nMlycNpyTefITeA-1; Wed, 20 May 2026 18:22:07 -0400 X-MC-Unique: 8cmvjo5nMlycNpyTefITeA-1 X-Mimecast-MFC-AGG-ID: 8cmvjo5nMlycNpyTefITeA_1779315727 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-48feb029961so30574885e9.3 for ; Wed, 20 May 2026 15:22:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315726; x=1779920526; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=yH/Aq3jRD/J0BW4aKC2BcM/dVQFsNlzLQi2tKUktXO0=; b=HPRpjzxHTmol8CYYrlrrMtmIrf4jJN1d/vz7sCi+Xv6CfJVpUZfZCpI/9+KSRIrMGM EFhaQiWmHYcVq78ZKqw63w6NPYLDmDEVZk74o8R+7apGLV+h+pvK4TyySQQ97YR0beDK WdX4EgLzTvrob/Wup+EJ1s3rPDKBpnCvEA7+KOMFo3bC2Rhn7lEY17jSpjGzTHwBDREk hLt53oKBRL2RzsyQac2UF8WOIieLvkEAwIE3mnUiO26VK5sR2VakMbtsCdFeWSBLw33E FSY+iA4nAceeOAK7H3cJFw9QcvFM7YmD/UcbWEvlLTc7mvuFxbOgP7I0TODv9WG2sYxi 5z1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315726; x=1779920526; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yH/Aq3jRD/J0BW4aKC2BcM/dVQFsNlzLQi2tKUktXO0=; b=oBa6jchfBMCUj5UfZymQ+U++SBYq+7m0o3wZ1/z3k8F5wGwV77S+Sf5IJ9+wrav/Xr Qskh98ig1Uq5x3SYXebzNlPcYY6irvSr4JrDIG3hLaVgpFwgSu4+ogm/gy+aswNltwKb 1Rk0byfDimZ0p1zoMKEGVy2393ldIuc/g6Zcxo2iu0UA15ZTi9p160GnrVc7lC7gdBVP gtj5N9XddTnpspp1Dsi4fLGR9LgU7CDfQWWtwIjleVVqPRndCEAXx1LN6138dq7u7lFL W62TCvY60FViEkKpUE4UTgdmrMwCcG9DdHG2Vgjo6aXHqDC7WuNjJEbvuDWAvCGKJj2N nQ6w== X-Gm-Message-State: AOJu0YwtwD41evwEs9s6lM47cSQKli8dEl+otJoyit7qTqPByOkafnKy nXcKoZ1CgUIGh1NL7qdxAFEivYtPOZLdvyZnobBAe22ztcwNBzFbm1A073dTmIVXUiaG4hsx5fh oaRYKdhfxXqEXlgfbrM8f5WNEhZYQIGYlT05oBX9XxvmIuBImE39fm1dmtUTYl+G+FwkbG2U5A6 RrHR4FUcD6yDiwjKqrdLY/+rtyDYJgpK2/Vfyvx+rYB6M= X-Gm-Gg: Acq92OH2ZzIskbpTaNvO5nRCWC6lk2aj+1c5kjn6OpfVOuKGFYatZe4xxeOBfn1aFvw q6GA7Y7hvFjNq7/aQuX9ZgHKRFdvHXy711JdSBKuk75So6pyltYnX5x2feISNCSDSxUE3xmWYtT D75HgFo8WlccRCpXZS8gfiOZMk6EYztGH7LlnqJh3ziQry7fMeP/SDAltHtI/+DMJzz1m9LE83h N+72RNbKoYfLlygu/pfVU0tUqq196Z6T1UeKaaGPApv3Q5/HX8mCRXy0MeRXF2OOXUJMwMIA0LA EQT0yfbBHKxfGArZtAi2uAEJcA/FX3pgp8h23DZIcXC2WF248SM9SiSM3QXT9YVqzAVrFn+Dr1Q UmiSkX4S3ZxvNUGEYh//PlLmwm6kycVTB0hHQeHbIm9M= X-Received: by 2002:a05:600c:468b:b0:488:a824:fdff with SMTP id 5b1f17b1804b1-490360c53c8mr2627905e9.22.1779315726618; Wed, 20 May 2026 15:22:06 -0700 (PDT) X-Received: by 2002:a05:600c:468b:b0:488:a824:fdff with SMTP id 5b1f17b1804b1-490360c53c8mr2627125e9.22.1779315726026; Wed, 20 May 2026 15:22:06 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49035edd7cfsm1709205e9.27.2026.05.20.15.22.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:22:05 -0700 (PDT) Date: Wed, 20 May 2026 18:22:00 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 18/37] mm: vma_alloc_anon_folio_pmd: pass raw fault address to vma_alloc_folio Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that vma_alloc_folio aligns the address internally, drop the redundant HPAGE_PMD_MASK alignment at the callsite. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Reviewed-by: Gregory Price --- mm/huge_memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 970e077019b7..d689e6491ddb 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1337,7 +1337,7 @@ static struct folio *vma_alloc_anon_folio_pmd(struct = vm_area_struct *vma, const int order =3D HPAGE_PMD_ORDER; struct folio *folio; =20 - folio =3D vma_alloc_folio(gfp, order, vma, addr & HPAGE_PMD_MASK); + folio =3D vma_alloc_folio(gfp, order, vma, addr); =20 if (unlikely(!folio)) { count_vm_event(THP_FAULT_FALLBACK); --=20 MST From nobody Sun May 24 22:35:51 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 40FAF3C3459 for ; Wed, 20 May 2026 22:22:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315736; cv=none; b=lms4t6Rmj1+U0NXWK9bTUvW+rXKOFvdrBTH87LitqQypA4nu4osBtJrRCmLyoHR+Eu0JbLIdn1bgrGOIJQd6fZUEK0gmSeHzfmeYr7ImyrjhOXmJTlJXodYrWSkvSyYdLhRtJd/S9URy7oeYPPQZ7+/Mt/Hh3Nf6l6YHc5I3IyA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315736; c=relaxed/simple; bh=+9Q5fGlUrZYlSCWgLoOvdSgXnGUe3WMtEgfY+DGuKBc=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=c5tl1VgwXn0qf5O/E4eGXi0rr7yMi1y2MGpDt98MVbFOSQYNSWPnVc4MVLAOAnc8P5dcqN0AL9eV0N0GF6nf/C3efNpMACbs1YMbMn5BgISPxwHS1sjp75kzZ85WAeVKgiyOC65Fx4bQj6rJJT1LoMTusM04n4YhUemE/J9baf4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=gcZ2T4AC; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=OrSjq0xZ; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="gcZ2T4AC"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="OrSjq0xZ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315734; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=dwGlO3yG5EtH2T+AQMoYdXPNyVEDsUWFS+ozJXrj1vA=; b=gcZ2T4ACk2BL8KCO2q7IvGLy6+4O/uAfZ8JANfTpayir5/H5Op7cbFrkrge8LlPSsSpyg5 LEwHsaFOILu5ow8XcxMEuZrdF7XCLdn5+pEQaIa8LCuZgZ2lbnca1qZzCd+gk/xFYA9VQS 6HpasHKHN1Krn2jX010QkGUK7G7UPgE= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-653-UgdR6rqaMkKHs9j_McQXVQ-1; Wed, 20 May 2026 18:22:13 -0400 X-MC-Unique: UgdR6rqaMkKHs9j_McQXVQ-1 X-Mimecast-MFC-AGG-ID: UgdR6rqaMkKHs9j_McQXVQ_1779315732 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-43fe791a398so4315460f8f.0 for ; Wed, 20 May 2026 15:22:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315732; x=1779920532; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=dwGlO3yG5EtH2T+AQMoYdXPNyVEDsUWFS+ozJXrj1vA=; b=OrSjq0xZQMAEFH9hL1NFCMAWOBNd0OTRZ0M7+/cdSG0GJAewNkjyBTajcSaYFpZZX4 CLnj08zKDkqNq2hSGY0xxA/qQcwJErfGVY5HIZr/7ZRSgAybemoMA6Hnsn2sN4q84c/3 ZSBmnp8rPaDKWBPEB74oV5a1CeHsRAZs+GPfWaK7HKxIdZ8vZdmAlJ7Wt5tgat6lMbu0 I2GqYLVk+cLFyN2iB2mtlTOrWpoVYAN8okTD/GRRlFjJRfCSgyyYANlmWD5LfR/rZPnA llxJ795KmcHGGuQ1yPNbiwIpA613l0JU2q4dzjA5qAIIk9vhz0gMFVK0QvOf4XIuuUeW EDkA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315732; x=1779920532; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dwGlO3yG5EtH2T+AQMoYdXPNyVEDsUWFS+ozJXrj1vA=; b=U0MQwMrnEppJBZIQkanKP45BW58FxA8h7G8WsFatJu7ww4UWGam1sBXKfxoMg10dnI ui3dzPuuWTOh/4wCDa634sWXz9Ew5e+S5G1wmlB3Fjnr+tszyrlNI8oAQ+cQUZBwe1uW xRNOGG/+AaK326zRhz+eJK7YamoqbXPMR6d6u2BuMRLu5KgoIaeLdMJMT3wumapDfyK7 /LKge9DvUmD4KuMhOCP419zwS2RoseLXm23gpOvRWbPIOUYc8xvZA4JETfytYL+U3GV9 z/mpapher+ncykHFPAebI3/80IftzuRtZR0W1Qk8daHSrP32bf/yiNi8qovW/GKpSc7j 02Ow== X-Gm-Message-State: AOJu0Yztb/AvrAifpw0vEon+Pkyp+iH+iN2BRkaiAA8SPnaa0cOkREG8 AviT506cz+CQx/mSQxQq98yTjAiIfCmxoQCGnkg3+OXW1Q/AcDzEe02qydwR6nKgFtvxyVnxqY4 /es3QAbwHfjKgSq7CuSD2biSwS8tW2Jy2AVl1HGP5JTS/Mah+Dca5668IAUOJi2lKWbTDlM0dEs mid0o2/7yZnlPArYLq7l0+WAUvoeIlU64RWZj9nqCOCIE= X-Gm-Gg: Acq92OFDIULpOZjkputByOQSGdvaxDKitq+eIQ6cP3axu2cOp+IyRmO4Uf9ZXAAgwAS kAgWLzALo4Y5HDt8PRLYPqBqqJ1QKr+XpfNAYmlyiFGQYjMjmDAnANfV9qrAlhM6PZgkK9Bozv4 KQQgJqce8zsTnNIrkqGdbbt6gZgL9rPOU+dlEJFmjpZkXYlr0HtVjvr20iKnip+qGxHbUP+ey6Y 4wdllpZn/r1n+UNbY4hYISoAv6UysfSUMIuU8mhuMb4YKY1RnypSaqgyupJOKe6afszskOxAtyM uRJ3nakBAFjuO5TK3II//6uqApTam/CCcLatrkjPmaIk76ra2mOWa/w0Kb/dEAGYuMJ05/x7iUY RlAXzRmoKSgoNEtp5QQLXpB1MrmoL+Lagl9HegQTUSuc= X-Received: by 2002:a05:6000:402b:b0:45e:73b3:8118 with SMTP id ffacd0b85a97d-45ea3bee76fmr278682f8f.29.1779315731808; Wed, 20 May 2026 15:22:11 -0700 (PDT) X-Received: by 2002:a05:6000:402b:b0:45e:73b3:8118 with SMTP id ffacd0b85a97d-45ea3bee76fmr278599f8f.29.1779315731226; Wed, 20 May 2026 15:22:11 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45da0a19b1dsm56913999f8f.17.2026.05.20.15.22.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:22:10 -0700 (PDT) Date: Wed, 20 May 2026 18:22:06 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 19/37] mm: use __GFP_ZERO in vma_alloc_anon_folio_pmd Message-ID: <0ec029361949c492b10accc8cd787aa0d2fef0df.1779315441.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Convert vma_alloc_anon_folio_pmd() to pass __GFP_ZERO instead of zeroing at the callsite. post_alloc_hook uses the fault address passed through vma_alloc_folio for cache-friendly zeroing. Note: with __GFP_ZERO, the folio is zeroed before mem_cgroup_charge(). If the charge fails, the zeroing work is wasted. Previously zeroing was done after a successful charge. This is inherent to moving zeroing into the allocator. Signed-off-by: Michael S. Tsirkin Reviewed-by: Gregory Price Assisted-by: Claude:claude-opus-4-6 --- mm/huge_memory.c | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index d689e6491ddb..9845c920c29c 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1333,7 +1333,7 @@ EXPORT_SYMBOL_GPL(thp_get_unmapped_area); static struct folio *vma_alloc_anon_folio_pmd(struct vm_area_struct *vma, unsigned long addr) { - gfp_t gfp =3D vma_thp_gfp_mask(vma); + gfp_t gfp =3D vma_thp_gfp_mask(vma) | __GFP_ZERO; const int order =3D HPAGE_PMD_ORDER; struct folio *folio; =20 @@ -1356,14 +1356,6 @@ static struct folio *vma_alloc_anon_folio_pmd(struct= vm_area_struct *vma, } folio_throttle_swaprate(folio, gfp); =20 - /* - * When a folio is not zeroed during allocation (__GFP_ZERO not used) - * or user folios require special handling, folio_zero_user() is used to - * make sure that the page corresponding to the faulting address will be - * hot in the cache after zeroing. - */ - if (user_alloc_needs_zeroing()) - folio_zero_user(folio, addr); /* * The memory barrier inside __folio_mark_uptodate makes sure that * folio_zero_user writes become visible before the set_pmd_at() --=20 MST From nobody Sun May 24 22:35:51 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7C0643C0603 for ; Wed, 20 May 2026 22:22:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315743; cv=none; b=Vz/mZPXsZsBY/LtpMOFZk+2RudEyM0xrb5JKsUDro4O37gFJluzV2MORb4AShZAQ0ExKMs8RQ124lku+XdEMD63Hjj1a0gOKXNU6/L2hEbWp4YpdKZIX/BqqxFaEe6Ct7EC/kSee54wm4t2lrgIqTLQ/+nmptP42F2HMww8jBvs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315743; c=relaxed/simple; bh=MOLGrRKYtAzXHsjQPObl4upRAuBjzhEEowJkGu/kpHs=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Tm1gpV6cBP4Ok0Z0KFyFTSYNoO2z4Fe2cUi/oB4MzKQZZyOY3iaElvkCClMQYBc2fPClwxU7xJlQS6MQd4Aw/QiLyn7PdqV2e86zXMXGw1oBfB3Z3eXRtHoDopECdaUVGxzcEZRwMZ3K19OX40ANS44gSUaihGYGVR9f6w6YAkY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=gqCgxNZ4; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=VzErDH7g; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="gqCgxNZ4"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="VzErDH7g" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315739; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=YFnBVDBDzVeV9p9dYXmJ4JPUR3YMu/6T3H2GlWvoQjs=; b=gqCgxNZ4pPWItivvm6csFx3m0NiRc5w6Djv2sk6b3VrB0bdOSEKJs3aPnN79Hy1m+Y3dge IAYsxcRBQUPa8An7Ndbg+ZCaiHTl31H6o2cNiLAtOsosv05wnG5DsOf3w0e7xV3WZHxnWO oSYzx0+u5zcEEyLffByaVphyaQ6eyFY= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-320-YN04S1ntOkC0kxx9Ghpzew-1; Wed, 20 May 2026 18:22:18 -0400 X-MC-Unique: YN04S1ntOkC0kxx9Ghpzew-1 X-Mimecast-MFC-AGG-ID: YN04S1ntOkC0kxx9Ghpzew_1779315737 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-48eb0da933fso39142475e9.0 for ; Wed, 20 May 2026 15:22:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315737; x=1779920537; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=YFnBVDBDzVeV9p9dYXmJ4JPUR3YMu/6T3H2GlWvoQjs=; b=VzErDH7g5ZSYBFhWlnhHgIyJ2xO1bIsFGr059bud8qxdzC4JsE82qzRGyXL7kjq855 x/Sho+QJCRcfC4eBkwzb2VTlg6MNiJ0IwdEvP9r9+xoJckj3llP9SYtOATLGS4m9Td+K LVH7kqzUs7t8L3SxHLP2AiBD5YKhFvBrpV47WWxekXc7v1QXykP+wiePF7ZbKh9butVA KjbS1nhC8MmwLtE2+0AIdGrIH0Tv4Q63Mg0EhjPGRkyRcyegFaJxND67t5CcpRF5nyZU hzA2MwVL0MKdCGV7couTZyNVx92M7TZM+vEYegRofcAl5yiIJbZ3PkFPJvfI2N5XsYBL wlGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315737; x=1779920537; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=YFnBVDBDzVeV9p9dYXmJ4JPUR3YMu/6T3H2GlWvoQjs=; b=eDwY6zAoUzUKOFQFphj2MFI9cFmV64gZPptEdGTh0qvy2QpFeHm8DtM/taT9klkZoI wul01PEhXwZwIFfqvReRNM2VGIMwoIfn7lgNwoWzc3luAdpqNjjslIco3lBVb/N2glMY 5stimOJqFhKYTVTXqWKn5ADqCm+ez2YPh6pwIqWE72iw8GlCfUEXooSN+rmlXtT2vGYD r3t/a9bZfRtrJagxYf9TYLWX5VJtKxM3IyD29+6shSHj+vw1PSUuG85o7NiiA3EPGcIT bIWV2YJW1NgKlaFf279BGcTXGhVjJMtiMK3PYf6QLNDkHqlmqsD0zKV/8aaWOCykChTi KSAA== X-Gm-Message-State: AOJu0YxA9U2Q4riREgzIxdca4/gArYnDIhY9PK5azB/iVq7O8s+IKKDO q/TLd23f0/t9PxxxRi+yWtxQtFOnt4/sKqa06/42IQmjVvEjmZSELj/n5Bye9/TWbMeWu7jkPRc H6Xhh9swtCrLmhcrD3pAj0FFtsf0LXGw4wQOjCqhpVLV6lDXGrv2cuKohctqDk8IiwW090JV/Fz gxZkUNyc/aqmmuXTv417iIWNKYFXfPECAXhN3q8nTTo84= X-Gm-Gg: Acq92OFxdfObpOiSjajV/A+ViDF9ZflXV7UWvkLeu4vDPKgwzBEm1dj6vjNGTjWP7q2 8Zd/E8M+3LuYCBW7AAxz39q9EcA7Me31x0OBGePG6EnjV9BDN7NsXVQUD2b6yBCI0GpRIG8tmuH /VQCLuSQa6WHQXXenm245sf1S+IHoXfGkbBNhMsVXC5SQNH6JcQkku01l8JBD45KVQ0sjUfhXMH ilDiSBNo/ygD/M2qkNZk8DFiN43Lqcs/FDad0tZKBkwOpGy7B82BSlMZ+tbkApKmC0QSASeq1oq rEPZspliOOyMjJRqaxcovEWGlvC251oFCfZTuA5hQv4PZ6tnb4u63TeixRa7MN2t9i025tYO4j+ NKNFYecEB7spUxIIc8+JR8cY6ZE28ZbR31WcRoUPIKNDxM/d4T3KtTQ== X-Received: by 2002:a05:600c:32af:b0:488:9ed3:1492 with SMTP id 5b1f17b1804b1-4903605e3damr1812045e9.10.1779315736899; Wed, 20 May 2026 15:22:16 -0700 (PDT) X-Received: by 2002:a05:600c:32af:b0:488:9ed3:1492 with SMTP id 5b1f17b1804b1-4903605e3damr1811475e9.10.1779315736278; Wed, 20 May 2026 15:22:16 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49033d2fd6bsm20770385e9.1.2026.05.20.15.22.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:22:15 -0700 (PDT) Date: Wed, 20 May 2026 18:22:11 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 20/37] mm: hugetlb: add gfp parameter and skip zeroing for zeroed pages Message-ID: <4eb3feb0a01adc0f2f68e0d0ebf48a43e28c00f1.1779315441.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a gfp_t parameter to alloc_hugetlb_folio(). When __GFP_ZERO is set, the function guarantees the returned folio is zeroed: - Fresh allocations (buddy or gigantic): zeroed by post_alloc_hook via __GFP_ZERO, HPG_zeroed set by alloc_surplus_hugetlb_folio. - Pool pages with HPG_zeroed set: already zeroed, skip. - Pool pages without HPG_zeroed: zeroed via folio_zero_user(). The address parameter is renamed to user_addr; the function aligns it internally for reservation and NUMA policy lookups. For pages that need zeroing, user_addr is passed to folio_zero_user() for cache-friendly zeroing near the faulting subpage. All callers pass a page-aligned address; the hugetlb_no_page caller passes vmf->real_address & PAGE_MASK for consistency. HPG_zeroed (stored in hugetlb folio->private bits) tracks known-zero pool pages. It is set when alloc_surplus_hugetlb_folio allocates with __GFP_ZERO, and cleared in free_huge_folio when the page returns to the pool after userspace use. Suggested-by: Gregory Price Reviewed-by: Gregory Price Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- fs/hugetlbfs/inode.c | 3 +-- include/linux/hugetlb.h | 5 ++++- mm/hugetlb.c | 42 +++++++++++++++++++++++++++++------------ 3 files changed, 35 insertions(+), 15 deletions(-) diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c index 8b05bec08e04..5856a3530c7b 100644 --- a/fs/hugetlbfs/inode.c +++ b/fs/hugetlbfs/inode.c @@ -810,13 +810,12 @@ static long hugetlbfs_fallocate(struct file *file, in= t mode, loff_t offset, * folios in these areas, we need to consume the reserves * to keep reservation accounting consistent. */ - folio =3D alloc_hugetlb_folio(&pseudo_vma, addr, false); + folio =3D alloc_hugetlb_folio(&pseudo_vma, addr, false, __GFP_ZERO); if (IS_ERR(folio)) { mutex_unlock(&hugetlb_fault_mutex_table[hash]); error =3D PTR_ERR(folio); goto out; } - folio_zero_user(folio, addr); __folio_mark_uptodate(folio); error =3D hugetlb_add_to_page_cache(folio, mapping, index); if (unlikely(error)) { diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index f016bc2e8936..49e5557d6cc0 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -599,6 +599,7 @@ enum hugetlb_page_flags { HPG_vmemmap_optimized, HPG_raw_hwp_unreliable, HPG_cma, + HPG_zeroed, __NR_HPAGEFLAGS, }; =20 @@ -659,6 +660,7 @@ HPAGEFLAG(Freed, freed) HPAGEFLAG(VmemmapOptimized, vmemmap_optimized) HPAGEFLAG(RawHwpUnreliable, raw_hwp_unreliable) HPAGEFLAG(Cma, cma) +HPAGEFLAG(Zeroed, zeroed) =20 #ifdef CONFIG_HUGETLB_PAGE =20 @@ -706,7 +708,8 @@ int isolate_or_dissolve_huge_folio(struct folio *folio,= struct list_head *list); int replace_free_hugepage_folios(unsigned long start_pfn, unsigned long en= d_pfn); void wait_for_freed_hugetlb_folios(void); struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, - unsigned long addr, bool cow_from_owner); + unsigned long user_addr, bool cow_from_owner, + gfp_t gfp); struct folio *alloc_hugetlb_folio_nodemask(struct hstate *h, int preferred= _nid, nodemask_t *nmask, gfp_t gfp_mask, bool allow_alloc_fallback); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 3a6afbe99116..4ccf4eb91d92 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1712,6 +1712,9 @@ void free_huge_folio(struct folio *folio) bool restore_reserve; unsigned long flags; =20 + /* Page was mapped to userspace; no longer known-zero */ + folio_clear_hugetlb_zeroed(folio); + VM_BUG_ON_FOLIO(folio_ref_count(folio), folio); VM_BUG_ON_FOLIO(folio_mapcount(folio), folio); =20 @@ -2113,6 +2116,10 @@ static struct folio *alloc_surplus_hugetlb_folio(str= uct hstate *h, if (!folio) return NULL; =20 + /* Mark as known-zero only if __GFP_ZERO was requested */ + if (gfp_mask & __GFP_ZERO) + folio_set_hugetlb_zeroed(folio); + spin_lock_irq(&hugetlb_lock); /* * nr_huge_pages needs to be adjusted within the same lock cycle @@ -2176,11 +2183,11 @@ static struct folio *alloc_migrate_hugetlb_folio(st= ruct hstate *h, gfp_t gfp_mas */ static struct folio *alloc_buddy_hugetlb_folio_with_mpol(struct hstate *h, - struct vm_area_struct *vma, unsigned long addr) + struct vm_area_struct *vma, unsigned long addr, gfp_t gfp) { struct folio *folio =3D NULL; struct mempolicy *mpol; - gfp_t gfp_mask =3D htlb_alloc_mask(h); + gfp_t gfp_mask =3D htlb_alloc_mask(h) | gfp; int nid; nodemask_t *nodemask; =20 @@ -2877,16 +2884,19 @@ typedef enum { * When it's set, the allocation will bypass all vma level reservations. */ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, - unsigned long addr, bool cow_from_owner) + unsigned long user_addr, bool cow_from_owner, + gfp_t gfp) { struct hugepage_subpool *spool =3D subpool_vma(vma); struct hstate *h =3D hstate_vma(vma); + unsigned long addr =3D user_addr & huge_page_mask(h); struct folio *folio; long retval, gbl_chg, gbl_reserve; map_chg_state map_chg; int ret, idx; struct hugetlb_cgroup *h_cg =3D NULL; - gfp_t gfp =3D htlb_alloc_mask(h) | __GFP_RETRY_MAYFAIL; + + gfp |=3D htlb_alloc_mask(h) | __GFP_RETRY_MAYFAIL; =20 idx =3D hstate_index(h); =20 @@ -2954,13 +2964,12 @@ struct folio *alloc_hugetlb_folio(struct vm_area_st= ruct *vma, folio =3D dequeue_hugetlb_folio_vma(h, vma, addr, gbl_chg); if (!folio) { spin_unlock_irq(&hugetlb_lock); - folio =3D alloc_buddy_hugetlb_folio_with_mpol(h, vma, addr); + folio =3D alloc_buddy_hugetlb_folio_with_mpol(h, vma, user_addr, gfp); if (!folio) goto out_uncharge_cgroup; spin_lock_irq(&hugetlb_lock); list_add(&folio->lru, &h->hugepage_activelist); folio_ref_unfreeze(folio, 1); - /* Fall through */ } =20 /* @@ -2983,6 +2992,10 @@ struct folio *alloc_hugetlb_folio(struct vm_area_str= uct *vma, =20 spin_unlock_irq(&hugetlb_lock); =20 + if ((gfp & __GFP_ZERO) && !folio_test_hugetlb_zeroed(folio)) + folio_zero_user(folio, user_addr); + folio_clear_hugetlb_zeroed(folio); + hugetlb_set_folio_subpool(folio, spool); =20 if (map_chg !=3D MAP_CHG_ENFORCED) { @@ -4991,7 +5004,7 @@ int copy_hugetlb_page_range(struct mm_struct *dst, st= ruct mm_struct *src, spin_unlock(src_ptl); spin_unlock(dst_ptl); /* Do not use reserve as it's private owned */ - new_folio =3D alloc_hugetlb_folio(dst_vma, addr, false); + new_folio =3D alloc_hugetlb_folio(dst_vma, addr, false, 0); if (IS_ERR(new_folio)) { folio_put(pte_folio); ret =3D PTR_ERR(new_folio); @@ -5520,7 +5533,7 @@ static vm_fault_t hugetlb_wp(struct vm_fault *vmf) * be acquired again before returning to the caller, as expected. */ spin_unlock(vmf->ptl); - new_folio =3D alloc_hugetlb_folio(vma, vmf->address, cow_from_owner); + new_folio =3D alloc_hugetlb_folio(vma, vmf->address, cow_from_owner, 0); =20 if (IS_ERR(new_folio)) { /* @@ -5780,7 +5793,13 @@ static vm_fault_t hugetlb_no_page(struct address_spa= ce *mapping, goto out; } =20 - folio =3D alloc_hugetlb_folio(vma, vmf->address, false); + /* + * Passing vmf->real_address would work just as well, + * but PAGE_MASK helps make sure we never pass + * USER_ADDR_NONE by mistake. + */ + folio =3D alloc_hugetlb_folio(vma, vmf->real_address & PAGE_MASK, + false, __GFP_ZERO); if (IS_ERR(folio)) { /* * Returning error will result in faulting task being @@ -5800,7 +5819,6 @@ static vm_fault_t hugetlb_no_page(struct address_spac= e *mapping, ret =3D 0; goto out; } - folio_zero_user(folio, vmf->real_address); __folio_mark_uptodate(folio); new_folio =3D true; =20 @@ -6239,7 +6257,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, goto out; } =20 - folio =3D alloc_hugetlb_folio(dst_vma, dst_addr, false); + folio =3D alloc_hugetlb_folio(dst_vma, dst_addr, false, 0); if (IS_ERR(folio)) { pte_t *actual_pte =3D hugetlb_walk(dst_vma, dst_addr, PMD_SIZE); if (actual_pte) { @@ -6286,7 +6304,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, goto out; } =20 - folio =3D alloc_hugetlb_folio(dst_vma, dst_addr, false); + folio =3D alloc_hugetlb_folio(dst_vma, dst_addr, false, 0); if (IS_ERR(folio)) { folio_put(*foliop); ret =3D -ENOMEM; --=20 MST From nobody Sun May 24 22:35:51 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2ABC23C0606 for ; Wed, 20 May 2026 22:22:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315746; cv=none; b=cRFmCXGREU90AGBjiLO26FbueMHWnczQWE7pEeAgsZCKPZu1a68d6AOCJXPtbSqCs/xQQwXKsb7MVI1lG2hKwKrbKZMU5zWz1NBspgbkViZ2lEavNsqJPvDorqnMjy9GHSx1mVnhU+jf/ZnB7sY50JLOPl7PRuvB8ZLpbPetcf8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315746; c=relaxed/simple; bh=Axczvl8C15WZZkzyVXb9w4AL4G4ah5FFkNusKG6v0bk=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=JkViN9+ZnTQ4UHhL2q5xWtlj1zIQzYeAhH5KP/H/4MsJMULD5ayva14fDiIbjCjUOwuLNsWkNkUNNL8abePFAN280wjnQ9xewQtK3Fvi9nvMpvNhs/rhamNmRGxXiGWRMx4ddlSjd9RO1Bf6s5nFmKGHRIypps3gMOE0WtG0sfk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=M8Rzke9D; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=PA8A9GfH; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="M8Rzke9D"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="PA8A9GfH" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315744; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Ha0MKvEjPpqKYjU+TNR9cOwfxNwOYM3BFVBNkyNiIfo=; b=M8Rzke9Dcg0AYjal93ef4JM+IULMFy9nSs6XpFb4RwnTIENJHGkVqS26rIFVHGYW6A0okG dA5MA2URC/G3oxXfXa4q3NBp9F4ZfSKTNiClL69qmK+TtCxGdndb/V6aH1sk7kVd25QIIe BgJBjVKQZkbGb1k5XoiLe3BFRcQfeJw= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-111-yZh-gSlTPI2bNENjHFe6Mw-1; Wed, 20 May 2026 18:22:23 -0400 X-MC-Unique: yZh-gSlTPI2bNENjHFe6Mw-1 X-Mimecast-MFC-AGG-ID: yZh-gSlTPI2bNENjHFe6Mw_1779315742 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-44d83e45febso4336397f8f.0 for ; Wed, 20 May 2026 15:22:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315742; x=1779920542; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Ha0MKvEjPpqKYjU+TNR9cOwfxNwOYM3BFVBNkyNiIfo=; b=PA8A9GfHcWGmoIwQ2ECXHbI+/0UNrQic4DAWpby5cTP+sYs681LGKgUNU72rdDy1fH dflhc0a15L/PnriGSE8WQa3TeHR+Sx3owr3GiB7SUQNLF3eHmSymbg2C9+1vq8um2PsS wob1wMbhnKBP5izJcw1sUcYqdAWkoDBAWf+XdM4aGLIfDXuS03fY3TKtzfKH9+9PYfHt w4WNcC6I77+QihfLlDUsbu+UKcWdaPWe6xtYNdVFZA5gVTHAH864aEAUT7FZ5411qmbb gKCWmK/skDW9Q4Bo1jsuuuMdVsUFLSyZ1xoHJUSgyAUXRJhEJd3kNu8qev/aDXJubnZV vsaA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315742; x=1779920542; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ha0MKvEjPpqKYjU+TNR9cOwfxNwOYM3BFVBNkyNiIfo=; b=gDSp1LYUAc+VDA16YO6pngUGmbPZGYFZ4MnWa6ob5cYh9LWPADaKY8/qDOjFu9FYzP mtImZPAH5hsckuges+vFvN7syD95FJxyctFRjCjGN97+733ASOAqn0CLW5IJOVF/Vbm4 VVq962fpF0VIOL2BRoCp2V0gOLmRe64wx1tw3XbOzw2JniwSioam1nBRrWo6tj2Zlx+C 43qlQ9VPBVI0vdqakVxoarRWnrVUKA8vZVcAeFd6MHSPeo1tLsiFWip/7t013+JymOgk Xtj9nwrYh2+13coWsJI/g7LQgnk0mvQYVAQCYHBl6qiWNcnkboglJHZtZvQxFCu/nnzC j+oQ== X-Gm-Message-State: AOJu0Yy/Mo971mfw6PcHATQeqg5dMUtzsfJQ4zkL3MOZav5/a8sa+kjA PjKQ74PhrCcGFKdfuVQiZo5tf7KCl/8MzZblwpc+AmGDpUb3636iCEYmc9WbYg/LYIbOd4hcath rB+/KqywIn0AgoXUlXR3SRClgglMrCFD7HJxxtiYJaAAtSWrvQ4hu6GmiPbewITEh8wSye4kcmH w59GFLLs9IDmd1PkydjzL9k9/q+i4vS0CLmEszIPxJKVg= X-Gm-Gg: Acq92OF0Ofp1fYO/0GFPNwjxXOonpFDZklBCzz96lQ4QfxJTuHFH05NXhc+cMP1NFQf ShNqJ1ItVRmiKsSH9DBy/pY/Gcki5iMNc1JC7n8l1MyFCJ7bbcarV6kv+ZrfY6K7wVtMVo3aNoX coEx2ouUIAyExY6SpKMyKQ27Eb0sFllH/IfJNX+J/5Y4QfrnRYdqaqikGBazKf3Zm+KpUnwgqoq 6TmZ3sKRsrJh3cFDrZLViVuVruuBt0S6Z+wORyF+kCHLkuXTkrmoaF14BJgR+0CFMHQhhXrr9bf B2NShXnSC3rPvA84KH5YOkvfIamDDWGsqFWxy3Ld4izEMI3AYAyd7FRunvVt0eRE7bw9lLf0vj/ t3531ZBuiZ79o3SQ+bqc+4SfRHiXoI2TH+L13kiFZG2g= X-Received: by 2002:a05:600c:8286:b0:489:32b:ac0b with SMTP id 5b1f17b1804b1-4903606a0bdmr2450975e9.6.1779315741749; Wed, 20 May 2026 15:22:21 -0700 (PDT) X-Received: by 2002:a05:600c:8286:b0:489:32b:ac0b with SMTP id 5b1f17b1804b1-4903606a0bdmr2450295e9.6.1779315741251; Wed, 20 May 2026 15:22:21 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49033d3f99csm31986145e9.6.2026.05.20.15.22.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:22:20 -0700 (PDT) Date: Wed, 20 May 2026 18:22:16 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 21/37] mm: memfd: skip zeroing for zeroed hugetlb pool pages Message-ID: <05befd0e641749ea70c1289f5a6ea7edaed266d7.1779315441.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add bool *zeroed output to alloc_hugetlb_folio_reserve() so callers can check whether the pool page is known-zero. memfd's memfd_alloc_folio() uses this to skip the explicit folio_zero_user() when the page is already zero. This avoids redundant zeroing for memfd hugetlb pages that were pre-allocated into the pool and never mapped to userspace. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/cma.h | 3 ++- include/linux/hugetlb.h | 6 ++++-- mm/cma.c | 6 ++++-- mm/hugetlb.c | 11 +++++++++-- mm/hugetlb_cma.c | 4 ++-- mm/memfd.c | 14 ++++++++------ 6 files changed, 29 insertions(+), 15 deletions(-) diff --git a/include/linux/cma.h b/include/linux/cma.h index 8555d38a97b1..dee88909cf5d 100644 --- a/include/linux/cma.h +++ b/include/linux/cma.h @@ -53,7 +53,8 @@ extern bool cma_release(struct cma *cma, const struct pag= e *pages, unsigned long =20 struct page *cma_alloc_frozen(struct cma *cma, unsigned long count, unsigned int align, bool no_warn); -struct page *cma_alloc_frozen_compound(struct cma *cma, unsigned int order= ); +struct page *cma_alloc_frozen_compound(struct cma *cma, unsigned int order, + gfp_t caller_gfp); bool cma_release_frozen(struct cma *cma, const struct page *pages, unsigned long count); =20 diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 49e5557d6cc0..3d23267014e6 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -714,7 +714,8 @@ struct folio *alloc_hugetlb_folio_nodemask(struct hstat= e *h, int preferred_nid, nodemask_t *nmask, gfp_t gfp_mask, bool allow_alloc_fallback); struct folio *alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_= nid, - nodemask_t *nmask, gfp_t gfp_mask); + nodemask_t *nmask, gfp_t gfp_mask, + bool *zeroed); =20 int hugetlb_add_to_page_cache(struct folio *folio, struct address_space *m= apping, pgoff_t idx); @@ -1134,7 +1135,8 @@ static inline void wait_for_freed_hugetlb_folios(void) =20 static inline struct folio * alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_nid, - nodemask_t *nmask, gfp_t gfp_mask) + nodemask_t *nmask, gfp_t gfp_mask, + bool *zeroed) { return NULL; } diff --git a/mm/cma.c b/mm/cma.c index c7ca567f4c5c..27971f6264ab 100644 --- a/mm/cma.c +++ b/mm/cma.c @@ -924,9 +924,11 @@ struct page *cma_alloc_frozen(struct cma *cma, unsigne= d long count, return __cma_alloc_frozen(cma, count, align, gfp); } =20 -struct page *cma_alloc_frozen_compound(struct cma *cma, unsigned int order) +struct page *cma_alloc_frozen_compound(struct cma *cma, unsigned int order, + gfp_t caller_gfp) { - gfp_t gfp =3D GFP_KERNEL | __GFP_COMP | __GFP_NOWARN; + gfp_t gfp =3D GFP_KERNEL | __GFP_COMP | __GFP_NOWARN | + (caller_gfp & __GFP_ZERO); =20 return __cma_alloc_frozen(cma, 1 << order, order, gfp); } diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 4ccf4eb91d92..649972101ace 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -2208,7 +2208,7 @@ struct folio *alloc_buddy_hugetlb_folio_with_mpol(str= uct hstate *h, } =20 struct folio *alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_= nid, - nodemask_t *nmask, gfp_t gfp_mask) + nodemask_t *nmask, gfp_t gfp_mask, bool *zeroed) { struct folio *folio; =20 @@ -2224,6 +2224,12 @@ struct folio *alloc_hugetlb_folio_reserve(struct hst= ate *h, int preferred_nid, h->resv_huge_pages--; =20 spin_unlock_irq(&hugetlb_lock); + + if (zeroed && folio) { + *zeroed =3D folio_test_hugetlb_zeroed(folio); + folio_clear_hugetlb_zeroed(folio); + } + return folio; } =20 @@ -2308,7 +2314,8 @@ static int gather_surplus_pages(struct hstate *h, lon= g delta) * It is okay to use NUMA_NO_NODE because we use numa_mem_id() * down the road to pick the current node if that is the case. */ - folio =3D alloc_surplus_hugetlb_folio(h, htlb_alloc_mask(h), + folio =3D alloc_surplus_hugetlb_folio(h, + htlb_alloc_mask(h), NUMA_NO_NODE, &alloc_nodemask, USER_ADDR_NONE); if (!folio) { diff --git a/mm/hugetlb_cma.c b/mm/hugetlb_cma.c index 7693ccefd0c6..c9266b25be3d 100644 --- a/mm/hugetlb_cma.c +++ b/mm/hugetlb_cma.c @@ -35,14 +35,14 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order,= gfp_t gfp_mask, return NULL; =20 if (hugetlb_cma[nid]) - page =3D cma_alloc_frozen_compound(hugetlb_cma[nid], order); + page =3D cma_alloc_frozen_compound(hugetlb_cma[nid], order, gfp_mask); =20 if (!page && !(gfp_mask & __GFP_THISNODE)) { for_each_node_mask(node, *nodemask) { if (node =3D=3D nid || !hugetlb_cma[node]) continue; =20 - page =3D cma_alloc_frozen_compound(hugetlb_cma[node], order); + page =3D cma_alloc_frozen_compound(hugetlb_cma[node], order, gfp_mask); if (page) break; } diff --git a/mm/memfd.c b/mm/memfd.c index fb425f4e315f..5518f7d2d91f 100644 --- a/mm/memfd.c +++ b/mm/memfd.c @@ -69,6 +69,7 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgoff= _t idx) #ifdef CONFIG_HUGETLB_PAGE struct folio *folio; gfp_t gfp_mask; + bool zeroed; =20 if (is_file_hugepages(memfd)) { /* @@ -93,17 +94,18 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgo= ff_t idx) folio =3D alloc_hugetlb_folio_reserve(h, numa_node_id(), NULL, - gfp_mask); + gfp_mask, + &zeroed); if (folio) { u32 hash; =20 /* - * Zero the folio to prevent information leaks to userspace. - * Use folio_zero_user() which is optimized for huge/gigantic - * pages. Pass 0 as addr_hint since this is not a faulting path - * and we don't have a user virtual address yet. + * Zero the folio to prevent information leaks to + * userspace. Skip if the pool page is known-zero + * (HPG_zeroed set during pool pre-allocation). */ - folio_zero_user(folio, 0); + if (!zeroed) + folio_zero_user(folio, 0); =20 /* * Mark the folio uptodate before adding to page cache, --=20 MST From nobody Sun May 24 22:35:51 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 489883CAE83 for ; Wed, 20 May 2026 22:22:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315753; cv=none; b=XHZo9m/SirgJOEEGZdvcgmSmDAonLGsoCUaSXfUeQCUu3ar1Z3jAd8wvtUSYnZRqxeQseudhlxg67HHJqElMtJjgxwdkJ+5a+sbW0ZQGLNPwtkM6B5MKrmbuU7YY4kaOxl/iazX7hs1qB61NSLeFkBkavCkg9AKV+o8ySEPyMsA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315753; c=relaxed/simple; bh=ZyZ1Q3SGthpxwZPpcR+XbFTSCrHBCLmyR21l6pltuGc=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=S3iZJ2yfgMPl/SD3CmJCieq2drO03TJYidBjZsXGDpDE5mq6MZ7t3tHqxeC5VHLDBFaZ+2Y8YVGwmb9BuR6QVrdyFzI3nbcPcdia1+RjeawX1oAkwf2NBVIx7SdUJYrJi/EACQ08p6QYlsJfQnDYGnQI+2L+UZKbVG2oqD5XH/M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=KYvmqAnL; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=Ejzx9SPx; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="KYvmqAnL"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="Ejzx9SPx" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315750; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=1YVd6WxtbkfB/mW7jT3E6E8CanzKli79KiDV6z3ud74=; b=KYvmqAnL7Cc4+fdP7+OzT1AfjC9g8eUs3QhusqQBJoYkdFosgOzm9ff40yQn/qalRHgs4S OWZOxphVnpo7rsmHMwR9TlP4VvFSKf9sf0YQiWHEOtJwgIMO84umoplOBEyy5C5wUFWTPG /BjctpQ6YXyk3FF5i4E7Io6Wyf6vYRQ= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-614-YpVrAzxHNDitSRj2TKgR9g-1; Wed, 20 May 2026 18:22:28 -0400 X-MC-Unique: YpVrAzxHNDitSRj2TKgR9g-1 X-Mimecast-MFC-AGG-ID: YpVrAzxHNDitSRj2TKgR9g_1779315747 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-44a52d5e572so4014896f8f.3 for ; Wed, 20 May 2026 15:22:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315747; x=1779920547; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=1YVd6WxtbkfB/mW7jT3E6E8CanzKli79KiDV6z3ud74=; b=Ejzx9SPx0/lyqOv8DKrccAM48QRwj0rv1YFUCOeGuuGbg29eVs8150hTVVeHUa6k6B 47dZfeSyRPBSJwixKLgEpfp9Ouucj5toDvGjSt1f5EI61VfhR8qolWdwC/9yipuKrev5 d+2wnZGULGPvy8Q53RqkPil1xffMfr4A7Ek8eEvxMfs+ejZ4FvcgDE/vEbYHrnHFd8LA L6xw5OEf4Mx40c8bt2aPLLEeGuqOrheDqqJqDbXLHOVGjsw9Acex36PkGzhj9CzV1W77 4M+JKLInw8vDxoae04IOoYwtN5+eYTP8y518aNN1jvpKm7rqTuaiGHqn2WLVkWHYkByo HYVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315747; x=1779920547; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1YVd6WxtbkfB/mW7jT3E6E8CanzKli79KiDV6z3ud74=; b=Kc6dk48HTc0AjGHrA0MhGfa/pTMKyfmWRxVWEDcQK+mc8H9beCTp0mrbMDS9D9OueI Jn75ne6cSYNr8iJ9vNfaH6NPVfKyLKX4I+CH/LYktda9F+7xJ9G25FOHF7U34EU2CFVa Yy2+V0XFcRFOQWTddVVQOw0futzGCxN+oViIuFJLt0ae7dduYZPvnHZklQP2p/Gq2mRt a3eaSmr4EPeSvbFlGPII6CRISYO64okZ9ySsaUAr9+p/fqQS7hkATfP3KLm9Wk8OZxiI oDNIUGQvW+NaZUkPP3/gKwL/6zVffExr2gQaKAfaonKsD3BVgTOVTGctapWM4yxVmGf6 QwJA== X-Gm-Message-State: AOJu0YyvMo629KD63NPGsvQvz6rXdZHtsgxT3MqzjmS1snGStuw9lXga do3uVNz9GL7hA3f+07+pG8e93WIfNfvq56jfXX5jSdnIK1WyPwFxpXUHD2Zgqv8XlVnCpBkHRJy AuxLCAp/L6IdMCsHZ2FmCOVEMpUc4cz/UlDbsDfkEh48HB3PhU8nwvzKKJL8Z6VRDH+UG7FgABh 1T/1fckmsqn6FgMKzcVev7gARYJY7QwY0RXJaA+jaBNMc= X-Gm-Gg: Acq92OEY0gGQ9Zg6yhcUDpWDjf/lGtr6EwXNCnSUuCVfSE0FEnDwqgz2kLLzpEm7kKn 2cNU1dKEDNEycl/Vl/8CAd42ptRTdWWGm4SPZ3MITq7eOY/MUiJTjjwJa1K4GmQhhkdTiWJX5hP NvAwEmNQpjT/JBmpeAIXgjfUeNB13IaAXoD544/xKJivyj/Ec0qCv2A7Ah2kQip/shOxO0Cz1PJ mt7eRjWtnwXX8D4nFR+aShRTK1e2KsxWYwHHfWyaXyA7MsqDOBGCM+Fvq06/rqM1YDNaua4lI0K OVprUvRBrxUe+2wOCJiw4ld2zGFaQv8w8Se1S8wva7b/vQXDqAWwITo20fl3rGtBzag8iDdtptK HXhHZitHSE78MXyeh6PvQo0pTQHnuu6MO39u1KwWP6U4= X-Received: by 2002:a05:6000:4310:b0:43d:e31:68d1 with SMTP id ffacd0b85a97d-45ea38ea8c2mr301753f8f.21.1779315747040; Wed, 20 May 2026 15:22:27 -0700 (PDT) X-Received: by 2002:a05:6000:4310:b0:43d:e31:68d1 with SMTP id ffacd0b85a97d-45ea38ea8c2mr301669f8f.21.1779315746363; Wed, 20 May 2026 15:22:26 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45da0fe1a41sm61751303f8f.31.2026.05.20.15.22.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:22:25 -0700 (PDT) Date: Wed, 20 May 2026 18:22:21 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 22/37] mm: page_reporting: skip redundant zeroing of host-zeroed reported pages Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When a guest reports free pages to the hypervisor via the page reporting framework (used by virtio-balloon and hv_balloon), the host typically zeros those pages when reclaiming their backing memory. However, when those pages are later allocated in the guest, post_alloc_hook() unconditionally zeros them again if __GFP_ZERO is set. This double-zeroing is wasteful, especially for large pages. Avoid redundant zeroing: - Add a host_zeroes_pages flag to page_reporting_dev_info, allowing drivers to declare that their host zeros reported pages on reclaim. A static key (page_reporting_host_zeroes) gates the fast path. - Add PG_zeroed page flag (sharing PG_private bit) to mark pages that have been zeroed by the host. Set it in page_reporting_drain() after the host reports them. - Thread the zeroed bool through rmqueue -> prep_new_page -> post_alloc_hook, where it skips redundant zeroing for __GFP_ZERO allocations. No driver sets host_zeroes_pages yet; a follow-up patch to virtio_balloon is needed to opt in. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- include/linux/page-flags.h | 9 +++++ include/linux/page_reporting.h | 3 ++ mm/compaction.c | 6 ++- mm/internal.h | 2 +- mm/page_alloc.c | 68 +++++++++++++++++++++++----------- mm/page_reporting.c | 14 ++++++- mm/page_reporting.h | 12 ++++++ 7 files changed, 88 insertions(+), 26 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 0e03d816e8b9..4ee64134acc3 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -135,6 +135,8 @@ enum pageflags { PG_swapcache =3D PG_owner_priv_1, /* Swap page: swp_entry_t in private */ /* Some filesystems */ PG_checked =3D PG_owner_priv_1, + /* Page contents are known to be zero */ + PG_zeroed =3D PG_private, =20 /* * Depending on the way an anonymous folio can be mapped into a page @@ -673,6 +675,13 @@ FOLIO_TEST_CLEAR_FLAG_FALSE(young) FOLIO_FLAG_FALSE(idle) #endif =20 +/* + * PageZeroed() tracks pages known to be zero. The allocator + * uses this to skip redundant zeroing in post_alloc_hook(). + */ +__PAGEFLAG(Zeroed, zeroed, PF_NO_COMPOUND) +#define __PG_ZEROED (1UL << PG_zeroed) + /* * PageReported() is used to track reported free pages within the Buddy * allocator. We can use the non-atomic version of the test and set diff --git a/include/linux/page_reporting.h b/include/linux/page_reporting.h index 5ab5be02fa15..c331c6b36687 100644 --- a/include/linux/page_reporting.h +++ b/include/linux/page_reporting.h @@ -14,6 +14,9 @@ struct page_reporting_dev_info { int (*report)(struct page_reporting_dev_info *prdev, struct scatterlist *sg, unsigned int nents); =20 + /* If true, host zeros reported pages on reclaim */ + bool host_zeroes_pages; + /* work struct for processing reports */ struct delayed_work work; =20 diff --git a/mm/compaction.c b/mm/compaction.c index 4336e433c99b..8000fc5e0a2e 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -82,7 +82,8 @@ static inline bool is_via_compact_memory(int order) { ret= urn false; } =20 static struct page *mark_allocated_noprof(struct page *page, unsigned int = order, gfp_t gfp_flags) { - post_alloc_hook(page, order, __GFP_MOVABLE, USER_ADDR_NONE); + __ClearPageZeroed(page); + post_alloc_hook(page, order, __GFP_MOVABLE, false, USER_ADDR_NONE); set_page_refcounted(page); return page; } @@ -1849,9 +1850,10 @@ static struct folio *compaction_alloc_noprof(struct = folio *src, unsigned long da set_page_private(&freepage[size], start_order); } dst =3D (struct folio *)freepage; + __ClearPageZeroed(&dst->page); if (order) prep_compound_page(&dst->page, order); - post_alloc_hook(&dst->page, order, __GFP_MOVABLE, USER_ADDR_NONE); + post_alloc_hook(&dst->page, order, __GFP_MOVABLE, false, USER_ADDR_NONE); set_page_refcounted(&dst->page); cc->nr_freepages -=3D 1 << order; cc->nr_migratepages -=3D 1 << order; diff --git a/mm/internal.h b/mm/internal.h index 389098200aa6..fd910743ddc3 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -928,7 +928,7 @@ static inline void init_compound_tail(struct page *tail, } =20 void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flag= s, - unsigned long user_addr); + bool zeroed, unsigned long user_addr); extern bool free_pages_prepare(struct page *page, unsigned int order); =20 extern int user_min_free_kbytes; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index ad0655387e9d..5158f7e23d18 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1746,6 +1746,7 @@ static __always_inline void page_del_and_expand(struc= t zone *zone, bool was_reported =3D page_reported(page); =20 __del_page_from_free_list(page, zone, high, migratetype); + nr_pages -=3D expand(zone, page, low, high, migratetype, was_reported); account_freepages(zone, -nr_pages, migratetype); } @@ -1818,8 +1819,10 @@ static inline bool should_skip_init(gfp_t flags) return (flags & __GFP_SKIP_ZERO); } =20 + inline void post_alloc_hook(struct page *page, unsigned int order, - gfp_t gfp_flags, unsigned long user_addr) + gfp_t gfp_flags, bool zeroed, + unsigned long user_addr) { bool init =3D !want_init_on_free() && want_init_on_alloc(gfp_flags) && !should_skip_init(gfp_flags); @@ -1828,6 +1831,14 @@ inline void post_alloc_hook(struct page *page, unsig= ned int order, =20 set_page_private(page, 0); =20 + /* + * If the page is zeroed, skip memory initialization. + * We still need to handle tag zeroing separately since the host + * does not know about memory tags. + */ + if (zeroed && init && !zero_tags) + init =3D false; + arch_alloc_page(page, order); debug_pagealloc_map_pages(page, 1 << order); =20 @@ -1870,7 +1881,7 @@ inline void post_alloc_hook(struct page *page, unsign= ed int order, * through a user-congruent mapping. Host-zeroed pages * (zeroed flag) don't need this: physical RAM is clean. */ - if (!init && (gfp_flags & __GFP_ZERO) && + if (!zeroed && !init && (gfp_flags & __GFP_ZERO) && user_addr !=3D USER_ADDR_NONE && user_alloc_needs_zeroing()) init =3D true; @@ -1903,13 +1914,13 @@ inline void post_alloc_hook(struct page *page, unsi= gned int order, } =20 static void prep_new_page(struct page *page, unsigned int order, gfp_t gfp= _flags, - unsigned int alloc_flags, - unsigned long user_addr) + unsigned int alloc_flags, bool zeroed, + unsigned long user_addr) { if (order && (gfp_flags & __GFP_COMP)) prep_compound_page(page, order); =20 - post_alloc_hook(page, order, gfp_flags, user_addr); + post_alloc_hook(page, order, gfp_flags, zeroed, user_addr); =20 /* * page is set pfmemalloc when ALLOC_NO_WATERMARKS was necessary to @@ -3175,6 +3186,7 @@ int __isolate_free_page(struct page *page, unsigned i= nt order) } =20 del_page_from_free_list(page, zone, order, mt); + __ClearPageZeroed(page); =20 /* * Set the pageblock if the isolated page is at least half of a @@ -3247,7 +3259,7 @@ static inline void zone_statistics(struct zone *prefe= rred_zone, struct zone *z, static __always_inline struct page *rmqueue_buddy(struct zone *preferred_zone, struct zone *zone, unsigned int order, unsigned int alloc_flags, - int migratetype) + int migratetype, bool *zeroed) { struct page *page; unsigned long flags; @@ -3282,6 +3294,8 @@ struct page *rmqueue_buddy(struct zone *preferred_zon= e, struct zone *zone, } } spin_unlock_irqrestore(&zone->lock, flags); + *zeroed =3D PageZeroed(page); + __ClearPageZeroed(page); } while (check_new_pages(page, order)); =20 /* @@ -3350,10 +3364,9 @@ static int nr_pcp_alloc(struct per_cpu_pages *pcp, s= truct zone *zone, int order) /* Remove page from the per-cpu list, caller must protect the list */ static inline struct page *__rmqueue_pcplist(struct zone *zone, unsigned int order, - int migratetype, - unsigned int alloc_flags, + int migratetype, unsigned int alloc_flags, struct per_cpu_pages *pcp, - struct list_head *list) + struct list_head *list, bool *zeroed) { struct page *page; =20 @@ -3388,6 +3401,8 @@ struct page *__rmqueue_pcplist(struct zone *zone, uns= igned int order, page =3D list_first_entry(list, struct page, pcp_list); list_del(&page->pcp_list); pcp->count -=3D 1 << order; + *zeroed =3D PageZeroed(page); + __ClearPageZeroed(page); } while (check_new_pages(page, order)); =20 return page; @@ -3396,7 +3411,8 @@ struct page *__rmqueue_pcplist(struct zone *zone, uns= igned int order, /* Lock and remove page from the per-cpu list */ static struct page *rmqueue_pcplist(struct zone *preferred_zone, struct zone *zone, unsigned int order, - int migratetype, unsigned int alloc_flags) + int migratetype, unsigned int alloc_flags, + bool *zeroed) { struct per_cpu_pages *pcp; struct list_head *list; @@ -3414,7 +3430,8 @@ static struct page *rmqueue_pcplist(struct zone *pref= erred_zone, */ pcp->free_count >>=3D 1; list =3D &pcp->lists[order_to_pindex(migratetype, order)]; - page =3D __rmqueue_pcplist(zone, order, migratetype, alloc_flags, pcp, li= st); + page =3D __rmqueue_pcplist(zone, order, migratetype, alloc_flags, + pcp, list, zeroed); pcp_spin_unlock(pcp); if (page) { __count_zid_vm_events(PGALLOC, page_zonenum(page), 1 << order); @@ -3439,19 +3456,19 @@ static inline struct page *rmqueue(struct zone *preferred_zone, struct zone *zone, unsigned int order, gfp_t gfp_flags, unsigned int alloc_flags, - int migratetype) + int migratetype, bool *zeroed) { struct page *page; =20 if (likely(pcp_allowed_order(order))) { page =3D rmqueue_pcplist(preferred_zone, zone, order, - migratetype, alloc_flags); + migratetype, alloc_flags, zeroed); if (likely(page)) goto out; } =20 page =3D rmqueue_buddy(preferred_zone, zone, order, alloc_flags, - migratetype); + migratetype, zeroed); =20 out: /* Separate test+clear to avoid unnecessary atomics */ @@ -3842,6 +3859,7 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int o= rder, int alloc_flags, struct pglist_data *last_pgdat =3D NULL; bool last_pgdat_dirty_ok =3D false; bool no_fallback; + bool zeroed; bool skip_kswapd_nodes =3D nr_online_nodes > 1; bool skipped_kswapd_nodes =3D false; =20 @@ -3986,10 +4004,11 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int= order, int alloc_flags, =20 try_this_zone: page =3D rmqueue(zonelist_zone(ac->preferred_zoneref), zone, order, - gfp_mask, alloc_flags, ac->migratetype); + gfp_mask, alloc_flags, ac->migratetype, + &zeroed); if (page) { prep_new_page(page, order, gfp_mask, alloc_flags, - ac->user_addr); + zeroed, ac->user_addr); =20 return page; } else { @@ -4216,9 +4235,11 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigne= d int order, count_vm_event(COMPACTSTALL); =20 /* Prep a captured page if available */ - if (page) - prep_new_page(page, order, gfp_mask, alloc_flags, + if (page) { + __ClearPageZeroed(page); + prep_new_page(page, order, gfp_mask, alloc_flags, false, ac->user_addr); + } =20 /* Try get a page from the freelist if available */ if (!page) @@ -5191,6 +5212,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, /* Attempt the batch allocation */ pcp_list =3D &pcp->lists[order_to_pindex(ac.migratetype, 0)]; while (nr_populated < nr_pages) { + bool zeroed =3D false; =20 /* Skip existing pages */ if (page_array[nr_populated]) { @@ -5199,7 +5221,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, } =20 page =3D __rmqueue_pcplist(zone, 0, ac.migratetype, alloc_flags, - pcp, pcp_list); + pcp, pcp_list, &zeroed); if (unlikely(!page)) { /* Try and allocate at least one page */ if (!nr_account) { @@ -5210,7 +5232,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, } nr_account++; =20 - prep_new_page(page, 0, gfp, 0, USER_ADDR_NONE); + prep_new_page(page, 0, gfp, 0, zeroed, USER_ADDR_NONE); set_page_refcounted(page); page_array[nr_populated++] =3D page; } @@ -6950,7 +6972,8 @@ static void split_free_frozen_pages(struct list_head = *list, gfp_t gfp_mask) list_for_each_entry_safe(page, next, &list[order], lru) { int i; =20 - post_alloc_hook(page, order, gfp_mask, USER_ADDR_NONE); + __ClearPageZeroed(page); + post_alloc_hook(page, order, gfp_mask, false, USER_ADDR_NONE); if (!order) continue; =20 @@ -7156,8 +7179,9 @@ static int __alloc_contig_frozen_range(unsigned long = start, unsigned long end, } else if (start =3D=3D outer_start && end =3D=3D outer_end && is_power_o= f_2(end - start)) { struct page *head =3D pfn_to_page(start); =20 + __ClearPageZeroed(head); check_new_pages(head, order); - prep_new_page(head, order, gfp_mask, 0, user_addr); + prep_new_page(head, order, gfp_mask, 0, false, user_addr); } else { ret =3D -EINVAL; WARN(true, "PFN range: requested [%lu, %lu), allocated [%lu, %lu)\n", diff --git a/mm/page_reporting.c b/mm/page_reporting.c index ae8d55f89f25..e42f857485ce 100644 --- a/mm/page_reporting.c +++ b/mm/page_reporting.c @@ -50,6 +50,8 @@ EXPORT_SYMBOL_GPL(page_reporting_order); #define PAGE_REPORTING_DELAY (2 * HZ) static struct page_reporting_dev_info __rcu *pr_dev_info __read_mostly; =20 +DEFINE_STATIC_KEY_FALSE(page_reporting_host_zeroes); + enum { PAGE_REPORTING_IDLE =3D 0, PAGE_REPORTING_REQUESTED, @@ -129,8 +131,11 @@ page_reporting_drain(struct page_reporting_dev_info *p= rdev, * report on the new larger page when we make our way * up to that higher order. */ - if (PageBuddy(page) && buddy_order(page) =3D=3D order) + if (PageBuddy(page) && buddy_order(page) =3D=3D order) { __SetPageReported(page); + if (page_reporting_host_zeroes_pages()) + __SetPageZeroed(page); + } } while ((sg =3D sg_next(sg))); =20 /* reinitialize scatterlist now that it is empty */ @@ -393,6 +398,10 @@ int page_reporting_register(struct page_reporting_dev_= info *prdev) /* Assign device to allow notifications */ rcu_assign_pointer(pr_dev_info, prdev); =20 + /* enable zeroed page optimization if host zeroes reported pages */ + if (prdev->host_zeroes_pages) + static_branch_enable(&page_reporting_host_zeroes); + /* enable page reporting notification */ if (!static_key_enabled(&page_reporting_enabled)) { static_branch_enable(&page_reporting_enabled); @@ -417,6 +426,9 @@ void page_reporting_unregister(struct page_reporting_de= v_info *prdev) =20 /* Flush any existing work, and lock it out */ cancel_delayed_work_sync(&prdev->work); + + if (prdev->host_zeroes_pages) + static_branch_disable(&page_reporting_host_zeroes); } =20 mutex_unlock(&page_reporting_mutex); diff --git a/mm/page_reporting.h b/mm/page_reporting.h index c51dbc228b94..736ea7b37e9e 100644 --- a/mm/page_reporting.h +++ b/mm/page_reporting.h @@ -15,6 +15,13 @@ DECLARE_STATIC_KEY_FALSE(page_reporting_enabled); extern unsigned int page_reporting_order; void __page_reporting_notify(void); =20 +DECLARE_STATIC_KEY_FALSE(page_reporting_host_zeroes); + +static inline bool page_reporting_host_zeroes_pages(void) +{ + return static_branch_unlikely(&page_reporting_host_zeroes); +} + static inline bool page_reported(struct page *page) { return static_branch_unlikely(&page_reporting_enabled) && @@ -46,6 +53,11 @@ static inline void page_reporting_notify_free(unsigned i= nt order) #else /* CONFIG_PAGE_REPORTING */ #define page_reported(_page) false =20 +static inline bool page_reporting_host_zeroes_pages(void) +{ + return false; +} + static inline void page_reporting_notify_free(unsigned int order) { } --=20 MST From nobody Sun May 24 22:35:51 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7AF1F3CB2E0 for ; Wed, 20 May 2026 22:22:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315756; cv=none; b=c5uogAOjH4a3HzjDiqIcsvpafR4OBRzbFl1YK1Gg0N+xx8+ABWbYkahnBPEPfQT+RnAZJLhv0fwd4zSE9g2/zjzD6SbC08tUxzvKiswd1ASoueX6RGTCmskR4AM85aAgIpSTFhr+YoEYrloFEpnMeX+KOUq0OPqcp89w///rO/M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315756; c=relaxed/simple; bh=v8XIPZddn0cM1ZAYTFFDjJe4mcF8BIRTQpXVMkTIuNs=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=ikvjhWFpGs1J2WxIYh9LGkZ+TyUI05Roh0Oe6jZ2U/+BroE10U0wl8GaC+eCDDTf0346VgQanm+wVtKDZioX1pYb8v4UcHZvXTuv9TIMYJyLZgtEX3RLXXtOu2pFy6iFgtOyXRDANREh7wV8UIkWHVil2e8iieeEihXEqDm5AZw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Tv1qTEPa; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=jCppVe29; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Tv1qTEPa"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="jCppVe29" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315754; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Fkdba+Ei5cbofNgiigEWYwJVG7QGPYteKxTN237/DT4=; b=Tv1qTEPa7hIQ2mAfBgYWcJbCz1ysnfMOTCh9cyFrBkgM0sbv6f7rgTYF68ENNeUzo6uEef Ma01FiRGcOxXAKWX1elg+uRXn3MgI6qWxAon7bitQROc5IJq6/MrZ0ALcLJMqKyJp126ya syWxEEEiOGBsYJxomAosDkZW+27JwlM= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-70-7SuLQ4YkMQKd0_5U26WEWQ-1; Wed, 20 May 2026 18:22:33 -0400 X-MC-Unique: 7SuLQ4YkMQKd0_5U26WEWQ-1 X-Mimecast-MFC-AGG-ID: 7SuLQ4YkMQKd0_5U26WEWQ_1779315752 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-43d789cebcfso4212059f8f.1 for ; Wed, 20 May 2026 15:22:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315752; x=1779920552; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Fkdba+Ei5cbofNgiigEWYwJVG7QGPYteKxTN237/DT4=; b=jCppVe29gFBNPgFzzfKdAXqSpWKcjzXew/EJpd64oyvVA2pksacojohpkbpG8a22Au dsBPGQ9v8stBOYsc8hKh7S23uUFRHA14vAjJld0sqR/mAHuJef2GDKiGBH1m9B2v8jQC 3PYZyqGHrV1qWuKCwcFQBkAElfeloGTAxJd7eGy9lXRtIc6R69nKrX3ANducrUvVNNjf 1KbjMd/ifyaNpm+BRbK9dbBwrp9H5RiL3bMiKZtBPEaR+J4ONYG8Hsrsxo+pFWy9i2+f rXeQ3bt21NQPEgsUmAZ4TbWcMFLBg9HWPiQAtbQd66GCZPjlvQWh0IUKUcXiqMucuvqw p+9A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315752; x=1779920552; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Fkdba+Ei5cbofNgiigEWYwJVG7QGPYteKxTN237/DT4=; b=WEJVIjASRj24JYuw860QXFMsXzJdHOiOfx4cFVDPCFnZVFDj+vkG/Wqva+RalzpBmx ny+X1UjaCYhzvCbGEz+ntDag6QddaJ+JdtETBj5IYYosDjXzzyrEJhvms8c4P3rDMvJF fJYGM1t0p/oXkBb3+5NBlaU2HEVyi/ljCkC4kanD7/bGoixMfezNdSTa13nWGR53qkjT Wxa1rDQmGjBVVjpIMB9PgXof1LXzcmwBxX1ysBpFHlhstrYdiiVU5WWsaXu53BijgeS1 InwquNxHhXrpF9lUmlUKAXxHRz99AcqMWz7i1beFpAqQ39G2CqctXg7uxcWmdpHFDu4T k3fw== X-Gm-Message-State: AOJu0Yx51NqcI0m1r1lxx02YT4ro/jPks7kvLMDta/PcDVGhSIKQUXeO tha1g1ZhzQT/s68VJ3th/ba5PwXjyj4E8uTBINZbJGbtnVIg5Ac1P87iF92gqDwvd0d+Uz8WDoX gKese4vuLvE0UhwMlLwxeDTMRA4km+kpWZTVX9cOqvO2cnuJu+wjmf8XIhAur5q6jnkYZr1/5DS tu8SSDKGQbPdBV79UTA/mrcWYVKHFWWwdX4DdxaV7Ur2c= X-Gm-Gg: Acq92OFSNhtVPkgVzvYdPFXw4eVjvi0iriovN9mUVogsETLVwaEZlL3O9rYDXDcl4MS CTOdQ/n4pwl3BRcxSiEu8iTwQUsFUYEljSeG4gY5S1SLubGz0ShrldAfc3bOLwUZGyKEP0inprV gIrZiS9NntjlPhAuwz6IAiQSaIURdwi8AlQCy8IsUp/Ecvt4qpFM149oF+THl4zKWYldTw+u+Vf mjdF9bVJr7zvUEVbfHq/vx9gnw4Dwn2D0IDwhJbqW3OTNwr/a43qlRtv7uuFBogJs+hjGCqTPZ0 /Vk39hF8cp0aZpvmXAJ+L1J+m7ss/Ir3qEUJ3tshG/fM+3DlGCdGJQTjamKxcgou5JIwKtPnhId /FOtF/O7pzNbEI4V7+X74uWBt047goIS0H862o8J1am4= X-Received: by 2002:a05:600c:6b12:b0:488:7d5b:125d with SMTP id 5b1f17b1804b1-49035f2e4f2mr2683695e9.7.1779315751964; Wed, 20 May 2026 15:22:31 -0700 (PDT) X-Received: by 2002:a05:600c:6b12:b0:488:7d5b:125d with SMTP id 5b1f17b1804b1-49035f2e4f2mr2683135e9.7.1779315751494; Wed, 20 May 2026 15:22:31 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45d9ed2ffdfsm53025915f8f.15.2026.05.20.15.22.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:22:31 -0700 (PDT) Date: Wed, 20 May 2026 18:22:26 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 23/37] mm: page_alloc: use aliasing checks instead of user_alloc_needs_zeroing Message-ID: <51265bb3f8374b093b91b07628d418375faacb9f.1779315441.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Replace user_alloc_needs_zeroing() with the direct aliasing checks (cpu_dcache_is_aliasing() || cpu_icache_is_aliasing()) in the post_alloc_hook aliasing guard. user_alloc_needs_zeroing() includes a !init_on_alloc term that means "allocator didn't zero this page." But in this guard's context (!zeroed && !init && __GFP_ZERO), we already know the page is zero; init incorporates init_on_alloc via want_init_on_alloc(). The only question left is whether the cache architecture needs the data re-zeroed through a congruent mapping, which is purely cpu_dcache_is_aliasing() || cpu_icache_is_aliasing(). On non-aliasing architectures with init_on_free=3Dtrue and init_on_alloc=3Dfalse, this avoids a redundant re-zero of an already-zero page. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/page_alloc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 5158f7e23d18..4cb7e779a6c5 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1883,7 +1883,7 @@ inline void post_alloc_hook(struct page *page, unsign= ed int order, */ if (!zeroed && !init && (gfp_flags & __GFP_ZERO) && user_addr !=3D USER_ADDR_NONE && - user_alloc_needs_zeroing()) + (cpu_dcache_is_aliasing() || cpu_icache_is_aliasing())) init =3D true; /* * If memory is still not initialized, initialize it now. --=20 MST From nobody Sun May 24 22:35:51 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7AF353CC33F for ; Wed, 20 May 2026 22:22:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315762; cv=none; b=YbfRuJez7W8BuSKFiQMPJOd4MQcPYY0R8DOxhtIj+PKZFLQcZ8Tu0lqQrNYDBk++LJbM8w6D3xR6RhQUJRoFsyPdOwFmh9NXfJi2f/LJc/d4a8k/oIsq9I5ESPYN8rOdBH/0XSIx/ue58KyyjAYh1hP7fQkdwdFyX3eit6EHgzc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315762; c=relaxed/simple; bh=h05Q2nCn+o78fr2Uz8tfGJ7k+kZvskvt29CW+J/CzdE=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=WFEnyeVuad7fM2EOzZsvsfYKFbFNvlwwbgRax85+rnTm1jJJo5kdv4CQWK7dgMYSidVNTOEc75zqLqjXbaYQpbJExU14hTDjifS0kCfnzFZr2XMMl1Z8IzgR+H2xa3HTcJW6JVGtg5AF5iyalx0WyBCXqiXhBJKXBdTCteA5xYg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=SnQIQfPe; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=InxaoE6P; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="SnQIQfPe"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="InxaoE6P" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315759; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ZUJUQcTEBW8Gz9Bvs58sr0Q3FF0IZpsM1OCMvoStcIE=; b=SnQIQfPekwx3WUJI6XlVlApvTGuyUMTttq/3IgawxE57zhnwDz0C3YnytfaBwGwNkadWMA mWIvN8yC7IgHVuhsWj0rjaI3od+GL22tU1chEd1CA7f0yWxZk5I0gPE7av0gFx06vwmkY6 EPhTJrndAgubRTWSNpSfWTPao5cG7aM= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-152-bEsmkh8IOwSalqrU4mU-8Q-1; Wed, 20 May 2026 18:22:38 -0400 X-MC-Unique: bEsmkh8IOwSalqrU4mU-8Q-1 X-Mimecast-MFC-AGG-ID: bEsmkh8IOwSalqrU4mU-8Q_1779315757 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-43d7b7bacddso3992694f8f.0 for ; Wed, 20 May 2026 15:22:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315757; x=1779920557; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=ZUJUQcTEBW8Gz9Bvs58sr0Q3FF0IZpsM1OCMvoStcIE=; b=InxaoE6Pz6KPOfgFomfRjP0w+Gsb8BulI9UkR4iJlAHMRihkFbexIIZeuPu6uyHQ5X jtufJYgxMJJzMvUWqNEGvIctVedTFYMEYVzcxr0oRIbS4BfdDBqWGItZig5Hu+ZirHKu EU+X09yy/+b3D7c/KIWfS7u3437NIG1LkY2TnA3eBCzkloGZGhtIJ17hRqLC/qhwI1KM GBo9hvpfIOtlyrMhiaZRlBIutrMPSY3pBIh0Bh9b8n6Jn5pfujQqMfB8c/9W1Ec2+0S6 f3Cleiu4BvGPIHlgSSDIggF0E4FYD+2LH086LM/Zd0mbjsTFNQkSJy7VPUnw+SGaTsAv iW1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315757; x=1779920557; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZUJUQcTEBW8Gz9Bvs58sr0Q3FF0IZpsM1OCMvoStcIE=; b=Z83+5kHL3Z1GYsXXgUY4zZwrKY4+ONnmvD1FfVflQMIvJujiH7HDK5UC68XESNavwW msiax/ZNOyKB8SyYLcCyF1pBFZy/oH3bVW0rfq9U/TOhgvAqOhE6HfCGpTDY6ybbaXeL AQih8d9xepl9K4SDdcMWtiCJaGuz5aZCgesdD/36i6eba1PPAY8y7SOht6dVE1skOITZ l3ytqmbFq4/z9Uye1cowOzGLPcMM/n2IllOSHoc94XXV+MwNslatLRiEyNXkCx5mw8ym TT/9Y5HhmYZsjFBFmNEibkxsnAHRd51gEyj2QEsAOTXJpt5S4828AezZlW9NI61miBFP soaQ== X-Gm-Message-State: AOJu0YyL3wHJgQmkNU7O4+5lZM8UWbmzwfcLHVmR6+JX121M6ibxChK2 FqgfI3gctUSivyHGkOob/KgG2J4iQkKigsiIRKe7Xb1gcZzpV6vd2PVX6bvVcmUp21+Gd3KokTi vH0SS4GV+p5ZOCAN0uJ0RrRK5XA4laVQjxZYkpBoz4zqMgZwQd0GY04kUJfUnyQXzUJt6uvHGlr TjSoiXhWu/h/8EoggjZsULe5zNufR7RruGLqUJK/Y5EuI= X-Gm-Gg: Acq92OFF5sS67LtU//L5yiW22rI9PIDhvsTgU9cOPbdqusjjp91h4j6lhIcq4ZJ9lXZ 63AtSLKUTXtlRlylt9ZcdOC8PBpMqE+X+zAMRywksHv+Te0cxhiDOzIea6sLP8uqAehIr51bEs5 BhR8RYzWtnuViXqqFPnXAHdn3WmLNtpuKpPdVK3+9vCO/tAhhmywIIHIzenXZ9/G2S11e1Zqaed UHJT7nhyDgH6r0DlDkrZq4umzNuCtYi3XS2ebifVGp0kF20RdPNArTgGoygUarK0oKjzsZhROvn D/QCDV9kBQ8DRMQGnyZspIp8pV3XBI0SF4S04cjdQSFTcDrEa3U1gb2ZEdQ2TEwFBZryp/Y++Nf evf3x9g4K8F18PtrCKYbkDdSWl6a9XmgdRZ+5xmBpf2c= X-Received: by 2002:a05:6000:2883:b0:43d:67d1:2022 with SMTP id ffacd0b85a97d-45ea38be3aemr335125f8f.19.1779315757155; Wed, 20 May 2026 15:22:37 -0700 (PDT) X-Received: by 2002:a05:6000:2883:b0:43d:67d1:2022 with SMTP id ffacd0b85a97d-45ea38be3aemr335047f8f.19.1779315756519; Wed, 20 May 2026 15:22:36 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45da15a6454sm58643722f8f.34.2026.05.20.15.22.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:22:36 -0700 (PDT) Date: Wed, 20 May 2026 18:22:31 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 24/37] mm: page_reporting: add per-page zeroed bitmap for host feedback Message-ID: <56b8f941ffcdef444d8c2aee62846bb8ad53ed14.1779315441.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The host may skip zeroing some reported pages (e.g., due to alignment constraints or bounce buffer fallback in QEMU). Currently, when host_zeroes_pages is set, all reported pages are unconditionally marked PG_zeroed - even ones the host did not actually zero. Add a zeroed_bitmap to page_reporting_dev_info that the report() callback can use to indicate which pages were actually zeroed. The driver's report() callback is responsible for managing the bitmap: zeroing it before sending pages to the host, then setting bits for pages the host actually zeroed. page_reporting_drain() checks the bitmap per-page in addition to the global host_zeroes_pages flag. No driver sets host_zeroes_pages yet, so the static key is off and the bitmap is never read. Behavior is unchanged. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- include/linux/page_reporting.h | 7 +++++++ mm/page_reporting.c | 8 ++++++-- 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/include/linux/page_reporting.h b/include/linux/page_reporting.h index c331c6b36687..e2e6a487ddab 100644 --- a/include/linux/page_reporting.h +++ b/include/linux/page_reporting.h @@ -17,6 +17,13 @@ struct page_reporting_dev_info { /* If true, host zeros reported pages on reclaim */ bool host_zeroes_pages; =20 + /* + * Per-page zeroed status, indexed by scatterlist position. + * The driver's report() callback must clear the bitmap, + * then set bits for pages that were actually zeroed. + */ + DECLARE_BITMAP(zeroed_bitmap, PAGE_REPORTING_CAPACITY); + /* work struct for processing reports */ struct delayed_work work; =20 diff --git a/mm/page_reporting.c b/mm/page_reporting.c index e42f857485ce..691a9e66aa5b 100644 --- a/mm/page_reporting.c +++ b/mm/page_reporting.c @@ -108,6 +108,7 @@ page_reporting_drain(struct page_reporting_dev_info *pr= dev, struct scatterlist *sgl, unsigned int nents, bool reported) { struct scatterlist *sg =3D sgl; + unsigned int i =3D 0; =20 /* * Drain the now reported pages back into their respective @@ -122,7 +123,7 @@ page_reporting_drain(struct page_reporting_dev_info *pr= dev, =20 /* If the pages were not reported due to error skip flagging */ if (!reported) - continue; + goto next; =20 /* * If page was not commingled with another page we can @@ -133,9 +134,12 @@ page_reporting_drain(struct page_reporting_dev_info *p= rdev, */ if (PageBuddy(page) && buddy_order(page) =3D=3D order) { __SetPageReported(page); - if (page_reporting_host_zeroes_pages()) + if (page_reporting_host_zeroes_pages() && + test_bit(i, prdev->zeroed_bitmap)) __SetPageZeroed(page); } +next: + i++; } while ((sg =3D sg_next(sg))); =20 /* reinitialize scatterlist now that it is empty */ --=20 MST From nobody Sun May 24 22:35:51 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 31D993C1973 for ; Wed, 20 May 2026 22:22:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315768; cv=none; b=pGMwLUsnIFMujkLfZ0tQFf+e5d8rE9NXBMnGav0DwOLPXABCVy/MaELzuCVmPidgYU1HG4oV9IXSi+FxQ5/hQM12CNYWXO4T5f8GMOM2sJvqMhCyLLI7mANBSlvsIHPhhG08ZwnHXUtgd6klNZuoia1YCvIpS1wjvz2A20crRIM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315768; c=relaxed/simple; bh=c6xJpyp5zpjmRc9jnwYRt4nCMF1SinUlHasxCFwTPkI=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=VAXf0tRFozYEnYrwep3QHBLIIIENknZWFDWiiFqnt8lCSu6OjZos2ySMkFQUprMNKENazLWnZMdk0BxrU4A3+XQCmRcF2zEfFqlF17h5g7t0kH0MeEy1WGAiVPOOYIUVG87JrU9NyKojiVfbm4BOyEthnM8GdNpBw8yv+XyexbQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Sow1jMbS; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=Y9ToiNET; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Sow1jMbS"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="Y9ToiNET" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315765; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=qdOI4mS59hNGv2oV9kd67xyAhp7b5Wts0l8Olu5Azv0=; b=Sow1jMbSo7ZN7nO7vn+mmMQV/fXuVtVuJjONeLyxiXsqzoUIuMYQEI9fKVBGtyCPvwgITr jBkSo1rRrh33eHgg+kykRQJ9xDUR8VaMFA55e9q9eRCzyIwunRo6BuMjE0n3piAStIz2IO iwupDhM4qzdBoOpmAlHNVgapKpe3h3s= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-641-gSczKFQ5NU6yoJUDxXsvZQ-1; Wed, 20 May 2026 18:22:44 -0400 X-MC-Unique: gSczKFQ5NU6yoJUDxXsvZQ-1 X-Mimecast-MFC-AGG-ID: gSczKFQ5NU6yoJUDxXsvZQ_1779315763 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-4497a0e3acaso5150289f8f.0 for ; Wed, 20 May 2026 15:22:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315763; x=1779920563; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=qdOI4mS59hNGv2oV9kd67xyAhp7b5Wts0l8Olu5Azv0=; b=Y9ToiNETUyTgx4ISIIzBgtouXjBO/pg/IcX/Bwfy58sJWea0TDPq8V/7h7xYkEckas 6vfqxRDMjMCugbkHce8rfV9Yf6fAV3Su/XvvgjFI8WyBi8BogpQOQ7hBUUdLoVle345m WUVaNRwiQr9LLKAEhzOzacjfkUENqtj/QSD7xlH5+/1SYYxccvA/DXOY7ZM7SmCwJt1T np86otCsHDaes5S3pWUsoSTfGpVm3oY624M+VBOXVKHmxZk8cvMrQmzVtAcEot2IbFn7 drNZNYWAvLVYBLuXyuVYwK0PSrhtAJKctd1C35TfE4c0tWkB5Mnop3kI1pFYjmrtPgP7 AFCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315763; x=1779920563; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qdOI4mS59hNGv2oV9kd67xyAhp7b5Wts0l8Olu5Azv0=; b=YYRIyybzQrKAk+n6G3iK7euk6f2V2muXIcp1KJshKnDv3IfldzkVWwCuTeBbfrOk// ijYCAsJGxPy+/bvFiJTOWiBLj+I61TsWN7A339qCb/rJ/fogEbhPQ1pZd4yHJRUjdHJ4 fObseLGEGW/vvsxyGgplNNLjJp6WqWO8uNRrLVLfNeuiB4ImlPjhbjhpJVtM2Wu8fm+F Wr3mJJ0u31xUFKvVJ5M74vK11BviBBDw/o1Db52kyvt/N0CtSz1/HGnsByMi+IBAs0nD RGY8gaKT4zek/ZjxesIwWCq+QFsvliwayhYS8lbT+W1nzfQ3TxC/fk7WtGHsKpRPE3fs 9beA== X-Gm-Message-State: AOJu0YxR1GOS512ydzWPg/iRLGYqtGrnmSkZ2YUf/awe+8th/kbFec1d I5vGlqKSC8+YGlILLCcs9wEu5ntkGMitOjemJLgw4QIlFcBdgPirnvVl/hpOj1v9ja7yMtYDDln AGQSXV9vGAFqj04Du5sMC/bCUGqykXmKTtfXJ26q+bXuYjS8YB+328J+IvAeq04f/Dt8bXw7sj7 jsQ72mSMyJ/suOJOKCEdcxHpZ1KR29nxVD4GXcLQtwNuk= X-Gm-Gg: Acq92OFile6PPdufjPi5XlU0ODH6YS04lkyQ8hvmaMbJ/d4Gn6tJVaEVf5w00bnr1TE iTmrlR8nMzpuSY0z1eK/MWDUtty9Y7uLT9naWMNrFyRdJQpx4IUrxyWheWNp3vZ9kb9KGxHU5nN abW2tpw/Bw8KS18OLuKXPBA5CB5qqhl2qdypRtYydNJOobdu1zEH6Uh9U1jfAU6pEFbBbHym9ng +br02fCoDPu7CB/LYwUYBaamWiUh3IzhZyR5rCAWv37yqBECyJEX3K+btDJOhi568D+Gzjl1oAw UfUGpqjhxLTxMPQZ48StChrVYJOJ/E1hAycCZgdSjw4SGSqQ+Hu9vfWmw65VI6ZuIO8vdw/1f49 6Jcue/OFZWZz4PSMFHO5dwMkVdzRX9xBHw8JNmU7XWt2J+Ew+DV4k8g== X-Received: by 2002:a5d:5e92:0:b0:456:15fa:9eba with SMTP id ffacd0b85a97d-45ea3df6cebmr217178f8f.43.1779315762793; Wed, 20 May 2026 15:22:42 -0700 (PDT) X-Received: by 2002:a5d:5e92:0:b0:456:15fa:9eba with SMTP id ffacd0b85a97d-45ea3df6cebmr217100f8f.43.1779315762157; Wed, 20 May 2026 15:22:42 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45d9adc2209sm54072453f8f.0.2026.05.20.15.22.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:22:41 -0700 (PDT) Date: Wed, 20 May 2026 18:22:36 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 25/37] mm: page_alloc: clear PG_zeroed on buddy merge if not both zero Message-ID: <68d88e8d227865ec4f8513d47bdbe06d0482851a.1779315441.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When two buddy pages merge in __free_one_page(), preserve PG_zeroed on the merged page only if both buddies have the flag set. Otherwise clear it. The merged page would inherit PG_zeroed, and a later __GFP_ZERO allocation would skip zeroing stale data in the non-zero half. Signed-off-by: Michael S. Tsirkin Reviewed-by: Gregory Price Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- include/linux/page-flags.h | 1 + mm/page_alloc.c | 15 ++++++++++++++- 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 4ee64134acc3..ff0b192b38e5 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -680,6 +680,7 @@ FOLIO_FLAG_FALSE(idle) * uses this to skip redundant zeroing in post_alloc_hook(). */ __PAGEFLAG(Zeroed, zeroed, PF_NO_COMPOUND) +CLEARPAGEFLAG(Zeroed, zeroed, PF_NO_COMPOUND) #define __PG_ZEROED (1UL << PG_zeroed) =20 /* diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 4cb7e779a6c5..ff614e422eec 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -940,10 +940,14 @@ static inline void __free_one_page(struct page *page, unsigned long buddy_pfn =3D 0; unsigned long combined_pfn; struct page *buddy; + bool buddy_zeroed; + bool page_zeroed; bool to_tail; =20 VM_BUG_ON(!zone_is_initialized(zone)); - VM_BUG_ON_PAGE(page->flags.f & PAGE_FLAGS_CHECK_AT_PREP, page); + /* PG_zeroed (aliased to PG_private) is valid on free-list pages */ + VM_BUG_ON_PAGE(page->flags.f & + (PAGE_FLAGS_CHECK_AT_PREP & ~__PG_ZEROED), page); =20 VM_BUG_ON(migratetype =3D=3D -1); VM_BUG_ON_PAGE(pfn & ((1 << order) - 1), page); @@ -978,6 +982,8 @@ static inline void __free_one_page(struct page *page, goto done_merging; } =20 + buddy_zeroed =3D PageZeroed(buddy); + /* * Our buddy is free or it is CONFIG_DEBUG_PAGEALLOC guard page, * merge with it and move up one order. @@ -996,10 +1002,17 @@ static inline void __free_one_page(struct page *page, change_pageblock_range(buddy, order, migratetype); } =20 + page_zeroed =3D PageZeroed(page); + __ClearPageZeroed(page); + __ClearPageZeroed(buddy); + combined_pfn =3D buddy_pfn & pfn; page =3D page + (combined_pfn - pfn); pfn =3D combined_pfn; order++; + + if (page_zeroed && buddy_zeroed) + __SetPageZeroed(page); } =20 done_merging: --=20 MST From nobody Sun May 24 22:35:51 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4E2673BFAD4 for ; Wed, 20 May 2026 22:22:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315772; cv=none; b=rRKdCwm1g5/68s871liFK5SdClAx7P9lESxicyd3CgrqDU66fTjJCgKxASk346ziNjnrNN0xqekVszmBT1bp6BqpvS1cyss5iZYdpxOw115OC9u0Zii0kWHyf2ZgjvS+DIWa8PsPOWwBS3jqbT4iNVQUknvuyggiPrB318LxCx0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315772; c=relaxed/simple; bh=5mf25/lf1DdZ8laPTDH+Cdq3xa7RrEIj1yvI/mG2wH0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Vj692gjtmArifR9lhasPH/9VuyR+vMxqyvB8pz/b2jLawGH+t1+C99vCmQ05mC2e9ZIISP867Hed3aCjssQ4YoHWAEUzBg3v77GnZ+DIVF/RLF93Wnv5U6SFPL2/W9wPEMQyjLrFoTVCgg1y2f16G5NOUeOfgupAUQCyQxcuvh0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=SnYVyqli; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=ls1A5p/G; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="SnYVyqli"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="ls1A5p/G" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315770; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=19buVa4Rma7mh18uS22QrkEHb5BZSKvMOxe4gXL+M+k=; b=SnYVyqlirTTtXhMir4KaNdDYZnmA27RKHkjTQY1rQglsGpOXbf+fE9Y4lC6L1qp5YeDudj 66bfUzTSL5ndUQ3506wbW/VFH3lxw+wPyGlwowEKo6HmUIz3iHlpfOtyKPHTJIZYXsKFy2 KUgdhqvLq9fP9jqShUEZ+2y7SP3M944= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-295-PjtqVb2bNh-8mWSJ_G7V8Q-1; Wed, 20 May 2026 18:22:49 -0400 X-MC-Unique: PjtqVb2bNh-8mWSJ_G7V8Q-1 X-Mimecast-MFC-AGG-ID: PjtqVb2bNh-8mWSJ_G7V8Q_1779315768 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-48ff0eb77b5so51033355e9.0 for ; Wed, 20 May 2026 15:22:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315768; x=1779920568; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=19buVa4Rma7mh18uS22QrkEHb5BZSKvMOxe4gXL+M+k=; b=ls1A5p/GCtZbUmqMsglb0U2Lwyr8EzKBW9BE7ZHAVe+da/EvKzaBvw7V8FMJKlrSfw O0xkeCr3cAEoOtVwVmXgNDYofxqmQhr4fMwRR2jXpRAoLpSPh4MwDcPXYDDAsgl82183 JBMB89EOxzksuGvUbeuk5muP76FdAaxHAq5Z3UJbxHLOnPmgkoYXPeJrAsmd7ehZBG6n B43q0D3HCfhD5AQXAzBEPF/48u0/o75+fQRt8VjEN0zMjNPmbQfyPiU4e5JqII7iPrEV EUDju0hMj/VTfjcrY+jQZFSDenEU2or+GyAA040+H1E7WrbxzzoxfAHLCGmojVz8JLIW +4pQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315768; x=1779920568; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=19buVa4Rma7mh18uS22QrkEHb5BZSKvMOxe4gXL+M+k=; b=heZVCCRV13QDoyVEcTaomXnhYojd9thTRAMy+3T+va/BO053lCLJKKmAtJ13C/NFU3 UYBOXbryTlfzAx/QZMfyf3XzwIpxz77EQUfyqKsu3UrGKKpFT/si3HpzNaSCfSe+CG8q xhE+QQ+zkHNcV1ptLn3gXW6Ihsn4GInvtoTbq+1dEhUy0k181U9gtKZrFYBPYuaZX0sG wSA0tBfvCmlo9wpqspDNnfAmXtBZgGFlPGB+Wmh+mje6cKfcQBsGMjd8YAQBncHY21t9 8V16IsWRX73Fr9QvcA6UzYhpe9VsODis+1BoskF9DWYHJzl5Lub8iLS9rgd5BDTFIxSn Owkg== X-Gm-Message-State: AOJu0YzodKVn8XMOax/aQ4KIYMQLAcl8ngOsqyOnR+D7/BLipxMBEOIg SYndh5FeuWj+P7cTFeWGWlKhYDFOnPH3o1MwVeA02KFXPR2sWAK5qUrSEd6p9XS575I05sAFhjX DhcKjOEtw8RkgvwoGCPE3I0QBESbSiDWXZ654uVSBe+nhVx/AsS/ny2BAhwBteWLIE2qWNVHwpU Yq8d3HT+OJ1OJyT6T4kEy00/voov75xeqDFCazUMdb0e0= X-Gm-Gg: Acq92OHUTep0BJlg53WxY+bhUWB5iBDLhgJ+uxgUtfQcu6O5Lwr6noAKJsfQv4Bx/qf BVPtVcablNkx7/HMPLNhCii9LycyBGamf0xW+JpmMVT0DM8HuFDSo7TM5A5TOyfTgDXM4ORwmPP tsFi+8LJeMZDITTygPqnHJGiV6sDw6IdKCeIHWAbizN0goBGmHqyrryVqXKGJfT/AEIdhK0tpmx OYJgRoElZk/18XZ4eE/BmfsmkBzguXC33nUh5xYpxccTXTk1c1ArH8XejhZJNWwy2cu30NiquvA qyZyvnKVc0IXohT7VGZXRscqbv0q3RzAK1OWseJjjrqMasEedzHpruiT9xh8Koi20F4+TBpU8RN boOoIcJ464QbE5H2DKkeOR8SbIGiNqWRg7tKl2EnG4Vw= X-Received: by 2002:a05:600c:c086:b0:489:1c5f:3a9e with SMTP id 5b1f17b1804b1-4903606bb2cmr2183615e9.13.1779315767778; Wed, 20 May 2026 15:22:47 -0700 (PDT) X-Received: by 2002:a05:600c:c086:b0:489:1c5f:3a9e with SMTP id 5b1f17b1804b1-4903606bb2cmr2182985e9.13.1779315767276; Wed, 20 May 2026 15:22:47 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49035f52a0dsm1490795e9.24.2026.05.20.15.22.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:22:46 -0700 (PDT) Date: Wed, 20 May 2026 18:22:42 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 26/37] mm: page_alloc: preserve PG_zeroed in page_del_and_expand Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Propagate PG_zeroed through buddy splits in page_del_and_expand() and try_to_claim_block(). When a zeroed high-order page is split to satisfy a smaller allocation, the sub-pages placed back on the free lists keep PG_zeroed. Signed-off-by: Michael S. Tsirkin Reviewed-by: Gregory Price Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/page_alloc.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index ff614e422eec..72e52f049cf0 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1715,7 +1715,8 @@ struct page *__pageblock_pfn_to_page(unsigned long st= art_pfn, * -- nyc */ static inline unsigned int expand(struct zone *zone, struct page *page, in= t low, - int high, int migratetype, bool reported) + int high, int migratetype, bool reported, + bool zeroed) { unsigned int size =3D 1 << high; unsigned int nr_added =3D 0; @@ -1746,6 +1747,8 @@ static inline unsigned int expand(struct zone *zone, = struct page *page, int low, */ if (reported) __SetPageReported(&page[size]); + if (zeroed) + __SetPageZeroed(&page[size]); } =20 return nr_added; @@ -1757,10 +1760,12 @@ static __always_inline void page_del_and_expand(str= uct zone *zone, { int nr_pages =3D 1 << high; bool was_reported =3D page_reported(page); + bool was_zeroed =3D PageZeroed(page); =20 __del_page_from_free_list(page, zone, high, migratetype); =20 - nr_pages -=3D expand(zone, page, low, high, migratetype, was_reported); + nr_pages -=3D expand(zone, page, low, high, migratetype, was_reported, + was_zeroed); account_freepages(zone, -nr_pages, migratetype); } =20 @@ -2356,11 +2361,12 @@ try_to_claim_block(struct zone *zone, struct page *= page, if (current_order >=3D pageblock_order) { unsigned int nr_added; bool was_reported =3D page_reported(page); + bool was_zeroed =3D PageZeroed(page); =20 del_page_from_free_list(page, zone, current_order, block_type); change_pageblock_range(page, current_order, start_type); nr_added =3D expand(zone, page, order, current_order, start_type, - was_reported); + was_reported, was_zeroed); account_freepages(zone, nr_added, start_type); return page; } --=20 MST From nobody Sun May 24 22:35:51 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 71F173C141F for ; Wed, 20 May 2026 22:22:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315778; cv=none; b=RM3wPH4FekOLzQukfMV6E6dsHd8FsMzDs8q+pxoQAP+EFKKoXrMuvJtuAc0XT76P+oosyeuxn/zIT52tQqmVaQpWFqdbmYqC54s8/EwtxvQc1iu3CYowq6/BmZ1WTQ9GYVPxQTHYPovT9PrTD1Muw7A43Fir2PwM3kBh9DgtaCk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315778; c=relaxed/simple; bh=NEUFfCyJTTjkAORA91qK34iLisrtuxPbcfRlH9SPC6Q=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=g4BuKhXiX0MxJI4SV5tD7KmqXWTRL3Ra210FMlvXDJiPT4yNEp59oYk8zp/tymjngdcat4/v34vS3TfwabFC2UJsa9GEbMRlQTv6cZP7RpcdmyYv6nxZ9LyxLFBseYmYK7TrOVQM3nLc1HNIFBMe6StyvuLclqpSgmjW8LXsF4Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=eXgqmevv; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=mF6bEu3n; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="eXgqmevv"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="mF6bEu3n" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315776; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=TrGGx5WHa8djlqQq7KjWOiK6GbnTz3UET362lIFlLIk=; b=eXgqmevvLDR4oRp2y93dTmB0mPqFYKHq1JyrKVFXMETkW4g/rKvBDt2bDxSCwxlr0rfPIl qU3tyAodUCX2P03e0U9ni8BXSgyH76CMTeFxeaX09m70OqAZcud6iWJUBe3GNhaSDC45cc vHlvVS/ivNS3r8EagF5nx9u79O0N+F4= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-394-0izGewMmMoGwZQgR-7BS3w-1; Wed, 20 May 2026 18:22:54 -0400 X-MC-Unique: 0izGewMmMoGwZQgR-7BS3w-1 X-Mimecast-MFC-AGG-ID: 0izGewMmMoGwZQgR-7BS3w_1779315773 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-488d1b5bca0so22074975e9.2 for ; Wed, 20 May 2026 15:22:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315773; x=1779920573; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=TrGGx5WHa8djlqQq7KjWOiK6GbnTz3UET362lIFlLIk=; b=mF6bEu3nJKJc734WNeeSvIC2fFpqMI0HTdR+L5/bVKOOMOqMOujWpo9+ppQaVT6e9r Bz+ILlbiHZIvHJgtvL4VcGO8OCdAA8vj/QqUfUNw1ejpFrbzONEBTMJfTxVWOEco2iDA WkxGQPz4GjItpo3+I+QMruHm1tnN9XBz9uBHdNyAc7OzAEwh7hFxpEUiKio92tISkk6N 0MIqLXpNcweEp2SI/0pI25onkC0h7N0mEqLwdp5Qk0MKIXrm5ixuN83GIghl2AdTRZWn GNxjgqPeRWEBe/BZ1zi89//I9E3RgkUOJhCYEZXQQOw32zXQ2YoAE2U5aMc10bXBKLdF Oj3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315773; x=1779920573; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TrGGx5WHa8djlqQq7KjWOiK6GbnTz3UET362lIFlLIk=; b=HHkrlHv5REO4DZxkCh9YKOTMjFZkoZzZIhCctDZDAh+T8xoj8ywtBi1IxCo3RvFJYE CaUq7+focqErMxsEmvClRx2DZoMm6HkXIp/0D3X3aAMhm+nkBNCr8hzPcH8sGoeiPwEF xF3kr0//tALOz8tXcJt8Uj170AUKKwG6uhoUYJT0ysoiITdZ4v1UxiUpqmFqUgY01t0o DAiRPg6ABoGGKtathHWYlOxlpoIv+6cF+/pYm9Zbs708F2qgDaCZD1bvE/bTf5V4rOZW y8kpvX+Tp475V0cXIrYPiTmkT+zU4Q7J9QIXlpeHvpSV5LA1igI7d0CymAHS73yvy1XX wEwA== X-Gm-Message-State: AOJu0Yy+YlCfoPFj95RyIkti7348W0386qwoau3OQ/PkhhhPPztMVUI8 o/trlOLdK8U9Iplr1f3yUrsbDv5hUShbGqCuwweE0FxakG9lsx8Mj68zXBIxES541QCW3uI3FXB 1qQqmucb8ZhNnpD7VBqseXXyXUSSxtpSxV3OxY2e+gdAwA3wC12+8p98ADfg8ZrCrpiNrO5pA4Z oAuWy1HYDQe+JtAiUs+I7IJL/zTFXQPnobe+PNl3lw8T0= X-Gm-Gg: Acq92OHeGWMd7aM2jTN86x7nt0U5dN+ivvFMYoRq2CnwLtWTSdrHPVoSXmDiE89LC6c yQeGhb21Zy8VczMWVcvAhvCgG4EeTX6qVmB7jgjj4m2xHxnCc+nFf3zAIFx1wS8uVnwsmyCTvdc HBEx0McjSqKASgw5ONsjr2SnQgY9EQUGCsf+NxXInBqOLM1oqs8sqnwBBV9SKp2wdJQhghLHtcT UbS5ybSXk3nSO/Tl17Ht0xqZcY/rA88m9HErR59sfmmZ/3yokj5q2Ff5CGuigF9Zx0n0rVIRnt+ 58SAe3RYRjtTlTTWuNL452KaUF5EwfbYrLZ91sMMYT5RvCnpDdzG0srpyzk1PEuK4sQsyjIt7ox loiHkgifHo1YWw4k+QZ9c9Zp7yWi/n8QXVjvg9Z/5UVU= X-Received: by 2002:a05:600c:840f:b0:48f:d1c0:721e with SMTP id 5b1f17b1804b1-490360539b9mr3370895e9.12.1779315773033; Wed, 20 May 2026 15:22:53 -0700 (PDT) X-Received: by 2002:a05:600c:840f:b0:48f:d1c0:721e with SMTP id 5b1f17b1804b1-490360539b9mr3370055e9.12.1779315772346; Wed, 20 May 2026 15:22:52 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49033d525ddsm23122965e9.6.2026.05.20.15.22.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:22:51 -0700 (PDT) Date: Wed, 20 May 2026 18:22:47 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 27/37] virtio_balloon: submit reported pages as individual buffers Message-ID: <0050d961a817b109ac032d121f095fb1fcc1fc5a.1779315441.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Submit each reported page as a separate virtqueue buffer instead of one buffer with an sg list of all pages. This avoids indirect descriptor allocation (kmalloc in the reporting path) and gives per-page used length feedback from the device. On error, the already-queued pages are kicked and drained before the error is returned. The caller (page_reporting_drain) then marks the batch as unreported, which is conservative but correct. Note: if the virtqueue is broken, wait_event on virtqueue_get_buf hangs. This is a pre-existing issue: the old single-buffer code had the same hang. EVENT_IDX is not a concern: callbacks were never disabled, so the virtqueue manages used_event automatically. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- drivers/virtio/virtio_balloon.c | 40 +++++++++++++++++++++------------ 1 file changed, 26 insertions(+), 14 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index 6a1a610c2cb1..53b4a3984e7d 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -187,7 +187,9 @@ static void tell_host(struct virtio_balloon *vb, struct= virtqueue *vq) =20 sg_init_one(&sg, vb->pfns, sizeof(vb->pfns[0]) * vb->num_pfns); =20 - /* We should always be able to add one buffer to an empty queue. */ + /* We made sure the vq is large enough so we should always + * be able to add one buffer to an empty queue. + */ virtqueue_add_outbuf(vq, &sg, 1, vb, GFP_KERNEL); virtqueue_kick(vq); =20 @@ -202,25 +204,35 @@ static int virtballoon_free_page_report(struct page_r= eporting_dev_info *pr_dev_i struct virtio_balloon *vb =3D container_of(pr_dev_info, struct virtio_balloon, pr_dev_info); struct virtqueue *vq =3D vb->reporting_vq; - unsigned int unused, err; + unsigned int i, err =3D 0; =20 /* We should always be able to add these buffers to an empty queue. */ - err =3D virtqueue_add_inbuf(vq, sg, nents, vb, GFP_NOWAIT); + for (i =3D 0; i < nents; i++) { + struct scatterlist one; =20 - /* - * In the extremely unlikely case that something has occurred and we - * are able to trigger an error we will simply display a warning - * and exit without actually processing the pages. - */ - if (WARN_ON_ONCE(err)) - return err; + sg_init_table(&one, 1); + sg_set_page(&one, sg_page(&sg[i]), sg[i].length, + sg[i].offset); + err =3D virtqueue_add_inbuf(vq, &one, 1, &sg[i], GFP_NOWAIT); + if (WARN_ON_ONCE(err)) { + nents =3D i; + break; + } + } =20 - virtqueue_kick(vq); + if (nents) { + virtqueue_kick(vq); =20 - /* When host has read buffer, this completes via balloon_ack */ - wait_event(vb->acked, virtqueue_get_buf(vq, &unused)); + /* When host has read buffer, this completes via balloon_ack */ + for (i =3D 0; i < nents; i++) { + unsigned int unused; =20 - return 0; + wait_event(vb->acked, + virtqueue_get_buf(vq, &unused)); + } + } + + return err; } =20 static void set_page_pfns(struct virtio_balloon *vb, --=20 MST From nobody Sun May 24 22:35:51 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6193D3C1992 for ; Wed, 20 May 2026 22:23:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315782; cv=none; b=DQxIKYPt46wW25V+eZpi0jmLts+hWaGrbFoTLJ3mveX49+iH0q02/vBXlzPKtd0Imb5CuOaTGjdP46hSNQVeT5SOq47XHWB+UfLvHCzrDO4wn2HUhOnb1EUyRGFRtp+uLu7VJG5YtLq7EHbah/cas1HCPLTnw6gQiOVbof9q5pU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315782; c=relaxed/simple; bh=b3DH92jukQeNkUO526fVTXypKGZyggM7uuOadXDZMdI=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=WJUPJkviWgBQV7ogKZ4VMt+IOMeu9ZyKX3THsyzammGa1d7Og+DreBlqth+wUKR9CqpwQ0ZnXVxQsEJ0F3SV6ZmJf7YPjieOy0H9WEl5cpM/oWb7J+tS6xLThT+YmmKKWj5syr3V3q+aVuSQMaqhksoMbB/qvYLOd6Otex8+KWQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=a2/5yx4z; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=jGpnlgnu; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="a2/5yx4z"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="jGpnlgnu" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315780; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=FmUnWEHUs3R8Yu/59LXfqjcAZ9F41NBb5tClVv8g9W0=; b=a2/5yx4zv9yQKy0uxf7IbsUHCCUgVapn4KMQo/9eRbD5iMD2vVZ+3G/C44AZPk3HTcRURk kIfvTby3PAqWIJU+D4bWe0GsTyo7B5R5CnZwRU5RhZT2mD5tAuGNrllzD1NvkB2cKALdxA gSBgAQU5Z661tg6MtST6U6XTtyWbkng= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-561-64SSNv9eOsCytBBSHrWleQ-1; Wed, 20 May 2026 18:22:59 -0400 X-MC-Unique: 64SSNv9eOsCytBBSHrWleQ-1 X-Mimecast-MFC-AGG-ID: 64SSNv9eOsCytBBSHrWleQ_1779315778 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-48a589c7879so41307995e9.1 for ; Wed, 20 May 2026 15:22:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315778; x=1779920578; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=FmUnWEHUs3R8Yu/59LXfqjcAZ9F41NBb5tClVv8g9W0=; b=jGpnlgnum5PocuB5qDxe1gJiPjmFfLHghFJ7tOmdvm6Ui/rwBpS1DsBvOd125JAXKg wlLzfnOGmZ2qFAs0Q8l0mGEwaNLIzWU4qN12bNDOC50NZ+RpAcbR+dRrqgOXzU4dX3oT J7jEIPi+jHh3Aoz37OZZu/lWmoO152DQj60wFIJ7Wd2YE9pQAmFmO2UecGIPi7dmKbxv 4yigd4HJqTfkhvsMmPmtY69pJk4/LhM83j0zxa4vju4a5mcqGsgZsER6I897VC/u03DA ScyLyme+u2Z92tAj1qR9iOKlgfrgO2GWvKJ6BtE4b4TBeAT2GoZqytYynAprHCtjSQbz iwng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315778; x=1779920578; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FmUnWEHUs3R8Yu/59LXfqjcAZ9F41NBb5tClVv8g9W0=; b=sYRBqVKf2ODSfey0Qz9WErTpMH5wolSjwiyAilNXxrv6K/wfLi4eU9It3GQkcJHzzK KLuuUpHb/Xh8LLRW/8i6MJDCDOZ+Ht+cyFu+HoRFIeThMa5AmidukCr2PtpGI/MHjusZ 9c5d3j97UEBOtfQ4hzh1wrMykMt9QoNXZtDoUfv/LU1M5zdcRXoVMxrZb4fnUC8TKGjR pulw6aFxykGf2qoG39Tvb7JBSAATR2NE5eI3Qcmqsk5lkFpEsuRIFaslE6UU6vW2XKzk YBJLZNQB/4UxkU9D/fgCPLJZQwiFn8jqeJnzcy+10J8rOpZILu/dEKvjYxy8WxRaKo1m DsKw== X-Gm-Message-State: AOJu0YyuphVQHEnn6ECyxgkIUPtvXj4zCtZ9BQuaLgPf7oIXVCGCHs9h mFi7R8uCXE5hJ+DuMXWd6BgjSNqVdJ2eD21rhoSIalZaiEbpI+QqAC7RpTqvTBLGVeLf9MEuED0 V/dXOJly3w3w0r3VZVchv9oOCvwD3goUQNyexQnxbc7YlDWHto87OGVky3xyyvpN002Qu8OJE2C ZO0Hd/uiZQ5+rvrU+uTS9U+UEUD0yBaQFaAb0dhx1FUI4= X-Gm-Gg: Acq92OFHDkfuy8q+PNoafOjxtdBve0bfBZeW+Ya6qnnC3N3Q2/NH7ApRWQcV/iChMDp vUIuzAgkmciy5V3FwNly0AP45EJ8N51+5vQg76J7MU8N70YJ3FeYedfW/S55FuaDG7v9oEf2+Qx ZoPtVPWaLVpNvmlibkYS1kpucX7Pv5LEzGK4088patu45UfjG19fnSA697NtilFI6ztBiVEVp5P poSPORv7bY1e7D6n4ogqHlDV4ixJK3/vrdLSB9nrCyCCn6/Fi5lKJ8gGFEbkkXuzkLOeO517S4y E4B27Al5c7TzRF9NH9q1Bs4r51uekgf3HG0MnA4IscvydAojl+Zi+HUgMjWTcR+X5RujX6in8JJ kPtY7oilqjEi4ClYQqmsg0pIDtVnAWOQT119V/F6fG1I= X-Received: by 2002:a05:600d:8496:20b0:48f:e230:29f4 with SMTP id 5b1f17b1804b1-490360dc195mr1175415e9.15.1779315777862; Wed, 20 May 2026 15:22:57 -0700 (PDT) X-Received: by 2002:a05:600d:8496:20b0:48f:e230:29f4 with SMTP id 5b1f17b1804b1-490360dc195mr1174845e9.15.1779315777370; Wed, 20 May 2026 15:22:57 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49033d3514esm23238795e9.3.2026.05.20.15.22.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:22:56 -0700 (PDT) Date: Wed, 20 May 2026 18:22:52 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 28/37] virtio_balloon: disable indirect descriptors Message-ID: <83beaef00179365fc6418a5d0aa4fa649d926834.1779315441.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Balloon submits at most 1-2 scatter-gather entries per buffer and virtqueue sizes are 128+, so indirect descriptors are unnecessary. Disabling them avoids a GFP_KERNEL allocation inside virtqueue_add_sgs when VIRTIO_RING_F_INDIRECT_DESC is negotiated. This allocation could trigger OOM reclaim while balloon_lock is held, leading to a deadlock: OOM notifier -> leak_balloon -> balloon_lock. With single-buffer submissions (previous patch) and no indirect descriptors, virtqueue_add never allocates memory. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- drivers/virtio/virtio_balloon.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index 53b4a3984e7d..1fa1c7fa285f 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -7,6 +7,7 @@ */ =20 #include +#include #include #include #include @@ -1175,6 +1176,13 @@ static int virtballoon_validate(struct virtio_device= *vdev) else if (!virtio_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_POISON)) __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_REPORTING); =20 + /* + * Balloon submits 1-2 sg entries max per buffer, virtqueue + * sizes are 128+. Disable indirect descriptors to avoid + * GFP_KERNEL allocation in virtqueue_add under balloon_lock, + * which could deadlock via OOM -> oom_notify -> leak_balloon. + */ + __virtio_clear_bit(vdev, VIRTIO_RING_F_INDIRECT_DESC); __virtio_clear_bit(vdev, VIRTIO_F_ACCESS_PLATFORM); return 0; } --=20 MST From nobody Sun May 24 22:35:51 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 871673C3459 for ; Wed, 20 May 2026 22:23:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315787; cv=none; b=HH9EdwTDkgymoDfcMpG7KkVEn4uC8jSUhjrm6LXshyan1PjCGp/5du/oNjDG0mpnRAw3/AQU5KeDX0yelN5jTykq3NdRFrJ9Dzc0Iaw/upPsLo5xM3flG8jm1AVxfa20st276I/1LutyeXoipVLx9aYOhI/FQQvCMeGcn2F9n9Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315787; c=relaxed/simple; bh=flUZ+cOTUcPYfWzZ31XAct9k/k2k4jVzLXPQVMmAbwo=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=tttv/VTIfE8MkIE3qmMZlywtvWkVX3Wr3L0o5lq9KZsMH3Y8V29ZNCN8ylpqHIhCVdnqvJHFl1Wt0A84+gHLEA4PLVkaQJ+7nPOfvxAiDOuqCnWZBsAY0iEXZpMiXOP+OPXXW88m4QpiZJgCz1JONJtw3T1H38lJ4kNWG/kZHak= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=dcck6qxZ; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=TkNIUBbs; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="dcck6qxZ"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="TkNIUBbs" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315785; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=uUKKOK0OmR1aDSnh65Nr9KPiL6ZvxY2ICpkCGdvKy4Q=; b=dcck6qxZmAGcANwTowxz3lo0FaT21luv0324uIfQodWOMv0R409hVp8FqxxxVahJkXrKGD gi7+rZPkIP1iLRyMVM/kxBCWG67u6rPugOFOILloUmVFJrnGbkpMXRfayWxZpW/N6uZIpL AyMN6ge5SG3tmNEDPJnf9IUawnJ2D4s= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-651-13KqapE8NoaMa5SYPsQOFg-1; Wed, 20 May 2026 18:23:04 -0400 X-MC-Unique: 13KqapE8NoaMa5SYPsQOFg-1 X-Mimecast-MFC-AGG-ID: 13KqapE8NoaMa5SYPsQOFg_1779315783 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-48fdca5c638so39671635e9.1 for ; Wed, 20 May 2026 15:23:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315783; x=1779920583; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=uUKKOK0OmR1aDSnh65Nr9KPiL6ZvxY2ICpkCGdvKy4Q=; b=TkNIUBbs2pL7HBi471mklKw1DXqzeU2tTHvFEsANl1pn9mnlvNWn4xOYP7RJU0tjNw 9T6LzEP+OwEEi3RhCFe4jmSwo6vgbjizDEl9OWJHghx6YViKfJhxRpZLz0UgP55Bcj9Q DPZvLCSH0/7QyeKSygINO7KQqS4wQOj2dCCw1dzBtFEiw3KGt/OFc0zHV6QADUQTGP75 4jjCjrBjQFy1vXUsYYBTvWhie+FsWK7SOqu8JE2uexhXCavVn6vG8e86O78M7RcNlvcj D8fyVt2+LR7mo1rFT/4+UtpieYLjMogD8Ngd6S6b1Hu2YCcylbDuRCJx83FSN45mcmr3 plOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315783; x=1779920583; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uUKKOK0OmR1aDSnh65Nr9KPiL6ZvxY2ICpkCGdvKy4Q=; b=oTKyL+PWdN0Z1ER1DwPFAmJV85t9IfxpiwiWcfYRcxx7tE6KwgklULr5yBlIpf09pb 0ZCTg2gmuiekrQ0jiDnyE6UdGBWUbgG5VZEVl/+izV4XCKVisLWvZnKL1pf4G44fWF/A J39kriponQRW5TvAhCROxPPeJefOpy94GNJEZZ9oZmR63I4HD+bZVMQ61ZUEIPGFzZf2 4Av53NbXwMXW/G0jMtKWiJpCGEGXnc+kzO7rYpR0MPkCS1gaeOBLyq0ODJTDFt0qIkQC Xv03QXNcXxgj2a+wH5vqwlHfHdxwftd/t0TiE2NIA6harJLlio8sjVreJsvggGDU5iST tWDA== X-Gm-Message-State: AOJu0Yz3fFbrGJzKbQGYd7/GJ/ugJGKStDzJat38n3FHoKmnNc8yXfz1 b63Y4mUafqCJCQyGsBuuddeKKe26yDqzWV+5qCeE4kNux8ijduXDciX/WqU9ztNb+S6R8gdWL6g hQd5NyhXoEi/c11RdgNwFZ2luEzgOjmVjRIin2uIyhPsD7BzGbDtfxIXxIr6fGZFL/5e27jUe0O KB9ocFm+Jn1tTo497ABXdbOjiVxYeEn3Q5G652n2ZsR3M= X-Gm-Gg: Acq92OGVuu/q6fqRJTaDGnUfxbKVSw4Knr1uPm+Fctd+vAu3eeTMsoIBdr0PmsV2kBX ypcIcnhm8/QEIc0kP2Mxo/B5kth9sMCUdR2e6MdTBg2jHW/NMvQ2XTbY2ahC4R7O4ihAl9B2r7Z xm9M+X5mEjzp5U02Ob7RZi0OQGl/CaVDOiBE8pFfk/iT2K8/wuqRtcvQztStYv7rlX/GSq6UZTz tJh/5/bZrdNYaMDOZd7c1O3d3EC0pUrr9Tzhndg5/RiCqKEG+TUv7+TWnC4QJdPlzyanujgT5gT nUkYL1Rz4cxZtsr0+wU9G6moOsyZuPXi+Q97b9oZtECH/v+dJ6MRswO6dvQdhtBLU5FAgQHEAho yNfAUCiAH3FxUb2bNB6oYbosBk3VKAfid/qNBObxukto= X-Received: by 2002:a05:600c:31a9:b0:48f:e6de:1cba with SMTP id 5b1f17b1804b1-49036089574mr1979165e9.17.1779315783076; Wed, 20 May 2026 15:23:03 -0700 (PDT) X-Received: by 2002:a05:600c:31a9:b0:48f:e6de:1cba with SMTP id 5b1f17b1804b1-49036089574mr1978505e9.17.1779315782397; Wed, 20 May 2026 15:23:02 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49033d3e9cbsm16828155e9.4.2026.05.20.15.22.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:23:01 -0700 (PDT) Date: Wed, 20 May 2026 18:22:57 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 29/37] mm: page_reporting: add flush parameter with page budget Message-ID: <842171d4147bb97a96b87386772a4ee2f058fabe.1779315441.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a write-only module parameter 'flush' that triggers immediate page reporting. The value specifies a page budget: at least this many pages (at page_reporting_order) will be reported, or all unreported pages if fewer remain. The actual number reported may exceed the budget since each reporting pass processes a full cycle across all zones. This is helpful when there is a lot of memory freed quickly, and a single cycle may not process all free pages due to internal budget limits. echo 512 > /sys/module/page_reporting/parameters/flush Note: the set callback runs under kernel param_lock, so writing this parameter blocks other built-in parameter writes until the flush loop completes. This is acceptable for a privileged debug/test parameter. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/page_reporting.c | 54 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/mm/page_reporting.c b/mm/page_reporting.c index 691a9e66aa5b..8b278a494ea5 100644 --- a/mm/page_reporting.c +++ b/mm/page_reporting.c @@ -358,6 +358,60 @@ static void page_reporting_process(struct work_struct = *work) static DEFINE_MUTEX(page_reporting_mutex); DEFINE_STATIC_KEY_FALSE(page_reporting_enabled); =20 +static int page_reporting_flush_set(const char *val, + const struct kernel_param *kp) +{ + struct page_reporting_dev_info *prdev; + unsigned int budget; + int err; + + err =3D kstrtouint(val, 0, &budget); + if (err) + return err; + if (!budget) + return 0; + + mutex_lock(&page_reporting_mutex); + prdev =3D rcu_dereference_protected(pr_dev_info, + lockdep_is_held(&page_reporting_mutex)); + if (prdev) { + unsigned int reported; + bool interrupted =3D false; + + for (reported =3D 0; reported < budget; + reported +=3D min(prdev->capacity, budget - reported)) { + /* + * First flush completes any previously scheduled + * reporting work. Then request a new reporting + * cycle and flush again to execute it. + */ + flush_delayed_work(&prdev->work); + __page_reporting_request(prdev); + flush_delayed_work(&prdev->work); + if (atomic_read(&prdev->state) =3D=3D PAGE_REPORTING_IDLE) + break; + if (signal_pending(current)) { + interrupted =3D true; + break; + } + } + if (interrupted) { + mutex_unlock(&page_reporting_mutex); + return -EINTR; + } + } + mutex_unlock(&page_reporting_mutex); + return 0; +} + +static const struct kernel_param_ops flush_ops =3D { + .set =3D page_reporting_flush_set, + .get =3D param_get_uint, +}; +static unsigned int page_reporting_flush; +module_param_cb(flush, &flush_ops, &page_reporting_flush, 0200); +MODULE_PARM_DESC(flush, "Report at least N pages at page_reporting_order, = or until all reported"); + int page_reporting_register(struct page_reporting_dev_info *prdev) { int err =3D 0; --=20 MST From nobody Sun May 24 22:35:51 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3A6C13C5535 for ; Wed, 20 May 2026 22:23:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315794; cv=none; b=XkHhHKUHT3xLsXFd2wlkn4IprmD7fqzGPnXr62Z+YIqJpGcqGKHbAi6mrooL3D6o+4WP8qbVLNW5mMgdy4rQ/pdIh9lBWV1T9C6NSsky53sbhVvivqWu/TMcHdFTxlaE04DIsAndGT7RDMYquD7CNjZ4YUec+xoHWUiD30w7Gvk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315794; c=relaxed/simple; bh=gX3aTYbHX/GyGonutM5PlW3FkeXMFkBsDPOEXbSa+qI=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Ahi8PIQ5w0Ha6Pu47gqU1zM356VJTedLWwsGVFvSKCDOVnAVWtkG286OzcwX66ESsKwF5fwJwK+ZYvX2bhzkEybK8PHWl66MpcCd/54+L/xF38lgEpH0aa2xkrre0tx5RY6D9ERoyWmxcHJZqZ4oE8kECfP1/g2Om8FBBQydYKk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=inAXTcAr; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=db9VyuCX; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="inAXTcAr"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="db9VyuCX" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315792; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=yuucL8pRyKGlavgC043yPLfH9MArN/0/T/V0zfd6qpo=; b=inAXTcArsJ/GPlS5MvDbgaWhy/O89WUY8UXFKM1c+yrh4HA1JO/NoB+RmwaAKw6ALQYHXd rgN0eY5DFbPhr4TzJHRctzt2oABDBgqiWGwRNfqU8YvB2aRxP0rWWFKIXfM1y+uYPulqwP DMDJ8EB+qh3lU7/5xRSwlXHEWzt7oSQ= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-166-ZPlarE5bPkOGHuepFS9QVA-1; Wed, 20 May 2026 18:23:09 -0400 X-MC-Unique: ZPlarE5bPkOGHuepFS9QVA-1 X-Mimecast-MFC-AGG-ID: ZPlarE5bPkOGHuepFS9QVA_1779315789 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-45e2f74da10so5973311f8f.2 for ; Wed, 20 May 2026 15:23:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315788; x=1779920588; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=yuucL8pRyKGlavgC043yPLfH9MArN/0/T/V0zfd6qpo=; b=db9VyuCXbgqIG29bWhl6vjwCrzXuVd9nVOkOX+mf11usIbY35uE098XK1YA/v//FoA 702u5n/jJwB+r7nCJPEiUc0RjeapSu355SeFOeDaFYrp3gqVpF6yZrF+kZuWiGStuAf2 25L8dMqxoqLE+VXc4Phmh2szAaWGeup0V9Ox2rVgeyB06AMABtNQoCldTNX4bGgjwQ/t Fu1GhElldtPYVL+dUH6gb9FaOLMDDuA3OMs8s0DntALzg9TxyN+kkS8YHa6w990fimU5 qdfuKyzNhp/gEJ+n7D/aryMMPL15ZO3Q/izGuriNPFpAcuFv9ro2x3O79Q2uYhufqeQ2 2f0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315788; x=1779920588; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yuucL8pRyKGlavgC043yPLfH9MArN/0/T/V0zfd6qpo=; b=O3uebXIFznFSLV0LFdvI594engsOHi+dGwLRNrA8zphb0rwwHBJpKU5WDQVRJxgCv0 hFwkrLMfImq0VpAniss1nosOJLWHaOjXsWMa5ObBiUfXsOEmRiaSG9aOerwZDcVfdIXr m4dYGQ8fTHIWjq5+VF9+KdxUVe+/gdYCThyzOUtTaXh473xO0aQs2UHbi4aF28wKt+bZ znskbOle+Ow38IS8t8u8W7jnZvxSdyXeEVurWAo6+IVL5NO5WrKdOtG3qXVyOmugowxL fw7XHmczWFGqGI9sILNqjGaVOFHrRzTT1CnCgM96YlGjwVuPNZhn0eZUoX0MfZMI2a/N 7+xA== X-Gm-Message-State: AOJu0YzPLsVlmn56e5BerLZLc155ENjOOYDVhsbZJFIIFco9tW6s6W3m UGLIFlCKezYaTAFRUIeE8TyFoe5KfHRNRhikLP6mxFsiZuYhuKdNjvdRkSMWnrNZCWtKpro4+j7 1eREZFIXmWivjeD2tNXTSuQ4kNeEkjeFYiTZS/LX4V4DVEi60dYBoGRzqvjy9GRyCI3tfcr2tov Bde32MovorRo3Heo4IifAjqNswIJDjPKqmSHzgzayE8nQ= X-Gm-Gg: Acq92OGDVDTAhZKjM9hcZsMrU0o6BCtVWSCqZN0FTWRgX96r0qjtjWgcdjPpDSkTL6E s7gzgEkz/HdgFX4Qq35gDmuNKGS8QZ7NR3NPCt7sFrLLB5xAeuOfMnakbfFjnIwSCkPOdrL+q5F CNQKkqs6h64QnJaDNzZYMP8LAdKiORVg3kGEoh0gVzc26obou0CItreex/pUIggDxKyIsELD7Xs nnb7ywDKP5QozaEzo/IKzsc6QkSvaBsgow1ncqCwGYo1A4RmIl6FVcuiMUCXReIBNG1AsdqJIxH zXTt5Bnb8tSjuZAsDC4N9Znm+NeoPCKi88wcadx3z98hYCLGbplq4wMRwGtZ1+ZWd7bmgcfHjin O+iz95dg9DsazWFg48vPab2/zhMYZgPXqMtbvg8AXvsY= X-Received: by 2002:a5d:5c81:0:b0:43c:f3ef:ee36 with SMTP id ffacd0b85a97d-45ea3ff957amr218759f8f.33.1779315788444; Wed, 20 May 2026 15:23:08 -0700 (PDT) X-Received: by 2002:a5d:5c81:0:b0:43c:f3ef:ee36 with SMTP id ffacd0b85a97d-45ea3ff957amr218690f8f.33.1779315787876; Wed, 20 May 2026 15:23:07 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45da0fe1a41sm61756873f8f.31.2026.05.20.15.23.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:23:07 -0700 (PDT) Date: Wed, 20 May 2026 18:23:02 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 30/37] mm: page_alloc: propagate PG_zeroed in split_large_buddy Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When splitting a large buddy page, propagate the PG_zeroed flag to each sub-page before freeing it. __free_pages_prepare clears all flags (including PG_zeroed), so the flag must be re-set on each fragment after the split. This ensures that the buddy merge logic can see PG_zeroed on pages that were part of a larger zeroed block. Signed-off-by: Michael S. Tsirkin Reviewed-by: Gregory Price Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/page_alloc.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 72e52f049cf0..702fa2ced1e1 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1524,6 +1524,7 @@ static void split_large_buddy(struct zone *zone, stru= ct page *page, { unsigned long end =3D pfn + (1 << order); bool reported =3D PageReported(page); + bool zeroed =3D PageZeroed(page); =20 VM_WARN_ON_ONCE(!IS_ALIGNED(pfn, 1 << order)); /* Caller removed page from freelist, buddy info cleared! */ @@ -1537,6 +1538,8 @@ static void split_large_buddy(struct zone *zone, stru= ct page *page, =20 if (reported) __SetPageReported(page); + if (zeroed) + __SetPageZeroed(page); __free_one_page(page, pfn, zone, order, mt, fpi); pfn +=3D 1 << order; if (pfn =3D=3D end) --=20 MST From nobody Sun May 24 22:35:51 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3D6293E3DA2 for ; Wed, 20 May 2026 22:23:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315798; cv=none; b=Td+dFscQQwwEq38H2ftpJwDzXCowFPtAs7ja1K/j0l/4BXiuXEjGjP7ZhubuXTXfJCA7Y6gw1PFjDaeaqqNFSYN6L+lUCTuKrfjUG5wyNgfBykbJoH6wEibs6gFLaeTIh5jsuAPPG/MHZbAIDBuu2nMtzlNS0IyK1nn/8PDCRMQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315798; c=relaxed/simple; bh=o1q520LUtIf3WEFA+mnt5VZVOCpKdiGNtHAvZKuDwl4=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=bmlKc9DMyUKUjh13ZqxrH2vJI4WNa0+XVTai1wGy2YBqprPkfJ+k9KFRM74myDL5oxnCphDfZ6d4CGhSyOBfalJLxg3yA8VvZ037Q1arhVvI5K9UBw0j2H7Ym1doR6CGiTgyl4q9/YXDPb7qmTajaKNx3+bHpNY9q3PmLR1k+R4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=gM8ly0/g; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=DHIBe+xT; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="gM8ly0/g"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="DHIBe+xT" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315796; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=PhUe/N+eFNvMvlBoMUh1oQQhfmsYB8gjnR/tug0hsFs=; b=gM8ly0/gT6k8YFQGXfmx/J4+shqp/WYOeko3CDS9oHOElS0Owrf9Pz5DAhDmTnEpkooT+B +vGpbpthEuQb3BxsQDW23Yosmt9jrzAPRomnJ/s1mCRZMXLhLS09sqAFOp9e7G/cie7Q2l 6Z2kiXR2M6TJPKM+aky1ake+Eb1Bm+M= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-649-yNlZyUZsOj6S5il-a-_rPw-1; Wed, 20 May 2026 18:23:15 -0400 X-MC-Unique: yNlZyUZsOj6S5il-a-_rPw-1 X-Mimecast-MFC-AGG-ID: yNlZyUZsOj6S5il-a-_rPw_1779315794 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-44d83e45febso4336640f8f.0 for ; Wed, 20 May 2026 15:23:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315793; x=1779920593; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=PhUe/N+eFNvMvlBoMUh1oQQhfmsYB8gjnR/tug0hsFs=; b=DHIBe+xTP0FDNzXozesKCQPmm4g2rQ0ejo+SX51iVb2wgznL3nimR5B8aUr3Iy4fOj 7kEOahUHgP4GxH2ckgtqqA4UqUm7lPIUdhdBGBJz4Cn4p3lYB/y3KtZ1oc5sh8JAvTD8 4Zg0nCU259UI1DBSgwrK0gpqfbDot0KJ2zT1gRWSLiVIkgdt8Krp55Dw3O68210XcIll n01o2reXIjGNan8zLmd5DRTbFFrIpUZB8LqpaKBozx6lcRyXjxlWDckQGQ8I7Oml2o8M OqDSqEYXsQ6MjoVRviZeWFhR9Djjhf3vShULecCheREIa/2wxdblMRtMB6/WDiJ1Y9Bl EPfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315793; x=1779920593; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PhUe/N+eFNvMvlBoMUh1oQQhfmsYB8gjnR/tug0hsFs=; b=f73IkKoNSRnbrYToocwLUxixeFFf+rExPMcapSUwcYd4Y7fV3hlhBA/McFyUcVfLCf dDHvI51J4dqTp1s15qMz7WhFWmPEsE4yeJcFAvMFrEdUgg04gMXVkuDoCVTYdjzA7yaV mRKkaC921rsNEbXTIr5IS+2SSICDpQ4ZlzJxvn4eAV+3KeUiMMzVNg9WHq71SVeXi1I/ vO1ZYtpVPG0aVlLFqE9eYnIe0dofkcIFWIHDQDNyz86Ce5Ye891N9eIyw7bPOWETI+T/ 1I0otOA00S3VaVeyZQLrRglglkJJpti40aXJhMQh/koSJl44RoydlQQxat3ZscmBQ3D2 1THw== X-Gm-Message-State: AOJu0YyptVDW2rF2sFiN33YqNmhDlhmfPUjxdJp3fQh7MZewIK0M+gFJ Dwy/DtHNznwdBBR2w+k9QqsxC43Aj9gnsuCcKeFefOBBBCYAC9Uej4XQjcvT+B2O256GRf6UZvn /FTWhfzKMA4kejmggSBq5ymlAFO45ED7GibzeS2mH+/LlMNXRNBTOLkl3A+mkHR7JIUnivBILH+ MOsLJZoBdxlvu2SKEjI7nHQeqNxfCzhaHryDtUc4scw68= X-Gm-Gg: Acq92OFL/V7gPGeoQxsLsNjPkH7nGln0J+TDaPohdOfipfhefkAEzkxe1JWvpfanP5k i2Bl5Ya7Tg8oRLibRI96wxJTRlesWSIcMxMRE9lKqiqRtm6h3yt2mJzmM7ROFBebTmdS9fX/weD tGxj9t6HEPQgEr8Qo2y4hQMO3xFgotVMMn9QbTjfH0+jsRwwfYLUsyVg2XhFjiArqDv4I9VWPGr 6auhbovA+1+izY4qFfWrwp4fYcSxG3cikmJ5EYrJD3uGw8bzXiqL0zYzOiKEfMwCVx3RJR45srI +2CtaIdPsAY80iCj1emDdo0lkB9NCJB2yOCKIvcZ1q78UdCiIY2hQwufkGCKa+hbJk1lcS7pjWQ JRSF/fyLstkx2MfqEyjmtt9Y1EN6dVDiVqi5vBqPK0SA= X-Received: by 2002:a05:600d:6446:20b0:48f:c649:e6fa with SMTP id 5b1f17b1804b1-49035f6841amr2480315e9.15.1779315793512; Wed, 20 May 2026 15:23:13 -0700 (PDT) X-Received: by 2002:a05:600d:6446:20b0:48f:c649:e6fa with SMTP id 5b1f17b1804b1-49035f6841amr2479685e9.15.1779315792933; Wed, 20 May 2026 15:23:12 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45da0fe13a7sm59877057f8f.29.2026.05.20.15.23.08 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:23:12 -0700 (PDT) Date: Wed, 20 May 2026 18:23:08 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 31/37] virtio_balloon: skip zeroing for host-zeroed reported pages Message-ID: <4753c094c752fc97c7d3372b5ca7db9414b42ca3.1779315441.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Implement VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED (per virtio spec proposal): when negotiated, the device initializes reported pages (zeros, or poison_val if PAGE_POISON). Check per-page used length returned by the device to determine which reported pages were zeroed. If used_len matches the page size, the device successfully initialized the page (e.g. via MADV_DONTNEED), and we set the corresponding zeroed_bitmap bit. Gate host_zeroes_pages on the feature bit and page content: when PAGE_POISON is negotiated with non-zero poison_val, the device fills with poison not zeros, so pages are not zeroed. Clear the feature in validate() if REPORTING is not present or if PAGE_POISON is active with non-zero poison_val. See the virtio spec change: https://github.com/oasis-tcs/virtio-spec/issues/244 Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- drivers/virtio/virtio_balloon.c | 22 ++++++++++++++++++++-- include/uapi/linux/virtio_balloon.h | 1 + 2 files changed, 21 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index 1fa1c7fa285f..e3afa6f32ba5 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -207,6 +207,8 @@ static int virtballoon_free_page_report(struct page_rep= orting_dev_info *pr_dev_i struct virtqueue *vq =3D vb->reporting_vq; unsigned int i, err =3D 0; =20 + bitmap_zero(pr_dev_info->zeroed_bitmap, nents); + /* We should always be able to add these buffers to an empty queue. */ for (i =3D 0; i < nents; i++) { struct scatterlist one; @@ -226,10 +228,14 @@ static int virtballoon_free_page_report(struct page_r= eporting_dev_info *pr_dev_i =20 /* When host has read buffer, this completes via balloon_ack */ for (i =3D 0; i < nents; i++) { - unsigned int unused; + struct scatterlist *entry; + unsigned int used_len; =20 wait_event(vb->acked, - virtqueue_get_buf(vq, &unused)); + (entry =3D virtqueue_get_buf(vq, &used_len))); + if (used_len =3D=3D entry->length) + set_bit(entry - sg, + pr_dev_info->zeroed_bitmap); } } =20 @@ -1051,6 +1057,9 @@ static int virtballoon_probe(struct virtio_device *vd= ev) #endif =20 vb->pr_dev_info.capacity =3D capacity; + vb->pr_dev_info.host_zeroes_pages =3D + virtio_has_feature(vdev, + VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED); err =3D page_reporting_register(&vb->pr_dev_info); if (err) goto out_unregister_oom; @@ -1176,6 +1185,14 @@ static int virtballoon_validate(struct virtio_device= *vdev) else if (!virtio_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_POISON)) __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_REPORTING); =20 + if (!virtio_has_feature(vdev, VIRTIO_BALLOON_F_REPORTING)) + __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED); + + /* Device fills with poison_val, not zeros; disable zeroed hint */ + if (virtio_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_POISON) && + !want_init_on_free()) + __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED); + /* * Balloon submits 1-2 sg entries max per buffer, virtqueue * sizes are 128+. Disable indirect descriptors to avoid @@ -1194,6 +1211,7 @@ static unsigned int features[] =3D { VIRTIO_BALLOON_F_FREE_PAGE_HINT, VIRTIO_BALLOON_F_PAGE_POISON, VIRTIO_BALLOON_F_REPORTING, + VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED, }; =20 static struct virtio_driver virtio_balloon_driver =3D { diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virti= o_balloon.h index ee35a372805d..13074631f300 100644 --- a/include/uapi/linux/virtio_balloon.h +++ b/include/uapi/linux/virtio_balloon.h @@ -37,6 +37,7 @@ #define VIRTIO_BALLOON_F_FREE_PAGE_HINT 3 /* VQ to report free pages */ #define VIRTIO_BALLOON_F_PAGE_POISON 4 /* Guest is using page poisoning */ #define VIRTIO_BALLOON_F_REPORTING 5 /* Page reporting virtqueue */ +#define VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED 6 /* Device initializes repo= rted pages */ =20 /* Size of a PFN in the balloon interface. */ #define VIRTIO_BALLOON_PFN_SHIFT 12 --=20 MST From nobody Sun May 24 22:35:51 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E6D693BFE3F for ; Wed, 20 May 2026 22:23:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315803; cv=none; b=Eins+9bKQ7KUAdnkFHgmBut70Fnsjr8TYSf+NVAQ6nFrgjO81EVbcBIJKgHu4O2tFcHhzd1ihKJb6iyRpsfvg2uHIyN6FbFBkezKXgzBbjIcoLhi2FFrSKcE7ZaEfB8uzVnskm78nsx9s7Q5mskgZALa40GEoRB0T+m/J85PJ5c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315803; c=relaxed/simple; bh=KQP828o7Oia72X2oJuTNYtM9uBAruxIc45lg9ub3w4U=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=rA28PmJBArNLTlzkOHE+GOAvnj/wQ/FnPsKft+ZIx9sw++kcsPCAkY+Bd+dT7tIhEqkdajEkLJZsZLM28VagCagA7SSrTVv2oywbuPlvk2x2NRB7X9rmqUV8GhoWseMSAID2jvwVADc7nLBhKgRJtbeC68B2Wx2F+QpAGd2yDXs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=LB8j++J8; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=kJ9dJ+Y6; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="LB8j++J8"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="kJ9dJ+Y6" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315801; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=3kNatCmWHyttmbkexMBnMmdZ3Q708953lxRldkYnzX4=; b=LB8j++J8kRN7/TDR9UqB7BQ6Rq/+PwznDCN/Mix4r7my3jTxqeyCOFMF3zELmM+iR4ONj0 37xLJzKahMwF/SWkvtQ01q2iKZt/hVEjuG/19ALeckQmEB86ptNnPQ81VkAqVcdvl8Eurz gpqTNLDEURiEbySk3xjw9wQ3bR0+aec= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-461-sv4xB7JVMAGMnqWZ_c4ISw-1; Wed, 20 May 2026 18:23:19 -0400 X-MC-Unique: sv4xB7JVMAGMnqWZ_c4ISw-1 X-Mimecast-MFC-AGG-ID: sv4xB7JVMAGMnqWZ_c4ISw_1779315799 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-48feb029961so30577515e9.3 for ; Wed, 20 May 2026 15:23:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315798; x=1779920598; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=3kNatCmWHyttmbkexMBnMmdZ3Q708953lxRldkYnzX4=; b=kJ9dJ+Y62WeB1kd+eok8XqSL/9tI8pvZmU5UxcXYsEMhV57y2qdIr/Zts7TRU9Fbqu ZWytuXQyd2NJ7GM98uauc0WKyfuwfES59uO9TGGjdM6mj3BtfClI+aenRHbokjzigTf3 lTr4/Thqtuh2cvWVPEbtUzZqZfVA4XNkwdvOKBgt/K76f4mdSSFNn9syNThCsTuvTIqS 8+CpY+l8F4mAHlrwSNOORp8HpFdtuwbeWgWv1rc7HzWX2ndwa7Rhc5TWVIvApYqYonON NyTuKq4xAwEmjc62QVNVYtpoi4j0YzP53gXO/RNssv0nwFgH8M6zDuOpHuhoPFnM3+03 m8CA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315798; x=1779920598; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3kNatCmWHyttmbkexMBnMmdZ3Q708953lxRldkYnzX4=; b=PcYpd543TlQ5Ox3/TsZ8PYF13F4dhK/WRVIAZlo8Cigay1WHIZLPzTkSRNz0slNtzG OHodL1YXrbuCt5dWvy037bsnsoKCR/LAW8PpWNmFznLdESUdEWMM0eceEeCPLxNr/OZU owJGa/YsUgNc8iGB9Plyy6wbLHpFwYQt5B9e/45zAofw05I1uZ1Wyt7qIdup9gRgOTlT qHmM4my+dve/3zH+UiUn01f0AfIJjDxWSx3JpyQ/8kJvU7eufwPDwkWHwjpttHACBJyM 0qgPIucLOo+jbgI7C9+FgfhQmghtBP/0kfgFuQZ88borpyE2JOSodi2VMHcqfA10ddQe j0hg== X-Gm-Message-State: AOJu0YxQXAV8t7DmDavl9veCnnSRjg0crCCGGgTomftsiaoDFXLvkodZ FWM3vpx4QI4Kph4an944zukxs0bG4cLOLjyCJqYB/iaxWUbbpxMpZjxsbifa876eaa2PmJCdTg0 VxYcaeKvk1jz9E4wp9WJ5GtWstEzAd9R9RGCClOk9beVamLdNHPknEns4B0WHvQxq6wlK+sC5Ia u/DhRMdAOEfkctVqOAhr9yzzB72cKuC3Pa04Z7UlUmT0M= X-Gm-Gg: Acq92OHl6o3YgixWrRuCANkqewoQzqs3mwZjKFHnmjmrgSXqDbLjExZ/Ev5JMDeDLlb d0qlFQnzlxMLecKO9Uzu6y/opq5D4esmntYUjEEk8vBmd6lo8qVbVR3SuYirbKmNp8heyeCs9ZK 3XDoIUpteNbRAU2E8Q9MpINmJH47vRFvqv/Y+l9VJUXbz7Y1DQnaMGIr4IVycGHAeD1hN2ULOTB uya5LPqrF2Jq2fsfXRnpK7gjUjUMHVAeLlMdWRJwPDxZBA2qYhMXQ0PIYa9WhWyV7rM0C1SgnrM 9UetYWYNA5c2FD3TPyJkX2pekQkh8TZeA9rwNouc6lYFGC/zvEboznVVkuULexbOqkLSn9Gj4jR pO523zO/TL8irZM+IiWM++yzUKj7l9HnNJVftOnFVGfx+JEgCWdg9AQ== X-Received: by 2002:a05:600c:6211:b0:48f:d1b8:9aa4 with SMTP id 5b1f17b1804b1-4903604cac9mr2998555e9.7.1779315798563; Wed, 20 May 2026 15:23:18 -0700 (PDT) X-Received: by 2002:a05:600c:6211:b0:48f:d1b8:9aa4 with SMTP id 5b1f17b1804b1-4903604cac9mr2997825e9.7.1779315797919; Wed, 20 May 2026 15:23:17 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49033d3f0d1sm23277085e9.5.2026.05.20.15.23.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:23:17 -0700 (PDT) Date: Wed, 20 May 2026 18:23:13 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 32/37] virtio_balloon: disable reporting zeroed optimization for confidential guests Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In confidential computing environments (TDX, SEV-SNP), the host is untrusted and may lie about zeroing reported pages. Clear DEVICE_INIT_REPORTED in validate() so the guest does not skip re-zeroing based on hints from an untrusted device. Note: currently REPORTING remains enabled and VIRTIO_F_ACCESS_PLATFORM is cleared in CC environments. This is known to leak free page physical addresses to the host. Whether that, or ballooning in general, is a security concern in CC is up to the user. This patch only disables our new zeroed-page hints where the host is untrusted. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- drivers/virtio/virtio_balloon.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index e3afa6f32ba5..bf1172ad5419 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -19,6 +19,7 @@ #include #include #include +#include =20 /* * Balloon device works in 4K page units. So each page is pointed to by @@ -1193,6 +1194,8 @@ static int virtballoon_validate(struct virtio_device = *vdev) !want_init_on_free()) __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED); =20 + if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) + __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED); /* * Balloon submits 1-2 sg entries max per buffer, virtqueue * sizes are 128+. Disable indirect descriptors to avoid --=20 MST From nobody Sun May 24 22:35:51 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 411E23F23B7 for ; Wed, 20 May 2026 22:23:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315808; cv=none; b=ZsPtv6Q1YWgnjG5a1tLqnoUqyQnLuTcKZpPDbQfepXk2gqlWesEmN/Ac6gvdt3loHa3Kc0DAOIQFyywYPSasc8GWbT4vIcXal40bZdo7CjHI1Mc5IpfSSmguYHXyQ9EB/VIFFeRfHfVcwqNP6u0aEFKdXaOyqQkm/3bww7p82gA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315808; c=relaxed/simple; bh=6Lm0/uxEF7Jrx82xgvKLV/wotxiG6uz9abF+ckORdaA=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=aUpMvEDtQNuiAYmCCmjS53UJItzQYeKPTTxohsyyvogDGqWxgikJz/2i+eIyIq24HD3/U2ZEzaHlxFA4gI0hakshFIINuapgMWFUzK2IQAbL6zStQE/72c+Y4zIOLgXZq3ck2CiEkpinry7GvCY4h0iDBg3rL6bidj0+Gl3RX0Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=VGHak+2D; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=MXRvYr3e; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="VGHak+2D"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="MXRvYr3e" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315806; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=4vz+PjXrtbljoW8SfvrGsI8KrTf/iN/5SV/3uQ1i74A=; b=VGHak+2DCD3M5vDUfGSi+2F3C7AXMXT9QzBSZxzKVEkUEAQHq5HobX9BsGRH5MRpBCRQz7 89Ixg0pwr+o9ZKRWGMq27VcbS46SWlksIWfKgmieJENY4/W3R0JVhh1xYd0tUHsesJ/35f n5sbYMyaMBkiKN7KFkGL/E6SSX5lIoU= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-271-WhKWmYhFNSubsq1l7BIt-g-1; Wed, 20 May 2026 18:23:24 -0400 X-MC-Unique: WhKWmYhFNSubsq1l7BIt-g-1 X-Mimecast-MFC-AGG-ID: WhKWmYhFNSubsq1l7BIt-g_1779315804 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-45dbbbf8d57so7164226f8f.3 for ; Wed, 20 May 2026 15:23:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315804; x=1779920604; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=4vz+PjXrtbljoW8SfvrGsI8KrTf/iN/5SV/3uQ1i74A=; b=MXRvYr3eDlVjGR/cX/q6ofLTGz/Iypc1s4CCXqv7aqZ9wMWajlV596kDItbTwstT7K PxxRMfEVmf8S9ZOONtJKvLpEyfrxhpI4vA4nS1NF5qAaTIw23Khqki8kuTBHbYYwoD2v YpmxdtIb6IzUqrAF/q9OeySiV7/ZKJc/c4jNawnflzxpjvxIGqyExlWUvuYFQOk0i+hJ KJSyHU0+RBRqLETHmHf+gkyTw9i4L2VIQnjJOXPlkKsQApDrx1Vb7m6IZaXkcl+IFrqm Nqt/PUB0TKszIB84BQO1N20g1pE3Uwurx1t4rArOusUrDZAn7uRIhK3T2tiZmBWQLA6G Jv3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315804; x=1779920604; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4vz+PjXrtbljoW8SfvrGsI8KrTf/iN/5SV/3uQ1i74A=; b=VjP4o25KNB3sJtkFS1F+Wt2p6v8tlh5IJX0Gg9CFMsMoRFBgAEpPHGBTzWb9GKCYxb 48kwW4B8vygwpmghhCHCmk9jyxGdE6OtJ+jmOkSTZY23IWeCUTZB9xD70I02nVTKTX9/ EvLQ+e/LpMxWWto1JCmsQ8kQzhhwbkOm+Yh0r+hKsCgoMKJRqsFlSq46MpmJUADdaRcP WgZ6oOBNhSnJg5cpHzsFUgQ8sKi44tnkJSlYtWMe8caK4JlKI9UCQ3JspqosshhFXIFg WPFtkzqBDaULq2u+GDtmkUzqRasFiFKCxhRMXH1XSWHDFBeR8E+zUSDdwiiF1FPUQslN HmhQ== X-Gm-Message-State: AOJu0YzYoTtartmsXL0frxISCFPegYTuGsDh5449Z3WbAOYPZZRO89WB p0QOxqD0r8x2069tnjkavpNPo3MbBh5yB49jl31nQlLE7s9XjbSSn4mI82tlpukzeUdeaZ9xV+I jxxntXOuB8lv9RKBAGJsEdAZP2vrJDhpJ3SijAW9AiK4Tki6F7v/oSq/UwmEcs2Uq7YecFKTLJ4 tbsiOyuY9EpfmqPy2lEp157MkOiXQJE95oxrlupxDRWqY= X-Gm-Gg: Acq92OEp3vuQ4XsbQhr19+sEewuW3JQ9d8AF60kPKGa5Ghsg+OAvsZGB4pksEP4J/mF Zt+p9K/sY3xD6LyHpNH0PsYJYZrlai4IraUtp5lf1BKFP8Re05WWXMj4K41o42bpePvLWPYugrD WwBs9kXeNspcGyp7ZliazKo+l7iW69ThjsB3VSuti3ENFpdo/wip0sUmqe1Rf3mdlKBBSfniFjv Tqfw0D6+QV74eUHxsa2LNGqMyCBLC64Cy/8fmiR2ZoOdl+WNpKmKyzoyzFxdXN+UkJwCrDlvIx7 N3mq1/LS1fJ9tb8TqPMyrkMTgX0L33h7kkWpvDX4BRSDLrjk6qjy+ZVIdZHN3b1QLCRHNwImWYm TePAOpHN+IBeJeOngDOmPbbkeyWCWPsVh6njGiY6vtGo= X-Received: by 2002:a05:6000:2089:b0:43d:73de:abd2 with SMTP id ffacd0b85a97d-45ea3ae8801mr287906f8f.26.1779315803561; Wed, 20 May 2026 15:23:23 -0700 (PDT) X-Received: by 2002:a05:6000:2089:b0:43d:73de:abd2 with SMTP id ffacd0b85a97d-45ea3ae8801mr287814f8f.26.1779315802825; Wed, 20 May 2026 15:23:22 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45d9ed2ffdfsm53030712f8f.15.2026.05.20.15.23.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:23:22 -0700 (PDT) Date: Wed, 20 May 2026 18:23:18 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 33/37] mm: add free_frozen_pages_zeroed Message-ID: <321e9f1a5a15a80dfb25a9b8aa76657ce27b2b2d.1779315441.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add free_frozen_pages_zeroed(page, order) to free a frozen page while marking it as zeroed, so the next allocation can skip redundant zeroing. An FPI_ZEROED internal flag carries the hint through the free path. PageZeroed is set after __free_pages_prepare() clears all flags, so the hint survives on the free list. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/gfp.h | 1 + mm/internal.h | 1 - mm/page_alloc.c | 29 ++++++++++++++++++++++++++++- 3 files changed, 29 insertions(+), 2 deletions(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index 73109d4e31a4..d24b61e45861 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -384,6 +384,7 @@ __meminit void *alloc_pages_exact_nid_noprof(int nid, s= ize_t size, gfp_t gfp_mas extern void __free_pages(struct page *page, unsigned int order); extern void free_pages_nolock(struct page *page, unsigned int order); extern void free_pages(unsigned long addr, unsigned int order); +void free_frozen_pages_zeroed(struct page *page, unsigned int order); =20 #define __free_page(page) __free_pages((page), 0) #define free_page(addr) free_pages((addr), 0) diff --git a/mm/internal.h b/mm/internal.h index fd910743ddc3..4af5e72742ba 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -938,7 +938,6 @@ struct page *__alloc_frozen_pages_noprof(gfp_t, unsigne= d int order, int nid, #define __alloc_frozen_pages(...) \ alloc_hooks(__alloc_frozen_pages_noprof(__VA_ARGS__)) void free_frozen_pages(struct page *page, unsigned int order); -void free_frozen_pages_zeroed(struct page *page, unsigned int order); void free_unref_folios(struct folio_batch *fbatch); =20 #ifdef CONFIG_NUMA diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 702fa2ced1e1..399e2038dcd9 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -90,6 +90,13 @@ typedef int __bitwise fpi_t; /* Free the page without taking locks. Rely on trylock only. */ #define FPI_TRYLOCK ((__force fpi_t)BIT(2)) =20 +/* + * The page contents are known to be zero (e.g., the host zeroed them + * during balloon deflate). Set PageZeroed after free so the next + * allocation can skip redundant zeroing. + */ +#define FPI_ZEROED ((__force fpi_t)BIT(3)) + /* prevent >1 _updater_ of zone percpu pageset ->high and ->batch fields */ static DEFINE_MUTEX(pcp_batch_high_lock); #define MIN_PERCPU_PAGELIST_HIGH_FRACTION (8) @@ -1599,8 +1606,19 @@ static void __free_pages_ok(struct page *page, unsig= ned int order, unsigned long pfn =3D page_to_pfn(page); struct zone *zone =3D page_zone(page); =20 - if (__free_pages_prepare(page, order, fpi_flags)) + if (__free_pages_prepare(page, order, fpi_flags)) { + /* + * Only mark as zeroed if the page is actually still zero. + * kernel_poison_pages() overwrites with PAGE_POISON (0xAA) + * when CONFIG_PAGE_POISONING is enabled. With init_on_free, + * kernel_init_pages() re-zeroes after poisoning, so the + * page is still zero. + */ + if ((fpi_flags & FPI_ZEROED) && + (!page_poisoning_enabled_static() || want_init_on_free())) + __SetPageZeroed(page); free_one_page(zone, page, pfn, order, fpi_flags); + } } =20 void __meminit __free_pages_core(struct page *page, unsigned int order, @@ -3021,6 +3039,9 @@ static void __free_frozen_pages(struct page *page, un= signed int order, if (!__free_pages_prepare(page, order, fpi_flags)) return; =20 + if (fpi_flags & FPI_ZEROED) + __SetPageZeroed(page); + /* * We only track unmovable, reclaimable and movable on pcp lists. * Place ISOLATE pages on the isolated list because they are being @@ -3059,6 +3080,12 @@ void free_frozen_pages(struct page *page, unsigned i= nt order) __free_frozen_pages(page, order, FPI_NONE); } =20 +void free_frozen_pages_zeroed(struct page *page, unsigned int order) +{ + __free_frozen_pages(page, order, FPI_ZEROED); +} +EXPORT_SYMBOL(free_frozen_pages_zeroed); + void free_frozen_pages_nolock(struct page *page, unsigned int order) { __free_frozen_pages(page, order, FPI_TRYLOCK); --=20 MST From nobody Sun May 24 22:35:51 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 856E93CAE63 for ; Wed, 20 May 2026 22:23:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315812; cv=none; b=PqgEycJY38Ua5iOR3YAT1EODehdMvUpcyTCnaU5KIlrYlVHDRJvcugWC5qMZQM8aZ3giOybWjIsxO2mv9LAVYeu5IGH/imN2/2xNCY57CSuE4xuRc4vkEIj22pFXmzKrkEgoc684dbFC2xJKxumc93Mne+OgMyFoew08Lxo8jDA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315812; c=relaxed/simple; bh=V5Y93H57q2xi9ZVdVLl8OLD+jeqpAWNwQzgfVZDhJj0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=kbMEg8QKCG5iZLtqgiLp8myHWqErAj34t3AZQvOfStt2qhvjQVfHud88WK4Y+wre8mw68R/NCZyOdrrvKZ1QnfS9IW9L2nnUgeOxnyZ1RKVjwtejgDeK/ufmO7VI6n+mRFZnwwji95gSCB9J+V1YevLVFcEa2DCkqjXgRpNphAc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=JLtuYJ0i; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=FQgng3UN; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="JLtuYJ0i"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="FQgng3UN" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315810; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=lR7ppjC8ZhT95gw1LzQhYf3JteDpTHqAJsB6tR6PXLg=; b=JLtuYJ0iJGgxFXQuw8KgfeAuZ3HA6opjUsP5ho2xCL74ShJjUAdYUNH6VIOlZexT6eRgqz 0dYIvS26nV/MV5t25I7e0uUJqsfMHXHINqAZSnxZq8j6z7Y6iiOMIqly1Uh1Soh2oX9fZ1 xPm/pe4ewK4dzJH2oP56vUWOElFcfcw= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-447-pFek1Jy9NLWadqsLS1hMqA-1; Wed, 20 May 2026 18:23:29 -0400 X-MC-Unique: pFek1Jy9NLWadqsLS1hMqA-1 X-Mimecast-MFC-AGG-ID: pFek1Jy9NLWadqsLS1hMqA_1779315808 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-447f2ce5556so4363991f8f.2 for ; Wed, 20 May 2026 15:23:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315808; x=1779920608; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=lR7ppjC8ZhT95gw1LzQhYf3JteDpTHqAJsB6tR6PXLg=; b=FQgng3UNKN90tAadL7Zv5XnIUg9+loktcEMv5L4jR9JcPGjs2Fk+p2TlnsxVjs7bWz 0wClLTfb8HC7YqxqZQVllgiRwGnG4p0Di+X2jio6jMONKjUG4R1X1j+UEDVFyNBliZYL mcndjwQ0laas7Viene4FPsU5tKFdlOaPpVy0M5VAPZXOCy2aDC5+D7Qs70AqtIvnpd4w chweZ+FmGXxz8Em8iQImn5BPe44ses6LeUwvW5OPxRvpqX5l7uQGw2z9Tvjhp3YfLQVJ fYpgbi2Mll/agY7+IOoU9vB8sBEXyE4VzmbiEWU+SHlj8Mv7rQx/UfiZT8XV2uEU3urn kilg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315808; x=1779920608; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lR7ppjC8ZhT95gw1LzQhYf3JteDpTHqAJsB6tR6PXLg=; b=N0S0ADajAKwSif/JG+iHtFfwEhJ7gpSPvmC7IoRdnRxfJe1Q35eCtAeI+8UYzUxBW/ 4wz3wf63YjdAbNl71H54PgZkt9m1DMCiWABM98n6C2/3xpcyF6SKAyGngrT/508IiTvh cLTkGl0ux6jl66ctisQFl8wKqtHtb8TKlDgKnTTlTXNqAJU618o+65fjSKpIjDgIGBWq FNxu975i5NiInrvmtgxu2zzsFf/Xck85LwN1WZDAl5oN9rv03BOIHt2rZL1fRe9wVa5L 1H9oh9b8hz3V7PtbLQy0XcAfHjFu+azClAS11pN6uf4NYS4ghGC9x0Qx1UDSzdK4bJx8 B4Nw== X-Gm-Message-State: AOJu0Yz1wuQ3sbjZjDYA/5uwRIf7CpHj+hJPSbzN3tuqE2OSajqTDDbn VK7msBCPRCK2jakaCpZ9wg5eSdawV11C2JYTtS/2UzrjPMELHC/g+Xu9yxGmccXmGe/bGlRJlHo q7DIqqpTdTwBwc+M9uwxX9Z26xfhhm9+n3VWm/3pOUYmm4DPIW9h+4Ydl2W9QU++19Tp80NiR4i 37Gm70+5aVow8p/PORyiO3WaLDNI69RdowcYXvRnQspoM= X-Gm-Gg: Acq92OFx/ZvcGigLkDI4bSyyqrItcl8E2WIZLNVyRmwSRJg1qxPSgsX2JLAo/9T9clj +86h5mICTXedTfMySlHAEdPz5Qohf0QEmyiQEmJ4BESQk2/WBoznWJ8iSVDzOIVNzFNbYxJ2SEs ROR7pfie1t0ariAmqepJ4la3fT1QH6aqaIKx7aVAIYkKNFzO/0sFBjQOue0EEBhUm5iMczm01ZQ Aa3mN0LkHmknJ7ydXVBFmi9cLycgh7dp5JA0OuPWzwyTVRmQbSZTGQu0hWotCUUmO3/c8r4Ofoj klmpmTCgJgPd094Yib2uDc9JVqAXvqK2/DoxchczSRAsqSKSZvKCHRgRUIZF4AbE8XRYq6NjBwR +Id6uSUIGaDkRn0kKSt0diZKOIoC5LupaMynh6gMiXAI= X-Received: by 2002:a05:6000:2883:b0:43d:67d1:2022 with SMTP id ffacd0b85a97d-45ea38be3aemr338334f8f.19.1779315808260; Wed, 20 May 2026 15:23:28 -0700 (PDT) X-Received: by 2002:a05:6000:2883:b0:43d:67d1:2022 with SMTP id ffacd0b85a97d-45ea38be3aemr338257f8f.19.1779315807764; Wed, 20 May 2026 15:23:27 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45d9e768072sm56994075f8f.5.2026.05.20.15.23.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:23:27 -0700 (PDT) Date: Wed, 20 May 2026 18:23:23 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 34/37] mm: page_alloc: skip kernel_init_pages for FPI_ZEROED when safe Message-ID: <86f98d31a9b7989c699bad2e7f1fcc977a58d0a1.1779315441.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In __free_pages_prepare(), when FPI_ZEROED is set the page is already known to be zero. We can skip kernel_init_pages() if page poisoning is not enabled (because poison would overwrite the zeroes). This avoids redundant zeroing work when freeing pages that are already known to contain all zeros. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/page_alloc.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 399e2038dcd9..90a92be96ebf 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1447,7 +1447,14 @@ __always_inline bool __free_pages_prepare(struct pag= e *page, if (kasan_has_integrated_init()) init =3D false; } - if (init) + /* + * Skip redundant zeroing when the page is already known-zero + * (FPI_ZEROED) and page poisoning did not overwrite it. + * When page_poisoning is enabled, kernel_poison_pages above + * wrote PAGE_POISON (0xAA), so we must re-zero. + */ + if (init && !((fpi_flags & FPI_ZEROED) && + !page_poisoning_enabled_static())) kernel_init_pages(page, 1 << order); =20 /* --=20 MST From nobody Sun May 24 22:35:51 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3F0193BFE4C for ; Wed, 20 May 2026 22:23:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315818; cv=none; b=crb5RS+3k4N193neFTTHdbkmNFzhQ8Oga2lMFb73YLYO+b2iK5m9YyxNv5fs7MErir/NbiD35huuIwRdQp71Em7eEQ47ZRQI0g/lny4FAvxcCZ8iUIpy8De6yeTnDnG8fdNlx50ryQu+c2Rze8KlZWSFdhcBMhigr+xylbTk4/U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315818; c=relaxed/simple; bh=4jLzODooKNrRdgluzPWerRBde7vslo8ZvEbzyAfj6vs=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=uL+d8oEvig3i9wkN5HkFEJuzNUUF1U9cFJX+bUYaeNWODjz/vVMx/3Gq27is1J4yR2OvmM8GsPy/zJXWXmNVaOAMy8YmSqgbBq1+cNTY8MhC8jdcCroryAX2yWoMaXyUS1XMxUGp+NxTUxHUItjaYuxoMBsC/wuUni1V43LGbHI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=UDVJCnMe; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=i2zJUhn+; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="UDVJCnMe"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="i2zJUhn+" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315816; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=RfirkbM5TmZ61fE0cxqW40X8dDTC2oX5xU5kp1FvlUs=; b=UDVJCnMeqL9Y804VfU1aS1nPg5k86wTtG5afXRosp90JyOiM6lAMhXNUJSrTRZkUr3AXmn fpZTF+fSvHggmZ9x2N4/77gHpm4tCl4Ye9tjvoGmcD6OCLMHFWSSg2uy0Zrv7NMzTqheW1 Qh8jgka2cC0UZsbzqs7e3lgtr2L8CcE= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-385-bSqVn1dSOMqDzhN-b9Nhcg-1; Wed, 20 May 2026 18:23:34 -0400 X-MC-Unique: bSqVn1dSOMqDzhN-b9Nhcg-1 X-Mimecast-MFC-AGG-ID: bSqVn1dSOMqDzhN-b9Nhcg_1779315814 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-48fe44ce385so43116705e9.0 for ; Wed, 20 May 2026 15:23:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315813; x=1779920613; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=RfirkbM5TmZ61fE0cxqW40X8dDTC2oX5xU5kp1FvlUs=; b=i2zJUhn+h40vRXyy98eHacGk85O/p46nBPQpstfgGvKGv2V3iRrSJGexMGMpzmnyI0 7c1LNVOncCbbOdF3E6TNNJn/9okx1CG4+HHT4XttNN6of8KESMYyqTqQovwLGr56e2T7 7KLO8UJusFi2yXz8QIQ53UPj0EmYw2gfN5oPXBWX89hkKMSRLfFR9OJIj+OTZAgdQX1X P/N4VqJJy8IwlrCY2rXU20TLVs6KaM2itLOiRy1PFOB97RqYSBwsgX4/VZKIVknEUg1i ifg6E2hN/ymkIFxH2k+PEFSL+UrLYl3kSDJdVljQDd6/1+OW5p/2c28h3Cg9+OYyxyAN pHyw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315813; x=1779920613; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RfirkbM5TmZ61fE0cxqW40X8dDTC2oX5xU5kp1FvlUs=; b=EXFFGVWFz8rbLmMbF+FLshwYHik768VM1tI8simRZLwSLrLo5tgk/3nNn1IckIJOq7 IwPUKO95ytDnrtf91LR/CMA0JDSH/gbS/eiwzStEYv6PlTGjf0Mi/KOkV2k019LnZZlU 6IcFRW5czLqx3pSAqYaIRwFfnD4mR4zWftXEjt1XoP2oR5RkrIIXvTe2qB5Qczw8lsqz FDQPvGd/GjmH0qzQapJLVd22uSMee2lO3uafgBPzTGLXsdxX/KG9uyZfWvaIFREOxNgT 9ghKuKrfT/EWjf5GMLUKQ7mZ22ihJVWRs8QTwAfykrTxobcNIgSwXb4TVERlLkTSHJr8 zE3w== X-Gm-Message-State: AOJu0YxJna2Q0OpBJMJ7CJOxi71Lbu2KJFmCk/VCgsirPFZVCtasmgG7 LYC9sQ1TBVPpnYqvhtJBkEls7oovhdDZEcpQ/yI/H0Qf7tCyegp6Hj3W4blu/+YJBK+y1LiFNRF NUId3F+St3hNn7Eaq75iWpQo7SMiJIZLO/pDZ2iGYavahtkO+Bk+Nz1Rm201ghhs9tvPmJ3dXPP EKZW8aQe8309V8GanJJXEVfcxXy6d+f5pVVU4V4BBizVU= X-Gm-Gg: Acq92OG4rp0wAPjCsGUxE8iNwqw+WLyxD61hq7B4oRAWmLat/dH15dnkH3aViXw/A5f b+xVWcivIeQHe6K0XFIkZrIL+t/4l4gQW2keSM0By9GiU+FYgiYRePCsGiFUhG54erGOIF8KfmV xZuPT4DD8uYy3on0ziBm0K+yq1dyPgKIflG8TLVoKXsa5ewOiJi82wGB4YobT2aNrGI6Znaj1rK bMypztP9X5hNxWxmwA9u7s5iQF77101qKbHRKI3cnIZDpTDViHXsZbnVvPApwex0fe8sDL4mF2p BntgX+SbV+ylkfA+qsmNe8aU7tbY8+G5kiYXRVtv8NCYZNuk+qqSpAY2ILe8c86TcNiWQwMfRru cOu4eTQv7+Fv3iW4K9JgTqF2ZhEfT3XNRMzLhsitM2NU= X-Received: by 2002:a05:600c:848d:b0:48e:635a:18d7 with SMTP id 5b1f17b1804b1-49035fbaad0mr3145165e9.0.1779315813538; Wed, 20 May 2026 15:23:33 -0700 (PDT) X-Received: by 2002:a05:600c:848d:b0:48e:635a:18d7 with SMTP id 5b1f17b1804b1-49035fbaad0mr3144455e9.0.1779315812865; Wed, 20 May 2026 15:23:32 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49033d8e24bsm20927915e9.10.2026.05.20.15.23.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:23:32 -0700 (PDT) Date: Wed, 20 May 2026 18:23:27 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli , Qi Zheng , Shakeel Butt , Youngjun Park Subject: [PATCH v8 35/37] mm: add put_page_zeroed and folio_put_zeroed Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add put_page_zeroed() / folio_put_zeroed() for callers that hold a reference to a page known to be zeroed. If this drops the last reference, the zeroed hint is propagated to the buddy allocator. If someone else still holds a reference, the hint is simply lost - this is best-effort. This is useful for balloon drivers during deflation: the host has already zeroed the pages, and the balloon is typically the sole owner. But if the page happens to be shared, silently dropping the hint is safe and avoids the need for callers to check the refcount. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/mm.h | 13 +++++++++++++ mm/swap.c | 20 ++++++++++++++++++-- 2 files changed, 31 insertions(+), 2 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index d1e768dcda13..86638ffaad01 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1913,6 +1913,7 @@ static inline struct folio *virt_to_folio(const void = *x) } =20 void __folio_put(struct folio *folio); +void __folio_put_zeroed(struct folio *folio); =20 void split_page(struct page *page, unsigned int order); void folio_copy(struct folio *dst, struct folio *src); @@ -2090,6 +2091,18 @@ static inline void folio_put(struct folio *folio) __folio_put(folio); } =20 +/* Caller must be sole owner to guarantee page is still zero */ +static inline void folio_put_zeroed(struct folio *folio) +{ + if (folio_put_testzero(folio)) + __folio_put_zeroed(folio); +} + +static inline void put_page_zeroed(struct page *page) +{ + folio_put_zeroed(page_folio(page)); +} + /** * folio_put_refs - Reduce the reference count on a folio. * @folio: The folio. diff --git a/mm/swap.c b/mm/swap.c index 5cc44f0de987..ecec780172ad 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -94,13 +94,15 @@ static void page_cache_release(struct folio *folio) lruvec_unlock_irqrestore(lruvec, flags); } =20 -void __folio_put(struct folio *folio) +static void ___folio_put(struct folio *folio, bool zeroed) { + /* zeroed hint ignored for now, no current user */ if (unlikely(folio_is_zone_device(folio))) { free_zone_device_folio(folio); return; } =20 + /* zeroed hint ignored for now, no current user */ if (folio_test_hugetlb(folio)) { free_huge_folio(folio); return; @@ -109,10 +111,24 @@ void __folio_put(struct folio *folio) page_cache_release(folio); folio_unqueue_deferred_split(folio); mem_cgroup_uncharge(folio); - free_frozen_pages(&folio->page, folio_order(folio)); + if (zeroed) + free_frozen_pages_zeroed(&folio->page, folio_order(folio)); + else + free_frozen_pages(&folio->page, folio_order(folio)); +} + +void __folio_put(struct folio *folio) +{ + ___folio_put(folio, false); } EXPORT_SYMBOL(__folio_put); =20 +void __folio_put_zeroed(struct folio *folio) +{ + ___folio_put(folio, true); +} +EXPORT_SYMBOL(__folio_put_zeroed); + typedef void (*move_fn_t)(struct lruvec *lruvec, struct folio *folio); =20 static void lru_add(struct lruvec *lruvec, struct folio *folio) --=20 MST From nobody Sun May 24 22:35:51 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 84B643C3429 for ; Wed, 20 May 2026 22:23:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315824; cv=none; b=VteRumh1ZhSmCvhXCz/lvjE9hLdud1EJjpClI+reJr1HF4E8vsWqq49QTZj+1F0BpTbVPbe+Xk2mbw+x63hyfdQW+M/NaQns5xpJQNXKXwvVrDc7JijWUmleycO5OJhTCRZMVOfy9m3TaaBOlKHSTix4WikDHz3grWPeTA1rWNM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315824; c=relaxed/simple; bh=ERxCchXSRe8nQPvT0YItbiAmsZsxvWJbwQShAennuxg=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=ZJoqqwjLtVbHZPLHk65t/QfyZu86AIH/iF+OsMBiKUqWCJsfz436h9t5LWE4c4KV+XhXGP7WfJMDFdSQkt3CXBsE/rDaRofzflV+BoW695WCHfOb5pxYGnsjU8AHtkdxBqsmBV8pNdoLVlSls4dh05v2llE/1w8I5c5EAF6d1AU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=S+GMXOjl; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=Gpubcf+U; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="S+GMXOjl"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="Gpubcf+U" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315821; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=vswTwl3UmjKbzRe2lrt2y8QDjC9hyCS6oEWOwDRxv4k=; b=S+GMXOjlKhKCtNbIFe55Nebb6ug4yj7Uce4WKlnMGHZRiccSMsB76IfMWZUaRQOjk+EvW8 rdj8PymXjcciAMp+o+hawwrhD7xB5jaPxqlIh66GCyz41ZsW87BomPl6+MOrAWGqaCZMgI kyYapcO2Rn5wornOv2cLbq8EtsQ7P14= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-193-h7n6-qbnNXKguG2jqSaCPg-1; Wed, 20 May 2026 18:23:40 -0400 X-MC-Unique: h7n6-qbnNXKguG2jqSaCPg-1 X-Mimecast-MFC-AGG-ID: h7n6-qbnNXKguG2jqSaCPg_1779315819 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-45ea19f2564so299988f8f.2 for ; Wed, 20 May 2026 15:23:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315819; x=1779920619; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=vswTwl3UmjKbzRe2lrt2y8QDjC9hyCS6oEWOwDRxv4k=; b=Gpubcf+UKE1A0SDCqaAPDdQ5mGOgeKZoQcqN2fDh7JWj3ebTKPP1AWX0aIoLPWGI6O oNGgbKJNJ0qWt8iQ9FZ21yK7YY1Rtf0wHwdw/ZwQJL4jlWH8j4ZxbvDRA6QIDM7cQE/U 0W/2ctwcVrhg4mLqG8NWkJpECoQL82lcJXEsgeAZcVqUag0NkvLS5aEwAB1DmEGJWel2 MQnaixJOUPwlrUTIMPnp4VjI3wyT7sBRBjPU0FTN99fAp9Y+H+PrA+r4fxjLHfR9uGMn 2549MtJs3UFiYvhg33tJkn5W1Uly1dicFo5QKLMKrh4vpqWLwMlIgjbvXneP4USlcy3w U1Tg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315819; x=1779920619; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vswTwl3UmjKbzRe2lrt2y8QDjC9hyCS6oEWOwDRxv4k=; b=SF9dsxhSLFRALhxLnC/Omoy/uue/4a75b4+7fcs1KKPB9dAGhUJxBqvF3NVZ8hRmED 3F1MbEmkYguG130zCp1wMrC6wkbtyCyQmMwgz9txE9yom37WsqZnJfeLMTYShCjpr2I2 TScyIDsnEU1F2ld6k9NVFyzbeLuPJpZ6cnH0pfzylH2HAAmyTst1N+4y3qjKRmYIXYGR s1J0ZpqGG5/gwEgDAB6qA8TQAfA1AWtC0aWeH6wDONqUaAWVp5HICHNWiAzU15zS5xBV y8t8J6KaOEOX73UrXRxRXkvA9lnx9/qTBVKSri89cZm13ZH3Bqh1LB1DwE1Ti8Ce6z9u bTzQ== X-Gm-Message-State: AOJu0YyQIbr9yMTz4bexMLSKuAgb4zOsqs5nvXuBGam1eT/VTFq6nHRh FJpR1PGiClcTK1stiNpybiLX+N2MOm/pBLz/60Wei4vjjLp0KMslw+7nPYSPSU9sOUh98yPQVyW URdUuQEZMXphBM3WrUyMB+/OEP8v/1tVbFoHWVSoyU37xQ+PW29ih88sU1nCYqkQoR7KLrBRq1t KfLR5dkVX7s5RRhe7HPG5TCxGmzg3iD5AmNuxQwiXIPK0= X-Gm-Gg: Acq92OGVS07FZ/st8LgNG9e89VEFr+VbUr9e4LVoxhwba+17TxxTqddX3paYTUdG2w5 VMlb87jBvhGYFKHf/skijk1GuE/EZYZSkBTofEq/GOXek8FuDR3UvxojveRQWoBLMV/SdVNjapX WKRFZ4H06W0qIrukFMMpZTAydbeeHy1xI8vLrOLMShTR5NJCHldmefg7BGxsztOyJuZom4TPnXC P7D/ucymIfi9wH5M4AlSL3OkPU+vGGmkUk6ea8M0T6XxUrYpngu7qtB80njA4QPnIw38Px/D9FJ MNZTVI/IeDYR+8fK9j09esZoNUIOeGbEXoc4bwzA+pyWIiW+A4eMWxh/DnNFIPuU55MDyAxVfOy 2Gjk4eooDp31F+xsn+Da69T1rnYUHYRaHwNKtDu146Mrb8dIxLHpBrA== X-Received: by 2002:a05:6000:2912:b0:441:1e41:19c with SMTP id ffacd0b85a97d-45ea3aeb9b1mr279104f8f.20.1779315818574; Wed, 20 May 2026 15:23:38 -0700 (PDT) X-Received: by 2002:a05:6000:2912:b0:441:1e41:19c with SMTP id ffacd0b85a97d-45ea3aeb9b1mr279019f8f.20.1779315817859; Wed, 20 May 2026 15:23:37 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-45e7c22d8b7sm28194903f8f.6.2026.05.20.15.23.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:23:37 -0700 (PDT) Date: Wed, 20 May 2026 18:23:33 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 36/37] virtio_balloon: implement VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE Message-ID: <2536c13da81f68ba400221bc7370d68365de3817.1779315441.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When the device offers DEVICE_INIT_ON_INFLATE (bit 7), the device initializes inflated pages and returns a per-page bitmap indicating which pages were successfully initialized. The driver appends a device-writable bitmap buffer to each inflate descriptor chain via virtqueue_add_sgs. After the host acknowledges, the driver checks bitmap bits (bounded by used_len) and marks pages with SetPageZeroed. tell_host() returns used_len from virtqueue_get_buf(). Bitmap reads are bounded: fill_balloon() and virtballoon_migratepage() only trust bits within the used_len range. On deflate, release_pages_balloon checks PageZeroed per page and uses put_page_zeroed for pages the host initialized, propagating the zeroed hint to the buddy allocator. If inflate_vq has fewer than 2 descriptors, probe fails with -ENOSPC. If PAGE_POISON is negotiated with non-zero poison_val, the feature is cleared in validate(). See the virtio spec change: https://lore.kernel.org/all/9c69b992c3dd83dfef3db92cd86b2fd8a0730d48.177773= 1396.git.mst@redhat.com Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- drivers/virtio/virtio_balloon.c | 107 ++++++++++++++++++++++++---- include/uapi/linux/virtio_balloon.h | 1 + 2 files changed, 96 insertions(+), 12 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index bf1172ad5419..4abc7b3b991c 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -20,6 +20,7 @@ #include #include #include +#include =20 /* * Balloon device works in 4K page units. So each page is pointed to by @@ -122,6 +123,13 @@ struct virtio_balloon { struct virtqueue *reporting_vq; struct page_reporting_dev_info pr_dev_info; =20 + /* No DMA alignment needed: ACCESS_PLATFORM is cleared, + * so virtio bypasses the DMA API. If this ever changes, + * add ____dma_from_device_aligned here. + */ + /* Bitmap returned by host for DEVICE_INIT_ON_INFLATE */ + DECLARE_BITMAP(inflate_bitmap, VIRTIO_BALLOON_ARRAY_PFNS_MAX); + /* State for keeping the wakeup_source active while adjusting the balloon= */ spinlock_t wakeup_lock; bool processing_wakeup_event; @@ -182,22 +190,33 @@ static void balloon_ack(struct virtqueue *vq) wake_up(&vb->acked); } =20 -static void tell_host(struct virtio_balloon *vb, struct virtqueue *vq) +static unsigned int tell_host(struct virtio_balloon *vb, struct virtqueue = *vq) { - struct scatterlist sg; + struct scatterlist sg_out, sg_in; + struct scatterlist *sgs[] =3D { &sg_out, &sg_in }; unsigned int len; =20 - sg_init_one(&sg, vb->pfns, sizeof(vb->pfns[0]) * vb->num_pfns); + sg_init_one(&sg_out, vb->pfns, sizeof(vb->pfns[0]) * vb->num_pfns); =20 /* We made sure the vq is large enough so we should always * be able to add one buffer to an empty queue. */ - virtqueue_add_outbuf(vq, &sg, 1, vb, GFP_KERNEL); + if (vq =3D=3D vb->inflate_vq && + virtio_has_feature(vb->vdev, + VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE)) { + unsigned int bitmap_bytes; + + bitmap_bytes =3D DIV_ROUND_UP(vb->num_pfns, 8); + bitmap_zero(vb->inflate_bitmap, vb->num_pfns); + sg_init_one(&sg_in, vb->inflate_bitmap, bitmap_bytes); + virtqueue_add_sgs(vq, sgs, 1, 1, vb, GFP_KERNEL); + } else { + virtqueue_add_outbuf(vq, &sg_out, 1, vb, GFP_KERNEL); + } virtqueue_kick(vq); =20 - /* When host has read buffer, this completes via balloon_ack */ wait_event(vb->acked, virtqueue_get_buf(vq, &len)); - + return len; } =20 static int virtballoon_free_page_report(struct page_reporting_dev_info *pr= _dev_info, @@ -300,8 +319,37 @@ static unsigned int fill_balloon(struct virtio_balloon= *vb, size_t num) =20 num_allocated_pages =3D vb->num_pfns; /* Did we get any? */ - if (vb->num_pfns !=3D 0) - tell_host(vb, vb->inflate_vq); + if (vb->num_pfns !=3D 0) { + unsigned int used_len =3D tell_host(vb, vb->inflate_vq); + + if (virtio_has_feature(vb->vdev, + VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE)) { + unsigned int i; + unsigned int valid_bits =3D used_len * 8; + + for (i =3D 0; i < vb->num_pfns; + i +=3D VIRTIO_BALLOON_PAGES_PER_PAGE) { + unsigned int pfn, j; + bool zeroed =3D true; + + if (i + VIRTIO_BALLOON_PAGES_PER_PAGE > valid_bits) + break; + for (j =3D 0; j < VIRTIO_BALLOON_PAGES_PER_PAGE; j++) { + if (!test_bit_le(i + j, vb->inflate_bitmap)) { + zeroed =3D false; + break; + } + } + if (zeroed) { + pfn =3D virtio32_to_cpu(vb->vdev, + vb->pfns[i]); + __SetPageZeroed(pfn_to_page(pfn >> + (PAGE_SHIFT - + VIRTIO_BALLOON_PFN_SHIFT))); + } + } + } + } mutex_unlock(&vb->balloon_lock); =20 return num_allocated_pages; @@ -314,7 +362,12 @@ static void release_pages_balloon(struct virtio_balloo= n *vb, =20 list_for_each_entry_safe(page, next, pages, lru) { list_del(&page->lru); - put_page(page); /* balloon reference */ + if (PageZeroed(page)) { + __ClearPageZeroed(page); + put_page_zeroed(page); + } else { + put_page(page); + } } } =20 @@ -861,8 +914,27 @@ static int virtballoon_migratepage(struct balloon_dev_= info *vb_dev_info, /* balloon's page migration 1st step -- inflate "newpage" */ vb->num_pfns =3D VIRTIO_BALLOON_PAGES_PER_PAGE; set_page_pfns(vb, vb->pfns, newpage); - tell_host(vb, vb->inflate_vq); + { + unsigned int used_len =3D tell_host(vb, vb->inflate_vq); =20 + if (virtio_has_feature(vb->vdev, + VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE) && + used_len >=3D DIV_ROUND_UP(VIRTIO_BALLOON_PAGES_PER_PAGE, 8)) { + unsigned int j; + bool zeroed =3D true; + + for (j =3D 0; j < VIRTIO_BALLOON_PAGES_PER_PAGE; j++) { + if (!test_bit_le(j, vb->inflate_bitmap)) { + zeroed =3D false; + break; + } + } + if (zeroed) + __SetPageZeroed(newpage); + } + } + + __ClearPageZeroed(page); /* balloon's page migration 2nd step -- deflate "page" */ vb->num_pfns =3D VIRTIO_BALLOON_PAGES_PER_PAGE; set_page_pfns(vb, vb->pfns, page); @@ -966,6 +1038,12 @@ static int virtballoon_probe(struct virtio_device *vd= ev) if (err) goto out_free_vb; =20 + if (virtio_has_feature(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE) && + virtqueue_get_vring_size(vb->inflate_vq) < 2) { + err =3D -ENOSPC; + goto out_del_vqs; + } + if (!virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_DEFLATE_ON_OOM)) vb->vb_dev_info.adjust_managed_page_count =3D true; #ifdef CONFIG_BALLOON_MIGRATION @@ -1191,11 +1269,15 @@ static int virtballoon_validate(struct virtio_devic= e *vdev) =20 /* Device fills with poison_val, not zeros; disable zeroed hint */ if (virtio_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_POISON) && - !want_init_on_free()) + !want_init_on_free()) { __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED); + __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE); + } =20 - if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) + if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) { __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED); + __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE); + } /* * Balloon submits 1-2 sg entries max per buffer, virtqueue * sizes are 128+. Disable indirect descriptors to avoid @@ -1215,6 +1297,7 @@ static unsigned int features[] =3D { VIRTIO_BALLOON_F_PAGE_POISON, VIRTIO_BALLOON_F_REPORTING, VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED, + VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE, }; =20 static struct virtio_driver virtio_balloon_driver =3D { diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virti= o_balloon.h index 13074631f300..cbaf18e0b17c 100644 --- a/include/uapi/linux/virtio_balloon.h +++ b/include/uapi/linux/virtio_balloon.h @@ -38,6 +38,7 @@ #define VIRTIO_BALLOON_F_PAGE_POISON 4 /* Guest is using page poisoning */ #define VIRTIO_BALLOON_F_REPORTING 5 /* Page reporting virtqueue */ #define VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED 6 /* Device initializes repo= rted pages */ +#define VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE 7 /* Device initializes pa= ges on inflate */ =20 /* Size of a PFN in the balloon interface. */ #define VIRTIO_BALLOON_PFN_SHIFT 12 --=20 MST From nobody Sun May 24 22:35:51 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2AC4E3BF66C for ; Wed, 20 May 2026 22:23:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315828; cv=none; b=BOZVckSveaFuvNKTuanX5mv0bMqz7bLA4SHTf7JP21o24UpBzx42UN7IUwc8zo+Zak3TNwYq4sYxFRT+1R3bGk7PVGXYTc+IR9uDGaDyPMLmw0ZrNYX/qozfY/YErWWQjIy5belprHqESL11McU2s0XDj+d7vbuFU1+uZKqA5+4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1779315828; c=relaxed/simple; bh=p/E5hRnS5VZWIkTtV/Cpgy5sbv8MptFrLHFQQR1g+04=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=mjwzplxtmROK7KtDINVflNMnaamQHQVFkg4eECJIlbHHYttNxoWuskxYrOoRGr0SZwtbM3mMv9GoLtXIcFmKDWbNzk3w3KNQwTOevnH/m8BtEfGrnlz89LIRMfvHSZXCV+lyqu++dPyeKIZ0SAqcDxUmEETbaZFr+DUifkP4VQc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=DkmY1d18; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=bdFCwt9H; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="DkmY1d18"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="bdFCwt9H" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1779315826; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=K72Es9G8n7K8ecgoDN9zfD/eO+Zd/boK65D6pcX9pF4=; b=DkmY1d18mGktbE6OtmtMa3JI6U5xYmmgO3Vxv+5M7D3aAkb4sJT/XgaLTVikIdeRTX4tE/ VHu8kbdYVm9D7Pi0lN0nd+h3vzXshIbqJywA55+CS0h80rsZpRmHGVWgI86OgKrs7+CbYF oioGWAB/SPHL5BUyjgdpR/0iksm4VIo= Received: from mail-wm1-f70.google.com (mail-wm1-f70.google.com [209.85.128.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-604-sXBNJ1zQO0OrH2zWx_SyqQ-1; Wed, 20 May 2026 18:23:44 -0400 X-MC-Unique: sXBNJ1zQO0OrH2zWx_SyqQ-1 X-Mimecast-MFC-AGG-ID: sXBNJ1zQO0OrH2zWx_SyqQ_1779315823 Received: by mail-wm1-f70.google.com with SMTP id 5b1f17b1804b1-48e89faa62eso32669535e9.1 for ; Wed, 20 May 2026 15:23:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1779315823; x=1779920623; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=K72Es9G8n7K8ecgoDN9zfD/eO+Zd/boK65D6pcX9pF4=; b=bdFCwt9H0yN9elfKoVcfkNJx4vS5+yLhpfmVdqQVI52uHkcYlI3Xrp6vBBVBCYCbYP hcXHvAgfN/UYMHJkPG9z6U3zDuW8VVOrOeLmO1FO7nSA7DPmTI5f+gZN0Pt1wooE+psv ipeMTX3QfHRDaEqKNRa0dJnYZXH71E3AaMOKRrxVTJ6/8+m6nF6tId3aJgmao3e6HAlS ABinoqKNqv9KM2e8Y9aJDqmEmM2Z6VjspwYbaRD+FERAl6UfUqYq9keTT6demswBnvA7 25MAcQw1CGhECoWunOBlpGi4a9EaS7EU8Rr13zFICdZ9bmVr9sZq1ZScECq2Zp7uzqmP xQAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1779315823; x=1779920623; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=K72Es9G8n7K8ecgoDN9zfD/eO+Zd/boK65D6pcX9pF4=; b=KJNe2aflfnGU7x9eSLoDflPRQI0t5uls6lSrcQQMD24xozAy9nbdBSUgKiYmXhb2Pa w4I6tn0n6EthkVNoOwSyNpMlNg1hO/mxoHY99jp5vGeET+Fws7dsNLFpZUe8ZaMo8RQ+ p1gFpuEhlqPOF9lAK98+kv1OEFBLmY8Weq0nJoKrALnEng0hSjriGHfC/SXp0Z1Fe7aM pqtNIixqK6jpQgY0fbymgIMsUX770LdAcaONnKwBDCmzMcRX5BbN4HNFMUZ+ouqCfzk8 pexUggawb6nMOW7tmMXVAnmPZzqwEmZg42xgNgZ2EEykWPn35Bu5uXOdD87vU/EZRq9c JXuA== X-Gm-Message-State: AOJu0Yx4ZUk2UTOJMMcyv5GA9pWuE9Cpes7gIm3S3rzQ8A706i0geA5v OTLnLgN2ThlvT0SpHKtKPUQXNsfYN53kEMDcumSa6SRm0+zkORXiZTFEa6RnJg5h257gTLgoAMB EdF2NQZlLWMLTRuGh//hfYd1O1nswaqqpKeo8//6/YCXWcq3YZZJOrzBujtsczA4dsbXr1EaTyQ wWewsoEcqS0jbb/IS+66t0ZWi6Q8g7iQ7nN0CLYpmCNqU= X-Gm-Gg: Acq92OGwyvLPTGpIZKzZ6QhmZip3hHwVkyUWmmLN77oD17glzz0q5FcNLxC/b/z25U6 nWWpTIYrn0VJtNYfundJkFJNho5/ug9j8FzyoxqP2mnYVzJ+9OqR1MhKsLIouLzJvT7KYKNiz/P 3/8EMsBW415k4QlkHpcl+S5DNeAWcywceA9hLV6WvleI3Z0qwoeiUi1V/F3jsziGN2lZd6eFly8 I3pWr9KXl1ux0G+4gZKCeUL1d5B/r34M7QM8lJ+ju3BP8+dskaisUPsY0dsaY9fPTdRxm50Ulop jG73uGETTccZC05Iav6upqGrV3zMks4F8mtrEBUH7GBHdVkTBmSZNaf6ArxT3hqSVi8RfseLk2x u7uwIvPYywzfopRfB8R104v7W7wvTNhmvM6YXEfY6XUg= X-Received: by 2002:a05:600c:858d:b0:48f:f7cc:f703 with SMTP id 5b1f17b1804b1-490360e4c2amr1777965e9.30.1779315823447; Wed, 20 May 2026 15:23:43 -0700 (PDT) X-Received: by 2002:a05:600c:858d:b0:48f:f7cc:f703 with SMTP id 5b1f17b1804b1-490360e4c2amr1777275e9.30.1779315822889; Wed, 20 May 2026 15:23:42 -0700 (PDT) Received: from redhat.com (IGLD-80-230-25-45.inter.net.il. [80.230.25.45]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-49033d58838sm19711575e9.11.2026.05.20.15.23.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 May 2026 15:23:42 -0700 (PDT) Date: Wed, 20 May 2026 18:23:38 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v8 37/37] mm: balloon: use put_page_zeroed for zeroed balloon pages Message-ID: <61af3e2e956952715b26b51be4afa4ca573ab631.1779315441.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When a balloon page marked PageZeroed is freed during migration, use put_page_zeroed() to propagate the zeroed hint to the buddy allocator. Previously the hint was silently lost via plain put_page(). Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/balloon.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/mm/balloon.c b/mm/balloon.c index 96a8f1e20bc6..6c9dd8ab0c5d 100644 --- a/mm/balloon.c +++ b/mm/balloon.c @@ -324,7 +324,15 @@ static int balloon_page_migrate(struct page *newpage, = struct page *page, balloon_page_finalize(page); spin_unlock_irqrestore(&balloon_pages_lock, flags); =20 - put_page(page); + if (PageZeroed(page)) { + /* Atomic to serialize with memory_failure's + * TestSetPageHWPoison; not under zone->lock here. + */ + ClearPageZeroed(page); + put_page_zeroed(page); + } else { + put_page(page); + } =20 return 0; } --=20 MST