From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 335943AFD0E for ; Mon, 8 Jun 2026 08:34:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907672; cv=none; b=D8KgaqLTIdvJSVA6lsWWBeywq1gdl5RmOTP6kFVVULT9Nh8a7uJXn5Dlt0eHSCGhcw21Edo3g2JV3wabMllIeyLE+K9MHf7vGyoUaYzOjGKxpsfMpRaWj9yMdz+/oPBu6IJ4nLwst6rfewe5j5VSxt7Frk0UKPuQyh2Y19jD7vA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907672; c=relaxed/simple; bh=yI1iGkmiSzMFJtnw1lJVP5u1P/QKkpQHXLglYhfy84A=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=PdvisSWZ61nkJR63RttRl+aYIO+lrmctZb7HwVfxpNvikByRR4dLFK2UWCuv+ccVbH+ILY+BLwuYM83h/JPGFQ9ONYaXtvnu6aCYIPqpNxdk7HbT+v/6K+tJ/hToE76PUSmDOifhH2Fg5hoTspMxsZSm9dB46THyqEphO782G14= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=FAGS5DA2; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=I6eaH8ug; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="FAGS5DA2"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="I6eaH8ug" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907668; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=315e7bSessxyz662R8s4IU7FTBDlj+mVbEZvVNtQrTU=; b=FAGS5DA2XHd3Prj76/6miRgXLK53dgOAlHzeT8idn2hg8gE/oJ4X93blTaCj28WdYwFkh9 7Ix0GBMQRDP9NwWboKxqAcXoLLDMq+VLdO1PkAHnZI86qGRKQ1yaxt5vviSqGHGaGan4LN pxXpkZihIUdifKYqWSWtzXYXQgOQugo= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-447-vK-HXtqYNvuIoiT2ykcWkQ-1; Mon, 08 Jun 2026 04:34:27 -0400 X-MC-Unique: vK-HXtqYNvuIoiT2ykcWkQ-1 X-Mimecast-MFC-AGG-ID: vK-HXtqYNvuIoiT2ykcWkQ_1780907666 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-490b8adf8b8so38328095e9.0 for ; Mon, 08 Jun 2026 01:34:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907666; x=1781512466; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=315e7bSessxyz662R8s4IU7FTBDlj+mVbEZvVNtQrTU=; b=I6eaH8ugbcvhTTmtioaCWe2bW5JuEPayuf3U6evSSlATOM/3eoqcYuRFseiCr+6upl JC7Jpk6dbEGZ2icwu77uMzSJwJB85Np7aAjt2fZhKHwVsBEOv9bY0DUOLMXkr/M31g2F E2Vl+DUM98ciDJ7OhAx9aLWqCp+ax8/nlJC5VN8uPTOyTwXzgJHF3hHvJyYOv9FCl7Ft RWaP5dMDACu4og4pG4Vs6YoczTEVwfhXwnB1xgCjURLKvSF9PAOcK9p3KySuViDuKFQl e0vkKVmhFfc0PZ75UJEOZ/pXS7gl7HqBBjgpp9tAJzRhYzC1B4EL61anwu0uuFitV9xB KDnA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907666; x=1781512466; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=315e7bSessxyz662R8s4IU7FTBDlj+mVbEZvVNtQrTU=; b=osHqJPGgxnPP8lyZyjkGypii9/xgcWDrUSt3Kgun/u0eqt8iHT4pYl/k42spARSyxi qaGwHL3fZxUBldCGWlTTEPx643E0iGtgWFhhrx/7J14pVNPMvYk0kCU9xfYP5NsoJuam boQ4DSPeTQ7tRMT+5b+cbkO1JvET85mgjl6nMSEjrqtxX/PKAXZ/Hde/8eZP982zi+Sl f3FKsNgPruwvczjbxl5Zn46WpF5crYiicX91nEH3dwbKfzlkmh7THQ2WVr+R8kEyd6Jp UUqpF8HYIHbIdI+pvKmx/fvKbbRoEc3hjylJg4BXEOQ3n/v18eixvKGGJ4q0t41SIMKB gmBQ== X-Gm-Message-State: AOJu0Yxh+u5gtqIkI/2LDLCFsCjeiChzs2w8mqLEb4GV+JrrYBevp/bi 4PBfq3v4gDZvi6zfbmSrb5T6pvZSJuEy9YeLZWZlJmQvmCrX2klBJ2lGIRFb0NNgKWQW0dwYucs xknG//MOS7RGxcM7FPcRcfZE/szqoD9+Kw0XUE8gpN7HdzPjv0HXWLA2InOh2aexAiEhKh+oOE2 Hk3G2vJkYYI8CacKdiPIcPkBURmpTFFNESXeS7dC90oZE= X-Gm-Gg: Acq92OHy+bEl56cTwKY9fll6jJeVLhy/v/x2YaPibjhutgkMGvKmYDQvFGe1FLfzt/S Q/vVMJGIVNQMsde7oDeYrM+V3f9iEpX0tqBm0aP2PNZ2uyxhCYF3rOMc+uJfSxsmIhqjiQOf6qD JBYwp7+2Z7Qzrl3U4J8lnvZ2QaaZOvKC6XaeAU08sQ8wudV1Do9xbp+LkjxxIO8zBacIsLuqQJX jdUPtGNauqiYMQaavay3/I15wN1iiBvRQMI0v0HwUefdKk3KexPm28GGW7jQi9flO/VdZxFeccn w/GGzj2EZVvz5YHuss7CJrSzOdfOe5U1kMra/l+PozVUCCFlLT1iHPsl1KVWf13B+Lo5qgBy/Kg vr+vsd8PhVLGIHaaZUGAsgCc= X-Received: by 2002:a05:600c:4708:b0:490:bd66:e522 with SMTP id 5b1f17b1804b1-490c26038d2mr235216635e9.29.1780907665500; Mon, 08 Jun 2026 01:34:25 -0700 (PDT) X-Received: by 2002:a05:600c:4708:b0:490:bd66:e522 with SMTP id 5b1f17b1804b1-490c26038d2mr235213495e9.29.1780907663108; Mon, 08 Jun 2026 01:34:23 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-490bc3d66c8sm356368295e9.10.2026.06.08.01.34.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:34:22 -0700 (PDT) Date: Mon, 8 Jun 2026 04:34:06 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 01/37] mm: mempolicy: fix interleave index calculation Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The NUMA interleave index was computed as two separate terms: *ilx +=3D vma->vm_pgoff >> order; *ilx +=3D (addr - vma->vm_start) >> (PAGE_SHIFT + order); This has two problems: 1. When vm_start is not aligned to the folio size, the subtraction before the shift lets low bits affect the result via borrows. 2. For file-backed VMAs, shifting vm_pgoff and the VMA offset independently loses carries between them, giving wrong chunk indices when vm_pgoff is not aligned to order. Combine into a single expression that adds vm_pgoff and the page-granularity VMA offset first, then shifts once: *ilx +=3D (vma->vm_pgoff + (addr >> PAGE_SHIFT) - (vma->vm_start >> PAGE_SHIFT)) >> order; For anonymous VMAs, vm_pgoff equals vm_start >> PAGE_SHIFT, so the vm_pgoff and vm_start terms cancel and the result reduces to addr >> (PAGE_SHIFT + order), same as before. For file-backed VMAs, the sum vm_pgoff + (addr >> PAGE_SHIFT) - (vm_start >> PAGE_SHIFT) gives the file page offset of addr. Shifting by order gives the correct file chunk index. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Reviewed-by: Gregory Price --- mm/mempolicy.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 4e4421b22b59..d139b074a599 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -2048,8 +2048,9 @@ struct mempolicy *get_vma_policy(struct vm_area_struc= t *vma, pol =3D get_task_policy(current); if (pol->mode =3D=3D MPOL_INTERLEAVE || pol->mode =3D=3D MPOL_WEIGHTED_INTERLEAVE) { - *ilx +=3D vma->vm_pgoff >> order; - *ilx +=3D (addr - vma->vm_start) >> (PAGE_SHIFT + order); + *ilx +=3D (vma->vm_pgoff + + (addr >> PAGE_SHIFT) - + (vma->vm_start >> PAGE_SHIFT)) >> order; } return pol; } --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8D40C3B14B8 for ; Mon, 8 Jun 2026 08:34:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907688; cv=none; b=bayFQjU2hcUZZ1XD9IDnpDgwVPNPC0X2qd9wBXr+upSeTB5iwvfMEkxTkRrzX5spIDcJAhRLa/1XzEtOHPUCQRJMgxk/++VBcrzcwdifG6tWtWvH0Ep/sLlWFG+0smmsfmxUTHZj4vS0oG0+3KB0mAumkIOhLtaeOodrjM2dER4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907688; c=relaxed/simple; bh=lgTFAAybKpw8Lv7QpTHpNFrj++hl60eW9AsRt64i/XU=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=StF3/i7dhcO1gVcIB5Z+QIxIqfHWYceNuYQ2MlZgL9acK8FJVR8iEVY6E3SVK2c1QB/H5GVzczv/my8321975OytUIK+lPepRT9Ja3XbfjV8Xv6jylUiJrkSurZrvbctbB0wk06H1oztfmnze+HLy7c2pGLi0UZI1Dg2Pll8lvg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=jAWWEee+; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=rdEO8HWN; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="jAWWEee+"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="rdEO8HWN" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907682; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=NUdBC37dWtUhtr3poKTRLbvv+0k/F3lQxPVXzGOdb1o=; b=jAWWEee+7tFInba1r5hF5OKU0EepwJpjJLcF+GGPX5njoWRYjSbaesl5CKyueLL0rgNLCK t6JancLe4Kkj9eXujurciXEWtAnuhQ1KfQkaslsIxC4XWyMErZ512tJhZzwm0XIt1yUMVf Ei4GjXmq0nobvareNynCKIWyZFKSHtc= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-115-7p0B6uTiOc6TO2pDmsGHtw-1; Mon, 08 Jun 2026 04:34:41 -0400 X-MC-Unique: 7p0B6uTiOc6TO2pDmsGHtw-1 X-Mimecast-MFC-AGG-ID: 7p0B6uTiOc6TO2pDmsGHtw_1780907680 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-45efa2f7009so2516387f8f.3 for ; Mon, 08 Jun 2026 01:34:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907680; x=1781512480; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=NUdBC37dWtUhtr3poKTRLbvv+0k/F3lQxPVXzGOdb1o=; b=rdEO8HWNFGyjyaLic7fZxa9O1MBGjkXyfukFzmEbQelHaMRJ4/EN8pmndsR11bNXtb daP/UaOr+HucqpBa/ySrfQSXa3sLbzvvvRx/KY2Qzv751DLrhTLkld6mZvM8m4FhyH2Z BXRV1AtGtEe9jO03h2spPf6oDZ85S0ZpvnhqSQDNvOhgif/EaSAMPxupkdp4Ga8bbCix SuuM00XsPD+s003P3VuldR9xjSJ6bI0xkZ7zxehEBbKs5klZ8DjBIRd1xjKMtAghlXrC zNXuQUpi9a4fSgvCcv8zwIZ4DYpXB2TGh3zeX9zBtXRhMx1M+UdJW60sRC/2EgxPQO28 PcdQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907680; x=1781512480; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=NUdBC37dWtUhtr3poKTRLbvv+0k/F3lQxPVXzGOdb1o=; b=tJ5Ua4PZcmMWpdf6uewrNR8xZN4I+FgcCUj4/DPpBPkT6/JBhwQrRqL98sfL69yFyf L0fHvRI+jCIJqRdsoQ0LHnHS3/isNXCKClIRKL91N/6iDu8wqUdc61BQcJPdGGwHrzAL 9EqWqjWKL1Au7CYufd4fGvVy2KKwOoBKJ+gn/qzHAtavvj7O/L9Gw2vOgYsAXKFqTENe R2737fLuGqQVyGXUNlUel/Tr3EB96Sx+uFgrWLb1PLqgfOdipqjgpr4Sqk7laKEFvCm0 yotp89e6jMocZFd5Nb1TX3FKqRu47JetV2i88b9j0p/oG/k8jgOuj3VYmWcaPXS6VM37 czLw== X-Gm-Message-State: AOJu0YwcZTr7YiBR/yZfoECAbCoqg8cHXeBM2KWZs6Wn/w2U5oxsaRR4 qrgGub9ANk1PiCIi9bg4J70YJJS4cqc9+7Aa84lnfkFMm2z0v8+M0nEi/XuJmfcwU0lp7wZiZcm MOKzAZ5n2BHaTdWToTSDx+UobsAeteZx0MEh44qJZkqXMa6MHGsrjkB9wbut0Oo/m4GRjIzp0eh FjveqZ/6pJnbKoA4W0igVTRDXcw8BRilfja5sVEa+vN6U= X-Gm-Gg: Acq92OEgqJxuDS6K/fuVsNgaBEw3y4eY2RKDLiy2B+S+lIIqtdYWLpdK+51N09goX0k mMNRoVQqEeGoseswT1D9l3kv0FHMqCDyGGHW+nvnNdsorMNVvFXeqJC/QFKJ3ORVwf34qP5o3Ru M1ckZvFz0rL9WiFHUnlIp77RQ0v5jvO/+7JYVJcbHi4vkdz3brR2739hYTK/zSdeuSPsfgN7/Ku YIcOZt99/XDCm6IEfV2TyfW0BEkVhnOA20IZsRImkgLUXt3Tfq6dHZK4nSP++WzsUMkvGnCxx12 OYsGm/rmgCenuOkjugkgOSfqN91zqIa2eDkk/0HZHDsHG0SBk/GefNLuBdHXUPxqrqTJ5RF9yKf u2VK/BF2STojiyoVuxqfN+fs= X-Received: by 2002:a05:600c:154d:b0:490:44eb:c1ec with SMTP id 5b1f17b1804b1-490c260f3cdmr236254925e9.27.1780907679688; Mon, 08 Jun 2026 01:34:39 -0700 (PDT) X-Received: by 2002:a05:600c:154d:b0:490:44eb:c1ec with SMTP id 5b1f17b1804b1-490c260f3cdmr236253855e9.27.1780907679052; Mon, 08 Jun 2026 01:34:39 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-490bc3d6c65sm366951975e9.12.2026.06.08.01.34.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:34:38 -0700 (PDT) Date: Mon, 8 Jun 2026 04:34:23 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli , Miaohe Lin Subject: [PATCH v10 02/37] mm: memory-failure: serialize TestSetPageHWPoison with zone->lock Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" TestSetPageHWPoison() is called without zone->lock, so its atomic update to page->flags can race with non-atomic flag operations that run under zone->lock in the buddy allocator. In particular, __free_pages_prepare() does: page->flags.f &=3D ~PAGE_FLAGS_CHECK_AT_PREP; This non-atomic read-modify-write, while correctly excluding __PG_HWPOISON from the mask, can still lose a concurrent TestSetPageHWPoison if the read happens before the poison bit is set and the write happens after. Follow-up patches in this series add similar non-atomic flag operations as well. Fix by acquiring zone->lock around TestSetPageHWPoison and around ClearPageHWPoison in the retry path. This serializes with all buddy flag manipulation. The cost is negligible: one lock/unlock in an extremely rare path (hardware memory errors). Note: SetPageHWPoison and TestClearPageHWPoison calls elsewhere in this file operate on pages already removed from the buddy allocator or on non-buddy pages (DAX, hugetlb), so they do not need zone->lock protection. Acked-by: Miaohe Lin Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/memory-failure.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/mm/memory-failure.c b/mm/memory-failure.c index ee42d4361309..3880486028a1 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -2348,6 +2348,8 @@ int memory_failure(unsigned long pfn, int flags) unsigned long page_flags; bool retry =3D true; int hugetlb =3D 0; + struct zone *zone; + unsigned long mf_flags; =20 if (!sysctl_memory_failure_recovery) panic("Memory failure on page %lx", pfn); @@ -2390,7 +2392,11 @@ int memory_failure(unsigned long pfn, int flags) if (hugetlb) goto unlock_mutex; =20 + /* Serialize with non-atomic buddy flag operations */ + zone =3D page_zone(p); + spin_lock_irqsave(&zone->lock, mf_flags); if (TestSetPageHWPoison(p)) { + spin_unlock_irqrestore(&zone->lock, mf_flags); res =3D -EHWPOISON; if (flags & MF_ACTION_REQUIRED) res =3D kill_accessing_process(current, pfn, flags); @@ -2399,6 +2405,7 @@ int memory_failure(unsigned long pfn, int flags) action_result(pfn, MF_MSG_ALREADY_POISONED, MF_FAILED); goto unlock_mutex; } + spin_unlock_irqrestore(&zone->lock, mf_flags); =20 /* * We need/can do nothing about count=3D0 pages. @@ -2420,7 +2427,10 @@ int memory_failure(unsigned long pfn, int flags) } else { /* We lost the race, try again */ if (retry) { + /* Serialize with non-atomic buddy flag operations */ + spin_lock_irqsave(&zone->lock, mf_flags); ClearPageHWPoison(p); + spin_unlock_irqrestore(&zone->lock, mf_flags); retry =3D false; goto try_again; } --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B1A0B3AFD16 for ; Mon, 8 Jun 2026 08:34:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907713; cv=none; b=ZsOAbDa3pa8IavMc8UHV33vpH8UZQq8XO9NW7awFmduFE5VjwQTmEwuuy9hei8PVgCS+wYICvKitrPiYud1lVn7N2/XgM+tYyawb0q21K1XvCwbCJ1mrQUurfp8fhnjfhCflzKeYwskuYbbhThVsQnMW3kjm6DkDj0dMWYk2pQU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907713; c=relaxed/simple; bh=9FkU2pdAQ7jw+NV4TdycQajDAfwAm44W9k92o71ADO8=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=IVJ4rjPteArRNCeAjOSbEme1ks60TgfhOl/b6NV4tfOyFKoUNN4vWHgnUW4lPrmEoeiB+Cnp0TyBnLf8Mz6x0hkngNcZ5Nys5O6dKwWuVElOQu5c1/a0ueUbnT4gMeNNB+NSy8Ai3y4vPtfwd1x2bDNrgglfV7lV10+9n84ClVs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=GlvlVEzX; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=nAurjDSW; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="GlvlVEzX"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="nAurjDSW" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907693; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=BG5vFIZuWkBfCpPnsH5jvRyFO8VXn5agYH6VILwADzw=; b=GlvlVEzXF2/x4tTmzwSSU3dKXk6YHAkjBM2z8Zw5T/FHg8lI01TnFdXVANRpFzkIHpNJi8 GAyxBUdFxcEeY/sAxJ3uQDEZcsgJkgeUUVS6X0mQpXs4xbMWo2h97zIxB95f3Lw2432TOs 9mwSM31G34DNB7h3qL2G5NZdlLAOfdg= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-453-kKiZ3_74P9aFzL32niqs0w-1; Mon, 08 Jun 2026 04:34:52 -0400 X-MC-Unique: kKiZ3_74P9aFzL32niqs0w-1 X-Mimecast-MFC-AGG-ID: kKiZ3_74P9aFzL32niqs0w_1780907691 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-490addd42bbso25944205e9.3 for ; Mon, 08 Jun 2026 01:34:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907691; x=1781512491; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=BG5vFIZuWkBfCpPnsH5jvRyFO8VXn5agYH6VILwADzw=; b=nAurjDSWRz+WK/aOMw/hzwHjkaasX+pkmUWaQzcnRdAMPROglSVafIBWNbwUuvrkKO kSZsbREA8xHUQT1jviyCSq1cQBuG84L1jwsV7XK5BsFbWKio+Tjr7+qrNetG1nuAHugr VVAOLZvpJ5ruBH0OMKRlMbJxTlii3lcuvyRO7NrzwnMSubxQOdXNH8xreW1/nGne7bia uRSalCiQ8vBtqaKw2hT1hvXmiuvlVyyKkXzvlTIxOjjCvc0Zf+siGmjMSr/ch70fYipJ pGwKzFzoZk7cjxeb2tNLn+X41l3vyqK/8f9mH4/3sDjzWF7deuWADRKFFeVcMBNLzJkG eDbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907691; x=1781512491; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BG5vFIZuWkBfCpPnsH5jvRyFO8VXn5agYH6VILwADzw=; b=Soe/QSChRmYBOKuntHh9HfRorciInpfS5aQNN2hcDSA88DZ/3dUAwLNjCHKdjXyEsW WjSJX5YVFAybjzRiR9S5xMrNiYpht4mv3ybeTRVGpT1kX6ea+eiw/DMYSNbsglDQB6+/ QYxlTSatrURbT6wyW4ocnPjgOSnQPzU0+CHThdztr8tE2NqIaTmOs+nP0WB4L8OBiOyS /oVQm//oPtoemJ06bKA/Kgnu2ePvYRm/78l30rMsDkkhGfpMLhFnpHLmXdM60VKsl1gC 6QQ5Jm42cV91HFlN5Gf4pjgMm+8yIMTGEI8s/7kjFS+LBbrrMqXR1igYem8BcaWBWXb1 mlAQ== X-Gm-Message-State: AOJu0YwtoC1DrE0hQCagsMYUEF4kIXQCQE/nD+SJqXJbi0JfndwF2rBG wJrOfKndeiSkIIlJXAteLdc7c+MC8oqQEMJZ88GQHM8harjk8QnoU4+3es4XdcR9luho82ilSy1 WoJogceP3XVi5JmiTaUVmPysCPvHuCRcn/s17nX5LApwODFq7nbJ9KjQ7izKQeuSIB9wVq9cTAb 17x62rCkzUrZfjvmpACbqbjeNRZTNp0lkDqtNBNRIhElQ= X-Gm-Gg: Acq92OFEvPsQOs0VcVUn/SgjpWG0toS+GO9MzehGXSg1p5Jy5EJ3eucm1W7uVE/+joo ngNtYUwn1X11vulkDqNUoyUnFdZc6A7DCvKuOS0GAK1pQ0aMoqcvF9uEgmwgViqyQ9fo0csAkub HLeqUBcxiRthPCVYFVI3HOY3BvN35U/qlsHuQEOk2rp5hasM920xKjEywBnvGU1/uk2N+qWJQjR H5Ii3Q3IVSQCbvdNp9cC0X9J2fyvhpm5Hu0EBig3ppLMDjrysPatPft229cbt5vcnwgexYAquFF UIhk3piXcjnH82eSmlUTrV40zZYesEsMc2HVW0oyWASdL+Nn/mXidPPf5KCdoVX0Gcrg0UAc4CV 3tzfftjfGnPPPQ9qwPWnxWyI= X-Received: by 2002:a05:600c:a011:b0:490:58ef:cea7 with SMTP id 5b1f17b1804b1-490c25ee058mr246872965e9.22.1780907690796; Mon, 08 Jun 2026 01:34:50 -0700 (PDT) X-Received: by 2002:a05:600c:a011:b0:490:58ef:cea7 with SMTP id 5b1f17b1804b1-490c25ee058mr246870995e9.22.1780907690081; Mon, 08 Jun 2026 01:34:50 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4601f2ed944sm52009994f8f.13.2026.06.08.01.34.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:34:49 -0700 (PDT) Date: Mon, 8 Jun 2026 04:34:39 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 03/37] mm: page_alloc: propagate PageReported flag across buddy splits Message-ID: <1a407d39a9ebd95e9de67e44f9ad2db37c9f31e9.1780906288.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When a reported free page is split via expand() to satisfy a smaller allocation, the sub-pages placed back on the free lists lose the PageReported flag. This means they will be unnecessarily re-reported to the hypervisor in the next reporting cycle, wasting work. While I was unable to quantify the performance difference, it is an obvious waste, even if small. Propagate the PageReported flag to sub-pages during expand(), both in page_del_and_expand() and try_to_claim_block(), so split_large_buddy() also propagates PageReported via a bool parameter: the caller saves PageReported before del_page_from_free_list() clears it, then passes the saved value. The flag is set after __free_one_page() with a PageBuddy check, matching the page_reporting_drain() pattern. Free-path callers pass false (freshly freed pages are never reported). that they are recognized as already-reported. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/page_alloc.c | 32 +++++++++++++++++++++++++------- 1 file changed, 25 insertions(+), 7 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index d49c254174da..8dae5b3f5876 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1502,7 +1502,8 @@ static void free_pcppages_bulk(struct zone *zone, int= count, =20 /* Split a multi-block free page into its individual pageblocks. */ static void split_large_buddy(struct zone *zone, struct page *page, - unsigned long pfn, int order, fpi_t fpi) + unsigned long pfn, int order, fpi_t fpi, + bool reported) { unsigned long end =3D pfn + (1 << order); =20 @@ -1517,6 +1518,8 @@ static void split_large_buddy(struct zone *zone, stru= ct page *page, int mt =3D get_pfnblock_migratetype(page, pfn); =20 __free_one_page(page, pfn, zone, order, mt, fpi); + if (reported && PageBuddy(page) && buddy_order(page) =3D=3D order) + __SetPageReported(page); pfn +=3D 1 << order; if (pfn =3D=3D end) break; @@ -1559,11 +1562,12 @@ static void free_one_page(struct zone *zone, struct= page *page, llist_for_each_entry_safe(p, tmp, llnode, pcp_llist) { unsigned int p_order =3D p->private; =20 - split_large_buddy(zone, p, page_to_pfn(p), p_order, fpi_flags); + split_large_buddy(zone, p, page_to_pfn(p), p_order, + fpi_flags, false); __count_vm_events(PGFREE, 1 << p_order); } } - split_large_buddy(zone, page, pfn, order, fpi_flags); + split_large_buddy(zone, page, pfn, order, fpi_flags, false); spin_unlock_irqrestore(&zone->lock, flags); =20 __count_vm_events(PGFREE, 1 << order); @@ -1694,7 +1698,7 @@ struct page *__pageblock_pfn_to_page(unsigned long st= art_pfn, * -- nyc */ static inline unsigned int expand(struct zone *zone, struct page *page, in= t low, - int high, int migratetype) + int high, int migratetype, bool reported) { unsigned int size =3D 1 << high; unsigned int nr_added =3D 0; @@ -1716,6 +1720,15 @@ static inline unsigned int expand(struct zone *zone,= struct page *page, int low, __add_to_free_list(&page[size], zone, high, migratetype, false); set_buddy_order(&page[size], high); nr_added +=3D size; + + /* + * The parent page has been reported to the host. The + * sub-pages are part of the same reported block, so mark + * them reported too. This avoids re-reporting pages that + * the host already knows about. + */ + if (reported) + __SetPageReported(&page[size]); } =20 return nr_added; @@ -1726,9 +1739,10 @@ static __always_inline void page_del_and_expand(stru= ct zone *zone, int high, int migratetype) { int nr_pages =3D 1 << high; + bool was_reported =3D page_reported(page); =20 __del_page_from_free_list(page, zone, high, migratetype); - nr_pages -=3D expand(zone, page, low, high, migratetype); + nr_pages -=3D expand(zone, page, low, high, migratetype, was_reported); account_freepages(zone, -nr_pages, migratetype); } =20 @@ -2116,11 +2130,13 @@ static bool __move_freepages_block_isolate(struct z= one *zone, /* We're a part of a larger buddy */ if (PageBuddy(buddy) && buddy_order(buddy) > pageblock_order) { int order =3D buddy_order(buddy); + bool reported =3D PageReported(buddy); =20 del_page_from_free_list(buddy, zone, order, get_pfnblock_migratetype(buddy, buddy_pfn)); toggle_pageblock_isolate(page, isolate); - split_large_buddy(zone, buddy, buddy_pfn, order, FPI_NONE); + split_large_buddy(zone, buddy, buddy_pfn, order, FPI_NONE, + reported); return true; } =20 @@ -2283,10 +2299,12 @@ try_to_claim_block(struct zone *zone, struct page *= page, /* Take ownership for orders >=3D pageblock_order */ if (current_order >=3D pageblock_order) { unsigned int nr_added; + bool was_reported =3D page_reported(page); =20 del_page_from_free_list(page, zone, current_order, block_type); change_pageblock_range(page, current_order, start_type); - nr_added =3D expand(zone, page, order, current_order, start_type); + nr_added =3D expand(zone, page, order, current_order, start_type, + was_reported); account_freepages(zone, nr_added, start_type); return page; } --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CDED23B27C8 for ; Mon, 8 Jun 2026 08:35:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907711; cv=none; b=iWllMcifAGdiNGH+HNv8ElcXgJQKtd6oCowzSOUI/lSJbmEnwTEo7ZzcxYmx2RUyP3bXN5T2XYi+w3Mb6yrjfQEfv4Yx3HZ691wGFu3cFGykhwo99Ge1CIG4uz0gaLz0yIAlVo5WU/AwLcABQhv3Wg7knnlrfAWH148lXxi+eO4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907711; c=relaxed/simple; bh=VM/bNOZTb0RCzRWmIDoLOJCCyLB90lnxDaW3yIFW1Xo=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=VQUr56cPC0qQ2AoO4cvIHyFQOVCz50abbDv65KbT6qX5OXOM7229IcuZfihDmj7ZHilLkCohxPMiCbgpvaIg8LydYVIAi23/VPMqkMSuX8ecFsHEizgGx+E6FElIpKkW6UZO+plVG+UuwFWvUtiE+sipZP+XHLj2cS4SCFVnyD0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=DKXKgou2; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=aAk4RBCI; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="DKXKgou2"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="aAk4RBCI" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907700; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=/LdI09yfbzRCb1twey6OkL3QcRw2oFWT9IrdWUT5qiU=; b=DKXKgou2i5/+AhjWscN6hari9eLSz26/2j577Yo/+hTye0MmUV+Ma+fZcKnqQXFDju9ab6 Uis5gkGT7rfKzwF/F1SB71Wn/fdt5j0VV5O8I7p4w/+qYNShs/nDEid+Quvvym7CGVcGln RwA2U2T7TZTW6bZPWYFTPlk+N2KOqvA= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-711-TCUFKFj6OTG7jp5jazpGJw-1; Mon, 08 Jun 2026 04:34:59 -0400 X-MC-Unique: TCUFKFj6OTG7jp5jazpGJw-1 X-Mimecast-MFC-AGG-ID: TCUFKFj6OTG7jp5jazpGJw_1780907698 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-45eef10d5ebso1943832f8f.0 for ; Mon, 08 Jun 2026 01:34:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907698; x=1781512498; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=/LdI09yfbzRCb1twey6OkL3QcRw2oFWT9IrdWUT5qiU=; b=aAk4RBCIfwGe/c8fGRu0T/BSvIKrbXuxUrWyWd3ympr/bVnCAkIKjSo7/bHd8IrRgj lDdXg83wYVlpqsDo98ii0TbZGgc7OUQ9jV3NJEJ03+tY80cwJIMITU2tdog9uLd0r3rK G+AmY5vOCN8yEXQVuosGyE7TQNSCcIPA9m5XZTC85+Gbd02rpOQbBVE/LRWp2gRyto4/ +zEj/4VOcg9Cs/bj9QokLWu2VYclGkRU+/hX+O8hAia+vw0sxg4WFH0RkWRAQA4oAwDn pzxZMzGD7P9LdmkPYCoQiQmlnTqVQAbJS3Zc6mHbFDlePGnz1HFLJDF4NIcNH9RYIsIP ZOtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907698; x=1781512498; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/LdI09yfbzRCb1twey6OkL3QcRw2oFWT9IrdWUT5qiU=; b=athxAV6sAczVR2BuO5a2QRi7b+kY+Z/ISdqk4z9b1lZpPUjurmQK8QbTNg9O8DDikb 2heKmPshh5mEltloWBBcIK/PBYz8VM+dSNzLKYnQHNsYtns9Bbj0CSND975QAJ266j6+ jmbLA0MnVcKlN3oSqu2PP1v/DBVJAUjoUxfD/s8hNep5lvibsfM5GRs7Wn1T2cqu7FLe U5Ph9Zxvcp43B3xVszNLUCukUF7esSAhyFYIBJRNr24uFroH1votcA2YvE7DKZKijzHA elHDn1SREbFiGg4G1763hXRxNbH4asJufN4hZszQtPw/8pAbwJQcrdfZgNx37J78pnOX 2d7g== X-Gm-Message-State: AOJu0YwxJMlxbQks4VkV1lV9g2uD9fWJ2D6TwYXjNzckSwXrRSWrpAUo myGwbPMVmqz4psl2jShHeiLgxwE+QUjxN+3yfdBPkqtQTw4ZMdRrVqanzlasUMxKTUp+i9yohej TUhemcBLOzf2UyqsQXHA5XRA5wRPM0AHM695sYqe3XuE97joIPQ4AaA87J4Vr6vxdn06L+U6TCm eJfevus570YOa6Rrv0Zm7ODP3ZIvo9qFaWRmDPAMl01oY= X-Gm-Gg: Acq92OG9KWUVLrb73129zXhiQ5KEYSXiXIZ7gCe4Vzbbjhj0kg3t7Q6Zd5Icp+kLiMU ASpV7ty9teP10tES0wUu+d3SJ+2BVQ+95D2NoT10yKxPit1qx2c95NjhbexPwb/qnoKzZaE4EDt w9NCq+PaEE8yk+laYgP92hQAyybMksJ34nS02gV0nDixaNUb2ocwWtMv8gqbXMZaZNuxv/Rw5ia WWh5nt3IhyXRkHDOag5fsUc2hL6wMC562+HQENLct5k39Eo/EhRzOjKvXtpEPcMuINJmWcav+g1 bE3k9xxGpqvyVOZiuwEkevuYpXEFnoOxBSGO9kPjok7nOe0zXxeKwvy8xxSXt0IbJT1sVYbzOn+ 8aU5K/GncySR0HTixp85TLwo= X-Received: by 2002:a5d:660c:0:b0:460:1695:89be with SMTP id ffacd0b85a97d-460302f1a80mr16882993f8f.17.1780907698219; Mon, 08 Jun 2026 01:34:58 -0700 (PDT) X-Received: by 2002:a5d:660c:0:b0:460:1695:89be with SMTP id ffacd0b85a97d-460302f1a80mr16882899f8f.17.1780907697659; Mon, 08 Jun 2026 01:34:57 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4601f2dcae2sm53186372f8f.6.2026.06.08.01.34.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:34:57 -0700 (PDT) Date: Mon, 8 Jun 2026 04:34:50 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 04/37] mm: page_reporting: allow driver to set batch capacity Message-ID: <776d350bffd86c5776485ffc4f24e668df36367e.1780906288.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a capacity field to page_reporting_dev_info so drivers can control the maximum number of pages per report batch. This is useful when the driver needs to reserve virtqueue descriptors for metadata (e.g., a bitmap buffer) alongside the page buffers. The value is capped at PAGE_REPORTING_CAPACITY. If unset (0), defaults to PAGE_REPORTING_CAPACITY. The virtio_balloon driver sets capacity to the reporting virtqueue size, letting page_reporting adapt to whatever the device provides. Note: capacity need not be a power of 2. The DIV_ROUND_UP in page_reporting_cycle() uses integer division, which the compiler handles efficiently. Rounding would halve the batch size for non-power-of-2 virtqueue sizes, wasting capacity. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- drivers/virtio/virtio_balloon.c | 5 +---- include/linux/page_reporting.h | 3 +++ mm/page_reporting.c | 25 ++++++++++++++----------- 3 files changed, 18 insertions(+), 15 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index f6c2dff33f8a..6a1a610c2cb1 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -1017,10 +1017,6 @@ static int virtballoon_probe(struct virtio_device *v= dev) unsigned int capacity; =20 capacity =3D virtqueue_get_vring_size(vb->reporting_vq); - if (capacity < PAGE_REPORTING_CAPACITY) { - err =3D -ENOSPC; - goto out_unregister_oom; - } =20 vb->pr_dev_info.order =3D PAGE_REPORTING_ORDER_UNSPECIFIED; =20 @@ -1041,6 +1037,7 @@ static int virtballoon_probe(struct virtio_device *vd= ev) vb->pr_dev_info.order =3D 5; #endif =20 + vb->pr_dev_info.capacity =3D capacity; err =3D page_reporting_register(&vb->pr_dev_info); if (err) goto out_unregister_oom; diff --git a/include/linux/page_reporting.h b/include/linux/page_reporting.h index 9d4ca5c218a0..5ab5be02fa15 100644 --- a/include/linux/page_reporting.h +++ b/include/linux/page_reporting.h @@ -22,6 +22,9 @@ struct page_reporting_dev_info { =20 /* Minimal order of page reporting */ unsigned int order; + + /* Max pages per report batch (default PAGE_REPORTING_CAPACITY) */ + unsigned int capacity; }; =20 /* Tear-down and bring-up for page reporting devices */ diff --git a/mm/page_reporting.c b/mm/page_reporting.c index 7418f2e500bb..5b6b17f67131 100644 --- a/mm/page_reporting.c +++ b/mm/page_reporting.c @@ -174,10 +174,10 @@ page_reporting_cycle(struct page_reporting_dev_info *= prdev, struct zone *zone, * list processed. This should result in us reporting all pages on * an idle system in about 30 seconds. * - * The division here should be cheap since PAGE_REPORTING_CAPACITY - * should always be a power of 2. + * The division here uses integer division; capacity need + * not be a power of 2. */ - budget =3D DIV_ROUND_UP(area->nr_free, PAGE_REPORTING_CAPACITY * 16); + budget =3D DIV_ROUND_UP(area->nr_free, prdev->capacity * 16); =20 /* loop through free list adding unreported pages to sg list */ list_for_each_entry_safe(page, next, list, lru) { @@ -222,10 +222,10 @@ page_reporting_cycle(struct page_reporting_dev_info *= prdev, struct zone *zone, spin_unlock_irq(&zone->lock); =20 /* begin processing pages in local list */ - err =3D prdev->report(prdev, sgl, PAGE_REPORTING_CAPACITY); + err =3D prdev->report(prdev, sgl, prdev->capacity); =20 /* reset offset since the full list was reported */ - *offset =3D PAGE_REPORTING_CAPACITY; + *offset =3D prdev->capacity; =20 /* update budget to reflect call to report function */ budget--; @@ -234,7 +234,7 @@ page_reporting_cycle(struct page_reporting_dev_info *pr= dev, struct zone *zone, spin_lock_irq(&zone->lock); =20 /* flush reported pages from the sg list */ - page_reporting_drain(prdev, sgl, PAGE_REPORTING_CAPACITY, !err); + page_reporting_drain(prdev, sgl, prdev->capacity, !err); =20 /* * Reset next to first entry, the old next isn't valid @@ -260,13 +260,13 @@ static int page_reporting_process_zone(struct page_reporting_dev_info *prdev, struct scatterlist *sgl, struct zone *zone) { - unsigned int order, mt, leftover, offset =3D PAGE_REPORTING_CAPACITY; + unsigned int order, mt, leftover, offset =3D prdev->capacity; unsigned long watermark; int err =3D 0; =20 /* Generate minimum watermark to be able to guarantee progress */ watermark =3D low_wmark_pages(zone) + - (PAGE_REPORTING_CAPACITY << page_reporting_order); + (prdev->capacity << page_reporting_order); =20 /* * Cancel request if insufficient free memory or if we failed @@ -290,7 +290,7 @@ page_reporting_process_zone(struct page_reporting_dev_i= nfo *prdev, } =20 /* report the leftover pages before going idle */ - leftover =3D PAGE_REPORTING_CAPACITY - offset; + leftover =3D prdev->capacity - offset; if (leftover) { sgl =3D &sgl[offset]; err =3D prdev->report(prdev, sgl, leftover); @@ -322,11 +322,11 @@ static void page_reporting_process(struct work_struct= *work) atomic_set(&prdev->state, state); =20 /* allocate scatterlist to store pages being reported on */ - sgl =3D kmalloc_objs(*sgl, PAGE_REPORTING_CAPACITY); + sgl =3D kmalloc_objs(*sgl, prdev->capacity); if (!sgl) goto err_out; =20 - sg_init_table(sgl, PAGE_REPORTING_CAPACITY); + sg_init_table(sgl, prdev->capacity); =20 for_each_zone(zone) { err =3D page_reporting_process_zone(prdev, sgl, zone); @@ -377,6 +377,9 @@ int page_reporting_register(struct page_reporting_dev_i= nfo *prdev) page_reporting_order =3D pageblock_order; } =20 + if (!prdev->capacity || prdev->capacity > PAGE_REPORTING_CAPACITY) + prdev->capacity =3D PAGE_REPORTING_CAPACITY; + /* initialize state and work structures */ atomic_set(&prdev->state, PAGE_REPORTING_IDLE); INIT_DELAYED_WORK(&prdev->work, &page_reporting_process); --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 010213B14D4 for ; Mon, 8 Jun 2026 08:35:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907721; cv=none; b=Y98u3udRY9TGWsIr3vWYN0fW42NQAowvjakwL5X6Z9nVpFU/DVsH3z4rsQKMbZQrVjCI5rqWnSeMaCLL0iIkGpyAoRiy8EU3bVXzzy1SYCJjrtej0mDW+2hWgRk/gKJfIvhmFl8wmW81COHHHnJggsvSOOMF0lnTL7dvF8bevI0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907721; c=relaxed/simple; bh=Sfj5gt6MvIEmG0HbHoKsMbRpZjmrEHXu8jQM2CsCIY8=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=TEu27HMlyX1QwoOwEotoEcoKoOv/JjeM5smq0V1F5GQuxuAjMuoQNgMoeQB2od/jHX4YE4z/JgQYUozSKQgI1wb2XfgAHe06YP9igtJMMetJPQlUmJ+LFAgdhYJxxpRiVes3W5Dlu8eGKZuZ0tDowlfiyHYshUtupd4F6TkxBhA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Ne8mE17G; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=DtFkyhEq; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Ne8mE17G"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="DtFkyhEq" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907713; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=lrvJesBjI5N3mqdjRc1dMOyxu185onYyp291JtEzMjE=; b=Ne8mE17GRmRUhWRufmS4hY3aVr1coZHmySuo9SoFUCl/Es4iN17qAv3thKqH+pwTpHkcRE El8YyTKDPzQ8t/g+i30+vQizMgRbeyefU1GAxQCrfi/YLD+vDC408ekxpK7Fx8Ht0j+2nt 0MeN6fexUDwP8eg2cblLUpX/7ptwKVM= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-682-7Cl64OnJNWO51GHKAQzAHg-1; Mon, 08 Jun 2026 04:35:06 -0400 X-MC-Unique: 7Cl64OnJNWO51GHKAQzAHg-1 X-Mimecast-MFC-AGG-ID: 7Cl64OnJNWO51GHKAQzAHg_1780907706 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-490ae3bcf4cso19713905e9.3 for ; Mon, 08 Jun 2026 01:35:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907706; x=1781512506; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=lrvJesBjI5N3mqdjRc1dMOyxu185onYyp291JtEzMjE=; b=DtFkyhEqHDXiBFPWvDGNOBtNPn0yD/hRecDfh9xweMFy1TQOFixMuYWN0Wi3R9a5Qi uSPLNQVu1pGFpzc1gqpFEKMPzUdWl7NQOXItBD3nOR7Y1GLnLuIR6QVa41sIb2PqM7bk JbLIzqLld2Qh1NYu0hnnsBVlkkmQSBu+5LN3sZmvuIbM0nrVGQ0Xytp7R0JMwroCj/FA 3QbiBorBCO4f1Ul206J1Jh6+MIvV3xLU8k1eF9qtmFPRNAnVieKUkuXZHW6WakeGKgeo JtcI6CCZGuG+V2yBoPP3Wter4jrtfme7hZ02yBCE1bHTa7TPc6/Dqg0bWiuQ7LOXqYLy NTTg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907706; x=1781512506; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lrvJesBjI5N3mqdjRc1dMOyxu185onYyp291JtEzMjE=; b=lFcUTnsUSvvsPBPO9LdmsNE9ojzTyUStfI0psyCknbhzqwfbkgtzLZqHQZ0K9/Hh/W IAxWDIO9oWAwGjZt6dKbAjGx/1/WNrLS1bMBZBsCk5GU3ocizc2brDeyN8HR/4jVZ3Cu YBWRMzmn2/hatAfAQBC0k485pqGqk3F6HSNodFSNpDYm1pnHuE9LH2E1AJT+HVWa6yLg fJpZn9c0iuSN+Q3gcnnKSveJm29VVGkLhGz7/KqdCTRtJBELPHVBFaZB6z7dw3fXjhJB ltyCeirDbHcCrCGfndz8bFEKbXE+T48U2/DhBOU+Y61x/lBuJ+vAZKDKNS5nlmusqKRA ocTA== X-Gm-Message-State: AOJu0YxKU3GiIg60R/9fh8xMsXncMIMwtZbdzK8/UOGEH0y15jaRaQtH Clp1x2R/5e8niVPwpNbGCMc0HWu04tAa7ooK5eSsDYoh8wZTf/LWwx55K4oA1nWwnKYw09Ux5tK OY/QfZfFzbkn+YJjXpPLAU+VP0eukl6CTyL2z0SnIg9dMOdMwoIPsOu1WRa+DQXBfABl5D7kgSV kE6MjnPhfLcDbqezjM8GMT6IuDbxNTreW/IkF7S4+iLKI= X-Gm-Gg: Acq92OHFXQPKVDsAXug8mQADrMjPumIf4vZ1K0Bha52Ecj40KgJ9wu4RJSD7wieJDYZ qkDGWsJG4UQPKsc9D7erRUyywbEMjlvwetXw7qejsYmxR75phIyeAM5KXZvGXmkG7sESNXx60IT BUPmRESi3oFpd8dFg1iZ54kQjrvxZxvrML1xX4yIB0Au26JoQvbtm2AkRZH8r4PaFk/G0SOETKn yAnmGbBFKYrbo0rhkhgZ7fzjvwYFppeQrHyPcRXgnBz8aY7LO2HdQOvx4ZWnpL097UVM8xa8TYT HBfpeu39dZ5SSMHL6xfbKjshWc3Ze6aE28Bp8lI/MqJd/rW6Sz55zAUdFuklB2cASewO1o8xr+C OgITe7r6PYxTaAnXK4COp5go= X-Received: by 2002:a05:600c:6087:b0:48a:8b02:ae91 with SMTP id 5b1f17b1804b1-490c25b0231mr252123705e9.11.1780907705569; Mon, 08 Jun 2026 01:35:05 -0700 (PDT) X-Received: by 2002:a05:600c:6087:b0:48a:8b02:ae91 with SMTP id 5b1f17b1804b1-490c25b0231mr252122505e9.11.1780907704887; Mon, 08 Jun 2026 01:35:04 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-490bc391aaasm433938045e9.1.2026.06.08.01.34.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:35:04 -0700 (PDT) Date: Mon, 8 Jun 2026 04:34:57 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 05/37] mm: hugetlb: remove dead alloc_hugetlb_folio stub Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Remove the !CONFIG_HUGETLB_PAGE stub for alloc_hugetlb_folio(). The stub is dead code: all callers are in mm/hugetlb.c (CONFIG_HUGETLB_PAGE) or fs/hugetlbfs/inode.c (CONFIG_HUGETLBFS), and CONFIG_HUGETLB_PAGE is def_bool HUGETLBFS with nothing selecting it independently. The stub is also broken: it returns NULL, but all callers check IS_ERR(folio), so a NULL return would not be caught and would crash on the subsequent folio dereference. Remove it now since follow-up patches change the signature of alloc_hugetlb_folio and would otherwise need to update the broken stub too. Reviewed-by: Gregory Price Assisted-by: Claude:claude-opus-4-6 Reviewed-by: Dev Jain Signed-off-by: Michael S. Tsirkin Reviewed-by: Lorenzo Stoakes --- include/linux/hugetlb.h | 7 ------- 1 file changed, 7 deletions(-) diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 5957bc25efa8..1f7ae6609e51 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -1123,13 +1123,6 @@ static inline void wait_for_freed_hugetlb_folios(voi= d) { } =20 -static inline struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, - unsigned long addr, - bool cow_from_owner) -{ - return NULL; -} - static inline struct folio * alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_nid, nodemask_t *nmask, gfp_t gfp_mask) --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A5D803B19BC for ; Mon, 8 Jun 2026 08:35:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907727; cv=none; b=qX2hDV63FijWBx4XhXIvLnD2z5gm5d4LB3eoYwkkKgyuP1De3JrgYR5qDm1B4LBMl57wKwJxAx+ByoolCB7cpLFpTHA0cb24z/U3HKOUqzShoFGPlfwxqO1vvBCeb86dfVENpY2D/95S4Qszg7+oEwaynat4NjVzAAj310qKo14= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907727; c=relaxed/simple; bh=WqVTVeEY5sjLY71Wx43CJrvkZxe/1qqCM34phrgRwA4=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=fNyba5WA6U7cCujjCCUA29VLwO1//1RnJyW7FL4vIpWPWlnhCXWq/H4gz5W6/lrTv+J7QLnw55lzd3Vu0b7S9o20Bbvz6xj4fo93amhM54mnqKz0++9fs6TxLMz56N8Enct5qhfVseOpiZb7ZS+4XwINBCzRcaEpoFExvzyuvL0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=HGRFKGFA; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=t1JKegBe; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="HGRFKGFA"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="t1JKegBe" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907720; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=QTsyEmHKpfx3QY2xerCkWdwHjor0zxZF5cs8sc/X2mI=; b=HGRFKGFApOfq18NwQKmtYNxiBhOw8LdW+qFXo6OnpwQTfwaK1gjhjMEw/OlpT0twdhnAf3 0nlcbzBrQs2xYhsmpng9/Mwg4ca3t/BOrftwsRJHJtFfrkJ3KDMVcbxWUdojRq7YpWrOXs fkc4+lO/Po4M+7dyoumIh4LyJc16QFo= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-588-xr4kFDYxOpKc1BNPlM_UYQ-1; Mon, 08 Jun 2026 04:35:19 -0400 X-MC-Unique: xr4kFDYxOpKc1BNPlM_UYQ-1 X-Mimecast-MFC-AGG-ID: xr4kFDYxOpKc1BNPlM_UYQ_1780907718 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-490a767b782so32495355e9.2 for ; Mon, 08 Jun 2026 01:35:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907718; x=1781512518; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=QTsyEmHKpfx3QY2xerCkWdwHjor0zxZF5cs8sc/X2mI=; b=t1JKegBe+ebRICH45pTeO42HMLB40kPtX2a+N+ZS+wmGfdGD/S2xEwDZnkAA61ofew VrRtu01qOT0BJUOtgEQgLjBnC0teddePzF/AarFpPNU/oIHb+zt+iBHwLTB65quft9wU HO1ESw71c2iYLN6RWHHtaeHGozvZfPICOYQTlfOTEJ+7yHTcy5Z4Av9UfOxRafrhgiqj 4V52J7M8+KrjYxDHRffJXTToc/HNVLcWO86rjQBSh/f58x/hd6On/pJHd1iehnCOGl0a UI8caLQSIcbkKZOpZ2q/0Nhece5nZZReCquTDI8c3hgacomERafKQekPcSEQD3eqCJiz OYbQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907718; x=1781512518; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QTsyEmHKpfx3QY2xerCkWdwHjor0zxZF5cs8sc/X2mI=; b=fTpAcNzOqzGQg8CLqzdnwe5jaSrrakV8Jf8gtculDe18hxcieDWhEkkz4pQQY59rUn 7nc0tXuUhHSZroIeQ+s4FyBg8jhWPPMbvFqHOXkJiJ7XMI0F6VK/1kMJaZwAfPEHwNTn CbgCrUYw9jW82UKQflWO40Dopqu+Ja/AWwfjIQBj5+wbN1ULmB/8t1fqUh02ErKdLnDH wJg7MGJ2oOXx3itNghxL33kmimmFxoML2dO5lXgcBLCb4UTTUVVB8d0X6F6mfhp99sOq 27kSn7LO+ka2zqRvNcR2RhRbPTbDQBLF7lfF4ApF7TrUG3AVvl1qkmcjxjRLC2ZJs9a7 ZP/w== X-Gm-Message-State: AOJu0Yy2Aif3Y92AifYH2YVfpmWiJHg6/cvDRYx7DldwAfi2c1Nlsh7b 21hd/sy7vhc1ApGpXTdOSZbtWb3eb3XmOhDcfTpBRYeje4ZCHdZZZm6IOuttHurNqlb3M6O0/n+ ac/WTzHxcI/ZRlGIq4EL6nIan0rJUvXT0PJ4SuzMTLQtkFEv7xX0xCsbuHwLnJTD7fanapk82Pa P8u7DHaaOT05Q5LtgCTdqVTQfYVIPiGuwow4qiMZ/r/78= X-Gm-Gg: Acq92OEBvzJhhZZhrfwGhOapDAUiQDikICPqUT/iSAYUCTa4vfHpABcfNMMq+eEEv87 eeC3A4wFHP6vNWM4XHSAWnjR4UmuIEKhQb8Oz773mO41gjNGZFE7mpRqltjFXbpeHtetU6kn+T+ 6yYt7EGDl2+pNmA0ognBFgq+/qS5LxbLMdqlo9VcfZMuYrM6HGojcdOkqtwHCmZnkOyik2Zuswp +1DFmAWmGh/V0fcJF/GffykDiuC8ljdt1F2jSWfz0dTn8CIHHEMT9eEElapS8MILWBpFDneBeDe kW+DwdGrhF+uHejTkUTiYNGzwF4ChF+JQvVXcZluAJoicG1W4uvceEGC7JgQceWpCTzPIP+2nMJ 4saOoABny3hURLVot/Xc06Dg= X-Received: by 2002:a05:600c:19d3:b0:490:48df:2793 with SMTP id 5b1f17b1804b1-490c25fa410mr242691395e9.26.1780907718085; Mon, 08 Jun 2026 01:35:18 -0700 (PDT) X-Received: by 2002:a05:600c:19d3:b0:490:48df:2793 with SMTP id 5b1f17b1804b1-490c25fa410mr242690165e9.26.1780907717394; Mon, 08 Jun 2026 01:35:17 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-490c2d37edbsm282225125e9.2.2026.06.08.01.35.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:35:16 -0700 (PDT) Date: Mon, 8 Jun 2026 04:35:05 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 06/37] mm: move vma_alloc_folio_noprof to page_alloc.c Message-ID: <5a3034dc7b3d5ea1edcc8b9baff63c980b7ae434.1780906288.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move vma_alloc_folio_noprof() from an inline in gfp.h (for !NUMA) and mempolicy.c (for NUMA) to page_alloc.c. This prepares for a subsequent patch that will thread user_addr through the allocator: having vma_alloc_folio_noprof in page_alloc.c means user_addr can be passed to the internal allocation path without changing public API signatures or duplicating plumbing in both gfp.h and mempolicy.c. The !NUMA path gains the VM_DROPPABLE -> __GFP_NOWARN check that the NUMA path already had. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- include/linux/gfp.h | 9 ++------- mm/mempolicy.c | 32 -------------------------------- mm/page_alloc.c | 43 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 45 insertions(+), 39 deletions(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index 51ef13ed756e..7ccbda35b9ad 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -318,13 +318,13 @@ static inline struct page *alloc_pages_node_noprof(in= t nid, gfp_t gfp_mask, =20 #define alloc_pages_node(...) alloc_hooks(alloc_pages_node_noprof(__VA_= ARGS__)) =20 +struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, + struct vm_area_struct *vma, unsigned long addr); #ifdef CONFIG_NUMA struct page *alloc_pages_noprof(gfp_t gfp, unsigned int order); struct folio *folio_alloc_noprof(gfp_t gfp, unsigned int order); struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order, struct mempolicy *mpol, pgoff_t ilx, int nid); -struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, struct vm_area_= struct *vma, - unsigned long addr); #else static inline struct page *alloc_pages_noprof(gfp_t gfp_mask, unsigned int= order) { @@ -339,11 +339,6 @@ static inline struct folio *folio_alloc_mpol_noprof(gf= p_t gfp, unsigned int orde { return folio_alloc_noprof(gfp, order); } -static inline struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, - struct vm_area_struct *vma, unsigned long addr) -{ - return folio_alloc_noprof(gfp, order); -} #endif =20 #define alloc_pages(...) alloc_hooks(alloc_pages_noprof(__VA_ARGS__)) diff --git a/mm/mempolicy.c b/mm/mempolicy.c index d139b074a599..a1707ad498a8 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -2516,38 +2516,6 @@ struct folio *folio_alloc_mpol_noprof(gfp_t gfp, uns= igned int order, return page_rmappable_folio(page); } =20 -/** - * vma_alloc_folio - Allocate a folio for a VMA. - * @gfp: GFP flags. - * @order: Order of the folio. - * @vma: Pointer to VMA. - * @addr: Virtual address of the allocation. Must be inside @vma. - * - * Allocate a folio for a specific address in @vma, using the appropriate - * NUMA policy. The caller must hold the mmap_lock of the mm_struct of the - * VMA to prevent it from going away. Should be used for all allocations - * for folios that will be mapped into user space, excepting hugetlbfs, and - * excepting where direct use of folio_alloc_mpol() is more appropriate. - * - * Return: The folio on success or NULL if allocation fails. - */ -struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, struct vm_area_= struct *vma, - unsigned long addr) -{ - struct mempolicy *pol; - pgoff_t ilx; - struct folio *folio; - - if (vma->vm_flags & VM_DROPPABLE) - gfp |=3D __GFP_NOWARN; - - pol =3D get_vma_policy(vma, addr, order, &ilx); - folio =3D folio_alloc_mpol_noprof(gfp, order, pol, ilx, numa_node_id()); - mpol_cond_put(pol); - return folio; -} -EXPORT_SYMBOL(vma_alloc_folio_noprof); - struct page *alloc_frozen_pages_noprof(gfp_t gfp, unsigned order) { struct mempolicy *pol =3D &default_policy; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 8dae5b3f5876..6a605d05e8cd 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -5286,6 +5286,49 @@ struct folio *__folio_alloc_noprof(gfp_t gfp, unsign= ed int order, int preferred_ } EXPORT_SYMBOL(__folio_alloc_noprof); =20 +#ifdef CONFIG_NUMA +/** + * vma_alloc_folio - Allocate a folio for a VMA. + * @gfp: GFP flags. + * @order: Order of the folio. + * @vma: Pointer to VMA. + * @addr: Virtual address of the allocation. Must be inside @vma. + * + * Allocate a folio for a specific address in @vma, using the appropriate + * NUMA policy. The caller must hold the mmap_lock of the mm_struct of the + * VMA to prevent it from going away. Should be used for all allocations + * for folios that will be mapped into user space, excepting hugetlbfs, and + * excepting where direct use of folio_alloc_mpol() is more appropriate. + * + * Return: The folio on success or NULL if allocation fails. + */ +struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, + struct vm_area_struct *vma, unsigned long addr) +{ + struct mempolicy *pol; + pgoff_t ilx; + struct folio *folio; + + if (vma->vm_flags & VM_DROPPABLE) + gfp |=3D __GFP_NOWARN; + + pol =3D get_vma_policy(vma, addr, order, &ilx); + folio =3D folio_alloc_mpol_noprof(gfp, order, pol, ilx, numa_node_id()); + mpol_cond_put(pol); + return folio; +} +#else +struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, + struct vm_area_struct *vma, unsigned long addr) +{ + if (vma->vm_flags & VM_DROPPABLE) + gfp |=3D __GFP_NOWARN; + + return folio_alloc_noprof(gfp, order); +} +#endif +EXPORT_SYMBOL(vma_alloc_folio_noprof); + /* * Common helper functions. Never use with __GFP_HIGHMEM because the retur= ned * address cannot represent highmem pages. Use alloc_pages and then kmap if --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A202D3B14B7 for ; Mon, 8 Jun 2026 08:35:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907739; cv=none; b=Wca5/gkS90S65rv3bB5umw1AnVaCr0o8Xt6+Ipf/wkd+yHgPA0b1Va32lyaK3XGr2bgnOYW9RkzQiF+YqmB9hPmxAFxlB2jOagI4QZuqyi4FVP5Y7ZlFmhx18fhzoFSESl8PuA8BktA8S8lQDQaZxM5aS2yUUcCJ2ucwGOGjvOU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907739; c=relaxed/simple; bh=f5U7wk39Pib0dJgeQgZbB+NuIGFstsUUa/l16uySjxc=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=FFsUzqXAvCU7yGw04izAxsLR1ZeYpF/T1iQSAkesF7FH2u72sJR0mO9SjVcykkq2rUO2/ibsjQDhjFrEGNn4tCvNnQJEaEHcKy/8W+kM/cexQkQFt/TX2HFNvJ3pz9IWT/4YWCIN0+5LgEOuBJZP7Dvu2L4yD0G/ydLj7yMmwtE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=SzHvvVMP; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=XZIBOaSW; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="SzHvvVMP"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="XZIBOaSW" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907732; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=w2/zUcZBQmNnar2to9U33SyX02FPb9ifpYfk2LvDheA=; b=SzHvvVMPfPttPcbwb4LMPJyotYcqqBUrrlQDVmHtsRNYuWwNDRlX0C4wcYlGl/iIJfUbkL oxHIWu4JD80WQV82XBMidLAd3230wOX5N5WzqX+HE6rz7kp14Z2bW+X+PKyT6rILT0+8eq 0fvMafIlY3dXZTVWN+y3Yyv802pKn70= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-253-mEIXTBeRNA6T-t_wyHZf5A-1; Mon, 08 Jun 2026 04:35:31 -0400 X-MC-Unique: mEIXTBeRNA6T-t_wyHZf5A-1 X-Mimecast-MFC-AGG-ID: mEIXTBeRNA6T-t_wyHZf5A_1780907730 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-490b61243easo21727735e9.2 for ; Mon, 08 Jun 2026 01:35:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907730; x=1781512530; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=w2/zUcZBQmNnar2to9U33SyX02FPb9ifpYfk2LvDheA=; b=XZIBOaSWJR3eHHj1Q/KRsdLD2nSwyd9fzFQ5ELUK1/QISjizNg38+GGKAJ0CqHUT3d qDjQTlrdcY2ZQXAMG0c022JcM0+26jWpFW6uQz2LJnrjA0JHzmC4AtCyBu1+F+sGrcKv 3bcYm1SR+8iGbiM1TWSdjhSsfUuBigG+Mir4k2mpJIbgysAXrHMz1uu2OGhfmsPsXjUa LU2XNh/8u6EIQ1diYGdPtvC5I+JZIj49m+e1daQHWt0N2F4zh47Q1tR8MrfaCUCE1/Ll j3KaN498P8gZyGJV+PaAFVMvwRB62731YDbpWFcRhTxRVNpfoSFqzdOotT014811JYom kTRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907730; x=1781512530; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=w2/zUcZBQmNnar2to9U33SyX02FPb9ifpYfk2LvDheA=; b=RdZYP/Vx1rSSOBmJEMiCqUykbNJQkyKguAO9lIOtao1sz5Lx26x4QutEgLHd+zllb3 VSGwAufbAQQsh+o8HnkvD+86JSYWFUdXoKPq5fL8J9CSNMLtEWM8Wc16Z9MkvjAAP7+Q jaLKEpFfAY4krG9hkveAo2viIKnz4TE1PQ0hH8SGwxEEg/Nfn0xjgjysuJ0nuFcYWNCt GY8jKee8+aigkvUV13LJsElH8hLkW/HUQSa2yhHnMW3CYNJpuCXwRff1CLwR+pNWwLV6 U42gS4kSx5eXM7ZJmZXB0IcyYIknn+rj1M5Ufj1mnWkAErMiWHGRYL4rrkJseQM0R41K S0YQ== X-Gm-Message-State: AOJu0YxyISft2JA9a3NexMwau4PK9YEpbfXv2MG2mc70YG42X389tVQ2 rl7VZcYTpv6p8l2m1IShQNPj3SCTQxUbhRSb3jJVGHkbLmeoBSP+MlwSY2k7YBx8KZ0T+514w+n V/c27J2zmSgo0coNKyND0+gS/Hsez+5BGcZWKhO9H1WCmRsiE95N/RQTbUSHZDWmxwde5t8IZMz aljZR2zpF1j2xXIWlzdzM9Bv3HsuWQu1W9X++vyNptnT8= X-Gm-Gg: Acq92OG0IeljPfJLuykicOUecRhq59yR6TPAae5IjDoR4frhlbiH+GS7X0LTADjsCBk ROFXIoemFSmDO384aVlQu5+2RQUwwmx/bBlD/s+uP2zMuufPmeLlnwMUDMQkPXRpqKjlj3AGUSZ CzVzDbeDekYNyQn3Dk74pwX5ENjzGdDp72crcMVOIkXwhPV7M/ill6RktyzKSBAgTgmHpURXs9D Z5J3Y8DsB+9aE+aL9dpaTNpUTcvDU9il6VUiEXvlZltpAN6v0P6+Itdld9vDLXbp7cOpAeOCs99 HRMAp0CWBXUPQJqOA6z6Z5RktDQEUMVJtyAPXkizg1LfrmRCR08WWC3k3/bBIYCfrE05469rBgx DJpM1RhvooTgTfIp3qbPvGTs= X-Received: by 2002:a05:600c:5248:b0:48f:d612:3c4a with SMTP id 5b1f17b1804b1-490c25604demr238926255e9.1.1780907729585; Mon, 08 Jun 2026 01:35:29 -0700 (PDT) X-Received: by 2002:a05:600c:5248:b0:48f:d612:3c4a with SMTP id 5b1f17b1804b1-490c25604demr238925325e9.1.1780907728919; Mon, 08 Jun 2026 01:35:28 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4601f351d69sm91125727f8f.29.2026.06.08.01.35.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:35:28 -0700 (PDT) Date: Mon, 8 Jun 2026 04:35:17 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 07/37] mm: thread user_addr through page allocator for cache-friendly zeroing Message-ID: <50d410b47fe3f45327783e05bd306d5eaab75e65.1780906288.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Thread a user virtual address from vma_alloc_folio() down through the page allocator to post_alloc_hook(). This is plumbing preparation for a subsequent patch that will use user_addr to call folio_zero_user() for cache-friendly zeroing of user pages. The user_addr is stored in struct alloc_context and flows through: vma_alloc_folio -> folio_alloc_mpol -> __alloc_pages_mpol -> __alloc_frozen_pages -> get_page_from_freelist -> prep_new_page -> post_alloc_hook USER_ADDR_NONE ((unsigned long)-1) is used for non-user allocations, since address 0 is a valid userspace mapping. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- include/linux/gfp.h | 2 +- mm/compaction.c | 5 ++--- mm/hugetlb.c | 36 ++++++++++++++++++++---------------- mm/internal.h | 21 ++++++++++++++++++--- mm/mempolicy.c | 44 ++++++++++++++++++++++++++++++++------------ mm/mmap.c | 6 ++++++ mm/page_alloc.c | 44 +++++++++++++++++++++++++++++--------------- mm/slub.c | 4 ++-- 8 files changed, 110 insertions(+), 52 deletions(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index 7ccbda35b9ad..ee35c5367abc 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -337,7 +337,7 @@ static inline struct folio *folio_alloc_noprof(gfp_t gf= p, unsigned int order) static inline struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned in= t order, struct mempolicy *mpol, pgoff_t ilx, int nid) { - return folio_alloc_noprof(gfp, order); + return __folio_alloc_noprof(gfp, order, numa_node_id(), NULL); } #endif =20 diff --git a/mm/compaction.c b/mm/compaction.c index 3648ce22c807..72684fe81e83 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -82,7 +82,7 @@ static inline bool is_via_compact_memory(int order) { ret= urn false; } =20 static struct page *mark_allocated_noprof(struct page *page, unsigned int = order, gfp_t gfp_flags) { - post_alloc_hook(page, order, __GFP_MOVABLE); + post_alloc_hook(page, order, __GFP_MOVABLE, USER_ADDR_NONE); set_page_refcounted(page); return page; } @@ -1849,8 +1849,7 @@ static struct folio *compaction_alloc_noprof(struct f= olio *src, unsigned long da set_page_private(&freepage[size], start_order); } dst =3D (struct folio *)freepage; - - post_alloc_hook(&dst->page, order, __GFP_MOVABLE); + post_alloc_hook(&dst->page, order, __GFP_MOVABLE, USER_ADDR_NONE); set_page_refcounted(&dst->page); if (order) prep_compound_page(&dst->page, order); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 4b80b167cc9c..f3bc15a7889a 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1786,7 +1786,8 @@ struct address_space *hugetlb_folio_mapping_lock_writ= e(struct folio *folio) } =20 static struct folio *alloc_buddy_frozen_folio(int order, gfp_t gfp_mask, - int nid, nodemask_t *nmask, nodemask_t *node_alloc_noretry) + int nid, nodemask_t *nmask, nodemask_t *node_alloc_noretry, + unsigned long addr) { struct folio *folio; bool alloc_try_hard =3D true; @@ -1803,7 +1804,7 @@ static struct folio *alloc_buddy_frozen_folio(int ord= er, gfp_t gfp_mask, if (alloc_try_hard) gfp_mask |=3D __GFP_RETRY_MAYFAIL; =20 - folio =3D (struct folio *)__alloc_frozen_pages(gfp_mask, order, nid, nmas= k); + folio =3D (struct folio *)__alloc_frozen_pages(gfp_mask, order, nid, nmas= k, addr); =20 /* * If we did not specify __GFP_RETRY_MAYFAIL, but still got a @@ -1832,7 +1833,7 @@ static struct folio *alloc_buddy_frozen_folio(int ord= er, gfp_t gfp_mask, =20 static struct folio *only_alloc_fresh_hugetlb_folio(struct hstate *h, gfp_t gfp_mask, int nid, nodemask_t *nmask, - nodemask_t *node_alloc_noretry) + nodemask_t *node_alloc_noretry, unsigned long addr) { struct folio *folio; int order =3D huge_page_order(h); @@ -1844,7 +1845,7 @@ static struct folio *only_alloc_fresh_hugetlb_folio(s= truct hstate *h, folio =3D alloc_gigantic_frozen_folio(order, gfp_mask, nid, nmask); else folio =3D alloc_buddy_frozen_folio(order, gfp_mask, nid, nmask, - node_alloc_noretry); + node_alloc_noretry, addr); if (folio) init_new_hugetlb_folio(folio); return folio; @@ -1858,11 +1859,12 @@ static struct folio *only_alloc_fresh_hugetlb_folio= (struct hstate *h, * pages is zero, and the accounting must be done in the caller. */ static struct folio *alloc_fresh_hugetlb_folio(struct hstate *h, - gfp_t gfp_mask, int nid, nodemask_t *nmask) + gfp_t gfp_mask, int nid, nodemask_t *nmask, + unsigned long addr) { struct folio *folio; =20 - folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, NULL); + folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, NULL, a= ddr); if (folio) hugetlb_vmemmap_optimize_folio(h, folio); return folio; @@ -1902,7 +1904,7 @@ static struct folio *alloc_pool_huge_folio(struct hst= ate *h, struct folio *folio; =20 folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, node, - nodes_allowed, node_alloc_noretry); + nodes_allowed, node_alloc_noretry, USER_ADDR_NONE); if (folio) return folio; } @@ -2071,7 +2073,8 @@ int dissolve_free_hugetlb_folios(unsigned long start_= pfn, unsigned long end_pfn) * Allocates a fresh surplus page from the page allocator. */ static struct folio *alloc_surplus_hugetlb_folio(struct hstate *h, - gfp_t gfp_mask, int nid, nodemask_t *nmask) + gfp_t gfp_mask, int nid, nodemask_t *nmask, + unsigned long addr) { struct folio *folio =3D NULL; =20 @@ -2083,7 +2086,7 @@ static struct folio *alloc_surplus_hugetlb_folio(stru= ct hstate *h, goto out_unlock; spin_unlock_irq(&hugetlb_lock); =20 - folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask); + folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, addr); if (!folio) return NULL; =20 @@ -2126,7 +2129,7 @@ static struct folio *alloc_migrate_hugetlb_folio(stru= ct hstate *h, gfp_t gfp_mas if (hstate_is_gigantic(h)) return NULL; =20 - folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask); + folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, nid, nmask, USER_ADDR_NO= NE); if (!folio) return NULL; =20 @@ -2162,14 +2165,14 @@ struct folio *alloc_buddy_hugetlb_folio_with_mpol(s= truct hstate *h, if (mpol_is_preferred_many(mpol)) { gfp_t gfp =3D gfp_mask & ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL); =20 - folio =3D alloc_surplus_hugetlb_folio(h, gfp, nid, nodemask); + folio =3D alloc_surplus_hugetlb_folio(h, gfp, nid, nodemask, addr); =20 /* Fallback to all nodes if page=3D=3DNULL */ nodemask =3D NULL; } =20 if (!folio) - folio =3D alloc_surplus_hugetlb_folio(h, gfp_mask, nid, nodemask); + folio =3D alloc_surplus_hugetlb_folio(h, gfp_mask, nid, nodemask, addr); mpol_cond_put(mpol); return folio; } @@ -2276,7 +2279,8 @@ static int gather_surplus_pages(struct hstate *h, lon= g delta) * down the road to pick the current node if that is the case. */ folio =3D alloc_surplus_hugetlb_folio(h, htlb_alloc_mask(h), - NUMA_NO_NODE, &alloc_nodemask); + NUMA_NO_NODE, &alloc_nodemask, + USER_ADDR_NONE); if (!folio) { alloc_ok =3D false; break; @@ -2682,7 +2686,7 @@ static int alloc_and_dissolve_hugetlb_folio(struct fo= lio *old_folio, spin_unlock_irq(&hugetlb_lock); gfp_mask =3D htlb_alloc_mask(h) | __GFP_THISNODE; new_folio =3D alloc_fresh_hugetlb_folio(h, gfp_mask, - nid, NULL); + nid, NULL, USER_ADDR_NONE); if (!new_folio) return -ENOMEM; goto retry; @@ -3380,13 +3384,13 @@ static void __init hugetlb_hstate_alloc_pages_oneno= de(struct hstate *h, int nid) gfp_t gfp_mask =3D htlb_alloc_mask(h) | __GFP_THISNODE; =20 folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, - &node_states[N_MEMORY], NULL); + &node_states[N_MEMORY], NULL, USER_ADDR_NONE); if (!folio && !list_empty(&folio_list) && hugetlb_vmemmap_optimizable_size(h)) { prep_and_add_allocated_folios(h, &folio_list); INIT_LIST_HEAD(&folio_list); folio =3D only_alloc_fresh_hugetlb_folio(h, gfp_mask, nid, - &node_states[N_MEMORY], NULL); + &node_states[N_MEMORY], NULL, USER_ADDR_NONE); } if (!folio) break; diff --git a/mm/internal.h b/mm/internal.h index 5a2ddcf68e0b..9d2198114510 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -662,6 +662,16 @@ void calculate_min_free_kbytes(void); int __meminit init_per_zone_wmark_min(void); void page_alloc_sysctl_init(void); =20 +/* + * Sentinel for user_addr: indicates a non-user allocation. + * Cannot use 0 because address 0 is a valid userspace mapping. + * (unsigned long)-1 is safe because: + * 1. vm_end =3D addr + len <=3D TASK_SIZE, and vm_end is exclusive, + * so -1 is never inside any VMA. + * 2. It will only be compared to page-aligned addresses. + */ +#define USER_ADDR_NONE ((unsigned long)-1) + /* * Structure for holding the mostly immutable allocation parameters passed * between functions involved in allocations, including the alloc_pages* @@ -693,6 +703,7 @@ struct alloc_context { */ enum zone_type highest_zoneidx; bool spread_dirty_pages; + unsigned long user_addr; }; =20 /* @@ -916,13 +927,14 @@ static inline void init_compound_tail(struct page *ta= il, prep_compound_tail(tail, head, order); } =20 -void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flag= s); +void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flag= s, + unsigned long user_addr); extern bool free_pages_prepare(struct page *page, unsigned int order); =20 extern int user_min_free_kbytes; =20 struct page *__alloc_frozen_pages_noprof(gfp_t, unsigned int order, int ni= d, - nodemask_t *); + nodemask_t *, unsigned long user_addr); #define __alloc_frozen_pages(...) \ alloc_hooks(__alloc_frozen_pages_noprof(__VA_ARGS__)) void free_frozen_pages(struct page *page, unsigned int order); @@ -930,10 +942,13 @@ void free_unref_folios(struct folio_batch *fbatch); =20 #ifdef CONFIG_NUMA struct page *alloc_frozen_pages_noprof(gfp_t, unsigned int order); +struct folio *folio_alloc_mpol_user_noprof(gfp_t gfp, unsigned int order, + struct mempolicy *pol, pgoff_t ilx, int nid, + unsigned long user_addr); #else static inline struct page *alloc_frozen_pages_noprof(gfp_t gfp, unsigned i= nt order) { - return __alloc_frozen_pages_noprof(gfp, order, numa_node_id(), NULL); + return __alloc_frozen_pages_noprof(gfp, order, numa_node_id(), NULL, USER= _ADDR_NONE); } #endif =20 diff --git a/mm/mempolicy.c b/mm/mempolicy.c index a1707ad498a8..f573ff32e94d 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -2413,7 +2413,8 @@ bool mempolicy_in_oom_domain(struct task_struct *tsk, } =20 static struct page *alloc_pages_preferred_many(gfp_t gfp, unsigned int ord= er, - int nid, nodemask_t *nodemask) + int nid, nodemask_t *nodemask, + unsigned long user_addr) { struct page *page; gfp_t preferred_gfp; @@ -2426,25 +2427,29 @@ static struct page *alloc_pages_preferred_many(gfp_= t gfp, unsigned int order, */ preferred_gfp =3D gfp | __GFP_NOWARN; preferred_gfp &=3D ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL); - page =3D __alloc_frozen_pages_noprof(preferred_gfp, order, nid, nodemask); + page =3D __alloc_frozen_pages_noprof(preferred_gfp, order, nid, + nodemask, user_addr); if (!page) - page =3D __alloc_frozen_pages_noprof(gfp, order, nid, NULL); + page =3D __alloc_frozen_pages_noprof(gfp, order, nid, NULL, + user_addr); =20 return page; } =20 /** - * alloc_pages_mpol - Allocate pages according to NUMA mempolicy. + * __alloc_pages_mpol - Allocate pages according to NUMA mempolicy. * @gfp: GFP flags. * @order: Order of the page allocation. * @pol: Pointer to the NUMA mempolicy. * @ilx: Index for interleave mempolicy (also distinguishes alloc_pages()). * @nid: Preferred node (usually numa_node_id() but @mpol may override it). + * @user_addr: User fault address for cache-friendly zeroing, or USER_ADDR= _NONE. * * Return: The page on success or NULL if allocation fails. */ -static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order, - struct mempolicy *pol, pgoff_t ilx, int nid) +static struct page *__alloc_pages_mpol(gfp_t gfp, unsigned int order, + struct mempolicy *pol, pgoff_t ilx, int nid, + unsigned long user_addr) { nodemask_t *nodemask; struct page *page; @@ -2452,7 +2457,8 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsig= ned int order, nodemask =3D policy_nodemask(gfp, pol, ilx, &nid); =20 if (pol->mode =3D=3D MPOL_PREFERRED_MANY) - return alloc_pages_preferred_many(gfp, order, nid, nodemask); + return alloc_pages_preferred_many(gfp, order, nid, nodemask, + user_addr); =20 if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && /* filter "hugepage" allocation, unless from alloc_pages() */ @@ -2476,7 +2482,7 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsig= ned int order, */ page =3D __alloc_frozen_pages_noprof( gfp | __GFP_THISNODE | __GFP_NORETRY, order, - nid, NULL); + nid, NULL, user_addr); if (page || !(gfp & __GFP_DIRECT_RECLAIM)) return page; /* @@ -2488,7 +2494,7 @@ static struct page *alloc_pages_mpol(gfp_t gfp, unsig= ned int order, } } =20 - page =3D __alloc_frozen_pages_noprof(gfp, order, nid, nodemask); + page =3D __alloc_frozen_pages_noprof(gfp, order, nid, nodemask, user_addr= ); =20 if (unlikely(pol->mode =3D=3D MPOL_INTERLEAVE || pol->mode =3D=3D MPOL_WEIGHTED_INTERLEAVE) && page) { @@ -2504,11 +2510,18 @@ static struct page *alloc_pages_mpol(gfp_t gfp, uns= igned int order, return page; } =20 -struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order, +static struct page *alloc_pages_mpol(gfp_t gfp, unsigned int order, struct mempolicy *pol, pgoff_t ilx, int nid) { - struct page *page =3D alloc_pages_mpol(gfp | __GFP_COMP, order, pol, - ilx, nid); + return __alloc_pages_mpol(gfp, order, pol, ilx, nid, USER_ADDR_NONE); +} + +struct folio *folio_alloc_mpol_user_noprof(gfp_t gfp, unsigned int order, + struct mempolicy *pol, pgoff_t ilx, int nid, + unsigned long user_addr) +{ + struct page *page =3D __alloc_pages_mpol(gfp | __GFP_COMP, order, pol, + ilx, nid, user_addr); if (!page) return NULL; =20 @@ -2516,6 +2529,13 @@ struct folio *folio_alloc_mpol_noprof(gfp_t gfp, uns= igned int order, return page_rmappable_folio(page); } =20 +struct folio *folio_alloc_mpol_noprof(gfp_t gfp, unsigned int order, + struct mempolicy *pol, pgoff_t ilx, int nid) +{ + return folio_alloc_mpol_user_noprof(gfp, order, pol, ilx, nid, + USER_ADDR_NONE); +} + struct page *alloc_frozen_pages_noprof(gfp_t gfp, unsigned order) { struct mempolicy *pol =3D &default_policy; diff --git a/mm/mmap.c b/mm/mmap.c index 5754d1c36462..73413cebc418 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -855,6 +855,12 @@ __get_unmapped_area(struct file *file, unsigned long a= ddr, unsigned long len, if (IS_ERR_VALUE(addr)) return addr; =20 + /* + * The check below ensures vm_end =3D addr + len <=3D TASK_SIZE. + * Since (unsigned long)-1 (USER_ADDR_NONE) >=3D TASK_SIZE and + * vm_end is exclusive, USER_ADDR_NONE is thus never a valid + * userspace address. + */ if (addr > TASK_SIZE - len) return -ENOMEM; if (offset_in_page(addr)) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 6a605d05e8cd..21b52c879751 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1815,7 +1815,7 @@ static inline bool should_skip_init(gfp_t flags) } =20 inline void post_alloc_hook(struct page *page, unsigned int order, - gfp_t gfp_flags) + gfp_t gfp_flags, unsigned long user_addr) { const bool zero_tags =3D gfp_flags & __GFP_ZEROTAGS; bool init =3D !want_init_on_free() && want_init_on_alloc(gfp_flags) && @@ -1870,9 +1870,10 @@ inline void post_alloc_hook(struct page *page, unsig= ned int order, } =20 static void prep_new_page(struct page *page, unsigned int order, gfp_t gfp= _flags, - unsigned int alloc_flags) + unsigned int alloc_flags, + unsigned long user_addr) { - post_alloc_hook(page, order, gfp_flags); + post_alloc_hook(page, order, gfp_flags, user_addr); =20 if (order && (gfp_flags & __GFP_COMP)) prep_compound_page(page, order); @@ -3956,7 +3957,8 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int o= rder, int alloc_flags, page =3D rmqueue(zonelist_zone(ac->preferred_zoneref), zone, order, gfp_mask, alloc_flags, ac->migratetype); if (page) { - prep_new_page(page, order, gfp_mask, alloc_flags); + prep_new_page(page, order, gfp_mask, alloc_flags, + ac->user_addr); =20 return page; } else { @@ -4184,7 +4186,8 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned= int order, =20 /* Prep a captured page if available */ if (page) - prep_new_page(page, order, gfp_mask, alloc_flags); + prep_new_page(page, order, gfp_mask, alloc_flags, + ac->user_addr); =20 /* Try get a page from the freelist if available */ if (!page) @@ -5061,7 +5064,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, struct zoneref *z; struct per_cpu_pages *pcp; struct list_head *pcp_list; - struct alloc_context ac; + struct alloc_context ac =3D { .user_addr =3D USER_ADDR_NONE }; gfp_t alloc_gfp; unsigned int alloc_flags =3D ALLOC_WMARK_LOW; int nr_populated =3D 0, nr_account =3D 0; @@ -5176,7 +5179,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, } nr_account++; =20 - prep_new_page(page, 0, gfp, 0); + prep_new_page(page, 0, gfp, 0, USER_ADDR_NONE); set_page_refcounted(page); page_array[nr_populated++] =3D page; } @@ -5201,12 +5204,13 @@ EXPORT_SYMBOL_GPL(alloc_pages_bulk_noprof); * This is the 'heart' of the zoned buddy allocator. */ struct page *__alloc_frozen_pages_noprof(gfp_t gfp, unsigned int order, - int preferred_nid, nodemask_t *nodemask) + int preferred_nid, nodemask_t *nodemask, + unsigned long user_addr) { struct page *page; unsigned int alloc_flags =3D ALLOC_WMARK_LOW; gfp_t alloc_gfp; /* The gfp_t that was actually used for allocation */ - struct alloc_context ac =3D { }; + struct alloc_context ac =3D { .user_addr =3D user_addr }; =20 /* * There are several places where we assume that the order value is sane @@ -5267,10 +5271,12 @@ EXPORT_SYMBOL(__alloc_frozen_pages_noprof); =20 struct page *__alloc_pages_noprof(gfp_t gfp, unsigned int order, int preferred_nid, nodemask_t *nodemask) + { struct page *page; =20 - page =3D __alloc_frozen_pages_noprof(gfp, order, preferred_nid, nodemask); + page =3D __alloc_frozen_pages_noprof(gfp, order, preferred_nid, + nodemask, USER_ADDR_NONE); if (page) set_page_refcounted(page); return page; @@ -5313,7 +5319,8 @@ struct folio *vma_alloc_folio_noprof(gfp_t gfp, int o= rder, gfp |=3D __GFP_NOWARN; =20 pol =3D get_vma_policy(vma, addr, order, &ilx); - folio =3D folio_alloc_mpol_noprof(gfp, order, pol, ilx, numa_node_id()); + folio =3D folio_alloc_mpol_user_noprof(gfp, order, pol, ilx, + numa_node_id(), addr); mpol_cond_put(pol); return folio; } @@ -5321,10 +5328,17 @@ struct folio *vma_alloc_folio_noprof(gfp_t gfp, int= order, struct folio *vma_alloc_folio_noprof(gfp_t gfp, int order, struct vm_area_struct *vma, unsigned long addr) { + struct page *page; + if (vma->vm_flags & VM_DROPPABLE) gfp |=3D __GFP_NOWARN; =20 - return folio_alloc_noprof(gfp, order); + page =3D __alloc_frozen_pages_noprof(gfp | __GFP_COMP, order, + numa_node_id(), NULL, addr); + if (!page) + return NULL; + set_page_refcounted(page); + return page_rmappable_folio(page); } #endif EXPORT_SYMBOL(vma_alloc_folio_noprof); @@ -6905,7 +6919,7 @@ static void split_free_frozen_pages(struct list_head = *list, gfp_t gfp_mask) list_for_each_entry_safe(page, next, &list[order], lru) { int i; =20 - post_alloc_hook(page, order, gfp_mask); + post_alloc_hook(page, order, gfp_mask, USER_ADDR_NONE); if (!order) continue; =20 @@ -7111,7 +7125,7 @@ int alloc_contig_frozen_range_noprof(unsigned long st= art, unsigned long end, struct page *head =3D pfn_to_page(start); =20 check_new_pages(head, order); - prep_new_page(head, order, gfp_mask, 0); + prep_new_page(head, order, gfp_mask, 0, USER_ADDR_NONE); } else { ret =3D -EINVAL; WARN(true, "PFN range: requested [%lu, %lu), allocated [%lu, %lu)\n", @@ -7776,7 +7790,7 @@ struct page *alloc_frozen_pages_nolock_noprof(gfp_t g= fp_flags, int nid, unsigned gfp_t alloc_gfp =3D __GFP_NOWARN | __GFP_ZERO | __GFP_NOMEMALLOC | __GFP_= COMP | gfp_flags; unsigned int alloc_flags =3D ALLOC_TRYLOCK; - struct alloc_context ac =3D { }; + struct alloc_context ac =3D { .user_addr =3D USER_ADDR_NONE }; struct page *page; =20 VM_WARN_ON_ONCE(gfp_flags & ~__GFP_ACCOUNT); diff --git a/mm/slub.c b/mm/slub.c index a2bf3756ca7d..f397fa2f3f80 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -3275,7 +3275,7 @@ static inline struct slab *alloc_slab_page(gfp_t flag= s, int node, else if (node =3D=3D NUMA_NO_NODE) page =3D alloc_frozen_pages(flags, order); else - page =3D __alloc_frozen_pages(flags, order, node, NULL); + page =3D __alloc_frozen_pages(flags, order, node, NULL, USER_ADDR_NONE); =20 if (!page) return NULL; @@ -5236,7 +5236,7 @@ static void *___kmalloc_large_node(size_t size, gfp_t= flags, int node) if (node =3D=3D NUMA_NO_NODE) page =3D alloc_frozen_pages_noprof(flags, order); else - page =3D __alloc_frozen_pages_noprof(flags, order, node, NULL); + page =3D __alloc_frozen_pages_noprof(flags, order, node, NULL, USER_ADDR= _NONE); =20 if (page) { ptr =3D page_address(page); --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B36603955DD for ; Mon, 8 Jun 2026 08:35:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907766; cv=none; b=oXat9MWCkURctNuYDtBRicBr6yeeYQjYL/nhhisqPAbFRL7MWtZY2vzhjZXNCdpUgvw0GqNBhws3j/5dSbp08Wr4oykE0ZiYkSA2+Piq2SeQNuNzw3ih39ZaQSxxF/6oeAhaWvsgDEx7/QhrRlhMuIW5YG91I2wfTjXgGSDng/w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907766; c=relaxed/simple; bh=VvYEZY8DbRfnOqU9SNzEdKKP+Kx+uDwCJkKNzhWK0fc=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=ZORDV1n9uCVqzD53jwYiUDvVVHstQQUBKBvIDZw0CNSl4TUmmjJrKri08RGh09RcWojqAGmZPKSfLypxGGuw2RbbOJV69hn6RkRJyIuAXuki/i9HoF/BHZWLQXQVHHlxGRhRFcsLBzk/sLZTtvKiD2yN2XCxKCTk/OunF34/Ne4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=c3wqB9hY; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=Bg6HP6do; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="c3wqB9hY"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="Bg6HP6do" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907758; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=OcVo+/Lds4YxddLi3UfNSteOvi0JEMISiAqD3/N1YSI=; b=c3wqB9hY472bXHN0m28ksX9jAWkzuEfRf4FObbAfwBuF/lm99nlWprXZtAOhoxLm0Ktzj/ eXMCC+UXHWMh1dlWmSAq8uRLdRWYLpXyU/WW3FQCDEzVfa0UWiFboNNrHs1RBiIgMEitRE VJ8/VnzLk3Y0ulzbSz8m80t+K95AvT8= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-340-4H4DAZNFMOSWLuqIJhOdUQ-1; Mon, 08 Jun 2026 04:35:47 -0400 X-MC-Unique: 4H4DAZNFMOSWLuqIJhOdUQ-1 X-Mimecast-MFC-AGG-ID: 4H4DAZNFMOSWLuqIJhOdUQ_1780907746 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-4601c9b630fso1627511f8f.0 for ; Mon, 08 Jun 2026 01:35:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907746; x=1781512546; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=OcVo+/Lds4YxddLi3UfNSteOvi0JEMISiAqD3/N1YSI=; b=Bg6HP6doMddpIcDcHk/v5luheQ/8ry7EpxqVQT6mxwLvipVeLeEKW126LlF/jIK0zT vDTGEgQdrHDntqBs0gessKzNWHwUJO83rnqX/MLvn4YaGafid0RIITMrVSNa3SC+DNKI htHWMQRnRZNPhGwjB89Ed6hD0lQzAu8Q35DjIhckbEpcS5p5xK4aH8GAwjthNpGhvcsF 9CGsHkOOyiaEPpY5xvlwuVbfUoWuRJikYfW9pFKulwkz8yj8hxhpvA6gq5iARtRfVZVZ P5fdMkCfmW1hNf7mWSgb+Vl12nSZ1LVQmhyccibnSCbxVG54zTpUuRJ8XbifdmeArx9v q2Jw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907746; x=1781512546; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OcVo+/Lds4YxddLi3UfNSteOvi0JEMISiAqD3/N1YSI=; b=saO7gwy7S5w6am5bq+VdIvJei9F62uTjRSNamxKmUj761Fjlddy5fyI6GhxEyAkryI M+tSLhTmHJFUjXkup7CBsRGwvvBoaRKeBVYsLF9d+CMS8SNOtqRUPUNZybd6dE2Z851f EqW3laxvD1giiQtd8f/mbfQOPN1CkO79SVpyo3PtgVkrwg7M4HJD2EZ/YhBmi1FatdXd UW+dEYaU4ALaRKIkrUMKxOdPVrzeqx2KQSz7ZMazbLYf2VAbxjmiqaJgBjkf18g1Snn4 Ej/6aIDZBETfIJEf3faVNvLThfmEQ22slMl9OdYd3THtuEkjyTRlH+NHd0G+9FEDp7Pt X+bA== X-Gm-Message-State: AOJu0Yx0Iyj3m3rdqNmxpKhlW7aqDB13rmbaer8Bat7XeCa5ISNtAvM0 jyZiSmrkvAaFpwkIUDYCusztU7qVR59MPz7cgm2auYZ2dZK4ecEW0CST/VBEqs+l/wXqIMC6CZe UfEQCHVCKVL5H+7zRheJ84GlT32KPBX7eN5ElEyjLfIle49jsXZKUnbw5RTI+ZPFMivqVBw/NJ1 sOIZcnyXRL1HKFjLgH9oZ46QYO+8NUCdQMGJYAONTJmTM= X-Gm-Gg: Acq92OEaDhhWyaKsI3cfm1PX1WWzpATUbWlhtcYucUqt6jYcw2gZKkf008NhQ+k6wHL TU0O5yveVAvWXahdxLfih2UYbW+tTeur0eWED0ftb6wkR7K1IiLU4Oit042SfX3osQ/HQIvJof5 +TOBcYz2hxmeik7hLLhi7koS0z308Rlz/uW0poIl1TmRqpkJxjNaDTi1JXmvz+7H4Fs5Yj/LPPi w6y9PbqiXhq00iFsmSVjcGkDJtU6Y0mVjh5ChrwFd6ZiqkRiW95pc56B+udzxnrvHDOsStSx9gq 2XrNwb1hkPz9QjXWWP73oNEMFrn/n/e0IkCJm23Sx/amny0d1Ghc5KqHsTmnylExGryDv60bN2U PC6XG0ZeSrjam+oNae1dxp1U= X-Received: by 2002:a05:600c:e548:10b0:490:cf76:dea4 with SMTP id 5b1f17b1804b1-490cf76dec2mr27271375e9.2.1780907744827; Mon, 08 Jun 2026 01:35:44 -0700 (PDT) X-Received: by 2002:a05:600c:e548:10b0:490:cf76:dea4 with SMTP id 5b1f17b1804b1-490cf76dec2mr27270255e9.2.1780907744100; Mon, 08 Jun 2026 01:35:44 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4601f2dc577sm50742305f8f.3.2026.06.08.01.35.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:35:43 -0700 (PDT) Date: Mon, 8 Jun 2026 04:35:29 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 08/37] mm: add alloc_contig_frozen_pages_user for cache-friendly zeroing Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a _user variant of alloc_contig_frozen_pages that accepts a user_addr parameter for cache-friendly zeroing of contiguous allocations. No functional change; all existing callers continue to pass USER_ADDR_NONE. Note for reviewers: non-compound contiguous allocations are zeroed via kernel_init_pages, same as before this patch. There is no fault address because these allocations are not from the page fault path. For compound allocations, user_addr reaches post_alloc_hook() which calls folio_zero_user() with the dcache flush on cache-aliasing architectures. Note about Sashiko (sashiko.dev) false positives: sashiko flags two issues here: (1) user_addr silently ignored for non-compound allocations, and (2) post_alloc_hook ignores user_addr. Both are false positives: (1) non-compound contiguous allocations have no fault address to pass, and (2) post_alloc_hook does use user_addr when it is not USER_ADDR_NONE. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/gfp.h | 6 ++++++ mm/page_alloc.c | 42 ++++++++++++++++++++++++++++++++---------- 2 files changed, 38 insertions(+), 10 deletions(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index ee35c5367abc..73109d4e31a4 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -453,6 +453,12 @@ struct page *alloc_contig_frozen_pages_noprof(unsigned= long nr_pages, #define alloc_contig_frozen_pages(...) \ alloc_hooks(alloc_contig_frozen_pages_noprof(__VA_ARGS__)) =20 +struct page *alloc_contig_frozen_pages_user_noprof(unsigned long nr_pages, + gfp_t gfp_mask, int nid, nodemask_t *nodemask, + unsigned long user_addr); +#define alloc_contig_frozen_pages_user(...) \ + alloc_hooks(alloc_contig_frozen_pages_user_noprof(__VA_ARGS__)) + struct page *alloc_contig_pages_noprof(unsigned long nr_pages, gfp_t gfp_m= ask, int nid, nodemask_t *nodemask); #define alloc_contig_pages(...) \ diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 21b52c879751..6d3f284c607d 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -6975,13 +6975,15 @@ static void __free_contig_frozen_range(unsigned lon= g pfn, unsigned long nr_pages } =20 /** - * alloc_contig_frozen_range() -- tries to allocate given range of frozen = pages + * __alloc_contig_frozen_range() -- tries to allocate given range of froze= n pages * @start: start PFN to allocate * @end: one-past-the-last PFN to allocate * @alloc_flags: allocation information * @gfp_mask: GFP mask. Node/zone/placement hints are ignored; only some * action and reclaim modifiers are supported. Reclaim modifiers * control allocation behavior during compaction/migration/reclaim. + * @user_addr: user virtual address for cache-friendly zeroing, or + * USER_ADDR_NONE for kernel allocations. * * The PFN range does not have to be pageblock aligned. The PFN range must * belong to a single zone. @@ -6997,8 +6999,9 @@ static void __free_contig_frozen_range(unsigned long = pfn, unsigned long nr_pages * * Return: zero on success or negative error code. */ -int alloc_contig_frozen_range_noprof(unsigned long start, unsigned long en= d, - acr_flags_t alloc_flags, gfp_t gfp_mask) +static int __alloc_contig_frozen_range(unsigned long start, unsigned long = end, + acr_flags_t alloc_flags, gfp_t gfp_mask, + unsigned long user_addr) { const unsigned int order =3D ilog2(end - start); unsigned long outer_start, outer_end; @@ -7125,7 +7128,7 @@ int alloc_contig_frozen_range_noprof(unsigned long st= art, unsigned long end, struct page *head =3D pfn_to_page(start); =20 check_new_pages(head, order); - prep_new_page(head, order, gfp_mask, 0, USER_ADDR_NONE); + prep_new_page(head, order, gfp_mask, 0, user_addr); } else { ret =3D -EINVAL; WARN(true, "PFN range: requested [%lu, %lu), allocated [%lu, %lu)\n", @@ -7135,6 +7138,13 @@ int alloc_contig_frozen_range_noprof(unsigned long s= tart, unsigned long end, undo_isolate_page_range(start, end); return ret; } + +int alloc_contig_frozen_range_noprof(unsigned long start, unsigned long en= d, + acr_flags_t alloc_flags, gfp_t gfp_mask) +{ + return __alloc_contig_frozen_range(start, end, alloc_flags, gfp_mask, + USER_ADDR_NONE); +} EXPORT_SYMBOL(alloc_contig_frozen_range_noprof); =20 /** @@ -7227,14 +7237,16 @@ static bool zone_spans_last_pfn(const struct zone *= zone, return zone_spans_pfn(zone, last_pfn); } =20 -/** - * alloc_contig_frozen_pages() -- tries to find and allocate contiguous ra= nge of frozen pages +/* + * alloc_contig_frozen_pages_user_noprof() -- allocate contiguous frozen p= ages with user address * @nr_pages: Number of contiguous pages to allocate * @gfp_mask: GFP mask. Node/zone/placement hints limit the search; only s= ome * action and reclaim modifiers are supported. Reclaim modifiers * control allocation behavior during compaction/migration/reclaim. * @nid: Target node * @nodemask: Mask for other possible nodes + * @user_addr: user virtual address for cache-friendly zeroing, or + * USER_ADDR_NONE for kernel allocations. * * This routine is a wrapper around alloc_contig_frozen_range(). It scans = over * zones on an applicable zonelist to find a contiguous pfn range which ca= n then @@ -7253,8 +7265,9 @@ static bool zone_spans_last_pfn(const struct zone *zo= ne, * * Return: pointer to contiguous frozen pages on success, or NULL if not s= uccessful. */ -struct page *alloc_contig_frozen_pages_noprof(unsigned long nr_pages, - gfp_t gfp_mask, int nid, nodemask_t *nodemask) +struct page *alloc_contig_frozen_pages_user_noprof(unsigned long nr_pages, + gfp_t gfp_mask, int nid, nodemask_t *nodemask, + unsigned long user_addr) { unsigned long ret, pfn, flags; struct zonelist *zonelist; @@ -7282,10 +7295,11 @@ struct page *alloc_contig_frozen_pages_noprof(unsig= ned long nr_pages, * win the race and cause allocation to fail. */ spin_unlock_irqrestore(&zone->lock, flags); - ret =3D alloc_contig_frozen_range_noprof(pfn, + ret =3D __alloc_contig_frozen_range(pfn, pfn + nr_pages, ACR_FLAGS_NONE, - gfp_mask); + gfp_mask, + user_addr); if (!ret) return pfn_to_page(pfn); spin_lock_irqsave(&zone->lock, flags); @@ -7307,6 +7321,14 @@ struct page *alloc_contig_frozen_pages_noprof(unsign= ed long nr_pages, } return NULL; } +EXPORT_SYMBOL(alloc_contig_frozen_pages_user_noprof); + +struct page *alloc_contig_frozen_pages_noprof(unsigned long nr_pages, + gfp_t gfp_mask, int nid, nodemask_t *nodemask) +{ + return alloc_contig_frozen_pages_user_noprof(nr_pages, gfp_mask, nid, + nodemask, USER_ADDR_NONE); +} EXPORT_SYMBOL(alloc_contig_frozen_pages_noprof); =20 /** --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2A9533B1EEF for ; Mon, 8 Jun 2026 08:36:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907771; cv=none; b=Rr96ZdYoVTpmoFzcBVFp1Ny9xj0BZYYtqnN155sjPecjJhttYMFb0UqL3MutGRMOE7mrGoRqDujBP3TlhltGYoT7IDzSWR9Jr48Ulnc7DaLwefubNmFkilU4023ygK2cxACm+Xt3oL7f5WrUzAeFTDotv2mKKWaY1DjozRMiKYg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907771; c=relaxed/simple; bh=vsZYN921BalqZjT91Mi+kZYvPDWSYuKfHJCbMesIOlQ=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=BrlStP/LTQZGHVtfS7eVoyx2qPQkFt/QiunntPM/G3R3dGUcW957OBiOk6r0cxuyBMiPZkaKuvhNfe+Els9P2AXNDe6f2zfXkeBZByNkcu1kXqXYAC7wciQdBiMZu4vJUgAGGJw+rx4RY8L+fQl3nXfhRvbyoAXnYyM8DcD6HX4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=bPGpzErd; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=JByTWpfm; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="bPGpzErd"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="JByTWpfm" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907765; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=hi0+oeNecn4eOEAyUzBbvo1srbN9H1YpFSy6pKNCWwo=; b=bPGpzErds6FSQ98XFDWl5Aa7Z5tMKZsOnMGhTCu+rQYWzh+vlu2vqzaz5De5o8bXO1tGIv nIcuDdtLmY5PfwivCE44HVGFFLS/3jd+TVfV8t6SMtA41a3l4AwyI+3gxwlXrcMgrd+FmN 19uYXZ2nmWW0nwlL0pZtn8SnxdLmAdQ= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-596-yNOeaHE5OWqRIQe1lrlhZw-1; Mon, 08 Jun 2026 04:36:04 -0400 X-MC-Unique: yNOeaHE5OWqRIQe1lrlhZw-1 X-Mimecast-MFC-AGG-ID: yNOeaHE5OWqRIQe1lrlhZw_1780907763 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-46010bc0f1eso2338137f8f.3 for ; Mon, 08 Jun 2026 01:36:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907763; x=1781512563; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=hi0+oeNecn4eOEAyUzBbvo1srbN9H1YpFSy6pKNCWwo=; b=JByTWpfmMnUXIqq3KpTTxUfGyyr5MMrhBQ/k8li0qiT42/yrMKSxwuTH/5P9vRb29/ zILWLCC9ecdIiSAL1foEN7tNUQtBq0dYnk6Uom3RqPyYSBOh6QhxBYw+O3DyAqDypjgh cTOk+qnI40TBRiXyq7yucp4qGSi6+jGGwv4h57hsWg6/L+714P8og31Evtmsp+yqtjXa xKhjIMWYwPHM1ZcXX/4B1Aih+dch1g4bWrVvucvC8RTlRC8We36P0otI7ony53W/qpGI juaEJOjiukBL02LxfsdL2lFGwnc3oqnx8OZZs3Cm5ERwiC/6PdFKO1VAj852Gro5tl5/ j51w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907763; x=1781512563; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hi0+oeNecn4eOEAyUzBbvo1srbN9H1YpFSy6pKNCWwo=; b=Vi+Y2Goe9qJJ6Lm+8M8vFhH5szKERvi8NknVF0tppWf8L6iLFZO+5cS1JbqdvaUyQO Zu4MD0bvNSq+PO4YeqsCkJbWjPipxjLi4CzD+9o/6w4xw5KcfOwm9akOsOhVLpP3LRrf uqNnvbn/Bp5FaSQ4SDfZimkZ3n9ZcrlD8kP0y/oLIZWA/hUa2pkkIV3X48Sa6Y8laqCo 06SgDUg9hT2BNjg/lZQ6jwDwi8xUahds40jjplYy5HFLfrgzjTJmDRHqi36743ciYrYg 3YPzoRAhj1qI6aZODhCHgSLq+1g13R+FU2S3/mtY/3nWn0FsU3Yr5HAWVW+/J9w+peiJ hGmw== X-Gm-Message-State: AOJu0YzWZk8toglIad8POSXLpXN4hK/pbYz32/+hE3DoOyMwG+eeQcpp nomju9sqKa7S20c7gtV1t95WkDsp58Ib/judy6yC4hS+sT3Sgv+wsnkNwLdz1+nBtZYo7ex1Deh yj7IGoZGMkwvXktRzvmFjH54poWdzJUDXUph3vOsguEug/RNbvnpUWb6CfSB700+sKB6ldlXUwO 8jvuFcil+zmw9cjW5kDp1S08tBu6/pmmkZpP19aZvLIrI= X-Gm-Gg: Acq92OErV7GelUBXmlL0j+skwyojdrrjbUyQdlZMAwW4OrJSn2Ai5kdE5oFEAwnTTWn am7jzi9YjXO1Ha/WEC1mvY49VeYSpf6QbwBP8wOX8y3UcLetjji0y8/ulS4pZlVEOsYCHobb1Pe Zm8mN2dmFoHdEShUMUwmKKmQz40IdlRlmZ7POyShYWRPT+YL7JMeTUtyqP9ik3G3vZTDNl88dve Irzb+gL3CXZcHFB/JDg7YgwceoXLmAS0dMvEtgtj7yFNcAQFbEOOReIqH1oczfLiorJIovpYVbh usghqaQucTITteUDX49iOxkBEckGgmtrU0Q582I74kdEmrrlWrrVCX9BVjvSu7P+8oH/DxTbPBF 9+zaS8o9dIrVEXq+7buQOREM= X-Received: by 2002:a05:600c:6096:b0:490:b115:e03f with SMTP id 5b1f17b1804b1-490c25ba56cmr240972165e9.8.1780907762625; Mon, 08 Jun 2026 01:36:02 -0700 (PDT) X-Received: by 2002:a05:600c:6096:b0:490:b115:e03f with SMTP id 5b1f17b1804b1-490c25ba56cmr240971115e9.8.1780907762056; Mon, 08 Jun 2026 01:36:02 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-490c2c9ea37sm425199295e9.0.2026.06.08.01.35.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:36:01 -0700 (PDT) Date: Mon, 8 Jun 2026 04:35:44 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 09/37] mm: hugetlb: thread user_addr through gigantic page allocation Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Thread the user_addr parameter through alloc_gigantic_frozen_folio so that gigantic page allocations can benefit from cache-friendly zeroing. Note: the CMA path (hugetlb_cma_alloc_frozen_folio) does not receive user_addr because CMA uses alloc_contig_frozen_pages, not the _user variant. CMA-allocated pages get zeroed via the normal __GFP_ZERO path without cache-friendly addressing. This is acceptable: gigantic pages are rare and the CMA path is a fallback when buddy allocation fails. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/hugetlb.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index f3bc15a7889a..5d7e546565f5 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1355,7 +1355,7 @@ static struct folio *dequeue_hugetlb_folio_vma(struct= hstate *h, =20 #if defined(CONFIG_ARCH_HAS_GIGANTIC_PAGE) && defined(CONFIG_CONTIG_ALLOC) static struct folio *alloc_gigantic_frozen_folio(int order, gfp_t gfp_mask, - int nid, nodemask_t *nodemask) + int nid, nodemask_t *nodemask, unsigned long addr) { struct folio *folio; =20 @@ -1366,13 +1366,15 @@ static struct folio *alloc_gigantic_frozen_folio(in= t order, gfp_t gfp_mask, if (hugetlb_cma_exclusive_alloc()) return NULL; =20 - folio =3D (struct folio *)alloc_contig_frozen_pages(1 << order, gfp_mask, - nid, nodemask); + folio =3D (struct folio *)alloc_contig_frozen_pages_user(1 << order, + gfp_mask, + nid, nodemask, + addr); return folio; } #else /* !CONFIG_ARCH_HAS_GIGANTIC_PAGE || !CONFIG_CONTIG_ALLOC */ static struct folio *alloc_gigantic_frozen_folio(int order, gfp_t gfp_mask= , int nid, - nodemask_t *nodemask) + nodemask_t *nodemask, unsigned long addr) { return NULL; } @@ -1842,7 +1844,8 @@ static struct folio *only_alloc_fresh_hugetlb_folio(s= truct hstate *h, nid =3D numa_mem_id(); =20 if (order_is_gigantic(order)) - folio =3D alloc_gigantic_frozen_folio(order, gfp_mask, nid, nmask); + folio =3D alloc_gigantic_frozen_folio(order, gfp_mask, nid, nmask, + addr); else folio =3D alloc_buddy_frozen_folio(order, gfp_mask, nid, nmask, node_alloc_noretry, addr); --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1DBF53B14BD for ; Mon, 8 Jun 2026 08:36:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907800; cv=none; b=NAkf57HyrpGQ5OvQrJNFeruOIcPbCediGIbpJd0jdAj38DyPCc1xjAfJLPkreYon7V1FSLrdfiduRwbCPlrO/lDjn+ItwI9LdekreUhzTvdz9W4bwJI98IZkr1qfuhtYkcy5WVGVzUTaoy4vMHvbBIyJ17FWVtsy7o/fdFIdAaA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907800; c=relaxed/simple; bh=92V//TJ1Mkotj2ZpeYBESUtzgMLjmko9VnfBRr6iZOA=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Rh7t0SQOJpHSL3byEl91RkR/dLiB/pQKd27hxh4LawHlYmI2Qn6wBmNp0Ak41wV2BuwctOwZ/gsl3RrPfTpM3AYrERv4NYj/nDVVS4LcZT1kab35U49W67vVmRFWujHSEhdqxLkr3PioUSmXjy8Ra5UqeAdWHEy3clDLCJzffEs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=aLmZi9yH; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=oYuFt6v6; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="aLmZi9yH"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="oYuFt6v6" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907791; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=FnyUq+qDaMsFj8Ekur/kDRnbeSK9pjyeC2qTYvUeJTc=; b=aLmZi9yHdS2MBxEjwNlakx0LR0mbcd6Nfh1jYfx1MvU2TwiULcIUxEWwhLjWyFvwOd5PFR ahJL3GQn2zCG0Gde2cSmQ0SbWFeHyajnTx7mhFji/pdyGH1I55J5uWq4gCtb7N2c1VEjOg lKRZeh2gzAST84G/VpWiemtgXg9yAVU= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-601-2BwiciaLMLWVXlxQWYh7bw-1; Mon, 08 Jun 2026 04:36:23 -0400 X-MC-Unique: 2BwiciaLMLWVXlxQWYh7bw-1 X-Mimecast-MFC-AGG-ID: 2BwiciaLMLWVXlxQWYh7bw_1780907781 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-490b5d2e394so29567785e9.3 for ; Mon, 08 Jun 2026 01:36:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907781; x=1781512581; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=FnyUq+qDaMsFj8Ekur/kDRnbeSK9pjyeC2qTYvUeJTc=; b=oYuFt6v6I/Kv1nOyhkdmghB1U3RfYkTGq0QZpOl125HP6CHbZBfs9J7+HjlMHdVojM eI3WxXz4/dMMvbmEmKqVjsQJPpVdFfNoDtIek2pZ8x2LIn4VRD44KI+5a6n9xYU2qA1M nlbBwLJ0oTgnXsoUnPD9YbYPayU3WjA9B8AMPJym0PQif8KIhkKJ7dYn+P8tYFJI0kLF sGQs0JmAoMLuhyZdhDzsJmvORv9xp07UstBf7FYQ4ENYBWOEt2omcS8edLbiPk4/fSzs PWrUursbj/gjzaUlSWSB31kjVcYUF9/dyCma7q15Pd5Eo8Lp3BaKJuc3ta+A41FJf6NG l8wQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907781; x=1781512581; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=FnyUq+qDaMsFj8Ekur/kDRnbeSK9pjyeC2qTYvUeJTc=; b=Hu/t70joO6OK6148SRUbXIhxH7tihgFJxFzO+7aEOm6frEPmD+7pgmqeKRs+mMT9QK 5Nlds2aP6T5ICS8JJ1mmQhZtchM+eu7TXgQmlMasJ8wSR21rh8ixssNu4iqKPKdPJgjc grFRalP1PhcH9IQnWh3xYJREQ5rBcJOhkzjEEpXm69opPQCI+0q0iEOub/Me1yL59aQO iOJpFwhK9HweGDMJFyaZtl/uzQh27XndwZZ5F/AjG3iC+NQgZFVtrjKL5LRrfBbGeLcE oO9Am9QPiQxDtKnPGUh0VBPaCpTt5pRZYBadSukJLn4gp1xrQgnwHdgOY6Uu97OmhPYA I4xg== X-Gm-Message-State: AOJu0YwkwIetHiDuWEcsiiobu89B/PDTpwimdTXuK9uBAQseLsYJh19F LA+psoHXuZiiU0byID7TrVpg7Pn1Lhxrrry5m01hEdCK+d5/FNYCCY1WadEzx9ASwm5a0RYn0tb nlsFLRHOLdnsWoGrxo4e0JMckr0OWmOV6whElDpJ+yRISC4go87x1JcVSFWI53fb439/gbUPITt RLNGyQEJuixLq9Es7yW/8Kt9z47BBJdXtOiAcbzJIjgDE= X-Gm-Gg: Acq92OGwQzZQ9NgcAsqjshO4QQvcroyyj5TOAyDPk6npOq2nHCC0PhCjITq6elz9TcX i+y6jcLmBZfZRAoF+EK9dNEXSafTPcJgTJuDy2XlXpQNcW7iuOfKSlLRs153dgIZifNIrHXfXsE WezekkKc88jssjm8z7bA16875mu0idgMEMkEAOnYxy/P2nwU4CgaeDePHKncPGTevhed/sr3+9j QKLJxYFqAfu4j/fhq/gsX4ZOBPb1QkDdQKoohES+Tilp/T7TAYsFAAnVJXhEWO+6Zl8K6F2510l D4OxI653AyCkn6fsUsNngsWSO8VVhoL6NXOWhGbzN21gDkXaKh5wpSRiuOze7ppCrxO0xnY5z91 6ZClijtqI7ED8ZSBN7UWsLwU= X-Received: by 2002:a05:600c:4e4b:b0:490:bfda:7d7e with SMTP id 5b1f17b1804b1-490c258999amr250329395e9.3.1780907781008; Mon, 08 Jun 2026 01:36:21 -0700 (PDT) X-Received: by 2002:a05:600c:4e4b:b0:490:bfda:7d7e with SMTP id 5b1f17b1804b1-490c258999amr250328155e9.3.1780907780459; Mon, 08 Jun 2026 01:36:20 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-490bc3918fcsm427602815e9.3.2026.06.08.01.36.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:36:19 -0700 (PDT) Date: Mon, 8 Jun 2026 04:36:02 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 10/37] mm: add folio_zero_user stub for configs without THP/HUGETLBFS Message-ID: <60d48e3c6f03238f385cdb8e48f6d8ffd4cff387.1780906288.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" folio_zero_user() is defined in mm/memory.c under CONFIG_TRANSPARENT_HUGEPAGE || CONFIG_HUGETLBFS. A subsequent patch will call it from post_alloc_hook() for all user page zeroing, so configs without THP or HUGETLBFS will need a stub. Add a stub that uses clear_user_highpages() with aligned addr_hint. Without THP/HUGETLBFS, only order-0 user pages are allocated, so the locality optimization in the real folio_zero_user() (zero near the faulting address last) is not needed. This also matches what vma_alloc_zeroed_movable_folio currently does. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/folio_zero.h | 18 ++++++++++++++++++ mm/page_alloc.c | 1 + 2 files changed, 19 insertions(+) create mode 100644 mm/folio_zero.h diff --git a/mm/folio_zero.h b/mm/folio_zero.h new file mode 100644 index 000000000000..c135b3a34da8 --- /dev/null +++ b/mm/folio_zero.h @@ -0,0 +1,18 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef MM_FOLIO_ZERO_H +#define MM_FOLIO_ZERO_H + +#include + +#if defined(CONFIG_TRANSPARENT_HUGEPAGE) || defined(CONFIG_HUGETLBFS) +void folio_zero_user(struct folio *folio, unsigned long addr_hint); +#else +static inline void folio_zero_user(struct folio *folio, unsigned long addr= _hint) +{ + unsigned long base =3D ALIGN_DOWN(addr_hint, folio_size(folio)); + + clear_user_highpages(&folio->page, base, folio_nr_pages(folio)); +} +#endif + +#endif /* MM_FOLIO_ZERO_H */ diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 6d3f284c607d..0943ab724032 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -17,6 +17,7 @@ #include #include #include +#include "folio_zero.h" #include #include #include --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 353DD3AFD0B for ; Mon, 8 Jun 2026 08:36:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907809; cv=none; b=P4IP60Aqdp1VmB5Xk6xywsLCoZ2dhNiPIJTC9UNq3JhU6D6iNoyGkaZppjl/vPrQE2mchCgqxJudegy0H3uGQq7iQN7opN6KOdL3dUJquhCBRUfg5yEsnb/kb+PtKzu0zEZC6d0isFtAXUW5+QRlF10TgPhcRuGdh7VVdvPd0fE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907809; c=relaxed/simple; bh=pjwppoBrYRW1HQas7rkHTtLdWmdFN3pMjiX+S9iYC1E=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=H5IkR0AJRT43bUmkzmGzAZw5PKROcHsunxWvly1YzE7Jst8MkUSXziNnsD+5JzW+8Zxjji1oW2Qdhja1K8wWrBoXQW/M2muehVZcgpOpgdV0xayZAvH9psKWVFVy85Cxo+GtPfb/2Pz25CXbYo17JxvjwKNOmkSIRDL8JpaPW+4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=BrPXVh5+; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=HUG1KOYJ; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="BrPXVh5+"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="HUG1KOYJ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907801; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=8KJh4wvwDWohkhm2jB8sA/e6an56FCy42Cqcxiv2qXU=; b=BrPXVh5++1SufKZI3QM6wBBC7NGV+qEC1gUNq5R8H0u+jrik8DLCrVirFTf5u8QZvarCL4 Aiu13G93ZS+q4439E+5IyC7oYcobfKTMwWK6FAgT7x7noMebgnnrjgzj1RE/9mOaiAzC5e 4OvdjGPRJDv2eS2FYweG94rNGOsLZ0Y= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-402-7umxmQt0MeWi2pGOXe9qSw-1; Mon, 08 Jun 2026 04:36:39 -0400 X-MC-Unique: 7umxmQt0MeWi2pGOXe9qSw-1 X-Mimecast-MFC-AGG-ID: 7umxmQt0MeWi2pGOXe9qSw_1780907799 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-46010392f89so3680385f8f.2 for ; Mon, 08 Jun 2026 01:36:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907799; x=1781512599; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=8KJh4wvwDWohkhm2jB8sA/e6an56FCy42Cqcxiv2qXU=; b=HUG1KOYJ8D4JjZWJ3SkiN3iUEivraz/eTHDDITvdsDfcoiy6GS1dH0u3uvLuaJvefm C6Ib7GGTwdAUxGchy3bpPm2capcm+vGFGMLbNTdl7o1fVwyRFm6khO40u41fslkx2eih LMN5wAPfqfA0BM8odizP6OvLRklmh2f9r9mJTQ4Ubcd80j1mtfY4IMRzgBhZvCQZwk8K yVG4UKnMirK1XXyubwdsF2lhzjQJTpmovfjDsUetDHYy0iOSq53hkRGDc/oCZih5Q3O4 ROhBcbDGhk1yWWeB3L7Jsu/sXkaYS+c5tfTsIAp1FOATcyRSszK0fFnvh/IDxIQMBRZW 4tUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907799; x=1781512599; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8KJh4wvwDWohkhm2jB8sA/e6an56FCy42Cqcxiv2qXU=; b=Onv9KBZBQA2Q7jOFCWNhxmuJP0b5tVZpYKRw6daaPRIJUmIuzjCzeICAcLk9qRKDtc Eb4dE93qVxykw1TvpJl6nRv8ylR7BSCbulqQM8yA1lxM2Bhr/EzPy3/wqjHeU5L5iDNM /mA9FPQKKPNBoqf7s52m6rtdy8eP5bDsTcFn4E/RMjFhHHJMrIa8ZSU8xwHEtrk9S8U3 3w0YuGxZc6DSretQT2WcDEFi1D4egGdBAmapskhGskUM6oluqKqQZs90o9KXFK6pQ3Zt DK97p2jWVNDZSuIZlCX33bo1G0yqpMpECSYL4PXFFzJjTKJBLXZVAfiPv7UIaeUkcCkb hwZQ== X-Gm-Message-State: AOJu0Yyxet+z1fZBhMfbptEoO9YgoBvb3Pqry940DkjQrwN4gqOZlGDT 1lkyoXB3YaBIz+h0G2IvYf3UMlGtwrRlTM2Xpfqw72axAam9ctzHNo5+e8ypmBCZhJlB871dFgM shZ2bzsDupFAnOvqKW3YdqSpyxZFhcLacVlPV3cRoS8yYrcFrIohUy0c6LQFJl+E2FN8kGngS2g bX/RryCgzzX+fJlgvCWedqoMfLcM1G8ZU+R94lD0bbWJo= X-Gm-Gg: Acq92OGNQQvXhFSIY23sMYBItrUEHu1qIKivk0OObpVA1CNzpsfLh1j+HJMmGfCOfYZ Lf+Icf3w7eU7TmfnBF0YaApNCItctAJZZ2Vi7FFjzF7sS6YmrBHO9TV3OQ4EFWZ/pxmtSr/mAZi VAwFSolIHUhzYJRi2xyE8zDCU9mfUrBkObbxiMLQvdHu6e4SMJR80I14945HBMVoHvXmDBbwZxr VTyV33yG4RqfcJYXK5ozVnQMCrTSw0K4/Da6k0cfhUGKMv/y1F8b551KVzWajCH5S4so5cgs4vb fz29Pod1Jm/gaJjK6OUwOsyjQz06GvSdslNUeBcSu1E38f711o1ZvLIfIpfR680Kj1RSw0EAaxz dmjj0yH6Xa4VUJPUdakSOI1Y= X-Received: by 2002:a5d:658c:0:b0:45d:7bf0:c7d3 with SMTP id ffacd0b85a97d-46030643aa6mr15913854f8f.19.1780907798531; Mon, 08 Jun 2026 01:36:38 -0700 (PDT) X-Received: by 2002:a5d:658c:0:b0:45d:7bf0:c7d3 with SMTP id ffacd0b85a97d-46030643aa6mr15913766f8f.19.1780907797944; Mon, 08 Jun 2026 01:36:37 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4601f0a43e9sm50455796f8f.0.2026.06.08.01.36.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:36:37 -0700 (PDT) Date: Mon, 8 Jun 2026 04:36:20 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 11/37] mm: page_alloc: move prep_compound_page before post_alloc_hook Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Move prep_compound_page() before post_alloc_hook() in prep_new_page(). The next patch adds a folio_zero_user() call to post_alloc_hook(), which uses folio_nr_pages() to determine how many pages to zero. Without compound metadata set up first, folio_nr_pages() returns 1 for higher-order allocations, so only the first page would be zeroed. All other operations in post_alloc_hook() (arch_alloc_page, KASAN, debug, page owner, etc.) use raw page pointers with explicit order counts and are unaffected by this reordering. Also reorder compaction_alloc_noprof() for consistency. Compaction currently passes USER_ADDR_NONE so folio_zero_user() is not called there, but keeping the same ordering avoids a future tripping hazard. Reviewed-by: Gregory Price Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/compaction.c | 4 ++-- mm/page_alloc.c | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index 72684fe81e83..4336e433c99b 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -1849,10 +1849,10 @@ static struct folio *compaction_alloc_noprof(struct= folio *src, unsigned long da set_page_private(&freepage[size], start_order); } dst =3D (struct folio *)freepage; - post_alloc_hook(&dst->page, order, __GFP_MOVABLE, USER_ADDR_NONE); - set_page_refcounted(&dst->page); if (order) prep_compound_page(&dst->page, order); + post_alloc_hook(&dst->page, order, __GFP_MOVABLE, USER_ADDR_NONE); + set_page_refcounted(&dst->page); cc->nr_freepages -=3D 1 << order; cc->nr_migratepages -=3D 1 << order; return page_rmappable_folio(&dst->page); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 0943ab724032..4676fd49819e 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1874,11 +1874,11 @@ static void prep_new_page(struct page *page, unsign= ed int order, gfp_t gfp_flags unsigned int alloc_flags, unsigned long user_addr) { - post_alloc_hook(page, order, gfp_flags, user_addr); - if (order && (gfp_flags & __GFP_COMP)) prep_compound_page(page, order); =20 + post_alloc_hook(page, order, gfp_flags, user_addr); + /* * page is set pfmemalloc when ALLOC_NO_WATERMARKS was necessary to * allocate the page. The expectation is that the caller is taking --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8B4753B3C12 for ; Mon, 8 Jun 2026 08:36:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907822; cv=none; b=FzTklu71PaSyC41Frf7RnwZ5oJFGj8feRrlbx1ABCS4hBMK9hCU+MvGwfLuVXkzBtDJ1OeEk/pkaIYmuBOpKIvN1v3Pg2AwvuQxOlP2iKcB7AA2aSTIOfmbmvT3RCw2t+3WLrCN9jkR+5woa7KhphBagu1+bWeMNZOhwrCsqSc0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907822; c=relaxed/simple; bh=y+61YvUymd++YcmeYOOWPNpu5ycFdBgEhoiyVksZMFs=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=HrNW7Zg25nMZw/QXRn8p8Xhy6PcptUuPYhJDQA7zdQX1/PHMGHU1JH3aWtuu6pXIlkO9iFWOFFu/rGesE6NeDOQ5noeXHs7K9DpEAtjN+pe2w1fEBbNuUV93WeVEL7RbhNdpmWE3TVYWuU8oefaLElj0zEO4HtNSKUMrTvIzpC4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=BneHsa1B; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=T+sWV4YW; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="BneHsa1B"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="T+sWV4YW" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907815; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=AR0svr1Wd/OcNae1i4toNU59Y7b0yC0WCO1bghbpkVI=; b=BneHsa1BpvEcXMOMe/1Pt681Ii5Lv7M41cUyVd8Iqkzn789l/WCNGXmYHQmnXfuQB5S9/u 4rUFUuh6zZMk4ojqqZTVuOKvYscOl33kolTYIKuKq80SaELj+t+ugxiI7yb9FZ2AtlS8Y2 6MVGq9UCcyUUVHxYDFkaxzR2grrSXY8= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-88-Zs3lGsGRPzqjWgewi3ArCQ-1; Mon, 08 Jun 2026 04:36:53 -0400 X-MC-Unique: Zs3lGsGRPzqjWgewi3ArCQ-1 X-Mimecast-MFC-AGG-ID: Zs3lGsGRPzqjWgewi3ArCQ_1780907812 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-490bde3d239so31373385e9.1 for ; Mon, 08 Jun 2026 01:36:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907812; x=1781512612; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=AR0svr1Wd/OcNae1i4toNU59Y7b0yC0WCO1bghbpkVI=; b=T+sWV4YWb8mvzDZzbqOeNKoOnAvRVGld8RoxVec7glzwwSX52AJFpzeE+f8lmKbahz m6lQcz14aeJjysJjmsq8LM3kC8HRBthIA7LtKFU9dqtfKD2IQWucdErDNVXgfHDsRBiq 7KnYajrGnYXwDFybc7PU4SnOxROHXX4zTfCTQ8V3dKUGFsxcRQbEiDBncWZIJj9nGiju yJQIvau4PuCw08GFTZPHLw4hd/wu7fJEotIZyKd1NeajdnfnYkTMrE08vasEefSS25XN jij2yS+QgMf1U4XLRD1wpYHPnR0blQyReB1zYuBqlWwZ7eSPHtlIO4VC3gvjkQMEx4yy c/5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907812; x=1781512612; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AR0svr1Wd/OcNae1i4toNU59Y7b0yC0WCO1bghbpkVI=; b=LN94RTCjO3v5OzwQ/zZGHdt8cSiDLsf2zCRBbpGoGDM4kKyNSjFqwU1qqhdp96HwT+ IaDso2qL4bNuPgNo3NOhn8AfSpfRKAi+Z8TS/wCF0N7dr/382lJqoi1saIuUUbVPewHi WNvSbsmOBCT33WA6G6EoaIVXUSE20vRV8g6rkX10hmKgIAhH6TpMBvYNrxHmLYmlEfcG ppXc9vw9fSP6KNaZ8ABRQTp2gAa9xGm5S5fZBcmw6/i7YGaDUizv2qtEg4iLeHRHgJxA t6n4U0EnnU0dBOwzX+zw0LqeewgE+WkWRAYVd5z1rNyVuwQjwniVpXBd11rWFrkic04O xyMA== X-Gm-Message-State: AOJu0Yy6l4Gr22A1JMxs+MNcdR4KiA/VBq+fIuVK1vw7H+svDd3uni7n FxgzvZbl7LOjZfPVqFwOV1jQhFNkerW1OMfVKUol8LPeQq76QDKCHcYdC4E/KWIrH4fFqwJo8Cz 0lbuQoFL4VmV97LA6pXKhi+l4l2RBi56D2KxuO5Ez0FkqgzEHeKaR0sBt6SrhamshyLy4ytoVgZ +7VgTfGOx6VWxnd14XXQPLzQZtoRV6EPWzP+YajC3h5DU= X-Gm-Gg: Acq92OGj+g1GCRy+Y4qysTveHKTpxwq0q6IxGWirBaVB2/lzinsT1nSlYHzMPsNw9Tt HWJ7lb2itvYs2b/xpky3nx6AS1Zi3WnCGlOKe7AxMK2KxLidEqd18/23BJP0NlYJ8oVcPV/xHpe YBStwL/yptRza3L7BBSFJc4kWkFEL7XZGqEDPVpze6iLq5+yMEpEUAMrX1Xm+jjkJfXOunOSJnw hzQUutZPeypAHdsLYgki8O2rBUoFpvAYOuDiV2tyuqS9bNiEvMLOPBBnfkAAbaqhDoOVTUvutYV oV125z24t3e7gJsGwD9beClzu00HstQnU6PMn+zZ4jYh7zMxh+f9XjX1AnAKjPGrydCk2IhQivU koKv9bOqXwQponPEbu+uBIZo= X-Received: by 2002:a05:600c:4ec7:b0:490:b8ee:d6a5 with SMTP id 5b1f17b1804b1-490c2589927mr241065885e9.6.1780907811971; Mon, 08 Jun 2026 01:36:51 -0700 (PDT) X-Received: by 2002:a05:600c:4ec7:b0:490:b8ee:d6a5 with SMTP id 5b1f17b1804b1-490c2589927mr241064915e9.6.1780907811451; Mon, 08 Jun 2026 01:36:51 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-490bc39e024sm500006545e9.4.2026.06.08.01.36.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:36:51 -0700 (PDT) Date: Mon, 8 Jun 2026 04:36:38 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 12/37] mm: use folio_zero_user for user pages in post_alloc_hook Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When post_alloc_hook() needs to zero a page for an explicit __GFP_ZERO allocation for a user page (user_addr is set), use folio_zero_us= er() instead of kernel_init_pages(). This zeros near the faulting address last, keeping those cachelines hot for the impending user access. folio_zero_user() is only used for explicit __GFP_ZERO, not for init_on_alloc. On architectures with virtually-indexed caches (e.g., ARM), clear_user_highpage() performs per-line cache operations; using it for init_on_alloc would add overhead that kernel_init_pages() avoids (the page fault path flushes the cache at PTE installation time regardless). No functional change yet: current callers do not pass __GFP_ZERO for user pages (they zero at the callsite instead). Subsequent patches will convert them. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/page_alloc.c | 35 ++++++++++++++++++++++++++++++++--- 1 file changed, 32 insertions(+), 3 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 4676fd49819e..d4fbf1861a8a 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1861,9 +1861,38 @@ inline void post_alloc_hook(struct page *page, unsig= ned int order, for (i =3D 0; i !=3D 1 << order; ++i) page_kasan_tag_reset(page + i); } - /* If memory is still not initialized, initialize it now. */ - if (init) - kernel_init_pages(page, 1 << order); + /* + * On architectures with cache aliasing, pages zeroed via the + * kernel direct map (e.g. init_on_free) must be re-zeroed + * through a user-congruent mapping. Host-zeroed pages + * (zeroed flag) don't need this: physical RAM is clean. + */ + if (!init && (gfp_flags & __GFP_ZERO) && + user_addr !=3D USER_ADDR_NONE && + user_alloc_needs_zeroing()) + init =3D true; + /* + * If memory is still not initialized, initialize it now. + * When __GFP_ZERO was explicitly requested and user_addr is set, + * use folio_zero_user() which zeros near the faulting address + * last, keeping those cachelines hot. For init_on_alloc, use + * kernel_init_pages() to avoid unnecessary cache flush overhead + * on architectures with virtually-indexed caches. + */ + if (init) { + if ((gfp_flags & __GFP_ZERO) && user_addr !=3D USER_ADDR_NONE) { + /* + * folio_zero_user relies on folio_nr_pages which + * requires __GFP_COMP for order > 0. All user folio + * allocations set __GFP_COMP via __folio_alloc. + * user_addr !=3D USER_ADDR_NONE implies sleepable + * context (user page fault). + */ + VM_WARN_ON_ONCE(order && !(gfp_flags & __GFP_COMP)); + folio_zero_user(page_folio(page), user_addr); + } else + kernel_init_pages(page, 1 << order); + } =20 set_page_owner(page, order, gfp_flags); page_table_check_alloc(page, order); --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BFD123B0ACB for ; Mon, 8 Jun 2026 08:37:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907845; cv=none; b=Uxt1gb89dEF3qAlD+Jdtjn1iLWvaX6pL7o9UoJ8rsrAC6bwWNhUdBluxewkKzZpaCSNl2CGEB6G4UWTFt7ydpU3oU9Zf1H7Q9v4esPzQbigmFFM6dfmamID4e4JO3JHyG2KBYUTw52CV6nQU66iTPswVWoGsDev49bs3T7G4T3g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907845; c=relaxed/simple; bh=K42PFe6eLQ++q/LDnGpg3rhqNCEgPv/tPyRqfoqSjPs=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=IEM+2u92mX8hHaOWVjQfuNStST1IqrQcUbKB9f5xWJQcCTzU3mxVe4D7Old1lICCP340s459bHbElPSKtLHUDb1eErdHcoSi3NOZOxnYcQQP3V8JjOxvNXPga5Tysf3HMGmLwrP7gHiRYNFDF16TIgL2o63yEP7kPdORkqAtM34= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=bOgDZ6Zx; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=VHD+ZOa5; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="bOgDZ6Zx"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="VHD+ZOa5" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907837; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=A0GMQhTq7ZqaOlkJiK0dDinX7wgaWZxvF9PWDzI1B0I=; b=bOgDZ6Zx4YFrUWxzFrYh5VPJ00N3aWv4rVKmUiVQMbWQp7xThGiX1dMp0NQU8h14CkOvD1 gxaIIX3/ZQgEgQpoPhXwhZoyuZo0BDcpxAhL66RdE33AQ35V4iFRq8wMIl+DJHM7EZxY05 8G7Xh+a/WDr6wK3bXFTH+BIeVgRsFgU= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-36-1zHhiuknMLG0Fw9MR5DVHQ-1; Mon, 08 Jun 2026 04:37:11 -0400 X-MC-Unique: 1zHhiuknMLG0Fw9MR5DVHQ-1 X-Mimecast-MFC-AGG-ID: 1zHhiuknMLG0Fw9MR5DVHQ_1780907830 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-4601daf4c65so2320105f8f.2 for ; Mon, 08 Jun 2026 01:37:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907830; x=1781512630; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=A0GMQhTq7ZqaOlkJiK0dDinX7wgaWZxvF9PWDzI1B0I=; b=VHD+ZOa5esc2ZRFWZK4M01mdEjB8UGO7MvRGf+WVkr+mxwMtU/nZApmO/g5iphJlwh Xc8RHEUNn/tdRhjZk6d8p7r4vaKFRgJv2jb2pYYvfDkhkMEuRODEr2FZkGGMoFlri7kG csP8p2JFmwtffcJganEx62H4xt0keYl0CNU86fIydeusm6UqO0JOfGBU1CXwLCk31wYz k+NVb4ErQaTxPnaMWisWubBEEuiQX55qDvUgHcX3hCfDtcOIrwZ4V+fjoa5QsCStsqFO tzu1mGewn3UB2xNp8K6o0Qg0rQkFwzsKDy9O/0oJn8rC++iDG8oDUipLZgj5LaclvZmt ZsCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907830; x=1781512630; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=A0GMQhTq7ZqaOlkJiK0dDinX7wgaWZxvF9PWDzI1B0I=; b=gCTT70or6+UG3Oft8O1HVUV5ZHcSwCFj2TO1K3r1IwM9szv2SbVdJtEabCw7elO3Ry hhIhsWFDSLQ6OJdOYclnADmQAsTZNlvagk9rKFFC65IvygBTU9W53EI+1HhmKbXeRWy9 sCW+hZ+6rF30/V/qAFk010xCXqdU7i3hfzjakHL15RI4EVimXkb5jQ4PIqSB0nOrDYAz Fr4mIF8mJ/4hYRkzKlbqQDIf7/BUHqRkNO2cEgeDTc/x+gM8YwOEczMPbZLCs9Qed9Qf zhK2RNtJufvSXyUH1z7FKrABPXP+RZaJA5/3qDHo/GaUQbAqzn7ZFoeQVwoLbT98YXfA N4AQ== X-Gm-Message-State: AOJu0YzBq4J8WesVa5DpPRNfrg0LABD7leFaEkq2fDCQ3Hi5W6k7BzXP u/O9KjcuwpIh+Y5wKfJQbixEmmCn2eLsaRP/nwyo3hxHfGA6614QKn6vmF6Vc+hN48p8fmHHxla pAeyjeS2EyTz8eW/mJyJ/RVC0tpdWUKqAMTg/94yaieJNfqdiRn0/o08a61PrSyX+oypUOsrmiH W502m9W/oNAA5bFpNA6BM1GLHYtpFqeIqyQQ+kTDjEUcM= X-Gm-Gg: Acq92OHcy0kxI6siaw/Dm+u1n/7FNNNMr9LwCY/9RdSmU9w/PieIy1UBMO9FmglzpZD fqtdTjazBM95+34Zm59jNu0g69ym2jcfmKxcxA79sWGSj2PI4ocMdnrbl9mlrFyRjxjm+guaJTX dfUW1byXhg75sOynVUEUqp69BfMGaV09Et6Wy9FFm2J50H/W1QA2++qBmh1zyzjyuybtPPZguzP As8jY3VyZYzgMQcBPSjGxlYDxGIESk/uVeA/wquBYF8zrlYBnJImtFjLd/N9A0JoMcLm7+SeFyO nZ674BCad3vHnm3oUq17xXNafAELb5Qb8/LHjyVffTHck8EhrKdY8mSm9GbbwhpN04OGwpf2mci 7UzufAEo+uV4P1szW2zjI8gE= X-Received: by 2002:a05:6000:137a:b0:43c:fdd:ea96 with SMTP id ffacd0b85a97d-46030648ba8mr16182381f8f.26.1780907828703; Mon, 08 Jun 2026 01:37:08 -0700 (PDT) X-Received: by 2002:a05:6000:137a:b0:43c:fdd:ea96 with SMTP id ffacd0b85a97d-46030648ba8mr16182281f8f.26.1780907827908; Mon, 08 Jun 2026 01:37:07 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4601f2f5612sm48449848f8f.15.2026.06.08.01.36.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:37:07 -0700 (PDT) Date: Mon, 8 Jun 2026 04:36:51 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 13/37] mm: use __GFP_ZERO in vma_alloc_zeroed_movable_folio Message-ID: <862a59e98104db76ee3998460bfc55e937c218c4.1780906288.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that post_alloc_hook() handles cache-friendly user page zeroing via folio_zero_user(), convert vma_alloc_zeroed_movable_folio() to pass __GFP_ZERO instead of zeroing at the callsite. Note: before this series, replacing clear_user_highpage() with __GFP_ZERO was unsafe on cache-aliasing architectures because __GFP_ZERO uses clear_page() without a dcache flush. With this series, it is safe if the caller passes a valid user address (not USER_ADDR_NONE) to vma_alloc_folio() etc., which delivers it to post_alloc_hook() for the dcache flush via folio_zero_user(). It is only unsafe if USER_ADDR_NONE is passed. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/highmem.h | 9 ++------- 1 file changed, 2 insertions(+), 7 deletions(-) diff --git a/include/linux/highmem.h b/include/linux/highmem.h index d7aac9de1c8a..8b0afaabbc6e 100644 --- a/include/linux/highmem.h +++ b/include/linux/highmem.h @@ -320,13 +320,8 @@ static inline struct folio *vma_alloc_zeroed_movable_folio(struct vm_area_struct *vma, unsigned long vaddr) { - struct folio *folio; - - folio =3D vma_alloc_folio(GFP_HIGHUSER_MOVABLE, 0, vma, vaddr); - if (folio && user_alloc_needs_zeroing()) - clear_user_highpage(&folio->page, vaddr); - - return folio; + return vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, + 0, vma, vaddr); } #endif =20 --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B99333B3C0E for ; Mon, 8 Jun 2026 08:37:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907856; cv=none; b=rt7imrS4WgbPinoYZCUYQibIkeRJ31rQaqe4/M8NsVvx2bW61ZjCaCAfUsAjyzukLwAMYEtAqlOJadYtviK2DDj+wYEVbM5ORiwYUFcnvvIGAFis4rdpeymvNuUmY/jgDSN8cQCjEOkykyNcxoeTKX1Xnb3Bix8scNs899hu/jk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907856; c=relaxed/simple; bh=OHWxGJkJbozNCm8KAs+71lxyLWxsCrak///2e95AF4U=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=fdbC87GO/DObYxcBxdABFz8Fu7fnOX9jzqoKK4V4U+efzGOOVFsxx82IKJc7c3klWA5ja6GaolT3ozbQGGMXP/ca4xk/3+ukdnaeEMN4E5wvTCPk7OUeYgfIdTVZXsEYHRe2STAXFReQzTDekmnKwUeOp3lbjWFKTq+Bk2vVoVk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=C4mYpJ4h; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=DqZeLELI; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="C4mYpJ4h"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="DqZeLELI" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907848; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=tltobG27QNK6CnGoNCrb5i/6pgzOGIIyEcQIJDK7QRA=; b=C4mYpJ4heQzCYvdeu/nEgxW+X1RzKSmckIqC2soB/fOjfu/8+oRTRNBtZ0IpwdetPkq6DM LfUNk7hy7WxU/I0I3V3ctAE0oqIuiGj2GnMH6KyDMyjmARoAbX6nhc57fJJBNyzFQy0yFN Kaj76xu8M4bYornHny05vuDyOXX82Lc= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-660-7mMXZoMVPqWT9eWq3POQMQ-1; Mon, 08 Jun 2026 04:37:27 -0400 X-MC-Unique: 7mMXZoMVPqWT9eWq3POQMQ-1 X-Mimecast-MFC-AGG-ID: 7mMXZoMVPqWT9eWq3POQMQ_1780907846 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-490ae461f8dso29989475e9.1 for ; Mon, 08 Jun 2026 01:37:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907846; x=1781512646; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=tltobG27QNK6CnGoNCrb5i/6pgzOGIIyEcQIJDK7QRA=; b=DqZeLELIbNoC/sWKq7PP1z3ZvRRIfg4C8VX+UxGGOOXLqUxolObQ6TrDCe2m9MDkrj Y3Iz3kxZjXFT8+eeUoVyJXCPIb4fYlgyHf4njl5HvMaECBm1g9z03SwooUdmbdGM8yA+ 1ynEOltDmUcfEnyaEnssg9+kLfpAmk9N6Wsba4/pW2JMUwYlPy/cAVYXa5K0yHBzaxv4 MbOlKp0R9M7KvBrCSUp3tDNFM4m3yz3cMISvoW4bgFMsuFJzd+kW+rtoplJdVSHQ4LtO 123rysbIpvkINK27S/UI3PdcrEJCYg/b/O/43cgT9iJb6Ipy1l8q8f0Qc3LjGkCMAG2O 75mw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907846; x=1781512646; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tltobG27QNK6CnGoNCrb5i/6pgzOGIIyEcQIJDK7QRA=; b=p0Ny2A9f+PEKjJYEBNpTwrWX/4jqdIxvd/oOUHjfLpG8W8M2TIVmu6XHYbDg4nTlX9 kmdVYBV3k13g9Ujjk07jBaU/JT1oMlhCdTExnWNZ5tO66QD65biXw4Du8MdF5HPyOjv8 1EH/XIRa1bd2Oyy6fL8E/qXwRfHqCWusX7yllZfwmjCHgDnSCdfIjfbGNmvjVYa3P0+G HMXZpOz28FWyzX0qZzXGLrwqyTEFtCNyli4bZwX907M+RT02Jft0AprdbH5ABDdINUvg ly6RVlKt4V4qEsrlpFBFP4+73ouQs3jNTk5saOIVGyROlrFLiKVrZ9ISQb8SDNF6S3uO uKTg== X-Gm-Message-State: AOJu0YwwDYVuT4AedvTw77v1WhTtY1+lEAyM0ibrmh5dN8fIQDAXJN10 +y2ZV2E4WuPYQ+msvwZuixYDSGq/cwbhNTRlxYDzoAlnICMD2JA9Kumnvmf4Y1yodJjy3aZmiml hIknjifBlXcs3CFvF7qZnN+/0vh+DTG5Ah3WqYc9cTXEF0+m4JgxDeCVyK0OSiiuh8AX0peW6wu SHOnmExsiyMR8KTTr24YYMQODOYnH18+gibawyqdtogSg= X-Gm-Gg: Acq92OFBgJ7EjeqStzXmgY3GI8l88isMOg/Aq416B6+N5aQFP4VVUAfruTw86QnjhoC pymbLxEMucufz3WC3UkK6noBZp0Zwgua5nLpXri6FxLSus4RciZrREIAacGpgNRP+icsJhYuVft MVc7J5p54cpwHvHfxPfQBzW7zDMLgHBJzD6en6pjJriFx6UN+82zuS+IdZLgnXI00IzyZAN2EMb ik1QbZI1vpukf3jbBPR2t+JvNQ1XIBqaYY0Pl1qA3NX4uLzDh/aREDKoz+XNsABjC9FUEMT1RKQ 9gnLseqymC5znUwxIY/Bmh2wsyZE0rdZJPlp4NKGt74bibuw+V0VVUo2kYqF5aiq26nsUnn4LWe VZK2VXePS1taQeqhDYkt9Sq4= X-Received: by 2002:a05:600c:4fc6:b0:48f:f7cc:f6f5 with SMTP id 5b1f17b1804b1-490c2614167mr244773485e9.31.1780907845823; Mon, 08 Jun 2026 01:37:25 -0700 (PDT) X-Received: by 2002:a05:600c:4fc6:b0:48f:f7cc:f6f5 with SMTP id 5b1f17b1804b1-490c2614167mr244772255e9.31.1780907845077; Mon, 08 Jun 2026 01:37:25 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-490bc3e5a00sm375786835e9.15.2026.06.08.01.37.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:37:24 -0700 (PDT) Date: Mon, 8 Jun 2026 04:37:08 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli , Magnus Lindholm , Greg Ungerer , Geert Uytterhoeven Subject: [PATCH v10 14/37] mm: remove arch vma_alloc_zeroed_movable_folio overrides Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that the generic vma_alloc_zeroed_movable_folio() uses __GFP_ZERO, the arch-specific macros on alpha, m68k, s390, and x86 that did the same thing are redundant. Remove them. arm64 is not affected: it has a real function override that handles MTE tag zeroing, not just __GFP_ZERO. Suggested-by: David Hildenbrand Acked-by: Magnus Lindholm Acked-by: Greg Ungerer Acked-by: Geert Uytterhoeven # m68k Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Reviewed-by: Gregory Price --- arch/alpha/include/asm/page.h | 3 --- arch/m68k/include/asm/page_no.h | 3 --- arch/s390/include/asm/page.h | 3 --- arch/x86/include/asm/page.h | 3 --- include/linux/highmem.h | 8 +++++--- 5 files changed, 5 insertions(+), 15 deletions(-) diff --git a/arch/alpha/include/asm/page.h b/arch/alpha/include/asm/page.h index 59d01f9b77f6..4327029cd660 100644 --- a/arch/alpha/include/asm/page.h +++ b/arch/alpha/include/asm/page.h @@ -12,9 +12,6 @@ =20 extern void clear_page(void *page); =20 -#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ - vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr) - extern void copy_page(void * _to, void * _from); #define copy_user_page(to, from, vaddr, pg) copy_page(to, from) =20 diff --git a/arch/m68k/include/asm/page_no.h b/arch/m68k/include/asm/page_n= o.h index d2532bc407ef..f511b763a235 100644 --- a/arch/m68k/include/asm/page_no.h +++ b/arch/m68k/include/asm/page_no.h @@ -12,9 +12,6 @@ extern unsigned long memory_end; =20 #define copy_user_page(to, from, vaddr, pg) copy_page(to, from) =20 -#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ - vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr) - #define __pa(vaddr) ((unsigned long)(vaddr)) #define __va(paddr) ((void *)((unsigned long)(paddr))) =20 diff --git a/arch/s390/include/asm/page.h b/arch/s390/include/asm/page.h index 56da819a79e6..e995d2a413f9 100644 --- a/arch/s390/include/asm/page.h +++ b/arch/s390/include/asm/page.h @@ -67,9 +67,6 @@ static inline void copy_page(void *to, void *from) =20 #define copy_user_page(to, from, vaddr, pg) copy_page(to, from) =20 -#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ - vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr) - #ifdef CONFIG_STRICT_MM_TYPECHECKS #define STRICT_MM_TYPECHECKS #endif diff --git a/arch/x86/include/asm/page.h b/arch/x86/include/asm/page.h index 416dc88e35c1..92fa975b46f3 100644 --- a/arch/x86/include/asm/page.h +++ b/arch/x86/include/asm/page.h @@ -28,9 +28,6 @@ static inline void copy_user_page(void *to, void *from, u= nsigned long vaddr, copy_page(to, from); } =20 -#define vma_alloc_zeroed_movable_folio(vma, vaddr) \ - vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr) - #ifndef __pa #define __pa(x) __phys_addr((unsigned long)(x)) #endif diff --git a/include/linux/highmem.h b/include/linux/highmem.h index 8b0afaabbc6e..642718a50c27 100644 --- a/include/linux/highmem.h +++ b/include/linux/highmem.h @@ -303,7 +303,6 @@ static inline void clear_user_highpages(struct page *pa= ge, unsigned long vaddr, #endif } =20 -#ifndef vma_alloc_zeroed_movable_folio /** * vma_alloc_zeroed_movable_folio - Allocate a zeroed page for a VMA. * @vma: The VMA the page is to be allocated for. @@ -317,12 +316,15 @@ static inline void clear_user_highpages(struct page *= page, unsigned long vaddr, * we are out of memory. */ static inline -struct folio *vma_alloc_zeroed_movable_folio(struct vm_area_struct *vma, +struct folio *vma_alloc_zeroed_movable_folio_noprof(struct vm_area_struct = *vma, unsigned long vaddr) { - return vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, + return vma_alloc_folio_noprof(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr); } +#ifndef vma_alloc_zeroed_movable_folio +#define vma_alloc_zeroed_movable_folio(...) \ + alloc_hooks(vma_alloc_zeroed_movable_folio_noprof(__VA_ARGS__)) #endif =20 static inline void clear_highpage(struct page *page) --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 828473B27D8 for ; Mon, 8 Jun 2026 08:37:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907870; cv=none; b=GanrFCFrPnpCUJKKkTfVmVHSD+q1csEin4ChCrZ9Ta1nAUPgS5z+pfZSK/RVQutCa5HKn6TmOaIqdZUbSTo1wJrofYGsN8un3OC5pqEiroCDFz/zJuUFz1qEI+wW36V5gtgXhhVzCMh1KZSL0XAmbUZQoh5PGUw74P5+coYDuNM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907870; c=relaxed/simple; bh=1zyTUpR/sEiqxyRibjnxckO9DMK4cSEFuMxiLpElRYk=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=hHLO2zjL7fk1xJ3qdn7ZFohqHWFcfCjvB6VyPhJFKYoe2hPBEgwnQ3ZstcItADleMxzXH8FkGEYOqia5ByWiDgXRNJtVHrcbpJ1UWgDVAsr4/9hxjJr0b88y8eQAVq6LSYd5F3xkUj+MEQoHbjHL9xv7NNL5QEhft3QuQE7tfQk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=a/5XROuV; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=k74TZXbc; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="a/5XROuV"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="k74TZXbc" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907864; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=5nfUkrh3UXXGBEBXNDRnqOIydAOk6SYQWb2HusQDLsM=; b=a/5XROuVFhFXf8AlswwpL9GXWpSJHYH6hwUdVBWff4UwjwzCN2NEiP3TXF+LD2GNacTzxF qHNST+PsVMeOPnnimWjLCv7T7LXRR4mUQs392Dq6G0qe92ieoYYhWnUkPXOwzF8rFISd72 rdDBOhmQOkBwevN0znrrCC+smvlg2oU= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-277-U65kSEs7M9K2aJ8PpKuV1w-1; Mon, 08 Jun 2026 04:37:43 -0400 X-MC-Unique: U65kSEs7M9K2aJ8PpKuV1w-1 X-Mimecast-MFC-AGG-ID: U65kSEs7M9K2aJ8PpKuV1w_1780907862 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-45eec2ba533so2790903f8f.3 for ; Mon, 08 Jun 2026 01:37:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907862; x=1781512662; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=5nfUkrh3UXXGBEBXNDRnqOIydAOk6SYQWb2HusQDLsM=; b=k74TZXbcHF1LDr8JcQnKN5cP073QTdkBpRWdrG7sgZo8F8HciPNDMHRvzzrEmZQuMb SFpc2UOQcfUyeyuEwuYYQC83nVVKFQFwGApNYgX7E9vwzyWPNFVhBitt9nMWMcx+T8/3 EA0GL9mCPti0vPqT+gETTfGil6aEpspxG+Yxk7NBgFQ1LRQvbUdpCcLvkYpF0J/Yr/UN N16NiNHJpG9HzETDa+jMe9f7fLnDWs91TT5b+En7/D61tb4rXDMHbesTdQ4gFvspgDlH gmACo92NnnqW/m3LQmNuWBCErrqKrNhpWSnWBaoaf20quqs4uUp4VxThz+blxVqKkoWj E0RQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907862; x=1781512662; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=5nfUkrh3UXXGBEBXNDRnqOIydAOk6SYQWb2HusQDLsM=; b=oV84Dzt1r83iPG3neArEQmf8WI3FZIccUc6OnENeOkliYjQrt9egJmHox6tttMT/Sv CB0dUiRKtEp4bgs/6GEpL4g0xx6h7iVCw4/m97cMR9mZ3hyby0TN3HEnnPQ/7rIsAd8F ItzbAOetn/Bf5aALi55mAzRPLUHzNOJOmeNsZXNRjezzKBTF4cMLsv0K5HSwiaAeGQyC 6Z97PAJ+YSgJ5utcdAvUMi8p+q1H0jEmyZzbvab3/vzFEmc9qflKrslbSTaf2YVp1cit wO7CYDAmAXsAdyvw+miIkD57ZwXwia9DckKWITLjLJPv65Kl4pDiXe51DFdA5YbQp2Wc F5Ow== X-Gm-Message-State: AOJu0YxRvcjmeXdy6kX+bGvEsyv77BX1W0V/rTazFOa66spyML5j3dwv 18L/xpg3izjnmvtYcFyTCNq5RhlxwhuTCXjKNpXAYLi4In+wlmi8Kq2JbbpfbWxYBpqAmBtCT75 3d9YtiuoGf+ntIy0BLhql3Y79DXscym2kpaSnzDrzoqnSB6vrzVBZ1xjh23S44FGKbUhjDN0Hs+ /SscCdKO/6I3pEkE6Kkj1hcsaYduQdk7SgqVXmqUn2Aqo= X-Gm-Gg: Acq92OHmO1YkX/PRF1CevjxknNUWDm9KVU50mSlT4OXmTqUR7QWqNXO+CrJDlJDsIYE WwNDnnsKNt8fexWlnKKMhaL5ALAn3kClJDZwntubYxFa6VLC9FHZi1BARsoxt3iq3/YTJ2/6Fov a5txieGVyLEtZwCgD7NXguFSFfxrgX0GZ/ihMt1ynwDm2SeNKK81HXzmNEQ0Zw0lJ6bsoyKwE4g JmM53novVACTRhikVoiCO48jVdD8u0FgxbG1Triddpl8X960iPg8jU5Sf2KSRadhyTqcf37vbMt DJJ+739CAVeMnN16BSELJhLCgvJVO3EpjK1wzg9JAyeuJZngYAzwTcjas4bL5OOGM3RVlpajxvg C7zHreR+7qQVaz484920Nhf4= X-Received: by 2002:a05:6000:25c4:b0:45e:e3ee:98b1 with SMTP id ffacd0b85a97d-4603061f968mr23332612f8f.32.1780907861851; Mon, 08 Jun 2026 01:37:41 -0700 (PDT) X-Received: by 2002:a05:6000:25c4:b0:45e:e3ee:98b1 with SMTP id ffacd0b85a97d-4603061f968mr23332507f8f.32.1780907861145; Mon, 08 Jun 2026 01:37:41 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4601f368e9fsm49322612f8f.37.2026.06.08.01.37.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:37:40 -0700 (PDT) Date: Mon, 8 Jun 2026 04:37:25 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 15/37] mm: alloc_anon_folio: pass raw fault address to vma_alloc_folio Message-ID: <2e931dc3daa76b57eaedd5b2d3d49f1075797252.1780906288.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Pass vmf->address directly instead of ALIGN_DOWN(vmf->address, ...). NUMA interleave is not affected: the ilx calculation in get_vma_policy() shifts addr >> PAGE_SHIFT >> order, which drops sub-page bits regardless of alignment. post_alloc_hook will use the raw address for cache-friendly zeroing via folio_zero_user(). Signed-off-by: Michael S. Tsirkin Reviewed-by: Gregory Price Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/memory.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 86a973119bd4..21f640674c4f 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5268,8 +5268,7 @@ static struct folio *alloc_anon_folio(struct vm_fault= *vmf) /* Try allocating the highest of the remaining orders. */ gfp =3D vma_thp_gfp_mask(vma); while (orders) { - addr =3D ALIGN_DOWN(vmf->address, PAGE_SIZE << order); - folio =3D vma_alloc_folio(gfp, order, vma, addr); + folio =3D vma_alloc_folio(gfp, order, vma, vmf->address); if (folio) { if (mem_cgroup_charge(folio, vma->vm_mm, gfp)) { count_mthp_stat(order, MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE); --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DB1BB3B3BFE for ; Mon, 8 Jun 2026 08:37:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907881; cv=none; b=kmOVKLcU2SrxBxcYXvIRD96n5eWUV+JOuyCEVD6nBa9TnzUmoTMFeqWr2bGKlHsgy3YLrjYh2SEwiailwTox/rlFPMN7s4nA4c5zKUX+JBzjA6vsib44S4Gi8wMvKayTBz/dQdWP2qOTrvy1pfbC1zxzY4wsRwuLPKLFta7IG/o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907881; c=relaxed/simple; bh=TnddcwSWkBVpMxUBPESy99JTSMn25mXLp3afUCWRn7U=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=RDz+7t7FSKM3w7I01bn7Nti1iyZopbJJmGlXXLWpVXYge1IbtaECazthqRFnCzbNBSzUlWo+BPGz1uMqByYeUgLUT7lUTJ1wwb3d+nvrha5AtjCE1BevXI4Hzp1nNj4C6rU1FS5x+99itcrxTWxOYVmSHwTXXBlqXrjYKqBQh4U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=U8PBVLBm; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=JpIok/ZG; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="U8PBVLBm"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="JpIok/ZG" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907871; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=Zy4rtaIprLz/f2YXF/2fwOOOF4wr4AdMW87y/Btd5qo=; b=U8PBVLBmP96HJJDJdtgrEKs7d0gWNuRRD/mS+4EgusKdFAHiSxjmPwdaFbbD4d8ISG19dz QdLkkaysLpBL+4aTtTZOORN5QqAry9xehrT1FY88CyLR1MpuFYebLEcHegMQkpzD2cylXJ fMQeWVz5xasbitNA7+JjV6bN3kqdzkk= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-88-9P5-6xwjMtGn-9e6LMyCsg-1; Mon, 08 Jun 2026 04:37:50 -0400 X-MC-Unique: 9P5-6xwjMtGn-9e6LMyCsg-1 X-Mimecast-MFC-AGG-ID: 9P5-6xwjMtGn-9e6LMyCsg_1780907870 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-490a786f987so47024965e9.3 for ; Mon, 08 Jun 2026 01:37:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907869; x=1781512669; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=Zy4rtaIprLz/f2YXF/2fwOOOF4wr4AdMW87y/Btd5qo=; b=JpIok/ZGWj65xny7yMgNlOzRehX1crG4LioqEk2nvHh8iCRuFI/VLE4JB+I0ygBfYK qVsDtnbuTWh7UwWhhcBk5mHhJPzuz7DchS34y5mIifQfONzkTdlDk6lKulJqZsQhH6VF 9sqRz0ZNSOxfvfL47HTlyrjkh2PYcOcRZE6esJTSXQWLBQn9suH1b7/jfiCbwTpOpiA4 Cp+IpkXE9i+EEie4ToehP4n3OlmnfTUVE6ULKq/AkJWcC8m//4vZZ1II37HMkfumbGU1 UA7ldS/DfmGig7Bx3mx1kyFxloAkoc3ezXrlQA0iltB0Wi6qMZs4nIkDGrORzOs2Onv2 wapA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907869; x=1781512669; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Zy4rtaIprLz/f2YXF/2fwOOOF4wr4AdMW87y/Btd5qo=; b=libLwjIDkoefYxVs/3rLiF/Xfgu7Ik7kt8xEZ+J1jM25TWO4O1jJ3CRHz6DMGybSAf yC090XS5sSzj3BWQkeilW8ZXE+5iK0wBS9KmnmJknnvwnJSe033niQ9QdhMkEeoTVwj+ raAOVguWxyKf+YCD78m2ZoYItF9oCq63QfQgjkesHztwQTy3aVOqHl5TAXYBwRRZGJL0 E57I46/Zp9GEZiX5wvyz133w+BcAA32t4QuGpKB7sY4Zl37TswgkEYrKZ6FT2XBU/nmw Ic5q8uwec/vZW5PIP7/zO2NvFNgAg4doK0Zvg3KQYs+aEkONj19bnYmk5+hgnXQh8qwW FVDw== X-Gm-Message-State: AOJu0Yy0+PiAiZr+jTq9Y7Weav8qqKiPVCcWxQ0wsnqd7TPPFlOCcnxC SJ0RVYdK7aKUrhyed5gHK7p/U9IkuIA6sIbgmjfNtEfViI6EsuA0mQSX5mgbqnLLfZds8iYN5km oTpEhNcXkbiTuZ3HMhPyvbXdmil7NPWtiWYx8G7zq0iOT0QDYM9PiRA9VsDEEFuwcY1S8UL3Hk3 SfIK0JsHQkLebeQFCC6MaHp7j7L/dVpvxd4I28M7uSPEc= X-Gm-Gg: Acq92OG2QiKyXxTsZ1O8fZ18H0GiBTH0vT8aUHE8kFsScAiVF62Xi9MsRgk1V1+F8QF TwzjgP+88EdsxjJMHj2Tw5xTDUcDtUOKOfKqOrahL+CZfePaORmPXLDgUE6cZBWTqZ+yUs8p4Wx qoD6Pw/FitYhdQwxPEVe5a82Hx2qgDSoeUexk7dWspJoAmHFmgnK2aGPXESnFqQm6hVBQtQIYqn +7jbsFVl5rRLhZkA01D7RvDh5sqAn0v376xE2clqlEZpYguYGT8NSSL900EGmfXR7JR65iCVSyy zcNFPrASZQ0eAC3rW4ATilVhosUZZHURKVrg8m/GWVU8BS6bCeCdm4bKof/TxGKPr8/FDxJoYmV i3HgVup/3Ac9pG0ISvL/0Hvw= X-Received: by 2002:a05:600c:1549:b0:490:c7dd:7cc2 with SMTP id 5b1f17b1804b1-490c7dd7defmr154053705e9.24.1780907869364; Mon, 08 Jun 2026 01:37:49 -0700 (PDT) X-Received: by 2002:a05:600c:1549:b0:490:c7dd:7cc2 with SMTP id 5b1f17b1804b1-490c7dd7defmr154052515e9.24.1780907868720; Mon, 08 Jun 2026 01:37:48 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-490bc3cc140sm450465175e9.9.2026.06.08.01.37.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:37:48 -0700 (PDT) Date: Mon, 8 Jun 2026 04:37:41 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 16/37] mm: alloc_swap_folio: pass raw fault address to vma_alloc_folio Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Same change as the previous patch but for alloc_swap_folio: pass vmf->address directly instead of ALIGN_DOWN(vmf->address, ...). Note: NUMA interleave is not affected by the raw address; the ilx calculation shifts addr >> PAGE_SHIFT >> order, dropping sub-page bits regardless of alignment. Signed-off-by: Michael S. Tsirkin Reviewed-by: Gregory Price Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/memory.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 21f640674c4f..6c14b90f558e 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4750,8 +4750,7 @@ static struct folio *alloc_swap_folio(struct vm_fault= *vmf) /* Try allocating the highest of the remaining orders. */ gfp =3D vma_thp_gfp_mask(vma); while (orders) { - addr =3D ALIGN_DOWN(vmf->address, PAGE_SIZE << order); - folio =3D vma_alloc_folio(gfp, order, vma, addr); + folio =3D vma_alloc_folio(gfp, order, vma, vmf->address); if (folio) { if (!mem_cgroup_swapin_charge_folio(folio, vma->vm_mm, gfp, entry)) --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D2D7636F430 for ; Mon, 8 Jun 2026 08:38:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907896; cv=none; b=ctJOc4uFrN8gGU4Aa7lufLQuFiBCDHbEA5d3+YNag8W3/XNvIRMIs0zP+Ui/w0AUUdgT5sj4sH+iIU7R9PIzQx/zD29pzhwcsadvd8mjBDk7acfz5thKJ0ISf0BqvrAZ+ghxCxxGvOhdNd/nY6l0wh/ovGXwMWVJAcAUGn/k8v8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907896; c=relaxed/simple; bh=ItpkCdtgXylgGwPy7PtKJSb5szTx3fSHDwdNXrpaaFw=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=GcAwML2w3sqWuMI3eZ4mTZ78wyhOS+m+1HfHMqyR+UsQvdzuF+rJraAfFoVFn3Neoi6Q5aCEOYHqFxeRbcVKiGAH9xnsSeR9NF4+ROin1InUtPYhOrZ1cw8UXYEo7WjTw98DFodBfYa+7Uxkgk9ZKJZsIys16PjMsPDNH54eWjI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=SdlQ5VxG; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=K9bOhhdq; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="SdlQ5VxG"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="K9bOhhdq" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907883; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=hKnQ6S1r5njcc65wkmVimKLPjyeNQHeCJ4YbfSWW3Go=; b=SdlQ5VxGgE8Hk+WMZwIuJh/mvlfa+bFb7pdh05kcDmCl1ilLdz7Ueuao+bGTxmHNKZIESS cbKb8em2XKCBdwmi8aP2NWTI+fI+7M3JIOp6gEf+U6YiHVjv15jgA+zK9NFDwDKbKguAwO 4DVlWxkGo630fQeBEK4oF87cfvWjPOY= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-583-xnlZ0_eQNASQu_s1wQwsDg-1; Mon, 08 Jun 2026 04:38:02 -0400 X-MC-Unique: xnlZ0_eQNASQu_s1wQwsDg-1 X-Mimecast-MFC-AGG-ID: xnlZ0_eQNASQu_s1wQwsDg_1780907881 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-490b4d3d3e6so33362055e9.0 for ; Mon, 08 Jun 2026 01:38:01 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907881; x=1781512681; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=hKnQ6S1r5njcc65wkmVimKLPjyeNQHeCJ4YbfSWW3Go=; b=K9bOhhdqG9Xe8JW/Fhx0j5XM3i1VZESGM+MQqK4jGT3sIFl2SjEF/eAPrlONBaaDAV 5xKl6bjnynPGxquiOGhxaU7gN/4HOJzihTNQJYrxeSOfm/Nldr6A8Rj3LA0PMw5odPp+ WZ1gBblUIzTftlXX1+jTnrFW+BaIHo/qn9rYyc/9ds8G6KGjrAbaNNnd/BHw/mxwOuDx 7D/OUS6/eEURjlWtMXD+NwbRe15FX9ORivk2pWaJj75lFDsyiQAC664ldFoKSmu1ir9V WChlundlZVP6WovufI12cFx14bhVAeudacxQzGEDk7yiXZXsHltD11Sojg98FktnTKhj Kfsw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907881; x=1781512681; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hKnQ6S1r5njcc65wkmVimKLPjyeNQHeCJ4YbfSWW3Go=; b=tYZG/cKSRIooEx1DlvbUTQvqqK20JlfQbmxVNCXqRUYFY7qrpplaAntYmydmzMTQDe 3/LQtPlj079MOHH+psZ+h3O+4L5lKxOZ4wVbtPs32QA6Xse7UIR9Lrj6StLCXmC6kGPc CkSlUQSDnT4JMvgUlNP2osG5OgsdIosuY6dEVL8Lhc4CIGj/e3LWGtTfevQHJQKTM2aw jpAq+MfPcfdujBMjJjkJvnB4zLQAuRvRThYu1ZLFQGXJNSkUPtpUSkkp5II36zVOzSOl uMp8ibFiBRuiigHpbNkBqEX/+h1K3r6OCCh9nsOf3uvFxuI3Nv/AmJiz4Uz6SRmHUTxN 95Ig== X-Gm-Message-State: AOJu0YxnItxhvrsg76lwmcBA9oxxnVRtRiKJLTe9SOtF7qTuS1qj2gUp qK6Y2apctmfZORsY2cFuFcdIXTMyESt8P58whAtfZh23VziuY/A3gwjk7eQnafxR1Kr9MswSdaS Ffmxd8gDlFvSk0EAOm8LvPlj9UuSbUg2C9fPr/LKVRnOSr1/MQqM0I5aNtJgRbW+UwVAeyIkFwC QocYGVTwFXMU0km30QDvQjfFdtHmJ3fpbRM5MLfSPas5U= X-Gm-Gg: Acq92OF95fCPF4g3nEsXAZffZt+ZKRQSNVXuOqNCN3KZPa7NdfIcR7uPGUtMEMbMRCc S9kcRXP35xFUsY4d+ovruroXD3fxvf7pzw5BMk2jWGueF7TOThDRlLWdPU8cUBK0b244IQDuZCa Un39k2/ws1NZgRVB66ddw/OxUMxmyXpH8tw0moibj0UAyZMzeTLmEbi9t3ypZWrOwswkhz8AwM4 EWY8KWAfHHAR4WrXXhuRCTLpRDGkI0PnTK26qnQewwtoqcALZE32l9HozveTCGd4mlyHHt/b7xZ EORPiqw7pmRlo1yT4sHccf2vrBr8yNNudT9SCivHbyJY9JqbG9/OePvZm2glG5FzGOq4BGkLsbA Nj7XDlraY2LwA6gf60Z7NbbY= X-Received: by 2002:a05:600c:b90:b0:490:44eb:c1ea with SMTP id 5b1f17b1804b1-490c2613da2mr253384815e9.24.1780907880660; Mon, 08 Jun 2026 01:38:00 -0700 (PDT) X-Received: by 2002:a05:600c:b90:b0:490:44eb:c1ea with SMTP id 5b1f17b1804b1-490c2613da2mr253383525e9.24.1780907879858; Mon, 08 Jun 2026 01:37:59 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4601f35eae5sm51110392f8f.33.2026.06.08.01.37.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:37:59 -0700 (PDT) Date: Mon, 8 Jun 2026 04:37:48 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 17/37] mm: page_reporting: skip redundant zeroing of host-zeroed reported pages Message-ID: <6a2f93e4447e55ada6ccf0f4d5c64e8d41a848d4.1780906288.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When a guest reports free pages to the hypervisor via the page reporting framework (used by virtio-balloon and hv_balloon), the host typically zeros those pages when reclaiming their backing memory. However, when those pages are later allocated in the guest, post_alloc_hook() unconditionally zeros them again if __GFP_ZERO is set. This double-zeroing is wasteful, especially for large pages. Avoid redundant zeroing: - Add a host_zeroes_pages flag to page_reporting_dev_info, allowing drivers to declare that their host zeros reported pages on reclaim. A static key (page_reporting_host_zeroes) gates the fast path. - Add PG_zeroed page flag (sharing PG_private bit) to mark pages that have been zeroed by the host. Set it in page_reporting_drain() after the host reports them. - Thread the zeroed bool through rmqueue -> prep_new_page -> post_alloc_hook, where it skips redundant zeroing for __GFP_ZERO allocations. Currently the PG_zeroed hint can be lost when pages are split (expand) or merged in the buddy allocator. This is harmless: losing the hint just means the page gets re-zeroed, which is correct but suboptimal. Follow-up patches propagate PG_zeroed across splits and merges to preserve the hint on common paths. No driver sets host_zeroes_pages yet; a follow-up patch to virtio_balloon is needed to opt in. PG_zeroed pages may pass through PCP lists before being freed. This is safe: __free_pages_prepare clears all PAGE_FLAGS_CHECK_AT_PREP flags (including PG_zeroed/PG_private) before the page re-enters the buddy allocator. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- include/linux/page-flags.h | 9 +++++ include/linux/page_reporting.h | 3 ++ mm/compaction.c | 6 ++- mm/internal.h | 2 +- mm/page_alloc.c | 68 +++++++++++++++++++++++----------- mm/page_reporting.c | 14 ++++++- mm/page_reporting.h | 12 ++++++ 7 files changed, 88 insertions(+), 26 deletions(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 7223f6f4e2b4..91f8ddb1d512 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -135,6 +135,8 @@ enum pageflags { PG_swapcache =3D PG_owner_priv_1, /* Swap page: swp_entry_t in private */ /* Some filesystems */ PG_checked =3D PG_owner_priv_1, + /* Page contents are known to be zero */ + PG_zeroed =3D PG_private, =20 /* * Depending on the way an anonymous folio can be mapped into a page @@ -673,6 +675,13 @@ FOLIO_TEST_CLEAR_FLAG_FALSE(young) FOLIO_FLAG_FALSE(idle) #endif =20 +/* + * PageZeroed() tracks pages known to be zero. The allocator + * uses this to skip redundant zeroing in post_alloc_hook(). + */ +__PAGEFLAG(Zeroed, zeroed, PF_NO_COMPOUND) +#define __PG_ZEROED (1UL << PG_zeroed) + /* * PageReported() is used to track reported free pages within the Buddy * allocator. We can use the non-atomic version of the test and set diff --git a/include/linux/page_reporting.h b/include/linux/page_reporting.h index 5ab5be02fa15..c331c6b36687 100644 --- a/include/linux/page_reporting.h +++ b/include/linux/page_reporting.h @@ -14,6 +14,9 @@ struct page_reporting_dev_info { int (*report)(struct page_reporting_dev_info *prdev, struct scatterlist *sg, unsigned int nents); =20 + /* If true, host zeros reported pages on reclaim */ + bool host_zeroes_pages; + /* work struct for processing reports */ struct delayed_work work; =20 diff --git a/mm/compaction.c b/mm/compaction.c index 4336e433c99b..8000fc5e0a2e 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -82,7 +82,8 @@ static inline bool is_via_compact_memory(int order) { ret= urn false; } =20 static struct page *mark_allocated_noprof(struct page *page, unsigned int = order, gfp_t gfp_flags) { - post_alloc_hook(page, order, __GFP_MOVABLE, USER_ADDR_NONE); + __ClearPageZeroed(page); + post_alloc_hook(page, order, __GFP_MOVABLE, false, USER_ADDR_NONE); set_page_refcounted(page); return page; } @@ -1849,9 +1850,10 @@ static struct folio *compaction_alloc_noprof(struct = folio *src, unsigned long da set_page_private(&freepage[size], start_order); } dst =3D (struct folio *)freepage; + __ClearPageZeroed(&dst->page); if (order) prep_compound_page(&dst->page, order); - post_alloc_hook(&dst->page, order, __GFP_MOVABLE, USER_ADDR_NONE); + post_alloc_hook(&dst->page, order, __GFP_MOVABLE, false, USER_ADDR_NONE); set_page_refcounted(&dst->page); cc->nr_freepages -=3D 1 << order; cc->nr_migratepages -=3D 1 << order; diff --git a/mm/internal.h b/mm/internal.h index 9d2198114510..4af5e72742ba 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -928,7 +928,7 @@ static inline void init_compound_tail(struct page *tail, } =20 void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flag= s, - unsigned long user_addr); + bool zeroed, unsigned long user_addr); extern bool free_pages_prepare(struct page *page, unsigned int order); =20 extern int user_min_free_kbytes; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index d4fbf1861a8a..45e824b1ec75 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1743,6 +1743,7 @@ static __always_inline void page_del_and_expand(struc= t zone *zone, bool was_reported =3D page_reported(page); =20 __del_page_from_free_list(page, zone, high, migratetype); + nr_pages -=3D expand(zone, page, low, high, migratetype, was_reported); account_freepages(zone, -nr_pages, migratetype); } @@ -1815,8 +1816,10 @@ static inline bool should_skip_init(gfp_t flags) return (flags & __GFP_SKIP_ZERO); } =20 + inline void post_alloc_hook(struct page *page, unsigned int order, - gfp_t gfp_flags, unsigned long user_addr) + gfp_t gfp_flags, bool zeroed, + unsigned long user_addr) { const bool zero_tags =3D gfp_flags & __GFP_ZEROTAGS; bool init =3D !want_init_on_free() && want_init_on_alloc(gfp_flags) && @@ -1825,6 +1828,14 @@ inline void post_alloc_hook(struct page *page, unsig= ned int order, =20 set_page_private(page, 0); =20 + /* + * If the page is zeroed, skip memory initialization. + * We still need to handle tag zeroing separately since the host + * does not know about memory tags. + */ + if (zeroed && init && !zero_tags) + init =3D false; + arch_alloc_page(page, order); debug_pagealloc_map_pages(page, 1 << order); =20 @@ -1867,7 +1878,7 @@ inline void post_alloc_hook(struct page *page, unsign= ed int order, * through a user-congruent mapping. Host-zeroed pages * (zeroed flag) don't need this: physical RAM is clean. */ - if (!init && (gfp_flags & __GFP_ZERO) && + if (!zeroed && !init && (gfp_flags & __GFP_ZERO) && user_addr !=3D USER_ADDR_NONE && user_alloc_needs_zeroing()) init =3D true; @@ -1900,13 +1911,13 @@ inline void post_alloc_hook(struct page *page, unsi= gned int order, } =20 static void prep_new_page(struct page *page, unsigned int order, gfp_t gfp= _flags, - unsigned int alloc_flags, - unsigned long user_addr) + unsigned int alloc_flags, bool zeroed, + unsigned long user_addr) { if (order && (gfp_flags & __GFP_COMP)) prep_compound_page(page, order); =20 - post_alloc_hook(page, order, gfp_flags, user_addr); + post_alloc_hook(page, order, gfp_flags, zeroed, user_addr); =20 /* * page is set pfmemalloc when ALLOC_NO_WATERMARKS was necessary to @@ -3174,6 +3185,7 @@ int __isolate_free_page(struct page *page, unsigned i= nt order) } =20 del_page_from_free_list(page, zone, order, mt); + __ClearPageZeroed(page); =20 /* * Set the pageblock if the isolated page is at least half of a @@ -3246,7 +3258,7 @@ static inline void zone_statistics(struct zone *prefe= rred_zone, struct zone *z, static __always_inline struct page *rmqueue_buddy(struct zone *preferred_zone, struct zone *zone, unsigned int order, unsigned int alloc_flags, - int migratetype) + int migratetype, bool *zeroed) { struct page *page; unsigned long flags; @@ -3281,6 +3293,8 @@ struct page *rmqueue_buddy(struct zone *preferred_zon= e, struct zone *zone, } } spin_unlock_irqrestore(&zone->lock, flags); + *zeroed =3D PageZeroed(page); + __ClearPageZeroed(page); } while (check_new_pages(page, order)); =20 /* @@ -3349,10 +3363,9 @@ static int nr_pcp_alloc(struct per_cpu_pages *pcp, s= truct zone *zone, int order) /* Remove page from the per-cpu list, caller must protect the list */ static inline struct page *__rmqueue_pcplist(struct zone *zone, unsigned int order, - int migratetype, - unsigned int alloc_flags, + int migratetype, unsigned int alloc_flags, struct per_cpu_pages *pcp, - struct list_head *list) + struct list_head *list, bool *zeroed) { struct page *page; =20 @@ -3387,6 +3400,8 @@ struct page *__rmqueue_pcplist(struct zone *zone, uns= igned int order, page =3D list_first_entry(list, struct page, pcp_list); list_del(&page->pcp_list); pcp->count -=3D 1 << order; + *zeroed =3D PageZeroed(page); + __ClearPageZeroed(page); } while (check_new_pages(page, order)); =20 return page; @@ -3395,7 +3410,8 @@ struct page *__rmqueue_pcplist(struct zone *zone, uns= igned int order, /* Lock and remove page from the per-cpu list */ static struct page *rmqueue_pcplist(struct zone *preferred_zone, struct zone *zone, unsigned int order, - int migratetype, unsigned int alloc_flags) + int migratetype, unsigned int alloc_flags, + bool *zeroed) { struct per_cpu_pages *pcp; struct list_head *list; @@ -3413,7 +3429,8 @@ static struct page *rmqueue_pcplist(struct zone *pref= erred_zone, */ pcp->free_count >>=3D 1; list =3D &pcp->lists[order_to_pindex(migratetype, order)]; - page =3D __rmqueue_pcplist(zone, order, migratetype, alloc_flags, pcp, li= st); + page =3D __rmqueue_pcplist(zone, order, migratetype, alloc_flags, + pcp, list, zeroed); pcp_spin_unlock(pcp); if (page) { __count_zid_vm_events(PGALLOC, page_zonenum(page), 1 << order); @@ -3438,19 +3455,19 @@ static inline struct page *rmqueue(struct zone *preferred_zone, struct zone *zone, unsigned int order, gfp_t gfp_flags, unsigned int alloc_flags, - int migratetype) + int migratetype, bool *zeroed) { struct page *page; =20 if (likely(pcp_allowed_order(order))) { page =3D rmqueue_pcplist(preferred_zone, zone, order, - migratetype, alloc_flags); + migratetype, alloc_flags, zeroed); if (likely(page)) goto out; } =20 page =3D rmqueue_buddy(preferred_zone, zone, order, alloc_flags, - migratetype); + migratetype, zeroed); =20 out: /* Separate test+clear to avoid unnecessary atomics */ @@ -3841,6 +3858,7 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int o= rder, int alloc_flags, struct pglist_data *last_pgdat =3D NULL; bool last_pgdat_dirty_ok =3D false; bool no_fallback; + bool zeroed; bool skip_kswapd_nodes =3D nr_online_nodes > 1; bool skipped_kswapd_nodes =3D false; =20 @@ -3985,10 +4003,11 @@ get_page_from_freelist(gfp_t gfp_mask, unsigned int= order, int alloc_flags, =20 try_this_zone: page =3D rmqueue(zonelist_zone(ac->preferred_zoneref), zone, order, - gfp_mask, alloc_flags, ac->migratetype); + gfp_mask, alloc_flags, ac->migratetype, + &zeroed); if (page) { prep_new_page(page, order, gfp_mask, alloc_flags, - ac->user_addr); + zeroed, ac->user_addr); =20 return page; } else { @@ -4215,9 +4234,11 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigne= d int order, count_vm_event(COMPACTSTALL); =20 /* Prep a captured page if available */ - if (page) - prep_new_page(page, order, gfp_mask, alloc_flags, + if (page) { + __ClearPageZeroed(page); + prep_new_page(page, order, gfp_mask, alloc_flags, false, ac->user_addr); + } =20 /* Try get a page from the freelist if available */ if (!page) @@ -5190,6 +5211,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, /* Attempt the batch allocation */ pcp_list =3D &pcp->lists[order_to_pindex(ac.migratetype, 0)]; while (nr_populated < nr_pages) { + bool zeroed =3D false; =20 /* Skip existing pages */ if (page_array[nr_populated]) { @@ -5198,7 +5220,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, } =20 page =3D __rmqueue_pcplist(zone, 0, ac.migratetype, alloc_flags, - pcp, pcp_list); + pcp, pcp_list, &zeroed); if (unlikely(!page)) { /* Try and allocate at least one page */ if (!nr_account) { @@ -5209,7 +5231,7 @@ unsigned long alloc_pages_bulk_noprof(gfp_t gfp, int = preferred_nid, } nr_account++; =20 - prep_new_page(page, 0, gfp, 0, USER_ADDR_NONE); + prep_new_page(page, 0, gfp, 0, zeroed, USER_ADDR_NONE); set_page_refcounted(page); page_array[nr_populated++] =3D page; } @@ -6949,7 +6971,8 @@ static void split_free_frozen_pages(struct list_head = *list, gfp_t gfp_mask) list_for_each_entry_safe(page, next, &list[order], lru) { int i; =20 - post_alloc_hook(page, order, gfp_mask, USER_ADDR_NONE); + __ClearPageZeroed(page); + post_alloc_hook(page, order, gfp_mask, false, USER_ADDR_NONE); if (!order) continue; =20 @@ -7157,8 +7180,9 @@ static int __alloc_contig_frozen_range(unsigned long = start, unsigned long end, } else if (start =3D=3D outer_start && end =3D=3D outer_end && is_power_o= f_2(end - start)) { struct page *head =3D pfn_to_page(start); =20 + __ClearPageZeroed(head); check_new_pages(head, order); - prep_new_page(head, order, gfp_mask, 0, user_addr); + prep_new_page(head, order, gfp_mask, 0, false, user_addr); } else { ret =3D -EINVAL; WARN(true, "PFN range: requested [%lu, %lu), allocated [%lu, %lu)\n", diff --git a/mm/page_reporting.c b/mm/page_reporting.c index 5b6b17f67131..84ebc4547119 100644 --- a/mm/page_reporting.c +++ b/mm/page_reporting.c @@ -50,6 +50,8 @@ EXPORT_SYMBOL_GPL(page_reporting_order); #define PAGE_REPORTING_DELAY (2 * HZ) static struct page_reporting_dev_info __rcu *pr_dev_info __read_mostly; =20 +DEFINE_STATIC_KEY_FALSE(page_reporting_host_zeroes); + enum { PAGE_REPORTING_IDLE =3D 0, PAGE_REPORTING_REQUESTED, @@ -129,8 +131,11 @@ page_reporting_drain(struct page_reporting_dev_info *p= rdev, * report on the new larger page when we make our way * up to that higher order. */ - if (PageBuddy(page) && buddy_order(page) =3D=3D order) + if (PageBuddy(page) && buddy_order(page) =3D=3D order) { __SetPageReported(page); + if (page_reporting_host_zeroes_pages()) + __SetPageZeroed(page); + } } while ((sg =3D sg_next(sg))); =20 /* reinitialize scatterlist now that it is empty */ @@ -390,6 +395,10 @@ int page_reporting_register(struct page_reporting_dev_= info *prdev) /* Assign device to allow notifications */ rcu_assign_pointer(pr_dev_info, prdev); =20 + /* enable zeroed page optimization if host zeroes reported pages */ + if (prdev->host_zeroes_pages) + static_branch_enable(&page_reporting_host_zeroes); + /* enable page reporting notification */ if (!static_key_enabled(&page_reporting_enabled)) { static_branch_enable(&page_reporting_enabled); @@ -414,6 +423,9 @@ void page_reporting_unregister(struct page_reporting_de= v_info *prdev) =20 /* Flush any existing work, and lock it out */ cancel_delayed_work_sync(&prdev->work); + + if (prdev->host_zeroes_pages) + static_branch_disable(&page_reporting_host_zeroes); } =20 mutex_unlock(&page_reporting_mutex); diff --git a/mm/page_reporting.h b/mm/page_reporting.h index c51dbc228b94..736ea7b37e9e 100644 --- a/mm/page_reporting.h +++ b/mm/page_reporting.h @@ -15,6 +15,13 @@ DECLARE_STATIC_KEY_FALSE(page_reporting_enabled); extern unsigned int page_reporting_order; void __page_reporting_notify(void); =20 +DECLARE_STATIC_KEY_FALSE(page_reporting_host_zeroes); + +static inline bool page_reporting_host_zeroes_pages(void) +{ + return static_branch_unlikely(&page_reporting_host_zeroes); +} + static inline bool page_reported(struct page *page) { return static_branch_unlikely(&page_reporting_enabled) && @@ -46,6 +53,11 @@ static inline void page_reporting_notify_free(unsigned i= nt order) #else /* CONFIG_PAGE_REPORTING */ #define page_reported(_page) false =20 +static inline bool page_reporting_host_zeroes_pages(void) +{ + return false; +} + static inline void page_reporting_notify_free(unsigned int order) { } --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B19983AFD19 for ; Mon, 8 Jun 2026 08:38:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907899; cv=none; b=c1QsdiVasPgIn5OYiOwCSc0HySH2uOPqym8Jn1ueuU4qrUsWDxg7j4y63HDvUU7QHJTDNW5e2RpjHjp80Hb3v14XhPWNn55biIjC0RDMG9pF55ZxVN6RtW0AguXGvhFsVTjzRvv/qykxSsMOrlJLd6QZAPAN+DISU5MWIkPzUDg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907899; c=relaxed/simple; bh=8+ZahGpeDELrqcGB1YJH6oTjGbft7NUn5eY1+HiJ8Dk=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=tuyidnTaZlmLtsdWcZksLk/XW3xln4w3lgd5RKcUaQAnNwZh6ZpTjKhlh9A2KuK5xmm8I76/E8FgUVDD3IHSBC9yt5CRik8qI8MuPBHyFXtaQG4m2wPaBMKh2s8e/gsMJt2foMs7cfq/JQ3ovk4D8xY9GTh22RD+vZLqKcIngGA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=UTdlxRAw; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=UptaY7Vk; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="UTdlxRAw"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="UptaY7Vk" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907896; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=k8mD++2dfZtiFiQOTpn1O7CzJ+9UexLHDFBwODnyZcw=; b=UTdlxRAw9WKvG72HuGbt/XkR0RdTAVPCKET8v4J3nrZJyUcgYUfEETk4QZXDsMdfAJizIz AQDmC4V9haRQAtvwffjmhUaT7BRhfX3OBWv0J3UeFRIh8jOHMaSYGGPhWq0Wda1gQaaa0P yxHw9Ez3QN89A4+dUsPQkB+tIlBGWpQ= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-629-QYCgmaanOYCCmay2lYYkSA-1; Mon, 08 Jun 2026 04:38:10 -0400 X-MC-Unique: QYCgmaanOYCCmay2lYYkSA-1 X-Mimecast-MFC-AGG-ID: QYCgmaanOYCCmay2lYYkSA_1780907889 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-45efa12a788so2884472f8f.2 for ; Mon, 08 Jun 2026 01:38:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907889; x=1781512689; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=k8mD++2dfZtiFiQOTpn1O7CzJ+9UexLHDFBwODnyZcw=; b=UptaY7Vk6zyOKDvgMp0+2/H01Xuk+YgAgWbkUzVeg2AozMkOrB6k4jKpiFmTe1oC3S WKh0rAQ8dU8YuO0K51J2chp13CmuMsC5jEArzioE2e8qdkptcr3s5ukp6IEI8xvYVls6 hg9LIk7Vdo0LwwwVmAcvI73eZT4xetRYaIiB9fBYq5tQZwkSG/poETVlQAm+YFc9lQJb 6XdBGiDaShnbggoQp3Hs6GOhlZAqZ6xHI8qFBrMjbBXSElqEWFMaGbFhpWXdBjC0fglB Hv2oM8Owl+TcruYsSsz/8tQHA79DhT6GV8lxtWVx2pyhW/+J5u41y394feeIPSqrAQSf rtdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907889; x=1781512689; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=k8mD++2dfZtiFiQOTpn1O7CzJ+9UexLHDFBwODnyZcw=; b=bU5RkkGJKBIsbyRQFAz65TFbDsgyFXdNaQP0Au7bBkluijx92L5uT4WwR32FmTSFSZ m7ekUrCicHa7Yn9tTmOkJtTxC+sphKsGac+jtXhW0DR6pyAmZ0L0lsk/eQDWiBWsPhVo r5UPmT+ZnQm/EVrQmaQd7NkiXmKkwsFgeLymCeQmoOgpTb/CbHne3qo0YO2tVO1EE2X7 cxkaAaDAKXWFQ19USsY7zxoucslFyKAs0uAbiBbyNUIwo9d5X1VnB+sBry37lrjM1nzO HyD/haF90mb+Gq/dwDU9htZjt1ulEBLVX1CKTHt/ztbxMRGkatSpzfB3uFh9SOGqzIjP QhZQ== X-Gm-Message-State: AOJu0Yy4iW+8WVO2NxS48/rShIosMwKbFtd2X1Dgvv+xHur/mhCT/ReL pis7LuPCWlbzjAfeFETU+sM8zyRy7YutOB3LYWosyhf4ESXTVAgkkIOcsZsKkIY4EQvUcSycVIy MMqV5ItmmZDQrTifeDa4m15EO/bMuAHmpWz6FJIfT+P9XQ8tCfjbNFutJrWoRsWuEBn/z9bPT/I dsi/wxx+OOOOAdUQYVpVsDcg6ZQFBJ2SpsBlJRhPO1c2U= X-Gm-Gg: Acq92OHDqsS3elHsve1MaZkZKwm9GTaebOVLaFITC3tqS26ZQgjxpoLCZ521YiYLV8v lATKtQrqR4zx8a/czZBWTqatlbqZYu1Bj//9t9IGer/XgByKQoxvVvyrrQ8QS1T+IBf0Ce3ROJ+ xUBYK4NW5s6VbGSfT9+GIPlZ26wNN3zCmtNGjrBvoZHvm5p6AWvCUjjS77xSROq9cUaDWPDOqik gNXJCe/EetU07T76vWIjkV8bOOClku6Vnr9MpvyP28rtSq8IzLis1qunLHM91lXfIYgmnp9XAcW +GtnHmMpbk/Ow13mQWroU34dr0arnmAALLhjSAECtusgu0+8ZE1VWdnGsOF5a9+KX+YfZhBOXoS Ye6iKs9GFygSWvLfb2UgQKOg= X-Received: by 2002:a05:600c:c491:b0:490:bb44:3f8b with SMTP id 5b1f17b1804b1-490c2605385mr243137435e9.17.1780907889256; Mon, 08 Jun 2026 01:38:09 -0700 (PDT) X-Received: by 2002:a05:600c:c491:b0:490:bb44:3f8b with SMTP id 5b1f17b1804b1-490c2605385mr243136515e9.17.1780907888480; Mon, 08 Jun 2026 01:38:08 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-490bc4082c3sm362037795e9.13.2026.06.08.01.38.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:38:07 -0700 (PDT) Date: Mon, 8 Jun 2026 04:38:00 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 18/37] mm: page_alloc: use aliasing checks instead of user_alloc_needs_zeroing Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Replace user_alloc_needs_zeroing() with the direct aliasing checks (cpu_dcache_is_aliasing() || cpu_icache_is_aliasing()) in the post_alloc_hook aliasing guard. user_alloc_needs_zeroing() includes a !init_on_alloc term that means "allocator didn't zero this page." But in this guard's context (!zeroed && !init && __GFP_ZERO), we already know the page is zero; init incorporates init_on_alloc via want_init_on_alloc(). The only question left is whether the cache architecture needs the data re-zeroed through a congruent mapping, which is purely cpu_dcache_is_aliasing() || cpu_icache_is_aliasing(). On non-aliasing architectures with init_on_free=3Dtrue and init_on_alloc=3Dfalse, this avoids a redundant re-zero of an already-zero page. Note on PowerPC: PowerPC overrides clear_user_page to call flush_dcache_page after clear_page, but on freshly allocated pages PG_dcache_clean is already clear (cleared by __free_pages_prepare), so flush_dcache_page is a no-op. Skipping this here thus has no effect. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/page_alloc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 45e824b1ec75..edfc83571985 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1880,7 +1880,7 @@ inline void post_alloc_hook(struct page *page, unsign= ed int order, */ if (!zeroed && !init && (gfp_flags & __GFP_ZERO) && user_addr !=3D USER_ADDR_NONE && - user_alloc_needs_zeroing()) + (cpu_dcache_is_aliasing() || cpu_icache_is_aliasing())) init =3D true; /* * If memory is still not initialized, initialize it now. --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EB3203AFD0D for ; Mon, 8 Jun 2026 08:38:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907901; cv=none; b=i63ZkCrI0GURaGmfVIJw2eJqxSK4O50ErhWIJu53edDpby7zrjox5cYIf/qfnKIjcbohoE2iga+UjGeIrAD+KvYKyo3JsUtF91bovBmk1Fc/gGNwZYBedblHd68rx0ngnoCg2rUtA2z9UH3HIjaHB3D4sqXzJivf4QWX+etW5Ic= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907901; c=relaxed/simple; bh=BXaZfFdflynFOGogrQYYCO11NJz9qF4gKEWS8jDZf8c=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=ui2/lZreyMA5bGe/dMv5xhy2rOLu4A+vQ5BEqKooUGGw51NvJJF70M5MFOdFT6qNvmYy22TBmk2g6Xs2FR9HjtTG5XwgMzErtPrcTaSlgJaM/D0DttQeVTXgj1put+ZBMppmD7gV/vRd422Sn08ZHeFH3LPWJ/CDX9smNk2r1xc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=TjA82VPn; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=grbw7E3j; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="TjA82VPn"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="grbw7E3j" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907899; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=wwQyRM0soCCwqjRtQ3TNBnEtxbXipQm56goEaNWxWT0=; b=TjA82VPn9Jw9Gio6M2bAUY1lHGFCMViTHntESvJvjPq25VAfIGEz2kP6KwByziV8tp2fae e8x4RAB66moneWaUoXHB2lmbPS4x3nBIOq4WFW50cbZi0O47cb2nDs182tTu2n2qwd7y31 FIPLHXieDOON3uA9ZfCAMNK4yrLULPY= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-691-aKwU4ot5M1K6nAFzmta7UQ-1; Mon, 08 Jun 2026 04:38:17 -0400 X-MC-Unique: aKwU4ot5M1K6nAFzmta7UQ-1 X-Mimecast-MFC-AGG-ID: aKwU4ot5M1K6nAFzmta7UQ_1780907897 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-46010392f89so3681649f8f.2 for ; Mon, 08 Jun 2026 01:38:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907897; x=1781512697; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=wwQyRM0soCCwqjRtQ3TNBnEtxbXipQm56goEaNWxWT0=; b=grbw7E3jXr4kr/DKFq+XiAXo9JQxi7KxWPvZ509AEkNPpYlBt4fQtuEL8T/XSkXHKC ucXWHQzf7N7ZX7VkCnB/VehxChH4ttHvoL4mglaa+HEL6OYt7OjoLhTeGvda7fB4Qd6H 8P3THxNfZmTSl1D34AixLmmdP92teFWoPeUCfRnJxVPLz7GJ9DjhSpQbE5x3BtROU2uv iZTAU73abKR4ZcUR9ZMDHUEY8ReCpXC6uEjj4dMji5ASMgFCt0xM/66O0OyaCucqJ/p3 tXQsSp0eTfh59oAnYlPpciSJlXFq7DMsOvgTnD+Sr3sKvlnbxoPx9owN8tSlj1vL4+v3 7yNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907897; x=1781512697; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wwQyRM0soCCwqjRtQ3TNBnEtxbXipQm56goEaNWxWT0=; b=dskdD9E8yEQ1dy0KpXdFwRjf3PlZY7n40vF6vHhK+ehxVI9/aDAxG+nsr/HpG58t/5 ujbsj0ieWvVFSZ4vx9s/Mex+nxk7zuWMqo3KsaV/6vzrxCviY2eSFqmNzDThrmruGgNQ sPsw2yH88cnCvxbFBftZiFg6FW5wMJf5Y+YYBrdi0L177Msh7df9vrkFsNVFnDo8djFk ZIv0Mu3bTZJgrIUrNksW4+7HWTj4G/m7aTrB2cBDTsHSOeRyJk2YP/pUjVqlYuDxaksr eWSbhBi9cZ3NUsolVQa9V3YE9771A1iCVJI5cjO7Ik8ZOi/l3V3uTDYZO6YwNxx5UZ8Q xZAA== X-Gm-Message-State: AOJu0Yxah9C6lUg+4M66HrB3ZXQQFUDwVvXe+3c9n7JSJEn6YzOPGNNK DZUJJN0mBEL/M1vrlohuy+VMlYqOyDSXkwSVm3qaABARogiQt7QCnJdtVfpCzfTD4UbHBssJWp9 QWtRgl5qGMLWNeduPkLs96ch6+Hec4Z0rX93YIuSQAGx5tXVKzUDHYpVDQn9BlsizJDxfR2Szqv FKEWKTgr8XI44V+wGqAYKLuWgRsuDPE1oSh/35LDMeNts= X-Gm-Gg: Acq92OEJizB8VBqcdrNZqjn8R0l4aJyLLyoJ0BcMJwydmRtEowOuVVjxuW+ZVk0w3oX Jr6yqN9hiNCcdj8DV2dZ3zgWu0KErKyYTgmzcGobDZJ4TjAnMR6Ewkv6CnjcaTd20kh+QksJO6Y cfkijXkuLGv3H6Jw7DNuXivDU9xLdkgV39t+Q7ui7P+lK2SGjPlDw1UAOAErB3kMvjxOOXKtxJV GGbNjxwBjrGK5YZMNIjBSbOcS1ygUr9vcC/boEtxHUfcGZ8lk3nuv/m0wwkcrb8MwyOlkTtrPT5 Nyezm2g7pUmTC8UdqzPH2LqWpbGKtrcjpsgWRazpckTT7YZVwLXtfo9/ru21775nMlD3tg+GE4Q aWNsKaZ8L21HgaCWNe/cOXCM= X-Received: by 2002:a5d:624a:0:b0:452:3677:3fb3 with SMTP id ffacd0b85a97d-460304ebbe1mr15950868f8f.1.1780907896453; Mon, 08 Jun 2026 01:38:16 -0700 (PDT) X-Received: by 2002:a5d:624a:0:b0:452:3677:3fb3 with SMTP id ffacd0b85a97d-460304ebbe1mr15950779f8f.1.1780907895867; Mon, 08 Jun 2026 01:38:15 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4601f344541sm47271286f8f.22.2026.06.08.01.38.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:38:15 -0700 (PDT) Date: Mon, 8 Jun 2026 04:38:08 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 19/37] mm: page_alloc: clear PG_zeroed on buddy merge if not both zero Message-ID: <918897302a4cab9f476625adc238ec65a688d1d7.1780906288.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When two buddy pages merge in __free_one_page(), preserve PG_zeroed on the merged page only if both buddies have the flag set. Otherwise clear it. The merged page would inherit PG_zeroed, and a later __GFP_ZERO allocation would skip zeroing stale data in the non-zero half. Signed-off-by: Michael S. Tsirkin Reviewed-by: Gregory Price Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- include/linux/page-flags.h | 1 + mm/page_alloc.c | 15 ++++++++++++++- 2 files changed, 15 insertions(+), 1 deletion(-) diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 91f8ddb1d512..9365d59ac1d6 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -680,6 +680,7 @@ FOLIO_FLAG_FALSE(idle) * uses this to skip redundant zeroing in post_alloc_hook(). */ __PAGEFLAG(Zeroed, zeroed, PF_NO_COMPOUND) +CLEARPAGEFLAG(Zeroed, zeroed, PF_NO_COMPOUND) #define __PG_ZEROED (1UL << PG_zeroed) =20 /* diff --git a/mm/page_alloc.c b/mm/page_alloc.c index edfc83571985..a90bca5317c1 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -941,10 +941,14 @@ static inline void __free_one_page(struct page *page, unsigned long buddy_pfn =3D 0; unsigned long combined_pfn; struct page *buddy; + bool buddy_zeroed; + bool page_zeroed; bool to_tail; =20 VM_BUG_ON(!zone_is_initialized(zone)); - VM_BUG_ON_PAGE(page->flags.f & PAGE_FLAGS_CHECK_AT_PREP, page); + /* PG_zeroed (aliased to PG_private) is valid on free-list pages */ + VM_BUG_ON_PAGE(page->flags.f & + (PAGE_FLAGS_CHECK_AT_PREP & ~__PG_ZEROED), page); =20 VM_BUG_ON(migratetype =3D=3D -1); VM_BUG_ON_PAGE(pfn & ((1 << order) - 1), page); @@ -979,6 +983,8 @@ static inline void __free_one_page(struct page *page, goto done_merging; } =20 + buddy_zeroed =3D PageZeroed(buddy); + /* * Our buddy is free or it is CONFIG_DEBUG_PAGEALLOC guard page, * merge with it and move up one order. @@ -997,10 +1003,17 @@ static inline void __free_one_page(struct page *page, change_pageblock_range(buddy, order, migratetype); } =20 + page_zeroed =3D PageZeroed(page); + __ClearPageZeroed(page); + __ClearPageZeroed(buddy); + combined_pfn =3D buddy_pfn & pfn; page =3D page + (combined_pfn - pfn); pfn =3D combined_pfn; order++; + + if (page_zeroed && buddy_zeroed) + __SetPageZeroed(page); } =20 done_merging: --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DABB13B6340 for ; Mon, 8 Jun 2026 08:38:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907914; cv=none; b=YC2vKVF1m/dkU63CGDolM7jVQiulsqvgt7KcKayIZAPwgVQ6TbpyV2EfuhNpM+Yt65tc3+oc66XX1x8xCrhk12oGIfTWRxED3pX63cIuSBD5CrmSx2yd8z1rD28yOyB5pjU6PdUnYBL729RjWK835BUvTxr9vCo7KgMSQG9onSo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907914; c=relaxed/simple; bh=UxeyPjgwwid26PyqR+dr8ssEfWP9Pr8mV393MNefhtc=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=HC+XRUgL4aRtfW4meW62vkssEKhpbw/wveU9m5QNdxjmL59MXzY7kIfJNPEJaRRdIcoRwdLIIZgWcj5D+hKjA+L4KditAyHoNd+6onYIIatSO5jeSAu9G6xFSdpUdEwGw5wmREdqp3tuFg6JZfbsEgvqyXWyptrQtqq4/XR8VAE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Kws79ZkD; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=Zjrva3cn; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Kws79ZkD"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="Zjrva3cn" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907908; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=pR/GrYiW1j/TwC+RCSDEiDzn5vRq7z0PTWnhHKvSevw=; b=Kws79ZkDcXF5XOSVwOBOqAzQqx9/4EJXAsjqWXLjXNFJku/1D7Sa+ZECADXanoYvZvQwvU Ip2NHmlC1HMxX54bSfbGeSSPlB8ql5sIg3AbyCfP09SxcHCRbeXFqVJ+j0je9YVJt2Zpqm fXKwAYFjf+QWjmaMekKk9fewGTYym78= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-375-pGVpJlMxPOWrVq0AFElRBw-1; Mon, 08 Jun 2026 04:38:27 -0400 X-MC-Unique: pGVpJlMxPOWrVq0AFElRBw-1 X-Mimecast-MFC-AGG-ID: pGVpJlMxPOWrVq0AFElRBw_1780907906 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-45eee3f9f03so4049221f8f.1 for ; Mon, 08 Jun 2026 01:38:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907906; x=1781512706; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=pR/GrYiW1j/TwC+RCSDEiDzn5vRq7z0PTWnhHKvSevw=; b=Zjrva3cnIfWpG2HwaLvRvFJSZvSbwlA+LbVk/+b29m6+Of1/EaT/6y9S8jRdbwv7pV BTBfkxYsWtdmvLmT4gOcDq5i/fEPKsnIZUgSDaG7HT37MEX7myVOee3acMInYdqWVOGk 9Hf7wOp+V97o4hh2ICl0yCS5/7HZSLpJ6YTZApsxuWrnuf/ZpF0nvzEKfiR6nAcu13F2 hDM+ICOVIrfoxIWGyPMFCPTCuHBIe+miiJcoKLmfgPBs2F8N2G16Q0xCBWo9zVRqVyrc f45mp62CVrXDc4w8cbg269uUmluNjxinXP/f6ZFePGHnC5A1vD8ivYJqpa/HsCEc2gJf i+oA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907906; x=1781512706; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pR/GrYiW1j/TwC+RCSDEiDzn5vRq7z0PTWnhHKvSevw=; b=n8wykmzneRBotFTuRDRModw+UF9rEWvXlj3N7sA6y+JAVGdZ2ONLW0ic8A6QLMitcv KKzGwrhvXjhF+JB5JMJAwa8VZoU4tufgUwBR+AS1tpSoK8vaHrftKC4dWrfrajP0B4N0 2vb4HJhpiWoPP0TGcpKJFqw2oXUZ4YU/WwBPQlfKPcUHNFKkTXu7lvoD8M3gGRlcngWh YeMN0YPa+Ar20X2mtri1ry0jYn4/t1SB1eDjjF3etvJPNitypIE1Si5PAzJOautFcIgK nLLoWdwLHY4XFhYARvxWPP/ozo1w0Epg9+LCGiw+/y5RBHTfnmxZFeoTR3eSqdpf8Ye3 Yv1Q== X-Gm-Message-State: AOJu0Yx21LmX35n4Jenevj8mQ6nwuqD2BMHkmqqh5CoYF3P5fIUV2WYi DzV6+8fCuGC0nc3lGQH7A7At+MJRJM6o74uIZuE+Yj+996URDTZ1xqgGnaMUTBliwSZ1z5hqlRw POMyav5a3RV8ydTwD/Dw9BkGlybQWdIvY+PUrOWg13sM1+QPGYne6BaSrbxCrVcilZgdI0sFo5O zFiQp43DCRBYnsMZoWkFJo8lj2w8MMbrNXrTVR/8sHSsk= X-Gm-Gg: Acq92OEgjDAIR1iA6dJZunN4b5Ehcl9huxZtiUYyBAPwn6+t1F3gkMqS8t9ayaxMQLm 7eUYST0OUCpP9HuqYRw8ZOvJGQdi2mveFfYQvbQPJF9sxM8MiJyiqJaqzB4i8tndgSFWW3HxFJN fxcC3q+Y/cW//4UT/sCCeI1In36XHtiB4R1AeLa3E/lz8qLYZ1YsXRtS7rCRA8Nw/w2gqAk1cY2 bE3XHzu39vLe5kb3mXShWdcJEhstINpzmhiBkCROhNhREDoNHnVeC8GvCNJjAjOIvJzSgefKJ1Q yjHoBSsRjcS1lhpMe7TJK3vj6KTQ0j4ZwubdkVj9HjpVcOp2Ay7utN1c1ZeWNMiPmRlt5gxPVz9 qkTAdsyO4nGYse3FKsmF6DNk= X-Received: by 2002:a05:6000:2d10:b0:43f:dfb6:78e1 with SMTP id ffacd0b85a97d-460302e0a26mr15095040f8f.8.1780907905974; Mon, 08 Jun 2026 01:38:25 -0700 (PDT) X-Received: by 2002:a05:6000:2d10:b0:43f:dfb6:78e1 with SMTP id ffacd0b85a97d-460302e0a26mr15094967f8f.8.1780907905416; Mon, 08 Jun 2026 01:38:25 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4601f3444fesm51241760f8f.20.2026.06.08.01.38.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:38:24 -0700 (PDT) Date: Mon, 8 Jun 2026 04:38:16 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 20/37] mm: page_alloc: preserve PG_zeroed in page_del_and_expand Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Propagate PG_zeroed through buddy splits in page_del_and_expand() and try_to_claim_block(). When a zeroed high-order page is split to satisfy a smaller allocation, the sub-pages placed back on the free lists keep PG_zeroed. Signed-off-by: Michael S. Tsirkin Reviewed-by: Gregory Price Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/page_alloc.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index a90bca5317c1..7a6dedd716e2 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1712,7 +1712,8 @@ struct page *__pageblock_pfn_to_page(unsigned long st= art_pfn, * -- nyc */ static inline unsigned int expand(struct zone *zone, struct page *page, in= t low, - int high, int migratetype, bool reported) + int high, int migratetype, bool reported, + bool zeroed) { unsigned int size =3D 1 << high; unsigned int nr_added =3D 0; @@ -1743,6 +1744,8 @@ static inline unsigned int expand(struct zone *zone, = struct page *page, int low, */ if (reported) __SetPageReported(&page[size]); + if (zeroed) + __SetPageZeroed(&page[size]); } =20 return nr_added; @@ -1754,10 +1757,12 @@ static __always_inline void page_del_and_expand(str= uct zone *zone, { int nr_pages =3D 1 << high; bool was_reported =3D page_reported(page); + bool was_zeroed =3D PageZeroed(page); =20 __del_page_from_free_list(page, zone, high, migratetype); =20 - nr_pages -=3D expand(zone, page, low, high, migratetype, was_reported); + nr_pages -=3D expand(zone, page, low, high, migratetype, was_reported, + was_zeroed); account_freepages(zone, -nr_pages, migratetype); } =20 @@ -2355,11 +2360,12 @@ try_to_claim_block(struct zone *zone, struct page *= page, if (current_order >=3D pageblock_order) { unsigned int nr_added; bool was_reported =3D page_reported(page); + bool was_zeroed =3D PageZeroed(page); =20 del_page_from_free_list(page, zone, current_order, block_type); change_pageblock_range(page, current_order, start_type); nr_added =3D expand(zone, page, order, current_order, start_type, - was_reported); + was_reported, was_zeroed); account_freepages(zone, nr_added, start_type); return page; } --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9416D3B774A for ; Mon, 8 Jun 2026 08:38:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907924; cv=none; b=mKdJ1a9EUiLh+UT/k/d/DwAjcYab3KwA1+bCiQN5Z8JBJBfG5dzIvyqXkooMHwUtgNNF0dQQTQLWNdXiAFYFypcn+vrmUVqWqyOGSvUpm+e1Wp3mJnmc0eDsbWBkli8r+TL9nJuM+o9CF32YHFucRMmVCFwTJ6OqqLxO0VQ93c8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907924; c=relaxed/simple; bh=QZ3JwNzQr1I2N8bB1taSY/6E446pSRQWfElbnCUeVaI=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=a6LVHqF54iq6ysjm0KrkqYHXxp1bzSia/6PJRhBAyUZB5DG3g0Yb0IpOH3jPV2WA0AI/WyRvjIaeyE79d81fmKxnS4JUEfp9jr5AQzAIktjAUhF2Envq3QW5mGp96UZ2/yXObE2Um3rseRFJ8Wsy89DIpfDMe5MVax0emzmUt9Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=FgVBcoeO; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=ZMj5TceT; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="FgVBcoeO"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="ZMj5TceT" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907920; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=vosxR5Z06o9BwR/z8OHSdoefhvjInnjhiEJ23p/a38w=; b=FgVBcoeOakn0yVdM0k9ezq+vOwH6hwaYVJfu0myL9+mrn3VDHcv1cEvZbIS6HJxesOj/pU CZF6Afz2GWexOWT+hduGA8DwpJX241XrZagxUKtGlLVSB4pY5cftOUqSqv/cfH+gfYLTnM 1DpIKouKqHjFGyi6l/VbY1fvs4dPdKo= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-25-2RWITuSMMsqLOP5NwQ0AiQ-1; Mon, 08 Jun 2026 04:38:39 -0400 X-MC-Unique: 2RWITuSMMsqLOP5NwQ0AiQ-1 X-Mimecast-MFC-AGG-ID: 2RWITuSMMsqLOP5NwQ0AiQ_1780907918 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-490ae4d89e2so40911445e9.3 for ; Mon, 08 Jun 2026 01:38:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907918; x=1781512718; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=vosxR5Z06o9BwR/z8OHSdoefhvjInnjhiEJ23p/a38w=; b=ZMj5TceT1Mz5VlvmgbBY69tr61zAyhjXOMlvszieeq9ETahVQJHQBkeRQW12l51TCi XNJ1W1p4ptHoPPNTeger3aVvBSSVZOl1nrbMyKmE250CD4qyJWiwXMA7DigD/y0zKaRp c2/5JuRh8kGhNAeKXMm1QFB8bVky9Cc26Fx4LLnY2CfRBBQT8YCYiHqXO/S/lyh3fNTL 3KmuMW5/ggBSvk2e7lA4gkIwM+lIQ/TrxvcL6iUBZWnLiAx+UTgsoLCsitIbLG/x7by9 B9TC2o98qygiwtlLA0fsQeRhe6kW816KiWLY5XR5lfn5zmaV5ag7LfIc0bHee3gq2p3T su0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907918; x=1781512718; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vosxR5Z06o9BwR/z8OHSdoefhvjInnjhiEJ23p/a38w=; b=DNSbvSebdtJqgO47C9mBxlboIaIc6Dx4l7XksDvKM9HgfA9My+tpv6ZRr5zV4YoJWa dn5ZKE+DvL+m3zHQ31dFfCrj7gLPw9Pid3OIECbVItAsTe7zsvtXA3TjzaIvIdxiTfkI ohcwawZ5AAOxMWZihhZwRFfW0WTIZRxuz9QMurLv9pWG1ok4cXtbIkJKzvzdD0tvWbUA B4Y76iIOjyeIr1aB8EsxsGmwIkPOyAobrLkFcdqpzZaFHo3rvZplonsa4gcEFyFVpO0a paDXHMzLRX5kOTincB1bj29pFBUDiEvu1TfhchqwO94YnXmB35Xp83aWWEJUbZ3E72iZ lAjw== X-Gm-Message-State: AOJu0YwLLS3rk74+H6WYKKt9QLJxaUXUciD9Ywj/Ltl+DNpXLXR9Ia8e Lgkrq0BvO5iCEKPdlmN+fD6oq6lAY0cZgMy9fe+Cf6/8+T1FVgEMkxqQpEJdQXLRsBJReedo7Kz qnzQSjLrA/02Sz72DMaAOkzAAjFdEMuCCDC+i15FjJzWFAEk2kfhX+Te6sdyWKNRccNV9sdGl+x lNgVhhFMoS7tn41CFZtD13hHDLAsOvNmEjltGcyTjyXCY= X-Gm-Gg: Acq92OEhayCATNHtnTq8Ikk+Pb0b54DjsnTlhAJMzegr+ECUqtfAJm7BUXUapFi8hYS cne5m4EN9ltHem1zQ6jWWPjdym/szGWFDP662/7GhUjY8Z2ji0tCyJMNMmy4evQckifldfKbydn 41gjN36UArQm1KjVn4D1bTfJXiDHGDyEccsYCHoQJ4WLvFON51ZdVgc0TFzSwDXWFBxtNZRQUDU 0C8muNmEsSbmm6+yjVyt+dBwb/GXNMRkAyta+MtphFNzhQN4UKu7NM0CCSpBJWBDnCanpF/PhDI E6+sGD6lcmMLBmZdtemJmWHw7bePR9dS0BAhV1qrvjuctQM01vXOzQnUnNetn4R0c4rl2npDbEL EQSTqUNtyKG1Xog0nDhm1ab0= X-Received: by 2002:a05:600c:34c9:b0:490:688b:f10d with SMTP id 5b1f17b1804b1-490c25b3549mr252415345e9.7.1780907917607; Mon, 08 Jun 2026 01:38:37 -0700 (PDT) X-Received: by 2002:a05:600c:34c9:b0:490:688b:f10d with SMTP id 5b1f17b1804b1-490c25b3549mr252414265e9.7.1780907916933; Mon, 08 Jun 2026 01:38:36 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4601f2e4004sm50788900f8f.9.2026.06.08.01.38.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:38:36 -0700 (PDT) Date: Mon, 8 Jun 2026 04:38:25 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 21/37] mm: page_alloc: propagate PG_zeroed in split_large_buddy Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When splitting a large buddy page, propagate the PG_zeroed flag to each sub-page before freeing it. __free_pages_prepare clears all flags (including PG_zeroed), so the flag must be re-set on each fragment after the split. This ensures that the buddy merge logic can see PG_zeroed on pages that were part of a larger zeroed block. Signed-off-by: Michael S. Tsirkin Reviewed-by: Gregory Price Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/page_alloc.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 7a6dedd716e2..21f9e92922f1 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1520,6 +1520,7 @@ static void split_large_buddy(struct zone *zone, stru= ct page *page, bool reported) { unsigned long end =3D pfn + (1 << order); + bool zeroed =3D PageZeroed(page); =20 VM_WARN_ON_ONCE(!IS_ALIGNED(pfn, 1 << order)); /* Caller removed page from freelist, buddy info cleared! */ @@ -1531,6 +1532,8 @@ static void split_large_buddy(struct zone *zone, stru= ct page *page, do { int mt =3D get_pfnblock_migratetype(page, pfn); =20 + if (zeroed) + __SetPageZeroed(page); __free_one_page(page, pfn, zone, order, mt, fpi); if (reported && PageBuddy(page) && buddy_order(page) =3D=3D order) __SetPageReported(page); --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0BF053B7B8B for ; Mon, 8 Jun 2026 08:38:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907933; cv=none; b=F1QBFvE1f5Nwzl8QoxYiL0K1ino1SQoklIGbnf8mOeNvSI93y2qJrGg8SYpf/3FCESzNLLsBlLtJ13ahS2rKEo/JC46pU6lQ6TmQI3+3pN0s7Uo2PN42rN9hoCu5tDa/u+k/pRjKMXDUFe7pl7I1nzyuy2Hu0oFE1qsuWLkCiSw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907933; c=relaxed/simple; bh=XOxo/dtWeCKwLWC3rRtMtJbaGbjkk6GuaLkok0aBtBs=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=LOCO4f68qAggBHW7ghVwB7ZT1WnxG4LrevgvcvEwTPzr7shzQiq2eGcXUxnz2gAzUS5WzmvxBtDRiWollquLkyfwX8VsUaS30BWXAxNGZ3CEUmOPjrM5vWYZlSNmuj4M3IIomqsJTx6O4Wxhn7mVABqhGaXqh9wDpFdzs73JKFQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=DlTapet2; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=Urlpb5/2; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="DlTapet2"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="Urlpb5/2" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907929; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=1lvfH8Mz6BNBeGQvvcXVuozV5D9pusYvRbrmVhIfIf0=; b=DlTapet2T4ktiww2FSwxG+lE4KDjZIcaI4onjkGmuFPZWex1/xziDRMfmcplF7OvBOiwWC JxEsGtsQPVaNQBh8r61ys2VGLKBFvoEoUerImHnAt+jf9xvb2kC29uQJAp9LF8DNaeplHC 819B45EDsamu6FNS5ZmjsG62aaaFj3o= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-138-_Zy8ioxuO_ehsQ7hgBFV9A-1; Mon, 08 Jun 2026 04:38:47 -0400 X-MC-Unique: _Zy8ioxuO_ehsQ7hgBFV9A-1 X-Mimecast-MFC-AGG-ID: _Zy8ioxuO_ehsQ7hgBFV9A_1780907926 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-490ace40f13so18300575e9.3 for ; Mon, 08 Jun 2026 01:38:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907926; x=1781512726; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=1lvfH8Mz6BNBeGQvvcXVuozV5D9pusYvRbrmVhIfIf0=; b=Urlpb5/28rLuJcG8oBKSQPGzpKDKu+h2dCAY842GtiMjYWqDMigKGiqf8GdktvW7yw N1AytAJnsi6UU5klFh3n21cjHonMuR6sMcS8+0Q13r0Rbn/+o6vQD4cwqOiaTCCFOMnF DIK7KAc+dUR3iKJTZn0OX5afKJ/jqBWnTiMTAdXQqsir5BXmmI4BR+M9duLbWrHo6Th1 F7X94CpW7JK2/h5aDh3dKVF4g3zbiBnaI4qPI4fa9ONOxLe4LXCrAnaGD5oF4wvsdxFE 19oN1BtuCiQsVJODyrG7AltYqTv/kIXg3VYvGw0xk//yx8M2BzsxoydhMY277KF6Uas3 dzZA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907926; x=1781512726; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1lvfH8Mz6BNBeGQvvcXVuozV5D9pusYvRbrmVhIfIf0=; b=YZ5sjDrf0HtotmLRGhCYjDn/INFcUvvZikcqkcjjWwHeTlwR+IQRASar0aMWAorIYq WhWDyoI0WMOa++/RNnkeDUARMdZ3y1msj6RlRqqCli5RMhinyR0wIGn3xO0bde00/S+A w8/raGRudon5730LI7mUZbSJZvClxABEtrPwQ7ZF1diirIsb0oGitvpjcw1a7xVi/8cQ 716V016TuwEA2ujsS+QyugyC6n+mtzYi4tr9io6SUee9ABgYnHmqVOXtxV6UrjosDH/c JRwNLCGkxdAgIJOI6yJAFJM1Hys5hW0bKGh8/xVwjuMkr+spMC42YlMZdNQ0/OiLmQdP pD8Q== X-Gm-Message-State: AOJu0YzNWKwaiWvlqV1AlNhjZE4Dh5jbHMp2VCuSrWtufn1lWp0grh+A W5ubTuaLigcshkK8U5qA13hD6Evf9Du8t4AXRnbhxQHUJQGPOUFmNoPHc+GUUlXui810ahxwQ8O S9Ea5HFJNWiD6TohY9Z5BRKl+UEzNRGES4Ph3KJLTeocbedPTAqUH6cD4W9CJFr5fcK0F8z8O0d gEmKc1QA/M3GzPJfanb136KwZz2WiEdmn+NPhB7B3fNMA= X-Gm-Gg: Acq92OFvxzkBXGj/tu/n/QsUL/r1wzKmit5BFzZF0Dgj4bClCsKP9U/MONT2d2okB10 Y8q3A1xirAZWwZDc/P15OZrso74zUGdBD3R0cL7eqB3qM8Z7eLkKsn9944sVVZBp6uMts93zcW6 dhpoWXMU3CgtEPT73rCeqlqwgb628sO3GJdUrtVlzFUM8XqGvTyvG8QTEYAx4X/j+skXboQLpbm DawiYfJZWPEpDBFAUZN/Dk8tXCflyiSP6hwPUx/udZ4P2Wd7klRB1uLUeCFJ94ncd+nbqkgX1hE JH+YlLyNqVMhaujD8BwfCHXSWvty/Gnk1b5s3XqLL/BC1Qk6hlG5OPVRG3ZFfcEshNIebFpy36c QqbCnTsiU2nwtTAhWojg74Nc= X-Received: by 2002:a05:600c:628c:b0:48f:e26a:1744 with SMTP id 5b1f17b1804b1-490c25a87f5mr237272785e9.9.1780907926368; Mon, 08 Jun 2026 01:38:46 -0700 (PDT) X-Received: by 2002:a05:600c:628c:b0:48f:e26a:1744 with SMTP id 5b1f17b1804b1-490c25a87f5mr237271745e9.9.1780907925906; Mon, 08 Jun 2026 01:38:45 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4601f2eadefsm51854339f8f.11.2026.06.08.01.38.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:38:45 -0700 (PDT) Date: Mon, 8 Jun 2026 04:38:37 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 22/37] mm: add free_frozen_pages_zeroed Message-ID: <4220116b4d91f3e933fa76ad473106c2e48fd452.1780906288.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add free_frozen_pages_zeroed(page, order) to free a frozen page while marking it as zeroed, so the next allocation can skip redundant zeroing. An FPI_ZEROED internal flag carries the hint through the free path. PageZeroed is set after __free_pages_prepare() clears all flags, so the hint survives on the free list. __SetPageZeroed is non-atomic but safe here: the page is frozen (refcount 0) and not yet on any free list. Note: when want_init_on_free() zeroes the page via kernel_init_pages(), the page is zero but the direct-map cache lines may be dirty. A later patch (skip kernel_init_pages for FPI_ZEROED) avoids the redundant re-zero, and post_alloc_hook handles the dcache flush for user pages on aliasing architectures. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/gfp.h | 1 + mm/internal.h | 1 + mm/page_alloc.c | 23 ++++++++++++++++++++++- 3 files changed, 24 insertions(+), 1 deletion(-) diff --git a/include/linux/gfp.h b/include/linux/gfp.h index 73109d4e31a4..d24b61e45861 100644 --- a/include/linux/gfp.h +++ b/include/linux/gfp.h @@ -384,6 +384,7 @@ __meminit void *alloc_pages_exact_nid_noprof(int nid, s= ize_t size, gfp_t gfp_mas extern void __free_pages(struct page *page, unsigned int order); extern void free_pages_nolock(struct page *page, unsigned int order); extern void free_pages(unsigned long addr, unsigned int order); +void free_frozen_pages_zeroed(struct page *page, unsigned int order); =20 #define __free_page(page) __free_pages((page), 0) #define free_page(addr) free_pages((addr), 0) diff --git a/mm/internal.h b/mm/internal.h index 4af5e72742ba..fd910743ddc3 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -938,6 +938,7 @@ struct page *__alloc_frozen_pages_noprof(gfp_t, unsigne= d int order, int nid, #define __alloc_frozen_pages(...) \ alloc_hooks(__alloc_frozen_pages_noprof(__VA_ARGS__)) void free_frozen_pages(struct page *page, unsigned int order); +void free_frozen_pages_zeroed(struct page *page, unsigned int order); void free_unref_folios(struct folio_batch *fbatch); =20 #ifdef CONFIG_NUMA diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 21f9e92922f1..008f1a311c40 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -91,6 +91,13 @@ typedef int __bitwise fpi_t; /* Free the page without taking locks. Rely on trylock only. */ #define FPI_TRYLOCK ((__force fpi_t)BIT(2)) =20 +/* + * The page contents are known to be zero (e.g., the host zeroed them + * during balloon deflate). Set PageZeroed after free so the next + * allocation can skip redundant zeroing. + */ +#define FPI_ZEROED ((__force fpi_t)BIT(3)) + /* prevent >1 _updater_ of zone percpu pageset ->high and ->batch fields */ static DEFINE_MUTEX(pcp_batch_high_lock); #define MIN_PERCPU_PAGELIST_HIGH_FRACTION (8) @@ -1596,8 +1603,12 @@ static void __free_pages_ok(struct page *page, unsig= ned int order, unsigned long pfn =3D page_to_pfn(page); struct zone *zone =3D page_zone(page); =20 - if (__free_pages_prepare(page, order, fpi_flags)) + if (__free_pages_prepare(page, order, fpi_flags)) { + /* Don't mark zeroed if poison overwrote with 0xAA. */ + if ((fpi_flags & FPI_ZEROED) && !page_poisoning_enabled_static()) + __SetPageZeroed(page); free_one_page(zone, page, pfn, order, fpi_flags); + } } =20 void __meminit __free_pages_core(struct page *page, unsigned int order, @@ -3020,6 +3031,10 @@ static void __free_frozen_pages(struct page *page, u= nsigned int order, if (!__free_pages_prepare(page, order, fpi_flags)) return; =20 + /* Don't mark zeroed if poison overwrote with 0xAA. */ + if ((fpi_flags & FPI_ZEROED) && !page_poisoning_enabled_static()) + __SetPageZeroed(page); + /* * We only track unmovable, reclaimable and movable on pcp lists. * Place ISOLATE pages on the isolated list because they are being @@ -3058,6 +3073,12 @@ void free_frozen_pages(struct page *page, unsigned i= nt order) __free_frozen_pages(page, order, FPI_NONE); } =20 +void free_frozen_pages_zeroed(struct page *page, unsigned int order) +{ + __free_frozen_pages(page, order, FPI_ZEROED); +} +EXPORT_SYMBOL(free_frozen_pages_zeroed); + void free_frozen_pages_nolock(struct page *page, unsigned int order) { __free_frozen_pages(page, order, FPI_TRYLOCK); --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D6F7A3AB283 for ; Mon, 8 Jun 2026 08:38:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907940; cv=none; b=CKbWLXwK8TVOCRb0+IthTIqavbZbf+Ibo8rYjs7zeiuUi6M4M619MqSyjM1TzQlgwVbi5NdL8Ny7JEInXRtzyELGeyGuDgomGKmusPUq5euPJcX1zeQf2Iek+WU1dCCC3Ed/1UINI0b5wWx7CgQQA6SteuKNdGOCySJCg2SE4R0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907940; c=relaxed/simple; bh=9G4viGLxhnf9wEDx0vlsgDGsNz6CjvZFH6MPF7/wsQQ=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=LF2c8xmkyhFOfRh2hIFDXKuMCKum+w7/300E2xypi5A8nI7kCIx/xyIQLPONHjVMKtvb2bzLR9iKB7EgIaLbK24vNqyr1E2sW4oxxUj1beC1vlNTjW7cJu5clg8rVeVg2tLAhbhOJKsTM+FeMYAZR98OhAtIQ9+zekXjCddUEVE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=AFH0W5OE; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=ijDAX9b4; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="AFH0W5OE"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="ijDAX9b4" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907937; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=EnA3ajreXHx2VoG4/VPZjRVpyyF9Dgnshjyr2lhZIq8=; b=AFH0W5OEo2Qn1vreC/ANDj0Uke9EOnTnojRzUPtxHwhOyPLsqyhbJ4vnDk42ye+2NOc7+Y neqDHwvsJuKvjmSnIv17X98aQvT9RweaR5F78FD2T9YDfLXAcmtCCVyGgNEsVEchwRisLC Bn/b9HFG6dQKzmA/7qMgDS/81TtC+L4= Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-561-0m0VvXA2O96Pc83WzYjKMg-1; Mon, 08 Jun 2026 04:38:55 -0400 X-MC-Unique: 0m0VvXA2O96Pc83WzYjKMg-1 X-Mimecast-MFC-AGG-ID: 0m0VvXA2O96Pc83WzYjKMg_1780907935 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-4601102c30dso2291581f8f.2 for ; Mon, 08 Jun 2026 01:38:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907934; x=1781512734; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=EnA3ajreXHx2VoG4/VPZjRVpyyF9Dgnshjyr2lhZIq8=; b=ijDAX9b4rIkoT5a+5IEOvIWNNedIsVXlVE+77nexwSi/Qub/l+pZxcOl9GkPvMwma/ HY+/mH+7O90TuuY8PmUmjjvMiPIny3NkhmicBjQ4wQIawwihRDrjcC0xrKywsVZYu/jp YA/kThzorg5II86GwOxn4VtKrnlbJACpZOGInfjqL4UbS+QyVPnuZcYjyioWYVWBawtE yhzvkTbDefbIdzw5T023MzH6bWmZ1vxzD1eOGmIqhSlZ1NHhcUJLmFh341M2Fw+U8ehT UG9+vsw9Gg5LKtJKO/1QI/C7vu9OQdqlHTie78XcV7vyVS9SfCUPb69YaZZnyct7b+hb NA5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907934; x=1781512734; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EnA3ajreXHx2VoG4/VPZjRVpyyF9Dgnshjyr2lhZIq8=; b=TjxMc7xEe+lmwwrO35UmzpF1ha6Hponjl+Ya2ScMgGCbEqNy0pQ5QouDA08JTdGKGC qM/Lf95VREPg3pEbOfbhNXvx2LF2XcyNVgMnMkxM1XwUArG9PTDJDWNAK5sY68THm8jq E8wSR88iGf8Nc9y/u5f2gYTMFUp7XnKtTaEMPHQT5fvDGXeNqZsQgTt/dKwT7hj6Ka7Y aLsaGKNj+X30ECAZVyZ4sk7j04OOHuHRiZ+P9iM+DFzW234/YgjUfLU3gZEfEyHy715O Mv0o0fR9HNt1t6n5dZ0tQBsh0p+/4/Rfm16KE77GvVk6NhyStTGe+i/Bhl9SjRNaArWW Tp/Q== X-Gm-Message-State: AOJu0Yy8XwstTJE7fpOQ5V8v5etco6AmSyVUiI4idsye2k+LSbfIDa7U sQybBac7as61qCRkcfEgbY9/5HvDlAEEL6soFY9WreWXFoiwhKxQobxW5G1JWqZat8SMxDgREA/ i32BL922vb6qA1SadxiJ3s/jauTcC+lTvaELiIMWxKTltwtjC9GZtQ8diKr1u8fORy9E0sEfmHv 1W2oVjz2Au1K5hSfFCCiEZ7EY4ysR4JgvkynpWRgk82rc= X-Gm-Gg: Acq92OFFALfxRB+Ty8CGNl5LCID/2oiGW5fX8GzNiQk1HYhPYxPDNscSCULPTX6atF1 2Oo1unXENn1qfyffCVNXbLunHxoyxjFLC573cxTbT4/7bJSk1ZY+sIYZbYlfqCZJoeDMHaZAJuy Yuo/TDhwb0z9dv/FrnWLFeaitdFR06uXIVWoJGuaxoFnOI57VdaWhssUKD4Agqj7l0i4jw72Wv5 nrE5zQAK9v9+TphJNsdk7Xg/yQKZ243wZP/6ZBnfJo1I6cFTr+4AveqXrCXp2fADaHCKqQ7mErh 9HUoEip3m6HUDVb0p6GM7W0itkyNGKqdMhjS9sxb5FcOJM90ftgPYYtaxPaUOyEQo91rkrO7G37 8rTtqcmOhSXC0QZEbdSsxqt4= X-Received: by 2002:adf:fcce:0:b0:45e:df08:b18 with SMTP id ffacd0b85a97d-4603075ca65mr15338438f8f.32.1780907934439; Mon, 08 Jun 2026 01:38:54 -0700 (PDT) X-Received: by 2002:adf:fcce:0:b0:45e:df08:b18 with SMTP id ffacd0b85a97d-4603075ca65mr15338352f8f.32.1780907933822; Mon, 08 Jun 2026 01:38:53 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4601f3446aesm55104695f8f.24.2026.06.08.01.38.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:38:53 -0700 (PDT) Date: Mon, 8 Jun 2026 04:38:46 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 23/37] mm: page_alloc: skip kernel_init_pages for FPI_ZEROED when safe Message-ID: <7de6959a0fb1b255dc9bee2de494f51a6c21e468.1780906288.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In __free_pages_prepare(), when FPI_ZEROED is set the page is already known to be zero. We can skip kernel_init_pages() if page poisoning is not enabled (because poison would overwrite the zeroes). This avoids redundant zeroing work when freeing pages that are already known to contain all zeros. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/page_alloc.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 008f1a311c40..e3a7c40c769c 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1443,7 +1443,14 @@ __always_inline bool __free_pages_prepare(struct pag= e *page, if (kasan_has_integrated_init()) init =3D false; } - if (init) + /* + * Skip redundant zeroing when the page is already known-zero + * (FPI_ZEROED) and page poisoning did not overwrite it. + * When page_poisoning is enabled, kernel_poison_pages above + * wrote PAGE_POISON (0xAA), so we must re-zero. + */ + if (init && !((fpi_flags & FPI_ZEROED) && + !page_poisoning_enabled_static())) kernel_init_pages(page, 1 << order); =20 /* --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1BC093B895E for ; Mon, 8 Jun 2026 08:39:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907949; cv=none; b=lvs1TZ8dfQd5cX+wcBTaOIwucQbyA56GRqYwJkTAMHcN92NJ6lGylaHcGpDzcWVXl/bF3yLfIcJV4fUwmjLH+tzePpcMrbaCoPlCOrk2AUrtwvYCD1Eq/7l0ylFuABOygugFRJg1EjZNolSqZnEQIroOYkvWQ49leaxkonqGniE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907949; c=relaxed/simple; bh=JqpRnHMOZCHNy3L9y89dcwb7zbUTjUHqd7YR7olPip0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=f3qWQu3UJnFqZrbu+0RZUEsPL5OFp59AXBK1jMcPGofvPuWB4LyM+DYShHIpj8VuZ2kcC0k9n71rY/gI62YhIAqoW/u+FkhxXd7faKoRf03EXfi19w+1/AycPdwTNDyo7FuOWVjcjjx9R8jtqI08Rzjct77/9caevQ28yzaHJFc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=bVEXtFev; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=syHeloBy; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="bVEXtFev"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="syHeloBy" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907945; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=DMc8fDrj0D8JcjJ5dRd88tqBSMHmwk7bs26tvZRKcB8=; b=bVEXtFevJKB79zHKYaUfAZVYq5FkvbXosQ4kNoJ1Ky/zMh7zqDCF52oXC6yXHFmMDrMOaC 43DXRF6VC8NFbY9rN1xqQ1fYGCskYe99Mk538u2ya44JWvSAR+ZBy8O+ZY/ID5CQyh6bmV 2UzaY4LUu5aLz/ZGZMkMaqoYFl1AeEQ= Received: from mail-wm1-f71.google.com (mail-wm1-f71.google.com [209.85.128.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-108-2OA2lsmDMXiwczzIwWIElw-1; Mon, 08 Jun 2026 04:39:03 -0400 X-MC-Unique: 2OA2lsmDMXiwczzIwWIElw-1 X-Mimecast-MFC-AGG-ID: 2OA2lsmDMXiwczzIwWIElw_1780907943 Received: by mail-wm1-f71.google.com with SMTP id 5b1f17b1804b1-490b37e1f47so36123095e9.0 for ; Mon, 08 Jun 2026 01:39:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907942; x=1781512742; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=DMc8fDrj0D8JcjJ5dRd88tqBSMHmwk7bs26tvZRKcB8=; b=syHeloByWO/3ja62bJctJUW9RlVA7+bb5nMX6AQRuPuV8E+hCSCVrTFTuBxgYwLSxk 5VXnkF55EEO/K1MvhTaiuq7brROonoiQYXJqbDC2M5LOZoNkuKnJqLPR2hSffOlAxNeE twdraarMg8owdkPENlsDsJQRCIs/e3FL+RszlOdp+bCJAv8qdJQevz27quaX5UdSy9ul saO9X6RTiUQ5b1e9wOGziHwXlAhN1Vh4wj6bKyDPyjitCkOPe6Rtcxjm/GjoddQnEAVr xRWFm6dX/s42PSKtbUK91tp7aClLTa5CMz+DnGjvEnEtPo0FwvBjZ6x1qy7alBEqHN50 b8MQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907942; x=1781512742; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DMc8fDrj0D8JcjJ5dRd88tqBSMHmwk7bs26tvZRKcB8=; b=nu9pMZaAwyMB51nsE4pIMKOFyeww4cEa8Bdi31PvAyo41rT/Azg9zUtVJ0stLT7Ne4 AN1TeWxldSLPgeTEk9pYgi4CDAymw1kclLc4yZR2/XXlQgNF3AEzXbK3Ik8olMuOh7YX XFZYENxQ1sp17M+sJQFCVJYbh0YNSvaEs0yaFgbTGuL6Gw+7ZVfe66M6ysDBuiTL/ilh rGvBZDOAz5YDPv21+Ujjutxo92PfqXBcqQsBs7BSpICRBqIAfZSf9+/tOr+BypgBZ2to NpMmvypbd2Sg8yGLANEYkfsjNzhiYb0eVNiHK/kL8maGIue89vnLbRaGYhgNyke56uyi DPTw== X-Gm-Message-State: AOJu0YyjKdCltAx7nNbHXV/XfDbwwyi739nnoojHV3X8MELjCX6Yju0L VOKjKUlJ+A5UqYjVyoJHtbqQ10kMnwRbD2nNczBLon0zSXxefN4s8OmnUahw465I9WhT/gDEhQo 2W3FDREjMod1iPsgK2tBXfS+QkZHzkY7yB7Ej0P+BUaQFEBIS88Lxyg3jvAphteFz6mkSjcSLHn 7WF2l0WBE2d3PdFGArUDpOqhcarvme+lNsfb+TO0GvFso= X-Gm-Gg: Acq92OFzM/ZH25JRL+OAliTJ2xv6gy+TlXzKsrTeF1aa8w0g1jaxZQ2l1FdeVHrslhz cETGXBvEm2qN+7no3RfmaMHxX2Xpigd3Bsaq5MmWygEyarAea7L+pRpJLK+GMcq1KINUcBSSBCz 27AdXoC9Ol4Di5qG0g5BZvMSTJGyhAhnEHAJeogQXuBXy6QDgnfrS4OKCg2/6yVS8eCHsB+wZid M0sr0O0Xf/WUHen7cxF+GD3D/R5DLKLcT+n9GOivbXH3sdEWdPJzer/7WBzYNpQCCbQu3OENa3a 19UbomE1+a/xk94mZCKrJpFmbM/PHQkqbdeP28HtcJ3PkPyn1G2gQmVWaLtOHYCKoD4TZgrVV3x 44qwd+nl30ZxBRZUNowWhPl8= X-Received: by 2002:a05:600c:5252:b0:490:b2a6:8c1d with SMTP id 5b1f17b1804b1-490c25adb2fmr254958815e9.10.1780907942479; Mon, 08 Jun 2026 01:39:02 -0700 (PDT) X-Received: by 2002:a05:600c:5252:b0:490:b2a6:8c1d with SMTP id 5b1f17b1804b1-490c25adb2fmr254957625e9.10.1780907941897; Mon, 08 Jun 2026 01:39:01 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4601f2ed944sm52040479f8f.13.2026.06.08.01.38.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:39:01 -0700 (PDT) Date: Mon, 8 Jun 2026 04:38:54 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 24/37] mm: add put_page_zeroed and folio_put_zeroed Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add put_page_zeroed() / folio_put_zeroed() for callers that hold a reference to a page known to be zeroed. If this drops the last reference, the zeroed hint is propagated to the buddy allocator. If someone else still holds a reference, the hint is simply lost - this is best-effort. This is useful for balloon drivers during deflation: the host has already zeroed the pages, and the balloon is typically the sole owner. But if the page happens to be shared, silently dropping the hint is safe and avoids the need for callers to check the refcount. Note: put_page_zeroed uses folio_put_testzero() which only detects sole ownership at the instant of the atomic decrement. A concurrent reference holder (e.g. migration) means the hint is silently lost. This is by design: the zeroed hint is a performance optimization, not a correctness requirement. Losing it just means the next allocation re-zeroes the page. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/mm.h | 13 +++++++++++++ mm/swap.c | 20 ++++++++++++++++++-- 2 files changed, 31 insertions(+), 2 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 06bbe9eba636..79b3a8cb9a3b 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1913,6 +1913,7 @@ static inline struct folio *virt_to_folio(const void = *x) } =20 void __folio_put(struct folio *folio); +void __folio_put_zeroed(struct folio *folio); =20 void split_page(struct page *page, unsigned int order); void folio_copy(struct folio *dst, struct folio *src); @@ -2090,6 +2091,18 @@ static inline void folio_put(struct folio *folio) __folio_put(folio); } =20 +/* Caller must be sole owner to guarantee page is still zero */ +static inline void folio_put_zeroed(struct folio *folio) +{ + if (folio_put_testzero(folio)) + __folio_put_zeroed(folio); +} + +static inline void put_page_zeroed(struct page *page) +{ + folio_put_zeroed(page_folio(page)); +} + /** * folio_put_refs - Reduce the reference count on a folio. * @folio: The folio. diff --git a/mm/swap.c b/mm/swap.c index 5cc44f0de987..ecec780172ad 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -94,13 +94,15 @@ static void page_cache_release(struct folio *folio) lruvec_unlock_irqrestore(lruvec, flags); } =20 -void __folio_put(struct folio *folio) +static void ___folio_put(struct folio *folio, bool zeroed) { + /* zeroed hint ignored for now, no current user */ if (unlikely(folio_is_zone_device(folio))) { free_zone_device_folio(folio); return; } =20 + /* zeroed hint ignored for now, no current user */ if (folio_test_hugetlb(folio)) { free_huge_folio(folio); return; @@ -109,10 +111,24 @@ void __folio_put(struct folio *folio) page_cache_release(folio); folio_unqueue_deferred_split(folio); mem_cgroup_uncharge(folio); - free_frozen_pages(&folio->page, folio_order(folio)); + if (zeroed) + free_frozen_pages_zeroed(&folio->page, folio_order(folio)); + else + free_frozen_pages(&folio->page, folio_order(folio)); +} + +void __folio_put(struct folio *folio) +{ + ___folio_put(folio, false); } EXPORT_SYMBOL(__folio_put); =20 +void __folio_put_zeroed(struct folio *folio) +{ + ___folio_put(folio, true); +} +EXPORT_SYMBOL(__folio_put_zeroed); + typedef void (*move_fn_t)(struct lruvec *lruvec, struct folio *folio); =20 static void lru_add(struct lruvec *lruvec, struct folio *folio) --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5CA933BB670 for ; Mon, 8 Jun 2026 08:39:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907960; cv=none; b=aSVj+roUmbf3tdtq8hP3PWeLam5qbdodXL8MqbgXQsBT3STH1CCRl0FnGu1tgW7i2Li9NWZ7vCBxOjqDVFY8oPU3YGrj3+wNTIp+GXxjGxA9CmwiJACNMjOC7TjKa99/k2JfdNZrOfkwLauUg9hhvZ2fXFCIS3b+8u6B55TSrBo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907960; c=relaxed/simple; bh=GCycL8yBIVDI7qw2NQ3Ow74lnO1SdliLzn2m5qQWypg=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=lXWEWLhpWt4eV/ec5oasdnIf6J0qu0IVkKPN4RsDwaBI79U/IdiDcqkuBRghitbBcjaNoqT6vsKPXe2Gbru/k/S9nfDmgMGgNgU2Yk7N+PEHi0roTYk3xBQ9FAed4OR+moJ+EEm9CR1WfOtl/k5Mc/RqKVtzdu0awxG6Ctob4qA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Oc+QV+qs; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=nzEQA5ti; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Oc+QV+qs"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="nzEQA5ti" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907954; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=3+rjGl0pefUb9vVc/tCmuC8Z28GLOAKmmZQW2TUwDlo=; b=Oc+QV+qs9O3xfqQQflJcbNRabWfHjVlloaRFnzN3mCCW+M31WbwmvhXSWjtIknERJDhox4 N9tcvjXj0Uxs9VwL+poSFf45qk4+TPpwfqRz5Bigcu0sunhE49j5ETt9qlO3FY9oAAdM2t UJdQK0lJXAGBYK5UAWRjwhVEcWa0pvo= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-396-46nWSEFYNvulAwSad06P1g-1; Mon, 08 Jun 2026 04:39:12 -0400 X-MC-Unique: 46nWSEFYNvulAwSad06P1g-1 X-Mimecast-MFC-AGG-ID: 46nWSEFYNvulAwSad06P1g_1780907951 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-45efa12a788so2885226f8f.2 for ; Mon, 08 Jun 2026 01:39:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907951; x=1781512751; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=3+rjGl0pefUb9vVc/tCmuC8Z28GLOAKmmZQW2TUwDlo=; b=nzEQA5tiICNOagPm2zFLXj+kRP7aiQdgXoIpkOHtc5NMHIeHk/AL1i27YM9eMwfVLS pgP8dgVviuFvDwroo+cvlQov0a+8tvpzJbRe1Rb9ZvaklbN2SGWGBPF0sZNPCKCA8Bx+ QNDKL3Sx2Okw961S+wBLoUEkowBU9CanI0pM+HI8akpFud/AJQ4cnJe3rz93rhE7vuzB 2WPbo2B1Vs09qw6pl9ZlipNi0QmdGuzvvUdxC9NtXbGbZJRowVCiG1Jr3fLCj0YUXkRQ duj913qFHiHjH1TuaNO6CG5d2cOHGXeJMYZaou1OfosQgOuDc/H4LdnLZ19PTj+pZpem wXUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907951; x=1781512751; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3+rjGl0pefUb9vVc/tCmuC8Z28GLOAKmmZQW2TUwDlo=; b=JXxQ3KjRpruXZfx9QI1ePqEK4VXCxRbB96ELMwoMfF86Lf1umd1iD6RbGy9lWC05In kBKRPs6sdNryKJS5XfqiIOJG8oSGVTuqb8fpSgrIHdiXHdCpZTQMzjx66B0+vrIq/Byq zp4wpSzcufl64gped5KXorETHwwQsytakR2AF6/5DmxgdcYDZ2MaNYtzsYGV1YBMTsd6 e1wRxPJKdHg25A+T3K68qLdQp1dZgeX6baOMeFuNHVp3UMofH6TQp+NxNn4w6f+A8jSO +G3QVzsaa2fTLDZgYi/WIZD3BAwFBlpjSbte2X1uAeWToedPu3FtxU07bu0iBnJNnh0X Msxw== X-Gm-Message-State: AOJu0YyjKBx02+9GHn4xp+7Hi5klDDeRg0XB/zVqaGC5ios53pNWtIX/ P5nHbaBucSWAhypkVFwsew8MvvrX1Xfyz22pYBhT4Jn1c0/ozk03Z4gCTaybUFA2SnM4a8C3MFW kBv57ShYZL09sDwnhf2UPJn+l4PfJUREFhokbHiQwMg6lMAJ3/faOPEs9vBCnWFFxsHbve+laCt iep0Hz26mqkRlMxrbEiXQiIa3KpyRZxXyCf9ThSA/oQ3U= X-Gm-Gg: Acq92OH9xoUQPLqV6umJxd0KQsrtwR6ba3mKAvidB6ODLUiW9Gd76O1Zmo2/K+4S8CZ xIurWs2jPdtM5a/EJgHAM0c6I3VvWtfChXw6tFmQg7MSCDDPDBf5LOtH4cMPK9ntCj1tREwibPV 3XYCLPSwIPxutdWVunYTrdb8YCadCAErgMJRCHKK2V487M62W2DZI5yvYH83aJDwNfQKdwmGswc SJdUs5MwKuOcFSUsV0Dk75y5D3CXzwbBJQ8LXPGxJqcJZ1N1vPq6V56cFdO7MqTUs7zN960bujx SlgIh/PM6cibAOEaqIDQiu0IU4ep/Z4pJeQXFWxeCH2bB6NI3utWFyFHrn1Foy/nMaiXUiq9QD4 sPuaky+vXaIw9mzIyMFmK584= X-Received: by 2002:a5d:4a85:0:b0:45f:f142:d55a with SMTP id ffacd0b85a97d-460302e5e92mr18158194f8f.14.1780907950778; Mon, 08 Jun 2026 01:39:10 -0700 (PDT) X-Received: by 2002:a5d:4a85:0:b0:45f:f142:d55a with SMTP id ffacd0b85a97d-460302e5e92mr18158101f8f.14.1780907950019; Mon, 08 Jun 2026 01:39:10 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4601f2f67c6sm48744304f8f.16.2026.06.08.01.39.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:39:09 -0700 (PDT) Date: Mon, 8 Jun 2026 04:39:02 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 25/37] mm: use __GFP_ZERO in alloc_anon_folio Message-ID: <13ed2aa6607d510bd8dc6d602e96626511ee4fee.1780906288.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Convert alloc_anon_folio() to pass __GFP_ZERO instead of zeroing at the callsite. post_alloc_hook uses the fault address passed through vma_alloc_folio for cache-friendly zeroing. Note: before this series, replacing clear_user_highpage() with __GFP_ZERO was unsafe on cache-aliasing architectures because __GFP_ZERO uses clear_page() without a dcache flush. With this series, it is safe if the caller passes a valid user address (not USER_ADDR_NONE) to vma_alloc_folio() etc., which delivers it to post_alloc_hook() for the dcache flush via folio_zero_user(). It is only unsafe if USER_ADDR_NONE is passed. Note: with __GFP_ZERO, the folio is zeroed before mem_cgroup_charge(). If the charge fails, the zeroing work is wasted. Previously zeroing was done after a successful charge. This is inherent to moving zeroing into the allocator. Charge failures are rare (only at cgroup limits). Use folio_put_zeroed() on charge failure so the zeroed hint propagates to the buddy allocator, avoiding redundant re-zeroing on the next allocation attempt. Signed-off-by: Michael S. Tsirkin Reviewed-by: Gregory Price Assisted-by: Claude:claude-opus-4-6 --- mm/memory.c | 13 ++----------- 1 file changed, 2 insertions(+), 11 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 6c14b90f558e..6d6a3e1a02c1 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5265,25 +5265,16 @@ static struct folio *alloc_anon_folio(struct vm_fau= lt *vmf) goto fallback; =20 /* Try allocating the highest of the remaining orders. */ - gfp =3D vma_thp_gfp_mask(vma); + gfp =3D vma_thp_gfp_mask(vma) | __GFP_ZERO; while (orders) { folio =3D vma_alloc_folio(gfp, order, vma, vmf->address); if (folio) { if (mem_cgroup_charge(folio, vma->vm_mm, gfp)) { count_mthp_stat(order, MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE); - folio_put(folio); + folio_put_zeroed(folio); goto next; } folio_throttle_swaprate(folio, gfp); - /* - * When a folio is not zeroed during allocation - * (__GFP_ZERO not used) or user folios require special - * handling, folio_zero_user() is used to make sure - * that the page corresponding to the faulting address - * will be hot in the cache after zeroing. - */ - if (user_alloc_needs_zeroing()) - folio_zero_user(folio, vmf->address); return folio; } next: --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7A0583B8127 for ; Mon, 8 Jun 2026 08:39:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907965; cv=none; b=H2rU2VocKHOk9tJHGQYd2jItgjd6gwSQZFJD4MOa2TVnEGrqhpjWre2cAqrZqO+mmtByM/CN9u2P2t0fRBuqVi07uWOYO3V7ekzb4T7WHNUPjwcR/kCT4YiHqtlyusXQqWH0fySFDW/DAMcnZUs9zC4SR7Pn3uFUAvhXoUVdg8E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907965; c=relaxed/simple; bh=8lKGno9llqXVAoyhq8ZkwVVgxVUaSsktCgeYhaFtI3o=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=aOj+KmK/E2HNcCT+cdV+IgLJZIP8yj5YbwI7xwafLzyPiyCGa2i4zt4ZpSIozOitxtA4yIRHQDBQgJj9I2QVnQ2jTrwpl7+UZsd5ueT8nTXihGJLBKUxbGgxaOddDWXThSieVE1tsUCRUVbBplG4ChSul3w6ITq2numjq1YxE8g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=ANQAfEUL; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=tQn4TqjL; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ANQAfEUL"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="tQn4TqjL" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907961; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=sR3mqTwhRYGGG0tioR5exPzZ/dZTwNqp64a/18KVd0I=; b=ANQAfEULV8XmTV+vtAeOkdR6x4QT51Ts3l55wm9o7frJjduevHHK93WazvK8bTQO2YxnOr Vr+pRi35U8bcvcMKM8zcikFoEPkPjAxH7e7HLPlnvJlZWALrW3gR+g2K9zB8NNklr1TRNW YQix1BJAg6bCyS/glaIafxRBLEuWXRU= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-260-Wk_zWPdUP4GBJUBBVwnF7g-1; Mon, 08 Jun 2026 04:39:20 -0400 X-MC-Unique: Wk_zWPdUP4GBJUBBVwnF7g-1 X-Mimecast-MFC-AGG-ID: Wk_zWPdUP4GBJUBBVwnF7g_1780907959 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-45ef6417092so3350815f8f.2 for ; Mon, 08 Jun 2026 01:39:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907959; x=1781512759; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=sR3mqTwhRYGGG0tioR5exPzZ/dZTwNqp64a/18KVd0I=; b=tQn4TqjLLHepEPiPZlKEDeTXjy9bg6LqcLBLdoEqyzeVgje457BGiPXpV8RywnhWva hVCYL++d1IfikbN6Hkp09UAClQ2XYIGiCqf6vXUPmvdEa3JChjeS7vtAeift9pZfn4Rk dTf3w5MHYnze7QUa2KVwTcnpZh+CJY1AWieZOI5pxytvjkpPSr3ZXUhsS30ILInKpT59 npwt2WISfxleVlsxHM0V3p4yT2BH8LfFFwneJkblW0LrwTRP/KOCMcSlX82EUaAVl1ta JV5nBwza0igO14ihvRcpRqW5rfuMMyTOYKb2n/VsnTpzxYvYQ/Tq01H50cnl5qhSewvK o9TQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907959; x=1781512759; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sR3mqTwhRYGGG0tioR5exPzZ/dZTwNqp64a/18KVd0I=; b=pH+k1tHuTtoUUVIJwW+1ulup+cab4zKr0UBKbFzYZKF7HtqJJKspWhwGEJhS2wkWPC 5gy6j7UKp+avxlhY7v6RdGdhhgBXtlB9kU8l54J7LS8UkFKwTage1LVp1awr2L3gzsk0 +FuyUQCnuP2a8SgqysDMh7M1OUHYU4SwY6kLiPuQJjYSbVo2lYz9UDu4IIIQO2NAB0S3 Amdb2cHAB/Z1Jc0H84BTFURgF6+Z9YFJCEybzBHwSx0lOLdA23I9zvdvQiRPcESP18aj hDRWCiacAV/cHS/r/1BLnFkY06eNHjENoi7fnFA3Vn9bumHia63BH7DTCUSOisRIum1Z 0Keg== X-Gm-Message-State: AOJu0YxgML5CUTudQ1IJhuF/SvLJBf8IJZF4NW9YZFLTw+4DRQByDBvA V3ezjVFSe71Q5Cwu38UgCyYcmS786BtevOSGQaDAKpkulIQkmCwpzNRbWKP2x1fBkrP2Dtd1WPq jfA2/tTB5xECv8GSTvDG3j3p5l+e2hch+nIMgKgQyAY3zA5q8+ZtSlugnnXDFxgQMG2MCY+TODf FopKRfZm6hFKxZ7r4kz800xZqdDznc2pAOr7ST2FZ9OTQ= X-Gm-Gg: Acq92OGfHRoY/FQaOVK8Fm0Kn0JuhZMr2Og+AQPcGWUjMjM7r0BSZCi1cGnA9O/Etjj cyxpObpQ2gOxWEHjRsyhgp30mQHkWUJPtzNrWkcJVLeoPOVlWfod7bVyhRaPgSTJhyB15LxHbvz LbhnmOd2L2qf5y0s1NZYWCUWE4ULyFGxA7WUeR/nL6CF1v76pIy57jkc9tYokXWy/6w78TAip8V 81utYbBUqm2INxEB+E+Hbn5EgrMuhRuYCydYtBkV6AmJpUgBluioBoJx4kogDJXQqtFFAX6oxzD SemNh+1aRP8ZvSyg3ubYTtEviKjONRIflTQNPcZj2bTliDkvUfvWrhewDKUtYcZCwoXsR4i+E/+ 4HI62YNlXyd3BrucYvab0MoQ= X-Received: by 2002:a05:600c:4e94:b0:490:9588:bdae with SMTP id 5b1f17b1804b1-490c25e30e0mr244497825e9.18.1780907958679; Mon, 08 Jun 2026 01:39:18 -0700 (PDT) X-Received: by 2002:a05:600c:4e94:b0:490:9588:bdae with SMTP id 5b1f17b1804b1-490c25e30e0mr244496725e9.18.1780907958066; Mon, 08 Jun 2026 01:39:18 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-490bc3e59f5sm399310235e9.14.2026.06.08.01.39.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:39:17 -0700 (PDT) Date: Mon, 8 Jun 2026 04:39:10 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 26/37] mm: vma_alloc_anon_folio_pmd: pass raw fault address to vma_alloc_folio Message-ID: <94f64aeca62d8419d76f2cf82b36f1da6017a312.1780906288.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Drop the redundant HPAGE_PMD_MASK alignment at the callsite. NUMA interleave is not affected by the raw address; the ilx calculation shifts addr >> PAGE_SHIFT >> order, dropping sub-page bits regardless of alignment. post_alloc_hook will use the raw address for cache-friendly zeroing. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Reviewed-by: Gregory Price --- mm/huge_memory.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 970e077019b7..d689e6491ddb 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1337,7 +1337,7 @@ static struct folio *vma_alloc_anon_folio_pmd(struct = vm_area_struct *vma, const int order =3D HPAGE_PMD_ORDER; struct folio *folio; =20 - folio =3D vma_alloc_folio(gfp, order, vma, addr & HPAGE_PMD_MASK); + folio =3D vma_alloc_folio(gfp, order, vma, addr); =20 if (unlikely(!folio)) { count_vm_event(THP_FAULT_FALLBACK); --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 40D363BB670 for ; Mon, 8 Jun 2026 08:39:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907978; cv=none; b=UwWvfvnNu14uzJ27J9R08uwN2KYTWDzyyqnUM/3GcLApGjZjaFh96EqgAS2VGSL49bN6EfLtICRGHHxVFeBqevNk0DAGyi/gBFSERyQK97+/+QqsTgMXfU4uYJRWT71siwVyd/25Ng/tYQeRLsId1d5UXU35YErYxwvbGZ0p+xE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907978; c=relaxed/simple; bh=GbupeTEyC1nYpB3GHTrtS/PPh02DP7xvsmjmQMrjGAg=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=cuAi+/sjrOrk95+kVU3j/4O+sp1onHhrKm5aY+eM68Ftyejh2gXLuNy2l59ToMSBoYlcOJJ8mhE5ZjUJEpeSb0UL+61lkAryQvPSi2gpcAFv7hGqSNV7R6hES+9893kstgejXTPPknr1RVthBn2tcLBalbxTIuRGandTGgM0Chk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=TIJWC3VL; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=jvfAhMD/; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="TIJWC3VL"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="jvfAhMD/" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907974; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=jfQ8iRnJTK4n0bEkv49q5EiiPzQx8wAqaRkFNFNqcuU=; b=TIJWC3VLZM8qcwWG4C2KOilwTP6jXoR7a6LQLP6kEqmW+LcZFcVh1dda8vwJn1S8S29fpq 1shnULKrVcqpFTt39FyvlNXFESqycUYKvvGrvzFg6IlPijLtCoRNzGBuZBNQtKIn3Kp3qE 5JRolpvsxEg42NcjzG7h6yVQmNIMzg8= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-100-nd3Oioq6McChcbhjb8W0AQ-1; Mon, 08 Jun 2026 04:39:27 -0400 X-MC-Unique: nd3Oioq6McChcbhjb8W0AQ-1 X-Mimecast-MFC-AGG-ID: nd3Oioq6McChcbhjb8W0AQ_1780907966 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-45ef6ad76bfso1301664f8f.0 for ; Mon, 08 Jun 2026 01:39:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907966; x=1781512766; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=jfQ8iRnJTK4n0bEkv49q5EiiPzQx8wAqaRkFNFNqcuU=; b=jvfAhMD/zLuaiid/6DseBcrGBfPbzF4MUIacpIYarJnpP9GJkvHIPfIfNbWsERCGaq LsHFUes5tXboit2jchkeDHQc/lJCp6N56I8EU2PqppW9uks4A/9ThvA7LaXY4+a+CCN2 zJgr/MPBvFACbM4oHfdVLAmZx3gDXHbnQazKfOHeYxUsfYhp67j13vm6exzqnfzM95I4 4SZYo8tisrC/77kQwuVGZkq2uXpX+yrSq8lEVnr0tOkVDNyQvmlfLY+mg+QntnIfwcJB WvxthgmD+En/HZUkDtIZndUB5tQdN60o4X+zWhzvhNk5BG8DzHixYKi8/TtP5SpS6ikE fSqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907966; x=1781512766; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jfQ8iRnJTK4n0bEkv49q5EiiPzQx8wAqaRkFNFNqcuU=; b=XLp6Tie1nPcYPp24r0JUC0MJtZL/Vy0L7bPsofMrP/HQhjyI5H7wW65EoMBK8Us6eP z/lbG9VOFxMY/hV/IkfPTzslosvfUY9D4kzd7VBGHTICRdAiFLLSRyCT4xDUWUW9nfNX FfVAvBYmW/OFA46ENc32SXhuDFhlYA2mJ4i+Gm3YexcOG3KfaO3OkfMvZKUX2LiAX0gS hwjEg3vvOOJ5x1R61LbWGGDy9C17NO94jN9IIYeSSjVcpUaklokLxU2qMYP92/pa1/a5 euHaISGQDzZ8u+HJaWZyj77K12HJoC36sSETeZNUTYoay02XQrtIvo0gsmhK7z/ZC0AM R53w== X-Gm-Message-State: AOJu0Yyz5BDkKjkielKWqgbIjP21/IEgAgBG1PvP2NUZxl3/rW0pkNob HBDpGExEyykRCtSbtFykxDhu8nZ9oQvD34/wGlML0M7OuUbb4UvJNaPZatKjoLOGnwpP2KZp7xr J15YNwm94e1sTL/g8oMsnEHUtY0aapxiJkm2Dj9IFYmYTHhS4ayqnMGCCw11ICcXueQSl+e3ijI zV95GhLbhuGoE/0iisf+GEsWbadWKPCl4zKi7SvotxvoI= X-Gm-Gg: Acq92OET4EU3UmFzIFWBb9PWKhwXoeVF6Ck0tJ0tEdCtGYme/8uUmIADFYMfw8qhQOd 2S97Aam3iEQt0i5+uAs3Cc7LFuHCfYO4ym4F5lV9Hm+uz9glRSRIOj5WQwdJ4QdwSfEIAfhpFQl BYKqwXqxnS6PtqTIC7jm22ibRMqakd6W6jPvac8E/w4m2uVUoTSJKMPWyzBWpkryKUTWxc52uRY Ct4HUWpemDuT+qs6uw4zIwQMaV4gH1Ga65NCu72BcZ5zxEHaI7Lqrm5WMcGgwrL7C1nyFtjT4Dx +/GInU6O1WhYgPYgrW33r8mctnzPZ1mwMK8ImWtgNhVn0VUZS/3S+2CoDSi117NvDnc8relr8wt 3SbU1zQ5mvJABh8kZMT+DqH8= X-Received: by 2002:a05:600c:8b70:b0:490:ae94:a7e6 with SMTP id 5b1f17b1804b1-490c25c20e0mr225777375e9.24.1780907966289; Mon, 08 Jun 2026 01:39:26 -0700 (PDT) X-Received: by 2002:a05:600c:8b70:b0:490:ae94:a7e6 with SMTP id 5b1f17b1804b1-490c25c20e0mr225776435e9.24.1780907965793; Mon, 08 Jun 2026 01:39:25 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-490c2d30eeasm239643575e9.1.2026.06.08.01.39.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:39:25 -0700 (PDT) Date: Mon, 8 Jun 2026 04:39:18 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 27/37] mm: use __GFP_ZERO in vma_alloc_anon_folio_pmd Message-ID: <2ee827fed4765a155d2b56bb0e13d7ee5fc6dce8.1780906288.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Convert vma_alloc_anon_folio_pmd() to pass __GFP_ZERO instead of zeroing at the callsite. post_alloc_hook uses the fault address passed through vma_alloc_folio for cache-friendly zeroing. Note: before this series, replacing folio_zero_user() with __GFP_ZERO was unsafe on cache-aliasing architectures because __GFP_ZERO uses clear_page() without a dcache flush. With this series, it is safe if the caller passes a valid user address (not USER_ADDR_NONE) to vma_alloc_folio() etc., which delivers it to post_alloc_hook() for the dcache flush via folio_zero_user(). It is only unsafe if USER_ADDR_NONE is passed. Note: with __GFP_ZERO, the folio is zeroed before mem_cgroup_charge(). If the charge fails, the zeroing work is wasted. Previously zeroing was done after a successful charge. This is inherent to moving zeroing into the allocator. Charge failures are rare (only at cgroup limits). Use folio_put_zeroed() on charge failure so the zeroed hint propagates to the buddy allocator, avoiding redundant re-zeroing on the next allocation attempt. Signed-off-by: Michael S. Tsirkin Reviewed-by: Gregory Price Assisted-by: Claude:claude-opus-4-6 --- mm/huge_memory.c | 14 +++----------- 1 file changed, 3 insertions(+), 11 deletions(-) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index d689e6491ddb..0dec3c717ff2 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1333,7 +1333,7 @@ EXPORT_SYMBOL_GPL(thp_get_unmapped_area); static struct folio *vma_alloc_anon_folio_pmd(struct vm_area_struct *vma, unsigned long addr) { - gfp_t gfp =3D vma_thp_gfp_mask(vma); + gfp_t gfp =3D vma_thp_gfp_mask(vma) | __GFP_ZERO; const int order =3D HPAGE_PMD_ORDER; struct folio *folio; =20 @@ -1347,7 +1347,7 @@ static struct folio *vma_alloc_anon_folio_pmd(struct = vm_area_struct *vma, =20 VM_BUG_ON_FOLIO(!folio_test_large(folio), folio); if (mem_cgroup_charge(folio, vma->vm_mm, gfp)) { - folio_put(folio); + folio_put_zeroed(folio); count_vm_event(THP_FAULT_FALLBACK); count_vm_event(THP_FAULT_FALLBACK_CHARGE); count_mthp_stat(order, MTHP_STAT_ANON_FAULT_FALLBACK); @@ -1356,17 +1356,9 @@ static struct folio *vma_alloc_anon_folio_pmd(struct= vm_area_struct *vma, } folio_throttle_swaprate(folio, gfp); =20 - /* - * When a folio is not zeroed during allocation (__GFP_ZERO not used) - * or user folios require special handling, folio_zero_user() is used to - * make sure that the page corresponding to the faulting address will be - * hot in the cache after zeroing. - */ - if (user_alloc_needs_zeroing()) - folio_zero_user(folio, addr); /* * The memory barrier inside __folio_mark_uptodate makes sure that - * folio_zero_user writes become visible before the set_pmd_at() + * page zeroing becomes visible before the set_pmd_at() * write. */ __folio_mark_uptodate(folio); --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 04A1A3B3C13 for ; Mon, 8 Jun 2026 08:39:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907989; cv=none; b=Xuh+AJdukkd+WLSdK2HAlRp3wOZ2b6JWWThm8wgGn83QtSFO6qPnXcfIz0KMuMzp7PtCLEeqB4BIEyO7KbHMtMDzGsK1zREvKzq5qQFC7EdCk9o5BjJxmvJnV36oaotHE4vrXp7m/G2oinnlmeetQuBN/IGlC3RCwlDFyxzE1XY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907989; c=relaxed/simple; bh=CN8i6xYIZIGBMi7wcy0sE5TKRzHdryaGnwcrknomwE4=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=PVPTWhzq/MK+OAop+BL+u0GoeeWgdzx9H/NJOHKDnb0aQ/O6z5BEdJ3cEi2l5ruKrQ9Y6axcK2HT35Hfv1WGHNXAs3TvOOOSw5qAu0Xdcc2838tKpkU/VCko12a/j31q55QBVOAj68xvJsXT/TIFVnF45sdlBq9vut1foKWtoLM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=UzO25CmA; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=OkD85Dgi; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="UzO25CmA"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="OkD85Dgi" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907980; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=AjTJBqpyDofbRU0ObNPGnhPpa7CYDWMZBguCdtv7thM=; b=UzO25CmAEwlqIFruSOMKUlRbQPbB3Bj3mPxjpOFurGYjJln+Vlf83eSIfQPLXmTl6phxYA 39Ca3ry40c2BONTL10NmN5pQNdtuGG6e4r4TN8o8DiOojXKWzDNfpZJMGSFshTw7tLN2HA 4pOxI6QFbnmprtmU4iG8zrYD8qNime8= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-235-lw1Vs2WQMaqINHVHCaerqg-1; Mon, 08 Jun 2026 04:39:39 -0400 X-MC-Unique: lw1Vs2WQMaqINHVHCaerqg-1 X-Mimecast-MFC-AGG-ID: lw1Vs2WQMaqINHVHCaerqg_1780907978 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-490b2f22ea2so42659195e9.1 for ; Mon, 08 Jun 2026 01:39:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907978; x=1781512778; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=AjTJBqpyDofbRU0ObNPGnhPpa7CYDWMZBguCdtv7thM=; b=OkD85DgiCzlZZE0pZVIa7Ng8zfibrIvyxQbPKMt7GJapLmOONznpAH6Rtdz0MpoxLR IpVbgPQlkHAW+k6WtUwX2gQHrV4vyd54fRNaAprQYcCLgDkqI5ehZdotLP3eT26+jp0h yfUw1zLDRMqygUHwsPdw7kTP1UbujU7H9BSTNXOctoOa+emo7QV2XRxHHUxYK0r+1qwW nSjMPAAkBspYPTmW3vTv9H4cwXq4hwAoXkl7M9kyRaLRFVGMpltg1ntX7UuaZaCWJShE ijJTNN+Wf8eCMp7tRWBd0HT4gMXuGgJohMbgZtVMaF5FDoiI+l7+B78/IHjSi6tk9BJT kshQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907978; x=1781512778; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AjTJBqpyDofbRU0ObNPGnhPpa7CYDWMZBguCdtv7thM=; b=p75S1LzGC0UExZE4KjEhBt5bEV0sRnkODLOEDrj5qwDaaIBsRRz8A2eNzzbYVkOysZ UI9Fz6SVFzY6e81xGeczCOGVGEVZPpnYTsnw3oJKEZVnXxrnfXpKDAZ/YhQNnRZkW+Dt dH9XoBuABIH+EQC1FSBoTnWo1qG9CeQFcYqp3bJRDnxmpVgzL8H9UxGMSEnhpqQn74Kc x6SwSsXwtZ3YRUPGTMRwABU4Ev1Aj1vx5mloPXylVFKVswTZ8VfKszlN/m4ELzSTjYlF kagGzqV9WHwE0vVXZ8yGD2EJIQOvRuj5lGUYdxBPKiwn/8bEvrQ2sAhDVz4AgweEqilp c+eQ== X-Gm-Message-State: AOJu0YyN1RSY4PDLKGzm6a334BKXP+brqdlliBueuOkQINOnOra9QvnA Alj/RDgIpZpGUlmROxYJh5Ee5c+Qb2BJarcbDZ8DuP45JIdLcEMN0ZvWlSbiKB3b1PJRnAfElyv orwPC0Yu9a+SBcaWSUVdFQq0MBpbJ7NjTf756TQ2Pa8iu5suY42pHKIp16lqhEiu7YiDSemrIka sVrK9wM71yZA1xQMYnQquGRNPbrE1jEgIUOvSazh0gfmg= X-Gm-Gg: Acq92OHg3WDrZumYi2uHpCF1FTXd6chGV4JJ1wuxSD442YCRm8xEz1SjLTMf4M1y4Fm isK+U6KVoBs86FUjEcxWnnNbsQfd5b3VsLqr5nRRUrfD/TkLPIbMSMPJY3SqH0AgFbnYe9izWka M1t7NbjbOBber9WE2++XNYOus87VcfdFgRd9zfUBD/SV1GIdC6K5pjPpzSBm/p0tKkMrkXjroNo etnrrgodr7W3vlc/DxoZWItG2u9UClkQqxr9tVw5koP8zM2Fcz+vuqbVRGn654LwHL03PzeO/6T WBDSks/ouLhrOZ97xvJp1Il+RB5hV7d14nLekFGn/fEd7J5opRjx/8fdgmiCgo7QYyU6Thn0ZkS 5AYhTezgWC4T09oxRF69N92E= X-Received: by 2002:a05:600c:1d2a:b0:490:bbc1:d508 with SMTP id 5b1f17b1804b1-490c258f3femr244580505e9.1.1780907977738; Mon, 08 Jun 2026 01:39:37 -0700 (PDT) X-Received: by 2002:a05:600c:1d2a:b0:490:bbc1:d508 with SMTP id 5b1f17b1804b1-490c258f3femr244579295e9.1.1780907976956; Mon, 08 Jun 2026 01:39:36 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-490bc3fd502sm376493895e9.11.2026.06.08.01.39.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:39:36 -0700 (PDT) Date: Mon, 8 Jun 2026 04:39:26 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 28/37] mm: hugetlb: add gfp parameter and skip zeroing for zeroed pages Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a gfp_t parameter to alloc_hugetlb_folio(). When __GFP_ZERO is set, the function guarantees the returned folio is zeroed: - Fresh allocations (buddy or gigantic): zeroed by post_alloc_hook via __GFP_ZERO, HPG_zeroed set by alloc_surplus_hugetlb_folio. - Pool pages with HPG_zeroed set: already zeroed, skip. - Pool pages without HPG_zeroed: zeroed via folio_zero_user(). The address parameter is renamed to user_addr; the function aligns it internally for reservation and NUMA policy lookups. For pages that need zeroing, user_addr is passed to folio_zero_user() for cache-friendly zeroing near the faulting subpage. All callers pass a page-aligned address; the hugetlb_no_page caller passes vmf->real_address & PAGE_MASK for consistency. HPG_zeroed (stored in hugetlb folio->private bits) tracks known-zero pool pages. It is set when alloc_surplus_hugetlb_folio allocates with __GFP_ZERO, and cleared in free_huge_folio when the page returns to the pool after userspace use. Note: for gigantic CMA pages, __GFP_ZERO is passed through to cma_alloc_frozen_compound() via its caller_gfp parameter, so the pages ARE zeroed by the allocator. HPG_zeroed is only set when __GFP_ZERO was in the original gfp_mask. Pool pages allocated without __GFP_ZERO (e.g. by alloc_pool_huge_folio) do not get HPG_zeroed; they are zeroed later by folio_zero_user() at fault time. Note: with __GFP_ZERO, the folio is zeroed before mem_cgroup_charge_hugetlb(). If the charge fails, the zeroed folio is freed back. Before this patch it is zeroed after charge, so simply freeing after zeroing would be a regression. Thread a zeroed hint through free_huge_folio so surplus pages freed back to buddy preserve the zeroed state via free_frozen_pages_zeroed, avoiding redundant re-zeroing on the next allocation. Suggested-by: Gregory Price Reviewed-by: Gregory Price Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- fs/hugetlbfs/inode.c | 3 +- include/linux/hugetlb.h | 5 ++- mm/hugetlb.c | 78 +++++++++++++++++++++++++++-------------- 3 files changed, 57 insertions(+), 29 deletions(-) diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c index 78d61bf2bd9b..2c0c51fe9ec3 100644 --- a/fs/hugetlbfs/inode.c +++ b/fs/hugetlbfs/inode.c @@ -790,13 +790,12 @@ static long hugetlbfs_fallocate(struct file *file, in= t mode, loff_t offset, * folios in these areas, we need to consume the reserves * to keep reservation accounting consistent. */ - folio =3D alloc_hugetlb_folio(&pseudo_vma, addr, false); + folio =3D alloc_hugetlb_folio(&pseudo_vma, addr, false, __GFP_ZERO); if (IS_ERR(folio)) { mutex_unlock(&hugetlb_fault_mutex_table[hash]); error =3D PTR_ERR(folio); goto out; } - folio_zero_user(folio, addr); __folio_mark_uptodate(folio); error =3D hugetlb_add_to_page_cache(folio, mapping, index); if (unlikely(error)) { diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 1f7ae6609e51..06d033a57a61 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -593,6 +593,7 @@ enum hugetlb_page_flags { HPG_vmemmap_optimized, HPG_raw_hwp_unreliable, HPG_cma, + HPG_zeroed, __NR_HPAGEFLAGS, }; =20 @@ -653,6 +654,7 @@ HPAGEFLAG(Freed, freed) HPAGEFLAG(VmemmapOptimized, vmemmap_optimized) HPAGEFLAG(RawHwpUnreliable, raw_hwp_unreliable) HPAGEFLAG(Cma, cma) +HPAGEFLAG(Zeroed, zeroed) =20 #ifdef CONFIG_HUGETLB_PAGE =20 @@ -700,7 +702,8 @@ int isolate_or_dissolve_huge_folio(struct folio *folio,= struct list_head *list); int replace_free_hugepage_folios(unsigned long start_pfn, unsigned long en= d_pfn); void wait_for_freed_hugetlb_folios(void); struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, - unsigned long addr, bool cow_from_owner); + unsigned long user_addr, bool cow_from_owner, + gfp_t gfp); struct folio *alloc_hugetlb_folio_nodemask(struct hstate *h, int preferred= _nid, nodemask_t *nmask, gfp_t gfp_mask, bool allow_alloc_fallback); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 5d7e546565f5..ed00db703911 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1455,7 +1455,8 @@ void add_hugetlb_folio(struct hstate *h, struct folio= *folio, } =20 static void __update_and_free_hugetlb_folio(struct hstate *h, - struct folio *folio) + struct folio *folio, + bool zeroed) { bool clear_flag =3D folio_test_hugetlb_vmemmap_optimized(folio); =20 @@ -1506,6 +1507,8 @@ static void __update_and_free_hugetlb_folio(struct hs= tate *h, VM_BUG_ON_FOLIO(folio_ref_count(folio), folio); if (folio_test_hugetlb_cma(folio)) hugetlb_cma_free_frozen_folio(folio); + else if (zeroed) + free_frozen_pages_zeroed(&folio->page, folio_order(folio)); else free_frozen_pages(&folio->page, folio_order(folio)); } @@ -1545,7 +1548,7 @@ static void free_hpage_workfn(struct work_struct *wor= k) */ h =3D size_to_hstate(folio_size(folio)); =20 - __update_and_free_hugetlb_folio(h, folio); + __update_and_free_hugetlb_folio(h, folio, false); =20 cond_resched(); } @@ -1559,10 +1562,10 @@ static inline void flush_free_hpage_work(struct hst= ate *h) } =20 static void update_and_free_hugetlb_folio(struct hstate *h, struct folio *= folio, - bool atomic) + bool atomic, bool zeroed) { if (!folio_test_hugetlb_vmemmap_optimized(folio) || !atomic) { - __update_and_free_hugetlb_folio(h, folio); + __update_and_free_hugetlb_folio(h, folio, zeroed); return; } =20 @@ -1596,7 +1599,7 @@ static void bulk_vmemmap_restore_error(struct hstate = *h, spin_lock_irq(&hugetlb_lock); __folio_clear_hugetlb(folio); spin_unlock_irq(&hugetlb_lock); - update_and_free_hugetlb_folio(h, folio, false); + update_and_free_hugetlb_folio(h, folio, false, false); cond_resched(); } } else { @@ -1621,7 +1624,7 @@ static void bulk_vmemmap_restore_error(struct hstate = *h, spin_lock_irq(&hugetlb_lock); __folio_clear_hugetlb(folio); spin_unlock_irq(&hugetlb_lock); - update_and_free_hugetlb_folio(h, folio, false); + update_and_free_hugetlb_folio(h, folio, false, false); cond_resched(); break; } @@ -1664,7 +1667,7 @@ static void update_and_free_pages_bulk(struct hstate = *h, } =20 list_for_each_entry_safe(folio, t_folio, &non_hvo_folios, lru) { - update_and_free_hugetlb_folio(h, folio, false); + update_and_free_hugetlb_folio(h, folio, false, false); cond_resched(); } } @@ -1680,7 +1683,7 @@ struct hstate *size_to_hstate(unsigned long size) return NULL; } =20 -void free_huge_folio(struct folio *folio) +static void __free_huge_folio(struct folio *folio, bool zeroed) { /* * Can't pass hstate in here because it is called from the @@ -1692,6 +1695,9 @@ void free_huge_folio(struct folio *folio) bool restore_reserve; unsigned long flags; =20 + /* Page was mapped to userspace; no longer known-zero */ + folio_clear_hugetlb_zeroed(folio); + VM_BUG_ON_FOLIO(folio_ref_count(folio), folio); VM_BUG_ON_FOLIO(folio_mapcount(folio), folio); =20 @@ -1735,12 +1741,12 @@ void free_huge_folio(struct folio *folio) if (folio_test_hugetlb_temporary(folio)) { remove_hugetlb_folio(h, folio, false); spin_unlock_irqrestore(&hugetlb_lock, flags); - update_and_free_hugetlb_folio(h, folio, true); + update_and_free_hugetlb_folio(h, folio, true, zeroed); } else if (h->surplus_huge_pages_node[nid]) { /* remove the page from active list */ remove_hugetlb_folio(h, folio, true); spin_unlock_irqrestore(&hugetlb_lock, flags); - update_and_free_hugetlb_folio(h, folio, true); + update_and_free_hugetlb_folio(h, folio, true, zeroed); } else { arch_clear_hugetlb_flags(folio); enqueue_hugetlb_folio(h, folio); @@ -1748,6 +1754,11 @@ void free_huge_folio(struct folio *folio) } } =20 +void free_huge_folio(struct folio *folio) +{ + __free_huge_folio(folio, false); +} + /* * Must be called with the hugetlb lock held */ @@ -2031,7 +2042,7 @@ int dissolve_free_hugetlb_folio(struct folio *folio) rc =3D 0; } =20 - update_and_free_hugetlb_folio(h, folio, false); + update_and_free_hugetlb_folio(h, folio, false, false); return rc; } out: @@ -2093,6 +2104,10 @@ static struct folio *alloc_surplus_hugetlb_folio(str= uct hstate *h, if (!folio) return NULL; =20 + /* Mark as known-zero only if __GFP_ZERO was requested */ + if (gfp_mask & __GFP_ZERO) + folio_set_hugetlb_zeroed(folio); + spin_lock_irq(&hugetlb_lock); /* * nr_huge_pages needs to be adjusted within the same lock cycle @@ -2156,11 +2171,11 @@ static struct folio *alloc_migrate_hugetlb_folio(st= ruct hstate *h, gfp_t gfp_mas */ static struct folio *alloc_buddy_hugetlb_folio_with_mpol(struct hstate *h, - struct vm_area_struct *vma, unsigned long addr) + struct vm_area_struct *vma, unsigned long addr, gfp_t gfp) { struct folio *folio =3D NULL; struct mempolicy *mpol; - gfp_t gfp_mask =3D htlb_alloc_mask(h); + gfp_t gfp_mask =3D htlb_alloc_mask(h) | gfp; int nid; nodemask_t *nodemask; =20 @@ -2715,7 +2730,7 @@ static int alloc_and_dissolve_hugetlb_folio(struct fo= lio *old_folio, * Folio has been replaced, we can safely free the old one. */ spin_unlock_irq(&hugetlb_lock); - update_and_free_hugetlb_folio(h, old_folio, false); + update_and_free_hugetlb_folio(h, old_folio, false, false); } =20 return ret; @@ -2723,7 +2738,7 @@ static int alloc_and_dissolve_hugetlb_folio(struct fo= lio *old_folio, free_new: spin_unlock_irq(&hugetlb_lock); if (new_folio) - update_and_free_hugetlb_folio(h, new_folio, false); + update_and_free_hugetlb_folio(h, new_folio, false, false); =20 return ret; } @@ -2857,16 +2872,19 @@ typedef enum { * When it's set, the allocation will bypass all vma level reservations. */ struct folio *alloc_hugetlb_folio(struct vm_area_struct *vma, - unsigned long addr, bool cow_from_owner) + unsigned long user_addr, bool cow_from_owner, + gfp_t gfp) { struct hugepage_subpool *spool =3D subpool_vma(vma); struct hstate *h =3D hstate_vma(vma); + unsigned long addr =3D user_addr & huge_page_mask(h); struct folio *folio; long retval, gbl_chg, gbl_reserve; map_chg_state map_chg; int ret, idx; struct hugetlb_cgroup *h_cg =3D NULL; - gfp_t gfp =3D htlb_alloc_mask(h) | __GFP_RETRY_MAYFAIL; + + gfp |=3D htlb_alloc_mask(h) | __GFP_RETRY_MAYFAIL; =20 idx =3D hstate_index(h); =20 @@ -2934,13 +2952,12 @@ struct folio *alloc_hugetlb_folio(struct vm_area_st= ruct *vma, folio =3D dequeue_hugetlb_folio_vma(h, vma, addr, gbl_chg); if (!folio) { spin_unlock_irq(&hugetlb_lock); - folio =3D alloc_buddy_hugetlb_folio_with_mpol(h, vma, addr); + folio =3D alloc_buddy_hugetlb_folio_with_mpol(h, vma, user_addr, gfp); if (!folio) goto out_uncharge_cgroup; spin_lock_irq(&hugetlb_lock); list_add(&folio->lru, &h->hugepage_activelist); folio_ref_unfreeze(folio, 1); - /* Fall through */ } =20 /* @@ -2963,6 +2980,10 @@ struct folio *alloc_hugetlb_folio(struct vm_area_str= uct *vma, =20 spin_unlock_irq(&hugetlb_lock); =20 + if ((gfp & __GFP_ZERO) && !folio_test_hugetlb_zeroed(folio)) + folio_zero_user(folio, user_addr); + folio_clear_hugetlb_zeroed(folio); + hugetlb_set_folio_subpool(folio, spool); =20 if (map_chg !=3D MAP_CHG_ENFORCED) { @@ -2999,7 +3020,7 @@ struct folio *alloc_hugetlb_folio(struct vm_area_stru= ct *vma, lruvec_stat_mod_folio(folio, NR_HUGETLB, pages_per_huge_page(h)); =20 if (ret =3D=3D -ENOMEM) { - free_huge_folio(folio); + __free_huge_folio(folio, !!(gfp & __GFP_ZERO)); return ERR_PTR(-ENOMEM); } =20 @@ -4971,7 +4992,7 @@ int copy_hugetlb_page_range(struct mm_struct *dst, st= ruct mm_struct *src, spin_unlock(src_ptl); spin_unlock(dst_ptl); /* Do not use reserve as it's private owned */ - new_folio =3D alloc_hugetlb_folio(dst_vma, addr, false); + new_folio =3D alloc_hugetlb_folio(dst_vma, addr, false, 0); if (IS_ERR(new_folio)) { folio_put(pte_folio); ret =3D PTR_ERR(new_folio); @@ -5500,7 +5521,7 @@ static vm_fault_t hugetlb_wp(struct vm_fault *vmf) * be acquired again before returning to the caller, as expected. */ spin_unlock(vmf->ptl); - new_folio =3D alloc_hugetlb_folio(vma, vmf->address, cow_from_owner); + new_folio =3D alloc_hugetlb_folio(vma, vmf->address, cow_from_owner, 0); =20 if (IS_ERR(new_folio)) { /* @@ -5760,7 +5781,13 @@ static vm_fault_t hugetlb_no_page(struct address_spa= ce *mapping, goto out; } =20 - folio =3D alloc_hugetlb_folio(vma, vmf->address, false); + /* + * Passing vmf->real_address would work just as well, + * but PAGE_MASK helps make sure we never pass + * USER_ADDR_NONE by mistake. + */ + folio =3D alloc_hugetlb_folio(vma, vmf->real_address & PAGE_MASK, + false, __GFP_ZERO); if (IS_ERR(folio)) { /* * Returning error will result in faulting task being @@ -5780,7 +5807,6 @@ static vm_fault_t hugetlb_no_page(struct address_spac= e *mapping, ret =3D 0; goto out; } - folio_zero_user(folio, vmf->real_address); __folio_mark_uptodate(folio); new_folio =3D true; =20 @@ -6219,7 +6245,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, goto out; } =20 - folio =3D alloc_hugetlb_folio(dst_vma, dst_addr, false); + folio =3D alloc_hugetlb_folio(dst_vma, dst_addr, false, 0); if (IS_ERR(folio)) { pte_t *actual_pte =3D hugetlb_walk(dst_vma, dst_addr, PMD_SIZE); if (actual_pte) { @@ -6266,7 +6292,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, goto out; } =20 - folio =3D alloc_hugetlb_folio(dst_vma, dst_addr, false); + folio =3D alloc_hugetlb_folio(dst_vma, dst_addr, false, 0); if (IS_ERR(folio)) { folio_put(*foliop); ret =3D -ENOMEM; --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C18AE3BFAFA for ; Mon, 8 Jun 2026 08:39:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907998; cv=none; b=XsfL2ZfsLcBAO+wagVj6ZJ0fCxrNkYCKIMuA4WIif7Y8iRHVM7mxCEZPrtSTPJDd3izLekw6rzLv6garApWcGJa0rL5x2nlM2iBEBQ3PSMv8irJWB6h63frLBrMgplsyqJ4HlNWhKkTxh5A2bQSWAq7JcBdn2obRpCIB3Cv++tY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780907998; c=relaxed/simple; bh=uD9B0Z2r8dApvmwrr4a0YLqj2hTMaiDgCysJxcZ5WsU=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=KUl6tLIYY/IaZsvWd4zGyHlVcNkOiGKG7d6pAgGzlo9eMdy6kKty4+dGo434v0dkwpMUHsBivHYuiycrvSF0OP2n6ygpOWg8R/z41/mX0omPLEq85gsMe5aWWy2a3GopkyPNkAALzZRdsC/QEqNIXt3obdtcZnzWfIGZC+lN2qQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=djOaZnLL; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=qE0sxdoT; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="djOaZnLL"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="qE0sxdoT" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780907992; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=ScAoSfQrcD4R0Yhb9nAUBLjdu8qZL5GRGpRfEb8F90Q=; b=djOaZnLL68tiyDAA1RNRRLjf7i6w9D0+hyMhUbXNybuejkcs8cciJYlviaGWH3tnp2vxRn PmJ/bUGptBD9ZicZ3KwgOMLSQrQcjjalYs5EsgtDEYO/Arr9yjdgckpgYjuQMK8/YGwAwD tifQXUyF9w0v1taJAmkDO0kwnI6KZgo= Received: from mail-wm1-f72.google.com (mail-wm1-f72.google.com [209.85.128.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-410-9U33sh7WNliG3xWSe2iFCg-1; Mon, 08 Jun 2026 04:39:51 -0400 X-MC-Unique: 9U33sh7WNliG3xWSe2iFCg-1 X-Mimecast-MFC-AGG-ID: 9U33sh7WNliG3xWSe2iFCg_1780907990 Received: by mail-wm1-f72.google.com with SMTP id 5b1f17b1804b1-490b7959b9fso41826745e9.2 for ; Mon, 08 Jun 2026 01:39:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907990; x=1781512790; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=ScAoSfQrcD4R0Yhb9nAUBLjdu8qZL5GRGpRfEb8F90Q=; b=qE0sxdoT0LlQcezUjcJpe+ChEv1ZJ6WoapcZmuE+WqwwcOMjo59KdyWpyFObwF/Lfi Dim2mBGNOo0zC/shxQ07v0QeKDyBLts/ZlGTrBHhsIfEdWtCNd/sOsLNLTz+IZ3747aM Bjsau0nt/HWCPDJDgNVSXwQwY9fkeMENTY+7+4Q56AzQleEguJKUm1GneO3lfynX1oIX JWyatfKQGwp4hGE46fFiRPo2enog5VbAp5LQOOzxihUGa8dd8MnLk+N3plwGWkp84MAR U9bxqI1+2HLMQiqh1zHA4XyBqzw7GFCi5KWmODsbVq0f8tW2wsz1AOomtV4rYAFgxMAu /xLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907990; x=1781512790; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ScAoSfQrcD4R0Yhb9nAUBLjdu8qZL5GRGpRfEb8F90Q=; b=Yp8ffYY+peKLxNA120H4RcWzh7pEbKqpemRDiO381HsMtsf1FIluzzg2vPQ2/0ml3B honfBabA9b8CMzBszZTsWuypN5pbUFjCZ22EfetxcOM63JI9gc2cBahVY4Sw78/IBP1K pyFmCsQ1L5YO3NT7HNnQ/vkZdFmNOs/nWiwIEZEApNd8u2s98GPMrVqpbyWoercjsnil VYxAnXn7AEVkF0m1XJB91bLcmbuCXGX06L87/GGE7DZ/Z0b1Cvw9kXIaet3HzeJ3VbuO D0w2XpzMYohipMPaS0FhWODPmYMxdBVXRfNi5Hlg+tXsVPTzBnyZNcYUuTSKxPvrNX8W wDaw== X-Gm-Message-State: AOJu0YxsPOXmFtvUswi4kyPQM4ay7qzLmPcOPAF4hsQ4SxnMHdI4hUNT 6jQ0e2pjc/SCSL6hA1BBPINewF8PR3FgpSciz2YEkedMgll4PWYAUDvZ3NdJbUJTbL/Lvpudlnx XOYJeI7Mv2BZw1fcCDMtBxszP2Wux8sSuMjVvUnlwRpCV1kuunuVE26SVoHMXrFw5ampIMgh3f2 yWV6dY1dERtFii2m28D7OjgS2/6vkjvvs+bcgebd8i2xk= X-Gm-Gg: Acq92OGAhgkjZxIX1YXEnQ87sIfzqi8/WBfKxR5GxVoHJMTSVqrMTcC0FccP8DFR1Pm 8+SpJdsFGkjH6erUXA27Zv6W4q5mNjcDoggdmBAqhiS46tZEIg916cGRLk10jLO7AvJAhYoCUa6 LL74Y0TQg+YdzkEvTJ3qffbx83omV31Q2WhjW2s4eTmH/kxF87EyhiT4T69JqjPFrl9JWszbiTd sU0/ify30nXl9AaluBHm6iC2BIF5QrQOCael/slE0Y9SnQZnIsR4mZkEQo1LB1dXeStcjzRWU+6 91Ju4FsKQv4AItUAs7cbkZPGaSgCq8ZdK/mZ3GyreYjUJ2SJuFnEKqo2QaLUwSL/8dC9DoA0kol CFYScgBFYW9oQf/6sxA6s1NM= X-Received: by 2002:a05:600c:3e15:b0:490:adb6:793d with SMTP id 5b1f17b1804b1-490c25f67d9mr239705435e9.26.1780907989635; Mon, 08 Jun 2026 01:39:49 -0700 (PDT) X-Received: by 2002:a05:600c:3e15:b0:490:adb6:793d with SMTP id 5b1f17b1804b1-490c25f67d9mr239704145e9.26.1780907988974; Mon, 08 Jun 2026 01:39:48 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-490bc3d66c8sm356783495e9.10.2026.06.08.01.39.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:39:48 -0700 (PDT) Date: Mon, 8 Jun 2026 04:39:37 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 29/37] mm: memfd: skip zeroing for zeroed hugetlb pool pages Message-ID: <0d6c2d31f48ff454223ad4f1d37ef7b73263bf5d.1780906288.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add bool *zeroed output to alloc_hugetlb_folio_reserve() so callers can check whether the pool page is known-zero. memfd's memfd_alloc_folio() uses this to skip the explicit folio_zero_user() when the page is already zero. This avoids redundant zeroing for memfd hugetlb pages that were pre-allocated into the pool and never mapped to userspace. Note: HPG_zeroed is currently only set for surplus pages allocated with __GFP_ZERO (via alloc_surplus_hugetlb_folio), not for pool pages from alloc_pool_huge_folio. So the zeroed output from alloc_hugetlb_folio_reserve is typically false for pool-only reservations. It becomes true when surplus pages fill the reservation. The addr_hint 0 passed to folio_zero_user is acceptable for memfd: these pages are not mapped yet and will get proper dcache handling at mmap time via the page fault path. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- include/linux/cma.h | 3 ++- include/linux/hugetlb.h | 6 ++++-- mm/cma.c | 6 ++++-- mm/hugetlb.c | 11 +++++++++-- mm/hugetlb_cma.c | 4 ++-- mm/memfd.c | 14 ++++++++------ 6 files changed, 29 insertions(+), 15 deletions(-) diff --git a/include/linux/cma.h b/include/linux/cma.h index 8555d38a97b1..dee88909cf5d 100644 --- a/include/linux/cma.h +++ b/include/linux/cma.h @@ -53,7 +53,8 @@ extern bool cma_release(struct cma *cma, const struct pag= e *pages, unsigned long =20 struct page *cma_alloc_frozen(struct cma *cma, unsigned long count, unsigned int align, bool no_warn); -struct page *cma_alloc_frozen_compound(struct cma *cma, unsigned int order= ); +struct page *cma_alloc_frozen_compound(struct cma *cma, unsigned int order, + gfp_t caller_gfp); bool cma_release_frozen(struct cma *cma, const struct page *pages, unsigned long count); =20 diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 06d033a57a61..7eb529eabe99 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -708,7 +708,8 @@ struct folio *alloc_hugetlb_folio_nodemask(struct hstat= e *h, int preferred_nid, nodemask_t *nmask, gfp_t gfp_mask, bool allow_alloc_fallback); struct folio *alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_= nid, - nodemask_t *nmask, gfp_t gfp_mask); + nodemask_t *nmask, gfp_t gfp_mask, + bool *zeroed); =20 int hugetlb_add_to_page_cache(struct folio *folio, struct address_space *m= apping, pgoff_t idx); @@ -1128,7 +1129,8 @@ static inline void wait_for_freed_hugetlb_folios(void) =20 static inline struct folio * alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_nid, - nodemask_t *nmask, gfp_t gfp_mask) + nodemask_t *nmask, gfp_t gfp_mask, + bool *zeroed) { return NULL; } diff --git a/mm/cma.c b/mm/cma.c index c7ca567f4c5c..27971f6264ab 100644 --- a/mm/cma.c +++ b/mm/cma.c @@ -924,9 +924,11 @@ struct page *cma_alloc_frozen(struct cma *cma, unsigne= d long count, return __cma_alloc_frozen(cma, count, align, gfp); } =20 -struct page *cma_alloc_frozen_compound(struct cma *cma, unsigned int order) +struct page *cma_alloc_frozen_compound(struct cma *cma, unsigned int order, + gfp_t caller_gfp) { - gfp_t gfp =3D GFP_KERNEL | __GFP_COMP | __GFP_NOWARN; + gfp_t gfp =3D GFP_KERNEL | __GFP_COMP | __GFP_NOWARN | + (caller_gfp & __GFP_ZERO); =20 return __cma_alloc_frozen(cma, 1 << order, order, gfp); } diff --git a/mm/hugetlb.c b/mm/hugetlb.c index ed00db703911..a087e915783f 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -2196,7 +2196,7 @@ struct folio *alloc_buddy_hugetlb_folio_with_mpol(str= uct hstate *h, } =20 struct folio *alloc_hugetlb_folio_reserve(struct hstate *h, int preferred_= nid, - nodemask_t *nmask, gfp_t gfp_mask) + nodemask_t *nmask, gfp_t gfp_mask, bool *zeroed) { struct folio *folio; =20 @@ -2212,6 +2212,12 @@ struct folio *alloc_hugetlb_folio_reserve(struct hst= ate *h, int preferred_nid, h->resv_huge_pages--; =20 spin_unlock_irq(&hugetlb_lock); + + if (zeroed && folio) { + *zeroed =3D folio_test_hugetlb_zeroed(folio); + folio_clear_hugetlb_zeroed(folio); + } + return folio; } =20 @@ -2296,7 +2302,8 @@ static int gather_surplus_pages(struct hstate *h, lon= g delta) * It is okay to use NUMA_NO_NODE because we use numa_mem_id() * down the road to pick the current node if that is the case. */ - folio =3D alloc_surplus_hugetlb_folio(h, htlb_alloc_mask(h), + folio =3D alloc_surplus_hugetlb_folio(h, + htlb_alloc_mask(h), NUMA_NO_NODE, &alloc_nodemask, USER_ADDR_NONE); if (!folio) { diff --git a/mm/hugetlb_cma.c b/mm/hugetlb_cma.c index 7693ccefd0c6..c9266b25be3d 100644 --- a/mm/hugetlb_cma.c +++ b/mm/hugetlb_cma.c @@ -35,14 +35,14 @@ struct folio *hugetlb_cma_alloc_frozen_folio(int order,= gfp_t gfp_mask, return NULL; =20 if (hugetlb_cma[nid]) - page =3D cma_alloc_frozen_compound(hugetlb_cma[nid], order); + page =3D cma_alloc_frozen_compound(hugetlb_cma[nid], order, gfp_mask); =20 if (!page && !(gfp_mask & __GFP_THISNODE)) { for_each_node_mask(node, *nodemask) { if (node =3D=3D nid || !hugetlb_cma[node]) continue; =20 - page =3D cma_alloc_frozen_compound(hugetlb_cma[node], order); + page =3D cma_alloc_frozen_compound(hugetlb_cma[node], order, gfp_mask); if (page) break; } diff --git a/mm/memfd.c b/mm/memfd.c index abe13b291ddc..a99617a62e33 100644 --- a/mm/memfd.c +++ b/mm/memfd.c @@ -69,6 +69,7 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgoff= _t idx) #ifdef CONFIG_HUGETLB_PAGE struct folio *folio; gfp_t gfp_mask; + bool zeroed; =20 if (is_file_hugepages(memfd)) { /* @@ -93,17 +94,18 @@ struct folio *memfd_alloc_folio(struct file *memfd, pgo= ff_t idx) folio =3D alloc_hugetlb_folio_reserve(h, numa_node_id(), NULL, - gfp_mask); + gfp_mask, + &zeroed); if (folio) { u32 hash; =20 /* - * Zero the folio to prevent information leaks to userspace. - * Use folio_zero_user() which is optimized for huge/gigantic - * pages. Pass 0 as addr_hint since this is not a faulting path - * and we don't have a user virtual address yet. + * Zero the folio to prevent information leaks to + * userspace. Skip if the pool page is known-zero + * (HPG_zeroed set during pool pre-allocation). */ - folio_zero_user(folio, 0); + if (!zeroed) + folio_zero_user(folio, 0); =20 /* * Mark the folio uptodate before adding to page cache, --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C4F03C1402 for ; Mon, 8 Jun 2026 08:40:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780908005; cv=none; b=tz1mdsaU6ttB3vFyUPt/CmwJAxgqyfmpiCBK8lGkRb7Oy4UhIWwp63D2fP1qm2tAc2uUEJXsXL3h4MKdS9ZgBNAYgDRXG+YJ1yeOp0vc8W2SkCyGCBrMkP/d/BQm2QpoBG9vQ1d7fGF8/LdSmb3bdkKPTPsOHUwF1ha5p71/CvE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780908005; c=relaxed/simple; bh=5wGTQWwTRWiz92QjrAnqbfvJyoSWYuOrxYhosbDCVVo=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=b8hYgCVcHTP6Fch6/K1aUjHKOazfwwd26BW5ZT4L4uQYDi1TbvEi7CNgBS5cVmRRdwu3PEnx6vIn8NbgiDBshktBO51nLA2fK0aNNNXCYZwlIs1D+g+LeR1fe9Gopo9ctzCsySR6mJPoMdKN/zmXA9+mAb9KfsM/dq9Mtbga7To= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=NaTnWor3; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=C7pqns54; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="NaTnWor3"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="C7pqns54" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780908000; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=w/d/SpfWsyx4WFKJs+ZrOIC973gnVs56nqbJSdvDWbI=; b=NaTnWor3bRWWSxIbwABgGIAOYs6wJeflVzD8DUnbJjZj68pRBHJWJkKM/tvlxMZA6F+YwA ZIpB2ot0IY7VlQuDYCE03LvX4ZzAGS8z+EpHUPSslMc3TZT+lRsE9FHFP9hihO4sySoRFz dLhAbA6EQ+jS+gLnECj5juGkj6WADig= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-688-fmomM6H5Ne-pQVZPYFCr5A-1; Mon, 08 Jun 2026 04:39:59 -0400 X-MC-Unique: fmomM6H5Ne-pQVZPYFCr5A-1 X-Mimecast-MFC-AGG-ID: fmomM6H5Ne-pQVZPYFCr5A_1780907998 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-460129ef947so2392954f8f.1 for ; Mon, 08 Jun 2026 01:39:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780907998; x=1781512798; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=w/d/SpfWsyx4WFKJs+ZrOIC973gnVs56nqbJSdvDWbI=; b=C7pqns54dJYadr4RbB0Q5l5qNFAqssZOqua423Tgl0LLsJBaN0dpq+gjlEgEwcnAlc C4oNCLrnDWvbZqPmj+TGg3/sWek1gN51WAl9uhl8su/AUB+Jtjkj8ZAO2wjiyiFRVWtK ge9/8gX+deLiBimH8/1wcqlVo88B6nYca5cSesvidRQW4zClQfixWqzQaRTYW40Mo94/ GWXgdzWRiMPzQwIkvoga81YZ6Zwia/UIOtSWq4jXM2n2jmv4C40842g9sIxFfzlBJWms 1SHA1FydhRB26XnvhIedbfjVQosWq2DeYbJrUT7b8FojSgw+Ab4ydN/EOx2iMg8GI/J5 jqZw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780907998; x=1781512798; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=w/d/SpfWsyx4WFKJs+ZrOIC973gnVs56nqbJSdvDWbI=; b=cHvBELngfyKGb3BWIzbpsiGkk6/hPA8Lg6ND2QPdimgVsxd2BkeOsT5k7r4ioQyggp gA/BeJ3SLIusMDdp958U8fafQViAGGWD2uxIua8gRxB0aZu5t/iF2mDI35ONMgxMMN2i DBMtM8KbLB8Z90FOeaRcj96w+GJLbRxgebGn26f/mBmZGflQRrAXSEPFKwK8HIP1vhTo RMQsEkoWaHnwNrIJeAyMRrtgcxP3J0OwTPyttbF+47QT/bVzvE+q5wXRH+OV/YJc3O8a 5RCn4iv4C0akPiifhjBC0aoYAhY4LDjMFt7kHnux77LMHZcrvYvShwjkUxfhKHEcV6Am aNvA== X-Gm-Message-State: AOJu0YzfkZOxGXX+/EWLAMj8iwOBA+x1rrjeqfLOtUTVh+p/0FJzMscq FzKlBHUguDdYmw6h2wDrhyIyCOCxmuuhhwom9tiHSkByDC/TVZPROLKou0tyJIDh0HaeBEqkbnN NA75UdFVyAfeWLZ/jPCRd7PTC1A/JSgczE0nsZhseOrT+ncE/6JOA1CWCTdgPhvug5ndKatRlH1 xL90JrXK7hUTQrOOGafky7sjp2IpcQ9HCKDeIfjmzIwbY= X-Gm-Gg: Acq92OE0Um1+jiQspTw4Cg6fl06kEqXTL7Xdswpj9F1d3SVMqIovy5s1jekcZ2mueo4 Hvw/T7Eg80VJJ29r/nN0D7XstQDi7DJJl2MgeBv96zWCndzmijdWWexMGpWTLGwewT6pXiPXcT9 k4Uau+AyOgj+dIesuff6NvzaI819P593JwFMK/f6TepUGr7xnWNDP91euEthUlnx2AO++ejUbKt I6A4OXtjQtDyMrDdO71Krq9AaciNoq5AE9CJeU+iIOqyHqHqZy+UTYKghkj2obPOi50oFmMbjzw Hix/ezEnwoT3zmEgYEUOCTWc0aAml9Z9WozQ/DzotsZjPRXEJrWBzNa0lObFk0bhTuT4fre9p1F 08N2Y4wOHOtgB9vWJBdLXH5o= X-Received: by 2002:a5d:6e84:0:b0:45e:f2bd:2b17 with SMTP id ffacd0b85a97d-46030501f7fmr16740289f8f.21.1780907997549; Mon, 08 Jun 2026 01:39:57 -0700 (PDT) X-Received: by 2002:a5d:6e84:0:b0:45e:f2bd:2b17 with SMTP id ffacd0b85a97d-46030501f7fmr16740213f8f.21.1780907996901; Mon, 08 Jun 2026 01:39:56 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4601f344558sm48512372f8f.18.2026.06.08.01.39.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:39:56 -0700 (PDT) Date: Mon, 8 Jun 2026 04:39:49 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 30/37] mm: page_reporting: add per-page zeroed bitmap for host feedback Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The host may skip zeroing some reported pages (e.g., due to alignment constraints or bounce buffer fallback in QEMU). Currently, when host_zeroes_pages is set, all reported pages are unconditionally marked PG_zeroed - even ones the host did not actually zero. Add a zeroed_bitmap to page_reporting_dev_info that the report() callback can use to indicate which pages were actually zeroed. The driver's report() callback is responsible for managing the bitmap: zeroing it before sending pages to the host, then setting bits for pages the host actually zeroed. page_reporting_drain() checks the bitmap per-page in addition to the global host_zeroes_pages flag. No driver sets host_zeroes_pages yet, so the static key is off and the bitmap is never read. Behavior is unchanged. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- include/linux/page_reporting.h | 7 +++++++ mm/page_reporting.c | 8 ++++++-- 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/include/linux/page_reporting.h b/include/linux/page_reporting.h index c331c6b36687..e2e6a487ddab 100644 --- a/include/linux/page_reporting.h +++ b/include/linux/page_reporting.h @@ -17,6 +17,13 @@ struct page_reporting_dev_info { /* If true, host zeros reported pages on reclaim */ bool host_zeroes_pages; =20 + /* + * Per-page zeroed status, indexed by scatterlist position. + * The driver's report() callback must clear the bitmap, + * then set bits for pages that were actually zeroed. + */ + DECLARE_BITMAP(zeroed_bitmap, PAGE_REPORTING_CAPACITY); + /* work struct for processing reports */ struct delayed_work work; =20 diff --git a/mm/page_reporting.c b/mm/page_reporting.c index 84ebc4547119..9c39448e758b 100644 --- a/mm/page_reporting.c +++ b/mm/page_reporting.c @@ -108,6 +108,7 @@ page_reporting_drain(struct page_reporting_dev_info *pr= dev, struct scatterlist *sgl, unsigned int nents, bool reported) { struct scatterlist *sg =3D sgl; + unsigned int i =3D 0; =20 /* * Drain the now reported pages back into their respective @@ -122,7 +123,7 @@ page_reporting_drain(struct page_reporting_dev_info *pr= dev, =20 /* If the pages were not reported due to error skip flagging */ if (!reported) - continue; + goto next; =20 /* * If page was not commingled with another page we can @@ -133,9 +134,12 @@ page_reporting_drain(struct page_reporting_dev_info *p= rdev, */ if (PageBuddy(page) && buddy_order(page) =3D=3D order) { __SetPageReported(page); - if (page_reporting_host_zeroes_pages()) + if (page_reporting_host_zeroes_pages() && + test_bit(i, prdev->zeroed_bitmap)) __SetPageZeroed(page); } +next: + i++; } while ((sg =3D sg_next(sg))); =20 /* reinitialize scatterlist now that it is empty */ --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A36B53BFAD1 for ; Mon, 8 Jun 2026 08:40:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780908016; cv=none; b=SSzseJuHzy9yxW8++AiIdIED1mtZteLdRm0RCS5MBp/iMvldemv29qVK1Dq1wqUWWHMhMbHmOQ5jN+L4igeUapscUQNrsoLg1b5wt0CT/pdiG1S4G1g5XKQrhpxbJVsKJde6mTamypPd7aCUxVTWit870NnFYhHPcloPf2YsHFo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780908016; c=relaxed/simple; bh=JiJWE7l+ip28dmS9nLt+boIUyEWSXna1gjhWeHw0vBg=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=OvpQUsbyawHSGyt5kLNQUd9toJ9tHDFTV1vD0KcAzdDga3IwfdDxXCkYJHSg2CfcfbK/5uw2Yt4uwQVM9wHEY1Gf4gLIvp9lSMz7ZsrehVRL3nJH6KQiC41bz68bm0W8HJYNSQ/MRx0g6YIECLkx7LPKISJJpGCUkU6NFRkPy7I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=B0K0T8O/; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=sANmSgM6; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="B0K0T8O/"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="sANmSgM6" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780908013; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=jJIh5LHG5pOtGoQVym0SRNgbl1Zw+eEBG1qiVMPIhiA=; b=B0K0T8O/K8KJfRdqn5yhcSG5YUD2kOxaHXIFu+rOGvTHNugGjxRrGz7smFuXTEl2AXpKS+ rM8XSkLAHzpc8DyPBkQvuUIRO14XFw64X/n8psAnlOu+Cgz59VFvj/WodA9h8aGu4FVNP0 FGHSw7b7Su2BeqpXKhJb4sEbNJwNx8I= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-433-sEqNWKnZMxOBgR-v7ks1OA-1; Mon, 08 Jun 2026 04:40:07 -0400 X-MC-Unique: sEqNWKnZMxOBgR-v7ks1OA-1 X-Mimecast-MFC-AGG-ID: sEqNWKnZMxOBgR-v7ks1OA_1780908006 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-46016bedbaaso1388387f8f.1 for ; Mon, 08 Jun 2026 01:40:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780908006; x=1781512806; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=jJIh5LHG5pOtGoQVym0SRNgbl1Zw+eEBG1qiVMPIhiA=; b=sANmSgM6ZMQPSHvo4un0bOA/5FJYUkzS79ckGlPQPePZbxrw9rMAJ1qWpVtYWATuth xJqJH/n/cUTQXaBNaNbOOBYY5IZhFATPjL0laSQg1HaqHwkwhiDs+mibOpImVRwWvMBG VeX1TCkEGBO/0Lok7lnzfdVcKnVEJ8goDyXnZm9hXt4M8EUxMzGXxZ0P+JZ3SjDpKRO4 Q7ixs7rUF76ikY4Kvcjs8X4QMUuc3IYrDZnGzXAlMm2WHfIVP4TveVhEJJSn7I4EDbb9 SXPXeAGfOmWSCdTjnQE5N7jSjW1s9Eg5+guYjmgLcHf7knufPzAnFrVmh/J5YNYJYBKU 7QJA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780908006; x=1781512806; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jJIh5LHG5pOtGoQVym0SRNgbl1Zw+eEBG1qiVMPIhiA=; b=hhQyb5VrZgv2WuACL6BlJiPDKPiQZ9rcwa1oMDNEa6/9gZtoUGlXrpjDpSaq3OmzKN sWAMnC4pllaAeOFXBs2Bok5gomATEH9G92yqzKFtNl3PfXdQeM+1W8Tbwm4vzmSN816G ugTIrHhDgvYxm1CZ7ZFHkm8h8qOBevOONt2XYFf5ECnVvLIRYzjlqfuMuaBtf/JWXTne cVhXAKsYqVF4cgtF6/R3nk5CavIzL0TwwjTqEfekT9N8Dd2O3h6w3QfSOPqh7Z6jOJye NLLomZhKQhC6mds0uKSB748gl3Kl5gmtYHI1uyaMW5FDA2vgYdE4aj+Mlfk1BFATD8fQ +Dng== X-Gm-Message-State: AOJu0YxRrEp0BFoMrx55ZHI3+vtOUCsC38Dgb6zq7FHO/B+J+cYANP3x DYhP5XAUGjwO0fq8RhI2Vc0P96UeEQ3e0YkmoiunoujLfDf9Vmi7NmoABe0r/LGI0nTkWtuIfsF 3Xb0nnb7auMUDwvNpGZgf19IcRaBB0/x2ac2Y/mQDiBzSz+oEO2LDKqLZqfRHlUpHObmnbeuP6f y+bY+/A5g9DJ9Xs9kANaXx+gCjV4Wq/MnCy1Z+iyAAEno= X-Gm-Gg: Acq92OF68SoOXBzhdhSwuUUFz16JgVZqahTLLFeXte+eIMZbqH5YBvapRRnwNQ0fdsl MQwEC07NmgNGqdT47HUJGDCRdrkjCSld1V2E46u8OzRWDz874uo2Wmi/lG1zvQqXsxOdZiB+EPT kpagnvErQUoEVXKNGWMHAPE2mJ3q7LU/5/XiQlNFFsDYw1bsp1brUgTPGxqa0RTfx0XTHlwt1M0 +y36sjSzHpsHbuFDxtC1DWrkXGBSab/RxLL6stRp7dGfO3HkgNPkfCrNdBmiywdBKJQlgvxkSgA SqtBhdTNa2GM2qQGyRoazIy7/jPjwNSgSMn4q9J4k/NlvtbepdCZ8e7f/jsBCcgb+UbHr5VUoZt qbJ1HELoDDaRyNV8G8Je7QUQ= X-Received: by 2002:a5d:64e6:0:b0:460:3233:6700 with SMTP id ffacd0b85a97d-460323367c8mr22945834f8f.43.1780908006248; Mon, 08 Jun 2026 01:40:06 -0700 (PDT) X-Received: by 2002:a5d:64e6:0:b0:460:3233:6700 with SMTP id ffacd0b85a97d-460323367c8mr22945715f8f.43.1780908005559; Mon, 08 Jun 2026 01:40:05 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4602cda363bsm38411715f8f.31.2026.06.08.01.39.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:40:04 -0700 (PDT) Date: Mon, 8 Jun 2026 04:39:57 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 31/37] virtio_balloon: submit reported pages as individual buffers Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Submit each reported page as a separate virtqueue buffer instead of one buffer with an sg list of all pages. This avoids indirect descriptor allocation (kmalloc in the reporting path) and gives per-page used length feedback from the device. On error, the already-queued pages are kicked and drained before the error is returned. The caller (page_reporting_drain) then marks the batch as unreported, which is conservative but correct. Note: if the virtqueue is broken, wait_event on virtqueue_get_buf hangs. This is a pre-existing issue: the old single-buffer code had the same hang. EVENT_IDX is not a concern: callbacks were never disabled, so the virtqueue manages used_event automatically. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- drivers/virtio/virtio_balloon.c | 40 +++++++++++++++++++++------------ 1 file changed, 26 insertions(+), 14 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index 6a1a610c2cb1..53b4a3984e7d 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -187,7 +187,9 @@ static void tell_host(struct virtio_balloon *vb, struct= virtqueue *vq) =20 sg_init_one(&sg, vb->pfns, sizeof(vb->pfns[0]) * vb->num_pfns); =20 - /* We should always be able to add one buffer to an empty queue. */ + /* We made sure the vq is large enough so we should always + * be able to add one buffer to an empty queue. + */ virtqueue_add_outbuf(vq, &sg, 1, vb, GFP_KERNEL); virtqueue_kick(vq); =20 @@ -202,25 +204,35 @@ static int virtballoon_free_page_report(struct page_r= eporting_dev_info *pr_dev_i struct virtio_balloon *vb =3D container_of(pr_dev_info, struct virtio_balloon, pr_dev_info); struct virtqueue *vq =3D vb->reporting_vq; - unsigned int unused, err; + unsigned int i, err =3D 0; =20 /* We should always be able to add these buffers to an empty queue. */ - err =3D virtqueue_add_inbuf(vq, sg, nents, vb, GFP_NOWAIT); + for (i =3D 0; i < nents; i++) { + struct scatterlist one; =20 - /* - * In the extremely unlikely case that something has occurred and we - * are able to trigger an error we will simply display a warning - * and exit without actually processing the pages. - */ - if (WARN_ON_ONCE(err)) - return err; + sg_init_table(&one, 1); + sg_set_page(&one, sg_page(&sg[i]), sg[i].length, + sg[i].offset); + err =3D virtqueue_add_inbuf(vq, &one, 1, &sg[i], GFP_NOWAIT); + if (WARN_ON_ONCE(err)) { + nents =3D i; + break; + } + } =20 - virtqueue_kick(vq); + if (nents) { + virtqueue_kick(vq); =20 - /* When host has read buffer, this completes via balloon_ack */ - wait_event(vb->acked, virtqueue_get_buf(vq, &unused)); + /* When host has read buffer, this completes via balloon_ack */ + for (i =3D 0; i < nents; i++) { + unsigned int unused; =20 - return 0; + wait_event(vb->acked, + virtqueue_get_buf(vq, &unused)); + } + } + + return err; } =20 static void set_page_pfns(struct virtio_balloon *vb, --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 376703B5F59 for ; Mon, 8 Jun 2026 08:40:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780908019; cv=none; b=tcUk6AUrV4jXSaPKsoZUx9FHwPN9uPF+J7A1eXJFMEXp/9aHbpuLpL6itwqWBdIh27tvw3uSQQufzKlwu7pY+EiVXqfkvLGN8Wub0CkXeTFsptORM7/ve3it7TX7rA3RR3tozXwPB3OT9oPxInLjoBPlj+WpXZjNT0OTJaFpHnI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780908019; c=relaxed/simple; bh=CCqjoUNUurzYE4twri0CPtkeKv57rqlJVdiWTN8T65w=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=fQwe7gYBh85W9wBiwVRDYQetSZSSh/033Of+0Xh4O8gBp8q8HQWhxAdO6fKBT2fqNcfABQR2AE/Hy0FTVu9xzm9L05jMdh71m6OzCxdyj0jt032V5y7n0KLlqNMzuclMuQy3QWi+7Wu85whyRJsojArE/Rfvxym4nv+9SO8n3Ys= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=dENg5k3K; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=DJTQVPsP; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="dENg5k3K"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="DJTQVPsP" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780908017; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=vnDpUYDWGGzhd8LJgVzrd2lW64WfRqjlcwd6dZFOBgQ=; b=dENg5k3Kco/IxvFzlgNQNrruHfCEIKy4CByPCP1tE+7TeHklVbBp3Eduhxva04lNDHy5xF 5QiHuw4hwPM1jrEgAxnALL9uTvMkf9P3wXMOMzlpRKayPfERJ6hWEwXWKmLzsVstKhoMRO V2FoVAoGQIUMqVl8ILs08dihZiJ6igQ= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-2-x3H7OVYXO8-OikezdVHZag-1; Mon, 08 Jun 2026 04:40:16 -0400 X-MC-Unique: x3H7OVYXO8-OikezdVHZag-1 X-Mimecast-MFC-AGG-ID: x3H7OVYXO8-OikezdVHZag_1780908015 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-45efa2f7009so2519844f8f.3 for ; Mon, 08 Jun 2026 01:40:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780908015; x=1781512815; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=vnDpUYDWGGzhd8LJgVzrd2lW64WfRqjlcwd6dZFOBgQ=; b=DJTQVPsPZV4zkS8PNl6PrJnNGxDxgLy3IoOgjxY2c93Ph17QqGqAHnzf0cL0wPKlar /9FNut8Wf9a50FW7q3dbIArMFux2vYfs/VD7YvlA1yaUz7Hm+Ei9LZ7BCYY3kYWGx706 ARFqqAdejh3W7sXDS2zJTzAcnieHWqFDgYHZllrV8ImSnKp+BTX6IThvWlplW7aqEChy o+lTyagl1YgOgS1H1Wh7YtUhiRdUF2iMHPWgc1P5CMqUHp+cdz3v46uspJraLoCW/WFo VExpBU5RLQLG15xQaX1WZ4zy76UPOwia9VnFsADB0IDG87nIPQ94pN0BYXL+nzNaWdbE cLOQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780908015; x=1781512815; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vnDpUYDWGGzhd8LJgVzrd2lW64WfRqjlcwd6dZFOBgQ=; b=k6CkGOYYed/ZvloOEg4KP8K+l8LZtZX9TgefeQnbw+oH0gDSXnR41Yoi3dp0oxnI5+ UrVV8lSpMM1Mccm1Krau3aAIPYL0bfYj6FheB1b7PfiEnTam3nM8nlD5oAFQJDKPzhZb OWkVMF6jUvhyU8dfM2ORLqH6eVXVSohfwT0+MfgsQj1iPmdPcU0Dw6PDi7moxqNKWAc0 EraSIGYxVLJc6OogTHnKblG/ZI/itJLR3Q8GrVtgQ3XegjqS1C1w2eWndDwEVPVD5iT5 3bl8ZgjqFnIL+yfaNk0tgd0mnE/aupb4DQOw846VMfA2JP7l8Hw7zFo4CJBMjljeCaGD YNqA== X-Gm-Message-State: AOJu0YyouWUgHZTuC3XtVXqWLLk8csL92H+UPu4vuRt0ugjU0pck8VIM xIc8xnsglcrw6UwJ/lI1fv7WvN0ABa+zscH+PecFzI1MFWUewpvPV9fp6e3nh3oZpTceSZNrIlZ bJppJopQv34RDjsX0LLom08P6RHPnJq5oJGm5Upaj6Upcqi3NKHA2cyP6/iiQ7kEOwvb2kM0lws Omdhg3vYQhC6WMq+cdyng2/33Y8mKyrs1wNf8hUQApWYk= X-Gm-Gg: Acq92OGPUgSzPv51B3vtQwPOr+0gW7HQGA6Hgp8x6rCaK2Ug4BtPYms1s54cy9OZgAs 8G2T310BjMSMdtyO1M0ZI8iUmb5GsLX4jL1r6ZN1fY+TWhIGCxTxUORh01gENr71zrsZ2HEJCWe Sxwl+77IoBYlYN3jraNVyfFtldlwBfFY1cCJKTsZUZjgM6ykivTiM3h5Y8QZ+G6j3WI1cywFIDf xznnMDqjB0NC16Mc5u39pY/FtovO6lfLBBL59AQYwqeG1qgkAIJoBgnrPOC9oOguT2vWV5RwgcC zyL+udpdACk1zUzlpLSeejQ0JsvrA5WYQkvQ0v9pFRM1/mM6b/2F92UmMl/Uul7USVgCazcT0Sh oJmqr8Hc6Fvuzc7A05mX2Cbg= X-Received: by 2002:adf:f2cf:0:b0:460:e00:1213 with SMTP id ffacd0b85a97d-46030502efamr17987095f8f.23.1780908014416; Mon, 08 Jun 2026 01:40:14 -0700 (PDT) X-Received: by 2002:adf:f2cf:0:b0:460:e00:1213 with SMTP id ffacd0b85a97d-46030502efamr17987021f8f.23.1780908013858; Mon, 08 Jun 2026 01:40:13 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4601f34413csm49706045f8f.21.2026.06.08.01.40.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:40:13 -0700 (PDT) Date: Mon, 8 Jun 2026 04:40:05 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 32/37] virtio_balloon: disable indirect descriptors Message-ID: <3c48b12a88c03c9cf9ac2c7abaffbb8aedd470d9.1780906288.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" A follow-up patch (DEVICE_INIT_ON_INFLATE) adds an inbuf (bitmap) alongside the outbuf (pfns), making total_sg=3D2. With INDIRECT_DESC negotiated, virtqueue_use_indirect() triggers for total_sg > 1. Balloon never needs more than 2 entries and virtqueue sizes are 128+, so indirect descriptors are unnecessary. Disabling them avoids a GFP_KERNEL allocation inside virtqueue_add_sgs when VIRTIO_RING_F_INDIRECT_DESC is negotiated. This allocation could trigger OOM reclaim while balloon_lock is held, leading to a deadlock: OOM notifier -> leak_balloon -> balloon_lock. With single-buffer submissions (previous patch) and no indirect descriptors, virtqueue_add never allocates memory. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- drivers/virtio/virtio_balloon.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index 53b4a3984e7d..1fa1c7fa285f 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -7,6 +7,7 @@ */ =20 #include +#include #include #include #include @@ -1175,6 +1176,13 @@ static int virtballoon_validate(struct virtio_device= *vdev) else if (!virtio_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_POISON)) __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_REPORTING); =20 + /* + * Balloon submits 1-2 sg entries max per buffer, virtqueue + * sizes are 128+. Disable indirect descriptors to avoid + * GFP_KERNEL allocation in virtqueue_add under balloon_lock, + * which could deadlock via OOM -> oom_notify -> leak_balloon. + */ + __virtio_clear_bit(vdev, VIRTIO_RING_F_INDIRECT_DESC); __virtio_clear_bit(vdev, VIRTIO_F_ACCESS_PLATFORM); return 0; } --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CABC53B27F7 for ; Mon, 8 Jun 2026 08:40:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780908027; cv=none; b=geQMR7OiVV0fovf5QWAn16oDuRPhSxFM2wrqK1FWaU/oijbopQMBFgXww1RXiVjSgmAgQqhzfBCHy93w8MzhzJEmewEVK3R/Wn/huHOOQx4yaojqMspxHoPm2cc8ZHjMwOxLZaLO3yGcDfBs+EsPgoPBzbgXLpAUwsdoSq/UzFM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780908027; c=relaxed/simple; bh=lBJ4LLu1g6y3CucnwB/F1BfG3upMe4x3Hxcooku9Cnw=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=J3Xj6U000vKgVJf8IxxvGrx4kUy9R8Cc08Yk7QYAX7xligF+BtVCRfzGM3Jhfze14z3Qoh3wi89Jagsg3189Br280GhIeDn3Ec0x0IdbU8UkFg/me0YUhNq5B48vk4OlfX5epwdwMdXTYoVIED3MAWZRY6joiVweQGDhWjIDz0I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Z11GtX6W; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=a+psbBqN; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Z11GtX6W"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="a+psbBqN" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780908024; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=71/5gV60k4yG26aCAIxd/scVyaKUto/kd/wMt0/Zjmg=; b=Z11GtX6Wa3L/mhVNq2UoDXcYEB1Eo0ZQEYh861HYCA5g0IeP8iQAEVir3s3hbiiXLViHrE Pw4dOUspS+i+l/XwGlvBgFi+bjoa//GOOJhA7WrWQpudxgjc9bANCB7Un6+CIEyVIcgmU1 Gcjn9J9oOTqakBIj9rF/Tiu3uvR130g= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-66-zaLVg0aiPJaxJai_ZMASSg-1; Mon, 08 Jun 2026 04:40:23 -0400 X-MC-Unique: zaLVg0aiPJaxJai_ZMASSg-1 X-Mimecast-MFC-AGG-ID: zaLVg0aiPJaxJai_ZMASSg_1780908022 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-45ef0af9517so1922523f8f.3 for ; Mon, 08 Jun 2026 01:40:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780908022; x=1781512822; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=71/5gV60k4yG26aCAIxd/scVyaKUto/kd/wMt0/Zjmg=; b=a+psbBqN7re3iT5Dy8bwCy7GcGg2/8TUGbbeR+FERTnXEvChHMP0jdOWsndL9DU6p9 BEk2Bs/cVAVqf3ItBzh0u/DLRaA+Ce9UvMzwJQ7KusGdRBfQM1DfDRs62aNEa08BojVB pODXCAf1kYdsL1AzrJFDvLcH1z2C9E2AOiiLBYu9erbk4IQY3JXpVwxpWf2ITSbBZTaN Tzq2IibxoLCjRAa1aKuPkeGogMpog5vJylctAyX7H2nXdqDnpn0Rz5nBkn0nVpiNPNtk 0ni8ICKy8YEe++I/AKQwqGlvTEdAswnfll/XBP4o9BP6Zf6DCLJdoJ1Wokrnlg5BWJ0d ZSGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780908022; x=1781512822; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=71/5gV60k4yG26aCAIxd/scVyaKUto/kd/wMt0/Zjmg=; b=WKMLbDCUo03YP6vm9calAQBL1b6UV5qGG+KaHu/8TYRRq1Q3q4YHKCXTf1ZWdgthMV 7lTBRNWBapyf87BY5L+LbPAQem1fS+0SwX2QOiv8UE/wc3v3UC4anOXV0KiYwoVuw0Ge OszOoPS69XBdvKxqn+8opEoYgFCcFmDGRqXZhUPDBT7EkNmZcBKPKJlataIBKNEOuO24 fhKINO6rrP4AYeauQ6cPpPeD8HkBhw+q3dCG8bXpWXiwVOPqyNRibMNyA+UYFycXe6mb bSafjSO9CRoN9T6Shve07wWCaCDRXRmAPSmQef8qwJguqYmL89vxs9kXsi3mISiz8YS7 PZgw== X-Gm-Message-State: AOJu0YzpkZeTatChUWwFnHQ4fo4Nkgg4xXUNQBw3HnCRq9jxdNP8Id/2 3tDRVjqpYLWY0cSrQnerVSsGG2ij+KWjb6hlQ6FMsid/HF0hARZ0H2NQ6ba6HteTppVPmTABezp oJgk2H8c9QirJi3r9SUaUxa7htj1DbwAuaF2n0HCX2UwjST602HoHrSgoGPjpZzRCjy2b8D6UrO TldsotB2nGHmB6bgrWwOSEQ7oWes1crOaykgQVwXkqBCw= X-Gm-Gg: Acq92OH3AkaewiEHN/Y+jk7pHKJoR8OLpVk61uZXoMvIlOoWwGpEkHIzlBoODedZRAr n4iYwrHFKAHVrdWGfmsLDMAiOLCW4Fshb48Jb+VROES4DqQfuDmi2nOxwxXgluOYMhlE4fqSK4O v3MFws+i/NurIXxDS+3Bl+o+QDZ/NVWHDnSFsr5yasnQVWi7eVc+sXNiZn+xEDbZuc/sFjdvWrg 654Zod1gHUfDqyQBgpaVIvEdB9Xb9DKHMLimZA7zE5vnvxudsOmTaPVUNzRBsT2x5UOr+7h5VFL 8FQb3DGlUSimkbRiNqXN4VPadrr0TKlQ4WicU0IYrvLYRg7uyHpVN5MC/ALQSwp2zWzMjCBn7nn o1z0iRsj+vEYcGLJiXtz5sdA= X-Received: by 2002:adf:ec0f:0:b0:45e:db3b:921d with SMTP id ffacd0b85a97d-460307563f0mr16344601f8f.31.1780908022260; Mon, 08 Jun 2026 01:40:22 -0700 (PDT) X-Received: by 2002:adf:ec0f:0:b0:45e:db3b:921d with SMTP id ffacd0b85a97d-460307563f0mr16344502f8f.31.1780908021651; Mon, 08 Jun 2026 01:40:21 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4601f2f2710sm39846375f8f.14.2026.06.08.01.40.15 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:40:21 -0700 (PDT) Date: Mon, 8 Jun 2026 04:40:14 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 33/37] mm: page_reporting: add flush parameter with page budget Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add a write-only module parameter 'flush' that triggers immediate page reporting. The value specifies a page budget: at least this many pages (at page_reporting_order) will be reported, or all unreported pages if fewer remain. The actual number reported may exceed the budget since each reporting pass processes a full cycle across all zones. This is helpful when there is a lot of memory freed quickly, and a single cycle may not process all free pages due to internal budget limits. echo 512 > /sys/module/page_reporting/parameters/flush Note: the set callback runs under kernel param_lock, so writing this parameter blocks other built-in parameter writes until the flush loop completes. This is acceptable for a privileged debug/test parameter. Note: flush_delayed_work is called twice (once before the main loop, once after). The first call ensures any pending work completes before we submit new budget. The race with __page_reporting_request is benign: if a concurrent request arrives between our flushes, it queues new delayed work that will run on its own schedule. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- mm/page_reporting.c | 54 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) diff --git a/mm/page_reporting.c b/mm/page_reporting.c index 9c39448e758b..400150a2aa15 100644 --- a/mm/page_reporting.c +++ b/mm/page_reporting.c @@ -358,6 +358,60 @@ static void page_reporting_process(struct work_struct = *work) static DEFINE_MUTEX(page_reporting_mutex); DEFINE_STATIC_KEY_FALSE(page_reporting_enabled); =20 +static int page_reporting_flush_set(const char *val, + const struct kernel_param *kp) +{ + struct page_reporting_dev_info *prdev; + unsigned int budget; + int err; + + err =3D kstrtouint(val, 0, &budget); + if (err) + return err; + if (!budget) + return 0; + + mutex_lock(&page_reporting_mutex); + prdev =3D rcu_dereference_protected(pr_dev_info, + lockdep_is_held(&page_reporting_mutex)); + if (prdev) { + unsigned int reported; + bool interrupted =3D false; + + for (reported =3D 0; reported < budget; + reported +=3D min(prdev->capacity, budget - reported)) { + /* + * First flush completes any previously scheduled + * reporting work. Then request a new reporting + * cycle and flush again to execute it. + */ + flush_delayed_work(&prdev->work); + __page_reporting_request(prdev); + flush_delayed_work(&prdev->work); + if (atomic_read(&prdev->state) =3D=3D PAGE_REPORTING_IDLE) + break; + if (signal_pending(current)) { + interrupted =3D true; + break; + } + } + if (interrupted) { + mutex_unlock(&page_reporting_mutex); + return -EINTR; + } + } + mutex_unlock(&page_reporting_mutex); + return 0; +} + +static const struct kernel_param_ops flush_ops =3D { + .set =3D page_reporting_flush_set, + .get =3D param_get_uint, +}; +static unsigned int page_reporting_flush; +module_param_cb(flush, &flush_ops, &page_reporting_flush, 0200); +MODULE_PARM_DESC(flush, "Report at least N pages at page_reporting_order, = or until all reported"); + int page_reporting_register(struct page_reporting_dev_info *prdev) { int err =3D 0; --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4BA503B27C8 for ; Mon, 8 Jun 2026 08:40:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780908036; cv=none; b=Vrry7EmWikoVxiGKELgrTJOQO65IuULQr20R4FyISQ0GPdkmJDyeDH6UnRg/pcD7+ND/v3WTXTHpjQ4XUZdVZHFvWf8q3mVHby2ArkhEUeNKlCsSuf7QCFgmtzPg2WRZpixVUNQWtBc8W94qdk57CH4ZscAhwljcMm0mEmHy1Iw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780908036; c=relaxed/simple; bh=o1q520LUtIf3WEFA+mnt5VZVOCpKdiGNtHAvZKuDwl4=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=UwrTu3p8z6jjJ0CHi9GHlsE4s2H/rWGjvM+JGmm5FMarX6o+SM0iV4k3iiNa4bHactP+6kepuU6tDxRIpFjC4VWFLFnMk7krj8XrrucRGRz7IfEO7m3aX+z8XvrJhjff71BiZ0+1a6vN95yj6eKbjSDUGMEYJ+x6gIYVM+Z6Rro= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=H9eL3znh; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=aMbKFF0S; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="H9eL3znh"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="aMbKFF0S" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780908034; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=PhUe/N+eFNvMvlBoMUh1oQQhfmsYB8gjnR/tug0hsFs=; b=H9eL3znh8XZ7mIvETiHRlsiXixXzwTwILaQgoLLbjOC3+OgroRj8yz4Sh+1zXkGcFvqJjm ygie46lFslLuD7hViJEKKLZZ7EmYYMoU/jKVT+5yfh1YNt6WsCAhG9LJ6IJQECh82gBPu9 UrcS9WRYlUptyoksi62srhH1rxYzlz8= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-691-o0E4z6bEM-GrwDmjp2zgzQ-1; Mon, 08 Jun 2026 04:40:32 -0400 X-MC-Unique: o0E4z6bEM-GrwDmjp2zgzQ-1 X-Mimecast-MFC-AGG-ID: o0E4z6bEM-GrwDmjp2zgzQ_1780908031 Received: by mail-wm1-f69.google.com with SMTP id 5b1f17b1804b1-490c56e2576so22832735e9.3 for ; Mon, 08 Jun 2026 01:40:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780908031; x=1781512831; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=PhUe/N+eFNvMvlBoMUh1oQQhfmsYB8gjnR/tug0hsFs=; b=aMbKFF0SOowGoN+4BPv618svVwfWVmqYKLp9JPGkVgmMd+5uUExGECL1N2P3hHYlmp 4kucTG0I2RvSFexMEbFYbVcjhFxz/jQouQ54R5NoIsjvwA2sx1zQ0zcjPWqJRMXcPG9x nJmdR8th5YzLP+RSwrf6wgOM7Sh8jpWgHkYEjH34tGf/eOlRkPhYfdniFVKsDyyaZb6A ZTffL5hMBA9qg35PCax8BxDHwWycDV4zELWw1KLL0G+BylQQvbzJyj2g1jKSLEScb9mL OOLyh4Fy/oEBCCfaydFjsFs1eMP3/9ztu2bJJ4pZZ0dM+Ed0/xtiFIsNXb3NRu0dGAWA pIUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780908031; x=1781512831; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PhUe/N+eFNvMvlBoMUh1oQQhfmsYB8gjnR/tug0hsFs=; b=f07BQyR6rynpU+xkvSdickaf2PSQgsU/A2SGrsGgN2nwObfOXDMM1Wio1uiRCKBoLM vr2lprIaPRyPxWgvnhYLUt5AT47JFEJhZ/VFyZIqqqm3jB1aH8QEQm3qVGPyOta0BFir O9J4WfHLcWPcYQuX22HsKWr3nhaQwJ8701gcrTK9Xdq6eVCAChXgBBdcbVq6hcJM3HLD fSjBYjVJTTJzarNWLPnwKF3rZoYcU1XiEF/KqdtxWBh8apg29Me7xaF0QeAskXJeM+sQ UjHQHOnqa6z9lUMpqmLnGtsJXlnJRAVNwLp+ch3r+hNHniXaS7frQ9pkPaurBhHC9MUg j1IQ== X-Gm-Message-State: AOJu0YzxWBI37Zg8zEfa/HQciMnUvQr7zRrzZGPASfEGld+odYW5aCfR by98n1fO+fGh9iTH1/XdbPITGw/0TEYyNon5t6MJ10bZuenFo3mkEo7T9Aes3ruQ0x4q+W4/8tW B2hW6B7nSwgdTzTaQoS10Q2PZAbzVdtCOCQ2AnB8RShSZDkZVU92n9fEBc8tjTCvDg8EcQGYnZh FbJxW10qX/UoHNgpGQevd1f9z/BBz2yQYoVzzbeRUZVHM= X-Gm-Gg: Acq92OG5BN7XIg3By2vqu1D7alodjl+CZV3PQzkgdVx4OBCfIYxjo8LOB6AHDsLPW8O dafcGvGE4yxr1b0CNfyRpTnf/bCUmZz69iu2W403sKz+or1ZwstxDUQtza+hFYPCFvXDNA4SV1f rcABPXVAZow1Ic9/bkWJ/CuwXx5rnvd2nz/HNvyXjkzGH+NVbmCDmZkZJ0WjPb/27B0oAsFsj8e S8pT0ndb1UnH2Ndm3V8UXKMdUPG6KPLn2cFNeqhgXPHJ2yFi7GZ+ixe9gmQXQ0mmNAGKTu+FdwL gRSK3Al/00y9eO9j28ItnqlKLkZ/lSgdSHJSl6wCa5gpSgSFU4KUHKDk2fv9Wh4t//Td8gXqS7e Omt3WO4zROa88YadbJt9hxRc= X-Received: by 2002:a05:600c:620b:b0:48a:53ea:140b with SMTP id 5b1f17b1804b1-490c25cf45bmr250656325e9.28.1780908030846; Mon, 08 Jun 2026 01:40:30 -0700 (PDT) X-Received: by 2002:a05:600c:620b:b0:48a:53ea:140b with SMTP id 5b1f17b1804b1-490c25cf45bmr250655175e9.28.1780908030307; Mon, 08 Jun 2026 01:40:30 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4601f2dcae2sm53227082f8f.6.2026.06.08.01.40.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:40:29 -0700 (PDT) Date: Mon, 8 Jun 2026 04:40:21 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 34/37] virtio_balloon: skip zeroing for host-zeroed reported pages Message-ID: <4276165a2f7a1f8314341f639f2d9663808d7bd8.1780906288.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Implement VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED (per virtio spec proposal): when negotiated, the device initializes reported pages (zeros, or poison_val if PAGE_POISON). Check per-page used length returned by the device to determine which reported pages were zeroed. If used_len matches the page size, the device successfully initialized the page (e.g. via MADV_DONTNEED), and we set the corresponding zeroed_bitmap bit. Gate host_zeroes_pages on the feature bit and page content: when PAGE_POISON is negotiated with non-zero poison_val, the device fills with poison not zeros, so pages are not zeroed. Clear the feature in validate() if REPORTING is not present or if PAGE_POISON is active with non-zero poison_val. See the virtio spec change: https://github.com/oasis-tcs/virtio-spec/issues/244 Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- drivers/virtio/virtio_balloon.c | 22 ++++++++++++++++++++-- include/uapi/linux/virtio_balloon.h | 1 + 2 files changed, 21 insertions(+), 2 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index 1fa1c7fa285f..e3afa6f32ba5 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -207,6 +207,8 @@ static int virtballoon_free_page_report(struct page_rep= orting_dev_info *pr_dev_i struct virtqueue *vq =3D vb->reporting_vq; unsigned int i, err =3D 0; =20 + bitmap_zero(pr_dev_info->zeroed_bitmap, nents); + /* We should always be able to add these buffers to an empty queue. */ for (i =3D 0; i < nents; i++) { struct scatterlist one; @@ -226,10 +228,14 @@ static int virtballoon_free_page_report(struct page_r= eporting_dev_info *pr_dev_i =20 /* When host has read buffer, this completes via balloon_ack */ for (i =3D 0; i < nents; i++) { - unsigned int unused; + struct scatterlist *entry; + unsigned int used_len; =20 wait_event(vb->acked, - virtqueue_get_buf(vq, &unused)); + (entry =3D virtqueue_get_buf(vq, &used_len))); + if (used_len =3D=3D entry->length) + set_bit(entry - sg, + pr_dev_info->zeroed_bitmap); } } =20 @@ -1051,6 +1057,9 @@ static int virtballoon_probe(struct virtio_device *vd= ev) #endif =20 vb->pr_dev_info.capacity =3D capacity; + vb->pr_dev_info.host_zeroes_pages =3D + virtio_has_feature(vdev, + VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED); err =3D page_reporting_register(&vb->pr_dev_info); if (err) goto out_unregister_oom; @@ -1176,6 +1185,14 @@ static int virtballoon_validate(struct virtio_device= *vdev) else if (!virtio_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_POISON)) __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_REPORTING); =20 + if (!virtio_has_feature(vdev, VIRTIO_BALLOON_F_REPORTING)) + __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED); + + /* Device fills with poison_val, not zeros; disable zeroed hint */ + if (virtio_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_POISON) && + !want_init_on_free()) + __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED); + /* * Balloon submits 1-2 sg entries max per buffer, virtqueue * sizes are 128+. Disable indirect descriptors to avoid @@ -1194,6 +1211,7 @@ static unsigned int features[] =3D { VIRTIO_BALLOON_F_FREE_PAGE_HINT, VIRTIO_BALLOON_F_PAGE_POISON, VIRTIO_BALLOON_F_REPORTING, + VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED, }; =20 static struct virtio_driver virtio_balloon_driver =3D { diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virti= o_balloon.h index ee35a372805d..13074631f300 100644 --- a/include/uapi/linux/virtio_balloon.h +++ b/include/uapi/linux/virtio_balloon.h @@ -37,6 +37,7 @@ #define VIRTIO_BALLOON_F_FREE_PAGE_HINT 3 /* VQ to report free pages */ #define VIRTIO_BALLOON_F_PAGE_POISON 4 /* Guest is using page poisoning */ #define VIRTIO_BALLOON_F_REPORTING 5 /* Page reporting virtqueue */ +#define VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED 6 /* Device initializes repo= rted pages */ =20 /* Size of a PFN in the balloon interface. */ #define VIRTIO_BALLOON_PFN_SHIFT 12 --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 54A9E3B5854 for ; Mon, 8 Jun 2026 08:40:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780908050; cv=none; b=rBUY4B4KJ0txA/bFY+zait6VrJFEw+BZ5WPPPnX65cKdLMn80E+ygOy377q7kfnrVhAptBsPUJrmw4O5UR4q0msT3udR7eliTOHE3WhNuBclBWBcB7wHJvJpAx9UJzFpniTqXhckQLFmDyNNLmNVMf/UDRKKZpnfSBGrd/kw/+M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780908050; c=relaxed/simple; bh=t7dTtFcMNo1oGbZCogjoqyizO/j3E+pcvyk5ppy+DwY=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=oF3jVeD8Hw9SoQDZDV7G8TA9lLvr2vsiCyZj2pmGrLHXcG8X4Oeq7LC/I0TNx4/f1p8uQnlmXNf2AMuMn4nTm3CY0TnPi4/Xjs212u/8HwNiSrZO6I8jzi9UH1kWPm+MluOGH/j+dKdZfO0xIEIOTZx6+NlLG6xnuUo0WncmJFI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=CyScqIgk; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=d6Ky4wyE; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="CyScqIgk"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="d6Ky4wyE" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780908048; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=4h2My/REs5Vp77QAaORxbiGwFbWeluHTenTERVDNw80=; b=CyScqIgkvbPjDWFTomxC6UduwHqNrPKtxhhsiFk9IvXf8/yPwx9g1rpZMis+yocjJDDAzZ Pv8A/xZjjYn7PzRiz//2HwGWeM+8T/9K08bucmiibRHJSOhPUlBfXGNV9j9M+TRRE6Zy+r oI5Fv20T8/Jt8Xt9t8ENGFkNtQCTNQ0= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-9-TTXnBsLiObi6mwl6b2OO_Q-1; Mon, 08 Jun 2026 04:40:47 -0400 X-MC-Unique: TTXnBsLiObi6mwl6b2OO_Q-1 X-Mimecast-MFC-AGG-ID: TTXnBsLiObi6mwl6b2OO_Q_1780908046 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-45eebc943bfso2307147f8f.0 for ; Mon, 08 Jun 2026 01:40:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780908046; x=1781512846; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=4h2My/REs5Vp77QAaORxbiGwFbWeluHTenTERVDNw80=; b=d6Ky4wyEEm1um5rDNrvMov99suWtnYnJSNBcn2eAOdM3w5UwiAuwYuU+2LFRxchfBP EKhkF/wvJ7htGd2T1eS9NKsJ3yrw5mc4oaw9gjh5njZSScs4UMXcXqO5FrDejuisX9J/ JGlRPfSKM83S3zwBW24g0VbaHrfGNQsfELpiqJupRMmfDaiC4imQWbeR/9oUz3xqMyBF hzVzveCyTOS3RAnDRGnNzj6XDxjWNlCjlKHFogtOEPWZAzqY6CjM/J9KC87WtT1n29RY XTEP8+s7zWlC5mZelxjGaXQMKTSZYUzf92ejJNlP0KW4AzmHdJozBWGxTN7uDnMtJ9Zz jJzA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780908046; x=1781512846; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4h2My/REs5Vp77QAaORxbiGwFbWeluHTenTERVDNw80=; b=NNtADtUccofxj8KSzTtgAkT6TRiyKGv9CyMVF9xRdjA/B9qxkBm8ACGneWCOKgux4k o/SboqiAK6oB3/LPlyrNyRI2dssajRYOfi6VcjOadLd9hLMge7im6oBKPjTeGpTCH0rN T2go9MItKNNbeqSJGmBmvmpHweDAmUQEfWfgablUh6FSbwABn3FWo/ohBjNd14vW32/C afiwnZhhnwqFf4QYz5E1HAymfZB/9HUKmsn7prnEaWtN1j4QshQ+kMcruDg20/7y87pa N81ioVY2Mcn4hQxdlSm+RFBA00336IW6a/MnGhZSDykjS0R55qsue3ZYDO8XL3/TWRJM Ixpg== X-Gm-Message-State: AOJu0Ywx5nT9XZSw8DVaLxBy6lSIuBQ0uAxBRJErTwlfI6sSszGA2122 NxsqHQlI4uXDTbmlS0JBmH30Sqqtd8M2n6ybVw2ppcyYY1gnwyUQR+YA3VymQcQVRBx4HY8ldL/ Pi7yFuEFSTvOh3KNimdVA3qcYsdcMDlzOR1qrLJiZR+BiX/SbmOmUoXgtPEMIT4PmzBvngKPA1k AK9EL2Ytl4kZtGYFgIvOofi7TSpB85btStXW1DIfI/PRY= X-Gm-Gg: Acq92OGwlsKifJMQhi0lrzijBK8i7GxB2BDR4YwV5+hccCtv1LpWR1ITu0uPDyj04CA 33JDLYb0rDX+3x09o29pcK72MJ4lhzbak493DmRlGoeOjsTX2I4Mead/fjQDqAWko9FQIhAK7kQ yPGfiURFhb89TTor5XV6tW6NbchUkTCtyY10CaEKaoEP5DNcrx5WndIu96Yd+RWmhT4OZnY3aMV A0F6td3bn6AFuuVs590XAG8nXKNHgqplnwRYOWKsbsvMmujt8twH/RWJVV0Azxv4CyiwRAveG5z 6yGdPSJbPQ517L3AsrUjb5K8qODqS9iFvbb7qDsxfAHq7C62N6SKgzJDt/+7XFrGVu69QLPepjs NP3L4Lrp7B0qMxKYJqIZo+Hg= X-Received: by 2002:a05:6000:491a:b0:45e:f93a:2069 with SMTP id ffacd0b85a97d-46032db4e83mr21033570f8f.23.1780908045929; Mon, 08 Jun 2026 01:40:45 -0700 (PDT) X-Received: by 2002:a05:6000:491a:b0:45e:f93a:2069 with SMTP id ffacd0b85a97d-46032db4e83mr21033456f8f.23.1780908045356; Mon, 08 Jun 2026 01:40:45 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4601f344558sm48517487f8f.18.2026.06.08.01.40.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:40:44 -0700 (PDT) Date: Mon, 8 Jun 2026 04:40:30 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 35/37] virtio_balloon: disable reporting zeroed optimization for confidential guests Message-ID: <101bfae36b0f6445ff0189c2b0936648f3f46118.1780906288.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In confidential computing environments (TDX, SEV-SNP), the host is untrusted and may lie about zeroing reported pages. Clear DEVICE_INIT_REPORTED in validate() so the guest does not skip re-zeroing based on hints from an untrusted device. Note: currently REPORTING remains enabled and VIRTIO_F_ACCESS_PLATFORM is cleared in CC environments. This is known to leak free page physical addresses to the host. Whether that, or ballooning in general, is a security concern in CC is up to the user. This patch only disables our new zeroed-page hints where the host is untrusted. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- drivers/virtio/virtio_balloon.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index e3afa6f32ba5..bf1172ad5419 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -19,6 +19,7 @@ #include #include #include +#include =20 /* * Balloon device works in 4K page units. So each page is pointed to by @@ -1193,6 +1194,8 @@ static int virtballoon_validate(struct virtio_device = *vdev) !want_init_on_free()) __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED); =20 + if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) + __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED); /* * Balloon submits 1-2 sg entries max per buffer, virtqueue * sizes are 128+. Disable indirect descriptors to avoid --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F404A3B777B for ; Mon, 8 Jun 2026 08:40:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780908059; cv=none; b=jSuJL2q/soc1a0L8iL9T1O+0+3BdWhNnSh8/z9Q5XT8E3vkztSAKoQ3z0hvYWPiRS5iUsLo2vxK8qAzmX/pfz2jqI0EEkuaNucLaF6MGu+bNlsYeUDBJ3pTxdhn0qtJgRrj+CfSP7wYGPtXXzS4Cfuo9X5qBFwmLIMAKMYS/+gA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780908059; c=relaxed/simple; bh=vnQVlmpg+g0un4bccRkR6WRPQgTaTQW+RONtjX7TAow=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=EwhwZDdhyeYOLKrGMubQK7SVwAfxK1vaQhbwRwH/lxCVSbxUtPRZ59hGoACf2joD+JtqXEW1T35Ihjaaf1IG/4AzIU9Z1w4+e6ox8h4laHe4HIghpTy2+Taep9h1qwm58Ph93+MQVlV8rmZ1Ex8nTwUGCx/ykH0Pk1XJGyAfblA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=IDcIaJ83; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=rJsOfKXz; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="IDcIaJ83"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="rJsOfKXz" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780908057; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=P4H6i1o8NtKwI7+T7msfk1MScFiiY05veUMkpUANhzw=; b=IDcIaJ8368xWVvAknCr2GqCG7xe6Ev1IYnJ+EuM0InyxW6uAoto/CFmww3KWjliaZrYjT3 T2TyJmRSDI9OK+m9h5i0BvfSc9EcCVyhOK5aDbQPm35Ww1WHDoAyVTjB0r50+dp7m2yIEn Jy0G4lEF3xHey1F/dWownPUQJAYLmtY= Received: from mail-wr1-f71.google.com (mail-wr1-f71.google.com [209.85.221.71]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-130-Djb2YBe9NHKaj_HC4EUaxw-1; Mon, 08 Jun 2026 04:40:56 -0400 X-MC-Unique: Djb2YBe9NHKaj_HC4EUaxw-1 X-Mimecast-MFC-AGG-ID: Djb2YBe9NHKaj_HC4EUaxw_1780908055 Received: by mail-wr1-f71.google.com with SMTP id ffacd0b85a97d-46011aa5000so1700562f8f.3 for ; Mon, 08 Jun 2026 01:40:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780908055; x=1781512855; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=P4H6i1o8NtKwI7+T7msfk1MScFiiY05veUMkpUANhzw=; b=rJsOfKXzS2M4z5VaM7YrxG0kZaxelic6QF1Bd2X5/aVwekpjbJCfV0SLAbWj7tx97Y T5d3htQYslwytjGwZhxRPtQiwha2pTYuMCa2llz9ptKxURcfaYuonKjz9hNqZUYq9m/U KuMMko6iRlXACJy1/r8bTU4BV1JBTh+zgGJEDn1OFWPJ3Ya4zgopHIql23lXdb9IFiM3 MUrYFeZhVkq4+3VqQG3UnH4LOPzy4dIHEuCVPnHdJacufVsNlyjfWmLmIcPXYySrHJ50 7rYCzuZc8sJkFq2VzWnqkabS3VQ/GbEJODnUe0AsbJUXWs7e0PRE7gvZvbS/ucYSKwfZ 3mIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780908055; x=1781512855; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=P4H6i1o8NtKwI7+T7msfk1MScFiiY05veUMkpUANhzw=; b=REq6NvnPAkVplhOuxfR4jHCI1EoWDIR7zTmIw2+kryHocdHOlhFyMLGMnIeoYeylBQ mBkL+HBXpeulZiqQDj6ZBPiAxY82BK+ZU8JjF8Y5Ke+IENoey/5I5+uyVI+OMMi/9GYq e1As+mloM2e84sIda0gFaONoSk2G/ZaupiTABmuEs4jKAVs13a38Mkzkgrjr7ahdJErL ix2S4XMzOJEa7bU7WsMI1QyJ9QfkTda0QEb1yg/2M5duxMLRVsRdp1xFB8i0YDvnWAx+ AodsgELqjfJ9gubKO5IQxU3/0mmu/krKELa4X1lV7SySje4r6PVdqRsuOf7xAYaNWIjt u0pQ== X-Gm-Message-State: AOJu0YyeBqRwAC1Fh5+os/jUy1vIfK/pm058uvKk5hZxOuMWHlQa08f8 XauFICmEnBTTnmi/DjMU6FbJLUSTJrBwMC4ymWjxDvQdhOSAF8UM+37JUX1NGHXhaPnHlws3fl2 Xp+oJzEyz4AdSmB0/abgeLapdofQpHpFjM4uUwW/OoeYoNH2nCxDPmhNkligp65Tw9wHfYOBBrl 4guIPuJDjbFPCKjg9vLtEoinTIFwfjlpqn1b3/PfJV+4Q= X-Gm-Gg: Acq92OG5DiQHaY7KMvKYFKxj1sfMIKPkLS8ctrjY0dZy/FEGIO9+N+8OQLg7SEi/AjP /x3rb4PrVlam+E0HVzakD3XFUlZih9wrwmDqD0OA2O6xAQgTdWkT2lZfnjhDBRPrl9DUNrxrfy/ 1C413QD6qWLpVxI18sZ9WRyB7FBLExh12fVqNpB0z1UF3Nf15NyjE7fmmRKLI2jCA4niVBML5Gs TvGr4sg6wxsh8S2jo5ReFihuoGolYNlVjsS/V2Mf7O9B6dhyLY0KVyM5E4cAGDaU2Nf5lf5ChLE QciPlLo9aZTfAKrx6o3odX6Rm3Odq0Ju19EpvtaVmSPhzL/TcJedazTaQ4p3VM4vP6Z09y3t8qu vhXIWUwkny+H5Z1fDmycwLkc= X-Received: by 2002:a05:600c:c16f:b0:490:b8e6:be40 with SMTP id 5b1f17b1804b1-490c25fc0d5mr237785225e9.21.1780908054572; Mon, 08 Jun 2026 01:40:54 -0700 (PDT) X-Received: by 2002:a05:600c:c16f:b0:490:b8e6:be40 with SMTP id 5b1f17b1804b1-490c25fc0d5mr237784245e9.21.1780908053964; Mon, 08 Jun 2026 01:40:53 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-490bc3b5b06sm343164275e9.3.2026.06.08.01.40.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:40:53 -0700 (PDT) Date: Mon, 8 Jun 2026 04:40:45 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 36/37] mm: balloon: use put_page_zeroed for zeroed balloon pages Message-ID: <619342ca2c01c60862fe49cd4a790d171cd65ef3.1780906288.git.mst@redhat.com> References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When a balloon page marked PageZeroed is freed during migration, use put_page_zeroed() to propagate the zeroed hint to the buddy allocator. Previously the hint was silently lost via plain put_page(). No page has PageZeroed set yet; the next patch (VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE) will set it on pages the host has zeroed during inflate. Note: during balloon migration, the migration core holds an extra reference, so put_page_zeroed() will not be the final put. The zeroed hint is lost in that case, which is acceptable: it is a best-effort optimization. Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 --- mm/balloon.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/mm/balloon.c b/mm/balloon.c index 96a8f1e20bc6..6c9dd8ab0c5d 100644 --- a/mm/balloon.c +++ b/mm/balloon.c @@ -324,7 +324,15 @@ static int balloon_page_migrate(struct page *newpage, = struct page *page, balloon_page_finalize(page); spin_unlock_irqrestore(&balloon_pages_lock, flags); =20 - put_page(page); + if (PageZeroed(page)) { + /* Atomic to serialize with memory_failure's + * TestSetPageHWPoison; not under zone->lock here. + */ + ClearPageZeroed(page); + put_page_zeroed(page); + } else { + put_page(page); + } =20 return 0; } --=20 MST From nobody Mon Jun 8 10:56:14 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2D66B3B7B76 for ; Mon, 8 Jun 2026 08:41:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780908067; cv=none; b=e1rFg7UTXMWgUCSkxosAw1WIYvc+nLOIgREV4hbnyvDrmki7DoM1zL4GmOYTRkPKfn6A+7ktYwJpptOC+sbOvyDmwTctWacqmNF6AqWuDBVuWesomQmv83r+g9nkkTA6EqMNKCs8Eczzay7ESpaiNuwFblSWT8PUidQB11TYG9M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1780908067; c=relaxed/simple; bh=d4eezH+pcRDMoTex5H3SWhKAQD5cm7eZ/LyLch14vB0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=JpKZfPnRzRTSiS5ZLdx8U2VqHTw+AfI7FUKJBvcwbGX2PPkk+INm7eifaDrHGZ2rvr9cayPXIUUe0r0hFzar5rVMVXPDMQxQMeW3a9xfQQc3mAXpKzwNhr8NCUP2rK/OaihSEwNdGD8M9HLJ0AXRYIO/E8UCFKUOlzASvABf+5w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=I8EsqeaM; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=VJxDQtGd; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="I8EsqeaM"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="VJxDQtGd" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1780908065; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=4AnVGJ057Et7SfWebx9lNogVowiJVkQh33o78IbbqYo=; b=I8EsqeaM5jqdxGuhecUfYsa+g9chAiVYIM95CK9GPSals06lWyJx95EBk4aSj0Y0G6NkK9 6zPCsmbP2D1dspgoXZ76mZv/0CgveMzqEzU8v81g0gwlyHYxGCjN37M3OEFaUF3DRbHbkO GJg2JAmyAQme1wv9/r2ctYvVXhnREFY= Received: from mail-wr1-f70.google.com (mail-wr1-f70.google.com [209.85.221.70]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-534-iUBkw9RuNXG5VoRi3t_lPA-1; Mon, 08 Jun 2026 04:41:04 -0400 X-MC-Unique: iUBkw9RuNXG5VoRi3t_lPA-1 X-Mimecast-MFC-AGG-ID: iUBkw9RuNXG5VoRi3t_lPA_1780908063 Received: by mail-wr1-f70.google.com with SMTP id ffacd0b85a97d-4600c8cb13aso2428389f8f.1 for ; Mon, 08 Jun 2026 01:41:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1780908063; x=1781512863; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=4AnVGJ057Et7SfWebx9lNogVowiJVkQh33o78IbbqYo=; b=VJxDQtGdLnQa3fBcHFTZZKyQMxYFX7gAQTZEfvpoBBU5Nj+BDtanhrUX0ENMRfftXW T00NOJGg+MKom3TmeFAv+37xzZjyPqIFIW1Ng1ot1jwB8p3BtI0eV+4MliS3P2cZ20qE YWKuhR7c8o+cb43ksu5xAS6pXRqlS0rF9g1YZoJB62R+wTekK01jW6qMump0PgFtuCat /o8Mmqu3hxA3qjCYig7IpIehp9fR4iPkF1yAqZYtcgC5BIgsM2NUtpZ1arAEUfS70whp 9Ea1xO29nwGVCu/ZHAE3qVmHXlSI3AlcdmisqKIA5HacHTp5pIYj+0z7XqiXDiTgd9De Ic4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1780908063; x=1781512863; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-gg:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4AnVGJ057Et7SfWebx9lNogVowiJVkQh33o78IbbqYo=; b=g59NwRtFhHXlG4t1Ell2qXDTIpoZuEE6Yn7EIOIpszS0I8h8WaKwsetvOcin+xKaFF 9hApHncY6JlTCZVb+30E4LRwy/AALfRJzlPGJ3c2GXfl/tY9TCt1K0qbhchQ0OmnAydt oaFE87WMKlwdqJVPTCite1ECto7P1iQX8pJN8ecaJZ4A4LEW6Vbltd854+m1HepRPGvM qSE/8iJ+lvxOyPy7FjQXbJh8b/A1E8XjH3nkfGoIdpjJUAj74VXC1+Ut5i8CDDFcVzgd PuzhaEWnFr+tZQ80tKppFTKR/2+5b9Sb8GA0gHBn701h+ofnVJwtPmivBDHj3iAfRtb7 FnqA== X-Gm-Message-State: AOJu0YzcVQ+hUnPwtnSsNPzUg94EimHoQGgG4mjxPgAd8bZqjZG/DgVQ 3bhMi3pKchI1WMB8+BWEknkkZKioyUGqpE0l73uW9VKdGNAnT30C3MXXTHTzKMYFxzACEXJHOXk qMv9lbNeAXTp38VpvMDqULc75tE/EjJPvIG1ZoR4m+athsPyNhLSm8lHVeIDLNLiyPtH3AvlHdy +Q87YQJx6vKAPCXkrhbmu2Joo86MQTGszg6taOW/0AGgs= X-Gm-Gg: Acq92OFQYrQY0E9KLtS8hDfBxe1ZFcyjjXatu9EspRdHc8TKWlErftMElEUJGql0ind WMlyCAc8MK9qxsrmghSZA4KcHQ+cfk+cKe3jAcdOdawTHZ8d5XWCh5Kf5bhO+8U9rSSgukX+Xfs IOaTSKgXZ2VJjS6yb98syOaWxFpJfnxzxmOAo9HISSwE9KxIbcd7IQGv6+Nfs8ZsQwSL0CgW+w7 TY+ukRHXST3OqVVheLTuq2qxiZ1rSuWwfRYlDbH19AOuMggi+HVH6DNwfpjnGmftyIyVX9zTIVu Pq85cjVtmcr279B/I+J2s3H58wrxCrXWOqJLE9zrB0qlTn3Q9MysXRS55AGmM0D+UsGN69OyMbQ B4+aLamuAhHdvmfBjOtn2UuM= X-Received: by 2002:a05:6000:240f:b0:456:b23d:e57 with SMTP id ffacd0b85a97d-46030182732mr23389961f8f.0.1780908062722; Mon, 08 Jun 2026 01:41:02 -0700 (PDT) X-Received: by 2002:a05:6000:240f:b0:456:b23d:e57 with SMTP id ffacd0b85a97d-46030182732mr23389854f8f.0.1780908062126; Mon, 08 Jun 2026 01:41:02 -0700 (PDT) Received: from redhat.com ([31.152.37.159]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-4602cda363bsm38418630f8f.31.2026.06.08.01.40.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 08 Jun 2026 01:41:01 -0700 (PDT) Date: Mon, 8 Jun 2026 04:40:54 -0400 From: "Michael S. Tsirkin" To: linux-kernel@vger.kernel.org Cc: "David Hildenbrand (Arm)" , Jason Wang , Xuan Zhuo , Eugenio =?utf-8?B?UMOpcmV6?= , Muchun Song , Oscar Salvador , Andrew Morton , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Brendan Jackman , Johannes Weiner , Zi Yan , Baolin Wang , Nico Pache , Ryan Roberts , Dev Jain , Barry Song , Lance Yang , Hugh Dickins , Matthew Brost , Joshua Hahn , Rakie Kim , Byungchul Park , Gregory Price , Ying Huang , Alistair Popple , Christoph Lameter , David Rientjes , Roman Gushchin , Harry Yoo , Axel Rasmussen , Yuanchu Xie , Wei Xu , Chris Li , Kairui Song , Kemeng Shi , Nhat Pham , Baoquan He , virtualization@lists.linux.dev, linux-mm@kvack.org, Andrea Arcangeli Subject: [PATCH v10 37/37] virtio_balloon: implement VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE Message-ID: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Mailer: git-send-email 2.51.2.2891.g4157995a80.dirty X-Mutt-Fcc: =sent Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When the device offers DEVICE_INIT_ON_INFLATE (bit 7), the device initializes inflated pages and returns a per-page bitmap indicating which pages were successfully initialized. The driver appends a device-writable bitmap buffer to each inflate descriptor chain via virtqueue_add_sgs. After the host acknowledges, the driver checks bitmap bits (bounded by used_len) and marks pages with SetPageZeroed. tell_host() returns used_len from virtqueue_get_buf(). Bitmap reads are bounded: fill_balloon() and virtballoon_migratepage() only trust bits within the used_len range. On deflate, release_pages_balloon checks PageZeroed per page and uses put_page_zeroed for pages the host initialized, propagating the zeroed hint to the buddy allocator. If inflate_vq has fewer than 2 descriptors, probe fails with -ENOSPC. If PAGE_POISON is negotiated with non-zero poison_val, the feature is cleared in validate(). See the virtio spec change: https://lore.kernel.org/all/9c69b992c3dd83dfef3db92cd86b2fd8a0730d48.177773= 1396.git.mst@redhat.com Signed-off-by: Michael S. Tsirkin Assisted-by: Claude:claude-opus-4-6 Assisted-by: cursor-agent:GPT-5.4-xhigh --- drivers/virtio/virtio_balloon.c | 107 ++++++++++++++++++++++++---- include/linux/page-flags.h | 1 + include/uapi/linux/virtio_balloon.h | 1 + 3 files changed, 97 insertions(+), 12 deletions(-) diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloo= n.c index bf1172ad5419..0cd4fd06ca2d 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -20,6 +20,7 @@ #include #include #include +#include =20 /* * Balloon device works in 4K page units. So each page is pointed to by @@ -122,6 +123,13 @@ struct virtio_balloon { struct virtqueue *reporting_vq; struct page_reporting_dev_info pr_dev_info; =20 + /* No DMA alignment needed: ACCESS_PLATFORM is cleared, + * so virtio bypasses the DMA API. If this ever changes, + * add ____dma_from_device_aligned here. + */ + /* Bitmap returned by host for DEVICE_INIT_ON_INFLATE */ + DECLARE_BITMAP(inflate_bitmap, VIRTIO_BALLOON_ARRAY_PFNS_MAX); + /* State for keeping the wakeup_source active while adjusting the balloon= */ spinlock_t wakeup_lock; bool processing_wakeup_event; @@ -182,22 +190,33 @@ static void balloon_ack(struct virtqueue *vq) wake_up(&vb->acked); } =20 -static void tell_host(struct virtio_balloon *vb, struct virtqueue *vq) +static unsigned int tell_host(struct virtio_balloon *vb, struct virtqueue = *vq) { - struct scatterlist sg; + struct scatterlist sg_out, sg_in; + struct scatterlist *sgs[] =3D { &sg_out, &sg_in }; unsigned int len; =20 - sg_init_one(&sg, vb->pfns, sizeof(vb->pfns[0]) * vb->num_pfns); + sg_init_one(&sg_out, vb->pfns, sizeof(vb->pfns[0]) * vb->num_pfns); =20 /* We made sure the vq is large enough so we should always * be able to add one buffer to an empty queue. */ - virtqueue_add_outbuf(vq, &sg, 1, vb, GFP_KERNEL); + if (vq =3D=3D vb->inflate_vq && + virtio_has_feature(vb->vdev, + VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE)) { + unsigned int bitmap_bytes; + + bitmap_bytes =3D DIV_ROUND_UP(vb->num_pfns, 8); + bitmap_zero(vb->inflate_bitmap, vb->num_pfns); + sg_init_one(&sg_in, vb->inflate_bitmap, bitmap_bytes); + virtqueue_add_sgs(vq, sgs, 1, 1, vb, GFP_KERNEL); + } else { + virtqueue_add_outbuf(vq, &sg_out, 1, vb, GFP_KERNEL); + } virtqueue_kick(vq); =20 - /* When host has read buffer, this completes via balloon_ack */ wait_event(vb->acked, virtqueue_get_buf(vq, &len)); - + return len; } =20 static int virtballoon_free_page_report(struct page_reporting_dev_info *pr= _dev_info, @@ -300,8 +319,37 @@ static unsigned int fill_balloon(struct virtio_balloon= *vb, size_t num) =20 num_allocated_pages =3D vb->num_pfns; /* Did we get any? */ - if (vb->num_pfns !=3D 0) - tell_host(vb, vb->inflate_vq); + if (vb->num_pfns !=3D 0) { + unsigned int used_len =3D tell_host(vb, vb->inflate_vq); + + if (virtio_has_feature(vb->vdev, + VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE)) { + unsigned int i; + unsigned int valid_bits =3D used_len * 8; + + for (i =3D 0; i < vb->num_pfns; + i +=3D VIRTIO_BALLOON_PAGES_PER_PAGE) { + unsigned int pfn, j; + bool zeroed =3D true; + + if (i + VIRTIO_BALLOON_PAGES_PER_PAGE > valid_bits) + break; + for (j =3D 0; j < VIRTIO_BALLOON_PAGES_PER_PAGE; j++) { + if (!test_bit_le(i + j, vb->inflate_bitmap)) { + zeroed =3D false; + break; + } + } + if (zeroed) { + pfn =3D virtio32_to_cpu(vb->vdev, + vb->pfns[i]); + SetPageZeroed(pfn_to_page(pfn >> + (PAGE_SHIFT - + VIRTIO_BALLOON_PFN_SHIFT))); + } + } + } + } mutex_unlock(&vb->balloon_lock); =20 return num_allocated_pages; @@ -314,7 +362,12 @@ static void release_pages_balloon(struct virtio_balloo= n *vb, =20 list_for_each_entry_safe(page, next, pages, lru) { list_del(&page->lru); - put_page(page); /* balloon reference */ + if (PageZeroed(page)) { + ClearPageZeroed(page); + put_page_zeroed(page); + } else { + put_page(page); + } } } =20 @@ -861,8 +914,27 @@ static int virtballoon_migratepage(struct balloon_dev_= info *vb_dev_info, /* balloon's page migration 1st step -- inflate "newpage" */ vb->num_pfns =3D VIRTIO_BALLOON_PAGES_PER_PAGE; set_page_pfns(vb, vb->pfns, newpage); - tell_host(vb, vb->inflate_vq); + { + unsigned int used_len =3D tell_host(vb, vb->inflate_vq); =20 + if (virtio_has_feature(vb->vdev, + VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE) && + used_len >=3D DIV_ROUND_UP(VIRTIO_BALLOON_PAGES_PER_PAGE, 8)) { + unsigned int j; + bool zeroed =3D true; + + for (j =3D 0; j < VIRTIO_BALLOON_PAGES_PER_PAGE; j++) { + if (!test_bit_le(j, vb->inflate_bitmap)) { + zeroed =3D false; + break; + } + } + if (zeroed) + SetPageZeroed(newpage); + } + } + + ClearPageZeroed(page); /* balloon's page migration 2nd step -- deflate "page" */ vb->num_pfns =3D VIRTIO_BALLOON_PAGES_PER_PAGE; set_page_pfns(vb, vb->pfns, page); @@ -966,6 +1038,12 @@ static int virtballoon_probe(struct virtio_device *vd= ev) if (err) goto out_free_vb; =20 + if (virtio_has_feature(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE) && + virtqueue_get_vring_size(vb->inflate_vq) < 2) { + err =3D -ENOSPC; + goto out_del_vqs; + } + if (!virtio_has_feature(vb->vdev, VIRTIO_BALLOON_F_DEFLATE_ON_OOM)) vb->vb_dev_info.adjust_managed_page_count =3D true; #ifdef CONFIG_BALLOON_MIGRATION @@ -1191,11 +1269,15 @@ static int virtballoon_validate(struct virtio_devic= e *vdev) =20 /* Device fills with poison_val, not zeros; disable zeroed hint */ if (virtio_has_feature(vdev, VIRTIO_BALLOON_F_PAGE_POISON) && - !want_init_on_free()) + !want_init_on_free()) { __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED); + __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE); + } =20 - if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) + if (cc_platform_has(CC_ATTR_GUEST_MEM_ENCRYPT)) { __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED); + __virtio_clear_bit(vdev, VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE); + } /* * Balloon submits 1-2 sg entries max per buffer, virtqueue * sizes are 128+. Disable indirect descriptors to avoid @@ -1215,6 +1297,7 @@ static unsigned int features[] =3D { VIRTIO_BALLOON_F_PAGE_POISON, VIRTIO_BALLOON_F_REPORTING, VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED, + VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE, }; =20 static struct virtio_driver virtio_balloon_driver =3D { diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 9365d59ac1d6..caecf0fbf0e2 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -680,6 +680,7 @@ FOLIO_FLAG_FALSE(idle) * uses this to skip redundant zeroing in post_alloc_hook(). */ __PAGEFLAG(Zeroed, zeroed, PF_NO_COMPOUND) +SETPAGEFLAG(Zeroed, zeroed, PF_NO_COMPOUND) CLEARPAGEFLAG(Zeroed, zeroed, PF_NO_COMPOUND) #define __PG_ZEROED (1UL << PG_zeroed) =20 diff --git a/include/uapi/linux/virtio_balloon.h b/include/uapi/linux/virti= o_balloon.h index 13074631f300..cbaf18e0b17c 100644 --- a/include/uapi/linux/virtio_balloon.h +++ b/include/uapi/linux/virtio_balloon.h @@ -38,6 +38,7 @@ #define VIRTIO_BALLOON_F_PAGE_POISON 4 /* Guest is using page poisoning */ #define VIRTIO_BALLOON_F_REPORTING 5 /* Page reporting virtqueue */ #define VIRTIO_BALLOON_F_DEVICE_INIT_REPORTED 6 /* Device initializes repo= rted pages */ +#define VIRTIO_BALLOON_F_DEVICE_INIT_ON_INFLATE 7 /* Device initializes pa= ges on inflate */ =20 /* Size of a PFN in the balloon interface. */ #define VIRTIO_BALLOON_PFN_SHIFT 12 --=20 MST