From nobody Fri Sep 25 03:17:02 2026 Received: from mail-pj2-f12.google.com (mail-pj2-f12.google.com [74.125.227.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E96D2175A9D for ; Thu, 17 Sep 2026 05:29:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.227.140 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789622994; cv=none; b=iAKlTiuzoApzmV5osX/r9Ya8Qa2Dj7GRFB4pfoKua77vRSPHR6kktPYCBLgIKntFu6xsJTfN1yZYr3qXMna8kZwnm8ydsf5sZdZOIuy8bpBiq9eTXiUNNqburBm5/RMYrxyDwagc36RUxvpHniML6lXYM4u3ojz4/PMsyK8mGdo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789622994; c=relaxed/simple; bh=/nwSN6IrgrPDyhmpZwP1Nze8+BBC/+n6X5U6gITRzfg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=UjNnZmOlWesl8znz3ANXj/xUKT5/pB8tHVPbAa0eFirnSehGqmwGh3eNYyBhfGTJ4XtCUBkoS1K7dly2u3zdlHAmjlhxE1nPadwk53g5xWAQ/8W81sa9+o015C9/PlVtkyKDU+gPs8ouTUsDanmvcfl0kWweExtEnVmh5EgWGyk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=QZn6jqX6; arc=none smtp.client-ip=74.125.227.140 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QZn6jqX6" Received: by mail-pj2-f12.google.com with SMTP id 98e67ed59e1d1-39d654f02baso348790a91.3 for ; Wed, 16 Sep 2026 22:29:52 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1789622992; x=1790227792; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=aMxiXZ3VjT6yairvAxeBbjdHvNVDdfswnxYz1hMIZxE=; b=QZn6jqX674K6IjgxmSQAyAq9f9KhX+xp9FyBfNY7s3vnCO4z3WoK3RnZ7XoDWUXaLt Ap8WnY2dMYI0UxH5yRWKWeH9+Xmj26CXkU33tHSvR5F/QjMfTaPIaUuxIBkVxszj4uxU w1+e+xynoZUMVDxfpC7p3oiKa0xCiGaAnLZu6H6HhKynqljZQT7b1UEStxFxqxMyKfET yv+FSPHiQbDHlxvzLYMQ0wlWVUFMXDBqrdT6IYmzG6+wFcuh7AZ+/y5U+uTdoYyqlsNH kQSdB+K46PrWE4aW4dYl0uyEi0vkFViBd91tn8k0cuKgY7fgyyG/nLw8jl09/r5SbAZb 1kLw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1789622992; x=1790227792; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=aMxiXZ3VjT6yairvAxeBbjdHvNVDdfswnxYz1hMIZxE=; b=u+iBhVRGClLPEfNTudGTEzGlSw2TSOaUio2cMO9Xo2wRj2i+Omnjpb9xGm3KYWzIX/ prAX9VphpPN2nOatEtZq0b221/3FTtpw8+uaHtsbnGPBBgZ2mfoiGim6Y22OmDQA8VtY 6CS9RqJtV6cJwmFP0pHfD4RTk3qwTT2lTth5L05Gwjc433I2Yegl/njc3BaAVdIwOV9a QvaTv5A43bz03hCKsvGEIZgFlr7gAYrEsyv5cpYZZIPsQeWZTZVq+JJQ49Kw7HJPmjHR c5npKpvTp4ocG830a/sqN3C3yqwV/RvzbBrc4Xh+OFr+F0EDUqqU3PhKZSyGxRN143sf hn/Q== X-Forwarded-Encrypted: i=1; AKwUvBxXcOFtd/J1DN8dtVtK5rxrtWYM5gzI105tqE1a6g7D5S3bmhLEEbRvDKlqxXoPHrJUipsmSSU6J4ZVhbE=@vger.kernel.org X-Gm-Message-State: AFuF++nnD8ZEVWbBzPP51fZpQvf3KZkA9e+HAPl6WvO1Rz4MddD62fy2 8OVb1FgJuCM3YiRTd5/PBI37H3kNKb1dJQyhOEybTN+mJg7aDVyFO9GN X-Gm-Gg: AYBFou1kOqP/u4TwT5xCKWdrUAyPj89nMQs4asVndsKwxwwFgGiqUnrALiURbnZ+knk A6OoeV1EuUeGLClLM2OnUjwVN/L1uVoFxmvH2PaL4osQL7db3qOe8gMtN7KCSVEPHiPqCIH+rkX IN3Em1xR5qOyEDFiSSEwH/q2ZswnqhijpPdc1OeO8T203rdOpoB9Tl1VuENgaWDA9kpihq5nfGh xkhSnkekuJYvD+k+FtfcJwxF+NkM0fRkYMt3otjy+u4RoapXlrlmcVj06WsKhY2FrkekVoEr4m8 miihZC2jVTwwgxIShNi3BL0SwqdbkRzvLgWeTjT/eDicbw+CFO99fQ8tC0YJwVKbNGurB3jPs/e 35vHtgIGBpsFj2LvEGIgRtw6DxerA8dPKYgybNrjm2Vpob64ovan9vkok+AVwTcSoS/lmxTUP7Y 7FtOzpHNqrHGJQaF1oWaqCXHQkxqGTlXOr+ezKsxVUTr3FrthEApUdE0zwbl61vPCowsKk/mDej 2EpFH3yB4ahP3VYVNeiWajkX8w= X-Received: by 2002:a17:90b:3b91:b0:39d:ec43:3cc3 with SMTP id 98e67ed59e1d1-39e1e2f3e9fmr11172449a91.10.1789622992190; Wed, 16 Sep 2026 22:29:52 -0700 (PDT) Received: from mi-OptiPlex-7060.mioffice.cn ([43.224.245.234]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-39e3abada87sm1955836a91.14.2026.09.16.22.29.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Sep 2026 22:29:51 -0700 (PDT) From: Wen Jiang To: akpm@linux-foundation.org, catalin.marinas@arm.com, linux-mm@kvack.org, urezki@gmail.com, will@kernel.org Cc: Xueyuan.chen21@gmail.com, ajd@linux.ibm.com, anshuman.khandual@arm.com, baohua@kernel.org, chleroy@kernel.org, david@kernel.org, dev.jain@arm.com, jiangwen6@xiaomi.com, leo.yan@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, rppt@kernel.org, ryan.roberts@arm.com Subject: [PATCH v8 01/10] arm64/mm: add pte_set_huge() and pte_clear_huge() Date: Thu, 17 Sep 2026 13:29:24 +0800 Message-Id: <20260917052933.188679-2-jiangwenxiaomi@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260917052933.188679-1-jiangwenxiaomi@gmail.com> References: <20260917052933.188679-1-jiangwenxiaomi@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Wen Jiang vmap installs PTE-level block mappings (PTE_CONT on arm64) by reusing set_huge_pte_at() and huge_ptep_get_and_clear(), which are HugeTLB helpers gated by CONFIG_HUGETLB_PAGE. This makes the feature silently unavailable on CONFIG_HUGETLB_PAGE=3Dn kernels and couples mm/vmalloc.c to HugeTLB internals it does not otherwise need. Add pte_set_huge()/pte_clear_huge() to arm64, joining the existing pmd_set_huge()/pud_set_huge() family in mm/mmu.c. They build on __set_ptes() and __get_and_clear_full_ptes() directly, so they do not depend on CONFIG_HUGETLB_PAGE. There is no caller yet: mm/vmalloc.c is converted later in this series, once the generic fallbacks are in place. Signed-off-by: Wen Jiang --- arch/arm64/include/asm/pgtable.h | 6 ++++++ arch/arm64/mm/mmu.c | 19 +++++++++++++++++++ 2 files changed, 25 insertions(+) diff --git a/arch/arm64/include/asm/pgtable.h b/arch/arm64/include/asm/pgta= ble.h index 4dfa42b7d0535..6ee1522fac78f 100644 --- a/arch/arm64/include/asm/pgtable.h +++ b/arch/arm64/include/asm/pgtable.h @@ -1949,6 +1949,12 @@ static inline void clear_young_dirty_ptes(struct vm_= area_struct *vma, =20 #endif /* CONFIG_ARM64_CONTPTE */ =20 +#define __HAVE_ARCH_PTE_SET_HUGE +void pte_set_huge(pte_t *ptep, unsigned long addr, phys_addr_t phys, + pgprot_t prot, unsigned long size); +#define __HAVE_ARCH_PTE_CLEAR_HUGE +pte_t pte_clear_huge(pte_t *ptep, unsigned long addr, unsigned long size); + #endif /* !__ASSEMBLER__ */ =20 #endif /* __ASM_PGTABLE_H */ diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 92fedf4db9278..93b602a6c68a6 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1872,6 +1872,25 @@ int pmd_clear_huge(pmd_t *pmdp) return 1; } =20 +void pte_set_huge(pte_t *ptep, unsigned long addr, phys_addr_t phys, + pgprot_t prot, unsigned long size) +{ + unsigned long pfn =3D __phys_to_pfn(phys); + pte_t new_pte =3D pte_mkcont(pfn_pte(pfn, prot)); + unsigned int nr =3D size >> PAGE_SHIFT; + + VM_WARN_ON(!IS_ALIGNED(pfn, CONT_PTES)); + VM_WARN_ON(!IS_ALIGNED(size, CONT_PTE_SIZE)); + + __set_ptes(&init_mm, addr, ptep, new_pte, nr); +} + +pte_t pte_clear_huge(pte_t *ptep, unsigned long addr, unsigned long size) +{ + return __get_and_clear_full_ptes(&init_mm, addr, ptep, + size >> PAGE_SHIFT, 0); +} + static int __pmd_free_pte_page(pmd_t *pmdp, unsigned long addr, bool acquire_mmap_lock) { --=20 2.34.1 From nobody Fri Sep 25 03:17:02 2026 Received: from mail-pz2-f42.google.com (mail-pz2-f42.google.com [74.125.228.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EF0EC3A451D for ; Thu, 17 Sep 2026 05:29:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.228.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789623000; cv=none; b=ZVCp1dbym3E15mjK2jE9O1QzC1rBlQkAsWzVbCWpeGzloljaNf9JYHdbFAJHklHfMgeqltNclMqCQS9Z7dm5ltGOhhLjGoPFv5bMzL26vMnmx3JaK8CEj299tGqierhmc45ix9hqi9+A8guW/zD9vwJfPY6/L9ohZZFvRHZHM5A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789623000; c=relaxed/simple; bh=hQjwjA5LN+z5+t0OFB7/5zJnx8tmmFqbQZ0r7K29ATQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=iyETkbUdbv1/eFTFDHrxdOc/Eo1VAI+3aKO8Bf1aTPJa0/HHZ1BM5q07gAG/416bTFk6Q7hA4Mfoe4+VvacIj8PfNIOaGImRwxe8IazSG1MsmwM/iDtCNhahODZJX4lvOkXZwav6InTwg66GU1CGG+szfKaVYXPYwogJEzOSTqA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=iYg/Du7O; arc=none smtp.client-ip=74.125.228.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="iYg/Du7O" Received: by mail-pz2-f42.google.com with SMTP id 41be03b00d2f7-cc4d2fe2056so243754a12.2 for ; Wed, 16 Sep 2026 22:29:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1789622998; x=1790227798; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=Qf2/Psgvj1EaockCLCoqTAU5YfPvsk9Hzwk+BcXGZSE=; b=iYg/Du7OOSA9Te/Isndc4W9+3ELj+aoPZB6QCudY0RoV4QJEKs0CVLP6jWeEombrlm AZcOGim7u6RiCZhAzkorZ2a+z2xWlMS7Qj0NWmaU7AY7X7TFodglTYoqD10OkWx/i9PZ FjAcSFVk4MDFFr5OhuT13UfZ3dPEtlSMwKDvkHMjptZuj8J7I7Uk6A91DiV9Pmq6tskF OEMOcvnJAba63a0ZaYi6v6iA3TzfiK87IfiM738ZoyhDD2rniyJ1IqcTe9TBzhHnS1AK cWgrOwWZBcp8xR/lRw0ZY83Fcl8Jlv7ypcVp/R8vSZMPxiBxcpnR4pmRmwBHOP2JGv2e pUvw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1789622998; x=1790227798; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=Qf2/Psgvj1EaockCLCoqTAU5YfPvsk9Hzwk+BcXGZSE=; b=oQt9+289+szeteNKGmrHACYyChIJlnpuGuYp812YvtH6v4mrfcdvN0fWe2Ra8f1AVm WK6TOEx8AkomRajSS7EXD4YkPrvIdcPi8X7UbddP/slBLmHlkLesiWZtlg8FFveTtP+1 6g9QJYvxF0QUR7j0itxdO0gVJpSzIvUCrudfXW0Y/wBBfyZuC7VO/sEIiGi4tjXmKOyj Krd0ooLuujZmP6B9ewBAVAsnW7sUMtFoPSCnr89YWjCTAM7qqs1Mmw/wOFLpu3ZSzc/Q EskARjQ7G8G2+geYY0ywnjszV9ZM5hdmIrslRIAQK29vWLoIi1Mt01NuqZLZUkvp0A/e dFpQ== X-Forwarded-Encrypted: i=1; AKwUvBwwJH1Pfin7vp2lmkdSHzqq0vMkk6hl0lxZ2RscF7cz4Vi5YjDVShuneuZS7kAwSOl7BNrtCP/EqGZX+Bc=@vger.kernel.org X-Gm-Message-State: AFuF++n2UqX9byppJYfhfDKVYC/LSSRX8+P7h/l+/U4ZGbaEYtaC90JO aaC+psyjLMaJd0kq1/p3n98yoq7fPry/LPZVzqFNdTFueGLTGQ50X0PL X-Gm-Gg: AYBFou02rkmIALpzt1MFuUH8HcgqKEU6ieh5pU7PbOqIZhMmz+xd+G2Jgkv/W2wqtMO CJ4Kt86dg8bBDA3UREqZ5cHSeqXcsV6MSNMlb6F2f5Ay4r2LJ67lv0Exs53hcfYLbxpaSCr2JLS ztHy5IwGpAfoJDLNlujyJ9hSwFEXWOZo6nJdzCEm9qF+cQYDemnKCnuiFWk694bGsqOIyyaWWD+ VimSVMMn9ZtNDtcDeFJexSU7tKQH0IesIzTUwXzRxUBeDSaXFSadqHSsBXIU8ZzgP9fwyJF4B5t Z8Djf6zpNuC1Bk2cx2VwQXaTqv+6KWQ4GqadS0zTVCgfvDW+Nvcc2fVpOkQWWCGD13c1GI3beLb 5a5vHu8Gv33iuyflCDkDACl8HX2OWQgmCMoi03TXzgpNKSHfK9X9Qmd3XIbtgjVBVzHeVgbjD7m OYLzypMLxVnXaaQHjlYZAimnu1hnaJyP6LAA4BCJoVpcBlyLAFAG+/l4mDUo19yXRGX3g2tcJQS ygkestUjSpaRvsz4DKnxg/35hU= X-Received: by 2002:a17:90b:53c3:b0:398:c0ad:711c with SMTP id 98e67ed59e1d1-39e1e48a517mr11683515a91.15.1789622998238; Wed, 16 Sep 2026 22:29:58 -0700 (PDT) Received: from mi-OptiPlex-7060.mioffice.cn ([43.224.245.234]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-39e3abada87sm1955836a91.14.2026.09.16.22.29.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Sep 2026 22:29:57 -0700 (PDT) From: Wen Jiang To: akpm@linux-foundation.org, catalin.marinas@arm.com, linux-mm@kvack.org, urezki@gmail.com, will@kernel.org Cc: Xueyuan.chen21@gmail.com, ajd@linux.ibm.com, anshuman.khandual@arm.com, baohua@kernel.org, chleroy@kernel.org, david@kernel.org, dev.jain@arm.com, jiangwen6@xiaomi.com, leo.yan@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, rppt@kernel.org, ryan.roberts@arm.com Subject: [PATCH v8 02/10] powerpc/8xx: add pte_set_huge() Date: Thu, 17 Sep 2026 13:29:25 +0800 Message-Id: <20260917052933.188679-3-jiangwenxiaomi@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260917052933.188679-1-jiangwenxiaomi@gmail.com> References: <20260917052933.188679-1-jiangwenxiaomi@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Wen Jiang vmap installs PTE-level block mappings (SZ_16K/SZ_512K on powerpc/8xx) by reusing set_huge_pte_at(), a HugeTLB helper gated by CONFIG_HUGETLB_PAGE. This makes the feature silently unavailable on CONFIG_HUGETLB_PAGE=3Dn kernels and couples mm/vmalloc.c to HugeTLB internals it does not otherwise need. Add pte_set_huge() to powerpc/8xx, next to the existing pmd_clear_huge()/pud_clear_huge() in mm/nohash/8xx.c. It builds the huge PTE and writes it into the backing cells directly, without going through set_huge_pte_at(), so it does not depend on CONFIG_HUGETLB_PAGE. No pte_clear_huge() is needed: 8xx does not implement arch_vmap_pte_range_unmap_size(), so the vmap unmap path never takes the block-mapping branch. In practice PPC_8xx selects HUGETLBFS unconditionally, so CONFIG_HUGETLB_PAGE=3Dn does not occur there today; this is a decoupling cleanup rather than a new configuration. There is no caller yet: mm/vmalloc.c is converted later in this series, once the generic fallbacks are in place. Signed-off-by: Wen Jiang Reviewed-by: Christophe Leroy (CS GROUP) --- arch/powerpc/include/asm/nohash/32/pte-8xx.h | 4 +++ arch/powerpc/mm/nohash/8xx.c | 29 ++++++++++++++++++++ 2 files changed, 33 insertions(+) diff --git a/arch/powerpc/include/asm/nohash/32/pte-8xx.h b/arch/powerpc/in= clude/asm/nohash/32/pte-8xx.h index e2ea8ba9f8cae..1ee3f8f0de974 100644 --- a/arch/powerpc/include/asm/nohash/32/pte-8xx.h +++ b/arch/powerpc/include/asm/nohash/32/pte-8xx.h @@ -235,6 +235,10 @@ static inline pte_t ptep_get(pte_t *ptep) } #endif /* CONFIG_PPC_16K_PAGES */ =20 +#define __HAVE_ARCH_PTE_SET_HUGE +void pte_set_huge(pte_t *ptep, unsigned long addr, phys_addr_t phys, + pgprot_t prot, unsigned long size); + #endif =20 #endif /* __KERNEL__ */ diff --git a/arch/powerpc/mm/nohash/8xx.c b/arch/powerpc/mm/nohash/8xx.c index a9d3f4729eada..2518d2a6bdb41 100644 --- a/arch/powerpc/mm/nohash/8xx.c +++ b/arch/powerpc/mm/nohash/8xx.c @@ -220,3 +220,32 @@ int pmd_clear_huge(pmd_t *pmd) { return 0; } + +/* + * vmap PTE-level block mapping. Sets a present kernel mapping directly, + * so unlike set_huge_pte_at() it needs neither CONFIG_HUGETLB_PAGE nor + * set_pte_filter() (a no-op for non-exec kernel mappings). + */ +void pte_set_huge(pte_t *ptep, unsigned long addr, phys_addr_t phys, + pgprot_t prot, unsigned long size) +{ + pmd_t *pmdp =3D pmd_off(&init_mm, addr); + pte_t pte =3D pfn_pte(PHYS_PFN(phys), prot); + pte_basic_t val; + pte_basic_t *entry =3D (pte_basic_t *)ptep; + int num, i; + + pte =3D arch_make_huge_pte(pte, ilog2(size), 0); + val =3D pte_val(pte); + + /* + * Make sure hardware valid bit is not set. We don't do + * tlb flush for this update. + */ + VM_WARN_ON(pte_hw_valid(*ptep) && !pte_protnone(*ptep)); + + num =3D number_of_cells_per_pte(pmdp, val, 1); + + for (i =3D 0; i < num; i++, entry++, val +=3D SZ_4K) + *entry =3D val; +} --=20 2.34.1 From nobody Fri Sep 25 03:17:02 2026 Received: from mail-pj2-f13.google.com (mail-pj2-f13.google.com [74.125.227.141]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 069553FC5AE for ; Thu, 17 Sep 2026 05:30:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.227.141 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789623006; cv=none; b=KXigmdjzTJ7DG/pnM6Tja8ODnrHplI2Tzs0X7lkMyqh0z+cJ6VjxE+2ozpa0JzCNGqY4DeRy4Tvbn/DQ6Rnrc9WH9Sp0l1wIIYXfSu+t986Aa4PgIse4dmirWvXnOzC3hLlVOHYu4aIG8Ut2cfEOqv0IaABVa4bCNuA5yPp+2fs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789623006; c=relaxed/simple; bh=FqU9mUdoHISvILuaEFO8CBOPURlTl+emi5lOdPZslEE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Q3LJgizVYFjE+DJWQ2Zeec5CGv4B61dkm/zUKIK/hzbSecN3AwPVSP4g/WToHHSg6bZGr/wxyiukAbbbBkdcJ4jVBTMtmau170rQCSmxL+iqhB+KiYL3ztQvX9p6ZBnQpAERb1qb1ei3rhj6WHocaksDFx2lfWURzD4F9JFEw4U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=UniWIrpX; arc=none smtp.client-ip=74.125.227.141 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="UniWIrpX" Received: by mail-pj2-f13.google.com with SMTP id 98e67ed59e1d1-396ccd78e6eso146878a91.0 for ; Wed, 16 Sep 2026 22:30:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1789623004; x=1790227804; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=/iHoeiOn8M1Js0lz6hY05jc0JjGSYBNXCmk3v6q8ZOE=; b=UniWIrpXeSOyqGqapZUcH8UxfxtJWvLAUQUZ/al1VIj8e5vd7rAkcPZWn5g2wNu4zq PoSsigtUKE2X2YhGoAZriHCHcDtnR0Ru/a158E+pgSkSG68XrC8986IjOl2IwkQexBYy 4Zv7f+flA/q0Z4UjXsco9WZdbTpXWzkWtMLRddb521ZOoRmVmTY4Zaf2cQCF/thgGtEv ad/Cjn5d0KOlh2eiM55LA4aPHDIBf8UNOe39tmRDt7Otwp/lFv4pDmqe9dxgmzTjrLJk I6NdKHTTRe+0fZRJu8jGSPuDBijNpFaTEI+BhQNBMdNb9c+rzN31//lTeWEdyfKJIha9 GaRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1789623004; x=1790227804; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=/iHoeiOn8M1Js0lz6hY05jc0JjGSYBNXCmk3v6q8ZOE=; b=ip9BYqkgP0WvOOPbzHmovOJAyAhVjAEXoV3ptnb0LAu8OVDrUik2R9Ot+RqVkcmtJd /ySii8IlbToVPJ9bE5EqFmbzHfsyNL5NBfOu2pUnNbBnPR3W1eRm53c6KBq+eW76qbLQ rU2a/3YHDBwPwGGuY9a351dwkKksOoqvYDA96fDTX1uweGA0WbbXWJ3DJVfCLVbZZQ+y Bqj2WXsapCBT9k0XSdv1k1Thyv3ub5CU34Tymq5wWL75ysi0QFjKmgPS6ajH3YWEuJMs LWxcXg3VKgSHzo2igMRst5S/XlMRD8g8vzzDONwjsyf2JUU/CWBMLo4LSqot6rTkmKD5 ezog== X-Forwarded-Encrypted: i=1; AKwUvBxLxqzGHBP/3QiE1RlX/PhZhwXChO/4nbIjqv543O9e7z9nQ0FJ2vmyIkYgJDu4BUo4BJNhAG+y2/ijFIk=@vger.kernel.org X-Gm-Message-State: AFuF++ndyPw7//4HHVUxH4fmzXCwq8PfFMEsUmjIaeGcf4BMhUY29pFq z+JhUEDxplMeJKVQbz/rwHmeIDk4ZYh2HhGlyj+jBsmV7DpHHsAY2iZd X-Gm-Gg: AYBFou3XyODzCDhsUw0WdUcLhyOG35BpUWirDjXd47JRRr64rRZ0r/G77pz9Et/UbtD WYeRMoOVl+mPGxY3ScPUquycdz7O7J7PgUkThpdRNcsB100D5iM8U9Vj2F0vSOvAEIgKZNvwRRN 4f0BVmAbn3+1hwVPCZnvLZPDkovmHG7UWhssyyiGXbkv29JDL0DioR/MvZIVVEaZM9ebRmvzt2Q t7umTMKrGzQNmRwzMk03RM186qCzsQNB7otiaEwwM0kRM+6Vcj0s4I0GkC9POYe+puZXanfQiHY 6fHQ4bc+ktC1rLc72wZkXWLsCXvLIniUYX09pOqy9urlT7zCku8xv1uouK9DbqaegvPvFgeVkQ+ sWaUq/BiUc7k3XGHr1BVjtkJxabdawyYAoL/0R1ytUvoi0dDf4hEepRXAQ0/rgdeZvqTPNbZFM0 L7kaFhKsP0C2hCnubXlMWLco8XXxfTTHuaTNCcnYw7aUhmPqrh8J48PAl6BtG7EPbByNYVuCYEr +Cu0cdfvI2o5B+UZ5ZdGdGeAOQ= X-Received: by 2002:a17:90b:5484:b0:39d:b57e:b47d with SMTP id 98e67ed59e1d1-39e35f809c5mr2777472a91.8.1789623004300; Wed, 16 Sep 2026 22:30:04 -0700 (PDT) Received: from mi-OptiPlex-7060.mioffice.cn ([43.224.245.234]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-39e3abada87sm1955836a91.14.2026.09.16.22.29.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Sep 2026 22:30:03 -0700 (PDT) From: Wen Jiang To: akpm@linux-foundation.org, catalin.marinas@arm.com, linux-mm@kvack.org, urezki@gmail.com, will@kernel.org Cc: Xueyuan.chen21@gmail.com, ajd@linux.ibm.com, anshuman.khandual@arm.com, baohua@kernel.org, chleroy@kernel.org, david@kernel.org, dev.jain@arm.com, jiangwen6@xiaomi.com, leo.yan@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, rppt@kernel.org, ryan.roberts@arm.com Subject: [PATCH v8 03/10] mm/vmalloc: use pte_set_huge()/pte_clear_huge() for PTE-level block mappings Date: Thu, 17 Sep 2026 13:29:26 +0800 Message-Id: <20260917052933.188679-4-jiangwenxiaomi@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260917052933.188679-1-jiangwenxiaomi@gmail.com> References: <20260917052933.188679-1-jiangwenxiaomi@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Wen Jiang vmap installs PTE-level block mappings by reusing set_huge_pte_at() and huge_ptep_get_and_clear() under #ifdef CONFIG_HUGETLB_PAGE. This makes the feature silently unavailable on CONFIG_HUGETLB_PAGE=3Dn kernels and couples mm/vmalloc.c to HugeTLB internals it does not otherwise need. Now that arm64 and powerpc/8xx provide pte_set_huge() and pte_clear_huge(), add the generic fallbacks next to the existing pmd/pud_set_huge() family and convert vmap_pte_range() and vunmap_pte_range() to the new helpers. The CONFIG_HUGETLB_PAGE guards around the block-mapping paths are dropped, so PTE-level block mappings now also work on CONFIG_HUGETLB_PAGE=3Dn kernels, and mm/vmalloc.c no longer includes . The fallbacks exist only to keep the build working on architectures without PTE-level block mapping support. They are unreachable there: the callers only run when arch_vmap_pte_range_map_size() or arch_vmap_pte_range_unmap_size() return a size other than PAGE_SIZE, which requires an arch implementation. WARN_ON_ONCE() makes that explicit rather than silently doing nothing. Signed-off-by: Wen Jiang --- include/linux/pgtable.h | 29 +++++++++++++++++++++++++++++ mm/vmalloc.c | 19 ++++++------------- 2 files changed, 35 insertions(+), 13 deletions(-) diff --git a/include/linux/pgtable.h b/include/linux/pgtable.h index cdd68ed3ae1a9..349ced999f959 100644 --- a/include/linux/pgtable.h +++ b/include/linux/pgtable.h @@ -2134,6 +2134,35 @@ static inline int pmd_free_pte_page(pmd_t *pmd, unsi= gned long addr) } #endif /* CONFIG_HAVE_ARCH_HUGE_VMAP */ =20 +/* + * PTE-level block mappings for vmap. + * + * pte_set_huge() only has to be implemented by architectures whose + * arch_vmap_pte_range_map_size() can return a size other than PAGE_SIZE. + */ +#ifndef __HAVE_ARCH_PTE_SET_HUGE +static inline void pte_set_huge(pte_t *ptep, unsigned long addr, + phys_addr_t phys, pgprot_t prot, + unsigned long size) +{ + WARN_ON_ONCE(1); +} +#endif + +/* + * Likewise, pte_clear_huge() only has to be implemented by architectures + * whose arch_vmap_pte_range_unmap_size() can return a size other than + * PAGE_SIZE. + */ +#ifndef __HAVE_ARCH_PTE_CLEAR_HUGE +static inline pte_t pte_clear_huge(pte_t *ptep, unsigned long addr, + unsigned long size) +{ + WARN_ON_ONCE(1); + return __pte(0); +} +#endif + #ifndef __HAVE_ARCH_FLUSH_PMD_TLB_RANGE #ifdef CONFIG_TRANSPARENT_HUGEPAGE /* diff --git a/mm/vmalloc.c b/mm/vmalloc.c index bb6ae08d18f58..025b3a848d564 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -38,7 +38,6 @@ #include #include #include -#include #include #include #include @@ -98,7 +97,7 @@ static int vmap_pte_range(pmd_t *pmd, unsigned long addr,= unsigned long end, pte_t *pte; u64 pfn; struct page *page; - unsigned long size =3D PAGE_SIZE; + unsigned long size; =20 if (WARN_ON_ONCE(!PAGE_ALIGNED(end - addr))) return -EINVAL; @@ -119,17 +118,12 @@ static int vmap_pte_range(pmd_t *pmd, unsigned long a= ddr, unsigned long end, BUG(); } =20 -#ifdef CONFIG_HUGETLB_PAGE size =3D arch_vmap_pte_range_map_size(addr, end, pfn, max_page_shift); if (size !=3D PAGE_SIZE) { - pte_t entry =3D pfn_pte(pfn, prot); - - entry =3D arch_make_huge_pte(entry, ilog2(size), 0); - set_huge_pte_at(&init_mm, addr, pte, entry, size); + pte_set_huge(pte, addr, PFN_PHYS(pfn), prot, size); pfn +=3D PFN_DOWN(size); continue; } -#endif set_pte_at(&init_mm, addr, pte, pfn_pte(pfn, prot)); pfn++; } while (pte +=3D PFN_DOWN(size), addr +=3D size, addr !=3D end); @@ -368,25 +362,24 @@ static void vunmap_pte_range(pmd_t *pmd, unsigned lon= g addr, unsigned long end, { pte_t *pte; pte_t ptent; - unsigned long size =3D PAGE_SIZE; + unsigned long size; =20 pte =3D pte_offset_kernel(pmd, addr); lazy_mmu_mode_enable(); =20 do { -#ifdef CONFIG_HUGETLB_PAGE size =3D arch_vmap_pte_range_unmap_size(addr, pte); if (size !=3D PAGE_SIZE) { if (WARN_ON(!IS_ALIGNED(addr, size))) { addr =3D ALIGN_DOWN(addr, size); pte =3D PTR_ALIGN_DOWN(pte, sizeof(*pte) * (size >> PAGE_SHIFT)); } - ptent =3D huge_ptep_get_and_clear(&init_mm, addr, pte, size); + ptent =3D pte_clear_huge(pte, addr, size); if (WARN_ON(end - addr < size)) size =3D end - addr; - } else -#endif + } else { ptent =3D ptep_get_and_clear(&init_mm, addr, pte); + } WARN_ON(!pte_none(ptent) && !pte_present(ptent)); } while (pte +=3D (size >> PAGE_SHIFT), addr +=3D size, addr !=3D end); =20 --=20 2.34.1 From nobody Fri Sep 25 03:17:02 2026 Received: from mail-pj2-f13.google.com (mail-pj2-f13.google.com [74.125.227.141]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 35AFA37DAAE for ; Thu, 17 Sep 2026 05:30:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.227.141 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789623013; cv=none; b=fvQQ21jj/xe8WguFntkwNWIngcnlU58zXS3GOF1jT2o+S9tK8RSH3+JwtRvBOeRDmj5u6wUV9ew4aifXKBhdcFMyyEVWJH7KUUDTi4Uy9H0XrJ4IGucdnlBgbhOzAp/MoR+uRsjlY2oSWa4sBHGTnFGdxkIJ98cgzoRx9mivOTs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789623013; c=relaxed/simple; bh=yo3vtO2fmEY1otZDT0RqPL7Bfe1xvhyeqH2fwkoy46k=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=EXg1SSRDugom8n1VRG4FWVB7ZzhANJc4YeQRRK7bjp4kbkpQAqJ8UkuhuYpgqjrL/vNXt0DWN0r9Dku9R06hUtLzeS8kKy4Yg0KVJG3uLzvbVPBlaGUjhnwLBCdHvytlvzlktQDij/GWkz0gcGacjls2nImyc2djNdhNh5V/tM4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=aSO9T9Co; arc=none smtp.client-ip=74.125.227.141 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="aSO9T9Co" Received: by mail-pj2-f13.google.com with SMTP id d9443c01a7336-2db22383fe8so2717395ad.2 for ; Wed, 16 Sep 2026 22:30:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1789623011; x=1790227811; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=ynKYarepbWl9Bt2lXEeL1/lRyqq80+H1joWy3Adf4T8=; b=aSO9T9Co6ZbsnkB64IOmcx01OupH0wWgmSzphYZzQi4LQKFxL3Tu/6M9u21v7i7Ept ZJdnW5xdi/ioDqRNxwG7ZBonKDjQBJk/lx5YJwSv1A7sLpDBC3rwMq43p7mk0N+0R0IX TVUyKnxJfVRf4dsLqE0E6AV5YJ2w+Mg9r10H6JxZWR1bF4jnGvizKHjlCSwp+AZ96pe0 3foRKF6NtTnx8w75KofxvgXPx44B1NLGBPMNKbjmovC77g1l1hmGn4dodHdLBccdEN6P eh5+ghOg6T8A5EoZaqZro6dlj03eTMQ0hxv6mNI3eXC5MFwuBBQeQ1goSl6AOVt1uxmB kqyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1789623011; x=1790227811; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=ynKYarepbWl9Bt2lXEeL1/lRyqq80+H1joWy3Adf4T8=; b=c6+U4KYYIqQmsznZAHTJXY2Unw/qAmNRn7qpNcmfGnKSci024vB7BmIcFVvIYHUN0O sJAUJkTX2lD0bGTiua9AsrAgof824OA38ZCLgo1yrBi59VvL1WjDHsReC5io9KDSGnsn miM8YAXPb8TgllHyUHQxmdTyv/mKe1aCfGjemrhJNXNe9Egcrf8GFLCB6yAYRCh7EufZ +HmRdMuos8J6BG9HF3HgCFlZsNLMYJbleNhFbUeNcGykFB/JwoEXevbpANem7xJsXQy6 WFp9OOLekur/Lu58qReYGpHkUAIUvxZFD+Rt9D5uofatlCfY0BP0VqgeiB2eHF9i1UKW IJqg== X-Forwarded-Encrypted: i=1; AKwUvBxZfDOESJO7xMobIqCkElA1evlo6raRkx37Wo6nuwszeGMvWTRpBOd3I9jHI1n3NwLArzwVaNDTjHa4x04=@vger.kernel.org X-Gm-Message-State: AFuF++kZhiqUWYCzNVhtNwU36HcqskzjmJraaPGaHWB6owfssXlUVITc JJDqAoL+LAtN0W62qjXVxQvjLPrY95PDnMEIH2SLvk2CL9PnTDclW+XS X-Gm-Gg: AYBFou2DkUsOwnVlxVPnkW7oJ+MY7N218QY+VC79bmY/Rdh3xwVNy81IifXj8hushBu 5nbIRe8RHPXo1ISc9SQaMSkpDE+fzzdzVBQWGiTAycNgfA5n8uCvhA9G/CQLCxsrqXVKDjtuNkg scKmNYhS083RiJeQHy4fr37OziazyQq24R85rlMhbpOazpoLpHFn0kLWqxtwWNlgSjXmSB/ABpx X+YVogUhwoyBdrR+Q8CxWKR5GAv9/omtUxGj20ZoczV0ZouTMY61/skOF4lmKGDS9E/H9oqK62E D7UrPBDppon+vWAQCloToRIdon+EhwV2VXhqa1AnR6Epq9/CgS6GRWeu5hl99SXftkAVOJwuxmy 4Pad6tKJKH02iewVn7ttmEwoW1IR8JkyAlGygsd6+DRPc/Y/5FCHHyyQgqXQNgTTvz/h1mB7Xqb QdnKGoZMVIsfT74yhgNM9mjl6JCbe44xOP8csLXYVZev9vcXB88AQh8tteCGrkUR0yNkSwaU++e oNHGh/s1shfvezpTGszf5oGJgQ= X-Received: by 2002:a17:90b:2803:b0:39d:ef3e:9035 with SMTP id 98e67ed59e1d1-39e1e324463mr17718864a91.10.1789623011402; Wed, 16 Sep 2026 22:30:11 -0700 (PDT) Received: from mi-OptiPlex-7060.mioffice.cn ([43.224.245.234]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-39e3abada87sm1955836a91.14.2026.09.16.22.30.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Sep 2026 22:30:10 -0700 (PDT) From: Wen Jiang To: akpm@linux-foundation.org, catalin.marinas@arm.com, linux-mm@kvack.org, urezki@gmail.com, will@kernel.org Cc: Xueyuan.chen21@gmail.com, ajd@linux.ibm.com, anshuman.khandual@arm.com, baohua@kernel.org, chleroy@kernel.org, david@kernel.org, dev.jain@arm.com, jiangwen6@xiaomi.com, leo.yan@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, rppt@kernel.org, ryan.roberts@arm.com Subject: [PATCH v8 04/10] arm64/hugetlb: drop the init_mm special case in clear_flush() Date: Thu, 17 Sep 2026 13:29:27 +0800 Message-Id: <20260917052933.188679-5-jiangwenxiaomi@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260917052933.188679-1-jiangwenxiaomi@gmail.com> References: <20260917052933.188679-1-jiangwenxiaomi@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Wen Jiang clear_flush() calls flush_tlb_kernel_range() when mm is &init_mm. This was added by commit 06fc959fcff7 ("arm64/mm: Support huge pte-mapped pages in vmap") because vmap called set_huge_pte_at(&init_mm, ...). Now that vmap uses pte_set_huge()/pte_clear_huge(), mm/vmalloc.c is no longer a caller of set_huge_pte_at() on arm64. The only remaining callers passing &init_mm are in arch/powerpc, which uses its own implementation. Remove the dead branch. No functional change. Signed-off-by: Wen Jiang --- arch/arm64/mm/hugetlbpage.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 8e799c1fe0aa6..ce247b06fe03d 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -206,10 +206,7 @@ static void clear_flush(struct mm_struct *mm, for (i =3D 0; i < ncontig; i++, addr +=3D pgsize, ptep++) __ptep_get_and_clear_anysz(mm, addr, ptep, pgsize); =20 - if (mm =3D=3D &init_mm) - flush_tlb_kernel_range(saddr, addr); - else - __flush_hugetlb_tlb_range(&vma, saddr, addr, pgsize, TLBF_NOWALKCACHE); + __flush_hugetlb_tlb_range(&vma, saddr, addr, pgsize, TLBF_NOWALKCACHE); } =20 void set_huge_pte_at(struct mm_struct *mm, unsigned long addr, --=20 2.34.1 From nobody Fri Sep 25 03:17:02 2026 Received: from mail-pj2-f43.google.com (mail-pj2-f43.google.com [74.125.227.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BF7F536A34D for ; Thu, 17 Sep 2026 05:30:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.227.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789623021; cv=none; b=SpHsqyXkEerxunPWI3m5x08YiGGLZSnfXJFIoYy1V0DPjgr+05BrdQ15xg/LHUwUFfmEyJFq8TcfF+UC41LIiSLBz3IiKdWlpY/0cUVY8roPfXn2W5R0FaYcts0RoZEL+JR253pnRLzT3MW8C4Cb/7dS+LzBqG5SFjewo0xNoIQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789623021; c=relaxed/simple; bh=ufE2Uz6n3OfWWdak1O+/fDPxZjq1jLayzDJlYGRApYc=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=m1canr8T6YCu4wZnDOX/icaNUvGeb6ntvMzvf9zKrSNvx6BeCWqwXCotrXGzV7cLgUEfb+G4lHc1IyaQxEiXXUGQsZsxhHbvFZy0E1lb7E4ofpKDuJwwDH+BdMCkQcYiA1QXqSI4KjfDQeBYAuv2+7iDg3z5oDdcir0N7bBfjcM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Ibb2mcU/; arc=none smtp.client-ip=74.125.227.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Ibb2mcU/" Received: by mail-pj2-f43.google.com with SMTP id 98e67ed59e1d1-39b350c6920so379073a91.1 for ; Wed, 16 Sep 2026 22:30:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1789623019; x=1790227819; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=RRHIBzX1Ph+VWLlGic49OjVgc6epG84F5iB9tVuXpJQ=; b=Ibb2mcU/NuL+2IQeaVkUNh9Vn6hBJ8+rhC0rd39Gt33JU2CkIv8Xvdk/sNmsHB8nY9 ecjc0iaVVYjTepkolCkr4tdRwXOocZjgtuU3GiCRTGLSpftNdZS6dQuuz/pTUu2Pe4BY qpXPy6N3SA12QVIMz/ItUWHIxYiGiXEvKmrmLA/7hCexLyoUxhh2aC/jKVibXbPuFOXU Wdm/RmzA5IDuqSlyv4aFiPOP8ZFMGb57/7vKeWgyCiwyi1ZECx1HKvh16qPcvWvX28E/ IlZW0NeJlq/9RYL4j8IHuEVCl5CVdvlSlZL9e0Xpr9JDp/nVB2uPejUKVP+UJaeNe0eD T3Sg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1789623019; x=1790227819; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=RRHIBzX1Ph+VWLlGic49OjVgc6epG84F5iB9tVuXpJQ=; b=fWiuS56CKcvphcmpivWYMVtLN9iIMC7QDrnNBn+onjCuNKpDcOjgjNGA7f158HXgY8 8iRWkivkRa2c8XWi9avsG6jgYh0SWJ4VfJLLj1GHgywJN3Jjy9BNu4IhqGKvGHiZipK8 D8ArxHRwBr6tdTUA3YCQRsZ+1jbBRGdAjbZVkktwcmb2hxj1nN46eiKa6uoR9psdql3t 4aG0U/vjYqoQ+g/IKLQjMglh+ZVcUcAZjp1QgSG7BSIl5Jz/HeEP1rWtxBGDPo67HEdx qwY/LSwrgXRe/gfMAHVcGEWCVXjQjDjrzGiDxsRFOU7t1wPyc1dmvf5DMSrzRqwN8bRJ kLlA== X-Forwarded-Encrypted: i=1; AKwUvBy4Hm368nYGeFlGd3KKzKwXKhxRyP2udeqNkvsF4JCry185f/QjSAPu6YUd9fjJR6yqy/mNDE97IHwdYPg=@vger.kernel.org X-Gm-Message-State: AFuF++mPu94m9J8JCkyKYjbIP4CcveM3Jj/bmQsWe45+sRyPj67m6ZIj Ot240QLIPEgOCY6zPuV96yB61R6FgUolXJYcLPDgOOl8MPaS0C5XbW1F X-Gm-Gg: AYBFou3a6WCd+ZmHQau6APUw7WYEEwfA6FBjePP871v2R+ywfCFJqOCbLWo8AL7ZVR6 KTa8FjAGFueR4AC9mCnex9yU7/tp9ZqlNmi39bngJZfAEfo3Cy8mp4vJKThMAAk5fez+1/nRCwW a//e4p75wWmiaWq08haW1Tpanj2+vD0qIrYmE2wT+jH8hm444bH1t/c6puKsJf1u5Nc3+bZd0Dr LqyFZPA0Xe3AES3LMsYC2ZOUA7GXrjMt8QFXKBA8rpDvU2zKqChqFyxLwTEiDR5xnbtCLitwfGT ueXH/QjNxJJSinXg3t3JI3vSetH274Bhmu3dT8QBCFmOtNJ1CEHO64hDJqtsoR0Ve51dNz9/j5Y 33wBygEbVdzDepn2hsISZNEGA7JyWNOypQbzse2U2XqPaZ9V/NuBv9m7OqMJ+70Nj2Kuy8UY7Lw iUGArUkV3K7QXkqWXFsfvp/CxDLgCVXNXvrTIXfKX5taXV1kyQvQviEJ/wAmWgmMM8CEd/VOskl th9w97l+M6hWUvOimCqcmLFucfwCKxWs7n6Dg== X-Received: by 2002:a17:90a:d644:b0:39e:288:80c9 with SMTP id 98e67ed59e1d1-39e1e4d3fd1mr12367022a91.19.1789623018896; Wed, 16 Sep 2026 22:30:18 -0700 (PDT) Received: from mi-OptiPlex-7060.mioffice.cn ([43.224.245.234]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-39e3abada87sm1955836a91.14.2026.09.16.22.30.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Sep 2026 22:30:18 -0700 (PDT) From: Wen Jiang To: akpm@linux-foundation.org, catalin.marinas@arm.com, linux-mm@kvack.org, urezki@gmail.com, will@kernel.org Cc: Xueyuan.chen21@gmail.com, ajd@linux.ibm.com, anshuman.khandual@arm.com, baohua@kernel.org, chleroy@kernel.org, david@kernel.org, dev.jain@arm.com, jiangwen6@xiaomi.com, leo.yan@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, rppt@kernel.org, ryan.roberts@arm.com, Xueyuan Chen Subject: [PATCH v8 05/10] arm64/vmalloc: allow arch_vmap_pte_range_map_size() to batch multiple CONT_PTE Date: Thu, 17 Sep 2026 13:29:28 +0800 Message-Id: <20260917052933.188679-6-jiangwenxiaomi@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260917052933.188679-1-jiangwenxiaomi@gmail.com> References: <20260917052933.188679-1-jiangwenxiaomi@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: "Barry Song (Xiaomi)" Allow arch_vmap_pte_range_map_size to batch across multiple CONT_PTE blocks, reducing both PTE setup and TLB flush iterations. For CONT_PTE_SIZE-aligned ranges, return a mapping size that may cover multiple CONT_PTE blocks, capped below PMD_SIZE. These sizes are vmalloc mapping spans, not HugeTLB hstate sizes. Signed-off-by: Barry Song (Xiaomi) Signed-off-by: Wen Jiang Tested-by: Xueyuan Chen Tested-by: Leo Yan --- arch/arm64/include/asm/vmalloc.h | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/vmalloc.h b/arch/arm64/include/asm/vmal= loc.h index 4ec1acd3c1b34..4053b1ec1902e 100644 --- a/arch/arm64/include/asm/vmalloc.h +++ b/arch/arm64/include/asm/vmalloc.h @@ -23,10 +23,14 @@ static inline unsigned long arch_vmap_pte_range_map_siz= e(unsigned long addr, unsigned long end, u64 pfn, unsigned int max_page_shift) { + unsigned long size; + /* * If the block is at least CONT_PTE_SIZE in size, and is naturally * aligned in both virtual and physical space, then we can pte-map the * block using the PTE_CONT bit for more efficient use of the TLB. + * The returned mapping size may cover multiple CONT_PTE_SIZE blocks, + * capped below PMD_SIZE. */ if (max_page_shift < CONT_PTE_SHIFT) return PAGE_SIZE; @@ -40,7 +44,9 @@ static inline unsigned long arch_vmap_pte_range_map_size(= unsigned long addr, if (!IS_ALIGNED(PFN_PHYS(pfn), CONT_PTE_SIZE)) return PAGE_SIZE; =20 - return CONT_PTE_SIZE; + size =3D min3(end - addr, 1UL << max_page_shift, PMD_SIZE >> 1); + size =3D round_down(size, CONT_PTE_SIZE); + return size; } =20 #define arch_vmap_pte_range_unmap_size arch_vmap_pte_range_unmap_size --=20 2.34.1 From nobody Fri Sep 25 03:17:02 2026 Received: from mail-pj2-f13.google.com (mail-pj2-f13.google.com [74.125.227.141]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AB3322E738B for ; Thu, 17 Sep 2026 05:30:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.227.141 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789623029; cv=none; b=EX2P9vNqF7q+/ZhUstGozeq/4dJ4YV0/+w80QPYI8r9+1seSVfjwtLcV6v7DiluACEreIrJvvqyK1TafbVQh4TfUvUlm6OjP+6pLj68vCATVbFlDRoxGaxbX2YfzsSUmoztNtYXUgoB4QFmAMwOzaHp+QEO+N0QD7DpHgW+Ltf4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789623029; c=relaxed/simple; bh=nu9CVlYrXHpgLjfI48yaYuzW8VfJM48VdD20onjBsno=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=GQW99xHfLPh6c4TN0va5xs0VhoQa3F5gBefumJZ3bfTnZvsFgpkV/2ul/XO8bcgpVC7wbNFo2ZE/85ymH0LaK3uQnFUUOt4hIqvnUQ7+b2xiu4tQdnGK5Ic+etYduQ9s3UDaJCwsH/JlTlLHutuxzrHEaejpLfN8bSWrycoa8zs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Xl/3/k1r; arc=none smtp.client-ip=74.125.227.141 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Xl/3/k1r" Received: by mail-pj2-f13.google.com with SMTP id 98e67ed59e1d1-39e03468a5fso389478a91.0 for ; Wed, 16 Sep 2026 22:30:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1789623027; x=1790227827; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=G8iXtJFaILHaNQYB7TRtsKAUY5yHxckNGkamGztIiuQ=; b=Xl/3/k1rTS2dlblvy8eSmme3m0mekQJKVrf8rOI6c3EBgAMou/d7odx3V3Q5udnm2A 3Zqdyscdxv68Gm/OisCy/55ofJXMk99tUU+XB5pg4l04aEyiuZ8GnbqZXB97qEHJjOzt Zty0ZfpLZi9QPoF8USDd3raPrG97iS722YSiX8ooFNbqyN+s7BgiV0ycRGHIfvGq2n3Z O74GPXFK7eewT+sZfT0/NC8uq2GIii0UJrBJJVmsjIYci8XTNkBCtpf5NK4k2vzR4ceS zAKs18TkdO33ksY3J+AVwfN+Y5+N3obociagDe5NZxSorMzLF9cVcGatudATeA9oTN9w or3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1789623027; x=1790227827; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=G8iXtJFaILHaNQYB7TRtsKAUY5yHxckNGkamGztIiuQ=; b=I/UNgSUA75xUH5HhDl7hVzKmHlc4y2mnPtSrwFDyxMpU+MBpbBiEr/HGuqcvgz8PGC CkY8weoSo+OhdEZS9pM6KfEdsHk+78Tnoq3FOay4tJlRFuKJtMzrA7ylYykQMDBXJ+37 RhXanbwv+9OZjfEb5zcVdApBufdkqNsIxHWWqRg7PCgcwc+9ZQ9uqFjCNDTs03q3l6Q/ G+GwpVAzVpKDsW19KYM1Lcf85LuARTJ3lmE2GovUE4+0DVG3Aj2qkWmy1CskzYp8Rehk 0BqQWJ6XdUyvYE7YMpcoS+sksHppkG1twPhrJJkVgrsvXZRMGbCuPmG5jhUe7aMmMA+D IQmQ== X-Forwarded-Encrypted: i=1; AKwUvBy8ThTiqhNStVdos04BQAxJjNLtG07wAvGipMNF2JHrRJrJ5x42g1JY5mTYw7iRc+PCYJEiFdYmyP26KBs=@vger.kernel.org X-Gm-Message-State: AFuF++mZJfRbqBQ2Ic0WEajYudSdySg+4eyHRpytQQcXEyOcrk/N7uVW xY8dZvOD84qVv/m8BeLuLaJ04MX4Ex7HRDmlPBi97vjQwlhzS87HLtqy X-Gm-Gg: AYBFou32H/1UtOWKwSiQ+y+vtXZgAc2ZYyP2PvyQ3/JJNQcCLBaD2vQfty8U7dOCtxI mwMlhMVjK0ls71t+Fr2GAe9EhsIjtO4WdgURTe40gPR5Gi4FDeJnZ0tvvBN81TCZbO2UgGYMQK0 4XrrKz2xnkBiZPJKUqqNOvOZFU5BRvplGi/xkA/9am0Wka4FL9E8dcfX61uVrrjchCDq3ARm2yX vUm2hLLwtGRotoZ/LAusCAN5NBLRaUMShzFJAay4/AX3Odf6uNEWYljOKv7bSB0JAD7dzse1pqW W2pPcoyC4u13LGT8T+bMyxR1BLQxn9Ctfy9R1JmbVks3Yf2ayyihlDiOTp+Vjrb08xUK5fjTzAw 4o/4zomHiwXn/Cc/GSUVS9eBxMn04UgG/r9Vz0ENLge3CNpYFrlJSBU7k39hhucObkPNUwf8VpB 2ZUt0tSF/8PDpE3cErGZ/Cjkok1jRtccl5ZsKRY5EH5ApoJ3VEWmHyx1DAH4yJYhEHYbj5tbsBj 4SOu87cP9nt5QqFTK6pXwCO+tw= X-Received: by 2002:a17:90b:4e88:b0:398:cb56:e92 with SMTP id 98e67ed59e1d1-39e1e30ba45mr13210240a91.11.1789623025225; Wed, 16 Sep 2026 22:30:25 -0700 (PDT) Received: from mi-OptiPlex-7060.mioffice.cn ([43.224.245.234]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-39e3abada87sm1955836a91.14.2026.09.16.22.30.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Sep 2026 22:30:24 -0700 (PDT) From: Wen Jiang To: akpm@linux-foundation.org, catalin.marinas@arm.com, linux-mm@kvack.org, urezki@gmail.com, will@kernel.org Cc: Xueyuan.chen21@gmail.com, ajd@linux.ibm.com, anshuman.khandual@arm.com, baohua@kernel.org, chleroy@kernel.org, david@kernel.org, dev.jain@arm.com, jiangwen6@xiaomi.com, leo.yan@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, rppt@kernel.org, ryan.roberts@arm.com, Xueyuan Chen Subject: [PATCH v8 06/10] mm/vmalloc: extract vmap_set_ptes() to consolidate PTE mapping logic Date: Thu, 17 Sep 2026 13:29:29 +0800 Message-Id: <20260917052933.188679-7-jiangwenxiaomi@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260917052933.188679-1-jiangwenxiaomi@gmail.com> References: <20260917052933.188679-1-jiangwenxiaomi@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Wen Jiang Extract the common PTE mapping logic from vmap_pte_range() into a shared helper vmap_set_ptes(). This handles both CONT_PTE and regular PTE mappings in a single function, preparing for the next patch which will extend vmap_pages_pte_range() to also use this helper. No functional change. Signed-off-by: Wen Jiang Tested-by: Xueyuan Chen Tested-by: Leo Yan --- mm/vmalloc.c | 36 +++++++++++++++++++++++++++--------- 1 file changed, 27 insertions(+), 9 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 025b3a848d564..a0508bdb7d36f 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -90,6 +90,29 @@ struct vfree_deferred { static DEFINE_PER_CPU(struct vfree_deferred, vfree_deferred); =20 /*** Page table manipulation functions ***/ + +/* + * Try contiguous mappings at the PTE level for arches which support them,= and if + * requested by the caller. Fall back to PAGE_SIZE mappings otherwise. + * + * Return: mapping size. + */ +static __always_inline unsigned long vmap_set_ptes(pte_t *ptep, + unsigned long addr, unsigned long end, u64 pfn, + pgprot_t prot, unsigned int max_page_shift) +{ + unsigned long size; + + size =3D arch_vmap_pte_range_map_size(addr, end, pfn, max_page_shift); + if (size !=3D PAGE_SIZE) { + pte_set_huge(ptep, addr, PFN_PHYS(pfn), prot, size); + return size; + } + + set_pte_at(&init_mm, addr, ptep, pfn_pte(pfn, prot)); + return PAGE_SIZE; +} + static int vmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long en= d, phys_addr_t phys_addr, pgprot_t prot, unsigned int max_page_shift, pgtbl_mod_mask *mask) @@ -98,6 +121,7 @@ static int vmap_pte_range(pmd_t *pmd, unsigned long addr= , unsigned long end, u64 pfn; struct page *page; unsigned long size; + unsigned int steps; =20 if (WARN_ON_ONCE(!PAGE_ALIGNED(end - addr))) return -EINVAL; @@ -118,15 +142,9 @@ static int vmap_pte_range(pmd_t *pmd, unsigned long ad= dr, unsigned long end, BUG(); } =20 - size =3D arch_vmap_pte_range_map_size(addr, end, pfn, max_page_shift); - if (size !=3D PAGE_SIZE) { - pte_set_huge(pte, addr, PFN_PHYS(pfn), prot, size); - pfn +=3D PFN_DOWN(size); - continue; - } - set_pte_at(&init_mm, addr, pte, pfn_pte(pfn, prot)); - pfn++; - } while (pte +=3D PFN_DOWN(size), addr +=3D size, addr !=3D end); + size =3D vmap_set_ptes(pte, addr, end, pfn, prot, max_page_shift); + steps =3D PFN_DOWN(size); + } while (pte +=3D steps, pfn +=3D steps, addr +=3D size, addr !=3D end); =20 lazy_mmu_mode_disable(); *mask |=3D PGTBL_PTE_MODIFIED; --=20 2.34.1 From nobody Fri Sep 25 03:17:02 2026 Received: from mail-pz2-f12.google.com (mail-pz2-f12.google.com [74.125.228.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BC9B62E738B for ; Thu, 17 Sep 2026 05:30:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.228.12 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789623034; cv=none; b=nNjoyGLI45Z68T4B7qDJ79X/ep2BfkEgOLeguVLRCReWa/EeB+jD1aTykE4QyVLsnkxBdl5jnoyJxNuNx0L/zs1fP0cjaerVfgo5KWzUsHnB9UXxkyDIOvThw6gkE8h1NTI5ITCJZXXWccw6YqcYUjJchTCnCR9TH+eQrY+K3Wo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789623034; c=relaxed/simple; bh=uMUKiMDN8NyTCo/fPbIx3j/DhN59lS9Na64iD8H+2NQ=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=kOoq0qGzzCMze7ffaC1U58+Kd2apbSD1KaTG/CnQ+rJwrDe2kDcEpDwsf2eGpLHnPnEGWc507xUH56tgfOy4UpQOdAL+v1RiWvqyTfpSG+Rzs2HUnqdm9NpmmPEbZFmtKch6zimj0YNSLMPqirugxZa8VFF3cyveVJTkgkhOGI4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=KNOdZWg3; arc=none smtp.client-ip=74.125.228.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="KNOdZWg3" Received: by mail-pz2-f12.google.com with SMTP id 41be03b00d2f7-cc1cea4bfd1so295641a12.0 for ; Wed, 16 Sep 2026 22:30:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1789623032; x=1790227832; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=XToDi6fIsVYoV27kQlhNobBUYsIv/tm0eptMLl1rq74=; b=KNOdZWg3+qNi4Y+n7txqbbNwR8fJwAsNRvUUGnhXwjpG3h/eLOnnGphzWTv2g/3mNd cxy1CvApdVVWXI3ZIS1vrWb0XHIlV6hVuK8RbB2zykade84UozGBR99IHdxOYqd5Ije4 wFUW1XZUcNlsVtX/of82EiEz28UiFjfrN+WSyhsDnxyDNYLUzetUVgljYi/1467kcqD4 Ufghv3aYDGipD3tzKaSj/1WARPG++3ODOrU1SV3PweK7PoGiqSpdPm7+SVV1zi9w/G9A 4St+lre/qIx+3ZgTjFoKpsO8iWypgURY7EoM+g0nUHFqFJMB0fIVSGFz2HjmzjO7Uj/M mLcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1789623032; x=1790227832; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=XToDi6fIsVYoV27kQlhNobBUYsIv/tm0eptMLl1rq74=; b=g44xcILtFn5In6bjZoO+WmK0VxEPGKiR1bsUYx0UoHZq49SB+od1x1ICsIUjq1SwN4 ZA8QChKSED6cescv2X0ebG7KqpOu4y6Ke3WSSHt++cjdvTYlzBH0E0HLr5YEgRjdNg98 ZuEnD+Mp8iDEhjZVSt3PRwDXeTageRsuFYtENiTog9cVJ0obCh2h6/suF+/1HAoxNm9P qbSnR7XRv/HNbjFkeCyiK3Fzbvh12aatQpfvUq4fv9IXTgBRcDsXAxr1JOlZbE4wKpjK rElMAKgZB7/EcTZat8BbHiTvR6bLFQnwA9kQtlCh6iSco/E7YZe1dF9SS//vCB56kRiK nfCw== X-Forwarded-Encrypted: i=1; AKwUvBwy2pnsceRPF0E2iy04tzOD+3ImIBdJlDsGbPQJndK+DaGwDqSV2Bzym2mh777bvJZZdpIMRMhCs/nmDa8=@vger.kernel.org X-Gm-Message-State: AFuF++l1eE8RpjtL/s4XK+ohizBGg7J77gcVIh7c+cC+rmL7hUWQ4Evo BPGjskBV4odUcGSMVxYulNj8m0fymQoVrejd1s3tbL2/W+okW8TxZL4P X-Gm-Gg: AYBFou3nN2mbHCZjxWIYVVnxTz1cm9h6Segb7ENpiE0i7nvzZvDY8lXMbzxdXTGEnXv WgDOHrMoZhep6PK841eF50XpHrmnbGOnhKeCSjTFhj1PqtVsmRh5keylL9Vgm7+vv86TL6Eoq/5 aR8xQIAi/ndU6uS3hdUM11kl4VtNPi9xPJATQSDJJ0Wxz4SPLvilDfEtN2bcMO5m+UB+MFgWXcj lWdnCXXToIF2JDSPijRTHqnyx2FrMkLuOay004gpIhbBktrd7F/+82ocdPVBmrDpZvjcUg4pZP3 WrGh+x2BGK7vr5IqOquCJnX5huhm4zuzjXH5mdE1CoLsL01evR0k+Lw+DoszX7ZeDiHjLpJyBmF mfahuzUN7KY9RCLN4lbjGWBVI9b/HrH/S0BK7WlgL8yhJCbpmpQNVO7f4Q5GYE0UxCjshU8SDgc rELtgqPfgoCwWf+e5UWfHFxzsv+GNIrry2jcIgLn1KXCFIteCSHjQkbPZH9T9PkaMzRWhqyM8Rw xjG/yh2TCuKo/5ktT9Y4STjtjg= X-Received: by 2002:a17:90b:4a51:b0:39d:ee20:1e28 with SMTP id 98e67ed59e1d1-39e1e510f73mr15610978a91.21.1789623031887; Wed, 16 Sep 2026 22:30:31 -0700 (PDT) Received: from mi-OptiPlex-7060.mioffice.cn ([43.224.245.234]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-39e3abada87sm1955836a91.14.2026.09.16.22.30.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Sep 2026 22:30:31 -0700 (PDT) From: Wen Jiang To: akpm@linux-foundation.org, catalin.marinas@arm.com, linux-mm@kvack.org, urezki@gmail.com, will@kernel.org Cc: Xueyuan.chen21@gmail.com, ajd@linux.ibm.com, anshuman.khandual@arm.com, baohua@kernel.org, chleroy@kernel.org, david@kernel.org, dev.jain@arm.com, jiangwen6@xiaomi.com, leo.yan@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, rppt@kernel.org, ryan.roberts@arm.com, Xueyuan Chen Subject: [PATCH v8 07/10] mm/vmalloc: extend page table walk to support larger page_shift sizes and eliminate page table rewalk Date: Thu, 17 Sep 2026 13:29:30 +0800 Message-Id: <20260917052933.188679-8-jiangwenxiaomi@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260917052933.188679-1-jiangwenxiaomi@gmail.com> References: <20260917052933.188679-1-jiangwenxiaomi@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: "Barry Song (Xiaomi)" vmap_pages_range_noflush_walk() (formerly vmap_small_pages_range_noflush()) provides a clean interface by taking struct page **pages and mapping them via direct PTE iteration. This avoids the page table rewalk seen when using vmap_range_noflush() for page_shift values other than PAGE_SHIFT. Extend it to support larger page_shift values, and add PMD- and contiguous-PTE mappings as well. Rename it to vmap_pages_range_noflush_walk() since it now handles more than just small pages. For vmalloc() allocations with VM_ALLOW_HUGE_VMAP, we no longer need to iterate over pages one by one via vmap_range_noflush(), which would otherwise lead to page table rewalk. The code is now unified with the PAGE_SHIFT case by simply calling vmap_pages_range_noflush_walk(). Signed-off-by: Barry Song (Xiaomi) Signed-off-by: Wen Jiang Tested-by: Xueyuan Chen Tested-by: Leo Yan Reviewed-by: Dev Jain --- mm/vmalloc.c | 84 +++++++++++++++++++++++++++++++--------------------- 1 file changed, 50 insertions(+), 34 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index a0508bdb7d36f..cf1daf931a6f6 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -535,8 +535,10 @@ void vunmap_range(unsigned long addr, unsigned long en= d) =20 static int vmap_pages_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, pgprot_t prot, struct page **pages, int *nr, - pgtbl_mod_mask *mask) + pgtbl_mod_mask *mask, unsigned int shift) { + unsigned long pfn, size; + unsigned int steps; int err =3D 0; pte_t *pte; =20 @@ -567,9 +569,10 @@ static int vmap_pages_pte_range(pmd_t *pmd, unsigned l= ong addr, break; } =20 - set_pte_at(&init_mm, addr, pte, mk_pte(page, prot)); - (*nr)++; - } while (pte++, addr +=3D PAGE_SIZE, addr !=3D end); + pfn =3D page_to_pfn(page); + size =3D vmap_set_ptes(pte, addr, end, pfn, prot, shift); + steps =3D PFN_DOWN(size); + } while (pte +=3D steps, *nr +=3D steps, addr +=3D size, addr !=3D end); =20 lazy_mmu_mode_disable(); *mask |=3D PGTBL_PTE_MODIFIED; @@ -579,60 +582,88 @@ static int vmap_pages_pte_range(pmd_t *pmd, unsigned = long addr, =20 static int vmap_pages_pmd_range(pud_t *pud, unsigned long addr, unsigned long end, pgprot_t prot, struct page **pages, int *nr, - pgtbl_mod_mask *mask) + pgtbl_mod_mask *mask, unsigned int shift) { pmd_t *pmd; unsigned long next; + int err; =20 pmd =3D pmd_alloc_track(&init_mm, pud, addr, mask); if (!pmd) return -ENOMEM; do { next =3D pmd_addr_end(addr, end); - if (vmap_pages_pte_range(pmd, addr, next, prot, pages, nr, mask)) - return -ENOMEM; + + if (shift >=3D PMD_SHIFT) { + struct page *page =3D pages[*nr]; + phys_addr_t phys_addr; + + if (WARN_ON(!page)) + return -ENOMEM; + if (WARN_ON(!pfn_valid(page_to_pfn(page)))) + return -EINVAL; + + phys_addr =3D page_to_phys(page); + + if (vmap_try_huge_pmd(pmd, addr, next, phys_addr, prot, shift)) { + *mask |=3D PGTBL_PMD_MODIFIED; + *nr +=3D 1 << (PMD_SHIFT - PAGE_SHIFT); + continue; + } + } + err =3D vmap_pages_pte_range(pmd, addr, next, prot, pages, nr, mask, shi= ft); + if (err) + return err; } while (pmd++, addr =3D next, addr !=3D end); return 0; } =20 static int vmap_pages_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end, pgprot_t prot, struct page **pages, int *nr, - pgtbl_mod_mask *mask) + pgtbl_mod_mask *mask, unsigned int shift) { pud_t *pud; unsigned long next; + int err; =20 pud =3D pud_alloc_track(&init_mm, p4d, addr, mask); if (!pud) return -ENOMEM; do { next =3D pud_addr_end(addr, end); - if (vmap_pages_pmd_range(pud, addr, next, prot, pages, nr, mask)) - return -ENOMEM; + err =3D vmap_pages_pmd_range(pud, addr, next, prot, pages, nr, mask, shi= ft); + if (err) + return err; } while (pud++, addr =3D next, addr !=3D end); return 0; } =20 static int vmap_pages_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end, pgprot_t prot, struct page **pages, int *nr, - pgtbl_mod_mask *mask) + pgtbl_mod_mask *mask, unsigned int shift) { p4d_t *p4d; unsigned long next; + int err; =20 p4d =3D p4d_alloc_track(&init_mm, pgd, addr, mask); if (!p4d) return -ENOMEM; do { next =3D p4d_addr_end(addr, end); - if (vmap_pages_pud_range(p4d, addr, next, prot, pages, nr, mask)) - return -ENOMEM; + err =3D vmap_pages_pud_range(p4d, addr, next, prot, pages, nr, mask, shi= ft); + if (err) + return err; } while (p4d++, addr =3D next, addr !=3D end); return 0; } =20 -static int vmap_small_pages_range_noflush(unsigned long addr, unsigned lon= g end, - pgprot_t prot, struct page **pages) +/* + * It can take an array of pages which are not all contiguous, but it + * may have contiguous chunks, as hinted by @shift. + */ +static int vmap_pages_range_noflush_walk(unsigned long addr, unsigned long= end, + pgprot_t prot, struct page **pages, unsigned int shift) { unsigned long start =3D addr; pgd_t *pgd; @@ -647,7 +678,7 @@ static int vmap_small_pages_range_noflush(unsigned long= addr, unsigned long end, next =3D pgd_addr_end(addr, end); if (pgd_bad(*pgd)) mask |=3D PGTBL_PGD_MODIFIED; - err =3D vmap_pages_p4d_range(pgd, addr, next, prot, pages, &nr, &mask); + err =3D vmap_pages_p4d_range(pgd, addr, next, prot, pages, &nr, &mask, s= hift); if (err) break; } while (pgd++, addr =3D next, addr !=3D end); @@ -670,27 +701,12 @@ static int vmap_small_pages_range_noflush(unsigned lo= ng addr, unsigned long end, int __vmap_pages_range_noflush(unsigned long addr, unsigned long end, pgprot_t prot, struct page **pages, unsigned int page_shift) { - unsigned int i, nr =3D (end - addr) >> PAGE_SHIFT; - WARN_ON(page_shift < PAGE_SHIFT); =20 - if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMALLOC) || - page_shift =3D=3D PAGE_SHIFT) - return vmap_small_pages_range_noflush(addr, end, prot, pages); + if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMALLOC)) + page_shift =3D PAGE_SHIFT; =20 - for (i =3D 0; i < nr; i +=3D 1U << (page_shift - PAGE_SHIFT)) { - int err; - - err =3D vmap_range_noflush(addr, addr + (1UL << page_shift), - page_to_phys(pages[i]), prot, - page_shift); - if (err) - return err; - - addr +=3D 1UL << page_shift; - } - - return 0; + return vmap_pages_range_noflush_walk(addr, end, prot, pages, page_shift); } =20 int vmap_pages_range_noflush(unsigned long addr, unsigned long end, --=20 2.34.1 From nobody Fri Sep 25 03:17:02 2026 Received: from mail-pz2-f17.google.com (mail-pz2-f17.google.com [74.125.228.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 190CA3546DE for ; Thu, 17 Sep 2026 05:30:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.228.17 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789623040; cv=none; b=eXA+I9z/Sx2sAmc4Q8pEU1unrcJJSWSFtpVxQDBlJ+fwYC+PxeNfqNBpTNdyoLtRtqsJOXzFY2Bmpdzy5zSgLBG6OLWWvPl0R7IdwKdU3LAPT5mzS0j4DiL5UBLMvXZhJbVZP16UDIvTOTKohJw4+lBOTFTEgSL1a4ALtzXPDCo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789623040; c=relaxed/simple; bh=8yVhZ3fE/86hGPVlLHCwsytF1kuIHqv5hZXT51LmS14=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=djPhWlwHmlcXglayxYMnYc43Xu9ZSAsTjot8Q9KcCaoht1JuCbqiSUkDgNOvBofjzRtqm6EM6kCYUcN1ARl5WOagGTLBacMnXU6+/beil8UVCzXzizC3DvQRVOsADlMbyHLDhOtrYq+S5isFZngsPfaxlh5c1xZo+iZ1lKp7ysY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=JNi1oe+f; arc=none smtp.client-ip=74.125.228.17 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="JNi1oe+f" Received: by mail-pz2-f17.google.com with SMTP id 41be03b00d2f7-cc4c3304784so299068a12.3 for ; Wed, 16 Sep 2026 22:30:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1789623038; x=1790227838; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=NzaNd9dh5gQiSMMbEDA26vNGMF5WFO7Tl8lfuiT6OXc=; b=JNi1oe+f9tI0s588VKQrW23NRsTC/PKiryTHoMmZ4zVpSJuVbg8OfSxzQhYmgbZ46H GirW+wbeca85hf1ZnPs9sYCvWBHI4BM9sf136DySthwrKo7jkRzdFHCy/GPX9woAmXhh m+tH0kWMCeEOft27Fi5g4LOb6tRMKhauyGvOe6dOyOpmvLctF5ul/oYKVMAbdwec68yK ZHUS/G01o79YZB1J701068LgDSk2iuUly4R7TA8kETPKTemaS0gHPK83WCsQNPOyNqcl vKYyuH9LaE6Y7CVPKVXzYmU7LnM3fG5GxPD4L8m4qLW9zfva/VG4IuxfKR6NhmfKPTlm vTww== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1789623038; x=1790227838; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=NzaNd9dh5gQiSMMbEDA26vNGMF5WFO7Tl8lfuiT6OXc=; b=BEsCyCG3aGIAKO+wTFVC7EJnToQMqAZKbRLmHkWHEntp8ED0MVadaK4KvRcXxy4hmQ QoRL0D9lNneHF/J7dkzxxAhfaJXV0YMIUx4eGxljcZdBAjjsbmFqRcH5U8x9UlZ+Lmby zzBFusnIiHrvXxPJd3Cax5KlcX7iMficBixQyhW1FMR/BgfC/esFp4oIRvwe7rpQhacl k2zBLx4Gsq8ooeTLYk/bU8n5kidnXa2e67Mi2jOfshJPL2kRJ50lXozcgFfpPz8wvP0o 5x+Uztj/G5wY6UGMs1km+IOc+qX0b3U9r5Eq3repbjTBfw6P2FaqwQpvZHgJhZQXn7nf OZGw== X-Forwarded-Encrypted: i=1; AKwUvBw7pRlK30vZenOvb/C0ViHwIaOory5jzhzc7dp+MBT3rj1LLh7BHf8e7Elqe2XREyqhVu1vzKX96h4q5jk=@vger.kernel.org X-Gm-Message-State: AFuF++meuYIVOk02uFanQ+xc3rcuAiKCTdtFkCBEjhX6dpQ+nvnxkK+C qV+oTBbV6xC47GdQz8uH7mLbpqwLRnPFVQnjE5l7YvgMVlLA/U2r0XVC X-Gm-Gg: AYBFou2EJlgNLFHnxqOyELQRV2EcyFfxt628eNvN/YPIODvu+tCHfoUHdbD0Nq4qxq7 LStQSxHT4ccqboEFKsAG5dVW96MazIUdZIhYe59WqPmOGmAP1X/Q88cIC8tcwhs7DElOpfZrLfZ J+usrry/xelLjhJq79rsrN/AKdNcOxskWnw/bA5ruzsWsqx5vGFKcrgc9HM1njyT3SKUb+DR1km zejrC0KGLly+Brmv6WJtghK5VIzmk9cKTG4UkNtU5acWsCPAcVaTKmr5+Q4pWtVy35v47fty8WT 6IUkOX5bpNI1Alc3XMVKojXyfZqyAi5eqMi8aT+8qdtK1u4E9T8yoknoTpX2SwAdrdMMFfuWkit 7RJ68PPuskfNUR4w13PbbNlmHRjxuqMcujKRKWjNxttnXCF4JCLfjwzjXOlvNkNLyxUgk37n8AB tEI/zQ2alse1yY99tLXA29ry7yQ4V1sNBq6jc9EBjVC5uT4Z6ChVSf2QOTR4SRwSnKKFwOuau91 eM4sl15zZtYOfQcPauvJqbclzU= X-Received: by 2002:a17:90b:3b91:b0:381:6c5:3f63 with SMTP id 98e67ed59e1d1-39e1e2bf7a7mr16057715a91.6.1789623038178; Wed, 16 Sep 2026 22:30:38 -0700 (PDT) Received: from mi-OptiPlex-7060.mioffice.cn ([43.224.245.234]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-39e3abada87sm1955836a91.14.2026.09.16.22.30.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Sep 2026 22:30:37 -0700 (PDT) From: Wen Jiang To: akpm@linux-foundation.org, catalin.marinas@arm.com, linux-mm@kvack.org, urezki@gmail.com, will@kernel.org Cc: Xueyuan.chen21@gmail.com, ajd@linux.ibm.com, anshuman.khandual@arm.com, baohua@kernel.org, chleroy@kernel.org, david@kernel.org, dev.jain@arm.com, jiangwen6@xiaomi.com, leo.yan@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, rppt@kernel.org, ryan.roberts@arm.com, Xueyuan Chen Subject: [PATCH v8 08/10] mm/vmalloc: extract vm_shift() to consolidate mapping shift selection Date: Thu, 17 Sep 2026 13:29:31 +0800 Message-Id: <20260917052933.188679-9-jiangwenxiaomi@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260917052933.188679-1-jiangwenxiaomi@gmail.com> References: <20260917052933.188679-1-jiangwenxiaomi@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Wen Jiang Extract the vmalloc mapping shift selection from __vmalloc_node_range_noprof() into vm_shift(), preparing for reuse by the vmap batching path. No functional change. Suggested-by: Dev Jain Signed-off-by: Wen Jiang Tested-by: Xueyuan Chen Tested-by: Leo Yan --- mm/vmalloc.c | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index cf1daf931a6f6..ed868afd8194d 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -3532,6 +3532,14 @@ void vunmap(const void *addr) } EXPORT_SYMBOL(vunmap); =20 +static inline unsigned int vm_shift(pgprot_t prot, unsigned long size) +{ + if (arch_vmap_pmd_supported(prot) && size >=3D PMD_SIZE) + return PMD_SHIFT; + + return arch_vmap_pte_supported_shift(size); +} + /** * vmap - map an array of pages into virtually contiguous space * @pages: array of page pointers @@ -4039,11 +4047,7 @@ void *__vmalloc_node_range_noprof(unsigned long size= , unsigned long align, * supporting them. */ =20 - if (arch_vmap_pmd_supported(prot) && size >=3D PMD_SIZE) - shift =3D PMD_SHIFT; - else - shift =3D arch_vmap_pte_supported_shift(size); - + shift =3D vm_shift(prot, size); align =3D max(original_align, 1UL << shift); } =20 --=20 2.34.1 From nobody Fri Sep 25 03:17:02 2026 Received: from mail-pj2-f42.google.com (mail-pj2-f42.google.com [74.125.227.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 550763A874A for ; Thu, 17 Sep 2026 05:30:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.227.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789623046; cv=none; b=ulA0m0SuYuMFq4E2ibFDsO5Xp29XHTLW83CMHpb2FJakkposVSF53fizbgLIOoxQ0WB8blfWl16znUqKHcbh1BkGuW/KRGIQjdyPjoqgHTdDD4OE0o+6HlYyt15zibFa7opD6jBdd+vZkwuX/pSSk40RmxEG4P8r3aIhpU4PAho= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789623046; c=relaxed/simple; bh=nxcc+X5rXpZz7j+4HigRc/Xt8kSFUwFf7kdydb8URt8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=dR13ujRCopDMthxfH6Hrs/BLzLNJED8+saMj2lFr3iCCjuXqXHxyd7HknnG+k6QYRLvIZalIAvxwrB66f0WDDRLCT0sIHc+wAZmhSwioWMIZqPiP42jKn7SbNYaZW1K8+F7CyuGHiVCkKWy8C2wXNoNcHbuCF3J7IOTMYstKKXQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=baNJi4IQ; arc=none smtp.client-ip=74.125.227.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="baNJi4IQ" Received: by mail-pj2-f42.google.com with SMTP id 98e67ed59e1d1-39b350c69b4so337108a91.2 for ; Wed, 16 Sep 2026 22:30:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1789623045; x=1790227845; darn=vger.kernel.org; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:from:to:cc:subject :date:message-id:reply-to:content-type; bh=yJbFWtXzdjoxqLLRP1e19uuFJMa1NhU30c4WN8jSX0Q=; b=baNJi4IQWggydvmZ8lFJ0Y/IbqPiMccYptMpybHVIcNapekYPe2N17WCqQcdx/RbO3 qL6xvNVQyqzhxeBvAY6mX7Y6Nh+gHsNKdKdjsO7Q8/fogsKRG9PUqXmnXzjcXRCcY8mM do3hWITe1u7wCkbn17DyJzXKR+XkTKiht7hisxsL1uXBvcpQo8L1cDPugUkbr7tww/Qd SseLMl9LfmVSJ0bHl4oLOcDR+nWHQxcgCagxlLn1SSeMNHIB17YIiM4qPxceL5t7yTyD s3Xy3vkI5y2jG9mP6pwbmHAKs2YS8BxUZKnLRS7KhxEfqA/LRna6wT2OM2er8u4btFL1 2ceQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1789623045; x=1790227845; h=content-transfer-encoding:content-type:mime-version:references :in-reply-to:message-id:date:subject:cc:to:from:x-gm-gg :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to :content-type; bh=yJbFWtXzdjoxqLLRP1e19uuFJMa1NhU30c4WN8jSX0Q=; b=xGI6uiNpe2tklHOk8QpH+dWJKsdxs9D3KZ6OG94PM7ImYr8zYKG7HMI1gDXUGVXmW+ HIZxm5lxcR1WB0aI000iEjMIhYKa9NtZjfqT5dcxiCU6n+Xds0kXL53YKd1Tn313TbTI 987eMJSp/WfwPl2mH1/p9QPXG1j7JRvza2qmlPvT9aXKMQwEvpqk5CNnTqOPcM+p4NML nsTr6e0eQ86fX1A44hM4MTV3jJuZqcuBPTaKcb6NzZZ1v0pFJMK2hv7VFLJR39ucutlr wUWElE0UgTNUn4twJowXKfIE1NtiOgu1mI8sSnHqj83UlvBVlhBlEgQDqyasxZJePEZ8 qftw== X-Forwarded-Encrypted: i=1; AKwUvByMEi9I3U4pdnkbHlft6CB+sAgv5KNbzZcXCjXDrPePnIB3gZr/ECAYh2M1o5a4eP/Van4to7hzKXJ58MU=@vger.kernel.org X-Gm-Message-State: AFuF++n5n+H8jPjx3T60eU+BPETazl8yqKxZmmzubOSv/yvAsMH75WOo 0CykFxoIXiChJWHNf+bGpWM/nwdDDUUdmfJ6qvl4w7U53Zcmuv/sP3OL X-Gm-Gg: AYBFou3l5ruA0JIn83swj5Q1NrDR3uEXG4pmSmoLNH9hpJ9YH5SLdi/oO5YLi6Eiwy+ HtVx5Ulm9696/rZWhbTdizjLVzYOQFe+OjFl8VCXa++ilTnH2VZY5fD6XC9JX3gOWJR2dAC0sYf 0D1gb3YuY1l/ywLJIrpuv7mGvonnJ198bstcvyzUABu6IVv1fliPbOVdW9Wl9hRVq7DnGmreseD BcDftUhDxQji07jzEoXHvMzzq5vSOE252i3w7Mbj7Bg11Y8BlCBPoba6dracQPSsRv/RtMEr5dJ 3z0FQkOJ7GiKYEbkvFQVO9L3izFlPOy7/c4Aulw7KNrcsygT5eucLjk4xWXlol1BA+NRDcJigpi oVvpVPhSxZQ6A7PwScSv7XGpiFbRZoxUq2zgOy2KtidadpvlsYfQ0rQQHgv9Iz/TGsYXMRRcj1P BtzO47A9hqnbwsIKtjF8M/SzRzz6ONDkmXqSARopYv7j7pFCPM/m71K3ZIz+zjGAD3+OzX0hNNg zEaBy5QrjC9eV6G4fOg3U3K57w= X-Received: by 2002:a17:90b:4d07:b0:38e:2517:5d1f with SMTP id 98e67ed59e1d1-39e1e2ee113mr11267726a91.9.1789623044645; Wed, 16 Sep 2026 22:30:44 -0700 (PDT) Received: from mi-OptiPlex-7060.mioffice.cn ([43.224.245.234]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-39e3abada87sm1955836a91.14.2026.09.16.22.30.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Sep 2026 22:30:44 -0700 (PDT) From: Wen Jiang To: akpm@linux-foundation.org, catalin.marinas@arm.com, linux-mm@kvack.org, urezki@gmail.com, will@kernel.org Cc: Xueyuan.chen21@gmail.com, ajd@linux.ibm.com, anshuman.khandual@arm.com, baohua@kernel.org, chleroy@kernel.org, david@kernel.org, dev.jain@arm.com, jiangwen6@xiaomi.com, leo.yan@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, rppt@kernel.org, ryan.roberts@arm.com, Xueyuan Chen Subject: [PATCH v8 09/10] mm/vmalloc: map contiguous pages in batches for vmap() if possible Date: Thu, 17 Sep 2026 13:29:32 +0800 Message-Id: <20260917052933.188679-10-jiangwenxiaomi@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260917052933.188679-1-jiangwenxiaomi@gmail.com> References: <20260917052933.188679-1-jiangwenxiaomi@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: "Barry Song (Xiaomi)" In many cases, the pages passed to vmap() may include high-order pages. For example, the systemheap often allocates pages in descending order: order 8, then 4, then 0. Currently, vmap() iterates over every page individually=E2=80=94even pages inside a high-order block are handled one by one. This patch detects physically contiguous pages (regardless of whether they are compound or non-compound) by scanning with num_pages_contiguous(), and maps them as a single contiguous block whenever possible. The mapping order is determined by taking the minimum of the contiguous page count and the pfn alignment, allowing graceful degradation when pfn alignment is less than the contiguous range. Pages with the same page_shift are coalesced and mapped via vmap_pages_range_noflush_walk() to avoid page table rewalk. As users typically allocate memory in descending orders (e.g. 8 =E2=86=92 4 =E2=86=92 0), once an order-0 page is encountered, we stop sc= anning for contiguous pages since subsequent pages are likely order-0 as well. Signed-off-by: Barry Song (Xiaomi) Co-developed-by: Dev Jain Signed-off-by: Dev Jain Signed-off-by: Wen Jiang Tested-by: Xueyuan Chen Tested-by: Leo Yan --- mm/vmalloc.c | 82 ++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 80 insertions(+), 2 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index ed868afd8194d..bce2d2c09f546 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -3540,6 +3540,85 @@ static inline unsigned int vm_shift(pgprot_t prot, u= nsigned long size) return arch_vmap_pte_supported_shift(size); } =20 +static inline int get_vmap_batch_order(struct page **pages, + pgprot_t prot, unsigned int nr_pages) +{ + unsigned long pfn; + unsigned int nr_contig; + int order; + + if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMAP)) + return 0; + + /* Limit nr_pages by pfn alignment */ + pfn =3D page_to_pfn(*pages); + if (pfn > 0) + nr_pages =3D min_t(size_t, nr_pages, 1UL << __ffs(pfn)); + + nr_contig =3D num_pages_contiguous(pages, nr_pages); + if (nr_contig < 2) + return 0; + + order =3D ilog2(nr_contig); + + if (vm_shift(prot, PAGE_SIZE << order) =3D=3D PAGE_SHIFT) + return 0; + + return order; +} + +static int vmap_pages_range_batched(unsigned long addr, unsigned long end, + pgprot_t prot, struct page **pages) +{ + const unsigned int nr_pages =3D (end - addr) >> PAGE_SHIFT; + unsigned int prev_shift =3D 0, batch_idx =3D 0; + unsigned long batch_start =3D addr, batch_end =3D addr; + int err; + + err =3D kmsan_vmap_pages_range_noflush(addr, end, prot, pages, + PAGE_SHIFT, GFP_KERNEL); + + if (err) + goto out; + + for (unsigned int i =3D 0; i < nr_pages; ) { + unsigned int shift =3D PAGE_SHIFT + + get_vmap_batch_order(pages + i, prot, nr_pages - i); + + if (!i) + prev_shift =3D shift; + + if (shift !=3D prev_shift) { + err =3D vmap_pages_range_noflush_walk(batch_start, batch_end, + prot, pages + batch_idx, prev_shift); + if (err) + goto out; + prev_shift =3D shift; + batch_start =3D batch_end; + batch_idx =3D i; + } + + /* + * Once we fail to batch pages, we expect to fail batching + * for all remaining pages, so just give up. + */ + if (shift =3D=3D PAGE_SHIFT) + break; + + batch_end +=3D 1UL << shift; + i +=3D 1U << (shift - PAGE_SHIFT); + } + + /* Remaining */ + if (batch_start < end) + err =3D vmap_pages_range_noflush_walk(batch_start, end, prot, + pages + batch_idx, prev_shift); + +out: + flush_cache_vmap(addr, end); + return err; +} + /** * vmap - map an array of pages into virtually contiguous space * @pages: array of page pointers @@ -3583,8 +3662,7 @@ void *vmap(struct page **pages, unsigned int count, return NULL; =20 addr =3D (unsigned long)area->addr; - if (vmap_pages_range(addr, addr + size, pgprot_nx(prot), - pages, PAGE_SHIFT) < 0) { + if (vmap_pages_range_batched(addr, addr + size, pgprot_nx(prot), pages) <= 0) { vunmap(area->addr); return NULL; } --=20 2.34.1 From nobody Fri Sep 25 03:17:02 2026 Received: from mail-pj2-f12.google.com (mail-pj2-f12.google.com [74.125.227.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BF4293546DE for ; Thu, 17 Sep 2026 05:30:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.227.140 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789623053; cv=none; b=ZMQzkAY8fhoAGfe4vJjUi4REUNu3Mni/uzpQ9Kt+eNH5cG4RVUpnj3hcHL4K4yzaUTbuYvXTZH1GCiCFGX43T47F85+n+tXjkjJJv7tODK8G18oebJvA1JTIYUD1y1GlpcPoWWL8zbrdc38NsiBnJ5lY/UeaOy+ykT4jXnH596o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1789623053; c=relaxed/simple; bh=x7tKdQKG0sWWcxPRX+uUg1aYDnd/OPPFByjyr1enwkU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=cL2cJeoEHpDw3C4Wv2VBvu8wrOIF2ksQ3AyX+RNU5VVd0lxKVkAHZz7mFHHuY84mYddCLAvKtcxaxdK3IGQVqYdashdM7zQJqhx3MXzTOb6s9uewvgeDMQnr++rA669oRmyhiApYZcDLx2KV9EOdsFVKJOwwnq7ohdy4bUW4YFY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ggCgWO7r; arc=none smtp.client-ip=74.125.227.140 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ggCgWO7r" Received: by mail-pj2-f12.google.com with SMTP id d9443c01a7336-2d8fdc579daso4584705ad.1 for ; Wed, 16 Sep 2026 22:30:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1789623051; x=1790227851; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=/8rYVhg/v6BbsXS7Hdm4I0mlpI54bk8irnJCIlnxFJI=; b=ggCgWO7rVrOciBHAnAwhiTPaAF+iPfbXMiik5N9TXTVDgbmSWi8GkVkjb0T8aul798 BLgLCXLyHJtgOBXiPEjA4eUfj7IXZ64ak3UmUrYLQ7myHCOibsRZ+QdHYVLK8dKxZ0nv j0yHep/LgblNlstKNohfypQlGTwqneuvNsRK7iq4d1QoWb35nLbNwVivo4RiQDq5Em/C Kb8fpPDiMKuF0svgVShkieTqw2iN4ccO1IwPZ3AXlNKpJQz8yFu+eQ3J3AEyBOWD/vVd VvUPna90+iXSCOneYxV5AgR+hP3yFFtAU8RrbMH0YCqigpArFqJVO595n6ZIPVNlfq5v wqUw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1789623051; x=1790227851; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=/8rYVhg/v6BbsXS7Hdm4I0mlpI54bk8irnJCIlnxFJI=; b=oU3INVGYefxj+2bNkeQg5z8II8ncc50x6a1EPDIXPrvxsU3tBAuNNj6k0Ms/0f7P9V z892AkYjiOev17GHw/Ef5MP4r56qyjbJJGA+nz0hxyxssNV3oaeCrOm9eLWVXOzML/K/ 3b0NCOQht1OxK0vb5Ckki/8KhG04mTTVyH/p0y2meE334DkC778sYPmHcj6kEQN8dKrZ fkq/+hCotwEGdxMdNaqzRihEz/RIwBIY9UL6OigIvhgW+YDKy6TRwqfhEf8os8wqamix hdK9u6+dNpbBX3AUb++wO8ELwgHIJl0O00BVdB65gK6pn69zqPR7gdoRT/D1ALkXRhzD SGiA== X-Forwarded-Encrypted: i=1; AKwUvBw/KI4oJcfD5MZDf5zBXpt/IfGTY5hROiQ7orL5dLhNihn/J8k/lApE/MVziZmggbBUjBeWyZMnmfO2p5Y=@vger.kernel.org X-Gm-Message-State: AFuF++k/uivkOdoE/qw35Pf5cyCu6BEUvwQwtjb+LFYSDztgWm4xBKlH 99DKwa3JXADWxspFbdQlJ066lcAgyHpd5v3sWgu0EumG/R1EryqSNJ1E X-Gm-Gg: AYBFou1HmEnxcwRJ9wVU4Cu2rR4fAWBBITHuJCowSqMhDpIFVUCUZ4+p3yeCh930yV5 Xv/lQPLk3tLrYMwpeLJlS9Oslb6poxFoCkoHetaEXej4Uix99PAOar5VAYoHiNn17rHJA/k+gkc 77wRN+Z+y3zbKw1yWnqsNv5EuhNMTV8tFPdiPQKrQ519Q59Ji2U12s/yn0V7V6M358yVoKi3JkT LnVA6zoiKEsWaS/kNO9t/v1LEE9B5oA05da7VUqJKVteZ59u8b/kUZjIrwP8jer6jrEj4vh9aZ4 xshbvN6eoLBXYlvNqS2RdK2ohgFBWaG4ZoT02XS79Ju5/AB3DV5zPLZIhhW70xS3pqTuUitVrAE 5pvqSCn6w8gAOdyN/aBird6g60k6lo9z0ZFCk+bP+KwH591gttkZrmZcdt0sFgkOB7fu8yU19jD jats6eMZ4uqxIo15hRfZvoLS0PkXJdK9z6L+5O2NAQF4n/pNQKXUo9dBmbPOSGCISScCq6eyXj7 W2pa44kiLswWJ8U5WXy3Ml6YKmZcCdLpNUImF8= X-Received: by 2002:a17:90b:2b4c:b0:39d:f024:c7ec with SMTP id 98e67ed59e1d1-39e1e484b5emr13289860a91.16.1789623051007; Wed, 16 Sep 2026 22:30:51 -0700 (PDT) Received: from mi-OptiPlex-7060.mioffice.cn ([43.224.245.234]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-39e3abada87sm1955836a91.14.2026.09.16.22.30.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 16 Sep 2026 22:30:50 -0700 (PDT) From: Wen Jiang To: akpm@linux-foundation.org, catalin.marinas@arm.com, linux-mm@kvack.org, urezki@gmail.com, will@kernel.org Cc: Xueyuan.chen21@gmail.com, ajd@linux.ibm.com, anshuman.khandual@arm.com, baohua@kernel.org, chleroy@kernel.org, david@kernel.org, dev.jain@arm.com, jiangwen6@xiaomi.com, leo.yan@arm.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, rppt@kernel.org, ryan.roberts@arm.com, Xueyuan Chen Subject: [PATCH v8 10/10] mm/vmalloc: align vm_area so vmap() can batch mappings Date: Thu, 17 Sep 2026 13:29:33 +0800 Message-Id: <20260917052933.188679-11-jiangwenxiaomi@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260917052933.188679-1-jiangwenxiaomi@gmail.com> References: <20260917052933.188679-1-jiangwenxiaomi@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: "Barry Song (Xiaomi)" Try to align the vmap virtual address to PMD_SHIFT or a larger PTE mapping size hinted by the architecture, so contiguous pages can be batch-mapped when setting PMD or PTE entries. We do not expect any significant overhead for this. Add __get_vm_area_node_aligned_caller() as a wrapper over __get_vm_area_node() to simplify repeated calls with fixed arguments. Signed-off-by: Barry Song (Xiaomi) Signed-off-by: Wen Jiang Tested-by: Xueyuan Chen Tested-by: Leo Yan Reviewed-by: Uladzislau Rezki (Sony) Reviewed-by: Dev Jain --- mm/vmalloc.c | 35 ++++++++++++++++++++++++++++++++++- 1 file changed, 34 insertions(+), 1 deletion(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index bce2d2c09f546..84413def1d771 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -3306,6 +3306,14 @@ struct vm_struct *get_vm_area_caller(unsigned long s= ize, unsigned long flags, NUMA_NO_NODE, GFP_KERNEL, caller); } =20 +static struct vm_struct *__get_vm_area_node_aligned_caller(unsigned long s= ize, + unsigned long align, unsigned long flags, const void *caller) +{ + return __get_vm_area_node(size, align, PAGE_SHIFT, flags, + VMALLOC_START, VMALLOC_END, + NUMA_NO_NODE, GFP_KERNEL, caller); +} + /** * find_vm_area - find a continuous kernel virtual area * @addr: base address @@ -3619,6 +3627,30 @@ static int vmap_pages_range_batched(unsigned long ad= dr, unsigned long end, return err; } =20 +static struct vm_struct *vmap_get_aligned_vm_area(unsigned long size, + unsigned long flags, pgprot_t prot, const void *caller) +{ + struct vm_struct *vm_area; + unsigned int shift; + + if (arch_vmap_pmd_supported(prot) && size >=3D PMD_SIZE) { + vm_area =3D __get_vm_area_node_aligned_caller(size, PMD_SIZE, + flags, caller); + if (vm_area) + return vm_area; + } + + shift =3D arch_vmap_pte_supported_shift(size); + if (shift > PAGE_SHIFT) { + vm_area =3D __get_vm_area_node_aligned_caller(size, 1UL << shift, + flags, caller); + if (vm_area) + return vm_area; + } + + return __get_vm_area_node_aligned_caller(size, PAGE_SIZE, flags, caller); +} + /** * vmap - map an array of pages into virtually contiguous space * @pages: array of page pointers @@ -3657,7 +3689,8 @@ void *vmap(struct page **pages, unsigned int count, return NULL; =20 size =3D (unsigned long)count << PAGE_SHIFT; - area =3D get_vm_area_caller(size, flags, __builtin_return_address(0)); + area =3D vmap_get_aligned_vm_area(size, flags, prot, + __builtin_return_address(0)); if (!area) return NULL; =20 --=20 2.34.1