From nobody Fri Jun 12 14:18:14 2026 Received: from mail-pg1-f174.google.com (mail-pg1-f174.google.com [209.85.215.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4272E3815FE for ; Thu, 14 May 2026 09:41:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778751704; cv=none; b=iz5cJIhSNhHEYJgR14ZzNo7e68DUUsID4ewOFs9u+fO/XQw6O9q44eR0KWRk+WZ+UuJsWcY+WckLwY4rZ5mROGd58K7L+X1tKn3nXYq1axRl3ulDjR85TGHu4+Qt56JczApXsUwPA578PbAvliGVo5Uk7baYp4RWMNgETFySAr8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778751704; c=relaxed/simple; bh=CjuFSFevwXnhqOk1t7jckI2gag1P2Xtfjv6l5xGEXFo=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=LkYZNfiikDjCkBuydLe4JAlBIpIu4ZRHIg6p5PpoHsLU3gRe8ANdoZfI/gNEOpZt0EE9XVklz2eGQ56zyMFZwIMoI1yU2hGE3W/OWCV7/x7klgjJ+WP3uWFbDCcSCHg7VeSVieDSZq1hgmrySqjMs44Aub69wuawVRAzzdOcj/U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=hjba0mXM; arc=none smtp.client-ip=209.85.215.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hjba0mXM" Received: by mail-pg1-f174.google.com with SMTP id 41be03b00d2f7-c82a6278a4cso2037665a12.3 for ; Thu, 14 May 2026 02:41:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1778751702; x=1779356502; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gRd7Ld3GiaVdtiEOte4Ul5KoBjr/FrRH6YpNK5Is7Rw=; b=hjba0mXMaH5dr8jE7LXG4eR9rcp2fA8SPbfw8vOP0K5FmpntOtI1Q5/d8NKww6OkO+ c72/x30dz3jQmX4kklTE8J1w24U6QNgTMBHUFs9FSmNMMu0khADJfWowQ8mIcaaPzwu8 fM4TivuNuJds1xLwAiCnN+tTOrl+QZo6ESGkcwKUQMGr4mQAv69UW0sC5pZj5e8K3v4o q8Va2Wqeh71Asg9KLEBFtmGLOvzEK/Wkq7wX1lBFDW9lw0UcSNVX16RNNPCwKgAHw/dK C04RdoFh94pIkdWxFv4HnNjsI3O/O2b9FgcEnFT+Cr6YEJCmWBhAzA2Q7SvO89VqTrjI j+CA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778751702; x=1779356502; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=gRd7Ld3GiaVdtiEOte4Ul5KoBjr/FrRH6YpNK5Is7Rw=; b=dFgzNFYRobot1TzqMxBEtwBKQ3hsf59amddRp+gT8ifLGrK9dfIP2NtqtV7K6l6kE9 Y5TKfCt2kCBINbnE9+eGaeQA/5xp/IOOvRjf32msaQZhPW3q1+xIBLsSFEvv+57znwB7 IGv/BNAW7/Parg4w1LcAbXB/diat6dKvZzW1CrUbi86u2OZ0z+3FpMMo5Z9IIFwS3OO6 TBd8JXWNjbO8fVE5PQ8V/dQbFQOYD4AxKYW3Jk8cZxgO39OwYehDcEz8DmF7D8lrvhES eTlwvJBo922zlWr1wnpD+m/K+XOa/WT28lOm3TC1nCgml/b6hY5uvkHg0Yi0Ea6KqK6H RcCg== X-Forwarded-Encrypted: i=1; AFNElJ9vcykm3RwXyWCObQLRQaRtWfRCNMQRKlA4lic6BtihSBGWu9nDxcwaIit9e/F+bG7qhh2dpvJnZaXRVTs=@vger.kernel.org X-Gm-Message-State: AOJu0YwcMCTIWCvcCV7wVpDKxSuAhM3rOjvQIlSmVNbnLTOoN0DXWvwm YYLpvYvc5WguO4k40ifn6gH9jCn0Hu6muZGfarW4QuiqCryhRf5mVsiiW0ZyZvp/t4E= X-Gm-Gg: Acq92OEj2O6DL0a30gVjpEVX8nXYFtZt09hckNPyW1+tNgywfGcZsPmt+snrI1jHKwb SJJ0UfOwAwNSw6OAGjkw8VOjZNuepb8OY7Z4CSZsONUKHxaLX/bCoyZ2JG+oiP1ruoDMjoURbjr L/Za5G0anM69grGq2m/xDJHLerzJpicKj2Q8X1e6hvKNo6O/av1VOBCz2M67ZSYHLOTRIGNjv21 ODXcWSznujl9jVSziWCr3Sk4eUxITa5qCcyyTe3F0joV9lPiYvKm9j/zDoDnU2zHjIe6+11q3SU sGVv7I8NJRx3av0dQaBwJwcNFzU1ptHlIxeX62LNxsFo8PD+VMRdXcTW4zaqTJJknttKuSk1uaY lvjNTXu6liey174tlmtispF/b2oOIGPWmD/pP2DwlSPHzjxnGy46Nt5GHnuXRG/DkegfJ9+WdCA sxvfzVAinT+QZHtah5U7ijvCJz/tbCh+vZdVah0f7b8yAYuAPgBRt2QlaJ X-Received: by 2002:a05:6a20:9389:b0:398:a1ca:7a2a with SMTP id adf61e73a8af0-3afb052129cmr7620166637.48.1778751702487; Thu, 14 May 2026 02:41:42 -0700 (PDT) Received: from mi-OptiPlex-7060.mioffice.cn ([43.224.245.234]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-c82bb114a70sm2351244a12.22.2026.05.14.02.41.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 May 2026 02:41:42 -0700 (PDT) From: Wen Jiang X-Google-Original-From: Wen Jiang To: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, catalin.marinas@arm.com, will@kernel.org, akpm@linux-foundation.org, urezki@gmail.com Cc: baohua@kernel.org, Xueyuan.chen21@gmail.com, dev.jain@arm.com, rppt@kernel.org, david@kernel.org, ryan.roberts@arm.com, anshuman.khandual@arm.com, ajd@linux.ibm.com, linux-kernel@vger.kernel.org, Wen Jiang , Xueyuan Chen Subject: [PATCH v2 1/7] arm64/hugetlb: Extend batching of multiple CONT_PTE in a single PTE setup Date: Thu, 14 May 2026 17:41:02 +0800 Message-Id: <20260514094108.2016201-2-jiangwen6@xiaomi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260514094108.2016201-1-jiangwen6@xiaomi.com> References: <20260514094108.2016201-1-jiangwen6@xiaomi.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: "Barry Song (Xiaomi)" For sizes aligned to CONT_PTE_SIZE and smaller than PMD_SIZE, we can batch CONT_PTE settings instead of handling them individually. Signed-off-by: Barry Song (Xiaomi) Signed-off-by: Wen Jiang Tested-by: Xueyuan Chen --- arch/arm64/mm/hugetlbpage.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index 30772a909..d477a9dd1 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -110,6 +110,12 @@ static inline int num_contig_ptes(unsigned long size, = size_t *pgsize) contig_ptes =3D CONT_PTES; break; default: + if (size > 0 && size < PMD_SIZE && + IS_ALIGNED(size, CONT_PTE_SIZE)) { + contig_ptes =3D size >> PAGE_SHIFT; + *pgsize =3D PAGE_SIZE; + break; + } WARN_ON(!__hugetlb_valid_size(size)); } =20 @@ -359,6 +365,10 @@ pte_t arch_make_huge_pte(pte_t entry, unsigned int shi= ft, vm_flags_t flags) case CONT_PTE_SIZE: return pte_mkcont(entry); default: + if (pagesize > 0 && pagesize < PMD_SIZE && + IS_ALIGNED(pagesize, CONT_PTE_SIZE)) + return pte_mkcont(entry); + break; } pr_warn("%s: unrecognized huge page size 0x%lx\n", --=20 2.34.1 From nobody Fri Jun 12 14:18:14 2026 Received: from mail-pg1-f174.google.com (mail-pg1-f174.google.com [209.85.215.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8C3CD3A168E for ; Thu, 14 May 2026 09:41:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778751709; cv=none; b=d6LaRaqnEWFzWRFoGv+MS+aTQiKWp4QvCX//AjxFdQfitenrjyBaJBsAGbWwsZ9CqRdvlBFAl8gcaT0g4OZ7bUT+K20i7U9WZ02QAcTlhwbDaM9xIR5xfYXlFXltTejFwYWLV1AUdk1lyOQSBGm8sOEzJkUOUpcRIMoqbxzG710= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778751709; c=relaxed/simple; bh=BZN3J4TDBwxVNoJbkUmYV/DxYikTLyIlESQRFD/HceU=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=nkZGPiBQMmoz1eBIpsG87mAL+9oGyXZBAvm8ceRIKOXJyQsc991wAfoo2k9P1bFgSm8j6Glxh99YolWkLwoa3N8aPeZq2QNydS0E+JA+yrV/WJEImHEl0+Jr97XF+MKmhrDwWDATEdgHIu86DVGdxzoequ1uAGllA1Tx5ERNNmY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=YEMZQH1q; arc=none smtp.client-ip=209.85.215.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="YEMZQH1q" Received: by mail-pg1-f174.google.com with SMTP id 41be03b00d2f7-c80167f5716so3453942a12.2 for ; Thu, 14 May 2026 02:41:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1778751708; x=1779356508; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4izeGUE5yc6CqsffIoYTcEPD1HGMIlHVDl1/1KC/ZM0=; b=YEMZQH1qKQftGI1RXz7ki31POwI/wfbZX6k8TkLlB23hZNawr7oFjAOsXd7Em4rd1a YsBJrBUg/tn9c+ox6FSxxVpErJPUi66MfZoERVguEgZsxkdme5p6ldJSiPdJEZHE3vRM WNOJIFMRl36+KioZsR0z9VqHkukbKNOflNhVVrtXTIHyWpanae9ari2d/MPweHMx4NOx Az7CgRZ6o1s5PyKlU0smQxzD5czl+anR2DrGvS8Kn7+2yI9oqXSSe9Au0u6ppCSsmOwh M8e2WfpNC1Uiv7LEa1JSZ4VxawgUr3JeYM9Kw6j1yKJQBYczEveJSPzskSNCM26AEyMx Gnjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778751708; x=1779356508; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=4izeGUE5yc6CqsffIoYTcEPD1HGMIlHVDl1/1KC/ZM0=; b=QTwGIdhCc2hyU5yCto5XJgHJKX/Qn0lpF0XB+LS6pxZ6+S2UpaL1ot6WiMnIV3c/Xt e7SniVihAED8zGjRomYGbui3k+ieDfyI2ILFDbpWMDoWB/+Fzj7prsd4zRU5yzfztgoR O4gbEaB4bcRb5BLWdpTCIj2gskFv0kmwlsOnTC+uyXCEFfjABsSLPwT6AejKn7q0rvji DTdjIg0INR7pz4S+7aLBPs2oYkB4eyFRk7eRJaqrwsReayfu6PF77oG5gk3zQ3MZLNTa P948dmau3VzJkRBlV98Rl5EPvYWOiW/ViRTIpxmJgZFQjoYmVShidMgYUVITYn2a9HOr Umpw== X-Forwarded-Encrypted: i=1; AFNElJ9kwI2HXsvM2Sv8UJUWzhjxqJ8TVxqIPrns5gu+sQNYcIHpS7UHgmR53trHDCFY9nRu4U7I1NCunovIPx8=@vger.kernel.org X-Gm-Message-State: AOJu0Yxd2ztTvjQ2GrPBf9ShjdcC7NySJfoWTCYKhttR9yU1lLmaUEjV FYco+BT0sIos53MSmBl9XHvdGLpd/53lyc2uocozrc+JgKtcNnMNX2Dd X-Gm-Gg: Acq92OHecTKzvY7kI+Ddgukw+yYrgluVa5iORRBCROaAjiggNno3wjvxtHwmzf2dMyP 9YjyMb+2Awq2DQQU8MlnRS/68AMmnfgZ3m5OxHUCkjEB3m8TrT4Z/mHYhhPqZS8aYsisywbtRa8 GYVlFsNDQYTUGBp19K34mfVokMm8OaJF2tGtAvQe9J5XP/SKMiYoXhEAR91WSDndpEZJJ2ekP3q 4/948Igg0zY+l+hhyR4oEAU8b62LGIlFjJmn3UXeJPlt+jjVjtCvc+PuuyDHFgvbZjIOiD3pkox rJSVelH4eYagCL8GL39HadbKzlHWoxQm+Uvbb9Y+W85BrlBjLSi/ruoNydGhflJNvSREuLhfpRq ZDbXSmakDwrqesxw9ZeTqT2h99ptC4gEpZ3YRaE8+09Ge+CYMxt+d56R6KzoG49CrXjzTqbqvHx 4WHbR39H+m5e2mVUOgtacuqWT8Q6Zzsx10UiBZPTtLen71sY6q7WOG+8zq X-Received: by 2002:a05:6300:6c07:b0:398:abe1:eaac with SMTP id adf61e73a8af0-3af8137634cmr5712299637.27.1778751707654; Thu, 14 May 2026 02:41:47 -0700 (PDT) Received: from mi-OptiPlex-7060.mioffice.cn ([43.224.245.234]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-c82bb114a70sm2351244a12.22.2026.05.14.02.41.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 May 2026 02:41:47 -0700 (PDT) From: Wen Jiang X-Google-Original-From: Wen Jiang To: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, catalin.marinas@arm.com, will@kernel.org, akpm@linux-foundation.org, urezki@gmail.com Cc: baohua@kernel.org, Xueyuan.chen21@gmail.com, dev.jain@arm.com, rppt@kernel.org, david@kernel.org, ryan.roberts@arm.com, anshuman.khandual@arm.com, ajd@linux.ibm.com, linux-kernel@vger.kernel.org, Wen Jiang , Xueyuan Chen Subject: [PATCH v2 2/7] arm64/vmalloc: Allow arch_vmap_pte_range_map_size to batch multiple CONT_PTE Date: Thu, 14 May 2026 17:41:03 +0800 Message-Id: <20260514094108.2016201-3-jiangwen6@xiaomi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260514094108.2016201-1-jiangwen6@xiaomi.com> References: <20260514094108.2016201-1-jiangwen6@xiaomi.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: "Barry Song (Xiaomi)" Allow arch_vmap_pte_range_map_size to batch multiple CONT_PTE hugepages, reducing both PTE setup and TLB flush iterations. Signed-off-by: Barry Song (Xiaomi) Signed-off-by: Wen Jiang Tested-by: Xueyuan Chen --- arch/arm64/include/asm/vmalloc.h | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/vmalloc.h b/arch/arm64/include/asm/vmal= loc.h index 4ec1acd3c..9eea06d0f 100644 --- a/arch/arm64/include/asm/vmalloc.h +++ b/arch/arm64/include/asm/vmalloc.h @@ -23,6 +23,8 @@ static inline unsigned long arch_vmap_pte_range_map_size(= unsigned long addr, unsigned long end, u64 pfn, unsigned int max_page_shift) { + unsigned long size; + /* * If the block is at least CONT_PTE_SIZE in size, and is naturally * aligned in both virtual and physical space, then we can pte-map the @@ -40,7 +42,9 @@ static inline unsigned long arch_vmap_pte_range_map_size(= unsigned long addr, if (!IS_ALIGNED(PFN_PHYS(pfn), CONT_PTE_SIZE)) return PAGE_SIZE; =20 - return CONT_PTE_SIZE; + size =3D min3(end - addr, 1UL << max_page_shift, PMD_SIZE >> 1); + size =3D 1UL << (fls(size) - 1); + return size; } =20 #define arch_vmap_pte_range_unmap_size arch_vmap_pte_range_unmap_size --=20 2.34.1 From nobody Fri Jun 12 14:18:14 2026 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BAAAC3A168E for ; Thu, 14 May 2026 09:41:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778751715; cv=none; b=lwySwc8eT2G+7qGg1aUB/LyXmH/P96Mp4lDk1DV8ViniFKHHVYpZVaSIzai1M1lK7b5ngFzPreDg6pyG4tzEmpnSuF4/LrjPoEesf7Fx8qZZ6w8LBXW5EyFcl0O8NA8HFIbhJvlAK5Ai8iljQ6OI/yOhKSTx80rDDPraxuVzG8A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778751715; c=relaxed/simple; bh=tzYKuF3rV2XdOfpQu9eYuMgVaBuh4flMpdNgBiTP7HE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=WtH3lU41Fml09tULwS78xTQCCbIeDad7x3W1A7NVPBwLk2/YmT2VyWG7/pyx5Unv3GcjzIOXR0cxfHEQldVwba32bJlMNwAOPMx65RR89fSPa7MXcQczDN1OZcdGzt4Jca7kluuCDB32w1J3N9JydPWHj187sCGKzjSi70IezN8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Kv5VT389; arc=none smtp.client-ip=209.85.210.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Kv5VT389" Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-835c98b659eso5208985b3a.0 for ; Thu, 14 May 2026 02:41:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1778751713; x=1779356513; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=F795Xv5J46ljVi2+6iHBT2O82gjybUH4psWzRnSB2TA=; b=Kv5VT389DhD5UTpFD8vbpQjSq7FBjT9llgg8I4y5MJAWX6ld/b/eOjw3jiaG24VK8C C/8bCm9fSkRt+fwoch1Vq916NDPDdQBt8gVd/JKfdgOcE/O3nYOh1xsPDq9wZKSnxf+q dYTlAWuqZQR73z1OwycEfKWfT286IGicFM6ZsCRDldnKKwJUT6BvwDcn+fwlG284GUuS RA+CmztVBMoc3DnopNXNP8iiHBbJ8CI1yfl15x1376j3KN+LqQCYFUoZP1hfoFZi8cRo BoFvWF3Oej0a631VnNG59BMog3cAia2rL5Mt4ufNrpxaFlsHMuZDZHltYNOXgGZHRtIc Tk1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778751713; x=1779356513; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=F795Xv5J46ljVi2+6iHBT2O82gjybUH4psWzRnSB2TA=; b=IGF2h1gjnfgsDosIZnFNdSDPinkNDPoH2Wcfw9y3z/L+Gkd9Kz9rFFzdNDbOnVqtIf CxvDbn2knZX2sZAkn20IJphXF2NCyEdL7RiqZUfxA7CFjTLw3NeaUN4/JiHxWiu9sdcr btbrcrYjpU4fqQfPg7DzoLsm+3vtMgSqcvBEfKH+76C9NiQLrTs5POhn1ZqgVCNpNpB2 dTf8v/Tgz/DYbK8Qgkn7h+ev1GfZHNyY5MM+l8EesY18MB+sTH6c5ImDB04fkf0cB8nI aC3EodVu4la75Uc5N3GEoOdqL3DO2RpZvPBuUklSB8hvodl+Uxj7QR3IpW1oNk8auO6M w7+g== X-Forwarded-Encrypted: i=1; AFNElJ8+uShgJDZm4fwHE59u2uvFuocqlFiTV7Zf0imhJ5H+qVLvtircLTy345pu312uYbeM5Nnyo7qWBWp0Bo0=@vger.kernel.org X-Gm-Message-State: AOJu0YwiQFR/NhK5fNH/BVKWINXmlxKijv/S34e9Ff39hxVGcfofuyBd 6y191OcfsAJ7PprE03qAN3mmwy18lX3qVxtMP7QZ4Hlpba89Wn/hS4NS X-Gm-Gg: Acq92OFKmwwgrHET7lspEXljGPgzwUlIGBmxXumR8ldhYYvR3ir9BrL4HP2PMSsBzzn PAcYopWP/+2mBqRJehDHZYLIjBLabafmVH8oCiYydGljKq2PwMUo3XPvSieiwmgo127LiL0q+wJ sgKcPOs5qHjW/EwRokxE5kKWyrWozmrDd8qSWZdPF9JnKkPmZwVKwy2VPwGiazkGKpysKhx5veJ NbmN9ZJhYdpDI9dqA7p+h6fvoXn1demDj9v1O03xA6Ih3s3pgFZDLdIaFTPIeN3887QRZoAxgO+ pEhWLPped1JCZLJl9CShsq6Q7uVNiB4vjazrLq5JgVhGSIlR9c3qBqii8FYB36bhvuaByXlX6eL tjaoeFI4/SCzlNJlWnLdToGGJLQsyWlO/y22i81bQeaAp0KsTrvBBQzoj6HGD4f0XbACqy852nv B5rEg4e/4+3rGa+TQcp8MUGfXAgp39HJ+xRq8+DdvrfAPeSFCAZDpMmNtq X-Received: by 2002:a05:6a20:2589:b0:39b:d937:8010 with SMTP id adf61e73a8af0-3af83289f88mr8096707637.45.1778751712910; Thu, 14 May 2026 02:41:52 -0700 (PDT) Received: from mi-OptiPlex-7060.mioffice.cn ([43.224.245.234]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-c82bb114a70sm2351244a12.22.2026.05.14.02.41.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 May 2026 02:41:52 -0700 (PDT) From: Wen Jiang X-Google-Original-From: Wen Jiang To: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, catalin.marinas@arm.com, will@kernel.org, akpm@linux-foundation.org, urezki@gmail.com Cc: baohua@kernel.org, Xueyuan.chen21@gmail.com, dev.jain@arm.com, rppt@kernel.org, david@kernel.org, ryan.roberts@arm.com, anshuman.khandual@arm.com, ajd@linux.ibm.com, linux-kernel@vger.kernel.org, Wen Jiang , Xueyuan Chen Subject: [PATCH v2 3/7] mm/vmalloc: Extract vmap_set_ptes() to consolidate PTE mapping logic Date: Thu, 14 May 2026 17:41:04 +0800 Message-Id: <20260514094108.2016201-4-jiangwen6@xiaomi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260514094108.2016201-1-jiangwen6@xiaomi.com> References: <20260514094108.2016201-1-jiangwen6@xiaomi.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Extract the common PTE mapping logic from vmap_pte_range() into a shared helper vmap_set_ptes(). This handles both CONT_PTE and regular PTE mappings in a single function, preparing for the next patch which will extend vmap_pages_pte_range() to also use this helper. The #ifdef CONFIG_HUGETLB_PAGE guard is moved inside vmap_set_ptes(), so callers no longer need to handle the conditional compilation. Signed-off-by: Wen Jiang Tested-by: Xueyuan Chen --- mm/vmalloc.c | 49 ++++++++++++++++++++++++++++++++++--------------- 1 file changed, 34 insertions(+), 15 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 3e9e5156f..9bfd0aa34 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -91,6 +91,35 @@ struct vfree_deferred { static DEFINE_PER_CPU(struct vfree_deferred, vfree_deferred); =20 /*** Page table manipulation functions ***/ + +/* + * Set PTE mappings for the given PFN. Try CONT_PTE mappings first when + * supported, otherwise fall back to PAGE_SIZE mappings. + * + * Return: mapping size. + */ +static __always_inline unsigned long vmap_set_ptes(pte_t *pte, + unsigned long addr, unsigned long end, u64 pfn, + pgprot_t prot, unsigned int max_page_shift) +{ +#ifdef CONFIG_HUGETLB_PAGE + if (max_page_shift > PAGE_SHIFT) { + unsigned long size; + + size =3D arch_vmap_pte_range_map_size(addr, end, pfn, max_page_shift); + if (size !=3D PAGE_SIZE) { + pte_t entry =3D pfn_pte(pfn, prot); + + entry =3D arch_make_huge_pte(entry, ilog2(size), 0); + set_huge_pte_at(&init_mm, addr, pte, entry, size); + return size; + } + } +#endif + set_pte_at(&init_mm, addr, pte, pfn_pte(pfn, prot)); + return PAGE_SIZE; +} + static int vmap_pte_range(pmd_t *pmd, unsigned long addr, unsigned long en= d, phys_addr_t phys_addr, pgprot_t prot, unsigned int max_page_shift, pgtbl_mod_mask *mask) @@ -98,7 +127,8 @@ static int vmap_pte_range(pmd_t *pmd, unsigned long addr= , unsigned long end, pte_t *pte; u64 pfn; struct page *page; - unsigned long size =3D PAGE_SIZE; + unsigned long size; + unsigned int steps; =20 if (WARN_ON_ONCE(!PAGE_ALIGNED(end - addr))) return -EINVAL; @@ -119,20 +149,9 @@ static int vmap_pte_range(pmd_t *pmd, unsigned long ad= dr, unsigned long end, BUG(); } =20 -#ifdef CONFIG_HUGETLB_PAGE - size =3D arch_vmap_pte_range_map_size(addr, end, pfn, max_page_shift); - if (size !=3D PAGE_SIZE) { - pte_t entry =3D pfn_pte(pfn, prot); - - entry =3D arch_make_huge_pte(entry, ilog2(size), 0); - set_huge_pte_at(&init_mm, addr, pte, entry, size); - pfn +=3D PFN_DOWN(size); - continue; - } -#endif - set_pte_at(&init_mm, addr, pte, pfn_pte(pfn, prot)); - pfn++; - } while (pte +=3D PFN_DOWN(size), addr +=3D size, addr !=3D end); + size =3D vmap_set_ptes(pte, addr, end, pfn, prot, max_page_shift); + steps =3D PFN_DOWN(size); + } while (pte +=3D steps, pfn +=3D steps, addr +=3D size, addr !=3D end); =20 lazy_mmu_mode_disable(); *mask |=3D PGTBL_PTE_MODIFIED; --=20 2.34.1 From nobody Fri Jun 12 14:18:14 2026 Received: from mail-pj1-f51.google.com (mail-pj1-f51.google.com [209.85.216.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 265B53A168E for ; Thu, 14 May 2026 09:41:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778751720; cv=none; b=SZnVqlPdEwHezb6czpJ83NkclBLdBjjP725EGfv3gIFGgXGEILPtQLzSoUn9LEBbACoEsAFqpbI8HTSPN0bdO9FY6Ahn0V5Paf0vuvKQiLqMxnaO4fQLs8BGkCbrT+xXCIV2uscC+fGvrwa0N5BmSamW3IWlP5+JZuzwxwklCCk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778751720; c=relaxed/simple; bh=ryDdaEHstL9MaJXM55j47MViyY/dZmsXY4ga/TFHH6s=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=P2ObVr5TETRmIEpwLfVhf32RYz/2EgH3UiYHCenf1scB8GOfqpA8zklG8IZaCYUlXjXWUncSYz/Y5LkQw0uxTAa/B7qbr3czA8gB0OPF4DpJZome5p+h9BkNVn1uEJmCdX+VmuCcIunWkDO33s9cIJg+zG79erZKu4lsEE08Quo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Q7OMkMPK; arc=none smtp.client-ip=209.85.216.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Q7OMkMPK" Received: by mail-pj1-f51.google.com with SMTP id 98e67ed59e1d1-36608b2f2dcso5583990a91.2 for ; Thu, 14 May 2026 02:41:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1778751718; x=1779356518; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=XAa7S8+lmtacBoTWuk3/nEzfs1vLgSAXTfB0g6A9cN4=; b=Q7OMkMPKN1GPzERqBo33cr9pgjn4R56eqCsLIlVYkhgvSVW6N+eGXbHgwKA8IqB8DI s+znJFo6XcBjXKqzKxyDr/VTlAavRsKLjIP0Yt5/N2B2Ll5UZKNENpfE1haP1XrciK3V RwZNlN8IHfNiarkaumTKrHDXxSGE9XwucJ571CWiVAmIInXuvbXFHS0ZwrmAeeUDyUdl ie8NBHJSfd2gXVH1C16l4cwM9VSmniVCS7Uu0f9k7WhMYyMIT4CKl0ayKw9wcRbXtnuE JSr0OEKpCsbj/hCxKon2KMIIxirA5OFw5Y7DnykFsY2FGKfJoCZX+Ccf4WV0Od4C7kGk aO0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778751718; x=1779356518; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=XAa7S8+lmtacBoTWuk3/nEzfs1vLgSAXTfB0g6A9cN4=; b=KZdw2oz5qBwQRqstGZaydDBHm+yIFiFDdn4ehc+q1lhSTvkTy78Ttmd4lFpr0ILtQl RFqBPgohBjXBdxPhMmw67Fyn+xeri+k8eN0R4uwv26Gy0BwfJ0fMNXEZjkXSOciSISFw HRoCsTVxRlG+SrrjgFOamNTAF+CrZ9C19E9WL62g9cdaIOoircXdzDj6hd7IjJDjLLDB Dh6V+xcf71S3anOaddpCi/Z0bhMnVRQHoTQbTWcBDD9mlqwrLdGliGMP/rMcPFqpAynD 0UfQEL5Y0zrtksuQ3xOIjd9SEee6ZCBuEEyUxfJdhk6yxd5vryu0KvWUIGGblvzQFy0p rpSw== X-Forwarded-Encrypted: i=1; AFNElJ87FM3m3/VPxJ4LHm0ZYDmfJC/M2+8cl3dAOkjMiXYtSKJS5Ylz4aPrt0znb/xlOcqZcfibDLGGptxbDNM=@vger.kernel.org X-Gm-Message-State: AOJu0Yw+0w06OTZ0Dno2fzp4comNX2/jQY7dEzVkHRHC6EJpnkMBjwT7 P4na7qCbkZ/y1aNtGC81K4jbzKeDVxQCkThBuaHiZPQFuRgKw56igkd1 X-Gm-Gg: Acq92OEzlL8krBOdlpJWlUx61VQvrm6UtIV74+NNAazjOU7m3DGL3oOHN5QunJNwiAx fT+zsKJ1K5P6LoqK2hD/ZjOgc8ZENljBLyYv29vr6CwCcmC85/kb6CyNhokxypsUWbZgzvai5l1 62oy+5KMv5HPD6bVJzxtn/XyAlg7sHRp/iOZH751UyMBFazb1ZruptwVWScNP9fabIJMZE6zpjP XnIvQmxpJhI2UWIyDbEDKhwxMhGoqB3f6+kOPPkmhf2duOcGuVujjvKT209N2S4psIw3MRxkIpY Ol7zBAQ5s1uvQsEubpCPowZmIyX7hBXRltEe5TpOAZJnJMKVGS3jIMfQT6p0HdAU0U/nhKiH6oh jF/OMjM4t6zJBfDmqjpv78kXD0K3jCiY4b/Zkpgegfbp4HQ1OweHRXhDCEBQbIPxq72tB9MyJ32 zqFNlCIRQcMb0RCS5i4OaUjBmuLPSHfhIuhSlmg6UK7FVyDA== X-Received: by 2002:a17:90a:1c97:b0:368:ddd7:abcd with SMTP id 98e67ed59e1d1-368fb7992e1mr3845479a91.27.1778751718260; Thu, 14 May 2026 02:41:58 -0700 (PDT) Received: from mi-OptiPlex-7060.mioffice.cn ([43.224.245.234]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-c82bb114a70sm2351244a12.22.2026.05.14.02.41.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 May 2026 02:41:57 -0700 (PDT) From: Wen Jiang X-Google-Original-From: Wen Jiang To: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, catalin.marinas@arm.com, will@kernel.org, akpm@linux-foundation.org, urezki@gmail.com Cc: baohua@kernel.org, Xueyuan.chen21@gmail.com, dev.jain@arm.com, rppt@kernel.org, david@kernel.org, ryan.roberts@arm.com, anshuman.khandual@arm.com, ajd@linux.ibm.com, linux-kernel@vger.kernel.org, Wen Jiang , Xueyuan Chen Subject: [PATCH v2 4/7] mm/vmalloc: Extend page table walk to support larger page_shift sizes and eliminate page table rewalk Date: Thu, 14 May 2026 17:41:05 +0800 Message-Id: <20260514094108.2016201-5-jiangwen6@xiaomi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260514094108.2016201-1-jiangwen6@xiaomi.com> References: <20260514094108.2016201-1-jiangwen6@xiaomi.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: "Barry Song (Xiaomi)" vmap_pages_range_noflush_walk() (formerly vmap_small_pages_range_noflush()) provides a clean interface by taking struct page **pages and mapping them via direct PTE iteration. This avoids the page table rewalk seen when using vmap_range_noflush() for page_shift values other than PAGE_SHIFT. Extend it to support larger page_shift values, and add PMD- and contiguous-PTE mappings as well. Rename it to vmap_pages_range_noflush_walk= () since it now handles more than just small pages. For vmalloc() allocations with VM_ALLOW_HUGE_VMAP, we no longer need to iterate over pages one by one via vmap_range_noflush(), which would otherwise lead to page table rewalk. The code is now unified with the PAGE_SHIFT case by simply calling vmap_pages_range_noflush_walk(). Signed-off-by: Barry Song (Xiaomi) Signed-off-by: Wen Jiang Tested-by: Xueyuan Chen --- mm/vmalloc.c | 64 +++++++++++++++++++++++++++------------------------- 1 file changed, 33 insertions(+), 31 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 9bfd0aa34..516d40650 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -543,8 +543,10 @@ void vunmap_range(unsigned long addr, unsigned long en= d) =20 static int vmap_pages_pte_range(pmd_t *pmd, unsigned long addr, unsigned long end, pgprot_t prot, struct page **pages, int *nr, - pgtbl_mod_mask *mask) + pgtbl_mod_mask *mask, unsigned int shift) { + unsigned long pfn, size; + unsigned int steps; int err =3D 0; pte_t *pte; =20 @@ -575,9 +577,10 @@ static int vmap_pages_pte_range(pmd_t *pmd, unsigned l= ong addr, break; } =20 - set_pte_at(&init_mm, addr, pte, mk_pte(page, prot)); - (*nr)++; - } while (pte++, addr +=3D PAGE_SIZE, addr !=3D end); + pfn =3D page_to_pfn(page); + size =3D vmap_set_ptes(pte, addr, end, pfn, prot, shift); + steps =3D PFN_DOWN(size); + } while (pte +=3D steps, *nr +=3D steps, addr +=3D size, addr !=3D end); =20 lazy_mmu_mode_disable(); *mask |=3D PGTBL_PTE_MODIFIED; @@ -587,7 +590,7 @@ static int vmap_pages_pte_range(pmd_t *pmd, unsigned lo= ng addr, =20 static int vmap_pages_pmd_range(pud_t *pud, unsigned long addr, unsigned long end, pgprot_t prot, struct page **pages, int *nr, - pgtbl_mod_mask *mask) + pgtbl_mod_mask *mask, unsigned int shift) { pmd_t *pmd; unsigned long next; @@ -597,7 +600,20 @@ static int vmap_pages_pmd_range(pud_t *pud, unsigned l= ong addr, return -ENOMEM; do { next =3D pmd_addr_end(addr, end); - if (vmap_pages_pte_range(pmd, addr, next, prot, pages, nr, mask)) + + if (shift =3D=3D PMD_SHIFT) { + struct page *page =3D pages[*nr]; + phys_addr_t phys_addr =3D page_to_phys(page); + + if (vmap_try_huge_pmd(pmd, addr, next, phys_addr, prot, + shift)) { + *mask |=3D PGTBL_PMD_MODIFIED; + *nr +=3D 1 << (shift - PAGE_SHIFT); + continue; + } + } + + if (vmap_pages_pte_range(pmd, addr, next, prot, pages, nr, mask, shift)) return -ENOMEM; } while (pmd++, addr =3D next, addr !=3D end); return 0; @@ -605,7 +621,7 @@ static int vmap_pages_pmd_range(pud_t *pud, unsigned lo= ng addr, =20 static int vmap_pages_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end, pgprot_t prot, struct page **pages, int *nr, - pgtbl_mod_mask *mask) + pgtbl_mod_mask *mask, unsigned int shift) { pud_t *pud; unsigned long next; @@ -615,7 +631,7 @@ static int vmap_pages_pud_range(p4d_t *p4d, unsigned lo= ng addr, return -ENOMEM; do { next =3D pud_addr_end(addr, end); - if (vmap_pages_pmd_range(pud, addr, next, prot, pages, nr, mask)) + if (vmap_pages_pmd_range(pud, addr, next, prot, pages, nr, mask, shift)) return -ENOMEM; } while (pud++, addr =3D next, addr !=3D end); return 0; @@ -623,7 +639,7 @@ static int vmap_pages_pud_range(p4d_t *p4d, unsigned lo= ng addr, =20 static int vmap_pages_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end, pgprot_t prot, struct page **pages, int *nr, - pgtbl_mod_mask *mask) + pgtbl_mod_mask *mask, unsigned int shift) { p4d_t *p4d; unsigned long next; @@ -633,14 +649,14 @@ static int vmap_pages_p4d_range(pgd_t *pgd, unsigned = long addr, return -ENOMEM; do { next =3D p4d_addr_end(addr, end); - if (vmap_pages_pud_range(p4d, addr, next, prot, pages, nr, mask)) + if (vmap_pages_pud_range(p4d, addr, next, prot, pages, nr, mask, shift)) return -ENOMEM; } while (p4d++, addr =3D next, addr !=3D end); return 0; } =20 -static int vmap_small_pages_range_noflush(unsigned long addr, unsigned lon= g end, - pgprot_t prot, struct page **pages) +static int vmap_pages_range_noflush_walk(unsigned long addr, unsigned long= end, + pgprot_t prot, struct page **pages, unsigned int shift) { unsigned long start =3D addr; pgd_t *pgd; @@ -655,7 +671,7 @@ static int vmap_small_pages_range_noflush(unsigned long= addr, unsigned long end, next =3D pgd_addr_end(addr, end); if (pgd_bad(*pgd)) mask |=3D PGTBL_PGD_MODIFIED; - err =3D vmap_pages_p4d_range(pgd, addr, next, prot, pages, &nr, &mask); + err =3D vmap_pages_p4d_range(pgd, addr, next, prot, pages, &nr, &mask, s= hift); if (err) break; } while (pgd++, addr =3D next, addr !=3D end); @@ -678,27 +694,13 @@ static int vmap_small_pages_range_noflush(unsigned lo= ng addr, unsigned long end, int __vmap_pages_range_noflush(unsigned long addr, unsigned long end, pgprot_t prot, struct page **pages, unsigned int page_shift) { - unsigned int i, nr =3D (end - addr) >> PAGE_SHIFT; - WARN_ON(page_shift < PAGE_SHIFT); =20 - if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMALLOC) || - page_shift =3D=3D PAGE_SHIFT) - return vmap_small_pages_range_noflush(addr, end, prot, pages); - - for (i =3D 0; i < nr; i +=3D 1U << (page_shift - PAGE_SHIFT)) { - int err; - - err =3D vmap_range_noflush(addr, addr + (1UL << page_shift), - page_to_phys(pages[i]), prot, - page_shift); - if (err) - return err; + if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMALLOC)) + page_shift =3D PAGE_SHIFT; =20 - addr +=3D 1UL << page_shift; - } - - return 0; + return vmap_pages_range_noflush_walk(addr, end, prot, pages, + min(page_shift, PMD_SHIFT)); } =20 int vmap_pages_range_noflush(unsigned long addr, unsigned long end, --=20 2.34.1 From nobody Fri Jun 12 14:18:14 2026 Received: from mail-pg1-f176.google.com (mail-pg1-f176.google.com [209.85.215.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 65F6B3A8722 for ; Thu, 14 May 2026 09:42:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778751725; cv=none; b=I4Pllw/upVx9eU5l8kHIVbNYPesDvfZi9Fp0tV+MV7CaP2ncA95XksWjKEjOSTeAFbV1q7KY4uOyejEwFG5ewm02WUeNvMEtO6UQAKcV9LF1K6A35vtSqYxLh5fzFIiZXF+npPdbjKR6TMYsxWUSOy0Bs2cx/ByBv9JYvL2BB3w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778751725; c=relaxed/simple; bh=AonKCnwqO2HIRBa6OxCuT1KzKIDFrUrkqow/SjOHkQ0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=p1bEDPDkOQrkVp1tGw01u8SX2dItVjC5Ntwho0Dro+BLn3sX2Ox9qOQCxNX0WiYdBFOWhqqVev3FACCDP0fHsUB8W0ygndW2CDxHpPiBbvtkFHRMeYW4041TuDZ5gDAVrCz1VNCb5WlC3JlQFgMAw4jw4uDJW3MoWAq/1U7rfGE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=sfba11lm; arc=none smtp.client-ip=209.85.215.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="sfba11lm" Received: by mail-pg1-f176.google.com with SMTP id 41be03b00d2f7-c7ffe8eeaf2so3341503a12.0 for ; Thu, 14 May 2026 02:42:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1778751724; x=1779356524; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ug5dkzDgJ2Vn9RO/6uwBXc0123ip/xCq3qTJ4G4/vnU=; b=sfba11lmJNZSyUID25RIr2l3xKFM8QWnEzW+71f7mlOmYnMOmvd0SL9Dt/2eevMWz9 6fPjGi4imIM8VOf79gRQDDFs/Cg8QYaGCQgbdEl5VNcW62RwXN9hRsZQS4G0rjA+lc2z l/y2HF5WNl2f6lCNk8qC0jbOz7diWUYM1xyxImhvP0aCqkmhsqCY7BvEGA1LopeZapXy TyUpe1h2SKNLe5IDKRLnAeyHyn3k51LNA8O9c1+5xM81OyUoNTrqC7sE2IA9Mxcj09dU H7dupWN/dcxL/z9KwH2RZtu4BJ2OMh2yJllftxE2+Aui4QOMIcV/lcYwrLG0PojAX9H9 XbDg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778751724; x=1779356524; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=ug5dkzDgJ2Vn9RO/6uwBXc0123ip/xCq3qTJ4G4/vnU=; b=QZMicPuvkd4cuVDxFkBghYhvzZQB3nvspZVXJpykhEPaUU2jSFKJLbr/IbqTP424xj ylWoVUguBbbr8q76erJCL3rB/vlJl871bU73cbfXGFk45bl9Px45d+BpqDnjBNM3A5MH iKYJybK0ZTJSGxbfKUm06zgc5UNtQNTmPYcOl9rz7LiBZ0RBBCcaXunVpf+LRSx6gY+Z KDEfCS7ZjMfTbagQlI7Pbk3IyAXgF+0JRfrvYlR0jBJAF6Z8F1vAmHlIcSHBWLmna4az hrGJqgGT2jKSaNymOONYWoSY4qKnYElnGlRnmxZ8ksS8ILy+tBe2RMEr9qXE6MOg7Bob NFiA== X-Forwarded-Encrypted: i=1; AFNElJ8+4pXInwn6x6QvKZwNU8y85dWKVG8lMT2xNScTnAR3OM21FTw4ADKpGo0/Xl/YxFvkyYWohmZu1wV03gg=@vger.kernel.org X-Gm-Message-State: AOJu0YzLdOWvnEUcOHzJziF9HXdRwL6zAUYeJugfXzxO1UftZA8VplsO 4V49F8NV8PfmPN/PkkyN8Jg3fD2keQvhLriBnwUTNs7EnOxx+rKrk3tY X-Gm-Gg: Acq92OGQI8XXxh9wpFT/bycylZJ/1rV8Tlfy8h/9oQLd2g2G+EJsOtS7MBoxcrXKtEp 7qZBlNXWBdxhymjBxhMmrIzo5S7u4P9FdIQkzS2U6z4cBbFagvGSvxyZiskRjHjNWETMTbp6gek mleZlo46vErhAUtjDXtrWRfqgVS0XK78gfxK5uK3YUwoACeLOCUYAJTowhlJYqlE6JyH5BvcQnt TO3LDT/En9vfFzEZhV+eAUeuB+i4Pk8tViXykaxH/Irl0oCT4y4epzJTVWuD8C1vF1z0g9bBoP3 pdgpR8Uj3M+SO509iy++WC9C5mZ0iojFQsF1mJ5x8YOWP/MKZgwr345rhLh+MCm08wBRHu1UR+C voLJ8v+zZuPiKLXIKJkd+D3SnFOznZjOLOhjmOROBXPHW5lC9B6o2StwO8NfjKtU8RVlVIEnNj3 /5hyjM+ymPIrg6v6KJa/mUC2Xz8+dHBvBR9Cvc1DsFtSqe5Q== X-Received: by 2002:a05:6a20:3c8e:b0:39b:da83:91aa with SMTP id adf61e73a8af0-3afb2342a48mr7860002637.51.1778751723547; Thu, 14 May 2026 02:42:03 -0700 (PDT) Received: from mi-OptiPlex-7060.mioffice.cn ([43.224.245.234]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-c82bb114a70sm2351244a12.22.2026.05.14.02.41.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 May 2026 02:42:03 -0700 (PDT) From: Wen Jiang X-Google-Original-From: Wen Jiang To: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, catalin.marinas@arm.com, will@kernel.org, akpm@linux-foundation.org, urezki@gmail.com Cc: baohua@kernel.org, Xueyuan.chen21@gmail.com, dev.jain@arm.com, rppt@kernel.org, david@kernel.org, ryan.roberts@arm.com, anshuman.khandual@arm.com, ajd@linux.ibm.com, linux-kernel@vger.kernel.org, Wen Jiang , Xueyuan Chen Subject: [PATCH v2 5/7] mm/vmalloc: map contiguous pages in batches for vmap() if possible Date: Thu, 14 May 2026 17:41:06 +0800 Message-Id: <20260514094108.2016201-6-jiangwen6@xiaomi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260514094108.2016201-1-jiangwen6@xiaomi.com> References: <20260514094108.2016201-1-jiangwen6@xiaomi.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: "Barry Song (Xiaomi)" In many cases, the pages passed to vmap() may include high-order pages. For example, the systemheap often allocates pages in descending order: order 8, then 4, then 0. Currently, vmap() iterates over every page individually=E2=80=94even pages inside a high-order block are handled one by one. This patch detects physically contiguous pages (regardless of whether they are compound or non-compound) by scanning with num_pages_contiguous(), and maps them as a single contiguous block whenever possible. The first page's pfn must be aligned to the mapping order for the batched mapping to be used. Pages with the same page_shift are coalesced and mapped via vmap_pages_range_noflush_walk() to avoid page table rewalk. Signed-off-by: Barry Song (Xiaomi) Co-developed-by: Dev Jain Signed-off-by: Dev Jain Signed-off-by: Wen Jiang Tested-by: Xueyuan Chen --- mm/vmalloc.c | 75 ++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 73 insertions(+), 2 deletions(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 516d40650..c30a7673e 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -3520,6 +3520,77 @@ void vunmap(const void *addr) } EXPORT_SYMBOL(vunmap); =20 +static inline int get_vmap_batch_order(struct page **pages, + unsigned int max_steps, unsigned int idx) +{ + unsigned int nr_contig; + int order; + + if (!IS_ENABLED(CONFIG_HAVE_ARCH_HUGE_VMAP) || + ioremap_max_page_shift =3D=3D PAGE_SHIFT) + return 0; + + nr_contig =3D num_pages_contiguous(&pages[idx], max_steps); + if (nr_contig < 2) + return 0; + + order =3D fls(nr_contig) - 1; + + if (arch_vmap_pte_supported_shift(PAGE_SIZE << order) =3D=3D PAGE_SHIFT) + return 0; + + /* Ensure the first page's pfn is aligned to the order */ + if (!IS_ALIGNED(page_to_pfn(pages[idx]), 1 << order)) + return 0; + + return order; +} + +static int __vmap_huge(unsigned long addr, unsigned long end, + pgprot_t prot, struct page **pages) +{ + unsigned int count =3D (end - addr) >> PAGE_SHIFT; + unsigned int prev_shift =3D 0, idx =3D 0; + unsigned long map_addr =3D addr; + int err; + + err =3D kmsan_vmap_pages_range_noflush(addr, end, prot, pages, + PAGE_SHIFT, GFP_KERNEL); + if (err) + goto out; + + for (unsigned int i =3D 0; i < count; ) { + unsigned int shift =3D PAGE_SHIFT + + get_vmap_batch_order(pages, count - i, i); + + if (!i) + prev_shift =3D shift; + + if (shift !=3D prev_shift) { + err =3D vmap_pages_range_noflush_walk(map_addr, addr, + prot, pages + idx, + min(prev_shift, PMD_SHIFT)); + if (err) + goto out; + prev_shift =3D shift; + map_addr =3D addr; + idx =3D i; + } + + addr +=3D 1UL << shift; + i +=3D 1U << (shift - PAGE_SHIFT); + } + + /* Remaining */ + if (map_addr < end) + err =3D vmap_pages_range_noflush_walk(map_addr, end, + prot, pages + idx, min(prev_shift, PMD_SHIFT)); + +out: + flush_cache_vmap(addr, end); + return err; +} + /** * vmap - map an array of pages into virtually contiguous space * @pages: array of page pointers @@ -3563,8 +3634,8 @@ void *vmap(struct page **pages, unsigned int count, return NULL; =20 addr =3D (unsigned long)area->addr; - if (vmap_pages_range(addr, addr + size, pgprot_nx(prot), - pages, PAGE_SHIFT) < 0) { + if (__vmap_huge(addr, addr + size, pgprot_nx(prot), + pages) < 0) { vunmap(area->addr); return NULL; } --=20 2.34.1 From nobody Fri Jun 12 14:18:14 2026 Received: from mail-pg1-f175.google.com (mail-pg1-f175.google.com [209.85.215.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ADE803A8735 for ; Thu, 14 May 2026 09:42:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778751731; cv=none; b=fb7n05z7vkrFXNhPCH6ewGroVE9eSTU3hb0JCDtSOpLoER1YV0ytNL+8SmJGxXkwr+MjKoDvglw9lkltWYijo0dgjywgYRDVMccKZDn4zyd0x8MH5kX5szi+QvUxStViznskrhI2tyKqpnFQLj4UYBt6tLutRWVzVREzPCRWhDM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778751731; c=relaxed/simple; bh=TmAkXaAw5cJHa+mEh0TSy8Kn5ny/TDE2/zEY35TBm2k=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=VIkMttRu9NGWJATVmqex4NPNAElP8YeBkuBKqzvPOi54gSyIobK9MEBCiLyhZUpi+oZj/lz8s2lbGf1wJ898lL71JPpfSWwjTtNNoBNtW9Q1HnUWVJ6fgH6zVB/YX8GwAiCz2wFte/tA57rv3+BHUuGmU6UPUCKHRi6VwJ/5j9k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=DB8CMlHN; arc=none smtp.client-ip=209.85.215.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="DB8CMlHN" Received: by mail-pg1-f175.google.com with SMTP id 41be03b00d2f7-c80167f56cdso3250685a12.3 for ; Thu, 14 May 2026 02:42:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1778751729; x=1779356529; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rrNbLANiBcQr27ZSAlrK2cxuCnVeM2v9Np9AjqW/V2E=; b=DB8CMlHNZ4YLNwZAtMEri5DjJT5d1U76uncbMR6k9rE6U0nkXV+itYYQ80TFv2T8D2 22bAsCXo8dPoXJdReYxZYm5+vnUeAQvjbBqXQCfChChe1JF6GUphOlU+36Xg71Xc4LkK +UKdqh7ODbr1Taag0rwwv22IqfJXJrCNjvz9ga1i//0t/y6/eA5ozyKV6E5rjpSqZpxA TAWujVFiWodLercgXs0oxORgC4FKjS9DWpugSKxe8z2i06uQWXbER7RNZdk0oNtMxcqq dovILiiChCzfgMDnRWMlFN4JpBqo1wRU6B120s/nsiBWssJclC4UVvz0m43/4RpfpHSQ MRIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778751729; x=1779356529; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=rrNbLANiBcQr27ZSAlrK2cxuCnVeM2v9Np9AjqW/V2E=; b=FftgoSHY6ChVKE8rSM9ef+XGSwtFyiwyeRevvGUvbmU3ymgPa28A9ItNNg56ePt0/p lXdY8XiZ5lskmZ5PBsRFwmtNpmItg8iJeMZXImthh08lwz1WGTzwuwWR2A/Hdq2ymg5k uL7s4E2OwXc8NLZ1jqlMrPUqsDq10jiNsFiHIOrHikCEQA+aDdGYhc0qFfpsPpeTj0yd YPVjg4tn7/yjoB3m/jAmbSehCB2osTibc1KebekzuT13qGqevhey8zXBzJ5Brmi1oU1c FPfOHgTT8nQWE/norIRl8yqnGJi/1P52DUKO9YTVUAc5vVpNePxstApxky9EffC/CcH+ Gihg== X-Forwarded-Encrypted: i=1; AFNElJ9mr6hJeTGMwOKfixkLHSEAkM4uD+vNDXWi88m8JonjuvzJgDANUgvpgXfZy6Yi090b1P72mrtZzMmOKf0=@vger.kernel.org X-Gm-Message-State: AOJu0YyX8FKg9nq1FitYBe4YmwyykYaibhe3ISvs9xdoDaCdrVlNULeJ hgNC6qGAev0rFkEllitWLsKNDgUhF5NtKtIeczF6ZqsULVr2E5nUOBgc X-Gm-Gg: Acq92OFZZYSyiomVBMJ1TZ4bhffB9gxPjgYHbfBGgGtWZTlg5Y1fcQDVd19XIqdUC1M o7DAxgvzghRye+DA4tj5TmSL9Y2WdOqr0+1UBMTjxp1B2jyPWQeeuPZrOd0iluc35ppLxuxFaBC AFXYjCvAP9fIA6+rSlCvLpK/6BLDMCHjfP5CjgyUfpO5nNezDIHg+W+7WxdaTfLuTbJCpDOTC7u IsVl9HEtdY2k8zBgriOx0+2MPtQEwnIgTK09F/G4gyRvXdIhTRGGsFaxF1L++qsEqXTXWIWMNjE 5vi0PxWudI7etr2eUC0OOEE4xxiLG+bL40fDV3XbreIIM7n68DQ5Tw+oCRqi9yE4JKDk3LE34Tw R2At8zXfKDWTTzErMU84yKK/4/UlAzaL2/pedgIniI1l5fNbiL1NAIzwPbao/ZnufcS9sbprvj2 hkSj2lIsC7aW3F0MZ0+OG/u8pGqYAqJFd4dceajraK1RMv8w== X-Received: by 2002:a05:6a21:3390:b0:3aa:60e0:b2ed with SMTP id adf61e73a8af0-3afb1009aabmr7737701637.27.1778751728731; Thu, 14 May 2026 02:42:08 -0700 (PDT) Received: from mi-OptiPlex-7060.mioffice.cn ([43.224.245.234]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-c82bb114a70sm2351244a12.22.2026.05.14.02.42.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 May 2026 02:42:08 -0700 (PDT) From: Wen Jiang X-Google-Original-From: Wen Jiang To: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, catalin.marinas@arm.com, will@kernel.org, akpm@linux-foundation.org, urezki@gmail.com Cc: baohua@kernel.org, Xueyuan.chen21@gmail.com, dev.jain@arm.com, rppt@kernel.org, david@kernel.org, ryan.roberts@arm.com, anshuman.khandual@arm.com, ajd@linux.ibm.com, linux-kernel@vger.kernel.org, Wen Jiang , Xueyuan Chen Subject: [PATCH v2 6/7] mm/vmalloc: align vm_area so vmap() can batch mappings Date: Thu, 14 May 2026 17:41:07 +0800 Message-Id: <20260514094108.2016201-7-jiangwen6@xiaomi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260514094108.2016201-1-jiangwen6@xiaomi.com> References: <20260514094108.2016201-1-jiangwen6@xiaomi.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: "Barry Song (Xiaomi)" Try to align the vmap virtual address to PMD_SHIFT or a larger PTE mapping size hinted by the architecture, so contiguous pages can be batch-mapped when setting PMD or PTE entries. Signed-off-by: Barry Song (Xiaomi) Signed-off-by: Wen Jiang Tested-by: Xueyuan Chen --- mm/vmalloc.c | 31 ++++++++++++++++++++++++++++++- 1 file changed, 30 insertions(+), 1 deletion(-) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index c30a7673e..b3389c8f1 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -3591,6 +3591,35 @@ static int __vmap_huge(unsigned long addr, unsigned = long end, return err; } =20 +static struct vm_struct *get_aligned_vm_area(unsigned long size, unsigned = long flags) +{ + unsigned int shift =3D (size >=3D PMD_SIZE) ? PMD_SHIFT : + arch_vmap_pte_supported_shift(size); + struct vm_struct *vm_area =3D NULL; + + /* + * Try to allocate an aligned vm_area so contiguous pages can be + * mapped in batches. + */ + while (1) { + unsigned long align =3D 1UL << shift; + + vm_area =3D __get_vm_area_node(size, align, PAGE_SHIFT, flags, + VMALLOC_START, VMALLOC_END, + NUMA_NO_NODE, GFP_KERNEL, + __builtin_return_address(0)); + if (vm_area || shift <=3D PAGE_SHIFT) + goto out; + if (shift =3D=3D PMD_SHIFT) + shift =3D arch_vmap_pte_supported_shift(size); + else if (shift > PAGE_SHIFT) + shift =3D PAGE_SHIFT; + } + +out: + return vm_area; +} + /** * vmap - map an array of pages into virtually contiguous space * @pages: array of page pointers @@ -3629,7 +3658,7 @@ void *vmap(struct page **pages, unsigned int count, return NULL; =20 size =3D (unsigned long)count << PAGE_SHIFT; - area =3D get_vm_area_caller(size, flags, __builtin_return_address(0)); + area =3D get_aligned_vm_area(size, flags); if (!area) return NULL; =20 --=20 2.34.1 From nobody Fri Jun 12 14:18:14 2026 Received: from mail-pg1-f176.google.com (mail-pg1-f176.google.com [209.85.215.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C6C1E3A6F03 for ; Thu, 14 May 2026 09:42:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778751736; cv=none; b=CgG97xhcu9AIPgvn5RSSdpEIzObPcA+03fOQ1+9GgTkOWyJx4yWECazrIMIE4AOVtAuQi1YQkxXoEGV2lhHObZSStbTSDS6S+B3pefguOr0AApJYASNIjfUO+JcYqfHt4oqc4Othkc6SqeUtZNGfMsuN0TGimEln7o24l26w+lQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1778751736; c=relaxed/simple; bh=5fy9jItbAgSlWzh5B1zFJI28iVDR6TcG/Hh2Y8lnfyE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=VUcFSU61HjGRDN3FTm+Rc/y+wOl1Ob8VHnOOar+SBz3EfdfV2mm2/D3c08iJgGLlcyN1tPjHksnT6NsWOkGfsCh69YsGPT0NG13mUgQKWcrWtJjaM3cy/IwLSGhEXAquua1oPl6flXyIVVN6NW1a5E/BJjeTm3vsphzBON2E/uU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=mDjiVDck; arc=none smtp.client-ip=209.85.215.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="mDjiVDck" Received: by mail-pg1-f176.google.com with SMTP id 41be03b00d2f7-c8025500cc7so5633043a12.1 for ; Thu, 14 May 2026 02:42:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20251104; t=1778751734; x=1779356534; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=t6f7VCuNqJSNbGtIB9f0dkJuFICST/hbl5c+UwAGsc0=; b=mDjiVDck5COCg+WK4YEu18urU6BrQ/vdxuCfKFLjACQvCZUXHHNjpgHYruGmfkWKzL Aamegfl148dsUSi3RE/ebPeM3zhNli+MdwVninudnMZCqu0GQpuPI/OHws0Bk+iuu6Ms LWsglA3dEzLf4fRCSUHT+zOZWtgoDqZStg1FSIcusVEISkpQ4a/mV4+SbXhbOkZOPZXt iKuHCZSausAL4epe5UVUSBW41BJudC3Zi6bATDB/p5cWyQwdsHxIakEHsvxtS3T4YorW b7OqPEMRwUdA4FNh5YiKuFBz3fij9vel2q263MQQwyF2h47ht2t8gGBDkc34BQiT9ql4 NzvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1778751734; x=1779356534; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to; bh=t6f7VCuNqJSNbGtIB9f0dkJuFICST/hbl5c+UwAGsc0=; b=TMkGLE2o3GaHGrxF7DFzR703tiPxm8YIhNuRUwcXBDhZlhpKA8MZnU3D7orzPIi6pP ILZwvoqg/VfkS9EXrm8aeZ3OulIgBSoJT/bjIWQ2ffySKr4ZkJVH56gh0Ab70+YcrExg iPJVlNC2VCd5nfY06JAfMKkEQY8zsA78AmeQZmBrIwowtnZyLfre7ZZIGGlaSolT95Hm rZIxIRjO1VHdrWA8Acb5yCWG0kEGzZK4mBnNiosGhZfBwLyN7jbEDNjKLtvRw+GplvT8 xoFTHuKYWKL2oEXLZMJ9fBjf9jPssdN8ltF7uAhD+txqQVzFVaMpgjtXrUgPlNv+t+zs /0lg== X-Forwarded-Encrypted: i=1; AFNElJ+QdrY6TPE5IsERvwJsl6G7DIRzhiocVQoNHGp8R+BlhcADOhp/2MBypcTJp1aK3FdTpFVs5D6R7O3Bu2Q=@vger.kernel.org X-Gm-Message-State: AOJu0YxMA1wqVosmTD1QqSrMTGbl9Jcv3E13eRuCMlxqGRYd521QR1RG 5Qwc20Zewhm87K5SveGj95stgiK6gZgA57lwLHdnEwo08y3UKi3ov+OJ X-Gm-Gg: Acq92OF8/l8y8Q7gpgkNN8RuBbsaZ4r128cwcUNNnptbngWqVYMHcw16t0j+gnrm+a4 xyYNdpA5mduLHt0tataxbNCKjZhxj1evig4MNbEUR6Eu4nCSwtNZI8FfZAWjc1iqtWU7fc00gQc N6ErIy1FiynYnQGzv0r8X7f4bRKng8W4aZcSO7YK+59Wd/lWUFrlpagsR9ZTsFqd9fRssr95BGV bwkksO+bgKoohyhUPHDt1NMaNnd5hRtI0C9DyLqsr1kfzqpcxd4vaMCo8sIi7RGjq9yvs9RjN2k Emnhq5zLGLgwYJ4PsniZWf5+oeTLjnPj8JwgGhSDJi6MMPwDOGGE2uGISRWzW1rbFeSt409GB8s HF8WoravOlTSfTWSAAg9MYeA/Gw/DBbpzAVOHCOCvKHORxp83a7Aq3g5wEXzLgoE8DJ2Jo+vNkk d+WuPp1+i6UvlzL1RY7UGP63wtetajDM2qSGYaZkdT4RXo4iaD1e6JHXHc X-Received: by 2002:a05:6a20:7354:b0:39f:3ca8:a33b with SMTP id adf61e73a8af0-3afad844bf8mr7760924637.17.1778751733927; Thu, 14 May 2026 02:42:13 -0700 (PDT) Received: from mi-OptiPlex-7060.mioffice.cn ([43.224.245.234]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-c82bb114a70sm2351244a12.22.2026.05.14.02.42.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 May 2026 02:42:13 -0700 (PDT) From: Wen Jiang X-Google-Original-From: Wen Jiang To: linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org, catalin.marinas@arm.com, will@kernel.org, akpm@linux-foundation.org, urezki@gmail.com Cc: baohua@kernel.org, Xueyuan.chen21@gmail.com, dev.jain@arm.com, rppt@kernel.org, david@kernel.org, ryan.roberts@arm.com, anshuman.khandual@arm.com, ajd@linux.ibm.com, linux-kernel@vger.kernel.org, Wen Jiang , Xueyuan Chen Subject: [PATCH v2 7/7] mm/vmalloc: Stop scanning for compound pages after encountering small pages in vmap Date: Thu, 14 May 2026 17:41:08 +0800 Message-Id: <20260514094108.2016201-8-jiangwen6@xiaomi.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260514094108.2016201-1-jiangwen6@xiaomi.com> References: <20260514094108.2016201-1-jiangwen6@xiaomi.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable From: "Barry Song (Xiaomi)" Users typically allocate memory in descending orders, e.g. 8 =E2=86=92 4 =E2=86=92 0. Once an order-0 page is encountered, subsequent pages are likely to also be order-0, so we stop scanning for compound pages at that point. Signed-off-by: Barry Song (Xiaomi) Signed-off-by: Wen Jiang Tested-by: Xueyuan Chen --- mm/vmalloc.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index b3389c8f1..60579bfbf 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -3576,6 +3576,12 @@ static int __vmap_huge(unsigned long addr, unsigned = long end, map_addr =3D addr; idx =3D i; } + /* + * Once small pages are encountered, the remaining pages + * are likely small as well + */ + if (shift =3D=3D PAGE_SHIFT) + break; =20 addr +=3D 1UL << shift; i +=3D 1U << (shift - PAGE_SHIFT); --=20 2.34.1