From nobody Mon Feb 9 04:28:58 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C0BDC77B75 for ; Tue, 23 May 2023 17:10:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238016AbjEWRK5 (ORCPT ); Tue, 23 May 2023 13:10:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45326 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229506AbjEWRKz (ORCPT ); Tue, 23 May 2023 13:10:55 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3546BB5 for ; Tue, 23 May 2023 10:10:54 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id BF26D6347B for ; Tue, 23 May 2023 17:10:53 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B0CA5C433EF; Tue, 23 May 2023 17:10:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1684861853; bh=iYEi+B9sslkLJiksbVO2kVvlfMuA7iAdpf02AyS8igQ=; h=From:To:Cc:Subject:Date:From; b=PhDRPvYZILhYEYCmQtt/slf03Yh5MHR8+iyRxWyhTh0eq+BhqtiOWmp610Nbsl1oh zpMvW7cAZ7KmLytiPFn5OwUU7+NH2z0jMIFQ2xdKZU5QrSiMMsHbXcSToHXwnN33D2 R53shFR+HN4q3PTe3gqtoRz5M/oYvlNGV+NYyt+ctmKTaLaDcEtX1DL74HQRqGyg/d /IuV75WmSi2z9yzargQmPsWNyXPRCST0duDsSFI06XF9mPeBUnX3yVhu5b9AnzYHjx iK2GXx+uNq0FtyeDKxJjxeUxUhiE4/cDDyDuFbNVkvz0mZEpMEkOVfcoquSUIfTZpB nekLt4xeKLHQg== From: Jisheng Zhang To: Paul Walmsley , Palmer Dabbelt , Albert Ou Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, Suren Baghdasaryan Subject: [PATCH] riscv: mm: try VMA lock-based page fault handling first Date: Wed, 24 May 2023 00:59:42 +0800 Message-Id: <20230523165942.2630-1-jszhang@kernel.org> X-Mailer: git-send-email 2.40.0 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Attempt VMA lock-based page fault handling first, and fall back to the existing mmap_lock-based handling if that fails. A simple running the ebizzy benchmark on Lichee Pi 4A shows that PER_VMA_LOCK can improve the ebizzy benchmark by about 32.68%. In theory, the more CPUs, the bigger improvement, but I don't have any HW platform which has more than 4 CPUs. This is the riscv variant of "x86/mm: try VMA lock-based page fault handling first". Signed-off-by: Jisheng Zhang Reviewed-by: Guo Ren Reviewed-by: Kefeng Wang Reviewed-by: Suren Baghdasaryan --- Any performance numbers are welcome! Especially the numbers on HW platforms with 8 or more CPUs. arch/riscv/Kconfig | 1 + arch/riscv/mm/fault.c | 33 +++++++++++++++++++++++++++++++++ 2 files changed, 34 insertions(+) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 62e84fee2cfd..b958f67f9a12 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -42,6 +42,7 @@ config RISCV select ARCH_SUPPORTS_DEBUG_PAGEALLOC if MMU select ARCH_SUPPORTS_HUGETLBFS if MMU select ARCH_SUPPORTS_PAGE_TABLE_CHECK if MMU + select ARCH_SUPPORTS_PER_VMA_LOCK if MMU select ARCH_USE_MEMTEST select ARCH_USE_QUEUED_RWLOCKS select ARCH_WANT_DEFAULT_TOPDOWN_MMAP_LAYOUT if MMU diff --git a/arch/riscv/mm/fault.c b/arch/riscv/mm/fault.c index 8685f85a7474..eccdddf26f4b 100644 --- a/arch/riscv/mm/fault.c +++ b/arch/riscv/mm/fault.c @@ -286,6 +286,36 @@ void handle_page_fault(struct pt_regs *regs) flags |=3D FAULT_FLAG_WRITE; else if (cause =3D=3D EXC_INST_PAGE_FAULT) flags |=3D FAULT_FLAG_INSTRUCTION; +#ifdef CONFIG_PER_VMA_LOCK + if (!(flags & FAULT_FLAG_USER)) + goto lock_mmap; + + vma =3D lock_vma_under_rcu(mm, addr); + if (!vma) + goto lock_mmap; + + if (unlikely(access_error(cause, vma))) { + vma_end_read(vma); + goto lock_mmap; + } + + fault =3D handle_mm_fault(vma, addr, flags | FAULT_FLAG_VMA_LOCK, regs); + vma_end_read(vma); + + if (!(fault & VM_FAULT_RETRY)) { + count_vm_vma_lock_event(VMA_LOCK_SUCCESS); + goto done; + } + count_vm_vma_lock_event(VMA_LOCK_RETRY); + + if (fault_signal_pending(fault, regs)) { + if (!user_mode(regs)) + no_context(regs, addr); + return; + } +lock_mmap: +#endif /* CONFIG_PER_VMA_LOCK */ + retry: mmap_read_lock(mm); vma =3D find_vma(mm, addr); @@ -355,6 +385,9 @@ void handle_page_fault(struct pt_regs *regs) =20 mmap_read_unlock(mm); =20 +#ifdef CONFIG_PER_VMA_LOCK +done: +#endif if (unlikely(fault & VM_FAULT_ERROR)) { tsk->thread.bad_cause =3D cause; mm_fault_error(regs, addr, fault); --=20 2.40.1