From nobody Mon Feb 9 10:50:17 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AC968127B4E; Thu, 4 Apr 2024 13:37:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712237830; cv=none; b=c4CZr+PkHkRDaqQT+rPg/zMlisxeuBHn9USZrmlR+ZQ+ZLrMWpJ/k5yod4QQ4PA3T/NrTq4CclwSLII2X+wd4hT3FJO/F3cGqXT7t4SHNVR067Q1tNp4tYgnSdujAdv2oYopYj3DOsI0wBZCt0kTCKUh3oTds3NtmdPNnqs/RR4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712237830; c=relaxed/simple; bh=jxLpmh8VTKLl4QE0vKweacztjDkwg/HfKpebb5mL/xY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=bcsXnwKZ3RC0BD769sgm3mngKCnmqEr+AVDG5kbnzx48kOuBindMnhQNrTa+KDXWmq6kf/S+l2aELcE4QebiEVUoU/MXW48x/mdg+EF6jEmyN4g7wBzn5qoFLrSNdAvN6jQEClPw+JAWMvQgS51ms8kSRW0b3YFxiOPAjUFJPr4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id A1A4AC433C7; Thu, 4 Apr 2024 13:37:07 +0000 (UTC) From: Huacai Chen To: Andrew Morton , Huacai Chen Cc: loongarch@lists.linux.dev, linux-mm@kvack.org, Xuefeng Li , Guo Ren , Xuerui Wang , Jiaxun Yang , linux-kernel@vger.kernel.org, loongson-kernel@lists.loongnix.cn, Huacai Chen Subject: [PATCH 1/3] mm: Move lowmem_page_address() a little later Date: Thu, 4 Apr 2024 21:36:34 +0800 Message-ID: <20240404133642.971583-2-chenhuacai@loongson.cn> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240404133642.971583-1-chenhuacai@loongson.cn> References: <20240404133642.971583-1-chenhuacai@loongson.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" LoongArch will override page_to_virt() which use page_address() in the KFENCE case (by defining WANT_PAGE_VIRTUAL/HASHED_PAGE_VIRTUAL). So move lowmem_page_address() a little later to avoid such build errors: error: implicit declaration of function 'page_address'. Signed-off-by: Huacai Chen Acked-by: Andrew Morton --- include/linux/mm.h | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 0436b919f1c7..7b0ee64225de 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2207,11 +2207,6 @@ static inline int arch_make_folio_accessible(struct = folio *folio) */ #include =20 -static __always_inline void *lowmem_page_address(const struct page *page) -{ - return page_to_virt(page); -} - #if defined(CONFIG_HIGHMEM) && !defined(WANT_PAGE_VIRTUAL) #define HASHED_PAGE_VIRTUAL #endif @@ -2234,6 +2229,11 @@ void set_page_address(struct page *page, void *virtu= al); void page_address_init(void); #endif =20 +static __always_inline void *lowmem_page_address(const struct page *page) +{ + return page_to_virt(page); +} + #if !defined(HASHED_PAGE_VIRTUAL) && !defined(WANT_PAGE_VIRTUAL) #define page_address(page) lowmem_page_address(page) #define set_page_address(page, address) do { } while(0) --=20 2.43.0 From nobody Mon Feb 9 10:50:17 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 00BE912838A; Thu, 4 Apr 2024 13:37:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712237836; cv=none; b=nIlFPX6+zDmL0vQ3yIkfRvoU0mV6ilC5Ka1Ph1PIqcMJw1GHcZ2b7uIv1H6UK63ZShWGSI3TTZc6pYuptQYBrvD4m83j8IJOcRvkHr/6bzHviqVawHdmD0ELDUyCpPy0jP6IUtfadNH49tiMklLtnLNopSGL0VLFeG1jh0Od5IE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712237836; c=relaxed/simple; bh=ddqupzXUZSofI+WMDJou/va1wQCQQkBHUreBxZzCrko=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=m310uWQaTpqffO5D981dgI2oMILqTDwgSxn3sJfovLpLiJKwIu/KyjyWE8wsKFW1pms5E7D5hCwbtC/yIkEV9cVvut2nWfVc17t0ljEvL0/0U3rhBpiL1YaEJLleO0HaLgRxHb47Q8Z3WXv53gkMUgQQ8hhgiCoVnjgXC/Z6yOM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3756EC433C7; Thu, 4 Apr 2024 13:37:13 +0000 (UTC) From: Huacai Chen To: Andrew Morton , Huacai Chen Cc: loongarch@lists.linux.dev, linux-mm@kvack.org, Xuefeng Li , Guo Ren , Xuerui Wang , Jiaxun Yang , linux-kernel@vger.kernel.org, loongson-kernel@lists.loongnix.cn, Huacai Chen Subject: [PATCH 2/3] LoongArch: Make {virt, phys, page, pfn} translation work with KFENCE Date: Thu, 4 Apr 2024 21:36:35 +0800 Message-ID: <20240404133642.971583-3-chenhuacai@loongson.cn> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240404133642.971583-1-chenhuacai@loongson.cn> References: <20240404133642.971583-1-chenhuacai@loongson.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" KFENCE changes virt_to_page() to be able to translate tlb mapped virtual addresses, but forget to change virt_to_phys()/phys_to_virt() and other translation functions as well. This patch fix it, otherwise some drivers (such as nvme and virtio-blk) cannot work with KFENCE. All {virt, phys, page, pfn} translation functions are updated: 1, virt_to_pfn()/pfn_to_virt(); 2, virt_to_page()/page_to_virt(); 3, virt_to_phys()/phys_to_virt(). DMW/TLB mapped addresses are distinguished by comparing the vaddress with vm_map_base in virt_to_xyz(), and we define WANT_PAGE_VIRTUAL in the KFENCE case for the reverse translations, xyz_to_virt(). Signed-off-by: Huacai Chen --- arch/loongarch/include/asm/io.h | 20 +++++++++++++++----- arch/loongarch/include/asm/kfence.h | 9 +++++++++ arch/loongarch/include/asm/page.h | 26 +++++++++++++++++++++++++- arch/loongarch/mm/pgtable.c | 4 ++-- 4 files changed, 51 insertions(+), 8 deletions(-) diff --git a/arch/loongarch/include/asm/io.h b/arch/loongarch/include/asm/i= o.h index 4a8adcca329b..c2f9979b2979 100644 --- a/arch/loongarch/include/asm/io.h +++ b/arch/loongarch/include/asm/io.h @@ -14,11 +14,6 @@ #include #include =20 -/* - * Change "struct page" to physical address. - */ -#define page_to_phys(page) ((phys_addr_t)page_to_pfn(page) << PAGE_SHIFT) - extern void __init __iomem *early_ioremap(u64 phys_addr, unsigned long siz= e); extern void __init early_iounmap(void __iomem *addr, unsigned long size); =20 @@ -73,6 +68,21 @@ extern void __memcpy_fromio(void *to, const volatile voi= d __iomem *from, size_t =20 #define __io_aw() mmiowb() =20 +#ifdef CONFIG_KFENCE +#define virt_to_phys(kaddr) \ +({ \ + (likely((unsigned long)kaddr < vm_map_base)) ? __pa((unsigned long)kaddr)= : \ + page_to_phys(tlb_virt_to_page((unsigned long)kaddr)) + offset_in_page((un= signed long)kaddr);\ +}) + +#define phys_to_virt(paddr) \ +({ \ + extern char *__kfence_pool; \ + (unlikely(__kfence_pool =3D=3D NULL)) ? __va((unsigned long)paddr) : \ + page_address(phys_to_page((unsigned long)paddr)) + offset_in_page((unsign= ed long)paddr);\ +}) +#endif + #include =20 #define ARCH_HAS_VALID_PHYS_ADDR_RANGE diff --git a/arch/loongarch/include/asm/kfence.h b/arch/loongarch/include/a= sm/kfence.h index 6c82aea1c993..a6a5760da3a3 100644 --- a/arch/loongarch/include/asm/kfence.h +++ b/arch/loongarch/include/asm/kfence.h @@ -16,6 +16,7 @@ static inline bool arch_kfence_init_pool(void) { int err; + char *kaddr, *vaddr; char *kfence_pool =3D __kfence_pool; struct vm_struct *area; =20 @@ -35,6 +36,14 @@ static inline bool arch_kfence_init_pool(void) return false; } =20 + kaddr =3D kfence_pool; + vaddr =3D __kfence_pool; + while (kaddr < kfence_pool + KFENCE_POOL_SIZE) { + set_page_address(virt_to_page(kaddr), vaddr); + kaddr +=3D PAGE_SIZE; + vaddr +=3D PAGE_SIZE; + } + return true; } =20 diff --git a/arch/loongarch/include/asm/page.h b/arch/loongarch/include/asm= /page.h index 44027060c54a..e85df33f11c7 100644 --- a/arch/loongarch/include/asm/page.h +++ b/arch/loongarch/include/asm/page.h @@ -78,7 +78,26 @@ typedef struct { unsigned long pgprot; } pgprot_t; struct page *dmw_virt_to_page(unsigned long kaddr); struct page *tlb_virt_to_page(unsigned long kaddr); =20 -#define virt_to_pfn(kaddr) PFN_DOWN(PHYSADDR(kaddr)) +#define pfn_to_phys(pfn) __pfn_to_phys(pfn) +#define phys_to_pfn(paddr) __phys_to_pfn(paddr) + +#define page_to_phys(page) pfn_to_phys(page_to_pfn(page)) +#define phys_to_page(paddr) pfn_to_page(phys_to_pfn(paddr)) + +#ifndef CONFIG_KFENCE + +#define page_to_virt(page) __va(page_to_phys(page)) +#define virt_to_page(kaddr) phys_to_page(__pa(kaddr)) + +#else + +#define WANT_PAGE_VIRTUAL + +#define page_to_virt(page) \ +({ \ + extern char *__kfence_pool; \ + (__kfence_pool =3D=3D NULL) ? __va(page_to_phys(page)) : page_address(pag= e); \ +}) =20 #define virt_to_page(kaddr) \ ({ \ @@ -86,6 +105,11 @@ struct page *tlb_virt_to_page(unsigned long kaddr); dmw_virt_to_page((unsigned long)kaddr) : tlb_virt_to_page((unsigned long)= kaddr);\ }) =20 +#endif + +#define pfn_to_virt(pfn) page_to_virt(pfn_to_page(pfn)) +#define virt_to_pfn(kaddr) page_to_pfn(virt_to_page(kaddr)) + extern int __virt_addr_valid(volatile void *kaddr); #define virt_addr_valid(kaddr) __virt_addr_valid((volatile void *)(kaddr)) =20 diff --git a/arch/loongarch/mm/pgtable.c b/arch/loongarch/mm/pgtable.c index 2aae72e63871..bda018150000 100644 --- a/arch/loongarch/mm/pgtable.c +++ b/arch/loongarch/mm/pgtable.c @@ -11,13 +11,13 @@ =20 struct page *dmw_virt_to_page(unsigned long kaddr) { - return pfn_to_page(virt_to_pfn(kaddr)); + return phys_to_page(__pa(kaddr)); } EXPORT_SYMBOL(dmw_virt_to_page); =20 struct page *tlb_virt_to_page(unsigned long kaddr) { - return pfn_to_page(pte_pfn(*virt_to_kpte(kaddr))); + return phys_to_page(pfn_to_phys(pte_pfn(*virt_to_kpte(kaddr)))); } EXPORT_SYMBOL(tlb_virt_to_page); =20 --=20 2.43.0 From nobody Mon Feb 9 10:50:17 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 70C19128362; Thu, 4 Apr 2024 13:37:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712237843; cv=none; b=DVJOBGSnl3/6N8YxZs4J0V0OKxsNraNW6H0VE0JeqYCNgTSIQtgK4m2qZvd8je2A58iWbmy0dDGe5/nsFejYNRPKztr6fByzC3sJpqbtsgctRfM0ruUtcFh8zCS7gZrj/TaJfp9fJw5ZCMjXBcHdX5s6YDICWjv5aoj7F3VXlxk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1712237843; c=relaxed/simple; bh=L5JuVTwYjhYNtocqaBlZrcI4/RRu7dj6AmX+ESPnNdg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tECuZZQSqTJZjGFrfGiGUROpqWk88vcADcDBNUsLYT10bhsiWJ+aLTaZ4S4UJQWsV72xs3Pt2M0mbKUv26qAUL0SWd8EGqqaEZI3gOrmvyl+/MSVy6apioLTSCYxIWh1eF6elOoe8vg6NAdAI3VYrmVvbktsIB1Tj4gBdPaGpPA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1989BC433F1; Thu, 4 Apr 2024 13:37:19 +0000 (UTC) From: Huacai Chen To: Andrew Morton , Huacai Chen Cc: loongarch@lists.linux.dev, linux-mm@kvack.org, Xuefeng Li , Guo Ren , Xuerui Wang , Jiaxun Yang , linux-kernel@vger.kernel.org, loongson-kernel@lists.loongnix.cn, Huacai Chen , Guenter Roeck Subject: [PATCH 3/3] LoongArch: Make virt_addr_valid()/__virt_addr_valid() work with KFENCE Date: Thu, 4 Apr 2024 21:36:36 +0800 Message-ID: <20240404133642.971583-4-chenhuacai@loongson.cn> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240404133642.971583-1-chenhuacai@loongson.cn> References: <20240404133642.971583-1-chenhuacai@loongson.cn> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When enabling both CONFIG_KFENCE and CONFIG_DEBUG_SG, I get the following backtraces when running LongArch kernels. [ 2.496257] kernel BUG at include/linux/scatterlist.h:187! ... [ 2.501925] Call Trace: [ 2.501950] [<9000000004ad59c4>] sg_init_one+0xac/0xc0 [ 2.502204] [<9000000004a438f8>] do_test_kpp+0x278/0x6e4 [ 2.502353] [<9000000004a43dd4>] alg_test_kpp+0x70/0xf4 [ 2.502494] [<9000000004a41b48>] alg_test+0x128/0x690 [ 2.502631] [<9000000004a3d898>] cryptomgr_test+0x20/0x40 [ 2.502775] [<90000000041b4508>] kthread+0x138/0x158 [ 2.502912] [<9000000004161c48>] ret_from_kernel_thread+0xc/0xa4 The backtrace is always similar but not exactly the same. It is always triggered from cryptomgr_test, but not always from the same test. Analysis shows that with CONFIG_KFENCE active, the address returned from kmalloc() and friends is not always below vm_map_base. It is allocated by kfence_alloc() which at least sometimes seems to get its memory from an address space above vm_map_base. This causes __virt_addr_valid() to return false for the affected objects. Let __virt_addr_valid() return 1 for kfence pool addresses, this make virt_addr_valid()/__virt_addr_valid() work with KFENCE. Reported-by: Guenter Roeck Suggested-by: Guenter Roeck Signed-off-by: Huacai Chen --- arch/loongarch/mm/mmap.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/arch/loongarch/mm/mmap.c b/arch/loongarch/mm/mmap.c index a9630a81b38a..89af7c12e8c0 100644 --- a/arch/loongarch/mm/mmap.c +++ b/arch/loongarch/mm/mmap.c @@ -4,6 +4,7 @@ */ #include #include +#include #include #include #include @@ -111,6 +112,9 @@ int __virt_addr_valid(volatile void *kaddr) { unsigned long vaddr =3D (unsigned long)kaddr; =20 + if (is_kfence_address((void *)kaddr)) + return 1; + if ((vaddr < PAGE_OFFSET) || (vaddr >=3D vm_map_base)) return 0; =20 --=20 2.43.0