From nobody Tue Nov 26 08:24:58 2024 Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1FF6D20402E for ; Fri, 18 Oct 2024 17:30:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729272626; cv=none; b=RXQAG9gfLGBZsJwXZzPy9WCgWCBEoDf4DE9/KWDxp7mDRtYe9CLfuJwKuitDcOWlgesTVz/9rUHbu9NqfsFTK8SSlJhnGtTdhnkQytbmBDDPk3DVfYmu+B/6ZnvcmdeFDYF0NHrvhlmzaGzPzSGveIl4+VaS5D1J/rYTO6Q3P9E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729272626; c=relaxed/simple; bh=1ZXY/6ndXY0AQOuA82oxShaciH67q8+j3sjgy9ktttk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=sMSUJIriF32JEMJUMaeP8yOCnH0FHtqqIAEsRdXl3uBZV5Ds2CFuOJjnrEZy/HPE5ohW7cBaDahd3D+wilat2c1XyxESyKcNhCliRD9/C4YofXGKniHnHn1z12P+zkN8hmdCTRvVi8xqw281VmZANfcbJ7qQxy4u377mjYB3sss= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=GSU4UB9w; arc=none smtp.client-ip=209.85.210.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="GSU4UB9w" Received: by mail-pf1-f176.google.com with SMTP id d2e1a72fcca58-71e4244fdc6so1725338b3a.0 for ; Fri, 18 Oct 2024 10:30:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729272624; x=1729877424; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ZFpXrhpElIx5U0caqw8WXCQdQvrJMpAq6qiRpCPXwmQ=; b=GSU4UB9wKP5sNB1ZL6hCsJwcOzQgP3t2oG3ysgC+LmpH6pLI+CT+Za+paX8A/ZJt1B 9SSJQBQhTTziAD/iAPzPbv4bYhEQQ08tL+4nuGDVHvywjhf5zknhFJ3lWw9bbiyEEf5E HFlSiskMgoBn3kLhOgVLYvukn+zeoIYe+Q4fodpu6bV7LTE8uy3vHD7h19Q6X+fy74Jy bzPN1m2gMVgi4bD9lBBT7UpgHwHvR4b63MEVDuSaeFH4PKMpE7d4w3Og3xgIoVWJMVEW x+WoPhiVLMm3WwBWwxoMu82fhNBTCejEOdCVw8gDWmV3HCir0TOEzhbBsE0r2BWgGm+U s7gQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729272624; x=1729877424; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ZFpXrhpElIx5U0caqw8WXCQdQvrJMpAq6qiRpCPXwmQ=; b=DDHs3ORJXN5mNzB34mW4LEc66h2rR3cYyLVKmbyAqa/sgpeZk9NYagqS0BRoBjv1sa 0MSkQhppM/QBuRvWR7OS6cGHxWROAPPz5PwAkXyaE4k42hb0QGWivX7btw1rBJcF91xy +0jL29EQNHA1xmGIHt2n1eVBXoNIMmMlN9tHijqpkh6hidwAUQZnLMS6M9qlKh5I+UKG f4YlWaomGwM21qudz+f/oRfTXXfz2Fc+Jt+vkpcZw/VWyWvdt//cpWkXG/M2KgvZRZk2 /Xrn4DhypHsxdI/yUDqnwP5qe1I09I0b3TOnhd7CdVJfgh9qQ+lw6GwC/+abt2u4H2Ez 5FhA== X-Forwarded-Encrypted: i=1; AJvYcCWHkKqZSSAH4cOs4vRBqoJIoO1GYPJ5U3WAMAuPfAY10OaTmY6ZyP2GMzxcWWXlYj7hff6U073r0mkA32I=@vger.kernel.org X-Gm-Message-State: AOJu0YzqdlHMQNICF/ufkfG3qYt/GFbTF50b05vYASvkfgF+MK4CXP9y hwvS6sopXYp/uGYGeHYZntv0Uys2j1YZQnXGqst1n1z0tlAkGt8nxGJMDtgw X-Google-Smtp-Source: AGHT+IFcGGMjhgZmKijPwzMX2TIIKyjbIB1NxgoXGiQqvtxq79BYQZAFUknM7+tFqktbTIEQT/2tPA== X-Received: by 2002:a05:6a00:1390:b0:71e:148c:4611 with SMTP id d2e1a72fcca58-71ea3124252mr4810440b3a.6.1729272624262; Fri, 18 Oct 2024 10:30:24 -0700 (PDT) Received: from dw-tp.ibmuc.com ([171.76.80.151]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71ea3311f51sm1725242b3a.36.2024.10.18.10.30.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Oct 2024 10:30:23 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linuxppc-dev@lists.ozlabs.org Cc: kasan-dev@googlegroups.com, linux-mm@kvack.org, Marco Elver , Alexander Potapenko , Heiko Carstens , Michael Ellerman , Nicholas Piggin , Madhavan Srinivasan , Christophe Leroy , Hari Bathini , "Aneesh Kumar K . V" , Donet Tom , Pavithra Prakash , LKML , "Ritesh Harjani (IBM)" , Disha Goel Subject: [PATCH v3 01/12] powerpc: mm/fault: Fix kfence page fault reporting Date: Fri, 18 Oct 2024 22:59:42 +0530 Message-ID: X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" copy_from_kernel_nofault() can be called when doing read of /proc/kcore. /proc/kcore can have some unmapped kfence objects which when read via copy_from_kernel_nofault() can cause page faults. Since *_nofault() functions define their own fixup table for handling fault, use that instead of asking kfence to handle such faults. Hence we search the exception tables for the nip which generated the fault. If there is an entry then we let the fixup table handler handle the page fault by returning an error from within ___do_page_fault(). This can be easily triggered if someone tries to do dd from /proc/kcore. dd if=3D/proc/kcore of=3D/dev/null bs=3D1M =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D BUG: KFENCE: invalid read in copy_from_kernel_nofault+0xb0/0x1c8 Invalid read at 0x000000004f749d2e: copy_from_kernel_nofault+0xb0/0x1c8 0xc0000000057f7950 read_kcore_iter+0x41c/0x9ac proc_reg_read_iter+0xe4/0x16c vfs_read+0x2e4/0x3b0 ksys_read+0x88/0x154 system_call_exception+0x124/0x340 system_call_common+0x160/0x2c4 BUG: KFENCE: use-after-free read in copy_from_kernel_nofault+0xb0/0x1c8 Use-after-free read at 0x000000008fbb08ad (in kfence-#0): copy_from_kernel_nofault+0xb0/0x1c8 0xc0000000057f7950 read_kcore_iter+0x41c/0x9ac proc_reg_read_iter+0xe4/0x16c vfs_read+0x2e4/0x3b0 ksys_read+0x88/0x154 system_call_exception+0x124/0x340 system_call_common+0x160/0x2c4 Fixes: 90cbac0e995d ("powerpc: Enable KFENCE for PPC32") Suggested-by: Christophe Leroy Reported-by: Disha Goel Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Christophe Leroy --- arch/powerpc/mm/fault.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/arch/powerpc/mm/fault.c b/arch/powerpc/mm/fault.c index 81c77ddce2e3..316f5162ffc4 100644 --- a/arch/powerpc/mm/fault.c +++ b/arch/powerpc/mm/fault.c @@ -439,10 +439,17 @@ static int ___do_page_fault(struct pt_regs *regs, uns= igned long address, /* * The kernel should never take an execute fault nor should it * take a page fault to a kernel address or a page fault to a user - * address outside of dedicated places + * address outside of dedicated places. + * + * Rather than kfence directly reporting false negatives, search whether + * the NIP belongs to the fixup table for cases where fault could come + * from functions like copy_from_kernel_nofault(). */ if (unlikely(!is_user && bad_kernel_fault(regs, error_code, address, is_w= rite))) { - if (kfence_handle_page_fault(address, is_write, regs)) + + if (is_kfence_address((void *)address) && + !search_exception_tables(instruction_pointer(regs)) && + kfence_handle_page_fault(address, is_write, regs)) return 0; return SIGSEGV; -- 2.46.0 From nobody Tue Nov 26 08:24:58 2024 Received: from mail-pf1-f182.google.com (mail-pf1-f182.google.com [209.85.210.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4366F2040BB for ; Fri, 18 Oct 2024 17:30:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729272632; cv=none; b=MMZp8Sy8EMxQVA3p0pTcvpA0O3mtn+/wJPcn/Pmt60SrESJXCt5h7RCmU3Tqwtkb6uxuCVayulFdQ2aXtbJlZ7pFjW9oV3mnXG0oNoNWUmD9GEedfu3c7c4bBVdd2Sgr33QsczXPgpn05qSujeuG+IQjWS53GcdS6AFjvp/smGE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729272632; c=relaxed/simple; bh=mAFFiFfCRXSppOaaN188Oqy1JAdt7Lpi79C9uG0gueA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=V7vahSRBaGr7IqdLY7N8mSKXrLkJuuWSKczMCdke5yqPVOUzwcF15Z0p9U0UiJzkwCH3kNJfpT9deo1qt6YxgK1zaJ7sybR7xNKYS2mBRxkRKdrLfpTs6z1xvS+K2YRiR8s+sCsIZoKlU/U3DRritk/toz0r22BFmyvwl5l/eqg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=eQBv5G2N; arc=none smtp.client-ip=209.85.210.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="eQBv5G2N" Received: by mail-pf1-f182.google.com with SMTP id d2e1a72fcca58-71e6f085715so1847435b3a.2 for ; Fri, 18 Oct 2024 10:30:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729272629; x=1729877429; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Y7m6nw0SlLNWzWwVTYubL573BHmMjcLP6yP7O1snSbI=; b=eQBv5G2NHOwCNsGJmXbNZzxeCtRsMnZ70lqhWVXqiJ5sQep86LTiAzHE7hqHYyjTfN QqG4dihxI2c3htnigTDkcqJ24RcryKFf4F/J41TJXWOarkclsZdje6J1aWgOUEP619Iz LfwkH/2ocngJbR6nT9LajWn7guzFqPTOe+IAPvEpas3lCbJRVH6A859dAhE01gp13Zbj 4s4u9IcYtuUj3YgOEV1/WdPrbUzOMI0HO2zg87NtKE/UMTw+qfACGsb0z2HMQyvX+TGn XM+riwfgXcs69iVyf9xdaJ7x5CMnh474se9+3BHUmCdNcVrFl8xAfJzHfGu3biQVotVp +pIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729272629; x=1729877429; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Y7m6nw0SlLNWzWwVTYubL573BHmMjcLP6yP7O1snSbI=; b=n4ujNcLNhxMPXFAkTGfbXwQeRZViECYBXYXZIRuHj+r6swfm/EsdVfqaA+khOlBtAZ vfTayU7Qn5yngxIhrOys+UNTojTQjJ8cR/hHMlVd3wkgFzarrmiaAiNNtUHMY8wJhmcp XJ3AwV00ibjKIVAUJ0F0nYZJ+IrLzdn03AiGN5GK4I+YFCGbsrJMOT3a5J5d4HWu+G4O s4n54Mf0Xj03sNOdwOI6xCfXCUcMzis/eki90T8td0/y2gCAwURHvKRQE+hZMSNtKvWa 5vcPc8bfHoueIOz+MWFbKC2jwhZPLhx/rgKkaC0B8SeIXEGqA8QPX6MTN6Yd0g0RJWdz c4mA== X-Forwarded-Encrypted: i=1; AJvYcCVpGA4F/ayYKAOFn/bXJFG0Z/psf8muz8YrsCtoyA2f2PJSvgwrD4ec9POg2yCuOphlHIPyyU+IIBGomCo=@vger.kernel.org X-Gm-Message-State: AOJu0YzYDwHQHS9rk1wPsGZmp0wNoD8yj/b+ZwKVqN/DzvNYHlQYsOEf 3gwqPxzjkrlEttoNBu6GafhBE50i7wO3CfNEiDYKFbbNf3mYLSJB X-Google-Smtp-Source: AGHT+IH29uhXoWddJW63rmOfdUmm3F60bMAd8Lb1nxktQLiJXibpBjYFtPZvdFulLgU3PQV9sp2ESA== X-Received: by 2002:a05:6a00:3e0f:b0:71e:4a1b:2204 with SMTP id d2e1a72fcca58-71ea331b398mr4152501b3a.25.1729272629392; Fri, 18 Oct 2024 10:30:29 -0700 (PDT) Received: from dw-tp.ibmuc.com ([171.76.80.151]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71ea3311f51sm1725242b3a.36.2024.10.18.10.30.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Oct 2024 10:30:28 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linuxppc-dev@lists.ozlabs.org Cc: kasan-dev@googlegroups.com, linux-mm@kvack.org, Marco Elver , Alexander Potapenko , Heiko Carstens , Michael Ellerman , Nicholas Piggin , Madhavan Srinivasan , Christophe Leroy , Hari Bathini , "Aneesh Kumar K . V" , Donet Tom , Pavithra Prakash , LKML , "Ritesh Harjani (IBM)" Subject: [PATCH v3 02/12] book3s64/hash: Remove kfence support temporarily Date: Fri, 18 Oct 2024 22:59:43 +0530 Message-ID: <1761bc39674473c8878dedca15e0d9a0d3a1b528.1729271995.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Kfence on book3s Hash on pseries is anyways broken. It fails to boot due to RMA size limitation. That is because, kfence with Hash uses debug_pagealloc infrastructure. debug_pagealloc allocates linear map for entire dram size instead of just kfence relevant objects. This means for 16TB of DRAM it will require (16TB >> PAGE_SHIFT) which is 256MB which is half of RMA region on P8. crash kernel reserves 256MB and we also need 2048 * 16KB * 3 for emergency stack and some more for paca allocations. That means there is not enough memory for reserving the full linear map in the RMA region, if the DRAM size is too big (>=3D16TB) (The issue is seen above 8TB with crash kernel 256 MB reservation). Now Kfence does not require linear memory map for entire DRAM. It only needs for kfence objects. So this patch temporarily removes the kfence functionality since debug_pagealloc code needs some refactoring. We will bring in kfence on Hash support in later patches. Signed-off-by: Ritesh Harjani (IBM) --- arch/powerpc/include/asm/kfence.h | 5 +++++ arch/powerpc/mm/book3s64/hash_utils.c | 16 +++++++++++----- 2 files changed, 16 insertions(+), 5 deletions(-) diff --git a/arch/powerpc/include/asm/kfence.h b/arch/powerpc/include/asm/k= fence.h index fab124ada1c7..f3a9476a71b3 100644 --- a/arch/powerpc/include/asm/kfence.h +++ b/arch/powerpc/include/asm/kfence.h @@ -10,6 +10,7 @@ =20 #include #include +#include =20 #ifdef CONFIG_PPC64_ELF_ABI_V1 #define ARCH_FUNC_PREFIX "." @@ -25,6 +26,10 @@ static inline void disable_kfence(void) =20 static inline bool arch_kfence_init_pool(void) { +#ifdef CONFIG_PPC64 + if (!radix_enabled()) + return false; +#endif return !kfence_disabled; } #endif diff --git a/arch/powerpc/mm/book3s64/hash_utils.c b/arch/powerpc/mm/book3s= 64/hash_utils.c index a408ef7d850e..e22a8f540193 100644 --- a/arch/powerpc/mm/book3s64/hash_utils.c +++ b/arch/powerpc/mm/book3s64/hash_utils.c @@ -431,7 +431,7 @@ int htab_bolt_mapping(unsigned long vstart, unsigned lo= ng vend, break; =20 cond_resched(); - if (debug_pagealloc_enabled_or_kfence() && + if (debug_pagealloc_enabled() && (paddr >> PAGE_SHIFT) < linear_map_hash_count) linear_map_hash_slots[paddr >> PAGE_SHIFT] =3D ret | 0x80; } @@ -814,7 +814,7 @@ static void __init htab_init_page_sizes(void) bool aligned =3D true; init_hpte_page_sizes(); =20 - if (!debug_pagealloc_enabled_or_kfence()) { + if (!debug_pagealloc_enabled()) { /* * Pick a size for the linear mapping. Currently, we only * support 16M, 1M and 4K which is the default @@ -1134,7 +1134,7 @@ static void __init htab_initialize(void) =20 prot =3D pgprot_val(PAGE_KERNEL); =20 - if (debug_pagealloc_enabled_or_kfence()) { + if (debug_pagealloc_enabled()) { linear_map_hash_count =3D memblock_end_of_DRAM() >> PAGE_SHIFT; linear_map_hash_slots =3D memblock_alloc_try_nid( linear_map_hash_count, 1, MEMBLOCK_LOW_LIMIT, @@ -2120,7 +2120,7 @@ void hpt_do_stress(unsigned long ea, unsigned long hp= te_group) } } =20 -#if defined(CONFIG_DEBUG_PAGEALLOC) || defined(CONFIG_KFENCE) +#ifdef CONFIG_DEBUG_PAGEALLOC static DEFINE_RAW_SPINLOCK(linear_map_hash_lock); =20 static void kernel_map_linear_page(unsigned long vaddr, unsigned long lmi) @@ -2194,7 +2194,13 @@ int hash__kernel_map_pages(struct page *page, int nu= mpages, int enable) local_irq_restore(flags); return 0; } -#endif /* CONFIG_DEBUG_PAGEALLOC || CONFIG_KFENCE */ +#else /* CONFIG_DEBUG_PAGEALLOC */ +int hash__kernel_map_pages(struct page *page, int numpages, + int enable) +{ + return 0; +} +#endif /* CONFIG_DEBUG_PAGEALLOC */ =20 void hash__setup_initial_memory_limit(phys_addr_t first_memblock_base, phys_addr_t first_memblock_size) --=20 2.46.0 From nobody Tue Nov 26 08:24:58 2024 Received: from mail-pg1-f175.google.com (mail-pg1-f175.google.com [209.85.215.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6099C204943 for ; Fri, 18 Oct 2024 17:30:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729272637; cv=none; b=DxSB5t90bM+qZL30vzwWSHsENvhB0/WEWG57xc2nrBSgu2RInymSJqN+zvniJxRPqE0bN+NTtruc42ucIpDz01cEZdDGJyF1ziCKBq58dnzz1OBbQG3Gd2sks+zGTXLftAdKnOtl2K39qX3dx/SH89eJpFGQpI3ITKDpDxACnWY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729272637; c=relaxed/simple; bh=qf4m6j7PPG75kG42Tc21klYl7CuPttrtdBNmIdfIt/0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lHoxkEUl0L6Q8wKFNKgIbp7cuhLJtGJ8GOpe/ymUJCVvDJSExNTuIKQfPgmaqEYNxGtPgqsktY3jzsrOd9BnHyAx0dZ/csBgnO2TgkkXmUvSq0ZggbCbnSBOLgph8wwTNnfZIpmwjqzQT2NjLXMjyEXub8GN2foWFzgWMwrUp0I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Cet/MpG4; arc=none smtp.client-ip=209.85.215.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Cet/MpG4" Received: by mail-pg1-f175.google.com with SMTP id 41be03b00d2f7-7db637d1e4eso2453300a12.2 for ; Fri, 18 Oct 2024 10:30:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729272634; x=1729877434; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=PKeOGzBwhkORap+bFaXRZ5k5ETEZTGkTXkc6r1BfTnM=; b=Cet/MpG4kvfdCQe8kcz0OglK7FZtxONAfapZkQnAjRLbCPXKA+tuGZp5N3vzZsC2LK Y3XFF+G11g9TnuZBPgznfYFL8cOvqQMQm6mdz3+xvfNQdn1Jm3NwTXVoRf3bp67/qY2Q nM/iviDVPEx1QZw5JlTYeAYoAuXPBepVBHjUWC0zscem62bJRVVtjg4oPfx5wS5xe1/i KqLRl5P5pUOchJiISf2H92nSXhn4eK7Y/zBGiEBMmAtDnjrFVYbeDkSk7+OtWVKCB7d/ 3NP0XxbiPptY6L/ZjGRVFTpjaV7qE8kwdwyM0djtQKwShQ6facU+adB2n96v2Ow+2oNX ByTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729272634; x=1729877434; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=PKeOGzBwhkORap+bFaXRZ5k5ETEZTGkTXkc6r1BfTnM=; b=QRS4XzdtZjztVGXAnZbJ58HxsoO5jV7sRVHdjFZQU2pWz4e3m37nRkcETwSoH4vKtp v8omNnOShYZp4ssaH4UVf15f0rcqeV5CIcDDrXqxBivAQDSHxRZk+AzaA8LAj0dSqwPQ on4/Q4Z8sHuasR4nnkXWYDyEUSH+fdgpd0FYOwoyhtv7Srt/uC8csACcVpZ0tUDMjy+G Md3D//7Kihz3117KUeGMDiNP54xsZbOrIBZYUxycK/MC3tM8g77IPgMMa/VHN4VPKvZS RVzvLsJDVydtmi1J8b7y0rICsVkeDRhnFCIplRXzoxdg8FW3M1Wp+vA5yHNWwrzsbka7 t4fA== X-Forwarded-Encrypted: i=1; AJvYcCWYg3U6VId7bHjeoDuXYtiGy7FGalvftgpWSju1d3OfYQeQAHA3TzHSCFe9LeN1A+Kzqd+TNxMkssQxcfU=@vger.kernel.org X-Gm-Message-State: AOJu0YxwhBPBIg627KKhDnX8VjeaHHkLBv0CdJVJkDjdacDbjlllws8E W+XvS23LGXorbsb/U4DWx7S7SroZ3Mal08NCkZxAACbqpAeo/sgP X-Google-Smtp-Source: AGHT+IEHDVJvXPkTfxkfLhiJ891PgPvrAItwAzBFVy3rywsUuhpfmzOVbkrBnF6delxgn8dQGKkFxQ== X-Received: by 2002:a05:6a20:d98:b0:1cf:38b0:57ff with SMTP id adf61e73a8af0-1d92c5ac311mr4139493637.48.1729272634341; Fri, 18 Oct 2024 10:30:34 -0700 (PDT) Received: from dw-tp.ibmuc.com ([171.76.80.151]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71ea3311f51sm1725242b3a.36.2024.10.18.10.30.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Oct 2024 10:30:33 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linuxppc-dev@lists.ozlabs.org Cc: kasan-dev@googlegroups.com, linux-mm@kvack.org, Marco Elver , Alexander Potapenko , Heiko Carstens , Michael Ellerman , Nicholas Piggin , Madhavan Srinivasan , Christophe Leroy , Hari Bathini , "Aneesh Kumar K . V" , Donet Tom , Pavithra Prakash , LKML , "Ritesh Harjani (IBM)" Subject: [PATCH v3 03/12] book3s64/hash: Refactor kernel linear map related calls Date: Fri, 18 Oct 2024 22:59:44 +0530 Message-ID: <56c610310aa50b5417976a39c5f15b78bc76c764.1729271995.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This just brings all linear map related handling at one place instead of having those functions scattered in hash_utils file. Makes it easy for review. No functionality changes in this patch. Signed-off-by: Ritesh Harjani (IBM) --- arch/powerpc/mm/book3s64/hash_utils.c | 164 +++++++++++++------------- 1 file changed, 82 insertions(+), 82 deletions(-) diff --git a/arch/powerpc/mm/book3s64/hash_utils.c b/arch/powerpc/mm/book3s= 64/hash_utils.c index e22a8f540193..fb2f717e9e74 100644 --- a/arch/powerpc/mm/book3s64/hash_utils.c +++ b/arch/powerpc/mm/book3s64/hash_utils.c @@ -273,6 +273,88 @@ void hash__tlbiel_all(unsigned int action) WARN(1, "%s called on pre-POWER7 CPU\n", __func__); } =20 +#ifdef CONFIG_DEBUG_PAGEALLOC +static DEFINE_RAW_SPINLOCK(linear_map_hash_lock); + +static void kernel_map_linear_page(unsigned long vaddr, unsigned long lmi) +{ + unsigned long hash; + unsigned long vsid =3D get_kernel_vsid(vaddr, mmu_kernel_ssize); + unsigned long vpn =3D hpt_vpn(vaddr, vsid, mmu_kernel_ssize); + unsigned long mode =3D htab_convert_pte_flags(pgprot_val(PAGE_KERNEL), HP= TE_USE_KERNEL_KEY); + long ret; + + hash =3D hpt_hash(vpn, PAGE_SHIFT, mmu_kernel_ssize); + + /* Don't create HPTE entries for bad address */ + if (!vsid) + return; + + if (linear_map_hash_slots[lmi] & 0x80) + return; + + ret =3D hpte_insert_repeating(hash, vpn, __pa(vaddr), mode, + HPTE_V_BOLTED, + mmu_linear_psize, mmu_kernel_ssize); + + BUG_ON (ret < 0); + raw_spin_lock(&linear_map_hash_lock); + BUG_ON(linear_map_hash_slots[lmi] & 0x80); + linear_map_hash_slots[lmi] =3D ret | 0x80; + raw_spin_unlock(&linear_map_hash_lock); +} + +static void kernel_unmap_linear_page(unsigned long vaddr, unsigned long lm= i) +{ + unsigned long hash, hidx, slot; + unsigned long vsid =3D get_kernel_vsid(vaddr, mmu_kernel_ssize); + unsigned long vpn =3D hpt_vpn(vaddr, vsid, mmu_kernel_ssize); + + hash =3D hpt_hash(vpn, PAGE_SHIFT, mmu_kernel_ssize); + raw_spin_lock(&linear_map_hash_lock); + if (!(linear_map_hash_slots[lmi] & 0x80)) { + raw_spin_unlock(&linear_map_hash_lock); + return; + } + hidx =3D linear_map_hash_slots[lmi] & 0x7f; + linear_map_hash_slots[lmi] =3D 0; + raw_spin_unlock(&linear_map_hash_lock); + if (hidx & _PTEIDX_SECONDARY) + hash =3D ~hash; + slot =3D (hash & htab_hash_mask) * HPTES_PER_GROUP; + slot +=3D hidx & _PTEIDX_GROUP_IX; + mmu_hash_ops.hpte_invalidate(slot, vpn, mmu_linear_psize, + mmu_linear_psize, + mmu_kernel_ssize, 0); +} + +int hash__kernel_map_pages(struct page *page, int numpages, int enable) +{ + unsigned long flags, vaddr, lmi; + int i; + + local_irq_save(flags); + for (i =3D 0; i < numpages; i++, page++) { + vaddr =3D (unsigned long)page_address(page); + lmi =3D __pa(vaddr) >> PAGE_SHIFT; + if (lmi >=3D linear_map_hash_count) + continue; + if (enable) + kernel_map_linear_page(vaddr, lmi); + else + kernel_unmap_linear_page(vaddr, lmi); + } + local_irq_restore(flags); + return 0; +} +#else /* CONFIG_DEBUG_PAGEALLOC */ +int hash__kernel_map_pages(struct page *page, int numpages, + int enable) +{ + return 0; +} +#endif /* CONFIG_DEBUG_PAGEALLOC */ + /* * 'R' and 'C' update notes: * - Under pHyp or KVM, the updatepp path will not set C, thus it *will* @@ -2120,88 +2202,6 @@ void hpt_do_stress(unsigned long ea, unsigned long h= pte_group) } } =20 -#ifdef CONFIG_DEBUG_PAGEALLOC -static DEFINE_RAW_SPINLOCK(linear_map_hash_lock); - -static void kernel_map_linear_page(unsigned long vaddr, unsigned long lmi) -{ - unsigned long hash; - unsigned long vsid =3D get_kernel_vsid(vaddr, mmu_kernel_ssize); - unsigned long vpn =3D hpt_vpn(vaddr, vsid, mmu_kernel_ssize); - unsigned long mode =3D htab_convert_pte_flags(pgprot_val(PAGE_KERNEL), HP= TE_USE_KERNEL_KEY); - long ret; - - hash =3D hpt_hash(vpn, PAGE_SHIFT, mmu_kernel_ssize); - - /* Don't create HPTE entries for bad address */ - if (!vsid) - return; - - if (linear_map_hash_slots[lmi] & 0x80) - return; - - ret =3D hpte_insert_repeating(hash, vpn, __pa(vaddr), mode, - HPTE_V_BOLTED, - mmu_linear_psize, mmu_kernel_ssize); - - BUG_ON (ret < 0); - raw_spin_lock(&linear_map_hash_lock); - BUG_ON(linear_map_hash_slots[lmi] & 0x80); - linear_map_hash_slots[lmi] =3D ret | 0x80; - raw_spin_unlock(&linear_map_hash_lock); -} - -static void kernel_unmap_linear_page(unsigned long vaddr, unsigned long lm= i) -{ - unsigned long hash, hidx, slot; - unsigned long vsid =3D get_kernel_vsid(vaddr, mmu_kernel_ssize); - unsigned long vpn =3D hpt_vpn(vaddr, vsid, mmu_kernel_ssize); - - hash =3D hpt_hash(vpn, PAGE_SHIFT, mmu_kernel_ssize); - raw_spin_lock(&linear_map_hash_lock); - if (!(linear_map_hash_slots[lmi] & 0x80)) { - raw_spin_unlock(&linear_map_hash_lock); - return; - } - hidx =3D linear_map_hash_slots[lmi] & 0x7f; - linear_map_hash_slots[lmi] =3D 0; - raw_spin_unlock(&linear_map_hash_lock); - if (hidx & _PTEIDX_SECONDARY) - hash =3D ~hash; - slot =3D (hash & htab_hash_mask) * HPTES_PER_GROUP; - slot +=3D hidx & _PTEIDX_GROUP_IX; - mmu_hash_ops.hpte_invalidate(slot, vpn, mmu_linear_psize, - mmu_linear_psize, - mmu_kernel_ssize, 0); -} - -int hash__kernel_map_pages(struct page *page, int numpages, int enable) -{ - unsigned long flags, vaddr, lmi; - int i; - - local_irq_save(flags); - for (i =3D 0; i < numpages; i++, page++) { - vaddr =3D (unsigned long)page_address(page); - lmi =3D __pa(vaddr) >> PAGE_SHIFT; - if (lmi >=3D linear_map_hash_count) - continue; - if (enable) - kernel_map_linear_page(vaddr, lmi); - else - kernel_unmap_linear_page(vaddr, lmi); - } - local_irq_restore(flags); - return 0; -} -#else /* CONFIG_DEBUG_PAGEALLOC */ -int hash__kernel_map_pages(struct page *page, int numpages, - int enable) -{ - return 0; -} -#endif /* CONFIG_DEBUG_PAGEALLOC */ - void hash__setup_initial_memory_limit(phys_addr_t first_memblock_base, phys_addr_t first_memblock_size) { --=20 2.46.0 From nobody Tue Nov 26 08:24:58 2024 Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4C478204F61 for ; Fri, 18 Oct 2024 17:30:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729272644; cv=none; b=PoXOnO1R3nfGN5CIWwMBwgYh5n3oBvpcAcw9rxR2xSmKDvPyQcQc54gX1eGE56DcVsOuLa7RiyySpdrBoz1QR9s+DD6R9Nhtic/9vW9jRI2N/CRqrZ9hKpSyATcy3w5jnZQOh/tEvtM68fIvbs+e8/t7nvPIo0v6uapvm7ktqH8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729272644; c=relaxed/simple; bh=Eu79YTiDNt99GzvYca1lqmYdNCIQyv54j7t8FXnFaIY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qPbaGxZXPpe9EwvV7NLJdOC2NuB2GI4BI8eNng0zgtK1fPuI/h08AvBzD+LfDCT8jdaek/UorMHmfQfAYvebrjaEuCm7EHp1CgiYQHlXgHbsaLZRFN3ZMjEoPUEX4XqEPSL3AiUERingel67yDNwFHByZXJ/2v2u7+hZiYJ8k/k= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Cd9VROW/; arc=none smtp.client-ip=209.85.210.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Cd9VROW/" Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-71eb1d0e3c2so29086b3a.2 for ; Fri, 18 Oct 2024 10:30:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729272641; x=1729877441; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Jw41Uo4vlK/av70SyuzQ/EcBacKxJaXG8uVQm7tNSaA=; b=Cd9VROW/chSLaFW65w/Dhp+Sskt0juhr2oLkPTSl65IaAmCKrvOkPvQxtsK2reiUYg trY5rseGvFrkgcCfAF946Z4HSOk5NFjF+fcG4QSjQrITMwzeWFcQPEs8LTWe+nfCWtIl RWVeOnXyZkWPQAGuHCmyrkHxm+30g4S7HYScD2qiVvoEvJq01H4ODE6J357Wjazdx81A LaxOLHek+4h3XeGRtn6wtRKELmcfkuiDquRlP9LXy8jwu/7USORGSo+Mj/NBNUPWU3Vf SeDpk9oitHMc4A+WEsO4vhm9T/8rQEcGGmwtyYxhL9MGCw8cF2qujdk5U6XaFzbi+EW1 TeCw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729272641; x=1729877441; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Jw41Uo4vlK/av70SyuzQ/EcBacKxJaXG8uVQm7tNSaA=; b=QZsOq7hSMqe9tNlnALq59Cg2ZmzXGTfNrvf8OR/ffl8q1ivvJeXz192YgcveRY9NIP MHzjtzbYSZbWC5kVHhj9Eu0lqsxTB6MkgsSm7Cf9en4e3malMnUsAH2OpBZFe6O99nnO xAAN7/WJffYd63OfQ0/ahq2Tn7QTMmRNtZuDcI4dXOHeTMOuh0mlkqu+ySJDkr4VR+uH UI/N/ZHKfodXYncO9TYN9iGf9r0/M0g3qG349x3vCML9R9hFYb+XHdmf+Wj5h/6cTjfu +johhKMwO3JU3/Fvz+I1h1GrModsOxhAp6wOjbEJ5R1tplnNEJyXbvu9yPxw2VvxPLiM Fnjg== X-Forwarded-Encrypted: i=1; AJvYcCXBu5j0g/JRSOIlocHTyMokbaMaNgh2oHY6Ns+iqGaiV3uLAw+2lrlWXd2a935R2J99GGwnIbXZa0WmgvA=@vger.kernel.org X-Gm-Message-State: AOJu0YzT/uiKfxWKar1bmnIuA4QaQkZ0W4BC3+xN3oB9sEixd3xW/e/M 325TY8ZQ2mcTz2DnvZN+kLRA7xyRzeOlTyU1ZE3bioJ0OV3P1Jfv X-Google-Smtp-Source: AGHT+IHA05XWUstb450eeCq2YkOYG3UKP/gDh/dgZ6794ND1I3UZATF+JDsVTkAs4xXtweTSIyj5Uw== X-Received: by 2002:aa7:86da:0:b0:71e:b1dc:f255 with SMTP id d2e1a72fcca58-71eb1dcf5e7mr118528b3a.9.1729272641491; Fri, 18 Oct 2024 10:30:41 -0700 (PDT) Received: from dw-tp.ibmuc.com ([171.76.80.151]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71ea3311f51sm1725242b3a.36.2024.10.18.10.30.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Oct 2024 10:30:40 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linuxppc-dev@lists.ozlabs.org Cc: kasan-dev@googlegroups.com, linux-mm@kvack.org, Marco Elver , Alexander Potapenko , Heiko Carstens , Michael Ellerman , Nicholas Piggin , Madhavan Srinivasan , Christophe Leroy , Hari Bathini , "Aneesh Kumar K . V" , Donet Tom , Pavithra Prakash , LKML , "Ritesh Harjani (IBM)" Subject: [PATCH v3 04/12] book3s64/hash: Add hash_debug_pagealloc_add_slot() function Date: Fri, 18 Oct 2024 22:59:45 +0530 Message-ID: <026f0aaa1dddd89154dc8d20ceccfca4f63ccf79.1729271995.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This adds hash_debug_pagealloc_add_slot() function instead of open coding that in htab_bolt_mapping(). This is required since we will be separating kfence functionality to not depend upon debug_pagealloc. No functionality change in this patch. Signed-off-by: Ritesh Harjani (IBM) --- arch/powerpc/mm/book3s64/hash_utils.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/arch/powerpc/mm/book3s64/hash_utils.c b/arch/powerpc/mm/book3s= 64/hash_utils.c index fb2f717e9e74..de3cabd66812 100644 --- a/arch/powerpc/mm/book3s64/hash_utils.c +++ b/arch/powerpc/mm/book3s64/hash_utils.c @@ -328,6 +328,14 @@ static void kernel_unmap_linear_page(unsigned long vad= dr, unsigned long lmi) mmu_kernel_ssize, 0); } =20 +static inline void hash_debug_pagealloc_add_slot(phys_addr_t paddr, int sl= ot) +{ + if (!debug_pagealloc_enabled()) + return; + if ((paddr >> PAGE_SHIFT) < linear_map_hash_count) + linear_map_hash_slots[paddr >> PAGE_SHIFT] =3D slot | 0x80; +} + int hash__kernel_map_pages(struct page *page, int numpages, int enable) { unsigned long flags, vaddr, lmi; @@ -353,6 +361,7 @@ int hash__kernel_map_pages(struct page *page, int numpa= ges, { return 0; } +static inline void hash_debug_pagealloc_add_slot(phys_addr_t paddr, int sl= ot) {} #endif /* CONFIG_DEBUG_PAGEALLOC */ =20 /* @@ -513,9 +522,7 @@ int htab_bolt_mapping(unsigned long vstart, unsigned lo= ng vend, break; =20 cond_resched(); - if (debug_pagealloc_enabled() && - (paddr >> PAGE_SHIFT) < linear_map_hash_count) - linear_map_hash_slots[paddr >> PAGE_SHIFT] =3D ret | 0x80; + hash_debug_pagealloc_add_slot(paddr, ret); } return ret < 0 ? ret : 0; } --=20 2.46.0 From nobody Tue Nov 26 08:24:58 2024 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5E2E9204F7E for ; Fri, 18 Oct 2024 17:30:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729272649; cv=none; b=OtUWfaqXQEmKa6iFmrRehWsaVpXZZRDbnYPfxLQdsxphnuppBdWoTXZinJn/icgmwlyJM/VPeudSjdT0Chjll1NwHrOv03ZSjGJxO1aYDLx5y0cI5nAvD+TaNNE3/+yFzQ5EjpNxFgP0M7BgXks8ntbv05vqLVIJtHYTVWFVrqc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729272649; c=relaxed/simple; bh=E1R3Fouo1t2LcLOJd8YnFBZMjXJ6qjnl450cBxYMuHc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=OG7ZKhPcLkU9CQZHY30nUMuKYnTWgmeGRdlKcpHI6shJObGdqYGguOJOhS26vfRWXOznK+28fGE8ThcQV6W7c/MkF3nGx/F92bd5JyMj6nBqXUjaYJMqI7nHHRQC+WK3VYUFj0fzXm5DY9ub0ORpnBR+JCZHAA7bjzY2BSKISfU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ZfFuc2s0; arc=none smtp.client-ip=209.85.210.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ZfFuc2s0" Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-71e579abb99so1707090b3a.2 for ; Fri, 18 Oct 2024 10:30:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729272646; x=1729877446; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DfUSdtWOnTBS3aUaneALBJjz6yO/1MpkxmVLk8hKBQI=; b=ZfFuc2s0+HsxfAphN9wVfwOqxCyFrsja5boTwpQha2SC4RTOFzxvqCB7M/M9qhqTDM MX+xpirtKgHSewC9BtJdmvwvNyM2NUtoS3srHxMtqDJp0YEGC1EH6NMQ+6Ff53ZIc7Jy EfUPRiqTP/Dal9seRt5quBjUMiT6di27oTrLTAHz1JqFFcybwtocFUaBOgbuV8e2aaw9 nt7+ZKcrE7UUIMxrqQU+bbVq0hmCzAJJVq0vqo2SQfj4WAvKPTaoYhYWLtlX7EYz+FeN 8BwhYZsLhX6EiHEuZlDoSeUiojJkBWBzumjzfjKz4jWogyBlwiPqkpIVKZ+0K1b4+nLi n0tQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729272646; x=1729877446; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DfUSdtWOnTBS3aUaneALBJjz6yO/1MpkxmVLk8hKBQI=; b=Lr1Y/SQZ9ywyB/Ov0yrXxUG55KPQb9exaTg7WFVqRfV5NqySUnCkj4yvDHTewIpH3T XHbHNyQO2osdq6fsH240hFjXM0BjjXwbpthhh6F4J9rXQNCLwqy3FI2N4Peh/7nHFDOo ggxVnZ5KpwLMTg/Xq7djBpECwxteKlluNtalfWPfdUMkFrITsYZVJykZXuM08UdPoWOG WKrnqT/4ZmC8wPmlkBcqMeYDGeiFGZ9oD1elE999jj4/FK1kPo4qeDnG/1zkq7DzvCU4 blin3Velg7j7NZiLTIRva6bnZ3fbIf1Nb4qtUMUm3lfy6Ujyz32A3asW3IlxceZXailg Iw3w== X-Forwarded-Encrypted: i=1; AJvYcCXjUNfwx9JYNcjEKx/+EAknRsfnNDLap62Yq8N4yFWT9kp4HkEmVzqexukZhaOZOqm+C+TJq3qDEdKxESA=@vger.kernel.org X-Gm-Message-State: AOJu0YyCs2D10vVqg89moDQDbifAcewWVyWRb5W7VnRAPi7FZyztVWiz 9A7SFuRPvuzKvTitAhxvtm8hmJtMSx5x+ktvmvXSDMdT94htoKSXIv0prrm6 X-Google-Smtp-Source: AGHT+IEH/PHoy0SvfRofUPBZ0xaaeQuP6mqbjcomFuXt7qmLHoVJL01TXC7JEI5XORTcA+JdNts71g== X-Received: by 2002:a05:6a00:6618:b0:71e:4e2a:38c4 with SMTP id d2e1a72fcca58-71ea31e53d5mr3448284b3a.14.1729272646416; Fri, 18 Oct 2024 10:30:46 -0700 (PDT) Received: from dw-tp.ibmuc.com ([171.76.80.151]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71ea3311f51sm1725242b3a.36.2024.10.18.10.30.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Oct 2024 10:30:45 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linuxppc-dev@lists.ozlabs.org Cc: kasan-dev@googlegroups.com, linux-mm@kvack.org, Marco Elver , Alexander Potapenko , Heiko Carstens , Michael Ellerman , Nicholas Piggin , Madhavan Srinivasan , Christophe Leroy , Hari Bathini , "Aneesh Kumar K . V" , Donet Tom , Pavithra Prakash , LKML , "Ritesh Harjani (IBM)" Subject: [PATCH v3 05/12] book3s64/hash: Add hash_debug_pagealloc_alloc_slots() function Date: Fri, 18 Oct 2024 22:59:46 +0530 Message-ID: X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This adds hash_debug_pagealloc_alloc_slots() function instead of open coding that in htab_initialize(). This is required since we will be separating the kfence functionality to not depend upon debug_pagealloc. Now that everything required for debug_pagealloc is under a #ifdef config. Bring in linear_map_hash_slots and linear_map_hash_count variables under the same config too. Signed-off-by: Ritesh Harjani (IBM) --- arch/powerpc/mm/book3s64/hash_utils.c | 29 ++++++++++++++++----------- 1 file changed, 17 insertions(+), 12 deletions(-) diff --git a/arch/powerpc/mm/book3s64/hash_utils.c b/arch/powerpc/mm/book3s= 64/hash_utils.c index de3cabd66812..0b63acf62d1d 100644 --- a/arch/powerpc/mm/book3s64/hash_utils.c +++ b/arch/powerpc/mm/book3s64/hash_utils.c @@ -123,8 +123,6 @@ EXPORT_SYMBOL_GPL(mmu_slb_size); #ifdef CONFIG_PPC_64K_PAGES int mmu_ci_restrictions; #endif -static u8 *linear_map_hash_slots; -static unsigned long linear_map_hash_count; struct mmu_hash_ops mmu_hash_ops __ro_after_init; EXPORT_SYMBOL(mmu_hash_ops); =20 @@ -274,6 +272,8 @@ void hash__tlbiel_all(unsigned int action) } =20 #ifdef CONFIG_DEBUG_PAGEALLOC +static u8 *linear_map_hash_slots; +static unsigned long linear_map_hash_count; static DEFINE_RAW_SPINLOCK(linear_map_hash_lock); =20 static void kernel_map_linear_page(unsigned long vaddr, unsigned long lmi) @@ -328,6 +328,19 @@ static void kernel_unmap_linear_page(unsigned long vad= dr, unsigned long lmi) mmu_kernel_ssize, 0); } =20 +static inline void hash_debug_pagealloc_alloc_slots(void) +{ + if (!debug_pagealloc_enabled()) + return; + linear_map_hash_count =3D memblock_end_of_DRAM() >> PAGE_SHIFT; + linear_map_hash_slots =3D memblock_alloc_try_nid( + linear_map_hash_count, 1, MEMBLOCK_LOW_LIMIT, + ppc64_rma_size, NUMA_NO_NODE); + if (!linear_map_hash_slots) + panic("%s: Failed to allocate %lu bytes max_addr=3D%pa\n", + __func__, linear_map_hash_count, &ppc64_rma_size); +} + static inline void hash_debug_pagealloc_add_slot(phys_addr_t paddr, int sl= ot) { if (!debug_pagealloc_enabled()) @@ -361,6 +374,7 @@ int hash__kernel_map_pages(struct page *page, int numpa= ges, { return 0; } +static inline void hash_debug_pagealloc_alloc_slots(void) {} static inline void hash_debug_pagealloc_add_slot(phys_addr_t paddr, int sl= ot) {} #endif /* CONFIG_DEBUG_PAGEALLOC */ =20 @@ -1223,16 +1237,7 @@ static void __init htab_initialize(void) =20 prot =3D pgprot_val(PAGE_KERNEL); =20 - if (debug_pagealloc_enabled()) { - linear_map_hash_count =3D memblock_end_of_DRAM() >> PAGE_SHIFT; - linear_map_hash_slots =3D memblock_alloc_try_nid( - linear_map_hash_count, 1, MEMBLOCK_LOW_LIMIT, - ppc64_rma_size, NUMA_NO_NODE); - if (!linear_map_hash_slots) - panic("%s: Failed to allocate %lu bytes max_addr=3D%pa\n", - __func__, linear_map_hash_count, &ppc64_rma_size); - } - + hash_debug_pagealloc_alloc_slots(); /* create bolted the linear mapping in the hash table */ for_each_mem_range(i, &base, &end) { size =3D end - base; --=20 2.46.0 From nobody Tue Nov 26 08:24:58 2024 Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 01785205122 for ; Fri, 18 Oct 2024 17:30:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729272653; cv=none; b=JaVHGxULrgRHCdfxO3N0cmo2aBfTf7SavwreKHTJVAJ0Ckx+UZhqFW4nb4mZGBpKZPNZz5P537wk+7qZ2Z+otQ/bhB7TEyM6TCvayg1IKuAStr9YeUEpM/HAsztS/YxXcyRui51aOLmtB66UJ9Et2Vh0eXWY+vSp4WaJnSpBlas= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729272653; c=relaxed/simple; bh=8xMWMAm2lHPS6fjdW0x2wv3+VGcvv1GToMNZcFopFrc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=p5u34wgpbRk5KuqIXxY/Emr8qNSWCBCYo9C1yVUDxrD8pgEOdx0cF/Yo6LBfOyN/tXyHBDJDOW30C7K4ueQIpNmOza1IE6aJ9k12RRGicjPSNkrFVSu/ylXwjYQfy94WgUHYZL0Wh2MacyxZjiTc1akG3rB3vH1pWOFFcJoeUwE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=nEr/I+RD; arc=none smtp.client-ip=209.85.210.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="nEr/I+RD" Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-71e592d7f6eso1565400b3a.3 for ; Fri, 18 Oct 2024 10:30:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729272651; x=1729877451; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/yT2xkfY2cuEcnEiccNoddLAZz1lOngFGOhNqKs9rtU=; b=nEr/I+RDOtE64fYHtxTHhuDXN3mokG/KJp+cIjI+aqULjQZKr8yX4Z5bDrScr8Bf5j 9TOM44AVdO5oEl2tu1/qVUQarA5xm41W03I7lbgpaA2X3Pu8mTfufqoLGPsgHGe1MWFL AZMbb+rfVxYSrL+UuKF11Wg4L7KMW+jVHf8VRK8EXLxhUGqXffHW/by1oAGi8i1CVA4r LPynxyUQV+qKwNrkZ0uCnuNo88ak43Chaczz6lPq48LkFALONL8xyYeZClJ3gjpe+R+7 Q8NmmN57tLyr7GME/Vdu8eTkPXq91Tu2ik0cbpGhm7XSj7/GppWwSXngo4gDZa5trl39 ACqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729272651; x=1729877451; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/yT2xkfY2cuEcnEiccNoddLAZz1lOngFGOhNqKs9rtU=; b=RofIpk2VQRvJBBe4biTF7wAXiSfSOfzrj5TrYnWNRbp8WpNpqF227FlZPMRC3/Jdxv LfQ3xlSKDdMgfcGdYbzl2GyqrYEfiK0qjQ4TMl/bXOr1nTtdP0savLpAA0mH76O0PJQe GpUHZnvQL2M2rJWuCp+3HNqfvET6lgEbSP0TbEetrFjuvm7r8YRpRBzCIRmbppSPng0e F8viWeCyaQ6xmr6aFilvviMOtmZWzSJj25WPgkbEIu//o/ywiY8rzJTjOML2CTTvJ2fG s4P1vRVPj1UYUp/zhTkePlGGjA5Qu1DGWMMDXX/P5ZedCG8scTvNQyrRnga3+cXXQEb2 t8rw== X-Forwarded-Encrypted: i=1; AJvYcCVwJFG3zmC7XQs8SvfZmryn0p0U9oz5ahU8727uhwtJTTgYJ4aCO6nVz7iqg9ft+tTuOeZmc/RWTOCjB4Y=@vger.kernel.org X-Gm-Message-State: AOJu0Yz/z6APm7f1aFjZdurhapAJWUJ3MbaXy+ARsg2V+XBAuhtSNdOV IpqgYe7rM5flscrwX+dFJGJQv4bIwhrBz/U/ztcZgZdmFFzjSa3M X-Google-Smtp-Source: AGHT+IHW0pQgponGx/gVpoZ4bowEpREiydwVrhTTv3Smj0fSfHzFmSnPLW4FIXTVD3Eg6sTX/fZl5A== X-Received: by 2002:a05:6a21:3a96:b0:1d8:fcf2:9ce2 with SMTP id adf61e73a8af0-1d92c57db72mr3781064637.44.1729272651290; Fri, 18 Oct 2024 10:30:51 -0700 (PDT) Received: from dw-tp.ibmuc.com ([171.76.80.151]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71ea3311f51sm1725242b3a.36.2024.10.18.10.30.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Oct 2024 10:30:50 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linuxppc-dev@lists.ozlabs.org Cc: kasan-dev@googlegroups.com, linux-mm@kvack.org, Marco Elver , Alexander Potapenko , Heiko Carstens , Michael Ellerman , Nicholas Piggin , Madhavan Srinivasan , Christophe Leroy , Hari Bathini , "Aneesh Kumar K . V" , Donet Tom , Pavithra Prakash , LKML , "Ritesh Harjani (IBM)" Subject: [PATCH v3 06/12] book3s64/hash: Refactor hash__kernel_map_pages() function Date: Fri, 18 Oct 2024 22:59:47 +0530 Message-ID: <0cb8ddcccdcf61ea06ab4d92aacd770c16cc0f2c.1729271995.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This refactors hash__kernel_map_pages() function to call hash_debug_pagealloc_map_pages(). This will come useful when we will add kfence support. No functionality changes in this patch. Signed-off-by: Ritesh Harjani (IBM) --- arch/powerpc/mm/book3s64/hash_utils.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/mm/book3s64/hash_utils.c b/arch/powerpc/mm/book3s= 64/hash_utils.c index 0b63acf62d1d..ab50bb33a390 100644 --- a/arch/powerpc/mm/book3s64/hash_utils.c +++ b/arch/powerpc/mm/book3s64/hash_utils.c @@ -349,7 +349,8 @@ static inline void hash_debug_pagealloc_add_slot(phys_a= ddr_t paddr, int slot) linear_map_hash_slots[paddr >> PAGE_SHIFT] =3D slot | 0x80; } =20 -int hash__kernel_map_pages(struct page *page, int numpages, int enable) +static int hash_debug_pagealloc_map_pages(struct page *page, int numpages, + int enable) { unsigned long flags, vaddr, lmi; int i; @@ -368,6 +369,12 @@ int hash__kernel_map_pages(struct page *page, int nump= ages, int enable) local_irq_restore(flags); return 0; } + +int hash__kernel_map_pages(struct page *page, int numpages, int enable) +{ + return hash_debug_pagealloc_map_pages(page, numpages, enable); +} + #else /* CONFIG_DEBUG_PAGEALLOC */ int hash__kernel_map_pages(struct page *page, int numpages, int enable) --=20 2.46.0 From nobody Tue Nov 26 08:24:58 2024 Received: from mail-pg1-f180.google.com (mail-pg1-f180.google.com [209.85.215.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 81F5C205122 for ; Fri, 18 Oct 2024 17:30:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729272659; cv=none; b=jgLyALLMTJG8XHtlUlTBUQrOKy4NWecf873ISRNBhk5pdNRPe2SXNCLrqTf81Td3dFi0+RgCfP5eNjqh+RKcTqdXxWtrGWVKFykA40EdQuFvo6a3Y8ZrFcbcHukT5fbPvfCjRoJFF8EPjme6penetem7B2ZjzNx3w3UREi9qr3c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729272659; c=relaxed/simple; bh=vwo5Vjz6fiE2bBnN6H3uBn6axl3BU1PmSdcjm6+/cIo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TAFFx7mHz1pCJ80T9H+dUDBY0hJmoq/53ITrqGSfRlNCLsTKrbFQS6c0Lnq+yT23AKVsWiefalwCRCG3irtRroNEO43ndrV0d3vbWdeBGSVgJYBJ/EBSUG2qVvfNTQVkRyckpRZ0NCli5b5qDxJzgyUPzQty0IMt9u6515FpUwI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=gPP/10/0; arc=none smtp.client-ip=209.85.215.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="gPP/10/0" Received: by mail-pg1-f180.google.com with SMTP id 41be03b00d2f7-7d916b6a73aso1549447a12.1 for ; Fri, 18 Oct 2024 10:30:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729272657; x=1729877457; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=SYv3qxbJ8XzSYVM3O5y7o2xiEmCUj8b+8LHRXlL50QE=; b=gPP/10/0EIieVyt3uWCYCSk2KK7uc3covXIWsnNR0PTVitBfGgxLeX7NV0qRQOqnki Kc/Atjrl0ax8nmQhYJRqEfXk5SKUTHQCK4CZWoHYHNyJBEbwWFg9L+uR4BD57E1TaWHj rO702iOrqBB7YHmb2vUbXifFf7A9VgQVTLvG8wcjXcY6mY5jjGrqfA8jbG1bfuqxAoUl lmPCfOjpqTMTQy2SlChqZHtDb8aWfpin7qbCSCkWiLQ6S6Y8Sg+cRpBp/viVzhqwlIpq kaxeZGVUQNvSJlO+UOJX5kFS9CbaUxXZGFkEvSxMLjqbRI7U7FWymuZTIaN0hv2C553h w89A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729272657; x=1729877457; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=SYv3qxbJ8XzSYVM3O5y7o2xiEmCUj8b+8LHRXlL50QE=; b=lUUn21w27rcSi9nFRj8rJs8pqbXSykVvbGjKWy61ZFm7h9CDi3gb/vCO56nb7sL5cE 9GQZxPEVf8PbqppdKaORPOvemq+q1EqQ5CnxZ5ZAD+/8lUexAPEGjz6TJ7SVm2vK8ubt BIqlj4i4Apr49QqZEQN25slLx+Hb8PwaQ2gLhOwuRojlJwee4CQWIFuuotvJwpNc+Hij Q/B76F7fBZTSztl6MjlKNCtPgJmDu8PcubnXJIHQ+10/iEbrmSSKIUQCBJHmDzAmm45r W7tY/NUaMw9ZMwVGd1IFw0anbwWEB82+VvzyAaH37Bj2UhTmWgiBAN5I9xqPVxvQd+po PYnw== X-Forwarded-Encrypted: i=1; AJvYcCW2CQuoO5IXhLYZkRMMS7zRR5VFNYssMQrbGczq8ZBAXcpSFghJZAoJbFcSuZEw/8Rh5BQdljqU+pA+J8g=@vger.kernel.org X-Gm-Message-State: AOJu0YxO0G5Q9f439Ek0ONUhVKoJ8W1fUxIsrrWjdSEvVIM6PwlvAhsG TvFEtIH1i6cV9CVePk6BQbh8oFJQ9r0q8prRo2LEYPm665MJJ00M X-Google-Smtp-Source: AGHT+IGzdUgXGWS3Rnfne00N0lzoqmJmv13oE/GEcsOOigglv7dM20Xc4zaopt4Axws81Q8JXv67zw== X-Received: by 2002:a05:6a21:a4c1:b0:1d8:f1f4:f4ee with SMTP id adf61e73a8af0-1d92c4baaaemr4986521637.8.1729272656623; Fri, 18 Oct 2024 10:30:56 -0700 (PDT) Received: from dw-tp.ibmuc.com ([171.76.80.151]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71ea3311f51sm1725242b3a.36.2024.10.18.10.30.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Oct 2024 10:30:55 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linuxppc-dev@lists.ozlabs.org Cc: kasan-dev@googlegroups.com, linux-mm@kvack.org, Marco Elver , Alexander Potapenko , Heiko Carstens , Michael Ellerman , Nicholas Piggin , Madhavan Srinivasan , Christophe Leroy , Hari Bathini , "Aneesh Kumar K . V" , Donet Tom , Pavithra Prakash , LKML , "Ritesh Harjani (IBM)" Subject: [PATCH v3 07/12] book3s64/hash: Make kernel_map_linear_page() generic Date: Fri, 18 Oct 2024 22:59:48 +0530 Message-ID: <5b67df7b29e68d7c78d6fc1f42d41137299bac6b.1729271995.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Currently kernel_map_linear_page() function assumes to be working on linear_map_hash_slots array. But since in later patches we need a separate linear map array for kfence, hence make kernel_map_linear_page() take a linear map array and lock in it's function argument. This is needed to separate out kfence from debug_pagealloc infrastructure. Signed-off-by: Ritesh Harjani (IBM) --- arch/powerpc/mm/book3s64/hash_utils.c | 47 ++++++++++++++------------- 1 file changed, 25 insertions(+), 22 deletions(-) diff --git a/arch/powerpc/mm/book3s64/hash_utils.c b/arch/powerpc/mm/book3s= 64/hash_utils.c index ab50bb33a390..11975a2f7403 100644 --- a/arch/powerpc/mm/book3s64/hash_utils.c +++ b/arch/powerpc/mm/book3s64/hash_utils.c @@ -272,11 +272,8 @@ void hash__tlbiel_all(unsigned int action) } =20 #ifdef CONFIG_DEBUG_PAGEALLOC -static u8 *linear_map_hash_slots; -static unsigned long linear_map_hash_count; -static DEFINE_RAW_SPINLOCK(linear_map_hash_lock); - -static void kernel_map_linear_page(unsigned long vaddr, unsigned long lmi) +static void kernel_map_linear_page(unsigned long vaddr, unsigned long idx, + u8 *slots, raw_spinlock_t *lock) { unsigned long hash; unsigned long vsid =3D get_kernel_vsid(vaddr, mmu_kernel_ssize); @@ -290,7 +287,7 @@ static void kernel_map_linear_page(unsigned long vaddr,= unsigned long lmi) if (!vsid) return; =20 - if (linear_map_hash_slots[lmi] & 0x80) + if (slots[idx] & 0x80) return; =20 ret =3D hpte_insert_repeating(hash, vpn, __pa(vaddr), mode, @@ -298,36 +295,40 @@ static void kernel_map_linear_page(unsigned long vadd= r, unsigned long lmi) mmu_linear_psize, mmu_kernel_ssize); =20 BUG_ON (ret < 0); - raw_spin_lock(&linear_map_hash_lock); - BUG_ON(linear_map_hash_slots[lmi] & 0x80); - linear_map_hash_slots[lmi] =3D ret | 0x80; - raw_spin_unlock(&linear_map_hash_lock); + raw_spin_lock(lock); + BUG_ON(slots[idx] & 0x80); + slots[idx] =3D ret | 0x80; + raw_spin_unlock(lock); } =20 -static void kernel_unmap_linear_page(unsigned long vaddr, unsigned long lm= i) +static void kernel_unmap_linear_page(unsigned long vaddr, unsigned long id= x, + u8 *slots, raw_spinlock_t *lock) { - unsigned long hash, hidx, slot; + unsigned long hash, hslot, slot; unsigned long vsid =3D get_kernel_vsid(vaddr, mmu_kernel_ssize); unsigned long vpn =3D hpt_vpn(vaddr, vsid, mmu_kernel_ssize); =20 hash =3D hpt_hash(vpn, PAGE_SHIFT, mmu_kernel_ssize); - raw_spin_lock(&linear_map_hash_lock); - if (!(linear_map_hash_slots[lmi] & 0x80)) { - raw_spin_unlock(&linear_map_hash_lock); + raw_spin_lock(lock); + if (!(slots[idx] & 0x80)) { + raw_spin_unlock(lock); return; } - hidx =3D linear_map_hash_slots[lmi] & 0x7f; - linear_map_hash_slots[lmi] =3D 0; - raw_spin_unlock(&linear_map_hash_lock); - if (hidx & _PTEIDX_SECONDARY) + hslot =3D slots[idx] & 0x7f; + slots[idx] =3D 0; + raw_spin_unlock(lock); + if (hslot & _PTEIDX_SECONDARY) hash =3D ~hash; slot =3D (hash & htab_hash_mask) * HPTES_PER_GROUP; - slot +=3D hidx & _PTEIDX_GROUP_IX; + slot +=3D hslot & _PTEIDX_GROUP_IX; mmu_hash_ops.hpte_invalidate(slot, vpn, mmu_linear_psize, mmu_linear_psize, mmu_kernel_ssize, 0); } =20 +static u8 *linear_map_hash_slots; +static unsigned long linear_map_hash_count; +static DEFINE_RAW_SPINLOCK(linear_map_hash_lock); static inline void hash_debug_pagealloc_alloc_slots(void) { if (!debug_pagealloc_enabled()) @@ -362,9 +363,11 @@ static int hash_debug_pagealloc_map_pages(struct page = *page, int numpages, if (lmi >=3D linear_map_hash_count) continue; if (enable) - kernel_map_linear_page(vaddr, lmi); + kernel_map_linear_page(vaddr, lmi, + linear_map_hash_slots, &linear_map_hash_lock); else - kernel_unmap_linear_page(vaddr, lmi); + kernel_unmap_linear_page(vaddr, lmi, + linear_map_hash_slots, &linear_map_hash_lock); } local_irq_restore(flags); return 0; --=20 2.46.0 From nobody Tue Nov 26 08:24:58 2024 Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5CE73205AA5 for ; Fri, 18 Oct 2024 17:31:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729272664; cv=none; b=OPaw0hyfmIX+H95HeF2fwXkxsoS2fI1JxNfHEeOPdoT9ajUgSJlq50VFsCLAnX8OaVqVudJqEPD43c/mMM2bBLr2K2xqTeMwEAZXBnmiVDsJX8BjPLj25y/fJZb5/pfkRBsNAUhjCx2rsYh56V9+yaniQtyCg6ED8D7dlIUpKbI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729272664; c=relaxed/simple; bh=YLI0r8N5pRa8ImeMxY5MDp0IiftU++zZzlcW81OslQo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kSBLBYrEfqgLXClt9SKT7Crrtlj2EJ6W8RT4sipSqbIn9JbxF7PgHRJHuTsA2IeOzQzg5bPWgu+agWanTqvJBctrhvJvqHFH38/qcQARM65prfj3Vl/zF38Ehb0vGwH9ysiZryYPzoc2B48BYPJxbk2szPFduJ2f7VEiou2VFbw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=NfN757dz; arc=none smtp.client-ip=209.85.210.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="NfN757dz" Received: by mail-pf1-f172.google.com with SMTP id d2e1a72fcca58-71e70c32cd7so1998283b3a.1 for ; Fri, 18 Oct 2024 10:31:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729272661; x=1729877461; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=c7nu+U6UkRZF2cuN8FLbB8nvBf84Wg4YjQThyWhRKHk=; b=NfN757dzhgwnB7z6Z3tUtmdL3z+R6uma6ge30WORzQzDqvA0AoGuiiOhZW2tTEaAUq zBHSCw1BGjwb7AhtvgtgkSearPo45z7fuOLZ/ahl7BWI2mABN9lebYJSlmGknhBK5nvE I2vKN5p9/RA51+TwOPsOOugn/YsX4hyy8YXmzuYY3712YhFcZxUIVk2RQzLRj+L/tJ7S bqn/lTKJ3egmpqA0oaTE77cI87lyKxzZLr+/MTk8mvTrYVT+AtTkOC5VVXcIgyEucdfX eaCcM8fQUQwunS+OIS8p/rHW6t7wfNEV6DrOzOy2wmlotN0SKYdW4JzdKUcN9ywBGSid Tx5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729272661; x=1729877461; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=c7nu+U6UkRZF2cuN8FLbB8nvBf84Wg4YjQThyWhRKHk=; b=ZZYgsSJ5+Vth1xXZs/Uk+Gj2ckWlSPtxK1ljpJTkUzOiT2YPLrf+Pq6nkzFKIA79vZ NR8lwiP/H5kMGr+Hj+gbdz5ANUZlWGGIE7xWy+qAAYwUfBy+9lKWWJIwmQ7Ge9i5w5eq TD04k8YIjx4vJKsOKzL3ySce9NeFMFxPyQ6gTn0qABI8yBC6FdDTSk+NeiDYqkDASelH jIUt8ELdr13O/0cETMpPtBuQTs55XtxB5dD8eZVTcPcZGNCXoak/Ie9GO1M2Oq0Js/mv uQmjyhonr29LETkUgzeHgQwC7VXKpd5TWWElHuGAeP7lPE7lemzsUiIFjDvVtYC2Oyxw G4yg== X-Forwarded-Encrypted: i=1; AJvYcCVayrzL1RDw5rixY/W2BJYQrva1pvovMWhMYYZB97BeCZUtWkZqCc0K7hZk4kZfxbc/ll7R24VjAbSjhME=@vger.kernel.org X-Gm-Message-State: AOJu0YxZ07jBDdctKWcnREt1pyIMIf2YfrRWEg8jROKi8UKpBRjyMnVr FPtV2UpjdvJRFnFElbJiCL5uvOM4M6L/nHixs1j1CAx/Uz1XpEwn X-Google-Smtp-Source: AGHT+IHJYrrrqJRfAumO9Q20FqjOKsTO5quh/OtUWisgh37hgw56Q3PPkpt1yyNYXV1te9hYA/UiKQ== X-Received: by 2002:a05:6a21:1519:b0:1d8:a3ab:7228 with SMTP id adf61e73a8af0-1d92c3323d2mr4881461637.0.1729272661553; Fri, 18 Oct 2024 10:31:01 -0700 (PDT) Received: from dw-tp.ibmuc.com ([171.76.80.151]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71ea3311f51sm1725242b3a.36.2024.10.18.10.30.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Oct 2024 10:31:00 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linuxppc-dev@lists.ozlabs.org Cc: kasan-dev@googlegroups.com, linux-mm@kvack.org, Marco Elver , Alexander Potapenko , Heiko Carstens , Michael Ellerman , Nicholas Piggin , Madhavan Srinivasan , Christophe Leroy , Hari Bathini , "Aneesh Kumar K . V" , Donet Tom , Pavithra Prakash , LKML , "Ritesh Harjani (IBM)" Subject: [PATCH v3 08/12] book3s64/hash: Disable debug_pagealloc if it requires more memory Date: Fri, 18 Oct 2024 22:59:49 +0530 Message-ID: X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Make size of the linear map to be allocated in RMA region to be of ppc64_rma_size / 4. If debug_pagealloc requires more memory than that then do not allocate any memory and disable debug_pagealloc. Signed-off-by: Ritesh Harjani (IBM) --- arch/powerpc/mm/book3s64/hash_utils.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/mm/book3s64/hash_utils.c b/arch/powerpc/mm/book3s= 64/hash_utils.c index 11975a2f7403..f51f2cd9bf22 100644 --- a/arch/powerpc/mm/book3s64/hash_utils.c +++ b/arch/powerpc/mm/book3s64/hash_utils.c @@ -331,9 +331,19 @@ static unsigned long linear_map_hash_count; static DEFINE_RAW_SPINLOCK(linear_map_hash_lock); static inline void hash_debug_pagealloc_alloc_slots(void) { + unsigned long max_hash_count =3D ppc64_rma_size / 4; + if (!debug_pagealloc_enabled()) return; linear_map_hash_count =3D memblock_end_of_DRAM() >> PAGE_SHIFT; + if (unlikely(linear_map_hash_count > max_hash_count)) { + pr_info("linear map size (%llu) greater than 4 times RMA region (%llu). = Disabling debug_pagealloc\n", + ((u64)linear_map_hash_count << PAGE_SHIFT), + ppc64_rma_size); + linear_map_hash_count =3D 0; + return; + } + linear_map_hash_slots =3D memblock_alloc_try_nid( linear_map_hash_count, 1, MEMBLOCK_LOW_LIMIT, ppc64_rma_size, NUMA_NO_NODE); @@ -344,7 +354,7 @@ static inline void hash_debug_pagealloc_alloc_slots(voi= d) =20 static inline void hash_debug_pagealloc_add_slot(phys_addr_t paddr, int sl= ot) { - if (!debug_pagealloc_enabled()) + if (!debug_pagealloc_enabled() || !linear_map_hash_count) return; if ((paddr >> PAGE_SHIFT) < linear_map_hash_count) linear_map_hash_slots[paddr >> PAGE_SHIFT] =3D slot | 0x80; @@ -356,6 +366,9 @@ static int hash_debug_pagealloc_map_pages(struct page *= page, int numpages, unsigned long flags, vaddr, lmi; int i; =20 + if (!debug_pagealloc_enabled() || !linear_map_hash_count) + return 0; + local_irq_save(flags); for (i =3D 0; i < numpages; i++, page++) { vaddr =3D (unsigned long)page_address(page); --=20 2.46.0 From nobody Tue Nov 26 08:24:58 2024 Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9D304205AC2 for ; Fri, 18 Oct 2024 17:31:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729272669; cv=none; b=dosy66uc4K+l5rPG/YM8m3QcsxnqlF2HQX8YovfaehSW6MDoQovULeKaB1IN/R/7w4fpTntmC++Nc2Wk3Xt7hU5bBDz94q+BxfhD/CdYKgaLVbSrxo7vcf486aemsYMk6c8Oi/W28qqc7exr60ye5G9x+HA2wG4DAcd+Pt2Ap2U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729272669; c=relaxed/simple; bh=0ZU3PBuusYz0BC8RPwMscqOGtYWO9M254Vd7xILudqU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=CmH4AR+dToAw4Ng+0rVYQEQQmQgO+ZpTaRbY5bGU5CUY45dEywWSPvS7znz24+VS45aQVpIgsbIUGZTxv28/E86V1eXQ01UqmNl5LtxH3tHkOpZu13m8Qai7OxrpU8Qkv0CPfvsBA4HDmVUvTfZhZio3gRLv2ugrNnbk0/rMFbE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=eX7KpWI7; arc=none smtp.client-ip=209.85.210.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="eX7KpWI7" Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-71e7086c231so1828618b3a.0 for ; Fri, 18 Oct 2024 10:31:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729272667; x=1729877467; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=EPaaeu37JE/wM6SvauUhRnxy0xWFM+VjQPTyakMCkTU=; b=eX7KpWI7Ft/IoJRr0+qPd22FJBYSBn9n1/JeK5zkC866Lu45+2cnhsOF+r1pnBKPtL Si/Tj4Risfbh8Sb/Zz4rgND0NyERN20keFKSTMtlR6EVs5albiffLhHqmb2Btyc5WN3e gcV9LMsDHX3csVSh4esjSkb3x9SNBWJFqdaKs8ey75ZuxygNr3n1TEkulp3Z2PErt8fC RrUQkqGCWmQp0y4ZyFSkGpKZ2NDRmQGzsv92y+U4seQgm5WXaKpyDVuC8ZyuLuBJi2nV zZ6PyxA+rX7eVhqWjFu8lccMPxngX59jhFS1R3/Qb0/XMrcTSvCqjagPOf8Zrf+X95Gw Z7yg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729272667; x=1729877467; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=EPaaeu37JE/wM6SvauUhRnxy0xWFM+VjQPTyakMCkTU=; b=A4i2T4Fgneqd65W+fu7MufCRqqxO/lG4AYc2SDFemF6y9tuvfXXnKsO5XMQC1BRbjS yh7roBKiinyeIkTZz999qVH5AsYOwPba9wHlNx3nwWDjEcGocNSD9vukqWXDOEYBCT5o zr7jazegEH70SAIu/ZFi2bCyphG/PViRaxq1W5hmfV5yVK2gvRCA3sZMbDC1LamrkKDk zxgwYBjhR/qqUN9u3nU4CS549MpODFLlVUxi51o3DyHULr5wQSf440Yl7uRIK3Lb2ocp gCchAtvzSkX7VaGL8mcpv3h43bcoYBG013GSkiiZmeNGtLZJEN4B6sgvtQMiJ/LPMZtE Ruug== X-Forwarded-Encrypted: i=1; AJvYcCUtFFSgBqZTzbAVK2MLLHVS3RL1FpgegJ7icxX6soDdvNLbVGU1RWonFnKPfHz6DbA/x9O7+vn8D9aFTpU=@vger.kernel.org X-Gm-Message-State: AOJu0YyaxUpCYxg4A6p9PfCh5xY7ORCdcK2RUSSKG6qUjMpZ9cEFrLlv Yb49SVXoXnqEHRz6PjbAFV+GNeHg6bNUdqYSr9jRjYAgwlkOebmV X-Google-Smtp-Source: AGHT+IEEe+ByOf1Adz9XWU2Tes8g7tZEwmexWC38hJ63m0WcdhEx6XCci2ar7YRCIZhlcwi/ce3HQw== X-Received: by 2002:a05:6a00:2d25:b0:71e:5950:97d2 with SMTP id d2e1a72fcca58-71ea3328e08mr4294187b3a.17.1729272666771; Fri, 18 Oct 2024 10:31:06 -0700 (PDT) Received: from dw-tp.ibmuc.com ([171.76.80.151]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71ea3311f51sm1725242b3a.36.2024.10.18.10.31.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Oct 2024 10:31:05 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linuxppc-dev@lists.ozlabs.org Cc: kasan-dev@googlegroups.com, linux-mm@kvack.org, Marco Elver , Alexander Potapenko , Heiko Carstens , Michael Ellerman , Nicholas Piggin , Madhavan Srinivasan , Christophe Leroy , Hari Bathini , "Aneesh Kumar K . V" , Donet Tom , Pavithra Prakash , LKML , "Ritesh Harjani (IBM)" Subject: [PATCH v3 09/12] book3s64/hash: Add kfence functionality Date: Fri, 18 Oct 2024 22:59:50 +0530 Message-ID: <5c2b61941b344077a2b8654dab46efa0322af3af.1729271995.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Now that linear map functionality of debug_pagealloc is made generic, enable kfence to use this generic infrastructure. 1. Define kfence related linear map variables. - u8 *linear_map_kf_hash_slots; - unsigned long linear_map_kf_hash_count; - DEFINE_RAW_SPINLOCK(linear_map_kf_hash_lock); 2. The linear map size allocated in RMA region is quite small (KFENCE_POOL_SIZE >> PAGE_SHIFT) which is 512 bytes by default. 3. kfence pool memory is reserved using memblock_phys_alloc() which has can come from anywhere. (default 255 objects =3D> ((1+255) * 2) << PAGE_SHIFT =3D 32MB) 4. The hash slot information for kfence memory gets added in linear map in hash_linear_map_add_slot() (which also adds for debug_pagealloc). Reported-by: Pavithra Prakash Signed-off-by: Ritesh Harjani (IBM) --- arch/powerpc/include/asm/kfence.h | 5 - arch/powerpc/mm/book3s64/hash_utils.c | 162 +++++++++++++++++++++++--- 2 files changed, 149 insertions(+), 18 deletions(-) diff --git a/arch/powerpc/include/asm/kfence.h b/arch/powerpc/include/asm/k= fence.h index f3a9476a71b3..fab124ada1c7 100644 --- a/arch/powerpc/include/asm/kfence.h +++ b/arch/powerpc/include/asm/kfence.h @@ -10,7 +10,6 @@ =20 #include #include -#include =20 #ifdef CONFIG_PPC64_ELF_ABI_V1 #define ARCH_FUNC_PREFIX "." @@ -26,10 +25,6 @@ static inline void disable_kfence(void) =20 static inline bool arch_kfence_init_pool(void) { -#ifdef CONFIG_PPC64 - if (!radix_enabled()) - return false; -#endif return !kfence_disabled; } #endif diff --git a/arch/powerpc/mm/book3s64/hash_utils.c b/arch/powerpc/mm/book3s= 64/hash_utils.c index f51f2cd9bf22..558d6f5202b9 100644 --- a/arch/powerpc/mm/book3s64/hash_utils.c +++ b/arch/powerpc/mm/book3s64/hash_utils.c @@ -40,6 +40,7 @@ #include #include #include +#include =20 #include #include @@ -66,6 +67,7 @@ #include #include #include +#include =20 #include =20 @@ -271,7 +273,7 @@ void hash__tlbiel_all(unsigned int action) WARN(1, "%s called on pre-POWER7 CPU\n", __func__); } =20 -#ifdef CONFIG_DEBUG_PAGEALLOC +#if defined(CONFIG_DEBUG_PAGEALLOC) || defined(CONFIG_KFENCE) static void kernel_map_linear_page(unsigned long vaddr, unsigned long idx, u8 *slots, raw_spinlock_t *lock) { @@ -325,11 +327,13 @@ static void kernel_unmap_linear_page(unsigned long va= ddr, unsigned long idx, mmu_linear_psize, mmu_kernel_ssize, 0); } +#endif =20 +#ifdef CONFIG_DEBUG_PAGEALLOC static u8 *linear_map_hash_slots; static unsigned long linear_map_hash_count; static DEFINE_RAW_SPINLOCK(linear_map_hash_lock); -static inline void hash_debug_pagealloc_alloc_slots(void) +static void hash_debug_pagealloc_alloc_slots(void) { unsigned long max_hash_count =3D ppc64_rma_size / 4; =20 @@ -352,7 +356,8 @@ static inline void hash_debug_pagealloc_alloc_slots(voi= d) __func__, linear_map_hash_count, &ppc64_rma_size); } =20 -static inline void hash_debug_pagealloc_add_slot(phys_addr_t paddr, int sl= ot) +static inline void hash_debug_pagealloc_add_slot(phys_addr_t paddr, + int slot) { if (!debug_pagealloc_enabled() || !linear_map_hash_count) return; @@ -386,20 +391,148 @@ static int hash_debug_pagealloc_map_pages(struct pag= e *page, int numpages, return 0; } =20 -int hash__kernel_map_pages(struct page *page, int numpages, int enable) +#else /* CONFIG_DEBUG_PAGEALLOC */ +static inline void hash_debug_pagealloc_alloc_slots(void) {} +static inline void hash_debug_pagealloc_add_slot(phys_addr_t paddr, int sl= ot) {} +static int __maybe_unused +hash_debug_pagealloc_map_pages(struct page *page, int numpages, int enable) { - return hash_debug_pagealloc_map_pages(page, numpages, enable); + return 0; } +#endif /* CONFIG_DEBUG_PAGEALLOC */ =20 -#else /* CONFIG_DEBUG_PAGEALLOC */ -int hash__kernel_map_pages(struct page *page, int numpages, - int enable) +#ifdef CONFIG_KFENCE +static u8 *linear_map_kf_hash_slots; +static unsigned long linear_map_kf_hash_count; +static DEFINE_RAW_SPINLOCK(linear_map_kf_hash_lock); + +static phys_addr_t kfence_pool; + +static inline void hash_kfence_alloc_pool(void) +{ + + /* allocate linear map for kfence within RMA region */ + linear_map_kf_hash_count =3D KFENCE_POOL_SIZE >> PAGE_SHIFT; + linear_map_kf_hash_slots =3D memblock_alloc_try_nid( + linear_map_kf_hash_count, 1, + MEMBLOCK_LOW_LIMIT, ppc64_rma_size, + NUMA_NO_NODE); + if (!linear_map_kf_hash_slots) { + pr_err("%s: memblock for linear map (%lu) failed\n", __func__, + linear_map_kf_hash_count); + goto err; + } + + /* allocate kfence pool early */ + kfence_pool =3D memblock_phys_alloc_range(KFENCE_POOL_SIZE, PAGE_SIZE, + MEMBLOCK_LOW_LIMIT, MEMBLOCK_ALLOC_ANYWHERE); + if (!kfence_pool) { + pr_err("%s: memblock for kfence pool (%lu) failed\n", __func__, + KFENCE_POOL_SIZE); + memblock_free(linear_map_kf_hash_slots, + linear_map_kf_hash_count); + linear_map_kf_hash_count =3D 0; + goto err; + } + memblock_mark_nomap(kfence_pool, KFENCE_POOL_SIZE); + + return; +err: + pr_info("Disabling kfence\n"); + disable_kfence(); +} + +static inline void hash_kfence_map_pool(void) +{ + unsigned long kfence_pool_start, kfence_pool_end; + unsigned long prot =3D pgprot_val(PAGE_KERNEL); + + if (!kfence_pool) + return; + + kfence_pool_start =3D (unsigned long) __va(kfence_pool); + kfence_pool_end =3D kfence_pool_start + KFENCE_POOL_SIZE; + __kfence_pool =3D (char *) kfence_pool_start; + BUG_ON(htab_bolt_mapping(kfence_pool_start, kfence_pool_end, + kfence_pool, prot, mmu_linear_psize, + mmu_kernel_ssize)); + memblock_clear_nomap(kfence_pool, KFENCE_POOL_SIZE); +} + +static inline void hash_kfence_add_slot(phys_addr_t paddr, int slot) { + unsigned long vaddr =3D (unsigned long) __va(paddr); + unsigned long lmi =3D (vaddr - (unsigned long)__kfence_pool) + >> PAGE_SHIFT; + + if (!kfence_pool) + return; + BUG_ON(!is_kfence_address((void *)vaddr)); + BUG_ON(lmi >=3D linear_map_kf_hash_count); + linear_map_kf_hash_slots[lmi] =3D slot | 0x80; +} + +static int hash_kfence_map_pages(struct page *page, int numpages, int enab= le) +{ + unsigned long flags, vaddr, lmi; + int i; + + WARN_ON_ONCE(!linear_map_kf_hash_count); + local_irq_save(flags); + for (i =3D 0; i < numpages; i++, page++) { + vaddr =3D (unsigned long)page_address(page); + lmi =3D (vaddr - (unsigned long)__kfence_pool) >> PAGE_SHIFT; + + /* Ideally this should never happen */ + if (lmi >=3D linear_map_kf_hash_count) { + WARN_ON_ONCE(1); + continue; + } + + if (enable) + kernel_map_linear_page(vaddr, lmi, + linear_map_kf_hash_slots, + &linear_map_kf_hash_lock); + else + kernel_unmap_linear_page(vaddr, lmi, + linear_map_kf_hash_slots, + &linear_map_kf_hash_lock); + } + local_irq_restore(flags); return 0; } -static inline void hash_debug_pagealloc_alloc_slots(void) {} -static inline void hash_debug_pagealloc_add_slot(phys_addr_t paddr, int sl= ot) {} -#endif /* CONFIG_DEBUG_PAGEALLOC */ +#else +static inline void hash_kfence_alloc_pool(void) {} +static inline void hash_kfence_map_pool(void) {} +static inline void hash_kfence_add_slot(phys_addr_t paddr, int slot) {} +static int __maybe_unused +hash_kfence_map_pages(struct page *page, int numpages, int enable) +{ + return 0; +} +#endif + +#if defined(CONFIG_DEBUG_PAGEALLOC) || defined(CONFIG_KFENCE) +int hash__kernel_map_pages(struct page *page, int numpages, int enable) +{ + void *vaddr =3D page_address(page); + + if (is_kfence_address(vaddr)) + return hash_kfence_map_pages(page, numpages, enable); + else + return hash_debug_pagealloc_map_pages(page, numpages, enable); +} + +static void hash_linear_map_add_slot(phys_addr_t paddr, int slot) +{ + if (is_kfence_address(__va(paddr))) + hash_kfence_add_slot(paddr, slot); + else + hash_debug_pagealloc_add_slot(paddr, slot); +} +#else +static void hash_linear_map_add_slot(phys_addr_t paddr, int slot) {} +#endif =20 /* * 'R' and 'C' update notes: @@ -559,7 +692,8 @@ int htab_bolt_mapping(unsigned long vstart, unsigned lo= ng vend, break; =20 cond_resched(); - hash_debug_pagealloc_add_slot(paddr, ret); + /* add slot info in debug_pagealloc / kfence linear map */ + hash_linear_map_add_slot(paddr, ret); } return ret < 0 ? ret : 0; } @@ -940,7 +1074,7 @@ static void __init htab_init_page_sizes(void) bool aligned =3D true; init_hpte_page_sizes(); =20 - if (!debug_pagealloc_enabled()) { + if (!debug_pagealloc_enabled_or_kfence()) { /* * Pick a size for the linear mapping. Currently, we only * support 16M, 1M and 4K which is the default @@ -1261,6 +1395,7 @@ static void __init htab_initialize(void) prot =3D pgprot_val(PAGE_KERNEL); =20 hash_debug_pagealloc_alloc_slots(); + hash_kfence_alloc_pool(); /* create bolted the linear mapping in the hash table */ for_each_mem_range(i, &base, &end) { size =3D end - base; @@ -1277,6 +1412,7 @@ static void __init htab_initialize(void) BUG_ON(htab_bolt_mapping(base, base + size, __pa(base), prot, mmu_linear_psize, mmu_kernel_ssize)); } + hash_kfence_map_pool(); memblock_set_current_limit(MEMBLOCK_ALLOC_ANYWHERE); =20 /* --=20 2.46.0 From nobody Tue Nov 26 08:24:58 2024 Received: from mail-pg1-f176.google.com (mail-pg1-f176.google.com [209.85.215.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 402D8205AC6 for ; Fri, 18 Oct 2024 17:31:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729272675; cv=none; b=bUIcJ8UPFeZIh8lNrf+zMml49hOq/36qKH8qPE6YgxAgjzhs0xbpCBCxYGXnNmiSr6nw75hsdwRmqsWsk6Lhjw9/28EGwYnOHQO5iXGAlYsgfLjVxY9Ey8kej0Isw74EBhAcUVVyLiqDCr1kFdgOdTWXbAN2BjhH6KaSuhmNOzE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729272675; c=relaxed/simple; bh=OtgBNzwT10q1fgJ73my79dtV2veuEsi813cYK1MYapI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Cych6q8TS/0sf1gMAs3Vt7bdb6RaY6NZ4AI1xiPU/niVxK7I3NTAsBvMv49KWB2CVJhNHCwbs8+mc7kndcDewrdjsMgPMKcTFfvsY3cSBbqHtP48INMCU359gsalWHp1dkpGDxpXWZiorhHvJFKg9VJ4alttuxS0krCiKCwTtg8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=BdCBp7la; arc=none smtp.client-ip=209.85.215.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="BdCBp7la" Received: by mail-pg1-f176.google.com with SMTP id 41be03b00d2f7-7ea0ff74b15so1592197a12.3 for ; Fri, 18 Oct 2024 10:31:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729272672; x=1729877472; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=maqsHecWoRrqy0atrv0EioRwOZB99TyG0W2HP4lfnAU=; b=BdCBp7laJl9ycTaWhrrR/hUZfz/XPh/fUh4VqhkuSUQzB23NhxIyA60nhPspxtjydk ZVWBZLtMgYO7i8EMT6EHcFUP1Z7NVvP8a/esxdpZ895IIXfn2+K5M1JNIMhCfK8znAfq g8h4hTy0DNSYEbNTK1YgLtd0psgcV5WljTnj25dlEhEmcH+DRDghDqt0rC25aJZq/D5G 6uc9iJbjAC7ctHKKwgjVQbYE1kjS2hpS2CbTD/HpDvUuP0y8CGo/JeJWt0H4WQi7BgW8 jCOAf4sWxXK8DMA04dLdAh3Q+JmkzkoUhsLyAeqRd8WcTNp/lxHplMiELo90ygIIKBw9 o7hg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729272672; x=1729877472; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=maqsHecWoRrqy0atrv0EioRwOZB99TyG0W2HP4lfnAU=; b=Nj2IXXX0ew9TE5fWvwZgqpD5fe6WilNQB009dU2sBXWfv3m0niEqhzWkeouy380yRM yFDXvKlQ8p2FTIJL7pzX0AxrDORYtPlaBQPy0Z7eAzIDlnH14DpVMXaL5tKLJvuep99N 6PvkQ4DSrhr4Se9fVChImjgRo5jjVP6Oj5EoQs6H4LwKfy6IxC63yVU5ecrGR+O8a8Cw 7KWMaEwwU+3YcWByHZVLafaL3wee792zEMxrarWNNH4stlk0ZdDAqI3iCh4hMxlbjPes 5Okt2cwCw2od8btkSmHP245FE6oEOtNidAND/9Wuh0jRHGrd7q2QtOwJnKKkSzEomgUm cjKg== X-Forwarded-Encrypted: i=1; AJvYcCW5hoxL4QPYekgGzobr+75ZejmcxO+TK7/W0wC5AjuP6HuhGepKImBm1LT15O+dJncuUVC9odTL6abp9Oc=@vger.kernel.org X-Gm-Message-State: AOJu0YxQ+Aa4N+ntQmP29bkvUXW3NkQEPySaxRgJc+vdMglKJSqm4UwB ZGyH4/GEUqJ8AK0Uxy+gtI1t+ifuggzPUNfDhfziAis4E5xSfUST X-Google-Smtp-Source: AGHT+IGCcfcvV3hhkWCRVZ4L4+07qgofxPBOFcgBhHnT233XbATfjI7cFHkLMcuk8gJsatcgaCUiZQ== X-Received: by 2002:a05:6a20:43a0:b0:1d9:a90:8879 with SMTP id adf61e73a8af0-1d92c502994mr4770966637.21.1729272672394; Fri, 18 Oct 2024 10:31:12 -0700 (PDT) Received: from dw-tp.ibmuc.com ([171.76.80.151]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71ea3311f51sm1725242b3a.36.2024.10.18.10.31.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Oct 2024 10:31:11 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linuxppc-dev@lists.ozlabs.org Cc: kasan-dev@googlegroups.com, linux-mm@kvack.org, Marco Elver , Alexander Potapenko , Heiko Carstens , Michael Ellerman , Nicholas Piggin , Madhavan Srinivasan , Christophe Leroy , Hari Bathini , "Aneesh Kumar K . V" , Donet Tom , Pavithra Prakash , LKML , "Ritesh Harjani (IBM)" Subject: [PATCH v3 10/12] book3s64/radix: Refactoring common kfence related functions Date: Fri, 18 Oct 2024 22:59:51 +0530 Message-ID: X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Both radix and hash on book3s requires to detect if kfence early init is enabled or not. Hash needs to disable kfence if early init is not enabled because with kfence the linear map is mapped using PAGE_SIZE rather than 16M mapping. We don't support multiple page sizes for slb entry used for kernel linear map in book3s64. This patch refactors out the common functions required to detect kfence early init is enabled or not. Signed-off-by: Ritesh Harjani (IBM) --- arch/powerpc/include/asm/kfence.h | 8 ++++++-- arch/powerpc/mm/book3s64/pgtable.c | 13 +++++++++++++ arch/powerpc/mm/book3s64/radix_pgtable.c | 12 ------------ arch/powerpc/mm/init-common.c | 1 + 4 files changed, 20 insertions(+), 14 deletions(-) diff --git a/arch/powerpc/include/asm/kfence.h b/arch/powerpc/include/asm/k= fence.h index fab124ada1c7..1f7cab58ab2c 100644 --- a/arch/powerpc/include/asm/kfence.h +++ b/arch/powerpc/include/asm/kfence.h @@ -15,7 +15,7 @@ #define ARCH_FUNC_PREFIX "." #endif =20 -#ifdef CONFIG_KFENCE +extern bool kfence_early_init; extern bool kfence_disabled; =20 static inline void disable_kfence(void) @@ -27,7 +27,11 @@ static inline bool arch_kfence_init_pool(void) { return !kfence_disabled; } -#endif + +static inline bool kfence_early_init_enabled(void) +{ + return IS_ENABLED(CONFIG_KFENCE) && kfence_early_init; +} =20 #ifdef CONFIG_PPC64 static inline bool kfence_protect_page(unsigned long addr, bool protect) diff --git a/arch/powerpc/mm/book3s64/pgtable.c b/arch/powerpc/mm/book3s64/= pgtable.c index f4d8d3c40e5c..1563a8c28feb 100644 --- a/arch/powerpc/mm/book3s64/pgtable.c +++ b/arch/powerpc/mm/book3s64/pgtable.c @@ -37,6 +37,19 @@ EXPORT_SYMBOL(__pmd_frag_nr); unsigned long __pmd_frag_size_shift; EXPORT_SYMBOL(__pmd_frag_size_shift); =20 +#ifdef CONFIG_KFENCE +extern bool kfence_early_init; +static int __init parse_kfence_early_init(char *arg) +{ + int val; + + if (get_option(&arg, &val)) + kfence_early_init =3D !!val; + return 0; +} +early_param("kfence.sample_interval", parse_kfence_early_init); +#endif + #ifdef CONFIG_TRANSPARENT_HUGEPAGE /* * This is called when relaxing access to a hugepage. It's also called in = the page diff --git a/arch/powerpc/mm/book3s64/radix_pgtable.c b/arch/powerpc/mm/boo= k3s64/radix_pgtable.c index b0d927009af8..311e2112d782 100644 --- a/arch/powerpc/mm/book3s64/radix_pgtable.c +++ b/arch/powerpc/mm/book3s64/radix_pgtable.c @@ -363,18 +363,6 @@ static int __meminit create_physical_mapping(unsigned = long start, } =20 #ifdef CONFIG_KFENCE -static bool __ro_after_init kfence_early_init =3D !!CONFIG_KFENCE_SAMPLE_I= NTERVAL; - -static int __init parse_kfence_early_init(char *arg) -{ - int val; - - if (get_option(&arg, &val)) - kfence_early_init =3D !!val; - return 0; -} -early_param("kfence.sample_interval", parse_kfence_early_init); - static inline phys_addr_t alloc_kfence_pool(void) { phys_addr_t kfence_pool; diff --git a/arch/powerpc/mm/init-common.c b/arch/powerpc/mm/init-common.c index 2978fcbe307e..745097554bea 100644 --- a/arch/powerpc/mm/init-common.c +++ b/arch/powerpc/mm/init-common.c @@ -33,6 +33,7 @@ bool disable_kuep =3D !IS_ENABLED(CONFIG_PPC_KUEP); bool disable_kuap =3D !IS_ENABLED(CONFIG_PPC_KUAP); #ifdef CONFIG_KFENCE bool __ro_after_init kfence_disabled; +bool __ro_after_init kfence_early_init =3D !!CONFIG_KFENCE_SAMPLE_INTERVAL; #endif =20 static int __init parse_nosmep(char *p) --=20 2.46.0 From nobody Tue Nov 26 08:24:58 2024 Received: from mail-pg1-f176.google.com (mail-pg1-f176.google.com [209.85.215.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CFF9F205E06 for ; Fri, 18 Oct 2024 17:31:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729272679; cv=none; b=lNwUeyZQT9K03tgfhju4YBAAzHo2Cij38TkDX1MRQpMiOZ51Eo9atqmgHIa5FThpZxoxwnrqtLE3oZk0UBVjvl0PljIsXi8luMhFhYXhcuFA4cMgIyw0v2pCOdHbwyYryo5tW2PMg2R8HJp0eD3BLDsGI26aBJhgKZqvcPu5lSc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729272679; c=relaxed/simple; bh=dfXVagvKCrtVIK4lQC9jHDb/cboUbJl9srs8w+qOFtg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MBk6xmf/CsFizCAYVwz+IEPy+6Wbu5QeGBL4/xt+d/wlCHdo6v68LiPVL4sFopSaLW7Z71c6esyTDSKVl+YKoZM8DbBUUKCjDjPsPp2pIYo5MsKJGQ0CT4o8hsnCGC4XSoSEseJHbfRi2QD8kuO3rwy/LxhePf2RY+nqoNvsE20= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=N3MCwPtK; arc=none smtp.client-ip=209.85.215.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="N3MCwPtK" Received: by mail-pg1-f176.google.com with SMTP id 41be03b00d2f7-7db908c9c83so1573523a12.2 for ; Fri, 18 Oct 2024 10:31:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729272677; x=1729877477; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=M+pQKseSbUwQ7nDgMGbwYxJsHyJez2ZBxycyN9iER7I=; b=N3MCwPtKI+88BAI0jeG6mPQUyvz/qGEHP7FKzjIPuwtsn91S8IHAYIBCdda3smj5xA 81o8bS6++qYOP30cfoVicxQTrOhl+Xnphc5uZb63u3/5IA9zFeG9K/Vy3F91RI6e+LTF uN2M20QsWc0/R4mJgunLGE3bk1ppKPFGLgZoUb+VxCtrRkCZHYr68RycfyogiRjl5aNI w5Nr9z2NkVAepoXrgjdDRo3j2gYRwJHUyTq4YW7y9rJMdMp7m5iA6ANhqOEp4LPr3HII 3STAh3X+t/SXsPoPRiry4XZDlo77GEdex6R2qln5uMorjED01HvooAJlcqdhOW2o/uCX RgTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729272677; x=1729877477; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=M+pQKseSbUwQ7nDgMGbwYxJsHyJez2ZBxycyN9iER7I=; b=gejKZGgvJccDUnFZWFkARByHuQndJPCNK6loDsDZ9pQtBuOdjsuF3DkMAqrP3hkQg5 6wA8T8lHX6z3uNC+kbxJCxbkUsuJJ03fzKtgm+TP2LXX3hWIXeefPsk3ryFvSpAW5gml BaWmV/fVif6erRz8sQ+3r5obxtWlTPVJ7VC08jfEdMUBHWtweu8e76fF5itTvOB5l/iU 5tLGWb+pLd0/YSmwvc1RKU4EjZITY9JRDboE7cIk/kQyaS9/Zi8AVKdOjb7S4VXY3bHo LQnmsl9afL6++e25gALGNz3vcnmRx8rz2WMR47S7rp4LOP4NrOouyhcyeEqRQ54/APx7 oc6Q== X-Forwarded-Encrypted: i=1; AJvYcCUXN8dD/MEqAQBrtlE2GaY5MfofkM1+NXFg+6nSwE6qepczG+ZXwR4rO32ogKecLUFVtSJSoNLh1s84L4k=@vger.kernel.org X-Gm-Message-State: AOJu0YxFao/XWw2t1rjeJDmyyOJckwt1WZkXQtIy7Dtoytm7Mxo8zRLQ 5BhjGc/hPjK4gEhXe5o3z8KV0Ap1+EQ0L7TQfd+Dk8F5kyu4kkhG X-Google-Smtp-Source: AGHT+IFVWe1xUi/sLyBKrxUXKe/50HIPEi+N9laTmy/mjDBIPEDqbRrd1jNCsq8LdLHdy/KQ/lg9Ig== X-Received: by 2002:a05:6a21:168d:b0:1d9:61b:9607 with SMTP id adf61e73a8af0-1d92c4a0231mr4271987637.6.1729272677039; Fri, 18 Oct 2024 10:31:17 -0700 (PDT) Received: from dw-tp.ibmuc.com ([171.76.80.151]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71ea3311f51sm1725242b3a.36.2024.10.18.10.31.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Oct 2024 10:31:16 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linuxppc-dev@lists.ozlabs.org Cc: kasan-dev@googlegroups.com, linux-mm@kvack.org, Marco Elver , Alexander Potapenko , Heiko Carstens , Michael Ellerman , Nicholas Piggin , Madhavan Srinivasan , Christophe Leroy , Hari Bathini , "Aneesh Kumar K . V" , Donet Tom , Pavithra Prakash , LKML , "Ritesh Harjani (IBM)" Subject: [PATCH v3 11/12] book3s64/hash: Disable kfence if not early init Date: Fri, 18 Oct 2024 22:59:52 +0530 Message-ID: <4a6eea8cfd1cd28fccfae067026bff30cbec1d4b.1729271995.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Enable kfence on book3s64 hash only when early init is enabled. This is because, kfence could cause the kernel linear map to be mapped at PAGE_SIZE level instead of 16M (which I guess we don't want). Also currently there is no way to - 1. Make multiple page size entries for the SLB used for kernel linear map. 2. No easy way of getting the hash slot details after the page table mapping for kernel linear setup. So even if kfence allocate the pool in late init, we won't be able to get the hash slot details in kfence linear map. Thus this patch disables kfence on hash if kfence early init is not enabled. Signed-off-by: Ritesh Harjani (IBM) --- arch/powerpc/mm/book3s64/hash_utils.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/mm/book3s64/hash_utils.c b/arch/powerpc/mm/book3s= 64/hash_utils.c index 558d6f5202b9..2f5dd6310a8f 100644 --- a/arch/powerpc/mm/book3s64/hash_utils.c +++ b/arch/powerpc/mm/book3s64/hash_utils.c @@ -410,6 +410,8 @@ static phys_addr_t kfence_pool; =20 static inline void hash_kfence_alloc_pool(void) { + if (!kfence_early_init_enabled()) + goto err; =20 /* allocate linear map for kfence within RMA region */ linear_map_kf_hash_count =3D KFENCE_POOL_SIZE >> PAGE_SHIFT; @@ -1074,7 +1076,7 @@ static void __init htab_init_page_sizes(void) bool aligned =3D true; init_hpte_page_sizes(); =20 - if (!debug_pagealloc_enabled_or_kfence()) { + if (!debug_pagealloc_enabled() && !kfence_early_init_enabled()) { /* * Pick a size for the linear mapping. Currently, we only * support 16M, 1M and 4K which is the default --=20 2.46.0 From nobody Tue Nov 26 08:24:58 2024 Received: from mail-pf1-f170.google.com (mail-pf1-f170.google.com [209.85.210.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D978120694D for ; Fri, 18 Oct 2024 17:31:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729272684; cv=none; b=Ad5+gtqLi0tzeFCHhUcXC4RaIrfPRBYbpjklnX69uloLAuAbrDMvyZPJRw19cDml2nkeH0H5tXmqS0fiCBDRga8r6r4azfp8JuFUeHJ7MpV1g9f2mv6MXc8YPqBNQe2dMxzQCZpTGh8lpMPnrgNLbMYl36JldUnzuoKvvT2sTdg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1729272684; c=relaxed/simple; bh=91fYtR+ANu7O/Y5U9VPeZVNe/1K1yAoWvPLfU0bA2lE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jUESKLro39jny0xLiBONZsajT4A1B2LjGsQZUcMAvmKXU66rxO8bUSfjmuWwqHmebIoGUvSm13R0XCmHW80XKCUEkE90Rmq+C4k2DXOxZ5NtpjFu1gxcbpwNHk5tL4Yp6n+zXhBwjyOQ7KdAQRTA8g9HNKRoIQvwpPJw81qxxzk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Owap3dec; arc=none smtp.client-ip=209.85.210.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Owap3dec" Received: by mail-pf1-f170.google.com with SMTP id d2e1a72fcca58-71e93d551a3so1607556b3a.1 for ; Fri, 18 Oct 2024 10:31:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1729272682; x=1729877482; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1QLaMyzhWv0h56bpJNLe2thoYBckgfAdM8HvqGdr41M=; b=Owap3decxqfrICg1DHETviJCxqOstxFhcH4Lrb06+YTMul4uoJT02FK5zdI0CjR4/J hdkeV+JUQsDcse89h6+mvsK4y652x1cMREraYDxwdWF3lzXX0SL9lI/bnxZpMmQi8UW8 BYjVdfm9/s7UlsuGqUdC3W3ZOr8rIKg9wlrHJe8e+C0ul03ltaC5R/wzWMa0xNHOxVIl SLDO1dgeDyEZYr/1ObB6vfc6rXGrV7PHiFpl8YkZ5RUFkR42rhOpjdTNs+jnVO2M1UYM AJ6IJV9u/+xbtHw7jkyh/Pd8h7ktZFc6ksO7jzfD62HoRiLZwe+lqXIw4SJZl9FRX9f/ lHJw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1729272682; x=1729877482; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1QLaMyzhWv0h56bpJNLe2thoYBckgfAdM8HvqGdr41M=; b=dHosBWQfd3pIlFkpb4EaLHuKCl82uT1Gs+u+45i2p0GqNsf+dFm8phuOaECgbk69Pa FwaLo/GiytEEjADU4HUNe3+pruqGEiBd9Z9XWLhJqiEUFBUuVTd9Co1eL07QHPJyB9NO ZRyyPcvz4rNtyLxEHjWfk+AWvZ1F2xr9qfAapDu5iGuu+Gt2g+xklAl4wIRwp4HcFka/ VfGlvmWgAFzoFmqQVpwPDsT/cKfTh5vnMcNWYM4JNrNU+dTKRSEHNpekigCOjVr1FVlW 7TOnY/iXpIludi0yXnpVMFfwnGdJvxdb8FFmYHU4L7nQWGZIkZeG8tAhS7ezhYwrDb6f Gcyg== X-Forwarded-Encrypted: i=1; AJvYcCUSjozEv1B8tFG4yTHoTLTg1UYw5pHXdWBTiwHJ+JclVrcjzjBDGWyHwLvhUDU7NEKEPduueT4cwVk8mFQ=@vger.kernel.org X-Gm-Message-State: AOJu0Yw48wwhG6EKhavPIzy+LqvC/JKxJVRa2tVj836W4tmgNrmDI9Gv dL1aZvArZziUuVhOFBoXH4ntu2RCcCcepARavJd2K2AxrOq50xdI X-Google-Smtp-Source: AGHT+IFTSMl8QW5ycN4XEBU+MJ3K9NfWvdEojOEp7QlJ+1fDl8flZ96NMrdY+IbWH9uWHYWgBiWu4A== X-Received: by 2002:a05:6a00:cd4:b0:71e:cf8:d6f1 with SMTP id d2e1a72fcca58-71ea316bf4emr3925702b3a.14.1729272682003; Fri, 18 Oct 2024 10:31:22 -0700 (PDT) Received: from dw-tp.ibmuc.com ([171.76.80.151]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-71ea3311f51sm1725242b3a.36.2024.10.18.10.31.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 18 Oct 2024 10:31:21 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linuxppc-dev@lists.ozlabs.org Cc: kasan-dev@googlegroups.com, linux-mm@kvack.org, Marco Elver , Alexander Potapenko , Heiko Carstens , Michael Ellerman , Nicholas Piggin , Madhavan Srinivasan , Christophe Leroy , Hari Bathini , "Aneesh Kumar K . V" , Donet Tom , Pavithra Prakash , LKML , "Ritesh Harjani (IBM)" Subject: [PATCH v3 12/12] book3s64/hash: Early detect debug_pagealloc size requirement Date: Fri, 18 Oct 2024 22:59:53 +0530 Message-ID: X-Mailer: git-send-email 2.46.0 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Add hash_supports_debug_pagealloc() helper to detect whether debug_pagealloc can be supported on hash or not. This checks for both, whether debug_pagealloc config is enabled and the linear map should fit within rma_size/4 region size. This can then be used early during htab_init_page_sizes() to decide linear map pagesize if hash supports either debug_pagealloc or kfence. Signed-off-by: Ritesh Harjani (IBM) --- arch/powerpc/mm/book3s64/hash_utils.c | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/arch/powerpc/mm/book3s64/hash_utils.c b/arch/powerpc/mm/book3s= 64/hash_utils.c index 2f5dd6310a8f..2674f763f5db 100644 --- a/arch/powerpc/mm/book3s64/hash_utils.c +++ b/arch/powerpc/mm/book3s64/hash_utils.c @@ -329,25 +329,26 @@ static void kernel_unmap_linear_page(unsigned long va= ddr, unsigned long idx, } #endif =20 +static inline bool hash_supports_debug_pagealloc(void) +{ + unsigned long max_hash_count =3D ppc64_rma_size / 4; + unsigned long linear_map_count =3D memblock_end_of_DRAM() >> PAGE_SHIFT; + + if (!debug_pagealloc_enabled() || linear_map_count > max_hash_count) + return false; + return true; +} + #ifdef CONFIG_DEBUG_PAGEALLOC static u8 *linear_map_hash_slots; static unsigned long linear_map_hash_count; static DEFINE_RAW_SPINLOCK(linear_map_hash_lock); static void hash_debug_pagealloc_alloc_slots(void) { - unsigned long max_hash_count =3D ppc64_rma_size / 4; - - if (!debug_pagealloc_enabled()) - return; - linear_map_hash_count =3D memblock_end_of_DRAM() >> PAGE_SHIFT; - if (unlikely(linear_map_hash_count > max_hash_count)) { - pr_info("linear map size (%llu) greater than 4 times RMA region (%llu). = Disabling debug_pagealloc\n", - ((u64)linear_map_hash_count << PAGE_SHIFT), - ppc64_rma_size); - linear_map_hash_count =3D 0; + if (!hash_supports_debug_pagealloc()) return; - } =20 + linear_map_hash_count =3D memblock_end_of_DRAM() >> PAGE_SHIFT; linear_map_hash_slots =3D memblock_alloc_try_nid( linear_map_hash_count, 1, MEMBLOCK_LOW_LIMIT, ppc64_rma_size, NUMA_NO_NODE); @@ -1076,7 +1077,7 @@ static void __init htab_init_page_sizes(void) bool aligned =3D true; init_hpte_page_sizes(); =20 - if (!debug_pagealloc_enabled() && !kfence_early_init_enabled()) { + if (!hash_supports_debug_pagealloc() && !kfence_early_init_enabled()) { /* * Pick a size for the linear mapping. Currently, we only * support 16M, 1M and 4K which is the default --=20 2.46.0