From nobody Sun Oct 5 05:29:00 2025 Received: from mail-lf1-f53.google.com (mail-lf1-f53.google.com [209.85.167.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E8A362405E1; Sun, 10 Aug 2025 12:58:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754830690; cv=none; b=mZrMcLPwibFYnwW/Id0qCgTBqytkNC5KEauJ8XJI7PgFxU0Cu7mWxawu4eQK2KyoyIX6UU0EYPCIEdCYoW6+GcrHTCALska7u6jN7Gcn0HdL6WtxU22iP9u3z6a0FiZoQFGGRGEauwIq9OPSG4kFrZ1s+3Xvs/Xw2UfZa+ZBLLs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754830690; c=relaxed/simple; bh=ZOE7uoE4a1soLzGxbvmVG2dcRCVD/i2iAU6H4uPhAF8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Wq/FK8Ihsd87LH+QAZqSAo8siwjInClkHct5RIU0QMH4Achosz6zhwnnwGD4O7Vk4JJ0Wul8vML7uFdRBqA6DIPZCb8f8dIN9r6Wz5FdtSAGtb7+ueYAxeKwFIjCoUlJbTdRsFU9tJn+aeg2xqDg+AXqGKBoB3hyDrRWHSQLcCY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=eGgrxA83; arc=none smtp.client-ip=209.85.167.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="eGgrxA83" Received: by mail-lf1-f53.google.com with SMTP id 2adb3069b0e04-55b8bf653dfso3857739e87.0; Sun, 10 Aug 2025 05:58:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1754830686; x=1755435486; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jKeR5uHoWIcIuAKYQgjQzhpWKD2jCeMi5KWikEai+a4=; b=eGgrxA83ym9Y+0kHi5k5uaUWeYzlHtO7U7F0ucEpIQo/M8aCxz6DS+H+fbyslqiFpL dcPUOMM6ojBW+SBaBylCjvGJoMklhVeWaXm9ihRlKQQSX6BK9OhvtjV7BB27RSTxAH4Q PpvHtXCprqbfKwg6sqq1P96Ctb8UMsi8S42qdQghl6UeaJyfLhyMT/hdTDbuun//ukgf 9RQ0PYe8H1OTxW3/2gaf87XHvFHeDPwg7ahA8ynb65teU8hUxP6L3MNdpXk0Y4t2BplY uqiVmV67PAvnGBavcreFuhc//s6vE0Va9wngklmUABvmtZ3/lP/mpefRQoQ0XIpCM7GQ SImg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754830686; x=1755435486; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jKeR5uHoWIcIuAKYQgjQzhpWKD2jCeMi5KWikEai+a4=; b=IZmHzHXwZfBEtbyWqJRLnPtydhVMsA9W19/Z8EGMyk9S0YpbF8p3liTuSQ+K0U1Oo1 9NS183o+7rPf+MLylUmVNt3Q8vvW8Ei3r+2mzW0kxt0sSwCLqOPyezG6qMNUP/IDVBg8 XTIB+FRDSz6PkIrs2LgOe88eJbnFahcp2XremXdxRfpjGva9hALV+jw5/lNI6x1SsPGC kVXaVyYXCbq+IAiMQhhoNHDi8Dy6qo/Zz55LzyIXXZ6x7I621ecMrEElKyMpEADGeumU XcT0kQZe/euBBT800vLwXI5Q/ibOSaYMOahh6TCtUxyQraSm/SrGrzN+rXDMgS4DXNWc EMkg== X-Forwarded-Encrypted: i=1; AJvYcCWQcbpEuG1vg19Z38NP96t6/uLp9qFcFDWWhSEJcksiNQ5UWuJpzwl4Tf6jTEG5+Isr1MoI6jDnvmfn5DU=@vger.kernel.org, AJvYcCX+bDBWrVPoFwAxfGO42GLs9L6GwLOueQPeOr7Mn+EKXdbMMavX1pBuI8+oXX4jhtcMLGbF6jPQgdu9Vw==@vger.kernel.org X-Gm-Message-State: AOJu0YwWsKLjxUEignyRrvfZXfBniynwfrYOKLfuTZ6qK0/CzmTXIKnt Bzdv1Qd697vQM5jwgQ5rQTCvsEqUDCJSw8wG63NkzORAutHhHCEe0C5M X-Gm-Gg: ASbGncsBDB+nUvfWtGciQQxiUMQngop7nxfFPMz4lNJek35aeDbk8FBJQ+sp8pqKQDR l4lWvU0vHYe8akRcDB+M9NkghVAZgrauUxRPZ/7lhcTSkOvMRc3ujaJ2WEvm1sK1KRjv3LoTwtg gTc8qTtAO1tXxypEHA8ilqe38vv/HZfy8zZ73aEIEZvID0GH1+rfoA3/Pd7IxSNCBQp4YMhzYZU YGIGFjtsCX79mn6dw7kso5CiAsRS9TLNBY72l07IAtLFmJ0yP8CrBVfVbwMFVKfysmJzvCLqVa7 GgP3Csjh0zX7Y2vzr7H2PoLnJKfhBa2UlLC9EKPehfD+wRN1Ec5TrOR2cb7yxhsZb1ubHL0hAuC NsFASUx7xP5zujtSpcI3C1h1bM0zgx794TO4oBg== X-Google-Smtp-Source: AGHT+IErXkbMxszWeWK4iBKntLkDp5w8ZvgbUHZU1lvQV/xiB2iEZXUIzYb1B+mD0a2Z3z67sFMnHA== X-Received: by 2002:a05:6512:33c9:b0:55b:95a2:d70f with SMTP id 2adb3069b0e04-55cc00ec563mr3184446e87.18.1754830685714; Sun, 10 Aug 2025 05:58:05 -0700 (PDT) Received: from localhost.localdomain ([2a03:32c0:2e:37dd:bfc4:9fdc:ddc6:5962]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-55b88c9908esm3804561e87.76.2025.08.10.05.57.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 10 Aug 2025 05:58:05 -0700 (PDT) From: Sabyrzhan Tasbolatov To: ryabinin.a.a@gmail.com, christophe.leroy@csgroup.eu, bhe@redhat.com, hca@linux.ibm.com, andreyknvl@gmail.com, akpm@linux-foundation.org, zhangqing@loongson.cn, chenhuacai@loongson.cn, davidgow@google.com, glider@google.com, dvyukov@google.com, alexghiti@rivosinc.com Cc: alex@ghiti.fr, agordeev@linux.ibm.com, vincenzo.frascino@arm.com, elver@google.com, kasan-dev@googlegroups.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, loongarch@lists.linux.dev, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-um@lists.infradead.org, linux-mm@kvack.org, snovitoll@gmail.com Subject: [PATCH v6 1/2] kasan: introduce ARCH_DEFER_KASAN and unify static key across modes Date: Sun, 10 Aug 2025 17:57:45 +0500 Message-Id: <20250810125746.1105476-2-snovitoll@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250810125746.1105476-1-snovitoll@gmail.com> References: <20250810125746.1105476-1-snovitoll@gmail.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Introduce CONFIG_ARCH_DEFER_KASAN to identify architectures [1] that need to defer KASAN initialization until shadow memory is properly set up, and unify the static key infrastructure across all KASAN modes. [1] PowerPC, UML, LoongArch selects ARCH_DEFER_KASAN. The core issue is that different architectures haveinconsistent approaches to KASAN readiness tracking: - PowerPC, LoongArch, and UML arch, each implement own kasan_arch_is_ready() - Only HW_TAGS mode had a unified static key (kasan_flag_enabled) - Generic and SW_TAGS modes relied on arch-specific solutions or always-on behavior This patch addresses the fragmentation in KASAN initialization across architectures by introducing a unified approach that eliminates duplicate static keys and arch-specific kasan_arch_is_ready() implementations. Let's replace kasan_arch_is_ready() with existing kasan_enabled() check, which examines the static key being enabled if arch selects ARCH_DEFER_KASAN or has HW_TAGS mode support. For other arch, kasan_enabled() checks the enablement during compile time. Now KASAN users can use a single kasan_enabled() check everywhere. Closes: https://bugzilla.kernel.org/show_bug.cgi?id=3D217049 Signed-off-by: Sabyrzhan Tasbolatov Reviewed-by: Christophe Leroy Reviewed-by: Ritesh Harjani (IBM) #powerpc --- Changes in v6: - Added more details in git commit message - Fixed commenting format per coding style in UML (Christophe Leroy) - Changed exporting to GPL for kasan_flag_enabled (Christophe Leroy) - Converted ARCH_DEFER_KASAN to def_bool depending on KASAN to avoid arch users to have `if KASAN` condition (Christophe Leroy) - Forgot to add __init for kasan_init in UML Changes in v5: - Unified patches where arch (powerpc, UML, loongarch) selects ARCH_DEFER_KASAN in the first patch not to break bisectability - Removed kasan_arch_is_ready completely as there is no user - Removed __wrappers in v4, left only those where it's necessary due to different implementations Changes in v4: - Fixed HW_TAGS static key functionality (was broken in v3) - Merged configuration and implementation for atomicity --- arch/loongarch/Kconfig | 1 + arch/loongarch/include/asm/kasan.h | 7 ------ arch/loongarch/mm/kasan_init.c | 8 +++---- arch/powerpc/Kconfig | 1 + arch/powerpc/include/asm/kasan.h | 12 ---------- arch/powerpc/mm/kasan/init_32.c | 2 +- arch/powerpc/mm/kasan/init_book3e_64.c | 2 +- arch/powerpc/mm/kasan/init_book3s_64.c | 6 +---- arch/um/Kconfig | 1 + arch/um/include/asm/kasan.h | 5 ++-- arch/um/kernel/mem.c | 13 ++++++++--- include/linux/kasan-enabled.h | 32 ++++++++++++++++++-------- include/linux/kasan.h | 6 +++++ lib/Kconfig.kasan | 12 ++++++++++ mm/kasan/common.c | 17 ++++++++++---- mm/kasan/generic.c | 19 +++++++++++---- mm/kasan/hw_tags.c | 9 +------- mm/kasan/kasan.h | 8 ++++++- mm/kasan/shadow.c | 12 +++++----- mm/kasan/sw_tags.c | 1 + mm/kasan/tags.c | 2 +- 21 files changed, 106 insertions(+), 70 deletions(-) diff --git a/arch/loongarch/Kconfig b/arch/loongarch/Kconfig index f0abc38c40ac..e449e3fcecf9 100644 --- a/arch/loongarch/Kconfig +++ b/arch/loongarch/Kconfig @@ -9,6 +9,7 @@ config LOONGARCH select ACPI_PPTT if ACPI select ACPI_SYSTEM_POWER_STATES_SUPPORT if ACPI select ARCH_BINFMT_ELF_STATE + select ARCH_NEEDS_DEFER_KASAN select ARCH_DISABLE_KASAN_INLINE select ARCH_ENABLE_MEMORY_HOTPLUG select ARCH_ENABLE_MEMORY_HOTREMOVE diff --git a/arch/loongarch/include/asm/kasan.h b/arch/loongarch/include/as= m/kasan.h index 62f139a9c87d..0e50e5b5e056 100644 --- a/arch/loongarch/include/asm/kasan.h +++ b/arch/loongarch/include/asm/kasan.h @@ -66,7 +66,6 @@ #define XKPRANGE_WC_SHADOW_OFFSET (KASAN_SHADOW_START + XKPRANGE_WC_KASAN_= OFFSET) #define XKVRANGE_VC_SHADOW_OFFSET (KASAN_SHADOW_START + XKVRANGE_VC_KASAN_= OFFSET) =20 -extern bool kasan_early_stage; extern unsigned char kasan_early_shadow_page[PAGE_SIZE]; =20 #define kasan_mem_to_shadow kasan_mem_to_shadow @@ -75,12 +74,6 @@ void *kasan_mem_to_shadow(const void *addr); #define kasan_shadow_to_mem kasan_shadow_to_mem const void *kasan_shadow_to_mem(const void *shadow_addr); =20 -#define kasan_arch_is_ready kasan_arch_is_ready -static __always_inline bool kasan_arch_is_ready(void) -{ - return !kasan_early_stage; -} - #define addr_has_metadata addr_has_metadata static __always_inline bool addr_has_metadata(const void *addr) { diff --git a/arch/loongarch/mm/kasan_init.c b/arch/loongarch/mm/kasan_init.c index d2681272d8f0..170da98ad4f5 100644 --- a/arch/loongarch/mm/kasan_init.c +++ b/arch/loongarch/mm/kasan_init.c @@ -40,11 +40,9 @@ static pgd_t kasan_pg_dir[PTRS_PER_PGD] __initdata __ali= gned(PAGE_SIZE); #define __pte_none(early, pte) (early ? pte_none(pte) : \ ((pte_val(pte) & _PFN_MASK) =3D=3D (unsigned long)__pa(kasan_early_shadow_= page))) =20 -bool kasan_early_stage =3D true; - void *kasan_mem_to_shadow(const void *addr) { - if (!kasan_arch_is_ready()) { + if (!kasan_enabled()) { return (void *)(kasan_early_shadow_page); } else { unsigned long maddr =3D (unsigned long)addr; @@ -298,7 +296,8 @@ void __init kasan_init(void) kasan_populate_early_shadow(kasan_mem_to_shadow((void *)VMALLOC_START), kasan_mem_to_shadow((void *)KFENCE_AREA_END)); =20 - kasan_early_stage =3D false; + /* Enable KASAN here before kasan_mem_to_shadow(). */ + kasan_init_generic(); =20 /* Populate the linear mapping */ for_each_mem_range(i, &pa_start, &pa_end) { @@ -329,5 +328,4 @@ void __init kasan_init(void) =20 /* At this point kasan is fully initialized. Enable error messages */ init_task.kasan_depth =3D 0; - pr_info("KernelAddressSanitizer initialized.\n"); } diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index 93402a1d9c9f..4730c676b6bf 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -122,6 +122,7 @@ config PPC # Please keep this list sorted alphabetically. # select ARCH_32BIT_OFF_T if PPC32 + select ARCH_NEEDS_DEFER_KASAN if PPC_RADIX_MMU select ARCH_DISABLE_KASAN_INLINE if PPC_RADIX_MMU select ARCH_DMA_DEFAULT_COHERENT if !NOT_COHERENT_CACHE select ARCH_ENABLE_MEMORY_HOTPLUG diff --git a/arch/powerpc/include/asm/kasan.h b/arch/powerpc/include/asm/ka= san.h index b5bbb94c51f6..957a57c1db58 100644 --- a/arch/powerpc/include/asm/kasan.h +++ b/arch/powerpc/include/asm/kasan.h @@ -53,18 +53,6 @@ #endif =20 #ifdef CONFIG_KASAN -#ifdef CONFIG_PPC_BOOK3S_64 -DECLARE_STATIC_KEY_FALSE(powerpc_kasan_enabled_key); - -static __always_inline bool kasan_arch_is_ready(void) -{ - if (static_branch_likely(&powerpc_kasan_enabled_key)) - return true; - return false; -} - -#define kasan_arch_is_ready kasan_arch_is_ready -#endif =20 void kasan_early_init(void); void kasan_mmu_init(void); diff --git a/arch/powerpc/mm/kasan/init_32.c b/arch/powerpc/mm/kasan/init_3= 2.c index 03666d790a53..1d083597464f 100644 --- a/arch/powerpc/mm/kasan/init_32.c +++ b/arch/powerpc/mm/kasan/init_32.c @@ -165,7 +165,7 @@ void __init kasan_init(void) =20 /* At this point kasan is fully initialized. Enable error messages */ init_task.kasan_depth =3D 0; - pr_info("KASAN init done\n"); + kasan_init_generic(); } =20 void __init kasan_late_init(void) diff --git a/arch/powerpc/mm/kasan/init_book3e_64.c b/arch/powerpc/mm/kasan= /init_book3e_64.c index 60c78aac0f63..0d3a73d6d4b0 100644 --- a/arch/powerpc/mm/kasan/init_book3e_64.c +++ b/arch/powerpc/mm/kasan/init_book3e_64.c @@ -127,7 +127,7 @@ void __init kasan_init(void) =20 /* Enable error messages */ init_task.kasan_depth =3D 0; - pr_info("KASAN init done\n"); + kasan_init_generic(); } =20 void __init kasan_late_init(void) { } diff --git a/arch/powerpc/mm/kasan/init_book3s_64.c b/arch/powerpc/mm/kasan= /init_book3s_64.c index 7d959544c077..dcafa641804c 100644 --- a/arch/powerpc/mm/kasan/init_book3s_64.c +++ b/arch/powerpc/mm/kasan/init_book3s_64.c @@ -19,8 +19,6 @@ #include #include =20 -DEFINE_STATIC_KEY_FALSE(powerpc_kasan_enabled_key); - static void __init kasan_init_phys_region(void *start, void *end) { unsigned long k_start, k_end, k_cur; @@ -92,11 +90,9 @@ void __init kasan_init(void) */ memset(kasan_early_shadow_page, 0, PAGE_SIZE); =20 - static_branch_inc(&powerpc_kasan_enabled_key); - /* Enable error messages */ init_task.kasan_depth =3D 0; - pr_info("KASAN init done\n"); + kasan_init_generic(); } =20 void __init kasan_early_init(void) { } diff --git a/arch/um/Kconfig b/arch/um/Kconfig index 9083bfdb7735..1d4def0db841 100644 --- a/arch/um/Kconfig +++ b/arch/um/Kconfig @@ -5,6 +5,7 @@ menu "UML-specific options" config UML bool default y + select ARCH_NEEDS_DEFER_KASAN if STATIC_LINK select ARCH_WANTS_DYNAMIC_TASK_STRUCT select ARCH_HAS_CACHE_LINE_SIZE select ARCH_HAS_CPU_FINALIZE_INIT diff --git a/arch/um/include/asm/kasan.h b/arch/um/include/asm/kasan.h index f97bb1f7b851..b54a4e937fd1 100644 --- a/arch/um/include/asm/kasan.h +++ b/arch/um/include/asm/kasan.h @@ -24,10 +24,9 @@ =20 #ifdef CONFIG_KASAN void kasan_init(void); -extern int kasan_um_is_ready; =20 -#ifdef CONFIG_STATIC_LINK -#define kasan_arch_is_ready() (kasan_um_is_ready) +#if defined(CONFIG_STATIC_LINK) && defined(CONFIG_KASAN_INLINE) +#error UML does not work in KASAN_INLINE mode with STATIC_LINK enabled! #endif #else static inline void kasan_init(void) { } diff --git a/arch/um/kernel/mem.c b/arch/um/kernel/mem.c index 76bec7de81b5..32e3b1972dc1 100644 --- a/arch/um/kernel/mem.c +++ b/arch/um/kernel/mem.c @@ -21,10 +21,10 @@ #include #include #include +#include =20 #ifdef CONFIG_KASAN -int kasan_um_is_ready; -void kasan_init(void) +void __init kasan_init(void) { /* * kasan_map_memory will map all of the required address space and @@ -32,7 +32,11 @@ void kasan_init(void) */ kasan_map_memory((void *)KASAN_SHADOW_START, KASAN_SHADOW_SIZE); init_task.kasan_depth =3D 0; - kasan_um_is_ready =3D true; + /* + * Since kasan_init() is called before main(), + * KASAN is initialized but the enablement is deferred after + * jump_label_init(). See arch_mm_preinit(). + */ } =20 static void (*kasan_init_ptr)(void) @@ -58,6 +62,9 @@ static unsigned long brk_end; =20 void __init arch_mm_preinit(void) { + /* Safe to call after jump_label_init(). Enables KASAN. */ + kasan_init_generic(); + /* clear the zero-page */ memset(empty_zero_page, 0, PAGE_SIZE); =20 diff --git a/include/linux/kasan-enabled.h b/include/linux/kasan-enabled.h index 6f612d69ea0c..9eca967d8526 100644 --- a/include/linux/kasan-enabled.h +++ b/include/linux/kasan-enabled.h @@ -4,32 +4,46 @@ =20 #include =20 -#ifdef CONFIG_KASAN_HW_TAGS - +#if defined(CONFIG_ARCH_DEFER_KASAN) || defined(CONFIG_KASAN_HW_TAGS) +/* + * Global runtime flag for KASAN modes that need runtime control. + * Used by ARCH_DEFER_KASAN architectures and HW_TAGS mode. + */ DECLARE_STATIC_KEY_FALSE(kasan_flag_enabled); =20 +/* + * Runtime control for shadow memory initialization or HW_TAGS mode. + * Uses static key for architectures that need deferred KASAN or HW_TAGS. + */ static __always_inline bool kasan_enabled(void) { return static_branch_likely(&kasan_flag_enabled); } =20 -static inline bool kasan_hw_tags_enabled(void) +static inline void kasan_enable(void) { - return kasan_enabled(); + static_branch_enable(&kasan_flag_enabled); } - -#else /* CONFIG_KASAN_HW_TAGS */ - -static inline bool kasan_enabled(void) +#else +/* For architectures that can enable KASAN early, use compile-time check. = */ +static __always_inline bool kasan_enabled(void) { return IS_ENABLED(CONFIG_KASAN); } =20 +static inline void kasan_enable(void) {} +#endif /* CONFIG_ARCH_DEFER_KASAN || CONFIG_KASAN_HW_TAGS */ + +#ifdef CONFIG_KASAN_HW_TAGS +static inline bool kasan_hw_tags_enabled(void) +{ + return kasan_enabled(); +} +#else static inline bool kasan_hw_tags_enabled(void) { return false; } - #endif /* CONFIG_KASAN_HW_TAGS */ =20 #endif /* LINUX_KASAN_ENABLED_H */ diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 890011071f2b..51a8293d1af6 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -543,6 +543,12 @@ void kasan_report_async(void); =20 #endif /* CONFIG_KASAN_HW_TAGS */ =20 +#ifdef CONFIG_KASAN_GENERIC +void __init kasan_init_generic(void); +#else +static inline void kasan_init_generic(void) { } +#endif + #ifdef CONFIG_KASAN_SW_TAGS void __init kasan_init_sw_tags(void); #else diff --git a/lib/Kconfig.kasan b/lib/Kconfig.kasan index f82889a830fa..a4bb610a7a6f 100644 --- a/lib/Kconfig.kasan +++ b/lib/Kconfig.kasan @@ -19,6 +19,18 @@ config ARCH_DISABLE_KASAN_INLINE Disables both inline and stack instrumentation. Selected by architectures that do not support these instrumentation types. =20 +config ARCH_NEEDS_DEFER_KASAN + bool + +config ARCH_DEFER_KASAN + def_bool y + depends on KASAN && ARCH_NEEDS_DEFER_KASAN + help + Architectures should select this if they need to defer KASAN + initialization until shadow memory is properly set up. This + enables runtime control via static keys. Otherwise, KASAN uses + compile-time constants for better performance. + config CC_HAS_KASAN_GENERIC def_bool $(cc-option, -fsanitize=3Dkernel-address) =20 diff --git a/mm/kasan/common.c b/mm/kasan/common.c index 9142964ab9c9..e3765931a31f 100644 --- a/mm/kasan/common.c +++ b/mm/kasan/common.c @@ -32,6 +32,15 @@ #include "kasan.h" #include "../slab.h" =20 +#if defined(CONFIG_ARCH_DEFER_KASAN) || defined(CONFIG_KASAN_HW_TAGS) +/* + * Definition of the unified static key declared in kasan-enabled.h. + * This provides consistent runtime enable/disable across KASAN modes. + */ +DEFINE_STATIC_KEY_FALSE(kasan_flag_enabled); +EXPORT_SYMBOL_GPL(kasan_flag_enabled); +#endif + struct slab *kasan_addr_to_slab(const void *addr) { if (virt_addr_valid(addr)) @@ -246,7 +255,7 @@ static inline void poison_slab_object(struct kmem_cache= *cache, void *object, bool __kasan_slab_pre_free(struct kmem_cache *cache, void *object, unsigned long ip) { - if (!kasan_arch_is_ready() || is_kfence_address(object)) + if (is_kfence_address(object)) return false; return check_slab_allocation(cache, object, ip); } @@ -254,7 +263,7 @@ bool __kasan_slab_pre_free(struct kmem_cache *cache, vo= id *object, bool __kasan_slab_free(struct kmem_cache *cache, void *object, bool init, bool still_accessible) { - if (!kasan_arch_is_ready() || is_kfence_address(object)) + if (is_kfence_address(object)) return false; =20 /* @@ -293,7 +302,7 @@ bool __kasan_slab_free(struct kmem_cache *cache, void *= object, bool init, =20 static inline bool check_page_allocation(void *ptr, unsigned long ip) { - if (!kasan_arch_is_ready()) + if (!kasan_enabled()) return false; =20 if (ptr !=3D page_address(virt_to_head_page(ptr))) { @@ -522,7 +531,7 @@ bool __kasan_mempool_poison_object(void *ptr, unsigned = long ip) return true; } =20 - if (is_kfence_address(ptr) || !kasan_arch_is_ready()) + if (is_kfence_address(ptr)) return true; =20 slab =3D folio_slab(folio); diff --git a/mm/kasan/generic.c b/mm/kasan/generic.c index d54e89f8c3e7..b413c46b3e04 100644 --- a/mm/kasan/generic.c +++ b/mm/kasan/generic.c @@ -36,6 +36,17 @@ #include "kasan.h" #include "../slab.h" =20 +/* + * Initialize Generic KASAN and enable runtime checks. + * This should be called from arch kasan_init() once shadow memory is read= y. + */ +void __init kasan_init_generic(void) +{ + kasan_enable(); + + pr_info("KernelAddressSanitizer initialized (generic)\n"); +} + /* * All functions below always inlined so compiler could * perform better optimizations in each of __asan_loadX/__assn_storeX @@ -165,7 +176,7 @@ static __always_inline bool check_region_inline(const v= oid *addr, size_t size, bool write, unsigned long ret_ip) { - if (!kasan_arch_is_ready()) + if (!kasan_enabled()) return true; =20 if (unlikely(size =3D=3D 0)) @@ -193,7 +204,7 @@ bool kasan_byte_accessible(const void *addr) { s8 shadow_byte; =20 - if (!kasan_arch_is_ready()) + if (!kasan_enabled()) return true; =20 shadow_byte =3D READ_ONCE(*(s8 *)kasan_mem_to_shadow(addr)); @@ -495,7 +506,7 @@ static void release_alloc_meta(struct kasan_alloc_meta = *meta) =20 static void release_free_meta(const void *object, struct kasan_free_meta *= meta) { - if (!kasan_arch_is_ready()) + if (!kasan_enabled()) return; =20 /* Check if free meta is valid. */ @@ -562,7 +573,7 @@ void kasan_save_alloc_info(struct kmem_cache *cache, vo= id *object, gfp_t flags) kasan_save_track(&alloc_meta->alloc_track, flags); } =20 -void kasan_save_free_info(struct kmem_cache *cache, void *object) +void __kasan_save_free_info(struct kmem_cache *cache, void *object) { struct kasan_free_meta *free_meta; =20 diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index 9a6927394b54..c8289a3feabf 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -45,13 +45,6 @@ static enum kasan_arg kasan_arg __ro_after_init; static enum kasan_arg_mode kasan_arg_mode __ro_after_init; static enum kasan_arg_vmalloc kasan_arg_vmalloc __initdata; =20 -/* - * Whether KASAN is enabled at all. - * The value remains false until KASAN is initialized by kasan_init_hw_tag= s(). - */ -DEFINE_STATIC_KEY_FALSE(kasan_flag_enabled); -EXPORT_SYMBOL(kasan_flag_enabled); - /* * Whether the selected mode is synchronous, asynchronous, or asymmetric. * Defaults to KASAN_MODE_SYNC. @@ -260,7 +253,7 @@ void __init kasan_init_hw_tags(void) kasan_init_tags(); =20 /* KASAN is now initialized, enable it. */ - static_branch_enable(&kasan_flag_enabled); + kasan_enable(); =20 pr_info("KernelAddressSanitizer initialized (hw-tags, mode=3D%s, vmalloc= =3D%s, stacktrace=3D%s)\n", kasan_mode_info(), diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 129178be5e64..8a9d8a6ea717 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -398,7 +398,13 @@ depot_stack_handle_t kasan_save_stack(gfp_t flags, dep= ot_flags_t depot_flags); void kasan_set_track(struct kasan_track *track, depot_stack_handle_t stack= ); void kasan_save_track(struct kasan_track *track, gfp_t flags); void kasan_save_alloc_info(struct kmem_cache *cache, void *object, gfp_t f= lags); -void kasan_save_free_info(struct kmem_cache *cache, void *object); + +void __kasan_save_free_info(struct kmem_cache *cache, void *object); +static inline void kasan_save_free_info(struct kmem_cache *cache, void *ob= ject) +{ + if (kasan_enabled()) + __kasan_save_free_info(cache, object); +} =20 #ifdef CONFIG_KASAN_GENERIC bool kasan_quarantine_put(struct kmem_cache *cache, void *object); diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c index d2c70cd2afb1..2e126cb21b68 100644 --- a/mm/kasan/shadow.c +++ b/mm/kasan/shadow.c @@ -125,7 +125,7 @@ void kasan_poison(const void *addr, size_t size, u8 val= ue, bool init) { void *shadow_start, *shadow_end; =20 - if (!kasan_arch_is_ready()) + if (!kasan_enabled()) return; =20 /* @@ -150,7 +150,7 @@ EXPORT_SYMBOL_GPL(kasan_poison); #ifdef CONFIG_KASAN_GENERIC void kasan_poison_last_granule(const void *addr, size_t size) { - if (!kasan_arch_is_ready()) + if (!kasan_enabled()) return; =20 if (size & KASAN_GRANULE_MASK) { @@ -390,7 +390,7 @@ int kasan_populate_vmalloc(unsigned long addr, unsigned= long size) unsigned long shadow_start, shadow_end; int ret; =20 - if (!kasan_arch_is_ready()) + if (!kasan_enabled()) return 0; =20 if (!is_vmalloc_or_module_addr((void *)addr)) @@ -560,7 +560,7 @@ void kasan_release_vmalloc(unsigned long start, unsigne= d long end, unsigned long region_start, region_end; unsigned long size; =20 - if (!kasan_arch_is_ready()) + if (!kasan_enabled()) return; =20 region_start =3D ALIGN(start, KASAN_MEMORY_PER_SHADOW_PAGE); @@ -611,7 +611,7 @@ void *__kasan_unpoison_vmalloc(const void *start, unsig= ned long size, * with setting memory tags, so the KASAN_VMALLOC_INIT flag is ignored. */ =20 - if (!kasan_arch_is_ready()) + if (!kasan_enabled()) return (void *)start; =20 if (!is_vmalloc_or_module_addr(start)) @@ -636,7 +636,7 @@ void *__kasan_unpoison_vmalloc(const void *start, unsig= ned long size, */ void __kasan_poison_vmalloc(const void *start, unsigned long size) { - if (!kasan_arch_is_ready()) + if (!kasan_enabled()) return; =20 if (!is_vmalloc_or_module_addr(start)) diff --git a/mm/kasan/sw_tags.c b/mm/kasan/sw_tags.c index b9382b5b6a37..c75741a74602 100644 --- a/mm/kasan/sw_tags.c +++ b/mm/kasan/sw_tags.c @@ -44,6 +44,7 @@ void __init kasan_init_sw_tags(void) per_cpu(prng_state, cpu) =3D (u32)get_cycles(); =20 kasan_init_tags(); + kasan_enable(); =20 pr_info("KernelAddressSanitizer initialized (sw-tags, stacktrace=3D%s)\n", str_on_off(kasan_stack_collection_enabled())); diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c index d65d48b85f90..b9f31293622b 100644 --- a/mm/kasan/tags.c +++ b/mm/kasan/tags.c @@ -142,7 +142,7 @@ void kasan_save_alloc_info(struct kmem_cache *cache, vo= id *object, gfp_t flags) save_stack_info(cache, object, flags, false); } =20 -void kasan_save_free_info(struct kmem_cache *cache, void *object) +void __kasan_save_free_info(struct kmem_cache *cache, void *object) { save_stack_info(cache, object, 0, true); } --=20 2.34.1