From nobody Sat Oct 4 20:54:34 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 21EEF27FB3E; Tue, 12 Aug 2025 13:26:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005214; cv=none; b=Qmt/e0jDo57oBX7m/KQwozUtQ/1wy7bbutoop+miVQOk1uDX13yP03qRzbp8ZQZjnS9PGcImm6j3vlgNYz2XudfP0ZBGF0tkbEkjquMc5T8OEAVKNz19BfJzk2t9oL/rn63LcqPjHtK3hFT5JlE4nQEQveh1mAUAcKW+ZyT+9Wo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005214; c=relaxed/simple; bh=t2SLP7Q10LFKMhAOypv/6H2o8JXEilxwoe/3oShHQgM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=m9y8AS8AG5+BejI+501hUgKgt7N62ouHLDHK6xkCJq99FWgo0ZAKQMzKrny6e3anbXM22r+4lqSA9jQ6JZ0jG6l36nH7AYyjLPyJF5tnd0l4Ymjy1+ibkBHu6jC78CqBx0IxV67tvIbPYWz+tjJVC2RFyF1kBdDmdTFmU4kIZzc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=HKPqmlOh; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="HKPqmlOh" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1755005213; x=1786541213; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=t2SLP7Q10LFKMhAOypv/6H2o8JXEilxwoe/3oShHQgM=; b=HKPqmlOhMHyMHn6slzSt4hkeyCgfel6ITWrm0UJRi8VlcRCqjPK675H+ hKy4wxd8EdqALs3ttltPT6zBooNYlXgXow6ojYNLneII/0lRa3hpOZFIj QXdW6XDQ9iuD3AooUbpFQ7NE+NtFFFUDAQsZPRJdd8X8ZaTm3RWmbzv3s Fqqjb5/T8wVGFl0ywXoeyH5Y8tjyW6g/y0dQ8ZMqEDGSZ9/urcP2/fdxA Bj/lnJTcN+zuFW+4opZn/sYFk2GUCOGGlXxdmFD2h8EaF/Tg0b6KaggjU 6rMHqI4FdBpC3LwyyWsUZsXlJsThrqMBLSNFB3/xwf/mJ/cb7B0LDgEBz Q==; X-CSE-ConnectionGUID: P+AxH26fS4OArTyUkgQRvA== X-CSE-MsgGUID: 4yJBEOKmTGW4wD8LqE7aSA== X-IronPort-AV: E=McAfee;i="6800,10657,11520"; a="60903174" X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="60903174" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:25:22 -0700 X-CSE-ConnectionGUID: 1w2jEZ//QCW7oiX+WFh/vw== X-CSE-MsgGUID: LyynSEAjR+uvqbS3d1NVXQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="165831306" Received: from vpanait-mobl.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.245.54]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:24:59 -0700 From: Maciej Wieczor-Retman To: nathan@kernel.org, arnd@arndb.de, broonie@kernel.org, Liam.Howlett@oracle.com, urezki@gmail.com, will@kernel.org, kaleshsingh@google.com, rppt@kernel.org, leitao@debian.org, coxu@redhat.com, surenb@google.com, akpm@linux-foundation.org, luto@kernel.org, jpoimboe@kernel.org, changyuanl@google.com, hpa@zytor.com, dvyukov@google.com, kas@kernel.org, corbet@lwn.net, vincenzo.frascino@arm.com, smostafa@google.com, nick.desaulniers+lkml@gmail.com, morbo@google.com, andreyknvl@gmail.com, alexander.shishkin@linux.intel.com, thiago.bauermann@linaro.org, catalin.marinas@arm.com, ryabinin.a.a@gmail.com, jan.kiszka@siemens.com, jbohac@suse.cz, dan.j.williams@intel.com, joel.granados@kernel.org, baohua@kernel.org, kevin.brodsky@arm.com, nicolas.schier@linux.dev, pcc@google.com, andriy.shevchenko@linux.intel.com, wei.liu@kernel.org, bp@alien8.de, ada.coupriediaz@arm.com, xin@zytor.com, pankaj.gupta@amd.com, vbabka@suse.cz, glider@google.com, jgross@suse.com, kees@kernel.org, jhubbard@nvidia.com, joey.gouly@arm.com, ardb@kernel.org, thuth@redhat.com, pasha.tatashin@soleen.com, kristina.martsenko@arm.com, bigeasy@linutronix.de, maciej.wieczor-retman@intel.com, lorenzo.stoakes@oracle.com, jason.andryuk@amd.com, david@redhat.com, graf@amazon.com, wangkefeng.wang@huawei.com, ziy@nvidia.com, mark.rutland@arm.com, dave.hansen@linux.intel.com, samuel.holland@sifive.com, kbingham@kernel.org, trintaeoitogc@gmail.com, scott@os.amperecomputing.com, justinstitt@google.com, kuan-ying.lee@canonical.com, maz@kernel.org, tglx@linutronix.de, samitolvanen@google.com, mhocko@suse.com, nunodasneves@linux.microsoft.com, brgerst@gmail.com, willy@infradead.org, ubizjak@gmail.com, peterz@infradead.org, mingo@redhat.com, sohil.mehta@intel.com Cc: linux-mm@kvack.org, linux-kbuild@vger.kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org, llvm@lists.linux.dev, kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 01/18] kasan: sw_tags: Use arithmetic shift for shadow computation Date: Tue, 12 Aug 2025 15:23:37 +0200 Message-ID: <6c04b001999b845f9f601ae42ccd116dcca6b4f1.1755004923.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Samuel Holland Currently, kasan_mem_to_shadow() uses a logical right shift, which turns canonical kernel addresses into non-canonical addresses by clearing the high KASAN_SHADOW_SCALE_SHIFT bits. The value of KASAN_SHADOW_OFFSET is then chosen so that the addition results in a canonical address for the shadow memory. For KASAN_GENERIC, this shift/add combination is ABI with the compiler, because KASAN_SHADOW_OFFSET is used in compiler-generated inline tag checks[1], which must only attempt to dereference canonical addresses. However, for KASAN_SW_TAGS we have some freedom to change the algorithm without breaking the ABI. Because TBI is enabled for kernel addresses, the top bits of shadow memory addresses computed during tag checks are irrelevant, and so likewise are the top bits of KASAN_SHADOW_OFFSET. This is demonstrated by the fact that LLVM uses a logical right shift in the tag check fast path[2] but a sbfx (signed bitfield extract) instruction in the slow path[3] without causing any issues. Using an arithmetic shift in kasan_mem_to_shadow() provides a number of benefits: 1) The memory layout doesn't change but is easier to understand. KASAN_SHADOW_OFFSET becomes a canonical memory address, and the shifted pointer becomes a negative offset, so KASAN_SHADOW_OFFSET =3D=3D KASAN_SHADOW_END regardless of the shift amount or the size of the virtual address space. 2) KASAN_SHADOW_OFFSET becomes a simpler constant, requiring only one instruction to load instead of two. Since it must be loaded in each function with a tag check, this decreases kernel text size by 0.5%. 3) This shift and the sign extension from kasan_reset_tag() can be combined into a single sbfx instruction. When this same algorithm change is applied to the compiler, it removes an instruction from each inline tag check, further reducing kernel text size by an additional 4.6%. These benefits extend to other architectures as well. On RISC-V, where the baseline ISA does not shifted addition or have an equivalent to the sbfx instruction, loading KASAN_SHADOW_OFFSET is reduced from 3 to 2 instructions, and kasan_mem_to_shadow(kasan_reset_tag(addr)) similarly combines two consecutive right shifts. Link: https://github.com/llvm/llvm-project/blob/llvmorg-20-init/llvm/lib/Tr= ansforms/Instrumentation/AddressSanitizer.cpp#L1316 [1] Link: https://github.com/llvm/llvm-project/blob/llvmorg-20-init/llvm/lib/Tr= ansforms/Instrumentation/HWAddressSanitizer.cpp#L895 [2] Link: https://github.com/llvm/llvm-project/blob/llvmorg-20-init/llvm/lib/Ta= rget/AArch64/AArch64AsmPrinter.cpp#L669 [3] Signed-off-by: Samuel Holland Signed-off-by: Maciej Wieczor-Retman --- Changelog v4: (Maciej) - Revert x86 to signed mem_to_shadow mapping. - Remove last two paragraphs since they were just poorer duplication of the comments in kasan_non_canonical_hook(). Changelog v3: (Maciej) - Fix scripts/gdb/linux/kasan.py so the new signed mem_to_shadow() is reflected there. - Fix Documentation/arch/arm64/kasan-offsets.sh to take new offsets into account. - Made changes to the kasan_non_canonical_hook() according to upstream discussion. Settled on overflow on both ranges and separate checks for x86 and arm. Changelog v2: (Maciej) - Correct address range that's checked in kasan_non_canonical_hook(). Adjust the comment inside. - Remove part of comment from arch/arm64/include/asm/memory.h. - Append patch message paragraph about the overflow in kasan_non_canonical_hook(). Documentation/arch/arm64/kasan-offsets.sh | 8 +++-- arch/arm64/Kconfig | 10 +++---- arch/arm64/include/asm/memory.h | 14 ++++++++- arch/arm64/mm/kasan_init.c | 7 +++-- include/linux/kasan.h | 10 +++++-- mm/kasan/report.c | 36 ++++++++++++++++++++--- scripts/gdb/linux/kasan.py | 3 ++ scripts/gdb/linux/mm.py | 5 ++-- 8 files changed, 75 insertions(+), 18 deletions(-) mode change 100644 =3D> 100755 Documentation/arch/arm64/kasan-offsets.sh diff --git a/Documentation/arch/arm64/kasan-offsets.sh b/Documentation/arch= /arm64/kasan-offsets.sh old mode 100644 new mode 100755 index 2dc5f9e18039..ce777c7c7804 --- a/Documentation/arch/arm64/kasan-offsets.sh +++ b/Documentation/arch/arm64/kasan-offsets.sh @@ -5,8 +5,12 @@ =20 print_kasan_offset () { printf "%02d\t" $1 - printf "0x%08x00000000\n" $(( (0xffffffff & (-1 << ($1 - 1 - 32))) \ - - (1 << (64 - 32 - $2)) )) + if [[ $2 -ne 4 ]] then + printf "0x%08x00000000\n" $(( (0xffffffff & (-1 << ($1 - 1 - 32))) \ + - (1 << (64 - 32 - $2)) )) + else + printf "0x%08x00000000\n" $(( (0xffffffff & (-1 << ($1 - 1 - 32))) )) + fi } =20 echo KASAN_SHADOW_SCALE_SHIFT =3D 3 diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index e9bbfacc35a6..82cbfc7d1233 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -431,11 +431,11 @@ config KASAN_SHADOW_OFFSET default 0xdffffe0000000000 if ARM64_VA_BITS_42 && !KASAN_SW_TAGS default 0xdfffffc000000000 if ARM64_VA_BITS_39 && !KASAN_SW_TAGS default 0xdffffff800000000 if ARM64_VA_BITS_36 && !KASAN_SW_TAGS - default 0xefff800000000000 if (ARM64_VA_BITS_48 || (ARM64_VA_BITS_52 && != ARM64_16K_PAGES)) && KASAN_SW_TAGS - default 0xefffc00000000000 if (ARM64_VA_BITS_47 || ARM64_VA_BITS_52) && A= RM64_16K_PAGES && KASAN_SW_TAGS - default 0xeffffe0000000000 if ARM64_VA_BITS_42 && KASAN_SW_TAGS - default 0xefffffc000000000 if ARM64_VA_BITS_39 && KASAN_SW_TAGS - default 0xeffffff800000000 if ARM64_VA_BITS_36 && KASAN_SW_TAGS + default 0xffff800000000000 if (ARM64_VA_BITS_48 || (ARM64_VA_BITS_52 && != ARM64_16K_PAGES)) && KASAN_SW_TAGS + default 0xffffc00000000000 if (ARM64_VA_BITS_47 || ARM64_VA_BITS_52) && A= RM64_16K_PAGES && KASAN_SW_TAGS + default 0xfffffe0000000000 if ARM64_VA_BITS_42 && KASAN_SW_TAGS + default 0xffffffc000000000 if ARM64_VA_BITS_39 && KASAN_SW_TAGS + default 0xfffffff800000000 if ARM64_VA_BITS_36 && KASAN_SW_TAGS default 0xffffffffffffffff =20 config UNWIND_TABLES diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memor= y.h index 5213248e081b..277d56ceeb01 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -89,7 +89,15 @@ * * KASAN_SHADOW_END is defined first as the shadow address that correspond= s to * the upper bound of possible virtual kernel memory addresses UL(1) << 64 - * according to the mapping formula. + * according to the mapping formula. For Generic KASAN, the address in the + * mapping formula is treated as unsigned (part of the compiler's ABI), so= the + * end of the shadow memory region is at a large positive offset from + * KASAN_SHADOW_OFFSET. For Software Tag-Based KASAN, the address in the + * formula is treated as signed. Since all kernel addresses are negative, = they + * map to shadow memory below KASAN_SHADOW_OFFSET, making KASAN_SHADOW_OFF= SET + * itself the end of the shadow memory region. (User pointers are positive= and + * would map to shadow memory above KASAN_SHADOW_OFFSET, but shadow memory= is + * not allocated for them.) * * KASAN_SHADOW_START is defined second based on KASAN_SHADOW_END. The sha= dow * memory start must map to the lowest possible kernel virtual memory addr= ess @@ -100,7 +108,11 @@ */ #if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) #define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) +#ifdef CONFIG_KASAN_GENERIC #define KASAN_SHADOW_END ((UL(1) << (64 - KASAN_SHADOW_SCALE_SHIFT)) + KAS= AN_SHADOW_OFFSET) +#else +#define KASAN_SHADOW_END KASAN_SHADOW_OFFSET +#endif #define _KASAN_SHADOW_START(va) (KASAN_SHADOW_END - (UL(1) << ((va) - KASA= N_SHADOW_SCALE_SHIFT))) #define KASAN_SHADOW_START _KASAN_SHADOW_START(vabits_actual) #define PAGE_END KASAN_SHADOW_START diff --git a/arch/arm64/mm/kasan_init.c b/arch/arm64/mm/kasan_init.c index d541ce45daeb..dc2de12c4f26 100644 --- a/arch/arm64/mm/kasan_init.c +++ b/arch/arm64/mm/kasan_init.c @@ -198,8 +198,11 @@ static bool __init root_level_aligned(u64 addr) /* The early shadow maps everything to a single page of zeroes */ asmlinkage void __init kasan_early_init(void) { - BUILD_BUG_ON(KASAN_SHADOW_OFFSET !=3D - KASAN_SHADOW_END - (1UL << (64 - KASAN_SHADOW_SCALE_SHIFT))); + if (IS_ENABLED(CONFIG_KASAN_GENERIC)) + BUILD_BUG_ON(KASAN_SHADOW_OFFSET !=3D + KASAN_SHADOW_END - (1UL << (64 - KASAN_SHADOW_SCALE_SHIFT))); + else + BUILD_BUG_ON(KASAN_SHADOW_OFFSET !=3D KASAN_SHADOW_END); BUILD_BUG_ON(!IS_ALIGNED(_KASAN_SHADOW_START(VA_BITS), SHADOW_ALIGN)); BUILD_BUG_ON(!IS_ALIGNED(_KASAN_SHADOW_START(VA_BITS_MIN), SHADOW_ALIGN)); BUILD_BUG_ON(!IS_ALIGNED(KASAN_SHADOW_END, SHADOW_ALIGN)); diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 890011071f2b..b396feca714f 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -61,8 +61,14 @@ int kasan_populate_early_shadow(const void *shadow_start, #ifndef kasan_mem_to_shadow static inline void *kasan_mem_to_shadow(const void *addr) { - return (void *)((unsigned long)addr >> KASAN_SHADOW_SCALE_SHIFT) - + KASAN_SHADOW_OFFSET; + void *scaled; + + if (IS_ENABLED(CONFIG_KASAN_GENERIC)) + scaled =3D (void *)((unsigned long)addr >> KASAN_SHADOW_SCALE_SHIFT); + else + scaled =3D (void *)((long)addr >> KASAN_SHADOW_SCALE_SHIFT); + + return KASAN_SHADOW_OFFSET + scaled; } #endif =20 diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 62c01b4527eb..93c6cadb0765 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -642,11 +642,39 @@ void kasan_non_canonical_hook(unsigned long addr) const char *bug_type; =20 /* - * All addresses that came as a result of the memory-to-shadow mapping - * (even for bogus pointers) must be >=3D KASAN_SHADOW_OFFSET. + * For Generic KASAN, kasan_mem_to_shadow() uses the logical right shift + * and never overflows with the chosen KASAN_SHADOW_OFFSET values (on + * both x86 and arm64). Thus, the possible shadow addresses (even for + * bogus pointers) belong to a single contiguous region that is the + * result of kasan_mem_to_shadow() applied to the whole address space. */ - if (addr < KASAN_SHADOW_OFFSET) - return; + if (IS_ENABLED(CONFIG_KASAN_GENERIC)) { + if (addr < (u64)kasan_mem_to_shadow((void *)(0UL)) || + addr > (u64)kasan_mem_to_shadow((void *)(~0UL))) + return; + } + + /* + * For Software Tag-Based KASAN, kasan_mem_to_shadow() uses the + * arithmetic shift. Normally, this would make checking for a possible + * shadow address complicated, as the shadow address computation + * operation would overflow only for some memory addresses. However, due + * to the chosen KASAN_SHADOW_OFFSET values and the fact the + * kasan_mem_to_shadow() only operates on pointers with the tag reset, + * the overflow always happens. + * + * For arm64, the top byte of the pointer gets reset to 0xFF. Thus, the + * possible shadow addresses belong to a region that is the result of + * kasan_mem_to_shadow() applied to the memory range + * [0xFF000000000000, 0xFFFFFFFFFFFFFFFF]. Despite the overflow, the + * resulting possible shadow region is contiguous, as the overflow + * happens for both 0xFF000000000000 and 0xFFFFFFFFFFFFFFFF. + */ + if (IS_ENABLED(CONFIG_KASAN_SW_TAGS) && IS_ENABLED(CONFIG_ARM64)) { + if (addr < (u64)kasan_mem_to_shadow((void *)(0xFFUL << 56)) || + addr > (u64)kasan_mem_to_shadow((void *)(~0UL))) + return; + } =20 orig_addr =3D (unsigned long)kasan_shadow_to_mem((void *)addr); =20 diff --git a/scripts/gdb/linux/kasan.py b/scripts/gdb/linux/kasan.py index 56730b3fde0b..fca39968d308 100644 --- a/scripts/gdb/linux/kasan.py +++ b/scripts/gdb/linux/kasan.py @@ -8,6 +8,7 @@ =20 import gdb from linux import constants, mm +from ctypes import c_int64 as s64 =20 def help(): t =3D """Usage: lx-kasan_mem_to_shadow [Hex memory addr] @@ -39,6 +40,8 @@ class KasanMemToShadow(gdb.Command): else: help() def kasan_mem_to_shadow(self, addr): + if constants.CONFIG_KASAN_SW_TAGS: + addr =3D s64(addr) return (addr >> self.p_ops.KASAN_SHADOW_SCALE_SHIFT) + self.p_ops.= KASAN_SHADOW_OFFSET =20 KasanMemToShadow() diff --git a/scripts/gdb/linux/mm.py b/scripts/gdb/linux/mm.py index 7571aebbe650..2e63f3dedd53 100644 --- a/scripts/gdb/linux/mm.py +++ b/scripts/gdb/linux/mm.py @@ -110,12 +110,13 @@ class aarch64_page_ops(): self.KERNEL_END =3D gdb.parse_and_eval("_end") =20 if constants.LX_CONFIG_KASAN_GENERIC or constants.LX_CONFIG_KASAN_= SW_TAGS: + self.KASAN_SHADOW_OFFSET =3D constants.LX_CONFIG_KASAN_SHADOW_= OFFSET if constants.LX_CONFIG_KASAN_GENERIC: self.KASAN_SHADOW_SCALE_SHIFT =3D 3 + self.KASAN_SHADOW_END =3D (1 << (64 - self.KASAN_SHADOW_SC= ALE_SHIFT)) + self.KASAN_SHADOW_OFFSET else: self.KASAN_SHADOW_SCALE_SHIFT =3D 4 - self.KASAN_SHADOW_OFFSET =3D constants.LX_CONFIG_KASAN_SHADOW_= OFFSET - self.KASAN_SHADOW_END =3D (1 << (64 - self.KASAN_SHADOW_SCALE_= SHIFT)) + self.KASAN_SHADOW_OFFSET + self.KASAN_SHADOW_END =3D self.KASAN_SHADOW_OFFSET self.PAGE_END =3D self.KASAN_SHADOW_END - (1 << (self.vabits_a= ctual - self.KASAN_SHADOW_SCALE_SHIFT)) else: self.PAGE_END =3D self._PAGE_END(self.VA_BITS_MIN) --=20 2.50.1 From nobody Sat Oct 4 20:54:34 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D402727FB1E; Tue, 12 Aug 2025 13:26:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005217; cv=none; b=Da1uc4jb8LlH+xTiUBN6LtkN2sKsVC31M3U0KTlU+Vq/NCRrpI+Z+c3iLBy9495A2dzY6UWWDq8fuZGbFhgUBE65dXOWtieDSwX0Mf6gwTHFJGr7YgsL2XG5wvtGSZXIcqZTh0aftQ1nRKRs2GdnvgNlYosgPtrciY997jwZGxc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005217; c=relaxed/simple; bh=9/RZLCMjXQxGvwHsrxYm6WSdmas+FAtxJYkFWNEbaFU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nI3dXYd3phC0XqRDhLkxLEtfmZAkm2ynvzlMX8+8/e+lWXdV+wtLElUZYP5m4yk/mbr1Fns8tylUBag222iHJBrip5g7JJGZtuqPf7d96akrNSxENFVyyRvvGj3bkjyKkkjIFLiFvZyL8/vN1r8FVV47nBqJUWGR9ZelnMN/onU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=IEnMVdw1; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="IEnMVdw1" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1755005216; x=1786541216; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=9/RZLCMjXQxGvwHsrxYm6WSdmas+FAtxJYkFWNEbaFU=; b=IEnMVdw1sUKPcMnz7Fo2YZ/gjMRLShxTvtQk/Bx8pVeA1kmj9P6B+OZT kleKFVhDqQGdlFGKXFSikZurV00VN4Rrt3Izbasm+IedCeQMPbp3uwm3/ SiAVOXguLKKg6eaU5TpoB1I94lBV2Jh5VgDf+g+RfOJCANrQSWrcdfkWr Anno6asytPxEz8OYzRgYnBiYlXzWHpjlPb4bVigOYpwFNSc1Eqpy4VQ+H dIps7SY/LbYcokaJyBNJPbxoLr493K3Q9VsAIds7krDafDz2d1b+d45gg VI+ug0Xp4DaG3IfrT/sUV3iR3VYTgT3jndaIGHDWzEhRMjbtOFRo0BWqW g==; X-CSE-ConnectionGUID: PvUtcdLjSXy+RDYJkW8HWw== X-CSE-MsgGUID: fbfd+KFhQzSyiOwSjtdNGQ== X-IronPort-AV: E=McAfee;i="6800,10657,11520"; a="60903230" X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="60903230" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:25:44 -0700 X-CSE-ConnectionGUID: G0cKUUX3QdeIuuUQDf731Q== X-CSE-MsgGUID: HyifQRzJTUaZ4oYqgT0L6w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="165831317" Received: from vpanait-mobl.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.245.54]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:25:22 -0700 From: Maciej Wieczor-Retman To: nathan@kernel.org, arnd@arndb.de, broonie@kernel.org, Liam.Howlett@oracle.com, urezki@gmail.com, will@kernel.org, kaleshsingh@google.com, rppt@kernel.org, leitao@debian.org, coxu@redhat.com, surenb@google.com, akpm@linux-foundation.org, luto@kernel.org, jpoimboe@kernel.org, changyuanl@google.com, hpa@zytor.com, dvyukov@google.com, kas@kernel.org, corbet@lwn.net, vincenzo.frascino@arm.com, smostafa@google.com, nick.desaulniers+lkml@gmail.com, morbo@google.com, andreyknvl@gmail.com, alexander.shishkin@linux.intel.com, thiago.bauermann@linaro.org, catalin.marinas@arm.com, ryabinin.a.a@gmail.com, jan.kiszka@siemens.com, jbohac@suse.cz, dan.j.williams@intel.com, joel.granados@kernel.org, baohua@kernel.org, kevin.brodsky@arm.com, nicolas.schier@linux.dev, pcc@google.com, andriy.shevchenko@linux.intel.com, wei.liu@kernel.org, bp@alien8.de, ada.coupriediaz@arm.com, xin@zytor.com, pankaj.gupta@amd.com, vbabka@suse.cz, glider@google.com, jgross@suse.com, kees@kernel.org, jhubbard@nvidia.com, joey.gouly@arm.com, ardb@kernel.org, thuth@redhat.com, pasha.tatashin@soleen.com, kristina.martsenko@arm.com, bigeasy@linutronix.de, maciej.wieczor-retman@intel.com, lorenzo.stoakes@oracle.com, jason.andryuk@amd.com, david@redhat.com, graf@amazon.com, wangkefeng.wang@huawei.com, ziy@nvidia.com, mark.rutland@arm.com, dave.hansen@linux.intel.com, samuel.holland@sifive.com, kbingham@kernel.org, trintaeoitogc@gmail.com, scott@os.amperecomputing.com, justinstitt@google.com, kuan-ying.lee@canonical.com, maz@kernel.org, tglx@linutronix.de, samitolvanen@google.com, mhocko@suse.com, nunodasneves@linux.microsoft.com, brgerst@gmail.com, willy@infradead.org, ubizjak@gmail.com, peterz@infradead.org, mingo@redhat.com, sohil.mehta@intel.com Cc: linux-mm@kvack.org, linux-kbuild@vger.kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org, llvm@lists.linux.dev, kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 02/18] kasan: sw_tags: Support tag widths less than 8 bits Date: Tue, 12 Aug 2025 15:23:38 +0200 Message-ID: <780347f3897ea97e90968de028c9dd02f466204e.1755004923.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Samuel Holland Allow architectures to override KASAN_TAG_KERNEL in asm/kasan.h. This is needed on RISC-V, which supports 57-bit virtual addresses and 7-bit pointer tags. For consistency, move the arm64 MTE definition of KASAN_TAG_MIN to asm/kasan.h, since it is also architecture-dependent; RISC-V's equivalent extension is expected to support 7-bit hardware memory tags. Reviewed-by: Andrey Konovalov Signed-off-by: Samuel Holland Signed-off-by: Maciej Wieczor-Retman --- arch/arm64/include/asm/kasan.h | 6 ++++-- arch/arm64/include/asm/uaccess.h | 1 + include/linux/kasan-tags.h | 13 ++++++++----- 3 files changed, 13 insertions(+), 7 deletions(-) diff --git a/arch/arm64/include/asm/kasan.h b/arch/arm64/include/asm/kasan.h index e1b57c13f8a4..4ab419df8b93 100644 --- a/arch/arm64/include/asm/kasan.h +++ b/arch/arm64/include/asm/kasan.h @@ -6,8 +6,10 @@ =20 #include #include -#include -#include + +#ifdef CONFIG_KASAN_HW_TAGS +#define KASAN_TAG_MIN 0xF0 /* minimum value for random tags */ +#endif =20 #define arch_kasan_set_tag(addr, tag) __tag_set(addr, tag) #define arch_kasan_reset_tag(addr) __tag_reset(addr) diff --git a/arch/arm64/include/asm/uaccess.h b/arch/arm64/include/asm/uacc= ess.h index 5b91803201ef..f890dadc7b4e 100644 --- a/arch/arm64/include/asm/uaccess.h +++ b/arch/arm64/include/asm/uaccess.h @@ -22,6 +22,7 @@ #include #include #include +#include #include #include #include diff --git a/include/linux/kasan-tags.h b/include/linux/kasan-tags.h index 4f85f562512c..e07c896f95d3 100644 --- a/include/linux/kasan-tags.h +++ b/include/linux/kasan-tags.h @@ -2,13 +2,16 @@ #ifndef _LINUX_KASAN_TAGS_H #define _LINUX_KASAN_TAGS_H =20 +#include + +#ifndef KASAN_TAG_KERNEL #define KASAN_TAG_KERNEL 0xFF /* native kernel pointers tag */ -#define KASAN_TAG_INVALID 0xFE /* inaccessible memory tag */ -#define KASAN_TAG_MAX 0xFD /* maximum value for random tags */ +#endif + +#define KASAN_TAG_INVALID (KASAN_TAG_KERNEL - 1) /* inaccessible memory ta= g */ +#define KASAN_TAG_MAX (KASAN_TAG_KERNEL - 2) /* maximum value for random = tags */ =20 -#ifdef CONFIG_KASAN_HW_TAGS -#define KASAN_TAG_MIN 0xF0 /* minimum value for random tags */ -#else +#ifndef KASAN_TAG_MIN #define KASAN_TAG_MIN 0x00 /* minimum value for random tags */ #endif =20 --=20 2.50.1 From nobody Sat Oct 4 20:54:34 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1C224280CF6; Tue, 12 Aug 2025 13:26:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005219; cv=none; b=CDkefsZIe9TfNma2+OAxDT8HdlUw/SXFx5t4b1ccB3NNnmfVmcXNxxV3LoZD+n002ebwIgOzKfTF+pfpcS9QKmxXA1Yh2j8NmMDrPCFVjBv2syr9n0xACpPPvfe3Avv4yjYbMo4yxvsiSg4tuPLpJxvD1dmE0juZydEMPspOaGE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005219; c=relaxed/simple; bh=svwcdRMIJEw5xAT6fmvSyI7M9DNKZMAHSataOorEZqU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZGsnRbLvW6642ReOrtSHyRh108HP8tGqp6bD74EAitvIEe5KOeq2heHj49FVjAzPWKr9v0FH2bmDhGQ8QysLzUK7e9E3pfVgnGWRDXn6S8tliDJyfUBPGEYzUp61QPgSfxps93S/VYMP92uy6aGWQZp2edO3mtHduJFdCCd2S7o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=IZdmiskg; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="IZdmiskg" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1755005219; x=1786541219; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=svwcdRMIJEw5xAT6fmvSyI7M9DNKZMAHSataOorEZqU=; b=IZdmiskgb5rDGcCNmk4lW22n1/SBDCq86PwNht72yAv51OWcu8vK5VVB UCKld6lmNKqwVf+mYYp6vF2+dnWcOhXDN2hk161wRtuzo3lEIC/V7C4rG 3bfAmbQ2Zq4iYl42SVQ1Hh+MO1OAOc5jOZR1pOIL4HR0ngyFQoI8sUAQ0 N+l9rczPfaAJpWlENvqWy8ovpikSkrEJfR7pdTjAQyCP7md6YsT4c0hl9 ahHVMhRuIlnAoi2isrXh28IpEtxlcjSAib19noAWxKaZyXqpamoy5wEXZ BLScwgzs3eMTxVIp/MUvlNrsVeSi135P1xZ4APwwHfF7HL+ReBLP5X0zJ w==; X-CSE-ConnectionGUID: fdgl1WClTLueQSpdMteGkg== X-CSE-MsgGUID: o9Gd2+xjTlOV9vqeNY02AA== X-IronPort-AV: E=McAfee;i="6800,10657,11520"; a="60903292" X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="60903292" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:26:10 -0700 X-CSE-ConnectionGUID: fEBGC8HBRSeUSf2GiP4qhQ== X-CSE-MsgGUID: dKdrPa1TQk6ZCZHGur3yuw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="165831361" Received: from vpanait-mobl.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.245.54]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:25:43 -0700 From: Maciej Wieczor-Retman To: nathan@kernel.org, arnd@arndb.de, broonie@kernel.org, Liam.Howlett@oracle.com, urezki@gmail.com, will@kernel.org, kaleshsingh@google.com, rppt@kernel.org, leitao@debian.org, coxu@redhat.com, surenb@google.com, akpm@linux-foundation.org, luto@kernel.org, jpoimboe@kernel.org, changyuanl@google.com, hpa@zytor.com, dvyukov@google.com, kas@kernel.org, corbet@lwn.net, vincenzo.frascino@arm.com, smostafa@google.com, nick.desaulniers+lkml@gmail.com, morbo@google.com, andreyknvl@gmail.com, alexander.shishkin@linux.intel.com, thiago.bauermann@linaro.org, catalin.marinas@arm.com, ryabinin.a.a@gmail.com, jan.kiszka@siemens.com, jbohac@suse.cz, dan.j.williams@intel.com, joel.granados@kernel.org, baohua@kernel.org, kevin.brodsky@arm.com, nicolas.schier@linux.dev, pcc@google.com, andriy.shevchenko@linux.intel.com, wei.liu@kernel.org, bp@alien8.de, ada.coupriediaz@arm.com, xin@zytor.com, pankaj.gupta@amd.com, vbabka@suse.cz, glider@google.com, jgross@suse.com, kees@kernel.org, jhubbard@nvidia.com, joey.gouly@arm.com, ardb@kernel.org, thuth@redhat.com, pasha.tatashin@soleen.com, kristina.martsenko@arm.com, bigeasy@linutronix.de, maciej.wieczor-retman@intel.com, lorenzo.stoakes@oracle.com, jason.andryuk@amd.com, david@redhat.com, graf@amazon.com, wangkefeng.wang@huawei.com, ziy@nvidia.com, mark.rutland@arm.com, dave.hansen@linux.intel.com, samuel.holland@sifive.com, kbingham@kernel.org, trintaeoitogc@gmail.com, scott@os.amperecomputing.com, justinstitt@google.com, kuan-ying.lee@canonical.com, maz@kernel.org, tglx@linutronix.de, samitolvanen@google.com, mhocko@suse.com, nunodasneves@linux.microsoft.com, brgerst@gmail.com, willy@infradead.org, ubizjak@gmail.com, peterz@infradead.org, mingo@redhat.com, sohil.mehta@intel.com Cc: linux-mm@kvack.org, linux-kbuild@vger.kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org, llvm@lists.linux.dev, kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 03/18] kasan: Fix inline mode for x86 tag-based mode Date: Tue, 12 Aug 2025 15:23:39 +0200 Message-ID: <0ee6ca89ff7617fe7a4eda63ef5cf376d609ccdd.1755004923.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The LLVM compiler uses hwasan-instrument-with-calls parameter to setup inline or outline mode in tag-based KASAN. If zeroed, it means the instrumentation implementation will be pasted into each relevant location along with KASAN related constants during compilation. If set to one all function instrumentation will be done with function calls instead. The default hwasan-instrument-with-calls value for the x86 architecture in the compiler is "1", which is not true for other architectures. Because of this, enabling inline mode in software tag-based KASAN doesn't work on x86 as the kernel script doesn't zero out the parameter and always sets up the outline mode. Explicitly zero out hwasan-instrument-with-calls when enabling inline mode in tag-based KASAN. Signed-off-by: Maciej Wieczor-Retman --- Changelog v3: - Add this patch to the series. scripts/Makefile.kasan | 3 +++ 1 file changed, 3 insertions(+) diff --git a/scripts/Makefile.kasan b/scripts/Makefile.kasan index 693dbbebebba..2c7be96727ac 100644 --- a/scripts/Makefile.kasan +++ b/scripts/Makefile.kasan @@ -76,8 +76,11 @@ CFLAGS_KASAN :=3D -fsanitize=3Dkernel-hwaddress RUSTFLAGS_KASAN :=3D -Zsanitizer=3Dkernel-hwaddress \ -Zsanitizer-recover=3Dkernel-hwaddress =20 +# LLVM sets hwasan-instrument-with-calls to 1 on x86 by default. Set it to= 0 +# when inline mode is enabled. ifdef CONFIG_KASAN_INLINE kasan_params +=3D hwasan-mapping-offset=3D$(KASAN_SHADOW_OFFSET) + kasan_params +=3D hwasan-instrument-with-calls=3D0 else kasan_params +=3D hwasan-instrument-with-calls=3D1 endif --=20 2.50.1 From nobody Sat Oct 4 20:54:34 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 18FF4281531; Tue, 12 Aug 2025 13:27:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005221; cv=none; b=mIsqewJXZ+ZQDSkSCwtvsc+UJ1rdZ3BS1MHu1UUnJUTNn+FwMTRkUtPQOo2/LPCQwa1aWfhKpGKD4jPcPEZFLN7TUq/5CkRSAtxUggEoOiJhTKvp7MblUYa3SKlI6HFwN5Jv/3AH5nX5dcnDrNNFYkEv/2T4iJW2bZg9MFCZ+qo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005221; c=relaxed/simple; bh=umSIEr/jpouxzkGHeJJ2QvhqsNKgw+HWF35dK6bLmas=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=aBIbSt1k8hhQQ/GMDekFwXpu9RKdEeFNuannKEYV3hgHOI4Z1Lx+3O9yrqa2Ofpmw+E4bhX8qwr+fwexAaZ70upc+uv4jMC0MQXq5ugZQjV0SJRz/cnytJGELm9QcO8AuUOq7++4kYaKgKcefVE1aiUoViro3uW40tol7BvCbPE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=gkO1hVUT; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="gkO1hVUT" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1755005221; x=1786541221; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=umSIEr/jpouxzkGHeJJ2QvhqsNKgw+HWF35dK6bLmas=; b=gkO1hVUTfBgcsoGnDeJZ9CuedMz7RoOXXlO/RN5kMeamtYKfc2JIpf5y km+qtLjPtOMmLapadj7b947nc7FJDX+vkAJvPT3WGGC0fjv1TSY+EklMy FVgHcfKyG0j6FwlVkzvF6q+d7BARouoCsgnqFfvg9eBR16ZDCgQtIchii t4qOBibwBYHFlbK0KfFZAQyX82xytc0qhfggtWcR0w7ZPWZ7Qo3NKKn2Q NxCmTDhJnXN+a0QKLVrJctGqjYkCEFyhmqbvrlOLxUOvpTc6d/WQOuirt 6/LvWYTkdDEfzNuYmwKaQ9lsb8+rsdAZZRO+ZPikm1LtLbI95RYpf5Eke A==; X-CSE-ConnectionGUID: iop5kIC2TJWYPPl+KZtxpw== X-CSE-MsgGUID: NmivGVN5SWmtTz9dg05OuQ== X-IronPort-AV: E=McAfee;i="6800,10657,11520"; a="60903350" X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="60903350" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:26:33 -0700 X-CSE-ConnectionGUID: ZdcdAsOcQWWTFu4OTS0rRA== X-CSE-MsgGUID: cCyGXKgLSeWES9FNw89vig== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="165831396" Received: from vpanait-mobl.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.245.54]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:26:09 -0700 From: Maciej Wieczor-Retman To: nathan@kernel.org, arnd@arndb.de, broonie@kernel.org, Liam.Howlett@oracle.com, urezki@gmail.com, will@kernel.org, kaleshsingh@google.com, rppt@kernel.org, leitao@debian.org, coxu@redhat.com, surenb@google.com, akpm@linux-foundation.org, luto@kernel.org, jpoimboe@kernel.org, changyuanl@google.com, hpa@zytor.com, dvyukov@google.com, kas@kernel.org, corbet@lwn.net, vincenzo.frascino@arm.com, smostafa@google.com, nick.desaulniers+lkml@gmail.com, morbo@google.com, andreyknvl@gmail.com, alexander.shishkin@linux.intel.com, thiago.bauermann@linaro.org, catalin.marinas@arm.com, ryabinin.a.a@gmail.com, jan.kiszka@siemens.com, jbohac@suse.cz, dan.j.williams@intel.com, joel.granados@kernel.org, baohua@kernel.org, kevin.brodsky@arm.com, nicolas.schier@linux.dev, pcc@google.com, andriy.shevchenko@linux.intel.com, wei.liu@kernel.org, bp@alien8.de, ada.coupriediaz@arm.com, xin@zytor.com, pankaj.gupta@amd.com, vbabka@suse.cz, glider@google.com, jgross@suse.com, kees@kernel.org, jhubbard@nvidia.com, joey.gouly@arm.com, ardb@kernel.org, thuth@redhat.com, pasha.tatashin@soleen.com, kristina.martsenko@arm.com, bigeasy@linutronix.de, maciej.wieczor-retman@intel.com, lorenzo.stoakes@oracle.com, jason.andryuk@amd.com, david@redhat.com, graf@amazon.com, wangkefeng.wang@huawei.com, ziy@nvidia.com, mark.rutland@arm.com, dave.hansen@linux.intel.com, samuel.holland@sifive.com, kbingham@kernel.org, trintaeoitogc@gmail.com, scott@os.amperecomputing.com, justinstitt@google.com, kuan-ying.lee@canonical.com, maz@kernel.org, tglx@linutronix.de, samitolvanen@google.com, mhocko@suse.com, nunodasneves@linux.microsoft.com, brgerst@gmail.com, willy@infradead.org, ubizjak@gmail.com, peterz@infradead.org, mingo@redhat.com, sohil.mehta@intel.com Cc: linux-mm@kvack.org, linux-kbuild@vger.kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org, llvm@lists.linux.dev, kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 04/18] x86: Add arch specific kasan functions Date: Tue, 12 Aug 2025 15:23:40 +0200 Message-ID: X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" KASAN's software tag-based mode needs multiple macros/functions to handle tag and pointer interactions - to set, retrieve and reset tags from the top bits of a pointer. Mimic functions currently used by arm64 but change the tag's position to bits [60:57] in the pointer. Signed-off-by: Maciej Wieczor-Retman --- Changelog v4: - Rewrite __tag_set() without pointless casts and make it more readable. Changelog v3: - Reorder functions so that __tag_*() etc are above the arch_kasan_*() ones. - Remove CONFIG_KASAN condition from __tag_set() arch/x86/include/asm/kasan.h | 36 ++++++++++++++++++++++++++++++++++-- 1 file changed, 34 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/kasan.h b/arch/x86/include/asm/kasan.h index d7e33c7f096b..1963eb2fcff3 100644 --- a/arch/x86/include/asm/kasan.h +++ b/arch/x86/include/asm/kasan.h @@ -3,6 +3,8 @@ #define _ASM_X86_KASAN_H =20 #include +#include +#include #define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) #define KASAN_SHADOW_SCALE_SHIFT 3 =20 @@ -24,8 +26,37 @@ KASAN_SHADOW_SCALE_SHIFT))) =20 #ifndef __ASSEMBLER__ +#include +#include +#include + +#ifdef CONFIG_KASAN_SW_TAGS + +#define __tag_shifted(tag) FIELD_PREP(GENMASK_ULL(60, 57), tag) +#define __tag_reset(addr) (sign_extend64((u64)(addr), 56)) +#define __tag_get(addr) ((u8)FIELD_GET(GENMASK_ULL(60, 57), (u64)addr)) +#else +#define __tag_shifted(tag) 0UL +#define __tag_reset(addr) (addr) +#define __tag_get(addr) 0 +#endif /* CONFIG_KASAN_SW_TAGS */ + +static inline void *__tag_set(const void *__addr, u8 tag) +{ + u64 addr =3D (u64)__addr; + + addr &=3D ~__tag_shifted(KASAN_TAG_MASK); + addr |=3D __tag_shifted(tag); + + return (void *)addr; +} + +#define arch_kasan_set_tag(addr, tag) __tag_set(addr, tag) +#define arch_kasan_reset_tag(addr) __tag_reset(addr) +#define arch_kasan_get_tag(addr) __tag_get(addr) =20 #ifdef CONFIG_KASAN + void __init kasan_early_init(void); void __init kasan_init(void); void __init kasan_populate_shadow_for_vaddr(void *va, size_t size, int nid= ); @@ -34,8 +65,9 @@ static inline void kasan_early_init(void) { } static inline void kasan_init(void) { } static inline void kasan_populate_shadow_for_vaddr(void *va, size_t size, int nid) { } -#endif =20 -#endif +#endif /* CONFIG_KASAN */ + +#endif /* __ASSEMBLER__ */ =20 #endif --=20 2.50.1 From nobody Sat Oct 4 20:54:34 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B6AD928314E; Tue, 12 Aug 2025 13:27:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005224; cv=none; b=jbqX0UVFoQ7AZpO2kUdFdsqVXlqcmuSfySGSb1fZNc02IgDYTGARMCY/Pemc9jS0apOy7mk8kwmhCr6KYXr4K0pJGFDawja2ZSriZ4z0mz9vKo1n2iVOQCTlMSw2RGu2N4xIlJqG0D570BKYA6A72tefpR1ROcRXN34LpUceIcQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005224; c=relaxed/simple; bh=mZLJdH3rbK1bxCuEzF7KjKtbxlAVDYknI9S994S9uz4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=GDxBOt2MdCg4TOuncA95XQAEjN2DRMOS+RfuBnBEBjlBg5rm2rBGAD/OBkbBBN+F1d8qAcSaGuqyUfGIMe4HP3mMmCaOGkXakoWc6k7n4w/kefbD456TAtq7Z1wDJftN2OHsnjxfytILipnOW3oNgFJgImM9T/qpXcB1rynMfcM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=RiM0QO5D; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="RiM0QO5D" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1755005223; x=1786541223; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=mZLJdH3rbK1bxCuEzF7KjKtbxlAVDYknI9S994S9uz4=; b=RiM0QO5DHaJjWTHT1DZVDOV2cfv4S+IQ+/gUEY1Doph15f0QsQOcfcnH ZMJT1jJ2yFyYQngLoQuj/RpdnC28eEq+CIpALICg75sPPOufkUX7oPEmH 2tzJFlHw4awBCfZyX3qkpqR1ofQZRJjynhzPsc9ATAWjjuvPcQ1YBiYEV K0ThuYgD+2c3npyOcVQPNTum2iQffHydDWwdh/9oiUqplsexzTWCYnjau AOOubnaBT2+6R18ZMAkstsWuTcjtonvs9D4hAOrXPZbjTKpPtnht7OwfY lVMKYhOCc2+sLZCxByRvQX5w5+oKIF7dwHab8sLQgiZXy4+y+vlJWwp9K Q==; X-CSE-ConnectionGUID: gQVy1ST9Sz+NZku6AGzNWw== X-CSE-MsgGUID: CI9BGR9YS+ahIdC60O4eBg== X-IronPort-AV: E=McAfee;i="6800,10657,11520"; a="60903400" X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="60903400" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:26:55 -0700 X-CSE-ConnectionGUID: 0gOP6hCJQl2mMyP4g+OHwQ== X-CSE-MsgGUID: 99luYSpdRRigAj3rcRdRgg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="165831406" Received: from vpanait-mobl.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.245.54]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:26:32 -0700 From: Maciej Wieczor-Retman To: nathan@kernel.org, arnd@arndb.de, broonie@kernel.org, Liam.Howlett@oracle.com, urezki@gmail.com, will@kernel.org, kaleshsingh@google.com, rppt@kernel.org, leitao@debian.org, coxu@redhat.com, surenb@google.com, akpm@linux-foundation.org, luto@kernel.org, jpoimboe@kernel.org, changyuanl@google.com, hpa@zytor.com, dvyukov@google.com, kas@kernel.org, corbet@lwn.net, vincenzo.frascino@arm.com, smostafa@google.com, nick.desaulniers+lkml@gmail.com, morbo@google.com, andreyknvl@gmail.com, alexander.shishkin@linux.intel.com, thiago.bauermann@linaro.org, catalin.marinas@arm.com, ryabinin.a.a@gmail.com, jan.kiszka@siemens.com, jbohac@suse.cz, dan.j.williams@intel.com, joel.granados@kernel.org, baohua@kernel.org, kevin.brodsky@arm.com, nicolas.schier@linux.dev, pcc@google.com, andriy.shevchenko@linux.intel.com, wei.liu@kernel.org, bp@alien8.de, ada.coupriediaz@arm.com, xin@zytor.com, pankaj.gupta@amd.com, vbabka@suse.cz, glider@google.com, jgross@suse.com, kees@kernel.org, jhubbard@nvidia.com, joey.gouly@arm.com, ardb@kernel.org, thuth@redhat.com, pasha.tatashin@soleen.com, kristina.martsenko@arm.com, bigeasy@linutronix.de, maciej.wieczor-retman@intel.com, lorenzo.stoakes@oracle.com, jason.andryuk@amd.com, david@redhat.com, graf@amazon.com, wangkefeng.wang@huawei.com, ziy@nvidia.com, mark.rutland@arm.com, dave.hansen@linux.intel.com, samuel.holland@sifive.com, kbingham@kernel.org, trintaeoitogc@gmail.com, scott@os.amperecomputing.com, justinstitt@google.com, kuan-ying.lee@canonical.com, maz@kernel.org, tglx@linutronix.de, samitolvanen@google.com, mhocko@suse.com, nunodasneves@linux.microsoft.com, brgerst@gmail.com, willy@infradead.org, ubizjak@gmail.com, peterz@infradead.org, mingo@redhat.com, sohil.mehta@intel.com Cc: linux-mm@kvack.org, linux-kbuild@vger.kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org, llvm@lists.linux.dev, kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 05/18] kasan: arm64: x86: Make special tags arch specific Date: Tue, 12 Aug 2025 15:23:41 +0200 Message-ID: <788b6f9a1f5adf2b69564489d16de436ba1bdc3c.1755004923.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" KASAN's tag-based mode defines multiple special tag values. They're reserved for: - Native kernel value. On arm64 it's 0xFF and it causes an early return in the tag checking function. - Invalid value. 0xFE marks an area as freed / unallocated. It's also the value that is used to initialize regions of shadow memory. - Max value. 0xFD is the highest value that can be randomly generated for a new tag. Metadata macro is also defined: - Tag width equal to 8. Tag-based mode on x86 is going to use 4 bit wide tags so all the above values need to be changed accordingly. Make native kernel tag arch specific for x86 and arm64. Replace hardcoded kernel tag value and tag width with macros in KASAN's non-arch specific code. Signed-off-by: Maciej Wieczor-Retman --- Changelog v4: - Move KASAN_TAG_MASK to kasan-tags.h. Changelog v2: - Remove risc-v from the patch. MAINTAINERS | 2 +- arch/arm64/include/asm/kasan-tags.h | 9 +++++++++ arch/x86/include/asm/kasan-tags.h | 9 +++++++++ include/linux/kasan-tags.h | 10 +++++++++- include/linux/kasan.h | 4 +++- include/linux/mm.h | 6 +++--- include/linux/mmzone.h | 1 - include/linux/page-flags-layout.h | 9 +-------- 8 files changed, 35 insertions(+), 15 deletions(-) create mode 100644 arch/arm64/include/asm/kasan-tags.h create mode 100644 arch/x86/include/asm/kasan-tags.h diff --git a/MAINTAINERS b/MAINTAINERS index fe168477caa4..7ce8c6b86e3d 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -13166,7 +13166,7 @@ L: kasan-dev@googlegroups.com S: Maintained B: https://bugzilla.kernel.org/buglist.cgi?component=3DSanitizers&product= =3DMemory%20Management F: Documentation/dev-tools/kasan.rst -F: arch/*/include/asm/*kasan.h +F: arch/*/include/asm/*kasan*.h F: arch/*/mm/kasan_init* F: include/linux/kasan*.h F: lib/Kconfig.kasan diff --git a/arch/arm64/include/asm/kasan-tags.h b/arch/arm64/include/asm/k= asan-tags.h new file mode 100644 index 000000000000..8cb12ebae57f --- /dev/null +++ b/arch/arm64/include/asm/kasan-tags.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_KASAN_TAGS_H +#define __ASM_KASAN_TAGS_H + +#define KASAN_TAG_KERNEL 0xFF /* native kernel pointers tag */ + +#define KASAN_TAG_WIDTH 8 + +#endif /* ASM_KASAN_TAGS_H */ diff --git a/arch/x86/include/asm/kasan-tags.h b/arch/x86/include/asm/kasan= -tags.h new file mode 100644 index 000000000000..68ba385bc75c --- /dev/null +++ b/arch/x86/include/asm/kasan-tags.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_KASAN_TAGS_H +#define __ASM_KASAN_TAGS_H + +#define KASAN_TAG_KERNEL 0xF /* native kernel pointers tag */ + +#define KASAN_TAG_WIDTH 4 + +#endif /* ASM_KASAN_TAGS_H */ diff --git a/include/linux/kasan-tags.h b/include/linux/kasan-tags.h index e07c896f95d3..fe80fa8f3315 100644 --- a/include/linux/kasan-tags.h +++ b/include/linux/kasan-tags.h @@ -2,7 +2,15 @@ #ifndef _LINUX_KASAN_TAGS_H #define _LINUX_KASAN_TAGS_H =20 -#include +#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) +#include +#endif + +#ifndef KASAN_TAG_WIDTH +#define KASAN_TAG_WIDTH 0 +#endif + +#define KASAN_TAG_MASK ((1UL << KASAN_TAG_WIDTH) - 1) =20 #ifndef KASAN_TAG_KERNEL #define KASAN_TAG_KERNEL 0xFF /* native kernel pointers tag */ diff --git a/include/linux/kasan.h b/include/linux/kasan.h index b396feca714f..54481f8c30c5 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -40,7 +40,9 @@ typedef unsigned int __bitwise kasan_vmalloc_flags_t; =20 #ifdef CONFIG_KASAN_SW_TAGS /* This matches KASAN_TAG_INVALID. */ -#define KASAN_SHADOW_INIT 0xFE +#ifndef KASAN_SHADOW_INIT +#define KASAN_SHADOW_INIT KASAN_TAG_INVALID +#endif #else #define KASAN_SHADOW_INIT 0 #endif diff --git a/include/linux/mm.h b/include/linux/mm.h index 1ae97a0b8ec7..bb494cb1d5af 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1692,7 +1692,7 @@ static inline u8 page_kasan_tag(const struct page *pa= ge) =20 if (kasan_enabled()) { tag =3D (page->flags >> KASAN_TAG_PGSHIFT) & KASAN_TAG_MASK; - tag ^=3D 0xff; + tag ^=3D KASAN_TAG_KERNEL; } =20 return tag; @@ -1705,7 +1705,7 @@ static inline void page_kasan_tag_set(struct page *pa= ge, u8 tag) if (!kasan_enabled()) return; =20 - tag ^=3D 0xff; + tag ^=3D KASAN_TAG_KERNEL; old_flags =3D READ_ONCE(page->flags); do { flags =3D old_flags; @@ -1724,7 +1724,7 @@ static inline void page_kasan_tag_reset(struct page *= page) =20 static inline u8 page_kasan_tag(const struct page *page) { - return 0xff; + return KASAN_TAG_KERNEL; } =20 static inline void page_kasan_tag_set(struct page *page, u8 tag) { } diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 0c5da9141983..c139fb3d862d 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -1166,7 +1166,6 @@ static inline bool zone_is_empty(struct zone *zone) #define NODES_MASK ((1UL << NODES_WIDTH) - 1) #define SECTIONS_MASK ((1UL << SECTIONS_WIDTH) - 1) #define LAST_CPUPID_MASK ((1UL << LAST_CPUPID_SHIFT) - 1) -#define KASAN_TAG_MASK ((1UL << KASAN_TAG_WIDTH) - 1) #define ZONEID_MASK ((1UL << ZONEID_SHIFT) - 1) =20 static inline enum zone_type page_zonenum(const struct page *page) diff --git a/include/linux/page-flags-layout.h b/include/linux/page-flags-l= ayout.h index 760006b1c480..b2cc4cb870e0 100644 --- a/include/linux/page-flags-layout.h +++ b/include/linux/page-flags-layout.h @@ -3,6 +3,7 @@ #define PAGE_FLAGS_LAYOUT_H =20 #include +#include #include =20 /* @@ -72,14 +73,6 @@ #define NODE_NOT_IN_PAGE_FLAGS 1 #endif =20 -#if defined(CONFIG_KASAN_SW_TAGS) -#define KASAN_TAG_WIDTH 8 -#elif defined(CONFIG_KASAN_HW_TAGS) -#define KASAN_TAG_WIDTH 4 -#else -#define KASAN_TAG_WIDTH 0 -#endif - #ifdef CONFIG_NUMA_BALANCING #define LAST__PID_SHIFT 8 #define LAST__PID_MASK ((1 << LAST__PID_SHIFT)-1) --=20 2.50.1 From nobody Sat Oct 4 20:54:34 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 18A2A283FE2; Tue, 12 Aug 2025 13:27:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005238; cv=none; b=QPl45t7dWljNzCyHN8wRjCCYW/Fw106vseTaIP4s6uI6zsIZAGnI9AUTvW7PBB2U0qXM6kHVEXDCA17dyYG1T3kZQdejpAQe5jkcqBvNMoeGxuAq7E/fM0dcF6DSgV3xVdlG2PswMSn01yQHAoxXvb/OJm8LuZEEoTNA10h5FEw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005238; c=relaxed/simple; bh=a8brDXoNQXGsX0O3+U1wqzz6qH0YVwQ2IYUmoUvYEIA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=seKnx5QjuSScHAEiOOp8o101VEXICP4vDicbdtYMlyxto+cY/LT2GL0DEu7BP41rrSApcUDPzisfOc2Itx5E/7LpSCbqSrfFmt3r0um2og4zmTaZKw4pqSlOXDrN+QOSXPbAa1McwewUaKfWo0OtLJwWTzzU143vvHOoxamhPwE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Fo0Vm+Cg; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Fo0Vm+Cg" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1755005238; x=1786541238; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=a8brDXoNQXGsX0O3+U1wqzz6qH0YVwQ2IYUmoUvYEIA=; b=Fo0Vm+Cgv2xRxiSOTgYYPQfWq5G2niXASlxfNuFP0Jk35f8B13CnHJVp chtRalOYmYbcJ3wPD+hEDoeuH8fOAYZ/D5EKQz3ufU1S3nySYIUN3kaf8 5xFdY09ItYwfqyMLGciDkX9cVwuvtK4B9+ZtGyN9DgEdE1QHOxck1cGxU mh0h67u2/6vzqVo8zwnW4hYyA8zTCBTencL+9sH15nZwjxqgzu2XNOy44 F7WDDQCS+eKjGx4eAQOPpHu7M27ZgkbJ88KPVvqdHYAiCOr+8ikXa5O4O at+TXbkjYm48sC/4S60SP5GB80Kcr4cT+4BGnzJQPENljXTdtGgovVu+1 w==; X-CSE-ConnectionGUID: +ETZIlvCRfqUYgJ7i6N2HQ== X-CSE-MsgGUID: CWJv2eUgShCjJS70xinZkw== X-IronPort-AV: E=McAfee;i="6800,10657,11520"; a="60903476" X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="60903476" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:27:17 -0700 X-CSE-ConnectionGUID: wC4pRigPTk65sPn+FhCwlg== X-CSE-MsgGUID: 65CUt4VoQ8Opy1+rKR26tg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="165831425" Received: from vpanait-mobl.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.245.54]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:26:55 -0700 From: Maciej Wieczor-Retman To: nathan@kernel.org, arnd@arndb.de, broonie@kernel.org, Liam.Howlett@oracle.com, urezki@gmail.com, will@kernel.org, kaleshsingh@google.com, rppt@kernel.org, leitao@debian.org, coxu@redhat.com, surenb@google.com, akpm@linux-foundation.org, luto@kernel.org, jpoimboe@kernel.org, changyuanl@google.com, hpa@zytor.com, dvyukov@google.com, kas@kernel.org, corbet@lwn.net, vincenzo.frascino@arm.com, smostafa@google.com, nick.desaulniers+lkml@gmail.com, morbo@google.com, andreyknvl@gmail.com, alexander.shishkin@linux.intel.com, thiago.bauermann@linaro.org, catalin.marinas@arm.com, ryabinin.a.a@gmail.com, jan.kiszka@siemens.com, jbohac@suse.cz, dan.j.williams@intel.com, joel.granados@kernel.org, baohua@kernel.org, kevin.brodsky@arm.com, nicolas.schier@linux.dev, pcc@google.com, andriy.shevchenko@linux.intel.com, wei.liu@kernel.org, bp@alien8.de, ada.coupriediaz@arm.com, xin@zytor.com, pankaj.gupta@amd.com, vbabka@suse.cz, glider@google.com, jgross@suse.com, kees@kernel.org, jhubbard@nvidia.com, joey.gouly@arm.com, ardb@kernel.org, thuth@redhat.com, pasha.tatashin@soleen.com, kristina.martsenko@arm.com, bigeasy@linutronix.de, maciej.wieczor-retman@intel.com, lorenzo.stoakes@oracle.com, jason.andryuk@amd.com, david@redhat.com, graf@amazon.com, wangkefeng.wang@huawei.com, ziy@nvidia.com, mark.rutland@arm.com, dave.hansen@linux.intel.com, samuel.holland@sifive.com, kbingham@kernel.org, trintaeoitogc@gmail.com, scott@os.amperecomputing.com, justinstitt@google.com, kuan-ying.lee@canonical.com, maz@kernel.org, tglx@linutronix.de, samitolvanen@google.com, mhocko@suse.com, nunodasneves@linux.microsoft.com, brgerst@gmail.com, willy@infradead.org, ubizjak@gmail.com, peterz@infradead.org, mingo@redhat.com, sohil.mehta@intel.com Cc: linux-mm@kvack.org, linux-kbuild@vger.kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org, llvm@lists.linux.dev, kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 06/18] x86: Reset tag for virtual to physical address conversions Date: Tue, 12 Aug 2025 15:23:42 +0200 Message-ID: <01e62233dcc39aeb8d640eb3ee794f5da533f2a3.1755004923.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Any place where pointer arithmetic is used to convert a virtual address into a physical one can raise errors if the virtual address is tagged. Reset the pointer's tag by sign extending the tag bits in macros that do pointer arithmetic in address conversions. There will be no change in compiled code with KASAN disabled since the compiler will optimize the __tag_reset() out. Signed-off-by: Maciej Wieczor-Retman --- Changelog v4: - Simplify page_to_virt() by removing pointless casts. - Remove change in __is_canonical_address() because it's taken care of in a later patch due to a LAM compatible definition of canonical. arch/x86/include/asm/page.h | 14 +++++++++++--- arch/x86/include/asm/page_64.h | 2 +- arch/x86/mm/physaddr.c | 1 + 3 files changed, 13 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/page.h b/arch/x86/include/asm/page.h index 9265f2fca99a..15c95e96fd15 100644 --- a/arch/x86/include/asm/page.h +++ b/arch/x86/include/asm/page.h @@ -7,6 +7,7 @@ #ifdef __KERNEL__ =20 #include +#include =20 #ifdef CONFIG_X86_64 #include @@ -41,7 +42,7 @@ static inline void copy_user_page(void *to, void *from, u= nsigned long vaddr, #define __pa(x) __phys_addr((unsigned long)(x)) #endif =20 -#define __pa_nodebug(x) __phys_addr_nodebug((unsigned long)(x)) +#define __pa_nodebug(x) __phys_addr_nodebug((unsigned long)(__tag_reset(x)= )) /* __pa_symbol should be used for C visible symbols. This seems to be the official gcc blessed way to do such arithmetic. */ /* @@ -65,9 +66,16 @@ static inline void copy_user_page(void *to, void *from, = unsigned long vaddr, * virt_to_page(kaddr) returns a valid pointer if and only if * virt_addr_valid(kaddr) returns true. */ -#define virt_to_page(kaddr) pfn_to_page(__pa(kaddr) >> PAGE_SHIFT) + +#ifdef CONFIG_KASAN_SW_TAGS +#define page_to_virt(x) ({ \ + void *__addr =3D __va(page_to_pfn((struct page *)x) << PAGE_SHIFT); \ + __tag_set(__addr, page_kasan_tag(x)); \ +}) +#endif +#define virt_to_page(kaddr) pfn_to_page(__pa((void *)__tag_reset(kaddr)) >= > PAGE_SHIFT) extern bool __virt_addr_valid(unsigned long kaddr); -#define virt_addr_valid(kaddr) __virt_addr_valid((unsigned long) (kaddr)) +#define virt_addr_valid(kaddr) __virt_addr_valid((unsigned long)(__tag_res= et(kaddr))) =20 static __always_inline void *pfn_to_kaddr(unsigned long pfn) { diff --git a/arch/x86/include/asm/page_64.h b/arch/x86/include/asm/page_64.h index 015d23f3e01f..de68ac40dba2 100644 --- a/arch/x86/include/asm/page_64.h +++ b/arch/x86/include/asm/page_64.h @@ -33,7 +33,7 @@ static __always_inline unsigned long __phys_addr_nodebug(= unsigned long x) extern unsigned long __phys_addr(unsigned long); extern unsigned long __phys_addr_symbol(unsigned long); #else -#define __phys_addr(x) __phys_addr_nodebug(x) +#define __phys_addr(x) __phys_addr_nodebug(__tag_reset(x)) #define __phys_addr_symbol(x) \ ((unsigned long)(x) - __START_KERNEL_map + phys_base) #endif diff --git a/arch/x86/mm/physaddr.c b/arch/x86/mm/physaddr.c index fc3f3d3e2ef2..7f2b11308245 100644 --- a/arch/x86/mm/physaddr.c +++ b/arch/x86/mm/physaddr.c @@ -14,6 +14,7 @@ #ifdef CONFIG_DEBUG_VIRTUAL unsigned long __phys_addr(unsigned long x) { + x =3D __tag_reset(x); unsigned long y =3D x - __START_KERNEL_map; =20 /* use the carry flag to determine if x was < __START_KERNEL_map */ --=20 2.50.1 From nobody Sat Oct 4 20:54:34 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 12F7828153D; Tue, 12 Aug 2025 13:27:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005260; cv=none; b=V3Flz1CQZM+a1tXNtgN/T89+d2ucDID4sfPHTBC0dBYZSWOaUwqWw0Y4ihxwSpwB535YpuBwJwsOjnUdlPST2VpU2JtpwDAIetcsBLtWfsb6GfwN8XPZ6ugE3pU4es3n5X2QqQvHhbP1RN76AwCXj9pY2WAfTLeEmchM/U8UFik= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005260; c=relaxed/simple; bh=WytGKsbpVx+YjGfDckn4ztihfy/XnHxiwtMc2ueYMYI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=h+etjhFto/14X5RNk7wkoOglCNmIIK9Geh4M8PjZ/cxxgGHiEuVGBG1Ynabu6o6nsFiq/oBzzshsMFKF1kn8ZTpxIb9MmZ57CGYe6GaY/LRrw6nViwtjzoBQ84HCXsHxtnjNDccL3xmUrnRTA6VrwRfEsnaXkO8KKmvJrwLnGO8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=L+LGP85a; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="L+LGP85a" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1755005260; x=1786541260; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=WytGKsbpVx+YjGfDckn4ztihfy/XnHxiwtMc2ueYMYI=; b=L+LGP85alfWS2LanLsShitiahDbLftGu7ZecMK0HXqFNaSw3HO4Iip3I m6NwRizP9z4xuMZ+882UPYJJ03fNabxlypK/CRQBGgh6BnIeBRYCp/XCR 9t9q72HQBBHKv9Q3G9yma8IcF0DvCSQdevDDt6GaS3951q3p9LCQauh/W zDHWM/o0ZZX9LzHAjnWHLXPel1Bu10Anh1jk//fSfeMAdkBKlQz+YbYHC A8+vzyl9M+baH1yOdgwWKnhlndEw3meGpuGAir+0Pey2RITTIq+p00DiV IeGiYDpDxB3MRd3gUquvPDAAy5VmN8cl0sWrBU/N3PoBGytPH1A2SXR3g A==; X-CSE-ConnectionGUID: ear2osQsTqavaX3FvMQjnQ== X-CSE-MsgGUID: FZ9oKwwxQACeIwU0fg3RUg== X-IronPort-AV: E=McAfee;i="6800,10657,11520"; a="60903512" X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="60903512" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:27:38 -0700 X-CSE-ConnectionGUID: 7sr1bGkITqqOrRUhUWybYQ== X-CSE-MsgGUID: Oy2/bGKxRJGKw5AHGzSS+w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="165831445" Received: from vpanait-mobl.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.245.54]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:27:16 -0700 From: Maciej Wieczor-Retman To: nathan@kernel.org, arnd@arndb.de, broonie@kernel.org, Liam.Howlett@oracle.com, urezki@gmail.com, will@kernel.org, kaleshsingh@google.com, rppt@kernel.org, leitao@debian.org, coxu@redhat.com, surenb@google.com, akpm@linux-foundation.org, luto@kernel.org, jpoimboe@kernel.org, changyuanl@google.com, hpa@zytor.com, dvyukov@google.com, kas@kernel.org, corbet@lwn.net, vincenzo.frascino@arm.com, smostafa@google.com, nick.desaulniers+lkml@gmail.com, morbo@google.com, andreyknvl@gmail.com, alexander.shishkin@linux.intel.com, thiago.bauermann@linaro.org, catalin.marinas@arm.com, ryabinin.a.a@gmail.com, jan.kiszka@siemens.com, jbohac@suse.cz, dan.j.williams@intel.com, joel.granados@kernel.org, baohua@kernel.org, kevin.brodsky@arm.com, nicolas.schier@linux.dev, pcc@google.com, andriy.shevchenko@linux.intel.com, wei.liu@kernel.org, bp@alien8.de, ada.coupriediaz@arm.com, xin@zytor.com, pankaj.gupta@amd.com, vbabka@suse.cz, glider@google.com, jgross@suse.com, kees@kernel.org, jhubbard@nvidia.com, joey.gouly@arm.com, ardb@kernel.org, thuth@redhat.com, pasha.tatashin@soleen.com, kristina.martsenko@arm.com, bigeasy@linutronix.de, maciej.wieczor-retman@intel.com, lorenzo.stoakes@oracle.com, jason.andryuk@amd.com, david@redhat.com, graf@amazon.com, wangkefeng.wang@huawei.com, ziy@nvidia.com, mark.rutland@arm.com, dave.hansen@linux.intel.com, samuel.holland@sifive.com, kbingham@kernel.org, trintaeoitogc@gmail.com, scott@os.amperecomputing.com, justinstitt@google.com, kuan-ying.lee@canonical.com, maz@kernel.org, tglx@linutronix.de, samitolvanen@google.com, mhocko@suse.com, nunodasneves@linux.microsoft.com, brgerst@gmail.com, willy@infradead.org, ubizjak@gmail.com, peterz@infradead.org, mingo@redhat.com, sohil.mehta@intel.com Cc: linux-mm@kvack.org, linux-kbuild@vger.kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org, llvm@lists.linux.dev, kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 07/18] mm: x86: Untag addresses in EXECMEM_ROX related pointer arithmetic Date: Tue, 12 Aug 2025 15:23:43 +0200 Message-ID: X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" ARCH_HAS_EXECMEM_ROX was re-enabled in x86 at Linux 6.14 release. Related code has multiple spots where page virtual addresses end up used as arguments in arithmetic operations. Combined with enabled tag-based KASAN it can result in pointers that don't point where they should or logical operations not giving expected results. vm_reset_perms() calculates range's start and end addresses using min() and max() functions. To do that it compares pointers but some are not tagged - addr variable is, start and end variables aren't. within() and within_range() can receive tagged addresses which get compared to untagged start and end variables. Reset tags in addresses used as function arguments in min(), max(), within() and within_range(). execmem_cache_add() adds tagged pointers to a maple tree structure, which then are incorrectly compared when walking the tree. That results in different pointers being returned later and page permission violation errors panicking the kernel. Reset tag of the address range inserted into the maple tree inside execmem_cache_add(). Signed-off-by: Maciej Wieczor-Retman --- Changelog v4: - Add patch to the series. arch/x86/mm/pat/set_memory.c | 1 + mm/execmem.c | 4 +++- mm/vmalloc.c | 4 ++-- 3 files changed, 6 insertions(+), 3 deletions(-) diff --git a/arch/x86/mm/pat/set_memory.c b/arch/x86/mm/pat/set_memory.c index 8834c76f91c9..1f14a1297db0 100644 --- a/arch/x86/mm/pat/set_memory.c +++ b/arch/x86/mm/pat/set_memory.c @@ -222,6 +222,7 @@ static inline void cpa_inc_lp_preserved(int level) { } static inline int within(unsigned long addr, unsigned long start, unsigned long end) { + addr =3D (unsigned long)kasan_reset_tag((void *)addr); return addr >=3D start && addr < end; } =20 diff --git a/mm/execmem.c b/mm/execmem.c index 0822305413ec..743fa4a8c069 100644 --- a/mm/execmem.c +++ b/mm/execmem.c @@ -191,6 +191,8 @@ static int execmem_cache_add_locked(void *ptr, size_t s= ize, gfp_t gfp_mask) unsigned long lower, upper; void *area =3D NULL; =20 + addr =3D arch_kasan_reset_tag(addr); + lower =3D addr; upper =3D addr + size - 1; =20 @@ -216,7 +218,7 @@ static int execmem_cache_add(void *ptr, size_t size, gf= p_t gfp_mask) static bool within_range(struct execmem_range *range, struct ma_state *mas, size_t size) { - unsigned long addr =3D mas->index; + unsigned long addr =3D arch_kasan_reset_tag(mas->index); =20 if (addr >=3D range->start && addr + size < range->end) return true; diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 6dbcdceecae1..83d666e4837a 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -3328,8 +3328,8 @@ static void vm_reset_perms(struct vm_struct *area) unsigned long page_size; =20 page_size =3D PAGE_SIZE << page_order; - start =3D min(addr, start); - end =3D max(addr + page_size, end); + start =3D min((unsigned long)arch_kasan_reset_tag(addr), start); + end =3D max((unsigned long)arch_kasan_reset_tag(addr) + page_size, end); flush_dmap =3D 1; } } --=20 2.50.1 From nobody Sat Oct 4 20:54:34 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EA005280CF6; Tue, 12 Aug 2025 13:28:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005284; cv=none; b=dGrvBszyJqXwZ8Lnxz0eYzEzBNsYEseRMYRruE07i2shqdj6YsrZfq3+ftJ5AIfMiBlX05pRpEmcbHm8si7kkEdtUE4QtzCPRwyCbu/qEHoFmdHadBz6ZB8GizuslJZkZL4t5dTgQ/k10M53gH1Twqm+I582OJDp1JE0qrnnSoA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005284; c=relaxed/simple; bh=lO1wQ1uF80MsG8cgULvpACCbJsuEPvTrA5VlE9nVfaY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jm3z/7YyxqHhmioKpR1uC8iSkLIXq5i5R6fr4TpFrnELUUA08ippKwdDAtZmJ9ce9zf42LVt5kzBV2/x0RM53CpunOpg9uUJZV8LWgal8+mAt9gvox+ZY344M3jFDljtf/mA/AtzuDYQKEJsCYh/I20fNu8eXkxDC3IqHz4DQCc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=RWRSvSNu; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="RWRSvSNu" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1755005284; x=1786541284; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=lO1wQ1uF80MsG8cgULvpACCbJsuEPvTrA5VlE9nVfaY=; b=RWRSvSNux+R9BDMXkB5Tnu/1Y7qgOJqnGx6Hr+boLqoFPfRoC2f9dVae RSSi9kL3L8k/U5Dx5DDDQpNXuSDsLNtO6JKbieQFCFEq9WOxNRT5VAJwh atQYqheTeQuR/3zCmRaWthmbCN36hLAQmpWmapnyJ/GicmhC7QTBx1KvD PDLm7xrQDwQDwxMk1GXmxHKrjO/hW3cuAWzvBrdB8zdnXb/bmjSQemSAi KaJyk0Gb99qwO/ruUP4m2l+brDu0UPFN7APOe3S2e4aWM8qngHtUSFuRf ktR/1zvdvH63e/NFmnFSjC3ThuUCa8cdSNnkmSucc8x5g/dJ4Ie0dXaM7 A==; X-CSE-ConnectionGUID: pLeCkwtTTz2ppOKLxCTlWg== X-CSE-MsgGUID: +HrhbGABR0uQ5KNwbOEsjQ== X-IronPort-AV: E=McAfee;i="6800,10657,11520"; a="60903567" X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="60903567" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:28:02 -0700 X-CSE-ConnectionGUID: 63q+EOftQWWQMSotLrPagg== X-CSE-MsgGUID: fl7/gRLpTkieCVLNXAAdgQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="165831460" Received: from vpanait-mobl.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.245.54]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:27:38 -0700 From: Maciej Wieczor-Retman To: nathan@kernel.org, arnd@arndb.de, broonie@kernel.org, Liam.Howlett@oracle.com, urezki@gmail.com, will@kernel.org, kaleshsingh@google.com, rppt@kernel.org, leitao@debian.org, coxu@redhat.com, surenb@google.com, akpm@linux-foundation.org, luto@kernel.org, jpoimboe@kernel.org, changyuanl@google.com, hpa@zytor.com, dvyukov@google.com, kas@kernel.org, corbet@lwn.net, vincenzo.frascino@arm.com, smostafa@google.com, nick.desaulniers+lkml@gmail.com, morbo@google.com, andreyknvl@gmail.com, alexander.shishkin@linux.intel.com, thiago.bauermann@linaro.org, catalin.marinas@arm.com, ryabinin.a.a@gmail.com, jan.kiszka@siemens.com, jbohac@suse.cz, dan.j.williams@intel.com, joel.granados@kernel.org, baohua@kernel.org, kevin.brodsky@arm.com, nicolas.schier@linux.dev, pcc@google.com, andriy.shevchenko@linux.intel.com, wei.liu@kernel.org, bp@alien8.de, ada.coupriediaz@arm.com, xin@zytor.com, pankaj.gupta@amd.com, vbabka@suse.cz, glider@google.com, jgross@suse.com, kees@kernel.org, jhubbard@nvidia.com, joey.gouly@arm.com, ardb@kernel.org, thuth@redhat.com, pasha.tatashin@soleen.com, kristina.martsenko@arm.com, bigeasy@linutronix.de, maciej.wieczor-retman@intel.com, lorenzo.stoakes@oracle.com, jason.andryuk@amd.com, david@redhat.com, graf@amazon.com, wangkefeng.wang@huawei.com, ziy@nvidia.com, mark.rutland@arm.com, dave.hansen@linux.intel.com, samuel.holland@sifive.com, kbingham@kernel.org, trintaeoitogc@gmail.com, scott@os.amperecomputing.com, justinstitt@google.com, kuan-ying.lee@canonical.com, maz@kernel.org, tglx@linutronix.de, samitolvanen@google.com, mhocko@suse.com, nunodasneves@linux.microsoft.com, brgerst@gmail.com, willy@infradead.org, ubizjak@gmail.com, peterz@infradead.org, mingo@redhat.com, sohil.mehta@intel.com Cc: linux-mm@kvack.org, linux-kbuild@vger.kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org, llvm@lists.linux.dev, kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 08/18] x86: Physical address comparisons in fill_p*d/pte Date: Tue, 12 Aug 2025 15:23:44 +0200 Message-ID: X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Calculating page offset returns a pointer without a tag. When comparing the calculated offset to a tagged page pointer an error is raised because they are not equal. Change pointer comparisons to physical address comparisons as to avoid issues with tagged pointers that pointer arithmetic would create. Open code pte_offset_kernel(), pmd_offset(), pud_offset() and p4d_offset(). Because one parameter is always zero and the rest of the function insides are enclosed inside __va(), removing that layer lowers the complexity of final assembly. Signed-off-by: Maciej Wieczor-Retman --- Changelog v2: - Open code *_offset() to avoid it's internal __va(). arch/x86/mm/init_64.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index 76e33bd7c556..51a247e258b1 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -251,7 +251,10 @@ static p4d_t *fill_p4d(pgd_t *pgd, unsigned long vaddr) if (pgd_none(*pgd)) { p4d_t *p4d =3D (p4d_t *)spp_getpage(); pgd_populate(&init_mm, pgd, p4d); - if (p4d !=3D p4d_offset(pgd, 0)) + + if (__pa(p4d) !=3D (pgtable_l5_enabled() ? + __pa(pgd) : + (unsigned long)pgd_val(*pgd) & PTE_PFN_MASK)) printk(KERN_ERR "PAGETABLE BUG #00! %p <-> %p\n", p4d, p4d_offset(pgd, 0)); } @@ -263,7 +266,7 @@ static pud_t *fill_pud(p4d_t *p4d, unsigned long vaddr) if (p4d_none(*p4d)) { pud_t *pud =3D (pud_t *)spp_getpage(); p4d_populate(&init_mm, p4d, pud); - if (pud !=3D pud_offset(p4d, 0)) + if (__pa(pud) !=3D (p4d_val(*p4d) & p4d_pfn_mask(*p4d))) printk(KERN_ERR "PAGETABLE BUG #01! %p <-> %p\n", pud, pud_offset(p4d, 0)); } @@ -275,7 +278,7 @@ static pmd_t *fill_pmd(pud_t *pud, unsigned long vaddr) if (pud_none(*pud)) { pmd_t *pmd =3D (pmd_t *) spp_getpage(); pud_populate(&init_mm, pud, pmd); - if (pmd !=3D pmd_offset(pud, 0)) + if (__pa(pmd) !=3D (pud_val(*pud) & pud_pfn_mask(*pud))) printk(KERN_ERR "PAGETABLE BUG #02! %p <-> %p\n", pmd, pmd_offset(pud, 0)); } @@ -287,7 +290,7 @@ static pte_t *fill_pte(pmd_t *pmd, unsigned long vaddr) if (pmd_none(*pmd)) { pte_t *pte =3D (pte_t *) spp_getpage(); pmd_populate_kernel(&init_mm, pmd, pte); - if (pte !=3D pte_offset_kernel(pmd, 0)) + if (__pa(pte) !=3D (pmd_val(*pmd) & pmd_pfn_mask(*pmd))) printk(KERN_ERR "PAGETABLE BUG #03!\n"); } return pte_offset_kernel(pmd, vaddr); --=20 2.50.1 From nobody Sat Oct 4 20:54:34 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 67A3628000C; Tue, 12 Aug 2025 13:28:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005308; cv=none; b=W9CmOCqv9LD3bzyrtzF+y2aE4kz8HBM3HgWMUVPbX6ACplJE6MxbDdN/n13JlcII+daT5L/wYtsjEk69XT6c7X1mh3Ga/wnJBXPt7sfTcVvE61e95aPOo6jyLCC7ev5fqvuCYwNOMjjxyxbyBiUkndBlTs5zS3y4uHFs27LZTlI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005308; c=relaxed/simple; bh=c7Af/YAFKCbSIcmDUGPF1EcBvvAVYWDWdLscrGxcJaU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RJ8tZHQqYzPn7fht83aowl7hKdKxmmsBAAa4HQ/8lMwjqAdNZ1ThkuXsxvfGjiL/FYxeRvjmeoL3QrOMIerz9tBPIFy7AVH4i8zl4USg2b4G8AIaQTveI/l31S0gKRmW8n6wLXmbCYzq9vHXJvXtZMSz3GPi60b6+X1oByuEPxo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=JavcaNMc; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="JavcaNMc" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1755005307; x=1786541307; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=c7Af/YAFKCbSIcmDUGPF1EcBvvAVYWDWdLscrGxcJaU=; b=JavcaNMcPItqXKs1C0i8eeXgPQfbvEmU0NFw0fXXlg2xKV/p76Gi76kQ wK2dP3q8n597kyBV604FfYr38AcBD4Hqkg/uULEHHD0fTSasg4WWADZ/z q4yl7MY6gnQDNVa0JnA4aKjVluwIYncvs/N9tZQKlVRWB/+1LteZl1ZWH CSfSbNo1tc0g7iWvKRc+5NUUJPMT8U/FBNeogaJv92Am1n4LiHe3vM9OS TYnM28dUL9Ao/7MsEyC3HoWyTVr0jDo3opZNpRU5cSlr+Bf4VEyWulnWB Hnkn4Lf4xajhs2FLQN7oBsb0qQnHx8uOcCqGCASwEtJVmjKGx6nv5x6qv w==; X-CSE-ConnectionGUID: hiQ+PCM2S1avnaAzypLirA== X-CSE-MsgGUID: yq5Ai1tsRpCtKW4gDCY3fw== X-IronPort-AV: E=McAfee;i="6800,10657,11520"; a="60903623" X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="60903623" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:28:26 -0700 X-CSE-ConnectionGUID: RvG1WbQvQdy6dQLlny3rTg== X-CSE-MsgGUID: gCS5TnjYQ6isOtqp7VNDUw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="165831505" Received: from vpanait-mobl.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.245.54]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:28:02 -0700 From: Maciej Wieczor-Retman To: nathan@kernel.org, arnd@arndb.de, broonie@kernel.org, Liam.Howlett@oracle.com, urezki@gmail.com, will@kernel.org, kaleshsingh@google.com, rppt@kernel.org, leitao@debian.org, coxu@redhat.com, surenb@google.com, akpm@linux-foundation.org, luto@kernel.org, jpoimboe@kernel.org, changyuanl@google.com, hpa@zytor.com, dvyukov@google.com, kas@kernel.org, corbet@lwn.net, vincenzo.frascino@arm.com, smostafa@google.com, nick.desaulniers+lkml@gmail.com, morbo@google.com, andreyknvl@gmail.com, alexander.shishkin@linux.intel.com, thiago.bauermann@linaro.org, catalin.marinas@arm.com, ryabinin.a.a@gmail.com, jan.kiszka@siemens.com, jbohac@suse.cz, dan.j.williams@intel.com, joel.granados@kernel.org, baohua@kernel.org, kevin.brodsky@arm.com, nicolas.schier@linux.dev, pcc@google.com, andriy.shevchenko@linux.intel.com, wei.liu@kernel.org, bp@alien8.de, ada.coupriediaz@arm.com, xin@zytor.com, pankaj.gupta@amd.com, vbabka@suse.cz, glider@google.com, jgross@suse.com, kees@kernel.org, jhubbard@nvidia.com, joey.gouly@arm.com, ardb@kernel.org, thuth@redhat.com, pasha.tatashin@soleen.com, kristina.martsenko@arm.com, bigeasy@linutronix.de, maciej.wieczor-retman@intel.com, lorenzo.stoakes@oracle.com, jason.andryuk@amd.com, david@redhat.com, graf@amazon.com, wangkefeng.wang@huawei.com, ziy@nvidia.com, mark.rutland@arm.com, dave.hansen@linux.intel.com, samuel.holland@sifive.com, kbingham@kernel.org, trintaeoitogc@gmail.com, scott@os.amperecomputing.com, justinstitt@google.com, kuan-ying.lee@canonical.com, maz@kernel.org, tglx@linutronix.de, samitolvanen@google.com, mhocko@suse.com, nunodasneves@linux.microsoft.com, brgerst@gmail.com, willy@infradead.org, ubizjak@gmail.com, peterz@infradead.org, mingo@redhat.com, sohil.mehta@intel.com Cc: linux-mm@kvack.org, linux-kbuild@vger.kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org, llvm@lists.linux.dev, kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 09/18] x86: KASAN raw shadow memory PTE init Date: Tue, 12 Aug 2025 15:23:45 +0200 Message-ID: <38129f9031dfefc3e9465a0bf06fd32256cdc157.1755004923.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In KASAN's generic mode the default value in shadow memory is zero. During initialization of shadow memory pages they are allocated and zeroed. In KASAN's tag-based mode the default tag for the arm64 architecture is 0xFE which corresponds to any memory that should not be accessed. On x86 (where tags are 4-bit wide instead of 8-bit wide) that tag is 0xE so during the initializations all the bytes in shadow memory pages should be filled with it. Use memblock_alloc_try_nid_raw() instead of memblock_alloc_try_nid() to avoid zeroing out the memory so it can be set with the KASAN invalid tag. Signed-off-by: Maciej Wieczor-Retman --- Changelog v2: - Remove dense mode references, use memset() instead of kasan_poison(). arch/x86/mm/kasan_init_64.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c index 0539efd0d216..e8a451cafc8c 100644 --- a/arch/x86/mm/kasan_init_64.c +++ b/arch/x86/mm/kasan_init_64.c @@ -34,6 +34,18 @@ static __init void *early_alloc(size_t size, int nid, bo= ol should_panic) return ptr; } =20 +static __init void *early_raw_alloc(size_t size, int nid, bool should_pani= c) +{ + void *ptr =3D memblock_alloc_try_nid_raw(size, size, + __pa(MAX_DMA_ADDRESS), MEMBLOCK_ALLOC_ACCESSIBLE, nid); + + if (!ptr && should_panic) + panic("%pS: Failed to allocate page, nid=3D%d from=3D%lx\n", + (void *)_RET_IP_, nid, __pa(MAX_DMA_ADDRESS)); + + return ptr; +} + static void __init kasan_populate_pmd(pmd_t *pmd, unsigned long addr, unsigned long end, int nid) { @@ -63,8 +75,9 @@ static void __init kasan_populate_pmd(pmd_t *pmd, unsigne= d long addr, if (!pte_none(*pte)) continue; =20 - p =3D early_alloc(PAGE_SIZE, nid, true); - entry =3D pfn_pte(PFN_DOWN(__pa(p)), PAGE_KERNEL); + p =3D early_raw_alloc(PAGE_SIZE, nid, true); + memset(p, PAGE_SIZE, KASAN_SHADOW_INIT); + entry =3D pfn_pte(PFN_DOWN(__pa_nodebug(p)), PAGE_KERNEL); set_pte_at(&init_mm, addr, pte, entry); } while (pte++, addr +=3D PAGE_SIZE, addr !=3D end); } @@ -436,7 +449,7 @@ void __init kasan_init(void) * it may contain some garbage. Now we can clear and write protect it, * since after the TLB flush no one should write to it. */ - memset(kasan_early_shadow_page, 0, PAGE_SIZE); + memset(kasan_early_shadow_page, KASAN_SHADOW_INIT, PAGE_SIZE); for (i =3D 0; i < PTRS_PER_PTE; i++) { pte_t pte; pgprot_t prot; --=20 2.50.1 From nobody Sat Oct 4 20:54:34 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 28F8F280A5B; Tue, 12 Aug 2025 13:28:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005330; cv=none; b=IiBS7E8Lt9NitsuEO9awxHy3sRWcgXC2znpys5yBQtdkLLJDygNR88MGQ+WLfkv6RhLfMbbcjD52V9vAwzayXzIpcY0SJnunMnY8HlxXeuXyVe9Uvu6fxrrd/wWIy8ldUuqTBOAooWGazDj4WIgBLCRwI4l+VxW0eKdAb8IjfDI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005330; c=relaxed/simple; bh=LqACkkb9/6GBECPhMHTz7bJjgo3j4ob5b0Xf9fj/ZIk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eNxpx1xjBcA6REC1tmpUzHlHVB+QIzT0rNIjKQ0qztRwCHRmPNVTKeS8VlxYNlQXQ9wD6WsdVDqUqvIsmJTWnJFfFm5ygNHZ8nv0zVHfsBJJuPyMlfec3YSZMrXVQB1wE//JQtMK0mJhoXmo1/RBieTdyuQaywMB96Pjx5pd1Ss= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=dohaW+no; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="dohaW+no" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1755005330; x=1786541330; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=LqACkkb9/6GBECPhMHTz7bJjgo3j4ob5b0Xf9fj/ZIk=; b=dohaW+noLHnLLCJiFw/rddU1pJROWA3EUzafWIi5bXn0fVvNCnlB1LPo /NM8jkXKGuL01hePazEnxfkfmX8P0Yy0EmUXMblXftC9omg3sjniNJyY9 d7S9Usqh1RP+UNHVQVtsI647Wk8pkILiC3EPCA63U//or8yI4f8oLjIzv MPW0GhYOgpjP51wzESCuOmNaJRK9Thjfeov8ONrHFSTnkJw/ttv/QXHFQ F4b1RHPgsqEiZOjUC1kyMIapF0XQ7wLWlyixOd2MzGeqBCtSNUlz+3LhH Wj5cdczdxB7TvCp655rE/2eqwpJIytXZMTOyW3nveqAb0Jdef+rQPTpBz Q==; X-CSE-ConnectionGUID: zcIL7XpUSXynhAIdUkYQ/A== X-CSE-MsgGUID: PVoEeay9SuGwIKaH54EdAQ== X-IronPort-AV: E=McAfee;i="6800,10657,11520"; a="60903690" X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="60903690" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:28:49 -0700 X-CSE-ConnectionGUID: 91sElzL6QCmJr0sQT27i3w== X-CSE-MsgGUID: 58vZSmAIQMuehLmPiYiDkA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="165831526" Received: from vpanait-mobl.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.245.54]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:28:25 -0700 From: Maciej Wieczor-Retman To: nathan@kernel.org, arnd@arndb.de, broonie@kernel.org, Liam.Howlett@oracle.com, urezki@gmail.com, will@kernel.org, kaleshsingh@google.com, rppt@kernel.org, leitao@debian.org, coxu@redhat.com, surenb@google.com, akpm@linux-foundation.org, luto@kernel.org, jpoimboe@kernel.org, changyuanl@google.com, hpa@zytor.com, dvyukov@google.com, kas@kernel.org, corbet@lwn.net, vincenzo.frascino@arm.com, smostafa@google.com, nick.desaulniers+lkml@gmail.com, morbo@google.com, andreyknvl@gmail.com, alexander.shishkin@linux.intel.com, thiago.bauermann@linaro.org, catalin.marinas@arm.com, ryabinin.a.a@gmail.com, jan.kiszka@siemens.com, jbohac@suse.cz, dan.j.williams@intel.com, joel.granados@kernel.org, baohua@kernel.org, kevin.brodsky@arm.com, nicolas.schier@linux.dev, pcc@google.com, andriy.shevchenko@linux.intel.com, wei.liu@kernel.org, bp@alien8.de, ada.coupriediaz@arm.com, xin@zytor.com, pankaj.gupta@amd.com, vbabka@suse.cz, glider@google.com, jgross@suse.com, kees@kernel.org, jhubbard@nvidia.com, joey.gouly@arm.com, ardb@kernel.org, thuth@redhat.com, pasha.tatashin@soleen.com, kristina.martsenko@arm.com, bigeasy@linutronix.de, maciej.wieczor-retman@intel.com, lorenzo.stoakes@oracle.com, jason.andryuk@amd.com, david@redhat.com, graf@amazon.com, wangkefeng.wang@huawei.com, ziy@nvidia.com, mark.rutland@arm.com, dave.hansen@linux.intel.com, samuel.holland@sifive.com, kbingham@kernel.org, trintaeoitogc@gmail.com, scott@os.amperecomputing.com, justinstitt@google.com, kuan-ying.lee@canonical.com, maz@kernel.org, tglx@linutronix.de, samitolvanen@google.com, mhocko@suse.com, nunodasneves@linux.microsoft.com, brgerst@gmail.com, willy@infradead.org, ubizjak@gmail.com, peterz@infradead.org, mingo@redhat.com, sohil.mehta@intel.com Cc: linux-mm@kvack.org, linux-kbuild@vger.kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org, llvm@lists.linux.dev, kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 10/18] x86: LAM compatible non-canonical definition Date: Tue, 12 Aug 2025 15:23:46 +0200 Message-ID: <5dee53bb1044787199e143f7b5f6ec13204a3029.1755004923.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" For an address to be canonical it has to have its top bits equal to each other. The number of bits depends on the paging level and whether they're supposed to be ones or zeroes depends on whether the address points to kernel or user space. With Linear Address Masking (LAM) enabled, the definition of linear address canonicality is modified. Not all of the previously required bits need to be equal, only the first and last from the previously equal bitmask. So for example a 5-level paging kernel address needs to have bits [63] and [56] set. Add separate __canonical_address() implementation for CONFIG_KASAN_SW_TAGS since it's the only thing right now that enables LAM for kernel addresses (LAM_SUP bit in CR4). Signed-off-by: Maciej Wieczor-Retman --- Changelog v4: - Add patch to the series. arch/x86/include/asm/page.h | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/arch/x86/include/asm/page.h b/arch/x86/include/asm/page.h index 15c95e96fd15..97de2878f0b3 100644 --- a/arch/x86/include/asm/page.h +++ b/arch/x86/include/asm/page.h @@ -82,10 +82,20 @@ static __always_inline void *pfn_to_kaddr(unsigned long= pfn) return __va(pfn << PAGE_SHIFT); } =20 +/* + * CONFIG_KASAN_SW_TAGS requires LAM which changes the canonicality checks. + */ +#ifdef CONFIG_KASAN_SW_TAGS +static __always_inline u64 __canonical_address(u64 vaddr, u8 vaddr_bits) +{ + return (vaddr | BIT_ULL(63) | BIT_ULL(vaddr_bits - 1)); +} +#else static __always_inline u64 __canonical_address(u64 vaddr, u8 vaddr_bits) { return ((s64)vaddr << (64 - vaddr_bits)) >> (64 - vaddr_bits); } +#endif =20 static __always_inline u64 __is_canonical_address(u64 vaddr, u8 vaddr_bits) { --=20 2.50.1 From nobody Sat Oct 4 20:54:34 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0E5EA284B39; Tue, 12 Aug 2025 13:29:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005354; cv=none; b=bToH1H5kewLtmaVR9/gSCEAHbc3xjUvAZhm1k8jsWajKLfC6lU1tp1b/b2R2EzUSOIP+BgVdGkuYedWPulQ0HwWDZR8jgs3BxjHOaWt+R1asMumVRxaFndWd8S9Qs+Kejcev0UkkbI1SsaHOMslG2PFBQlZhJBjBAVhnQb2Osok= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005354; c=relaxed/simple; bh=UlbhFGz226HIxGPpl32TNHWjf4s+OegZMHU4Z5wVVAQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Ph5f7c3G71fMk9P/q2fKSjbr5NsXUgEO7Na7u7RrWzF8k+dnQMTVPfgVBgg9vj4Uku3MckW/hXujLDw0wcwmn1VxzSoo3PPg1jwrYncYlS5nRjovSEuIsdqmC3O0fku7QJkB1tPPW1e/3Mx0dCQcM7rvclFl3XYnHExylNbKJeA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=W1Jxq65g; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="W1Jxq65g" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1755005354; x=1786541354; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=UlbhFGz226HIxGPpl32TNHWjf4s+OegZMHU4Z5wVVAQ=; b=W1Jxq65gFZuaXsYz54sumoPhCdtFIDm6YDCUPpYIkfZs1+Q/oam2JOy/ Ezvk4Hu4TXPQaBQalfgPf3VgVr7Jqbim8rfrQyy/jqseJj1osOGjzaQhm kVv0JdRPQvBtnJxLt96dbbuYA3iGMDWuBVq8FNQouOW3kj2Lbm+jKLWka M+Rdgawi5jXtAe2/6wiHQ1PdbRGRo/NFC0wNMcfT/8l8VUKbRBWF6X3/O iYMTKHYObuTZqkW72EouKCg2t2pfMw15U0CmRdNmWE7SUrjEoMCqX7h9l LCoqDWbRiTW7LH1B6bx+3FIEz9vrnNs3yyPaWdNBTBBPp9Z1UlPgDloy4 w==; X-CSE-ConnectionGUID: qjui5eutSt6IgjUiJT+9pQ== X-CSE-MsgGUID: Fokb0MiNSWqyuQI1xKLpyg== X-IronPort-AV: E=McAfee;i="6800,10657,11520"; a="60903741" X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="60903741" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:29:12 -0700 X-CSE-ConnectionGUID: QX9ghsjwQgadzn721t5Dqw== X-CSE-MsgGUID: JxTTgOkVRwaxXFC7TVl+Qg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="165831554" Received: from vpanait-mobl.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.245.54]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:28:48 -0700 From: Maciej Wieczor-Retman To: nathan@kernel.org, arnd@arndb.de, broonie@kernel.org, Liam.Howlett@oracle.com, urezki@gmail.com, will@kernel.org, kaleshsingh@google.com, rppt@kernel.org, leitao@debian.org, coxu@redhat.com, surenb@google.com, akpm@linux-foundation.org, luto@kernel.org, jpoimboe@kernel.org, changyuanl@google.com, hpa@zytor.com, dvyukov@google.com, kas@kernel.org, corbet@lwn.net, vincenzo.frascino@arm.com, smostafa@google.com, nick.desaulniers+lkml@gmail.com, morbo@google.com, andreyknvl@gmail.com, alexander.shishkin@linux.intel.com, thiago.bauermann@linaro.org, catalin.marinas@arm.com, ryabinin.a.a@gmail.com, jan.kiszka@siemens.com, jbohac@suse.cz, dan.j.williams@intel.com, joel.granados@kernel.org, baohua@kernel.org, kevin.brodsky@arm.com, nicolas.schier@linux.dev, pcc@google.com, andriy.shevchenko@linux.intel.com, wei.liu@kernel.org, bp@alien8.de, ada.coupriediaz@arm.com, xin@zytor.com, pankaj.gupta@amd.com, vbabka@suse.cz, glider@google.com, jgross@suse.com, kees@kernel.org, jhubbard@nvidia.com, joey.gouly@arm.com, ardb@kernel.org, thuth@redhat.com, pasha.tatashin@soleen.com, kristina.martsenko@arm.com, bigeasy@linutronix.de, maciej.wieczor-retman@intel.com, lorenzo.stoakes@oracle.com, jason.andryuk@amd.com, david@redhat.com, graf@amazon.com, wangkefeng.wang@huawei.com, ziy@nvidia.com, mark.rutland@arm.com, dave.hansen@linux.intel.com, samuel.holland@sifive.com, kbingham@kernel.org, trintaeoitogc@gmail.com, scott@os.amperecomputing.com, justinstitt@google.com, kuan-ying.lee@canonical.com, maz@kernel.org, tglx@linutronix.de, samitolvanen@google.com, mhocko@suse.com, nunodasneves@linux.microsoft.com, brgerst@gmail.com, willy@infradead.org, ubizjak@gmail.com, peterz@infradead.org, mingo@redhat.com, sohil.mehta@intel.com Cc: linux-mm@kvack.org, linux-kbuild@vger.kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org, llvm@lists.linux.dev, kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 11/18] x86: LAM initialization Date: Tue, 12 Aug 2025 15:23:47 +0200 Message-ID: <94461b3ac97f13073c8db552f90952aa7edf503a.1755004923.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" To make use of KASAN's tag based mode on x86, Linear Address Masking (LAM) needs to be enabled. To do that the 28th bit in CR4 has to be set. Set the bit in early memory initialization. When launching secondary CPUs the LAM bit gets lost. To avoid this add it in a mask in head_64.S. The bitmask permits some bits of CR4 to pass from the primary CPU to the secondary CPUs without being cleared. Signed-off-by: Maciej Wieczor-Retman --- arch/x86/kernel/head_64.S | 3 +++ arch/x86/mm/init.c | 3 +++ 2 files changed, 6 insertions(+) diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 3e9b3a3bd039..18ca77daa481 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -209,6 +209,9 @@ SYM_INNER_LABEL(common_startup_64, SYM_L_LOCAL) * there will be no global TLB entries after the execution." */ movl $(X86_CR4_PAE | X86_CR4_LA57), %edx +#ifdef CONFIG_ADDRESS_MASKING + orl $X86_CR4_LAM_SUP, %edx +#endif #ifdef CONFIG_X86_MCE /* * Preserve CR4.MCE if the kernel will enable #MC support. diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c index bb57e93b4caf..756bd96c3b8b 100644 --- a/arch/x86/mm/init.c +++ b/arch/x86/mm/init.c @@ -763,6 +763,9 @@ void __init init_mem_mapping(void) probe_page_size_mask(); setup_pcid(); =20 + if (boot_cpu_has(X86_FEATURE_LAM) && IS_ENABLED(CONFIG_KASAN_SW_TAGS)) + cr4_set_bits_and_update_boot(X86_CR4_LAM_SUP); + #ifdef CONFIG_X86_64 end =3D max_pfn << PAGE_SHIFT; #else --=20 2.50.1 From nobody Sat Oct 4 20:54:34 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8AD6B1FDA94; Tue, 12 Aug 2025 13:29:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005376; cv=none; b=ekghHj2P7u7ZAdLabVFzpHT7vyGVxLrMkBkPk0MLGUPH0EXw+jUutmslaFxfUfA1phogqM+QhnFUO8Xj/24JWAhV0Ct4FuM9cLJ1gna0/bX5YoMEKo3RBp4OIaLtkyutn34uFwSMSzbPuGZ55Duysr16FCH4pnVU1bLtKNU7zrM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005376; c=relaxed/simple; bh=9AhOaWTatNb4QMfmox2H6CEYyqT3KnhFIot1r52xKmg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cJoElwlB/GlZpH8FwhqRFK+/j4YK9luQvorkfPK+XFkTcrbygm2UvYsi6CWwCmXzt9b06sz+S8tWFImeMF5YMAzYjC8u84FncgkrwdFl7z0WafZL1p0kX/wyFwdgAS3KeSWKQynKtQFTGosdoljS67t/RrRW85uI1GZC5qQD+Lw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=dxXQMyrK; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="dxXQMyrK" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1755005375; x=1786541375; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=9AhOaWTatNb4QMfmox2H6CEYyqT3KnhFIot1r52xKmg=; b=dxXQMyrKsH7Jyak4u9OBrQP/i1i2aALoCY7NNFLedp5Yz2g2X6+qQeGo nNbBcK72oWlAnAvSZ9a8a13qaGZu+A6TSif/ucrF89EDeFDmDMBo3I1pu d4dkksejp/LPcMYE1iObLcsvrUjHp1r2s2ZF801R/Y1Y0jvrydkApyoRG EprNjezSdDQrxfX9ku/ZRi2Zr80jttaeJk4g9XZ2me6Ppez15+0dfcTqM bZqdvCgZ+Pkv5DTY2WW/0myjmyFFnZg7IaUZwDllTduSawyyoERDk/197 1hcSWfaf/uRxu0HVcPfHKZFOhvajoOGezVF5LeQzXtWq2hxppub35S1FD A==; X-CSE-ConnectionGUID: iTkOM0YRQ8Kj38R4ZT19Bg== X-CSE-MsgGUID: 0VoXRUPYTLeBIRb0ccFt6Q== X-IronPort-AV: E=McAfee;i="6800,10657,11520"; a="60903795" X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="60903795" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:29:34 -0700 X-CSE-ConnectionGUID: t6kjFU4oRZiLZmMpMij85g== X-CSE-MsgGUID: TDLcIyj9Qy+UaemOVoZKVg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="165831582" Received: from vpanait-mobl.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.245.54]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:29:12 -0700 From: Maciej Wieczor-Retman To: nathan@kernel.org, arnd@arndb.de, broonie@kernel.org, Liam.Howlett@oracle.com, urezki@gmail.com, will@kernel.org, kaleshsingh@google.com, rppt@kernel.org, leitao@debian.org, coxu@redhat.com, surenb@google.com, akpm@linux-foundation.org, luto@kernel.org, jpoimboe@kernel.org, changyuanl@google.com, hpa@zytor.com, dvyukov@google.com, kas@kernel.org, corbet@lwn.net, vincenzo.frascino@arm.com, smostafa@google.com, nick.desaulniers+lkml@gmail.com, morbo@google.com, andreyknvl@gmail.com, alexander.shishkin@linux.intel.com, thiago.bauermann@linaro.org, catalin.marinas@arm.com, ryabinin.a.a@gmail.com, jan.kiszka@siemens.com, jbohac@suse.cz, dan.j.williams@intel.com, joel.granados@kernel.org, baohua@kernel.org, kevin.brodsky@arm.com, nicolas.schier@linux.dev, pcc@google.com, andriy.shevchenko@linux.intel.com, wei.liu@kernel.org, bp@alien8.de, ada.coupriediaz@arm.com, xin@zytor.com, pankaj.gupta@amd.com, vbabka@suse.cz, glider@google.com, jgross@suse.com, kees@kernel.org, jhubbard@nvidia.com, joey.gouly@arm.com, ardb@kernel.org, thuth@redhat.com, pasha.tatashin@soleen.com, kristina.martsenko@arm.com, bigeasy@linutronix.de, maciej.wieczor-retman@intel.com, lorenzo.stoakes@oracle.com, jason.andryuk@amd.com, david@redhat.com, graf@amazon.com, wangkefeng.wang@huawei.com, ziy@nvidia.com, mark.rutland@arm.com, dave.hansen@linux.intel.com, samuel.holland@sifive.com, kbingham@kernel.org, trintaeoitogc@gmail.com, scott@os.amperecomputing.com, justinstitt@google.com, kuan-ying.lee@canonical.com, maz@kernel.org, tglx@linutronix.de, samitolvanen@google.com, mhocko@suse.com, nunodasneves@linux.microsoft.com, brgerst@gmail.com, willy@infradead.org, ubizjak@gmail.com, peterz@infradead.org, mingo@redhat.com, sohil.mehta@intel.com Cc: linux-mm@kvack.org, linux-kbuild@vger.kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org, llvm@lists.linux.dev, kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 12/18] x86: Minimal SLAB alignment Date: Tue, 12 Aug 2025 15:23:48 +0200 Message-ID: X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" 8 byte minimal SLAB alignment interferes with KASAN's granularity of 16 bytes. It causes a lot of out-of-bounds errors for unaligned 8 byte allocations. Compared to a kernel with KASAN disabled, the memory footprint increases because all kmalloc-8 allocations now are realized as kmalloc-16, which has twice the object size. But more meaningfully, when compared to a kernel with generic KASAN enabled, there is no difference. Because of redzones in generic KASAN, kmalloc-8' and kmalloc-16' object size is the same (48 bytes). So changing the minimal SLAB alignment of the tag-based mode doesn't have any negative impact when compared to the other software KASAN mode. Adjust x86 minimal SLAB alignment to match KASAN granularity size. Signed-off-by: Maciej Wieczor-Retman --- Changelog v4: - Extend the patch message with some more context and impact information. Changelog v3: - Fix typo in patch message 4 -> 16. - Change define location to arch/x86/include/asm/cache.c. arch/x86/include/asm/cache.h | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/arch/x86/include/asm/cache.h b/arch/x86/include/asm/cache.h index 69404eae9983..3232583b5487 100644 --- a/arch/x86/include/asm/cache.h +++ b/arch/x86/include/asm/cache.h @@ -21,4 +21,8 @@ #endif #endif =20 +#ifdef CONFIG_KASAN_SW_TAGS +#define ARCH_SLAB_MINALIGN (1ULL << KASAN_SHADOW_SCALE_SHIFT) +#endif + #endif /* _ASM_X86_CACHE_H */ --=20 2.50.1 From nobody Sat Oct 4 20:54:34 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E96AF27E070; Tue, 12 Aug 2025 13:29:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005400; cv=none; b=EvH18KL6kWZXvKaqmCDv6wwQJlb9eUJgfyilLvIUvrMxr42SA98taWcxjGZ7g6sBYKYZc+0bQklstXSHYARSYWo1RmCfpgVSFLdR1Vn+ceTo8xvLwor714Fq65BmdEytX1RXIn50gq2B1i/InML69JZS7MEv0yhdHz9efgLPMOM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005400; c=relaxed/simple; bh=s8GTuO6BhzUjxSM1kGBYFltc6MUMI6BMbrDsjAC5xo8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LidNO23eNIXHtPQXe3IEgX9mrrOOyVR0/ip/xeh8vi/7ebIoO4oe3w6mYjavuwlu8KFzsg9qyWmCjsMaqOPxlUdlvGcQJMt1ofltdqvZfo1+suadmd5WJih6r87U+CyuwWbICz/W2ypvtlQqGo/O1+P+c6yePoMMcjFZHdlPLSo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=O5ms1iwj; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="O5ms1iwj" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1755005400; x=1786541400; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=s8GTuO6BhzUjxSM1kGBYFltc6MUMI6BMbrDsjAC5xo8=; b=O5ms1iwjG+roKdf7o/OWqgTTck3ipMoXK6NJ0Lu2BHGZke2pJYkAKFPl GvLSTbPa9j12QGqg0MXD5x9y1JRhRynLyNPC2tHcyyPRZCSylXkVrFwtx /93S68K9N5xi2FwYrSGnLb7ME9VzaMdfqNIJGr4xtV0PBu7j2vJRPHQ2Z iLNt7tXqSNyYr+nN4HHsiIt6l6YbDKhgeCZfx6ljinMbZVb5r4iehLMSx 7ou4unJMtrgXxuoCmLD2pkaZfqM0ub1C5ZvCfGzPmtqLWIW7DGaCuG8lc Hnlfp//pjeA5lp7ReYyaXeBMV4K0ia5IbXeuyMaikueVWoHMRlbNU7mx5 w==; X-CSE-ConnectionGUID: p4KwOclKTqSOjCkKH41dOg== X-CSE-MsgGUID: bKB9gV8IR/inSZ9n3tXEHw== X-IronPort-AV: E=McAfee;i="6800,10657,11520"; a="60903862" X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="60903862" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:29:58 -0700 X-CSE-ConnectionGUID: vu28n5RXQyCx06dXuuNJDg== X-CSE-MsgGUID: KI/7/IerRmiIwK8D24KJGg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="165831619" Received: from vpanait-mobl.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.245.54]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:29:34 -0700 From: Maciej Wieczor-Retman To: nathan@kernel.org, arnd@arndb.de, broonie@kernel.org, Liam.Howlett@oracle.com, urezki@gmail.com, will@kernel.org, kaleshsingh@google.com, rppt@kernel.org, leitao@debian.org, coxu@redhat.com, surenb@google.com, akpm@linux-foundation.org, luto@kernel.org, jpoimboe@kernel.org, changyuanl@google.com, hpa@zytor.com, dvyukov@google.com, kas@kernel.org, corbet@lwn.net, vincenzo.frascino@arm.com, smostafa@google.com, nick.desaulniers+lkml@gmail.com, morbo@google.com, andreyknvl@gmail.com, alexander.shishkin@linux.intel.com, thiago.bauermann@linaro.org, catalin.marinas@arm.com, ryabinin.a.a@gmail.com, jan.kiszka@siemens.com, jbohac@suse.cz, dan.j.williams@intel.com, joel.granados@kernel.org, baohua@kernel.org, kevin.brodsky@arm.com, nicolas.schier@linux.dev, pcc@google.com, andriy.shevchenko@linux.intel.com, wei.liu@kernel.org, bp@alien8.de, ada.coupriediaz@arm.com, xin@zytor.com, pankaj.gupta@amd.com, vbabka@suse.cz, glider@google.com, jgross@suse.com, kees@kernel.org, jhubbard@nvidia.com, joey.gouly@arm.com, ardb@kernel.org, thuth@redhat.com, pasha.tatashin@soleen.com, kristina.martsenko@arm.com, bigeasy@linutronix.de, maciej.wieczor-retman@intel.com, lorenzo.stoakes@oracle.com, jason.andryuk@amd.com, david@redhat.com, graf@amazon.com, wangkefeng.wang@huawei.com, ziy@nvidia.com, mark.rutland@arm.com, dave.hansen@linux.intel.com, samuel.holland@sifive.com, kbingham@kernel.org, trintaeoitogc@gmail.com, scott@os.amperecomputing.com, justinstitt@google.com, kuan-ying.lee@canonical.com, maz@kernel.org, tglx@linutronix.de, samitolvanen@google.com, mhocko@suse.com, nunodasneves@linux.microsoft.com, brgerst@gmail.com, willy@infradead.org, ubizjak@gmail.com, peterz@infradead.org, mingo@redhat.com, sohil.mehta@intel.com Cc: linux-mm@kvack.org, linux-kbuild@vger.kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org, llvm@lists.linux.dev, kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 13/18] kasan: arm64: x86: Handle int3 for inline KASAN reports Date: Tue, 12 Aug 2025 15:23:49 +0200 Message-ID: <9030d5a35eb5a3831319881cb8cb040aad65b7b6.1755004923.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Inline KASAN on x86 does tag mismatch reports by passing the faulty address and metadata through the INT3 instruction - scheme that's setup in the LLVM's compiler code (specifically HWAddressSanitizer.cpp). Add a kasan hook to the INT3 handling function. Disable KASAN in an INT3 core kernel selftest function since it can raise a false tag mismatch report and potentially panic the kernel. Make part of that hook - which decides whether to die or recover from a tag mismatch - arch independent to avoid duplicating a long comment on both x86 and arm64 architectures. Signed-off-by: Maciej Wieczor-Retman --- Changelog v4: - Make kasan_handler() a stub in a header file. Remove #ifdef from traps.c. - Consolidate the "recover" comment into one place. - Make small changes to the patch message. MAINTAINERS | 2 +- arch/arm64/kernel/traps.c | 17 +---------------- arch/x86/include/asm/kasan.h | 26 ++++++++++++++++++++++++++ arch/x86/kernel/alternative.c | 4 +++- arch/x86/kernel/traps.c | 4 ++++ arch/x86/mm/Makefile | 2 ++ arch/x86/mm/kasan_inline.c | 23 +++++++++++++++++++++++ include/linux/kasan.h | 24 ++++++++++++++++++++++++ 8 files changed, 84 insertions(+), 18 deletions(-) create mode 100644 arch/x86/mm/kasan_inline.c diff --git a/MAINTAINERS b/MAINTAINERS index 7ce8c6b86e3d..3daeeaf67546 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -13167,7 +13167,7 @@ S: Maintained B: https://bugzilla.kernel.org/buglist.cgi?component=3DSanitizers&product= =3DMemory%20Management F: Documentation/dev-tools/kasan.rst F: arch/*/include/asm/*kasan*.h -F: arch/*/mm/kasan_init* +F: arch/*/mm/kasan_* F: include/linux/kasan*.h F: lib/Kconfig.kasan F: mm/kasan/ diff --git a/arch/arm64/kernel/traps.c b/arch/arm64/kernel/traps.c index f528b6041f6a..b9bdabc14ad1 100644 --- a/arch/arm64/kernel/traps.c +++ b/arch/arm64/kernel/traps.c @@ -1068,22 +1068,7 @@ int kasan_brk_handler(struct pt_regs *regs, unsigned= long esr) =20 kasan_report(addr, size, write, pc); =20 - /* - * The instrumentation allows to control whether we can proceed after - * a crash was detected. This is done by passing the -recover flag to - * the compiler. Disabling recovery allows to generate more compact - * code. - * - * Unfortunately disabling recovery doesn't work for the kernel right - * now. KASAN reporting is disabled in some contexts (for example when - * the allocator accesses slab object metadata; this is controlled by - * current->kasan_depth). All these accesses are detected by the tool, - * even though the reports for them are not printed. - * - * This is something that might be fixed at some point in the future. - */ - if (!recover) - die("Oops - KASAN", regs, esr); + kasan_inline_recover(recover, "Oops - KASAN", regs, esr); =20 /* If thread survives, skip over the brk instruction and continue: */ arm64_skip_faulting_instruction(regs, AARCH64_INSN_SIZE); diff --git a/arch/x86/include/asm/kasan.h b/arch/x86/include/asm/kasan.h index 1963eb2fcff3..5bf38bb836e1 100644 --- a/arch/x86/include/asm/kasan.h +++ b/arch/x86/include/asm/kasan.h @@ -6,7 +6,28 @@ #include #include #define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) +#ifdef CONFIG_KASAN_SW_TAGS + +/* + * LLVM ABI for reporting tag mismatches in inline KASAN mode. + * On x86 the INT3 instruction is used to carry metadata in RAX + * to the KASAN report. + * + * SIZE refers to how many bytes the faulty memory access + * requested. + * WRITE bit, when set, indicates the access was a write, otherwise + * it was a read. + * RECOVER bit, when set, should allow the kernel to carry on after + * a tag mismatch. Otherwise die() is called. + */ +#define KASAN_RAX_RECOVER 0x20 +#define KASAN_RAX_WRITE 0x10 +#define KASAN_RAX_SIZE_MASK 0x0f +#define KASAN_RAX_SIZE(rax) (1 << ((rax) & KASAN_RAX_SIZE_MASK)) + +#else #define KASAN_SHADOW_SCALE_SHIFT 3 +#endif =20 /* * Compiler uses shadow offset assuming that addresses start @@ -35,10 +56,15 @@ #define __tag_shifted(tag) FIELD_PREP(GENMASK_ULL(60, 57), tag) #define __tag_reset(addr) (sign_extend64((u64)(addr), 56)) #define __tag_get(addr) ((u8)FIELD_GET(GENMASK_ULL(60, 57), (u64)addr)) +bool kasan_inline_handler(struct pt_regs *regs); #else #define __tag_shifted(tag) 0UL #define __tag_reset(addr) (addr) #define __tag_get(addr) 0 +static inline bool kasan_inline_handler(struct pt_regs *regs) +{ + return false; +} #endif /* CONFIG_KASAN_SW_TAGS */ =20 static inline void *__tag_set(const void *__addr, u8 tag) diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index 2a330566e62b..4cb085daad31 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -2228,7 +2228,7 @@ int3_exception_notify(struct notifier_block *self, un= signed long val, void *data } =20 /* Must be noinline to ensure uniqueness of int3_selftest_ip. */ -static noinline void __init int3_selftest(void) +static noinline __no_sanitize_address void __init int3_selftest(void) { static __initdata struct notifier_block int3_exception_nb =3D { .notifier_call =3D int3_exception_notify, @@ -2236,6 +2236,7 @@ static noinline void __init int3_selftest(void) }; unsigned int val =3D 0; =20 + kasan_disable_current(); BUG_ON(register_die_notifier(&int3_exception_nb)); =20 /* @@ -2253,6 +2254,7 @@ static noinline void __init int3_selftest(void) =20 BUG_ON(val !=3D 1); =20 + kasan_enable_current(); unregister_die_notifier(&int3_exception_nb); } =20 diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 0f6f187b1a9e..2a119279980f 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -912,6 +912,10 @@ static bool do_int3(struct pt_regs *regs) if (kprobe_int3_handler(regs)) return true; #endif + + if (kasan_inline_handler(regs)) + return true; + res =3D notify_die(DIE_INT3, "int3", regs, 0, X86_TRAP_BP, SIGTRAP); =20 return res =3D=3D NOTIFY_STOP; diff --git a/arch/x86/mm/Makefile b/arch/x86/mm/Makefile index 5b9908f13dcf..1dc18090cbe7 100644 --- a/arch/x86/mm/Makefile +++ b/arch/x86/mm/Makefile @@ -36,7 +36,9 @@ obj-$(CONFIG_PTDUMP) +=3D dump_pagetables.o obj-$(CONFIG_PTDUMP_DEBUGFS) +=3D debug_pagetables.o =20 KASAN_SANITIZE_kasan_init_$(BITS).o :=3D n +KASAN_SANITIZE_kasan_inline.o :=3D n obj-$(CONFIG_KASAN) +=3D kasan_init_$(BITS).o +obj-$(CONFIG_KASAN_SW_TAGS) +=3D kasan_inline.o =20 KMSAN_SANITIZE_kmsan_shadow.o :=3D n obj-$(CONFIG_KMSAN) +=3D kmsan_shadow.o diff --git a/arch/x86/mm/kasan_inline.c b/arch/x86/mm/kasan_inline.c new file mode 100644 index 000000000000..9f85dfd1c38b --- /dev/null +++ b/arch/x86/mm/kasan_inline.c @@ -0,0 +1,23 @@ +// SPDX-License-Identifier: GPL-2.0 +#include +#include + +bool kasan_inline_handler(struct pt_regs *regs) +{ + int metadata =3D regs->ax; + u64 addr =3D regs->di; + u64 pc =3D regs->ip; + bool recover =3D metadata & KASAN_RAX_RECOVER; + bool write =3D metadata & KASAN_RAX_WRITE; + size_t size =3D KASAN_RAX_SIZE(metadata); + + if (user_mode(regs)) + return false; + + if (!kasan_report((void *)addr, size, write, pc)) + return false; + + kasan_inline_recover(recover, "Oops - KASAN", regs, metadata, die); + + return true; +} diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 54481f8c30c5..8691ad870f3b 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -663,4 +663,28 @@ void kasan_non_canonical_hook(unsigned long addr); static inline void kasan_non_canonical_hook(unsigned long addr) { } #endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ =20 +#ifdef CONFIG_KASAN_SW_TAGS +/* + * The instrumentation allows to control whether we can proceed after + * a crash was detected. This is done by passing the -recover flag to + * the compiler. Disabling recovery allows to generate more compact + * code. + * + * Unfortunately disabling recovery doesn't work for the kernel right + * now. KASAN reporting is disabled in some contexts (for example when + * the allocator accesses slab object metadata; this is controlled by + * current->kasan_depth). All these accesses are detected by the tool, + * even though the reports for them are not printed. + * + * This is something that might be fixed at some point in the future. + */ +static inline void kasan_inline_recover( + bool recover, char *msg, struct pt_regs *regs, unsigned long err, + void die_fn(const char *str, struct pt_regs *regs, long err)) +{ + if (!recover) + die_fn(msg, regs, err); +} +#endif + #endif /* LINUX_KASAN_H */ --=20 2.50.1 From nobody Sat Oct 4 20:54:34 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DB40D283C9E; Tue, 12 Aug 2025 13:30:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005425; cv=none; b=SSAlmkooWizOUHcTpY1Uczewrl95jOY3LcaY5f+Ix+xixapVgDbtnu5EYfVqY04G2hQ+gEx9GJT8siD+R/9Qm8Tn7DzdAHu4p0lzr8P2qLWaI6Mv5Hi1fyv40GH/1k0jxk8/5OEe1zWyxE5+sQml73oDSaDyEpmxxMOeQ+JgIfA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005425; c=relaxed/simple; bh=Z/kDn3NFABCWRH7hqk/aYpfugnY8BGjn3l5jxoLYjts=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=YaAMNnrgOhTAc7nS6C5Mis3kg7eSTazI8SUrhbsq+uNodu7m8mi3GT2SGh64gXSTnaomRkmIl77hbYpw1yyJlmTqLNMqhiOfpacI4H8d7aDcs+qB+fr91McpPlCe7ffXVh467JXZXNFgdm1BWdvZokGXs7bk6TEeC+AKv8y/XPs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=T8BjC18w; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="T8BjC18w" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1755005424; x=1786541424; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Z/kDn3NFABCWRH7hqk/aYpfugnY8BGjn3l5jxoLYjts=; b=T8BjC18wBYWexyUPECVg0o56O98qxKgdIHfcXcTCX9QJVEt6SXddG+eG +XBQo7bgXyZ70ZOtWFRfpF3kj42fYCOdVs2YwZnEhmLfzhXm+VHh4ypyK VnyXuyd6VARecvJmzxUewg8W/WDcaZSKBczUfix6WoxrjZXaQ/kmImKXz U4gTlpMR/SLzjZ9zKGdSrxW4RFFymDfjwYFuzpKipmi6WcJtETZM1CrDu 2PazCBLxg7ZNdRYltqgN40l+TSVBd6uTpppO8KjGyHXPoFfMb59q9vH87 6S0jqHmuBe8zu2cVmed0zuaQs9NiAAaD5JgYrzYyJT+a1mq4M/6gK91Ss A==; X-CSE-ConnectionGUID: xwnm88smQEmhn+HIZ/MVZg== X-CSE-MsgGUID: 85EmKvBZS++yfZQfnlO1+A== X-IronPort-AV: E=McAfee;i="6800,10657,11520"; a="60903934" X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="60903934" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:30:24 -0700 X-CSE-ConnectionGUID: O4o3sBLXTKmQMTR9muYdMA== X-CSE-MsgGUID: lT/GoXCxT2av3FFB71gMVw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="165831775" Received: from vpanait-mobl.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.245.54]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:29:58 -0700 From: Maciej Wieczor-Retman To: nathan@kernel.org, arnd@arndb.de, broonie@kernel.org, Liam.Howlett@oracle.com, urezki@gmail.com, will@kernel.org, kaleshsingh@google.com, rppt@kernel.org, leitao@debian.org, coxu@redhat.com, surenb@google.com, akpm@linux-foundation.org, luto@kernel.org, jpoimboe@kernel.org, changyuanl@google.com, hpa@zytor.com, dvyukov@google.com, kas@kernel.org, corbet@lwn.net, vincenzo.frascino@arm.com, smostafa@google.com, nick.desaulniers+lkml@gmail.com, morbo@google.com, andreyknvl@gmail.com, alexander.shishkin@linux.intel.com, thiago.bauermann@linaro.org, catalin.marinas@arm.com, ryabinin.a.a@gmail.com, jan.kiszka@siemens.com, jbohac@suse.cz, dan.j.williams@intel.com, joel.granados@kernel.org, baohua@kernel.org, kevin.brodsky@arm.com, nicolas.schier@linux.dev, pcc@google.com, andriy.shevchenko@linux.intel.com, wei.liu@kernel.org, bp@alien8.de, ada.coupriediaz@arm.com, xin@zytor.com, pankaj.gupta@amd.com, vbabka@suse.cz, glider@google.com, jgross@suse.com, kees@kernel.org, jhubbard@nvidia.com, joey.gouly@arm.com, ardb@kernel.org, thuth@redhat.com, pasha.tatashin@soleen.com, kristina.martsenko@arm.com, bigeasy@linutronix.de, maciej.wieczor-retman@intel.com, lorenzo.stoakes@oracle.com, jason.andryuk@amd.com, david@redhat.com, graf@amazon.com, wangkefeng.wang@huawei.com, ziy@nvidia.com, mark.rutland@arm.com, dave.hansen@linux.intel.com, samuel.holland@sifive.com, kbingham@kernel.org, trintaeoitogc@gmail.com, scott@os.amperecomputing.com, justinstitt@google.com, kuan-ying.lee@canonical.com, maz@kernel.org, tglx@linutronix.de, samitolvanen@google.com, mhocko@suse.com, nunodasneves@linux.microsoft.com, brgerst@gmail.com, willy@infradead.org, ubizjak@gmail.com, peterz@infradead.org, mingo@redhat.com, sohil.mehta@intel.com Cc: linux-mm@kvack.org, linux-kbuild@vger.kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org, llvm@lists.linux.dev, kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 14/18] kasan: x86: Apply multishot to the inline report handler Date: Tue, 12 Aug 2025 15:23:50 +0200 Message-ID: <8ace14464d88b51f309b289874760b5d6265e438.1755004923.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" KASAN by default reports only one tag mismatch and based on other command line parameters either keeps going or panics. The multishot mechanism - enabled either through a command line parameter or by inline enable/disable function calls - lifts that restriction and allows an infinite number of tag mismatch reports to be shown. Inline KASAN uses the INT3 instruction to pass metadata to the report handling function. Currently the "recover" field in that metadata is broken in the compiler layer and causes every inline tag mismatch to panic the kernel. Check the multishot state in the KASAN hook called inside the INT3 handling function. Signed-off-by: Maciej Wieczor-Retman --- Changelog v4: - Add this patch to the series. arch/x86/mm/kasan_inline.c | 3 +++ include/linux/kasan.h | 3 +++ mm/kasan/report.c | 8 +++++++- 3 files changed, 13 insertions(+), 1 deletion(-) diff --git a/arch/x86/mm/kasan_inline.c b/arch/x86/mm/kasan_inline.c index 9f85dfd1c38b..f837caf32e6c 100644 --- a/arch/x86/mm/kasan_inline.c +++ b/arch/x86/mm/kasan_inline.c @@ -17,6 +17,9 @@ bool kasan_inline_handler(struct pt_regs *regs) if (!kasan_report((void *)addr, size, write, pc)) return false; =20 + if (kasan_multi_shot_enabled()) + return true; + kasan_inline_recover(recover, "Oops - KASAN", regs, metadata, die); =20 return true; diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 8691ad870f3b..7a2527794549 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -663,7 +663,10 @@ void kasan_non_canonical_hook(unsigned long addr); static inline void kasan_non_canonical_hook(unsigned long addr) { } #endif /* CONFIG_KASAN_GENERIC || CONFIG_KASAN_SW_TAGS */ =20 +bool kasan_multi_shot_enabled(void); + #ifdef CONFIG_KASAN_SW_TAGS + /* * The instrumentation allows to control whether we can proceed after * a crash was detected. This is done by passing the -recover flag to diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 93c6cadb0765..cfa2da0e2985 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -121,6 +121,12 @@ static void report_suppress_stop(void) #endif } =20 +bool kasan_multi_shot_enabled(void) +{ + return test_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags); +} +EXPORT_SYMBOL(kasan_multi_shot_enabled); + /* * Used to avoid reporting more than one KASAN bug unless kasan_multi_shot * is enabled. Note that KASAN tests effectively enable kasan_multi_shot @@ -128,7 +134,7 @@ static void report_suppress_stop(void) */ static bool report_enabled(void) { - if (test_bit(KASAN_BIT_MULTI_SHOT, &kasan_flags)) + if (kasan_multi_shot_enabled()) return true; return !test_and_set_bit(KASAN_BIT_REPORTED, &kasan_flags); } --=20 2.50.1 From nobody Sat Oct 4 20:54:34 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4A23227F003; Tue, 12 Aug 2025 13:30:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005454; cv=none; b=E/7H+mDH0w2OiUGtsRHoNEcOCPfk8RcQJhkTQwlYlcJsfqQyz3kVERLmfOx8ENe3jdNZVDBanxr1TaWQrn5DYlGBrJ2L0bYMRSojDh75p58Ddb10wtHtTgB4xSVxdJnH2+gYRMab3v2gx7QNfDKx+b2RiHlpEmPjo3sX5wCsLoE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005454; c=relaxed/simple; bh=bLAP3Uvsa+ddGeiTQivKwBmzgkLDslUubgZ6G8l99Io=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=csEAbywegd3rIvU6wmE3h/4VhBiAHb9GMQcxTriedZ9VRnGbPOxZJtJgoyAVyTmOLEROpQF5OODGi6GxLUtUflCuf9sNKEbLG0k1/5PJJ8k6PISqP6BEM+FECu2ECOlfcxW/evWMrTELk4/IluATvYmlUyoG9svZrn8idXfvxZc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Bb5zXZft; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Bb5zXZft" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1755005454; x=1786541454; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=bLAP3Uvsa+ddGeiTQivKwBmzgkLDslUubgZ6G8l99Io=; b=Bb5zXZftTPnFwAw9jCXpAHiVzVOin2kbVi3y6n9AJKRaU55oR2GStiUD lHo/cVFTrd7s1Eq846GnJG1r+bJRBzipcZzHIHZYvHE8OQNS4IdRyiUb4 Sgi83rR/j3svoFBVckjBJVfOF9O1R+iP7P0lr1dupF4qb3ZmIuq+23Ztx 9iS62FKjivRbQAhXxBFfxnXqdI5fUmSSFXSh4W0YReCD6snnRqn/7uNmX InK8NkZgq5NqfnPqs7J02i9Cl4Ox1AHPDmOLtIhAZJqZGelApYekSzkv7 OL0Oafz91fll0WeH9RtPq64XnE68fRzEhhK0wFJsROcivXBhB/Xokg1P/ w==; X-CSE-ConnectionGUID: d/cVEAm5Sa+VweNn2IJz3g== X-CSE-MsgGUID: DPPWhDX5QP2J/4WFIcqt6Q== X-IronPort-AV: E=McAfee;i="6800,10657,11520"; a="60903992" X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="60903992" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:30:53 -0700 X-CSE-ConnectionGUID: Qs0HalWXTxanbhip8kE5PA== X-CSE-MsgGUID: lUt/yXQuTg+taKHifBrAyQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="165832023" Received: from vpanait-mobl.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.245.54]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:30:21 -0700 From: Maciej Wieczor-Retman To: nathan@kernel.org, arnd@arndb.de, broonie@kernel.org, Liam.Howlett@oracle.com, urezki@gmail.com, will@kernel.org, kaleshsingh@google.com, rppt@kernel.org, leitao@debian.org, coxu@redhat.com, surenb@google.com, akpm@linux-foundation.org, luto@kernel.org, jpoimboe@kernel.org, changyuanl@google.com, hpa@zytor.com, dvyukov@google.com, kas@kernel.org, corbet@lwn.net, vincenzo.frascino@arm.com, smostafa@google.com, nick.desaulniers+lkml@gmail.com, morbo@google.com, andreyknvl@gmail.com, alexander.shishkin@linux.intel.com, thiago.bauermann@linaro.org, catalin.marinas@arm.com, ryabinin.a.a@gmail.com, jan.kiszka@siemens.com, jbohac@suse.cz, dan.j.williams@intel.com, joel.granados@kernel.org, baohua@kernel.org, kevin.brodsky@arm.com, nicolas.schier@linux.dev, pcc@google.com, andriy.shevchenko@linux.intel.com, wei.liu@kernel.org, bp@alien8.de, ada.coupriediaz@arm.com, xin@zytor.com, pankaj.gupta@amd.com, vbabka@suse.cz, glider@google.com, jgross@suse.com, kees@kernel.org, jhubbard@nvidia.com, joey.gouly@arm.com, ardb@kernel.org, thuth@redhat.com, pasha.tatashin@soleen.com, kristina.martsenko@arm.com, bigeasy@linutronix.de, maciej.wieczor-retman@intel.com, lorenzo.stoakes@oracle.com, jason.andryuk@amd.com, david@redhat.com, graf@amazon.com, wangkefeng.wang@huawei.com, ziy@nvidia.com, mark.rutland@arm.com, dave.hansen@linux.intel.com, samuel.holland@sifive.com, kbingham@kernel.org, trintaeoitogc@gmail.com, scott@os.amperecomputing.com, justinstitt@google.com, kuan-ying.lee@canonical.com, maz@kernel.org, tglx@linutronix.de, samitolvanen@google.com, mhocko@suse.com, nunodasneves@linux.microsoft.com, brgerst@gmail.com, willy@infradead.org, ubizjak@gmail.com, peterz@infradead.org, mingo@redhat.com, sohil.mehta@intel.com Cc: linux-mm@kvack.org, linux-kbuild@vger.kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org, llvm@lists.linux.dev, kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 15/18] kasan: x86: Logical bit shift for kasan_mem_to_shadow Date: Tue, 12 Aug 2025 15:23:51 +0200 Message-ID: X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" While generally tag-based KASAN adopts an arithemitc bit shift to convert a memory address to a shadow memory address, it doesn't work for all cases on x86. Testing different shadow memory offsets proved that either 4 or 5 level paging didn't work correctly or inline mode ran into issues. Thus the best working scheme is the logical bit shift and non-canonical shadow offset that x86 uses for generic KASAN, of course adjusted for the increased granularity from 8 to 16 bytes. Add an arch specific implementation of kasan_mem_to_shadow() that uses the logical bit shift. The non-canonical hook tries to calculate whether an address came from kasan_mem_to_shadow(). First it checks whether this address fits into the legal set of values possible to output from the mem to shadow function. Tie both generic and tag-based x86 KASAN modes to the address range check associated with generic KASAN. Signed-off-by: Maciej Wieczor-Retman --- Changelog v4: - Add this patch to the series. arch/x86/include/asm/kasan.h | 8 ++++++++ mm/kasan/report.c | 5 +++-- 2 files changed, 11 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/kasan.h b/arch/x86/include/asm/kasan.h index 5bf38bb836e1..f3e34a9754d2 100644 --- a/arch/x86/include/asm/kasan.h +++ b/arch/x86/include/asm/kasan.h @@ -53,6 +53,14 @@ =20 #ifdef CONFIG_KASAN_SW_TAGS =20 +static inline void *__kasan_mem_to_shadow(const void *addr) +{ + return (void *)((unsigned long)addr >> KASAN_SHADOW_SCALE_SHIFT) + + KASAN_SHADOW_OFFSET; +} + +#define kasan_mem_to_shadow(addr) __kasan_mem_to_shadow(addr) + #define __tag_shifted(tag) FIELD_PREP(GENMASK_ULL(60, 57), tag) #define __tag_reset(addr) (sign_extend64((u64)(addr), 56)) #define __tag_get(addr) ((u8)FIELD_GET(GENMASK_ULL(60, 57), (u64)addr)) diff --git a/mm/kasan/report.c b/mm/kasan/report.c index cfa2da0e2985..11c8b3ddb4cc 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -648,13 +648,14 @@ void kasan_non_canonical_hook(unsigned long addr) const char *bug_type; =20 /* - * For Generic KASAN, kasan_mem_to_shadow() uses the logical right shift + * For Generic KASAN and Software Tag-Based mode on the x86 + * architecture, kasan_mem_to_shadow() uses the logical right shift * and never overflows with the chosen KASAN_SHADOW_OFFSET values (on * both x86 and arm64). Thus, the possible shadow addresses (even for * bogus pointers) belong to a single contiguous region that is the * result of kasan_mem_to_shadow() applied to the whole address space. */ - if (IS_ENABLED(CONFIG_KASAN_GENERIC)) { + if (IS_ENABLED(CONFIG_KASAN_GENERIC) || IS_ENABLED(CONFIG_X86_64)) { if (addr < (u64)kasan_mem_to_shadow((void *)(0UL)) || addr > (u64)kasan_mem_to_shadow((void *)(~0UL))) return; --=20 2.50.1 From nobody Sat Oct 4 20:54:34 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 84364280CC9; Tue, 12 Aug 2025 13:31:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005471; cv=none; b=Y0X8mAKxLgIqcHBZF7qraRwNidmC8Im2RY7GAQwlk7447gYI6COq0OSaQ4caZS9m8JOetHsFUG2t0fQ+zKCY3Dy6MypKxqSBDL3SKH2QzpIftYAQT5q2Jj7XAPNkod4+ixCkAnwhMgyTYnZfJKnik41Dg276K4Eg0Bhaa37Ew2Y= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005471; c=relaxed/simple; bh=HiW8X4qWeN2pPHljIwWnxU5OYoff7Zyp0Lg8pvxu5HY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=f7cEDKrB63gLpUXA3kr6iKAE54TYwG36T1r4z5OHOzTL/GmmCzFs0ytnooXZkB7v2J/rWuLTjiViW5BiFpyHaMbKtL/wTIGnsdnWh6y2ZaVj8liJ29cOmm4atjxI0XJz2S2wgzFr35shtz6HR8zfXwNt8dxR/RK2bg+EzSTfH7Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=K5w7pXMm; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="K5w7pXMm" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1755005470; x=1786541470; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=HiW8X4qWeN2pPHljIwWnxU5OYoff7Zyp0Lg8pvxu5HY=; b=K5w7pXMm8m1/9sx95JwMhSoppwNScXnA+H8t9O1P55sGUKxhtkuCfhWg b1ATX2/kzcQueqq63fFBHqjjfLKNM3fucRu2/rdaglhAGSNTunaPlupZx RA13SbA/ysaA6yBKw/kJm2koHLglhjGdX+CWsOdyrpH7BNc1kIoIvauLF mPxtqEPksJZAhJdCWoUXblEuSxcItHdgJOvxwEdeby5ooAIkQ06oy5zwq 2qTSVAsyrVJ886LDxGLFvF0SCjUxGh5UpERua+QT/63EgQA9VIAkidkCu Qbcod+rVNYsrPBiqlsv3hoqvL8qGOZCNYhpXuj1SAHopSim+P3LqbM4AS Q==; X-CSE-ConnectionGUID: CevhBVW/Tqaqm0g1T2S42A== X-CSE-MsgGUID: ZC1hLSxmS7+p+HjJXD/MMw== X-IronPort-AV: E=McAfee;i="6800,10657,11520"; a="60904064" X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="60904064" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:31:09 -0700 X-CSE-ConnectionGUID: 310Nw93RRXy9OmSdy8cVzA== X-CSE-MsgGUID: xzKkOVDzSASmChbZfONGbQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="165832087" Received: from vpanait-mobl.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.245.54]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:30:43 -0700 From: Maciej Wieczor-Retman To: nathan@kernel.org, arnd@arndb.de, broonie@kernel.org, Liam.Howlett@oracle.com, urezki@gmail.com, will@kernel.org, kaleshsingh@google.com, rppt@kernel.org, leitao@debian.org, coxu@redhat.com, surenb@google.com, akpm@linux-foundation.org, luto@kernel.org, jpoimboe@kernel.org, changyuanl@google.com, hpa@zytor.com, dvyukov@google.com, kas@kernel.org, corbet@lwn.net, vincenzo.frascino@arm.com, smostafa@google.com, nick.desaulniers+lkml@gmail.com, morbo@google.com, andreyknvl@gmail.com, alexander.shishkin@linux.intel.com, thiago.bauermann@linaro.org, catalin.marinas@arm.com, ryabinin.a.a@gmail.com, jan.kiszka@siemens.com, jbohac@suse.cz, dan.j.williams@intel.com, joel.granados@kernel.org, baohua@kernel.org, kevin.brodsky@arm.com, nicolas.schier@linux.dev, pcc@google.com, andriy.shevchenko@linux.intel.com, wei.liu@kernel.org, bp@alien8.de, ada.coupriediaz@arm.com, xin@zytor.com, pankaj.gupta@amd.com, vbabka@suse.cz, glider@google.com, jgross@suse.com, kees@kernel.org, jhubbard@nvidia.com, joey.gouly@arm.com, ardb@kernel.org, thuth@redhat.com, pasha.tatashin@soleen.com, kristina.martsenko@arm.com, bigeasy@linutronix.de, maciej.wieczor-retman@intel.com, lorenzo.stoakes@oracle.com, jason.andryuk@amd.com, david@redhat.com, graf@amazon.com, wangkefeng.wang@huawei.com, ziy@nvidia.com, mark.rutland@arm.com, dave.hansen@linux.intel.com, samuel.holland@sifive.com, kbingham@kernel.org, trintaeoitogc@gmail.com, scott@os.amperecomputing.com, justinstitt@google.com, kuan-ying.lee@canonical.com, maz@kernel.org, tglx@linutronix.de, samitolvanen@google.com, mhocko@suse.com, nunodasneves@linux.microsoft.com, brgerst@gmail.com, willy@infradead.org, ubizjak@gmail.com, peterz@infradead.org, mingo@redhat.com, sohil.mehta@intel.com Cc: linux-mm@kvack.org, linux-kbuild@vger.kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org, llvm@lists.linux.dev, kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 16/18] mm: Unpoison pcpu chunks with base address tag Date: Tue, 12 Aug 2025 15:23:52 +0200 Message-ID: X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The problem presented here is related to NUMA systems and tag-based KASAN mode. It can be explained in the following points: 1. There can be more than one virtual memory chunk. 2. Chunk's base address has a tag. 3. The base address points at the first chunk and thus inherits the tag of the first chunk. 4. The subsequent chunks will be accessed with the tag from the first chunk. 5. Thus, the subsequent chunks need to have their tag set to match that of the first chunk. Refactor code by moving it into a helper in preparation for the actual fix. Signed-off-by: Maciej Wieczor-Retman --- Changelog v4: - Redo the patch message numbered list. - Do the refactoring in this patch and move additions to the next new one. Changelog v3: - Remove last version of this patch that just resets the tag on base_addr and add this patch that unpoisons all areas with the same tag instead. include/linux/kasan.h | 10 ++++++++++ mm/kasan/hw_tags.c | 11 +++++++++++ mm/kasan/shadow.c | 10 ++++++++++ mm/vmalloc.c | 4 +--- 4 files changed, 32 insertions(+), 3 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 7a2527794549..3ec432d7df9a 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -613,6 +613,13 @@ static __always_inline void kasan_poison_vmalloc(const= void *start, __kasan_poison_vmalloc(start, size); } =20 +void __kasan_unpoison_vmap_areas(struct vm_struct **vms, int nr_vms); +static __always_inline void kasan_unpoison_vmap_areas(struct vm_struct **v= ms, int nr_vms) +{ + if (kasan_enabled()) + __kasan_unpoison_vmap_areas(vms, nr_vms); +} + #else /* CONFIG_KASAN_VMALLOC */ =20 static inline void kasan_populate_early_vm_area_shadow(void *start, @@ -637,6 +644,9 @@ static inline void *kasan_unpoison_vmalloc(const void *= start, static inline void kasan_poison_vmalloc(const void *start, unsigned long s= ize) { } =20 +static inline void kasan_unpoison_vmap_areas(struct vm_struct **vms, int n= r_vms) +{ } + #endif /* CONFIG_KASAN_VMALLOC */ =20 #if (defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)) && \ diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c index 9a6927394b54..1f569df313c3 100644 --- a/mm/kasan/hw_tags.c +++ b/mm/kasan/hw_tags.c @@ -382,6 +382,17 @@ void __kasan_poison_vmalloc(const void *start, unsigne= d long size) */ } =20 +void __kasan_unpoison_vmap_areas(struct vm_struct **vms, int nr_vms) +{ + int area; + + for (area =3D 0 ; area < nr_vms ; area++) { + vms[area]->addr =3D __kasan_unpoison_vmalloc( + vms[area]->addr, vms[area]->size, + KASAN_VMALLOC_PROT_NORMAL); + } +} + #endif =20 void kasan_enable_hw_tags(void) diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c index d2c70cd2afb1..b41f74d68916 100644 --- a/mm/kasan/shadow.c +++ b/mm/kasan/shadow.c @@ -646,6 +646,16 @@ void __kasan_poison_vmalloc(const void *start, unsigne= d long size) kasan_poison(start, size, KASAN_VMALLOC_INVALID, false); } =20 +void __kasan_unpoison_vmap_areas(struct vm_struct **vms, int nr_vms) +{ + int area; + + for (area =3D 0 ; area < nr_vms ; area++) { + kasan_poison(vms[area]->addr, vms[area]->size, + arch_kasan_get_tag(vms[area]->addr), false); + } +} + #else /* CONFIG_KASAN_VMALLOC */ =20 int kasan_alloc_module_shadow(void *addr, size_t size, gfp_t gfp_mask) diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 83d666e4837a..72eecc8b087a 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -4847,9 +4847,7 @@ struct vm_struct **pcpu_get_vm_areas(const unsigned l= ong *offsets, * With hardware tag-based KASAN, marking is skipped for * non-VM_ALLOC mappings, see __kasan_unpoison_vmalloc(). */ - for (area =3D 0; area < nr_vms; area++) - vms[area]->addr =3D kasan_unpoison_vmalloc(vms[area]->addr, - vms[area]->size, KASAN_VMALLOC_PROT_NORMAL); + kasan_unpoison_vmap_areas(vms, nr_vms); =20 kfree(vas); return vms; --=20 2.50.1 From nobody Sat Oct 4 20:54:34 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4EC18284B58; Tue, 12 Aug 2025 13:31:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005493; cv=none; b=DCZm6V6On0f+tS2aioDufq6NFx4Yi7onoR/IMf1um2h1lw4T5jBJWs3nEtoRSU8Y2dY1YniO/Osa/eKSOHATlwNxnzTbCkBYrWTEHuWTQOCVS92g1a3sgsGKZ+ue8jxoXkmcdEEoFAQ8irh3i50sITSriy9NeVbiE18cTh77yuQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005493; c=relaxed/simple; bh=okYAGWwGwYjZD7NZd1rejvGMvVZcH4Qcm5xKkCEr0jA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EPcHBneIak03KJVD6hbCKDooDHq2uplmVUii3Qkyz6ztDa/cMBHfJG3fJEvXURbaNr5+BCdg7ga92q3J5HbKqLmc93xfyagcDZ60fWOPBi2RTfd2lsxukolcDWXuT5klUwxG6l/ahg9+52WlT2yAm4T1sDrvJokgnicgLWWH7tI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=N3t2UaKe; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="N3t2UaKe" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1755005493; x=1786541493; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=okYAGWwGwYjZD7NZd1rejvGMvVZcH4Qcm5xKkCEr0jA=; b=N3t2UaKeggDZWfhQEF/4UjZ8KJF2ueIh0xb15qjiRYSXbYcUB9GmqftN KV2mkph8gXJ2cGCLiX/18eJc0v48uQmF5P7YO4vfqZo/CFzNRxAvTSTiN LB8bv9RRafDVPUSqdy6ZzlUugRcidjNZ37fgNnmOws9K3FD1UlZTQotZE yL1v28RXm1B8QE5/6Y+s7Y0DG2OwdS82XhpdtdPOqGZGHDnpsNCSxY43X BLEDZ2gia5cIU1kQrnDF40PcgKmZS8jDITTI2yvgm/7GlnHZBO4xJgZwN 4lEkkuRLdYOk9FIdwVe73vJ6MG6TgzbpJWUws2xfgOl9u413Enslblv7+ Q==; X-CSE-ConnectionGUID: pS1cJ2z/Q9qWNQKR8Z4DjQ== X-CSE-MsgGUID: Y3FVhXtWSv+n3QtdbzoZ4w== X-IronPort-AV: E=McAfee;i="6800,10657,11520"; a="60904135" X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="60904135" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:31:32 -0700 X-CSE-ConnectionGUID: +neUmvM8S5Otg3Vc0KnEog== X-CSE-MsgGUID: oyi06oYNTk2MyPzECSScWQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="165832138" Received: from vpanait-mobl.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.245.54]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:31:09 -0700 From: Maciej Wieczor-Retman To: nathan@kernel.org, arnd@arndb.de, broonie@kernel.org, Liam.Howlett@oracle.com, urezki@gmail.com, will@kernel.org, kaleshsingh@google.com, rppt@kernel.org, leitao@debian.org, coxu@redhat.com, surenb@google.com, akpm@linux-foundation.org, luto@kernel.org, jpoimboe@kernel.org, changyuanl@google.com, hpa@zytor.com, dvyukov@google.com, kas@kernel.org, corbet@lwn.net, vincenzo.frascino@arm.com, smostafa@google.com, nick.desaulniers+lkml@gmail.com, morbo@google.com, andreyknvl@gmail.com, alexander.shishkin@linux.intel.com, thiago.bauermann@linaro.org, catalin.marinas@arm.com, ryabinin.a.a@gmail.com, jan.kiszka@siemens.com, jbohac@suse.cz, dan.j.williams@intel.com, joel.granados@kernel.org, baohua@kernel.org, kevin.brodsky@arm.com, nicolas.schier@linux.dev, pcc@google.com, andriy.shevchenko@linux.intel.com, wei.liu@kernel.org, bp@alien8.de, ada.coupriediaz@arm.com, xin@zytor.com, pankaj.gupta@amd.com, vbabka@suse.cz, glider@google.com, jgross@suse.com, kees@kernel.org, jhubbard@nvidia.com, joey.gouly@arm.com, ardb@kernel.org, thuth@redhat.com, pasha.tatashin@soleen.com, kristina.martsenko@arm.com, bigeasy@linutronix.de, maciej.wieczor-retman@intel.com, lorenzo.stoakes@oracle.com, jason.andryuk@amd.com, david@redhat.com, graf@amazon.com, wangkefeng.wang@huawei.com, ziy@nvidia.com, mark.rutland@arm.com, dave.hansen@linux.intel.com, samuel.holland@sifive.com, kbingham@kernel.org, trintaeoitogc@gmail.com, scott@os.amperecomputing.com, justinstitt@google.com, kuan-ying.lee@canonical.com, maz@kernel.org, tglx@linutronix.de, samitolvanen@google.com, mhocko@suse.com, nunodasneves@linux.microsoft.com, brgerst@gmail.com, willy@infradead.org, ubizjak@gmail.com, peterz@infradead.org, mingo@redhat.com, sohil.mehta@intel.com Cc: linux-mm@kvack.org, linux-kbuild@vger.kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org, llvm@lists.linux.dev, kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 17/18] mm: Unpoison vms[area] addresses with a common tag Date: Tue, 12 Aug 2025 15:23:53 +0200 Message-ID: X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The problem presented here is related to NUMA systems and tag-based KASAN mode. It can be explained in the following points: 1. There can be more than one virtual memory chunk. 2. Chunk's base address has a tag. 3. The base address points at the first chunk and thus inherits the tag of the first chunk. 4. The subsequent chunks will be accessed with the tag from the first chunk. 5. Thus, the subsequent chunks need to have their tag set to match that of the first chunk. Unpoison all vms[]->addr memory and pointers with the same tag to resolve the mismatch. Signed-off-by: Maciej Wieczor-Retman --- Changelog v4: - Move tagging the vms[]->addr to this new patch and leave refactoring there. - Comment the fix to provide some context. mm/kasan/shadow.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c index b41f74d68916..ee2488371784 100644 --- a/mm/kasan/shadow.c +++ b/mm/kasan/shadow.c @@ -646,13 +646,21 @@ void __kasan_poison_vmalloc(const void *start, unsign= ed long size) kasan_poison(start, size, KASAN_VMALLOC_INVALID, false); } =20 +/* + * A tag mismatch happens when calculating per-cpu chunk addresses, because + * they all inherit the tag from vms[0]->addr, even when nr_vms is bigger + * than 1. This is a problem because all the vms[]->addr come from separate + * allocations and have different tags so while the calculated address is + * correct the tag isn't. + */ void __kasan_unpoison_vmap_areas(struct vm_struct **vms, int nr_vms) { int area; =20 for (area =3D 0 ; area < nr_vms ; area++) { kasan_poison(vms[area]->addr, vms[area]->size, - arch_kasan_get_tag(vms[area]->addr), false); + arch_kasan_get_tag(vms[0]->addr), false); + arch_kasan_set_tag(vms[area]->addr, arch_kasan_get_tag(vms[0]->addr)); } } =20 --=20 2.50.1 From nobody Sat Oct 4 20:54:34 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 43A7127FB3E; Tue, 12 Aug 2025 13:31:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005519; cv=none; b=lOwBJFNo/JmlZQDiIUj1AKI/QBcBwtWh5+YEj3wRh6n/YPIEZiOja1P/j/4CoSva5jG5EP5mST4z/18PARL1NfRTrL95i/TPgXINggMqI7craeY8NhWt5iFbrCPCtKCvCMCoYZtppa2+/3MuqpayBVGLvoC923qsKEHcqdnq/Aw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755005519; c=relaxed/simple; bh=Vl29c6D1eR0f0U4vY2WrG9uFQFyXVuYuiDzYnN2VaZk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IhxLeKCdH/Y3xTFkWQRsHx8/VGVYbmQxtXUYexvtzLm3HjX4si7wNDZw/LU1sC0JWYFsRk19NZhkeN9Dyazm0Hsc9gsQMsR5hMYpOD78S6Mk3nSj4bPX4wR02daBbgUqwfT10QD1c6tMDPevUCKRc9L0xpfWPSqe2BLdYzzA8SM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=BvCs7dxe; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="BvCs7dxe" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1755005519; x=1786541519; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Vl29c6D1eR0f0U4vY2WrG9uFQFyXVuYuiDzYnN2VaZk=; b=BvCs7dxeahxYboCZmojBIuouvOGg4gMIZpUQQFFHWJX/z1zF9GGjmlW2 X4BCMW0vlmEnzwcMQ5IgqVyf4sC52WjFb2vjml+ZgmP4limPvPgy3lsZO qYVTW3hbaLJWxlp8rt+3C+xgaHvbdickvZNiYZQQ4KDlYNP+BZnj6ZbqR /InZUyUps7gePABqszrFwfO31sSN8Hlde1QAIVw3XpS/7XGBY6Tgr4Dag kLfMMa1Qs9cCquqj8Vy1rit1sGPlBKj5mxExc/acbBrsQwbsp0KAr77SU XpElpnNyOvwd/TxAjRITUJAdiCddpWCiAMq6UtA8+F/4bmkkMtaItW6or g==; X-CSE-ConnectionGUID: m0nCU0fBQcSod0S4QQRB+w== X-CSE-MsgGUID: 1GfOCaKmSzSQcPOvAer+1Q== X-IronPort-AV: E=McAfee;i="6800,10657,11520"; a="60904210" X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="60904210" Received: from orviesa009.jf.intel.com ([10.64.159.149]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:31:57 -0700 X-CSE-ConnectionGUID: uGwnKofIQG2MO3tLlIilYQ== X-CSE-MsgGUID: HOOQUTfdSZuSH6R+h2pTMg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.17,284,1747724400"; d="scan'208";a="165832158" Received: from vpanait-mobl.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.245.54]) by orviesa009-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 12 Aug 2025 06:31:31 -0700 From: Maciej Wieczor-Retman To: nathan@kernel.org, arnd@arndb.de, broonie@kernel.org, Liam.Howlett@oracle.com, urezki@gmail.com, will@kernel.org, kaleshsingh@google.com, rppt@kernel.org, leitao@debian.org, coxu@redhat.com, surenb@google.com, akpm@linux-foundation.org, luto@kernel.org, jpoimboe@kernel.org, changyuanl@google.com, hpa@zytor.com, dvyukov@google.com, kas@kernel.org, corbet@lwn.net, vincenzo.frascino@arm.com, smostafa@google.com, nick.desaulniers+lkml@gmail.com, morbo@google.com, andreyknvl@gmail.com, alexander.shishkin@linux.intel.com, thiago.bauermann@linaro.org, catalin.marinas@arm.com, ryabinin.a.a@gmail.com, jan.kiszka@siemens.com, jbohac@suse.cz, dan.j.williams@intel.com, joel.granados@kernel.org, baohua@kernel.org, kevin.brodsky@arm.com, nicolas.schier@linux.dev, pcc@google.com, andriy.shevchenko@linux.intel.com, wei.liu@kernel.org, bp@alien8.de, ada.coupriediaz@arm.com, xin@zytor.com, pankaj.gupta@amd.com, vbabka@suse.cz, glider@google.com, jgross@suse.com, kees@kernel.org, jhubbard@nvidia.com, joey.gouly@arm.com, ardb@kernel.org, thuth@redhat.com, pasha.tatashin@soleen.com, kristina.martsenko@arm.com, bigeasy@linutronix.de, maciej.wieczor-retman@intel.com, lorenzo.stoakes@oracle.com, jason.andryuk@amd.com, david@redhat.com, graf@amazon.com, wangkefeng.wang@huawei.com, ziy@nvidia.com, mark.rutland@arm.com, dave.hansen@linux.intel.com, samuel.holland@sifive.com, kbingham@kernel.org, trintaeoitogc@gmail.com, scott@os.amperecomputing.com, justinstitt@google.com, kuan-ying.lee@canonical.com, maz@kernel.org, tglx@linutronix.de, samitolvanen@google.com, mhocko@suse.com, nunodasneves@linux.microsoft.com, brgerst@gmail.com, willy@infradead.org, ubizjak@gmail.com, peterz@infradead.org, mingo@redhat.com, sohil.mehta@intel.com Cc: linux-mm@kvack.org, linux-kbuild@vger.kernel.org, linux-arm-kernel@lists.infradead.org, x86@kernel.org, llvm@lists.linux.dev, kasan-dev@googlegroups.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v4 18/18] x86: Make software tag-based kasan available Date: Tue, 12 Aug 2025 15:23:54 +0200 Message-ID: <414e258b828b4710966c3864aebc7a9bc598eb4f.1755004923.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.50.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Make CONFIG_KASAN_SW_TAGS available for x86 machines if they have ADDRESS_MASKING enabled (LAM) as that works similarly to Top-Byte Ignore (TBI) that allows the software tag-based mode on arm64 platform. Set scale macro based on KASAN mode: in software tag-based mode 16 bytes of memory map to one shadow byte and 8 in generic mode. Disable CONFIG_KASAN_INLINE and CONFIG_KASAN_STACK when CONFIG_KASAN_SW_TAGS is enabled on x86 until the appropriate compiler support is available. Signed-off-by: Maciej Wieczor-Retman --- Changelog v4: - Add x86 specific kasan_mem_to_shadow(). - Revert x86 to the older unsigned KASAN_SHADOW_OFFSET. Do the same to KASAN_SHADOW_START/END. - Modify scripts/gdb/linux/kasan.py to keep x86 using unsigned offset. - Disable inline and stack support when software tags are enabled on x86. Changelog v3: - Remove runtime_const from previous patch and merge the rest here. - Move scale shift definition back to header file. - Add new kasan offset for software tag based mode. - Fix patch message typo 32 -> 16, and 16 -> 8. - Update lib/Kconfig.kasan with x86 now having software tag-based support. Changelog v2: - Remove KASAN dense code. Documentation/arch/x86/x86_64/mm.rst | 6 ++++-- arch/x86/Kconfig | 4 +++- arch/x86/boot/compressed/misc.h | 1 + arch/x86/include/asm/kasan.h | 1 + arch/x86/kernel/setup.c | 2 ++ lib/Kconfig.kasan | 3 ++- scripts/gdb/linux/kasan.py | 4 ++-- 7 files changed, 15 insertions(+), 6 deletions(-) diff --git a/Documentation/arch/x86/x86_64/mm.rst b/Documentation/arch/x86/= x86_64/mm.rst index a6cf05d51bd8..ccbdbb4cda36 100644 --- a/Documentation/arch/x86/x86_64/mm.rst +++ b/Documentation/arch/x86/x86_64/mm.rst @@ -60,7 +60,8 @@ Complete virtual memory map with 4-level page tables ffffe90000000000 | -23 TB | ffffe9ffffffffff | 1 TB | ... unused= hole ffffea0000000000 | -22 TB | ffffeaffffffffff | 1 TB | virtual me= mory map (vmemmap_base) ffffeb0000000000 | -21 TB | ffffebffffffffff | 1 TB | ... unused= hole - ffffec0000000000 | -20 TB | fffffbffffffffff | 16 TB | KASAN shad= ow memory + ffffec0000000000 | -20 TB | fffffbffffffffff | 16 TB | KASAN shad= ow memory (generic mode) + fffff40000000000 | -8 TB | fffffbffffffffff | 8 TB | KASAN shad= ow memory (software tag-based mode) __________________|____________|__________________|_________|___________= _________________________________________________ | | Identical = layout to the 56-bit one from here on: @@ -130,7 +131,8 @@ Complete virtual memory map with 5-level page tables ffd2000000000000 | -11.5 PB | ffd3ffffffffffff | 0.5 PB | ... unused= hole ffd4000000000000 | -11 PB | ffd5ffffffffffff | 0.5 PB | virtual me= mory map (vmemmap_base) ffd6000000000000 | -10.5 PB | ffdeffffffffffff | 2.25 PB | ... unused= hole - ffdf000000000000 | -8.25 PB | fffffbffffffffff | ~8 PB | KASAN shad= ow memory + ffdf000000000000 | -8.25 PB | fffffbffffffffff | ~8 PB | KASAN shad= ow memory (generic mode) + ffeffc0000000000 | -6 PB | fffffbffffffffff | 4 PB | KASAN shad= ow memory (software tag-based mode) __________________|____________|__________________|_________|___________= _________________________________________________ | | Identical = layout to the 47-bit one from here on: diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index b8df57ac0f28..f44fec1190b6 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -69,6 +69,7 @@ config X86 select ARCH_CLOCKSOURCE_INIT select ARCH_CONFIGURES_CPU_MITIGATIONS select ARCH_CORRECT_STACKTRACE_ON_KRETPROBE + select ARCH_DISABLE_KASAN_INLINE if X86_64 && KASAN_SW_TAGS select ARCH_ENABLE_HUGEPAGE_MIGRATION if X86_64 && HUGETLB_PAGE && MIGRAT= ION select ARCH_ENABLE_MEMORY_HOTPLUG if X86_64 select ARCH_ENABLE_MEMORY_HOTREMOVE if MEMORY_HOTPLUG @@ -199,6 +200,7 @@ config X86 select HAVE_ARCH_JUMP_LABEL_RELATIVE select HAVE_ARCH_KASAN if X86_64 select HAVE_ARCH_KASAN_VMALLOC if X86_64 + select HAVE_ARCH_KASAN_SW_TAGS if ADDRESS_MASKING select HAVE_ARCH_KFENCE select HAVE_ARCH_KMSAN if X86_64 select HAVE_ARCH_KGDB @@ -403,7 +405,7 @@ config AUDIT_ARCH =20 config KASAN_SHADOW_OFFSET hex - depends on KASAN + default 0xeffffc0000000000 if KASAN_SW_TAGS default 0xdffffc0000000000 =20 config HAVE_INTEL_TXT diff --git a/arch/x86/boot/compressed/misc.h b/arch/x86/boot/compressed/mis= c.h index db1048621ea2..ded92b439ada 100644 --- a/arch/x86/boot/compressed/misc.h +++ b/arch/x86/boot/compressed/misc.h @@ -13,6 +13,7 @@ #undef CONFIG_PARAVIRT_SPINLOCKS #undef CONFIG_KASAN #undef CONFIG_KASAN_GENERIC +#undef CONFIG_KASAN_SW_TAGS =20 #define __NO_FORTIFY =20 diff --git a/arch/x86/include/asm/kasan.h b/arch/x86/include/asm/kasan.h index f3e34a9754d2..385f4e9daab3 100644 --- a/arch/x86/include/asm/kasan.h +++ b/arch/x86/include/asm/kasan.h @@ -7,6 +7,7 @@ #include #define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) #ifdef CONFIG_KASAN_SW_TAGS +#define KASAN_SHADOW_SCALE_SHIFT 4 =20 /* * LLVM ABI for reporting tag mismatches in inline KASAN mode. diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index 1b2edd07a3e1..5b819f84f6db 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -1207,6 +1207,8 @@ void __init setup_arch(char **cmdline_p) =20 kasan_init(); =20 + kasan_init_sw_tags(); + /* * Sync back kernel address range. * diff --git a/lib/Kconfig.kasan b/lib/Kconfig.kasan index f82889a830fa..9ddbc6aeb5d5 100644 --- a/lib/Kconfig.kasan +++ b/lib/Kconfig.kasan @@ -100,7 +100,8 @@ config KASAN_SW_TAGS =20 Requires GCC 11+ or Clang. =20 - Supported only on arm64 CPUs and relies on Top Byte Ignore. + Supported on arm64 CPUs that support Top Byte Ignore and on x86 CPUs + that support Linear Address Masking. =20 Consumes about 1/16th of available memory at kernel start and add an overhead of ~20% for dynamic allocations. diff --git a/scripts/gdb/linux/kasan.py b/scripts/gdb/linux/kasan.py index fca39968d308..4b86202b155f 100644 --- a/scripts/gdb/linux/kasan.py +++ b/scripts/gdb/linux/kasan.py @@ -7,7 +7,7 @@ # =20 import gdb -from linux import constants, mm +from linux import constants, utils, mm from ctypes import c_int64 as s64 =20 def help(): @@ -40,7 +40,7 @@ class KasanMemToShadow(gdb.Command): else: help() def kasan_mem_to_shadow(self, addr): - if constants.CONFIG_KASAN_SW_TAGS: + if constants.CONFIG_KASAN_SW_TAGS and not utils.is_target_arch('x8= 6'): addr =3D s64(addr) return (addr >> self.p_ops.KASAN_SHADOW_SCALE_SHIFT) + self.p_ops.= KASAN_SHADOW_OFFSET =20 --=20 2.50.1