From nobody Sun Dec 14 13:53:05 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 93C8421638A; Tue, 4 Feb 2025 17:34:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.20 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690491; cv=none; b=jNeOFguVb/M0Rxm/+TYyfOJhylkuZHnJW0TKBDW6WkK9v4hIeynW7bsSQ6eSAaV2awIZgKfO+bFOkC9QiXzqcz1kJu0hA1GgMalX6GfBIoNuSlfH25eikdjbVQqExicStAdhcSlTxIgEr9ll2eEkpk1FeaNwTdARVZrLXEWPcRU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690491; c=relaxed/simple; bh=LrFP/S58o6ICJkrRaOG0zWteTvtIiSjG9jGn/c23LW4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TJTLdR2gYRTC4GWVFLm4zUktZX/pTELUxZu3QIUDU35C95bAaJl9CZzAJxL8ubF1lEHs6V4lZi/2J7cD18mq/AL3SYKTkTQY5JQdgA0JofGoUkJ4vDrHAJDZAuquoXJNq0UHSDR0lfz7aGcUmSdXCwqXJp4dl4VCkBhCFYsfxIU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Ie/fh6xi; arc=none smtp.client-ip=198.175.65.20 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Ie/fh6xi" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690489; x=1770226489; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=LrFP/S58o6ICJkrRaOG0zWteTvtIiSjG9jGn/c23LW4=; b=Ie/fh6xiqA66AZcYQ4Y6orvCz4F+I7OgQtmMq3hkvXhFg0wI6qGc1M0l 16382plnKRpFu35/2VUtIw8fLA0L0AO4pOfxriHivoBuXqooHcjtcYlzU PrgKOqI0VL9F/WYaqj8pGDZo2vNjQHTazzFSjhwmWx5ST7uCaqYnYvR2S isXiBe4usojrGFk2FQ0xAMTsmwRthgr5JSGN4sFcurUYZe0yiTNErIbeR qfwabJrGAwTf3HF1clPTzspGTBHGA2gBiDT5F8gx/FIeSDi3hjnnLF5Pi vWQB8qNxEMS7npas28kV5fDS5kS1n31hvv7hQA/YNkDgZn+2KBqqC6M0S Q==; X-CSE-ConnectionGUID: evVFWeqPRvyAL/ae7HZXjw== X-CSE-MsgGUID: kSExue3dQvi6+3DisJMjlQ== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38930326" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38930326" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:34:49 -0800 X-CSE-ConnectionGUID: hsBgrPAEQzS1k8EGLwQyaA== X-CSE-MsgGUID: MO+fRiNSTeubDAn41XyOaA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147866143" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:34:37 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 01/15] kasan: Allocation enhancement for dense tag-based mode Date: Tue, 4 Feb 2025 18:33:42 +0100 Message-ID: <808cc6516f47d5f5e811d2c237983767952f3743.1738686764.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Tag-based KASAN (on arm64) works by generating a random 8-bit tag and putting it in both the top byte of the pointer (that points to the allocated memory) and into all bytes of shadow memory that correspond to the chunk of allocated regular memory. Each byte of shadow memory covers a 16 byte chunk of allocated memory - a value called KASAN granularity. This means that out-of-bounds memory accesses that happen inside the 16 bytes can't be caught. The dense mode offers reducing the tag width from 8 to 4 bits and storing two tags in one byte of shadow memory - one in the upper 4 bits of the byte and one in the lower 4. This way one byte of shadow memory can cover 32 bytes of allocated memory while still keeping the "16 bytes per one tag" granularity. The lower 4 bits of each shadow byte map bytes of memory with offsets 0-15 and the upper 4 bits map offsets 16-31. Example: The example below shows how the shadow memory looks like after allocating 48 bytes of memory in both normal tag-based mode and the dense mode. The contents of shadow memory are overlaid onto address offsets that they relate to in the allocated kernel memory. Each cell | | symbolizes one byte of shadow memory. =3D The regular tag based mode: - Randomly generated 8-bit tag equals 0xAB. - 0xFE is the tag that symbolizes unallocated memory. Shadow memory contents: | 0xAB | 0xAB | 0xAB | 0xFE | Shadow memory address offsets: 0 1 2 3 4 Allocated memory address offsets: 0 16 32 48 64 =3D The dense tag based mode: - Randomly generated 4-bit tag equals 0xC. - 0xE is the tag that symbolizes unallocated memory. Shadow memory contents: |0xC 0xC |0xC 0xE |0xE 0xE |0xE 0xE | Shadow memory address offsets: 0 1 2 3 4 Allocated memory address offsets: 0 32 64 96 128 Add a new config option and defines that can override the standard system of one tag per one shadow byte. Add alternative version of the kasan_poison() that deals with tags not being aligned to byte size in shadow memory. Signed-off-by: Maciej Wieczor-Retman --- include/linux/kasan.h | 18 ++++++++++++++++++ lib/Kconfig.kasan | 21 +++++++++++++++++++++ mm/kasan/kasan.h | 4 +--- mm/kasan/shadow.c | 33 ++++++++++++++++++++++++++++++--- 4 files changed, 70 insertions(+), 6 deletions(-) diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 03b440658817..ea0f5acd875b 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -35,6 +35,24 @@ typedef unsigned int __bitwise kasan_vmalloc_flags_t; =20 /* Software KASAN implementations use shadow memory. */ =20 +#ifdef CONFIG_KASAN_SW_TAGS_DENSE +#define KASAN_GRANULE_SHIFT (KASAN_SHADOW_SCALE_SHIFT - 1) +#define KASAN_SHADOW_SCALE_SIZE (1UL << KASAN_SHADOW_SCALE_SHIFT) +static inline u8 kasan_dense_tag(u8 tag) +{ + return (tag << KASAN_TAG_WIDTH | tag); +} +#else +#define KASAN_GRANULE_SHIFT KASAN_SHADOW_SCALE_SHIFT +#define KASAN_SHADOW_SCALE_SIZE (1UL << KASAN_GRANULE_SHIFT) +static inline u8 kasan_dense_tag(u8 tag) +{ + return tag; +} +#endif + +#define KASAN_GRANULE_SIZE (1UL << KASAN_GRANULE_SHIFT) + #ifdef CONFIG_KASAN_SW_TAGS /* This matches KASAN_TAG_INVALID. */ #define KASAN_SHADOW_INIT 0xFE diff --git a/lib/Kconfig.kasan b/lib/Kconfig.kasan index 98016e137b7f..d08b4e9bf477 100644 --- a/lib/Kconfig.kasan +++ b/lib/Kconfig.kasan @@ -19,6 +19,13 @@ config ARCH_DISABLE_KASAN_INLINE Disables both inline and stack instrumentation. Selected by architectures that do not support these instrumentation types. =20 +config ARCH_HAS_KASAN_SW_TAGS_DENSE + bool + help + Enables option to compile tag-based KASAN with densely packed tags - + two 4-bit tags per one byte of shadow memory. Set on architectures + that have 4-bit tag macros. + config CC_HAS_KASAN_GENERIC def_bool $(cc-option, -fsanitize=3Dkernel-address) =20 @@ -223,4 +230,18 @@ config KASAN_EXTRA_INFO boot parameter, it will add 8 * stack_ring_size bytes of additional memory consumption. =20 +config KASAN_SW_TAGS_DENSE + bool "Two 4-bit tags in one shadow memory byte" + depends on KASAN_SW_TAGS + depends on ARCH_HAS_KASAN_SW_TAGS_DENSE + help + Enables packing two tags into one shadow byte to half the memory usage + compared to normal tag-based mode. + + After setting this option, tag width macro is set to 4 and size macros + are adjusted based on used KASAN_SHADOW_SCALE_SHIFT. + + ARCH_HAS_KASAN_SW_TAGS_DENSE is needed for this option since the + special tag macros need to be properly set for 4-bit wide tags. + endif # KASAN diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 72da5ddcceaa..0e04c5e2c405 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -128,9 +128,7 @@ static inline bool kasan_requires_meta(void) =20 #endif /* CONFIG_KASAN_GENERIC */ =20 -#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS) -#define KASAN_GRANULE_SIZE (1UL << KASAN_SHADOW_SCALE_SHIFT) -#else +#ifdef CONFIG_KASAN_HW_TAGS #include #define KASAN_GRANULE_SIZE MTE_GRANULE_SIZE #endif diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c index d6210ca48dda..368503f54b87 100644 --- a/mm/kasan/shadow.c +++ b/mm/kasan/shadow.c @@ -123,7 +123,8 @@ EXPORT_SYMBOL(__hwasan_memcpy); =20 void kasan_poison(const void *addr, size_t size, u8 value, bool init) { - void *shadow_start, *shadow_end; + u8 *shadow_start, *shadow_end, *shadow_start_aligned, *shadow_end_aligned= , tag; + u64 addr64, addr_start_aligned, addr_end_aligned; =20 if (!kasan_arch_is_ready()) return; @@ -134,16 +135,42 @@ void kasan_poison(const void *addr, size_t size, u8 v= alue, bool init) * addresses to this function. */ addr =3D kasan_reset_tag(addr); + addr64 =3D (u64)addr; =20 - if (WARN_ON((unsigned long)addr & KASAN_GRANULE_MASK)) + if (WARN_ON(addr64 & KASAN_GRANULE_MASK)) return; if (WARN_ON(size & KASAN_GRANULE_MASK)) return; =20 shadow_start =3D kasan_mem_to_shadow(addr); shadow_end =3D kasan_mem_to_shadow(addr + size); + addr_start_aligned =3D round_up(addr64, KASAN_SHADOW_SCALE_SIZE); + addr_end_aligned =3D round_down(addr64 + size, KASAN_SHADOW_SCALE_SIZE); + shadow_start_aligned =3D kasan_mem_to_shadow((void *)addr_start_aligned); + shadow_end_aligned =3D kasan_mem_to_shadow((void *)addr_end_aligned); + + /* If size is empty just return. */ + if (!size) + return; =20 - __memset(shadow_start, value, shadow_end - shadow_start); + /* Memset the first unaligned tag in shadow memory. */ + if (addr64 % KASAN_SHADOW_SCALE_SIZE) { + tag =3D *shadow_start & KASAN_TAG_MASK; + tag |=3D value << KASAN_TAG_WIDTH; + *shadow_start =3D tag; + } + + /* Memset the middle aligned part in shadow memory. */ + tag =3D kasan_dense_tag(value); + __memset(shadow_start_aligned, tag, shadow_end_aligned - shadow_start_ali= gned); + + /* Memset the last unaligned tag in shadow memory. */ + if ((addr64 + size) % KASAN_SHADOW_SCALE_SIZE) { + tag =3D KASAN_TAG_MASK << KASAN_TAG_WIDTH; + tag &=3D *shadow_end; + tag |=3D value; + *shadow_end =3D tag; + } } EXPORT_SYMBOL_GPL(kasan_poison); =20 --=20 2.47.1 From nobody Sun Dec 14 13:53:05 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 41A5C215F7E; Tue, 4 Feb 2025 17:35:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.20 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690508; cv=none; b=LG8gxg9syT65UYrHtmeDY0c51I7gver0+eqgcYd2rmtr2FgZytU3Sg5ChXFMkk5U+LVzJNlW51qSVFDJjDFo1MP+NqlVpgsT6OZj48lSTAo9d/r7EUugnCTrlxOgrSAjnXWV37QgOHfmkS8VYfzvIqvPRqWE+/D1YaGXnes8IwA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690508; c=relaxed/simple; bh=aHGhpjph+eDn968NL6XwKtEp1WbZxR8Rwtm1kIwOPTc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TtyVm/kHC9sw1c5yXqJuCkq2PXvZrH1je9IJnNpe3BlKQps4YLVaqBQUSy4JJzb/QIYq2HCN2f3GyugH/4eMjdM0ilzQ1bDBawoPjKSHJ59SguBzhwGxq7b4QtWSLildCuO+Wbg63tIo3dug2oEkemp3DkKr4ccNlyKcGmdIlF0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Rufk0rAb; arc=none smtp.client-ip=198.175.65.20 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Rufk0rAb" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690506; x=1770226506; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=aHGhpjph+eDn968NL6XwKtEp1WbZxR8Rwtm1kIwOPTc=; b=Rufk0rAbWg2sNISQsQphLLXIzm50vQe2J5XKJ47wS9ocHJVCOz1dc2RV a3p30EuwmX6PAv9QhFiy+tNnjk2MF+gZs8I6dvJvnX/j0GESFppww3LAj s+l68OXJW4LrJXyMpA/5lG0vz70+lxlgNJ7jP1oXOPhAaNOb6FDwfhu+h TJPxPOQSC1Bsw3IWCvkNCfi4n00u/a758N6LyOqOmcTd8QS659Yz+IBBY gwa40HkYNni2MxC4OiQyjXhjFgDAzW0Y6VtvKIy9B89jOWPRPZHrgOoMc ygTZcJNK/CapE0pJod+by0Wtg02AiGo+iBg8NBxdGKnYtbSHElAutvgEK g==; X-CSE-ConnectionGUID: 901WlPTFRd++68aeK17uAQ== X-CSE-MsgGUID: EjlGCcXJTMGhoGoz6n6qFA== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38930393" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38930393" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:35:04 -0800 X-CSE-ConnectionGUID: eBbq5MWaSZONJhS+dGPi8A== X-CSE-MsgGUID: mjasgO5WTcSBmxRL24BE0w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147866217" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:34:49 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 02/15] kasan: Tag checking with dense tag-based mode Date: Tue, 4 Feb 2025 18:33:43 +0100 Message-ID: <8f790bb7e166c1ea2e5003318149eb1d7aba3596.1738686764.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In KASAN's tag-based mode (arm64) when a memory access occurs, the tag stored in the top 8 bits of the pointer is compared with tags saved in the region of the shadow memory that maps to memory the pointer points to. If any of the tags in the shadow memory region do not match the one stored in the pointer an error report is generated. With the introduction of the dense mode, tags won't necessarily occupy whole bytes of shadow memory if the previously allocated memory wasn't aligned to 32 bytes - which is the coverage of one shadow byte. Add an alternative implementation of kasan_check_range() that performs special checks on first and last bytes of shadow memory ranges if the originally allocated memory wasn't aligned to 32 bytes. Signed-off-by: Maciej Wieczor-Retman --- include/linux/kasan.h | 47 +++++++++++++++------- mm/kasan/Makefile | 3 ++ mm/kasan/dense.c | 83 +++++++++++++++++++++++++++++++++++++++ mm/kasan/kasan.h | 2 +- mm/kasan/report.c | 2 +- mm/kasan/report_sw_tags.c | 12 ++---- mm/kasan/sw_tags.c | 8 ++++ 7 files changed, 133 insertions(+), 24 deletions(-) create mode 100644 mm/kasan/dense.c diff --git a/include/linux/kasan.h b/include/linux/kasan.h index ea0f5acd875b..5a3e9bec21c2 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -33,6 +33,20 @@ typedef unsigned int __bitwise kasan_vmalloc_flags_t; =20 #include =20 +#ifndef kasan_mem_to_shadow +static inline void *kasan_mem_to_shadow(const void *addr) +{ + void *scaled; + + if (IS_ENABLED(CONFIG_KASAN_GENERIC)) + scaled =3D (void *)((unsigned long)addr >> KASAN_SHADOW_SCALE_SHIFT); + else + scaled =3D (void *)((long)addr >> KASAN_SHADOW_SCALE_SHIFT); + + return KASAN_SHADOW_OFFSET + scaled; +} +#endif + /* Software KASAN implementations use shadow memory. */ =20 #ifdef CONFIG_KASAN_SW_TAGS_DENSE @@ -53,6 +67,25 @@ static inline u8 kasan_dense_tag(u8 tag) =20 #define KASAN_GRANULE_SIZE (1UL << KASAN_GRANULE_SHIFT) =20 +#ifdef CONFIG_KASAN_SW_TAGS_DENSE +static inline u8 kasan_get_shadow_tag(const void *ptr) +{ + u8 shadow_byte =3D *(u8 *)kasan_mem_to_shadow(ptr); + unsigned long addr =3D (unsigned long)ptr; + int shift; + + shift =3D !!(addr & KASAN_GRANULE_SIZE) * KASAN_TAG_WIDTH; + shadow_byte >>=3D shift; + + return shadow_byte & KASAN_TAG_KERNEL; +} +#else +static inline u8 kasan_get_shadow_tag(const void *addr) +{ + return (*(u8 *)kasan_mem_to_shadow(addr)); +} +#endif + #ifdef CONFIG_KASAN_SW_TAGS /* This matches KASAN_TAG_INVALID. */ #define KASAN_SHADOW_INIT 0xFE @@ -73,20 +106,6 @@ extern p4d_t kasan_early_shadow_p4d[MAX_PTRS_PER_P4D]; int kasan_populate_early_shadow(const void *shadow_start, const void *shadow_end); =20 -#ifndef kasan_mem_to_shadow -static inline void *kasan_mem_to_shadow(const void *addr) -{ - void *scaled; - - if (IS_ENABLED(CONFIG_KASAN_GENERIC)) - scaled =3D (void *)((unsigned long)addr >> KASAN_SHADOW_SCALE_SHIFT); - else - scaled =3D (void *)((long)addr >> KASAN_SHADOW_SCALE_SHIFT); - - return KASAN_SHADOW_OFFSET + scaled; -} -#endif - int kasan_add_zero_shadow(void *start, unsigned long size); void kasan_remove_zero_shadow(void *start, unsigned long size); =20 diff --git a/mm/kasan/Makefile b/mm/kasan/Makefile index b88543e5c0cc..3a460abd4c18 100644 --- a/mm/kasan/Makefile +++ b/mm/kasan/Makefile @@ -5,6 +5,7 @@ KCOV_INSTRUMENT :=3D n =20 # Disable ftrace to avoid recursion. CFLAGS_REMOVE_common.o =3D $(CC_FLAGS_FTRACE) +CFLAGS_REMOVE_dense.o =3D $(CC_FLAGS_FTRACE) CFLAGS_REMOVE_generic.o =3D $(CC_FLAGS_FTRACE) CFLAGS_REMOVE_init.o =3D $(CC_FLAGS_FTRACE) CFLAGS_REMOVE_quarantine.o =3D $(CC_FLAGS_FTRACE) @@ -24,6 +25,7 @@ CC_FLAGS_KASAN_RUNTIME +=3D -fno-stack-protector CC_FLAGS_KASAN_RUNTIME +=3D -DDISABLE_BRANCH_PROFILING =20 CFLAGS_common.o :=3D $(CC_FLAGS_KASAN_RUNTIME) +CFLAGS_dense.o :=3D $(CC_FLAGS_KASAN_RUNTIME) CFLAGS_generic.o :=3D $(CC_FLAGS_KASAN_RUNTIME) CFLAGS_init.o :=3D $(CC_FLAGS_KASAN_RUNTIME) CFLAGS_quarantine.o :=3D $(CC_FLAGS_KASAN_RUNTIME) @@ -49,6 +51,7 @@ RUSTFLAGS_kasan_test_rust.o :=3D $(RUSTFLAGS_KASAN) CFLAGS_kasan_test_module.o :=3D $(CFLAGS_KASAN_TEST) =20 obj-y :=3D common.o report.o +obj-$(CONFIG_KASAN_SW_TAGS_DENSE) +=3D dense.o obj-$(CONFIG_KASAN_GENERIC) +=3D init.o generic.o report_generic.o shadow.= o quarantine.o obj-$(CONFIG_KASAN_HW_TAGS) +=3D hw_tags.o report_hw_tags.o tags.o report_= tags.o obj-$(CONFIG_KASAN_SW_TAGS) +=3D init.o report_sw_tags.o shadow.o sw_tags.= o tags.o report_tags.o diff --git a/mm/kasan/dense.c b/mm/kasan/dense.c new file mode 100644 index 000000000000..306bbbfdce29 --- /dev/null +++ b/mm/kasan/dense.c @@ -0,0 +1,83 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include "kasan.h" + +static __always_inline bool kasan_check_range_inline(const void *addr, + size_t size, bool write, + unsigned long ret_ip) +{ + u8 *shadow_first, *shadow_last, *shadow, *shadow_first_aligned, *shadow_l= ast_aligned; + u64 addr_start_aligned, addr_end_aligned; + u8 tag, kasan_granule_offset; + size_t aligned_size; + void *untagged_addr; + + if (unlikely(size =3D=3D 0)) + return true; + + if (unlikely(addr + size < addr)) + return !kasan_report(addr, size, write, ret_ip); + + tag =3D get_tag((const void *)addr); + + /* + * Ignore accesses for pointers tagged with native kernel + * pointer tag to suppress false positives caused by kmap. + * + * Some kernel code was written to account for archs that don't keep + * high memory mapped all the time, but rather map and unmap particular + * pages when needed. Instead of storing a pointer to the kernel memory, + * this code saves the address of the page structure and offset within + * that page for later use. Those pages are then mapped and unmapped + * with kmap/kunmap when necessary and virt_to_page is used to get the + * virtual address of the page. For arm64 (that keeps the high memory + * mapped all the time), kmap is turned into a page_address call. + + * The issue is that with use of the page_address + virt_to_page + * sequence the top byte value of the original pointer gets lost (gets + * set to KASAN_TAG_KERNEL). + */ + if (tag =3D=3D KASAN_TAG_KERNEL) + return true; + + untagged_addr =3D kasan_reset_tag((void *)round_down((u64)addr, KASAN_GRA= NULE_SIZE)); + if (unlikely(!addr_has_metadata(untagged_addr))) + return !kasan_report(addr, size, write, ret_ip); + + kasan_granule_offset =3D ((u64)addr & KASAN_GRANULE_MASK); + aligned_size =3D round_up(size + kasan_granule_offset, KASAN_GRANULE_SIZE= ); + shadow_first =3D kasan_mem_to_shadow(untagged_addr); + shadow_last =3D kasan_mem_to_shadow(untagged_addr + aligned_size); + addr_start_aligned =3D round_up((u64)untagged_addr, KASAN_SHADOW_SCALE_SI= ZE); + addr_end_aligned =3D round_down((u64)untagged_addr + aligned_size, KASAN_= SHADOW_SCALE_SIZE); + shadow_first_aligned =3D kasan_mem_to_shadow((void *)addr_start_aligned); + shadow_last_aligned =3D kasan_mem_to_shadow((void *)addr_end_aligned); + + /* Check the first unaligned tag in shadow memory. */ + if ((u64)untagged_addr % KASAN_SHADOW_SCALE_SIZE) { + if (unlikely((*shadow_first >> KASAN_TAG_WIDTH) !=3D tag)) + return !kasan_report(addr, size, write, ret_ip); + } + + /* Check the middle aligned part in shadow memory. */ + for (shadow =3D shadow_first_aligned; shadow < shadow_last_aligned; shado= w++) { + if (unlikely(*shadow !=3D ((tag << KASAN_TAG_WIDTH) | tag))) + return !kasan_report(addr, size, write, ret_ip); + } + + /* Check the last unaligned tag in shadow memory. */ + if (((u64)untagged_addr + aligned_size) % KASAN_SHADOW_SCALE_SIZE) { + if (unlikely((*shadow_last & KASAN_TAG_MASK) !=3D tag)) + return !kasan_report(addr, size, write, ret_ip); + } + + return true; +} + +#if IS_ENABLED(CONFIG_KASAN_SW_TAGS_DENSE) +bool kasan_check_range(const void *addr, size_t size, bool write, + unsigned long ret_ip) +{ + return kasan_check_range_inline(addr, size, write, ret_ip); +} +#endif diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index 0e04c5e2c405..d29bd0e65020 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -183,7 +183,7 @@ static inline bool kasan_requires_meta(void) #define META_BYTES_PER_BLOCK 1 #define META_BLOCKS_PER_ROW 16 #define META_BYTES_PER_ROW (META_BLOCKS_PER_ROW * META_BYTES_PER_BLOCK) -#define META_MEM_BYTES_PER_ROW (META_BYTES_PER_ROW * KASAN_GRANULE_SIZE) +#define META_MEM_BYTES_PER_ROW (META_BYTES_PER_ROW * KASAN_SHADOW_SCALE_SI= ZE) #define META_ROWS_AROUND_ADDR 2 =20 #define KASAN_STACK_DEPTH 64 diff --git a/mm/kasan/report.c b/mm/kasan/report.c index c08097715686..ee9e406b0cdb 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -436,7 +436,7 @@ static int meta_pointer_offset(const void *row, const v= oid *addr) * plus 1 byte for space. */ return 3 + (BITS_PER_LONG / 8) * 2 + - (addr - row) / KASAN_GRANULE_SIZE * 3 + 1; + (addr - row) / KASAN_SHADOW_SCALE_SIZE * 3 + 1; } =20 static void print_memory_metadata(const void *addr) diff --git a/mm/kasan/report_sw_tags.c b/mm/kasan/report_sw_tags.c index 689e94f9fe3c..1ac5c7a9011d 100644 --- a/mm/kasan/report_sw_tags.c +++ b/mm/kasan/report_sw_tags.c @@ -39,7 +39,7 @@ const void *kasan_find_first_bad_addr(const void *addr, s= ize_t size) if (!addr_has_metadata(p)) return p; =20 - while (p < end && tag =3D=3D *(u8 *)kasan_mem_to_shadow(p)) + while (p < end && tag =3D=3D kasan_get_shadow_tag(p)) p +=3D KASAN_GRANULE_SIZE; =20 return p; @@ -48,7 +48,6 @@ const void *kasan_find_first_bad_addr(const void *addr, s= ize_t size) size_t kasan_get_alloc_size(void *object, struct kmem_cache *cache) { size_t size =3D 0; - u8 *shadow; =20 /* * Skip the addr_has_metadata check, as this function only operates on @@ -59,13 +58,11 @@ size_t kasan_get_alloc_size(void *object, struct kmem_c= ache *cache) * The loop below returns 0 for freed objects, for which KASAN cannot * calculate the allocation size based on the metadata. */ - shadow =3D (u8 *)kasan_mem_to_shadow(object); while (size < cache->object_size) { - if (*shadow !=3D KASAN_TAG_INVALID) + if (kasan_get_shadow_tag(object + size) !=3D KASAN_TAG_INVALID) size +=3D KASAN_GRANULE_SIZE; else return size; - shadow++; } =20 return cache->object_size; @@ -78,9 +75,8 @@ void kasan_metadata_fetch_row(char *buffer, void *row) =20 void kasan_print_tags(u8 addr_tag, const void *addr) { - u8 *shadow =3D (u8 *)kasan_mem_to_shadow(addr); - - pr_err("Pointer tag: [%02x], memory tag: [%02x]\n", addr_tag, *shadow); + pr_err("Pointer tag: [%02x], memory tag: [%02x]\n", addr_tag, + kasan_get_shadow_tag(addr)); } =20 #ifdef CONFIG_KASAN_STACK diff --git a/mm/kasan/sw_tags.c b/mm/kasan/sw_tags.c index 32435d33583a..7a6b8ea9bf78 100644 --- a/mm/kasan/sw_tags.c +++ b/mm/kasan/sw_tags.c @@ -79,6 +79,7 @@ u8 __hwasan_generate_tag(void) } EXPORT_SYMBOL(__hwasan_generate_tag); =20 +#if !IS_ENABLED(CONFIG_KASAN_SW_TAGS_DENSE) bool kasan_check_range(const void *addr, size_t size, bool write, unsigned long ret_ip) { @@ -127,17 +128,24 @@ bool kasan_check_range(const void *addr, size_t size,= bool write, =20 return true; } +#endif =20 bool kasan_byte_accessible(const void *addr) { u8 tag =3D get_tag(addr); void *untagged_addr =3D kasan_reset_tag(addr); u8 shadow_byte; + int shift; =20 if (!addr_has_metadata(untagged_addr)) return false; =20 shadow_byte =3D READ_ONCE(*(u8 *)kasan_mem_to_shadow(untagged_addr)); + if (IS_ENABLED(CONFIG_KASAN_SW_TAGS_DENSE)) { + shift =3D !!((u64)addr & BIT(KASAN_TAG_WIDTH)) * KASAN_TAG_WIDTH; + shadow_byte =3D (shadow_byte >> shift) & KASAN_TAG_KERNEL; + } + return tag =3D=3D KASAN_TAG_KERNEL || tag =3D=3D shadow_byte; } =20 --=20 2.47.1 From nobody Sun Dec 14 13:53:05 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BE6D12165E8; Tue, 4 Feb 2025 17:35:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.20 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690519; cv=none; b=dFXOvvHKGENHWZ9ZQ7hszbmLg2My8fJCK2DT+sc/Ug9n9/6yZ3vIA92hBmyvrL9L7VNABtm0Vl2rHc9hZlmgqAtbI2iCF3TMZl95Mlb5lfg7KE7hqfooybMKZZZaXy5GPXAvZ6LFh7VchKaG2u++5MM3yojEHWXRHAh/1QKS3hg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690519; c=relaxed/simple; bh=edAJD92hFjQk8fjbvzxioatc9um2IkZNk+/tzcDM8p4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PeOta3Pc6o9wAgKOGopkSqWOiLsKs0IORo6axEEinJJS7FvS/hAHNFQhUI7VLBlnjiLaHPDD6DsuTQt10p4YUZHVpUIGlODwUiKHog7G/rXAEEmrX90qU1bDhdF21syhz3bNaAlwj1Ogr5mPhRf4BA0fwV66nBgp1qQepD4Gr/I= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=dsm4X8Hk; arc=none smtp.client-ip=198.175.65.20 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="dsm4X8Hk" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690517; x=1770226517; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=edAJD92hFjQk8fjbvzxioatc9um2IkZNk+/tzcDM8p4=; b=dsm4X8HkmGZRRVndEXeUODKR1LloYDT5lxd9vQYXP+3Kc3U3zXOoP1aM 7E0poUQovfpoB/WkFF4HBz3rr2QyGRj1Hk7WAMbMBvAuoDXMs41qEu52r 4HBZWNllNenWQlao8MwX8gMBIwEiuVjnONP5SRmAJaxKdqyaDUxmLuzKQ VBzl1cIupxsmB6kzY28oVS8inSoFDfGSXQrFatvp3qeQb2ioS9bTgl3fu jB5rA9oLRdcMehRSiBKmajUijPqcFLuoii7LjRayTuYge9jruLubMiU2k SpdUjAi4ZZGHTO0tEDAd8Uv3SToG4pDitcfFBsu9npDcAsahyzCN32rzm g==; X-CSE-ConnectionGUID: K0LzWR/FTyqTOJl3jkXdAw== X-CSE-MsgGUID: euzW0rMrRhCdGdt4lrLVag== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38930452" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38930452" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:35:14 -0800 X-CSE-ConnectionGUID: GXavZLsESKizsbio/E/Srw== X-CSE-MsgGUID: e0b52Ax9SOSm68zuxVJ9gQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147866342" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:35:02 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 03/15] kasan: Vmalloc dense tag-based mode support Date: Tue, 4 Feb 2025 18:33:44 +0100 Message-ID: X-Mailer: git-send-email 2.47.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" To use KASAN with the vmalloc allocator multiple functions are implemented that deal with full pages of memory. Many of these functions are hardcoded to deal with byte aligned shadow memory regions by using __memset(). With the introduction of the dense mode, tags won't necessarily occupy whole bytes of shadow memory if the previously allocated memory wasn't aligned to 32 bytes - which is the coverage of one shadow byte. Change __memset() calls to kasan_poison(). With dense tag-based mode enabled that will take care of any unaligned tags in shadow memory. Signed-off-by: Maciej Wieczor-Retman --- mm/kasan/kasan.h | 2 +- mm/kasan/shadow.c | 14 ++++++-------- 2 files changed, 7 insertions(+), 9 deletions(-) diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index d29bd0e65020..a56aadd51485 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -135,7 +135,7 @@ static inline bool kasan_requires_meta(void) =20 #define KASAN_GRANULE_MASK (KASAN_GRANULE_SIZE - 1) =20 -#define KASAN_MEMORY_PER_SHADOW_PAGE (KASAN_GRANULE_SIZE << PAGE_SHIFT) +#define KASAN_MEMORY_PER_SHADOW_PAGE (KASAN_SHADOW_SCALE_SIZE << PAGE_SHIF= T) =20 #ifdef CONFIG_KASAN_GENERIC #define KASAN_PAGE_FREE 0xFF /* freed page */ diff --git a/mm/kasan/shadow.c b/mm/kasan/shadow.c index 368503f54b87..94f51046e6ae 100644 --- a/mm/kasan/shadow.c +++ b/mm/kasan/shadow.c @@ -332,7 +332,7 @@ static int kasan_populate_vmalloc_pte(pte_t *ptep, unsi= gned long addr, if (!page) return -ENOMEM; =20 - __memset((void *)page, KASAN_VMALLOC_INVALID, PAGE_SIZE); + kasan_poison((void *)page, PAGE_SIZE, KASAN_VMALLOC_INVALID, false); pte =3D pfn_pte(PFN_DOWN(__pa(page)), PAGE_KERNEL); =20 spin_lock(&init_mm.page_table_lock); @@ -357,9 +357,6 @@ int kasan_populate_vmalloc(unsigned long addr, unsigned= long size) if (!is_vmalloc_or_module_addr((void *)addr)) return 0; =20 - shadow_start =3D (unsigned long)kasan_mem_to_shadow((void *)addr); - shadow_end =3D (unsigned long)kasan_mem_to_shadow((void *)addr + size); - /* * User Mode Linux maps enough shadow memory for all of virtual memory * at boot, so doesn't need to allocate more on vmalloc, just clear it. @@ -368,12 +365,12 @@ int kasan_populate_vmalloc(unsigned long addr, unsign= ed long size) * reason. */ if (IS_ENABLED(CONFIG_UML)) { - __memset((void *)shadow_start, KASAN_VMALLOC_INVALID, shadow_end - shado= w_start); + kasan_poison((void *)addr, size, KASAN_VMALLOC_INVALID, false); return 0; } =20 - shadow_start =3D PAGE_ALIGN_DOWN(shadow_start); - shadow_end =3D PAGE_ALIGN(shadow_end); + shadow_start =3D PAGE_ALIGN_DOWN((unsigned long)kasan_mem_to_shadow((void= *)addr)); + shadow_end =3D PAGE_ALIGN((unsigned long)kasan_mem_to_shadow((void *)addr= + size)); =20 ret =3D apply_to_page_range(&init_mm, shadow_start, shadow_end - shadow_start, @@ -546,7 +543,8 @@ void kasan_release_vmalloc(unsigned long start, unsigne= d long end, if (shadow_end > shadow_start) { size =3D shadow_end - shadow_start; if (IS_ENABLED(CONFIG_UML)) { - __memset(shadow_start, KASAN_SHADOW_INIT, shadow_end - shadow_start); + kasan_poison((void *)region_start, region_start - region_end, + KASAN_VMALLOC_INVALID, false); return; } apply_to_existing_page_range(&init_mm, --=20 2.47.1 From nobody Sun Dec 14 13:53:05 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 19E2321639D; Tue, 4 Feb 2025 17:35:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.20 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690528; cv=none; b=IPGVCSPKemz2Etje4MVpBaocKnU0NCr2zBxIOYCgT6cnAPqjUc9avK+88P7J4WX6QD9KCNpUHPkLEElcIhOAIHKF2eaLzdnO/WjZ0yeMaUZMu3k8pv5Ww6jPp56zXBlfrOYp1/V2FDY7zEmuIbYC/iZO4NGm8w8EUKer1zcJ4nM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690528; c=relaxed/simple; bh=7SPlp9QEiWP+nB12GdV91yF1dqiTepZdICL7iZuq4h4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=i06K4oQXEGZpNJmPrXSl/FQCA5et3tVM1kFH7KhiQUyJzddQPI8JAqR7XXIfh1ji3y/SOKRb4pZaTGpzfkTwHJfc1hW+JoVDdqh3ge08l26rIW81XZpox+KFBbPe0d/nhuFc5Xr3MTMjIka+y5cLr6/ezIceuzBm/0P2B341F6Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=mpFAEIXA; arc=none smtp.client-ip=198.175.65.20 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="mpFAEIXA" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690527; x=1770226527; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=7SPlp9QEiWP+nB12GdV91yF1dqiTepZdICL7iZuq4h4=; b=mpFAEIXAsrrIMBAcnIvCKJz0uXT/PV3jYGdsm2nGrycSggR92krYDUDV z60eDyZ65u18zUxm1kRCJcpBr3p+37+AlhMOZ0EEIoRxJ26Mb+qdvqGbc YA/Mof5ZB8+G+LYxLDwTGj4mODCPKpqA2+eQqGH050Su+kuJD7qhZrpBR 6l4dmgLhYm8P+2Yf4UJoZhByx7cJweIzinBUzOOiF2NCADNz6VVF2dPtL ENh8gKGRbgQBLcnB0plRnGQwdOa0s2XrL8oysy4Jyxi7WlC+rG2d+HdRy uMxJtn774w/BI47RG4J1YLPe5nTY7l44Vh5BRxkmYQCuZnQ1ZnZocaT0n g==; X-CSE-ConnectionGUID: 7/LRegh0TN+f0pdyn8H7DQ== X-CSE-MsgGUID: 6VXRX2c/SgS1AhAUeHEvJQ== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38930508" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38930508" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:35:25 -0800 X-CSE-ConnectionGUID: /sEOqI4kSuqYrX4FgQuozA== X-CSE-MsgGUID: zBO7dwN1RyyrRSQp4bH21Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147866447" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:35:14 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 04/15] kasan: arm64: x86: risc-v: Make special tags arch specific Date: Tue, 4 Feb 2025 18:33:45 +0100 Message-ID: X-Mailer: git-send-email 2.47.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" KASAN's tag-based mode defines multiple special tag values. They're reserved for: - Native kernel value. On arm64 it's 0xFF and it causes an early return in the tag checking function. - Invalid value. 0xFE marks an area as freed / unallocated. It's also the value that is used to initialize regions of shadow memory. - Max value. 0xFD is the highest value that can be randomly generated for a new tag. Metadata macro is also defined: - Tag width equal to 8. Tag-based mode on x86 is going to use 4 bit wide tags so all the above values need to be changed accordingly. Make tags arch specific for x86, risc-v and arm64. On x86 the values just lose the top 4 bits. Replace hardcoded kernel tag value and tag width with macros in KASAN's non-arch specific code. Signed-off-by: Maciej Wieczor-Retman Acked-by: Palmer Dabbelt # RISC-V --- MAINTAINERS | 2 +- arch/arm64/include/asm/kasan-tags.h | 9 +++++++++ arch/riscv/include/asm/kasan-tags.h | 12 ++++++++++++ arch/riscv/include/asm/kasan.h | 4 ---- arch/x86/include/asm/kasan-tags.h | 9 +++++++++ include/linux/kasan-tags.h | 12 +++++++++++- include/linux/kasan.h | 4 +++- include/linux/mm.h | 6 +++--- include/linux/page-flags-layout.h | 7 +------ 9 files changed, 49 insertions(+), 16 deletions(-) create mode 100644 arch/arm64/include/asm/kasan-tags.h create mode 100644 arch/riscv/include/asm/kasan-tags.h create mode 100644 arch/x86/include/asm/kasan-tags.h diff --git a/MAINTAINERS b/MAINTAINERS index b878ddc99f94..45671faa3b6f 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -12227,7 +12227,7 @@ L: kasan-dev@googlegroups.com S: Maintained B: https://bugzilla.kernel.org/buglist.cgi?component=3DSanitizers&product= =3DMemory%20Management F: Documentation/dev-tools/kasan.rst -F: arch/*/include/asm/*kasan.h +F: arch/*/include/asm/*kasan*.h F: arch/*/mm/kasan_init* F: include/linux/kasan*.h F: lib/Kconfig.kasan diff --git a/arch/arm64/include/asm/kasan-tags.h b/arch/arm64/include/asm/k= asan-tags.h new file mode 100644 index 000000000000..9e835da95f6b --- /dev/null +++ b/arch/arm64/include/asm/kasan-tags.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_KASAN_TAGS_H +#define __ASM_KASAN_TAGS_H + +#define KASAN_TAG_KERNEL 0xFF /* native kernel pointers tag */ + +#define KASAN_TAG_WIDTH 8 + +#endif /* ASM_KASAN_TAGS_H */ diff --git a/arch/riscv/include/asm/kasan-tags.h b/arch/riscv/include/asm/k= asan-tags.h new file mode 100644 index 000000000000..83d7dcc8af74 --- /dev/null +++ b/arch/riscv/include/asm/kasan-tags.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_KASAN_TAGS_H +#define __ASM_KASAN_TAGS_H + +#ifdef CONFIG_KASAN_SW_TAGS +#define KASAN_TAG_KERNEL 0x7f /* native kernel pointers tag */ +#endif + +#define KASAN_TAG_WIDTH 8 + +#endif /* ASM_KASAN_TAGS_H */ + diff --git a/arch/riscv/include/asm/kasan.h b/arch/riscv/include/asm/kasan.h index f6b378ba936d..27938e0d5233 100644 --- a/arch/riscv/include/asm/kasan.h +++ b/arch/riscv/include/asm/kasan.h @@ -41,10 +41,6 @@ =20 #define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) =20 -#ifdef CONFIG_KASAN_SW_TAGS -#define KASAN_TAG_KERNEL 0x7f /* native kernel pointers tag */ -#endif - #define arch_kasan_set_tag(addr, tag) __tag_set(addr, tag) #define arch_kasan_reset_tag(addr) __tag_reset(addr) #define arch_kasan_get_tag(addr) __tag_get(addr) diff --git a/arch/x86/include/asm/kasan-tags.h b/arch/x86/include/asm/kasan= -tags.h new file mode 100644 index 000000000000..68ba385bc75c --- /dev/null +++ b/arch/x86/include/asm/kasan-tags.h @@ -0,0 +1,9 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __ASM_KASAN_TAGS_H +#define __ASM_KASAN_TAGS_H + +#define KASAN_TAG_KERNEL 0xF /* native kernel pointers tag */ + +#define KASAN_TAG_WIDTH 4 + +#endif /* ASM_KASAN_TAGS_H */ diff --git a/include/linux/kasan-tags.h b/include/linux/kasan-tags.h index e07c896f95d3..b4aacfa8709b 100644 --- a/include/linux/kasan-tags.h +++ b/include/linux/kasan-tags.h @@ -2,7 +2,17 @@ #ifndef _LINUX_KASAN_TAGS_H #define _LINUX_KASAN_TAGS_H =20 -#include +#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) +#include +#endif + +#ifdef CONFIG_KASAN_SW_TAGS_DENSE +#define KASAN_TAG_WIDTH 4 +#endif + +#ifndef KASAN_TAG_WIDTH +#define KASAN_TAG_WIDTH 0 +#endif =20 #ifndef KASAN_TAG_KERNEL #define KASAN_TAG_KERNEL 0xFF /* native kernel pointers tag */ diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 5a3e9bec21c2..83146367170a 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -88,7 +88,9 @@ static inline u8 kasan_get_shadow_tag(const void *addr) =20 #ifdef CONFIG_KASAN_SW_TAGS /* This matches KASAN_TAG_INVALID. */ -#define KASAN_SHADOW_INIT 0xFE +#ifndef KASAN_SHADOW_INIT +#define KASAN_SHADOW_INIT KASAN_TAG_INVALID +#endif #else #define KASAN_SHADOW_INIT 0 #endif diff --git a/include/linux/mm.h b/include/linux/mm.h index 61fff5d34ed5..ddca2f63a5f6 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1813,7 +1813,7 @@ static inline u8 page_kasan_tag(const struct page *pa= ge) =20 if (kasan_enabled()) { tag =3D (page->flags >> KASAN_TAG_PGSHIFT) & KASAN_TAG_MASK; - tag ^=3D 0xff; + tag ^=3D KASAN_TAG_KERNEL; } =20 return tag; @@ -1826,7 +1826,7 @@ static inline void page_kasan_tag_set(struct page *pa= ge, u8 tag) if (!kasan_enabled()) return; =20 - tag ^=3D 0xff; + tag ^=3D KASAN_TAG_KERNEL; old_flags =3D READ_ONCE(page->flags); do { flags =3D old_flags; @@ -1845,7 +1845,7 @@ static inline void page_kasan_tag_reset(struct page *= page) =20 static inline u8 page_kasan_tag(const struct page *page) { - return 0xff; + return KASAN_TAG_KERNEL; } =20 static inline void page_kasan_tag_set(struct page *page, u8 tag) { } diff --git a/include/linux/page-flags-layout.h b/include/linux/page-flags-l= ayout.h index 7d79818dc065..ac3576f409ad 100644 --- a/include/linux/page-flags-layout.h +++ b/include/linux/page-flags-layout.h @@ -3,6 +3,7 @@ #define PAGE_FLAGS_LAYOUT_H =20 #include +#include #include =20 /* @@ -72,12 +73,6 @@ #define NODE_NOT_IN_PAGE_FLAGS 1 #endif =20 -#if defined(CONFIG_KASAN_SW_TAGS) || defined(CONFIG_KASAN_HW_TAGS) -#define KASAN_TAG_WIDTH 8 -#else -#define KASAN_TAG_WIDTH 0 -#endif - #ifdef CONFIG_NUMA_BALANCING #define LAST__PID_SHIFT 8 #define LAST__PID_MASK ((1 << LAST__PID_SHIFT)-1) --=20 2.47.1 From nobody Sun Dec 14 13:53:05 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4D8172163AA; Tue, 4 Feb 2025 17:35:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.20 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690541; cv=none; b=mb31CloaXqkvGnn7N9aiKEEm4Fnn7T4TQuheLpJR/udo+HvM9Yvj4KrfY3gve3NdVvHrAnULdmy5ROcHNJ+Y02K3pIU8nNxDRuPBUn06XjCAMTnEN2TlOtLUym2i/4wzI+40A9mDUCqadElgbthQJ5dGGARFAk7u7UyaKVrpnMg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690541; c=relaxed/simple; bh=1nix+N5DUro8RwKo5+pglZtqohhVbR1ohGEV2E0y1dM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qXjS4tfS/6554A6RwZkvy6XeJ7i5ZjBIIzGM3Un5wceuuXBB0/iLeud2SZRQY1ZTek4nD+yBnZNfjtyIHsznad9Env+t8U5AA+S+EgpyByPhJyFM5M5PNNG7M3ZCmP7PHZvRZot9vshQ8R83QJLzbv6qA3CutXomsVV2vquDxmk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=YrldGYiJ; arc=none smtp.client-ip=198.175.65.20 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="YrldGYiJ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690539; x=1770226539; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=1nix+N5DUro8RwKo5+pglZtqohhVbR1ohGEV2E0y1dM=; b=YrldGYiJkWVWWXF5kYu19Jo1gWt5VOhdWzSu6yg/TefEg3Qif1IhKVIi 6HAqZuM1sUQMcSmX1TmbferFMZNToJKyV/9u/ht9wPSuYaTJye5U0yztV YJEEvr9AmO2OX37iDhCQ3vP1YJbLY9zvw4oDZq0RvTNxSJmh9Mv9+94Kn uLX8TXNUFC5FQq47iZVcACvsS12swwtk9kX5c9OjHBwTbWFwez+/a3D7b btjZLGa5Eb8cCJsUwSq3M8WjFxuO8vTwm8rudi3LlHn51A71vnOKPpYsD XhIk+N0XzUvF+LazP2Unb37uB78AVEyD9BHukCZlxvdQ82KABR12QXlX2 g==; X-CSE-ConnectionGUID: /KfoAHlBTk2U/xpWpDt1Sw== X-CSE-MsgGUID: T4RgynpVSru/cJfq5lJvkw== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38930585" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38930585" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:35:38 -0800 X-CSE-ConnectionGUID: BjNSJDU0RVuIyvdxJ63vGQ== X-CSE-MsgGUID: INVQujqZTM+d6XKry25vsA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147866530" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:35:26 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 05/15] x86: Add arch specific kasan functions Date: Tue, 4 Feb 2025 18:33:46 +0100 Message-ID: <911ad4b9f001bca4c274b60144b1db80eab2015f.1738686764.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" KASAN's software tag-based mode needs multiple macros/functions to handle tag and pointer interactions - mainly to set and retrieve tags from the top bits of a pointer. Mimic functions currently used by arm64 but change the tag's position to bits [60:57] in the pointer. Signed-off-by: Maciej Wieczor-Retman --- arch/x86/include/asm/kasan.h | 32 ++++++++++++++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/kasan.h b/arch/x86/include/asm/kasan.h index de75306b932e..8829337a75fa 100644 --- a/arch/x86/include/asm/kasan.h +++ b/arch/x86/include/asm/kasan.h @@ -3,6 +3,8 @@ #define _ASM_X86_KASAN_H =20 #include +#include +#include #define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) #define KASAN_SHADOW_SCALE_SHIFT 3 =20 @@ -24,8 +26,33 @@ KASAN_SHADOW_SCALE_SHIFT))) =20 #ifndef __ASSEMBLY__ +#include +#include +#include + +#define arch_kasan_set_tag(addr, tag) __tag_set(addr, tag) +#define arch_kasan_reset_tag(addr) __tag_reset(addr) +#define arch_kasan_get_tag(addr) __tag_get(addr) + +#ifdef CONFIG_KASAN_SW_TAGS + +#define __tag_shifted(tag) FIELD_PREP(GENMASK_ULL(60, 57), tag) +#define __tag_reset(addr) (sign_extend64((u64)(addr), 56)) +#define __tag_get(addr) ((u8)FIELD_GET(GENMASK_ULL(60, 57), (u64)addr)) +#else +#define __tag_shifted(tag) 0UL +#define __tag_reset(addr) (addr) +#define __tag_get(addr) 0 +#endif /* CONFIG_KASAN_SW_TAGS */ =20 #ifdef CONFIG_KASAN + +static inline const void *__tag_set(const void *addr, u8 tag) +{ + u64 __addr =3D (u64)addr & ~__tag_shifted(KASAN_TAG_KERNEL); + return (const void *)(__addr | __tag_shifted(tag)); +} + void __init kasan_early_init(void); void __init kasan_init(void); void __init kasan_populate_shadow_for_vaddr(void *va, size_t size, int nid= ); @@ -34,8 +61,9 @@ static inline void kasan_early_init(void) { } static inline void kasan_init(void) { } static inline void kasan_populate_shadow_for_vaddr(void *va, size_t size, int nid) { } -#endif =20 -#endif +#endif /* CONFIG_KASAN */ + +#endif /* __ASSEMBLY__ */ =20 #endif --=20 2.47.1 From nobody Sun Dec 14 13:53:05 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 29C032163AA; Tue, 4 Feb 2025 17:35:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.20 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690552; cv=none; b=fyKulo3BXULFyX1gfz+vYUFiXej4JB096Dvk7ISRMI82IngcO4ZZJbDp3gbbsL311CKsLUP/i6Md43cT+XKLShrR8MOs3STo6w8OmLTK6E2/JPRllpMmB+2+14ThqsRZtyMy6cmcdqbC582022jydI7T1aP6KLhDkYELxvCjgjU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690552; c=relaxed/simple; bh=GgxfVN861pKW86Az0//GU2BTNPe6s6rSZNyJJoV8Pn4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=WHmdLjdoo0GBqcVj9T30H0Cky5OniuZiuhCcKoVF7bQCUg7xS1/4dzcrdxHX4515xLHEt6Xm5wEQFQ14oCSSfuY7qxTCNE3v1AW7gM11IQsKuX7odtQ3j+cE5DQt5wrfsDKOsuvjKbHrJ3Ql53s3SzePmImnPgoLZtXmaw6Vhrg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=jVrYBjQh; arc=none smtp.client-ip=198.175.65.20 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="jVrYBjQh" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690551; x=1770226551; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=GgxfVN861pKW86Az0//GU2BTNPe6s6rSZNyJJoV8Pn4=; b=jVrYBjQhO2QmuAwBzqwkPwrrzPU5Nwr4rnCsPXzPE3azWzRnsV67F5cE qNaKF77o2/MmWr94b1YMv4AcIUFwBJ534hDfJR/2DskqfHhFNoDehhdpX mWEhUhIbGxgFXSwt75w0F7bhUk+0/ET4eLWXewymsvIYOCG/bmzJAlO0m w+IyyhKVsrDy9S9K5UTe4J6Gi3d+Z+kFtbdZCQu2A6BehBIdrxNTLVC1j GMcXqKzSy4l5bVvz2kwUoW1XBKV6HG53eqSypQstIEht+w2kTSMfuf8lp TfZD7LaUD1OOyrZDye0efbG5oEuStbEk8Paq4DI6elvgaAIcTAG1cn0d8 g==; X-CSE-ConnectionGUID: aQ3yqOPmRuSPJyPVNNbHFA== X-CSE-MsgGUID: uyTPCFycSHKblo6IJDVjlQ== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38930640" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38930640" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:35:50 -0800 X-CSE-ConnectionGUID: EjCbNRnNQ/eG0RktDBjJIQ== X-CSE-MsgGUID: zLXQup0VQWiJeXH9lbUyzw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147866602" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:35:38 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 06/15] x86: Reset tag for virtual to physical address conversions Date: Tue, 4 Feb 2025 18:33:47 +0100 Message-ID: <80aa9a4c633502b5330c40f8b2d4da705dca92e7.1738686764.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Any place where pointer arithmetic is used to convert a virtual address into a physical one can raise errors if the virtual address is tagged. Reset the pointer's tag by sign extending the tag bits in macros that do pointer arithmetic in address conversions. There will be no change in compiled code with KASAN disabled since the compiler will optimize the __tag_reset() out. Signed-off-by: Maciej Wieczor-Retman --- arch/x86/include/asm/page.h | 17 +++++++++++++---- arch/x86/include/asm/page_64.h | 2 +- arch/x86/mm/physaddr.c | 1 + 3 files changed, 15 insertions(+), 5 deletions(-) diff --git a/arch/x86/include/asm/page.h b/arch/x86/include/asm/page.h index 1b93ff80b43b..09c3914d8ce4 100644 --- a/arch/x86/include/asm/page.h +++ b/arch/x86/include/asm/page.h @@ -7,6 +7,7 @@ #ifdef __KERNEL__ =20 #include +#include =20 #ifdef CONFIG_X86_64 #include @@ -41,7 +42,7 @@ static inline void copy_user_page(void *to, void *from, u= nsigned long vaddr, #define __pa(x) __phys_addr((unsigned long)(x)) #endif =20 -#define __pa_nodebug(x) __phys_addr_nodebug((unsigned long)(x)) +#define __pa_nodebug(x) __phys_addr_nodebug((unsigned long)(__tag_reset(x)= )) /* __pa_symbol should be used for C visible symbols. This seems to be the official gcc blessed way to do such arithmetic. */ /* @@ -65,9 +66,17 @@ static inline void copy_user_page(void *to, void *from, = unsigned long vaddr, * virt_to_page(kaddr) returns a valid pointer if and only if * virt_addr_valid(kaddr) returns true. */ -#define virt_to_page(kaddr) pfn_to_page(__pa(kaddr) >> PAGE_SHIFT) + +#ifdef CONFIG_KASAN_SW_TAGS +#define page_to_virt(x) ({ \ + __typeof__(x) __page =3D x; \ + void *__addr =3D __va(page_to_pfn((__typeof__(x))__tag_reset(__page)) << = PAGE_SHIFT); \ + (void *)__tag_set((const void *)__addr, page_kasan_tag(__page)); \ +}) +#endif +#define virt_to_page(kaddr) pfn_to_page(__pa((void *)__tag_reset(kaddr)) >= > PAGE_SHIFT) extern bool __virt_addr_valid(unsigned long kaddr); -#define virt_addr_valid(kaddr) __virt_addr_valid((unsigned long) (kaddr)) +#define virt_addr_valid(kaddr) __virt_addr_valid((unsigned long)(__tag_res= et(kaddr))) =20 static __always_inline void *pfn_to_kaddr(unsigned long pfn) { @@ -81,7 +90,7 @@ static __always_inline u64 __canonical_address(u64 vaddr,= u8 vaddr_bits) =20 static __always_inline u64 __is_canonical_address(u64 vaddr, u8 vaddr_bits) { - return __canonical_address(vaddr, vaddr_bits) =3D=3D vaddr; + return __canonical_address(vaddr, vaddr_bits) =3D=3D __tag_reset(vaddr); } =20 #endif /* __ASSEMBLY__ */ diff --git a/arch/x86/include/asm/page_64.h b/arch/x86/include/asm/page_64.h index f3d257c45225..6e24aeff36eb 100644 --- a/arch/x86/include/asm/page_64.h +++ b/arch/x86/include/asm/page_64.h @@ -33,7 +33,7 @@ static __always_inline unsigned long __phys_addr_nodebug(= unsigned long x) extern unsigned long __phys_addr(unsigned long); extern unsigned long __phys_addr_symbol(unsigned long); #else -#define __phys_addr(x) __phys_addr_nodebug(x) +#define __phys_addr(x) __phys_addr_nodebug(__tag_reset(x)) #define __phys_addr_symbol(x) \ ((unsigned long)(x) - __START_KERNEL_map + phys_base) #endif diff --git a/arch/x86/mm/physaddr.c b/arch/x86/mm/physaddr.c index fc3f3d3e2ef2..7f2b11308245 100644 --- a/arch/x86/mm/physaddr.c +++ b/arch/x86/mm/physaddr.c @@ -14,6 +14,7 @@ #ifdef CONFIG_DEBUG_VIRTUAL unsigned long __phys_addr(unsigned long x) { + x =3D __tag_reset(x); unsigned long y =3D x - __START_KERNEL_map; =20 /* use the carry flag to determine if x was < __START_KERNEL_map */ --=20 2.47.1 From nobody Sun Dec 14 13:53:05 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 97A8F216383; Tue, 4 Feb 2025 17:36:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.20 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690564; cv=none; b=T1xBSkZuW3GDYY6bHly2wGeIT69LHgICrJ4Zl6/Uo3S4vEcT7HrgruKuMtcEGEOroGoQMmSf592Odior9coUYA5YQ4nAbplSt2YKRP/Dh+uz7zs1HusMOHiyGSfneubH9orcwW9qGgnPtGE8pLDj4WudRjJqsz/CPgt16csYtCo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690564; c=relaxed/simple; bh=PewXt3mq2ZzovNzL7I8Fp3xpOP5/pryHgDbcA5gr/7A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lQ2NtiQTHbDK32UbAV9LGkWgYhnBu+Y8txmqCsMtx0/dniYLHBpTIYe7htmZFv5mj7G/63dxa26TLMFJHQIggilcRqUpA0TaiKsidD58WngZvZAneYsTgYkRDk4WqHQxHxwJ8BcV9dSPoFXq/uVl9928q8J9824lAPhZpnmkwvs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=jWwCojzY; arc=none smtp.client-ip=198.175.65.20 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="jWwCojzY" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690562; x=1770226562; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=PewXt3mq2ZzovNzL7I8Fp3xpOP5/pryHgDbcA5gr/7A=; b=jWwCojzYdr/+UtD3oziggohUdthZQHHTjWF/Sxw8vrndR08tKcZrrlB/ eHDvmqTsx80M5qj3DNc2uNiflBoQ/b4eXEPT9u/Sndd47JJTf5fYxmeVR ovT65YY5xMer9vS3bP9sqGJeEhHa4TIh2LrSGfGLtxw7wOnpomrowCuUD 9w21fw44JakFSn37Es9KzlN8rYanpfPL9vaFTVioCFyYazKZGTzW55zBR F5O4Q6Pm1ppGsRnC+6UnfwTcKeIaE4ZAcG/1GGrK/6NC4c6lG2sJcGrEG HH6p20dMuuSGaq/tiSOYzLz4Q1IaZTsItmVZBrr96mzA9WyoKKBjRzt+p w==; X-CSE-ConnectionGUID: fW3P6L61TpGZ0fae9ArqHg== X-CSE-MsgGUID: 4vtfLn6TRzSQAGKjC9Icww== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38930679" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38930679" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:36:02 -0800 X-CSE-ConnectionGUID: PpX6VWM3RaGBO1JoNfSfGA== X-CSE-MsgGUID: vYDFTil1TqeMDIrSmNQ1tw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147866647" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:35:50 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 07/15] mm: Pcpu chunk address tag reset Date: Tue, 4 Feb 2025 18:33:48 +0100 Message-ID: X-Mailer: git-send-email 2.47.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The problem presented here is related to NUMA systems and tag-based KASAN mode. Getting to it can be explained in the following points: 1. A new chunk is created with pcpu_create_chunk() and vm_structs are allocated. On systems with one NUMA node only one is allocated, but with more NUMA nodes at least a second one will be allocated too. 2. chunk->base_addr is assigned the modified value of vms[0]->addr and thus inherits the tag of this allocated structure. 3. In pcpu_alloc() for each possible cpu pcpu_chunk_addr() is executed which calculates per cpu pointers that correspond to the vms structure addresses. The calculations are based on adding an offset from a table to chunk->base_addr. Here the problem presents itself since for addresses based on vms[1] and up, the tag will be different than the ones based on vms[0] (base_addr). The tag mismatch happens and an error is reported. Reset the base_addr tag, since it will disable tag checks for pointers derived arithmetically from base_addr that would inherit its tag. Signed-off-by: Maciej Wieczor-Retman --- mm/percpu-vm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/mm/percpu-vm.c b/mm/percpu-vm.c index cd69caf6aa8d..e13750d804f7 100644 --- a/mm/percpu-vm.c +++ b/mm/percpu-vm.c @@ -347,7 +347,7 @@ static struct pcpu_chunk *pcpu_create_chunk(gfp_t gfp) } =20 chunk->data =3D vms; - chunk->base_addr =3D vms[0]->addr - pcpu_group_offsets[0]; + chunk->base_addr =3D kasan_reset_tag(vms[0]->addr) - pcpu_group_offsets[0= ]; =20 pcpu_stats_chunk_alloc(); trace_percpu_create_chunk(chunk->base_addr); --=20 2.47.1 From nobody Sun Dec 14 13:53:05 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0D691216388; Tue, 4 Feb 2025 17:36:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.20 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690577; cv=none; b=WUJ4r79scyTXviMtpz9vRJnKEb8EcshOR6VtLES94X+3lL71Cf6v8HGHJW10e7co0R0W9DzdikHk1d/GVmze5DtcO1K6q1rU5HRqPt0zmHnRyiYDcDgxMWtM4/1FSb+o65WPBk4eKg7dB2LKt87HVgb/V41qLPJbNqVlWK4Rv7Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690577; c=relaxed/simple; bh=/9iE/zrVNGKs+zQWoSO6YvnUY6c4YJpTItZxJ7K+vOY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=FFObmrnRdS4H/UtO8UT0HfycqWLb9GAwDhQQ2yKMvjdPwpuTFcaBd0VlLvOFuM/nXdCGne8NzWRLD7UWAZyU7elrZbLnUzIhxlt4Uy3DptzWAmC/QGW8b9/AD3/8OaGUjhxxd6x14Oo/cb6tEcGdvcx0gtSyWhYMXW4+Et8Ve/Y= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=VW8l/rCH; arc=none smtp.client-ip=198.175.65.20 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="VW8l/rCH" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690576; x=1770226576; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=/9iE/zrVNGKs+zQWoSO6YvnUY6c4YJpTItZxJ7K+vOY=; b=VW8l/rCHQgEsPCF63BbFyXJe7tmcE7VopuNG71RyRKwAVbVQbw+Kp8d4 TMJH9KZYqlQGg7x3ZcH1vb+X9emRWQnlKeADY1cjMDFUcqDnJm1nd3Hgn xkazmaOylHUkOAVIvgEPQp8cRnQk9kVdo+cP45wl0nQ58TtxfZSdgSQM8 6qHBi0whMI+TKh1wWT6/sdl5DaASKZSAt9adGqgNwJ6rQHzWgOh7Khtum mKRVnz9dumB99SLNBN/7FDQfCCjeVhSPIcd5dLkYLPWZrVQGv8FMB0VPd IO6zyJWIwiDKQ+55cvKmAD6ahP2mX+x+rw4WeSQKvOGmFPZozNHyohxOA Q==; X-CSE-ConnectionGUID: FOr61SjhSc+XNUnyRcGEaA== X-CSE-MsgGUID: C8zHQcGeSDSaRh/eRFa7bg== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38930746" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38930746" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:36:14 -0800 X-CSE-ConnectionGUID: +qvwGRPBT+u0hTrJ0ps4gQ== X-CSE-MsgGUID: gKQBPywxQQq2MfdmEZxhgA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147866742" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:36:02 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 08/15] x86: Physical address comparisons in fill_p*d/pte Date: Tue, 4 Feb 2025 18:33:49 +0100 Message-ID: <2c2a71ec844db597f30754dd79faf87c9de0b21f.1738686764.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Calculating page offset returns a pointer without a tag. When comparing the calculated offset to a tagged page pointer an error is raised because they are not equal. Change pointer comparisons to physical address comparisons as to avoid issues in KASAN that pointer arithmetic would create. Signed-off-by: Maciej Wieczor-Retman --- arch/x86/mm/init_64.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/x86/mm/init_64.c b/arch/x86/mm/init_64.c index ff253648706f..bb101412424a 100644 --- a/arch/x86/mm/init_64.c +++ b/arch/x86/mm/init_64.c @@ -251,7 +251,7 @@ static p4d_t *fill_p4d(pgd_t *pgd, unsigned long vaddr) if (pgd_none(*pgd)) { p4d_t *p4d =3D (p4d_t *)spp_getpage(); pgd_populate(&init_mm, pgd, p4d); - if (p4d !=3D p4d_offset(pgd, 0)) + if (__pa(p4d) !=3D __pa(p4d_offset(pgd, 0))) printk(KERN_ERR "PAGETABLE BUG #00! %p <-> %p\n", p4d, p4d_offset(pgd, 0)); } @@ -263,7 +263,7 @@ static pud_t *fill_pud(p4d_t *p4d, unsigned long vaddr) if (p4d_none(*p4d)) { pud_t *pud =3D (pud_t *)spp_getpage(); p4d_populate(&init_mm, p4d, pud); - if (pud !=3D pud_offset(p4d, 0)) + if (__pa(pud) !=3D __pa(pud_offset(p4d, 0))) printk(KERN_ERR "PAGETABLE BUG #01! %p <-> %p\n", pud, pud_offset(p4d, 0)); } @@ -275,7 +275,7 @@ static pmd_t *fill_pmd(pud_t *pud, unsigned long vaddr) if (pud_none(*pud)) { pmd_t *pmd =3D (pmd_t *) spp_getpage(); pud_populate(&init_mm, pud, pmd); - if (pmd !=3D pmd_offset(pud, 0)) + if (__pa(pmd) !=3D __pa(pmd_offset(pud, 0))) printk(KERN_ERR "PAGETABLE BUG #02! %p <-> %p\n", pmd, pmd_offset(pud, 0)); } @@ -287,7 +287,7 @@ static pte_t *fill_pte(pmd_t *pmd, unsigned long vaddr) if (pmd_none(*pmd)) { pte_t *pte =3D (pte_t *) spp_getpage(); pmd_populate_kernel(&init_mm, pmd, pte); - if (pte !=3D pte_offset_kernel(pmd, 0)) + if (__pa(pte) !=3D __pa(pte_offset_kernel(pmd, 0))) printk(KERN_ERR "PAGETABLE BUG #03!\n"); } return pte_offset_kernel(pmd, vaddr); --=20 2.47.1 From nobody Sun Dec 14 13:53:05 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3B4012163AC; Tue, 4 Feb 2025 17:36:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.20 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690588; cv=none; b=qqKocCJK/JXf+JKlUBp2YH+x+lh0ETXONXBLaYaXqO4PAiOvOlRv9xcbcW6j+0LxkFjjjcC2a9ofrwv9O/OZrf8vXanVPlkAhEBKqxf3yP+5W3mdmDHG1kc03JXb7oEoMGLZuqD0fdQHO8le0mMLeN8eiUevF8bOA/1iczJ0dMk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690588; c=relaxed/simple; bh=ZPDKrg9roFM67mEQ2fI48/uwWPYPBpbE1ZWp7rfkzQw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZnQTX6D8NBkbY9xxR375WWtT7GCyGoGzRspDCDMKmzmuA42BRW7ZpCVl4Kh1l4nGCoBhM+hAmpExdKb6hCYi+20aW+zfGkilde06aHwZUM/XEkmUjGpotpXTEnM9qZWDH2frS5aZZtU0JU8OWnXcMoXBVw/KINsuz1r/55NqNfs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=OzKLEZRc; arc=none smtp.client-ip=198.175.65.20 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="OzKLEZRc" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690587; x=1770226587; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ZPDKrg9roFM67mEQ2fI48/uwWPYPBpbE1ZWp7rfkzQw=; b=OzKLEZRcGnEafPuWzqcBa2H/6WpLfuxAPAQKLEg+q0BEnRqRSm549NYD CJd00DexTfsg5MFYeUUdGd7nX9aNaXTU57Gwcx1LKymI7cM7Sib/0jzbe Y9ZbE3nOGscJs7gB7xwfuBBre2j8iA9T2jBSfUGfhd1tvFEeSV1VqdOMM TIO8+ZF3S1VXNPkSX0eEpdl9ZQD2MckykDnD2uY2ZdZW8cra2Eh5PM1pF MGwFiR+HFhadiAKiNJL4kgU9Y2rEDnxOdKHQ06PnpATfH1MKE5WVQO55x DBY+/6GIcbm3lrik/UrpKk9ylLYZ+EsDRzDcdfhO4hR7bkYQZkXS5/ROQ Q==; X-CSE-ConnectionGUID: 6PEHu10CSJqCmwuWjtPhNA== X-CSE-MsgGUID: 5hN2IxyfSsKcu0JwTBADuA== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38930840" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38930840" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:36:26 -0800 X-CSE-ConnectionGUID: jXIRAGRVSi6AV3rBRiAebw== X-CSE-MsgGUID: yAFfDOxmTeCrIA01iFfVXA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147866806" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:36:14 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 09/15] x86: Physical address comparison in current_mm pgd check Date: Tue, 4 Feb 2025 18:33:50 +0100 Message-ID: X-Mailer: git-send-email 2.47.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" With KASAN software tag-based mode enabled PGD pointer stored in current_mm structure is tagged while the same pointer computed through __va(read_cr3_pa()) ends up with the tag space filled with ones. Use current_mm->pgd' physical address and drop the __va() so the VM_WARN_ON_ONCE can work properly and not report false positives while KASAN is enabled. Signed-off-by: Maciej Wieczor-Retman --- arch/x86/mm/tlb.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 86593d1b787d..95e3dc1fb766 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -1295,7 +1295,7 @@ bool nmi_uaccess_okay(void) if (loaded_mm !=3D current_mm) return false; =20 - VM_WARN_ON_ONCE(current_mm->pgd !=3D __va(read_cr3_pa())); + VM_WARN_ON_ONCE(__pa(current_mm->pgd) !=3D read_cr3_pa()); =20 return true; } --=20 2.47.1 From nobody Sun Dec 14 13:53:05 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A95F52163AF; Tue, 4 Feb 2025 17:36:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.20 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690602; cv=none; b=Vv3amQtm5o22aDAOZmbilrQWOFCo/x0GDMb//t50IAbYTeQbQVYKPzVmglfbrKCxwVHMTpPpCMOUFiW/X+A7ivcQ3Ov5xIZz5U4b7ETUoTFdxOiiS66HnbtahEmaOIQpm4J1k9wSY+dOj/kpJ0iQ+M5IVfIqtFDvWUxOntnCdPc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690602; c=relaxed/simple; bh=J5wZ/mcDbil10hn8JDYxev0M9sT2GI5gsQFk6+vTupM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=s9Tt9L2cllKyO11Tc5+TL8sF2lt3WwRnW1tyIBNilG2QIhUxu02vb0B/yoaqDxPCt0H9Gz1tiUmffzwcvdYY3IhBVbxwpN3e12nnWG6az4LgA/myeHhA8YxDdBO23L4/NuEIQRYetNyohVpdO4tHnn82oiHyUPY0VsRmT9Kibz4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=T/lBYmY0; arc=none smtp.client-ip=198.175.65.20 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="T/lBYmY0" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690600; x=1770226600; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=J5wZ/mcDbil10hn8JDYxev0M9sT2GI5gsQFk6+vTupM=; b=T/lBYmY04F0gTR2DdBG1G7rqG9VCDAOmjBx2xooFhvjknmdPbSa0o/Dx Jz+O6tMUV1pg7H8QJiHz7H68RY6YLGAbfnis4m2AEoPjEWQa6w8MBIhB6 dhtJrx4j8O0/orz1p3N9nzH9p5XOdoOni3CtfwrIAifzvOnudQ6Vwiwdp 7wX8l3EFrPrCbKQBdv/8QOHL+64mQoxIVYbeTA9EguGLN9TWTRS8/w0Zh u/cNHAh3JbCgAtJvwSQm8kxUD9yjlsnMm5npYS6eylscnJ/KZllQO0Mnv Zj72IALpqACWv9K+xLUTqA5pxF3ENgZqG/4hCFiuAJGt2SZOtZQNpz6J0 w==; X-CSE-ConnectionGUID: 1j8/gvxiSB6YasberLKXTA== X-CSE-MsgGUID: klFLnf7SSSqTnq63tS04eg== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38930924" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38930924" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:36:39 -0800 X-CSE-ConnectionGUID: euT5Fx0UQBaZT6g1J2Ba6Q== X-CSE-MsgGUID: VDPCZKL7T1GLAmeyS4orWQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147866863" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:36:27 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 10/15] x86: KASAN raw shadow memory PTE init Date: Tue, 4 Feb 2025 18:33:51 +0100 Message-ID: <28ddfb1694b19278405b4934f37d398794409749.1738686764.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In KASAN's generic mode the default value in shadow memory is zero. During initialization of shadow memory pages they are allocated and zeroed. In KASAN's tag-based mode the default tag for the arm64 architecture is 0xFE which corresponds to any memory that should not be accessed. On x86 (where tags are 4-bit wide instead of 8-bit wide) that tag is 0xE so during the initializations all the bytes in shadow memory pages should be filled with 0xE or 0xEE if two tags should be packed in one shadow byte. Use memblock_alloc_try_nid_raw() instead of memblock_alloc_try_nid() to avoid zeroing out the memory so it can be set with the KASAN invalid tag. Signed-off-by: Maciej Wieczor-Retman --- arch/x86/mm/kasan_init_64.c | 19 ++++++++++++++++--- include/linux/kasan.h | 25 +++++++++++++++++++++++++ mm/kasan/kasan.h | 19 ------------------- 3 files changed, 41 insertions(+), 22 deletions(-) diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c index 9dddf19a5571..55d468d83682 100644 --- a/arch/x86/mm/kasan_init_64.c +++ b/arch/x86/mm/kasan_init_64.c @@ -35,6 +35,18 @@ static __init void *early_alloc(size_t size, int nid, bo= ol should_panic) return ptr; } =20 +static __init void *early_raw_alloc(size_t size, int nid, bool should_pani= c) +{ + void *ptr =3D memblock_alloc_try_nid_raw(size, size, + __pa(MAX_DMA_ADDRESS), MEMBLOCK_ALLOC_ACCESSIBLE, nid); + + if (!ptr && should_panic) + panic("%pS: Failed to allocate page, nid=3D%d from=3D%lx\n", + (void *)_RET_IP_, nid, __pa(MAX_DMA_ADDRESS)); + + return ptr; +} + static void __init kasan_populate_pmd(pmd_t *pmd, unsigned long addr, unsigned long end, int nid) { @@ -64,8 +76,9 @@ static void __init kasan_populate_pmd(pmd_t *pmd, unsigne= d long addr, if (!pte_none(*pte)) continue; =20 - p =3D early_alloc(PAGE_SIZE, nid, true); - entry =3D pfn_pte(PFN_DOWN(__pa(p)), PAGE_KERNEL); + p =3D early_raw_alloc(PAGE_SIZE, nid, true); + memset(p, PAGE_SIZE, kasan_dense_tag(KASAN_SHADOW_INIT)); + entry =3D pfn_pte(PFN_DOWN(__pa_nodebug(p)), PAGE_KERNEL); set_pte_at(&init_mm, addr, pte, entry); } while (pte++, addr +=3D PAGE_SIZE, addr !=3D end); } @@ -437,7 +450,7 @@ void __init kasan_init(void) * it may contain some garbage. Now we can clear and write protect it, * since after the TLB flush no one should write to it. */ - memset(kasan_early_shadow_page, 0, PAGE_SIZE); + kasan_poison(kasan_early_shadow_page, PAGE_SIZE, KASAN_SHADOW_INIT, false= ); for (i =3D 0; i < PTRS_PER_PTE; i++) { pte_t pte; pgprot_t prot; diff --git a/include/linux/kasan.h b/include/linux/kasan.h index 83146367170a..af8272c74409 100644 --- a/include/linux/kasan.h +++ b/include/linux/kasan.h @@ -151,6 +151,31 @@ static __always_inline void kasan_unpoison_range(const= void *addr, size_t size) __kasan_unpoison_range(addr, size); } =20 +#ifdef CONFIG_KASAN_HW_TAGS + +static inline void kasan_poison(const void *addr, size_t size, u8 value, b= ool init) +{ + if (WARN_ON((unsigned long)addr & KASAN_GRANULE_MASK)) + return; + if (WARN_ON(size & KASAN_GRANULE_MASK)) + return; + + hw_set_mem_tag_range(kasan_reset_tag(addr), size, value, init); +} + +#else /* CONFIG_KASAN_HW_TAGS */ + +/** + * kasan_poison - mark the memory range as inaccessible + * @addr - range start address, must be aligned to KASAN_GRANULE_SIZE + * @size - range size, must be aligned to KASAN_GRANULE_SIZE + * @value - value that's written to metadata for the range + * @init - whether to initialize the memory range (only for hardware tag-b= ased) + */ +void kasan_poison(const void *addr, size_t size, u8 value, bool init); + +#endif /* CONFIG_KASAN_HW_TAGS */ + void __kasan_poison_pages(struct page *page, unsigned int order, bool init= ); static __always_inline void kasan_poison_pages(struct page *page, unsigned int order, bool init) diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h index a56aadd51485..2405477c5899 100644 --- a/mm/kasan/kasan.h +++ b/mm/kasan/kasan.h @@ -466,16 +466,6 @@ static inline u8 kasan_random_tag(void) { return 0; } =20 #ifdef CONFIG_KASAN_HW_TAGS =20 -static inline void kasan_poison(const void *addr, size_t size, u8 value, b= ool init) -{ - if (WARN_ON((unsigned long)addr & KASAN_GRANULE_MASK)) - return; - if (WARN_ON(size & KASAN_GRANULE_MASK)) - return; - - hw_set_mem_tag_range(kasan_reset_tag(addr), size, value, init); -} - static inline void kasan_unpoison(const void *addr, size_t size, bool init) { u8 tag =3D get_tag(addr); @@ -497,15 +487,6 @@ static inline bool kasan_byte_accessible(const void *a= ddr) =20 #else /* CONFIG_KASAN_HW_TAGS */ =20 -/** - * kasan_poison - mark the memory range as inaccessible - * @addr - range start address, must be aligned to KASAN_GRANULE_SIZE - * @size - range size, must be aligned to KASAN_GRANULE_SIZE - * @value - value that's written to metadata for the range - * @init - whether to initialize the memory range (only for hardware tag-b= ased) - */ -void kasan_poison(const void *addr, size_t size, u8 value, bool init); - /** * kasan_unpoison - mark the memory range as accessible * @addr - range start address, must be aligned to KASAN_GRANULE_SIZE --=20 2.47.1 From nobody Sun Dec 14 13:53:05 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C49AF2165F4; Tue, 4 Feb 2025 17:36:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.20 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690613; cv=none; b=R4qwmHRSlbjGDeq33JDQamYokpvycGO/irhqSoIeK3sbPt3nJDK7/Q3ehHAvjw2HnqIOxZjimQQfWGwdvhF3AlLJME4gkaWJXqSyYReHCEdHKZtvCx7ZmfUOVhCy2vKgQZ5ES/NyY16n4eplzaY1pDZ9+TU3qg6dj/CzpYOAtNU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690613; c=relaxed/simple; bh=EUgvSoD6ZXvZMdQY8EXK092WIVXF++/3+F17U+GxOWU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=CMuAB3lv3r3A7etTG7NFCKeO+cQEnMC0s/qAu7RUahnANh505NaYFHGKjQ7FjWxQKsuLbsKNIr3EzzbRJLW+jonW47Ny5aXN1jb4bAZ7a71D7vpRDNGIPTmcn2V9YB/lG86DyfAzv/akBff/rp0OoSFspk+yJMNgzgvbygnX0vE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=jN/G8y/b; arc=none smtp.client-ip=198.175.65.20 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="jN/G8y/b" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690612; x=1770226612; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=EUgvSoD6ZXvZMdQY8EXK092WIVXF++/3+F17U+GxOWU=; b=jN/G8y/byn5XTgdkIACw6XW7C1VohBQMyPrFtalPd9T9ZlGzT/yUb7TN 3E46uSHOlZxq0AEg8aAQjd6g4ctgIlSVqm9PmU1MAoDYUds6RlFKhgyc6 Ow/TlbC4p61EYVro48hfLmZKMbf3yurlo/FW/zNYno/IKe1TWp2j7Yq+E kLXbd6jOgTOK1Y134xkVQga8EM5TnW088inVReoH9F3DehcD5yeH+XXfA XqAvK0ilUcadBDq/DU2AzZG0dKG15CYtsV4stA7EVpG7KNbBoHUPfFS0d x0AgB/Resr/q0BoQt8CYyhP4KP+AvWgeVyMbUvz1e6ZiKOC9CiulCtOo0 g==; X-CSE-ConnectionGUID: yQMj6mn5T9GPqqzzU9jkaQ== X-CSE-MsgGUID: gpAF3kYNRJ2ZJgmKgl0Pbw== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38930994" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38930994" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:36:51 -0800 X-CSE-ConnectionGUID: P6du7V6VScafjuMIXvnbWA== X-CSE-MsgGUID: W7s9nIAMS+GBPujRL+zY2g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147866889" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:36:39 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 11/15] x86: LAM initialization Date: Tue, 4 Feb 2025 18:33:52 +0100 Message-ID: <01104816cdd0d430ac843847a8056d07b8770be0.1738686764.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" To make use of KASAN's tag based mode on x86 Linear Address Masking (LAM) needs to be enabled. To do that the 28th bit in CR4 needs to be set. Set the bit in early memory initialization. When launching secondary CPUs the LAM bit gets lost. To avoid this it needs to get added in a mask in head_64.S. The bit mask permits some bits of CR4 to pass from the primary CPU to the secondary CPUs without being cleared. Signed-off-by: Maciej Wieczor-Retman --- arch/x86/kernel/head_64.S | 3 +++ arch/x86/mm/init.c | 3 +++ 2 files changed, 6 insertions(+) diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 16752b8dfa89..7cdafcedbc70 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -199,6 +199,9 @@ SYM_INNER_LABEL(common_startup_64, SYM_L_LOCAL) * there will be no global TLB entries after the execution." */ movl $(X86_CR4_PAE | X86_CR4_LA57), %edx +#ifdef CONFIG_ADDRESS_MASKING + orl $X86_CR4_LAM_SUP, %edx +#endif #ifdef CONFIG_X86_MCE /* * Preserve CR4.MCE if the kernel will enable #MC support. diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c index eb503f53c319..4dc3679fedd1 100644 --- a/arch/x86/mm/init.c +++ b/arch/x86/mm/init.c @@ -756,6 +756,9 @@ void __init init_mem_mapping(void) probe_page_size_mask(); setup_pcid(); =20 + if (boot_cpu_has(X86_FEATURE_LAM) && IS_ENABLED(CONFIG_KASAN_SW_TAGS)) + cr4_set_bits_and_update_boot(X86_CR4_LAM_SUP); + #ifdef CONFIG_X86_64 end =3D max_pfn << PAGE_SHIFT; #else --=20 2.47.1 From nobody Sun Dec 14 13:53:05 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 15E192163AA; Tue, 4 Feb 2025 17:37:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.20 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690627; cv=none; b=suFrfrutL4nderrj8Jsb9vLA8qhUQqFj6gqeuh9FjYps6QhsAvfKLSpzt6Dt2DCqG5AvdwZcq1upS8akU1v2K4MwMLR2yO5Uy1TC/9iWxU7vJxi4rptqU53nbaBuPRDzP/748xDgYtNNXau97AnIi5lxYttF3sMcuLLp3rDJVwE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690627; c=relaxed/simple; bh=r2iiDiPZOS4/QANoocz7hbq1LBuYUYYysQaiQVX4JUg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nOrfn2Jm7lz9UDUADxwD2UmINo9nSVeM7X2MKF/n0NT/ngwq6Ibb5w3qPCHtCcQBW4FvbvjkzzTD2lF7eaIYieJSsO8M+t8LV+YHcXPihlPKCX5X+w9XHKgpmxGgJbZRb/bqgQz0e2Ym3Fo4hz35dKaCKJhiT4WwOe72YCbYqy8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=A9DnbX4Z; arc=none smtp.client-ip=198.175.65.20 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="A9DnbX4Z" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690626; x=1770226626; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=r2iiDiPZOS4/QANoocz7hbq1LBuYUYYysQaiQVX4JUg=; b=A9DnbX4Z9IwBYz3VpbeJxIgfP3l52T38fvZC/JCnHlJRKN4J+P47wTOG nWu9Pqab15e01pHqHX/3KmwEr8/EL2suF4dYHr3ixz0lwEDlKIjqp/dMw CbltiCEzVwKYJBZ6zIJ17R1KyuOr8hqVbHtmwasZ39cn4Td5J1COGQNJz i7ejcJOxS+W2mK/nUh8UfWloYT1jPRr8Uzwv2A591dzilinxCy5sRU229 KJMEmWTTjR5RwrkKbxuFBXBWMR2WzcOWUzA4EvtMHQfDbSz+gsdPP/Xmz QaDvMHhwyMGyerdQHPZp3EYAdkfG1uMweLAumqZYKKQ79/F1kElLn+Qz6 A==; X-CSE-ConnectionGUID: Jlv3wYWESBmqrrSs8UzHKw== X-CSE-MsgGUID: gq2SdZbzRdu//H0avhk1Cw== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38931048" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38931048" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:37:04 -0800 X-CSE-ConnectionGUID: F7WrZE34TJC30qp8xAUvrQ== X-CSE-MsgGUID: 1Ko21FfoQgSOsBU65eanYQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147866985" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:36:51 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 12/15] x86: Minimal SLAB alignment Date: Tue, 4 Feb 2025 18:33:53 +0100 Message-ID: <162610a0af3e04e2f42872401461b1d62ec78fbd.1738686764.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Adjust x86 minimal SLAB alignment to match KASAN granularity size. In tag-based mode the size changes to 16 bytes so the value needs to be 4. Signed-off-by: Maciej Wieczor-Retman --- arch/x86/include/asm/kasan.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/x86/include/asm/kasan.h b/arch/x86/include/asm/kasan.h index 8829337a75fa..f7a8d3763615 100644 --- a/arch/x86/include/asm/kasan.h +++ b/arch/x86/include/asm/kasan.h @@ -36,6 +36,8 @@ =20 #ifdef CONFIG_KASAN_SW_TAGS =20 +#define ARCH_SLAB_MINALIGN (1ULL << KASAN_GRANULE_SHIFT) + #define __tag_shifted(tag) FIELD_PREP(GENMASK_ULL(60, 57), tag) #define __tag_reset(addr) (sign_extend64((u64)(addr), 56)) #define __tag_get(addr) ((u8)FIELD_GET(GENMASK_ULL(60, 57), (u64)addr)) --=20 2.47.1 From nobody Sun Dec 14 13:53:05 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4276A216E08; Tue, 4 Feb 2025 17:37:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.20 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690642; cv=none; b=MWIw3GqXuGRtKLtw6DI4LJPKx9ZHiHulccSVbWBt4tkrKdJ6dTvuLvjx7s1WkMGxgVo1vMTFquuRLwYlCENSWDWy6P8lIjaMjXks4kZVixMCNGCMh4mdeGYzY+3bNcBINHrvgZKFvDGmg/d8ozxo0e1LxxQo0y6KEpvODynBOK4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690642; c=relaxed/simple; bh=tgQnPnMq4WHkStd3vJbMqoHpzLn5Ygn4fGXf7vNlMB4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=n6vPfSndX9/sIQXCOQEZQgTV6aP7roqJB6WNWGl1ZttbWOWv5h0KCpNubOUyfdwz07ANNtP5XT7oCHC27ZvMLT0idEsTXYLCniNalgv4swizYXVkP8a1kV9yTvyXe9jF5aMPBX6rzmEWH+G/DaWDAYu4lU57FlSCo4A0KGV89AA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=mpn66V7Z; arc=none smtp.client-ip=198.175.65.20 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="mpn66V7Z" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690640; x=1770226640; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=tgQnPnMq4WHkStd3vJbMqoHpzLn5Ygn4fGXf7vNlMB4=; b=mpn66V7ZCkd2BFGVuiPJv/udkt8S/bI20qdpRxKfo+QtVDNGynuO5w4K CBa443d16u0qsGIX0jQDmGYOLKaEjBw/o3xtQb/v6EzSbGd2sQzUb12AU lKv1EW8Dg/qz+IR1I+q4sf8mwQTtrN2Tk9WZtxJ375Et/rn1RWJMRTdiV 9Iy3NiJ5zxsczH9/VRjHIaHpoVmGXSXBsOH/zD/HaywjtGlG21A0p6Bu9 pj2ZsiSe9ko8Z5FrpneonRGrSc4k4OnpigLQ1aaxTo5J4R34LmJdJb84r W8n3aZifM6hGBQQo1eKzeXL0BNYM8sTXqDV6wXF+xxMjhyfTCbFgpGd6t A==; X-CSE-ConnectionGUID: TqlvJK9cS+CCqYQaN1J0Jw== X-CSE-MsgGUID: PZse8TRoTrCLEOebUF4uYA== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38931119" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38931119" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:37:19 -0800 X-CSE-ConnectionGUID: OH9xu6cPRWustG/bFhP1dw== X-CSE-MsgGUID: vAuLaG4TRpmkRprgXY7/Og== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147867096" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:37:04 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 13/15] x86: runtime_const used for KASAN_SHADOW_END Date: Tue, 4 Feb 2025 18:33:54 +0100 Message-ID: <5d0f9dbd0f7c2326229f2a1f3dcedd46842a9615.1738686764.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" On x86, generic KASAN is setup in a way that needs a single KASAN_SHADOW_OFFSET value for both 4 and 5 level paging. It's required to facilitate boot time switching and it's a compiler ABI so it can't be changed during runtime. Software tag-based mode doesn't tie shadow start and end to any linear addresses as part of the compiler ABI so it can be changed during runtime. This notion, for KASAN purposes, allows to optimize out macros such us pgtable_l5_enabled() which would otherwise be used in every single KASAN related function. Use runtime_const infrastructure with pgtable_l5_enabled() to initialize the end address of KASAN's shadow address space. It's a good choice since in software tag based mode KASAN_SHADOW_OFFSET and KASAN_SHADOW_END refer to the same value and the offset in kasan_mem_to_shadow() is a signed negative value. Setup KASAN_SHADOW_END values so that they're aligned to 4TB in 4-level paging mode and to 2PB in 5-level paging mode. Also update x86 memory map documentation. Signed-off-by: Maciej Wieczor-Retman --- Documentation/arch/x86/x86_64/mm.rst | 6 ++++-- arch/x86/Kconfig | 3 +-- arch/x86/include/asm/kasan.h | 14 +++++++++++++- arch/x86/kernel/vmlinux.lds.S | 1 + arch/x86/mm/kasan_init_64.c | 5 ++++- 5 files changed, 23 insertions(+), 6 deletions(-) diff --git a/Documentation/arch/x86/x86_64/mm.rst b/Documentation/arch/x86/= x86_64/mm.rst index 35e5e18c83d0..4e8c04d71a13 100644 --- a/Documentation/arch/x86/x86_64/mm.rst +++ b/Documentation/arch/x86/x86_64/mm.rst @@ -48,7 +48,8 @@ Complete virtual memory map with 4-level page tables ffffe90000000000 | -23 TB | ffffe9ffffffffff | 1 TB | ... unused= hole ffffea0000000000 | -22 TB | ffffeaffffffffff | 1 TB | virtual me= mory map (vmemmap_base) ffffeb0000000000 | -21 TB | ffffebffffffffff | 1 TB | ... unused= hole - ffffec0000000000 | -20 TB | fffffbffffffffff | 16 TB | KASAN shad= ow memory + ffffec0000000000 | -20 TB | fffffbffffffffff | 16 TB | KASAN shad= ow memory (generic mode) + fffff80000000000 | -8 TB | fffffc0000000000 | 4 TB | KASAN shad= ow memory (software tag-based mode) __________________|____________|__________________|_________|___________= _________________________________________________ | | Identical = layout to the 56-bit one from here on: @@ -107,7 +108,8 @@ Complete virtual memory map with 5-level page tables ffd2000000000000 | -11.5 PB | ffd3ffffffffffff | 0.5 PB | ... unused= hole ffd4000000000000 | -11 PB | ffd5ffffffffffff | 0.5 PB | virtual me= mory map (vmemmap_base) ffd6000000000000 | -10.5 PB | ffdeffffffffffff | 2.25 PB | ... unused= hole - ffdf000000000000 | -8.25 PB | fffffbffffffffff | ~8 PB | KASAN shad= ow memory + ffdf000000000000 | -8.25 PB | fffffbffffffffff | ~8 PB | KASAN shad= ow memory (generic mode) + ffe8000000000000 | -6 PB | fff0000000000000 | 2 PB | KASAN shad= ow memory (software tag-based mode) __________________|____________|__________________|_________|___________= _________________________________________________ | | Identical = layout to the 47-bit one from here on: diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 7b9a7e8f39ac..dfec7bc692d4 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -392,8 +392,7 @@ config AUDIT_ARCH =20 config KASAN_SHADOW_OFFSET hex - depends on KASAN - default 0xdffffc0000000000 + default 0xdffffc0000000000 if KASAN_GENERIC =20 config HAVE_INTEL_TXT def_bool y diff --git a/arch/x86/include/asm/kasan.h b/arch/x86/include/asm/kasan.h index f7a8d3763615..79151356d5f2 100644 --- a/arch/x86/include/asm/kasan.h +++ b/arch/x86/include/asm/kasan.h @@ -5,7 +5,7 @@ #include #include #include -#define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) + #define KASAN_SHADOW_SCALE_SHIFT 3 =20 /* @@ -14,6 +14,8 @@ * for kernel really starts from compiler's shadow offset + * 'kernel address space start' >> KASAN_SHADOW_SCALE_SHIFT */ +#ifdef CONFIG_KASAN_GENERIC +#define KASAN_SHADOW_OFFSET _AC(CONFIG_KASAN_SHADOW_OFFSET, UL) #define KASAN_SHADOW_START (KASAN_SHADOW_OFFSET + \ ((-1UL << __VIRTUAL_MASK_SHIFT) >> \ KASAN_SHADOW_SCALE_SHIFT)) @@ -24,12 +26,22 @@ #define KASAN_SHADOW_END (KASAN_SHADOW_START + \ (1ULL << (__VIRTUAL_MASK_SHIFT - \ KASAN_SHADOW_SCALE_SHIFT))) +#endif + =20 #ifndef __ASSEMBLY__ +#include #include #include #include =20 +#ifdef CONFIG_KASAN_SW_TAGS +extern unsigned long KASAN_SHADOW_END_RC; +#define KASAN_SHADOW_END runtime_const_ptr(KASAN_SHADOW_END_RC) +#define KASAN_SHADOW_OFFSET KASAN_SHADOW_END +#define KASAN_SHADOW_START (KASAN_SHADOW_END - ((UL(1)) << (__VIRTUAL_MASK= _SHIFT - KASAN_SHADOW_SCALE_SHIFT))) +#endif + #define arch_kasan_set_tag(addr, tag) __tag_set(addr, tag) #define arch_kasan_reset_tag(addr) __tag_reset(addr) #define arch_kasan_get_tag(addr) __tag_get(addr) diff --git a/arch/x86/kernel/vmlinux.lds.S b/arch/x86/kernel/vmlinux.lds.S index feb8102a9ca7..46183f7439c9 100644 --- a/arch/x86/kernel/vmlinux.lds.S +++ b/arch/x86/kernel/vmlinux.lds.S @@ -359,6 +359,7 @@ SECTIONS =20 RUNTIME_CONST_VARIABLES RUNTIME_CONST(ptr, USER_PTR_MAX) + RUNTIME_CONST(ptr, KASAN_SHADOW_END_RC) =20 . =3D ALIGN(PAGE_SIZE); =20 diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c index 55d468d83682..0f8190e0e5f6 100644 --- a/arch/x86/mm/kasan_init_64.c +++ b/arch/x86/mm/kasan_init_64.c @@ -358,6 +358,9 @@ void __init kasan_init(void) int i; =20 memcpy(early_top_pgt, init_top_pgt, sizeof(early_top_pgt)); + unsigned long KASAN_SHADOW_END_RC =3D pgtable_l5_enabled() ? 0xfff0000000= 000000 : 0xfffffc0000000000; + + runtime_const_init(ptr, KASAN_SHADOW_END_RC); =20 /* * We use the same shadow offset for 4- and 5-level paging to @@ -372,7 +375,7 @@ void __init kasan_init(void) * bunch of things like kernel code, modules, EFI mapping, etc. * We need to take extra steps to not overwrite them. */ - if (pgtable_l5_enabled()) { + if (pgtable_l5_enabled() && !IS_ENABLED(CONFIG_KASAN_SW_TAGS)) { void *ptr; =20 ptr =3D (void *)pgd_page_vaddr(*pgd_offset_k(KASAN_SHADOW_END)); --=20 2.47.1 From nobody Sun Dec 14 13:53:05 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5FD112165E8; Tue, 4 Feb 2025 17:37:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.20 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690651; cv=none; b=dnsUyelbWAjtU/ejfR+r03BFvoITr6Q8c4lLfr/M97EAW4aRjlJNzJxd4xIN6VObP2K7p5GZYAue2tsdFWh2r7AnjZGjm2pYrKnPpsKZBiJLqRigXkMRf1mjRbPqefuUNIY2PWlfa44VaqafBeoSR+Abz/WbcNfdJU8vIC9VJIQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690651; c=relaxed/simple; bh=oiRBgvokfmZMzIN+jtwGJEePgjLFxs7ade5tHmSo4go=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=puBU+/sQ0C6paUOFHKbD9ZKh/IvHqLmtBc06ez87gJKq+5Q28J3IqzYr6mvL5kWYglUrAOX6tEfQiQchgMWTmvwOL0HDKH6z/OgDVIQzK4/LDYKLbj6dJ6SjhlUG7W7JggH+maTeR9rwpj12XHbk/6soFHqMuxepPLJdmqwzwxU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=AT+emMBs; arc=none smtp.client-ip=198.175.65.20 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="AT+emMBs" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690649; x=1770226649; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=oiRBgvokfmZMzIN+jtwGJEePgjLFxs7ade5tHmSo4go=; b=AT+emMBscQCyn5fPQ0oRIf8s1+/d9RwkDxYMHXX17mDxoicd+LJWOuql geskYX11c+AAzWrHvldMohSNmfnEGiQdy+kv/pJnxUrlpFswzzsAzI0Cq jy13QgoMsIBxhruW2ZqAMoFWabJM8BDv5EC5BuIGZzwJZ4HBMe6fP2g62 R+eV2bJlg52AwWy1Ou8c5BEZMWqD6jJkWdrgGEeZjoibPrWWGT3fPxdS0 R0QO1iDNLI7pnezuNXF7v3EB5fBt3Q7HFszVJd2HgnP+JGLDm6cStpnN8 S94DXFk0kg2w/nXh7IFPyousj4GdP0EiGRz8RkhxDl8vSLWJIIjrjjjW3 A==; X-CSE-ConnectionGUID: TUiqvFdBQDmPQDh4wL0PMg== X-CSE-MsgGUID: PsWR+aN/S5KFnRReagCVAw== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38931159" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38931159" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:37:29 -0800 X-CSE-ConnectionGUID: VKIBhH62QgyxuyV1BEgDvg== X-CSE-MsgGUID: /T7gEKWsTDWNscbEm9C8IA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147867163" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:37:17 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 14/15] x86: Make software tag-based kasan available Date: Tue, 4 Feb 2025 18:33:55 +0100 Message-ID: <794a931acfb8e73e28c02932ef08bed9254f164e.1738686764.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Make CONFIG_KASAN_SW_TAGS available for x86 machines if they have ADDRESS_MASKING enabled (LAM) as that works similarly to Top-Byte Ignore (TBI) that allows the software tag-based mode on arm64 platform. Set scale macro based on KASAN mode: in software tag-based mode 32 bytes of memory map to one shadow byte and 16 in generic mode. Signed-off-by: Maciej Wieczor-Retman --- arch/x86/Kconfig | 8 ++++++++ arch/x86/boot/compressed/misc.h | 2 ++ arch/x86/include/asm/kasan.h | 2 +- arch/x86/kernel/setup.c | 2 ++ 4 files changed, 13 insertions(+), 1 deletion(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index dfec7bc692d4..afbcf27ad278 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -36,6 +36,7 @@ config X86_64 select ARCH_HAS_ELFCORE_COMPAT select ZONE_DMA32 select EXECMEM if DYNAMIC_FTRACE + select ARCH_HAS_KASAN_SW_TAGS_DENSE =20 config FORCE_DYNAMIC_FTRACE def_bool y @@ -190,6 +191,7 @@ config X86 select HAVE_ARCH_JUMP_LABEL_RELATIVE select HAVE_ARCH_KASAN if X86_64 select HAVE_ARCH_KASAN_VMALLOC if X86_64 + select HAVE_ARCH_KASAN_SW_TAGS if ADDRESS_MASKING select HAVE_ARCH_KFENCE select HAVE_ARCH_KMSAN if X86_64 select HAVE_ARCH_KGDB @@ -394,6 +396,12 @@ config KASAN_SHADOW_OFFSET hex default 0xdffffc0000000000 if KASAN_GENERIC =20 +config KASAN_SHADOW_SCALE_SHIFT + int + default 5 if KASAN_SW_TAGS_DENSE + default 4 if KASAN_SW_TAGS + default 3 + config HAVE_INTEL_TXT def_bool y depends on INTEL_IOMMU && ACPI diff --git a/arch/x86/boot/compressed/misc.h b/arch/x86/boot/compressed/mis= c.h index dd8d1a85f671..397a70558ffa 100644 --- a/arch/x86/boot/compressed/misc.h +++ b/arch/x86/boot/compressed/misc.h @@ -13,6 +13,8 @@ #undef CONFIG_PARAVIRT_SPINLOCKS #undef CONFIG_KASAN #undef CONFIG_KASAN_GENERIC +#undef CONFIG_KASAN_SW_TAGS +#undef CONFIG_KASAN_SW_TAGS_DENSE =20 #define __NO_FORTIFY =20 diff --git a/arch/x86/include/asm/kasan.h b/arch/x86/include/asm/kasan.h index 79151356d5f2..99ff4ae83bf7 100644 --- a/arch/x86/include/asm/kasan.h +++ b/arch/x86/include/asm/kasan.h @@ -6,7 +6,7 @@ #include #include =20 -#define KASAN_SHADOW_SCALE_SHIFT 3 +#define KASAN_SHADOW_SCALE_SHIFT CONFIG_KASAN_SHADOW_SCALE_SHIFT =20 /* * Compiler uses shadow offset assuming that addresses start diff --git a/arch/x86/kernel/setup.c b/arch/x86/kernel/setup.c index f1fea506e20f..c300274e205a 100644 --- a/arch/x86/kernel/setup.c +++ b/arch/x86/kernel/setup.c @@ -1121,6 +1121,8 @@ void __init setup_arch(char **cmdline_p) =20 kasan_init(); =20 + kasan_init_sw_tags(); + /* * Sync back kernel address range. * --=20 2.47.1 From nobody Sun Dec 14 13:53:05 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.20]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F3F5E2185A8; Tue, 4 Feb 2025 17:37:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.20 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690663; cv=none; b=fyU8+EKmpaGg4X7iD7tTnstD1fL/TV/wvZ6AWT59xuwJdzczcyE8ZrrbVyW50tmsRHkM9CyelmB6bJHL8pVXJAUgwF+ZegmS70lE77hczYef/3N7474MVKWVESkG4MZ/OpluG5/KWVxve5GioabyicStJMyoimu4Pdepk9FmZMY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738690663; c=relaxed/simple; bh=Ht7/TOmkT+iwiNKb8KTAnbXFoYcUy6oM0kb2gCQ20Wc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JsUOUR4nqPYmfku8p69iEh51606tztEWGm6OPp1YRBVM4mLEZwrD3nq9H7O4n5JeqvaoY2zpH/0X46RfINwWTQhNmGQeNd6eNueZdY54GKeFRQ1bFSE3Xl0Mcp/MM6oN25DuohWT2BrM0wpnnO0bN0ZSYaygdy0jFELth8+nlVk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=F7XPcWxS; arc=none smtp.client-ip=198.175.65.20 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="F7XPcWxS" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1738690662; x=1770226662; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Ht7/TOmkT+iwiNKb8KTAnbXFoYcUy6oM0kb2gCQ20Wc=; b=F7XPcWxSQup1ygaEE4vV/PWueMPDhbOr0FzR6ddIpOXvVpoZSVPg1nE/ kcG1v8fwsa+EsIZRi0rP4HHzE1wXY+tHjxZHWq+nRwAE/oSQRCCYpDqBQ yijZXa1EbEH+U5IFB4twvX2G5i6atp0HhpKOlXXXTkOnVFh/DOerx5R1F kccGlRtx0y8D6qB9vcEAKUcdYGlcHpipjgdDkBiDQ3CpHHtopKJZKABzo 4JSJ5HgXSZ+iO5rwG0DmxYGlVNK/7O9FT5/ZFJCa5KJll48xBpoV+tarO tyVf4cv8Lm7byHgk7l/qd+uz9xXXSm+3A6rvFasEDSM1DZ3vQOy+h/v6G w==; X-CSE-ConnectionGUID: w+VdoRepR26y1WNGgzMUGA== X-CSE-MsgGUID: ZXlxW/+1Q0q1VA++y681uw== X-IronPort-AV: E=McAfee;i="6700,10204,11336"; a="38931217" X-IronPort-AV: E=Sophos;i="6.13,259,1732608000"; d="scan'208";a="38931217" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by orvoesa112.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:37:41 -0800 X-CSE-ConnectionGUID: UAr5/zW7Rha56MjtNiYK0Q== X-CSE-MsgGUID: 2nPTC7p0TeC/cQLyC4Ywqw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,224,1728975600"; d="scan'208";a="147867266" Received: from mjarzebo-mobl1.ger.corp.intel.com (HELO wieczorr-mobl1.intel.com) ([10.245.244.61]) by smtpauth.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 04 Feb 2025 09:37:29 -0800 From: Maciej Wieczor-Retman To: luto@kernel.org, xin@zytor.com, kirill.shutemov@linux.intel.com, palmer@dabbelt.com, tj@kernel.org, andreyknvl@gmail.com, brgerst@gmail.com, ardb@kernel.org, dave.hansen@linux.intel.com, jgross@suse.com, will@kernel.org, akpm@linux-foundation.org, arnd@arndb.de, corbet@lwn.net, maciej.wieczor-retman@intel.com, dvyukov@google.com, richard.weiyang@gmail.com, ytcoode@gmail.com, tglx@linutronix.de, hpa@zytor.com, seanjc@google.com, paul.walmsley@sifive.com, aou@eecs.berkeley.edu, justinstitt@google.com, jason.andryuk@amd.com, glider@google.com, ubizjak@gmail.com, jannh@google.com, bhe@redhat.com, vincenzo.frascino@arm.com, rafael.j.wysocki@intel.com, ndesaulniers@google.com, mingo@redhat.com, catalin.marinas@arm.com, junichi.nomura@nec.com, nathan@kernel.org, ryabinin.a.a@gmail.com, dennis@kernel.org, bp@alien8.de, kevinloughlin@google.com, morbo@google.com, dan.j.williams@intel.com, julian.stecklina@cyberus-technology.de, peterz@infradead.org, cl@linux.com, kees@kernel.org Cc: kasan-dev@googlegroups.com, x86@kernel.org, linux-arm-kernel@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, linux-doc@vger.kernel.org Subject: [PATCH 15/15] kasan: Add mititgation and debug modes Date: Tue, 4 Feb 2025 18:33:56 +0100 Message-ID: <450a1fe078b0e07bf2e4f3098c9110c9959c6524.1738686764.git.maciej.wieczor-retman@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" With smaller memory footprint KASAN could be used in production systems. One problem is that saving stacktraces slowes memory allocation substantially - with KASAN enabled up to 90% of time spent on kmalloc() is spent on saving the stacktrace. Add mitigation mode to allow the option for running KASAN focused on performance and security. In mitigation mode disable saving stacktraces and set fault mode to always panic on KASAN error as a security mechanism. Signed-off-by: Maciej Wieczor-Retman --- lib/Kconfig.kasan | 28 ++++++++++++++++++++++++++++ mm/kasan/report.c | 4 ++++ mm/kasan/tags.c | 5 +++++ 3 files changed, 37 insertions(+) diff --git a/lib/Kconfig.kasan b/lib/Kconfig.kasan index d08b4e9bf477..6daa62b40dea 100644 --- a/lib/Kconfig.kasan +++ b/lib/Kconfig.kasan @@ -244,4 +244,32 @@ config KASAN_SW_TAGS_DENSE ARCH_HAS_KASAN_SW_TAGS_DENSE is needed for this option since the special tag macros need to be properly set for 4-bit wide tags. =20 +choice + prompt "KASAN operation mode" + default KASAN_OPERATION_DEBUG + help + Choose between the mitigation or debug operation modes. + + The first one disables stacktrace saving and enables panic on error. + Faster memory allocation but less information. The second one is the + default where KASAN operates with full functionality. + +config KASAN_OPERATION_DEBUG + bool "Debug operation mode" + depends on KASAN + help + The default mode. Full functionality and all boot parameters + available. + +config KASAN_OPERATION_MITIGATION + bool "Mitigation operation mode" + depends on KASAN + help + Operation mode dedicated at faster operation at the cost of less + information collection. Disables stacktrace saving for faster + allocations and forces panic on KASAN error to mitigate malicious + attacks. + +endchoice + endif # KASAN diff --git a/mm/kasan/report.c b/mm/kasan/report.c index ee9e406b0cdb..ae989d3bd919 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -47,7 +47,11 @@ enum kasan_arg_fault { KASAN_ARG_FAULT_PANIC_ON_WRITE, }; =20 +#ifdef CONFIG_KASAN_OPERATION_MITIGATION +static enum kasan_arg_fault kasan_arg_fault __ro_after_init =3D KASAN_ARG_= FAULT_PANIC; +#else static enum kasan_arg_fault kasan_arg_fault __ro_after_init =3D KASAN_ARG_= FAULT_DEFAULT; +#endif =20 /* kasan.fault=3Dreport/panic */ static int __init early_kasan_fault(char *arg) diff --git a/mm/kasan/tags.c b/mm/kasan/tags.c index c111d98961ed..2414cddeaaf3 100644 --- a/mm/kasan/tags.c +++ b/mm/kasan/tags.c @@ -78,6 +78,11 @@ early_param("kasan.stack_ring_size", early_kasan_flag_st= ack_ring_size); =20 void __init kasan_init_tags(void) { + if (IS_ENABLED(CONFIG_KASAN_OPERATION_MITIGATION)) { + static_branch_disable(&kasan_flag_stacktrace); + return; + } + switch (kasan_arg_stacktrace) { case KASAN_ARG_STACKTRACE_DEFAULT: /* Default is specified by kasan_flag_stacktrace definition. */ --=20 2.47.1