From nobody Wed Oct 8 06:47:47 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B66F626CE1D; Tue, 1 Jul 2025 09:59:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363946; cv=none; b=PRFjhT79eHtnsxpwFKQPEBCb3lv2S+vS1fHdug41Mwj8XRGy9zIzW9DYKuefln+JlhMSZFj7u/1I/GwLADXYfMPHZu9EMkIwHsZpPFnaWK7V0pGp41XyeUZgSuMGu+3KPeP9DhVZPmpVCJVKWmJIBQ7o2gK6r/XMJdK7OZymMmA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363946; c=relaxed/simple; bh=+sLhOPcFptW2woVp03R+FsF4pqvxAwrxx4TJdCebibo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=VzHFG/qeL269gx7kVvI0MzZgPF4EL1cb4PNVhXmm8eEivEecC1pHYcvkCpOVKZ0pyYm5yh4gUvy4rBOipw8PAEV/Jb1WwxwiSh6FV/qr96anF3iNBQu71dWdSgGm+oKpF95hlyN2dD6mZKIoLxMe1vwZGcERpfzWFK+EPzaCWoo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.helo=mgamail.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=jLPpVKeG; arc=none smtp.client-ip=198.175.65.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.helo=mgamail.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="jLPpVKeG" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1751363944; x=1782899944; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=+sLhOPcFptW2woVp03R+FsF4pqvxAwrxx4TJdCebibo=; b=jLPpVKeGWpQDg5FNXE6wPnQ8Dz2ylovxE3u4nyPwtdOofEGMOSTY3AjI 9QnQAmnAEH5Ya5dsnO+dfJA0E6dxdUWmQra7Nxla7mE0L2068rhp9aAtU 5UmKla39Cpjp48jLoO32oqyWEPjJWF/6470Yzs4gs5PxrjHs5ewKCSvI8 9u1bBebu3BbUnRR0F/UTuM5NZxv5Bc5qWyJQ2lB2/trEJsCKjB5PptWPp woRjGLHD8ZC8xadZtgLY7yiiMqRDGhtafGvTHkNQ2gaui9M2Lj0jPMEiL Xu4IB+uN0949ozStQCFQbeOJjOkpqrojQFMoiqiMET04Sd8U15L717h7U Q==; X-CSE-ConnectionGUID: Yrtkro2YQWGWdA0+s5rO7Q== X-CSE-MsgGUID: Cw5PiH9cSyWwBKrKkSyZXw== X-IronPort-AV: E=McAfee;i="6800,10657,11480"; a="71048638" X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="71048638" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Jul 2025 02:59:03 -0700 X-CSE-ConnectionGUID: BBS8SzKIRwiiHIOkHJAuLA== X-CSE-MsgGUID: CZI8LRe7T4G+FU+u70N2wg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="177390794" Received: from black.fi.intel.com ([10.237.72.28]) by fmviesa002.fm.intel.com with ESMTP; 01 Jul 2025 02:58:51 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id 2344992; Tue, 01 Jul 2025 12:58:50 +0300 (EEST) From: "Kirill A. Shutemov" To: Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Ard Biesheuvel , "Paul E. McKenney" , Josh Poimboeuf , Xiongwei Song , Xin Li , "Mike Rapoport (IBM)" , Brijesh Singh , Michael Roth , Tony Luck , Alexey Kardashevskiy , Alexander Shishkin Cc: Jonathan Corbet , Sohil Mehta , Ingo Molnar , Pawan Gupta , Daniel Sneddon , Kai Huang , Sandipan Das , Breno Leitao , Rick Edgecombe , Alexei Starovoitov , Hou Tao , Juergen Gross , Vegard Nossum , Kees Cook , Eric Biggers , Jason Gunthorpe , "Masami Hiramatsu (Google)" , Andrew Morton , Luis Chamberlain , Yuntao Wang , Rasmus Villemoes , Christophe Leroy , Tejun Heo , Changbin Du , Huang Shijie , Geert Uytterhoeven , Namhyung Kim , Arnaldo Carvalho de Melo , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-efi@vger.kernel.org, linux-mm@kvack.org, "Kirill A. Shutemov" Subject: [PATCHv8 01/17] x86/cpu: Enumerate the LASS feature bits Date: Tue, 1 Jul 2025 12:58:30 +0300 Message-ID: <20250701095849.2360685-2-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> References: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Sohil Mehta Linear Address Space Separation (LASS) is a security feature that intends to prevent malicious virtual address space accesses across user/kernel mode. Such mode based access protection already exists today with paging and features such as SMEP and SMAP. However, to enforce these protections, the processor must traverse the paging structures in memory. Malicious software can use timing information resulting from this traversal to determine details about the paging structures, and these details may also be used to determine the layout of the kernel memory. The LASS mechanism provides the same mode-based protections as paging but without traversing the paging structures. Because the protections enforced by LASS are applied before paging, software will not be able to derive paging-based timing information from the various caching structures such as the TLBs, mid-level caches, page walker, data caches, etc. LASS enforcement relies on the typical kernel implementation to divide the 64-bit virtual address space into two halves: Addr[63]=3D0 -> User address space Addr[63]=3D1 -> Kernel address space Any data access or code execution across address spaces typically results in a #GP fault. The LASS enforcement for kernel data access is dependent on CR4.SMAP being set. The enforcement can be disabled by toggling the RFLAGS.AC bit similar to SMAP. Define the CPU feature bits to enumerate this feature and include feature dependencies to reflect the same. LASS provides protection against a class of speculative attacks, such as SLAM[1]. Add the "lass" flag to /proc/cpuinfo to indicate that the feature is supported by hardware and enabled by the kernel. This allows userspace to determine if the setup is secure against such attacks. [1] https://download.vusec.net/papers/slam_sp24.pdf Co-developed-by: Yian Chen Signed-off-by: Yian Chen Signed-off-by: Sohil Mehta Signed-off-by: Alexander Shishkin Signed-off-by: Kirill A. Shutemov Reviewed-by: Borislav Petkov (AMD) Reviewed-by: Xin Li (Intel) --- arch/x86/Kconfig.cpufeatures | 4 ++++ arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/uapi/asm/processor-flags.h | 2 ++ arch/x86/kernel/cpu/cpuid-deps.c | 1 + tools/arch/x86/include/asm/cpufeatures.h | 1 + 5 files changed, 9 insertions(+) diff --git a/arch/x86/Kconfig.cpufeatures b/arch/x86/Kconfig.cpufeatures index 250c10627ab3..733d5aff2456 100644 --- a/arch/x86/Kconfig.cpufeatures +++ b/arch/x86/Kconfig.cpufeatures @@ -124,6 +124,10 @@ config X86_DISABLED_FEATURE_PCID def_bool y depends on !X86_64 =20 +config X86_DISABLED_FEATURE_LASS + def_bool y + depends on X86_32 + config X86_DISABLED_FEATURE_PKU def_bool y depends on !X86_INTEL_MEMORY_PROTECTION_KEYS diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpuf= eatures.h index b78af55aa22e..8eef1ad7aca2 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -313,6 +313,7 @@ #define X86_FEATURE_SM4 (12*32+ 2) /* SM4 instructions */ #define X86_FEATURE_AVX_VNNI (12*32+ 4) /* "avx_vnni" AVX VNNI instructio= ns */ #define X86_FEATURE_AVX512_BF16 (12*32+ 5) /* "avx512_bf16" AVX512 BFLOAT= 16 instructions */ +#define X86_FEATURE_LASS (12*32+ 6) /* "lass" Linear Address Space Separa= tion */ #define X86_FEATURE_CMPCCXADD (12*32+ 7) /* CMPccXADD instructio= ns */ #define X86_FEATURE_ARCH_PERFMON_EXT (12*32+ 8) /* Intel Architectural Per= fMon Extension */ #define X86_FEATURE_FZRM (12*32+10) /* Fast zero-length REP MOVSB */ diff --git a/arch/x86/include/uapi/asm/processor-flags.h b/arch/x86/include= /uapi/asm/processor-flags.h index f1a4adc78272..81d0c8bf1137 100644 --- a/arch/x86/include/uapi/asm/processor-flags.h +++ b/arch/x86/include/uapi/asm/processor-flags.h @@ -136,6 +136,8 @@ #define X86_CR4_PKE _BITUL(X86_CR4_PKE_BIT) #define X86_CR4_CET_BIT 23 /* enable Control-flow Enforcement Technology = */ #define X86_CR4_CET _BITUL(X86_CR4_CET_BIT) +#define X86_CR4_LASS_BIT 27 /* enable Linear Address Space Separation supp= ort */ +#define X86_CR4_LASS _BITUL(X86_CR4_LASS_BIT) #define X86_CR4_LAM_SUP_BIT 28 /* LAM for supervisor pointers */ #define X86_CR4_LAM_SUP _BITUL(X86_CR4_LAM_SUP_BIT) =20 diff --git a/arch/x86/kernel/cpu/cpuid-deps.c b/arch/x86/kernel/cpu/cpuid-d= eps.c index 46efcbd6afa4..98d0cdd82574 100644 --- a/arch/x86/kernel/cpu/cpuid-deps.c +++ b/arch/x86/kernel/cpu/cpuid-deps.c @@ -89,6 +89,7 @@ static const struct cpuid_dep cpuid_deps[] =3D { { X86_FEATURE_SHSTK, X86_FEATURE_XSAVES }, { X86_FEATURE_FRED, X86_FEATURE_LKGS }, { X86_FEATURE_SPEC_CTRL_SSBD, X86_FEATURE_SPEC_CTRL }, + { X86_FEATURE_LASS, X86_FEATURE_SMAP }, {} }; =20 diff --git a/tools/arch/x86/include/asm/cpufeatures.h b/tools/arch/x86/incl= ude/asm/cpufeatures.h index ee176236c2be..4473a6f7800b 100644 --- a/tools/arch/x86/include/asm/cpufeatures.h +++ b/tools/arch/x86/include/asm/cpufeatures.h @@ -313,6 +313,7 @@ #define X86_FEATURE_SM4 (12*32+ 2) /* SM4 instructions */ #define X86_FEATURE_AVX_VNNI (12*32+ 4) /* "avx_vnni" AVX VNNI instructio= ns */ #define X86_FEATURE_AVX512_BF16 (12*32+ 5) /* "avx512_bf16" AVX512 BFLOAT= 16 instructions */ +#define X86_FEATURE_LASS (12*32+ 6) /* "lass" Linear Address Space Separa= tion */ #define X86_FEATURE_CMPCCXADD (12*32+ 7) /* CMPccXADD instructio= ns */ #define X86_FEATURE_ARCH_PERFMON_EXT (12*32+ 8) /* Intel Architectural Per= fMon Extension */ #define X86_FEATURE_FZRM (12*32+10) /* Fast zero-length REP MOVSB */ --=20 2.47.2 From nobody Wed Oct 8 06:47:47 2025 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ADC1A26CE12; Tue, 1 Jul 2025 09:59:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.12 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363945; cv=none; b=PSfSU3M5k+77OWGeEiC5ogzv5JZ4EtB4RJWVldpOzJ/QIdjsk0j2KUD+v9CyEHwuqYNRMUwSQoFD7Bi9ZA+ZT2STPt94VHaZwdK++cBmLu9Vwc37PaqtzaKqJKyO+/3VP5gAEXjZvXQ5hkFhIpUHF7wLYqXqv5+fjQIn7HScDr8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363945; c=relaxed/simple; bh=MH1IaDp7hDo603CvtHddFJjtTuSQLBftTzwh28qZu1M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=S99OGOcO/TvGlZiyGyPJ2vqrMkzmqi11lo+GUf/0MgJW6elhELmYR7EI1Hx9oINtk+Vgl1pUYX0PfcWafLskiyYfk4A6dwGp3nT1praeLzm/9kJlmQ0RJaeU5SNGtO64Ymwa9pfDhAqFu2qoEtfzwbs6D3m4S3caE1uZUXLsPc0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.helo=mgamail.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=b09akE0g; arc=none smtp.client-ip=192.198.163.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.helo=mgamail.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="b09akE0g" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1751363944; x=1782899944; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=MH1IaDp7hDo603CvtHddFJjtTuSQLBftTzwh28qZu1M=; b=b09akE0gkoxZH7eLMMtU+1/kfGDrkQwMsWJpPnwi0SMBcuSDLhLtlAJW tYm2RJPjAJDFRuTBH8X38CGkewQh89t5e2BNyyZ4RTtyc9mOa8k7CHDaR ZagxdFbKsbdMOOIJLKhrPHQd9ePviy7i11OpPO0AKnt8ddnY4Rkzj+Zw4 jQdce78nulsEfMr409azo4xQv1dCVZzl3dQnZRGWY5bvqnJUnmrb9TYJe U+HK3YLpwzUcWt322TqYbUThaJls4MjSpNblstup+uO2E2Eq/MWhISrKW KgAnuWYykARz2evt0VF3iCh+dd37c48Ee9D0UCak9BLdF1lvl/rak6gr+ A==; X-CSE-ConnectionGUID: lneFF278QA6mUaOemILZcA== X-CSE-MsgGUID: B/yPu5S4Se+y8eC7hm0EwQ== X-IronPort-AV: E=McAfee;i="6800,10657,11480"; a="57427942" X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="57427942" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Jul 2025 02:59:03 -0700 X-CSE-ConnectionGUID: OH/TrHlOTry9t2jFI1A5rA== X-CSE-MsgGUID: OQFG+yXBQoip9IzJR/wuMg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="190896388" Received: from black.fi.intel.com ([10.237.72.28]) by orviesa001.jf.intel.com with ESMTP; 01 Jul 2025 02:58:51 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id 3697E376; Tue, 01 Jul 2025 12:58:50 +0300 (EEST) From: "Kirill A. Shutemov" To: Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Ard Biesheuvel , "Paul E. McKenney" , Josh Poimboeuf , Xiongwei Song , Xin Li , "Mike Rapoport (IBM)" , Brijesh Singh , Michael Roth , Tony Luck , Alexey Kardashevskiy , Alexander Shishkin Cc: Jonathan Corbet , Sohil Mehta , Ingo Molnar , Pawan Gupta , Daniel Sneddon , Kai Huang , Sandipan Das , Breno Leitao , Rick Edgecombe , Alexei Starovoitov , Hou Tao , Juergen Gross , Vegard Nossum , Kees Cook , Eric Biggers , Jason Gunthorpe , "Masami Hiramatsu (Google)" , Andrew Morton , Luis Chamberlain , Yuntao Wang , Rasmus Villemoes , Christophe Leroy , Tejun Heo , Changbin Du , Huang Shijie , Geert Uytterhoeven , Namhyung Kim , Arnaldo Carvalho de Melo , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-efi@vger.kernel.org, linux-mm@kvack.org, "Kirill A. Shutemov" Subject: [PATCHv8 02/17] x86/asm: Introduce inline memcpy and memset Date: Tue, 1 Jul 2025 12:58:31 +0300 Message-ID: <20250701095849.2360685-3-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> References: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Extract memcpy and memset functions from copy_user_generic() and __clear_user(). They can be used as inline memcpy and memset instead of the GCC builtins whenever necessary. LASS requires them to handle text_poke. Originally-by: Peter Zijlstra Link: https://lore.kernel.org/all/20241029184840.GJ14555@noisy.programming.= kicks-ass.net/ Signed-off-by: Kirill A. Shutemov --- arch/x86/include/asm/string.h | 46 +++++++++++++++++++++++++++++++ arch/x86/include/asm/uaccess_64.h | 38 +++++++------------------ arch/x86/lib/clear_page_64.S | 13 +++++++-- 3 files changed, 67 insertions(+), 30 deletions(-) diff --git a/arch/x86/include/asm/string.h b/arch/x86/include/asm/string.h index c3c2c1914d65..17f6b5bfa8c1 100644 --- a/arch/x86/include/asm/string.h +++ b/arch/x86/include/asm/string.h @@ -1,6 +1,52 @@ /* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_X86_STRING_H +#define _ASM_X86_STRING_H + +#include +#include +#include + #ifdef CONFIG_X86_32 # include #else # include #endif + +#ifdef CONFIG_X86_64 +#define ALT_64(orig, alt, feat) ALTERNATIVE(orig, alt, feat) +#else +#define ALT_64(orig, alt, feat) orig "\n" +#endif + +static __always_inline void *__inline_memcpy(void *to, const void *from, s= ize_t len) +{ + void *ret =3D to; + + asm volatile("1:\n\t" + ALT_64("rep movsb", + "call rep_movs_alternative", ALT_NOT(X86_FEATURE_FSRM)) + "2:\n\t" + _ASM_EXTABLE_UA(1b, 2b) + : "+c" (len), "+D" (to), "+S" (from), ASM_CALL_CONSTRAINT + : : "memory", _ASM_AX); + + return ret + len; +} + +static __always_inline void *__inline_memset(void *addr, int v, size_t len) +{ + void *ret =3D addr; + + asm volatile("1:\n\t" + ALT_64("rep stosb", + "call rep_stos_alternative", ALT_NOT(X86_FEATURE_FSRM)) + "2:\n\t" + _ASM_EXTABLE_UA(1b, 2b) + : "+c" (len), "+D" (addr), ASM_CALL_CONSTRAINT + : "a" ((uint8_t)v) + : "memory", _ASM_SI, _ASM_DX); + + return ret + len; +} + +#endif /* _ASM_X86_STRING_H */ diff --git a/arch/x86/include/asm/uaccess_64.h b/arch/x86/include/asm/uacce= ss_64.h index c8a5ae35c871..eb531e13e659 100644 --- a/arch/x86/include/asm/uaccess_64.h +++ b/arch/x86/include/asm/uaccess_64.h @@ -13,6 +13,7 @@ #include #include #include +#include =20 /* * Virtual variable: there's no actual backing store for this, @@ -118,21 +119,12 @@ rep_movs_alternative(void *to, const void *from, unsi= gned len); static __always_inline __must_check unsigned long copy_user_generic(void *to, const void *from, unsigned long len) { + void *ret; + stac(); - /* - * If CPU has FSRM feature, use 'rep movs'. - * Otherwise, use rep_movs_alternative. - */ - asm volatile( - "1:\n\t" - ALTERNATIVE("rep movsb", - "call rep_movs_alternative", ALT_NOT(X86_FEATURE_FSRM)) - "2:\n" - _ASM_EXTABLE_UA(1b, 2b) - :"+c" (len), "+D" (to), "+S" (from), ASM_CALL_CONSTRAINT - : : "memory", "rax"); + ret =3D __inline_memcpy(to, from, len); clac(); - return len; + return ret - to; } =20 static __always_inline __must_check unsigned long @@ -178,25 +170,15 @@ rep_stos_alternative(void __user *addr, unsigned long= len); =20 static __always_inline __must_check unsigned long __clear_user(void __user= *addr, unsigned long size) { + void *ptr =3D (__force void *)addr; + void *ret; + might_fault(); stac(); - - /* - * No memory constraint because it doesn't change any memory gcc - * knows about. - */ - asm volatile( - "1:\n\t" - ALTERNATIVE("rep stosb", - "call rep_stos_alternative", ALT_NOT(X86_FEATURE_FSRS)) - "2:\n" - _ASM_EXTABLE_UA(1b, 2b) - : "+c" (size), "+D" (addr), ASM_CALL_CONSTRAINT - : "a" (0)); - + ret =3D __inline_memset(ptr, 0, size); clac(); =20 - return size; + return ret - ptr; } =20 static __always_inline unsigned long clear_user(void __user *to, unsigned = long n) diff --git a/arch/x86/lib/clear_page_64.S b/arch/x86/lib/clear_page_64.S index a508e4a8c66a..47b613690f84 100644 --- a/arch/x86/lib/clear_page_64.S +++ b/arch/x86/lib/clear_page_64.S @@ -55,17 +55,26 @@ SYM_FUNC_END(clear_page_erms) EXPORT_SYMBOL_GPL(clear_page_erms) =20 /* - * Default clear user-space. + * Default memset. * Input: * rdi destination + * rsi scratch * rcx count - * rax is zero + * al is value * * Output: * rcx: uncleared bytes or 0 if successful. + * rdx: clobbered */ SYM_FUNC_START(rep_stos_alternative) ANNOTATE_NOENDBR + + movzbq %al, %rsi + movabs $0x0101010101010101, %rax + + /* RDX:RAX =3D RAX * RSI */ + mulq %rsi + cmpq $64,%rcx jae .Lunrolled =20 --=20 2.47.2 From nobody Wed Oct 8 06:47:47 2025 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8BD5F26E706; Tue, 1 Jul 2025 09:59:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.12 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363947; cv=none; b=VFJL74vBd5231s/JUmoaVRh5VamDlz3o2SF/xR7IEcMJX2fg6sWPNl6RQiuNWwP7rgUTcVYhPv9bwFpqZTmHc9awBWrKxt3TrbX+ZrZje9T5jrjvcBSMYHGYMSRcGWnk7qkhIOgDeSox7Bb75UI/G6pPJNJWxLt2Lftd+MX0mqE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363947; c=relaxed/simple; bh=y0vn0lcrJETMXdw4LaYM0L/E+gW//EAzSxILZz0c9ew=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Y0CuFgWPiTXtjV6Pw3pgOZsh2AgDa7bTGYj5AvziXxZsDzrrDgpMXCZvYq9/310KAG+5NT6lguoO1LNEthzw4O/gzdbWWuS0engEULvnLtsSFacDNd6bQm0gB11dITmhwzAik+w/GOqdx8g6n9y1x2m6O2w2B3V7sTd1orcErOE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.helo=mgamail.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=FCpTGRmz; arc=none smtp.client-ip=192.198.163.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.helo=mgamail.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="FCpTGRmz" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1751363946; x=1782899946; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=y0vn0lcrJETMXdw4LaYM0L/E+gW//EAzSxILZz0c9ew=; b=FCpTGRmzfshdF+FsGlYDTiHYI1QR2TMbTquAkTuqu9H9RBMhDYt7ZMcj KViqOjQi93KNJP4hdESeHCoPpLzwV018yKZsxAGxsgnVf3FBD3UA3Kzp3 tx9hTo/L+BOOQtzalQA7QOa1vfHTbNVSUcDNLhNXK/Z7HMpmsPJAxo3ZI iKN3a8PTrI64uXBW35wRtHBQqhmwKBJn831IXfxLtb5bZNNUoeHgFbl0r lazT8BMLYZwyhgLru63ZygAiiBuSrWBwE6qFhIiznDJ+H1BkomxqfeowY hGb9KvRJjYeNUPg49ZrvjkvrqeXB1kpOgSYpJAQqFBRc90dbsoaF6aDKq w==; X-CSE-ConnectionGUID: YwrEUj3DR4aRjrDrEv/WYA== X-CSE-MsgGUID: Ijodw2LKS1q2yrLfax5gkQ== X-IronPort-AV: E=McAfee;i="6800,10657,11480"; a="57427975" X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="57427975" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Jul 2025 02:59:05 -0700 X-CSE-ConnectionGUID: 6jGzRELIQ7uUqZi1iPqqRA== X-CSE-MsgGUID: TCotVfYvQPq6dUnaXH4C2Q== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="190896398" Received: from black.fi.intel.com ([10.237.72.28]) by orviesa001.jf.intel.com with ESMTP; 01 Jul 2025 02:58:53 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id 48BFC3EA; Tue, 01 Jul 2025 12:58:50 +0300 (EEST) From: "Kirill A. Shutemov" To: Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Ard Biesheuvel , "Paul E. McKenney" , Josh Poimboeuf , Xiongwei Song , Xin Li , "Mike Rapoport (IBM)" , Brijesh Singh , Michael Roth , Tony Luck , Alexey Kardashevskiy , Alexander Shishkin Cc: Jonathan Corbet , Sohil Mehta , Ingo Molnar , Pawan Gupta , Daniel Sneddon , Kai Huang , Sandipan Das , Breno Leitao , Rick Edgecombe , Alexei Starovoitov , Hou Tao , Juergen Gross , Vegard Nossum , Kees Cook , Eric Biggers , Jason Gunthorpe , "Masami Hiramatsu (Google)" , Andrew Morton , Luis Chamberlain , Yuntao Wang , Rasmus Villemoes , Christophe Leroy , Tejun Heo , Changbin Du , Huang Shijie , Geert Uytterhoeven , Namhyung Kim , Arnaldo Carvalho de Melo , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-efi@vger.kernel.org, linux-mm@kvack.org, "Kirill A. Shutemov" Subject: [PATCHv8 03/17] x86/alternatives: Disable LASS when patching kernel alternatives Date: Tue, 1 Jul 2025 12:58:32 +0300 Message-ID: <20250701095849.2360685-4-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> References: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Sohil Mehta For patching, the kernel initializes a temporary mm area in the lower half of the address range. See commit 4fc19708b165 ("x86/alternatives: Initialize temporary mm for patching"). Disable LASS enforcement during patching to avoid triggering a #GP fault. The objtool warns due to a call to a non-allowed function that exists outside of the stac/clac guard, or references to any function with a dynamic function pointer inside the guard. See the Objtool warnings section #9 in the document tools/objtool/Documentation/objtool.txt. Considering that patching is usually small, replace the memcpy and memset functions in the text poking functions with their inline versions respectively. Signed-off-by: Sohil Mehta Signed-off-by: Alexander Shishkin Signed-off-by: Kirill A. Shutemov --- arch/x86/include/asm/smap.h | 33 +++++++++++++++++++++++++++++++-- arch/x86/kernel/alternative.c | 14 ++++++++++++-- 2 files changed, 43 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/smap.h b/arch/x86/include/asm/smap.h index 4f84d421d1cf..d0cc24348641 100644 --- a/arch/x86/include/asm/smap.h +++ b/arch/x86/include/asm/smap.h @@ -23,18 +23,47 @@ =20 #else /* __ASSEMBLER__ */ =20 +/* + * The CLAC/STAC instructions toggle the enforcement of X86_FEATURE_SMAP a= nd + * X86_FEATURE_LASS. + * + * SMAP enforcement is based on the _PAGE_BIT_USER bit in the page tables:= the + * kernel is not allowed to touch pages with the bit set unless the AC bit= is + * set. + * + * LASS enforcement is based on bit 63 of the virtual address. The kernel = is + * not allowed to touch memory in the lower half of the virtual address sp= ace + * unless the AC bit is set. + * + * Use stac()/clac() when accessing userspace (_PAGE_USER) mappings, + * regardless of location. + * + * Use lass_stac()/lass_clac() when accessing kernel mappings (!_PAGE_USER) + * in the lower half of the address space. + * + * Note: a barrier is implicit in alternative(). + */ + static __always_inline void clac(void) { - /* Note: a barrier is implicit in alternative() */ alternative("", "clac", X86_FEATURE_SMAP); } =20 static __always_inline void stac(void) { - /* Note: a barrier is implicit in alternative() */ alternative("", "stac", X86_FEATURE_SMAP); } =20 +static __always_inline void lass_clac(void) +{ + alternative("", "clac", X86_FEATURE_LASS); +} + +static __always_inline void lass_stac(void) +{ + alternative("", "stac", X86_FEATURE_LASS); +} + static __always_inline unsigned long smap_save(void) { unsigned long flags; diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c index ea1d984166cd..3d2bcb7682eb 100644 --- a/arch/x86/kernel/alternative.c +++ b/arch/x86/kernel/alternative.c @@ -2447,16 +2447,26 @@ void __init_or_module text_poke_early(void *addr, c= onst void *opcode, __ro_after_init struct mm_struct *text_poke_mm; __ro_after_init unsigned long text_poke_mm_addr; =20 +/* + * Text poking creates and uses a mapping in the lower half of the + * address space. Relax LASS enforcement when accessing the poking + * address. + */ + static void text_poke_memcpy(void *dst, const void *src, size_t len) { - memcpy(dst, src, len); + lass_stac(); + __inline_memcpy(dst, src, len); + lass_clac(); } =20 static void text_poke_memset(void *dst, const void *src, size_t len) { int c =3D *(const int *)src; =20 - memset(dst, c, len); + lass_stac(); + __inline_memset(dst, c, len); + lass_clac(); } =20 typedef void text_poke_f(void *dst, const void *src, size_t len); --=20 2.47.2 From nobody Wed Oct 8 06:47:47 2025 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E7B7B26E71A; Tue, 1 Jul 2025 09:59:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.12 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363947; cv=none; b=OoY8ttTBE9XT5LNA4ZdSNzTIQ6I1zxdjPrT/clW4eaw7O7/mzwGgMA4+slH5IHzswg5n2XoG0rfq70OC0Vp/OtyvU1KtpBHKjfhoC9l7hPY4OLc/7v90PPcFguf9Til2Q21QckL68g3Q3xF/f0utjw3ETnK/W+rlPs5Ap/nbdy0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363947; c=relaxed/simple; bh=KVi1SbCxLhPwipXkfMiClroXvuKW+0f0MpKQ0MUg/MY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EmY6Qni2dP750sXLVM7YEklZh9sWHC3CvYn8odvHA/KluI4IW3lFM0D4M2Gp7uuYd1TSAdhjICT9eqfyvKTP2gyxwzIBXgyQcIJzRVDNOoEwpf0Fh8+V7+sQkIiD/Z9K1xc3eWtwUxUvOYVfbQ6ONeeh4DBbwP+0v04SyuXa1PM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.helo=mgamail.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=Qd1AipEd; arc=none smtp.client-ip=192.198.163.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.helo=mgamail.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="Qd1AipEd" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1751363946; x=1782899946; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=KVi1SbCxLhPwipXkfMiClroXvuKW+0f0MpKQ0MUg/MY=; b=Qd1AipEd3Xi0LD0w8LvZC9Sm57jwEsg3vQjDgG53PUy1nBqDejMpgs5b 5v+OXWwNN1KzQG+KNso1/n+s0ki9A4Xe27/zAYlbtPbp/nqQ+kAxynxbr Yl2/zBL4BejIPV4fsT44D37LCKxPi/x3b72OcB9nmZPKG11CfZA7wYd2l 4JgDlyGQUk3EOIgtVO3mvVfMQh2FYOsw1PT9mMNjnpzb2wCNta4lju1dn +IZ6DuawAlKKlA5MpUJlTVIZsmvxqzLm4hwmNiwAIben5F+55OVTH6Ru1 Ur5Tr/VpSS5y8HSGrWVx7xJWMoYgzgN5S2jmsUWERwErLnFqHWg927Q4f Q==; X-CSE-ConnectionGUID: R9KsAzu1SDKgaTDELFnl5g== X-CSE-MsgGUID: KNw+4+q0S9+1SedtSTcHKA== X-IronPort-AV: E=McAfee;i="6800,10657,11480"; a="57428008" X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="57428008" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Jul 2025 02:59:05 -0700 X-CSE-ConnectionGUID: t/+qn6kkRnqxouZqA5qu+g== X-CSE-MsgGUID: nXMyxAdjQnWehTndjUuilA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="190896400" Received: from black.fi.intel.com ([10.237.72.28]) by orviesa001.jf.intel.com with ESMTP; 01 Jul 2025 02:58:53 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id 5690B3F5; Tue, 01 Jul 2025 12:58:50 +0300 (EEST) From: "Kirill A. Shutemov" To: Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Ard Biesheuvel , "Paul E. McKenney" , Josh Poimboeuf , Xiongwei Song , Xin Li , "Mike Rapoport (IBM)" , Brijesh Singh , Michael Roth , Tony Luck , Alexey Kardashevskiy , Alexander Shishkin Cc: Jonathan Corbet , Sohil Mehta , Ingo Molnar , Pawan Gupta , Daniel Sneddon , Kai Huang , Sandipan Das , Breno Leitao , Rick Edgecombe , Alexei Starovoitov , Hou Tao , Juergen Gross , Vegard Nossum , Kees Cook , Eric Biggers , Jason Gunthorpe , "Masami Hiramatsu (Google)" , Andrew Morton , Luis Chamberlain , Yuntao Wang , Rasmus Villemoes , Christophe Leroy , Tejun Heo , Changbin Du , Huang Shijie , Geert Uytterhoeven , Namhyung Kim , Arnaldo Carvalho de Melo , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-efi@vger.kernel.org, linux-mm@kvack.org, "Kirill A. Shutemov" Subject: [PATCHv8 04/17] x86/cpu: Defer CR pinning setup until after EFI initialization Date: Tue, 1 Jul 2025 12:58:33 +0300 Message-ID: <20250701095849.2360685-5-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> References: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alexander Shishkin In order to map the EFI runtime services, set_virtual_address_map() needs to be called, which resides in the lower half of the address space. This means that LASS needs to be temporarily disabled around this call. This can only be done before the CR pinning is set up. Move CR pinning setup behind the EFI initialization. Wrapping efi_enter_virtual_mode() into lass_disable/enable_enforcement() is not enough because AC flag gates data accesses, but not instruction fetch. Clearing the CR4 bit is required. Signed-off-by: Alexander Shishkin Suggested-by: Kirill A. Shutemov Signed-off-by: Kirill A. Shutemov --- arch/x86/kernel/cpu/common.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 4f430be285de..9918121e0adc 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -2081,7 +2081,6 @@ static __init void identify_boot_cpu(void) enable_sep_cpu(); #endif cpu_detect_tlb(&boot_cpu_data); - setup_cr_pinning(); =20 tsx_init(); tdx_init(); @@ -2532,10 +2531,14 @@ void __init arch_cpu_finalize_init(void) =20 /* * This needs to follow the FPU initializtion, since EFI depends on it. + * + * EFI twiddles CR4.LASS. Do it before CR pinning. */ if (efi_enabled(EFI_RUNTIME_SERVICES)) efi_enter_virtual_mode(); =20 + setup_cr_pinning(); + /* * Ensure that access to the per CPU representation has the initial * boot CPU configuration. --=20 2.47.2 From nobody Wed Oct 8 06:47:47 2025 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6616726FDAC; Tue, 1 Jul 2025 09:59:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.12 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363949; cv=none; b=Qd4oJIvPKvDJcCMwhlcNVDM9QNc6I8+KArewxCkA3nUmL1YtsXuk6ewkB/EJxdF202OYu6wm/Aal2moXgkRF3JDzopHxKtUjb59pyYnF0tyjptAIwgfYGXheHu5hOZZ3ZR/6jQwk//iRuDVulLz03jTeE9W5k/42loPh3OJ13AY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363949; c=relaxed/simple; bh=G+eP9Bw73xI4j+67SYkgeX9LiSQjfbtHwX1bukiOpM0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=sq+oxj/4DdFVEE2LXgsD/x+58rNLJoAehcKsJSVn/Ax06sIhFOYklhRTVn9eHPkoN1P53P0W5vr0zFiW9Il407/tL9dLulW1aCTxoBROXCp6KfRWNCuVtyvtcyF32jC23BZ/VXGcUYdVUo0v0RwmWQESdctd1VLwSqS059S2MTU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.helo=mgamail.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=T47dJNTG; arc=none smtp.client-ip=192.198.163.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.helo=mgamail.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="T47dJNTG" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1751363948; x=1782899948; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=G+eP9Bw73xI4j+67SYkgeX9LiSQjfbtHwX1bukiOpM0=; b=T47dJNTGqyLtldhrhszu5ZTemlhhgi4EA0pymKiOS4XTSv1396RTdwj8 VH5ZR84ez+parwzeKwCPe7N0hdHdhy4z6eZ2btCAgwdloL2XlJnfR49Mp 5J6St1h0atQydOocS+5uSYOjsKdTlKt/zXeBLTpQuB1+q152r7K1ZhENa VOaoVdVRc83ggqwT2VfTMnHUdZQ1aKd3Jb2Gio8FgcWo1JC2UoFCt0z+6 aUBg0Ihugn2c0H2ovLSHpmsFvZY5DvCJp24p0eiQ3MsExAL2Y++5z8Kxx 4wcdSdIS8yxkZC6RNu6WOCTNK0mHurPsCuYy7NaNZE5Zj4MwbiFV+8NSp Q==; X-CSE-ConnectionGUID: UmqLEIhvTDCKmtt0e6mUuQ== X-CSE-MsgGUID: 57h9T2ckSJuMIGzs5Esuqw== X-IronPort-AV: E=McAfee;i="6800,10657,11480"; a="57428029" X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="57428029" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Jul 2025 02:59:07 -0700 X-CSE-ConnectionGUID: qbhrAEsuT3OS1w2+RDuY5g== X-CSE-MsgGUID: WBcnX9YsScuHLqGF0AKp1g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="190896404" Received: from black.fi.intel.com ([10.237.72.28]) by orviesa001.jf.intel.com with ESMTP; 01 Jul 2025 02:58:55 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id 67888417; Tue, 01 Jul 2025 12:58:50 +0300 (EEST) From: "Kirill A. Shutemov" To: Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Ard Biesheuvel , "Paul E. McKenney" , Josh Poimboeuf , Xiongwei Song , Xin Li , "Mike Rapoport (IBM)" , Brijesh Singh , Michael Roth , Tony Luck , Alexey Kardashevskiy , Alexander Shishkin Cc: Jonathan Corbet , Sohil Mehta , Ingo Molnar , Pawan Gupta , Daniel Sneddon , Kai Huang , Sandipan Das , Breno Leitao , Rick Edgecombe , Alexei Starovoitov , Hou Tao , Juergen Gross , Vegard Nossum , Kees Cook , Eric Biggers , Jason Gunthorpe , "Masami Hiramatsu (Google)" , Andrew Morton , Luis Chamberlain , Yuntao Wang , Rasmus Villemoes , Christophe Leroy , Tejun Heo , Changbin Du , Huang Shijie , Geert Uytterhoeven , Namhyung Kim , Arnaldo Carvalho de Melo , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-efi@vger.kernel.org, linux-mm@kvack.org, "Kirill A. Shutemov" Subject: [PATCHv8 05/17] efi: Disable LASS around set_virtual_address_map() EFI call Date: Tue, 1 Jul 2025 12:58:34 +0300 Message-ID: <20250701095849.2360685-6-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> References: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alexander Shishkin Of all the EFI runtime services, set_virtual_address_map() is the only one that is called at its lower mapping, which LASS prohibits regardless of EFLAGS.AC setting. The only way to allow this to happen is to disable LASS in the CR4 register. Disable LASS around this low address EFI call. Signed-off-by: Alexander Shishkin Signed-off-by: Kirill A. Shutemov --- arch/x86/platform/efi/efi.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/arch/x86/platform/efi/efi.c b/arch/x86/platform/efi/efi.c index 463b784499a8..5b23c0daedef 100644 --- a/arch/x86/platform/efi/efi.c +++ b/arch/x86/platform/efi/efi.c @@ -787,6 +787,7 @@ static void __init __efi_enter_virtual_mode(void) int count =3D 0, pg_shift =3D 0; void *new_memmap =3D NULL; efi_status_t status; + unsigned long lass; unsigned long pa; =20 if (efi_alloc_page_tables()) { @@ -825,11 +826,25 @@ static void __init __efi_enter_virtual_mode(void) =20 efi_sync_low_kernel_mappings(); =20 + /* + * set_virtual_address_map() is the only service located at lower + * addresses, so LASS has to be disabled around it. + * + * Note that flipping RFLAGS.AC is not sufficient for this, as it only + * permits data accesses and not instruction fetch. The entire LASS + * needs to be disabled. + */ + lass =3D cr4_read_shadow() & X86_CR4_LASS; + cr4_clear_bits(lass); + status =3D efi_set_virtual_address_map(efi.memmap.desc_size * count, efi.memmap.desc_size, efi.memmap.desc_version, (efi_memory_desc_t *)pa, efi_systab_phys); + + cr4_set_bits(lass); + if (status !=3D EFI_SUCCESS) { pr_err("Unable to switch EFI into virtual mode (status=3D%lx)!\n", status); --=20 2.47.2 From nobody Wed Oct 8 06:47:47 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 889BA26FA59; Tue, 1 Jul 2025 09:59:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363948; cv=none; b=a863aFmyy5bG+s6joJqfmJ4PewOf+OPHV3YHmROY8zzLxNRJ7hjqQZJzmRhV79IFGb5/FFBZt+Ypms9uNe8L+HlHp8cd3E5Fj/vA/4pqgQSDuZBVz27YILW93Cvhvn/venlg4paxpnkL7CT+g+wU7djY2gtoG5SxZgdB6QAXn9k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363948; c=relaxed/simple; bh=TQ5RCfOQ/5tuZqESL8hi2WCyzYpg/ALQ/w3Eenbo4Ow=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BUJuMGn7HEeBj8ecycG2FtSCBAtAdGCSbZsIaCHg5d2q3gAgozF+tMukD9ahYkgh+kjxG07hAGBSs/0tgbVa8zT1eFrcPIvW7F6/BWh2ECRLH4ld/15ndHG9X7xMDqYwKXuc0mVThaVYuY1hS98pN0DbWOwJ4Ug+dR9d1pF5SgU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.helo=mgamail.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=WhJDZRRT; arc=none smtp.client-ip=198.175.65.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.helo=mgamail.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="WhJDZRRT" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1751363947; x=1782899947; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=TQ5RCfOQ/5tuZqESL8hi2WCyzYpg/ALQ/w3Eenbo4Ow=; b=WhJDZRRTqqAbzW/GY3lIx0eFqekyRkc7f/JOlzhxuEKS/pBWqqUY2MUM V0pwiJNxmgTd1bXYjVvzhJo2K3xLZAspm0TikqfEfUpFuuuk0WeT4eYfH P5qsGjN8svtRHitVcStkccIH/MZDg7aAA3CqKya6Aa2FT0PixbAWfJuj7 q4Qcop9rbpsJvf2THrXALhC8FigX/5eNLtCGaz5Dir7E05Ju631gttaRY 5k1VYUvq6ftM6rdPiZPm6wn58ZQ6gyeShJZL94hS3HGULfLdSw+2b6Hkl iD65z8WvuCDjxWb4+8XFoKqVUEKm5rLmMtrhICaLV3CHyiGG2c4VRHMSM A==; X-CSE-ConnectionGUID: So6E2Nw/TnyAQQ+27VsnEA== X-CSE-MsgGUID: HeZ5O2Q4StasPOcy0F+trA== X-IronPort-AV: E=McAfee;i="6800,10657,11480"; a="71048649" X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="71048649" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Jul 2025 02:59:07 -0700 X-CSE-ConnectionGUID: ulSrwC4NR46aDPQOLhrZ5Q== X-CSE-MsgGUID: kv43OEAgSp+0GwwuAzuZkw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="177390811" Received: from black.fi.intel.com ([10.237.72.28]) by fmviesa002.fm.intel.com with ESMTP; 01 Jul 2025 02:58:55 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id 744944B1; Tue, 01 Jul 2025 12:58:50 +0300 (EEST) From: "Kirill A. Shutemov" To: Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Ard Biesheuvel , "Paul E. McKenney" , Josh Poimboeuf , Xiongwei Song , Xin Li , "Mike Rapoport (IBM)" , Brijesh Singh , Michael Roth , Tony Luck , Alexey Kardashevskiy , Alexander Shishkin Cc: Jonathan Corbet , Sohil Mehta , Ingo Molnar , Pawan Gupta , Daniel Sneddon , Kai Huang , Sandipan Das , Breno Leitao , Rick Edgecombe , Alexei Starovoitov , Hou Tao , Juergen Gross , Vegard Nossum , Kees Cook , Eric Biggers , Jason Gunthorpe , "Masami Hiramatsu (Google)" , Andrew Morton , Luis Chamberlain , Yuntao Wang , Rasmus Villemoes , Christophe Leroy , Tejun Heo , Changbin Du , Huang Shijie , Geert Uytterhoeven , Namhyung Kim , Arnaldo Carvalho de Melo , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-efi@vger.kernel.org, linux-mm@kvack.org, "Kirill A. Shutemov" Subject: [PATCHv8 06/17] x86/vsyscall: Do not require X86_PF_INSTR to emulate vsyscall Date: Tue, 1 Jul 2025 12:58:35 +0300 Message-ID: <20250701095849.2360685-7-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> References: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" emulate_vsyscall() expects to see X86_PF_INSTR in PFEC on a vsyscall page fault, but the CPU does not report X86_PF_INSTR if neither X86_FEATURE_NX nor X86_FEATURE_SMEP are enabled. X86_FEATURE_NX should be enabled on nearly all 64-bit CPUs, except for early P4 processors that did not support this feature. Instead of explicitly checking for X86_PF_INSTR, compare the fault address against RIP. On machines with X86_FEATURE_NX enabled, issue a warning if RIP is equal to fault address but X86_PF_INSTR is absent. Originally-by: Dave Hansen Link: https://lore.kernel.org/all/bd81a98b-f8d4-4304-ac55-d4151a1a77ab@inte= l.com Signed-off-by: Kirill A. Shutemov Reported-by: Andrew Cooper Reviewed-by: Andrew Cooper --- arch/x86/entry/vsyscall/vsyscall_64.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/arch/x86/entry/vsyscall/vsyscall_64.c b/arch/x86/entry/vsyscal= l/vsyscall_64.c index c9103a6fa06e..0b0e0283994f 100644 --- a/arch/x86/entry/vsyscall/vsyscall_64.c +++ b/arch/x86/entry/vsyscall/vsyscall_64.c @@ -124,7 +124,8 @@ bool emulate_vsyscall(unsigned long error_code, if ((error_code & (X86_PF_WRITE | X86_PF_USER)) !=3D X86_PF_USER) return false; =20 - if (!(error_code & X86_PF_INSTR)) { + /* Avoid emulation unless userspace was executing from vsyscall page: */ + if (address !=3D regs->ip) { /* Failed vsyscall read */ if (vsyscall_mode =3D=3D EMULATE) return false; @@ -136,13 +137,16 @@ bool emulate_vsyscall(unsigned long error_code, return false; } =20 + + /* X86_PF_INSTR is only set when NX is supported: */ + if (cpu_feature_enabled(X86_FEATURE_NX)) + WARN_ON_ONCE(!(error_code & X86_PF_INSTR)); + /* * No point in checking CS -- the only way to get here is a user mode * trap to a high address, which means that we're in 64-bit user code. */ =20 - WARN_ON_ONCE(address !=3D regs->ip); - if (vsyscall_mode =3D=3D NONE) { warn_bad_vsyscall(KERN_INFO, regs, "vsyscall attempted with vsyscall=3Dnone"); --=20 2.47.2 From nobody Wed Oct 8 06:47:47 2025 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3AAE3270EDD; Tue, 1 Jul 2025 09:59:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.12 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363950; cv=none; b=KBRewqQntubOpjUmZJkfBhWvwAQ2I+oyJCjOiMK1C13KFSMIw+beYZrNAWFy5U9EtaZMRAANVg+eesimdP0WkbHbACSBDS45SSLkYFlqPM1Y6eP18hs96dA16/oAHq8Dt6FZl0gNPWdKtRhNkpxuXHyI+y8a+ljjJwA82LYayZo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363950; c=relaxed/simple; bh=AhbyuBrFy6ZhBp5R/WtWxW6rQmI0xvLZeZC7IlrtM/I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qtqOPg10hnD2gZPGlJyFN9TK5JCWN0sb5OU/PztCidmQkkyLhlCYuMRXsRrvRVdhB2Sm5ydP52ZC78m6am8++BmxpIs13dlFLBFdIIo7gJXRee5057CCDcfg87hdbg12P5cSfhf+rJa3oB1MottEbDa+fnz9M6LBhx6bGMbOR70= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.helo=mgamail.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=EkC1feT4; arc=none smtp.client-ip=192.198.163.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.helo=mgamail.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="EkC1feT4" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1751363949; x=1782899949; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=AhbyuBrFy6ZhBp5R/WtWxW6rQmI0xvLZeZC7IlrtM/I=; b=EkC1feT4AkhF5YosOWSce4Av6pPb5Trfvuf2dM+WSdZB8bOKqmLVTRlw LQHs94kVpCCSU2xFSJTAo/z9tlhk6XapKdPaaaVz6B3Jh16ZzvZjGz7hd 1hnE31AU9SKN/JB26h65kf/BZ6u2VAoZk9u8tZU1wiM7LrU50pJAJtqZz dieOrumOre436clOJQNAoMTTEXGxRdCkivrDdJDvgR8TDvUY6qS8SgnGK Lpv1F5mg6Pj7E2+EthbhHOPR8OoD8/9T2O67rw9MjttC3DAWOfyeoem/+ zeCLjYLJadRgCucM2VPMaG3nXoYM2rJfvIJgn/b7ZslAV+w7xBOG66A7I A==; X-CSE-ConnectionGUID: wmXC1aP3Tzq/c0mvGg2+bw== X-CSE-MsgGUID: C2zAzkQTTCCqkYt3za7gKQ== X-IronPort-AV: E=McAfee;i="6800,10657,11480"; a="57428054" X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="57428054" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Jul 2025 02:59:08 -0700 X-CSE-ConnectionGUID: TW34rlASSy+SrR+yVajNOg== X-CSE-MsgGUID: c/1eMKOIT7iCrLMo4ccEFw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="190896413" Received: from black.fi.intel.com ([10.237.72.28]) by orviesa001.jf.intel.com with ESMTP; 01 Jul 2025 02:58:57 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id 812A256D; Tue, 01 Jul 2025 12:58:50 +0300 (EEST) From: "Kirill A. Shutemov" To: Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Ard Biesheuvel , "Paul E. McKenney" , Josh Poimboeuf , Xiongwei Song , Xin Li , "Mike Rapoport (IBM)" , Brijesh Singh , Michael Roth , Tony Luck , Alexey Kardashevskiy , Alexander Shishkin Cc: Jonathan Corbet , Sohil Mehta , Ingo Molnar , Pawan Gupta , Daniel Sneddon , Kai Huang , Sandipan Das , Breno Leitao , Rick Edgecombe , Alexei Starovoitov , Hou Tao , Juergen Gross , Vegard Nossum , Kees Cook , Eric Biggers , Jason Gunthorpe , "Masami Hiramatsu (Google)" , Andrew Morton , Luis Chamberlain , Yuntao Wang , Rasmus Villemoes , Christophe Leroy , Tejun Heo , Changbin Du , Huang Shijie , Geert Uytterhoeven , Namhyung Kim , Arnaldo Carvalho de Melo , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-efi@vger.kernel.org, linux-mm@kvack.org, "Kirill A. Shutemov" Subject: [PATCHv8 07/17] x86/vsyscall: Reorganize the #PF emulation code Date: Tue, 1 Jul 2025 12:58:36 +0300 Message-ID: <20250701095849.2360685-8-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> References: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Sohil Mehta Separate out the actual vsyscall emulation from the page fault specific handling in preparation for the upcoming #GP fault emulation. No functional change intended. Signed-off-by: Sohil Mehta Signed-off-by: Alexander Shishkin Signed-off-by: Kirill A. Shutemov Acked-by: Dave Hansen --- arch/x86/entry/vsyscall/vsyscall_64.c | 52 ++++++++++++++------------- arch/x86/include/asm/vsyscall.h | 8 ++--- arch/x86/mm/fault.c | 2 +- 3 files changed, 33 insertions(+), 29 deletions(-) diff --git a/arch/x86/entry/vsyscall/vsyscall_64.c b/arch/x86/entry/vsyscal= l/vsyscall_64.c index 0b0e0283994f..25f94ac5fd35 100644 --- a/arch/x86/entry/vsyscall/vsyscall_64.c +++ b/arch/x86/entry/vsyscall/vsyscall_64.c @@ -112,36 +112,13 @@ static bool write_ok_or_segv(unsigned long ptr, size_= t size) } } =20 -bool emulate_vsyscall(unsigned long error_code, - struct pt_regs *regs, unsigned long address) +static bool __emulate_vsyscall(struct pt_regs *regs, unsigned long address) { unsigned long caller; int vsyscall_nr, syscall_nr, tmp; long ret; unsigned long orig_dx; =20 - /* Write faults or kernel-privilege faults never get fixed up. */ - if ((error_code & (X86_PF_WRITE | X86_PF_USER)) !=3D X86_PF_USER) - return false; - - /* Avoid emulation unless userspace was executing from vsyscall page: */ - if (address !=3D regs->ip) { - /* Failed vsyscall read */ - if (vsyscall_mode =3D=3D EMULATE) - return false; - - /* - * User code tried and failed to read the vsyscall page. - */ - warn_bad_vsyscall(KERN_INFO, regs, "vsyscall read attempt denied -- look= up the vsyscall kernel parameter if you need a workaround"); - return false; - } - - - /* X86_PF_INSTR is only set when NX is supported: */ - if (cpu_feature_enabled(X86_FEATURE_NX)) - WARN_ON_ONCE(!(error_code & X86_PF_INSTR)); - /* * No point in checking CS -- the only way to get here is a user mode * trap to a high address, which means that we're in 64-bit user code. @@ -274,6 +251,33 @@ bool emulate_vsyscall(unsigned long error_code, return true; } =20 +bool emulate_vsyscall_pf(unsigned long error_code, struct pt_regs *regs, + unsigned long address) +{ + /* Write faults or kernel-privilege faults never get fixed up. */ + if ((error_code & (X86_PF_WRITE | X86_PF_USER)) !=3D X86_PF_USER) + return false; + + if (address =3D=3D regs->ip) { + /* X86_PF_INSTR is only set when NX is supported: */ + if (cpu_feature_enabled(X86_FEATURE_NX)) + WARN_ON_ONCE(!(error_code & X86_PF_INSTR)); + + return __emulate_vsyscall(regs, address); + } + + /* Failed vsyscall read */ + if (vsyscall_mode =3D=3D EMULATE) + return false; + + /* + * User code tried and failed to read the vsyscall page. + */ + warn_bad_vsyscall(KERN_INFO, regs, + "vsyscall read attempt denied -- look up the vsyscall kernel paramete= r if you need a workaround"); + return false; +} + /* * A pseudo VMA to allow ptrace access for the vsyscall page. This only * covers the 64bit vsyscall page now. 32bit has a real VMA now and does diff --git a/arch/x86/include/asm/vsyscall.h b/arch/x86/include/asm/vsyscal= l.h index 472f0263dbc6..214977f4fa11 100644 --- a/arch/x86/include/asm/vsyscall.h +++ b/arch/x86/include/asm/vsyscall.h @@ -14,12 +14,12 @@ extern void set_vsyscall_pgtable_user_bits(pgd_t *root); * Called on instruction fetch fault in vsyscall page. * Returns true if handled. */ -extern bool emulate_vsyscall(unsigned long error_code, - struct pt_regs *regs, unsigned long address); +extern bool emulate_vsyscall_pf(unsigned long error_code, + struct pt_regs *regs, unsigned long address); #else static inline void map_vsyscall(void) {} -static inline bool emulate_vsyscall(unsigned long error_code, - struct pt_regs *regs, unsigned long address) +static inline bool emulate_vsyscall_pf(unsigned long error_code, + struct pt_regs *regs, unsigned long address) { return false; } diff --git a/arch/x86/mm/fault.c b/arch/x86/mm/fault.c index 998bd807fc7b..fbcc2da75fd6 100644 --- a/arch/x86/mm/fault.c +++ b/arch/x86/mm/fault.c @@ -1316,7 +1316,7 @@ void do_user_addr_fault(struct pt_regs *regs, * to consider the PF_PK bit. */ if (is_vsyscall_vaddr(address)) { - if (emulate_vsyscall(error_code, regs, address)) + if (emulate_vsyscall_pf(error_code, regs, address)) return; } #endif --=20 2.47.2 From nobody Wed Oct 8 06:47:47 2025 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6D1F926CE12; Tue, 1 Jul 2025 09:59:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.12 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363951; cv=none; b=iskjNZ6/Ei5h0hREkZ14vrDQ0DMD2sMCHi81uMFnvVqk8+6JgYaC0GxEUyZA1UzgfJsP6O4EK3J9iN76gm4vMXeYlNDyGAZEljeCJtyQSYSohdJEnsL1NXU7ztvthoFbJtsZI7t/7N4uG57x6ZrK5UbrM/vXqQXFAxkcm23pjks= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363951; c=relaxed/simple; bh=GkZc9oX0bGyfhB00ib+Vypspe5Y0XCe0GUwAqpAwtwU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mKldqxCfxszmrwOFmtfW5Dd+J6BtVBgdMycfSdnvRqYQeaaw+mq5vNiyf29B9prZKoLYXd9b6lHNZUoBMWtAzjh8UhNXD3emsgZHMi5KsjHfuWExolEV7SeF5Drr0pmyj7VBFdsMW96lIlCQVgUjN51S/QUE0snBoIEYv66KWI0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.helo=mgamail.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=nZ+aRhVf; arc=none smtp.client-ip=192.198.163.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.helo=mgamail.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="nZ+aRhVf" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1751363950; x=1782899950; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=GkZc9oX0bGyfhB00ib+Vypspe5Y0XCe0GUwAqpAwtwU=; b=nZ+aRhVfP+bkQjhVhfbzqjeJx9dLAUW5macl9Bp0cQVkQls6E0KHmZkB vpnTdqc9pB0l3H7wiYpnhvGQxIfYiLqPx1BfkahaaniIa5KVCQzz9z17s Kjt9MFvr4iEgpVNSWdO4DNSflCl8DJd1UR9rs22rESbAnAXntPeGrY78a O7/JvFFdwhpPo0XhMTr6M6E1EKcP9b1HWtXt6SDhLr7pviF9OBJ0RzBe4 iMQp5tzdr9nyStgMGdjZxApL9H+ig+T0zOrqjwX/YPTmek0B08ov996nq 6VPOm7bEvxIwW6gNxxI3dSYF+YynIazyU49z1Nmcty4pgfGPyS2YHiK3a g==; X-CSE-ConnectionGUID: U9Rlzg8vT+yQc8hO2hyvVg== X-CSE-MsgGUID: V41VM3cQQ0uV1FaLpAh2fA== X-IronPort-AV: E=McAfee;i="6800,10657,11480"; a="57428077" X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="57428077" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Jul 2025 02:59:08 -0700 X-CSE-ConnectionGUID: cgfsiWB6Smqc33N8baKJsg== X-CSE-MsgGUID: GsXzxampTQuf3+ieGEQ5Fg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="190896417" Received: from black.fi.intel.com ([10.237.72.28]) by orviesa001.jf.intel.com with ESMTP; 01 Jul 2025 02:58:57 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id 8DF45608; Tue, 01 Jul 2025 12:58:50 +0300 (EEST) From: "Kirill A. Shutemov" To: Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Ard Biesheuvel , "Paul E. McKenney" , Josh Poimboeuf , Xiongwei Song , Xin Li , "Mike Rapoport (IBM)" , Brijesh Singh , Michael Roth , Tony Luck , Alexey Kardashevskiy , Alexander Shishkin Cc: Jonathan Corbet , Sohil Mehta , Ingo Molnar , Pawan Gupta , Daniel Sneddon , Kai Huang , Sandipan Das , Breno Leitao , Rick Edgecombe , Alexei Starovoitov , Hou Tao , Juergen Gross , Vegard Nossum , Kees Cook , Eric Biggers , Jason Gunthorpe , "Masami Hiramatsu (Google)" , Andrew Morton , Luis Chamberlain , Yuntao Wang , Rasmus Villemoes , Christophe Leroy , Tejun Heo , Changbin Du , Huang Shijie , Geert Uytterhoeven , Namhyung Kim , Arnaldo Carvalho de Melo , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-efi@vger.kernel.org, linux-mm@kvack.org, "Kirill A. Shutemov" Subject: [PATCHv8 08/17] x86/traps: Consolidate user fixups in exc_general_protection() Date: Tue, 1 Jul 2025 12:58:37 +0300 Message-ID: <20250701095849.2360685-9-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> References: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Sohil Mehta Move the UMIP exception fixup along with the other user mode fixups, that is, under the common "if (user_mode(regs))" condition where the rest of the fixups reside. No functional change intended. Suggested-by: Dave Hansen Signed-off-by: Sohil Mehta Signed-off-by: Alexander Shishkin Signed-off-by: Kirill A. Shutemov Acked-by: Dave Hansen --- arch/x86/kernel/traps.c | 8 +++----- arch/x86/kernel/umip.c | 3 +++ 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index c5c897a86418..10856e0ac46c 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -800,11 +800,6 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection) =20 cond_local_irq_enable(regs); =20 - if (static_cpu_has(X86_FEATURE_UMIP)) { - if (user_mode(regs) && fixup_umip_exception(regs)) - goto exit; - } - if (v8086_mode(regs)) { local_irq_enable(); handle_vm86_fault((struct kernel_vm86_regs *) regs, error_code); @@ -819,6 +814,9 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection) if (fixup_vdso_exception(regs, X86_TRAP_GP, error_code, 0)) goto exit; =20 + if (fixup_umip_exception(regs)) + goto exit; + gp_user_force_sig_segv(regs, X86_TRAP_GP, error_code, desc); goto exit; } diff --git a/arch/x86/kernel/umip.c b/arch/x86/kernel/umip.c index 5a4b21389b1d..80f2ad26363c 100644 --- a/arch/x86/kernel/umip.c +++ b/arch/x86/kernel/umip.c @@ -343,6 +343,9 @@ bool fixup_umip_exception(struct pt_regs *regs) void __user *uaddr; struct insn insn; =20 + if (!cpu_feature_enabled(X86_FEATURE_UMIP)) + return false; + if (!regs) return false; =20 --=20 2.47.2 From nobody Wed Oct 8 06:47:47 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A75F327145B; Tue, 1 Jul 2025 09:59:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363951; cv=none; b=WZTYVUXkvV9WtnyNqzjA9pDEDv6Dy050MR1dxLACwdSw4l5SUchIhiX8L6euVjf7X5TDVYjdiL1K8Tq1wYDGGH7S+yHjCmMVXBQjJUi3+4/ki+26fY0C7fwRbExdC0w0qpn8LHsxuhbAXa5Jfak7+YX04OdD+vpXhGxfUHHu/wk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363951; c=relaxed/simple; bh=Y6wMD+f3zIu+lL8OF4jucYTQX8f5E+hJrr2IotIkhTA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qrgO1cBLxCo3ENU3bvDrPJqQLyHH2d4JwCyq7GozvhrOS2mCzk2eESlQ6kmmTijF3lFaIKCZy+QV0Qnj03jM1PGiI1wm/JTp8A44LwbCHNe/TS9RxTJCW29WaC3xdg9qkjQUzlXzPAtUviTKCPmeWl4WMcgwW7xvmpD2/JjyeaQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.helo=mgamail.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=kjeXVxIF; arc=none smtp.client-ip=198.175.65.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.helo=mgamail.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="kjeXVxIF" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1751363950; x=1782899950; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Y6wMD+f3zIu+lL8OF4jucYTQX8f5E+hJrr2IotIkhTA=; b=kjeXVxIFkLTUtS9bLn+hbN+enwixkKQ9BlrDjic5pQWCvamV7TbLtdeQ hsE3O6LM7Ayl8SHLw0No1Zx/K26RfH0bHVOuzOdmaAJfhyOA3uNYI9hs7 4ta2I4lboZZ9/4KZoo+wewHy4EwRDFJhDcpudRB0GdIXPckn9q5Uk9PAY DX/VbaEMxuSyjwtJuHkxPoXuII30NIMRSBZ/5sOakluLC5Q9PppZPqCIz OZI9YChpiMnh3qR4MZ8rqsc/RiHQCc7hT0tDM7jWqZHaIqrl3/VBRCttJ suH/ESIXl5u6ViSjuTuoBZh+AJbQ2rdN/Mkq/DyNRHlNLFxo7irY5zJ/M Q==; X-CSE-ConnectionGUID: L/cu2zDcT9mLlZxKdatVJA== X-CSE-MsgGUID: 86CEY7AiSOy4/CO02BvU0A== X-IronPort-AV: E=McAfee;i="6800,10657,11480"; a="71048670" X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="71048670" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Jul 2025 02:59:10 -0700 X-CSE-ConnectionGUID: 50qUsqSFTjCgdkVjSs68gQ== X-CSE-MsgGUID: Vgf+hGwgSeeLMGPVR5qUIw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="177390828" Received: from black.fi.intel.com ([10.237.72.28]) by fmviesa002.fm.intel.com with ESMTP; 01 Jul 2025 02:58:58 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id 9FBA1627; Tue, 01 Jul 2025 12:58:50 +0300 (EEST) From: "Kirill A. Shutemov" To: Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Ard Biesheuvel , "Paul E. McKenney" , Josh Poimboeuf , Xiongwei Song , Xin Li , "Mike Rapoport (IBM)" , Brijesh Singh , Michael Roth , Tony Luck , Alexey Kardashevskiy , Alexander Shishkin Cc: Jonathan Corbet , Sohil Mehta , Ingo Molnar , Pawan Gupta , Daniel Sneddon , Kai Huang , Sandipan Das , Breno Leitao , Rick Edgecombe , Alexei Starovoitov , Hou Tao , Juergen Gross , Vegard Nossum , Kees Cook , Eric Biggers , Jason Gunthorpe , "Masami Hiramatsu (Google)" , Andrew Morton , Luis Chamberlain , Yuntao Wang , Rasmus Villemoes , Christophe Leroy , Tejun Heo , Changbin Du , Huang Shijie , Geert Uytterhoeven , Namhyung Kim , Arnaldo Carvalho de Melo , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-efi@vger.kernel.org, linux-mm@kvack.org, "Kirill A. Shutemov" Subject: [PATCHv8 09/17] x86/vsyscall: Add vsyscall emulation for #GP Date: Tue, 1 Jul 2025 12:58:38 +0300 Message-ID: <20250701095849.2360685-10-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> References: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Sohil Mehta The legacy vsyscall page is mapped at a fixed address in the kernel address range 0xffffffffff600000-0xffffffffff601000. Prior to LASS being introduced, a legacy vsyscall page access from userspace would always generate a page fault. The kernel emulates the execute (XONLY) accesses in the page fault handler and returns back to userspace with the appropriate register values. Since LASS intercepts these accesses before the paging structures are traversed it generates a general protection fault instead of a page fault. The #GP fault doesn't provide much information in terms of the error code. So, use the faulting RIP which is preserved in the user registers to emulate the vsyscall access without going through complex instruction decoding. Signed-off-by: Sohil Mehta Signed-off-by: Alexander Shishkin Signed-off-by: Kirill A. Shutemov --- arch/x86/entry/vsyscall/vsyscall_64.c | 14 +++++++++++++- arch/x86/include/asm/vsyscall.h | 6 ++++++ arch/x86/kernel/traps.c | 4 ++++ 3 files changed, 23 insertions(+), 1 deletion(-) diff --git a/arch/x86/entry/vsyscall/vsyscall_64.c b/arch/x86/entry/vsyscal= l/vsyscall_64.c index 25f94ac5fd35..be77385b311e 100644 --- a/arch/x86/entry/vsyscall/vsyscall_64.c +++ b/arch/x86/entry/vsyscall/vsyscall_64.c @@ -23,7 +23,7 @@ * soon be no new userspace code that will ever use a vsyscall. * * The code in this file emulates vsyscalls when notified of a page - * fault to a vsyscall address. + * fault or a general protection fault to a vsyscall address. */ =20 #include @@ -278,6 +278,18 @@ bool emulate_vsyscall_pf(unsigned long error_code, str= uct pt_regs *regs, return false; } =20 +bool emulate_vsyscall_gp(struct pt_regs *regs) +{ + if (!cpu_feature_enabled(X86_FEATURE_LASS)) + return false; + + /* Emulate only if the RIP points to the vsyscall address */ + if (!is_vsyscall_vaddr(regs->ip)) + return false; + + return __emulate_vsyscall(regs, regs->ip); +} + /* * A pseudo VMA to allow ptrace access for the vsyscall page. This only * covers the 64bit vsyscall page now. 32bit has a real VMA now and does diff --git a/arch/x86/include/asm/vsyscall.h b/arch/x86/include/asm/vsyscal= l.h index 214977f4fa11..4eb8d3673223 100644 --- a/arch/x86/include/asm/vsyscall.h +++ b/arch/x86/include/asm/vsyscall.h @@ -16,6 +16,7 @@ extern void set_vsyscall_pgtable_user_bits(pgd_t *root); */ extern bool emulate_vsyscall_pf(unsigned long error_code, struct pt_regs *regs, unsigned long address); +extern bool emulate_vsyscall_gp(struct pt_regs *regs); #else static inline void map_vsyscall(void) {} static inline bool emulate_vsyscall_pf(unsigned long error_code, @@ -23,6 +24,11 @@ static inline bool emulate_vsyscall_pf(unsigned long err= or_code, { return false; } + +static inline bool emulate_vsyscall_gp(struct pt_regs *regs) +{ + return false; +} #endif =20 /* diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 10856e0ac46c..40e34bb66d7c 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -69,6 +69,7 @@ #include #include #include +#include =20 #ifdef CONFIG_X86_64 #include @@ -817,6 +818,9 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection) if (fixup_umip_exception(regs)) goto exit; =20 + if (emulate_vsyscall_gp(regs)) + goto exit; + gp_user_force_sig_segv(regs, X86_TRAP_GP, error_code, desc); goto exit; } --=20 2.47.2 From nobody Wed Oct 8 06:47:47 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5E70127281A; Tue, 1 Jul 2025 09:59:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363954; cv=none; b=RnubQ9xS4LMMElQdNSj9hJMeth+MPhYLun60Eiuekdk28Vntw1+RN+ckiwAqFkNy/Dsa26wk/EGg+5r9vduuW7F4Ma2864SgPoRjRAD+ZcudOQnrEwtL/MgoKo/J7L7GK6jbyDiZW+02nvTRaoHbl0fR8mP4Y0uct2rkhBSdHRw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363954; c=relaxed/simple; bh=gRYbIS7Dgh1noZwFu/j8FTMijwGvmqKuvAcfA8tsGH0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LTpPaxN7iTtdgKSv6InBzQ5oJN3uJStibDwnBFHikpL/jcKnLQvzN4N53yjLC6Zigp64dSSbBhHk6RfXwhUYaSXZ1CAIPIhGsbnkPkhnUaPZaFf4l8DpUlAuMXNx725qlplAniz7Lcz9XHuMQBVGHF9ZAvo1xpFqVLuF6ia3qpI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.helo=mgamail.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=FYdQjS3m; arc=none smtp.client-ip=198.175.65.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.helo=mgamail.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="FYdQjS3m" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1751363952; x=1782899952; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=gRYbIS7Dgh1noZwFu/j8FTMijwGvmqKuvAcfA8tsGH0=; b=FYdQjS3mLgRtT0eD4mwRyp182zpCCWW0JZfLzSaYdCVXBwGyhO3lvhLU EXmLyy5QXfhjw+AhvhX8RnRpSW5yM3vDsnGhvm88akw4ZqbRjdbw5UXMz XCkKtfRGGHlflucOewrDRF8bFUImwfdHCeVJ7xeqjUZY1SUTPXAr9v4NU CQ/lL7Ng40v7yRsGR3p8ZAPrtj5O/G2AEZ9MuXt7iRr7OEF0S0gAJhGEk lcbNZmN0K/fr+R0O2Att6lQEAi4+l2ZBU00nW617Ip0v09QMOyzYNpVJM jCvoQz7+y2lFWZFpH2rEAMR3PWj6kq2jv3GzZxsWXs/N84KB2g5lLsYiQ g==; X-CSE-ConnectionGUID: bFWgVS+UQH6AJmUEf8XdMA== X-CSE-MsgGUID: DccjpOVBRQqFeCvsRT2z8A== X-IronPort-AV: E=McAfee;i="6800,10657,11480"; a="71048691" X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="71048691" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Jul 2025 02:59:10 -0700 X-CSE-ConnectionGUID: zWH4JyydQyiqU6Pd+Unzqw== X-CSE-MsgGUID: aYY1iB0TQ/KOPkw2mkRjpQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="177390831" Received: from black.fi.intel.com ([10.237.72.28]) by fmviesa002.fm.intel.com with ESMTP; 01 Jul 2025 02:58:58 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id AC55F679; Tue, 01 Jul 2025 12:58:50 +0300 (EEST) From: "Kirill A. Shutemov" To: Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Ard Biesheuvel , "Paul E. McKenney" , Josh Poimboeuf , Xiongwei Song , Xin Li , "Mike Rapoport (IBM)" , Brijesh Singh , Michael Roth , Tony Luck , Alexey Kardashevskiy , Alexander Shishkin Cc: Jonathan Corbet , Sohil Mehta , Ingo Molnar , Pawan Gupta , Daniel Sneddon , Kai Huang , Sandipan Das , Breno Leitao , Rick Edgecombe , Alexei Starovoitov , Hou Tao , Juergen Gross , Vegard Nossum , Kees Cook , Eric Biggers , Jason Gunthorpe , "Masami Hiramatsu (Google)" , Andrew Morton , Luis Chamberlain , Yuntao Wang , Rasmus Villemoes , Christophe Leroy , Tejun Heo , Changbin Du , Huang Shijie , Geert Uytterhoeven , Namhyung Kim , Arnaldo Carvalho de Melo , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-efi@vger.kernel.org, linux-mm@kvack.org, "Kirill A. Shutemov" Subject: [PATCHv8 10/17] x86/vsyscall: Disable LASS if vsyscall mode is set to EMULATE Date: Tue, 1 Jul 2025 12:58:39 +0300 Message-ID: <20250701095849.2360685-11-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> References: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Sohil Mehta The EMULATE mode of vsyscall maps the vsyscall page into user address space which can be read directly by the user application. This mode has been deprecated recently and can only be enabled from a special command line parameter vsyscall=3Demulate. See commit bf00745e7791 ("x86/vsyscall: Remove CONFIG_LEGACY_VSYSCALL_EMULATE") Fixing the LASS violations during the EMULATE mode would need complex instruction decoding since the resulting #GP fault does not include any useful error information and the vsyscall address is not readily available in the RIP. At this point, no one is expected to be using the insecure and deprecated EMULATE mode. The rare usages that need support probably don't care much about security anyway. Disable LASS when EMULATE mode is requested during command line parsing to avoid breaking user software. LASS will be supported if vsyscall mode is set to XONLY or NONE. Signed-off-by: Sohil Mehta Signed-off-by: Alexander Shishkin Signed-off-by: Kirill A. Shutemov --- Documentation/admin-guide/kernel-parameters.txt | 4 +++- arch/x86/entry/vsyscall/vsyscall_64.c | 7 +++++++ 2 files changed, 10 insertions(+), 1 deletion(-) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentatio= n/admin-guide/kernel-parameters.txt index f1f2c0874da9..796c987372df 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -7926,7 +7926,9 @@ =20 emulate Vsyscalls turn into traps and are emulated reasonably safely. The vsyscall page is - readable. + readable. This disables the Linear + Address Space Separation (LASS) security + feature and makes the system less secure. =20 xonly [default] Vsyscalls turn into traps and are emulated reasonably safely. The vsyscall diff --git a/arch/x86/entry/vsyscall/vsyscall_64.c b/arch/x86/entry/vsyscal= l/vsyscall_64.c index be77385b311e..d37df40bfb26 100644 --- a/arch/x86/entry/vsyscall/vsyscall_64.c +++ b/arch/x86/entry/vsyscall/vsyscall_64.c @@ -63,6 +63,13 @@ static int __init vsyscall_setup(char *str) else return -EINVAL; =20 + if (cpu_feature_enabled(X86_FEATURE_LASS) && + vsyscall_mode =3D=3D EMULATE) { + cr4_clear_bits(X86_CR4_LASS); + setup_clear_cpu_cap(X86_FEATURE_LASS); + pr_warn_once("x86/cpu: Disabling LASS support due to vsyscall=3Demulate= \n"); + } + return 0; } =20 --=20 2.47.2 From nobody Wed Oct 8 06:47:47 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 70DEC272E44; Tue, 1 Jul 2025 09:59:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363953; cv=none; b=Z638Vio0h0+uZeGnCDOwfeChkmotiQS6mclaqZxO6RLXSiKsXeuvHCtAtN6KYp2ncTk2Hyyyq/9M14cUIh85fZDbEYAI/MXMWcLSFM1wof69Ynj1U01lRFzWMc+FWE3gPGS+zX3pgICBzGNlpeUHEyerEVdwITAg3l0hNu7clOw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363953; c=relaxed/simple; bh=r8+6XTFMer08iaSTrK7+AgP3gettkLSvH05lTk9iIws=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tjFtZlXEyDVtZz8S02dvNmB4Q+IimZAHBN0rITVtE2zYJqs80DEyYXhG8bNq0UF4Ktk/qYNl1cGIdG81u4AfhhTZdpywtrLam9qEQrFiGVf8Xk/TC81r40NE+Iwi87vO2TUIhjHAuZeSOnbSavFUv2glNONGrsJq3ZIfOa2j2fU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.helo=mgamail.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=VxpPzHls; arc=none smtp.client-ip=198.175.65.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.helo=mgamail.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="VxpPzHls" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1751363952; x=1782899952; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=r8+6XTFMer08iaSTrK7+AgP3gettkLSvH05lTk9iIws=; b=VxpPzHlsEhPr3TKqrxcg1hZ104A+GdskIHCRklOiNM4kH9PDqHdmXMVj GX6R1dl+VApXBM4VqMHG9THuZF0Yjth3SiKsQpFHG+ucaYoHZb9h1Ldo/ F2CPNo5RemUob/gUS5hemZgDC71q/H/kmAUsYTQ2tINK46/ogepnya4Gi PsKSOXn4RUgPEWn0mXafgXnR3NDK8fgIoYmKQARtUTfFqhHZvz9EGan3y +YfLqpDe48WW5bgUegYK/YA4o0n9x1gOYIQ6BHXGt5Ajr5fHgvgkL7UnR wXGJ4LcGvWzDxDPT3sbM5Md1sr18y43tUHzQCJRLGFLniyzsv7ZoSZBkp A==; X-CSE-ConnectionGUID: lGWuOh/PR06YmjArZo2AdA== X-CSE-MsgGUID: EWRCLhobQ7qHvTX/J9UFpw== X-IronPort-AV: E=McAfee;i="6800,10657,11480"; a="71048712" X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="71048712" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Jul 2025 02:59:11 -0700 X-CSE-ConnectionGUID: arooEZEwTHWXkcqynRtFQg== X-CSE-MsgGUID: tYpqEaW/R7mjeQ4zpLJpPQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="177390834" Received: from black.fi.intel.com ([10.237.72.28]) by fmviesa002.fm.intel.com with ESMTP; 01 Jul 2025 02:58:59 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id B93C6778; Tue, 01 Jul 2025 12:58:50 +0300 (EEST) From: "Kirill A. Shutemov" To: Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Ard Biesheuvel , "Paul E. McKenney" , Josh Poimboeuf , Xiongwei Song , Xin Li , "Mike Rapoport (IBM)" , Brijesh Singh , Michael Roth , Tony Luck , Alexey Kardashevskiy , Alexander Shishkin Cc: Jonathan Corbet , Sohil Mehta , Ingo Molnar , Pawan Gupta , Daniel Sneddon , Kai Huang , Sandipan Das , Breno Leitao , Rick Edgecombe , Alexei Starovoitov , Hou Tao , Juergen Gross , Vegard Nossum , Kees Cook , Eric Biggers , Jason Gunthorpe , "Masami Hiramatsu (Google)" , Andrew Morton , Luis Chamberlain , Yuntao Wang , Rasmus Villemoes , Christophe Leroy , Tejun Heo , Changbin Du , Huang Shijie , Geert Uytterhoeven , Namhyung Kim , Arnaldo Carvalho de Melo , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-efi@vger.kernel.org, linux-mm@kvack.org, "Kirill A. Shutemov" Subject: [PATCHv8 11/17] x86/cpu: Set LASS CR4 bit as pinning sensitive Date: Tue, 1 Jul 2025 12:58:40 +0300 Message-ID: <20250701095849.2360685-12-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> References: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Yian Chen Security features such as LASS are not expected to be disabled once initialized. Add LASS to the CR4 pinned mask. Signed-off-by: Yian Chen Signed-off-by: Alexander Shishkin Reviewed-by: Tony Luck Signed-off-by: Kirill A. Shutemov Reviewed-by: Sohil Mehta --- arch/x86/kernel/cpu/common.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 9918121e0adc..1552c7510380 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -403,7 +403,8 @@ static __always_inline void setup_umip(struct cpuinfo_x= 86 *c) =20 /* These bits should not change their value after CPU init is finished. */ static const unsigned long cr4_pinned_mask =3D X86_CR4_SMEP | X86_CR4_SMAP= | X86_CR4_UMIP | - X86_CR4_FSGSBASE | X86_CR4_CET | X86_CR4_FRED; + X86_CR4_FSGSBASE | X86_CR4_CET | X86_CR4_FRED | + X86_CR4_LASS; static DEFINE_STATIC_KEY_FALSE_RO(cr_pinning); static unsigned long cr4_pinned_bits __ro_after_init; =20 --=20 2.47.2 From nobody Wed Oct 8 06:47:47 2025 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7CF80272E7E; Tue, 1 Jul 2025 09:59:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.12 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363954; cv=none; b=kCjxh9vWy1ufRonnpcuzpabt1FJkpRQ9ICvl8DHVpx0al982EvtV3HHSfKiSlrxWkR2C5Ymp/HEhNDPomXrUtsR2Ejl83TZqH36VGYjXHMiLip+Y9j71II8vILXDi3iGOAD7GLbLbV4OURJ2MWzYH9nT6d9PTK/GLnaf+oB0H5w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363954; c=relaxed/simple; bh=PwAnS8GCFl9R0ZnXkmWVweujaDST7jWhau/vhrZXQZY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RYcY2IzKbFrASIxCDjkRKTJPOAhoQ33LnrguLRO0qaqcC88ZD5wAYG176YsTvKVZ/5kXBJ0lVWuW3u4HLp6vz/AF2OUHjpKhni0WKrgPPV/ZOPTkE33hbr4g8Nx1xw4j55UVdun/eARIc3luBmDjLvTTPg3nP/qeRRU9GYERJdg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.helo=mgamail.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=PpANrVFW; arc=none smtp.client-ip=192.198.163.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.helo=mgamail.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="PpANrVFW" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1751363953; x=1782899953; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=PwAnS8GCFl9R0ZnXkmWVweujaDST7jWhau/vhrZXQZY=; b=PpANrVFWnFQzS82pSI2u6KBOWervEQGW+XRgHh+uCVfiH+zXzgiTZ3lQ NM/WrGDaRSGov5R20VVecFm5uRpmBC61OtPGB9wezMH7QoUf7RvuvCvRr G24EDe0Uu/TF5ycb3kcQLBvXlQCc6Ezl7ie4TWoWgMFLZtVAJqE/jfF+U 8aWlOJPbRhnTi0iVRVxrjSKEVDpkhglIjhIXhbOMzmFyKcEmmxAz73hZc IxNCfxevjmoU7bi7j1aIRIb4E+DgHgKsOTv4QSJntfdkah8C7lcqnW/Ry PiNe/FvaK9FKcWbYLOnctjNDr+6AV5KTv01jrscmuxq9X0cPSsy2QqgP4 Q==; X-CSE-ConnectionGUID: ujCE2VTPQLeHs0IuVgZV/A== X-CSE-MsgGUID: rk6M+QOSSXewlvCnZougyA== X-IronPort-AV: E=McAfee;i="6800,10657,11480"; a="57428114" X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="57428114" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Jul 2025 02:59:12 -0700 X-CSE-ConnectionGUID: FEfNDtZ7Sau6zPbETLZ06Q== X-CSE-MsgGUID: wH3R9HGFQ0yeIauuf+vyYw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="190896431" Received: from black.fi.intel.com ([10.237.72.28]) by orviesa001.jf.intel.com with ESMTP; 01 Jul 2025 02:59:00 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id C5534862; Tue, 01 Jul 2025 12:58:50 +0300 (EEST) From: "Kirill A. Shutemov" To: Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Ard Biesheuvel , "Paul E. McKenney" , Josh Poimboeuf , Xiongwei Song , Xin Li , "Mike Rapoport (IBM)" , Brijesh Singh , Michael Roth , Tony Luck , Alexey Kardashevskiy , Alexander Shishkin Cc: Jonathan Corbet , Sohil Mehta , Ingo Molnar , Pawan Gupta , Daniel Sneddon , Kai Huang , Sandipan Das , Breno Leitao , Rick Edgecombe , Alexei Starovoitov , Hou Tao , Juergen Gross , Vegard Nossum , Kees Cook , Eric Biggers , Jason Gunthorpe , "Masami Hiramatsu (Google)" , Andrew Morton , Luis Chamberlain , Yuntao Wang , Rasmus Villemoes , Christophe Leroy , Tejun Heo , Changbin Du , Huang Shijie , Geert Uytterhoeven , Namhyung Kim , Arnaldo Carvalho de Melo , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-efi@vger.kernel.org, linux-mm@kvack.org, "Kirill A. Shutemov" Subject: [PATCHv8 12/17] x86/traps: Communicate a LASS violation in #GP message Date: Tue, 1 Jul 2025 12:58:41 +0300 Message-ID: <20250701095849.2360685-13-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> References: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alexander Shishkin Provide a more helpful message on #GP when a kernel side LASS violation is detected. A NULL pointer dereference is reported if a LASS violation occurs due to accessing the first page frame. Signed-off-by: Alexander Shishkin Signed-off-by: Kirill A. Shutemov --- arch/x86/kernel/traps.c | 24 +++++++++++++++++++----- 1 file changed, 19 insertions(+), 5 deletions(-) diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 40e34bb66d7c..5206eb0ab01a 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -636,7 +636,16 @@ DEFINE_IDTENTRY(exc_bounds) enum kernel_gp_hint { GP_NO_HINT, GP_NON_CANONICAL, - GP_CANONICAL + GP_CANONICAL, + GP_LASS_VIOLATION, + GP_NULL_POINTER, +}; + +static const char * const kernel_gp_hint_help[] =3D { + [GP_NON_CANONICAL] =3D "probably for non-canonical address", + [GP_CANONICAL] =3D "maybe for address", + [GP_LASS_VIOLATION] =3D "LASS prevented access to address", + [GP_NULL_POINTER] =3D "kernel NULL pointer dereference", }; =20 /* @@ -672,6 +681,12 @@ static enum kernel_gp_hint get_kernel_gp_address(struc= t pt_regs *regs, if (*addr < ~__VIRTUAL_MASK && *addr + insn.opnd_bytes - 1 > __VIRTUAL_MASK) return GP_NON_CANONICAL; + else if (*addr < ~__VIRTUAL_MASK && + cpu_feature_enabled(X86_FEATURE_LASS)) { + if (*addr < PAGE_SIZE) + return GP_NULL_POINTER; + return GP_LASS_VIOLATION; + } #endif =20 return GP_CANONICAL; @@ -833,11 +848,10 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection) else hint =3D get_kernel_gp_address(regs, &gp_addr); =20 - if (hint !=3D GP_NO_HINT) + if (hint !=3D GP_NO_HINT) { snprintf(desc, sizeof(desc), GPFSTR ", %s 0x%lx", - (hint =3D=3D GP_NON_CANONICAL) ? "probably for non-canonical address" - : "maybe for address", - gp_addr); + kernel_gp_hint_help[hint], gp_addr); + } =20 /* * KASAN is interested only in the non-canonical case, clear it --=20 2.47.2 From nobody Wed Oct 8 06:47:47 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 15F84221281; Tue, 1 Jul 2025 09:59:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363964; cv=none; b=AhUcQn37vB+iADKWEJbKbg5icZdocQlx+nZ7nXq6cSGdQjua2vTyj9+d2D240//Ewp+T9c6ugNPFUVcF3S/4u/pH6tZizgL7tv3iLasFHfmINhgZFcd1f3KAyfYyemIdVMAkqtxdWrhAAelLf2Luv1CCfQ9UqKXFf5YflPrFtl8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363964; c=relaxed/simple; bh=BDM4GULv098OkgaG9+t85KmDsV77Wx362tvBjQprKsg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SHOBImQcp1mzdzMliGp9O22Z6tjN98OYNkdPPgWkRspspuVkw7og8pMptpOBpFSJUhsQiyyR7wiOjC8gPh66dJxPYeRupC1oiCUZe0MPrLuIgTgFKa5VFjsVR0ILcFNxnYD8PV0gYZ0/Us+6t/6PfUNoFzcbJ+ZGxyeHA6CNZIk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.helo=mgamail.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=WZg4iR85; arc=none smtp.client-ip=198.175.65.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.helo=mgamail.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="WZg4iR85" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1751363964; x=1782899964; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=BDM4GULv098OkgaG9+t85KmDsV77Wx362tvBjQprKsg=; b=WZg4iR850EfFVt82YCjwO7jp06CE5I/5OKYy+6Y2yPrlITRgN+v9Dlkm PeeOx4fdgeiqDkxHj+yFveOSq1dxH6aWK9LjgHE+KQTFvn4JuOeyNDu1D 0ror5A72WSdLLg0hq6O3QPnXwiQr+VrHmoFxOObFSDLC6e2WX1yl/grSD l3pHfZxzdbloPITZjE6xf7YwNdPA1h+hwH0ZqZXG0QuQoz3+lzDQOqiHt BBsVNZ8kSQvowsVxVlshXbyfZL8tG6snge6hQrqs7wzcmb1ZXxNU9TQXT C2l+HqCaMEPsc63wYZho2SkZ6jKM/dJE7EVn44eU2LsiGN8nTWse/aBNY Q==; X-CSE-ConnectionGUID: EOtLsbgcQ4q/DV2cab/MAw== X-CSE-MsgGUID: HBqwSWHBSEelqS1mn/extg== X-IronPort-AV: E=McAfee;i="6800,10657,11480"; a="71048738" X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="71048738" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Jul 2025 02:59:23 -0700 X-CSE-ConnectionGUID: SwaahxybTzSPHC8U1d1nQQ== X-CSE-MsgGUID: vDnNRCLrTGWlKa5z5g43AA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="177390839" Received: from black.fi.intel.com ([10.237.72.28]) by fmviesa002.fm.intel.com with ESMTP; 01 Jul 2025 02:59:01 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id D19BA892; Tue, 01 Jul 2025 12:58:50 +0300 (EEST) From: "Kirill A. Shutemov" To: Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Ard Biesheuvel , "Paul E. McKenney" , Josh Poimboeuf , Xiongwei Song , Xin Li , "Mike Rapoport (IBM)" , Brijesh Singh , Michael Roth , Tony Luck , Alexey Kardashevskiy , Alexander Shishkin Cc: Jonathan Corbet , Sohil Mehta , Ingo Molnar , Pawan Gupta , Daniel Sneddon , Kai Huang , Sandipan Das , Breno Leitao , Rick Edgecombe , Alexei Starovoitov , Hou Tao , Juergen Gross , Vegard Nossum , Kees Cook , Eric Biggers , Jason Gunthorpe , "Masami Hiramatsu (Google)" , Andrew Morton , Luis Chamberlain , Yuntao Wang , Rasmus Villemoes , Christophe Leroy , Tejun Heo , Changbin Du , Huang Shijie , Geert Uytterhoeven , Namhyung Kim , Arnaldo Carvalho de Melo , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-efi@vger.kernel.org, linux-mm@kvack.org, "Kirill A. Shutemov" Subject: [PATCHv8 13/17] x86/traps: Generalize #GP address decode and hint code Date: Tue, 1 Jul 2025 12:58:42 +0300 Message-ID: <20250701095849.2360685-14-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> References: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Handlers for #GP and #SS will now share code to decode the exception address and retrieve the exception hint string. The helper, enum, and array should be renamed as they are no longer specific to #GP. No functional change intended. Signed-off-by: Kirill A. Shutemov --- arch/x86/kernel/traps.c | 62 ++++++++++++++++++++--------------------- 1 file changed, 31 insertions(+), 31 deletions(-) diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index 5206eb0ab01a..ceb091f17a5b 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -633,28 +633,28 @@ DEFINE_IDTENTRY(exc_bounds) cond_local_irq_disable(regs); } =20 -enum kernel_gp_hint { - GP_NO_HINT, - GP_NON_CANONICAL, - GP_CANONICAL, - GP_LASS_VIOLATION, - GP_NULL_POINTER, +enum kernel_exc_hint { + EXC_NO_HINT, + EXC_NON_CANONICAL, + EXC_CANONICAL, + EXC_LASS_VIOLATION, + EXC_NULL_POINTER, }; =20 -static const char * const kernel_gp_hint_help[] =3D { - [GP_NON_CANONICAL] =3D "probably for non-canonical address", - [GP_CANONICAL] =3D "maybe for address", - [GP_LASS_VIOLATION] =3D "LASS prevented access to address", - [GP_NULL_POINTER] =3D "kernel NULL pointer dereference", +static const char * const kernel_exc_hint_help[] =3D { + [EXC_NON_CANONICAL] =3D "probably for non-canonical address", + [EXC_CANONICAL] =3D "maybe for address", + [EXC_LASS_VIOLATION] =3D "LASS prevented access to address", + [EXC_NULL_POINTER] =3D "kernel NULL pointer dereference", }; =20 /* - * When an uncaught #GP occurs, try to determine the memory address access= ed by - * the instruction and return that address to the caller. Also, try to fig= ure - * out whether any part of the access to that address was non-canonical. + * When an uncaught #GP/#SS occurs, try to determine the memory address ac= cessed + * by the instruction and return that address to the caller. Also, try to + * figure out whether any part of the access to that address was non-canon= ical. */ -static enum kernel_gp_hint get_kernel_gp_address(struct pt_regs *regs, - unsigned long *addr) +static enum kernel_exc_hint get_kernel_exc_address(struct pt_regs *regs, + unsigned long *addr) { u8 insn_buf[MAX_INSN_SIZE]; struct insn insn; @@ -662,15 +662,15 @@ static enum kernel_gp_hint get_kernel_gp_address(stru= ct pt_regs *regs, =20 if (copy_from_kernel_nofault(insn_buf, (void *)regs->ip, MAX_INSN_SIZE)) - return GP_NO_HINT; + return EXC_NO_HINT; =20 ret =3D insn_decode_kernel(&insn, insn_buf); if (ret < 0) - return GP_NO_HINT; + return EXC_NO_HINT; =20 *addr =3D (unsigned long)insn_get_addr_ref(&insn, regs); if (*addr =3D=3D -1UL) - return GP_NO_HINT; + return EXC_NO_HINT; =20 #ifdef CONFIG_X86_64 /* @@ -680,16 +680,16 @@ static enum kernel_gp_hint get_kernel_gp_address(stru= ct pt_regs *regs, */ if (*addr < ~__VIRTUAL_MASK && *addr + insn.opnd_bytes - 1 > __VIRTUAL_MASK) - return GP_NON_CANONICAL; + return EXC_NON_CANONICAL; else if (*addr < ~__VIRTUAL_MASK && cpu_feature_enabled(X86_FEATURE_LASS)) { if (*addr < PAGE_SIZE) - return GP_NULL_POINTER; - return GP_LASS_VIOLATION; + return EXC_NULL_POINTER; + return EXC_LASS_VIOLATION; } #endif =20 - return GP_CANONICAL; + return EXC_CANONICAL; } =20 #define GPFSTR "general protection fault" @@ -808,8 +808,8 @@ static void gp_user_force_sig_segv(struct pt_regs *regs= , int trapnr, DEFINE_IDTENTRY_ERRORCODE(exc_general_protection) { char desc[sizeof(GPFSTR) + 50 + 2*sizeof(unsigned long) + 1] =3D GPFSTR; - enum kernel_gp_hint hint =3D GP_NO_HINT; - unsigned long gp_addr; + enum kernel_exc_hint hint =3D EXC_NO_HINT; + unsigned long exc_addr; =20 if (user_mode(regs) && try_fixup_enqcmd_gp()) return; @@ -846,21 +846,21 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection) if (error_code) snprintf(desc, sizeof(desc), "segment-related " GPFSTR); else - hint =3D get_kernel_gp_address(regs, &gp_addr); + hint =3D get_kernel_exc_address(regs, &exc_addr); =20 - if (hint !=3D GP_NO_HINT) { + if (hint !=3D EXC_NO_HINT) { snprintf(desc, sizeof(desc), GPFSTR ", %s 0x%lx", - kernel_gp_hint_help[hint], gp_addr); + kernel_exc_hint_help[hint], exc_addr); } =20 /* * KASAN is interested only in the non-canonical case, clear it * otherwise. */ - if (hint !=3D GP_NON_CANONICAL) - gp_addr =3D 0; + if (hint !=3D EXC_NON_CANONICAL) + exc_addr =3D 0; =20 - die_addr(desc, regs, error_code, gp_addr); + die_addr(desc, regs, error_code, exc_addr); =20 exit: cond_local_irq_disable(regs); --=20 2.47.2 From nobody Wed Oct 8 06:47:47 2025 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.10]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A19F226FD8F; Tue, 1 Jul 2025 09:59:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.10 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363966; cv=none; b=YXpgNyTcSq3V7khxph5nxKPlJe52wq70azp2deTZOwKZVCXis7in5pKfUbQh5vTbK1jyoBasKIXz5Hsswido3V18r2L7F+F8FFga2jEnzbJFjSKp4NoLkdm25qiiS70MRju+E1kWj04nh6err9rEmHYxlSybRJRtnEmkoInlKn0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363966; c=relaxed/simple; bh=qO+8BSDMXd1kOh4db6N91L5SEjVjt9oi3Mr4Xl+KkEE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lwxD3QuzyRn1TG745MShYJ7qm30JsrUeuYrWGcX1GLiqBBXYK43mk9CBfUEX2Ahn9iLsXPIZ5d1bEAEl6rLPq3NqH3Kz8IIjpKZK8SsvXisD3D/BggeWaSgTBsc4j4wITfBTPMuVi/PBJYJGdHeqVI+yPf/PJkBCu63p1mSz9uw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.helo=mgamail.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=WmaMpHVG; arc=none smtp.client-ip=198.175.65.10 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.helo=mgamail.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="WmaMpHVG" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1751363965; x=1782899965; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=qO+8BSDMXd1kOh4db6N91L5SEjVjt9oi3Mr4Xl+KkEE=; b=WmaMpHVGop6nEsng66U5j/rGLfp6s+xASHTJvTDP0HKupiRdPkOuBcco x7v/AioGDfewyMmkm0oEzddI72nJx6jh/TCTKyDTWhepuUCVdMmPJL42G 52SvoLlU4ughO6r2uH21n+YPfGPI+KTCy3d831Xze+sgdwsTtvm7Be2cv tN6gF89CaZu4Ndg3hiexe8q6G8Nqiw8HT7AX3IMzcRLDD3WsHbNFqDrsY i+UWX2mz+BF0MtvhFmrZQTfGLnZWQhpCBwkf6T0KlUjfzkMHTXQZTBRXg IOVlZSmGcsNcMrx7ENhJYl5fG44jwyow/3aJ2DJdxyoYbltfI5FKphYAI w==; X-CSE-ConnectionGUID: INofQOkhR8uKwSMrC/jCHA== X-CSE-MsgGUID: B+hk4MSjQXmi70Z6aharZg== X-IronPort-AV: E=McAfee;i="6800,10657,11480"; a="71048766" X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="71048766" Received: from fmviesa002.fm.intel.com ([10.60.135.142]) by orvoesa102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Jul 2025 02:59:23 -0700 X-CSE-ConnectionGUID: aOqy9tIvQbafvLKOgawDGg== X-CSE-MsgGUID: auP5LPesQ/m7qenBhDetrg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="177390854" Received: from black.fi.intel.com ([10.237.72.28]) by fmviesa002.fm.intel.com with ESMTP; 01 Jul 2025 02:59:01 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id DDBB9893; Tue, 01 Jul 2025 12:58:50 +0300 (EEST) From: "Kirill A. Shutemov" To: Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Ard Biesheuvel , "Paul E. McKenney" , Josh Poimboeuf , Xiongwei Song , Xin Li , "Mike Rapoport (IBM)" , Brijesh Singh , Michael Roth , Tony Luck , Alexey Kardashevskiy , Alexander Shishkin Cc: Jonathan Corbet , Sohil Mehta , Ingo Molnar , Pawan Gupta , Daniel Sneddon , Kai Huang , Sandipan Das , Breno Leitao , Rick Edgecombe , Alexei Starovoitov , Hou Tao , Juergen Gross , Vegard Nossum , Kees Cook , Eric Biggers , Jason Gunthorpe , "Masami Hiramatsu (Google)" , Andrew Morton , Luis Chamberlain , Yuntao Wang , Rasmus Villemoes , Christophe Leroy , Tejun Heo , Changbin Du , Huang Shijie , Geert Uytterhoeven , Namhyung Kim , Arnaldo Carvalho de Melo , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-efi@vger.kernel.org, linux-mm@kvack.org, "Kirill A. Shutemov" Subject: [PATCHv8 14/17] x86/traps: Handle LASS thrown #SS Date: Tue, 1 Jul 2025 12:58:43 +0300 Message-ID: <20250701095849.2360685-15-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> References: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" LASS throws a #GP for any violations except for stack register accesses, in which case it throws a #SS instead. Handle this similarly to how other LASS violations are handled. In case of FRED, before handling #SS as LASS violation, kernel has to check if there's a fixup for the exception. It can address #SS due to invalid user context on ERETU. See 5105e7687ad3 ("x86/fred: Fixup fault on ERETU by jumping to fred_entrypoint_user") for more details. Co-developed-by: Alexander Shishkin Signed-off-by: Alexander Shishkin Signed-off-by: Kirill A. Shutemov --- arch/x86/kernel/traps.c | 39 +++++++++++++++++++++++++++++++++------ 1 file changed, 33 insertions(+), 6 deletions(-) diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c index ceb091f17a5b..f9ca5b911141 100644 --- a/arch/x86/kernel/traps.c +++ b/arch/x86/kernel/traps.c @@ -418,12 +418,6 @@ DEFINE_IDTENTRY_ERRORCODE(exc_segment_not_present) SIGBUS, 0, NULL); } =20 -DEFINE_IDTENTRY_ERRORCODE(exc_stack_segment) -{ - do_error_trap(regs, error_code, "stack segment", X86_TRAP_SS, SIGBUS, - 0, NULL); -} - DEFINE_IDTENTRY_ERRORCODE(exc_alignment_check) { char *str =3D "alignment check"; @@ -866,6 +860,39 @@ DEFINE_IDTENTRY_ERRORCODE(exc_general_protection) cond_local_irq_disable(regs); } =20 +#define SSFSTR "stack segment fault" + +DEFINE_IDTENTRY_ERRORCODE(exc_stack_segment) +{ + if (user_mode(regs)) + goto error_trap; + + if (cpu_feature_enabled(X86_FEATURE_FRED) && + fixup_exception(regs, X86_TRAP_SS, error_code, 0)) + return; + + if (cpu_feature_enabled(X86_FEATURE_LASS)) { + enum kernel_exc_hint hint; + unsigned long exc_addr; + + hint =3D get_kernel_exc_address(regs, &exc_addr); + if (hint !=3D EXC_NO_HINT) { + printk(SSFSTR ", %s 0x%lx", kernel_exc_hint_help[hint], + exc_addr); + } + + if (hint !=3D EXC_NON_CANONICAL) + exc_addr =3D 0; + + die_addr(SSFSTR, regs, error_code, exc_addr); + return; + } + +error_trap: + do_error_trap(regs, error_code, "stack segment", X86_TRAP_SS, SIGBUS, + 0, NULL); +} + static bool do_int3(struct pt_regs *regs) { int res; --=20 2.47.2 From nobody Wed Oct 8 06:47:48 2025 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DFD31273D94; Tue, 1 Jul 2025 09:59:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.12 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363956; cv=none; b=SjpneiIyTSiYIH7nxzTxaKtV8XuVe94W2QyBPovAh9v7ZE1vUT1X61YtCg5Y8Rd5mzJkD3HyQXIr9XhfeR9tu0835cQDk5oBPdZG0cLfBtdq5IGbfHonwOAvaL9BSsFqIT5DWhdikK+X/EF3tBjkQKb6acIt9cY7QQKXXWStV4I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363956; c=relaxed/simple; bh=UXQro1nhuXHnLN9WvBTrdf+zqqavYnpmltBFM1Abf3w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bO2QzOnHEx39XvSNTXJhC317ym8JCgOeiGnzQQEp995FkbQ2e57iwhjePOxemrTuDf5HmOMP/D1QOsGJKRm8fxKI8c19ZTPCuOribsdcyv5E8VETpUVLpDM9SheLrzACs1Y9X5nVk2FbyWYG/lni94Pe2brhxn0B7vEBiuzPBdM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.helo=mgamail.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=ejfPYMLg; arc=none smtp.client-ip=192.198.163.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.helo=mgamail.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="ejfPYMLg" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1751363955; x=1782899955; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=UXQro1nhuXHnLN9WvBTrdf+zqqavYnpmltBFM1Abf3w=; b=ejfPYMLgrwFcX95hulEJYO1dBdCj7KFTKjxDc7AGqQzsCUUsaNBVgjHE xYpWCfJ9PecjQaYeNnMafanPBw7vphf+Zlf5jg8nTl2ZkGB5kgvZdRN7r TxUR/KSOjTRo1dKuH+EH56XhWfXZXp921LLdZkzitB5UsiAja5z3EiHxS U+WNv2CGbgPXMUgUgE+YdsTswHCjfZ1JwCN+Oj4YsbqnpYDOlVaZtP39u 9D1lBABuGojAMCkahypsKBIEo+qJXw1PAU9RVOEjRskbmhQ3g9bDTduXb iM6txixxAtHzNECuRyOZ+Y2beOzAWxLBvzTng6U0t3dntY0GWbZs2crFi Q==; X-CSE-ConnectionGUID: j0lR17VMRZ6ISkkdrlJQ/Q== X-CSE-MsgGUID: KgZ9nYxySSyLsdbpwKZkIQ== X-IronPort-AV: E=McAfee;i="6800,10657,11480"; a="57428151" X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="57428151" Received: from orviesa001.jf.intel.com ([10.64.159.141]) by fmvoesa106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Jul 2025 02:59:14 -0700 X-CSE-ConnectionGUID: 2JOVmV3pT+qr+n/Ir31q3A== X-CSE-MsgGUID: xXwqX3z+S2eyf6yOYGz41w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="190896444" Received: from black.fi.intel.com ([10.237.72.28]) by orviesa001.jf.intel.com with ESMTP; 01 Jul 2025 02:59:03 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id E9CF28E9; Tue, 01 Jul 2025 12:58:50 +0300 (EEST) From: "Kirill A. Shutemov" To: Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Ard Biesheuvel , "Paul E. McKenney" , Josh Poimboeuf , Xiongwei Song , Xin Li , "Mike Rapoport (IBM)" , Brijesh Singh , Michael Roth , Tony Luck , Alexey Kardashevskiy , Alexander Shishkin Cc: Jonathan Corbet , Sohil Mehta , Ingo Molnar , Pawan Gupta , Daniel Sneddon , Kai Huang , Sandipan Das , Breno Leitao , Rick Edgecombe , Alexei Starovoitov , Hou Tao , Juergen Gross , Vegard Nossum , Kees Cook , Eric Biggers , Jason Gunthorpe , "Masami Hiramatsu (Google)" , Andrew Morton , Luis Chamberlain , Yuntao Wang , Rasmus Villemoes , Christophe Leroy , Tejun Heo , Changbin Du , Huang Shijie , Geert Uytterhoeven , Namhyung Kim , Arnaldo Carvalho de Melo , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-efi@vger.kernel.org, linux-mm@kvack.org, "Kirill A. Shutemov" Subject: [PATCHv8 15/17] x86/cpu: Make LAM depend on LASS Date: Tue, 1 Jul 2025 12:58:44 +0300 Message-ID: <20250701095849.2360685-16-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> References: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Alexander Shishkin To prevent exploits for Spectre based on LAM as demonstrated by the whitepaper [1], make LAM depend on LASS, which avoids this type of vulnerability. [1] https://download.vusec.net/papers/slam_sp24.pdf Signed-off-by: Alexander Shishkin Signed-off-by: Kirill A. Shutemov Reviewed-by: Sohil Mehta --- arch/x86/kernel/cpu/cpuid-deps.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/x86/kernel/cpu/cpuid-deps.c b/arch/x86/kernel/cpu/cpuid-d= eps.c index 98d0cdd82574..11bb9ed40140 100644 --- a/arch/x86/kernel/cpu/cpuid-deps.c +++ b/arch/x86/kernel/cpu/cpuid-deps.c @@ -90,6 +90,7 @@ static const struct cpuid_dep cpuid_deps[] =3D { { X86_FEATURE_FRED, X86_FEATURE_LKGS }, { X86_FEATURE_SPEC_CTRL_SSBD, X86_FEATURE_SPEC_CTRL }, { X86_FEATURE_LASS, X86_FEATURE_SMAP }, + { X86_FEATURE_LAM, X86_FEATURE_LASS }, {} }; =20 --=20 2.47.2 From nobody Wed Oct 8 06:47:48 2025 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 82DA22701D9; Tue, 1 Jul 2025 09:59:35 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.12 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363977; cv=none; b=mD4t+gKbcuttmoR1MpLiE4Pk3lYyROLYdoR+WkLIQsZOrXE00DdWQ6jyhQuME6p6WQ2PD9Fwq037ccIwq9g80eLlWxQhiXcHMbNDSWnpUREUF1AONndebAAes82JLUuPX8uMYnZ1nMx3goLV2g9ii929/Git3jchFplR/PMyTlY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363977; c=relaxed/simple; bh=OjJNhJk2T736RflVUZxtae35YdvRzXSJjxBWWJGThLU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=oSqAXxI8r0lRVnFYW38S07T6OcU10h7fSbabwg4j1+FCnfSe7dWe3lio7gBIRhtbUxvxE6y+Kah9NrSwl82+jd8GWLcQ/WkO99r1j08n1uwKX/N4IyYJuU13IYMXk9vA5tCvrlJwzUhb5R9drP1cxskmH4KyQTspvqcGGki13eM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.helo=mgamail.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=YP/DMTij; arc=none smtp.client-ip=192.198.163.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.helo=mgamail.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="YP/DMTij" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1751363976; x=1782899976; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=OjJNhJk2T736RflVUZxtae35YdvRzXSJjxBWWJGThLU=; b=YP/DMTijQqpMtGIMzsPTvzIDrMnKZGHi02km6MRFFDk64YfrpWHJIyYv x0GGEdjlcvWiWVODmyDkT3AGH5voTsMTK5joT1tawgVlJwmnIvNmIa45t gw4aPgObBjgfJUKrALzpc0ZW95SW2vALY7vq1YSLTDKCYB8Et1YvPjK4I TBnY+8sV3hgbeWWMrglYmjQkm4lBsbtbr9YVD30xgNNjJBh7ZYnfe/8NK +12ecyr0vLBc1p0V2WdyVUE/25wMEWK6CcuzlTZ2rOuQ+OnSCsdZNtRvZ f40irAaT07Vzp0IKNAPSKH39auRPWv9texeWUK5De7NqygA5dNtB0VGVY g==; X-CSE-ConnectionGUID: OaEi3tHTT7CmW2iPncda9Q== X-CSE-MsgGUID: eG0Y97r4TbCA9smnaRNWHg== X-IronPort-AV: E=McAfee;i="6800,10657,11480"; a="57428212" X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="57428212" Received: from orviesa008.jf.intel.com ([10.64.159.148]) by fmvoesa106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Jul 2025 02:59:15 -0700 X-CSE-ConnectionGUID: wkESWtlpRQ+HNh/NQSpitw== X-CSE-MsgGUID: ldjqWFxdSzaadhor3s4cTA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="154216196" Received: from black.fi.intel.com ([10.237.72.28]) by orviesa008.jf.intel.com with ESMTP; 01 Jul 2025 02:59:03 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id 03531928; Tue, 01 Jul 2025 12:58:50 +0300 (EEST) From: "Kirill A. Shutemov" To: Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Ard Biesheuvel , "Paul E. McKenney" , Josh Poimboeuf , Xiongwei Song , Xin Li , "Mike Rapoport (IBM)" , Brijesh Singh , Michael Roth , Tony Luck , Alexey Kardashevskiy , Alexander Shishkin Cc: Jonathan Corbet , Sohil Mehta , Ingo Molnar , Pawan Gupta , Daniel Sneddon , Kai Huang , Sandipan Das , Breno Leitao , Rick Edgecombe , Alexei Starovoitov , Hou Tao , Juergen Gross , Vegard Nossum , Kees Cook , Eric Biggers , Jason Gunthorpe , "Masami Hiramatsu (Google)" , Andrew Morton , Luis Chamberlain , Yuntao Wang , Rasmus Villemoes , Christophe Leroy , Tejun Heo , Changbin Du , Huang Shijie , Geert Uytterhoeven , Namhyung Kim , Arnaldo Carvalho de Melo , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-efi@vger.kernel.org, linux-mm@kvack.org, "Kirill A. Shutemov" Subject: [PATCHv8 16/17] x86/cpu: Enable LASS during CPU initialization Date: Tue, 1 Jul 2025 12:58:45 +0300 Message-ID: <20250701095849.2360685-17-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> References: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Sohil Mehta Being a security feature, enable LASS by default if the platform supports it. While at it, get rid of the comment above the SMAP/SMEP/UMIP/LASS setup instead of updating it to mention LASS as well, as the whole sequence is quite self-explanatory. Signed-off-by: Sohil Mehta Signed-off-by: Alexander Shishkin Signed-off-by: Kirill A. Shutemov --- arch/x86/kernel/cpu/common.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 1552c7510380..97a228f917a9 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -401,6 +401,12 @@ static __always_inline void setup_umip(struct cpuinfo_= x86 *c) cr4_clear_bits(X86_CR4_UMIP); } =20 +static __always_inline void setup_lass(struct cpuinfo_x86 *c) +{ + if (cpu_feature_enabled(X86_FEATURE_LASS)) + cr4_set_bits(X86_CR4_LASS); +} + /* These bits should not change their value after CPU init is finished. */ static const unsigned long cr4_pinned_mask =3D X86_CR4_SMEP | X86_CR4_SMAP= | X86_CR4_UMIP | X86_CR4_FSGSBASE | X86_CR4_CET | X86_CR4_FRED | @@ -1975,10 +1981,10 @@ static void identify_cpu(struct cpuinfo_x86 *c) /* Disable the PN if appropriate */ squash_the_stupid_serial_number(c); =20 - /* Set up SMEP/SMAP/UMIP */ setup_smep(c); setup_smap(c); setup_umip(c); + setup_lass(c); =20 /* Enable FSGSBASE instructions if available. */ if (cpu_has(c, X86_FEATURE_FSGSBASE)) { --=20 2.47.2 From nobody Wed Oct 8 06:47:48 2025 Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.12]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E498E2701BD; Tue, 1 Jul 2025 09:59:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.12 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363975; cv=none; b=uQGIV15CB6VpCxn2//Jrow22uDmx3ic+uexL7SPT9W9m092bHx212j9bVgkUmxdEl4H2DqNnMiI1NJa8x6KBQL3JPnw1gdCpGTVz8fvOJUUs6iIr8jOgRt2L4haMnrR1VxmvRIOXuC7rLGKI8Wen4GQZxvo0EknZ9WTFxj0iXkY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1751363975; c=relaxed/simple; bh=gDZAIaId/LE+Cg3g34PHSl3JoHL8xvSE+GTgwe6XlKg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qvSxmLZA+rLDTZ9LK+vmiJsY3Va0T+S7w0i37iH2KKhF34qJTxifneJLyqyKhBWNqZ2sqEbKCj6jY0umrJsEawoMVcv5Vt1qEOyLrLTcsIqwsor6RV1GecP4rkSycVzoxuH0wzWOWDokOCBLSQwQea2Q1DClYu+tSvoQeBsjAmE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com; spf=none smtp.helo=mgamail.intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=co2VDTSD; arc=none smtp.client-ip=192.198.163.12 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.intel.com Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.helo=mgamail.intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="co2VDTSD" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1751363974; x=1782899974; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=gDZAIaId/LE+Cg3g34PHSl3JoHL8xvSE+GTgwe6XlKg=; b=co2VDTSD0PYrflX1PuFRQgcJTNlr5UgfEObQLTlhJ/tYPA9aMC5R6t7g bJTZeOU7VTMmVojLB5zSRGeG2mWQAQgLZkpmHvnW8YQ3PB4uY3nOJSPBN /unXuSbju3vCtlmzMX1EJwP9rySnq8XrkperkEPVFDkR5nnGb2GJiVwYz EeDtsi1AlFU0qTZ7qWx7D+xJSoXNIBY+/8QZ01RRlVte4NJhR3PcaatPw m1GJXhWN7yqcp+qui77ztvjQDAWItm9ChUlMOhMiZUH7lWlB28q3u2MQZ MR34Ik5zaBx7IoFrxJLJp2sAh6DBUlWSJqxCq0eK54H4kbRQrFk6LqhRI Q==; X-CSE-ConnectionGUID: QFfVSuAORNWfDPa0BYSQyQ== X-CSE-MsgGUID: sDMBNmCbR/2Z9f3nD7z88Q== X-IronPort-AV: E=McAfee;i="6800,10657,11480"; a="57428180" X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="57428180" Received: from orviesa008.jf.intel.com ([10.64.159.148]) by fmvoesa106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 01 Jul 2025 02:59:15 -0700 X-CSE-ConnectionGUID: VVUtxzUdTiK9KNiZjO9P5g== X-CSE-MsgGUID: VdrXhZFsR1OJpKY88wtqxg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.16,279,1744095600"; d="scan'208";a="154216194" Received: from black.fi.intel.com ([10.237.72.28]) by orviesa008.jf.intel.com with ESMTP; 01 Jul 2025 02:59:03 -0700 Received: by black.fi.intel.com (Postfix, from userid 1000) id 103AF983; Tue, 01 Jul 2025 12:58:51 +0300 (EEST) From: "Kirill A. Shutemov" To: Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , x86@kernel.org, "H. Peter Anvin" , Peter Zijlstra , Ard Biesheuvel , "Paul E. McKenney" , Josh Poimboeuf , Xiongwei Song , Xin Li , "Mike Rapoport (IBM)" , Brijesh Singh , Michael Roth , Tony Luck , Alexey Kardashevskiy , Alexander Shishkin Cc: Jonathan Corbet , Sohil Mehta , Ingo Molnar , Pawan Gupta , Daniel Sneddon , Kai Huang , Sandipan Das , Breno Leitao , Rick Edgecombe , Alexei Starovoitov , Hou Tao , Juergen Gross , Vegard Nossum , Kees Cook , Eric Biggers , Jason Gunthorpe , "Masami Hiramatsu (Google)" , Andrew Morton , Luis Chamberlain , Yuntao Wang , Rasmus Villemoes , Christophe Leroy , Tejun Heo , Changbin Du , Huang Shijie , Geert Uytterhoeven , Namhyung Kim , Arnaldo Carvalho de Melo , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-efi@vger.kernel.org, linux-mm@kvack.org, "Kirill A. Shutemov" Subject: [PATCHv8 17/17] x86: Re-enable Linear Address Masking Date: Tue, 1 Jul 2025 12:58:46 +0300 Message-ID: <20250701095849.2360685-18-kirill.shutemov@linux.intel.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> References: <20250701095849.2360685-1-kirill.shutemov@linux.intel.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This reverts commit 3267cb6d3a174ff83d6287dcd5b0047bbd912452. LASS mitigates the Spectre based on LAM (SLAM) [1] and the previous commit made LAM depend on LASS, so we no longer need to disable LAM at compile time, so revert the commit that disables LAM. Adjust USER_PTR_MAX if LAM enabled, allowing tag bits to be set for userspace pointers. The value for the constant is defined in a way to avoid overflow compiler warning on 32-bit config. [1] https://download.vusec.net/papers/slam_sp24.pdf Signed-off-by: Kirill A. Shutemov Cc: Pawan Gupta Reviewed-by: Sohil Mehta --- arch/x86/Kconfig | 1 - arch/x86/kernel/cpu/common.c | 5 +---- 2 files changed, 1 insertion(+), 5 deletions(-) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 71019b3b54ea..2b48e916b754 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -2181,7 +2181,6 @@ config RANDOMIZE_MEMORY_PHYSICAL_PADDING config ADDRESS_MASKING bool "Linear Address Masking support" depends on X86_64 - depends on COMPILE_TEST || !CPU_MITIGATIONS # wait for LASS help Linear Address Masking (LAM) modifies the checking that is applied to 64-bit linear addresses, allowing software to use of the diff --git a/arch/x86/kernel/cpu/common.c b/arch/x86/kernel/cpu/common.c index 97a228f917a9..6f2ae9e702bc 100644 --- a/arch/x86/kernel/cpu/common.c +++ b/arch/x86/kernel/cpu/common.c @@ -2558,11 +2558,8 @@ void __init arch_cpu_finalize_init(void) if (IS_ENABLED(CONFIG_X86_64)) { unsigned long USER_PTR_MAX =3D TASK_SIZE_MAX; =20 - /* - * Enable this when LAM is gated on LASS support if (cpu_feature_enabled(X86_FEATURE_LAM)) - USER_PTR_MAX =3D (1ul << 63) - PAGE_SIZE; - */ + USER_PTR_MAX =3D (-1UL >> 1) & PAGE_MASK; runtime_const_init(ptr, USER_PTR_MAX); =20 /* --=20 2.47.2