From nobody Wed Dec 17 06:08:50 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED7DBC433FE for ; Thu, 6 Oct 2022 16:03:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229714AbiJFQD4 (ORCPT ); Thu, 6 Oct 2022 12:03:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43442 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232000AbiJFQDr (ORCPT ); Thu, 6 Oct 2022 12:03:47 -0400 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3D9CFB1BB8 for ; Thu, 6 Oct 2022 09:03:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1665072226; x=1696608226; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=UDzg+9uANYdDBV+Ra3lQotFnvAmXOEkeoHs76rgla5s=; b=i9mtznLfYzD/k+LIldIHb7B3DihI2xGbwXQnlyRDCv5od7RrW+A+LjVs Z1n9wNaR4fU07fyiGbNfg9I6Ng46kMAkap2brJ47vcfJrXQs5mErEjEIO EIpl4/b9BYdlzmYML6fqI0A+hk/FXuav8OZm2LVbTFkL9SC3TfAzEmqGv QMcXL9TF+4UjL3C7xN9R0JUSgp4DpVnDEd07SLQZxV3MIzzXgYQm0Aylu 8O/FlVWLdhR/o0SMb7Q/41wWNyB+1GMMbITtqR55uy4BUFl3B4vB+SC5n loOHwAD1+AqX0oNMFrUiIZXNoH+IJccbsLa7rSSfbfB98YsbsyshhBHYs g==; X-IronPort-AV: E=McAfee;i="6500,9779,10492"; a="304480698" X-IronPort-AV: E=Sophos;i="5.95,164,1661842800"; d="scan'208";a="304480698" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Oct 2022 09:02:36 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10491"; a="687486224" X-IronPort-AV: E=Sophos;i="5.95,164,1661842800"; d="scan'208";a="687486224" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga008.fm.intel.com with ESMTP; 06 Oct 2022 09:02:35 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com Subject: [PATCH 1/6] x86/cpufeature: add cpu feature bit for LKGS Date: Thu, 6 Oct 2022 08:40:36 -0700 Message-Id: <20221006154041.13001-2-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221006154041.13001-1-xin3.li@intel.com> References: <20221006154041.13001-1-xin3.li@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: "H. Peter Anvin (Intel)" Add the CPU feature bit for LKGS (Load "Kernel" GS). LKGS instruction is introduced with Intel FRED (flexible return and event delivery) specificaton https://cdrdv2.intel.com/v1/dl/getContent/678938. LKGS behaves like the MOV to GS instruction except that it loads the base address into the IA32_KERNEL_GS_BASE MSR instead of the GS segment=E2=80=99s descriptor cache, which is exactly what Linux kernel does to load a user level GS base. Thus, with LKGS, there is no need to SWAPGS away from the kernel GS base. Signed-off-by: H. Peter Anvin (Intel) Signed-off-by: Xin Li --- arch/x86/include/asm/cpufeatures.h | 1 + tools/arch/x86/include/asm/cpufeatures.h | 1 + 2 files changed, 2 insertions(+) diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpuf= eatures.h index ef4775c6db01..459fb0c21dd4 100644 --- a/arch/x86/include/asm/cpufeatures.h +++ b/arch/x86/include/asm/cpufeatures.h @@ -308,6 +308,7 @@ /* Intel-defined CPU features, CPUID level 0x00000007:1 (EAX), word 12 */ #define X86_FEATURE_AVX_VNNI (12*32+ 4) /* AVX VNNI instructions */ #define X86_FEATURE_AVX512_BF16 (12*32+ 5) /* AVX512 BFLOAT16 instruction= s */ +#define X86_FEATURE_LKGS (12*32+ 18) /* Load "kernel" (userspace) gs */ =20 /* AMD-defined CPU features, CPUID level 0x80000008 (EBX), word 13 */ #define X86_FEATURE_CLZERO (13*32+ 0) /* CLZERO instruction */ diff --git a/tools/arch/x86/include/asm/cpufeatures.h b/tools/arch/x86/incl= ude/asm/cpufeatures.h index ef4775c6db01..459fb0c21dd4 100644 --- a/tools/arch/x86/include/asm/cpufeatures.h +++ b/tools/arch/x86/include/asm/cpufeatures.h @@ -308,6 +308,7 @@ /* Intel-defined CPU features, CPUID level 0x00000007:1 (EAX), word 12 */ #define X86_FEATURE_AVX_VNNI (12*32+ 4) /* AVX VNNI instructions */ #define X86_FEATURE_AVX512_BF16 (12*32+ 5) /* AVX512 BFLOAT16 instruction= s */ +#define X86_FEATURE_LKGS (12*32+ 18) /* Load "kernel" (userspace) gs */ =20 /* AMD-defined CPU features, CPUID level 0x80000008 (EBX), word 13 */ #define X86_FEATURE_CLZERO (13*32+ 0) /* CLZERO instruction */ --=20 2.34.1 From nobody Wed Dec 17 06:08:50 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA601C433F5 for ; Thu, 6 Oct 2022 16:03:55 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232020AbiJFQDy (ORCPT ); Thu, 6 Oct 2022 12:03:54 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43364 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232001AbiJFQDr (ORCPT ); Thu, 6 Oct 2022 12:03:47 -0400 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id AFF23ABD5D for ; Thu, 6 Oct 2022 09:03:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1665072226; x=1696608226; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=LPkLGLb6ct3NaqkfE/fcmAkN388PX6Fn1d7MFn2Rg6I=; b=b+BeOhmOTpR+IzH9SdsPYxkCa+McXP5HqwGj3fayUSzWbV/gwNRR+nUs FpQKlGVeJ7d020jFwCVbztyE1H4bDjOzbaeaShDDVCtLh89th1d7acARg PzMyS7DcHF2UyWKe2ZTuY6omC6pOoj9JmVBKYhQhIKKDrgerxkR0NVx6a p2B1zBjOtaCqKG6jdbtQFXRdpoCE1J8Xp6uOIG2GBWd4is4YgXn013ifh LGqRIEPgk/kWygf5DvCPWoYOo/dfT637GQx9sIK8oPE9Sin7KadELohhX 1evXnSHtwazhM0TdpK9uxgxzxvQ2mzzdbstY3WlPc23OElegyb+R71+gv A==; X-IronPort-AV: E=McAfee;i="6500,9779,10492"; a="304480701" X-IronPort-AV: E=Sophos;i="5.95,164,1661842800"; d="scan'208";a="304480701" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Oct 2022 09:02:36 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10491"; a="687486225" X-IronPort-AV: E=Sophos;i="5.95,164,1661842800"; d="scan'208";a="687486225" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga008.fm.intel.com with ESMTP; 06 Oct 2022 09:02:35 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com Subject: [PATCH 2/6] x86/opcode: add LKGS instruction to x86-opcode-map Date: Thu, 6 Oct 2022 08:40:37 -0700 Message-Id: <20221006154041.13001-3-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221006154041.13001-1-xin3.li@intel.com> References: <20221006154041.13001-1-xin3.li@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "H. Peter Anvin (Intel)" Add the instruction opcode used by LKGS. Opcode number is per public FRED draft spec v3.0 https://cdrdv2.intel.com/v1/dl/getContent/678938. Signed-off-by: H. Peter Anvin (Intel) Signed-off-by: Xin Li --- arch/x86/lib/x86-opcode-map.txt | 1 + tools/arch/x86/lib/x86-opcode-map.txt | 1 + 2 files changed, 2 insertions(+) diff --git a/arch/x86/lib/x86-opcode-map.txt b/arch/x86/lib/x86-opcode-map.= txt index d12d1358f96d..5168ee0360b2 100644 --- a/arch/x86/lib/x86-opcode-map.txt +++ b/arch/x86/lib/x86-opcode-map.txt @@ -1047,6 +1047,7 @@ GrpTable: Grp6 3: LTR Ew 4: VERR Ew 5: VERW Ew +6: LKGS Ew (F2) EndTable =20 GrpTable: Grp7 diff --git a/tools/arch/x86/lib/x86-opcode-map.txt b/tools/arch/x86/lib/x86= -opcode-map.txt index d12d1358f96d..5168ee0360b2 100644 --- a/tools/arch/x86/lib/x86-opcode-map.txt +++ b/tools/arch/x86/lib/x86-opcode-map.txt @@ -1047,6 +1047,7 @@ GrpTable: Grp6 3: LTR Ew 4: VERR Ew 5: VERW Ew +6: LKGS Ew (F2) EndTable =20 GrpTable: Grp7 --=20 2.34.1 From nobody Wed Dec 17 06:08:50 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9EDC3C433FE for ; Thu, 6 Oct 2022 16:04:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232011AbiJFQEA (ORCPT ); Thu, 6 Oct 2022 12:04:00 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43362 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232005AbiJFQDr (ORCPT ); Thu, 6 Oct 2022 12:03:47 -0400 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 12895B2749 for ; Thu, 6 Oct 2022 09:03:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1665072227; x=1696608227; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=1YbzoiUXSvQWzBUraiDzjfmE9NvY0qFctz1F+7sN7Sw=; b=mdItGj5/ZlUGLztKQ4SN94mHK8pnNDvRGRwHfDW4BlB5uh2Y+2+nU0FL 8VcHdRDuC+tKqvx6FDg2eF1eC7oHP97s/GyaFAVthSc1J2BLPoVtfpJC0 JNfdzv/jMTUKGYSrCJCMIIGvJkjC8rjs0NaG1bJYtVwBJ8Mo8JHO13z9c 4XuT096EDmbsJp+7aboFJZlJTASlcW6WmIJT7oYS0d3VzNzFfIuZ9sdk3 nij9g9RztbJpHd5ufOY0kJZoQ5B96KMQUnyLmoFx9v1jUQd41BGWpYpKu ecUvmZg45mkJNPjNDZ54b48Awv9t0aNFYMcyAW1CeODXH/naj4CpU/g6Q w==; X-IronPort-AV: E=McAfee;i="6500,9779,10492"; a="304480708" X-IronPort-AV: E=Sophos;i="5.95,164,1661842800"; d="scan'208";a="304480708" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Oct 2022 09:02:36 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10491"; a="687486227" X-IronPort-AV: E=Sophos;i="5.95,164,1661842800"; d="scan'208";a="687486227" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga008.fm.intel.com with ESMTP; 06 Oct 2022 09:02:35 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com Subject: [PATCH 3/6] x86/gsseg: make asm_load_gs_index() take an u16 Date: Thu, 6 Oct 2022 08:40:38 -0700 Message-Id: <20221006154041.13001-4-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221006154041.13001-1-xin3.li@intel.com> References: <20221006154041.13001-1-xin3.li@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "H. Peter Anvin (Intel)" Let gcc know that only the low 16 bits of load_gs_index() argument actually matter. It might allow it to create slightly better code. However, do not propagate this into the prototypes of functions that end up being paravirtualized, to avoid unnecessary changes. Signed-off-by: H. Peter Anvin (Intel) Signed-off-by: Xin Li --- arch/x86/entry/entry_64.S | 2 +- arch/x86/include/asm/special_insns.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index 9953d966d124..e0c48998d2fb 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -779,7 +779,7 @@ _ASM_NOKPROBE(common_interrupt_return) =20 /* * Reload gs selector with exception handling - * edi: new selector + * di: new selector * * Is in entry.text as it shouldn't be instrumented. */ diff --git a/arch/x86/include/asm/special_insns.h b/arch/x86/include/asm/sp= ecial_insns.h index 35f709f619fb..a71d0e8d4684 100644 --- a/arch/x86/include/asm/special_insns.h +++ b/arch/x86/include/asm/special_insns.h @@ -120,7 +120,7 @@ static inline void native_wbinvd(void) asm volatile("wbinvd": : :"memory"); } =20 -extern asmlinkage void asm_load_gs_index(unsigned int selector); +extern asmlinkage void asm_load_gs_index(u16 selector); =20 static inline void native_load_gs_index(unsigned int selector) { --=20 2.34.1 From nobody Wed Dec 17 06:08:50 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 27CD0C433F5 for ; Thu, 6 Oct 2022 16:04:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232000AbiJFQEF (ORCPT ); Thu, 6 Oct 2022 12:04:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43346 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231792AbiJFQDr (ORCPT ); Thu, 6 Oct 2022 12:03:47 -0400 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 38009B2745 for ; Thu, 6 Oct 2022 09:03:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1665072227; x=1696608227; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=UaC8ceQnideTemOSR0AnD9gGv296yv6KtEQKYgHeIg4=; b=jxKw+8UVS/xHiX5TS3LjHyJCmJTJs9BXkzfcPSg0mOfbZazHa1KbiOT1 hCrFLfpv3R+8jlxSGN11DE5wM+AYbSti8w555KqJp7IZ1PckRNuUArTz6 w0F9JSk98HwVkqPE5ch0MJJ8cj6aRPbtEu4uiJy95dm6ENsVamTahiSPA AfJwjjsXUJE2VRg+gHjTajBVOpn6DzaF93CbvertipTVbCw2Ac0pWVtH4 BqYis5aZBtze0i2mx2MGpm5EQWtqC63ogvJaAcosqSNwww4mpBXzBNcwI kqYLbyMWIGFSrqt4xrcrRmofaqNCHvAyy+0ySq05pHCKLtrySdie5Br86 A==; X-IronPort-AV: E=McAfee;i="6500,9779,10492"; a="304480712" X-IronPort-AV: E=Sophos;i="5.95,164,1661842800"; d="scan'208";a="304480712" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Oct 2022 09:02:36 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10491"; a="687486230" X-IronPort-AV: E=Sophos;i="5.95,164,1661842800"; d="scan'208";a="687486230" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga008.fm.intel.com with ESMTP; 06 Oct 2022 09:02:36 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com Subject: [PATCH 4/6] x86/gsseg: move local_irq_save/restore() into asm_load_gs_index() Date: Thu, 6 Oct 2022 08:40:39 -0700 Message-Id: <20221006154041.13001-5-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221006154041.13001-1-xin3.li@intel.com> References: <20221006154041.13001-1-xin3.li@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "H. Peter Anvin (Intel)" The need to disable/enable interrupts around asm_load_gs_index() is a consequence of the implementation. Prepare for using the LKGS instruction, which is locally atomic and therefore doesn't need interrupts disabled. Signed-off-by: H. Peter Anvin (Intel) Signed-off-by: Xin Li --- arch/x86/entry/entry_64.S | 26 +++++++++++++++++++++----- arch/x86/include/asm/special_insns.h | 4 ---- 2 files changed, 21 insertions(+), 9 deletions(-) diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index e0c48998d2fb..cc6ba6672418 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -778,19 +778,35 @@ SYM_CODE_END(common_interrupt_return) _ASM_NOKPROBE(common_interrupt_return) =20 /* - * Reload gs selector with exception handling + * Reload gs selector with exception handling. This is used only on + * native, so using swapgs, pushf, popf, cli, sti, ... directly is fine. + * * di: new selector + * rax: scratch register * * Is in entry.text as it shouldn't be instrumented. + * + * Note: popf is slow, so use pushf to read IF and then execute cli/sti + * if necessary. */ SYM_FUNC_START(asm_load_gs_index) FRAME_BEGIN + pushf + pop %rax + andl $X86_EFLAGS_IF, %eax /* Interrupts enabled? */ + jz 1f + cli +1: swapgs .Lgs_change: ANNOTATE_NOENDBR // error_entry movl %edi, %gs 2: ALTERNATIVE "", "mfence", X86_BUG_SWAPGS_FENCE swapgs + testl %eax, %eax + jz 3f + sti +3: FRAME_END RET =20 @@ -799,12 +815,12 @@ SYM_FUNC_START(asm_load_gs_index) swapgs /* switch back to user gs */ .macro ZAP_GS /* This can't be a string because the preprocessor needs to see it. */ - movl $__USER_DS, %eax - movl %eax, %gs + movl $__USER_DS, %edi + movl %edi, %gs .endm ALTERNATIVE "", "ZAP_GS", X86_BUG_NULL_SEG - xorl %eax, %eax - movl %eax, %gs + xorl %edi, %edi + movl %edi, %gs jmp 2b =20 _ASM_EXTABLE(.Lgs_change, .Lbad_gs) diff --git a/arch/x86/include/asm/special_insns.h b/arch/x86/include/asm/sp= ecial_insns.h index a71d0e8d4684..6de00dec6564 100644 --- a/arch/x86/include/asm/special_insns.h +++ b/arch/x86/include/asm/special_insns.h @@ -124,11 +124,7 @@ extern asmlinkage void asm_load_gs_index(u16 selector); =20 static inline void native_load_gs_index(unsigned int selector) { - unsigned long flags; - - local_irq_save(flags); asm_load_gs_index(selector); - local_irq_restore(flags); } =20 static inline unsigned long __read_cr4(void) --=20 2.34.1 From nobody Wed Dec 17 06:08:50 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E7C6BC433F5 for ; Thu, 6 Oct 2022 16:04:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232010AbiJFQEI (ORCPT ); Thu, 6 Oct 2022 12:04:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43338 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232004AbiJFQDs (ORCPT ); Thu, 6 Oct 2022 12:03:48 -0400 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 85EC5B14E9 for ; Thu, 6 Oct 2022 09:03:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1665072227; x=1696608227; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=juR16UITaT3MxswvlHsDaympRivW8hA0yaAGOHpwels=; b=Nn1+qBbR+WZ13vZdNujYMPhoypfyvc3cS4U7bTv5MP241b+PWmRbmyox pFna9wrayJwYYBNVNKQJBl2kgCk5BXEl2AhlP5LQwFvdyLrjIKhZf156X OwNUnDXvDlQybisM5o3q/iXv05X8nzc6AjXYfaJ2lkrGpp31FcuJkCFo1 j5oA89vONeXKQhutC/2PZ+lHCMVzG8feavQg9UnIOt7D1VO0dsPJjLv2q EKPfhbECNb0J1uKIzqjJumIrm06QIyxyR+AX9DTme6LbZ81ALYzu7IQro NvfYgdYq7+cya0QLuBxyWK4UEtSh8h4m4NLNzDLnLqk9NSMruoWMgxD0P Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10492"; a="304480716" X-IronPort-AV: E=Sophos;i="5.95,164,1661842800"; d="scan'208";a="304480716" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Oct 2022 09:02:36 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10491"; a="687486234" X-IronPort-AV: E=Sophos;i="5.95,164,1661842800"; d="scan'208";a="687486234" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga008.fm.intel.com with ESMTP; 06 Oct 2022 09:02:36 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com Subject: [PATCH 5/6] x86/gsseg: move load_gs_index() to its own header file Date: Thu, 6 Oct 2022 08:40:40 -0700 Message-Id: <20221006154041.13001-6-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221006154041.13001-1-xin3.li@intel.com> References: <20221006154041.13001-1-xin3.li@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "H. Peter Anvin (Intel)" depends on , so in order to be able to use alternatives in native_load_gs_index(), factor it out into a separate header file. Signed-off-by: H. Peter Anvin (Intel) Signed-off-by: Xin Li --- arch/x86/ia32/ia32_signal.c | 1 + arch/x86/include/asm/gsseg.h | 32 ++++++++++++++++++++++++++++ arch/x86/include/asm/mmu_context.h | 1 + arch/x86/include/asm/special_insns.h | 17 --------------- arch/x86/kernel/paravirt.c | 1 + arch/x86/kernel/tls.c | 1 + 6 files changed, 36 insertions(+), 17 deletions(-) create mode 100644 arch/x86/include/asm/gsseg.h diff --git a/arch/x86/ia32/ia32_signal.c b/arch/x86/ia32/ia32_signal.c index c9c3859322fa..14c739303099 100644 --- a/arch/x86/ia32/ia32_signal.c +++ b/arch/x86/ia32/ia32_signal.c @@ -34,6 +34,7 @@ #include #include #include +#include =20 static inline void reload_segments(struct sigcontext_32 *sc) { diff --git a/arch/x86/include/asm/gsseg.h b/arch/x86/include/asm/gsseg.h new file mode 100644 index 000000000000..5e3b56a17098 --- /dev/null +++ b/arch/x86/include/asm/gsseg.h @@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +#ifndef _ASM_X86_GSSEG_H +#define _ASM_X86_GSSEG_H + +#include +#include + +#ifdef CONFIG_X86_64 + +extern asmlinkage void asm_load_gs_index(u16 selector); + +static inline void native_load_gs_index(unsigned int selector) +{ + asm_load_gs_index(selector); +} + +#endif /* CONFIG_X86_64 */ + +#ifndef CONFIG_PARAVIRT_XXL + +static inline void load_gs_index(unsigned int selector) +{ +#ifdef CONFIG_X86_64 + native_load_gs_index(selector); +#else + loadsegment(gs, selector); +#endif +} + +#endif /* CONFIG_PARAVIRT_XXL */ + +#endif /* _ASM_X86_GSSEG_H */ diff --git a/arch/x86/include/asm/mmu_context.h b/arch/x86/include/asm/mmu_= context.h index b8d40ddeab00..e01aa74a6de7 100644 --- a/arch/x86/include/asm/mmu_context.h +++ b/arch/x86/include/asm/mmu_context.h @@ -12,6 +12,7 @@ #include #include #include +#include =20 extern atomic64_t last_mm_ctx_id; =20 diff --git a/arch/x86/include/asm/special_insns.h b/arch/x86/include/asm/sp= ecial_insns.h index 6de00dec6564..cfd9499b617c 100644 --- a/arch/x86/include/asm/special_insns.h +++ b/arch/x86/include/asm/special_insns.h @@ -120,13 +120,6 @@ static inline void native_wbinvd(void) asm volatile("wbinvd": : :"memory"); } =20 -extern asmlinkage void asm_load_gs_index(u16 selector); - -static inline void native_load_gs_index(unsigned int selector) -{ - asm_load_gs_index(selector); -} - static inline unsigned long __read_cr4(void) { return native_read_cr4(); @@ -180,16 +173,6 @@ static inline void wbinvd(void) native_wbinvd(); } =20 - -static inline void load_gs_index(unsigned int selector) -{ -#ifdef CONFIG_X86_64 - native_load_gs_index(selector); -#else - loadsegment(gs, selector); -#endif -} - #endif /* CONFIG_PARAVIRT_XXL */ =20 static inline void clflush(volatile void *__p) diff --git a/arch/x86/kernel/paravirt.c b/arch/x86/kernel/paravirt.c index 7ca2d46c08cc..00f6a92551d2 100644 --- a/arch/x86/kernel/paravirt.c +++ b/arch/x86/kernel/paravirt.c @@ -32,6 +32,7 @@ #include #include #include +#include =20 /* * nop stub, which must not clobber anything *including the stack* to diff --git a/arch/x86/kernel/tls.c b/arch/x86/kernel/tls.c index 3c883e064242..3ffbab0081f4 100644 --- a/arch/x86/kernel/tls.c +++ b/arch/x86/kernel/tls.c @@ -12,6 +12,7 @@ #include #include #include +#include =20 #include "tls.h" =20 --=20 2.34.1 From nobody Wed Dec 17 06:08:50 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8951BC433FE for ; Thu, 6 Oct 2022 16:03:15 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231982AbiJFQDN (ORCPT ); Thu, 6 Oct 2022 12:03:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42806 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231968AbiJFQDI (ORCPT ); Thu, 6 Oct 2022 12:03:08 -0400 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9AFDFAE228 for ; Thu, 6 Oct 2022 09:03:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1665072187; x=1696608187; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=TYDLP/WLagDk6RRKpxF1aIZ7QTqIfIktvLR1UyU4GrE=; b=g3QiBSh2sWgslY+YgKSUpLOo+GKlO1AsiQRinu6nZGFUpr0jMJHgtmvX uVk2hAPAuGTSEkzmlfMrKd3TV8S1IjRzr3Rd5Ntt6t345UBgFGSr+n9pb E50aY82ocCXPlWcc7Vz2u2vE9B2ezTCM6qbVgnxyMPYMGQ4Ng27T4+Bg8 lXt/naDrQuKBziNjFNAwfn02/5jD+LKUbYY40BU6MSC/WhqcOrivNApg/ YhXWkX69oCZcHr4m2BOU82O3ESiRKVtUq3XObxrnAIFivXxr8Msuco6zQ Nvnagk82J1gg3LWrVbX0DV1OPor8k6PNayyCA3H1n2Tn8OlmINSppxjnh w==; X-IronPort-AV: E=McAfee;i="6500,9779,10491"; a="286709428" X-IronPort-AV: E=Sophos;i="5.95,164,1661842800"; d="scan'208";a="286709428" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 06 Oct 2022 09:02:37 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10491"; a="687486237" X-IronPort-AV: E=Sophos;i="5.95,164,1661842800"; d="scan'208";a="687486237" Received: from unknown (HELO fred..) ([172.25.112.68]) by fmsmga008.fm.intel.com with ESMTP; 06 Oct 2022 09:02:36 -0700 From: Xin Li To: linux-kernel@vger.kernel.org, x86@kernel.org Cc: tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com Subject: [PATCH 6/6] x86/gsseg: use the LKGS instruction if available for load_gs_index() Date: Thu, 6 Oct 2022 08:40:41 -0700 Message-Id: <20221006154041.13001-7-xin3.li@intel.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221006154041.13001-1-xin3.li@intel.com> References: <20221006154041.13001-1-xin3.li@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" From: "H. Peter Anvin (Intel)" The LKGS instruction atomically loads a segment descriptor into the %gs descriptor registers, *except* that %gs.base is unchanged, and the base is instead loaded into MSR_IA32_KERNEL_GS_BASE, which is exactly what we want this function to do. Signed-off-by: H. Peter Anvin (Intel) Signed-off-by: Xin Li --- arch/x86/include/asm/gsseg.h | 28 +++++++++++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/gsseg.h b/arch/x86/include/asm/gsseg.h index 5e3b56a17098..b8a6a98d88b8 100644 --- a/arch/x86/include/asm/gsseg.h +++ b/arch/x86/include/asm/gsseg.h @@ -3,15 +3,41 @@ #define _ASM_X86_GSSEG_H =20 #include + +#include +#include +#include #include +#include =20 #ifdef CONFIG_X86_64 =20 extern asmlinkage void asm_load_gs_index(u16 selector); =20 +#define LKGS_DI _ASM_BYTES(0xf2,0x0f,0x00,0xf7) + static inline void native_load_gs_index(unsigned int selector) { - asm_load_gs_index(selector); + u16 sel =3D selector; + + /* + * Note: the fixup is used for the LKGS instruction, but + * it needs to be attached to the primary instruction sequence + * as it isn't something that gets patched. + * + * %rax is provided to the assembly routine as a scratch + * register. + */ + alternative_io("1: call asm_load_gs_index\n" + ".pushsection \".fixup\",\"ax\"\n" + "2: xorl %k[sel], %k[sel]\n" + " jmp 1b\n" + ".popsection\n" + _ASM_EXTABLE(1b, 2b), + _ASM_BYTES(0x3e) LKGS_DI, + X86_FEATURE_LKGS, + ASM_OUTPUT2([sel] "+D" (sel), ASM_CALL_CONSTRAINT), + ASM_NO_INPUT_CLOBBER(_ASM_AX)); } =20 #endif /* CONFIG_X86_64 */ --=20 2.34.1