From nobody Wed Dec 17 10:19:57 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4146EC77B73 for ; Thu, 20 Apr 2023 14:16:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232000AbjDTOQX (ORCPT ); Thu, 20 Apr 2023 10:16:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39534 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231928AbjDTOQQ (ORCPT ); Thu, 20 Apr 2023 10:16:16 -0400 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 070D64217; Thu, 20 Apr 2023 07:16:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1682000175; x=1713536175; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=X5UoG8xozByeuHc7S3vpOJzvxFI1farY3KuVu17e7rI=; b=i1G+nSwSA3A/rIQhPBT0v/OzlJBcAwumWOj9gIYVd3wNc8/8p6TRMtWx iev1XfOCWlYOvonefpGvtKruF9q1IYjr1hWTMtugs3P0gH/nrCrRfXih/ 9bS38O/3ySWmIoN396nteyaq3sTvxapc4xrPPL2YsxSfAaXksUC70+Lw+ Fn5GoqtkP2ZDeoLkbY6/9Qs+d2GVZZIKxRjf+t92nNziyU5DVeqM4+FzX GWYgJ48RCWH6nY5xCFN07D1U2cpORw5kAWZrekH2JFKYkZdQgT97m9/Kn V2qZiD9nJvLH8yPqfZ4O1Z9uCfwIUBYMAOR7ZUlFYCSZtLMee2Q+cd3cs A==; X-IronPort-AV: E=McAfee;i="6600,9927,10686"; a="343217782" X-IronPort-AV: E=Sophos;i="5.99,212,1677571200"; d="scan'208";a="343217782" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Apr 2023 07:16:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10686"; a="816028815" X-IronPort-AV: E=Sophos;i="5.99,212,1677571200"; d="scan'208";a="816028815" Received: from arthur-vostro-3668.sh.intel.com ([10.238.200.53]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Apr 2023 07:16:00 -0700 From: Zeng Guang To: Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , H Peter Anvin , kvm@vger.kernel.org Cc: x86@kernel.org, linux-kernel@vger.kernel.org, Gao Chao , Zeng Guang Subject: [PATCH 1/6] KVM: x86: Virtualize CR4.LASS Date: Thu, 20 Apr 2023 21:37:19 +0800 Message-Id: <20230420133724.11398-2-guang.zeng@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230420133724.11398-1-guang.zeng@intel.com> References: <20230420133724.11398-1-guang.zeng@intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Virtualize CR4.LASS[bit 27] under KVM control instead of being guest-owned as CR4.LASS generally set once for each vCPU at boot time and won't be toggled at runtime. Besides, only if VM has LASS capability enumerated with CPUID.(EAX=3D07H.ECX=3D1):EAX.LASS[bit 6], KVM allows guest software to be = able to set CR4.LASS. By design CR4.LASS can be manipulated by nested guest as well. Notes: Setting CR4.LASS to 1 enable LASS in IA-32e mode. It doesn't take effect in legacy mode even if CR4.LASS is set. Signed-off-by: Zeng Guang Reviewed-by: Binbin Wu --- arch/x86/include/asm/kvm_host.h | 2 +- arch/x86/kvm/vmx/vmx.c | 3 +++ arch/x86/kvm/x86.h | 2 ++ 3 files changed, 6 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 6aaae18f1854..8ff89a52ef66 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -125,7 +125,7 @@ | X86_CR4_PGE | X86_CR4_PCE | X86_CR4_OSFXSR | X86_CR4_PCIDE \ | X86_CR4_OSXSAVE | X86_CR4_SMEP | X86_CR4_FSGSBASE \ | X86_CR4_OSXMMEXCPT | X86_CR4_LA57 | X86_CR4_VMXE \ - | X86_CR4_SMAP | X86_CR4_PKE | X86_CR4_UMIP)) + | X86_CR4_SMAP | X86_CR4_PKE | X86_CR4_UMIP | X86_CR4_LASS)) =20 #define CR8_RESERVED_BITS (~(unsigned long)X86_CR8_TPR) =20 diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index 43ff3276918b..c923d7599d71 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -7569,6 +7569,9 @@ static void nested_vmx_cr_fixed1_bits_update(struct k= vm_vcpu *vcpu) cr4_fixed1_update(X86_CR4_UMIP, ecx, feature_bit(UMIP)); cr4_fixed1_update(X86_CR4_LA57, ecx, feature_bit(LA57)); =20 + entry =3D kvm_find_cpuid_entry_index(vcpu, 0x7, 1); + cr4_fixed1_update(X86_CR4_LASS, eax, feature_bit(LASS)); + #undef cr4_fixed1_update } =20 diff --git a/arch/x86/kvm/x86.h b/arch/x86/kvm/x86.h index 7c8a30d44c29..218f4c73789a 100644 --- a/arch/x86/kvm/x86.h +++ b/arch/x86/kvm/x86.h @@ -475,6 +475,8 @@ bool kvm_msr_allowed(struct kvm_vcpu *vcpu, u32 index, = u32 type); __reserved_bits |=3D X86_CR4_VMXE; \ if (!__cpu_has(__c, X86_FEATURE_PCID)) \ __reserved_bits |=3D X86_CR4_PCIDE; \ + if (!__cpu_has(__c, X86_FEATURE_LASS)) \ + __reserved_bits |=3D X86_CR4_LASS; \ __reserved_bits; \ }) =20 --=20 2.27.0 From nobody Wed Dec 17 10:19:57 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 20C9CC77B7C for ; Thu, 20 Apr 2023 14:16:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232031AbjDTOQ2 (ORCPT ); Thu, 20 Apr 2023 10:16:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39524 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231933AbjDTOQR (ORCPT ); Thu, 20 Apr 2023 10:16:17 -0400 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1349A449D; Thu, 20 Apr 2023 07:16:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1682000176; x=1713536176; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=8vcqBoPnJSsP+PR1vJco3USQMMRGQCpM1hb/P9YwlSA=; b=ADzl7iY/e2QalQndR+u7ksiViYrax1kHYFcrYyBakxt5E6xxOO4UIoDQ ZpCixLwPLvwrQZHj8XVsK4asqZR9riZUcbU5eykkxDn3Ibcupv98kaHkr iwAdL3FyGrKo/KN+vvs4IRvIVLSpG9QBduMS/ff/renm/G4aQ/wO1g0+L VFtvtA+jbmhFR9c6OJPO6eH2NumFywRit6twF73GosmwB1dFws3QMQFxQ GEKmyEWi2aSi8qmgP+qYAXtElo9c3YI1YC7ovSRruAKt312qMuxSXqlRD K3r2o7YAkq9jj3lB19EuLLsayhC2WSodf0hdbTiYbnSrJxrxwz6wfccDW Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10686"; a="343217803" X-IronPort-AV: E=Sophos;i="5.99,212,1677571200"; d="scan'208";a="343217803" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Apr 2023 07:16:06 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10686"; a="816028837" X-IronPort-AV: E=Sophos;i="5.99,212,1677571200"; d="scan'208";a="816028837" Received: from arthur-vostro-3668.sh.intel.com ([10.238.200.53]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Apr 2023 07:16:03 -0700 From: Zeng Guang To: Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , H Peter Anvin , kvm@vger.kernel.org Cc: x86@kernel.org, linux-kernel@vger.kernel.org, Gao Chao , Zeng Guang Subject: [PATCH 2/6] KVM: VMX: Add new ops in kvm_x86_ops for LASS violation check Date: Thu, 20 Apr 2023 21:37:20 +0800 Message-Id: <20230420133724.11398-3-guang.zeng@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230420133724.11398-1-guang.zeng@intel.com> References: <20230420133724.11398-1-guang.zeng@intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Intel introduce LASS (Linear Address Separation) feature providing an independent mechanism to achieve the mode-based protection. LASS partitions 64-bit linear address space into two halves, user-mode address (LA[bit 63]=3D0) and supervisor-mode address (LA[bit 63]=3D1). It stops any code execution or data access 1. from user mode to supervisor-mode address space 2. from supervisor mode to user-mode address space and generates LASS violation fault accordingly. A supervisor mode data access causes a LASS violation only if supervisor mode access protection is enabled (CR4.SMAP =3D 1) and either RFLAGS.AC =3D= 0 or the access implicitly accesses a system data structure. Following are the rule of LASS violation check on the linear address(LA). User access to supervisor-mode address space: LA[bit 63] && (CPL =3D=3D 3) Supervisor access to user-mode address space: Instruction fetch: !LA[bit 63] && (CPL < 3) Data access: !LA[bit 63] && (CR4.SMAP=3D=3D1) && ((RFLAGS.AC =3D=3D 0 && CPL < 3) || Implicit supervisor access) Add new ops in kvm_x86_ops to do LASS violation check. Signed-off-by: Zeng Guang --- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 5 +++ arch/x86/kvm/vmx/vmx.c | 55 ++++++++++++++++++++++++++++++ arch/x86/kvm/vmx/vmx.h | 2 ++ 4 files changed, 63 insertions(+) diff --git a/arch/x86/include/asm/kvm-x86-ops.h b/arch/x86/include/asm/kvm-= x86-ops.h index abccd51dcfca..f76c07f2674b 100644 --- a/arch/x86/include/asm/kvm-x86-ops.h +++ b/arch/x86/include/asm/kvm-x86-ops.h @@ -131,6 +131,7 @@ KVM_X86_OP(msr_filter_changed) KVM_X86_OP(complete_emulated_msr) KVM_X86_OP(vcpu_deliver_sipi_vector) KVM_X86_OP_OPTIONAL_RET0(vcpu_get_apicv_inhibit_reasons); +KVM_X86_OP_OPTIONAL_RET0(check_lass); =20 #undef KVM_X86_OP #undef KVM_X86_OP_OPTIONAL diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_hos= t.h index 8ff89a52ef66..31fb8699a1ff 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -69,6 +69,9 @@ #define KVM_X86_NOTIFY_VMEXIT_VALID_BITS (KVM_X86_NOTIFY_VMEXIT_ENABLED | \ KVM_X86_NOTIFY_VMEXIT_USER) =20 +/* x86-specific emulation flags */ +#define KVM_X86_EMULFLAG_SKIP_LASS _BITULL(1) + /* x86-specific vcpu->requests bit members */ #define KVM_REQ_MIGRATE_TIMER KVM_ARCH_REQ(0) #define KVM_REQ_REPORT_TPR_ACCESS KVM_ARCH_REQ(1) @@ -1706,6 +1709,8 @@ struct kvm_x86_ops { * Returns vCPU specific APICv inhibit reasons */ unsigned long (*vcpu_get_apicv_inhibit_reasons)(struct kvm_vcpu *vcpu); + + bool (*check_lass)(struct kvm_vcpu *vcpu, u64 access, u64 la, u64 flags); }; =20 struct kvm_x86_nested_ops { diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index c923d7599d71..581327ede66a 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -8070,6 +8070,59 @@ static void vmx_vm_destroy(struct kvm *kvm) free_pages((unsigned long)kvm_vmx->pid_table, vmx_get_pid_table_order(kvm= )); } =20 +/* + * Determine whether an access to the linear address causes a LASS violati= on. + * LASS protection is only effective in long mode. As a prerequisite, call= er + * should make sure VM running in long mode and invoke this api to do LASS + * violation check. + */ +bool __vmx_check_lass(struct kvm_vcpu *vcpu, u64 access, u64 la, u64 flags) +{ + bool user_mode, user_as, rflags_ac; + + if (!!(flags & KVM_X86_EMULFLAG_SKIP_LASS) || + !kvm_is_cr4_bit_set(vcpu, X86_CR4_LASS)) + return false; + + WARN_ON_ONCE(!is_long_mode(vcpu)); + + user_as =3D !(la >> 63); + + /* + * An access is a supervisor-mode access if CPL < 3 or if it implicitly + * accesses a system data structure. For implicit accesses to system + * data structure, the processor acts as if RFLAGS.AC is clear. + */ + if (access & PFERR_IMPLICIT_ACCESS) { + user_mode =3D false; + rflags_ac =3D false; + } else { + user_mode =3D vmx_get_cpl(vcpu) =3D=3D 3; + if (!user_mode) + rflags_ac =3D !!(kvm_get_rflags(vcpu) & X86_EFLAGS_AC); + } + + if (user_mode !=3D user_as) { + /* + * Supervisor-mode _data_ accesses to user address space + * cause LASS violations only if SMAP is enabled. + */ + if (!user_mode && !(access & PFERR_FETCH_MASK)) { + return kvm_is_cr4_bit_set(vcpu, X86_CR4_SMAP) && + !rflags_ac; + } else { + return true; + } + } + + return false; +} + +static bool vmx_check_lass(struct kvm_vcpu *vcpu, u64 access, u64 la, u64 = flags) +{ + return is_long_mode(vcpu) && __vmx_check_lass(vcpu, access, la, flags); +} + static struct kvm_x86_ops vmx_x86_ops __initdata =3D { .name =3D "kvm_intel", =20 @@ -8207,6 +8260,8 @@ static struct kvm_x86_ops vmx_x86_ops __initdata =3D { .complete_emulated_msr =3D kvm_complete_insn_gp, =20 .vcpu_deliver_sipi_vector =3D kvm_vcpu_deliver_sipi_vector, + + .check_lass =3D vmx_check_lass, }; =20 static unsigned int vmx_handle_intel_pt_intr(void) diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h index a3da84f4ea45..6569385a5978 100644 --- a/arch/x86/kvm/vmx/vmx.h +++ b/arch/x86/kvm/vmx/vmx.h @@ -433,6 +433,8 @@ void vmx_enable_intercept_for_msr(struct kvm_vcpu *vcpu= , u32 msr, int type); u64 vmx_get_l2_tsc_offset(struct kvm_vcpu *vcpu); u64 vmx_get_l2_tsc_multiplier(struct kvm_vcpu *vcpu); =20 +bool __vmx_check_lass(struct kvm_vcpu *vcpu, u64 access, u64 la, u64 flags= ); + static inline void vmx_set_intercept_for_msr(struct kvm_vcpu *vcpu, u32 ms= r, int type, bool value) { --=20 2.27.0 From nobody Wed Dec 17 10:19:57 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 82D02C77B73 for ; Thu, 20 Apr 2023 14:16:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232050AbjDTOQc (ORCPT ); Thu, 20 Apr 2023 10:16:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39536 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231926AbjDTOQS (ORCPT ); Thu, 20 Apr 2023 10:16:18 -0400 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C355540DB; Thu, 20 Apr 2023 07:16:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1682000177; x=1713536177; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=DAVFBOgJwmbdVgPRUhLaouaTcJcJ1MnxKOkOedpXi1U=; b=hvxPxf42OVYIY39GpTsSGfjw3rEsjqU8OSQGrVMBo+TS7Zr5FBUgnRSH x9/1/I+C81J//Z8BwPYFAMiNfk7lmvCspzcL9GHPdqqvzLaLklDtJJJDb IlPOeKRc4cb/knbAOYa8Pl23xS/KeAHt2qmNqD/ENAhkb2n8YvTUtFrsf +de6EuiBVkd4ItdWjNYjrBVpooQM+tX3rVooeiq/sL0Udriz73HbfqMgp v32cRjiNYQDxs509N7Tqe3Xa2oof77Xx8KqBxrQlSMRej9lqKk2XmO5RT k64Nxa+Hni1IT+W5Y4CSHfWeUDqNilUBNVD0fU7tBcW90mhaUxRmeQw0E Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10686"; a="343217826" X-IronPort-AV: E=Sophos;i="5.99,212,1677571200"; d="scan'208";a="343217826" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Apr 2023 07:16:10 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10686"; a="816028858" X-IronPort-AV: E=Sophos;i="5.99,212,1677571200"; d="scan'208";a="816028858" Received: from arthur-vostro-3668.sh.intel.com ([10.238.200.53]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Apr 2023 07:16:06 -0700 From: Zeng Guang To: Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , H Peter Anvin , kvm@vger.kernel.org Cc: x86@kernel.org, linux-kernel@vger.kernel.org, Gao Chao , Zeng Guang Subject: [PATCH 3/6] KVM: x86: Add emulator helper for LASS violation check Date: Thu, 20 Apr 2023 21:37:21 +0800 Message-Id: <20230420133724.11398-4-guang.zeng@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230420133724.11398-1-guang.zeng@intel.com> References: <20230420133724.11398-1-guang.zeng@intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" When LASS is enabled, KVM need apply LASS violation check to instruction emulations. Add helper for the usage of x86 emulator to perform LASS protection. Signed-off-by: Zeng Guang --- arch/x86/kvm/kvm_emulate.h | 1 + arch/x86/kvm/x86.c | 9 +++++++++ 2 files changed, 10 insertions(+) diff --git a/arch/x86/kvm/kvm_emulate.h b/arch/x86/kvm/kvm_emulate.h index 2d9662be8333..1c55247d52d7 100644 --- a/arch/x86/kvm/kvm_emulate.h +++ b/arch/x86/kvm/kvm_emulate.h @@ -224,6 +224,7 @@ struct x86_emulate_ops { int (*leave_smm)(struct x86_emulate_ctxt *ctxt); void (*triple_fault)(struct x86_emulate_ctxt *ctxt); int (*set_xcr)(struct x86_emulate_ctxt *ctxt, u32 index, u64 xcr); + bool (*check_lass)(struct x86_emulate_ctxt *ctxt, u64 access, u64 la, u64= flags); }; =20 /* Type, address-of, and value of an instruction's operand. */ diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c index 87feb1249ad6..704c5e4b9e76 100644 --- a/arch/x86/kvm/x86.c +++ b/arch/x86/kvm/x86.c @@ -8193,6 +8193,14 @@ static void emulator_vm_bugged(struct x86_emulate_ct= xt *ctxt) kvm_vm_bugged(kvm); } =20 +static bool emulator_check_lass(struct x86_emulate_ctxt *ctxt, + u64 access, u64 la, u64 flags) +{ + struct kvm_vcpu *vcpu =3D emul_to_vcpu(ctxt); + + return static_call(kvm_x86_check_lass)(vcpu, access, la, flags); +} + static const struct x86_emulate_ops emulate_ops =3D { .vm_bugged =3D emulator_vm_bugged, .read_gpr =3D emulator_read_gpr, @@ -8237,6 +8245,7 @@ static const struct x86_emulate_ops emulate_ops =3D { .leave_smm =3D emulator_leave_smm, .triple_fault =3D emulator_triple_fault, .set_xcr =3D emulator_set_xcr, + .check_lass =3D emulator_check_lass, }; =20 static void toggle_interruptibility(struct kvm_vcpu *vcpu, u32 mask) --=20 2.27.0 From nobody Wed Dec 17 10:19:57 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A5F0C77B73 for ; Thu, 20 Apr 2023 14:16:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231909AbjDTOQe (ORCPT ); Thu, 20 Apr 2023 10:16:34 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39524 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231956AbjDTOQV (ORCPT ); Thu, 20 Apr 2023 10:16:21 -0400 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8BFEF40EE; Thu, 20 Apr 2023 07:16:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1682000179; x=1713536179; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=vRwyjqtJWovvtpBFmwcHt38o9HcKHdqbwSJI0tUzb2s=; b=VC7rCWEgc5Qg7C7YhhHfBYQLK6cl2TKYQdNQd+Gu+reHNUdNzVyuYDJE sNEbtCTn3dH/OvCDDSRvDyk8jFIlHNyfUF1r5gLF0cqRhz7ff6M44tFYA U5RJUsfFcJrVvYC/rbfePfPU3XLRS8MIksPyQwSvvHGylxdQnmmgGuEN3 LmP7+Tr1+bHbiPhfXZC8O0SjeaDek8FKwOmptF18t1X9mz5Vb6eJS0cv5 V2hdf4Q30QzwWun6SU0JHTNOJHkgsfc6WG3C/DFwV5lIy3PaOQ2aKE72F hhwMH+l/OX5Wo9HoszMZNy6CQq5SF5xjgYJIt12DdC2AMMKAWvHGI5CVR Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10686"; a="343217849" X-IronPort-AV: E=Sophos;i="5.99,212,1677571200"; d="scan'208";a="343217849" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Apr 2023 07:16:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10686"; a="816028886" X-IronPort-AV: E=Sophos;i="5.99,212,1677571200"; d="scan'208";a="816028886" Received: from arthur-vostro-3668.sh.intel.com ([10.238.200.53]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Apr 2023 07:16:10 -0700 From: Zeng Guang To: Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , H Peter Anvin , kvm@vger.kernel.org Cc: x86@kernel.org, linux-kernel@vger.kernel.org, Gao Chao , Zeng Guang Subject: [PATCH 4/6] KVM: x86: LASS protection on KVM emulation when LASS enabled Date: Thu, 20 Apr 2023 21:37:22 +0800 Message-Id: <20230420133724.11398-5-guang.zeng@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230420133724.11398-1-guang.zeng@intel.com> References: <20230420133724.11398-1-guang.zeng@intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Do LASS violation check for instructions emulated by KVM. Note that for instructions executed in the guest directly, hardware will perform the check. Not all instruction emulation leads to accesses to guest linear addresses because 1) some instrutions like CPUID, RDMSR, don't take memory as operands 2) instruction fetch in most cases is already done inside the guest. Four cases in which kvm may access guest linear addresses are identified by code inspection: - KVM emulator uses segmented address for instruction fetches or data accesses. - For implicit data access, KVM emulator gets address to a system data structure(GDT/LDT/IDT/TR). - For VMX instruction emulation, KVM gets the address from "VM-exit instruction information" field in VMCS. - For SGX ENCLS instruction emulation, KVM gets the address from registers. LASS violation check applies to these linear address so as to enforce mode-based protections as hardware behaves. As exceptions, the target memory address of emulation of invlpg, branch and call instructions doesn't require LASS violation check. Signed-off-by: Zeng Guang --- arch/x86/kvm/emulate.c | 36 +++++++++++++++++++++++++++++++----- arch/x86/kvm/vmx/nested.c | 3 +++ arch/x86/kvm/vmx/sgx.c | 2 ++ 3 files changed, 36 insertions(+), 5 deletions(-) diff --git a/arch/x86/kvm/emulate.c b/arch/x86/kvm/emulate.c index 5cc3efa0e21c..a9a022fd712e 100644 --- a/arch/x86/kvm/emulate.c +++ b/arch/x86/kvm/emulate.c @@ -687,7 +687,8 @@ static __always_inline int __linearize(struct x86_emula= te_ctxt *ctxt, struct segmented_address addr, unsigned *max_size, unsigned size, bool write, bool fetch, - enum x86emul_mode mode, ulong *linear) + enum x86emul_mode mode, ulong *linear, + u64 flags) { struct desc_struct desc; bool usable; @@ -695,6 +696,7 @@ static __always_inline int __linearize(struct x86_emula= te_ctxt *ctxt, u32 lim; u16 sel; u8 va_bits; + u64 access =3D fetch ? PFERR_FETCH_MASK : 0; =20 la =3D seg_base(ctxt, addr.seg) + addr.ea; *max_size =3D 0; @@ -740,6 +742,10 @@ static __always_inline int __linearize(struct x86_emul= ate_ctxt *ctxt, } break; } + + if (ctxt->ops->check_lass(ctxt, access, *linear, flags)) + goto bad; + if (la & (insn_alignment(ctxt, size) - 1)) return emulate_gp(ctxt, 0); return X86EMUL_CONTINUE; @@ -757,7 +763,7 @@ static int linearize(struct x86_emulate_ctxt *ctxt, { unsigned max_size; return __linearize(ctxt, addr, &max_size, size, write, false, - ctxt->mode, linear); + ctxt->mode, linear, 0); } =20 static inline int assign_eip(struct x86_emulate_ctxt *ctxt, ulong dst) @@ -770,7 +776,10 @@ static inline int assign_eip(struct x86_emulate_ctxt *= ctxt, ulong dst) =20 if (ctxt->op_bytes !=3D sizeof(unsigned long)) addr.ea =3D dst & ((1UL << (ctxt->op_bytes << 3)) - 1); - rc =3D __linearize(ctxt, addr, &max_size, 1, false, true, ctxt->mode, &li= near); + + /* LASS doesn't apply to address for branch and call instructions */ + rc =3D __linearize(ctxt, addr, &max_size, 1, false, true, ctxt->mode, + &linear, KVM_X86_EMULFLAG_SKIP_LASS); if (rc =3D=3D X86EMUL_CONTINUE) ctxt->_eip =3D addr.ea; return rc; @@ -845,6 +854,13 @@ static inline int jmp_rel(struct x86_emulate_ctxt *ctx= t, int rel) static int linear_read_system(struct x86_emulate_ctxt *ctxt, ulong linear, void *data, unsigned size) { + if (ctxt->ops->check_lass(ctxt, PFERR_IMPLICIT_ACCESS, linear, 0)) { + ctxt->exception.vector =3D GP_VECTOR; + ctxt->exception.error_code =3D 0; + ctxt->exception.error_code_valid =3D true; + return X86EMUL_PROPAGATE_FAULT; + } + return ctxt->ops->read_std(ctxt, linear, data, size, &ctxt->exception, tr= ue); } =20 @@ -852,6 +868,13 @@ static int linear_write_system(struct x86_emulate_ctxt= *ctxt, ulong linear, void *data, unsigned int size) { + if (ctxt->ops->check_lass(ctxt, PFERR_IMPLICIT_ACCESS, linear, 0)) { + ctxt->exception.vector =3D GP_VECTOR; + ctxt->exception.error_code =3D 0; + ctxt->exception.error_code_valid =3D true; + return X86EMUL_PROPAGATE_FAULT; + } + return ctxt->ops->write_std(ctxt, linear, data, size, &ctxt->exception, t= rue); } =20 @@ -907,7 +930,7 @@ static int __do_insn_fetch_bytes(struct x86_emulate_ctx= t *ctxt, int op_size) * against op_size. */ rc =3D __linearize(ctxt, addr, &max_size, 0, false, true, ctxt->mode, - &linear); + &linear, 0); if (unlikely(rc !=3D X86EMUL_CONTINUE)) return rc; =20 @@ -3432,8 +3455,11 @@ static int em_invlpg(struct x86_emulate_ctxt *ctxt) { int rc; ulong linear; + unsigned max_size; =20 - rc =3D linearize(ctxt, ctxt->src.addr.mem, 1, false, &linear); + /* LASS doesn't apply to the memory address for invlpg */ + rc =3D __linearize(ctxt, ctxt->src.addr.mem, &max_size, 1, false, false, + ctxt->mode, &linear, KVM_X86_EMULFLAG_SKIP_LASS); if (rc =3D=3D X86EMUL_CONTINUE) ctxt->ops->invlpg(ctxt, linear); /* Disable writeback. */ diff --git a/arch/x86/kvm/vmx/nested.c b/arch/x86/kvm/vmx/nested.c index c8ae9d0e59b3..55c88c4593a6 100644 --- a/arch/x86/kvm/vmx/nested.c +++ b/arch/x86/kvm/vmx/nested.c @@ -4974,6 +4974,9 @@ int get_vmx_mem_address(struct kvm_vcpu *vcpu, unsign= ed long exit_qualification, * destination for long mode! */ exn =3D is_noncanonical_address(*ret, vcpu); + + if (!exn) + exn =3D __vmx_check_lass(vcpu, 0, *ret, 0); } else { /* * When not in long mode, the virtual/linear address is diff --git a/arch/x86/kvm/vmx/sgx.c b/arch/x86/kvm/vmx/sgx.c index b12da2a6dec9..30cb5d0980be 100644 --- a/arch/x86/kvm/vmx/sgx.c +++ b/arch/x86/kvm/vmx/sgx.c @@ -37,6 +37,8 @@ static int sgx_get_encls_gva(struct kvm_vcpu *vcpu, unsig= ned long offset, fault =3D true; } else if (likely(is_long_mode(vcpu))) { fault =3D is_noncanonical_address(*gva, vcpu); + if (!fault) + fault =3D __vmx_check_lass(vcpu, 0, *gva, 0); } else { *gva &=3D 0xffffffff; fault =3D (s.unusable) || --=20 2.27.0 From nobody Wed Dec 17 10:19:57 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 39995C77B76 for ; Thu, 20 Apr 2023 14:16:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232067AbjDTOQi (ORCPT ); Thu, 20 Apr 2023 10:16:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39536 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231962AbjDTOQV (ORCPT ); Thu, 20 Apr 2023 10:16:21 -0400 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 967FE5594; Thu, 20 Apr 2023 07:16:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1682000180; x=1713536180; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=/9rOO3Vbv54KntD8O0wgn+L44l9VgaKRnztR0QPc7F0=; b=FGLh1XN6L0CmuLbhgRPOwS0kpy2wfbadqE3Vrb+pdgG60Jx9yNssS98X TKHqfr5a4AwcdCr/ljmidxJ8209H29UESkdaNwNmzvCTSo092PF8PoOEz nWjmNf9I8JCLeBT9avg8JBQKSXr3whtGJXIxBqsfdkamiFbab2exkapZK GLRZhXykzXToKbj5/lBDTxlybZFUqY/1WxlHWHYuDHRxcYdUI12bV97Fp sjnoNBpc/cz3+XQ2RT1vqEJMVukI6OTJZ4Mcf0sclr0+Hc/FmXUKtex6E QklRIumkp76Ks6MjLrw/t+rDsEy3DA9m1Ufn8GFNg5W1D/mPHU9uGail6 Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10686"; a="343217863" X-IronPort-AV: E=Sophos;i="5.99,212,1677571200"; d="scan'208";a="343217863" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Apr 2023 07:16:16 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10686"; a="816028909" X-IronPort-AV: E=Sophos;i="5.99,212,1677571200"; d="scan'208";a="816028909" Received: from arthur-vostro-3668.sh.intel.com ([10.238.200.53]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Apr 2023 07:16:13 -0700 From: Zeng Guang To: Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , H Peter Anvin , kvm@vger.kernel.org Cc: x86@kernel.org, linux-kernel@vger.kernel.org, Gao Chao , Zeng Guang Subject: [PATCH 5/6] KVM: x86: Advertise LASS CPUID to user space Date: Thu, 20 Apr 2023 21:37:23 +0800 Message-Id: <20230420133724.11398-6-guang.zeng@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230420133724.11398-1-guang.zeng@intel.com> References: <20230420133724.11398-1-guang.zeng@intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" LASS (Linear-address space separation) is an independent mechanism to enforce the mode-based protection that can prevent user-mode accesses to supervisor-mode addresses, and vice versa. Because the LASS protections are applied before paging, malicious software can not acquire any paging-based timing information to compromise the security of system. The CPUID bit definition to support LASS: CPUID.(EAX=3D07H.ECX=3D1):EAX.LASS[bit 6] Advertise LASS to user space to support LASS virtualization. Signed-off-by: Zeng Guang --- arch/x86/kvm/cpuid.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index ba7f7abc8964..5facb8037140 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -663,8 +663,8 @@ void kvm_set_cpu_caps(void) kvm_cpu_cap_set(X86_FEATURE_SPEC_CTRL_SSBD); =20 kvm_cpu_cap_mask(CPUID_7_1_EAX, - F(AVX_VNNI) | F(AVX512_BF16) | F(CMPCCXADD) | F(AMX_FP16) | - F(AVX_IFMA) + F(AVX_VNNI) | F(AVX512_BF16) | F(LASS) | F(CMPCCXADD) | + F(AMX_FP16) | F(AVX_IFMA) ); =20 kvm_cpu_cap_init_kvm_defined(CPUID_7_1_EDX, --=20 2.27.0 From nobody Wed Dec 17 10:19:57 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 139F9C77B72 for ; Thu, 20 Apr 2023 14:16:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232076AbjDTOQo (ORCPT ); Thu, 20 Apr 2023 10:16:44 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39524 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231969AbjDTOQW (ORCPT ); Thu, 20 Apr 2023 10:16:22 -0400 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3A78C5FFB; Thu, 20 Apr 2023 07:16:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1682000181; x=1713536181; h=from:to:cc:subject:date:message-id:in-reply-to: references; bh=mBqjl0yhEeb8HytbOFw3rMDfti8h06Ban5qDOvLFqy8=; b=fQwdg0q9wJaME2h+ePIxss6z6WIOGNZOE+lkFBr7SdxMsNulLMQ5g+za EdaxQQDy6y5r5TgvdfEu+ilPQLvTy3frTK/keponnTSbMjylWgl/bOfA0 ijM0RO5EOtRQd/GJqeI92GVIaaVnbaU6pcHaR0Xgb1FTI2u/9fjzI1uXO XEoPrbcqZiz4nIjyDWB5kngCVo2aVw2RXdawkv7RmK9iUykiN6tsUxbR+ xaYvJdAEDOyHGPxLy4Tpf0Nd5WJvC7tTSOzeFm60J+6O4NM+uyxYKUaVl Z2zAXaOV4rNnnKjgE84G6LbRfZz5JPPgdZMSUCsEn2/ezi7udenIbY77k g==; X-IronPort-AV: E=McAfee;i="6600,9927,10686"; a="343217882" X-IronPort-AV: E=Sophos;i="5.99,212,1677571200"; d="scan'208";a="343217882" Received: from orsmga004.jf.intel.com ([10.7.209.38]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Apr 2023 07:16:19 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10686"; a="816028928" X-IronPort-AV: E=Sophos;i="5.99,212,1677571200"; d="scan'208";a="816028928" Received: from arthur-vostro-3668.sh.intel.com ([10.238.200.53]) by orsmga004-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 20 Apr 2023 07:16:16 -0700 From: Zeng Guang To: Paolo Bonzini , Sean Christopherson , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , H Peter Anvin , kvm@vger.kernel.org Cc: x86@kernel.org, linux-kernel@vger.kernel.org, Gao Chao , Zeng Guang Subject: [PATCH 6/6] KVM: x86: Set KVM LASS based on hardware capability Date: Thu, 20 Apr 2023 21:37:24 +0800 Message-Id: <20230420133724.11398-7-guang.zeng@intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230420133724.11398-1-guang.zeng@intel.com> References: <20230420133724.11398-1-guang.zeng@intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Host kernel may clear LASS capability in boot_cpu_data.x86_capability besides explicitly using clearcpuid parameter. That will cause guest not being able to manage LASS independently. So set KVM LASS directly based on hardware capability to eliminate the dependency. Add new helper functions to facilitate getting result of CPUID sub-leaf. Signed-off-by: Zeng Guang --- arch/x86/include/asm/cpuid.h | 36 ++++++++++++++++++++++++++++++++++++ arch/x86/kvm/cpuid.c | 4 ++++ 2 files changed, 40 insertions(+) diff --git a/arch/x86/include/asm/cpuid.h b/arch/x86/include/asm/cpuid.h index 9bee3e7bf973..a25dd00b7c0a 100644 --- a/arch/x86/include/asm/cpuid.h +++ b/arch/x86/include/asm/cpuid.h @@ -127,6 +127,42 @@ static inline unsigned int cpuid_edx(unsigned int op) return edx; } =20 +static inline unsigned int cpuid_count_eax(unsigned int op, int count) +{ + unsigned int eax, ebx, ecx, edx; + + cpuid_count(op, count, &eax, &ebx, &ecx, &edx); + + return eax; +} + +static inline unsigned int cpuid_count_ebx(unsigned int op, int count) +{ + unsigned int eax, ebx, ecx, edx; + + cpuid_count(op, count, &eax, &ebx, &ecx, &edx); + + return ebx; +} + +static inline unsigned int cpuid_count_ecx(unsigned int op, int count) +{ + unsigned int eax, ebx, ecx, edx; + + cpuid_count(op, count, &eax, &ebx, &ecx, &edx); + + return ecx; +} + +static inline unsigned int cpuid_count_edx(unsigned int op, int count) +{ + unsigned int eax, ebx, ecx, edx; + + cpuid_count(op, count, &eax, &ebx, &ecx, &edx); + + return edx; +} + static __always_inline bool cpuid_function_is_indexed(u32 function) { switch (function) { diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index 5facb8037140..e99b99ebe1fe 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -667,6 +667,10 @@ void kvm_set_cpu_caps(void) F(AMX_FP16) | F(AVX_IFMA) ); =20 + /* Set LASS based on hardware capability */ + if (cpuid_count_eax(7, 1) & F(LASS)) + kvm_cpu_cap_set(X86_FEATURE_LASS); + kvm_cpu_cap_init_kvm_defined(CPUID_7_1_EDX, F(AVX_VNNI_INT8) | F(AVX_NE_CONVERT) | F(PREFETCHITI) ); --=20 2.27.0