From nobody Thu Oct 2 07:48:26 2025 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5664D2FC87A for ; Fri, 19 Sep 2025 22:33:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758321237; cv=none; b=JIopwvaDK+wPoA5G/1z8ZP54kLeP3diucdcO/Cs0+97RCj1/9Rnl7Hirxb+Dd8hKhm4Zj4/AoQXRe01I6nNj6mF1I8LWo4c0mB5TUdBd8gYKoFKgNEsUyp3dN038nBqlPkzgTIXpn7TUwX/qzs+Iy615CtJMSJTI6kbhpDHi7Cw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758321237; c=relaxed/simple; bh=H85izHo3sM6kNkmT1jTPpHBNMJmymsnKRWYhCXf1xIQ=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=rtKtfRqBodMFk6+Vb/nJdrc0F+NBiwksH7eqx4AGeQhcn2s2kBGLPKc8Mryxbn8nDQbs3LQnbrMdn06wuF3A8zqtvt+6SjbqdLpoG5dYFr45FsZMgAvxX0sa8IHUl49T9QOMeeTTdgmuGn3dAEet3AEFeNzLRTQEA6tyIzwd2cE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=w56vZVoe; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="w56vZVoe" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-24ca417fb41so26512765ad.1 for ; Fri, 19 Sep 2025 15:33:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1758321235; x=1758926035; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=PjK5+Yt+wjh8VShHpuKoKCtX5MZ19p55Nm6f0snPAks=; b=w56vZVoenyKCclyorDg9e9tyS218pPHL4xT1MHrtabRmtukWMBy/tosrL19uEDdmE3 RQubKiLp0ZHiZr8/zNF1ifUZXdNfFjVznlLoczYKlrbe2wYLIPAcBh0AL4AGFFgpzwVd uReFVRlL8ESlYc4RGxr2GjR1uoosC9wUdqFZK4EK222jCqk3O0eOAWTPQpr3o9tEh0XJ Mpo/XHzSMJME1fOgPIoibdtVjszzPRPrqsoiIGo9tNcHFQ+Z/4USf8g/XLdEvvt8hxtw R6stVGquatqB2lQJte+m+fKyDaYsWIApOWx4yaakh7W+1mqy+mifXNTR89aK5OWcTWIM +u4Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1758321235; x=1758926035; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=PjK5+Yt+wjh8VShHpuKoKCtX5MZ19p55Nm6f0snPAks=; b=qmYKUpUxTpEaUfLzA4pZFbYqD/mKOU30JEIS9IFfwWegJlAof/geCuJvlfyfbcRR34 DjVGmFwyUNOrUSWuHSzAbGBCyHneQf5WwpggE8IZL2ffyEHdBtCzmkGPs6vWXXP5c2gd WlnHwgoeI2XJX0lG4WY+M4Sc77EWyTHKoLw3R/OhTy6pDLTkUtcrncGq4QG2pu4xNt3j TFTkVUbxL+2e8Mxxq8N9X3wKuwwfx0Sb5OsFcXIYbSERjt7v8nTnx7bY+54wmTTZ0YuF G4JI3ME+190Dzud6qR6eBdI3KrkBfJTOjOnkniWE+b/iR5Kb76TFp7n+Fjtt9RJwXx9K XYrA== X-Forwarded-Encrypted: i=1; AJvYcCWLAEEBSaYE0vHNp7tVKUfJK5R/EVRdvOXEEv9e4kg6OIm7U/z03WPEi7y7LENZlePM1RLUypSSyPhHfE4=@vger.kernel.org X-Gm-Message-State: AOJu0YzHHWMaMBN1XQgUkuLAbV5q776U7OEQVUGrA7i7THE115MrN6hA 21W1pvu10y8rscwHnwFmWX/8zZ89OL5zL71XOTGnauiBkk0T3BInkGZcqqIpFwdCWiiN+EnUJKK BajUoEQ== X-Google-Smtp-Source: AGHT+IH7oW74ad5r7dOGMu9LUzDr9y7Upw5+k1X7yrhcCg7TokB+Tkp43mKDPjj87CuGA4qb8Q9yBtJXo5I= X-Received: from pjbsf15.prod.google.com ([2002:a17:90b:51cf:b0:325:220a:dd41]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:db12:b0:251:493c:43e3 with SMTP id d9443c01a7336-269ba516161mr66907895ad.31.1758321234774; Fri, 19 Sep 2025 15:33:54 -0700 (PDT) Reply-To: Sean Christopherson Date: Fri, 19 Sep 2025 15:32:35 -0700 In-Reply-To: <20250919223258.1604852-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250919223258.1604852-1-seanjc@google.com> X-Mailer: git-send-email 2.51.0.470.ga7dc726c21-goog Message-ID: <20250919223258.1604852-29-seanjc@google.com> Subject: [PATCH v16 28/51] KVM: x86: Enable CET virtualization for VMX and advertise to userspace From: Sean Christopherson To: Paolo Bonzini , Sean Christopherson Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Tom Lendacky , Mathias Krause , John Allen , Rick Edgecombe , Chao Gao , Binbin Wu , Xiaoyao Li , Maxim Levitsky , Zhang Yi Z , Xin Li Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Yang Weijiang Add support for the LOAD_CET_STATE VM-Enter and VM-Exit controls, the CET XFEATURE bits in XSS, and advertise support for IBT and SHSTK to userspace. Explicitly clear IBT and SHSTK onn SVM, as additional work is needed to enable CET on SVM, e.g. to context switch S_CET and other state. Disable KVM CET feature if unrestricted_guest is unsupported/disabled as KVM does not support emulating CET, as running without Unrestricted Guest can result in KVM emulating large swaths of guest code. While it's highly unlikely any guest will trigger emulation while also utilizing IBT or SHSTK, there's zero reason to allow CET without Unrestricted Guest as that combination should only be possible when explicitly disabling unrestricted_guest for testing purposes. Disable CET if VMX_BASIC[bit56] =3D=3D 0, i.e. if hardware strictly enforces the presence of an Error Code based on exception vector, as attempting to inject a #CP with an Error Code (#CP architecturally has an Error Code) will fail due to the #CP vector historically not having an Error Code. Clear S_CET and SSP-related VMCS on "reset" to emulate the architectural of CET MSRs and SSP being reset to 0 after RESET, power-up and INIT. Note, KVM already clears guest CET state that is managed via XSTATE in kvm_xstate_reset(). Signed-off-by: Yang Weijiang Signed-off-by: Mathias Krause Tested-by: Mathias Krause Tested-by: John Allen Tested-by: Rick Edgecombe Signed-off-by: Chao Gao [sean: move some bits to separate patches, massage changelog] Signed-off-by: Sean Christopherson Reviewed-by: Binbin Wu Reviewed-by: Xiaoyao Li --- arch/x86/include/asm/vmx.h | 1 + arch/x86/kvm/cpuid.c | 2 ++ arch/x86/kvm/svm/svm.c | 4 ++++ arch/x86/kvm/vmx/capabilities.h | 5 +++++ arch/x86/kvm/vmx/vmx.c | 30 +++++++++++++++++++++++++++++- arch/x86/kvm/vmx/vmx.h | 6 ++++-- 6 files changed, 45 insertions(+), 3 deletions(-) diff --git a/arch/x86/include/asm/vmx.h b/arch/x86/include/asm/vmx.h index ce10a7e2d3d9..c85c50019523 100644 --- a/arch/x86/include/asm/vmx.h +++ b/arch/x86/include/asm/vmx.h @@ -134,6 +134,7 @@ #define VMX_BASIC_DUAL_MONITOR_TREATMENT BIT_ULL(49) #define VMX_BASIC_INOUT BIT_ULL(54) #define VMX_BASIC_TRUE_CTLS BIT_ULL(55) +#define VMX_BASIC_NO_HW_ERROR_CODE_CC BIT_ULL(56) =20 static inline u32 vmx_basic_vmcs_revision_id(u64 vmx_basic) { diff --git a/arch/x86/kvm/cpuid.c b/arch/x86/kvm/cpuid.c index b5c4cb13630c..b861a88083e1 100644 --- a/arch/x86/kvm/cpuid.c +++ b/arch/x86/kvm/cpuid.c @@ -946,6 +946,7 @@ void kvm_set_cpu_caps(void) VENDOR_F(WAITPKG), F(SGX_LC), F(BUS_LOCK_DETECT), + X86_64_F(SHSTK), ); =20 /* @@ -990,6 +991,7 @@ void kvm_set_cpu_caps(void) F(AMX_INT8), F(AMX_BF16), F(FLUSH_L1D), + F(IBT), ); =20 if (boot_cpu_has(X86_FEATURE_AMD_IBPB_RET) && diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index 67f4eed01526..73dde1645e46 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -5221,6 +5221,10 @@ static __init void svm_set_cpu_caps(void) kvm_caps.supported_perf_cap =3D 0; kvm_caps.supported_xss =3D 0; =20 + /* KVM doesn't yet support CET virtualization for SVM. */ + kvm_cpu_cap_clear(X86_FEATURE_SHSTK); + kvm_cpu_cap_clear(X86_FEATURE_IBT); + /* CPUID 0x80000001 and 0x8000000A (SVM features) */ if (nested) { kvm_cpu_cap_set(X86_FEATURE_SVM); diff --git a/arch/x86/kvm/vmx/capabilities.h b/arch/x86/kvm/vmx/capabilitie= s.h index 59c83888bdc0..02aadb9d730e 100644 --- a/arch/x86/kvm/vmx/capabilities.h +++ b/arch/x86/kvm/vmx/capabilities.h @@ -73,6 +73,11 @@ static inline bool cpu_has_vmx_basic_inout(void) return vmcs_config.basic & VMX_BASIC_INOUT; } =20 +static inline bool cpu_has_vmx_basic_no_hw_errcode_cc(void) +{ + return vmcs_config.basic & VMX_BASIC_NO_HW_ERROR_CODE_CC; +} + static inline bool cpu_has_virtual_nmis(void) { return vmcs_config.pin_based_exec_ctrl & PIN_BASED_VIRTUAL_NMIS && diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index a7d9e60b2771..69e35440cee7 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -2615,6 +2615,7 @@ static int setup_vmcs_config(struct vmcs_config *vmcs= _conf, { VM_ENTRY_LOAD_IA32_EFER, VM_EXIT_LOAD_IA32_EFER }, { VM_ENTRY_LOAD_BNDCFGS, VM_EXIT_CLEAR_BNDCFGS }, { VM_ENTRY_LOAD_IA32_RTIT_CTL, VM_EXIT_CLEAR_IA32_RTIT_CTL }, + { VM_ENTRY_LOAD_CET_STATE, VM_EXIT_LOAD_CET_STATE }, }; =20 memset(vmcs_conf, 0, sizeof(*vmcs_conf)); @@ -4881,6 +4882,14 @@ void vmx_vcpu_reset(struct kvm_vcpu *vcpu, bool init= _event) =20 vmcs_write32(VM_ENTRY_INTR_INFO_FIELD, 0); /* 22.2.1 */ =20 + if (kvm_cpu_cap_has(X86_FEATURE_SHSTK)) { + vmcs_writel(GUEST_SSP, 0); + vmcs_writel(GUEST_INTR_SSP_TABLE, 0); + } + if (kvm_cpu_cap_has(X86_FEATURE_IBT) || + kvm_cpu_cap_has(X86_FEATURE_SHSTK)) + vmcs_writel(GUEST_S_CET, 0); + kvm_make_request(KVM_REQ_APIC_PAGE_RELOAD, vcpu); =20 vpid_sync_context(vmx->vpid); @@ -6348,6 +6357,10 @@ void dump_vmcs(struct kvm_vcpu *vcpu) if (vmcs_read32(VM_EXIT_MSR_STORE_COUNT) > 0) vmx_dump_msrs("guest autostore", &vmx->msr_autostore.guest); =20 + if (vmentry_ctl & VM_ENTRY_LOAD_CET_STATE) + pr_err("S_CET =3D 0x%016lx, SSP =3D 0x%016lx, SSP TABLE =3D 0x%016lx\n", + vmcs_readl(GUEST_S_CET), vmcs_readl(GUEST_SSP), + vmcs_readl(GUEST_INTR_SSP_TABLE)); pr_err("*** Host State ***\n"); pr_err("RIP =3D 0x%016lx RSP =3D 0x%016lx\n", vmcs_readl(HOST_RIP), vmcs_readl(HOST_RSP)); @@ -6378,6 +6391,10 @@ void dump_vmcs(struct kvm_vcpu *vcpu) vmcs_read64(HOST_IA32_PERF_GLOBAL_CTRL)); if (vmcs_read32(VM_EXIT_MSR_LOAD_COUNT) > 0) vmx_dump_msrs("host autoload", &vmx->msr_autoload.host); + if (vmexit_ctl & VM_EXIT_LOAD_CET_STATE) + pr_err("S_CET =3D 0x%016lx, SSP =3D 0x%016lx, SSP TABLE =3D 0x%016lx\n", + vmcs_readl(HOST_S_CET), vmcs_readl(HOST_SSP), + vmcs_readl(HOST_INTR_SSP_TABLE)); =20 pr_err("*** Control State ***\n"); pr_err("CPUBased=3D0x%08x SecondaryExec=3D0x%08x TertiaryExec=3D0x%016llx= \n", @@ -7959,7 +7976,6 @@ static __init void vmx_set_cpu_caps(void) kvm_cpu_cap_set(X86_FEATURE_UMIP); =20 /* CPUID 0xD.1 */ - kvm_caps.supported_xss =3D 0; if (!cpu_has_vmx_xsaves()) kvm_cpu_cap_clear(X86_FEATURE_XSAVES); =20 @@ -7971,6 +7987,18 @@ static __init void vmx_set_cpu_caps(void) =20 if (cpu_has_vmx_waitpkg()) kvm_cpu_cap_check_and_set(X86_FEATURE_WAITPKG); + + /* + * Disable CET if unrestricted_guest is unsupported as KVM doesn't + * enforce CET HW behaviors in emulator. On platforms with + * VMX_BASIC[bit56] =3D=3D 0, inject #CP at VMX entry with error code + * fails, so disable CET in this case too. + */ + if (!cpu_has_load_cet_ctrl() || !enable_unrestricted_guest || + !cpu_has_vmx_basic_no_hw_errcode_cc()) { + kvm_cpu_cap_clear(X86_FEATURE_SHSTK); + kvm_cpu_cap_clear(X86_FEATURE_IBT); + } } =20 static bool vmx_is_io_intercepted(struct kvm_vcpu *vcpu, diff --git a/arch/x86/kvm/vmx/vmx.h b/arch/x86/kvm/vmx/vmx.h index 23d6e89b96f2..af8224e074ee 100644 --- a/arch/x86/kvm/vmx/vmx.h +++ b/arch/x86/kvm/vmx/vmx.h @@ -484,7 +484,8 @@ static inline u8 vmx_get_rvi(void) VM_ENTRY_LOAD_IA32_EFER | \ VM_ENTRY_LOAD_BNDCFGS | \ VM_ENTRY_PT_CONCEAL_PIP | \ - VM_ENTRY_LOAD_IA32_RTIT_CTL) + VM_ENTRY_LOAD_IA32_RTIT_CTL | \ + VM_ENTRY_LOAD_CET_STATE) =20 #define __KVM_REQUIRED_VMX_VM_EXIT_CONTROLS \ (VM_EXIT_SAVE_DEBUG_CONTROLS | \ @@ -506,7 +507,8 @@ static inline u8 vmx_get_rvi(void) VM_EXIT_LOAD_IA32_EFER | \ VM_EXIT_CLEAR_BNDCFGS | \ VM_EXIT_PT_CONCEAL_PIP | \ - VM_EXIT_CLEAR_IA32_RTIT_CTL) + VM_EXIT_CLEAR_IA32_RTIT_CTL | \ + VM_EXIT_LOAD_CET_STATE) =20 #define KVM_REQUIRED_VMX_PIN_BASED_VM_EXEC_CONTROL \ (PIN_BASED_EXT_INTR_MASK | \ --=20 2.51.0.470.ga7dc726c21-goog