From nobody Mon Feb 9 11:32:33 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1622121948; cv=none; d=zohomail.com; s=zohoarc; b=BePec/FZZjGG3sBsd49yCbrWB6PEoJ2toVAf1L/7ZhCrgDM+FauiaUeCKizUv6g5Ij5I4Fsx43pW5eX4BXa3nvIKmurumW0iQJ0qehiq2QCOpDgMSbxpRW8dOzwCHllEKzNYlpVOPlwjfRDF1AKMD2B/bwUah7YwMAxqcH9YTJc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1622121948; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=32u6NevcWXrp2rAJe4kgTeUnpLM4zy2jsJ/zhedOyfw=; b=CpJmXjdnoKDtq8OBGVgMSpxx18OSgVzz5qlzF1VeYQ17sMoDplhnSNKnoO/2UvxYtGeUayZxVd59koSPzyetpiAFiNhJcILaoSXSK+Mf27+HeO6GA2FX6fw+l66Y1ASdmKc8vDSJNPUUfYiTDFjMIYKy2AufyTIgLxASH5z5JNw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1622121948285231.25268424092326; Thu, 27 May 2021 06:25:48 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.133372.248642 (Exim 4.92) (envelope-from ) id 1lmG11-0006YF-AZ; Thu, 27 May 2021 13:25:35 +0000 Received: by outflank-mailman (output) from mailman id 133372.248642; Thu, 27 May 2021 13:25:35 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmG11-0006Y4-79; Thu, 27 May 2021 13:25:35 +0000 Received: by outflank-mailman (input) for mailman id 133372; Thu, 27 May 2021 13:25:33 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmG0z-0006Vp-Bx for xen-devel@lists.xenproject.org; Thu, 27 May 2021 13:25:33 +0000 Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id d5590923-66fe-478c-957a-1b070d481308; Thu, 27 May 2021 13:25:31 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d5590923-66fe-478c-957a-1b070d481308 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1622121931; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=m9QjH0cMMuHCQVPpDVT4R/qor7cqmBRJec5W9WeQMWI=; b=CCvrJWHEFqL/KPm9u7kpq+Uv9f3MXekSOuzCymZoEeV3n21th/xziL9C GfV3bC8fwZvyA3+LPM9+Mu53rWCLBPwDaZGdwjfyR0a3P15xr5EPc7HlZ 0xVC8JFrLiccPLjEXhElrJ0GrUIcBqt3/LRjCS0CIdpT8XIFLja8faFYr o=; Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none IronPort-SDR: RPC8Wl/AUgpsC6I+ScZyKvRYyEfw9QAESKL4QptfhG5ddSJOw/WTQQdTUsQxCkrGLfc4kbJblf v4bBHOObommyCNSwlKbezknANoDRFYsmrV9NQbLMUW+Zp3YAMhqGDa3ZqpZdNvy4KG7gQ3Ov6T YM2Bk2tXhOfGa8D6X4IddPhhc0vh18AbRt4kv3TjDYhIm+t9dwl1R2d8d0P5azv7CxmCbWlLhN 6pFViN5H6FpMm9HXB2yKQk88jwwV9VJcJtMN+5Jtkik2cy4NjY3CKL8JL6qXOUmXa6gHIuczce K5w= X-SBRS: 5.1 X-MesageID: 44745591 X-Ironport-Server: esa2.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED IronPort-HdrOrdr: A9a23:s3Gyoq/gv3zs8gQ3zrZuk+DgI+orL9Y04lQ7vn2YSXRuHPBw8P re5cjztCWE7gr5N0tBpTntAsW9qDbnhPtICOoqTNCftWvdyQiVxehZhOOIqVDd8m/Fh4pgPM 9bAtBD4bbLbGSS4/yU3ODBKadD/OW6 X-IronPort-AV: E=Sophos;i="5.82,334,1613451600"; d="scan'208";a="44745591" From: Andrew Cooper To: Xen-devel CC: Andrew Cooper , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Wei Liu Subject: [PATCH 1/3] x86/cpuid: Rework HLE and RTM handling Date: Thu, 27 May 2021 14:25:17 +0100 Message-ID: <20210527132519.21730-2-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20210527132519.21730-1-andrew.cooper3@citrix.com> References: <20210527132519.21730-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) The TAA mitigation offered the option to hide the HLE and RTM CPUID bits, which has caused some migration compatibility problems. These two bits are special. Annotate them with ! to emphasise this point. Hardware Lock Elision (HLE) may or may not be visible in CPUID, but is disabled in microcode on all CPUs, and has been removed from the architectu= re. Do not advertise it to VMs by default. Restricted Transactional Memory (RTM) may or may not be visible in CPUID, a= nd may or may not be configured in force-abort mode. Have tsx_init() note whether RTM has been configured into force-abort mode, so guest_common_feature_adjustments() can conditionally hide it from VMs by default. The host policy values for HLE/RTM may or may not be set, depending on any previous running kernel's choice of visibility, and Xen's choice. TSX is available on any CPU which enumerates a TSX-hiding mechanism, so instead of doing a two-step to clobber any hiding, scan CPUID, then set the visibility, just force visibility of the bits in the first place. With the HLE/RTM bits now unilaterally visible in the host policy, xc_cpuid_apply_policy() can construct a more appropriate policy out of thin air for pre-4.13 VMs with no CPUID data in their migration stream, and specifically one where HLE/RTM doesn't potentially disappear behind the back of a running VM. Fixes: 8c4330818f6 ("x86/spec-ctrl: Mitigate the TSX Asynchronous Abort sid= echannel") Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich Reviewed-by: Roger Pau Monn=C3=A9 --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 CC: Wei Liu --- tools/libs/guest/xg_cpuid_x86.c | 2 ++ xen/arch/x86/cpuid.c | 24 ++++++++++------------ xen/arch/x86/spec_ctrl.c | 3 --- xen/arch/x86/tsx.c | 31 +++++++++++++++++++++++++= ++-- xen/include/asm-x86/processor.h | 1 + xen/include/public/arch-x86/cpufeatureset.h | 4 ++-- 6 files changed, 44 insertions(+), 21 deletions(-) diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x8= 6.c index 1ebc108213..ec5a47fde4 100644 --- a/tools/libs/guest/xg_cpuid_x86.c +++ b/tools/libs/guest/xg_cpuid_x86.c @@ -511,6 +511,8 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t d= omid, bool restore, * so migrated-in VM's don't risk seeing features disappearing. */ p->basic.rdrand =3D test_bit(X86_FEATURE_RDRAND, host_featureset); + p->feat.hle =3D test_bit(X86_FEATURE_HLE, host_featureset); + p->feat.rtm =3D test_bit(X86_FEATURE_RTM, host_featureset); =20 if ( di.hvm ) { diff --git a/xen/arch/x86/cpuid.c b/xen/arch/x86/cpuid.c index 752bf244ea..55a7b16342 100644 --- a/xen/arch/x86/cpuid.c +++ b/xen/arch/x86/cpuid.c @@ -375,6 +375,16 @@ static void __init guest_common_default_feature_adjust= ments(uint32_t *fs) boot_cpu_data.x86 =3D=3D 6 && boot_cpu_data.x86_model =3D=3D 0x3a= && cpu_has_rdrand && !is_forced_cpu_cap(X86_FEATURE_RDRAND) ) __clear_bit(X86_FEATURE_RDRAND, fs); + + /* + * On certain hardware, speculative or errata workarounds can result in + * TSX being placed in "force-abort" mode, where it doesn't actually + * function as expected, but is technically compatible with the ISA. + * + * Do not advertise RTM to guests by default if it won't actually work. + */ + if ( rtm_disabled ) + __clear_bit(X86_FEATURE_RTM, fs); } =20 static void __init guest_common_feature_adjustments(uint32_t *fs) @@ -652,20 +662,6 @@ void recalculate_cpuid_policy(struct domain *d) __clear_bit(X86_FEATURE_SYSCALL, max_fs); } =20 - /* - * On hardware with MSR_TSX_CTRL, the admin may have elected to disable - * TSX and hide the feature bits. Migrating-in VMs may have been boot= ed - * pre-mitigation when the TSX features were visbile. - * - * This situation is compatible (albeit with a perf hit to any TSX cod= e in - * the guest), so allow the feature bits to remain set. - */ - if ( cpu_has_tsx_ctrl ) - { - __set_bit(X86_FEATURE_HLE, max_fs); - __set_bit(X86_FEATURE_RTM, max_fs); - } - /* Clamp the toolstacks choices to reality. */ for ( i =3D 0; i < ARRAY_SIZE(fs); i++ ) fs[i] &=3D max_fs[i]; diff --git a/xen/arch/x86/spec_ctrl.c b/xen/arch/x86/spec_ctrl.c index cd05f42394..f2782b2d55 100644 --- a/xen/arch/x86/spec_ctrl.c +++ b/xen/arch/x86/spec_ctrl.c @@ -1158,9 +1158,6 @@ void __init init_speculation_mitigations(void) ((hw_smt_enabled && opt_smt) || !boot_cpu_has(X86_FEATURE_SC_VERW_IDLE)) ) { - setup_clear_cpu_cap(X86_FEATURE_HLE); - setup_clear_cpu_cap(X86_FEATURE_RTM); - opt_tsx =3D 0; tsx_init(); } diff --git a/xen/arch/x86/tsx.c b/xen/arch/x86/tsx.c index 39e483640a..e09e819dce 100644 --- a/xen/arch/x86/tsx.c +++ b/xen/arch/x86/tsx.c @@ -15,6 +15,7 @@ */ int8_t __read_mostly opt_tsx =3D -1; int8_t __read_mostly cpu_has_tsx_ctrl =3D -1; +bool __read_mostly rtm_disabled; =20 static int __init parse_tsx(const char *s) { @@ -45,6 +46,30 @@ void tsx_init(void) rdmsrl(MSR_ARCH_CAPABILITIES, caps); =20 cpu_has_tsx_ctrl =3D !!(caps & ARCH_CAPS_TSX_CTRL); + + /* + * The TSX features (HLE/RTM) are handled specially. They both + * enumerate features but, on certain parts, have mechanisms to be + * hidden without disrupting running software. + * + * At the moment, we're running in an unknown context (WRT hiding - + * particularly if another fully fledged kernel ran before us) and + * depending on user settings, may elect to continue hiding them f= rom + * native CPUID instructions. + * + * Xen doesn't use TSX itself, but use cpu_has_{hle,rtm} for vario= us + * system reasons, mostly errata detection, so the meaning is more + * useful as "TSX infrastructure available", as opposed to "featur= es + * advertised and working". + * + * Force the features to be visible in Xen's view if we see any of= the + * infrastructure capable of hiding them. + */ + if ( cpu_has_tsx_ctrl ) + { + setup_force_cpu_cap(X86_FEATURE_HLE); + setup_force_cpu_cap(X86_FEATURE_RTM); + } } =20 if ( cpu_has_tsx_ctrl ) @@ -53,9 +78,11 @@ void tsx_init(void) =20 rdmsrl(MSR_TSX_CTRL, val); =20 + /* Check bottom bit only. Higher bits are various sentinels. */ + rtm_disabled =3D !(opt_tsx & 1); + val &=3D ~(TSX_CTRL_RTM_DISABLE | TSX_CTRL_CPUID_CLEAR); - /* Check bottom bit only. Higher bits are various sentinals. */ - if ( !(opt_tsx & 1) ) + if ( rtm_disabled ) val |=3D TSX_CTRL_RTM_DISABLE | TSX_CTRL_CPUID_CLEAR; =20 wrmsrl(MSR_TSX_CTRL, val); diff --git a/xen/include/asm-x86/processor.h b/xen/include/asm-x86/processo= r.h index 83143d4df8..bc4dc69253 100644 --- a/xen/include/asm-x86/processor.h +++ b/xen/include/asm-x86/processor.h @@ -627,6 +627,7 @@ static inline uint8_t get_cpu_family(uint32_t raw, uint= 8_t *model, } =20 extern int8_t opt_tsx, cpu_has_tsx_ctrl; +extern bool rtm_disabled; void tsx_init(void); =20 enum ap_boot_method { diff --git a/xen/include/public/arch-x86/cpufeatureset.h b/xen/include/publ= ic/arch-x86/cpufeatureset.h index c42f56bdd4..b65af42436 100644 --- a/xen/include/public/arch-x86/cpufeatureset.h +++ b/xen/include/public/arch-x86/cpufeatureset.h @@ -197,14 +197,14 @@ XEN_CPUFEATURE(FSGSBASE, 5*32+ 0) /*A {RD,WR}{F= S,GS}BASE instructions */ XEN_CPUFEATURE(TSC_ADJUST, 5*32+ 1) /*S TSC_ADJUST MSR available */ XEN_CPUFEATURE(SGX, 5*32+ 2) /* Software Guard extensions */ XEN_CPUFEATURE(BMI1, 5*32+ 3) /*A 1st bit manipulation extension= s */ -XEN_CPUFEATURE(HLE, 5*32+ 4) /*A Hardware Lock Elision */ +XEN_CPUFEATURE(HLE, 5*32+ 4) /*!a Hardware Lock Elision */ XEN_CPUFEATURE(AVX2, 5*32+ 5) /*A AVX2 instructions */ XEN_CPUFEATURE(FDP_EXCP_ONLY, 5*32+ 6) /*! x87 FDP only updated on except= ion. */ XEN_CPUFEATURE(SMEP, 5*32+ 7) /*S Supervisor Mode Execution Prot= ection */ XEN_CPUFEATURE(BMI2, 5*32+ 8) /*A 2nd bit manipulation extension= s */ XEN_CPUFEATURE(ERMS, 5*32+ 9) /*A Enhanced REP MOVSB/STOSB */ XEN_CPUFEATURE(INVPCID, 5*32+10) /*H Invalidate Process Context ID = */ -XEN_CPUFEATURE(RTM, 5*32+11) /*A Restricted Transactional Memor= y */ +XEN_CPUFEATURE(RTM, 5*32+11) /*!A Restricted Transactional Memor= y */ XEN_CPUFEATURE(PQM, 5*32+12) /* Platform QoS Monitoring */ XEN_CPUFEATURE(NO_FPU_SEL, 5*32+13) /*! FPU CS/DS stored as zero */ XEN_CPUFEATURE(MPX, 5*32+14) /*s Memory Protection Extensions */ --=20 2.11.0