From nobody Mon Feb 9 18:18:33 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass(p=reject dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1622121959; cv=none; d=zohomail.com; s=zohoarc; b=dRKJZjFaAGzBFr9oh7a1JjB9KXUtjVJG18bJfK6c9AbS0YID8sBjmZfVCoJcb0AOJjv/fe3mZzQSQJJESb5OD3NnuhbZE3e+xkS7G0GuVmCBPrNvqO7mzoelM6AHmND3ORs4V2ujvDqpNh0rlwU73/INiARSgh/WlXXhFV0V4H8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1622121959; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=yFuUSMrhL6AOMmssLDB/pX8etPeEthHG1ntkUAEoF84=; b=BDVy2gN95/pOxRpDzhRNrPTVK51YQH90fwL3Dy3Rg0utWPvIc9GSH8/1t7GRse1+3EquS3E5No04oINlGyxmli77Yg86NI84Yc5bhBDaNzorglrnqSMQEjtXYgglbSfv7cHuLlUSGmxBfT6CFkBNdSVYEMpfY5oeSKuG+DhJUrI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of lists.xenproject.org designates 192.237.175.120 as permitted sender) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=pass header.from= (p=reject dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1622121959935955.53343378874; Thu, 27 May 2021 06:25:59 -0700 (PDT) Received: from list by lists.xenproject.org with outflank-mailman.133375.248664 (Exim 4.92) (envelope-from ) id 1lmG1B-0007NE-2n; Thu, 27 May 2021 13:25:45 +0000 Received: by outflank-mailman (output) from mailman id 133375.248664; Thu, 27 May 2021 13:25:45 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmG1A-0007Mu-Uy; Thu, 27 May 2021 13:25:44 +0000 Received: by outflank-mailman (input) for mailman id 133375; Thu, 27 May 2021 13:25:43 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1lmG19-0006Vp-Ai for xen-devel@lists.xenproject.org; Thu, 27 May 2021 13:25:43 +0000 Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id cf710b61-4d57-4556-b0a8-9d504384264d; Thu, 27 May 2021 13:25:34 +0000 (UTC) X-Outflank-Mailman: Message body and most headers restored to incoming version X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: cf710b61-4d57-4556-b0a8-9d504384264d DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1622121933; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=hbSlZ0t4S9qZJpcVYcxxqV9AHueCrBm5p1+6QD2gkF4=; b=csYiggvlHewb61k3daWwtqDdhjw4KFDx9PFeGnnpHI6UfkzHLkd0UPOJ G0mQPlqcEXjzbPBZLxaKjIzWVUWO/fpssFFNyxb87fNQt+h1FcrQBXqFK kqi2XzBBTub5ShVWT4i/yaUoO0E2rn8hy+2ozVNlbq9ZDv3VkXEHRnV55 w=; Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none IronPort-SDR: ctaQJrHedHNfZbi6TXiP7BmYX0i1rU9TXyaIQ/8KlLN8FshKN2kUXpPMnC9bm7KSL/Pxc1gFpL /25S6z2vUk5qBFuqjpppiefS7ySJ6/FWs+0L0DcrRuKsMYY8XqO9nAbHvZHuoJBLq4EGG4p1gr Z6FLRC05zgQLO/7B5sVfZjDRlQOoZF6CFNuOFGTtjpBn5jfNPnfehX7hhfGpH0hNaYsQizx22v JXoJkfzWljgcev3I62BY2uGZhMpeLdSAeCZrvWYVBdafbJHC9vkbPYBn7Vp6J1jfa4fq4wwxnq Ptg= X-SBRS: 5.1 X-MesageID: 44745603 X-Ironport-Server: esa2.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED IronPort-HdrOrdr: A9a23:VsOdxaGa6A9bvP6FpLqE0seALOsnbusQ8zAXP0AYc31om6uj5r iTdZUgpGbJYVkqKRIdcLy7V5VoBEmskaKdgrNhW4tKPjOW2ldARbsKheCJrlHd8m/Fh4lgPM 9bAtND4bbLbWSS4/yV3ODBKadE/OW6 X-IronPort-AV: E=Sophos;i="5.82,334,1613451600"; d="scan'208";a="44745603" From: Andrew Cooper To: Xen-devel CC: Andrew Cooper , Jan Beulich , =?UTF-8?q?Roger=20Pau=20Monn=C3=A9?= , Wei Liu Subject: [PATCH 3/3] x86/tsx: Deprecate vpmu=rtm-abort and use tsx= instead Date: Thu, 27 May 2021 14:25:19 +0100 Message-ID: <20210527132519.21730-4-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20210527132519.21730-1-andrew.cooper3@citrix.com> References: <20210527132519.21730-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @citrix.com) This reuses the rtm_disable infrastructure, so CPUID derivation works prope= rly when TSX is disabled in favour of working PCR3. vpmu=3D is not a supported feature, and having this functionality under tsx= =3D centralises all TSX handling. Signed-off-by: Andrew Cooper Acked-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monn=C3=A9 CC: Wei Liu --- docs/misc/xen-command-line.pandoc | 40 +++++++++++++++--------------- xen/arch/x86/cpu/intel.c | 3 --- xen/arch/x86/cpu/vpmu.c | 4 +-- xen/arch/x86/tsx.c | 51 +++++++++++++++++++++++++++++++++++= ++-- xen/include/asm-x86/vpmu.h | 1 - 5 files changed, 70 insertions(+), 29 deletions(-) diff --git a/docs/misc/xen-command-line.pandoc b/docs/misc/xen-command-line= .pandoc index c32a397a12..a6facc33ea 100644 --- a/docs/misc/xen-command-line.pandoc +++ b/docs/misc/xen-command-line.pandoc @@ -2296,14 +2296,21 @@ pages) must also be specified via the tbuf_size par= ameter. =20 Controls for the use of Transactional Synchronization eXtensions. =20 -On Intel parts released in Q3 2019 (with updated microcode), and future pa= rts, -a control has been introduced which allows TSX to be turned off. +Several microcode updates are relevant: =20 -On systems with the ability to turn TSX off, this boolean offers system wi= de -control of whether TSX is enabled or disabled. + * March 2019, fixing the TSX memory ordering errata on all TSX-enabled CP= Us + to date. Introduced MSR_TSX_FORCE_ABORT on SKL/SKX/KBL/WHL/CFL parts. = The + errata workaround uses Performance Counter 3, so the user can select + between working TSX and working perfcounters. =20 -On parts vulnerable to CVE-2019-11135 / TSX Asynchronous Abort, the follow= ing -logic applies: + * November 2019, fixing the TSX Async Abort speculative vulnerability. + Introduced MSR_TSX_CTRL on all TSX-enabled MDS_NO parts to date, + CLX/WHL-R/CFL-R, with the controls becoming architectural moving forward + and formally retiring HLE from the architecture. The user can disable = TSX + to mitigate TAA, and elect to hide the HLE/RTM CPUID bits. + +On systems with the ability to disable TSX off, this boolean offers system +wide control of whether TSX is enabled or disabled. =20 * An explicit `tsx=3D` choice is honoured, even if it is `true` and would result in a vulnerable system. @@ -2311,10 +2318,14 @@ logic applies: * When no explicit `tsx=3D` choice is given, parts vulnerable to TAA will= be mitigated by disabling TSX, as this is the lowest overhead option. =20 - * If the use of TSX is important, the more expensive TAA mitigations can = be + If the use of TSX is important, the more expensive TAA mitigations can = be opted in to with `smt=3D0 spec-ctrl=3Dmd-clear`, at which point TSX wil= l remain active by default. =20 + * When no explicit `tsx=3D` option is given, parts susceptible to the mem= ory + ordering errata default to `true` to enable working TSX. Alternatively, + selecting `tsx=3D0` will disable TSX and restore PCR3 to a working stat= e. + ### ucode > `=3D List of [ | scan=3D, nmi=3D, allow-same=3D ]` =20 @@ -2456,20 +2467,7 @@ provide access to a wealth of low level processor in= formation. =20 * The `arch` option allows access to the pre-defined architectural event= s. =20 -* The `rtm-abort` boolean controls a trade-off between working Restricted - Transactional Memory, and working performance counters. - - All processors released to date (Q1 2019) supporting Transactional Mem= ory - Extensions suffer an erratum which has been addressed in microcode. - - Processors based on the Skylake microarchitecture with up-to-date - microcode internally use performance counter 3 to work around the erra= tum. - A consequence is that the counter gets reprogrammed whenever an `XBEGI= N` - instruction is executed. - - An alternative mode exists where PCR3 behaves as before, at the cost of - `XBEGIN` unconditionally aborting. Enabling `rtm-abort` mode will - activate this alternative mode. +* The `rtm-abort` boolean has been superseded. Use `tsx=3D0` instead. =20 *Warning:* As the virtualisation is not 100% safe, don't use the vpmu flag on diff --git a/xen/arch/x86/cpu/intel.c b/xen/arch/x86/cpu/intel.c index 37439071d9..abf8e206d7 100644 --- a/xen/arch/x86/cpu/intel.c +++ b/xen/arch/x86/cpu/intel.c @@ -356,9 +356,6 @@ static void Intel_errata_workarounds(struct cpuinfo_x86= *c) (c->x86_model =3D=3D 29 || c->x86_model =3D=3D 46 || c->x86_model =3D= =3D 47)) __set_bit(X86_FEATURE_CLFLUSH_MONITOR, c->x86_capability); =20 - if (cpu_has_tsx_force_abort && opt_rtm_abort) - wrmsrl(MSR_TSX_FORCE_ABORT, TSX_FORCE_ABORT_RTM); - probe_c3_errata(c); } =20 diff --git a/xen/arch/x86/cpu/vpmu.c b/xen/arch/x86/cpu/vpmu.c index d8659c63f8..16e91a3694 100644 --- a/xen/arch/x86/cpu/vpmu.c +++ b/xen/arch/x86/cpu/vpmu.c @@ -49,7 +49,6 @@ CHECK_pmu_params; static unsigned int __read_mostly opt_vpmu_enabled; unsigned int __read_mostly vpmu_mode =3D XENPMU_MODE_OFF; unsigned int __read_mostly vpmu_features =3D 0; -bool __read_mostly opt_rtm_abort; =20 static DEFINE_SPINLOCK(vpmu_lock); static unsigned vpmu_count; @@ -79,7 +78,8 @@ static int __init parse_vpmu_params(const char *s) else if ( !cmdline_strcmp(s, "arch") ) vpmu_features |=3D XENPMU_FEATURE_ARCH_ONLY; else if ( (val =3D parse_boolean("rtm-abort", s, ss)) >=3D 0 ) - opt_rtm_abort =3D val; + printk(XENLOG_WARNING + "'rtm-abort=3D' superseded. Use 'tsx=3D' i= nstead\n"); else rc =3D -EINVAL; =20 diff --git a/xen/arch/x86/tsx.c b/xen/arch/x86/tsx.c index 98ecb71a4a..338191df7f 100644 --- a/xen/arch/x86/tsx.c +++ b/xen/arch/x86/tsx.c @@ -6,7 +6,9 @@ * Valid values: * 1 =3D> Explicit tsx=3D1 * 0 =3D> Explicit tsx=3D0 - * -1 =3D> Default, implicit tsx=3D1, may change to 0 to mitigate TAA + * -1 =3D> Default, altered to 0/1 (if unspecified) by: + * - TAA heuristics/settings for speculative safety + * - "TSX vs PCR3" select for TSX memory ordering safety * -3 =3D> Implicit tsx=3D1 (feed-through from spec-ctrl=3D0) * * This is arranged such that the bottom bit encodes whether TSX is actual= ly @@ -50,6 +52,26 @@ void tsx_init(void) =20 cpu_has_tsx_ctrl =3D !!(caps & ARCH_CAPS_TSX_CTRL); =20 + if ( cpu_has_tsx_force_abort ) + { + /* + * On an early TSX-enable Skylake part subject to the memory + * ordering erratum, with at least the March 2019 microcode. + */ + + /* + * If no explicit tsx=3D option is provided, pick a default. + * + * This deliberately overrides the implicit opt_tsx=3D-3 from + * `spec-ctrl=3D0` because: + * - parse_spec_ctrl() ran before any CPU details where know. + * - We now know we're running on a CPU not affected by TAA (as + * TSX_FORCE_ABORT is enumerated). + */ + if ( opt_tsx < 0 ) + opt_tsx =3D 1; + } + /* * The TSX features (HLE/RTM) are handled specially. They both * enumerate features but, on certain parts, have mechanisms to be @@ -75,6 +97,12 @@ void tsx_init(void) } } =20 + /* + * Note: MSR_TSX_CTRL is enumerated on TSX-enabled MDS_NO and later pa= rts. + * MSR_TSX_FORCE_ABORT is enumerated on TSX-enabled pre-MDS_NO Skylake + * parts only. The two features are on a disjoint set of CPUs, and not + * offered to guests by hypervisors. + */ if ( cpu_has_tsx_ctrl ) { uint32_t hi, lo; @@ -90,9 +118,28 @@ void tsx_init(void) =20 wrmsr(MSR_TSX_CTRL, lo, hi); } + else if ( cpu_has_tsx_force_abort ) + { + /* + * On an early TSX-enable Skylake part subject to the memory order= ing + * erratum, with at least the March 2019 microcode. + */ + uint32_t hi, lo; + + rdmsr(MSR_TSX_FORCE_ABORT, lo, hi); + + /* Check bottom bit only. Higher bits are various sentinels. */ + rtm_disabled =3D !(opt_tsx & 1); + + lo &=3D ~TSX_FORCE_ABORT_RTM; + if ( rtm_disabled ) + lo |=3D TSX_FORCE_ABORT_RTM; + + wrmsr(MSR_TSX_FORCE_ABORT, lo, hi); + } else if ( opt_tsx >=3D 0 ) printk_once(XENLOG_WARNING - "MSR_TSX_CTRL not available - Ignoring tsx=3D setting\= n"); + "TSX controls not available - Ignoring tsx=3D setting\= n"); } =20 /* diff --git a/xen/include/asm-x86/vpmu.h b/xen/include/asm-x86/vpmu.h index 55f85ba00f..4b0a6ba3da 100644 --- a/xen/include/asm-x86/vpmu.h +++ b/xen/include/asm-x86/vpmu.h @@ -126,7 +126,6 @@ static inline int vpmu_do_rdmsr(unsigned int msr, uint6= 4_t *msr_content) =20 extern unsigned int vpmu_mode; extern unsigned int vpmu_features; -extern bool opt_rtm_abort; =20 /* Context switch */ static inline void vpmu_switch_from(struct vcpu *prev) --=20 2.11.0