[PATCH] x86/cpu-policy: Advertise MSR_ARCH_CAPS to guests by default

Andrew Cooper posted 1 patch 9 months, 1 week ago
Patches applied successfully (tree, apply log)
git fetch https://gitlab.com/xen-project/patchew/xen tags/patchew/20230725213611.2405811-1-andrew.cooper3@citrix.com
There is a newer version of this series
xen/arch/x86/cpu-policy.c                   | 11 -----------
xen/include/public/arch-x86/cpufeatureset.h |  2 +-
2 files changed, 1 insertion(+), 12 deletions(-)
[PATCH] x86/cpu-policy: Advertise MSR_ARCH_CAPS to guests by default
Posted by Andrew Cooper 9 months, 1 week ago
With xl/libxl now able to control the policy bits for MSR_ARCH_CAPS, it is
safe to advertise to guests by default.  In turn, we don't need the special
case to expose details to dom0.

This advertises MSR_ARCH_CAPS to guests on *all* Intel hardware, even if the
register content ends up being empty.  This is necessary in order to safely
level two hosts which cross the Broadwell/Skylake divide.

On Cascade Lake and later hardware, guests can now see RDCL_NO (not vulnerable
to Meltdown) amongst others.  This causes substantial performance
improvements, as guests are no longer applying software mitigations in cases
where they don't need to.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>

Not to go in before Roger's libxl changes...
---
 xen/arch/x86/cpu-policy.c                   | 11 -----------
 xen/include/public/arch-x86/cpufeatureset.h |  2 +-
 2 files changed, 1 insertion(+), 12 deletions(-)

diff --git a/xen/arch/x86/cpu-policy.c b/xen/arch/x86/cpu-policy.c
index f40eeb8be8dc..1f954d4e5940 100644
--- a/xen/arch/x86/cpu-policy.c
+++ b/xen/arch/x86/cpu-policy.c
@@ -888,17 +888,6 @@ void __init init_dom0_cpuid_policy(struct domain *d)
     if ( cpu_has_itsc )
         p->extd.itsc = true;
 
-    /*
-     * Expose the "hardware speculation behaviour" bits of ARCH_CAPS to dom0,
-     * so dom0 can turn off workarounds as appropriate.  Temporary, until the
-     * domain policy logic gains a better understanding of MSRs.
-     */
-    if ( is_hardware_domain(d) && cpu_has_arch_caps )
-    {
-        p->feat.arch_caps = true;
-        p->arch_caps.raw = host_cpu_policy.arch_caps.raw;
-    }
-
     /* Apply dom0-cpuid= command line settings, if provided. */
     if ( dom0_cpuid_cmdline )
     {
diff --git a/xen/include/public/arch-x86/cpufeatureset.h b/xen/include/public/arch-x86/cpufeatureset.h
index ce7407d6a10c..6d20810cb9d1 100644
--- a/xen/include/public/arch-x86/cpufeatureset.h
+++ b/xen/include/public/arch-x86/cpufeatureset.h
@@ -271,7 +271,7 @@ XEN_CPUFEATURE(AVX512_FP16,   9*32+23) /*A  AVX512 FP16 instructions */
 XEN_CPUFEATURE(IBRSB,         9*32+26) /*A  IBRS and IBPB support (used by Intel) */
 XEN_CPUFEATURE(STIBP,         9*32+27) /*A  STIBP */
 XEN_CPUFEATURE(L1D_FLUSH,     9*32+28) /*S  MSR_FLUSH_CMD and L1D flush. */
-XEN_CPUFEATURE(ARCH_CAPS,     9*32+29) /*!a IA32_ARCH_CAPABILITIES MSR */
+XEN_CPUFEATURE(ARCH_CAPS,     9*32+29) /*!A IA32_ARCH_CAPABILITIES MSR */
 XEN_CPUFEATURE(CORE_CAPS,     9*32+30) /*   IA32_CORE_CAPABILITIES MSR */
 XEN_CPUFEATURE(SSBD,          9*32+31) /*A  MSR_SPEC_CTRL.SSBD available */
 
-- 
2.30.2


Re: [PATCH] x86/cpu-policy: Advertise MSR_ARCH_CAPS to guests by default
Posted by Jan Beulich 9 months, 1 week ago
On 25.07.2023 23:36, Andrew Cooper wrote:
> With xl/libxl now able to control the policy bits for MSR_ARCH_CAPS, it is
> safe to advertise to guests by default.  In turn, we don't need the special
> case to expose details to dom0.
> 
> This advertises MSR_ARCH_CAPS to guests on *all* Intel hardware, even if the
> register content ends up being empty.  This is necessary in order to safely
> level two hosts which cross the Broadwell/Skylake divide.

Could you expand on this, so one won't have to dig out details from
elsewhere?

> On Cascade Lake and later hardware, guests can now see RDCL_NO (not vulnerable
> to Meltdown) amongst others.  This causes substantial performance
> improvements, as guests are no longer applying software mitigations in cases
> where they don't need to.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Roger Pau Monné <roger.pau@citrix.com>
> CC: Wei Liu <wl@xen.org>
> 
> Not to go in before Roger's libxl changes...

I was about to comment on the "now able to", when I saw this.

Preferably with a little bit more detail added to the description
Reviewed-by: Jan Beulich <jbeulich@suse.com>

Jan

[PATCH v2] x86/cpu-policy: Advertise MSR_ARCH_CAPS to guests by default
Posted by Andrew Cooper 9 months, 1 week ago
With xl/libxl now able to control the policy bits for MSR_ARCH_CAPS, it is
safe to advertise to guests by default.  In turn, we don't need the special
case to expose details to dom0.

This advertises MSR_ARCH_CAPS to guests on *all* Intel hardware, even if the
register content ends up being empty.

  - Advertising ARCH_CAPS and not RSBA signals "retpoline is safe here and
    everywhere you might migrate to".  This is important because it avoids the
    guest kernel needing to rely on model checks.

  - Alternatively, levelling for safety across the Broadwell/Skylake divide
    requires advertising ARCH_CAPS and RSBA, meaning "retpoline not safe on
    some hardware you might migrate to".

On Cascade Lake and later hardware, guests can now see RDCL_NO (not vulnerable
to Meltdown) amongst others.  This causes substantial performance
improvements, as guests are no longer applying software mitigations in cases
where they don't need to.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
Reviewed-by: Jan Beulich <jbeulich@suse.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Wei Liu <wl@xen.org>

Not to go in before Roger's libxl changes...

v2:
 * Extend the commit message.
---
 xen/arch/x86/cpu-policy.c                   | 11 -----------
 xen/include/public/arch-x86/cpufeatureset.h |  2 +-
 2 files changed, 1 insertion(+), 12 deletions(-)

diff --git a/xen/arch/x86/cpu-policy.c b/xen/arch/x86/cpu-policy.c
index f40eeb8be8dc..1f954d4e5940 100644
--- a/xen/arch/x86/cpu-policy.c
+++ b/xen/arch/x86/cpu-policy.c
@@ -888,17 +888,6 @@ void __init init_dom0_cpuid_policy(struct domain *d)
     if ( cpu_has_itsc )
         p->extd.itsc = true;
 
-    /*
-     * Expose the "hardware speculation behaviour" bits of ARCH_CAPS to dom0,
-     * so dom0 can turn off workarounds as appropriate.  Temporary, until the
-     * domain policy logic gains a better understanding of MSRs.
-     */
-    if ( is_hardware_domain(d) && cpu_has_arch_caps )
-    {
-        p->feat.arch_caps = true;
-        p->arch_caps.raw = host_cpu_policy.arch_caps.raw;
-    }
-
     /* Apply dom0-cpuid= command line settings, if provided. */
     if ( dom0_cpuid_cmdline )
     {
diff --git a/xen/include/public/arch-x86/cpufeatureset.h b/xen/include/public/arch-x86/cpufeatureset.h
index ce7407d6a10c..6d20810cb9d1 100644
--- a/xen/include/public/arch-x86/cpufeatureset.h
+++ b/xen/include/public/arch-x86/cpufeatureset.h
@@ -271,7 +271,7 @@ XEN_CPUFEATURE(AVX512_FP16,   9*32+23) /*A  AVX512 FP16 instructions */
 XEN_CPUFEATURE(IBRSB,         9*32+26) /*A  IBRS and IBPB support (used by Intel) */
 XEN_CPUFEATURE(STIBP,         9*32+27) /*A  STIBP */
 XEN_CPUFEATURE(L1D_FLUSH,     9*32+28) /*S  MSR_FLUSH_CMD and L1D flush. */
-XEN_CPUFEATURE(ARCH_CAPS,     9*32+29) /*!a IA32_ARCH_CAPABILITIES MSR */
+XEN_CPUFEATURE(ARCH_CAPS,     9*32+29) /*!A IA32_ARCH_CAPABILITIES MSR */
 XEN_CPUFEATURE(CORE_CAPS,     9*32+30) /*   IA32_CORE_CAPABILITIES MSR */
 XEN_CPUFEATURE(SSBD,          9*32+31) /*A  MSR_SPEC_CTRL.SSBD available */
 
-- 
2.30.2