[RFC 00/10] i386/cpu: Cache CPUID fixup, Intel cache model & topo CPUID enhencement

Zhao Liu posted 10 patches 6 months, 3 weeks ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20250423114702.1529340-1-zhao1.liu@intel.com
Maintainers: Paolo Bonzini <pbonzini@redhat.com>, Zhao Liu <zhao1.liu@intel.com>, Marcelo Tosatti <mtosatti@redhat.com>
target/i386/cpu.c     | 346 ++++++++++++++++++++++++++++++++++++++++--
target/i386/cpu.h     |   9 ++
target/i386/kvm/kvm.c |   2 +-
3 files changed, 343 insertions(+), 14 deletions(-)
[RFC 00/10] i386/cpu: Cache CPUID fixup, Intel cache model & topo CPUID enhencement
Posted by Zhao Liu 6 months, 3 weeks ago
Hi all,

(Since patches 1 and 2 involve changes to x86 vendors other than Intel,
I have also cc'd friends from AMD and Zhaoxin.)

These are the ones I was going to clean up a long time ago:
 * Fixup CPUID 0x80000005 & 0x80000006 for Intel (and Zhaoxin now).
 * Add cache model for Intel CPUs.
 * Enable 0x1f CPUID leaf for specific Intel CPUs, which already have
   this leaf on host by default.

Overall, the enhancements to the Intel CPU models are still based on
feedback received over time, for a long time...

I'll introduce my changes one by one in the order of importance as I
see it. (The doc update is missing in this version.)


Intel Cache Model
=================

AMD has supports cache model for a long time. And this feature strats
from the Eduardo's idea [1].

Unfortunately, Intel does not support this, and I have received some
feedback (from Tejus on mail list [2] and kvm forum, and from Jason).

Additionally, after clearly defining the cache topology for QEMU's
cache model, outdated cache models can easily raise more questions. For
example, the default legacy cache model's L3 is per die, but SPR's
real L3 is per socket. Users may question how the L3 topology changes
when multiple dies are created (discussed with Daniel on [3]).

So, in this series, I have added cache models for SRF, GNR, and SPR
(because these are the only machines I can find at the moment :-) ).

Note that the cache models are based on the Scalable Performance (SP)
version, and the Xeon Advanced Performance (AP) version may have
different cache sizes. However, SP is sufficient as the default cache
model baseline. In the future, I will consider adding additional
parameters in "smp-cache" to adjust cache sizes to meet different needs.

[1]: https://lore.kernel.org/qemu-devel/20180320175427.GU3417@localhost.localdomain/
[2]: https://lore.kernel.org/qemu-devel/6766AC1F-96D1-41F0-AAEB-CE4158662A51@nutanix.com/
[3]: https://lore.kernel.org/qemu-devel/ZkTrsDdyGRFzVULG@redhat.com/

0x1f CPUID by default (for some CPUs)
=====================================

Once the cache model can be clearly defined, another issue is the
topology.

Currently, the cache topology is actually tied to the CPU topology.
However, in recent Intel CPUs (from cascadelake-AP - 2nd xeon [4]),
CPU topology information is primarily expressed using the 0x1f leaf.

Due to compatibility issues and historical reasons, the Guest's 0x1f
is not unconditionally exposed.

The discrepancy between having 0x1f on the Host but not on the Guest
does indeed cause problems (Manish mentioned in [5]).

Manish and Xiaoyao (for TDX) both attempted to enable 0x1f by default
for Intel CPUs [6] [7], but following Igor's suggestion, it is more
appropriate to enable it by default only for certain CPU models [8]. 

So, as I update the CPU model at this time, I think it's time to revisit
the community's idea (referencing patch 7, where I "took the liberty" to
merge the property-related work pieces from Manish and Xiaoyao, based on
a TDX patch from Xiaoyao [9]).

I enable the 0x1f leaf for SRF, GNR and SPR by default for better
emulation of real silicons.

[4]: https://lore.kernel.org/qemu-devel/ZpoWskY4XE%2F98jss@intel.com/
[5]: https://lore.kernel.org/qemu-devel/PH0PR02MB738410511BF51B12DB09BE6CF6AC2@PH0PR02MB7384.namprd02.prod.outlook.com/
[6]: https://lore.kernel.org/qemu-devel/20240722101859.47408-1-manish.mishra@nutanix.com/
[7]: https://lore.kernel.org/qemu-devel/20240813033145.279307-1-xiaoyao.li@intel.com/
[8]: https://lore.kernel.org/qemu-devel/20240723170321.0ef780c5@imammedo.users.ipa.redhat.com/
[9]: https://lore.kernel.org/qemu-devel/20250401130205.2198253-34-xiaoyao.li@intel.com/


CPUID 0x80000005 & 0x80000006 Fix
=================================

CPUID[0x80000005] is reserved for Intel, and Intel only supports
CPUID[0x80000006].ECX. And becuase AMD requires lines_per_tag to be not
0, which blocks Intel's new cache model.

Therefore, fix these 2 leaves for Intel (and Zhaoxin - which follows
Intel's SDM).

Thanks and Best Regards,
Zhao
---
Manish Mishra (1):
  i386/cpu: Add a "cpuid-0x1f" property

Xiaoyao Li (1):
  i386/cpu: Introduce enable_cpuid_0x1f to force exposing CPUID 0x1f

Zhao Liu (8):
  i386/cpu: Mark CPUID[0x80000005] as reserved for Intel
  i386/cpu: Fix CPUID[0x80000006] for Intel CPU
  i386/cpu: Introduce cache model for SierraForest
  i386/cpu: Introduce cache model for GraniteRapids
  i386/cpu: Introduce cache model for SapphireRapids
  i386/cpu: Enable 0x1f leaf for SierraForest by default
  i386/cpu: Enable 0x1f leaf for GraniteRapids by default
  i386/cpu: Enable 0x1f leaf for SapphireRapids by default

 target/i386/cpu.c     | 346 ++++++++++++++++++++++++++++++++++++++++--
 target/i386/cpu.h     |   9 ++
 target/i386/kvm/kvm.c |   2 +-
 3 files changed, 343 insertions(+), 14 deletions(-)

-- 
2.34.1
Re: [RFC 00/10] i386/cpu: Cache CPUID fixup, Intel cache model & topo CPUID enhencement
Posted by Zhao Liu 6 months, 3 weeks ago
On Wed, Apr 23, 2025 at 07:46:52PM +0800, Zhao Liu wrote:
> Date: Wed, 23 Apr 2025 19:46:52 +0800
> From: Zhao Liu <zhao1.liu@intel.com>
> Subject: [RFC 00/10] i386/cpu: Cache CPUID fixup, Intel cache model & topo
>  CPUID enhencement
> X-Mailer: git-send-email 2.34.1
> 
> Hi all,
> 
> (Since patches 1 and 2 involve changes to x86 vendors other than Intel,
> I have also cc'd friends from AMD and Zhaoxin.)
> 
> These are the ones I was going to clean up a long time ago:
>  * Fixup CPUID 0x80000005 & 0x80000006 for Intel (and Zhaoxin now).
>  * Add cache model for Intel CPUs.
>  * Enable 0x1f CPUID leaf for specific Intel CPUs, which already have
>    this leaf on host by default.
> 
> Overall, the enhancements to the Intel CPU models are still based on
> feedback received over time, for a long time...
> 
> I'll introduce my changes one by one in the order of importance as I
> see it. (The doc update is missing in this version.)
> 
> 
> Intel Cache Model
> =================
> 
> AMD has supports cache model for a long time. And this feature strats
> from the Eduardo's idea [1].
> 
> Unfortunately, Intel does not support this, and I have received some
> feedback (from Tejus on mail list [2] and kvm forum, and from Jason).

I need to add more background:

the legacy "host-cache-info" is becoming failing... On SRF, we have
observed that it cannot accurately identify cache topology, so we have
to use "smp-cache" to set the cache topology.

However, once "host-cache-info" is disabled, we lose the cache info
that matches the real silicon... Therefore, we can only add the cache
model for the named CPU model.
Re: [RFC 00/10] i386/cpu: Cache CPUID fixup, Intel cache model & topo CPUID enhencement
Posted by Ewan Hai 5 months, 3 weeks ago

On 4/23/25 7:46 PM, Zhao Liu wrote:
> Hi all,
> 
> (Since patches 1 and 2 involve changes to x86 vendors other than Intel,
> I have also cc'd friends from AMD and Zhaoxin.)
> 
> These are the ones I was going to clean up a long time ago:
>   * Fixup CPUID 0x80000005 & 0x80000006 for Intel (and Zhaoxin now).
>   * Add cache model for Intel CPUs.
>   * Enable 0x1f CPUID leaf for specific Intel CPUs, which already have
>     this leaf on host by default.

If you run into vendor specific branches while refactoring the topology-related 
code, please feel free to treat Intel and Zhaoxin as one class. For every 
topology CPUID leaf(0x0B, 0x1F, ...) so far, Zhaoxin has followed the Intel SDM 
definition exactly.
Re: [RFC 00/10] i386/cpu: Cache CPUID fixup, Intel cache model & topo CPUID enhencement
Posted by Zhao Liu 5 months, 3 weeks ago
On Mon, May 26, 2025 at 06:52:41PM +0800, Ewan Hai wrote:
> Date: Mon, 26 May 2025 18:52:41 +0800
> From: Ewan Hai <ewanhai-oc@zhaoxin.com>
> Subject: Re: [RFC 00/10] i386/cpu: Cache CPUID fixup, Intel cache model &
>  topo CPUID enhencement
> 
> 
> 
> On 4/23/25 7:46 PM, Zhao Liu wrote:
> > Hi all,
> > 
> > (Since patches 1 and 2 involve changes to x86 vendors other than Intel,
> > I have also cc'd friends from AMD and Zhaoxin.)
> > 
> > These are the ones I was going to clean up a long time ago:
> >   * Fixup CPUID 0x80000005 & 0x80000006 for Intel (and Zhaoxin now).
> >   * Add cache model for Intel CPUs.
> >   * Enable 0x1f CPUID leaf for specific Intel CPUs, which already have
> >     this leaf on host by default.
> 
> If you run into vendor specific branches while refactoring the
> topology-related code, please feel free to treat Intel and Zhaoxin as one
> class. For every topology CPUID leaf(0x0B, 0x1F, ...) so far, Zhaoxin has
> followed the Intel SDM definition exactly.

Thank you for your confirmation. I'll post v2 soon (If things go well,
it'll be in the next two weeks. :-) )

Regards,
Zhao
Re: [RFC 00/10] i386/cpu: Cache CPUID fixup, Intel cache model & topo CPUID enhencement
Posted by Ewan Hai 5 months, 3 weeks ago

On 5/27/25 5:19 PM, Zhao Liu wrote:
> 
> On Mon, May 26, 2025 at 06:52:41PM +0800, Ewan Hai wrote:
>> Date: Mon, 26 May 2025 18:52:41 +0800
>> From: Ewan Hai <ewanhai-oc@zhaoxin.com>
>> Subject: Re: [RFC 00/10] i386/cpu: Cache CPUID fixup, Intel cache model &
>>   topo CPUID enhencement
>>
>>
>>
>> On 4/23/25 7:46 PM, Zhao Liu wrote:
>>> Hi all,
>>>
>>> (Since patches 1 and 2 involve changes to x86 vendors other than Intel,
>>> I have also cc'd friends from AMD and Zhaoxin.)
>>>
>>> These are the ones I was going to clean up a long time ago:
>>>    * Fixup CPUID 0x80000005 & 0x80000006 for Intel (and Zhaoxin now).
>>>    * Add cache model for Intel CPUs.
>>>    * Enable 0x1f CPUID leaf for specific Intel CPUs, which already have
>>>      this leaf on host by default.
>>
>> If you run into vendor specific branches while refactoring the
>> topology-related code, please feel free to treat Intel and Zhaoxin as one
>> class. For every topology CPUID leaf(0x0B, 0x1F, ...) so far, Zhaoxin has
>> followed the Intel SDM definition exactly.
> 
> Thank you for your confirmation. I'll post v2 soon (If things go well,
> it'll be in the next two weeks. :-) )

No rush, everyone is busy, maintainers especially so. Just handle it whenever it 
best fits your schedule.