Hi All,
This is the v2 RFC of the Named CPU models support for Arm64 in QEMU.
v2 implements all of Planned items that were mentioned in v1.
v1 link: https://lore.kernel.org/qemu-arm/20260513163356.3033159-1-shaju.abraham@nutanix.com/
Background:
=============================================================
Currently QEMU only support -cpu host/max for KVM on Arm64. This disables
live migration of VMs between hosts that differ in CPU features.
This RFC proposes hierarchical named models for Arm64 under KVM, this
provides strict contract of CPU features that will be exposed to the guest
for a given named model. Models can be customized further with feature
properties (like feat_AES=off, pauth=on, sve=off, etc...).
The design has three layers: an ARM ID-register field table (single
source of truth for fields, safe-rules, defaults, arch-defined values); a
property layer on top (single-field, fractional, and composite properties);
and a hierarchical named-model layer that picks values for those properties
with parent-chain inheritance.
The series also implements QMP commands that lets the management layer query
which models a host can run, what each model resolves to, and which feature
values the host supports.
Relationship with "customizable host model" series:
============================================================
This RFC overlaps with customizable host model series [1]. The two series
solve different parts of the same problem: [1] provides the lower layer
(per-field SYSREG_<REG>_<FIELD> u64 properties on -cpu host,
writable-mask probing, ID-register writeback).
This series adds the layers above that, like:
hierarchical named CPU models, default-from-zero baselines, fractional and
composite properties for named models (sve, pauth, expose-cache),
property/value names, and QMP introspection (blockers via safe-rule
validation, supported values, new query-arm-cpu-props-info). This series
also fixes the host-supports-Nested-Virtualziation-while-qemu-booted-without-it
writeback case [2] that surfaces once [1]'s writeback lands.
Long term plan is to rebase this series on top of [1]. Few difficulties
in that will be augmenting the ID regs fields table with safe rules and
default values, which Eric correctly mentioned in [1] that it cannot
be inferred from Registers.json.
For property names and values we have no strong preference,
<SYSREG>_<REG>_<FIELD> with uint64 values or "feat_*" properties with
string values (SYSREG_ID_AA64ISAR0_AES=2/feat_AES="pmull").
[1]: https://lore.kernel.org/qemu-arm/20260519132905.145643-1-eric.auger@redhat.com/
[2]: See patch 29 of this series.
TL;DR examples:
# Boot with Grace
qemu-system-aarch64 -cpu grace-v1 -machine virt,accel=kvm ...
# Grace with a feature disabled
qemu-system-aarch64 -cpu grace-v1,feat_SHA1=off ...
# Host passthrough with individual feature control
qemu-system-aarch64 -cpu host,feat_AES=aes ...
# named CPU model with composite properties
qemu-system-aarch64 -cpu grace-v1,pauth=off,sve=on ...
# QMP -cpu host query-cpu-model-expansion:
"model": {
"name": "host",
"props": {
"cpu_partnum": 3407,
"feat_SM4": "on",
"feat_SM3": "on",
....
}
}
# QMP -cpu graviton3-v1 query-cpu-model-expansion:
"model": {
"name": "graviton3-v1",
"props": {
"cpu_partnum": 3392,
"feat_SM4": "on",
"feat_SM3": "on",
"feat_RNDR": "on",
....
}
}
# QMP query-cpu-definitions (on Grace host) (blockers for graviton3-v1):
{
"name": "grace-v1",
"typename": "grace-v1-arm-cpu",
"static": false,
"deprecated": false
},
{
"name": "graviton3-v1",
"typename": "graviton3-v1-arm-cpu",
"unavailable-features": [
"hw_prop_TGRAN4_2",
"hw_prop_TGRAN64_2",
"hw_prop_TGRAN16_2",
"hw_prop_APA",
"feat_RNDR"
],
"static": false,
"deprecated": false
},
....
# new QMP query-arm-cpu-props-info (on Grace host):
{
"name": "feat_SHA1",
"type": "boolean",
"supported-values": [
"on",
"off"
]
},
{
"name": "feat_AES",
"type": "string",
"supported-values": [
"off",
"aes",
"pmull"
]
},
....
Changes since v1:
- Added `query-cpu-definitions` and `query-cpu-model-expansion` QMP
support for named models and accompanying properties.
- Introduced `query-arm-cpu-props-info` QMP command that returns
list of all newly supported properties for -cpu host, and its supported
values.
- Add composite properties for named models which acts as master "on"/"off"
switches for features that compose of broader range of ID registers
fields like pauth and sve.
- Added `expose-cache` composite with other cache related properties to
allow named models to expose cache information to the guest.
- Cache the host isar view with and without EL2 support, and make
-cpu host with writeback work on host kernel that supports
virtualization.
- Validate the cpu->isar.idregs before writing back to KVM to catch early
errors.
- Properties and models are now defined in file rather than using .h.inc
files.
Tested on NVIDIA Grace and AWS Graviton3 hosts (6.18); booted grace-v1,
graviton3-v1, neoverse-v2-v1, neoverse-v1-v1, and -cpu host with per-feature
overrides.
Plans for v3:
- Some documentation and unit tests.
- More properties based on what more the latest kernel supports.
- pmu composite property.
- Trying to rebase on top of [1].
- Splitting this huge series into 3-4 sub series.
Warm Regards,
Khushit, Shaju
Khushit Shah (20):
target/arm: Introduce data-structures for the ARM property layer
target/arm: Add all ARM64 properties to host model
target/arm: Add Nvidia Grace named model
target/arm/kvm: enable writable implementation ID registers
target/aarm: Validate cpu->isar.idregs[] before writeback
target/arm/kvm: handle DCZID_EL0 specially
target/arm: skip GIC, COPDBG and PMU fields during KVM writeback
target/arm: Add composite property type to model definitions
target/arm: define pauth composite property
target/arm: define sve composite property
target/arm: Introduce stub files required for qmp support
target/arm/qmp: add named models and properties to cpu-model-expansion
target/arm/kvm: introduce kvm_arm_get_host_isar helper
target/arm/qmp: add query-arm-cpu-props-info
target/arm: Report "off" for ID fields gated by vCPU init flags
target/arm/qmp: hook blockers in query-cpu-definitions
target/arm: Support exposing cache information for named cpu models
target/arm: Provide default cache hierarchy
target/arm: supported-values and blockers for CCSIDR cache properties
target/arm/kvm: fix host model writeback when kernel supports EL2
Shaju Abraham (9):
target/arm: named_cpu_model: define containers for ID registers and
fields
target/arm: Add ID Register field descriptions in cpu-idregs.h.inc
target/arm: Add MIDR, REVIDR, AIDR and extra ID regs to cpu-sysregs
target/arm: Generate ARM64 ID registers and field tables
target/arm: Replace FIELD() macros with IDREG_FIELD expansion
target/arm: Define ARM properties
target/arm: Add ID register field helper functions
target/arm: Add named cpu model infra + graviton3 named model
target/arm/kvm: Writeback modified ID registers to KVM
hw/arm/virt.c | 28 +-
qapi/misc-arm.json | 41 +
stubs/qmp-arm-gic.c | 6 +
target/arm/arm-cpu-models-stub.c | 18 +
target/arm/arm-cpu-models.c | 620 ++++++++++
target/arm/arm-cpu-models.h | 49 +
target/arm/arm-cpu-props-stub.c | 36 +
target/arm/arm-cpu-props.c | 1212 +++++++++++++++++++
target/arm/arm-cpu-props.h | 70 ++
target/arm/arm-qmp-cmds.c | 128 +-
target/arm/cpu-features.h | 246 +---
target/arm/cpu-idregs.c | 508 ++++++++
target/arm/cpu-idregs.h | 119 ++
target/arm/cpu-idregs.h.inc | 1898 ++++++++++++++++++++++++++++++
target/arm/cpu-sysregs.h.inc | 5 +
target/arm/cpu.h | 3 +
target/arm/cpu64.c | 31 +-
target/arm/kvm-stub.c | 7 +-
target/arm/kvm.c | 356 +++++-
target/arm/kvm_arm.h | 7 +-
target/arm/meson.build | 11 +-
target/arm/trace-events | 1 +
22 files changed, 5148 insertions(+), 252 deletions(-)
create mode 100644 target/arm/arm-cpu-models-stub.c
create mode 100644 target/arm/arm-cpu-models.c
create mode 100644 target/arm/arm-cpu-models.h
create mode 100644 target/arm/arm-cpu-props-stub.c
create mode 100644 target/arm/arm-cpu-props.c
create mode 100644 target/arm/arm-cpu-props.h
create mode 100644 target/arm/cpu-idregs.c
create mode 100644 target/arm/cpu-idregs.h
create mode 100644 target/arm/cpu-idregs.h.inc
--
2.52.0