Hej folks,

I'm looking into supporting migration between 2 Ampere Altra (Max)
machines (using Neoverse-N1). They are almost identical regarding
their feature id register state except for CTR_EL0.DIC which is set
on one machine but not the other.

CTR_EL0 is currently marked as invariant and migrating a VM between
those 2 machines using qemu fails.

Changes RFC [0] -> V1 [1]:
 * store the emulated value per VM and not per VCPU
 * allow to change more values than just the DIC bit
 * only trap guest access to that reg when needed
 * make sure to not present the guest with an inconsistent register set
Changes V1 -> V2:
 * implemented Marc's suggestion for keeping registers consistent while
   not breaking userspace ABI / expectations (I hope correctly this time)
 * keep the shadowed value valid at all time
 * unify the code to setup traps

Note:
* in order to switch off CTR_EL0.IDC userspace first has to set up
  CLIDR_EL1 accordingly
* reading CCSIDR before and after changing the cache size in CTR_EL0
  could result in different values (but only if CCSIDR was not changed
  from the generated value)
* I'll prepare a testcase for the next revision

Thanks,
Sebastian

[0]: https://lore.kernel.org/all/20240318111636.10613-1-sebott@redhat.com/T/
[1]: https://lore.kernel.org/lkml/20240405120108.11844-1-sebott@redhat.com/T/

Sebastian Ott (6):
  KVM: arm64: change return value in arm64_check_features()
  KVM: arm64: unify trap setup code
  KVM: arm64: maintain per VM value for CTR_EL0
  KVM: arm64: add emulation for CTR_EL0 register
  KVM: arm64: show writable masks for feature registers
  KVM: arm64: rename functions for invariant sys regs

 arch/arm64/include/asm/kvm_emulate.h |  37 -----
 arch/arm64/include/asm/kvm_host.h    |   4 +-
 arch/arm64/kvm/arm.c                 |   3 +-
 arch/arm64/kvm/sys_regs.c            | 225 ++++++++++++++++++++-------
 4 files changed, 173 insertions(+), 96 deletions(-)

-- 
2.42.0

Hi Sebastian,

On Fri, Apr 26, 2024 at 12:49:44PM +0200, Sebastian Ott wrote:
> Hej folks,
> 
> I'm looking into supporting migration between 2 Ampere Altra (Max)
> machines (using Neoverse-N1). They are almost identical regarding
> their feature id register state except for CTR_EL0.DIC which is set
> on one machine but not the other.
> 
> CTR_EL0 is currently marked as invariant and migrating a VM between
> those 2 machines using qemu fails.

I left some feedback on the series, but in addition to that would it be
possible to augment the set_id_regs selftest to exercise the CTR_EL0
mutability?

-- 
Thanks,
Oliver

Hi Oliver,

On Wed, 1 May 2024, Oliver Upton wrote:
> On Fri, Apr 26, 2024 at 12:49:44PM +0200, Sebastian Ott wrote:
>> Hej folks,
>>
>> I'm looking into supporting migration between 2 Ampere Altra (Max)
>> machines (using Neoverse-N1). They are almost identical regarding
>> their feature id register state except for CTR_EL0.DIC which is set
>> on one machine but not the other.
>>
>> CTR_EL0 is currently marked as invariant and migrating a VM between
>> those 2 machines using qemu fails.
>
> I left some feedback on the series, but in addition to that would it be
> possible to augment the set_id_regs selftest to exercise the CTR_EL0
> mutability?

Yes, sure!

Thanks,
Sebastian