Hi guys,

This is another try to allow userspace to change ID_AA64PFR1_EL1, and we want to
give userspace the ability to control the visible feature set for a VM, which
could be used by userspace in such a way to transparently migrate VMs.

The patch series have four part:

The first patch disable those fields which KVM doesn't know how to handle, so
KVM will only expose value 0 of those fields to the guest.

The second patch check the FEAT_SSBS in guest IDREG instead of the cpu
capability.

The third patch allow userspace to change ID_AA64PFR1_EL1, it only advertise the
fields known to KVM and leave others unadvertise.

The fourth patch adds the kselftest to test if userspace can change the
ID_AA64PFR1_EL1.

Besides, I also noticed there is another patch [1] which try to make the
ID_AA64PFR1_EL1 writable. This patch [1] is try to enable GCS on baremental, and
add GCS support for the guest. What I understand is if we have GCS support on
baremental, it will be clear to how to handle them in KVM. And same for other
fields like NMI, THE, DF2, MTEX.. At that time, they can be writable.

[1] [PATCH v9 13/39] KVM: arm64: Manage GCS registers for guests
    https://lore.kernel.org/all/20240625-arm64-gcs-v9-13-0f634469b8f0@kernel.org/

Changelog:
----------
v4 -> v5:
  * Only advertise fields which KVM know how to handle to userspace, leave
    others unadvertised.
  * Add a new patch to check FEAT_SSBS in IDREG instead of cpu capability.
  * Tweak the kselftest writable fields.
  * Improve the commit message.

v3 -> v4:
  * Add a new patch to disable some feature which KVM doesn't know how to
    handle in the register accessor.
  * Handle all the fields in the register.
  * Fixes a small cnt issue in kselftest.

v2 -> v3:
  * Give more description about why only part of the fields can be writable.
  * Updated the writable mask by referring the latest ARM spec.

v1 -> v2:
  * Tackling the full register instead of single field.
  * Changing the patch title and commit message.

RFCv1 -> v1:
  * Fix the compilation error.
  * Delete the machine specific information and make the description more
    generable.

RFCv1: https://lore.kernel.org/all/20240612023553.127813-1-shahuang@redhat.com/
v1: https://lore.kernel.org/all/20240617075131.1006173-1-shahuang@redhat.com/
v2: https://lore.kernel.org/all/20240618063808.1040085-1-shahuang@redhat.com/
v3: https://lore.kernel.org/all/20240628060454.1936886-2-shahuang@redhat.com/
v4: https://lore.kernel.org/all/20240718035017.434996-1-shahuang@redhat.com/

Shaoqin Huang (4):
  KVM: arm64: Disable fields that KVM doesn't know how to handle in
    ID_AA64PFR1_EL1
  KVM: arm64: Use kvm_has_feat() to check if FEAT_SSBS is advertised to
    the guest
  KVM: arm64: Allow userspace to change ID_AA64PFR1_EL1
  KVM: selftests: aarch64: Add writable test for ID_AA64PFR1_EL1

 arch/arm64/kvm/hypercalls.c                   | 12 +++++-----
 arch/arm64/kvm/sys_regs.c                     | 22 ++++++++++++++++++-
 .../selftests/kvm/aarch64/set_id_regs.c       | 14 +++++++++---
 3 files changed, 38 insertions(+), 10 deletions(-)

-- 
2.40.1

On Tue, 23 Jul 2024 03:19:59 -0400, Shaoqin Huang wrote:
> This is another try to allow userspace to change ID_AA64PFR1_EL1, and we want to
> give userspace the ability to control the visible feature set for a VM, which
> could be used by userspace in such a way to transparently migrate VMs.
> 
> The patch series have four part:
> 
> The first patch disable those fields which KVM doesn't know how to handle, so
> KVM will only expose value 0 of those fields to the guest.
> 
> [...]

Applied to next, thanks!

[1/4] KVM: arm64: Disable fields that KVM doesn't know how to handle in ID_AA64PFR1_EL1
      commit: ffe68b2d19a5a84440fea99a732cfc3b157559eb
[2/4] KVM: arm64: Use kvm_has_feat() to check if FEAT_SSBS is advertised to the guest
      commit: e8d164974cfa46fe5ec87869c8a7113641f322d5
[3/4] KVM: arm64: Allow userspace to change ID_AA64PFR1_EL1
      commit: 78c4446b5f957fb14737582e503b1b25f66edc45
[4/4] KVM: selftests: aarch64: Add writable test for ID_AA64PFR1_EL1
      commit: dc9b5d7e0bd40e68a94013766b27be3dda10c006

Cheers,

	M.
-- 
Without deviation from the norm, progress is not possible.

On Tue, 23 Jul 2024 08:19:59 +0100,
Shaoqin Huang <shahuang@redhat.com> wrote:
> 
> Hi guys,
> 
> This is another try to allow userspace to change ID_AA64PFR1_EL1, and we want to
> give userspace the ability to control the visible feature set for a VM, which
> could be used by userspace in such a way to transparently migrate VMs.


I think this looks OK now, thanks for going through the motions and
doing the right thing.

What is missing is similar handling for 32bit ID registers, but I'm
not sure we keen on going down that road -- machines capable of
running those are on their way out. This can be done later anyway,
should anyone care.

>
> The patch series have four part:
> 
> The first patch disable those fields which KVM doesn't know how to handle, so
> KVM will only expose value 0 of those fields to the guest.
> 
> The second patch check the FEAT_SSBS in guest IDREG instead of the cpu
> capability.
> 
> The third patch allow userspace to change ID_AA64PFR1_EL1, it only advertise the
> fields known to KVM and leave others unadvertise.
> 
> The fourth patch adds the kselftest to test if userspace can change the
> ID_AA64PFR1_EL1.
> 
> Besides, I also noticed there is another patch [1] which try to make the
> ID_AA64PFR1_EL1 writable. This patch [1] is try to enable GCS on baremental, and
> add GCS support for the guest. What I understand is if we have GCS support on
> baremental, it will be clear to how to handle them in KVM. And same for other
> fields like NMI, THE, DF2, MTEX.. At that time, they can be
> writable.

I expect that Broonie would look into this as part of his series.

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.

On Sun, Aug 25, 2024 at 05:46:36PM +0100, Marc Zyngier wrote:
> On Tue, 23 Jul 2024 08:19:59 +0100,
> Shaoqin Huang <shahuang@redhat.com> wrote:
> > 
> > Hi guys,
> > 
> > This is another try to allow userspace to change ID_AA64PFR1_EL1, and we want to
> > give userspace the ability to control the visible feature set for a VM, which
> > could be used by userspace in such a way to transparently migrate VMs.
> 
> 
> I think this looks OK now, thanks for going through the motions and
> doing the right thing.
> 
> What is missing is similar handling for 32bit ID registers, but I'm
> not sure we keen on going down that road -- machines capable of
> running those are on their way out. This can be done later anyway,
> should anyone care.

The Aarch32 ID registers need doing - we've already established that
fact. Sadly, you decided you wouldn't respond to my patch addressing
one of the Aarch32 ID registers despite me sending follow-ups to nicely
ask you about this - you seemed to go utterly silent on it.

The Aarch32 ID registers have changed value between different kernel
versions, and given that QEMU saves and restores _all_ ID registers,
changes to these ID registers cause a regression if one attempts to
migrate VMs between one kernel version and the next. It doesn't even
have to be between two physical machines. Libvirt supports managed-
saving on reboot, where it saves an image of a VM at shutdown, and
restores it at the next reboot. These changes in ID registers render
effectively data loss in VMs that have been managed-saved - the
saved state of the VM has to either be destroyed, or the host kernel
reverted back and _never_ moved forward.

As you don't seem to be keen to address this (by ignoring my emails
on the topic, and now suggesting in your response above that you're
not keen to do anything with the Aarch32 ID registers, I guess this
just means that KVM on Aarch64 is going to forever suck.

I'm sure Oliver will recall my emails on this which you've decided to
ignore... he was supportive of my efforts to address this.

-- 
*** please note that I probably will only be occasionally responsive
*** for an unknown period of time due to recent eye surgery making
*** reading quite difficult.

RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!

On Sun, 25 Aug 2024 18:09:48 +0100,
"Russell King (Oracle)" <linux@armlinux.org.uk> wrote:
> 
> On Sun, Aug 25, 2024 at 05:46:36PM +0100, Marc Zyngier wrote:
> > On Tue, 23 Jul 2024 08:19:59 +0100,
> > Shaoqin Huang <shahuang@redhat.com> wrote:
> > > 
> > > Hi guys,
> > > 
> > > This is another try to allow userspace to change ID_AA64PFR1_EL1, and we want to
> > > give userspace the ability to control the visible feature set for a VM, which
> > > could be used by userspace in such a way to transparently migrate VMs.
> > 
> > 
> > I think this looks OK now, thanks for going through the motions and
> > doing the right thing.
> > 
> > What is missing is similar handling for 32bit ID registers, but I'm
> > not sure we keen on going down that road -- machines capable of
> > running those are on their way out. This can be done later anyway,
> > should anyone care.
> 
> The Aarch32 ID registers need doing - we've already established that
> fact. Sadly, you decided you wouldn't respond to my patch addressing
> one of the Aarch32 ID registers despite me sending follow-ups to nicely
> ask you about this - you seemed to go utterly silent on it.

No, Russell. *you* went utterly silent after your May patch. You sent
an RFC, to which people responded. Given that your last email on the
subject was almost 4 months ago and that you never brought the subject
up again, it can't be that big a deal.

To me, an RFC means "I have this idea, and I'm not sure how to do it".
An RFC is usually only a proof of concept that has no purpose being
taken at face value. If you want a patch taken seriously, don't send
it as an RFC. And send it again if nobody replies. It's not that this
is anything new.

> The Aarch32 ID registers have changed value between different kernel
> versions, and given that QEMU saves and restores _all_ ID registers,
> changes to these ID registers cause a regression if one attempts to
> migrate VMs between one kernel version and the next. It doesn't even
> have to be between two physical machines. Libvirt supports managed-
> saving on reboot, where it saves an image of a VM at shutdown, and
> restores it at the next reboot. These changes in ID registers render
> effectively data loss in VMs that have been managed-saved - the
> saved state of the VM has to either be destroyed, or the host kernel
> reverted back and _never_ moved forward.
>
> As you don't seem to be keen to address this (by ignoring my emails
> on the topic, and now suggesting in your response above that you're
> not keen to do anything with the Aarch32 ID registers, I guess this
> just means that KVM on Aarch64 is going to forever suck.

I'm fine with that. Nobody is forced to use it, and I don't feel the
need to put extra effort on things I don't care about any more.
AArch32 support is one of these things, amongst many others.

If you want the support to improve, I suggest you send patches. And
send them again if no reply shows up in a timely manner.  Because
you're probably the last person who gives a damn about the AArch32
support in KVM. And if not even you can be bothered to fix it, then
support for AArch32 EL1 should probably be removed altogether (I'm all
for deleting unused code).

> I'm sure Oliver will recall my emails on this which you've decided to
> ignore... he was supportive of my efforts to address this.

I'm supportive as well. I'm just not going to fix it for you.

	M.

-- 
Without deviation from the norm, progress is not possible.

Hi Marc, Oliver,

Kindly ping for this series. Welcome your feedbacks. I wonder if this 
implementation satisfy your expectation?

Thanks,
Shaoqin

On 7/23/24 15:19, Shaoqin Huang wrote:
> Hi guys,
> 
> This is another try to allow userspace to change ID_AA64PFR1_EL1, and we want to
> give userspace the ability to control the visible feature set for a VM, which
> could be used by userspace in such a way to transparently migrate VMs.
> 
> The patch series have four part:
> 
> The first patch disable those fields which KVM doesn't know how to handle, so
> KVM will only expose value 0 of those fields to the guest.
> 
> The second patch check the FEAT_SSBS in guest IDREG instead of the cpu
> capability.
> 
> The third patch allow userspace to change ID_AA64PFR1_EL1, it only advertise the
> fields known to KVM and leave others unadvertise.
> 
> The fourth patch adds the kselftest to test if userspace can change the
> ID_AA64PFR1_EL1.
> 
> Besides, I also noticed there is another patch [1] which try to make the
> ID_AA64PFR1_EL1 writable. This patch [1] is try to enable GCS on baremental, and
> add GCS support for the guest. What I understand is if we have GCS support on
> baremental, it will be clear to how to handle them in KVM. And same for other
> fields like NMI, THE, DF2, MTEX.. At that time, they can be writable.
> 
> [1] [PATCH v9 13/39] KVM: arm64: Manage GCS registers for guests
>      https://lore.kernel.org/all/20240625-arm64-gcs-v9-13-0f634469b8f0@kernel.org/
> 
> Changelog:
> ----------
> v4 -> v5:
>    * Only advertise fields which KVM know how to handle to userspace, leave
>      others unadvertised.
>    * Add a new patch to check FEAT_SSBS in IDREG instead of cpu capability.
>    * Tweak the kselftest writable fields.
>    * Improve the commit message.
> 
> v3 -> v4:
>    * Add a new patch to disable some feature which KVM doesn't know how to
>      handle in the register accessor.
>    * Handle all the fields in the register.
>    * Fixes a small cnt issue in kselftest.
> 
> v2 -> v3:
>    * Give more description about why only part of the fields can be writable.
>    * Updated the writable mask by referring the latest ARM spec.
> 
> v1 -> v2:
>    * Tackling the full register instead of single field.
>    * Changing the patch title and commit message.
> 
> RFCv1 -> v1:
>    * Fix the compilation error.
>    * Delete the machine specific information and make the description more
>      generable.
> 
> RFCv1: https://lore.kernel.org/all/20240612023553.127813-1-shahuang@redhat.com/
> v1: https://lore.kernel.org/all/20240617075131.1006173-1-shahuang@redhat.com/
> v2: https://lore.kernel.org/all/20240618063808.1040085-1-shahuang@redhat.com/
> v3: https://lore.kernel.org/all/20240628060454.1936886-2-shahuang@redhat.com/
> v4: https://lore.kernel.org/all/20240718035017.434996-1-shahuang@redhat.com/
> 
> Shaoqin Huang (4):
>    KVM: arm64: Disable fields that KVM doesn't know how to handle in
>      ID_AA64PFR1_EL1
>    KVM: arm64: Use kvm_has_feat() to check if FEAT_SSBS is advertised to
>      the guest
>    KVM: arm64: Allow userspace to change ID_AA64PFR1_EL1
>    KVM: selftests: aarch64: Add writable test for ID_AA64PFR1_EL1
> 
>   arch/arm64/kvm/hypercalls.c                   | 12 +++++-----
>   arch/arm64/kvm/sys_regs.c                     | 22 ++++++++++++++++++-
>   .../selftests/kvm/aarch64/set_id_regs.c       | 14 +++++++++---
>   3 files changed, 38 insertions(+), 10 deletions(-)
> 

-- 
Shaoqin