docs/system/arm/cpu-features.rst | 5 +++ target/arm/cpu.h | 6 +++ target/arm/kvm-consts.h | 2 + target/arm/kvm.c | 70 +++++++++++++++++++++++++++++++- 4 files changed, 82 insertions(+), 1 deletion(-)
This series adds a vcpu knob to request a specific PSCI version from KVM via the KVM_REG_ARM_PSCI_VERSION FW register. Note: in order to support PSCI v0.1 we need to drop vcpu initialization with KVM_CAP_ARM_PSCI_0_2 in that case. Alternatively we could limit support to versions >=0.2 . Sebastian Ott (2): target/arm/kvm: add constants for new PSCI versions target/arm/kvm: add kvm-psci-version vcpu property docs/system/arm/cpu-features.rst | 5 +++ target/arm/cpu.h | 6 +++ target/arm/kvm-consts.h | 2 + target/arm/kvm.c | 70 +++++++++++++++++++++++++++++++- 4 files changed, 82 insertions(+), 1 deletion(-) -- 2.42.0
On Thu, 11 Sept 2025 at 15:49, Sebastian Ott <sebott@redhat.com> wrote: > > This series adds a vcpu knob to request a specific PSCI version > from KVM via the KVM_REG_ARM_PSCI_VERSION FW register. > > Note: in order to support PSCI v0.1 we need to drop vcpu > initialization with KVM_CAP_ARM_PSCI_0_2 in that case. > Alternatively we could limit support to versions >=0.2 . > > Sebastian Ott (2): > target/arm/kvm: add constants for new PSCI versions > target/arm/kvm: add kvm-psci-version vcpu property Could we have some rationale, please? What's the use case where you might need to specify a particular PSCI version? thanks -- PMM
On Thu, 11 Sep 2025, Peter Maydell wrote: > On Thu, 11 Sept 2025 at 15:49, Sebastian Ott <sebott@redhat.com> wrote: >> >> This series adds a vcpu knob to request a specific PSCI version >> from KVM via the KVM_REG_ARM_PSCI_VERSION FW register. >> >> Note: in order to support PSCI v0.1 we need to drop vcpu >> initialization with KVM_CAP_ARM_PSCI_0_2 in that case. >> Alternatively we could limit support to versions >=0.2 . >> >> Sebastian Ott (2): >> target/arm/kvm: add constants for new PSCI versions >> target/arm/kvm: add kvm-psci-version vcpu property > > Could we have some rationale, please? What's the use case > where you might need to specify a particular PSCI version? The use case is migrating between different host kernel versions. Per default the kernel reports the latest PSCI version in the KVM_REG_ARM_PSCI_VERSION register (for KVM_CAP_ARM_PSCI_0_2) - when that differs between source and target a migration will fail. This property allows to request a PSCI version that is supported by both sides. Specifically I want to support migration between host kernels with and without the following Linux commit: 8be82d536a9f KVM: arm64: Add support for PSCI v1.2 and v1.3 Regards, Sebastian
On Thu, 11 Sept 2025 at 16:59, Sebastian Ott <sebott@redhat.com> wrote: > > On Thu, 11 Sep 2025, Peter Maydell wrote: > > On Thu, 11 Sept 2025 at 15:49, Sebastian Ott <sebott@redhat.com> wrote: > >> > >> This series adds a vcpu knob to request a specific PSCI version > >> from KVM via the KVM_REG_ARM_PSCI_VERSION FW register. > >> > >> Note: in order to support PSCI v0.1 we need to drop vcpu > >> initialization with KVM_CAP_ARM_PSCI_0_2 in that case. > >> Alternatively we could limit support to versions >=0.2 . > >> > >> Sebastian Ott (2): > >> target/arm/kvm: add constants for new PSCI versions > >> target/arm/kvm: add kvm-psci-version vcpu property > > > > Could we have some rationale, please? What's the use case > > where you might need to specify a particular PSCI version? > > The use case is migrating between different host kernel versions. > Per default the kernel reports the latest PSCI version in the > KVM_REG_ARM_PSCI_VERSION register (for KVM_CAP_ARM_PSCI_0_2) - > when that differs between source and target a migration will fail. > > This property allows to request a PSCI version that is supported by > both sides. Specifically I want to support migration between host > kernels with and without the following Linux commit: > 8be82d536a9f KVM: arm64: Add support for PSCI v1.2 and v1.3 So if the destination kernel is post that commit and the source kernel pre-dates it, do we fail migration? Or is this only a migration failure when the destination doesn't support the PSCI version we defaulted to at the source end? thanks -- PMM
On Thu, 11 Sep 2025, Peter Maydell wrote: > On Thu, 11 Sept 2025 at 16:59, Sebastian Ott <sebott@redhat.com> wrote: >> >> On Thu, 11 Sep 2025, Peter Maydell wrote: >>> On Thu, 11 Sept 2025 at 15:49, Sebastian Ott <sebott@redhat.com> wrote: >>>> >>>> This series adds a vcpu knob to request a specific PSCI version >>>> from KVM via the KVM_REG_ARM_PSCI_VERSION FW register. >>>> >>>> Note: in order to support PSCI v0.1 we need to drop vcpu >>>> initialization with KVM_CAP_ARM_PSCI_0_2 in that case. >>>> Alternatively we could limit support to versions >=0.2 . >>>> >>>> Sebastian Ott (2): >>>> target/arm/kvm: add constants for new PSCI versions >>>> target/arm/kvm: add kvm-psci-version vcpu property >>> >>> Could we have some rationale, please? What's the use case >>> where you might need to specify a particular PSCI version? >> >> The use case is migrating between different host kernel versions. >> Per default the kernel reports the latest PSCI version in the >> KVM_REG_ARM_PSCI_VERSION register (for KVM_CAP_ARM_PSCI_0_2) - >> when that differs between source and target a migration will fail. >> >> This property allows to request a PSCI version that is supported by >> both sides. Specifically I want to support migration between host >> kernels with and without the following Linux commit: >> 8be82d536a9f KVM: arm64: Add support for PSCI v1.2 and v1.3 > > So if the destination kernel is post that commit and the > source kernel pre-dates it, do we fail migration? This case works with current qemu without any changes, since on target qemu would write the register value it has stored from the source side (QEMU_PSCI_VERSION_1_1) and thus requests kvm on target to emulate that version. > Or is > this only a migration failure when the destination doesn't > support the PSCI version we defaulted to at the source end? Yes, this doesn't work with current qemu. On target qemu would write QEMU_PSCI_VERSION_1_3 to the KVM_REG_ARM_PSCI_VERSION register but that kernel doesn't know this version and the migration will fail. With this series you could request QEMU_PSCI_VERSION_1_1 on the source kernel to allow a migration. (For RHEL we have distro specific machine types that will use this property to handle that via compats.) Sebastian
On Thu, 11 Sept 2025 at 17:29, Sebastian Ott <sebott@redhat.com> wrote: > > On Thu, 11 Sep 2025, Peter Maydell wrote: > > On Thu, 11 Sept 2025 at 16:59, Sebastian Ott <sebott@redhat.com> wrote: > >> > >> On Thu, 11 Sep 2025, Peter Maydell wrote: > >>> On Thu, 11 Sept 2025 at 15:49, Sebastian Ott <sebott@redhat.com> wrote: > >>>> > >>>> This series adds a vcpu knob to request a specific PSCI version > >>>> from KVM via the KVM_REG_ARM_PSCI_VERSION FW register. > >>>> > >>>> Note: in order to support PSCI v0.1 we need to drop vcpu > >>>> initialization with KVM_CAP_ARM_PSCI_0_2 in that case. > >>>> Alternatively we could limit support to versions >=0.2 . > >>>> > >>>> Sebastian Ott (2): > >>>> target/arm/kvm: add constants for new PSCI versions > >>>> target/arm/kvm: add kvm-psci-version vcpu property > >>> > >>> Could we have some rationale, please? What's the use case > >>> where you might need to specify a particular PSCI version? > >> > >> The use case is migrating between different host kernel versions. > >> Per default the kernel reports the latest PSCI version in the > >> KVM_REG_ARM_PSCI_VERSION register (for KVM_CAP_ARM_PSCI_0_2) - > >> when that differs between source and target a migration will fail. > >> > >> This property allows to request a PSCI version that is supported by > >> both sides. Specifically I want to support migration between host > >> kernels with and without the following Linux commit: > >> 8be82d536a9f KVM: arm64: Add support for PSCI v1.2 and v1.3 > > > > So if the destination kernel is post that commit and the > > source kernel pre-dates it, do we fail migration? > > This case works with current qemu without any changes, since on > target qemu would write the register value it has stored from > the source side (QEMU_PSCI_VERSION_1_1) and thus requests kvm > on target to emulate that version. > > > Or is > > this only a migration failure when the destination doesn't > > support the PSCI version we defaulted to at the source end? > > Yes, this doesn't work with current qemu. On target qemu would > write QEMU_PSCI_VERSION_1_3 to the KVM_REG_ARM_PSCI_VERSION > register but that kernel doesn't know this version and the > migration will fail. I was under the impression that trying to migrate backwards from a newer kernel to an older one was likely to fail for various reasons (notably "new kernel reports a new system register the old one doesn't") ? Perhaps we should think about the problem in a wider scope than just the PSCI version... thanks -- PMM
On Thu, 11 Sep 2025, Peter Maydell wrote: > On Thu, 11 Sept 2025 at 17:29, Sebastian Ott <sebott@redhat.com> wrote: >> >> On Thu, 11 Sep 2025, Peter Maydell wrote: >>> On Thu, 11 Sept 2025 at 16:59, Sebastian Ott <sebott@redhat.com> wrote: >>>> >>>> On Thu, 11 Sep 2025, Peter Maydell wrote: >>>>> On Thu, 11 Sept 2025 at 15:49, Sebastian Ott <sebott@redhat.com> wrote: >>>>>> >>>>>> This series adds a vcpu knob to request a specific PSCI version >>>>>> from KVM via the KVM_REG_ARM_PSCI_VERSION FW register. >>>>>> >>>>>> Note: in order to support PSCI v0.1 we need to drop vcpu >>>>>> initialization with KVM_CAP_ARM_PSCI_0_2 in that case. >>>>>> Alternatively we could limit support to versions >=0.2 . >>>>>> >>>>>> Sebastian Ott (2): >>>>>> target/arm/kvm: add constants for new PSCI versions >>>>>> target/arm/kvm: add kvm-psci-version vcpu property >>>>> >>>>> Could we have some rationale, please? What's the use case >>>>> where you might need to specify a particular PSCI version? >>>> >>>> The use case is migrating between different host kernel versions. >>>> Per default the kernel reports the latest PSCI version in the >>>> KVM_REG_ARM_PSCI_VERSION register (for KVM_CAP_ARM_PSCI_0_2) - >>>> when that differs between source and target a migration will fail. >>>> >>>> This property allows to request a PSCI version that is supported by >>>> both sides. Specifically I want to support migration between host >>>> kernels with and without the following Linux commit: >>>> 8be82d536a9f KVM: arm64: Add support for PSCI v1.2 and v1.3 >>> >>> So if the destination kernel is post that commit and the >>> source kernel pre-dates it, do we fail migration? >> >> This case works with current qemu without any changes, since on >> target qemu would write the register value it has stored from >> the source side (QEMU_PSCI_VERSION_1_1) and thus requests kvm >> on target to emulate that version. >> >>> Or is >>> this only a migration failure when the destination doesn't >>> support the PSCI version we defaulted to at the source end? >> >> Yes, this doesn't work with current qemu. On target qemu would >> write QEMU_PSCI_VERSION_1_3 to the KVM_REG_ARM_PSCI_VERSION >> register but that kernel doesn't know this version and the >> migration will fail. > > I was under the impression that trying to migrate backwards > from a newer kernel to an older one was likely to fail > for various reasons (notably "new kernel reports a new > system register the old one doesn't") ? Perhaps we should > think about the problem in a wider scope than just the > PSCI version... Yes we already are ;-) See this series from Cornelia: https://lore.kernel.org/qemu-devel/20250414163849.321857-1-cohuck@redhat.com/ And this from Eric: https://lore.kernel.org/qemu-devel/20250911134324.3702720-1-eric.auger@redhat.com/ Both will help mitigate register differences for a backwards/downgrade migration. Sebastian
Hi Peter, On 9/11/25 6:46 PM, Sebastian Ott wrote: > On Thu, 11 Sep 2025, Peter Maydell wrote: >> On Thu, 11 Sept 2025 at 17:29, Sebastian Ott <sebott@redhat.com> wrote: >>> >>> On Thu, 11 Sep 2025, Peter Maydell wrote: >>>> On Thu, 11 Sept 2025 at 16:59, Sebastian Ott <sebott@redhat.com> >>>> wrote: >>>>> >>>>> On Thu, 11 Sep 2025, Peter Maydell wrote: >>>>>> On Thu, 11 Sept 2025 at 15:49, Sebastian Ott <sebott@redhat.com> >>>>>> wrote: >>>>>>> >>>>>>> This series adds a vcpu knob to request a specific PSCI version >>>>>>> from KVM via the KVM_REG_ARM_PSCI_VERSION FW register. >>>>>>> >>>>>>> Note: in order to support PSCI v0.1 we need to drop vcpu >>>>>>> initialization with KVM_CAP_ARM_PSCI_0_2 in that case. >>>>>>> Alternatively we could limit support to versions >=0.2 . >>>>>>> >>>>>>> Sebastian Ott (2): >>>>>>> target/arm/kvm: add constants for new PSCI versions >>>>>>> target/arm/kvm: add kvm-psci-version vcpu property >>>>>> >>>>>> Could we have some rationale, please? What's the use case >>>>>> where you might need to specify a particular PSCI version? >>>>> >>>>> The use case is migrating between different host kernel versions. >>>>> Per default the kernel reports the latest PSCI version in the >>>>> KVM_REG_ARM_PSCI_VERSION register (for KVM_CAP_ARM_PSCI_0_2) - >>>>> when that differs between source and target a migration will fail. >>>>> >>>>> This property allows to request a PSCI version that is supported by >>>>> both sides. Specifically I want to support migration between host >>>>> kernels with and without the following Linux commit: >>>>> 8be82d536a9f KVM: arm64: Add support for PSCI v1.2 and v1.3 >>>> >>>> So if the destination kernel is post that commit and the >>>> source kernel pre-dates it, do we fail migration? >>> >>> This case works with current qemu without any changes, since on >>> target qemu would write the register value it has stored from >>> the source side (QEMU_PSCI_VERSION_1_1) and thus requests kvm >>> on target to emulate that version. >>> >>>> Or is >>>> this only a migration failure when the destination doesn't >>>> support the PSCI version we defaulted to at the source end? >>> >>> Yes, this doesn't work with current qemu. On target qemu would >>> write QEMU_PSCI_VERSION_1_3 to the KVM_REG_ARM_PSCI_VERSION >>> register but that kernel doesn't know this version and the >>> migration will fail. >> >> I was under the impression that trying to migrate backwards >> from a newer kernel to an older one was likely to fail >> for various reasons (notably "new kernel reports a new >> system register the old one doesn't") ? Perhaps we should >> think about the problem in a wider scope than just the >> PSCI version... > > Yes we already are ;-) See this series from Cornelia: > https://lore.kernel.org/qemu-devel/20250414163849.321857-1-cohuck@redhat.com/ > > > And this from Eric: > https://lore.kernel.org/qemu-devel/20250911134324.3702720-1-eric.auger@redhat.com/ > the above series especially handles a class of migration errors where the source host kernel exposes more KVM regs to userspace than destination host kernel. In that case, currently, the vcpu state cannot be migrated. I should have called that: mitigation of* "failed to load cpu:cpreg_vmstate_array_len" migration errors. Sebastian tries to handle a change in the default value of a pseudo FW register. We would like to have a compat to keep the old value for old machine types. Thanks Eric * > > Both will help mitigate register differences for a backwards/downgrade > migration. > > Sebastian >
© 2016 - 2025 Red Hat, Inc.