From nobody Thu Nov 28 05:29:12 2024 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 289F421F40C; Fri, 4 Oct 2024 15:29:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728055781; cv=none; b=iOujTWtl004+CTVlifH9NiBOP5yunbOXjrNbQmRYOjzoF5Sudn8yJ2GrgyDrghTDVOep5QUkNyerOI99x6ZSHWrj9ReNOJvqM/usaLhHgpSs3ov89ugb54Wr2VWRDRrct9UmW039CDSRmfRqKCv21S94EKKCkznirt3aros8PNg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1728055781; c=relaxed/simple; bh=U7GZCtTIlQ59g4iAqZVgaKc7tyXpAtMbwy1ujEg22mY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=QRWH7vWlqKDEYJNGJ+pUggDGkwpwUjK8SxyvzLLrBxCy0D+XG8wPZEUwJJiU0skvQFDa6mduq3OSkxa82BU8w94y1ThN33oN4B6pH0pIqPurfTdrBmLCAePPXoht8cCOKQ/Oyor7NLDfqrxVaUMl5w2NG1zwiHtOoVbONVEY+Js= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 554C51516; Fri, 4 Oct 2024 08:30:09 -0700 (PDT) Received: from e122027.cambridge.arm.com (unknown [10.1.25.25]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id DC7813F640; Fri, 4 Oct 2024 08:29:35 -0700 (PDT) From: Steven Price To: kvm@vger.kernel.org, kvmarm@lists.linux.dev Cc: Steven Price , Catalin Marinas , Marc Zyngier , Will Deacon , James Morse , Oliver Upton , Suzuki K Poulose , Zenghui Yu , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Joey Gouly , Alexandru Elisei , Christoffer Dall , Fuad Tabba , linux-coco@lists.linux.dev, Ganapatrao Kulkarni , Gavin Shan , Shanker Donthineni , Alper Gun , "Aneesh Kumar K . V" Subject: [PATCH v5 18/43] arm64: RME: Handle realm enter/exit Date: Fri, 4 Oct 2024 16:27:39 +0100 Message-Id: <20241004152804.72508-19-steven.price@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20241004152804.72508-1-steven.price@arm.com> References: <20241004152804.72508-1-steven.price@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Entering a realm is done using a SMC call to the RMM. On exit the exit-codes need to be handled slightly differently to the normal KVM path so define our own functions for realm enter/exit and hook them in if the guest is a realm guest. Signed-off-by: Steven Price --- Changes since v4: * Rename handle_rme_exit() to handle_rec_exit() * Move the loop to copy registers into the REC enter structure from the to rec_exit_handlers callbacks to kvm_rec_enter(). This fixes a bug where the handler exits to user space and user space wants to modify the GPRS. * Some code rearrangement in rec_exit_ripas_change(). Changes since v2: * realm_set_ipa_state() now provides an output parameter for the top_iap that was changed. Use this to signal the VMM with the correct range that has been transitioned. * Adapt to previous patch changes. --- arch/arm64/include/asm/kvm_rme.h | 3 + arch/arm64/kvm/Makefile | 2 +- arch/arm64/kvm/arm.c | 19 +++- arch/arm64/kvm/rme-exit.c | 179 +++++++++++++++++++++++++++++++ arch/arm64/kvm/rme.c | 19 ++++ 5 files changed, 216 insertions(+), 6 deletions(-) create mode 100644 arch/arm64/kvm/rme-exit.c diff --git a/arch/arm64/include/asm/kvm_rme.h b/arch/arm64/include/asm/kvm_= rme.h index c064bfb080ad..889fe120283a 100644 --- a/arch/arm64/include/asm/kvm_rme.h +++ b/arch/arm64/include/asm/kvm_rme.h @@ -96,6 +96,9 @@ void kvm_realm_destroy_rtts(struct kvm *kvm, u32 ia_bits); int kvm_create_rec(struct kvm_vcpu *vcpu); void kvm_destroy_rec(struct kvm_vcpu *vcpu); =20 +int kvm_rec_enter(struct kvm_vcpu *vcpu); +int handle_rec_exit(struct kvm_vcpu *vcpu, int rec_run_status); + void kvm_realm_unmap_range(struct kvm *kvm, unsigned long ipa, u64 size, diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile index ce8a10d3161d..0170e902fb63 100644 --- a/arch/arm64/kvm/Makefile +++ b/arch/arm64/kvm/Makefile @@ -24,7 +24,7 @@ kvm-y +=3D arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.= o \ vgic/vgic-mmio.o vgic/vgic-mmio-v2.o \ vgic/vgic-mmio-v3.o vgic/vgic-kvm-device.o \ vgic/vgic-its.o vgic/vgic-debug.o \ - rme.o + rme.o rme-exit.o =20 kvm-$(CONFIG_HW_PERF_EVENTS) +=3D pmu-emul.o pmu.o kvm-$(CONFIG_ARM64_PTR_AUTH) +=3D pauth.o diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index ecce40a35cd0..273c08bb4a05 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1278,7 +1278,10 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) trace_kvm_entry(*vcpu_pc(vcpu)); guest_timing_enter_irqoff(); =20 - ret =3D kvm_arm_vcpu_enter_exit(vcpu); + if (vcpu_is_rec(vcpu)) + ret =3D kvm_rec_enter(vcpu); + else + ret =3D kvm_arm_vcpu_enter_exit(vcpu); =20 vcpu->mode =3D OUTSIDE_GUEST_MODE; vcpu->stat.exits++; @@ -1332,10 +1335,13 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) =20 local_irq_enable(); =20 - trace_kvm_exit(ret, kvm_vcpu_trap_get_class(vcpu), *vcpu_pc(vcpu)); - /* Exit types that need handling before we can be preempted */ - handle_exit_early(vcpu, ret); + if (!vcpu_is_rec(vcpu)) { + trace_kvm_exit(ret, kvm_vcpu_trap_get_class(vcpu), + *vcpu_pc(vcpu)); + + handle_exit_early(vcpu, ret); + } =20 preempt_enable(); =20 @@ -1358,7 +1364,10 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) ret =3D ARM_EXCEPTION_IL; } =20 - ret =3D handle_exit(vcpu, ret); + if (vcpu_is_rec(vcpu)) + ret =3D handle_rec_exit(vcpu, ret); + else + ret =3D handle_exit(vcpu, ret); } =20 /* Tell userspace about in-kernel device output levels */ diff --git a/arch/arm64/kvm/rme-exit.c b/arch/arm64/kvm/rme-exit.c new file mode 100644 index 000000000000..e96ea308212c --- /dev/null +++ b/arch/arm64/kvm/rme-exit.c @@ -0,0 +1,179 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright (C) 2023 ARM Ltd. + */ + +#include +#include +#include + +#include +#include +#include +#include + +typedef int (*exit_handler_fn)(struct kvm_vcpu *vcpu); + +static int rec_exit_reason_notimpl(struct kvm_vcpu *vcpu) +{ + struct realm_rec *rec =3D &vcpu->arch.rec; + + pr_err("[vcpu %d] Unhandled exit reason from realm (ESR: %#llx)\n", + vcpu->vcpu_id, rec->run->exit.esr); + return -ENXIO; +} + +static int rec_exit_sync_dabt(struct kvm_vcpu *vcpu) +{ + return kvm_handle_guest_abort(vcpu); +} + +static int rec_exit_sync_iabt(struct kvm_vcpu *vcpu) +{ + struct realm_rec *rec =3D &vcpu->arch.rec; + + pr_err("[vcpu %d] Unhandled instruction abort (ESR: %#llx).\n", + vcpu->vcpu_id, rec->run->exit.esr); + return -ENXIO; +} + +static int rec_exit_sys_reg(struct kvm_vcpu *vcpu) +{ + struct realm_rec *rec =3D &vcpu->arch.rec; + unsigned long esr =3D kvm_vcpu_get_esr(vcpu); + int rt =3D kvm_vcpu_sys_get_rt(vcpu); + bool is_write =3D !(esr & 1); + int ret; + + if (is_write) + vcpu_set_reg(vcpu, rt, rec->run->exit.gprs[0]); + + ret =3D kvm_handle_sys_reg(vcpu); + + if (ret >=3D 0 && !is_write) + rec->run->enter.gprs[0] =3D vcpu_get_reg(vcpu, rt); + + return ret; +} + +static exit_handler_fn rec_exit_handlers[] =3D { + [0 ... ESR_ELx_EC_MAX] =3D rec_exit_reason_notimpl, + [ESR_ELx_EC_SYS64] =3D rec_exit_sys_reg, + [ESR_ELx_EC_DABT_LOW] =3D rec_exit_sync_dabt, + [ESR_ELx_EC_IABT_LOW] =3D rec_exit_sync_iabt +}; + +static int rec_exit_psci(struct kvm_vcpu *vcpu) +{ + struct realm_rec *rec =3D &vcpu->arch.rec; + int i; + + for (i =3D 0; i < REC_RUN_GPRS; i++) + vcpu_set_reg(vcpu, i, rec->run->exit.gprs[i]); + + return kvm_smccc_call_handler(vcpu); +} + +static int rec_exit_ripas_change(struct kvm_vcpu *vcpu) +{ + struct kvm *kvm =3D vcpu->kvm; + struct realm *realm =3D &kvm->arch.realm; + struct realm_rec *rec =3D &vcpu->arch.rec; + unsigned long base =3D rec->run->exit.ripas_base; + unsigned long top =3D rec->run->exit.ripas_top; + unsigned long ripas =3D rec->run->exit.ripas_value; + unsigned long top_ipa; + int ret; + + if (!realm_is_addr_protected(realm, base) || + !realm_is_addr_protected(realm, top - 1)) { + kvm_err("Invalid RIPAS_CHANGE for %#lx - %#lx, ripas: %#lx\n", + base, top, ripas); + return -EINVAL; + } + + kvm_mmu_topup_memory_cache(&vcpu->arch.mmu_page_cache, + kvm_mmu_cache_min_pages(vcpu->arch.hw_mmu)); + write_lock(&kvm->mmu_lock); + ret =3D realm_set_ipa_state(vcpu, base, top, ripas, &top_ipa); + write_unlock(&kvm->mmu_lock); + + WARN(ret && ret !=3D -ENOMEM, + "Unable to satisfy RIPAS_CHANGE for %#lx - %#lx, ripas: %#lx\n", + base, top, ripas); + + /* Exit to VMM to complete the change */ + kvm_prepare_memory_fault_exit(vcpu, base, top_ipa - base, false, false, + ripas =3D=3D RMI_RAM); + + return 0; +} + +static void update_arch_timer_irq_lines(struct kvm_vcpu *vcpu) +{ + struct realm_rec *rec =3D &vcpu->arch.rec; + + __vcpu_sys_reg(vcpu, CNTV_CTL_EL0) =3D rec->run->exit.cntv_ctl; + __vcpu_sys_reg(vcpu, CNTV_CVAL_EL0) =3D rec->run->exit.cntv_cval; + __vcpu_sys_reg(vcpu, CNTP_CTL_EL0) =3D rec->run->exit.cntp_ctl; + __vcpu_sys_reg(vcpu, CNTP_CVAL_EL0) =3D rec->run->exit.cntp_cval; + + kvm_realm_timers_update(vcpu); +} + +/* + * Return > 0 to return to guest, < 0 on error, 0 (and set exit_reason) on + * proper exit to userspace. + */ +int handle_rec_exit(struct kvm_vcpu *vcpu, int rec_run_ret) +{ + struct realm_rec *rec =3D &vcpu->arch.rec; + u8 esr_ec =3D ESR_ELx_EC(rec->run->exit.esr); + unsigned long status, index; + + status =3D RMI_RETURN_STATUS(rec_run_ret); + index =3D RMI_RETURN_INDEX(rec_run_ret); + + /* + * If a PSCI_SYSTEM_OFF request raced with a vcpu executing, we might + * see the following status code and index indicating an attempt to run + * a REC when the RD state is SYSTEM_OFF. In this case, we just need to + * return to user space which can deal with the system event or will try + * to run the KVM VCPU again, at which point we will no longer attempt + * to enter the Realm because we will have a sleep request pending on + * the VCPU as a result of KVM's PSCI handling. + */ + if (status =3D=3D RMI_ERROR_REALM && index =3D=3D 1) { + vcpu->run->exit_reason =3D KVM_EXIT_UNKNOWN; + return 0; + } + + if (rec_run_ret) + return -ENXIO; + + vcpu->arch.fault.esr_el2 =3D rec->run->exit.esr; + vcpu->arch.fault.far_el2 =3D rec->run->exit.far; + vcpu->arch.fault.hpfar_el2 =3D rec->run->exit.hpfar; + + update_arch_timer_irq_lines(vcpu); + + /* Reset the emulation flags for the next run of the REC */ + rec->run->enter.flags =3D 0; + + switch (rec->run->exit.exit_reason) { + case RMI_EXIT_SYNC: + return rec_exit_handlers[esr_ec](vcpu); + case RMI_EXIT_IRQ: + case RMI_EXIT_FIQ: + return 1; + case RMI_EXIT_PSCI: + return rec_exit_psci(vcpu); + case RMI_EXIT_RIPAS_CHANGE: + return rec_exit_ripas_change(vcpu); + } + + kvm_pr_unimpl("Unsupported exit reason: %u\n", + rec->run->exit.exit_reason); + vcpu->run->exit_reason =3D KVM_EXIT_INTERNAL_ERROR; + return 0; +} diff --git a/arch/arm64/kvm/rme.c b/arch/arm64/kvm/rme.c index 1fa9991d708b..4c0751231810 100644 --- a/arch/arm64/kvm/rme.c +++ b/arch/arm64/kvm/rme.c @@ -899,6 +899,25 @@ void kvm_destroy_realm(struct kvm *kvm) kvm_free_stage2_pgd(&kvm->arch.mmu); } =20 +int kvm_rec_enter(struct kvm_vcpu *vcpu) +{ + struct realm_rec *rec =3D &vcpu->arch.rec; + + switch (rec->run->exit.exit_reason) { + case RMI_EXIT_HOST_CALL: + case RMI_EXIT_PSCI: + for (int i =3D 0; i < REC_RUN_GPRS; i++) + rec->run->enter.gprs[i] =3D vcpu_get_reg(vcpu, i); + break; + } + + if (kvm_realm_state(vcpu->kvm) !=3D REALM_STATE_ACTIVE) + return -EINVAL; + + return rmi_rec_enter(virt_to_phys(rec->rec_page), + virt_to_phys(rec->run)); +} + static void free_rec_aux(struct page **aux_pages, unsigned int num_aux) { --=20 2.34.1