From nobody Mon Apr 6 18:28:59 2026 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D82D82D7DE4; Wed, 18 Mar 2026 15:57:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=217.140.110.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773849447; cv=none; b=giBMFkz3vv1FdSljrz+Rg02QNKG1DHSmss8s4uNgH4b1DNK3ITCpkjR2AO/AUvtKETkk6JkQgG6SkuAQkA8+AlvbAbce+QXUXeRngUE7pqrcWd74zlM/fOA1ggxEm7q2MgfwJL8u4eAy4xhvtPJx5RgFJfaO8yQ3Qx1ZnVd3o5Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1773849447; c=relaxed/simple; bh=fZtPYshS4jt9m5V6dNmRy6ebCHivCSGGwPHPUhVpHh8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rk1lyAqXMvlIgl5Obms7Gi901r5sLNOAxb52Yu64Q2dlxcBo+ldOXJUPGOcRJxoV45sL1ojECckZsO6wNBdK7f9IhRy+hm9h2s3Hd/IPxPbN2MhdniAkQn1BCWIt+zLPKCEceGnyUC/mgVYY45vZvH5AwreUh4yrVf9xwJvXBMQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com; spf=pass smtp.mailfrom=arm.com; arc=none smtp.client-ip=217.140.110.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=arm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2A38922BE; Wed, 18 Mar 2026 08:57:12 -0700 (PDT) Received: from e122027.arm.com (unknown [10.57.61.122]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 3E6043F73B; Wed, 18 Mar 2026 08:57:15 -0700 (PDT) From: Steven Price To: kvm@vger.kernel.org, kvmarm@lists.linux.dev Cc: Steven Price , Catalin Marinas , Marc Zyngier , Will Deacon , James Morse , Oliver Upton , Suzuki K Poulose , Zenghui Yu , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Joey Gouly , Alexandru Elisei , Christoffer Dall , Fuad Tabba , linux-coco@lists.linux.dev, Ganapatrao Kulkarni , Gavin Shan , Shanker Donthineni , Alper Gun , "Aneesh Kumar K . V" , Emi Kisanuki , Vishal Annapurve Subject: [PATCH v13 48/48] [WIP] arm64: RMI: Add support for SRO Date: Wed, 18 Mar 2026 15:54:12 +0000 Message-ID: <20260318155413.793430-49-steven.price@arm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20260318155413.793430-1-steven.price@arm.com> References: <20260318155413.793430-1-steven.price@arm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" RMM v2.0 introduces the concept of "Stateful RMI Operations" (SRO). This means that an SMC can return with an operation still in progress. The host is expected to continue the operation until it reaches a conclusion (either success or failure). During this process the RMM can request addition memory ('donate') or hand memory back to the host ('reclaim'). The host can request an operation is cancelled, but must still continue the operation until it has completed (otherwise the incomplete operation may cause future RMM operations to fail). SROs may request memory and these operations sometimes have to be performed in places where memory allocation may not be possible. To deal with this a SRO may be started, but then cancelled when the memory allocation cannot be completed. The reclaimed memory will then be stored in a struct rmi_sro_state object with the intention that once Linux has returned to a state where memory allocation is possible, the failed allocation can be reattempted (with GFP flags enabling sleeping and/or direct reclaim) and the SRO operation reattempted (after acquiring the necessary locks). In the worst case this may require several attempts (if the RMM makes several memory requests) but should always make forward progress. This patch is currently a work-in-progress showing the general structure of how this should work and implementing SROs for two operations (RMI_REC_CREATE and RMI_REC_DESTROY). I'm aware there is missing error-checking and there are some details in the specification that need clarifying. These operations are also 'easy' in that we don't have restrictions on memory allocation in these contexts. Signed-off-by: Steven Price --- arch/arm64/include/asm/kvm_rmi.h | 8 - arch/arm64/include/asm/rmi_cmds.h | 71 +++--- arch/arm64/include/asm/rmi_smc.h | 27 +-- arch/arm64/kvm/rmi.c | 385 +++++++++++++++++++++++------- 4 files changed, 347 insertions(+), 144 deletions(-) diff --git a/arch/arm64/include/asm/kvm_rmi.h b/arch/arm64/include/asm/kvm_= rmi.h index b914b9a84bd8..e1f5523c2dfa 100644 --- a/arch/arm64/include/asm/kvm_rmi.h +++ b/arch/arm64/include/asm/kvm_rmi.h @@ -51,7 +51,6 @@ enum realm_state { * @state: The lifetime state machine for the realm * @rd: Kernel mapping of the Realm Descriptor (RD) * @params: Parameters for the RMI_REALM_CREATE command - * @num_aux: The number of auxiliary pages required by the RMM * @ia_bits: Number of valid Input Address bits in the IPA */ struct realm { @@ -60,7 +59,6 @@ struct realm { void *rd; struct realm_params *params; =20 - unsigned long num_aux; unsigned int ia_bits; }; =20 @@ -75,12 +73,6 @@ struct realm { struct realm_rec { unsigned long mpidr; void *rec_page; - /* - * REC_PARAMS_AUX_GRANULES is the maximum number of 4K granules that - * the RMM can require. The array is sized to be large enough for the - * maximum number of host sized pages that could be required. - */ - struct page *aux_pages[(REC_PARAMS_AUX_GRANULES * SZ_4K) >> PAGE_SHIFT]; struct rec_run *run; }; =20 diff --git a/arch/arm64/include/asm/rmi_cmds.h b/arch/arm64/include/asm/rmi= _cmds.h index 9c4f83644a61..b761d74e1273 100644 --- a/arch/arm64/include/asm/rmi_cmds.h +++ b/arch/arm64/include/asm/rmi_cmds.h @@ -17,6 +17,27 @@ struct rtt_entry { int ripas; }; =20 +#define RMI_MAX_ADDR_LIST 256 + +struct rmi_sro_state { + struct arm_smccc_1_2_regs regs; + + unsigned long addr_count; + unsigned long addr_list[RMI_MAX_ADDR_LIST]; +}; + +#define rmi_init_sro(...) ({ \ + struct rmi_sro_state *sro =3D kmalloc_obj(*sro); \ + if (sro) \ + *sro =3D (struct rmi_sro_state){.regs =3D {__VA_ARGS__}}; \ + sro; \ +}) + +#define rmi_smccc(...) do { \ + arm_smccc_1_1_invoke(__VA_ARGS__); \ +} while (RMI_RETURN_STATUS(res.a0) =3D=3D RMI_BUSY || \ + RMI_RETURN_STATUS(res.a0) =3D=3D RMI_BLOCKED) + /** * rmi_rmm_config_get() - Get the system configuration * @cfg_ptr: PA of a struct rmm_config @@ -410,29 +431,7 @@ static inline int rmi_realm_destroy(unsigned long rd) } =20 /** - * rmi_rec_aux_count() - Get number of auxiliary granules required - * @rd: PA of the RD - * @aux_count: Number of granules written to this pointer - * - * A REC may require extra auxiliary granules to be delegated for the RMM = to - * store metadata (not visible to the normal world) in. This function prov= ides - * the number of granules that are required. - * - * Return: RMI return code - */ -static inline int rmi_rec_aux_count(unsigned long rd, unsigned long *aux_c= ount) -{ - struct arm_smccc_res res; - - arm_smccc_1_1_invoke(SMC_RMI_REC_AUX_COUNT, rd, &res); - - if (aux_count) - *aux_count =3D res.a1; - return res.a0; -} - -/** - * rmi_rec_create() - Create a REC + * rmi_rec_create_sro_init() - Init an SRO to create a REC * @rd: PA of the RD * @rec: PA of the target REC * @params: PA of REC parameters @@ -440,33 +439,27 @@ static inline int rmi_rec_aux_count(unsigned long rd,= unsigned long *aux_count) * Create a REC using the parameters specified in the struct rec_params po= inted * to by @params. * - * Return: RMI return code + * Returns: Allocated SRO object */ -static inline int rmi_rec_create(unsigned long rd, unsigned long rec, - unsigned long params) +static inline struct rmi_sro_state * +rmi_rec_create_sro_init(unsigned long rd, + unsigned long rec, + unsigned long params) { - struct arm_smccc_res res; - - arm_smccc_1_1_invoke(SMC_RMI_REC_CREATE, rd, rec, params, &res); - - return res.a0; + return rmi_init_sro(SMC_RMI_REC_CREATE, rd, rec, params); } =20 /** - * rmi_rec_destroy() - Destroy a REC + * rmi_rec_destroy_sro_init() - Init an SRO to destroy a REC * @rec: PA of the target REC * * Destroys a REC. The REC must not be running. * - * Return: RMI return code + * Return: Allocated SRO object */ -static inline int rmi_rec_destroy(unsigned long rec) +static inline struct rmi_sro_state *rmi_rec_destroy_sro_init(unsigned long= rec) { - struct arm_smccc_res res; - - arm_smccc_1_1_invoke(SMC_RMI_REC_DESTROY, rec, &res); - - return res.a0; + return rmi_init_sro(SMC_RMI_REC_DESTROY, rec); } =20 /** diff --git a/arch/arm64/include/asm/rmi_smc.h b/arch/arm64/include/asm/rmi_= smc.h index 049d71470486..fa23818e1b4c 100644 --- a/arch/arm64/include/asm/rmi_smc.h +++ b/arch/arm64/include/asm/rmi_smc.h @@ -38,7 +38,6 @@ #define SMC_RMI_PSCI_COMPLETE SMC_RMI_CALL(0x0164) #define SMC_RMI_FEATURES SMC_RMI_CALL(0x0165) #define SMC_RMI_RTT_FOLD SMC_RMI_CALL(0x0166) -#define SMC_RMI_REC_AUX_COUNT SMC_RMI_CALL(0x0167) // #define SMC_RMI_RTT_INIT_RIPAS SMC_RMI_CALL(0x0168) #define SMC_RMI_RTT_SET_RIPAS SMC_RMI_CALL(0x0169) #define SMC_RMI_VSMMU_CREATE SMC_RMI_CALL(0x016a) @@ -180,11 +179,18 @@ #define RMI_ADDR_TYPE_SINGLE 1 #define RMI_ADDR_TYPE_LIST 2 =20 -#define RMI_ADDR_RANGE_SIZE(ar) (FIELD_GET(GENMASK(1, 0), (ar))) -#define RMI_ADDR_RANGE_COUNT(ar) (FIELD_GET(GENMASK(PAGE_SHIFT - 1, 2), \ +#define RMI_ADDR_RANGE_SIZE_MASK GENMASK(1, 0) +#define RMI_ADDR_RANGE_COUNT_MASK GENMASK(PAGE_SHIFT - 1, 2) +#define RMI_ADDR_RANGE_ADDR_MASK (PAGE_MASK & GENMASK(51, 0)) +#define RMI_ADDR_RANGE_STATE_MASK BIT(63) + +#define RMI_ADDR_RANGE_SIZE(ar) (FIELD_GET(RMI_ADDR_RANGE_SIZE_MASK, \ + (ar))) +#define RMI_ADDR_RANGE_COUNT(ar) (FIELD_GET(RMI_ADDR_RANGE_COUNT_MASK, \ + (ar))) +#define RMI_ADDR_RANGE_ADDR(ar) ((ar) & RMI_ADDR_RANGE_ADDR_MASK) +#define RMI_ADDR_RANGE_STATE(ar) (FIELD_GET(RMI_ADDR_RANGE_STATE_MASK, \ (ar))) -#define RMI_ADDR_RANGE_ADDR(ar) ((ar) & PAGE_MASK & GENMASK(51, 0)) -#define RMI_ADDR_RANGE_STATE(ar) (FIELD_GET(BIT(63), (ar))) =20 enum rmi_ripas { RMI_EMPTY =3D 0, @@ -295,8 +301,6 @@ struct realm_params { =20 #define REC_PARAMS_FLAG_RUNNABLE BIT_ULL(0) =20 -#define REC_PARAMS_AUX_GRANULES 16 - struct rec_params { union { /* 0x0 */ u64 flags; @@ -312,14 +316,7 @@ struct rec_params { }; union { /* 0x300 */ u64 gprs[REC_CREATE_NR_GPRS]; - u8 padding3[0x500]; - }; - union { /* 0x800 */ - struct { - u64 num_rec_aux; - u64 aux[REC_PARAMS_AUX_GRANULES]; - }; - u8 padding4[0x800]; + u8 padding3[0xd00]; }; }; =20 diff --git a/arch/arm64/kvm/rmi.c b/arch/arm64/kvm/rmi.c index e76e58762f55..10ff1c3bddaf 100644 --- a/arch/arm64/kvm/rmi.c +++ b/arch/arm64/kvm/rmi.c @@ -20,6 +20,295 @@ static unsigned long rmm_feat_reg1; =20 #define RMM_L2_BLOCK_SIZE PMD_SIZE =20 +static int delegate_range(phys_addr_t phys, unsigned long size); +static int undelegate_range(phys_addr_t phys, unsigned long size); + +static unsigned long donate_req_to_size(unsigned long donatereq) +{ + unsigned long unit_size =3D RMI_DONATE_SIZE(donatereq); + + switch (unit_size) { + case 0: + return PAGE_SIZE; + case 1: + return PMD_SIZE; + case 2: + return PUD_SIZE; + case 3: + return P4D_SIZE; + } + unreachable(); +} + +static void rmi_smccc_invoke(struct arm_smccc_1_2_regs *regs_in, + struct arm_smccc_1_2_regs *regs_out) +{ + struct arm_smccc_1_2_regs regs =3D *regs_in; + unsigned long status; + + do { + arm_smccc_1_2_invoke(®s, regs_out); + status =3D RMI_RETURN_STATUS(regs_out->a0); + } while (status =3D=3D RMI_BUSY || status =3D=3D RMI_BLOCKED); +} + +static int rmi_sro_donate_contig(struct rmi_sro_state *sro, + unsigned long sro_handle, + unsigned long donatereq, + struct arm_smccc_1_2_regs *out_regs, + gfp_t gfp) +{ + unsigned long unit_size =3D RMI_DONATE_SIZE(donatereq); + unsigned long count =3D RMI_DONATE_COUNT(donatereq); + unsigned long state =3D RMI_DONATE_STATE(donatereq); + unsigned long size; + unsigned long addr_range; + struct page *pages; + phys_addr_t phys; + struct arm_smccc_1_2_regs regs =3D { + SMC_RMI_OP_MEM_DONATE, + sro_handle + }; + + for (int i =3D 0; i < sro->addr_count; i++) { + unsigned long entry =3D sro->addr_list[i]; + + if (RMI_ADDR_RANGE_SIZE(entry) =3D=3D unit_size && + RMI_ADDR_RANGE_COUNT(entry) =3D=3D count && + RMI_ADDR_RANGE_STATE(entry) =3D=3D state) { + sro->addr_count--; + swap(sro->addr_list[sro->addr_count], + sro->addr_list[i]); + + goto out; + } + } + + size =3D donate_req_to_size(donatereq) * count; + + pages =3D alloc_pages(gfp, get_order(size)); + if (!pages) + return -ENOMEM; + phys =3D page_to_phys(pages); + + if (state =3D=3D RMI_OP_MEM_DELEGATED) { + if (delegate_range(phys, size)) { + __free_pages(pages, get_order(size)); + return -ENXIO; + } + } + + addr_range =3D phys & RMI_ADDR_RANGE_ADDR_MASK; + FIELD_MODIFY(RMI_ADDR_RANGE_SIZE_MASK, &addr_range, unit_size); + FIELD_MODIFY(RMI_ADDR_RANGE_COUNT_MASK, &addr_range, count); + FIELD_MODIFY(RMI_ADDR_RANGE_STATE_MASK, &addr_range, state); + + sro->addr_list[sro->addr_count] =3D addr_range; + +out: + regs.a2 =3D virt_to_phys(&sro->addr_list[sro->addr_count]); + regs.a3 =3D 1; + rmi_smccc_invoke(®s, out_regs); + + unsigned long donated_granules =3D out_regs->a1; + + WARN_ON(donated_granules > 1); + if (WARN_ON(donated_granules =3D=3D 0)) { + sro->addr_count++; + return 0; + } + + return 0; +} + +static int rmi_sro_donate_noncontig(struct rmi_sro_state *sro, + unsigned long sro_handle, + unsigned long donatereq, + struct arm_smccc_1_2_regs *out_regs, + gfp_t gfp) +{ + unsigned long unit_size =3D RMI_DONATE_SIZE(donatereq); + unsigned long count =3D RMI_DONATE_COUNT(donatereq); + unsigned long state =3D RMI_DONATE_STATE(donatereq); + unsigned long found =3D 0; + unsigned long addr_list_start =3D sro->addr_count; + struct arm_smccc_1_2_regs regs =3D { + SMC_RMI_OP_MEM_DONATE, + sro_handle + }; + + for (int i =3D 0; i < addr_list_start && found < count; i++) { + unsigned long entry =3D sro->addr_list[i]; + + if (RMI_ADDR_RANGE_SIZE(entry) =3D=3D unit_size && + RMI_ADDR_RANGE_COUNT(entry) =3D=3D 1 && + RMI_ADDR_RANGE_STATE(entry) =3D=3D state) { + addr_list_start--; + swap(sro->addr_list[addr_list_start], + sro->addr_list[i]); + found++; + i--; + } + } + + while (found < count) { + unsigned long addr_range; + unsigned long size =3D donate_req_to_size(donatereq); + + struct page *pages =3D alloc_pages(gfp, get_order(size)); + phys_addr_t phys; + + if (!pages) + return -ENOMEM; + + phys =3D page_to_phys(pages); + + if (state =3D=3D RMI_OP_MEM_DELEGATED) { + if (delegate_range(phys, size)) { + __free_pages(pages, get_order(size)); + return -ENXIO; + } + } + + addr_range =3D phys & RMI_ADDR_RANGE_ADDR_MASK; + FIELD_MODIFY(RMI_ADDR_RANGE_SIZE_MASK, &addr_range, unit_size); + FIELD_MODIFY(RMI_ADDR_RANGE_COUNT_MASK, &addr_range, 1); + FIELD_MODIFY(RMI_ADDR_RANGE_STATE_MASK, &addr_range, state); + + sro->addr_list[sro->addr_count++] =3D addr_range; + found++; + } + + regs.a2 =3D virt_to_phys(&sro->addr_list[addr_list_start]); + regs.a3 =3D found; + rmi_smccc_invoke(®s, out_regs); + + unsigned long donated_granules =3D out_regs->a1; + + while (donated_granules < found) { + swap(sro->addr_list[addr_list_start++], + sro->addr_list[--sro->addr_count]); + found--; + } + sro->addr_count -=3D donated_granules; + + return 0; +} + +static int rmi_sro_donate(struct rmi_sro_state *sro, + unsigned long sro_handle, + unsigned long donatereq, + struct arm_smccc_1_2_regs *regs, + gfp_t gfp) +{ + unsigned long count =3D RMI_DONATE_COUNT(donatereq); + + if (WARN_ON(!count)) + return 0; + + if (RMI_DONATE_CONTIG(donatereq)) { + return rmi_sro_donate_contig(sro, sro_handle, donatereq, + regs, gfp); + } else { + return rmi_sro_donate_noncontig(sro, sro_handle, donatereq, + regs, gfp); + } +} + +static int rmi_sro_reclaim(struct rmi_sro_state *sro, + unsigned long sro_handle, + struct arm_smccc_1_2_regs *out_regs) +{ + struct arm_smccc_1_2_regs regs =3D { + SMC_RMI_OP_MEM_RECLAIM, + sro_handle, + virt_to_phys(&sro->addr_list[sro->addr_count]), + RMI_MAX_ADDR_LIST - sro->addr_count + }; + rmi_smccc_invoke(®s, out_regs); + sro->addr_count +=3D out_regs->a1; + + return 0; +} + +static void rmi_sro_free(struct rmi_sro_state *sro) +{ + for (int i =3D 0; i < sro->addr_count; i++) { + unsigned long entry =3D sro->addr_list[i]; + unsigned long addr =3D RMI_ADDR_RANGE_ADDR(entry); + unsigned long unit_size =3D RMI_ADDR_RANGE_SIZE(entry); + unsigned long count =3D RMI_ADDR_RANGE_COUNT(entry); + unsigned long state =3D RMI_ADDR_RANGE_STATE(entry); + unsigned long size =3D donate_req_to_size(unit_size) * count; + + if (state =3D=3D RMI_OP_MEM_DELEGATED) { + if (WARN_ON(undelegate_range(addr, size))) { + /* Leak the pages */ + continue; + } + } + __free_pages(phys_to_page(addr), get_order(size)); + } + + sro->addr_count =3D 0; +} + +DEFINE_FREE(sro, struct rmi_sro_state *, if (_T) rmi_sro_free(_T)) + +static unsigned long rmi_sro_execute(struct rmi_sro_state *sro) +{ + unsigned long sro_handle; + struct arm_smccc_1_2_regs regs; + struct arm_smccc_1_2_regs *regs_in =3D &sro->regs; + + rmi_smccc_invoke(regs_in, ®s); + + sro_handle =3D regs.a1; + + while (RMI_RETURN_STATUS(regs.a0) =3D=3D RMI_INCOMPLETE) { + bool can_cancel =3D RMI_RETURN_CANCANCEL(regs.a0); + int ret; + + switch (RMI_RETURN_MEMREQ(regs.a0)) { + case RMI_OP_MEM_REQ_NONE: + regs =3D (struct arm_smccc_1_2_regs){ + SMC_RMI_OP_CONTINUE, sro_handle, 0 + }; + rmi_smccc_invoke(®s, ®s); + break; + case RMI_OP_MEM_REQ_DONATE: + ret =3D rmi_sro_donate(sro, sro_handle, regs.a2, ®s, + GFP_KERNEL); + break; + case RMI_OP_MEM_REQ_RECLAIM: + ret =3D rmi_sro_reclaim(sro, sro_handle, ®s); + break; + default: + ret =3D WARN_ON(1); + break; + } + + if (ret) { + if (can_cancel) { + /* + * FIXME: Handle cancelling properly! + * + * If the operation has failed due to memory + * allocation failure then the information on + * the memory allocation should be saved, so + * that the allocation can be repeated outside + * of any context which prevented the + * allocation. + */ + } + if (WARN_ON(ret)) + return ret; + } + } + + return regs.a0; +} + static inline unsigned long rmi_rtt_level_mapsize(int level) { if (WARN_ON(level > RMM_RTT_MAX_LEVEL)) @@ -795,12 +1084,6 @@ static int realm_create_rd(struct kvm *kvm) goto out_undelegate_tables; } =20 - if (WARN_ON(rmi_rec_aux_count(rd_phys, &realm->num_aux))) { - WARN_ON(rmi_realm_destroy(rd_phys)); - r =3D -ENXIO; - goto out_undelegate_tables; - } - realm->rd =3D rd; WRITE_ONCE(realm->state, REALM_STATE_NEW); /* The realm is up, free the parameters. */ @@ -1432,65 +1715,9 @@ int noinstr kvm_rec_enter(struct kvm_vcpu *vcpu) return ret; } =20 -static void free_rec_aux(struct page **aux_pages, - unsigned int num_aux) -{ - unsigned int i; - unsigned int page_count =3D 0; - - for (i =3D 0; i < num_aux; i++) { - struct page *aux_page =3D aux_pages[page_count++]; - phys_addr_t aux_page_phys =3D page_to_phys(aux_page); - - if (!WARN_ON(undelegate_page(aux_page_phys))) - __free_page(aux_page); - aux_page_phys +=3D PAGE_SIZE; - } -} - -static int alloc_rec_aux(struct page **aux_pages, - u64 *aux_phys_pages, - unsigned int num_aux) -{ - struct page *aux_page; - unsigned int i; - int ret; - - for (i =3D 0; i < num_aux; i++) { - phys_addr_t aux_page_phys; - - aux_page =3D alloc_page(GFP_KERNEL); - if (!aux_page) { - ret =3D -ENOMEM; - goto out_err; - } - - aux_page_phys =3D page_to_phys(aux_page); - if (delegate_page(aux_page_phys)) { - ret =3D -ENXIO; - goto err_undelegate; - } - aux_phys_pages[i] =3D aux_page_phys; - aux_pages[i] =3D aux_page; - } - - return 0; -err_undelegate: - while (i > 0) { - i--; - if (WARN_ON(undelegate_page(aux_phys_pages[i]))) { - /* Leak the page if the undelegate fails */ - goto out_err; - } - } - __free_page(aux_page); -out_err: - free_rec_aux(aux_pages, i); - return ret; -} - static int kvm_create_rec(struct kvm_vcpu *vcpu) { + struct rmi_sro_state *sro __free(sro) =3D NULL; struct user_pt_regs *vcpu_regs =3D vcpu_gp_regs(vcpu); unsigned long mpidr =3D kvm_vcpu_get_mpidr_aff(vcpu); struct realm *realm =3D &vcpu->kvm->arch.realm; @@ -1538,18 +1765,17 @@ static int kvm_create_rec(struct kvm_vcpu *vcpu) goto out_free_pages; } =20 - r =3D alloc_rec_aux(rec->aux_pages, params->aux, realm->num_aux); - if (r) - goto out_undelegate_rmm_rec; - - params->num_rec_aux =3D realm->num_aux; params->mpidr =3D mpidr; =20 - if (rmi_rec_create(virt_to_phys(realm->rd), - rec_page_phys, - virt_to_phys(params))) { + sro =3D rmi_rec_create_sro_init(virt_to_phys(realm->rd), + rec_page_phys, virt_to_phys(params)); + if (!sro) { + r =3D -ENOMEM; + goto out_undelegate_rmm_rec; + } + if (rmi_sro_execute(sro)) { r =3D -ENXIO; - goto out_free_rec_aux; + goto out_undelegate_rmm_rec; } =20 rec->mpidr =3D mpidr; @@ -1557,8 +1783,6 @@ static int kvm_create_rec(struct kvm_vcpu *vcpu) free_page((unsigned long)params); return 0; =20 -out_free_rec_aux: - free_rec_aux(rec->aux_pages, realm->num_aux); out_undelegate_rmm_rec: if (WARN_ON(undelegate_page(rec_page_phys))) rec->rec_page =3D NULL; @@ -1572,7 +1796,7 @@ static int kvm_create_rec(struct kvm_vcpu *vcpu) =20 void kvm_destroy_rec(struct kvm_vcpu *vcpu) { - struct realm *realm =3D &vcpu->kvm->arch.realm; + struct rmi_sro_state *sro __free(sro) =3D NULL; struct realm_rec *rec =3D &vcpu->arch.rec; unsigned long rec_page_phys; =20 @@ -1588,15 +1812,12 @@ void kvm_destroy_rec(struct kvm_vcpu *vcpu) =20 rec_page_phys =3D virt_to_phys(rec->rec_page); =20 - /* - * The REC and any AUX pages cannot be reclaimed until the REC is - * destroyed. So if the REC destroy fails then the REC page and any AUX - * pages will be leaked. - */ - if (WARN_ON(rmi_rec_destroy(rec_page_phys))) + sro =3D rmi_rec_destroy_sro_init(rec_page_phys); + if (WARN_ON(!sro)) return; =20 - free_rec_aux(rec->aux_pages, realm->num_aux); + if (WARN_ON(rmi_sro_execute(sro))) + return; =20 free_delegated_page(rec_page_phys); } --=20 2.43.0