From nobody Mon Sep 28 06:34:45 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CF206318EEE; Tue, 25 Aug 2026 16:01:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787673681; cv=none; b=KA+y8RHMdgUfuoo8zuB33hQBA2yOrAo867JDzlRDjXtaKVHW4rqiA8Ig+yTzQm0Em2J+8LHIEm3rm7duCL1nX/mljj/A4Ba22eCWX3WAk7Mi/jKFsWPSQ6oWY5qZYRAyYp5rAioyfbF7nyjpCXxHlhhOv/dpYaimO58HSXQV/kY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787673681; c=relaxed/simple; bh=btC+NUubidig9Ed/d2ywLChFedyhilKG2Dr7V4xBm7s=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=D6gkozWkfTgErNjXnfuv1s81QPNiYumGDc0zhvDZD30ZVc+ZYMMdwFhUK3AiInrQWsqXET3QEH9A6aR0mDo0xPUp8JxF5FkypCy5cDkTtb7IGCisrNxCvFGkyiYAIXQnlwDveZnvJ3Md4igq3qXfHzCn2UOf1u3oHXV0mM6XBr4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=fyGw1Fw3; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="fyGw1Fw3" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C359A1F00A3A; Tue, 25 Aug 2026 16:01:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1787673679; bh=RCpZ6I8gJKC84YSYpuYu+qK0qlSMETmmTK2jYvJjU/M=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=fyGw1Fw3VY1oc8Q/kFgwnp4YgxP5B8a521/LXjnvvkbQDm5Zxx5mCzZwqNefHTn2j jJ3NeLgzsmLkFQdX2ZhB2X7iCy6B4YFFG7o9ca6yKN3EpnBwbgjb/Am6bjEAtYA5ct rDaJSCPSKTMgYhpbsyBEnQyr50HN3EfE3lkGWbK3KRgHC5QhCvJZKGaRnNjHnUnhV2 G1iGYchCDv3g5fii58hD4i8YL5W0nIt0p20HIL3UoArG3ZAjnl51xwtnPsfRsgPWzr J2RMIXfAzf3FK2TgtaATJO/a02CVo1u6xEPZ0Gpwc4ApVW47R4I+U0KPEs+7UTGCD1 /FLss+AplflKQ== From: "Lorenzo Stoakes (ARM)" Date: Tue, 25 Aug 2026 17:00:35 +0100 Subject: [PATCH 1/8] KVM: arm64: Propagate and use esr in s2fd when handling guest aborts Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260825-kvm-arm-prefault-v1-1-befe8947702e@kernel.org> References: <20260825-kvm-arm-prefault-v1-0-befe8947702e@kernel.org> In-Reply-To: <20260825-kvm-arm-prefault-v1-0-befe8947702e@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , Fuad Tabba , Joey Gouly , Steffen Eiden , Suzuki K Poulose , Zenghui Yu , Paolo Bonzini , Jonathan Corbet Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Jack Thomson , Jack Thomson , Alexandru Elisei , Vincent Donnefort , "Aneesh Kumar K.V" , Sean Christopherson , Claudio Imbrenda , Leo Soares Passos , "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=19029; i=ljs@kernel.org; h=from:subject:message-id; bh=btC+NUubidig9Ed/d2ywLChFedyhilKG2Dr7V4xBm7s=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLJ69zg1NgamZcllhrX3rGYxFnPY+vGy55QfCzkzjVOTZ k48wLC2o5SFQYyLQVZMkeX5F/H9QSJh8zov+LvBzGFlAhnCwMUpABPR4mRkWL5v/oZJzyyuVt87 79BYwP4gd7FBvp1F7Pav999UB03ff4Phr+Tb9Ym6exN6fz35c+9rtNyuCv5Zi665ad2/1l3sFLt OlQsA X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 kvm_handle_guest_abort() establishes a kvm_s2_fault_desc data structure, s2fd, to store and propagate state to either pkvm_mem_abort(), gmem_abort() or user_mem_abort() handlers. Each of these, however, examines the Exception Syndrome Register (ESR) via s2fd->vcpu. Introduce an s2fd->esr field to abstract this and propagate it to callers. The value of this (beyond refactoring) is to be able to later generate faults with a synthetic esr, specifically to implement stage 2 page table pre-faulting. Abstract esr-specific predicates and helpers to the esr.h header and either have vcpu wrappers call these, or eliminate them if they are not used elsewhere. Provide kvm_s2_fault_is_[write,exec,perm]() helpers for convenience. Since kvm_s2_fault_map() either sets perm_fault_granule to the permission fault granule or 0 if not a permission fault, implement kvm_s2_perm_fault_granule() to do this directly. Abort handlers which use kvm_s2_fault_desc - gmem_abort() and user_mem_abort() - now only reference s2fd->esr and do not look it up in any other way, which makes it safe to pass a synthetic s2fd->esr value to these functions. No functional change intended. Signed-off-by: Lorenzo Stoakes (ARM) Tested-by: Fuad Tabba < fuad.tabba@linux.dev> --- arch/arm64/include/asm/esr.h | 129 +++++++++++++++++++++++++------= ---- arch/arm64/include/asm/kvm_emulate.h | 52 ++++---------- arch/arm64/kvm/mmu.c | 80 +++++++++++++--------- 3 files changed, 156 insertions(+), 105 deletions(-) diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h index f816f5d77f1a..162e90c832e9 100644 --- a/arch/arm64/include/asm/esr.h +++ b/arch/arm64/include/asm/esr.h @@ -437,6 +437,32 @@ #ifndef __ASSEMBLER__ #include =20 +static inline u8 esr_trap_get_class(unsigned long esr) +{ + return ESR_ELx_EC(esr); +} + +static inline bool esr_trap_is_iabt(unsigned long esr) +{ + return esr_trap_get_class(esr) =3D=3D ESR_ELx_EC_IABT_LOW; +} + +static inline bool esr_abt_is_s1ptw(unsigned long esr) +{ + return esr & ESR_ELx_S1PTW; +} + +/* Always check for S1PTW *before* using this. */ +static inline bool esr_dabt_is_write(unsigned long esr) +{ + return esr & ESR_ELx_WNR; +} + +static inline bool esr_dabt_is_cm(unsigned long esr) +{ + return esr & ESR_ELx_CM; +} + static inline unsigned long esr_brk_comment(unsigned long esr) { return esr & ESR_ELx_BRK64_ISS_COMMENT_MASK; @@ -460,75 +486,104 @@ static inline bool esr_is_ubsan_brk(unsigned long es= r) return (esr_brk_comment(esr) & ~UBSAN_BRK_MASK) =3D=3D UBSAN_BRK_IMM; } =20 +static inline u8 esr_fsc_get_fault(unsigned long esr) +{ + return esr & ESR_ELx_FSC; +} + static inline bool esr_fsc_is_translation_fault(unsigned long esr) { - esr =3D esr & ESR_ELx_FSC; + const u8 fault =3D esr_fsc_get_fault(esr); =20 - return (esr =3D=3D ESR_ELx_FSC_FAULT_L(3)) || - (esr =3D=3D ESR_ELx_FSC_FAULT_L(2)) || - (esr =3D=3D ESR_ELx_FSC_FAULT_L(1)) || - (esr =3D=3D ESR_ELx_FSC_FAULT_L(0)) || - (esr =3D=3D ESR_ELx_FSC_FAULT_L(-1)); + return (fault =3D=3D ESR_ELx_FSC_FAULT_L(3)) || + (fault =3D=3D ESR_ELx_FSC_FAULT_L(2)) || + (fault =3D=3D ESR_ELx_FSC_FAULT_L(1)) || + (fault =3D=3D ESR_ELx_FSC_FAULT_L(0)) || + (fault =3D=3D ESR_ELx_FSC_FAULT_L(-1)); } =20 static inline bool esr_fsc_is_permission_fault(unsigned long esr) { - esr =3D esr & ESR_ELx_FSC; + const u8 fault =3D esr_fsc_get_fault(esr); =20 - return (esr =3D=3D ESR_ELx_FSC_PERM_L(3)) || - (esr =3D=3D ESR_ELx_FSC_PERM_L(2)) || - (esr =3D=3D ESR_ELx_FSC_PERM_L(1)) || - (esr =3D=3D ESR_ELx_FSC_PERM_L(0)); + return (fault =3D=3D ESR_ELx_FSC_PERM_L(3)) || + (fault =3D=3D ESR_ELx_FSC_PERM_L(2)) || + (fault =3D=3D ESR_ELx_FSC_PERM_L(1)) || + (fault =3D=3D ESR_ELx_FSC_PERM_L(0)); } =20 static inline bool esr_fsc_is_access_flag_fault(unsigned long esr) { - esr =3D esr & ESR_ELx_FSC; + const u8 fault =3D esr_fsc_get_fault(esr); =20 - return (esr =3D=3D ESR_ELx_FSC_ACCESS_L(3)) || - (esr =3D=3D ESR_ELx_FSC_ACCESS_L(2)) || - (esr =3D=3D ESR_ELx_FSC_ACCESS_L(1)) || - (esr =3D=3D ESR_ELx_FSC_ACCESS_L(0)); + return (fault =3D=3D ESR_ELx_FSC_ACCESS_L(3)) || + (fault =3D=3D ESR_ELx_FSC_ACCESS_L(2)) || + (fault =3D=3D ESR_ELx_FSC_ACCESS_L(1)) || + (fault =3D=3D ESR_ELx_FSC_ACCESS_L(0)); } =20 static inline bool esr_fsc_is_excl_atomic_fault(unsigned long esr) { - esr =3D esr & ESR_ELx_FSC; - - return esr =3D=3D ESR_ELx_FSC_EXCL_ATOMIC; + return esr_fsc_get_fault(esr) =3D=3D ESR_ELx_FSC_EXCL_ATOMIC; } =20 static inline bool esr_fsc_is_addr_sz_fault(unsigned long esr) { - esr &=3D ESR_ELx_FSC; + const u8 fault =3D esr_fsc_get_fault(esr); + + return (fault =3D=3D ESR_ELx_FSC_ADDRSZ_L(3)) || + (fault =3D=3D ESR_ELx_FSC_ADDRSZ_L(2)) || + (fault =3D=3D ESR_ELx_FSC_ADDRSZ_L(1)) || + (fault =3D=3D ESR_ELx_FSC_ADDRSZ_L(0)) || + (fault =3D=3D ESR_ELx_FSC_ADDRSZ_L(-1)); +} + +static inline bool esr_abt_is_exec_fault(unsigned long esr) +{ + return esr_trap_is_iabt(esr) && !esr_abt_is_s1ptw(esr); +} =20 - return (esr =3D=3D ESR_ELx_FSC_ADDRSZ_L(3)) || - (esr =3D=3D ESR_ELx_FSC_ADDRSZ_L(2)) || - (esr =3D=3D ESR_ELx_FSC_ADDRSZ_L(1)) || - (esr =3D=3D ESR_ELx_FSC_ADDRSZ_L(0)) || - (esr =3D=3D ESR_ELx_FSC_ADDRSZ_L(-1)); +static inline bool esr_abt_is_sea(unsigned long esr) +{ + const u8 fault =3D esr_fsc_get_fault(esr); + + switch (fault) { + case ESR_ELx_FSC_EXTABT: + case ESR_ELx_FSC_SEA_TTW(-1) ... ESR_ELx_FSC_SEA_TTW(3): + case ESR_ELx_FSC_SECC: + case ESR_ELx_FSC_SECC_TTW(-1) ... ESR_ELx_FSC_SECC_TTW(3): + return true; + default: + return false; + } +} + +/* Not valid for negative levels. */ +static inline u64 esr_fsc_get_level(unsigned long esr) +{ + return esr & ESR_ELx_FSC_LEVEL; } =20 static inline bool esr_fsc_is_sea_ttw(unsigned long esr) { - esr =3D esr & ESR_ELx_FSC; + const u8 fault =3D esr_fsc_get_fault(esr); =20 - return (esr =3D=3D ESR_ELx_FSC_SEA_TTW(3)) || - (esr =3D=3D ESR_ELx_FSC_SEA_TTW(2)) || - (esr =3D=3D ESR_ELx_FSC_SEA_TTW(1)) || - (esr =3D=3D ESR_ELx_FSC_SEA_TTW(0)) || - (esr =3D=3D ESR_ELx_FSC_SEA_TTW(-1)); + return (fault =3D=3D ESR_ELx_FSC_SEA_TTW(3)) || + (fault =3D=3D ESR_ELx_FSC_SEA_TTW(2)) || + (fault =3D=3D ESR_ELx_FSC_SEA_TTW(1)) || + (fault =3D=3D ESR_ELx_FSC_SEA_TTW(0)) || + (fault =3D=3D ESR_ELx_FSC_SEA_TTW(-1)); } =20 static inline bool esr_fsc_is_secc_ttw(unsigned long esr) { - esr =3D esr & ESR_ELx_FSC; + const u8 fault =3D esr_fsc_get_fault(esr); =20 - return (esr =3D=3D ESR_ELx_FSC_SECC_TTW(3)) || - (esr =3D=3D ESR_ELx_FSC_SECC_TTW(2)) || - (esr =3D=3D ESR_ELx_FSC_SECC_TTW(1)) || - (esr =3D=3D ESR_ELx_FSC_SECC_TTW(0)) || - (esr =3D=3D ESR_ELx_FSC_SECC_TTW(-1)); + return (fault =3D=3D ESR_ELx_FSC_SECC_TTW(3)) || + (fault =3D=3D ESR_ELx_FSC_SECC_TTW(2)) || + (fault =3D=3D ESR_ELx_FSC_SECC_TTW(1)) || + (fault =3D=3D ESR_ELx_FSC_SECC_TTW(0)) || + (fault =3D=3D ESR_ELx_FSC_SECC_TTW(-1)); } =20 /* Indicate whether ESR.EC=3D=3D0x1A is for an ERETAx instruction */ diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/= kvm_emulate.h index a3c1928bdf74..811d7a68a9f9 100644 --- a/arch/arm64/include/asm/kvm_emulate.h +++ b/arch/arm64/include/asm/kvm_emulate.h @@ -411,18 +411,13 @@ static __always_inline int kvm_vcpu_dabt_get_rd(const= struct kvm_vcpu *vcpu) =20 static __always_inline bool kvm_vcpu_abt_iss1tw(const struct kvm_vcpu *vcp= u) { - return !!(kvm_vcpu_get_esr(vcpu) & ESR_ELx_S1PTW); + return esr_abt_is_s1ptw(kvm_vcpu_get_esr(vcpu)); } =20 /* Always check for S1PTW *before* using this. */ static __always_inline bool kvm_vcpu_dabt_iswrite(const struct kvm_vcpu *v= cpu) { - return kvm_vcpu_get_esr(vcpu) & ESR_ELx_WNR; -} - -static inline bool kvm_vcpu_dabt_is_cm(const struct kvm_vcpu *vcpu) -{ - return !!(kvm_vcpu_get_esr(vcpu) & ESR_ELx_CM); + return esr_dabt_is_write(kvm_vcpu_get_esr(vcpu)); } =20 static __always_inline unsigned int kvm_vcpu_dabt_get_as(const struct kvm_= vcpu *vcpu) @@ -438,17 +433,12 @@ static __always_inline bool kvm_vcpu_trap_il_is32bit(= const struct kvm_vcpu *vcpu =20 static __always_inline u8 kvm_vcpu_trap_get_class(const struct kvm_vcpu *v= cpu) { - return ESR_ELx_EC(kvm_vcpu_get_esr(vcpu)); + return esr_trap_get_class(kvm_vcpu_get_esr(vcpu)); } =20 static inline bool kvm_vcpu_trap_is_iabt(const struct kvm_vcpu *vcpu) { - return kvm_vcpu_trap_get_class(vcpu) =3D=3D ESR_ELx_EC_IABT_LOW; -} - -static inline bool kvm_vcpu_trap_is_exec_fault(const struct kvm_vcpu *vcpu) -{ - return kvm_vcpu_trap_is_iabt(vcpu) && !kvm_vcpu_abt_iss1tw(vcpu); + return esr_trap_is_iabt(kvm_vcpu_get_esr(vcpu)); } =20 static __always_inline u8 kvm_vcpu_trap_get_fault(const struct kvm_vcpu *v= cpu) @@ -468,26 +458,9 @@ bool kvm_vcpu_trap_is_translation_fault(const struct k= vm_vcpu *vcpu) return esr_fsc_is_translation_fault(kvm_vcpu_get_esr(vcpu)); } =20 -static inline -u64 kvm_vcpu_trap_get_perm_fault_granule(const struct kvm_vcpu *vcpu) -{ - unsigned long esr =3D kvm_vcpu_get_esr(vcpu); - - BUG_ON(!esr_fsc_is_permission_fault(esr)); - return BIT(ARM64_HW_PGTABLE_LEVEL_SHIFT(esr & ESR_ELx_FSC_LEVEL)); -} - static __always_inline bool kvm_vcpu_abt_issea(const struct kvm_vcpu *vcpu) { - switch (kvm_vcpu_trap_get_fault(vcpu)) { - case ESR_ELx_FSC_EXTABT: - case ESR_ELx_FSC_SEA_TTW(-1) ... ESR_ELx_FSC_SEA_TTW(3): - case ESR_ELx_FSC_SECC: - case ESR_ELx_FSC_SECC_TTW(-1) ... ESR_ELx_FSC_SECC_TTW(3): - return true; - default: - return false; - } + return esr_abt_is_sea(kvm_vcpu_get_esr(vcpu)); } =20 static __always_inline int kvm_vcpu_sys_get_rt(struct kvm_vcpu *vcpu) @@ -496,9 +469,9 @@ static __always_inline int kvm_vcpu_sys_get_rt(struct k= vm_vcpu *vcpu) return ESR_ELx_SYS64_ISS_RT(esr); } =20 -static inline bool kvm_is_write_fault(struct kvm_vcpu *vcpu) +static inline bool esr_abt_is_write_fault(unsigned long esr) { - if (kvm_vcpu_abt_iss1tw(vcpu)) { + if (esr_abt_is_s1ptw(esr)) { /* * Only a permission fault on a S1PTW should be * considered as a write. Otherwise, page tables baked @@ -511,13 +484,18 @@ static inline bool kvm_is_write_fault(struct kvm_vcpu= *vcpu) * first), then a permission fault to allow the flags * to be set. */ - return kvm_vcpu_trap_is_permission_fault(vcpu); + return esr_fsc_is_permission_fault(esr); } =20 - if (kvm_vcpu_trap_is_iabt(vcpu)) + if (esr_trap_is_iabt(esr)) return false; =20 - return kvm_vcpu_dabt_iswrite(vcpu); + return esr_dabt_is_write(esr); +} + +static inline bool kvm_is_write_fault(struct kvm_vcpu *vcpu) +{ + return esr_abt_is_write_fault(kvm_vcpu_get_esr(vcpu)); } =20 static inline unsigned long kvm_vcpu_get_mpidr_aff(struct kvm_vcpu *vcpu) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 74e7e7f7564c..30d605e87b01 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1603,12 +1603,38 @@ struct kvm_s2_fault_desc { struct kvm_s2_trans *nested; struct kvm_memory_slot *memslot; unsigned long hva; + unsigned long esr; }; =20 +static bool kvm_s2_fault_is_perm(const struct kvm_s2_fault_desc *s2fd) +{ + return esr_fsc_is_permission_fault(s2fd->esr); +} + +static bool kvm_s2_fault_is_exec(const struct kvm_s2_fault_desc *s2fd) +{ + return esr_abt_is_exec_fault(s2fd->esr); +} + +static bool kvm_s2_fault_is_write(const struct kvm_s2_fault_desc *s2fd) +{ + return esr_abt_is_write_fault(s2fd->esr); +} + +static u64 kvm_s2_perm_fault_granule(const struct kvm_s2_fault_desc *s2fd) +{ + u64 level; + + if (!kvm_s2_fault_is_perm(s2fd)) + return 0; + level =3D esr_fsc_get_level(s2fd->esr); + return BIT(ARM64_HW_PGTABLE_LEVEL_SHIFT(level)); +} + static int gmem_abort(const struct kvm_s2_fault_desc *s2fd) { bool write_fault, exec_fault; - bool perm_fault =3D kvm_vcpu_trap_is_permission_fault(s2fd->vcpu); + const bool perm_fault =3D kvm_s2_fault_is_perm(s2fd); enum kvm_pgtable_walk_flags flags =3D KVM_PGTABLE_WALK_SHARED; enum kvm_pgtable_prot prot =3D KVM_PGTABLE_PROT_R; struct kvm_pgtable *pgt =3D s2fd->vcpu->arch.hw_mmu->pgt; @@ -1632,8 +1658,8 @@ static int gmem_abort(const struct kvm_s2_fault_desc = *s2fd) else gfn =3D s2fd->fault_ipa >> PAGE_SHIFT; =20 - write_fault =3D kvm_is_write_fault(s2fd->vcpu); - exec_fault =3D kvm_vcpu_trap_is_exec_fault(s2fd->vcpu); + write_fault =3D kvm_s2_fault_is_write(s2fd); + exec_fault =3D kvm_s2_fault_is_exec(s2fd); =20 VM_WARN_ON_ONCE(write_fault && exec_fault); =20 @@ -1840,11 +1866,6 @@ static short kvm_s2_resolve_vma_size(const struct kv= m_s2_fault_desc *s2fd, return vma_shift; } =20 -static bool kvm_s2_fault_is_perm(const struct kvm_s2_fault_desc *s2fd) -{ - return kvm_vcpu_trap_is_permission_fault(s2fd->vcpu); -} - static int kvm_s2_fault_get_vma_info(const struct kvm_s2_fault_desc *s2fd, struct kvm_s2_fault_vma_info *s2vi) { @@ -1910,7 +1931,7 @@ static int kvm_s2_fault_pin_pfn(const struct kvm_s2_f= ault_desc *s2fd, return ret; =20 s2vi->pfn =3D __kvm_faultin_pfn(s2fd->memslot, get_canonical_gfn(s2fd, s2= vi), - kvm_is_write_fault(s2fd->vcpu) ? FOLL_WRITE : 0, + kvm_s2_fault_is_write(s2fd) ? FOLL_WRITE : 0, &s2vi->map_writable, &s2vi->page); if (unlikely(is_error_noslot_pfn(s2vi->pfn))) { if (s2vi->pfn =3D=3D KVM_PFN_ERR_HWPOISON) { @@ -1968,7 +1989,7 @@ static int kvm_s2_fault_compute_prot(const struct kvm= _s2_fault_desc *s2fd, { struct kvm *kvm =3D s2fd->vcpu->kvm; =20 - if (kvm_vcpu_trap_is_exec_fault(s2fd->vcpu) && s2vi->map_non_cacheable) + if (kvm_s2_fault_is_exec(s2fd) && s2vi->map_non_cacheable) return -ENOEXEC; =20 /* @@ -1977,7 +1998,7 @@ static int kvm_s2_fault_compute_prot(const struct kvm= _s2_fault_desc *s2fd, * and trigger the exception here. Since the memslot is valid, inject * the fault back to the guest. */ - if (esr_fsc_is_excl_atomic_fault(kvm_vcpu_get_esr(s2fd->vcpu))) { + if (esr_fsc_is_excl_atomic_fault(s2fd->esr)) { kvm_inject_dabt_excl_atomic(s2fd->vcpu, kvm_vcpu_get_hfar(s2fd->vcpu)); return 1; } @@ -1986,13 +2007,13 @@ static int kvm_s2_fault_compute_prot(const struct k= vm_s2_fault_desc *s2fd, =20 if (s2vi->map_writable && (s2vi->device || !memslot_is_logging(s2fd->memslot) || - kvm_is_write_fault(s2fd->vcpu))) + kvm_s2_fault_is_write(s2fd))) *prot |=3D KVM_PGTABLE_PROT_W; =20 if (s2fd->nested) *prot =3D adjust_nested_fault_perms(s2fd->nested, *prot); =20 - if (kvm_vcpu_trap_is_exec_fault(s2fd->vcpu)) + if (kvm_s2_fault_is_exec(s2fd)) *prot |=3D KVM_PGTABLE_PROT_X; =20 if (s2vi->map_non_cacheable) @@ -2034,8 +2055,7 @@ static int kvm_s2_fault_map(const struct kvm_s2_fault= _desc *s2fd, if (mmu_invalidate_retry(kvm, s2vi->mmu_seq)) goto out_unlock; =20 - perm_fault_granule =3D (kvm_s2_fault_is_perm(s2fd) ? - kvm_vcpu_trap_get_perm_fault_granule(s2fd->vcpu) : 0); + perm_fault_granule =3D kvm_s2_perm_fault_granule(s2fd); mapping_size =3D s2vi->vma_pagesize; pfn =3D s2vi->pfn; gfn =3D s2vi->gfn; @@ -2103,7 +2123,7 @@ static int kvm_s2_fault_map(const struct kvm_s2_fault= _desc *s2fd, =20 static int user_mem_abort(const struct kvm_s2_fault_desc *s2fd) { - bool perm_fault =3D kvm_vcpu_trap_is_permission_fault(s2fd->vcpu); + const bool perm_fault =3D kvm_s2_fault_is_perm(s2fd); struct kvm_s2_fault_vma_info s2vi =3D {}; enum kvm_pgtable_prot prot; void *memcache; @@ -2250,7 +2270,7 @@ int kvm_handle_guest_sea(struct kvm_vcpu *vcpu) int kvm_handle_guest_abort(struct kvm_vcpu *vcpu) { struct kvm_s2_trans nested_trans, *nested =3D NULL; - unsigned long esr; + const unsigned long esr =3D kvm_vcpu_get_esr(vcpu); phys_addr_t fault_ipa; /* The address we faulted on */ phys_addr_t ipa; /* Always the IPA in the L1 guest phys space */ struct kvm_memory_slot *memslot; @@ -2259,11 +2279,9 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu) gfn_t gfn; int ret, idx; =20 - if (kvm_vcpu_abt_issea(vcpu)) + if (esr_abt_is_sea(esr)) return kvm_handle_guest_sea(vcpu); =20 - esr =3D kvm_vcpu_get_esr(vcpu); - /* * The fault IPA should be reliable at this point as we're not dealing * with an SEA. @@ -2272,7 +2290,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu) if (KVM_BUG_ON(ipa =3D=3D INVALID_GPA, vcpu->kvm)) return -EFAULT; =20 - is_iabt =3D kvm_vcpu_trap_is_iabt(vcpu); + is_iabt =3D esr_trap_is_iabt(esr); =20 if (esr_fsc_is_translation_fault(esr)) { /* Beyond sanitised PARange (which is the IPA limit) */ @@ -2289,7 +2307,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu) } } =20 - trace_kvm_guest_fault(*vcpu_pc(vcpu), kvm_vcpu_get_esr(vcpu), + trace_kvm_guest_fault(*vcpu_pc(vcpu), esr, kvm_vcpu_get_hfar(vcpu), fault_ipa); =20 /* Check the stage-2 fault is trans. fault or write fault */ @@ -2298,9 +2316,9 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu) !esr_fsc_is_access_flag_fault(esr) && !esr_fsc_is_excl_atomic_fault(esr)) { kvm_err("Unsupported FSC: EC=3D%#x xFSC=3D%#lx ESR_EL2=3D%#lx\n", - kvm_vcpu_trap_get_class(vcpu), - (unsigned long)kvm_vcpu_trap_get_fault(vcpu), - (unsigned long)kvm_vcpu_get_esr(vcpu)); + esr_trap_get_class(esr), + (unsigned long)esr_fsc_get_fault(esr), + (unsigned long)esr); return -EFAULT; } =20 @@ -2349,7 +2367,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu) gfn =3D ipa >> PAGE_SHIFT; memslot =3D gfn_to_memslot(vcpu->kvm, gfn); hva =3D gfn_to_hva_memslot_prot(memslot, gfn, &writable); - write_fault =3D kvm_is_write_fault(vcpu); + write_fault =3D esr_abt_is_write_fault(esr); if (kvm_is_error_hva(hva) || (write_fault && !writable)) { /* * The guest has put either its instructions or its page-tables @@ -2362,7 +2380,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu) goto out; } =20 - if (kvm_vcpu_abt_iss1tw(vcpu)) { + if (esr_abt_is_s1ptw(esr)) { ret =3D kvm_inject_sea_dabt(vcpu, kvm_vcpu_get_hfar(vcpu)); goto out_unlock; } @@ -2377,7 +2395,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu) * So let's assume that the guest is just being * cautious, and skip the instruction. */ - if (kvm_is_error_hva(hva) && kvm_vcpu_dabt_is_cm(vcpu)) { + if (kvm_is_error_hva(hva) && esr_dabt_is_cm(esr)) { kvm_incr_pc(vcpu); ret =3D 1; goto out_unlock; @@ -2409,14 +2427,14 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu) .nested =3D nested, .memslot =3D memslot, .hva =3D hva, + .esr =3D esr, }; =20 if (kvm_vm_is_protected(vcpu->kvm)) { ret =3D pkvm_mem_abort(&s2fd); } else { - VM_WARN_ON_ONCE(kvm_vcpu_trap_is_permission_fault(vcpu) && - !write_fault && - !kvm_vcpu_trap_is_exec_fault(vcpu)); + VM_WARN_ON_ONCE(kvm_s2_fault_is_perm(&s2fd) && !write_fault && + !kvm_s2_fault_is_exec(&s2fd)); =20 if (kvm_slot_has_gmem(memslot)) ret =3D gmem_abort(&s2fd); --=20 2.55.0 From nobody Mon Sep 28 06:34:45 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D27DF33B96B; Tue, 25 Aug 2026 16:01:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787673687; cv=none; b=QBcbTN2ILamMVacNZm2f9HjNYRzirwNrkt6TGVDYiUjV8e1Cyo9MY6WRw1deU5nrh/JUG9GLbrZi0C2AIz8CbTNGf9g4Fqt8Zjf8CBW2B/pCcb6d1nKDLxnXghX0obdGMFwyTQtm6pNKCHFH1AquezdstTsppAWAx5dvtukbe6U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787673687; c=relaxed/simple; bh=J6uxqTSKPaKAgm7bOeHIZU2H7nzo5abmcoeBjqEoyv8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=lmM9wfFyn55nrSNEufkGb//6Qr1zIqBtKaWjcXQI37jmSDEBwIjVvS38NoA2GxrblSBdlBNgfVuoS5nsQyPsAv/mRIj7ayJoG0vkrkhKLy0GrWw3CyIGK3GCe95vy2X1FHuFlfSB53XLkY16lL1+fazqPJDlgOoC9a6nDnWAZVE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=QKpA7JMw; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="QKpA7JMw" Received: by smtp.kernel.org (Postfix) with ESMTPSA id EC4A11F000E9; Tue, 25 Aug 2026 16:01:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1787673685; bh=SdxVL4YItxylvL76H0ucWMH8iwm4yxQfCs5MMvXiowY=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=QKpA7JMwXATmgC6S/XdkcFxJAMz3wLFJzxSWQSkaI7KV7z7NC9GjnfW091Hqa8Iip ef3k/3kfXysldY+lEzUnLHlj81QC/9G8DZWzcCCvwLczRJOAoOefKIc/kqSXgw2HDN OwxFkRS+V+uDZlXANpXZGuTdC9GyMe/DcxGpt+o0TPpHrqxc7K+VxNe28ztn9Btkm7 ePSuzgEkhm7LAW0k25eo5nu9FZGFdgYIpn/0CAn0I8KtS61Hin4lWmFVgMtG97I2XX ZIz4tyYXiYPBwCCtDWSBL3gPhDCKtFejrxSLNx86tIpdh3m/GX/MKFVkF9MAbSAICC d2qjHlVYZNuSw== From: "Lorenzo Stoakes (ARM)" Date: Tue, 25 Aug 2026 17:00:36 +0100 Subject: [PATCH 2/8] KVM: arm64: Propagate and use mmu in s2fd when handling guest aborts Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260825-kvm-arm-prefault-v1-2-befe8947702e@kernel.org> References: <20260825-kvm-arm-prefault-v1-0-befe8947702e@kernel.org> In-Reply-To: <20260825-kvm-arm-prefault-v1-0-befe8947702e@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , Fuad Tabba , Joey Gouly , Steffen Eiden , Suzuki K Poulose , Zenghui Yu , Paolo Bonzini , Jonathan Corbet Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Jack Thomson , Jack Thomson , Alexandru Elisei , Vincent Donnefort , "Aneesh Kumar K.V" , Sean Christopherson , Claudio Imbrenda , Leo Soares Passos , "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=3989; i=ljs@kernel.org; h=from:subject:message-id; bh=J6uxqTSKPaKAgm7bOeHIZU2H7nzo5abmcoeBjqEoyv8=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLJ69zgVn1obl+0s6LK75LHanZ5V2qp/P3AGmf51316gu /PgkgWyHaUsDGJcDLJiiizPv4jvDxIJm9d5wd8NZg4rE8gQBi5OAZjITg1Ghi+LshwY9ObfKT5x 7rrYszrbZdN6nXkinjKvT2cyPrzNspPhr2Di9PU1Yv0HT3HdOa1xxsvsHI+cz4zssha/M1PKmez 9WAE= X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 kvm_handle_guest_abort() establishes a kvm_s2_fault_desc data structure, s2fd, to store and propagate state to either pkvm_mem_abort(), gmem_abort() or user_mem_abort() handlers. Each of these, however, examines the state of the stage 2 MMU via vcpu->arch.hw_mmu. Introduce an s2fd->mmu field to abstract this and propagate it to callers. Similar to adding the esr field, this allows injection of synthetic faults with the ultimate intention of implementing stage 2 page table pre-faulting. No functional change intended. Signed-off-by: Lorenzo Stoakes (ARM) Tested-by: Fuad Tabba < fuad.tabba@linux.dev> --- arch/arm64/kvm/mmu.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 30d605e87b01..80cb520e25b9 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1604,6 +1604,7 @@ struct kvm_s2_fault_desc { struct kvm_memory_slot *memslot; unsigned long hva; unsigned long esr; + struct kvm_s2_mmu *mmu; }; =20 static bool kvm_s2_fault_is_perm(const struct kvm_s2_fault_desc *s2fd) @@ -1637,7 +1638,7 @@ static int gmem_abort(const struct kvm_s2_fault_desc = *s2fd) const bool perm_fault =3D kvm_s2_fault_is_perm(s2fd); enum kvm_pgtable_walk_flags flags =3D KVM_PGTABLE_WALK_SHARED; enum kvm_pgtable_prot prot =3D KVM_PGTABLE_PROT_R; - struct kvm_pgtable *pgt =3D s2fd->vcpu->arch.hw_mmu->pgt; + struct kvm_pgtable *pgt =3D s2fd->mmu->pgt; unsigned long mmu_seq; struct page *page; struct kvm *kvm =3D s2fd->vcpu->kvm; @@ -1735,7 +1736,7 @@ static int pkvm_mem_abort(const struct kvm_s2_fault_d= esc *s2fd) { unsigned int flags =3D FOLL_HWPOISON | FOLL_LONGTERM | FOLL_WRITE; struct kvm_vcpu *vcpu =3D s2fd->vcpu; - struct kvm_pgtable *pgt =3D vcpu->arch.hw_mmu->pgt; + struct kvm_pgtable *pgt =3D s2fd->mmu->pgt; struct mm_struct *mm =3D current->mm; struct kvm *kvm =3D vcpu->kvm; void *hyp_memcache; @@ -2050,7 +2051,7 @@ static int kvm_s2_fault_map(const struct kvm_s2_fault= _desc *s2fd, int ret; =20 kvm_fault_lock(kvm); - pgt =3D s2fd->vcpu->arch.hw_mmu->pgt; + pgt =3D s2fd->mmu->pgt; ret =3D -EAGAIN; if (mmu_invalidate_retry(kvm, s2vi->mmu_seq)) goto out_unlock; @@ -2271,6 +2272,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu) { struct kvm_s2_trans nested_trans, *nested =3D NULL; const unsigned long esr =3D kvm_vcpu_get_esr(vcpu); + struct kvm_s2_mmu *mmu =3D vcpu->arch.hw_mmu; phys_addr_t fault_ipa; /* The address we faulted on */ phys_addr_t ipa; /* Always the IPA in the L1 guest phys space */ struct kvm_memory_slot *memslot; @@ -2300,7 +2302,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu) } =20 /* Falls between the IPA range and the PARange? */ - if (fault_ipa >=3D BIT_ULL(VTCR_EL2_IPA(vcpu->arch.hw_mmu->vtcr))) { + if (fault_ipa >=3D BIT_ULL(VTCR_EL2_IPA(mmu->vtcr))) { fault_ipa |=3D FAR_TO_FIPA_OFFSET(kvm_vcpu_get_hfar(vcpu)); =20 return kvm_inject_sea(vcpu, is_iabt, fault_ipa); @@ -2337,8 +2339,8 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu) * nothing to walk and we treat it as a 1:1 before going through the * canonical translation. */ - if (kvm_is_nested_s2_mmu(vcpu->kvm,vcpu->arch.hw_mmu) && - vcpu->arch.hw_mmu->nested_stage2_enabled) { + if (kvm_is_nested_s2_mmu(vcpu->kvm, mmu) && + mmu->nested_stage2_enabled) { u32 esr; =20 ret =3D kvm_walk_nested_s2(vcpu, fault_ipa, &nested_trans); @@ -2413,7 +2415,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu) } =20 /* Userspace should not be able to register out-of-bounds IPAs */ - VM_BUG_ON(ipa >=3D kvm_phys_size(vcpu->arch.hw_mmu)); + VM_BUG_ON(ipa >=3D kvm_phys_size(mmu)); =20 if (esr_fsc_is_access_flag_fault(esr)) { handle_access_fault(vcpu, fault_ipa); @@ -2428,6 +2430,7 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu) .memslot =3D memslot, .hva =3D hva, .esr =3D esr, + .mmu =3D mmu, }; =20 if (kvm_vm_is_protected(vcpu->kvm)) { --=20 2.55.0 From nobody Mon Sep 28 06:34:45 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7E4872D46CE; Tue, 25 Aug 2026 16:01:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787673692; cv=none; b=W8AdVFXl7JMx4W/rlYYovVdF8BhpfMRPXA8nqOoBe6lAZJEEFFQeJSDrKXSZqJxzEIWfKDWEGJnyZ81jU1xEZwZmF9Kv+mMEA1PuGSwbwi67+m0VAXnjK2ma/0qcNbcmSTbiSY2X9NFgIRUuNqmfrUG00a2KtsxfJjmyqUphdos= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787673692; c=relaxed/simple; bh=OK8qMkoBFUsKSDW1WXYaJvEYhrPVl7NXbf8oJ3V+T3w=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=a/EoAqdXL0u3nmCsuhbMMNq+P5mSAQaL2vZ03Z+NXiDLSoMMrpBTXT7uKQKFmBOZtR2LHK/2Jp1sfgTXQP2LgSKEbrMQq6eCyxBb7c1noYJO5rcVG2J6Mx3lewREKdKiaUIG7Uiz6TLv4ajvLwGPf9QV5PumnThfzO7mx9/b+nc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=dHVqBfDS; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="dHVqBfDS" Received: by smtp.kernel.org (Postfix) with ESMTPSA id E1A1A1F00A3A; Tue, 25 Aug 2026 16:01:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1787673691; bh=2n/GT1zNcErOZDVCRG/HrutbSxrG2jWsh4QWz8aUK78=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=dHVqBfDStyFZhJftkYg6JAcTecPfldfshUIibfnQn0qeJVNaujjSLd6QcRAmlIRXf tTdHX3/a0nT8Hf2scWZ/o2rQ6nx/67V89ay+ONEXUq6aQ8Rm4AxNd3m0NwOk3UYgJm bVuEat7hL3MtjEf9FSh5iHr7SvtyvoQwT1X3z7olEPI7QlmA+rDexmL9ltF33TOOUD NNfdSnAfxZ/FwY0G3aq6lRTN4hdvcVD/dmrDytX67c9FAGK/UkHwS2bgvzhSH6/sHB Vl3QawoYBQfhM7anQ84gCbKoPubESoBoOX3vJukavt8xVf04nGHECoA8X8NeuIwjvk Sl/WWQKBAa1UA== From: "Lorenzo Stoakes (ARM)" Date: Tue, 25 Aug 2026 17:00:37 +0100 Subject: [PATCH 3/8] KVM: arm64: Propagate and use kvm_s2_fault_result on S2 fault Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260825-kvm-arm-prefault-v1-3-befe8947702e@kernel.org> References: <20260825-kvm-arm-prefault-v1-0-befe8947702e@kernel.org> In-Reply-To: <20260825-kvm-arm-prefault-v1-0-befe8947702e@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , Fuad Tabba , Joey Gouly , Steffen Eiden , Suzuki K Poulose , Zenghui Yu , Paolo Bonzini , Jonathan Corbet Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Jack Thomson , Jack Thomson , Alexandru Elisei , Vincent Donnefort , "Aneesh Kumar K.V" , Sean Christopherson , Claudio Imbrenda , Leo Soares Passos , "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=4633; i=ljs@kernel.org; h=from:subject:message-id; bh=OK8qMkoBFUsKSDW1WXYaJvEYhrPVl7NXbf8oJ3V+T3w=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLJ69zgdV7sdlbN67b4bp9Qnd+2ZwBXv6ZRSMyWBYafTD 6b1M/4+7ShlYRDjYpAVU2R5/kV8f5BI2LzOC/5uMHNYmUCGMHBxCsBE3kxnZHj/SLt2l/3/4Oen tq3p/dVhKHUs9ZhC9hflxkr29ZPaBBUZ/lmHSc57WP9DL7rR4+8NuSVptx/YJn6Purp10u1VqzY 9fs0AAA== X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 When stage 2 page tables fault the net result may either be that a page is mapped, an error occurred or the fault should be retried (-EAGAIN). When a fault succeeds it may be upgraded to a PMD size via transparent_hugepage_adjust(). In order to support KVM pre-faulting the outcome of the fault and the mapping size must be recorded. Track this in the new kvm_s2_fault_result struct, which is threaded through gmem_abort(), user_mem_abort() and kvm_s2_fault_map(). PKVM and SEA aren't relevant to synthetic pre-faulting so neither kvm_inject_sea() nor pkvm_mem_abort() are altered. Actual hardware faulting doesn't require this information, so kvm_handle_guest_abort() simply passes NULL kvm_s2_fault_result to gmem_abort() and user_mem_abort(). Faults are necessarily ephemeral and pre-faulting can't guarantee what may happen in parallel, so do not store the GFN or PFN in kvm_s2_fault_result. Pre-faulting only needs to know what was mapped in at the point of the fault. This struct could be replaced with a pointer to an unsigned long, however it's clearer to separate out the mapped flag and having a struct allows us to easily add additional fields in future as needed. No functional change intended. Suggested-by: Vincent Donnefort Signed-off-by: Lorenzo Stoakes (ARM) Tested-by: Fuad Tabba < fuad.tabba@linux.dev> --- arch/arm64/kvm/mmu.c | 33 +++++++++++++++++++++++++++------ 1 file changed, 27 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 80cb520e25b9..da15da4e40e6 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -1607,6 +1607,11 @@ struct kvm_s2_fault_desc { struct kvm_s2_mmu *mmu; }; =20 +struct kvm_s2_fault_result { + unsigned long mapping_size; + bool mapped; +}; + static bool kvm_s2_fault_is_perm(const struct kvm_s2_fault_desc *s2fd) { return esr_fsc_is_permission_fault(s2fd->esr); @@ -1632,7 +1637,17 @@ static u64 kvm_s2_perm_fault_granule(const struct kv= m_s2_fault_desc *s2fd) return BIT(ARM64_HW_PGTABLE_LEVEL_SHIFT(level)); } =20 -static int gmem_abort(const struct kvm_s2_fault_desc *s2fd) +static void populate_fault_result(struct kvm_s2_fault_result *result, + unsigned long mapping_size) +{ + /* A THP upgrade may have altered mapping size. */ + result->mapping_size =3D mapping_size; + /* -EAGAIN is swallowed so be explicit when we actually map. */ + result->mapped =3D true; +} + +static int gmem_abort(const struct kvm_s2_fault_desc *s2fd, + struct kvm_s2_fault_result *result) { bool write_fault, exec_fault; const bool perm_fault =3D kvm_s2_fault_is_perm(s2fd); @@ -1714,6 +1729,8 @@ static int gmem_abort(const struct kvm_s2_fault_desc = *s2fd) if ((prot & KVM_PGTABLE_PROT_W) && !ret) mark_page_dirty_in_slot(kvm, s2fd->memslot, gfn); =20 + if (result && !ret) + populate_fault_result(result, PAGE_SIZE); return ret !=3D -EAGAIN ? ret : 0; } =20 @@ -2038,7 +2055,8 @@ static int kvm_s2_fault_compute_prot(const struct kvm= _s2_fault_desc *s2fd, static int kvm_s2_fault_map(const struct kvm_s2_fault_desc *s2fd, const struct kvm_s2_fault_vma_info *s2vi, enum kvm_pgtable_prot prot, - void *memcache) + void *memcache, + struct kvm_s2_fault_result *result) { enum kvm_pgtable_walk_flags flags =3D KVM_PGTABLE_WALK_SHARED; bool writable =3D prot & KVM_PGTABLE_PROT_W; @@ -2117,12 +2135,15 @@ static int kvm_s2_fault_map(const struct kvm_s2_fau= lt_desc *s2fd, mark_page_dirty_in_slot(kvm, s2fd->memslot, gpa_to_gfn(ipa)); } =20 + if (result && !ret) + populate_fault_result(result, mapping_size); if (ret !=3D -EAGAIN) return ret; return 0; } =20 -static int user_mem_abort(const struct kvm_s2_fault_desc *s2fd) +static int user_mem_abort(const struct kvm_s2_fault_desc *s2fd, + struct kvm_s2_fault_result *result) { const bool perm_fault =3D kvm_s2_fault_is_perm(s2fd); struct kvm_s2_fault_vma_info s2vi =3D {}; @@ -2161,7 +2182,7 @@ static int user_mem_abort(const struct kvm_s2_fault_d= esc *s2fd) return ret; } =20 - return kvm_s2_fault_map(s2fd, &s2vi, prot, memcache); + return kvm_s2_fault_map(s2fd, &s2vi, prot, memcache, result); } =20 /* Resolve the access fault by making the page young again. */ @@ -2440,9 +2461,9 @@ int kvm_handle_guest_abort(struct kvm_vcpu *vcpu) !kvm_s2_fault_is_exec(&s2fd)); =20 if (kvm_slot_has_gmem(memslot)) - ret =3D gmem_abort(&s2fd); + ret =3D gmem_abort(&s2fd, NULL); else - ret =3D user_mem_abort(&s2fd); + ret =3D user_mem_abort(&s2fd, NULL); } =20 if (ret =3D=3D 0) --=20 2.55.0 From nobody Mon Sep 28 06:34:45 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B3EB3334C3C; Tue, 25 Aug 2026 16:01:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787673699; cv=none; b=IlyIRzgbj+qRWdb8IuEvnIvxZii9aciPGXA3jy0Su2Telr84nOrHU6aQgL0+dCkeJ+1tZOq7IU3ygdAthyiAkRoXBc+uq9/6YEYe1PYwGe/fgQ9DXhVbaPLUq/+V2u1u35ITD5MwhFhUKGiQouzKGSgDbfWc0Biy0wplhHcoVSY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787673699; c=relaxed/simple; bh=o4Auwo51lsUqNX84mp915x6tL1LeRNAk/x5+YdBJpXA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=jM4h+jG50VcAKwwkzNTGYSpuEDg91ASIQVcSH3xVKpomlQil5kQdoUwDzU7ynLKGifAZidNGdUhEtYPydfqnroEqVfybTGRYu6CwtxnnUH6JgpUbmOJLNc6dWt6mHNKvmqW7OBJjW2mjHuAtDWksIURye6OWZS8qigcWVvvB1m4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=aXRqIKi1; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="aXRqIKi1" Received: by smtp.kernel.org (Postfix) with ESMTPSA id DEC5A1F000E9; Tue, 25 Aug 2026 16:01:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1787673697; bh=sdQhYGYcudMsM4aRFVyqBxg6xpTZPha1ewHoePoR7aw=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=aXRqIKi1O4MVUlxU7MWBEHuIxZ+FlMT5aBPk3pLpoak20xiBsD99u9fHw7PRrB7tK zCN3YvE70fu/XZTd7Gwh2YPHXCuUTwzIy4pHdpLys5883PLEQeOBeHsxewsBWnrFV5 85i8f7W30+JJKlhacf9HdzEL5KHcM9Ayq6FIus/vN4/eZbkM6BTNoQD6kqWHv1jnNJ IzjzO7BhC6HgeYOWtsUvt7B1edroTSxBCy7L030Rw0HogmdOYEWpK4icCbECT+wTB6 LFUruvf/7Id4EGEZ0Lehnvg4gaYlk7oXV84ZPb0k+FHEeQlDnRrSN7imQcZWsTukDI JOmGs6fQ7UB7g== From: "Lorenzo Stoakes (ARM)" Date: Tue, 25 Aug 2026 17:00:38 +0100 Subject: [PATCH 4/8] KVM: arm64: Pass walk flags to kvm_pgtable_get_leaf() Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260825-kvm-arm-prefault-v1-4-befe8947702e@kernel.org> References: <20260825-kvm-arm-prefault-v1-0-befe8947702e@kernel.org> In-Reply-To: <20260825-kvm-arm-prefault-v1-0-befe8947702e@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , Fuad Tabba , Joey Gouly , Steffen Eiden , Suzuki K Poulose , Zenghui Yu , Paolo Bonzini , Jonathan Corbet Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Jack Thomson , Jack Thomson , Alexandru Elisei , Vincent Donnefort , "Aneesh Kumar K.V" , Sean Christopherson , Claudio Imbrenda , Leo Soares Passos , "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=5556; i=ljs@kernel.org; h=from:subject:message-id; bh=SNPM+GS9oSB1Lz0eqMxBtW49x04aXuFubXrjawMYwg4=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLJ69zitKmbnmme2w0QuxO2Agn+xdfWFI/Pd9Qz+86VKT 0zklPTvKGVhEONikBVTZHn+RXx/kEjYvM4L/m4wc1iZQIYwcHEKwET+r2X4zfLycNzmxrvvnWNk 5kXpHHwR0RvIK7jvtOp/p9zKRgO+pwx/Rd02rc2/ZaNWtffOwduyM+qWvX7KuFzydwv36jP2KZv k+AA= X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 From: Jack Thomson Allow callers of kvm_pgtable_get_leaf() to specify the page-table walk flags, in preparation for performing walks under the MMU read lock. Reading a stage-2 leaf while only holding the read lock requires KVM_PGTABLE_WALK_SHARED: parallel faults (which also only hold the read lock) can unlink table pages and free them via RCU, so the walker must be inside an RCU read-side critical section, which the shared walk flag provides via kvm_pgtable_walk_begin(). All existing callers either hold the write lock, walk with interrupts disabled, or run at hyp where shared walks are rejected; they keep the current behaviour by passing no flags. No functional change intended. Signed-off-by: Jack Thomson Signed-off-by: Lorenzo Stoakes (ARM) Tested-by: Fuad Tabba < fuad.tabba@linux.dev> --- arch/arm64/include/asm/kvm_pgtable.h | 5 ++++- arch/arm64/kvm/hyp/nvhe/mem_protect.c | 10 +++++----- arch/arm64/kvm/hyp/pgtable.c | 5 +++-- arch/arm64/kvm/mmu.c | 2 +- arch/arm64/kvm/nested.c | 2 +- 5 files changed, 14 insertions(+), 10 deletions(-) diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/include/asm/= kvm_pgtable.h index 41a8687938eb..d0167f7dfbee 100644 --- a/arch/arm64/include/asm/kvm_pgtable.h +++ b/arch/arm64/include/asm/kvm_pgtable.h @@ -859,6 +859,8 @@ int kvm_pgtable_walk(struct kvm_pgtable *pgt, u64 addr,= u64 size, * @addr: Input address for the start of the walk. * @ptep: Pointer to storage for the retrieved PTE. * @level: Pointer to storage for the level of the retrieved PTE. + * @flags: Flags to control the page-table walk + * (see struct kvm_pgtable_visit_ctx). * * The offset of @addr within a page is ignored. * @@ -869,7 +871,8 @@ int kvm_pgtable_walk(struct kvm_pgtable *pgt, u64 addr,= u64 size, * Return: 0 on success, negative error code on failure. */ int kvm_pgtable_get_leaf(struct kvm_pgtable *pgt, u64 addr, - kvm_pte_t *ptep, s8 *level); + kvm_pte_t *ptep, s8 *level, + enum kvm_pgtable_walk_flags flags); =20 /** * kvm_pgtable_stage2_pte_prot() - Retrieve the protection attributes of a diff --git a/arch/arm64/kvm/hyp/nvhe/mem_protect.c b/arch/arm64/kvm/hyp/nvh= e/mem_protect.c index 39aa8911f62c..23a57312b981 100644 --- a/arch/arm64/kvm/hyp/nvhe/mem_protect.c +++ b/arch/arm64/kvm/hyp/nvhe/mem_protect.c @@ -532,7 +532,7 @@ static int host_stage2_adjust_range(u64 addr, struct kv= m_mem_range *range) int ret; =20 hyp_assert_lock_held(&host_mmu.lock); - ret =3D kvm_pgtable_get_leaf(&host_mmu.pgt, addr, &pte, &level); + ret =3D kvm_pgtable_get_leaf(&host_mmu.pgt, addr, &pte, &level, 0); if (ret) return ret; =20 @@ -910,7 +910,7 @@ static int get_valid_guest_pte(struct pkvm_hyp_vm *vm, = u64 ipa, kvm_pte_t *ptep, s8 level; int ret; =20 - ret =3D kvm_pgtable_get_leaf(&vm->pgt, ipa, &pte, &level); + ret =3D kvm_pgtable_get_leaf(&vm->pgt, ipa, &pte, &level, 0); if (ret) return ret; if (guest_pte_is_poisoned(pte)) @@ -959,7 +959,7 @@ int __pkvm_vcpu_in_poison_fault(struct pkvm_hyp_vcpu *h= yp_vcpu) ipa |=3D FAR_TO_FIPA_OFFSET(kvm_vcpu_get_hfar(&hyp_vcpu->vcpu)); =20 guest_lock_component(vm); - ret =3D kvm_pgtable_get_leaf(&vm->pgt, ipa, &pte, &level); + ret =3D kvm_pgtable_get_leaf(&vm->pgt, ipa, &pte, &level, 0); if (ret) goto unlock; =20 @@ -1315,7 +1315,7 @@ static int host_stage2_get_guest_info(phys_addr_t phy= s, struct pkvm_hyp_vm **vm, return -EPERM; } =20 - ret =3D kvm_pgtable_get_leaf(&host_mmu.pgt, phys, &pte, &level); + ret =3D kvm_pgtable_get_leaf(&host_mmu.pgt, phys, &pte, &level, 0); if (ret) return ret; =20 @@ -1544,7 +1544,7 @@ static int __check_host_shared_guest(struct pkvm_hyp_= vm *vm, u64 *__phys, u64 ip s8 level; int ret; =20 - ret =3D kvm_pgtable_get_leaf(&vm->pgt, ipa, &pte, &level); + ret =3D kvm_pgtable_get_leaf(&vm->pgt, ipa, &pte, &level, 0); if (ret) return ret; if (!kvm_pte_valid(pte)) diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgtable.c index b74dd5ce1efd..347eec3957d6 100644 --- a/arch/arm64/kvm/hyp/pgtable.c +++ b/arch/arm64/kvm/hyp/pgtable.c @@ -298,12 +298,13 @@ static int leaf_walker(const struct kvm_pgtable_visit= _ctx *ctx, } =20 int kvm_pgtable_get_leaf(struct kvm_pgtable *pgt, u64 addr, - kvm_pte_t *ptep, s8 *level) + kvm_pte_t *ptep, s8 *level, + enum kvm_pgtable_walk_flags flags) { struct leaf_walk_data data; struct kvm_pgtable_walker walker =3D { .cb =3D leaf_walker, - .flags =3D KVM_PGTABLE_WALK_LEAF, + .flags =3D flags | KVM_PGTABLE_WALK_LEAF, .arg =3D &data, }; int ret; diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index da15da4e40e6..16299004f229 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -866,7 +866,7 @@ static int get_user_mapping_size(struct kvm *kvm, u64 a= ddr) * IPI-ing threads). */ local_irq_save(flags); - ret =3D kvm_pgtable_get_leaf(&pgt, addr, &pte, &level); + ret =3D kvm_pgtable_get_leaf(&pgt, addr, &pte, &level, 0); local_irq_restore(flags); =20 if (ret) diff --git a/arch/arm64/kvm/nested.c b/arch/arm64/kvm/nested.c index 17123f0b6dab..3488d5ad225b 100644 --- a/arch/arm64/kvm/nested.c +++ b/arch/arm64/kvm/nested.c @@ -654,7 +654,7 @@ static u8 get_guest_mapping_ttl(struct kvm_s2_mmu *mmu,= u64 addr) return 0; =20 tmp &=3D ~(sz - 1); - if (kvm_pgtable_get_leaf(mmu->pgt, tmp, &pte, NULL)) + if (kvm_pgtable_get_leaf(mmu->pgt, tmp, &pte, NULL, 0)) goto again; if (!(pte & PTE_VALID)) goto again; --=20 2.55.0 From nobody Mon Sep 28 06:34:45 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CA6FA3242BD; Tue, 25 Aug 2026 16:01:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787673705; cv=none; b=WKLe+cd2fKIAgQcPg6QQqg/anmoUPEi37Q7c1AyNk3SSiYbmxw2zldrA78diH3AX9pD9eqhUguPGesDUzhKgTKvfBUu4H5CMBFI2yjc7nQ5I8v2KY+g+kwWQKFhj7sIXviYxY8+rvO47zXeAiKVYJyTqTpZcGFoTq/t6yXlmQnk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787673705; c=relaxed/simple; bh=O5R3W+EAGtE/pAeolIY67pfITb9DRfGm0N1r8RGkQu0=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=oscRE3dB1lGTlqIgLSSFJUiZkrCqn/cj5yuU+ML6TQRpGn8GGvy5XzV/hciw14EpT8e5aAlpcFGQMHiHIVkTecbZUue3w+u8yHXA8uJQEf72sJi9KTuhQpF524TIImfXBtwbSA0pvTaBxWM3O+bYwiT1BI8cwc+JP/kFan3jJ4g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=lQUKFJi1; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="lQUKFJi1" Received: by smtp.kernel.org (Postfix) with ESMTPSA id D2C9F1F00A3A; Tue, 25 Aug 2026 16:01:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1787673703; bh=vItrVbNElvJ8tDNO/bq+S6qyucbi87j1O2ToAdzbEpc=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=lQUKFJi1sa7TctT7/aIIKj/GyVJjaM1CPJd1+VoUyVSWYSJ5U8TDzy2sdKbr8eKad DVsVRPitH5m92ELsqPgnmaQpWiNEa4FjRXHF4MZVNwHra5gZayTGzqGZEddv1FzLfG aDzQDtb5rc2CJgsab27CjjUWBpM/YxSYDMSbcTU7nf9dXUqXgu1xW1Tt+y2qE7vQES NbecBYW3o+D6p7K8Te1+DV2Gq8KCO3uOu63vnZMIP2PJSHYMzoXookJdW/CG3d+Jw6 wDLebKA7VH8XsQszHK9oZI3ozKWA/A9vyeiUhcK1AMlidbhiroPoMAfKJ+xIqP3aNm vPZXBmWA8mIUA== From: "Lorenzo Stoakes (ARM)" Date: Tue, 25 Aug 2026 17:00:39 +0100 Subject: [PATCH 5/8] KVM: arm64: Implement KVM_PRE_FAULT_MEMORY Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260825-kvm-arm-prefault-v1-5-befe8947702e@kernel.org> References: <20260825-kvm-arm-prefault-v1-0-befe8947702e@kernel.org> In-Reply-To: <20260825-kvm-arm-prefault-v1-0-befe8947702e@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , Fuad Tabba , Joey Gouly , Steffen Eiden , Suzuki K Poulose , Zenghui Yu , Paolo Bonzini , Jonathan Corbet Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Jack Thomson , Jack Thomson , Alexandru Elisei , Vincent Donnefort , "Aneesh Kumar K.V" , Sean Christopherson , Claudio Imbrenda , Leo Soares Passos , "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=16453; i=ljs@kernel.org; h=from:subject:message-id; bh=O5R3W+EAGtE/pAeolIY67pfITb9DRfGm0N1r8RGkQu0=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLJ69ziVvHQzmJyyNivclP/BtWoflwqG29JSZy9cmSaem 21h08LbUcrCIMbFICumyPL8i/j+IJGweZ0X/N1g5rAygQxh4OIUgIncbGL4Kz5V7nrlJ3NubYei JI+521LOsK+wefV7jtPXb9FRvX+sXRkZek8f+qHXYGfy7ODSTxWODR0t3nd9+fscm46nXYpJEqr iAgA= X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 Implement KVM stage 2 page table pre-faulting for the arm64 architecture. Do the trivial plumbing by selecting CONFIG_KVM_GENERIC_PRE_FAULT_MEMORY in Kconfig, returning 1 in kvm_vm_ioctl_check_extension() for KVM_CAP_PRE_FAULT_MEMORY but false in kvm_pkvm_ext_allowed() to disallow its use for pKVM. Add kvm_arch_vcpu_pre_fault_memory() to actually implement the feature by pre-faulting stage 2 page tables for a specific GPA. Previous commits added the required foundations - gmem_abort() and user_mem_abort() determine esr and mmu from s2fd->esr and s2fd->mmu respectively, the granule size is returned by them in kvm_s2_fault_result and kvm_pgtable_get_leaf() accepts a walk flags parameter. Additionally, kvm_pgtable_get_leaf() now accepts a walk flag, meaning its page table walk can be performed with KVM_PGTABLE_WALK_SHARED set and thus can be performed under a read mmu_lock. With these changes in place, implement pre-faulting by first trying a page table walk under the MMU read lock then, if it fails, injecting a synthetic data abort at the page table level at which the page table walk failed. This is necessarily racey as reclaim might happen at any time. Successfully pre-faulting can therefore only guarantee that each GPA was observed to be mapped at least once. Protected KVM (pKVM) is not supported at all because pKVM creates VMs and vCPUs when first run, meaning any attempt to pre-fault prior to this cannot succeed. Since the sensible use case for pre-faulting is doing so prior to vCPU run, and supporting only online pKVM vCPUs is confusing and inconsistent, simply don't support this at all. There is a subtlety when retrieving the hva: it seems natural to use gfn_to_hva_memslot(), but this errors out for read-only memslots. Since pre-faulting should fault in both read-only and read/write hvas, this isn't the correct API to use here. gfn_to_hva_memslot_prot() allows retrieval of read-only hvas, but has unclear semantics, so introduce gfn_to_hva_memslot_read() to wrap it. A retry mechanic is implemented when user_mem_abort() or gmem_abort() fail to map memory due to a benign failure where a hardware abort would not result in an error. These occur when the abort handler was raced by either an invalidation MMU notifier or a racing abort path. Since these faults are highly likely to succeed on immediate retry, retry up to MAX_PRE_FAULT_RETRIES times, after which -EAGAIN is ultimately returned to the caller. An invalid memslot (i.e. a memslot with the KVM_MEMSLOT_INVALID flag set) also results in the operation returning -EAGAIN without a retry mechanic. This is because an invalid memslot means the pre-fault operation raced with memslot reclaim, and since the SRCU lock is held, progress cannot be made. Therefore, when this happens, -EAGAIN indicates that userland should retry the ioctl, which will bounce the SRCU lock and permit forward progress. Finally, update the KVM API documentation to describe the changes, providing arm64-specific implementation details. This work is based with gratitude on Jack Thomson's original series, its previous revisions and the feedback they received. Link: https://patch.msgid.link/20260612162354.73378-1-jackabt.amazon@gmail.= com/ Signed-off-by: Lorenzo Stoakes (ARM) Tested-by: Fuad Tabba < fuad.tabba@linux.dev> --- Documentation/virt/kvm/api.rst | 17 +++- arch/arm64/include/asm/kvm_pkvm.h | 2 +- arch/arm64/kvm/Kconfig | 1 + arch/arm64/kvm/arm.c | 1 + arch/arm64/kvm/mmu.c | 192 ++++++++++++++++++++++++++++++++++= +--- 5 files changed, 198 insertions(+), 15 deletions(-) diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst index 4988c32df4bf..6246f4a56bb9 100644 --- a/Documentation/virt/kvm/api.rst +++ b/Documentation/virt/kvm/api.rst @@ -6485,7 +6485,7 @@ See KVM_SET_USER_MEMORY_REGION2 for additional detail= s. --------------------------- =20 :Capability: KVM_CAP_PRE_FAULT_MEMORY -:Architectures: none +:Architectures: x86, s390, arm64 :Type: vcpu ioctl :Parameters: struct kvm_pre_fault_memory (in/out) :Returns: 0 if at least one page is processed, < 0 on error @@ -6493,12 +6493,14 @@ See KVM_SET_USER_MEMORY_REGION2 for additional deta= ils. Errors: =20 =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D + EAGAIN A race occurred before progress was made, but a retry may suc= ceed. EINVAL The specified `gpa` and `size` were invalid (e.g. not page aligned, causes an overflow, or size is zero), or the VM is UCONTROL (s390). ENOENT The specified `gpa` is outside defined memslots. EINTR An unmasked signal is pending and no page was processed. EFAULT The parameter address was invalid. + EHWPOISON A poisoned host page was encountered. EOPNOTSUPP Mapping memory for a GPA is unsupported by the hypervisor, and/or for the current vCPU state/mode. EIO unexpected error conditions (also causes a WARN) @@ -6518,7 +6520,16 @@ Errors: KVM_PRE_FAULT_MEMORY populates KVM's stage-2 page tables used to map memory for the current vCPU state. KVM maps memory as if the vCPU generated a stage-2 read page fault, e.g. faults in memory as needed, but doesn't break -CoW. On x86, KVM does not mark any newly created stage-2 PTE as Accessed. +CoW. On arm64, KVM marks both existing and newly created stage-2 PTEs as +Accessed, on x86 it does not and for s390 it is not applicable. + +On arm64, a GPA is interpreted as an IPA, and never interpreted as the IPA +of a nested guest. Pre-faulting only populates canonical stage 2 page +tables. + +The feature is not supported on arm64 if the protected KVM (pKVM) feature +is enabled, as that results in vCPUs being instantiated on first run, which +renders pre-faulting useless. =20 In the case of confidential VM types where there is an initial set up of private guest memory before the guest is 'finalized'/measured, this ioctl @@ -6533,7 +6544,7 @@ When the ioctl returns, the input values are updated = to point to the remaining range. If `size` > 0 on return, the caller can just issue the ioctl again with the same `struct kvm_map_memory` argument. =20 -Shadow page tables cannot support this ioctl because they +On x86, Shadow page tables cannot support this ioctl because they are indexed by virtual address or nested guest physical address. Calling this ioctl when the guest is using shadow page tables (for example because it is running a nested guest with nested page tables) diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm= _pkvm.h index beea00e693a0..4d6e5765e9e5 100644 --- a/arch/arm64/include/asm/kvm_pkvm.h +++ b/arch/arm64/include/asm/kvm_pkvm.h @@ -44,9 +44,9 @@ static inline bool kvm_pkvm_ext_allowed(struct kvm *kvm, = long ext) case KVM_CAP_ARM_PTRAUTH_GENERIC: return true; case KVM_CAP_ARM_MTE: - return false; case KVM_CAP_ARM_EAGER_SPLIT_CHUNK_SIZE: case KVM_CAP_ARM_SUPPORTED_BLOCK_SIZES: + case KVM_CAP_PRE_FAULT_MEMORY: return false; default: return !kvm || !kvm_vm_is_protected(kvm); diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig index 449154f9a485..71233068b7cb 100644 --- a/arch/arm64/kvm/Kconfig +++ b/arch/arm64/kvm/Kconfig @@ -37,6 +37,7 @@ menuconfig KVM select SCHED_INFO select GUEST_PERF_EVENTS if PERF_EVENTS select KVM_GUEST_MEMFD + select KVM_GENERIC_PRE_FAULT_MEMORY help Support hosting virtualized guest machines. =20 diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index ccae82c1242b..58ac70f31d84 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -393,6 +393,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long = ext) case KVM_CAP_COUNTER_OFFSET: case KVM_CAP_ARM_WRITABLE_IMP_ID_REGS: case KVM_CAP_ARM_SEA_TO_USER: + case KVM_CAP_PRE_FAULT_MEMORY: r =3D 1; break; case KVM_CAP_SET_GUEST_DEBUG2: diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 16299004f229..401ae538ae75 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -5,6 +5,7 @@ */ =20 #include +#include #include #include #include @@ -1554,9 +1555,9 @@ static void *get_mmu_memcache(struct kvm_vcpu *vcpu) return &vcpu->arch.pkvm_memcache; } =20 -static int topup_mmu_memcache(struct kvm_vcpu *vcpu, void *memcache) +static int topup_mmu_memcache(struct kvm_s2_mmu *mmu, void *memcache) { - int min_pages =3D kvm_mmu_cache_min_pages(vcpu->arch.hw_mmu); + const int min_pages =3D kvm_mmu_cache_min_pages(mmu); =20 if (!is_protected_kvm_enabled()) return kvm_mmu_topup_memory_cache(memcache, min_pages); @@ -1605,6 +1606,7 @@ struct kvm_s2_fault_desc { unsigned long hva; unsigned long esr; struct kvm_s2_mmu *mmu; + bool pre_fault; }; =20 struct kvm_s2_fault_result { @@ -1664,7 +1666,7 @@ static int gmem_abort(const struct kvm_s2_fault_desc = *s2fd, =20 if (!perm_fault) { memcache =3D get_mmu_memcache(s2fd->vcpu); - ret =3D topup_mmu_memcache(s2fd->vcpu, memcache); + ret =3D topup_mmu_memcache(s2fd->mmu, memcache); if (ret) return ret; } @@ -1761,7 +1763,7 @@ static int pkvm_mem_abort(const struct kvm_s2_fault_d= esc *s2fd) int ret; =20 hyp_memcache =3D get_mmu_memcache(vcpu); - ret =3D topup_mmu_memcache(vcpu, hyp_memcache); + ret =3D topup_mmu_memcache(s2fd->mmu, hyp_memcache); if (ret) return -ENOMEM; =20 @@ -1953,6 +1955,8 @@ static int kvm_s2_fault_pin_pfn(const struct kvm_s2_f= ault_desc *s2fd, &s2vi->map_writable, &s2vi->page); if (unlikely(is_error_noslot_pfn(s2vi->pfn))) { if (s2vi->pfn =3D=3D KVM_PFN_ERR_HWPOISON) { + if (s2fd->pre_fault) + return -EHWPOISON; kvm_send_hwpoison_signal(s2fd->hva, __ffs(s2vi->vma_pagesize)); return 0; } @@ -2163,7 +2167,7 @@ static int user_mem_abort(const struct kvm_s2_fault_d= esc *s2fd, memcache =3D get_mmu_memcache(s2fd->vcpu); if (!perm_fault || memslot_is_logging(s2fd->memslot) || is_protected_kvm_enabled()) { - ret =3D topup_mmu_memcache(s2fd->vcpu, memcache); + ret =3D topup_mmu_memcache(s2fd->mmu, memcache); if (ret) return ret; } @@ -2185,18 +2189,22 @@ static int user_mem_abort(const struct kvm_s2_fault= _desc *s2fd, return kvm_s2_fault_map(s2fd, &s2vi, prot, memcache, result); } =20 -/* Resolve the access fault by making the page young again. */ -static void handle_access_fault(struct kvm_vcpu *vcpu, phys_addr_t fault_i= pa) +static void __handle_access_fault(struct kvm_pgtable *pgt, phys_addr_t fau= lt_ipa) { enum kvm_pgtable_walk_flags flags =3D KVM_PGTABLE_WALK_SHARED; - struct kvm_s2_mmu *mmu; =20 trace_kvm_access_fault(fault_ipa); + KVM_PGT_FN(kvm_pgtable_stage2_mkyoung)(pgt, fault_ipa, flags); +} + +/* Resolve the access fault by making the page young again. */ +static void handle_access_fault(struct kvm_vcpu *vcpu, phys_addr_t fault_i= pa) +{ + struct kvm_s2_mmu *mmu; =20 - read_lock(&vcpu->kvm->mmu_lock); + guard(read_lock)(&vcpu->kvm->mmu_lock); mmu =3D vcpu->arch.hw_mmu; - KVM_PGT_FN(kvm_pgtable_stage2_mkyoung)(mmu->pgt, fault_ipa, flags); - read_unlock(&vcpu->kvm->mmu_lock); + __handle_access_fault(mmu->pgt, fault_ipa); } =20 /* @@ -2834,3 +2842,165 @@ void kvm_toggle_cache(struct kvm_vcpu *vcpu, bool w= as_enabled) =20 trace_kvm_toggle_cache(*vcpu_pc(vcpu), was_enabled, now_enabled); } + +static bool kvm_pte_young_s2(kvm_pte_t pte) +{ + return pte & KVM_PTE_LEAF_ATTR_LO_S2_AF; +} + +static void kvm_pte_mkyoung_s2(struct kvm_pgtable *pgt, gpa_t gpa) +{ + struct kvm *kvm =3D kvm_s2_mmu_to_kvm(pgt->mmu); + + lockdep_assert_held(&kvm->mmu_lock); + /* Despite its name, doesn't fault here. */ + __handle_access_fault(pgt, gpa); +} + +/* + * Try to walk to the specified GPA in canonical mmu - if unmapped returns= 0, if + * mapped returns the granule size, otherwise returns an error. + */ +static long kvm_walk_s2(struct kvm_pgtable *pgt, + gpa_t gpa, s8 *level) +{ + struct kvm *kvm =3D kvm_s2_mmu_to_kvm(pgt->mmu); + kvm_pte_t pte; + long ret; + + guard(read_lock)(&kvm->mmu_lock); + + ret =3D kvm_pgtable_get_leaf(pgt, gpa, &pte, level, + KVM_PGTABLE_WALK_SHARED); + if (ret) + return ret; + /* Unpopulated, must fault. */ + if (!kvm_pte_valid(pte)) + return 0; + /* Walked the entry so mark young. */ + if (!kvm_pte_young_s2(pte)) + kvm_pte_mkyoung_s2(pgt, gpa); + return kvm_granule_size(*level); +} + +/* Synthesised data abort at specified page table level. */ +#define PRE_FAULT_ESR(level) \ + ((ESR_ELx_EC_DABT_LOW << ESR_ELx_EC_SHIFT) | \ + ESR_ELx_IL | ESR_ELx_FSC_FAULT_L(level)) + +/* Retrieve either a read-only or a read/write hva. */ +static hva_t gfn_to_hva_memslot_read(struct kvm_memory_slot *slot, gfn_t g= fn) +{ + return gfn_to_hva_memslot_prot(slot, gfn, /*writable=3D*/NULL); +} + +static long __pre_fault_s2(struct kvm_s2_mmu *mmu, struct kvm_vcpu *vcpu, + gpa_t gpa, struct kvm_memory_slot *memslot, s8 level) +{ + const bool is_gmem =3D kvm_slot_has_gmem(memslot); + const gfn_t gfn =3D gpa_to_gfn(gpa); + const hva_t hva =3D is_gmem ? 0 : gfn_to_hva_memslot_read(memslot, gfn); + const struct kvm_s2_fault_desc s2fd =3D { + .vcpu =3D vcpu, + .fault_ipa =3D gpa, + .nested =3D NULL, + .memslot =3D memslot, + .hva =3D hva, + .esr =3D PRE_FAULT_ESR(level), + .mmu =3D mmu, + .pre_fault =3D true, + }; + struct kvm_s2_fault_result result =3D {}; + long ret; + + if (kvm_is_error_hva(hva)) + return -EFAULT; + + if (is_gmem) + ret =3D gmem_abort(&s2fd, &result); + else + ret =3D user_mem_abort(&s2fd, &result); + if (IS_ERR_VALUE(ret)) + return ret; + if (!result.mapped) + return -EAGAIN; + return result.mapping_size; +} + +static long pre_fault_s2(struct kvm_s2_mmu *mmu, struct kvm_vcpu *vcpu, + gpa_t gpa, struct kvm_memory_slot *memslot) +{ + s8 level =3D KVM_PGTABLE_LAST_LEVEL; + long ret; + + /* Try a walk first. */ + ret =3D kvm_walk_s2(mmu->pgt, gpa, &level); + if (ret) + return ret; + /* OK, have to fault page in. */ + return __pre_fault_s2(mmu, vcpu, gpa, memslot, level); +} + +static unsigned long +pre_fault_bytes_consumed(gpa_t gpa, unsigned long granule_size, + unsigned long bytes_remaining) +{ + /* Granules are always a power-of-2. */ + const unsigned long granule_bytes_remaining =3D + granule_size - (gpa % granule_size); + + return min(granule_bytes_remaining, bytes_remaining); +} + +/* If you lose the race this many times, time to give up. */ +#define MAX_PRE_FAULT_RETRIES 3 + +/** + * kvm_arch_vcpu_pre_fault_memory - pre-fault stage-2 page tables for the + * specified GPA. + * @vcpu: The VCPU pointer + * @range: {gpa, size, flags} tuple + * + * The mapping performed is always best-effort - faulting in is necessarily + * racey. The ranges faulted in are canonical, nested page tables are igno= red. + * + * If the GPA is already mapped, the page table entry is marked young. + * + * @range->gpa specifies the GPA to pre-fault, @range->size specifies how = many + * bytes remain to be pre-faulted and @range->flags is reserved and must b= e 0. + * + * Returns: the number of bytes the pre-fault consumed, or an error. + */ +long kvm_arch_vcpu_pre_fault_memory(struct kvm_vcpu *vcpu, + struct kvm_pre_fault_memory *range) +{ + struct kvm *kvm =3D vcpu->kvm; + const u64 bytes_remaining =3D range->size; + struct kvm_s2_mmu *mmu =3D &kvm->arch.mmu; /* Canonical. */ + struct kvm_memory_slot *memslot; + const gpa_t gpa =3D range->gpa; + int num_retries =3D 0; + long ret; + + /* + * pKVM is unsupported as their vCPUs are instantiated on first run and + * pre-faulting only running vCPUs would be inconsistent and confusing. + */ + if (is_protected_kvm_enabled()) + return -EOPNOTSUPP; + + memslot =3D gfn_to_memslot(kvm, gpa_to_gfn(gpa)); + if (!memslot) + return -ENOENT; + /* SRCU must be released for progress and only userland can do that. */ + if (memslot->flags & KVM_MEMSLOT_INVALID) + return -EAGAIN; + + do { + ret =3D pre_fault_s2(mmu, vcpu, gpa, memslot); + } while (ret =3D=3D -EAGAIN && num_retries++ < MAX_PRE_FAULT_RETRIES); + + if (IS_ERR_VALUE(ret)) + return ret; + return pre_fault_bytes_consumed(gpa, ret, bytes_remaining); +} --=20 2.55.0 From nobody Mon Sep 28 06:34:45 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5B3D334041D; Tue, 25 Aug 2026 16:01:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787673710; cv=none; b=LRrJIpn5yBnv/xEV0ZzrJl5TBJxATpL3C9e5pTHisDv324zQgF+lQe9CDWqnpWxtLu1j1vvntyx8hzvklEdsyDEIwKoNBnIAhsYWRbglQRoC/LaQGDdsJO6+assTfJ7M1UXuwwC11NkQz/wIrZqEFzmpxoRfT45OtjhfX2c+/yo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787673710; c=relaxed/simple; bh=IZ/Xs5TvGOQD7uw0TKqgAfrUBwPUr+hKALeLx177jmU=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=ankVbQWc1t0oqfpomebgems2jjJ4/e+BCaIgjQyx3Zm+Qu5Ki+puMfp0mxotrvxOUbCAjlq8z8BBW5ggqRlT8mYqnK2zrUUs5MOltsNEPZGEgIHrvO11wxKimrbVA+dDHr6garkwy+PD915AiEtUUymQCtiaKp9XojX/yOXqflI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=gNyhkK1J; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="gNyhkK1J" Received: by smtp.kernel.org (Postfix) with ESMTPSA id C7BCA1F000E9; Tue, 25 Aug 2026 16:01:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1787673709; bh=2QmTKWBAQNYmpr1gvX/qlAg50jxayUk+Ya6iK3WkxcY=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=gNyhkK1JMWcr3YP3IzLLN1sB2uxkEZtANLng+EY+H6V7Xqt8FiWpTP2E8Ik9tgwOW B7MGAL5rUBqJYwftpVXaaVy/RUqwkc5DIKfmrz6goizPmWdVZhEDV5NfNx1AQwHuZh MZikVf/oza+X3vU8UyG46+eQ4+JjjjJPuMac3dGR8xgByEwLOnBxJKLgcx8Jp0sps3 j+iIQobJw0xeRel5/cEfcyW7Q1/b+yH7EY71Gu45mGAnFuFCUrqCCIV8F4iQbrnyY7 alhP8mVwGENMRw1IPCTIKaZgSi52pf9OTc7RfXv+U5hSdp57MUOFtTSrts+H0WoVNo EKFkQyazOhEMg== From: "Lorenzo Stoakes (ARM)" Date: Tue, 25 Aug 2026 17:00:40 +0100 Subject: [PATCH 6/8] KVM: selftests: Enable pre_fault_memory_test for arm64 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260825-kvm-arm-prefault-v1-6-befe8947702e@kernel.org> References: <20260825-kvm-arm-prefault-v1-0-befe8947702e@kernel.org> In-Reply-To: <20260825-kvm-arm-prefault-v1-0-befe8947702e@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , Fuad Tabba , Joey Gouly , Steffen Eiden , Suzuki K Poulose , Zenghui Yu , Paolo Bonzini , Jonathan Corbet Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Jack Thomson , Jack Thomson , Alexandru Elisei , Vincent Donnefort , "Aneesh Kumar K.V" , Sean Christopherson , Claudio Imbrenda , Leo Soares Passos , "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=7536; i=ljs@kernel.org; h=from:subject:message-id; bh=ipOghwD1cyVJpKN91gzBqnU0uoGWRVuwEpkouBW6IVo=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLJ69zgf6zhyxHjJ/4qU1g06p5L3SETIs23uyje1l6rRd Mw922DfUcrCIMbFICumyPL8i/j+IJGweZ0X/N1g5rAygQxh4OIUgIkkJjH8szd46tllFfT2zZNJ gqu/qFu4RW+w2Rqlx+2/qvXGj20FRgz/807J+zcXZ2+a/dXveVRjnszH/Ncz/mWuiJnD8WnOUeV z7AA= X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 From: Jack Thomson Enable the pre_fault_memory_test to run on arm64 by making it work with different guest page sizes and testing multiple guest configurations. Update the test_assert to compare against the UCALL_EXIT_REASON, for portability, as arm64 exits with KVM_EXIT_MMIO while x86 uses KVM_EXIT_IO. Signed-off-by: Jack Thomson [ljs: merge conflict resolution with commit c847704619da] Signed-off-by: Lorenzo Stoakes (ARM) Tested-by: Fuad Tabba < fuad.tabba@linux.dev> --- tools/testing/selftests/kvm/Makefile.kvm | 1 + .../testing/selftests/kvm/pre_fault_memory_test.c | 117 +++++++++++++++++= ---- 2 files changed, 96 insertions(+), 22 deletions(-) diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selft= ests/kvm/Makefile.kvm index 5f8d17ee5f67..66a9a4e62888 100644 --- a/tools/testing/selftests/kvm/Makefile.kvm +++ b/tools/testing/selftests/kvm/Makefile.kvm @@ -199,6 +199,7 @@ TEST_GEN_PROGS_arm64 +=3D guest_memfd_test TEST_GEN_PROGS_arm64 +=3D mmu_stress_test TEST_GEN_PROGS_arm64 +=3D rseq_test TEST_GEN_PROGS_arm64 +=3D steal_time +TEST_GEN_PROGS_arm64 +=3D pre_fault_memory_test =20 TEST_GEN_PROGS_s390 =3D $(TEST_GEN_PROGS_COMMON) TEST_GEN_PROGS_s390 +=3D s390/memop diff --git a/tools/testing/selftests/kvm/pre_fault_memory_test.c b/tools/te= sting/selftests/kvm/pre_fault_memory_test.c index a0fcae3cb7a8..ee77c51cc566 100644 --- a/tools/testing/selftests/kvm/pre_fault_memory_test.c +++ b/tools/testing/selftests/kvm/pre_fault_memory_test.c @@ -12,19 +12,29 @@ #include #include #include +#include =20 /* Arbitrarily chosen values */ -#define TEST_SIZE (SZ_2M + PAGE_SIZE) -#define TEST_NPAGES (TEST_SIZE / PAGE_SIZE) +#define TEST_BASE_SIZE SZ_2M #define TEST_SLOT 10 =20 +/* Storage of test info to share with guest code */ +struct test_config { + u64 page_size; + u64 test_size; + u64 test_num_pages; +}; + +static struct test_config test_config; + static void guest_code(u64 base_gva) { volatile u64 val __used; + struct test_config *config =3D &test_config; int i; =20 - for (i =3D 0; i < TEST_NPAGES; i++) { - u64 *src =3D (u64 *)(base_gva + i * PAGE_SIZE); + for (i =3D 0; i < config->test_num_pages; i++) { + u64 *src =3D (u64 *)(base_gva + i * config->page_size); =20 val =3D *src; } @@ -57,7 +67,7 @@ static void *delete_slot_worker(void *__data) cpu_relax(); =20 vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS, data->gpa, - TEST_SLOT, TEST_NPAGES, data->flags); + TEST_SLOT, test_config.test_num_pages, data->flags); =20 return NULL; } @@ -150,8 +160,8 @@ static void pre_fault_memory(struct kvm_vcpu *vcpu, u64= base_gpa, u64 offset, /* * Assert success if prefaulting the entire range should succeed, i.e. * complete with no bytes remaining. Otherwise prefaulting should have - * failed due to ENOENT (due to RET_PF_EMULATE for emulated MMIO when - * no memslot exists). + * failed due to ENOENT (no memslot exists for the GPA; on x86 this + * surfaces via RET_PF_EMULATE). */ if (!expected_left) TEST_ASSERT_VM_VCPU_IOCTL(!ret, KVM_PRE_FAULT_MEMORY, ret, vcpu->vm); @@ -160,39 +170,72 @@ static void pre_fault_memory(struct kvm_vcpu *vcpu, u= 64 base_gpa, u64 offset, KVM_PRE_FAULT_MEMORY, ret, vcpu->vm); } =20 -static void __test_pre_fault_memory(unsigned long vm_type, bool private) +struct test_params { + unsigned long vm_type; + bool private; +}; + +static void __test_pre_fault_memory(enum vm_guest_mode guest_mode, void *a= rg) { - gpa_t gpa, gva, alignment, guest_page_size; + gpa_t gpa, gva, alignment, guest_page_size, host_page_size; + struct test_params *p =3D arg; const struct vm_shape shape =3D { - .mode =3D VM_MODE_DEFAULT, - .type =3D vm_type, + .mode =3D guest_mode, + .type =3D p->vm_type, }; struct kvm_vcpu *vcpu; + struct kvm_run *run; struct kvm_vm *vm; struct ucall uc; =20 + pr_info("Testing guest mode: %s\n", vm_guest_mode_string(guest_mode)); + vm =3D vm_create_shape_with_one_vcpu(shape, &vcpu, guest_code); =20 - alignment =3D guest_page_size =3D vm_guest_mode_params[VM_MODE_DEFAULT].p= age_size; - gpa =3D (vm->max_gfn - TEST_NPAGES) * guest_page_size; + guest_page_size =3D vm_guest_mode_params[guest_mode].page_size; + host_page_size =3D getpagesize(); + + test_config.page_size =3D guest_page_size; + test_config.test_size =3D align_up(TEST_BASE_SIZE + test_config.page_size, + host_page_size); + test_config.test_num_pages =3D vm_calc_num_guest_pages(vm->mode, test_con= fig.test_size); + + gpa =3D (vm->max_gfn - test_config.test_num_pages) * test_config.page_siz= e; alignment =3D SZ_2M; + alignment =3D max(alignment, host_page_size); gpa =3D align_down(gpa, alignment); gva =3D gpa & ((1ULL << (vm->va_bits - 1)) - 1); =20 - vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS, gpa, TEST_SLOT, - TEST_NPAGES, private ? KVM_MEM_GUEST_MEMFD : 0); - virt_map(vm, gva, gpa, TEST_NPAGES); + vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS, + gpa, TEST_SLOT, test_config.test_num_pages, + p->private ? KVM_MEM_GUEST_MEMFD : 0); + virt_map(vm, gva, gpa, test_config.test_num_pages); =20 - if (private) - vm_mem_set_private(vm, gpa, TEST_SIZE); + if (p->private) + vm_mem_set_private(vm, gpa, test_config.test_size); =20 - pre_fault_memory(vcpu, gpa, 0, SZ_2M, 0, private); - pre_fault_memory(vcpu, gpa, SZ_2M, PAGE_SIZE * 2, PAGE_SIZE, private); - pre_fault_memory(vcpu, gpa, TEST_SIZE, PAGE_SIZE, PAGE_SIZE, private); + pre_fault_memory(vcpu, gpa, 0, test_config.test_size, 0, p->private); + /* Retry the same range after the first prefault attempt. */ + pre_fault_memory(vcpu, gpa, 0, test_config.test_size, 0, p->private); + pre_fault_memory(vcpu, gpa, + test_config.test_size - host_page_size, + host_page_size * 2, host_page_size, p->private); + pre_fault_memory(vcpu, gpa, test_config.test_size, + host_page_size, host_page_size, p->private); =20 vcpu_args_set(vcpu, 1, gva); + + /* Export the shared variables to the guest. */ + sync_global_to_guest(vm, test_config); + vcpu_run(vcpu); =20 + run =3D vcpu->run; + TEST_ASSERT(run->exit_reason =3D=3D UCALL_EXIT_REASON, + "Wanted %s, got exit reason: %u (%s)", + exit_reason_str(UCALL_EXIT_REASON), + run->exit_reason, exit_reason_str(run->exit_reason)); + switch (get_ucall(vcpu, &uc)) { case UCALL_ABORT: REPORT_GUEST_ASSERT(uc); @@ -209,16 +252,46 @@ static void __test_pre_fault_memory(unsigned long vm_= type, bool private) =20 static void test_pre_fault_memory(unsigned long vm_type, bool private) { + struct test_params p =3D { + .vm_type =3D vm_type, + .private =3D private, + }; + if (vm_type && !(kvm_check_cap(KVM_CAP_VM_TYPES) & BIT(vm_type))) { pr_info("Skipping tests for vm_type 0x%lx\n", vm_type); return; } =20 - __test_pre_fault_memory(vm_type, private); + for_each_guest_mode(__test_pre_fault_memory, &p); +} + +static void help(char *name) +{ + puts(""); + printf("usage: %s [-h] [-m mode]\n", name); + puts(""); + guest_modes_help(); + puts(""); } =20 int main(int argc, char *argv[]) { + int opt; + + guest_modes_append_default(); + + while ((opt =3D getopt(argc, argv, "hm:")) !=3D -1) { + switch (opt) { + case 'm': + guest_modes_cmdline(optarg); + break; + case 'h': + default: + help(argv[0]); + exit(0); + } + } + TEST_REQUIRE(kvm_check_cap(KVM_CAP_PRE_FAULT_MEMORY)); =20 test_pre_fault_memory(0, false); --=20 2.55.0 From nobody Mon Sep 28 06:34:45 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9930F346FB3; Tue, 25 Aug 2026 16:01:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787673716; cv=none; b=nNNZHxmJMxVw5glmL8oajGh+m5X0xGsgeqwvAg0N8i1619yoDcgD8tRh7hwlC5oGkXJlxHn8vgnTlKgyGtCY/kXwf1jRfNsyEJob8l/a+xnpmx/W21j2jTxLx79Xi7dh8qAr9x69bsGweN+kvuqoO2uthfrKsXry0GusMGQXfEQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787673716; c=relaxed/simple; bh=2mli7Z7Iv34/jPztedKMCf45PGYRXNpv3Rf35FR6iow=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=s7ekGOQfrX9TIQTiST8oZ63v31KmvqwTryWVuDTp1a+QUH/k6laCaohP1rsqtYi6a64jYschlZGF+2NZhSWxR7USv8D19x9YXHeBGHPKKGo/mt68ANCLHhFubSVReLCRNiH5KNDp/E5urNmW/BWSe0J8YHwms6kOL6FaEhEH6OQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=eNYuUnqC; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="eNYuUnqC" Received: by smtp.kernel.org (Postfix) with ESMTPSA id BC3D51F00A3A; Tue, 25 Aug 2026 16:01:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1787673715; bh=cRk7lD7czZ0zVPvtZdVAKc/8RTbrtJAsMOSGvFeFLfA=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=eNYuUnqCPLVIB87qE4SeFKDRvJqxiDjo0944zDhCuee8dUFKmVm9Kmr8UNsiwJEXs pyCes0N4LPtWxeY/14IytRcjf1TaBQR5boFFP8/ammwTtuyWPyFwMR6PUEovPbZVI5 WqbUceY5RBlwVfILRhEVK3vMBbDqkuDTw1fjywItkXmezmv3mIdsBf2HBQpAlWJOq6 lZNoVd0MAzSo4eAI4JIP6uWTDuJpUyrIeAub8avvCfCIWZME14vZqt9O9cAwMTcg+A d1IVzlIXGrlfsulILyOQwnNhV4qq5ouzYHqzhWqxz8HMPnq0NabfCStsj95cVpL4Fy PDRxhneLvC0sQ== From: "Lorenzo Stoakes (ARM)" Date: Tue, 25 Aug 2026 17:00:41 +0100 Subject: [PATCH 7/8] KVM: selftests: Add option for different backing in pre-fault tests Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260825-kvm-arm-prefault-v1-7-befe8947702e@kernel.org> References: <20260825-kvm-arm-prefault-v1-0-befe8947702e@kernel.org> In-Reply-To: <20260825-kvm-arm-prefault-v1-0-befe8947702e@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , Fuad Tabba , Joey Gouly , Steffen Eiden , Suzuki K Poulose , Zenghui Yu , Paolo Bonzini , Jonathan Corbet Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Jack Thomson , Jack Thomson , Alexandru Elisei , Vincent Donnefort , "Aneesh Kumar K.V" , Sean Christopherson , Claudio Imbrenda , Leo Soares Passos , "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=6683; i=ljs@kernel.org; h=from:subject:message-id; bh=PO6VwHVzr1ywPi/wKPjgqxPqz8SXyRcry63dvjVJNh0=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLJ69zgf8LrlIipT1BSXtGfvU9a+I+/unv8pW8Yb+cQ8f nJNmwtvRykLgxgXg6yYIsvzL+L7g0TC5nVe8HeDmcPKBDKEgYtTACay/T8jQ9fuB2zNBf4vd+Q7 GZxK0dAVdw1fm7nzp0bdhgOTGt8q/2dkODDDJFRZ5oBZ4MolKzyaewOu+HrXGVe94Z7F/Gt+d+l CVgA= X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 From: Jack Thomson Add a -s option to specify different memory backing types for the pre-fault tests (e.g. anonymous, hugetlb), allowing testing of the pre-fault functionality across different memory configurations. Signed-off-by: Jack Thomson Signed-off-by: Lorenzo Stoakes (ARM) Tested-by: Fuad Tabba < fuad.tabba@linux.dev> --- .../testing/selftests/kvm/pre_fault_memory_test.c | 51 +++++++++++++++---= ---- 1 file changed, 36 insertions(+), 15 deletions(-) diff --git a/tools/testing/selftests/kvm/pre_fault_memory_test.c b/tools/te= sting/selftests/kvm/pre_fault_memory_test.c index ee77c51cc566..702c72bbfa96 100644 --- a/tools/testing/selftests/kvm/pre_fault_memory_test.c +++ b/tools/testing/selftests/kvm/pre_fault_memory_test.c @@ -46,6 +46,7 @@ struct slot_worker_data { struct kvm_vm *vm; gpa_t gpa; u32 flags; + enum vm_mem_backing_src_type mem_backing_src; bool worker_ready; bool prefault_ready; bool recreate_slot; @@ -66,14 +67,16 @@ static void *delete_slot_worker(void *__data) while (!READ_ONCE(data->recreate_slot)) cpu_relax(); =20 - vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS, data->gpa, + vm_userspace_mem_region_add(vm, data->mem_backing_src, data->gpa, TEST_SLOT, test_config.test_num_pages, data->flags); =20 return NULL; } =20 static void pre_fault_memory(struct kvm_vcpu *vcpu, u64 base_gpa, u64 offs= et, - u64 size, u64 expected_left, bool private) + u64 size, u64 expected_left, + enum vm_mem_backing_src_type mem_backing_src, + bool private) { struct kvm_pre_fault_memory range =3D { .gpa =3D base_gpa + offset, @@ -84,6 +87,7 @@ static void pre_fault_memory(struct kvm_vcpu *vcpu, u64 b= ase_gpa, u64 offset, .vm =3D vcpu->vm, .gpa =3D base_gpa, .flags =3D private ? KVM_MEM_GUEST_MEMFD : 0, + .mem_backing_src =3D mem_backing_src, }; bool slot_recreated =3D false; pthread_t slot_worker; @@ -173,11 +177,13 @@ static void pre_fault_memory(struct kvm_vcpu *vcpu, u= 64 base_gpa, u64 offset, struct test_params { unsigned long vm_type; bool private; + enum vm_mem_backing_src_type mem_backing_src; }; =20 static void __test_pre_fault_memory(enum vm_guest_mode guest_mode, void *a= rg) { gpa_t gpa, gva, alignment, guest_page_size, host_page_size; + gpa_t backing_src_pagesz, mem_page_size; struct test_params *p =3D arg; const struct vm_shape shape =3D { .mode =3D guest_mode, @@ -189,24 +195,28 @@ static void __test_pre_fault_memory(enum vm_guest_mod= e guest_mode, void *arg) struct ucall uc; =20 pr_info("Testing guest mode: %s\n", vm_guest_mode_string(guest_mode)); + pr_info("Testing memory backing src type: %s\n", + vm_mem_backing_src_alias(p->mem_backing_src)->name); =20 vm =3D vm_create_shape_with_one_vcpu(shape, &vcpu, guest_code); =20 guest_page_size =3D vm_guest_mode_params[guest_mode].page_size; host_page_size =3D getpagesize(); + backing_src_pagesz =3D get_backing_src_pagesz(p->mem_backing_src); + mem_page_size =3D max(host_page_size, backing_src_pagesz); =20 test_config.page_size =3D guest_page_size; test_config.test_size =3D align_up(TEST_BASE_SIZE + test_config.page_size, - host_page_size); + mem_page_size); test_config.test_num_pages =3D vm_calc_num_guest_pages(vm->mode, test_con= fig.test_size); =20 gpa =3D (vm->max_gfn - test_config.test_num_pages) * test_config.page_siz= e; alignment =3D SZ_2M; - alignment =3D max(alignment, host_page_size); + alignment =3D max(alignment, mem_page_size); gpa =3D align_down(gpa, alignment); gva =3D gpa & ((1ULL << (vm->va_bits - 1)) - 1); =20 - vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS, + vm_userspace_mem_region_add(vm, p->mem_backing_src, gpa, TEST_SLOT, test_config.test_num_pages, p->private ? KVM_MEM_GUEST_MEMFD : 0); virt_map(vm, gva, gpa, test_config.test_num_pages); @@ -214,14 +224,18 @@ static void __test_pre_fault_memory(enum vm_guest_mod= e guest_mode, void *arg) if (p->private) vm_mem_set_private(vm, gpa, test_config.test_size); =20 - pre_fault_memory(vcpu, gpa, 0, test_config.test_size, 0, p->private); + pre_fault_memory(vcpu, gpa, 0, test_config.test_size, 0, + p->mem_backing_src, p->private); /* Retry the same range after the first prefault attempt. */ - pre_fault_memory(vcpu, gpa, 0, test_config.test_size, 0, p->private); + pre_fault_memory(vcpu, gpa, 0, test_config.test_size, 0, + p->mem_backing_src, p->private); pre_fault_memory(vcpu, gpa, test_config.test_size - host_page_size, - host_page_size * 2, host_page_size, p->private); + host_page_size * 2, host_page_size, + p->mem_backing_src, p->private); pre_fault_memory(vcpu, gpa, test_config.test_size, - host_page_size, host_page_size, p->private); + host_page_size, host_page_size, + p->mem_backing_src, p->private); =20 vcpu_args_set(vcpu, 1, gva); =20 @@ -250,11 +264,13 @@ static void __test_pre_fault_memory(enum vm_guest_mod= e guest_mode, void *arg) kvm_vm_free(vm); } =20 -static void test_pre_fault_memory(unsigned long vm_type, bool private) +static void test_pre_fault_memory(unsigned long vm_type, enum vm_mem_backi= ng_src_type backing_src, + bool private) { struct test_params p =3D { .vm_type =3D vm_type, .private =3D private, + .mem_backing_src =3D backing_src, }; =20 if (vm_type && !(kvm_check_cap(KVM_CAP_VM_TYPES) & BIT(vm_type))) { @@ -268,23 +284,28 @@ static void test_pre_fault_memory(unsigned long vm_ty= pe, bool private) static void help(char *name) { puts(""); - printf("usage: %s [-h] [-m mode]\n", name); + printf("usage: %s [-h] [-m mode] [-s mem-type]\n", name); puts(""); guest_modes_help(); + backing_src_help("-s"); puts(""); } =20 int main(int argc, char *argv[]) { + enum vm_mem_backing_src_type backing =3D DEFAULT_VM_MEM_SRC; int opt; =20 guest_modes_append_default(); =20 - while ((opt =3D getopt(argc, argv, "hm:")) !=3D -1) { + while ((opt =3D getopt(argc, argv, "hm:s:")) !=3D -1) { switch (opt) { case 'm': guest_modes_cmdline(optarg); break; + case 's': + backing =3D parse_backing_src_type(optarg); + break; case 'h': default: help(argv[0]); @@ -294,10 +315,10 @@ int main(int argc, char *argv[]) =20 TEST_REQUIRE(kvm_check_cap(KVM_CAP_PRE_FAULT_MEMORY)); =20 - test_pre_fault_memory(0, false); + test_pre_fault_memory(0, backing, false); #ifdef __x86_64__ - test_pre_fault_memory(KVM_X86_SW_PROTECTED_VM, false); - test_pre_fault_memory(KVM_X86_SW_PROTECTED_VM, true); + test_pre_fault_memory(KVM_X86_SW_PROTECTED_VM, backing, false); + test_pre_fault_memory(KVM_X86_SW_PROTECTED_VM, backing, true); #endif return 0; } --=20 2.55.0 From nobody Mon Sep 28 06:34:45 2026 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-alma10-1.taild15c8.ts.net [100.103.45.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C6920340282; Tue, 25 Aug 2026 16:02:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=100.103.45.18 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787673723; cv=none; b=AwnyUn0Q07jTmRBhCG+M16VrQiUKEjsv668HW6HDowX72YXTpfCoGFh3YHOExvvVz+Q43qp9X6z5bzEsZSJBlmIp7sEkL3MJUayK1CJZIt6s5aFKGHbk2JY785r064EigpaBNURGaRB3FQNRkWGsVf/M8Hpm95Pz1Q9T0fz4HSs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1787673723; c=relaxed/simple; bh=bIU/5QqTt2sESium66hjDrMvxvN9oUs89PlIyr+xpII=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=aUZLZto5KmOSkoErLAguezbT0gCerBnRiRrY05T8J4baOmLD4Q8HiAZhSzXnHQ1hbpEwo1WLkEUQWG6iBvRg01tEX1YqeTseXpYA04bIkQY5B6ZRxidAT26AKl+KaHBi6dG3cwmkI6WnylmUj+8RK35nH2R+zH6JR6J1EgP+EH4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ZQ2fKSMB; arc=none smtp.client-ip=100.103.45.18 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZQ2fKSMB" Received: by smtp.kernel.org (Postfix) with ESMTPSA id B32971F000E9; Tue, 25 Aug 2026 16:01:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel.org; s=k20260515; t=1787673721; bh=tcKcgTzP/MaC1TKqD0rXntx2zTo0kP8ybCcblQSok4Q=; h=From:Date:Subject:References:In-Reply-To:To:Cc; b=ZQ2fKSMB51ICm8a+uQ7XhMAdLeWtrzaJjIxUqvCXFYKol3tlFUd4AeOGCcJf8JVWB AcbHuBEKV6I+Cf+jJD2cmjF5CVXyFzNayEUqS34AZwMmakPJshLz8TZ8XAmvfRX9y1 2ZdOmbtRt8cUJUUXMAUsIjeggGUOmSHg2rKxekC+1NV7XfvPm8ZiV5XIkIhHdPEfvH fzfzT/iKbB6F+55FhYFN75bOwWayybPWtneyUEHzX/3tZDCkbNyU578qTAyLn9JEMw k5bLFZDq14Lpvx0vUz2QjEYdIlUGSDHEnIJDtQADg3HebMt3dA7WxkCzXjhqxB7Z/H bjljMnkItf9Rw== From: "Lorenzo Stoakes (ARM)" Date: Tue, 25 Aug 2026 17:00:42 +0100 Subject: [PATCH 8/8] KVM: selftests: Add nested pre-fault test for arm64 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260825-kvm-arm-prefault-v1-8-befe8947702e@kernel.org> References: <20260825-kvm-arm-prefault-v1-0-befe8947702e@kernel.org> In-Reply-To: <20260825-kvm-arm-prefault-v1-0-befe8947702e@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , Fuad Tabba , Joey Gouly , Steffen Eiden , Suzuki K Poulose , Zenghui Yu , Paolo Bonzini , Jonathan Corbet Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, Jack Thomson , Jack Thomson , Alexandru Elisei , Vincent Donnefort , "Aneesh Kumar K.V" , Sean Christopherson , Claudio Imbrenda , Leo Soares Passos , "Lorenzo Stoakes (ARM)" X-Mailer: b4 0.14.3 X-Developer-Signature: v=1; a=openpgp-sha256; l=8041; i=ljs@kernel.org; h=from:subject:message-id; bh=uxEB4tWFsMMdxFr1BeaQVWSs7h76x9U+I63sFxRh9hg=; b=owGbwMvMwCV2fu7ZrsZH9SKMp9WSGLJ69zgrtNl/tlib23ByjswG0aAD067fmScoZsIqZ/0rb d8s/z3WHaUsDGJcDLJiiizPv4jvDxIJm9d5wd8NZg4rE8gQBi5OAZjI8TkM/2y9Ik/v1grsPSJU 0617vtbsU+/yFTFx9pF9970/nk1QD2NkWLfgRvf2Mv1qbbFZCSZuQkpKHtbPllgoLN+W6SK086A DBwA= X-Developer-Key: i=ljs@kernel.org; a=openpgp; fpr=E7F417BF5214569E89D04F46CF9DCD8A81E27F14 From: Jack Thomson Add an arm64 nested-virt selftest for KVM_PRE_FAULT_MEMORY. The guest enters vEL1 and exits to userspace with a nested/shadow stage-2 MMU as the vCPU's last-run context. Before prefaulting, userspace enables HCR_EL2.VM and points VTTBR_EL2 at an empty nested stage-2 root. A prefault implementation that incorrectly treats the userspace GPA as an L2 IPA will fail the ioctl; the correct path targets the canonical stage-2 and succeeds. Restore the original nested state before resuming the guest, then touch the prefaulted range to check that vEL1 still runs correctly. Signed-off-by: Jack Thomson [ljs: partial progress, >4 KiB pgsize, commit msg, comment fixups] Signed-off-by: Lorenzo Stoakes (ARM) Tested-by: Fuad Tabba < fuad.tabba@linux.dev> --- tools/testing/selftests/kvm/Makefile.kvm | 1 + .../selftests/kvm/arm64/nv_pre_fault_memory_test.c | 206 +++++++++++++++++= ++++ 2 files changed, 207 insertions(+) diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selft= ests/kvm/Makefile.kvm index 66a9a4e62888..1e0edd0f99f7 100644 --- a/tools/testing/selftests/kvm/Makefile.kvm +++ b/tools/testing/selftests/kvm/Makefile.kvm @@ -173,6 +173,7 @@ TEST_GEN_PROGS_arm64 +=3D arm64/debug-exceptions TEST_GEN_PROGS_arm64 +=3D arm64/hello_el2 TEST_GEN_PROGS_arm64 +=3D arm64/host_sve TEST_GEN_PROGS_arm64 +=3D arm64/hypercalls +TEST_GEN_PROGS_arm64 +=3D arm64/nv_pre_fault_memory_test TEST_GEN_PROGS_arm64 +=3D arm64/external_aborts TEST_GEN_PROGS_arm64 +=3D arm64/mmio_sign_ext TEST_GEN_PROGS_arm64 +=3D arm64/page_fault_test diff --git a/tools/testing/selftests/kvm/arm64/nv_pre_fault_memory_test.c b= /tools/testing/selftests/kvm/arm64/nv_pre_fault_memory_test.c new file mode 100644 index 000000000000..b3833aea2a34 --- /dev/null +++ b/tools/testing/selftests/kvm/arm64/nv_pre_fault_memory_test.c @@ -0,0 +1,206 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * nv_pre_fault_memory_test - Test KVM_PRE_FAULT_MEMORY on a vCPU whose + * last-run context is nested. + * + * The guest starts at vEL2, mirrors its EL2 translation regime into the + * real EL1 registers, drops HCR_EL2.TGE and ERETs to vEL1, then exits to + * userspace from vEL1 so that the vCPU's last-run context selects a + * shadow stage-2 MMU. Userspace then enables an empty nested stage-2 + * before prefaulting. Prefaulting must target the canonical stage-2, + * regardless of the vCPU's nested state. + */ +#include "kvm_util.h" +#include "processor.h" +#include "test_util.h" +#include "ucall.h" + +#include +#include + +#define TEST_MEM_SLOT 10 +#define NESTED_S2_ROOT_SLOT 11 +#define TEST_MEM_SIZE SZ_2M +#define TEST_MEM_GPA SZ_1G +#define NESTED_S2_ROOT_GPA (TEST_MEM_GPA + TEST_MEM_SIZE) + +struct nested_s2_state { + u64 hcr_el2; + u64 vttbr_el2; +}; + +static void guest_el1_code(void) +{ + u64 offset; + + GUEST_ASSERT_EQ(get_current_el(), 1); + + /* Exit to userspace with the vEL1 (nested) context live. */ + GUEST_SYNC(1); + + /* + * Touch the prefaulted range. vstage-2 is disabled, so the shadow + * stage-2 is a 1:1 view of the canonical IPA space. + */ + for (offset =3D 0; offset < TEST_MEM_SIZE; offset +=3D SZ_4K) + READ_ONCE(*(u64 *)(TEST_MEM_GPA + offset)); + + GUEST_DONE(); +} + +static void guest_code(void) +{ + u64 sp; + + GUEST_ASSERT_EQ(get_current_el(), 2); + + /* + * Mirror the EL2 translation regime into the real EL1 registers so + * that vEL1 runs on the test's stage-1 page tables. With E2H=3D1, the + * _EL1 accessors read the EL2 registers, and the _EL12 accessors + * write the real EL1 registers. + */ + write_sysreg_s(read_sysreg(sctlr_el1), SYS_SCTLR_EL12); + write_sysreg_s(read_sysreg(tcr_el1), SYS_TCR_EL12); + write_sysreg_s(read_sysreg(ttbr0_el1), SYS_TTBR0_EL12); + write_sysreg_s(read_sysreg(mair_el1), SYS_MAIR_EL12); + write_sysreg_s(read_sysreg(cpacr_el1), SYS_CPACR_EL12); + + /* Run vEL1 on the same stack. */ + asm volatile("mov %0, sp" : "=3Dr"(sp)); + write_sysreg(sp, sp_el1); + + /* + * Drop TGE so that vEL1 is a nested context rather than host EL0. + * KVM backs it with a shadow stage-2 MMU even though vstage-2 is + * disabled (HCR_EL2.VM=3D0). + */ + write_sysreg(read_sysreg(hcr_el2) & ~HCR_EL2_TGE, hcr_el2); + isb(); + + write_sysreg(PSR_MODE_EL1h | PSR_F_BIT | PSR_I_BIT | PSR_A_BIT | + PSR_D_BIT, spsr_el2); + write_sysreg((u64)guest_el1_code, elr_el2); + asm volatile("eret"); + + GUEST_ASSERT(false); +} + +static void pre_fault(struct kvm_vcpu *vcpu, u64 gpa, u64 size) +{ + struct kvm_pre_fault_memory range =3D { + .gpa =3D gpa, + .size =3D size, + }; + int ret; + + do { + ret =3D __vcpu_ioctl(vcpu, KVM_PRE_FAULT_MEMORY, &range); + } while ((!ret && range.size) || + (ret < 0 && (errno =3D=3D EINTR || errno =3D=3D EAGAIN))); + + TEST_ASSERT(!ret, "KVM_PRE_FAULT_MEMORY failed, ret: %d errno: %d", + ret, errno); + TEST_ASSERT_EQ(range.size, 0); +} + +static struct nested_s2_state enable_empty_nested_s2(struct kvm_vcpu *vcpu) +{ + struct nested_s2_state state =3D { + .hcr_el2 =3D vcpu_get_reg(vcpu, KVM_ARM64_SYS_REG(SYS_HCR_EL2)), + .vttbr_el2 =3D vcpu_get_reg(vcpu, + KVM_ARM64_SYS_REG(SYS_VTTBR_EL2)), + }; + + TEST_ASSERT(!(state.hcr_el2 & HCR_EL2_TGE), + "vCPU should be in nested/vEL1 context"); + + vcpu_set_reg(vcpu, KVM_ARM64_SYS_REG(SYS_VTTBR_EL2), + NESTED_S2_ROOT_GPA); + vcpu_set_reg(vcpu, KVM_ARM64_SYS_REG(SYS_HCR_EL2), + state.hcr_el2 | HCR_EL2_VM); + + return state; +} + +static void restore_nested_s2(struct kvm_vcpu *vcpu, + struct nested_s2_state *state) +{ + vcpu_set_reg(vcpu, KVM_ARM64_SYS_REG(SYS_HCR_EL2), state->hcr_el2); + vcpu_set_reg(vcpu, KVM_ARM64_SYS_REG(SYS_VTTBR_EL2), + state->vttbr_el2); +} + +int main(void) +{ + struct nested_s2_state s2; + struct kvm_vcpu_init init; + struct kvm_vcpu *vcpu; + struct kvm_vm *vm; + struct ucall uc; + u64 npages; + + TEST_REQUIRE(kvm_check_cap(KVM_CAP_ARM_EL2)); + TEST_REQUIRE(kvm_check_cap(KVM_CAP_PRE_FAULT_MEMORY)); + + vm =3D vm_create(1); + + kvm_get_default_vcpu_target(vm, &init); + init.features[0] |=3D BIT(KVM_ARM_VCPU_HAS_EL2); + vcpu =3D aarch64_vcpu_add(vm, 0, &init, guest_code); + kvm_arch_vm_finalize_vcpus(vm); + + npages =3D TEST_MEM_SIZE / vm->page_size; + vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS, TEST_MEM_GPA, + TEST_MEM_SLOT, npages, 0); + virt_map(vm, TEST_MEM_GPA, TEST_MEM_GPA, npages); + + vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS, + NESTED_S2_ROOT_GPA, NESTED_S2_ROOT_SLOT, + vm_adjust_num_guest_pages(vm->mode, 1), 0); + + /* Run the guest until it has ERET'd from vEL2 to vEL1. */ + vcpu_run(vcpu); + switch (get_ucall(vcpu, &uc)) { + case UCALL_SYNC: + TEST_ASSERT_EQ(uc.args[1], 1); + break; + case UCALL_ABORT: + REPORT_GUEST_ASSERT(uc); + break; + default: + TEST_FAIL("Unhandled ucall: %ld", uc.cmd); + } + + /* + * The vCPU's last-run context is vEL1, backed by a shadow stage-2 + * MMU. Enable nested stage-2 with an empty root so that the ioctl + * fails if it tries to interpret the userspace GPA as an L2 IPA. + * + * Prefault in two halves so that the second ioctl exercises a + * repeated shadow-MMU attach and canonical stage-2 swap. + * + * (Note that an implementation that wrongly populates shadow + * stage-2 page tables would not be caught as userland can't + * inspect these.) + */ + s2 =3D enable_empty_nested_s2(vcpu); + pre_fault(vcpu, TEST_MEM_GPA, TEST_MEM_SIZE / 2); + pre_fault(vcpu, TEST_MEM_GPA + TEST_MEM_SIZE / 2, TEST_MEM_SIZE / 2); + restore_nested_s2(vcpu, &s2); + + /* Resume at vEL1 and touch the prefaulted range. */ + vcpu_run(vcpu); + switch (get_ucall(vcpu, &uc)) { + case UCALL_DONE: + break; + case UCALL_ABORT: + REPORT_GUEST_ASSERT(uc); + break; + default: + TEST_FAIL("Unhandled ucall: %ld", uc.cmd); + } + + kvm_vm_free(vm); + return 0; +} --=20 2.55.0