From nobody Thu Nov 28 12:36:37 2024 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B7ED91C0DF5; Tue, 1 Oct 2024 11:31:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727782276; cv=none; b=CiuNJ1V7Wiv/7tRX0qzO5khUChNb3aEDpj3WO6fkHhhglm/yY7RhEC9rO8Vu8vZ52jXv1VfdcAxyELHQmKJ8Q+Omr2CCVZQDEMlEkztH6BWQDE1czkrrvreIV00JIkxZhmskMx2l+WDTCpHG2g42YPUfZAZ9UYbP9kXbIgf8xnc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727782276; c=relaxed/simple; bh=/8kAC4uyafUpiRRGXlRqbKMN4dLMMXoXHxOXxaPQDDQ=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=A2SG+4iNT6yd6XYIJquAFaxjGNKJj0r2MHYKgKLaBUnIXlEulXjipKrysWN8z/FFq2AJBjAhVkpEszG3PWooMZgGgexcHy6ITe0LlWzBnm6pU0wRV9HbsfDZ4euGt/RAG3wZF5frJI4M81WYo8i6Rt3JNJIoXujzyM97S4rpBFQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=T5Umuq7l; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="T5Umuq7l" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 30BC4C4CECD; Tue, 1 Oct 2024 11:31:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727782276; bh=/8kAC4uyafUpiRRGXlRqbKMN4dLMMXoXHxOXxaPQDDQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=T5Umuq7ljlHWXhSLe/eIw8E95ukpdJgREL82FZYdJJxHAVrGCZSWDIckIz7LVxQKp 3tIWa9HnmYvQl2nd2+VeaCoz0P3x9R+DWElTguisIqt3Wosl/cFoZyIEb9Sr6WiPUx rsNggS0pfUwCI5P0uYcNEUYH6FBQftvAJZn92YAx01DQS9eyiFqReamTnES6Vw0q/L rqcrPFCFINIW7a3eSb3ySpan+9FOrd1kn0HMD0Mjkvrp4+w2D26bQbLHMwZDeqXeM+ qyPtRFnc0ZjgMjtQxYSobYY4RkH9iV1FuiGbjsWgNGmVrqDdpxeDPnX85em1jr7JSg BZxCJowBjoO/A== From: Mark Brown Date: Tue, 01 Oct 2024 12:29:06 +0100 Subject: [PATCH v7 1/4] arm64/fpsimd: Introduce __bit_to_vl() helper Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20241001-kvm-arm64-fix-pkvm-sve-vl-v7-1-7b0171a36695@kernel.org> References: <20241001-kvm-arm64-fix-pkvm-sve-vl-v7-0-7b0171a36695@kernel.org> In-Reply-To: <20241001-kvm-arm64-fix-pkvm-sve-vl-v7-0-7b0171a36695@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Fuad Tabba Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=3064; i=broonie@kernel.org; h=from:subject:message-id; bh=/8kAC4uyafUpiRRGXlRqbKMN4dLMMXoXHxOXxaPQDDQ=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBm+918K2dkt+RyTe17B6hQuE/kqMhTsruB9gMWraGq ay4fjSqJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZvvdfAAKCRAk1otyXVSH0ADvB/ 47xx2/156+0n1vm9GXaOjaUWX7fxRYKsC/VK02LjPI2bwp0DxUXHhJAidXhkijDjq6DuXua6jkqdDD kEQN3rBkcn/KcMLk/jkYToMkAcWfeoRFqWzIXZrGQR+3j5E4NHoGxv5yFTKNPNyNkkd3sNW2ReTBne ni4Hi3asHXhG20ls6oh42qwpnDXWAvCG3InygpC60/mxi7LuGdvDeIW6y5gxMzqdjRmP/vy0A1MoYr wdM8ices0RXqLg/lepRarbl0U+kacax9uEc9njx4VrUGsjIXne2rIMqG6bTgYK43yXlruLL0J3lRPj Mscn4z+5Oi8cIZwzIk2cHF7yKWo2S9 X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB In all cases where we use the existing __bit_to_vq() helper we immediately convert the result into a VL. Provide and use __bit_to_vl() doing this directly. Acked-by: Catalin Marinas Signed-off-by: Mark Brown --- arch/arm64/include/asm/fpsimd.h | 4 ++++ arch/arm64/kernel/fpsimd.c | 12 ++++++------ 2 files changed, 10 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsim= d.h index f2a84efc3618..3c30da35c285 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -170,6 +170,10 @@ static inline unsigned int __bit_to_vq(unsigned int bi= t) return SVE_VQ_MAX - bit; } =20 +static inline unsigned int __bit_to_vl(unsigned int bit) +{ + return sve_vl_from_vq(__bit_to_vq(bit)); +} =20 struct vl_info { enum vec_type type; diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 77006df20a75..8a080dbd8988 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -530,7 +530,7 @@ static unsigned int find_supported_vector_length(enum v= ec_type type, =20 bit =3D find_next_bit(info->vq_map, SVE_VQ_MAX, __vq_to_bit(sve_vq_from_vl(vl))); - return sve_vl_from_vq(__bit_to_vq(bit)); + return __bit_to_vl(bit); } =20 #if defined(CONFIG_ARM64_SVE) && defined(CONFIG_SYSCTL) @@ -1103,7 +1103,7 @@ int vec_verify_vq_map(enum vec_type type) * Mismatches above sve_max_virtualisable_vl are fine, since * no guest is allowed to configure ZCR_EL2.LEN to exceed this: */ - if (sve_vl_from_vq(__bit_to_vq(b)) <=3D info->max_virtualisable_vl) { + if (__bit_to_vl(b) <=3D info->max_virtualisable_vl) { pr_warn("%s: cpu%d: Unsupported vector length(s) present\n", info->name, smp_processor_id()); return -EINVAL; @@ -1169,7 +1169,7 @@ void __init sve_setup(void) set_bit(__vq_to_bit(SVE_VQ_MIN), info->vq_map); =20 max_bit =3D find_first_bit(info->vq_map, SVE_VQ_MAX); - info->max_vl =3D sve_vl_from_vq(__bit_to_vq(max_bit)); + info->max_vl =3D __bit_to_vl(max_bit); =20 /* * For the default VL, pick the maximum supported value <=3D 64. @@ -1188,7 +1188,7 @@ void __init sve_setup(void) /* No virtualisable VLs? This is architecturally forbidden. */ info->max_virtualisable_vl =3D SVE_VQ_MIN; else /* b + 1 < SVE_VQ_MAX */ - info->max_virtualisable_vl =3D sve_vl_from_vq(__bit_to_vq(b + 1)); + info->max_virtualisable_vl =3D __bit_to_vl(b + 1); =20 if (info->max_virtualisable_vl > info->max_vl) info->max_virtualisable_vl =3D info->max_vl; @@ -1305,10 +1305,10 @@ void __init sme_setup(void) WARN_ON(bitmap_empty(info->vq_map, SVE_VQ_MAX)); =20 min_bit =3D find_last_bit(info->vq_map, SVE_VQ_MAX); - info->min_vl =3D sve_vl_from_vq(__bit_to_vq(min_bit)); + info->min_vl =3D __bit_to_vl(min_bit); =20 max_bit =3D find_first_bit(info->vq_map, SVE_VQ_MAX); - info->max_vl =3D sve_vl_from_vq(__bit_to_vq(max_bit)); + info->max_vl =3D __bit_to_vl(max_bit); =20 WARN_ON(info->min_vl > info->max_vl); =20 --=20 2.39.2 From nobody Thu Nov 28 12:36:37 2024 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 704331C1ACC; Tue, 1 Oct 2024 11:31:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727782279; cv=none; b=YyvlVGBf0+NZgiqylwJCVvP5xUK2lzTmkE8fX6DLXknkH3NfHBcHhgE6QNzZ6sstLvvzDQmyj1qEelL9nos2BeoMChSWg6q0tKV2P+i+fTNiNF2oLDPevlWBvubEbIPNmXD0VfsNpMcDUgv6tD0lA4wfrmi0feFivTJWGmRa4YE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727782279; c=relaxed/simple; bh=T8FjmI+5TxAQIynPAIQoQkfPmrHt9l4M/rmCIDXWNk8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Ie8k8Tx89ahz+lkFdlcQDLCKyTJo4Y5BwDlZPPu/f/uUsobZp/lsaltIHKcwtBbREngpVLA3920LtbpIerqvDn4J5PdXXnLD1F48AfSpqbEx71RtZC7eK6fDhPY6VW7g5Z/qcD9lKd18q9tER+RzKy6A1PlXJTGlbQUX1PWsnGI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=Niupd6pT; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="Niupd6pT" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 08E53C4CEC6; Tue, 1 Oct 2024 11:31:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727782279; bh=T8FjmI+5TxAQIynPAIQoQkfPmrHt9l4M/rmCIDXWNk8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Niupd6pTw/ZNvqwxoEImGBsdNK8Psgwq8K6LKc4ziELCbvvS0OVtx/T0EN6fBiPnd pK8ioArLpJv2NZWn/en39KA5U3aZxJ5xDZgQyWgwrTvrRakq1H1La3O0Pnjt5MVn/j JLQKYgTtF2yfhUs/+j2BS0YHQE8vmptykquie25DsL2apAXGouXTx6nveONf6Vlg9w G8On63/UDgyvG4R85fTUfi3kkUTR5MGXXwNf3lJI/TAEV2HZpNcqFr8hQOhLAkedU6 yWVwbIo8sfF0/bdaEWixDvv3Vzy4qnFTVFtVs5Up8V9QjfxAGyyCCb7/fJOPj/jWCP hvED56DYkN/Ag== From: Mark Brown Date: Tue, 01 Oct 2024 12:29:07 +0100 Subject: [PATCH v7 2/4] arm64/fpsimd: Discover maximum vector length implemented by any CPU Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20241001-kvm-arm64-fix-pkvm-sve-vl-v7-2-7b0171a36695@kernel.org> References: <20241001-kvm-arm64-fix-pkvm-sve-vl-v7-0-7b0171a36695@kernel.org> In-Reply-To: <20241001-kvm-arm64-fix-pkvm-sve-vl-v7-0-7b0171a36695@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Fuad Tabba Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=4205; i=broonie@kernel.org; h=from:subject:message-id; bh=T8FjmI+5TxAQIynPAIQoQkfPmrHt9l4M/rmCIDXWNk8=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBm+919ztNN3f1p7RnUQbLl+kj+u0eNguOZjwmLwrCH RRxsJ7uJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZvvdfQAKCRAk1otyXVSH0HE+B/ 9TyWHjKOKfGCjZIC6IfjLAEsXXFeXkrEwn6BlS9R81DdltxbYEk4weYzWvbTIe7A2JO52fqD51KLVn 4Fl7BLZny2+KsuDXIjp8LRdmXg+vmeKJ6DPY8pLQYCDUy4JtexwtLVDT/ptuX+FYC5gZaV84wgCM5Y E0kZPQMf3JM3vRzMo7SbQiQdZF9Rkio1ZRlvaNriPgg9SJdXitsGSr9ciTQ3iIInNXbHuYVl+wH5yM /DnCYlZRIs13UaBO2wxIqrFCcIIzL15k1c8cgZXWaeh6YlxTmcD+mBBbRYTRAaJgzpu7IZvRlCR08Y aD7eXunfqinvaatVVlbQ2Jb7R/a8tx X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB When discovering the vector lengths for SVE and SME we do not currently record the maximum VL supported on any individual CPU. This is expected to be the same for all CPUs but the architecture allows asymmetry, if we do encounter an asymmetric system then some CPUs may support VLs higher than the maximum Linux will use. Since the pKVM hypervisor needs to support saving and restoring anything the host can physically set it needs to know the maximum value any CPU could have, add support for enumerating it and validation for late CPUs. Acked-by: Catalin Marinas Signed-off-by: Mark Brown --- arch/arm64/include/asm/fpsimd.h | 13 +++++++++++++ arch/arm64/kernel/fpsimd.c | 26 +++++++++++++++++++++++++- 2 files changed, 38 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsim= d.h index 3c30da35c285..754b0eb09fe8 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -186,6 +186,9 @@ struct vl_info { int max_vl; int max_virtualisable_vl; =20 + /* Maximum vector length observed on any CPU */ + int max_cpu_vl; + /* * Set of available vector lengths, * where length vq encoded as bit __vq_to_bit(vq): @@ -276,6 +279,11 @@ static inline int vec_max_virtualisable_vl(enum vec_ty= pe type) return vl_info[type].max_virtualisable_vl; } =20 +static inline int vec_max_cpu_vl(enum vec_type type) +{ + return vl_info[type].max_cpu_vl; +} + static inline int sve_max_vl(void) { return vec_max_vl(ARM64_VEC_SVE); @@ -286,6 +294,11 @@ static inline int sve_max_virtualisable_vl(void) return vec_max_virtualisable_vl(ARM64_VEC_SVE); } =20 +static inline int sve_max_cpu_vl(void) +{ + return vec_max_cpu_vl(ARM64_VEC_SVE); +} + /* Ensure vq >=3D SVE_VQ_MIN && vq <=3D SVE_VQ_MAX before calling this fun= ction */ static inline bool vq_available(enum vec_type type, unsigned int vq) { diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 8a080dbd8988..0bf0837d4adb 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -129,6 +129,7 @@ __ro_after_init struct vl_info vl_info[ARM64_VEC_MAX] = =3D { .min_vl =3D SVE_VL_MIN, .max_vl =3D SVE_VL_MIN, .max_virtualisable_vl =3D SVE_VL_MIN, + .max_cpu_vl =3D SVE_VL_MIN, }, #endif #ifdef CONFIG_ARM64_SME @@ -1041,8 +1042,13 @@ static void vec_probe_vqs(struct vl_info *info, void __init vec_init_vq_map(enum vec_type type) { struct vl_info *info =3D &vl_info[type]; + unsigned long b; + vec_probe_vqs(info, info->vq_map); bitmap_copy(info->vq_partial_map, info->vq_map, SVE_VQ_MAX); + + b =3D find_first_bit(info->vq_map, SVE_VQ_MAX); + info->max_cpu_vl =3D __bit_to_vl(b); } =20 /* @@ -1054,11 +1060,16 @@ void vec_update_vq_map(enum vec_type type) { struct vl_info *info =3D &vl_info[type]; DECLARE_BITMAP(tmp_map, SVE_VQ_MAX); + unsigned long b; =20 vec_probe_vqs(info, tmp_map); bitmap_and(info->vq_map, info->vq_map, tmp_map, SVE_VQ_MAX); bitmap_or(info->vq_partial_map, info->vq_partial_map, tmp_map, SVE_VQ_MAX); + + b =3D find_first_bit(tmp_map, SVE_VQ_MAX); + if (__bit_to_vl(b) > info->max_cpu_vl) + info->max_cpu_vl =3D __bit_to_vl(b); } =20 /* @@ -1069,10 +1080,23 @@ int vec_verify_vq_map(enum vec_type type) { struct vl_info *info =3D &vl_info[type]; DECLARE_BITMAP(tmp_map, SVE_VQ_MAX); - unsigned long b; + unsigned long b, max_vl; =20 vec_probe_vqs(info, tmp_map); =20 + /* + * Currently the maximum VL is only used for pKVM which + * doesn't allow late CPUs but we don't expect asymmetry and + * if we encounter any then future users will need handling so + * warn if we see anything. + */ + max_vl =3D __bit_to_vl(find_first_bit(tmp_map, SVE_VQ_MAX)); + if (max_vl > info->max_cpu_vl) { + pr_warn("%s: cpu%d: increases maximum VL to %lu\n", + info->name, smp_processor_id(), max_vl); + info->max_cpu_vl =3D max_vl; + } + bitmap_complement(tmp_map, tmp_map, SVE_VQ_MAX); if (bitmap_intersects(tmp_map, info->vq_map, SVE_VQ_MAX)) { pr_warn("%s: cpu%d: Required vector length(s) missing\n", --=20 2.39.2 From nobody Thu Nov 28 12:36:37 2024 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 01EC51C2429; Tue, 1 Oct 2024 11:31:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727782282; cv=none; b=ItKZaFdBxAzGchGULKOYeZwTCUazvGpcxlVq8m5tNpJ17bLVYVNOT4E2F97K3b2GfEgdqlGXjeSrmM10JP4sbBqnyPUpQj+HzjsWS36TU6QXzIYgdukDgVT7WJqTkf+V6i1TmI6o3QXcJlqQh6XXYzrHd69r7ZZ/6fww3Bdy1UQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727782282; c=relaxed/simple; bh=KBunKuidQbGsl1sTPDqT1ibl0GGL5dnEQFUfFDn0MP8=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=TpPlcxYGyLYNPxnvdtkB++bp4kvTG30XdoZz57aqWYaf90P8yLs3YyGAYqj+aIfvbfJ4QIxwfOVL4YqozaBFil/aqO6T46I9FZZU4m1N0w0fDQ9TggeaCgyxDsz7qZNJZbchVqpCiO+NUJFQPIqhsgnA40NbANYD5BGDxWwnfOg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=f2utS53a; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="f2utS53a" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 96B0CC4CEC6; Tue, 1 Oct 2024 11:31:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727782281; bh=KBunKuidQbGsl1sTPDqT1ibl0GGL5dnEQFUfFDn0MP8=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=f2utS53aPNMtAdR2kLn87aubYqKj6soMe/MmzTiwRNusjBY9Kvtd6t8lsrYC66qcT 2caAWmgTZjnL+MHHlShC8N1E5MZh1IxjUtqpkiWJaPvzht7rXS+SsapTNPfR7AnDBL kd0KFzAU0ScB+S3JML/yRqKEEu9pw3uN7bwg+5xpWUHyNm1helyuJvmXq9TPBplb8A nPPMnzlWOtlnFGdwgGboF1PdVpxfvmyVjzv6qNv1tFMQljPYocbhF/KUky0C4mpNup HMgVOpIx79OsWdmel+ctIodRvkcMSPFpOxF+QM8QYCqtz6ulqYb8ZUfQ85kJvuaUhG HMwZEG1sWDMQQ== From: Mark Brown Date: Tue, 01 Oct 2024 12:29:08 +0100 Subject: [PATCH v7 3/4] KVM: arm64: Fix FFR offset calculation for pKVM host state save and restore Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20241001-kvm-arm64-fix-pkvm-sve-vl-v7-3-7b0171a36695@kernel.org> References: <20241001-kvm-arm64-fix-pkvm-sve-vl-v7-0-7b0171a36695@kernel.org> In-Reply-To: <20241001-kvm-arm64-fix-pkvm-sve-vl-v7-0-7b0171a36695@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Fuad Tabba Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=3206; i=broonie@kernel.org; h=from:subject:message-id; bh=KBunKuidQbGsl1sTPDqT1ibl0GGL5dnEQFUfFDn0MP8=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBm+91952Fz2ONJP3ZDAU2Oql7Admcy1BKuIxh9+12i 7GtyhbuJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZvvdfQAKCRAk1otyXVSH0PTkB/ 407zY0DEiqK+eF3/i3xkdy/woU0hcDfHxTybQd/T6pvgXBawysTNgYjQ2CogGCSBbw2pDvtNcedfI3 0IVNF17OOUZiVB7M0DGQjX4C85sN4i+JFZ/Hm0uVzONX/hAWX3wPafc6YTKZc2kzsmQGz4IE7PnNNt sVBUmV/EeSNHYzoqbJfdHp2+sb36lNGBL3fzSUOPs8Ubt16DA8/+rYRzbtQcM+RjSw/cJ0kY8k0FTA rax6ye6OXZGFwiJGCDaYpnZlUDwxpl9QKgYDr11tRok3EbtntYmW6c4HzPhv1i/k7JWARv+cB3Dod7 6JZh3CHiXQ9r4iQB/izB/VgNa4fdJm X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB When saving and restoring the SVE state for the host we configure the hardware for the maximum VL it supports, but when calculating offsets in memory we use the maximum usable VL for the host. Since these two values may not be the same this may result in data corruption in the case where the PE supports a VL larger than the maximum usable VL for the host. We can just read the current VL from the hardware with an instruction so use that instead of a saved value, we need to correct the value used to lay out the stored data and this makes it clear that the layout is consistent with the hardware configuration. Fixes: b5b9955617bc ("KVM: arm64: Eagerly restore host fpsimd/sve state in = pKVM") Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_hyp.h | 1 + arch/arm64/kvm/hyp/fpsimd.S | 5 +++++ arch/arm64/kvm/hyp/include/hyp/switch.h | 2 +- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 2 +- 4 files changed, 8 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_= hyp.h index c838309e4ec4..6b074f4d48b2 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -113,6 +113,7 @@ void __fpsimd_save_state(struct user_fpsimd_state *fp_r= egs); void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs); void __sve_save_state(void *sve_pffr, u32 *fpsr, int save_ffr); void __sve_restore_state(void *sve_pffr, u32 *fpsr, int restore_ffr); +int __sve_get_vl(void); =20 u64 __guest_enter(struct kvm_vcpu *vcpu); =20 diff --git a/arch/arm64/kvm/hyp/fpsimd.S b/arch/arm64/kvm/hyp/fpsimd.S index e950875e31ce..d272dbf36da8 100644 --- a/arch/arm64/kvm/hyp/fpsimd.S +++ b/arch/arm64/kvm/hyp/fpsimd.S @@ -31,3 +31,8 @@ SYM_FUNC_START(__sve_save_state) sve_save 0, x1, x2, 3 ret SYM_FUNC_END(__sve_save_state) + +SYM_FUNC_START(__sve_get_vl) + _sve_rdvl 0, 1 + ret +SYM_FUNC_END(__sve_get_vl) diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/i= nclude/hyp/switch.h index 46d52e8a3df3..58f60ea95a35 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -339,7 +339,7 @@ static inline void __hyp_sve_save_host(void) =20 sve_state->zcr_el1 =3D read_sysreg_el1(SYS_ZCR); write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL2); - __sve_save_state(sve_state->sve_regs + sve_ffr_offset(kvm_host_sve_max_vl= ), + __sve_save_state(sve_state->sve_regs + sve_ffr_offset(__sve_get_vl()), &sve_state->fpsr, true); } diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/h= yp-main.c index 87692b566d90..48adcd005079 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -49,7 +49,7 @@ static void __hyp_sve_restore_host(void) * supported by the system (or limited at EL3). */ write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL2); - __sve_restore_state(sve_state->sve_regs + sve_ffr_offset(kvm_host_sve_max= _vl), + __sve_restore_state(sve_state->sve_regs + sve_ffr_offset(__sve_get_vl()), &sve_state->fpsr, true); write_sysreg_el1(sve_state->zcr_el1, SYS_ZCR); --=20 2.39.2 From nobody Thu Nov 28 12:36:37 2024 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6029E1C2DA3; Tue, 1 Oct 2024 11:31:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727782284; cv=none; b=U69XjM+iu/kFHd/Rq86FVl0JIQ2B/9PLuC7d4WLvqGvTunGJwUJL9609NvuwlYqG5Plft7bruzL7gWAquOfTEcKk6HDlCIgRfFDI1p0x92zcj3glVpxtgIXy+ZDpffc0Kw+wpbeaUgFS4MM4iFFcPV7pahzkMrWuE01ykf+eNT4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1727782284; c=relaxed/simple; bh=3H3rN27/T4dSvW1Uedlxa4dx04M+fhPbfJn2xQeiUto=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=Gs8UwUZASqBmtOKCunf4TWzehwN6URiI6qG7dxUN0ytxQNaqPiBlpbMVPLcackwwCkWMPaXkCs/e6TRvcph9sW/KfZvK4A3Zqp39zZq4e3omi+y8N6L84v/y1MvxdLV2z7LZRSqleowj/4k2DIBMx5gpxABpoHMGYlhvenPPSjM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ev0zcnej; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ev0zcnej" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 065F8C4CECD; Tue, 1 Oct 2024 11:31:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1727782284; bh=3H3rN27/T4dSvW1Uedlxa4dx04M+fhPbfJn2xQeiUto=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=ev0zcnejLlg4untSFDKPs0uu9yRjLpeF29NXm6Fyn2yh3/+5F6eKQ+i/efPs3+p7m Ms97pZVloGyzj0h7PgGqWHkpDCdU2MDWFiWnNyBAlGYlbBbgkT8of9E3yp0dHCzQjc FG49kMw6VnsY+KD8+Bu2FZAUZQFnS/gitVh92IdizoZi2bkwMrboc7ogwwjzfbrFq/ itKBJbxfIL3lh2wVsGkt/RfR3TrVW5P6/hfsI6PJHU46H27qv9ZFa4NUTB/rCpWR1l qJewLv3219nczc442IMtmEliB4fnOxgEqrFnASQNOgAQaA24w9cHZVlGiF76UuT7fU RsaaVHjOy2elA== From: Mark Brown Date: Tue, 01 Oct 2024 12:29:09 +0100 Subject: [PATCH v7 4/4] KVM: arm64: Avoid underallocating storage for host SVE state Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20241001-kvm-arm64-fix-pkvm-sve-vl-v7-4-7b0171a36695@kernel.org> References: <20241001-kvm-arm64-fix-pkvm-sve-vl-v7-0-7b0171a36695@kernel.org> In-Reply-To: <20241001-kvm-arm64-fix-pkvm-sve-vl-v7-0-7b0171a36695@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Fuad Tabba Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, Mark Brown X-Mailer: b4 0.15-dev-99b12 X-Developer-Signature: v=1; a=openpgp-sha256; l=4775; i=broonie@kernel.org; h=from:subject:message-id; bh=3H3rN27/T4dSvW1Uedlxa4dx04M+fhPbfJn2xQeiUto=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBm+91+bqwK4f/pJu2CGuxMaixni9UrdS9SNp0K0H+g BdW8tDmJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZvvdfgAKCRAk1otyXVSH0O8MB/ 9PffATY1K7tJ1hlmtmShlhAHdohSC3qAnH0uEWLq5dujSFDSuS8JMjvRLtBWwAApYUjohwazjqUMGr S6MQmMdtfoV58SR9WNsybgfttZz3/uUpVu8hWjJLRHIKm7drRnIjuhK2oerBmf4UaS6EmeiZ816hvb qV4L4r53Jp/fm1zDfziPzD8Vu5vCb5UwJ7HOmd7vAhyehJ4jkNnLG/kF63IHnl/hDaw2po5EjH/fCp oNaK0u1sYxpx592tHvJj/Orl/hKz2D7Wtuu6M199308Qhe+CKvn4rLBNdxudXs0uEG5R1ot344mBT2 y5b51UB7U5839ImBX7zF9B/CbsLJlo X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB We size the allocation for the host SVE state using the maximum VL shared by all CPUs in the host. As observed during review on an asymmetric system this may be less than the maximum VL supported on some of the CPUs. Since the pKVM hypervisor saves and restores the host state using the maximum VL for the current CPU this may lead to buffer overflows, fix this by changing pKVM to use the maximum VL for any CPU to size allocations and limit host configurations. Fixes: 66d5b53e20a6 ("KVM: arm64: Allocate memory mapped at hyp for host sv= e state in pKVM") Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 2 +- arch/arm64/include/asm/kvm_hyp.h | 2 +- arch/arm64/include/asm/kvm_pkvm.h | 2 +- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 4 ++-- arch/arm64/kvm/hyp/nvhe/pkvm.c | 2 +- arch/arm64/kvm/reset.c | 6 +++--- 6 files changed, 9 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm= _host.h index 329619c6fa96..612a5adc2dbf 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -76,7 +76,7 @@ static inline enum kvm_mode kvm_get_mode(void) { return K= VM_MODE_NONE; }; DECLARE_STATIC_KEY_FALSE(userspace_irqchip_in_use); =20 extern unsigned int __ro_after_init kvm_sve_max_vl; -extern unsigned int __ro_after_init kvm_host_sve_max_vl; +extern unsigned int __ro_after_init kvm_host_sve_max_cpu_vl; int __init kvm_arm_init_sve(void); =20 u32 __attribute_const__ kvm_target_cpu(void); diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_= hyp.h index 6b074f4d48b2..19f3ae9f05a9 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -144,6 +144,6 @@ extern u64 kvm_nvhe_sym(id_aa64smfr0_el1_sys_val); =20 extern unsigned long kvm_nvhe_sym(__icache_flags); extern unsigned int kvm_nvhe_sym(kvm_arm_vmid_bits); -extern unsigned int kvm_nvhe_sym(kvm_host_sve_max_vl); +extern unsigned int kvm_nvhe_sym(kvm_host_sve_max_cpu_vl); =20 #endif /* __ARM64_KVM_HYP_H__ */ diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm= _pkvm.h index cd56acd9a842..6fc0cf42fca3 100644 --- a/arch/arm64/include/asm/kvm_pkvm.h +++ b/arch/arm64/include/asm/kvm_pkvm.h @@ -134,7 +134,7 @@ static inline size_t pkvm_host_sve_state_size(void) return 0; =20 return size_add(sizeof(struct cpu_sve_state), - SVE_SIG_REGS_SIZE(sve_vq_from_vl(kvm_host_sve_max_vl))); + SVE_SIG_REGS_SIZE(sve_vq_from_vl(kvm_host_sve_max_cpu_vl))); } =20 #endif /* __ARM64_KVM_PKVM_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/h= yp-main.c index 48adcd005079..234372bef85c 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -99,8 +99,8 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu) hyp_vcpu->vcpu.arch.ctxt =3D host_vcpu->arch.ctxt; =20 hyp_vcpu->vcpu.arch.sve_state =3D kern_hyp_va(host_vcpu->arch.sve_state); - /* Limit guest vector length to the maximum supported by the host. */ - hyp_vcpu->vcpu.arch.sve_max_vl =3D min(host_vcpu->arch.sve_max_vl, kvm_ho= st_sve_max_vl); + /* Limit guest vector length to the maximum supported by any CPU. */ + hyp_vcpu->vcpu.arch.sve_max_vl =3D min(host_vcpu->arch.sve_max_vl, kvm_ho= st_sve_max_cpu_vl); =20 hyp_vcpu->vcpu.arch.hw_mmu =3D host_vcpu->arch.hw_mmu; =20 diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c index 187a5f4d56c0..770d66491b76 100644 --- a/arch/arm64/kvm/hyp/nvhe/pkvm.c +++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c @@ -18,7 +18,7 @@ unsigned long __icache_flags; /* Used by kvm_get_vttbr(). */ unsigned int kvm_arm_vmid_bits; =20 -unsigned int kvm_host_sve_max_vl; +unsigned int kvm_host_sve_max_cpu_vl; =20 /* * Set trap register values based on features in ID_AA64PFR0. diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c index 0b0ae5ae7bc2..6c87d01514ff 100644 --- a/arch/arm64/kvm/reset.c +++ b/arch/arm64/kvm/reset.c @@ -32,7 +32,7 @@ =20 /* Maximum phys_shift supported for any VM on this host */ static u32 __ro_after_init kvm_ipa_limit; -unsigned int __ro_after_init kvm_host_sve_max_vl; +unsigned int __ro_after_init kvm_host_sve_max_cpu_vl; =20 /* * ARMv8 Reset Values @@ -52,8 +52,8 @@ int __init kvm_arm_init_sve(void) { if (system_supports_sve()) { kvm_sve_max_vl =3D sve_max_virtualisable_vl(); - kvm_host_sve_max_vl =3D sve_max_vl(); - kvm_nvhe_sym(kvm_host_sve_max_vl) =3D kvm_host_sve_max_vl; + kvm_host_sve_max_cpu_vl =3D sve_max_cpu_vl(); + kvm_nvhe_sym(kvm_host_sve_max_cpu_vl) =3D kvm_host_sve_max_cpu_vl; =20 /* * The get_sve_reg()/set_sve_reg() ioctl interface will need --=20 2.39.2