From nobody Thu Jun 25 07:05:25 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id E6179C433EF for ; Tue, 22 Feb 2022 16:53:54 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234382AbiBVQyT (ORCPT ); Tue, 22 Feb 2022 11:54:19 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33200 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232223AbiBVQyR (ORCPT ); Tue, 22 Feb 2022 11:54:17 -0500 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D017C38DB0 for ; Tue, 22 Feb 2022 08:53:51 -0800 (PST) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-2d07ae11462so146914547b3.8 for ; Tue, 22 Feb 2022 08:53:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:cc; bh=HjTQZhkAd40lRuvS6cRV70Tpnwq9dG8Hrk7s02bufrw=; b=l9gvPlFOgpm93Vwqfc4x5Vuvn4H98TzVtnPztFKq1RVJHuCOwjRWnhPGwbxmv2gBju qni22+lZAoWeuHyAnrHLkUxmGVg1MGvOPtvysJUTpVnxGE9XNCgdGG3ihOcgDh4Ud0VO t81B3SWw8tC8AP32YUnfVl7SNtvpxpe1ER/dZExOrnQ3bwmGSIvoILXJpsKsbYOeEn5X eNRw3J9xbd+PwHflrNv9W7603a/Ig+5VqL31ZqaFq/h7ygZQBquGNSxpt5WNmcJEKeuh 799vMF4hFNUjwmMyaOlRkDXrCuhduGcEePOXQj+fAOq4j8VbFHa6wTIBzVfUzsaX0ujb tebQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:cc; bh=HjTQZhkAd40lRuvS6cRV70Tpnwq9dG8Hrk7s02bufrw=; b=sDmp9dFDLdaEbcbHl9InkAEhhOi6yRb7QOGP9rnm8jFnRPUmzbGZ4EqLHtt2YA5B3H AaOAXxbklt0uq+dPQz1/H4odQlwuhssgIfheLLxRSKRWOBLsziuTPVJcIMLA4Eg/L3dF OZ0jiHPXxD4VGqtblYIm7IrdMTnVN3H0zP66iOGlNrF3Amyzcl8sAOvahjFpziRTBW/2 v0xcreuLo4e+ru5lrWUPPacsri8MMsyf1zrA8+X6nWiLOU9W+fvc4dVaFzXkybz6bXty kXSEViPhQWob010nq3Oy0iROvsiroAZUi1sievC6TeLrVve261xJ35/FXq6HB65h0wvg 6vLQ== X-Gm-Message-State: AOAM530aM1Dr3E0H6m6oJoECfbwRFna3UenUuSk8sc0EtKpziyWA/iJ2 JE3hUm0DHM+DZbYm5VWVZ8/ZFmjXrGokwVXGcw== X-Google-Smtp-Source: ABdhPJxqQdinLFWWaI+jsvW0KvgGVTkFzIldtere0/r0cpO5BgzuX/8Yyc3hdk7P6zZ0tb1mTeOyerwEEGUUJI16nw== X-Received: from kaleshsingh.mtv.corp.google.com ([2620:15c:211:200:5db7:1235:b3dd:cfcb]) (user=kaleshsingh job=sendgmr) by 2002:a25:4dd7:0:b0:621:a740:3988 with SMTP id a206-20020a254dd7000000b00621a7403988mr24600128ybb.58.1645548831008; Tue, 22 Feb 2022 08:53:51 -0800 (PST) Date: Tue, 22 Feb 2022 08:51:02 -0800 In-Reply-To: <20220222165212.2005066-1-kaleshsingh@google.com> Message-Id: <20220222165212.2005066-2-kaleshsingh@google.com> Mime-Version: 1.0 References: <20220222165212.2005066-1-kaleshsingh@google.com> X-Mailer: git-send-email 2.35.1.473.g83b2b277ed-goog Subject: [PATCH v2 1/9] KVM: arm64: Introduce hyp_alloc_private_va_range() From: Kalesh Singh Cc: will@kernel.org, maz@kernel.org, qperret@google.com, tabba@google.com, surenb@google.com, kernel-team@android.com, Kalesh Singh , Catalin Marinas , James Morse , Alexandru Elisei , Suzuki K Poulose , Ard Biesheuvel , Mark Rutland , Pasha Tatashin , Joey Gouly , Peter Collingbourne , Andrew Walbran , Andrew Scull , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" hyp_alloc_private_va_range() can be used to reserve private VA ranges in the nVHE hypervisor. Also update __create_hyp_private_mapping() to allow specifying an alignment for the private VA mapping. These will be used to implement stack guard pages for KVM nVHE hypervisor (nVHE Hyp mode / not pKVM), in a subsequent patch in the series. Signed-off-by: Kalesh Singh --- arch/arm64/include/asm/kvm_mmu.h | 4 +++ arch/arm64/kvm/mmu.c | 61 +++++++++++++++++++++----------- 2 files changed, 44 insertions(+), 21 deletions(-) diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_= mmu.h index 81839e9a8a24..0b0c71302b92 100644 --- a/arch/arm64/include/asm/kvm_mmu.h +++ b/arch/arm64/include/asm/kvm_mmu.h @@ -153,6 +153,10 @@ static __always_inline unsigned long __kern_hyp_va(uns= igned long v) int kvm_share_hyp(void *from, void *to); void kvm_unshare_hyp(void *from, void *to); int create_hyp_mappings(void *from, void *to, enum kvm_pgtable_prot prot); +unsigned long hyp_alloc_private_va_range(size_t size, size_t align); +int __create_hyp_private_mapping(phys_addr_t phys_addr, size_t size, + size_t align, unsigned long *haddr, + enum kvm_pgtable_prot prot); int create_hyp_io_mappings(phys_addr_t phys_addr, size_t size, void __iomem **kaddr, void __iomem **haddr); diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index bc2aba953299..e5abcce44ad0 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -457,22 +457,16 @@ int create_hyp_mappings(void *from, void *to, enum kv= m_pgtable_prot prot) return 0; } =20 -static int __create_hyp_private_mapping(phys_addr_t phys_addr, size_t size, - unsigned long *haddr, - enum kvm_pgtable_prot prot) + +/* + * Allocates a private VA range below io_map_base. + * + * @size: The size of the VA range to reserve. + * @align: The required alignment for the allocation. + */ +unsigned long hyp_alloc_private_va_range(size_t size, size_t align) { unsigned long base; - int ret =3D 0; - - if (!kvm_host_owns_hyp_mappings()) { - base =3D kvm_call_hyp_nvhe(__pkvm_create_private_mapping, - phys_addr, size, prot); - if (IS_ERR_OR_NULL((void *)base)) - return PTR_ERR((void *)base); - *haddr =3D base; - - return 0; - } =20 mutex_lock(&kvm_hyp_pgd_mutex); =20 @@ -484,8 +478,8 @@ static int __create_hyp_private_mapping(phys_addr_t phy= s_addr, size_t size, * * The allocated size is always a multiple of PAGE_SIZE. */ - size =3D PAGE_ALIGN(size + offset_in_page(phys_addr)); - base =3D io_map_base - size; + base =3D io_map_base - PAGE_ALIGN(size); + base =3D ALIGN_DOWN(base, align); =20 /* * Verify that BIT(VA_BITS - 1) hasn't been flipped by @@ -493,20 +487,45 @@ static int __create_hyp_private_mapping(phys_addr_t p= hys_addr, size_t size, * overflowed the idmap/IO address range. */ if ((base ^ io_map_base) & BIT(VA_BITS - 1)) - ret =3D -ENOMEM; + base =3D (unsigned long)ERR_PTR(-ENOMEM); else io_map_base =3D base; =20 mutex_unlock(&kvm_hyp_pgd_mutex); =20 + return base; +} + +int __create_hyp_private_mapping(phys_addr_t phys_addr, size_t size, + size_t align, unsigned long *haddr, + enum kvm_pgtable_prot prot) +{ + unsigned long addr; + int ret =3D 0; + + if (!kvm_host_owns_hyp_mappings()) { + addr =3D kvm_call_hyp_nvhe(__pkvm_create_private_mapping, + phys_addr, size, prot); + if (IS_ERR_OR_NULL((void *)addr)) + return PTR_ERR((void *)addr); + *haddr =3D addr; + + return 0; + } + + size +=3D offset_in_page(phys_addr); + addr =3D hyp_alloc_private_va_range(size, align); + if (IS_ERR_OR_NULL((void *)addr)) + return PTR_ERR((void *)addr); + if (ret) goto out; =20 - ret =3D __create_hyp_mappings(base, size, phys_addr, prot); + ret =3D __create_hyp_mappings(addr, size, phys_addr, prot); if (ret) goto out; =20 - *haddr =3D base + offset_in_page(phys_addr); + *haddr =3D addr + offset_in_page(phys_addr); out: return ret; } @@ -537,7 +556,7 @@ int create_hyp_io_mappings(phys_addr_t phys_addr, size_= t size, return 0; } =20 - ret =3D __create_hyp_private_mapping(phys_addr, size, + ret =3D __create_hyp_private_mapping(phys_addr, size, PAGE_SIZE, &addr, PAGE_HYP_DEVICE); if (ret) { iounmap(*kaddr); @@ -564,7 +583,7 @@ int create_hyp_exec_mappings(phys_addr_t phys_addr, siz= e_t size, =20 BUG_ON(is_kernel_in_hyp_mode()); =20 - ret =3D __create_hyp_private_mapping(phys_addr, size, + ret =3D __create_hyp_private_mapping(phys_addr, size, PAGE_SIZE, &addr, PAGE_HYP_EXEC); if (ret) { *haddr =3D NULL; --=20 2.35.1.473.g83b2b277ed-goog From nobody Thu Jun 25 07:05:25 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0CB78C433EF for ; Tue, 22 Feb 2022 16:55:01 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234395AbiBVQzZ (ORCPT ); Tue, 22 Feb 2022 11:55:25 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38548 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233461AbiBVQzX (ORCPT ); Tue, 22 Feb 2022 11:55:23 -0500 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9037D16BCF5 for ; Tue, 22 Feb 2022 08:54:57 -0800 (PST) Received: by mail-yb1-xb4a.google.com with SMTP id k10-20020a056902070a00b0062469b00335so8994297ybt.14 for ; Tue, 22 Feb 2022 08:54:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:cc; bh=ZlyAAqqDhAE4WxbyZ31+jkqH/YpnLCKmJnh7POCxqTo=; b=TIooTTyH5x1F5nebrNIhkxrh2HuWrtPHN3fTIyBzs1a6KH6OGiKUp1PWOlO6NXsbQT k7h9zWn15cvo5sZhfXstOZ7RpAD+Gat9qLxoStKOCv0K160fImoXF6iu00kBuJnVyT6n YeThCRS0IPdnxIFck9JWsb0bj7sYMYA9LYn4MWVPyD2giNP+0cLZ8uSvz0IwXqXOXaOu G/dlwHj4gGGF4Y6nSgBvhRU/4Yq7YkrEaClamVuzK+tUFIXdOsSYoS/NWO0g8LCGrMsX It+EBn3ntQqg8tQNu7EC0xgAKjXEbbsSJs8P+w445Q4D6gLEB+GMhwcdWzAmnJsKbA4J Pgrw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:cc; bh=ZlyAAqqDhAE4WxbyZ31+jkqH/YpnLCKmJnh7POCxqTo=; b=FEq/NQkzuc0UgMysaEDwxjybb2bBdaIwEmZLnR1lNaAbRIZjcsarYXt7tbxbrA8Z0t uzMHmo1Jj/EfCHhUYwC8GPZhctgk6ojIBiJufpNN+ROJgyumkipBA2ERAm6jWg0Fk5j/ Jzw2AHwh/4fGmzt7A+ynqS9RhYCCDfEFEZVuo/A5zGfAmOW4a2SPlUsJRZLrhJ03UxtZ r2mH3vWzyCyU6P03J9OSGSEJTraUZCei2zFwGcwZ9GrotiQ1cKdxzLHKi5PZ3Zs/88IS 8i8VJN8OjUedrg01f7heV+oCuxyzk7JKHgjbccDNhouVoBMDl8oLjedy5QT3sIw9q7Rr jtWw== X-Gm-Message-State: AOAM531yM8UeMgEK79bprCwCy+FTXydRI+ruik0gMEBCm/q5lAEI4cJU Gcj4jdNARtqlGzSgGOrOINttCQAAh9apF+To5g== X-Google-Smtp-Source: ABdhPJwTHC6zcsc1CP1g9QcEYL9chbpjl/nOUt74qFxAcpxZyUBNLrnrRhwJaWgirDTS7AoSkxlDAybxGjmkVJWUJw== X-Received: from kaleshsingh.mtv.corp.google.com ([2620:15c:211:200:5db7:1235:b3dd:cfcb]) (user=kaleshsingh job=sendgmr) by 2002:a81:911:0:b0:2d2:c5c6:b4a1 with SMTP id 17-20020a810911000000b002d2c5c6b4a1mr24349571ywj.48.1645548896812; Tue, 22 Feb 2022 08:54:56 -0800 (PST) Date: Tue, 22 Feb 2022 08:51:03 -0800 In-Reply-To: <20220222165212.2005066-1-kaleshsingh@google.com> Message-Id: <20220222165212.2005066-3-kaleshsingh@google.com> Mime-Version: 1.0 References: <20220222165212.2005066-1-kaleshsingh@google.com> X-Mailer: git-send-email 2.35.1.473.g83b2b277ed-goog Subject: [PATCH v2 2/9] KVM: arm64: Introduce pkvm_alloc_private_va_range() From: Kalesh Singh Cc: will@kernel.org, maz@kernel.org, qperret@google.com, tabba@google.com, surenb@google.com, kernel-team@android.com, Kalesh Singh , Catalin Marinas , James Morse , Alexandru Elisei , Suzuki K Poulose , Ard Biesheuvel , Mark Rutland , Pasha Tatashin , Joey Gouly , Peter Collingbourne , Andrew Scull , Zenghui Yu , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" pkvm_hyp_alloc_private_va_range() can be used to reserve private VA ranges in the pKVM nVHE hypervisor (). Also update __pkvm_create_private_mapping() to allow specifying an alignment for the private VA mapping. These will be used to implement stack guard pages for pKVM nVHE hypervisor (in a subsequent patch in the series). Credits to Quentin Perret for the idea of moving private VA allocation out of __pkvm_create_private_mapping() Signed-off-by: Kalesh Singh --- Changes in v2: - Allow specifying an alignment for the private VA allocations, per Marc arch/arm64/kvm/hyp/include/nvhe/mm.h | 3 +- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 5 +-- arch/arm64/kvm/hyp/nvhe/mm.c | 49 +++++++++++++++++++--------- arch/arm64/kvm/mmu.c | 2 +- 4 files changed, 39 insertions(+), 20 deletions(-) diff --git a/arch/arm64/kvm/hyp/include/nvhe/mm.h b/arch/arm64/kvm/hyp/incl= ude/nvhe/mm.h index 2d08510c6cc1..05d06ad00347 100644 --- a/arch/arm64/kvm/hyp/include/nvhe/mm.h +++ b/arch/arm64/kvm/hyp/include/nvhe/mm.h @@ -20,7 +20,8 @@ int pkvm_cpu_set_vector(enum arm64_hyp_spectre_vector slo= t); int pkvm_create_mappings(void *from, void *to, enum kvm_pgtable_prot prot); int pkvm_create_mappings_locked(void *from, void *to, enum kvm_pgtable_pro= t prot); unsigned long __pkvm_create_private_mapping(phys_addr_t phys, size_t size, - enum kvm_pgtable_prot prot); + size_t align, enum kvm_pgtable_prot prot); +unsigned long pkvm_alloc_private_va_range(size_t size, size_t align); =20 static inline void hyp_vmemmap_range(phys_addr_t phys, unsigned long size, unsigned long *start, unsigned long *end) diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/h= yp-main.c index 5e2197db0d32..96b2312a0f1d 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -158,9 +158,10 @@ static void handle___pkvm_create_private_mapping(struc= t kvm_cpu_context *host_ct { DECLARE_REG(phys_addr_t, phys, host_ctxt, 1); DECLARE_REG(size_t, size, host_ctxt, 2); - DECLARE_REG(enum kvm_pgtable_prot, prot, host_ctxt, 3); + DECLARE_REG(size_t, align, host_ctxt, 3); + DECLARE_REG(enum kvm_pgtable_prot, prot, host_ctxt, 4); =20 - cpu_reg(host_ctxt, 1) =3D __pkvm_create_private_mapping(phys, size, prot); + cpu_reg(host_ctxt, 1) =3D __pkvm_create_private_mapping(phys, size, align= , prot); } =20 static void handle___pkvm_prot_finalize(struct kvm_cpu_context *host_ctxt) diff --git a/arch/arm64/kvm/hyp/nvhe/mm.c b/arch/arm64/kvm/hyp/nvhe/mm.c index 526a7d6fa86f..298fbbe4651d 100644 --- a/arch/arm64/kvm/hyp/nvhe/mm.c +++ b/arch/arm64/kvm/hyp/nvhe/mm.c @@ -37,26 +37,46 @@ static int __pkvm_create_mappings(unsigned long start, = unsigned long size, return err; } =20 -unsigned long __pkvm_create_private_mapping(phys_addr_t phys, size_t size, - enum kvm_pgtable_prot prot) +/* + * Allocates a private VA range above __io_map_base. + * + * @size: The size of the VA range to reserve. + * @align: The required alignment for the allocation. + */ +unsigned long pkvm_alloc_private_va_range(size_t size, size_t align) { - unsigned long addr; - int err; + unsigned long base, addr; =20 hyp_spin_lock(&pkvm_pgd_lock); =20 - size =3D PAGE_ALIGN(size + offset_in_page(phys)); - addr =3D __io_map_base; - __io_map_base +=3D size; + addr =3D ALIGN(__io_map_base, align); + + /* The allocated size is always a multiple of PAGE_SIZE */ + base =3D addr + PAGE_ALIGN(size); =20 /* Are we overflowing on the vmemmap ? */ - if (__io_map_base > __hyp_vmemmap) { - __io_map_base -=3D size; + if (base > __hyp_vmemmap) addr =3D (unsigned long)ERR_PTR(-ENOMEM); + else + __io_map_base =3D base; + + hyp_spin_unlock(&pkvm_pgd_lock); + + return addr; +} + +unsigned long __pkvm_create_private_mapping(phys_addr_t phys, size_t size, + size_t align, enum kvm_pgtable_prot prot) +{ + unsigned long addr; + int err; + + size +=3D offset_in_page(phys); + addr =3D pkvm_alloc_private_va_range(size, align); + if (IS_ERR((void *)addr)) goto out; - } =20 - err =3D kvm_pgtable_hyp_map(&pkvm_pgtable, addr, size, phys, prot); + err =3D __pkvm_create_mappings(addr, size, phys, prot); if (err) { addr =3D (unsigned long)ERR_PTR(err); goto out; @@ -64,8 +84,6 @@ unsigned long __pkvm_create_private_mapping(phys_addr_t p= hys, size_t size, =20 addr =3D addr + offset_in_page(phys); out: - hyp_spin_unlock(&pkvm_pgd_lock); - return addr; } =20 @@ -152,9 +170,8 @@ int hyp_map_vectors(void) return 0; =20 phys =3D __hyp_pa(__bp_harden_hyp_vecs); - bp_base =3D (void *)__pkvm_create_private_mapping(phys, - __BP_HARDEN_HYP_VECS_SZ, - PAGE_HYP_EXEC); + bp_base =3D (void *)__pkvm_create_private_mapping(phys, __BP_HARDEN_HYP_V= ECS_SZ, + PAGE_SIZE, PAGE_HYP_EXEC); if (IS_ERR_OR_NULL(bp_base)) return PTR_ERR(bp_base); =20 diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index e5abcce44ad0..18a711d6a52f 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -505,7 +505,7 @@ int __create_hyp_private_mapping(phys_addr_t phys_addr,= size_t size, =20 if (!kvm_host_owns_hyp_mappings()) { addr =3D kvm_call_hyp_nvhe(__pkvm_create_private_mapping, - phys_addr, size, prot); + phys_addr, size, align, prot); if (IS_ERR_OR_NULL((void *)addr)) return PTR_ERR((void *)addr); *haddr =3D addr; --=20 2.35.1.473.g83b2b277ed-goog From nobody Thu Jun 25 07:05:25 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1E059C433F5 for ; Tue, 22 Feb 2022 16:56:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234402AbiBVQ4c (ORCPT ); Tue, 22 Feb 2022 11:56:32 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39500 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230519AbiBVQ4a (ORCPT ); Tue, 22 Feb 2022 11:56:30 -0500 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 604A116BCF5 for ; Tue, 22 Feb 2022 08:56:05 -0800 (PST) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-2d61b4ef6cdso145981507b3.11 for ; Tue, 22 Feb 2022 08:56:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:cc; bh=R7NUgyhmz2Vzm1IbfrrDYD1qNpQGYjcjDbFVXUaMLH8=; b=KI3O3p6WIXcE6n5c1xlRqNCzstNa7C+hluEFjVsr0MOzveDFLD2VUOGSWGeKG0RHJy T/yqV6+ri+p/3rX+b1pYMjSNo3Ovj9yyZbKA1607HqCA3kbya8vyKQ2GDgWlYG9l1znh rWdySSjzOH0K6sfRsXIWIRMQFAHMD5jl44ausSef3FnfQgFOmyhWcQwfQoMj6etEc/LI hJHRQ66bszx5zigZGXaE9ntUDiXKmNtjFdm5S5h0xTnO2PAF2yNKcyiTDlX9Dq5Ziu1q t3RdfNDzIKa7Bq1TCGBRbeYsC4mW70m5JWLL7NBzuv6XS1rF0YDiOJcHIe7I88ZhGfi5 7YDA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:cc; bh=R7NUgyhmz2Vzm1IbfrrDYD1qNpQGYjcjDbFVXUaMLH8=; b=5pCIMjBHoSmDnMdhJCa9qPw4/phSfhou2ejPaTQbYvBvjD8oexTgTklZ5EsHOCrrLE jboPtOVHi8hHCkUJsmE7kM6oU8imr00nHtQGKigOnolY/eXVH/e5GXUtHC4FM+b5ZFrP cv/EhWU7teJrZJW++CKnNBPlbbYORnPejUbR9epRnPZ8onCFuCw9AMkfjLX01eVYosm7 /r1oyglsnK57OiGYfhrGvqtD6FugIzN0pLKWHvD4FcewLhLcbn19gkYPuCVAOUQHlA5F 5zYXe4G6AxXPEMsL2bEit+Mje57RUmI0s7uKw/BXyTEDEAoJ/pY/6NQ3sRgxXUeWoZjm UYOA== X-Gm-Message-State: AOAM53186SQ2x/v/TWXkEmnGuJre0+hiBk1jutePYuPU8FFfSYCo1PPW LmjBESxkzZgUva0ICIh9O+ikF4NOIHWvjCqAlg== X-Google-Smtp-Source: ABdhPJzj4uTOWudxzzJokeTZXJ0Xc+OcxmAl9Nm7IKWdiyC4Dj/qqu6z+Dkz3NZLlc5P6ZEzL8Pz6NCeaRU6Ltd2cA== X-Received: from kaleshsingh.mtv.corp.google.com ([2620:15c:211:200:5db7:1235:b3dd:cfcb]) (user=kaleshsingh job=sendgmr) by 2002:a05:6902:ca:b0:5ff:5f2d:b533 with SMTP id i10-20020a05690200ca00b005ff5f2db533mr23967292ybs.606.1645548964571; Tue, 22 Feb 2022 08:56:04 -0800 (PST) Date: Tue, 22 Feb 2022 08:51:04 -0800 In-Reply-To: <20220222165212.2005066-1-kaleshsingh@google.com> Message-Id: <20220222165212.2005066-4-kaleshsingh@google.com> Mime-Version: 1.0 References: <20220222165212.2005066-1-kaleshsingh@google.com> X-Mailer: git-send-email 2.35.1.473.g83b2b277ed-goog Subject: [PATCH v2 3/9] KVM: arm64: Add guard pages for KVM nVHE hypervisor stack From: Kalesh Singh Cc: will@kernel.org, maz@kernel.org, qperret@google.com, tabba@google.com, surenb@google.com, kernel-team@android.com, Kalesh Singh , Catalin Marinas , James Morse , Alexandru Elisei , Suzuki K Poulose , Ard Biesheuvel , Mark Rutland , Pasha Tatashin , Joey Gouly , Peter Collingbourne , Andrew Scull , Paolo Bonzini , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Maps the stack pages in the flexible private VA range and allocates guard pages below the stack as unbacked VA space. The stack is aligned to twice its size to aid overflow detection (implemented in a subsequent patch in the series). Signed-off-by: Kalesh Singh --- arch/arm64/include/asm/kvm_asm.h | 1 + arch/arm64/kvm/arm.c | 32 +++++++++++++++++++++++++++++--- 2 files changed, 30 insertions(+), 3 deletions(-) diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_= asm.h index d5b0386ef765..2e277f2ed671 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -169,6 +169,7 @@ struct kvm_nvhe_init_params { unsigned long tcr_el2; unsigned long tpidr_el2; unsigned long stack_hyp_va; + unsigned long stack_pa; phys_addr_t pgd_pa; unsigned long hcr_el2; unsigned long vttbr; diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index ecc5958e27fe..7e2e680c3ffb 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -1541,7 +1541,6 @@ static void cpu_prepare_hyp_mode(int cpu) tcr |=3D (idmap_t0sz & GENMASK(TCR_TxSZ_WIDTH - 1, 0)) << TCR_T0SZ_OFFSET; params->tcr_el2 =3D tcr; =20 - params->stack_hyp_va =3D kern_hyp_va(per_cpu(kvm_arm_hyp_stack_page, cpu)= + PAGE_SIZE); params->pgd_pa =3D kvm_mmu_get_httbr(); if (is_protected_kvm_enabled()) params->hcr_el2 =3D HCR_HOST_NVHE_PROTECTED_FLAGS; @@ -1990,14 +1989,41 @@ static int init_hyp_mode(void) * Map the Hyp stack pages */ for_each_possible_cpu(cpu) { + struct kvm_nvhe_init_params *params =3D per_cpu_ptr_nvhe_sym(kvm_init_pa= rams, cpu); char *stack_page =3D (char *)per_cpu(kvm_arm_hyp_stack_page, cpu); - err =3D create_hyp_mappings(stack_page, stack_page + PAGE_SIZE, - PAGE_HYP); + unsigned long stack_hyp_va, guard_hyp_va; =20 + /* + * Private mappings are allocated downwards from io_map_base + * so allocate the stack first then the guard page. + * + * The stack is aligned to twice its size to facilitate overflow + * detection. + */ + err =3D __create_hyp_private_mapping(__pa(stack_page), PAGE_SIZE, + PAGE_SIZE * 2, &stack_hyp_va, PAGE_HYP); if (err) { kvm_err("Cannot map hyp stack\n"); goto out_err; } + + /* Allocate unbacked private VA range for stack guard page */ + guard_hyp_va =3D hyp_alloc_private_va_range(PAGE_SIZE, PAGE_SIZE); + if (IS_ERR((void *)guard_hyp_va)) { + err =3D PTR_ERR((void *)guard_hyp_va); + kvm_err("Cannot allocate hyp stack guard page\n"); + goto out_err; + } + + /* + * Save the stack PA in nvhe_init_params. This will be needed to recreate + * the stack mapping in protected nVHE mode. __hyp_pa() won't do the rig= ht + * thing there, since the stack has been mapped in the flexible private + * VA space. + */ + params->stack_pa =3D __pa(stack_page) + PAGE_SIZE; + + params->stack_hyp_va =3D stack_hyp_va + PAGE_SIZE; } =20 for_each_possible_cpu(cpu) { --=20 2.35.1.473.g83b2b277ed-goog From nobody Thu Jun 25 07:05:25 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1612BC433F5 for ; Tue, 22 Feb 2022 16:57:16 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234422AbiBVQ5k (ORCPT ); Tue, 22 Feb 2022 11:57:40 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:43388 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233144AbiBVQ5i (ORCPT ); Tue, 22 Feb 2022 11:57:38 -0500 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DC00716BFAD for ; Tue, 22 Feb 2022 08:57:11 -0800 (PST) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-2d07ae11460so145910127b3.7 for ; Tue, 22 Feb 2022 08:57:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:cc; bh=knnzmuohJ2ih2y2NTiUEzUxj92qYLV4FmoYUekHSvUw=; b=tazhhxtdUk3rQttzgDm+V0n+dY4v2ZnB4i8xG8ax2HZdSoHAffJo42UTzU3IHvzAqj CzDDQtHudJDmOpjvt+NTOxsHiMMcd5AR1KeV4KZ04Anj9Hvg2wo5WdkKx0uhJOAuX1UG DMG5h+1UR544FLvyjxNA1/VvkZavi0pxiwKfoHP3qdoBAJ8849pS8kTNoGD5QX5f9/Px 48yknJpXBRw8xvBSwtZigYe1MGyv8eQBBWwjuMT8BSujAmFufVqRsnYE2LCmZqywZs8i wJddWj0UZSrpFMMQpEijc1j/Xch9CqJ1J2YUynBS8lvvuY7OGu4etP9u4qE8wqmUHhIw yLwg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:cc; bh=knnzmuohJ2ih2y2NTiUEzUxj92qYLV4FmoYUekHSvUw=; b=Xe91ktN0nz4w2p6o9pV1v0wApCbo6UaLaeuPYe45EQbzSk64vQZvT72LAB9FO9RrmX xN5Wg8mZ4aBISVHiinRGGt5aYR5PpjCCM40Zl/0+mwECR581bifkJyipKWhc2dZ4RNL6 kvpoz2aXJbfrXoguQ4ynICHKbbZ3xNv5kdigDG5vE82x1Qj7MawXuufXgpHG2X+jk+17 j+Wpnt9zwM9+0lpFs5dhtiT3W2h9ZjwbVSXUZHTnncr0RCGmQjLoeoY8ljVa6lgbciA8 tdSSdwDokx577mWJZ+sFc4ipEXCP56AVkN3QJZGPtE5l0QRMZP1sMIYa0TqfFfPpjHQ6 c9Tw== X-Gm-Message-State: AOAM533n76ItwTRSOBBPIpeh+7FaW4OiC4J7NMks7qREoj/z04nSlB16 OZMv4ni8o3w57ntUAI41J0IBjqRWNd3WoztbZg== X-Google-Smtp-Source: ABdhPJxsM9h50TZI3B0JLyBOhiXv0umcrv6dsBugtZNBeZtlKN8O9HTo3L2ie+qpRSFxA92HD2CPBV8kKYkIX9PJAQ== X-Received: from kaleshsingh.mtv.corp.google.com ([2620:15c:211:200:5db7:1235:b3dd:cfcb]) (user=kaleshsingh job=sendgmr) by 2002:a25:4cc1:0:b0:623:ca02:c1e5 with SMTP id z184-20020a254cc1000000b00623ca02c1e5mr22952503yba.95.1645549030879; Tue, 22 Feb 2022 08:57:10 -0800 (PST) Date: Tue, 22 Feb 2022 08:51:05 -0800 In-Reply-To: <20220222165212.2005066-1-kaleshsingh@google.com> Message-Id: <20220222165212.2005066-5-kaleshsingh@google.com> Mime-Version: 1.0 References: <20220222165212.2005066-1-kaleshsingh@google.com> X-Mailer: git-send-email 2.35.1.473.g83b2b277ed-goog Subject: [PATCH v2 4/9] KVM: arm64: Add guard pages for pKVM (protected nVHE) hypervisor stack From: Kalesh Singh Cc: will@kernel.org, maz@kernel.org, qperret@google.com, tabba@google.com, surenb@google.com, kernel-team@android.com, Kalesh Singh , Catalin Marinas , James Morse , Alexandru Elisei , Suzuki K Poulose , Ard Biesheuvel , Mark Rutland , Pasha Tatashin , Joey Gouly , Peter Collingbourne , Andrew Scull , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Maps the stack pages in the flexible private VA range and allocates guard pages below the stack as unbacked VA space. The stack is aligned to twice its size to aid overflow detection (implemented in a subsequent patch in the series). Signed-off-by: Kalesh Singh --- arch/arm64/kvm/hyp/nvhe/setup.c | 25 +++++++++++++++++++++---- 1 file changed, 21 insertions(+), 4 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/setup.c b/arch/arm64/kvm/hyp/nvhe/setu= p.c index 27af337f9fea..69df21320b09 100644 --- a/arch/arm64/kvm/hyp/nvhe/setup.c +++ b/arch/arm64/kvm/hyp/nvhe/setup.c @@ -105,11 +105,28 @@ static int recreate_hyp_mappings(phys_addr_t phys, un= signed long size, if (ret) return ret; =20 - end =3D (void *)per_cpu_ptr(&kvm_init_params, i)->stack_hyp_va; + /* + * Private mappings are allocated upwards from __io_map_base + * so allocate the guard page first then the stack. + */ + start =3D (void *)pkvm_alloc_private_va_range(PAGE_SIZE, PAGE_SIZE); + if (IS_ERR_OR_NULL(start)) + return PTR_ERR(start); + + /* + * The stack is aligned to twice its size to facilitate overflow + * detection. + */ + end =3D (void *)per_cpu_ptr(&kvm_init_params, i)->stack_pa; start =3D end - PAGE_SIZE; - ret =3D pkvm_create_mappings(start, end, PAGE_HYP); - if (ret) - return ret; + start =3D (void *)__pkvm_create_private_mapping((phys_addr_t)start, + PAGE_SIZE, PAGE_SIZE * 2, PAGE_HYP); + if (IS_ERR_OR_NULL(start)) + return PTR_ERR(start); + end =3D start + PAGE_SIZE; + + /* Update stack_hyp_va to end of the stack's private VA range */ + per_cpu_ptr(&kvm_init_params, i)->stack_hyp_va =3D (unsigned long) end; } =20 /* --=20 2.35.1.473.g83b2b277ed-goog From nobody Thu Jun 25 07:05:25 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3AF5FC433EF for ; Tue, 22 Feb 2022 16:58:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234430AbiBVQ64 (ORCPT ); Tue, 22 Feb 2022 11:58:56 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:49644 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231211AbiBVQ6x (ORCPT ); Tue, 22 Feb 2022 11:58:53 -0500 Received: from mail-yb1-xb4a.google.com (mail-yb1-xb4a.google.com [IPv6:2607:f8b0:4864:20::b4a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 481E116C4C4 for ; Tue, 22 Feb 2022 08:58:27 -0800 (PST) Received: by mail-yb1-xb4a.google.com with SMTP id i205-20020a2522d6000000b00622c778ac7cso24923379ybi.3 for ; Tue, 22 Feb 2022 08:58:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:cc; bh=LdOkseL/SVBmCmTgjcnNq9QzVzqqoPlVFVsUav/JFK0=; b=FNoLI1ytq66gPDfA9ohgBGXwe3ZXAMfQPU+vmETuazTxn1eRt4sfndjKQl5vNg1s6N aKsNQ+NyKz9hsu+jiKv0znYXLeJd3n/+di/bVx097otwKJ+Pm8qhNs67V2lcMqwTVhI1 wW9Xhb5tkIIU2yAp/NzXxghYQ+kMzgy+o2wG8PnfZAZIib9of+6+4qFBo+n7HfiKcBUT zgEVEI8Bx5lOplWa+VY0r+2kHSy9Bk1YC2Ipw+jivWuz7/saxNwcRLetDoqydMzCZ/+o DlsgOQQ986VSJnpFZSHCEQu861DTK2XF2rz2ttpajHfiMXQ90DWFxCmT05fG0qHlS8Nr E9dA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:cc; bh=LdOkseL/SVBmCmTgjcnNq9QzVzqqoPlVFVsUav/JFK0=; b=P5iEr0f8TTnbCkU9p3fC0NrZsPOvuP/tcgxLu7dXMMuTm7e+6s1fNbVfxs82gU/B8H sk0168bIWP1Jzg4/yxaKC5xkmFXQgrPJV11eTD/cVB11GXi0g4Gvvso9zkLf2u57ge43 utTntKwW4WHkVj6lr260XMvOyesN7H2fzpAVnKM53F8PyC+WdnKkeYQyxE2OiDfeClqv fUsFdZQCudLYvXSLdsbPKMFeaHL5HtMn32hr8LFQHgQRUXJ5xrJeF85oo6rNzfsCJjQO 0p0KXkOdTkau6A2e0/pD2QNWblUDacsODfhdfndHbThK11KrF3vws2pAuuH3r9Xov7iM /oBg== X-Gm-Message-State: AOAM530wnHpdhOsOq718ZP46ZPdULo83PRerX5xXTcwVuiaW8wSchdN+ ixTapr5qIwMop7UlI5rld3S/1whvFD2T82nPSw== X-Google-Smtp-Source: ABdhPJxTMt4EP42XEzXDEACbHfSQnsjw9YnBj/jsWJgMJWkojIR6N0I/bKfbgjEHFp9OFaaN7ZlREpz8dxBvYs+xsg== X-Received: from kaleshsingh.mtv.corp.google.com ([2620:15c:211:200:5db7:1235:b3dd:cfcb]) (user=kaleshsingh job=sendgmr) by 2002:a81:b11:0:b0:2d7:3775:1eb9 with SMTP id 17-20020a810b11000000b002d737751eb9mr10900452ywl.99.1645549106468; Tue, 22 Feb 2022 08:58:26 -0800 (PST) Date: Tue, 22 Feb 2022 08:51:06 -0800 In-Reply-To: <20220222165212.2005066-1-kaleshsingh@google.com> Message-Id: <20220222165212.2005066-6-kaleshsingh@google.com> Mime-Version: 1.0 References: <20220222165212.2005066-1-kaleshsingh@google.com> X-Mailer: git-send-email 2.35.1.473.g83b2b277ed-goog Subject: [PATCH v2 5/9] arm64: asm: Introduce test_sp_overflow macro From: Kalesh Singh Cc: will@kernel.org, maz@kernel.org, qperret@google.com, tabba@google.com, surenb@google.com, kernel-team@android.com, Kalesh Singh , Catalin Marinas , James Morse , Alexandru Elisei , Suzuki K Poulose , Ard Biesheuvel , Mark Rutland , Pasha Tatashin , Joey Gouly , Peter Collingbourne , Andrew Scull , Paolo Bonzini , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Quentin Perret The asm entry code in the kernel uses a trick to check if VMAP'd stacks have overflowed by aligning them at THREAD_SHIFT * 2 granularity and checking the SP's THREAD_SHIFT bit. Protected KVM will soon make use of a similar trick to detect stack overflows, so factor out the asm code in a re-usable macro. Signed-off-by: Quentin Perret [Kalesh - Resolve minor conflicts] Signed-off-by: Kalesh Singh --- arch/arm64/include/asm/assembler.h | 11 +++++++++++ arch/arm64/kernel/entry.S | 7 +------ 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/as= sembler.h index e8bd0af0141c..ad40eb0eee83 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -850,4 +850,15 @@ alternative_endif =20 #endif /* GNU_PROPERTY_AARCH64_FEATURE_1_DEFAULT */ =20 +/* + * Test whether the SP has overflowed, without corrupting a GPR. + */ +.macro test_sp_overflow shift, label + add sp, sp, x0 // sp' =3D sp + x0 + sub x0, sp, x0 // x0' =3D sp' - x0 =3D (sp + x0) - x0 =3D sp + tbnz x0, #\shift, \label + sub x0, sp, x0 // x0'' =3D sp' - x0' =3D (sp + x0) - sp =3D x0 + sub sp, sp, x0 // sp'' =3D sp' - x0 =3D (sp + x0) - x0 =3D sp +.endm + #endif /* __ASM_ASSEMBLER_H */ diff --git a/arch/arm64/kernel/entry.S b/arch/arm64/kernel/entry.S index 772ec2ecf488..ce99ee30c77e 100644 --- a/arch/arm64/kernel/entry.S +++ b/arch/arm64/kernel/entry.S @@ -53,15 +53,10 @@ alternative_else_nop_endif sub sp, sp, #PT_REGS_SIZE #ifdef CONFIG_VMAP_STACK /* - * Test whether the SP has overflowed, without corrupting a GPR. * Task and IRQ stacks are aligned so that SP & (1 << THREAD_SHIFT) * should always be zero. */ - add sp, sp, x0 // sp' =3D sp + x0 - sub x0, sp, x0 // x0' =3D sp' - x0 =3D (sp + x0) - x0 =3D sp - tbnz x0, #THREAD_SHIFT, 0f - sub x0, sp, x0 // x0'' =3D sp' - x0' =3D (sp + x0) - sp =3D x0 - sub sp, sp, x0 // sp'' =3D sp' - x0 =3D (sp + x0) - x0 =3D sp + test_sp_overflow THREAD_SHIFT, 0f b el\el\ht\()_\regsize\()_\label =20 0: --=20 2.35.1.473.g83b2b277ed-goog From nobody Thu Jun 25 07:05:25 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67181C433F5 for ; Tue, 22 Feb 2022 16:59:35 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234446AbiBVQ77 (ORCPT ); Tue, 22 Feb 2022 11:59:59 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55198 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233343AbiBVQ75 (ORCPT ); Tue, 22 Feb 2022 11:59:57 -0500 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 67CE116C4D1 for ; Tue, 22 Feb 2022 08:59:32 -0800 (PST) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-2d6994a6942so144242377b3.17 for ; Tue, 22 Feb 2022 08:59:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:cc; bh=kA1uHTXTZgvCDEwUv0Venj02VbjFVl5MyBDxltrhOyc=; b=NP6fnietNid07WODom2crvgVApuYNwyBc+vaQeT9ELb0Wby0y027V1WoutortBDdER 6VKH2m5g/hhqE8jP+GV8Z52SXKndZFpGpqKGnvqTJ6saOgO9yiXq/Sl8KOE48ShXSFrf Yd/GQs7iTARZWcbcAbs2DZcziHvlIf4ws8th2T86PRoxGLRCAo79SHT5a0UzBseydVww c/kdTBUTDmKW4WhBmbRdCfs4Sm1ZyK65IyLMvKPqQFMBa2uiSRCXXe8ja7vMgkYdHVOd FDQ7AFIgQCuxZCe7VWKGtZS0vLPpQ4mLhJFEH4IE0gr/JO65ZVTSGIb2gzhuMTjBEe1w 9t+A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:cc; bh=kA1uHTXTZgvCDEwUv0Venj02VbjFVl5MyBDxltrhOyc=; b=eDjOWfIo2bmmatAkxeRJT9A2HVV0AO7Djm8cCgheIbzbt7TOZVPEi9XYBJm6REpUgt kbrMz6/RJHDxAX6fswwzKSRv95B4Z35qH9RmNQqai9TZhxap59fs1ncu0OXFoZfOZS/o dFC7RbioUBJoBACrpXTklOzcRsgT4pVEPO6kW5628WREQOZH/MBP6gtwdx1DT5nT9HPn RepUzZPR7PN2AKd4AMuyW3dU7leenhgpcN4/T1Q0cM1AJQBN6GOB/Rce9141C3YDK4ts S1nXg6PvGgOdZhGGCfvSPHvew4m5toCtdGttMGMq1QM5wvVwuMwWhd2EmDZwgxqiJgrI i9sA== X-Gm-Message-State: AOAM533x02fkjgpRA55wHW0ED00fQtHyveRM1S/kmK1RHqSb6Ue0uku2 D8/h8lNid62yp1dve0BbwMqO5BaWOw2ADf9YTQ== X-Google-Smtp-Source: ABdhPJw2THd1z2iAbfih7pFjkHmBG5AlQKbmEqgBiWlGmwrM/dGUOKWBoivdDJ4o6t47SOdmT8K0alB2LRMYcbGgbA== X-Received: from kaleshsingh.mtv.corp.google.com ([2620:15c:211:200:5db7:1235:b3dd:cfcb]) (user=kaleshsingh job=sendgmr) by 2002:a81:138a:0:b0:2d6:b391:90cd with SMTP id 132-20020a81138a000000b002d6b39190cdmr23404107ywt.480.1645549171601; Tue, 22 Feb 2022 08:59:31 -0800 (PST) Date: Tue, 22 Feb 2022 08:51:07 -0800 In-Reply-To: <20220222165212.2005066-1-kaleshsingh@google.com> Message-Id: <20220222165212.2005066-7-kaleshsingh@google.com> Mime-Version: 1.0 References: <20220222165212.2005066-1-kaleshsingh@google.com> X-Mailer: git-send-email 2.35.1.473.g83b2b277ed-goog Subject: [PATCH v2 6/9] KVM: arm64: Detect and handle hypervisor stack overflows From: Kalesh Singh Cc: will@kernel.org, maz@kernel.org, qperret@google.com, tabba@google.com, surenb@google.com, kernel-team@android.com, Kalesh Singh , Catalin Marinas , James Morse , Alexandru Elisei , Suzuki K Poulose , Ard Biesheuvel , Mark Rutland , Pasha Tatashin , Joey Gouly , Peter Collingbourne , Andrew Scull , Paolo Bonzini , Zenghui Yu , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The hypervisor stacks (for both nVHE Hyp mode and nVHE protected mode) are aligned to twice their size (PAGE_SIZE), meaning that any valid stack address has PAGE_SHIFT bit as 0. This allows us to conveniently check for overflow in the exception entry without corrupting any GPRs. We won't recover from a stack overflow so panic the hypervisor. Signed-off-by: Kalesh Singh Reported-by: kernel test robot --- arch/arm64/kvm/hyp/nvhe/host.S | 16 ++++++++++++++++ arch/arm64/kvm/hyp/nvhe/switch.c | 5 +++++ 2 files changed, 21 insertions(+) diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index 3d613e721a75..78e4b612ac06 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -153,6 +153,10 @@ SYM_FUNC_END(__host_hvc) =20 .macro invalid_host_el2_vect .align 7 + + /* Test stack overflow without corrupting GPRs */ + test_sp_overflow PAGE_SHIFT, .L__hyp_sp_overflow\@ + /* If a guest is loaded, panic out of it. */ stp x0, x1, [sp, #-16]! get_loaded_vcpu x0, x1 @@ -165,6 +169,18 @@ SYM_FUNC_END(__host_hvc) * been partially clobbered by __host_enter. */ b hyp_panic + +.L__hyp_sp_overflow\@: + /* + * Reset SP to the top of the stack, to allow handling the hyp_panic. + * This corrupts the stack but is ok, since we won't be attempting + * any unwinding here. + */ + ldr_this_cpu x0, kvm_init_params + NVHE_INIT_STACK_HYP_VA, x1 + mov sp, x0 + + bl hyp_panic_bad_stack + ASM_BUG() .endm =20 .macro invalid_host_el1_vect diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/swi= tch.c index 6410d21d8695..5a2e1ab79913 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -369,6 +369,11 @@ void __noreturn hyp_panic(void) unreachable(); } =20 +void __noreturn hyp_panic_bad_stack(void) +{ + hyp_panic(); +} + asmlinkage void kvm_unexpected_el2_exception(void) { return __kvm_unexpected_el2_exception(); --=20 2.35.1.473.g83b2b277ed-goog From nobody Thu Jun 25 07:05:25 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3670CC433F5 for ; Tue, 22 Feb 2022 17:00:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233620AbiBVRBG (ORCPT ); Tue, 22 Feb 2022 12:01:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:56212 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231290AbiBVRBF (ORCPT ); Tue, 22 Feb 2022 12:01:05 -0500 Received: from mail-pg1-x549.google.com (mail-pg1-x549.google.com [IPv6:2607:f8b0:4864:20::549]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A061016C4EB for ; Tue, 22 Feb 2022 09:00:39 -0800 (PST) Received: by mail-pg1-x549.google.com with SMTP id p21-20020a631e55000000b00372d919267cso11653224pgm.1 for ; Tue, 22 Feb 2022 09:00:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:cc; bh=5qb5Zh5e8x0hA/HCW8SK7letHwHax9Wce8Hz1k3JTdg=; b=ITgvGI+DDxrws5QD+aqNmanS1XXPkR5OB1mc/GjeEFKe1PUFInLivQOj2yo7QUg+Ms jiQqEBIgx8+pm/kTGUv4kJbSou6k/zNoYhVYDkZAoPQcycS+1hRaIFS0LySP1N6gBZzN UaUG12NsYuyKvxTJaL6deitrMeqYZr6WLyZD3VhFhAjEtFzDl9XHbYq3JslYm4d8OYjz X/npdOM7xhWI6vlrhC6uIMRYM7xjLfG1ykah3QVJqU0WAp7G1+U7twKq7GAqvOV9yNkk pKJSH3usycLKHhOjCm8vwteuS3A7gapZDcF2n0sPj1Iupc/Hz3asBMWMKtVvwjT0Yhf5 /1Rw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:cc; bh=5qb5Zh5e8x0hA/HCW8SK7letHwHax9Wce8Hz1k3JTdg=; b=1vYKjfxTpGnw50sdbPPfeoDZAK1X6gmopRE2UuKaNCdEjem2Lx8JND5w37qTH0/j+h Vlceo89YYJO8sUnSfjt3VsWJLVan2Z0einAVgdjSehZ7y+21pT4xFrGsgk8xbd/CZoUX yOBMfEhbm7bdFF4zklGCMPrJJTJUjkZjxE55JWJM01a0zHijNvYEwd2Xi2LauyvlEFKf CUNqzwKvBcU7hviVHpJuLzzfYd3TAsfN2B6LtCDEMjM3dljaB3PIpvxwVVwzlh0ORuVd saTpa90ITFJuo4m4ZPNr0Bs0T/RiLeSOzO4wQrhQGrXWxqlS0BR2XageHu0LTkqA5gji YdtA== X-Gm-Message-State: AOAM533enYHbduRE82RGsLiajDPehIoqXDMIhZJWv0P0mPRoST0J4D9/ 0ww7ZUJJni/psHdU6qiCdXrqV4TCYSUC5yR7qw== X-Google-Smtp-Source: ABdhPJzH5Lfs/f/UpeVcOR5FEQsrODh9JTFqCt6nAPQwaG7BkrPBoWZU4Etxumn3mAF9Ju4JlTfm2YcSMf2h76tr4w== X-Received: from kaleshsingh.mtv.corp.google.com ([2620:15c:211:200:5db7:1235:b3dd:cfcb]) (user=kaleshsingh job=sendgmr) by 2002:a05:6a00:2296:b0:4e1:3029:ee2 with SMTP id f22-20020a056a00229600b004e130290ee2mr25396129pfe.22.1645549239077; Tue, 22 Feb 2022 09:00:39 -0800 (PST) Date: Tue, 22 Feb 2022 08:51:08 -0800 In-Reply-To: <20220222165212.2005066-1-kaleshsingh@google.com> Message-Id: <20220222165212.2005066-8-kaleshsingh@google.com> Mime-Version: 1.0 References: <20220222165212.2005066-1-kaleshsingh@google.com> X-Mailer: git-send-email 2.35.1.473.g83b2b277ed-goog Subject: [PATCH v2 7/9] KVM: arm64: Add hypervisor overflow stack From: Kalesh Singh Cc: will@kernel.org, maz@kernel.org, qperret@google.com, tabba@google.com, surenb@google.com, kernel-team@android.com, Kalesh Singh , Catalin Marinas , James Morse , Alexandru Elisei , Suzuki K Poulose , Ard Biesheuvel , Mark Rutland , Pasha Tatashin , Joey Gouly , Peter Collingbourne , Andrew Scull , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Allocate and switch to 16-byte aligned secondary stack on overflow. This provides us stack space to better handle overflows; and is used in a subsequent patch to dump the hypervisor stacktrace. The overflow stack is only allocated if CONFIG_NVHE_EL2_DEBUG is enabled, as hypervisor stacktraces is a debug feature dependent on CONFIG_NVHE_EL2_DEBUG. Signed-off-by: Kalesh Singh --- arch/arm64/kvm/hyp/nvhe/host.S | 5 +++++ arch/arm64/kvm/hyp/nvhe/switch.c | 5 +++++ 2 files changed, 10 insertions(+) diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index 78e4b612ac06..751a4b9e429f 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -171,6 +171,10 @@ SYM_FUNC_END(__host_hvc) b hyp_panic =20 .L__hyp_sp_overflow\@: +#ifdef CONFIG_NVHE_EL2_DEBUG + /* Switch to the overflow stack */ + adr_this_cpu sp, hyp_overflow_stack + PAGE_SIZE, x0 +#else /* * Reset SP to the top of the stack, to allow handling the hyp_panic. * This corrupts the stack but is ok, since we won't be attempting @@ -178,6 +182,7 @@ SYM_FUNC_END(__host_hvc) */ ldr_this_cpu x0, kvm_init_params + NVHE_INIT_STACK_HYP_VA, x1 mov sp, x0 +#endif =20 bl hyp_panic_bad_stack ASM_BUG() diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/swi= tch.c index 5a2e1ab79913..2accc158210f 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -34,6 +34,11 @@ DEFINE_PER_CPU(struct kvm_host_data, kvm_host_data); DEFINE_PER_CPU(struct kvm_cpu_context, kvm_hyp_ctxt); DEFINE_PER_CPU(unsigned long, kvm_hyp_vector); =20 +#ifdef CONFIG_NVHE_EL2_DEBUG +DEFINE_PER_CPU(unsigned long [PAGE_SIZE/sizeof(long)], hyp_overflow_stack) + __aligned(16); +#endif + static void __activate_traps(struct kvm_vcpu *vcpu) { u64 val; --=20 2.35.1.473.g83b2b277ed-goog From nobody Thu Jun 25 07:05:25 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2890CC433EF for ; Tue, 22 Feb 2022 17:01:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234476AbiBVRCM (ORCPT ); Tue, 22 Feb 2022 12:02:12 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58626 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233605AbiBVRCK (ORCPT ); Tue, 22 Feb 2022 12:02:10 -0500 Received: from mail-yw1-x114a.google.com (mail-yw1-x114a.google.com [IPv6:2607:f8b0:4864:20::114a]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C58062B19A for ; Tue, 22 Feb 2022 09:01:43 -0800 (PST) Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-2d11b6259adso147253097b3.19 for ; Tue, 22 Feb 2022 09:01:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:cc; bh=EEB0cF7W8Z4tzEJGAGYjBKXuJWIAUS9G1/uPvryZBvQ=; b=p7Id9dQujHyG29UdzWat8WdvrrPOw9krxSXiqk00tl1BZL9gYpFz1Lm+YwRDklhGUH 150VLbRfa0vcn/ERLt+En0egKXNgOC8lOqXzts0iqORAExDDaPBq3HdTShbvH1CSuDZK lCiPBIiXIti2UHtIQayQ0NSEjtzeT40JmK27ityUxx5sAtcfCy46cvf6FedW4XfLEfmZ fhR9famMNOpZEMx80sWjahdWq+EzlpJ4922/FHyVPoD0Ln4NAfbQO+vCazvCjeJOFkzM uavkGk1fmEM0XM2PtoEUoda5VCyP1ut3qzRDpZH2wkQuOscNUT3ISzbq5dQ2OWa6ykos IBYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:cc; bh=EEB0cF7W8Z4tzEJGAGYjBKXuJWIAUS9G1/uPvryZBvQ=; b=KJTd2J2kZpU2uI1WufAse5T/S2uUL2yhpHPoOAseNvvp8nxgghH2YpGkz4mWWIrsd+ Jtbhyv5iLI1PGWnWCitmHFwLP01jd258Wdllxp7fQEAVUAD54qPK1yw8w5RtEznSeF8j n2KnfOBj/TtfODEK4Og2eTHCxstzydx+0XOHAycVJXbjApdK0MM2jdD/ppxec8/Kq9r6 GypaJ6IVIYSAqNY+ATRJ7cyk53yG/9pe1ZsWfZmfQc64Mphz1T1aCBe3PkafcMl1Ur2L DvsBA6+pQF/rL1orD669dRbUcIHUsUd/ASfGz8uodeCzi1NTljht2jIN3l7tXhi0bnuO /v4g== X-Gm-Message-State: AOAM533x5kOPkqXWiIETRHbkqLdH8ZipdWjDckiJuOc38ob5hjcwGflK 3Zig2fy8wrvW0mjwn/3ecSRwQXXv0IXVhrE0Yg== X-Google-Smtp-Source: ABdhPJzEcdFuLjZ+IU/YVKrt6YVo9vU91uda9Y6U5r89FElLqqMG4NnbgdLFIeyeY+V8GzaBnO8HGFz4Jr+YvAi4wg== X-Received: from kaleshsingh.mtv.corp.google.com ([2620:15c:211:200:5db7:1235:b3dd:cfcb]) (user=kaleshsingh job=sendgmr) by 2002:a25:84cf:0:b0:621:a740:3249 with SMTP id x15-20020a2584cf000000b00621a7403249mr23161492ybm.275.1645549303006; Tue, 22 Feb 2022 09:01:43 -0800 (PST) Date: Tue, 22 Feb 2022 08:51:09 -0800 In-Reply-To: <20220222165212.2005066-1-kaleshsingh@google.com> Message-Id: <20220222165212.2005066-9-kaleshsingh@google.com> Mime-Version: 1.0 References: <20220222165212.2005066-1-kaleshsingh@google.com> X-Mailer: git-send-email 2.35.1.473.g83b2b277ed-goog Subject: [PATCH v2 8/9] KVM: arm64: Unwind and dump nVHE HYP stacktrace From: Kalesh Singh Cc: will@kernel.org, maz@kernel.org, qperret@google.com, tabba@google.com, surenb@google.com, kernel-team@android.com, Kalesh Singh , Catalin Marinas , James Morse , Alexandru Elisei , Suzuki K Poulose , Ard Biesheuvel , Mark Rutland , Pasha Tatashin , Joey Gouly , Peter Collingbourne , Andrew Scull , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Unwind the stack in EL1, when CONFIG_NVHE_EL2_DEBUG is enabled. This is possible because CONFIG_NVHE_EL2_DEBUG disables the host stage 2 protection which allows host to access the hypervisor stack pages in EL1. Unwinding and dumping hyp call traces is gated on CONFIG_NVHE_EL2_DEBUG to avoid the potential leaking of information to the host. A simple stack overflow test produces the following output: [ 580.376051][ T412] kvm: nVHE hyp panic at: ffffffc0116145c4! [ 580.378034][ T412] kvm [412]: nVHE HYP call trace: [ 580.378591][ T412] kvm [412]: [] [ 580.378993][ T412] kvm [412]: [] [ 580.379386][ T412] kvm [412]: [] // Non-terminating= recursive call [ 580.379772][ T412] kvm [412]: [] [ 580.380158][ T412] kvm [412]: [] [ 580.380544][ T412] kvm [412]: [] [ 580.380928][ T412] kvm [412]: [] . . . Since nVHE hyp symbols are not included by kallsyms to avoid issues with aliasing, we fallback to the vmlinux addresses. Symbolizing the addresses is handled in the next patch in this series. Signed-off-by: Kalesh Singh --- Changes in v2: - Add cpu_prepare_nvhe_panic_info() - Move updating the panic info to hyp_panic(), so that unwinding also works for conventional nVHE Hyp-mode. arch/arm64/include/asm/kvm_asm.h | 17 ++ arch/arm64/kvm/Kconfig | 5 +- arch/arm64/kvm/Makefile | 1 + arch/arm64/kvm/arm.c | 2 +- arch/arm64/kvm/handle_exit.c | 3 + arch/arm64/kvm/hyp/nvhe/switch.c | 19 ++ arch/arm64/kvm/stacktrace.c | 290 +++++++++++++++++++++++++++++++ arch/arm64/kvm/stacktrace.h | 17 ++ 8 files changed, 351 insertions(+), 3 deletions(-) create mode 100644 arch/arm64/kvm/stacktrace.c create mode 100644 arch/arm64/kvm/stacktrace.h diff --git a/arch/arm64/include/asm/kvm_asm.h b/arch/arm64/include/asm/kvm_= asm.h index 2e277f2ed671..af44b3a0596b 100644 --- a/arch/arm64/include/asm/kvm_asm.h +++ b/arch/arm64/include/asm/kvm_asm.h @@ -176,6 +176,23 @@ struct kvm_nvhe_init_params { unsigned long vtcr; }; =20 +#ifdef CONFIG_NVHE_EL2_DEBUG +/* + * Used by the host in EL1 to dump the nVHE hypervisor backtrace on + * hyp_panic. This is possible because CONFIG_NVHE_EL2_DEBUG disables + * the host stage 2 protection. See: __hyp_do_panic() + * + * @hyp_stack_base: hyp VA of the hyp_stack base. + * @hyp_overflow_stack_base: hyp VA of the hyp_overflow_stack base. + * @start_fp: hyp FP where the hyp backtrace should begin. + */ +struct kvm_nvhe_panic_info { + unsigned long hyp_stack_base; + unsigned long hyp_overflow_stack_base; + unsigned long start_fp; +}; +#endif + /* Translate a kernel address @ptr into its equivalent linear mapping */ #define kvm_ksym_ref(ptr) \ ({ \ diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig index 8a5fbbf084df..75f2c8255ff0 100644 --- a/arch/arm64/kvm/Kconfig +++ b/arch/arm64/kvm/Kconfig @@ -51,8 +51,9 @@ config NVHE_EL2_DEBUG depends on KVM help Say Y here to enable the debug mode for the non-VHE KVM EL2 object. - Failure reports will BUG() in the hypervisor. This is intended for - local EL2 hypervisor development. + Failure reports will BUG() in the hypervisor; and panics will print + the hypervisor call stack. This is intended for local EL2 hypervisor + development. =20 If unsure, say N. =20 diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile index 91861fd8b897..262b5c58cc62 100644 --- a/arch/arm64/kvm/Makefile +++ b/arch/arm64/kvm/Makefile @@ -23,6 +23,7 @@ kvm-y +=3D arm.o mmu.o mmio.o psci.o hypercalls.o pvtime.= o \ vgic/vgic-its.o vgic/vgic-debug.o =20 kvm-$(CONFIG_HW_PERF_EVENTS) +=3D pmu-emul.o +kvm-$(CONFIG_NVHE_EL2_DEBUG) +=3D stacktrace.o =20 always-y :=3D hyp_constants.h hyp-constants.s =20 diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 7e2e680c3ffb..491cf1eb28f6 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -49,7 +49,7 @@ DEFINE_STATIC_KEY_FALSE(kvm_protected_mode_initialized); =20 DECLARE_KVM_HYP_PER_CPU(unsigned long, kvm_hyp_vector); =20 -static DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_page); +DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_page); unsigned long kvm_arm_hyp_percpu_base[NR_CPUS]; DECLARE_KVM_NVHE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); =20 diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index e3140abd2e2e..b038c32a3236 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -23,6 +23,7 @@ =20 #define CREATE_TRACE_POINTS #include "trace_handle_exit.h" +#include "stacktrace.h" =20 typedef int (*exit_handle_fn)(struct kvm_vcpu *); =20 @@ -326,6 +327,8 @@ void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, = u64 spsr, kvm_err("nVHE hyp panic at: %016llx!\n", elr_virt + hyp_offset); } =20 + hyp_dump_backtrace(hyp_offset); + /* * Hyp has panicked and we're going to handle that by panicking the * kernel. The kernel offset will be revealed in the panic so we're diff --git a/arch/arm64/kvm/hyp/nvhe/switch.c b/arch/arm64/kvm/hyp/nvhe/swi= tch.c index 2accc158210f..57ab23f03b1e 100644 --- a/arch/arm64/kvm/hyp/nvhe/switch.c +++ b/arch/arm64/kvm/hyp/nvhe/switch.c @@ -37,6 +37,23 @@ DEFINE_PER_CPU(unsigned long, kvm_hyp_vector); #ifdef CONFIG_NVHE_EL2_DEBUG DEFINE_PER_CPU(unsigned long [PAGE_SIZE/sizeof(long)], hyp_overflow_stack) __aligned(16); +DEFINE_PER_CPU(struct kvm_nvhe_panic_info, kvm_panic_info); + +DECLARE_PER_CPU(struct kvm_nvhe_panic_info, kvm_panic_info); + +static void cpu_prepare_nvhe_panic_info(void) +{ + struct kvm_nvhe_panic_info *panic_info =3D this_cpu_ptr(&kvm_panic_info); + struct kvm_nvhe_init_params *params =3D this_cpu_ptr(&kvm_init_params); + + panic_info->hyp_stack_base =3D (unsigned long)(params->stack_hyp_va - PAG= E_SIZE); + panic_info->hyp_overflow_stack_base =3D (unsigned long)this_cpu_ptr(hyp_o= verflow_stack); + panic_info->start_fp =3D (unsigned long)__builtin_frame_address(0); +} +#else +static void cpu_prepare_nvhe_panic_info(void) +{ +} #endif =20 static void __activate_traps(struct kvm_vcpu *vcpu) @@ -360,6 +377,8 @@ void __noreturn hyp_panic(void) struct kvm_cpu_context *host_ctxt; struct kvm_vcpu *vcpu; =20 + cpu_prepare_nvhe_panic_info(); + host_ctxt =3D &this_cpu_ptr(&kvm_host_data)->host_ctxt; vcpu =3D host_ctxt->__hyp_running_vcpu; =20 diff --git a/arch/arm64/kvm/stacktrace.c b/arch/arm64/kvm/stacktrace.c new file mode 100644 index 000000000000..cdd672bf0ea8 --- /dev/null +++ b/arch/arm64/kvm/stacktrace.c @@ -0,0 +1,290 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Stack unwinder for EL2 nVHE hypervisor. + * + * Code mostly copied from the arm64 kernel stack unwinder + * and adapted to the nVHE hypervisor. + * + * See: arch/arm64/kernel/stacktrace.c + * + * CONFIG_NVHE_EL2_DEBUG disables the host stage-2 protection + * allowing us to access the hypervisor stack pages and + * consequently unwind its stack from the host in EL1. + * + * See: __hyp_do_panic() + */ + +#include +#include +#include +#include "stacktrace.h" + +DECLARE_PER_CPU(unsigned long, kvm_arm_hyp_stack_page); +DECLARE_KVM_NVHE_PER_CPU(unsigned long [PAGE_SIZE/sizeof(long)], hyp_overf= low_stack); +DECLARE_KVM_NVHE_PER_CPU(struct kvm_nvhe_panic_info, kvm_panic_info); + +enum hyp_stack_type { + HYP_STACK_TYPE_UNKNOWN, + HYP_STACK_TYPE_HYP, + HYP_STACK_TYPE_OVERFLOW, + __NR_HYP_STACK_TYPES +}; + +struct hyp_stack_info { + unsigned long low; + unsigned long high; + enum hyp_stack_type type; +}; + +/* + * A snapshot of a frame record or fp/lr register values, along with some + * accounting information necessary for robust unwinding. + * + * @fp: The fp value in the frame record (or the real fp) + * @pc: The pc value calculated from lr in the frame record. + * + * @stacks_done: Stacks which have been entirely unwound, for which it is = no + * longer valid to unwind to. + * + * @prev_fp: The fp that pointed to this frame record, or a synthetic = value + * of 0. This is used to ensure that within a stack, each + * subsequent frame record is at an increasing address. + * @prev_type: The type of stack this frame record was on, or a synthetic + * value of HYP_STACK_TYPE_UNKNOWN. This is used to detect a + * transition from one stack to another. + */ +struct hyp_stackframe { + unsigned long fp; + unsigned long pc; + DECLARE_BITMAP(stacks_done, __NR_HYP_STACK_TYPES); + unsigned long prev_fp; + enum hyp_stack_type prev_type; +}; + +static inline bool __on_hyp_stack(unsigned long hyp_sp, unsigned long size, + unsigned long low, unsigned long high, + enum hyp_stack_type type, + struct hyp_stack_info *info) +{ + if (!low) + return false; + + if (hyp_sp < low || hyp_sp + size < hyp_sp || hyp_sp + size > high) + return false; + + if (info) { + info->low =3D low; + info->high =3D high; + info->type =3D type; + } + return true; +} + +static inline bool on_hyp_overflow_stack(unsigned long hyp_sp, unsigned lo= ng size, + struct hyp_stack_info *info) +{ + struct kvm_nvhe_panic_info *panic_info =3D this_cpu_ptr_nvhe_sym(kvm_pani= c_info); + unsigned long low =3D (unsigned long)panic_info->hyp_overflow_stack_base; + unsigned long high =3D low + PAGE_SIZE; + + return __on_hyp_stack(hyp_sp, size, low, high, HYP_STACK_TYPE_OVERFLOW, i= nfo); +} + +static inline bool on_hyp_stack(unsigned long hyp_sp, unsigned long size, + struct hyp_stack_info *info) +{ + struct kvm_nvhe_panic_info *panic_info =3D this_cpu_ptr_nvhe_sym(kvm_pani= c_info); + unsigned long low =3D (unsigned long)panic_info->hyp_stack_base; + unsigned long high =3D low + PAGE_SIZE; + + return __on_hyp_stack(hyp_sp, size, low, high, HYP_STACK_TYPE_HYP, info); +} + +static inline bool on_hyp_accessible_stack(unsigned long hyp_sp, unsigned = long size, + struct hyp_stack_info *info) +{ + if (info) + info->type =3D HYP_STACK_TYPE_UNKNOWN; + + if (on_hyp_stack(hyp_sp, size, info)) + return true; + if (on_hyp_overflow_stack(hyp_sp, size, info)) + return true; + + return false; +} + +static unsigned long __hyp_stack_kern_va(unsigned long hyp_va) +{ + struct kvm_nvhe_panic_info *panic_info =3D this_cpu_ptr_nvhe_sym(kvm_pani= c_info); + unsigned long hyp_base, kern_base, hyp_offset; + + hyp_base =3D (unsigned long)panic_info->hyp_stack_base; + hyp_offset =3D hyp_va - hyp_base; + + kern_base =3D (unsigned long)*this_cpu_ptr(&kvm_arm_hyp_stack_page); + + return kern_base + hyp_offset; +} + +static unsigned long __hyp_overflow_stack_kern_va(unsigned long hyp_va) +{ + struct kvm_nvhe_panic_info *panic_info =3D this_cpu_ptr_nvhe_sym(kvm_pani= c_info); + unsigned long hyp_base, kern_base, hyp_offset; + + hyp_base =3D (unsigned long)panic_info->hyp_overflow_stack_base; + hyp_offset =3D hyp_va - hyp_base; + + kern_base =3D (unsigned long)this_cpu_ptr_nvhe_sym(hyp_overflow_stack); + + return kern_base + hyp_offset; +} + +/* + * Convert hypervisor stack VA to a kernel VA. + * + * The hypervisor stack is mapped in the flexible 'private' VA range, to a= llow + * for guard pages below the stack. Consequently, the fixed offset address + * translation macros won't work here. + * + * The kernel VA is calculated as an offset from the kernel VA of the hype= rvisor + * stack base. See: __hyp_stack_kern_va(), __hyp_overflow_stack_kern_va() + */ +static unsigned long hyp_stack_kern_va(unsigned long hyp_va, + enum hyp_stack_type stack_type) +{ + switch (stack_type) { + case HYP_STACK_TYPE_HYP: + return __hyp_stack_kern_va(hyp_va); + case HYP_STACK_TYPE_OVERFLOW: + return __hyp_overflow_stack_kern_va(hyp_va); + default: + return 0UL; + } +} + +/* + * Unwind from one frame record (A) to the next frame record (B). + * + * We terminate early if the location of B indicates a malformed chain of = frame + * records (e.g. a cycle), determined based on the location and fp value o= f A + * and the location (but not the fp value) of B. + */ +static int notrace hyp_unwind_frame(struct hyp_stackframe *frame) +{ + unsigned long fp =3D frame->fp, fp_kern_va; + struct hyp_stack_info info; + + if (fp & 0x7) + return -EINVAL; + + if (!on_hyp_accessible_stack(fp, 16, &info)) + return -EINVAL; + + if (test_bit(info.type, frame->stacks_done)) + return -EINVAL; + + /* + * As stacks grow downward, any valid record on the same stack must be + * at a strictly higher address than the prior record. + * + * Stacks can nest in the following order: + * + * HYP -> OVERFLOW + * + * ... but the nesting itself is strict. Once we transition from one + * stack to another, it's never valid to unwind back to that first + * stack. + */ + if (info.type =3D=3D frame->prev_type) { + if (fp <=3D frame->prev_fp) + return -EINVAL; + } else { + set_bit(frame->prev_type, frame->stacks_done); + } + + /* Translate the hyp stack address to a kernel address */ + fp_kern_va =3D hyp_stack_kern_va(fp, info.type); + if (!fp_kern_va) + return -EINVAL; + + /* + * Record this frame record's values and location. The prev_fp and + * prev_type are only meaningful to the next hyp_unwind_frame() + * invocation. + */ + frame->fp =3D READ_ONCE_NOCHECK(*(unsigned long *)(fp_kern_va)); + /* PC =3D LR - 4; All aarch64 instructions are 32-bits in size */ + frame->pc =3D READ_ONCE_NOCHECK(*(unsigned long *)(fp_kern_va + 8)) - 4; + frame->prev_fp =3D fp; + frame->prev_type =3D info.type; + + return 0; +} + +/* + * AArch64 PCS assigns the frame pointer to x29. + * + * A simple function prologue looks like this: + * sub sp, sp, #0x10 + * stp x29, x30, [sp] + * mov x29, sp + * + * A simple function epilogue looks like this: + * mov sp, x29 + * ldp x29, x30, [sp] + * add sp, sp, #0x10 + */ +static void hyp_start_backtrace(struct hyp_stackframe *frame, unsigned lon= g fp) +{ + frame->fp =3D fp; + + /* + * Prime the first unwind. + * + * In hyp_unwind_frame() we'll check that the FP points to a valid + * stack, which can't be HYP_STACK_TYPE_UNKNOWN, and the first unwind + * will be treated as a transition to whichever stack that happens to + * be. The prev_fp value won't be used, but we set it to 0 such that + * it is definitely not an accessible stack address. The first frame + * (hyp_panic()) is skipped, so we also set PC to 0. + */ + bitmap_zero(frame->stacks_done, __NR_HYP_STACK_TYPES); + frame->pc =3D frame->prev_fp =3D 0; + frame->prev_type =3D HYP_STACK_TYPE_UNKNOWN; +} + +static void hyp_dump_backtrace_entry(unsigned long hyp_pc, unsigned long h= yp_offset) +{ + unsigned long va_mask =3D GENMASK_ULL(vabits_actual - 1, 0); + + hyp_pc &=3D va_mask; /* Mask tags */ + hyp_pc +=3D hyp_offset; + + kvm_err("[<%016lx>]\n", hyp_pc); +} + +void hyp_dump_backtrace(unsigned long hyp_offset) +{ + struct kvm_nvhe_panic_info *panic_info =3D this_cpu_ptr_nvhe_sym(kvm_pani= c_info); + struct hyp_stackframe frame; + int frame_nr =3D 0; + int skip =3D 1; /* Skip the first frame: hyp_panic() */ + + kvm_err("nVHE HYP call trace:\n"); + + hyp_start_backtrace(&frame, (unsigned long)panic_info->start_fp); + + do { + if (skip) { + skip--; + continue; + } + + hyp_dump_backtrace_entry(frame.pc, hyp_offset); + + frame_nr++; + } while (!hyp_unwind_frame(&frame)); + + kvm_err("---- end of nVHE HYP call trace ----\n"); +} diff --git a/arch/arm64/kvm/stacktrace.h b/arch/arm64/kvm/stacktrace.h new file mode 100644 index 000000000000..40c397394b9b --- /dev/null +++ b/arch/arm64/kvm/stacktrace.h @@ -0,0 +1,17 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Stack unwinder for EL2 nVHE hypervisor. + */ + +#ifndef __KVM_HYP_STACKTRACE_H +#define __KVM_HYP_STACKTRACE_H + +#ifdef CONFIG_NVHE_EL2_DEBUG +void hyp_dump_backtrace(unsigned long hyp_offset); +#else +static inline void hyp_dump_backtrace(unsigned long hyp_offset) +{ +} +#endif /* CONFIG_NVHE_EL2_DEBUG */ + +#endif /* __KVM_HYP_STACKTRACE_H */ --=20 2.35.1.473.g83b2b277ed-goog From nobody Thu Jun 25 07:05:25 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8217DC433FE for ; Tue, 22 Feb 2022 17:02:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234497AbiBVRDR (ORCPT ); Tue, 22 Feb 2022 12:03:17 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32928 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232240AbiBVRDP (ORCPT ); Tue, 22 Feb 2022 12:03:15 -0500 Received: from mail-yb1-xb49.google.com (mail-yb1-xb49.google.com [IPv6:2607:f8b0:4864:20::b49]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 04E4650E0D for ; Tue, 22 Feb 2022 09:02:50 -0800 (PST) Received: by mail-yb1-xb49.google.com with SMTP id k10-20020a056902070a00b0062469b00335so9019428ybt.14 for ; Tue, 22 Feb 2022 09:02:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:cc; bh=beGtXg45IQ4dKCY/A8ewT2jAssjG8oCd/wil2F8VoF8=; b=Qmw8f9ngENw67mW7ZY926jso9fM6FSWsadBt2xuNCmEM/wL9yNNx+++6QAzW5Geysa X0CVpzvJkNzo1xClPSFGBfQoQ9e82eUu5kgF0cfza/NxsGQMydRbfBIhqKkBzLs5aCzK VBY/ce5OQiMkErnhinkSoy35Z7xGC9tfoOBy0pVxgyAmPZog1qhtESjBaWlKEIEeimgX bKnw6hLvfmjhuk5Ps7Kiuf9lL11CkBXEiVCRXEz1k0Pj8iOLK68P3f2mF/3G5iD/5IP2 26qsHRaykVbWjx15kawdg3HR402vdDmc9IFBtIz1CQ/1k/qSnS4QDbrQcuG21X7huvEk B9pw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:cc; bh=beGtXg45IQ4dKCY/A8ewT2jAssjG8oCd/wil2F8VoF8=; b=RFVbxtaBYfZoXwGCu/XOp2pHpM3T74HwS4lqmwC4talSVmUxJRCibMIwfnDN04DD+8 R0TgiASd3W3/irVB3/7Plngd/edvlalaZSKxn5O6YKC9XVNIiZygmQuOY40Wl7sSq4u7 zIWADkH5tyWE0Rkp1lhoL/CPKgOgTjsW4cjzUd6wTK0umC4ttzceWF9pBx5RthiWfka7 ye6cOX7I5dQTpDnnrT9kEc25iVVFpLxxrbjh44OEIm0u+1NnqBIzV6W16WNliBoNNC2b oQYQVd4R92fEOt9u9yw0FzjmALejxXLWtIU996qdmtjxIRWlbdqFbyadmYigARtyxVyF /O7w== X-Gm-Message-State: AOAM532XY2Nt5ybdPGK8bH2eR5V+++tzeFiUtJn1OYT1ZKkIpATp28b1 CdVitTrBh+lGVIJtDkF5VggFO9kxLQM2ibz00Q== X-Google-Smtp-Source: ABdhPJx9ya/dSuGbtyc2Drwg2dL1gTZ3RclR9peZ4NgVIVWBnHY+dyxNePng3kg7EOauiAGLQaMx/zndC7RygsTqtw== X-Received: from kaleshsingh.mtv.corp.google.com ([2620:15c:211:200:5db7:1235:b3dd:cfcb]) (user=kaleshsingh job=sendgmr) by 2002:a25:d60c:0:b0:610:dc8d:b3bd with SMTP id n12-20020a25d60c000000b00610dc8db3bdmr24399063ybg.561.1645549369167; Tue, 22 Feb 2022 09:02:49 -0800 (PST) Date: Tue, 22 Feb 2022 08:51:10 -0800 In-Reply-To: <20220222165212.2005066-1-kaleshsingh@google.com> Message-Id: <20220222165212.2005066-10-kaleshsingh@google.com> Mime-Version: 1.0 References: <20220222165212.2005066-1-kaleshsingh@google.com> X-Mailer: git-send-email 2.35.1.473.g83b2b277ed-goog Subject: [PATCH v2 9/9] KVM: arm64: Symbolize the nVHE HYP backtrace From: Kalesh Singh Cc: will@kernel.org, maz@kernel.org, qperret@google.com, tabba@google.com, surenb@google.com, kernel-team@android.com, Kalesh Singh , Catalin Marinas , James Morse , Alexandru Elisei , Suzuki K Poulose , Ard Biesheuvel , Mark Rutland , Pasha Tatashin , Joey Gouly , Peter Collingbourne , Andrew Walbran , Andrew Scull , Paolo Bonzini , Andrew Jones , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.cs.columbia.edu To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Reintroduce the __kvm_nvhe_ symbols in kallsyms, ignoring the local symbols in this namespace. The local symbols are not informative and can cause aliasing issues when symbolizing the addresses. With the necessary symbols now in kallsyms we can symbolize nVHE stacktrace addresses using the %pB print format specifier. Some sample call traces: ------- [ 167.018598][ T407] kvm [407]: nVHE hyp panic at: [] _= _kvm_nvhe_overflow_stack+0x10/0x34! [ 167.020841][ T407] kvm [407]: nVHE HYP call trace: [ 167.021371][ T407] kvm [407]: [] __kvm_nvhe_hyp_panic= _bad_stack+0xc/0x10 [ 167.021972][ T407] kvm [407]: [] __kvm_nvhe___kvm_hyp= _host_vector+0x248/0x794 [ 167.022572][ T407] kvm [407]: [] __kvm_nvhe_overflow_= stack+0x20/0x34 [ 167.023135][ T407] kvm [407]: [] __kvm_nvhe_overflow_= stack+0x20/0x34 [ 167.023699][ T407] kvm [407]: [] __kvm_nvhe_overflow_= stack+0x20/0x34 [ 167.024261][ T407] kvm [407]: [] __kvm_nvhe_overflow_= stack+0x20/0x34 . . . ------- [ 166.161699][ T409] kvm [409]: Invalid host exception to nVHE hyp! [ 166.163789][ T409] kvm [409]: nVHE HYP call trace: [ 166.164709][ T409] kvm [409]: [] __kvm_nvhe_handle___= kvm_vcpu_run+0x198/0x21c [ 166.165352][ T409] kvm [409]: [] __kvm_nvhe_handle_tr= ap+0xa4/0x124 [ 166.165911][ T409] kvm [409]: [] __kvm_nvhe___host_ex= it+0x60/0x64 [ 166.166657][ T409] Kernel panic - not syncing: HYP panic: . . . ------- Signed-off-by: Kalesh Singh --- Changes in v2: - Fix printk warnings - %p expects (void *) arch/arm64/kvm/handle_exit.c | 13 +++++-------- arch/arm64/kvm/stacktrace.c | 2 +- scripts/kallsyms.c | 2 +- 3 files changed, 7 insertions(+), 10 deletions(-) diff --git a/arch/arm64/kvm/handle_exit.c b/arch/arm64/kvm/handle_exit.c index b038c32a3236..1b953005d301 100644 --- a/arch/arm64/kvm/handle_exit.c +++ b/arch/arm64/kvm/handle_exit.c @@ -296,13 +296,8 @@ void __noreturn __cold nvhe_hyp_panic_handler(u64 esr,= u64 spsr, u64 elr_in_kimg =3D __phys_to_kimg(elr_phys); u64 hyp_offset =3D elr_in_kimg - kaslr_offset() - elr_virt; u64 mode =3D spsr & PSR_MODE_MASK; + u64 panic_addr =3D elr_virt + hyp_offset; =20 - /* - * The nVHE hyp symbols are not included by kallsyms to avoid issues - * with aliasing. That means that the symbols cannot be printed with the - * "%pS" format specifier, so fall back to the vmlinux address if - * there's no better option. - */ if (mode !=3D PSR_MODE_EL2t && mode !=3D PSR_MODE_EL2h) { kvm_err("Invalid host exception to nVHE hyp!\n"); } else if (ESR_ELx_EC(esr) =3D=3D ESR_ELx_EC_BRK64 && @@ -322,9 +317,11 @@ void __noreturn __cold nvhe_hyp_panic_handler(u64 esr,= u64 spsr, if (file) kvm_err("nVHE hyp BUG at: %s:%u!\n", file, line); else - kvm_err("nVHE hyp BUG at: %016llx!\n", elr_virt + hyp_offset); + kvm_err("nVHE hyp BUG at: [<%016llx>] %pB!\n", panic_addr, + (void *)panic_addr); } else { - kvm_err("nVHE hyp panic at: %016llx!\n", elr_virt + hyp_offset); + kvm_err("nVHE hyp panic at: [<%016llx>] %pB!\n", panic_addr, + (void *)panic_addr); } =20 hyp_dump_backtrace(hyp_offset); diff --git a/arch/arm64/kvm/stacktrace.c b/arch/arm64/kvm/stacktrace.c index cdd672bf0ea8..896c225a4a89 100644 --- a/arch/arm64/kvm/stacktrace.c +++ b/arch/arm64/kvm/stacktrace.c @@ -261,7 +261,7 @@ static void hyp_dump_backtrace_entry(unsigned long hyp_= pc, unsigned long hyp_off hyp_pc &=3D va_mask; /* Mask tags */ hyp_pc +=3D hyp_offset; =20 - kvm_err("[<%016lx>]\n", hyp_pc); + kvm_err("[<%016lx>] %pB\n", hyp_pc, (void *)hyp_pc); } =20 void hyp_dump_backtrace(unsigned long hyp_offset) diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c index 54ad86d13784..19aba43d9da4 100644 --- a/scripts/kallsyms.c +++ b/scripts/kallsyms.c @@ -111,7 +111,7 @@ static bool is_ignored_symbol(const char *name, char ty= pe) ".LASANPC", /* s390 kasan local symbols */ "__crc_", /* modversions */ "__efistub_", /* arm64 EFI stub namespace */ - "__kvm_nvhe_", /* arm64 non-VHE KVM namespace */ + "__kvm_nvhe_$", /* arm64 local symbols in non-VHE KVM namespace */ "__AArch64ADRPThunk_", /* arm64 lld */ "__ARMV5PILongThunk_", /* arm lld */ "__ARMV7PILongThunk_", --=20 2.35.1.473.g83b2b277ed-goog