From nobody Sat Nov 23 21:29:44 2024 Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4567E194082 for ; Fri, 8 Nov 2024 21:15:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731100539; cv=none; b=RIfmUD99S6qQXz2bcLM0C/vL6u+yno0A8LoHBY2expCLNkkLC2qo/BqVcivUlfqQwZ0hlfk4GDoGx2y9qj1Z2eop0+YXjKK6ffu3HRvwB3M/MntjuxkwRxQvbIhPns2XUikmn6afzLipHNHmc+LMztVxqDaCDTMFsEzKw2luIkY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731100539; c=relaxed/simple; bh=3mnwDE8EcYDJohsTNnQuX49X1JYTz6SUGnbFTkORf0o=; h=Date:Mime-Version:Message-ID:Subject:From:To:Cc:Content-Type; b=oTojfsw/3XNV0Q7ITfDWVgl8MlR6vc6CNhrruWTVGRKKaARH+/mLGs4/RYryg51Hsn9a8xn98HQa8Wgvhfw9CdFyQPLnmXwq5y98Hmi2h7exMJsalIC1l/CCbNMP7R7dpBDoVbNUXoeOCw8g/S7zMJLTgAGbDlQD1uQ2mZfDQtk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--kaleshsingh.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=DQNMLimT; arc=none smtp.client-ip=209.85.128.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--kaleshsingh.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="DQNMLimT" Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-6ea8d6fb2ffso50131577b3.2 for ; Fri, 08 Nov 2024 13:15:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1731100536; x=1731705336; darn=vger.kernel.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=Xj9B9RMnkmM/PP2fh4BdeD15ANP1xy3HAxUiNCCHfSo=; b=DQNMLimTYqCLqEvf14/Riynhyt5gEQ/XB5N8aDWC6R8U9A5oBmTT7ggMNvq00YQ8Oa Uup4SG7avuAMQ8V1nfAD6FzpAFbUUQOEsWE6ual1cd3wBFGT+qdcDHZpZPdKBOO26SCg 5UINL32FsxM+taRpDy8etizMPErZ1Q0fAwWCJn5Y5Mn5n2olJdK4NinrZnjpXjoQjASx mqYtbE6pXEwEUEXY2qecet7yDZsWUqFFbe0tyxgJCWFCVWVEzxz8qW7IFBq+a6d7BHIm xHrPLSrffwfz/E2y27D+yio7KEtYyBgKHTDt4S5aoAJNGE1nTGth4G1qoq6Ihi8w8n0D XROQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731100536; x=1731705336; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=Xj9B9RMnkmM/PP2fh4BdeD15ANP1xy3HAxUiNCCHfSo=; b=ZVEvj1KjeW3EpU9+qjTH8cE+7cgrH7PCd2yCVZCBMKbKUS5nb4CTVIVt06Q5APpm0k mndfE/UVfXrf0HuHxY5ByJPtG6drlTU8YGZFLfkUMurVFxaOO6JOKtYN7Ie00h1s4XpU /DYdd0dqPH1hmcrM1TimcN//HT6tXWjsdliRA/x+gt3krC64h9+4anBcZJXxI3kuVsw5 XXmzG+Nhrd7Q4B1PCuOzX7kINhdENZk69EurgyX/ypH7bsnaApkeC/AxXo/tFk55CXBf toJXjsGOngrb6FMvABTzjI1ds1lUpSG6AbmbWj94mEbdUBSNN3546WEYdldVStmsWrld UkLA== X-Forwarded-Encrypted: i=1; AJvYcCWy2YI2OcW7qC128KHx/AiCJLKJpHpsJshy+w+VcOnT35iq64tjQ9RTU4HunaLJQ/cUpHAtA529mOiD2iw=@vger.kernel.org X-Gm-Message-State: AOJu0Yzf6fBHH0L9qb8XYVLtra8dX+g1CrHn1/tZ8ayPjicPzttt/5nk tn+MKvtF6mJ7bn3Bm5CtOvOvSVQ63vrAWH6oSk1kChnT0o3ZhVXqH+KGqAqMksl+xP7QxHLjmqX CTnM+tstm9wPb0K6iRcn1nA== X-Google-Smtp-Source: AGHT+IEsHFWnlCL6mSsTKptJZufFOreEVWGFRJq416Pbau44wC4RVCznab2s8sBpPto1P5Y2xzKAJABWpyJ20H2TbA== X-Received: from kalesh.mtv.corp.google.com ([2a00:79e0:2e3f:8:bf60:92db:a254:85c0]) (user=kaleshsingh job=sendgmr) by 2002:a05:690c:7307:b0:6ea:3c62:17c1 with SMTP id 00721157ae682-6eaddd75f83mr406757b3.1.1731100536154; Fri, 08 Nov 2024 13:15:36 -0800 (PST) Date: Fri, 8 Nov 2024 13:14:00 -0800 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 X-Mailer: git-send-email 2.47.0.277.g8800431eea-goog Message-ID: <20241108211446.3304809-1-kaleshsingh@google.com> Subject: [PATCH] arm64: kvm: Make nvhe stack size configurable From: Kalesh Singh To: will@kernel.org, qperret@google.com, maz@kernel.org, broonie@kernel.org, mark.rutland@arm.com Cc: keir@google.com, vdonnefort@google.com, kernel-team@android.com, android-mm@google.com, Kalesh Singh , Catalin Marinas , Oliver Upton , Joey Gouly , Suzuki K Poulose , Zenghui Yu , Ard Biesheuvel , D Scott Phillips , Andrey Konovalov , Ankit Agrawal , Wang Jinchao , "Madhavan T. Venkataraman" , "=?UTF-8?q?Pierre-Cl=C3=A9ment=20Tosi?=" , Bjorn Helgaas , Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In order to make the nVHE stack size easily configurable, introduce NVHE_STACK_SHIFT which must be >=3D PAGE_SHIFT. The default stack size remains 1 page (no functional change) Downstream vendor features which require a larger stack can configure it by setting CONFIG_NVHE_STACK_SHIFT. Cc: Marc Zyngier Cc: Mark Brown Cc: Mark Rutland Cc: Quentin Perret Cc: Will Deacon Signed-off-by: Kalesh Singh --- arch/arm64/include/asm/memory.h | 5 ++++- arch/arm64/include/asm/stacktrace/nvhe.h | 2 +- arch/arm64/kvm/Kconfig | 14 ++++++++++++++ arch/arm64/kvm/arm.c | 18 +++++++++--------- arch/arm64/kvm/hyp/nvhe/host.S | 4 ++-- arch/arm64/kvm/hyp/nvhe/mm.c | 12 ++++++------ arch/arm64/kvm/hyp/nvhe/stacktrace.c | 4 ++-- arch/arm64/kvm/mmu.c | 12 ++++++------ arch/arm64/kvm/stacktrace.c | 6 +++--- 9 files changed, 47 insertions(+), 30 deletions(-) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memor= y.h index 0480c61dbb4f..55e592351749 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -145,13 +145,16 @@ =20 #define OVERFLOW_STACK_SIZE SZ_4K =20 +#define NVHE_STACK_SHIFT CONFIG_NVHE_STACK_SHIFT +#define NVHE_STACK_SIZE (UL(1) << NVHE_STACK_SHIFT) + /* * With the minimum frame size of [x29, x30], exactly half the combined * sizes of the hyp and overflow stacks is the maximum size needed to * save the unwinded stacktrace; plus an additional entry to delimit the * end. */ -#define NVHE_STACKTRACE_SIZE ((OVERFLOW_STACK_SIZE + PAGE_SIZE) / 2 + size= of(long)) +#define NVHE_STACKTRACE_SIZE ((OVERFLOW_STACK_SIZE + NVHE_STACK_SIZE) / 2 = + sizeof(long)) =20 /* * Alignment of kernel segments (e.g. .text, .data). diff --git a/arch/arm64/include/asm/stacktrace/nvhe.h b/arch/arm64/include/= asm/stacktrace/nvhe.h index 44759281d0d4..171f9edef49f 100644 --- a/arch/arm64/include/asm/stacktrace/nvhe.h +++ b/arch/arm64/include/asm/stacktrace/nvhe.h @@ -47,7 +47,7 @@ static inline void kvm_nvhe_unwind_init(struct unwind_sta= te *state, =20 DECLARE_KVM_NVHE_PER_CPU(unsigned long [OVERFLOW_STACK_SIZE/sizeof(long)],= overflow_stack); DECLARE_KVM_NVHE_PER_CPU(struct kvm_nvhe_stacktrace_info, kvm_stacktrace_i= nfo); -DECLARE_PER_CPU(unsigned long, kvm_arm_hyp_stack_page); +DECLARE_PER_CPU(unsigned long, kvm_arm_hyp_stack_base); =20 void kvm_nvhe_dump_backtrace(unsigned long hyp_offset); =20 diff --git a/arch/arm64/kvm/Kconfig b/arch/arm64/kvm/Kconfig index ead632ad01b4..7aaf0951086c 100644 --- a/arch/arm64/kvm/Kconfig +++ b/arch/arm64/kvm/Kconfig @@ -66,6 +66,20 @@ config PROTECTED_NVHE_STACKTRACE =20 If unsure, or not using protected nVHE (pKVM), say N. =20 +config NVHE_STACK_SHIFT + int "Size of the nVHE stack as a power of 2" + depends on KVM + range 12 13 if ARM64_4K_PAGES + default "12" if ARM64_4K_PAGES + range 14 15 if ARM64_16K_PAGES + default "14" if ARM64_16K_PAGES + range 16 17 if ARM64_64K_PAGES + default "16" if ARM64_64K_PAGES + help + This option specifies the size of the KVM nVHE stack (for both + conventional nVHE and pKVM modes) as a power of 2 -- which must + be at least as large as PAGE_SIZE. + config PTDUMP_STAGE2_DEBUGFS bool "Present the stage-2 pagetables to debugfs" depends on KVM diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c index 48cafb65d6ac..d19bffbfcb9e 100644 --- a/arch/arm64/kvm/arm.c +++ b/arch/arm64/kvm/arm.c @@ -61,7 +61,7 @@ static enum kvm_wfx_trap_policy kvm_wfe_trap_policy __rea= d_mostly =3D KVM_WFX_NOTR =20 DECLARE_KVM_HYP_PER_CPU(unsigned long, kvm_hyp_vector); =20 -DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_page); +DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_base); DECLARE_KVM_NVHE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params); =20 DECLARE_KVM_NVHE_PER_CPU(struct kvm_cpu_context, kvm_hyp_ctxt); @@ -2359,7 +2359,7 @@ static void __init teardown_hyp_mode(void) =20 free_hyp_pgds(); for_each_possible_cpu(cpu) { - free_page(per_cpu(kvm_arm_hyp_stack_page, cpu)); + free_pages(per_cpu(kvm_arm_hyp_stack_base, cpu), NVHE_STACK_SHIFT - PAGE= _SHIFT); free_pages(kvm_nvhe_sym(kvm_arm_hyp_percpu_base)[cpu], nvhe_percpu_order= ()); =20 if (free_sve) { @@ -2547,15 +2547,15 @@ static int __init init_hyp_mode(void) * Allocate stack pages for Hypervisor-mode */ for_each_possible_cpu(cpu) { - unsigned long stack_page; + unsigned long stack_base; =20 - stack_page =3D __get_free_page(GFP_KERNEL); - if (!stack_page) { + stack_base =3D __get_free_pages(GFP_KERNEL, NVHE_STACK_SHIFT - PAGE_SHIF= T); + if (!stack_base) { err =3D -ENOMEM; goto out_err; } =20 - per_cpu(kvm_arm_hyp_stack_page, cpu) =3D stack_page; + per_cpu(kvm_arm_hyp_stack_base, cpu) =3D stack_base; } =20 /* @@ -2624,9 +2624,9 @@ static int __init init_hyp_mode(void) */ for_each_possible_cpu(cpu) { struct kvm_nvhe_init_params *params =3D per_cpu_ptr_nvhe_sym(kvm_init_pa= rams, cpu); - char *stack_page =3D (char *)per_cpu(kvm_arm_hyp_stack_page, cpu); + char *stack_base =3D (char *)per_cpu(kvm_arm_hyp_stack_base, cpu); =20 - err =3D create_hyp_stack(__pa(stack_page), ¶ms->stack_hyp_va); + err =3D create_hyp_stack(__pa(stack_base), ¶ms->stack_hyp_va); if (err) { kvm_err("Cannot map hyp stack\n"); goto out_err; @@ -2638,7 +2638,7 @@ static int __init init_hyp_mode(void) * __hyp_pa() won't do the right thing there, since the stack * has been mapped in the flexible private VA space. */ - params->stack_pa =3D __pa(stack_page); + params->stack_pa =3D __pa(stack_base); } =20 for_each_possible_cpu(cpu) { diff --git a/arch/arm64/kvm/hyp/nvhe/host.S b/arch/arm64/kvm/hyp/nvhe/host.S index 3d610fc51f4d..58f0cb2298cc 100644 --- a/arch/arm64/kvm/hyp/nvhe/host.S +++ b/arch/arm64/kvm/hyp/nvhe/host.S @@ -188,12 +188,12 @@ SYM_FUNC_END(__host_hvc) =20 /* * Test whether the SP has overflowed, without corrupting a GPR. - * nVHE hypervisor stacks are aligned so that the PAGE_SHIFT bit + * nVHE hypervisor stacks are aligned so that the NVHE_STACK_SHIFT bit * of SP should always be 1. */ add sp, sp, x0 // sp' =3D sp + x0 sub x0, sp, x0 // x0' =3D sp' - x0 =3D (sp + x0) - x0 =3D sp - tbz x0, #PAGE_SHIFT, .L__hyp_sp_overflow\@ + tbz x0, #NVHE_STACK_SHIFT, .L__hyp_sp_overflow\@ sub x0, sp, x0 // x0'' =3D sp' - x0' =3D (sp + x0) - sp =3D x0 sub sp, sp, x0 // sp'' =3D sp' - x0 =3D (sp + x0) - x0 =3D sp =20 diff --git a/arch/arm64/kvm/hyp/nvhe/mm.c b/arch/arm64/kvm/hyp/nvhe/mm.c index 8850b591d775..f41c7440b34b 100644 --- a/arch/arm64/kvm/hyp/nvhe/mm.c +++ b/arch/arm64/kvm/hyp/nvhe/mm.c @@ -360,10 +360,10 @@ int pkvm_create_stack(phys_addr_t phys, unsigned long= *haddr) =20 prev_base =3D __io_map_base; /* - * Efficient stack verification using the PAGE_SHIFT bit implies + * Efficient stack verification using the NVHE_STACK_SHIFT bit implies * an alignment of our allocation on the order of the size. */ - size =3D PAGE_SIZE * 2; + size =3D NVHE_STACK_SIZE * 2; addr =3D ALIGN(__io_map_base, size); =20 ret =3D __pkvm_alloc_private_va_range(addr, size); @@ -373,12 +373,12 @@ int pkvm_create_stack(phys_addr_t phys, unsigned long= *haddr) * at the higher address and leave the lower guard page * unbacked. * - * Any valid stack address now has the PAGE_SHIFT bit as 1 + * Any valid stack address now has the NVHE_STACK_SHIFT bit as 1 * and addresses corresponding to the guard page have the - * PAGE_SHIFT bit as 0 - this is used for overflow detection. + * NVHE_STACK_SHIFT bit as 0 - this is used for overflow detection. */ - ret =3D kvm_pgtable_hyp_map(&pkvm_pgtable, addr + PAGE_SIZE, - PAGE_SIZE, phys, PAGE_HYP); + ret =3D kvm_pgtable_hyp_map(&pkvm_pgtable, addr + NVHE_STACK_SIZE, + NVHE_STACK_SIZE, phys, PAGE_HYP); if (ret) __io_map_base =3D prev_base; } diff --git a/arch/arm64/kvm/hyp/nvhe/stacktrace.c b/arch/arm64/kvm/hyp/nvhe= /stacktrace.c index ed6b58b19cfa..5b6eeab1a774 100644 --- a/arch/arm64/kvm/hyp/nvhe/stacktrace.c +++ b/arch/arm64/kvm/hyp/nvhe/stacktrace.c @@ -28,7 +28,7 @@ static void hyp_prepare_backtrace(unsigned long fp, unsig= ned long pc) struct kvm_nvhe_stacktrace_info *stacktrace_info =3D this_cpu_ptr(&kvm_st= acktrace_info); struct kvm_nvhe_init_params *params =3D this_cpu_ptr(&kvm_init_params); =20 - stacktrace_info->stack_base =3D (unsigned long)(params->stack_hyp_va - PA= GE_SIZE); + stacktrace_info->stack_base =3D (unsigned long)(params->stack_hyp_va - NV= HE_STACK_SIZE); stacktrace_info->overflow_stack_base =3D (unsigned long)this_cpu_ptr(over= flow_stack); stacktrace_info->fp =3D fp; stacktrace_info->pc =3D pc; @@ -54,7 +54,7 @@ static struct stack_info stackinfo_get_hyp(void) { struct kvm_nvhe_init_params *params =3D this_cpu_ptr(&kvm_init_params); unsigned long high =3D params->stack_hyp_va; - unsigned long low =3D high - PAGE_SIZE; + unsigned long low =3D high - NVHE_STACK_SIZE; =20 return (struct stack_info) { .low =3D low, diff --git a/arch/arm64/kvm/mmu.c b/arch/arm64/kvm/mmu.c index 0f7658aefa1a..e366ed8650fe 100644 --- a/arch/arm64/kvm/mmu.c +++ b/arch/arm64/kvm/mmu.c @@ -704,10 +704,10 @@ int create_hyp_stack(phys_addr_t phys_addr, unsigned = long *haddr) =20 mutex_lock(&kvm_hyp_pgd_mutex); /* - * Efficient stack verification using the PAGE_SHIFT bit implies + * Efficient stack verification using the NVHE_STACK_SHIFT bit implies * an alignment of our allocation on the order of the size. */ - size =3D PAGE_SIZE * 2; + size =3D NVHE_STACK_SIZE * 2; base =3D ALIGN_DOWN(io_map_base - size, size); =20 ret =3D __hyp_alloc_private_va_range(base); @@ -724,12 +724,12 @@ int create_hyp_stack(phys_addr_t phys_addr, unsigned = long *haddr) * at the higher address and leave the lower guard page * unbacked. * - * Any valid stack address now has the PAGE_SHIFT bit as 1 + * Any valid stack address now has the NVHE_STACK_SHIFT bit as 1 * and addresses corresponding to the guard page have the - * PAGE_SHIFT bit as 0 - this is used for overflow detection. + * NVHE_STACK_SHIFT bit as 0 - this is used for overflow detection. */ - ret =3D __create_hyp_mappings(base + PAGE_SIZE, PAGE_SIZE, phys_addr, - PAGE_HYP); + ret =3D __create_hyp_mappings(base + NVHE_STACK_SIZE, NVHE_STACK_SIZE, + phys_addr, PAGE_HYP); if (ret) kvm_err("Cannot map hyp stack\n"); =20 diff --git a/arch/arm64/kvm/stacktrace.c b/arch/arm64/kvm/stacktrace.c index 3ace5b75813b..b9744a932920 100644 --- a/arch/arm64/kvm/stacktrace.c +++ b/arch/arm64/kvm/stacktrace.c @@ -50,7 +50,7 @@ static struct stack_info stackinfo_get_hyp(void) struct kvm_nvhe_stacktrace_info *stacktrace_info =3D this_cpu_ptr_nvhe_sym(kvm_stacktrace_info); unsigned long low =3D (unsigned long)stacktrace_info->stack_base; - unsigned long high =3D low + PAGE_SIZE; + unsigned long high =3D low + NVHE_STACK_SIZE; =20 return (struct stack_info) { .low =3D low, @@ -60,8 +60,8 @@ static struct stack_info stackinfo_get_hyp(void) =20 static struct stack_info stackinfo_get_hyp_kern_va(void) { - unsigned long low =3D (unsigned long)*this_cpu_ptr(&kvm_arm_hyp_stack_pag= e); - unsigned long high =3D low + PAGE_SIZE; + unsigned long low =3D (unsigned long)*this_cpu_ptr(&kvm_arm_hyp_stack_bas= e); + unsigned long high =3D low + NVHE_STACK_SIZE; =20 return (struct stack_info) { .low =3D low, --=20 2.47.0.277.g8800431eea-goog