From nobody Mon Oct 6 06:29:00 2025 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5C9A6248F7C for ; Thu, 24 Jul 2025 23:37:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753400247; cv=none; b=jcGilpQh2W2NzVEf6NCnHA+PrURGIzSdZAwXcU44Kt+ahXcDjs1gMaLs2baHBCB585IdmTF9EZOF1hynYeeddrSL5AK+BhiEYOWYHTlecY7X7QzaebDbNyJmmpf+hVsWxEZN4PLT/oUIv2zii0Y6FtdbXjm67cZS4rwT3xOKEdQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1753400247; c=relaxed/simple; bh=g8myJdAMdYL2MhaKnL63Vuez4fkKjNIq12NTzG9z8qA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=RngKf0ayGOlxCulk9OYDYggU9njk6ePSbBmg+0esO/ieyo+dusoQzdZYbGbJjpvHN+oHeh3IVsOzo3arupUzDluQdMnh6AzMbEIlv/EKOI1hyT1SmQzfvCrVM/Ha9tihc8GOdn14Mu/8zYcoC7245EH21ace9bPI55NDZY/hXg4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=eO/rBZtL; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="eO/rBZtL" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-23649faf69fso12758745ad.0 for ; Thu, 24 Jul 2025 16:37:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1753400246; x=1754005046; darn=vger.kernel.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=cR/QY88sdxyw3B/3rrgSlLa8+7hhea91nQDWrYR/AlU=; b=eO/rBZtLJx0jFIyPeMhldXls+i7NMnkM4iHFtBbY9MdKNEHTCuUNKCZL/H6SxFVx1E HycbVsWfZ0lYGPwbbsU99pTpmA0KNQsLyZ9oc0XLMz+JKCkKjs+q95U9GySOizNGh4A1 VZVCGzM75h9kuNcGHNZclUZe/dsvDp+NhrmuIoEcOWVLZhK+/f9MhujzrajWwUcH1S66 bW70pHU7mlaZ1xlplem6jmG9V5OqWlsRH1dgf657mlvQvMBPYC7mFu2MtVKKsQAqwp5B EmqUsO0/2oEBhobae95VDuy3qkI9XWBdGo8Mpmpb04CUrPxFNc1Aj2Z35XqGscCNmZac s2hg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1753400246; x=1754005046; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=cR/QY88sdxyw3B/3rrgSlLa8+7hhea91nQDWrYR/AlU=; b=ufrZWRCv+pIxOTWFDihFV4nGEbYRRKUqDgBkmEsJfIb8pUGBtd+4PuT5cVyJOjFNmC 4Pw/Hon8oL9VYne3hK0sioghWMVHIhn7H4ZTBS50yFP7VYVuAvpoQvEgF2GiE2FFnKVL JAqGMtuzwcughpvcP54DR/3TMbKEWXg++dGdaSgxR/ze69sCFPU7DZUeEnhSZ/LAGHJ9 bOjm6V/IMGGufdiumOQo1k3yj5/lEwpPuaX/P0RvkYWHKGM8foT5Jgi7fb+N7a1Il/j5 wvaBkwBcJCz2cUeOjA461V+Rus8Snoxc57XnSRmUHkjcr0z5N8t4Us9PbwpWfCut/bom Dnag== X-Forwarded-Encrypted: i=1; AJvYcCVG4lJZvMfN0zCm7fO8PPqtPasLi5l6WT0FCVsCEPZyHPghVMeYJ86gjpYu63oPPcRW6+Yvc2Ij3gRksDE=@vger.kernel.org X-Gm-Message-State: AOJu0YywQa1Rh2XItWt/ifx77r+j6VOZjw7Nt7CwzWJCPL941ZGN8bE/ KPCbnkHrWVuKQcNIHbuSvwsGV04X6gabWuA01Wh62zy/xqhZWxuOQCWS4lWoldJJjBg= X-Gm-Gg: ASbGncu3IoSkxxvZ5dGrhQ63XhgCd04dG+CqltOWt4uVVA8/xOQcWduBBkuU8LXBEhD TfRUkZ9u1aTZsvO8VnuicYSdRljcyztewQUud/K4PW0oipGdHqQ2bxvmnAHqRXx4TOYXx9HsXSh rFT5pfvNrhNxc8j9faKHMliR8jcDEvh/imTvZUqcvGBeJDApLPIQ9FpqcZ59AsMMT1JKQX7Tlhs ipj6qU2pZD9VsWyAZoOkKHgzx7B2WHx580PNzpenDxOGExA56kWwKmweHIatIxbH5RGJsiRYW1J M+wH4hUQLXNOg1QIv57ZXAOVQAMNpQMKFtdBZ4L0Yj/FBV58o8a5P712sGXXgm6BH/GugK9lc// kK0Y/kpzXatFY+pdrLr9Z4P7t5K89HZRN X-Google-Smtp-Source: AGHT+IFpPmZfsrgw5MGLmfyvQ9TeyCoMqEZ9ePbdOyQJB+JX01/i58SgR22VLSNFSyWsuZBztHnwLA== X-Received: by 2002:a17:902:ce01:b0:235:1706:2002 with SMTP id d9443c01a7336-23f98083375mr117426355ad.0.1753400245784; Thu, 24 Jul 2025 16:37:25 -0700 (PDT) Received: from debug.ba.rivosinc.com ([64.71.180.162]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-23fa48bc706sm23598685ad.106.2025.07.24.16.37.23 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Jul 2025 16:37:25 -0700 (PDT) From: Deepak Gupta Date: Thu, 24 Jul 2025 16:37:02 -0700 Subject: [PATCH 09/11] riscv: scs: add hardware shadow stack support to scs Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20250724-riscv_kcfi-v1-9-04b8fa44c98c@rivosinc.com> References: <20250724-riscv_kcfi-v1-0-04b8fa44c98c@rivosinc.com> In-Reply-To: <20250724-riscv_kcfi-v1-0-04b8fa44c98c@rivosinc.com> To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Alexandre Ghiti , Masahiro Yamada , Nathan Chancellor , Nicolas Schier , Andrew Morton , David Hildenbrand , Lorenzo Stoakes , "Liam R. Howlett" , Vlastimil Babka , Mike Rapoport , Suren Baghdasaryan , Michal Hocko , Nick Desaulniers , Bill Wendling , Monk Chiang , Kito Cheng , Justin Stitt Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-kbuild@vger.kernel.org, linux-mm@kvack.org, llvm@lists.linux.dev, rick.p.edgecombe@intel.com, broonie@kernel.org, cleger@rivosinc.com, samitolvanen@google.com, apatel@ventanamicro.com, ajones@ventanamicro.com, conor.dooley@microchip.com, charlie@rivosinc.com, samuel.holland@sifive.com, bjorn@rivosinc.com, fweimer@redhat.com, jeffreyalaw@gmail.com, heinrich.schuchardt@canonical.com, andrew@sifive.com, ved@rivosinc.com, Deepak Gupta X-Mailer: b4 0.13.0 Adding support for hardware support for shadow call stack on riscv. This patch enables scs_* macros to use zicfiss shadow stack pointer (CSR_SSP) instead of relying on `gp`. Since zicfiss based shadow stack needs to have correct encoding set in PTE init shadow stack can't be established too early. It has to be setup after `setup_vm` is called. Thus `scs_load_init_stack` is noped out if CONFIG_ARCH_HAS_KERNEL_SHADOW_STACK is not selected. Adds `arch_scs_store` that can be used in generic scs magic store routine. Signed-off-by: Deepak Gupta --- arch/riscv/include/asm/asm.h | 2 +- arch/riscv/include/asm/scs.h | 48 +++++++++++++++++++++++++++++++++++-----= ---- arch/riscv/kernel/entry.S | 14 ++++++------- arch/riscv/kernel/head.S | 4 ++-- 4 files changed, 49 insertions(+), 19 deletions(-) diff --git a/arch/riscv/include/asm/asm.h b/arch/riscv/include/asm/asm.h index a8a2af6dfe9d..256aff523dd4 100644 --- a/arch/riscv/include/asm/asm.h +++ b/arch/riscv/include/asm/asm.h @@ -110,7 +110,7 @@ REG_L \dst, 0(\dst) .endm =20 -#ifdef CONFIG_SHADOW_CALL_STACK +#if defined(CONFIG_SHADOW_CALL_STACK) && !defined(CONFIG_ARCH_HAS_KERNEL_S= HADOW_STACK) /* gp is used as the shadow call stack pointer instead */ .macro load_global_pointer .endm diff --git a/arch/riscv/include/asm/scs.h b/arch/riscv/include/asm/scs.h index 0e45db78b24b..e70e6ef14bc5 100644 --- a/arch/riscv/include/asm/scs.h +++ b/arch/riscv/include/asm/scs.h @@ -9,46 +9,76 @@ =20 /* Load init_shadow_call_stack to gp. */ .macro scs_load_init_stack +#ifndef CONFIG_ARCH_HAS_KERNEL_SHADOW_STACK la gp, init_shadow_call_stack XIP_FIXUP_OFFSET gp +#endif .endm =20 /* Load the per-CPU IRQ shadow call stack to gp. */ -.macro scs_load_irq_stack tmp +.macro scs_load_irq_stack tmp tmp1 +#ifdef CONFIG_ARCH_HAS_KERNEL_SHADOW_STACK + load_per_cpu \tmp1, irq_shadow_call_stack_ptr, \tmp + li \tmp, 4096 + add \tmp, \tmp, \tmp1 + csrw CSR_SSP, \tmp +#else load_per_cpu gp, irq_shadow_call_stack_ptr, \tmp +#endif .endm =20 /* Load task_scs_sp(current) to gp. */ -.macro scs_load_current +.macro scs_load_current tmp +#ifdef CONFIG_ARCH_HAS_KERNEL_SHADOW_STACK + REG_L \tmp, TASK_TI_SCS_SP(tp) + csrw CSR_SSP, \tmp +#else REG_L gp, TASK_TI_SCS_SP(tp) +#endif .endm =20 /* Load task_scs_sp(current) to gp, but only if tp has changed. */ -.macro scs_load_current_if_task_changed prev +.macro scs_load_current_if_task_changed prev tmp beq \prev, tp, _skip_scs - scs_load_current + scs_load_current \tmp _skip_scs: .endm =20 /* Save gp to task_scs_sp(current). */ -.macro scs_save_current +.macro scs_save_current tmp +#ifdef CONFIG_ARCH_HAS_KERNEL_SHADOW_STACK + csrr \tmp, CSR_SSP + REG_S \tmp, TASK_TI_SCS_SP(tp) +#else REG_S gp, TASK_TI_SCS_SP(tp) +#endif .endm =20 #else /* CONFIG_SHADOW_CALL_STACK */ =20 .macro scs_load_init_stack .endm -.macro scs_load_irq_stack tmp +.macro scs_load_irq_stack tmp tmp1 .endm -.macro scs_load_current +.macro scs_load_current tmp .endm -.macro scs_load_current_if_task_changed prev +.macro scs_load_current_if_task_changed prev tmp .endm -.macro scs_save_current +.macro scs_save_current tmp .endm =20 #endif /* CONFIG_SHADOW_CALL_STACK */ #endif /* __ASSEMBLY__ */ =20 +#ifdef CONFIG_ARCH_HAS_KERNEL_SHADOW_STACK +#define arch_scs_store(ss_addr, magic_val) do { \ + asm volatile ("ssamoswap.d %0, %2, %1" \ + : "=3Dr" (magic_val), "+A" (*ss_addr) \ + : "r" (magic_val) \ + : "memory"); \ + } while (0) +#else +#define arch_scs_store(ss_addr, magic_val) do {} while (0) +#endif + #endif /* _ASM_SCS_H */ diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S index 3f0890b9c0b9..800a5ab763af 100644 --- a/arch/riscv/kernel/entry.S +++ b/arch/riscv/kernel/entry.S @@ -199,7 +199,7 @@ SYM_CODE_START(handle_exception) load_global_pointer =20 /* Load the kernel shadow call stack pointer if coming from userspace */ - scs_load_current_if_task_changed s5 + scs_load_current_if_task_changed s5 t0 =20 #ifdef CONFIG_RISCV_ISA_V_PREEMPTIVE move a0, sp @@ -260,7 +260,7 @@ SYM_CODE_START_NOALIGN(ret_from_exception) REG_S s0, TASK_TI_KERNEL_SP(tp) =20 /* Save the kernel shadow call stack pointer */ - scs_save_current + scs_save_current t0 =20 /* * Save TP into the scratch register , so we can find the kernel data @@ -382,8 +382,8 @@ SYM_FUNC_START(call_on_irq_stack) addi s0, sp, STACKFRAME_SIZE_ON_STACK =20 /* Switch to the per-CPU shadow call stack */ - scs_save_current - scs_load_irq_stack t0 + scs_save_current t0 + scs_load_irq_stack t0 t1 =20 /* Switch to the per-CPU IRQ stack and call the handler */ load_per_cpu t0, irq_stack_ptr, t1 @@ -393,7 +393,7 @@ SYM_FUNC_START(call_on_irq_stack) jalr a1 =20 /* Switch back to the thread shadow call stack */ - scs_load_current + scs_load_current t0 =20 /* Switch back to the thread stack and restore ra and s0 */ addi sp, s0, -STACKFRAME_SIZE_ON_STACK @@ -440,7 +440,7 @@ SYM_FUNC_START(__switch_to) REG_S s0, TASK_THREAD_SUM_RA(a3) =20 /* Save the kernel shadow call stack pointer */ - scs_save_current + scs_save_current t0 /* Restore context from next->thread */ REG_L s0, TASK_THREAD_SUM_RA(a4) li s1, SR_SUM @@ -463,7 +463,7 @@ SYM_FUNC_START(__switch_to) /* The offset of thread_info in task_struct is zero. */ move tp, a1 /* Switch to the next shadow call stack */ - scs_load_current + scs_load_current t0 ret SYM_FUNC_END(__switch_to) =20 diff --git a/arch/riscv/kernel/head.S b/arch/riscv/kernel/head.S index 59af044bf85c..366e15a9280a 100644 --- a/arch/riscv/kernel/head.S +++ b/arch/riscv/kernel/head.S @@ -184,7 +184,7 @@ secondary_start_sbi: REG_S a0, (a1) 1: #endif - scs_load_current + scs_load_current t0 =20 #if defined(CONFIG_RISCV_SBI) && defined(CONFIG_RISCV_KERNEL_CFI) li a7, SBI_EXT_FWFT @@ -367,7 +367,7 @@ SYM_CODE_START(_start_kernel) REG_S a0, (a1) 1: #endif - scs_load_current + scs_load_current t0 =20 #if defined(CONFIG_RISCV_SBI) && defined(CONFIG_RISCV_KERNEL_CFI) li a7, SBI_EXT_FWFT --=20 2.43.0