From nobody Thu Jan 1 11:01:33 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5ABADC001E0 for ; Mon, 23 Oct 2023 08:29:30 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230191AbjJWI3a (ORCPT ); Mon, 23 Oct 2023 04:29:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58060 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229492AbjJWI32 (ORCPT ); Mon, 23 Oct 2023 04:29:28 -0400 Received: from mail-pf1-x433.google.com (mail-pf1-x433.google.com [IPv6:2607:f8b0:4864:20::433]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4CD21DD for ; Mon, 23 Oct 2023 01:29:27 -0700 (PDT) Received: by mail-pf1-x433.google.com with SMTP id d2e1a72fcca58-6bd32d1a040so3015102b3a.3 for ; Mon, 23 Oct 2023 01:29:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1698049767; x=1698654567; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yC7lpeLF6zJ2U1K9d3gKHK6RO2c15uLNrRvH89Y0oYM=; b=EgA7QFtiTyRVdMGiCGCgsrhZUC5LD1B6etQWLWgwXgHdvjpSJa+aWvlsQ/7yM+0Q8C +xy6om9MYlRelRiI9CAkL+Uk0NMl9Er5BenQHs6PXthbl4v5P2mKEbSdtBu5G+uGVgo2 HqruKzYqnjweoAZG0ghGoeQ3gCanp4xy/KTRr7qjxyMVO/OW2EfR4Y5H1siQliUPBYkh 70HLB8D7mR14EJrWIKLU8BrEwEwsYlIVVq+SWqmpjj8NY9PEL85qnyECZAjMWOnjx68U Qrkzla17YSfr7nqGYOl2jlD6hjOTLuZfZkhTggK1qzfcEdT2HyMIcU3G9Yz6xZLpcPUS tuQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698049767; x=1698654567; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yC7lpeLF6zJ2U1K9d3gKHK6RO2c15uLNrRvH89Y0oYM=; b=jTzG76B4VveQz8c0/JkfMw3bHZzHfdtrYYGXtgd7GH2wnYbUhV2O9Vw531l7MijM0H FkStwogxVXG6GgGJ9rPY/ZtBaQXdjpeZO8zvKsaWvcRDymbqrbXRGJ+K1Fp3nwrMazvG 00OefDb9cJFi8Rd3QCKGFQiY0zthzL9TKYnO1cy5ljzM9slkDmhg8Y5HhWsmVTKvTlyl reei8S2pnIGg1/9BlVQc8hRJZFknFT5BDgIo3ZJqjp1Chx6vibHQeO2/O/gNbMLM9wYF VbDFFrwjY9sWynpMQtUlGc3YMl+FCexxcnNxnoGvBQJm+Og+80MrzDhhvMfiraBfAoSR I7oQ== X-Gm-Message-State: AOJu0YysldadpmPuQyXdK9UPUuu8J0GipwmA8oAqcrHqPYSS0WWbGt6p iZQy83Hzubfgfa4+HYalgHDUjg== X-Google-Smtp-Source: AGHT+IHCH1nMZJNeGDEKv3u1kEAVWCuStY+cy0vOx97pHNcDM4nqv54AkAVkcpOIZIr2dvR9THZ/jQ== X-Received: by 2002:a05:6a20:a108:b0:14c:daa9:5e22 with SMTP id q8-20020a056a20a10800b0014cdaa95e22mr10514808pzk.45.1698049766796; Mon, 23 Oct 2023 01:29:26 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id d15-20020a170903230f00b001b8b07bc600sm5415805plh.186.2023.10.23.01.29.21 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 23 Oct 2023 01:29:26 -0700 (PDT) From: Xu Lu To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, tglx@linutronix.de, maz@kernel.org, anup@brainfault.org, atishp@atishpatra.org Cc: dengliang.1214@bytedance.com, liyu.yukiteru@bytedance.com, sunjiadong.lff@bytedance.com, xieyongji@bytedance.com, lihangjing@bytedance.com, chaiwen.cc@bytedance.com, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Xu Lu Subject: [RFC 01/12] riscv: Introduce CONFIG_RISCV_PSEUDO_NMI Date: Mon, 23 Oct 2023 16:29:00 +0800 Message-Id: <20231023082911.23242-2-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20231023082911.23242-1-luxu.kernel@bytedance.com> References: <20231023082911.23242-1-luxu.kernel@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This commit introduces a new config RISCV_PSEUDO_NMI to control whether enabling the pseudo NMI feature on RISC-V. Signed-off-by: Xu Lu --- arch/riscv/Kconfig | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index d607ab0f7c6d..487e4293f31e 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -669,6 +669,16 @@ config RISCV_BOOT_SPINWAIT =20 If unsure what to do here, say N. =20 +config RISCV_PSEUDO_NMI + bool "Support for NMI-like interrupts" + depends on !RISCV_M_MODE + default n + help + Adds support for mimicking Non-Maskable Interrupts through the use of + CSR_IE register. + + If unsure, say N. + config ARCH_SUPPORTS_KEXEC def_bool MMU =20 --=20 2.20.1 From nobody Thu Jan 1 11:01:33 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D1300C001E0 for ; Mon, 23 Oct 2023 08:30:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232341AbjJWIaF (ORCPT ); Mon, 23 Oct 2023 04:30:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58086 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233042AbjJWI35 (ORCPT ); Mon, 23 Oct 2023 04:29:57 -0400 Received: from mail-pl1-x62b.google.com (mail-pl1-x62b.google.com [IPv6:2607:f8b0:4864:20::62b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 49595DA for ; Mon, 23 Oct 2023 01:29:33 -0700 (PDT) Received: by mail-pl1-x62b.google.com with SMTP id d9443c01a7336-1ca816f868fso17588505ad.1 for ; Mon, 23 Oct 2023 01:29:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1698049772; x=1698654572; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=RLEuNYLzmNdpMekkYxMIqBfr7ettCjIz89PT5TQjtwM=; b=DfWQ88JR9HltN/3MCZzOr7Y3fI0keDoSDgYyekrnXQrK4JxKk1suF7tBkLUXc6lg7t 55mUqLjtngB7yH+C+46LIwDkgiF0xEw0vkDkF3m7aISLpe6B/Y2w+LezS43R1CxOQyqL bw7BXGKTCe7JQtIq1FrhdyvgS9D3QYnMAzI/oY42TA9xvLM9dP/tleOYidyXiyzRSITv BmZT8+u9khgysnhNVvJnMqJJraAAg6reI9U7J68TLNd9/hKjOa6fHvzgzTGFWUSseqE9 xdjT5/0DDkcqTDrEa+1207a1Fl2UzAxCPblQo8PnxjbtTJTFURfBhbfS4sLISkGjF/6W GeIA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698049772; x=1698654572; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=RLEuNYLzmNdpMekkYxMIqBfr7ettCjIz89PT5TQjtwM=; b=ETe5euxh2Ik1CWv2P6Y6Ok0mSFWTMxGT0kmwnbyoL5AKOKb2IObUoJHrBAe9cY5sSi q3gbhmuXYlvw/31rHy87PhaZsVR9T+J5AxARBRKHxA/e5DTfTDNrX55YJLQPBkLWhcn4 jANaRYO93oqgI3FiS+vrN695h6ElX2C5JH2TaleFQusyaLtZKHLBk2O8W3KayjXfMao6 zUY2ihQGTzuRR3bLuX96cOSzu2u638WsMmNM5UDSxsiRas35pYvof8OgARvauR5TH3Z2 syNmdgEjU3vWvDivkCleIJApBkeLTKDrdd8xLZYJgk03SZosftiCyJZN1Jo/XJkwGR+S zbbA== X-Gm-Message-State: AOJu0Yz4deLE6Fq7k1rMGXpcilquV8lG6WJGuuUpkhN2Dbj8WUriaZaJ B6TTAYIpas84isBBDSadTywCgA== X-Google-Smtp-Source: AGHT+IFwiuZzN1VxYpGUrbYGS2JDDA5Axr6zrj45LnHXNTd5WQzI3Ka79FvuRD+C40wTzv0jOoJZUg== X-Received: by 2002:a17:902:dad1:b0:1c5:befa:d81d with SMTP id q17-20020a170902dad100b001c5befad81dmr8117421plx.10.1698049772541; Mon, 23 Oct 2023 01:29:32 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id d15-20020a170903230f00b001b8b07bc600sm5415805plh.186.2023.10.23.01.29.27 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 23 Oct 2023 01:29:32 -0700 (PDT) From: Xu Lu To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, tglx@linutronix.de, maz@kernel.org, anup@brainfault.org, atishp@atishpatra.org Cc: dengliang.1214@bytedance.com, liyu.yukiteru@bytedance.com, sunjiadong.lff@bytedance.com, xieyongji@bytedance.com, lihangjing@bytedance.com, chaiwen.cc@bytedance.com, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Xu Lu Subject: [RFC 02/12] riscv: Make CSR_IE register part of context Date: Mon, 23 Oct 2023 16:29:01 +0800 Message-Id: <20231023082911.23242-3-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20231023082911.23242-1-luxu.kernel@bytedance.com> References: <20231023082911.23242-1-luxu.kernel@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This commit makes CSR_IE register part of thread context. Kernel nowadays saves and restores irq status of each thread via CSR_STATUS register. When a thread traps into kernel, irq status of it is automatically stored in SR_PIE field of CSR_STATUS by hardware. And when kernel returns back, irq status will be automatically restored from CSR_STATUS. Things get different when we switch to CSR_IE masking for irq disabling. Hardware won't save or restore CSR_IE value during traps. In this case, when trapped into kernel, we should save CSR_IE value for previous thread manually and then clear all CSR_IE bits to disable irqs during traps. Also, we should manually restore SIE field of CSR_STATUS as we do not depends on it to disable irqs. When kernel returns back, we manually restore CSR_IE value from previous saved value. Signed-off-by: Xu Lu Signed-off-by: Hangjing Li Reviewed-by: Liang Deng Reviewed-by: Yu Li --- arch/riscv/include/asm/csr.h | 17 +++++++++++++++++ arch/riscv/include/asm/ptrace.h | 3 +++ arch/riscv/kernel/asm-offsets.c | 3 +++ arch/riscv/kernel/entry.S | 13 +++++++++++++ arch/riscv/kernel/process.c | 6 ++++++ arch/riscv/kernel/suspend_entry.S | 1 + drivers/clocksource/timer-clint.c | 4 ++++ drivers/clocksource/timer-riscv.c | 4 ++++ drivers/irqchip/irq-riscv-intc.c | 4 ++++ drivers/perf/riscv_pmu_sbi.c | 4 ++++ 10 files changed, 59 insertions(+) diff --git a/arch/riscv/include/asm/csr.h b/arch/riscv/include/asm/csr.h index 777cb8299551..6520bd826d52 100644 --- a/arch/riscv/include/asm/csr.h +++ b/arch/riscv/include/asm/csr.h @@ -7,6 +7,7 @@ #define _ASM_RISCV_CSR_H =20 #include +#include #include =20 /* Status register flags */ @@ -451,6 +452,22 @@ #define IE_TIE (_AC(0x1, UL) << RV_IRQ_TIMER) #define IE_EIE (_AC(0x1, UL) << RV_IRQ_EXT) =20 +#ifdef CONFIG_RISCV_PSEUDO_NMI +#define IRQS_ENABLED_IE (IE_SIE | IE_TIE | IE_EIE) +#define irqs_enabled_ie \ +({ \ + unsigned long __v; \ + asm (ALTERNATIVE( \ + "li %0, " __stringify(IRQS_ENABLED_IE) "\n\t" \ + "nop", \ + "li %0, " __stringify(IRQS_ENABLED_IE | SIP_LCOFIP),\ + 0, RISCV_ISA_EXT_SSCOFPMF, \ + CONFIG_RISCV_PSEUDO_NMI) \ + : "=3Dr"(__v) : : ); \ + __v; \ +}) +#endif /* CONFIG_RISCV_PSEUDO_NMI */ + #ifndef __ASSEMBLY__ =20 #define csr_swap(csr, val) \ diff --git a/arch/riscv/include/asm/ptrace.h b/arch/riscv/include/asm/ptrac= e.h index b5b0adcc85c1..b57d3a6b232f 100644 --- a/arch/riscv/include/asm/ptrace.h +++ b/arch/riscv/include/asm/ptrace.h @@ -47,6 +47,9 @@ struct pt_regs { unsigned long t6; /* Supervisor/Machine CSRs */ unsigned long status; +#ifdef CONFIG_RISCV_PSEUDO_NMI + unsigned long ie; +#endif unsigned long badaddr; unsigned long cause; /* a0 value before the syscall */ diff --git a/arch/riscv/kernel/asm-offsets.c b/arch/riscv/kernel/asm-offset= s.c index d6a75aac1d27..165f6f9fc458 100644 --- a/arch/riscv/kernel/asm-offsets.c +++ b/arch/riscv/kernel/asm-offsets.c @@ -112,6 +112,9 @@ void asm_offsets(void) OFFSET(PT_GP, pt_regs, gp); OFFSET(PT_ORIG_A0, pt_regs, orig_a0); OFFSET(PT_STATUS, pt_regs, status); +#ifdef CONFIG_RISCV_PSEUDO_NMI + OFFSET(PT_IE, pt_regs, ie); +#endif OFFSET(PT_BADADDR, pt_regs, badaddr); OFFSET(PT_CAUSE, pt_regs, cause); =20 diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S index 143a2bb3e697..19ba7c4520b9 100644 --- a/arch/riscv/kernel/entry.S +++ b/arch/riscv/kernel/entry.S @@ -65,6 +65,10 @@ _save_context: REG_S s3, PT_BADADDR(sp) REG_S s4, PT_CAUSE(sp) REG_S s5, PT_TP(sp) +#ifdef CONFIG_RISCV_PSEUDO_NMI + csrr s0, CSR_IE + REG_S s0, PT_IE(sp) +#endif /* CONFIG_RISCV_PSEUDO_NMI */ =20 /* * Set the scratch register to 0, so that if a recursive exception @@ -153,6 +157,11 @@ SYM_CODE_START_NOALIGN(ret_from_exception) csrw CSR_STATUS, a0 csrw CSR_EPC, a2 =20 +#ifdef CONFIG_RISCV_PSEUDO_NMI + REG_L s0, PT_IE(sp) + csrw CSR_IE, s0 +#endif /* CONFIG_RISCV_PSEUDO_NMI */ + REG_L x1, PT_RA(sp) REG_L x3, PT_GP(sp) REG_L x4, PT_TP(sp) @@ -251,6 +260,10 @@ restore_caller_reg: REG_S s3, PT_BADADDR(sp) REG_S s4, PT_CAUSE(sp) REG_S s5, PT_TP(sp) +#ifdef CONFIG_RISCV_PSEUDO_NMI + csrr s0, CSR_IE + REG_S s0, PT_IE(sp) +#endif /* CONFIG_RISCV_PSEUDO_NMI */ move a0, sp tail handle_bad_stack SYM_CODE_END(handle_kernel_stack_overflow) diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c index e32d737e039f..9663bae23c57 100644 --- a/arch/riscv/kernel/process.c +++ b/arch/riscv/kernel/process.c @@ -115,6 +115,9 @@ void start_thread(struct pt_regs *regs, unsigned long p= c, unsigned long sp) { regs->status =3D SR_PIE; +#ifdef CONFIG_RISCV_PSEUDO_NMI + regs->ie =3D irqs_enabled_ie; +#endif if (has_fpu()) { regs->status |=3D SR_FS_INITIAL; /* @@ -189,6 +192,9 @@ int copy_thread(struct task_struct *p, const struct ker= nel_clone_args *args) childregs->gp =3D gp_in_global; /* Supervisor/Machine, irqs on: */ childregs->status =3D SR_PP | SR_PIE; +#ifdef CONFIG_RISCV_PSEUDO_NMI + childregs->ie =3D irqs_enabled_ie; +#endif =20 p->thread.s[0] =3D (unsigned long)args->fn; p->thread.s[1] =3D (unsigned long)args->fn_arg; diff --git a/arch/riscv/kernel/suspend_entry.S b/arch/riscv/kernel/suspend_= entry.S index f7960c7c5f9e..6825f4836be4 100644 --- a/arch/riscv/kernel/suspend_entry.S +++ b/arch/riscv/kernel/suspend_entry.S @@ -47,6 +47,7 @@ ENTRY(__cpu_suspend_enter) REG_S t0, (SUSPEND_CONTEXT_REGS + PT_EPC)(a0) csrr t0, CSR_STATUS REG_S t0, (SUSPEND_CONTEXT_REGS + PT_STATUS)(a0) + /* There is no need to save CSR_IE as it is maintained in memory */ csrr t0, CSR_TVAL REG_S t0, (SUSPEND_CONTEXT_REGS + PT_BADADDR)(a0) csrr t0, CSR_CAUSE diff --git a/drivers/clocksource/timer-clint.c b/drivers/clocksource/timer-= clint.c index 9a55e733ae99..bdc10be9d3b4 100644 --- a/drivers/clocksource/timer-clint.c +++ b/drivers/clocksource/timer-clint.c @@ -114,7 +114,9 @@ static int clint_clock_next_event(unsigned long delta, void __iomem *r =3D clint_timer_cmp + cpuid_to_hartid_map(smp_processor_id()); =20 +#ifndef CONFIG_RISCV_PSEUDO_NMI csr_set(CSR_IE, IE_TIE); +#endif writeq_relaxed(clint_get_cycles64() + delta, r); return 0; } @@ -155,7 +157,9 @@ static irqreturn_t clint_timer_interrupt(int irq, void = *dev_id) { struct clock_event_device *evdev =3D this_cpu_ptr(&clint_clock_event); =20 +#ifndef CONFIG_RISCV_PSEUDO_NMI csr_clear(CSR_IE, IE_TIE); +#endif evdev->event_handler(evdev); =20 return IRQ_HANDLED; diff --git a/drivers/clocksource/timer-riscv.c b/drivers/clocksource/timer-= riscv.c index da3071b387eb..b730e01a7f02 100644 --- a/drivers/clocksource/timer-riscv.c +++ b/drivers/clocksource/timer-riscv.c @@ -36,7 +36,9 @@ static int riscv_clock_next_event(unsigned long delta, { u64 next_tval =3D get_cycles64() + delta; =20 +#ifndef CONFIG_RISCV_PSEUDO_NMI csr_set(CSR_IE, IE_TIE); +#endif if (static_branch_likely(&riscv_sstc_available)) { #if defined(CONFIG_32BIT) csr_write(CSR_STIMECMP, next_tval & 0xFFFFFFFF); @@ -119,7 +121,9 @@ static irqreturn_t riscv_timer_interrupt(int irq, void = *dev_id) { struct clock_event_device *evdev =3D this_cpu_ptr(&riscv_clock_event); =20 +#ifndef CONFIG_RISCV_PSEUDO_NMI csr_clear(CSR_IE, IE_TIE); +#endif evdev->event_handler(evdev); =20 return IRQ_HANDLED; diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-i= ntc.c index e8d01b14ccdd..7fad1ba37e5c 100644 --- a/drivers/irqchip/irq-riscv-intc.c +++ b/drivers/irqchip/irq-riscv-intc.c @@ -39,12 +39,16 @@ static asmlinkage void riscv_intc_irq(struct pt_regs *r= egs) =20 static void riscv_intc_irq_mask(struct irq_data *d) { +#ifndef CONFIG_RISCV_PSEUDO_NMI csr_clear(CSR_IE, BIT(d->hwirq)); +#endif } =20 static void riscv_intc_irq_unmask(struct irq_data *d) { +#ifndef CONFIG_RISCV_PSEUDO_NMI csr_set(CSR_IE, BIT(d->hwirq)); +#endif } =20 static void riscv_intc_irq_eoi(struct irq_data *d) diff --git a/drivers/perf/riscv_pmu_sbi.c b/drivers/perf/riscv_pmu_sbi.c index 96c7f670c8f0..995b501ec721 100644 --- a/drivers/perf/riscv_pmu_sbi.c +++ b/drivers/perf/riscv_pmu_sbi.c @@ -778,7 +778,9 @@ static int pmu_sbi_starting_cpu(unsigned int cpu, struc= t hlist_node *node) if (riscv_pmu_use_irq) { cpu_hw_evt->irq =3D riscv_pmu_irq; csr_clear(CSR_IP, BIT(riscv_pmu_irq_num)); +#ifndef CONFIG_RISCV_PSEUDO_NMI csr_set(CSR_IE, BIT(riscv_pmu_irq_num)); +#endif enable_percpu_irq(riscv_pmu_irq, IRQ_TYPE_NONE); } =20 @@ -789,7 +791,9 @@ static int pmu_sbi_dying_cpu(unsigned int cpu, struct h= list_node *node) { if (riscv_pmu_use_irq) { disable_percpu_irq(riscv_pmu_irq); +#ifndef CONFIG_RISCV_PSEUDO_NMI csr_clear(CSR_IE, BIT(riscv_pmu_irq_num)); +#endif } =20 /* Disable all counters access for user mode now */ --=20 2.20.1 From nobody Thu Jan 1 11:01:33 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id ADCA3CDB474 for ; Mon, 23 Oct 2023 08:29:42 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232974AbjJWI3m (ORCPT ); Mon, 23 Oct 2023 04:29:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58074 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229868AbjJWI3k (ORCPT ); Mon, 23 Oct 2023 04:29:40 -0400 Received: from mail-pf1-x429.google.com (mail-pf1-x429.google.com [IPv6:2607:f8b0:4864:20::429]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BFD21DD for ; Mon, 23 Oct 2023 01:29:38 -0700 (PDT) Received: by mail-pf1-x429.google.com with SMTP id d2e1a72fcca58-6bd32d1a040so3015197b3a.3 for ; Mon, 23 Oct 2023 01:29:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1698049778; x=1698654578; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=at7wi3me+Z61wYFd7J2d3j74V70AFyRlLZ+mJARIjAE=; b=jolbgEvqeuI6JHbIiGOBuLMlL1/o93PhR8i2WR023e0Xd9zQOpaJUdD/NZXBHxb9wF /lqv7hExIXx+nQJjLQX/6Uv3TXE4JhRObxZFHRqKHWx2UWLNJVLUUuIvqbXN5Dz71wvw 6t2VJGMSF8MCZtbzYoIj64xjTpGcMemFcjBJS+LZHOxNQt6ouDqGi3VQC+/HVQjejvb7 SMsUKV/+6gJDOktDVtxSyQA2ST+HwW6PgHjZs5ZWMdextMgdTpl7FiEy7lL+8vup6ttx v6Oy/RZjWl8bwGkdkkDssV9tXIbKQlV62XZIw6Cfg8qkrtAF/42bi38eTxGCXDV+/Cpa Fzag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698049778; x=1698654578; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=at7wi3me+Z61wYFd7J2d3j74V70AFyRlLZ+mJARIjAE=; b=Esja8U4FG07Tk9rFMR8AdAyCcroPybmzXvCv1Sq7CLO3Dga19Rhxyovg+oBg+UezKv 3zaLC88p0UZBQAscBwiAKflxqZXyrrwVge4h6tjP6G+RFjINPL9efoEEe8KhSy6NiUz5 aTi3SkSTDevkRMwRv9TSlFOA2tOKnQb5K3xbZgSgnpfSZECpcSBs1uDrcE0tP9xnXszC xshT7AjbEu5HALxDyAOuyGN+gJFAPrvuapYCLrpLOBLlqWUYqWkxnMFE+K8LHuUNl5AW Kfw/ACeJNlq4SqYFIRAgVje/qFsg4H07GZyPsXXRCm82cNCBz16mDvpsd6+m/Q08+bqS p1Cg== X-Gm-Message-State: AOJu0YwFxQD6JF9IuEFN9QAFIh0HfJ1b/KyEEUCebsnF10OSDg4v17mA L7Tal9tlRH3/OIlZVzxxwAubCg== X-Google-Smtp-Source: AGHT+IHAxAXPREhEH68MgYTJlOqTtsbSB0gFuNM7voWNCv0htKF7/ukJX/oxNlGp1Ca/X+qqi24MCg== X-Received: by 2002:a05:6a20:7489:b0:16b:afc2:3b68 with SMTP id p9-20020a056a20748900b0016bafc23b68mr10853205pzd.25.1698049778199; Mon, 23 Oct 2023 01:29:38 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id d15-20020a170903230f00b001b8b07bc600sm5415805plh.186.2023.10.23.01.29.32 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 23 Oct 2023 01:29:37 -0700 (PDT) From: Xu Lu To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, tglx@linutronix.de, maz@kernel.org, anup@brainfault.org, atishp@atishpatra.org Cc: dengliang.1214@bytedance.com, liyu.yukiteru@bytedance.com, sunjiadong.lff@bytedance.com, xieyongji@bytedance.com, lihangjing@bytedance.com, chaiwen.cc@bytedance.com, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Xu Lu Subject: [RFC 03/12] riscv: Switch to CSR_IE masking when disabling irqs Date: Mon, 23 Oct 2023 16:29:02 +0800 Message-Id: <20231023082911.23242-4-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20231023082911.23242-1-luxu.kernel@bytedance.com> References: <20231023082911.23242-1-luxu.kernel@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This commit switch the way of disabling irqs to CSR_IE masking. After CSR_IE has been made a part of context, now we can safely switch to CSR_IE masking when disabling irqs. Signed-off-by: Xu Lu Signed-off-by: Hangjing Li Reviewed-by: Liang Deng Reviewed-by: Yu Li --- arch/riscv/include/asm/irqflags.h | 58 +++++++++++++++++++++++++++++++ arch/riscv/include/asm/ptrace.h | 4 +++ arch/riscv/kernel/entry.S | 7 +++- arch/riscv/kernel/head.S | 10 ++++++ 4 files changed, 78 insertions(+), 1 deletion(-) diff --git a/arch/riscv/include/asm/irqflags.h b/arch/riscv/include/asm/irq= flags.h index 08d4d6a5b7e9..e0ff37315178 100644 --- a/arch/riscv/include/asm/irqflags.h +++ b/arch/riscv/include/asm/irqflags.h @@ -10,6 +10,62 @@ #include #include =20 +#ifdef CONFIG_RISCV_PSEUDO_NMI + +static inline void local_irq_switch_on(void) +{ + csr_set(CSR_STATUS, SR_IE); +} + +static inline void local_irq_switch_off(void) +{ + csr_clear(CSR_STATUS, SR_IE); +} + +/* read interrupt enabled status */ +static inline unsigned long arch_local_save_flags(void) +{ + return csr_read(CSR_IE); +} + +/* unconditionally enable interrupts */ +static inline void arch_local_irq_enable(void) +{ + csr_set(CSR_IE, irqs_enabled_ie); +} + +/* unconditionally disable interrupts */ +static inline void arch_local_irq_disable(void) +{ + csr_clear(CSR_IE, irqs_enabled_ie); +} + +/* get status and disable interrupts */ +static inline unsigned long arch_local_irq_save(void) +{ + return csr_read_clear(CSR_IE, irqs_enabled_ie); +} + +/* test flags */ +static inline int arch_irqs_disabled_flags(unsigned long flags) +{ + return (flags !=3D irqs_enabled_ie); +} + +/* test hardware interrupt enable bit */ +static inline int arch_irqs_disabled(void) +{ + return arch_irqs_disabled_flags(arch_local_save_flags()); +} + +/* set interrupt enabled status */ +static inline void arch_local_irq_restore(unsigned long flags) +{ + csr_write(CSR_IE, flags); +} + +#else /* CONFIG_RISCV_PSEUDO_NMI */ + /* read interrupt enabled status */ static inline unsigned long arch_local_save_flags(void) { @@ -52,4 +108,6 @@ static inline void arch_local_irq_restore(unsigned long = flags) csr_set(CSR_STATUS, flags & SR_IE); } =20 +#endif /* !CONFIG_RISCV_PSEUDO_NMI */ + #endif /* _ASM_RISCV_IRQFLAGS_H */ diff --git a/arch/riscv/include/asm/ptrace.h b/arch/riscv/include/asm/ptrac= e.h index b57d3a6b232f..e552e7fb46f3 100644 --- a/arch/riscv/include/asm/ptrace.h +++ b/arch/riscv/include/asm/ptrace.h @@ -178,7 +178,11 @@ static inline unsigned long regs_get_kernel_argument(s= truct pt_regs *regs, =20 static inline int regs_irqs_disabled(struct pt_regs *regs) { +#ifdef CONFIG_RISCV_PSEUDO_NMI + return (regs->ie !=3D irqs_enabled_ie); +#else return !(regs->status & SR_PIE); +#endif } =20 #endif /* __ASSEMBLY__ */ diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S index 19ba7c4520b9..d1f28dab02f7 100644 --- a/arch/riscv/kernel/entry.S +++ b/arch/riscv/kernel/entry.S @@ -66,8 +66,13 @@ _save_context: REG_S s4, PT_CAUSE(sp) REG_S s5, PT_TP(sp) #ifdef CONFIG_RISCV_PSEUDO_NMI - csrr s0, CSR_IE + csrrw s0, CSR_IE, x0 REG_S s0, PT_IE(sp) + andi s1, s1, SR_PIE + beqz s1, 1f + li s1, SR_IE + csrs CSR_STATUS, s1 +1: #endif /* CONFIG_RISCV_PSEUDO_NMI */ =20 /* diff --git a/arch/riscv/kernel/head.S b/arch/riscv/kernel/head.S index 3710ea5d160f..4f9446defacd 100644 --- a/arch/riscv/kernel/head.S +++ b/arch/riscv/kernel/head.S @@ -169,6 +169,10 @@ secondary_start_sbi: call relocate_enable_mmu #endif call setup_trap_vector +#ifdef CONFIG_RISCV_PSEUDO_NMI + li t0, SR_IE + csrs CSR_STATUS, t0 +#endif tail smp_callin #endif /* CONFIG_SMP */ =20 @@ -320,6 +324,12 @@ clear_bss_done: #ifdef CONFIG_KASAN call kasan_early_init #endif + +#ifdef CONFIG_RISCV_PSEUDO_NMI + li t0, SR_IE + csrs CSR_STATUS, t0 +#endif + /* Start the kernel */ call soc_early_init tail start_kernel --=20 2.20.1 From nobody Thu Jan 1 11:01:33 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6325C001E0 for ; Mon, 23 Oct 2023 08:30:22 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230097AbjJWIaV (ORCPT ); Mon, 23 Oct 2023 04:30:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37596 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233030AbjJWIaI (ORCPT ); Mon, 23 Oct 2023 04:30:08 -0400 Received: from mail-pf1-x42b.google.com (mail-pf1-x42b.google.com [IPv6:2607:f8b0:4864:20::42b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 56B4BD78 for ; Mon, 23 Oct 2023 01:29:44 -0700 (PDT) Received: by mail-pf1-x42b.google.com with SMTP id d2e1a72fcca58-6ba172c5f3dso2364945b3a.0 for ; Mon, 23 Oct 2023 01:29:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1698049784; x=1698654584; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bTPdJw2kGsvWjCGGMUBG2xz20Ji8Kpf17R3X8M8D7Aw=; b=QXrBeRI9/K2u7fFuEiXyWhP+s+WDdIkIDNK7dgyOyREi7X/AaaFR5ZxO59sCIK1Be7 Hfg3pqCjVLhgx0fEYu6As3sAciDBzXuQrxwSR/RBKYwrEJOX7uPyozsjX/w3qPivaLoh Gu7FbhsbtVvYjV1bETgjQhzEGooawk4MrbmPh3ZeujuRuvvbb/6x4VwF1DvsNoUtPkW8 QvExDqWKO6QeUSMU9V2kTRRhBAabH6QUaabugN6ZhkcG6SSMA1ji9g9y7DJeHRB66Mee PSYa6t5j6d323VNRb/gg46biF2cL6CXMMRl2RFsptajj6DHo8vmwKMedhIPN9kSQIMba Ltfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698049784; x=1698654584; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bTPdJw2kGsvWjCGGMUBG2xz20Ji8Kpf17R3X8M8D7Aw=; b=gitXaW1cLgpjpWc0NeLY7z8etkWB3ItUNFAgjg9o2IFMLw0hfEF1Ni8LoGiLNpnE6u jNLW4epXAZGq8pA/Ft6PyCu6s+ugTKpKT9vR4kbI1ZdeTz13iNBNm9JvTNU5+m4/Mydf 5m94Af+AvG9iXro4Fq5EzEXye9vvIRCH9lSKk9NurMiiUSW6HUNA29LXMLpmRUNlDyvC yWfggeSjCc4DXAmoEn8ZEy3pSP28mFxRFrYH/oxSit+j1TAfsdQ5N58biuN+21KYz1LG 8sih9c4YBPBVW99Y95JQczdSexOdJX9Cn4Sol3ESwAmi47zEjt6s7kV/YVufrPtXi1PT Dxjw== X-Gm-Message-State: AOJu0Yz0QVI9h96brbS7Ikz3SuFmwJc159u2ytwvARrhg34P3CKUGLBg e60K6gGXLV5FeS7qsVreH43TIw== X-Google-Smtp-Source: AGHT+IHy0ee6CNfNaI0e1G2cSIdQi3bfs3nWqXWt1dZTbNcZKi88wQbICOA1lqr6IbE3K2mo4LQppQ== X-Received: by 2002:a05:6a21:3398:b0:174:1466:359c with SMTP id yy24-20020a056a21339800b001741466359cmr6820626pzb.61.1698049783818; Mon, 23 Oct 2023 01:29:43 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id d15-20020a170903230f00b001b8b07bc600sm5415805plh.186.2023.10.23.01.29.38 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 23 Oct 2023 01:29:43 -0700 (PDT) From: Xu Lu To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, tglx@linutronix.de, maz@kernel.org, anup@brainfault.org, atishp@atishpatra.org Cc: dengliang.1214@bytedance.com, liyu.yukiteru@bytedance.com, sunjiadong.lff@bytedance.com, xieyongji@bytedance.com, lihangjing@bytedance.com, chaiwen.cc@bytedance.com, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Xu Lu Subject: [RFC 04/12] riscv: Switch back to CSR_STATUS masking when going idle Date: Mon, 23 Oct 2023 16:29:03 +0800 Message-Id: <20231023082911.23242-5-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20231023082911.23242-1-luxu.kernel@bytedance.com> References: <20231023082911.23242-1-luxu.kernel@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The WFI instruction makes current core stall until interrupt happens. In WFI's implementation, core can only be waken up from interrupt which is both pending in CSR_IP and enabled in CSR_IE. After we switch to CSR_IE masking for irq disabling, WFI instruction can never resume execution if CSR_IE is masked. This commit handles this special case. When WFI instruction is called with CSR_IE masked, we unmask CSR_IE first and disable irqs in traditional CSR_STATUS way instead. Signed-off-by: Xu Lu --- arch/riscv/include/asm/processor.h | 4 ++++ arch/riscv/kernel/irq.c | 17 +++++++++++++++++ 2 files changed, 21 insertions(+) diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/pr= ocessor.h index 3e23e1786d05..ab9b2b974979 100644 --- a/arch/riscv/include/asm/processor.h +++ b/arch/riscv/include/asm/processor.h @@ -111,10 +111,14 @@ extern void start_thread(struct pt_regs *regs, extern unsigned long __get_wchan(struct task_struct *p); =20 =20 +#ifndef CONFIG_RISCV_PSEUDO_NMI static inline void wait_for_interrupt(void) { __asm__ __volatile__ ("wfi"); } +#else +void wait_for_interrupt(void); +#endif =20 struct device_node; int riscv_of_processor_hartid(struct device_node *node, unsigned long *har= tid); diff --git a/arch/riscv/kernel/irq.c b/arch/riscv/kernel/irq.c index 9cc0a7669271..e7dfd68e9ca3 100644 --- a/arch/riscv/kernel/irq.c +++ b/arch/riscv/kernel/irq.c @@ -15,6 +15,23 @@ #include #include =20 +#ifdef CONFIG_RISCV_PSEUDO_NMI + +void wait_for_interrupt(void) +{ + if (irqs_disabled()) { + local_irq_switch_off(); + local_irq_enable(); + __asm__ __volatile__ ("wfi"); + local_irq_disable(); + local_irq_switch_on(); + } else { + __asm__ __volatile__ ("wfi"); + } +} + +#endif /* CONFIG_RISCV_PSEUDO_NMI */ + static struct fwnode_handle *(*__get_intc_node)(void); =20 void riscv_set_intc_hwnode_fn(struct fwnode_handle *(*fn)(void)) --=20 2.20.1 From nobody Thu Jan 1 11:01:33 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84DEDC001E0 for ; Mon, 23 Oct 2023 08:29:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233037AbjJWI35 (ORCPT ); Mon, 23 Oct 2023 04:29:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37610 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232982AbjJWI3v (ORCPT ); Mon, 23 Oct 2023 04:29:51 -0400 Received: from mail-pl1-x62e.google.com (mail-pl1-x62e.google.com [IPv6:2607:f8b0:4864:20::62e]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1F23CD68 for ; Mon, 23 Oct 2023 01:29:50 -0700 (PDT) Received: by mail-pl1-x62e.google.com with SMTP id d9443c01a7336-1bdf4752c3cso15992645ad.2 for ; Mon, 23 Oct 2023 01:29:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1698049789; x=1698654589; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=4WLOSQOA5Tm0Adw6Ut8ujQkn4XXYrNjbdXHKCGaB3LM=; b=iLxR9CpPHj5MF5lOEEwlXsnK9mVa74W1k3vTSLD7gZlk6/+nytiotJkBdq1fclkXTO ndO4S8Ys9lefTYja8AhjD2oUy4fizVh2ATwxMEtEi64rwKKA/1wwnmLgX6ub2uwgVjXJ V6x8BKU2UU77QDAl3SRrTP+yPewkIixiXSFjq+Sgl0twKDPXVdXrkKjQekfLcPIcBQSJ QUL4Dfp7E6cqOPO6XMk/CuIhsB5Llo7CcyM5+LNA+L84tbW1a7erxFzPvljqAjqsTFMY kWHcJWdgJ2RZDK/ONbKSSY0OZIJAmSm3EYOgESFNezsJ4r4GeDZkY32KRLaZJ5h+QNud QAlQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698049789; x=1698654589; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4WLOSQOA5Tm0Adw6Ut8ujQkn4XXYrNjbdXHKCGaB3LM=; b=G2Tqt395QWhPS+PDBtx8bPqZk+1D+sanX7fWf8/egX1MVSEpk4/enLmMaQYP24xIgP akZn40PAgVP0/745/FyljFVep3A2bCPDnI8qskAsOfpvIOmbOPTmGe4rW1qnWbuy9yCu SEXmQRhlChb3l4XvV4q+hemEZ2ofC95G5ILkjiAQpqRymCnCMIOv9XNHG41mVJUnLk47 SN50V5OutgVKeIBYdVchFfui7c+JBMEDmFtkSAGK4nnEnNXBkckeF8wckmqX0P4RZ+nx h5gfQeBbF0p30nQ957n/H4OluxicArSUvfqMC0FBBIkWoIwzCrUuVC1wTMhhD+ktSJO5 +WJg== X-Gm-Message-State: AOJu0YwYXHXW5dNhOZoNfcQAQOXvDLbrUUUUrV/2hyOZjUzB8zbF1h2Q 7av7ceczofkvRan61HhtBpWKnw== X-Google-Smtp-Source: AGHT+IHnm6GiYKkee1BBGlEg9/2GVy9OXofYG3BGHLaj+6z7TyuT5to3Ev1GjwdsT/XgsL2yDJJgGg== X-Received: by 2002:a17:902:e548:b0:1c9:b187:4d84 with SMTP id n8-20020a170902e54800b001c9b1874d84mr8674687plf.14.1698049789506; Mon, 23 Oct 2023 01:29:49 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id d15-20020a170903230f00b001b8b07bc600sm5415805plh.186.2023.10.23.01.29.44 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 23 Oct 2023 01:29:49 -0700 (PDT) From: Xu Lu To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, tglx@linutronix.de, maz@kernel.org, anup@brainfault.org, atishp@atishpatra.org Cc: dengliang.1214@bytedance.com, liyu.yukiteru@bytedance.com, sunjiadong.lff@bytedance.com, xieyongji@bytedance.com, lihangjing@bytedance.com, chaiwen.cc@bytedance.com, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Xu Lu Subject: [RFC 05/12] riscv: kvm: Switch back to CSR_STATUS masking when entering guest Date: Mon, 23 Oct 2023 16:29:04 +0800 Message-Id: <20231023082911.23242-6-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20231023082911.23242-1-luxu.kernel@bytedance.com> References: <20231023082911.23242-1-luxu.kernel@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" When kvm enters vcpu, it first disables local irqs before preparing vcpu context and uses SRET instruction to enter guest mode after vcpu context is ready, which automatically restores guest's irq status. However, after we switch to CSR_IE masking for interrupt disabling, the SRET instruction itself can not restore guest's irq status correctly as interrupts are still masked by CSR_IE. This commit handles this special case by switching to traditional CSR_STATUS way to disable irqs before entering guest mode. Signed-off-by: Xu Lu --- arch/riscv/include/asm/irqflags.h | 3 +++ arch/riscv/kvm/vcpu.c | 18 +++++++++++++----- 2 files changed, 16 insertions(+), 5 deletions(-) diff --git a/arch/riscv/include/asm/irqflags.h b/arch/riscv/include/asm/irq= flags.h index e0ff37315178..60c19f8b57f0 100644 --- a/arch/riscv/include/asm/irqflags.h +++ b/arch/riscv/include/asm/irqflags.h @@ -64,6 +64,9 @@ static inline void arch_local_irq_restore(unsigned long f= lags) csr_write(CSR_IE, flags); } =20 +#define local_irq_enable_vcpu_run local_irq_switch_on +#define local_irq_disable_vcpu_run local_irq_switch_off + #else /* CONFIG_RISCV_PSEUDO_NMI */ =20 /* read interrupt enabled status */ diff --git a/arch/riscv/kvm/vcpu.c b/arch/riscv/kvm/vcpu.c index 82229db1ce73..233408247da7 100644 --- a/arch/riscv/kvm/vcpu.c +++ b/arch/riscv/kvm/vcpu.c @@ -621,6 +621,14 @@ static void noinstr kvm_riscv_vcpu_enter_exit(struct k= vm_vcpu *vcpu) guest_state_exit_irqoff(); } =20 +#ifndef local_irq_enable_vcpu_run +#define local_irq_enable_vcpu_run local_irq_enable +#endif + +#ifndef local_irq_disable_vcpu_run +#define local_irq_disable_vcpu_run local_irq_disable +#endif + int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) { int ret; @@ -685,7 +693,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) continue; } =20 - local_irq_disable(); + local_irq_disable_vcpu_run(); =20 /* * Ensure we set mode to IN_GUEST_MODE after we disable @@ -712,7 +720,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) kvm_request_pending(vcpu) || xfer_to_guest_mode_work_pending()) { vcpu->mode =3D OUTSIDE_GUEST_MODE; - local_irq_enable(); + local_irq_enable_vcpu_run(); preempt_enable(); kvm_vcpu_srcu_read_lock(vcpu); continue; @@ -757,12 +765,12 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) * recognised, so we just hope that the CPU takes any pending * interrupts between the enable and disable. */ - local_irq_enable(); - local_irq_disable(); + local_irq_enable_vcpu_run(); + local_irq_disable_vcpu_run(); =20 guest_timing_exit_irqoff(); =20 - local_irq_enable(); + local_irq_enable_vcpu_run(); =20 preempt_enable(); =20 --=20 2.20.1 From nobody Thu Jan 1 11:01:33 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F1845CDB474 for ; Mon, 23 Oct 2023 08:30:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233060AbjJWIaM (ORCPT ); Mon, 23 Oct 2023 04:30:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35390 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233058AbjJWI35 (ORCPT ); Mon, 23 Oct 2023 04:29:57 -0400 Received: from mail-pf1-x436.google.com (mail-pf1-x436.google.com [IPv6:2607:f8b0:4864:20::436]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id B762CD7B for ; Mon, 23 Oct 2023 01:29:55 -0700 (PDT) Received: by mail-pf1-x436.google.com with SMTP id d2e1a72fcca58-6b36e1fcee9so2386712b3a.3 for ; Mon, 23 Oct 2023 01:29:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1698049795; x=1698654595; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=AUO1/BVdvM4H5C6LBNItdP0IIJipRGNxmDV7eD7nopg=; b=eIYgUBkXxqcUP0Fa4z7vj8CIE5bLNxOer2Zrmzreg36NkvNydrY5vMC1ckvF/Ei60M 0cmk2xdY6RkcYFLQ4QG4z9dO9cNwvm/UKD1Wojx7RSkgNDTJ2phfJ8JoBwfIGd7xL4W7 K1L8g/zVyGF3JT7EbxnhYabXmBv40ESR4rqqcf8ffNLbUTKHrsjW9fo72FGc31s50fTR x07UFMAMOMe9N1VrgS3RcdypsifHCmUvus/kaEGh7q3EFhTY8vj+14yMUTas77+PaqZJ Iij0X4vIlEjVECHAFjLDqVGbdgqWgE26Uvg8ZItAlDWiA6SlYslVhw5qhdDq0Ci/XtYZ PL6w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698049795; x=1698654595; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AUO1/BVdvM4H5C6LBNItdP0IIJipRGNxmDV7eD7nopg=; b=eOG3RQnWcde32Y9SWVSREX51lxF3U92XdFmLyPu3NqvqbVcjEy1W4VQ2Xoz7FjK62d XZD+Nk4AvvjxI+L9U+WlT5Z2Zt+Q4/IYfNPp4zPfqtREYHsm3odnf+Nl+U1yOegzlNpI Bjy1K8R+xKVA/CgYWC6lAA/dsRqBQvm1xg3xL3uARy4NmVIar0u7TtSihK9INmAIT1jD Xl/iPEf+IDHi7p+3DgjcpeNSoEBXa9Mr2ScgN0TQGunSKKnQP83+j8NZNJsPx+dQ/PZF b2bEFcNR4E5UT+UMv9VnmhoNcuG0P61Dk0X1nRoIJSOfzF9qcBMM1V6g3SnoGyF7fZaG yYkA== X-Gm-Message-State: AOJu0YzlYWwmrGSTxK7F8YkcMVKm+weIiHIV8tTKx7/jShuoZm4ZjqW1 bQ47tdsQ1nF6d3IBRhemtPdO5g== X-Google-Smtp-Source: AGHT+IEuewyI9+tw5QJUm2oN6guHecDPjWsmm8bO+1FZNslxYqHTr+PcfG3+59e4kYZAI8n0lGmkLw== X-Received: by 2002:a05:6a20:2615:b0:151:7d4c:899c with SMTP id i21-20020a056a20261500b001517d4c899cmr4935811pze.25.1698049795184; Mon, 23 Oct 2023 01:29:55 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id d15-20020a170903230f00b001b8b07bc600sm5415805plh.186.2023.10.23.01.29.49 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 23 Oct 2023 01:29:54 -0700 (PDT) From: Xu Lu To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, tglx@linutronix.de, maz@kernel.org, anup@brainfault.org, atishp@atishpatra.org Cc: dengliang.1214@bytedance.com, liyu.yukiteru@bytedance.com, sunjiadong.lff@bytedance.com, xieyongji@bytedance.com, lihangjing@bytedance.com, chaiwen.cc@bytedance.com, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Xu Lu Subject: [RFC 06/12] riscv: Allow requesting irq as pseudo NMI Date: Mon, 23 Oct 2023 16:29:05 +0800 Message-Id: <20231023082911.23242-7-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20231023082911.23242-1-luxu.kernel@bytedance.com> References: <20231023082911.23242-1-luxu.kernel@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This commit implements pseudo NMI callbacks for riscv_intc_irq chip. We use an immediate macro to denote NMIs of each cpu. Each bit of it represents an irq. Bit 1 means corresponding irq is registered as NMI while bit 0 means not. Signed-off-by: Xu Lu Signed-off-by: Hangjing Li Reviewed-by: Liang Deng Reviewed-by: Yu Li --- arch/riscv/include/asm/irqflags.h | 17 ++++++++++++++ drivers/irqchip/irq-riscv-intc.c | 38 +++++++++++++++++++++++++++++++ 2 files changed, 55 insertions(+) diff --git a/arch/riscv/include/asm/irqflags.h b/arch/riscv/include/asm/irq= flags.h index 60c19f8b57f0..9700a17a003a 100644 --- a/arch/riscv/include/asm/irqflags.h +++ b/arch/riscv/include/asm/irqflags.h @@ -12,6 +12,23 @@ =20 #ifdef CONFIG_RISCV_PSEUDO_NMI =20 +#define __ALLOWED_NMI_MASK 0 +#define ALLOWED_NMI_MASK (__ALLOWED_NMI_MASK & irqs_enabled_ie) + +static inline bool nmi_allowed(int irq) +{ + return (BIT(irq) & ALLOWED_NMI_MASK); +} + +static inline bool is_nmi(int irq) +{ + return (BIT(irq) & ALLOWED_NMI_MASK); +} + +static inline void set_nmi(int irq) {} + +static inline void unset_nmi(int irq) {} + static inline void local_irq_switch_on(void) { csr_set(CSR_STATUS, SR_IE); diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-i= ntc.c index 7fad1ba37e5c..83a0a744fce6 100644 --- a/drivers/irqchip/irq-riscv-intc.c +++ b/drivers/irqchip/irq-riscv-intc.c @@ -67,11 +67,49 @@ static void riscv_intc_irq_eoi(struct irq_data *d) */ } =20 +#ifdef CONFIG_RISCV_PSEUDO_NMI + +static int riscv_intc_irq_nmi_setup(struct irq_data *d) +{ + unsigned int hwirq =3D d->hwirq; + struct irq_desc *desc =3D irq_to_desc(d->irq); + + if (WARN_ON((hwirq >=3D BITS_PER_LONG) || !nmi_allowed(hwirq))) + return -EINVAL; + + desc->handle_irq =3D handle_percpu_devid_fasteoi_nmi; + set_nmi(hwirq); + + return 0; +} + +static void riscv_intc_irq_nmi_teardown(struct irq_data *d) +{ + unsigned int hwirq =3D d->hwirq; + struct irq_desc *desc =3D irq_to_desc(d->irq); + + if (WARN_ON(hwirq >=3D BITS_PER_LONG)) + return; + + if (WARN_ON(!is_nmi(hwirq))) + return; + + desc->handle_irq =3D handle_percpu_devid_irq; + unset_nmi(hwirq); +} + +#endif /* CONFIG_RISCV_PSEUDO_NMI */ + static struct irq_chip riscv_intc_chip =3D { .name =3D "RISC-V INTC", .irq_mask =3D riscv_intc_irq_mask, .irq_unmask =3D riscv_intc_irq_unmask, .irq_eoi =3D riscv_intc_irq_eoi, +#ifdef CONFIG_RISCV_PSEUDO_NMI + .irq_nmi_setup =3D riscv_intc_irq_nmi_setup, + .irq_nmi_teardown =3D riscv_intc_irq_nmi_teardown, + .flags =3D IRQCHIP_SUPPORTS_NMI, +#endif }; =20 static int riscv_intc_domain_map(struct irq_domain *d, unsigned int irq, --=20 2.20.1 From nobody Thu Jan 1 11:01:33 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0BCE0C001E0 for ; Mon, 23 Oct 2023 08:30:48 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229492AbjJWIar (ORCPT ); Mon, 23 Oct 2023 04:30:47 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37678 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233202AbjJWIaa (ORCPT ); Mon, 23 Oct 2023 04:30:30 -0400 Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com [IPv6:2607:f8b0:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4F43CD7D for ; Mon, 23 Oct 2023 01:30:04 -0700 (PDT) Received: by mail-pl1-x631.google.com with SMTP id d9443c01a7336-1c8a1541232so25448305ad.0 for ; Mon, 23 Oct 2023 01:30:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1698049801; x=1698654601; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=KDj8p8VlOUWmt6ePZtJj52Ju7p2Y1636xaXynBRmH4M=; b=ESLob1UbjBeQXWmATGlGDVFX/ccJq/psY6rKeSPpA5LQihEo8YeLQ2FFAjwkd+VUaw IwAVoLM51QGFiGje6BpmxP4gHZBXyYZ2HFrQmWyiTYJZx9PvHYex7AMKKesIGwi4JDrK HLwT2vhrf1ognNZ/vRWda4g56Ide3aPgJMiaStUTt/Q3Id5T7hWGmqDfFg+QOGxtDj3B IynOaUV03m2vtY943kZctQ/GfLjxWUAU4aUe51V/NxQNhsHTlr2aSk50s+bZjTAIUPDk 2J1izRBehLaBLMV4cfZLxWX6682m6qDZ/gtd3LmkUHEFXeP3MYy66gTS5bFWL8cvBxYl M3Og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698049801; x=1698654601; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=KDj8p8VlOUWmt6ePZtJj52Ju7p2Y1636xaXynBRmH4M=; b=umF16mmFxUbnA0UdABWz3UOgj2BQ9nANr0CaEpRxMeMGWoaKUbqvl+OLz8uDXKHwHM /wJ20truLDvLbREgfKIdTIC+SKj6bhbzZkubMS6hayGkw1I5SBIE90+U4h69/Ue7zwaR RgYYMpcrdpP9TVWiCNRr/k2orsyjv0ajjTMRY5mZizzqiY3aop4/OkocVThoW3hMhKOH pvQA4Ivj5ddPYmrsdVsFKdFvUYVA/eHevJLymjZocGTBeFMDlxRuSBN/lYnejt0NARLl Lh4/oVA2llQvKG3/gk/olSzG+Vc/kxl2qfGT5xPt1uhcC4T91/ZO7zBBbomaNfytuiYT OQuA== X-Gm-Message-State: AOJu0YxWkKGr27je7VEH/XpUbcBiMWfKl/AcVaXR1d6rJHV6c4RxV15t +XXc2g4xvvXdtiw7dXOf+posUQ== X-Google-Smtp-Source: AGHT+IHcCdTfYDcXkZ0Qt5csB3IUm6TeiHVMhTQKrBuA3FQKpUEaJPQvyodQ8j5twuKfm3ICNLk/lg== X-Received: by 2002:a17:902:c602:b0:1c8:7822:e5c3 with SMTP id r2-20020a170902c60200b001c87822e5c3mr7531739plr.47.1698049800869; Mon, 23 Oct 2023 01:30:00 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id d15-20020a170903230f00b001b8b07bc600sm5415805plh.186.2023.10.23.01.29.55 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 23 Oct 2023 01:30:00 -0700 (PDT) From: Xu Lu To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, tglx@linutronix.de, maz@kernel.org, anup@brainfault.org, atishp@atishpatra.org Cc: dengliang.1214@bytedance.com, liyu.yukiteru@bytedance.com, sunjiadong.lff@bytedance.com, xieyongji@bytedance.com, lihangjing@bytedance.com, chaiwen.cc@bytedance.com, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Xu Lu Subject: [RFC 07/12] riscv: Handle pseudo NMI in arch irq handler Date: Mon, 23 Oct 2023 16:29:06 +0800 Message-Id: <20231023082911.23242-8-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20231023082911.23242-1-luxu.kernel@bytedance.com> References: <20231023082911.23242-1-luxu.kernel@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This commit handles pseudo NMI in arch irq handler. We enter NMI context before handling NMI and keeps all interrupts disabled during NMI handling to avoid interrupt nesting. Signed-off-by: Xu Lu Signed-off-by: Hangjing Li --- drivers/irqchip/irq-riscv-intc.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-i= ntc.c index 83a0a744fce6..c672c0c64d5d 100644 --- a/drivers/irqchip/irq-riscv-intc.c +++ b/drivers/irqchip/irq-riscv-intc.c @@ -20,6 +20,26 @@ =20 static struct irq_domain *intc_domain; =20 +#ifdef CONFIG_RISCV_PSEUDO_NMI + +static asmlinkage void riscv_intc_irq(struct pt_regs *regs) +{ + unsigned long cause =3D regs->cause & ~CAUSE_IRQ_FLAG; + + if (unlikely(cause >=3D BITS_PER_LONG)) + panic("unexpected interrupt cause"); + + if (is_nmi(cause)) { + nmi_enter(); + generic_handle_domain_nmi(intc_domain, cause); + nmi_exit(); + } else { + generic_handle_domain_irq(intc_domain, cause); + } +} + +#else /* CONFIG_RISCV_PSEUDO_NMI */ + static asmlinkage void riscv_intc_irq(struct pt_regs *regs) { unsigned long cause =3D regs->cause & ~CAUSE_IRQ_FLAG; @@ -30,6 +50,8 @@ static asmlinkage void riscv_intc_irq(struct pt_regs *reg= s) generic_handle_domain_irq(intc_domain, cause); } =20 +#endif /* CONFIG_RISCV_PSEUDO_NMI */ + /* * On RISC-V systems local interrupts are masked or unmasked by writing * the SIE (Supervisor Interrupt Enable) CSR. As CSRs can only be written --=20 2.20.1 From nobody Thu Jan 1 11:01:33 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DC173C001E0 for ; Mon, 23 Oct 2023 08:30:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233241AbjJWIax (ORCPT ); Mon, 23 Oct 2023 04:30:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55356 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233145AbjJWIaf (ORCPT ); Mon, 23 Oct 2023 04:30:35 -0400 Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com [IPv6:2607:f8b0:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C58A4188 for ; Mon, 23 Oct 2023 01:30:07 -0700 (PDT) Received: by mail-pl1-x631.google.com with SMTP id d9443c01a7336-1c9e072472bso18124415ad.2 for ; Mon, 23 Oct 2023 01:30:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1698049806; x=1698654606; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=tFjJUDD1OgrURVFCNfyqhpsKd5+0kFPMkXoyM2g55XE=; b=kACoJMEKjIPMrZ4Kknmviad8kSSliwgMgQYkJPgoxtR6vhlBtJJA6uZN+aHqgK2p+L 2QsN5+EsN/32NDecwdcuhWQfc39gw7samaX1KHnKlYMvVcz2ikw4fiMrdPt1dayKlxoQ uB1QTi/siS4x0+kubKEsBJF1mcPh/faReofT3PNNCO7T6CFUjvcT+gH5quZAEm28omiP nnw0H4yOBkvHz4q7/9nK+FhnExrxz7Ijh8VP4Iy2AJkHY+VdvaJu4tXFvdhEFgeHZi9o blgFvnwKOT8piy4+YxB4/0YY60Jg96rPt+XufaRWmb/PU9+ZhE9jcnWMEvDtOqHAJGt5 Hc4A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698049806; x=1698654606; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=tFjJUDD1OgrURVFCNfyqhpsKd5+0kFPMkXoyM2g55XE=; b=csy5dY4RkgYOwAhuMdFElYp38jVRDLhIz8pZvfuTMheMaRzGrGIVIiMrtQy0Mn1+In o41I0WK64Ad8TIAFc0bJj9zlhWItvl1xnPnCmfk6Lc1ILWKfFcGHVxTMLXn0T6lrHpEG +NWU7LuUcp6rk6FKYEqTOzYd13Jo8nOF/AXHoiQGiXAyz3J4JM1/Ea5gF2qpNyzSXRtK t+4eoI2vC85f4BZQmkuXMNb7LYJxLa4IMK+2JJuhqH10S5hxGuNxiueBzrTObF2BxF8p Ysg7HqUNoTT9EXmrUfP2FZaGEWxtOy3wJvliRTx9UFjMmXVVbBFI7W3AykUdXjKsAvJ6 ifZg== X-Gm-Message-State: AOJu0YzUsagKoOuHjYmi0uXjXuH84t0/vP30oaOIPm1bepkF7IBCdEya +WdHsXzyIIgMlin7CwUoU79MO9yZcs6BknaKPp0= X-Google-Smtp-Source: AGHT+IHxUbIFNlkoPGxo4yFGNIhvxArkkFh19bAzzG/qz3kZJnf4Ua56mvZKJnXAyEFXPQjbUdh3rQ== X-Received: by 2002:a17:902:ecc6:b0:1c9:cc41:76e4 with SMTP id a6-20020a170902ecc600b001c9cc4176e4mr5988168plh.10.1698049806521; Mon, 23 Oct 2023 01:30:06 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id d15-20020a170903230f00b001b8b07bc600sm5415805plh.186.2023.10.23.01.30.01 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 23 Oct 2023 01:30:06 -0700 (PDT) From: Xu Lu To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, tglx@linutronix.de, maz@kernel.org, anup@brainfault.org, atishp@atishpatra.org Cc: dengliang.1214@bytedance.com, liyu.yukiteru@bytedance.com, sunjiadong.lff@bytedance.com, xieyongji@bytedance.com, lihangjing@bytedance.com, chaiwen.cc@bytedance.com, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Xu Lu Subject: [RFC 08/12] riscv: Enable NMIs during irqs disabled context Date: Mon, 23 Oct 2023 16:29:07 +0800 Message-Id: <20231023082911.23242-9-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20231023082911.23242-1-luxu.kernel@bytedance.com> References: <20231023082911.23242-1-luxu.kernel@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This commit allows NMIs to happen even when irqs are disabled. When disabling irqs, we mask all normal irqs via clearing corresponding bits in CSR_IE while leaving NMI bits alone. Signed-off-by: Xu Lu --- arch/riscv/include/asm/irqflags.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/riscv/include/asm/irqflags.h b/arch/riscv/include/asm/irq= flags.h index 9700a17a003a..42f7803582df 100644 --- a/arch/riscv/include/asm/irqflags.h +++ b/arch/riscv/include/asm/irqflags.h @@ -54,13 +54,13 @@ static inline void arch_local_irq_enable(void) /* unconditionally disable interrupts */ static inline void arch_local_irq_disable(void) { - csr_clear(CSR_IE, irqs_enabled_ie); + csr_clear(CSR_IE, ~ALLOWED_NMI_MASK); } =20 /* get status and disable interrupts */ static inline unsigned long arch_local_irq_save(void) { - return csr_read_clear(CSR_IE, irqs_enabled_ie); + return csr_read_clear(CSR_IE, ~ALLOWED_NMI_MASK); } =20 /* test flags */ --=20 2.20.1 From nobody Thu Jan 1 11:01:33 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BBCF9C001E0 for ; Mon, 23 Oct 2023 08:30:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233310AbjJWIa6 (ORCPT ); Mon, 23 Oct 2023 04:30:58 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55398 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229809AbjJWIah (ORCPT ); Mon, 23 Oct 2023 04:30:37 -0400 Received: from mail-pf1-x42c.google.com (mail-pf1-x42c.google.com [IPv6:2607:f8b0:4864:20::42c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 1091510EF for ; Mon, 23 Oct 2023 01:30:12 -0700 (PDT) Received: by mail-pf1-x42c.google.com with SMTP id d2e1a72fcca58-6ba54c3ed97so2910294b3a.2 for ; Mon, 23 Oct 2023 01:30:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1698049812; x=1698654612; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=CJ95gw/OAzQTy7Pqa/IxTLGW5cdMj2QHsOf1G6j+k+o=; b=S30YYUPeF3K4Wg7u5gv9HP8YR1365PNv7lH+yH8yqCc8lgXpWtld+GOUV6U7GTyVS+ z2i6EMgOwPqz2g8s079d1IGwRG5mdFGLL3yrri0vfqCM82IaDJBhdWfQbCiDszl8lqST nunaU991Zqd9Qv2rNXHHeyN9d1luOOD1BBb59nG9LQfmipfpOxjHyBeFD9h53TnHfvPO VLFwmsehR+/ER7sRRODTWF7nO1Q+1KoTh54UE/ZaIU6yJf62qYWBlDeVMe18atuoKP4W I2tppbSMuDVaigItRRWfAPpa2GVrMq8AQs1ViWm0O7V32Z7kbGWfO2iqWqcU7WSXqpCL OwWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698049812; x=1698654612; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=CJ95gw/OAzQTy7Pqa/IxTLGW5cdMj2QHsOf1G6j+k+o=; b=Ak2HaZ06WNAFnN0NNl6spF8BOxZHeBrgNwEhlfph8vyjcO1osWBFe0hIUSBuIWpsjg Ysd3sC6hkhm6SLeWtZ3cp/sfb7gcKmjT4xkHEyWYqdXvl/4f7V8laj7B24yNewX5hitQ sg9S5//TrfeBT9z1rfEtRsl3JBdlRpM4INwJxPRwyKfZde6tYOizMFE7bwBAaimxwV2n IU0HvtG0H+RwIYRuGesOWLukMH5j2ch7Cp5LumNA2Cmqpv8oTaa/ZAYJliflInauYp9M wGw4+SzQqsO8iFAJIw0/43iB1ZBroEXvMj80vJphwzn6o8HZd/pA05xnlqciu/D1neRf MTUw== X-Gm-Message-State: AOJu0YxuMO1DjDMAL8wE7Mo+I8cGITcLCCHMsCovbRFyN6gcoGE/uXr2 MsKGOAC4iI08QoCmcaRumvt90Q== X-Google-Smtp-Source: AGHT+IGiT6MKYuJi2ZVpq5CAD0Z+TUVQ/Ug4skMI+tZ5HzA4krihsDz81Ez71goYp3lfV7OVtvYOHA== X-Received: by 2002:a05:6a20:1611:b0:17b:8016:6f79 with SMTP id l17-20020a056a20161100b0017b80166f79mr10196547pzj.3.1698049812218; Mon, 23 Oct 2023 01:30:12 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id d15-20020a170903230f00b001b8b07bc600sm5415805plh.186.2023.10.23.01.30.06 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 23 Oct 2023 01:30:11 -0700 (PDT) From: Xu Lu To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, tglx@linutronix.de, maz@kernel.org, anup@brainfault.org, atishp@atishpatra.org Cc: dengliang.1214@bytedance.com, liyu.yukiteru@bytedance.com, sunjiadong.lff@bytedance.com, xieyongji@bytedance.com, lihangjing@bytedance.com, chaiwen.cc@bytedance.com, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Xu Lu Subject: [RFC 09/12] riscv: Enable NMIs during exceptions Date: Mon, 23 Oct 2023 16:29:08 +0800 Message-Id: <20231023082911.23242-10-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20231023082911.23242-1-luxu.kernel@bytedance.com> References: <20231023082911.23242-1-luxu.kernel@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" We have switched the way of disabling irqs to CSR_IE masking. But hardware still automatically clearing SIE field of CSR_STATUS whenever thread traps into kernel, which disabling all irqs including NMIs. This commit re-enables NMIs and normal irqs during exceptions by setting the SIE field in CSR_STATUS and restoring NMI and irq bits in CSR_IE. Signed-off-by: Xu Lu --- arch/riscv/include/asm/irqflags.h | 13 +++++++++++++ arch/riscv/include/asm/switch_to.h | 7 +++++++ arch/riscv/kernel/traps.c | 10 ++++++++++ 3 files changed, 30 insertions(+) diff --git a/arch/riscv/include/asm/irqflags.h b/arch/riscv/include/asm/irq= flags.h index 42f7803582df..6a709e9c69ca 100644 --- a/arch/riscv/include/asm/irqflags.h +++ b/arch/riscv/include/asm/irqflags.h @@ -29,6 +29,16 @@ static inline void set_nmi(int irq) {} =20 static inline void unset_nmi(int irq) {} =20 +static inline void enable_nmis(void) +{ + csr_set(CSR_IE, ALLOWED_NMI_MASK); +} + +static inline void disable_nmis(void) +{ + csr_clear(CSR_IE, ALLOWED_NMI_MASK); +} + static inline void local_irq_switch_on(void) { csr_set(CSR_STATUS, SR_IE); @@ -128,6 +138,9 @@ static inline void arch_local_irq_restore(unsigned long= flags) csr_set(CSR_STATUS, flags & SR_IE); } =20 +static inline void enable_nmis(void) {} +static inline void disable_nmis(void) {} + #endif /* !CONFIG_RISCV_PSEUDO_NMI */ =20 #endif /* _ASM_RISCV_IRQFLAGS_H */ diff --git a/arch/riscv/include/asm/switch_to.h b/arch/riscv/include/asm/sw= itch_to.h index a727be723c56..116cffeaa6bf 100644 --- a/arch/riscv/include/asm/switch_to.h +++ b/arch/riscv/include/asm/switch_to.h @@ -84,4 +84,11 @@ do { \ ((last) =3D __switch_to(__prev, __next)); \ } while (0) =20 +#ifdef CONFIG_RISCV_PSEUDO_NMI + +#define prepare_arch_switch(next) disable_nmis() +#define finish_arch_post_lock_switch() enable_nmis() + +#endif /* CONFIG_RISCV_PSEUDO_NMI */ + #endif /* _ASM_RISCV_SWITCH_TO_H */ diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c index fae8f610d867..63d3c1417563 100644 --- a/arch/riscv/kernel/traps.c +++ b/arch/riscv/kernel/traps.c @@ -135,7 +135,9 @@ asmlinkage __visible __trap_section void name(struct pt= _regs *regs) \ { \ if (user_mode(regs)) { \ irqentry_enter_from_user_mode(regs); \ + enable_nmis(); \ do_trap_error(regs, signo, code, regs->epc, "Oops - " str); \ + disable_nmis(); \ irqentry_exit_to_user_mode(regs); \ } else { \ irqentry_state_t state =3D irqentry_nmi_enter(regs); \ @@ -292,8 +294,12 @@ asmlinkage __visible __trap_section void do_trap_break= (struct pt_regs *regs) if (user_mode(regs)) { irqentry_enter_from_user_mode(regs); =20 + enable_nmis(); + handle_break(regs); =20 + disable_nmis(); + irqentry_exit_to_user_mode(regs); } else { irqentry_state_t state =3D irqentry_nmi_enter(regs); @@ -338,10 +344,14 @@ asmlinkage __visible noinstr void do_page_fault(struc= t pt_regs *regs) { irqentry_state_t state =3D irqentry_enter(regs); =20 + enable_nmis(); + handle_page_fault(regs); =20 local_irq_disable(); =20 + disable_nmis(); + irqentry_exit(regs, state); } #endif --=20 2.20.1 From nobody Thu Jan 1 11:01:33 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99FB9CDB474 for ; Mon, 23 Oct 2023 08:31:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233416AbjJWIbK (ORCPT ); Mon, 23 Oct 2023 04:31:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32966 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233057AbjJWIaq (ORCPT ); Mon, 23 Oct 2023 04:30:46 -0400 Received: from mail-pj1-x1031.google.com (mail-pj1-x1031.google.com [IPv6:2607:f8b0:4864:20::1031]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ECF77D6B for ; Mon, 23 Oct 2023 01:30:18 -0700 (PDT) Received: by mail-pj1-x1031.google.com with SMTP id 98e67ed59e1d1-27d1373f631so2375943a91.1 for ; Mon, 23 Oct 2023 01:30:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1698049818; x=1698654618; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+zaYgIQbfD9rezIYpqI9laPzzTdWx+ZAuyZYYGD6lvk=; b=Kjd3jevJivPEcsw5F67N8jy0YvnVBqHR5+uIDJ+9La/cvROPJqBXHLpWV36WidC2Dt RqgxAytav/Ur5ma2mfgDfcqN6jrfiMzQxwbb8dN3lsRIktkn6tuMHhGHKADFrFeJfrOu vd6vVbV8RzC9b3Nb5A7o5+iw7fF/QxQ4sd6vym6sd+EsWiiO81QOgRu3Mfol527titTO U/mt7XTu5CiNZYWEkpb4fElOcSDDuM+n5n8UAyRv/DbiqwpaSUaSULuhBTkIqBC/Ciro gsgWWvvEGSv0OIUlxFW8VCegUd4FS9f158yDWngOMSyfvFVSgpOEiUqcizwGCh6+Tgvr U16Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698049818; x=1698654618; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+zaYgIQbfD9rezIYpqI9laPzzTdWx+ZAuyZYYGD6lvk=; b=Bc8xPho61ChO1r5kDpCTLZSReGnYFms6YVjqwM8jucDTmXdnDTDc2LkKM5uZ2rn9GU 3s+sAVDO0PfyyUypJrTdRxvAWPg3VmjVhQZYTIBeYwb1BmPUgl6/Oyk4WfLkbIECf6bg j6LCfYZckZWErH+K9VmJvPa+oSBv/ggTvAFi1cPU2X655B5D6ZQwZhdwetDmTaQIRF0I GtoJfVG30Ght9RcC8bG15Zn2d5uL43s/kieDDSQ0UyAhaiM/tNzqQ0W8b/xX8nwas135 vtxFjms9EDdpr8gWZc2EEkthRk1pvldndgIs+R5OKyT98uR9dphQx7V6O/oP93F+0xcd +HLg== X-Gm-Message-State: AOJu0YzYWyEA9iJeBwps4Y+MdrtzGXLaxDcEZvhlfwjTiXNalNYakGx4 HaPAiA1GJaufDj+AIFZW2YDZmA== X-Google-Smtp-Source: AGHT+IFpt0TBJ9j577UmWU9zDZbpQtF3JzZZq4hl33QCJ42B75FqE8SvRamj8B8r3i2TgYxF2UyAzw== X-Received: by 2002:a17:90b:3e85:b0:27d:1567:f51d with SMTP id rj5-20020a17090b3e8500b0027d1567f51dmr9084887pjb.47.1698049817843; Mon, 23 Oct 2023 01:30:17 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id d15-20020a170903230f00b001b8b07bc600sm5415805plh.186.2023.10.23.01.30.12 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 23 Oct 2023 01:30:17 -0700 (PDT) From: Xu Lu To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, tglx@linutronix.de, maz@kernel.org, anup@brainfault.org, atishp@atishpatra.org Cc: dengliang.1214@bytedance.com, liyu.yukiteru@bytedance.com, sunjiadong.lff@bytedance.com, xieyongji@bytedance.com, lihangjing@bytedance.com, chaiwen.cc@bytedance.com, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Xu Lu Subject: [RFC 10/12] riscv: Enable NMIs during interrupt handling Date: Mon, 23 Oct 2023 16:29:09 +0800 Message-Id: <20231023082911.23242-11-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20231023082911.23242-1-luxu.kernel@bytedance.com> References: <20231023082911.23242-1-luxu.kernel@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" Hardware automatically clearing SIE field of CSR_STATUS whenever thread traps into kernel by interrupt, disabling all irqs including NMIs during interrupt handling. This commit re-enable NMIs during interrupt handling by setting the SIE field in CSR_STATUS and restoring NMIs bits in CSR_IE. Normal interrupts are still disabled during interrupt handling and NMIs are also disabled during NMIs handling to avoid nesting. Signed-off-by: Xu Lu Signed-off-by: Hangjing Li Reviewed-by: Liang Deng Reviewed-by: Yu Li --- arch/riscv/kernel/traps.c | 44 +++++++++++++++++++++++--------- drivers/irqchip/irq-riscv-intc.c | 2 ++ 2 files changed, 34 insertions(+), 12 deletions(-) diff --git a/arch/riscv/kernel/traps.c b/arch/riscv/kernel/traps.c index 63d3c1417563..185743edfa09 100644 --- a/arch/riscv/kernel/traps.c +++ b/arch/riscv/kernel/traps.c @@ -356,20 +356,11 @@ asmlinkage __visible noinstr void do_page_fault(struc= t pt_regs *regs) } #endif =20 -static void noinstr handle_riscv_irq(struct pt_regs *regs) +static void noinstr do_interrupt(struct pt_regs *regs) { struct pt_regs *old_regs; =20 - irq_enter_rcu(); old_regs =3D set_irq_regs(regs); - handle_arch_irq(regs); - set_irq_regs(old_regs); - irq_exit_rcu(); -} - -asmlinkage void noinstr do_irq(struct pt_regs *regs) -{ - irqentry_state_t state =3D irqentry_enter(regs); #ifdef CONFIG_IRQ_STACKS if (on_thread_stack()) { ulong *sp =3D per_cpu(irq_stack_ptr, smp_processor_id()) @@ -382,7 +373,9 @@ asmlinkage void noinstr do_irq(struct pt_regs *regs) "addi s0, sp, 2*"RISCV_SZPTR "\n" "move sp, %[sp] \n" "move a0, %[regs] \n" - "call handle_riscv_irq \n" + "la t0, handle_arch_irq \n" + REG_L" t1, (t0) \n" + "jalr t1 \n" "addi sp, s0, -2*"RISCV_SZPTR"\n" REG_L" s0, (sp) \n" "addi sp, sp, "RISCV_SZPTR "\n" @@ -398,11 +391,38 @@ asmlinkage void noinstr do_irq(struct pt_regs *regs) "memory"); } else #endif - handle_riscv_irq(regs); + handle_arch_irq(regs); + set_irq_regs(old_regs); +} + +static __always_inline void __do_nmi(struct pt_regs *regs) +{ + irqentry_state_t state =3D irqentry_nmi_enter(regs); + + do_interrupt(regs); + + irqentry_nmi_exit(regs, state); +} + +static __always_inline void __do_irq(struct pt_regs *regs) +{ + irqentry_state_t state =3D irqentry_enter(regs); + + irq_enter_rcu(); + do_interrupt(regs); + irq_exit_rcu(); =20 irqentry_exit(regs, state); } =20 +asmlinkage void noinstr do_irq(struct pt_regs *regs) +{ + if (IS_ENABLED(CONFIG_RISCV_PSEUDO_NMI) && regs_irqs_disabled(regs)) + __do_nmi(regs); + else + __do_irq(regs); +} + #ifdef CONFIG_GENERIC_BUG int is_valid_bugaddr(unsigned long pc) { diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-riscv-i= ntc.c index c672c0c64d5d..80ed8606e04d 100644 --- a/drivers/irqchip/irq-riscv-intc.c +++ b/drivers/irqchip/irq-riscv-intc.c @@ -34,7 +34,9 @@ static asmlinkage void riscv_intc_irq(struct pt_regs *reg= s) generic_handle_domain_nmi(intc_domain, cause); nmi_exit(); } else { + enable_nmis(); generic_handle_domain_irq(intc_domain, cause); + disable_nmis(); } } =20 --=20 2.20.1 From nobody Thu Jan 1 11:01:33 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 833B3CDB474 for ; Mon, 23 Oct 2023 08:31:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233156AbjJWIbR (ORCPT ); Mon, 23 Oct 2023 04:31:17 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55426 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233064AbjJWIas (ORCPT ); Mon, 23 Oct 2023 04:30:48 -0400 Received: from mail-pl1-x631.google.com (mail-pl1-x631.google.com [IPv6:2607:f8b0:4864:20::631]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 16A0010EA for ; Mon, 23 Oct 2023 01:30:24 -0700 (PDT) Received: by mail-pl1-x631.google.com with SMTP id d9443c01a7336-1c9e95aa02dso21512265ad.0 for ; Mon, 23 Oct 2023 01:30:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1698049823; x=1698654623; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0DBuRxeBwydw4BwBQS/PBMNuUJKwvOuK50YBTu/VQqs=; b=KxlUY0bdG+FdiA3aRJmxHmFrT8czPG8ktWnSfbAuxJ187b7aBmevmaWvZkDAhm9zaR z8VegjvSmMIa2QZ8GqQOQwtZEfoPt6aN5P9HZ2iBdy4D7IWbGC5tSUIg7sVjTfi2jJ17 tKAR09xEJOktQgKsPukRgSIeSpb7qztR1hnHZDX4qZtDObtfssk3R39f7DHLhScPfRgc OKQ4peFFyxDPmhFts5fWtt7R2lIMq+7iGydLsHEII+D725ZOD3GW0wmbFrOUmjeAQP0/ aY7WId0cAQqI4/TBWfWDf9wqS1oA+2iQOH50c3f6uCNZstxDJ2Wm3w92O6G2EzESiTfX NT1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698049823; x=1698654623; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0DBuRxeBwydw4BwBQS/PBMNuUJKwvOuK50YBTu/VQqs=; b=uz9GDxWsXdFX4lHoNit6cq1DGiIaPyVmdsQQ/jifIiOiAan6pNFwdVSeKiwKpjeoow yeMUoQMXL8k7Xn3KETPRdFvYKNwV8CR607F7CyAj1sSeIYdjj2DcqbwWFPQ0JZMJt6aP kPfVz3uWpW3lPbzTb1chDkeDyIZhCnV/jx4eqC4dWEttnJCf/Ohqj5cmWxgr/ozHiylP s2LF9EoEPk2L1pnwlh3w2NthEkEILHcvUUwqjFaRDMcDYrztauAx2jlIUIz5lgHP+Soz KzyQThjbNVUpmQjQ2lL365odOlaFe9Thia3maVU20EhUbu8Eah1/WxemPTE7gVfK2xAR +dOQ== X-Gm-Message-State: AOJu0Yx6zHj7ieHnWwoU+mJsZk9njlFYECp1dmx3fQU1mAk0ncv6GxBw JObes6vvRLbAf4l6nn9WyBEffQ== X-Google-Smtp-Source: AGHT+IFbYL9LzR1f7qPWMFMhpIw/jq2fW6XuDR85z/dEj8js9eHh6BWdpBWgonHNNRMuACn/U0IMZw== X-Received: by 2002:a17:903:32d2:b0:1c6:d0a:cf01 with SMTP id i18-20020a17090332d200b001c60d0acf01mr16447416plr.11.1698049823515; Mon, 23 Oct 2023 01:30:23 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id d15-20020a170903230f00b001b8b07bc600sm5415805plh.186.2023.10.23.01.30.18 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 23 Oct 2023 01:30:23 -0700 (PDT) From: Xu Lu To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, tglx@linutronix.de, maz@kernel.org, anup@brainfault.org, atishp@atishpatra.org Cc: dengliang.1214@bytedance.com, liyu.yukiteru@bytedance.com, sunjiadong.lff@bytedance.com, xieyongji@bytedance.com, lihangjing@bytedance.com, chaiwen.cc@bytedance.com, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Xu Lu Subject: [RFC 11/12] riscv: Request pmu overflow interrupt as NMI Date: Mon, 23 Oct 2023 16:29:10 +0800 Message-Id: <20231023082911.23242-12-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20231023082911.23242-1-luxu.kernel@bytedance.com> References: <20231023082911.23242-1-luxu.kernel@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This commit registers pmu overflow interrupt as NMI to improve the accuracy of perf sampling. Signed-off-by: Xu Lu --- arch/riscv/include/asm/irqflags.h | 2 +- drivers/perf/riscv_pmu_sbi.c | 23 +++++++++++++++++++---- 2 files changed, 20 insertions(+), 5 deletions(-) diff --git a/arch/riscv/include/asm/irqflags.h b/arch/riscv/include/asm/irq= flags.h index 6a709e9c69ca..be840e297559 100644 --- a/arch/riscv/include/asm/irqflags.h +++ b/arch/riscv/include/asm/irqflags.h @@ -12,7 +12,7 @@ =20 #ifdef CONFIG_RISCV_PSEUDO_NMI =20 -#define __ALLOWED_NMI_MASK 0 +#define __ALLOWED_NMI_MASK BIT(IRQ_PMU_OVF) #define ALLOWED_NMI_MASK (__ALLOWED_NMI_MASK & irqs_enabled_ie) =20 static inline bool nmi_allowed(int irq) diff --git a/drivers/perf/riscv_pmu_sbi.c b/drivers/perf/riscv_pmu_sbi.c index 995b501ec721..85abb7dd43b9 100644 --- a/drivers/perf/riscv_pmu_sbi.c +++ b/drivers/perf/riscv_pmu_sbi.c @@ -760,6 +760,7 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void *d= ev) =20 static int pmu_sbi_starting_cpu(unsigned int cpu, struct hlist_node *node) { + int ret =3D 0; struct riscv_pmu *pmu =3D hlist_entry_safe(node, struct riscv_pmu, node); struct cpu_hw_events *cpu_hw_evt =3D this_cpu_ptr(pmu->hw_events); =20 @@ -778,20 +779,30 @@ static int pmu_sbi_starting_cpu(unsigned int cpu, str= uct hlist_node *node) if (riscv_pmu_use_irq) { cpu_hw_evt->irq =3D riscv_pmu_irq; csr_clear(CSR_IP, BIT(riscv_pmu_irq_num)); -#ifndef CONFIG_RISCV_PSEUDO_NMI +#ifdef CONFIG_RISCV_PSEUDO_NMI + ret =3D prepare_percpu_nmi(riscv_pmu_irq); + if (ret !=3D 0) { + pr_err("Failed to prepare percpu nmi:%d\n", ret); + return ret; + } + enable_percpu_nmi(riscv_pmu_irq, IRQ_TYPE_NONE); +#else csr_set(CSR_IE, BIT(riscv_pmu_irq_num)); -#endif enable_percpu_irq(riscv_pmu_irq, IRQ_TYPE_NONE); +#endif } =20 - return 0; + return ret; } =20 static int pmu_sbi_dying_cpu(unsigned int cpu, struct hlist_node *node) { if (riscv_pmu_use_irq) { +#ifdef CONFIG_RISCV_PSEUDO_NMI + disable_percpu_nmi(riscv_pmu_irq); + teardown_percpu_nmi(riscv_pmu_irq); +#else disable_percpu_irq(riscv_pmu_irq); -#ifndef CONFIG_RISCV_PSEUDO_NMI csr_clear(CSR_IE, BIT(riscv_pmu_irq_num)); #endif } @@ -835,7 +846,11 @@ static int pmu_sbi_setup_irqs(struct riscv_pmu *pmu, s= truct platform_device *pde return -ENODEV; } =20 +#ifdef CONFIG_RISCV_PSEUDO_NMI + ret =3D request_percpu_nmi(riscv_pmu_irq, pmu_sbi_ovf_handler, "riscv-pmu= ", hw_events); +#else ret =3D request_percpu_irq(riscv_pmu_irq, pmu_sbi_ovf_handler, "riscv-pmu= ", hw_events); +#endif if (ret) { pr_err("registering percpu irq failed [%d]\n", ret); return ret; --=20 2.20.1 From nobody Thu Jan 1 11:01:33 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FBCACDB474 for ; Mon, 23 Oct 2023 08:31:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233206AbjJWIbX (ORCPT ); Mon, 23 Oct 2023 04:31:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:51074 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233303AbjJWIa5 (ORCPT ); Mon, 23 Oct 2023 04:30:57 -0400 Received: from mail-pf1-x42d.google.com (mail-pf1-x42d.google.com [IPv6:2607:f8b0:4864:20::42d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BAA36D7F for ; Mon, 23 Oct 2023 01:30:30 -0700 (PDT) Received: by mail-pf1-x42d.google.com with SMTP id d2e1a72fcca58-6bd0e1b1890so2155755b3a.3 for ; Mon, 23 Oct 2023 01:30:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bytedance.com; s=google; t=1698049829; x=1698654629; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qwqRYcNnh3bL2bnUHbsBXwm7mmQqzZo0pVLQzAgbeF0=; b=ZIJ3GaaKbcga50C/dczsvfOIZV8U+JvyFtPaKRxNMgLu+KiqGYMhAuw9avgnk2yeLy QbP+xyj7FkSP5VtNlHnJU3B3WNh+9ZN8GudRSuoeLVkfdEpCj4VI1t0eedIL2S9HKTAQ JwI1VKPPYAMvmo42+zqvG2EiubACFBU25T1aTw6UNruI+zHShNBsD9+pcH399WQ8y2kT utVRjCq4num5qoPFTx70Wu8FPRJcTvvn8KKygVbqPFyQF7/U49hwMwZaqFpOYGxNs3Hd LfUB0+Li3YduCbfadzRq8Gv1ymQPz+gW+jhWZnXHiG/ur5TyQRk+4neH/SbnuV5WmXq/ /tYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1698049829; x=1698654629; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qwqRYcNnh3bL2bnUHbsBXwm7mmQqzZo0pVLQzAgbeF0=; b=ihWK3M5ys8WzQSut+0G6tssraBCjuIy/uNGHd0NdAydjX3fnV+9IspqKaXyNpwhmaD GilFsKf8aYI5Hi0Y2H8yC2/RpuTf9OSHbYwZain/wMiCr2C9tECG3ylZ6yH4cemWkMNs WYRAlKhtnQtZ68K3VZU5mFQd5Y5jM+/PbSLNR4EMuPKujOBhnIpNiPsav+IfFAzLfCcH ldmJC0BBpvxhHWfgYFBRAkn8siaE626uCll4oKViDakWziEEk7tcymf3jo66oQnx8Las lExMAiqHceegs+kVBQwb4UFWtgZvF6+Bu2hCVIrqgiuGTGsIi2vXq4HChdBIpcw2FMuW rCiw== X-Gm-Message-State: AOJu0YxzMkFwo/TGjdCOkpvdBcGtDkP6iPmawvQU8Wg5E/dbYqBXIjCu P6TwgyA/eE2mJ3Q9Z1OrDZztXw== X-Google-Smtp-Source: AGHT+IGyFbUXBRXdZQh2eZcjKZcfVc6v/Ld3/FBhlNzBXs9NtpZ6v0cDf/3miDn1ZPth5cbBQ//TYA== X-Received: by 2002:a05:6a21:498b:b0:17b:d39c:1d6 with SMTP id ax11-20020a056a21498b00b0017bd39c01d6mr5328684pzc.30.1698049829431; Mon, 23 Oct 2023 01:30:29 -0700 (PDT) Received: from J9GPGXL7NT.bytedance.net ([203.208.167.147]) by smtp.gmail.com with ESMTPSA id d15-20020a170903230f00b001b8b07bc600sm5415805plh.186.2023.10.23.01.30.23 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Mon, 23 Oct 2023 01:30:28 -0700 (PDT) From: Xu Lu To: paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu, tglx@linutronix.de, maz@kernel.org, anup@brainfault.org, atishp@atishpatra.org Cc: dengliang.1214@bytedance.com, liyu.yukiteru@bytedance.com, sunjiadong.lff@bytedance.com, xieyongji@bytedance.com, lihangjing@bytedance.com, chaiwen.cc@bytedance.com, linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, Xu Lu Subject: [RFC 12/12] riscv: Enable CONFIG_RISCV_PSEUDO_NMI in default Date: Mon, 23 Oct 2023 16:29:11 +0800 Message-Id: <20231023082911.23242-13-luxu.kernel@bytedance.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20231023082911.23242-1-luxu.kernel@bytedance.com> References: <20231023082911.23242-1-luxu.kernel@bytedance.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" This commit enables CONFIG_RISCV_PSEUDO_NMI in default. Now pseudo NMI feature is defaultly enabled on RISC-V. Signed-off-by: Xu Lu --- arch/riscv/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 487e4293f31e..ecccdc91563f 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -672,7 +672,7 @@ config RISCV_BOOT_SPINWAIT config RISCV_PSEUDO_NMI bool "Support for NMI-like interrupts" depends on !RISCV_M_MODE - default n + default y help Adds support for mimicking Non-Maskable Interrupts through the use of CSR_IE register. --=20 2.20.1