From nobody Tue Dec 16 13:37:30 2025 Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4CBB82D640A for ; Wed, 5 Nov 2025 08:28:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762331336; cv=none; b=Rfv1tTJl/jUsIclp3AFm0JxbNS+JFTf9dbYPwDkKPrAI4WMALJp2x+gw09XYPXlD2oE6LaUbRLdh4CG5jeLj0WdniB8ZSIpfWR+gOTpeqE34z7AwIX3BZGR68XT6t8j/21kmCyJ9tdVba1QxUc2eCjapLmutX4kCC/DioH8VSk0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762331336; c=relaxed/simple; bh=YD1DGc1PTTzmFAGHhSNrHRTvJkRtZfZ1qpUIc0VusL0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=lbWxUHuZu4HZUaYDqxei833oBc2auebowBX+126VtMBd1mE6KKA7u2O9AEwbae0L6oNby7qifn8HlYdwTxRBddxjBFRFduJFzRtMgxXxVSri9LP/2FUxn6X2TOAcGZdpnpLdlrxggSuEgGFa+MDWpOO8agYPp4cnc0GBhCIyhcw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc.com header.i=@rivosinc.com header.b=ShVK0YTS; arc=none smtp.client-ip=209.85.214.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc.com header.i=@rivosinc.com header.b="ShVK0YTS" Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-294fd2ca6acso5161575ad.0 for ; Wed, 05 Nov 2025 00:28:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc.com; s=google; t=1762331333; x=1762936133; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mxVWTJdT+SCjk5O8ZmtBkLAMOboy+sCiTeWzaOD3L7c=; b=ShVK0YTSvDfTVYKyfdoqQOagSaBTDIUuwymTHoPAAPadL9xsq8UgKEfQn4eN4+lecM 79WBWv87nhl//oFNcEab6WzxvoCtSjZYaW6ZkhWTzk4oFAdUyu+qIiU14wcbN6Zoa+ed +7uWaw2oIV8oO3id9x8tzFjMwIDQKryyXhp0NPIePHU4/ASMhp6voIvd1web3imWZS3h +2RR6wUQY00ktmPZsRYUAQx0fWx1ehBmKEDz16ojzVLX3IULUUN8YeQHb2VsL3JDYpjl RfazJoSH1d/v7b9gm/X9SA9DfIox/zXDnEU9fX6NQRswd4rFcrtpPqI3FrC+OJCrjRGo Xh6g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1762331334; x=1762936134; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mxVWTJdT+SCjk5O8ZmtBkLAMOboy+sCiTeWzaOD3L7c=; b=K5KmlfwC5qWgfLH16DelJ/JBwKiVMFWTwucfRg4gDVC2Fgb9JMS1HeoZ3yCoQl17sc BE6lfquTAnOxF0D363OMpjowOWgtsfw6GNOp4XjZ+QxQU70AZwm1qV1HT8BTmUzHaYPT DtXTT+3hxGw2vvwoUFOmWPG7u3AvxbSmvR0ANWOabd2lhDnqBVaVP+qYYDpHA5u8C02j gr5fMwnh4LrY1FvRpGUY6o4lw8hIUDYblVUCtpfeKVwQHJSCi/TXFMn0hx5poUcxjX1P M/7/hHTtJSpMJSC2XOvY44nEXymi7+Vzzdklzm+OfKYbrUF6sN1KKfevOrpLhjzzWRql 1T1Q== X-Forwarded-Encrypted: i=1; AJvYcCW/xDzEsk22dixfA+tgcYas9vzzWF9DsdgiFGJUfhJyjJd9aI7tqZdJ9kBlFllXjCxSqv9qVuZ+vN2HS6M=@vger.kernel.org X-Gm-Message-State: AOJu0Yx2Gn36KbLca8xp/5sGywVU14ey2VNRyIcYGMaQz8gltWK/0PUX AUMKxlAZnzUeFNrOutBt2MCxLvlGtXTGXAWOMr/93GGDYXNJQ0GTkhK2IGOVJpOafsw= X-Gm-Gg: ASbGncsTax7Bqc8EMRupcxgukdH6guYZH/O9UVb7CEnX1g3vdbE0K9A/jyK6HSQVXsm 9kPrA03BvigbUWIlLrPw9mS64AEj522QmELgmvyxZRhfL0fus4MO+DM95e91GfPq0qR1eMmKNe/ fyfjr0lh2LHMCTsHBO1FDySbUCsOmY7CXhI+/vE802/0dqF7DeIiyhO9sNJN/HSlnteg6HswlHd pEoYaND+eum9I0dMEDQF92k4/86CZzOGShcRE77ouoHYe9s/Gvrk4fYT03dpdWHriMGZ2CPZbee 2k2phQDX1432zxFg2c4yXny//mduTYFCTxqeuqxR/xvDbUWZjlOOat2m6JtpX3h0szYR7GvYWxr T28HBRpCuUKEyADS71jum4dKu6CPcD0nJtD4csznMnPZP2BWFldt84yFy X-Google-Smtp-Source: AGHT+IGoN1GXGikcvAvUXnBnSY8dbKotW+FyvTG4CG4Bx86TVuBznGgWug72QAB0JjPEWtQzHPyqhw== X-Received: by 2002:a17:903:2346:b0:269:aba9:ffd7 with SMTP id d9443c01a7336-295f9483fb3mr69028445ad.25.1762331333570; Wed, 05 Nov 2025 00:28:53 -0800 (PST) Received: from cleger.eu.int ([2001:41d0:420:f300::]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-29601a741e3sm51870875ad.98.2025.11.05.00.28.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Nov 2025 00:28:53 -0800 (PST) From: =?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?= To: Paul Walmsley , Palmer Dabbelt , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: =?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?= , Himanshu Chauhan , Anup Patel , Xu Lu , Atish Patra , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Yunhui Cui Subject: [PATCH v8 1/5] riscv: add SBI SSE extension definitions Date: Wed, 5 Nov 2025 08:26:33 +0000 Message-ID: <20251105082639.342973-2-cleger@rivosinc.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251105082639.342973-1-cleger@rivosinc.com> References: <20251105082639.342973-1-cleger@rivosinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Add needed definitions for SBI Supervisor Software Events extension [1]. This extension enables the SBI to inject events into supervisor software much like ARM SDEI. [1] https://lists.riscv.org/g/tech-prs/message/515 Signed-off-by: Cl=C3=A9ment L=C3=A9ger --- arch/riscv/include/asm/sbi.h | 61 ++++++++++++++++++++++++++++++++++++ 1 file changed, 61 insertions(+) diff --git a/arch/riscv/include/asm/sbi.h b/arch/riscv/include/asm/sbi.h index ccc77a89b1e2..0f47e6f03b7e 100644 --- a/arch/riscv/include/asm/sbi.h +++ b/arch/riscv/include/asm/sbi.h @@ -37,6 +37,7 @@ enum sbi_ext_id { SBI_EXT_NACL =3D 0x4E41434C, SBI_EXT_FWFT =3D 0x46574654, SBI_EXT_MPXY =3D 0x4D505859, + SBI_EXT_SSE =3D 0x535345, =20 /* Experimentals extensions must lie within this range */ SBI_EXT_EXPERIMENTAL_START =3D 0x08000000, @@ -505,6 +506,66 @@ enum sbi_mpxy_rpmi_attribute_id { #define SBI_MPXY_CHAN_CAP_SEND_WITHOUT_RESP BIT(4) #define SBI_MPXY_CHAN_CAP_GET_NOTIFICATIONS BIT(5) =20 +enum sbi_ext_sse_fid { + SBI_SSE_EVENT_ATTR_READ =3D 0, + SBI_SSE_EVENT_ATTR_WRITE, + SBI_SSE_EVENT_REGISTER, + SBI_SSE_EVENT_UNREGISTER, + SBI_SSE_EVENT_ENABLE, + SBI_SSE_EVENT_DISABLE, + SBI_SSE_EVENT_COMPLETE, + SBI_SSE_EVENT_INJECT, + SBI_SSE_HART_UNMASK, + SBI_SSE_HART_MASK, +}; + +enum sbi_sse_state { + SBI_SSE_STATE_UNUSED =3D 0, + SBI_SSE_STATE_REGISTERED =3D 1, + SBI_SSE_STATE_ENABLED =3D 2, + SBI_SSE_STATE_RUNNING =3D 3, +}; + +/* SBI SSE Event Attributes. */ +enum sbi_sse_attr_id { + SBI_SSE_ATTR_STATUS =3D 0x00000000, + SBI_SSE_ATTR_PRIO =3D 0x00000001, + SBI_SSE_ATTR_CONFIG =3D 0x00000002, + SBI_SSE_ATTR_PREFERRED_HART =3D 0x00000003, + SBI_SSE_ATTR_ENTRY_PC =3D 0x00000004, + SBI_SSE_ATTR_ENTRY_ARG =3D 0x00000005, + SBI_SSE_ATTR_INTERRUPTED_SEPC =3D 0x00000006, + SBI_SSE_ATTR_INTERRUPTED_FLAGS =3D 0x00000007, + SBI_SSE_ATTR_INTERRUPTED_A6 =3D 0x00000008, + SBI_SSE_ATTR_INTERRUPTED_A7 =3D 0x00000009, + + SBI_SSE_ATTR_MAX =3D 0x0000000A +}; + +#define SBI_SSE_ATTR_STATUS_STATE_OFFSET 0 +#define SBI_SSE_ATTR_STATUS_STATE_MASK 0x3 +#define SBI_SSE_ATTR_STATUS_PENDING_OFFSET 2 +#define SBI_SSE_ATTR_STATUS_INJECT_OFFSET 3 + +#define SBI_SSE_ATTR_CONFIG_ONESHOT BIT(0) + +#define SBI_SSE_ATTR_INTERRUPTED_FLAGS_SSTATUS_SPP BIT(0) +#define SBI_SSE_ATTR_INTERRUPTED_FLAGS_SSTATUS_SPIE BIT(1) +#define SBI_SSE_ATTR_INTERRUPTED_FLAGS_HSTATUS_SPV BIT(2) +#define SBI_SSE_ATTR_INTERRUPTED_FLAGS_HSTATUS_SPVP BIT(3) + +#define SBI_SSE_EVENT_LOCAL_HIGH_PRIO_RAS 0x00000000 +#define SBI_SSE_EVENT_LOCAL_DOUBLE_TRAP 0x00000001 +#define SBI_SSE_EVENT_GLOBAL_HIGH_PRIO_RAS 0x00008000 +#define SBI_SSE_EVENT_LOCAL_PMU_OVERFLOW 0x00010000 +#define SBI_SSE_EVENT_LOCAL_LOW_PRIO_RAS 0x00100000 +#define SBI_SSE_EVENT_GLOBAL_LOW_PRIO_RAS 0x00108000 +#define SBI_SSE_EVENT_LOCAL_SOFTWARE_INJECTED 0xffff0000 +#define SBI_SSE_EVENT_GLOBAL_SOFTWARE_INJECTED 0xffff8000 + +#define SBI_SSE_EVENT_PLATFORM BIT(14) +#define SBI_SSE_EVENT_GLOBAL BIT(15) + /* SBI spec version fields */ #define SBI_SPEC_VERSION_DEFAULT 0x1 #define SBI_SPEC_VERSION_MAJOR_SHIFT 24 --=20 2.43.0 From nobody Tue Dec 16 13:37:30 2025 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 448C42C0285 for ; Wed, 5 Nov 2025 08:29:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762331345; cv=none; b=SLVNUKLU0c3MIUN0NkC+LBItJWX0SDqEyTFisuLwE201ogiOcvw6AUqQgZ8jRrV7GK61Q1XXQoX1gtd98gifFoRIc6IAGH4hZjvsZu7wDCAZpJkto7dV4wBnOI57kCgY5DdgA+x4aPrF3p4Q5NnCLkzaKBFg47H+PUxmynaG3ME= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762331345; c=relaxed/simple; bh=15CWRnlubK5IUWGnaPJbLVWSFVFZMPUG8nbQfKHB4f8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=chFWiY8kFmHxZUh+gBJOdvSHBlZi975Q22H2Zi7hnK1NCcHDZf8PCj5t2MYRZmcpIpe3+IHjYsG9wUYPmV96AZNEk29/C5e/yWLzy1mLj4GGfHLR/42Dbmllx3j8mMshSgD3purZUbbYeuxemakicqo2n0hCnqucxpw5QEwhFKo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc.com header.i=@rivosinc.com header.b=YP5R8aHY; arc=none smtp.client-ip=209.85.214.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc.com header.i=@rivosinc.com header.b="YP5R8aHY" Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-29568d93e87so34407085ad.2 for ; Wed, 05 Nov 2025 00:29:02 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc.com; s=google; t=1762331342; x=1762936142; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=anlUB3rXmmV23za2cZAX6kM83tarAqtEt9FUbE3gVZg=; b=YP5R8aHY/Rtt/IZWQ3MTQSDfgooEi/2EU1erepWUnCerX6oF789lUA7wRJsBmdmPHl 7WzWVMVNCOthxZ++ceEUHRKkLXivXhJ7BvpsrDnCAcZD4BGJHGz66PCVk4R05KcEpjjz NvMJ5FGHGIJ0uLUVI72U1Xfl6bqoEwmoIXUL4Y7lOD5D50MV5JzDc/rujQEeEJ9QdaaE 4RCT9d6OE7TtyiM/Xy7J1qjA9DRodgt9rNGKP6a/zhC4YveMhqQ/bUi28D9dT70t9eNo pAS+k618FVH4FvlNLKPfO4ToHI7VT47AJfRTfR1wo5g4vHAi2VwQTOIx27f+m4ZrFehT UYng== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1762331342; x=1762936142; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=anlUB3rXmmV23za2cZAX6kM83tarAqtEt9FUbE3gVZg=; b=eIJ7sCyRGtR8V0DFuaQtJ9mGVJ1bx9k5GFH593qYetwwxC7XQAqdko5yf7MCTuyb2K UG7nBJR2KK1vT6eUuQ4bAADb74g1Ym1BX4lrYXxJEKB05if+DGpDwOVAMyU7eG97kZ8x gP2grqSaCjF+kZUQgzJMzs2UfivGhK/qKvGdHYO6DiOh1O79QEUCL8hYne4l5l9g918J l8XJov26EbcEvuLTHwcR+xR7KXMLkmyRIltexDJPwXG/33Q5viZb3qof1kJ+yDLmb8La KKYaOeHSz7JP+/MHJHE6PGfvlxepdrOFZdZXlao83ccRQt40yu5z/Q8Az/60xNd5mHBM 2c6A== X-Forwarded-Encrypted: i=1; AJvYcCUJmg8VE/xQsrhmeE2QmoDvRbwrsX1REuhjcHg9MjjCJ80tOPGuT4jV7qWY2Cd+ZPHjwlLAGLu1urzvOzc=@vger.kernel.org X-Gm-Message-State: AOJu0YzKVi6/fvJWo91OFj5Ulxy0hCunZ7E8hmGDvIS4hkI71kKAkYD6 E1yOUkLftJqPgC/Zy0hVZ6qJ+++deNNeWU5CrypIp/ecPYfV54+Hh3EYpqwUTKjGN54= X-Gm-Gg: ASbGncvXP1EKaEMcNhJcLI0sIpccAjoHHG/cusTSMl995sVA3NRDCzdu8aOB7+tY9XB oz5GHCJT7bD6oXNGoywFERPOejEJex65ZMYhIcwvERyN4zJZyKuuMpLhOLwn8YD2kdR7o1S1KNT vIYAco4LGHuJgS5knv+n8MXy+xYA7hkKafeyXbSViRtv11cWoq70tL8moU8Dbsis6wTqXyqxD3G qPV1rQ18LC5XgGEcCrrrxK0X92/SwOiD2xD5X6XdcLs1RfuKVzIy2lUR9c1ZXl+DuP5KDvFFYNJ qixZWZQT/TCcu4VwwmYKPKr/cm4xs0eKvSNlHcVOT1HlNB6Vj0HrOX6aucp2HBMkKPY48L3/oQl g9mfPfRwlWwddABXyiSwuExymuCxceqdnWWEp/Mn6cWBQ07rkoHDGwFnv9yZLZR5kHcg= X-Google-Smtp-Source: AGHT+IEHRv9sGZgg4O10g1SzBN2n4TT/3gcAW3pvJKqPdS5cgXt2xL3Bi+wAIWW6gKq7wqZNbqi4BA== X-Received: by 2002:a17:903:944:b0:269:b2e5:ee48 with SMTP id d9443c01a7336-2962ae91307mr35889295ad.52.1762331342396; Wed, 05 Nov 2025 00:29:02 -0800 (PST) Received: from cleger.eu.int ([2001:41d0:420:f300::]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-29601a741e3sm51870875ad.98.2025.11.05.00.28.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Nov 2025 00:29:01 -0800 (PST) From: =?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?= To: Paul Walmsley , Palmer Dabbelt , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: =?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?= , Himanshu Chauhan , Anup Patel , Xu Lu , Atish Patra , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Yunhui Cui Subject: [PATCH v8 2/5] riscv: add support for SBI Supervisor Software Events extension Date: Wed, 5 Nov 2025 08:26:34 +0000 Message-ID: <20251105082639.342973-3-cleger@rivosinc.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251105082639.342973-1-cleger@rivosinc.com> References: <20251105082639.342973-1-cleger@rivosinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The SBI SSE extension allows the supervisor software to be notified by the SBI of specific events that are not maskable. The context switch is handled partially by the firmware which will save registers a6 and a7. When entering kernel we can rely on these 2 registers to setup the stack and save all the registers. Since SSE events can be delivered at any time to the kernel (including during exception handling, we need a way to locate the current_task for context tracking. On RISC-V, it is sotred in scratch when in user space or tp when in kernel space (in which case SSCRATCH is zero). But at a at the beginning of exception handling, SSCRATCH is used to swap tp and check the origin of the exception. If interrupted at that point, then, there is no way to reliably know were is located the current task_struct. Even checking the interruption location won't work as SSE event can be nested on top of each other so the original interruption site might be lost at some point. In order to retrieve it reliably, store the current task in an additional __sse_entry_task per_cpu array. This array is then used to retrieve the current task based on the hart ID that is passed to the SSE event handler in a6. That being said, the way the current task struct is stored should probably be reworked to find a better reliable alternative. Since each events (and each CPU for local events) have their own context and can preempt each other, allocate a stack (and a shadow stack if needed for each of them (and for each cpu for local events). When completing the event, if we were coming from kernel with interrupts disabled, simply return there. If coming from userspace or kernel with interrupts enabled, simulate an interrupt exception by setting IE_SIE in CSR_IP to allow delivery of signals to user task. For instance this can happen, when a RAS event has been generated by a user application and a SIGBUS has been sent to a task. Signed-off-by: Cl=C3=A9ment L=C3=A9ger --- arch/riscv/include/asm/asm.h | 14 ++- arch/riscv/include/asm/scs.h | 7 ++ arch/riscv/include/asm/sse.h | 47 +++++++ arch/riscv/include/asm/switch_to.h | 14 +++ arch/riscv/include/asm/thread_info.h | 1 + arch/riscv/kernel/Makefile | 1 + arch/riscv/kernel/asm-offsets.c | 14 +++ arch/riscv/kernel/sbi_sse.c | 162 ++++++++++++++++++++++++ arch/riscv/kernel/sbi_sse_entry.S | 178 +++++++++++++++++++++++++++ 9 files changed, 435 insertions(+), 3 deletions(-) create mode 100644 arch/riscv/include/asm/sse.h create mode 100644 arch/riscv/kernel/sbi_sse.c create mode 100644 arch/riscv/kernel/sbi_sse_entry.S diff --git a/arch/riscv/include/asm/asm.h b/arch/riscv/include/asm/asm.h index ac28066bb564..a981d520f6d3 100644 --- a/arch/riscv/include/asm/asm.h +++ b/arch/riscv/include/asm/asm.h @@ -84,16 +84,24 @@ .endm =20 #ifdef CONFIG_SMP -.macro asm_per_cpu dst sym tmp - lw \tmp, TASK_TI_CPU_NUM(tp) - slli \tmp, \tmp, RISCV_LGPTR +.macro asm_per_cpu_with_cpu dst sym tmp cpu + slli \tmp, \cpu, RISCV_LGPTR la \dst, __per_cpu_offset add \dst, \dst, \tmp REG_L \tmp, 0(\dst) la \dst, \sym add \dst, \dst, \tmp .endm + +.macro asm_per_cpu dst sym tmp + lw \tmp, TASK_TI_CPU_NUM(tp) + asm_per_cpu_with_cpu \dst \sym \tmp \tmp +.endm #else /* CONFIG_SMP */ +.macro asm_per_cpu_with_cpu dst sym tmp cpu + la \dst, \sym +.endm + .macro asm_per_cpu dst sym tmp la \dst, \sym .endm diff --git a/arch/riscv/include/asm/scs.h b/arch/riscv/include/asm/scs.h index ab7714aa93bd..fd8242c54e90 100644 --- a/arch/riscv/include/asm/scs.h +++ b/arch/riscv/include/asm/scs.h @@ -18,6 +18,11 @@ load_per_cpu gp, irq_shadow_call_stack_ptr, \tmp .endm =20 +/* Load the per-CPU IRQ shadow call stack to gp. */ +.macro scs_load_sse_stack reg_evt + REG_L gp, SSE_REG_EVT_SHADOW_STACK(\reg_evt) +.endm + /* Load task_scs_sp(current) to gp. */ .macro scs_load_current REG_L gp, TASK_TI_SCS_SP(tp) @@ -41,6 +46,8 @@ .endm .macro scs_load_irq_stack tmp .endm +.macro scs_load_sse_stack reg_evt +.endm .macro scs_load_current .endm .macro scs_load_current_if_task_changed prev diff --git a/arch/riscv/include/asm/sse.h b/arch/riscv/include/asm/sse.h new file mode 100644 index 000000000000..d3ce8c2b5221 --- /dev/null +++ b/arch/riscv/include/asm/sse.h @@ -0,0 +1,47 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2024 Rivos Inc. + */ +#ifndef __ASM_SSE_H +#define __ASM_SSE_H + +#include + +#ifdef CONFIG_RISCV_SBI_SSE + +struct sse_event_interrupted_state { + unsigned long a6; + unsigned long a7; +}; + +struct sse_event_arch_data { + void *stack; + void *shadow_stack; + unsigned long tmp; + struct sse_event_interrupted_state interrupted; + unsigned long interrupted_phys; + u32 evt_id; + unsigned int hart_id; + unsigned int cpu_id; +}; + +static inline bool sse_event_is_global(u32 evt) +{ + return !!(evt & SBI_SSE_EVENT_GLOBAL); +} + +void arch_sse_event_update_cpu(struct sse_event_arch_data *arch_evt, int c= pu); +int arch_sse_init_event(struct sse_event_arch_data *arch_evt, u32 evt_id, + int cpu); +void arch_sse_free_event(struct sse_event_arch_data *arch_evt); +int arch_sse_register_event(struct sse_event_arch_data *arch_evt); + +void sse_handle_event(struct sse_event_arch_data *arch_evt, + struct pt_regs *regs); +asmlinkage void handle_sse(void); +asmlinkage void do_sse(struct sse_event_arch_data *arch_evt, + struct pt_regs *reg); + +#endif + +#endif diff --git a/arch/riscv/include/asm/switch_to.h b/arch/riscv/include/asm/sw= itch_to.h index 0e71eb82f920..70e68e630216 100644 --- a/arch/riscv/include/asm/switch_to.h +++ b/arch/riscv/include/asm/switch_to.h @@ -88,6 +88,19 @@ static inline void __switch_to_envcfg(struct task_struct= *next) :: "r" (next->thread.envcfg) : "memory"); } =20 +#ifdef CONFIG_RISCV_SBI_SSE +DECLARE_PER_CPU(struct task_struct *, __sbi_sse_entry_task); + +static inline void __switch_sbi_sse_entry_task(struct task_struct *next) +{ + __this_cpu_write(__sbi_sse_entry_task, next); +} +#else +static inline void __switch_sbi_sse_entry_task(struct task_struct *next) +{ +} +#endif + extern struct task_struct *__switch_to(struct task_struct *, struct task_struct *); =20 @@ -122,6 +135,7 @@ do { \ if (switch_to_should_flush_icache(__next)) \ local_flush_icache_all(); \ __switch_to_envcfg(__next); \ + __switch_sbi_sse_entry_task(__next); \ ((last) =3D __switch_to(__prev, __next)); \ } while (0) =20 diff --git a/arch/riscv/include/asm/thread_info.h b/arch/riscv/include/asm/= thread_info.h index 836d80dd2921..c5e1159cedcf 100644 --- a/arch/riscv/include/asm/thread_info.h +++ b/arch/riscv/include/asm/thread_info.h @@ -36,6 +36,7 @@ #define OVERFLOW_STACK_SIZE SZ_4K =20 #define IRQ_STACK_SIZE THREAD_SIZE +#define SSE_STACK_SIZE THREAD_SIZE =20 #ifndef __ASSEMBLER__ =20 diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index f60fce69b725..22bc8bdac079 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -99,6 +99,7 @@ obj-$(CONFIG_DYNAMIC_FTRACE) +=3D mcount-dyn.o obj-$(CONFIG_PERF_EVENTS) +=3D perf_callchain.o obj-$(CONFIG_HAVE_PERF_REGS) +=3D perf_regs.o obj-$(CONFIG_RISCV_SBI) +=3D sbi.o sbi_ecall.o +obj-$(CONFIG_RISCV_SBI_SSE) +=3D sbi_sse.o sbi_sse_entry.o ifeq ($(CONFIG_RISCV_SBI), y) obj-$(CONFIG_SMP) +=3D sbi-ipi.o obj-$(CONFIG_SMP) +=3D cpu_ops_sbi.o diff --git a/arch/riscv/kernel/asm-offsets.c b/arch/riscv/kernel/asm-offset= s.c index 7d42d3b8a32a..dc456a4a0080 100644 --- a/arch/riscv/kernel/asm-offsets.c +++ b/arch/riscv/kernel/asm-offsets.c @@ -15,6 +15,8 @@ #include #include #include +#include +#include #include =20 void asm_offsets(void); @@ -529,4 +531,16 @@ void asm_offsets(void) DEFINE(FREGS_A6, offsetof(struct __arch_ftrace_regs, a6)); DEFINE(FREGS_A7, offsetof(struct __arch_ftrace_regs, a7)); #endif + +#ifdef CONFIG_RISCV_SBI_SSE + OFFSET(SSE_REG_EVT_STACK, sse_event_arch_data, stack); + OFFSET(SSE_REG_EVT_SHADOW_STACK, sse_event_arch_data, shadow_stack); + OFFSET(SSE_REG_EVT_TMP, sse_event_arch_data, tmp); + OFFSET(SSE_REG_HART_ID, sse_event_arch_data, hart_id); + OFFSET(SSE_REG_CPU_ID, sse_event_arch_data, cpu_id); + + DEFINE(SBI_EXT_SSE, SBI_EXT_SSE); + DEFINE(SBI_SSE_EVENT_COMPLETE, SBI_SSE_EVENT_COMPLETE); + DEFINE(ASM_NR_CPUS, NR_CPUS); +#endif } diff --git a/arch/riscv/kernel/sbi_sse.c b/arch/riscv/kernel/sbi_sse.c new file mode 100644 index 000000000000..f30ae9ff77c9 --- /dev/null +++ b/arch/riscv/kernel/sbi_sse.c @@ -0,0 +1,162 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Copyright (C) 2025 Rivos Inc. + */ +#include +#include +#include +#include + +#include +#include +#include +#include +#include + +DEFINE_PER_CPU(struct task_struct *, __sbi_sse_entry_task); + +void __weak sse_handle_event(struct sse_event_arch_data *arch_evt, struct = pt_regs *regs) +{ +} + +void do_sse(struct sse_event_arch_data *arch_evt, struct pt_regs *regs) +{ + nmi_enter(); + + /* Retrieve missing GPRs from SBI */ + sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_ATTR_READ, arch_evt->evt_id, + SBI_SSE_ATTR_INTERRUPTED_A6, + (SBI_SSE_ATTR_INTERRUPTED_A7 - SBI_SSE_ATTR_INTERRUPTED_A6) + 1, + arch_evt->interrupted_phys, 0, 0); + + memcpy(®s->a6, &arch_evt->interrupted, sizeof(arch_evt->interrupted)); + + sse_handle_event(arch_evt, regs); + + nmi_exit(); +} + +static void *alloc_to_stack_pointer(void *alloc) +{ + return alloc ? alloc + SSE_STACK_SIZE : NULL; +} + +static void *stack_pointer_to_alloc(void *stack) +{ + return stack - SSE_STACK_SIZE; +} + +#ifdef CONFIG_VMAP_STACK +static void *sse_stack_alloc(unsigned int cpu) +{ + void *stack =3D arch_alloc_vmap_stack(SSE_STACK_SIZE, cpu_to_node(cpu)); + + return alloc_to_stack_pointer(stack); +} + +static void sse_stack_free(void *stack) +{ + vfree(stack_pointer_to_alloc(stack)); +} + +static void arch_sse_stack_cpu_sync(struct sse_event_arch_data *arch_evt) +{ + void *p_stack =3D arch_evt->stack; + unsigned long stack =3D (unsigned long)stack_pointer_to_alloc(p_stack); + unsigned long stack_end =3D stack + SSE_STACK_SIZE; + + /* + * Flush the tlb to avoid taking any exception when accessing the + * vmapped stack inside the SSE handler + */ + if (sse_event_is_global(arch_evt->evt_id)) + flush_tlb_kernel_range(stack, stack_end); + else + local_flush_tlb_kernel_range(stack, stack_end); +} +#else /* CONFIG_VMAP_STACK */ +static void *sse_stack_alloc(unsigned int cpu) +{ + void *stack =3D kmalloc(SSE_STACK_SIZE, GFP_KERNEL); + + return alloc_to_stack_pointer(stack); +} + +static void sse_stack_free(void *stack) +{ + kfree(stack_pointer_to_alloc(stack)); +} + +static void arch_sse_stack_cpu_sync(struct sse_event_arch_data *arch_evt) = {} +#endif /* CONFIG_VMAP_STACK */ + +static int sse_init_scs(int cpu, struct sse_event_arch_data *arch_evt) +{ + void *stack; + + if (!scs_is_enabled()) + return 0; + + stack =3D scs_alloc(cpu_to_node(cpu)); + if (!stack) + return -ENOMEM; + + arch_evt->shadow_stack =3D stack; + + return 0; +} + +void arch_sse_event_update_cpu(struct sse_event_arch_data *arch_evt, int c= pu) +{ + arch_evt->cpu_id =3D cpu; + arch_evt->hart_id =3D cpuid_to_hartid_map(cpu); +} + +int arch_sse_init_event(struct sse_event_arch_data *arch_evt, u32 evt_id, + int cpu) +{ + void *stack; + + arch_evt->evt_id =3D evt_id; + stack =3D sse_stack_alloc(cpu); + if (!stack) + return -ENOMEM; + + arch_evt->stack =3D stack; + + if (sse_init_scs(cpu, arch_evt)) { + sse_stack_free(arch_evt->stack); + return -ENOMEM; + } + + if (sse_event_is_global(evt_id)) { + arch_evt->interrupted_phys =3D + virt_to_phys(&arch_evt->interrupted); + } else { + arch_evt->interrupted_phys =3D + per_cpu_ptr_to_phys(&arch_evt->interrupted); + } + + arch_sse_event_update_cpu(arch_evt, cpu); + + return 0; +} + +void arch_sse_free_event(struct sse_event_arch_data *arch_evt) +{ + scs_free(arch_evt->shadow_stack); + sse_stack_free(arch_evt->stack); +} + +int arch_sse_register_event(struct sse_event_arch_data *arch_evt) +{ + struct sbiret sret; + + arch_sse_stack_cpu_sync(arch_evt); + + sret =3D sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_REGISTER, arch_evt->evt_id, + (unsigned long)handle_sse, (unsigned long)arch_evt, 0, + 0, 0); + + return sbi_err_map_linux_errno(sret.error); +} diff --git a/arch/riscv/kernel/sbi_sse_entry.S b/arch/riscv/kernel/sbi_sse_= entry.S new file mode 100644 index 000000000000..b3f308a15912 --- /dev/null +++ b/arch/riscv/kernel/sbi_sse_entry.S @@ -0,0 +1,178 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2025 Rivos Inc. + */ + +#include +#include + +#include +#include +#include + +/* When entering handle_sse, the following registers are set: + * a6: contains the hartid + * a7: contains a sse_event_arch_data struct pointer + */ +SYM_CODE_START(handle_sse) + /* Save stack temporarily */ + REG_S sp, SSE_REG_EVT_TMP(a7) + /* Set entry stack */ + REG_L sp, SSE_REG_EVT_STACK(a7) + + addi sp, sp, -(PT_SIZE_ON_STACK) + REG_S ra, PT_RA(sp) + REG_S s0, PT_S0(sp) + REG_S s1, PT_S1(sp) + REG_S s2, PT_S2(sp) + REG_S s3, PT_S3(sp) + REG_S s4, PT_S4(sp) + REG_S s5, PT_S5(sp) + REG_S s6, PT_S6(sp) + REG_S s7, PT_S7(sp) + REG_S s8, PT_S8(sp) + REG_S s9, PT_S9(sp) + REG_S s10, PT_S10(sp) + REG_S s11, PT_S11(sp) + REG_S tp, PT_TP(sp) + REG_S t0, PT_T0(sp) + REG_S t1, PT_T1(sp) + REG_S t2, PT_T2(sp) + REG_S t3, PT_T3(sp) + REG_S t4, PT_T4(sp) + REG_S t5, PT_T5(sp) + REG_S t6, PT_T6(sp) + REG_S gp, PT_GP(sp) + REG_S a0, PT_A0(sp) + REG_S a1, PT_A1(sp) + REG_S a2, PT_A2(sp) + REG_S a3, PT_A3(sp) + REG_S a4, PT_A4(sp) + REG_S a5, PT_A5(sp) + + /* Retrieve entry sp */ + REG_L a4, SSE_REG_EVT_TMP(a7) + /* Save CSRs */ + csrr a0, CSR_EPC + csrr a1, CSR_SSTATUS + csrr a2, CSR_STVAL + csrr a3, CSR_SCAUSE + + REG_S a0, PT_EPC(sp) + REG_S a1, PT_STATUS(sp) + REG_S a2, PT_BADADDR(sp) + REG_S a3, PT_CAUSE(sp) + REG_S a4, PT_SP(sp) + + /* Disable user memory access and floating/vector computing */ + li t0, SR_SUM | SR_FS_VS + csrc CSR_STATUS, t0 + + load_global_pointer + scs_load_sse_stack a7 + +#ifdef CONFIG_SMP + lw t4, SSE_REG_HART_ID(a7) + lw t3, SSE_REG_CPU_ID(a7) + + bne t4, a6, .Lfind_hart_id_slowpath + +.Lcpu_id_found: +#else + mv t3, zero +#endif + + asm_per_cpu_with_cpu t2 __sbi_sse_entry_task t1 t3 + REG_L tp, 0(t2) + + mv a1, sp /* pt_regs on stack */ + + /* + * Save sscratch for restoration since we might have interrupted the + * kernel in early exception path and thus, we don't know the content of + * sscratch. + */ + csrrw s4, CSR_SSCRATCH, x0 + + mv a0, a7 + + call do_sse + + csrw CSR_SSCRATCH, s4 + + REG_L a0, PT_STATUS(sp) + REG_L a1, PT_EPC(sp) + REG_L a2, PT_BADADDR(sp) + REG_L a3, PT_CAUSE(sp) + csrw CSR_SSTATUS, a0 + csrw CSR_EPC, a1 + csrw CSR_STVAL, a2 + csrw CSR_SCAUSE, a3 + + REG_L ra, PT_RA(sp) + REG_L s0, PT_S0(sp) + REG_L s1, PT_S1(sp) + REG_L s2, PT_S2(sp) + REG_L s3, PT_S3(sp) + REG_L s4, PT_S4(sp) + REG_L s5, PT_S5(sp) + REG_L s6, PT_S6(sp) + REG_L s7, PT_S7(sp) + REG_L s8, PT_S8(sp) + REG_L s9, PT_S9(sp) + REG_L s10, PT_S10(sp) + REG_L s11, PT_S11(sp) + REG_L tp, PT_TP(sp) + REG_L t0, PT_T0(sp) + REG_L t1, PT_T1(sp) + REG_L t2, PT_T2(sp) + REG_L t3, PT_T3(sp) + REG_L t4, PT_T4(sp) + REG_L t5, PT_T5(sp) + REG_L t6, PT_T6(sp) + REG_L gp, PT_GP(sp) + REG_L a0, PT_A0(sp) + REG_L a1, PT_A1(sp) + REG_L a2, PT_A2(sp) + REG_L a3, PT_A3(sp) + REG_L a4, PT_A4(sp) + REG_L a5, PT_A5(sp) + + REG_L sp, PT_SP(sp) + + li a7, SBI_EXT_SSE + li a6, SBI_SSE_EVENT_COMPLETE + ecall + +#ifdef CONFIG_SMP +.Lfind_hart_id_slowpath: + + /* Restore current task struct from __sbi_sse_entry_task */ + li t1, ASM_NR_CPUS + /* Slowpath to find the CPU id associated to the hart id */ + la t0, __cpuid_to_hartid_map + +.Lhart_id_loop: + REG_L t2, 0(t0) + beq t2, a6, .Lcpu_id_found + + /* Increment pointer and CPU number */ + addi t3, t3, 1 + addi t0, t0, RISCV_SZPTR + bltu t3, t1, .Lhart_id_loop + + /* + * This should never happen since we expect the hart_id to match one + * of our CPU, but better be safe than sorry + */ + la tp, init_task + la a0, sse_hart_id_panic_string + la t0, panic + jalr t0 +#endif + +SYM_CODE_END(handle_sse) + +SYM_DATA_START_LOCAL(sse_hart_id_panic_string) + .ascii "Unable to match hart_id with cpu\0" +SYM_DATA_END(sse_hart_id_panic_string) --=20 2.43.0 From nobody Tue Dec 16 13:37:30 2025 Received: from mail-pl1-f182.google.com (mail-pl1-f182.google.com [209.85.214.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 813EF2D738F for ; Wed, 5 Nov 2025 08:29:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762331354; cv=none; b=B78B2EvkrkTf5eDRQH92j8cpyXAZRmzRUCGVHn0ZsKFMJnqmIgNjPAIljZJBKg3bllJMlaOY1Ciip9hnY0INsldBojLiTBHjHEWwqgqIMPEuTb4V9ATA6eks31KruGS2QFSR6vPkF/TrkiyaUy30NxU4rU77btXAnWhSf9Z2eCQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762331354; c=relaxed/simple; bh=7fglIJGknytXC7EQcwsjCzo4jWgRk8/Rh6eN6fCzY1g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=V6qs3De5j0zdgNltgTAbQUDzNpeIW5zA6nKq7aalylQA8Y9f9F8KnRJdjeF2amdky/GYlhTTdGJLk881+OA3Y0a07QkcP1QrsMTyPBL+C1zEvPNoRtlnHg8exErNIoEk6mRZ9jUN5peaHahHmo+kSc8N+T7gV+l4IWzM3X0YyJg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc.com header.i=@rivosinc.com header.b=GC7HSHT+; arc=none smtp.client-ip=209.85.214.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc.com header.i=@rivosinc.com header.b="GC7HSHT+" Received: by mail-pl1-f182.google.com with SMTP id d9443c01a7336-294fe7c2e69so61163065ad.0 for ; Wed, 05 Nov 2025 00:29:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc.com; s=google; t=1762331352; x=1762936152; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=mf41sNUeGzW5wXaLxNujZAvPXMLpV6g78NpsX8NDBNI=; b=GC7HSHT+OoxnJLV8B1cBL7pS/XBzZIpDaywURNnnw0LlZyBJJGX+abO8avxpkf0ou6 p0+wdwEfoA4vQWpfGdjTP/uQzxL7OLYsYEvDD0oDHKlgeKiL7U7W+eHt6tB0WgRTHLif u/cL82PyjLa5lUfZg+pn+2iFY+QeK3iDGaULWex6cNveju6Whycc2J4+/41oF8KpUT0D W+/7BqKEluUGqx5++zEVBVV0kOPbqc/SwG1wq7GZrZOqUN0ptfPvwCA3PtQr//WOZ68d KrFaveuWtE3PvdgvDQjbg8ZuDUyZzm1WnZ84Xfe1xxSxpzg49C+swx/sXXpOnurjURVC Ka4g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1762331352; x=1762936152; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=mf41sNUeGzW5wXaLxNujZAvPXMLpV6g78NpsX8NDBNI=; b=AX6pIHXzLuTvqxk6EYMk9BWT6CZqbU+33BjpE26b6O5fZgExxVm3pVJYsMF1h8x5Ml g5UvvfH0M793cJ6xeYkyJdvb8Mcus79Vu2NQ1JiTNiKMVUhWZxo4E77GBE1xzjgJ03HO 3jAZC3WEqZhbTID6yyvpRS96B0K0Hq24HZpC7dmkV4dqFEN69d0i4MtgR1TOD8J9xbuA fSnMY25hgFv7Qw/N0QU7TA9MOn8NRKRct570MrX73hTIsb1aw1H3cSvD3rkTP5vuxKDl SQavt+H6DdhctY8RvCXroDyDQUtJ+R5uxEB3tL8KycMcRXA0aVhmSMDIsN9IG3bkH7+c cavA== X-Forwarded-Encrypted: i=1; AJvYcCWvhS3ywsmBGthpWPC1BS+4YfuR+IfHkL+YK3shM3LfAvJ4cXNbSMDuYwZnn+JQeToaOW+qfXzTAqPwPH8=@vger.kernel.org X-Gm-Message-State: AOJu0Yxkl3kaaHWhXOKTkPS1zHOV17S/sFmmTr6M63U3IgBBfY6/nUWq aPBNrcvWdctNXe1hOUDo3iaoTjKRZ3Mtu/tceYys8hcARuHu+iTispV8arhOgcyrCkc= X-Gm-Gg: ASbGncsctAgMlrAELOwf+CE8ssY89/begcRbGVTWnzmrSshZltHrcXM2Kdx2IQA8jy2 a+6Z6ULil85fQu1nRj+ZAAAPqQBOMDQRzlqpwulnG4XqlYXtdp7hFrNuabNQM9a9ILBihU1KId2 mcaJ8gwpmLKn1oMM1U+yUBw9Wt5pcfzSJf8OxHloDG1WV4X6ijYQo34NHvCIEMmv4+7vZvgFQbC cqiuDRD5PNRA5mvc801YCncnsPnWAY1bmIC99vlNawbqlz0BTy4RfsHxMtWnCKCgPyYXjKIZC+I TnHNzwQVUBQoxKdLvJW4hiy1xIJhKK/c7YoU+NxomyE3hIIkhM7owtrwXpwP7PfsdCAHJdB1zPz Bml0jGuJw5z71OCSok/Lsx72ArXWChmhO+e8OIYq0YXTgaXHVQxkq693h X-Google-Smtp-Source: AGHT+IGaa9rkx0tEgDIGzvQq5Gbb3WmX8d59GIZdj0I5KA2BYoFxV6YfJVLJ2Oa0cCGOHw+LbSavnw== X-Received: by 2002:a17:902:ef09:b0:295:5d0b:e119 with SMTP id d9443c01a7336-2962adf02f0mr35769345ad.26.1762331351888; Wed, 05 Nov 2025 00:29:11 -0800 (PST) Received: from cleger.eu.int ([2001:41d0:420:f300::]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-29601a741e3sm51870875ad.98.2025.11.05.00.29.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Nov 2025 00:29:11 -0800 (PST) From: =?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?= To: Paul Walmsley , Palmer Dabbelt , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: =?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?= , Himanshu Chauhan , Anup Patel , Xu Lu , Atish Patra , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Yunhui Cui , Conor Dooley Subject: [PATCH v8 3/5] drivers: firmware: add riscv SSE support Date: Wed, 5 Nov 2025 08:26:35 +0000 Message-ID: <20251105082639.342973-4-cleger@rivosinc.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251105082639.342973-1-cleger@rivosinc.com> References: <20251105082639.342973-1-cleger@rivosinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Add driver level interface to use RISC-V SSE arch support. This interface allows registering SSE handlers, and receive them. This will be used by PMU and GHES driver. Co-developed-by: Himanshu Chauhan Signed-off-by: Himanshu Chauhan Signed-off-by: Cl=C3=A9ment L=C3=A9ger Acked-by: Conor Dooley --- MAINTAINERS | 15 + drivers/firmware/Kconfig | 1 + drivers/firmware/Makefile | 1 + drivers/firmware/riscv/Kconfig | 15 + drivers/firmware/riscv/Makefile | 3 + drivers/firmware/riscv/riscv_sbi_sse.c | 694 +++++++++++++++++++++++++ include/linux/riscv_sbi_sse.h | 57 ++ 7 files changed, 786 insertions(+) create mode 100644 drivers/firmware/riscv/Kconfig create mode 100644 drivers/firmware/riscv/Makefile create mode 100644 drivers/firmware/riscv/riscv_sbi_sse.c create mode 100644 include/linux/riscv_sbi_sse.h diff --git a/MAINTAINERS b/MAINTAINERS index 46bd8e033042..f664fca6c9df 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -22105,6 +22105,13 @@ T: git git://git.kernel.org/pub/scm/linux/kernel/g= it/iommu/linux.git F: Documentation/devicetree/bindings/iommu/riscv,iommu.yaml F: drivers/iommu/riscv/ =20 +RISC-V FIRMWARE DRIVERS +M: Conor Dooley +L: linux-riscv@lists.infradead.org +S: Maintained +T: git git://git.kernel.org/pub/scm/linux/kernel/git/conor/linux.git +F: drivers/firmware/riscv/* + RISC-V MICROCHIP FPGA SUPPORT M: Conor Dooley M: Daire McNamara @@ -22184,6 +22191,14 @@ F: arch/riscv/boot/dts/spacemit/ N: spacemit K: spacemit =20 +RISC-V SSE DRIVER +M: Cl=C3=A9ment L=C3=A9ger +R: Himanshu Chauhan +L: linux-riscv@lists.infradead.org +S: Maintained +F: drivers/firmware/riscv/riscv_sse.c +F: include/linux/riscv_sse.h + RISC-V THEAD SoC SUPPORT M: Drew Fustini M: Guo Ren diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig index bbd2155d8483..1894df87b08e 100644 --- a/drivers/firmware/Kconfig +++ b/drivers/firmware/Kconfig @@ -294,6 +294,7 @@ source "drivers/firmware/meson/Kconfig" source "drivers/firmware/microchip/Kconfig" source "drivers/firmware/psci/Kconfig" source "drivers/firmware/qcom/Kconfig" +source "drivers/firmware/riscv/Kconfig" source "drivers/firmware/samsung/Kconfig" source "drivers/firmware/smccc/Kconfig" source "drivers/firmware/tegra/Kconfig" diff --git a/drivers/firmware/Makefile b/drivers/firmware/Makefile index 4ddec2820c96..6cdd84570ea7 100644 --- a/drivers/firmware/Makefile +++ b/drivers/firmware/Makefile @@ -34,6 +34,7 @@ obj-y +=3D efi/ obj-y +=3D imx/ obj-y +=3D psci/ obj-y +=3D qcom/ +obj-y +=3D riscv/ obj-y +=3D samsung/ obj-y +=3D smccc/ obj-y +=3D tegra/ diff --git a/drivers/firmware/riscv/Kconfig b/drivers/firmware/riscv/Kconfig new file mode 100644 index 000000000000..ed5b663ac5f9 --- /dev/null +++ b/drivers/firmware/riscv/Kconfig @@ -0,0 +1,15 @@ +# SPDX-License-Identifier: GPL-2.0-only +menu "Risc-V Specific firmware drivers" +depends on RISCV + +config RISCV_SBI_SSE + bool "Enable SBI Supervisor Software Events support" + depends on RISCV_SBI + default y + help + The Supervisor Software Events support allows the SBI to deliver + NMI-like notifications to the supervisor mode software. When enabled, + this option provides support to register callbacks on specific SSE + events. + +endmenu diff --git a/drivers/firmware/riscv/Makefile b/drivers/firmware/riscv/Makef= ile new file mode 100644 index 000000000000..c8795d4bbb2e --- /dev/null +++ b/drivers/firmware/riscv/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 + +obj-$(CONFIG_RISCV_SBI_SSE) +=3D riscv_sbi_sse.o diff --git a/drivers/firmware/riscv/riscv_sbi_sse.c b/drivers/firmware/risc= v/riscv_sbi_sse.c new file mode 100644 index 000000000000..c7f29b10cdfb --- /dev/null +++ b/drivers/firmware/riscv/riscv_sbi_sse.c @@ -0,0 +1,694 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Copyright (C) 2025 Rivos Inc. + */ + +#define pr_fmt(fmt) "sse: " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +struct sse_event { + struct list_head list; + u32 evt_id; + u32 priority; + sse_event_handler_fn *handler; + void *handler_arg; + /* Only valid for global events */ + unsigned int cpu; + + union { + struct sse_registered_event *global; + struct sse_registered_event __percpu *local; + }; +}; + +static int sse_hp_state; +static bool sse_available __ro_after_init; +static DEFINE_SPINLOCK(events_list_lock); +static LIST_HEAD(events); +static DEFINE_MUTEX(sse_mutex); + +struct sse_registered_event { + struct sse_event_arch_data arch; + struct sse_event *event; + unsigned long attr; + bool is_enabled; +}; + +void sse_handle_event(struct sse_event_arch_data *arch_event, + struct pt_regs *regs) +{ + int ret; + struct sse_registered_event *reg_evt =3D + container_of(arch_event, struct sse_registered_event, arch); + struct sse_event *evt =3D reg_evt->event; + + ret =3D evt->handler(evt->evt_id, evt->handler_arg, regs); + if (ret) + pr_warn("event %x handler failed with error %d\n", evt->evt_id, ret); +} + +static struct sse_event *sse_event_get(u32 evt) +{ + struct sse_event *event =3D NULL; + + scoped_guard(spinlock, &events_list_lock) { + list_for_each_entry(event, &events, list) { + if (event->evt_id =3D=3D evt) + return event; + } + } + + return NULL; +} + +static phys_addr_t sse_event_get_attr_phys(struct sse_registered_event *re= g_evt) +{ + phys_addr_t phys; + void *addr =3D ®_evt->attr; + + if (sse_event_is_global(reg_evt->event->evt_id)) + phys =3D virt_to_phys(addr); + else + phys =3D per_cpu_ptr_to_phys(addr); + + return phys; +} + +static struct sse_registered_event *sse_get_reg_evt(struct sse_event *even= t) +{ + if (sse_event_is_global(event->evt_id)) + return event->global; + else + return per_cpu_ptr(event->local, smp_processor_id()); +} + +static int sse_sbi_event_func(struct sse_event *event, unsigned long func) +{ + struct sbiret ret; + u32 evt =3D event->evt_id; + struct sse_registered_event *reg_evt =3D sse_get_reg_evt(event); + + ret =3D sbi_ecall(SBI_EXT_SSE, func, evt, 0, 0, 0, 0, 0); + if (ret.error) { + pr_warn("Failed to execute func %lx, event %x, error %ld\n", + func, evt, ret.error); + return sbi_err_map_linux_errno(ret.error); + } + + if (func =3D=3D SBI_SSE_EVENT_DISABLE) + reg_evt->is_enabled =3D false; + else if (func =3D=3D SBI_SSE_EVENT_ENABLE) + reg_evt->is_enabled =3D true; + + return 0; +} + +int sse_event_disable_local(struct sse_event *event) +{ + return sse_sbi_event_func(event, SBI_SSE_EVENT_DISABLE); +} +EXPORT_SYMBOL_GPL(sse_event_disable_local); + +int sse_event_enable_local(struct sse_event *event) +{ + return sse_sbi_event_func(event, SBI_SSE_EVENT_ENABLE); +} +EXPORT_SYMBOL_GPL(sse_event_enable_local); + +static int sse_event_attr_get_no_lock(struct sse_registered_event *reg_evt, + unsigned long attr_id, unsigned long *val) +{ + struct sbiret sret; + u32 evt =3D reg_evt->event->evt_id; + unsigned long phys; + + phys =3D sse_event_get_attr_phys(reg_evt); + + sret =3D sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_ATTR_READ, evt, attr_id, 1, + phys, 0, 0); + if (sret.error) { + pr_debug("Failed to get event %x attr %lx, error %ld\n", evt, + attr_id, sret.error); + return sbi_err_map_linux_errno(sret.error); + } + + *val =3D reg_evt->attr; + + return 0; +} + +static int sse_event_attr_set_nolock(struct sse_registered_event *reg_evt, + unsigned long attr_id, unsigned long val) +{ + struct sbiret sret; + u32 evt =3D reg_evt->event->evt_id; + unsigned long phys; + + reg_evt->attr =3D val; + phys =3D sse_event_get_attr_phys(reg_evt); + + sret =3D sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_ATTR_WRITE, evt, attr_id, 1, + phys, 0, 0); + if (sret.error) + pr_debug("Failed to set event %x attr %lx, error %ld\n", evt, + attr_id, sret.error); + + return sbi_err_map_linux_errno(sret.error); +} + +static void sse_global_event_update_cpu(struct sse_event *event, + unsigned int cpu) +{ + struct sse_registered_event *reg_evt =3D event->global; + + event->cpu =3D cpu; + arch_sse_event_update_cpu(®_evt->arch, cpu); +} + +static int sse_event_set_target_cpu_nolock(struct sse_event *event, + unsigned int cpu) +{ + unsigned long hart_id =3D cpuid_to_hartid_map(cpu); + struct sse_registered_event *reg_evt =3D event->global; + u32 evt =3D event->evt_id; + bool was_enabled; + int ret; + + if (!sse_event_is_global(evt)) + return -EINVAL; + + was_enabled =3D reg_evt->is_enabled; + if (was_enabled) + sse_event_disable_local(event); + + ret =3D sse_event_attr_set_nolock(reg_evt, SBI_SSE_ATTR_PREFERRED_HART, + hart_id); + if (ret =3D=3D 0) + sse_global_event_update_cpu(event, cpu); + + if (was_enabled) + sse_event_enable_local(event); + + return 0; +} + +int sse_event_set_target_cpu(struct sse_event *event, unsigned int cpu) +{ + int ret; + + scoped_guard(mutex, &sse_mutex) { + scoped_guard(cpus_read_lock) { + if (!cpu_online(cpu)) + return -EINVAL; + + ret =3D sse_event_set_target_cpu_nolock(event, cpu); + } + } + + return ret; +} +EXPORT_SYMBOL_GPL(sse_event_set_target_cpu); + +static int sse_event_init_registered(unsigned int cpu, + struct sse_registered_event *reg_evt, + struct sse_event *event) +{ + reg_evt->event =3D event; + + return arch_sse_init_event(®_evt->arch, event->evt_id, cpu); +} + +static void sse_event_free_registered(struct sse_registered_event *reg_evt) +{ + arch_sse_free_event(®_evt->arch); +} + +static int sse_event_alloc_global(struct sse_event *event) +{ + int err; + struct sse_registered_event *reg_evt; + + reg_evt =3D kzalloc(sizeof(*reg_evt), GFP_KERNEL); + if (!reg_evt) + return -ENOMEM; + + event->global =3D reg_evt; + err =3D sse_event_init_registered(smp_processor_id(), reg_evt, event); + if (err) + kfree(reg_evt); + + return err; +} + +static int sse_event_alloc_local(struct sse_event *event) +{ + int err; + unsigned int cpu, err_cpu; + struct sse_registered_event *reg_evt; + struct sse_registered_event __percpu *reg_evts; + + reg_evts =3D alloc_percpu(struct sse_registered_event); + if (!reg_evts) + return -ENOMEM; + + event->local =3D reg_evts; + + for_each_possible_cpu(cpu) { + reg_evt =3D per_cpu_ptr(reg_evts, cpu); + err =3D sse_event_init_registered(cpu, reg_evt, event); + if (err) { + err_cpu =3D cpu; + goto err_free_per_cpu; + } + } + + return 0; + +err_free_per_cpu: + for_each_possible_cpu(cpu) { + if (cpu =3D=3D err_cpu) + break; + reg_evt =3D per_cpu_ptr(reg_evts, cpu); + sse_event_free_registered(reg_evt); + } + + free_percpu(reg_evts); + + return err; +} + +static struct sse_event *sse_event_alloc(u32 evt, u32 priority, + sse_event_handler_fn *handler, + void *arg) +{ + int err; + struct sse_event *event; + + event =3D kzalloc(sizeof(*event), GFP_KERNEL); + if (!event) + return ERR_PTR(-ENOMEM); + + event->evt_id =3D evt; + event->priority =3D priority; + event->handler_arg =3D arg; + event->handler =3D handler; + + if (sse_event_is_global(evt)) + err =3D sse_event_alloc_global(event); + else + err =3D sse_event_alloc_local(event); + + if (err) { + kfree(event); + return ERR_PTR(err); + } + + return event; +} + +static int sse_sbi_register_event(struct sse_event *event, + struct sse_registered_event *reg_evt) +{ + int ret; + + ret =3D sse_event_attr_set_nolock(reg_evt, SBI_SSE_ATTR_PRIO, + event->priority); + if (ret) + return ret; + + return arch_sse_register_event(®_evt->arch); +} + +static int sse_event_register_local(struct sse_event *event) +{ + int ret; + struct sse_registered_event *reg_evt; + + reg_evt =3D per_cpu_ptr(event->local, smp_processor_id()); + ret =3D sse_sbi_register_event(event, reg_evt); + if (ret) + pr_debug("Failed to register event %x: err %d\n", event->evt_id, + ret); + + return ret; +} + +static int sse_sbi_unregister_event(struct sse_event *event) +{ + return sse_sbi_event_func(event, SBI_SSE_EVENT_UNREGISTER); +} + +struct sse_per_cpu_evt { + struct sse_event *event; + unsigned long func; + cpumask_t error; +}; + +static void sse_event_per_cpu_func(void *info) +{ + int ret; + struct sse_per_cpu_evt *cpu_evt =3D info; + + if (cpu_evt->func =3D=3D SBI_SSE_EVENT_REGISTER) + ret =3D sse_event_register_local(cpu_evt->event); + else + ret =3D sse_sbi_event_func(cpu_evt->event, cpu_evt->func); + + if (ret) + cpumask_set_cpu(smp_processor_id(), &cpu_evt->error); +} + +static void sse_event_free(struct sse_event *event) +{ + unsigned int cpu; + struct sse_registered_event *reg_evt; + + if (sse_event_is_global(event->evt_id)) { + sse_event_free_registered(event->global); + kfree(event->global); + } else { + for_each_possible_cpu(cpu) { + reg_evt =3D per_cpu_ptr(event->local, cpu); + sse_event_free_registered(reg_evt); + } + free_percpu(event->local); + } + + kfree(event); +} + +static int sse_on_each_cpu(struct sse_event *event, unsigned long func, + unsigned long revert_func) +{ + struct sse_per_cpu_evt cpu_evt; + + cpu_evt.event =3D event; + cpumask_clear(&cpu_evt.error); + cpu_evt.func =3D func; + on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1); + /* + * If there are some error reported by CPUs, revert event state on the + * other ones + */ + if (!cpumask_empty(&cpu_evt.error)) { + cpumask_t revert; + + cpumask_andnot(&revert, cpu_online_mask, &cpu_evt.error); + cpu_evt.func =3D revert_func; + on_each_cpu_mask(&revert, sse_event_per_cpu_func, &cpu_evt, 1); + + return -EIO; + } + + return 0; +} + +int sse_event_enable(struct sse_event *event) +{ + int ret =3D 0; + + scoped_guard(mutex, &sse_mutex) { + scoped_guard(cpus_read_lock) { + if (sse_event_is_global(event->evt_id)) { + ret =3D sse_event_enable_local(event); + } else { + ret =3D sse_on_each_cpu(event, + SBI_SSE_EVENT_ENABLE, + SBI_SSE_EVENT_DISABLE); + } + } + } + return ret; +} +EXPORT_SYMBOL_GPL(sse_event_enable); + +static int sse_events_mask(void) +{ + struct sbiret ret; + + ret =3D sbi_ecall(SBI_EXT_SSE, SBI_SSE_HART_MASK, 0, 0, 0, 0, 0, 0); + + return sbi_err_map_linux_errno(ret.error); +} + +static int sse_events_unmask(void) +{ + struct sbiret ret; + + ret =3D sbi_ecall(SBI_EXT_SSE, SBI_SSE_HART_UNMASK, 0, 0, 0, 0, 0, 0); + + return sbi_err_map_linux_errno(ret.error); +} + +static void sse_event_disable_nolock(struct sse_event *event) +{ + struct sse_per_cpu_evt cpu_evt; + + if (sse_event_is_global(event->evt_id)) { + sse_event_disable_local(event); + } else { + cpu_evt.event =3D event; + cpu_evt.func =3D SBI_SSE_EVENT_DISABLE; + on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1); + } +} + +void sse_event_disable(struct sse_event *event) +{ + scoped_guard(mutex, &sse_mutex) { + scoped_guard(cpus_read_lock) { + sse_event_disable_nolock(event); + } + } +} +EXPORT_SYMBOL_GPL(sse_event_disable); + +struct sse_event *sse_event_register(u32 evt, u32 priority, + sse_event_handler_fn *handler, void *arg) +{ + struct sse_event *event; + int cpu; + int ret =3D 0; + + if (!sse_available) + return ERR_PTR(-EOPNOTSUPP); + + guard(mutex)(&sse_mutex); + if (sse_event_get(evt)) + return ERR_PTR(-EEXIST); + + event =3D sse_event_alloc(evt, priority, handler, arg); + if (IS_ERR(event)) + return event; + + scoped_guard(cpus_read_lock) { + if (sse_event_is_global(evt)) { + unsigned long preferred_hart; + + ret =3D sse_event_attr_get_no_lock(event->global, + SBI_SSE_ATTR_PREFERRED_HART, + &preferred_hart); + if (ret) + goto err_event_free; + + cpu =3D riscv_hartid_to_cpuid(preferred_hart); + sse_global_event_update_cpu(event, cpu); + + ret =3D sse_sbi_register_event(event, event->global); + if (ret) + goto err_event_free; + + } else { + ret =3D sse_on_each_cpu(event, SBI_SSE_EVENT_REGISTER, + SBI_SSE_EVENT_DISABLE); + if (ret) + goto err_event_free; + } + } + + scoped_guard(spinlock, &events_list_lock) + list_add(&event->list, &events); + + return event; + +err_event_free: + sse_event_free(event); + + return ERR_PTR(ret); +} +EXPORT_SYMBOL_GPL(sse_event_register); + +static void sse_event_unregister_nolock(struct sse_event *event) +{ + struct sse_per_cpu_evt cpu_evt; + + if (sse_event_is_global(event->evt_id)) { + sse_sbi_unregister_event(event); + } else { + cpu_evt.event =3D event; + cpu_evt.func =3D SBI_SSE_EVENT_UNREGISTER; + on_each_cpu(sse_event_per_cpu_func, &cpu_evt, 1); + } +} + +void sse_event_unregister(struct sse_event *event) +{ + scoped_guard(mutex, &sse_mutex) { + scoped_guard(cpus_read_lock) + sse_event_unregister_nolock(event); + + scoped_guard(spinlock, &events_list_lock) + list_del(&event->list); + + sse_event_free(event); + } +} +EXPORT_SYMBOL_GPL(sse_event_unregister); + +static int sse_cpu_online(unsigned int cpu) +{ + struct sse_event *event; + + scoped_guard(spinlock, &events_list_lock) { + list_for_each_entry(event, &events, list) { + if (sse_event_is_global(event->evt_id)) + continue; + + sse_event_register_local(event); + if (sse_get_reg_evt(event)) + sse_event_enable_local(event); + } + } + + /* Ready to handle events. Unmask SSE. */ + return sse_events_unmask(); +} + +static int sse_cpu_teardown(unsigned int cpu) +{ + int ret =3D 0; + unsigned int next_cpu; + struct sse_event *event; + struct sse_registered_event *reg_evt; + + /* Mask the sse events */ + ret =3D sse_events_mask(); + if (ret) + return ret; + + scoped_guard(spinlock, &events_list_lock) { + list_for_each_entry(event, &events, list) { + /* Disable local event on current cpu */ + if (!sse_event_is_global(event->evt_id)) { + reg_evt =3D sse_get_reg_evt(event); + if (reg_evt->is_enabled) + sse_event_disable_local(event); + + sse_sbi_unregister_event(event); + continue; + } + + if (event->cpu !=3D smp_processor_id()) + continue; + + /* Update destination hart for global event */ + next_cpu =3D cpumask_any_but(cpu_online_mask, cpu); + ret =3D sse_event_set_target_cpu_nolock(event, next_cpu); + } + } + + return ret; +} + +static int sse_pm_notifier(struct notifier_block *nb, unsigned long action, + void *data) +{ + WARN_ON_ONCE(preemptible()); + + switch (action) { + case CPU_PM_ENTER: + sse_events_mask(); + break; + case CPU_PM_EXIT: + case CPU_PM_ENTER_FAILED: + sse_events_unmask(); + break; + default: + return NOTIFY_DONE; + } + + return NOTIFY_OK; +} + +static struct notifier_block sse_pm_nb =3D { + .notifier_call =3D sse_pm_notifier, +}; + +/* + * Mask all CPUs and unregister all events on panic, reboot or kexec. + */ +static int sse_reboot_notifier(struct notifier_block *nb, unsigned long ac= tion, + void *data) +{ + cpuhp_remove_state(sse_hp_state); + + return NOTIFY_OK; +} + +static struct notifier_block sse_reboot_nb =3D { + .notifier_call =3D sse_reboot_notifier, +}; + +static int __init sse_init(void) +{ + int ret; + + if (sbi_probe_extension(SBI_EXT_SSE) <=3D 0) { + pr_info("Missing SBI SSE extension\n"); + return -EOPNOTSUPP; + } + pr_info("SBI SSE extension detected\n"); + + ret =3D cpu_pm_register_notifier(&sse_pm_nb); + if (ret) { + pr_warn("Failed to register CPU PM notifier...\n"); + return ret; + } + + ret =3D register_reboot_notifier(&sse_reboot_nb); + if (ret) { + pr_warn("Failed to register reboot notifier...\n"); + goto remove_cpupm; + } + + ret =3D cpuhp_setup_state(CPUHP_AP_ONLINE_DYN, "riscv/sse:online", + sse_cpu_online, sse_cpu_teardown); + if (ret < 0) + goto remove_reboot; + + sse_hp_state =3D ret; + sse_available =3D true; + + return 0; + +remove_reboot: + unregister_reboot_notifier(&sse_reboot_nb); + +remove_cpupm: + cpu_pm_unregister_notifier(&sse_pm_nb); + + return ret; +} +arch_initcall(sse_init); diff --git a/include/linux/riscv_sbi_sse.h b/include/linux/riscv_sbi_sse.h new file mode 100644 index 000000000000..84165cb5f2d0 --- /dev/null +++ b/include/linux/riscv_sbi_sse.h @@ -0,0 +1,57 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (C) 2025 Rivos Inc. + */ + +#ifndef __LINUX_RISCV_SBI_SSE_H +#define __LINUX_RISCV_SBI_SSE_H + +#include +#include + +struct sse_event; +struct pt_regs; + +typedef int (sse_event_handler_fn)(u32 event_num, void *arg, + struct pt_regs *regs); + +#ifdef CONFIG_RISCV_SBI_SSE + +struct sse_event *sse_event_register(u32 event_num, u32 priority, + sse_event_handler_fn *handler, void *arg); + +void sse_event_unregister(struct sse_event *evt); + +int sse_event_set_target_cpu(struct sse_event *sse_evt, unsigned int cpu); + +int sse_event_enable(struct sse_event *sse_evt); + +void sse_event_disable(struct sse_event *sse_evt); + +int sse_event_enable_local(struct sse_event *sse_evt); +int sse_event_disable_local(struct sse_event *sse_evt); + +#else +static inline struct sse_event *sse_event_register(u32 event_num, u32 prio= rity, + sse_event_handler_fn *handler, + void *arg) +{ + return ERR_PTR(-EOPNOTSUPP); +} + +static inline void sse_event_unregister(struct sse_event *evt) {} + +static inline int sse_event_set_target_cpu(struct sse_event *sse_evt, + unsigned int cpu) +{ + return -EOPNOTSUPP; +} + +static inline int sse_event_enable(struct sse_event *sse_evt) +{ + return -EOPNOTSUPP; +} + +static inline void sse_event_disable(struct sse_event *sse_evt) {} +#endif +#endif /* __LINUX_RISCV_SBI_SSE_H */ --=20 2.43.0 From nobody Tue Dec 16 13:37:30 2025 Received: from mail-pl1-f171.google.com (mail-pl1-f171.google.com [209.85.214.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 42D9F2D9EC8 for ; Wed, 5 Nov 2025 08:29:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762331363; cv=none; b=A4I0JmH0O6dYVAxxJ+8p3GiNXwsgtfnK/ii+rhkutxBH4PSBULktZaEEt2JY5luQRtfkbgYonYpZSjevvcv2LEyYKCuqmg6xhhsWrxGGQj+oFa8P5mN1VssoNeYtdPoYecNE5PYzXav5txjDyuLxX8m9FwPQrjq3Oq3Qzz2ngBw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762331363; c=relaxed/simple; bh=rko0iUJF0zDKww/HfwrbBNAFt59i3B+7ifqQ30itxtc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=UeLrqaHFrULj2cJabvWgVgpp2u5cThB2zYNTSPQeuG+hGxVPMrWRHwGO3reUfIDWcprfWS74ZnIYaRlPQGhKobSi84SYc4iKHlNaSutdx/B0UCyto/QpZBOaHr5wK6E+3gIeSzCzWOrHoh28Hg4e0LSIFuQbXEcQ7rO9mKQM5so= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc.com header.i=@rivosinc.com header.b=ZPoTlw0S; arc=none smtp.client-ip=209.85.214.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc.com header.i=@rivosinc.com header.b="ZPoTlw0S" Received: by mail-pl1-f171.google.com with SMTP id d9443c01a7336-2957850c63bso6891665ad.0 for ; Wed, 05 Nov 2025 00:29:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc.com; s=google; t=1762331360; x=1762936160; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UpAlHv9OQWYvUlKWgP1gdNckfEi918WeQvUwQAzIeaA=; b=ZPoTlw0Skc3bVUQAODr1x5Vj5I4QA2l0s/6koDHSZuxHTx2zChsGAEbUe4inVbHS0n 2iiyEgqtSPdYl4Arq0EKcjHpCYLaiLhS32f8oFC0RBsku/JrDnGjLeuGa+/cySRcZ3fA +kIIWTNFq7cezsKV8wZ2hGLeZ7qFglqhHq05BE4LvEje2FNcoVmj32HjPLLF4d+qoxEd 2y2s1RJ3JNpyeKul+HSBCejfdu50wt7pZ4qLAL5vXN63SwaNIfRJOMaZnEDFVoZGyCsb Kgo/kpucsh7Q18G8ytcdABzeAKnKuzyphYI9ydEx9YfEXuiI1I0inraEQosKXghkDCe+ AgSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1762331360; x=1762936160; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UpAlHv9OQWYvUlKWgP1gdNckfEi918WeQvUwQAzIeaA=; b=Oj1INvCYw1hYkHVEI4jEe9EHiqJxhcItS12FRXuu4TK+iTHNHvVpVApKRg6ArkoVrt lnwKce51WCmGQ1c3uhp6LtTHUy8Mx64rNOoJOj8Y9nPddeHfj4iWFLqxTQgUzKT/O8G4 vCSdrx4za8+hx94H5OsdwOiz3t/buaIe1pUOdy+luat2I+8u/Eqr4mWsbSj7dkCeAjlu ctxC1Ns+DEKLaz4XherSnpeNqfdXrMp4dQzKwBpgz8bNXqkDtORkXyd6TLsXDQoLzx5L 2/Iuncxc4lDR2i5nmuAcpGIn9gBM1Hjz738/FB4wphY6bXSLhhdLuBAEvvQtwGkuNSm/ QwXg== X-Forwarded-Encrypted: i=1; AJvYcCVlGAMlx+NziPa4+nRC4r+ckecmB2xAgQvGL0PMJaKg8ZzKP9+qUvprVDYjBQEfAc2nR/kUwlQM3+sWOUQ=@vger.kernel.org X-Gm-Message-State: AOJu0YyJ0eLFINWCg2LREDehhlNPueuLjtszesrUYFPrRv+9NCv6TijT ktqtR6XLsmhlx7Lf1NrKTqKysn6FZgVB7CLXOrrXys12fh9lEGkkOpmicamczbjrsMg= X-Gm-Gg: ASbGncuJdPTO9bLCBhbHLdz+n70BA/6JlE9cXWwjCcek0vvknPEa/Ae1lw0Yn+Rwa7s Vrqdx4XM9TH8bHzzgWW+TeURHIwFfkovcCwtf6P/LQyTiXqpZkN6UUxn5gtmo5+dTjxwHwThZ6h xbOFzyW60iErCIdteeg0VgwhZU3Cr4lQ3imIEWKmUVlCM98hbxQgA+kYmL8bXU+F0/qIHRx3pbz 98OHZ94/I1T+RWgav7hjU46jyL8lhl7yFaL7wuMEpeffEmUrPDzxexU0QCTi0CRI8zlcydH3tJB xQi3HFasl+cSIzBUjSw9jlHF5T+rZBCMG5K4rpLrXll49utIKeYfcXZOY7XHFSv1VjgScMX2Dyi HseZIfVOA13NxN32joh7eaRnsa0gvX1lufyxZc8Une/XiAQkuec5UzrPK X-Google-Smtp-Source: AGHT+IF2Z6homljqV5rqhE78buYnfAbixEFj/1q40usfOdn/g6EqgmSRtkgSbkwfy2jLGxHxkCHQEw== X-Received: by 2002:a17:902:e5cb:b0:273:ef4c:60f3 with SMTP id d9443c01a7336-295f937c8a7mr84723755ad.4.1762331360459; Wed, 05 Nov 2025 00:29:20 -0800 (PST) Received: from cleger.eu.int ([2001:41d0:420:f300::]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-29601a741e3sm51870875ad.98.2025.11.05.00.29.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Nov 2025 00:29:19 -0800 (PST) From: =?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?= To: Paul Walmsley , Palmer Dabbelt , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: =?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?= , Himanshu Chauhan , Anup Patel , Xu Lu , Atish Patra , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Yunhui Cui Subject: [PATCH v8 4/5] perf: RISC-V: add support for SSE event Date: Wed, 5 Nov 2025 08:26:36 +0000 Message-ID: <20251105082639.342973-5-cleger@rivosinc.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251105082639.342973-1-cleger@rivosinc.com> References: <20251105082639.342973-1-cleger@rivosinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable In order to use SSE within PMU drivers, register a SSE handler for the local PMU event. Reuse the existing overflow IRQ handler and pass appropriate pt_regs. Add a config option RISCV_PMU_SSE to select event delivery via SSE events. Signed-off-by: Cl=C3=A9ment L=C3=A9ger --- drivers/perf/Kconfig | 10 +++++ drivers/perf/riscv_pmu.c | 23 +++++++++++ drivers/perf/riscv_pmu_sbi.c | 71 +++++++++++++++++++++++++++++----- include/linux/perf/riscv_pmu.h | 5 +++ 4 files changed, 99 insertions(+), 10 deletions(-) diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig index 638321fc9800..d6ffc2b036e5 100644 --- a/drivers/perf/Kconfig +++ b/drivers/perf/Kconfig @@ -105,6 +105,16 @@ config RISCV_PMU_SBI full perf feature support i.e. counter overflow, privilege mode filtering, counter configuration. =20 +config RISCV_PMU_SBI_SSE + depends on RISCV_PMU && RISCV_SBI_SSE + bool "RISC-V PMU SSE events" + default n + help + Say y if you want to use SSE events to deliver PMU interrupts. This + provides a way to profile the kernel at any level by using NMI-like + SSE events. SSE events being really intrusive, this option allows + to select it only if needed. + config STARFIVE_STARLINK_PMU depends on ARCH_STARFIVE || COMPILE_TEST depends on 64BIT diff --git a/drivers/perf/riscv_pmu.c b/drivers/perf/riscv_pmu.c index 7644147d50b4..dda2814801c0 100644 --- a/drivers/perf/riscv_pmu.c +++ b/drivers/perf/riscv_pmu.c @@ -13,6 +13,7 @@ #include #include #include +#include #include #include =20 @@ -254,6 +255,24 @@ void riscv_pmu_start(struct perf_event *event, int fla= gs) perf_event_update_userpage(event); } =20 +#ifdef CONFIG_RISCV_PMU_SBI_SSE +static void riscv_pmu_disable(struct pmu *pmu) +{ + struct riscv_pmu *rvpmu =3D to_riscv_pmu(pmu); + + if (rvpmu->sse_evt) + sse_event_disable_local(rvpmu->sse_evt); +} + +static void riscv_pmu_enable(struct pmu *pmu) +{ + struct riscv_pmu *rvpmu =3D to_riscv_pmu(pmu); + + if (rvpmu->sse_evt) + sse_event_enable_local(rvpmu->sse_evt); +} +#endif + static int riscv_pmu_add(struct perf_event *event, int flags) { struct riscv_pmu *rvpmu =3D to_riscv_pmu(event->pmu); @@ -411,6 +430,10 @@ struct riscv_pmu *riscv_pmu_alloc(void) .event_mapped =3D riscv_pmu_event_mapped, .event_unmapped =3D riscv_pmu_event_unmapped, .event_idx =3D riscv_pmu_event_idx, +#ifdef CONFIG_RISCV_PMU_SBI_SSE + .pmu_enable =3D riscv_pmu_enable, + .pmu_disable =3D riscv_pmu_disable, +#endif .add =3D riscv_pmu_add, .del =3D riscv_pmu_del, .start =3D riscv_pmu_start, diff --git a/drivers/perf/riscv_pmu_sbi.c b/drivers/perf/riscv_pmu_sbi.c index e255c1b069ec..c852f64a5022 100644 --- a/drivers/perf/riscv_pmu_sbi.c +++ b/drivers/perf/riscv_pmu_sbi.c @@ -17,6 +17,7 @@ #include #include #include +#include #include #include #include @@ -1038,10 +1039,10 @@ static void pmu_sbi_start_overflow_mask(struct risc= v_pmu *pmu, pmu_sbi_start_ovf_ctrs_sbi(cpu_hw_evt, ctr_ovf_mask); } =20 -static irqreturn_t pmu_sbi_ovf_handler(int irq, void *dev) +static irqreturn_t pmu_sbi_ovf_handler(struct cpu_hw_events *cpu_hw_evt, + struct pt_regs *regs, bool from_sse) { struct perf_sample_data data; - struct pt_regs *regs; struct hw_perf_event *hw_evt; union sbi_pmu_ctr_info *info; int lidx, hidx, fidx; @@ -1049,7 +1050,6 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void = *dev) struct perf_event *event; u64 overflow; u64 overflowed_ctrs =3D 0; - struct cpu_hw_events *cpu_hw_evt =3D dev; u64 start_clock =3D sched_clock(); struct riscv_pmu_snapshot_data *sdata =3D cpu_hw_evt->snapshot_addr; =20 @@ -1059,13 +1059,15 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, voi= d *dev) /* Firmware counter don't support overflow yet */ fidx =3D find_first_bit(cpu_hw_evt->used_hw_ctrs, RISCV_MAX_COUNTERS); if (fidx =3D=3D RISCV_MAX_COUNTERS) { - csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num)); + if (!from_sse) + csr_clear(CSR_SIP, BIT(riscv_pmu_irq_num)); return IRQ_NONE; } =20 event =3D cpu_hw_evt->events[fidx]; if (!event) { - ALT_SBI_PMU_OVF_CLEAR_PENDING(riscv_pmu_irq_mask); + if (!from_sse) + ALT_SBI_PMU_OVF_CLEAR_PENDING(riscv_pmu_irq_mask); return IRQ_NONE; } =20 @@ -1080,16 +1082,16 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, voi= d *dev) =20 /* * Overflow interrupt pending bit should only be cleared after stopping - * all the counters to avoid any race condition. + * all the counters to avoid any race condition. When using SSE, + * interrupt is cleared when stopping counters. */ - ALT_SBI_PMU_OVF_CLEAR_PENDING(riscv_pmu_irq_mask); + if (!from_sse) + ALT_SBI_PMU_OVF_CLEAR_PENDING(riscv_pmu_irq_mask); =20 /* No overflow bit is set */ if (!overflow) return IRQ_NONE; =20 - regs =3D get_irq_regs(); - for_each_set_bit(lidx, cpu_hw_evt->used_hw_ctrs, RISCV_MAX_COUNTERS) { struct perf_event *event =3D cpu_hw_evt->events[lidx]; =20 @@ -1145,6 +1147,51 @@ static irqreturn_t pmu_sbi_ovf_handler(int irq, void= *dev) return IRQ_HANDLED; } =20 +static irqreturn_t pmu_sbi_ovf_irq_handler(int irq, void *dev) +{ + return pmu_sbi_ovf_handler(dev, get_irq_regs(), false); +} + +#ifdef CONFIG_RISCV_PMU_SBI_SSE +static int pmu_sbi_ovf_sse_handler(u32 evt, void *arg, struct pt_regs *reg= s) +{ + struct cpu_hw_events __percpu *hw_events =3D arg; + struct cpu_hw_events *hw_event =3D raw_cpu_ptr(hw_events); + + pmu_sbi_ovf_handler(hw_event, regs, true); + + return 0; +} + +static int pmu_sbi_setup_sse(struct riscv_pmu *pmu) +{ + int ret; + struct sse_event *evt; + struct cpu_hw_events __percpu *hw_events =3D pmu->hw_events; + + evt =3D sse_event_register(SBI_SSE_EVENT_LOCAL_PMU_OVERFLOW, 0, + pmu_sbi_ovf_sse_handler, hw_events); + if (IS_ERR(evt)) + return PTR_ERR(evt); + + ret =3D sse_event_enable(evt); + if (ret) { + sse_event_unregister(evt); + return ret; + } + + pr_info("using SSE for PMU event delivery\n"); + pmu->sse_evt =3D evt; + + return ret; +} +#else +static int pmu_sbi_setup_sse(struct riscv_pmu *pmu) +{ + return -EOPNOTSUPP; +} +#endif + static int pmu_sbi_starting_cpu(unsigned int cpu, struct hlist_node *node) { struct riscv_pmu *pmu =3D hlist_entry_safe(node, struct riscv_pmu, node); @@ -1195,6 +1242,10 @@ static int pmu_sbi_setup_irqs(struct riscv_pmu *pmu,= struct platform_device *pde struct cpu_hw_events __percpu *hw_events =3D pmu->hw_events; struct irq_domain *domain =3D NULL; =20 + ret =3D pmu_sbi_setup_sse(pmu); + if (!ret) + return 0; + if (riscv_isa_extension_available(NULL, SSCOFPMF)) { riscv_pmu_irq_num =3D RV_IRQ_PMU; riscv_pmu_use_irq =3D true; @@ -1229,7 +1280,7 @@ static int pmu_sbi_setup_irqs(struct riscv_pmu *pmu, = struct platform_device *pde return -ENODEV; } =20 - ret =3D request_percpu_irq(riscv_pmu_irq, pmu_sbi_ovf_handler, "riscv-pmu= ", hw_events); + ret =3D request_percpu_irq(riscv_pmu_irq, pmu_sbi_ovf_irq_handler, "riscv= -pmu", hw_events); if (ret) { pr_err("registering percpu irq failed [%d]\n", ret); return ret; diff --git a/include/linux/perf/riscv_pmu.h b/include/linux/perf/riscv_pmu.h index f82a28040594..08fdcf6baf4e 100644 --- a/include/linux/perf/riscv_pmu.h +++ b/include/linux/perf/riscv_pmu.h @@ -28,6 +28,8 @@ =20 #define RISCV_PMU_CONFIG1_GUEST_EVENTS 0x1 =20 +struct sse_event; + struct cpu_hw_events { /* currently enabled events */ int n_events; @@ -54,6 +56,9 @@ struct riscv_pmu { char *name; =20 irqreturn_t (*handle_irq)(int irq_num, void *dev); +#ifdef CONFIG_RISCV_PMU_SBI_SSE + struct sse_event *sse_evt; +#endif =20 unsigned long cmask; u64 (*ctr_read)(struct perf_event *event); --=20 2.43.0 From nobody Tue Dec 16 13:37:30 2025 Received: from mail-pg1-f177.google.com (mail-pg1-f177.google.com [209.85.215.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9DB2028CF4A for ; Wed, 5 Nov 2025 08:29:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762331376; cv=none; b=CzQUNiJlUypvWn2U/kXjyb5E/G6Me5j4SRSFpc6oK75BOGL8vOoyWP7VxJaTr/qkTFbMGWxcx7LiXlyYXt3vnCp8jtNXpbqLBkFRA2qpXqqcPpfgDrP5jrV8yzkEmeDbVhuUdRnR8sfanWsOT7C8dPOO+fFg9ytOtbRlI5q1UDM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762331376; c=relaxed/simple; bh=5vLgHfJ5+oIZrkhpfvtFhd1B+CgmWC+9azY2vGK1S3I=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=tUWv7oSi9wq8PqEz6fhenSf3AJgbhAimSZ7gRN9e341jqZeuZCk6h2g3M67bwcQn8jWY6J7TViIA3ZquNfED+z7M6AedzbPDpwYLU+NSH5hOttuamlKBUhuwRb0gPvIdboF9WvWekp1tLHhLe9CpJJlAYvn2U6yAOlt0ehxj52E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc.com header.i=@rivosinc.com header.b=WxvuPXHA; arc=none smtp.client-ip=209.85.215.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc.com header.i=@rivosinc.com header.b="WxvuPXHA" Received: by mail-pg1-f177.google.com with SMTP id 41be03b00d2f7-b9a6ff216faso3059029a12.3 for ; Wed, 05 Nov 2025 00:29:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc.com; s=google; t=1762331374; x=1762936174; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+LvaX4EPEoHtGWdcOKi4KdBLQBe4M94Gewe05R3R7ko=; b=WxvuPXHA0ALlhbmzUXz5Dxi1BY36e7tzqGRO1J27Cu9kdRi20gGc8NQHiOspnYY9vt Cd4KNO0N8IIHOrEE5qFzZgHA4sFi49WJq8f2c37+gvcbtzldFODwHqaohVqoKMIjObYK U7p7ydBslusXRwM31GyBcdZsi1bBIUvO1jLwGR5SRlGOpZapx58d/+XPvQ2y2cv74VLe Rq7ek1Gm5zK9pWeRm8QpDg1GlTgkP0+VJq7x/cAMjjtJD+fzp+sOJezIo6mYoA8QRVIL N/lLo3/UM197h3G3G+PfzRl5lApUKfMAmhnumIu4S/3Yie9GoRQwiwg/O8hp4UYXKwZN iigw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1762331374; x=1762936174; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+LvaX4EPEoHtGWdcOKi4KdBLQBe4M94Gewe05R3R7ko=; b=t6VMSKOLh2JdU68TOWsKL/nmzCV/4Up0ZQ06Onsyc8qOt3dsmH9gMBEfeSbC2s6fWU qXRxouxZS1K2B0YKl8mBDzqG07HYLUVFobiie0qxsKec6/Fo2dybQDZuOUtqRvolE1T6 Pskep4Y6IxZpwlwUKX+FnWKcP5ntI5tZW9GYscb+LQqkG/DQj73D1bKSEqRLuJ8bt+F2 4M+AQgFX6bFqT9n1YXwMk1nL6ROO8v+X0h4Xv42ONJnkN1Vfi0+nkoPcOVrFAfMz+Tzu lOyt45TOK7jDlxQayz9K2JOiKNaulFD+IeO/5Zp7q9FYcohHXhxkDv4DKhmtJomdq71l gZpQ== X-Forwarded-Encrypted: i=1; AJvYcCXsuV/8f9iDhVCHbtxf0oxgjrad0y7f4/5c6B6XK2KPsDknZupM9t4RYwH1S7hfO7i/trxHKzvmPPB4VDE=@vger.kernel.org X-Gm-Message-State: AOJu0YwKkyGacFh9uV7ot7263RdcfMKuBjtfnpFVLoeB2OjmDWxuaIO8 hic+8yQ9AVr/TA5glHlFzq58P6YfP39ruTZvSaDM/nCnSveFmpOP2wQKX8QuCcsaZvg= X-Gm-Gg: ASbGncsKjSBXePs/Gr4YtFSIS88Y3zofMnWI2lUsrVoBMAOuV18IhIZyMEYEnJ89hN4 xS31ovduYp6wAWdsIdL97HfPBZJG5Ot9x4YdtiqYdvTmVVQGO4Sfol6iJKfCNxmKOzG1rQ0Lk4Q P4sM3M8Lgy1W/wKdDOhogJRSkqE+0ZO4snr1G4/z9ZZEWVwie3x0CUUAFxFe7yWmIbdg8GUvV/X Oexsn8EIxhUQcAfVryBE7XxtbtKFEpMbavVRP2q8qPkLlzYaWh7tS6brAIdfIcf3IWs2UTSuSiR 75Tqld6cB+unh3FYGDmbPOmTb8g6BAWXa9LAfj8kcaN+4RoAXz+KJQW47MRFm2AEM9XnpkFNET3 +QC0j2rOIdZFW10+yTyKB6lhgOVCACwNIj+QIslInx+BCv5kX/StG7Y+qc8PwZYxV9mw= X-Google-Smtp-Source: AGHT+IFOtwzVqigPG5Pv48WZCvUZZFKqHqAR0nRj7q7nG+pF4gXpcJEujy77eb39aoy7P6BCN42M3w== X-Received: by 2002:a17:902:fc8e:b0:295:8c51:64ff with SMTP id d9443c01a7336-2962ad95d73mr37428255ad.29.1762331369297; Wed, 05 Nov 2025 00:29:29 -0800 (PST) Received: from cleger.eu.int ([2001:41d0:420:f300::]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-29601a741e3sm51870875ad.98.2025.11.05.00.29.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 05 Nov 2025 00:29:28 -0800 (PST) From: =?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?= To: Paul Walmsley , Palmer Dabbelt , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: =?UTF-8?q?Cl=C3=A9ment=20L=C3=A9ger?= , Himanshu Chauhan , Anup Patel , Xu Lu , Atish Patra , =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Yunhui Cui Subject: [PATCH v8 5/5] selftests/riscv: add SSE test module Date: Wed, 5 Nov 2025 08:26:37 +0000 Message-ID: <20251105082639.342973-6-cleger@rivosinc.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20251105082639.342973-1-cleger@rivosinc.com> References: <20251105082639.342973-1-cleger@rivosinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable This module, once loaded, will execute a series of tests using the SSE framework. The provided script will check for any error reported by the test module. Signed-off-by: Cl=C3=A9ment L=C3=A9ger --- tools/testing/selftests/riscv/Makefile | 2 +- tools/testing/selftests/riscv/sse/Makefile | 5 + .../selftests/riscv/sse/module/Makefile | 16 + .../riscv/sse/module/riscv_sse_test.c | 511 ++++++++++++++++++ .../selftests/riscv/sse/run_sse_test.sh | 44 ++ 5 files changed, 577 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/riscv/sse/Makefile create mode 100644 tools/testing/selftests/riscv/sse/module/Makefile create mode 100644 tools/testing/selftests/riscv/sse/module/riscv_sse_test= .c create mode 100644 tools/testing/selftests/riscv/sse/run_sse_test.sh diff --git a/tools/testing/selftests/riscv/Makefile b/tools/testing/selftes= ts/riscv/Makefile index 099b8c1f46f8..c62f58414b29 100644 --- a/tools/testing/selftests/riscv/Makefile +++ b/tools/testing/selftests/riscv/Makefile @@ -5,7 +5,7 @@ ARCH ?=3D $(shell uname -m 2>/dev/null || echo not) =20 ifneq (,$(filter $(ARCH),riscv)) -RISCV_SUBTARGETS ?=3D abi hwprobe mm sigreturn vector +RISCV_SUBTARGETS ?=3D abi hwprobe mm sigreturn vector sse else RISCV_SUBTARGETS :=3D endif diff --git a/tools/testing/selftests/riscv/sse/Makefile b/tools/testing/sel= ftests/riscv/sse/Makefile new file mode 100644 index 000000000000..67eaee06f213 --- /dev/null +++ b/tools/testing/selftests/riscv/sse/Makefile @@ -0,0 +1,5 @@ +TEST_GEN_MODS_DIR :=3D module + +TEST_FILES :=3D run_sse_test.sh + +include ../../lib.mk diff --git a/tools/testing/selftests/riscv/sse/module/Makefile b/tools/test= ing/selftests/riscv/sse/module/Makefile new file mode 100644 index 000000000000..02018f083456 --- /dev/null +++ b/tools/testing/selftests/riscv/sse/module/Makefile @@ -0,0 +1,16 @@ +TESTMODS_DIR :=3D $(realpath $(dir $(abspath $(lastword $(MAKEFILE_LIST)))= )) +KDIR ?=3D /lib/modules/$(shell uname -r)/build + +obj-m +=3D riscv_sse_test.o + +# Ensure that KDIR exists, otherwise skip the compilation +modules: +ifneq ("$(wildcard $(KDIR))", "") + $(Q)$(MAKE) -C $(KDIR) modules KBUILD_EXTMOD=3D$(TESTMODS_DIR) +endif + +# Ensure that KDIR exists, otherwise skip the clean target +clean: +ifneq ("$(wildcard $(KDIR))", "") + $(Q)$(MAKE) -C $(KDIR) clean KBUILD_EXTMOD=3D$(TESTMODS_DIR) +endif diff --git a/tools/testing/selftests/riscv/sse/module/riscv_sse_test.c b/to= ols/testing/selftests/riscv/sse/module/riscv_sse_test.c new file mode 100644 index 000000000000..fcff1e346fee --- /dev/null +++ b/tools/testing/selftests/riscv/sse/module/riscv_sse_test.c @@ -0,0 +1,511 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * Copyright (C) 2025 Rivos Inc. + */ + +#define pr_fmt(fmt) "riscv_sse_test: " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include +#include + +#define RUN_LOOP_COUNT 1000 +#define SSE_FAILED_PREFIX "FAILED: " +#define sse_err(...) pr_err(SSE_FAILED_PREFIX __VA_ARGS__) + +struct sse_event_desc { + u32 evt_id; + const char *name; + bool can_inject; +}; + +static struct sse_event_desc sse_event_descs[] =3D { + { + .evt_id =3D SBI_SSE_EVENT_LOCAL_HIGH_PRIO_RAS, + .name =3D "local_high_prio_ras", + }, + { + .evt_id =3D SBI_SSE_EVENT_LOCAL_DOUBLE_TRAP, + .name =3D "local_double_trap", + }, + { + .evt_id =3D SBI_SSE_EVENT_GLOBAL_HIGH_PRIO_RAS, + .name =3D "global_high_prio_ras", + }, + { + .evt_id =3D SBI_SSE_EVENT_LOCAL_PMU_OVERFLOW, + .name =3D "local_pmu_overflow", + }, + { + .evt_id =3D SBI_SSE_EVENT_LOCAL_LOW_PRIO_RAS, + .name =3D "local_low_prio_ras", + }, + { + .evt_id =3D SBI_SSE_EVENT_GLOBAL_LOW_PRIO_RAS, + .name =3D "global_low_prio_ras", + }, + { + .evt_id =3D SBI_SSE_EVENT_LOCAL_SOFTWARE_INJECTED, + .name =3D "local_software_injected", + }, + { + .evt_id =3D SBI_SSE_EVENT_GLOBAL_SOFTWARE_INJECTED, + .name =3D "global_software_injected", + } +}; + +static struct sse_event_desc *sse_get_evt_desc(u32 evt) +{ + int i; + + for (i =3D 0; i < ARRAY_SIZE(sse_event_descs); i++) { + if (sse_event_descs[i].evt_id =3D=3D evt) + return &sse_event_descs[i]; + } + + return NULL; +} + +static const char *sse_evt_name(u32 evt) +{ + struct sse_event_desc *desc =3D sse_get_evt_desc(evt); + + return desc ? desc->name : NULL; +} + +static bool sse_test_can_inject_event(u32 evt) +{ + struct sse_event_desc *desc =3D sse_get_evt_desc(evt); + + return desc ? desc->can_inject : false; +} + +static struct sbiret sbi_sse_ecall(int fid, unsigned long arg0, unsigned l= ong arg1) +{ + return sbi_ecall(SBI_EXT_SSE, fid, arg0, arg1, 0, 0, 0, 0); +} + +static int sse_event_attr_get(u32 evt, unsigned long attr_id, + unsigned long *val) +{ + struct sbiret sret; + unsigned long *attr_buf, phys; + + attr_buf =3D kmalloc(sizeof(unsigned long), GFP_KERNEL); + if (!attr_buf) + return -ENOMEM; + + phys =3D virt_to_phys(attr_buf); + + sret =3D sbi_ecall(SBI_EXT_SSE, SBI_SSE_EVENT_ATTR_READ, evt, attr_id, 1, + phys, 0, 0); + if (sret.error) + return sbi_err_map_linux_errno(sret.error); + + *val =3D *attr_buf; + + return 0; +} + +static int sse_test_signal(u32 evt, unsigned int cpu) +{ + unsigned int hart_id =3D cpuid_to_hartid_map(cpu); + struct sbiret ret; + + ret =3D sbi_sse_ecall(SBI_SSE_EVENT_INJECT, evt, hart_id); + if (ret.error) { + sse_err("Failed to signal event %x, error %ld\n", evt, ret.error); + return sbi_err_map_linux_errno(ret.error); + } + + return 0; +} + +static int sse_test_inject_event(struct sse_event *event, u32 evt, unsigne= d int cpu) +{ + int res; + unsigned long status; + + if (sse_event_is_global(evt)) { + /* + * Due to the fact the completion might happen faster than + * the call to SBI_SSE_COMPLETE in the handler, if the event was + * running on another CPU, we need to wait for the event status + * to be !RUNNING. + */ + do { + res =3D sse_event_attr_get(evt, SBI_SSE_ATTR_STATUS, &status); + if (res) { + sse_err("Failed to get status for evt %x, error %d\n", evt, res); + return res; + } + status =3D status & SBI_SSE_ATTR_STATUS_STATE_MASK; + } while (status =3D=3D SBI_SSE_STATE_RUNNING); + + res =3D sse_event_set_target_cpu(event, cpu); + if (res) { + sse_err("Failed to set cpu for evt %x, error %d\n", evt, res); + return res; + } + } + + return sse_test_signal(evt, cpu); +} + +struct fast_test_arg { + u32 evt; + int cpu; + bool completion; +}; + +static int sse_test_handler(u32 evt, void *arg, struct pt_regs *regs) +{ + int ret =3D 0; + struct fast_test_arg *targ =3D arg; + u32 test_evt =3D READ_ONCE(targ->evt); + int cpu =3D READ_ONCE(targ->cpu); + + if (evt !=3D test_evt) { + sse_err("Received SSE event id %x instead of %x\n", test_evt, evt); + ret =3D -EINVAL; + } + + if (cpu !=3D smp_processor_id()) { + sse_err("Received SSE event %d on CPU %d instead of %d\n", evt, smp_proc= essor_id(), + cpu); + ret =3D -EINVAL; + } + + WRITE_ONCE(targ->completion, true); + + return ret; +} + +static void sse_run_fast_test(struct fast_test_arg *test_arg, struct sse_e= vent *event, u32 evt) +{ + unsigned long timeout; + int ret, cpu; + + for_each_online_cpu(cpu) { + WRITE_ONCE(test_arg->completion, false); + WRITE_ONCE(test_arg->cpu, cpu); + /* Test arg is used on another CPU */ + smp_wmb(); + + ret =3D sse_test_inject_event(event, evt, cpu); + if (ret) { + sse_err("event %s injection failed, err %d\n", sse_evt_name(evt), ret); + return; + } + + timeout =3D jiffies + HZ / 100; + /* We can not use since they are not NMI safe */ + while (!READ_ONCE(test_arg->completion) && + time_before(jiffies, timeout)) { + cpu_relax(); + } + if (!time_before(jiffies, timeout)) { + sse_err("Failed to wait for event %s completion on CPU %d\n", + sse_evt_name(evt), cpu); + return; + } + } +} + +static void sse_test_injection_fast(void) +{ + int i, ret =3D 0, j; + u32 evt; + struct fast_test_arg test_arg; + struct sse_event *event; + + pr_info("Starting SSE test (fast)\n"); + + for (i =3D 0; i < ARRAY_SIZE(sse_event_descs); i++) { + evt =3D sse_event_descs[i].evt_id; + WRITE_ONCE(test_arg.evt, evt); + + if (!sse_event_descs[i].can_inject) + continue; + + event =3D sse_event_register(evt, 0, sse_test_handler, + (void *)&test_arg); + if (IS_ERR(event)) { + sse_err("Failed to register event %s, err %ld\n", sse_evt_name(evt), + PTR_ERR(event)); + goto out; + } + + ret =3D sse_event_enable(event); + if (ret) { + sse_err("Failed to enable event %s, err %d\n", sse_evt_name(evt), ret); + goto err_unregister; + } + + pr_info("Starting testing event %s\n", sse_evt_name(evt)); + + for (j =3D 0; j < RUN_LOOP_COUNT; j++) + sse_run_fast_test(&test_arg, event, evt); + + pr_info("Finished testing event %s\n", sse_evt_name(evt)); + + sse_event_disable(event); +err_unregister: + sse_event_unregister(event); + } +out: + pr_info("Finished SSE test (fast)\n"); +} + +struct priority_test_arg { + unsigned long evt; + struct sse_event *event; + bool called; + u32 prio; + struct priority_test_arg *next_evt_arg; + void (*check_func)(struct priority_test_arg *arg); +}; + +static int sse_hi_priority_test_handler(u32 evt, void *arg, + struct pt_regs *regs) +{ + struct priority_test_arg *targ =3D arg; + struct priority_test_arg *next =3D READ_ONCE(targ->next_evt_arg); + + WRITE_ONCE(targ->called, 1); + + if (next) { + sse_test_signal(next->evt, smp_processor_id()); + if (!READ_ONCE(next->called)) { + sse_err("Higher priority event %s was not handled %s\n", + sse_evt_name(next->evt), sse_evt_name(evt)); + } + } + + return 0; +} + +static int sse_low_priority_test_handler(u32 evt, void *arg, struct pt_reg= s *regs) +{ + struct priority_test_arg *targ =3D arg; + struct priority_test_arg *next =3D READ_ONCE(targ->next_evt_arg); + + WRITE_ONCE(targ->called, 1); + + if (next) { + sse_test_signal(next->evt, smp_processor_id()); + if (READ_ONCE(next->called)) { + sse_err("Lower priority event %s was handle before %s\n", + sse_evt_name(next->evt), sse_evt_name(evt)); + } + } + + return 0; +} + +static void sse_test_injection_priority_arg(struct priority_test_arg *args= , unsigned int args_size, + sse_event_handler_fn handler, const char *test_name) +{ + unsigned int i; + int ret; + struct sse_event *event; + struct priority_test_arg *arg, *first_arg =3D NULL, *prev_arg =3D NULL; + + pr_info("Starting SSE priority test (%s)\n", test_name); + for (i =3D 0; i < args_size; i++) { + arg =3D &args[i]; + + if (!sse_test_can_inject_event(arg->evt)) + continue; + + WRITE_ONCE(arg->called, false); + WRITE_ONCE(arg->next_evt_arg, NULL); + if (prev_arg) + WRITE_ONCE(prev_arg->next_evt_arg, arg); + + prev_arg =3D arg; + + if (!first_arg) + first_arg =3D arg; + + event =3D sse_event_register(arg->evt, arg->prio, handler, (void *)arg); + if (IS_ERR(event)) { + sse_err("Failed to register event %s, err %ld\n", sse_evt_name(arg->evt= ), + PTR_ERR(event)); + goto release_events; + } + arg->event =3D event; + + if (sse_event_is_global(arg->evt)) { + /* Target event at current CPU */ + ret =3D sse_event_set_target_cpu(event, smp_processor_id()); + if (ret) { + sse_err("Failed to set event %s target CPU, err %d\n", + sse_evt_name(arg->evt), ret); + goto release_events; + } + } + + ret =3D sse_event_enable(event); + if (ret) { + sse_err("Failed to enable event %s, err %d\n", sse_evt_name(arg->evt), = ret); + goto release_events; + } + } + + if (!first_arg) { + sse_err("No injectable event available\n"); + return; + } + + /* Inject first event, handler should trigger the others in chain. */ + ret =3D sse_test_inject_event(first_arg->event, first_arg->evt, smp_proce= ssor_id()); + if (ret) { + sse_err("SSE event %s injection failed\n", sse_evt_name(first_arg->evt)); + goto release_events; + } + + /* + * Event are injected directly on the current CPU after calling sse_test_= inject_event() + * so that execution is premmpted right away, no need to wait for timeout. + */ + arg =3D first_arg; + while (arg) { + if (!READ_ONCE(arg->called)) { + sse_err("Event %s handler was not called\n", + sse_evt_name(arg->evt)); + ret =3D -EINVAL; + } + + event =3D arg->event; + arg =3D READ_ONCE(arg->next_evt_arg); + } + +release_events: + + arg =3D first_arg; + while (arg) { + event =3D arg->event; + if (!event) + break; + + sse_event_disable(event); + sse_event_unregister(event); + arg =3D READ_ONCE(arg->next_evt_arg); + } + + pr_info("Finished SSE priority test (%s)\n", test_name); +} + +static void sse_test_injection_priority(void) +{ + struct priority_test_arg default_hi_prio_args[] =3D { + { .evt =3D SBI_SSE_EVENT_GLOBAL_SOFTWARE_INJECTED }, + { .evt =3D SBI_SSE_EVENT_LOCAL_SOFTWARE_INJECTED }, + { .evt =3D SBI_SSE_EVENT_GLOBAL_LOW_PRIO_RAS }, + { .evt =3D SBI_SSE_EVENT_LOCAL_LOW_PRIO_RAS }, + { .evt =3D SBI_SSE_EVENT_LOCAL_PMU_OVERFLOW }, + { .evt =3D SBI_SSE_EVENT_GLOBAL_HIGH_PRIO_RAS }, + { .evt =3D SBI_SSE_EVENT_LOCAL_DOUBLE_TRAP }, + { .evt =3D SBI_SSE_EVENT_LOCAL_HIGH_PRIO_RAS }, + }; + + struct priority_test_arg default_low_prio_args[] =3D { + { .evt =3D SBI_SSE_EVENT_LOCAL_HIGH_PRIO_RAS }, + { .evt =3D SBI_SSE_EVENT_LOCAL_DOUBLE_TRAP }, + { .evt =3D SBI_SSE_EVENT_GLOBAL_HIGH_PRIO_RAS }, + { .evt =3D SBI_SSE_EVENT_LOCAL_PMU_OVERFLOW }, + { .evt =3D SBI_SSE_EVENT_LOCAL_LOW_PRIO_RAS }, + { .evt =3D SBI_SSE_EVENT_GLOBAL_LOW_PRIO_RAS }, + { .evt =3D SBI_SSE_EVENT_LOCAL_SOFTWARE_INJECTED }, + { .evt =3D SBI_SSE_EVENT_GLOBAL_SOFTWARE_INJECTED }, + + }; + struct priority_test_arg set_prio_args[] =3D { + { .evt =3D SBI_SSE_EVENT_GLOBAL_SOFTWARE_INJECTED, .prio =3D 5 }, + { .evt =3D SBI_SSE_EVENT_LOCAL_SOFTWARE_INJECTED, .prio =3D 10 }, + { .evt =3D SBI_SSE_EVENT_GLOBAL_LOW_PRIO_RAS, .prio =3D 15 }, + { .evt =3D SBI_SSE_EVENT_LOCAL_LOW_PRIO_RAS, .prio =3D 20 }, + { .evt =3D SBI_SSE_EVENT_LOCAL_PMU_OVERFLOW, .prio =3D 25 }, + { .evt =3D SBI_SSE_EVENT_GLOBAL_HIGH_PRIO_RAS, .prio =3D 30 }, + { .evt =3D SBI_SSE_EVENT_LOCAL_DOUBLE_TRAP, .prio =3D 35 }, + { .evt =3D SBI_SSE_EVENT_LOCAL_HIGH_PRIO_RAS, .prio =3D 40 }, + }; + + struct priority_test_arg same_prio_args[] =3D { + { .evt =3D SBI_SSE_EVENT_LOCAL_PMU_OVERFLOW, .prio =3D 0 }, + { .evt =3D SBI_SSE_EVENT_LOCAL_HIGH_PRIO_RAS, .prio =3D 10 }, + { .evt =3D SBI_SSE_EVENT_LOCAL_SOFTWARE_INJECTED, .prio =3D 10 }, + { .evt =3D SBI_SSE_EVENT_GLOBAL_SOFTWARE_INJECTED, .prio =3D 10 }, + { .evt =3D SBI_SSE_EVENT_GLOBAL_HIGH_PRIO_RAS, .prio =3D 20 }, + }; + + sse_test_injection_priority_arg(default_hi_prio_args, ARRAY_SIZE(default_= hi_prio_args), + sse_hi_priority_test_handler, "high"); + + sse_test_injection_priority_arg(default_low_prio_args, ARRAY_SIZE(default= _low_prio_args), + sse_low_priority_test_handler, "low"); + + sse_test_injection_priority_arg(set_prio_args, ARRAY_SIZE(set_prio_args), + sse_low_priority_test_handler, "set"); + + sse_test_injection_priority_arg(same_prio_args, ARRAY_SIZE(same_prio_args= ), + sse_low_priority_test_handler, "same_prio_args"); +} + +static bool sse_get_inject_status(u32 evt) +{ + int ret; + unsigned long val; + + /* Check if injection is supported */ + ret =3D sse_event_attr_get(evt, SBI_SSE_ATTR_STATUS, &val); + if (ret) + return false; + + return !!(val & BIT(SBI_SSE_ATTR_STATUS_INJECT_OFFSET)); +} + +static void sse_init_events(void) +{ + int i; + + for (i =3D 0; i < ARRAY_SIZE(sse_event_descs); i++) { + struct sse_event_desc *desc =3D &sse_event_descs[i]; + + desc->can_inject =3D sse_get_inject_status(desc->evt_id); + if (!desc->can_inject) + pr_info("Can not inject event %s, tests using this event will be skippe= d\n", + desc->name); + } +} + +static int __init sse_test_init(void) +{ + sse_init_events(); + + sse_test_injection_fast(); + sse_test_injection_priority(); + + return 0; +} + +static void __exit sse_test_exit(void) +{ +} + +module_init(sse_test_init); +module_exit(sse_test_exit); + +MODULE_LICENSE("GPL"); +MODULE_AUTHOR("Cl=C3=A9ment L=C3=A9ger "); +MODULE_DESCRIPTION("Test module for SSE"); diff --git a/tools/testing/selftests/riscv/sse/run_sse_test.sh b/tools/test= ing/selftests/riscv/sse/run_sse_test.sh new file mode 100644 index 000000000000..888bc4a99cb3 --- /dev/null +++ b/tools/testing/selftests/riscv/sse/run_sse_test.sh @@ -0,0 +1,44 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# +# Copyright (C) 2025 Rivos Inc. + +MODULE_NAME=3Driscv_sse_test +DRIVER=3D"./module/${MODULE_NAME}.ko" + +check_test_failed_prefix() { + if dmesg | grep -q "${MODULE_NAME}: FAILED:";then + echo "${MODULE_NAME} failed, please check dmesg" + exit 1 + fi +} + +# Kselftest framework requirement - SKIP code is 4. +ksft_skip=3D4 + +check_test_requirements() +{ + uid=3D$(id -u) + if [ $uid -ne 0 ]; then + echo "$0: Must be run as root" + exit $ksft_skip + fi + + if ! which insmod > /dev/null 2>&1; then + echo "$0: You need insmod installed" + exit $ksft_skip + fi + + if [ ! -f $DRIVER ]; then + echo "$0: You need to compile ${MODULE_NAME} module" + exit $ksft_skip + fi +} + +check_test_requirements + +insmod $DRIVER > /dev/null 2>&1 +rmmod $MODULE_NAME +check_test_failed_prefix + +exit 0 --=20 2.43.0