From nobody Tue Dec 16 22:17:05 2025 Received: from mail-wm1-f44.google.com (mail-wm1-f44.google.com [209.85.128.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AF5141C0DEE for ; Wed, 29 May 2024 18:54:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.44 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717008874; cv=none; b=f/pJekA1lvIkRyW7xELoH+zcQSiBstEKd8DfUH0JN7deGI/wmWO0+ZTe5qiXhEsBPq/MbyBi56x0SjofywzrT8grHu0wqSbo8tRqlzy67rdqrvOeBJwLBtUdUt3Hk2EtFFK5PPc6EVmHrDP7ChLx7ZSDqSvC/htfOsa9xC2J/Nk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717008874; c=relaxed/simple; bh=vJl9tQ1Pqf9QGonXEZ2iQ0jeNMSCWZFnU88m6CXe9t8=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=LV2pu39+Q2q37iKfgpWkUxMLs+WuYLg4Em5IQ9k8Uuyl0T+JPmllQLdqsWOOWXv+S8q2cbVVGa4IPrV5cDLng7mZTdnNr4dfXikR8DmJxUbjn6zSC964o82jcNqowgJdWFyjRHypWSGjQYUw2bj+f54XKIDi96m9JJX5py32T6g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=B7t1zk4w; arc=none smtp.client-ip=209.85.128.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="B7t1zk4w" Received: by mail-wm1-f44.google.com with SMTP id 5b1f17b1804b1-4201986d60aso296455e9.3 for ; Wed, 29 May 2024 11:54:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1717008871; x=1717613671; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=688BMYms93lIkdSUxmL3fbJu+G2/bRYi5WNxrC0uGt8=; b=B7t1zk4w+lS+6fejmWMRCnqIqLH+yDS/VC3D3KnHm37scq6XHxH5qT/r4oudrL2b62 y0txXlkoOmkbhKNguBGQboAjQqx/q5aDYEMYOAXei6xEAwCo0X3HYCZ3egLQIj6QrOxJ AjFC0HLielbh/z6mHRrdIwikD6mNrdDH9tUXvJhGunfUlS2L0AvVr6KVjoClkAs+C1cc K6DitnoI19YYp/NXX79oGrViR2KSexj/eRNVbsw53alWIqOTqv9CKPQpZun3I3mgtODV t53WiPPHtQE5oasYrHspszZgqCJB0S8bhRdYiGZFNrVt2abPULeo1i/6bc12gFFaFSEb L8WQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717008871; x=1717613671; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=688BMYms93lIkdSUxmL3fbJu+G2/bRYi5WNxrC0uGt8=; b=EiLKRFlIcIxWcmq3R2uXt8HOurkMT4/g3aAhQjpm6tCFKTZ58NjV/NRrdZ7Isxj4yb 2TDqdeEMR1FqtVi/mIAG9tDSIy763spvydujhBW/QCgqwphO+W/MYq9w/JVp8+JeYg4x ebASNq9wRJrXL+BIk90hrkUtWo8s9vMYM9a0LOcZOodqzQJgzGvKReWDN0R3cOIUiGvX ooHiCt/EpMKiyfYXN/vX58xmDSdMH+si0zd9vYBUKYO8jr8NYDtEOmllDRhfZqES+qvT rNPO1EfzgFR7HvEHZAvQnxpZo+qYa6h5JpZjVqfKz028X5IEA8K1JQltUNTP4p+u9r9O w7Qg== X-Gm-Message-State: AOJu0YxD/adLS1NqD1tFw3UzS4jLjzPWAPMKLAenp2aQLl58Kt5u+ErA 4qE3RbQE98hhzFCzq0ku+tTQKXsEa/4iCjt3GUZIkyvDAGIwW7mmYtzNJOB9el0tzSawuaSYdB/ p X-Google-Smtp-Source: AGHT+IFXjh/sO2hf9fpunrpzb5/Ur0BabTosyM0Or0Urw/4lUp/bZLnXNorWqgLPv7xYyl4q0uveYw== X-Received: by 2002:a05:600c:4ed0:b0:421:22c4:db60 with SMTP id 5b1f17b1804b1-4212781b273mr404635e9.23.1717008870867; Wed, 29 May 2024 11:54:30 -0700 (PDT) Received: from rkanwal-XPS-15-9520.Home ([2a02:c7c:7527:ee00:7446:71c1:a41a:da9b]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4212706a23csm2787885e9.27.2024.05.29.11.54.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 May 2024 11:54:30 -0700 (PDT) From: Rajnesh Kanwal To: linux-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org, linux-riscv@lists.infradead.org, adrian.hunter@intel.com, alexander.shishkin@linux.intel.com, ajones@ventanamicro.com, anup@brainfault.org, acme@kernel.org, atishp@rivosinc.com, beeman@rivosinc.com, brauner@kernel.org, conor@kernel.org, heiko@sntech.de, irogers@google.com, mingo@redhat.com, james.clark@arm.com, renyu.zj@linux.alibaba.com, jolsa@kernel.org, jisheng.teoh@starfivetech.com, palmer@dabbelt.com, tech-control-transfer-records@lists.riscv.org, will@kernel.org, kaiwenxue1@gmail.com, Rajnesh Kanwal Subject: [PATCH RFC 1/6] perf: Increase the maximum number of samples to 256. Date: Wed, 29 May 2024 19:53:32 +0100 Message-Id: <20240529185337.182722-2-rkanwal@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240529185337.182722-1-rkanwal@rivosinc.com> References: <20240529185337.182722-1-rkanwal@rivosinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" RISCV CTR extension support a maximum depth of 256 last branch records. The 127 entries limit results in corrupting CTR entries for RISC-V if configured to be 256 entries. This will not impact any other architectures as it is just increasing maximum limit of possible entries. Signed-off-by: Rajnesh Kanwal --- tools/perf/util/machine.c | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/tools/perf/util/machine.c b/tools/perf/util/machine.c index 527517db3182..ec12f0199d46 100644 --- a/tools/perf/util/machine.c +++ b/tools/perf/util/machine.c @@ -2254,25 +2254,32 @@ static void save_iterations(struct iterations *iter, iter->cycles +=3D be[i].flags.cycles; } =20 -#define CHASHSZ 127 -#define CHASHBITS 7 -#define NO_ENTRY 0xff +#define CHASHBITS 8 +#define NO_ENTRY 0xffU =20 -#define PERF_MAX_BRANCH_DEPTH 127 +#define PERF_MAX_BRANCH_DEPTH 256 =20 /* Remove loops. */ +/* Note: Last entry (i=3D=3Dff) will never be checked against NO_ENTRY + * so it's safe to have an unsigned char array to process 256 entries + * without causing clash between last entry and NO_ENTRY value. + */ static int remove_loops(struct branch_entry *l, int nr, struct iterations *iter) { int i, j, off; - unsigned char chash[CHASHSZ]; + unsigned char chash[PERF_MAX_BRANCH_DEPTH]; =20 memset(chash, NO_ENTRY, sizeof(chash)); =20 - BUG_ON(PERF_MAX_BRANCH_DEPTH > 255); + BUG_ON(PERF_MAX_BRANCH_DEPTH > 256); =20 for (i =3D 0; i < nr; i++) { - int h =3D hash_64(l[i].from, CHASHBITS) % CHASHSZ; + /* Remainder division by PERF_MAX_BRANCH_DEPTH is not + * needed as hash_64 will anyway limit the hash + * to CHASHBITS + */ + int h =3D hash_64(l[i].from, CHASHBITS); =20 /* no collision handling for now */ if (chash[h] =3D=3D NO_ENTRY) { --=20 2.34.1 From nobody Tue Dec 16 22:17:05 2025 Received: from mail-wr1-f51.google.com (mail-wr1-f51.google.com [209.85.221.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C6A911C2312 for ; Wed, 29 May 2024 18:54:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717008875; cv=none; b=hVMJS8F6L8sL17nGtP8wdtZWN5sKk3bT7YZM6UsAu86oM4+S9n/DKIygazSnuA8ainSLKeRVwPK90iCJyXGcuA6TUyIvtTF5y0VJ8S8fAvuwwZep5DUIU/a8LEgVkLE8NLs0dVrTZCgO+BgKkgVQxefdSgfh3S8WRYyHrLXNYtU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717008875; c=relaxed/simple; bh=cxmtmKz3E4f94afNiy72jSn+uklIphZ48Rhdqmo7TFI=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=cuxbpm/H9wt+fCapvSOI24Wzs5YQYGAmS5NTaM42FVNIRb8LwhbabuagVikxhMy+94auM7O2pg/wivSaHwNt9WPdVSlt0bq6B4bFFumtZsmPsudpQLbzkeBJstNUMGYdxMqiqet2tS+INLuHP5KYvDmzICusIBVPE1yXkz5Gvhg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=PkslBels; arc=none smtp.client-ip=209.85.221.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="PkslBels" Received: by mail-wr1-f51.google.com with SMTP id ffacd0b85a97d-354de3c5d00so20231f8f.1 for ; Wed, 29 May 2024 11:54:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1717008872; x=1717613672; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9bkrwFm3shuYsCyMH7CeeRCQ3k6hoLktrnsincGFelA=; b=PkslBelsXIwtqdornnU135YD2O0Nb8KaXLpKAcQ+R/ytmxrPk2UDu2lK5MxejiDxfu vyMYnBrxZisYWjHAYR3l36PmA3PDRlrJODZ56bviN2CIOboNjCtopDjJgvItK83YTOUc 5S+SjotwhkwexD1w1foWM6QPw90CyFyBLh/U48CjCVn9vZk5WT0bMvtL1GKQzfSM57gV wHhH5JadINFX0Iy62QplGfEVwTSSyp8cNEGB0dzdtCxXOi60sFD8fHwXcv6k39iioqL4 OCof61XVbs1iQ2xWXbN+6qSdbxpogBnZisDwmoS3ZPQIoWUzEOkH9qN9Oym1Z9+CSJd6 GJmA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717008872; x=1717613672; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9bkrwFm3shuYsCyMH7CeeRCQ3k6hoLktrnsincGFelA=; b=LZoFI1xtlMWAu6O6elchLGv2Xt7mbsOnHUe5TIeexK1GbxATCqhIEK7tukvC0GV35r hN1Ymd60534fboWB5gw+rwq9mR4P/7FwRUjkayCROC5q95UMyFfMFkQTKOJ04F9LLA2K v6jMaFn8p5NbEFu1WknusgNVAFR7GEEObZNc9KOf+Cp4xm4wpLEDq4XtHU74bv3eKKiE 46Udc6q+/BQkO9o8vuM4XbrTqNmsm4sGCc5g5H7mYv22n8obHsnukSLgFI+cEYfw+BsK G7z2JDDaJgVhphZL0OfxFBUs9t0vlpB9KgCvs1LD1QZzIBHOFBg9urvzlAFblnscNfsn /nGg== X-Gm-Message-State: AOJu0YxAC6AnfFpO48G+rvYKhDx5BIp0S1yvXw6rdbzK4QDZ1bc/c/IK HhH1hDh1XBVfNX21AGP4f+ST55LazyJYWCMAk8RWDBIhu+P9fG12S3wQRwdrEt95P1iTxN5QWZg t X-Google-Smtp-Source: AGHT+IFoQdIT2bGEbgTFZXCZgpArQHuG+mz5mTYos9yf7MRsly5SzLPt+k6vmtWPXeZ8W9+MCZpm8w== X-Received: by 2002:adf:e3c1:0:b0:34d:9b4d:2a43 with SMTP id ffacd0b85a97d-35c7672a78bmr2946621f8f.0.1717008871994; Wed, 29 May 2024 11:54:31 -0700 (PDT) Received: from rkanwal-XPS-15-9520.Home ([2a02:c7c:7527:ee00:7446:71c1:a41a:da9b]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4212706a23csm2787885e9.27.2024.05.29.11.54.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 May 2024 11:54:31 -0700 (PDT) From: Rajnesh Kanwal To: linux-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org, linux-riscv@lists.infradead.org, adrian.hunter@intel.com, alexander.shishkin@linux.intel.com, ajones@ventanamicro.com, anup@brainfault.org, acme@kernel.org, atishp@rivosinc.com, beeman@rivosinc.com, brauner@kernel.org, conor@kernel.org, heiko@sntech.de, irogers@google.com, mingo@redhat.com, james.clark@arm.com, renyu.zj@linux.alibaba.com, jolsa@kernel.org, jisheng.teoh@starfivetech.com, palmer@dabbelt.com, tech-control-transfer-records@lists.riscv.org, will@kernel.org, kaiwenxue1@gmail.com, Rajnesh Kanwal Subject: [PATCH RFC 2/6] riscv: perf: Add Control transfer records CSR definations. Date: Wed, 29 May 2024 19:53:33 +0100 Message-Id: <20240529185337.182722-3-rkanwal@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240529185337.182722-1-rkanwal@rivosinc.com> References: <20240529185337.182722-1-rkanwal@rivosinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Adding CSR defines for RISCV Control Transfer Records extension [0] along with bit-field macros for each CSR. [0]: https://github.com/riscv/riscv-control-transfer-records Signed-off-by: Rajnesh Kanwal --- arch/riscv/include/asm/csr.h | 83 ++++++++++++++++++++++++++++++++++++ 1 file changed, 83 insertions(+) diff --git a/arch/riscv/include/asm/csr.h b/arch/riscv/include/asm/csr.h index 701963b64fc4..a80a2ee9d44e 100644 --- a/arch/riscv/include/asm/csr.h +++ b/arch/riscv/include/asm/csr.h @@ -309,6 +309,85 @@ =20 #define CSR_SSCOUNTOVF 0xda0 =20 +/* M-mode Control Transfer Records CSRs */ +#define CSR_MCTRCTL 0x34e + +/* S-mode Control Transfer Records CSRs */ +#define CSR_SCTRCTL 0x14e +#define CSR_SCTRSTATUS 0x14f +#define CSR_SCTRDEPTH 0x15f + +/* VS-mode Control Transfer Records CSRs */ +#define CSR_VSCTRCTL 0x24e + +/* xctrtl CSR bits. */ +#define CTRCTL_U_ENABLE _AC(0x1, UL) +#define CTRCTL_S_ENABLE _AC(0x2, UL) +#define CTRCTL_M_ENABLE _AC(0x4, UL) +#define CTRCTL_RASEMU _AC(0x80, UL) +#define CTRCTL_STE _AC(0x100, UL) +#define CTRCTL_MTE _AC(0x200, UL) +#define CTRCTL_BPFRZ _AC(0x800, UL) +#define CTRCTL_LCOFIFRZ _AC(0x1000, UL) +#define CTRCTL_EXCINH _AC(0x200000000, UL) +#define CTRCTL_INTRINH _AC(0x400000000, UL) +#define CTRCTL_TRETINH _AC(0x800000000, UL) +#define CTRCTL_NTBREN _AC(0x1000000000, UL) +#define CTRCTL_TKBRINH _AC(0x2000000000, UL) +#define CTRCTL_INDCALL_INH _AC(0x10000000000, UL) +#define CTRCTL_DIRCALL_INH _AC(0x20000000000, UL) +#define CTRCTL_INDJUMP_INH _AC(0x40000000000, UL) +#define CTRCTL_DIRJUMP_INH _AC(0x80000000000, UL) +#define CTRCTL_CORSWAP_INH _AC(0x100000000000, UL) +#define CTRCTL_RET_INH _AC(0x200000000000, UL) +#define CTRCTL_INDOJUMP_INH _AC(0x400000000000, UL) +#define CTRCTL_DIROJUMP_INH _AC(0x800000000000, UL) + +/* sctrstatus CSR bits. */ +#define SCTRSTATUS_WRPTR_MASK 0xFF +#define SCTRSTATUS_FROZEN _AC(0x80000000, UL) + +#ifdef CONFIG_RISCV_M_MODE +#define CTRCTL_KERNEL_ENABLE CTRCTL_M_ENABLE +#else +#define CTRCTL_KERNEL_ENABLE CTRCTL_S_ENABLE +#endif + +/* sctrdepth CSR bits. */ +#define SCTRDEPTH_MASK 0x7 + +#define SCTRDEPTH_MIN 0x0 /* 16 Entries. */ +#define SCTRDEPTH_MAX 0x4 /* 256 Entries. */ + +/* ctrsource, ctrtarget and ctrdata CSR bits. */ +#define CTRSOURCE_VALID 0x1ULL +#define CTRTARGET_MISP 0x1ULL + +#define CTRDATA_TYPE_MASK 0xF +#define CTRDATA_CCV 0x8000 +#define CTRDATA_CCM_MASK 0xFFF0000 +#define CTRDATA_CCE_MASK 0xF0000000 + +#define CTRDATA_TYPE_NONE 0 +#define CTRDATA_TYPE_EXCEPTION 1 +#define CTRDATA_TYPE_INTERRUPT 2 +#define CTRDATA_TYPE_TRAP_RET 3 +#define CTRDATA_TYPE_NONTAKEN_BRANCH 4 +#define CTRDATA_TYPE_TAKEN_BRANCH 5 +#define CTRDATA_TYPE_RESERVED_6 6 +#define CTRDATA_TYPE_RESERVED_7 7 +#define CTRDATA_TYPE_INDIRECT_CALL 8 +#define CTRDATA_TYPE_DIRECT_CALL 9 +#define CTRDATA_TYPE_INDIRECT_JUMP 10 +#define CTRDATA_TYPE_DIRECT_JUMP 11 +#define CTRDATA_TYPE_CO_ROUTINE_SWAP 12 +#define CTRDATA_TYPE_RETURN 13 +#define CTRDATA_TYPE_OTHER_INDIRECT_JUMP 14 +#define CTRDATA_TYPE_OTHER_DIRECT_JUMP 15 + +#define CTR_ENTRIES_FIRST 0x200 +#define CTR_ENTRIES_LAST 0x2ff + #define CSR_SSTATUS 0x100 #define CSR_SIE 0x104 #define CSR_STVEC 0x105 @@ -490,6 +569,8 @@ # define CSR_TOPEI CSR_MTOPEI # define CSR_TOPI CSR_MTOPI =20 +# define CSR_CTRCTL CSR_MCTRCTL + # define SR_IE SR_MIE # define SR_PIE SR_MPIE # define SR_PP SR_MPP @@ -520,6 +601,8 @@ # define CSR_TOPEI CSR_STOPEI # define CSR_TOPI CSR_STOPI =20 +# define CSR_CTRCTL CSR_SCTRCTL + # define SR_IE SR_SIE # define SR_PIE SR_SPIE # define SR_PP SR_SPP --=20 2.34.1 From nobody Tue Dec 16 22:17:05 2025 Received: from mail-wr1-f49.google.com (mail-wr1-f49.google.com [209.85.221.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A6B0E1C2319 for ; Wed, 29 May 2024 18:54:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717008876; cv=none; b=tvU+d3xDvT7jaUrJu7oimop3Oz6sNPzgibktsqYQ16Cp2rKz6KbtUbbLA2l6W4qnOWEs9XU5du6VFyAfykMMqAce1AzJPETDRmfj4h9zlptSE7HzVr0XDka6PAXb6qZSnK35QUt5dsxEyQ8/iH4D0IZas6m0bOz4jVd4Ymp5gk0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717008876; c=relaxed/simple; bh=fMpo7uyRB4y7EFUUS5JpPLDb1F+bGUsB2lDFWPSLOEg=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=iTo2O+PNyuQhiMuoeaXuUnyRE9+uyllwGhz6EVrtoSAsRIwxaHwSc0XRtGCXs3yAEF6aKh7HDW89Osx8GnyUSSNWN2DJyKDg+VmAIlUyx9OFy4Bk0TKNTF4ixt0x2OvjHdgsmSOgjvDUFWe231/fsV9kFPqIw9SGo3xFmQNQ1to= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=JpKCexS0; arc=none smtp.client-ip=209.85.221.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="JpKCexS0" Received: by mail-wr1-f49.google.com with SMTP id ffacd0b85a97d-354de3c5d00so20244f8f.1 for ; Wed, 29 May 2024 11:54:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1717008873; x=1717613673; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=BSkmcpX/DU8tSqXScAea3itOYSZ+Zk7JjFi4MBclVMg=; b=JpKCexS0d5fubLu2T0o7lTYPo82rcTpuKXaxl+1KMQScc0K0Kk5R4vazXhqZaIH2qT e3BJZ1TJluS6P+kS48FWAWm3xmRnabAsjTrCuSUnRyOz+JcxH3tSd/fa7gC43oDsGylD k/NRvn5dHkiWQVbOt8L0tbO8KJluxTDAKpfYtw4E8isxs0H4KLTtkUJPaYiQciplLQWH yAzLYnXRm3Ho2yas/2LFti748WYC/rt9xc1/UJfOb/LPQct20dxRP+ELWKmYh1rBfqXw CHJYvVLC7xouduyAcJX18k03ixLxBZB36nlY0EvFjSKeBLgjZi743fKcNr+oXJC0+7Cp hoHA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717008873; x=1717613673; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=BSkmcpX/DU8tSqXScAea3itOYSZ+Zk7JjFi4MBclVMg=; b=oK7oojJHHKEMeL7gmqC9+XElXGLuU2GAoBJJqwtlU16iezSChuClW7CsdjwSLHNHBK u+3WUIZF4oxMqaCGDSa8QWWRf+579ghDTlqsUUqcZORG2osTCbSE5zLZ5civXNe03yp+ r11rbyBnB88xCIRkziHp9i6c8ddirlgAvSbM1u0aEcmd3VfswsjmcPh5vwPdOlnLkmf8 K+s6eTIvAqRlG7AIUzZGmXpCuitm/TeDmZcaeZrl/u3RiA3GZF5f16Ci0kYNIHAFrwXc 4jrNxoFwdamNyEJBapiNQ5aV+f64kCHcUlqW2cxykgVCgGwI3bfc4n1KNw0CavFD8IlB AHhg== X-Gm-Message-State: AOJu0YwHht0AQ0CK6HHcIiGi8evvTZWdOdXXzd3kbljx4JGa4QPKr5QH FhM9G4BK7EdFGI6RgQnZaTlfTw7P3BVYwBiD2vQjlk6ZTEw1ZskbUxfQ7LJ4aNtFuFenRNsUy6q g X-Google-Smtp-Source: AGHT+IHHl9KK9z/L8izKsRybmVqfP8bMjXHwVAHsTsF/TnNuIRcfgvHrqZiLQ7fq4FiWPMrXEUFK7A== X-Received: by 2002:adf:f6cd:0:b0:354:fce5:4cc3 with SMTP id ffacd0b85a97d-35c7a8477f9mr2527916f8f.19.1717008872954; Wed, 29 May 2024 11:54:32 -0700 (PDT) Received: from rkanwal-XPS-15-9520.Home ([2a02:c7c:7527:ee00:7446:71c1:a41a:da9b]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4212706a23csm2787885e9.27.2024.05.29.11.54.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 May 2024 11:54:32 -0700 (PDT) From: Rajnesh Kanwal To: linux-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org, linux-riscv@lists.infradead.org, adrian.hunter@intel.com, alexander.shishkin@linux.intel.com, ajones@ventanamicro.com, anup@brainfault.org, acme@kernel.org, atishp@rivosinc.com, beeman@rivosinc.com, brauner@kernel.org, conor@kernel.org, heiko@sntech.de, irogers@google.com, mingo@redhat.com, james.clark@arm.com, renyu.zj@linux.alibaba.com, jolsa@kernel.org, jisheng.teoh@starfivetech.com, palmer@dabbelt.com, tech-control-transfer-records@lists.riscv.org, will@kernel.org, kaiwenxue1@gmail.com, Rajnesh Kanwal Subject: [PATCH RFC 3/6] riscv: perf: Add Control Transfer Records extension parsing Date: Wed, 29 May 2024 19:53:34 +0100 Message-Id: <20240529185337.182722-4-rkanwal@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240529185337.182722-1-rkanwal@rivosinc.com> References: <20240529185337.182722-1-rkanwal@rivosinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Adding CTR extension in ISA extension map to lookup for extension availability. Signed-off-by: Rajnesh Kanwal --- arch/riscv/include/asm/hwcap.h | 4 ++++ arch/riscv/kernel/cpufeature.c | 2 ++ 2 files changed, 6 insertions(+) diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h index b8cc459ee8a4..aff5ef398671 100644 --- a/arch/riscv/include/asm/hwcap.h +++ b/arch/riscv/include/asm/hwcap.h @@ -86,6 +86,8 @@ #define RISCV_ISA_EXT_SSCCFG 77 #define RISCV_ISA_EXT_SMCDELEG 78 #define RISCV_ISA_EXT_SMCNTRPMF 79 +#define RISCV_ISA_EXT_SMCTR 80 +#define RISCV_ISA_EXT_SSCTR 81 =20 #define RISCV_ISA_EXT_XLINUXENVCFG 127 =20 @@ -95,9 +97,11 @@ #ifdef CONFIG_RISCV_M_MODE #define RISCV_ISA_EXT_SxAIA RISCV_ISA_EXT_SMAIA #define RISCV_ISA_EXT_SxCSRIND RISCV_ISA_EXT_SMCSRIND +#define RISCV_ISA_EXT_SxCTR RISCV_ISA_EXT_SMCTR #else #define RISCV_ISA_EXT_SxAIA RISCV_ISA_EXT_SSAIA #define RISCV_ISA_EXT_SxCSRIND RISCV_ISA_EXT_SSCSRIND +#define RISCV_ISA_EXT_SxCTR RISCV_ISA_EXT_SSCTR #endif =20 #endif /* _ASM_RISCV_HWCAP_H */ diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index d1fb6a8c5492..4334d822b2f2 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -298,6 +298,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] =3D { __RISCV_ISA_EXT_DATA(zvkt, RISCV_ISA_EXT_ZVKT), __RISCV_ISA_EXT_DATA(smaia, RISCV_ISA_EXT_SMAIA), __RISCV_ISA_EXT_DATA(smcdeleg, RISCV_ISA_EXT_SMCDELEG), + __RISCV_ISA_EXT_DATA(smctr, RISCV_ISA_EXT_SMCTR), __RISCV_ISA_EXT_DATA(smstateen, RISCV_ISA_EXT_SMSTATEEN), __RISCV_ISA_EXT_DATA(smcntrpmf, RISCV_ISA_EXT_SMCNTRPMF), __RISCV_ISA_EXT_DATA(smcsrind, RISCV_ISA_EXT_SMCSRIND), @@ -305,6 +306,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] =3D { __RISCV_ISA_EXT_DATA(sscsrind, RISCV_ISA_EXT_SSCSRIND), __RISCV_ISA_EXT_DATA(ssccfg, RISCV_ISA_EXT_SSCCFG), __RISCV_ISA_EXT_DATA(sscofpmf, RISCV_ISA_EXT_SSCOFPMF), + __RISCV_ISA_EXT_DATA(ssctr, RISCV_ISA_EXT_SSCTR), __RISCV_ISA_EXT_DATA(sstc, RISCV_ISA_EXT_SSTC), __RISCV_ISA_EXT_DATA(svinval, RISCV_ISA_EXT_SVINVAL), __RISCV_ISA_EXT_DATA(svnapot, RISCV_ISA_EXT_SVNAPOT), --=20 2.34.1 From nobody Tue Dec 16 22:17:05 2025 Received: from mail-wm1-f51.google.com (mail-wm1-f51.google.com [209.85.128.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3F4A51C2305 for ; Wed, 29 May 2024 18:54:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717008878; cv=none; b=rgvEMW/XSNmbUCN0bQVSLR+Um3ZzT9Vg65f0xN7c52kYfuca4TBZG1/pv7G/cxLs/pM6TkOU4NULqvka7j37K6iPK4v9iBZxyaHzGipQDMcs5fA567JZ7k47sB/qLdUIa9f8fWolW77vARB1RJVibaqMMBf0tW86J79bzFzFhHE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717008878; c=relaxed/simple; bh=ey72yrLeOq5dkzbSyIayVN5J/Pk9SHcSRo8jLUBOHDE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=HYyA8n7lxeqyKSlaOw2qJF/uTaNkCTnEj1TGDNTDT6VIFz7d7SY3cf6CHbRAqnLpD9Rflj9FPy18dFI2qy5Qa3mcX9jaFCp9c3x3epQXWRmKEhuFpwpy8YFgCs35+8JqtHYY0Ygpr4JHPdGSGHQ+gYNwjwVdnLJi+3lKz6w5WNI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=VOnnQHql; arc=none smtp.client-ip=209.85.128.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="VOnnQHql" Received: by mail-wm1-f51.google.com with SMTP id 5b1f17b1804b1-4211249fbafso304495e9.3 for ; Wed, 29 May 2024 11:54:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1717008874; x=1717613674; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=b7vDEMF2Cse3GKKKWWJPFMdZWN9OHL0jv0BqXqFqZx0=; b=VOnnQHqlyXC01EgFqQW41VhI05t3lG2PiUCMjGVc8aVqjzYocGzklvTrJy+/mCbbTD eqBkqYvWWMC2433HPbOpMK3+T6MJK9rkerAgXgwx1JvKuCJmEHrDohcq8/jmTC8sM5J6 IZffwupmHFYqjjiG4Aldyud6xCggSX3dDtd26Kg83qHmwlY0//de4rUjawAtUFjYNRBG dAt6WQlqR64kT1v8oqW/rxobRiHX49MTvMWytaLtv0Kj8nneZ4hEl8vAkw67O3oC2KPv OHgbGLX14legP2cZkTE+PyrL3zo8mgzBajh4yPcARxttQcnOBvC2kdwGXE/ceMy3+lDq ZkrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717008874; x=1717613674; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=b7vDEMF2Cse3GKKKWWJPFMdZWN9OHL0jv0BqXqFqZx0=; b=cqWNP1iSca+zOtrlQe61oT7T/NyvSRI8+yoZ1E1jpjXwOO/UP9VlzoQycK9iBa/M/d wteTJ4Jac5eO6zWMNbXElglLaxYgyt8+nSPoWy645oPeiPO77Pji//Q8AYDXjkqx0MpV sXfMAnnaBcgLZ1/XIm3XAnMh9gndxB5XD7MFeW+uEEc6KTrYfrSRbwIr0j9AGw5hwOL/ 11H13bJvtcB0x7b2+RkCqI1BG0w5I9vCcxzUZsrvn9BRqN8W9va4oMWid/q+fPkX7PaR TkQItzaJNuky7h0zvemuKkTms7+4gmxmY5SIcGqhE/b60k/7EMDXM+O+AgeTFezOo754 FATQ== X-Gm-Message-State: AOJu0Yzys3YQk9zLyMzNdZ5zLztWtb+DoJbUG/gKZhpDjs6Eu7ebZu2u 5UgWMdxbrMqvlhAtjJMHw31k9CaCofKTfYDt4f++Stzi7CMzMcpM5Hv9kyWKRl2lHJjZ8cDiqX8 z X-Google-Smtp-Source: AGHT+IG0aFRXbEPemdv8ro3r0XN3H/rTAxOwWSQmMr3iD1+X74b6QY7y655sl+IdhOd5xfv/zlwFfw== X-Received: by 2002:a05:600c:1d9a:b0:421:2065:3799 with SMTP id 5b1f17b1804b1-42127929560mr208155e9.29.1717008873888; Wed, 29 May 2024 11:54:33 -0700 (PDT) Received: from rkanwal-XPS-15-9520.Home ([2a02:c7c:7527:ee00:7446:71c1:a41a:da9b]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4212706a23csm2787885e9.27.2024.05.29.11.54.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 May 2024 11:54:33 -0700 (PDT) From: Rajnesh Kanwal To: linux-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org, linux-riscv@lists.infradead.org, adrian.hunter@intel.com, alexander.shishkin@linux.intel.com, ajones@ventanamicro.com, anup@brainfault.org, acme@kernel.org, atishp@rivosinc.com, beeman@rivosinc.com, brauner@kernel.org, conor@kernel.org, heiko@sntech.de, irogers@google.com, mingo@redhat.com, james.clark@arm.com, renyu.zj@linux.alibaba.com, jolsa@kernel.org, jisheng.teoh@starfivetech.com, palmer@dabbelt.com, tech-control-transfer-records@lists.riscv.org, will@kernel.org, kaiwenxue1@gmail.com, Rajnesh Kanwal Subject: [PATCH RFC 4/6] riscv: perf: Add infrastructure for Control Transfer Record Date: Wed, 29 May 2024 19:53:35 +0100 Message-Id: <20240529185337.182722-5-rkanwal@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240529185337.182722-1-rkanwal@rivosinc.com> References: <20240529185337.182722-1-rkanwal@rivosinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" To support Control Transfer Records (CTR) extension, we need to extend the riscv_pmu framework with some basic infrastructure for branch stack samplin= g. Subsequent patches will use this to add support for CTR in the riscv_pmu_dev driver. With CTR, the branches are stored into a hardware FIFO, which will be sampl= ed by software when perf events overflow. A task may be context- switched betw= een overflows, and to avoid leaking samples we need to clear the last task's records when a task is context-switched In. To do this we will be using the pmu::sched_task() callback added in this patch. Signed-off-by: Rajnesh Kanwal --- drivers/perf/riscv_pmu_common.c | 15 +++++++++++++++ drivers/perf/riscv_pmu_dev.c | 9 +++++++++ include/linux/perf/riscv_pmu.h | 16 ++++++++++++++++ 3 files changed, 40 insertions(+) diff --git a/drivers/perf/riscv_pmu_common.c b/drivers/perf/riscv_pmu_commo= n.c index b4efdddb2ad9..e794675e4944 100644 --- a/drivers/perf/riscv_pmu_common.c +++ b/drivers/perf/riscv_pmu_common.c @@ -159,6 +159,19 @@ u64 riscv_pmu_ctr_get_width_mask(struct perf_event *ev= ent) return GENMASK_ULL(cwidth, 0); } =20 +static void riscv_pmu_sched_task(struct perf_event_pmu_context *pmu_ctx, + bool sched_in) +{ + struct riscv_pmu *pmu; + + if (!pmu_ctx) + return; + + pmu =3D to_riscv_pmu(pmu_ctx->pmu); + if (pmu->sched_task) + pmu->sched_task(pmu_ctx, sched_in); +} + u64 riscv_pmu_event_update(struct perf_event *event) { struct riscv_pmu *rvpmu =3D to_riscv_pmu(event->pmu); @@ -406,6 +419,7 @@ struct riscv_pmu *riscv_pmu_alloc(void) for_each_possible_cpu(cpuid) { cpuc =3D per_cpu_ptr(pmu->hw_events, cpuid); cpuc->n_events =3D 0; + cpuc->ctr_users =3D 0; for (i =3D 0; i < RISCV_MAX_COUNTERS; i++) cpuc->events[i] =3D NULL; } @@ -419,6 +433,7 @@ struct riscv_pmu *riscv_pmu_alloc(void) .start =3D riscv_pmu_start, .stop =3D riscv_pmu_stop, .read =3D riscv_pmu_read, + .sched_task =3D riscv_pmu_sched_task, }; =20 return pmu; diff --git a/drivers/perf/riscv_pmu_dev.c b/drivers/perf/riscv_pmu_dev.c index 5ca8a909f3ab..40ae5fc897a3 100644 --- a/drivers/perf/riscv_pmu_dev.c +++ b/drivers/perf/riscv_pmu_dev.c @@ -670,6 +670,14 @@ static void rvpmu_sbi_ctr_stop(struct perf_event *even= t, unsigned long flag) hwc->idx, sbi_err_map_linux_errno(ret.error)); } =20 +static void pmu_sched_task(struct perf_event_pmu_context *pmu_ctx, + bool sched_in) +{ + struct riscv_pmu *pmu =3D to_riscv_pmu(pmu_ctx->pmu); + + /* Call CTR specific Sched hook. */ +} + static int rvpmu_sbi_find_num_ctrs(void) { struct sbiret ret; @@ -1494,6 +1502,7 @@ static int rvpmu_device_probe(struct platform_device = *pdev) pmu->event_mapped =3D rvpmu_event_mapped; pmu->event_unmapped =3D rvpmu_event_unmapped; pmu->csr_index =3D rvpmu_csr_index; + pmu->sched_task =3D pmu_sched_task; =20 ret =3D cpuhp_state_add_instance(CPUHP_AP_PERF_RISCV_STARTING, &pmu->node= ); if (ret) diff --git a/include/linux/perf/riscv_pmu.h b/include/linux/perf/riscv_pmu.h index 425edd6685a9..5a6b840018bd 100644 --- a/include/linux/perf/riscv_pmu.h +++ b/include/linux/perf/riscv_pmu.h @@ -33,6 +33,13 @@ #define RISCV_PMU_CYCLE_FIXED_CTR_MASK 0x01 #define RISCV_PMU_INSTRUCTION_FIXED_CTR_MASK 0x04 =20 +#define MAX_BRANCH_RECORDS 256 + +struct branch_records { + struct perf_branch_stack branch_stack; + struct perf_branch_entry branch_entries[MAX_BRANCH_RECORDS]; +}; + struct cpu_hw_events { /* currently enabled events */ int n_events; @@ -44,6 +51,12 @@ struct cpu_hw_events { DECLARE_BITMAP(used_hw_ctrs, RISCV_MAX_COUNTERS); /* currently enabled firmware counters */ DECLARE_BITMAP(used_fw_ctrs, RISCV_MAX_COUNTERS); + + /* Saved branch records. */ + struct branch_records *branches; + + /* Active events requesting branch records */ + int ctr_users; }; =20 struct riscv_pmu { @@ -64,10 +77,13 @@ struct riscv_pmu { void (*event_mapped)(struct perf_event *event, struct mm_struct *mm); void (*event_unmapped)(struct perf_event *event, struct mm_struct *mm); uint8_t (*csr_index)(struct perf_event *event); + void (*sched_task)(struct perf_event_pmu_context *ctx, bool sched_in); =20 struct cpu_hw_events __percpu *hw_events; struct hlist_node node; struct notifier_block riscv_pm_nb; + + unsigned int ctr_depth; }; =20 #define to_riscv_pmu(p) (container_of(p, struct riscv_pmu, pmu)) --=20 2.34.1 From nobody Tue Dec 16 22:17:05 2025 Received: from mail-wm1-f49.google.com (mail-wm1-f49.google.com [209.85.128.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 639F41C6896 for ; Wed, 29 May 2024 18:54:37 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717008879; cv=none; b=k1M4wdZkH+7TrgX4mDohHE/pIy8TXlH8d/B9CiqbFMdIxJ4PvJEfW54djmt3/+BIw4g3IJvcsxqf3Fs4fGrC9bpyCMTPlxjfrfveUhOtodIwuKvoHpe6DfWQBGy200EIouMQV9idWnhscyr3C8M3TSCo042c+QBMASD8sbvHVTM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717008879; c=relaxed/simple; bh=acsH4CXflg8MWwftRGHOCAfjnNVksPUpC1VI99j84PE=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=s2L+vliVUBMYgpWbA2Rf9ES0gbd7yTHT5Qr6Ecosw16nekhDKxNMgnkHprO5VF6yKxvKuPaQfDDSXOAMhkg34SsAaRqwSBB7tJhWzvI4RlHEiZQNUiSL9PH2XMckZl29tm2CuZ6sB10/ezjgxMHd5z9EloMJHKHkbvrIEZyMYM8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=z33m9uzE; arc=none smtp.client-ip=209.85.128.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="z33m9uzE" Received: by mail-wm1-f49.google.com with SMTP id 5b1f17b1804b1-42017f8de7aso502495e9.1 for ; Wed, 29 May 2024 11:54:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1717008875; x=1717613675; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=0o+B1b3b9otKzfoRya637jHIfmbAgjTIaV6zgoFCmBw=; b=z33m9uzEBrrPztugdjncliVvPsAxS0TbDE1qOsWerMogqKoEfTEF/iCSqEUoLpHFVs H6Oexj4ngz1X39VkgcSn2TmM4roKdLurkiseYNVRFEDoYkmMlj3Gwtwncb7SLigytkR7 vxK+MCad4qJMcyJNWH4hk2TDULB8gDD9sPNTYQsPXxNJ6T1/ropOoDksJn4voLKAGVzr YUIescN6e2NedpST2v6IF8akoID2o00b+L5gZlVjOvyOyiHUrJ7MZ1HsI0bgN8hnhgBQ A3SnJYOlfbCTjM266aiR3eSEerZnFJUsV5wm9WCZCwvQ07Ral4/xDZdeEXyqo0m2n2/Z AUQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717008875; x=1717613675; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0o+B1b3b9otKzfoRya637jHIfmbAgjTIaV6zgoFCmBw=; b=kYTUsFyF8o6s/XwGwqA9ssdp6eO+Q7Sx9IGhmnz7SNHI2AmA4fl36L3e36Aa7lb8i0 Ry6MFWRoiubymp+SUi/xjiYXORRoqw4k6XGO8DVv5EyVe/G4hPVNItwQIsD07iS6ftbd 5IhX/sph94za0dz02rpWxkLgSwiktTjpL8O2zbs0pZZPsAjGgAnEyWI/L/sSuyzcAkkP utKBJzOi236v7mDUCDKULi+8RahPcKo+DjNBdkhnKOQd7xToqRmhcW5umHxpgm6Xqk85 Pat+NUU0Y/sn4SMRveMArFMHFYhHPZU8WJ5375HjJAxmwUGbYN93UoOV97eOX/5Xvk36 rcuA== X-Gm-Message-State: AOJu0YxGdeGGbvxHkmUndOxjhMC126a+uvTPqsOLl4bxI+ZL1CSiqAWK ENUVCWldUryVHEHRp36Ui+v2uvp6auAy/k3SDIynoRaY/Bv/M1RjniJiVvzBueIe7HkfbN9rlsS v X-Google-Smtp-Source: AGHT+IErFHVKwDzA+pRnaE1cowj8Cur30CsYYW0yRqy5Cyv1C3pRbjxr/J2rzgCC4GvWtUmPlzLsaw== X-Received: by 2002:a05:600c:a03:b0:416:2471:e102 with SMTP id 5b1f17b1804b1-4212792c381mr196775e9.37.1717008875165; Wed, 29 May 2024 11:54:35 -0700 (PDT) Received: from rkanwal-XPS-15-9520.Home ([2a02:c7c:7527:ee00:7446:71c1:a41a:da9b]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4212706a23csm2787885e9.27.2024.05.29.11.54.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 May 2024 11:54:34 -0700 (PDT) From: Rajnesh Kanwal To: linux-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org, linux-riscv@lists.infradead.org, adrian.hunter@intel.com, alexander.shishkin@linux.intel.com, ajones@ventanamicro.com, anup@brainfault.org, acme@kernel.org, atishp@rivosinc.com, beeman@rivosinc.com, brauner@kernel.org, conor@kernel.org, heiko@sntech.de, irogers@google.com, mingo@redhat.com, james.clark@arm.com, renyu.zj@linux.alibaba.com, jolsa@kernel.org, jisheng.teoh@starfivetech.com, palmer@dabbelt.com, tech-control-transfer-records@lists.riscv.org, will@kernel.org, kaiwenxue1@gmail.com, Rajnesh Kanwal Subject: [PATCH RFC 5/6] riscv: perf: Add driver for Control Transfer Records Ext. Date: Wed, 29 May 2024 19:53:36 +0100 Message-Id: <20240529185337.182722-6-rkanwal@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240529185337.182722-1-rkanwal@rivosinc.com> References: <20240529185337.182722-1-rkanwal@rivosinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This adds support for CTR Ext defined in [0]. The extension allows to records a maximum for 256 last branch records. CTR extension depends on s[m|s]csrind and Sscofpmf extensions. Signed-off-by: Rajnesh Kanwal --- MAINTAINERS | 1 + drivers/perf/Kconfig | 11 + drivers/perf/Makefile | 1 + drivers/perf/riscv_ctr.c | 469 +++++++++++++++++++++++++++++++++ include/linux/perf/riscv_pmu.h | 33 +++ 5 files changed, 515 insertions(+) create mode 100644 drivers/perf/riscv_ctr.c diff --git a/MAINTAINERS b/MAINTAINERS index d6b42d5f62da..868e4b0808ab 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -19056,6 +19056,7 @@ M: Atish Patra R: Anup Patel L: linux-riscv@lists.infradead.org S: Supported +F: drivers/perf/riscv_ctr.c F: drivers/perf/riscv_pmu_common.c F: drivers/perf/riscv_pmu_dev.c F: drivers/perf/riscv_pmu_legacy.c diff --git a/drivers/perf/Kconfig b/drivers/perf/Kconfig index 3c37577b25f7..cca6598be739 100644 --- a/drivers/perf/Kconfig +++ b/drivers/perf/Kconfig @@ -110,6 +110,17 @@ config ANDES_CUSTOM_PMU =20 If you don't know what to do here, say "Y". =20 +config RISCV_CTR + bool "Enable support for Control Transfer Records (CTR)" + depends on PERF_EVENTS && RISCV_PMU + default y + help + Enable support for Control Transfer Records (CTR) which + allows recording branches, Jumps, Calls, returns etc taken in an + execution path. This also supports privilege based filtering. It + captures additional relevant information such as cycle count, + branch misprediction etc. + config ARM_PMU_ACPI depends on ARM_PMU && ACPI def_bool y diff --git a/drivers/perf/Makefile b/drivers/perf/Makefile index ba809cc069d5..364b1f66f410 100644 --- a/drivers/perf/Makefile +++ b/drivers/perf/Makefile @@ -16,6 +16,7 @@ obj-$(CONFIG_RISCV_PMU_COMMON) +=3D riscv_pmu_common.o obj-$(CONFIG_RISCV_PMU_LEGACY) +=3D riscv_pmu_legacy.o obj-$(CONFIG_RISCV_PMU) +=3D riscv_pmu_dev.o obj-$(CONFIG_STARFIVE_STARLINK_PMU) +=3D starfive_starlink_pmu.o +obj-$(CONFIG_RISCV_CTR) +=3D riscv_ctr.o obj-$(CONFIG_THUNDERX2_PMU) +=3D thunderx2_pmu.o obj-$(CONFIG_XGENE_PMU) +=3D xgene_pmu.o obj-$(CONFIG_ARM_SPE_PMU) +=3D arm_spe_pmu.o diff --git a/drivers/perf/riscv_ctr.c b/drivers/perf/riscv_ctr.c new file mode 100644 index 000000000000..95fda1edda4f --- /dev/null +++ b/drivers/perf/riscv_ctr.c @@ -0,0 +1,469 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Control transfer records extension Helpers. + * + * Copyright (C) 2024 Rivos Inc. + * + * Author: Rajnesh Kanwal + */ + +#define pr_fmt(fmt) "CTR: " fmt + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define CTR_BRANCH_FILTERS_INH (CTRCTL_EXCINH | \ + CTRCTL_INTRINH | \ + CTRCTL_TRETINH | \ + CTRCTL_TKBRINH | \ + CTRCTL_INDCALL_INH | \ + CTRCTL_DIRCALL_INH | \ + CTRCTL_INDJUMP_INH | \ + CTRCTL_DIRJUMP_INH | \ + CTRCTL_CORSWAP_INH | \ + CTRCTL_RET_INH | \ + CTRCTL_INDOJUMP_INH | \ + CTRCTL_DIROJUMP_INH) + +#define CTR_BRANCH_ENABLE_BITS (CTRCTL_KERNEL_ENABLE | CTRCTL_U_ENABLE) + +/* Branch filters not-supported by CTR extension. */ +#define CTR_EXCLUDE_BRANCH_FILTERS (PERF_SAMPLE_BRANCH_ABORT_TX | \ + PERF_SAMPLE_BRANCH_IN_TX | \ + PERF_SAMPLE_BRANCH_PRIV_SAVE | \ + PERF_SAMPLE_BRANCH_NO_TX | \ + PERF_SAMPLE_BRANCH_COUNTERS) + +/* Branch filters supported by CTR extension. */ +#define CTR_ALLOWED_BRANCH_FILTERS (PERF_SAMPLE_BRANCH_USER | \ + PERF_SAMPLE_BRANCH_KERNEL | \ + PERF_SAMPLE_BRANCH_HV | \ + PERF_SAMPLE_BRANCH_ANY | \ + PERF_SAMPLE_BRANCH_ANY_CALL | \ + PERF_SAMPLE_BRANCH_ANY_RETURN | \ + PERF_SAMPLE_BRANCH_IND_CALL | \ + PERF_SAMPLE_BRANCH_COND | \ + PERF_SAMPLE_BRANCH_IND_JUMP | \ + PERF_SAMPLE_BRANCH_HW_INDEX | \ + PERF_SAMPLE_BRANCH_NO_FLAGS | \ + PERF_SAMPLE_BRANCH_NO_CYCLES | \ + PERF_SAMPLE_BRANCH_CALL_STACK | \ + PERF_SAMPLE_BRANCH_CALL | \ + PERF_SAMPLE_BRANCH_TYPE_SAVE) + +#define CTR_PERF_BRANCH_FILTERS (CTR_ALLOWED_BRANCH_FILTERS | \ + CTR_EXCLUDE_BRANCH_FILTERS) + +static u64 allowed_filters __read_mostly; + +struct ctr_regset { + unsigned long src; + unsigned long target; + unsigned long ctr_data; +}; + +static inline u64 get_ctr_src_reg(unsigned int ctr_idx) +{ + return csr_ind_read(CSR_IREG, CTR_ENTRIES_FIRST, ctr_idx); +} + +static inline u64 get_ctr_tgt_reg(unsigned int ctr_idx) +{ + return csr_ind_read(CSR_IREG2, CTR_ENTRIES_FIRST, ctr_idx); +} + +static inline u64 get_ctr_data_reg(unsigned int ctr_idx) +{ + return csr_ind_read(CSR_IREG3, CTR_ENTRIES_FIRST, ctr_idx); +} + +static inline bool ctr_record_valid(u64 ctr_src) +{ + return !!FIELD_GET(CTRSOURCE_VALID, ctr_src); +} + +static inline int ctr_get_mispredict(u64 ctr_target) +{ + return FIELD_GET(CTRTARGET_MISP, ctr_target); +} + +static inline unsigned int ctr_get_cycles(u64 ctr_data) +{ + const unsigned int cce =3D FIELD_GET(CTRDATA_CCE_MASK, ctr_data); + const unsigned int ccm =3D FIELD_GET(CTRDATA_CCM_MASK, ctr_data); + + if (ctr_data & CTRDATA_CCV) + return 0; + + /* Formula to calculate cycles from spec: (2^12 + CCM) << CCE-1 */ + if (cce > 0) + return (4096 + ccm) << (cce - 1); + + return FIELD_GET(CTRDATA_CCM_MASK, ctr_data); +} + +static inline unsigned int ctr_get_type(u64 ctr_data) +{ + return FIELD_GET(CTRDATA_TYPE_MASK, ctr_data); +} + +static inline unsigned int ctr_get_depth(u64 ctr_depth) +{ + /* Depth table from CTR Spec: 2.4 sctrdepth. + * + * sctrdepth.depth Depth + * 000 - 16 + * 001 - 32 + * 010 - 64 + * 011 - 128 + * 100 - 256 + * + * Depth =3D 16 * 2 ^ (ctrdepth.depth) + * or + * Depth =3D 16 << ctrdepth.depth. + */ + return 16 << FIELD_GET(SCTRDEPTH_MASK, ctr_depth); +} + +/* Reads CTR entry at idx and stores it in entry struct. */ +static bool capture_ctr_regset(struct ctr_regset *entry, unsigned int idx) +{ + entry->src =3D get_ctr_src_reg(idx); + + if (!ctr_record_valid(entry->src)) + return false; + + entry->src =3D entry->src & (~CTRSOURCE_VALID); + entry->target =3D get_ctr_tgt_reg(idx); + entry->ctr_data =3D get_ctr_data_reg(idx); + + return true; +} + +static u64 branch_type_to_ctr(int branch_type) +{ + u64 config =3D CTR_BRANCH_FILTERS_INH | CTRCTL_LCOFIFRZ; + + if (branch_type & PERF_SAMPLE_BRANCH_USER) + config |=3D CTRCTL_U_ENABLE; + + if (branch_type & PERF_SAMPLE_BRANCH_KERNEL) + config |=3D CTRCTL_KERNEL_ENABLE; + + if (branch_type & PERF_SAMPLE_BRANCH_HV) { + if (riscv_isa_extension_available(NULL, h)) + config |=3D CTRCTL_KERNEL_ENABLE; + } + + if (branch_type & PERF_SAMPLE_BRANCH_ANY) { + config &=3D ~CTR_BRANCH_FILTERS_INH; + return config; + } + + if (branch_type & PERF_SAMPLE_BRANCH_ANY_CALL) { + config &=3D ~CTRCTL_INDCALL_INH; + config &=3D ~CTRCTL_DIRCALL_INH; + config &=3D ~CTRCTL_EXCINH; + config &=3D ~CTRCTL_INTRINH; + } + + if (branch_type & PERF_SAMPLE_BRANCH_ANY_RETURN) + config &=3D ~(CTRCTL_RET_INH | CTRCTL_TRETINH); + + if (branch_type & PERF_SAMPLE_BRANCH_IND_CALL) + config &=3D ~CTRCTL_INDCALL_INH; + + if (branch_type & PERF_SAMPLE_BRANCH_COND) + config &=3D ~CTRCTL_TKBRINH; + + if (branch_type & PERF_SAMPLE_BRANCH_CALL_STACK) { + config &=3D ~(CTRCTL_INDCALL_INH | CTRCTL_DIRCALL_INH | + CTRCTL_RET_INH); + config |=3D CTRCTL_RASEMU; + } + + if (branch_type & PERF_SAMPLE_BRANCH_IND_JUMP) { + config &=3D ~CTRCTL_INDJUMP_INH; + config &=3D ~CTRCTL_INDOJUMP_INH; + } + + if (branch_type & PERF_SAMPLE_BRANCH_CALL) + config &=3D ~CTRCTL_DIRCALL_INH; + + return config; +} + +static const int ctr_perf_map[] =3D { + [CTRDATA_TYPE_NONE] =3D PERF_BR_UNKNOWN, + [CTRDATA_TYPE_EXCEPTION] =3D PERF_BR_SYSCALL, + [CTRDATA_TYPE_INTERRUPT] =3D PERF_BR_IRQ, + [CTRDATA_TYPE_TRAP_RET] =3D PERF_BR_ERET, + [CTRDATA_TYPE_NONTAKEN_BRANCH] =3D PERF_BR_COND, + [CTRDATA_TYPE_TAKEN_BRANCH] =3D PERF_BR_COND, + [CTRDATA_TYPE_RESERVED_6] =3D PERF_BR_UNKNOWN, + [CTRDATA_TYPE_RESERVED_7] =3D PERF_BR_UNKNOWN, + [CTRDATA_TYPE_INDIRECT_CALL] =3D PERF_BR_IND_CALL, + [CTRDATA_TYPE_DIRECT_CALL] =3D PERF_BR_CALL, + [CTRDATA_TYPE_INDIRECT_JUMP] =3D PERF_BR_UNCOND, + [CTRDATA_TYPE_DIRECT_JUMP] =3D PERF_BR_UNKNOWN, + [CTRDATA_TYPE_CO_ROUTINE_SWAP] =3D PERF_BR_UNKNOWN, + [CTRDATA_TYPE_RETURN] =3D PERF_BR_RET, + [CTRDATA_TYPE_OTHER_INDIRECT_JUMP] =3D PERF_BR_IND, + [CTRDATA_TYPE_OTHER_DIRECT_JUMP] =3D PERF_BR_UNKNOWN, +}; + +static void ctr_set_perf_entry_type(struct perf_branch_entry *entry, + u64 ctr_data) +{ + int ctr_type =3D ctr_get_type(ctr_data); + + entry->type =3D ctr_perf_map[ctr_type]; + if (entry->type =3D=3D PERF_BR_UNKNOWN) + pr_warn("%d - unknown branch type captured\n", ctr_type); +} + +static void capture_ctr_flags(struct perf_branch_entry *entry, + struct perf_event *event, u64 ctr_data, + u64 ctr_target) +{ + if (branch_sample_type(event)) + ctr_set_perf_entry_type(entry, ctr_data); + + if (!branch_sample_no_cycles(event)) + entry->cycles =3D ctr_get_cycles(ctr_data); + + if (!branch_sample_no_flags(event)) { + entry->abort =3D 0; + entry->mispred =3D ctr_get_mispredict(ctr_target); + entry->predicted =3D !entry->mispred; + } + + if (branch_sample_priv(event)) + entry->priv =3D PERF_BR_PRIV_UNKNOWN; +} + + +static void ctr_regset_to_branch_entry(struct cpu_hw_events *cpuc, + struct perf_event *event, + struct ctr_regset *regset, + unsigned int idx) +{ + struct perf_branch_entry *entry =3D &cpuc->branches->branch_entries[idx]; + + perf_clear_branch_entry_bitfields(entry); + entry->from =3D regset->src; + entry->to =3D regset->target & (~CTRTARGET_MISP); + capture_ctr_flags(entry, event, regset->ctr_data, regset->target); +} + +static void ctr_read_entries(struct cpu_hw_events *cpuc, + struct perf_event *event, + unsigned int depth) +{ + struct ctr_regset entry =3D {}; + u64 ctr_ctl; + int i; + + ctr_ctl =3D csr_read_clear(CSR_CTRCTL, CTR_BRANCH_ENABLE_BITS); + + for (i =3D 0; i < depth; i++) { + if (!capture_ctr_regset(&entry, i)) + break; + + ctr_regset_to_branch_entry(cpuc, event, &entry, i); + } + + csr_set(CSR_CTRCTL, ctr_ctl & CTR_BRANCH_ENABLE_BITS); + + cpuc->branches->branch_stack.nr =3D i; + cpuc->branches->branch_stack.hw_idx =3D 0; +} + +bool riscv_pmu_ctr_valid(struct perf_event *event) +{ + u64 branch_type =3D event->attr.branch_sample_type; + + if (branch_type & ~allowed_filters) { + pr_debug_once("Requested branch filters not supported 0x%llx\n", + branch_type & ~allowed_filters); + return false; + } + + return true; +} + +void riscv_pmu_ctr_consume(struct cpu_hw_events *cpuc, struct perf_event *= event) +{ + unsigned int depth =3D to_riscv_pmu(event->pmu)->ctr_depth; + + ctr_read_entries(cpuc, event, depth); + + /* Clear frozen bit. */ + csr_clear(CSR_SCTRSTATUS, SCTRSTATUS_FROZEN); +} + +static void riscv_pmu_ctr_clear(void) +{ + /* FIXME: Replace with sctrclr instruction once support is merged + * into toolchain. + */ + asm volatile(".4byte 0x10400073\n" ::: "memory"); + csr_write(CSR_SCTRSTATUS, 0); +} + +/* + * On context switch in, we need to make sure no samples from previous user + * are left in the CTR. + * + * On ctxswin, sched_in =3D true, called after the PMU has started + * On ctxswout, sched_in =3D false, called before the PMU is stopped + */ +void riscv_pmu_ctr_sched_task(struct perf_event_pmu_context *pmu_ctx, + bool sched_in) +{ + struct riscv_pmu *rvpmu =3D to_riscv_pmu(pmu_ctx->pmu); + struct cpu_hw_events *cpuc =3D this_cpu_ptr(rvpmu->hw_events); + + if (cpuc->ctr_users && sched_in) + riscv_pmu_ctr_clear(); +} + +void riscv_pmu_ctr_enable(struct perf_event *event) +{ + struct riscv_pmu *rvpmu =3D to_riscv_pmu(event->pmu); + struct cpu_hw_events *cpuc =3D this_cpu_ptr(rvpmu->hw_events); + u64 branch_type =3D event->attr.branch_sample_type; + u64 ctr; + + if (!cpuc->ctr_users++ && !event->total_time_running) + riscv_pmu_ctr_clear(); + + ctr =3D branch_type_to_ctr(branch_type); + csr_write(CSR_CTRCTL, ctr); + + perf_sched_cb_inc(event->pmu); +} + +void riscv_pmu_ctr_disable(struct perf_event *event) +{ + struct riscv_pmu *rvpmu =3D to_riscv_pmu(event->pmu); + struct cpu_hw_events *cpuc =3D this_cpu_ptr(rvpmu->hw_events); + + /* Clear CTRCTL to disable the recording. */ + csr_write(CSR_CTRCTL, 0); + + cpuc->ctr_users--; + WARN_ON_ONCE(cpuc->ctr_users < 0); + + perf_sched_cb_dec(event->pmu); +} + +/* + * Check for hardware supported perf filters here. To avoid missing + * any new added filter in perf, we do a BUILD_BUG_ON check, so make sure + * to update CTR_ALLOWED_BRANCH_FILTERS or CTR_EXCLUDE_BRANCH_FILTERS + * defines when adding support for it in below function. + */ +static void __init check_available_filters(void) +{ + u64 ctr_ctl; + + /* + * Ensure both perf branch filter allowed and exclude + * masks are always in sync with the generic perf ABI. + */ + BUILD_BUG_ON(CTR_PERF_BRANCH_FILTERS !=3D (PERF_SAMPLE_BRANCH_MAX - 1)); + + allowed_filters =3D PERF_SAMPLE_BRANCH_USER | + PERF_SAMPLE_BRANCH_KERNEL | + PERF_SAMPLE_BRANCH_ANY | + PERF_SAMPLE_BRANCH_HW_INDEX | + PERF_SAMPLE_BRANCH_NO_FLAGS | + PERF_SAMPLE_BRANCH_NO_CYCLES | + PERF_SAMPLE_BRANCH_TYPE_SAVE; + + csr_write(CSR_CTRCTL, ~0); + ctr_ctl =3D csr_read(CSR_CTRCTL); + + if (riscv_isa_extension_available(NULL, h)) + allowed_filters |=3D PERF_SAMPLE_BRANCH_HV; + + if (ctr_ctl & (CTRCTL_INDCALL_INH | CTRCTL_DIRCALL_INH)) + allowed_filters |=3D PERF_SAMPLE_BRANCH_ANY_CALL; + + if (ctr_ctl & (CTRCTL_RET_INH | CTRCTL_TRETINH)) + allowed_filters |=3D PERF_SAMPLE_BRANCH_ANY_RETURN; + + if (ctr_ctl & CTRCTL_INDCALL_INH) + allowed_filters |=3D PERF_SAMPLE_BRANCH_IND_CALL; + + if (ctr_ctl & CTRCTL_TKBRINH) + allowed_filters |=3D PERF_SAMPLE_BRANCH_COND; + + if (ctr_ctl & CTRCTL_RASEMU) + allowed_filters |=3D PERF_SAMPLE_BRANCH_CALL_STACK; + + if (ctr_ctl & (CTRCTL_INDOJUMP_INH | CTRCTL_INDJUMP_INH)) + allowed_filters |=3D PERF_SAMPLE_BRANCH_IND_JUMP; + + if (ctr_ctl & CTRCTL_DIRCALL_INH) + allowed_filters |=3D PERF_SAMPLE_BRANCH_CALL; +} + +void riscv_pmu_ctr_starting_cpu(void) +{ + if (!riscv_isa_extension_available(NULL, SxCTR) || + !riscv_isa_extension_available(NULL, SSCOFPMF) || + !riscv_isa_extension_available(NULL, SxCSRIND)) + return; + + /* Set depth to maximum. */ + csr_write(CSR_SCTRDEPTH, SCTRDEPTH_MASK); +} + +void riscv_pmu_ctr_dying_cpu(void) +{ + if (!riscv_isa_extension_available(NULL, SxCTR) || + !riscv_isa_extension_available(NULL, SSCOFPMF) || + !riscv_isa_extension_available(NULL, SxCSRIND)) + return; + + /* Clear and reset CTR CSRs. */ + csr_write(CSR_SCTRDEPTH, 0); + csr_write(CSR_CTRCTL, 0); + riscv_pmu_ctr_clear(); +} + +void __init riscv_pmu_ctr_init(struct riscv_pmu *riscv_pmu) +{ + if (!riscv_isa_extension_available(NULL, SxCTR) || + !riscv_isa_extension_available(NULL, SSCOFPMF) || + !riscv_isa_extension_available(NULL, SxCSRIND)) + return; + + check_available_filters(); + + /* Set depth to maximum. */ + csr_write(CSR_SCTRDEPTH, SCTRDEPTH_MASK); + riscv_pmu->ctr_depth =3D ctr_get_depth(csr_read(CSR_SCTRDEPTH)); + + pr_info("Perf CTR available, with %d depth\n", riscv_pmu->ctr_depth); +} + +void __init riscv_pmu_ctr_finish(struct riscv_pmu *riscv_pmu) +{ + if (!riscv_pmu_ctr_supported(riscv_pmu)) + return; + + csr_write(CSR_SCTRDEPTH, 0); + csr_write(CSR_CTRCTL, 0); + riscv_pmu_ctr_clear(); + riscv_pmu->ctr_depth =3D 0; +} diff --git a/include/linux/perf/riscv_pmu.h b/include/linux/perf/riscv_pmu.h index 5a6b840018bd..455d2386936f 100644 --- a/include/linux/perf/riscv_pmu.h +++ b/include/linux/perf/riscv_pmu.h @@ -104,6 +104,39 @@ struct riscv_pmu *riscv_pmu_alloc(void); int riscv_pmu_get_hpm_info(u32 *hw_ctr_width, u32 *num_hw_ctr); #endif =20 +static inline bool riscv_pmu_ctr_supported(struct riscv_pmu *pmu) +{ + return !!pmu->ctr_depth; +} + #endif /* CONFIG_RISCV_PMU_COMMON */ =20 +#ifdef CONFIG_RISCV_CTR + +bool riscv_pmu_ctr_valid(struct perf_event *event); +void riscv_pmu_ctr_consume(struct cpu_hw_events *cpuc, struct perf_event *= event); +void riscv_pmu_ctr_sched_task(struct perf_event_pmu_context *pmu_ctx, bool= sched_in); +void riscv_pmu_ctr_enable(struct perf_event *event); +void riscv_pmu_ctr_disable(struct perf_event *event); +void riscv_pmu_ctr_dying_cpu(void); +void riscv_pmu_ctr_starting_cpu(void); +void riscv_pmu_ctr_init(struct riscv_pmu *riscv_pmu); +void riscv_pmu_ctr_finish(struct riscv_pmu *riscv_pmu); + +#else + +static inline bool riscv_pmu_ctr_valid(struct perf_event *event) { return = false; } +static inline void riscv_pmu_ctr_consume(struct cpu_hw_events *cpuc, + struct perf_event *event) { } +static inline void riscv_pmu_ctr_sched_task(struct perf_event_pmu_context = *, + bool sched_in) { } +static inline void riscv_pmu_ctr_enable(struct perf_event *event) { } +static inline void riscv_pmu_ctr_disable(struct perf_event *event) { } +static inline void riscv_pmu_ctr_dying_cpu(void) { } +static inline void riscv_pmu_ctr_starting_cpu(void) { } +static inline void riscv_pmu_ctr_init(struct riscv_pmu *riscv_pmu) { } +static inline void riscv_pmu_ctr_finish(struct riscv_pmu *riscv_pmu) { } + +#endif /* CONFIG_RISCV_CTR */ + #endif /* _RISCV_PMU_H */ --=20 2.34.1 From nobody Tue Dec 16 22:17:05 2025 Received: from mail-wm1-f48.google.com (mail-wm1-f48.google.com [209.85.128.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 30FE71C230E for ; Wed, 29 May 2024 18:54:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.48 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717008879; cv=none; b=U9mt14C5YapfZADHbrTVLOlteMh1rMB+jm/emi7aXbXU5noYzszLKXjAdz8DsliQjlWgLDtnVTE2TjoH+CY0i+CJoHO1MSvHAX0v8o0rzOjk/H3T1jwsGPduWiQnPVac/1H2LUTotOm4h+pHdmoaQZG8bm1kuWDwieCpjWvN8fM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717008879; c=relaxed/simple; bh=fhyOW32KF7x1nOg5NvGrhHnYTYAqHGUN810iLS+POK0=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=Br1y4J8a9PwvpXiS5ptflKi3p1ihSkYEGkbc3c1DxD3WyfKpwdXR1Y76b4CIybQyeCQQbECeGsyBPMUG5bNYWvTYqsDag8QAZ94lImjEZH8UcmSEkMwZRT+f5nKG9/hKAgntc5YpgQHAwNeLP4kKsV/HVShVo3bhoEbSOZDAD9g= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com; spf=pass smtp.mailfrom=rivosinc.com; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b=Q8dA1usY; arc=none smtp.client-ip=209.85.128.48 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=rivosinc.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=rivosinc-com.20230601.gappssmtp.com header.i=@rivosinc-com.20230601.gappssmtp.com header.b="Q8dA1usY" Received: by mail-wm1-f48.google.com with SMTP id 5b1f17b1804b1-42120fc8cbfso358265e9.2 for ; Wed, 29 May 2024 11:54:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1717008876; x=1717613676; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=U+2U8h+2rvtM/KDIv8ZmxGGBqHz1DJ98r4cPzpRskE8=; b=Q8dA1usY15b8WQ+g36JDmCaqZrRlLcNcwX+fkhSDvbQ4jMTMiZOfZZTYuLpRGTm3kE 8p2jfSzp3bm5nk6yzXQyB/zk8gbTi8M8c3x/oE8Lmm6+Qhws8kd5+qSKkVR56WWDaF7a D751SZznnFXI21jvW5Or8VJ+bgmCD2JaO2y+aSnIvznWM3RINVvkd7q+Mqmj9rmXXD9s rXj/GqrsjZAOG6HIuTcz0iCi5O8isXLQ1bHSjkTQ7RVfD66sKp+mYvnKwvBipPcw4Z61 aJobQE+UWSFfqkojekkPMO/nC1o+t/ZlkqjGPxOHuv72GwCUw8tqYyk73bVYF7EA8z1F h5DA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1717008876; x=1717613676; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=U+2U8h+2rvtM/KDIv8ZmxGGBqHz1DJ98r4cPzpRskE8=; b=jKAod/5WozILD/HtvSoIGdFOCXd/HQj/qg/pTO39Rf68xmt/yItzKOYYK4YbXWK8yr jXzrWbGm1skJqzvDv7XOIq+vXpe8oGevsQtbejGqDgWYZYu0YsxG9jogGrmo0vY5yWK3 RqyBGWMbrB+CEztS7dsKU+QReYx55LJ8tX89cMhkGohKzF+DMiDOpe126HJMf+xelv9Q L8uNFDPszndjNlgr9tQkLx/DXlGjPGQESr+GlI4dKx5RKcH8+c00H8eh04KMlev0msQH iUilTernFJmRSTfNSkopd/qwuZCh1cTHV+qyLtTDzqHxDS9L0iX/krHDt3Y05Fza6pxn D8Zw== X-Gm-Message-State: AOJu0YyLLUuRxB+Fz1ETdmvKGShlFxOOqPIkSPy+5Hr10Ieepmteosuz BWa15gfZ7bZgYA1Tlvido3KNgndzV/Z7DcNmzUAGf/VbQjgFCYZsTQ1437QVddGqrbdxDBJiFks b X-Google-Smtp-Source: AGHT+IHyilA5pn7bTk5wG0if56ayFdi//JSBq/3E0JiJr67lveFf/JA8ZYRTkFP5SiaOqO8femclbQ== X-Received: by 2002:a7b:c391:0:b0:420:1db0:53c1 with SMTP id 5b1f17b1804b1-421279375bamr150315e9.41.1717008876402; Wed, 29 May 2024 11:54:36 -0700 (PDT) Received: from rkanwal-XPS-15-9520.Home ([2a02:c7c:7527:ee00:7446:71c1:a41a:da9b]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-4212706a23csm2787885e9.27.2024.05.29.11.54.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 29 May 2024 11:54:35 -0700 (PDT) From: Rajnesh Kanwal To: linux-kernel@vger.kernel.org Cc: linux-perf-users@vger.kernel.org, linux-riscv@lists.infradead.org, adrian.hunter@intel.com, alexander.shishkin@linux.intel.com, ajones@ventanamicro.com, anup@brainfault.org, acme@kernel.org, atishp@rivosinc.com, beeman@rivosinc.com, brauner@kernel.org, conor@kernel.org, heiko@sntech.de, irogers@google.com, mingo@redhat.com, james.clark@arm.com, renyu.zj@linux.alibaba.com, jolsa@kernel.org, jisheng.teoh@starfivetech.com, palmer@dabbelt.com, tech-control-transfer-records@lists.riscv.org, will@kernel.org, kaiwenxue1@gmail.com, Rajnesh Kanwal Subject: [PATCH RFC 6/6] riscv: perf: Integrate CTR Ext support in riscv_pmu_dev driver Date: Wed, 29 May 2024 19:53:37 +0100 Message-Id: <20240529185337.182722-7-rkanwal@rivosinc.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240529185337.182722-1-rkanwal@rivosinc.com> References: <20240529185337.182722-1-rkanwal@rivosinc.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" This integrates recently added CTR ext support in riscv_pmu_dev driver to enable branch stack sampling using PMU events. This mainly adds CTR enable/disable callbacks in rvpmu_ctr_stop() and rvpmu_ctr_start() function to start/stop branch recording along with the event. PMU overflow handler rvpmu_ovf_handler() is also updated to sample CTR entries in case of the overflow for the particular event programmed to records branches. The recorded entries are fed to core perf for further processing. Signed-off-by: Rajnesh Kanwal --- drivers/perf/riscv_pmu_common.c | 3 +- drivers/perf/riscv_pmu_dev.c | 77 +++++++++++++++++++++++++++------ 2 files changed, 65 insertions(+), 15 deletions(-) diff --git a/drivers/perf/riscv_pmu_common.c b/drivers/perf/riscv_pmu_commo= n.c index e794675e4944..e1f3a33b479f 100644 --- a/drivers/perf/riscv_pmu_common.c +++ b/drivers/perf/riscv_pmu_common.c @@ -326,8 +326,7 @@ static int riscv_pmu_event_init(struct perf_event *even= t) u64 event_config =3D 0; uint64_t cmask; =20 - /* driver does not support branch stack sampling */ - if (has_branch_stack(event)) + if (has_branch_stack(event) && !riscv_pmu_ctr_supported(rvpmu)) return -EOPNOTSUPP; =20 hwc->flags =3D 0; diff --git a/drivers/perf/riscv_pmu_dev.c b/drivers/perf/riscv_pmu_dev.c index 40ae5fc897a3..1b2c04c35bed 100644 --- a/drivers/perf/riscv_pmu_dev.c +++ b/drivers/perf/riscv_pmu_dev.c @@ -675,7 +675,7 @@ static void pmu_sched_task(struct perf_event_pmu_contex= t *pmu_ctx, { struct riscv_pmu *pmu =3D to_riscv_pmu(pmu_ctx->pmu); =20 - /* Call CTR specific Sched hook. */ + riscv_pmu_ctr_sched_task(pmu_ctx, sched_in); } =20 static int rvpmu_sbi_find_num_ctrs(void) @@ -935,17 +935,25 @@ static irqreturn_t rvpmu_ovf_handler(int irq, void *d= ev) hw_evt =3D &event->hw; riscv_pmu_event_update(event); perf_sample_data_init(&data, 0, hw_evt->last_period); - if (riscv_pmu_event_set_period(event)) { - /* - * Unlike other ISAs, RISC-V don't have to disable interrupts - * to avoid throttling here. As per the specification, the - * interrupt remains disabled until the OF bit is set. - * Interrupts are enabled again only during the start. - * TODO: We will need to stop the guest counters once - * virtualization support is added. - */ - perf_event_overflow(event, &data, regs); + if (!riscv_pmu_event_set_period(event)) + continue; + + if (needs_branch_stack(event)) { + riscv_pmu_ctr_consume(cpu_hw_evt, event); + perf_sample_save_brstack( + &data, event, + &cpu_hw_evt->branches->branch_stack, NULL); } + + /* + * Unlike other ISAs, RISC-V don't have to disable interrupts + * to avoid throttling here. As per the specification, the + * interrupt remains disabled until the OF bit is set. + * Interrupts are enabled again only during the start. + * TODO: We will need to stop the guest counters once + * virtualization support is added. + */ + perf_event_overflow(event, &data, regs); } =20 rvpmu_start_overflow_mask(pmu, overflowed_ctrs); @@ -1103,10 +1111,12 @@ static void rvpmu_ctr_start(struct perf_event *even= t, u64 ival) else rvpmu_sbi_ctr_start(event, ival); =20 - if ((hwc->flags & PERF_EVENT_FLAG_USER_ACCESS) && (hwc->flags & PERF_EVENT_FLAG_USER_READ_CNT)) rvpmu_set_scounteren((void *)event); + + if (needs_branch_stack(event)) + riscv_pmu_ctr_enable(event); } =20 static void rvpmu_ctr_stop(struct perf_event *event, unsigned long flag) @@ -1128,6 +1138,9 @@ static void rvpmu_ctr_stop(struct perf_event *event, = unsigned long flag) } else { rvpmu_sbi_ctr_stop(event, flag); } + + if (needs_branch_stack(event)) + riscv_pmu_ctr_disable(event); } =20 static int rvpmu_find_ctrs(void) @@ -1161,6 +1174,9 @@ static int rvpmu_find_ctrs(void) =20 static int rvpmu_event_map(struct perf_event *event, u64 *econfig) { + if (needs_branch_stack(event) && !riscv_pmu_ctr_valid(event)) + return -EOPNOTSUPP; + if (static_branch_likely(&riscv_pmu_cdeleg_available) && !pmu_sbi_is_fw_e= vent(event)) return rvpmu_deleg_event_map(event, econfig); else @@ -1207,6 +1223,8 @@ static int rvpmu_starting_cpu(unsigned int cpu, struc= t hlist_node *node) enable_percpu_irq(riscv_pmu_irq, IRQ_TYPE_NONE); } =20 + riscv_pmu_ctr_starting_cpu(); + return 0; } =20 @@ -1218,6 +1236,7 @@ static int rvpmu_dying_cpu(unsigned int cpu, struct h= list_node *node) =20 /* Disable all counters access for user mode now */ csr_write(CSR_SCOUNTEREN, 0x0); + riscv_pmu_ctr_dying_cpu(); =20 return 0; } @@ -1331,6 +1350,29 @@ static void riscv_pmu_destroy(struct riscv_pmu *pmu) cpuhp_state_remove_instance(CPUHP_AP_PERF_RISCV_STARTING, &pmu->node); } =20 +static int branch_records_alloc(struct riscv_pmu *pmu) +{ + struct branch_records __percpu *tmp_alloc_ptr; + struct branch_records *records; + struct cpu_hw_events *events; + int cpu; + + if (!riscv_pmu_ctr_supported(pmu)) + return 0; + + tmp_alloc_ptr =3D alloc_percpu_gfp(struct branch_records, GFP_KERNEL); + if (!tmp_alloc_ptr) + return -ENOMEM; + + for_each_possible_cpu(cpu) { + events =3D per_cpu_ptr(pmu->hw_events, cpu); + records =3D per_cpu_ptr(tmp_alloc_ptr, cpu); + events->branches =3D records; + } + + return 0; +} + static void rvpmu_event_init(struct perf_event *event) { /* @@ -1490,6 +1532,12 @@ static int rvpmu_device_probe(struct platform_device= *pdev) pmu->pmu.attr_groups =3D riscv_cdeleg_pmu_attr_groups; else pmu->pmu.attr_groups =3D riscv_sbi_pmu_attr_groups; + + riscv_pmu_ctr_init(pmu); + ret =3D branch_records_alloc(pmu); + if (ret) + goto out_ctr_finish; + pmu->cmask =3D cmask; pmu->ctr_start =3D rvpmu_ctr_start; pmu->ctr_stop =3D rvpmu_ctr_stop; @@ -1506,7 +1554,7 @@ static int rvpmu_device_probe(struct platform_device = *pdev) =20 ret =3D cpuhp_state_add_instance(CPUHP_AP_PERF_RISCV_STARTING, &pmu->node= ); if (ret) - return ret; + goto out_ctr_finish; =20 ret =3D riscv_pm_pmu_register(pmu); if (ret) @@ -1523,6 +1571,9 @@ static int rvpmu_device_probe(struct platform_device = *pdev) out_unregister: riscv_pmu_destroy(pmu); =20 +out_ctr_finish: + riscv_pmu_ctr_finish(pmu); + out_free: kfree(pmu); return ret; --=20 2.34.1