From nobody Tue Sep 29 02:33:43 2026 Received: from mail-pg1-f199.google.com (mail-pg1-f199.google.com [209.85.215.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2D6BA38B131 for ; Thu, 13 Aug 2026 05:24:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.199 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786598678; cv=none; b=ZvwfKGI+OM5Bn0HT59Vl2rXDr7pQ+8RMYMCJZ29nwT0/jM0wBmyKBtwHRKuiSUOGzunQiJMj8oI+C75pCkgM3Exg41vkhVU8lM9NKF0Yz/SVaMewAAxh9QKL0kexBKIzg9vVHaCGJ6fFUz4epqXSgrzmjgd5E+l/WWTCzrQjJKE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786598678; c=relaxed/simple; bh=I+QbmftyQ3/m5lci7+ClT0Xvzu0nyW5ktrtiFWBA31U=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=iJtlL05qPaEpioGyeUQhqWDpJQmYW2Z6a2fubsZuOMrqeSIJY32gLIuBdaXo7J9cHkCDESH2NbzIh8OUCqjY13MmHzzFrik+o+CjNXDpAjbAQMP8JRF96TGiiioW+AwNSl/E+65n/AO80RrIoFDh8rz0yh2zfjlyoagQHK878HA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=ZW86xS/o; arc=none smtp.client-ip=209.85.215.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="ZW86xS/o" Received: by mail-pg1-f199.google.com with SMTP id 41be03b00d2f7-cbe77d6864dso2677986a12.2 for ; Wed, 12 Aug 2026 22:24:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1786598664; x=1787203464; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=/IZ5ILsrdyS+CjgceIxUMriYsJo8p9ZfwvLEd5ItbUU=; b=ZW86xS/o2sf23sjti+JW0ADq34DqZJ3r3Che2kMmCD74MgVgJ/kLJBEGNw3UQeUacR Wh9396OfDLlJvHxhIntxaLrYuzSj2NQ1WaSYSakh8kleC9gNeEZuwCr+2xWOh+01fTPI kQwkt7TgTDdB7jWljL1w4jigYcKrzXafVSvVkGG41zPZaruZWEkxZPpX+HO47U4OOQHF s7//81Ixm0DBICJXYRROpGFl/4y6y1vLuqGE//ny5E1rVuDV+nIebxZUJqEWVq53ZCmB 4RIJfW77Oy8yvuFGXL+k5AnuyzsLtNEE7Ca3cv0jDhnR4lK443t7ttsX1c88lcBxrJqW Y9fQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786598664; x=1787203464; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=/IZ5ILsrdyS+CjgceIxUMriYsJo8p9ZfwvLEd5ItbUU=; b=st9jcS/gifVfZdxYw73tl9anHBJGJvy15f6IRvlbmvL6ZFRBna/JeyGDzTocssdqZQ 1vjDGqDbsHnRPP5OS7iNUbyZViM4hbf4RWU4OyEyNaNjBxV6+4Cr8Og84xqUYMWEkg19 sbfwrP6n68tmt+zK/p3RiIQujMgv7BPRxEvi87Okny1mmemUfLGL4MU7Va71Hv5l6dCr JhlOIPgL3qluRTajQWYDaJLTsKKnP0IEWDgLm0nh/b+17ib6dBy+Aw27pnxHTymNL76F 3LIdKkmOWJzIToCDm9ziTY8wC1Lo2b/iP3jfuyoaHbBrRpYOdd3XH4trXSezWZq8pu2e raiw== X-Forwarded-Encrypted: i=1; AHgh+RrCQzwCUoD58WCc/ECWnoqbLVEE0j0yg+W+0GHuUI0jQDHOCse3JJUxBxIK5HlpDuRBD6tGMZ3RMkk+lC0=@vger.kernel.org X-Gm-Message-State: AOJu0YzSyXyQN3xwFEQdRUbR7O1WKVqTYmNLU58xEavM4M6aVihSiyMS 7ZZQcyRX6JzTme8ID7pYuYuxUaADzAyRcyjBB6z4HtiwJlA4vG+TIyzQZ9ipLLGqJi2N6Gxi+Lj Fgg30yw+GOw== X-Received: from dyz7.prod.google.com ([2002:a05:693c:4087:b0:30f:4402:de4f]) (user=irogers job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a20:2e81:b0:3cc:55ed:dbfa with SMTP id adf61e73a8af0-3cc55edf3d8mr3000593637.27.1786598663981; Wed, 12 Aug 2026 22:24:23 -0700 (PDT) Date: Wed, 12 Aug 2026 22:24:17 -0700 In-Reply-To: <20260813052419.2770841-1-irogers@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260813043533.2744442-1-irogers@google.com> <20260813052419.2770841-1-irogers@google.com> X-Mailer: git-send-email 2.55.0.691.gc56d675ccc-goog Message-ID: <20260813052419.2770841-2-irogers@google.com> Subject: [PATCH v7 1/2] perf: Add Raspberry Pi BCM2835 AXI PMU driver From: Ian Rogers To: irogers@google.com, linux-perf-users@vger.kernel.org, linux-rpi-kernel@lists.infradead.org Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, mark.rutland@arm.com, u.kleine-koenig@baylibre.com, will@kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Signed-off-by: Ian Rogers --- drivers/perf/rpi_axi_pmu.c | 1993 ++++++++++++++++++++++++++++++++++++ 1 file changed, 1993 insertions(+) create mode 100644 drivers/perf/rpi_axi_pmu.c diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c new file mode 100644 index 000000000000..bd3c79dd2ff8 --- /dev/null +++ b/drivers/perf/rpi_axi_pmu.c @@ -0,0 +1,1993 @@ +// SPDX-License-Identifier: GPL-2.0-only + +/** + * DOC: Raspberry Pi AXI Bus Performance Monitoring Unit (PMU) Driver + * + * This driver exposes the performance monitoring hardware on Raspberry Pi + * System-on-Chips to the Linux perf subsystem: + * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2= 835/2836/2837/2711). + * + * Architecture Overview: + * ---------------------- + * The Broadcom AXI performance hardware provides up to two independent mo= nitors: + * 1. System Monitor (MON__SYSTEM =3D 0): + * Monitors system-level AXI traffic (ARM CPU L2/UC, DMA, V3D, ISP, HVS= , PCIe/RP1). + * Directly memory-mapped via ARM physical IO memory space (MMIO). + * Read latency: ~10-20 nanoseconds (fast, atomic-safe, non-blocking). + * + * 2. VPU Monitor (MON__VPU =3D 1): + * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, = etc.). + * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/= GET_PERIPH_REG). + * Read latency: ~10-100 microseconds (IPC over VPU mailbox). + * + * Synchronization & Concurrency Model: + * ------------------------------------ + * - Spinlock (pmu->lock): + * Protects active event array (events[]), event generation sequence cou= nters (event_gen[]), + * bus watcher allocation/refcounting, active_vpu_events counter, and MM= IO register updates + * (MON__SYSTEM) against SMP race conditions and ABA pointer recycling r= aces. + * + * - Mutex (pmu->vpu_mutex): + * Serializes VideoCore Mailbox IPC transactions (MON__VPU) in process c= ontext, + * preventing concurrent mailbox buffer corruption across multiple CPUs. + * + * - Cached Async VPU Reads & Multiplexing (MON__VPU): + * Polled periodically in process context by vpu_work when active_vpu_ev= ents > 0. + * Uses PERF_HES_UPTODATE state flag to safely establish counter baselin= es during + * event rotation / multiplexing. User read() syscalls return cached cum= ulative event counter + * instantly without blocking. + */ + +#include +#include +#include +#include +#include + +#if LINUX_VERSION_CODE < KERNEL_VERSION(6, 13, 0) +static inline void rpi_hrtimer_setup(struct hrtimer *timer, + enum hrtimer_restart (*function)(struct hrtimer *), + clockid_t clock_id, enum hrtimer_mode mode) +{ + hrtimer_init(timer, clock_id, mode); + timer->function =3D function; +} +#define hrtimer_setup rpi_hrtimer_setup +#endif +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +/* --- PLATFORM CONSTANTS & ENUMERATIONS --------------------------- */ + +/** + * enum rpi_axi_chip - Supported Broadcom SoC generations + * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, = Zero/W) + */ +enum rpi_axi_chip { + CHIP_BCM2835 =3D 0, +}; + +enum monitor { + MON__SYSTEM =3D 0, + MON__VPU, + MON__MAX +}; + +/* Number of hardware bus watcher units per monitor */ +#define NUM_BUS_WATCHERS_PER_MONITOR 3 + +/** + * enum bcm2835_system_bus - AXI buses monitored by System Monitor on BCM2= 835-BCM2711 (RPi 1-4) + * @BCM2835_SB__DMA_L2: DMA engine L2 cache interconnect bus + * @BCM2835_SB__TRANS: Transposer engine bus (image format rotation & 2D m= atrix conversion) + * @BCM2835_SB__JPEG: Hardware JPEG codec acceleration bus + * @BCM2835_SB__SYSTEM_UC: System Uncached memory bus + * @BCM2835_SB__DMA_UC: DMA Uncached memory bus + * @BCM2835_SB__SYSTEM_L2: System main L2 cache bus + * @BCM2835_SB__CCP2TX: Compact Camera Port 2 (CCP2) transmitter bus + * @BCM2835_SB__MPHI_RX: Message Passing Host Interface (MPHI) receive bus + * @BCM2835_SB__MPHI_TX: Message Passing Host Interface (MPHI) transmit bus + * @BCM2835_SB__HVS: Hardware Video Scaler (HVS) multi-layer display compo= sition engine bus + * @BCM2835_SB__H264: H.264 / AVC hardware video encoder/decoder bus + * @BCM2835_SB__ISP: Image Sensor Processor (ISP) camera processing pipeli= ne bus + * @BCM2835_SB__V3D: VideoCore V3D 3D graphics hardware pipeline bus + * @BCM2835_SB__PERIPHERAL: System peripherals bus (UART, SPI, I2C, GPIO, = PWM, PCM) + * @BCM2835_SB__CPU_UC: ARM CPU Uncached memory bus + * @BCM2835_SB__CPU_L2: ARM CPU L2 cache bus + * @BCM2835_SB__MAX: Total count of monitored system buses on BCM2835-BCM2= 711 + */ +enum bcm2835_system_bus { + BCM2835_SB__DMA_L2 =3D 0, + BCM2835_SB__TRANS, + BCM2835_SB__JPEG, + BCM2835_SB__SYSTEM_UC, + BCM2835_SB__DMA_UC, + BCM2835_SB__SYSTEM_L2, + BCM2835_SB__CCP2TX, + BCM2835_SB__MPHI_RX, + BCM2835_SB__MPHI_TX, + BCM2835_SB__HVS, + BCM2835_SB__H264, + BCM2835_SB__ISP, + BCM2835_SB__V3D, + BCM2835_SB__PERIPHERAL, + BCM2835_SB__CPU_UC, + BCM2835_SB__CPU_L2, + BCM2835_SB__MAX +}; + + +/** + * enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (R= Pi 1-4) + * @VPU__VPU1_D_L2: VideoCore VPU Core 1 Data L2 cache bus + * @VPU__VPU0_D_L2: VideoCore VPU Core 0 Data L2 cache bus + * @VPU__VPU1_I_L2: VideoCore VPU Core 1 Instruction L2 cache bus + * @VPU__VPU0_I_L2: VideoCore VPU Core 0 Instruction L2 cache bus + * @VPU__SYSTEM_L2: VPU System L2 cache interconnect bus + * @VPU__L2_FLUSH: VPU L2 cache flush controller bus + * @VPU__DMA_L2: VPU DMA L2 cache interconnect bus + * @VPU__VPU1_D_UC: VideoCore VPU Core 1 Data Uncached memory bus + * @VPU__VPU0_D_UC: VideoCore VPU Core 0 Data Uncached memory bus + * @VPU__VPU1_I_UC: VideoCore VPU Core 1 Instruction Uncached memory bus + * @VPU__VPU0_I_UC: VideoCore VPU Core 0 Instruction Uncached memory bus + * @VPU__SYSTEM_UC: VPU System Uncached memory bus + * @VPU__L2_OUT: VPU L2 cache outbound memory bus + * @VPU__DMA_UC: VPU DMA Uncached memory bus + * @VPU__SDRAM: VPU SDRAM memory controller bus + * @VPU__L2_IN: VPU L2 cache inbound memory bus + * @VPU__MAX: Total count of monitored VPU buses + */ +enum vpu_bus { + VPU__VPU1_D_L2 =3D 0, + VPU__VPU0_D_L2, + VPU__VPU1_I_L2, + VPU__VPU0_I_L2, + VPU__SYSTEM_L2, + VPU__L2_FLUSH, + VPU__DMA_L2, + VPU__VPU1_D_UC, + VPU__VPU0_D_UC, + VPU__VPU1_I_UC, + VPU__VPU0_I_UC, + VPU__SYSTEM_UC, + VPU__L2_OUT, + VPU__DMA_UC, + VPU__SDRAM, + VPU__L2_IN, + VPU__MAX +}; + +/** + * enum counter - 32-bit hardware performance counter metrics per bus watc= her unit + * @CNT__ATRANS: Total address phase transaction count + * @CNT__ATWAIT: Total address phase wait / stall cycles + * @CNT__WTRANS: Total write data phase transaction count + * @CNT__WTWAIT: Total write data phase wait / stall cycles + * @CNT__RTRANS: Total read data phase transaction count + * @CNT__RTWAIT: Total read data phase wait / stall cycles + * @CNT__MAX: Total metric counters per watcher unit + */ +enum counter { + CNT__ATRANS =3D 0, + CNT__ATWAIT, + CNT__WTRANS, + CNT__WTWAIT, + CNT__RTRANS, + CNT__RTWAIT, + CNT__MAX +}; + +/** + * enum bcm2835_filter - AXI master ID filter options for BCM2835-BCM2711 = (RPi 1-4) + * @BCM2835_FLT__NONE: Disable master ID filtering (monitor all traffic on= bus) + * @BCM2835_FLT__CORE0_V: VideoCore Core 0 master ID + * @BCM2835_FLT__ICACHE0: CPU Core 0 Instruction Cache master ID + * @BCM2835_FLT__DCACHE0: CPU Core 0 Data Cache master ID + * @BCM2835_FLT__CORE1_V: VideoCore Core 1 master ID + * @BCM2835_FLT__ICACHE1: CPU Core 1 Instruction Cache master ID + * @BCM2835_FLT__DCACHE1: CPU Core 1 Data Cache master ID + * @BCM2835_FLT__L2_MAIN: Main L2 cache controller master ID + * @BCM2835_FLT__HOST_PORT: Host Interface Port 0 master ID + * @BCM2835_FLT__HOST_PORT2: Host Interface Port 1 master ID + * @BCM2835_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID + * @BCM2835_FLT__ISP: Image Sensor Processor (ISP) camera pipeline master = ID + * @BCM2835_FLT__VIDEO_DCT: Discrete Cosine Transform (DCT) hardware accel= erator master ID + * @BCM2835_FLT__VIDEO_SD2AXI: SD card to AXI bridge master ID + * @BCM2835_FLT__CAM0: Camera Unicam 0 receiver master ID + * @BCM2835_FLT__CAM1: Camera Unicam 1 receiver master ID + * @BCM2835_FLT__DMA0: System DMA Channel 0 master ID + * @BCM2835_FLT__DMA1: System DMA Channel 1 master ID + * @BCM2835_FLT__DMA2_VPU: VPU DMA engine master ID + * @BCM2835_FLT__JPEG: JPEG decoder hardware master ID + * @BCM2835_FLT__VIDEO_CME: Motion Estimation hardware accelerator master = ID + * @BCM2835_FLT__TRANSPOSER: Image Transposer engine master ID + * @BCM2835_FLT__VIDEO_FME: Fractional Motion Estimation hardware master ID + * @BCM2835_FLT__CCP2TX: Compact Camera Port 2 transmitter master ID + * @BCM2835_FLT__USB: USB 2.0 Host/OTG controller master ID + * @BCM2835_FLT__V3D0: VideoCore V3D graphics pipe 0 master ID + * @BCM2835_FLT__V3D1: VideoCore V3D graphics pipe 1 master ID + * @BCM2835_FLT__V3D2: VideoCore V3D graphics pipe 2 master ID + * @BCM2835_FLT__AVE: Audio-Video Engine master ID + * @BCM2835_FLT__DEBUG: ARM JTAG/CoreSight debug unit master ID + * @BCM2835_FLT__CPU: Generic ARM CPU cluster master ID + * @BCM2835_FLT__M30: Reserved master ID slot 30 + * @BCM2835_FLT__MAX: Maximum filter ID count + */ +enum bcm2835_filter { + BCM2835_FLT__NONE =3D 0, + BCM2835_FLT__CORE0_V, + BCM2835_FLT__ICACHE0, + BCM2835_FLT__DCACHE0, + BCM2835_FLT__CORE1_V, + BCM2835_FLT__ICACHE1, + BCM2835_FLT__DCACHE1, + BCM2835_FLT__L2_MAIN, + BCM2835_FLT__HOST_PORT, + BCM2835_FLT__HOST_PORT2, + BCM2835_FLT__HVS, + BCM2835_FLT__ISP, + BCM2835_FLT__VIDEO_DCT, + BCM2835_FLT__VIDEO_SD2AXI, + BCM2835_FLT__CAM0, + BCM2835_FLT__CAM1, + BCM2835_FLT__DMA0, + BCM2835_FLT__DMA1, + BCM2835_FLT__DMA2_VPU, + BCM2835_FLT__JPEG, + BCM2835_FLT__VIDEO_CME, + BCM2835_FLT__TRANSPOSER, + BCM2835_FLT__VIDEO_FME, + BCM2835_FLT__CCP2TX, + BCM2835_FLT__USB, + BCM2835_FLT__V3D0, + BCM2835_FLT__V3D1, + BCM2835_FLT__V3D2, + BCM2835_FLT__AVE, + BCM2835_FLT__DEBUG, + BCM2835_FLT__CPU, + BCM2835_FLT__M30, + BCM2835_FLT__MAX +}; + + +/* Hardware register offsets & control bitwise constants */ +#define GEN_CTRL 0x00 +#define GEN_CTL_ENABLE_BIT BIT(0) +#define GEN_CTL_RESET_BIT BIT(1) +#define GEN_CTL_WATCH_BIT BIT(2) + +#define BW_PITCH 0x40 +#define BW0_CTRL 0x40 +#define BW1_CTRL 0x80 +#define BW2_CTRL 0xc0 + +#define BW_ATRANS_OFFSET 0x04 +#define BW_ATWAIT_OFFSET 0x08 +#define BW_AMAX_OFFSET 0x0c +#define BW_WTRANS_OFFSET 0x10 +#define BW_WTWAIT_OFFSET 0x14 +#define BW_WMAX_OFFSET 0x18 +#define BW_RTRANS_OFFSET 0x1c +#define BW_RTWAIT_OFFSET 0x20 +#define BW_RMAX_OFFSET 0x24 + +#define BW_CTRL_RESET_BIT BIT(31) +#define BW_CTRL_ENABLE_BIT BIT(30) +#define BW_CTRL_ENABLE_ID_FILTER_BIT BIT(29) +#define BW_CTRL_LIMIT_HALT_BIT BIT(28) + +#define BW_CTRL_BUS_WATCH_SHIFT 0 +#define BW_CTRL_BUS_WATCH_MASK GENMASK(5, 0) +#define BW_CTRL_BUS_FILTER_SHIFT 8 +#define BW_CTRL_BUS_FILTER_MASK GENMASK(12, 8) + +#define RPI_AXI_PMU_TIMER_INTERVAL ms_to_ktime(2000) + +static enum cpuhp_state rpi_axi_pmu_cpuhp_state; + +/* --- PMU API & CONFIG DECODING ---------------------------------- */ + +#define PMU_NAME "rpi_axi_pmu" + +/** + * config_to_filter() - Extracts AXI filter ID from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Filter ID value (bits 10-14). + */ +static int config_to_filter(__u64 config) +{ + return (config >> 10) & 0x1F; +} + +/** + * config_to_monitor() - Extracts Monitor ID from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Monitor enum (bit 9: 0 =3D System, 1 =3D VPU). + */ +static enum monitor config_to_monitor(__u64 config) +{ + return (config >> 9) & 1; +} + +/** + * config_to_bus() - Extracts bus index from perf event config + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Bus index (bits 4-8). + */ +static int config_to_bus(__u64 config) +{ + return (config >> 4) & 0x1F; +} + +/** + * config_to_counter() - Extracts metric counter type from perf event conf= ig + * @config: 64-bit config value from struct perf_event_attr + * + * Return: Counter enum (bits 0-3). + */ +static enum counter config_to_counter(__u64 config) +{ + return config & 0xF; +} + +struct rpi_axi_pmu; + +/** + * config_is_valid() - Validates whether event config bitfields match SoC = capabilities + * @pmu: Pointer to rpi_axi_pmu driver context + * @config: 64-bit config value from struct perf_event_attr + * + * Return: true if valid for the detected chip, false otherwise. + */ +static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config); + +/** + * struct rpi_axi_hw_events - Hardware resource tracking per monitor + * @monitored_bus: Array tracking which bus index is assigned to each of t= he 3 watchers + * @filter: Array tracking the filter applied to each watcher + * @refcount: Reference count of active events sharing each watcher + * @num_monitored: Total count of active watchers in use + * @num_started: Total count of running events on this monitor + * @enabled: Hardware enabled status for each watcher + * @monitor_running: Flag indicating if the hardware monitor loop is globa= lly active + * @vpu_disable_pending: Array tracking asynchronous hardware disable requ= ests for VPU + */ +struct rpi_axi_hw_events { + int monitored_bus[NUM_BUS_WATCHERS_PER_MONITOR]; + int filter[NUM_BUS_WATCHERS_PER_MONITOR]; + int refcount[NUM_BUS_WATCHERS_PER_MONITOR]; + int num_monitored; + int num_started; +bool monitor_running; + bool enabled[NUM_BUS_WATCHERS_PER_MONITOR]; + bool vpu_disable_pending[NUM_BUS_WATCHERS_PER_MONITOR]; +}; + +/** + * rpi_axi_hw_events__init() - Resets hardware watcher tracking data + * @hw_events: Pointer to rpi_axi_hw_events structure + */ +static void rpi_axi_hw_events__init(struct rpi_axi_hw_events *hw_events) +{ + hw_events->num_monitored =3D 0; + hw_events->num_started =3D 0; + hw_events->monitor_running =3D false; + for (int i =3D 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) { + hw_events->monitored_bus[i] =3D -1; + hw_events->filter[i] =3D BCM2835_FLT__NONE; + hw_events->refcount[i] =3D 0; + hw_events->enabled[i] =3D false; + } +} + +/** + * rpi_axi_hw_events__get_alloc_event_idx() - Allocates or reuses a bus wa= tcher index + * @hw_events: Pointer to hardware watcher tracking state + * @event: Pointer to perf_event being initialized + * + * Return: Watcher index (0..2) on success, -1 if all 3 watchers are busy. + */ +static int rpi_axi_hw_events__get_alloc_event_idx(struct rpi_axi_hw_events= *hw_events, + const struct perf_event *event) +{ + int bus =3D config_to_bus(event->attr.config); + int filter =3D config_to_filter(event->attr.config); + + for (int i =3D 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) { + if (hw_events->monitored_bus[i] =3D=3D bus && hw_events->filter[i] =3D= =3D filter) { + hw_events->refcount[i]++; + return i; + } + } + if (hw_events->num_monitored =3D=3D NUM_BUS_WATCHERS_PER_MONITOR) + return -1; + + for (int i =3D 0; i < NUM_BUS_WATCHERS_PER_MONITOR; i++) { + if (hw_events->monitored_bus[i] =3D=3D -1) { + hw_events->monitored_bus[i] =3D bus; + hw_events->filter[i] =3D filter; + hw_events->refcount[i] =3D 1; + hw_events->vpu_disable_pending[i] =3D false; + hw_events->num_monitored++; + return i; + } + } + return -1; +} + +/* Maximum simultaneous active perf_events tracked by PMU */ +#define RPI_AXI_MAX_EVENTS 60 + +/** + * struct rpi_axi_pmu - Root PMU driver context + * @pmu: Core Linux perf PMU structure + * @pdev: Owning platform_device pointer + * @chip: Detected Broadcom SoC generation + * @firmware: Raspberry Pi firmware handle for VideoCore mailbox calls (BC= M2835-BCM2711) + * @cpu: CPU core assigned to process uncore PMU events + * @cpuhp_node: Dynamic CPU hotplug instance node + * @lock: Spinlock protecting events[] list, watcher refcounts, and MMIO c= ounter updates + * @vpu_mutex: Mutex serializing VideoCore Mailbox IPC transactions in pro= cess context + * @hrtimer: High-resolution timer for periodic 32-bit counter overflow po= lling + * @vpu_work: Deferred work structure for safe process-context VPU mailbox= reads + * @active_events: Count of active perf_events currently monitored + * @active_vpu_events: Count of active VPU perf_events currently monitored + * @events: Array of active perf_event pointers + * @event_gen: Per-slot generation sequence counters to prevent ABA pointe= r recycling races + * @monitor: Per-monitor state array (System and VPU) + */ +struct rpi_axi_pmu { + struct pmu pmu; + struct platform_device *pdev; + enum rpi_axi_chip chip; + struct rpi_firmware *firmware; + + int cpu; + struct hlist_node cpuhp_node; + + spinlock_t lock; + struct mutex vpu_mutex; + + struct hrtimer hrtimer; + struct work_struct vpu_work; + int active_events; + int active_vpu_events; + struct perf_event *events[RPI_AXI_MAX_EVENTS]; + u64 event_gen[RPI_AXI_MAX_EVENTS]; + + struct { + struct rpi_axi_hw_events hw_events; + bool use_mailbox_interface; + union { + u32 mailbox; + void __iomem *base_address; + }; + } monitor[MON__MAX]; +}; +#define pmu_to_rpi_axi_pmu(p) (container_of(p, struct rpi_axi_pmu, pmu)) + +static bool config_is_valid(struct rpi_axi_pmu *pmu, __u64 config) +{ + enum monitor mon =3D config_to_monitor(config); + int bus =3D config_to_bus(config); + int filter =3D config_to_filter(config); + int counter =3D config_to_counter(config); + + if (config >> 15 !=3D 0) + return false; + + if (mon >=3D MON__MAX) + return false; + + if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_a= ddress) + return false; + + if (mon =3D=3D MON__SYSTEM) { + if (bus >=3D BCM2835_SB__MAX) + return false; + } else { + if (bus >=3D VPU__MAX) + return false; + } + if (filter >=3D BCM2835_FLT__MAX) + return false; + return counter < CNT__MAX; +} + +PMU_FORMAT_ATTR(filter, "config:10-14"); +PMU_FORMAT_ATTR(monitor, "config:9"); +PMU_FORMAT_ATTR(bus, "config:4-8"); +PMU_FORMAT_ATTR(counter, "config:0-3"); + +static struct attribute *rpi_axi_pmu_formats_attr[] =3D { + &format_attr_filter.attr, + &format_attr_monitor.attr, + &format_attr_bus.attr, + &format_attr_counter.attr, + NULL, +}; + +static const struct attribute_group rpi_axi_pmu_format_group =3D { + .name =3D "format", + .attrs =3D rpi_axi_pmu_formats_attr, +}; + +/* + * Event Attribute Scaling & Unit Definitions: + * + * Uncore AXI bus transaction events (_rtrans, _wtrans, _atrans) report ha= rdware + * transaction beats. On Broadcom BCM2835 AXI interconnects, single-beat + * transactions transfer 32 bytes per beat, while contiguous DMA page burs= ts may + * use 64-byte double-beats. Setting .scale=3D"32" and .unit=3D"Bytes" pro= vides a + * close (~95%) byte throughput approximation in perf stat output (account= ing for + * 64-byte burst beats and VideoCore L2 cache prefetch hits). + */ +/* --- SYSFS NAMED EVENT ALIASES ------------------------------------ */ + +/* DMA Engine L2 Cache Bus Events (bus=3D0, BCM2835_SB__DMA_L2) */ +PMU_EVENT_ATTR_STRING(dma_l2_atrans, rpi_axi_pmu_event_dma_l2_atrans, + "monitor=3D0,bus=3D0,counter=3D0"); +PMU_EVENT_ATTR_STRING(dma_l2_atwait, rpi_axi_pmu_event_dma_l2_atwait, + "monitor=3D0,bus=3D0,counter=3D1"); +PMU_EVENT_ATTR_STRING(dma_l2_wtrans, rpi_axi_pmu_event_dma_l2_wtrans, + "monitor=3D0,bus=3D0,counter=3D2"); +PMU_EVENT_ATTR_STRING(dma_l2_wtrans.scale, rpi_axi_pmu_event_dma_l2_wtrans= _scale, + "32"); +PMU_EVENT_ATTR_STRING(dma_l2_wtrans.unit, rpi_axi_pmu_event_dma_l2_wtrans_= unit, + "Bytes"); +PMU_EVENT_ATTR_STRING(dma_l2_wtwait, rpi_axi_pmu_event_dma_l2_wtwait, + "monitor=3D0,bus=3D0,counter=3D3"); +PMU_EVENT_ATTR_STRING(dma_l2_rtrans, rpi_axi_pmu_event_dma_l2_rtrans, + "monitor=3D0,bus=3D0,counter=3D4"); +PMU_EVENT_ATTR_STRING(dma_l2_rtrans.scale, rpi_axi_pmu_event_dma_l2_rtrans= _scale, + "32"); +PMU_EVENT_ATTR_STRING(dma_l2_rtrans.unit, rpi_axi_pmu_event_dma_l2_rtrans_= unit, + "Bytes"); +PMU_EVENT_ATTR_STRING(dma_l2_rtwait, rpi_axi_pmu_event_dma_l2_rtwait, + "monitor=3D0,bus=3D0,counter=3D5"); + +/* Image Transposer Engine Events (bus=3D1, BCM2835_SB__TRANS: format rota= tion) */ +PMU_EVENT_ATTR_STRING(trans_atrans, rpi_axi_pmu_event_trans_atrans, + "monitor=3D0,bus=3D1,counter=3D0"); +PMU_EVENT_ATTR_STRING(trans_atwait, rpi_axi_pmu_event_trans_atwait, + "monitor=3D0,bus=3D1,counter=3D1"); +PMU_EVENT_ATTR_STRING(trans_wtrans, rpi_axi_pmu_event_trans_wtrans, + "monitor=3D0,bus=3D1,counter=3D2"); +PMU_EVENT_ATTR_STRING(trans_wtwait, rpi_axi_pmu_event_trans_wtwait, + "monitor=3D0,bus=3D1,counter=3D3"); +PMU_EVENT_ATTR_STRING(trans_rtrans, rpi_axi_pmu_event_trans_rtrans, + "monitor=3D0,bus=3D1,counter=3D4"); +PMU_EVENT_ATTR_STRING(trans_rtwait, rpi_axi_pmu_event_trans_rtwait, + "monitor=3D0,bus=3D1,counter=3D5"); + +/* Hardware JPEG Codec Accelerator Bus Events (bus=3D2, BCM2835_SB__JPEG) = */ +PMU_EVENT_ATTR_STRING(jpeg_atrans, rpi_axi_pmu_event_jpeg_atrans, + "monitor=3D0,bus=3D2,counter=3D0"); +PMU_EVENT_ATTR_STRING(jpeg_atwait, rpi_axi_pmu_event_jpeg_atwait, + "monitor=3D0,bus=3D2,counter=3D1"); +PMU_EVENT_ATTR_STRING(jpeg_wtrans, rpi_axi_pmu_event_jpeg_wtrans, + "monitor=3D0,bus=3D2,counter=3D2"); +PMU_EVENT_ATTR_STRING(jpeg_wtwait, rpi_axi_pmu_event_jpeg_wtwait, + "monitor=3D0,bus=3D2,counter=3D3"); +PMU_EVENT_ATTR_STRING(jpeg_rtrans, rpi_axi_pmu_event_jpeg_rtrans, + "monitor=3D0,bus=3D2,counter=3D4"); +PMU_EVENT_ATTR_STRING(jpeg_rtwait, rpi_axi_pmu_event_jpeg_rtwait, + "monitor=3D0,bus=3D2,counter=3D5"); + +/* System Uncached Memory Bus Events (bus=3D3, BCM2835_SB__SYSTEM_UC) */ +PMU_EVENT_ATTR_STRING(system_uc_atrans, rpi_axi_pmu_event_system_uc_atrans, + "monitor=3D0,bus=3D3,counter=3D0"); +PMU_EVENT_ATTR_STRING(system_uc_atwait, rpi_axi_pmu_event_system_uc_atwait, + "monitor=3D0,bus=3D3,counter=3D1"); +PMU_EVENT_ATTR_STRING(system_uc_wtrans, rpi_axi_pmu_event_system_uc_wtrans, + "monitor=3D0,bus=3D3,counter=3D2"); +PMU_EVENT_ATTR_STRING(system_uc_wtwait, rpi_axi_pmu_event_system_uc_wtwait, + "monitor=3D0,bus=3D3,counter=3D3"); +PMU_EVENT_ATTR_STRING(system_uc_rtrans, rpi_axi_pmu_event_system_uc_rtrans, + "monitor=3D0,bus=3D3,counter=3D4"); +PMU_EVENT_ATTR_STRING(system_uc_rtwait, rpi_axi_pmu_event_system_uc_rtwait, + "monitor=3D0,bus=3D3,counter=3D5"); + +/* DMA Engine Uncached Memory Bus Events (bus=3D4, BCM2835_SB__DMA_UC) */ +PMU_EVENT_ATTR_STRING(dma_uc_atrans, rpi_axi_pmu_event_dma_uc_atrans, + "monitor=3D0,bus=3D4,counter=3D0"); +PMU_EVENT_ATTR_STRING(dma_uc_atwait, rpi_axi_pmu_event_dma_uc_atwait, + "monitor=3D0,bus=3D4,counter=3D1"); +PMU_EVENT_ATTR_STRING(dma_uc_wtrans, rpi_axi_pmu_event_dma_uc_wtrans, + "monitor=3D0,bus=3D4,counter=3D2"); +PMU_EVENT_ATTR_STRING(dma_uc_wtwait, rpi_axi_pmu_event_dma_uc_wtwait, + "monitor=3D0,bus=3D4,counter=3D3"); +PMU_EVENT_ATTR_STRING(dma_uc_rtrans, rpi_axi_pmu_event_dma_uc_rtrans, + "monitor=3D0,bus=3D4,counter=3D4"); +PMU_EVENT_ATTR_STRING(dma_uc_rtwait, rpi_axi_pmu_event_dma_uc_rtwait, + "monitor=3D0,bus=3D4,counter=3D5"); + +/* System Main L2 Cache Bus Events (bus=3D5, BCM2835_SB__SYSTEM_L2) */ +PMU_EVENT_ATTR_STRING(system_l2_atrans, rpi_axi_pmu_event_system_l2_atrans, + "monitor=3D0,bus=3D5,counter=3D0"); +PMU_EVENT_ATTR_STRING(system_l2_atwait, rpi_axi_pmu_event_system_l2_atwait, + "monitor=3D0,bus=3D5,counter=3D1"); +PMU_EVENT_ATTR_STRING(system_l2_wtrans, rpi_axi_pmu_event_system_l2_wtrans, + "monitor=3D0,bus=3D5,counter=3D2"); +PMU_EVENT_ATTR_STRING(system_l2_wtrans.scale, rpi_axi_pmu_event_system_l2_= wtrans_scale, + "32"); +PMU_EVENT_ATTR_STRING(system_l2_wtrans.unit, rpi_axi_pmu_event_system_l2_w= trans_unit, + "Bytes"); +PMU_EVENT_ATTR_STRING(system_l2_wtwait, rpi_axi_pmu_event_system_l2_wtwait, + "monitor=3D0,bus=3D5,counter=3D3"); +PMU_EVENT_ATTR_STRING(system_l2_rtrans, rpi_axi_pmu_event_system_l2_rtrans, + "monitor=3D0,bus=3D5,counter=3D4"); +PMU_EVENT_ATTR_STRING(system_l2_rtrans.scale, rpi_axi_pmu_event_system_l2_= rtrans_scale, + "32"); +PMU_EVENT_ATTR_STRING(system_l2_rtrans.unit, rpi_axi_pmu_event_system_l2_r= trans_unit, + "Bytes"); +PMU_EVENT_ATTR_STRING(system_l2_rtwait, rpi_axi_pmu_event_system_l2_rtwait, + "monitor=3D0,bus=3D5,counter=3D5"); + +/* Legacy Camera & Message Passing Bus Events (bus=3D6-8, CCP2TX, MPHI_RX,= MPHI_TX) */ +PMU_EVENT_ATTR_STRING(ccp2tx_atrans, rpi_axi_pmu_event_ccp2tx_atrans, + "monitor=3D0,bus=3D6,counter=3D0"); +PMU_EVENT_ATTR_STRING(mphi_rx_atrans, rpi_axi_pmu_event_mphi_rx_atrans, + "monitor=3D0,bus=3D7,counter=3D0"); +PMU_EVENT_ATTR_STRING(mphi_tx_atrans, rpi_axi_pmu_event_mphi_tx_atrans, + "monitor=3D0,bus=3D8,counter=3D0"); + +/* Hardware Video Scaler Display Engine Bus Events (bus=3D9, BCM2835_SB__H= VS) */ +PMU_EVENT_ATTR_STRING(hvs_atrans, rpi_axi_pmu_event_hvs_atrans, + "monitor=3D0,bus=3D9,counter=3D0"); +PMU_EVENT_ATTR_STRING(hvs_atwait, rpi_axi_pmu_event_hvs_atwait, + "monitor=3D0,bus=3D9,counter=3D1"); +PMU_EVENT_ATTR_STRING(hvs_wtrans, rpi_axi_pmu_event_hvs_wtrans, + "monitor=3D0,bus=3D9,counter=3D2"); +PMU_EVENT_ATTR_STRING(hvs_wtrans.scale, rpi_axi_pmu_event_hvs_wtrans_scale, + "32"); +PMU_EVENT_ATTR_STRING(hvs_wtrans.unit, rpi_axi_pmu_event_hvs_wtrans_unit, + "Bytes"); +PMU_EVENT_ATTR_STRING(hvs_wtwait, rpi_axi_pmu_event_hvs_wtwait, + "monitor=3D0,bus=3D9,counter=3D3"); +PMU_EVENT_ATTR_STRING(hvs_rtrans, rpi_axi_pmu_event_hvs_rtrans, + "monitor=3D0,bus=3D9,counter=3D4"); +PMU_EVENT_ATTR_STRING(hvs_rtrans.scale, rpi_axi_pmu_event_hvs_rtrans_scale, + "32"); +PMU_EVENT_ATTR_STRING(hvs_rtrans.unit, rpi_axi_pmu_event_hvs_rtrans_unit, + "Bytes"); +PMU_EVENT_ATTR_STRING(hvs_rtwait, rpi_axi_pmu_event_hvs_rtwait, + "monitor=3D0,bus=3D9,counter=3D5"); + +/* H.264 Video Encoder/Decoder Bus Events (bus=3D10, BCM2835_SB__H264) */ +PMU_EVENT_ATTR_STRING(h264_atrans, rpi_axi_pmu_event_h264_atrans, + "monitor=3D0,bus=3D10,counter=3D0"); +PMU_EVENT_ATTR_STRING(h264_atwait, rpi_axi_pmu_event_h264_atwait, + "monitor=3D0,bus=3D10,counter=3D1"); +PMU_EVENT_ATTR_STRING(h264_wtrans, rpi_axi_pmu_event_h264_wtrans, + "monitor=3D0,bus=3D10,counter=3D2"); +PMU_EVENT_ATTR_STRING(h264_wtwait, rpi_axi_pmu_event_h264_wtwait, + "monitor=3D0,bus=3D10,counter=3D3"); +PMU_EVENT_ATTR_STRING(h264_rtrans, rpi_axi_pmu_event_h264_rtrans, + "monitor=3D0,bus=3D10,counter=3D4"); +PMU_EVENT_ATTR_STRING(h264_rtwait, rpi_axi_pmu_event_h264_rtwait, + "monitor=3D0,bus=3D10,counter=3D5"); + +/* Image Sensor Processor Camera Pipeline Bus Events (bus=3D11, BCM2835_SB= __ISP) */ +PMU_EVENT_ATTR_STRING(isp_atrans, rpi_axi_pmu_event_isp_atrans, + "monitor=3D0,bus=3D11,counter=3D0"); +PMU_EVENT_ATTR_STRING(isp_atwait, rpi_axi_pmu_event_isp_atwait, + "monitor=3D0,bus=3D11,counter=3D1"); +PMU_EVENT_ATTR_STRING(isp_wtrans, rpi_axi_pmu_event_isp_wtrans, + "monitor=3D0,bus=3D11,counter=3D2"); +PMU_EVENT_ATTR_STRING(isp_wtwait, rpi_axi_pmu_event_isp_wtwait, + "monitor=3D0,bus=3D11,counter=3D3"); +PMU_EVENT_ATTR_STRING(isp_rtrans, rpi_axi_pmu_event_isp_rtrans, + "monitor=3D0,bus=3D11,counter=3D4"); +PMU_EVENT_ATTR_STRING(isp_rtwait, rpi_axi_pmu_event_isp_rtwait, + "monitor=3D0,bus=3D11,counter=3D5"); + +/* VideoCore V3D 3D Graphics Hardware Pipeline Bus Events (bus=3D12, BCM28= 35_SB__V3D) */ +PMU_EVENT_ATTR_STRING(v3d_atrans, rpi_axi_pmu_event_v3d_atrans, + "monitor=3D0,bus=3D12,counter=3D0"); +PMU_EVENT_ATTR_STRING(v3d_atwait, rpi_axi_pmu_event_v3d_atwait, + "monitor=3D0,bus=3D12,counter=3D1"); +PMU_EVENT_ATTR_STRING(v3d_wtrans, rpi_axi_pmu_event_v3d_wtrans, + "monitor=3D0,bus=3D12,counter=3D2"); +PMU_EVENT_ATTR_STRING(v3d_wtrans.scale, rpi_axi_pmu_event_v3d_wtrans_scale, + "32"); +PMU_EVENT_ATTR_STRING(v3d_wtrans.unit, rpi_axi_pmu_event_v3d_wtrans_unit, + "Bytes"); +PMU_EVENT_ATTR_STRING(v3d_wtwait, rpi_axi_pmu_event_v3d_wtwait, + "monitor=3D0,bus=3D12,counter=3D3"); +PMU_EVENT_ATTR_STRING(v3d_rtrans, rpi_axi_pmu_event_v3d_rtrans, + "monitor=3D0,bus=3D12,counter=3D4"); +PMU_EVENT_ATTR_STRING(v3d_rtrans.scale, rpi_axi_pmu_event_v3d_rtrans_scale, + "32"); +PMU_EVENT_ATTR_STRING(v3d_rtrans.unit, rpi_axi_pmu_event_v3d_rtrans_unit, + "Bytes"); +PMU_EVENT_ATTR_STRING(v3d_rtwait, rpi_axi_pmu_event_v3d_rtwait, + "monitor=3D0,bus=3D12,counter=3D5"); + +/* Low-Speed Peripherals Bus Events (bus=3D13, BCM2835_SB__PERIPHERAL: UAR= T, SPI, I2C, GPIO) */ +PMU_EVENT_ATTR_STRING(peripheral_atrans, rpi_axi_pmu_event_peripheral_atra= ns, + "monitor=3D0,bus=3D13,counter=3D0"); + +/* CPU Uncached Memory Bus Events (bus=3D14, BCM2835_SB__CPU_UC) */ +PMU_EVENT_ATTR_STRING(cpu_uc_atrans, rpi_axi_pmu_event_cpu_uc_atrans, + "monitor=3D0,bus=3D14,counter=3D0"); +PMU_EVENT_ATTR_STRING(cpu_uc_atwait, rpi_axi_pmu_event_cpu_uc_atwait, + "monitor=3D0,bus=3D14,counter=3D1"); +PMU_EVENT_ATTR_STRING(cpu_uc_wtrans, rpi_axi_pmu_event_cpu_uc_wtrans, + "monitor=3D0,bus=3D14,counter=3D2"); +PMU_EVENT_ATTR_STRING(cpu_uc_wtwait, rpi_axi_pmu_event_cpu_uc_wtwait, + "monitor=3D0,bus=3D14,counter=3D3"); +PMU_EVENT_ATTR_STRING(cpu_uc_rtrans, rpi_axi_pmu_event_cpu_uc_rtrans, + "monitor=3D0,bus=3D14,counter=3D4"); +PMU_EVENT_ATTR_STRING(cpu_uc_rtwait, rpi_axi_pmu_event_cpu_uc_rtwait, + "monitor=3D0,bus=3D14,counter=3D5"); + +/* CPU L2 Cache Bus Events (bus=3D15, BCM2835_SB__CPU_L2) */ +PMU_EVENT_ATTR_STRING(cpu_l2_atrans, rpi_axi_pmu_event_cpu_l2_atrans, + "monitor=3D0,bus=3D15,counter=3D0"); +PMU_EVENT_ATTR_STRING(cpu_l2_atwait, rpi_axi_pmu_event_cpu_l2_atwait, + "monitor=3D0,bus=3D15,counter=3D1"); +PMU_EVENT_ATTR_STRING(cpu_l2_wtrans, rpi_axi_pmu_event_cpu_l2_wtrans, + "monitor=3D0,bus=3D15,counter=3D2"); +PMU_EVENT_ATTR_STRING(cpu_l2_wtwait, rpi_axi_pmu_event_cpu_l2_wtwait, + "monitor=3D0,bus=3D15,counter=3D3"); +PMU_EVENT_ATTR_STRING(cpu_l2_rtrans, rpi_axi_pmu_event_cpu_l2_rtrans, + "monitor=3D0,bus=3D15,counter=3D4"); +PMU_EVENT_ATTR_STRING(cpu_l2_rtwait, rpi_axi_pmu_event_cpu_l2_rtwait, + "monitor=3D0,bus=3D15,counter=3D5"); + +/* --- VIDEOCORE VPU MONITOR EVENTS (MON__VPU =3D 1, RPi 1-4) ------------= ------ */ + +/* VideoCore Core 0 & Core 1 Data/Instruction L2 Cache Bus Events */ +PMU_EVENT_ATTR_STRING(vpu1_d_l2_atrans, rpi_axi_pmu_event_vpu1_d_l2_atrans, + "monitor=3D1,bus=3D0,counter=3D0"); +PMU_EVENT_ATTR_STRING(vpu0_d_l2_atrans, rpi_axi_pmu_event_vpu0_d_l2_atrans, + "monitor=3D1,bus=3D1,counter=3D0"); +PMU_EVENT_ATTR_STRING(vpu1_i_l2_atrans, rpi_axi_pmu_event_vpu1_i_l2_atrans, + "monitor=3D1,bus=3D2,counter=3D0"); +PMU_EVENT_ATTR_STRING(vpu0_i_l2_atrans, rpi_axi_pmu_event_vpu0_i_l2_atrans, + "monitor=3D1,bus=3D3,counter=3D0"); + +/* VPU System L2 Cache Interconnect Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_system_l2_atrans, rpi_axi_pmu_event_vpu_system_l= 2_atrans, + "monitor=3D1,bus=3D4,counter=3D0"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_wtrans, rpi_axi_pmu_event_vpu_system_l= 2_wtrans, + "monitor=3D1,bus=3D4,counter=3D2"); +PMU_EVENT_ATTR_STRING(vpu_system_l2_rtrans, rpi_axi_pmu_event_vpu_system_l= 2_rtrans, + "monitor=3D1,bus=3D4,counter=3D4"); + +/* VPU Cache Flush Controller & VPU DMA L2 Cache Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_l2_flush_atrans, rpi_axi_pmu_event_vpu_l2_flush_= atrans, + "monitor=3D1,bus=3D5,counter=3D0"); +PMU_EVENT_ATTR_STRING(vpu_dma_l2_atrans, rpi_axi_pmu_event_vpu_dma_l2_atra= ns, + "monitor=3D1,bus=3D6,counter=3D0"); + +/* VideoCore Core 0 & Core 1 Data/Instruction Uncached Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu1_d_uc_atrans, rpi_axi_pmu_event_vpu1_d_uc_atrans, + "monitor=3D1,bus=3D7,counter=3D0"); +PMU_EVENT_ATTR_STRING(vpu0_d_uc_atrans, rpi_axi_pmu_event_vpu0_d_uc_atrans, + "monitor=3D1,bus=3D8,counter=3D0"); +PMU_EVENT_ATTR_STRING(vpu1_i_uc_atrans, rpi_axi_pmu_event_vpu1_i_uc_atrans, + "monitor=3D1,bus=3D9,counter=3D0"); +PMU_EVENT_ATTR_STRING(vpu0_i_uc_atrans, rpi_axi_pmu_event_vpu0_i_uc_atrans, + "monitor=3D1,bus=3D10,counter=3D0"); + +/* VPU System Uncached Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_system_uc_atrans, rpi_axi_pmu_event_vpu_system_u= c_atrans, + "monitor=3D1,bus=3D11,counter=3D0"); +PMU_EVENT_ATTR_STRING(vpu_system_uc_wtrans, rpi_axi_pmu_event_vpu_system_u= c_wtrans, + "monitor=3D1,bus=3D11,counter=3D2"); +PMU_EVENT_ATTR_STRING(vpu_system_uc_rtrans, rpi_axi_pmu_event_vpu_system_u= c_rtrans, + "monitor=3D1,bus=3D11,counter=3D4"); + +/* VPU L2 Cache Outbound & VPU DMA Uncached Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_l2_out_atrans, rpi_axi_pmu_event_vpu_l2_out_atra= ns, + "monitor=3D1,bus=3D12,counter=3D0"); +PMU_EVENT_ATTR_STRING(vpu_dma_uc_atrans, rpi_axi_pmu_event_vpu_dma_uc_atra= ns, + "monitor=3D1,bus=3D13,counter=3D0"); + +/* VPU SDRAM Memory Controller Bus Events & Stall Wait Cycles */ +PMU_EVENT_ATTR_STRING(vpu_sdram_atrans, rpi_axi_pmu_event_vpu_sdram_atrans, + "monitor=3D1,bus=3D14,counter=3D0"); +PMU_EVENT_ATTR_STRING(vpu_sdram_atwait, rpi_axi_pmu_event_vpu_sdram_atwait, + "monitor=3D1,bus=3D14,counter=3D1"); +PMU_EVENT_ATTR_STRING(vpu_sdram_wtrans, rpi_axi_pmu_event_vpu_sdram_wtrans, + "monitor=3D1,bus=3D14,counter=3D2"); +PMU_EVENT_ATTR_STRING(vpu_sdram_wtwait, rpi_axi_pmu_event_vpu_sdram_wtwait, + "monitor=3D1,bus=3D14,counter=3D3"); +PMU_EVENT_ATTR_STRING(vpu_sdram_rtrans, rpi_axi_pmu_event_vpu_sdram_rtrans, + "monitor=3D1,bus=3D14,counter=3D4"); +PMU_EVENT_ATTR_STRING(vpu_sdram_rtwait, rpi_axi_pmu_event_vpu_sdram_rtwait, + "monitor=3D1,bus=3D14,counter=3D5"); + +/* VPU L2 Cache Inbound Memory Bus Events */ +PMU_EVENT_ATTR_STRING(vpu_l2_in_atrans, rpi_axi_pmu_event_vpu_l2_in_atrans, + "monitor=3D1,bus=3D15,counter=3D0"); + +/* --- AXI MASTER FILTERED ALIASES (RPi 1-4, BCM2835-BCM2711) ------------= --- */ + +/* CPU Core 0 & Core 1 Instruction and Data Cache Filtered Events */ +PMU_EVENT_ATTR_STRING(cpu0_icache_rtrans, rpi_axi_pmu_event_cpu0_icache_rt= rans, + "monitor=3D0,bus=3D15,counter=3D4,filter=3D2"); +PMU_EVENT_ATTR_STRING(cpu0_dcache_rtrans, rpi_axi_pmu_event_cpu0_dcache_rt= rans, + "monitor=3D0,bus=3D15,counter=3D4,filter=3D3"); +PMU_EVENT_ATTR_STRING(cpu0_dcache_wtrans, rpi_axi_pmu_event_cpu0_dcache_wt= rans, + "monitor=3D0,bus=3D15,counter=3D2,filter=3D3"); +PMU_EVENT_ATTR_STRING(cpu1_icache_rtrans, rpi_axi_pmu_event_cpu1_icache_rt= rans, + "monitor=3D0,bus=3D15,counter=3D4,filter=3D5"); +PMU_EVENT_ATTR_STRING(cpu1_dcache_rtrans, rpi_axi_pmu_event_cpu1_dcache_rt= rans, + "monitor=3D0,bus=3D15,counter=3D4,filter=3D6"); +PMU_EVENT_ATTR_STRING(cpu1_dcache_wtrans, rpi_axi_pmu_event_cpu1_dcache_wt= rans, + "monitor=3D0,bus=3D15,counter=3D2,filter=3D6"); + +/* System DMA Channels 0 & 1 L2 Cache Filtered Events */ +PMU_EVENT_ATTR_STRING(dma0_l2_rtrans, rpi_axi_pmu_event_dma0_l2_rtrans, + "monitor=3D0,bus=3D0,counter=3D4,filter=3D16"); +PMU_EVENT_ATTR_STRING(dma0_l2_wtrans, rpi_axi_pmu_event_dma0_l2_wtrans, + "monitor=3D0,bus=3D0,counter=3D2,filter=3D16"); +PMU_EVENT_ATTR_STRING(dma1_l2_rtrans, rpi_axi_pmu_event_dma1_l2_rtrans, + "monitor=3D0,bus=3D0,counter=3D4,filter=3D17"); +PMU_EVENT_ATTR_STRING(dma1_l2_wtrans, rpi_axi_pmu_event_dma1_l2_wtrans, + "monitor=3D0,bus=3D0,counter=3D2,filter=3D17"); + +/* VideoCore V3D Pipes 0 & 1 L2 Cache Filtered Events */ +PMU_EVENT_ATTR_STRING(v3d0_system_l2_rtrans, rpi_axi_pmu_event_v3d0_system= _l2_rtrans, + "monitor=3D0,bus=3D5,counter=3D4,filter=3D25"); +PMU_EVENT_ATTR_STRING(v3d0_system_l2_wtrans, rpi_axi_pmu_event_v3d0_system= _l2_wtrans, + "monitor=3D0,bus=3D5,counter=3D2,filter=3D25"); +PMU_EVENT_ATTR_STRING(v3d1_system_l2_rtrans, rpi_axi_pmu_event_v3d1_system= _l2_rtrans, + "monitor=3D0,bus=3D5,counter=3D4,filter=3D26"); +PMU_EVENT_ATTR_STRING(v3d1_system_l2_wtrans, rpi_axi_pmu_event_v3d1_system= _l2_wtrans, + "monitor=3D0,bus=3D5,counter=3D2,filter=3D26"); + +/* HVS Display Engine, ISP Camera, and USB 2.0 Host Controller Filtered Ev= ents */ +PMU_EVENT_ATTR_STRING(hvs_system_l2_rtrans, rpi_axi_pmu_event_hvs_system_l= 2_rtrans, + "monitor=3D0,bus=3D5,counter=3D4,filter=3D10"); +PMU_EVENT_ATTR_STRING(hvs_system_l2_wtrans, rpi_axi_pmu_event_hvs_system_l= 2_wtrans, + "monitor=3D0,bus=3D5,counter=3D2,filter=3D10"); +PMU_EVENT_ATTR_STRING(isp_system_l2_rtrans, rpi_axi_pmu_event_isp_system_l= 2_rtrans, + "monitor=3D0,bus=3D5,counter=3D4,filter=3D11"); +PMU_EVENT_ATTR_STRING(isp_system_l2_wtrans, rpi_axi_pmu_event_isp_system_l= 2_wtrans, + "monitor=3D0,bus=3D5,counter=3D2,filter=3D11"); +PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_pmu_event_usb_system_l= 2_rtrans, + "monitor=3D0,bus=3D5,counter=3D4,filter=3D24"); +PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l= 2_wtrans, + "monitor=3D0,bus=3D5,counter=3D2,filter=3D24"); + +static struct attribute *rpi_axi_pmu_events_attrs[] =3D { + /* System Monitor events */ + &rpi_axi_pmu_event_dma_l2_atrans.attr.attr, + &rpi_axi_pmu_event_dma_l2_atwait.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_dma_l2_wtwait.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_dma_l2_rtwait.attr.attr, + + &rpi_axi_pmu_event_trans_atrans.attr.attr, + &rpi_axi_pmu_event_trans_atwait.attr.attr, + &rpi_axi_pmu_event_trans_wtrans.attr.attr, + &rpi_axi_pmu_event_trans_wtwait.attr.attr, + &rpi_axi_pmu_event_trans_rtrans.attr.attr, + &rpi_axi_pmu_event_trans_rtwait.attr.attr, + + &rpi_axi_pmu_event_jpeg_atrans.attr.attr, + &rpi_axi_pmu_event_jpeg_atwait.attr.attr, + &rpi_axi_pmu_event_jpeg_wtrans.attr.attr, + &rpi_axi_pmu_event_jpeg_wtwait.attr.attr, + &rpi_axi_pmu_event_jpeg_rtrans.attr.attr, + &rpi_axi_pmu_event_jpeg_rtwait.attr.attr, + + &rpi_axi_pmu_event_system_uc_atrans.attr.attr, + &rpi_axi_pmu_event_system_uc_atwait.attr.attr, + &rpi_axi_pmu_event_system_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_system_uc_wtwait.attr.attr, + &rpi_axi_pmu_event_system_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_system_uc_rtwait.attr.attr, + + &rpi_axi_pmu_event_dma_uc_atrans.attr.attr, + &rpi_axi_pmu_event_dma_uc_atwait.attr.attr, + &rpi_axi_pmu_event_dma_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_dma_uc_wtwait.attr.attr, + &rpi_axi_pmu_event_dma_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_dma_uc_rtwait.attr.attr, + + &rpi_axi_pmu_event_system_l2_atrans.attr.attr, + &rpi_axi_pmu_event_system_l2_atwait.attr.attr, + &rpi_axi_pmu_event_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_system_l2_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_system_l2_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_system_l2_wtwait.attr.attr, + &rpi_axi_pmu_event_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_system_l2_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_system_l2_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_system_l2_rtwait.attr.attr, + + &rpi_axi_pmu_event_ccp2tx_atrans.attr.attr, + &rpi_axi_pmu_event_mphi_rx_atrans.attr.attr, + &rpi_axi_pmu_event_mphi_tx_atrans.attr.attr, + + &rpi_axi_pmu_event_hvs_atrans.attr.attr, + &rpi_axi_pmu_event_hvs_atwait.attr.attr, + &rpi_axi_pmu_event_hvs_wtrans.attr.attr, + &rpi_axi_pmu_event_hvs_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_hvs_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_hvs_wtwait.attr.attr, + &rpi_axi_pmu_event_hvs_rtrans.attr.attr, + &rpi_axi_pmu_event_hvs_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_hvs_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_hvs_rtwait.attr.attr, + + &rpi_axi_pmu_event_h264_atrans.attr.attr, + &rpi_axi_pmu_event_h264_atwait.attr.attr, + &rpi_axi_pmu_event_h264_wtrans.attr.attr, + &rpi_axi_pmu_event_h264_wtwait.attr.attr, + &rpi_axi_pmu_event_h264_rtrans.attr.attr, + &rpi_axi_pmu_event_h264_rtwait.attr.attr, + + &rpi_axi_pmu_event_isp_atrans.attr.attr, + &rpi_axi_pmu_event_isp_atwait.attr.attr, + &rpi_axi_pmu_event_isp_wtrans.attr.attr, + &rpi_axi_pmu_event_isp_wtwait.attr.attr, + &rpi_axi_pmu_event_isp_rtrans.attr.attr, + &rpi_axi_pmu_event_isp_rtwait.attr.attr, + + &rpi_axi_pmu_event_v3d_atrans.attr.attr, + &rpi_axi_pmu_event_v3d_atwait.attr.attr, + &rpi_axi_pmu_event_v3d_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d_wtrans_scale.attr.attr, + &rpi_axi_pmu_event_v3d_wtrans_unit.attr.attr, + &rpi_axi_pmu_event_v3d_wtwait.attr.attr, + &rpi_axi_pmu_event_v3d_rtrans.attr.attr, + &rpi_axi_pmu_event_v3d_rtrans_scale.attr.attr, + &rpi_axi_pmu_event_v3d_rtrans_unit.attr.attr, + &rpi_axi_pmu_event_v3d_rtwait.attr.attr, + + &rpi_axi_pmu_event_peripheral_atrans.attr.attr, + + &rpi_axi_pmu_event_cpu_uc_atrans.attr.attr, + &rpi_axi_pmu_event_cpu_uc_atwait.attr.attr, + &rpi_axi_pmu_event_cpu_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_cpu_uc_wtwait.attr.attr, + &rpi_axi_pmu_event_cpu_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu_uc_rtwait.attr.attr, + + &rpi_axi_pmu_event_cpu_l2_atrans.attr.attr, + &rpi_axi_pmu_event_cpu_l2_atwait.attr.attr, + &rpi_axi_pmu_event_cpu_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_cpu_l2_wtwait.attr.attr, + &rpi_axi_pmu_event_cpu_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu_l2_rtwait.attr.attr, + + /* VPU Monitor events (RPi 1-4) */ + &rpi_axi_pmu_event_vpu1_d_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_d_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu1_i_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_i_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_vpu_l2_flush_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_dma_l2_atrans.attr.attr, + &rpi_axi_pmu_event_vpu1_d_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_d_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu1_i_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu0_i_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_uc_wtrans.attr.attr, + &rpi_axi_pmu_event_vpu_system_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_vpu_l2_out_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_dma_uc_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_atrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_atwait.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_wtrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_wtwait.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_rtrans.attr.attr, + &rpi_axi_pmu_event_vpu_sdram_rtwait.attr.attr, + &rpi_axi_pmu_event_vpu_l2_in_atrans.attr.attr, + + /* Filtered Event Aliases (RPi 1-4) */ + &rpi_axi_pmu_event_cpu0_icache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu0_dcache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu0_dcache_wtrans.attr.attr, + &rpi_axi_pmu_event_cpu1_icache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu1_dcache_rtrans.attr.attr, + &rpi_axi_pmu_event_cpu1_dcache_wtrans.attr.attr, + &rpi_axi_pmu_event_dma0_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_dma0_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_dma1_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_dma1_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d0_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_v3d0_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d1_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_v3d1_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_hvs_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_hvs_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_isp_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_isp_system_l2_wtrans.attr.attr, + &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr, + &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr, + + NULL, +}; + +/** + * rpi_axi_pmu_events_is_visible() - Sysfs attribute visibility callback f= or event aliases + * @kobj: Pointer to sysfs kobject for events directory + * @attr: Pointer to sysfs attribute being queried + * @unused: Index (unused) + * + * Dynamically filters sysfs event aliases based on the detected Broadcom = SoC generation: + * + * Return: attr->mode (0444) if visible on current SoC, 0 to hide. + */ +static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj, + struct attribute *attr, int unused) +{ + return attr->mode; +} + +static const struct attribute_group rpi_axi_pmu_events_group =3D { + .name =3D "events", + .attrs =3D rpi_axi_pmu_events_attrs, + .is_visible =3D rpi_axi_pmu_events_is_visible, +}; + +/** + * cpumask_show() - Sysfs attribute callback to display assigned PMU CPU c= ore + * @dev: Pointer to device structure + * @attr: Pointer to device attribute + * @buf: Output buffer for cpumask string representation + * + * Return: Number of bytes written to buffer. + */ +static ssize_t cpumask_show(struct device *dev, struct device_attribute *a= ttr, char *buf) +{ + struct pmu *pmu =3D dev_get_drvdata(dev); + struct rpi_axi_pmu *rpi_pmu =3D pmu_to_rpi_axi_pmu(pmu); + + return cpumap_print_to_pagebuf(true, buf, cpumask_of(rpi_pmu->cpu)); +} +static DEVICE_ATTR_RO(cpumask); + +static struct attribute *rpi_axi_pmu_cpumask_attrs[] =3D { + &dev_attr_cpumask.attr, + NULL, +}; + +static const struct attribute_group rpi_axi_pmu_cpumask_group =3D { + .attrs =3D rpi_axi_pmu_cpumask_attrs, +}; + +static const struct attribute_group *rpi_axi_pmu_attr_groups[] =3D { + &rpi_axi_pmu_format_group, + &rpi_axi_pmu_events_group, + &rpi_axi_pmu_cpumask_group, + NULL, +}; + +/** + * rpi_axi_pmu__validate_event() - Validates single event resource capabil= ity + * @pmu: Pointer to core PMU struct + * @fake_hw_events: Temporary array of watcher structures for pre-flight c= heck + * @event: Pointer to perf_event being validated + * + * Return: true if event can be scheduled, false otherwise. + */ +static bool rpi_axi_pmu__validate_event(struct pmu *pmu, + struct rpi_axi_hw_events fake_hw_events[MON__MAX], + struct perf_event *event) +{ + enum monitor mon; + + if (is_software_event(event)) + return true; + if (event->pmu !=3D pmu) + return false; + + mon =3D config_to_monitor(event->attr.config); + return rpi_axi_hw_events__get_alloc_event_idx(&fake_hw_events[mon], event= ) >=3D 0; +} + +/** + * rpi_axi_pmu__validate_group() - Validates event group scheduleability + * @event: Pointer to leader or sibling perf_event + * + * Simulates bus watcher allocation across both monitors to guarantee all = events + * in the group can run simultaneously. + * + * Return: true if group is valid, false otherwise. + */ +static bool rpi_axi_pmu__validate_group(struct perf_event *event) +{ + struct perf_event *sibling, *leader =3D event->group_leader; + struct rpi_axi_hw_events fake_hw_events[MON__MAX]; + + rpi_axi_hw_events__init(&fake_hw_events[MON__SYSTEM]); + rpi_axi_hw_events__init(&fake_hw_events[MON__VPU]); + + if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, leader)) + return false; + + for_each_sibling_event(sibling, leader) { + if (!rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, sibling)) + return false; + } + return rpi_axi_pmu__validate_event(event->pmu, fake_hw_events, event); +} + +/** + * rpi_axi_pmu_event_init() - Perf callback to initialize a perf_event + * @event: Pointer to perf_event to initialize + * + * Return: 0 on success, negative error code on failure. + */ +static int rpi_axi_pmu_event_init(struct perf_event *event) +{ + struct rpi_axi_pmu *pmu; + struct device *dev; + + if (event->attr.type !=3D event->pmu->type) + return -ENOENT; + + pmu =3D pmu_to_rpi_axi_pmu(event->pmu); + dev =3D pmu->pmu.dev; + + if (!config_is_valid(pmu, event->attr.config)) { + dev_dbg(dev, "Invalid event config\n"); + return -EINVAL; + } + + if (is_sampling_event(event)) { + dev_dbg(dev, "Sampling not supported\n"); + return -EOPNOTSUPP; + } + + if (event->cpu < 0) { + dev_dbg(dev, "Per-task data not supported\n"); + return -EOPNOTSUPP; + } + + if (event->cpu !=3D pmu->cpu) { + dev_dbg(dev, "Can only bind to PMU CPU %d\n", pmu->cpu); + return -EINVAL; + } + + if (!rpi_axi_pmu__validate_group(event)) { + dev_dbg(dev, "Invalid event or grouping of events\n"); + return -EINVAL; + } + event->hw.idx =3D -1; + return 0; +} + +/** + * set_monitor_control() - Writes to global monitor control register + * @pmu: Pointer to PMU context + * @mon: Monitor selection (MON__SYSTEM or MON__VPU) + * @set: Control bitmask to write + */ +static void set_monitor_control(struct rpi_axi_pmu *pmu, enum monitor mon,= u32 set) +{ + if (pmu->monitor[mon].use_mailbox_interface) { + u32 tmp[3] =3D {pmu->monitor[mon].mailbox + GEN_CTRL, 1, set}; + int err; + + might_sleep(); + lockdep_assert_held(&pmu->vpu_mutex); + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled())) + return; + + err =3D rpi_firmware_property(pmu->firmware, + RPI_FIRMWARE_SET_PERIPH_REG, + tmp, sizeof(tmp)); + + if (err < 0 || tmp[1] !=3D 1) + dev_err(&pmu->pdev->dev, "Failed to set monitor control\n"); + } else { + lockdep_assert_held(&pmu->lock); + writel(set, pmu->monitor[mon].base_address + GEN_CTRL); + } +} + +static int watcher_offset(const struct rpi_axi_pmu *pmu, enum monitor mon,= int idx) +{ + return BW0_CTRL + idx * BW_PITCH; +} + +/** + * set_bus_watcher_control() - Writes to a specific bus watcher control re= gister + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + * @set: Control bitmask to write + */ +static void set_bus_watcher_control(struct rpi_axi_pmu *pmu, enum monitor = mon, int idx, u32 set) +{ + int watcher =3D watcher_offset(pmu, mon, idx); + + if (pmu->monitor[mon].use_mailbox_interface) { + u32 tmp[3] =3D {pmu->monitor[mon].mailbox + watcher, 1, set}; + int err; + + might_sleep(); + lockdep_assert_held(&pmu->vpu_mutex); + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled())) + return; + + err =3D rpi_firmware_property(pmu->firmware, + RPI_FIRMWARE_SET_PERIPH_REG, + tmp, sizeof(tmp)); + if (err < 0 || tmp[1] !=3D 1) + dev_err(&pmu->pdev->dev, "Failed to set bus watcher control\n"); + } else { + lockdep_assert_held(&pmu->lock); + writel(set, pmu->monitor[mon].base_address + watcher); + } +} + +/** + * rpi_axi_pmu_enable_bus_watcher() - Enables a hardware bus watcher unit + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + * @bus: Monitored AXI bus index + * @filter: AXI master ID filter + */ +static void rpi_axi_pmu_enable_bus_watcher(struct rpi_axi_pmu *pmu, enum m= onitor mon, + int idx, int bus, int filter) +{ + int bus_control; + + if (pmu->monitor[mon].hw_events.enabled[idx]) + return; + + bus_control =3D BW_CTRL_ENABLE_BIT | (bus & 0x3F); + if (filter) { + bus_control |=3D BW_CTRL_ENABLE_ID_FILTER_BIT; + bus_control |=3D (filter & 0x1F) << BW_CTRL_BUS_FILTER_SHIFT; + } + set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT); + set_bus_watcher_control(pmu, mon, idx, bus_control); + + if (!pmu->monitor[mon].hw_events.monitor_running) { + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT); + set_monitor_control(pmu, mon, GEN_CTL_ENABLE_BIT | GEN_CTL_WATCH_BIT); + } +} + +/** + * rpi_axi_pmu_disable_bus_watcher() - Resets and disables a hardware bus = watcher unit + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + */ +static void rpi_axi_pmu_disable_bus_watcher(struct rpi_axi_pmu *pmu, enum = monitor mon, int idx) +{ + + set_bus_watcher_control(pmu, mon, idx, BW_CTRL_RESET_BIT); +} + +static int counter_offset(enum counter counter) +{ + switch (counter) { + case CNT__ATRANS: return BW_ATRANS_OFFSET; + case CNT__ATWAIT: return BW_ATWAIT_OFFSET; + case CNT__WTRANS: return BW_WTRANS_OFFSET; + case CNT__WTWAIT: return BW_WTWAIT_OFFSET; + case CNT__RTRANS: return BW_RTRANS_OFFSET; + case CNT__RTWAIT: return BW_RTWAIT_OFFSET; + default: return 0; + } +} + +/** + * rpi_axi_pmu_read_counter() - Reads raw 32-bit hardware counter from wat= cher + * @pmu: Pointer to PMU context + * @mon: Monitor selection + * @idx: Watcher index (0..2) + * @counter: Metric counter type (enum counter, 0..8) + * + * Return: 32-bit raw hardware counter value. + */ +static u32 rpi_axi_pmu_read_counter(struct rpi_axi_pmu *pmu, enum monitor = mon, int idx, + enum counter counter) +{ + int watcher =3D watcher_offset(pmu, mon, idx); + int offset =3D counter_offset(counter); + u32 ret; + + if (!pmu->monitor[mon].hw_events.enabled[idx]) + return 0; + + if (pmu->monitor[mon].use_mailbox_interface) { + u32 tmp[3] =3D { + pmu->monitor[mon].mailbox + watcher + offset, + 1, -1 + }; + int err; + + might_sleep(); + lockdep_assert_held(&pmu->vpu_mutex); + if (WARN_ON_ONCE(in_interrupt() || irqs_disabled())) + return -1; + + err =3D rpi_firmware_property(pmu->firmware, + RPI_FIRMWARE_GET_PERIPH_REG, + tmp, sizeof(tmp)); + + if (err < 0 || tmp[1] !=3D 1) { + dev_err(&pmu->pdev->dev, "Failed to read bus watcher\n"); + /* + * Return U32_MAX on IPC failure to indicate invalid read. + * Valid hardware counters are 31-bit and masked to 0x7FFFFFFF, + * so U32_MAX is unambiguously an error state. + */ + return U32_MAX; + } + ret =3D tmp[2] & 0x7FFFFFFF; + } else { + void __iomem *addr =3D pmu->monitor[mon].base_address + watcher + offset; + + lockdep_assert_held(&pmu->lock); + ret =3D readl(addr) & 0x7FFFFFFF; + } + return ret; +} + +/** + * rpi_axi_pmu_read() - Perf callback to update event counter value + * @event: Pointer to perf_event being read + * + * Thread-safe counter update implementation: + * - System Monitor (MON__SYSTEM): Uses fast MMIO (~15ns), serialized via = pmu->lock spinlock. + * - VPU Monitor (MON__VPU): Async background polling in process context (= vpu_work). + * Direct read() syscalls return cached count immediately without blocki= ng. + */ +static void rpi_axi_pmu_read(struct perf_event *event) +{ + struct rpi_axi_pmu *pmu =3D pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon =3D config_to_monitor(event->attr.config); + enum counter counter =3D config_to_counter(event->attr.config); + u64 prev_count, new_count; + unsigned long flags; + u32 delta; + + /* Mailbox VPU counters are polled asynchronously in background vpu_work. + * MMIO monitors (System) are read synchronously. + */ + if (event->hw.idx < 0) + return; + + if (pmu->monitor[mon].use_mailbox_interface) + return; + + spin_lock_irqsave(&pmu->lock, flags); + if (!pmu->monitor[mon].hw_events.enabled[event->hw.idx] || + (event->hw.state & PERF_HES_STOPPED)) { + spin_unlock_irqrestore(&pmu->lock, flags); + return; + } + prev_count =3D local64_read(&event->hw.prev_count); + new_count =3D rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter); + if (new_count =3D=3D U32_MAX) { + /* + * U32_MAX indicates a hardware or IPC read failure. Ignore the update + * to prevent spurious artificial counter spikes from underflows. + */ + spin_unlock_irqrestore(&pmu->lock, flags); + return; + } + local64_set(&event->hw.prev_count, new_count); + delta =3D (new_count - prev_count) & 0x7FFFFFFF; + local64_add(delta, &event->count); + spin_unlock_irqrestore(&pmu->lock, flags); +} + +/** + * rpi_axi_pmu_vpu_work_handler() - Background work handler for VPU monito= r counter reads + * @work: Pointer to work_struct inside struct rpi_axi_pmu + * + * Runs in kernel process context where sleeping is allowed. Reads fresh V= PU counter values over + * VideoCore Mailbox IPC under vpu_mutex and updates the cached perf event= count safely. + * ABA pointer recycling races are prevented by validating per-slot sequen= ce counters (event_gen). + * Multiplexing counter rotation baselines are established using the PERF_= HES_UPTODATE flag. + */ +static void rpi_axi_pmu_vpu_work_handler(struct work_struct *work) +{ + struct rpi_axi_pmu *pmu =3D container_of(work, struct rpi_axi_pmu, vpu_wo= rk); + + might_sleep(); + mutex_lock(&pmu->vpu_mutex); + spin_lock_irq(&pmu->lock); + + for (int idx =3D 0; idx < NUM_BUS_WATCHERS_PER_MONITOR; idx++) { + if (pmu->monitor[MON__VPU].hw_events.vpu_disable_pending[idx]) { + pmu->monitor[MON__VPU].hw_events.vpu_disable_pending[idx] =3D false; + if (pmu->monitor[MON__VPU].use_mailbox_interface) { + spin_unlock_irq(&pmu->lock); + rpi_axi_pmu_disable_bus_watcher(pmu, MON__VPU, idx); + spin_lock_irq(&pmu->lock); + } else { + rpi_axi_pmu_disable_bus_watcher(pmu, MON__VPU, idx); + } + } + } + + for (int i =3D 0; i < RPI_AXI_MAX_EVENTS; i++) { + struct perf_event *event =3D pmu->events[i]; + enum counter counter; + u64 gen; + int idx; + u64 prev_count, new_count; + u32 delta; + + if (!event || (event->hw.state & PERF_HES_STOPPED) || + config_to_monitor(event->attr.config) !=3D MON__VPU) + continue; + + gen =3D pmu->event_gen[i]; + counter =3D config_to_counter(event->attr.config); + idx =3D event->hw.idx; + + /* If VPU bus watcher is not enabled on hardware, enable it in process c= ontext */ + if (!pmu->monitor[MON__VPU].hw_events.enabled[idx]) { + int bus =3D pmu->monitor[MON__VPU].hw_events.monitored_bus[idx]; + int filter =3D pmu->monitor[MON__VPU].hw_events.filter[idx]; + + if (pmu->monitor[MON__VPU].use_mailbox_interface) { + spin_unlock_irq(&pmu->lock); + rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter); + spin_lock_irq(&pmu->lock); + } else { + rpi_axi_pmu_enable_bus_watcher(pmu, MON__VPU, idx, bus, filter); + } + pmu->monitor[MON__VPU].hw_events.monitor_running =3D true; + } + + /* Drop spinlock during Mailbox IPC read only if using mailbox interface= */ + if (pmu->monitor[MON__VPU].use_mailbox_interface) { + spin_unlock_irq(&pmu->lock); + new_count =3D rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter); + spin_lock_irq(&pmu->lock); + } else { + new_count =3D rpi_axi_pmu_read_counter(pmu, MON__VPU, idx, counter); + } + + /* + * U32_MAX indicates a hardware or IPC read failure. Ignore the update + * to prevent spurious artificial counter spikes from underflows. + */ + if (new_count =3D=3D U32_MAX) + continue; + + /* + * Verify event pointer and generation sequence counter match + * (ABA pointer recycling race prevention). + */ + if (pmu->events[i] =3D=3D event && + pmu->event_gen[i] =3D=3D gen && + !(event->hw.state & PERF_HES_STOPPED)) { + pmu->monitor[MON__VPU].hw_events.enabled[idx] =3D true; + if (!(event->hw.state & PERF_HES_UPTODATE)) { + /* Initial baseline read for newly started/rotated VPU event */ + local64_set(&event->hw.prev_count, new_count); + event->hw.state |=3D PERF_HES_UPTODATE; + } else { + prev_count =3D local64_read(&event->hw.prev_count); + local64_set(&event->hw.prev_count, new_count); + delta =3D (new_count - prev_count) & 0x7FFFFFFF; + local64_add(delta, &event->count); + } + } + } + + if (pmu->monitor[MON__VPU].hw_events.num_monitored =3D=3D 0 && + pmu->monitor[MON__VPU].hw_events.monitor_running) { + if (pmu->monitor[MON__VPU].use_mailbox_interface) { + spin_unlock_irq(&pmu->lock); + set_monitor_control(pmu, MON__VPU, GEN_CTL_RESET_BIT); + spin_lock_irq(&pmu->lock); + } else { + set_monitor_control(pmu, MON__VPU, GEN_CTL_RESET_BIT); + } + pmu->monitor[MON__VPU].hw_events.monitor_running =3D false; + } + + spin_unlock_irq(&pmu->lock); + mutex_unlock(&pmu->vpu_mutex); +} + +/** + * rpi_axi_pmu_timer_handler() - Periodic hrtimer callback for 32-bit coun= ter overflow polling + * @timer: Pointer to hrtimer structure + * + * Runs in hard IRQ / atomic context. + * Holding pmu->lock, reads System monitor counters (MON__SYSTEM) via fast= MMIO. + * Schedules background work (vpu_work) if active_vpu_events > 0. + * + * Return: HRTIMER_RESTART. + */ +static enum hrtimer_restart rpi_axi_pmu_timer_handler(struct hrtimer *time= r) +{ + struct rpi_axi_pmu *pmu =3D container_of(timer, struct rpi_axi_pmu, hrtim= er); + unsigned long flags; + + spin_lock_irqsave(&pmu->lock, flags); + if (pmu->active_events =3D=3D 0) { + spin_unlock_irqrestore(&pmu->lock, flags); + return HRTIMER_NORESTART; + } + for (int i =3D 0; i < RPI_AXI_MAX_EVENTS; i++) { + struct perf_event *event =3D pmu->events[i]; + enum counter counter; + u64 prev_count, new_count; + u32 delta; + + if (!event || (event->hw.state & PERF_HES_STOPPED) || + config_to_monitor(event->attr.config) !=3D MON__SYSTEM) + continue; + + counter =3D config_to_counter(event->attr.config); + prev_count =3D local64_read(&event->hw.prev_count); + new_count =3D rpi_axi_pmu_read_counter(pmu, MON__SYSTEM, + event->hw.idx, counter); + /* + * U32_MAX indicates a hardware or IPC read failure. Ignore the update + * to prevent spurious artificial counter spikes from underflows. + */ + if (new_count !=3D U32_MAX) { + local64_set(&event->hw.prev_count, new_count); + delta =3D (new_count - prev_count) & 0x7FFFFFFF; + local64_add(delta, &event->count); + } + } + + if (pmu->active_vpu_events > 0) + schedule_work(&pmu->vpu_work); + + hrtimer_forward_now(timer, RPI_AXI_PMU_TIMER_INTERVAL); + spin_unlock_irqrestore(&pmu->lock, flags); + return HRTIMER_RESTART; +} + +/** + * rpi_axi_pmu_start() - Starts monitoring on an allocated bus watcher + * @event: Pointer to perf_event being started + * @flags: Start control flags + */ +static void rpi_axi_pmu_start(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu =3D pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon =3D config_to_monitor(event->attr.config); + enum counter counter =3D config_to_counter(event->attr.config); + unsigned long spinflags; + + spin_lock_irqsave(&pmu->lock, spinflags); + event->hw.state =3D 0; + pmu->monitor[mon].hw_events.num_started++; + if (mon =3D=3D MON__SYSTEM) { + int bus =3D pmu->monitor[mon].hw_events.monitored_bus[event->hw.idx]; + int filter =3D pmu->monitor[mon].hw_events.filter[event->hw.idx]; + + rpi_axi_pmu_enable_bus_watcher(pmu, mon, event->hw.idx, bus, filter); + pmu->monitor[mon].hw_events.enabled[event->hw.idx] =3D true; + pmu->monitor[mon].hw_events.monitor_running =3D true; + local64_set(&event->hw.prev_count, + rpi_axi_pmu_read_counter(pmu, mon, event->hw.idx, counter)); + event->hw.state |=3D PERF_HES_UPTODATE; + } else if (mon =3D=3D MON__VPU) { + schedule_work(&pmu->vpu_work); + } + spin_unlock_irqrestore(&pmu->lock, spinflags); +} + +/** + * rpi_axi_pmu_add() - Allocates a bus watcher resource and adds event to = PMU + * @event: Pointer to perf_event being added + * @flags: Add control flags (e.g. PERF_EF_START) + * + * Return: 0 on success, -EAGAIN if no watcher slot is available. + */ +static int rpi_axi_pmu_add(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu =3D pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon =3D config_to_monitor(event->attr.config); + struct rpi_axi_hw_events *hw_events =3D &pmu->monitor[mon].hw_events; + unsigned long spinflags; + int idx, slot =3D -1; + + spin_lock_irqsave(&pmu->lock, spinflags); + idx =3D rpi_axi_hw_events__get_alloc_event_idx(hw_events, event); + if (idx < 0) { + spin_unlock_irqrestore(&pmu->lock, spinflags); + return -EAGAIN; + } + + for (int i =3D 0; i < RPI_AXI_MAX_EVENTS; i++) { + if (!pmu->events[i]) { + slot =3D i; + pmu->events[i] =3D event; + pmu->event_gen[i]++; + break; + } + } + + if (slot < 0) { + hw_events->refcount[idx]--; + if (hw_events->refcount[idx] =3D=3D 0) { + hw_events->monitored_bus[idx] =3D -1; + hw_events->filter[idx] =3D BCM2835_FLT__NONE; + hw_events->num_monitored--; + } + spin_unlock_irqrestore(&pmu->lock, spinflags); + return -ENOSPC; + } + + pmu->active_events++; + if (mon =3D=3D MON__VPU) + pmu->active_vpu_events++; + + if (pmu->active_events =3D=3D 1) + hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL, + HRTIMER_MODE_REL_SOFT); + + event->hw.idx =3D idx; + event->hw.state =3D PERF_HES_STOPPED; + spin_unlock_irqrestore(&pmu->lock, spinflags); + + if (flags & PERF_EF_START) + rpi_axi_pmu_start(event, /*flags=3D*/0); + return 0; +} + +/** + * rpi_axi_pmu_stop() - Stops counter monitoring for an event + * @event: Pointer to perf_event being stopped + * @flags: Stop control flags (e.g. PERF_EF_UPDATE) + */ +static void rpi_axi_pmu_stop(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu =3D pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon =3D config_to_monitor(event->attr.config); + unsigned long spinflags; + + if (event->hw.state & PERF_HES_STOPPED) + return; + + if (flags & PERF_EF_UPDATE) + rpi_axi_pmu_read(event); + + spin_lock_irqsave(&pmu->lock, spinflags); + if (flags & PERF_EF_UPDATE) + event->hw.state |=3D PERF_HES_UPTODATE; + event->hw.state |=3D PERF_HES_STOPPED; + pmu->monitor[mon].hw_events.num_started--; + spin_unlock_irqrestore(&pmu->lock, spinflags); +} + +/** + * rpi_axi_pmu_del() - Deallocates watcher resource and removes event from= PMU + * @event: Pointer to perf_event being removed + * @flags: Delete control flags + */ +static void rpi_axi_pmu_del(struct perf_event *event, int flags) +{ + struct rpi_axi_pmu *pmu =3D pmu_to_rpi_axi_pmu(event->pmu); + enum monitor mon =3D config_to_monitor(event->attr.config); + unsigned long spinflags; + int idx =3D event->hw.idx; + + if (idx < 0) + return; + + rpi_axi_pmu_stop(event, PERF_EF_UPDATE); + + spin_lock_irqsave(&pmu->lock, spinflags); + for (int i =3D 0; i < RPI_AXI_MAX_EVENTS; i++) { + if (pmu->events[i] =3D=3D event) { + pmu->events[i] =3D NULL; + pmu->event_gen[i]++; + break; + } + } + + if (pmu->monitor[mon].hw_events.monitored_bus[idx] >=3D 0) { + pmu->monitor[mon].hw_events.refcount[idx]--; + if (pmu->monitor[mon].hw_events.refcount[idx] =3D=3D 0) { + pmu->monitor[mon].hw_events.monitored_bus[idx] =3D -1; + pmu->monitor[mon].hw_events.filter[idx] =3D BCM2835_FLT__NONE; + pmu->monitor[mon].hw_events.num_monitored--; + if (mon =3D=3D MON__SYSTEM) { + pmu->monitor[MON__SYSTEM].hw_events.enabled[idx] =3D false; + rpi_axi_pmu_disable_bus_watcher(pmu, mon, idx); + if (pmu->monitor[mon].hw_events.num_monitored =3D=3D 0) { + set_monitor_control(pmu, mon, GEN_CTL_RESET_BIT); + pmu->monitor[mon].hw_events.monitor_running =3D false; + } + } else if (mon =3D=3D MON__VPU) { + pmu->monitor[MON__VPU].hw_events.enabled[idx] =3D false; + pmu->monitor[MON__VPU].hw_events.vpu_disable_pending[idx] =3D true; + schedule_work(&pmu->vpu_work); + } + } + } + + event->hw.idx =3D -1; + pmu->active_events--; + if (mon =3D=3D MON__VPU) + pmu->active_vpu_events--; + + if (pmu->active_events =3D=3D 0) { + spin_unlock_irqrestore(&pmu->lock, spinflags); + return; + } + spin_unlock_irqrestore(&pmu->lock, spinflags); +} + +/** + * rpi_axi_pmu_offline_cpu() - CPU hotplug notifier callback when designat= ed CPU goes offline + * @cpu: CPU core number being taken offline + * @node: Pointer to hlist_node inside struct rpi_axi_pmu + * + * Return: 0. + */ +static int rpi_axi_pmu_offline_cpu(unsigned int cpu, struct hlist_node *no= de) +{ + struct rpi_axi_pmu *pmu =3D hlist_entry_safe(node, struct rpi_axi_pmu, cp= uhp_node); + unsigned long spinflags; + unsigned int target; + + if (cpu !=3D pmu->cpu) + return 0; + + target =3D cpumask_any_but(cpu_online_mask, cpu); + if (target >=3D nr_cpu_ids) + return 0; + + perf_pmu_migrate_context(&pmu->pmu, cpu, target); + pmu->cpu =3D target; + + spin_lock_irqsave(&pmu->lock, spinflags); + if (pmu->active_events > 0) { + spin_unlock_irqrestore(&pmu->lock, spinflags); + hrtimer_cancel(&pmu->hrtimer); + spin_lock_irqsave(&pmu->lock, spinflags); + if (pmu->active_events > 0) + hrtimer_start(&pmu->hrtimer, RPI_AXI_PMU_TIMER_INTERVAL, + HRTIMER_MODE_REL_SOFT); + } + spin_unlock_irqrestore(&pmu->lock, spinflags); + + return 0; +} + +/** + * rpi_axi_pmu__init() - Internal PMU driver initialization called during = probe + * @pmu: Pointer to PMU context + * @pdev: Platform device pointer + * + * Return: 0 on success, negative error code on failure. + */ +static int rpi_axi_pmu__init(struct rpi_axi_pmu *pmu, struct platform_devi= ce *pdev) +{ + struct device *dev =3D &pdev->dev; + struct device_node *fw_node; + int ret; + + spin_lock_init(&pmu->lock); + mutex_init(&pmu->vpu_mutex); + + pmu->chip =3D (enum rpi_axi_chip)(uintptr_t)of_device_get_match_data(dev); + + pmu->pmu =3D (struct pmu) { + .module =3D THIS_MODULE, + .attr_groups =3D rpi_axi_pmu_attr_groups, + .task_ctx_nr =3D perf_invalid_context, + .event_init =3D rpi_axi_pmu_event_init, + .add =3D rpi_axi_pmu_add, + .del =3D rpi_axi_pmu_del, + .start =3D rpi_axi_pmu_start, + .stop =3D rpi_axi_pmu_stop, + .read =3D rpi_axi_pmu_read, + .capabilities =3D PERF_PMU_CAP_NO_EXCLUDE, + }; + pmu->pdev =3D pdev; + pmu->cpu =3D cpumask_first(cpu_online_mask); + hrtimer_setup(&pmu->hrtimer, rpi_axi_pmu_timer_handler, CLOCK_MONOTONIC, + HRTIMER_MODE_REL_SOFT); + INIT_WORK(&pmu->vpu_work, rpi_axi_pmu_vpu_work_handler); + + if (pmu->chip =3D=3D CHIP_BCM2835) { + pmu->monitor[MON__SYSTEM].use_mailbox_interface =3D false; + fw_node =3D of_parse_phandle(dev->of_node, "firmware", 0); + if (fw_node) { + pmu->firmware =3D rpi_firmware_get(fw_node); + of_node_put(fw_node); + } + if (!pmu->firmware) + pmu->monitor[MON__VPU].use_mailbox_interface =3D false; + else + pmu->monitor[MON__VPU].use_mailbox_interface =3D true; + } else { + pmu->monitor[MON__SYSTEM].use_mailbox_interface =3D false; + pmu->monitor[MON__VPU].use_mailbox_interface =3D false; + } + + for (int i =3D 0; i < MON__MAX; i++) { + rpi_axi_hw_events__init(&pmu->monitor[i].hw_events); + + if (pmu->monitor[i].use_mailbox_interface) { + struct resource *resource =3D platform_get_resource(pdev, IORESOURCE_ME= M, i); + + if (!resource) { + dev_err(dev, "Error reading mailbox resource %d\n", i); + ret =3D -EINVAL; + goto err_firmware_put; + } + pmu->monitor[i].mailbox =3D (u32)resource->start; + } else { + struct resource *resource =3D platform_get_resource(pdev, IORESOURCE_ME= M, i); + + if (!resource) + continue; + + pmu->monitor[i].base_address =3D devm_ioremap_resource(dev, resource); + if (IS_ERR(pmu->monitor[i].base_address)) { + ret =3D PTR_ERR(pmu->monitor[i].base_address); + dev_err(dev, "Error devm_ioremap_resource failed %d\n", ret); + goto err_firmware_put; + } + } + } + + ret =3D cpuhp_state_add_instance(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_nod= e); + if (ret) { + dev_err(dev, "Failed to add cpuhp instance %d\n", ret); + goto err_teardown; + } + + ret =3D perf_pmu_register(&pmu->pmu, PMU_NAME, /*type=3D*/-1); + if (ret) { + dev_err(dev, "PMU register failed %d\n", ret); + goto err_cpuhp_remove; + } + + return 0; + +err_cpuhp_remove: + cpuhp_state_remove_instance_nocalls(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_= node); +err_teardown: + hrtimer_cancel(&pmu->hrtimer); + flush_work(&pmu->vpu_work); +err_firmware_put: + if (pmu->firmware) + rpi_firmware_put(pmu->firmware); + return ret; +} + +/** + * rpi_axi_pmu__exit() - Internal PMU teardown called during remove + * @pmu: Pointer to PMU context + */ +static void rpi_axi_pmu__exit(struct rpi_axi_pmu *pmu) +{ + perf_pmu_unregister(&pmu->pmu); + cpuhp_state_remove_instance_nocalls(rpi_axi_pmu_cpuhp_state, &pmu->cpuhp_= node); + hrtimer_cancel(&pmu->hrtimer); + flush_work(&pmu->vpu_work); + if (pmu->firmware) + rpi_firmware_put(pmu->firmware); +} + +/* --- DRIVER ENTRY POINTS ----------------------------------------- */ + +/** + * rpi_axi_pmu_probe() - Platform driver probe entry point + * @pdev: Pointer to platform_device + * + * Return: 0 on success, negative error code on failure. + */ +static int rpi_axi_pmu_probe(struct platform_device *pdev) +{ + struct rpi_axi_pmu *pmu; + + pmu =3D devm_kzalloc(&pdev->dev, sizeof(*pmu), GFP_KERNEL); + if (!pmu) + return -ENOMEM; + + platform_set_drvdata(pdev, pmu); + return rpi_axi_pmu__init(pmu, pdev); +} + +/** + * rpi_axi_pmu_remove() - Platform driver remove entry point + * @pdev: Pointer to platform_device + */ +static void rpi_axi_pmu_remove(struct platform_device *pdev) +{ + struct rpi_axi_pmu *pmu; + + pmu =3D platform_get_drvdata(pdev); + rpi_axi_pmu__exit(pmu); +} + +/* Devices matching this driver in Device Tree */ +static const struct of_device_id rpi_axi_pmu_match[] =3D { + { + .compatible =3D "brcm,bcm2835-axiperf", + .data =3D (void *)CHIP_BCM2835, + }, + { + .compatible =3D "brcm,bcm2711-axiperf", + .data =3D (void *)CHIP_BCM2835, + }, + { } +}; +MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match); + +static struct platform_driver rpi_axi_pmu_driver =3D { + .probe =3D rpi_axi_pmu_probe, + .remove =3D rpi_axi_pmu_remove, + .driver =3D { + .name =3D PMU_NAME, + .of_match_table =3D of_match_ptr(rpi_axi_pmu_match), + .suppress_bind_attrs =3D true, + }, +}; + +/** + * rpi_axi_pmu_driver_init() - Module initialization entry point + * + * Registers the dynamic CPU hotplug state and the platform driver. + * + * Return: 0 on success, negative error code on failure. + */ +static int __init rpi_axi_pmu_driver_init(void) +{ + int ret; + + ret =3D cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN, "perf/rpi_axi_pmu:on= line", + NULL, rpi_axi_pmu_offline_cpu); + if (ret < 0) + return ret; + + rpi_axi_pmu_cpuhp_state =3D ret; + + ret =3D platform_driver_register(&rpi_axi_pmu_driver); + if (ret) + cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state); + + return ret; +} +module_init(rpi_axi_pmu_driver_init); + +/** + * rpi_axi_pmu_driver_exit() - Module cleanup entry point + * + * Unregisters the platform driver and CPU hotplug state. + */ +static void __exit rpi_axi_pmu_driver_exit(void) +{ + platform_driver_unregister(&rpi_axi_pmu_driver); + cpuhp_remove_multi_state(rpi_axi_pmu_cpuhp_state); +} +module_exit(rpi_axi_pmu_driver_exit); + +MODULE_LICENSE("GPL"); --=20 2.55.0.691.gc56d675ccc-goog From nobody Tue Sep 29 02:33:43 2026 Received: from mail-pg1-f199.google.com (mail-pg1-f199.google.com [209.85.215.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9781A3D1AA6 for ; Thu, 13 Aug 2026 05:24:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.199 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786598681; cv=none; b=Z5C8Ex0L9LlIECGNBFPwm3DjQ9X7Kp+IqJlMVPq7BUwDOdtbGNaeYxTHq6+365k/dzRPhGUNmD5LaNbHz0PIJQOJC8cDdDqvnykqy+4IbPuNqwRb9AtYe37OPDU8atKCWnOS8b37es9LOONgVc4BXTDvSKWWJURBpFEtK5DmE6c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1786598681; c=relaxed/simple; bh=C6T519CxdXc6NmXgLM+Mb6hJ4E0SAmAz01fF653GqCU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=AKMfK4qO56XLsgQgt/DwFPH/kq8YFkkoWWgN2hk0GwOujJBg26/3+/+VcMlmbNMPKo2CU3cmD2uRlbWxlj31pVliBVitc+oVEAV608aG8pUxW8Y/sngy6FSzoVKLCm5Nq2QLnNwFv9Q2OW6pf+3ZeCL1EqSIQk0mZbi2NKw8FMM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=vSEEmZ3l; arc=none smtp.client-ip=209.85.215.199 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--irogers.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="vSEEmZ3l" Received: by mail-pg1-f199.google.com with SMTP id 41be03b00d2f7-cbee5bab340so2019659a12.2 for ; Wed, 12 Aug 2026 22:24:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20251104; t=1786598666; x=1787203466; darn=vger.kernel.org; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:from:to:cc:subject:date:message-id:reply-to :content-type; bh=ydkjDPY8OIg1uOSSajI8hORDW4sYt4UTkqDjJueQMEE=; b=vSEEmZ3leL9h/xY/lhn7uuRDLKMnwaq6mUlO502zU4g67v89NYvk9lDDShmafus+sL 0pW1dGOvZihfIN1w3HV1SyeU60uPKsN73tNWQp/GQ5V9PNNKrIt5wGwOeq5m7F++PYh0 xRBZojlicbYjRg/mkAI0C7XfLWd8CuGFPsCo8D0m/UiNrmm3+P+/Og0n1vhjzaODtBcE Ifg5YJk2RG7Z8usl3Z320jnkmNc3FeRflUZI5PhHhvRZQahfM8A86ST6jlLllcKT64E9 U5gb0a9ufS+N9qx/NGhwinVBmeKREvki5VYCQ0Y5w1YBpdMDBQSQT2P8Z2k/Ll+CAaXg OKNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1786598666; x=1787203466; h=content-type:cc:to:from:subject:message-id:references:mime-version :in-reply-to:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to:content-type; bh=ydkjDPY8OIg1uOSSajI8hORDW4sYt4UTkqDjJueQMEE=; b=mGPs6EheYKqKEKgfqmQlK0NvDLL1Qm94lyt3Wy9c6C+jYpsOQZYmWbU+QeY24kOAPv sJD0P6v4idf5ZB59T5f93620QgUITl9lcySRXpi7/YTBSLKtg6BzlxElaj4zyq7j6fu6 sPmzmYeZZJSGHP45K4j1eAGAimD14rXFND6PyEwQ0/ogBxx1m3vwNSI02mGB0vlG61Ea jAvNHX4fjzsZ6vNViZXu+hQ90/JEsxpb1stwqlzp2ordbKCTl1Fp6VBaIBgMbhL2XLkW W+5f7b35KPn3OQqArFGw3QaoFeRvuMex7DlkEsq7U0gshBtr/lUjKtUu63KSHd63VLGA VpgQ== X-Forwarded-Encrypted: i=1; AHgh+Rqd5KuTB1NzNOS3BiJrTm6VXWW8aX3g5z7eFz8pjcZRyp0EJENffm3Cea0COYHbgnXjcDyscTJkpwnSm/U=@vger.kernel.org X-Gm-Message-State: AOJu0YzrZBAVjGQWbGNWjaqNut1P2fsh8ckRy0jHM8ttrHQtFez0l2Rr 2zApaenuDUZwSGy/Mx90La3rzYcWWJ3+oI5XPIbUy1xWdWgP9cXsfyRatOL+8t1kOHFqJfcIdyt Us6NnkIVgzA== X-Received: from dlbvs14.prod.google.com ([2002:a05:7022:3f0e:b0:13d:2928:7996]) (user=irogers job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6a21:4cc3:b0:3c3:7cfe:b32a with SMTP id adf61e73a8af0-3cc55484a40mr4993845637.27.1786598666065; Wed, 12 Aug 2026 22:24:26 -0700 (PDT) Date: Wed, 12 Aug 2026 22:24:18 -0700 In-Reply-To: <20260813052419.2770841-1-irogers@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20260813043533.2744442-1-irogers@google.com> <20260813052419.2770841-1-irogers@google.com> X-Mailer: git-send-email 2.55.0.691.gc56d675ccc-goog Message-ID: <20260813052419.2770841-3-irogers@google.com> Subject: [PATCH v7 2/2] perf: Add Raspberry Pi 5 (BCM2712) AXI PMU support From: Ian Rogers To: irogers@google.com, linux-perf-users@vger.kernel.org, linux-rpi-kernel@lists.infradead.org Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, mark.rutland@arm.com, u.kleine-koenig@baylibre.com, will@kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Signed-off-by: Ian Rogers --- drivers/perf/rpi_axi_pmu.c | 308 ++++++++++++++++++++++++++++++++++++- 1 file changed, 300 insertions(+), 8 deletions(-) diff --git a/drivers/perf/rpi_axi_pmu.c b/drivers/perf/rpi_axi_pmu.c index bd3c79dd2ff8..a843e8636d81 100644 --- a/drivers/perf/rpi_axi_pmu.c +++ b/drivers/perf/rpi_axi_pmu.c @@ -6,6 +6,7 @@ * This driver exposes the performance monitoring hardware on Raspberry Pi * System-on-Chips to the Linux perf subsystem: * - Raspberry Pi 1, 2, 3, 4, Compute Modules 1-4, Zero, Zero W (SoCs BCM2= 835/2836/2837/2711). + * - Raspberry Pi 5, Compute Module 5 (SoC BCM2712). * * Architecture Overview: * ---------------------- @@ -19,6 +20,7 @@ * Monitors VideoCore VPU buses (VPU0/1 Data/Instruction L2/UC, SDRAM, = etc.). * Accessible through VideoCore firmware mailbox IPC (RPI_FIRMWARE_SET/= GET_PERIPH_REG). * Read latency: ~10-100 microseconds (IPC over VPU mailbox). + * On BCM2712 (RPi 5), VPU Mailbox IPC is replaced by direct PCIe and M= MIO mapping. * * Synchronization & Concurrency Model: * ------------------------------------ @@ -72,9 +74,11 @@ static inline void rpi_hrtimer_setup(struct hrtimer *tim= er, /** * enum rpi_axi_chip - Supported Broadcom SoC generations * @CHIP_BCM2835: BCM2835 / BCM2836 / BCM2837 / BCM2711 (RPi 1-4, CM 1-4, = Zero/W) + * @CHIP_BCM2712: BCM2712 (RPi 5, CM 5) */ enum rpi_axi_chip { CHIP_BCM2835 =3D 0, + CHIP_BCM2712, }; =20 enum monitor { @@ -126,6 +130,43 @@ enum bcm2835_system_bus { BCM2835_SB__MAX }; =20 +/** + * enum bcm2712_system_bus - AXI buses monitored by System Monitor on BCM2= 712 (RPi 5) + * @BCM2712_SB__DMA_L2: DMA engine L2 cache interconnect bus + * @BCM2712_SB__TRANS: Transposer engine bus + * @BCM2712_SB__JPEG: Hardware JPEG codec acceleration bus + * @BCM2712_SB__SYSTEM_UC: System Uncached memory bus + * @BCM2712_SB__DMA_UC: DMA Uncached memory bus + * @BCM2712_SB__SYSTEM_L2: System main L2 cache bus + * @BCM2712_SB__PCIE_RP1: PCIe 2.0 x4 RP1 Southbridge link bus (USB 3.0, E= thernet, I/O) + * @BCM2712_SB__HEVC_DEC: HEVC (H.265) hardware video decoder bus + * @BCM2712_SB__A76_DSU_L3: Quad-core Arm Cortex-A76 DynamIQ Shared Unit (= DSU) L3 bus + * @BCM2712_SB__HVS: Hardware Video Scaler (HVS) display composition engin= e bus + * @BCM2712_SB__V3D7: VideoCore VII 3D graphics hardware pipeline bus + * @BCM2712_SB__ISP: Image Sensor Processor (ISP) camera pipeline bus + * @BCM2712_SB__PERIPHERAL: System peripherals bus + * @BCM2712_SB__CPU_UC: CPU Uncached memory bus + * @BCM2712_SB__CPU_L2: CPU L2 cache bus + * @BCM2712_SB__MAX: Total count of monitored system buses on BCM2712 + */ +enum bcm2712_system_bus { + BCM2712_SB__DMA_L2 =3D 0, + BCM2712_SB__TRANS, + BCM2712_SB__JPEG, + BCM2712_SB__SYSTEM_UC, + BCM2712_SB__DMA_UC, + BCM2712_SB__SYSTEM_L2, + BCM2712_SB__PCIE_RP1, + BCM2712_SB__HEVC_DEC, + BCM2712_SB__A76_DSU_L3, + BCM2712_SB__HVS, + BCM2712_SB__V3D7, + BCM2712_SB__ISP, + BCM2712_SB__PERIPHERAL, + BCM2712_SB__CPU_UC, + BCM2712_SB__CPU_L2, + BCM2712_SB__MAX +}; =20 /** * enum vpu_bus - AXI buses monitored by VPU Monitor on BCM2835-BCM2711 (R= Pi 1-4) @@ -259,6 +300,93 @@ enum bcm2835_filter { BCM2835_FLT__MAX }; =20 +/** + * enum bcm2712_filter - AXI master ID filter options for BCM2712 (RPi 5) + * @BCM2712_FLT__NONE: Disable master ID filtering (monitor all traffic on= bus) + * @BCM2712_FLT__VPU_UC0: VPU Uncached 0 master ID + * @BCM2712_FLT__VPU_IC0: VPU I-Cache 0 master ID + * @BCM2712_FLT__VPU_DC0: VPU D-Cache 0 master ID + * @BCM2712_FLT__VPU_UC1: VPU Uncached 1 master ID + * @BCM2712_FLT__VPU_IC1: VPU I-Cache 1 master ID + * @BCM2712_FLT__VPU_DC1: VPU D-Cache 1 master ID + * @BCM2712_FLT__VPU_L2: VPU L2 Cache master ID + * @BCM2712_FLT__DMA2: DMA2 master ID + * @BCM2712_FLT__VPU_DEBUG: VPU Debug master ID + * @BCM2712_FLT__ARM: Arm Quad-Core CPU Cluster master ID + * @BCM2712_FLT__DMA0: DMA0 master ID + * @BCM2712_FLT__DMA1: DMA1 master ID + * @BCM2712_FLT__RAAGA: RAAGA Audio Engine master ID + * @BCM2712_FLT__BBSI: BBSI master ID + * @BCM2712_FLT__PCIE0: RP1 PCIe Southbridge 0 master ID + * @BCM2712_FLT__PCIE1: PCIe 1 master ID + * @BCM2712_FLT__PCIE2: PCIe 2 master ID + * @BCM2712_FLT__UMR: UMR master ID + * @BCM2712_FLT__SAGE: SAGE master ID + * @BCM2712_FLT__HVDP: HVDP master ID + * @BCM2712_FLT__BSP: BSP master ID + * @BCM2712_FLT__HVS: Hardware Video Scaler (HVS) display engine master ID + * @BCM2712_FLT__HVS_WMK: HVS Watermark master ID + * @BCM2712_FLT__MOP0: MOP0 master ID + * @BCM2712_FLT__MOP1: MOP1 master ID + * @BCM2712_FLT__MBVN: MBVN master ID + * @BCM2712_FLT__DSI: DSI Display Interface master ID + * @BCM2712_FLT__XPT: XPT master ID + * @BCM2712_FLT__EMMC0: SD/eMMC Controller 0 master ID + * @BCM2712_FLT__GENET: Gigabit Ethernet Controller master ID + * @BCM2712_FLT__USB: USB Controller master ID + * @BCM2712_FLT__MAX: Maximum filter ID count for BCM2712 + */ +enum bcm2712_filter { + BCM2712_FLT__NONE =3D 0, + BCM2712_FLT__VPU_UC0 =3D 1, + BCM2712_FLT__VPU_IC0 =3D 2, + BCM2712_FLT__VPU_DC0 =3D 3, + BCM2712_FLT__VPU_UC1 =3D 4, + BCM2712_FLT__VPU_IC1 =3D 5, + BCM2712_FLT__VPU_DC1 =3D 6, + BCM2712_FLT__VPU_L2 =3D 7, + BCM2712_FLT__DMA2 =3D 8, + BCM2712_FLT__VPU_DEBUG =3D 9, + BCM2712_FLT__ARM =3D 10, + BCM2712_FLT__DMA0 =3D 11, + BCM2712_FLT__DMA1 =3D 12, + BCM2712_FLT__RAAGA =3D 13, + BCM2712_FLT__BBSI =3D 14, + BCM2712_FLT__PCIE0 =3D 15, + BCM2712_FLT__PCIE1 =3D 16, + BCM2712_FLT__PCIE2 =3D 17, + BCM2712_FLT__UMR =3D 18, + BCM2712_FLT__SAGE =3D 19, + BCM2712_FLT__HVDP =3D 20, + BCM2712_FLT__BSP =3D 21, + BCM2712_FLT__HVS =3D 22, + BCM2712_FLT__HVS_WMK =3D 23, + BCM2712_FLT__MOP0 =3D 24, + BCM2712_FLT__MOP1 =3D 25, + BCM2712_FLT__MBVN =3D 26, + BCM2712_FLT__DSI =3D 27, + BCM2712_FLT__XPT =3D 28, + BCM2712_FLT__EMMC0 =3D 29, + BCM2712_FLT__GENET =3D 30, + BCM2712_FLT__USB =3D 31, + BCM2712_FLT__MAX =3D 32 +}; + +/* Compile-time static assertions verifying cross-generation enum equivale= nce */ +static_assert((int)BCM2835_FLT__NONE =3D=3D (int)BCM2712_FLT__NONE, + "Filter NONE enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__DMA_L2 =3D=3D (int)BCM2712_SB__DMA_L2, + "DMA_L2 bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__TRANS =3D=3D (int)BCM2712_SB__TRANS, + "TRANS bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__JPEG =3D=3D (int)BCM2712_SB__JPEG, + "JPEG bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__SYSTEM_UC =3D=3D (int)BCM2712_SB__SYSTEM_UC, + "SYSTEM_UC bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__DMA_UC =3D=3D (int)BCM2712_SB__DMA_UC, + "DMA_UC bus enum value mismatch across BCM2835 and BCM2712"); +static_assert((int)BCM2835_SB__SYSTEM_L2 =3D=3D (int)BCM2712_SB__SYSTEM_L2, + "SYSTEM_L2 bus enum value mismatch across BCM2835 and BCM2712"); =20 /* Hardware register offsets & control bitwise constants */ #define GEN_CTRL 0x00 @@ -435,7 +563,7 @@ static int rpi_axi_hw_events__get_alloc_event_idx(struc= t rpi_axi_hw_events *hw_e * struct rpi_axi_pmu - Root PMU driver context * @pmu: Core Linux perf PMU structure * @pdev: Owning platform_device pointer - * @chip: Detected Broadcom SoC generation + * @chip: Detected Broadcom SoC generation (CHIP_BCM2835 or CHIP_BCM2712) * @firmware: Raspberry Pi firmware handle for VideoCore mailbox calls (BC= M2835-BCM2711) * @cpu: CPU core assigned to process uncore PMU events * @cpuhp_node: Dynamic CPU hotplug instance node @@ -495,15 +623,24 @@ static bool config_is_valid(struct rpi_axi_pmu *pmu, = __u64 config) if (!pmu->monitor[mon].use_mailbox_interface && !pmu->monitor[mon].base_a= ddress) return false; =20 - if (mon =3D=3D MON__SYSTEM) { - if (bus >=3D BCM2835_SB__MAX) + if (pmu->chip =3D=3D CHIP_BCM2712) { + if (mon !=3D MON__SYSTEM) + return false; + if (bus >=3D BCM2712_SB__MAX) + return false; + if (filter >=3D BCM2712_FLT__MAX) return false; } else { - if (bus >=3D VPU__MAX) + if (mon =3D=3D MON__SYSTEM) { + if (bus >=3D BCM2835_SB__MAX) + return false; + } else { + if (bus >=3D VPU__MAX) + return false; + } + if (filter >=3D BCM2835_FLT__MAX) return false; } - if (filter >=3D BCM2835_FLT__MAX) - return false; return counter < CNT__MAX; } =20 @@ -529,7 +666,7 @@ static const struct attribute_group rpi_axi_pmu_format_= group =3D { * Event Attribute Scaling & Unit Definitions: * * Uncore AXI bus transaction events (_rtrans, _wtrans, _atrans) report ha= rdware - * transaction beats. On Broadcom BCM2835 AXI interconnects, single-beat + * transaction beats. On Broadcom BCM2835-BCM2712 AXI interconnects, singl= e-beat * transactions transfer 32 bytes per beat, while contiguous DMA page burs= ts may * use 64-byte double-beats. Setting .scale=3D"32" and .unit=3D"Bytes" pro= vides a * close (~95%) byte throughput approximation in perf stat output (account= ing for @@ -867,6 +1004,75 @@ PMU_EVENT_ATTR_STRING(usb_system_l2_rtrans, rpi_axi_p= mu_event_usb_system_l2_rtra PMU_EVENT_ATTR_STRING(usb_system_l2_wtrans, rpi_axi_pmu_event_usb_system_l= 2_wtrans, "monitor=3D0,bus=3D5,counter=3D2,filter=3D24"); =20 +/* --- RASPBERRY PI 5 (BCM2712) EXPANDED EVENT ALIASES -------------------= --- */ + +/* PCIe 2.0 x4 RP1 Southbridge Link Bus Events & Stall Wait Cycles (bus=3D= 6, BCM2712_SB__PCIE_RP1) */ +PMU_EVENT_ATTR_STRING(pcie_rp1_atrans, rpi_axi_pmu_event_pcie_rp1_atrans, + "monitor=3D0,bus=3D6,counter=3D0"); +PMU_EVENT_ATTR_STRING(pcie_rp1_atwait, rpi_axi_pmu_event_pcie_rp1_atwait, + "monitor=3D0,bus=3D6,counter=3D1"); +PMU_EVENT_ATTR_STRING(pcie_rp1_wtrans, rpi_axi_pmu_event_pcie_rp1_wtrans, + "monitor=3D0,bus=3D6,counter=3D2"); +PMU_EVENT_ATTR_STRING(pcie_rp1_wtwait, rpi_axi_pmu_event_pcie_rp1_wtwait, + "monitor=3D0,bus=3D6,counter=3D3"); +PMU_EVENT_ATTR_STRING(pcie_rp1_rtrans, rpi_axi_pmu_event_pcie_rp1_rtrans, + "monitor=3D0,bus=3D6,counter=3D4"); +PMU_EVENT_ATTR_STRING(pcie_rp1_rtwait, rpi_axi_pmu_event_pcie_rp1_rtwait, + "monitor=3D0,bus=3D6,counter=3D5"); + +/* HEVC (H.265) Hardware Video Decoder Bus Events (bus=3D7, BCM2712_SB__HE= VC_DEC) */ +PMU_EVENT_ATTR_STRING(hevc_dec_atrans, rpi_axi_pmu_event_hevc_dec_atrans, + "monitor=3D0,bus=3D7,counter=3D0"); +PMU_EVENT_ATTR_STRING(hevc_dec_wtrans, rpi_axi_pmu_event_hevc_dec_wtrans, + "monitor=3D0,bus=3D7,counter=3D2"); +PMU_EVENT_ATTR_STRING(hevc_dec_rtrans, rpi_axi_pmu_event_hevc_dec_rtrans, + "monitor=3D0,bus=3D7,counter=3D4"); + +/* Arm Cortex-A76 DSU L3 Interconnect Bus Events (bus=3D8, BCM2712_SB__A76= _DSU_L3) */ +PMU_EVENT_ATTR_STRING(a76_dsu_l3_atrans, rpi_axi_pmu_event_a76_dsu_l3_atra= ns, + "monitor=3D0,bus=3D8,counter=3D0"); +PMU_EVENT_ATTR_STRING(a76_dsu_l3_wtrans, rpi_axi_pmu_event_a76_dsu_l3_wtra= ns, + "monitor=3D0,bus=3D8,counter=3D2"); +PMU_EVENT_ATTR_STRING(a76_dsu_l3_rtrans, rpi_axi_pmu_event_a76_dsu_l3_rtra= ns, + "monitor=3D0,bus=3D8,counter=3D4"); + +/* VideoCore VII 3D Graphics Pipeline Bus Events (bus=3D10, BCM2712_SB__V3= D7) */ +PMU_EVENT_ATTR_STRING(v3d7_atrans, rpi_axi_pmu_event_v3d7_atrans, + "monitor=3D0,bus=3D10,counter=3D0"); +PMU_EVENT_ATTR_STRING(v3d7_wtrans, rpi_axi_pmu_event_v3d7_wtrans, + "monitor=3D0,bus=3D10,counter=3D2"); +PMU_EVENT_ATTR_STRING(v3d7_rtrans, rpi_axi_pmu_event_v3d7_rtrans, + "monitor=3D0,bus=3D10,counter=3D4"); + +/* + * Quad-Core Arm Cortex-A76 Cores 0-3 I-Cache & D-Cache Filtered Events + * (bus=3D14, BCM2712_SB__CPU_L2) + */ +PMU_EVENT_ATTR_STRING(bcm2712_peripheral_rtrans, rpi_axi_pmu_event_bcm2712= _peripheral_rtrans, + "monitor=3D0,bus=3D12,counter=3D4"); +PMU_EVENT_ATTR_STRING(bcm2712_peripheral_wtrans, rpi_axi_pmu_event_bcm2712= _peripheral_wtrans, + "monitor=3D0,bus=3D12,counter=3D2"); +PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_rtrans, rpi_axi_pmu_event_bcm2712_cpu= _uc_rtrans, + "monitor=3D0,bus=3D13,counter=3D4"); +PMU_EVENT_ATTR_STRING(bcm2712_cpu_uc_wtrans, rpi_axi_pmu_event_bcm2712_cpu= _uc_wtrans, + "monitor=3D0,bus=3D13,counter=3D2"); + +/* BCM2712 Master ID Filtered Events */ +PMU_EVENT_ATTR_STRING(arm_rtrans, rpi_axi_pmu_event_arm_rtrans, + "monitor=3D0,bus=3D14,counter=3D4,filter=3D10"); +PMU_EVENT_ATTR_STRING(arm_wtrans, rpi_axi_pmu_event_arm_wtrans, + "monitor=3D0,bus=3D14,counter=3D2,filter=3D10"); +PMU_EVENT_ATTR_STRING(pcie0_rtrans, rpi_axi_pmu_event_pcie0_rtrans, + "monitor=3D0,bus=3D6,counter=3D4,filter=3D15"); +PMU_EVENT_ATTR_STRING(pcie0_wtrans, rpi_axi_pmu_event_pcie0_wtrans, + "monitor=3D0,bus=3D6,counter=3D2,filter=3D15"); +PMU_EVENT_ATTR_STRING(hvs_filtered_rtrans, rpi_axi_pmu_event_hvs_filtered_= rtrans, + "monitor=3D0,bus=3D9,counter=3D4,filter=3D22"); +PMU_EVENT_ATTR_STRING(emmc0_rtrans, rpi_axi_pmu_event_emmc0_rtrans, + "monitor=3D0,bus=3D5,counter=3D4,filter=3D29"); +PMU_EVENT_ATTR_STRING(emmc0_wtrans, rpi_axi_pmu_event_emmc0_wtrans, + "monitor=3D0,bus=3D5,counter=3D2,filter=3D29"); + static struct attribute *rpi_axi_pmu_events_attrs[] =3D { /* System Monitor events */ &rpi_axi_pmu_event_dma_l2_atrans.attr.attr, @@ -1024,6 +1230,38 @@ static struct attribute *rpi_axi_pmu_events_attrs[] = =3D { &rpi_axi_pmu_event_usb_system_l2_rtrans.attr.attr, &rpi_axi_pmu_event_usb_system_l2_wtrans.attr.attr, =20 + /* RPi 5 (BCM2712) Expanded Event Aliases */ + &rpi_axi_pmu_event_pcie_rp1_atrans.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_atwait.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_wtrans.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_wtwait.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_rtrans.attr.attr, + &rpi_axi_pmu_event_pcie_rp1_rtwait.attr.attr, + + &rpi_axi_pmu_event_hevc_dec_atrans.attr.attr, + &rpi_axi_pmu_event_hevc_dec_wtrans.attr.attr, + &rpi_axi_pmu_event_hevc_dec_rtrans.attr.attr, + + &rpi_axi_pmu_event_a76_dsu_l3_atrans.attr.attr, + &rpi_axi_pmu_event_a76_dsu_l3_wtrans.attr.attr, + &rpi_axi_pmu_event_a76_dsu_l3_rtrans.attr.attr, + + &rpi_axi_pmu_event_v3d7_atrans.attr.attr, + &rpi_axi_pmu_event_v3d7_wtrans.attr.attr, + &rpi_axi_pmu_event_v3d7_rtrans.attr.attr, + + &rpi_axi_pmu_event_bcm2712_peripheral_rtrans.attr.attr, + &rpi_axi_pmu_event_bcm2712_peripheral_wtrans.attr.attr, + &rpi_axi_pmu_event_bcm2712_cpu_uc_rtrans.attr.attr, + &rpi_axi_pmu_event_bcm2712_cpu_uc_wtrans.attr.attr, + + &rpi_axi_pmu_event_arm_rtrans.attr.attr, + &rpi_axi_pmu_event_arm_wtrans.attr.attr, + &rpi_axi_pmu_event_pcie0_rtrans.attr.attr, + &rpi_axi_pmu_event_pcie0_wtrans.attr.attr, + &rpi_axi_pmu_event_hvs_filtered_rtrans.attr.attr, + &rpi_axi_pmu_event_emmc0_rtrans.attr.attr, + &rpi_axi_pmu_event_emmc0_wtrans.attr.attr, NULL, }; =20 @@ -1034,12 +1272,62 @@ static struct attribute *rpi_axi_pmu_events_attrs[]= =3D { * @unused: Index (unused) * * Dynamically filters sysfs event aliases based on the detected Broadcom = SoC generation: + * - On BCM2712 (RPi 5): Hides VideoCore VPU Mailbox IPC events (monitor= =3D1). + * - On BCM2835-BCM2711 (RPi 1-4): Hides Cortex-A76 and RP1 PCIe specific = events. * * Return: attr->mode (0444) if visible on current SoC, 0 to hide. */ static umode_t rpi_axi_pmu_events_is_visible(struct kobject *kobj, struct attribute *attr, int unused) { + struct device *dev =3D kobj_to_dev(kobj); + struct pmu *pmu =3D dev_get_drvdata(dev); + struct rpi_axi_pmu *rpi_pmu =3D pmu_to_rpi_axi_pmu(pmu); + struct perf_pmu_events_attr *pmu_attr; + + pmu_attr =3D container_of(attr, struct perf_pmu_events_attr, attr.attr); + + if (rpi_pmu->chip =3D=3D CHIP_BCM2712) { + /* + * On RPi 5 (BCM2712), hide legacy VPU Mailbox IPC events (monitor=3D1) + * and legacy RPi 1-4 System Monitor aliases + */ + if (strstr(pmu_attr->event_str, "monitor=3D1") || + strstr(attr->name, "cpu0_") || + strstr(attr->name, "cpu1_") || + strstr(attr->name, "dma0_") || + strstr(attr->name, "dma1_") || + strstr(attr->name, "v3d0_") || + strstr(attr->name, "v3d1_") || + strstr(attr->name, "hvs_system") || + strstr(attr->name, "isp_system") || + strstr(attr->name, "usb_system") || + strstr(attr->name, "ccp2tx_") || + strstr(attr->name, "mphi_") || + strstr(attr->name, "h264_") || + strstr(attr->name, "v3d_") || + (strstr(attr->name, "peripheral_") && !strstr(attr->name, "bcm2712_"= )) || + (strstr(attr->name, "cpu_uc_") && !strstr(attr->name, "bcm2712_")) || + strstr(attr->name, "cpu_l2_") || + strstr(attr->name, "l2_control_") || + strstr(attr->name, "system_control_") || + strstr(attr->name, "direct_control_") || + strstr(attr->name, "direct_data_")) + return 0; + } else { + /* On RPi 1-4 (BCM2835-BCM2711), hide RPi 5 specific events */ + if (strstr(attr->name, "pcie_rp1") || + strstr(attr->name, "a76_") || + strstr(attr->name, "v3d7_") || + strstr(attr->name, "hevc_dec") || + strstr(attr->name, "arm_") || + strstr(attr->name, "pcie0_") || + strstr(attr->name, "hvs_filtered") || + strstr(attr->name, "emmc0_") || + strstr(attr->name, "bcm2712_")) + return 0; + } + return attr->mode; } =20 @@ -1370,7 +1658,7 @@ static void rpi_axi_pmu_read(struct perf_event *event) u32 delta; =20 /* Mailbox VPU counters are polled asynchronously in background vpu_work. - * MMIO monitors (System) are read synchronously. + * MMIO monitors (System and BCM2712 VPU) are read synchronously. */ if (event->hw.idx < 0) return; @@ -1938,6 +2226,10 @@ static const struct of_device_id rpi_axi_pmu_match[]= =3D { .compatible =3D "brcm,bcm2711-axiperf", .data =3D (void *)CHIP_BCM2835, }, + { + .compatible =3D "brcm,bcm2712-axiperf", + .data =3D (void *)CHIP_BCM2712, + }, { } }; MODULE_DEVICE_TABLE(of, rpi_axi_pmu_match); --=20 2.55.0.691.gc56d675ccc-goog