From nobody Sun Jun 14 21:07:05 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 004F2C433F5 for ; Wed, 11 May 2022 14:25:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244587AbiEKOZg (ORCPT ); Wed, 11 May 2022 10:25:36 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53222 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242913AbiEKOZV (ORCPT ); Wed, 11 May 2022 10:25:21 -0400 Received: from desiato.infradead.org (desiato.infradead.org [IPv6:2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 8D06A40A3B for ; Wed, 11 May 2022 07:25:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=fzXxRG6OXG8jw/YQezAyrq7G35GCWfwkvISstTTxtus=; b=BdcmOLm2rz344t6tbIbh6CFOC8 AvsfNgbdPXnGb5G9LrIlEaTB17d8qe9Y21KRpRS0ER9wTdMjrs/EOKmspYoQXKiJ66mcQaxyYL8+B LFAyt53fU1q2GR6wZmSyAjC4w9gtoFbuvvwkHuZ44b3Ik5o41uuua7Nrt5cUYunu8mcSVIz/Y3uUP nhUvO9PlZigtTlQVTOpvIeV5ak5psuqVw2stonZVGkVmqrWsW/qejQgVUM5kqXa1C9v3yHeK1L95u r6XEeDmRNUkz5sFv5nKfo4W0Kmq01GBwdU0hYhWXfg1YKoVgyoSYYUGgNzhfje5KZFCQhaFpUh2d3 /eGXE/3A==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1nonH5-00DBu4-1R; Wed, 11 May 2022 14:25:11 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 3454430047E; Wed, 11 May 2022 16:25:09 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 1E5D920236A42; Wed, 11 May 2022 16:25:09 +0200 (CEST) Message-ID: <20220511142345.084235472@infradead.org> User-Agent: quilt/0.66 Date: Wed, 11 May 2022 16:20:38 +0200 From: Peter Zijlstra To: x86@kernel.org, kan.liang@linux.intel.com, eranian@google.com Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, acme@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, namhyung@kernel.org Subject: [PATCH 1/5] perf/x86/amd: Fix AMD BRS period adjustment References: <20220511142037.353492804@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" There's two problems with the current amd_brs_adjust_period() code: - it isn't in fact AMD specific and wil always adjust the period; - it adjusts the period, while it should only adjust the event count, resulting in repoting a short period. Fix this by using x86_pmu.limit_period, this makes it specific to the AMD BRS case and ensures only the event count is adjusted while the reported period is unmodified. Fixes: ba2fe7500845 ("perf/x86/amd: Add AMD branch sampling period adjustme= nt") Signed-off-by: Peter Zijlstra (Intel) --- arch/x86/events/amd/core.c | 13 +++++++++++++ arch/x86/events/core.c | 7 ------- arch/x86/events/perf_event.h | 18 ------------------ 3 files changed, 13 insertions(+), 25 deletions(-) --- a/arch/x86/events/amd/core.c +++ b/arch/x86/events/amd/core.c @@ -1258,6 +1258,18 @@ static void amd_pmu_sched_task(struct pe amd_pmu_brs_sched_task(ctx, sched_in); } =20 +static u64 amd_pmu_limit_period(struct perf_event *event, u64 left) +{ + /* + * Decrease period by the depth of the BRS feature to get the last N + * taken branches and approximate the desired period + */ + if (has_branch_stack(event) && left > x86_pmu.lbr_nr) + left -=3D x86_pmu.lbr_nr; + + return left; +} + static __initconst const struct x86_pmu amd_pmu =3D { .name =3D "AMD", .handle_irq =3D amd_pmu_handle_irq, @@ -1418,6 +1430,7 @@ static int __init amd_core_pmu_init(void if (boot_cpu_data.x86 >=3D 0x19 && !amd_brs_init()) { x86_pmu.get_event_constraints =3D amd_get_event_constraints_f19h; x86_pmu.sched_task =3D amd_pmu_sched_task; + x86_pmu.limit_period =3D amd_pmu_limit_period; /* * put_event_constraints callback same as Fam17h, set above */ --- a/arch/x86/events/core.c +++ b/arch/x86/events/core.c @@ -1375,13 +1375,6 @@ int x86_perf_event_set_period(struct per return x86_pmu.set_topdown_event_period(event); =20 /* - * decrease period by the depth of the BRS feature to get - * the last N taken branches and approximate the desired period - */ - if (has_branch_stack(event)) - period =3D amd_brs_adjust_period(period); - - /* * If we are way outside a reasonable range then just skip forward: */ if (unlikely(left <=3D -period)) { --- a/arch/x86/events/perf_event.h +++ b/arch/x86/events/perf_event.h @@ -1254,14 +1254,6 @@ static inline void amd_pmu_brs_del(struc } =20 void amd_pmu_brs_sched_task(struct perf_event_context *ctx, bool sched_in); - -static inline s64 amd_brs_adjust_period(s64 period) -{ - if (period > x86_pmu.lbr_nr) - return period - x86_pmu.lbr_nr; - - return period; -} #else static inline int amd_brs_init(void) { @@ -1290,11 +1282,6 @@ static inline void amd_pmu_brs_sched_tas { } =20 -static inline s64 amd_brs_adjust_period(s64 period) -{ - return period; -} - static inline void amd_brs_enable_all(void) { } @@ -1324,11 +1311,6 @@ static inline void amd_brs_enable_all(vo static inline void amd_brs_disable_all(void) { } - -static inline s64 amd_brs_adjust_period(s64 period) -{ - return period; -} #endif /* CONFIG_CPU_SUP_AMD */ =20 static inline int is_pebs_pt(struct perf_event *event) From nobody Sun Jun 14 21:07:05 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A05E8C433EF for ; Wed, 11 May 2022 14:25:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244589AbiEKOZm (ORCPT ); Wed, 11 May 2022 10:25:42 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53232 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244560AbiEKOZV (ORCPT ); Wed, 11 May 2022 10:25:21 -0400 Received: from desiato.infradead.org (desiato.infradead.org [IPv6:2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 07F9F633A0 for ; Wed, 11 May 2022 07:25:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=69XAIoGDwel9FBx285nF3+WxoH4y8KYpY4nBuO7IJh8=; b=Ih99V/BJlIdC9sEbXKvAS+KVKa niWZ3kd8r8UynTXMsHglgGBNYaVYw0KfvDk0n4WrpYbZM0tCJhgs4B3M8lbeGXmB0nW8sBqCaNwRO sJfJEXF6/hHt5MninTZuIX/eRvfoXmGC6yLDfVGcjERDhNvyoSBlBCo6aRkRKH3grZoGN07X5pTNc oHgjfoh8+xtW7NvS8b4gfudf9+mq875JSCiRxqzOIVKrM3OHqOUNxDciXzynLrph8o2+HDoToy57r aqTMnbh0F9ppSjqXb5VR/382NHAmqQ2g+IWxmCcrEVX4oJkv27pKtTfYWtF1dhMLMgublJ2BY+5Py LXA2d6BQ==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1nonH5-00DBu5-0m; Wed, 11 May 2022 14:25:11 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 37B993005B9; Wed, 11 May 2022 16:25:09 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 224C4201E6488; Wed, 11 May 2022 16:25:09 +0200 (CEST) Message-ID: <20220511142345.146224050@infradead.org> User-Agent: quilt/0.66 Date: Wed, 11 May 2022 16:20:39 +0200 From: Peter Zijlstra To: x86@kernel.org, kan.liang@linux.intel.com, eranian@google.com Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, acme@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, namhyung@kernel.org Subject: [PATCH 2/5] perf/x86: Add two more x86_pmu methods References: <20220511142037.353492804@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In order to clean up x86_perf_event_{set_period,update)() start by adding them as x86_pmu methods. Signed-off-by: Peter Zijlstra (Intel) --- arch/x86/events/core.c | 22 +++++++++++++++++----- arch/x86/events/perf_event.h | 5 +++++ 2 files changed, 22 insertions(+), 5 deletions(-) --- a/arch/x86/events/core.c +++ b/arch/x86/events/core.c @@ -72,6 +72,9 @@ DEFINE_STATIC_CALL_NULL(x86_pmu_add, *x DEFINE_STATIC_CALL_NULL(x86_pmu_del, *x86_pmu.del); DEFINE_STATIC_CALL_NULL(x86_pmu_read, *x86_pmu.read); =20 +DEFINE_STATIC_CALL_NULL(x86_pmu_set_period, *x86_pmu.set_period); +DEFINE_STATIC_CALL_NULL(x86_pmu_update, *x86_pmu.update); + DEFINE_STATIC_CALL_NULL(x86_pmu_schedule_events, *x86_pmu.schedule_e= vents); DEFINE_STATIC_CALL_NULL(x86_pmu_get_event_constraints, *x86_pmu.get_event_= constraints); DEFINE_STATIC_CALL_NULL(x86_pmu_put_event_constraints, *x86_pmu.put_event_= constraints); @@ -1518,7 +1521,7 @@ static void x86_pmu_start(struct perf_ev =20 if (flags & PERF_EF_RELOAD) { WARN_ON_ONCE(!(event->hw.state & PERF_HES_UPTODATE)); - x86_perf_event_set_period(event); + static_call(x86_pmu_set_period)(event); } =20 event->hw.state =3D 0; @@ -1610,7 +1613,7 @@ void x86_pmu_stop(struct perf_event *eve * Drain the remaining delta count out of a event * that we are disabling: */ - x86_perf_event_update(event); + static_call(x86_pmu_update)(event); hwc->state |=3D PERF_HES_UPTODATE; } } @@ -1700,7 +1703,7 @@ int x86_pmu_handle_irq(struct pt_regs *r =20 event =3D cpuc->events[idx]; =20 - val =3D x86_perf_event_update(event); + val =3D static_call(x86_pmu_update)(event); if (val & (1ULL << (x86_pmu.cntval_bits - 1))) continue; =20 @@ -1709,7 +1712,7 @@ int x86_pmu_handle_irq(struct pt_regs *r */ handled++; =20 - if (!x86_perf_event_set_period(event)) + if (!static_call(x86_pmu_set_period)(event)) continue; =20 perf_sample_data_init(&data, 0, event->hw.last_period); @@ -2023,6 +2026,9 @@ static void x86_pmu_static_call_update(v static_call_update(x86_pmu_del, x86_pmu.del); static_call_update(x86_pmu_read, x86_pmu.read); =20 + static_call_update(x86_pmu_set_period, x86_pmu.set_period); + static_call_update(x86_pmu_update, x86_pmu.update); + static_call_update(x86_pmu_schedule_events, x86_pmu.schedule_events); static_call_update(x86_pmu_get_event_constraints, x86_pmu.get_event_const= raints); static_call_update(x86_pmu_put_event_constraints, x86_pmu.put_event_const= raints); @@ -2042,7 +2048,7 @@ static void x86_pmu_static_call_update(v =20 static void _x86_pmu_read(struct perf_event *event) { - x86_perf_event_update(event); + static_call(x86_pmu_update)(event); } =20 void x86_pmu_show_pmu_cap(int num_counters, int num_counters_fixed, @@ -2148,6 +2154,12 @@ static int __init init_hw_perf_events(vo if (!x86_pmu.guest_get_msrs) x86_pmu.guest_get_msrs =3D (void *)&__static_call_return0; =20 + if (!x86_pmu.set_period) + x86_pmu.set_period =3D x86_perf_event_set_period; + + if (!x86_pmu.update) + x86_pmu.update =3D x86_perf_event_update; + x86_pmu_static_call_update(); =20 /* --- a/arch/x86/events/perf_event.h +++ b/arch/x86/events/perf_event.h @@ -735,6 +735,8 @@ struct x86_pmu { void (*add)(struct perf_event *); void (*del)(struct perf_event *); void (*read)(struct perf_event *event); + int (*set_period)(struct perf_event *event); + u64 (*update)(struct perf_event *event); int (*hw_config)(struct perf_event *event); int (*schedule_events)(struct cpu_hw_events *cpuc, int n, int *assign); unsigned eventsel; @@ -1031,6 +1033,9 @@ static struct perf_pmu_format_hybrid_att struct pmu *x86_get_pmu(unsigned int cpu); extern struct x86_pmu x86_pmu __read_mostly; =20 +DECLARE_STATIC_CALL(x86_pmu_set_period, *x86_pmu.set_period); +DECLARE_STATIC_CALL(x86_pmu_update, *x86_pmu.update); + static __always_inline struct x86_perf_task_context_opt *task_context_opt(= void *ctx) { if (static_cpu_has(X86_FEATURE_ARCH_LBR)) From nobody Sun Jun 14 21:07:05 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 58557C433EF for ; Wed, 11 May 2022 14:25:31 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244579AbiEKOZ2 (ORCPT ); Wed, 11 May 2022 10:25:28 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53196 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S242047AbiEKOZT (ORCPT ); Wed, 11 May 2022 10:25:19 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EB62653E3C for ; Wed, 11 May 2022 07:25:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=6ox9SsphMCojor9sb9ByAEKufXYN5OOq6Tkw9aA9+6M=; b=G56+Su1vl2cHiOT37LoU1/eZPA 1THQ7HDyWKaAeDxgzf2FkRqTP7RGgnohQrgDPQU8Dbu3lAMaveoD5ksSlIyQ1kpe6zMXyYGRx2F30 z6XEDB0qlTnh4CvcK3F1kbRxc8he94sGX9Yl4XwSXFNNtGSV9WK9s46rn3TjaS8R/TEjyfZlLGp/U sPibh9LKH30r4JuaPuQbOsteocnWM6OAQPiCg7yDTsyI8ALXbAil9jIZjWNH4C8/Lwylw8Gs1nEVZ 0LvQOkhK2wC/rYSgCITIl5xo/fWi/pLReL5KkVJaOuHGoRwYNlDKJ2UPl0RjH0Gd+z+/cTsv00qKV OT83n4lg==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1nonH5-005VvH-2N; Wed, 11 May 2022 14:25:11 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 3A80F300750; Wed, 11 May 2022 16:25:09 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 25052203BF8F2; Wed, 11 May 2022 16:25:09 +0200 (CEST) Message-ID: <20220511142345.209020116@infradead.org> User-Agent: quilt/0.66 Date: Wed, 11 May 2022 16:20:40 +0200 From: Peter Zijlstra To: x86@kernel.org, kan.liang@linux.intel.com, eranian@google.com Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, acme@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, namhyung@kernel.org Subject: [PATCH 3/5] perf/x86/intel: Move the topdown stuff into the intel driver References: <20220511142037.353492804@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Use the new x86_pmu::{set_period,update}() methods to push the topdown stuff into the Intel driver, where it belongs. Signed-off-by: Peter Zijlstra (Intel) --- arch/x86/events/core.c | 7 ------- arch/x86/events/intel/core.c | 28 +++++++++++++++++++++++++--- 2 files changed, 25 insertions(+), 10 deletions(-) --- a/arch/x86/events/core.c +++ b/arch/x86/events/core.c @@ -119,9 +119,6 @@ u64 x86_perf_event_update(struct perf_ev if (unlikely(!hwc->event_base)) return 0; =20 - if (unlikely(is_topdown_count(event)) && x86_pmu.update_topdown_event) - return x86_pmu.update_topdown_event(event); - /* * Careful: an NMI might modify the previous event value. * @@ -1373,10 +1370,6 @@ int x86_perf_event_set_period(struct per if (unlikely(!hwc->event_base)) return 0; =20 - if (unlikely(is_topdown_count(event)) && - x86_pmu.set_topdown_event_period) - return x86_pmu.set_topdown_event_period(event); - /* * If we are way outside a reasonable range then just skip forward: */ --- a/arch/x86/events/intel/core.c +++ b/arch/x86/events/intel/core.c @@ -2301,7 +2301,7 @@ static void intel_pmu_nhm_workaround(voi for (i =3D 0; i < 4; i++) { event =3D cpuc->events[i]; if (event) - x86_perf_event_update(event); + static_call(x86_pmu_update)(event); } =20 for (i =3D 0; i < 4; i++) { @@ -2316,7 +2316,7 @@ static void intel_pmu_nhm_workaround(voi event =3D cpuc->events[i]; =20 if (event) { - x86_perf_event_set_period(event); + static_call(x86_pmu_set_period)(event); __x86_pmu_enable_event(&event->hw, ARCH_PERFMON_EVENTSEL_ENABLE); } else @@ -2793,7 +2793,7 @@ static void intel_pmu_add_event(struct p */ int intel_pmu_save_and_restart(struct perf_event *event) { - x86_perf_event_update(event); + static_call(x86_pmu_update)(event); /* * For a checkpointed counter always reset back to 0. This * avoids a situation where the counter overflows, aborts the @@ -2805,9 +2805,27 @@ int intel_pmu_save_and_restart(struct pe wrmsrl(event->hw.event_base, 0); local64_set(&event->hw.prev_count, 0); } + return static_call(x86_pmu_set_period)(event); +} + +static int intel_pmu_set_period(struct perf_event *event) +{ + if (unlikely(is_topdown_count(event)) && + x86_pmu.set_topdown_event_period) + return x86_pmu.set_topdown_event_period(event); + return x86_perf_event_set_period(event); } =20 +static u64 intel_pmu_update(struct perf_event *event) +{ + if (unlikely(is_topdown_count(event)) && + x86_pmu.update_topdown_event) + return x86_pmu.update_topdown_event(event); + + return x86_perf_event_update(event); +} + static void intel_pmu_reset(void) { struct debug_store *ds =3D __this_cpu_read(cpu_hw_events.ds); @@ -4635,6 +4653,10 @@ static __initconst const struct x86_pmu .enable_all =3D core_pmu_enable_all, .enable =3D core_pmu_enable_event, .disable =3D x86_pmu_disable_event, + + .set_period =3D intel_pmu_set_period, + .update =3D intel_pmu_update, + .hw_config =3D core_pmu_hw_config, .schedule_events =3D x86_schedule_events, .eventsel =3D MSR_ARCH_PERFMON_EVENTSEL0, From nobody Sun Jun 14 21:07:05 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6A170C433EF for ; Wed, 11 May 2022 14:25:25 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244568AbiEKOZX (ORCPT ); Wed, 11 May 2022 10:25:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53194 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229790AbiEKOZT (ORCPT ); Wed, 11 May 2022 10:25:19 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E9BD640A3B for ; Wed, 11 May 2022 07:25:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=rLl5J2DLGbXLQLGLVKmwEFUM1mLMY8M0xRVjlffGnsU=; b=q9t0EthDCuaahUXWGXSW15D4rB Nx39cShAfuD3IONFAluhd2In0ym90Ye6I9rTyDBakNP4GacdJGA1ARxYA3D3smwR2F8aIzR1CshR9 lN29Znudh3Zhg9X9MY87VoOl06vPZSsXftzubIjR+i3kSfShSUKwR84lYsermWQk0SmMVlJYty2X7 HXN7V+/GykB+K5KkQPYjDugFq2ys4E262veU4867ihtjESl+XoXxhGBIyIYqL/j0WsZhFMfzZon2S Lw0hEfLDeOxgovZXnqMM3sSCBCQvkskTg/KsOdZc0LA7ii1fQ4Q3BUEqv815uzIG4UZSpEBi9e5/+ lhnyXCHw==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1nonH5-005VvF-21; Wed, 11 May 2022 14:25:11 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 45E2D30080B; Wed, 11 May 2022 16:25:09 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 2B7BA203BFA31; Wed, 11 May 2022 16:25:09 +0200 (CEST) Message-ID: <20220511142345.289907761@infradead.org> User-Agent: quilt/0.66 Date: Wed, 11 May 2022 16:20:41 +0200 From: Peter Zijlstra To: x86@kernel.org, kan.liang@linux.intel.com, eranian@google.com Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, acme@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, namhyung@kernel.org Subject: [PATCH 4/5] perf/x86: Change x86_pmu::limit_period signature References: <20220511142037.353492804@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In preparation for making it a static_call, change the signature. Signed-off-by: Peter Zijlstra (Intel) --- arch/x86/events/amd/core.c | 8 +++----- arch/x86/events/core.c | 18 ++++++++---------- arch/x86/events/intel/core.c | 19 ++++++++----------- arch/x86/events/perf_event.h | 2 +- 4 files changed, 20 insertions(+), 27 deletions(-) --- a/arch/x86/events/amd/core.c +++ b/arch/x86/events/amd/core.c @@ -1258,16 +1258,14 @@ static void amd_pmu_sched_task(struct pe amd_pmu_brs_sched_task(ctx, sched_in); } =20 -static u64 amd_pmu_limit_period(struct perf_event *event, u64 left) +static void amd_pmu_limit_period(struct perf_event *event, s64 *left) { /* * Decrease period by the depth of the BRS feature to get the last N * taken branches and approximate the desired period */ - if (has_branch_stack(event) && left > x86_pmu.lbr_nr) - left -=3D x86_pmu.lbr_nr; - - return left; + if (has_branch_stack(event) && *left > x86_pmu.lbr_nr) + *left -=3D x86_pmu.lbr_nr; } =20 static __initconst const struct x86_pmu amd_pmu =3D { --- a/arch/x86/events/core.c +++ b/arch/x86/events/core.c @@ -621,8 +621,9 @@ int x86_pmu_hw_config(struct perf_event event->hw.config |=3D event->attr.config & X86_RAW_EVENT_MASK; =20 if (event->attr.sample_period && x86_pmu.limit_period) { - if (x86_pmu.limit_period(event, event->attr.sample_period) > - event->attr.sample_period) + s64 left =3D event->attr.sample_period; + x86_pmu.limit_period(event, &left); + if (left > event->attr.sample_period) return -EINVAL; } =20 @@ -1386,19 +1387,14 @@ int x86_perf_event_set_period(struct per hwc->last_period =3D period; ret =3D 1; } - /* - * Quirk: certain CPUs dont like it if just 1 hw_event is left: - */ - if (unlikely(left < 2)) - left =3D 2; =20 if (left > x86_pmu.max_period) left =3D x86_pmu.max_period; =20 if (x86_pmu.limit_period) - left =3D x86_pmu.limit_period(event, left); + x86_pmu.limit_period(event, &left); =20 - per_cpu(pmc_prev_left[idx], smp_processor_id()) =3D left; + this_cpu_write(pmc_prev_left[idx], left); =20 /* * The hw event starts counting from this event offset, @@ -2672,7 +2668,9 @@ static int x86_pmu_check_period(struct p return -EINVAL; =20 if (value && x86_pmu.limit_period) { - if (x86_pmu.limit_period(event, value) > value) + s64 left =3D value; + x86_pmu.limit_period(event, &left); + if (left > value) return -EINVAL; } =20 --- a/arch/x86/events/intel/core.c +++ b/arch/x86/events/intel/core.c @@ -4244,28 +4244,25 @@ static u8 adl_get_hybrid_cpu_type(void) * Therefore the effective (average) period matches the requested period, * despite coarser hardware granularity. */ -static u64 bdw_limit_period(struct perf_event *event, u64 left) +static void bdw_limit_period(struct perf_event *event, s64 *left) { if ((event->hw.config & INTEL_ARCH_EVENT_MASK) =3D=3D X86_CONFIG(.event=3D0xc0, .umask=3D0x01)) { - if (left < 128) - left =3D 128; - left &=3D ~0x3fULL; + if (*left < 128) + *left =3D 128; + *left &=3D ~0x3fULL; } - return left; } =20 -static u64 nhm_limit_period(struct perf_event *event, u64 left) +static void nhm_limit_period(struct perf_event *event, s64 *left) { - return max(left, 32ULL); + *left =3D max(*left, 32LL); } =20 -static u64 spr_limit_period(struct perf_event *event, u64 left) +static void spr_limit_period(struct perf_event *event, s64 *left) { if (event->attr.precise_ip =3D=3D 3) - return max(left, 128ULL); - - return left; + *left =3D max(*left, 128LL); } =20 PMU_FORMAT_ATTR(event, "config:0-7" ); --- a/arch/x86/events/perf_event.h +++ b/arch/x86/events/perf_event.h @@ -773,7 +773,7 @@ struct x86_pmu { struct event_constraint *event_constraints; struct x86_pmu_quirk *quirks; int perfctr_second_write; - u64 (*limit_period)(struct perf_event *event, u64 l); + void (*limit_period)(struct perf_event *event, s64 *l); =20 /* PMI handler bits */ unsigned int late_ack :1, From nobody Sun Jun 14 21:07:05 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DFA91C433F5 for ; Wed, 11 May 2022 14:25:34 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S244596AbiEKOZc (ORCPT ); Wed, 11 May 2022 10:25:32 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53198 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232399AbiEKOZT (ORCPT ); Wed, 11 May 2022 10:25:19 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 41C9963394 for ; Wed, 11 May 2022 07:25:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=RrsRwnufM8aJb8X9Do0ueNVFWhI9Rg8k/jxvL6pTy3A=; b=vnzOmGsJ8JHF5tstWbXoHMfj1B yet9pcHyTbhthoPq81ecsWA4MmEu0qPUOXOm3kD0AxKaRrUBfYz+kzBvrzC1xr3a7PuvUDZ8YQ1I7 2QxmPAu3g8eilLhw3+nFUiTgD4DRE5IoPiuYuYIk4O8PvR3XuWFldF5aCMIUNEHPEmQ6rc4ScgYTF aBYHYUzEyEUL0+NQNN5TgWP+vBQ5UlioCreu9ADuqI0GUHTcZPeGDJ3yGQtdr+s2RY1DNyNJqLtqs oP2UGPL9aASxNV1DvWesK4gvE5ug7avOO0hskbN83R3pkSzHWLOVLHe7TNtWi5fadrvLfMypH/Xnk cFd8s/Sw==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1nonH5-005VvM-LM; Wed, 11 May 2022 14:25:11 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 85EEE300859; Wed, 11 May 2022 16:25:09 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 0) id 2FE7C203C276F; Wed, 11 May 2022 16:25:09 +0200 (CEST) Message-ID: <20220511142345.354695274@infradead.org> User-Agent: quilt/0.66 Date: Wed, 11 May 2022 16:20:42 +0200 From: Peter Zijlstra To: x86@kernel.org, kan.liang@linux.intel.com, eranian@google.com Cc: linux-kernel@vger.kernel.org, peterz@infradead.org, acme@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, namhyung@kernel.org Subject: [PATCH 5/5] perf/x86: Add a x86_pmu::limit_period static_call References: <20220511142037.353492804@infradead.org> MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Avoid a branch and indirect call. Signed-off-by: Peter Zijlstra (Intel) --- arch/x86/events/core.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) --- a/arch/x86/events/core.c +++ b/arch/x86/events/core.c @@ -72,8 +72,9 @@ DEFINE_STATIC_CALL_NULL(x86_pmu_add, *x DEFINE_STATIC_CALL_NULL(x86_pmu_del, *x86_pmu.del); DEFINE_STATIC_CALL_NULL(x86_pmu_read, *x86_pmu.read); =20 -DEFINE_STATIC_CALL_NULL(x86_pmu_set_period, *x86_pmu.set_period); -DEFINE_STATIC_CALL_NULL(x86_pmu_update, *x86_pmu.update); +DEFINE_STATIC_CALL_NULL(x86_pmu_set_period, *x86_pmu.set_period); +DEFINE_STATIC_CALL_NULL(x86_pmu_update, *x86_pmu.update); +DEFINE_STATIC_CALL_NULL(x86_pmu_limit_period, *x86_pmu.limit_period); =20 DEFINE_STATIC_CALL_NULL(x86_pmu_schedule_events, *x86_pmu.schedule_e= vents); DEFINE_STATIC_CALL_NULL(x86_pmu_get_event_constraints, *x86_pmu.get_event_= constraints); @@ -1391,8 +1392,7 @@ int x86_perf_event_set_period(struct per if (left > x86_pmu.max_period) left =3D x86_pmu.max_period; =20 - if (x86_pmu.limit_period) - x86_pmu.limit_period(event, &left); + static_call_cond(x86_pmu_limit_period)(event, &left); =20 this_cpu_write(pmc_prev_left[idx], left); =20 @@ -2017,6 +2017,7 @@ static void x86_pmu_static_call_update(v =20 static_call_update(x86_pmu_set_period, x86_pmu.set_period); static_call_update(x86_pmu_update, x86_pmu.update); + static_call_update(x86_pmu_limit_period, x86_pmu.limit_period); =20 static_call_update(x86_pmu_schedule_events, x86_pmu.schedule_events); static_call_update(x86_pmu_get_event_constraints, x86_pmu.get_event_const= raints); From nobody Sun Jun 14 21:07:05 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id AD2FDC433EF for ; Wed, 11 May 2022 15:35:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343763AbiEKPfD (ORCPT ); Wed, 11 May 2022 11:35:03 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57730 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S244494AbiEKPfC (ORCPT ); Wed, 11 May 2022 11:35:02 -0400 Received: from desiato.infradead.org (desiato.infradead.org [IPv6:2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A2BA562BF8 for ; Wed, 11 May 2022 08:34:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=y+SBYyRLp190lyhAhB9+txzCudgdVn83IFku4q2UpiI=; b=W6mS9Eai/Z5pjlxzsAPFQvsRlY wcDDvWD9+nBz1rjwweVulL1LtnRXcru3r8shsbbqXkYuIOO/gO8WaskIoqE8R62zpHJIwQHy+dOlo 48gGe4kQJFqVB6E5W19o7WSf18r3Q5R6lNeisizy4yxMJLsQ/4f3jsRk+mT0l8Cxx2Kuig3v2Zrem ZmJMSMHyl06+Bu89FCBx2mcjHjqV5OP900H3dmQnl4fpq4G9KwKRdtD29LIPL2VcyhdW6g3UJnjMt WufnTU272CgIshDTz3TKIe/rbqM59IXcn8Bc61O+OKMXuvRW7WMC3l692bydBEiMnkKrzJa1MR7JM axkSfN4g==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1nooMS-00DCk5-O8; Wed, 11 May 2022 15:34:49 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id C5A4C3002BE; Wed, 11 May 2022 17:34:46 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 1000) id 8819F2038BE61; Wed, 11 May 2022 17:34:46 +0200 (CEST) Date: Wed, 11 May 2022 17:34:46 +0200 From: Peter Zijlstra To: x86@kernel.org, kan.liang@linux.intel.com, eranian@google.com Cc: linux-kernel@vger.kernel.org, acme@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, namhyung@kernel.org Subject: [RFC][PATCH 6/5] perf/x86/intel: Remove x86_pmu::set_topdown_event_period Message-ID: References: <20220511142037.353492804@infradead.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20220511142037.353492804@infradead.org> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Subject: perf/x86/intel: Remove x86_pmu::set_topdown_event_period From: Peter Zijlstra Date: Wed May 11 16:41:25 CEST 2022 Now that it is all internal to the intel driver, remove x86_pmu::set_topdown_event_period. Signed-off-by: Peter Zijlstra (Intel) --- arch/x86/events/intel/core.c | 16 ++++++++++------ arch/x86/events/perf_event.h | 1 - 2 files changed, 10 insertions(+), 7 deletions(-) --- a/arch/x86/events/intel/core.c +++ b/arch/x86/events/intel/core.c @@ -2520,6 +2520,8 @@ static int adl_set_topdown_event_period( return icl_set_topdown_event_period(event); } =20 +DEFINE_STATIC_CALL(intel_pmu_set_topdown_event_period, x86_perf_event_set_= period); + static inline u64 icl_get_metrics_event_value(u64 metric, u64 slots, int i= dx) { u32 val; @@ -2810,9 +2812,8 @@ int intel_pmu_save_and_restart(struct pe =20 static int intel_pmu_set_period(struct perf_event *event) { - if (unlikely(is_topdown_count(event)) && - x86_pmu.set_topdown_event_period) - return x86_pmu.set_topdown_event_period(event); + if (unlikely(is_topdown_count(event))) + return static_call(intel_pmu_set_topdown_event_period)(event); =20 return x86_perf_event_set_period(event); } @@ -6191,7 +6192,8 @@ __init int intel_pmu_init(void) intel_pmu_pebs_data_source_skl(pmem); x86_pmu.num_topdown_events =3D 4; x86_pmu.update_topdown_event =3D icl_update_topdown_event; - x86_pmu.set_topdown_event_period =3D icl_set_topdown_event_period; + static_call_update(intel_pmu_set_topdown_event_period, + &icl_set_topdown_event_period); pr_cont("Icelake events, "); name =3D "icelake"; break; @@ -6228,7 +6230,8 @@ __init int intel_pmu_init(void) intel_pmu_pebs_data_source_skl(pmem); x86_pmu.num_topdown_events =3D 8; x86_pmu.update_topdown_event =3D icl_update_topdown_event; - x86_pmu.set_topdown_event_period =3D icl_set_topdown_event_period; + static_call_update(intel_pmu_set_topdown_event_period, + &icl_set_topdown_event_period); pr_cont("Sapphire Rapids events, "); name =3D "sapphire_rapids"; break; @@ -6261,7 +6264,8 @@ __init int intel_pmu_init(void) intel_pmu_pebs_data_source_skl(false); x86_pmu.num_topdown_events =3D 8; x86_pmu.update_topdown_event =3D adl_update_topdown_event; - x86_pmu.set_topdown_event_period =3D adl_set_topdown_event_period; + static_call_update(intel_pmu_set_topdown_event_period, + &adl_set_topdown_event_period); =20 x86_pmu.filter_match =3D intel_pmu_filter_match; x86_pmu.get_event_constraints =3D adl_get_event_constraints; --- a/arch/x86/events/perf_event.h +++ b/arch/x86/events/perf_event.h @@ -879,7 +879,6 @@ struct x86_pmu { */ int num_topdown_events; u64 (*update_topdown_event)(struct perf_event *event); - int (*set_topdown_event_period)(struct perf_event *event); =20 /* * perf task context (i.e. struct perf_event_context::task_ctx_data) From nobody Sun Jun 14 21:07:05 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 87E21C433EF for ; Wed, 11 May 2022 15:35:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1343785AbiEKPfi (ORCPT ); Wed, 11 May 2022 11:35:38 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:58280 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234832AbiEKPfg (ORCPT ); Wed, 11 May 2022 11:35:36 -0400 Received: from casper.infradead.org (casper.infradead.org [IPv6:2001:8b0:10b:1236::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CE53369491 for ; Wed, 11 May 2022 08:35:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=4T/P7nk5HDT/LMOEvNiZF4F+08yR/q62Va+kEhtGeks=; b=pIWiZygdgVImGPwxabMGgcMTJv ACM/Zq3wQU0a+0hXIvEDKj3o6NCMsVbpelz7e8GE0C3ivuCd9YjBh/1S6T3dBW1lGv64wXvqjFPDj H9Ilm+339EGWEcjhBF9jjGMG0kmfU2qToNqRHkBPLGWbHOTr3EEpRH16L7elpC1H8DsemJphwkvSY /yUQtcsv7L2Ev+bgNmxbtJUQo/taHrSyGX4ErgJ1R17XWYN840NJbaS8756JCkzYlbaU3ARjNjcvY Qs9171GIEeBwLqlMsgiJSEItRjrWfUN6E7AJLXqEbOIuOiGY9NpnXRrvhHRVs4HEryQ0W6o7CEtOw 4+IVRTxQ==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by casper.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1nooN4-005Ywf-Pq; Wed, 11 May 2022 15:35:26 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 36F4230080B; Wed, 11 May 2022 17:35:26 +0200 (CEST) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 1000) id 2717E203BFA30; Wed, 11 May 2022 17:35:26 +0200 (CEST) Date: Wed, 11 May 2022 17:35:26 +0200 From: Peter Zijlstra To: x86@kernel.org, kan.liang@linux.intel.com, eranian@google.com Cc: linux-kernel@vger.kernel.org, acme@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, namhyung@kernel.org Subject: [RFC][PATCH 7/5] perf/x86/intel: Remove x86_pmu::update_topdown_event Message-ID: References: <20220511142037.353492804@infradead.org> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20220511142037.353492804@infradead.org> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Subject: perf/x86/intel: Remove x86_pmu::update_topdown_event From: Peter Zijlstra Date: Wed May 11 17:02:05 CEST 2022 Not that it is all internal to the intel driver, remove x86_pmu::update_topdown_event. Assumes that is_topdown_count(event) can only be true when the hardware has topdown stuff and the function is set. Signed-off-by: Peter Zijlstra (Intel) --- arch/x86/events/intel/core.c | 22 ++++++++++++---------- arch/x86/events/perf_event.h | 1 - 2 files changed, 12 insertions(+), 11 deletions(-) --- a/arch/x86/events/intel/core.c +++ b/arch/x86/events/intel/core.c @@ -2672,6 +2672,7 @@ static u64 adl_update_topdown_event(stru return icl_update_topdown_event(event); } =20 +DEFINE_STATIC_CALL(intel_pmu_update_topdown_event, x86_perf_event_update); =20 static void intel_pmu_read_topdown_event(struct perf_event *event) { @@ -2683,7 +2684,7 @@ static void intel_pmu_read_topdown_event return; =20 perf_pmu_disable(event->pmu); - x86_pmu.update_topdown_event(event); + static_call(intel_pmu_update_topdown_event)(event); perf_pmu_enable(event->pmu); } =20 @@ -2691,7 +2692,7 @@ static void intel_pmu_read_event(struct { if (event->hw.flags & PERF_X86_EVENT_AUTO_RELOAD) intel_pmu_auto_reload_read(event); - else if (is_topdown_count(event) && x86_pmu.update_topdown_event) + else if (is_topdown_count(event)) intel_pmu_read_topdown_event(event); else x86_perf_event_update(event); @@ -2820,9 +2821,8 @@ static int intel_pmu_set_period(struct p =20 static u64 intel_pmu_update(struct perf_event *event) { - if (unlikely(is_topdown_count(event)) && - x86_pmu.update_topdown_event) - return x86_pmu.update_topdown_event(event); + if (unlikely(is_topdown_count(event))) + return static_call(intel_pmu_update_topdown_event)(event); =20 return x86_perf_event_update(event); } @@ -2950,8 +2950,7 @@ static int handle_pmi_common(struct pt_r */ if (__test_and_clear_bit(GLOBAL_STATUS_PERF_METRICS_OVF_BIT, (unsigned lo= ng *)&status)) { handled++; - if (x86_pmu.update_topdown_event) - x86_pmu.update_topdown_event(NULL); + static_call(intel_pmu_update_topdown_event)(NULL); } =20 /* @@ -6191,7 +6190,8 @@ __init int intel_pmu_init(void) x86_pmu.lbr_pt_coexist =3D true; intel_pmu_pebs_data_source_skl(pmem); x86_pmu.num_topdown_events =3D 4; - x86_pmu.update_topdown_event =3D icl_update_topdown_event; + static_call_update(intel_pmu_update_topdown_event, + &icl_update_topdown_event); static_call_update(intel_pmu_set_topdown_event_period, &icl_set_topdown_event_period); pr_cont("Icelake events, "); @@ -6229,7 +6229,8 @@ __init int intel_pmu_init(void) x86_pmu.lbr_pt_coexist =3D true; intel_pmu_pebs_data_source_skl(pmem); x86_pmu.num_topdown_events =3D 8; - x86_pmu.update_topdown_event =3D icl_update_topdown_event; + static_call_update(intel_pmu_update_topdown_event, + &icl_update_topdown_event); static_call_update(intel_pmu_set_topdown_event_period, &icl_set_topdown_event_period); pr_cont("Sapphire Rapids events, "); @@ -6263,7 +6264,8 @@ __init int intel_pmu_init(void) x86_pmu.lbr_pt_coexist =3D true; intel_pmu_pebs_data_source_skl(false); x86_pmu.num_topdown_events =3D 8; - x86_pmu.update_topdown_event =3D adl_update_topdown_event; + static_call_update(intel_pmu_update_topdown_event, + &adl_update_topdown_event); static_call_update(intel_pmu_set_topdown_event_period, &adl_set_topdown_event_period); =20 --- a/arch/x86/events/perf_event.h +++ b/arch/x86/events/perf_event.h @@ -878,7 +878,6 @@ struct x86_pmu { * Intel perf metrics */ int num_topdown_events; - u64 (*update_topdown_event)(struct perf_event *event); =20 /* * perf task context (i.e. struct perf_event_context::task_ctx_data)