From nobody Sat Nov 23 15:14:10 2024 Received: from galois.linutronix.de (Galois.linutronix.de [193.142.43.55]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6E76E13C8F3; Tue, 19 Nov 2024 08:48:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=193.142.43.55 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732006137; cv=none; b=RW2DvxVN9DA95s0SAHTzP/X7fff0HMUS45/ZfsNM/22gSRDVDZy1xd2t+Lhsi4grxNZaedlTBivWXnFs9gbX3OdMOXGwFWXgf5prHdX5SCwB9dzPQfvic+Wqh/cCcdqxLBEMHN1RTxCvoccswlBspd2fJM1hoPrZMuZfBdRtWuw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732006137; c=relaxed/simple; bh=afao0aY+yNSeIT6M3FOH/a+FU3dQYIPiuRR+jeiz/2s=; h=Date:From:To:Subject:Cc:In-Reply-To:References:MIME-Version: Message-ID:Content-Type; b=n5eeAY/pVoU913BmJKzN/h48qbnw3hq+S5QeP/qgEsQJuPAfzJakywsIPSU0wyuGFfytz6Ur+/1eL84iY+pNzkYaWThycQI0FZlwDHQq2/t0SeyHkRRuKLM9DVtGcFCkao9ipbcJqHr4KO8grYdB0Ldm9TTbKPk/UdvKr5tXVbA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de; spf=pass smtp.mailfrom=linutronix.de; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=ugEtOosD; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b=EHzRW2EM; arc=none smtp.client-ip=193.142.43.55 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linutronix.de Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linutronix.de Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="ugEtOosD"; dkim=permerror (0-bit key) header.d=linutronix.de header.i=@linutronix.de header.b="EHzRW2EM" Date: Tue, 19 Nov 2024 08:48:50 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1732006131; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=H59Af6/KjQaKhGwtXsUbYVDmnGdfpqZ7rmNWeaRXoWM=; b=ugEtOosDWWj3XcBtxegCPsVy46VgyigSYqjgmez2tV0VR/CXY4h+I49RJEDmH5RTi7AWAj Kk5GNyJ8PMXsIImaeQy7HJlaQ1ZghEg8Mua2bHJn5CrYz2o3ppO01Cn3M/GDxUdCw7iOZv 7GprkGhynJLu3prodJ85vtS9dqlvl87y8kFc2gxSxkDbLDrrcdf58KRjHW54VWmrWOOQ4p PDHzU7aZmBDip3HfTgTopmYY8N45UoK/tP9wALLVicaZ/uJLET69mgBHf7IE4Wa94mco0w +ofwylD4iIoRBHmVXiOOPS/Z+jqgXIZbUPYhCpFaP6nM1eGcy63zG8QIqwANrg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1732006131; h=from:from:sender:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=H59Af6/KjQaKhGwtXsUbYVDmnGdfpqZ7rmNWeaRXoWM=; b=EHzRW2EMp+oqI6HFWEZ+fTZXXeskUjZlh/csw381oYFECQhJVKkyc7fKvjooHgsDsBm8we pWpMmhPnzrz+suDA== From: "tip-bot2 for Yabin Cui" Sender: tip-bot2@linutronix.de Reply-to: linux-kernel@vger.kernel.org To: linux-tip-commits@vger.kernel.org Subject: [tip: perf/core] perf/core: Save raw sample data conditionally based on sample type Cc: Yabin Cui , Ingo Molnar , Ian Rogers , Namhyung Kim , x86@kernel.org, linux-kernel@vger.kernel.org In-Reply-To: <20240515193610.2350456-2-yabinc@google.com> References: <20240515193610.2350456-2-yabinc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-ID: <173200613028.412.14433264233875847097.tip-bot2@tip-bot2> Robot-ID: Robot-Unsubscribe: Contact to get blacklisted from these emails Precedence: bulk Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable The following commit has been merged into the perf/core branch of tip: Commit-ID: b9c44b91476b67327a521568a854babecc4070ab Gitweb: https://git.kernel.org/tip/b9c44b91476b67327a521568a854babec= c4070ab Author: Yabin Cui AuthorDate: Wed, 15 May 2024 12:36:07 -07:00 Committer: Ingo Molnar CommitterDate: Tue, 19 Nov 2024 09:23:42 +01:00 perf/core: Save raw sample data conditionally based on sample type Currently, space for raw sample data is always allocated within sample records for both BPF output and tracepoint events. This leads to unused space in sample records when raw sample data is not requested. This patch enforces checking sample type of an event in perf_sample_save_raw_data(). So raw sample data will only be saved if explicitly requested, reducing overhead when it is not needed. Fixes: 0a9081cf0a11 ("perf/core: Add perf_sample_save_raw_data() helper") Signed-off-by: Yabin Cui Signed-off-by: Ingo Molnar Reviewed-by: Ian Rogers Acked-by: Namhyung Kim Link: https://lore.kernel.org/r/20240515193610.2350456-2-yabinc@google.com --- arch/s390/kernel/perf_cpum_cf.c | 2 +- arch/s390/kernel/perf_pai_crypto.c | 2 +- arch/s390/kernel/perf_pai_ext.c | 2 +- arch/x86/events/amd/ibs.c | 2 +- include/linux/perf_event.h | 6 +++++- kernel/events/core.c | 35 ++++++++++++++--------------- kernel/trace/bpf_trace.c | 11 ++++----- 7 files changed, 34 insertions(+), 26 deletions(-) diff --git a/arch/s390/kernel/perf_cpum_cf.c b/arch/s390/kernel/perf_cpum_c= f.c index e2e0aa4..c3075e4 100644 --- a/arch/s390/kernel/perf_cpum_cf.c +++ b/arch/s390/kernel/perf_cpum_cf.c @@ -981,7 +981,7 @@ static int cfdiag_push_sample(struct perf_event *event, if (event->attr.sample_type & PERF_SAMPLE_RAW) { raw.frag.size =3D cpuhw->usedss; raw.frag.data =3D cpuhw->stop; - perf_sample_save_raw_data(&data, &raw); + perf_sample_save_raw_data(&data, event, &raw); } =20 overflow =3D perf_event_overflow(event, &data, ®s); diff --git a/arch/s390/kernel/perf_pai_crypto.c b/arch/s390/kernel/perf_pai= _crypto.c index fa73254..10725f5 100644 --- a/arch/s390/kernel/perf_pai_crypto.c +++ b/arch/s390/kernel/perf_pai_crypto.c @@ -478,7 +478,7 @@ static int paicrypt_push_sample(size_t rawsize, struct = paicrypt_map *cpump, if (event->attr.sample_type & PERF_SAMPLE_RAW) { raw.frag.size =3D rawsize; raw.frag.data =3D cpump->save; - perf_sample_save_raw_data(&data, &raw); + perf_sample_save_raw_data(&data, event, &raw); } =20 overflow =3D perf_event_overflow(event, &data, ®s); diff --git a/arch/s390/kernel/perf_pai_ext.c b/arch/s390/kernel/perf_pai_ex= t.c index 7f462be..a8f0bad 100644 --- a/arch/s390/kernel/perf_pai_ext.c +++ b/arch/s390/kernel/perf_pai_ext.c @@ -503,7 +503,7 @@ static int paiext_push_sample(size_t rawsize, struct pa= iext_map *cpump, if (event->attr.sample_type & PERF_SAMPLE_RAW) { raw.frag.size =3D rawsize; raw.frag.data =3D cpump->save; - perf_sample_save_raw_data(&data, &raw); + perf_sample_save_raw_data(&data, event, &raw); } =20 overflow =3D perf_event_overflow(event, &data, ®s); diff --git a/arch/x86/events/amd/ibs.c b/arch/x86/events/amd/ibs.c index e91970b..c3a2f6f 100644 --- a/arch/x86/events/amd/ibs.c +++ b/arch/x86/events/amd/ibs.c @@ -1118,7 +1118,7 @@ fail: .data =3D ibs_data.data, }, }; - perf_sample_save_raw_data(&data, &raw); + perf_sample_save_raw_data(&data, event, &raw); } =20 if (perf_ibs =3D=3D &perf_ibs_op) diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index cb99ec8..f7c0a3f 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -1287,12 +1287,18 @@ static inline void perf_sample_save_callchain(struc= t perf_sample_data *data, } =20 static inline void perf_sample_save_raw_data(struct perf_sample_data *data, + struct perf_event *event, struct perf_raw_record *raw) { struct perf_raw_frag *frag =3D &raw->frag; u32 sum =3D 0; int size; =20 + if (!(event->attr.sample_type & PERF_SAMPLE_RAW)) + return; + if (WARN_ON_ONCE(data->sample_flags & PERF_SAMPLE_RAW)) + return; + do { sum +=3D frag->size; if (perf_raw_frag_last(frag)) diff --git a/kernel/events/core.c b/kernel/events/core.c index 1869164..3670b91 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -10451,9 +10451,9 @@ static struct pmu perf_tracepoint =3D { }; =20 static int perf_tp_filter_match(struct perf_event *event, - struct perf_sample_data *data) + struct perf_raw_record *raw) { - void *record =3D data->raw->frag.data; + void *record =3D raw->frag.data; =20 /* only top level events have filters set */ if (event->parent) @@ -10465,7 +10465,7 @@ static int perf_tp_filter_match(struct perf_event *= event, } =20 static int perf_tp_event_match(struct perf_event *event, - struct perf_sample_data *data, + struct perf_raw_record *raw, struct pt_regs *regs) { if (event->hw.state & PERF_HES_STOPPED) @@ -10476,7 +10476,7 @@ static int perf_tp_event_match(struct perf_event *e= vent, if (event->attr.exclude_kernel && !user_mode(regs)) return 0; =20 - if (!perf_tp_filter_match(event, data)) + if (!perf_tp_filter_match(event, raw)) return 0; =20 return 1; @@ -10502,6 +10502,7 @@ EXPORT_SYMBOL_GPL(perf_trace_run_bpf_submit); static void __perf_tp_event_target_task(u64 count, void *record, struct pt_regs *regs, struct perf_sample_data *data, + struct perf_raw_record *raw, struct perf_event *event) { struct trace_entry *entry =3D record; @@ -10511,13 +10512,17 @@ static void __perf_tp_event_target_task(u64 count= , void *record, /* Cannot deliver synchronous signal to other task. */ if (event->attr.sigtrap) return; - if (perf_tp_event_match(event, data, regs)) + if (perf_tp_event_match(event, raw, regs)) { + perf_sample_data_init(data, 0, 0); + perf_sample_save_raw_data(data, event, raw); perf_swevent_event(event, count, data, regs); + } } =20 static void perf_tp_event_target_task(u64 count, void *record, struct pt_regs *regs, struct perf_sample_data *data, + struct perf_raw_record *raw, struct perf_event_context *ctx) { unsigned int cpu =3D smp_processor_id(); @@ -10525,15 +10530,15 @@ static void perf_tp_event_target_task(u64 count, = void *record, struct perf_event *event, *sibling; =20 perf_event_groups_for_cpu_pmu(event, &ctx->pinned_groups, cpu, pmu) { - __perf_tp_event_target_task(count, record, regs, data, event); + __perf_tp_event_target_task(count, record, regs, data, raw, event); for_each_sibling_event(sibling, event) - __perf_tp_event_target_task(count, record, regs, data, sibling); + __perf_tp_event_target_task(count, record, regs, data, raw, sibling); } =20 perf_event_groups_for_cpu_pmu(event, &ctx->flexible_groups, cpu, pmu) { - __perf_tp_event_target_task(count, record, regs, data, event); + __perf_tp_event_target_task(count, record, regs, data, raw, event); for_each_sibling_event(sibling, event) - __perf_tp_event_target_task(count, record, regs, data, sibling); + __perf_tp_event_target_task(count, record, regs, data, raw, sibling); } } =20 @@ -10551,15 +10556,10 @@ void perf_tp_event(u16 event_type, u64 count, voi= d *record, int entry_size, }, }; =20 - perf_sample_data_init(&data, 0, 0); - perf_sample_save_raw_data(&data, &raw); - perf_trace_buf_update(record, event_type); =20 hlist_for_each_entry_rcu(event, head, hlist_entry) { - if (perf_tp_event_match(event, &data, regs)) { - perf_swevent_event(event, count, &data, regs); - + if (perf_tp_event_match(event, &raw, regs)) { /* * Here use the same on-stack perf_sample_data, * some members in data are event-specific and @@ -10569,7 +10569,8 @@ void perf_tp_event(u16 event_type, u64 count, void = *record, int entry_size, * because data->sample_flags is set. */ perf_sample_data_init(&data, 0, 0); - perf_sample_save_raw_data(&data, &raw); + perf_sample_save_raw_data(&data, event, &raw); + perf_swevent_event(event, count, &data, regs); } } =20 @@ -10586,7 +10587,7 @@ void perf_tp_event(u16 event_type, u64 count, void = *record, int entry_size, goto unlock; =20 raw_spin_lock(&ctx->lock); - perf_tp_event_target_task(count, record, regs, &data, ctx); + perf_tp_event_target_task(count, record, regs, &data, &raw, ctx); raw_spin_unlock(&ctx->lock); unlock: rcu_read_unlock(); diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index fdab7ec..162bacf 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -619,7 +619,8 @@ static const struct bpf_func_proto bpf_perf_event_read_= value_proto =3D { =20 static __always_inline u64 __bpf_perf_event_output(struct pt_regs *regs, struct bpf_map *map, - u64 flags, struct perf_sample_data *sd) + u64 flags, struct perf_raw_record *raw, + struct perf_sample_data *sd) { struct bpf_array *array =3D container_of(map, struct bpf_array, map); unsigned int cpu =3D smp_processor_id(); @@ -644,6 +645,8 @@ __bpf_perf_event_output(struct pt_regs *regs, struct bp= f_map *map, if (unlikely(event->oncpu !=3D cpu)) return -EOPNOTSUPP; =20 + perf_sample_save_raw_data(sd, event, raw); + return perf_event_output(event, sd, regs); } =20 @@ -687,9 +690,8 @@ BPF_CALL_5(bpf_perf_event_output, struct pt_regs *, reg= s, struct bpf_map *, map, } =20 perf_sample_data_init(sd, 0, 0); - perf_sample_save_raw_data(sd, &raw); =20 - err =3D __bpf_perf_event_output(regs, map, flags, sd); + err =3D __bpf_perf_event_output(regs, map, flags, &raw, sd); out: this_cpu_dec(bpf_trace_nest_level); preempt_enable(); @@ -748,9 +750,8 @@ u64 bpf_event_output(struct bpf_map *map, u64 flags, vo= id *meta, u64 meta_size, =20 perf_fetch_caller_regs(regs); perf_sample_data_init(sd, 0, 0); - perf_sample_save_raw_data(sd, &raw); =20 - ret =3D __bpf_perf_event_output(regs, map, flags, sd); + ret =3D __bpf_perf_event_output(regs, map, flags, &raw, sd); out: this_cpu_dec(bpf_event_output_nest_level); preempt_enable();