From nobody Sat Feb 14 05:36:51 2026 Received: from mail-yw1-f201.google.com (mail-yw1-f201.google.com [209.85.128.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0891D159598 for ; Wed, 15 May 2024 19:36:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715801790; cv=none; b=MoHNkhwU+7PP7kKuHlTKPvVGLfdC33Rqfo6ATrmDNawyuhpQ25uHf6A0WHwTs1vA5sqixHugvEAMrNbA691mA+uE0114IhcQ+tpATheZSAnhXu/K/QpKxgYJHgG+AUBR9GfMbWTRNwJ4rs1uDBvlFH33HNSrlffwJP9oYRBDAEk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715801790; c=relaxed/simple; bh=EbuXkutu4SiTY7XkcXY+EZ/i0vH1OfoP6NR93JYJ3KY=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=kBKFV2fG1MA/B3YzhH2x8SmQCDX1bYrpbZ+BhpG6klWr9FtqcuY+0Lt+0nm/peXy1RB0wTS/ZsZjUpBklzl+imW/r2+cyAvNNZHMEx/WU3M7C2fA+9RRXAhTISRp0c/kEUKrKPsUnnq63qu9/2fqmo0fpBDOK+y3TVlANF1cdsk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--yabinc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=z+HFtJ5/; arc=none smtp.client-ip=209.85.128.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--yabinc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="z+HFtJ5/" Received: by mail-yw1-f201.google.com with SMTP id 00721157ae682-6204c4f4240so113864347b3.3 for ; Wed, 15 May 2024 12:36:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1715801787; x=1716406587; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=fXwOZAIdSfp5mZkveOIuW8YalkQhwYauV3LK0Wa4qgM=; b=z+HFtJ5/XGGXdtid/MQl/UdDr6JOHrSR9A+ToALPSWn4D6mnCl1MYla4zQUU40FMrI O03ScbCMr7AQw5JLRTqErY5UrSnOuP++PZWlGWxrVkohJpx+PDRtQLOPCE8Rj9YZbjel gvfRzyfReeyJRtxutoS0vi8bSv1pQgcTELVltJvvVn3gVtP8C+Xpl4h5sEjfovLv6Q25 cjVKYTYODcRrym+oesy9DevKgaf60jQyjASSXdRbQ4w991H6mp7rKurzSv7RIDY6Tey5 EOQA6HNFrPfuPSolnGNFQ3ImUiXU2dQ/W9gOoiZuXawubeyQeEJkeKf+JYMjKE6npfZX Pbkw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715801787; x=1716406587; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=fXwOZAIdSfp5mZkveOIuW8YalkQhwYauV3LK0Wa4qgM=; b=PHGUBAiZUQU3edS/Gbkc09wV/PeYzOTa4jdIzql9bCqLJiGp/LjS3OUmJPCK308l3X VIC8+LMfAO4CT6675WWU6eqPcGtWXHCD0O4EphEMSWJef84YPz9jXSdW3EauWzebraw8 9PLLLUJchXgXRcCfAeL0bmCmgvaK68I4djipo5O17CU5a4o2DaJE/0mLH2I0qW7Zy8pr mSFB5+136/1taCerycCNE0vQZ2ZR5bOlZ7P8Kw+dxuT4RFIscZjs+d8fvUzTktMikXDf sfmfj8HMKwvh75erkZ28YKfPMinXf3u8xbS4jncEJczEs7zEIHrW+KxHJeJ6NCJae1Qi Yaqw== X-Forwarded-Encrypted: i=1; AJvYcCWElaHZnUPA41MB8jLWNvkVPfmhWnv9CXnj4IT26DyYSXtCxj6K6WjiJL9fFoOldF4FdUFGVKj/5H9G5VPuXl1GdZ8YJQn8bKhw1riI X-Gm-Message-State: AOJu0Yzy66E+5gB1W5ElSVVb2hPX6oILz/zpFZXE/dEeSgQi8IoHHMuG bmxTcR0Rbp3D9rVCVbj7qe5kvMkYEXYmxwW11jJ/NUD/x2W1G9qrnge1dMjuc7S0JoyXr7peBGu m X-Google-Smtp-Source: AGHT+IGzo1N8KtCx5N6jvWrTB8oDA5Ct6uYEeZeEijzfl2sTvDtmFEz2XqRAbCQe/iFUSRDGRG0s2JUqckM= X-Received: from yabinc-desktop.mtv.corp.google.com ([2620:15c:211:202:6e4e:954d:1e49:f87c]) (user=yabinc job=sendgmr) by 2002:a81:6d97:0:b0:627:3c45:4a90 with SMTP id 00721157ae682-6273c454b1emr3280237b3.4.1715801787120; Wed, 15 May 2024 12:36:27 -0700 (PDT) Date: Wed, 15 May 2024 12:36:07 -0700 In-Reply-To: <20240515193610.2350456-1-yabinc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240515193610.2350456-1-yabinc@google.com> X-Mailer: git-send-email 2.45.0.rc1.225.g2a3ae87e7f-goog Message-ID: <20240515193610.2350456-2-yabinc@google.com> Subject: [PATCH v5 1/3] perf/core: Save raw sample data conditionally based on sample type From: Yabin Cui To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Mark Rutland , Alexander Shishkin , Jiri Olsa , Ian Rogers , Adrian Hunter Cc: linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org, Yabin Cui Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Currently, space for raw sample data is always allocated within sample records for both BPF output and tracepoint events. This leads to unused space in sample records when raw sample data is not requested. This patch enforces checking sample type of an event in perf_sample_save_raw_data(). So raw sample data will only be saved if explicitly requested, reducing overhead when it is not needed. Fixes: 0a9081cf0a11 ("perf/core: Add perf_sample_save_raw_data() helper") Signed-off-by: Yabin Cui Acked-by: Namhyung Kim --- arch/s390/kernel/perf_cpum_cf.c | 2 +- arch/s390/kernel/perf_pai_crypto.c | 2 +- arch/s390/kernel/perf_pai_ext.c | 2 +- arch/x86/events/amd/ibs.c | 2 +- include/linux/perf_event.h | 6 +++++ kernel/events/core.c | 35 +++++++++++++++--------------- kernel/trace/bpf_trace.c | 11 +++++----- 7 files changed, 34 insertions(+), 26 deletions(-) diff --git a/arch/s390/kernel/perf_cpum_cf.c b/arch/s390/kernel/perf_cpum_c= f.c index 1434642e9cba..9bf9464109e9 100644 --- a/arch/s390/kernel/perf_cpum_cf.c +++ b/arch/s390/kernel/perf_cpum_cf.c @@ -971,7 +971,7 @@ static int cfdiag_push_sample(struct perf_event *event, if (event->attr.sample_type & PERF_SAMPLE_RAW) { raw.frag.size =3D cpuhw->usedss; raw.frag.data =3D cpuhw->stop; - perf_sample_save_raw_data(&data, &raw); + perf_sample_save_raw_data(&data, event, &raw); } =20 overflow =3D perf_event_overflow(event, &data, ®s); diff --git a/arch/s390/kernel/perf_pai_crypto.c b/arch/s390/kernel/perf_pai= _crypto.c index 4ad472d130a3..2fb8aeba4872 100644 --- a/arch/s390/kernel/perf_pai_crypto.c +++ b/arch/s390/kernel/perf_pai_crypto.c @@ -444,7 +444,7 @@ static int paicrypt_push_sample(size_t rawsize, struct = paicrypt_map *cpump, if (event->attr.sample_type & PERF_SAMPLE_RAW) { raw.frag.size =3D rawsize; raw.frag.data =3D cpump->save; - perf_sample_save_raw_data(&data, &raw); + perf_sample_save_raw_data(&data, event, &raw); } =20 overflow =3D perf_event_overflow(event, &data, ®s); diff --git a/arch/s390/kernel/perf_pai_ext.c b/arch/s390/kernel/perf_pai_ex= t.c index a6da7e0cc7a6..b2914df2107a 100644 --- a/arch/s390/kernel/perf_pai_ext.c +++ b/arch/s390/kernel/perf_pai_ext.c @@ -458,7 +458,7 @@ static int paiext_push_sample(size_t rawsize, struct pa= iext_map *cpump, if (event->attr.sample_type & PERF_SAMPLE_RAW) { raw.frag.size =3D rawsize; raw.frag.data =3D cpump->save; - perf_sample_save_raw_data(&data, &raw); + perf_sample_save_raw_data(&data, event, &raw); } =20 overflow =3D perf_event_overflow(event, &data, ®s); diff --git a/arch/x86/events/amd/ibs.c b/arch/x86/events/amd/ibs.c index e91970b01d62..c3a2f6f57770 100644 --- a/arch/x86/events/amd/ibs.c +++ b/arch/x86/events/amd/ibs.c @@ -1118,7 +1118,7 @@ static int perf_ibs_handle_irq(struct perf_ibs *perf_= ibs, struct pt_regs *iregs) .data =3D ibs_data.data, }, }; - perf_sample_save_raw_data(&data, &raw); + perf_sample_save_raw_data(&data, event, &raw); } =20 if (perf_ibs =3D=3D &perf_ibs_op) diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index a5304ae8c654..fefac1a57b56 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -1243,12 +1243,18 @@ static inline void perf_sample_save_callchain(struc= t perf_sample_data *data, } =20 static inline void perf_sample_save_raw_data(struct perf_sample_data *data, + struct perf_event *event, struct perf_raw_record *raw) { struct perf_raw_frag *frag =3D &raw->frag; u32 sum =3D 0; int size; =20 + if (!(event->attr.sample_type & PERF_SAMPLE_RAW)) + return; + if (WARN_ON_ONCE(data->sample_flags & PERF_SAMPLE_RAW)) + return; + do { sum +=3D frag->size; if (perf_raw_frag_last(frag)) diff --git a/kernel/events/core.c b/kernel/events/core.c index 6b0a66ed2ae3..28e4e777e82a 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -10228,9 +10228,9 @@ static struct pmu perf_tracepoint =3D { }; =20 static int perf_tp_filter_match(struct perf_event *event, - struct perf_sample_data *data) + struct perf_raw_record *raw) { - void *record =3D data->raw->frag.data; + void *record =3D raw->frag.data; =20 /* only top level events have filters set */ if (event->parent) @@ -10242,7 +10242,7 @@ static int perf_tp_filter_match(struct perf_event *= event, } =20 static int perf_tp_event_match(struct perf_event *event, - struct perf_sample_data *data, + struct perf_raw_record *raw, struct pt_regs *regs) { if (event->hw.state & PERF_HES_STOPPED) @@ -10253,7 +10253,7 @@ static int perf_tp_event_match(struct perf_event *e= vent, if (event->attr.exclude_kernel && !user_mode(regs)) return 0; =20 - if (!perf_tp_filter_match(event, data)) + if (!perf_tp_filter_match(event, raw)) return 0; =20 return 1; @@ -10279,6 +10279,7 @@ EXPORT_SYMBOL_GPL(perf_trace_run_bpf_submit); static void __perf_tp_event_target_task(u64 count, void *record, struct pt_regs *regs, struct perf_sample_data *data, + struct perf_raw_record *raw, struct perf_event *event) { struct trace_entry *entry =3D record; @@ -10288,13 +10289,17 @@ static void __perf_tp_event_target_task(u64 count= , void *record, /* Cannot deliver synchronous signal to other task. */ if (event->attr.sigtrap) return; - if (perf_tp_event_match(event, data, regs)) + if (perf_tp_event_match(event, raw, regs)) { + perf_sample_data_init(data, 0, 0); + perf_sample_save_raw_data(data, event, raw); perf_swevent_event(event, count, data, regs); + } } =20 static void perf_tp_event_target_task(u64 count, void *record, struct pt_regs *regs, struct perf_sample_data *data, + struct perf_raw_record *raw, struct perf_event_context *ctx) { unsigned int cpu =3D smp_processor_id(); @@ -10302,15 +10307,15 @@ static void perf_tp_event_target_task(u64 count, = void *record, struct perf_event *event, *sibling; =20 perf_event_groups_for_cpu_pmu(event, &ctx->pinned_groups, cpu, pmu) { - __perf_tp_event_target_task(count, record, regs, data, event); + __perf_tp_event_target_task(count, record, regs, data, raw, event); for_each_sibling_event(sibling, event) - __perf_tp_event_target_task(count, record, regs, data, sibling); + __perf_tp_event_target_task(count, record, regs, data, raw, sibling); } =20 perf_event_groups_for_cpu_pmu(event, &ctx->flexible_groups, cpu, pmu) { - __perf_tp_event_target_task(count, record, regs, data, event); + __perf_tp_event_target_task(count, record, regs, data, raw, event); for_each_sibling_event(sibling, event) - __perf_tp_event_target_task(count, record, regs, data, sibling); + __perf_tp_event_target_task(count, record, regs, data, raw, sibling); } } =20 @@ -10328,15 +10333,10 @@ void perf_tp_event(u16 event_type, u64 count, voi= d *record, int entry_size, }, }; =20 - perf_sample_data_init(&data, 0, 0); - perf_sample_save_raw_data(&data, &raw); - perf_trace_buf_update(record, event_type); =20 hlist_for_each_entry_rcu(event, head, hlist_entry) { - if (perf_tp_event_match(event, &data, regs)) { - perf_swevent_event(event, count, &data, regs); - + if (perf_tp_event_match(event, &raw, regs)) { /* * Here use the same on-stack perf_sample_data, * some members in data are event-specific and @@ -10346,7 +10346,8 @@ void perf_tp_event(u16 event_type, u64 count, void = *record, int entry_size, * because data->sample_flags is set. */ perf_sample_data_init(&data, 0, 0); - perf_sample_save_raw_data(&data, &raw); + perf_sample_save_raw_data(&data, event, &raw); + perf_swevent_event(event, count, &data, regs); } } =20 @@ -10363,7 +10364,7 @@ void perf_tp_event(u16 event_type, u64 count, void = *record, int entry_size, goto unlock; =20 raw_spin_lock(&ctx->lock); - perf_tp_event_target_task(count, record, regs, &data, ctx); + perf_tp_event_target_task(count, record, regs, &data, &raw, ctx); raw_spin_unlock(&ctx->lock); unlock: rcu_read_unlock(); diff --git a/kernel/trace/bpf_trace.c b/kernel/trace/bpf_trace.c index f5154c051d2c..a7ec7a3eb7e1 100644 --- a/kernel/trace/bpf_trace.c +++ b/kernel/trace/bpf_trace.c @@ -620,7 +620,8 @@ static const struct bpf_func_proto bpf_perf_event_read_= value_proto =3D { =20 static __always_inline u64 __bpf_perf_event_output(struct pt_regs *regs, struct bpf_map *map, - u64 flags, struct perf_sample_data *sd) + u64 flags, struct perf_raw_record *raw, + struct perf_sample_data *sd) { struct bpf_array *array =3D container_of(map, struct bpf_array, map); unsigned int cpu =3D smp_processor_id(); @@ -645,6 +646,8 @@ __bpf_perf_event_output(struct pt_regs *regs, struct bp= f_map *map, if (unlikely(event->oncpu !=3D cpu)) return -EOPNOTSUPP; =20 + perf_sample_save_raw_data(sd, event, raw); + return perf_event_output(event, sd, regs); } =20 @@ -688,9 +691,8 @@ BPF_CALL_5(bpf_perf_event_output, struct pt_regs *, reg= s, struct bpf_map *, map, } =20 perf_sample_data_init(sd, 0, 0); - perf_sample_save_raw_data(sd, &raw); =20 - err =3D __bpf_perf_event_output(regs, map, flags, sd); + err =3D __bpf_perf_event_output(regs, map, flags, &raw, sd); out: this_cpu_dec(bpf_trace_nest_level); preempt_enable(); @@ -749,9 +751,8 @@ u64 bpf_event_output(struct bpf_map *map, u64 flags, vo= id *meta, u64 meta_size, =20 perf_fetch_caller_regs(regs); perf_sample_data_init(sd, 0, 0); - perf_sample_save_raw_data(sd, &raw); =20 - ret =3D __bpf_perf_event_output(regs, map, flags, sd); + ret =3D __bpf_perf_event_output(regs, map, flags, &raw, sd); out: this_cpu_dec(bpf_event_output_nest_level); preempt_enable(); --=20 2.45.0.rc1.225.g2a3ae87e7f-goog From nobody Sat Feb 14 05:36:51 2026 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C516515B0F2 for ; Wed, 15 May 2024 19:36:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715801793; cv=none; b=aLhbIJM72gchqDmJMepD0S0LvzTxwr5Xl/Ib9aC7jFSvxxut5L1WvyXEV65XZP6lRGt6008VhqJOeIm5H62Flyvqr6EfTIB3peVC/h10fOyyRpdqXcprxHQZQsuDhu0HeP7pr5S/ahaWIpfCB6t3ZRvc5eOMjPgF2uAIpVHavCs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715801793; c=relaxed/simple; bh=0lSt7QVBUAdOGQGjlbIPqXYDj232a+VWY9UlJ/ZrpNU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=ggU3oCNLp44jFJ0Yw9dYeF4fROpflXv6yOUw5j3C0opgkr91XUhd9Qvcoc86pnZNei+diBznTixK6PQDUkPuUSJWJfyJuf0RFddPUMBvK3bgFmwfraMuvSnrDQhismDZX2SW+MgzLuseND3REggOHmf1mVy1pQcBNcJ43Wb2VqI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--yabinc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=nqcJeDG4; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--yabinc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="nqcJeDG4" Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-de604ccb373so12202303276.2 for ; Wed, 15 May 2024 12:36:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1715801791; x=1716406591; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=YlmXNn2pYx8wzd3I64r7rBzQTUUgwVUdbiBLIDfo5/M=; b=nqcJeDG4VS51tGH0eNlWJoGayc4RrT+yo5IBujXLTc4nPTLlc2J4p0MMFR32Z7lSaz ppcz2KvOuj7LrhOsRg55PtbeID9JycUPJ3eBGVK/tNGq4VucWozqkxdSfOLbbpO17prQ sWuIwN/jjBE0ZVMEaXfgWjGSIq9S/BjpTOsyJ0pFRhip7B5v0uD4dEKyeFnTS0D5n18u ZOtkM/+FANKK4yv6kaJytaB+sGQZ0xeirNUOBMRqpsASkfeSJfn3MyxcKjiA2GpNJCoe VYBzMiX3v+wqmc/4Hv15pHNVBYWLdjNPr+sHBe+DMe/BXRA9qH02YVCEQXwRfaEcAhmy sgIg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715801791; x=1716406591; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=YlmXNn2pYx8wzd3I64r7rBzQTUUgwVUdbiBLIDfo5/M=; b=LZaYHLDo3myNlfw70OOkkypmDJhtemvxM9KNoLqz8O9BN0zmXKmEFjYWXht1xT7ke/ v3Bo2r3ejMQ1OUbc3gPCuq2PoOfhWF/hStxTo828Saw7t7SWYfOa7bncOdh+wmD6M5CO gc9DQ9gjU1t4h9VU0v+oMCaxH9mVmXdg79Hippk+Wxx88oVshQt8QYJtNJnxwnoJAkED zAuP6llICULGEaLBMNwpZWkLu+mNOnk4d4VW4T9jACKEhE12tpGEGZN0KMltlKCSzjKN AYQtnIspiEe+7L1DHHO6uPetW7rxCv4ctAOIVATNyw7kJ1UMO2qrnMchCYpwVqQPGHNZ PgvQ== X-Forwarded-Encrypted: i=1; AJvYcCVLgIjeKHUzj+E7LFGyKmvrBxT31cQJWA0+hHPm0RP1g4ZtQF9apXN6ptIkK1dXugEvLjBc439VFTBrIkrw1oEZS5mBzArh82uXStJf X-Gm-Message-State: AOJu0Yx/uUsLPQ5ZzcW4o1t5RZibpilvYey+VFWVUv4U1lEtS5SPN/6W tdPXEfZNhANCF6uR+g/zRlKQf+hdHoA5BxEfkKSXCJdsUmzhP7yG1Hk/T4Ipq/AUH6K+FKk+45b R X-Google-Smtp-Source: AGHT+IGytyoLkOH431aR7mF4/3pxrMXnoPzF9EPSCOORKbwwc0MQGbSUWr23nIhRDL3iPMPF3Uv+te+uzcs= X-Received: from yabinc-desktop.mtv.corp.google.com ([2620:15c:211:202:6e4e:954d:1e49:f87c]) (user=yabinc job=sendgmr) by 2002:a05:6902:2b83:b0:de5:2ce1:b62d with SMTP id 3f1490d57ef6-dee4f4bfba9mr1543914276.10.1715801790944; Wed, 15 May 2024 12:36:30 -0700 (PDT) Date: Wed, 15 May 2024 12:36:08 -0700 In-Reply-To: <20240515193610.2350456-1-yabinc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240515193610.2350456-1-yabinc@google.com> X-Mailer: git-send-email 2.45.0.rc1.225.g2a3ae87e7f-goog Message-ID: <20240515193610.2350456-3-yabinc@google.com> Subject: [PATCH v5 2/3] perf/core: Check sample_type in perf_sample_save_callchain From: Yabin Cui To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Mark Rutland , Alexander Shishkin , Jiri Olsa , Ian Rogers , Adrian Hunter Cc: linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org, Yabin Cui Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Check sample_type in perf_sample_save_callchain() to prevent saving callchain data when it isn't required. Suggested-by: Namhyung Kim Signed-off-by: Yabin Cui Acked-by: Namhyung Kim --- arch/x86/events/amd/ibs.c | 3 +-- arch/x86/events/intel/ds.c | 6 ++---- include/linux/perf_event.h | 5 +++++ 3 files changed, 8 insertions(+), 6 deletions(-) diff --git a/arch/x86/events/amd/ibs.c b/arch/x86/events/amd/ibs.c index c3a2f6f57770..f02939655b2a 100644 --- a/arch/x86/events/amd/ibs.c +++ b/arch/x86/events/amd/ibs.c @@ -1129,8 +1129,7 @@ static int perf_ibs_handle_irq(struct perf_ibs *perf_= ibs, struct pt_regs *iregs) * recorded as part of interrupt regs. Thus we need to use rip from * interrupt regs while unwinding call stack. */ - if (event->attr.sample_type & PERF_SAMPLE_CALLCHAIN) - perf_sample_save_callchain(&data, event, iregs); + perf_sample_save_callchain(&data, event, iregs); =20 throttle =3D perf_event_overflow(event, &data, ®s); out: diff --git a/arch/x86/events/intel/ds.c b/arch/x86/events/intel/ds.c index e010bfed8417..c2b5585aa6d1 100644 --- a/arch/x86/events/intel/ds.c +++ b/arch/x86/events/intel/ds.c @@ -1655,8 +1655,7 @@ static void setup_pebs_fixed_sample_data(struct perf_= event *event, * previous PMI context or an (I)RET happened between the record and * PMI. */ - if (sample_type & PERF_SAMPLE_CALLCHAIN) - perf_sample_save_callchain(data, event, iregs); + perf_sample_save_callchain(data, event, iregs); =20 /* * We use the interrupt regs as a base because the PEBS record does not @@ -1823,8 +1822,7 @@ static void setup_pebs_adaptive_sample_data(struct pe= rf_event *event, * previous PMI context or an (I)RET happened between the record and * PMI. */ - if (sample_type & PERF_SAMPLE_CALLCHAIN) - perf_sample_save_callchain(data, event, iregs); + perf_sample_save_callchain(data, event, iregs); =20 *regs =3D *iregs; /* The ip in basic is EventingIP */ diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index fefac1a57b56..bda066c33120 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -1235,6 +1235,11 @@ static inline void perf_sample_save_callchain(struct= perf_sample_data *data, { int size =3D 1; =20 + if (!(event->attr.sample_type & PERF_SAMPLE_CALLCHAIN)) + return; + if (WARN_ON_ONCE(data->sample_flags & PERF_SAMPLE_CALLCHAIN)) + return; + data->callchain =3D perf_callchain(event, regs); size +=3D data->callchain->nr; =20 --=20 2.45.0.rc1.225.g2a3ae87e7f-goog From nobody Sat Feb 14 05:36:51 2026 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F2E8E15B13B for ; Wed, 15 May 2024 19:36:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715801797; cv=none; b=CsM1PuOev/zuFsN5Aj8OMwyolqDbKEaUpTULbXbAJ1au3Qf7riJUmgdA+W2gWJc8CZTRdwYguzQGIVzkiLs0y0gQ+6tPePLYusAtBfnn3F44MsbjH5Asx3FxbmsV6+Dj0N5lZsBBinUt7Rk33LmXcLk3WSEq6tiYB15y9LuvICU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715801797; c=relaxed/simple; bh=he0/u3sIGVL/RVwt13BLNfoDrXSI661dD7cIPhfWeMI=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=OT549tqQuXoJWRWrK8l5wrGOWM3zXJYq+2bqK4MUmSFDqMmPu+eIFh+atKjvJWos46rQBtc9tUZsZOH05lwxWr1pgDVUY3STmMLLYyyRksLKCX2BYIWftonHXfnKqrRKXExejF/5Mabv/zyf2P9QKf9pvyyytf/av+BZxYTvEyw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--yabinc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=uTpcroEI; arc=none smtp.client-ip=209.85.219.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--yabinc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="uTpcroEI" Received: by mail-yb1-f201.google.com with SMTP id 3f1490d57ef6-de74a2635e2so11860966276.3 for ; Wed, 15 May 2024 12:36:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1715801794; x=1716406594; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=QOOQJ5w73fGp6GcpDlduyDPXgxiHotLSQ/tbSKNqQEg=; b=uTpcroEIiBPYyGyoRuRXv4CbbyMtFk/CWKyic7H8usc4kemD/OPXGxT1o0gcFB36ik 6Uqzo22OcRKDPrB9DuLrAZBMz27YcDxiNxGpdXCJLmufZA1EhgMZnwwwRm1/p8nDN1w/ UU7+xvMYGe5QeQC/axsKnSJ2gJ/k5skNJYbMHzK/Qr8iJ1MoZpnHphtqqGr6icZK/i15 Mboc1Rgf8wTdFHDMiCboZy64VdIpCeEsz0O96mDv4GlLae7V/iqsxmx8JgBN3bzLbgYZ yra2WTqmeOUJLuQUaJW/JI8NIrShcVT9cxkDAw0IjARbMBL7w1YbJKGfpLN1wVhkwb/S hFWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715801794; x=1716406594; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=QOOQJ5w73fGp6GcpDlduyDPXgxiHotLSQ/tbSKNqQEg=; b=qHT5LDoeaqyDAryjnJDnrpG05E55IJ8WMR0RcRJUOWdoRbtfkdiENtyx6YaMqLK8QR MzK2uR3jehLWyMwvjC7qH++jadTK2WadzsVEIIpGt6NPBJ2vzmA0mbyL4WT7WqgrPtsJ stHNNiMsfINXSN+fkv3HUcIkANti2Zw2A22sOwWGo8hJNTAXWKnaLkbYe8P/6a7z+Yu7 6PH5E/vejC8ILVm/631+uYT9JByrjTSWlbGx3ebmaXCdnU5h0gPN3fhfEX3KXbGdGToj gpfvAYBZWEQ/zC7FVOomW+aq6N2zU+UseODvpUSQtCWapuF1RRw7Q2k1oRyw+R90tz05 ehXA== X-Forwarded-Encrypted: i=1; AJvYcCVTcoyliLq0qj4CqGa3nlx7cUqaVKll9Wcc+Jq6jJAtI9eZxSuHnTeRpRgGWxGfxml95Htr+VCCe3ZYP6LMsbmMqPrkHKNyq382nI30 X-Gm-Message-State: AOJu0Yyu0+kv0vEYoSminkTOyD7th5DvJHHv8NcpGyGFYAco793Q1aZz x+mHBCII2g6KZB3YpBe6dINlk7iZr1Ks+4vnxKGBOY3hfon6Ytyc/n6UCa5vn7Z5fGSJ7EKvL6h + X-Google-Smtp-Source: AGHT+IF6QmVbhQuJj9RtRACtFUo4iDmGwzViKpWJl+nMgnxXUnqlxhoXTKA1SYrr2yOInQW9GSC3sD6FIl8= X-Received: from yabinc-desktop.mtv.corp.google.com ([2620:15c:211:202:6e4e:954d:1e49:f87c]) (user=yabinc job=sendgmr) by 2002:a05:6902:1026:b0:dee:6147:7e26 with SMTP id 3f1490d57ef6-dee6bf164abmr1178181276.11.1715801794050; Wed, 15 May 2024 12:36:34 -0700 (PDT) Date: Wed, 15 May 2024 12:36:09 -0700 In-Reply-To: <20240515193610.2350456-1-yabinc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240515193610.2350456-1-yabinc@google.com> X-Mailer: git-send-email 2.45.0.rc1.225.g2a3ae87e7f-goog Message-ID: <20240515193610.2350456-4-yabinc@google.com> Subject: [PATCH v5 3/3] perf/core: Check sample_type in perf_sample_save_brstack From: Yabin Cui To: Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Mark Rutland , Alexander Shishkin , Jiri Olsa , Ian Rogers , Adrian Hunter Cc: linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org, Yabin Cui Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Check sample_type in perf_sample_save_brstack() to prevent saving branch stack data when it isn't required. Suggested-by: Namhyung Kim Signed-off-by: Yabin Cui Acked-by: Namhyung Kim --- arch/x86/events/amd/core.c | 3 +-- arch/x86/events/core.c | 3 +-- arch/x86/events/intel/ds.c | 3 +-- include/linux/perf_event.h | 15 ++++++++++----- 4 files changed, 13 insertions(+), 11 deletions(-) diff --git a/arch/x86/events/amd/core.c b/arch/x86/events/amd/core.c index 1fc4ce44e743..f71a9ef932c5 100644 --- a/arch/x86/events/amd/core.c +++ b/arch/x86/events/amd/core.c @@ -998,8 +998,7 @@ static int amd_pmu_v2_handle_irq(struct pt_regs *regs) if (!x86_perf_event_set_period(event)) continue; =20 - if (has_branch_stack(event)) - perf_sample_save_brstack(&data, event, &cpuc->lbr_stack, NULL); + perf_sample_save_brstack(&data, event, &cpuc->lbr_stack, NULL); =20 if (perf_event_overflow(event, &data, regs)) x86_pmu_stop(event, 0); diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c index 5b0dd07b1ef1..ff5577315938 100644 --- a/arch/x86/events/core.c +++ b/arch/x86/events/core.c @@ -1702,8 +1702,7 @@ int x86_pmu_handle_irq(struct pt_regs *regs) =20 perf_sample_data_init(&data, 0, event->hw.last_period); =20 - if (has_branch_stack(event)) - perf_sample_save_brstack(&data, event, &cpuc->lbr_stack, NULL); + perf_sample_save_brstack(&data, event, &cpuc->lbr_stack, NULL); =20 if (perf_event_overflow(event, &data, regs)) x86_pmu_stop(event, 0); diff --git a/arch/x86/events/intel/ds.c b/arch/x86/events/intel/ds.c index c2b5585aa6d1..f25236ffa28f 100644 --- a/arch/x86/events/intel/ds.c +++ b/arch/x86/events/intel/ds.c @@ -1754,8 +1754,7 @@ static void setup_pebs_fixed_sample_data(struct perf_= event *event, if (x86_pmu.intel_cap.pebs_format >=3D 3) setup_pebs_time(event, data, pebs->tsc); =20 - if (has_branch_stack(event)) - perf_sample_save_brstack(data, event, &cpuc->lbr_stack, NULL); + perf_sample_save_brstack(data, event, &cpuc->lbr_stack, NULL); } =20 static void adaptive_pebs_save_regs(struct pt_regs *regs, diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index bda066c33120..f6637f3cbe84 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -1276,6 +1276,11 @@ static inline void perf_sample_save_raw_data(struct = perf_sample_data *data, data->sample_flags |=3D PERF_SAMPLE_RAW; } =20 +static inline bool has_branch_stack(struct perf_event *event) +{ + return event->attr.sample_type & PERF_SAMPLE_BRANCH_STACK; +} + static inline void perf_sample_save_brstack(struct perf_sample_data *data, struct perf_event *event, struct perf_branch_stack *brs, @@ -1283,6 +1288,11 @@ static inline void perf_sample_save_brstack(struct p= erf_sample_data *data, { int size =3D sizeof(u64); /* nr */ =20 + if (!has_branch_stack(event)) + return; + if (WARN_ON_ONCE(data->sample_flags & PERF_SAMPLE_BRANCH_STACK)) + return; + if (branch_sample_hw_index(event)) size +=3D sizeof(u64); size +=3D brs->nr * sizeof(struct perf_branch_entry); @@ -1658,11 +1668,6 @@ extern void perf_bp_event(struct perf_event *event, = void *data); # define perf_arch_bpf_user_pt_regs(regs) regs #endif =20 -static inline bool has_branch_stack(struct perf_event *event) -{ - return event->attr.sample_type & PERF_SAMPLE_BRANCH_STACK; -} - static inline bool needs_branch_stack(struct perf_event *event) { return event->attr.branch_sample_type !=3D 0; --=20 2.45.0.rc1.225.g2a3ae87e7f-goog