From nobody Tue Dec 23 16:20:57 2025 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C2805E545 for ; Mon, 13 Jan 2025 05:23:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.202 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736745792; cv=none; b=kLQ1oW76LDIc6znDUqI4STzGLHbJSHoYPYzl/mv1yjC8msrfaUblot/L/qztMS7FUAaAXu3So028i32S8C+T3Pq/ybU/U9RHRAymhs/XhLTKA3d2DE0AJ+9d32Fx4u8xVAzUIEdLbG1RaITMR6TjD7wBcktQ+DV5Vf+c038sO1s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1736745792; c=relaxed/simple; bh=pQOb1PSWvkn5i2/mDD/OCElWZVdAqdPQiu1a5zPnVkc=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=m4vzpgmk0/wsRpE+rHw9HXPldKqwxE3M0uYX6ogVP27195qxNZMJvBdK4XhNYL7Fn9GICcVkHUIwNWYjfmcDSTDVoiLkADS3wBQstLX2DUg/xYYbaFaNr776rlCUarixo9gQMYPqI1aTTj7wQ569rCdi21bJbhCiYi5mxreAUEg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ctshao.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=qGDb84OK; arc=none smtp.client-ip=209.85.214.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ctshao.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="qGDb84OK" Received: by mail-pl1-f202.google.com with SMTP id d9443c01a7336-2162f80040aso64494165ad.1 for ; Sun, 12 Jan 2025 21:23:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1736745790; x=1737350590; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=VVfSqsB1425hgeh1CCvgyoytNRCKm7B7/XnFcKeLkPw=; b=qGDb84OKMaH8E/XPWhfu6YqFC4903e1J0YeLAVZufEHbCQITZdpsmTI3z1RspsomWK pAZI5NMU0b8fY7WS60l9w2ub9l3+Q7OEU9wAJnn5bQh3Vz68IC53pFP4SYmppOBMF7Be tzqvqJtmS5XI6/cNy1G7kf+GXpJCnJtyHNTuUBXagolnTl4cVgg9qhx+hMzeJdr7AjGI f2ZUS51bNbl3sOW13M0xTbFdWEsmhxJoK0OIH6bZHvXRutudk66oyVYOJ5zdy39HFlI8 2xFR26QVT9fk24FClofRzWQi5bHhNpDTu8w1V3rglK/IYLU+we79EPn5VDpJ2xGWxfTz 502w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1736745790; x=1737350590; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=VVfSqsB1425hgeh1CCvgyoytNRCKm7B7/XnFcKeLkPw=; b=SZ1u3C69rluKHn3znFpCDRBmV1sC+Te93x/NeugD6YVLff07eXF7mQQht5bqebiPhi 7qSghRzdxN2S5JXaoDGqtLkH11ABqt19pcMCOM91xdHfDfpIKN7I8gxdQ2iKhfATvkxN 3m8ZlyrmjmO2PHqRaJ9NxYlzUlj/9uESd7jRYc+4K8a3hU44mzn1qNkz/Cqs6KWlq2sY HJ4XSOgqqDcS7rF0eyDuaP8iuU84xxlw2rkcILZh3SV/Ng5WWBFA1lzxbD0BiJt79nDS lTXxttIC7oAutV+1HA+ZuCnLnP235885R9eUCySiT+9VQjtOSVtzPH9d6TpLbrWfrYjx fIqA== X-Gm-Message-State: AOJu0YxBqNKzwChuCtFJ+0ULsE35jowiJdBiT+Ti1yiLcW2Stbx036KA C8jIuINcs0m4F07zeE6uap/V2gno8ofXmO2b9JUfRNu2Cfk2cDmukzfBSqArejt5LBQGtsdsN3c 9A8yJ/AoVcQFALZ0DI9IoWNNdQkFAb3aXWhwpAm2Er7EFTTVUaTg+95v4H0PQOPaiK7BGjkNmGQ xWDNb6oKxy/f5h22Xn4xH3BGNJUekq7SdQw8idJzWT X-Google-Smtp-Source: AGHT+IEZWkENHpl86IzSgwf0OFY8eoSHS59AUCniUII1KvxAtH7cndK43/Nilx8npxLdrV5enN/fkbSZh4M= X-Received: from pgbgb2.prod.google.com ([2002:a05:6a02:4b42:b0:86d:55d8:7944]) (user=ctshao job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:ecc7:b0:216:3b31:56c2 with SMTP id d9443c01a7336-21a83fded24mr314809615ad.53.1736745790291; Sun, 12 Jan 2025 21:23:10 -0800 (PST) Date: Sun, 12 Jan 2025 21:20:15 -0800 In-Reply-To: <20250113052220.2105645-1-ctshao@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250113052220.2105645-1-ctshao@google.com> X-Mailer: git-send-email 2.47.1.688.g23fc6f90ad-goog Message-ID: <20250113052220.2105645-3-ctshao@google.com> Subject: [PATCH v2 2/4] perf lock: Retrieve owner callstack in bpf program From: Chun-Tse Shao To: linux-kernel@vger.kernel.org Cc: Chun-Tse Shao , peterz@infradead.org, mingo@redhat.com, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com, linux-perf-users@vger.kernel.org, bpf@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Tracing owner callstack in `contention_begin()` and `contention_end()`, and storing in `owner_lock_stat` bpf map. Signed-off-by: Chun-Tse Shao --- .../perf/util/bpf_skel/lock_contention.bpf.c | 152 +++++++++++++++++- 1 file changed, 151 insertions(+), 1 deletion(-) diff --git a/tools/perf/util/bpf_skel/lock_contention.bpf.c b/tools/perf/ut= il/bpf_skel/lock_contention.bpf.c index 05da19fdab23..3f47fbfa237c 100644 --- a/tools/perf/util/bpf_skel/lock_contention.bpf.c +++ b/tools/perf/util/bpf_skel/lock_contention.bpf.c @@ -7,6 +7,7 @@ #include =20 #include "lock_data.h" +#include =20 /* for collect_lock_syms(). 4096 was rejected by the verifier */ #define MAX_CPUS 1024 @@ -178,6 +179,9 @@ int data_fail; int task_map_full; int data_map_full; =20 +struct task_struct *bpf_task_from_pid(s32 pid) __ksym; +void bpf_task_release(struct task_struct *p) __ksym; + static inline __u64 get_current_cgroup_id(void) { struct task_struct *task; @@ -407,6 +411,60 @@ int contention_begin(u64 *ctx) pelem->flags =3D (__u32)ctx[1]; =20 if (needs_callstack) { + u32 i =3D 0; + int owner_pid; + unsigned long *entries; + struct task_struct *task; + cotd *data; + + if (!lock_owner) + goto contention_begin_skip_owner_callstack; + + task =3D get_lock_owner(pelem->lock, pelem->flags); + if (!task) + goto contention_begin_skip_owner_callstack; + + owner_pid =3D BPF_CORE_READ(task, pid); + + entries =3D bpf_map_lookup_elem(&owner_stacks_entries, &i); + if (!entries) + goto contention_begin_skip_owner_callstack; + for (i =3D 0; i < max_stack; i++) + entries[i] =3D 0x0; + + task =3D bpf_task_from_pid(owner_pid); + if (task) { + bpf_get_task_stack(task, entries, + max_stack * sizeof(unsigned long), + 0); + bpf_task_release(task); + } + + data =3D bpf_map_lookup_elem(&contention_owner_tracing, + &(pelem->lock)); + + // Contention just happens, or corner case `lock` is owned by + // process not `owner_pid`. + if (!data || data->pid !=3D owner_pid) { + cotd first =3D { + .pid =3D owner_pid, + .timestamp =3D pelem->timestamp, + .count =3D 1, + }; + bpf_map_update_elem(&contention_owner_tracing, + &(pelem->lock), &first, BPF_ANY); + bpf_map_update_elem(&contention_owner_stacks, + &(pelem->lock), entries, BPF_ANY); + } + // Contention is going on and new waiter joins. + else { + __sync_fetch_and_add(&data->count, 1); + // TODO: Since for owner the callstack would change at + // different time, We should check and report if the + // callstack is different with the recorded one in + // `contention_owner_stacks`. + } +contention_begin_skip_owner_callstack: pelem->stack_id =3D bpf_get_stackid(ctx, &stacks, BPF_F_FAST_STACK_CMP | stack_skip); if (pelem->stack_id < 0) @@ -443,6 +501,7 @@ int contention_end(u64 *ctx) struct tstamp_data *pelem; struct contention_key key =3D {}; struct contention_data *data; + __u64 timestamp; __u64 duration; bool need_delete =3D false; =20 @@ -469,12 +528,103 @@ int contention_end(u64 *ctx) return 0; need_delete =3D true; } - duration =3D bpf_ktime_get_ns() - pelem->timestamp; + timestamp =3D bpf_ktime_get_ns(); + duration =3D timestamp - pelem->timestamp; if ((__s64)duration < 0) { __sync_fetch_and_add(&time_fail, 1); goto out; } =20 + if (needs_callstack && lock_owner) { + u64 owner_contention_time; + unsigned long *owner_stack; + struct contention_data *cdata; + cotd *otdata; + + otdata =3D bpf_map_lookup_elem(&contention_owner_tracing, + &(pelem->lock)); + owner_stack =3D bpf_map_lookup_elem(&contention_owner_stacks, + &(pelem->lock)); + if (!otdata || !owner_stack) + goto contention_end_skip_owner_callstack; + + owner_contention_time =3D timestamp - otdata->timestamp; + + // Update `owner_lock_stat` if `owner_stack` is + // available. + if (owner_stack[0] !=3D 0x0) { + cdata =3D bpf_map_lookup_elem(&owner_lock_stat, + owner_stack); + if (!cdata) { + struct contention_data first =3D { + .total_time =3D owner_contention_time, + .max_time =3D owner_contention_time, + .min_time =3D owner_contention_time, + .count =3D 1, + .flags =3D pelem->flags, + }; + bpf_map_update_elem(&owner_lock_stat, + owner_stack, &first, + BPF_ANY); + } else { + __sync_fetch_and_add(&cdata->total_time, + owner_contention_time); + __sync_fetch_and_add(&cdata->count, 1); + + /* FIXME: need atomic operations */ + if (cdata->max_time < owner_contention_time) + cdata->max_time =3D owner_contention_time; + if (cdata->min_time > owner_contention_time) + cdata->min_time =3D owner_contention_time; + } + } + + // No contention is going on, delete `lock` in + // `contention_owner_tracing` and + // `contention_owner_stacks` + if (otdata->count <=3D 1) { + bpf_map_delete_elem(&contention_owner_tracing, + &(pelem->lock)); + bpf_map_delete_elem(&contention_owner_stacks, + &(pelem->lock)); + } + // Contention is still going on, with a new owner + // (current task). `otdata` should be updated accordingly. + else { + (otdata->count)--; + + // If ctx[1] is not 0, the current task terminates lock + // waiting without acquiring it. Owner is not changed. + if (ctx[1] =3D=3D 0) { + u32 i =3D 0; + unsigned long *entries =3D bpf_map_lookup_elem( + &owner_stacks_entries, &i); + if (entries) { + for (i =3D 0; i < (u32)max_stack; i++) + entries[i] =3D 0x0; + + bpf_get_task_stack( + bpf_get_current_task_btf(), + entries, + max_stack * + sizeof(unsigned long), + 0); + bpf_map_update_elem( + &contention_owner_stacks, + &(pelem->lock), entries, + BPF_ANY); + } + + otdata->pid =3D pid; + otdata->timestamp =3D timestamp; + } + + bpf_map_update_elem(&contention_owner_tracing, + &(pelem->lock), otdata, BPF_ANY); + } + } +contention_end_skip_owner_callstack: + switch (aggr_mode) { case LOCK_AGGR_CALLER: key.stack_id =3D pelem->stack_id; --=20 2.47.1.688.g23fc6f90ad-goog