From nobody Sat Feb 7 14:04:26 2026 Received: from out-182.mta0.migadu.com (out-182.mta0.migadu.com [91.218.175.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F235C303C81 for ; Sun, 9 Nov 2025 16:36:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=91.218.175.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762706190; cv=none; b=oaEbmojZ/Hvgbd4mVADscaE3mtZNpfuEE15SYg2uwXdBfKZWpjxQMUXG0xj3lyTTiJIwgYoH3nm8tiSIpQz5G+CQDy9JGEP4yzPh6M34/kfWD4TADWPCkX23y9oKSJYkFjq8Q5XoLhv8I4gLBd5b1QZ0+OSdf1dipyrZy7tJOY0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762706190; c=relaxed/simple; bh=Mxr2x+lJJSwgGGpqUmT96at9v4/PYnLJ6k8ngEK7Vkc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pnGC/Sy6BP0RSkaJup1HITEZGsFg8laZCNnKgCe4nggB3+kwTcZDNzlSuSYB6Gw70IKWUO4ivp3zTrPWY4ahk4wLB8pVzVMePvm9TTQUdLo6YGcBpjfp9f5BBqVnq+wy8r8E01vpALMvSQbZp6Lo9nAhoNt/L3dHEhIgjw+9kqg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=DpeOT4eH; arc=none smtp.client-ip=91.218.175.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="DpeOT4eH" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1762706185; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=39rZiArGXeCxXvKdQzId38fSf/Sj1+1oORkoOlR75MI=; b=DpeOT4eHi+kcZoRKM50QoWCedVdNWMVcdKtcq2Ee+1KRyrdJSh9sUgGzAApP6xJkVVeAlU 8lKzt5oNPszxU5ykQxbIto4dZBgcdkmvFE0vEF0g9vHVbb1fcRd+cjr0F8yngOIBr+F7N7 xt3/Cv/AU9dXE+nzW4EVOg4DN+UiHsk= From: Tao Chen To: peterz@infradead.org, mingo@redhat.com, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com Cc: linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org, Tao Chen Subject: [PATCH bpf-next v5 1/3] perf: Refactor get_perf_callchain Date: Mon, 10 Nov 2025 00:35:57 +0800 Message-ID: <20251109163559.4102849-2-chen.dylane@linux.dev> In-Reply-To: <20251109163559.4102849-1-chen.dylane@linux.dev> References: <20251109163559.4102849-1-chen.dylane@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" From BPF stack map, we want to ensure that the callchain buffer will not be overwritten by other preemptive tasks. Peter suggested provide more flexible stack-sampling APIs, which can be used in BPF, and we can still use the perf callchain entry with the help of these APIs. The next patch will modify the BPF part. In the future, these APIs will also make it convenient for us to add stack-sampling kfuncs in the eBPF subsystem, just as Andrii and Alexei discussed earlier. Signed-off-by: Peter Zijlstra Signed-off-by: Tao Chen --- include/linux/perf_event.h | 9 +++++ kernel/events/callchain.c | 73 ++++++++++++++++++++++++-------------- 2 files changed, 56 insertions(+), 26 deletions(-) diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index fd1d91017b9..edd3058e4d8 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -67,6 +67,7 @@ struct perf_callchain_entry_ctx { u32 nr; short contexts; bool contexts_maxed; + bool add_mark; }; =20 typedef unsigned long (*perf_copy_f)(void *dst, const void *src, @@ -1718,6 +1719,14 @@ DECLARE_PER_CPU(struct perf_callchain_entry, perf_ca= llchain_entry); =20 extern void perf_callchain_user(struct perf_callchain_entry_ctx *entry, st= ruct pt_regs *regs); extern void perf_callchain_kernel(struct perf_callchain_entry_ctx *entry, = struct pt_regs *regs); + +extern void __init_perf_callchain_ctx(struct perf_callchain_entry_ctx *ctx, + struct perf_callchain_entry *entry, + u32 max_stack, bool add_mark); + +extern void __get_perf_callchain_kernel(struct perf_callchain_entry_ctx *c= tx, struct pt_regs *regs); +extern void __get_perf_callchain_user(struct perf_callchain_entry_ctx *ctx= , struct pt_regs *regs); + extern struct perf_callchain_entry * get_perf_callchain(struct pt_regs *regs, bool kernel, bool user, u32 max_stack, bool crosstask, bool add_mark); diff --git a/kernel/events/callchain.c b/kernel/events/callchain.c index 808c0d7a31f..fb1f26be297 100644 --- a/kernel/events/callchain.c +++ b/kernel/events/callchain.c @@ -216,13 +216,54 @@ static void fixup_uretprobe_trampoline_entries(struct= perf_callchain_entry *entr #endif } =20 +void __init_perf_callchain_ctx(struct perf_callchain_entry_ctx *ctx, + struct perf_callchain_entry *entry, + u32 max_stack, bool add_mark) + +{ + ctx->entry =3D entry; + ctx->max_stack =3D max_stack; + ctx->nr =3D entry->nr =3D 0; + ctx->contexts =3D 0; + ctx->contexts_maxed =3D false; + ctx->add_mark =3D add_mark; +} + +void __get_perf_callchain_kernel(struct perf_callchain_entry_ctx *ctx, str= uct pt_regs *regs) +{ + if (user_mode(regs)) + return; + + if (ctx->add_mark) + perf_callchain_store_context(ctx, PERF_CONTEXT_KERNEL); + perf_callchain_kernel(ctx, regs); +} + +void __get_perf_callchain_user(struct perf_callchain_entry_ctx *ctx, struc= t pt_regs *regs) +{ + int start_entry_idx; + + if (!user_mode(regs)) { + if (current->flags & (PF_KTHREAD | PF_USER_WORKER)) + return; + regs =3D task_pt_regs(current); + } + + if (ctx->add_mark) + perf_callchain_store_context(ctx, PERF_CONTEXT_USER); + + start_entry_idx =3D ctx->nr; + perf_callchain_user(ctx, regs); + fixup_uretprobe_trampoline_entries(ctx->entry, start_entry_idx); +} + struct perf_callchain_entry * get_perf_callchain(struct pt_regs *regs, bool kernel, bool user, u32 max_stack, bool crosstask, bool add_mark) { struct perf_callchain_entry *entry; struct perf_callchain_entry_ctx ctx; - int rctx, start_entry_idx; + int rctx; =20 /* crosstask is not supported for user stacks */ if (crosstask && user && !kernel) @@ -232,34 +273,14 @@ get_perf_callchain(struct pt_regs *regs, bool kernel,= bool user, if (!entry) return NULL; =20 - ctx.entry =3D entry; - ctx.max_stack =3D max_stack; - ctx.nr =3D entry->nr =3D 0; - ctx.contexts =3D 0; - ctx.contexts_maxed =3D false; + __init_perf_callchain_ctx(&ctx, entry, max_stack, add_mark); =20 - if (kernel && !user_mode(regs)) { - if (add_mark) - perf_callchain_store_context(&ctx, PERF_CONTEXT_KERNEL); - perf_callchain_kernel(&ctx, regs); - } - - if (user && !crosstask) { - if (!user_mode(regs)) { - if (current->flags & (PF_KTHREAD | PF_USER_WORKER)) - goto exit_put; - regs =3D task_pt_regs(current); - } + if (kernel) + __get_perf_callchain_kernel(&ctx, regs); =20 - if (add_mark) - perf_callchain_store_context(&ctx, PERF_CONTEXT_USER); - - start_entry_idx =3D entry->nr; - perf_callchain_user(&ctx, regs); - fixup_uretprobe_trampoline_entries(entry, start_entry_idx); - } + if (user && !crosstask) + __get_perf_callchain_user(&ctx, regs); =20 -exit_put: put_callchain_entry(rctx); =20 return entry; --=20 2.48.1 From nobody Sat Feb 7 14:04:26 2026 Received: from out-179.mta1.migadu.com (out-179.mta1.migadu.com [95.215.58.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5B4D6303C81 for ; Sun, 9 Nov 2025 16:37:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762706224; cv=none; b=rvgbSsKT94QV0Rk+axso2qsY+hERS6CdmH+LdOr/U3vFYVnrhdRpbStuz8Yujt2PNA3kDqcUZBAKFTFWCbZvM6qhk+P6V27F7Meqjg/d140R8BHU2FsvCrAau6ceo+S+FGKkqDISk4EBMYo+KnekD8rgSxUQL66Jfdi3yDtQNXc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762706224; c=relaxed/simple; bh=zMlAgK7Vfadh2JGekv30Yi3HICQKbllq6IxAioW86y4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BioTRaF8MMZ1z8bCwWvTcKGlFEnSJWPTlQ3X715xoKYpD+KjX1lWRU+9F4HcmhBbAjBO99JibkRsKm8bPk4WtU+i+c2a1w0tYEn7g9wf1PyxSe130p+5nboedRHTKFCwao7g4Ga3ymyqeuMcMZu4r9T+JNncaGYIrXNmoc9Fwm4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=ZjgQ5Bhx; arc=none smtp.client-ip=95.215.58.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="ZjgQ5Bhx" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1762706209; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5VAP3oB7GpBydxTF8/qJglhhf0NNtnfVJXegyQuiKgU=; b=ZjgQ5BhxFucJEi1DMFumNdX6jv1H42gt1f0a+84UtwSVFkPt/iBQjj4bGfwmj7LtoDI8kj W5xjz0CYwmOGcypnCZ/TbSodB7A5OjbwEYntS2Qhptp9SmZvZLlfqAiF2lPut9Yofb9TmT HJ/wb9HsLzGVXp5Yd8jQ4iHZmx2D9BU= From: Tao Chen To: peterz@infradead.org, mingo@redhat.com, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com Cc: linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org, Tao Chen , Yonghong Song Subject: [PATCH bpf-next v5 2/3] perf: Add atomic operation in get_recursion_context Date: Mon, 10 Nov 2025 00:35:58 +0800 Message-ID: <20251109163559.4102849-3-chen.dylane@linux.dev> In-Reply-To: <20251109163559.4102849-1-chen.dylane@linux.dev> References: <20251109163559.4102849-1-chen.dylane@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" From BPF side, preemption usually is enabled. Yonghong said, it is possible that both tasks (at process level) may reach right before "recursion[rctx]++;". In such cases, both tasks will be able to get buffer and this is not right. Signed-off-by: Yonghong Song Signed-off-by: Tao Chen --- kernel/events/internal.h | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/kernel/events/internal.h b/kernel/events/internal.h index d9cc5708309..684bde972ba 100644 --- a/kernel/events/internal.h +++ b/kernel/events/internal.h @@ -214,12 +214,9 @@ static inline int get_recursion_context(u8 *recursion) { unsigned char rctx =3D interrupt_context_level(); =20 - if (recursion[rctx]) + if (cmpxchg(&recursion[rctx], 0, 1) !=3D 0) return -1; =20 - recursion[rctx]++; - barrier(); - return rctx; } =20 --=20 2.48.1 From nobody Sat Feb 7 14:04:26 2026 Received: from out-179.mta1.migadu.com (out-179.mta1.migadu.com [95.215.58.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 50421303C81 for ; Sun, 9 Nov 2025 16:37:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=95.215.58.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762706236; cv=none; b=KWEaHJuf27uWciq9lVLRSMdCakINv1+RKPErJ9GLxpahxA7eXpGByEndqdZiPGoubM7LaaRR4ZNrOrx5qXn03p1AQiOTg5NAjefPR/H9OCHlDh74lHc8gPl7HFwqVqCaXD75IZPWzsuEFmeGEENcwrv+8qsfyJiK/qyibJFid4M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1762706236; c=relaxed/simple; bh=hNAWHlmQuvUiJrfA36o2iOZOu3sIu3Ko/4lg5IxIDTM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=plbsiuPrYgj5s+GsB1FlgUVNxDi1a7vW7ObcSU9jtSDy1wwHH+Ql+pHrlSRQgEQN7f+SIE+AquNLemfWNzh6C1giyWdobAf93mfgJrc3PZiy6kDUTZ/R/wGfmP0oEeNoS3gf3Uteq/TT40xPh7MH+VDPsHdvlu6LFBXy7RCp9qQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev; spf=pass smtp.mailfrom=linux.dev; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b=TwveU4X2; arc=none smtp.client-ip=95.215.58.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.dev Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.dev Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=linux.dev header.i=@linux.dev header.b="TwveU4X2" X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1762706231; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=K+FAv/HqxyqKXOjlEtMLSRilYuA/JiP8B+dO7mrVenw=; b=TwveU4X20p1iQZm6jneWjjD85e9m1xPGKJ2wF4jlbQRVn9pJI/+6mzuP7utgw8xay9P5a5 zL9WpGQ0CvB2tN9kBU9d4WxIp2jLKAi7ucG4xT5DJmbSb97GfErpSzF0+rVjL/T3WWqQev A+8uuXh/XvC3fACdkCLcTFI0aQSh/2M= From: Tao Chen To: peterz@infradead.org, mingo@redhat.com, acme@kernel.org, namhyung@kernel.org, mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org, irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com Cc: linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org, bpf@vger.kernel.org, Tao Chen Subject: [PATCH bpf-next v5 3/3] bpf: Hold the perf callchain entry until used completely Date: Mon, 10 Nov 2025 00:35:59 +0800 Message-ID: <20251109163559.4102849-4-chen.dylane@linux.dev> In-Reply-To: <20251109163559.4102849-1-chen.dylane@linux.dev> References: <20251109163559.4102849-1-chen.dylane@linux.dev> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Migadu-Flow: FLOW_OUT Content-Type: text/plain; charset="utf-8" As Alexei noted, get_perf_callchain() return values may be reused if a task is preempted after the BPF program enters migrate disable mode. The perf_callchain_entres has a small stack of entries, and we can reuse it as follows: 1. get the perf callchain entry 2. BPF use... 3. put the perf callchain entry Signed-off-by: Tao Chen --- kernel/bpf/stackmap.c | 62 ++++++++++++++++++++++++++++++++++--------- 1 file changed, 50 insertions(+), 12 deletions(-) diff --git a/kernel/bpf/stackmap.c b/kernel/bpf/stackmap.c index 2365541c81d..58b4432ab00 100644 --- a/kernel/bpf/stackmap.c +++ b/kernel/bpf/stackmap.c @@ -210,13 +210,12 @@ static void stack_map_get_build_id_offset(struct bpf_= stack_build_id *id_offs, } =20 static struct perf_callchain_entry * -get_callchain_entry_for_task(struct task_struct *task, u32 max_depth) +get_callchain_entry_for_task(int *rctx, struct task_struct *task, u32 max_= depth) { #ifdef CONFIG_STACKTRACE struct perf_callchain_entry *entry; - int rctx; =20 - entry =3D get_callchain_entry(&rctx); + entry =3D get_callchain_entry(rctx); =20 if (!entry) return NULL; @@ -238,8 +237,6 @@ get_callchain_entry_for_task(struct task_struct *task, = u32 max_depth) to[i] =3D (u64)(from[i]); } =20 - put_callchain_entry(rctx); - return entry; #else /* CONFIG_STACKTRACE */ return NULL; @@ -320,6 +317,31 @@ static long __bpf_get_stackid(struct bpf_map *map, return id; } =20 +static struct perf_callchain_entry * +bpf_get_perf_callchain(int *rctx, struct pt_regs *regs, bool kernel, bool = user, + int max_stack, bool crosstask) +{ + struct perf_callchain_entry_ctx ctx; + struct perf_callchain_entry *entry; + + entry =3D get_callchain_entry(rctx); + if (unlikely(!entry)) + return NULL; + + __init_perf_callchain_ctx(&ctx, entry, max_stack, false); + if (kernel) + __get_perf_callchain_kernel(&ctx, regs); + if (user && !crosstask) + __get_perf_callchain_user(&ctx, regs); + + return entry; +} + +static void bpf_put_perf_callchain(int rctx) +{ + put_callchain_entry(rctx); +} + BPF_CALL_3(bpf_get_stackid, struct pt_regs *, regs, struct bpf_map *, map, u64, flags) { @@ -328,20 +350,24 @@ BPF_CALL_3(bpf_get_stackid, struct pt_regs *, regs, s= truct bpf_map *, map, struct perf_callchain_entry *trace; bool kernel =3D !user; u32 max_depth; + int rctx, ret; =20 if (unlikely(flags & ~(BPF_F_SKIP_FIELD_MASK | BPF_F_USER_STACK | BPF_F_FAST_STACK_CMP | BPF_F_REUSE_STACKID))) return -EINVAL; =20 max_depth =3D stack_map_calculate_max_depth(map->value_size, elem_size, f= lags); - trace =3D get_perf_callchain(regs, kernel, user, max_depth, - false, false); + trace =3D bpf_get_perf_callchain(&rctx, regs, kernel, user, max_depth, + false); =20 if (unlikely(!trace)) /* couldn't fetch the stack trace */ return -EFAULT; =20 - return __bpf_get_stackid(map, trace, flags); + ret =3D __bpf_get_stackid(map, trace, flags); + bpf_put_perf_callchain(rctx); + + return ret; } =20 const struct bpf_func_proto bpf_get_stackid_proto =3D { @@ -435,6 +461,7 @@ static long __bpf_get_stack(struct pt_regs *regs, struc= t task_struct *task, bool kernel =3D !user; int err =3D -EINVAL; u64 *ips; + int rctx; =20 if (unlikely(flags & ~(BPF_F_SKIP_FIELD_MASK | BPF_F_USER_STACK | BPF_F_USER_BUILD_ID))) @@ -467,18 +494,26 @@ static long __bpf_get_stack(struct pt_regs *regs, str= uct task_struct *task, trace =3D trace_in; trace->nr =3D min_t(u32, trace->nr, max_depth); } else if (kernel && task) { - trace =3D get_callchain_entry_for_task(task, max_depth); + trace =3D get_callchain_entry_for_task(&rctx, task, max_depth); } else { - trace =3D get_perf_callchain(regs, kernel, user, max_depth, - crosstask, false); + trace =3D bpf_get_perf_callchain(&rctx, regs, kernel, user, max_depth, + crosstask); } =20 - if (unlikely(!trace) || trace->nr < skip) { + if (unlikely(!trace)) { if (may_fault) rcu_read_unlock(); goto err_fault; } =20 + if (trace->nr < skip) { + if (may_fault) + rcu_read_unlock(); + if (!trace_in) + bpf_put_perf_callchain(rctx); + goto err_fault; + } + trace_nr =3D trace->nr - skip; copy_len =3D trace_nr * elem_size; =20 @@ -497,6 +532,9 @@ static long __bpf_get_stack(struct pt_regs *regs, struc= t task_struct *task, if (may_fault) rcu_read_unlock(); =20 + if (!trace_in) + bpf_put_perf_callchain(rctx); + if (user_build_id) stack_map_get_build_id_offset(buf, trace_nr, user, may_fault); =20 --=20 2.48.1