From nobody Thu Sep 11 18:24:14 2025 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 94991C76196 for ; Fri, 7 Apr 2023 23:06:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229596AbjDGXGI (ORCPT ); Fri, 7 Apr 2023 19:06:08 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59152 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229451AbjDGXGB (ORCPT ); Fri, 7 Apr 2023 19:06:01 -0400 Received: from mail-pj1-x1049.google.com (mail-pj1-x1049.google.com [IPv6:2607:f8b0:4864:20::1049]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A8F66E07B for ; Fri, 7 Apr 2023 16:05:32 -0700 (PDT) Received: by mail-pj1-x1049.google.com with SMTP id s93-20020a17090a2f6600b0024670ac71caso141403pjd.4 for ; Fri, 07 Apr 2023 16:05:32 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1680908732; x=1683500732; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=T+Aq5EGpv0TdcyR3mJ0rG0U8bcZUTZ5BKGaHK0JrYVc=; b=joThEOBHVXoPtxtwqaFrm5dG382oUEuC1Z7udKCQMpKFT/BWvfI6nQPyjLU1LUL3te mBpw/CtuNM6bTwwDxWFVQdel2qstpt+P+m0KSEjKzA9E3PppJLphsjyrfJ5wk4ihmOml TvxpslPz0hEt4peTxtTYvBgcR4ns/Xv53JrIA4P71lrj4d3hUW7HgxVuTeOugl/9aWHw 5P4p+m3t/xDpCLHV/HmizJ+/zeD6yq+8GlycZd+pdDDku7Nu7TF3T8m1rGCuzJBXPBvZ TuPavi2Mt4Led/cG7CKvhPJy1AYMopuj+yb4W7zxIq9/TUAKGi5gTCFR02m4UWbKeQ+b KNmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680908732; x=1683500732; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=T+Aq5EGpv0TdcyR3mJ0rG0U8bcZUTZ5BKGaHK0JrYVc=; b=CRzXcqWfOToiR1iju53KK9GnGckRXyGBdlxi9z3TnIV0rPJnaNPttUlvWqfBqLSLJu ZyKcTh0mUinO8vS6TgKNv/yKfmDfWKVgsReEqbhI/zU6HVZu1D4EWuHdA1Xv0YfeYGyj 4jkJaSl+fRmg2+txni+pdkbn+7ZJDvuMQH8UfduBQr/SWTfof+mdUZeUEAxg2PXMl8NU JRKtkqK7pWhlGiFP5VQkitjdDHh1R2P0qC+7f8ltyNn4ffYZKq5K3tt4IlF/ATWpssUq 6+V0DfsnJ/ToH0jcAYlCQt7+5qmBoyy2/lxMO1dioAYeSZgQOWr8X3E7/NbyA/1wrZ6X LCqQ== X-Gm-Message-State: AAQBX9fkPmmIPDLdjDmHm6SXWMU1TitQWyYqSpT1bMNsvjREMBTQTBh7 cloV40ePnmvCI9UEbHXt6ZgDqjKq7rRdbXbeuoqA+2jiukzn/j2H4ywVwglxxEpKg/QtFe9rNT8 NgukkxPLa4Ob94oIVEc80roYToLaFRyRBNmTePZ0sWWQZ/vXoL+RsYwJPI0t8yneoZCxSTUk= X-Google-Smtp-Source: AKy350Yj18hj/bhFg7Ob768bL9JlrO2l4hoCellvR18Ea82ZFABDTbikdhH6gftC1NNZaAAneg89KZd0wMGH X-Received: from jstultz-noogler2.c.googlers.com ([fda3:e722:ac3:cc00:24:72f4:c0a8:600]) (user=jstultz job=sendgmr) by 2002:a63:58e:0:b0:50b:e80f:caff with SMTP id 136-20020a63058e000000b0050be80fcaffmr3764515pgf.0.1680908731909; Fri, 07 Apr 2023 16:05:31 -0700 (PDT) Date: Fri, 7 Apr 2023 23:05:26 +0000 Mime-Version: 1.0 X-Mailer: git-send-email 2.40.0.577.gac1e443424-goog Message-ID: <20230407230526.1685443-1-jstultz@google.com> Subject: [RESEND x4][PATCH v3] trace: Add trace points for tasklet entry/exit From: John Stultz To: LKML Cc: Lingutla Chandrasekhar , Steven Rostedt , Masami Hiramatsu , Thomas Gleixner , "Paul E. McKenney" , kernel-team@android.com, "J . Avila" , John Stultz Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Lingutla Chandrasekhar Tasklets are supposed to finish their work quickly and should not block the current running process, but it is not guaranteed that. Currently softirq_entry/exit can be used to know total tasklets execution time, but not helpful to track individual tasklet's execution time. With that we can't find any culprit tasklet function, which is taking more time. Add tasklet_entry/exit trace point support to track individual tasklet execution. Trivial usage example: # echo 1 > /sys/kernel/debug/tracing/events/irq/tasklet_entry/enable # echo 1 > /sys/kernel/debug/tracing/events/irq/tasklet_exit/enable # cat /sys/kernel/debug/tracing/trace # tracer: nop # # entries-in-buffer/entries-written: 4/4 #P:4 # # _-----=3D> irqs-off/BH-disabled # / _----=3D> need-resched # | / _---=3D> hardirq/softirq # || / _--=3D> preempt-depth # ||| / _-=3D> migrate-disable # |||| / delay # TASK-PID CPU# ||||| TIMESTAMP FUNCTION # | | | ||||| | | -0 [003] ..s1. 314.011428: tasklet_entry: tasklet= =3D0xffffa01ef8db2740 function=3Dtcp_tasklet_func -0 [003] ..s1. 314.011432: tasklet_exit: tasklet= =3D0xffffa01ef8db2740 function=3Dtcp_tasklet_func -0 [003] ..s1. 314.017369: tasklet_entry: tasklet= =3D0xffffa01ef8db2740 function=3Dtcp_tasklet_func -0 [003] ..s1. 314.017371: tasklet_exit: tasklet= =3D0xffffa01ef8db2740 function=3Dtcp_tasklet_func This patch has been carried in the Android tree for awhile so I wanted to submit it for review upstream. Feedback would be appreciated! Cc: Steven Rostedt Cc: Masami Hiramatsu Cc: Thomas Gleixner Cc: "Paul E. McKenney" Cc: kernel-team@android.com Reviewed-by: Steven Rostedt (Google) Signed-off-by: Lingutla Chandrasekhar [elavila: Port to android-mainline] Signed-off-by: J. Avila [jstultz: Rebased to upstream, cut unused trace points, added comments for the tracepoints, reworded commit] Signed-off-by: John Stultz --- "Helloooooo..." he yelled into the void. v2: * Added tasklet pointer to the trace event as suggested by Steven v3: * Minor tweak to commit to show usage and output example --- include/trace/events/irq.h | 47 ++++++++++++++++++++++++++++++++++++++ kernel/softirq.c | 9 ++++++-- 2 files changed, 54 insertions(+), 2 deletions(-) diff --git a/include/trace/events/irq.h b/include/trace/events/irq.h index eeceafaaea4c..a07b4607b663 100644 --- a/include/trace/events/irq.h +++ b/include/trace/events/irq.h @@ -160,6 +160,53 @@ DEFINE_EVENT(softirq, softirq_raise, TP_ARGS(vec_nr) ); =20 +DECLARE_EVENT_CLASS(tasklet, + + TP_PROTO(struct tasklet_struct *t, void *func), + + TP_ARGS(t, func), + + TP_STRUCT__entry( + __field( void *, tasklet) + __field( void *, func) + ), + + TP_fast_assign( + __entry->tasklet =3D t; + __entry->func =3D func; + ), + + TP_printk("tasklet=3D%ps function=3D%ps", __entry->tasklet, __entry->func) +); + +/** + * tasklet_entry - called immediately before the tasklet is run + * @t: tasklet pointer + * @func: tasklet callback or function being run + * + * Used to find individual tasklet execution time + */ +DEFINE_EVENT(tasklet, tasklet_entry, + + TP_PROTO(struct tasklet_struct *t, void *func), + + TP_ARGS(t, func) +); + +/** + * tasklet_exit - called immediately after the tasklet is run + * @t: tasklet pointer + * @func: tasklet callback or function being run + * + * Used to find individual tasklet execution time + */ +DEFINE_EVENT(tasklet, tasklet_exit, + + TP_PROTO(struct tasklet_struct *t, void *func), + + TP_ARGS(t, func) +); + #endif /* _TRACE_IRQ_H */ =20 /* This part must be outside protection */ diff --git a/kernel/softirq.c b/kernel/softirq.c index c8a6913c067d..1b725510dd0f 100644 --- a/kernel/softirq.c +++ b/kernel/softirq.c @@ -793,10 +793,15 @@ static void tasklet_action_common(struct softirq_acti= on *a, if (tasklet_trylock(t)) { if (!atomic_read(&t->count)) { if (tasklet_clear_sched(t)) { - if (t->use_callback) + if (t->use_callback) { + trace_tasklet_entry(t, t->callback); t->callback(t); - else + trace_tasklet_exit(t, t->callback); + } else { + trace_tasklet_entry(t, t->func); t->func(t->data); + trace_tasklet_exit(t, t->func); + } } tasklet_unlock(t); continue; --=20 2.40.0.577.gac1e443424-goog