From nobody Sun Oct 5 09:07:19 2025 Received: from mail-pg1-f201.google.com (mail-pg1-f201.google.com [209.85.215.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8206B2BEFFE for ; Wed, 6 Aug 2025 19:57:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510264; cv=none; b=FXS/KeeQOgChqHr4KcHYx+hYYKiZwNs03zi/4/FN50muaKKYX8NkBDVWqgzJjeK3dlDsz9pb7I1WPS/qOfjanZK99kooccBnqcwX4JU+lhxyCW/5k0g/M0RlKXa18GTVSu+n/g+qQMF9j6rWm98Z+H11gis5h7HYIRx3/to66cc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754510264; c=relaxed/simple; bh=YYCe+4eLi35qElSGVX2EX5EoBQNPxGJxcMCI2Jk+YMU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=hfi0KkoKmyJj5HOZ+zw7ylGyuwBHygBYJ0+zXHlD9dvXkgqsVLYwXsRjE/Gl3FPP6qJ4kfxjr+oQZsBm6sxzSZbuCNlmpCXbKlVgyqN0VADx4gHl80xCtqngFgROh9O/ESKsJbREM34lHmVtdIq1qwsftn8z4Mb7SgcIlZXwRPE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=nccQ4gdO; arc=none smtp.client-ip=209.85.215.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="nccQ4gdO" Received: by mail-pg1-f201.google.com with SMTP id 41be03b00d2f7-b423aba05feso137872a12.3 for ; Wed, 06 Aug 2025 12:57:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754510258; x=1755115058; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:from:to:cc:subject:date:message-id:reply-to; bh=RSqs7LjEbfI/o+XH+tbZdSeN9oFJhrETU4UrZ2e2+po=; b=nccQ4gdOx7uh0LSzRCCErY+WkYMUzWQPyrPzXgmeXUaMjpScuM3iuk/GWTr0JX77nQ 5eQOmW1WR/uWMWWxAq61fCwW5nb7+XcHa7PVTM7kTNF+w7OC75pNZLjGcfzp9MwHlvdd w61Hkzll6izQ1sfUNgiBjPAKnhSryGNtu0bYAmKuRjtH42myVvlhuOHqp6bxhv1MzExD hOlK92XLSXM6J1o9LYiNlvPY1wIbiMtQ4TusJ0a/803hrk9Rty/PzmEDcQtGE8G+RvUJ E++/mMN+0MHeEPSUSpjJOvZD2gBDZzKVJelAX+6ZGkKN0ccZhqLa5LMZW+3b9IQn8Sj9 w/QA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754510258; x=1755115058; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:reply-to:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=RSqs7LjEbfI/o+XH+tbZdSeN9oFJhrETU4UrZ2e2+po=; b=Q4NA+wUz5RSIAUxMZfxp2bOx0OB6bikQrGY+l4+ab368HzPMa3DlcYDb22CQ7FMmdp ANFKW5gm3EIRgJBzd/TsO1fLL3gTGJvpAXruz8cJNDLm5HTvffebCrwcUuBffvyyEGE4 jX5K/78OAaDe25p8bdPeuzghxEvEeRt6y+7IdlfrVGurkZVvFpyE5ym6OpP7DZsR6n9N F/POzDcfONPocHfVKDteSvkuuCoFK6G1thke2Xb9NKhUVOEO3IXtbZGUEICrojQiLzv2 EA6dQmaV/7N0dOfTpwGoZ8Y8NzDAKNO1kyRNOa/ZqeQX4R+cyPHyANRQIJW8zr3sKE+q V2lQ== X-Forwarded-Encrypted: i=1; AJvYcCVt+sWkcIds7zdi2cr7kyd2TgN8CpH6UgcJvuE4x7N4qfyAFhohD5Uuj4FXDKzWdW8iA/i6kvDdyvkoPy4=@vger.kernel.org X-Gm-Message-State: AOJu0Yx8MBwVCMLjvnpdU4KpMVKQ5A+p+PAj2FkoKHxSQjmokonILKTj Yf2gQNYuWjzTNt1J2OLe3O/hYBpkzv2bk0zGTLT+sb/p3oq/CmXbo9p+yPF9Q3DvXypZdrkLCMf QD16ZEQ== X-Google-Smtp-Source: AGHT+IHRaSY+k4TAZbl8ZMQSCfgVDMYJiwgIqFCqRbteQRg9trtxOd4uQTojLJyQdeJPXNfHx0P6bSsh+7Q= X-Received: from pjqq16.prod.google.com ([2002:a17:90b:5850:b0:31c:2fe4:33bd]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:1c10:b0:311:ef19:824d with SMTP id 98e67ed59e1d1-32166df97d9mr4595576a91.2.1754510257253; Wed, 06 Aug 2025 12:57:37 -0700 (PDT) Reply-To: Sean Christopherson Date: Wed, 6 Aug 2025 12:56:27 -0700 In-Reply-To: <20250806195706.1650976-1-seanjc@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250806195706.1650976-1-seanjc@google.com> X-Mailer: git-send-email 2.50.1.565.gc32cd1483b-goog Message-ID: <20250806195706.1650976-6-seanjc@google.com> Subject: [PATCH v5 05/44] perf: Clean up perf ctx time From: Sean Christopherson To: Marc Zyngier , Oliver Upton , Tianrui Zhao , Bibo Mao , Huacai Chen , Anup Patel , Paul Walmsley , Palmer Dabbelt , Albert Ou , Xin Li , "H. Peter Anvin" , Andy Lutomirski , Peter Zijlstra , Ingo Molnar , Arnaldo Carvalho de Melo , Namhyung Kim , Sean Christopherson , Paolo Bonzini Cc: linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, kvm@vger.kernel.org, loongarch@lists.linux.dev, kvm-riscv@lists.infradead.org, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-perf-users@vger.kernel.org, Kan Liang , Yongwei Ma , Mingwei Zhang , Xiong Zhang , Sandipan Das , Dapeng Mi Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: Kan Liang The current perf tracks two timestamps for the normal ctx and cgroup. The same type of variables and similar codes are used to track the timestamps. In the following patch, the third timestamp to track the guest time will be introduced. To avoid the code duplication, add a new struct perf_time_ctx and factor out a generic function update_perf_time_ctx(). No functional change. Suggested-by: Peter Zijlstra (Intel) Signed-off-by: Kan Liang Signed-off-by: Mingwei Zhang Signed-off-by: Sean Christopherson --- include/linux/perf_event.h | 13 +++---- kernel/events/core.c | 70 +++++++++++++++++--------------------- 2 files changed, 39 insertions(+), 44 deletions(-) diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index 63097beb5f02..186a026fb7fb 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -999,6 +999,11 @@ struct perf_event_groups { u64 index; }; =20 +struct perf_time_ctx { + u64 time; + u64 stamp; + u64 offset; +}; =20 /** * struct perf_event_context - event context structure @@ -1037,9 +1042,7 @@ struct perf_event_context { /* * Context clock, runs when context enabled. */ - u64 time; - u64 timestamp; - u64 timeoffset; + struct perf_time_ctx time; =20 /* * These fields let us detect when two contexts have both @@ -1172,9 +1175,7 @@ struct bpf_perf_event_data_kern { * This is a per-cpu dynamically allocated data structure. */ struct perf_cgroup_info { - u64 time; - u64 timestamp; - u64 timeoffset; + struct perf_time_ctx time; int active; }; =20 diff --git a/kernel/events/core.c b/kernel/events/core.c index bf0347231bd9..2cf988bdabf0 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -815,6 +815,24 @@ static void perf_ctx_enable(struct perf_event_context = *ctx, static void ctx_sched_out(struct perf_event_context *ctx, struct pmu *pmu,= enum event_type_t event_type); static void ctx_sched_in(struct perf_event_context *ctx, struct pmu *pmu, = enum event_type_t event_type); =20 +static inline void update_perf_time_ctx(struct perf_time_ctx *time, u64 no= w, bool adv) +{ + if (adv) + time->time +=3D now - time->stamp; + time->stamp =3D now; + + /* + * The above: time' =3D time + (now - timestamp), can be re-arranged + * into: time` =3D now + (time - timestamp), which gives a single value + * offset to compute future time without locks on. + * + * See perf_event_time_now(), which can be used from NMI context where + * it's (obviously) not possible to acquire ctx->lock in order to read + * both the above values in a consistent manner. + */ + WRITE_ONCE(time->offset, time->time - time->stamp); +} + #ifdef CONFIG_CGROUP_PERF =20 static inline bool @@ -856,7 +874,7 @@ static inline u64 perf_cgroup_event_time(struct perf_ev= ent *event) struct perf_cgroup_info *t; =20 t =3D per_cpu_ptr(event->cgrp->info, event->cpu); - return t->time; + return t->time.time; } =20 static inline u64 perf_cgroup_event_time_now(struct perf_event *event, u64= now) @@ -865,22 +883,11 @@ static inline u64 perf_cgroup_event_time_now(struct p= erf_event *event, u64 now) =20 t =3D per_cpu_ptr(event->cgrp->info, event->cpu); if (!__load_acquire(&t->active)) - return t->time; - now +=3D READ_ONCE(t->timeoffset); + return t->time.time; + now +=3D READ_ONCE(t->time.offset); return now; } =20 -static inline void __update_cgrp_time(struct perf_cgroup_info *info, u64 n= ow, bool adv) -{ - if (adv) - info->time +=3D now - info->timestamp; - info->timestamp =3D now; - /* - * see update_context_time() - */ - WRITE_ONCE(info->timeoffset, info->time - info->timestamp); -} - static inline void update_cgrp_time_from_cpuctx(struct perf_cpu_context *c= puctx, bool final) { struct perf_cgroup *cgrp =3D cpuctx->cgrp; @@ -894,7 +901,7 @@ static inline void update_cgrp_time_from_cpuctx(struct = perf_cpu_context *cpuctx, cgrp =3D container_of(css, struct perf_cgroup, css); info =3D this_cpu_ptr(cgrp->info); =20 - __update_cgrp_time(info, now, true); + update_perf_time_ctx(&info->time, now, true); if (final) __store_release(&info->active, 0); } @@ -917,7 +924,7 @@ static inline void update_cgrp_time_from_event(struct p= erf_event *event) * Do not update time when cgroup is not active */ if (info->active) - __update_cgrp_time(info, perf_clock(), true); + update_perf_time_ctx(&info->time, perf_clock(), true); } =20 static inline void @@ -941,7 +948,7 @@ perf_cgroup_set_timestamp(struct perf_cpu_context *cpuc= tx) for (css =3D &cgrp->css; css; css =3D css->parent) { cgrp =3D container_of(css, struct perf_cgroup, css); info =3D this_cpu_ptr(cgrp->info); - __update_cgrp_time(info, ctx->timestamp, false); + update_perf_time_ctx(&info->time, ctx->time.stamp, false); __store_release(&info->active, 1); } } @@ -1562,20 +1569,7 @@ static void __update_context_time(struct perf_event_= context *ctx, bool adv) =20 lockdep_assert_held(&ctx->lock); =20 - if (adv) - ctx->time +=3D now - ctx->timestamp; - ctx->timestamp =3D now; - - /* - * The above: time' =3D time + (now - timestamp), can be re-arranged - * into: time` =3D now + (time - timestamp), which gives a single value - * offset to compute future time without locks on. - * - * See perf_event_time_now(), which can be used from NMI context where - * it's (obviously) not possible to acquire ctx->lock in order to read - * both the above values in a consistent manner. - */ - WRITE_ONCE(ctx->timeoffset, ctx->time - ctx->timestamp); + update_perf_time_ctx(&ctx->time, now, adv); } =20 static void update_context_time(struct perf_event_context *ctx) @@ -1593,7 +1587,7 @@ static u64 perf_event_time(struct perf_event *event) if (is_cgroup_event(event)) return perf_cgroup_event_time(event); =20 - return ctx->time; + return ctx->time.time; } =20 static u64 perf_event_time_now(struct perf_event *event, u64 now) @@ -1607,9 +1601,9 @@ static u64 perf_event_time_now(struct perf_event *eve= nt, u64 now) return perf_cgroup_event_time_now(event, now); =20 if (!(__load_acquire(&ctx->is_active) & EVENT_TIME)) - return ctx->time; + return ctx->time.time; =20 - now +=3D READ_ONCE(ctx->timeoffset); + now +=3D READ_ONCE(ctx->time.offset); return now; } =20 @@ -11991,7 +11985,7 @@ static void task_clock_event_update(struct perf_eve= nt *event, u64 now) =20 static void task_clock_event_start(struct perf_event *event, int flags) { - local64_set(&event->hw.prev_count, event->ctx->time); + local64_set(&event->hw.prev_count, event->ctx->time.time); perf_swevent_start_hrtimer(event); } =20 @@ -11999,7 +11993,7 @@ static void task_clock_event_stop(struct perf_event= *event, int flags) { perf_swevent_cancel_hrtimer(event); if (flags & PERF_EF_UPDATE) - task_clock_event_update(event, event->ctx->time); + task_clock_event_update(event, event->ctx->time.time); } =20 static int task_clock_event_add(struct perf_event *event, int flags) @@ -12019,8 +12013,8 @@ static void task_clock_event_del(struct perf_event = *event, int flags) static void task_clock_event_read(struct perf_event *event) { u64 now =3D perf_clock(); - u64 delta =3D now - event->ctx->timestamp; - u64 time =3D event->ctx->time + delta; + u64 delta =3D now - event->ctx->time.stamp; + u64 time =3D event->ctx->time.time + delta; =20 task_clock_event_update(event, time); } --=20 2.50.1.565.gc32cd1483b-goog