From nobody Sun Feb 8 05:22:19 2026 Received: from mail-pg1-f174.google.com (mail-pg1-f174.google.com [209.85.215.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CEFAD1F6666 for ; Tue, 3 Dec 2024 14:28:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733236096; cv=none; b=LRmYYpZosjU3saObNGSsd9qyAq1uQbm3h5ydaLVydVGd3ZGTGvSGxbEzOMPp6yBByqlctOU6wmgZ4UPLpyS6P/Wjjm+nqaWbHDv/h1swJ2jhIWT7ZicEP64SCTCIM8MfOcZErrYYZFBoClBTIgBFb1nlQAWLr2dPelJ619gqP+4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733236096; c=relaxed/simple; bh=h2dIVnJ81F6uZYP1oUnf4CrFDhfMx5vUHlJpCOdGz8c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RP834N++MM25Z2Q0P9yZI/n5HGqYkKO99ZwxbFIXw240oIhvb5t/6/lr35aachxI147/WW6L9yut2MP5B5zOjEalAwJW7dlhnjAaOlGOBCnMYUSDz4qTyxC21sYwi1zqYyuvxi1+byI8DPtUxstyUK3bOmuQdyL++h4CzUSSy7o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=fip9F796; arc=none smtp.client-ip=209.85.215.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="fip9F796" Received: by mail-pg1-f174.google.com with SMTP id 41be03b00d2f7-7fbc65f6c72so5291108a12.1 for ; Tue, 03 Dec 2024 06:28:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733236094; x=1733840894; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=uXY9UT8RCb6a4PX7hbXExEdtMGrEre1BV6m7kXEjMlk=; b=fip9F796pGZWoWkn3/At2TZcl0I3wQyNiBGT5tJpk2Npm1JaYg1mNyIgJO9MZ0naEG aD/+xWkuIP4i8ahT/AVMzD519UZgaawJztXT+p+F+6fZ/LbnwR92aKBDTgiMs5X9r4D5 VX2exRzg0dGX8uOa/lly5vV8qQgjV9LlbfnQeN2G20xVqIrEJ4985Iabc1yaBhZ9qeoQ gflFGTNsYDEEMkSOMs7ZTuwxmXesSmt+DzmZKgA7DNqC9VfWeYkLsBFZP3Le3/zt+stW dKgHuncUm2LE5QC/2QCAW0w2Dqs7gNJdZGsyVJSgEEtIq590G4htKoQ+ay9ai/4nkObx Fvjw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733236094; x=1733840894; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=uXY9UT8RCb6a4PX7hbXExEdtMGrEre1BV6m7kXEjMlk=; b=fvqaZiMjSvlc/9DhhObmxhYp0X1iDoEGPtz8ldQEZnokeRKRsfRBpxq2biyBV7W1Ay 7XlvbV/PX8AFpj9rsW5dveH6oXITmWbJT+wyLv5flgUi1EgMD7CWDl1ueGKp1Po/BiO+ 4aPd8HNMREcpFNRzGbIz13tkGqfD11/d9wozGdE87ioIdPpnC1CDyO+p0lE0u61iMPF6 LQOIHr9kTdeAqkOJHrOznsixV5ksXaY92Rl5IuHbnb5zGA54ROlTWiARD4vgCulcmtmt Aer4HpHwpiPTiqbcLyCzM44S0/3ZBYRmVVwZtFT3otrsMXP70L+B9m1gANkHKNxEKdqA 5Kpw== X-Forwarded-Encrypted: i=1; AJvYcCVZMNnIId/8q+0Qc1AgUKbE1loneneJjiCSTtFlwJ10aCoqpMDiMCpz/vbdUi/bwxwjzvYjeJ4VmjNjzvM=@vger.kernel.org X-Gm-Message-State: AOJu0Yxp/oteuHLJBgOPQbRHRpQnW0Gd4X6+OeE+IUwTvu9jgleSzurp pHPh2JsjBS0W6actMRlmNUlHmJ5N2rlFN3oQ+U83o1Ds7mxwt+mvYDpspQ== X-Gm-Gg: ASbGncv5zZBa7O2VH7k0+OaQxlIf4QkGyVof4BHAfa6uU+gfSo5tOVTYuTAUootJyVd 92Ia3/Xn5UMGoARThEn8rvNSGVWjGNKpebQFyxJJL+qhMoYamdC7tcXzIL3Um8+Mg9KN+mhiD2a Kv1p2X3CrE786jN9lMNXKU+bCPW75Tnq8NTYNyhCu2lD6MZJ1UhXj8voNk8Eju83GdhUzDWxPv3 UoapwNR3panDpRSDEuo3JUW8fBxAmxv1UEjtW0ZBp05oknsytQerPMcdzdbzjJ5NaV3FLaG2lzo QSJUZQ== X-Google-Smtp-Source: AGHT+IE/KLhiic7WhAdlfQP7doDfiMSi12yMUnE+eYYoKvVI13GcDPjDKhryrlonfGZ4i4a7vfkvlw== X-Received: by 2002:a05:6a20:6a28:b0:1db:f01a:cf12 with SMTP id adf61e73a8af0-1e1653f28e6mr3848305637.34.1733236093907; Tue, 03 Dec 2024 06:28:13 -0800 (PST) Received: from localhost ([58.29.143.236]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72541849be0sm10481757b3a.198.2024.12.03.06.28.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 03 Dec 2024 06:28:13 -0800 (PST) From: Changwoo Min X-Google-Original-From: Changwoo Min To: tj@kernel.org, void@manifault.com, mingo@redhat.com, peterz@infradead.org Cc: changwoo@igalia.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: [PATCH v3 1/5] sched_ext: Implement scx_rq_clock_update/stale() Date: Tue, 3 Dec 2024 23:27:58 +0900 Message-ID: <20241203142802.36305-2-changwoo@igalia.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241203142802.36305-1-changwoo@igalia.com> References: <20241203142802.36305-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" scx_rq_clock_update() and scx_rq_clock_stale() manage the status of an rq clock when sched_ext is enabled. scx_rq_clock_update() keeps the rq clock in memory and its status valid. scx_rq_clock_stale() invalidates the current rq clock not to use the cached rq clock. Signed-off-by: Changwoo Min --- kernel/sched/sched.h | 48 +++++++++++++++++++++++++++++++------------- 1 file changed, 34 insertions(+), 14 deletions(-) diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 76f5f53a645f..da55b39c9e58 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -754,6 +754,7 @@ enum scx_rq_flags { SCX_RQ_BAL_PENDING =3D 1 << 2, /* balance hasn't run yet */ SCX_RQ_BAL_KEEP =3D 1 << 3, /* balance decided to keep current */ SCX_RQ_BYPASSING =3D 1 << 4, + SCX_RQ_CLK_VALID =3D 1 << 5, /* RQ clock is fresh and valid */ =20 SCX_RQ_IN_WAKEUP =3D 1 << 16, SCX_RQ_IN_BALANCE =3D 1 << 17, @@ -766,8 +767,9 @@ struct scx_rq { unsigned long ops_qseq; u64 extra_enq_flags; /* see move_task_to_local_dsq() */ u32 nr_running; - u32 flags; u32 cpuperf_target; /* [0, SCHED_CAPACITY_SCALE] */ + u64 clock; /* cached per-rq clock -- see scx_bpf_clock_get_ns() */ + u32 flags; bool cpu_released; cpumask_var_t cpus_to_kick; cpumask_var_t cpus_to_kick_if_idle; @@ -1717,6 +1719,37 @@ struct rq_flags { =20 extern struct balance_callback balance_push_callback; =20 +#ifdef CONFIG_SCHED_CLASS_EXT +extern const struct sched_class ext_sched_class; + +DECLARE_STATIC_KEY_FALSE(__scx_ops_enabled); /* SCX BPF scheduler loaded */ +DECLARE_STATIC_KEY_FALSE(__scx_switched_all); /* all fair class tasks on S= CX */ + +#define scx_enabled() static_branch_unlikely(&__scx_ops_enabled) +#define scx_switched_all() static_branch_unlikely(&__scx_switched_all) + +static inline void scx_rq_clock_update(struct rq *rq, u64 clock) +{ + if (scx_enabled()) { + rq->scx.clock =3D clock; + rq->scx.flags |=3D SCX_RQ_CLK_VALID; + } +} + +static inline void scx_rq_clock_stale(struct rq *rq) +{ + if (scx_enabled()) + rq->scx.flags &=3D ~SCX_RQ_CLK_VALID; +} + +#else /* !CONFIG_SCHED_CLASS_EXT */ +#define scx_enabled() false +#define scx_switched_all() false + +static inline void scx_rq_clock_update(struct rq *rq, u64 clock) {} +static inline void scx_rq_clock_stale(struct rq *rq) {} +#endif /* !CONFIG_SCHED_CLASS_EXT */ + /* * Lockdep annotation that avoids accidental unlocks; it's like a * sticky/continuous lockdep_assert_held(). @@ -2505,19 +2538,6 @@ extern const struct sched_class rt_sched_class; extern const struct sched_class fair_sched_class; extern const struct sched_class idle_sched_class; =20 -#ifdef CONFIG_SCHED_CLASS_EXT -extern const struct sched_class ext_sched_class; - -DECLARE_STATIC_KEY_FALSE(__scx_ops_enabled); /* SCX BPF scheduler loaded */ -DECLARE_STATIC_KEY_FALSE(__scx_switched_all); /* all fair class tasks on S= CX */ - -#define scx_enabled() static_branch_unlikely(&__scx_ops_enabled) -#define scx_switched_all() static_branch_unlikely(&__scx_switched_all) -#else /* !CONFIG_SCHED_CLASS_EXT */ -#define scx_enabled() false -#define scx_switched_all() false -#endif /* !CONFIG_SCHED_CLASS_EXT */ - /* * Iterate only active classes. SCX can take over all fair tasks or be * completely disabled. If the former, skip fair. If the latter, skip SCX. --=20 2.47.1 From nobody Sun Feb 8 05:22:19 2026 Received: from mail-pl1-f180.google.com (mail-pl1-f180.google.com [209.85.214.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 541D51F4733 for ; Tue, 3 Dec 2024 14:28:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733236103; cv=none; b=C7NBeae/YEbwhkLDva7QNrO62xYrApIE5J6/s+MeTOu8Is15t6XqvWuvpaPwAtqCEyVKrfqnnjb8lp9OONvyxfteiOz0pJO+a7W2AM76j6JwwSH2o5GsKZmN/6UcSjupg5b3zWc/ScJs+4WZbpzGR051dzGaZVWTmD79QJneIhg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733236103; c=relaxed/simple; bh=vQrVciJTDoDS5vnYIk1yX6n6NAutqvNy1gYAuXSRH5Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qZft4nvRQKLTUA2tMwCCKiPSbb53FAua09sl345O9Lvjlbi++mZm0fEKSgd+i6SXwv2KlH0naZgG/sldiS1mLzltl8wcVFtGukhaO+Niz4wIshZ5eXLppSb7JylGNQtHWQwGX6Ve5bRQs34ZelyuPddusEdz81N2oDgn+RLzrZY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=eQuQtKD4; arc=none smtp.client-ip=209.85.214.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="eQuQtKD4" Received: by mail-pl1-f180.google.com with SMTP id d9443c01a7336-2155157c31fso30057515ad.1 for ; Tue, 03 Dec 2024 06:28:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733236101; x=1733840901; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=UN9R12v3ErwGCI7mcp2L+jL7PAoJVHL8ztjnEIA4Wlc=; b=eQuQtKD4eHVJEdypcSFlHk1MmhZnh6WYX/eVH6kHwomhivOFt9GpNIgUUVilWEGP33 jfYR8BQtn/g16U6DtILxMBOfVe8WOY7FO0NUbmUuVGp5ccX2KvuIEI32HPzWSY1ah/sB dXVGAoAdtosMIdcIy64qyAibnJAD+mQ9ooflJWJ5U22Perf9+olqk0Hm4bid08csCId0 GHfeaq6+hPzqtHvTSXKHg7EUeAbotzQZ5o+sAdMu2dLlCS5J5RJhHtYIMWiGOq/Y34np 1UAHQpPxrJVxwrPGLExCv5FtRLEA2ppwMK2MG6A7tswzj8ZzJc/RdEeSbmkdYz/PMk/i NmIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733236102; x=1733840902; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UN9R12v3ErwGCI7mcp2L+jL7PAoJVHL8ztjnEIA4Wlc=; b=mE//+XLuQWR2MUaUhyuhGqJBeY9HAJI+v1sg4WducPxoOGKsVWJOOgewzChzUqSz7g aOnOE1IXSEbfgoQ3OQtuTRS6UobtfEqP7+vWTa6OuUkaZnJQkBuCooTwcYVHprar6o7W +OHFIfXOu32OMPEuUjOe2hgtqZaZRC8z5iEIni/s1Nx1i3ShcDiOLMjSebrBib8xGDix m+hKV3j7LyY2UTgg/8yHdqEZ2ELdGhEYDx3Dzo+/hFO55hQ6RpKHh0W/iIlgGvqQfTMa tPYAzkXOZoGEKymL2TawkgUN6bQ437X1yl6B+rUwguOeFVwtd4IoRALgaRDcZdyDuSQb 30bg== X-Forwarded-Encrypted: i=1; AJvYcCX8OHiRfGI7vPq5NkYdGjkOQL2TiaBxhHaRLm9+QY+WnavoGuIqMlpCNeDS7VH+ruXwuzruw980h2WcS40=@vger.kernel.org X-Gm-Message-State: AOJu0Yx6CA8kXH4TAeR/JlKUM1dkXHksd1q0g9I94yhbEkmoZKcWO+Bj bQ8Mil9c/uz8rN9XuX1I8Fp/Zg3HpN8UkTjSmqI6yOcPiljcY5VA X-Gm-Gg: ASbGncvQvdLnfQgjqCcZUQKEoBCRAYsG8dx7D0wSspSxw4Dx4KHJ+e3lZvu82yZ9C9a Syd5fCzgGJTjJca5BMgw3CQ/YSHHSd26yrduMGmFtoVrkqPqPs610C3i1/2AY47iU57wJEcPXt8 J2s9UVz48iMwhe1W4s3gXhAwK7iQ1dqqeKwO/J2C5OYqIVgl6rJoEQi/KLC0DbX93DQzsm0r17l 221kbFySUoETs6YP0yfdsqbP7zGvmaS7bYzWrWZmnL22+HiyEloRzWTYZV0sgrL0a86N78pTFb5 IsE0DQ== X-Google-Smtp-Source: AGHT+IGJY117emqiWiaAw7hJR2jgJXypH9JAa5pixPdjNfoRKlDOxchalfXCK9aU3kL1dpwjft0wjQ== X-Received: by 2002:a17:903:1744:b0:215:3998:189f with SMTP id d9443c01a7336-215bd45d400mr38752025ad.6.1733236101460; Tue, 03 Dec 2024 06:28:21 -0800 (PST) Received: from localhost ([58.29.143.236]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-21589aa5478sm41391925ad.59.2024.12.03.06.28.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 03 Dec 2024 06:28:21 -0800 (PST) From: Changwoo Min X-Google-Original-From: Changwoo Min To: tj@kernel.org, void@manifault.com, mingo@redhat.com, peterz@infradead.org Cc: changwoo@igalia.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: [PATCH v3 2/5] sched_ext: Manage the validity of scx_rq_clock Date: Tue, 3 Dec 2024 23:27:59 +0900 Message-ID: <20241203142802.36305-3-changwoo@igalia.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241203142802.36305-1-changwoo@igalia.com> References: <20241203142802.36305-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" An rq clock becomes valid when it is updated using update_rq_clock() and invalidated when the rq is unlocked using rq_unpin_lock(). Also, after long-running operations -- ops.running() and ops.update_idle() -- in a BPF scheduler, the sched_ext core invalidates the rq clock. Let's suppose the following timeline in the scheduler core: T1. rq_lock(rq) T2. update_rq_clock(rq) T3. a sched_ext BPF operation T4. rq_unlock(rq) T5. a sched_ext BPF operation T6. rq_lock(rq) T7. update_rq_clock(rq) For [T2, T4), we consider that rq clock is valid (SCX_RQ_CLK_VALID is set), so scx_bpf_clock_get_ns() calls during [T2, T4) (including T3) will return the rq clock updated at T2. For duration [T4, T7), when a BPF scheduler can still call scx_bpf_clock_get_ns() (T5), we consider the rq clock is invalid (SCX_RQ_CLK_VALID is unset at T4). So when calling scx_bpf_clock_get_ns() at T5, we will return a fresh clock value by calling sched_clock() internally. One example of calling scx_bpf_clock_get_ns(), when the rq clock is invalid (like T5), is in scx_central [1]. The scx_central scheduler uses a BPF timer for preemptive scheduling. In every msec, the timer callback checks if the currently running tasks exceed their timeslice. At the beginning of the BPF timer callback (central_timerfn in scx_central.bpf.c), scx_central gets the current time. When the BPF timer callback runs, the rq clock could be invalid, the same as T5. In this case, scx_bpf_clock_get_ns() returns a fresh clock value rather than returning the old one (T2). [1] https://github.com/sched-ext/scx/blob/main/scheds/c/scx_central.bpf.c Signed-off-by: Changwoo Min --- kernel/sched/core.c | 6 +++++- kernel/sched/ext.c | 3 +++ kernel/sched/sched.h | 2 +- 3 files changed, 9 insertions(+), 2 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 95e40895a519..ab8015c8cab4 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -789,6 +789,7 @@ static void update_rq_clock_task(struct rq *rq, s64 del= ta) void update_rq_clock(struct rq *rq) { s64 delta; + u64 clock; =20 lockdep_assert_rq_held(rq); =20 @@ -800,11 +801,14 @@ void update_rq_clock(struct rq *rq) SCHED_WARN_ON(rq->clock_update_flags & RQCF_UPDATED); rq->clock_update_flags |=3D RQCF_UPDATED; #endif + clock =3D sched_clock_cpu(cpu_of(rq)); + scx_rq_clock_update(rq, clock); =20 - delta =3D sched_clock_cpu(cpu_of(rq)) - rq->clock; + delta =3D clock - rq->clock; if (delta < 0) return; rq->clock +=3D delta; + update_rq_clock_task(rq, delta); } =20 diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index 7fff1d045477..ac279a657d50 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -2928,6 +2928,8 @@ static void set_next_task_scx(struct rq *rq, struct t= ask_struct *p, bool first) if (SCX_HAS_OP(running) && (p->scx.flags & SCX_TASK_QUEUED)) SCX_CALL_OP_TASK(SCX_KF_REST, running, p); =20 + scx_rq_clock_stale(rq); + clr_task_runnable(p, true); =20 /* @@ -3590,6 +3592,7 @@ void __scx_update_idle(struct rq *rq, bool idle) { int cpu =3D cpu_of(rq); =20 + scx_rq_clock_stale(rq); if (SCX_HAS_OP(update_idle) && !scx_rq_bypassing(rq)) { SCX_CALL_OP(SCX_KF_REST, update_idle, cpu_of(rq), idle); if (!static_branch_unlikely(&scx_builtin_idle_enabled)) diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index da55b39c9e58..7d26731c350f 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1779,7 +1779,7 @@ static inline void rq_unpin_lock(struct rq *rq, struc= t rq_flags *rf) if (rq->clock_update_flags > RQCF_ACT_SKIP) rf->clock_update_flags =3D RQCF_UPDATED; #endif - + scx_rq_clock_stale(rq); lockdep_unpin_lock(__rq_lockp(rq), rf->cookie); } =20 --=20 2.47.1 From nobody Sun Feb 8 05:22:19 2026 Received: from mail-pf1-f173.google.com (mail-pf1-f173.google.com [209.85.210.173]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DD96017BB16 for ; Tue, 3 Dec 2024 14:28:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.173 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733236109; cv=none; b=VRF8wM8eLldf/0YRaxyAvKqw10C5CWNjPCYBOhOQRRXmrgcB8bJ0chnbxVPdXBP8YhzAEP+rfQ0gKbAQNbEJvftLK7HKjqMJ5jzp9eXDxMqxTrHL33xvhdL3sv3Icteba9vyZfjoSk0zQZrv7JvtjtRhd1XWxy7KUpgv0ZICe2I= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733236109; c=relaxed/simple; bh=XP1bJ0NOVWvhnqquRkwIz/i6kZEfAZJaGcyYAhOVshM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qPMRDwUloACJtzW7fNBxtQwmIzbEwZKw1ka1xkofsMpgxc2D1MtwWJpur5zod7TRxdR5ogeNlqOQlY6EToaiCkLXLFAGHaX+7ThWHt2LBRwcYp54jUGsGuR/gKSt9CO3VdBR2fQmztSH9HvjHDrL8PmOxZn+8L2sA0dIVXyCRhs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=KX8TeSaK; arc=none smtp.client-ip=209.85.210.173 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="KX8TeSaK" Received: by mail-pf1-f173.google.com with SMTP id d2e1a72fcca58-7251d20e7f2so5565560b3a.0 for ; Tue, 03 Dec 2024 06:28:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733236107; x=1733840907; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vYP7n+ha6LjN/GOJcD6nH/LRPcnS5LKp91acoSKaYwA=; b=KX8TeSaKKOdi3b2+TSS43qbTOeHllSFBLiK05hpGvqEXFWbAPdPvbfCwalvrCZ5/Gy /mrbcZxAgM9las/nRtk02lNibEsCvbOE06sGReCCw8NuS6LH+cMhzQm/ecTjUCR9rv4h ovG3kb7YYlzO6ayuKu+bE4VM8HDdwShJSAf7eJJGSIPKn/YMwd+hyRToIUL3BZUKnCQw SVCEWtuEJgRN7lDdkMBLvsBUYzWzR2l/J8dvBxMup/Pp7Z9s5Cq5fsdUkrJhhBBSvQbm Dm2+kqB3iwbRJwNZnOPG0NuHQOc/3C5MN4XRdibtgL+dQjbP7UQPqq9s1x1+eflWQ3QA XNjQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733236107; x=1733840907; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vYP7n+ha6LjN/GOJcD6nH/LRPcnS5LKp91acoSKaYwA=; b=YGsxAwleau1gabW06y3TZkpYvdrk5iGXz72H5JiaI1Xe3/6r2IektjrgoZiFcIBLUu wTVreMF+ARz2xf7Ni86BKJkNJlZ3o4jFCEbkBlBAywM7n85l7gXJWcsZtEqMrO0cLuzI IrnfXJlHShCj0aW203iKSMgRm1iDRRatvKuNuCidSx+37YLTarq6shru2OGIEpuf+hDK J5N1gi1iHAyvVHupCTWXHrSUPs+tP5alCqdhX20C1Ev6Nm4a++CopEWwLR8zUwqM0OPj /xTSjQROcu0D/1OIofp0Cc8MfZEKQEV7lfuRJCXjm3W23BCvy9oF3xYUzIaajfTViVmi P5Dw== X-Forwarded-Encrypted: i=1; AJvYcCURvtSmXxyfAnrWpSDlSd0EXjuALe+sfoJawcJ13aVWyppJyGSrNIEFhzhFqQMcnieULlXj1bLpQJA7N3Y=@vger.kernel.org X-Gm-Message-State: AOJu0YyLDLTdmH/WPSebNat5I0eRKO411rkVFtr9wbmp58kIw1sqRPhf e5zTWh/ESs1+vV7USVR5YL/2f0lVO77MOY5cDstUo+g04H3pH6ox X-Gm-Gg: ASbGnct8smVEI7jz0Xy5mAZ80V8pEIqA8ZjG/MeGRbntqUVV+Dw0hJlNGV4xW0Pb4qa oexX6PdB+HlT0M21ZC/kV7Cl5J8k7XLnWNBpdZT4+SxFByfiX02h/XqHONV0U+CDthkCaTwmd+p oTutvIYRM9X9CvcI9I+H8RY2poZxeMA7M2YBPpxwt2YTvViuL/qAMfjsDTcfp8ujcV5hsgCUIHu Br0CsVeEP5XNjJn0c2BgcX8mnJ7wWAEhB5mDDdByjrNvaTX9lRwaQjY5m5x4ZlnjN/p1h5+aAG5 N74EeA== X-Google-Smtp-Source: AGHT+IGzEvUFUhuv6DWW7JtgRJA5rpEQD9VBCT18VtfYwwYuviw9CPrCi9Odv+s0ttu7vLN4Mk4pGQ== X-Received: by 2002:a17:90b:3e83:b0:2ee:b8ac:73c1 with SMTP id 98e67ed59e1d1-2ef011db5f7mr3511314a91.2.1733236106902; Tue, 03 Dec 2024 06:28:26 -0800 (PST) Received: from localhost ([58.29.143.236]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2eeaef494fcsm4686117a91.43.2024.12.03.06.28.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 03 Dec 2024 06:28:26 -0800 (PST) From: Changwoo Min X-Google-Original-From: Changwoo Min To: tj@kernel.org, void@manifault.com, mingo@redhat.com, peterz@infradead.org Cc: changwoo@igalia.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: [PATCH v3 3/5] sched_ext: Implement scx_bpf_clock_get_ns() Date: Tue, 3 Dec 2024 23:28:00 +0900 Message-ID: <20241203142802.36305-4-changwoo@igalia.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241203142802.36305-1-changwoo@igalia.com> References: <20241203142802.36305-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Returns a high-performance monotonically non-decreasing clock for the current CPU. The clock returned is in nanoseconds. It provides the following properties: 1) High performance: Many BPF schedulers call bpf_ktime_get_ns() frequently to account for execution time and track tasks' runtime properties. Unfortunately, in some hardware platforms, bpf_ktime_get_ns() -- which eventually reads a hardware timestamp counter -- is neither performant nor scalable. scx_bpf_clock_get_ns() aims to provide a high-performance clock by using the rq clock in the scheduler core whenever possible. 2) High enough resolution for the BPF scheduler use cases: In most BPF scheduler use cases, the required clock resolution is lower than the most accurate hardware clock (e.g., rdtsc in x86). scx_bpf_clock_get_ns() basically uses the rq clock in the scheduler core whenever it is valid. It considers that the rq clock is valid from the time the rq clock is updated (update_rq_clock) until the rq is unlocked (rq_unpin_lock). In addition, it invalidates the rq clock after long operations -- ops.running() and ops.update_idle() -- in a BPF scheduler. 3) Monotonically non-decreasing clock for the same CPU: scx_bpf_clock_get_ns() guarantees the clock never goes backward when comparing them in the same CPU. On the other hand, when comparing clocks in different CPUs, there is no such guarantee -- the clock can go backward. It provides a monotonically *non-decreasing* clock so that it would provide the same clock values in two different scx_bpf_clock_get_ns() calls in the same CPU during the same period of when the rq clock is valid. Signed-off-by: Changwoo Min --- kernel/sched/ext.c | 71 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 71 insertions(+) diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index ac279a657d50..9e2656e21593 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -7546,6 +7546,76 @@ __bpf_kfunc struct cgroup *scx_bpf_task_cgroup(struc= t task_struct *p) } #endif =20 +/** + * scx_bpf_clock_get_ns - Returns a high-performance monotonically + * non-decreasing clock for the current CPU. The clock returned is in + * nanoseconds. + * + * It provides the following properties: + * + * 1) High performance: Many BPF schedulers call bpf_ktime_get_ns() freque= ntly + * to account for execution time and track tasks' runtime properties. + * Unfortunately, in some hardware platforms, bpf_ktime_get_ns() -- which + * eventually reads a hardware timestamp counter -- is neither performant= nor + * scalable. scx_bpf_clock_get_ns() aims to provide a high-performance cl= ock + * by using the rq clock in the scheduler core whenever possible. + * + * 2) High enough resolution for the BPF scheduler use cases: In most BPF + * scheduler use cases, the required clock resolution is lower than the m= ost + * accurate hardware clock (e.g., rdtsc in x86). scx_bpf_clock_get_ns() + * basically uses the rq clock in the scheduler core whenever it is valid. + * It considers that the rq clock is valid from the time the rq clock is + * updated (update_rq_clock) until the rq is unlocked (rq_unpin_lock). + * In addition, it invalidates the rq clock after long operations -- + * ops.running() and ops.update_idle(). + * + * 3) Monotonically non-decreasing clock for the same CPU: + * scx_bpf_clock_get_ns() guarantees the clock never goes backward when + * comparing them in the same CPU. On the other hand, when comparing cloc= ks + * in different CPUs, there is no such guarantee -- the clock can go back= ward. + * It provides a monotonically *non-decreasing* clock so that it would pr= ovide + * the same clock values in two different scx_bpf_clock_get_ns() calls in= the + * same CPU during the same period of when the rq clock is valid. + */ +__bpf_kfunc u64 scx_bpf_clock_get_ns(void) +{ + static DEFINE_PER_CPU(u64, prev_clk); + struct rq *rq =3D this_rq(); + u64 pr_clk, cr_clk; + + preempt_disable(); + pr_clk =3D __this_cpu_read(prev_clk); + + /* + * If the rq clock is invalid, start a new rq clock period + * with a fresh sched_clock(). + */ + if (!(rq->scx.flags & SCX_RQ_CLK_VALID)) { + cr_clk =3D sched_clock(); + scx_rq_clock_update(rq, cr_clk); + } + /* + * If the rq clock is valid, use the cached rq clock + * whenever the clock does not go backward. + */ + else { + cr_clk =3D rq->scx.clock; + /* + * If the clock goes backward, start a new rq clock period + * with a fresh sched_clock(). + */ + if (pr_clk > cr_clk) { + cr_clk =3D sched_clock(); + scx_rq_clock_update(rq, cr_clk); + } + } + + __this_cpu_write(prev_clk, cr_clk); + preempt_enable(); + + return cr_clk; +} + __bpf_kfunc_end_defs(); =20 BTF_KFUNCS_START(scx_kfunc_ids_any) @@ -7577,6 +7647,7 @@ BTF_ID_FLAGS(func, scx_bpf_cpu_rq) #ifdef CONFIG_CGROUP_SCHED BTF_ID_FLAGS(func, scx_bpf_task_cgroup, KF_RCU | KF_ACQUIRE) #endif +BTF_ID_FLAGS(func, scx_bpf_clock_get_ns) BTF_KFUNCS_END(scx_kfunc_ids_any) =20 static const struct btf_kfunc_id_set scx_kfunc_set_any =3D { --=20 2.47.1 From nobody Sun Feb 8 05:22:19 2026 Received: from mail-pg1-f175.google.com (mail-pg1-f175.google.com [209.85.215.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8835B1F472A for ; Tue, 3 Dec 2024 14:28:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733236119; cv=none; b=uodnXAQB22Rb7H9O/eL1m5LP/cLgfVvpBTnLp7wjZyKK9aPgQ98bCx+eZ0K68mE7HIyqmH69C+U/Z2IYyWgHV7XvKAMbohiw1ot2sBiMOj6uD/Det6/DXzNQEM4H8zfgkIoBZTyZFnbcuY2IKhcKckZBvokOmyRPm5QtG+bZqEY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733236119; c=relaxed/simple; bh=CiA7urzExORBNfNnKVkDnfqRTn3t7XF1Gk8zmHVsPIM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=UKffikaTo1GgxdnhCnN/eoTVcfT+w9QcnDVoR0DYkggxTJy41phmL7KaiE1iHchr2apPl+tM7nV6qWR9g/QWgnElc/cUz+JnZ55eQWHi7XS2jylU7Ck7JPMN+kH8zUFk0eySKoc4CBrnBE2XVrt2ESXvWveiTB0/5KRH5k4Pc2U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=QM/BhQ8J; arc=none smtp.client-ip=209.85.215.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="QM/BhQ8J" Received: by mail-pg1-f175.google.com with SMTP id 41be03b00d2f7-7fc93152edcso4136545a12.0 for ; Tue, 03 Dec 2024 06:28:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733236118; x=1733840918; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=t5z6e0Wwzok4/c8JJ14VvrKpt6wQ9E07/XNoPDXalY0=; b=QM/BhQ8JzDGsh19ey/+pGkbSiSIGrq9V6L8/wjeTCekjXvCH+nsiCuwiEIJi5g3nB9 2e1JjZbRWm0vn3r5JciYF8VtZmvWJLFlcEcCfRwAaPCOl2rpatj48i+mkwpayD6f4Kxz uxQCafD7eL1ZFW+QGPWvNehVQmruA/DmdpTcaX0BhPgvhsgBLsHC3AnpsiTSGAdyqhM9 XGJUR7RYVxYBtLDchWDe4XelJyVldrgDs+T4mhm9qJyMDuvABP+bHgfkSHQP9GqYe7BW PVA1OgmJNsftiJ0tmOjWMt3N9SJWn1Y5oUON8BhguSrAz0U9Gr3zErqMUi/JsOn6jG+7 36XA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733236118; x=1733840918; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=t5z6e0Wwzok4/c8JJ14VvrKpt6wQ9E07/XNoPDXalY0=; b=iWBp2esXSU9TWomfIGCAtyW/5NMJmY7iVlNBHjWeBrSHUYv3238g7HBPHpdzGM63Zu YSqFZK/OsveOi0kDpJmT83qx1w2AjleEsYVOpJwscVpjoaBxe+atQHWSs7KN1TLWjbzR miUZ48TzlE0ILYB82r4oYgBNAXZTSzFuATgy7vcsOte8bbgfBOLrwbF92poLCF1esev+ ur1VlBrGo4eYFb5PrMaR8lGczGwluLgGd/ChshmUiqh2XrjX4/WYi1U/FKtD4/SFR1pW 3HLQoSZdnaDjDABSym35mWLJJ5lZRm3Dbri29hPLs8I/BjA2tuXzPjfTcPDbtQxks5R/ XFcA== X-Forwarded-Encrypted: i=1; AJvYcCUTPgbeELItGp3w15NIN5qMfRBftWrxh5dX3ZELs0055Ps1iZWcLk9BABwrcnGp/xnToKj8F2yasDPM+wE=@vger.kernel.org X-Gm-Message-State: AOJu0YxMGFNNgYSAfVFd697/YINTm440//ouAlIRCSNMFoIggdh2FVxh Ek6evI/WG9YuifXyJJBT6EqHlxTHh4jQooCqEYSW2lyTLcae6iQL X-Gm-Gg: ASbGncuu9UKJLa4oLVkA1OdAr4T8PkbFXLMBOTktLO4sH3mjWVjOMS83H1fMkxh1hbZ xqPt4wVkaBNfZfaQah9R+3+n1o4kzBMQ9qphT9MjqZnb5Uyjq5JA76hUri/AVjjBBo6Iejk09WD /JpVdRL86ERvjhxDdPhlVtnPkpChM+AIMxreLs3+b0C1pYk1DndURaycyIZh9fn8dLyNZEByQkN pVxZz8NhwthQXoD9jUVGeOb7dV2H3iCliP/MBg9qvnDohpG3Wp5LhGzmTR9kVPLR0Td4aZz3nax nlsS1A== X-Google-Smtp-Source: AGHT+IFKmA4cKMp0jI/Mp+cRDt5ueufgdNIF0oU4f10SCH9s78uN8Zp8/PfN2JyO5T6Wlka9ORZ0DA== X-Received: by 2002:a17:90b:1c89:b0:2ee:1ae:a247 with SMTP id 98e67ed59e1d1-2ee25ade311mr32989740a91.8.1733236117782; Tue, 03 Dec 2024 06:28:37 -0800 (PST) Received: from localhost ([58.29.143.236]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72541848b36sm10862079b3a.187.2024.12.03.06.28.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 03 Dec 2024 06:28:37 -0800 (PST) From: Changwoo Min X-Google-Original-From: Changwoo Min To: tj@kernel.org, void@manifault.com, mingo@redhat.com, peterz@infradead.org Cc: changwoo@igalia.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: [PATCH v3 4/5] sched_ext: Add scx_bpf_clock_get_ns() for BPF scheduler Date: Tue, 3 Dec 2024 23:28:01 +0900 Message-ID: <20241203142802.36305-5-changwoo@igalia.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241203142802.36305-1-changwoo@igalia.com> References: <20241203142802.36305-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" scx_bpf_clock_get_ns() is added to the header files so the BPF scheduler can use it. Signed-off-by: Changwoo Min --- tools/sched_ext/include/scx/common.bpf.h | 1 + tools/sched_ext/include/scx/compat.bpf.h | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/tools/sched_ext/include/scx/common.bpf.h b/tools/sched_ext/inc= lude/scx/common.bpf.h index 2f36b7b6418d..230c7f2e8ad6 100644 --- a/tools/sched_ext/include/scx/common.bpf.h +++ b/tools/sched_ext/include/scx/common.bpf.h @@ -72,6 +72,7 @@ bool scx_bpf_task_running(const struct task_struct *p) __= ksym; s32 scx_bpf_task_cpu(const struct task_struct *p) __ksym; struct rq *scx_bpf_cpu_rq(s32 cpu) __ksym; struct cgroup *scx_bpf_task_cgroup(struct task_struct *p) __ksym __weak; +u64 scx_bpf_clock_get_ns(void) __ksym __weak; =20 /* * Use the following as @it__iter when calling scx_bpf_dsq_move[_vtime]() = from diff --git a/tools/sched_ext/include/scx/compat.bpf.h b/tools/sched_ext/inc= lude/scx/compat.bpf.h index d56520100a26..d295c59e3f05 100644 --- a/tools/sched_ext/include/scx/compat.bpf.h +++ b/tools/sched_ext/include/scx/compat.bpf.h @@ -125,6 +125,11 @@ bool scx_bpf_dispatch_vtime_from_dsq___compat(struct b= pf_iter_scx_dsq *it__iter, false; \ }) =20 +#define scx_bpf_clock_get_ns() \ + (bpf_ksym_exists(scx_bpf_clock_get_ns) ? \ + scx_bpf_clock_get_ns() : \ + bpf_ktime_get_ns()) + /* * Define sched_ext_ops. This may be expanded to define multiple variants = for * backward compatibility. See compat.h::SCX_OPS_LOAD/ATTACH(). --=20 2.47.1 From nobody Sun Feb 8 05:22:19 2026 Received: from mail-pl1-f172.google.com (mail-pl1-f172.google.com [209.85.214.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ABE481F6662 for ; Tue, 3 Dec 2024 14:28:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733236125; cv=none; b=JN7z0HQ8q/hSc9K1WusDU3+xy9sqKJDTB6UE/o4iqNUgsFF84vQ/4fSZVrrMtf+hdW2NoUB8IYwcFtAhg9sA7hlvpme8pxLwC5dO64rd9NhcqZ1TBQxjWSzpaG2BckRn6eaxIz8PmRqY+iHmsgHp5kEMgLQRG9RgLV/2F1AmBgY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733236125; c=relaxed/simple; bh=Kzd/PUC5ctt74z6B6vUj4jyvkEy3XwQUHulW25s5gBs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bhrNIwIS5zrgR/3d/EihzKzIaz6mGKV8i+S/5djUCdwZgI6ayd86V4u8gvJmFcv907ajieon7QdqLufV11EPxWBGIOMrpYRNfEeIjFa5lLdc+Y91OowBoSwMvYDaxEjSfqSRZ2FGKx2KF2VY9NT0/mZZpkj3Tq03SGeWHb77ARs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=kFpju4tE; arc=none smtp.client-ip=209.85.214.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="kFpju4tE" Received: by mail-pl1-f172.google.com with SMTP id d9443c01a7336-215513ea198so28166245ad.1 for ; Tue, 03 Dec 2024 06:28:43 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733236123; x=1733840923; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Hrorv/GUnF7EJucVAVWvFSb+WFE0AwzJOhO/e5bho0Y=; b=kFpju4tE1u3azH/d6FTZbV8QxLr4W6X4aeKF2+DKZSURZFewcck+E+t700PDgMbpuJ Xgd6o9pONk6ISYpz6c/DNIhJtUPoB5S/cd8AK/k1IuexhJK7u5q52dieEgvAhz4Z2tWm CECyZmuA3WOVKZ2Ud2RsaQGac6ge5ab5kPv1A+LfyHoCPDJj58IQmMYXkl8Fv16kYnkk YZEKmeUwOKrqslZ0eMo3oCxyMr0KMWfg3H4VNOctVVuZ+v9yZdJTYK4FW9bQYHPx1V0S hmnEJypa6jYL417CxKJBAo1HdPyV3/S1iXixrYmx6wUtdJ7gyqbktSYFVAb/NyxJ2ljx Xaqw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733236123; x=1733840923; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Hrorv/GUnF7EJucVAVWvFSb+WFE0AwzJOhO/e5bho0Y=; b=sZ+3LkjYmTcfC4eWGaEs2gJwe5ckBBx90mN4HfLPGO7Azux6nHVhS3Px+bsntyYf0w ai8SAjS3kf9XM8ELTNd7ql9asDfPmiUbUycCbpnM6MjYYcQ9V4/NFdnalQTxDeoZplpw gaRWtzN3SAXAyqva4Q6FVWSJGWh7aZZYJWsWEKSnwm1blDL69SxKoWyrXapuUumh3Sqr 7iXmwZV7R0XcmNmfAJNXAoSpRrdF/gVWLMSACDMwDAfWqOWV1IKy2wA6v82/SX7HG4Qg HgNE8tc9WJH1k8MtkDxqhVpSv2Ej+YyRTOJSFGOLmDSUCTIhWDGddGR6ra0twmQ7U+02 /JIA== X-Forwarded-Encrypted: i=1; AJvYcCVnkmjNpTVyAQkaVIG8W7GvXir+rqml4mSIECZUGJV+XmrzRaoF+Spg6T6fIRnB4QN0OBNBNS0mujsJGgI=@vger.kernel.org X-Gm-Message-State: AOJu0YwE7GhMEPqb7DuTuvmoJN4xAlc+3plWYXvgKUcB/cw5VlxDE2uo 4voUUfWLicP31TIsfp6NozDmseHeb0LPGjeyUnmTym8xWLLhCFLW X-Gm-Gg: ASbGncss0NYAo3j2MJLUUdaGe9sVC94jIllJtWFyfYWb4JxyRGtrBOdX+bDRRbPahX6 0LjprjW3Y3lhv0QByNwPkunOFV0zNZOMWlnt6hysKS0uVSnBF+xAsOLeFGjmSOoi7W6aIuBdA15 p3+lIloiRR8ttG4iKIXaZ6JL4K4kKe/l+zEeK4vC33/p+0GBm/HnmN/72ZKEjOkFqIg+XrDnbne nuw37Ujkq0012b/8iGRVZVdtF3gCE5yHsB57/7+Umofw7xdM9wjreij6yYnR1uA5rj2C1b4DEry lEwTvg== X-Google-Smtp-Source: AGHT+IG2K5uZo4GqKpI5DVD7Drbl8U8kF581y/rMKG+4tvjTUKEugmTN1ITRDZlE5gV/OAxdW/NFhw== X-Received: by 2002:a17:902:db10:b0:215:65c2:f3f2 with SMTP id d9443c01a7336-21565c2f650mr218546115ad.6.1733236122922; Tue, 03 Dec 2024 06:28:42 -0800 (PST) Received: from localhost ([58.29.143.236]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-215964b6b76sm35501205ad.191.2024.12.03.06.28.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 03 Dec 2024 06:28:42 -0800 (PST) From: Changwoo Min X-Google-Original-From: Changwoo Min To: tj@kernel.org, void@manifault.com, mingo@redhat.com, peterz@infradead.org Cc: changwoo@igalia.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: [PATCH v3 5/5] sched_ext: Replace bpf_ktime_get_ns() to scx_bpf_clock_get_ns() Date: Tue, 3 Dec 2024 23:28:02 +0900 Message-ID: <20241203142802.36305-6-changwoo@igalia.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241203142802.36305-1-changwoo@igalia.com> References: <20241203142802.36305-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In the BPF schedulers that use bpf_ktime_get_ns() -- scx_central and scx_flatcg, replace bpf_ktime_get_ns() calls to scx_bpf_clock_get_ns(). Signed-off-by: Changwoo Min --- tools/sched_ext/scx_central.bpf.c | 4 ++-- tools/sched_ext/scx_flatcg.bpf.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tools/sched_ext/scx_central.bpf.c b/tools/sched_ext/scx_centra= l.bpf.c index e6fad6211f6c..cb7428b6a198 100644 --- a/tools/sched_ext/scx_central.bpf.c +++ b/tools/sched_ext/scx_central.bpf.c @@ -245,7 +245,7 @@ void BPF_STRUCT_OPS(central_running, struct task_struct= *p) s32 cpu =3D scx_bpf_task_cpu(p); u64 *started_at =3D ARRAY_ELEM_PTR(cpu_started_at, cpu, nr_cpu_ids); if (started_at) - *started_at =3D bpf_ktime_get_ns() ?: 1; /* 0 indicates idle */ + *started_at =3D scx_bpf_clock_get_ns() ?: 1; /* 0 indicates idle */ } =20 void BPF_STRUCT_OPS(central_stopping, struct task_struct *p, bool runnable) @@ -258,7 +258,7 @@ void BPF_STRUCT_OPS(central_stopping, struct task_struc= t *p, bool runnable) =20 static int central_timerfn(void *map, int *key, struct bpf_timer *timer) { - u64 now =3D bpf_ktime_get_ns(); + u64 now =3D scx_bpf_clock_get_ns(); u64 nr_to_kick =3D nr_queued; s32 i, curr_cpu; =20 diff --git a/tools/sched_ext/scx_flatcg.bpf.c b/tools/sched_ext/scx_flatcg.= bpf.c index 4e3afcd260bf..15351bf2f053 100644 --- a/tools/sched_ext/scx_flatcg.bpf.c +++ b/tools/sched_ext/scx_flatcg.bpf.c @@ -734,7 +734,7 @@ void BPF_STRUCT_OPS(fcg_dispatch, s32 cpu, struct task_= struct *prev) struct fcg_cpu_ctx *cpuc; struct fcg_cgrp_ctx *cgc; struct cgroup *cgrp; - u64 now =3D bpf_ktime_get_ns(); + u64 now =3D scx_bpf_clock_get_ns(); bool picked_next =3D false; =20 cpuc =3D find_cpu_ctx(); --=20 2.47.1