From nobody Sun Feb 8 12:43:29 2026 Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 00AEE19992E for ; Fri, 20 Dec 2024 06:20:34 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734675636; cv=none; b=YM/rei9coU1neYLZnAfNR1ojBdaOSG7oH9bFItFQIpAz226OSjj/1EOTYn0Ll242Gr2/qInFv2AcvC1IAo0FDVklN4GAUa5Sgi7tO0w3Nm1McwxK+wIZHuD2AQ/8Epg0tLVsOlLJx8Huje2C3qDfxy7bSUpvFzCOI/WVzCTvz9g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734675636; c=relaxed/simple; bh=TlSqGNDpjyEQ/sAS+WG9j8ZbPrrXCvTJG01tJ8bLoZ0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=bGPqLAugQC3j/YlUJWCiTdwRcaxwdpU3uGJCxgIOqB3p8i39FWqi24dnApCvGlBGh1vexH60VLVhN1ckQfvT/grTCngT5iG6ZHVuo7EF/rxb7KcDfwWB52G+K6FbE1+iR9l/Li4Xb5JUvwoSY+mhG86Fow/E4WShiFgnBE1WgI4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=dZmC5DxG; arc=none smtp.client-ip=209.85.210.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="dZmC5DxG" Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-725d9f57d90so1194107b3a.1 for ; Thu, 19 Dec 2024 22:20:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734675634; x=1735280434; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ot9Lx1zg/XLL07layOuusQRDG2mU5e3t2AM5FahhmOs=; b=dZmC5DxGiSFVv1xGJXrEl2e8cFdgVocZiIJyOWA7Pdh2zXqgt9o0FVzuR/VW4hDZLG /bXPx60sYw9aRWOegnLIXGNyMXfgjUanN8sZzY56gCyZAHK6mXP+tzTIfsndiNZMsCpD hbTfMJzWr5NV8EtNrbbN8t/Lca9SRxBAt+K15VeuUSrNq0TUkM1xAQBQXLarE1zp/u3B 9z4FO7p9wOK8mZ/U4aE3aHMlvaQNDuHdZRFRL8X+tKaHyJn7+XBAcW9HYFJRYvlXpux2 HkVEiBZ3etl1uzpcWUr0MRRuP6crb5qQWRLtuxWhHh3Sh/DofT7fbk5oZwnHJVFj3mSi r3aQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734675634; x=1735280434; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ot9Lx1zg/XLL07layOuusQRDG2mU5e3t2AM5FahhmOs=; b=VauzdgTgPNg+hska7/ZyO78icIUEePUR6elD9Q1aLYtvEepDQLchFTfm9NlR0misYr D47CYZyAsHnpKRK5iAcXyEc8CbRGrSZvwSOTc2fNHbc0+bTJ9Bbutyx7qTc6aXGgZQWX KPSXn63KjgavrsaTt4dTmIVpaDObccVArOcMWZoMdUWtp/T9hQpBsOwpG8VZbbEvrFcA 4bApZfx6WJxW8evBkRG5XaGw1U+sr7r/jNVvG53keF8+g1H2wxzN55h3ylEY+UbwS4rm xllSLEekB/KzeTaMrGXq64Rl4E/HsdScRFrKMUnNQCGfn+Bc4MmnXpo05BcUI94WT6hL 3hFQ== X-Forwarded-Encrypted: i=1; AJvYcCUrBzXUgBu00oGF58gnG8z+GY+kFITeoE5Jpl0JXSraIUAKy/bkDm8PL3/hnc5GM4bDX86np/6STUVjb2w=@vger.kernel.org X-Gm-Message-State: AOJu0YxzKE5gm4VGBbat/0DNtzs3nkGN5EDIrQjwLJ/KIZK50JVCS67N dmW5ga+ohgKRBiSQllOSp+ejP6koIW3iqx+WhA2CJAGjhS823fPYBVoXWg== X-Gm-Gg: ASbGncus2iuKIEF314fcRmFrCRVEPE7JY9NiqVMt0dhpCpvmGgV1AFTAIx/JIlB2oLd HhA9dctAmzzS53qTKKGbHNeUu7vPpxszOt0fNnWUJGbBBatQH0oKJwZXpi2QpOuY42aFFkfgr0o Bk6BLuuy3GgMJwL+TED8y7SStbm23qTZhB3HmWrouZDB1C6A00Q7b4tAXBbaPX54DMpTkgFDAw8 C0ZzSrICw9kdZsuOQ+1AyP+3icpzXwwIZ3sienWZ2mzf8A7ZraR4c7zt4tonB1K+sH0PikbiAPm iLW2WyT6OR4G X-Google-Smtp-Source: AGHT+IEANl5UiFOkGUoj32appW1tmSm4O1mRamD8CbklJhgQkYOyRx0aAIu2qMw9EqGr3TXpDOONbA== X-Received: by 2002:a05:6a00:1411:b0:725:f376:f4f4 with SMTP id d2e1a72fcca58-72abde01aa8mr2144438b3a.13.1734675634104; Thu, 19 Dec 2024 22:20:34 -0800 (PST) Received: from localhost ([58.29.143.236]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72aad8fb9a8sm2349000b3a.157.2024.12.19.22.20.33 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Dec 2024 22:20:33 -0800 (PST) From: Changwoo Min X-Google-Original-From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com, mingo@redhat.com, peterz@infradead.org Cc: changwoo@igalia.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: [PATCH v6 1/6] sched_ext: Relocate scx_enabled() related code Date: Fri, 20 Dec 2024 15:20:20 +0900 Message-ID: <20241220062025.27724-2-changwoo@igalia.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241220062025.27724-1-changwoo@igalia.com> References: <20241220062025.27724-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" scx_enabled() will be used in scx_rq_clock_update/invalidate() in the following patch, so relocate the scx_enabled() related code to the proper location. Signed-off-by: Changwoo Min Tested-by: Andrea Righi --- kernel/sched/sched.h | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 76f5f53a645f..440ecedf871b 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1717,6 +1717,19 @@ struct rq_flags { =20 extern struct balance_callback balance_push_callback; =20 +#ifdef CONFIG_SCHED_CLASS_EXT +extern const struct sched_class ext_sched_class; + +DECLARE_STATIC_KEY_FALSE(__scx_ops_enabled); /* SCX BPF scheduler loaded */ +DECLARE_STATIC_KEY_FALSE(__scx_switched_all); /* all fair class tasks on S= CX */ + +#define scx_enabled() static_branch_unlikely(&__scx_ops_enabled) +#define scx_switched_all() static_branch_unlikely(&__scx_switched_all) +#else /* !CONFIG_SCHED_CLASS_EXT */ +#define scx_enabled() false +#define scx_switched_all() false +#endif /* !CONFIG_SCHED_CLASS_EXT */ + /* * Lockdep annotation that avoids accidental unlocks; it's like a * sticky/continuous lockdep_assert_held(). @@ -2505,19 +2518,6 @@ extern const struct sched_class rt_sched_class; extern const struct sched_class fair_sched_class; extern const struct sched_class idle_sched_class; =20 -#ifdef CONFIG_SCHED_CLASS_EXT -extern const struct sched_class ext_sched_class; - -DECLARE_STATIC_KEY_FALSE(__scx_ops_enabled); /* SCX BPF scheduler loaded */ -DECLARE_STATIC_KEY_FALSE(__scx_switched_all); /* all fair class tasks on S= CX */ - -#define scx_enabled() static_branch_unlikely(&__scx_ops_enabled) -#define scx_switched_all() static_branch_unlikely(&__scx_switched_all) -#else /* !CONFIG_SCHED_CLASS_EXT */ -#define scx_enabled() false -#define scx_switched_all() false -#endif /* !CONFIG_SCHED_CLASS_EXT */ - /* * Iterate only active classes. SCX can take over all fair tasks or be * completely disabled. If the former, skip fair. If the latter, skip SCX. --=20 2.47.1 From nobody Sun Feb 8 12:43:29 2026 Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BE48119F43A for ; Fri, 20 Dec 2024 06:20:38 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734675640; cv=none; b=nrbKBEeIJstoG1IItHCOy431UnQbWnlhW859IdT0MqVT4mf8cGFRVOp1rNdmJ44zR13JLaMRXz9woi6Kd4A8iIkTgkFCvzzcpzri8lKs7BWN3KNWrWE9Y+C1m4aM2cYU8GT3BpV5XvnrOMTXTq0iJ9ByF5p+clfFKDPAAfXDQ9E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734675640; c=relaxed/simple; bh=LRV1YLe5fvzqVYPXiGRzo5vSBEim+526pcYQCkEgPh4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=QK8mMhkUEQbwziM3/m5OGBMQS37YCnGun0OmD6+jgV9xbDQrcVjLUNQ0FCZm36PrR+MbOVUQWGkhIY0vGZtN97UZDJsW15eENViIh2euVgbAL73WGdAegsPKGuAmGGTVujV8Ph2+w6FjeYbrdpHgneYTVl437QkeSZlX9Y86JsM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=nIODQVYK; arc=none smtp.client-ip=209.85.210.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="nIODQVYK" Received: by mail-pf1-f178.google.com with SMTP id d2e1a72fcca58-7265c18d79bso1812931b3a.3 for ; Thu, 19 Dec 2024 22:20:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734675638; x=1735280438; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Z7NA3+GcERPaGipSpS597mxR99dyljtPqLsP268sYPY=; b=nIODQVYKjhNLmUyPE7Z+DxRz1dHSjhPtIuNXi0xzSdOWspFIb3skvDKfL/eO2UcbYk FvlHe6ujECIvzEw1gl/thREj7UQJBZ3eR52PwZddn10DiuHhnMRSnwTVpMeqa58ORnxS TXa8yFYoTcyGp/ghjg7ufsEjBuezsXhz7z18TGyexulLdzU7iBnUuQX6FEDaO35e6JO1 2NsUOrQGpbi2kPCAFMn9yTF9YmFy6t7qS95ZB3BQRKVHEow8We+8uAS46RQJBrYAnisE i/eo1jAgQjSkzt8pmFs+DpAtos/YasfNlucnRnTxk+mI2Gj3LDuGTN7yAofhLND/d4J9 JKEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734675638; x=1735280438; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Z7NA3+GcERPaGipSpS597mxR99dyljtPqLsP268sYPY=; b=WL0XA/OGyIwPabfp7VBBbMmixjHXbkoVgjR0S98411qh3a6ewGmu4rvE7qoq6eUTxd rD0CTfuzy+2FdkpM+p36AgefoN5Ln1LsrSWFaZB9aKURy0NVSi2riTXivWOlr7Svc1n9 cbnxkuIVZ9PWMU+ZvXgf/nM643ABMhxlROvU9zMw/9X5Y5QI3JMBFdrpuwIiLP7x3atx 8/xTyPbCXfIpZkWlHQ+TzSozah7nzPT1BQyB7LOC2P4tAK6M9MLjVXaNzYk7DNW0OuHQ +nBCRVgU4A3M35bzo6BgWyQKO+JGtvI3mK0Woxz8pRn3Fe+CAAGdmAkuov0kNeZc7PCw GAkQ== X-Forwarded-Encrypted: i=1; AJvYcCWA2RIO+DkiNl/mU2U7EFUkPhfnY63s5i3s0Djnt+7wqDcQ/kKHXpJz0NCLPK28gVywn98cyGCBdH5sJG8=@vger.kernel.org X-Gm-Message-State: AOJu0Ywhe4wzwKKfLoIxe9UKD+McV+ciIyVQw4EstzA4n7fOKQ8MdEvD ASrICg9+Gh7eSCqQu5n7CWM6d1ROzZCZV1lFncqguvC0zcP4Wr4M X-Gm-Gg: ASbGncto34Z4DzdsK/eakRwcOR6GeatyRnrZDAxEgu+Oy+O3Bf60NCfOVMOH9tOOmnc tOKBfwhX7nI5z1jDqtYFneMB+rCR51Pi+elkscf0zS7sTqnlOLcwb9XNa9HVnKDYb21F/2cXex4 c5XyESghug9lZl5qC4X3e14+C2hWAlbxmF5JcmdZCLjEpVWAqUCnuvIFGeydncPdFFnZ8VUTHkW cCOANThsIbVY8M++t+Y0lFHnOv0qxF3ZePMNp9P28Rcy4q0D1ZfAuTKbp09GAZLtkSoCgBhByVp kUTBSyeT1D4f X-Google-Smtp-Source: AGHT+IFYG4zFy6YyxqXiB38t2dEwutkukyTXuOR2MFCwSJvsW+IunO3EiRE5mTBJC7nWcPiwhaRngQ== X-Received: by 2002:aa7:9315:0:b0:727:4e5e:881c with SMTP id d2e1a72fcca58-72abddd7b93mr1985918b3a.15.1734675637624; Thu, 19 Dec 2024 22:20:37 -0800 (PST) Received: from localhost ([58.29.143.236]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72aad8fb9a8sm2349132b3a.157.2024.12.19.22.20.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Dec 2024 22:20:37 -0800 (PST) From: Changwoo Min X-Google-Original-From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com, mingo@redhat.com, peterz@infradead.org Cc: changwoo@igalia.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: [PATCH v6 2/6] sched_ext: Implement scx_bpf_now_ns() Date: Fri, 20 Dec 2024 15:20:21 +0900 Message-ID: <20241220062025.27724-3-changwoo@igalia.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241220062025.27724-1-changwoo@igalia.com> References: <20241220062025.27724-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Returns a high-performance monotonically non-decreasing clock for the curre= nt CPU. The clock returned is in nanoseconds. It provides the following properties: 1) High performance: Many BPF schedulers call bpf_ktime_get_ns() frequently to account for execution time and track tasks' runtime properties. Unfortunately, in some hardware platforms, bpf_ktime_get_ns() -- which eventually reads a hardware timestamp counter -- is neither performant nor scalable. scx_bpf_now_ns() aims to provide a high-performance clock by using the rq clock in the scheduler core whenever possible. 2) High enough resolution for the BPF scheduler use cases: In most BPF scheduler use cases, the required clock resolution is lower than the most accurate hardware clock (e.g., rdtsc in x86). scx_bpf_now_ns() basically uses the rq clock in the scheduler core whenever it is valid. It considers that the rq clock is valid from the time the rq clock is updated (update_rq_clock) until the rq is unlocked (rq_unpin_lock). 3) Monotonically non-decreasing clock for the same CPU: scx_bpf_now_ns() guarantees the clock never goes backward when comparing them in the same CPU. On the other hand, when comparing clocks in different CPUs, there is no such guarantee -- the clock can go backward. It provides a monotonically *non-decreasing* clock so that it would provide the same clock values in two different scx_bpf_now_ns() calls in the same CPU during the same period of when the rq clock is valid. An rq clock becomes valid when it is updated using update_rq_clock() and invalidated when the rq is unlocked using rq_unpin_lock(). Let's suppose the following timeline in the scheduler core: T1. rq_lock(rq) T2. update_rq_clock(rq) T3. a sched_ext BPF operation T4. rq_unlock(rq) T5. a sched_ext BPF operation T6. rq_lock(rq) T7. update_rq_clock(rq) For [T2, T4), we consider that rq clock is valid (SCX_RQ_CLK_VALID is set), so scx_bpf_now_ns() calls during [T2, T4) (including T3) will return the rq clock updated at T2. For duration [T4, T7), when a BPF scheduler can still call scx_bpf_now_ns() (T5), we consider the rq clock is invalid (SCX_RQ_CLK_VALID is unset at T4). So when calling scx_bpf_now_ns() at T5, we will return a fresh clock value by calling sched_clock_cpu() internally. Also, to prevent getting outdated rq clocks from a previous scx scheduler, invalidate all the rq clocks when unloading a BPF scheduler. One example of calling scx_bpf_now_ns(), when the rq clock is invalid (like T5), is in scx_central [1]. The scx_central scheduler uses a BPF timer for preemptive scheduling. In every msec, the timer callback checks if the currently running tasks exceed their timeslice. At the beginning of the BPF timer callback (central_timerfn in scx_central.bpf.c), scx_central gets the current time. When the BPF timer callback runs, the rq clock could be invalid, the same as T5. In this case, scx_bpf_now_ns() returns a fresh clock value rather than returning the old one (T2). [1] https://github.com/sched-ext/scx/blob/main/scheds/c/scx_central.bpf.c Signed-off-by: Changwoo Min Tested-by: Andrea Righi --- kernel/sched/core.c | 6 +++- kernel/sched/ext.c | 75 +++++++++++++++++++++++++++++++++++++++++++- kernel/sched/sched.h | 26 +++++++++++++-- 3 files changed, 103 insertions(+), 4 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 95e40895a519..b0191977d919 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -789,6 +789,7 @@ static void update_rq_clock_task(struct rq *rq, s64 del= ta) void update_rq_clock(struct rq *rq) { s64 delta; + u64 clock; =20 lockdep_assert_rq_held(rq); =20 @@ -800,11 +801,14 @@ void update_rq_clock(struct rq *rq) SCHED_WARN_ON(rq->clock_update_flags & RQCF_UPDATED); rq->clock_update_flags |=3D RQCF_UPDATED; #endif + clock =3D sched_clock_cpu(cpu_of(rq)); + scx_rq_clock_update(rq, clock, true); =20 - delta =3D sched_clock_cpu(cpu_of(rq)) - rq->clock; + delta =3D clock - rq->clock; if (delta < 0) return; rq->clock +=3D delta; + update_rq_clock_task(rq, delta); } =20 diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index afeed9dfeecb..248b8c91149f 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -4910,7 +4910,7 @@ static void scx_ops_disable_workfn(struct kthread_wor= k *work) struct task_struct *p; struct rhashtable_iter rht_iter; struct scx_dispatch_q *dsq; - int i, kind; + int i, kind, cpu; =20 kind =3D atomic_read(&scx_exit_kind); while (true) { @@ -4993,6 +4993,15 @@ static void scx_ops_disable_workfn(struct kthread_wo= rk *work) scx_task_iter_stop(&sti); percpu_up_write(&scx_fork_rwsem); =20 + /* + * Invalidate all the rq clocks to prevent getting outdated + * rq clocks from a previous scx scheduler. + */ + for_each_possible_cpu(cpu) { + struct rq *rq =3D cpu_rq(cpu); + scx_rq_clock_invalidate(rq); + } + /* no task is on scx, turn off all the switches and flush in-progress cal= ls */ static_branch_disable(&__scx_ops_enabled); for (i =3D SCX_OPI_BEGIN; i < SCX_OPI_END; i++) @@ -7601,6 +7610,69 @@ __bpf_kfunc struct cgroup *scx_bpf_task_cgroup(struc= t task_struct *p) } #endif =20 +/** + * scx_bpf_now_ns - Returns a high-performance monotonically non-decreasing + * clock for the current CPU. The clock returned is in nanoseconds. + * + * It provides the following properties: + * + * 1) High performance: Many BPF schedulers call bpf_ktime_get_ns() freque= ntly + * to account for execution time and track tasks' runtime properties. + * Unfortunately, in some hardware platforms, bpf_ktime_get_ns() -- which + * eventually reads a hardware timestamp counter -- is neither performant= nor + * scalable. scx_bpf_now_ns() aims to provide a high-performance clock by + * using the rq clock in the scheduler core whenever possible. + * + * 2) High enough resolution for the BPF scheduler use cases: In most BPF + * scheduler use cases, the required clock resolution is lower than the m= ost + * accurate hardware clock (e.g., rdtsc in x86). scx_bpf_now_ns() basical= ly + * uses the rq clock in the scheduler core whenever it is valid. It consi= ders + * that the rq clock is valid from the time the rq clock is updated + * (update_rq_clock) until the rq is unlocked (rq_unpin_lock). + * + * 3) Monotonically non-decreasing clock for the same CPU: scx_bpf_now_ns() + * guarantees the clock never goes backward when comparing them in the sa= me + * CPU. On the other hand, when comparing clocks in different CPUs, there + * is no such guarantee -- the clock can go backward. It provides a + * monotonically *non-decreasing* clock so that it would provide the same + * clock values in two different scx_bpf_now_ns() calls in the same CPU + * during the same period of when the rq clock is valid. + */ +__bpf_kfunc u64 scx_bpf_now_ns(void) +{ + struct rq *rq; + u64 clock; + + preempt_disable(); + + /* + * If the rq clock is valid, use the cached rq clock. + * Otherwise, return a fresh rq glock. + * + * Note that scx_bpf_now_ns() is re-entrant between a process + * context and an interrupt context (e.g., timer interrupt). + * However, we don't need to consider the race between them + * because such race is not observable from a caller. + */ + rq =3D this_rq(); + clock =3D READ_ONCE(rq->scx.clock); + + if (!(READ_ONCE(rq->scx.flags) & SCX_RQ_CLK_VALID)) { + clock =3D sched_clock_cpu(cpu_of(rq)); + + /* + * The rq clock is updated outside of the rq lock. + * In this case, keep the updated rq clock invalid so the next + * kfunc call outside the rq lock gets a fresh rq clock. + */ + scx_rq_clock_update(rq, clock, false); + } + + preempt_enable(); + + return clock; +} + __bpf_kfunc_end_defs(); =20 BTF_KFUNCS_START(scx_kfunc_ids_any) @@ -7632,6 +7704,7 @@ BTF_ID_FLAGS(func, scx_bpf_cpu_rq) #ifdef CONFIG_CGROUP_SCHED BTF_ID_FLAGS(func, scx_bpf_task_cgroup, KF_RCU | KF_ACQUIRE) #endif +BTF_ID_FLAGS(func, scx_bpf_now_ns) BTF_KFUNCS_END(scx_kfunc_ids_any) =20 static const struct btf_kfunc_id_set scx_kfunc_set_any =3D { diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 440ecedf871b..53384012e77b 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -754,6 +754,7 @@ enum scx_rq_flags { SCX_RQ_BAL_PENDING =3D 1 << 2, /* balance hasn't run yet */ SCX_RQ_BAL_KEEP =3D 1 << 3, /* balance decided to keep current */ SCX_RQ_BYPASSING =3D 1 << 4, + SCX_RQ_CLK_VALID =3D 1 << 5, /* RQ clock is fresh and valid */ =20 SCX_RQ_IN_WAKEUP =3D 1 << 16, SCX_RQ_IN_BALANCE =3D 1 << 17, @@ -766,9 +767,10 @@ struct scx_rq { unsigned long ops_qseq; u64 extra_enq_flags; /* see move_task_to_local_dsq() */ u32 nr_running; - u32 flags; u32 cpuperf_target; /* [0, SCHED_CAPACITY_SCALE] */ bool cpu_released; + u32 flags; + u64 clock; /* current per-rq clock -- see scx_bpf_now_ns() */ cpumask_var_t cpus_to_kick; cpumask_var_t cpus_to_kick_if_idle; cpumask_var_t cpus_to_preempt; @@ -1725,9 +1727,29 @@ DECLARE_STATIC_KEY_FALSE(__scx_switched_all); /* all= fair class tasks on SCX */ =20 #define scx_enabled() static_branch_unlikely(&__scx_ops_enabled) #define scx_switched_all() static_branch_unlikely(&__scx_switched_all) + +static inline void scx_rq_clock_update(struct rq *rq, u64 clock, bool vali= d) +{ + if (!scx_enabled()) + return; + WRITE_ONCE(rq->scx.clock, clock); + if (valid) + WRITE_ONCE(rq->scx.flags, rq->scx.flags | SCX_RQ_CLK_VALID); +} + +static inline void scx_rq_clock_invalidate(struct rq *rq) +{ + if (!scx_enabled()) + return; + WRITE_ONCE(rq->scx.flags, rq->scx.flags & ~SCX_RQ_CLK_VALID); +} + #else /* !CONFIG_SCHED_CLASS_EXT */ #define scx_enabled() false #define scx_switched_all() false + +static inline void scx_rq_clock_update(struct rq *rq, u64 clock, bool vali= d) {} +static inline void scx_rq_clock_invalidate(struct rq *rq) {} #endif /* !CONFIG_SCHED_CLASS_EXT */ =20 /* @@ -1759,7 +1781,7 @@ static inline void rq_unpin_lock(struct rq *rq, struc= t rq_flags *rf) if (rq->clock_update_flags > RQCF_ACT_SKIP) rf->clock_update_flags =3D RQCF_UPDATED; #endif - + scx_rq_clock_invalidate(rq); lockdep_unpin_lock(__rq_lockp(rq), rf->cookie); } =20 --=20 2.47.1 From nobody Sun Feb 8 12:43:29 2026 Received: from mail-pj1-f53.google.com (mail-pj1-f53.google.com [209.85.216.53]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4EA4A19FA9D for ; Fri, 20 Dec 2024 06:20:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.53 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734675642; cv=none; b=h6BzSxchGXRsrJ0NsGD646j9vhqYBwXxV9eeleXRzhI/44Tv3sto/gcaGIomChAHwAuYC9jX6tVTSiElClv4npva0feNb18soOpaQHfzVeWbswziYF1gsqU4yUWKm4yx/19lmW8Na9v3YudjG5EuanS0ZyJkg9qy+26d3TsLs8g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734675642; c=relaxed/simple; bh=GZDtGSP3knc4XfhDJndoAF3aAtv9DQSb0onbM9mot6Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BRYRvNL9OFRAtTitLU9TnOFL6pc01iQkLwkbH+WQdNshNzu+K8Bdy6kFZu0oDvjv89EdpqzZH7CZrMEkKb4SH5hh6Y0x3qjs7AdzWCuaH5g5aetJrio33pRI6D1SFWaQagf3ftMBChXbw4dXm9/2uOFTIKNvr97DcTcAX74iFUA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=R+mBbBRe; arc=none smtp.client-ip=209.85.216.53 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="R+mBbBRe" Received: by mail-pj1-f53.google.com with SMTP id 98e67ed59e1d1-2ee50ffcf14so2171536a91.0 for ; Thu, 19 Dec 2024 22:20:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734675640; x=1735280440; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3qwdnhpnNZ/kDLebmCqGGOAqoWdsfoXXWVd5XUHDoNY=; b=R+mBbBRer0T8WdBX23NtDjdIBhrHwpleN8dFThiztMyBWC8fylyf93EKTxsoJZjvZN BwbyCl1rHHO1OimK+MYZcB3HKieuMDAV6uoPjZOhQdgVppHfT6qqDaTmGeVFhbRVp9yE YH4tclYHZj1ic91ggtk/SGEtEfYLMkEuk7a7xyvxUnbW8F4uqJLE11L32qwP/nCjypAp stC7aefcDaE4FddpvlYxBKrSnnSvsyAd7wAL7uJ/N3jqaimoFs2x8WSyrqhppXjk2auc vfyqeAjppA+h8OM8ZD11JomgAoz51d1upA7H6Y7E1UYdT5vq72q6IJBM5d1Maiftq9tK rFTA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734675640; x=1735280440; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3qwdnhpnNZ/kDLebmCqGGOAqoWdsfoXXWVd5XUHDoNY=; b=WFkrBI4BlCOsIsOtW/RU50ZhZfYYhhz4yLrwZWBTv2G+amalYzkbpM4XC8WCIOOu1H ECFEVziAGCckgi8K92Bku5VmVZcUkKNvueg6WTDDjh91xHo7AQK92iAjymPRw77ZaNYR p9bjRyvq9C4X1DIVst3twsDZeJBrixwYJsXwu9nfDyiIiCujve9ETMiJ6Zc/2MFGfEIP 0A+hztMjnaEbGVRHZzxVHgO97GKbu/7oOHpS/m/7piNj9udua3krRwBSeJirQRjpufju 62xYyD+eu1uIUk9qsc8xgLwwMrP51kjbB2VzXJ1oGIgEgy5JCQBIfRIC8vE+2HdFCut2 Fn7Q== X-Forwarded-Encrypted: i=1; AJvYcCU/oVxtbN0RCY3fnb6Z8oN+lQ3XP+hLZ8+n0Niqme1PqThinSHzuvsIIHTLPylgL55eDdYt9oqBQMVhGbo=@vger.kernel.org X-Gm-Message-State: AOJu0YwRvlbwm4QU2duAwJwn+dq3Z7A4RAnUkbrUcMnYYom+crf3gfQ+ U8GcsTMOdz2qEHMfhnqHjPYTSLIQCsRkeAJgRS3MzYXhRjS72AkaInsndg== X-Gm-Gg: ASbGncu7OxswgB2BFkBy7joXSSJ4xT9tAZU7jFHAGqkY/M0SMfaHdLp3IFHsaxgFICV 2UMI9bwbqX9JS4CV5OwDP2frf4Z7gFMuL6ZISZa6cWrKdtVZGbUTKEzgqxWmm4z4Bu1+6vmlgg+ DaMqbfSwZ3L5pZpjFK89oaQrV3ZWfvrexz5dI3ccF8I8HEt4arJDuSyIivSACJfCWnkOiPm0e1j /dwMr1C7AKbqNkuGymt6B3HdMsRsDk4qStyVkDPasDd/bZxdoK1e1U9p6SetBwy7vH0dInzyr0l JaAN9uSuvSjS X-Google-Smtp-Source: AGHT+IE9trKwnu2T1nHGleo7LJX45/FVjHYPX1YK5kl+fqFFmdPFZykZxfq0SULE2I1wifivu9mfiA== X-Received: by 2002:a17:90b:2d4e:b0:2ea:61c4:a443 with SMTP id 98e67ed59e1d1-2f44353f053mr9242355a91.4.1734675640530; Thu, 19 Dec 2024 22:20:40 -0800 (PST) Received: from localhost ([58.29.143.236]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-219dca02f8asm21963655ad.279.2024.12.19.22.20.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Dec 2024 22:20:40 -0800 (PST) From: Changwoo Min X-Google-Original-From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com, mingo@redhat.com, peterz@infradead.org Cc: changwoo@igalia.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: [PATCH v6 3/6] sched_ext: Add scx_bpf_now_ns() for BPF scheduler Date: Fri, 20 Dec 2024 15:20:22 +0900 Message-ID: <20241220062025.27724-4-changwoo@igalia.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241220062025.27724-1-changwoo@igalia.com> References: <20241220062025.27724-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" scx_bpf_now_ns() is added to the header files so the BPF scheduler can use it. Signed-off-by: Changwoo Min Tested-by: Andrea Righi --- tools/sched_ext/include/scx/common.bpf.h | 1 + tools/sched_ext/include/scx/compat.bpf.h | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/tools/sched_ext/include/scx/common.bpf.h b/tools/sched_ext/inc= lude/scx/common.bpf.h index 858ba1f438f6..79f0798a5350 100644 --- a/tools/sched_ext/include/scx/common.bpf.h +++ b/tools/sched_ext/include/scx/common.bpf.h @@ -76,6 +76,7 @@ bool scx_bpf_task_running(const struct task_struct *p) __= ksym; s32 scx_bpf_task_cpu(const struct task_struct *p) __ksym; struct rq *scx_bpf_cpu_rq(s32 cpu) __ksym; struct cgroup *scx_bpf_task_cgroup(struct task_struct *p) __ksym __weak; +u64 scx_bpf_now_ns(void) __ksym __weak; =20 /* * Use the following as @it__iter when calling scx_bpf_dsq_move[_vtime]() = from diff --git a/tools/sched_ext/include/scx/compat.bpf.h b/tools/sched_ext/inc= lude/scx/compat.bpf.h index d56520100a26..2f16f30ae741 100644 --- a/tools/sched_ext/include/scx/compat.bpf.h +++ b/tools/sched_ext/include/scx/compat.bpf.h @@ -125,6 +125,11 @@ bool scx_bpf_dispatch_vtime_from_dsq___compat(struct b= pf_iter_scx_dsq *it__iter, false; \ }) =20 +#define scx_bpf_now_ns() \ + (bpf_ksym_exists(scx_bpf_now_ns) ? \ + scx_bpf_now_ns() : \ + bpf_ktime_get_ns()) + /* * Define sched_ext_ops. This may be expanded to define multiple variants = for * backward compatibility. See compat.h::SCX_OPS_LOAD/ATTACH(). --=20 2.47.1 From nobody Sun Feb 8 12:43:29 2026 Received: from mail-pf1-f182.google.com (mail-pf1-f182.google.com [209.85.210.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 40ECE1A4E77 for ; Fri, 20 Dec 2024 06:20:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734675645; cv=none; b=nW9rPnavNGyNojLsi5pJokaC5jfe/yKlW9w4JW3Mp8NgHIrZNPXGJBr3mH7SS7NxjN05QkhmiBVQqRLUgx37YnJ87dVRiEHjjmTTBHILHv04TOSMjGuxC4wEVWsFLA9LoYh/YIv9rQnmN356AxDTeff9ofZXBWRKcfKKk0nVc6o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734675645; c=relaxed/simple; bh=tw/sZ6xyV0Pw2jZXspaHJWiWfZ9xSORxU2ivpIey0fc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RL0Jq2kurJVzAgTau/b4a9ceplcTT690MkEI1gPjhliZvzyvSdvvaflG6x2F6vl47zCMeyYnlnOChTYGtktO8+cH4/yCSYcP/VUg6SF5ISKPTKKG8fzC1G9Ev3JE066jG01ahDXYVjbMW8YD/WO64kz5fDvssXm6LtanWvI/4tE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ddcu/3Kp; arc=none smtp.client-ip=209.85.210.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ddcu/3Kp" Received: by mail-pf1-f182.google.com with SMTP id d2e1a72fcca58-725ee6f56b4so1403650b3a.3 for ; Thu, 19 Dec 2024 22:20:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734675643; x=1735280443; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=K16k0CtGst/uGn1z/pOeYRcdvDetSUcyUB4347UZ9Ic=; b=ddcu/3KpY9dAYAg7n50JJnkK+jPrKXkJdAEijwHRzpW0N+KUIDbsUGQ6zKbzrEVqDQ jVbe2z+qCSXtBWx8Zt29kgAqrcidypv0AVQBbB3qUy/UafoZZg9HNsSiw0Iy+V585fU7 caZnUgz+zxF3a5sYaDXlUe0kHyGGoxoINfmzaxuKjghsFMB32CSJoCWKjRFeg3h61s1e 4EjQ10G641h//g8bFlM8w7bbSjJo7WeUIzA+kMCRN40MSfEf3HgfHEW8h0FDtEnNSG7B VWRWoEzTaRoJNqvyyW9gUSqPvZXyMbhly1o2IUzH7CJhlztr3Z4MQOqH/Ka69jieSGOI Gpyg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734675643; x=1735280443; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=K16k0CtGst/uGn1z/pOeYRcdvDetSUcyUB4347UZ9Ic=; b=gx1dJj2auO2uvF8XdpIvXprBOU88XRmjZbE3Rn5wkDIL9gEOEx3QQIMh6msliiA5Yu j14LweyyjNpV5bheEgMfHqeeUyahTotFf5idu8IA37t7rLbeLuIenULZ1jHNskpH1abT Y0ZgNG7PFNWjd+mizqKKUPocXKiAiog012ozz2ovIHnpwG02HnU7dgPn0XilR+CdGuh3 FlPNMPsG3e1U2Gs9fdZFvXgVFrTVNRYgv+gyBlAsox6rSEoDAdne6gA7ffM1INFELnzL 3GUFaOz8OfKs9XyNsiTy1tuz7ecAcisEF2q9/dQfp9x8jd0fLpMKZdS78sTp9jomrrI4 migQ== X-Forwarded-Encrypted: i=1; AJvYcCX9Qg5hlXP7dBKk3HV7wkGKy0xOpCNkuq2sxHUWr+x1BuG2opLFFulpOLy8sFJ4V10c0K/mymiR9qDkqO8=@vger.kernel.org X-Gm-Message-State: AOJu0YzT0dkl+3R7DWCNoEjVeLhnVsB3Mjjx84TfAc+EghnJUGJevRfC D415mFViPp/nl7nHn4uv3ScSrARep5RSXpoe5rJZfLO3Hryfibl0 X-Gm-Gg: ASbGncuirFMBPPEixp6dusm4kMCuT9mMUz4TZ78e+U//zy5Bj5rrNdbwlsVVV1Le8Av TDGIAg6sUNrT+AdGQ7Ns0qJBxHl0NA0pDES+ir+tQP0q+18A5uZiNJif7K0zXnir/JeV4yvXE1j rBNRuCQDBuHkanq5HDGjskPSouyVBR5hi/ZAxcZiVmhXkf0FtutotkjpMKCyrWJ7ur621L8isCN 5epxpNoECdF6iYRcGhJm3146vbQhF31ff7/ttufRIfX2UexkmdFxgdHhaXjivK0ZUMf3NkaX6R+ XqlNbo7slB6t X-Google-Smtp-Source: AGHT+IEG8yKNHZTP2e3oJeY29omPQVOECmUYMI8B/vEBuZ866QlbRGQ/4iV6FrdGCZLOi1HpwCP6UA== X-Received: by 2002:a05:6a00:8c4:b0:727:3c37:d5f9 with SMTP id d2e1a72fcca58-72abdeff523mr2136986b3a.26.1734675643359; Thu, 19 Dec 2024 22:20:43 -0800 (PST) Received: from localhost ([58.29.143.236]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72aad90c1bcsm2411084b3a.187.2024.12.19.22.20.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Dec 2024 22:20:43 -0800 (PST) From: Changwoo Min X-Google-Original-From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com, mingo@redhat.com, peterz@infradead.org Cc: changwoo@igalia.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: [PATCH v6 4/6] sched_ext: Add time helpers for BPF schedulers Date: Fri, 20 Dec 2024 15:20:23 +0900 Message-ID: <20241220062025.27724-5-changwoo@igalia.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241220062025.27724-1-changwoo@igalia.com> References: <20241220062025.27724-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The following functions are added for BPF schedulers: - vtime_delta(after, before) - vtime_after(a, b) - vtime_before(a, b) - vtime_after_eq(a, b) - vtime_before_eq(a, b) - vtime_in_range(a, b, c) - vtime_in_range_open(a, b, c) Signed-off-by: Changwoo Min Tested-by: Andrea Righi --- tools/sched_ext/include/scx/common.bpf.h | 94 ++++++++++++++++++++++++ 1 file changed, 94 insertions(+) diff --git a/tools/sched_ext/include/scx/common.bpf.h b/tools/sched_ext/inc= lude/scx/common.bpf.h index 79f0798a5350..923bbf57e4f1 100644 --- a/tools/sched_ext/include/scx/common.bpf.h +++ b/tools/sched_ext/include/scx/common.bpf.h @@ -408,6 +408,100 @@ static __always_inline const struct cpumask *cast_mas= k(struct bpf_cpumask *mask) void bpf_rcu_read_lock(void) __ksym; void bpf_rcu_read_unlock(void) __ksym; =20 +/* + * Time helpers, most of which are from jiffies.h. + */ + +/** + * vtime_delta - Calculate the delta between new and old time stamp + * @after: first comparable as u64 + * @before: second comparable as u64 + * + * Return: the time difference, which is >=3D 0 + */ +static inline s64 vtime_delta(u64 after, u64 before) +{ + return (s64)(after - before) > 0 ? : 0; +} + +/** + * vtime_after - returns true if the time a is after time b. + * @a: first comparable as u64 + * @b: second comparable as u64 + * + * Do this with "<0" and ">=3D0" to only test the sign of the result. A + * good compiler would generate better code (and a really good compiler + * wouldn't care). Gcc is currently neither. + * + * Return: %true is time a is after time b, otherwise %false. + */ +static inline bool vtime_after(u64 a, u64 b) +{ + return (s64)(b - a) < 0; +} + +/** + * vtime_before - returns true if the time a is before time b. + * @a: first comparable as u64 + * @b: second comparable as u64 + * + * Return: %true is time a is before time b, otherwise %false. + */ +static inline bool vtime_before(u64 a, u64 b) +{ + return vtime_after(b, a); +} + +/** + * vtime_after_eq - returns true if the time a is after or the same as tim= e b. + * @a: first comparable as u64 + * @b: second comparable as u64 + * + * Return: %true is time a is after or the same as time b, otherwise %fals= e. + */ +static inline bool vtime_after_eq(u64 a, u64 b) +{ + return (s64)(a - b) >=3D 0; +} + +/** + * vtime_before_eq - returns true if the time a is before or the same as t= ime b. + * @a: first comparable as u64 + * @b: second comparable as u64 + * + * Return: %true is time a is before or the same as time b, otherwise %fal= se. + */ +static inline bool vtime_before_eq(u64 a, u64 b) +{ + return vtime_after_eq(b, a); +} + +/** + * vtime_in_range - Calculate whether a is in the range of [b, c]. + * @a: time to test + * @b: beginning of the range + * @c: end of the range + * + * Return: %true is time a is in the range [b, c], otherwise %false. + */ +static inline bool vtime_in_range(u64 a, u64 b, u64 c) +{ + return vtime_after_eq(a, b) && vtime_before_eq(a, c); +} + +/** + * vtime_in_range_open - Calculate whether a is in the range of [b, c). + * @a: time to test + * @b: beginning of the range + * @c: end of the range + * + * Return: %true is time a is in the range [b, c), otherwise %false. + */ +static inline bool vtime_in_range_open(u64 a, u64 b, u64 c) +{ + return vtime_after_eq(a, b) && vtime_before(a, c); +} + =20 /* * Other helpers --=20 2.47.1 From nobody Sun Feb 8 12:43:29 2026 Received: from mail-pj1-f49.google.com (mail-pj1-f49.google.com [209.85.216.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 550631A83E8 for ; Fri, 20 Dec 2024 06:20:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.49 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734675649; cv=none; b=OUraPo67fn9xpCJki+kZhmHZS9rRZaUYrx4LiIQu49FvMHqNZ81EzucL4irKf07tKUBzD3sHxVQqqUXnSdcIeZr+hS9CgV5/dBLj5Q2QoOgDvigG5qWnus5VoEdfPS/AuRN994eoFiDXJbh4rzc6iO3EWbsJBBxkcbElbJZEB0o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734675649; c=relaxed/simple; bh=4stYz4/gGsi209Zl3d3WdyhWsyzSZCdqMC7HTz5fhyA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kDXNT6O422FeGJEynFvsvpGSuluToLxrwvqSo0OUFCQa3iu+Runoee2dyR0nWpfjo70gC4KfN78wKNP/Z4kj4dPlD06r71F9EE/MVeOxoJbymAYfH66ug8rC7VP1Aq5c5ndE8gt3c93rBaqSw5xbsYl8DNOc1CytysXaMrb5Q74= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=ZwPbcXt7; arc=none smtp.client-ip=209.85.216.49 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="ZwPbcXt7" Received: by mail-pj1-f49.google.com with SMTP id 98e67ed59e1d1-2efe25558ddso1108370a91.2 for ; Thu, 19 Dec 2024 22:20:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734675646; x=1735280446; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bMZAxlMblUHwQnDYumYsih8SVwxSsCKPzL7XYsgcWJQ=; b=ZwPbcXt7U/hONvaGFj5ixj9QD2ZcKeTt6rdg1HhOmNRNt3om0GPdB1MJFWHislO+M0 0QMfkOsCw9lTKytzwruQz9hm2N+YdT8ecRdhK/uNCIKSjBxdxyhaiAoc7MR/72qd3ydD E5zVWLeh0BOgfBBvZj4FqlaH3U/LTxZsb3K0A8m6C2MGomY4Mi6lcdwMKITMQPciYhdB GP8QEdrnYYcFeT7s5eVZCFkXysFRwwBh5RGucbR4995dI/fjvJEtLEOSXFZd6J6SKw6h wSEED/bsiH/ccoGlX8jz6C2vgoi0Mpc1+m02+lhUMo1RBOmEJuoIbEIxlqcKpyh55BvK 4Z3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734675646; x=1735280446; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bMZAxlMblUHwQnDYumYsih8SVwxSsCKPzL7XYsgcWJQ=; b=bbIls2eANhZz6IIQywRP7Dsp1/Ep0Xk5A+PY42CVk08KCIK7IZrYxuu/JMgtpu8pii qsyRq7G/F6fRUQZeQXRTd3ToL5HzlLvwcYuaOnkMX0w0j7Ez8jThXCXJv2HXFxBqahiL IgM+A94ZWvnGqXE4HOLm6wHF2VbYGCsn0EvatOTCZRQVwzVXKNf9rTu6xd7+xYm7o/yb tKJ0GD6nS7YaRI1JJe0bgDDLHwJn7bfHU6y6iCaKJ1n6D4C15xX5z/bbRxca3Qxryci4 sUJIFQyorUvHTnC+A6Km+2BshI4mv10vGMgeN0CNWb/71lrg3/4INI+pbgib1qSZIzpL vX+g== X-Forwarded-Encrypted: i=1; AJvYcCXANbhjG/r27h/JHBTqfbXielIlb7I0nFNLcktRnooATQp5jGdBWIjOsWxaYA5Vti7Swd331tME/VTbe1Y=@vger.kernel.org X-Gm-Message-State: AOJu0YyHHbrN/063hAFJ9gLoGhPtMA8iVVZ8SRtVt6I5RpnD33/hNh9K Y2EbVaKrNyqrHAHj/s4GIUjoK+jPSn1PbihDf4QDsQuswWVCzeEu X-Gm-Gg: ASbGncs2b7wds5Qp33WHWm+mAISkEZyRMHg47dGzrweCLHj9Kk4eNBk0RKz29JenA0g zUa153htDpecFuYibmWJQFaLo7nn03OQ3xl6OnsNSlvNyIV9lwrTnDj2Oadqnvjr+rw63UOeoI8 niT0QgvLqoxWcE3isz0+YSYHR7E4TXrsZNejyK4KqDjGnCpEjyfST+sbIhCxKoRBvCTczXjIjbh wGG7pobxYE1nqRnC4vmSkqU8gru3f0Y4YOSNmX5IAsLhnciNMfOj6EcRKUrqaGm8hW1OcMgIdMv OwnGh7CQyEt4 X-Google-Smtp-Source: AGHT+IEa6I86huR52jK6ghjC4HzgolpGc+NUkmr0CQoA6ak3jVcLCoc4FcTFG2i2q2LV2za7K7v9qw== X-Received: by 2002:a17:90b:520a:b0:2ee:96a5:721c with SMTP id 98e67ed59e1d1-2f452e38d67mr2427858a91.21.1734675646580; Thu, 19 Dec 2024 22:20:46 -0800 (PST) Received: from localhost ([58.29.143.236]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2f2db9939b5sm4840865a91.1.2024.12.19.22.20.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Dec 2024 22:20:46 -0800 (PST) From: Changwoo Min X-Google-Original-From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com, mingo@redhat.com, peterz@infradead.org Cc: changwoo@igalia.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: [PATCH v6 5/6] sched_ext: Replace bpf_ktime_get_ns() to scx_bpf_now_ns() Date: Fri, 20 Dec 2024 15:20:24 +0900 Message-ID: <20241220062025.27724-6-changwoo@igalia.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241220062025.27724-1-changwoo@igalia.com> References: <20241220062025.27724-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In the BPF schedulers that use bpf_ktime_get_ns() -- scx_central and scx_flatcg, replace bpf_ktime_get_ns() calls to scx_bpf_now_ns(). Signed-off-by: Changwoo Min Tested-by: Andrea Righi --- tools/sched_ext/scx_central.bpf.c | 4 ++-- tools/sched_ext/scx_flatcg.bpf.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tools/sched_ext/scx_central.bpf.c b/tools/sched_ext/scx_centra= l.bpf.c index 2907df78241e..ea1d853b9dd4 100644 --- a/tools/sched_ext/scx_central.bpf.c +++ b/tools/sched_ext/scx_central.bpf.c @@ -245,7 +245,7 @@ void BPF_STRUCT_OPS(central_running, struct task_struct= *p) s32 cpu =3D scx_bpf_task_cpu(p); u64 *started_at =3D ARRAY_ELEM_PTR(cpu_started_at, cpu, nr_cpu_ids); if (started_at) - *started_at =3D bpf_ktime_get_ns() ?: 1; /* 0 indicates idle */ + *started_at =3D scx_bpf_now_ns() ?: 1; /* 0 indicates idle */ } =20 void BPF_STRUCT_OPS(central_stopping, struct task_struct *p, bool runnable) @@ -258,7 +258,7 @@ void BPF_STRUCT_OPS(central_stopping, struct task_struc= t *p, bool runnable) =20 static int central_timerfn(void *map, int *key, struct bpf_timer *timer) { - u64 now =3D bpf_ktime_get_ns(); + u64 now =3D scx_bpf_now_ns(); u64 nr_to_kick =3D nr_queued; s32 i, curr_cpu; =20 diff --git a/tools/sched_ext/scx_flatcg.bpf.c b/tools/sched_ext/scx_flatcg.= bpf.c index 3dbfa82883be..85e33e45f818 100644 --- a/tools/sched_ext/scx_flatcg.bpf.c +++ b/tools/sched_ext/scx_flatcg.bpf.c @@ -734,7 +734,7 @@ void BPF_STRUCT_OPS(fcg_dispatch, s32 cpu, struct task_= struct *prev) struct fcg_cpu_ctx *cpuc; struct fcg_cgrp_ctx *cgc; struct cgroup *cgrp; - u64 now =3D bpf_ktime_get_ns(); + u64 now =3D scx_bpf_now_ns(); bool picked_next =3D false; =20 cpuc =3D find_cpu_ctx(); --=20 2.47.1 From nobody Sun Feb 8 12:43:29 2026 Received: from mail-pg1-f181.google.com (mail-pg1-f181.google.com [209.85.215.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 198B01A83F8 for ; Fri, 20 Dec 2024 06:20:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734675651; cv=none; b=MWx3NuzcfCZSAaOT11/GgYJCYEnoEC20k+OUnj8tIqmNYtXKK9oL1j/dEzAMjOmF98xZJdQhzk9DNlzwvG2bNVg/cCo2i3HgOr5VxcjNIRh6PFpJiwjVRXvXF7a7qa6L7HDtuh4v570l8jsqZHViN3eEIb4/3TyUgh/otZToPtQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734675651; c=relaxed/simple; bh=QV1F6m7RtxUBxKyIhCZOeu8HRvxTozYvf6p2RqU2fss=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gIr3fNBsENKy6v4SCp+D+m/v1QurtUZe9eJQJDvylOVKTYA14mT9PpLH9bvULcn+ZqW1ywlkYw+LmnVZSKFvI43Z2w2bk47LoTSnlpd50U//MZVmrcqAKJKcNJq2T6sRhGVTel3Pov78KRQ2+3rvPXOxmn7gKmQ2tsaa21kTviM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=KEOR7UHL; arc=none smtp.client-ip=209.85.215.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="KEOR7UHL" Received: by mail-pg1-f181.google.com with SMTP id 41be03b00d2f7-7ee11ff7210so1143203a12.1 for ; Thu, 19 Dec 2024 22:20:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734675649; x=1735280449; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=rOt0ueFwgFWxNaJC91o0b6u/gfq61/y7Hzh7uJU5b4g=; b=KEOR7UHLyxbotC2yXMshWv2HtC65mb5bxpcojGiK74SpxqpCJxUKCyUFw5uy2vFUi3 RlkYWt8bF2ax7W7GSiNHMbos9vW/0XYUd0YXgOOitE/kQ4VOr09y39mjAGSXbsLA/a5V dcYOVN/BxosoHIbFZ0IviqhxIyjNyT+KcI5kqYuaE/hL4OjWyzE022hPr/A3vN0RuERt NJSlM/Cu4ZvVbFkqATjiftcwTicnaT6DWXzqIruChvlKcwjqHa0jh4dzXXQNjhzIM+Aw MfCbauc+Lc0RZdl/i78Mw8aLyaoCsNRoPmPVbCJh3J0Uk4Lt/ryPY8kXigp9Oy8e9/40 ejcA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734675649; x=1735280449; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=rOt0ueFwgFWxNaJC91o0b6u/gfq61/y7Hzh7uJU5b4g=; b=ne5HRDLqGAu6qYrDW6r8/7CE2A6UTYE6yT6pVvrEhz96tXT6gzlIabVoIRXCcXVgtZ sIEv+2rDDU26GiFqdzSBdfX9Sdq4mBA2YNuid8Y7UIOX1lcJ6esIeC5rkC8WuVrK6IXz er/tQ+Efxr8fylG86C8kYTcTtwhN4a4I89T2DySvL2FM0lGJjmGrHQdVVr4rPIol7zWJ oRByDroJRNuR0yk1JNTQeu/GiSzJIPfejunws4hADWn//B7bnSjA/V6Fsv+BlWonvLb4 T0+lmHrg5RmVEAC+n6hDHa/uQkDs6dO3/nl30i0a1EkPg6djiDrP/tsUXdROv7zSXOi2 BsZA== X-Forwarded-Encrypted: i=1; AJvYcCXBd3PR1oztMtltHu5zkVdE6Vo+hL3yL6/k+BIqsnD/QwEOrga/MQja2peQ7KC7aj3HRSdjk9JXVye0VTs=@vger.kernel.org X-Gm-Message-State: AOJu0YydXw6ErtkiDErM4zYAXus+bOiafhb64pJGTnD2+SS/GB1FXelv 1ZLHZB/0FIb6cmkz7DE/2eQPoYs0feEzhYzt5MqFffNM86HQPALG X-Gm-Gg: ASbGnctz+bm/+w03N0AcqdFsfCrSxW/uzZlvwf3Wo64tqWSB//Jiv/Yi2L0oP8BMriD k4uwBs76Bp5jXXQnGRejoIrXs1jC9mu3lpcMG3KWpAQC5cjRBmaj+pCWlutKddTCJKq/Tc2+40J oRCrZHvIlgG62nYAGfM5OdkFmwTHH36utf+pPrXvtWH0+p8rg+PWpxtmPkMkrkOzH448qYUnWpE mSHribHA5JdyNhavJsdA013nM/lJ4/rkjI9GzB4smZxdnfoLJ0jSkuoc9B/bF5U77Xd/J5sci90 EycUK7a+030q X-Google-Smtp-Source: AGHT+IFGeRm1JVD08XDR78PDc5sexwZuy4nZkHKia7QavwIxFOryV8m+EZJRqnnnMYVYs9gxAhk5YQ== X-Received: by 2002:a05:6a21:e8d:b0:1e1:adcd:eae5 with SMTP id adf61e73a8af0-1e5e0846948mr3134128637.42.1734675649462; Thu, 19 Dec 2024 22:20:49 -0800 (PST) Received: from localhost ([58.29.143.236]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72aad90b90asm2344027b3a.177.2024.12.19.22.20.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 19 Dec 2024 22:20:49 -0800 (PST) From: Changwoo Min X-Google-Original-From: Changwoo Min To: tj@kernel.org, void@manifault.com, arighi@nvidia.com, mingo@redhat.com, peterz@infradead.org Cc: changwoo@igalia.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: [PATCH v6 6/6] sched_ext: Use time helpers in BPF schedulers Date: Fri, 20 Dec 2024 15:20:25 +0900 Message-ID: <20241220062025.27724-7-changwoo@igalia.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241220062025.27724-1-changwoo@igalia.com> References: <20241220062025.27724-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Modify the BPF schedulers to use time helpers defined in common.bpf.h Signed-off-by: Changwoo Min Tested-by: Andrea Righi --- tools/sched_ext/scx_central.bpf.c | 5 ----- tools/sched_ext/scx_flatcg.bpf.c | 11 +++-------- tools/sched_ext/scx_simple.bpf.c | 5 ----- 3 files changed, 3 insertions(+), 18 deletions(-) diff --git a/tools/sched_ext/scx_central.bpf.c b/tools/sched_ext/scx_centra= l.bpf.c index ea1d853b9dd4..c3b6998ea83e 100644 --- a/tools/sched_ext/scx_central.bpf.c +++ b/tools/sched_ext/scx_central.bpf.c @@ -87,11 +87,6 @@ struct { __type(value, struct central_timer); } central_timer SEC(".maps"); =20 -static bool vtime_before(u64 a, u64 b) -{ - return (s64)(a - b) < 0; -} - s32 BPF_STRUCT_OPS(central_select_cpu, struct task_struct *p, s32 prev_cpu, u64 wake_flags) { diff --git a/tools/sched_ext/scx_flatcg.bpf.c b/tools/sched_ext/scx_flatcg.= bpf.c index 85e33e45f818..2735ec25e511 100644 --- a/tools/sched_ext/scx_flatcg.bpf.c +++ b/tools/sched_ext/scx_flatcg.bpf.c @@ -137,11 +137,6 @@ static u64 div_round_up(u64 dividend, u64 divisor) return (dividend + divisor - 1) / divisor; } =20 -static bool vtime_before(u64 a, u64 b) -{ - return (s64)(a - b) < 0; -} - static bool cgv_node_less(struct bpf_rb_node *a, const struct bpf_rb_node = *b) { struct cgv_node *cgc_a, *cgc_b; @@ -920,14 +915,14 @@ void BPF_STRUCT_OPS(fcg_cgroup_move, struct task_stru= ct *p, struct cgroup *from, struct cgroup *to) { struct fcg_cgrp_ctx *from_cgc, *to_cgc; - s64 vtime_delta; + s64 delta; =20 /* find_cgrp_ctx() triggers scx_ops_error() on lookup failures */ if (!(from_cgc =3D find_cgrp_ctx(from)) || !(to_cgc =3D find_cgrp_ctx(to)= )) return; =20 - vtime_delta =3D p->scx.dsq_vtime - from_cgc->tvtime_now; - p->scx.dsq_vtime =3D to_cgc->tvtime_now + vtime_delta; + delta =3D vtime_delta(p->scx.dsq_vtime, from_cgc->tvtime_now); + p->scx.dsq_vtime =3D to_cgc->tvtime_now + delta; } =20 s32 BPF_STRUCT_OPS_SLEEPABLE(fcg_init) diff --git a/tools/sched_ext/scx_simple.bpf.c b/tools/sched_ext/scx_simple.= bpf.c index 31f915b286c6..6561d400ae6d 100644 --- a/tools/sched_ext/scx_simple.bpf.c +++ b/tools/sched_ext/scx_simple.bpf.c @@ -52,11 +52,6 @@ static void stat_inc(u32 idx) (*cnt_p)++; } =20 -static inline bool vtime_before(u64 a, u64 b) -{ - return (s64)(a - b) < 0; -} - s32 BPF_STRUCT_OPS(simple_select_cpu, struct task_struct *p, s32 prev_cpu,= u64 wake_flags) { bool is_idle =3D false; --=20 2.47.1