From nobody Sun Feb 8 13:39:49 2026 Received: from mail-pf1-f171.google.com (mail-pf1-f171.google.com [209.85.210.171]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 33E7428399 for ; Mon, 2 Dec 2024 04:39:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.171 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733114347; cv=none; b=NS9W66NT3VIvHhh3MaLwwjo+Dg1TgBK9EA9f2uTbwsYHVdE5nvtEGyinHvoyKFqmma2rrvU4AFLKJEv9fTwn2sRTtcD3oabzV5CDAwCr6kHLhCEJOpGKR3R/a2sE9qHYbCNh0do9lkik+5juB65NKaCwbbQ7SgXbxW5I5Jr1k5g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733114347; c=relaxed/simple; bh=pPlr8cBAY8ZlWbRFSuS5YgZiPfbogN78FxsS+StV5ew=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZPDw7N9PMcNZXdy8/n1DJLmTQhG/LkYJH9PTMEtksjWtB6AC24VWMhpeos8l6MrAFGgv8H1v0OZ8EUGXrqnqc/RZRASV5pJz3JF/3kY+m2euIGrKdbDRwL2Ytmw5Ucjt6R0cELcd+NuG5ejczQNFRikuxbIw4q/UhlsLC8wgRqo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=iu5VadXH; arc=none smtp.client-ip=209.85.210.171 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="iu5VadXH" Received: by mail-pf1-f171.google.com with SMTP id d2e1a72fcca58-724d8422dbaso3117555b3a.0 for ; Sun, 01 Dec 2024 20:39:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733114345; x=1733719145; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=E7q6MX6sgwytwqgWeV3TDLk22z43nA7MC5ajmWt+B9A=; b=iu5VadXHnwtYHvx1429Ocuv17vMbUy1HF2Okqg0knLm33o2+gaMe5duhcUBTycKBba CMetESRLVNO9A2zHJPzNiKnrz7CBoZjuiB5d9K+nTRus+yU07IQy6gAXfDGne1A8NV6P uck7O5jpMribIyOotL6gHs8MjD2JUsANGBMlSiyEHCUeTxjUX49QRKpIZNojHDeTbBXA UVg4D+BIe2++jOrxxuecfsm14xBpOTX2khuC1je3wpADYOPPxSYrFhZXg9t2Ymw3fRyj edhItqvQhVRNXV74w4ZbC2txxjliBj5bAtHBU+Th1rGzFQp39E9O+uAyVsS8X3gBSWth bw1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733114345; x=1733719145; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=E7q6MX6sgwytwqgWeV3TDLk22z43nA7MC5ajmWt+B9A=; b=bnQDGg8rMUTKoVFRlK5C05DSlG5lqusovj+bz0dexwShP5IJYgtI6jeDd4GAFhEksP nBCcj91qr3MFdT5JXEzB2u7BfQJjRHHWLqheSp5sqqxR2MgeYOmdSTbe3/z4R1qfshEP 1JE/xMgO2mdmVjTQebEk/8HP8D2EPtCJNTI+nlJSZuPPDzNNbjPRtk+EY0IXb/LRmrSd vRQqWIi/xlZtbjJfp7VW9UHkJ/rJa1ALFJnr059swISHuixVQ9K8yDgl9pzlypsv5rZj p7BYRxHwyZ66G9VxDNjldXEVxvh1vbdKcyQ6npBUy4omV87C1ASq5Jgyv+G7nZrEAusW 8oQA== X-Forwarded-Encrypted: i=1; AJvYcCUFUQ9zk5z8ahNMbRVzTyQUUrPSQ1dzVXJ9FZeEDf0f33lXuVecqbhqIIDsHiAfG8ziyCv/B6sZ2CX7lsE=@vger.kernel.org X-Gm-Message-State: AOJu0YznDb7NPzkM+8bsj6YdJAmu8JCkVvKqOhoVOYQ3D3kmz6EH/38d k/6kGYq6k5lTVMkb39nrJgNYhMrhDAlDETC78nBLsfTtQqbxcR90 X-Gm-Gg: ASbGncu+HdIocxj4bRXfyyNQ1kScDETVEuoUnTaNYTVQ5ncomtNPPp9gznxxTrJ0hhd /o1K0yHoG+d3WMmNz3wma1Zjj0HgRJ+ibNzJmdqfRpiOsH7kEu3/phE8zmbUIso3ub3/lxNqXXZ fl1ggiH7xGY0ClSkzE/JrtpoIqDnhz9HiMH6Cvy6A3RfJjt2J4/PkLzPcfMcuL/HnTkDuuOsYLz 7gwJ4OYg5P8rCx3axoDtDcFpHvPKI4chj3NUUFZPcW+UX0tROw5a2COT5pKDIs6c83PEI919dri Gc11ZA== X-Google-Smtp-Source: AGHT+IFik/6cOdExAlO+rVaQ7ZjP8+HNg9j4UQF82xdYImQky2PyMu3+arlwqk96ftOcEAhgjsaydQ== X-Received: by 2002:a05:6a00:3c8e:b0:71e:cc7:c507 with SMTP id d2e1a72fcca58-725301832afmr32012816b3a.23.1733114344454; Sun, 01 Dec 2024 20:39:04 -0800 (PST) Received: from localhost ([58.29.143.236]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7254176f1b9sm7436080b3a.42.2024.12.01.20.39.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Dec 2024 20:39:03 -0800 (PST) From: Changwoo Min X-Google-Original-From: Changwoo Min To: tj@kernel.org, void@manifault.com, mingo@redhat.com, peterz@infradead.org Cc: changwoo@igalia.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: [PATCH v2 1/5] sched_ext: Implement scx_rq_clock_update/stale() Date: Mon, 2 Dec 2024 13:38:45 +0900 Message-ID: <20241202043849.1465664-2-changwoo@igalia.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241202043849.1465664-1-changwoo@igalia.com> References: <20241202043849.1465664-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" scx_rq_clock_update() and scx_rq_clock_stale() manage the status of an rq clock. scx_rq_clock_update() keeps the rq clock in memory and its status valid. scx_rq_clock_stale() invalidates the current rq clock not to use the cached rq clock. Signed-off-by: Changwoo Min --- kernel/sched/sched.h | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 76f5f53a645f..09d5a64f78ce 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -754,6 +754,7 @@ enum scx_rq_flags { SCX_RQ_BAL_PENDING =3D 1 << 2, /* balance hasn't run yet */ SCX_RQ_BAL_KEEP =3D 1 << 3, /* balance decided to keep current */ SCX_RQ_BYPASSING =3D 1 << 4, + SCX_RQ_CLK_VALID =3D 1 << 5, /* RQ clock is fresh and valid */ =20 SCX_RQ_IN_WAKEUP =3D 1 << 16, SCX_RQ_IN_BALANCE =3D 1 << 17, @@ -766,8 +767,9 @@ struct scx_rq { unsigned long ops_qseq; u64 extra_enq_flags; /* see move_task_to_local_dsq() */ u32 nr_running; - u32 flags; u32 cpuperf_target; /* [0, SCHED_CAPACITY_SCALE] */ + u64 clock; /* cached per-rq clock -- see scx_bpf_clock_get_ns() */ + u32 flags; bool cpu_released; cpumask_var_t cpus_to_kick; cpumask_var_t cpus_to_kick_if_idle; @@ -1353,6 +1355,24 @@ static inline void rq_set_donor(struct rq *rq, struc= t task_struct *t) /* Do nothing */ } =20 +#ifdef CONFIG_SCHED_CLASS_EXT +static inline void scx_rq_clock_update(struct rq *rq, u64 clock) +{ + rq->scx.clock =3D clock; + rq->scx.flags |=3D SCX_RQ_CLK_VALID; +} + +static inline void scx_rq_clock_stale(struct rq *rq) +{ + rq->scx.flags &=3D ~SCX_RQ_CLK_VALID; +} + +#else +static inline void scx_rq_clock_update(struct rq *rq, u64 clock) {} +static inline void scx_rq_clock_stale(struct rq *rq) {} + +#endif /* CONFIG_SCHED_CLASS_EXT */ + #ifdef CONFIG_SCHED_CORE static inline struct cpumask *sched_group_span(struct sched_group *sg); =20 --=20 2.47.1 From nobody Sun Feb 8 13:39:49 2026 Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 843761F8EE2 for ; Mon, 2 Dec 2024 04:39:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733114351; cv=none; b=LWJoBQuWw3Xn5Qmsv2cyHvKrAXmF/NHP2kxagnxYRIjgUs6PQUb0qmHE5z+ZXcazESQIIXtNrH73lIss9tI5tk8m3dy9n7ymxCtvidb3mz+WW78fS8OR2IVuL2nZDc8wMyVgP7Kd23JS6yNMG/48cv1Y+z7jWo90HqwRY/fNLJY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733114351; c=relaxed/simple; bh=LqVTHh10a/zWSmpfUFGG88RGCMLgEOiqgRiJBI5bTrg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ST9fALcGWBkeFqwB3pwHagiaPdM66RVnqv2TMzSLVCvI42OhX5Ub6NlwAJkChWLbxuZeMi7mggxOwrsSs1yvkNT6/+GnzTyzpMKUPmCa/gg7BGTE6CTvCjwM/Q4LNWidTaVb5h5uGn4pifle1ghTDx/hpzZ8/hvotLtC8cBYD0E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=IIW/JS2X; arc=none smtp.client-ip=209.85.216.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="IIW/JS2X" Received: by mail-pj1-f47.google.com with SMTP id 98e67ed59e1d1-2e9ff7a778cso3428824a91.1 for ; Sun, 01 Dec 2024 20:39:09 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733114349; x=1733719149; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Vgq4gZ3czszS9fpJ4ZhxgFfICr08ym8vUl7BLZh8zf0=; b=IIW/JS2XbOK5/OwIJTgplsqbTJMX3uRAyAzOqoIoXznjJEUbHvFFbBjqqe/BC4jU/i aXLHWxgpK7DyulnhGDBie4yPyiYxIaaxvI0y3VMFOIsEA21dwk11IQwDXcrgaU5jvyYJ J+BJEOStfCDWEB5tB3ultsFPCMzV6JX96clg4P8omIEt1BVfeduZjWuGlhYtZwpOXydx 2oBwbE3udJEXPFYGGVUzHctFYkfEvhaVbUjefs3T9EUc4uk7oI2oG2BGdTMpCBtVbbJ8 NxPXNykxknHHToVD9QZhxZ53El+DbMq2TR03CpMHQXp0WGt5HmyvyKetLAfpSczzo6O6 khGg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733114349; x=1733719149; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Vgq4gZ3czszS9fpJ4ZhxgFfICr08ym8vUl7BLZh8zf0=; b=LOCSM4jkq+xPFzMKzjKu+Hfti+OPA6u8rpSd9YmNYHuYbKnnLujn7IRzcXDgKqJGOc o6dmp/KiruESO/5OMU13J+IU6damQIgudQkEPbXFrkDk7mRyyZdzT8JF/Q7R4qVR8aWD FG2v2b2mGvDbZp9tb9Iy67pqcf5NBwv67jNpBlqSbNvnlULwqgtIOHoggqDphxgry1hZ 1OTKlc7bKl9y5rWyuf3+xS38TQnTEAQ0bFNRuyH3U6ewmQmldquZyTDd+/1eMO10tcZl /ncCNLS0rSNP9PhqoKkM9i9fYfXbNqnq8OG7CTVKYHlZ8i8kmCih4APGXqH46QDz06mc ITUA== X-Forwarded-Encrypted: i=1; AJvYcCV2aceUUotZYSuknR96LKSRheRoGRr3BE8sPcLVU9SpTRLvM69XmH7oY+m92JtxppJ6B8X9VaPIIl31sFA=@vger.kernel.org X-Gm-Message-State: AOJu0YzIcfGRbUneqUktvEOd2oA625nPgHAJMDm7u+9UPBWZXNzBqHLZ QHgN6H6/pDPkp1i6D24C36s8FpLXIZK1wpDcqA4MS3cehUAinzds X-Gm-Gg: ASbGncuWd8kCRLs20bmQ9NeX0SgYHMwwjItQ1Ln46Tp8/GYLC24ZwwntW//caDUGXTX n00EkWnUwFcd2CYgRWmc5vFJwQQcgT+Z8v3XoxvbvcSkpz6S9tA+tEwl42yVw1y73LbMjJb/EIo qgYL3Gpkmz4LgV3telRN9Veyul1kfifxPW1uX2yVVaaMNMq5XFfs8IQ9H8z3dhm5NiG0k2WWRGc 8u8RJm6PItHJDsIXNskQqZnOuoOAqfXKPeO7jXQNK6pjh6sMZW0PDh+2cQHSf/m0UrOAFePHdyT u8draA== X-Google-Smtp-Source: AGHT+IGRQ6fVVNbSZFwSFFRJDNIY2lw6ABSXN9rG5RLxCnKGLO5WuqSwuygQarwP4BpqxcBxNe1QwQ== X-Received: by 2002:a17:90b:3647:b0:2ee:7c22:6e7c with SMTP id 98e67ed59e1d1-2ee7c227f45mr12715840a91.13.1733114348786; Sun, 01 Dec 2024 20:39:08 -0800 (PST) Received: from localhost ([58.29.143.236]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2ee2aff1f34sm7131146a91.5.2024.12.01.20.39.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Dec 2024 20:39:08 -0800 (PST) From: Changwoo Min X-Google-Original-From: Changwoo Min To: tj@kernel.org, void@manifault.com, mingo@redhat.com, peterz@infradead.org Cc: changwoo@igalia.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: [PATCH v2 2/5] sched_ext: Manage the validity of scx_rq_clock Date: Mon, 2 Dec 2024 13:38:46 +0900 Message-ID: <20241202043849.1465664-3-changwoo@igalia.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241202043849.1465664-1-changwoo@igalia.com> References: <20241202043849.1465664-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" An rq clock becomes valid when it is updated using update_rq_clock() and invalidated when the rq is unlocked using rq_unpin_lock(). Also, after long-running operations -- ops.running() and ops.update_idle() -- in a BPF scheduler, the sched_ext core invalidates the rq clock. Let's suppose the following timeline in the scheduler core: T1. rq_lock(rq) T2. update_rq_clock(rq) T3. a sched_ext BPF operation T4. rq_unlock(rq) T5. a sched_ext BPF operation T6. rq_lock(rq) T7. update_rq_clock(rq) For [T2, T4), we consider that rq clock is valid (SCX_RQ_CLK_VALID is set), so scx_bpf_clock_get_ns() calls during [T2, T4) (including T3) will return the rq clock updated at T2. For duration [T4, T7), when a BPF scheduler can still call scx_bpf_clock_get_ns() (T5), we consider the rq clock is invalid (SCX_RQ_CLK_VALID is unset at T4). So when calling scx_bpf_clock_get_ns() at T5, we will return a fresh clock value by calling sched_clock() internally. One example of calling scx_bpf_clock_get_ns(), when the rq clock is invalid (like T5), is in scx_central [1]. The scx_central scheduler uses a BPF timer for preemptive scheduling. In every msec, the timer callback checks if the currently running tasks exceed their timeslice. At the beginning of the BPF timer callback (central_timerfn in scx_central.bpf.c), scx_central gets the current time. When the BPF timer callback runs, the rq clock could be invalid, the same as T5. In this case, scx_bpf_clock_get_ns() returns a fresh clock value rather than returning the old one (T2). [1] https://github.com/sched-ext/scx/blob/main/scheds/c/scx_central.bpf.c Signed-off-by: Changwoo Min --- kernel/sched/core.c | 6 +++++- kernel/sched/ext.c | 3 +++ kernel/sched/sched.h | 2 +- 3 files changed, 9 insertions(+), 2 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 95e40895a519..ab8015c8cab4 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -789,6 +789,7 @@ static void update_rq_clock_task(struct rq *rq, s64 del= ta) void update_rq_clock(struct rq *rq) { s64 delta; + u64 clock; =20 lockdep_assert_rq_held(rq); =20 @@ -800,11 +801,14 @@ void update_rq_clock(struct rq *rq) SCHED_WARN_ON(rq->clock_update_flags & RQCF_UPDATED); rq->clock_update_flags |=3D RQCF_UPDATED; #endif + clock =3D sched_clock_cpu(cpu_of(rq)); + scx_rq_clock_update(rq, clock); =20 - delta =3D sched_clock_cpu(cpu_of(rq)) - rq->clock; + delta =3D clock - rq->clock; if (delta < 0) return; rq->clock +=3D delta; + update_rq_clock_task(rq, delta); } =20 diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index 7fff1d045477..ac279a657d50 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -2928,6 +2928,8 @@ static void set_next_task_scx(struct rq *rq, struct t= ask_struct *p, bool first) if (SCX_HAS_OP(running) && (p->scx.flags & SCX_TASK_QUEUED)) SCX_CALL_OP_TASK(SCX_KF_REST, running, p); =20 + scx_rq_clock_stale(rq); + clr_task_runnable(p, true); =20 /* @@ -3590,6 +3592,7 @@ void __scx_update_idle(struct rq *rq, bool idle) { int cpu =3D cpu_of(rq); =20 + scx_rq_clock_stale(rq); if (SCX_HAS_OP(update_idle) && !scx_rq_bypassing(rq)) { SCX_CALL_OP(SCX_KF_REST, update_idle, cpu_of(rq), idle); if (!static_branch_unlikely(&scx_builtin_idle_enabled)) diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 09d5a64f78ce..1c1345274b02 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1766,7 +1766,7 @@ static inline void rq_unpin_lock(struct rq *rq, struc= t rq_flags *rf) if (rq->clock_update_flags > RQCF_ACT_SKIP) rf->clock_update_flags =3D RQCF_UPDATED; #endif - + scx_rq_clock_stale(rq); lockdep_unpin_lock(__rq_lockp(rq), rf->cookie); } =20 --=20 2.47.1 From nobody Sun Feb 8 13:39:49 2026 Received: from mail-pl1-f178.google.com (mail-pl1-f178.google.com [209.85.214.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EF0A71F8AF8 for ; Mon, 2 Dec 2024 04:39:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.178 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733114356; cv=none; b=OjkYE/MRp0LPHb9ote/x2T/ioTtGS5Qqdz2W0yiKa83Ls9RTV0W6OwPpX+LjhT0SW6KAxGtscB3Asv+5zdYO0OeeastL6ezHP+Y4waSN/bRLykwmnopEBl0rXK9GnHke6XpcpZYnyt7FeXRZY8Et5n396PUYswlqmM8Z6RC6Vkg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733114356; c=relaxed/simple; bh=XP1bJ0NOVWvhnqquRkwIz/i6kZEfAZJaGcyYAhOVshM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=iFg6T4ng0aTyjRnjSfaVqpXY/asvruZKaB2M6evy5aXDzfXAk3emVCyuW1+jFszZFvTRjgkCdIwKdTQU2vLezHPwCMjJeMAonVc6BBGtA13VSU++0vuN/agQsd/D61ftUrDwZVc6FTZaeyp8sy/2s2IOZTCJZ4RlExzGD/wsQko= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=hdkUCGRM; arc=none smtp.client-ip=209.85.214.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hdkUCGRM" Received: by mail-pl1-f178.google.com with SMTP id d9443c01a7336-21278f3d547so26321895ad.1 for ; Sun, 01 Dec 2024 20:39:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733114354; x=1733719154; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=vYP7n+ha6LjN/GOJcD6nH/LRPcnS5LKp91acoSKaYwA=; b=hdkUCGRMaSa4fXHnd2QG0ty62d5B5aMI/N7lUXrkC1ukFqmPT/60rOR9QAGfsHzeLo DUEclfSdRjl1Lghn3FLTMuv2H71hkh43g2DG08SHIPmepKDDO2u2srjOc0CKaUb8xBiE cNaGu8TLuG/OmhqU4J1omHEEeeB3JQytKyiITgmKsgVNUGhwn2OZZ2G4nOfeT8gXv8G2 9KuW22q/oarAcyCRWx/mP8oS/JW9bS2WDgpScjExMJqD0F5j0htDDznVyiCjs4OZIoxr m/syWsjImcoIPCUcBi3cVkNXe1XI9pyH+ibniw7oqXZQTLyB53qGo15EEnpSLFnpmCbi Ffxw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733114354; x=1733719154; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vYP7n+ha6LjN/GOJcD6nH/LRPcnS5LKp91acoSKaYwA=; b=Zjq3Z6aoW/38IRn67cm5jQ2ptI1gVAux6gfFsiVapu+DHaKBsqzDFctcb8jXNfxt1v cAL85Mn68Eqs30sWnRoMrrQkYSQwurf3nhOrHbcFEo6/o28SDCsL6Cp1kJgO7Xl+YBts dAtlI+x/cHbyZUY0J1gUbZPndSgXR5eBnfLR09FftKAVvSbmZw4mPkuxrzGIto3cukQz FA7erasn+9PE3oOZ7KTF0tMPW45jMTDYiNf2Y2Cpp7z8zq0HIiZCHUs2RApBmOd2cN3Q ZlZ31CGrFfS8Qg0lTCgq3/vLK6/7KQUKKyCHh99D99rpExkfdmGaUoXCOtwZfMXVrrSu im2A== X-Forwarded-Encrypted: i=1; AJvYcCXrSTd6CK709v3Ohj+rbigcMIc3fknHPZliyY/1o7fIFZIuq7EvbBo1w8UOBAnN8XDhSIbQklyxVubIrNw=@vger.kernel.org X-Gm-Message-State: AOJu0YxHKdkSys6wlJt2N3IbAxXA0y3I9RCp2Y5TltYgNV3sBXPFQ26y Spl2XOLceSU6y52hxW0szFJhWZ7PKo3LQwUeeaBCJu+7q1kTpiDqceKChw== X-Gm-Gg: ASbGncul+nki5YV0OQu0R37klyTa9uVZU++lPI2f3m7FI03wSQ1M7GNTKpe59zJtmn4 vPLYiU4TxQatZCG6QUIEv6LH7/0LpDIgYGZPWCKIHGFkxg5qE1AdvNMI6X7EUFhpuM+kQQZ1KRL 0WT4O4OW1foM7sPyqEWQRvyQoA5HrNnuFzanfFfXjBfc/YfRrngCpX+Y8kJl28Ikkmnppb8msBa G3kuQnHxRKZWiVvMPfE8pk+LWWo1FQKI6KYGdScN+ZDiP4P/xo095gDCvPoDZVDwt4kwFlR0iRb rmusyw== X-Google-Smtp-Source: AGHT+IEFxIWH9QkSFx4XNnPAXcBlCInFklU5TuiP5Qu3WROvroPGS3bMMGVM142hlGMVJZTxZR+0qQ== X-Received: by 2002:a17:902:c951:b0:215:75ca:6a0 with SMTP id d9443c01a7336-21575ca0788mr91397875ad.29.1733114354199; Sun, 01 Dec 2024 20:39:14 -0800 (PST) Received: from localhost ([58.29.143.236]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-215218f25easm66759555ad.27.2024.12.01.20.39.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Dec 2024 20:39:13 -0800 (PST) From: Changwoo Min X-Google-Original-From: Changwoo Min To: tj@kernel.org, void@manifault.com, mingo@redhat.com, peterz@infradead.org Cc: changwoo@igalia.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: [PATCH v2 3/5] sched_ext: Implement scx_bpf_clock_get_ns() Date: Mon, 2 Dec 2024 13:38:47 +0900 Message-ID: <20241202043849.1465664-4-changwoo@igalia.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241202043849.1465664-1-changwoo@igalia.com> References: <20241202043849.1465664-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Returns a high-performance monotonically non-decreasing clock for the current CPU. The clock returned is in nanoseconds. It provides the following properties: 1) High performance: Many BPF schedulers call bpf_ktime_get_ns() frequently to account for execution time and track tasks' runtime properties. Unfortunately, in some hardware platforms, bpf_ktime_get_ns() -- which eventually reads a hardware timestamp counter -- is neither performant nor scalable. scx_bpf_clock_get_ns() aims to provide a high-performance clock by using the rq clock in the scheduler core whenever possible. 2) High enough resolution for the BPF scheduler use cases: In most BPF scheduler use cases, the required clock resolution is lower than the most accurate hardware clock (e.g., rdtsc in x86). scx_bpf_clock_get_ns() basically uses the rq clock in the scheduler core whenever it is valid. It considers that the rq clock is valid from the time the rq clock is updated (update_rq_clock) until the rq is unlocked (rq_unpin_lock). In addition, it invalidates the rq clock after long operations -- ops.running() and ops.update_idle() -- in a BPF scheduler. 3) Monotonically non-decreasing clock for the same CPU: scx_bpf_clock_get_ns() guarantees the clock never goes backward when comparing them in the same CPU. On the other hand, when comparing clocks in different CPUs, there is no such guarantee -- the clock can go backward. It provides a monotonically *non-decreasing* clock so that it would provide the same clock values in two different scx_bpf_clock_get_ns() calls in the same CPU during the same period of when the rq clock is valid. Signed-off-by: Changwoo Min --- kernel/sched/ext.c | 71 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 71 insertions(+) diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index ac279a657d50..9e2656e21593 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -7546,6 +7546,76 @@ __bpf_kfunc struct cgroup *scx_bpf_task_cgroup(struc= t task_struct *p) } #endif =20 +/** + * scx_bpf_clock_get_ns - Returns a high-performance monotonically + * non-decreasing clock for the current CPU. The clock returned is in + * nanoseconds. + * + * It provides the following properties: + * + * 1) High performance: Many BPF schedulers call bpf_ktime_get_ns() freque= ntly + * to account for execution time and track tasks' runtime properties. + * Unfortunately, in some hardware platforms, bpf_ktime_get_ns() -- which + * eventually reads a hardware timestamp counter -- is neither performant= nor + * scalable. scx_bpf_clock_get_ns() aims to provide a high-performance cl= ock + * by using the rq clock in the scheduler core whenever possible. + * + * 2) High enough resolution for the BPF scheduler use cases: In most BPF + * scheduler use cases, the required clock resolution is lower than the m= ost + * accurate hardware clock (e.g., rdtsc in x86). scx_bpf_clock_get_ns() + * basically uses the rq clock in the scheduler core whenever it is valid. + * It considers that the rq clock is valid from the time the rq clock is + * updated (update_rq_clock) until the rq is unlocked (rq_unpin_lock). + * In addition, it invalidates the rq clock after long operations -- + * ops.running() and ops.update_idle(). + * + * 3) Monotonically non-decreasing clock for the same CPU: + * scx_bpf_clock_get_ns() guarantees the clock never goes backward when + * comparing them in the same CPU. On the other hand, when comparing cloc= ks + * in different CPUs, there is no such guarantee -- the clock can go back= ward. + * It provides a monotonically *non-decreasing* clock so that it would pr= ovide + * the same clock values in two different scx_bpf_clock_get_ns() calls in= the + * same CPU during the same period of when the rq clock is valid. + */ +__bpf_kfunc u64 scx_bpf_clock_get_ns(void) +{ + static DEFINE_PER_CPU(u64, prev_clk); + struct rq *rq =3D this_rq(); + u64 pr_clk, cr_clk; + + preempt_disable(); + pr_clk =3D __this_cpu_read(prev_clk); + + /* + * If the rq clock is invalid, start a new rq clock period + * with a fresh sched_clock(). + */ + if (!(rq->scx.flags & SCX_RQ_CLK_VALID)) { + cr_clk =3D sched_clock(); + scx_rq_clock_update(rq, cr_clk); + } + /* + * If the rq clock is valid, use the cached rq clock + * whenever the clock does not go backward. + */ + else { + cr_clk =3D rq->scx.clock; + /* + * If the clock goes backward, start a new rq clock period + * with a fresh sched_clock(). + */ + if (pr_clk > cr_clk) { + cr_clk =3D sched_clock(); + scx_rq_clock_update(rq, cr_clk); + } + } + + __this_cpu_write(prev_clk, cr_clk); + preempt_enable(); + + return cr_clk; +} + __bpf_kfunc_end_defs(); =20 BTF_KFUNCS_START(scx_kfunc_ids_any) @@ -7577,6 +7647,7 @@ BTF_ID_FLAGS(func, scx_bpf_cpu_rq) #ifdef CONFIG_CGROUP_SCHED BTF_ID_FLAGS(func, scx_bpf_task_cgroup, KF_RCU | KF_ACQUIRE) #endif +BTF_ID_FLAGS(func, scx_bpf_clock_get_ns) BTF_KFUNCS_END(scx_kfunc_ids_any) =20 static const struct btf_kfunc_id_set scx_kfunc_set_any =3D { --=20 2.47.1 From nobody Sun Feb 8 13:39:49 2026 Received: from mail-pf1-f181.google.com (mail-pf1-f181.google.com [209.85.210.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B301D1F8F04 for ; Mon, 2 Dec 2024 04:39:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733114360; cv=none; b=NOKamCCo+qkvLSUw449nYRZiWLPrbPFmZosUz1Z0njp2a4edBcKQyRS7l9KqIcQuhScnxltzf8mxEffsYm+jdt9J6t31oTZFO70wNWrpbD9YNCoobLKNd4v8Mhs4lbocjLvoPqalYxdETboUs1ZuQxXIdmsZiHlsZxpeIJTXehI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733114360; c=relaxed/simple; bh=CiA7urzExORBNfNnKVkDnfqRTn3t7XF1Gk8zmHVsPIM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DI8UvXsSCm/acrSFGULTxatKNkeLiCzse8XJG5A/UvKghwbkeFxkLO8f69aEYzGixhsfpUZyqO5ypYzHyBX62YBDcm1UAvixJNQwMDCqpVYc40nGdk0ex2QSI93r1yRUSH2bdJGaRU8V9ZmxWgIcC6MQPE26kmcWs3822+hmK4s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=GQ8rRxLR; arc=none smtp.client-ip=209.85.210.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="GQ8rRxLR" Received: by mail-pf1-f181.google.com with SMTP id d2e1a72fcca58-724e5fb3f9dso3127506b3a.3 for ; Sun, 01 Dec 2024 20:39:18 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733114358; x=1733719158; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=t5z6e0Wwzok4/c8JJ14VvrKpt6wQ9E07/XNoPDXalY0=; b=GQ8rRxLRk8qBZPpjkVsmTkQmf01Gh/C1PrblbalQwXX1kMR0Adlu2lhnWU/pRRZ4Md 7Bprfu4NsPWprR3K0bABmYazFT3oRcVxs1Cy/sZ99MnbB03oe+XjqpP3aZAjg5jLSRZW 5lL2AYtgTOB2EjxLdVyHXLfkL17LbLaX6EudpMc06dVL4xGp3JnIo4a2OfnERg4emVse v+LgZGF3S5kKzVNEs/etc42y7ltTFwJ4XrdP0TmJxi/NRxsiorhKfKfbpOKYlp+SyrOc VOq5UGYVkhcPf93XVYU6wxpz4wgR/SkQc0uRGq05kiWeX7vjw/JPVq5kvlTpAPVUXnOX /KRw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733114358; x=1733719158; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=t5z6e0Wwzok4/c8JJ14VvrKpt6wQ9E07/XNoPDXalY0=; b=AtW9oXU7mpdz9MXElGt1tHHQkhyCzqf5w6sVv6GajkxA9OHstWdQjs4xIiPjZmPWlp 3zap+ySyla6Q0/J7ePwxjc/Lo9PDl0R0kPEVrtKV35Duk+Sjavvir3QOgp/vLJOxBKEV TLyBJeFs1/ZKOTP4tbntioR47x2QngxJ4d802yLETyYDQm/YFkRCMGt57BHVlAjYVvz+ IovW/eyrM98upqxH3Mk3uyjwJ9GdB9f6Kpg7uCirD+pRSshiffPyXKBTosq3pDSZVrAQ MfHI8yhi3DvKyP9ZMWp8JLTxolVjCoIVskR1JKp73gtM8bw1zC5u5zQ6kyvMp1+/018j Uz3w== X-Forwarded-Encrypted: i=1; AJvYcCUmnZjt9ILK0uNPgS+niobOBeHrQMbT9ij3DNXwXVCLbgWMhtMkzl8ULW5/aBjo4rZKrXP4MYfga2NXpo8=@vger.kernel.org X-Gm-Message-State: AOJu0YxIR0mNBD3nkrdEBarRQIx1uQU0LAz58QH8vgugR/dFqvAwHTeZ tb3VzGQxue0AtHfZouGLqg5DfUfUdfSOfSQGRnzKCWBq8ZO2emDn X-Gm-Gg: ASbGncsEzK93XF10olOB8Wc05mo8V5Cy/mGcpf+xipz24i+0SeJoUi22mJKFyE+xriy H4r0DnkJfbjBvK0c2akOk5ZQKLwD3Ed9NCLqXkIA9Y9RC0loNsi9HGn36DO0JY1D3RW8xyZn9uL 2jXmHy6hl7YnSvpdbOogw7P0iEsinQHmTH89Fjk4GDGHeewk+n45/g3YV8ECEKh0EmdhF4zYE0K BzYYtdcXrqEGlgaHhUy5kZr+l5YC5D2fHZCexnl65lXNjD0QWj7GSZc7Mzdeu3xA3S/h+HPWL4o SE2jyA== X-Google-Smtp-Source: AGHT+IEaeNOvfUjlbtqHoKY56jCg6XhQk9IXFLBbMoRE/nbhRVXjUd0aG05P6S7NXiyEmpShabj2yQ== X-Received: by 2002:a05:6a00:2387:b0:71e:b1dc:f255 with SMTP id d2e1a72fcca58-7253004c844mr27338111b3a.9.1733114357948; Sun, 01 Dec 2024 20:39:17 -0800 (PST) Received: from localhost ([58.29.143.236]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7254176f77dsm7443163b3a.53.2024.12.01.20.39.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Dec 2024 20:39:17 -0800 (PST) From: Changwoo Min X-Google-Original-From: Changwoo Min To: tj@kernel.org, void@manifault.com, mingo@redhat.com, peterz@infradead.org Cc: changwoo@igalia.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: [PATCH v2 4/5] sched_ext: Add scx_bpf_clock_get_ns() for BPF scheduler Date: Mon, 2 Dec 2024 13:38:48 +0900 Message-ID: <20241202043849.1465664-5-changwoo@igalia.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241202043849.1465664-1-changwoo@igalia.com> References: <20241202043849.1465664-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" scx_bpf_clock_get_ns() is added to the header files so the BPF scheduler can use it. Signed-off-by: Changwoo Min --- tools/sched_ext/include/scx/common.bpf.h | 1 + tools/sched_ext/include/scx/compat.bpf.h | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/tools/sched_ext/include/scx/common.bpf.h b/tools/sched_ext/inc= lude/scx/common.bpf.h index 2f36b7b6418d..230c7f2e8ad6 100644 --- a/tools/sched_ext/include/scx/common.bpf.h +++ b/tools/sched_ext/include/scx/common.bpf.h @@ -72,6 +72,7 @@ bool scx_bpf_task_running(const struct task_struct *p) __= ksym; s32 scx_bpf_task_cpu(const struct task_struct *p) __ksym; struct rq *scx_bpf_cpu_rq(s32 cpu) __ksym; struct cgroup *scx_bpf_task_cgroup(struct task_struct *p) __ksym __weak; +u64 scx_bpf_clock_get_ns(void) __ksym __weak; =20 /* * Use the following as @it__iter when calling scx_bpf_dsq_move[_vtime]() = from diff --git a/tools/sched_ext/include/scx/compat.bpf.h b/tools/sched_ext/inc= lude/scx/compat.bpf.h index d56520100a26..d295c59e3f05 100644 --- a/tools/sched_ext/include/scx/compat.bpf.h +++ b/tools/sched_ext/include/scx/compat.bpf.h @@ -125,6 +125,11 @@ bool scx_bpf_dispatch_vtime_from_dsq___compat(struct b= pf_iter_scx_dsq *it__iter, false; \ }) =20 +#define scx_bpf_clock_get_ns() \ + (bpf_ksym_exists(scx_bpf_clock_get_ns) ? \ + scx_bpf_clock_get_ns() : \ + bpf_ktime_get_ns()) + /* * Define sched_ext_ops. This may be expanded to define multiple variants = for * backward compatibility. See compat.h::SCX_OPS_LOAD/ATTACH(). --=20 2.47.1 From nobody Sun Feb 8 13:39:49 2026 Received: from mail-pf1-f172.google.com (mail-pf1-f172.google.com [209.85.210.172]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8990C1F8EE5 for ; Mon, 2 Dec 2024 04:39:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.172 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733114363; cv=none; b=M80R5AH4nSdQKCrGgTx6rjLzgdjC3eWTGu3bOokld7DzhmkaiVRBJ0hxw0+iVw7P8JJka4GFB1o3bdd+CHaCoFw8hEfdTTZf8KlZVx8xr3VWNB+6MHPncwjTU/xTHOWc0/71d8BIDDWNlqCd1XTtLLRgqkED0DYfUC1NmTgBBC0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733114363; c=relaxed/simple; bh=Kzd/PUC5ctt74z6B6vUj4jyvkEy3XwQUHulW25s5gBs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TVJXgsFy2dATPw836QGZv+QhbMolgTSTiijquR1snLC75aBJrducumyduRiCjZofWqZpZc7Ps0Zq0cdaFolXNFTFfjBQZ3d9nByzLKFVgSQZMBqq03eutgDWj6JvcetHFmm8QIjVFjrB/QcxPyjlEvBRECtq79zX8aAHOWjD4TM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=hbskrb7m; arc=none smtp.client-ip=209.85.210.172 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="hbskrb7m" Received: by mail-pf1-f172.google.com with SMTP id d2e1a72fcca58-724f383c5bfso2749074b3a.1 for ; Sun, 01 Dec 2024 20:39:22 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733114362; x=1733719162; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Hrorv/GUnF7EJucVAVWvFSb+WFE0AwzJOhO/e5bho0Y=; b=hbskrb7mNhKH/JVgxz9DEG07IOkcRVGgmWiIx7WDrUNhMwG4vDIs884mEWNuMVcZC4 eoLmWKbt2DQJYpBGRsE8LJIgPayiZxMmA10bCFr96iUFKGr3yeAfN5Uil4OMKACsFxOC RlRfQuv5tOAxdJK1fgrGSm6ZMZMjJoKyGg60ll/XFpToMuJ7YW7kdOWX/Us41tu1R/WB xk4TFCVfauEuNNV5133oj0l1Na0aat+k8le7yjaXmRv7KQjrxns9W6evkeMTonPPUs3z DFmYoe+y7wVkxL9f6Y2CjzDxBPeqzORiVzR2CXFTm2FGgmznMC8MRpt9XLhIRDMzDDL0 P00g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733114362; x=1733719162; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Hrorv/GUnF7EJucVAVWvFSb+WFE0AwzJOhO/e5bho0Y=; b=ogP4NqjYtAnzY3f8WN6Yk1xPnCL7l74fXDNlGNTjSMiq620d0zqz2yS2dQxZtypSNF 4yfcHCJcaBdYwjt71oUWYq4lsMLXqhsXRsDMvv/lnQeZaRNdMtgfmrlijuwRP/yoo4y2 +QV+zVLfrojiuG/QqvhKoAP0ddCXoMsyYe9108+HtoQHEGCVAhsNf6+oSlkB8nsoJdft M2ThOdErXTu7kU2jxoczuXMmQBOi57iL4l9VMq19qJpGRS3zg9tLEEdBJyWON6eoYeLr qsjPkmwzWa8/qEx8rEQPiM96p9slr6X6LK5cm66Rq1LgwnjNyMY/ocPwBIYK8x+2XbEW b5cw== X-Forwarded-Encrypted: i=1; AJvYcCVfENzKGwIdi+0FawvpAdg8PGZiUhGIJIIOXTLXsLk82nMjI4gy4CJ+3fH++1z1jRgVLB6BzAHmMyoZTWk=@vger.kernel.org X-Gm-Message-State: AOJu0YzMpgYYyPsgmBgw676W66VZLdKPJy6YZilhNbvIPqaMMv3BXtah 9q8bgFOATEl8n/4eEgYN3OMGwRucR/tucol27O2JjFBfTuLBJqgX X-Gm-Gg: ASbGncsepcSsPH9Ji1AJ9TqXs2+YR1V/0al3ch8LGasK6B2phpMeuWNTgKGxdDhZgKt 2pqTJq/q4UdNwbnWXmV9RdMRwocdMw4lBbD4DX2thLL0IwZyClQr0NyU84B4X1xAm2WRDX+MLhY q30s5qSrSTAYSRiQGtre50iXsmdQu0qazw0HIc2Mqa7qrnzj4aCI4QzexsTm+KYA83A/JKgoF7X iGAaXxTnHe0otVVuuWRX79wOa1lNhREc9TW27MsgrIvr4Pc1MfmkXMhVKix88bX4uLhQHjdwP5u I9ASFg== X-Google-Smtp-Source: AGHT+IGNfxYs7dGFB/p293PItC99MrejYOrgMw0YTiPhRduHnTMChg1lC2uh6n+pQfLghwcAsvZOaw== X-Received: by 2002:a05:6a00:2351:b0:724:67dd:2615 with SMTP id d2e1a72fcca58-7253015446amr29616926b3a.22.1733114361847; Sun, 01 Dec 2024 20:39:21 -0800 (PST) Received: from localhost ([58.29.143.236]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7254176150dsm7687509b3a.23.2024.12.01.20.39.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Dec 2024 20:39:21 -0800 (PST) From: Changwoo Min X-Google-Original-From: Changwoo Min To: tj@kernel.org, void@manifault.com, mingo@redhat.com, peterz@infradead.org Cc: changwoo@igalia.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: [PATCH v2 5/5] sched_ext: Replace bpf_ktime_get_ns() to scx_bpf_clock_get_ns() Date: Mon, 2 Dec 2024 13:38:49 +0900 Message-ID: <20241202043849.1465664-6-changwoo@igalia.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241202043849.1465664-1-changwoo@igalia.com> References: <20241202043849.1465664-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In the BPF schedulers that use bpf_ktime_get_ns() -- scx_central and scx_flatcg, replace bpf_ktime_get_ns() calls to scx_bpf_clock_get_ns(). Signed-off-by: Changwoo Min --- tools/sched_ext/scx_central.bpf.c | 4 ++-- tools/sched_ext/scx_flatcg.bpf.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tools/sched_ext/scx_central.bpf.c b/tools/sched_ext/scx_centra= l.bpf.c index e6fad6211f6c..cb7428b6a198 100644 --- a/tools/sched_ext/scx_central.bpf.c +++ b/tools/sched_ext/scx_central.bpf.c @@ -245,7 +245,7 @@ void BPF_STRUCT_OPS(central_running, struct task_struct= *p) s32 cpu =3D scx_bpf_task_cpu(p); u64 *started_at =3D ARRAY_ELEM_PTR(cpu_started_at, cpu, nr_cpu_ids); if (started_at) - *started_at =3D bpf_ktime_get_ns() ?: 1; /* 0 indicates idle */ + *started_at =3D scx_bpf_clock_get_ns() ?: 1; /* 0 indicates idle */ } =20 void BPF_STRUCT_OPS(central_stopping, struct task_struct *p, bool runnable) @@ -258,7 +258,7 @@ void BPF_STRUCT_OPS(central_stopping, struct task_struc= t *p, bool runnable) =20 static int central_timerfn(void *map, int *key, struct bpf_timer *timer) { - u64 now =3D bpf_ktime_get_ns(); + u64 now =3D scx_bpf_clock_get_ns(); u64 nr_to_kick =3D nr_queued; s32 i, curr_cpu; =20 diff --git a/tools/sched_ext/scx_flatcg.bpf.c b/tools/sched_ext/scx_flatcg.= bpf.c index 4e3afcd260bf..15351bf2f053 100644 --- a/tools/sched_ext/scx_flatcg.bpf.c +++ b/tools/sched_ext/scx_flatcg.bpf.c @@ -734,7 +734,7 @@ void BPF_STRUCT_OPS(fcg_dispatch, s32 cpu, struct task_= struct *prev) struct fcg_cpu_ctx *cpuc; struct fcg_cgrp_ctx *cgc; struct cgroup *cgrp; - u64 now =3D bpf_ktime_get_ns(); + u64 now =3D scx_bpf_clock_get_ns(); bool picked_next =3D false; =20 cpuc =3D find_cpu_ctx(); --=20 2.47.1