From nobody Fri Nov 22 16:17:56 2024 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F22A414F70 for ; Sat, 16 Nov 2024 16:01:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731772921; cv=none; b=bdhfixUs4sPQV5rCZnphHuRIqaJZywLHuJZtJZEfrc8gZb7heOnFYzrMHRuBvnZ8TQ/TKMF0Vv0MnytRipBQLKHFIsVoJa9y35lG5rquunLenuEanmMC1hiqbOpVXzeTPIhP0iuhiGoKBNz7YWgvx1onIvyG7DE3QZISaSQRek0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731772921; c=relaxed/simple; bh=0vd+HZzJ5k+fFsPT6iNDQixWz/Op/NOEpANZ9hcb6EI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=py/BsDD0Pk+oh8rkJYNGCJeigcjCaUkJzPGKEZOGbyqsi7iVKnWc42GndaKP7oTK7h5fv0pcRblGQy9FLXWSor3Yeusgn06koLHd0yDvi16uBut/kuqZHLW93N1ZU+NIPtO2QKW8hFJ0gy92fOhJqhlm+b+jsvCJJM9/xh1lWTY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=T2AmXO43; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="T2AmXO43" Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-20cb47387ceso30314095ad.1 for ; Sat, 16 Nov 2024 08:01:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731772919; x=1732377719; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IuuWQGzQ1DfUt2ak4ZbC84lMZxrJ6yXnCz0y/5+ZzOA=; b=T2AmXO43hnCiwPMtv379vL5k7OjkIaDiasb8V5NE/o5vTlc4BaozgEoWWkPuT1NZsB 0gegVtyjeT6GcDj56DOMGc4N0kjkFN07MeBNbBjx+7YAUoO964J8JoaOSEOAcdylDzg0 +wua93HwuK4qSg+v00A2ODMdu0d6panpF7o87TKJxkuxbJsSWMzQiH/3lQ74hcMnZMhP J3fiCeAmHtBS2ClY5nRwuuxtoeItCcb9puS2eQa2IQHn6Me/YKX8CUpfjqv8RwExCB30 dwy9Xvq4fcTRLi5THEPMoEmUwFUKwDBr4LupoVoTDZXtnXyh44Jm2KoMG8dVV3ieIYxJ kS5g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731772919; x=1732377719; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IuuWQGzQ1DfUt2ak4ZbC84lMZxrJ6yXnCz0y/5+ZzOA=; b=RlPprmro0Fnc35ck3Nc+upoRCTUtQttk09WEA0jco2AY1r0ApdiHTa+ajwbQEZya9M jGWQbhS2Wdk5U8TXmcaQypYg/TntQ8xfZF2hU4mPA2SFBWEhD5CTBNsHzxrAF1jjlZMA ZemS5nUoREk+iAkJcqsjFtqPFuzEowJppbVHt/QvTvpPjwaT45da5o9j/TIIbG3TlYxm I0lPVYlrwoPfs1Rn+qTOozxWXiSfvqEcq9TTkgRCOZ4+zkElufz7LLiTQkQPi6zbXuxN beWDTCPqHpyFsrVCPOuhyn/JTn0e4djoQ9V11VYw/xF0s1BSsmJZ2RVMeppf0nYJZ4Ps Rudg== X-Forwarded-Encrypted: i=1; AJvYcCXwL8XnCYx11LlqotxqAxUnTxC6DH6hME0EdDbVowzLiaXv4KsPlQCcoVPCeCB4lLlJx798StyD1GeAm3c=@vger.kernel.org X-Gm-Message-State: AOJu0Yx3Ct0uWBBKKVHZUQZgIkOaiQa8orA6shXubQx8PXf3PQA+Na+m tKVue9SijJPJOZ44dbyU7ALw7ETvE9yR63G7WGlgorIWFLsUTw9Z X-Google-Smtp-Source: AGHT+IF4zrtBHANffc8d0oZ84RIc2au55hvsJo3/mPDF1rpu4iZzgSVCVnsFyVvIm7hJ6KI2EtO/RA== X-Received: by 2002:a17:903:2452:b0:20b:8776:4902 with SMTP id d9443c01a7336-211d0ebf325mr97950975ad.38.1731772919100; Sat, 16 Nov 2024 08:01:59 -0800 (PST) Received: from localhost ([58.29.143.236]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-211d0f34809sm28267705ad.124.2024.11.16.08.01.58 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 16 Nov 2024 08:01:58 -0800 (PST) From: Changwoo Min X-Google-Original-From: Changwoo Min To: tj@kernel.org, void@manifault.com Cc: mingo@redhat.com, peterz@infradead.org, changwoo@igalia.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: [PATCH 1/5] sched_ext: Implement scx_rq_clock_update/stale() Date: Sun, 17 Nov 2024 01:01:22 +0900 Message-ID: <20241116160126.29454-2-changwoo@igalia.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241116160126.29454-1-changwoo@igalia.com> References: <20241116160126.29454-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" scx_rq_clock_update() and scx_rq_clock_stale() manage the status of an rq clock. scx_rq_clock_update() keeps the rq clock in memory and its status valid. scx_rq_clock_stale() invalidates the current rq clock not to use the cached rq clock. Signed-off-by: Changwoo Min --- kernel/sched/sched.h | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 4d79804631e4..61efff790e24 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -754,6 +754,7 @@ enum scx_rq_flags { SCX_RQ_BAL_PENDING =3D 1 << 2, /* balance hasn't run yet */ SCX_RQ_BAL_KEEP =3D 1 << 3, /* balance decided to keep current */ SCX_RQ_BYPASSING =3D 1 << 4, + SCX_RQ_CLK_UPDATED =3D 1 << 5, /* RQ clock is updated by the core */ =20 SCX_RQ_IN_WAKEUP =3D 1 << 16, SCX_RQ_IN_BALANCE =3D 1 << 17, @@ -765,6 +766,7 @@ struct scx_rq { struct list_head ddsp_deferred_locals; /* deferred ddsps from enq */ unsigned long ops_qseq; u64 extra_enq_flags; /* see move_task_to_local_dsq() */ + u64 clock; /* cached per-rq clock -- see scx_bpf_clock_get_ns() */ u32 nr_running; u32 flags; u32 cpuperf_target; /* [0, SCHED_CAPACITY_SCALE] */ @@ -1345,6 +1347,24 @@ DECLARE_PER_CPU_SHARED_ALIGNED(struct rq, runqueues); #define cpu_curr(cpu) (cpu_rq(cpu)->curr) #define raw_rq() raw_cpu_ptr(&runqueues) =20 +#ifdef CONFIG_SCHED_CLASS_EXT +static inline void scx_rq_clock_update(struct rq *rq, u64 clock) +{ + rq->scx.clock =3D clock; + rq->scx.flags |=3D SCX_RQ_CLK_UPDATED; +} + +static inline void scx_rq_clock_stale(struct rq *rq) +{ + rq->scx.flags &=3D ~SCX_RQ_CLK_UPDATED; +} + +#else +static inline void scx_rq_clock_update(struct rq *rq, u64 clock) {} +static inline void scx_rq_clock_stale(struct rq *rq) {} + +#endif /* CONFIG_SCHED_CLASS_EXT */ + #ifdef CONFIG_SCHED_CORE static inline struct cpumask *sched_group_span(struct sched_group *sg); =20 --=20 2.47.0 From nobody Fri Nov 22 16:17:56 2024 Received: from mail-pl1-f174.google.com (mail-pl1-f174.google.com [209.85.214.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AC63F18DF6D for ; Sat, 16 Nov 2024 16:02:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731772925; cv=none; b=gqP/NGokaLw8VcgkJCBvUAtEm3twBixXbAxXvK17fvueVt4ObgM0Fo2JR4sFtnMh6LcSwX6bZwBH9ctqS9bVy8nAgU+pn8o1lwReNQ1byEiMF3qtJYdolsVPu/ZSHrLbk+mto23ir8azufW5Ae2t6nYqx8NJD93WgDkn79Sf3/c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731772925; c=relaxed/simple; bh=+MG52QQoyvMX4PhF1K/8jvNUimDPdhZ2sTZ6kij+eCo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jEsnTqTe98L5AOBnaAeA5yKM6yBy7tQP/tGNQ5ZlG86Erco8nkwBKCpHRftGBkJPVpD4Q6kZRp73ZqYx76sV/MJpSlcCXudosZmp+yFQ8sYKjGyxEiG6iCdErclcDLhcM30bKYbDbboUuW2QxCy4HvPfg8sxqkaPlnJCIQbCU9o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=L5PaUiGU; arc=none smtp.client-ip=209.85.214.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="L5PaUiGU" Received: by mail-pl1-f174.google.com with SMTP id d9443c01a7336-20c714cd9c8so4233745ad.0 for ; Sat, 16 Nov 2024 08:02:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731772923; x=1732377723; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=38PQp1RQQ2NMP1hcM2NKIXHciL2ilqE4M5FPbIzuslY=; b=L5PaUiGUbXBB2SdVNkV2IWSGLcQimmx6ywB+hK7XfXs5zIEwK5MYIH77fULjzjfYld LFYZQYAW5oVzWp4ttbUHHAr3sPVPdo2/Q/Q26hGN9m/1LfdlK0X5l1mjjnXlPHLAP14f dEYtHlgE3FVxBU9qXWnCg7bkCbxsIqayP1xCsQt/mwziLX6Z+8/T6erHP/dR6Es+ChoW tPdTD1LN6FtAW1ZYzHX+rbxSEPGdL+iHXRZZQaIsPVCetFLUJekFFCprk5lOIbEU4uBU C2/b0A0+AQL2EOxz5XGCUIBmahZPC/UePvU83z4Egt9B7mcMhwGHxznYSaaUDtdfUc0j U2fA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731772923; x=1732377723; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=38PQp1RQQ2NMP1hcM2NKIXHciL2ilqE4M5FPbIzuslY=; b=Lc9Q9aq+SHvMaOT0nRfz8xedDfigji81png3/O/mmfRLfDIsw+IWlRecYFGekU250h /evxkClZmydpjC1+4TuSdEenzXXkWu2uP7FA8heQrO/Nhpzx5P6JsOPaFbkQXLurypKe zF2TRl75aaYL4YL0nLuxmLJHJubTL9qF2vGo68eK6mUv/MhXkKRbMSfFb2j1+V2iw2pz jpSvbtyV61n+Nfa+ix3sgmmkHjB3RGvjDnqlFX0upFHoHA4S8aMfigrpCHd0a7pyPLRM B8fust/iCf2/36AMjnbLuw1zsnLMhNssgRyPgKbyxAsLTKDNXzHf8apzUQUUg9VxiKjY +b5Q== X-Forwarded-Encrypted: i=1; AJvYcCUmueibSe3MrlOP3YWC0B8LGOPqVycLW9wM/S2kLm8z+AQlM+TuEeuzo2+bchbmd3TlbZcl17XWyx2dvd0=@vger.kernel.org X-Gm-Message-State: AOJu0YxdT5ggBxTewHuAyGZ1qRbUHc+vzqXkuFiZRI5pLI7IWAGIaW3K 0501lQEgIpW0m2/I2dD+nXbrd8cYiZOP74Hd0M8psaG3eGFpr9my X-Google-Smtp-Source: AGHT+IEyY5xTEzo2l7XaqcLEW8MmJjYA8BSONOvXiTv670zrm9w9LaEoXycJgP5gUCdOsieDWorNyA== X-Received: by 2002:a05:6a20:729b:b0:1db:ec2b:7322 with SMTP id adf61e73a8af0-1dc90c18aedmr11919815637.46.1731772923095; Sat, 16 Nov 2024 08:02:03 -0800 (PST) Received: from localhost ([58.29.143.236]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7f8c1c3224asm3116635a12.29.2024.11.16.08.02.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 16 Nov 2024 08:02:02 -0800 (PST) From: Changwoo Min X-Google-Original-From: Changwoo Min To: tj@kernel.org, void@manifault.com Cc: mingo@redhat.com, peterz@infradead.org, changwoo@igalia.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: [PATCH 2/5] sched_ext: Manage the validity of scx_rq_clock Date: Sun, 17 Nov 2024 01:01:23 +0900 Message-ID: <20241116160126.29454-3-changwoo@igalia.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241116160126.29454-1-changwoo@igalia.com> References: <20241116160126.29454-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" An rq clock becomes valid when it is updated using update_rq_clock() and invalidated when the rq is unlocked using rq_unpin_lock(). Also, after long running operations -- ops.running() and ops.update_idle() -- in a BPF scheduler, the sched_ext core invalidates the rq clock. Signed-off-by: Changwoo Min --- kernel/sched/core.c | 6 +++++- kernel/sched/ext.c | 3 +++ kernel/sched/sched.h | 2 +- 3 files changed, 9 insertions(+), 2 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index a910a5b4c274..d0eb58b6a2da 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -784,6 +784,7 @@ static void update_rq_clock_task(struct rq *rq, s64 del= ta) void update_rq_clock(struct rq *rq) { s64 delta; + u64 clock; =20 lockdep_assert_rq_held(rq); =20 @@ -795,11 +796,14 @@ void update_rq_clock(struct rq *rq) SCHED_WARN_ON(rq->clock_update_flags & RQCF_UPDATED); rq->clock_update_flags |=3D RQCF_UPDATED; #endif + clock =3D sched_clock_cpu(cpu_of(rq)); + scx_rq_clock_update(rq, clock); =20 - delta =3D sched_clock_cpu(cpu_of(rq)) - rq->clock; + delta =3D clock - rq->clock; if (delta < 0) return; rq->clock +=3D delta; + update_rq_clock_task(rq, delta); } =20 diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index 9f9bc2930658..b8ad776ef516 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -2928,6 +2928,8 @@ static void set_next_task_scx(struct rq *rq, struct t= ask_struct *p, bool first) if (SCX_HAS_OP(running) && (p->scx.flags & SCX_TASK_QUEUED)) SCX_CALL_OP_TASK(SCX_KF_REST, running, p); =20 + scx_rq_clock_stale(rq); + clr_task_runnable(p, true); =20 /* @@ -3590,6 +3592,7 @@ void __scx_update_idle(struct rq *rq, bool idle) { int cpu =3D cpu_of(rq); =20 + scx_rq_clock_stale(rq); if (SCX_HAS_OP(update_idle) && !scx_rq_bypassing(rq)) { SCX_CALL_OP(SCX_KF_REST, update_idle, cpu_of(rq), idle); if (!static_branch_unlikely(&scx_builtin_idle_enabled)) diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 61efff790e24..03854ac9914b 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1758,7 +1758,7 @@ static inline void rq_unpin_lock(struct rq *rq, struc= t rq_flags *rf) if (rq->clock_update_flags > RQCF_ACT_SKIP) rf->clock_update_flags =3D RQCF_UPDATED; #endif - + scx_rq_clock_stale(rq); lockdep_unpin_lock(__rq_lockp(rq), rf->cookie); } =20 --=20 2.47.0 From nobody Fri Nov 22 16:17:56 2024 Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9B0F1191F9E for ; Sat, 16 Nov 2024 16:02:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731772929; cv=none; b=bgow8B1wvX/KiHnGUdXGm2bX3x2OJqz0ZMuRVgrM7sFcePNI3Baqok6fhKDozZj2xq7S/28hY6MQ9GXq1iZeqHaRHlXN6VrpeGoLRQ/uW1gc5sXeNA4+wiCnhlVjNMDIFveqmVfzePezR6EKC6ew9BfxkpxRcaqnw3rHeUPeopo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731772929; c=relaxed/simple; bh=URoaSsWNsSTDBVnXKWM7IW6JYC7xxG/Zo/nPyLKGfmE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Utj7Xy3ZzXGv5F9cdkieVVY5RlnkKrXxeoITW+cnnU1A9r1xLX0kvFtPh/HT7/yIyRXyjl7CBYmQz+8oYaf0qt2nUZzNU/l7vw0yhZh5WCOXkx98KYA4Ldq6PFGJq/8HiI7BGuIKxkiaYfiLQd6B/kwFB3+jHJamsRYn4Uw5BRM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=i6MHBgVE; arc=none smtp.client-ip=209.85.210.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="i6MHBgVE" Received: by mail-pf1-f177.google.com with SMTP id d2e1a72fcca58-72410cc7be9so2341245b3a.0 for ; Sat, 16 Nov 2024 08:02:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731772927; x=1732377727; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=QMycY57gYsPds40IIgx8o+5491JCQvawYObZO7Q8XFI=; b=i6MHBgVE8LzYeh+rDZ8bcWGrbqktqn02pQs95G6o50+ribFMDQVFhGam5MHQPrSOIR YkkgcErM2WTL99jTNfnio42Ngv3j5HRjKl4K498aro6J3pxIWOUTXMPZvLRw2teRI/hn SvVxUwhGpdtz/iBZ7GCE6aQDUGIcYYPcyjNhs1kAeKHBw4RQxrRX2dAOf8DJjQsvOGeg b5RAFCkqh010gXOkzvfNpxwl0QhugdudB/hztnhjvc1iWhV6zxKbNydVpMa/a6INi8xh CQ6m4QTGS9eTfVqAR7SOOXW8jdoWa9c7AdptRBjFAnvu+N7Xc97g6uIkz6t3rbRMENRe TG8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731772927; x=1732377727; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=QMycY57gYsPds40IIgx8o+5491JCQvawYObZO7Q8XFI=; b=J9GLP4oqayXUo439FJy0M5c3h0AFEZplMcKy+1GQQvDyA4eWMNvVZVLZX9q5BoVBr5 7avBM8qIoih7hpsEjfFj2Q8bzuYXge8xg5hbMePSoJVx6O4zpMF1UnJZh1phwIISehAm QopH3N9xWNppn2pinrVLpI7JINgSHGPzIXZd6HRgJwUQR29lhwdM4cDjklhjZHizRrym hvjKx4qjdtow2yLKOZ7Tp1TOX75+19jhPE4th3gqpKw/ALyJq0L0DZoEvgF3UoZ03Fc2 /nCMhSEkvT9I7lNpLNbpiyGhXhndZd3WxehtcKoxrn7ZSFYim1AT4z8EWOgm1KiFqjv0 5Pfg== X-Forwarded-Encrypted: i=1; AJvYcCVrD1Rj74OjzT4jHSp97cl7yjyKVoeY+NAR8w+LCJZoWO1C0EUyIilodb1++M5Vn8QNcewhr+FpUQRUiTI=@vger.kernel.org X-Gm-Message-State: AOJu0YzvVGt8C76fDI72dWg/SgQZSG0oHgsjZsUD/c7fioRTOpJ+KdpI EMQ/R2S2DlEiST7ViusVbtq5FM/9JpIZlPwEPFnAi+zCunQKHPif X-Google-Smtp-Source: AGHT+IE+DCkEW3YrSTAVNT5/vS7NbVd6a7tuTAH7r7kjprhrp2a0mchf2rouZw05bxQPg+RJhdFpxA== X-Received: by 2002:a17:90b:3504:b0:2ea:4150:3f71 with SMTP id 98e67ed59e1d1-2ea415042cfmr2204374a91.14.1731772926543; Sat, 16 Nov 2024 08:02:06 -0800 (PST) Received: from localhost ([58.29.143.236]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-211d0eca518sm29315795ad.88.2024.11.16.08.02.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 16 Nov 2024 08:02:06 -0800 (PST) From: Changwoo Min X-Google-Original-From: Changwoo Min To: tj@kernel.org, void@manifault.com Cc: mingo@redhat.com, peterz@infradead.org, changwoo@igalia.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: [PATCH 3/5] sched_ext: Implement scx_bpf_clock_get_ns() Date: Sun, 17 Nov 2024 01:01:24 +0900 Message-ID: <20241116160126.29454-4-changwoo@igalia.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241116160126.29454-1-changwoo@igalia.com> References: <20241116160126.29454-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Returns a high-performance monotonically non-decreasing clock for the current CPU. The clock returned is in nanoseconds. It provides the following properties: 1) High performance: Many BPF schedulers call bpf_ktime_get_ns() frequently to account for execution time and track tasks' runtime properties. Unfortunately, in some hardware platforms, bpf_ktime_get_ns() -- which eventually reads a hardware timestamp counter -- is neither performant nor scalable. scx_bpf_clock_get_ns() aims to provide a high-performance clock by using the rq clock in the scheduler core whenever possible. 2) High enough resolution for the BPF scheduler use cases: In most BPF scheduler use cases, the required clock resolution is lower than the most accurate hardware clock (e.g., rdtsc in x86). scx_bpf_clock_get_ns() basically uses the rq clock in the scheduler core whenever it is valid. It considers that the rq clock is valid from the time the rq clock is updated (update_rq_clock) until the rq is unlocked (rq_unpin_lock). In addition, it invalidates the rq clock after long operations -- ops.running() and ops.update_idle() -- in the BPF scheduler. 3) Monotonically non-decreasing clock for the same CPU: scx_bpf_clock_get_ns() guarantees the clock never goes backward when comparing them in the same CPU. On the other hand, when comparing clocks in different CPUs, there is no such guarantee -- the clock can go backward. It provides a monotonically *non-decreasing* clock so that it would provide the same clock values in two different scx_bpf_clock_get_ns() calls in the same CPU during the same period of when the rq clock is valid. Signed-off-by: Changwoo Min --- kernel/sched/ext.c | 71 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 71 insertions(+) diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index b8ad776ef516..b0374274ead2 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -7541,6 +7541,76 @@ __bpf_kfunc struct cgroup *scx_bpf_task_cgroup(struc= t task_struct *p) } #endif =20 +/** + * scx_bpf_clock_get_ns - Returns a high-performance monotonically + * non-decreasing clock for the current CPU. The clock returned is in + * nanoseconds. + * + * It provides the following properties: + * + * 1) High performance: Many BPF schedulers call bpf_ktime_get_ns() freque= ntly + * to account for execution time and track tasks' runtime properties. + * Unfortunately, in some hardware platforms, bpf_ktime_get_ns() -- which + * eventually reads a hardware timestamp counter -- is neither performant= nor + * scalable. scx_bpf_clock_get_ns() aims to provide a high-performance cl= ock + * by using the rq clock in the scheduler core whenever possible. + * + * 2) High enough resolution for the BPF scheduler use cases: In most BPF + * scheduler use cases, the required clock resolution is lower than the m= ost + * accurate hardware clock (e.g., rdtsc in x86). scx_bpf_clock_get_ns() + * basically uses the rq clock in the scheduler core whenever it is valid. + * It considers that the rq clock is valid from the time the rq clock is + * updated (update_rq_clock) until the rq is unlocked (rq_unpin_lock). + * In addition, it invalidates the rq clock after long operations -- + * ops.running() and ops.update_idle(). + * + * 3) Monotonically non-decreasing clock for the same CPU: + * scx_bpf_clock_get_ns() guarantees the clock never goes backward when + * comparing them in the same CPU. On the other hand, when comparing cloc= ks + * in different CPUs, there is no such guarantee -- the clock can go back= ward. + * It provides a monotonically *non-decreasing* clock so that it would pr= ovide + * the same clock values in two different scx_bpf_clock_get_ns() calls in= the + * same CPU during the same period of when the rq clock is valid. + */ +__bpf_kfunc u64 scx_bpf_clock_get_ns(void) +{ + static DEFINE_PER_CPU(u64, prev_clk); + struct rq *rq =3D this_rq(); + u64 pr_clk, cr_clk; + + preempt_disable(); + pr_clk =3D __this_cpu_read(prev_clk); + + /* + * If the rq clock is invalid, start a new rq clock period + * with a fresh sched_clock(). + */ + if (!(rq->scx.flags & SCX_RQ_CLK_UPDATED)) { + cr_clk =3D sched_clock(); + scx_rq_clock_update(rq, cr_clk); + } + /* + * If the rq clock is valid, use the cached rq clock + * whenever the clock does not go backward. + */ + else { + cr_clk =3D rq->scx.clock; + /* + * If the clock goes backward, start a new rq clock period + * with a fresh sched_clock(). + */ + if (pr_clk > cr_clk) { + cr_clk =3D sched_clock(); + scx_rq_clock_update(rq, cr_clk); + } + } + + __this_cpu_write(prev_clk, cr_clk); + preempt_enable(); + + return cr_clk; +} + __bpf_kfunc_end_defs(); =20 BTF_KFUNCS_START(scx_kfunc_ids_any) @@ -7572,6 +7642,7 @@ BTF_ID_FLAGS(func, scx_bpf_cpu_rq) #ifdef CONFIG_CGROUP_SCHED BTF_ID_FLAGS(func, scx_bpf_task_cgroup, KF_RCU | KF_ACQUIRE) #endif +BTF_ID_FLAGS(func, scx_bpf_clock_get_ns) BTF_KFUNCS_END(scx_kfunc_ids_any) =20 static const struct btf_kfunc_id_set scx_kfunc_set_any =3D { --=20 2.47.0 From nobody Fri Nov 22 16:17:56 2024 Received: from mail-pf1-f182.google.com (mail-pf1-f182.google.com [209.85.210.182]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0944C193091 for ; Sat, 16 Nov 2024 16:02:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731772932; cv=none; b=enSio9E+w0BQFqdG+8IYgJugcJeCMQ9awFQZ1JcrmN0ZUmYMLHQr5YMjQWfhBqwv4r68r/6GevQLrK6n2L69hXyvZLETTvq36cOz0AfhAQTLX30kvtBjwdGANVyVNDB/40Eh+X1R3/R07Hya68ZuwYKxoP6UGCf7vqhCWwp/XD0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731772932; c=relaxed/simple; bh=o0YDEjtcd9S385MR9NmR2UtjFN+yk2/00o7+024uHmg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=aoN/a960HJgOYzIflQvdFzxgeTxCIJiVYI33mwL168et6i1aKxWBeDCQPRtEOEPyAHlJlX80A0jYQyndEypCL5MD88wy1mvBjgjXRjpj3BEeP1+bH/iZRQQmpxXGFXzcJoEbXaX6A+RFpGADJPYl5yv1TF4sJbG0JSjmhLMVGUM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=er/M/OWY; arc=none smtp.client-ip=209.85.210.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="er/M/OWY" Received: by mail-pf1-f182.google.com with SMTP id d2e1a72fcca58-72487ebd2f5so429049b3a.1 for ; Sat, 16 Nov 2024 08:02:10 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731772930; x=1732377730; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=p7JCDTTn3vTxFGmkfNj8ZEUqCT3sK0X8BFkqLAI6rAs=; b=er/M/OWY7f4sTZMZClRH5p7RUR4cUHLG7NNPckI9x+pjU5npKRciRV4ywH8Wdd9hrQ Zeri3tnQBG5rZgneNzUl4qw6lU7LKrgN4tJ+SS2ZZX8dsbyGKHaKF1e2tRXKTlQCV7f1 IuCsw90egnDvlZ3gj06mmNBtHiJHiCG6LN10RSFL0vzpSd1m9YT9mcWbVRFRorBA2VWq Ws43wGRquQeercRGB2/490Y0G0s/GznqaT8iOLHGY9DKCpPgReMQVbIxiymYOHKVYGLK 9mwlqmLhrWQeBUJooUHVNSAOXZ4nvmm6Y4N3380GpntiFte6UutIAr/BrfQMTfvohPAT 1rdw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731772930; x=1732377730; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=p7JCDTTn3vTxFGmkfNj8ZEUqCT3sK0X8BFkqLAI6rAs=; b=U3yRLFYh1zhLHfWik3gNOZ5UXQKVhuXOGHvs980h8L41aa9Msq+DkLqMCqWHfApLVb F9q2z+6lOc+zL1Nbc7gjLWB+UwtztcmizU9/XNrcv25ai4pckDVlcKdRwZ9ZA6qyiJ6W HTngrsi+OS/e4Qt94nxiZxYUgAiyCURAQaHiWFlH6py+l8TksAyX91p4AYD3RoVRajjR 97imsLyIsBib8b9XOngQAJu3ise/GcbwijWx9p98yfobNJLWk1z39q299ztotupBFASS pyur0m8HfSxcYcNE2mcIp5nggiGim5PcBgtCh9yONYfPkEuhMpWB69iAd3T1nIKlude7 OKlg== X-Forwarded-Encrypted: i=1; AJvYcCWSLFxS7RQwNK/bgj5iBJaAx2iWvm6lz8Vd2Xj5sYObRS6+rm2fZ+7qhmC6rQjLcpP7l8eFR6FgJUktJv0=@vger.kernel.org X-Gm-Message-State: AOJu0YykCNhT16dYpah1NgMDJ6RRVzs0d9LS/g9T2AI5C/8AOn1+PYUN E8srUTdq5GNAG7lgspWtMSI+LXcW9XdFNuJ485ei7T/bsyN24Gly X-Google-Smtp-Source: AGHT+IEW3D+oOgg5ZmZqLrrYpAwHIzZ8WlZUJzEYdgoDk0OpEirTug6cOUYYnD5X4SsV8VqrXGnRhA== X-Received: by 2002:a05:6a00:10ce:b0:71e:3b8f:92e with SMTP id d2e1a72fcca58-72476b72b2bmr9148583b3a.3.1731772930052; Sat, 16 Nov 2024 08:02:10 -0800 (PST) Received: from localhost ([58.29.143.236]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-7247711de6fsm3291344b3a.61.2024.11.16.08.02.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 16 Nov 2024 08:02:09 -0800 (PST) From: Changwoo Min X-Google-Original-From: Changwoo Min To: tj@kernel.org, void@manifault.com Cc: mingo@redhat.com, peterz@infradead.org, changwoo@igalia.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: [PATCH 4/5] sched_ext: Add scx_bpf_clock_get_ns() for BPF scheduler Date: Sun, 17 Nov 2024 01:01:25 +0900 Message-ID: <20241116160126.29454-5-changwoo@igalia.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241116160126.29454-1-changwoo@igalia.com> References: <20241116160126.29454-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" scx_bpf_clock_get_ns() is added to the header files so the BPF scheduler can use it. Signed-off-by: Changwoo Min --- tools/sched_ext/include/scx/common.bpf.h | 1 + tools/sched_ext/include/scx/compat.bpf.h | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/tools/sched_ext/include/scx/common.bpf.h b/tools/sched_ext/inc= lude/scx/common.bpf.h index 2f36b7b6418d..230c7f2e8ad6 100644 --- a/tools/sched_ext/include/scx/common.bpf.h +++ b/tools/sched_ext/include/scx/common.bpf.h @@ -72,6 +72,7 @@ bool scx_bpf_task_running(const struct task_struct *p) __= ksym; s32 scx_bpf_task_cpu(const struct task_struct *p) __ksym; struct rq *scx_bpf_cpu_rq(s32 cpu) __ksym; struct cgroup *scx_bpf_task_cgroup(struct task_struct *p) __ksym __weak; +u64 scx_bpf_clock_get_ns(void) __ksym __weak; =20 /* * Use the following as @it__iter when calling scx_bpf_dsq_move[_vtime]() = from diff --git a/tools/sched_ext/include/scx/compat.bpf.h b/tools/sched_ext/inc= lude/scx/compat.bpf.h index d56520100a26..d295c59e3f05 100644 --- a/tools/sched_ext/include/scx/compat.bpf.h +++ b/tools/sched_ext/include/scx/compat.bpf.h @@ -125,6 +125,11 @@ bool scx_bpf_dispatch_vtime_from_dsq___compat(struct b= pf_iter_scx_dsq *it__iter, false; \ }) =20 +#define scx_bpf_clock_get_ns() \ + (bpf_ksym_exists(scx_bpf_clock_get_ns) ? \ + scx_bpf_clock_get_ns() : \ + bpf_ktime_get_ns()) + /* * Define sched_ext_ops. This may be expanded to define multiple variants = for * backward compatibility. See compat.h::SCX_OPS_LOAD/ATTACH(). --=20 2.47.0 From nobody Fri Nov 22 16:17:56 2024 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E73B9198848 for ; Sat, 16 Nov 2024 16:02:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731772935; cv=none; b=mX07bbbLXLlP8QKTZcA6/ks5X2WFdOBTFdeOfnld/PWr1Ihp8fmmt8M+S81bDTXFzEWr/Tavq4FTexjo+/iY8mizzOoxMcQ9fHpcFc27Ac/YfqJNYhjxtl9ROyYWMPhIeaIfyQFuMKTZEDHeREe5Gqlaqf/bEGyhW0C52S0XCqU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731772935; c=relaxed/simple; bh=29OhbxoZ0xm6G7JzBb4GdoFklR8a8xfzpQfSQfoBXk4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=acUVA01wLBHkotAXqMjuu+4wQJrMtkYK+kgoH2XFNfyb5bkg3RO1BdGBSKEkZbookqhZ24MHDm6VUydaNDmqj8JttRFbFXWsZXjU5lcDt47bZWMZ8EIwfvMmbNaVu41SWS+KqXN9UCNB8audtL57gK0f6zat2KBCLPEDRzCnzPg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=jy9WBBFd; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="jy9WBBFd" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-20caccadbeeso18597065ad.2 for ; Sat, 16 Nov 2024 08:02:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1731772933; x=1732377733; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=hEUGftoYqMghGeJCrzA2wJAFP+c9rSA4g2ZKWQjS6GI=; b=jy9WBBFdZWlX+aqc3w5GBlY18Piz8jo3rsA9Kh6jaEUW8wUlI8j4qBpZoTIPByVul/ Bqwudv+lHRoIoFHjoJxgQ2+r1Rv5Mmj5GUcXBNEjHEIs8HYd0j/WNo31lpKQRGXEyU7v PtWZPWKBUZsHv2cc2gLqzcRSAIgkNO383OPYt/01hrZClZdG9QxZctm0u7D1wQpDsqiG SVpOchQgrEUoSmo5u8LCEhO6phLaLTK5l64HpixhYhAxMC2udIoVJFKYsbbuB96Teei+ FgFFhtLMmR+qQJDsoQlxgKJi3UnUVYP19cjYvG4VV4PyJGJOBhpknGMOhapCbA3kVaW7 8Jeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1731772933; x=1732377733; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hEUGftoYqMghGeJCrzA2wJAFP+c9rSA4g2ZKWQjS6GI=; b=g3eUIQf6QttWMekU7dPCwlnlneZhyDn1KSm7CHe61Vd4sS1tFss82j7XxoSDC0R4Il eZdHW42fqFv0hrM2yRSO4gWXsDxwWnSwCueVX6WaYIrXW7UTEDH2NcOpWMxZkUTohDaj CzZHkl4qDq+2EmQN6e1N/vbBuxPDXLl5LDNsZebKLiFgZRetXgcG3fIX6Sw3B4PMH1Ay K439nqV3pOg799klcr/q/7rnRcuXtS34JTR4guP+m36wJaG5rFlcOpka/uYok06pYaKX +OJBItgCRWwzf2DNrnB50mTvXCq9yIoENJlCJIcc6H+Md8dzZGZrlncClRK1sS2bYq+9 Fiww== X-Forwarded-Encrypted: i=1; AJvYcCW8oJtmG+5T7OQ6DOtOx91FAYOIq1oeg9G5xNCyAAyLfvFWK5L7iGmvlo9Jak66Uq6jxGy2d44B22qro/E=@vger.kernel.org X-Gm-Message-State: AOJu0Yw1+3bhF0yxE5E8SaSpOW7+dkRiXbIyGxEyYkfvnsfuhhDE1OjS YyEmBxK/ay8M9gBbFmmQYhVvXUBCYT+HyCOliN9WfhRC70Yg3Y7c X-Google-Smtp-Source: AGHT+IFQzE03yOcJF6hT2Bip9XWJXr23JrMiXmrisg0f9+jeNvS/flZn9tQ/f2G0LVaDu+haL6Zunw== X-Received: by 2002:a17:902:d48e:b0:20c:e5b5:608a with SMTP id d9443c01a7336-211d0d5ea23mr94522405ad.5.1731772933238; Sat, 16 Nov 2024 08:02:13 -0800 (PST) Received: from localhost ([58.29.143.236]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-211d0f4535csm29137285ad.178.2024.11.16.08.02.12 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 16 Nov 2024 08:02:12 -0800 (PST) From: Changwoo Min X-Google-Original-From: Changwoo Min To: tj@kernel.org, void@manifault.com Cc: mingo@redhat.com, peterz@infradead.org, changwoo@igalia.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: [PATCH 5/5] sched_ext: Replace bpf_ktime_get_ns() to scx_bpf_clock_get_ns() Date: Sun, 17 Nov 2024 01:01:26 +0900 Message-ID: <20241116160126.29454-6-changwoo@igalia.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241116160126.29454-1-changwoo@igalia.com> References: <20241116160126.29454-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In the BPF schedulers that use bpf_ktime_get_ns() -- scx_central and scx_flatcg, replace bpf_ktime_get_ns() calls to scx_bpf_clock_get_ns(). Signed-off-by: Changwoo Min --- tools/sched_ext/scx_central.bpf.c | 4 ++-- tools/sched_ext/scx_flatcg.bpf.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tools/sched_ext/scx_central.bpf.c b/tools/sched_ext/scx_centra= l.bpf.c index e6fad6211f6c..cb7428b6a198 100644 --- a/tools/sched_ext/scx_central.bpf.c +++ b/tools/sched_ext/scx_central.bpf.c @@ -245,7 +245,7 @@ void BPF_STRUCT_OPS(central_running, struct task_struct= *p) s32 cpu =3D scx_bpf_task_cpu(p); u64 *started_at =3D ARRAY_ELEM_PTR(cpu_started_at, cpu, nr_cpu_ids); if (started_at) - *started_at =3D bpf_ktime_get_ns() ?: 1; /* 0 indicates idle */ + *started_at =3D scx_bpf_clock_get_ns() ?: 1; /* 0 indicates idle */ } =20 void BPF_STRUCT_OPS(central_stopping, struct task_struct *p, bool runnable) @@ -258,7 +258,7 @@ void BPF_STRUCT_OPS(central_stopping, struct task_struc= t *p, bool runnable) =20 static int central_timerfn(void *map, int *key, struct bpf_timer *timer) { - u64 now =3D bpf_ktime_get_ns(); + u64 now =3D scx_bpf_clock_get_ns(); u64 nr_to_kick =3D nr_queued; s32 i, curr_cpu; =20 diff --git a/tools/sched_ext/scx_flatcg.bpf.c b/tools/sched_ext/scx_flatcg.= bpf.c index 4e3afcd260bf..3be99f3c32fd 100644 --- a/tools/sched_ext/scx_flatcg.bpf.c +++ b/tools/sched_ext/scx_flatcg.bpf.c @@ -734,7 +734,7 @@ void BPF_STRUCT_OPS(fcg_dispatch, s32 cpu, struct task_= struct *prev) struct fcg_cpu_ctx *cpuc; struct fcg_cgrp_ctx *cgc; struct cgroup *cgrp; - u64 now =3D bpf_ktime_get_ns(); + u64 now =3D scx_bpf_op_clock_get_ns(); bool picked_next =3D false; =20 cpuc =3D find_cpu_ctx(); --=20 2.47.0