From nobody Wed Dec 17 17:53:15 2025 Received: from mail-oi1-f179.google.com (mail-oi1-f179.google.com [209.85.167.179]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8E2A443146 for ; Mon, 9 Dec 2024 06:15:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.179 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733724952; cv=none; b=YM+/8MUF2N0Cmi7eC1BVugJrDNMtTlfI+AqGSt9s+TJEM7JI5rGBmTlzsIVCAfWZXr6s6ZEQ9MmAM6XSN/nQ1XljYOrS5tG61yE0s9dqpm+tGFayj0kjOMkICUGzBwqQSWzuASVnEmumVk7GboA7CR9LRmx9z+Iaor0INA1FiLA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733724952; c=relaxed/simple; bh=Txl962X9D9KxVCAHDq5L6m6huEZFNRDOQ99W8Ng9mX4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=sUpK5A71seUyk1r4gvMCAAYYPWucoEkYheK/fRhU60aWMPFXZso8Rz36JzyLsQ9l0tJZOZX7hRBC0M5B1ikQMVPqWSly7Ihau//KZ8Bp0QKEOAlcTOvF8xFt3cNYoN9bgKuVuSK7MNkge7qPaaAe/DrEHPr+yKlZHKt8wkBnfro= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=BXEIq4Wq; arc=none smtp.client-ip=209.85.167.179 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="BXEIq4Wq" Received: by mail-oi1-f179.google.com with SMTP id 5614622812f47-3eb5a870158so188762b6e.3 for ; Sun, 08 Dec 2024 22:15:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733724949; x=1734329749; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pnNhPvtjjeA0SELvFx04GxV15CCO7vm1wX4v8xLLeWc=; b=BXEIq4WqU1bIv2ndIXzVSWUo0LNa6CKXV1b5kbYFG4HwEdF2uAEdYtMLBQIZy7LbAo JeCrHnFM8lkvhzjurHgdwyBIEk+5YOHSeCVLNltM1lo0A0ekplF5NqOmuR34tkYYsLdB M3JUIG45eC5CFK10dFeGTuKjScesNm1LuOu3S7RKJlASZdY11w0VaxkRgb/umnG6i803 qTvH9e/THZHstghWupxrMwu2cYe/b+Sn1UaLmN8X/NfjDE1vPC/9YwY7/Lfh+JxS1Fq8 c2NBp0FFMoz/3Gx3eTjO/rCvHbgqpmHaA72N+BshhybNvcq40urGsBxoa3bwkRvdLDG2 t6cQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733724949; x=1734329749; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pnNhPvtjjeA0SELvFx04GxV15CCO7vm1wX4v8xLLeWc=; b=wu1WFIrySYt+iLO5KTuAKKOgoahTVmI1GolhDvxckJGQAqACgpTOcNBfjdpnLn2+Fi jQgs/8ggv9pZ4loOigZyFF90xHPN9VH288vqqhlKvk1fPqzJAqYkzfx3Zcn1U1NJ/N7C rDnycqNw3BUxMKsKYlgRjLxt1OQFiW/Lq2IyNUeEexU2S6ut5YF6mqhwH6NSw88ezGtM 0/kSZXf9k30FZQV25pn21j2HeTUEmV4xBEykJx4LgxCsJ2SHHuo0Xx7KuLkbJQ08Xgkd 2mKM/KApioaxHDdbw3EJeyIjVAmC0LjAwHxv1Y8+x9ll31OGs2qMaQJDMxwysQjde5cd R/tw== X-Forwarded-Encrypted: i=1; AJvYcCUeqbJxOos308sULarIhuZ9e0MflwGo6UMMjhXEt2Dtz3sRwLeiu6Em8OiU8BCXZjeSU2pY5vSjpuJi3qg=@vger.kernel.org X-Gm-Message-State: AOJu0YyN2rIx8iSNIC9e41b2TWoVUiklRLEYmJMfFWU1p7lpwmxYiZmK UvqRS/7SIaaafQVraQmlDNHd1yyP79BO+LTLk0uOEY/99JJSsfBS X-Gm-Gg: ASbGncvpP0IDpjOpYawirwZ7SegNRBHD6TiX71fMIer6sreGuI2MUkYTIfot6zgrfYF UctORNxLVaIXyog3qnB+SjFNh5PR5134374PTeiGGMGbqx2XnxGGcGoOSv09pgqqDyj4yzR33qa NMSkHRl2Gs5iaRZo39N0aE0y4U8qErEFlCjINzXAkEFtveyK75QhYC4LNaXowu4kdQ6Z1mFWoYY HnWXgsx1VNef+uCEQVOEGV1EaXvkekxL9WasoUjbJzyNdUFQXnIEVnB/DDcZnhEzqRlMKDg+oLY 7hEG8w== X-Google-Smtp-Source: AGHT+IH0QB16al2MNtCP5J2c9l7nsN7n/OZFeMIS2zGWlp/k5mAHTZrWnn52QUb6jrJW8/Jmsl2a9A== X-Received: by 2002:a05:6808:124b:b0:3ea:5809:2835 with SMTP id 5614622812f47-3eb19ce7e2fmr10726413b6e.21.1733724949528; Sun, 08 Dec 2024 22:15:49 -0800 (PST) Received: from localhost ([58.29.143.236]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7fd416536dbsm2054166a12.16.2024.12.08.22.15.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 08 Dec 2024 22:15:49 -0800 (PST) From: Changwoo Min X-Google-Original-From: Changwoo Min To: tj@kernel.org, void@manifault.com, mingo@redhat.com, peterz@infradead.org Cc: changwoo@igalia.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: [PATCH v4 1/6] sched_ext: Relocate scx_enabled() related code Date: Mon, 9 Dec 2024 15:15:26 +0900 Message-ID: <20241209061531.257531-2-changwoo@igalia.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241209061531.257531-1-changwoo@igalia.com> References: <20241209061531.257531-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" scx_enabled() will be used in scx_rq_clock_update/stale() in the following patch, so relocate the scx_enabled() related code to the proper location. Signed-off-by: Changwoo Min Acked-by: Andrea Righi --- kernel/sched/sched.h | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 76f5f53a645f..440ecedf871b 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1717,6 +1717,19 @@ struct rq_flags { =20 extern struct balance_callback balance_push_callback; =20 +#ifdef CONFIG_SCHED_CLASS_EXT +extern const struct sched_class ext_sched_class; + +DECLARE_STATIC_KEY_FALSE(__scx_ops_enabled); /* SCX BPF scheduler loaded */ +DECLARE_STATIC_KEY_FALSE(__scx_switched_all); /* all fair class tasks on S= CX */ + +#define scx_enabled() static_branch_unlikely(&__scx_ops_enabled) +#define scx_switched_all() static_branch_unlikely(&__scx_switched_all) +#else /* !CONFIG_SCHED_CLASS_EXT */ +#define scx_enabled() false +#define scx_switched_all() false +#endif /* !CONFIG_SCHED_CLASS_EXT */ + /* * Lockdep annotation that avoids accidental unlocks; it's like a * sticky/continuous lockdep_assert_held(). @@ -2505,19 +2518,6 @@ extern const struct sched_class rt_sched_class; extern const struct sched_class fair_sched_class; extern const struct sched_class idle_sched_class; =20 -#ifdef CONFIG_SCHED_CLASS_EXT -extern const struct sched_class ext_sched_class; - -DECLARE_STATIC_KEY_FALSE(__scx_ops_enabled); /* SCX BPF scheduler loaded */ -DECLARE_STATIC_KEY_FALSE(__scx_switched_all); /* all fair class tasks on S= CX */ - -#define scx_enabled() static_branch_unlikely(&__scx_ops_enabled) -#define scx_switched_all() static_branch_unlikely(&__scx_switched_all) -#else /* !CONFIG_SCHED_CLASS_EXT */ -#define scx_enabled() false -#define scx_switched_all() false -#endif /* !CONFIG_SCHED_CLASS_EXT */ - /* * Iterate only active classes. SCX can take over all fair tasks or be * completely disabled. If the former, skip fair. If the latter, skip SCX. --=20 2.47.1 From nobody Wed Dec 17 17:53:15 2025 Received: from mail-pl1-f170.google.com (mail-pl1-f170.google.com [209.85.214.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1751E2147F8 for ; Mon, 9 Dec 2024 06:15:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733724957; cv=none; b=imWv8XWWHlxa5TqiQe6QG7jg59Wi7deKOgYmUhFh1PjEXMENrhAE3HB1/DLRUSBf0D1RdDfacn/EgqoVzqCaSgbX4IYbKX35sSiwra3gDnezfo+2k2yjxAPkSTxIxUHjbqFOPO0qq3jWN9i+TrRYq7HJmIxLjcq1B3ZN5y/WvKk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733724957; c=relaxed/simple; bh=MdDTxfGKx6bu9ZHjfydC1MEn4XRy5nf7GuWie5IjQzA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=L8bnDvqWQe1qH6xI8rvhnC9NIgt+EU+Wcuh5qxaz9SuQmMZ8mFxCRU1nsZjBBdKcFxBQepVQnk9m1fYKRcTb4HxlRH6xVGaRVnDBVxOVSxRYLdGPesNQ8ANVcX5wUaOXRMrttzLs5fWhl7ILaOu8NNVJmxL6YujXwov17UqLSVk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=di1kgpx5; arc=none smtp.client-ip=209.85.214.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="di1kgpx5" Received: by mail-pl1-f170.google.com with SMTP id d9443c01a7336-21661be2c2dso944425ad.1 for ; Sun, 08 Dec 2024 22:15:53 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733724953; x=1734329753; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=F7kQonWe5l9Kh79FyLWu6pMklDdWxAHpXcrT/0oQHYM=; b=di1kgpx5COvCAir0jmH+q7hzUlbJaavEEbmpsh/unJmkN2XndGEoN1JssHZ0I2jPR2 HpqH1FENwjzRsHGcp0U4rsRVgWhfcTUEI5gbk9cuV4WKntz0xEGhhzhH7SIjwwV32O+O VPFED8Uke1hHGv1AKJ0fcDhlfTckJE4apXHV8t6mROHOcyBk4xPrc33gfPRgIIQQ4y76 b1SYca2ic3KfsFWxZN4Z4quy+xWcIrJ3QWu8IjkRg+GET5Z/y4ceII8WwPNjUsY1ybRM qmJpRuRD3hJJlN8ocRE/xy6IzLJmShldX+XB1yxTtJisKJxiwAGhGiswGGxzQzIHsaYh hLoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733724953; x=1734329753; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=F7kQonWe5l9Kh79FyLWu6pMklDdWxAHpXcrT/0oQHYM=; b=tnQFrVmP9Ws/9bgbupKXrx0zY0nVyJvU7rb96My0UKThgtLIY0+kJHBUH4lZjsgO7z A2MfpdcPuT8astK+KGw6fSMroDMaX6mwVNgMVkyCtzubNJyF3hi4V07/dUtHxSPQulrb sWJwQ8jVGfaStxvITgBLONcXieTdjPz8JUDDP4YQ29zWqoQOuo5jZ6AEPOFPaWxj0uXX ajBZx/05m+PPs/azDnmvBzGfweS79gawig27ACa+nPUIM0OHFs1UMGk38GhlSu4ezBNH sciy7zRjYX6tpKXXZ4IbXZCA/Nh+vtu58Sx8Wqtv7VnDsHXRDnD0ME/6V+zLasR1Y6Pa JfvQ== X-Forwarded-Encrypted: i=1; AJvYcCWVzYtyY9N/qVZDl3AsTaxJZPBpHMoYGkgfw8o+XXyoXAjGNIEjN7ndfWcSPs2tNDXPQS8b5vaHmHUair4=@vger.kernel.org X-Gm-Message-State: AOJu0YzSRlK3BhWKv5WUYXL/ou4lMaPxC4/NHv/sWg+fHrIaoawDvUw6 RmgjNPaWuBIlak6YX1+IN4M3CQfBxPLHUuobvckJhf1Abe52KAUN X-Gm-Gg: ASbGnct+n5z69uhW8H1b7mMWUTktMcfmoucOXt7mwKaWyQLqSHqIZXKeqqkW7unUUE2 Sxwe46u+wxot6Rt/32o2ZOog5i4W9WjnhGfqpcL1Fd5E9ULabADErLs2og5gnJF4/rVwcwiP0y3 nzuy6xTHgY+q7AHFaG/kBMmHMgBwiei0XWu2g9yeCAlDZaeKR2EG/B6LaT5tYteE85k0LwXYQV6 rtyKfOj8FlHeXPi3RWkYLsxfOzXgSD7kf9M+7gio7wnFLMEcO5FmN5RKp/kfV8ONGwgHEYxdg+d Pufz4w== X-Google-Smtp-Source: AGHT+IEXT8C2guc2VNHTufgL0ACYxl6+A7855+73rckfBd3nDSUWho4w2D8ilsy0bUA5bU6T1nndZg== X-Received: by 2002:a17:902:ea02:b0:216:2b14:b625 with SMTP id d9443c01a7336-2162b14cdfamr132835015ad.31.1733724953257; Sun, 08 Dec 2024 22:15:53 -0800 (PST) Received: from localhost ([58.29.143.236]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2165590a8ebsm6925375ad.271.2024.12.08.22.15.52 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 08 Dec 2024 22:15:52 -0800 (PST) From: Changwoo Min X-Google-Original-From: Changwoo Min To: tj@kernel.org, void@manifault.com, mingo@redhat.com, peterz@infradead.org Cc: changwoo@igalia.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: [PATCH v4 2/6] sched_ext: Implement scx_rq_clock_update/stale() Date: Mon, 9 Dec 2024 15:15:27 +0900 Message-ID: <20241209061531.257531-3-changwoo@igalia.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241209061531.257531-1-changwoo@igalia.com> References: <20241209061531.257531-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" scx_rq_clock_update() and scx_rq_clock_stale() manage the status of an rq clock when sched_ext is enabled. scx_rq_clock_update() keeps the rq clock in memory and its status valid. scx_rq_clock_stale() invalidates the current rq clock not to use the cached rq clock. Signed-off-by: Changwoo Min Acked-by: Andrea Righi --- kernel/sched/sched.h | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 440ecedf871b..7e71d8685fcc 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -754,6 +754,7 @@ enum scx_rq_flags { SCX_RQ_BAL_PENDING =3D 1 << 2, /* balance hasn't run yet */ SCX_RQ_BAL_KEEP =3D 1 << 3, /* balance decided to keep current */ SCX_RQ_BYPASSING =3D 1 << 4, + SCX_RQ_CLK_VALID =3D 1 << 5, /* RQ clock is fresh and valid */ =20 SCX_RQ_IN_WAKEUP =3D 1 << 16, SCX_RQ_IN_BALANCE =3D 1 << 17, @@ -766,9 +767,11 @@ struct scx_rq { unsigned long ops_qseq; u64 extra_enq_flags; /* see move_task_to_local_dsq() */ u32 nr_running; - u32 flags; u32 cpuperf_target; /* [0, SCHED_CAPACITY_SCALE] */ bool cpu_released; + u32 flags; + u64 clock; /* current per-rq clock -- see scx_bpf_now_ns() */ + u64 prev_clock; /* previous per-rq clock -- see scx_bpf_now_ns() */ cpumask_var_t cpus_to_kick; cpumask_var_t cpus_to_kick_if_idle; cpumask_var_t cpus_to_preempt; @@ -1725,9 +1728,28 @@ DECLARE_STATIC_KEY_FALSE(__scx_switched_all); /* all= fair class tasks on SCX */ =20 #define scx_enabled() static_branch_unlikely(&__scx_ops_enabled) #define scx_switched_all() static_branch_unlikely(&__scx_switched_all) + +static inline void scx_rq_clock_update(struct rq *rq, u64 clock) +{ + if (scx_enabled()) { + rq->scx.prev_clock =3D rq->scx.clock; + rq->scx.clock =3D clock; + rq->scx.flags |=3D SCX_RQ_CLK_VALID; + } +} + +static inline void scx_rq_clock_stale(struct rq *rq) +{ + if (scx_enabled()) + rq->scx.flags &=3D ~SCX_RQ_CLK_VALID; +} + #else /* !CONFIG_SCHED_CLASS_EXT */ #define scx_enabled() false #define scx_switched_all() false + +static inline void scx_rq_clock_update(struct rq *rq, u64 clock) {} +static inline void scx_rq_clock_stale(struct rq *rq) {} #endif /* !CONFIG_SCHED_CLASS_EXT */ =20 /* --=20 2.47.1 From nobody Wed Dec 17 17:53:15 2025 Received: from mail-pl1-f175.google.com (mail-pl1-f175.google.com [209.85.214.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0D3822147F6 for ; Mon, 9 Dec 2024 06:15:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.175 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733724958; cv=none; b=C+zrAQ1xFaBjHeNHsZ7iADOG1o1xwl60YC9P784YDRFsGgjRjBNHGOOkGVWPJPtvt8YXM0jgjDacCDpmI2d+jaP8no3WmZ91Uo14El7caCsISaDVluAPK7FNM8bKhBukPy1HuZayVjb7443VdUbdtzVlbTrC7r7DImj698NGF5M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733724958; c=relaxed/simple; bh=5Z8mZYF8p+Smo6Mg7rjZn0niYCgmWyYNROi2qqxdffk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TTb5hQl/xZJvU5wVd49XFVggAEBXJiRm+Eqx26kVZWIkhDrpHfzjJbkM2bplhZINrds5+Ib/0rEMg/A0Q7z0AHJVMyvnFuV/JFMYZ4BeWDU78QHlLMgHhLsGOQORixtirS+afpMR5Rk3CZYn/KJWxYunaEIUDL9finK86YUpgMA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=YlC6cO4a; arc=none smtp.client-ip=209.85.214.175 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="YlC6cO4a" Received: by mail-pl1-f175.google.com with SMTP id d9443c01a7336-216426b0865so6038115ad.0 for ; Sun, 08 Dec 2024 22:15:56 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733724956; x=1734329756; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TTjz7Q36fm+eHR0LrCJEPj1whyTvpK9Y0P7GSkq40Sk=; b=YlC6cO4aIdlnykct+80WCS8Lg5IDKeNkV5yE76JbUvaB7+dAG3BKCmP7FyOBd9Ba1X v/9QW+QXcL8/QUdnOHCK+nCteL4DvaPeBAyryj5olWUe2G/wtMEtB7X02Zyu7ro0mmLA bGsTWRz/4nX8qRH7HPWl6hplH71F8haY/4bh9tQEo0WRFPOqaCy7xWQUKZVDVkU2o/oP 8ImtSi5BdN8S77V6i9g8rvlAfl43YtOwifVH2jjxXfr8uNPok03/SEt0GpZrWVhpxF3K 25arbphA5acI8wO8ijhMm3sICm3pLt93CQQYWpAa1iBp93C7XQ2glgDr3coz5x3IZVTw 3DNw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733724956; x=1734329756; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TTjz7Q36fm+eHR0LrCJEPj1whyTvpK9Y0P7GSkq40Sk=; b=Wu6u58+aylHxTQVDhxdhejIKWmi3QaO5oxIQu2tAwIJz10j/pCVQUiq1eXBdl9DgV5 VcJEjqv4QCfNyCe9mJzu8WRmi8N9wINPGPA9Fs08LFg6fH3ndHQcVGjKx/CqXLDCYJAh EwmpQQGBn1ENl+WnWGvmz1zw7TLNjxnxEJzJYPZVtHc5euaoBH5DmLMx2WutOyIYL2sb Zu/SBNcEN5kMx6aOu2LIzgTsWP4DHVmZ5gsTWsK5RSxZhNIrqU4yW66iF5PDGxSQ/1et kSGw/KGnc3Uy8ZA1Py4kfUFG1DLQ1YMolf0iQYH56ZZD4W6IDF8kEWVsuTlXxgGfVcg8 lFRg== X-Forwarded-Encrypted: i=1; AJvYcCUTkfuCesNRp/IDjlYEd6Fm/R8GhGXbpLvYHwIFrBfPtqS/yA1w6iNRNnN4g3y64Q+2jK9Wr8qDG//jhPY=@vger.kernel.org X-Gm-Message-State: AOJu0YzOZf0aSp7VDNHduQ13SEEIHTwFN7JDfbcA3HPtP4iAX3fq0wZQ 0OLn3nK28DRGL2+9mgVu0KfivPvbq8U5TPte0ISiyexwmSLTfBUB X-Gm-Gg: ASbGncvPmDmDCkw+/DvQ3+ZZrKjrElGp03btRYbwtoPsbVmHbT6K7yuTci4y/QCEkp5 Azqd/pnvtL4uPTDVCW7meZa7qs6ZWlCn01rPQ4MmkMMt4ifVWUk1fjKUgFRMTksP2+EEjs+Kd/+ uaWFdWYrxafJWmSKSimve27xG61+gsaJcegAi4neMzQgBEpn4vNbmOAJn7tOmnDJ9DGguZNVtBq 09K0DS5hVkEteSIXDlZ5m/4sZt2lJrf1kLH2pJzOgbHp2q7X/914qydXoz8rGv8oGmxIYFWzAyd JoKSSg== X-Google-Smtp-Source: AGHT+IGFnKTrwnNVlNBi0pRWyTxWgXXUXcq6GWbamGHw2JM+Z0reYdmXkftL6UEPzYa40jTFM969+w== X-Received: by 2002:a17:903:2a8b:b0:215:ba2b:cd51 with SMTP id d9443c01a7336-21614d3ace3mr179685945ad.15.1733724956327; Sun, 08 Dec 2024 22:15:56 -0800 (PST) Received: from localhost ([58.29.143.236]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-215f8f0913bsm65491515ad.185.2024.12.08.22.15.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 08 Dec 2024 22:15:56 -0800 (PST) From: Changwoo Min X-Google-Original-From: Changwoo Min To: tj@kernel.org, void@manifault.com, mingo@redhat.com, peterz@infradead.org Cc: changwoo@igalia.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: [PATCH v4 3/6] sched_ext: Manage the validity of scx_rq_clock Date: Mon, 9 Dec 2024 15:15:28 +0900 Message-ID: <20241209061531.257531-4-changwoo@igalia.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241209061531.257531-1-changwoo@igalia.com> References: <20241209061531.257531-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" An rq clock becomes valid when it is updated using update_rq_clock() and invalidated when the rq is unlocked using rq_unpin_lock(). Let's suppose the following timeline in the scheduler core: T1. rq_lock(rq) T2. update_rq_clock(rq) T3. a sched_ext BPF operation T4. rq_unlock(rq) T5. a sched_ext BPF operation T6. rq_lock(rq) T7. update_rq_clock(rq) For [T2, T4), we consider that rq clock is valid (SCX_RQ_CLK_VALID is set), so scx_bpf_now_ns() calls during [T2, T4) (including T3) will return the rq clock updated at T2. For duration [T4, T7), when a BPF scheduler can still call scx_bpf_now_ns() (T5), we consider the rq clock is invalid (SCX_RQ_CLK_VALID is unset at T4). So when calling scx_bpf_now_ns() at T5, we will return a fresh clock value by calling sched_clock() internally. One example of calling scx_bpf_now_ns(), when the rq clock is invalid (like T5), is in scx_central [1]. The scx_central scheduler uses a BPF timer for preemptive scheduling. In every msec, the timer callback checks if the currently running tasks exceed their timeslice. At the beginning of the BPF timer callback (central_timerfn in scx_central.bpf.c), scx_central gets the current time. When the BPF timer callback runs, the rq clock could be invalid, the same as T5. In this case, scx_bpf_now_ns() returns a fresh clock value rather than returning the old one (T2). [1] https://github.com/sched-ext/scx/blob/main/scheds/c/scx_central.bpf.c Signed-off-by: Changwoo Min Acked-by: Andrea Righi --- kernel/sched/core.c | 6 +++++- kernel/sched/sched.h | 2 +- 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 95e40895a519..ab8015c8cab4 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -789,6 +789,7 @@ static void update_rq_clock_task(struct rq *rq, s64 del= ta) void update_rq_clock(struct rq *rq) { s64 delta; + u64 clock; =20 lockdep_assert_rq_held(rq); =20 @@ -800,11 +801,14 @@ void update_rq_clock(struct rq *rq) SCHED_WARN_ON(rq->clock_update_flags & RQCF_UPDATED); rq->clock_update_flags |=3D RQCF_UPDATED; #endif + clock =3D sched_clock_cpu(cpu_of(rq)); + scx_rq_clock_update(rq, clock); =20 - delta =3D sched_clock_cpu(cpu_of(rq)) - rq->clock; + delta =3D clock - rq->clock; if (delta < 0) return; rq->clock +=3D delta; + update_rq_clock_task(rq, delta); } =20 diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 7e71d8685fcc..b9d7c638ec60 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -1781,7 +1781,7 @@ static inline void rq_unpin_lock(struct rq *rq, struc= t rq_flags *rf) if (rq->clock_update_flags > RQCF_ACT_SKIP) rf->clock_update_flags =3D RQCF_UPDATED; #endif - + scx_rq_clock_stale(rq); lockdep_unpin_lock(__rq_lockp(rq), rf->cookie); } =20 --=20 2.47.1 From nobody Wed Dec 17 17:53:15 2025 Received: from mail-pl1-f181.google.com (mail-pl1-f181.google.com [209.85.214.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D61CB2147F8 for ; Mon, 9 Dec 2024 06:16:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.181 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733724962; cv=none; b=ioPn3hoV9EDd+i+QDl60DAc5Y8+HHK8uXTGQ/6UXPDOPqDmDPxmV4ZI/W7VB37StDfc9+tVIm1cs3oiaiM9NuzUWZgUOUkzO76WlCrYUE6kto/BP/M3iNPh5h7y9SbdFig8e44RqmEJHW2GCi4u/31uCsNdKuH2mqXjAlwFIC0M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733724962; c=relaxed/simple; bh=PN6Iw3Dxk3S0kVn76rQkvmuHIz9L20xles6S0U2u3jg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BvOFbUKx3Ynq1p5PGjc/EHL+5DUnFxYgIPRyur0ulKtIxmac+cOv/1k7s4/edKDG+Pa3bnYjDLTKixbJQ0O13wgAfzIbbSXCozF2xZ5O2MTJwttUhl45gKfCVdTWV1ls5Hnujrm1KQFYhminyddejJXNB0Q3f7n+JLvsudblmFE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=O2SP04cd; arc=none smtp.client-ip=209.85.214.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="O2SP04cd" Received: by mail-pl1-f181.google.com with SMTP id d9443c01a7336-2165448243fso4665745ad.1 for ; Sun, 08 Dec 2024 22:16:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733724960; x=1734329760; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=fEf8zUxGouueZslZOLUKni+ZjYVAncqMk9D3zvb5k7I=; b=O2SP04cdZNS/9R18CQaN5qVNYfa8nKWnb9fDQsIS0VK3eRdDTxEz+LBhrqw6mwO66k hecL/GDfEnTnJTuaxVb4Te+l1K3ZcZKE3wGJC0KU6xkLm2e/XijJLOKt9PklypY5K3v9 /SCm3vXY2AfhbyNgS1al9PQUv+ephNhqu+BF+GKTEq7a06Gh4QJzN8gyFp2cGNhte1lp +UdOsGjemOLVTRcampWi5DHmHTAH/15XBTVp8Vgnarnl64lc54toD/LB4lha8iwhsYdK 9MncT/PXqoptenJUAhCFI3m7wsHcLc+qvRiI2OXFcpUQJRoA2Pk/K+N/+kT1H/EsLx7I VHeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733724960; x=1734329760; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fEf8zUxGouueZslZOLUKni+ZjYVAncqMk9D3zvb5k7I=; b=lA2wk0UVIEqfzVa7sAiUduXzlirAEmWuKqgRNBzVUS9dPPx8SCW32+I1BUPlMtJiGk 1Jg+XP4fQJK+gDkdYnpzEIi1XygPqwYBVFevJ+6lqWyvhO5xDJUty11QVcuFxCAPWBil NZiT9duYWX5IuNGI7ArjfS9LuY5LRCLAeYAQMGtO5YF02WyKkXUl9xloeXqU9oqT+fVZ 44TpGwQbuoubq9/94YuOaZzgwE5YhiAxY6gUuxO2jKpVOka3vx0MUjKztoQOjjsLcH+S 6cDyilQ1daCD+wVUlzoBas07kRc399W+XjBgqqR/eWkfF0GB4v/ViXX5SMkPorT7Z3jE pKYw== X-Forwarded-Encrypted: i=1; AJvYcCXM8kNva8Vnx9peZQNNkINdsk7oIy3U2dyQ6vmHMHhlUtTT6crtO+OCCTbtdLV+6440qLK5R5JlCU4xmDE=@vger.kernel.org X-Gm-Message-State: AOJu0Yx+fyMuBiv9tI3oebb17w+BCUpuGe6AKIJB1qzoxZP/kIq52x51 oXh0sds7Dc0ghzowuGJIebrKOYXPEfuSgegF95ujDpsTGryVblfkgsoFZQ== X-Gm-Gg: ASbGncvxzbY10NURCJ2pjUOlZGIZ5R2o3ZFM1SXPjtZMqYaatm8ignALVvR+mlcQW3D NGjdwGoMvtIvejhIvZaBJQchmSRM+RwZXiAR3wmR4xY9o2i/RvEcPsiUxa6FJq5nTMNLKTHfGcM CsZcV7aMMghZo1kHmQNgHXJPGjjWNUrCXeA+w8hU98Be7qpOflxCyGSAV69xxJNA0sMBZFOeMrn 1053cHsHUp+N+5CexmkAHNKraTDVEIaeBeUNiv4awF9mWfFaqPdpPEiH3DEAJjzqA4yc2Kyt4Ay yGsAvQ== X-Google-Smtp-Source: AGHT+IG64B1UnDqQhghjhSxh2tq6nF1Qq6Q36HyIm9VCwWqhe+o/79OCTs4qHrK3v0UWINjE8lBzBg== X-Received: by 2002:a17:902:d2c1:b0:216:4676:dfaf with SMTP id d9443c01a7336-2164676e34amr55038175ad.34.1733724959859; Sun, 08 Dec 2024 22:15:59 -0800 (PST) Received: from localhost ([58.29.143.236]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2162e22d4besm31147575ad.230.2024.12.08.22.15.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 08 Dec 2024 22:15:59 -0800 (PST) From: Changwoo Min X-Google-Original-From: Changwoo Min To: tj@kernel.org, void@manifault.com, mingo@redhat.com, peterz@infradead.org Cc: changwoo@igalia.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: [PATCH v4 4/6] sched_ext: Implement scx_bpf_now_ns() Date: Mon, 9 Dec 2024 15:15:29 +0900 Message-ID: <20241209061531.257531-5-changwoo@igalia.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241209061531.257531-1-changwoo@igalia.com> References: <20241209061531.257531-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Returns a high-performance monotonically non-decreasing clock for the curre= nt CPU. The clock returned is in nanoseconds. It provides the following properties: 1) High performance: Many BPF schedulers call bpf_ktime_get_ns() frequently to account for execution time and track tasks' runtime properties. Unfortunately, in some hardware platforms, bpf_ktime_get_ns() -- which eventually reads a hardware timestamp counter -- is neither performant nor scalable. scx_bpf_now_ns() aims to provide a high-performance clock by using the rq clock in the scheduler core whenever possible. 2) High enough resolution for the BPF scheduler use cases: In most BPF scheduler use cases, the required clock resolution is lower than the most accurate hardware clock (e.g., rdtsc in x86). scx_bpf_now_ns() basically uses the rq clock in the scheduler core whenever it is valid. It considers that the rq clock is valid from the time the rq clock is updated (update_rq_clock) until the rq is unlocked (rq_unpin_lock). 3) Monotonically non-decreasing clock for the same CPU: scx_bpf_now_ns() guarantees the clock never goes backward when comparing them in the same CPU. On the other hand, when comparing clocks in different CPUs, there is no such guarantee -- the clock can go backward. It provides a monotonically *non-decreasing* clock so that it would provide the same clock values in two different scx_bpf_now_ns() calls in the same CPU during the same period of when the rq clock is valid. Signed-off-by: Changwoo Min Acked-by: Andrea Righi --- kernel/sched/ext.c | 73 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 73 insertions(+) diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index 71342f3719c1..f0476d5dd6f5 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -7601,6 +7601,78 @@ __bpf_kfunc struct cgroup *scx_bpf_task_cgroup(struc= t task_struct *p) } #endif =20 +/** + * scx_bpf_now_ns - Returns a high-performance monotonically non-decreasing + * clock for the current CPU. The clock returned is in nanoseconds. + * + * It provides the following properties: + * + * 1) High performance: Many BPF schedulers call bpf_ktime_get_ns() freque= ntly + * to account for execution time and track tasks' runtime properties. + * Unfortunately, in some hardware platforms, bpf_ktime_get_ns() -- which + * eventually reads a hardware timestamp counter -- is neither performant= nor + * scalable. scx_bpf_now_ns() aims to provide a high-performance clock by + * using the rq clock in the scheduler core whenever possible. + * + * 2) High enough resolution for the BPF scheduler use cases: In most BPF + * scheduler use cases, the required clock resolution is lower than the m= ost + * accurate hardware clock (e.g., rdtsc in x86). scx_bpf_now_ns() basical= ly + * uses the rq clock in the scheduler core whenever it is valid. It consi= ders + * that the rq clock is valid from the time the rq clock is updated + * (update_rq_clock) until the rq is unlocked (rq_unpin_lock). + * + * 3) Monotonically non-decreasing clock for the same CPU: scx_bpf_now_ns() + * guarantees the clock never goes backward when comparing them in the sa= me + * CPU. On the other hand, when comparing clocks in different CPUs, there + * is no such guarantee -- the clock can go backward. It provides a + * monotonically *non-decreasing* clock so that it would provide the same + * clock values in two different scx_bpf_now_ns() calls in the same CPU + * during the same period of when the rq clock is valid. + */ +__bpf_kfunc u64 scx_bpf_now_ns(void) +{ + struct rq *rq; + u64 clock; + + preempt_disable(); + + /* + * If the rq clock is valid, use the cached rq clock + * whenever the clock does not go backward. + */ + rq =3D this_rq(); + clock =3D rq->scx.clock; + + if (!(rq->scx.flags & SCX_RQ_CLK_VALID) || + (rq->scx.prev_clock >=3D clock)) { + /* + * If the rq clock is invalid or goes backward, + * start a new rq clock period with a fresh sched_clock_cpu(). + * + * The cached rq clock can go backward because there is a + * race with a timer interrupt. Suppose that a timer interrupt + * occurred while running scx_bpf_now_ns() *after* reading the + * rq clock and *before* comparing the if condition. The timer + * interrupt will eventually call a BPF scheduler's ops.tick(), + * and the BPF scheduler can call scx_bpf_now_ns(). Since the + * scheduler core updates the rq clock before calling + * ops.tick(), the scx_bpf_now_ns() call will get the fresh + * clock. After handling the timer interrupt, the interrupted + * scx_bpf_now_ns() will be resumed, so the if condition will + * be compared. In this case, the clock, which was read before + * the timer interrupt, will be the same as rq->scx.prev_clock. + * When such a case is detected, start a new rq clock period + * with a fresh sched_clock_cpu(). + */ + clock =3D sched_clock_cpu(cpu_of(rq)); + scx_rq_clock_update(rq, clock); + } + + preempt_enable(); + + return clock; +} + __bpf_kfunc_end_defs(); =20 BTF_KFUNCS_START(scx_kfunc_ids_any) @@ -7632,6 +7704,7 @@ BTF_ID_FLAGS(func, scx_bpf_cpu_rq) #ifdef CONFIG_CGROUP_SCHED BTF_ID_FLAGS(func, scx_bpf_task_cgroup, KF_RCU | KF_ACQUIRE) #endif +BTF_ID_FLAGS(func, scx_bpf_now_ns) BTF_KFUNCS_END(scx_kfunc_ids_any) =20 static const struct btf_kfunc_id_set scx_kfunc_set_any =3D { --=20 2.47.1 From nobody Wed Dec 17 17:53:15 2025 Received: from mail-pf1-f176.google.com (mail-pf1-f176.google.com [209.85.210.176]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7C9C5214A89 for ; Mon, 9 Dec 2024 06:16:04 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.176 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733724965; cv=none; b=qbr7Fg9ieXCHYn9yf3j8xv48F2lw7UzGBrQ47ONweDFLuv7zeZlBCREdkCH+0FmR0VVyfvXSqPDAhpGqo8KeAKMmYTYZ6XMlNG80Ks7xEgdeg/szA9Ysyb86ghL1yKxdMfvklybrs/MXVNAesUc0WpQGe2M6jfnyKy4NDpXoU00= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733724965; c=relaxed/simple; bh=3MTZN/eJy+347t6uxFM4RqQzva5iqdXS/ZcOt/1Xa4o=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=G9z3IeSOSpcc7OiOeeA2LN1BDbRZKl16VylQKAiU1d/9aFNIO5QiFrvVcheZq5gy+V7niOIQqYH9EROg7LzEHjjva+TuxxnCDk4YKcvbzwD98UbGa6j+7VUCWgCPxaGeJKvBtVQl2a+DIuJEUSspv6+cBp9uwjQtuXQtYFy8LLI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=Jynot8yZ; arc=none smtp.client-ip=209.85.210.176 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="Jynot8yZ" Received: by mail-pf1-f176.google.com with SMTP id d2e1a72fcca58-725ee27e905so580044b3a.2 for ; Sun, 08 Dec 2024 22:16:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733724964; x=1734329764; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ogSxJ0qgSRVGfWLd2NgyVRKG8hMtwjWInerWH6gERr0=; b=Jynot8yZqm9O1/pxbaSTXZ9ga4KV3Bc4z1QJTLN/eKxn9dCUd/YRLqrNzq2gBst9XA BCzYRyDn6+N69/CnBM5ugKCViRThQH+W4UKVbv+y+PhX1PElk/WWBnFErwwfT7rHx+sh Tmyo57J4rxtX+fOF9un+wjL6P3ZFApUCIkPTwKXdX4I50+S2eWGKGQsqQpm7qtmO1DH0 o8ZQJbMwBKo9iA83yHlbRE7hm5GSNyHPybRz78rMFyf/ECgLjrZyKRBIjWrhlcgEYdGf y8GVLZ6ELoKf0KZhTcDIO4cFumRXm/7Waek68WzcaVjKSjHHwS3N2SvjAyO+Lmh8Vtht NOPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733724964; x=1734329764; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ogSxJ0qgSRVGfWLd2NgyVRKG8hMtwjWInerWH6gERr0=; b=uQzZdQgQN9nJXUF5VamW79Z6HuLnZk3kbu9bro0U5r05I31y4IkJamPul87l8QaHNR y4GnEK9dD1GQTOwnrGmhG4HLdLtSnjMczmr/kimOhywbqO0baoZwnnVkjoHvIT5itdgn 5mXlaFzCGJeEHNlyC4q0e0ee5WGnE/a1hbBk+/3I3liT7r9HK3FCJGmdPqQxXj1engI8 SRVC0ykeDqCiZBEdeiW4fvj+zKf2GEdncJ/te02fW64lujF7yb1wR6EuJn6Mr8lsQF2p l+kTlFipLkEJDiasEwmYmf1JduXYvABpCFnwpEC+7Z8gkcBMBtDn/tke+LmllX/yCkB/ 1DgA== X-Forwarded-Encrypted: i=1; AJvYcCUHQvp+Jh2C0kdBb2ku7+5gRS1UKeKbXuxarsfKkByBvAhF2gzt5TdsZr3MdHTEU89Gj+ffR9e7HSIl6DI=@vger.kernel.org X-Gm-Message-State: AOJu0Yx6CWzQkW2lzqbTtdZ7SZoIlmvtMhvrf4jeqdzEyJd7y4aPtEze UdaCEkxSfzvebDRw0F8JaXtv1FBG1FXOHQSmd4u1k5+aE2pjjLfp X-Gm-Gg: ASbGncscIic77g4A6mIslEyGME6An4vtquIO1CmYKTURdCnVhslufAHW56gpWD4bY/t pkWXK9bzOAoU/9zhRlDwuKdJogpGoe6vc1cDtVcohRghsatnLe1jrOpeuffFDxczjr3Z3RWR1VI JLCA9YYecIoeCuO3nTZV7ZSPgPWhpZZtUYDSV+nC5/17R+2urck4mOSW29UTljD/iJdnb09v6LT ZHnPUoOQLS80npYyjvyP1THPy2ZQrf7QWhE4WtTKQQAlo2MeNgsmJiIlN5swjNTb50odHsmG5Pa j25lmQ== X-Google-Smtp-Source: AGHT+IGkMLw/XguT2MSh4yw8vFbRBFPdO4mqrCvX61feUs351mK8m1b8xd1vSJnSRFsU2WFoYHWmLw== X-Received: by 2002:a05:6a00:4b50:b0:725:df1a:282 with SMTP id d2e1a72fcca58-725df1a17c6mr7207637b3a.10.1733724963584; Sun, 08 Dec 2024 22:16:03 -0800 (PST) Received: from localhost ([58.29.143.236]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-725e6e024e4sm1824259b3a.153.2024.12.08.22.16.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 08 Dec 2024 22:16:03 -0800 (PST) From: Changwoo Min X-Google-Original-From: Changwoo Min To: tj@kernel.org, void@manifault.com, mingo@redhat.com, peterz@infradead.org Cc: changwoo@igalia.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: [PATCH v4 5/6] sched_ext: Add scx_bpf_now_ns() for BPF scheduler Date: Mon, 9 Dec 2024 15:15:30 +0900 Message-ID: <20241209061531.257531-6-changwoo@igalia.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241209061531.257531-1-changwoo@igalia.com> References: <20241209061531.257531-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" scx_bpf_now_ns() is added to the header files so the BPF scheduler can use it. Signed-off-by: Changwoo Min Acked-by: Andrea Righi --- tools/sched_ext/include/scx/common.bpf.h | 1 + tools/sched_ext/include/scx/compat.bpf.h | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/tools/sched_ext/include/scx/common.bpf.h b/tools/sched_ext/inc= lude/scx/common.bpf.h index 625f5b046776..d8fd58d8c032 100644 --- a/tools/sched_ext/include/scx/common.bpf.h +++ b/tools/sched_ext/include/scx/common.bpf.h @@ -72,6 +72,7 @@ bool scx_bpf_task_running(const struct task_struct *p) __= ksym; s32 scx_bpf_task_cpu(const struct task_struct *p) __ksym; struct rq *scx_bpf_cpu_rq(s32 cpu) __ksym; struct cgroup *scx_bpf_task_cgroup(struct task_struct *p) __ksym __weak; +u64 scx_bpf_now_ns(void) __ksym __weak; =20 /* * Use the following as @it__iter when calling scx_bpf_dsq_move[_vtime]() = from diff --git a/tools/sched_ext/include/scx/compat.bpf.h b/tools/sched_ext/inc= lude/scx/compat.bpf.h index d56520100a26..2f16f30ae741 100644 --- a/tools/sched_ext/include/scx/compat.bpf.h +++ b/tools/sched_ext/include/scx/compat.bpf.h @@ -125,6 +125,11 @@ bool scx_bpf_dispatch_vtime_from_dsq___compat(struct b= pf_iter_scx_dsq *it__iter, false; \ }) =20 +#define scx_bpf_now_ns() \ + (bpf_ksym_exists(scx_bpf_now_ns) ? \ + scx_bpf_now_ns() : \ + bpf_ktime_get_ns()) + /* * Define sched_ext_ops. This may be expanded to define multiple variants = for * backward compatibility. See compat.h::SCX_OPS_LOAD/ATTACH(). --=20 2.47.1 From nobody Wed Dec 17 17:53:15 2025 Received: from mail-pg1-f180.google.com (mail-pg1-f180.google.com [209.85.215.180]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E44F221480E for ; Mon, 9 Dec 2024 06:16:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.215.180 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733724970; cv=none; b=fse8/OpNN8FDaNYCykwT1Ow5fcCWWcyXip5qPLZqJkERbDvpme47jlbXzTnXHRx9oBIdQ46pQuPwlmj7bMkSgxvTj2QyDGUvzycx/zfa4Q4O02j79Q5lgxy27Q37OgUDUtz+KZTNL+HFhM5K8PQRvepmoa63S+es9aOf6+6KFvQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733724970; c=relaxed/simple; bh=8xKUl/PksIoq2Rbneib2leLcMgymowSckqA99qxx/g4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=l1ZXV83Zo1AFG3CP1zEZh9z/azwXTiExnI/9CEa2+gTKJk/6aTM1qcCqjBABugHFi/KYkTdXsSq0GLWjFnzCAfadx3fwYXi/oGUggqlV6nU9D9iENlr550BUpN7unLYOF3HrUPyGcw7uPrCDuXL2ThqQUB10WoCCTOWugR3H6PM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=OtDPLb85; arc=none smtp.client-ip=209.85.215.180 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="OtDPLb85" Received: by mail-pg1-f180.google.com with SMTP id 41be03b00d2f7-7fd35b301bdso901378a12.2 for ; Sun, 08 Dec 2024 22:16:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1733724968; x=1734329768; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=47y2cVyd1kuIZD8NQ+KQkmd+50HMhaVqGUhTFqkPRtg=; b=OtDPLb85FZ6FH8HT5ZfK0KBpXMYOBrR8i2/p3PIFc+XkRQ841RQIT9ePyzDJaK4plB AtILbI8JdwgcoyYpLog1WWjuM8ZdG1skOG53xhMDOBI1a3rSIgW4P/j5vFfyAvOLX3FN /hNvNGBu4Jq1FNMJNc5ZvE1CyCfGcmP8ypiaA8LMdMZ8jwqfRPcyuXhITpakepPYG3LS JZ7XC8zBvdkOB9Sxoo+XalB1lqwN2oxjruEk3FvwYM+rv2il+QvwgBZq/98uVfu2+obH wGaetgeOy+XkDv9wRhDT5TN4g8Fqzywt4pOPcRl0AUdTqVUykdSBJX1P0khbZfAb3NbR 2plg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733724968; x=1734329768; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=47y2cVyd1kuIZD8NQ+KQkmd+50HMhaVqGUhTFqkPRtg=; b=bYkp6Uy0q2ZrkvkS65NFAbqQ9z/gDav6hbH7nF+zYIdBM0R0QVaoOBMOD2mW7XS3ng tYeec2AYnpj8fZpqCdUL3hYjGEZlW/bGWEP6wTTe0+q5WAWGAypFhip6HYVEdYnMYlZz DKehC3NBNXtMRkcgjiCXTiavdPlHuF6xr6MFNFmSPYcScNaiM+x2whxb1EEZPGl+4hn6 uI11hZRlwAS1lyNUNFlnuWvrLqAm17hXYmwyI8cO/ZB7LkWBUg0E3KXNx/7555x4xsSn qqiGKXinLRIfLxS3aCXBKRJAMyFUqd6pZhayz+TpKH1YyzgP1+6MCXeayr5waDH1z7sZ IBcg== X-Forwarded-Encrypted: i=1; AJvYcCVTbcV7qMIXEoEhIBY2C9rBwSratFhVRNN8EbNtlCGSs78q/Xt6TqgLeX2IE84D0Pfj7wmJD8jL0zkzDzc=@vger.kernel.org X-Gm-Message-State: AOJu0Ywt+nqUqfF+PhPy6sQ5nmDs6no2PCGyk+Hb5Yu4yQQvaYfypJh7 vAHL9Gz9sOa80ft1U0f0RvLOuJDhpcUeRAyTcDx3rP4ULHGHTJjR X-Gm-Gg: ASbGncu+QERz2Jf96wdKKCir836nOJKFmCxzdpVgslWfZpZH8lci4IfTdLRBOW9MaDI QoVvER7nDBP5J2LxFpCevNgBzlSGi9sffW5ztQiVItMLO1AW0SDCQPl/aVSTaoFXa3OoE3c/j+M W5Zx6RCyApe78Sd7oyHa8eLHw3EhKdpGAb+XmCP747Ywt98ILqB6k1Cdn+tOD7IQEjQ7sIQ+pL8 PiuUqJ89lHzee9Eqpe5o1uA5CrRm1xQT9AVSyGOk0CcWFUCc+asnZ5DQz0lDPzOr5mkcbm5QucW 9EyGmg== X-Google-Smtp-Source: AGHT+IGR6SDOInTOzs1ZgZz2jFp+v8LqNuzUwHxPV0vg7nvptPwzbdk6ljguSlV/VuELR3U+zEvc0w== X-Received: by 2002:a17:903:2344:b0:20c:6399:d637 with SMTP id d9443c01a7336-21614da3f2amr202143245ad.40.1733724968203; Sun, 08 Dec 2024 22:16:08 -0800 (PST) Received: from localhost ([58.29.143.236]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2162381c6basm39572265ad.121.2024.12.08.22.16.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 08 Dec 2024 22:16:07 -0800 (PST) From: Changwoo Min X-Google-Original-From: Changwoo Min To: tj@kernel.org, void@manifault.com, mingo@redhat.com, peterz@infradead.org Cc: changwoo@igalia.com, kernel-dev@igalia.com, linux-kernel@vger.kernel.org Subject: [PATCH v4 6/6] sched_ext: Replace bpf_ktime_get_ns() to scx_bpf_now_ns() Date: Mon, 9 Dec 2024 15:15:31 +0900 Message-ID: <20241209061531.257531-7-changwoo@igalia.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241209061531.257531-1-changwoo@igalia.com> References: <20241209061531.257531-1-changwoo@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" In the BPF schedulers that use bpf_ktime_get_ns() -- scx_central and scx_flatcg, replace bpf_ktime_get_ns() calls to scx_bpf_now_ns(). Signed-off-by: Changwoo Min Acked-by: Andrea Righi --- tools/sched_ext/scx_central.bpf.c | 4 ++-- tools/sched_ext/scx_flatcg.bpf.c | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/tools/sched_ext/scx_central.bpf.c b/tools/sched_ext/scx_centra= l.bpf.c index e6fad6211f6c..e64b7a429126 100644 --- a/tools/sched_ext/scx_central.bpf.c +++ b/tools/sched_ext/scx_central.bpf.c @@ -245,7 +245,7 @@ void BPF_STRUCT_OPS(central_running, struct task_struct= *p) s32 cpu =3D scx_bpf_task_cpu(p); u64 *started_at =3D ARRAY_ELEM_PTR(cpu_started_at, cpu, nr_cpu_ids); if (started_at) - *started_at =3D bpf_ktime_get_ns() ?: 1; /* 0 indicates idle */ + *started_at =3D scx_bpf_now_ns() ?: 1; /* 0 indicates idle */ } =20 void BPF_STRUCT_OPS(central_stopping, struct task_struct *p, bool runnable) @@ -258,7 +258,7 @@ void BPF_STRUCT_OPS(central_stopping, struct task_struc= t *p, bool runnable) =20 static int central_timerfn(void *map, int *key, struct bpf_timer *timer) { - u64 now =3D bpf_ktime_get_ns(); + u64 now =3D scx_bpf_now_ns(); u64 nr_to_kick =3D nr_queued; s32 i, curr_cpu; =20 diff --git a/tools/sched_ext/scx_flatcg.bpf.c b/tools/sched_ext/scx_flatcg.= bpf.c index 4e3afcd260bf..568581b15c44 100644 --- a/tools/sched_ext/scx_flatcg.bpf.c +++ b/tools/sched_ext/scx_flatcg.bpf.c @@ -734,7 +734,7 @@ void BPF_STRUCT_OPS(fcg_dispatch, s32 cpu, struct task_= struct *prev) struct fcg_cpu_ctx *cpuc; struct fcg_cgrp_ctx *cgc; struct cgroup *cgrp; - u64 now =3D bpf_ktime_get_ns(); + u64 now =3D scx_bpf_now_ns(); bool picked_next =3D false; =20 cpuc =3D find_cpu_ctx(); --=20 2.47.1