From nobody Sun Oct 5 03:38:30 2025 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8F07C38DD8 for ; Fri, 8 Aug 2025 20:03:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754683411; cv=none; b=py7HSlzD4V46GbKpnvwJjgHOoDwlYxK05u7ixAiA8TvO5RQLEhlLl8jvMajF9TnGHChYrF5KwZ+SFW+/ojV/Eyyo59F4vU4CglXs4vyI6S1T2joxvyQdCCeYjQaHm/FRojw/z5c+22Bdcueis3/9ihe5cVxZDa3WVmgSYI3iC1w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754683411; c=relaxed/simple; bh=QnGicouqjxX4375qN1r/AZitQYdLQdoftWkVyE7Gdxg=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=khdGgiv67U1An0tKMAkWgTXSLNtPPwnJY/8E72LhTfHL/EP3PO1nrHqyXUYh2dlQcxG1CJ1v/J7LhPKW26EOKyeJfXPsyO/7qFD2+K0ZcZ4HettgSzA3cYK6p5bzEMB9y2s4etFlYArgmiM/SxzAV+/PCtnrXgf/jH6wXGlTp4M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--jpiecuch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=4tpGnif3; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--jpiecuch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="4tpGnif3" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-459f53d29d2so5746305e9.1 for ; Fri, 08 Aug 2025 13:03:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754683408; x=1755288208; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=VQQTdrUE/FoLC6Ybw0i7vf2hpt8f5CSWzafPnqwUGXA=; b=4tpGnif3AKhxpp/gc7iWk0fsGMXb6mgdUTQ4jRonfKXsZz+k4VsY2WUq4FpTsdrCYq L3v45A3XEdKWOpagSPLvMrM11Y6Dh6SKxVvbsS4jLfUdcdUtu6oYkfyyENwLa0i2GvMz VniIoeK6Ryv4v8Ehz/cnjkZId7IXPC0dSHk0uD+2tINlN6na80Fss9TCoc+4tsuH+K/Q hhMtxNk5eI92Q71aHPx778IWRqPqN8qjX9i+yJ9rezs1zr/pJQrAIVplVczmUWV//URc AWNGHtakcOYznX8ifRlGCyPti4bjEg2CesO/S26ivWM69Az7MNKFUajnTXZAmOTaIzFh C8jA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754683408; x=1755288208; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=VQQTdrUE/FoLC6Ybw0i7vf2hpt8f5CSWzafPnqwUGXA=; b=dbDAJpMxn4JBBs2BnAB5qRWp1EYzVOc/u9XUddnTfqJX0+ESIGrGLhN7for67jiw/c z3S25nJHgltop9LATVQvxJ9yqpvLigE+QjM1yH25GWsP/kjYEQZBTRHfOGR6ATDkhs/S BO2pnv3tgI6CbkISvywkF7KTShBLmTDuP/kOqOR1XRSQ39WxYsyT1NYLgqwVpC5GFuyH x9H79nsDtcL7IhKzI4ONfnuavHlLk5BIgJjw6Y85l+qRZcBk4LXCfNH3y1ARjG+bAM57 zbjy1DHTHHm5aI5HsvVxTuKrSD/UyoaqjwJOWVfpWH8pDubZchPSuDIuDdyxgVMGI19i vGow== X-Gm-Message-State: AOJu0YyulGFWWPeoXxActHoPh3ZID7w08dsgO2ccwc+beS2XfPL7Gv1i qqHMxYhHKn/y3tdMUs3jEtYupnmPkw661oT6BZ3W0aGQO8PsuAzjdP8fU79EJ92KscjZGNJM2Te WXtJ7NBEz26BaSA== X-Google-Smtp-Source: AGHT+IEmV7GwfmoE8uf8gbaIXpmsbBca39VQZvtZ0R47yTjMkr0V4ZnD2QV24JHQM5FmkQ95r66GJNYWegaPNA== X-Received: from wmbhe3.prod.google.com ([2002:a05:600c:5403:b0:459:10dd:2e55]) (user=jpiecuch job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:4fcb:b0:459:db7b:988e with SMTP id 5b1f17b1804b1-459f4f3e883mr34390045e9.13.1754683407915; Fri, 08 Aug 2025 13:03:27 -0700 (PDT) Date: Fri, 8 Aug 2025 20:02:48 +0000 In-Reply-To: <20250808200250.2016584-1-jpiecuch@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250808200250.2016584-1-jpiecuch@google.com> X-Mailer: git-send-email 2.51.0.rc0.155.g4a0f42376b-goog Message-ID: <20250808200250.2016584-2-jpiecuch@google.com> Subject: [RFC PATCH 1/3] sched: add bool return value to sched_class::yield_task() From: Kuba Piecuch To: peterz@infradead.org, mingo@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, joshdon@google.com Cc: linux-kernel@vger.kernel.org, Kuba Piecuch Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The return value controls whether do_sched_yield() ends up calling schedule(). This can be used e.g. as an optimization when we're certain that the yield won't result in another task being scheduled. This patch does not change the current behavior, i.e. every call to do_sched_yield() will still go through to schedule(). Signed-off-by: Kuba Piecuch --- kernel/sched/deadline.c | 4 +++- kernel/sched/ext.c | 4 +++- kernel/sched/fair.c | 6 ++++-- kernel/sched/rt.c | 3 ++- kernel/sched/sched.h | 2 +- kernel/sched/stop_task.c | 2 +- kernel/sched/syscalls.c | 9 ++++++++- 7 files changed, 22 insertions(+), 8 deletions(-) diff --git a/kernel/sched/deadline.c b/kernel/sched/deadline.c index 89019a1408264..d8dcb73bd3433 100644 --- a/kernel/sched/deadline.c +++ b/kernel/sched/deadline.c @@ -2194,7 +2194,7 @@ static bool dequeue_task_dl(struct rq *rq, struct tas= k_struct *p, int flags) * yield_task_dl will indicate that some spare budget * is available for other task instances to use it. */ -static void yield_task_dl(struct rq *rq) +static bool yield_task_dl(struct rq *rq) { /* * We make the task go to sleep until its current deadline by @@ -2212,6 +2212,8 @@ static void yield_task_dl(struct rq *rq) * and double the fastpath cost. */ rq_clock_skip_update(rq); + + return true; } =20 #ifdef CONFIG_SMP diff --git a/kernel/sched/ext.c b/kernel/sched/ext.c index 7dd5cbcb7a069..dd0a0b6b7aa05 100644 --- a/kernel/sched/ext.c +++ b/kernel/sched/ext.c @@ -2502,7 +2502,7 @@ static bool dequeue_task_scx(struct rq *rq, struct ta= sk_struct *p, int deq_flags return true; } =20 -static void yield_task_scx(struct rq *rq) +static bool yield_task_scx(struct rq *rq) { struct scx_sched *sch =3D scx_root; struct task_struct *p =3D rq->curr; @@ -2511,6 +2511,8 @@ static void yield_task_scx(struct rq *rq) SCX_CALL_OP_2TASKS_RET(sch, SCX_KF_REST, yield, rq, p, NULL); else p->scx.slice =3D 0; + + return true; } =20 static bool yield_to_task_scx(struct rq *rq, struct task_struct *to) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 7a14da5396fb2..c06a2f8290822 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -9010,7 +9010,7 @@ static void put_prev_task_fair(struct rq *rq, struct = task_struct *prev, struct t /* * sched_yield() is very simple */ -static void yield_task_fair(struct rq *rq) +static bool yield_task_fair(struct rq *rq) { struct task_struct *curr =3D rq->curr; struct cfs_rq *cfs_rq =3D task_cfs_rq(curr); @@ -9020,7 +9020,7 @@ static void yield_task_fair(struct rq *rq) * Are we the only task in the tree? */ if (unlikely(rq->nr_running =3D=3D 1)) - return; + return true; =20 clear_buddies(cfs_rq, se); =20 @@ -9037,6 +9037,8 @@ static void yield_task_fair(struct rq *rq) rq_clock_skip_update(rq); =20 se->deadline +=3D calc_delta_fair(se->slice, se); + + return true; } =20 static bool yield_to_task_fair(struct rq *rq, struct task_struct *p) diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c index e40422c370335..1fa535457cc40 100644 --- a/kernel/sched/rt.c +++ b/kernel/sched/rt.c @@ -1533,9 +1533,10 @@ static void requeue_task_rt(struct rq *rq, struct ta= sk_struct *p, int head) } } =20 -static void yield_task_rt(struct rq *rq) +static bool yield_task_rt(struct rq *rq) { requeue_task_rt(rq, rq->curr, 0); + return true; } =20 #ifdef CONFIG_SMP diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 83e3aa9171429..8b2cd54a09942 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -2395,7 +2395,7 @@ struct sched_class { =20 void (*enqueue_task) (struct rq *rq, struct task_struct *p, int flags); bool (*dequeue_task) (struct rq *rq, struct task_struct *p, int flags); - void (*yield_task) (struct rq *rq); + bool (*yield_task) (struct rq *rq); bool (*yield_to_task)(struct rq *rq, struct task_struct *p); =20 void (*wakeup_preempt)(struct rq *rq, struct task_struct *p, int flags); diff --git a/kernel/sched/stop_task.c b/kernel/sched/stop_task.c index 058dd42e3d9b5..c6da16a39e08d 100644 --- a/kernel/sched/stop_task.c +++ b/kernel/sched/stop_task.c @@ -54,7 +54,7 @@ dequeue_task_stop(struct rq *rq, struct task_struct *p, i= nt flags) return true; } =20 -static void yield_task_stop(struct rq *rq) +static bool yield_task_stop(struct rq *rq) { BUG(); /* the stop task should never yield, its pointless. */ } diff --git a/kernel/sched/syscalls.c b/kernel/sched/syscalls.c index 547c1f05b667e..e708a31c8e313 100644 --- a/kernel/sched/syscalls.c +++ b/kernel/sched/syscalls.c @@ -1354,11 +1354,18 @@ static void do_sched_yield(void) { struct rq_flags rf; struct rq *rq; + bool should_resched; =20 rq =3D this_rq_lock_irq(&rf); =20 + should_resched =3D current->sched_class->yield_task(rq); + + if (unlikely(!should_resched)) { + rq_unlock_irq(rq, &rf); + return; + } + schedstat_inc(rq->yld_count); - current->sched_class->yield_task(rq); =20 preempt_disable(); rq_unlock_irq(rq, &rf); --=20 2.51.0.rc0.155.g4a0f42376b-goog From nobody Sun Oct 5 03:38:30 2025 Received: from mail-wm1-f73.google.com (mail-wm1-f73.google.com [209.85.128.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 69468155A4D for ; Fri, 8 Aug 2025 20:03:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.73 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754683412; cv=none; b=uuBZ6rV5MH9HcVIu03F3ByK/vfATbTZ8xugBvFqpKIyeLBvQOhzbSbnGC24wmcehBjLHmmeVnZRyWuTft4cyRegyo+W2SyFYs00UY2NgWtNdqEd0XdY1uPr/gp7sKJWFAPFHmufz/2AaJvKsqC/ON5YSyrHrEfs/l1c+/Xj/ldg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754683412; c=relaxed/simple; bh=GVJc/wZRynwuQ6rRZT+sEMoN0AfLOSNGlxqByyDWvgA=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=stSgsccQKBb7K2vMyxRuIamkfXlceeXi07VlEC0iP4J8RGgeqDMcPVqqVvhH1v7cJKgSd0/y6OZCdqSV2sO/1aTJZhK5O9dBe+MPPKvkT57PLPT6mJ8B/gAKZHzBP1Gf+eBYSfvkXKFnf2ktIwyBsaLePYCbw8PUbh3es6NtSzU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--jpiecuch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=L3LEQzrZ; arc=none smtp.client-ip=209.85.128.73 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--jpiecuch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="L3LEQzrZ" Received: by mail-wm1-f73.google.com with SMTP id 5b1f17b1804b1-459e30e4477so18734365e9.1 for ; Fri, 08 Aug 2025 13:03:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754683409; x=1755288209; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=X8wF8pEsyzI1TAHFsJQiDAZL8Oz2FlZ7WlVS+gzTnJM=; b=L3LEQzrZqWU+B6+xkn+AasIvnUfKM9zkyou5p35ZyNzpQPpa9cWEwKtS5UQWuVmofE dwrN0XHQuiCq+UG8Jkp0Da4/RBWeM0BNLRAwcUF3hb0BO97ismnjLwKhqlkY6cmN/jTO vMwmaHzA5Vwr2B36xRYpxT7BV8qH9Fzuhn1qhv4pxCHCJRjNsuYpBY6BjHE5+frof8bH E0FtdmesmyRhAbNMU9g40y4Vw2GSfCGkBy2CdEN+9XVehMPscWl6HQAV29cJjLgTZNSe eCtu8lrjT3NX4VKE9PhF7LkvG1irnyahuiXK4iK2CKXN+4LL3UxKd1MpNUs2NRqdljm7 4nQA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754683409; x=1755288209; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=X8wF8pEsyzI1TAHFsJQiDAZL8Oz2FlZ7WlVS+gzTnJM=; b=Hf9QJpXZn9k0EKJGSnt2htOw5KqMzHS40o5RaY0y1wwslKl+IAH8Za9KU9oyGI/3PN iJOn6AX2St3vZGljJmIUO/BDRo6cOdNMxNarn6JVohvsRLcPrX3PrAr7PpTfXR3PQ5Hq I6gCTrKwzFw57LxOrtTB39bvWo1OXbW5P7iRWFFNCcY0OkPolQsa+3zY1qIl+n2TQuTU BgBj9efnGbt17Jd/qZ8bIxMjfOIwi9kc5p4MSUl2lZ0mSb4esEtX1cSwqoHBIAUVcWiI I9XoM0VgmfhRRhgqEAXcZkFc2iMPn2mvuUNUsK3TbGLlDyxjoybX1tfl7zTlnE+lQ2Sn g+JQ== X-Gm-Message-State: AOJu0Yy7F5X7vz8ZT3elmHie7RLoKaBSOVBCdqGc/LaMW5bCyNfjr9X1 6PxBePLSikMUNvJw6GFZd1+Mr8176A+el/HLWO2pSr2DhkdhdAy3h/ZgglKILip6bZ9zueIjjNe Cy9Bx5k5QSxDNMw== X-Google-Smtp-Source: AGHT+IHFiFTYIQ+EVtzyPZqEFPvTOHnwmw/5R4XWNyE+q8wVxy7SuEA1K3BWF/A/oROA0xcl2eiuy7E2ax8skA== X-Received: from wmbhj11.prod.google.com ([2002:a05:600c:528b:b0:458:b662:fbc4]) (user=jpiecuch job=prod-delivery.src-stubby-dispatcher) by 2002:a05:6000:290c:b0:3b7:899c:e887 with SMTP id ffacd0b85a97d-3b900b4f687mr4034787f8f.24.1754683408678; Fri, 08 Aug 2025 13:03:28 -0700 (PDT) Date: Fri, 8 Aug 2025 20:02:49 +0000 In-Reply-To: <20250808200250.2016584-1-jpiecuch@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250808200250.2016584-1-jpiecuch@google.com> X-Mailer: git-send-email 2.51.0.rc0.155.g4a0f42376b-goog Message-ID: <20250808200250.2016584-3-jpiecuch@google.com> Subject: [RFC PATCH 2/3] sched/fair: don't schedule() in yield if nr_running == 1 From: Kuba Piecuch To: peterz@infradead.org, mingo@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, joshdon@google.com Cc: linux-kernel@vger.kernel.org, Kuba Piecuch Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" There's no need to schedule() if we know that there are no other tasks to pick. Signed-off-by: Kuba Piecuch --- kernel/sched/fair.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index c06a2f8290822..3f9bfc64e0bc5 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -9020,7 +9020,7 @@ static bool yield_task_fair(struct rq *rq) * Are we the only task in the tree? */ if (unlikely(rq->nr_running =3D=3D 1)) - return true; + return false; =20 clear_buddies(cfs_rq, se); =20 --=20 2.51.0.rc0.155.g4a0f42376b-goog From nobody Sun Oct 5 03:38:30 2025 Received: from mail-wm1-f74.google.com (mail-wm1-f74.google.com [209.85.128.74]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9B64222068F for ; Fri, 8 Aug 2025 20:03:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.74 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754683413; cv=none; b=SqY1910pgAU3tR28diuyk1Ni2I85LRonAglv9/3kAinx0bk2OhIOURCT/kzR50PqxrN1Ifp9kwNT8Ub8UtlKhGD8OgSI81770cuvMEcLkiBernOdwhgmtSRyFtEfGrJo34jpSwbwDsuSaJ9AgiymQouYoj40cWAI1CI2Stj5sCQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754683413; c=relaxed/simple; bh=NsA/CAR+RtyCAfhqNwO/OEoxhBdFN+ZLJMUpUfq+R8M=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=YtHcwZLZkepzexfEhu4RDqZkQQHq64H6RKuyVhmdDFJRisZfs/JDaV0GbULcb4DH+tqfWycyXfzgvCn0wcOb50OUTC7tky3yrO2BWxs+PtDV4Dh25jYP/vSYtQNBNcLun4yqEGuRspNOb5ClBzsx0PnUh6pc8RlbfFlalNxbnLQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--jpiecuch.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=vqYX0x1a; arc=none smtp.client-ip=209.85.128.74 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--jpiecuch.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="vqYX0x1a" Received: by mail-wm1-f74.google.com with SMTP id 5b1f17b1804b1-458c0c8d169so12706705e9.3 for ; Fri, 08 Aug 2025 13:03:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1754683410; x=1755288210; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=5OtIhhKwZ5LIpMwZNc9pJdMveQzWwOLEkahic0/q3KU=; b=vqYX0x1aW7cdz3UZEIM3LQ0Qb8/8eHRmZeZZC5P26mWPRFQSV5syBEa6cC1IMItLjN i/pP8INzxj3u4+r6fgIzrdIVLTtpV/8ryzqBE2GDfWudvFosx21daoA6VzGqVBmOFNLq sLnfOeCJYLGLNixN3K5aM8ICOln6ZuV/onUnxsaHTUDUl2JN+7TZuBO5ED1HX26eZZ1h XZv0zB3P3WXcxO4kKquz24SnXYo7IXfhSjJYJLBryRtf1Is/o0M/+iAHOUUtKhKytTxk xX3DaFxTqdtQkcgXgMNli2g548ryuSZOJOTcESyQKu2g8KbG6bTOCQP3Wtp793YPfqNS UVmw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1754683410; x=1755288210; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=5OtIhhKwZ5LIpMwZNc9pJdMveQzWwOLEkahic0/q3KU=; b=ONJXqOWZU/V49UGy+juuvf3VWEatLLn6HPS9h/VE0lunfPM2GeXYKUEe/npWgEFUtu dxyic+Jtg/GRHKudHWKeMVHabomA7ZvrRYyFRvtOynUkhtGypNq4k9gBkuHBItQEXcUP zkw7VHyiej0zSdstRiC6wBJ/2pyvsMn7JL/KA6c/uQDMPGYglesC2X98kmjZWb0SvCUs kkD3NV2Ad20QBa3KK27/zn+h3Y3lpc+75nf4K9dRzsvIY36+vgqOy0zK+6T9BhTJTjmX XhKt0oiLdsVh1OkOGne49Xq/2GJM71Zp7QZXmgNQKJPtrYVv8AT6DtdA3/DqHS40OndR 4BLQ== X-Gm-Message-State: AOJu0Ywa1bg4/HSjgSr1rB/lFZnfBHIniT9I6CfR/akB5OXaLMlKSbft 4PoSZroLQUNc7EnfTQ9VjPduOzwbPrGtkZ8XQ6pNC1630H76Ao0tzDSP/x5OWsg5iRTJYCq/Wbo mwvMNDzGBq0iWRQ== X-Google-Smtp-Source: AGHT+IFhEHRHvFN07ovdSLruJEqx2ftiwQyIcjpEWdBXuq9L2w9g3BjJDajHTGgVHcgWzbhDZDXtlsQ4ywyR2g== X-Received: from wmsr8.prod.google.com ([2002:a05:600c:8b08:b0:459:ddf8:fed5]) (user=jpiecuch job=prod-delivery.src-stubby-dispatcher) by 2002:a05:600c:840f:b0:456:19eb:2e09 with SMTP id 5b1f17b1804b1-459f4eba76amr37936965e9.8.1754683409865; Fri, 08 Aug 2025 13:03:29 -0700 (PDT) Date: Fri, 8 Aug 2025 20:02:50 +0000 In-Reply-To: <20250808200250.2016584-1-jpiecuch@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20250808200250.2016584-1-jpiecuch@google.com> X-Mailer: git-send-email 2.51.0.rc0.155.g4a0f42376b-goog Message-ID: <20250808200250.2016584-4-jpiecuch@google.com> Subject: [RFC PATCH 3/3] sched/fair: add debugfs knob for yield throttling From: Kuba Piecuch To: peterz@infradead.org, mingo@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, joshdon@google.com Cc: linux-kernel@vger.kernel.org, Kuba Piecuch Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" yield_interval_ns specifies the interval within which any given thread is allowed to yield at most once. Subsequent calls to sched_yield() within the interval simply return without calling schedule(). Allowing unlimited calls to sched_yield() allows for DoS-like behavior because threads can continually call into schedule() which results in various types of contention. For example, if a process has a profiling timer enabled, every call to update_curr() results in an atomic add to a shared process-wide variable p->signal->cputimer->cputime_atomic.sum_exec_runtime, performed in account_group_exec_runtime(). In a synthetic benchmark consisting of 80 threads (2 per core) calling sched_yield() in a busy loop with a profiling timer enabled, we have observed that ~80% of CPU time is spent in the single atomic add instruction. Setting yield_interval_ns to 10000 lowers that percentage to 1-2%, at the cost of decreasing the total number yields that end up calling schedule() by ~60%. The benchmark was run on an Intel Emerald Rapids CPU with 60 physical cores. Signed-off-by: Kuba Piecuch --- include/linux/sched.h | 2 ++ kernel/sched/core.c | 1 + kernel/sched/debug.c | 2 ++ kernel/sched/fair.c | 29 +++++++++++++++++++++++++++++ kernel/sched/sched.h | 2 ++ 5 files changed, 36 insertions(+) diff --git a/include/linux/sched.h b/include/linux/sched.h index aa9c5be7a6325..c637025792fc6 100644 --- a/include/linux/sched.h +++ b/include/linux/sched.h @@ -946,6 +946,8 @@ struct task_struct { =20 struct sched_info sched_info; =20 + ktime_t last_yield; + struct list_head tasks; #ifdef CONFIG_SMP struct plist_node pushable_tasks; diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 81c6df746df17..acc87c9ff5681 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -4493,6 +4493,7 @@ static void __sched_fork(unsigned long clone_flags, s= truct task_struct *p) { p->on_rq =3D 0; =20 + p->last_yield =3D ktime_set(0, 0); p->se.on_rq =3D 0; p->se.exec_start =3D 0; p->se.sum_exec_runtime =3D 0; diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c index 557246880a7e0..93d2c988d491d 100644 --- a/kernel/sched/debug.c +++ b/kernel/sched/debug.c @@ -512,6 +512,8 @@ static __init int sched_init_debug(void) debugfs_create_u32("latency_warn_ms", 0644, debugfs_sched, &sysctl_resche= d_latency_warn_ms); debugfs_create_u32("latency_warn_once", 0644, debugfs_sched, &sysctl_resc= hed_latency_warn_once); =20 + debugfs_create_u32("yield_interval_ns", 0644, debugfs_sched, &sysctl_sche= d_yield_interval); + #ifdef CONFIG_SMP debugfs_create_file("tunable_scaling", 0644, debugfs_sched, NULL, &sched_= scaling_fops); debugfs_create_u32("migration_cost_ns", 0644, debugfs_sched, &sysctl_sche= d_migration_cost); diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index 3f9bfc64e0bc5..39ca52128f502 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -81,6 +81,18 @@ static unsigned int normalized_sysctl_sched_base_slice = =3D 700000ULL; =20 __read_mostly unsigned int sysctl_sched_migration_cost =3D 500000UL; =20 +/* + * This interval controls how often a given CFS thread can yield. + * A given thread can only yield once within this interval. + * The throttling is accomplished by making calls to sched_yield() return + * without actually calling schedule(). + * A value of 0 means yields are not throttled. + * + * (default: 0, units: nanoseconds) + */ +__read_mostly unsigned int sysctl_sched_yield_interval; + + static int __init setup_sched_thermal_decay_shift(char *str) { pr_warn("Ignoring the deprecated sched_thermal_decay_shift=3D option\n"); @@ -9015,6 +9027,7 @@ static bool yield_task_fair(struct rq *rq) struct task_struct *curr =3D rq->curr; struct cfs_rq *cfs_rq =3D task_cfs_rq(curr); struct sched_entity *se =3D &curr->se; + ktime_t now, throttle_end_time; =20 /* * Are we the only task in the tree? @@ -9024,6 +9037,22 @@ static bool yield_task_fair(struct rq *rq) =20 clear_buddies(cfs_rq, se); =20 + if (unlikely(sysctl_sched_yield_interval)) { + /* + * Limit how often a given thread can call schedule() via + * sched_yield() to once every sysctl_sched_yield_interval + * nanoseconds. + */ + now =3D ktime_get(); + throttle_end_time =3D ktime_add_ns(curr->last_yield, + sysctl_sched_yield_interval); + + if (unlikely(ktime_before(now, throttle_end_time))) + return false; + + curr->last_yield =3D now; + } + update_rq_clock(rq); /* * Update run-time statistics of the 'current'. diff --git a/kernel/sched/sched.h b/kernel/sched/sched.h index 8b2cd54a09942..14e3d90b0df0e 100644 --- a/kernel/sched/sched.h +++ b/kernel/sched/sched.h @@ -2827,6 +2827,8 @@ extern __read_mostly unsigned int sysctl_sched_migrat= ion_cost; =20 extern unsigned int sysctl_sched_base_slice; =20 +extern __read_mostly unsigned int sysctl_sched_yield_interval; + extern int sysctl_resched_latency_warn_ms; extern int sysctl_resched_latency_warn_once; =20 --=20 2.51.0.rc0.155.g4a0f42376b-goog