From nobody Tue Feb 10 16:22:05 2026 Received: from mail-wr1-f42.google.com (mail-wr1-f42.google.com [209.85.221.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 78DB213D619 for ; Tue, 14 May 2024 23:41:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.221.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715730085; cv=none; b=Mnt5nyGkasqMEPPVOYX8gM5mN6Tgu0r4LuoQTlfQ1/FZwipn7mbcHU+674rc6XRfjaKYI0OVxMRcRByfa0sBU8QTj7wrlrlo1HD05OFmlva4O7kH5+xoIiJJ15J7QD0LAcTGgdcCDbL21emEMVJrbx4qpxn7o5U04H5o7289dU4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1715730085; c=relaxed/simple; bh=1Jx0O6Oo2nlnOfO/6AcY+Fr1NWNsgd0wps8Qhj6D2kY=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version; b=sGJZSVofUWlpBVMz5ozsvvvAvRT/Zh1JCTvH4Bkj7WeE4rPC68RUxWRUtwhwTc3mxDZl0P0kKBZsYzF0UJzrbiQcNbY7DZGk9aUR6lUR2FO31qsfueytDke3VAEbQFChCDq7Eg/oAoVc+HfJxCVpwgC2N4QaiDpZNIAggp4L1hs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io; spf=pass smtp.mailfrom=layalina.io; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b=PQaS2CRN; arc=none smtp.client-ip=209.85.221.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=layalina.io Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=layalina.io Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=layalina-io.20230601.gappssmtp.com header.i=@layalina-io.20230601.gappssmtp.com header.b="PQaS2CRN" Received: by mail-wr1-f42.google.com with SMTP id ffacd0b85a97d-34f0e55787aso4885788f8f.2 for ; Tue, 14 May 2024 16:41:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=layalina-io.20230601.gappssmtp.com; s=20230601; t=1715730081; x=1716334881; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=cyv4d0BNd/pX9bsHIQA0n79BqkHw7pTrOWISuu/E9o8=; b=PQaS2CRNfLcLHgt7tt1wA9DSSAegR1VrvWQIsMUTJTqDw9kMgqFq1d+p8IXVdrBk/R 63DhMLdI7sLDWAXZJB8L4j+dfc1b3dwJY4WXcJE+3P496Y09puEtYEqHhAJe/SHr213F eV4vaLiht8O+02vK2njTrAS236j8ILL5Ht+rXm+GYh5a4Nv70kSMNQdZ3mDTf2SY5w5F GrH94Qae9AFw2ahOBTww1ESp28i45W1H0HzKvzh3UfYLqQcgRd5Jp0hws6R3VoJxNPuw oZz18n8xvKiKLOgncaqcu8CVVWw/0DaYr6QiuZnih0ShSWGRCZwwpVLkLtxuLVCXDQPp UgoA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1715730081; x=1716334881; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=cyv4d0BNd/pX9bsHIQA0n79BqkHw7pTrOWISuu/E9o8=; b=XPXri+SqrKCdGDH14g16Y95JCVgf2z0DqRJpiZqjV2SpOi0LvaRNFnTNycjSxRBpYl awQRiID6XkrYnJsmHlyPn1/so+v1Cy7+ZbEBaFJVROxLoqnhW/6sm2mbW0qU4rRgk7Cv U42jHiSJm71cPMHU/jnD6TOaiYJG40Da1VXVGEvjz6uJFghiY1wYzz3SgYov9RCbHGZp cepTzpIDMK5sUphQ9U1KYABmBfXos1mBiJ7U4O9srLRt/pPUvJJqhBuFIWR2teFQOkNV o00ZkmnNsZpuGWGLTzkDqL4lKYXanKi2/tiW8oynLQXFOBsLvOsxXeRifLQgXQxpXFs5 443A== X-Forwarded-Encrypted: i=1; AJvYcCU6FM9ox9JdilI2vw4XyW57oFxWyc2jzjtkzE7mo/Uvemu3WvgCnv11p0YuOtVni0KYfdoGr31oUBLvnTOkO3dbxrjt44PwugmLG+ly X-Gm-Message-State: AOJu0Yy5Od2y6qoRSGj0dgvMTCHR+Sqba9bfi9qaIXyoQuDMs3CLEAU9 5hTWIk3L6BXcl20yjrxdmM56a2q4elq3Mf8bQFgIYL9i2LfvLsUHXAd8OPj4ToA= X-Google-Smtp-Source: AGHT+IEBy2w3iH8OKtwZdjojFHLLL37I8pr75pLUSWLE4x8ivC4Yc5Dp6aLTJywXCUWt2E70Qh3Gxw== X-Received: by 2002:adf:f250:0:b0:34d:414:5f99 with SMTP id ffacd0b85a97d-3504a735149mr8899514f8f.25.1715730080671; Tue, 14 May 2024 16:41:20 -0700 (PDT) Received: from airbuntu.. (host81-157-90-255.range81-157.btcentralplus.com. [81.157.90.255]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-3502b8a78cdsm14762308f8f.58.2024.05.14.16.41.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 14 May 2024 16:41:20 -0700 (PDT) From: Qais Yousef To: Ingo Molnar , Peter Zijlstra , Juri Lelli , Steven Rostedt Cc: Vincent Guittot , Daniel Bristot de Oliveira , Thomas Gleixner , Sebastian Andrzej Siewior , Alexander Viro , Christian Brauner , Andrew Morton , Jens Axboe , linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-mm@kvack.org, Qais Yousef Subject: [PATCH] sched/rt: Clean up usage of rt_task() Date: Wed, 15 May 2024 00:41:12 +0100 Message-Id: <20240514234112.792989-1-qyousef@layalina.io> X-Mailer: git-send-email 2.34.1 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" rt_task() checks if a task has RT priority. But depends on your dictionary, this could mean it belongs to RT class, or is a 'realtime' task, which includes RT and DL classes. Since this has caused some confusion already on discussion [1], it seemed a clean up is due. I define the usage of rt_task() to be tasks that belong to RT class. Make sure that it returns true only for RT class and audit the users and replace them with the new realtime_task() which returns true for RT and DL classes - the old behavior. Introduce similar realtime_prio() to create similar distinction to rt_prio() and update the users. Move MAX_DL_PRIO to prio.h so it can be used in the new definitions. Document the functions to make it more obvious what is the difference between them. PI-boosted tasks is a factor that must be taken into account when choosing which function to use. Rename task_is_realtime() to task_has_realtime_policy() as the old name is confusing against the new realtime_task(). No functional changes were intended. [1] https://lore.kernel.org/lkml/20240506100509.GL40213@noisy.programming.k= icks-ass.net/ Signed-off-by: Qais Yousef Reviewed-by: Phil Auld --- fs/select.c | 2 +- include/linux/ioprio.h | 2 +- include/linux/sched/deadline.h | 6 ++++-- include/linux/sched/prio.h | 1 + include/linux/sched/rt.h | 27 ++++++++++++++++++++++++++- kernel/locking/rtmutex.c | 4 ++-- kernel/locking/rwsem.c | 4 ++-- kernel/locking/ww_mutex.h | 2 +- kernel/sched/core.c | 6 +++--- kernel/time/hrtimer.c | 6 +++--- kernel/trace/trace_sched_wakeup.c | 2 +- mm/page-writeback.c | 4 ++-- mm/page_alloc.c | 2 +- 13 files changed, 48 insertions(+), 20 deletions(-) diff --git a/fs/select.c b/fs/select.c index 9515c3fa1a03..8d5c1419416c 100644 --- a/fs/select.c +++ b/fs/select.c @@ -82,7 +82,7 @@ u64 select_estimate_accuracy(struct timespec64 *tv) * Realtime tasks get a slack of 0 for obvious reasons. */ =20 - if (rt_task(current)) + if (realtime_task(current)) return 0; =20 ktime_get_ts64(&now); diff --git a/include/linux/ioprio.h b/include/linux/ioprio.h index db1249cd9692..6c00342b6166 100644 --- a/include/linux/ioprio.h +++ b/include/linux/ioprio.h @@ -40,7 +40,7 @@ static inline int task_nice_ioclass(struct task_struct *t= ask) { if (task->policy =3D=3D SCHED_IDLE) return IOPRIO_CLASS_IDLE; - else if (task_is_realtime(task)) + else if (task_has_realtime_policy(task)) return IOPRIO_CLASS_RT; else return IOPRIO_CLASS_BE; diff --git a/include/linux/sched/deadline.h b/include/linux/sched/deadline.h index df3aca89d4f5..5cb88b748ad6 100644 --- a/include/linux/sched/deadline.h +++ b/include/linux/sched/deadline.h @@ -10,8 +10,6 @@ =20 #include =20 -#define MAX_DL_PRIO 0 - static inline int dl_prio(int prio) { if (unlikely(prio < MAX_DL_PRIO)) @@ -19,6 +17,10 @@ static inline int dl_prio(int prio) return 0; } =20 +/* + * Returns true if a task has a priority that belongs to DL class. PI-boos= ted + * tasks will return true. Use dl_policy() to ignore PI-boosted tasks. + */ static inline int dl_task(struct task_struct *p) { return dl_prio(p->prio); diff --git a/include/linux/sched/prio.h b/include/linux/sched/prio.h index ab83d85e1183..6ab43b4f72f9 100644 --- a/include/linux/sched/prio.h +++ b/include/linux/sched/prio.h @@ -14,6 +14,7 @@ */ =20 #define MAX_RT_PRIO 100 +#define MAX_DL_PRIO 0 =20 #define MAX_PRIO (MAX_RT_PRIO + NICE_WIDTH) #define DEFAULT_PRIO (MAX_RT_PRIO + NICE_WIDTH / 2) diff --git a/include/linux/sched/rt.h b/include/linux/sched/rt.h index b2b9e6eb9683..b31be3c50152 100644 --- a/include/linux/sched/rt.h +++ b/include/linux/sched/rt.h @@ -7,18 +7,43 @@ struct task_struct; =20 static inline int rt_prio(int prio) +{ + if (unlikely(prio < MAX_RT_PRIO && prio >=3D MAX_DL_PRIO)) + return 1; + return 0; +} + +static inline int realtime_prio(int prio) { if (unlikely(prio < MAX_RT_PRIO)) return 1; return 0; } =20 +/* + * Returns true if a task has a priority that belongs to RT class. PI-boos= ted + * tasks will return true. Use rt_policy() to ignore PI-boosted tasks. + */ static inline int rt_task(struct task_struct *p) { return rt_prio(p->prio); } =20 -static inline bool task_is_realtime(struct task_struct *tsk) +/* + * Returns true if a task has a priority that belongs to RT or DL classes. + * PI-boosted tasks will return true. Use task_has_realtime_policy() to ig= nore + * PI-boosted tasks. + */ +static inline int realtime_task(struct task_struct *p) +{ + return realtime_prio(p->prio); +} + +/* + * Returns true if a task has a policy that belongs to RT or DL classes. + * PI-boosted tasks will return false. + */ +static inline bool task_has_realtime_policy(struct task_struct *tsk) { int policy =3D tsk->policy; =20 diff --git a/kernel/locking/rtmutex.c b/kernel/locking/rtmutex.c index 88d08eeb8bc0..55c9dab37f33 100644 --- a/kernel/locking/rtmutex.c +++ b/kernel/locking/rtmutex.c @@ -347,7 +347,7 @@ static __always_inline int __waiter_prio(struct task_st= ruct *task) { int prio =3D task->prio; =20 - if (!rt_prio(prio)) + if (!realtime_prio(prio)) return DEFAULT_PRIO; =20 return prio; @@ -435,7 +435,7 @@ static inline bool rt_mutex_steal(struct rt_mutex_waite= r *waiter, * Note that RT tasks are excluded from same priority (lateral) * steals to prevent the introduction of an unbounded latency. */ - if (rt_prio(waiter->tree.prio) || dl_prio(waiter->tree.prio)) + if (realtime_prio(waiter->tree.prio)) return false; =20 return rt_waiter_node_equal(&waiter->tree, &top_waiter->tree); diff --git a/kernel/locking/rwsem.c b/kernel/locking/rwsem.c index c6d17aee4209..ad8d4438bc91 100644 --- a/kernel/locking/rwsem.c +++ b/kernel/locking/rwsem.c @@ -631,7 +631,7 @@ static inline bool rwsem_try_write_lock(struct rw_semap= hore *sem, * if it is an RT task or wait in the wait queue * for too long. */ - if (has_handoff || (!rt_task(waiter->task) && + if (has_handoff || (!realtime_task(waiter->task) && !time_after(jiffies, waiter->timeout))) return false; =20 @@ -914,7 +914,7 @@ static bool rwsem_optimistic_spin(struct rw_semaphore *= sem) if (owner_state !=3D OWNER_WRITER) { if (need_resched()) break; - if (rt_task(current) && + if (realtime_task(current) && (prev_owner_state !=3D OWNER_WRITER)) break; } diff --git a/kernel/locking/ww_mutex.h b/kernel/locking/ww_mutex.h index 3ad2cc4823e5..fa4b416a1f62 100644 --- a/kernel/locking/ww_mutex.h +++ b/kernel/locking/ww_mutex.h @@ -237,7 +237,7 @@ __ww_ctx_less(struct ww_acquire_ctx *a, struct ww_acqui= re_ctx *b) int a_prio =3D a->task->prio; int b_prio =3D b->task->prio; =20 - if (rt_prio(a_prio) || rt_prio(b_prio)) { + if (realtime_prio(a_prio) || realtime_prio(b_prio)) { =20 if (a_prio > b_prio) return true; diff --git a/kernel/sched/core.c b/kernel/sched/core.c index 1a914388144a..27f15de3d099 100644 --- a/kernel/sched/core.c +++ b/kernel/sched/core.c @@ -162,7 +162,7 @@ static inline int __task_prio(const struct task_struct = *p) if (p->sched_class =3D=3D &stop_sched_class) /* trumps deadline */ return -2; =20 - if (rt_prio(p->prio)) /* includes deadline */ + if (realtime_prio(p->prio)) /* includes deadline */ return p->prio; /* [-1, 99] */ =20 if (p->sched_class =3D=3D &idle_sched_class) @@ -2198,7 +2198,7 @@ static int effective_prio(struct task_struct *p) * keep the priority unchanged. Otherwise, update priority * to the normal priority: */ - if (!rt_prio(p->prio)) + if (!realtime_prio(p->prio)) return p->normal_prio; return p->prio; } @@ -10282,7 +10282,7 @@ void normalize_rt_tasks(void) schedstat_set(p->stats.sleep_start, 0); schedstat_set(p->stats.block_start, 0); =20 - if (!dl_task(p) && !rt_task(p)) { + if (!realtime_task(p)) { /* * Renice negative nice level userspace * tasks back to 0: diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c index 70625dff62ce..4150e98847fa 100644 --- a/kernel/time/hrtimer.c +++ b/kernel/time/hrtimer.c @@ -1996,7 +1996,7 @@ static void __hrtimer_init_sleeper(struct hrtimer_sle= eper *sl, * expiry. */ if (IS_ENABLED(CONFIG_PREEMPT_RT)) { - if (task_is_realtime(current) && !(mode & HRTIMER_MODE_SOFT)) + if (task_has_realtime_policy(current) && !(mode & HRTIMER_MODE_SOFT)) mode |=3D HRTIMER_MODE_HARD; } =20 @@ -2096,7 +2096,7 @@ long hrtimer_nanosleep(ktime_t rqtp, const enum hrtim= er_mode mode, u64 slack; =20 slack =3D current->timer_slack_ns; - if (rt_task(current)) + if (realtime_task(current)) slack =3D 0; =20 hrtimer_init_sleeper_on_stack(&t, clockid, mode); @@ -2301,7 +2301,7 @@ schedule_hrtimeout_range_clock(ktime_t *expires, u64 = delta, * Override any slack passed by the user if under * rt contraints. */ - if (rt_task(current)) + if (realtime_task(current)) delta =3D 0; =20 hrtimer_init_sleeper_on_stack(&t, clock_id, mode); diff --git a/kernel/trace/trace_sched_wakeup.c b/kernel/trace/trace_sched_w= akeup.c index 0469a04a355f..19d737742e29 100644 --- a/kernel/trace/trace_sched_wakeup.c +++ b/kernel/trace/trace_sched_wakeup.c @@ -545,7 +545,7 @@ probe_wakeup(void *ignore, struct task_struct *p) * - wakeup_dl handles tasks belonging to sched_dl class only. */ if (tracing_dl || (wakeup_dl && !dl_task(p)) || - (wakeup_rt && !dl_task(p) && !rt_task(p)) || + (wakeup_rt && !realtime_task(p)) || (!dl_task(p) && (p->prio >=3D wakeup_prio || p->prio >=3D current->pr= io))) return; =20 diff --git a/mm/page-writeback.c b/mm/page-writeback.c index 3e19b87049db..7372e40f225d 100644 --- a/mm/page-writeback.c +++ b/mm/page-writeback.c @@ -418,7 +418,7 @@ static void domain_dirty_limits(struct dirty_throttle_c= ontrol *dtc) if (bg_thresh >=3D thresh) bg_thresh =3D thresh / 2; tsk =3D current; - if (rt_task(tsk)) { + if (realtime_task(tsk)) { bg_thresh +=3D bg_thresh / 4 + global_wb_domain.dirty_limit / 32; thresh +=3D thresh / 4 + global_wb_domain.dirty_limit / 32; } @@ -468,7 +468,7 @@ static unsigned long node_dirty_limit(struct pglist_dat= a *pgdat) else dirty =3D vm_dirty_ratio * node_memory / 100; =20 - if (rt_task(tsk)) + if (realtime_task(tsk)) dirty +=3D dirty / 4; =20 return dirty; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 14d39f34d336..0af24a60ade0 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -3877,7 +3877,7 @@ gfp_to_alloc_flags(gfp_t gfp_mask, unsigned int order) */ if (alloc_flags & ALLOC_MIN_RESERVE) alloc_flags &=3D ~ALLOC_CPUSET; - } else if (unlikely(rt_task(current)) && in_task()) + } else if (unlikely(realtime_task(current)) && in_task()) alloc_flags |=3D ALLOC_MIN_RESERVE; =20 alloc_flags =3D gfp_to_alloc_flags_cma(gfp_mask, alloc_flags); --=20 2.34.1