From nobody Sat Jul 25 21:21:07 2026 Received: from fanzine2.igalia.com (fanzine2.igalia.com [213.97.179.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8D6EF390991 for ; Mon, 13 Jul 2026 14:15:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.97.179.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783952105; cv=none; b=oEBzlZsZ+neKCEXljRpYybxjjxLpqIwkE6Zo01yxBrLnX6uTfRqVApoK9n4JJFckFgeXKw970ErgSkYLcAFTGQYrgPEWo6xRv4tBGc7qXUZv+5o9N8WquTBpLZHD0LMfCVdKYsLCBFr60mbDf5PSKCDJsmYV6GFuY8Zwy8UU6nk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783952105; c=relaxed/simple; bh=q/7jsvf1rgluMPK/WCYxCYqZMU2eVvIM4mBTtiqsWIg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=IZUsD+Z/o3qLv+3tMPBH3UDQ/ZtG2RpTGtxCoNHsmODpoXbQq1e+pv5RlHKYymFEC4N+pfeKW3Lxg9qlPjOGPnICm15DrreH7p2bCM4NJ58RrksekPH9lySlwiFNQZWNQuMOP8JW83C4ZrC9ZQLr4DpV5tuu3MqBjUKbpOoAX/c= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com; spf=pass smtp.mailfrom=igalia.com; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b=fR2maVA4; arc=none smtp.client-ip=213.97.179.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=igalia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="fR2maVA4" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=u7t6AOIDaNt7cBQ+Qzxy/mJOkT7jvyaowGm0Gpy5iZo=; b=fR2maVA43y0R5k/FS9keoWevoO s5L0dVkPpsuKln9pPb7KMh0gbUYxycgzy7mMB2QBGX1esPIKXvnFrYtFwWG/k3fMh3kVG5Wv0cqRK mxlbiFdfEli99aH/o6Ir+cKxq9yReYOOvx5WY1eoCGUjlMjHbRsAQkixGh8pz4oLMJBeYdP6ye43J tk0E7+Cv7UK5DneD7J0TX0eUSPfnTjR2B6PEZXFRyAcx9JwRuzTN2nRTz+m7JjVacf5V68ds8/LV1 HsE9wo+5DKRiw4j+Dse9YMeRkO/6LeFuAiiI3ZMa1zAI0NLNUqOZSIKwzPmidmObZITmb1AavEqSY dJX0Q3jg==; Received: from [90.240.106.137] (helo=localhost) by fanzine2.igalia.com with esmtpsa (Cipher TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1wjHQe-00EXHF-HP; Mon, 13 Jul 2026 16:14:40 +0200 From: Tvrtko Ursulin To: dri-devel@lists.freedesktop.org Cc: Boris Brezillon , Steven Price , Liviu Dudau , Chia-I Wu , Matthew Brost , kernel-dev@igalia.com, linux-kernel@vger.kernel.org, Tvrtko Ursulin , Chia-I Wu , Tejun Heo Subject: [RFC 1/2] workqueue: Add support for real-time workers Date: Mon, 13 Jul 2026 15:14:35 +0100 Message-ID: <20260713141436.21547-2-tvrtko.ursulin@igalia.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260713141436.21547-1-tvrtko.ursulin@igalia.com> References: <20260713141436.21547-1-tvrtko.ursulin@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" For use cases such as the DRM scheduler submitting work to the GPU on behalf of low latency userspace applications, where latter have sufficient privileges to have had successfuly obtained realtime Vulkan global priority, competing with random background CPU load can create large latency spikes which gets in the way of a smooth user experience. For these situations the existing WQ_HIPRI does not bring a noticeable improvemet and a stronger hint is needed. Lets add WQ_RTPRI which creates workers with a SCHED_FIFO scheduling class to improve this. We use a mininum priority level since we only care about winning the contest against normal background CPU load. We also limit the number of instantiated threads, both per workqueue to a maximum of two, and system- wide to a maximum of either two or one less than the number of online CPUs. The idea of that is to prevent system wide starvation cause by potentially misbehaving work items. Signed-off-by: Tvrtko Ursulin Cc: Boris Brezillon Cc: Chia-I Wu Cc: Liviu Dudau Cc: Matthew Brost Cc: Steven Price Cc: Tejun Heo --- include/linux/workqueue.h | 22 ++++-- kernel/workqueue.c | 154 ++++++++++++++++++++++++++++---------- 2 files changed, 131 insertions(+), 45 deletions(-) diff --git a/include/linux/workqueue.h b/include/linux/workqueue.h index 6177624539b3..bc313b9705cb 100644 --- a/include/linux/workqueue.h +++ b/include/linux/workqueue.h @@ -140,6 +140,12 @@ enum wq_affn_scope { WQ_AFFN_NR_TYPES, }; =20 +enum wq_priority { + WQ_PRIO_NORMAL =3D 0, + WQ_PRIO_HIGH =3D 1, + WQ_PRIO_RT =3D 2 +}; + /** * struct workqueue_attrs - A struct for workqueue attributes. * @@ -147,7 +153,12 @@ enum wq_affn_scope { */ struct workqueue_attrs { /** - * @nice: nice level + * @prio: priority level + */ + enum wq_priority prio; + + /** + * @nice: nice level for WQ_PRIO_HIGH */ int nice; =20 @@ -374,8 +385,9 @@ enum wq_flags { WQ_FREEZABLE =3D 1 << 2, /* freeze during suspend */ WQ_MEM_RECLAIM =3D 1 << 3, /* may be used for memory reclaim */ WQ_HIGHPRI =3D 1 << 4, /* high priority */ - WQ_CPU_INTENSIVE =3D 1 << 5, /* cpu intensive workqueue */ - WQ_SYSFS =3D 1 << 6, /* visible in sysfs, see workqueue_sysfs_register()= */ + WQ_RTPRI =3D 1 << 5, /* real-time priority */ + WQ_CPU_INTENSIVE =3D 1 << 6, /* cpu intensive workqueue */ + WQ_SYSFS =3D 1 << 7, /* visible in sysfs, see workqueue_sysfs_register()= */ =20 /* * Per-cpu workqueues are generally preferred because they tend to @@ -402,8 +414,8 @@ enum wq_flags { * * http://thread.gmane.org/gmane.linux.kernel/1480396 */ - WQ_POWER_EFFICIENT =3D 1 << 7, - WQ_PERCPU =3D 1 << 8, /* bound to a specific cpu */ + WQ_POWER_EFFICIENT =3D 1 << 8, + WQ_PERCPU =3D 1 << 9, /* bound to a specific cpu */ =20 __WQ_DESTROYING =3D 1 << 15, /* internal: workqueue is destroying */ __WQ_DRAINING =3D 1 << 16, /* internal: workqueue is draining */ diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 33b721a9af02..0473c009690f 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -80,9 +80,10 @@ enum worker_pool_flags { * BH pool is per-CPU and always DISASSOCIATED. */ POOL_BH =3D 1 << 0, /* is a BH pool */ - POOL_MANAGER_ACTIVE =3D 1 << 1, /* being managed */ - POOL_DISASSOCIATED =3D 1 << 2, /* cpu can't serve workers */ - POOL_BH_DRAINING =3D 1 << 3, /* draining after CPU offline */ + POOL_RT =3D 1 << 1, /* is a RT pool */ + POOL_MANAGER_ACTIVE =3D 1 << 2, /* being managed */ + POOL_DISASSOCIATED =3D 1 << 3, /* cpu can't serve workers */ + POOL_BH_DRAINING =3D 1 << 4, /* draining after CPU offline */ }; =20 enum worker_flags { @@ -104,7 +105,8 @@ enum work_cancel_flags { }; =20 enum wq_internal_consts { - NR_STD_WORKER_POOLS =3D 2, /* # standard pools per cpu */ + NR_STD_WORKER_POOLS =3D 3, /* # standard pools per cpu */ + NR_BH_WORKER_POOLS =3D 2, /* # bottom-half pools per cpu */ =20 UNBOUND_POOL_HASH_ORDER =3D 6, /* hashed by pool->attrs */ BUSY_WORKER_HASH_ORDER =3D 6, /* 64 pointers */ @@ -495,10 +497,10 @@ static bool wq_debug_force_rr_cpu =3D false; module_param_named(debug_force_rr_cpu, wq_debug_force_rr_cpu, bool, 0644); =20 /* to raise softirq for the BH worker pools on other CPUs */ -static DEFINE_PER_CPU_SHARED_ALIGNED(struct irq_work [NR_STD_WORKER_POOLS]= , bh_pool_irq_works); +static DEFINE_PER_CPU_SHARED_ALIGNED(struct irq_work [NR_BH_WORKER_POOLS],= bh_pool_irq_works); =20 /* the BH worker pools */ -static DEFINE_PER_CPU_SHARED_ALIGNED(struct worker_pool [NR_STD_WORKER_POO= LS], bh_worker_pools); +static DEFINE_PER_CPU_SHARED_ALIGNED(struct worker_pool [NR_BH_WORKER_POOL= S], bh_worker_pools); =20 /* the per-cpu worker pools */ static DEFINE_PER_CPU_SHARED_ALIGNED(struct worker_pool [NR_STD_WORKER_POO= LS], cpu_worker_pools); @@ -546,6 +548,8 @@ EXPORT_SYMBOL_GPL(system_bh_highpri_wq); struct workqueue_struct *system_dfl_long_wq __ro_after_init; EXPORT_SYMBOL_GPL(system_dfl_long_wq); =20 +static atomic_t total_rtpri_workers =3D ATOMIC_INIT(0); + static int worker_thread(void *__worker); static void workqueue_sysfs_unregister(struct workqueue_struct *wq); static void show_pwq(struct pool_workqueue *pwq); @@ -561,7 +565,7 @@ static void show_one_worker_pool(struct worker_pool *po= ol); =20 #define for_each_bh_worker_pool(pool, cpu) \ for ((pool) =3D &per_cpu(bh_worker_pools, cpu)[0]; \ - (pool) < &per_cpu(bh_worker_pools, cpu)[NR_STD_WORKER_POOLS]; \ + (pool) < &per_cpu(bh_worker_pools, cpu)[NR_BH_WORKER_POOLS]; \ (pool)++) =20 #define for_each_cpu_worker_pool(pool, cpu) \ @@ -1236,9 +1240,7 @@ static bool assign_work(struct work_struct *work, str= uct worker *worker, =20 static struct irq_work *bh_pool_irq_work(struct worker_pool *pool) { - int high =3D pool->attrs->nice =3D=3D HIGHPRI_NICE_LEVEL ? 1 : 0; - - return &per_cpu(bh_pool_irq_works, pool->cpu)[high]; + return &per_cpu(bh_pool_irq_works, pool->cpu)[pool->attrs->prio]; } =20 static void kick_bh_pool(struct worker_pool *pool) @@ -1251,7 +1253,7 @@ static void kick_bh_pool(struct worker_pool *pool) return; } #endif - if (pool->attrs->nice =3D=3D HIGHPRI_NICE_LEVEL) + if (pool->attrs->prio >=3D WQ_PRIO_HIGH) raise_softirq_irqoff(HI_SOFTIRQ); else raise_softirq_irqoff(TASKLET_SOFTIRQ); @@ -2801,13 +2803,16 @@ static int format_worker_id(char *buf, size_t size,= struct worker *worker, worker->rescue_wq->name); =20 if (pool) { - if (pool->cpu >=3D 0) + if (pool->cpu >=3D 0) { + const char *suffix[NR_STD_WORKER_POOLS] =3D { "", "H", "R" }; + return scnprintf(buf, size, "kworker/%d:%d%s", pool->cpu, worker->id, - pool->attrs->nice < 0 ? "H" : ""); - else + suffix[pool->attrs->prio]); + } else { return scnprintf(buf, size, "kworker/u%d:%d", pool->id, worker->id); + } } else { return scnprintf(buf, size, "kworker/dying"); } @@ -2863,7 +2868,11 @@ static struct worker *create_worker(struct worker_po= ol *pool) goto fail; } =20 - set_user_nice(worker->task, pool->attrs->nice); + if (pool->attrs->prio =3D=3D WQ_PRIO_RT) + sched_set_fifo_low(worker->task); + else if (pool->attrs->prio =3D=3D WQ_PRIO_HIGH) + set_user_nice(worker->task, pool->attrs->nice); + kthread_bind_mask(worker->task, pool_allowed_cpus(pool)); } =20 @@ -2876,6 +2885,9 @@ static struct worker *create_worker(struct worker_poo= l *pool) worker->pool->nr_workers++; worker_enter_idle(worker); =20 + if (pool->flags & POOL_RT) + atomic_inc(&total_rtpri_workers); + /* * @worker is waiting on a completion in kthread() and will trigger hung * check if not woken up soon. As kick_pool() is noop if @pool is empty, @@ -2909,6 +2921,8 @@ static void reap_dying_workers(struct list_head *cull= _list) list_for_each_entry_safe(worker, tmp, cull_list, entry) { list_del_init(&worker->entry); kthread_stop_put(worker->task); + if (worker->flags & WQ_RTPRI) + atomic_dec(&total_rtpri_workers); kfree(worker); } } @@ -3110,6 +3124,19 @@ __acquires(&pool->lock) mod_timer(&pool->mayday_timer, jiffies + MAYDAY_INITIAL_TIMEOUT); =20 while (true) { + if (pool->flags & POOL_RT) { + unsigned int max =3D num_online_cpus(); + + if (max > 2) + max =3D max - 1; + else + max =3D 2; + + /* Global cap on the number of RT workers */ + if (atomic_read(&total_rtpri_workers) >=3D max) + break; + } + if (create_worker(pool) || !need_to_create_worker(pool)) break; =20 @@ -3765,7 +3792,7 @@ static void drain_dead_softirq_workfn(struct work_str= uct *work) * don't hog this CPU's BH. */ if (repeat) { - if (pool->attrs->nice =3D=3D HIGHPRI_NICE_LEVEL) + if (pool->attrs->prio >=3D WQ_PRIO_HIGH) queue_work(system_bh_highpri_wq, work); else queue_work(system_bh_wq, work); @@ -3786,7 +3813,7 @@ void workqueue_softirq_dead(unsigned int cpu) { int i; =20 - for (i =3D 0; i < NR_STD_WORKER_POOLS; i++) { + for (i =3D 0; i < NR_BH_WORKER_POOLS; i++) { struct worker_pool *pool =3D &per_cpu(bh_worker_pools, cpu)[i]; struct wq_drain_dead_softirq_work dead_work; =20 @@ -3797,7 +3824,7 @@ void workqueue_softirq_dead(unsigned int cpu) dead_work.pool =3D pool; init_completion(&dead_work.done); =20 - if (pool->attrs->nice =3D=3D HIGHPRI_NICE_LEVEL) + if (pool->attrs->prio >=3D WQ_PRIO_HIGH) queue_work(system_bh_highpri_wq, &dead_work.work); else queue_work(system_bh_wq, &dead_work.work); @@ -4772,6 +4799,7 @@ struct workqueue_attrs *alloc_workqueue_attrs_noprof(= void) static void copy_workqueue_attrs(struct workqueue_attrs *to, const struct workqueue_attrs *from) { + to->prio =3D from->prio; to->nice =3D from->nice; cpumask_copy(to->cpumask, from->cpumask); cpumask_copy(to->__pod_cpumask, from->__pod_cpumask); @@ -4803,6 +4831,7 @@ static u32 wqattrs_hash(const struct workqueue_attrs = *attrs) { u32 hash =3D 0; =20 + hash =3D jhash_1word(attrs->prio, hash); hash =3D jhash_1word(attrs->nice, hash); hash =3D jhash_1word(attrs->affn_strict, hash); hash =3D jhash(cpumask_bits(attrs->__pod_cpumask), @@ -4817,6 +4846,8 @@ static u32 wqattrs_hash(const struct workqueue_attrs = *attrs) static bool wqattrs_equal(const struct workqueue_attrs *a, const struct workqueue_attrs *b) { + if (a->prio !=3D b->prio) + return false; if (a->nice !=3D b->nice) return false; if (a->affn_strict !=3D b->affn_strict) @@ -5603,11 +5634,17 @@ static void unbound_wq_update_pwq(struct workqueue_= struct *wq, int cpu) =20 static int alloc_and_link_pwqs(struct workqueue_struct *wq) { - bool highpri =3D wq->flags & WQ_HIGHPRI; - int cpu, ret; + int prio, cpu, ret; =20 lockdep_assert_held(&wq_pool_mutex); =20 + if (wq->flags & WQ_RTPRI) + prio =3D WQ_PRIO_RT; + else if (wq->flags & WQ_HIGHPRI) + prio =3D WQ_PRIO_HIGH; + else + prio =3D WQ_PRIO_NORMAL; + wq->cpu_pwq =3D alloc_percpu(struct pool_workqueue *); if (!wq->cpu_pwq) goto enomem; @@ -5624,7 +5661,7 @@ static int alloc_and_link_pwqs(struct workqueue_struc= t *wq) struct pool_workqueue **pwq_p; struct worker_pool *pool; =20 - pool =3D &(per_cpu_ptr(pools, cpu)[highpri]); + pool =3D &(per_cpu_ptr(pools, cpu)[prio]); pwq_p =3D per_cpu_ptr(wq->cpu_pwq, cpu); =20 *pwq_p =3D kmem_cache_alloc_node(pwq_cache, GFP_KERNEL, @@ -5644,14 +5681,14 @@ static int alloc_and_link_pwqs(struct workqueue_str= uct *wq) if (wq->flags & __WQ_ORDERED) { struct pool_workqueue *dfl_pwq; =20 - ret =3D apply_workqueue_attrs_locked(wq, ordered_wq_attrs[highpri]); + ret =3D apply_workqueue_attrs_locked(wq, ordered_wq_attrs[prio]); /* there should only be single pwq for ordering guarantee */ dfl_pwq =3D rcu_access_pointer(wq->dfl_pwq); WARN(!ret && (wq->pwqs.next !=3D &dfl_pwq->pwqs_node || wq->pwqs.prev !=3D &dfl_pwq->pwqs_node), "ordering guarantee broken for workqueue %s\n", wq->name); } else { - ret =3D apply_workqueue_attrs_locked(wq, unbound_std_wq_attrs[highpri]); + ret =3D apply_workqueue_attrs_locked(wq, unbound_std_wq_attrs[prio]); } =20 if (ret) @@ -5816,6 +5853,11 @@ static struct workqueue_struct *__alloc_workqueue(co= nst char *fmt, return NULL; } =20 + if (flags & WQ_RTPRI) { + if (WARN_ON_ONCE(flags & WQ_HIGHPRI)) + return NULL; + } + /* see the comment above the definition of WQ_POWER_EFFICIENT */ if ((flags & WQ_POWER_EFFICIENT) && wq_power_efficient) flags |=3D WQ_UNBOUND; @@ -5842,7 +5884,12 @@ static struct workqueue_struct *__alloc_workqueue(co= nst char *fmt, pr_warn_once("workqueue: name exceeds WQ_NAME_LEN. Truncating to: %s\n", wq->name); =20 - if (flags & WQ_BH) { + if (flags & WQ_RTPRI) { + /* + * RT workqueues are limited to max half of the online CPUs. + */ + max_active =3D min_t(int, max_active, num_online_cpus() / 2); + } else if (flags & WQ_BH) { /* * BH workqueues always share a single execution context per CPU * and don't impose any max_active limit. @@ -6344,17 +6391,26 @@ void print_worker_info(const char *log_lvl, struct = task_struct *task) } } =20 +static void pr_cont_bh_suffix(enum wq_priority prio) +{ + if (prio =3D=3D WQ_PRIO_RT) + pr_cont("bh-rt"); + else if (prio =3D=3D WQ_PRIO_HIGH) + pr_cont("bh-hi"); + else + pr_cont("bh"); +} + static void pr_cont_pool_info(struct worker_pool *pool) { pr_cont(" cpus=3D%*pbl", nr_cpumask_bits, pool->attrs->cpumask); if (pool->node !=3D NUMA_NO_NODE) pr_cont(" node=3D%d", pool->node); - pr_cont(" flags=3D0x%x", pool->flags); + pr_cont(" flags=3D0x%x ", pool->flags); if (pool->flags & POOL_BH) - pr_cont(" bh%s", - pool->attrs->nice =3D=3D HIGHPRI_NICE_LEVEL ? "-hi" : ""); + pr_cont_bh_suffix(pool->attrs->prio); else - pr_cont(" nice=3D%d", pool->attrs->nice); + pr_cont("prio=3D%d", pool->attrs->prio); } =20 static void pr_cont_worker_id(struct worker *worker) @@ -6362,8 +6418,7 @@ static void pr_cont_worker_id(struct worker *worker) struct worker_pool *pool =3D worker->pool; =20 if (pool->flags & POOL_BH) - pr_cont("bh%s", - pool->attrs->nice =3D=3D HIGHPRI_NICE_LEVEL ? "-hi" : ""); + pr_cont_bh_suffix(pool->attrs->prio); else pr_cont("%d%s", task_pid_nr(worker->task), worker->rescue_wq ? "(RESCUER)" : ""); @@ -7284,10 +7339,14 @@ static ssize_t wq_nice_show(struct device *dev, str= uct device_attribute *attr, char *buf) { struct workqueue_struct *wq =3D dev_to_wq(dev); - int written; + int written, nice; =20 mutex_lock(&wq->mutex); - written =3D scnprintf(buf, PAGE_SIZE, "%d\n", wq->unbound_attrs->nice); + if (wq->unbound_attrs->prio =3D=3D WQ_PRIO_RT) + nice =3D INT_MIN; + else + nice =3D wq->unbound_attrs->nice; + written =3D scnprintf(buf, PAGE_SIZE, "%d\n", nice); mutex_unlock(&wq->mutex); =20 return written; @@ -7313,13 +7372,20 @@ static ssize_t wq_nice_store(struct device *dev, st= ruct device_attribute *attr, { struct workqueue_struct *wq =3D dev_to_wq(dev); struct workqueue_attrs *attrs; - int ret =3D -ENOMEM; + int ret; =20 apply_wqattrs_lock(); =20 attrs =3D wq_sysfs_prep_attrs(wq); - if (!attrs) + if (!attrs) { + ret =3D -ENOMEM; goto out_unlock; + } + + if (attrs->prio =3D=3D WQ_PRIO_RT) { + ret =3D -EINVAL; + goto out_unlock; + } =20 if (sscanf(buf, "%d", &attrs->nice) =3D=3D 1 && attrs->nice >=3D MIN_NICE && attrs->nice <=3D MAX_NICE) @@ -7927,12 +7993,13 @@ static void __init restrict_unbound_cpumask(const c= har *name, const struct cpuma cpumask_and(wq_unbound_cpumask, wq_unbound_cpumask, mask); } =20 -static void __init init_cpu_worker_pool(struct worker_pool *pool, int cpu,= int nice) +static void __init init_cpu_worker_pool(struct worker_pool *pool, int cpu,= enum wq_priority prio, int nice) { BUG_ON(init_worker_pool(pool)); pool->cpu =3D cpu; cpumask_copy(pool->attrs->cpumask, cpumask_of(cpu)); cpumask_copy(pool->attrs->__pod_cpumask, cpumask_of(cpu)); + pool->attrs->prio =3D prio; pool->attrs->nice =3D nice; pool->attrs->affn_strict =3D true; pool->node =3D cpu_to_node(cpu); @@ -7956,8 +8023,9 @@ static void __init init_cpu_worker_pool(struct worker= _pool *pool, int cpu, int n void __init workqueue_init_early(void) { struct wq_pod_type *pt =3D &wq_pod_types[WQ_AFFN_SYSTEM]; - int std_nice[NR_STD_WORKER_POOLS] =3D { 0, HIGHPRI_NICE_LEVEL }; - void (*irq_work_fns[NR_STD_WORKER_POOLS])(struct irq_work *) =3D + int std_prio[NR_STD_WORKER_POOLS] =3D { 0, WQ_PRIO_HIGH, WQ_PRIO_RT }; + int std_nice[NR_STD_WORKER_POOLS] =3D { 0, HIGHPRI_NICE_LEVEL, 0 }; + void (*irq_work_fns[NR_BH_WORKER_POOLS])(struct irq_work *) =3D { bh_pool_kick_normal, bh_pool_kick_highpri }; int i, cpu; =20 @@ -8008,15 +8076,19 @@ void __init workqueue_init_early(void) =20 i =3D 0; for_each_bh_worker_pool(pool, cpu) { - init_cpu_worker_pool(pool, cpu, std_nice[i]); + init_cpu_worker_pool(pool, cpu, std_prio[i], std_nice[i]); pool->flags |=3D POOL_BH; init_irq_work(bh_pool_irq_work(pool), irq_work_fns[i]); i++; } =20 i =3D 0; - for_each_cpu_worker_pool(pool, cpu) - init_cpu_worker_pool(pool, cpu, std_nice[i++]); + for_each_cpu_worker_pool(pool, cpu) { + init_cpu_worker_pool(pool, cpu, std_prio[i], std_nice[i]); + if (i =3D=3D WQ_PRIO_RT) + pool->flags |=3D POOL_RT; + i++; + } } =20 /* create default unbound and ordered wq attrs */ @@ -8024,6 +8096,7 @@ void __init workqueue_init_early(void) struct workqueue_attrs *attrs; =20 BUG_ON(!(attrs =3D alloc_workqueue_attrs())); + attrs->prio =3D std_prio[i]; attrs->nice =3D std_nice[i]; unbound_std_wq_attrs[i] =3D attrs; =20 @@ -8032,6 +8105,7 @@ void __init workqueue_init_early(void) * guaranteed by max_active which is enforced by pwqs. */ BUG_ON(!(attrs =3D alloc_workqueue_attrs())); + attrs->prio =3D std_prio[i]; attrs->nice =3D std_nice[i]; attrs->ordered =3D true; ordered_wq_attrs[i] =3D attrs; --=20 2.54.0 From nobody Sat Jul 25 21:21:07 2026 Received: from fanzine2.igalia.com (fanzine2.igalia.com [213.97.179.56]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 938A62D060D for ; Mon, 13 Jul 2026 14:14:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=213.97.179.56 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783952103; cv=none; b=CITig1xx6laSwXU/FMW4khM/H2eGogulrW14FGgbhkbQFhjV12Hjq/dt7vye8Y5H6f7vH+7mPWKDmJHTuTyyMsl0PqJU8KVSqhZHiUoE4AxRhIxVQ7C40Q9cNzx3mtdF/R/iqGCQLTcsE43ditM3uNwJr9ixxVzc2BQx+qYDRSM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783952103; c=relaxed/simple; bh=lZD/MW6fqT3oLIbDXM87MlJxuwJvlrSn923hehR8GC0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=M1/dZ2psChRmHFlybgQO+xoqSShC2s3sJzvhcLQQc9F4/sqzfKaXmbVcjFUTdZ6sqejJR2krl1FOOqQ1S4y7Wo7+9WGnYr6OOb/aCj3fEisnJ+vsu02r2RpNYLkhie2HBnuTWMoX5gz+J2ohJ217mdBmQ5JAxhDIPEFdhYhY2xw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com; spf=pass smtp.mailfrom=igalia.com; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b=olPjHKk0; arc=none smtp.client-ip=213.97.179.56 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=igalia.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=igalia.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=igalia.com header.i=@igalia.com header.b="olPjHKk0" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=igalia.com; s=20170329; h=Content-Transfer-Encoding:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe: List-Post:List-Owner:List-Archive; bh=nBiCUD72qKVUcYFIKgH+mA1sMkpVbWa543v5wy2hy0s=; b=olPjHKk0EZPosfUC0zmiURZ5YX BOOOJn52JRH4bZnlf7G4VOrwHGA83N2iPvgyMEcxGYtHHSpmdVJcUAcyLeT7pICxioxuiJqM3NIaP 5ALPxsgG3KvZFVOuL6KgxZbQNjNe7AhsAe/zmbF0kuEgT/lUNx4LwZ/7gJBHoqYHG2dGpi2mtQhxP w19aRar+Jz+Oj9WP07KY6IxBko4dXMjwo3ggvGUjq2dxN0S+TcmQeGtNf2XXva0fgJkFD4dVBB3js clb1rtz4wfW6aghHWHxVbhzR1Bs9NRPrlvmWNLt2kEYGHPvYUvjmUYNiTj2xWiyyIgk51O15XBUzX zf6hAYvw==; Received: from [90.240.106.137] (helo=localhost) by fanzine2.igalia.com with esmtpsa (Cipher TLS1.3:ECDHE_SECP256R1__RSA_PSS_RSAE_SHA256__AES_256_GCM:256) (Exim) id 1wjHQf-00EXHP-8C; Mon, 13 Jul 2026 16:14:41 +0200 From: Tvrtko Ursulin To: dri-devel@lists.freedesktop.org Cc: Boris Brezillon , Steven Price , Liviu Dudau , Chia-I Wu , Matthew Brost , kernel-dev@igalia.com, linux-kernel@vger.kernel.org, Tvrtko Ursulin , Chia-I Wu , Tejun Heo Subject: [RFC 2/2] drm/panthor: Create per queue priority workqueues Date: Mon, 13 Jul 2026 15:14:36 +0100 Message-ID: <20260713141436.21547-3-tvrtko.ursulin@igalia.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260713141436.21547-1-tvrtko.ursulin@igalia.com> References: <20260713141436.21547-1-tvrtko.ursulin@igalia.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Split the single workqueue shared between the driver internal logic and DRM scheduler use into separate ones, where the DRM scheduler one is created per GPU priority level using the appropriate mapping to workqueue priorities. Low and medium GPU priority are served by a normal workqueue, high is server by a WQ_HIGHPRI instance, while realtime GPU priority is using the newly added WQ_RTPRI flag for lowest possible latency. These workqueues are device global and for all three we set the maximum concurrency to two in order to keep the GPU optimally fed with work. Signed-off-by: Tvrtko Ursulin Cc: Boris Brezillon Cc: Chia-I Wu Cc: Liviu Dudau Cc: Matthew Brost Cc: Steven Price Cc: Tejun Heo --- drivers/gpu/drm/panthor/panthor_sched.c | 38 +++++++++++++++++++++---- 1 file changed, 33 insertions(+), 5 deletions(-) diff --git a/drivers/gpu/drm/panthor/panthor_sched.c b/drivers/gpu/drm/pant= hor/panthor_sched.c index 2bee1c92fb9e..f01adc8c024c 100644 --- a/drivers/gpu/drm/panthor/panthor_sched.c +++ b/drivers/gpu/drm/panthor/panthor_sched.c @@ -152,11 +152,18 @@ struct panthor_scheduler { * * Used for the scheduler tick, group update or other kind of FW * event processing that can't be handled in the threaded interrupt - * path. Also passed to the drm_gpu_scheduler instances embedded - * in panthor_queue. + * path. */ struct workqueue_struct *wq; =20 + /** + * @submit_wq: Per priority workqueues for the DRM scheduler + * + * Passed to the drm_gpu_scheduler instances embedded + * in panthor_queue based on the queue priority. + */ + struct workqueue_struct *submit_wq[PANTHOR_CSG_PRIORITY_COUNT]; + /** * @heap_alloc_wq: Workqueue used to schedule tiler_oom works. * @@ -3488,7 +3495,6 @@ group_create_queue(struct panthor_group *group, { struct drm_sched_init_args sched_args =3D { .ops =3D &panthor_queue_sched_ops, - .submit_wq =3D group->ptdev->scheduler->wq, /* * The credit limit argument tells us the total number of * instructions across all CS slots in the ringbuffer, with @@ -3581,8 +3587,14 @@ group_create_queue(struct panthor_group *group, goto err_free_queue; } =20 + if (group->priority >=3D ARRAY_SIZE(group->ptdev->scheduler->submit_wq) || + !group->ptdev->scheduler->submit_wq[group->priority]) { + ret =3D -EINVAL; + goto err_free_queue; + } + sched_args.name =3D queue->name; - + sched_args.submit_wq =3D group->ptdev->scheduler->submit_wq[group->priori= ty]; ret =3D drm_sched_init(&queue->scheduler, &sched_args); if (ret) goto err_free_queue; @@ -4072,6 +4084,15 @@ static void panthor_sched_fini(struct drm_device *dd= ev, void *res) if (!sched || !sched->csg_slot_count) return; =20 + if (sched->submit_wq[PANTHOR_CSG_PRIORITY_MEDIUM]) + destroy_workqueue(sched->submit_wq[PANTHOR_CSG_PRIORITY_MEDIUM]); + + if (sched->submit_wq[PANTHOR_CSG_PRIORITY_HIGH]) + destroy_workqueue(sched->submit_wq[PANTHOR_CSG_PRIORITY_HIGH]); + + if (sched->submit_wq[PANTHOR_CSG_PRIORITY_RT]) + destroy_workqueue(sched->submit_wq[PANTHOR_CSG_PRIORITY_RT]); + if (sched->wq) destroy_workqueue(sched->wq); =20 @@ -4173,7 +4194,14 @@ int panthor_sched_init(struct panthor_device *ptdev) */ sched->heap_alloc_wq =3D alloc_workqueue("panthor-heap-alloc", WQ_UNBOUND= , 0); sched->wq =3D alloc_workqueue("panthor-csf-sched", WQ_MEM_RECLAIM | WQ_UN= BOUND, 0); - if (!sched->wq || !sched->heap_alloc_wq) { + sched->submit_wq[PANTHOR_CSG_PRIORITY_MEDIUM] =3D alloc_workqueue("pantho= r-drm", WQ_MEM_RECLAIM | WQ_UNBOUND, 2); + sched->submit_wq[PANTHOR_CSG_PRIORITY_LOW] =3D sched->submit_wq[PANTHOR_C= SG_PRIORITY_MEDIUM]; + sched->submit_wq[PANTHOR_CSG_PRIORITY_HIGH] =3D alloc_workqueue("panthor-= drm-high", WQ_HIGHPRI | WQ_MEM_RECLAIM | WQ_UNBOUND, 2); + sched->submit_wq[PANTHOR_CSG_PRIORITY_RT] =3D alloc_workqueue("panthor-dr= m-rt", WQ_RTPRI | WQ_MEM_RECLAIM | WQ_UNBOUND, 2); + if (!sched->wq || !sched->heap_alloc_wq || + !sched->submit_wq[PANTHOR_CSG_PRIORITY_MEDIUM] || + !sched->submit_wq[PANTHOR_CSG_PRIORITY_HIGH] || + !sched->submit_wq[PANTHOR_CSG_PRIORITY_RT]) { panthor_sched_fini(&ptdev->base, sched); drm_err(&ptdev->base, "Failed to allocate the workqueues"); return -ENOMEM; --=20 2.54.0