.../admin-guide/kernel-parameters.txt | 37 +++- Documentation/core-api/housekeeping.rst | 6 +- block/blk-mq-cpumap.c | 163 ++++++++++++++++-- block/blk-mq.c | 63 +++++++ drivers/scsi/aacraid/comminit.c | 3 +- include/linux/group_cpus.h | 3 + include/linux/sched/isolation.h | 1 + kernel/irq/affinity.c | 29 +++- kernel/sched/isolation.c | 18 ++ lib/group_cpus.c | 112 ++++++++++-- 10 files changed, 395 insertions(+), 40 deletions(-)
Hi Thomas, Jens, Sebastian, Frederic,
This is version 16 of the series enabling multiqueue devices and managed
interrupts to adhere to CPU isolation constraints. The series has been
rebased on v7.3-rc1-708-g893e11787f78.
Background and Problem
======================
Modern multiqueue storage controllers (e.g. NVMe devices) typically
allocate as many hardware queues and interrupt vectors as there are CPUs in
the system. When CPU isolation is requested via isolcpus, the kernel's
current mechanism "isolcpus=managed_irq" relies on post-allocation affinity
steering inside kernel/irq/manage.c. However, because drivers allocate
queues and vectors before affinity steering takes place, the number of
vectors frequently matches or exceeds the total CPU count. In such
scenarios, the post-allocation steering logic cannot avoid assigning
managed interrupts to isolated CPUs, effectively defeating CPU isolation
for real-time and latency-critical workloads.
Proposed solution and architecture
==================================
This series introduces a new CPU isolation feature,
"isolcpus=managed_irq_strict", designed to protect isolated cores from the
disruptive hardware interrupts generated by high-performance multiqueue
devices.
To prevent I/O stalls, the block layer is additionally hardened to reject
hotplug requests that attempt to offline a housekeeping CPU if it is the
last remaining CPU actively serving an online isolated core.
The complex "top-down" mask plumbing introduced in earlier revisions
(version 12), which modified struct irq_affinity and expanded block layer
APIs, has been abandoned. It is replaced by a centralised approach: direct
isolation querying via housekeeping_cpumask(HK_TYPE_MANAGED_IRQ_STRICT)
within the genirq/affinity subsystem. This architectural simplification
successfully decouples core changes from driver-specific implementations.
In version 15, this was addressed under the previously named
"isolcpus=io_queue". During subsequent review, Sebastian Andrzej Siewior
pointed out that the existing "isolcpus=managed_irq" implementation is
fundamentally broken on modern multiqueue hardware, and raised the question
of whether we should introduce another flag at all, or instead fix
"managed_irq" directly in place. As Sebastian observed, if
backward-compatibility concerns prevent modifying the legacy behaviour, a
"managed_irq_strict" designation represents the logical counterpart.
To make progress while keeping backwards compatibility completely safe,
this iteration adopts option 2: we drop the "io_queue" naming in favour of
"isolcpus=managed_irq_strict". It acts as the strict, pre-allocation
counterpart to "managed_irq", constraining both block layer queue mappings
and generic interrupt affinity masks exclusively to housekeeping CPUs.
Furthermore, if both options are supplied, "managed_irq_strict" takes
precedence.
Maintainer guidance requested
=============================
We would appreciate guidance from Frederic Weisbecker regarding preference
on this front:
1. Do you prefer keeping "managed_irq_strict" as an explicit
opt-in (as implemented in this series) to preserve historical
behaviour for existing users of "managed_irq"?
2. Or would you prefer upgrading the semantics of "managed_irq"
directly in place without introducing a new parameter, on the
grounds that the legacy post-allocation behaviour is already
broken for modern multiqueue hardware?
Please let me know your thoughts.
Changes since v15:
- Migrated feature naming from "isolcpus=io_queue" (HK_TYPE_IO_QUEUE) to
"isolcpus=managed_irq_strict" (HK_TYPE_MANAGED_IRQ_STRICT), establishing
it as the strict, pre-allocation counterpart to "managed_irq"
- Specifying "managed_irq_strict" automatically enables
HK_FLAG_MANAGED_IRQ so that dependent subsystems continue to function
- Cleaned up commit bodies across the series, stripping informal
scratchpads and adding Co-developed-by attribution where appropriate
- Added new patch 4/9 ("sched/isolation: Prevent out-of-bounds read in
isolcpus= boot parameter parser"). Incorporated prerequisite bugfix
resolving an out-of-bounds read in housekeeping_isolcpus_setup() when
sub-parameters lack trailing commas
- Defined HK_TYPE_MANAGED_IRQ_STRICT and HK_FLAG_MANAGED_IRQ_STRICT.
Ensured HK_FLAG_MANAGED_IRQ is set when managed_irq_strict is enabled
- Decoupled queue mapping from dynamic cpu_online_mask; eliminated the
blk_mq_validate() check and online cpumask snapshotting to avoid TOCTOU
races during concurrent CPU hotplug events
- Added cpuhp_tasks_frozen check to ensure suspend, hibernation, and
resume transitions are not aborted
- Added blk_mq_map_queue_type() check so CPU offline validation only
evaluates hardware contexts actively mapped to the CPU being offlined
- Enhanced warning message to report the offending CPU ID and queue
number
- Prioritised affd->calc_sets in irq_calc_affinity_vectors() so multi-set
drivers scale vector allocations with requested set sizes to prevent
cross-queue contention, while vector spreading remains strictly
constrained to housekeeping cores
- Documented "managed_irq_strict" in kernel-parameters.txt, detailing its
role in blk-mq queue distribution and genirq affinity masking, as well
as its superset relationship with legacy "managed_irq"
- Link to v15: https://lore.kernel.org/lkml/20260521232956.553287-1-atomlin@atomlin.com/
Changes since v14:
- Fixed a division-by-zero by ensuring group_mask_cpus_evenly() safely
frees its allocations and returns NULL instead of an empty array if the
provided mask yields zero groups.
- Fixed a device probe -ENOSPC regression in blk_mq_num_queues(). If the
housekeeping mask intersection evaluated to 0 (e.g., against a localised
NUMA node), min_not_zero() would erroneously return the absolute maximum
hardware queues. The result is now safely clamped to a minimum of 1.
- Added a mapping verification check to prevent unrelated housekeeping
CPUs from aborting the global hotplug offline sequence.
- Aligned the pr_warn format specifier with the unsigned int declaration
of hctx->queue_num in blk_mq_hctx_can_offline_hk_cpu().
- Link to v14: https://lore.kernel.org/lkml/20260520215030.496803-1-atomlin@atomlin.com/
Changes since v13:
- Removed ineffective data_race() annotations around mask and
cpu_present_mask pointers. Wrapping the pointers failed to suppress
KCSAN warnings for the underlying inline bitmap memory accesses.
- Fixed a silent validation bypass in blk_mq_map_hw_queues() caused by
overlapping IRQ affinity masks by removing the short-circuiting
optimisation and evaluating the active_hctx bitmap in a secondary pass.
- Restored topology-aware multi-queue fallback by correctly routing
missing IRQ affinity masks to the map_software path instead of the naive
map-all fallback.
- Dropped hctx->queue->disk->disk_name from warning to avoid a UAF.
- Fixed an isolation leak where excess allocated hardware queues were
improperly padded with irq_default_affinity. Because these queues are
marked as managed, they bypassed user-space IRQ balancing; they are now
safely padded with the housekeeping mask.
- Enforced the housekeeping vector cap prior to evaluating driver-provided
calc_sets() callbacks, preventing modern multi-queue drivers from
bypassing the cap and wasting memory on dead queues.
- Introduced a safety net to the vector calculation to prevent fatal
-ENOSPC device probe aborts on heavily isolated systems where the
housekeeping CPU count is lower than the device's structural minimum.
- Removed an inaccurate claim stating that the io_queue isolation flag
takes precedence over managed_irq. Both flags are parsed, evaluated, and
enforced entirely independently by their respective subsystems.
- Link to v13: https://lore.kernel.org/lkml/20260513005509.135966-1-atomlin@atomlin.com/
Changes since v12:
- Resolved TOCTOU race conditions against CPU hotplug events in
blk_mq_map_queues() and group_mask_cpus_evenly() by taking lockless
snapshots of the online CPU mask prior to algorithmic evaluation.
- Migrated the active_hctx tracking to a dynamically sized bitmap
(bitmap_zalloc), resolving a critical out-of-bounds memory write that
occurred when hardware queues exceeded the system CPU count.
- Wrapped the disk pointer fetch in blk_mq_hctx_can_offline_hk_cpu() with
READ_ONCE() to prevent a TOCTOU NULL pointer dereference against
concurrent device teardowns.
- Introduced bitmap_empty() checks to prevent the mapping logic from
routing unassigned CPUs into unallocated memory when all mapped CPUs are
offline, safely forcing a fallback mapping instead.
- Implemented a native two-stage distribution logic in
group_mask_cpus_evenly() that first prioritises physically present CPUs
to prevent I/O starvation before distributing remaining vectors to
non-present CPUs for hotplug safety.
- Restricted the maximum number of allocated vectors in
irq_calc_affinity_vectors() to the weight of the housekeeping mask,
preventing drivers from wasting memory on dead hardware queues that
physically cannot be routed.
- Added padding logic using irq_default_affinity for sets where isolation
constraints yield fewer masks than requested vectors, preserving the 1:1
hardware queue mapping sequence for subsequent sets.
- Fixed a logic flaw that prematurely rejected valid offline requests by
manually iterating over cpu_online_mask and reverse-mapping to
accurately detect isolated CPUs, properly permitting the offlining of
non-housekeeping CPUs.
- Corrected an absolute versus relative queue index calculation bug in
blk_mq_map_queues() that was overwriting loop iterations, by iterating
directly over the generated masks.
- Replaced scoped __free cleanups with traditional goto unwinding in the
block layer to align with subsystem styling guidelines.
- Refined the io_queue kernel command-line parameter documentation for
better clarity and precision.
- Link to v12: https://lore.kernel.org/lkml/20260422185215.100929-1-atomlin@atomlin.com/
Changes since v11:
- Removed duplicate paragraph from the commit message in patch 11
(Marco Crivellari)
- Ensure ZERO_SIZE_PTR is not returned by group_mask_cpus_evenly()
(Marco Crivellari)
- Link to v11: https://lore.kernel.org/lkml/20260416192942.1243421-1-atomlin@atomlin.com/
Changes since v10:
- Completely rewrote the isolcpus=io_queue documentation in
Documentation/admin-guide/kernel-parameters.txt to clarify its exclusive
application to managed IRQs, queue allocation limits, vector exhaustion
prevention, and hardware interrupt routing (Ming Lei)
- Fixed a stack frame bloat issue by avoiding the on-stack declaration of
struct cpumask (Waiman Long)
- Link to v10: https://lore.kernel.org/linux-nvme/20260401222312.772334-1-atomlin@atomlin.com/
Changes since v9:
- Fixed a page fault regression encountered when initialising secondary
queue maps (e.g. NVMe poll queues). Restored the qmap->queue_offset to
the mq_map assignment to ensure CPUs are strictly mapped to absolute
hardware indices (Keith Busch)
- Corrected the active_hctx tracker to utilise relative queue indices,
preventing out-of-bounds mask assignments
- Fixed the blk_mq_validate() sanity check to properly evaluate absolute
queue indices against the offset-adjusted loop index
- Corrected typographical errors within block/blk-mq-cpumap.c
(Keith Busch)
- Clarified the commit message regarding the removal of the !SMP fallback
code, explicitly noting that the core scheduler now mandates SMP
unconditionally (Sebastian Andrzej Siewior)
- Added missing "Signed-off-by:" tags to properly record the patch series
chain of custody
- Link to v9: https://lore.kernel.org/lkml/20260330221047.630206-1-atomlin@atomlin.com/
Changes since v8:
- Added "Reviewed-by:" tags
- Introduced irq_spread_hk_filter() to safely restrict managed IRQ
affinity to housekeeping CPUs (Thomas Gleixner)
- Removed the unsafe global static variable blk_hk_online_mask from
blk-mq-cpumap.c and blk-mq.c. blk_mq_online_queue_affinity() now returns
a stable pointer, delegating safe intersection to the callers to prevent
concurrent modification races (Thomas Gleixner, Hannes Reinecke)
- Resolved BUG: kernel NULL pointer dereference in __blk_mq_all_tag_iter
reported by the kernel test robot during cpuhotplug rcutorture stress
testing
- Link to v8: https://lore.kernel.org/lkml/20250905-isolcpus-io-queues-v8-0-885984c5daca@kernel.org/
Changes since v7:
- Added commit 524f5eea4bbe ("lib/group_cpus: remove !SMP code")
- Merged the new mapping logic directly into the existing function to
avoid special casing
- Refined the group_mask_cpus_evenly() implementation with the following
updates:
- Corrected the function name typo (changed group_masks_cpus_evenly to
group_mask_cpus_evenly)
- Updated the documentation comment to accurately reflect the function's
behavior
- Renamed the cpu_mask argument to mask for consistency
- Added a new patch for aacraid to include the missing number of queues
calculation
- Restricted updates to only affect SCSI drivers that support
PCI_IRQ_AFFINITY and do not utilise nvme-fabrics
- Removed the __free cleanup attribute usage for cpumask_var_t allocations
due to compatibility issues
- Updated the documentation to explicitly highlight the limitations
surrounding CPU offlining
- Collected accumulated Reviewed-by and Acked-by tags
- Link to v7: https://patch.msgid.link/20250702-isolcpus-io-queues-v7-0-557aa7eacce4@kernel.org
Changes since v6:
- Sent out the first part of the series independently:
https://lore.kernel.org/all/20250617-isolcpus-queue-counters-v1-0-13923686b54b@kernel.org/
- Added comprehensive kernel command-line documentation
- Added validation logic to ensure the resulting CPU-to-queue mapping is
fully operational
- Rewrote the isolcpus mapping code to properly account for active
hardware contexts (hctx)
- Introduced blk_mq_map_hk_irq_queues, which utilizes the mask retrieved
from irq_get_affinity()
- Refactored blk_mq_map_hk_queues to require the caller to explicitly test
for HK_TYPE_MANAGED_IRQ
- Link to v6: https://patch.msgid.link/20250424-isolcpus-io-queues-v6-0-9a53a870ca1f@kernel.org
Changes since v5:
- Reintroduced the io_queue type for the isolcpus kernel parameter
- Prevented the offlining of a housekeeping CPU if an isolated CPU is
still present, upgrading this behavior from a simple warning to a hard
restriction
- Link to v5: https://lore.kernel.org/r/20250110-isolcpus-io-queues-v5-0-0e4f118680b0@kernel.org
Changes since v4:
- Rebased the series onto the latest for-6.14/block branch.
- Updated the documentation regarding the managed_irq parameters
- Reworded the commit message for "blk-mq: issue warning when offlining
hctx with online isolcpus" for better clarity
- Split the input and output parameters in the patch "lib/group_cpus: let
group_cpu_evenly return number of groups"
- Dropped the patch "sched/isolation: document HK_TYPE housekeeping
option"
- Link to v4: https://lore.kernel.org/r/20241217-isolcpus-io-queues-v4-0-5d355fbb1e14@kernel.org
Changes since v3:
- Added the patch "blk-mq: issue warning when offlining hctx with online
isolcpus"
- Fixed the check in group_cpus_evenly(); the condition now properly uses
housekeeping_enabled() instead of cpumask_weight(), as the latter always
returns a valid mask
- Dropped the Fixes: tag from "lib/group_cpus.c: honor housekeeping config
when grouping CPUs"
- Fixed an overlong line warning in the patch "scsi: use block layer
helpers to calculate num of queues"
- Dropped the patch "sched/isolation: Add io_queue housekeeping option" in
favor of simply documenting the housekeeping hk_type enum
- Added the patch "lib/group_cpus: let group_cpu_evenly return number of
groups"
- Collected accumulated Reviewed-by and Acked-by tags
- Split the patchset by moving foundational changes into a separate
preparation series:
https://lore.kernel.org/linux-nvme/20241202-refactor-blk-affinity-helpers-v6-0-27211e9c2cd5@kernel.org/
- Link to v3: https://lore.kernel.org/r/20240806-isolcpus-io-queues-v3-0-da0eecfeaf8b@suse.de
Changes since v2:
- Integrated patches from Ming Lei
(https://lore.kernel.org/all/20210709081005.421340-1-ming.lei@redhat.com/):
"virtio: add APIs for retrieving vq affinity" and "blk-mq: introduce
blk_mq_dev_map_queues"
- Replaced all instances of blk_mq_pci_map_queues and
blk_mq_virtio_map_queues with the new unified blk_mq_dev_map_queues
- Updated and expanded the helper functions used for calculating the
number of queues
- Added the CPU-to-hctx mapping function specifically to support the
isolcpus=io_queue parameter
- Documented the hk_type enum and the newly introduced isolcpus=io_queue
parameter
- Added the patch "scsi: pm8001: do not overwrite PCI queue mapping"
- Link to v2: https://lore.kernel.org/r/20240627-isolcpus-io-queues-v2-0-26a32e3c4f75@suse.de
Changes since v1:
- Updated the feature documentation for clarity and completeness
- Split the blk/nvme-pci patch into smaller, logical commits
- Dropped the HK_TYPE_IO_QUEUE macro in favor of reusing
HK_TYPE_MANAGED_IRQ
- Link to v1: https://lore.kernel.org/r/20240621-isolcpus-io-queues-v1-0-8b169bf41083@suse.de
Aaron Tomlin (2):
sched/isolation: Prevent out-of-bounds read in isolcpus= boot
parameter parser
genirq/affinity: Restrict managed IRQ affinity to housekeeping CPUs
Daniel Wagner (7):
scsi: aacraid: use block layer helpers to calculate num of queues
lib/group_cpus: remove dead !SMP code
lib/group_cpus: Add group_mask_cpus_evenly()
isolation: Introduce managed_irq_strict isolcpus type
blk-mq: use hk cpus only when isolcpus=managed_irq_strict is enabled
blk-mq: prevent offlining hk CPUs with associated online isolated CPUs
docs: add managed_irq_strict flag to isolcpus
.../admin-guide/kernel-parameters.txt | 37 +++-
Documentation/core-api/housekeeping.rst | 6 +-
block/blk-mq-cpumap.c | 163 ++++++++++++++++--
block/blk-mq.c | 63 +++++++
drivers/scsi/aacraid/comminit.c | 3 +-
include/linux/group_cpus.h | 3 +
include/linux/sched/isolation.h | 1 +
kernel/irq/affinity.c | 29 +++-
kernel/sched/isolation.c | 18 ++
lib/group_cpus.c | 112 ++++++++++--
10 files changed, 395 insertions(+), 40 deletions(-)
--
2.55.0
Hi Aaron, Daniel,
Thanks for respinning this and for the move to
"isolcpus=managed_irq_strict".
Just a heads-up that I have picked up v16 and am backporting it to the
6.18 LTS tree so I can exercise the new parameter on our PREEMPT_RT
configuration. The goal is to validate the end-to-end behaviour of
managed_irq_strict on multiqueue hardware (NVMe and SCSI) with isolated
cores, in particular:
- managed IRQ affinity and the reserved pre/post vectors staying off
the isolated set and confined to housekeeping CPUs;
- the blk-mq hotplug guard that refuses to offline the last
housekeeping CPU still serving an online isolated core;
- queue/vector counts on systems where nr_vectors >= nr_cpus
(narrow housekeeping pools, many PFs + NVMe).
I will follow up on this thread with results (effective_affinity and
blk-mq queue mapping data, plus any backport-specific findings) once
testing on 6.18 LTS is complete.
On the maintainer question (option 1 vs option 2): from a
downstream/product standpoint I favour option 1, keeping
"managed_irq_strict" as an explicit opt-in. Leaving the legacy
"managed_irq" semantics unchanged avoids a silent behavioural change
for existing deployments, while the strict variant gives new users a
clear, predictable knob.
Thanks for driving this forward.
Ionut
Hi Aaron, Daniel,
As promised, here are the end-to-end results from exercising v16 on our
6.18 LTS PREEMPT_RT tree. Short version: managed_irq_strict behaves
exactly as documented on multiqueue NVMe with a deliberately narrow
housekeeping pool, including the nr_vectors >= nr_cpus case.
Series under test (as backported to 6.18 LTS)
---------------------------------------------
0001 scsi-aacraid: use block-layer helpers for queue count
0002 lib/group_cpus: remove dead SMP code
0003 lib/group_cpus: add group_mask_cpus_evenly()
0004 sched/isolation: prevent out-of-bounds read in isolcpus parsing
0005 isolation: introduce the managed_irq_strict isolcpus type
0006 blk-mq: use housekeeping CPUs only with isolcpus=managed_irq_strict
0007 blk-mq: prevent offlining housekeeping CPUs with online-associated
queues
0008 genirq/affinity: restrict managed IRQ affinity to housekeeping CPUs
0009 docs: document the managed_irq_strict isolcpus flag
0010 genirq/affinity: confine reserved pre/post vectors to housekeeping
CPUs
Note: 0010 is a local follow-up (see section 3), not part of the
upstream v16 series.
Test environment
----------------
Kernel: 6.18.15-rt and 6.18.52-rt (PREEMPT_RT), v16 backported
(6.18.52 is the current 6.18 LTS; behaviour is identical on
both, the data below is from 6.18.15)
CPU: single-socket Intel Xeon (Granite Rapids), 40C/80T,
1 NUMA node (all 80 CPUs on node0)
Storage: 2x NVMe (SK hynix PE9010), b6:00.0 and b7:00.0
NICs: 8-port E830-CC + 4-port E825-C (managed IRQs)
MSI-X: 2235 vectors allocated system-wide
cmdline (relevant bits):
nohz_full=2-38,42-78
isolcpus=nohz,domain,managed_irq_strict,2-38,42-78
rcu_nocbs=2-39,42-79 kthread_cpus=0-1,40-41 irqaffinity=39,79
=> isolated (isolcpus) = 2-38,42-78 (74 CPUs)
housekeeping (HK) = 0,1,39,40,41,79 (6 CPUs)
Because HK is only 6 CPUs while the box carries 2235 vectors, this is
squarely the "narrow housekeeping pool, nr_vectors >= nr_cpus" scenario.
1) blk-mq queue allocation and mapping (patches 6, 7)
-----------------------------------------------------
Both NVMe devices allocate exactly 6 hardware queues, matching the HK
weight. Each hctx is owned by exactly one HK CPU:
nvme0n1 / nvme1n1:
hctx0: 0 hctx1: 1 hctx2: 39
hctx3: 40 hctx4: 41 hctx5: 79
No isolated CPU owns a queue. The sysfs cpu_list only shows the HK CPUs
because blk_mq_map_swqueue() strips the isolated CPUs from hctx->cpumask
(as intended). The full software-context routing from debugfs confirms
the 74 isolated CPUs are still mapped onto these 6 HK queues rather than
left unserved, evenly (~12-13 isolated CPUs per queue), so I/O submitted
from an isolated core completes on a housekeeping CPU:
hctx0 (owner 0): 0,2,8,14,20,26,32,38,47,53,59,65,71,77
hctx1 (owner 1): 1,3,9,15,21,27,33,42,48,54,60,66,72,78
hctx2 (owner 39): 4,10,16,22,28,34,39,43,49,55,61,67,73
hctx3 (owner 40): 5,11,17,23,29,35,40,44,50,56,62,68,74
hctx4 (owner 41): 6,12,18,24,30,36,41,45,51,57,63,69,75
hctx5 (owner 79): 7,13,19,25,31,37,46,52,58,64,70,76,79
All 80 CPUs are covered exactly once. This also lines up with the
hotplug guard below: isolated CPU 3 is routed to hctx1 (owned by HK CPU
1), which is precisely why offlining CPU 1 is refused.
2) Managed IRQ affinity (patch 8)
---------------------------------
Every NVMe completion-queue vector has its effective_affinity confined
to the HK set:
nvme0q1..q6 eff = 0, 1, 39, 40, 41, 79
nvme1q1..q6 eff = 0, 1, 39, 40, 41, 79
No managed interrupt targets an isolated CPU.
3) Reserved pre/post vectors
----------------------------
The NVMe admin (pre_vector) queues also stay off the isolated set:
nvme0q0 eff = 79
nvme1q0 eff = 39
Note: in the stock series the reserved pre/post vectors still default to
irq_default_affinity, which on this config spans the isolated CPUs, so a
freshly probed admin queue can briefly land on an isolated core before
any userspace re-steer. On PREEMPT_RT that stray wakeup is precisely
what we are trying to avoid, so we carry a small follow-up that confines
the affd->pre_vectors / post_vectors to the housekeeping mask when
HK_TYPE_MANAGED_IRQ_STRICT is set. The affinities above are with that
follow-up applied. Happy to post it separately if there is interest in
closing that gap upstream.
4) blk-mq hotplug guard (patch 7)
---------------------------------
Offlining a housekeeping CPU that still serves an online isolated CPU is
refused:
# echo 0 > /sys/devices/system/cpu/cpu1/online
-> fails (-EINVAL)
blk-mq: cannot offline CPU 1: online isolated CPU 3 is still
mapped to hctx1
After offlining the dependent isolated CPUs first, offlining the HK CPU
succeeds. suspend/hibernate (cpuhp_tasks_frozen) is correctly exempted.
5) Vector count / probe robustness (patch 8)
--------------------------------------------
With 2235 vectors against 6 HK CPUs, no device aborted probe: no -ENOSPC
and no nr_io_queues=0. The minimum-vector safety net in
irq_calc_affinity_vectors() holds on this narrow-HK box.
Backport notes (6.18.15 / 6.18.52)
----------------------------------
Nothing behavioural, just tree deltas: the default isolcpus flag stays
HK_FLAG_DOMAIN (no HK_FLAG_DOMAIN_BOOT in this tree); the cpus_read_lock
around the cpu_possible_mask weight in irq_calc_affinity_vectors() is
retained; and the housekeeping.rst hunk from the docs patch is dropped
(that guide does not exist here).
6) Runtime completion locality (PREEMPT_RT threaded IRQs)
---------------------------------------------------------
fio randread 64k, 4 jobs, iodepth=64, 120s, pinned to a single isolated
CPU via a device-plugin extended resource on Kubernetes.
Results:
IOPS : 56.9k
Bandwidth : 3559 MiB/s (3732 MB/s), 417 GiB in 120s
clat : avg 4489us, stdev 30.99us, p99 4555us
total IOs : 6,833,178
While fio was submitting from that isolated CPU, every threaded NVMe
IRQ handler on the host was scheduled on a housekeeping CPU
(SCHED_FIFO, as expected on PREEMPT_RT). `ps -eLo psr,cls,comm |
grep irq.*nvme` during the run:
0 FF irq/195-nvme0q3 0 FF irq/196-nvme0q4 0 FF irq/198-nvme0q6
1 FF irq/187-nvme1q2 1 FF irq/188-nvme1q3 1 FF irq/194-nvme0q2
39 FF irq/185-nvme1q0
40 FF irq/186-nvme1q1 40 FF irq/189-nvme1q4 40 FF irq/193-nvme1q6
41 FF irq/191-nvme1q5 41 FF irq/192-nvme0q1 41 FF irq/197-nvme0q5
79 FF irq/190-nvme0q0
All 14 threaded handlers land on {0,1,39,40,41,79} = HK. Zero
irq/*-nvme* threads scheduled on any of the 74 isolated CPUs.
The /proc/interrupts delta over the 120s run confirms it at the counter
level. Every nvme* IRQ that fired incremented only on an HK CPU:
irq 192 nvme0q1 CPU0 : +6,809,271 (fio's IOs, one HK owner)
irq 187 nvme1q2 CPU1 : +684
irq 186 nvme1q1 CPU0 : +598
irq 189 nvme1q4 CPU40: +568
irq 191 nvme1q5 CPU41: +512
irq 188 nvme1q3 CPU39: +304
irq 193 nvme1q6 CPU79: +271
irq 196 nvme0q4 CPU40: +4
irq 194 nvme0q2 CPU1 : +3
Every single delta is on {0,1,39,40,41,79}. Zero completions on any of
the 74 isolated CPUs. I/O submitted by a task pinned to an isolated
core completes entirely on housekeeping cores.
Still on my list: a negative-control boot with plain managed_irq to
show the confinement disappears. I will follow up with that.
On option 1 vs option 2, my earlier preference stands: keep
managed_irq_strict as an explicit opt-in and leave managed_irq
semantics unchanged, to avoid a silent behavioural change for existing
deployments.
Tested-by: Ionut Nechita <ionut.nechita@windriver.com>
Thanks,
Ionut
On Thu, Sep 10, 2026 at 12:42:28PM -0400, Aaron Tomlin wrote:
> Hi Thomas, Jens, Sebastian, Frederic,
Adding Daniel, Martin, and Hannes on Cc.
> This is version 16 of the series enabling multiqueue devices and managed
> interrupts to adhere to CPU isolation constraints. The series has been
> rebased on v7.3-rc1-708-g893e11787f78.
>
> Background and Problem
> ======================
>
> Modern multiqueue storage controllers (e.g. NVMe devices) typically
> allocate as many hardware queues and interrupt vectors as there are CPUs in
> the system. When CPU isolation is requested via isolcpus, the kernel's
> current mechanism "isolcpus=managed_irq" relies on post-allocation affinity
> steering inside kernel/irq/manage.c. However, because drivers allocate
> queues and vectors before affinity steering takes place, the number of
> vectors frequently matches or exceeds the total CPU count. In such
> scenarios, the post-allocation steering logic cannot avoid assigning
> managed interrupts to isolated CPUs, effectively defeating CPU isolation
> for real-time and latency-critical workloads.
>
> Proposed solution and architecture
> ==================================
>
> This series introduces a new CPU isolation feature,
> "isolcpus=managed_irq_strict", designed to protect isolated cores from the
> disruptive hardware interrupts generated by high-performance multiqueue
> devices.
>
> To prevent I/O stalls, the block layer is additionally hardened to reject
> hotplug requests that attempt to offline a housekeeping CPU if it is the
> last remaining CPU actively serving an online isolated core.
>
> The complex "top-down" mask plumbing introduced in earlier revisions
> (version 12), which modified struct irq_affinity and expanded block layer
> APIs, has been abandoned. It is replaced by a centralised approach: direct
> isolation querying via housekeeping_cpumask(HK_TYPE_MANAGED_IRQ_STRICT)
> within the genirq/affinity subsystem. This architectural simplification
> successfully decouples core changes from driver-specific implementations.
>
>
> In version 15, this was addressed under the previously named
> "isolcpus=io_queue". During subsequent review, Sebastian Andrzej Siewior
> pointed out that the existing "isolcpus=managed_irq" implementation is
> fundamentally broken on modern multiqueue hardware, and raised the question
> of whether we should introduce another flag at all, or instead fix
> "managed_irq" directly in place. As Sebastian observed, if
> backward-compatibility concerns prevent modifying the legacy behaviour, a
> "managed_irq_strict" designation represents the logical counterpart.
>
> To make progress while keeping backwards compatibility completely safe,
> this iteration adopts option 2: we drop the "io_queue" naming in favour of
> "isolcpus=managed_irq_strict". It acts as the strict, pre-allocation
> counterpart to "managed_irq", constraining both block layer queue mappings
> and generic interrupt affinity masks exclusively to housekeeping CPUs.
> Furthermore, if both options are supplied, "managed_irq_strict" takes
> precedence.
>
> Maintainer guidance requested
> =============================
>
> We would appreciate guidance from Frederic Weisbecker regarding preference
> on this front:
> 1. Do you prefer keeping "managed_irq_strict" as an explicit
> opt-in (as implemented in this series) to preserve historical
> behaviour for existing users of "managed_irq"?
>
> 2. Or would you prefer upgrading the semantics of "managed_irq"
> directly in place without introducing a new parameter, on the
> grounds that the legacy post-allocation behaviour is already
> broken for modern multiqueue hardware?
>
>
> Please let me know your thoughts.
>
> Changes since v15:
>
> - Migrated feature naming from "isolcpus=io_queue" (HK_TYPE_IO_QUEUE) to
> "isolcpus=managed_irq_strict" (HK_TYPE_MANAGED_IRQ_STRICT), establishing
> it as the strict, pre-allocation counterpart to "managed_irq"
>
> - Specifying "managed_irq_strict" automatically enables
> HK_FLAG_MANAGED_IRQ so that dependent subsystems continue to function
>
> - Cleaned up commit bodies across the series, stripping informal
> scratchpads and adding Co-developed-by attribution where appropriate
>
> - Added new patch 4/9 ("sched/isolation: Prevent out-of-bounds read in
> isolcpus= boot parameter parser"). Incorporated prerequisite bugfix
> resolving an out-of-bounds read in housekeeping_isolcpus_setup() when
> sub-parameters lack trailing commas
>
> - Defined HK_TYPE_MANAGED_IRQ_STRICT and HK_FLAG_MANAGED_IRQ_STRICT.
> Ensured HK_FLAG_MANAGED_IRQ is set when managed_irq_strict is enabled
>
> - Decoupled queue mapping from dynamic cpu_online_mask; eliminated the
> blk_mq_validate() check and online cpumask snapshotting to avoid TOCTOU
> races during concurrent CPU hotplug events
>
> - Added cpuhp_tasks_frozen check to ensure suspend, hibernation, and
> resume transitions are not aborted
>
> - Added blk_mq_map_queue_type() check so CPU offline validation only
> evaluates hardware contexts actively mapped to the CPU being offlined
>
> - Enhanced warning message to report the offending CPU ID and queue
> number
>
> - Prioritised affd->calc_sets in irq_calc_affinity_vectors() so multi-set
> drivers scale vector allocations with requested set sizes to prevent
> cross-queue contention, while vector spreading remains strictly
> constrained to housekeeping cores
>
> - Documented "managed_irq_strict" in kernel-parameters.txt, detailing its
> role in blk-mq queue distribution and genirq affinity masking, as well
> as its superset relationship with legacy "managed_irq"
>
> - Link to v15: https://lore.kernel.org/lkml/20260521232956.553287-1-atomlin@atomlin.com/
>
> Changes since v14:
>
> - Fixed a division-by-zero by ensuring group_mask_cpus_evenly() safely
> frees its allocations and returns NULL instead of an empty array if the
> provided mask yields zero groups.
>
> - Fixed a device probe -ENOSPC regression in blk_mq_num_queues(). If the
> housekeeping mask intersection evaluated to 0 (e.g., against a localised
> NUMA node), min_not_zero() would erroneously return the absolute maximum
> hardware queues. The result is now safely clamped to a minimum of 1.
>
> - Added a mapping verification check to prevent unrelated housekeeping
> CPUs from aborting the global hotplug offline sequence.
>
> - Aligned the pr_warn format specifier with the unsigned int declaration
> of hctx->queue_num in blk_mq_hctx_can_offline_hk_cpu().
>
> - Link to v14: https://lore.kernel.org/lkml/20260520215030.496803-1-atomlin@atomlin.com/
>
> Changes since v13:
>
> - Removed ineffective data_race() annotations around mask and
> cpu_present_mask pointers. Wrapping the pointers failed to suppress
> KCSAN warnings for the underlying inline bitmap memory accesses.
>
> - Fixed a silent validation bypass in blk_mq_map_hw_queues() caused by
> overlapping IRQ affinity masks by removing the short-circuiting
> optimisation and evaluating the active_hctx bitmap in a secondary pass.
>
> - Restored topology-aware multi-queue fallback by correctly routing
> missing IRQ affinity masks to the map_software path instead of the naive
> map-all fallback.
>
> - Dropped hctx->queue->disk->disk_name from warning to avoid a UAF.
>
> - Fixed an isolation leak where excess allocated hardware queues were
> improperly padded with irq_default_affinity. Because these queues are
> marked as managed, they bypassed user-space IRQ balancing; they are now
> safely padded with the housekeeping mask.
>
> - Enforced the housekeeping vector cap prior to evaluating driver-provided
> calc_sets() callbacks, preventing modern multi-queue drivers from
> bypassing the cap and wasting memory on dead queues.
>
> - Introduced a safety net to the vector calculation to prevent fatal
> -ENOSPC device probe aborts on heavily isolated systems where the
> housekeeping CPU count is lower than the device's structural minimum.
>
> - Removed an inaccurate claim stating that the io_queue isolation flag
> takes precedence over managed_irq. Both flags are parsed, evaluated, and
> enforced entirely independently by their respective subsystems.
>
> - Link to v13: https://lore.kernel.org/lkml/20260513005509.135966-1-atomlin@atomlin.com/
>
> Changes since v12:
>
> - Resolved TOCTOU race conditions against CPU hotplug events in
> blk_mq_map_queues() and group_mask_cpus_evenly() by taking lockless
> snapshots of the online CPU mask prior to algorithmic evaluation.
>
> - Migrated the active_hctx tracking to a dynamically sized bitmap
> (bitmap_zalloc), resolving a critical out-of-bounds memory write that
> occurred when hardware queues exceeded the system CPU count.
>
> - Wrapped the disk pointer fetch in blk_mq_hctx_can_offline_hk_cpu() with
> READ_ONCE() to prevent a TOCTOU NULL pointer dereference against
> concurrent device teardowns.
>
> - Introduced bitmap_empty() checks to prevent the mapping logic from
> routing unassigned CPUs into unallocated memory when all mapped CPUs are
> offline, safely forcing a fallback mapping instead.
>
> - Implemented a native two-stage distribution logic in
> group_mask_cpus_evenly() that first prioritises physically present CPUs
> to prevent I/O starvation before distributing remaining vectors to
> non-present CPUs for hotplug safety.
>
> - Restricted the maximum number of allocated vectors in
> irq_calc_affinity_vectors() to the weight of the housekeeping mask,
> preventing drivers from wasting memory on dead hardware queues that
> physically cannot be routed.
>
> - Added padding logic using irq_default_affinity for sets where isolation
> constraints yield fewer masks than requested vectors, preserving the 1:1
> hardware queue mapping sequence for subsequent sets.
>
> - Fixed a logic flaw that prematurely rejected valid offline requests by
> manually iterating over cpu_online_mask and reverse-mapping to
> accurately detect isolated CPUs, properly permitting the offlining of
> non-housekeeping CPUs.
>
> - Corrected an absolute versus relative queue index calculation bug in
> blk_mq_map_queues() that was overwriting loop iterations, by iterating
> directly over the generated masks.
>
> - Replaced scoped __free cleanups with traditional goto unwinding in the
> block layer to align with subsystem styling guidelines.
>
> - Refined the io_queue kernel command-line parameter documentation for
> better clarity and precision.
>
> - Link to v12: https://lore.kernel.org/lkml/20260422185215.100929-1-atomlin@atomlin.com/
>
> Changes since v11:
>
> - Removed duplicate paragraph from the commit message in patch 11
> (Marco Crivellari)
>
> - Ensure ZERO_SIZE_PTR is not returned by group_mask_cpus_evenly()
> (Marco Crivellari)
>
> - Link to v11: https://lore.kernel.org/lkml/20260416192942.1243421-1-atomlin@atomlin.com/
>
> Changes since v10:
>
> - Completely rewrote the isolcpus=io_queue documentation in
> Documentation/admin-guide/kernel-parameters.txt to clarify its exclusive
> application to managed IRQs, queue allocation limits, vector exhaustion
> prevention, and hardware interrupt routing (Ming Lei)
>
> - Fixed a stack frame bloat issue by avoiding the on-stack declaration of
> struct cpumask (Waiman Long)
>
> - Link to v10: https://lore.kernel.org/linux-nvme/20260401222312.772334-1-atomlin@atomlin.com/
>
> Changes since v9:
>
> - Fixed a page fault regression encountered when initialising secondary
> queue maps (e.g. NVMe poll queues). Restored the qmap->queue_offset to
> the mq_map assignment to ensure CPUs are strictly mapped to absolute
> hardware indices (Keith Busch)
>
> - Corrected the active_hctx tracker to utilise relative queue indices,
> preventing out-of-bounds mask assignments
>
> - Fixed the blk_mq_validate() sanity check to properly evaluate absolute
> queue indices against the offset-adjusted loop index
>
> - Corrected typographical errors within block/blk-mq-cpumap.c
> (Keith Busch)
>
> - Clarified the commit message regarding the removal of the !SMP fallback
> code, explicitly noting that the core scheduler now mandates SMP
> unconditionally (Sebastian Andrzej Siewior)
>
> - Added missing "Signed-off-by:" tags to properly record the patch series
> chain of custody
>
> - Link to v9: https://lore.kernel.org/lkml/20260330221047.630206-1-atomlin@atomlin.com/
>
> Changes since v8:
>
> - Added "Reviewed-by:" tags
>
> - Introduced irq_spread_hk_filter() to safely restrict managed IRQ
> affinity to housekeeping CPUs (Thomas Gleixner)
>
> - Removed the unsafe global static variable blk_hk_online_mask from
> blk-mq-cpumap.c and blk-mq.c. blk_mq_online_queue_affinity() now returns
> a stable pointer, delegating safe intersection to the callers to prevent
> concurrent modification races (Thomas Gleixner, Hannes Reinecke)
>
> - Resolved BUG: kernel NULL pointer dereference in __blk_mq_all_tag_iter
> reported by the kernel test robot during cpuhotplug rcutorture stress
> testing
>
> - Link to v8: https://lore.kernel.org/lkml/20250905-isolcpus-io-queues-v8-0-885984c5daca@kernel.org/
>
> Changes since v7:
>
> - Added commit 524f5eea4bbe ("lib/group_cpus: remove !SMP code")
>
> - Merged the new mapping logic directly into the existing function to
> avoid special casing
>
> - Refined the group_mask_cpus_evenly() implementation with the following
> updates:
>
> - Corrected the function name typo (changed group_masks_cpus_evenly to
> group_mask_cpus_evenly)
>
> - Updated the documentation comment to accurately reflect the function's
> behavior
>
> - Renamed the cpu_mask argument to mask for consistency
>
> - Added a new patch for aacraid to include the missing number of queues
> calculation
>
> - Restricted updates to only affect SCSI drivers that support
> PCI_IRQ_AFFINITY and do not utilise nvme-fabrics
>
> - Removed the __free cleanup attribute usage for cpumask_var_t allocations
> due to compatibility issues
>
> - Updated the documentation to explicitly highlight the limitations
> surrounding CPU offlining
>
> - Collected accumulated Reviewed-by and Acked-by tags
>
> - Link to v7: https://patch.msgid.link/20250702-isolcpus-io-queues-v7-0-557aa7eacce4@kernel.org
>
> Changes since v6:
>
> - Sent out the first part of the series independently:
> https://lore.kernel.org/all/20250617-isolcpus-queue-counters-v1-0-13923686b54b@kernel.org/
>
> - Added comprehensive kernel command-line documentation
>
> - Added validation logic to ensure the resulting CPU-to-queue mapping is
> fully operational
>
> - Rewrote the isolcpus mapping code to properly account for active
> hardware contexts (hctx)
>
> - Introduced blk_mq_map_hk_irq_queues, which utilizes the mask retrieved
> from irq_get_affinity()
>
> - Refactored blk_mq_map_hk_queues to require the caller to explicitly test
> for HK_TYPE_MANAGED_IRQ
>
> - Link to v6: https://patch.msgid.link/20250424-isolcpus-io-queues-v6-0-9a53a870ca1f@kernel.org
>
> Changes since v5:
>
> - Reintroduced the io_queue type for the isolcpus kernel parameter
>
> - Prevented the offlining of a housekeeping CPU if an isolated CPU is
> still present, upgrading this behavior from a simple warning to a hard
> restriction
>
> - Link to v5: https://lore.kernel.org/r/20250110-isolcpus-io-queues-v5-0-0e4f118680b0@kernel.org
>
> Changes since v4:
>
> - Rebased the series onto the latest for-6.14/block branch.
>
> - Updated the documentation regarding the managed_irq parameters
>
> - Reworded the commit message for "blk-mq: issue warning when offlining
> hctx with online isolcpus" for better clarity
>
> - Split the input and output parameters in the patch "lib/group_cpus: let
> group_cpu_evenly return number of groups"
>
> - Dropped the patch "sched/isolation: document HK_TYPE housekeeping
> option"
>
> - Link to v4: https://lore.kernel.org/r/20241217-isolcpus-io-queues-v4-0-5d355fbb1e14@kernel.org
>
> Changes since v3:
>
> - Added the patch "blk-mq: issue warning when offlining hctx with online
> isolcpus"
>
> - Fixed the check in group_cpus_evenly(); the condition now properly uses
> housekeeping_enabled() instead of cpumask_weight(), as the latter always
> returns a valid mask
>
> - Dropped the Fixes: tag from "lib/group_cpus.c: honor housekeeping config
> when grouping CPUs"
>
> - Fixed an overlong line warning in the patch "scsi: use block layer
> helpers to calculate num of queues"
>
> - Dropped the patch "sched/isolation: Add io_queue housekeeping option" in
> favor of simply documenting the housekeeping hk_type enum
>
> - Added the patch "lib/group_cpus: let group_cpu_evenly return number of
> groups"
>
> - Collected accumulated Reviewed-by and Acked-by tags
>
> - Split the patchset by moving foundational changes into a separate
> preparation series:
> https://lore.kernel.org/linux-nvme/20241202-refactor-blk-affinity-helpers-v6-0-27211e9c2cd5@kernel.org/
>
> - Link to v3: https://lore.kernel.org/r/20240806-isolcpus-io-queues-v3-0-da0eecfeaf8b@suse.de
>
> Changes since v2:
>
> - Integrated patches from Ming Lei
> (https://lore.kernel.org/all/20210709081005.421340-1-ming.lei@redhat.com/):
> "virtio: add APIs for retrieving vq affinity" and "blk-mq: introduce
> blk_mq_dev_map_queues"
>
> - Replaced all instances of blk_mq_pci_map_queues and
> blk_mq_virtio_map_queues with the new unified blk_mq_dev_map_queues
>
> - Updated and expanded the helper functions used for calculating the
> number of queues
>
> - Added the CPU-to-hctx mapping function specifically to support the
> isolcpus=io_queue parameter
>
> - Documented the hk_type enum and the newly introduced isolcpus=io_queue
> parameter
>
> - Added the patch "scsi: pm8001: do not overwrite PCI queue mapping"
>
> - Link to v2: https://lore.kernel.org/r/20240627-isolcpus-io-queues-v2-0-26a32e3c4f75@suse.de
>
> Changes since v1:
>
> - Updated the feature documentation for clarity and completeness
>
> - Split the blk/nvme-pci patch into smaller, logical commits
>
> - Dropped the HK_TYPE_IO_QUEUE macro in favor of reusing
> HK_TYPE_MANAGED_IRQ
>
> - Link to v1: https://lore.kernel.org/r/20240621-isolcpus-io-queues-v1-0-8b169bf41083@suse.de
>
> Aaron Tomlin (2):
> sched/isolation: Prevent out-of-bounds read in isolcpus= boot
> parameter parser
> genirq/affinity: Restrict managed IRQ affinity to housekeeping CPUs
>
> Daniel Wagner (7):
> scsi: aacraid: use block layer helpers to calculate num of queues
> lib/group_cpus: remove dead !SMP code
> lib/group_cpus: Add group_mask_cpus_evenly()
> isolation: Introduce managed_irq_strict isolcpus type
> blk-mq: use hk cpus only when isolcpus=managed_irq_strict is enabled
> blk-mq: prevent offlining hk CPUs with associated online isolated CPUs
> docs: add managed_irq_strict flag to isolcpus
>
> .../admin-guide/kernel-parameters.txt | 37 +++-
> Documentation/core-api/housekeeping.rst | 6 +-
> block/blk-mq-cpumap.c | 163 ++++++++++++++++--
> block/blk-mq.c | 63 +++++++
> drivers/scsi/aacraid/comminit.c | 3 +-
> include/linux/group_cpus.h | 3 +
> include/linux/sched/isolation.h | 1 +
> kernel/irq/affinity.c | 29 +++-
> kernel/sched/isolation.c | 18 ++
> lib/group_cpus.c | 112 ++++++++++--
> 10 files changed, 395 insertions(+), 40 deletions(-)
>
> --
> 2.55.0
>
--
Aaron Tomlin
© 2016 - 2026 Red Hat, Inc.