From nobody Fri Nov 29 18:34:48 2024 Received: from mail-wm1-f47.google.com (mail-wm1-f47.google.com [209.85.128.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 44B0714B094; Mon, 16 Sep 2024 12:23:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726489403; cv=none; b=Tl9zrrHPgWrckzz2esg3RpM4cBircyoGhaJJL0wq0LygoHz1u2keBnnlxfol8L4fafUA3rxr1b8tHfGM93uGIFVCCbUV7clSyLD4+bHC6tMB2XZtxVPvsucQbkKPb5KY3zx5nXOi7xAySv+HDtTi8eDAl2a65IqfLEo1teWy0h4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726489403; c=relaxed/simple; bh=SxLsXlxTP2H5dHNDB2XckZNzj6N1DE/5WRZf4FZZvsE=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Pbd+a9OQtMRyZUNOCX0bGjGMgGkkspCOEYV0pqQqz1XtgLzZjWdWRHR6SHcSWP6CZkaehsWwEcQXkM4zGK1l3LNQkR5scdvITzRyS5wYg9BgxrPCGs1WhhnXt2p7XLjbRxXDSjyAjFAuyhQSe9Tcigtmy47f+UZAGkpK0fWqdrU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=gmail.com; arc=none smtp.client-ip=209.85.128.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-wm1-f47.google.com with SMTP id 5b1f17b1804b1-42cb58d810eso33802115e9.0; Mon, 16 Sep 2024 05:23:20 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726489399; x=1727094199; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=4XP7JYTPswuC1MNC97EWdOwUS12t/7UGSoTDEE9+Gmw=; b=C+hgv4K/NN+msVfa3b8k3R4UuWMw0OrEMIaZrZKcLIb9Zv21XFRsfdd3AhQT6xtT3X 4Hlsz9k25T+6iZqmA2E2cjJmm3eJFTqsiiHx44HFCWZwjPGVZuw4f9FKusLf3gs1I2gl 8q8FlUq8LjmibKwCipBOpR9MHZu5FeGEdGCwYKAOp1290fFI1op07/7Yp3bDx27kLV+0 A+ujwXQTZyYHzVx5HmgIZygcfhpSG5iypJbziY7SmidHndCB/OMARuGz/UBoB/X5Es3a VQCe0gC7ymwMl7a2z87aTbHQk7AhfQrsiIyPZm9m7iK/DZrXzUPo9w6cByP1mHcOdrBo hZAA== X-Forwarded-Encrypted: i=1; AJvYcCWjjH04tq5GUaqCVWdoIROU5oEkp16TiIbTS4m/Z62g9sCc6bIuycvlx/uTOMp5/0hzuWBH7Bai@vger.kernel.org, AJvYcCXurA7z4eD2XwaEr9ZmqGWuwzlC1v9GTamN9DVg9aoxR7z2zBuQJDyDHOxZsinbCAoOIV/ju2F7vBc9EEcB@vger.kernel.org X-Gm-Message-State: AOJu0YxJUlVde4kSxNsWKmT8RkaVkxw4LMNUtoIn7DZZy3KtW8YRaKZV GhKHqTd/J0skVIN7deTxmhAorf0hnRobVP1NgnX9IfQpuHzTdFI1 X-Google-Smtp-Source: AGHT+IFNI4iYpjMKYAcaWjWoU/iNfgpzX74OJKJcuWhd59WCiBCwFyoNGw6iT8rIvbAFpRO0GYaD9w== X-Received: by 2002:a5d:4d51:0:b0:374:d2a3:d213 with SMTP id ffacd0b85a97d-378c27b4d25mr8711614f8f.18.1726489399460; Mon, 16 Sep 2024 05:23:19 -0700 (PDT) Received: from costa-tp.. ([2a00:a041:e281:f300:ddd7:8878:b93d:7c0b]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-378e7800243sm7117272f8f.86.2024.09.16.05.23.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Sep 2024 05:23:18 -0700 (PDT) From: Costa Shulyupin To: longman@redhat.com, ming.lei@redhat.com, pauld@redhat.com, juri.lelli@redhat.com, vschneid@redhat.com, Michael Ellerman , Nicholas Piggin , Christophe Leroy , Naveen N Rao , Thomas Gleixner , Zefan Li , Tejun Heo , Johannes Weiner , =?UTF-8?q?Michal=20Koutn=C3=BD?= , Ingo Molnar , Peter Zijlstra , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Costa Shulyupin , Bjorn Helgaas , linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org Subject: [RFC PATCH v3 1/3] sched/isolation: Add infrastructure for dynamic CPU isolation Date: Mon, 16 Sep 2024 15:20:42 +0300 Message-ID: <20240916122044.3056787-2-costa.shul@redhat.com> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20240916122044.3056787-1-costa.shul@redhat.com> References: <20240916122044.3056787-1-costa.shul@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Introduce infrastructure function housekeeping_update() to change housekeeping_cpumask during runtime and adjust configurations of depended subsystems. Configuration adjustments of subsystems follow in subsequent patches. Parent patch: sched/isolation: Exclude dynamically isolated CPUs from housekeeping masks https://lore.kernel.org/lkml/20240821142312.236970-1-longman@redhat.com/ Signed-off-by: Costa Shulyupin --- Changes in v2: - remove unnecessary `err` variable - add for_each_clear_bit... to clear isolated CPUs - Address Gleixner's comments: - use WRITE_ONCE to change housekeeping.flags - use `struct cpumask *update` in signature of housekeeping_update v1: - https://lore.kernel.org/lkml/20240516190437.3545310-2-costa.shul@redhat.c= om/ --- kernel/sched/isolation.c | 38 +++++++++++++++++++++++++++++++++++--- 1 file changed, 35 insertions(+), 3 deletions(-) diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c index 3018ba81eb65d..3f24921b929a0 100644 --- a/kernel/sched/isolation.c +++ b/kernel/sched/isolation.c @@ -124,6 +124,36 @@ static void __init housekeeping_setup_type(enum hk_typ= e type, housekeeping_staging); } =20 +/* + * housekeeping_update - change housekeeping.cpumasks[type] and propagate = the + * change. + */ +static int housekeeping_update(enum hk_type type, const struct cpumask *up= date) +{ + struct { + struct cpumask changed; + struct cpumask enable; + struct cpumask disable; + } *masks; + + masks =3D kmalloc(sizeof(*masks), GFP_KERNEL); + if (!masks) + return -ENOMEM; + + lockdep_assert_cpus_held(); + cpumask_xor(&masks->changed, housekeeping_cpumask(type), update); + cpumask_and(&masks->enable, &masks->changed, update); + cpumask_andnot(&masks->disable, &masks->changed, update); + cpumask_copy(housekeeping.cpumasks[type], update); + WRITE_ONCE(housekeeping.flags, housekeeping.flags | BIT(type)); + if (!static_branch_unlikely(&housekeeping_overridden)) + static_key_enable_cpuslocked(&housekeeping_overridden.key); + + kfree(masks); + + return 0; +} + static int __init housekeeping_setup(char *str, unsigned long flags) { cpumask_var_t non_housekeeping_mask, housekeeping_staging; @@ -327,8 +357,11 @@ int housekeeping_exlude_isolcpus(const struct cpumask = *isolcpus, unsigned long f /* * Reset housekeeping to bootup default */ + + for_each_clear_bit(type, &boot_hk_flags, HK_TYPE_MAX) + housekeeping_update(type, cpu_possible_mask); for_each_set_bit(type, &boot_hk_flags, HK_TYPE_MAX) - cpumask_copy(housekeeping.cpumasks[type], boot_hk_cpumask); + housekeeping_update(type, boot_hk_cpumask); =20 WRITE_ONCE(housekeeping.flags, boot_hk_flags); if (!boot_hk_flags && static_key_enabled(&housekeeping_overridden)) @@ -355,9 +388,8 @@ int housekeeping_exlude_isolcpus(const struct cpumask *= isolcpus, unsigned long f cpumask_andnot(tmp_mask, src_mask, isolcpus); if (!cpumask_intersects(tmp_mask, cpu_online_mask)) return -EINVAL; /* Invalid isolated CPUs */ - cpumask_copy(housekeeping.cpumasks[type], tmp_mask); + housekeeping_update(type, tmp_mask); } - WRITE_ONCE(housekeeping.flags, boot_hk_flags | flags); excluded =3D true; if (!static_key_enabled(&housekeeping_overridden)) static_key_enable_cpuslocked(&housekeeping_overridden.key); --=20 2.45.0 From nobody Fri Nov 29 18:34:48 2024 Received: from mail-wm1-f42.google.com (mail-wm1-f42.google.com [209.85.128.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 687B815749A; Mon, 16 Sep 2024 12:23:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726489428; cv=none; b=bFEOUdpVG9SyT0Gzc/pAY0Kx/e5l37O86hXy8aABzU6Gcj8E8NuXXF9XPy23byvHIH7Byt5O8r6rBnptEHQqMXXyeBBVpUmOuGZvaY0LrbDakRZjpfqcEcuECh80NgcWwLnbnqLFjrb+N45mX+vqzI9fVvCASOIGseNGucL4AjY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726489428; c=relaxed/simple; bh=P7LkNy+NZSM2MrtRwNsi5xmMBEBwBhE+EM+OKKDCbyA=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gqKpRocMNnEwOuqqk3G0z2e+1sDoaQbvkt0eJzKaFhGIBGCuiQz1H3ApsSZbyIVvnkTyo+E10pgComo3OdyojDEfuoIDen67iikr4ISdygeX8jbkLvJYAOEBHl1XksWH671k+G0KFzuR2gdpYcRPjDXru1RnsDomUIXrHMpfpc8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=gmail.com; arc=none smtp.client-ip=209.85.128.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-wm1-f42.google.com with SMTP id 5b1f17b1804b1-42cb1758e41so25771195e9.1; Mon, 16 Sep 2024 05:23:46 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726489425; x=1727094225; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dDfojy5CrgHlgs0fxt38qSv2a++PUan1QC4a76dOnDc=; b=Jh1gqQ6xJ/fOufl1ZEIfxCwXW2+IwvDCljBHDirgOqkIx7mpQaHQwRRI1gAxnQGD/d SwgsbhmKZr2WF6pgf20N6stLqkn6HHlutkSatBHq6h1jVOmGPKWUeR06O56oMjxON8Mu nddAHn5n5D5VV6aMWXasmBpXPyTHZ76g+edUzi7tfTzKONoa8/ySJ8m6Aky9Cs8Ii5F1 RMJiILtwULEyeQOH5QnOePuiOztiOoWYjRuL1MS1YCceqMEM/DM9yqguAwPqIJNEWIeM CPiXrJFtzu/B7XI6TUaJu2B5TKSClrwx0x+vW1y20oAwVNrPrH8OSr81huP/6Kh98wVq vy/g== X-Forwarded-Encrypted: i=1; AJvYcCVymmR+XW2o0Hp1Q1jFN4u+hVQEIKQ01BvQJhlIHKf76s6ad1yzv/eiDglFKOMDN4AC+9YpdDqZmXw4M+Md@vger.kernel.org, AJvYcCWKP8KQNlSBxQ9gTNY9+vtCru7nIT8rXOa8K0RviQtduWLFYzBhaMb6C+E0BflPNv5ZywSsi0xM@vger.kernel.org X-Gm-Message-State: AOJu0YxGbfUEVEAs0gMVPNOgzkMuZPMI6zeYllZVME9oeLnZ//ABL8xL RywgkbXEPe4BSt2BwKGGI1420xSD5eNaqVSp+X5uu2ReelWuDTRf X-Google-Smtp-Source: AGHT+IFx/jTnwae8GjD9s+HXc1nBltwBGmeNBzQI1ep4kLkettAY/xLPtxREJvJrF8iUt004m0leaQ== X-Received: by 2002:a5d:4a52:0:b0:374:bde6:d375 with SMTP id ffacd0b85a97d-378d61f0d54mr5541099f8f.29.1726489424339; Mon, 16 Sep 2024 05:23:44 -0700 (PDT) Received: from costa-tp.. ([2a00:a041:e281:f300:ddd7:8878:b93d:7c0b]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-378e7800243sm7117272f8f.86.2024.09.16.05.23.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Sep 2024 05:23:43 -0700 (PDT) From: Costa Shulyupin To: longman@redhat.com, ming.lei@redhat.com, pauld@redhat.com, juri.lelli@redhat.com, vschneid@redhat.com, Michael Ellerman , Nicholas Piggin , Christophe Leroy , Naveen N Rao , Thomas Gleixner , Zefan Li , Tejun Heo , Johannes Weiner , =?UTF-8?q?Michal=20Koutn=C3=BD?= , Ingo Molnar , Peter Zijlstra , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Costa Shulyupin , Bjorn Helgaas , linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org Subject: [RFC PATCH v3 2/3] genirq/cpuhotplug: Adjust managed irqs according to change of housekeeping CPU Date: Mon, 16 Sep 2024 15:20:43 +0300 Message-ID: <20240916122044.3056787-3-costa.shul@redhat.com> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20240916122044.3056787-1-costa.shul@redhat.com> References: <20240916122044.3056787-1-costa.shul@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Interrupts disturb real-time tasks on affined cpus. To ensure CPU isolation for real-time tasks, interrupt handling must be adjusted accordingly. Non-managed interrupts can be configured from userspace, while managed interrupts require adjustments in kernelspace. Adjust status of managed interrupts according change of housekeeping CPUs to support dynamic CPU isolation. Signed-off-by: Costa Shulyupin --- The following code is a proof of concept to validate and review the correctness of the approach to solving the problem. C++ comments denote temporary comments. v2: - refactor irq_affinity_adjust(): - add more comments - add managed_irq_isolate() derived from migrate_one_irq(), irq_needs_fixup() and irq_fixup_move_pending() - use irq_set_affinity() instead of irq_set_affinity_locked - Addressed Gleixner's comments: - use `struct cpumask *` instead of `cpumask_var_t` in function signature - remove locking in irq_affinity_adjust() v1: - https://lore.kernel.org/lkml/20240516190437.3545310-5-costa.shul@redhat.c= om/ --- include/linux/irq.h | 2 + kernel/cgroup/cpuset.c | 1 + kernel/irq/cpuhotplug.c | 95 ++++++++++++++++++++++++++++++++++++++++ kernel/sched/isolation.c | 10 ++++- 4 files changed, 107 insertions(+), 1 deletion(-) diff --git a/include/linux/irq.h b/include/linux/irq.h index fa711f80957b6..4eb2e765dbd95 100644 --- a/include/linux/irq.h +++ b/include/linux/irq.h @@ -619,6 +619,8 @@ extern int irq_affinity_online_cpu(unsigned int cpu); # define irq_affinity_online_cpu NULL #endif =20 +int managed_irq_affinity_adjust(struct cpumask *enable_mask); + #if defined(CONFIG_SMP) && defined(CONFIG_GENERIC_PENDING_IRQ) void __irq_move_irq(struct irq_data *data); static inline void irq_move_irq(struct irq_data *data) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index bf60bdc973dd6..73b06b2cd91e3 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -90,6 +90,7 @@ static struct list_head remote_children; #define HOUSEKEEPING_FLAGS (BIT(HK_TYPE_TIMER) | BIT(HK_TYPE_RCU) |\ BIT(HK_TYPE_SCHED) | BIT(HK_TYPE_MISC) |\ BIT(HK_TYPE_DOMAIN) | BIT(HK_TYPE_WQ) |\ + BIT(HK_TYPE_MANAGED_IRQ) |\ BIT(HK_TYPE_KTHREAD)) =20 /* diff --git a/kernel/irq/cpuhotplug.c b/kernel/irq/cpuhotplug.c index ec2cdcd20bee7..adbe1f3e5bd22 100644 --- a/kernel/irq/cpuhotplug.c +++ b/kernel/irq/cpuhotplug.c @@ -252,3 +252,98 @@ int irq_affinity_online_cpu(unsigned int cpu) =20 return 0; } + +/* + * managed_irq_isolate() - Deactivate managed interrupts if necessary + */ +// derived from migrate_one_irq, irq_needs_fixup, irq_fixup_move_pending +static int managed_irq_isolate(struct irq_desc *desc) +{ + struct irq_data *d =3D irq_desc_get_irq_data(desc); + struct irq_chip *chip =3D irq_data_get_irq_chip(d); + const struct cpumask *a; + bool maskchip; + int err; + + /* + * Deactivate if: + * - Interrupt is managed + * - Interrupt is not per cpu + * - Interrupt is started + * - Effective affinity mask includes isolated CPUs + */ + if (!irqd_affinity_is_managed(d) || irqd_is_per_cpu(d) || !irqd_is_starte= d(d) + || cpumask_subset(irq_data_get_effective_affinity_mask(d), + housekeeping_cpumask(HK_TYPE_MANAGED_IRQ))) + return 0; + // TBD: it is required? + /* + * Complete an eventually pending irq move cleanup. If this + * interrupt was moved in hard irq context, then the vectors need + * to be cleaned up. It can't wait until this interrupt actually + * happens and this CPU was involved. + */ + irq_force_complete_move(desc); + + if (irqd_is_setaffinity_pending(d)) { + irqd_clr_move_pending(d); + if (cpumask_intersects(desc->pending_mask, + housekeeping_cpumask(HK_TYPE_MANAGED_IRQ))) + a =3D irq_desc_get_pending_mask(desc); + } else { + a =3D irq_data_get_affinity_mask(d); + } + + maskchip =3D chip->irq_mask && !irq_can_move_pcntxt(d) && !irqd_irq_maske= d(d); + if (maskchip) + chip->irq_mask(d); + + if (!cpumask_intersects(a, housekeeping_cpumask(HK_TYPE_MANAGED_IRQ))) { + /* + * Shut managed interrupt down and leave the affinity untouched. + * The effective affinity is reset to the first online CPU. + */ + irqd_set_managed_shutdown(d); + irq_shutdown_and_deactivate(desc); + return 0; + } + + /* + * Do not set the force argument of irq_do_set_affinity() as this + * disables the masking of offline CPUs from the supplied affinity + * mask and therefore might keep/reassign the irq to the outgoing + * CPU. + */ + err =3D irq_do_set_affinity(d, a, false); + if (err) + pr_warn_ratelimited("IRQ%u: set affinity failed(%d).\n", + d->irq, err); + + if (maskchip) + chip->irq_unmask(d); + + return err; +} + +/** managed_irq_affinity_adjust() - Deactivate of restore managed interrup= ts + * according to change of housekeeping cpumask. + * + * @enable_mask: CPUs for which interrupts should be restored + */ +int managed_irq_affinity_adjust(struct cpumask *enable_mask) +{ + unsigned int irq; + + for_each_active_irq(irq) { + struct irq_desc *desc =3D irq_to_desc(irq); + unsigned int cpu; + + for_each_cpu(cpu, enable_mask) + irq_restore_affinity_of_irq(desc, cpu); + raw_spin_lock(&desc->lock); + managed_irq_isolate(desc); + raw_spin_unlock(&desc->lock); + } + + return 0; +} diff --git a/kernel/sched/isolation.c b/kernel/sched/isolation.c index 3f24921b929a0..cd72300ec8b99 100644 --- a/kernel/sched/isolation.c +++ b/kernel/sched/isolation.c @@ -130,6 +130,8 @@ static void __init housekeeping_setup_type(enum hk_type= type, */ static int housekeeping_update(enum hk_type type, const struct cpumask *up= date) { + int err =3D 0; + struct { struct cpumask changed; struct cpumask enable; @@ -149,9 +151,15 @@ static int housekeeping_update(enum hk_type type, cons= t struct cpumask *update) if (!static_branch_unlikely(&housekeeping_overridden)) static_key_enable_cpuslocked(&housekeeping_overridden.key); =20 + switch (type) { + case HK_TYPE_MANAGED_IRQ: + err =3D managed_irq_affinity_adjust(&masks->enable); + break; + default: + } kfree(masks); =20 - return 0; + return err; } =20 static int __init housekeeping_setup(char *str, unsigned long flags) --=20 2.45.0 From nobody Fri Nov 29 18:34:48 2024 Received: from mail-wm1-f54.google.com (mail-wm1-f54.google.com [209.85.128.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 78B2A154BEB; Mon, 16 Sep 2024 12:24:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726489456; cv=none; b=MYapoJrLxCTEAVEEUlROxDxq+PEYpCGwLTRWdabnOQ0MDdQmtwQicLexTsa5T8Ri8RPwFE9FCAYX/BptuHnIgBpoivSCvxQgjQ8u253BteUxZk0WoPiAfFQzZ8ZQxy74I6Jv+QFi0i10wfhPyJjHUgQ41fR0MXH0Kjq4leto4Js= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1726489456; c=relaxed/simple; bh=MpCSgizxeCsUjdDKdGBrpG5aByiQairA6fYbS2P/elU=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KJ1mEGgt9cAKtajLVkXTij2pzZwD34MEGNpqP76FZ+zcmjjesQWEMohvSq0nO4l809CUje5j3JTB3lQXiJV/3+rO8gzJjgN5ZlDIgZ3JJ636A3KyHSih4P4gAO759pcFBJp8Hq7Sgij7jFyVraWcKuH4JoZomeKRDwkXbUfVvD0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=gmail.com; arc=none smtp.client-ip=209.85.128.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-wm1-f54.google.com with SMTP id 5b1f17b1804b1-42cacabd2e0so37266585e9.3; Mon, 16 Sep 2024 05:24:14 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1726489453; x=1727094253; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=UIQVZrqwrUMgVJOuTjxMJrg0tEe4W9lzFWo0vuBjDGA=; b=IOcjupPtp7pdRqbGrxbCDUTDF3d3gopD4q7mrlQgOnNulf+TaCr6QKsY4/hO3u9zoH tt0+AaeT0GuBg8f2Q944dlv3Ir9UQu83yUiqenR6JIiv7uR/gW5cSuUrW+9wjh552LqF KmnN/wpREs8aWbkJFphhj8jh4GmEFBRefvqNwVYn+KYx04dbewGssbncm/PpNN5Nu3yG XFk8Vz8ye/vNArBZN2+SpabpUD3kymto66W4poalG6qRjvBTmDdIVVLbZCYTq06SyF5k /IlQbMI6Mcx9mHffeFXQP7Y74Zy/EzeE5ZVessBD2/C0BKjjiK3SgYNDrzPo8/EJHsNn dzTg== X-Forwarded-Encrypted: i=1; AJvYcCVWYnWWAMV1b4rKFgKr+btXQZbtXH3vHwDtK/9K2U5toOGILF/3dyZ12Npga1qfXM0bqX1sYG3d@vger.kernel.org, AJvYcCWzJROgsOAw+MRUoPTIqLOkILUM7J+ZmZSTVxfi0VawNH0aE8A6IRg4m+6wfHBi3EMLiDA1oqu0MEIMeRpV@vger.kernel.org X-Gm-Message-State: AOJu0Yz7LDllsDE4ZSQHc5MhrzPE42HCOVbFqX1NMeiWa47C9cR823+B HgtNUir+iHUTVJwMOFSNf41diYOtumPfG0FMzkUvelaLpRwSD3qB X-Google-Smtp-Source: AGHT+IE0DmenEPK9NwFC2424pKeGC0BWrR50LQs1egXEXs39Z4HiNTrZQssjk5k4Z07jJC9ekhD5yg== X-Received: by 2002:a05:600c:2058:b0:42c:de9b:a1b5 with SMTP id 5b1f17b1804b1-42cde9ba226mr106150255e9.32.1726489452681; Mon, 16 Sep 2024 05:24:12 -0700 (PDT) Received: from costa-tp.. ([2a00:a041:e281:f300:ddd7:8878:b93d:7c0b]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-378e7800243sm7117272f8f.86.2024.09.16.05.24.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 16 Sep 2024 05:24:12 -0700 (PDT) From: Costa Shulyupin To: longman@redhat.com, ming.lei@redhat.com, pauld@redhat.com, juri.lelli@redhat.com, vschneid@redhat.com, Michael Ellerman , Nicholas Piggin , Christophe Leroy , Naveen N Rao , Thomas Gleixner , Zefan Li , Tejun Heo , Johannes Weiner , =?UTF-8?q?Michal=20Koutn=C3=BD?= , Ingo Molnar , Peter Zijlstra , Vincent Guittot , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Costa Shulyupin , Bjorn Helgaas , linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org Subject: [RFC PATCH v3 3/3] DO NOT MERGE: test for managed irqs adjustment Date: Mon, 16 Sep 2024 15:20:44 +0300 Message-ID: <20240916122044.3056787-4-costa.shul@redhat.com> X-Mailer: git-send-email 2.45.0 In-Reply-To: <20240916122044.3056787-1-costa.shul@redhat.com> References: <20240916122044.3056787-1-costa.shul@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" shell script for testing managed interrupts status adjustments. Targets: managed_irq_affinity_adjust(), irq_restore_affinity_of_irq(), managed_irq_isolate() Managed interrupts can be created in various ways. One of them: 1. create qcow2 image 2. run virtme-ng -v --cpus 4 --rw --user root \ --qemu-opts '\-drive id=3Dd1,if=3Dnone,file=3Dimage.qcow2 \ \-device nvme,id=3Di1,drive=3Dd1,serial=3D1,bootindex=3D2' Signed-off-by: Costa Shulyupin --- Changes in v2: - use shell script only v1: - https://lore.kernel.org/lkml/20240516190437.3545310-8-costa.shul@redhat.c= om/ --- tests/managed_irq.sh | 113 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 113 insertions(+) create mode 100755 tests/managed_irq.sh diff --git a/tests/managed_irq.sh b/tests/managed_irq.sh new file mode 100755 index 0000000000000..bd97f47991a0b --- /dev/null +++ b/tests/managed_irq.sh @@ -0,0 +1,113 @@ +#!/bin/zsh + +# shell script for testing IRQ status adjustment. +# Targets: managed_irq_affinity_adjust(), +# irq_restore_affinity_of_irq(), managed_irq_isolate() + +# cpu# to isolate +# +isolate=3D1 + +managed_affined=3D$( + cd /sys/kernel/debug/irq/irqs/; + grep -l -e "affinity: $isolate$" /dev/null $(grep -l IRQD_AFFINITY_MANAGE= D *) +) +test_irq=3D${managed_affined%% *} + +[ -z $test_irq ] && { echo No managed IRQs found;exit 1} + +rm -rf 0.irqs +cp -R /sys/kernel/debug/irq/irqs 0.irqs + +cd /sys/fs/cgroup/ +echo +cpuset > cgroup.subtree_control +mkdir -p test +echo isolated > test/cpuset.cpus.partition + +effective_affinity=3D/proc/irq/$test_irq/effective_affinity +test_irq_debug=3D/sys/kernel/debug/irq/irqs/$test_irq + +errors=3D0 + +check() +{ + local _status=3D$? + if [[ $_status =3D=3D 0 ]] + then + echo PASS + else + let errors+=3D1 + echo FAIL: + cat $test_irq_debug + fi + return $_status +} + +check_activated() +{ + echo "Check normal irq affinity" + test 0 -ne $((0x$(cat $effective_affinity) & 1 << $isolate)) + check + grep -q IRQD_ACTIVATED $test_irq_debug + check + grep -q IRQD_IRQ_STARTED $test_irq_debug + check + ! grep -q IRQD_IRQ_DISABLED $test_irq_debug + check + ! grep -q IRQD_IRQ_MASKED $test_irq_debug + check + ! grep -q IRQD_MANAGED_SHUTDOWN $test_irq_debug + check +} + +check_shutdown() +{ + echo "Check that irq affinity doesn't contain isolated cpu." + test 0 -eq $((0x$(cat $effective_affinity) & 1 << $isolate)) + check + ! grep -q IRQD_ACTIVATED $test_irq_debug + check + ! grep -q IRQD_IRQ_STARTED $test_irq_debug + check + grep -q IRQD_IRQ_DISABLED $test_irq_debug + check + grep -q IRQD_IRQ_MASKED $test_irq_debug + check + grep -q IRQD_MANAGED_SHUTDOWN $test_irq_debug + check +} + +echo "Isolating CPU #$isolate" +echo $isolate > test/cpuset.cpus + +check_shutdown + +echo Reset cpuset +echo "" > test/cpuset.cpus + +check_activated + +echo "Isolating CPU #$isolate again" +echo $isolate > test/cpuset.cpus + +check_shutdown() + +echo "Isolating CPU #3 and restore CPU #$isolate" +echo 3 > test/cpuset.cpus + +check_activated + +echo Reset cpuset +echo "" > test/cpuset.cpus + +rmdir test +cd - + +rm -rf final.irqs +cp -R /sys/kernel/debug/irq/irqs final.irqs + +echo Expected to be without major differences: +diff -r 0.irqs final.irqs && echo No differences + +echo errors=3D$errors +(return $errors) --=20 2.45.0