From nobody Sat Feb 7 23:34:02 2026 Received: from mail-wm1-f51.google.com (mail-wm1-f51.google.com [209.85.128.51]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E5EE51ABEB4; Sun, 1 Dec 2024 12:43:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.51 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733056991; cv=none; b=FXHDxoiFer3NG2vrODkWr5olKTAmmcwm5Gr690QpJrwgIOqoWAeBNNS4xXd1n+8CdW3+CzanV139/OQyrSv3wAGskzddcscV2Yr0rOy+HKfWnCpD0icQcT7X78xT9od6Fq8q1IQmE5a2YrXXXBf9yhwQl6luYNahb6qe2BzCg1Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733056991; c=relaxed/simple; bh=KfhRzWySzFUYAr+kVqtvBhG315vFirAFcq5NWWBMUY0=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=W3F5rcAhjiNoET9RZ37k1EiLdYAOGg8/vIs3xnrWKJ8AaGPPto8MXkk5MfLeFnKQtXP5vPejQ8U6UTM5Oc8NbgbTTbc1LoedFIluk1RkmP0W3DGW3zlYt+tF3nbmMoz2Tj48E37c8kieE5dMM9r2AdYVzSlcqrGlNqw9qjsfeAQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=gmail.com; arc=none smtp.client-ip=209.85.128.51 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-wm1-f51.google.com with SMTP id 5b1f17b1804b1-434a2033562so26853705e9.1; Sun, 01 Dec 2024 04:43:09 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733056988; x=1733661788; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oDZZ2nuy37BUtFZGESe/BsfLdL68Z5Xtb87JQGkcaNE=; b=VPpWaRoS/gllx5a/qf2zviCuCGeZvo/34CbgDKtf7ZevdCXqY8JFRUEjTQM7qkj94U fMiPs7ffFNfEuyD5ySHfg5FLT0mR+BLCI21zAFMTjQmvfTXntWOHyOU7JKaUyUAEbcIk 4N2pBgRfdguOZYKEJfPwKgOa8eSlgfYw8lzss1bSKMKBoCMsZxcG3gXyLqfs3yqo41bl lOfhhCdWSn4Wvd0xjUEyfOKD91aiFfvBrT3SAMPDmec+C7/ULeSQqrPofbm0/9BkRzuN JXrdNsVrew9pbV282pJCfK3XpaVEgeoLIm1wyO1acK2KEKPh4pRMo0hchdWHyDvea/SD aNsw== X-Forwarded-Encrypted: i=1; AJvYcCUWzqV9l0b4GLJc0qGPDv7fqmcCwQaUBRZT8eXnD698NeD4VTKc6l9TBm6eBBZv7I9LcnljjQ9hVYeJM1Uw@vger.kernel.org, AJvYcCW4Xvjm3HjOkglb470bYn+eQQCxYVA6gIlbasv7yOknMDVrKLxVYxuNB2ZRMPWFVhQS4VI62Qsn@vger.kernel.org X-Gm-Message-State: AOJu0YxR1w6cTTVZzEnUpW85WDtpiKMTacXNAhcjZ/oAPmHb7G2DevbL f0aGbkyxF0UQw38XyWFA65TDdRXRkTicrKIM1xHv0XcImVUOD/QG X-Gm-Gg: ASbGnctxHiMMqNJ/aROr3OVqUamsCIbKetKbGLhzFe7d5DlGjKQ+x2FkBAXWlkMLJ3m aXgnT7Gx+tPymgMi/wcMRdGt9C9sDPhR6AdZ/rrsQEOIXMmO16jODOp7QFpfkbetuC7SDk9+x0Z rQpiLDbzRf4yubiGOGBYrn1uZsZWOA7hajJ+vRiEullX03ChNlGbBhL1DIaJnhElPcj4cKhvYgO /MUicrxh8H4GfhHhV6xtKRQRTFlZpCar8hd7xV+m7C6ZWkUEjVXl2BfJGIka7myC+8= X-Google-Smtp-Source: AGHT+IFdXVdGGRVdmQ9j2nUN2ScMyJYwoD58U2MmR/tNNYmP4LdjN8IUbxXv3/RxyRcNLNoTXexpuw== X-Received: by 2002:a05:600c:458b:b0:434:a924:44e9 with SMTP id 5b1f17b1804b1-434a9dcfedfmr182085565e9.15.1733056988064; Sun, 01 Dec 2024 04:43:08 -0800 (PST) Received: from costa-tp.redhat.com ([2a00:a041:e280:5300:9068:704e:a31a:c135]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-434b0f70d91sm114434985e9.39.2024.12.01.04.43.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Dec 2024 04:43:06 -0800 (PST) From: Costa Shulyupin To: longman@redhat.com, ming.lei@redhat.com, pauld@redhat.com, juri.lelli@redhat.com, vschneid@redhat.com, Tejun Heo , Johannes Weiner , =?UTF-8?q?Michal=20Koutn=C3=BD?= , Thomas Gleixner , Costa Shulyupin , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org Subject: [RFC PATCH v4 1/4] cgroup/cpuset: Add HK_TYPE_MANAGED_IRQ to HOUSEKEEPING_FLAGS Date: Sun, 1 Dec 2024 14:42:41 +0200 Message-ID: <20241201124244.997754-2-costa.shul@redhat.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241201124244.997754-1-costa.shul@redhat.com> References: <20241201124244.997754-1-costa.shul@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" update_isolation_cpumasks() should also update housekeeping.cpumasks[HK_TYPE_MANAGED_IRQ] The patch amends https://lore.kernel.org/lkml/20240821142312.236970-3-longman@redhat.com/ Signed-off-by: Costa Shulyupin --- kernel/cgroup/cpuset.c | 1 + 1 file changed, 1 insertion(+) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index fb23eee3f18d..570941d782ef 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -90,6 +90,7 @@ static struct list_head remote_children; #define HOUSEKEEPING_FLAGS (BIT(HK_TYPE_TIMER) | BIT(HK_TYPE_RCU) |\ BIT(HK_TYPE_SCHED) | BIT(HK_TYPE_MISC) |\ BIT(HK_TYPE_DOMAIN) | BIT(HK_TYPE_WQ) |\ + BIT(HK_TYPE_MANAGED_IRQ) |\ BIT(HK_TYPE_KTHREAD)) =20 /* --=20 2.47.0 From nobody Sat Feb 7 23:34:02 2026 Received: from mail-wm1-f54.google.com (mail-wm1-f54.google.com [209.85.128.54]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 65C691B4F10; Sun, 1 Dec 2024 12:43:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.54 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733057002; cv=none; b=XC23y5Dwk0kbizYhjc/jzhNGATL/+S9JptC4uEsJYBOuXDbacS3N0SloelrkaS+nZHKs1zZRuEYy/nseZ224B3POrfspxvV0eq2dAfmGARwEFXKbwzYLKtzk9iaWJSBs1bEnoss4FrXR5H01iCfQaSozmMz8xPF7yOZbzbCfiZE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733057002; c=relaxed/simple; bh=gt4HCElhbhYUl7PT+3Z3hy7r8+X5V1aFs9mZhgDDa4g=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lXuBVhx8+rek6yUQeSDs4NR6j4KkXlKYuQMOmOfLQcYfMvszMJmZg9qVZVI9LkH3sQ+B/eVjK690My66zdT+aGOEIPf58WQaviTvL5rE4gb6m95v6RQNPbuq5r3UlrBv9kWOLdKopqO/04Hk+YMkHlu34z+K6xjw6eMa7ZYgaSk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=gmail.com; arc=none smtp.client-ip=209.85.128.54 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-wm1-f54.google.com with SMTP id 5b1f17b1804b1-434a10588f3so20380785e9.1; Sun, 01 Dec 2024 04:43:20 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733056999; x=1733661799; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VWr8vxwwo0ySNy1XE3FLPwaz+usxzE+up40Zsz5K9bM=; b=kRCdIBD9tPhlBDdMHG3BVqs81TxabEuVV4vhBCZ6+xcYQg+4fTD9KQLJLP4rGQCy6a rVb20MearlduQFyHZXdA85aPwiZ5EPFrrzceCPApxbU6vUE9GeZ5WbFe5UgEqM7pdo1z hG6keYwimrHkZfQuGSyzTn74UrYOSsY916i4m9SwDfWbAb7c0Wh0hlZuhIt21YJShTQk vLQcPNQkk/vjx5S3No5b9tz+A9u3nln54Q6dga+Zn7i4qIZzaZBkuxzX3/3Cum3MOVfz vGJH/toyGd9gdZxIwcX6SqQHQJlBLNXcPuAUkEhJR+0AYAl6OIrKJuPMuIrQOFjfkNqY 29Og== X-Forwarded-Encrypted: i=1; AJvYcCUlgjpf6sXSQyc4VSGs3zkCavTVmsO2xSd1HhYfNtgQMX0BWx134rOEefOCNHltDGBgDPcXZa9R@vger.kernel.org, AJvYcCWo3ebI19GxauGyw4d4ysZ77FW2SFCSkkaXUmfh31ir7W3gtS/sA+Hzori8lOKeIq5NULikW6hht/JKncOs@vger.kernel.org X-Gm-Message-State: AOJu0YyBABWnAl3ZAE0hdhcCHNQaPWFcFPYyHUT8TexXK+TbZhQFN40J CIwWvtrj7xMsGEbgiNGomWuq9IcbrQNWYuqxQXpLEwyre5cU/sEm X-Gm-Gg: ASbGnctl76sHbGHA82ru+uD42Cn7frdPnrTOTKGWO1D05jb2KbbslUXwl5rhTavnQnz dx+j4kEb+FU5IfJA7DEswF03AQXtYS4h+C0Nbjm3/cBF/ZlGNHwvH8gR9X6LxXYujLoPYCMLuQS wjEMSMFrEURLMhU53KRthLHgvfKO1q0Le2yH1I54ebn84wlDIOFglq9uM2nYumrMxHFya8W0ESb ZZCwKxLjeI1b5get39L4pD+RYpA/W6+8QIKAMX39UtTbx72adciYMGEKozDoNXQ/Bg= X-Google-Smtp-Source: AGHT+IHZeN6XY4DEY6A6dqVAV62+FxLC6/hUWxBqHcvVzwZN/CLzBgowoZRoNZwShLCfTrVQHlVErg== X-Received: by 2002:a05:600c:314a:b0:434:9e63:fb15 with SMTP id 5b1f17b1804b1-434a9dbb864mr149899785e9.1.1733056998529; Sun, 01 Dec 2024 04:43:18 -0800 (PST) Received: from costa-tp.redhat.com ([2a00:a041:e280:5300:9068:704e:a31a:c135]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-434b0f70d91sm114434985e9.39.2024.12.01.04.43.16 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Dec 2024 04:43:17 -0800 (PST) From: Costa Shulyupin To: longman@redhat.com, ming.lei@redhat.com, pauld@redhat.com, juri.lelli@redhat.com, vschneid@redhat.com, Tejun Heo , Johannes Weiner , =?UTF-8?q?Michal=20Koutn=C3=BD?= , Thomas Gleixner , Costa Shulyupin , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org Subject: [RFC PATCH v4 2/4] genirq/cpuhotplug: Dynamically isolate CPUs from managed interrupts Date: Sun, 1 Dec 2024 14:42:42 +0200 Message-ID: <20241201124244.997754-3-costa.shul@redhat.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241201124244.997754-1-costa.shul@redhat.com> References: <20241201124244.997754-1-costa.shul@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" After change of housekeeping_cpumask(HK_TYPE_MANAGED_IRQ) during runtime managed interrupts continue to run on isolated CPUs. Dynamic CPUs isolation is complex task. One of approaches is: 1. Set affected CPUs offline and disable relevant interrupts 2. Change housekeeping_cpumask 3. Set affected CPUs online and enable relevant interrupts irq_restore_affinity_of_irq() restores managed interrupts during complex CPU hotplug process of bringing back a CPU online. Leave the interrupts disabled those affinity doesn't intersect with new housekeeping_cpumask thereby ensuring isolation of the CPU from managed intrrupts. Signed-off-by: Costa Shulyupin --- kernel/irq/cpuhotplug.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/kernel/irq/cpuhotplug.c b/kernel/irq/cpuhotplug.c index ec2cdcd20bee..839d3e879c0d 100644 --- a/kernel/irq/cpuhotplug.c +++ b/kernel/irq/cpuhotplug.c @@ -218,6 +218,9 @@ static void irq_restore_affinity_of_irq(struct irq_desc= *desc, unsigned int cpu) if (desc->istate & IRQS_SUSPENDED) return; =20 + if (!cpumask_intersects(affinity, housekeeping_cpumask(HK_TYPE_MANAGED_IR= Q))) + return; + if (irqd_is_managed_and_shutdown(data)) irq_startup(desc, IRQ_RESEND, IRQ_START_COND); =20 --=20 2.47.0 From nobody Sat Feb 7 23:34:02 2026 Received: from mail-wm1-f43.google.com (mail-wm1-f43.google.com [209.85.128.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E31851B4F10; Sun, 1 Dec 2024 12:43:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733057012; cv=none; b=DmDjITEa+dDPrJq5Cg595GT3Mep0C/YJRmBSisCEgGe6EGXyaGHhAATYblFJ+bI8YV5086r2JwDftnXTrtnfqHhmkA4sJ9DMApvksLANal4KYRV7uOYMVL5qJc1tg1/vjteFuVzQM2zLZYZPM+N5wdgnslwuFrW44PWMqIzAbdk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733057012; c=relaxed/simple; bh=jFSka1yGhbHJNKx7+u+18P6j13WVA48HuQ6F6r0Jcow=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=baiTnNxO+y1jhKe5Z2SS6lfF8e7Fs1v05gsDOJWNwaa7FmChUborHsE7Xrn7IIePPzLm3lJAW9Mu64aPFmrcYgjqkjZPJ0DH4ytuO0bo4gd9Z9oGCZggZp9k7YI17WaAtNLoficGvh9Y45DyjXSnmRNn+YlNdxqwcGaaDMrjpUs= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=gmail.com; arc=none smtp.client-ip=209.85.128.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-wm1-f43.google.com with SMTP id 5b1f17b1804b1-434a2f3bae4so30788925e9.3; Sun, 01 Dec 2024 04:43:30 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733057009; x=1733661809; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qm5KHITgvJAlZwMn/vft2HYn9JiVpV45gh+d1dBBjCk=; b=H0HDA+T8KlMeO8YZNNBXCXbvBGgrCTEfJmsDsO38HJy4BWN+oOkcmsDk4XGPebirFj 0e4qi6eY/ZpcBr8jakhSpYI7n8EWMrcOUs73OLIR3GxLS5cGmWyDIOqre1QTeRcEDrWU R56t9WjGPachiSr+/qfnuRZQFzGysRDI+eiVQAh2+Lu6tq46vllwAcnY3owYbQLMpusR wee/hWRtBGdOT0HLpYFcXSf+GCOhPGzAkSBJz28JI6gNVKy/wCz/LqYZsbG8KQudtGGQ H/srk5ZVCMLOmnUHPiWUh90q8soOpi1Sk3TSdFuh61hqmE21kzDPkY7bVgy4As/L6FX/ Q2+g== X-Forwarded-Encrypted: i=1; AJvYcCUdZ3cejs2mNn0HtUYZLAahE08Yag93FSAfmvouGoU+GD9KJYxDed/X4SZjgv+wDIszemeWX8vHYne9Pa4M@vger.kernel.org, AJvYcCXuPtXNF8LxPrfLJq4yZLLbQGdGbck0wRoHvrBIrBh3+I7ujb5f34G4rpKc8Rb8xXo9iBSWXqMO@vger.kernel.org X-Gm-Message-State: AOJu0YzwE1hC7Z44tGpz7Xsaxu2I8yfh2e4UBb0f5xwg22Q+B05ME+Ir 3HE/DjjHpSaqE7rtOX1pkW5U3PVBAh+jKGPiEmzrhMHZ8nkxoiyp X-Gm-Gg: ASbGnctOjxqauhxwAqheTEN2rAqgkWaIhd/ZqYK/QBzIAo0+Oeg9Uq3YDuOwZpV03Zr 9Lz7W+FSWiTwMxQWAc6yxcl8K4qABM/V761bt9lekq6Fwdb2zIIiphIZxikxIbnsphXkx28lsIu Q05csb7CKGlqLGQbiyqqPuYd4gV57r+Fd4WkjUMLp2MTYJhmhgRo3v8WHTZQcHkFDI3hBBT9GZG DM6Y8Wrsu9I+rA9zkRIiB/a5yrCzSzLBl7mAasie4TzQLoQspnsjbHvVOpMDU12pBU= X-Google-Smtp-Source: AGHT+IG8nH9SSVD+lxWjB0L+a2E2bi9fBHSp8wh/OQyotTaqJTBfFi76+Ura6+CZGZS1obvkex1oZw== X-Received: by 2002:a05:600c:458b:b0:431:5a0e:fa2e with SMTP id 5b1f17b1804b1-434a9de3d04mr142985055e9.21.1733057009007; Sun, 01 Dec 2024 04:43:29 -0800 (PST) Received: from costa-tp.redhat.com ([2a00:a041:e280:5300:9068:704e:a31a:c135]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-434b0f70d91sm114434985e9.39.2024.12.01.04.43.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Dec 2024 04:43:28 -0800 (PST) From: Costa Shulyupin To: longman@redhat.com, ming.lei@redhat.com, pauld@redhat.com, juri.lelli@redhat.com, vschneid@redhat.com, Tejun Heo , Johannes Weiner , =?UTF-8?q?Michal=20Koutn=C3=BD?= , Thomas Gleixner , Costa Shulyupin , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org Subject: [RFC PATCH v4 3/4] cgroup/cpuset: Restart CPUs whose isolated_cpus bits have changed Date: Sun, 1 Dec 2024 14:42:43 +0200 Message-ID: <20241201124244.997754-4-costa.shul@redhat.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241201124244.997754-1-costa.shul@redhat.com> References: <20241201124244.997754-1-costa.shul@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" The goal is to dynamically isolate CPUs to prevent interference from housekeeping subsystems. The housekeeping CPU masks, set up by the "isolcpus" and "nohz_full" boot command line options, are used at boot time to exclude selected CPUs from running some kernel housekeeping subsystems to minimize interference with latency sensitive userspace applications such as DPDK. This options can only be changed with a reboot. This is a problem for containerized workloads running on OpenShift/Kubernetes where a mix of low latency and "normal" workloads can be created/destroyed dynamically and the number of CPUs allocated to each workload is often not known at boot time. CPU hotplug can be used to isolate CPUs by restarting related CPUs only, without complete reboot. Experimental solution. Automatically restart changed CPUs when the `isolated_cpus` is modified through the cgroup/cpuset interface. No additional manipulation of the CPU online status from userspace is required, and it remains compatible with existing software. cpu_device_down()/cpu_device_up() can't be called within subroutines of cpuset_write_resmask() because it locks `cpu_hotplug_lock` with cpus_read_lock() but _cpu_down()/_cpu_up() lock `cpu_hotplug_lock` with cpus_write_lock(). Intuitively the change of `isolated_cpus` should be performed between cpu_device_down() and cpu_device_up(). Since cpu_device_down(), at least for managed interrupts, doesn't depends on `isolated_cpus` and `housekeeping` it is more simple to call cpu_device_down() after change of `isolated_cpus` and `housekeeping` and cpus_read_unlock(). Signed-off-by: Costa Shulyupin --- kernel/cgroup/cpuset.c | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index 570941d782ef..d5d2b4036314 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -3131,6 +3131,27 @@ static void cpuset_attach(struct cgroup_taskset *tse= t) mutex_unlock(&cpuset_mutex); } =20 +/* + * Restart CPUs whose isolated_cpus bits have changed. + * Enforce subsystems to adopt the new isolated_cpus and housekeeping masks + * using CPU hotplug. + */ +static void propogate_isolated_cpus_change(struct cpumask *isolated_cpus_p= rev) +{ + unsigned int cpu; + + if (!isolated_cpus_prev) + return; + + for_each_online_cpu(cpu) { + if (cpumask_test_cpu(cpu, isolated_cpus_prev) !=3D + cpumask_test_cpu(cpu, isolated_cpus)) { + remove_cpu(cpu); + add_cpu(cpu); + } + } +} + /* * Common handling for a write to a "cpus" or "mems" file. */ @@ -3138,6 +3159,7 @@ ssize_t cpuset_write_resmask(struct kernfs_open_file = *of, char *buf, size_t nbytes, loff_t off) { struct cpuset *cs =3D css_cs(of_css(of)); + cpumask_var_t isolated_cpus_prev; struct cpuset *trialcs; int retval =3D -ENODEV; =20 @@ -3167,6 +3189,12 @@ ssize_t cpuset_write_resmask(struct kernfs_open_file= *of, =20 cpus_read_lock(); mutex_lock(&cpuset_mutex); + if (!alloc_cpumask_var(&isolated_cpus_prev, GFP_KERNEL)) { + retval =3D -ENOMEM; + goto out_unlock; + } + + cpumask_copy(isolated_cpus_prev, isolated_cpus); if (!is_cpuset_online(cs)) goto out_unlock; =20 @@ -3200,6 +3228,13 @@ ssize_t cpuset_write_resmask(struct kernfs_open_file= *of, kernfs_unbreak_active_protection(of->kn); css_put(&cs->css); flush_workqueue(cpuset_migrate_mm_wq); + + /* If isolated_cpus modified, the change must be propagated + * to all subsystems. + */ + propogate_isolated_cpus_change(isolated_cpus_prev); + free_cpumask_var(isolated_cpus_prev); + return retval ?: nbytes; } =20 --=20 2.47.0 From nobody Sat Feb 7 23:34:02 2026 Received: from mail-wm1-f52.google.com (mail-wm1-f52.google.com [209.85.128.52]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9ABAE1B5337; Sun, 1 Dec 2024 12:43:41 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.128.52 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733057023; cv=none; b=TayZFdn55JASXRj7hfAraIVC5MAhVBeOM27tyRjk1H3TuAyCO/5uX6stOGj/A53aHfnHf41M8DgbxXWN00ohdFSVCVF+kdGLs92ZjxCyFRIX2AqaUVj5kmZMbAEblw/8KctVXpRDahqT+lSGmkdnxbbntufpdLLx2vMPEig0mhY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733057023; c=relaxed/simple; bh=kEFz7s9cS8bt1A3BnJLkQ42vnQvrBH9vKlkQzNuay44=; h=From:To:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=iVUouJJPnfwb9/5JbJfZi0tCmYRHGMRrkRfRWMxTfhsuhUoR+hpkzRqB0cp+IzFMcdgSAUwI4N2o+N6NPx22kNnWqJVeNtUSd/WYDKi62lR0KDwsGeTzSTz4Ichmn0LGc/CMIcev0UhB8bbNsuFMQTw8/QkBiFiQhaw/p3bvTT4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=gmail.com; arc=none smtp.client-ip=209.85.128.52 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-wm1-f52.google.com with SMTP id 5b1f17b1804b1-434ab938e37so20449485e9.0; Sun, 01 Dec 2024 04:43:41 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1733057020; x=1733661820; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=hwTgWmg6Nk+SeybaZH1G2SoKFC/4ATTBRTyPLSQfoDo=; b=LreNNltUWimQmBYU6YEC7xNmpJx34PjtzrFCIZLSNewW8ACsypPqNvH+x4jm4hG44S zkllh8CkU+zO/Wxr0tEkcDFoO6TO+s7ZupJs+dAzLF9/FpUnmTUWfxyiSpeQzDNElYCg Tpv+zBw8MxKowu6phELgCg6fLDYGfq6Q9sEq7PTR7DpCvuckg2YAQ4FSjICJQVPZUvcu Kpsf1QPNqggN4bt9OSQnJ/5YiiC3iu0K70BQBOWAgJsXltsj2ohzS3DrDzR0q+h/ovjd 1x+HDabI/KdrN6z2xZh0SLaHgZC97q05BmmpUfmQsLbjdImA5pkGCcvl6NPDW+7H+L26 H/dA== X-Forwarded-Encrypted: i=1; AJvYcCUCd7JsGTxjrmQvdyM9Xyc6nr5af8MWcLMd/AZd1c+8p+JFU2lNmxujYJJxPF7n9MJU6TCGBQt6SSU/6cIw@vger.kernel.org, AJvYcCWVebBC9lKPJWS80U7rB3+LEKuKhRlwcmoa96l/MUenFt1fmB1Sup9mOl3EEHyfxjjZ7pU+8GSv@vger.kernel.org X-Gm-Message-State: AOJu0YxFopvbOuPrWzrnEY9O0dYyRNZ+a0nI4BldwqROkLRkYwCGtJJe 2Rf9HVL2i/54Tu7MAWEzOig1PUtayOvJnsbkRuVWPHCMZpcV7Y7caQBcuOScquc= X-Gm-Gg: ASbGncuXcYGjDjDU6WVxwifE26l2NBxO8pUlt0pchacFlHafJcrtiTJR1Uft3IzXeWA i+US4gZ5XYNno8875Pi6m9g6v4lgDdZl1D8VmJ0MB27EGD/QrlUKb0DfiXyh7ZvKsFDdG6o2Yd9 Y/XkTUZWn75AxDi9cNV/8ddb4l9AkvUPbsnIChqoNhss25xCMfiTIjYIdbRN/XnAHYYv7r35dRG KsufA9cB3ChIetKJKXSYJbvnHhgrkvRNAAsKzMwbXdBkU50Gid9BWRFBSbdoVIvrgI= X-Google-Smtp-Source: AGHT+IETkNM8+r32RNP4qhV2x33uBUhpJLEx/4ptuf4OfUoRO8MHH9mqdmcgeFc1qrYk89pxaaPaFg== X-Received: by 2002:a05:6000:1ac9:b0:382:4a7b:ba7b with SMTP id ffacd0b85a97d-385c6ee2174mr16120213f8f.48.1733057019624; Sun, 01 Dec 2024 04:43:39 -0800 (PST) Received: from costa-tp.redhat.com ([2a00:a041:e280:5300:9068:704e:a31a:c135]) by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-434b0f70d91sm114434985e9.39.2024.12.01.04.43.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 01 Dec 2024 04:43:38 -0800 (PST) From: Costa Shulyupin To: longman@redhat.com, ming.lei@redhat.com, pauld@redhat.com, juri.lelli@redhat.com, vschneid@redhat.com, Tejun Heo , Johannes Weiner , =?UTF-8?q?Michal=20Koutn=C3=BD?= , Thomas Gleixner , Costa Shulyupin , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org Subject: [RFC PATCH v4 4/4] DO NOT MERGE: Test CPU isolation from managed interrupts Date: Sun, 1 Dec 2024 14:42:44 +0200 Message-ID: <20241201124244.997754-5-costa.shul@redhat.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241201124244.997754-1-costa.shul@redhat.com> References: <20241201124244.997754-1-costa.shul@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Test CPU isolation to ensure it is unaffected by managed interrupts. Target: irq_restore_affinity_of_irq() Managed interrupts can be created in various ways. One of them: qemu-img create -f qcow2 test.qcow2 100M virtme-ng -v --cpus 4 --rw --user root \ --qemu-opts '\-drive id=3Dd1,if=3Dnone,file=3Dtest.qcow2 \ \-device nvme,id=3Di1,drive=3Dd1,serial=3D1,bootindex=3D2' Signed-off-by: Costa Shulyupin --- v4: - Remove /sys/devices/system/cpu/cpu$isolate/online v3: - No changes v2: - use shell script only --- MAINTAINERS | 2 + tests/managed_irq.sh | 135 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 137 insertions(+) create mode 100755 tests/managed_irq.sh diff --git a/MAINTAINERS b/MAINTAINERS index 1240e75ecf4b..4a753c2b34c1 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -26046,3 +26046,5 @@ S: Buried alive in reporters T: git git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git F: * F: */ + +# disable warning diff --git a/tests/managed_irq.sh b/tests/managed_irq.sh new file mode 100755 index 000000000000..3763183fc987 --- /dev/null +++ b/tests/managed_irq.sh @@ -0,0 +1,135 @@ +#!/bin/zsh +# SPDX-License-Identifier: GPL-2.0 + +# shell script for testing dynamic isolation of managed interrupts. +# Target: irq_restore_affinity_of_irq() + +# cpu# to isolate + +isolate=3D1 + +managed_affined=3D$( + cd /sys/kernel/debug/irq/irqs/; + grep -l -e "affinity: $isolate$" /dev/null $(grep -l IRQD_AFFINITY_MANAGE= D *) | + head -n1 +) +grep -l -e "affinity: $isolate$" /dev/null \ + $(grep -l IRQD_AFFINITY_MANAGED /sys/kernel/debug/irq/irqs/*) +test_irq=3D${managed_affined%% *} +echo test_irq=3D$test_irq + +[ -z $test_irq ] && { echo No managed IRQs found;exit 1} + +cp -R /sys/kernel/debug/irq/irqs 0.irqs + +# Restart CPUs without changing the isolated cpuset. +# Setup a baseline (the "control group") +# to compare against the test of isolated mask. + +echo 0 > /sys/devices/system/cpu/cpu$isolate/online +echo 0 > /sys/devices/system/cpu/cpu3/online +echo 1 > /sys/devices/system/cpu/cpu$isolate/online +echo 1 > /sys/devices/system/cpu/cpu3/online + +rm -rf baseline.irqs +cp -R /sys/kernel/debug/irq/irqs baseline.irqs + +cd /sys/fs/cgroup/ +echo +cpuset > cgroup.subtree_control +mkdir -p test +echo isolated > test/cpuset.cpus.partition + +effective_affinity=3D/proc/irq/$test_irq/effective_affinity +test_irq_debug=3D/sys/kernel/debug/irq/irqs/$test_irq + +errors=3D0 + +check() +{ + local _status=3D$? + if [[ $_status =3D=3D 0 ]] + then + echo PASS + else + let errors+=3D1 + echo "FAIL #$errors:" + cat $test_irq_debug + fi + return $_status +} + +check_activated() +{ + echo "Check normal irq affinity" + test 0 -ne $((0x$(cat $effective_affinity) & 1 << $isolate)) + check + grep -q IRQD_ACTIVATED $test_irq_debug + check + grep -q IRQD_IRQ_STARTED $test_irq_debug + check + ! grep -q IRQD_IRQ_DISABLED $test_irq_debug + check + ! grep -q IRQD_IRQ_MASKED $test_irq_debug + check + ! grep -q IRQD_MANAGED_SHUTDOWN $test_irq_debug + check +} + +check_shutdown() +{ + echo "Check that irq affinity doesn't contain isolated cpu." + test 0 -eq $((0x$(cat $effective_affinity) & 1 << $isolate)) + check + ! grep -q IRQD_ACTIVATED $test_irq_debug + check + ! grep -q IRQD_IRQ_STARTED $test_irq_debug + check + grep -q IRQD_IRQ_DISABLED $test_irq_debug + check + grep -q IRQD_IRQ_MASKED $test_irq_debug + check + grep -q IRQD_MANAGED_SHUTDOWN $test_irq_debug + check +} + +echo "Isolating CPU #$isolate" +echo $isolate > test/cpuset.cpus + +check_shutdown + +echo Reset cpuset +echo "" > test/cpuset.cpus + +check_activated + +echo "Isolating CPU #$isolate again" +echo $isolate > test/cpuset.cpus + +check_shutdown + +echo "Isolating CPU #3 and restore CPU #$isolate" +echo 3 > test/cpuset.cpus + +check_activated + +echo Reset cpuset +echo "" > test/cpuset.cpus + +rmdir test +cd - + +rm -rf final.irqs +cp -R /sys/kernel/debug/irq/irqs final.irqs + +sleep 1 # wait till IRQD_IRQ_INPROGRESS, IRQD_IRQ_MASKED + +if ! diff -r --ignore-matching-lines=3DVector:: baseline.irqs final.irqs; = then + echo diff failed; + let errors+=3D1 +fi + +# +# return zero on success or number of errors +# +echo errors=3D$errors +(return $errors) --=20 2.47.0