From nobody Fri Oct 10 21:10:18 2025 Received: from mail-pf1-f177.google.com (mail-pf1-f177.google.com [209.85.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3268029B77A for ; Thu, 12 Jun 2025 18:33:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.177 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749753230; cv=none; b=jltwiCwKY34LQLaOStpaEbtsWRt7NZSWDCxFjfUMzC9+5pqyoHDGQ4gQNH9jkRQIvddQlAbsb9g10n3qgFOMqH38TWRnMVQjGHpmO/0zrrDkJQUSLr/5pfHQyNlAwbsClYrxSCth/s4hE4t/q2jpZvrU8Bx07kh8dUihiJXf+nk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749753230; c=relaxed/simple; bh=haFgvWLLKNrwJaBpLIU0R+weECSTNAekip7enpQmJnY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tgQqdVBSzVqSXXOjPapjbG19A+GKbjiuyjVvSx9lWl3i/Nd84lkTV1gv8rkFibAJwX+dUEg8d1X1tVIjz2vBvOMBAGrtAae5Hz7WnxdkjTCtWCbqD1wGzh/WqOVm/NAC+PPKzVydPFybaHZCl4DfAsN6G//+dpWRbbzQNWK8ZoM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=NkCYD2PM; arc=none smtp.client-ip=209.85.210.177 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="NkCYD2PM" Received: by mail-pf1-f177.google.com with SMTP id d2e1a72fcca58-7425bd5a83aso1249341b3a.0 for ; Thu, 12 Jun 2025 11:33:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1749753228; x=1750358028; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Ml0lKVPVybvP0N92BPIyS7ApVn3dLoCM1ruXROi1PdY=; b=NkCYD2PM8sEXhP63suQDGobmqa2mU13nZRP5DriqljXz55u4ojTU7KtuBwGw6MaSlk 433/2ADFKtnqjoJhZrFgKndHETXlNf3Vt8G7Hxu+yuYj0BQda1hiyDC/gnkZKqNaOj4P sRJsNW3bPvBLsoqxRdekXOQEwQUnXHWq7u91Y= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1749753228; x=1750358028; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Ml0lKVPVybvP0N92BPIyS7ApVn3dLoCM1ruXROi1PdY=; b=MWR8mEB+FbrQy2lKAZAtY5zN+/JWNIgCly8CQoVVG0N4MtVXFmEu2SDdZWYvHcNs73 9tdvCou+yqYTMIhFGvEh5QaAExJP0KxpVgcWp0uZLu/2+tZTELzPO9X6qulgE192DMIq rRJJmLO9yNa8g8woIl8NPe9yWulbKOfS7P8n/rFFG3gZ+u0ywag9tqWtelcpCi40okWK TyupjUePIf2pRFl7I25tIX4gquKHDIpZYoAdVJhpsLIUPWgCfeHBYxvkHXjGC6H+7Gz5 Jsoi5TqgiAqQdxqOffFkOskAAaHT+OmE+Yxo0U6u8t0r757syNbJrrpkjk8tuTPkGOhY IISw== X-Forwarded-Encrypted: i=1; AJvYcCUhEpne4ICIHWK0420/V61fR51ebE18bDMX8LJqrdhPgElxwwsCecpmbeVwD/G8eajMHRUJn4JwSJN0vmg=@vger.kernel.org X-Gm-Message-State: AOJu0YxcS2wvT9117HUsqEkRoTYJo/b7EJHse5w30dkaWMJYTNGBO8ud TmuNoGGWm7RoXZSP7PE3DaO+aMX6fyhwSxSbTQmny0sugzESB9ZPGj2A9Eb00u3NTA== X-Gm-Gg: ASbGnct00Oscz2M+5Od22bTNQQoi9+yDGMglk9p9KMVPhtLnZVTR7urNsMqO0LLEIsy j9dGJBI9GBOi2SmnYIVGcORD17+ND/LHZ9lua2DyUdOfI3w+SVUUz3HyW3MO/6A4MitQQOC1hv0 gNDrpzivNgMKhTodhn36Tt9Mw/1fLacgZqSOcnpRvdHPX0KxBeCWzfzhqJtGnfY9kekONdSitAx mokVuBu/F1oX8CDvsEZQMy+h+H/1S4TwTD1sXWsIqLiuzVPYWV9ptXmeDG7lKC8NLJDFiauDQAC GNFzLB70Sz0t1jYDf92Ss6DxPrvgjSvwIh6R59aHcnRUEsJBcdxv1NXsb6qnxeJ7zZtD6fMS2kN uMslVqfQ9xhP064ltbJqHlJ3i X-Google-Smtp-Source: AGHT+IHln2HYr+WrcVg3RtojtpX6fOD0bSEeNgI9uoOtgX11XC4EPkgbC5OMG3J+OgA4Xhp+fJ3BPg== X-Received: by 2002:a05:6a00:2d8e:b0:736:ab49:d56 with SMTP id d2e1a72fcca58-7488f6e49f9mr406585b3a.1.1749753228440; Thu, 12 Jun 2025 11:33:48 -0700 (PDT) Received: from localhost ([2a00:79e0:2e14:7:e790:5956:5b47:d0a7]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-7488ffed070sm41340b3a.14.2025.06.12.11.33.47 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 12 Jun 2025 11:33:47 -0700 (PDT) From: Brian Norris To: Thomas Gleixner Cc: Douglas Anderson , David Stevens , linux-kernel@vger.kernel.org, Tsai Sung-Fu , Aleksandrs Vinarskis , Johan Hovold , Brian Norris Subject: [PATCH 6.16 1/2] genirq: Rebalance managed interrupts across multi-CPU hotplug Date: Thu, 12 Jun 2025 11:32:51 -0700 Message-ID: <20250612183303.3433234-2-briannorris@chromium.org> X-Mailer: git-send-email 2.50.0.rc1.591.g9c95f17f64-goog In-Reply-To: <20250612183303.3433234-1-briannorris@chromium.org> References: <20250612183303.3433234-1-briannorris@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Commit 788019eb559f ("genirq: Retain disable depth for managed interrupts across CPU hotplug") intended to only decrement the disable depth once per managed shutdown, but instead it decrements for each CPU hotplug in the affinity mask, until its depth reaches a point where it finally gets re-started. For example, consider: 1. Interrupt is affine to CPU {M,N} 2. disable_irq() -> depth is 1 3. CPU M goes offline -> interrupt migrates to CPU N / depth is still 1 4. CPU N goes offline -> irq_shutdown() / depth is 2 5. CPU N goes online -> irq_restore_affinity_of_irq() -> irqd_is_managed_and_shutdown()=3D=3Dtrue -> irq_startup_managed() -> depth is 1 6. CPU M goes online -> irq_restore_affinity_of_irq() -> irqd_is_managed_and_shutdown()=3D=3Dtrue -> irq_startup_managed() -> depth is 0 *** BUG: driver expects the interrupt is still disabled *** -> irq_startup() -> irqd_clr_managed_shutdown() 7. enable_irq() -> depth underflow / unbalanced enable_irq() warning We should clear the managed-shutdown flag at step 6, so that further hotplugs don't cause further imbalance. Note: while it might be cleaner to also remove the irqd_clr_managed_shutdown() from __irq_startup_managed() at the same time, this is currently not possible because of irq_update_affinity_desc() -- it sets IRQD_MANAGED_SHUTDOWN and expects irq_startup() to clear it. Fixes: 788019eb559f ("genirq: Retain disable depth for managed interrupts a= cross CPU hotplug") Signed-off-by: Brian Norris Tested-by: Aleksandrs Vinarskis # X1E, X1 --- kernel/irq/chip.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/kernel/irq/chip.c b/kernel/irq/chip.c index b0e0a7332993..2b274007e8ba 100644 --- a/kernel/irq/chip.c +++ b/kernel/irq/chip.c @@ -205,6 +205,14 @@ __irq_startup_managed(struct irq_desc *desc, const str= uct cpumask *aff, =20 void irq_startup_managed(struct irq_desc *desc) { + struct irq_data *d =3D irq_desc_get_irq_data(desc); + + /* + * Clear managed-shutdown flag, so we don't repeat managed-startup for + * multiple hotplugs, and cause imbalanced disable depth. + */ + irqd_clr_managed_shutdown(d); + /* * Only start it up when the disable depth is 1, so that a disable, * hotunplug, hotplug sequence does not end up enabling it during --=20 2.50.0.rc1.591.g9c95f17f64-goog From nobody Fri Oct 10 21:10:18 2025 Received: from mail-pf1-f174.google.com (mail-pf1-f174.google.com [209.85.210.174]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3F45C2D1F69 for ; Thu, 12 Jun 2025 18:33:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.210.174 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749753232; cv=none; b=R5MvFGuiLzBEeZ7wzvM7mu5SRMvuUV5i/5CzK5BPpKUDDrcGI//LEkrDNe9dVrIYGa48xDe1D1uyNitzVaUcwjxmy96zmC3w7cZOyzLS1n42ouFEyd/M/eVfMqTzl9YVFaqO1wOGjT4pqtiw90Zg23jM09y7sVGhf11nmr2TAIM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1749753232; c=relaxed/simple; bh=jm6hdSF9DDHht7VP6n7cX1vT1+eSt1dZthS7BFn28fA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=oNPI5s6AMrTrXp7I5f2VZD27WctJsXkj0JrGCFQN5IuJHyWwCry5hIMs0+3aBrFGxNf9q70AkrsMEtIKC/Bqdp7xt+GPMc1W//7+aFx/0hD9eKTXPKYCg4O/D82u41HgFkFqH5+tE2d/zIrMwWgPPYiTyAPRMXtxpTUV/tozV9Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org; spf=pass smtp.mailfrom=chromium.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b=PI+FSNa2; arc=none smtp.client-ip=209.85.210.174 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=chromium.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=chromium.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=chromium.org header.i=@chromium.org header.b="PI+FSNa2" Received: by mail-pf1-f174.google.com with SMTP id d2e1a72fcca58-748764d8540so1268030b3a.0 for ; Thu, 12 Jun 2025 11:33:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=chromium.org; s=google; t=1749753230; x=1750358030; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=jaV+jqSj0N9g0tjJ63kwte2Vfr32gC1ZJfj5xrckMhs=; b=PI+FSNa2sIg+sSKqy6Vhg+t+kS7vU/tQFzmzCSRx1GGIA7nYmGq/5jHP54S19YSCPy 7kyj1d8ELy2No+BhgL7tyFD00Cyc5km9OeyfUFaATs5Gauw+dDnRvG9O71b2O82OOHNG GTBJnGnBOEHZ4KnaoUTLaC3/89SnxkTD+ybZo= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1749753230; x=1750358030; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=jaV+jqSj0N9g0tjJ63kwte2Vfr32gC1ZJfj5xrckMhs=; b=EjovOvAkxI2Ny4a4XK8SKCNrF/LvGNJ19lfoLXyvUsCPSuFnHSaU02bl63I1kGLER6 KIQvbyUGJVeXS+/zQpSjk0Qdy4yhgc78ILrC3eCFT7xb5DRppJI6TEO7QSCwvDSMCOUu s+CU8P9hpoETUo5wAuQMz84+brNtUGEwiEn+Ba6TTQxIZTTegmq6DqpEUxp2BP3+pu9O n4JampAnm04pcfTCQR9/2mZs6ugnxG6Ine56K6k2Kewk9FcuIyCKWtmA8mWy6N5DuFTt urogxpq8izDnER1nnTqt1SNLImq2o8Px1WcIUgDqNqvnn1hiWInuLNPX5a02XxrMIx94 4Feg== X-Forwarded-Encrypted: i=1; AJvYcCVcF/urqRYQBlO/4EFhZYlz3ojKx+uXlI7uN08dPwZWFxJAiMa6qPrAYw3rZPbNkF/NSOj5J0HH4DOnODU=@vger.kernel.org X-Gm-Message-State: AOJu0YwwdMXLDZECGiL4QNL79XgqNjymWlKEY9h9fPPn61nfljJ9EZbU pLyDmeQrmFPhjXxN4h6ghkHnk4XZXdVR/9l2Cr7KA4RQ4wYuPQb+TLKdXDH5zNbQTg== X-Gm-Gg: ASbGncu/wIQJ7laLepNdTfyGHu6pGBPHTDn02+xul8nGseOPx2Zgbl7qe655RF60WQB nIy+MryOWEPfMVW2DDBapk764vz6ye7lqfMFgRTWqYVghAlVuWWBa4w8QNkzMA6w3jaPETDbjJn pagVrBXm5e3G1Htxx+XNiQsylE1C8jSqj2sXyKd3YeH1Zg8cfUByQtwTbRpqaXyt8viPj0h0Rew ajQUOUGKh/Zvk7LgBNlIqh49c0obQPFU1NES7PZsmGvpxbUAXw5o/maew0tGescJqxG75EvpCIQ XZJeSkjBxM70gL+gJXH15Azw4qFYlcyIHQQLVfdsflm7q8rYKrJp2xMa0D3D93EvJDCz/MZNfu+ MASCCrUo9g6PRxTSYMkRW6XR5H8YdqOCX8x4= X-Google-Smtp-Source: AGHT+IG2s/+PdI46v7+8VWpo7QZIQfG97uZbB3EVU5UKC/wKT4QaRs/Z8qP9FPCmd/a0BvOp9oNa0g== X-Received: by 2002:a05:6a21:999a:b0:1f5:8eec:e517 with SMTP id adf61e73a8af0-21f8660015bmr12380747637.9.1749753230534; Thu, 12 Jun 2025 11:33:50 -0700 (PDT) Received: from localhost ([2a00:79e0:2e14:7:e790:5956:5b47:d0a7]) by smtp.gmail.com with UTF8SMTPSA id d2e1a72fcca58-74890083034sm31888b3a.98.2025.06.12.11.33.49 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 12 Jun 2025 11:33:50 -0700 (PDT) From: Brian Norris To: Thomas Gleixner Cc: Douglas Anderson , David Stevens , linux-kernel@vger.kernel.org, Tsai Sung-Fu , Aleksandrs Vinarskis , Johan Hovold , Brian Norris Subject: [PATCH 6.16 2/2] genirq/cpuhotplug: Restore affinity even for suspended IRQ Date: Thu, 12 Jun 2025 11:32:52 -0700 Message-ID: <20250612183303.3433234-3-briannorris@chromium.org> X-Mailer: git-send-email 2.50.0.rc1.591.g9c95f17f64-goog In-Reply-To: <20250612183303.3433234-1-briannorris@chromium.org> References: <20250612183303.3433234-1-briannorris@chromium.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Commit 788019eb559f ("genirq: Retain disable depth for managed interrupts across CPU hotplug") tried to make managed shutdown/startup properly reference counted, but it missed the fact that the unplug and hotplug code had an intentional imbalance -- that they skip IRQS_SUSPENDED interrupts on the "restore" path. This means that if a managed-affinity interrupt was both suspended and managed-shutdown (such as may happen during system suspend / S3), we would skip calling irq_startup_managed() when resuming, and would again have an unbalanced depth -- this time, with a positive value (i.e., remaining unexpectedly masked). This IRQS_SUSPENDED check was introduced in commit a60dd06af674 ("genirq/cpuhotplug: Skip suspended interrupts when restoring affinity") for essentially the same reason as commit 788019eb559f -- that irq_startup() would unconditionally re-enable an interrupt too early. Because irq_startup_managed() now respsects the disable-depth count, we no longer need the IRQS_SUSPENDED check, and instead, it causes harm. Thus, drop the IRQS_SUSPENDED check, and restore balance. This effectively reverts commit a60dd06af674 ("genirq/cpuhotplug: Skip suspended interrupts when restoring affinity"), because it is replaced by commit 788019eb559f ("genirq: Retain disable depth for managed interrupts across CPU hotplug"). Fixes: 788019eb559f ("genirq: Retain disable depth for managed interrupts a= cross CPU hotplug") Reported-by: Aleksandrs Vinarskis Closes: https://lore.kernel.org/lkml/24ec4adc-7c80-49e9-93ee-19908a97ab84@g= mail.com/ Signed-off-by: Brian Norris Tested-by: Aleksandrs Vinarskis # X1E, X1 --- kernel/irq/cpuhotplug.c | 7 ------- 1 file changed, 7 deletions(-) diff --git a/kernel/irq/cpuhotplug.c b/kernel/irq/cpuhotplug.c index f07529ae4895..755346ea9819 100644 --- a/kernel/irq/cpuhotplug.c +++ b/kernel/irq/cpuhotplug.c @@ -210,13 +210,6 @@ static void irq_restore_affinity_of_irq(struct irq_des= c *desc, unsigned int cpu) !irq_data_get_irq_chip(data) || !cpumask_test_cpu(cpu, affinity)) return; =20 - /* - * Don't restore suspended interrupts here when a system comes back - * from S3. They are reenabled via resume_device_irqs(). - */ - if (desc->istate & IRQS_SUSPENDED) - return; - if (irqd_is_managed_and_shutdown(data)) irq_startup_managed(desc); =20 --=20 2.50.0.rc1.591.g9c95f17f64-goog