From nobody Wed Dec 17 08:51:18 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 591B2219E8 for ; Sun, 11 May 2025 16:35:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746981327; cv=none; b=eleyAhVecWA+xem8Nz2sltJqjEkxajRkuoKOesgzNbDBDF+FyMjh6Sp/4KFqU6B6jU64NB8KGPEN1ujUKntCHCEJoH46kMCsfKGggCOlnhyjJ/hd/tg9CUUf8GZUhcUOiRFwJv1947Opitov2JBg9UIjOMbgc7Yf3foEgqQgjCA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746981327; c=relaxed/simple; bh=GaOQnEbLL9GZs6ot00bxwo1vTakAiiwicLKpRgJPfmY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=aj8mc9EiDxfcWpPIh/H9/skrlIauGhpf0GA/7G57WuDijQ5niCy6gBWODz1+2qtmwiiAaGZcO/Pi4EeMrwG3Vv7+q8SxoTOATMl10Cpf5NY156JE4zh+vazp9OCmpch8pidC2KpAyqH8NX8XvTzv6r0o7/7fdmrlgsy3jFd/BT8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=fTTKFRdM; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="fTTKFRdM" Received: by smtp.kernel.org (Postfix) with ESMTPSA id F22E4C4CEE7; Sun, 11 May 2025 16:35:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1746981327; bh=GaOQnEbLL9GZs6ot00bxwo1vTakAiiwicLKpRgJPfmY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=fTTKFRdM3VnzPD88jFOf2OV/Rw99Y5XQlU1eZ6IgHDHKwj6ZnakTwEXRbIWvqGMFQ qJsyHuz6iw2qAOIeJylFbITR9Ls2d3geAGDGwJ4aQLjQGYG7vNLm96albse7nEdLje wESOJmauQV1z8L3epNxV85hxSCVp2SC3PZAf+jtlWoSQjlnab9xG+JXr/jnGJNteL2 67f2x0zVBwuUb/m+3LMKlsu80JJGqH2Tn9l+nZ2yt7oqTtCFV/xt3zDU2vnlus6pCI guT6JyN6qv6lvLzDPrJarxXzRgSlp6ZE6yPZ3qx6yibhOD9KkfLsLK+RyTxXZeb0EC TksPyGnhgkvgA== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1uE9e8-00Dt22-L7; Sun, 11 May 2025 17:35:24 +0100 From: Marc Zyngier To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: Thomas Gleixner , Lorenzo Pieralisi , Sascha Bischoff , Timothy Hayes Subject: [PATCH 1/4] genirq/msi: Add .msi_teardown() callback as the reverse of .msi_prepare() Date: Sun, 11 May 2025 17:35:17 +0100 Message-Id: <20250511163520.1307654-2-maz@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250511163520.1307654-1-maz@kernel.org> References: <20250511163520.1307654-1-maz@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, tglx@linutronix.de, lpieralisi@kernel.org, sascha.bischoff@arm.com, timothy.hayes@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Content-Type: text/plain; charset="utf-8" While the MSI ops do have a .msi_prepare() callback that is responsible for setting up the relevant (usually per-device) allocation, we don't have a callback reversing this setup. For this purpose, let's a .msi_teardown() callback. This is reliying on the msi_domain_info structure having a non-NULL alloc_data field. Nobody is populating this field yet, so there is no change in behaviour yet. Signed-off-by: Marc Zyngier --- include/linux/msi.h | 10 ++++++++-- kernel/irq/msi.c | 12 ++++++++++++ 2 files changed, 20 insertions(+), 2 deletions(-) diff --git a/include/linux/msi.h b/include/linux/msi.h index 86e42742fd0fb..0a44a2cba3105 100644 --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -420,6 +420,7 @@ struct msi_domain_info; * @msi_init: Domain specific init function for MSI interrupts * @msi_free: Domain specific function to free a MSI interrupts * @msi_prepare: Prepare the allocation of the interrupts in the domain + * @msi_teardown: Reverse the effects of @msi_prepare * @prepare_desc: Optional function to prepare the allocated MSI descriptor * in the domain * @set_desc: Set the msi descriptor for an interrupt @@ -435,8 +436,9 @@ struct msi_domain_info; * @get_hwirq, @msi_init and @msi_free are callbacks used by the underlying * irqdomain. * - * @msi_check, @msi_prepare, @prepare_desc and @set_desc are callbacks use= d by the - * msi_domain_alloc/free_irqs*() variants. + * @msi_check, @msi_prepare, @msi_teardown, @prepare_desc and + * @set_desc are callbacks used by the msi_domain_alloc/free_irqs*() + * variants. * * @domain_alloc_irqs, @domain_free_irqs can be used to override the * default allocation/free functions (__msi_domain_alloc/free_irqs). This @@ -458,6 +460,8 @@ struct msi_domain_ops { int (*msi_prepare)(struct irq_domain *domain, struct device *dev, int nvec, msi_alloc_info_t *arg); + void (*msi_teardown)(struct irq_domain *domain, + msi_alloc_info_t *arg); void (*prepare_desc)(struct irq_domain *domain, msi_alloc_info_t *arg, struct msi_desc *desc); void (*set_desc)(msi_alloc_info_t *arg, @@ -486,6 +490,7 @@ struct msi_domain_ops { * @handler: Optional: associated interrupt flow handler * @handler_data: Optional: associated interrupt flow handler data * @handler_name: Optional: associated interrupt flow handler name + * @alloc_data: Optional: associated interrupt allocation data * @data: Optional: domain specific data */ struct msi_domain_info { @@ -498,6 +503,7 @@ struct msi_domain_info { irq_flow_handler_t handler; void *handler_data; const char *handler_name; + msi_alloc_info_t *alloc_data; void *data; }; =20 diff --git a/kernel/irq/msi.c b/kernel/irq/msi.c index c05ba7ca00faa..a65ccf19b15d9 100644 --- a/kernel/irq/msi.c +++ b/kernel/irq/msi.c @@ -796,6 +796,11 @@ static int msi_domain_ops_prepare(struct irq_domain *d= omain, struct device *dev, return 0; } =20 +static void msi_domain_ops_teardown(struct irq_domain *domain, + msi_alloc_info_t *arg) +{ +} + static void msi_domain_ops_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc) { @@ -821,6 +826,7 @@ static struct msi_domain_ops msi_domain_ops_default =3D= { .get_hwirq =3D msi_domain_ops_get_hwirq, .msi_init =3D msi_domain_ops_init, .msi_prepare =3D msi_domain_ops_prepare, + .msi_teardown =3D msi_domain_ops_teardown, .set_desc =3D msi_domain_ops_set_desc, }; =20 @@ -842,6 +848,8 @@ static void msi_domain_update_dom_ops(struct msi_domain= _info *info) ops->msi_init =3D msi_domain_ops_default.msi_init; if (ops->msi_prepare =3D=3D NULL) ops->msi_prepare =3D msi_domain_ops_default.msi_prepare; + if (ops->msi_teardown =3D=3D NULL) + ops->msi_teardown =3D msi_domain_ops_default.msi_teardown; if (ops->set_desc =3D=3D NULL) ops->set_desc =3D msi_domain_ops_default.set_desc; } @@ -1088,6 +1096,10 @@ void msi_remove_device_irq_domain(struct device *dev= , unsigned int domid) =20 dev->msi.data->__domains[domid].domain =3D NULL; info =3D domain->host_data; + + if (info->alloc_data) + info->ops->msi_teardown(domain, info->alloc_data); + if (irq_domain_is_msi_device(domain)) fwnode =3D domain->fwnode; irq_domain_remove(domain); --=20 2.39.2 From nobody Wed Dec 17 08:51:18 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 73788258CFB for ; Sun, 11 May 2025 16:35:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746981327; cv=none; b=mXkb4XCLLrYIZgMjXzMQnR+dHtXi92BBTEnTIcHWcomsNaMxX8TdmCVumHR1zHUVwSr4tBv9SpQgvNWwUgN94t9hzyguvMHGmvCU9ikXJoxLc+ZaFs46viVKZVlfbW/Q2u2fQZxrSHVtwnyo/QFCmVoX5lldkttIOlCzWiwyq4g= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746981327; c=relaxed/simple; bh=MkHl02Hcg/C7P/syAv18JHro8Do6xpxM2jr5nO1nbsk=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=gvZY/warNoH/PAL9TFoA5Hae+S6BUs7q7gCclyXkXCuhX+sqIhRvOAlhNzAPnNGOrp7xZV8GDFXKhGDQ57b8cKwHrO3dD0N/qWg5qC3won458tyHt0Wki39mmop4BcSNPOhLTbNJ0D0XgZDLdgrYHPv9p+nHnCG55iEjK2xRvqo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=RJuzghMI; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="RJuzghMI" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 06494C4AF09; Sun, 11 May 2025 16:35:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1746981327; bh=MkHl02Hcg/C7P/syAv18JHro8Do6xpxM2jr5nO1nbsk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=RJuzghMIFy93VmeAdc0IgjOjuB/Xh/vCUxyzAT9AoC6LkyjWgQ/LQfDqcNh74AuKb oZSXhJtWB2L7razBEJO4q4Rv5zZhedjDrbYrm4fvGWTubJcoyOaACaksA3FtFVS2/x 4rk54zMacmJpoy7Wq4ekz3oqQJo2xFvAtnGFmI0EOf3Qg5S6ht1SmLCky5FxAcleDT Rs5DsrHLHxhlTN+rHO9qoYBKzIH/pt7SUL9AV3zdbgzNzBmF8g2uaft7Rv3xHvQGWk qePrDU9WxRQ3HbIE0hjKguulDSaqOXETnR1F6DpNYKrEm2D5IC9/IjBjvG3bH0M/Sc /r8NDLoCRUWHw== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1uE9e8-00Dt22-Th; Sun, 11 May 2025 17:35:24 +0100 From: Marc Zyngier To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: Thomas Gleixner , Lorenzo Pieralisi , Sascha Bischoff , Timothy Hayes Subject: [PATCH 2/4] irqchip/gic-v3-its: Implement .msi_teardown() callback Date: Sun, 11 May 2025 17:35:18 +0100 Message-Id: <20250511163520.1307654-3-maz@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250511163520.1307654-1-maz@kernel.org> References: <20250511163520.1307654-1-maz@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, tglx@linutronix.de, lpieralisi@kernel.org, sascha.bischoff@arm.com, timothy.hayes@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Content-Type: text/plain; charset="utf-8" We currently nuke the structure representing an endpoint device translating via an ITS on freeing the last LPI allocated for it. That's an unfortunate state of affair, as it is pretty common for a driver to allocate a single MSI, do something clever, teardown this MSI, and reallocate a whole bunch of them. The nvme driver does exactly that, amongst others. What happens in that case is that the core code is buggy enough to issue another .msi_prepare() call, even if it shouldn't. This luckily cancels the above behaviour and hides the problem. In order to fix the core code, let's start by implementing the new .msi_teardown() callback. Nothing calls it yet, so a side effect is that the its_dev structure will not be freed and that the DID will stay mapped. Not a big deal, and this will be solved in the following patch. Signed-off-by: Marc Zyngier --- drivers/irqchip/irq-gic-v3-its-msi-parent.c | 10 ++++ drivers/irqchip/irq-gic-v3-its.c | 56 +++++++++++++-------- 2 files changed, 45 insertions(+), 21 deletions(-) diff --git a/drivers/irqchip/irq-gic-v3-its-msi-parent.c b/drivers/irqchip/= irq-gic-v3-its-msi-parent.c index bdb04c8081480..76b94f55b00e2 100644 --- a/drivers/irqchip/irq-gic-v3-its-msi-parent.c +++ b/drivers/irqchip/irq-gic-v3-its-msi-parent.c @@ -159,6 +159,14 @@ static int its_pmsi_prepare(struct irq_domain *domain,= struct device *dev, dev, nvec, info); } =20 +static void its_msi_teardown(struct irq_domain *domain, msi_alloc_info_t *= info) +{ + struct msi_domain_info *msi_info; + + msi_info =3D msi_get_domain_info(domain->parent); + msi_info->ops->msi_teardown(domain->parent, info); +} + static bool its_init_dev_msi_info(struct device *dev, struct irq_domain *d= omain, struct irq_domain *real_parent, struct msi_domain_info *info) { @@ -182,6 +190,7 @@ static bool its_init_dev_msi_info(struct device *dev, s= truct irq_domain *domain, * %MSI_MAX_INDEX. */ info->ops->msi_prepare =3D its_pci_msi_prepare; + info->ops->msi_teardown =3D its_msi_teardown; break; case DOMAIN_BUS_DEVICE_MSI: case DOMAIN_BUS_WIRED_TO_MSI: @@ -190,6 +199,7 @@ static bool its_init_dev_msi_info(struct device *dev, s= truct irq_domain *domain, * size is also known at domain creation time. */ info->ops->msi_prepare =3D its_pmsi_prepare; + info->ops->msi_teardown =3D its_msi_teardown; break; default: /* Confused. How did the lib return true? */ diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-= its.c index 0115ad6c82593..3472b97477104 100644 --- a/drivers/irqchip/irq-gic-v3-its.c +++ b/drivers/irqchip/irq-gic-v3-its.c @@ -3620,8 +3620,43 @@ static int its_msi_prepare(struct irq_domain *domain= , struct device *dev, return err; } =20 +static void its_msi_teardown(struct irq_domain *domain, msi_alloc_info_t *= info) +{ + struct msi_domain_info *msi_info; + struct its_device *its_dev; + struct its_node *its; + u32 dev_id; + + dev_id =3D info->scratchpad[0].ul; + + msi_info =3D msi_get_domain_info(domain); + its =3D msi_info->data; + + guard(mutex)(&its->dev_alloc_lock); + + its_dev =3D its_find_device(its, dev_id); + + /* If the device is shared, keep everything around */ + if (its_dev->shared) + return; + + /* LPIs should have been already unmapped at this stage */ + if (WARN_ON_ONCE(!bitmap_empty(its_dev->event_map.lpi_map, + its_dev->event_map.nr_lpis))) + return; + + its_lpi_free(its_dev->event_map.lpi_map, + its_dev->event_map.lpi_base, + its_dev->event_map.nr_lpis); + + /* Unmap device/itt, and get rid of the tracking */ + its_send_mapd(its_dev, 0); + its_free_device(its_dev); +} + static struct msi_domain_ops its_msi_domain_ops =3D { .msi_prepare =3D its_msi_prepare, + .msi_teardown =3D its_msi_teardown, }; =20 static int its_irq_gic_domain_alloc(struct irq_domain *domain, @@ -3722,7 +3757,6 @@ static void its_irq_domain_free(struct irq_domain *do= main, unsigned int virq, { struct irq_data *d =3D irq_domain_get_irq_data(domain, virq); struct its_device *its_dev =3D irq_data_get_irq_chip_data(d); - struct its_node *its =3D its_dev->its; int i; =20 bitmap_release_region(its_dev->event_map.lpi_map, @@ -3736,26 +3770,6 @@ static void its_irq_domain_free(struct irq_domain *d= omain, unsigned int virq, irq_domain_reset_irq_data(data); } =20 - mutex_lock(&its->dev_alloc_lock); - - /* - * If all interrupts have been freed, start mopping the - * floor. This is conditioned on the device not being shared. - */ - if (!its_dev->shared && - bitmap_empty(its_dev->event_map.lpi_map, - its_dev->event_map.nr_lpis)) { - its_lpi_free(its_dev->event_map.lpi_map, - its_dev->event_map.lpi_base, - its_dev->event_map.nr_lpis); - - /* Unmap device/itt */ - its_send_mapd(its_dev, 0); - its_free_device(its_dev); - } - - mutex_unlock(&its->dev_alloc_lock); - irq_domain_free_irqs_parent(domain, virq, nr_irqs); } =20 --=20 2.39.2 From nobody Wed Dec 17 08:51:18 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 592052AF1C for ; Sun, 11 May 2025 16:35:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746981327; cv=none; b=qjn6gArMVzBNTOMP2IdrEuEbjczTdNpf6BmVAg2o+t29ikaWVNPQdEhtYJKj9s/Oxch1K4SfyEbXUmAW7Wd+ygc/5uxrjSHKjCSrtkeWCCcBdUCsL//HJ848uJ4M3h25zy6gPGEqOI3DFmuy3pPlZbS3YBIEsZ3SSYhhHL0Dm5o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746981327; c=relaxed/simple; bh=WM4iVRpLhz+0Y4LZdK+pjtHeHUh/P8d+elq7ZVpAd8k=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=R0gNRVT3TJoB5fnCbF9/RYX0AX6qhCSggQEtXt0up6lchAxs3fhpSRacQSiq9xtr4nv2HPmuKLsXJLEiOJ5R4qIOKGgbAQ9mQchcBFz/f9Us3Z2OzDpBdwH1Rj1GeT2c4nZGQbq4JStZhR7cblRUpK3z6i1PTE0rb78/K8KOZaM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=t4Alog4W; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="t4Alog4W" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 016D6C4CEF0; Sun, 11 May 2025 16:35:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1746981327; bh=WM4iVRpLhz+0Y4LZdK+pjtHeHUh/P8d+elq7ZVpAd8k=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=t4Alog4WA6uqvAd2IOMp0V2SmzhOtQhADFTx/KQTmEkcfvur5HmNd0mTB9/E6i8to mJSikDZOWrAEvS7HC6oFDzinJ9LToFXH7J3RMCJi/j2/qTpr2LjtlB5AmsL585VZaw FLe1DdnwFnm8AojwebaMPmcaDZUKKe9Y+xz75ihAPPk8d0/v97PzJt2to40NMSDAKz ULf56oicYum7pxGKeMrqpLqG3YA0Vn5Rdkito0hsfkLdrWejs/aA0G94Y8HiPz0CuR xX6tv25/gY1U39EcyPaImQp58R/jjEcaaAwPwQx6C9fbUxwyusUIGL2Hm0vFSodNS9 B2xeYZZ/3cXlg== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1uE9e9-00Dt22-4V; Sun, 11 May 2025 17:35:25 +0100 From: Marc Zyngier To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: Thomas Gleixner , Lorenzo Pieralisi , Sascha Bischoff , Timothy Hayes Subject: [PATCH 3/4] genirq/msi: Move prepare() call to per-device allocation Date: Sun, 11 May 2025 17:35:19 +0100 Message-Id: <20250511163520.1307654-4-maz@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250511163520.1307654-1-maz@kernel.org> References: <20250511163520.1307654-1-maz@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, tglx@linutronix.de, lpieralisi@kernel.org, sascha.bischoff@arm.com, timothy.hayes@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Content-Type: text/plain; charset="utf-8" The current device MSI infrastructure is subtly broken, as it will issue an .msi_prepare() callback into the MSI controller driver every time it needs to allocate an MSI. That's pretty wrong, as the contract between the MSI controller and the core code is that .msi_prepare() is called exactly once per device. This leads to some subtle breakage in said MSI controller drivers, as it gives the impression that there are multiple endpoints sharing a bus identifier (RID in PCI parlance, DID for GICv3+). It implies that whatever allocation the ITS driver (for example) has done on behalf of these devices cannot be undone, as there is no way to track the shared state. This is particularly bad for wire-MSI devices, for which .msi_prepare() is called for. each. input. line. To address this issue, move the call to .msi_prepare() to take place at the point of irq domain allocation, which is the only place that makes sense. The msi_alloc_info_t structure is made part of the msi_domain_template, so that its life-cycle is that of the domain as well. Finally, the msi_info::alloc_data field is made to point at this allocation tracking structure, ensuring that it is carried around the block. This is all pretty straightforward, except for the non-device-MSI leftovers, which still have to call .msi_prepare() at the old spot. One day... Signed-off-by: Marc Zyngier --- include/linux/msi.h | 2 ++ kernel/irq/msi.c | 39 ++++++++++++++++++++++++++++++++++----- 2 files changed, 36 insertions(+), 5 deletions(-) diff --git a/include/linux/msi.h b/include/linux/msi.h index 0a44a2cba3105..68a8b2d03eba9 100644 --- a/include/linux/msi.h +++ b/include/linux/msi.h @@ -513,12 +513,14 @@ struct msi_domain_info { * @chip: Interrupt chip for this domain * @ops: MSI domain ops * @info: MSI domain info data + * @arg: MSI domain allocation data (arch specific) */ struct msi_domain_template { char name[48]; struct irq_chip chip; struct msi_domain_ops ops; struct msi_domain_info info; + msi_alloc_info_t arg; }; =20 /* diff --git a/kernel/irq/msi.c b/kernel/irq/msi.c index a65ccf19b15d9..b8dc3289958c6 100644 --- a/kernel/irq/msi.c +++ b/kernel/irq/msi.c @@ -59,7 +59,8 @@ struct msi_ctrl { static void msi_domain_free_locked(struct device *dev, struct msi_ctrl *ct= rl); static unsigned int msi_domain_get_hwsize(struct device *dev, unsigned int= domid); static inline int msi_sysfs_create_group(struct device *dev); - +static int msi_domain_prepare_irqs(struct irq_domain *domain, struct devic= e *dev, + int nvec, msi_alloc_info_t *arg); =20 /** * msi_alloc_desc - Allocate an initialized msi_desc @@ -1025,6 +1026,7 @@ bool msi_create_device_irq_domain(struct device *dev,= unsigned int domid, bundle->info.ops =3D &bundle->ops; bundle->info.data =3D domain_data; bundle->info.chip_data =3D chip_data; + bundle->info.alloc_data =3D &bundle->arg; =20 pops =3D parent->msi_parent_ops; snprintf(bundle->name, sizeof(bundle->name), "%s%s-%s", @@ -1053,21 +1055,28 @@ bool msi_create_device_irq_domain(struct device *de= v, unsigned int domid, msi_lock_descs(dev); =20 if (WARN_ON_ONCE(msi_get_device_domain(dev, domid))) - goto fail; + goto fail_unlock; =20 if (!pops->init_dev_msi_info(dev, parent, parent, &bundle->info)) - goto fail; + goto fail_unlock; =20 domain =3D __msi_create_irq_domain(fwnode, &bundle->info, IRQ_DOMAIN_FLAG= _MSI_DEVICE, parent); if (!domain) - goto fail; + goto fail_unlock; =20 domain->dev =3D dev; dev->msi.data->__domains[domid].domain =3D domain; + + if (msi_domain_prepare_irqs(domain, dev, hwsize, &bundle->arg)) + goto fail; + msi_unlock_descs(dev); return true; =20 fail: + dev->msi.data->__domains[domid].domain =3D NULL; + irq_domain_remove(domain); +fail_unlock: msi_unlock_descs(dev); free_fwnode: irq_domain_free_fwnode(fwnalloced); @@ -1250,6 +1259,26 @@ static int msi_init_virq(struct irq_domain *domain, = int virq, unsigned int vflag return 0; } =20 +static int __populate_alloc_info(struct irq_domain *domain, struct device = *dev, + unsigned int nirqs, msi_alloc_info_t *arg) +{ + struct msi_domain_info *info =3D domain->host_data; + int ret =3D 0; + + /* + * If the caller has provided a template alloc info, use that. Once + * all users of msi_create_irq_domain() have been eliminated, this + * should be the only source of allocation information, and the + * prepare call below should be finally removed. + */ + if (info->alloc_data) + *arg =3D *info->alloc_data; + else + ret =3D msi_domain_prepare_irqs(domain, dev, nirqs, arg); + + return ret; +} + static int __msi_domain_alloc_irqs(struct device *dev, struct irq_domain *= domain, struct msi_ctrl *ctrl) { @@ -1262,7 +1291,7 @@ static int __msi_domain_alloc_irqs(struct device *dev= , struct irq_domain *domain unsigned long idx; int i, ret, virq; =20 - ret =3D msi_domain_prepare_irqs(domain, dev, ctrl->nirqs, &arg); + ret =3D __populate_alloc_info(domain, dev, ctrl->nirqs, &arg); if (ret) return ret; =20 --=20 2.39.2 From nobody Wed Dec 17 08:51:18 2025 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A2CF525CC5F for ; Sun, 11 May 2025 16:35:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746981327; cv=none; b=DYowFWSBd+/vSH1Lc14p2ovEFmdzSKzzjoS6EFb/uZKwcFrqs0cro2T8wnm10RBobqWDha7oTzPX/AXGMw6UDelwApNAfxxwDLFrJw2ntvMe/6Z8z5+/QK+0rvSRMlCMnQk5zQET78dENHKLiOVMlNd8al+gjn5of2uXyhqGlOE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1746981327; c=relaxed/simple; bh=yXnSYyP1mrYQ+gK1si8iEF8hympcNOwuZnqV8CQOOhY=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version; b=gclO79ntSfP1VTbP6i1ONRMJLSP8Jk3Eaq5Qdpr3JD6GmdO18SGxcZPJlAZ17pJypvus3LASuulqLL1tkF6bmedwreE07r3r2PzwgMi/mCBwej9WDkFDGsZysQRj6MwwR6ZXE64jNRLIIOb7J895aLy3OE8w6Osl3iAFZ1BZRyA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=jK4tUp/N; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="jK4tUp/N" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 473BCC4CEF2; Sun, 11 May 2025 16:35:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1746981327; bh=yXnSYyP1mrYQ+gK1si8iEF8hympcNOwuZnqV8CQOOhY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jK4tUp/NSPxJvfCScx7PH5tKDO238/6DSRZQRi8XZu9ooqolUp9JcE8LJHSxtjwE+ cH1wzPQhXz0Ri6NTnlZ4s/XgE38wdrYbW6PYjfDcN9urOIdLKMQcMLx74eJw/D/Dla wbFvvec2kmuu6qOcnJLTRkMDmm+SKg2LawB3W9p9wZ4ufRIBye+shDoMA9czT7wuSt lHKjSsC8ek3ORgoy9pcfVqoM4DSgtG+OFFviSRWLXJZD8nkL04raTRU87ARMhzQoCw h5ZOPITzuUt1wmw9FLCeGEG0tv1m1aEb/Ad91FZWfsHjtqNgJt6sDGXkUtcydsLIbK UOa/92k7HHVqg== Received: from sofa.misterjones.org ([185.219.108.64] helo=valley-girl.lan) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1uE9e9-00Dt22-AG; Sun, 11 May 2025 17:35:25 +0100 From: Marc Zyngier To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org Cc: Thomas Gleixner , Lorenzo Pieralisi , Sascha Bischoff , Timothy Hayes Subject: [PATCH 4/4] irqchip/gic-v3-its: Use allocation size from the prepare call Date: Sun, 11 May 2025 17:35:20 +0100 Message-Id: <20250511163520.1307654-5-maz@kernel.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20250511163520.1307654-1-maz@kernel.org> References: <20250511163520.1307654-1-maz@kernel.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, tglx@linutronix.de, lpieralisi@kernel.org, sascha.bischoff@arm.com, timothy.hayes@arm.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Content-Type: text/plain; charset="utf-8" Now that .msi_prepare() gets called at the right time and not with semi-random parameters, remove the ugly hack that tried to fix up the number of allocated vectors. It is now correct by construction. Signed-off-by: Marc Zyngier --- drivers/irqchip/irq-gic-v3-its-msi-parent.c | 19 ------------------- 1 file changed, 19 deletions(-) diff --git a/drivers/irqchip/irq-gic-v3-its-msi-parent.c b/drivers/irqchip/= irq-gic-v3-its-msi-parent.c index 76b94f55b00e2..2df75a758c104 100644 --- a/drivers/irqchip/irq-gic-v3-its-msi-parent.c +++ b/drivers/irqchip/irq-gic-v3-its-msi-parent.c @@ -67,17 +67,6 @@ static int its_pci_msi_prepare(struct irq_domain *domain= , struct device *dev, /* ITS specific DeviceID, as the core ITS ignores dev. */ info->scratchpad[0].ul =3D pci_msi_domain_get_msi_rid(domain->parent, pde= v); =20 - /* - * @domain->msi_domain_info->hwsize contains the size of the - * MSI[-X] domain, but vector allocation happens one by one. This - * needs some thought when MSI comes into play as the size of MSI - * might be unknown at domain creation time and therefore set to - * MSI_MAX_INDEX. - */ - msi_info =3D msi_get_domain_info(domain); - if (msi_info->hwsize > nvec) - nvec =3D msi_info->hwsize; - /* * Always allocate a power of 2, and special case device 0 for * broken systems where the DevID is not wired (and all devices @@ -143,14 +132,6 @@ static int its_pmsi_prepare(struct irq_domain *domain,= struct device *dev, /* ITS specific DeviceID, as the core ITS ignores dev. */ info->scratchpad[0].ul =3D dev_id; =20 - /* - * @domain->msi_domain_info->hwsize contains the size of the device - * domain, but vector allocation happens one by one. - */ - msi_info =3D msi_get_domain_info(domain); - if (msi_info->hwsize > nvec) - nvec =3D msi_info->hwsize; - /* Allocate at least 32 MSIs, and always as a power of 2 */ nvec =3D max_t(int, 32, roundup_pow_of_two(nvec)); =20 --=20 2.39.2