From nobody Sun Nov 24 07:39:08 2024 Received: from mail-ej1-f47.google.com (mail-ej1-f47.google.com [209.85.218.47]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6FFC7206973 for ; Wed, 6 Nov 2024 17:51:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.47 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730915469; cv=none; b=Cjl2wX/ILykDAve7Sspjcmqbfs+QLHtqmvbMNK/1hFlKddrqVagkLjl2Lx6Y7de2BR6PM9rhB+ekj0T0Oi1jAdwTCfjoR9ZkmbXII44/2FDscOjlhHLvhk/xzUm0MVieq1QnkFPAZehZDZUaATKjcOgwiM6b7CEQty/dS8bMytA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730915469; c=relaxed/simple; bh=kGFqsIlFumYIKvxXpVDJi2uzeK9bhsdbBadYRpJVUfs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cB+GfaR3BWE/gzb/Ey+hk6PLmqCTQ4HTCNKBtPIqMY/qG5w4eAH324npLnU5Xo5gL64Ly8W2rVUoq4n8i+DGFKsMiEQtHDHbLrPjT5RTzH/1xoQ3n0NU4QW4DL1K9ggleeGZrbU1EK28Q1gVwsXGyYPihUpIHqzcmIfAvc4/69s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ventanamicro.com; spf=pass smtp.mailfrom=ventanamicro.com; dkim=pass (2048-bit key) header.d=ventanamicro.com header.i=@ventanamicro.com header.b=YPn5eH69; arc=none smtp.client-ip=209.85.218.47 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ventanamicro.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ventanamicro.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ventanamicro.com header.i=@ventanamicro.com header.b="YPn5eH69" Received: by mail-ej1-f47.google.com with SMTP id a640c23a62f3a-a9e8522c10bso14961866b.1 for ; Wed, 06 Nov 2024 09:51:07 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ventanamicro.com; s=google; t=1730915466; x=1731520266; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=g96IwKhGkFLWIqbkT7NwpHd+U4DZ81t2GIHo2pf1780=; b=YPn5eH69uCOS65GIZg8ew0RrqLlhUqjm/LnpIILfhmBHvU6rAEfm7RxUxZ/EX9LK8O IKCQl0MI3ytCIVOQgghRbMqUsztym9b9mmVF4oNaue8mSxxF0JsslNwq+Ky/NJFcEh7N yJboavpJAA84fCgrW2WoXENLa04F0lgkHnRoZxzb8gb39QP+DaAyOy5xo58wjiW8gACp rdYyEhnVKXfrIwNtByouAZDIdjylnT0IgBf64XQF+ML8BTpWHLx84j4H2JCfbS6CE++l 6Oin874K6k4zMHUIsDnta4QxPM+C3NxUBCGp2pSDEWH/GFgOaO2mdtErL0oizU3YB5Sn 7/rQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730915466; x=1731520266; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=g96IwKhGkFLWIqbkT7NwpHd+U4DZ81t2GIHo2pf1780=; b=Fprp+6+BhLBuhkTVkJ4GhW0oe6vyZU4VP3WoNsPcyV131oU0F+1ZjNDP1ak8vSwhlH 2I+J0GztVqXinBNv2syZGJJXuSvzoLOifS7idsgEGCfWEVyLnU0KuNIZzqODLgU0QhpO +WsgzRZl9shLSIL7irW/Kvt5EUR7wMGYrI3x4PVE2QbK5JIy7CgD5tIGj9g2qlFQZBny 7LHYuN+SHdIZg2Cvlm7tvusdXJ7DAboUC1CWaBvPv0Zf25IzwHQjs3+Cr6PaeUNMLC7d mXIAm4tP64jCEdzNuPeUkGWR5wzAKjC+qXxxfE+tJYshfqf/qn2YZKhZKUcFRTutYgEc HWKA== X-Forwarded-Encrypted: i=1; AJvYcCV8Zo4oENWCSpLvChPHXRK70Zc4pnPdt88sGsQCcCb3A50o3hl7n9uLyIgQeiHVDW7vuPS0Khm4DWpTVsM=@vger.kernel.org X-Gm-Message-State: AOJu0YyZPGAkuMR/bPRi5KRJ6Bl2EFLHxv5YhIx6gusXdUbRzSasIuyH ofQmvKXOhh/0JFwFOsV+HX74t/wrN0zYScfe6dzH0tHDFC5O5s3IDz3e9mndfG4= X-Google-Smtp-Source: AGHT+IEFqB7EqZ9/qt6Tv+l9s3WePIJJrBkQGmuE+wise2peEn4wuYWmsni9+gmk+GDjL44CfjLWRg== X-Received: by 2002:a17:907:948d:b0:a9a:597:8cc9 with SMTP id a640c23a62f3a-a9ed50cb67dmr4422366b.12.1730915465830; Wed, 06 Nov 2024 09:51:05 -0800 (PST) Received: from localhost (2001-1ae9-1c2-4c00-20f-c6b4-1e57-7965.ip6.tmcz.cz. [2001:1ae9:1c2:4c00:20f:c6b4:1e57:7965]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a9eb16d4d6bsm309041166b.44.2024.11.06.09.51.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Nov 2024 09:51:05 -0800 (PST) From: Andrew Jones To: iommu@lists.linux.dev, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Cc: tjeznach@rivosinc.com, joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu Subject: [PATCH 1/2] iommu/riscv: Free irq vectors on pci remove Date: Wed, 6 Nov 2024 18:51:04 +0100 Message-ID: <20241106175102.219923-5-ajones@ventanamicro.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241106175102.219923-4-ajones@ventanamicro.com> References: <20241106175102.219923-4-ajones@ventanamicro.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" riscv_iommu_pci_probe() calls pci_alloc_irq_vectors() which states pci_free_irq_vectors() must be called on cleanup. Fixes: 68682e9578fb ("iommu/riscv: Add RISC-V IOMMU PCIe device driver") Signed-off-by: Andrew Jones --- drivers/iommu/riscv/iommu-pci.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/iommu/riscv/iommu-pci.c b/drivers/iommu/riscv/iommu-pc= i.c index c7a89143014c..25a27e627a0e 100644 --- a/drivers/iommu/riscv/iommu-pci.c +++ b/drivers/iommu/riscv/iommu-pci.c @@ -99,6 +99,7 @@ static void riscv_iommu_pci_remove(struct pci_dev *pdev) struct riscv_iommu_device *iommu =3D dev_get_drvdata(&pdev->dev); =20 riscv_iommu_remove(iommu); + pci_free_irq_vectors(pdev); } =20 static const struct pci_device_id riscv_iommu_pci_tbl[] =3D { --=20 2.47.0 From nobody Sun Nov 24 07:39:08 2024 Received: from mail-ej1-f43.google.com (mail-ej1-f43.google.com [209.85.218.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F23D8207218 for ; Wed, 6 Nov 2024 17:51:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.218.43 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730915470; cv=none; b=EqPnRFMNHmL7iIItA8UvYWI8KEiIztTVzc6WicSs6n2aa1we+R6MCcoXVldD2rjcurU1w9Xipmqk/3rHTdc1L8lxl0ptuAkGR1On5PZYXgkaP28jZMjxUw6GcXg1WIT6kjYfWE7ilb77a8iNMVKQ91WNeXlYwTnQxW0EvZJ+KbA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1730915470; c=relaxed/simple; bh=dIWVb2g9QRqcXTaV1s47gMyEVtB2onyQwlrAJa4Jj1E=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qJPGw2ENeBGiQSc2Mm51nubqwnyDUIN7c4ybmHmOUKHYACrO9PKcimVxV6J0YHhmOk0mQWLuGCAMISAyP7AivulR94K06cn8qXCe9SwqTmaxOpKYIecy57iM0/kggQz6zS+pcAVLSmpsDLRln5EPjEFiz1Cw2eEwG1INNgwY2u0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ventanamicro.com; spf=pass smtp.mailfrom=ventanamicro.com; dkim=pass (2048-bit key) header.d=ventanamicro.com header.i=@ventanamicro.com header.b=izl5UNHz; arc=none smtp.client-ip=209.85.218.43 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=ventanamicro.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ventanamicro.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ventanamicro.com header.i=@ventanamicro.com header.b="izl5UNHz" Received: by mail-ej1-f43.google.com with SMTP id a640c23a62f3a-a9ec86a67feso2664766b.1 for ; Wed, 06 Nov 2024 09:51:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ventanamicro.com; s=google; t=1730915467; x=1731520267; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=49hG3lsHGrJySpAEzyxRduAfSvP7r1BXPqy63OvAnoQ=; b=izl5UNHzJoVBRdT0WPqM1KjQkaDwaVf2oLbbE5zexih4M36jpqW/mHnuy/l8+QPW6D hthQchDEsWH+Ap3YT2C45VN8GlHwqa4HdotDvc8QxmTKIMOF6VUdU/xa7yZMO+PnDwTe 601EJsHo73/ojX4ldLoCfnn4vioPdJF/qTfnNOt2NfD6aq6GxV6RNIDqRXJ4+et9lwHS pWB8BgBZlqrwnbg4boFIRv4lVducSSlFgPWGX+28PuUNXanCDUNlD4gLrGkyx21C8LZ1 mQmWr2wncCkbWOpoaqoln3WptTyNwJy5nJpdKul2ZrMTOXUBUVPX+QW1PKNoXcvE/eJR rkFg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1730915467; x=1731520267; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=49hG3lsHGrJySpAEzyxRduAfSvP7r1BXPqy63OvAnoQ=; b=jSpjm5+FdC1fSp4xvNIbc2rKJgOuJpWcEzJpm+UJmjpOZ+RAw7C1OV2mWzcM43yWZj EJv0QF597GxGJsjVRbdyqWTbcQWd2veJTh15mslE0xdcWu4NNyLQErU0y3ttWC8V56Bw 6smsWHgGHC+eiuPbmnmOOzLawx2x0xHVH7C5iYnCzxNKDqBzycuzXj2L4Lz1/c+M7aSp sS4aUwijr4LqjZJ0cISeP6QplRUMJZ3ZJ2sKEyDQ9qVAVbrZm6rq2BAr9YAlYsBo2Js2 j2cjUTl9v5bF7Gf08ZuCF49GYykn+GrySC0SaSM+md+j/cyRlnh+NNSuEZ0rr8CEA3y/ tglg== X-Forwarded-Encrypted: i=1; AJvYcCWvgbqIDz5RNS2NrmPKvYnBYrPapGRbX6bfyp11XVbkwm2boXji614ULpkNIbKzpqQLGo2i+a1XhDzBxrk=@vger.kernel.org X-Gm-Message-State: AOJu0Yz6Fzcgv6q2mu+BHZDQBL9mJ1cutpd2HL1H/w0uHNqInFY2U7WS YwNJxcRXvD+Nssa0mFKrUKKS7wqE2JiBKsjiZAXNzxSLiI40W8FHVXgh3hu9V+M= X-Google-Smtp-Source: AGHT+IFfCwyaClKnReIWxUZd0iEfNRrNS6oO6x2m/KaQtRbS25PxoEe6F/CuTCRBg5p3XO9EVv7f2A== X-Received: by 2002:a17:907:7d93:b0:a9a:8a4:e079 with SMTP id a640c23a62f3a-a9de5edea84mr4205271366b.31.1730915467354; Wed, 06 Nov 2024 09:51:07 -0800 (PST) Received: from localhost (2001-1ae9-1c2-4c00-20f-c6b4-1e57-7965.ip6.tmcz.cz. [2001:1ae9:1c2:4c00:20f:c6b4:1e57:7965]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a9eb17d7328sm312443566b.129.2024.11.06.09.51.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Nov 2024 09:51:06 -0800 (PST) From: Andrew Jones To: iommu@lists.linux.dev, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org Cc: tjeznach@rivosinc.com, joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, paul.walmsley@sifive.com, palmer@dabbelt.com, aou@eecs.berkeley.edu Subject: [PATCH 2/2] iommu/riscv: Add support for platform msi Date: Wed, 6 Nov 2024 18:51:05 +0100 Message-ID: <20241106175102.219923-6-ajones@ventanamicro.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: <20241106175102.219923-4-ajones@ventanamicro.com> References: <20241106175102.219923-4-ajones@ventanamicro.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Apply platform_device_msi_init_and_alloc_irqs() to add support for MSIs when the IOMMU is a platform device. Signed-off-by: Andrew Jones --- drivers/iommu/riscv/iommu-platform.c | 102 ++++++++++++++++++++++----- 1 file changed, 84 insertions(+), 18 deletions(-) diff --git a/drivers/iommu/riscv/iommu-platform.c b/drivers/iommu/riscv/iom= mu-platform.c index da336863f152..89aa622bcbde 100644 --- a/drivers/iommu/riscv/iommu-platform.c +++ b/drivers/iommu/riscv/iommu-platform.c @@ -11,18 +11,41 @@ */ =20 #include +#include #include #include +#include =20 #include "iommu-bits.h" #include "iommu.h" =20 +static void riscv_iommu_write_msi_msg(struct msi_desc *desc, struct msi_ms= g *msg) +{ + struct device *dev =3D msi_desc_to_dev(desc); + struct riscv_iommu_device *iommu =3D dev_get_drvdata(dev); + u16 idx =3D desc->msi_index; + u64 addr; + + addr =3D ((u64)msg->address_hi << 32) | msg->address_lo; + + if (addr !=3D (addr & RISCV_IOMMU_MSI_CFG_TBL_ADDR)) + pr_warn_once("uh oh, the IOMMU can't send MSIs to 0x%llx, sending to 0x%= llx instead\n", + addr, addr & RISCV_IOMMU_MSI_CFG_TBL_ADDR); + + addr &=3D RISCV_IOMMU_MSI_CFG_TBL_ADDR; + + riscv_iommu_writeq(iommu, RISCV_IOMMU_REG_MSI_CFG_TBL_ADDR(idx), addr); + riscv_iommu_writel(iommu, RISCV_IOMMU_REG_MSI_CFG_TBL_DATA(idx), msg->dat= a); + riscv_iommu_writel(iommu, RISCV_IOMMU_REG_MSI_CFG_TBL_CTRL(idx), 0); +} + static int riscv_iommu_platform_probe(struct platform_device *pdev) { + enum riscv_iommu_igs_settings igs; struct device *dev =3D &pdev->dev; struct riscv_iommu_device *iommu =3D NULL; struct resource *res =3D NULL; - int vec; + int vec, ret; =20 iommu =3D devm_kzalloc(dev, sizeof(*iommu), GFP_KERNEL); if (!iommu) @@ -40,16 +63,6 @@ static int riscv_iommu_platform_probe(struct platform_de= vice *pdev) iommu->caps =3D riscv_iommu_readq(iommu, RISCV_IOMMU_REG_CAPABILITIES); iommu->fctl =3D riscv_iommu_readl(iommu, RISCV_IOMMU_REG_FCTL); =20 - /* For now we only support WSI */ - switch (FIELD_GET(RISCV_IOMMU_CAPABILITIES_IGS, iommu->caps)) { - case RISCV_IOMMU_CAPABILITIES_IGS_WSI: - case RISCV_IOMMU_CAPABILITIES_IGS_BOTH: - break; - default: - return dev_err_probe(dev, -ENODEV, - "unable to use wire-signaled interrupts\n"); - } - iommu->irqs_count =3D platform_irq_count(pdev); if (iommu->irqs_count <=3D 0) return dev_err_probe(dev, -ENODEV, @@ -57,13 +70,60 @@ static int riscv_iommu_platform_probe(struct platform_d= evice *pdev) if (iommu->irqs_count > RISCV_IOMMU_INTR_COUNT) iommu->irqs_count =3D RISCV_IOMMU_INTR_COUNT; =20 - for (vec =3D 0; vec < iommu->irqs_count; vec++) - iommu->irqs[vec] =3D platform_get_irq(pdev, vec); + igs =3D FIELD_GET(RISCV_IOMMU_CAPABILITIES_IGS, iommu->caps); + switch (igs) { + case RISCV_IOMMU_CAPABILITIES_IGS_BOTH: + case RISCV_IOMMU_CAPABILITIES_IGS_MSI: + if (is_of_node(dev->fwnode)) + of_msi_configure(dev, to_of_node(dev->fwnode)); + + if (!dev_get_msi_domain(dev)) { + dev_warn(dev, "failed to find an MSI domain"); + goto msi_fail; + } + + ret =3D platform_device_msi_init_and_alloc_irqs(dev, iommu->irqs_count, + riscv_iommu_write_msi_msg); + if (ret) { + dev_warn(dev, "failed to allocate MSIs"); + goto msi_fail; + } + + for (vec =3D 0; vec < iommu->irqs_count; vec++) + iommu->irqs[vec] =3D msi_get_virq(dev, vec); + + /* Enable message-signaled interrupts, fctl.WSI */ + if (iommu->fctl & RISCV_IOMMU_FCTL_WSI) { + iommu->fctl ^=3D RISCV_IOMMU_FCTL_WSI; + riscv_iommu_writel(iommu, RISCV_IOMMU_REG_FCTL, iommu->fctl); + } + + dev_info(dev, "using MSIs\n"); + break; + +msi_fail: + if (igs !=3D RISCV_IOMMU_CAPABILITIES_IGS_BOTH) { + dev_warn(dev, "\n"); + return dev_err_probe(dev, -ENODEV, + "unable to use wire-signaled interrupts\n"); + } + + dev_warn(dev, " - falling back to wired irqs\n"); + fallthrough; =20 - /* Enable wire-signaled interrupts, fctl.WSI */ - if (!(iommu->fctl & RISCV_IOMMU_FCTL_WSI)) { - iommu->fctl |=3D RISCV_IOMMU_FCTL_WSI; - riscv_iommu_writel(iommu, RISCV_IOMMU_REG_FCTL, iommu->fctl); + case RISCV_IOMMU_CAPABILITIES_IGS_WSI: + for (vec =3D 0; vec < iommu->irqs_count; vec++) + iommu->irqs[vec] =3D platform_get_irq(pdev, vec); + + /* Enable wire-signaled interrupts, fctl.WSI */ + if (!(iommu->fctl & RISCV_IOMMU_FCTL_WSI)) { + iommu->fctl |=3D RISCV_IOMMU_FCTL_WSI; + riscv_iommu_writel(iommu, RISCV_IOMMU_REG_FCTL, iommu->fctl); + } + dev_info(dev, "using wire-signaled interrupts\n"); + break; + default: + return dev_err_probe(dev, -ENODEV, "invalid IGS\n"); } =20 return riscv_iommu_init(iommu); @@ -71,7 +131,13 @@ static int riscv_iommu_platform_probe(struct platform_d= evice *pdev) =20 static void riscv_iommu_platform_remove(struct platform_device *pdev) { - riscv_iommu_remove(dev_get_drvdata(&pdev->dev)); + struct riscv_iommu_device *iommu =3D dev_get_drvdata(&pdev->dev); + bool msi =3D !(iommu->fctl & RISCV_IOMMU_FCTL_WSI); + + riscv_iommu_remove(iommu); + + if (msi) + platform_device_msi_free_irqs_all(&pdev->dev); }; =20 static const struct of_device_id riscv_iommu_of_match[] =3D { --=20 2.47.0