From nobody Tue Apr 7 00:25:24 2026 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F530C433FE for ; Fri, 7 Oct 2022 09:50:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229459AbiJGJuj (ORCPT ); Fri, 7 Oct 2022 05:50:39 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46680 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229591AbiJGJu2 (ORCPT ); Fri, 7 Oct 2022 05:50:28 -0400 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6CBDD9D527; Fri, 7 Oct 2022 02:50:24 -0700 (PDT) Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 2979jcWx004633; Fri, 7 Oct 2022 09:50:14 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=k23RzXU4mgAv59rxzFKF7MDBNPemeo6Xe8BqpTM84Mc=; b=HB2zYL4Ma7+AJ9YPDGdfstwffeeu6ni5KEGkgMQQ43yUihvShfyC/hexrayBXtApc7gA 4K3aY/fib7F6MBdAcQF8eB9ZcMZb/qxi+BwBmqVqRgoMgf65lckRxnSnag8elgZoaok8 KgYZviyjyOvC1rr58tzymvDyVV82Y8b8HGIZOzPHzzGaF82NE3bCLPcwh9GGmGj/GXiw z1YoXLhoSY+BOByUGx9fxuQPgNA5xAl+iPG+TN0st7Jg6qWwyDnxblXKhRpe+9y3UH/6 chyh5Z8p6sZVPQmWmSH+b2oZeuLaIlDwT+2v3wrDe/NFi9ib3IcGqQfyVnroGbP8uccF Qw== Received: from ppma06fra.de.ibm.com (48.49.7a9f.ip4.static.sl-reverse.com [159.122.73.72]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3k2hs00451-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 07 Oct 2022 09:50:13 +0000 Received: from pps.filterd (ppma06fra.de.ibm.com [127.0.0.1]) by ppma06fra.de.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 2979ajwW014491; Fri, 7 Oct 2022 09:50:11 GMT Received: from b06cxnps4075.portsmouth.uk.ibm.com (d06relay12.portsmouth.uk.ibm.com [9.149.109.197]) by ppma06fra.de.ibm.com with ESMTP id 3jxcthx6d2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 07 Oct 2022 09:50:11 +0000 Received: from d06av23.portsmouth.uk.ibm.com (d06av23.portsmouth.uk.ibm.com [9.149.105.59]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 2979o7Vb43778364 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 7 Oct 2022 09:50:08 GMT Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CFDADA4040; Fri, 7 Oct 2022 09:50:07 +0000 (GMT) Received: from d06av23.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5FD9CA404D; Fri, 7 Oct 2022 09:50:07 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.152.85.9]) by d06av23.portsmouth.uk.ibm.com (Postfix) with ESMTP; Fri, 7 Oct 2022 09:50:07 +0000 (GMT) From: Niklas Schnelle To: Matthew Rosato , Pierre Morel , iommu@lists.linux.dev Cc: linux-s390@vger.kernel.org, borntraeger@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com, gerald.schaefer@linux.ibm.com, agordeev@linux.ibm.com, svens@linux.ibm.com, joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, jgg@nvidia.com, linux-kernel@vger.kernel.org Subject: [PATCH v6 3/6] iommu/s390: Fix potential s390_domain aperture shrinking Date: Fri, 7 Oct 2022 11:50:02 +0200 Message-Id: <20221007095005.2017126-4-schnelle@linux.ibm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221007095005.2017126-1-schnelle@linux.ibm.com> References: <20221007095005.2017126-1-schnelle@linux.ibm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: kYD_OPVCcLN_mYM_zzT21-zi6gKZUO9c X-Proofpoint-GUID: kYD_OPVCcLN_mYM_zzT21-zi6gKZUO9c X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.528,FMLib:17.11.122.1 definitions=2022-10-06_05,2022-10-06_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 spamscore=0 malwarescore=0 suspectscore=0 adultscore=0 impostorscore=0 lowpriorityscore=0 mlxlogscore=999 clxscore=1015 phishscore=0 mlxscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2209130000 definitions=main-2210070058 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Type: text/plain; charset="utf-8" The s390 IOMMU driver currently sets the IOMMU domain's aperture to match the device specific DMA address range of the device that is first attached. This is not ideal. For one if the domain has no device attached in the meantime the aperture could be shrunk allowing translations outside the aperture to exist in the translation tables. Also this is a bit of a misuse of the aperture which really should describe what addresses can be translated and not some device specific limitations. Instead of misusing the aperture like this we can instead create reserved ranges for the ranges inaccessible to the attached devices allowing devices with overlapping ranges to still share an IOMMU domain. This also significantly simplifies s390_iommu_attach_device() allowing us to move the aperture check to the beginning of the function and removing the need to hold the device list's lock to check the aperture. As we then use the same aperture for all domains and it only depends on the table properties we can already check zdev->start_dma/end_dma at probe time and turn the check on attach into a WARN_ON(). Suggested-by: Jason Gunthorpe Reviewed-by: Matthew Rosato Signed-off-by: Niklas Schnelle --- v5->v6: - Return -EINVAL after WARN_ON() in attach v4->v5: - Make aperture check in attach a WARN_ON() and fail in probe if zdev->start_dma/end_dma doesn't git in aperture (Jason) drivers/iommu/s390-iommu.c | 63 ++++++++++++++++++++++++++------------ 1 file changed, 43 insertions(+), 20 deletions(-) diff --git a/drivers/iommu/s390-iommu.c b/drivers/iommu/s390-iommu.c index 399c31b97f65..65835a5ca328 100644 --- a/drivers/iommu/s390-iommu.c +++ b/drivers/iommu/s390-iommu.c @@ -62,6 +62,9 @@ static struct iommu_domain *s390_domain_alloc(unsigned do= main_type) kfree(s390_domain); return NULL; } + s390_domain->domain.geometry.force_aperture =3D true; + s390_domain->domain.geometry.aperture_start =3D 0; + s390_domain->domain.geometry.aperture_end =3D ZPCI_TABLE_SIZE_RT - 1; =20 spin_lock_init(&s390_domain->dma_table_lock); spin_lock_init(&s390_domain->list_lock); @@ -102,11 +105,15 @@ static int s390_iommu_attach_device(struct iommu_doma= in *domain, struct s390_domain *s390_domain =3D to_s390_domain(domain); struct zpci_dev *zdev =3D to_zpci_dev(dev); unsigned long flags; - int cc, rc =3D 0; + int cc; =20 if (!zdev) return -ENODEV; =20 + if (WARN_ON(domain->geometry.aperture_start > zdev->end_dma || + domain->geometry.aperture_end < zdev->start_dma)) + return -EINVAL; + if (zdev->s390_domain) __s390_iommu_detach_device(zdev); else if (zdev->dma_table) @@ -118,30 +125,14 @@ static int s390_iommu_attach_device(struct iommu_doma= in *domain, return -EIO; zdev->dma_table =3D s390_domain->dma_table; =20 - spin_lock_irqsave(&s390_domain->list_lock, flags); - /* First device defines the DMA range limits */ - if (list_empty(&s390_domain->devices)) { - domain->geometry.aperture_start =3D zdev->start_dma; - domain->geometry.aperture_end =3D zdev->end_dma; - domain->geometry.force_aperture =3D true; - /* Allow only devices with identical DMA range limits */ - } else if (domain->geometry.aperture_start !=3D zdev->start_dma || - domain->geometry.aperture_end !=3D zdev->end_dma) { - spin_unlock_irqrestore(&s390_domain->list_lock, flags); - rc =3D -EINVAL; - goto out_unregister; - } + zdev->dma_table =3D s390_domain->dma_table; zdev->s390_domain =3D s390_domain; + + spin_lock_irqsave(&s390_domain->list_lock, flags); list_add(&zdev->iommu_list, &s390_domain->devices); spin_unlock_irqrestore(&s390_domain->list_lock, flags); =20 return 0; - -out_unregister: - zpci_unregister_ioat(zdev, 0); - zdev->dma_table =3D NULL; - - return rc; } =20 static void s390_iommu_detach_device(struct iommu_domain *domain, @@ -155,10 +146,41 @@ static void s390_iommu_detach_device(struct iommu_dom= ain *domain, zpci_dma_init_device(zdev); } =20 +static void s390_iommu_get_resv_regions(struct device *dev, + struct list_head *list) +{ + struct zpci_dev *zdev =3D to_zpci_dev(dev); + struct iommu_resv_region *region; + + if (zdev->start_dma) { + region =3D iommu_alloc_resv_region(0, zdev->start_dma, 0, + IOMMU_RESV_RESERVED); + if (!region) + return; + list_add_tail(®ion->list, list); + } + + if (zdev->end_dma < ZPCI_TABLE_SIZE_RT - 1) { + region =3D iommu_alloc_resv_region(zdev->end_dma + 1, + ZPCI_TABLE_SIZE_RT - zdev->end_dma - 1, + 0, IOMMU_RESV_RESERVED); + if (!region) + return; + list_add_tail(®ion->list, list); + } +} + static struct iommu_device *s390_iommu_probe_device(struct device *dev) { struct zpci_dev *zdev =3D to_zpci_dev(dev); =20 + if (zdev->start_dma > zdev->end_dma || + zdev->start_dma > ZPCI_TABLE_SIZE_RT - 1) + return ERR_PTR(-EINVAL); + + if (zdev->end_dma > ZPCI_TABLE_SIZE_RT - 1) + zdev->end_dma =3D ZPCI_TABLE_SIZE_RT - 1; + return &zdev->iommu_dev; } =20 @@ -337,6 +359,7 @@ static const struct iommu_ops s390_iommu_ops =3D { .release_device =3D s390_iommu_release_device, .device_group =3D generic_device_group, .pgsize_bitmap =3D S390_IOMMU_PGSIZES, + .get_resv_regions =3D s390_iommu_get_resv_regions, .default_domain_ops =3D &(const struct iommu_domain_ops) { .attach_dev =3D s390_iommu_attach_device, .detach_dev =3D s390_iommu_detach_device, --=20 2.34.1