From nobody Wed Dec 17 14:20:38 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DC7111EE7D3; Fri, 11 Apr 2025 20:24:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744403090; cv=none; b=O42z+ZcmQE/5KfpVFcJWqAAg1wmOo7X73VPruB6/ICvXmYdR4pGr1eBU5AbZh4morPQN6y+fNR17XZwRoHhrephoC8TFAVQRjqoex7v3kF5oOoGHqL7obnWtAcRptwtduhoO5tik9Ovc1ivz0rdD1+8y+EnS7/yA12+gUiqpAX4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744403090; c=relaxed/simple; bh=x7B9MnN6xpz37Iys3AovmgLR2oS5tnCU9+OgjSjXdhU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=mNR/XCnJO1yl9EJSSYfb3QhHUVf9f62M7KYvwTa4z7ey8f2VFQ59ygdeWG5wZlG7M+3uNzKfcIKl3dxQywEg7sC8aFhYUXGs8GAsGgGvH7Le2PFtAawzfv0DhntJKBAWuY8LDSfhOxPWPaNkXT4OPQIvEIULYjByviLXF4uurUA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=C8NodISx; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="C8NodISx" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 53BII4PL022916; Fri, 11 Apr 2025 20:24:38 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=PQyazkUkM7A2DT0v2 mq2AovAAjWaC5LljDabv03UP2U=; b=C8NodISx6u4e+I6R+ZWI7twpQ9dztzpge JYDutqcoRfbiyCNB2c9ew0OA7uewbsIFNIV5stA1OWUWXZrTV1gi428I7NwREL58 qPSs7FZzOyA1xmnXDG230frJdYpbRkBB3pTieKRl1znLHlqG9yGNSf5qSp44FSN7 +HpLVGzj9r5wdGo1eBArEiC80HZRE+kTloiTIlcrkvIRKi4OA0f8NdmzwBjkJT2Y 9XeA8/5quaRuXpNaj0sxRLvxPY+fVvYKp3vG0Ve4HPvuA82dQoUa0VQcgKolUKGq rtecNqESmkZ42v6JRQKgypeNLHuS92BJirf7p/SEaq9/k2wqm8+vQ== Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 45y78x0va2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 11 Apr 2025 20:24:38 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 53BHeZDp024577; Fri, 11 Apr 2025 20:24:37 GMT Received: from smtprelay03.wdc07v.mail.ibm.com ([172.16.1.70]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 45ueutvx66-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 11 Apr 2025 20:24:37 +0000 Received: from smtpav02.dal12v.mail.ibm.com (smtpav02.dal12v.mail.ibm.com [10.241.53.101]) by smtprelay03.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 53BKOXTo27919098 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 11 Apr 2025 20:24:34 GMT Received: from smtpav02.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EFD2C5805A; Fri, 11 Apr 2025 20:24:35 +0000 (GMT) Received: from smtpav02.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0147F58051; Fri, 11 Apr 2025 20:24:35 +0000 (GMT) Received: from li-2311da4c-2e09-11b2-a85c-c003041e9174.ibm.com.com (unknown [9.61.105.190]) by smtpav02.dal12v.mail.ibm.com (Postfix) with ESMTP; Fri, 11 Apr 2025 20:24:34 +0000 (GMT) From: Matthew Rosato To: joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, gerald.schaefer@linux.ibm.com, schnelle@linux.ibm.com Cc: hca@linux.ibm.com, gor@linux.ibm.com, agordeev@linux.ibm.com, svens@linux.ibm.com, borntraeger@linux.ibm.com, clg@redhat.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v5 1/5] iommu/s390: set appropriate IOTA region type Date: Fri, 11 Apr 2025 16:24:29 -0400 Message-ID: <20250411202433.181683-2-mjrosato@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250411202433.181683-1-mjrosato@linux.ibm.com> References: <20250411202433.181683-1-mjrosato@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: f2UgZU-FSd_IIaL76mjJL0Lzw0zL6M2B X-Proofpoint-ORIG-GUID: f2UgZU-FSd_IIaL76mjJL0Lzw0zL6M2B X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1095,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-04-11_08,2025-04-10_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 bulkscore=0 clxscore=1015 impostorscore=0 priorityscore=1501 adultscore=0 spamscore=0 mlxlogscore=999 suspectscore=0 lowpriorityscore=0 mlxscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2502280000 definitions=main-2504110127 Content-Type: text/plain; charset="utf-8" When registering the I/O Translation Anchor, use the current table type stored in the s390_domain to set the appropriate region type indication. For the moment, the table type will always be stored as region third. Reviewed-by: Niklas Schnelle Signed-off-by: Matthew Rosato Tested-by: Niklas Schnelle --- drivers/iommu/s390-iommu.c | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/s390-iommu.c b/drivers/iommu/s390-iommu.c index e1c76e0f9c2b..cad032d4c9a6 100644 --- a/drivers/iommu/s390-iommu.c +++ b/drivers/iommu/s390-iommu.c @@ -31,6 +31,7 @@ struct s390_domain { unsigned long *dma_table; spinlock_t list_lock; struct rcu_head rcu; + u8 origin_type; }; =20 static struct iommu_domain blocking_domain; @@ -345,6 +346,7 @@ static struct iommu_domain *s390_domain_alloc_paging(st= ruct device *dev) s390_domain->domain.geometry.force_aperture =3D true; s390_domain->domain.geometry.aperture_start =3D 0; s390_domain->domain.geometry.aperture_end =3D ZPCI_TABLE_SIZE_RT - 1; + s390_domain->origin_type =3D ZPCI_TABLE_TYPE_RTX; =20 spin_lock_init(&s390_domain->list_lock); INIT_LIST_HEAD_RCU(&s390_domain->devices); @@ -381,6 +383,21 @@ static void zdev_s390_domain_update(struct zpci_dev *z= dev, spin_unlock_irqrestore(&zdev->dom_lock, flags); } =20 +static u64 get_iota_region_flag(struct s390_domain *domain) +{ + switch (domain->origin_type) { + case ZPCI_TABLE_TYPE_RTX: + return ZPCI_IOTA_RTTO_FLAG; + case ZPCI_TABLE_TYPE_RSX: + return ZPCI_IOTA_RSTO_FLAG; + case ZPCI_TABLE_TYPE_RFX: + return ZPCI_IOTA_RFTO_FLAG; + default: + WARN_ONCE(1, "Invalid IOMMU table (%x)\n", domain->origin_type); + return 0; + } +} + static int s390_iommu_domain_reg_ioat(struct zpci_dev *zdev, struct iommu_domain *domain, u8 *status) { @@ -399,7 +416,7 @@ static int s390_iommu_domain_reg_ioat(struct zpci_dev *= zdev, default: s390_domain =3D to_s390_domain(domain); iota =3D virt_to_phys(s390_domain->dma_table) | - ZPCI_IOTA_RTTO_FLAG; + get_iota_region_flag(s390_domain); rc =3D zpci_register_ioat(zdev, 0, zdev->start_dma, zdev->end_dma, iota, status); } --=20 2.49.0 From nobody Wed Dec 17 14:20:38 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5F1CA28CF4A; Fri, 11 Apr 2025 20:24:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744403097; cv=none; b=sWi0ysdBsA198+XUTzlcSFkaKfemlMkahfLcwfz1i2olYd7p/U2yS0UBUdGjwV6WI1V02TlvJMBKJdPVgtVPkuveSCIaBWUzHeSEccMf0+SeubP6e0/GtZiiJfn11PNO4bnoGOQerVgcFEJCf8ZVcOzGMJpnyDxlUVJXQVOBhE0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744403097; c=relaxed/simple; bh=2qKsH5kv0ncdsyHod9k9bno1V2Ucw112xluXhuZPimk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ci9HSbxXSTsnJGuihAgdJtJdQldlC7xX6fpJCd9fJBK8bcxsBVBS5GkHkFk5d1tTDbx4wlpCfgrJGzVi2sflP/vZhTsJCR3cgzwkVs5O1rFRLAqp5eyn6Ebf7oGeM+8lA8rZq33u7C5pOXKesZnRJWfSFz/h8kQQn3A2lgEoG10= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=DqPunv95; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="DqPunv95" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 53BIIP15023248; Fri, 11 Apr 2025 20:24:40 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=c39HqofJWxJJ4EWgu mfYfcuMArfFUHDA2BWbZqfDm44=; b=DqPunv95vZ3MStf2SfiShgbwNasmMZ2lZ T4kNCjobDwmxJ/qxFXgc2Ud4pD89wdUZVMavaso9SjGnw3I/abzHf5u9hnGonc9U qBRUFGoO5B0NNcTim/+IYNMnfqFb4c1HUJJH8bUpfhtrb8Djc1K0MrLELdixczbV Rggk4M/bc4ECxXJxI1J/z1CtRbtZb7iz3hyzR7jf2vMyWy77FIhJmGQxfX4D2E0s OL6x0l1dsprcrFZiENUctjM+4WK54zHEZoZsk99JuS7/nXzTHACx0OLDVddJ0O1m 7fOFU6sqbYRtmWhVDMINmsKEPZnu4pZs+LEdEq2rLUt3qBQfetkaQ== Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 45y343tb6m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 11 Apr 2025 20:24:39 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 53BJsN3l029592; Fri, 11 Apr 2025 20:24:39 GMT Received: from smtprelay04.wdc07v.mail.ibm.com ([172.16.1.71]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 45x1k7ajsp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 11 Apr 2025 20:24:39 +0000 Received: from smtpav02.dal12v.mail.ibm.com (smtpav02.dal12v.mail.ibm.com [10.241.53.101]) by smtprelay04.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 53BKOb4p63242600 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 11 Apr 2025 20:24:37 GMT Received: from smtpav02.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0E79B58051; Fri, 11 Apr 2025 20:24:37 +0000 (GMT) Received: from smtpav02.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 174A15805C; Fri, 11 Apr 2025 20:24:36 +0000 (GMT) Received: from li-2311da4c-2e09-11b2-a85c-c003041e9174.ibm.com.com (unknown [9.61.105.190]) by smtpav02.dal12v.mail.ibm.com (Postfix) with ESMTP; Fri, 11 Apr 2025 20:24:36 +0000 (GMT) From: Matthew Rosato To: joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, gerald.schaefer@linux.ibm.com, schnelle@linux.ibm.com Cc: hca@linux.ibm.com, gor@linux.ibm.com, agordeev@linux.ibm.com, svens@linux.ibm.com, borntraeger@linux.ibm.com, clg@redhat.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v5 2/5] iommu/s390: support cleanup of additional table regions Date: Fri, 11 Apr 2025 16:24:30 -0400 Message-ID: <20250411202433.181683-3-mjrosato@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250411202433.181683-1-mjrosato@linux.ibm.com> References: <20250411202433.181683-1-mjrosato@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: QpkBOdDKAG2umgOUiB-xaSBCz0M5atXL X-Proofpoint-ORIG-GUID: QpkBOdDKAG2umgOUiB-xaSBCz0M5atXL X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1095,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-04-11_08,2025-04-10_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 priorityscore=1501 adultscore=0 malwarescore=0 impostorscore=0 lowpriorityscore=0 spamscore=0 mlxscore=0 suspectscore=0 mlxlogscore=970 clxscore=1015 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2502280000 definitions=main-2504110127 Content-Type: text/plain; charset="utf-8" Extend the existing dma_cleanup_tables to also handle region second and region first tables. Reviewed-by: Niklas Schnelle Signed-off-by: Matthew Rosato Tested-by: Niklas Schnelle --- drivers/iommu/s390-iommu.c | 71 ++++++++++++++++++++++++++++++++++---- 1 file changed, 64 insertions(+), 7 deletions(-) diff --git a/drivers/iommu/s390-iommu.c b/drivers/iommu/s390-iommu.c index cad032d4c9a6..f2cda0ce0fe9 100644 --- a/drivers/iommu/s390-iommu.c +++ b/drivers/iommu/s390-iommu.c @@ -121,6 +121,22 @@ static inline int pt_entry_isvalid(unsigned long entry) return (entry & ZPCI_PTE_VALID_MASK) =3D=3D ZPCI_PTE_VALID; } =20 +static inline unsigned long *get_rf_rso(unsigned long entry) +{ + if ((entry & ZPCI_TABLE_TYPE_MASK) =3D=3D ZPCI_TABLE_TYPE_RFX) + return phys_to_virt(entry & ZPCI_RTE_ADDR_MASK); + else + return NULL; +} + +static inline unsigned long *get_rs_rto(unsigned long entry) +{ + if ((entry & ZPCI_TABLE_TYPE_MASK) =3D=3D ZPCI_TABLE_TYPE_RSX) + return phys_to_virt(entry & ZPCI_RTE_ADDR_MASK); + else + return NULL; +} + static inline unsigned long *get_rt_sto(unsigned long entry) { if ((entry & ZPCI_TABLE_TYPE_MASK) =3D=3D ZPCI_TABLE_TYPE_RTX) @@ -192,18 +208,59 @@ static void dma_free_seg_table(unsigned long entry) dma_free_cpu_table(sto); } =20 -static void dma_cleanup_tables(unsigned long *table) +static void dma_free_rt_table(unsigned long entry) { + unsigned long *rto =3D get_rs_rto(entry); int rtx; =20 - if (!table) + for (rtx =3D 0; rtx < ZPCI_TABLE_ENTRIES; rtx++) + if (reg_entry_isvalid(rto[rtx])) + dma_free_seg_table(rto[rtx]); + + dma_free_cpu_table(rto); +} + +static void dma_free_rs_table(unsigned long entry) +{ + unsigned long *rso =3D get_rf_rso(entry); + int rsx; + + for (rsx =3D 0; rsx < ZPCI_TABLE_ENTRIES; rsx++) + if (reg_entry_isvalid(rso[rsx])) + dma_free_rt_table(rso[rsx]); + + dma_free_cpu_table(rso); +} + +static void dma_cleanup_tables(struct s390_domain *domain) +{ + int rtx, rsx, rfx; + + if (!domain->dma_table) return; =20 - for (rtx =3D 0; rtx < ZPCI_TABLE_ENTRIES; rtx++) - if (reg_entry_isvalid(table[rtx])) - dma_free_seg_table(table[rtx]); + switch (domain->origin_type) { + case ZPCI_TABLE_TYPE_RFX: + for (rfx =3D 0; rfx < ZPCI_TABLE_ENTRIES; rfx++) + if (reg_entry_isvalid(domain->dma_table[rfx])) + dma_free_rs_table(domain->dma_table[rfx]); + break; + case ZPCI_TABLE_TYPE_RSX: + for (rsx =3D 0; rsx < ZPCI_TABLE_ENTRIES; rsx++) + if (reg_entry_isvalid(domain->dma_table[rsx])) + dma_free_rt_table(domain->dma_table[rsx]); + break; + case ZPCI_TABLE_TYPE_RTX: + for (rtx =3D 0; rtx < ZPCI_TABLE_ENTRIES; rtx++) + if (reg_entry_isvalid(domain->dma_table[rtx])) + dma_free_seg_table(domain->dma_table[rtx]); + break; + default: + WARN_ONCE(1, "Invalid IOMMU table (%x)\n", domain->origin_type); + return; + } =20 - dma_free_cpu_table(table); + dma_free_cpu_table(domain->dma_table); } =20 static unsigned long *dma_alloc_page_table(gfp_t gfp) @@ -358,7 +415,7 @@ static void s390_iommu_rcu_free_domain(struct rcu_head = *head) { struct s390_domain *s390_domain =3D container_of(head, struct s390_domain= , rcu); =20 - dma_cleanup_tables(s390_domain->dma_table); + dma_cleanup_tables(s390_domain); kfree(s390_domain); } =20 --=20 2.49.0 From nobody Wed Dec 17 14:20:38 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 15C6A27E1B0; Fri, 11 Apr 2025 20:24:50 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744403092; cv=none; b=VZusLKT/kIxAvmbe41c3R/S1zPjl9vmPrjInLbWAhx1kUlMliwzMhTiJO7clfsSUoP2hqbDQQYlV6teNwhfJ8Ls1XKE+IkD3QzpQ3KJrqT1ZxpELH/yuCNObUjC/XNFvHdZXD2Slm3FyHOSvFeh7wuhspapf0OD1Fq1olPzTqv8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744403092; c=relaxed/simple; bh=ZZnyG3zya5Po2F1xc2XCG4pomT8ljt2l/zbVN/JYOUA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=gV4aItGfPUDM6wAWYr9276s2l0QCoE6molGVPK8CY+Dqj9HxHJsXhdeHNvTQaasNjM3E5Ydj9DEnsEgF0PLCBpV2TT4o9mh7jZeBjwovges+eh2R7XGCqiapAI5fqy0EOUv7wipOYMaJFHAlsA9lL3IpBqpCZGihj+6C5O5ADAc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=W4XBu6YQ; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="W4XBu6YQ" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 53BII36X022894; Fri, 11 Apr 2025 20:24:41 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=BW2kWj1vzqNrV9mFj OjgpQ3BIDfQoxjiOkdFVgVgRuw=; b=W4XBu6YQ4gVo/VlOJTg1BZGml514lKZ4r zecgCxfg9iugBEGy7xF/EjcY5StqZEDgvMdC8BLvek0D/XhQVTWJuMTcFpV/WXxM KAh8IJwymqMTDikO8dcwqzoNUpZNiYVBvyg7FZ2GdiXNVFp9oTR7/4yZ+gn87wcQ J4h2rEAh5Lm4E01pxO1667DBKMplAqFKchd3VGFxy3VixjEO1bU3oI8lBQFHISC2 azQhRSCSFzke3VlikN8IDKe2zjD1x5Os66aJUlvibiOYyCo0Yqjz84LaWi7v6ZC2 Ky5ZQ+AylYg9dqjwN+I0wYbp/OLpoA6sf/vIWr7kdYNBvBbH/CHVw== Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 45y78x0va3-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 11 Apr 2025 20:24:40 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 53BIw4tO017825; Fri, 11 Apr 2025 20:24:39 GMT Received: from smtprelay05.wdc07v.mail.ibm.com ([172.16.1.72]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 45uh2m4hed-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 11 Apr 2025 20:24:39 +0000 Received: from smtpav02.dal12v.mail.ibm.com (smtpav02.dal12v.mail.ibm.com [10.241.53.101]) by smtprelay05.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 53BKOcaK64094584 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 11 Apr 2025 20:24:38 GMT Received: from smtpav02.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 318C35805E; Fri, 11 Apr 2025 20:24:38 +0000 (GMT) Received: from smtpav02.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2A59A5805C; Fri, 11 Apr 2025 20:24:37 +0000 (GMT) Received: from li-2311da4c-2e09-11b2-a85c-c003041e9174.ibm.com.com (unknown [9.61.105.190]) by smtpav02.dal12v.mail.ibm.com (Postfix) with ESMTP; Fri, 11 Apr 2025 20:24:37 +0000 (GMT) From: Matthew Rosato To: joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, gerald.schaefer@linux.ibm.com, schnelle@linux.ibm.com Cc: hca@linux.ibm.com, gor@linux.ibm.com, agordeev@linux.ibm.com, svens@linux.ibm.com, borntraeger@linux.ibm.com, clg@redhat.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v5 3/5] iommu/s390: support iova_to_phys for additional table regions Date: Fri, 11 Apr 2025 16:24:31 -0400 Message-ID: <20250411202433.181683-4-mjrosato@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250411202433.181683-1-mjrosato@linux.ibm.com> References: <20250411202433.181683-1-mjrosato@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: yS7iBEvGiNM2UF995yFmR50NIbNDUho2 X-Proofpoint-ORIG-GUID: yS7iBEvGiNM2UF995yFmR50NIbNDUho2 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1095,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-04-11_08,2025-04-10_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 bulkscore=0 clxscore=1015 impostorscore=0 priorityscore=1501 adultscore=0 spamscore=0 mlxlogscore=862 suspectscore=0 lowpriorityscore=0 mlxscore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2502280000 definitions=main-2504110127 Content-Type: text/plain; charset="utf-8" The origin_type of the dma_table is used to determine how many table levels must be traversed for the translation. Reviewed-by: Niklas Schnelle Signed-off-by: Matthew Rosato Tested-by: Niklas Schnelle --- arch/s390/include/asm/pci_dma.h | 2 ++ drivers/iommu/s390-iommu.c | 60 ++++++++++++++++++++++++++++++++- 2 files changed, 61 insertions(+), 1 deletion(-) diff --git a/arch/s390/include/asm/pci_dma.h b/arch/s390/include/asm/pci_dm= a.h index 42d7cc4262ca..8d8962e4fd58 100644 --- a/arch/s390/include/asm/pci_dma.h +++ b/arch/s390/include/asm/pci_dma.h @@ -55,6 +55,8 @@ enum zpci_ioat_dtype { #define ZPCI_PT_BITS 8 #define ZPCI_ST_SHIFT (ZPCI_PT_BITS + PAGE_SHIFT) #define ZPCI_RT_SHIFT (ZPCI_ST_SHIFT + ZPCI_TABLE_BITS) +#define ZPCI_RS_SHIFT (ZPCI_RT_SHIFT + ZPCI_TABLE_BITS) +#define ZPCI_RF_SHIFT (ZPCI_RS_SHIFT + ZPCI_TABLE_BITS) =20 #define ZPCI_RTE_FLAG_MASK 0x3fffUL #define ZPCI_RTE_ADDR_MASK (~ZPCI_RTE_FLAG_MASK) diff --git a/drivers/iommu/s390-iommu.c b/drivers/iommu/s390-iommu.c index f2cda0ce0fe9..338a7381e918 100644 --- a/drivers/iommu/s390-iommu.c +++ b/drivers/iommu/s390-iommu.c @@ -36,6 +36,16 @@ struct s390_domain { =20 static struct iommu_domain blocking_domain; =20 +static inline unsigned int calc_rfx(dma_addr_t ptr) +{ + return ((unsigned long)ptr >> ZPCI_RF_SHIFT) & ZPCI_INDEX_MASK; +} + +static inline unsigned int calc_rsx(dma_addr_t ptr) +{ + return ((unsigned long)ptr >> ZPCI_RS_SHIFT) & ZPCI_INDEX_MASK; +} + static inline unsigned int calc_rtx(dma_addr_t ptr) { return ((unsigned long)ptr >> ZPCI_RT_SHIFT) & ZPCI_INDEX_MASK; @@ -759,6 +769,51 @@ static int s390_iommu_map_pages(struct iommu_domain *d= omain, return rc; } =20 +static unsigned long *get_rso_from_iova(struct s390_domain *domain, + dma_addr_t iova) +{ + unsigned long *rfo; + unsigned long rfe; + unsigned int rfx; + + switch (domain->origin_type) { + case ZPCI_TABLE_TYPE_RFX: + rfo =3D domain->dma_table; + rfx =3D calc_rfx(iova); + rfe =3D READ_ONCE(rfo[rfx]); + if (!reg_entry_isvalid(rfe)) + return NULL; + return get_rf_rso(rfe); + case ZPCI_TABLE_TYPE_RSX: + return domain->dma_table; + default: + return NULL; + } +} + +static unsigned long *get_rto_from_iova(struct s390_domain *domain, + dma_addr_t iova) +{ + unsigned long *rso; + unsigned long rse; + unsigned int rsx; + + switch (domain->origin_type) { + case ZPCI_TABLE_TYPE_RFX: + case ZPCI_TABLE_TYPE_RSX: + rso =3D get_rso_from_iova(domain, iova); + rsx =3D calc_rsx(iova); + rse =3D READ_ONCE(rso[rsx]); + if (!reg_entry_isvalid(rse)) + return NULL; + return get_rs_rto(rse); + case ZPCI_TABLE_TYPE_RTX: + return domain->dma_table; + default: + return NULL; + } +} + static phys_addr_t s390_iommu_iova_to_phys(struct iommu_domain *domain, dma_addr_t iova) { @@ -772,10 +827,13 @@ static phys_addr_t s390_iommu_iova_to_phys(struct iom= mu_domain *domain, iova > domain->geometry.aperture_end) return 0; =20 + rto =3D get_rto_from_iova(s390_domain, iova); + if (!rto) + return 0; + rtx =3D calc_rtx(iova); sx =3D calc_sx(iova); px =3D calc_px(iova); - rto =3D s390_domain->dma_table; =20 rte =3D READ_ONCE(rto[rtx]); if (reg_entry_isvalid(rte)) { --=20 2.49.0 From nobody Wed Dec 17 14:20:38 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4F9D12900BA; Fri, 11 Apr 2025 20:24:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744403099; cv=none; b=knkTopsO5ZlDlDNcZhH9aXaPnegGlZotbtIe04mqrF2rR641xZrd/ub05nYS/SOpknzfC7YB/KcOWm+IZKHqOr21KFQgtkO+lxTkW/q23sR+AEtAEi1QhntGViwxaJPT5wm07etUFC4r3WPs9/tuR4P9pK2QiZltsiY9/2lJOss= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744403099; c=relaxed/simple; bh=78FVZaz/ifJxU5ZlGxgMG855kHor4oDTQVhQw0ARKm0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=LzRnY1CYes4RvwgpNhAsa7B+MX79+1jFcvqOI9jdXpAeTXmsGKRLSIm7Hz2w1RQkpvgEYQ41QmW8cWUurk1DpYua3Vi4jJ0Gb2YY+wChS3ow7FDb7OY0YRbRfAE1jG88zInxq95T64KSWKmECZk2Is/pGmsS4nXOX0TtA/+Nd+M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=Lf3Cj2AM; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="Lf3Cj2AM" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 53BII4Wp015152; Fri, 11 Apr 2025 20:24:42 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=0s2nYaP83mLmgojgx Ynn4A70gTj6tK12R3u0RycbEto=; b=Lf3Cj2AMJcGjbPc2K1HL7Qn51LKLKsyqa HB0dT1VYrxAQUOYkf012nzJs/1x+wXWgmNrd0M3prkCHpdDzIGU5yzY2HrdoK1l7 fEpD76jVZtYs6msdvAS3BFHLkh8dbf9lBC7snYPiehkPepyUdielBjKdHCGu3JGM cfyRE3/HvtTAb6M4CDVh95EQZkMngh7OH+HgjhbvqxLKVy5GYtOQBnOhgsLMgphQ 6UIHCPV4vE8sEld/SQ7aK5CwckCcRcbqUwbpPIhpSDqJI5WorurNxwgOM3gKK3qa oB4thECQp4luQEIrbKwOIo0ebDrssf0czAylncV2Prg0zLRpjC/Ag== Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 45y4gqhu2n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 11 Apr 2025 20:24:41 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 53BIZb2N014473; Fri, 11 Apr 2025 20:24:40 GMT Received: from smtprelay06.wdc07v.mail.ibm.com ([172.16.1.73]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 45ufup4p61-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 11 Apr 2025 20:24:40 +0000 Received: from smtpav02.dal12v.mail.ibm.com (smtpav02.dal12v.mail.ibm.com [10.241.53.101]) by smtprelay06.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 53BKOdxI25100912 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 11 Apr 2025 20:24:39 GMT Received: from smtpav02.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 44C615805C; Fri, 11 Apr 2025 20:24:39 +0000 (GMT) Received: from smtpav02.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4A9EA5805A; Fri, 11 Apr 2025 20:24:38 +0000 (GMT) Received: from li-2311da4c-2e09-11b2-a85c-c003041e9174.ibm.com.com (unknown [9.61.105.190]) by smtpav02.dal12v.mail.ibm.com (Postfix) with ESMTP; Fri, 11 Apr 2025 20:24:38 +0000 (GMT) From: Matthew Rosato To: joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, gerald.schaefer@linux.ibm.com, schnelle@linux.ibm.com Cc: hca@linux.ibm.com, gor@linux.ibm.com, agordeev@linux.ibm.com, svens@linux.ibm.com, borntraeger@linux.ibm.com, clg@redhat.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v5 4/5] iommu/s390: support map/unmap for additional table regions Date: Fri, 11 Apr 2025 16:24:32 -0400 Message-ID: <20250411202433.181683-5-mjrosato@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250411202433.181683-1-mjrosato@linux.ibm.com> References: <20250411202433.181683-1-mjrosato@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: fdWa-tmXObFFuDPuLkpwUrcWHG4kJ-bg X-Proofpoint-ORIG-GUID: fdWa-tmXObFFuDPuLkpwUrcWHG4kJ-bg X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1095,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-04-11_08,2025-04-10_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 mlxlogscore=999 bulkscore=0 adultscore=0 priorityscore=1501 spamscore=0 impostorscore=0 clxscore=1015 suspectscore=0 mlxscore=0 phishscore=0 malwarescore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2502280000 definitions=main-2504110127 Content-Type: text/plain; charset="utf-8" Map and unmap ops use the shared dma_walk_cpu_trans routine, update this using the origin_type of the dma_table to determine how many table levels must be walked. Signed-off-by: Matthew Rosato Reviewed-by: Niklas Schnelle Tested-by: Niklas Schnelle --- drivers/iommu/s390-iommu.c | 127 ++++++++++++++++++++++++++++++++++--- 1 file changed, 119 insertions(+), 8 deletions(-) diff --git a/drivers/iommu/s390-iommu.c b/drivers/iommu/s390-iommu.c index 338a7381e918..46f45b136993 100644 --- a/drivers/iommu/s390-iommu.c +++ b/drivers/iommu/s390-iommu.c @@ -67,6 +67,20 @@ static inline void set_pt_pfaa(unsigned long *entry, phy= s_addr_t pfaa) *entry |=3D (pfaa & ZPCI_PTE_ADDR_MASK); } =20 +static inline void set_rf_rso(unsigned long *entry, phys_addr_t rso) +{ + *entry &=3D ZPCI_RTE_FLAG_MASK; + *entry |=3D (rso & ZPCI_RTE_ADDR_MASK); + *entry |=3D ZPCI_TABLE_TYPE_RFX; +} + +static inline void set_rs_rto(unsigned long *entry, phys_addr_t rto) +{ + *entry &=3D ZPCI_RTE_FLAG_MASK; + *entry |=3D (rto & ZPCI_RTE_ADDR_MASK); + *entry |=3D ZPCI_TABLE_TYPE_RSX; +} + static inline void set_rt_sto(unsigned long *entry, phys_addr_t sto) { *entry &=3D ZPCI_RTE_FLAG_MASK; @@ -81,6 +95,22 @@ static inline void set_st_pto(unsigned long *entry, phys= _addr_t pto) *entry |=3D ZPCI_TABLE_TYPE_SX; } =20 +static inline void validate_rf_entry(unsigned long *entry) +{ + *entry &=3D ~ZPCI_TABLE_VALID_MASK; + *entry &=3D ~ZPCI_TABLE_OFFSET_MASK; + *entry |=3D ZPCI_TABLE_VALID; + *entry |=3D ZPCI_TABLE_LEN_RFX; +} + +static inline void validate_rs_entry(unsigned long *entry) +{ + *entry &=3D ~ZPCI_TABLE_VALID_MASK; + *entry &=3D ~ZPCI_TABLE_OFFSET_MASK; + *entry |=3D ZPCI_TABLE_VALID; + *entry |=3D ZPCI_TABLE_LEN_RSX; +} + static inline void validate_rt_entry(unsigned long *entry) { *entry &=3D ~ZPCI_TABLE_VALID_MASK; @@ -286,6 +316,70 @@ static unsigned long *dma_alloc_page_table(gfp_t gfp) return table; } =20 +static unsigned long *dma_walk_rs_table(unsigned long *rso, + dma_addr_t dma_addr, gfp_t gfp) +{ + unsigned int rsx =3D calc_rsx(dma_addr); + unsigned long old_rse, rse; + unsigned long *rsep, *rto; + + rsep =3D &rso[rsx]; + rse =3D READ_ONCE(*rsep); + if (reg_entry_isvalid(rse)) { + rto =3D get_rs_rto(rse); + } else { + rto =3D dma_alloc_cpu_table(gfp); + if (!rto) + return NULL; + + set_rs_rto(&rse, virt_to_phys(rto)); + validate_rs_entry(&rse); + entry_clr_protected(&rse); + + old_rse =3D cmpxchg(rsep, ZPCI_TABLE_INVALID, rse); + if (old_rse !=3D ZPCI_TABLE_INVALID) { + /* Somone else was faster, use theirs */ + dma_free_cpu_table(rto); + rto =3D get_rs_rto(old_rse); + } + } + return rto; +} + +static unsigned long *dma_walk_rf_table(unsigned long *rfo, + dma_addr_t dma_addr, gfp_t gfp) +{ + unsigned int rfx =3D calc_rfx(dma_addr); + unsigned long old_rfe, rfe; + unsigned long *rfep, *rso; + + rfep =3D &rfo[rfx]; + rfe =3D READ_ONCE(*rfep); + if (reg_entry_isvalid(rfe)) { + rso =3D get_rf_rso(rfe); + } else { + rso =3D dma_alloc_cpu_table(gfp); + if (!rso) + return NULL; + + set_rf_rso(&rfe, virt_to_phys(rso)); + validate_rf_entry(&rfe); + entry_clr_protected(&rfe); + + old_rfe =3D cmpxchg(rfep, ZPCI_TABLE_INVALID, rfe); + if (old_rfe !=3D ZPCI_TABLE_INVALID) { + /* Somone else was faster, use theirs */ + dma_free_cpu_table(rso); + rso =3D get_rf_rso(old_rfe); + } + } + + if (!rso) + return NULL; + + return dma_walk_rs_table(rso, dma_addr, gfp); +} + static unsigned long *dma_get_seg_table_origin(unsigned long *rtep, gfp_t = gfp) { unsigned long old_rte, rte; @@ -339,11 +433,31 @@ static unsigned long *dma_get_page_table_origin(unsig= ned long *step, gfp_t gfp) return pto; } =20 -static unsigned long *dma_walk_cpu_trans(unsigned long *rto, dma_addr_t dm= a_addr, gfp_t gfp) +static unsigned long *dma_walk_region_tables(struct s390_domain *domain, + dma_addr_t dma_addr, gfp_t gfp) { - unsigned long *sto, *pto; + switch (domain->origin_type) { + case ZPCI_TABLE_TYPE_RFX: + return dma_walk_rf_table(domain->dma_table, dma_addr, gfp); + case ZPCI_TABLE_TYPE_RSX: + return dma_walk_rs_table(domain->dma_table, dma_addr, gfp); + case ZPCI_TABLE_TYPE_RTX: + return domain->dma_table; + default: + return NULL; + } +} + +static unsigned long *dma_walk_cpu_trans(struct s390_domain *domain, + dma_addr_t dma_addr, gfp_t gfp) +{ + unsigned long *rto, *sto, *pto; unsigned int rtx, sx, px; =20 + rto =3D dma_walk_region_tables(domain, dma_addr, gfp); + if (!rto) + return NULL; + rtx =3D calc_rtx(dma_addr); sto =3D dma_get_seg_table_origin(&rto[rtx], gfp); if (!sto) @@ -690,8 +804,7 @@ static int s390_iommu_validate_trans(struct s390_domain= *s390_domain, int rc; =20 for (i =3D 0; i < nr_pages; i++) { - entry =3D dma_walk_cpu_trans(s390_domain->dma_table, dma_addr, - gfp); + entry =3D dma_walk_cpu_trans(s390_domain, dma_addr, gfp); if (unlikely(!entry)) { rc =3D -ENOMEM; goto undo_cpu_trans; @@ -706,8 +819,7 @@ static int s390_iommu_validate_trans(struct s390_domain= *s390_domain, undo_cpu_trans: while (i-- > 0) { dma_addr -=3D PAGE_SIZE; - entry =3D dma_walk_cpu_trans(s390_domain->dma_table, - dma_addr, gfp); + entry =3D dma_walk_cpu_trans(s390_domain, dma_addr, gfp); if (!entry) break; dma_update_cpu_trans(entry, 0, ZPCI_PTE_INVALID); @@ -724,8 +836,7 @@ static int s390_iommu_invalidate_trans(struct s390_doma= in *s390_domain, int rc =3D 0; =20 for (i =3D 0; i < nr_pages; i++) { - entry =3D dma_walk_cpu_trans(s390_domain->dma_table, dma_addr, - GFP_ATOMIC); + entry =3D dma_walk_cpu_trans(s390_domain, dma_addr, GFP_ATOMIC); if (unlikely(!entry)) { rc =3D -EINVAL; break; --=20 2.49.0 From nobody Wed Dec 17 14:20:38 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C2C7D28C5C6; Fri, 11 Apr 2025 20:24:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744403095; cv=none; b=PsYgOVHaLNokqMYgAJb6x4fJs+Si6wOh3um0BpEqC3V5YZpeSLkG9A5gavSn1BDmath8gcqxLZC19DmijS5lAGSui7yKjbqQqw0PXZpnQmagS5kx75sSe4KtQ9M9HFeA59T+xuVDUID1W4pFlh1tLVz6/LliyI5l2Kc9bLR17b8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1744403095; c=relaxed/simple; bh=OZlu8f89BYxknWQX/kA3bt6HxJ89EN0pJVICj/4m1H0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eIrXWnvbDvHWIGu6qkt6dexdA9GTK/birgCjhX5c3PlV+FrobhLPm3jWg+OM5JigTCMYOMx0GHgeELjA2jmHDqXYxAzMhpCGAZs+X7jEoZvq9ggfIDUgFezBNW0B4BJvFgm1ST9g3VayQVwupLmupYOlupqb6sEHstUFEq21XMU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=hQ2AiEVp; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="hQ2AiEVp" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 53BII4DN022316; Fri, 11 Apr 2025 20:24:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=Qov1yGhcG5wRklZOK SeRMBsnOSzne165QTDxfJ9nals=; b=hQ2AiEVpr2jp+6FslHYjsROpdPph/Ha92 TWPLvNgcd6RReSf86ZW5Bj9SMuG5N+BoGiPOx0GStggtGRoWS5C9LFOQ0inah7BC 3hk+MwdnMdujNTq5bKgPhkGcfGJGkkSQbuluQZA+OJj7MZsV8IkJrXyRnor1Gdh8 TBitV52Gzg8sV8jpfwGi85xozkVvDcehymd/tSsG7ciYjyNw4PCOXWKZwlnNLSFR YAiK1RaAIx8mCaUMqPzCr3i/aQ3QlmcIPX5qceKJCKZQ14Tz8zXebXXU5pAfh4Zl P5QatcWyIfNUR2X0LfzjWfWYt+wCeiRvTI4KWtOLQxc2crwPOv4DA== Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 45y343tb6p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 11 Apr 2025 20:24:43 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 53BHcN8J011078; Fri, 11 Apr 2025 20:24:42 GMT Received: from smtprelay07.wdc07v.mail.ibm.com ([172.16.1.74]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 45uf804ux8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 11 Apr 2025 20:24:41 +0000 Received: from smtpav02.dal12v.mail.ibm.com (smtpav02.dal12v.mail.ibm.com [10.241.53.101]) by smtprelay07.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 53BKOeLf28639926 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 11 Apr 2025 20:24:40 GMT Received: from smtpav02.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5F04F58051; Fri, 11 Apr 2025 20:24:40 +0000 (GMT) Received: from smtpav02.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 62C815805A; Fri, 11 Apr 2025 20:24:39 +0000 (GMT) Received: from li-2311da4c-2e09-11b2-a85c-c003041e9174.ibm.com.com (unknown [9.61.105.190]) by smtpav02.dal12v.mail.ibm.com (Postfix) with ESMTP; Fri, 11 Apr 2025 20:24:39 +0000 (GMT) From: Matthew Rosato To: joro@8bytes.org, will@kernel.org, robin.murphy@arm.com, gerald.schaefer@linux.ibm.com, schnelle@linux.ibm.com Cc: hca@linux.ibm.com, gor@linux.ibm.com, agordeev@linux.ibm.com, svens@linux.ibm.com, borntraeger@linux.ibm.com, clg@redhat.com, iommu@lists.linux.dev, linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org Subject: [PATCH v5 5/5] iommu/s390: allow larger region tables Date: Fri, 11 Apr 2025 16:24:33 -0400 Message-ID: <20250411202433.181683-6-mjrosato@linux.ibm.com> X-Mailer: git-send-email 2.49.0 In-Reply-To: <20250411202433.181683-1-mjrosato@linux.ibm.com> References: <20250411202433.181683-1-mjrosato@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: SG2dXj0RIqhu0JwkeonmFgGDZdaryUdI X-Proofpoint-ORIG-GUID: SG2dXj0RIqhu0JwkeonmFgGDZdaryUdI X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1095,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-04-11_08,2025-04-10_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 priorityscore=1501 adultscore=0 malwarescore=0 impostorscore=0 lowpriorityscore=0 spamscore=0 mlxscore=0 suspectscore=0 mlxlogscore=852 clxscore=1015 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.19.0-2502280000 definitions=main-2504110127 Content-Type: text/plain; charset="utf-8" Extend the aperture calculation to consider sizes beyond the maximum size of a region third table. Attempt to always use the smallest table size possible to avoid unnecessary extra steps during translation. Update reserved region calculations to use the appropriate table size. Signed-off-by: Matthew Rosato Reviewed-by: Niklas Schnelle Tested-by: Niklas Schnelle --- arch/s390/include/asm/pci_dma.h | 1 + drivers/iommu/s390-iommu.c | 70 ++++++++++++++++++++++++--------- 2 files changed, 53 insertions(+), 18 deletions(-) diff --git a/arch/s390/include/asm/pci_dma.h b/arch/s390/include/asm/pci_dm= a.h index 8d8962e4fd58..d12e17201661 100644 --- a/arch/s390/include/asm/pci_dma.h +++ b/arch/s390/include/asm/pci_dma.h @@ -25,6 +25,7 @@ enum zpci_ioat_dtype { #define ZPCI_KEY (PAGE_DEFAULT_KEY << 5) =20 #define ZPCI_TABLE_SIZE_RT (1UL << 42) +#define ZPCI_TABLE_SIZE_RS (1UL << 53) =20 #define ZPCI_IOTA_STO_FLAG (ZPCI_IOTA_IOT_ENABLED | ZPCI_KEY | ZPCI_IOTA_D= T_ST) #define ZPCI_IOTA_RTTO_FLAG (ZPCI_IOTA_IOT_ENABLED | ZPCI_KEY | ZPCI_IOTA_= DT_RT) diff --git a/drivers/iommu/s390-iommu.c b/drivers/iommu/s390-iommu.c index 46f45b136993..433b59f43530 100644 --- a/drivers/iommu/s390-iommu.c +++ b/drivers/iommu/s390-iommu.c @@ -511,9 +511,25 @@ static bool s390_iommu_capable(struct device *dev, enu= m iommu_cap cap) } } =20 +static inline u64 max_tbl_size(struct s390_domain *domain) +{ + switch (domain->origin_type) { + case ZPCI_TABLE_TYPE_RTX: + return ZPCI_TABLE_SIZE_RT - 1; + case ZPCI_TABLE_TYPE_RSX: + return ZPCI_TABLE_SIZE_RS - 1; + case ZPCI_TABLE_TYPE_RFX: + return U64_MAX; + default: + return 0; + } +} + static struct iommu_domain *s390_domain_alloc_paging(struct device *dev) { + struct zpci_dev *zdev =3D to_zpci_dev(dev); struct s390_domain *s390_domain; + u64 aperture_size; =20 s390_domain =3D kzalloc(sizeof(*s390_domain), GFP_KERNEL); if (!s390_domain) @@ -524,10 +540,26 @@ static struct iommu_domain *s390_domain_alloc_paging(= struct device *dev) kfree(s390_domain); return NULL; } + + aperture_size =3D min(s390_iommu_aperture, + zdev->end_dma - zdev->start_dma + 1); + if (aperture_size <=3D (ZPCI_TABLE_SIZE_RT - zdev->start_dma)) { + s390_domain->origin_type =3D ZPCI_TABLE_TYPE_RTX; + } else if (aperture_size <=3D (ZPCI_TABLE_SIZE_RS - zdev->start_dma) && + (zdev->dtsm & ZPCI_IOTA_DT_RS)) { + s390_domain->origin_type =3D ZPCI_TABLE_TYPE_RSX; + } else if (zdev->dtsm & ZPCI_IOTA_DT_RF) { + s390_domain->origin_type =3D ZPCI_TABLE_TYPE_RFX; + } else { + /* Assume RTX available */ + s390_domain->origin_type =3D ZPCI_TABLE_TYPE_RTX; + aperture_size =3D ZPCI_TABLE_SIZE_RT - zdev->start_dma; + } + zdev->end_dma =3D zdev->start_dma + aperture_size - 1; + s390_domain->domain.geometry.force_aperture =3D true; s390_domain->domain.geometry.aperture_start =3D 0; - s390_domain->domain.geometry.aperture_end =3D ZPCI_TABLE_SIZE_RT - 1; - s390_domain->origin_type =3D ZPCI_TABLE_TYPE_RTX; + s390_domain->domain.geometry.aperture_end =3D max_tbl_size(s390_domain); =20 spin_lock_init(&s390_domain->list_lock); INIT_LIST_HEAD_RCU(&s390_domain->devices); @@ -680,6 +712,8 @@ static void s390_iommu_get_resv_regions(struct device *= dev, { struct zpci_dev *zdev =3D to_zpci_dev(dev); struct iommu_resv_region *region; + u64 max_size, end_resv; + unsigned long flags; =20 if (zdev->start_dma) { region =3D iommu_alloc_resv_region(0, zdev->start_dma, 0, @@ -689,10 +723,21 @@ static void s390_iommu_get_resv_regions(struct device= *dev, list_add_tail(®ion->list, list); } =20 - if (zdev->end_dma < ZPCI_TABLE_SIZE_RT - 1) { - region =3D iommu_alloc_resv_region(zdev->end_dma + 1, - ZPCI_TABLE_SIZE_RT - zdev->end_dma - 1, - 0, IOMMU_RESV_RESERVED, GFP_KERNEL); + spin_lock_irqsave(&zdev->dom_lock, flags); + if (zdev->s390_domain->type =3D=3D IOMMU_DOMAIN_BLOCKED || + zdev->s390_domain->type =3D=3D IOMMU_DOMAIN_IDENTITY) { + spin_unlock_irqrestore(&zdev->dom_lock, flags); + return; + } + + max_size =3D max_tbl_size(to_s390_domain(zdev->s390_domain)); + spin_unlock_irqrestore(&zdev->dom_lock, flags); + + if (zdev->end_dma < max_size) { + end_resv =3D max_size - zdev->end_dma; + region =3D iommu_alloc_resv_region(zdev->end_dma + 1, end_resv, + 0, IOMMU_RESV_RESERVED, + GFP_KERNEL); if (!region) return; list_add_tail(®ion->list, list); @@ -708,13 +753,9 @@ static struct iommu_device *s390_iommu_probe_device(st= ruct device *dev) =20 zdev =3D to_zpci_dev(dev); =20 - if (zdev->start_dma > zdev->end_dma || - zdev->start_dma > ZPCI_TABLE_SIZE_RT - 1) + if (zdev->start_dma > zdev->end_dma) return ERR_PTR(-EINVAL); =20 - if (zdev->end_dma > ZPCI_TABLE_SIZE_RT - 1) - zdev->end_dma =3D ZPCI_TABLE_SIZE_RT - 1; - if (zdev->tlb_refresh) dev->iommu->shadow_on_flush =3D 1; =20 @@ -999,7 +1040,6 @@ struct zpci_iommu_ctrs *zpci_get_iommu_ctrs(struct zpc= i_dev *zdev) =20 int zpci_init_iommu(struct zpci_dev *zdev) { - u64 aperture_size; int rc =3D 0; =20 rc =3D iommu_device_sysfs_add(&zdev->iommu_dev, NULL, NULL, @@ -1017,12 +1057,6 @@ int zpci_init_iommu(struct zpci_dev *zdev) if (rc) goto out_sysfs; =20 - zdev->start_dma =3D PAGE_ALIGN(zdev->start_dma); - aperture_size =3D min3(s390_iommu_aperture, - ZPCI_TABLE_SIZE_RT - zdev->start_dma, - zdev->end_dma - zdev->start_dma + 1); - zdev->end_dma =3D zdev->start_dma + aperture_size - 1; - return 0; =20 out_sysfs: --=20 2.49.0