From nobody Sat Apr 11 20:14:10 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=reject dis=none) header.from=linux.ibm.com ARC-Seal: i=1; a=rsa-sha256; t=1775098390; cv=none; d=zohomail.com; s=zohoarc; b=aI0GG8FnQcActY1F4AEBBM2pn/JavpfNuv1KktpYyiJWgXgv8Of0auN3HctXuLKijXLhHgfKhWZGpHZNr3jvmUTUMLkSD6r1JX01PWHXzQdb3uLye1LzEtRSr3BVJfrLWT0KowHEqWhJx6ECQ/o6wsm7nvt1ecG3Rduu9XWkWys= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1775098390; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=UXYpNRzv4+eZcjhxh3B2xqMU0eQ8GnuF6qMEze3ITME=; b=LFPAyWyWJQrZuEf1UkncSd2PuJxADc6R8yun6XC1TMUwBmf30oDnEA+6NkozO2AVALUa2P+ITjOtkEyksIlzxBr8FfITYDWCZj0ryaFRTNlmoZ11PHGkOF98yrt69A+ZxCM0qWYHfH72g1AsB8uM/j8lxnFFRJkHiYiOQdv4RLY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 177509839071351.22633758389702; Wed, 1 Apr 2026 19:53:10 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1w88AY-0001MC-04; Wed, 01 Apr 2026 22:52:31 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w88AO-0001Kl-O0; Wed, 01 Apr 2026 22:52:22 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1w88AM-0007jI-CS; Wed, 01 Apr 2026 22:52:20 -0400 Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 631Nb3Tb445673; Thu, 2 Apr 2026 02:52:13 GMT Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4d66g233ca-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 02 Apr 2026 02:52:13 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 631Mjbeb021665; Thu, 2 Apr 2026 02:52:12 GMT Received: from smtprelay07.dal12v.mail.ibm.com ([172.16.1.9]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 4d6sasrj6n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 02 Apr 2026 02:52:12 +0000 Received: from smtpav06.wdc07v.mail.ibm.com (smtpav06.wdc07v.mail.ibm.com [10.39.53.233]) by smtprelay07.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 6322qB1R31326766 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 2 Apr 2026 02:52:11 GMT Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 699C1580F5; Thu, 2 Apr 2026 02:29:52 +0000 (GMT) Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DF90A580F0; Thu, 2 Apr 2026 02:29:51 +0000 (GMT) Received: from WIN-DU0DFC9G5VV.ibm.com (unknown [9.61.247.119]) by smtpav06.wdc07v.mail.ibm.com (Postfix) with ESMTP; Thu, 2 Apr 2026 02:29:51 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=UXYpNRzv4+eZcjhxh 3B2xqMU0eQ8GnuF6qMEze3ITME=; b=kNnF/CDQhWsBk88rTQQ1icqsD2thW+QVn zE9kpxB6ntHAGRO0CEn5TaRq38Yzs0U/Sv4AJAEWDULeZDHYj7dABZxuWaFZYGnG jA4Pq0wGUCgW4zNSFripJfNWxN9IwOCNgrzCYlGYdDtQ3Rnx8PEjIBtlAA0+BdYL HTHnYCmreQUZmmifv6iQEgCfFZw2hQBPAYnFdADBZmpm1Aw2ax1g8VsR7PLLkaIi v4jzazpbxr8g/AzeEWeeTORdqBoAAQzoTxcudZJ1rypzBSAu8OgBR+N7dOyGSIav Gh1FvkmE+nfewzOHLyy4zKEKezCaBtHreIP2C9pjEVHLAIHfiJ+8w== From: Konstantin Shkolnyy To: mjrosato@linux.ibm.com Cc: alifm@linux.ibm.com, richard.henderson@linaro.org, iii@linux.ibm.com, david@kernel.org, cohuck@redhat.com, pasic@linux.ibm.com, borntraeger@linux.ibm.com, qemu-s390x@nongnu.org, qemu-devel@nongnu.org Subject: [PATCH 01/15] s390x/pci: implement IOMMU replay Date: Wed, 1 Apr 2026 21:29:07 -0500 Message-Id: <20260402022921.298818-2-kshk@linux.ibm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260402022921.298818-1-kshk@linux.ibm.com> References: <20260402022921.298818-1-kshk@linux.ibm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=Fdo6BZ+6 c=1 sm=1 tr=0 ts=69cdd9dd cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=A5OVakUREuEA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=iQ6ETzBq9ecOQQE5vZCe:22 a=VnNF1IyMAAAA:8 a=CCwrLMjO1E3osjVnD3IA:9 X-Proofpoint-ORIG-GUID: UJTxQi_7XAXOMLUSnZVqkNUguQfEPsFC X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNDAyMDAyNCBTYWx0ZWRfX+09Eo32RIH7z gG/UeeZ/6DusMx6vHNO4BeNrJo+S+2umLra/SOe3wDIE58kml+8wEkrkMQvx+4FYtpTyMvHtbjw YWbmFBXnSUNSxilR3dwPJ30+GU+FM1jgD/ATC8kmccofXsxCZYuUbqCaaEBLto8gh3n5AL8FeaA eMKMQJVOAR9Bu3+h5VxMFEYYiNk8HaI0DhtLsrh7jUvtEx/mLdIn99qEO7ac4gm48mEN28sYyEW 0AhDI8ubkP+TGeTG7xes0PS9M23oJK7LurJ14t8o2lZb6P9Po7X859RfGfxCQTwasZl2VFw3WPH 24Mf9m+0K7CeAxaYH9MmiykzOR0DZF87AAWSbTT0irzuYAnN9KxNWnowzbR6aULcpty2L8e2WN7 DJxjqZ/Y6m2NDSVrHokHi/C5YIg71bSaci+/gXLJQi8OYxpaA1z9KrUZsrI5ad1towaxwgUj63A v0+pYZbEjNwcxNcOvwQ== X-Proofpoint-GUID: UJTxQi_7XAXOMLUSnZVqkNUguQfEPsFC X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-04-01_05,2026-04-01_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 clxscore=1015 bulkscore=0 suspectscore=0 priorityscore=1501 adultscore=0 malwarescore=0 phishscore=0 spamscore=0 lowpriorityscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2604020024 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.156.1; envelope-from=kshk@linux.ibm.com; helo=mx0a-001b2d01.pphosted.com X-Spam_score_int: -6 X-Spam_score: -0.7 X-Spam_bar: / X-Spam_report: (-0.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=1, RCVD_IN_VALIDITY_RPBL_BLOCKED=1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @ibm.com) X-ZM-MESSAGEID: 1775098392120154100 Content-Type: text/plain; charset="utf-8" From: Matthew Rosato There are a few scenarios where IOMMU replay can potentially be needed for zPCI device, namely VFIO device reset scenarios where the guest continues running and expects the contents of its IOMMU to be replayed upon IOAT re-registration and migration scenarios where the destination must reconstruct the IOMMU on the destination. zPCI migration is not supported yet, but the IOMMU replay function is implemented so that it can be called both from IOMMUMemoryRegionClass now and migration post_load later. Signed-off-by: Matthew Rosato --- hw/s390x/s390-pci-bus.c | 50 ++++++++++++++++++++++++++++---- hw/s390x/s390-pci-inst.c | 4 +-- include/hw/s390x/s390-pci-inst.h | 1 + 3 files changed, 47 insertions(+), 8 deletions(-) diff --git a/hw/s390x/s390-pci-bus.c b/hw/s390x/s390-pci-bus.c index 4de7b587e8..3665aba106 100644 --- a/hw/s390x/s390-pci-bus.c +++ b/hw/s390x/s390-pci-bus.c @@ -592,14 +592,52 @@ err: return ret; } =20 -static void s390_pci_iommu_replay(IOMMUMemoryRegion *iommu, +static void s390_pci_ioat_replay(S390PCIIOMMU *iommu) +{ + S390IOTLBEntry entry; + uint16_t error =3D 0; + uint32_t dma_avail; + hwaddr curr, end; + + curr =3D iommu->pba; + end =3D iommu->pal; + + if (iommu->dm_mr) { + /* If direct mapping is used, there are no guest tables to replay = */ + return; + } + + if (iommu->dma_limit) { + dma_avail =3D iommu->dma_limit->avail; + } else { + dma_avail =3D 1; + } + + while (curr < end) { + error =3D s390_guest_io_table_walk(iommu->g_iota, curr, &entry); + if (error) { + error_report("Failure to walk table during iommu remap"); + return; + } + + if (entry.perm !=3D IOMMU_NONE) { + if (dma_avail > 0) { + dma_avail =3D s390_pci_update_iotlb(iommu, &entry); + } else { + error_report("DMA mappings exhausted: iommu remap failed"); + return; + } + } + curr +=3D entry.len; + } +} + +static void s390_pci_iommu_replay(IOMMUMemoryRegion *mr, IOMMUNotifier *notifier) { - /* It's impossible to plug a pci device on s390x that already has iommu - * mappings which need to be replayed, that is due to the "one iommu p= er - * zpci device" construct. But when we support migration of vfio-pci - * devices in future, we need to revisit this. - */ + S390PCIIOMMU *iommu =3D container_of(mr, S390PCIIOMMU, iommu_mr); + + s390_pci_ioat_replay(iommu); } =20 static S390PCIIOMMU *s390_pci_get_iommu(S390pciState *s, PCIBus *bus, diff --git a/hw/s390x/s390-pci-inst.c b/hw/s390x/s390-pci-inst.c index 10066ca618..1834596076 100644 --- a/hw/s390x/s390-pci-inst.c +++ b/hw/s390x/s390-pci-inst.c @@ -613,8 +613,8 @@ int pcistg_service_call(S390CPU *cpu, uint8_t r1, uint8= _t r2, uintptr_t ra) return 0; } =20 -static uint32_t s390_pci_update_iotlb(S390PCIIOMMU *iommu, - S390IOTLBEntry *entry) +uint32_t s390_pci_update_iotlb(S390PCIIOMMU *iommu, + S390IOTLBEntry *entry) { S390IOTLBEntry *cache =3D g_hash_table_lookup(iommu->iotlb, &entry->io= va); IOMMUTLBEvent event =3D { diff --git a/include/hw/s390x/s390-pci-inst.h b/include/hw/s390x/s390-pci-i= nst.h index 5cb8da540b..c782990e3b 100644 --- a/include/hw/s390x/s390-pci-inst.h +++ b/include/hw/s390x/s390-pci-inst.h @@ -111,6 +111,7 @@ int mpcifc_service_call(S390CPU *cpu, uint8_t r1, uint6= 4_t fiba, uint8_t ar, int stpcifc_service_call(S390CPU *cpu, uint8_t r1, uint64_t fiba, uint8_t = ar, uintptr_t ra); void fmb_timer_free(S390PCIBusDevice *pbdev); +uint32_t s390_pci_update_iotlb(S390PCIIOMMU *iommu, S390IOTLBEntry *entry); =20 #define ZPCI_IO_BAR_MIN 0 #define ZPCI_IO_BAR_MAX 5 --=20 2.34.1