From nobody Sat Apr 11 18:34:14 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=reject dis=none) header.from=linux.ibm.com ARC-Seal: i=1; a=rsa-sha256; t=1775874522; cv=none; d=zohomail.com; s=zohoarc; b=m40qztVjwLEg/coOWsIt6khGgba3T26d5aEs1VG26P6Bv8SsPwEUNuRBoRzx+KaUKLCvshMR535hycjDemGb3uw5J+TE4s6JKTaQSx671wvkFQ1GiUIeEa2sdAvfv1VJC8T7wks41u3y2idoWMKOyx23W77ld8nar6alq+xGGSk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1775874522; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=zP+6BKHqwiPV9Z25tenrM6T/B/avjR59LPW9kUw8TqM=; b=GsYRIpxz6aYhdioCvYdnnZYwVH5ZZveuZ/mLlymJhFnnOzRSeKmXtEUPBxk32OOInUvBDl8NLdMEE4uBQ2M7IsqIRmoxbMp00iUxd8np9jVbqSjta+0pP4dk1lrKMgkPHXbTRD17z4BdVLkaUrScu4wBrXY9AWwFzNZCLXh73Ks= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.gnu.org (lists1p.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1775874522201936.9280006857199; Fri, 10 Apr 2026 19:28:42 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1wBO3s-000239-Kj; Fri, 10 Apr 2026 22:27:04 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists1p.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wBO3r-00021X-1w; Fri, 10 Apr 2026 22:27:03 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1wBO3o-0000LV-EG; Fri, 10 Apr 2026 22:27:01 -0400 Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 63B2PANr2382161; Sat, 11 Apr 2026 02:26:54 GMT Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4dfcqer35y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 11 Apr 2026 02:26:54 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 63B18fQL026665; Sat, 11 Apr 2026 02:26:53 GMT Received: from smtprelay02.dal12v.mail.ibm.com ([172.16.1.4]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 4dcmg8ayxd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sat, 11 Apr 2026 02:26:53 +0000 Received: from smtpav03.wdc07v.mail.ibm.com (smtpav03.wdc07v.mail.ibm.com [10.39.53.230]) by smtprelay02.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 63B2QqFx31392440 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sat, 11 Apr 2026 02:26:52 GMT Received: from smtpav03.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1798F5805A; Sat, 11 Apr 2026 02:26:52 +0000 (GMT) Received: from smtpav03.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7CF7058054; Sat, 11 Apr 2026 02:26:51 +0000 (GMT) Received: from WIN-DU0DFC9G5VV.ibm.com (unknown [9.61.252.248]) by smtpav03.wdc07v.mail.ibm.com (Postfix) with ESMTP; Sat, 11 Apr 2026 02:26:51 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=zP+6BKHqwiPV9Z25t enrM6T/B/avjR59LPW9kUw8TqM=; b=juDkDG6SK830bT03HP6walWr6twAu9V7y HTQoLiBE1WsTcR0a+/8NSr8ghJDHx1WUw5NUn34Pi28rl3q+3lQc12s5bGRuLllp HpXqaHcZkPFo0d/29YJIrwajZtz9IwMimAFYGGHGLH8oQedFYGfAvMgq3qOT17Ey Cpc36gojamj3JvJC8fl1lJ+C3ZVZx/uV/F5Dl86LcCxUDy2pfk8rJzbZ0qLW4ePE sWqoAor3MW+zA8LWv/iKAZhb2/jPeTyUuizNLF+nYVlqGp5GcIiQpY/+Hx8Gws0W Z8l5zFGp0v8FZlcc6ouhfJbWdW1Uk42aUKbRcNtbibYn7gbuPV9Ag== From: Konstantin Shkolnyy To: mjrosato@linux.ibm.com Cc: alifm@linux.ibm.com, richard.henderson@linaro.org, iii@linux.ibm.com, david@kernel.org, cohuck@redhat.com, pasic@linux.ibm.com, borntraeger@linux.ibm.com, qemu-s390x@nongnu.org, qemu-devel@nongnu.org, Konstantin Shkolnyy Subject: [PATCH v2 01/15] s390x/pci: implement IOMMU replay Date: Fri, 10 Apr 2026 21:26:04 -0500 Message-Id: <20260411022618.2136588-2-kshk@linux.ibm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260411022618.2136588-1-kshk@linux.ibm.com> References: <20260411022618.2136588-1-kshk@linux.ibm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNDExMDAxNiBTYWx0ZWRfXzzqcIjPWqk/i uO751AcYA3J3hJH02Xz6ErvKtdi+KDMvveSMe6InnG9YyvXQefQvrl93rIMQx2iB8ZcIeEt5a/L zHwICuUM5ca5UpRMjjCUhkQWkipEtPt+Cwy0rM0T1hUgL32FsyOwV8S9NIMrnsbMorHdZ7zHEUO OJFIKePM3OqdqCSEtIfZ//H5bzf2DTG9dWYStpX5vwjzxdumM+5uYJkzFC6PfwLNZg3+1NLlMVG HtUj3kdA9jj9X6CI//fs2qLIGG67zDxV/z/buCdfhr812iz2Cqs4cT8Kja3cpPm1novXTtrgfqH 67zYJiKvd5iWHwU6IXCu22csnnsHgL+LWI2hyCP+BrZ2AESt9kZrJu/HCZ8Qtvuqhw/mmbAcUpv Z+lHI9k49wlv/5TD0Nncnl/Cl+LhIdrwmJD5fyGm2ZJZV/Q2YDBme+rKH7C6N/6EWr2kyAnj8mK sAbfsvBFABGPKVeJBUw== X-Proofpoint-GUID: DmF5m4pZlyrW0gGhW2KXExB3aUbWZppj X-Proofpoint-ORIG-GUID: DmF5m4pZlyrW0gGhW2KXExB3aUbWZppj X-Authority-Analysis: v=2.4 cv=YemNIQRf c=1 sm=1 tr=0 ts=69d9b16e cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=A5OVakUREuEA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=V8glGbnc2Ofi9Qvn3v5h:22 a=VnNF1IyMAAAA:8 a=CCwrLMjO1E3osjVnD3IA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-04-10_07,2026-04-09_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 adultscore=0 priorityscore=1501 lowpriorityscore=0 clxscore=1015 impostorscore=0 bulkscore=0 malwarescore=0 suspectscore=0 phishscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2604010000 definitions=main-2604110016 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.158.5; envelope-from=kshk@linux.ibm.com; helo=mx0b-001b2d01.pphosted.com X-Spam_score_int: -26 X-Spam_score: -2.7 X-Spam_bar: -- X-Spam_report: (-2.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: qemu development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @ibm.com) X-ZM-MESSAGEID: 1775874523675154100 Content-Type: text/plain; charset="utf-8" From: Matthew Rosato There are a few scenarios where IOMMU replay can potentially be needed for zPCI device, namely VFIO device reset scenarios where the guest continues running and expects the contents of its IOMMU to be replayed upon IOAT re-registration and migration scenarios where the destination must reconstruct the IOMMU on the destination. zPCI migration is not supported yet, but the IOMMU replay function is implemented so that it can be called both from IOMMUMemoryRegionClass now and migration post_load later. Signed-off-by: Matthew Rosato Signed-off-by: Konstantin Shkolnyy --- hw/s390x/s390-pci-bus.c | 50 ++++++++++++++++++++++++++++---- hw/s390x/s390-pci-inst.c | 4 +-- include/hw/s390x/s390-pci-inst.h | 1 + 3 files changed, 47 insertions(+), 8 deletions(-) diff --git a/hw/s390x/s390-pci-bus.c b/hw/s390x/s390-pci-bus.c index 4de7b587e8..3665aba106 100644 --- a/hw/s390x/s390-pci-bus.c +++ b/hw/s390x/s390-pci-bus.c @@ -592,14 +592,52 @@ err: return ret; } =20 -static void s390_pci_iommu_replay(IOMMUMemoryRegion *iommu, +static void s390_pci_ioat_replay(S390PCIIOMMU *iommu) +{ + S390IOTLBEntry entry; + uint16_t error =3D 0; + uint32_t dma_avail; + hwaddr curr, end; + + curr =3D iommu->pba; + end =3D iommu->pal; + + if (iommu->dm_mr) { + /* If direct mapping is used, there are no guest tables to replay = */ + return; + } + + if (iommu->dma_limit) { + dma_avail =3D iommu->dma_limit->avail; + } else { + dma_avail =3D 1; + } + + while (curr < end) { + error =3D s390_guest_io_table_walk(iommu->g_iota, curr, &entry); + if (error) { + error_report("Failure to walk table during iommu remap"); + return; + } + + if (entry.perm !=3D IOMMU_NONE) { + if (dma_avail > 0) { + dma_avail =3D s390_pci_update_iotlb(iommu, &entry); + } else { + error_report("DMA mappings exhausted: iommu remap failed"); + return; + } + } + curr +=3D entry.len; + } +} + +static void s390_pci_iommu_replay(IOMMUMemoryRegion *mr, IOMMUNotifier *notifier) { - /* It's impossible to plug a pci device on s390x that already has iommu - * mappings which need to be replayed, that is due to the "one iommu p= er - * zpci device" construct. But when we support migration of vfio-pci - * devices in future, we need to revisit this. - */ + S390PCIIOMMU *iommu =3D container_of(mr, S390PCIIOMMU, iommu_mr); + + s390_pci_ioat_replay(iommu); } =20 static S390PCIIOMMU *s390_pci_get_iommu(S390pciState *s, PCIBus *bus, diff --git a/hw/s390x/s390-pci-inst.c b/hw/s390x/s390-pci-inst.c index 10066ca618..1834596076 100644 --- a/hw/s390x/s390-pci-inst.c +++ b/hw/s390x/s390-pci-inst.c @@ -613,8 +613,8 @@ int pcistg_service_call(S390CPU *cpu, uint8_t r1, uint8= _t r2, uintptr_t ra) return 0; } =20 -static uint32_t s390_pci_update_iotlb(S390PCIIOMMU *iommu, - S390IOTLBEntry *entry) +uint32_t s390_pci_update_iotlb(S390PCIIOMMU *iommu, + S390IOTLBEntry *entry) { S390IOTLBEntry *cache =3D g_hash_table_lookup(iommu->iotlb, &entry->io= va); IOMMUTLBEvent event =3D { diff --git a/include/hw/s390x/s390-pci-inst.h b/include/hw/s390x/s390-pci-i= nst.h index 5cb8da540b..c782990e3b 100644 --- a/include/hw/s390x/s390-pci-inst.h +++ b/include/hw/s390x/s390-pci-inst.h @@ -111,6 +111,7 @@ int mpcifc_service_call(S390CPU *cpu, uint8_t r1, uint6= 4_t fiba, uint8_t ar, int stpcifc_service_call(S390CPU *cpu, uint8_t r1, uint64_t fiba, uint8_t = ar, uintptr_t ra); void fmb_timer_free(S390PCIBusDevice *pbdev); +uint32_t s390_pci_update_iotlb(S390PCIIOMMU *iommu, S390IOTLBEntry *entry); =20 #define ZPCI_IO_BAR_MIN 0 #define ZPCI_IO_BAR_MAX 5 --=20 2.34.1