From nobody Sat Nov 15 05:12:47 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=reject dis=none) header.from=linux.ibm.com ARC-Seal: i=1; a=rsa-sha256; t=1756157192; cv=none; d=zohomail.com; s=zohoarc; b=NbNVwm0JdRLQdU/on/UTMl1FFEC2u0cJY0FgqgsWcvqhks6fzZ+Xk729S5fBK9kUEId6KZp4g8vuJrR7qU4KBPYuOkVRiR99vhPOQ7cK1ugn0Qk0vBliSqclocS5xQ0gZYwHG0qnpGBVRVr4tHeTIW3RWzC78+9hwO8Z1/0bkMQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1756157192; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=mE3qJAivminB5MrLqinfqvAmlvPJ3NNgZIEw4ZxYtQI=; b=JmuWVNllSlUeu1/TePMihzAItBAvOCSeicnhyflCJsmfe0hru4h6sBecNRyE1mDsQJLKRZlkKaM8j9RP/36tydGx8Cyith1m0Bs95FNPqjwksKqnq4k7P38az4WuAEHujR6PRJBSyEX+FVDXEg4bRzHhOISp/IL2RtU3NiqMjFA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1756157192488994.0496292142451; Mon, 25 Aug 2025 14:26:32 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uqego-00029h-Dz; Mon, 25 Aug 2025 17:25:20 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uqega-0001xz-KP; Mon, 25 Aug 2025 17:25:05 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uqegI-0001r5-Mg; Mon, 25 Aug 2025 17:25:03 -0400 Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57PKDSMs028933; Mon, 25 Aug 2025 21:24:40 GMT Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48q32vbfsx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 25 Aug 2025 21:24:40 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57PINS0V002612; Mon, 25 Aug 2025 21:24:39 GMT Received: from smtprelay01.wdc07v.mail.ibm.com ([172.16.1.68]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 48qt6m7juu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 25 Aug 2025 21:24:39 +0000 Received: from smtpav06.dal12v.mail.ibm.com (smtpav06.dal12v.mail.ibm.com [10.241.53.105]) by smtprelay01.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57PLObbA24576496 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 25 Aug 2025 21:24:37 GMT Received: from smtpav06.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5029D58043; Mon, 25 Aug 2025 21:24:37 +0000 (GMT) Received: from smtpav06.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B47195805E; Mon, 25 Aug 2025 21:24:36 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.255.253]) by smtpav06.dal12v.mail.ibm.com (Postfix) with ESMTP; Mon, 25 Aug 2025 21:24:36 +0000 (GMT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=mE3qJAivminB5MrLq infqvAmlvPJ3NNgZIEw4ZxYtQI=; b=gghQol3z6x2dyNOnZA57soTclGUSlGmBN lcExQSMCjGSDaCr8PqUouZrkk6X13jJvShxp+PD1VJTepDN5broZ33qRaznC8KCo oyDfopsUphrFZ28lIEkPvmaVq3a8Q5pv/xbS2lXvyYMXXihDiLSoHrr9pi530NUB EqKkU0g8e9ErrFlT9okt0TOu2vosT3k2XNTHV+nT/clSoP6W+yIPTzvdVxjHfr8C gxfHhi03TFoJqYu0vAIZwR08WKcPIrCop5nLCqRq6vNsnEDzQzZ1BdzVB7+FVhkc 9jFt5aNtR3KIzS0ngMOY8jSsFnNzh8y81xAauPQqzR/Sw+lwEvytQ== From: Farhan Ali To: qemu-devel@nongnu.org, qemu-s390x@nongnu.org Cc: alifm@linux.ibm.com, mjrosato@linux.ibm.com, thuth@redhat.com, alex.williamson@redhat.com, clg@redhat.com Subject: [PATCH v2 3/4] s390x/pci: Add PCI error handling for vfio pci devices Date: Mon, 25 Aug 2025 14:24:32 -0700 Message-ID: <20250825212434.2255-4-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250825212434.2255-1-alifm@linux.ibm.com> References: <20250825212434.2255-1-alifm@linux.ibm.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: OQMMLRmGiEyZif6qtqs4iIJ_o7fP6o-a X-Proofpoint-GUID: OQMMLRmGiEyZif6qtqs4iIJ_o7fP6o-a X-Authority-Analysis: v=2.4 cv=AfSxH2XG c=1 sm=1 tr=0 ts=68acd498 cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=2OwXVqhp2XgA:10 a=VnNF1IyMAAAA:8 a=jSCLr9NRB7a5bD1T82IA:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODIzMDAwMCBTYWx0ZWRfX9mjzghsiUw8g +LFLQVYQs32RQPimAksp32Sx7KT47RKKwA+nfaGc3cihDBtYmdeWesrHCq/7MPFMd/XAo5lU/P7 qsAkEkWCoxbRgVJoWawEYpYHbiB+FPUjJzzyBf944DdhP6Jr/LKNZUcpmiNjy8wYOM60Gk3y7ng a353Z38s3hoJf0hKzPSY/T0PesXK4mR/iMYWDKg78q1T+x+Aupy8EOMdREcwNNpTW+dN/jIl81p ls3Hvc4Jyq1f9cFCVAnioklgHKx3yuzQuAEUo8WtPAucTEv5KHH0NKpWTkE6p+roTJvWzb36wH6 KjRdPTyxUIuYgALPy04cXHO3r1vThAKg+r8QtKQUdQVBF59IwmvgIdHSsojrBLGxzJ8z1O3L0Hy e1+gJpb6 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-25_10,2025-08-20_03,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 suspectscore=0 malwarescore=0 adultscore=0 spamscore=0 priorityscore=1501 bulkscore=0 impostorscore=0 clxscore=1015 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2508230000 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=148.163.158.5; envelope-from=alifm@linux.ibm.com; helo=mx0b-001b2d01.pphosted.com X-Spam_score_int: -26 X-Spam_score: -2.7 X-Spam_bar: -- X-Spam_report: (-2.7 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @ibm.com) X-ZM-MESSAGEID: 1756157193153124100 Content-Type: text/plain; charset="utf-8" Add an s390x specific callback for vfio error handling. For s390x pci devic= es, we have platform specific error information. We need to retrieve this error information for passthrough devices. This is done via a memory region which exposes that information. Once this error information is retrieved we can then inject an error into the guest, and let the guest drive the recovery. Signed-off-by: Farhan Ali --- hw/s390x/s390-pci-bus.c | 5 +++ hw/s390x/s390-pci-vfio.c | 76 ++++++++++++++++++++++++++++++++ include/hw/s390x/s390-pci-bus.h | 1 + include/hw/s390x/s390-pci-vfio.h | 2 + 4 files changed, 84 insertions(+) diff --git a/hw/s390x/s390-pci-bus.c b/hw/s390x/s390-pci-bus.c index f87d2748b6..af42eb9938 100644 --- a/hw/s390x/s390-pci-bus.c +++ b/hw/s390x/s390-pci-bus.c @@ -158,6 +158,8 @@ static void s390_pci_perform_unplug(S390PCIBusDevice *p= bdev) { HotplugHandler *hotplug_ctrl; =20 + qemu_mutex_destroy(&pbdev->err_handler_lock); + if (pbdev->pft =3D=3D ZPCI_PFT_ISM) { notifier_remove(&pbdev->shutdown_notifier); } @@ -1140,6 +1142,7 @@ static void s390_pcihost_plug(HotplugHandler *hotplug= _dev, DeviceState *dev, pbdev->iommu->pbdev =3D pbdev; pbdev->state =3D ZPCI_FS_DISABLED; set_pbdev_info(pbdev); + qemu_mutex_init(&pbdev->err_handler_lock); =20 if (object_dynamic_cast(OBJECT(dev), "vfio-pci")) { /* @@ -1164,6 +1167,8 @@ static void s390_pcihost_plug(HotplugHandler *hotplug= _dev, DeviceState *dev, pbdev->iommu->dma_limit =3D s390_pci_start_dma_count(s, pbdev); /* Fill in CLP information passed via the vfio region */ s390_pci_get_clp_info(pbdev); + /* Setup error handler for error recovery */ + s390_pci_setup_err_handler(pbdev); if (!pbdev->interp) { /* Do vfio passthrough but intercept for I/O */ pbdev->fh |=3D FH_SHM_VFIO; diff --git a/hw/s390x/s390-pci-vfio.c b/hw/s390x/s390-pci-vfio.c index aaf91319b4..87ecd06a81 100644 --- a/hw/s390x/s390-pci-vfio.c +++ b/hw/s390x/s390-pci-vfio.c @@ -10,6 +10,7 @@ */ =20 #include "qemu/osdep.h" +#include "qemu/error-report.h" =20 #include #include @@ -103,6 +104,60 @@ void s390_pci_end_dma_count(S390pciState *s, S390PCIDM= ACount *cnt) } } =20 +static int s390_pci_get_feature_err(VFIOPCIDevice *vfio_pci, + struct vfio_device_feature_zpci_err *e= rr) +{ + int ret; + uint64_t buf[DIV_ROUND_UP(sizeof(struct vfio_device_feature) + + sizeof(struct vfio_device_feature_zpci_err), + sizeof(uint64_t))] =3D {}; + struct vfio_device_feature *feature =3D (struct vfio_device_feature *)= buf; + + feature->argsz =3D sizeof(buf); + feature->flags =3D VFIO_DEVICE_FEATURE_GET | VFIO_DEVICE_FEATURE_ZPCI_= ERROR; + ret =3D vfio_pci->vbasedev.io_ops->device_feature(&vfio_pci->vbasedev, + feature); + + if (ret) { + error_report("Failed feature get VFIO_DEVICE_FEATURE_ZPCI_ERROR" + " (rc=3D%d)", ret); + return ret; + } + + memcpy(err, (struct vfio_device_feature_zpci_err *) feature->data, + sizeof(struct vfio_device_feature_zpci_err)); + return 0; +} + +static void s390_pci_err_handler(VFIOPCIDevice *vfio_pci) +{ + S390PCIBusDevice *pbdev; + struct vfio_device_feature_zpci_err err; + int ret; + + pbdev =3D s390_pci_find_dev_by_target(s390_get_phb(), + DEVICE(&vfio_pci->pdev)->id); + + QEMU_LOCK_GUARD(&pbdev->err_handler_lock); + + ret =3D s390_pci_get_feature_err(vfio_pci, &err); + if (ret) { + return; + } + + pbdev->state =3D ZPCI_FS_ERROR; + s390_pci_generate_error_event(err.pec, pbdev->fh, pbdev->fid, 0, 0); + + while (err.pending_errors) { + ret =3D s390_pci_get_feature_err(vfio_pci, &err); + if (ret) { + return; + } + s390_pci_generate_error_event(err.pec, pbdev->fh, pbdev->fid, 0, 0= ); + } + return; +} + static void s390_pci_read_base(S390PCIBusDevice *pbdev, struct vfio_device_info *info) { @@ -369,3 +424,24 @@ void s390_pci_get_clp_info(S390PCIBusDevice *pbdev) s390_pci_read_util(pbdev, info); s390_pci_read_pfip(pbdev, info); } + +void s390_pci_setup_err_handler(S390PCIBusDevice *pbdev) +{ + int ret; + VFIOPCIDevice *vfio_pci =3D container_of(pbdev->pdev, VFIOPCIDevice, = pdev); + uint64_t buf[DIV_ROUND_UP(sizeof(struct vfio_device_feature), + sizeof(uint64_t))] =3D {}; + struct vfio_device_feature *feature =3D (struct vfio_device_feature *)= buf; + + feature->argsz =3D sizeof(buf); + feature->flags =3D VFIO_DEVICE_FEATURE_PROBE | VFIO_DEVICE_FEATURE_ZPC= I_ERROR; + + ret =3D vfio_pci->vbasedev.io_ops->device_feature(&vfio_pci->vbasedev, + feature); + + if (ret) { + info_report("Automated error recovery not available for passthroug= h device"); + return; + } + vfio_pci->arch_err_handler =3D s390_pci_err_handler; +} diff --git a/include/hw/s390x/s390-pci-bus.h b/include/hw/s390x/s390-pci-bu= s.h index 04944d4fed..3795e0bbfc 100644 --- a/include/hw/s390x/s390-pci-bus.h +++ b/include/hw/s390x/s390-pci-bus.h @@ -364,6 +364,7 @@ struct S390PCIBusDevice { bool forwarding_assist; bool aif; bool rtr_avail; + QemuMutex err_handler_lock; QTAILQ_ENTRY(S390PCIBusDevice) link; }; =20 diff --git a/include/hw/s390x/s390-pci-vfio.h b/include/hw/s390x/s390-pci-v= fio.h index ae1b126ff7..66b274293c 100644 --- a/include/hw/s390x/s390-pci-vfio.h +++ b/include/hw/s390x/s390-pci-vfio.h @@ -22,6 +22,7 @@ S390PCIDMACount *s390_pci_start_dma_count(S390pciState *s, void s390_pci_end_dma_count(S390pciState *s, S390PCIDMACount *cnt); bool s390_pci_get_host_fh(S390PCIBusDevice *pbdev, uint32_t *fh); void s390_pci_get_clp_info(S390PCIBusDevice *pbdev); +void s390_pci_setup_err_handler(S390PCIBusDevice *pbdev); #else static inline bool s390_pci_update_dma_avail(int fd, unsigned int *avail) { @@ -39,6 +40,7 @@ static inline bool s390_pci_get_host_fh(S390PCIBusDevice = *pbdev, uint32_t *fh) return false; } static inline void s390_pci_get_clp_info(S390PCIBusDevice *pbdev) { } +static inline void s390_pci_setup_err_handler(S390PCIBusDevice *pbdev) { } #endif =20 #endif --=20 2.43.0