From nobody Sat Oct 4 19:15:06 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7616B303C85; Wed, 13 Aug 2025 17:08:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755104910; cv=none; b=ib8tHnfJp4WXTPe3u4JVtYxdaCixVouSguMUT6/EBw+IKxPsLNkFYlaau4d3LHwblu4WwU1G7UgR1BLo9YLx3vYVSQ+pTnKdwqljF1o/jfZzQDCEnAfVz0eeGAmWYeiFruzOMRIFg8/g56p/SezHWrNYjlXTKpR7R40Q3+BDllg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755104910; c=relaxed/simple; bh=GpTnZzUi7WUVvEXofT8h2Pi8qGUoIzk8UCB5WABBuY0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=QMFLTR/C8RNMP7GCd37HHTY2ljBD45zfjrLbowT4ogQkl+upJI+u+vrdLB293NwpUd56yqptCnFgYbT2dT8h+pMFz+Y0bWWVz+uTsqCFoHOKjFebFWEahzQF2leAuy7zV7JDfRv0CSXpoaHDya7AtyFe8RKD+AEm1X0kDsNirWM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=pv4kyQY8; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="pv4kyQY8" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57DCPXDk028484; Wed, 13 Aug 2025 17:08:26 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=/JQ5jQDq4TI8LPK6T 1B7HMqojffE60cYmDeEejGCZxA=; b=pv4kyQY8BxPD+TFfqokZ21XYbPBL5d0re 9mAz7WAYDNWbWXxljtxhYluEhmV9A/bLnY57maSMDoDVu5iOZXVtA8rF2hkR7LDT IaOgzNCMeA1lGn+6Aqw8eS5Ll03T7P0CZBVLz1wYWCQXCU2Pl41Fy+l7sJhGgKSk DigkO7jnmzUsaVmN/5mN9/gM7sjvOyQwm1O4G2JXY3xgG1eOn9S6TuUZ+XvwcuBF qYJ1csYHMVlrMQ54W6D7kl6G5j27acwxl6UmvGHSeYD1kbipq4S0aUYHqoicn+jr SmP5cHOz7qJmLj/MmC3X+GnDfPIhW9kik/p+KTmipqltv5ZYdqWJw== Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48durudrch-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 13 Aug 2025 17:08:26 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57DGXi6t025654; Wed, 13 Aug 2025 17:08:25 GMT Received: from smtprelay02.dal12v.mail.ibm.com ([172.16.1.4]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 48ejvmg355-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 13 Aug 2025 17:08:25 +0000 Received: from smtpav03.wdc07v.mail.ibm.com (smtpav03.wdc07v.mail.ibm.com [10.39.53.230]) by smtprelay02.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57DH8O3u56230382 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 13 Aug 2025 17:08:24 GMT Received: from smtpav03.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E958B5805A; Wed, 13 Aug 2025 17:08:23 +0000 (GMT) Received: from smtpav03.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 05CEC5805D; Wed, 13 Aug 2025 17:08:23 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.255.61]) by smtpav03.wdc07v.mail.ibm.com (Postfix) with ESMTP; Wed, 13 Aug 2025 17:08:22 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: schnelle@linux.ibm.com, mjrosato@linux.ibm.com, alifm@linux.ibm.com, alex.williamson@redhat.com Subject: [PATCH v1 1/6] s390/pci: Restore airq unconditionally for the zPCI device Date: Wed, 13 Aug 2025 10:08:15 -0700 Message-ID: <20250813170821.1115-2-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250813170821.1115-1-alifm@linux.ibm.com> References: <20250813170821.1115-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: 5VKREChJFwabw5pmjUPsFrw8FkkC1uba X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODEyMDIyNCBTYWx0ZWRfX70jzmOP3nnbn 7UuX9JBcKR5mBKorLZoe9QlvQZUmKUungIozotV2Fioyv93H06O9iShC6r+coaJZPTiubVTd7IE HsN5wxrUx37IVWCFUU64RL712bas/wceI+iZXEJbR57dXmbqqSOefQp2cyOrEfgHTewiCsznanH qEeyKueqry+/YM4GrTJycInr52BB86p3cGe7uNVdd0TcEy+Y1I7zZfhRHtj6WbKzUhIuzXgWK6P zhnigak/A5jsu2Iwoqkfa2JjI8LZlkYpJwCVPpnFoci+MOjANCfTzn2qfUZKI+evOzhWDy5tHVc tWr8L+PFaLlfN0vUpgfggF9scq4Jsscv5JiSuwL1J1WVtC14tBUJLpLhN+6ffKWQTMb2+2gANio C6/oGjat X-Authority-Analysis: v=2.4 cv=QtNe3Uyd c=1 sm=1 tr=0 ts=689cc68a cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=2OwXVqhp2XgA:10 a=VnNF1IyMAAAA:8 a=rU-0L4YOitQFXdO97IcA:9 X-Proofpoint-ORIG-GUID: 5VKREChJFwabw5pmjUPsFrw8FkkC1uba X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-13_01,2025-08-11_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 priorityscore=1501 phishscore=0 clxscore=1011 malwarescore=0 spamscore=0 suspectscore=0 impostorscore=0 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2508120224 Content-Type: text/plain; charset="utf-8" Commit c1e18c17bda6 ("s390/pci: add zpci_set_irq()/zpci_clear_irq()"), introduced the zpci_set_irq() and zpci_clear_irq(), to be used while resetting a zPCI device. Commit da995d538d3a ("s390/pci: implement reset_slot for hotplug slot"), mentions zpci_clear_irq() being called in the path for zpci_hot_reset_devic= e(). But that is not the case anymore and these functions are not called outside of this file. However after a CLP disable/enable reset (zpci_hot_reset_device), the airq setup of the device will need to be restored. Since we are no longer calling zpci_clear_airq() in the reset path, we should restore the airq for device unconditionally. Signed-off-by: Farhan Ali --- arch/s390/pci/pci_irq.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/arch/s390/pci/pci_irq.c b/arch/s390/pci/pci_irq.c index 84482a921332..8b5493f0dee0 100644 --- a/arch/s390/pci/pci_irq.c +++ b/arch/s390/pci/pci_irq.c @@ -427,8 +427,7 @@ bool arch_restore_msi_irqs(struct pci_dev *pdev) { struct zpci_dev *zdev =3D to_zpci(pdev); =20 - if (!zdev->irqs_registered) - zpci_set_irq(zdev); + zpci_set_irq(zdev); return true; } =20 --=20 2.43.0 From nobody Sat Oct 4 19:15:06 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 03ABF2D12E6; Wed, 13 Aug 2025 17:08:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755104912; cv=none; b=XzS9h6sSDF5EVFh/hpyMbijI4mBzXaxV/4+GSdrKvwOIrzm/pVv/BlYoX359xexNcHfrauK5Yb8dwjNEI1pU2mTzWNbs6sHrfsaDmm0BHBZwo7AAp/qBOcMIr33cTG3HN65TB9EXTYKdJgjGmr/sO8f+aEfEpMLy8tHftgtWlvU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755104912; c=relaxed/simple; bh=V9RepE83IIiUTmqKpVJDxpRRNiKGDVzZe3FnWc+aLFA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=O9n8rLapKIIF+eGWd71ukP2zDxLTf7cXJx9/QYxWAkYZcNU4jHFZf7jWQcvMhdDLGLSlAcSxZjVzu7fSCv8Dm9UCSjTLVEsURRiJSE8ArsX8fn3Wh2ewXFqyPmhIQV9qZ3xgAcKZZZowqfSKbNMHI19voEhijmcbejaa7CD+4Jk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=ZV76fK2+; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="ZV76fK2+" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57DBNlmg030230; Wed, 13 Aug 2025 17:08:28 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=Gdzkp8P0Sl4AK+1Ci zpGJ3YvQvorUVDxbdF9qMhLtUo=; b=ZV76fK2+IOqf5X3qMn0pDlbokjvzUOPA/ helYsuxJH3waVlukhRqbLDw0NQlHM4xVwQDx4v88+llSrydiklBIXprQ1Dz0FcS6 ULPxJ4DFLrw9KYsPHzoZDBB/iS/39pGMoYTkrb9WL2iscVv9QyuouEv6QJ5RFvu0 N7GI5kOhxhNr0dy5WbZB2LkWcACI6c0dGSA6hlqGl0kNGapNmIoHfd0iigkMl5y0 hK+DwZxzy+WQRak1LbsLreg2XUoiFd/gsEyIx3IHmPx+ZeQjX8LE/J6LwzYXeXeG MP0Y8/g6G/Xq50fokDxPFrKhEq9unGq6cZL225gYjZ95P4/Y93AYw== Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48dwuddt5u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 13 Aug 2025 17:08:28 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57DFWO33026270; Wed, 13 Aug 2025 17:08:27 GMT Received: from smtprelay03.dal12v.mail.ibm.com ([172.16.1.5]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 48eh218ehw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 13 Aug 2025 17:08:27 +0000 Received: from smtpav03.wdc07v.mail.ibm.com (smtpav03.wdc07v.mail.ibm.com [10.39.53.230]) by smtprelay03.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57DH8P3f27460126 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 13 Aug 2025 17:08:25 GMT Received: from smtpav03.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6265058062; Wed, 13 Aug 2025 17:08:25 +0000 (GMT) Received: from smtpav03.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6D9535805A; Wed, 13 Aug 2025 17:08:24 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.255.61]) by smtpav03.wdc07v.mail.ibm.com (Postfix) with ESMTP; Wed, 13 Aug 2025 17:08:24 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: schnelle@linux.ibm.com, mjrosato@linux.ibm.com, alifm@linux.ibm.com, alex.williamson@redhat.com Subject: [PATCH v1 2/6] s390/pci: Update the logic for detecting passthrough device Date: Wed, 13 Aug 2025 10:08:16 -0700 Message-ID: <20250813170821.1115-3-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250813170821.1115-1-alifm@linux.ibm.com> References: <20250813170821.1115-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODEyMDIyNCBTYWx0ZWRfX1W0Q2mFDZDwc nhPSSVxYMuOK+fjFZuBicIMv3sVRoq6VFvpulgSH3Z7brFCFOC4wnLlrH4+KkQDOm5gjJ7GzBAi Ih7nsCS2nyqHGkNVXkQUOZ+5bxWpMqlkq3NiB7xvyYhKRfabvIvhLm4P7Im2iJPG0bySYsR+cHP P8YUuSNb9TQIvvB1TZidd6PxabQrmNi3WKZEEA94anKUqkUp4M869OVmL2/otDCTKEmtUIhMN+b g7n3kbkESHtm8k4BQ6u/Wu72ViSEH7s+VOM7v13E6CGWjOv3oPqIDcDVkodH8aF85YgpdQVrIoC 5jLh5sHKFDN41vAnL0m142Jhb7wvn47d5lPOw/xFjz3cxZXiF11PKQ+7xdwxUvCzZmwsh4Ys0Ie a3OHjuDJ X-Authority-Analysis: v=2.4 cv=d/31yQjE c=1 sm=1 tr=0 ts=689cc68c cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=2OwXVqhp2XgA:10 a=VnNF1IyMAAAA:8 a=wIibNaKEpnRuRg3xOlQA:9 X-Proofpoint-GUID: 99AX_1H_JwSpSKawa_EBjosIXo76-Uo7 X-Proofpoint-ORIG-GUID: 99AX_1H_JwSpSKawa_EBjosIXo76-Uo7 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-13_01,2025-08-11_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1011 suspectscore=0 impostorscore=0 priorityscore=1501 phishscore=0 spamscore=0 adultscore=0 bulkscore=0 malwarescore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2508120224 Content-Type: text/plain; charset="utf-8" We can now have userspace drivers (vfio-pci based) on s390x. The userspace drivers will not have any KVM fd and so no kzdev associated with them. So we need to update the logic for detecting passthrough devices to not depend on struct kvm_zdev. Signed-off-by: Farhan Ali --- arch/s390/include/asm/pci.h | 1 + arch/s390/pci/pci_event.c | 14 ++++---------- drivers/vfio/pci/vfio_pci_zdev.c | 9 ++++++++- 3 files changed, 13 insertions(+), 11 deletions(-) diff --git a/arch/s390/include/asm/pci.h b/arch/s390/include/asm/pci.h index 41f900f693d9..0705a2f52263 100644 --- a/arch/s390/include/asm/pci.h +++ b/arch/s390/include/asm/pci.h @@ -170,6 +170,7 @@ struct zpci_dev { =20 char res_name[16]; bool mio_capable; + bool mediated_recovery; struct zpci_bar_struct bars[PCI_STD_NUM_BARS]; =20 u64 start_dma; /* Start of available DMA addresses */ diff --git a/arch/s390/pci/pci_event.c b/arch/s390/pci/pci_event.c index d930416d4c90..541d536be052 100644 --- a/arch/s390/pci/pci_event.c +++ b/arch/s390/pci/pci_event.c @@ -61,16 +61,10 @@ static inline bool ers_result_indicates_abort(pci_ers_r= esult_t ers_res) } } =20 -static bool is_passed_through(struct pci_dev *pdev) +static bool needs_mediated_recovery(struct pci_dev *pdev) { struct zpci_dev *zdev =3D to_zpci(pdev); - bool ret; - - mutex_lock(&zdev->kzdev_lock); - ret =3D !!zdev->kzdev; - mutex_unlock(&zdev->kzdev_lock); - - return ret; + return zdev->mediated_recovery; } =20 static bool is_driver_supported(struct pci_driver *driver) @@ -194,7 +188,7 @@ static pci_ers_result_t zpci_event_attempt_error_recove= ry(struct pci_dev *pdev) } pdev->error_state =3D pci_channel_io_frozen; =20 - if (is_passed_through(pdev)) { + if (needs_mediated_recovery(pdev)) { pr_info("%s: Cannot be recovered in the host because it is a pass-throug= h device\n", pci_name(pdev)); status_str =3D "failed (pass-through)"; @@ -277,7 +271,7 @@ static void zpci_event_io_failure(struct pci_dev *pdev,= pci_channel_state_t es) * we will inject the error event and let the guest recover the device * itself. */ - if (is_passed_through(pdev)) + if (needs_mediated_recovery(pdev)) goto out; driver =3D to_pci_driver(pdev->dev.driver); if (driver && driver->err_handler && driver->err_handler->error_detected) diff --git a/drivers/vfio/pci/vfio_pci_zdev.c b/drivers/vfio/pci/vfio_pci_z= dev.c index 0990fdb146b7..a7bc23ce8483 100644 --- a/drivers/vfio/pci/vfio_pci_zdev.c +++ b/drivers/vfio/pci/vfio_pci_zdev.c @@ -148,6 +148,8 @@ int vfio_pci_zdev_open_device(struct vfio_pci_core_devi= ce *vdev) if (!zdev) return -ENODEV; =20 + zdev->mediated_recovery =3D true; + if (!vdev->vdev.kvm) return 0; =20 @@ -161,7 +163,12 @@ void vfio_pci_zdev_close_device(struct vfio_pci_core_d= evice *vdev) { struct zpci_dev *zdev =3D to_zpci(vdev->pdev); =20 - if (!zdev || !vdev->vdev.kvm) + if (!zdev) + return; + + zdev->mediated_recovery =3D false; + + if (!vdev->vdev.kvm) return; =20 if (zpci_kvm_hook.kvm_unregister) --=20 2.43.0 From nobody Sat Oct 4 19:15:06 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8A9232D7392; Wed, 13 Aug 2025 17:08:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755104913; cv=none; b=jsQ6b9jcWJ011BrIdz9eZlSIQbfaB+9Omea8mZvkmmRjgErFnismwUcp1X24mNIx+9Zf9FqLrT5/koIzUBZU3jgmdRE5T6cJFFlgV9Q6heHx9q8TRFtlt76K0gDoHFUwawyeIu+AjEapzzdifmxrJjn/hvqEzHmm2KcC5TFhv3c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755104913; c=relaxed/simple; bh=i+E+aH39o5jr/uCy1aRCF255mKXKVmkOKTvV8MXBnXI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KGKZ+cAc+idvbvIi5EdEJJVlgIf+D7ZNPzN+0BSBeM7I4dFz8n5hKh9yXHNo34vZTQjFTeT/MV4y2lTg4RKqr/ZmnGTifASHZ2KgVQdvIakLcCmCglbJJzmt79RnjzjaUq6VRcfCxUqzvmlewAgF9ZT9hQ0ncc4y2OEijmwbuiI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=WpgUEcLh; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="WpgUEcLh" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57DAwkme016338; Wed, 13 Aug 2025 17:08:29 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=a9TpNdff9Gjb4P9Gp SKIUjHy6e7Etdc7x4s2y4Bqv/k=; b=WpgUEcLhvB3qSkQxoxZ3FvXB+9/TI74L3 ZdYBgyfAbjGIgywtpsYrH7KZVCdZieDMLbrdW6LQfwJU9YU9gq/o94iO9sa12+yc 3+huQu1fhnSBDvdpVIFCuI33g5rkskkDCT7N7r+hgV7Gu/KR7MVzKqTmIYMUnYrM YPe1iXbKo1ufbWwi9W8djqbpCzsWrB3En5a3ZbfdZ05S1Li0EFguHnVlMcUnDbpw iV425g86foJZbCjorbUg+FjT2/jLg88Zpr4ueG2aKT/3Zymi7XJvzPs6QJ+FrQvZ SBX0wEEJRKcyvfuD+8FNGI9FLvPC90+Z+noQrYFUVfdKGUF4ev3mg== Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48dx14nr0k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 13 Aug 2025 17:08:29 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57DGxk86017600; Wed, 13 Aug 2025 17:08:28 GMT Received: from smtprelay04.wdc07v.mail.ibm.com ([172.16.1.71]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 48ekc3qyjc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 13 Aug 2025 17:08:28 +0000 Received: from smtpav03.wdc07v.mail.ibm.com (smtpav03.wdc07v.mail.ibm.com [10.39.53.230]) by smtprelay04.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57DH8QPs58065212 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 13 Aug 2025 17:08:26 GMT Received: from smtpav03.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 80CE958054; Wed, 13 Aug 2025 17:08:26 +0000 (GMT) Received: from smtpav03.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 91BBC5805A; Wed, 13 Aug 2025 17:08:25 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.255.61]) by smtpav03.wdc07v.mail.ibm.com (Postfix) with ESMTP; Wed, 13 Aug 2025 17:08:25 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: schnelle@linux.ibm.com, mjrosato@linux.ibm.com, alifm@linux.ibm.com, alex.williamson@redhat.com Subject: [PATCH v1 3/6] s390/pci: Store PCI error information for passthrough devices Date: Wed, 13 Aug 2025 10:08:17 -0700 Message-ID: <20250813170821.1115-4-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250813170821.1115-1-alifm@linux.ibm.com> References: <20250813170821.1115-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: n9bSgEDLoZHqMcj1o-SBYv-kbZbm0HmA X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODEyMDIyNCBTYWx0ZWRfX5uUP8rNqEDA9 8ZJ4GkbrVhS7K/7HrXBeJpThDWB5D7sUA4BxHfe9zFRTwh9gcvUcCm2fJtX7/OCB+2BvP6E6+Ed aSlbHNanNKhXsIDPNUyYWV2p5WtdfGiJpIB8WU2uqOF8Nw7xoDJnxR/2NP4IBp+ByPtbHDZIES7 JaC18l29D5+r2IGpeNtt+kqQHcqJOjLM9HJng6J7OwmBvvE+S4W3laqCSh0tVX5JvXOxCxHRg7v cjtBM0ut3DHxEgof98U297uF7Wzpwt6sWaXj1AAOmhj4UvigMEqzQztWXyTLKaR7kJr7hb7LOvO YOkzkPi051gM8/3iBp5Q1bA+/Ws1qdZrtZfcqJgaWW9i9sJYCoRnoSpKMSJqj7b/aHrWIbgwVhE RCYM8MYc X-Proofpoint-GUID: n9bSgEDLoZHqMcj1o-SBYv-kbZbm0HmA X-Authority-Analysis: v=2.4 cv=fLg53Yae c=1 sm=1 tr=0 ts=689cc68d cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=2OwXVqhp2XgA:10 a=VnNF1IyMAAAA:8 a=3QccEKBQ3G6yB81z3WwA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-13_01,2025-08-11_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 clxscore=1015 spamscore=0 priorityscore=1501 impostorscore=0 phishscore=0 malwarescore=0 bulkscore=0 suspectscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2508120224 Content-Type: text/plain; charset="utf-8" For a passthrough device we need co-operation from user space to recover the device. This would require to bubble up any error information to user space. Let's store this error information for passthrough devices, so it can be retrieved later. Signed-off-by: Farhan Ali --- arch/s390/include/asm/pci.h | 28 ++++++++++ arch/s390/pci/pci.c | 1 + arch/s390/pci/pci_event.c | 95 +++++++++++++++++++------------- drivers/vfio/pci/vfio_pci_zdev.c | 2 + 4 files changed, 88 insertions(+), 38 deletions(-) diff --git a/arch/s390/include/asm/pci.h b/arch/s390/include/asm/pci.h index 0705a2f52263..9b4abbd806f2 100644 --- a/arch/s390/include/asm/pci.h +++ b/arch/s390/include/asm/pci.h @@ -116,6 +116,31 @@ struct zpci_bus { enum pci_bus_speed max_bus_speed; }; =20 +/* Content Code Description for PCI Function Error */ +struct zpci_ccdf_err { + u32 reserved1; + u32 fh; /* function handle */ + u32 fid; /* function id */ + u32 ett : 4; /* expected table type */ + u32 mvn : 12; /* MSI vector number */ + u32 dmaas : 8; /* DMA address space */ + u32 reserved2 : 6; + u32 q : 1; /* event qualifier */ + u32 rw : 1; /* read/write */ + u64 faddr; /* failing address */ + u32 reserved3; + u16 reserved4; + u16 pec; /* PCI event code */ +} __packed; + +#define ZPCI_ERR_PENDING_MAX 16 +struct zpci_ccdf_pending { + size_t count; + int head; + int tail; + struct zpci_ccdf_err err[ZPCI_ERR_PENDING_MAX]; +}; + /* Private data per function */ struct zpci_dev { struct zpci_bus *zbus; @@ -192,6 +217,8 @@ struct zpci_dev { struct iommu_domain *s390_domain; /* attached IOMMU domain */ struct kvm_zdev *kzdev; struct mutex kzdev_lock; + struct zpci_ccdf_pending pending_errs; + struct mutex pending_errs_lock; spinlock_t dom_lock; /* protect s390_domain change */ }; =20 @@ -317,6 +344,7 @@ void zpci_debug_exit_device(struct zpci_dev *); int zpci_report_error(struct pci_dev *, struct zpci_report_error_header *); int zpci_clear_error_state(struct zpci_dev *zdev); int zpci_reset_load_store_blocked(struct zpci_dev *zdev); +void zpci_cleanup_pending_errors(struct zpci_dev *zdev); =20 #ifdef CONFIG_NUMA =20 diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c index cd6676c2d602..f795e05b5001 100644 --- a/arch/s390/pci/pci.c +++ b/arch/s390/pci/pci.c @@ -823,6 +823,7 @@ struct zpci_dev *zpci_create_device(u32 fid, u32 fh, en= um zpci_state state) mutex_init(&zdev->state_lock); mutex_init(&zdev->fmb_lock); mutex_init(&zdev->kzdev_lock); + mutex_init(&zdev->pending_errs_lock); =20 return zdev; =20 diff --git a/arch/s390/pci/pci_event.c b/arch/s390/pci/pci_event.c index 541d536be052..ac527410812b 100644 --- a/arch/s390/pci/pci_event.c +++ b/arch/s390/pci/pci_event.c @@ -18,23 +18,6 @@ #include "pci_bus.h" #include "pci_report.h" =20 -/* Content Code Description for PCI Function Error */ -struct zpci_ccdf_err { - u32 reserved1; - u32 fh; /* function handle */ - u32 fid; /* function id */ - u32 ett : 4; /* expected table type */ - u32 mvn : 12; /* MSI vector number */ - u32 dmaas : 8; /* DMA address space */ - u32 : 6; - u32 q : 1; /* event qualifier */ - u32 rw : 1; /* read/write */ - u64 faddr; /* failing address */ - u32 reserved3; - u16 reserved4; - u16 pec; /* PCI event code */ -} __packed; - /* Content Code Description for PCI Function Availability */ struct zpci_ccdf_avail { u32 reserved1; @@ -76,6 +59,41 @@ static bool is_driver_supported(struct pci_driver *drive= r) return true; } =20 +static void zpci_store_pci_error(struct pci_dev *pdev, + struct zpci_ccdf_err *ccdf) +{ + struct zpci_dev *zdev =3D to_zpci(pdev); + int i; + + mutex_lock(&zdev->pending_errs_lock); + if (zdev->pending_errs.count >=3D ZPCI_ERR_PENDING_MAX) { + pr_err("%s: Cannot store PCI error info for device", + pci_name(pdev)); + mutex_unlock(&zdev->pending_errs_lock); + return; + } + + i =3D zdev->pending_errs.tail % ZPCI_ERR_PENDING_MAX; + memcpy(&zdev->pending_errs.err[i], ccdf, sizeof(struct zpci_ccdf_err)); + zdev->pending_errs.tail++; + zdev->pending_errs.count++; + mutex_unlock(&zdev->pending_errs_lock); +} + +void zpci_cleanup_pending_errors(struct zpci_dev *zdev) +{ + struct pci_dev *pdev =3D NULL; + + mutex_lock(&zdev->pending_errs_lock); + pdev =3D pci_get_slot(zdev->zbus->bus, zdev->devfn); + if (zdev->pending_errs.count) + pr_err("%s: Unhandled PCI error events count=3D%zu", + pci_name(pdev), zdev->pending_errs.count); + memset(&zdev->pending_errs, 0, sizeof(struct zpci_ccdf_pending)); + mutex_unlock(&zdev->pending_errs_lock); +} +EXPORT_SYMBOL_GPL(zpci_cleanup_pending_errors); + static pci_ers_result_t zpci_event_notify_error_detected(struct pci_dev *p= dev, struct pci_driver *driver) { @@ -169,7 +187,8 @@ static pci_ers_result_t zpci_event_do_reset(struct pci_= dev *pdev, * and the platform determines which functions are affected for * multi-function devices. */ -static pci_ers_result_t zpci_event_attempt_error_recovery(struct pci_dev *= pdev) +static pci_ers_result_t zpci_event_attempt_error_recovery(struct pci_dev *= pdev, + struct zpci_ccdf_err *ccdf) { pci_ers_result_t ers_res =3D PCI_ERS_RESULT_DISCONNECT; struct zpci_dev *zdev =3D to_zpci(pdev); @@ -188,13 +207,6 @@ static pci_ers_result_t zpci_event_attempt_error_recov= ery(struct pci_dev *pdev) } pdev->error_state =3D pci_channel_io_frozen; =20 - if (needs_mediated_recovery(pdev)) { - pr_info("%s: Cannot be recovered in the host because it is a pass-throug= h device\n", - pci_name(pdev)); - status_str =3D "failed (pass-through)"; - goto out_unlock; - } - driver =3D to_pci_driver(pdev->dev.driver); if (!is_driver_supported(driver)) { if (!driver) { @@ -210,12 +222,22 @@ static pci_ers_result_t zpci_event_attempt_error_reco= very(struct pci_dev *pdev) goto out_unlock; } =20 + if (needs_mediated_recovery(pdev)) + zpci_store_pci_error(pdev, ccdf); + ers_res =3D zpci_event_notify_error_detected(pdev, driver); if (ers_result_indicates_abort(ers_res)) { status_str =3D "failed (abort on detection)"; goto out_unlock; } =20 + if (needs_mediated_recovery(pdev)) { + pr_info("%s: Recovering passthrough device\n", pci_name(pdev)); + ers_res =3D PCI_ERS_RESULT_RECOVERED; + status_str =3D "in progress"; + goto out_unlock; + } + if (ers_res !=3D PCI_ERS_RESULT_NEED_RESET) { ers_res =3D zpci_event_do_error_state_clear(pdev, driver); if (ers_result_indicates_abort(ers_res)) { @@ -258,25 +280,20 @@ static pci_ers_result_t zpci_event_attempt_error_reco= very(struct pci_dev *pdev) * @pdev: PCI function for which to report * @es: PCI channel failure state to report */ -static void zpci_event_io_failure(struct pci_dev *pdev, pci_channel_state_= t es) +static void zpci_event_io_failure(struct pci_dev *pdev, pci_channel_state_= t es, + struct zpci_ccdf_err *ccdf) { struct pci_driver *driver; =20 pci_dev_lock(pdev); pdev->error_state =3D es; - /** - * While vfio-pci's error_detected callback notifies user-space QEMU - * reacts to this by freezing the guest. In an s390 environment PCI - * errors are rarely fatal so this is overkill. Instead in the future - * we will inject the error event and let the guest recover the device - * itself. - */ + if (needs_mediated_recovery(pdev)) - goto out; + zpci_store_pci_error(pdev, ccdf); driver =3D to_pci_driver(pdev->dev.driver); if (driver && driver->err_handler && driver->err_handler->error_detected) driver->err_handler->error_detected(pdev, pdev->error_state); -out: + pci_dev_unlock(pdev); } =20 @@ -312,6 +329,7 @@ static void __zpci_event_error(struct zpci_ccdf_err *cc= df) pr_err("%s: Event 0x%x reports an error for PCI function 0x%x\n", pdev ? pci_name(pdev) : "n/a", ccdf->pec, ccdf->fid); =20 + if (!pdev) goto no_pdev; =20 @@ -322,12 +340,13 @@ static void __zpci_event_error(struct zpci_ccdf_err *= ccdf) break; case 0x0040: /* Service Action or Error Recovery Failed */ case 0x003b: - zpci_event_io_failure(pdev, pci_channel_io_perm_failure); + zpci_event_io_failure(pdev, pci_channel_io_perm_failure, ccdf); break; default: /* PCI function left in the error state attempt to recover */ - ers_res =3D zpci_event_attempt_error_recovery(pdev); + ers_res =3D zpci_event_attempt_error_recovery(pdev, ccdf); if (ers_res !=3D PCI_ERS_RESULT_RECOVERED) - zpci_event_io_failure(pdev, pci_channel_io_perm_failure); + zpci_event_io_failure(pdev, pci_channel_io_perm_failure, + ccdf); break; } pci_dev_put(pdev); diff --git a/drivers/vfio/pci/vfio_pci_zdev.c b/drivers/vfio/pci/vfio_pci_z= dev.c index a7bc23ce8483..2be37eab9279 100644 --- a/drivers/vfio/pci/vfio_pci_zdev.c +++ b/drivers/vfio/pci/vfio_pci_zdev.c @@ -168,6 +168,8 @@ void vfio_pci_zdev_close_device(struct vfio_pci_core_de= vice *vdev) =20 zdev->mediated_recovery =3D false; =20 + zpci_cleanup_pending_errors(zdev); + if (!vdev->vdev.kvm) return; =20 --=20 2.43.0 From nobody Sat Oct 4 19:15:06 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 783862E8DF1; Wed, 13 Aug 2025 17:08:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755104915; cv=none; b=MK+LwCZLM4cyVaLtH4N+6qk5CUnUgeU2XV51HgUcpn3mNmZUSquokih6zxChSyTADhorAx0KhmfaAWm1CywcxHHaTaK9um8kgP350nBELDEsPwE/WKtAVt8zgsW6LAveoue3qDMc0JCdgtE/uioRxf9DtZTYGrKUEGCwCylKXqE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755104915; c=relaxed/simple; bh=H0olbHu3gddiL94/NLGS2A4nkln3us/u5UpnT6aTIRg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KgcO8+03lAKzFT5sJMP32DPKWAnTtEaXeV6Q7BO/hswRyZy8A6xoDVFiuFNKqQKLj/WBdQ/qbbzCOf38SE9IMLLc3uugZx3w0dimI9KkeDdG2t3aYeoBhzy5oNqWL/7NcLQg4DFNLLe5Wcxi5g4LJIebgSqzReFhfR/NuKZfQDI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=Y0jvKmOo; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="Y0jvKmOo" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57DC2bkI016022; Wed, 13 Aug 2025 17:08:30 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=s4+QvAyAU4T2LjJZy CrXzDCEwplunuIPbU5pr6wVxMw=; b=Y0jvKmOoFhzALRfQWmF2jGT7/zP9B6rcj j4yd2VsKDgaIWI1baIOAENCARJyq96ReE/lhTZBQ6Z11pfl0mIkDbGOKnGuGMv6r 5h95C1aX7jlHWPUJ49zUEXg2Sxd9tZnJjelkKRbNYI723SMujF3HnUzOhbEENSw8 GcfjQR1Q4uqREmUJv1eWTr5mZFvpjsYW7tVxYlXB3Ck04VsLDKDA0c/Dbj/sTVlT zjBkpFcbFlJbLAwZDSTRecX8gN7tgpmIDFGPot9zu+jKuz2WMOPn+wdecxVD9kU8 CRY0UbGFaFx5XDEWfsrghY87k5xTNiPN/7OWSriJL+JoI13zcDPIA== Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48dx14nr0n-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 13 Aug 2025 17:08:30 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57DFWO34026270; Wed, 13 Aug 2025 17:08:28 GMT Received: from smtprelay05.wdc07v.mail.ibm.com ([172.16.1.72]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 48eh218ej5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 13 Aug 2025 17:08:28 +0000 Received: from smtpav03.wdc07v.mail.ibm.com (smtpav03.wdc07v.mail.ibm.com [10.39.53.230]) by smtprelay05.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57DH8RTm6619648 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 13 Aug 2025 17:08:27 GMT Received: from smtpav03.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8503F5805F; Wed, 13 Aug 2025 17:08:27 +0000 (GMT) Received: from smtpav03.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AC7BB58054; Wed, 13 Aug 2025 17:08:26 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.255.61]) by smtpav03.wdc07v.mail.ibm.com (Postfix) with ESMTP; Wed, 13 Aug 2025 17:08:26 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: schnelle@linux.ibm.com, mjrosato@linux.ibm.com, alifm@linux.ibm.com, alex.williamson@redhat.com Subject: [PATCH v1 4/6] vfio-pci/zdev: Setup a zpci memory region for error information Date: Wed, 13 Aug 2025 10:08:18 -0700 Message-ID: <20250813170821.1115-5-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250813170821.1115-1-alifm@linux.ibm.com> References: <20250813170821.1115-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: J8QleAuRj_xwqWXR5g6_Qvacnq8rWzrI X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODEyMDIyNCBTYWx0ZWRfX7R3ik6QwyMcK LFRSS1427xSogpnLpPF/n3gX6nbqQpQHNqjVTwciWRp2UEbOF7BLINnDGnGGOTRq9zRfsG2Xugz dvRq86HBW3eXsaCGxudQWtixpCZzz87GnLocLtcrhw31cSNhMsC2Exl9Mc99oYJH/Roahdu2shl Oov4lHKMPgp/ye8FBI412VWMFlsMvaUQntWupxAMBt9iVhIW3gvAiNExKdp0eg0Q1FJAnNkqGgb FfJFgbBUIp6ihJakFS93qRg0jwJC0u1qbG9SvcwYWR+0uamp1XGR5M4Zm84EJoDsrx2QBxoGjaK Sda8jxwR5UcJxcw27a1vN56IpBGXf5XlsrpnACzJygD2mFvsq+exQUxf4sniTk2Cw4mChnKl+aT pPjMTiJw X-Proofpoint-GUID: J8QleAuRj_xwqWXR5g6_Qvacnq8rWzrI X-Authority-Analysis: v=2.4 cv=fLg53Yae c=1 sm=1 tr=0 ts=689cc68e cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=2OwXVqhp2XgA:10 a=VnNF1IyMAAAA:8 a=F78Kq7YD1iUH8MtYH0wA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-13_01,2025-08-11_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 adultscore=0 clxscore=1011 spamscore=0 priorityscore=1501 impostorscore=0 phishscore=0 malwarescore=0 bulkscore=0 suspectscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2508120224 Content-Type: text/plain; charset="utf-8" For zpci devices, we have platform specific error information. To enable recovery for passthrough devices, we want to expose this error information to user space. We can expose this information via a device specific (read only) memory region. Signed-off-by: Farhan Ali --- drivers/vfio/pci/vfio_pci_zdev.c | 76 ++++++++++++++++++++++++++++++++ include/uapi/linux/vfio.h | 2 + include/uapi/linux/vfio_zdev.h | 5 +++ 3 files changed, 83 insertions(+) diff --git a/drivers/vfio/pci/vfio_pci_zdev.c b/drivers/vfio/pci/vfio_pci_z= dev.c index 2be37eab9279..818235b28caa 100644 --- a/drivers/vfio/pci/vfio_pci_zdev.c +++ b/drivers/vfio/pci/vfio_pci_zdev.c @@ -141,15 +141,91 @@ int vfio_pci_info_zdev_add_caps(struct vfio_pci_core_= device *vdev, return ret; } =20 +static ssize_t vfio_pci_zdev_read_err_region(struct vfio_pci_core_device *= vdev, + char __user *buf, size_t count, + loff_t *ppos, bool iswrite) +{ + struct zpci_dev *zdev =3D to_zpci(vdev->pdev); + struct zpci_ccdf_err err; + struct vfio_device_zpci_err_region *region; + unsigned int i =3D VFIO_PCI_OFFSET_TO_INDEX(*ppos) - VFIO_PCI_NUM_REGIONS; + int head =3D 0; + loff_t pos =3D *ppos & VFIO_PCI_OFFSET_MASK; + + region =3D vdev->region[i].data; + + if (!zdev) + return -ENODEV; + + if (pos + count > vdev->region[i].size || iswrite) + return -EINVAL; + + mutex_lock(&zdev->pending_errs_lock); + if (zdev->pending_errs.count) { + head =3D zdev->pending_errs.head % ZPCI_ERR_PENDING_MAX; + err =3D zdev->pending_errs.err[head]; + region->pec =3D err.pec; + zdev->pending_errs.head++; + zdev->pending_errs.count--; + region->pending_errors =3D zdev->pending_errs.count; + } + mutex_unlock(&zdev->pending_errs_lock); + + if (copy_to_user(buf, (void *)region + pos, count)) + count =3D -EFAULT; + + return count; +} + +static void vfio_pci_zdev_release_err_region(struct vfio_pci_core_device *= vdev, + struct vfio_pci_region *region) +{ + struct vfio_device_zpci_err_region *err_region =3D region->data; + + kfree(err_region); +} + +static const struct vfio_pci_regops vfio_pci_zdev_err_regops =3D { + .rw =3D vfio_pci_zdev_read_err_region, + .release =3D vfio_pci_zdev_release_err_region +}; + +static int vfio_pci_zdev_setup_err_region(struct vfio_pci_core_device *vde= v) +{ + struct vfio_device_zpci_err_region *region; + int ret =3D 0; + + region =3D kzalloc(sizeof(*region), GFP_KERNEL); + if (!region) + return -ENOMEM; + + ret =3D vfio_pci_core_register_dev_region(vdev, + PCI_VENDOR_ID_IBM | VFIO_REGION_TYPE_PCI_VENDOR_TYPE, + VFIO_REGION_SUBTYPE_IBM_ZPCI_ERROR_REGION, + &vfio_pci_zdev_err_regops, + sizeof(*region), VFIO_REGION_INFO_FLAG_READ, region); + + if (ret) + kfree(region); + + + return ret; +} + int vfio_pci_zdev_open_device(struct vfio_pci_core_device *vdev) { struct zpci_dev *zdev =3D to_zpci(vdev->pdev); + int ret; =20 if (!zdev) return -ENODEV; =20 zdev->mediated_recovery =3D true; =20 + ret =3D vfio_pci_zdev_setup_err_region(vdev); + if (ret) + return ret; + if (!vdev->vdev.kvm) return 0; =20 diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h index 75100bf009ba..452b87f3672e 100644 --- a/include/uapi/linux/vfio.h +++ b/include/uapi/linux/vfio.h @@ -369,6 +369,8 @@ struct vfio_region_info_cap_type { */ #define VFIO_REGION_SUBTYPE_IBM_NVLINK2_ATSD (1) =20 +#define VFIO_REGION_SUBTYPE_IBM_ZPCI_ERROR_REGION (2) + /* sub-types for VFIO_REGION_TYPE_GFX */ #define VFIO_REGION_SUBTYPE_GFX_EDID (1) =20 diff --git a/include/uapi/linux/vfio_zdev.h b/include/uapi/linux/vfio_zdev.h index 77f2aff1f27e..bcd06f334a42 100644 --- a/include/uapi/linux/vfio_zdev.h +++ b/include/uapi/linux/vfio_zdev.h @@ -82,4 +82,9 @@ struct vfio_device_info_cap_zpci_pfip { __u8 pfip[]; }; =20 +struct vfio_device_zpci_err_region { + __u16 pec; + int pending_errors; +}; + #endif --=20 2.43.0 From nobody Sat Oct 4 19:15:06 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E7F222E5B0D; Wed, 13 Aug 2025 17:08:32 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755104914; cv=none; b=mIVfzJ2iFvHxCmxqPHp/N6LRudSCk5Rf0BRwyqKWMnP4bmegUQlinBqtYHEdhD2iB/DuDci8uuj0UsCTbjnFIRwOiN+S1zYCMIoRuba9MogfvJwKOSMoDbKSPyTdJIRtBZi8WwpQxcRMpyjUreyI0k3Fm7q0LBpFkpSGZ4csZN4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755104914; c=relaxed/simple; bh=3T7kuQCYzVkhAWp2iffagprx/aWKjjw5I1uGWW0oPrI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=RpGE9e1uDGEzeX+ws/AmbT/HRevxooP/2sP3catD4LrZ7d/S9nE37HmPfJ19UAq883RkoqkJh4OftaalyGSI0vvp4wAK9Y77vl0np2/7h9qMrPIixGpojVDUy5klfPpEloTLnQN1gZiM+jMoQJm3HKhNSZIQJ8163D/J9D7U94w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=mvsBEYhM; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="mvsBEYhM" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57DCeGP1015905; Wed, 13 Aug 2025 17:08:31 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=6I9xSlQZgHAPcXdDL CXKu6L5Y+dGMZZl+x9bmedetxw=; b=mvsBEYhMOf5xzYSM4E/NCJFosy6WUp/g8 Wy+hgX1TyNGz8qrzYafsY8ZIQi4Rj3zIeJFL52hZRHVZVCJdIlNfDUQSvZtoHvXJ a3A9EXqjJmynCy4yY4XAUXll5sMi/D3W6Ebax0Kb4i8RDNYs+EhT9Us2hVR/ytzW B3ie2lIBi0Gc8Eo1crBDaZTpbNVE8iZOpmim5pOeJe06Limra6uHDZygyel7Alfd qnncKUqS9b8sVYkDrk23cOV7rpXYLxNRIUO5rwlQAw8fIxmqQmHu+PBYaYkngioe 8MSWeDFiL/gzVFXetCd1QGrSQ8tsXb5Fc84X3x/5BG0IlikQkYoMA== Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48dvrp5hqg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 13 Aug 2025 17:08:30 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57DG5hin028629; Wed, 13 Aug 2025 17:08:30 GMT Received: from smtprelay06.wdc07v.mail.ibm.com ([172.16.1.73]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 48ej5n871x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 13 Aug 2025 17:08:30 +0000 Received: from smtpav03.wdc07v.mail.ibm.com (smtpav03.wdc07v.mail.ibm.com [10.39.53.230]) by smtprelay06.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57DH8SjO5571164 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 13 Aug 2025 17:08:29 GMT Received: from smtpav03.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D91CC58054; Wed, 13 Aug 2025 17:08:28 +0000 (GMT) Received: from smtpav03.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B183958066; Wed, 13 Aug 2025 17:08:27 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.255.61]) by smtpav03.wdc07v.mail.ibm.com (Postfix) with ESMTP; Wed, 13 Aug 2025 17:08:27 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: schnelle@linux.ibm.com, mjrosato@linux.ibm.com, alifm@linux.ibm.com, alex.williamson@redhat.com Subject: [PATCH v1 5/6] vfio-pci/zdev: Perform platform specific function reset for zPCI Date: Wed, 13 Aug 2025 10:08:19 -0700 Message-ID: <20250813170821.1115-6-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250813170821.1115-1-alifm@linux.ibm.com> References: <20250813170821.1115-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODEyMDIxOSBTYWx0ZWRfXzfOY+vlojcEJ 4PN57fdy4ybEZdBQJRmY6a3Cp//Vh006IGyPwGOQMmR0tfZ70wp/c/wjkPqY5JXWlM+nnVxA0oD 7PIlIpCb0zWV4JC8NwXgs3cWbQiudfLYnjD1NaYM//MIeqHSY5MtxORIkdf6gkUYhGXyhQk1MDo a+O1WVeq0yK4YBK9pOEi87hDAEZSuMYM9j2vlIPEknN19Stfv/URpPOiFglf3MsyI2rB++1nofX 0Z+PnBb8CdykavMy3vdwXYpm2tvuBsQ2KEeEPGu50H1QExTa0D6YM2n8+QrNDBDNPyull9xO8Kr 6IMLTMAuh4WQa9q304ljiq3PvQH7caZBqrY0KPBp6hiZytEXywo/19bHXMEGduxVAgV/1uYTp0w XIsjB1PV X-Authority-Analysis: v=2.4 cv=GrpC+l1C c=1 sm=1 tr=0 ts=689cc68e cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=2OwXVqhp2XgA:10 a=VnNF1IyMAAAA:8 a=JRqH6569gI3gnDwHsrMA:9 X-Proofpoint-GUID: rxwv5PeXgM7_w2I8l-of36dC8Ju2r9DO X-Proofpoint-ORIG-GUID: rxwv5PeXgM7_w2I8l-of36dC8Ju2r9DO X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-13_01,2025-08-11_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1011 adultscore=0 spamscore=0 impostorscore=0 suspectscore=0 phishscore=0 bulkscore=0 priorityscore=1501 malwarescore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2508120219 Content-Type: text/plain; charset="utf-8" For zPCI devices we should drive a platform specific function reset as part of VFIO_DEVICE_RESET. This reset is needed recover a zPCI device in error state. Signed-off-by: Farhan Ali --- arch/s390/pci/pci.c | 1 + drivers/vfio/pci/vfio_pci_core.c | 4 ++++ drivers/vfio/pci/vfio_pci_priv.h | 5 ++++ drivers/vfio/pci/vfio_pci_zdev.c | 39 ++++++++++++++++++++++++++++++++ 4 files changed, 49 insertions(+) diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c index f795e05b5001..860a64993b58 100644 --- a/arch/s390/pci/pci.c +++ b/arch/s390/pci/pci.c @@ -788,6 +788,7 @@ int zpci_hot_reset_device(struct zpci_dev *zdev) =20 return rc; } +EXPORT_SYMBOL_GPL(zpci_hot_reset_device); =20 /** * zpci_create_device() - Create a new zpci_dev and add it to the zbus diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index 7dcf5439dedc..7220a22135a9 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -1227,6 +1227,10 @@ static int vfio_pci_ioctl_reset(struct vfio_pci_core= _device *vdev, */ vfio_pci_set_power_state(vdev, PCI_D0); =20 + ret =3D vfio_pci_zdev_reset(vdev); + if (ret && ret !=3D -ENODEV) + return ret; + ret =3D pci_try_reset_function(vdev->pdev); up_write(&vdev->memory_lock); =20 diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_p= riv.h index a9972eacb293..5288577b3170 100644 --- a/drivers/vfio/pci/vfio_pci_priv.h +++ b/drivers/vfio/pci/vfio_pci_priv.h @@ -86,6 +86,7 @@ int vfio_pci_info_zdev_add_caps(struct vfio_pci_core_devi= ce *vdev, struct vfio_info_cap *caps); int vfio_pci_zdev_open_device(struct vfio_pci_core_device *vdev); void vfio_pci_zdev_close_device(struct vfio_pci_core_device *vdev); +int vfio_pci_zdev_reset(struct vfio_pci_core_device *vdev); #else static inline int vfio_pci_info_zdev_add_caps(struct vfio_pci_core_device = *vdev, struct vfio_info_cap *caps) @@ -100,6 +101,10 @@ static inline int vfio_pci_zdev_open_device(struct vfi= o_pci_core_device *vdev) =20 static inline void vfio_pci_zdev_close_device(struct vfio_pci_core_device = *vdev) {} +int vfio_pci_zdev_reset(struct vfio_pci_core_device *vdev) +{ + return -ENODEV; +} #endif =20 static inline bool vfio_pci_is_vga(struct pci_dev *pdev) diff --git a/drivers/vfio/pci/vfio_pci_zdev.c b/drivers/vfio/pci/vfio_pci_z= dev.c index 818235b28caa..dd1919ccb3be 100644 --- a/drivers/vfio/pci/vfio_pci_zdev.c +++ b/drivers/vfio/pci/vfio_pci_zdev.c @@ -212,6 +212,45 @@ static int vfio_pci_zdev_setup_err_region(struct vfio_= pci_core_device *vdev) return ret; } =20 +int vfio_pci_zdev_reset(struct vfio_pci_core_device *vdev) +{ + struct zpci_dev *zdev =3D to_zpci(vdev->pdev); + int rc =3D -EIO; + + if (!zdev) + return -ENODEV; + + /* + * If we can't get the zdev->state_lock the device state is + * currently undergoing a transition and we bail out - just + * the same as if the device's state is not configured at all. + */ + if (!mutex_trylock(&zdev->state_lock)) + return rc; + + /* We can reset only if the function is configured */ + if (zdev->state !=3D ZPCI_FN_STATE_CONFIGURED) + goto out; + + rc =3D zpci_hot_reset_device(zdev); + if (rc !=3D 0) + goto out; + + if (!vdev->pci_saved_state) { + pci_err(vdev->pdev, "No saved available for the device"); + rc =3D -EIO; + goto out; + } + + pci_dev_lock(vdev->pdev); + pci_load_saved_state(vdev->pdev, vdev->pci_saved_state); + pci_restore_state(vdev->pdev); + pci_dev_unlock(vdev->pdev); +out: + mutex_unlock(&zdev->state_lock); + return rc; +} + int vfio_pci_zdev_open_device(struct vfio_pci_core_device *vdev) { struct zpci_dev *zdev =3D to_zpci(vdev->pdev); --=20 2.43.0 From nobody Sat Oct 4 19:15:06 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 346D9303C95; Wed, 13 Aug 2025 17:08:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755104916; cv=none; b=BuruKfqGoKR8/aDAdPsjVRZCBQtG5WZZuveDwdDYv6G1nWc49jR13XBkJL5lqN2A5zm0QWB5c3HXQbIgLH2YKDThUVW3qLjHII3T2LitEWd2dnipqLP5UOAVVJlWgC2S8ft+5MkDINkSFK2MWMPQlICWtip2NWNgLDR41CsS1kk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755104916; c=relaxed/simple; bh=xJeRrWUQX76AoRMTTqmxE3aohZSDfsAJ1LOzJ/NRqlo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DR7FGp2dvLS+7fH7xnZV0Wdkt3hrRJcF9daTeI1gdXO3zjCfgbMuLgOE2BjvbuH0uFMtfe5HrWCSrZ4COoUw7yW11ZN+/GZ0SUCl2k2VVClrwXxXtzsa4sl6sfmsCIagm7nwhyZhgl8GWvSz/QWY3fuXyeyb6veC+cM7KV7BlLg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=b0Fp/qNT; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="b0Fp/qNT" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57DBFb24025093; Wed, 13 Aug 2025 17:08:32 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=++097ZMnQ+6MhsbVK 0nfUKxQ9lpRaw6uDyOGWFXg2r8=; b=b0Fp/qNTToGs0m1odCWdUiCE3QC3/Llo9 Tppp+iPCMURVyjg1CmYsvOQrpzJMccCEceOOZeOT+BQ3+tqY2mJI7B6c42rxg4Da eEPVs2SgHKX2TcRbDMzWIBB6cNYBhU/NNfyBF1zuBILdJBS5CFwEi0yJdBOuzZR0 VwPluwZmSLg+LJBQesdaWwiw04peXYN2FwvTvKCKGbHflACp2ZRAFYmgJRSgFyyJ YzOl0Bono6YR93FG/osdtyF69OAERzNaUaS4a1+WXka39h2iLeb2iWrkTZFlK7nE rFpgw5srTs8xC2gfNRLV5n78ipruoH4qN7ND3qwIUz2OAwNnFqOrg== Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48ehaaa0ry-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 13 Aug 2025 17:08:32 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57DGKolU028647; Wed, 13 Aug 2025 17:08:31 GMT Received: from smtprelay07.wdc07v.mail.ibm.com ([172.16.1.74]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 48ej5n8720-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 13 Aug 2025 17:08:31 +0000 Received: from smtpav03.wdc07v.mail.ibm.com (smtpav03.wdc07v.mail.ibm.com [10.39.53.230]) by smtprelay07.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57DH8TFx18547278 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 13 Aug 2025 17:08:30 GMT Received: from smtpav03.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E03C75805D; Wed, 13 Aug 2025 17:08:29 +0000 (GMT) Received: from smtpav03.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 10CCD5805A; Wed, 13 Aug 2025 17:08:29 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.255.61]) by smtpav03.wdc07v.mail.ibm.com (Postfix) with ESMTP; Wed, 13 Aug 2025 17:08:28 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: schnelle@linux.ibm.com, mjrosato@linux.ibm.com, alifm@linux.ibm.com, alex.williamson@redhat.com Subject: [PATCH v1 6/6] vfio: Allow error notification and recovery for ISM device Date: Wed, 13 Aug 2025 10:08:20 -0700 Message-ID: <20250813170821.1115-7-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250813170821.1115-1-alifm@linux.ibm.com> References: <20250813170821.1115-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=KPRaDEFo c=1 sm=1 tr=0 ts=689cc690 cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=2OwXVqhp2XgA:10 a=VnNF1IyMAAAA:8 a=J_xkUs42lnTNXtJNwgwA:9 X-Proofpoint-ORIG-GUID: kTGjPGO53TDTmXq7EoQmN7QUWz69COI6 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODEyMDIyNCBTYWx0ZWRfX18FvEKhl1O2Q 9oWjALYTTUhlauptBeuq/uT3sRfe/tFF2zYUOp4WgerJH7eYBZXz7A+vKBfuhSlQ61bVHfOdWaa ELtAwTFLId85SCqxy5C9Hs/KJmEPVBh7UyvaxbiYQyp0j5nCCWflMNk/RwgahcVANmiWBaoXTpp eZx7lCJIXIMN5DwrYeI/0sSJXs0CxYJdfnWdUFFSEGGKt6CnoFTDDe/16qzaF+qhMgrM0eBYWJb GOqLLG2IEpei40m31StbC2xHw3kjCRjkYdowZELdeYrI54iOZ7udfyvemz6rxS0avDXnXfnnhyM +kiOOYauStUWW2F4/kBGDXN6oSdKDqzVLNYO9t6r6MwrioDOVsUJNHiBbX0DaKP0iIcpqXIgpfb 2axUG9SQ X-Proofpoint-GUID: kTGjPGO53TDTmXq7EoQmN7QUWz69COI6 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-13_01,2025-08-11_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 impostorscore=0 bulkscore=0 adultscore=0 priorityscore=1501 malwarescore=0 spamscore=0 clxscore=1015 suspectscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2508120224 Content-Type: text/plain; charset="utf-8" VFIO allows error recovery and notification for devices that are PCIe (and thus AER) capable. But for PCI devices on IBM s390 error recovery involves platform firmware and notification to operating system is done by architecture specific way. The Internal Shared Memory(ISM) device is a legacy PCI device (so not PCIe capable), but can still be recovered when notified of an error. Relax the PCIe only requirement for ISM devices, so passthrough ISM devices can be notified and recovered on error. Signed-off-by: Farhan Ali --- drivers/vfio/pci/vfio_pci_core.c | 18 ++++++++++++++++-- drivers/vfio/pci/vfio_pci_intrs.c | 2 +- drivers/vfio/pci/vfio_pci_priv.h | 3 +++ 3 files changed, 20 insertions(+), 3 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index 7220a22135a9..1faab80139c6 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -723,6 +723,20 @@ void vfio_pci_core_finish_enable(struct vfio_pci_core_= device *vdev) } EXPORT_SYMBOL_GPL(vfio_pci_core_finish_enable); =20 +bool vfio_pci_device_can_recover(struct vfio_pci_core_device *vdev) +{ + struct pci_dev *pdev =3D vdev->pdev; + + if (pci_is_pcie(pdev)) + return true; + + if (pdev->vendor =3D=3D PCI_VENDOR_ID_IBM && + pdev->device =3D=3D PCI_DEVICE_ID_IBM_ISM) + return true; + + return false; +} + static int vfio_pci_get_irq_count(struct vfio_pci_core_device *vdev, int i= rq_type) { if (irq_type =3D=3D VFIO_PCI_INTX_IRQ_INDEX) { @@ -749,7 +763,7 @@ static int vfio_pci_get_irq_count(struct vfio_pci_core_= device *vdev, int irq_typ return (flags & PCI_MSIX_FLAGS_QSIZE) + 1; } } else if (irq_type =3D=3D VFIO_PCI_ERR_IRQ_INDEX) { - if (pci_is_pcie(vdev->pdev)) + if (vfio_pci_device_can_recover(vdev)) return 1; } else if (irq_type =3D=3D VFIO_PCI_REQ_IRQ_INDEX) { return 1; @@ -1150,7 +1164,7 @@ static int vfio_pci_ioctl_get_irq_info(struct vfio_pc= i_core_device *vdev, case VFIO_PCI_REQ_IRQ_INDEX: break; case VFIO_PCI_ERR_IRQ_INDEX: - if (pci_is_pcie(vdev->pdev)) + if (vfio_pci_device_can_recover(vdev)) break; fallthrough; default: diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_= intrs.c index 123298a4dc8f..f5384086ac45 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -838,7 +838,7 @@ int vfio_pci_set_irqs_ioctl(struct vfio_pci_core_device= *vdev, uint32_t flags, case VFIO_PCI_ERR_IRQ_INDEX: switch (flags & VFIO_IRQ_SET_ACTION_TYPE_MASK) { case VFIO_IRQ_SET_ACTION_TRIGGER: - if (pci_is_pcie(vdev->pdev)) + if (vfio_pci_device_can_recover(vdev)) func =3D vfio_pci_set_err_trigger; break; } diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_p= riv.h index 5288577b3170..93c1e29fbbbb 100644 --- a/drivers/vfio/pci/vfio_pci_priv.h +++ b/drivers/vfio/pci/vfio_pci_priv.h @@ -36,6 +36,9 @@ ssize_t vfio_pci_config_rw(struct vfio_pci_core_device *v= dev, char __user *buf, ssize_t vfio_pci_bar_rw(struct vfio_pci_core_device *vdev, char __user *bu= f, size_t count, loff_t *ppos, bool iswrite); =20 +bool vfio_pci_device_can_recover(struct vfio_pci_core_device *vdev); + + #ifdef CONFIG_VFIO_PCI_VGA ssize_t vfio_pci_vga_rw(struct vfio_pci_core_device *vdev, char __user *bu= f, size_t count, loff_t *ppos, bool iswrite); --=20 2.43.0