From nobody Thu Oct 2 00:49:25 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 55521315D4D; Wed, 24 Sep 2025 17:16:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758734202; cv=none; b=L5js0ysJ4z0RhDMQ+aDFGdMGQbp8+LEDJvG+JYTckPR9e46IHj4/cuSsia/6l562u5GDWp57KIvwZbfAYhK9rNWowGbEIPkPRVYkLwlIk0tLZF+lMpm5peA4uJtdsnBrsRt77JoY8qPrIlC6MNjQCckKArzNGKJJS0nKmoa16t4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758734202; c=relaxed/simple; bh=8h6W/NxGRfhYZPUFoRt1M1HE4BoDsDTuTn4z2vyWDiE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Gk6yLezliv5u1hx8P/3n7+FsdBgR460+t5pYAFUbZ00Ffdgsz1BfkCawP6yNf+76FSJ/Kjws4liYNUgOH2jswzqXMqwXpmpfanHFoF6YUFu78Aevb2LXmBJyJ/THc+E5ADSshfCJLqAE5Ki9MaL/ixDI+7DnqRuxyLUp79yWS7o= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=AJ6hpUAr; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="AJ6hpUAr" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58O8VVXd009954; Wed, 24 Sep 2025 17:16:36 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=V4Gf2KInXPKS1c8fY bgOafCGDztl/YMJ+4YMF+tdaRY=; b=AJ6hpUArXsLFVLMoxctmcIHKUBjGGoLhb x7jDV9ITTZatI7WRYy1C1ehjKudBF7H/aQHpWn/DdTW+a8B3m1dzZ+KYx70Y9hWh 6uQU6NalhjooGCO0lua2mAtxacpLflBabLV5c7qwLdV+2YRPijFk5/reGDZ+wiI5 BLJ5wMq9SpegmKDaUBxMJnc+PsqCoBPZTPK5Qc075XN5lG4ajhB6Hw74CvU1zbjK peCYYRNAo2D1oy9Yo+VYwIDzV0GCRb+TRlYgfRQl2mF/zl5Aps6q7oF8Xv/K/wtg SoCGCGD9+07xJxsgk0wjt96ym+BHIHF3cbv+3ekhCCCBYEbpTRhmg== Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 499n0jru54-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 24 Sep 2025 17:16:35 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58OETxsF013599; Wed, 24 Sep 2025 17:16:34 GMT Received: from smtprelay06.dal12v.mail.ibm.com ([172.16.1.8]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 49a8tjhgxh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 24 Sep 2025 17:16:34 +0000 Received: from smtpav03.dal12v.mail.ibm.com (smtpav03.dal12v.mail.ibm.com [10.241.53.102]) by smtprelay06.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58OHGXUs21692972 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Sep 2025 17:16:33 GMT Received: from smtpav03.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id CFBE25803F; Wed, 24 Sep 2025 17:16:33 +0000 (GMT) Received: from smtpav03.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1EC635805A; Wed, 24 Sep 2025 17:16:33 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.252.148]) by smtpav03.dal12v.mail.ibm.com (Postfix) with ESMTP; Wed, 24 Sep 2025 17:16:33 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, clg@redhat.com, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v4 01/10] PCI: Avoid saving error values for config space Date: Wed, 24 Sep 2025 10:16:19 -0700 Message-ID: <20250924171628.826-2-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250924171628.826-1-alifm@linux.ibm.com> References: <20250924171628.826-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTIwMDAzMyBTYWx0ZWRfX/sdkH6OHo2PC cK86U1Cr/hcZH+xtfLg1WIo0XGmLpNHs//6lflMaETDjnSxhyifawrfPN/Dphcn95iczfe6A384 idi7sFyZi59oYGCMg2yJsEpZGZojzTaTXe2a9ZiGg2jRqHS2VPX4gdiWOclGBcCMoRM2sKqln8U QJHPai/ehrfeYaS9au9IiDuxDT8xKBEELr30/pqXc42IjRVDuWVkE6VIK7Zw+SXRJfl7Ls08oCl mfQ31VUetYUsBDHUlrcJAFjy1m1nltDIeGmudcQrqITC98UAC6qn2DOvwHbzrW/x+0lbkBaZwyY F3ZcnsZQF9x4IcJl38G1m+1CMRMiDP6SfBUfzpPIcSa5curZlxGI/bK8LaUce/v18qo48RYPOce V37wqqzz X-Authority-Analysis: v=2.4 cv=TOlFS0la c=1 sm=1 tr=0 ts=68d42773 cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=yJojWOMRYYMA:10 a=VnNF1IyMAAAA:8 a=HJkUYhVlXSnv9HZcNckA:9 X-Proofpoint-ORIG-GUID: 31nBXQQjTugvPz0NcQR0ocnXgWhWt-JT X-Proofpoint-GUID: 31nBXQQjTugvPz0NcQR0ocnXgWhWt-JT X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-24_04,2025-09-24_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 priorityscore=1501 phishscore=0 impostorscore=0 adultscore=0 suspectscore=0 spamscore=0 bulkscore=0 malwarescore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509200033 Content-Type: text/plain; charset="utf-8" The current reset process saves the device's config space state before reset and restores it afterward. However, when a device is in an error state before reset, config space reads may return error values instead of valid data. This results in saving corrupted values that get written back to the device during state restoration. Avoid saving the state of the config space when the device is in error. While restoring we only restore the state that can be restored through kernel data such as BARs or doesn't depend on the saved state. Signed-off-by: Farhan Ali --- drivers/pci/pci.c | 25 ++++++++++++++++++++++--- drivers/pci/pcie/aer.c | 3 +++ drivers/pci/pcie/dpc.c | 3 +++ drivers/pci/pcie/ptm.c | 3 +++ drivers/pci/tph.c | 3 +++ drivers/pci/vc.c | 3 +++ 6 files changed, 37 insertions(+), 3 deletions(-) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index b0f4d98036cd..a3d93d1baee7 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -1720,6 +1720,9 @@ static void pci_restore_pcie_state(struct pci_dev *de= v) struct pci_cap_saved_state *save_state; u16 *cap; =20 + if (!dev->state_saved) + return; + /* * Restore max latencies (in the LTR capability) before enabling * LTR itself in PCI_EXP_DEVCTL2. @@ -1775,6 +1778,9 @@ static void pci_restore_pcix_state(struct pci_dev *de= v) struct pci_cap_saved_state *save_state; u16 *cap; =20 + if (!dev->state_saved) + return; + save_state =3D pci_find_saved_cap(dev, PCI_CAP_ID_PCIX); pos =3D pci_find_capability(dev, PCI_CAP_ID_PCIX); if (!save_state || !pos) @@ -1792,6 +1798,14 @@ static void pci_restore_pcix_state(struct pci_dev *d= ev) int pci_save_state(struct pci_dev *dev) { int i; + u32 val; + + pci_read_config_dword(dev, PCI_COMMAND, &val); + if (PCI_POSSIBLE_ERROR(val)) { + pci_warn(dev, "Device config space inaccessible, will only be partially = restored\n"); + return -EIO; + } + /* XXX: 100% dword access ok here? */ for (i =3D 0; i < 16; i++) { pci_read_config_dword(dev, i * 4, &dev->saved_config_space[i]); @@ -1854,6 +1868,14 @@ static void pci_restore_config_space_range(struct pc= i_dev *pdev, =20 static void pci_restore_config_space(struct pci_dev *pdev) { + if (!pdev->state_saved) { + pci_warn(pdev, "No saved config space, restoring BARs\n"); + pci_restore_bars(pdev); + pci_write_config_word(pdev, PCI_COMMAND, + PCI_COMMAND_MEMORY | PCI_COMMAND_IO); + return; + } + if (pdev->hdr_type =3D=3D PCI_HEADER_TYPE_NORMAL) { pci_restore_config_space_range(pdev, 10, 15, 0, false); /* Restore BARs before the command register. */ @@ -1906,9 +1928,6 @@ static void pci_restore_rebar_state(struct pci_dev *p= dev) */ void pci_restore_state(struct pci_dev *dev) { - if (!dev->state_saved) - return; - pci_restore_pcie_state(dev); pci_restore_pasid_state(dev); pci_restore_pri_state(dev); diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c index e286c197d716..e6023ffbf94d 100644 --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -361,6 +361,9 @@ void pci_restore_aer_state(struct pci_dev *dev) if (!aer) return; =20 + if (!dev->state_saved) + return; + save_state =3D pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_ERR); if (!save_state) return; diff --git a/drivers/pci/pcie/dpc.c b/drivers/pci/pcie/dpc.c index fc18349614d7..e0d7034c2cd8 100644 --- a/drivers/pci/pcie/dpc.c +++ b/drivers/pci/pcie/dpc.c @@ -67,6 +67,9 @@ void pci_restore_dpc_state(struct pci_dev *dev) if (!pci_is_pcie(dev)) return; =20 + if (!dev->state_saved) + return; + save_state =3D pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_DPC); if (!save_state) return; diff --git a/drivers/pci/pcie/ptm.c b/drivers/pci/pcie/ptm.c index 65e4b008be00..78613000acfb 100644 --- a/drivers/pci/pcie/ptm.c +++ b/drivers/pci/pcie/ptm.c @@ -112,6 +112,9 @@ void pci_restore_ptm_state(struct pci_dev *dev) if (!ptm) return; =20 + if (!dev->state_saved) + return; + save_state =3D pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_PTM); if (!save_state) return; diff --git a/drivers/pci/tph.c b/drivers/pci/tph.c index cc64f93709a4..c0fa1b9a7a78 100644 --- a/drivers/pci/tph.c +++ b/drivers/pci/tph.c @@ -435,6 +435,9 @@ void pci_restore_tph_state(struct pci_dev *pdev) if (!pdev->tph_enabled) return; =20 + if (!pdev->state_saved) + return; + save_state =3D pci_find_saved_ext_cap(pdev, PCI_EXT_CAP_ID_TPH); if (!save_state) return; diff --git a/drivers/pci/vc.c b/drivers/pci/vc.c index a4ff7f5f66dd..00609c7e032a 100644 --- a/drivers/pci/vc.c +++ b/drivers/pci/vc.c @@ -391,6 +391,9 @@ void pci_restore_vc_state(struct pci_dev *dev) { int i; =20 + if (!dev->state_saved) + return; + for (i =3D 0; i < ARRAY_SIZE(vc_caps); i++) { int pos; struct pci_cap_saved_state *save_state; --=20 2.43.0 From nobody Thu Oct 2 00:49:25 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C992315D52; Wed, 24 Sep 2025 17:16:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758734202; cv=none; b=EL5I/AUp8hhS4Igu4XK4CiVLc7eNmawWgSNu2Jt0t1V/RBJXMlgRsf3XsUzRMuhcd/BUClcEQBY+cT/D3Dv6Xkq883YYPrHHauF1ZhYflTE+vkD+Rha74pNBfLVP+JRL8oHsReo2NB8aTowFEUz5zueHXxJl6K9ywEVoTrbBvz8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758734202; c=relaxed/simple; bh=YrxQ0WJe9dz6q4E5eGvh+vYwNIxItsbwqGpJD/pFKTw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=F+mw1CQ1/u+jnzfg+LQWYWFpn26naEi+7MUvjXcG806zkhZwcPvM9CHYg7OKSTabW7aK5u5Zl2jtpuaLn6A/rh/cYJhACpGVF8r9XwuP0UjK2E4TsZh+qXYm15kOvz63jeImCvAayOE3VjI74NphtD0+/oqkPn8hBhZacpZrAHo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=siNqHZt8; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="siNqHZt8" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58OGDqok017273; Wed, 24 Sep 2025 17:16:37 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=AVUF0BZZovq5IEjar f6BmNZ9GqdqEzXVBXdFy793V6o=; b=siNqHZt8z1iqbCPptlxhnxn9+C8q9NSfd 2volqaDSyMjUlctbQ4aJYa50BHP/CD9Pi/v/DTtHTdAMmDLBdcTfiGp081hidB1Q QYQSZcNAq5RIWPkz6Vt6zsrkV9baJxcj/9Hr2fp91evXLERtvVr2wyF/AdrkGLVA 04mADBn6WqMFEe0s1d0T/IexcE435rLOS6MhUEjP1T5y/rrz63LI4bRibRjbONcP UiZz1+1hycC7BoJjaoo32stjHncD/ZWHR0UgQGxHxAVHgHKbUy1UUZllZgzLsIDk K6RnMfOzsB4s/+fw7cy67RalBzCHylcLLII2eLZ2vbS7guMvWZmiw== Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 499ksc11p2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 24 Sep 2025 17:16:36 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58OELggF030367; Wed, 24 Sep 2025 17:16:35 GMT Received: from smtprelay07.dal12v.mail.ibm.com ([172.16.1.9]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 49a9a19f4d-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 24 Sep 2025 17:16:35 +0000 Received: from smtpav03.dal12v.mail.ibm.com (smtpav03.dal12v.mail.ibm.com [10.241.53.102]) by smtprelay07.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58OHGYdl28508918 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Sep 2025 17:16:34 GMT Received: from smtpav03.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B53805803F; Wed, 24 Sep 2025 17:16:34 +0000 (GMT) Received: from smtpav03.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id ECDC358056; Wed, 24 Sep 2025 17:16:33 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.252.148]) by smtpav03.dal12v.mail.ibm.com (Postfix) with ESMTP; Wed, 24 Sep 2025 17:16:33 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, clg@redhat.com, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v4 02/10] PCI: Add additional checks for flr reset Date: Wed, 24 Sep 2025 10:16:20 -0700 Message-ID: <20250924171628.826-3-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250924171628.826-1-alifm@linux.ibm.com> References: <20250924171628.826-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: JKn4S1BCIprkwT72N7oGBPZJFW-37Q_t X-Proofpoint-GUID: JKn4S1BCIprkwT72N7oGBPZJFW-37Q_t X-Authority-Analysis: v=2.4 cv=SdH3duRu c=1 sm=1 tr=0 ts=68d42774 cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=yJojWOMRYYMA:10 a=VnNF1IyMAAAA:8 a=8GLQ0EcgPjaQrAEZ2hkA:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTIwMDAyMCBTYWx0ZWRfX0OdcpLij4fSa POqoYEkGOe5d/N7HWkVhHK0s465/9dBgRfK5A/B7wfoqMZPCegJOURmDDjSkdtvdNreFmGaUZFQ Rcs9L+8X6zmXuysPQjyQQyIaac2YK0EMWkG2Au8gGeQ0l0SnRLWijUsrxEC2TaW34U9Prh373H7 ta0XngsNkx5gKrzsvzaITm/cElPas8gzM546KvQiY1PowT73ylGo6CAeTmpkWtCE093rT06vaBV 50mQCDr9Qkm5dQTKZs7kY4ZZZd7MsQdOWB9X/J/cFiM/EV6aVtX/En9vyIEnKJv7G+2lk/IVEzW BXVgJbjzycLz+H6Kg/0tjji8usGeZy37vHzpJqbfT0nfg1IGdkfVWKxGXAahc9j4/8HHBcFq7kB L83Dxlv+ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-24_04,2025-09-24_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 suspectscore=0 phishscore=0 spamscore=0 bulkscore=0 priorityscore=1501 clxscore=1015 adultscore=0 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509200020 Content-Type: text/plain; charset="utf-8" If a device is in an error state, then any reads of device registers can return error value. Add addtional checks to validate if a device is in an error state before doing an flr reset. Signed-off-by: Farhan Ali Reviewed-by: Benjamin Block --- drivers/pci/pci.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index a3d93d1baee7..327fefc6a1eb 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -4576,12 +4576,19 @@ EXPORT_SYMBOL_GPL(pcie_flr); */ int pcie_reset_flr(struct pci_dev *dev, bool probe) { + u32 reg; + if (dev->dev_flags & PCI_DEV_FLAGS_NO_FLR_RESET) return -ENOTTY; =20 if (!(dev->devcap & PCI_EXP_DEVCAP_FLR)) return -ENOTTY; =20 + if (pcie_capability_read_dword(dev, PCI_EXP_DEVCAP, ®)) { + pci_warn(dev, "Device unable to do an FLR\n"); + return -ENOTTY; + } + if (probe) return 0; =20 --=20 2.43.0 From nobody Thu Oct 2 00:49:25 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A8BF33164DC; Wed, 24 Sep 2025 17:16:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758734204; cv=none; b=kzY3Ex1dw9ggMS3qIBjXBxab7ddrvAYLS81jsdXmOpl0rUrashDHFF2xxt6SLqZHvPciaQkr9iL8fzUX7WKrIlJXipVeS20Xt6JUVVqXlMKVBsfpevQ6+vj1Xi4DXBFpT8Hqa9tA7+fH8M5sRHQZmamOim0UBvweiMUKON5GP2s= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758734204; c=relaxed/simple; bh=iUFfcJwCG6FBfxs5hXpJ5ztskkQYyvOnk7gsFW82Vys=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=q6qua+zCjeG2MMiU/jHN3PujRNT8M7hS5Tn2qEQda0iP3Ms05hnCODJJt7jHbeTw1X5kzYlBMIJK/s1+xmjsfe5xRb/2uQv/VZ6KuPELTvmUgqWzx4mK0QDmEPLB1nl//5haFinvWE9whF2VYFi7Un9utKUyetMXz8SQW0yRwe4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=XyQTvwlZ; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="XyQTvwlZ" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58OGgJcU028853; Wed, 24 Sep 2025 17:16:38 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=AZlxxd1WcazOIdq8Z AxncA3t4iY+7RfS/CJBSa31jKo=; b=XyQTvwlZ7q7F3eVG2N2iEpwlHpFV/Oolr od3vqvsTsnQQ1Qlx8yl3tc6ZKUa5P3eFPbocgCR1fRcFcZaSHt7xp9eKoopxHEYD An0+BZO2Z9RS2ucPpHSteup29mI36G51rZa2+X28m4PynmBpKQ++7brAiz+0A2dG FhmuffOcjaOGznAyD/uql9PjWP+wsGWtI9hZdHwX1f40KYm96/+b9g0B2tsoWMvV 0uDnIVqAJCV/PnL/HzzLGwzeSitma67pkbVWKXWUhkj/l1v+EW3SbV+UcGYMvC9r lsUPuM1lrS0UCd46HDCxlpKjfqh8ruxyGMiLQnBTclmNwPNCaSr+A== Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 499hpqggbx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 24 Sep 2025 17:16:38 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58OE45UZ013329; Wed, 24 Sep 2025 17:16:37 GMT Received: from smtprelay01.wdc07v.mail.ibm.com ([172.16.1.68]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 49cj348u51-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 24 Sep 2025 17:16:37 +0000 Received: from smtpav03.dal12v.mail.ibm.com (smtpav03.dal12v.mail.ibm.com [10.241.53.102]) by smtprelay01.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58OHGZjn32440602 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Sep 2025 17:16:36 GMT Received: from smtpav03.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A3F575803F; Wed, 24 Sep 2025 17:16:35 +0000 (GMT) Received: from smtpav03.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D0FFF58056; Wed, 24 Sep 2025 17:16:34 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.252.148]) by smtpav03.dal12v.mail.ibm.com (Postfix) with ESMTP; Wed, 24 Sep 2025 17:16:34 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, clg@redhat.com, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com, stable@vger.kernel.org Subject: [PATCH v4 03/10] PCI: Allow per function PCI slots Date: Wed, 24 Sep 2025 10:16:21 -0700 Message-ID: <20250924171628.826-4-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250924171628.826-1-alifm@linux.ibm.com> References: <20250924171628.826-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=FrEF/3rq c=1 sm=1 tr=0 ts=68d42776 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=yJojWOMRYYMA:10 a=VwQbUJbxAAAA:8 a=VnNF1IyMAAAA:8 a=K-1LnuYzVcpd1HOvht4A:9 X-Proofpoint-ORIG-GUID: HM120vcbSZvdrKj5Wv4otrRYOMcwZwrV X-Proofpoint-GUID: HM120vcbSZvdrKj5Wv4otrRYOMcwZwrV X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTE5MDIyNCBTYWx0ZWRfXy2zHvKEOOjLx ctOAjEpa/zeaeOo5llDRtZdYjhZo8cG80sQ+unKuH5PJdFkeoEX4WhfxSgVYwgI5km72HF6rxpd 7I/6aB7jABreLVdkuhPeRrspgxR9Y93XNv20K3GTV2CLUIGETjEvPKk9ziefT8M9XMyfi7gFISR YFpiGsvrK0QfG5LWAvLgZCLby4b6+ZEglJB9VI06wZORAZe7c2unfdY4xaKPxhOFx2aGZehLWBU Hmzq8VE9LPn3RvtxDRu+AHUCzOf2kPpq5jMP46YGbp5a7rqGrXAvOxjf31lEfhICQs/Q0XHIt/y 1QpDJj1AFk0iUAfzIAe32dr2bw8K5VgEwMofOxPndz4mCIdEe1j95HVISYT6+sbqK/HSjeOjYA3 Ei9HJN2x X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-24_04,2025-09-24_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 phishscore=0 priorityscore=1501 malwarescore=0 adultscore=0 clxscore=1015 impostorscore=0 spamscore=0 suspectscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509190224 Content-Type: text/plain; charset="utf-8" On s390 systems, which use a machine level hypervisor, PCI devices are always accessed through a form of PCI pass-through which fundamentally operates on a per PCI function granularity. This is also reflected in the s390 PCI hotplug driver which creates hotplug slots for individual PCI functions. Its reset_slot() function, which is a wrapper for zpci_hot_reset_device(), thus also resets individual functions. Currently, the kernel's PCI_SLOT() macro assigns the same pci_slot object to multifunction devices. This approach worked fine on s390 systems that only exposed virtual functions as individual PCI domains to the operating system. Since commit 44510d6fa0c0 ("s390/pci: Handling multifunctions") s390 supports exposing the topology of multifunction PCI devices by grouping them in a shared PCI domain. When attempting to reset a function through the hotplug driver, the shared slot assignment causes the wrong function to be reset instead of the intended one. It also leaks memory as we do create a pci_slot object for the function, but don't correctly free it in pci_slot_release(). Add a flag for struct pci_slot to allow per function PCI slots for functions managed through a hypervisor, which exposes individual PCI functions while retaining the topology. Fixes: 44510d6fa0c0 ("s390/pci: Handling multifunctions") Cc: Suggested-by: Niklas Schnelle Signed-off-by: Farhan Ali Reviewed-by: Benjamin Block --- drivers/pci/hotplug/s390_pci_hpc.c | 10 ++++++++-- drivers/pci/pci.c | 5 +++-- drivers/pci/slot.c | 14 +++++++++++--- include/linux/pci.h | 1 + 4 files changed, 23 insertions(+), 7 deletions(-) diff --git a/drivers/pci/hotplug/s390_pci_hpc.c b/drivers/pci/hotplug/s390_= pci_hpc.c index d9996516f49e..8b547de464bf 100644 --- a/drivers/pci/hotplug/s390_pci_hpc.c +++ b/drivers/pci/hotplug/s390_pci_hpc.c @@ -126,14 +126,20 @@ static const struct hotplug_slot_ops s390_hotplug_slo= t_ops =3D { =20 int zpci_init_slot(struct zpci_dev *zdev) { + int ret; char name[SLOT_NAME_SIZE]; struct zpci_bus *zbus =3D zdev->zbus; =20 zdev->hotplug_slot.ops =3D &s390_hotplug_slot_ops; =20 snprintf(name, SLOT_NAME_SIZE, "%08x", zdev->fid); - return pci_hp_register(&zdev->hotplug_slot, zbus->bus, - zdev->devfn, name); + ret =3D pci_hp_register(&zdev->hotplug_slot, zbus->bus, + zdev->devfn, name); + if (ret) + return ret; + + zdev->hotplug_slot.pci_slot->per_func_slot =3D 1; + return 0; } =20 void zpci_exit_slot(struct zpci_dev *zdev) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 327fefc6a1eb..530a793a332c 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -5057,8 +5057,9 @@ static int pci_reset_hotplug_slot(struct hotplug_slot= *hotplug, bool probe) =20 static int pci_dev_reset_slot_function(struct pci_dev *dev, bool probe) { - if (dev->multifunction || dev->subordinate || !dev->slot || - dev->dev_flags & PCI_DEV_FLAGS_NO_BUS_RESET) + if (dev->subordinate || !dev->slot || + dev->dev_flags & PCI_DEV_FLAGS_NO_BUS_RESET || + (dev->multifunction && !dev->slot->per_func_slot)) return -ENOTTY; =20 return pci_reset_hotplug_slot(dev->slot->hotplug, probe); diff --git a/drivers/pci/slot.c b/drivers/pci/slot.c index 50fb3eb595fe..51ee59e14393 100644 --- a/drivers/pci/slot.c +++ b/drivers/pci/slot.c @@ -63,6 +63,14 @@ static ssize_t cur_speed_read_file(struct pci_slot *slot= , char *buf) return bus_speed_read(slot->bus->cur_bus_speed, buf); } =20 +static bool pci_dev_matches_slot(struct pci_dev *dev, struct pci_slot *slo= t) +{ + if (slot->per_func_slot) + return dev->devfn =3D=3D slot->number; + + return PCI_SLOT(dev->devfn) =3D=3D slot->number; +} + static void pci_slot_release(struct kobject *kobj) { struct pci_dev *dev; @@ -73,7 +81,7 @@ static void pci_slot_release(struct kobject *kobj) =20 down_read(&pci_bus_sem); list_for_each_entry(dev, &slot->bus->devices, bus_list) - if (PCI_SLOT(dev->devfn) =3D=3D slot->number) + if (pci_dev_matches_slot(dev, slot)) dev->slot =3D NULL; up_read(&pci_bus_sem); =20 @@ -166,7 +174,7 @@ void pci_dev_assign_slot(struct pci_dev *dev) =20 mutex_lock(&pci_slot_mutex); list_for_each_entry(slot, &dev->bus->slots, list) - if (PCI_SLOT(dev->devfn) =3D=3D slot->number) + if (pci_dev_matches_slot(dev, slot)) dev->slot =3D slot; mutex_unlock(&pci_slot_mutex); } @@ -285,7 +293,7 @@ struct pci_slot *pci_create_slot(struct pci_bus *parent= , int slot_nr, =20 down_read(&pci_bus_sem); list_for_each_entry(dev, &parent->devices, bus_list) - if (PCI_SLOT(dev->devfn) =3D=3D slot_nr) + if (pci_dev_matches_slot(dev, slot)) dev->slot =3D slot; up_read(&pci_bus_sem); =20 diff --git a/include/linux/pci.h b/include/linux/pci.h index 59876de13860..9265f32d9786 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -78,6 +78,7 @@ struct pci_slot { struct list_head list; /* Node in list of slots */ struct hotplug_slot *hotplug; /* Hotplug info (move here) */ unsigned char number; /* PCI_SLOT(pci_dev->devfn) */ + unsigned int per_func_slot:1; /* Allow per function slot */ struct kobject kobj; }; =20 --=20 2.43.0 From nobody Thu Oct 2 00:49:25 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D38583191D7; Wed, 24 Sep 2025 17:16:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758734205; cv=none; b=TAO+YB+EHOjUhJk/P+DEQk7hjmKiDuK6cYnUdqpNlHYOk8b3qjgurAxlgYQDF2eONO18n63Pk0GnCQiCA8BEEBgO/vIYKpELSv/teoN1Puysi2fQmW+cGbHnx5fC+5PCwbuUssQ/CHTiTTCYm3B0OKfP8Us6UXD5PUsXILiP3to= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758734205; c=relaxed/simple; bh=zVzJdkD5hMbkEcIndfLw8MCcgvxNRuXkSJNgl1ThqUs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eU7KQ5QcqBV2mgicAz/6x2TYDsmFfsGnqUqpD1EEMEz8zZguZFWSIPfCqkccjOO2uxda0T3u+PsZbbrcJ4dk1dr6Gu5uRSu1p43yDGCEojkw4Neug9ZJQF6+DDsvruXWicpYr96uKBejSkF6RJjCqJQlb1ObbvJ7dQKJ2giYfpo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=Yd5gZW0u; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="Yd5gZW0u" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58O87E4q029365; Wed, 24 Sep 2025 17:16:39 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=mgLDKn5UCmN3nh4Mj hcGpsxIqwpvAX9zKj9v0JivNUE=; b=Yd5gZW0uzF4djSgfcESWB+6nhlOB4JeSY WKO2/TXZjNHjzgROnb1JOTVAcurYfEQ9RSdQBYrzo/JiSKhUXS/jUALknPe8OaP2 +cAmNPC6gk+3YY6NqVia07kRlHufEnrxCHpqBLgehyQrfvGFk5TocyVXV+wC+5uv g/HYmd+8uzN1bKzN1D690y3mCq8yhIujPYn8yo4NWOlGBo4miW3nn6udZXoJk+o/ FUrgoiKsSq+s0Gdh+CsxO+ECm/ezQ04yJ+UGaVpOFErDdtjXdN9WgGbwxeo8h8sR QlGCSRoV/IkzM82EezPSbC8VXv2cQz3qa6CYgsr+tulitFmKEQDZA== Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 499ksc11pa-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 24 Sep 2025 17:16:39 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58OE0FQ2013602; Wed, 24 Sep 2025 17:16:38 GMT Received: from smtprelay02.wdc07v.mail.ibm.com ([172.16.1.69]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 49a8tjhgxp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 24 Sep 2025 17:16:38 +0000 Received: from smtpav03.dal12v.mail.ibm.com (smtpav03.dal12v.mail.ibm.com [10.241.53.102]) by smtprelay02.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58OHGaRl25756394 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Sep 2025 17:16:36 GMT Received: from smtpav03.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 82A7658060; Wed, 24 Sep 2025 17:16:36 +0000 (GMT) Received: from smtpav03.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C3B775805A; Wed, 24 Sep 2025 17:16:35 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.252.148]) by smtpav03.dal12v.mail.ibm.com (Postfix) with ESMTP; Wed, 24 Sep 2025 17:16:35 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, clg@redhat.com, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v4 04/10] s390/pci: Add architecture specific resource/bus address translation Date: Wed, 24 Sep 2025 10:16:22 -0700 Message-ID: <20250924171628.826-5-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250924171628.826-1-alifm@linux.ibm.com> References: <20250924171628.826-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: KfzvwJpX1zoNqV11WroA5W5P9pLzypcF X-Proofpoint-GUID: KfzvwJpX1zoNqV11WroA5W5P9pLzypcF X-Authority-Analysis: v=2.4 cv=SdH3duRu c=1 sm=1 tr=0 ts=68d42777 cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=yJojWOMRYYMA:10 a=VnNF1IyMAAAA:8 a=yYx7OUe3D3zVL1aRsq0A:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTIwMDAyMCBTYWx0ZWRfX0S/yIOP0Fekk x1pNbTdMv42O74eQu1kEtW0eUz6zITFIM+9uUEgPJr5FIip+Lzbv0Ux4Hdk9b/wvn6SOVnCmPtL LuUFcPhFVf71aOAId2E2cEhJ6kXzGsrpetrd3fVwYoCtOa7moxSTe+o4e/rG2hhIbttJknX/mi3 TkyhrEowVIOO3AMEE4miaM+os3VePrLyYoCaaCEq+x16ID8Euzkn2em6Gn6ahzcANpKBBmSm/8o ANT5qZWU6/56iQVzkDMnQ57deKCigGoQ+AAzqal8etrpz5HYHpIbOVKXc0ytED4qoilavOI05kF IydRk/tegZmYZiWxeya/DJjwYkU8y+VOZov2fmA0lwtqcR3pT3eRWSVcpPU+9A/cDhJA+zlfx2p uZo8N876 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-24_04,2025-09-24_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 suspectscore=0 phishscore=0 spamscore=0 bulkscore=0 priorityscore=1501 clxscore=1015 adultscore=0 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509200020 Content-Type: text/plain; charset="utf-8" On s390 today we overwrite the PCI BAR resource address to either an artificial cookie address or MIO address. However this address is different from the bus address of the BARs programmed by firmware. The artificial cookie address was created to index into an array of function handles (zpci_iomap_start). The MIO (mapped I/O) addresses are provided by firmware but maybe different from the bus address. This creates an issue when trying to convert the BAR resource address to bus address using the generic pcibios_resource_to_bus(). Implement an architecture specific pcibios_resource_to_bus() function to correctly translate PCI BAR resource addresses to bus addresses for s390. Similarly add architecture specific pcibios_bus_to_resource function to do the reverse translation. Signed-off-by: Farhan Ali --- arch/s390/pci/pci.c | 74 +++++++++++++++++++++++++++++++++++++++ drivers/pci/host-bridge.c | 4 +-- 2 files changed, 76 insertions(+), 2 deletions(-) diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c index cd6676c2d602..3992ba44e1cf 100644 --- a/arch/s390/pci/pci.c +++ b/arch/s390/pci/pci.c @@ -264,6 +264,80 @@ resource_size_t pcibios_align_resource(void *data, con= st struct resource *res, return 0; } =20 +void pcibios_resource_to_bus(struct pci_bus *bus, struct pci_bus_region *r= egion, + struct resource *res) +{ + struct zpci_bus *zbus =3D bus->sysdata; + struct zpci_bar_struct *zbar; + struct zpci_dev *zdev; + + region->start =3D res->start; + region->end =3D res->end; + + for (int i =3D 0; i < ZPCI_FUNCTIONS_PER_BUS; i++) { + int j =3D 0; + + zbar =3D NULL; + zdev =3D zbus->function[i]; + if (!zdev) + continue; + + for (j =3D 0; j < PCI_STD_NUM_BARS; j++) { + if (zdev->bars[j].res->start =3D=3D res->start && + zdev->bars[j].res->end =3D=3D res->end && + res->flags & IORESOURCE_MEM) { + zbar =3D &zdev->bars[j]; + break; + } + } + + if (zbar) { + /* only MMIO is supported */ + region->start =3D zbar->val & PCI_BASE_ADDRESS_MEM_MASK; + if (zbar->val & PCI_BASE_ADDRESS_MEM_TYPE_64) + region->start |=3D (u64)zdev->bars[j + 1].val << 32; + + region->end =3D region->start + (1UL << zbar->size) - 1; + return; + } + } +} + +void pcibios_bus_to_resource(struct pci_bus *bus, struct resource *res, + struct pci_bus_region *region) +{ + struct zpci_bus *zbus =3D bus->sysdata; + struct zpci_dev *zdev; + resource_size_t start, end; + + res->start =3D region->start; + res->end =3D region->end; + + for (int i =3D 0; i < ZPCI_FUNCTIONS_PER_BUS; i++) { + zdev =3D zbus->function[i]; + if (!zdev || !zdev->has_resources) + continue; + + for (int j =3D 0; j < PCI_STD_NUM_BARS; j++) { + if (!zdev->bars[j].size) + continue; + + /* only MMIO is supported */ + start =3D zdev->bars[j].val & PCI_BASE_ADDRESS_MEM_MASK; + if (zdev->bars[j].val & PCI_BASE_ADDRESS_MEM_TYPE_64) + start |=3D (u64)zdev->bars[j + 1].val << 32; + + end =3D start + (1UL << zdev->bars[j].size) - 1; + + if (start =3D=3D region->start && end =3D=3D region->end) { + res->start =3D zdev->bars[j].res->start; + res->end =3D zdev->bars[j].res->end; + return; + } + } + } +} + void __iomem *ioremap_prot(phys_addr_t phys_addr, size_t size, pgprot_t prot) { diff --git a/drivers/pci/host-bridge.c b/drivers/pci/host-bridge.c index afa50b446567..56d62afb3afe 100644 --- a/drivers/pci/host-bridge.c +++ b/drivers/pci/host-bridge.c @@ -48,7 +48,7 @@ void pci_set_host_bridge_release(struct pci_host_bridge *= bridge, } EXPORT_SYMBOL_GPL(pci_set_host_bridge_release); =20 -void pcibios_resource_to_bus(struct pci_bus *bus, struct pci_bus_region *r= egion, +void __weak pcibios_resource_to_bus(struct pci_bus *bus, struct pci_bus_re= gion *region, struct resource *res) { struct pci_host_bridge *bridge =3D pci_find_host_bridge(bus); @@ -73,7 +73,7 @@ static bool region_contains(struct pci_bus_region *region= 1, return region1->start <=3D region2->start && region1->end >=3D region2->e= nd; } =20 -void pcibios_bus_to_resource(struct pci_bus *bus, struct resource *res, +void __weak pcibios_bus_to_resource(struct pci_bus *bus, struct resource *= res, struct pci_bus_region *region) { struct pci_host_bridge *bridge =3D pci_find_host_bridge(bus); --=20 2.43.0 From nobody Thu Oct 2 00:49:25 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8615931A549; Wed, 24 Sep 2025 17:16:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758734207; cv=none; b=ViDxDzU/V3K7zcH7I0Ds01xa0LA/X0Z8QEbyYP/o5f93nX3eq3tWS0Urmobl0M3i45mGqIZGHaKHoJSBsfwSnErxxevz66Lwxn/apX0djvmTu6uIQuk5OT8oVzlSrDMxexwwPjdOOP2enZNaZ6S2qZDF1XjFi2MfEAyEBpT+gGo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758734207; c=relaxed/simple; bh=1RXASWL+I3J3u2dKE3vN+USAb0FAaslxVixQNgxLNAY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=iTC/lbR9LAKcSX9V9YLg7FRSF4BrP17fzp75EY+yU9Uws8Bpz+SMtknq7fBf6iWLu23BEd8z8+SDARM2BXAWo0iQdwJM/AYxgU7si/xQvE01EIZVnujI8u4NOL3c1D+DTHUcvBEbM/Vp/S1I2UEsXvinQWuUgBSsCiRKz/aEEIo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=mE2cliMf; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="mE2cliMf" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58OBExXn027471; Wed, 24 Sep 2025 17:16:41 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=j8MbEgg7bHm6AdxqF MAtAs4Vl5MT9RULDEAjlayd9KM=; b=mE2cliMf3svjhuLnELwCZq2bLk0qwJ3fk sVYHxhTwYytbr/er5gZyhrR5EtBvQyFdQYBy7ka4+Tf9eta6Q01qYnx2eVsTAWNX 0o2Jp5wwUxAO4YK/icnGp8geZlpxtXVo/mPq1cQzoKIlLk+VrdRhd8QZ5wwqXM79 A8oabkHYCw21rfzld3klemFmjfg9DJjaj2VJCtWIfdPSoR1ot2NyVXidA39Z7Ji+ KZ6i+0jKVM1YLUJzBv4qwmpTBK5BcDyAudO3m9EZEAII2lR7BilPn7oaEGGB88/b T05x3Smhr3CQZuTvdowYskKUeS+1QMW1Kqvurp6HpJqGTZtFC0Y6A== Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 499n0jru5j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 24 Sep 2025 17:16:41 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58OE4Z6d019671; Wed, 24 Sep 2025 17:16:38 GMT Received: from smtprelay03.wdc07v.mail.ibm.com ([172.16.1.70]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 49a83k9mk8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 24 Sep 2025 17:16:38 +0000 Received: from smtpav03.dal12v.mail.ibm.com (smtpav03.dal12v.mail.ibm.com [10.241.53.102]) by smtprelay03.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58OHGQkJ3080786 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Sep 2025 17:16:27 GMT Received: from smtpav03.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5460F5803F; Wed, 24 Sep 2025 17:16:37 +0000 (GMT) Received: from smtpav03.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9C68E58064; Wed, 24 Sep 2025 17:16:36 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.252.148]) by smtpav03.dal12v.mail.ibm.com (Postfix) with ESMTP; Wed, 24 Sep 2025 17:16:36 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, clg@redhat.com, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v4 05/10] s390/pci: Restore IRQ unconditionally for the zPCI device Date: Wed, 24 Sep 2025 10:16:23 -0700 Message-ID: <20250924171628.826-6-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250924171628.826-1-alifm@linux.ibm.com> References: <20250924171628.826-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTIwMDAzMyBTYWx0ZWRfX5WyEbxE/Wp6k 6Nnp87qg51apiXHQRL7P9wzwcpjdv+GfrYldmQuugams+LkDcCUbAP2W2dngs0HkazRbuMe0gMW lGoHwD8Dl2BAMyVXLuo5NTAW/DXb+oDWYxD7ENzld+b2wxgJSfGcVkMPQLjsRzzhbxKstdSzgJN Ear775yoLZFtIVnMAbIzNYZB/DzJFnhsloPGiMTr10O0GMC6QH7mHeSxadNO3yhB3hJ5fKFipkE aGNrXJLYK3VyJUmFOBFNhlx8L63xg4xNUqbm+Dj3n0OiDxqEmGJjEmBQu8U3eQbwCxECdseQILP pw6Q7ud9WARQIQ+39jB3k4Ij/IpalpaPFhDmqrg2WmhKrjxaWzESjY85GIZcXFmVibcw33tFBzg NyICYRAS X-Authority-Analysis: v=2.4 cv=TOlFS0la c=1 sm=1 tr=0 ts=68d42779 cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=yJojWOMRYYMA:10 a=VnNF1IyMAAAA:8 a=ove13Onh8FEOJJhcmoIA:9 X-Proofpoint-ORIG-GUID: 2Kd3CNY_2zXrPRlixjk8Fr9ItOmlQFpK X-Proofpoint-GUID: 2Kd3CNY_2zXrPRlixjk8Fr9ItOmlQFpK X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-24_04,2025-09-24_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 priorityscore=1501 phishscore=0 impostorscore=0 adultscore=0 suspectscore=0 spamscore=0 bulkscore=0 malwarescore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509200033 Content-Type: text/plain; charset="utf-8" Commit c1e18c17bda6 ("s390/pci: add zpci_set_irq()/zpci_clear_irq()"), introduced the zpci_set_irq() and zpci_clear_irq(), to be used while resetting a zPCI device. Commit da995d538d3a ("s390/pci: implement reset_slot for hotplug slot"), mentions zpci_clear_irq() being called in the path for zpci_hot_reset_devic= e(). But that is not the case anymore and these functions are not called outside of this file. Instead zpci_hot_reset_device() relies on zpci_disable_device() also clearing the IRQs, but misses to reset the zdev->irqs_registered flag. However after a CLP disable/enable reset, the device's IRQ are unregistered, but the flag zdev->irq_registered does not get cleared. It creates an inconsistent state and so arch_restore_msi_irqs() doesn't correctly restore the device's IRQ. This becomes a problem when a PCI driver tries to restore the state of the device through pci_restore_state(). Restore IRQ unconditionally for the device and remove the irq_registered flag as its redundant. Signed-off-by: Farhan Ali Reviewed-by: Niklas Schnelle --- arch/s390/include/asm/pci.h | 1 - arch/s390/pci/pci_irq.c | 9 +-------- 2 files changed, 1 insertion(+), 9 deletions(-) diff --git a/arch/s390/include/asm/pci.h b/arch/s390/include/asm/pci.h index 41f900f693d9..aed19a1aa9d7 100644 --- a/arch/s390/include/asm/pci.h +++ b/arch/s390/include/asm/pci.h @@ -145,7 +145,6 @@ struct zpci_dev { u8 has_resources : 1; u8 is_physfn : 1; u8 util_str_avail : 1; - u8 irqs_registered : 1; u8 tid_avail : 1; u8 rtr_avail : 1; /* Relaxed translation allowed */ unsigned int devfn; /* DEVFN part of the RID*/ diff --git a/arch/s390/pci/pci_irq.c b/arch/s390/pci/pci_irq.c index 84482a921332..e73be96ce5fe 100644 --- a/arch/s390/pci/pci_irq.c +++ b/arch/s390/pci/pci_irq.c @@ -107,9 +107,6 @@ static int zpci_set_irq(struct zpci_dev *zdev) else rc =3D zpci_set_airq(zdev); =20 - if (!rc) - zdev->irqs_registered =3D 1; - return rc; } =20 @@ -123,9 +120,6 @@ static int zpci_clear_irq(struct zpci_dev *zdev) else rc =3D zpci_clear_airq(zdev); =20 - if (!rc) - zdev->irqs_registered =3D 0; - return rc; } =20 @@ -427,8 +421,7 @@ bool arch_restore_msi_irqs(struct pci_dev *pdev) { struct zpci_dev *zdev =3D to_zpci(pdev); =20 - if (!zdev->irqs_registered) - zpci_set_irq(zdev); + zpci_set_irq(zdev); return true; } =20 --=20 2.43.0 From nobody Thu Oct 2 00:49:25 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0B02531A7FE; Wed, 24 Sep 2025 17:16:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758734207; cv=none; b=YSR13kv+NtoPDkGDHwdsQsVDt2RtvJndstkvloOczO+XrhgJactlAMVhYD1kaUPHD3/lne6fQ12w5Et+j8fwoOFjUrZu/IzvI6fma5Asd73QY/0YuHFLzlWgdyLjAOjY/3JQdp81UDrMY6h+gJCqJgj+DWdv892+TSUhM7+d18o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758734207; c=relaxed/simple; bh=7f1tPgGaOdUema3daP0tIwh/qEBb+lZsUgVn6G7qQIk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=pTLDM0i3DxXTAAOhKgzdJOfRFALH9qGp7JqAMkIFW82PLjh8IHUw1ohoH5fVmFbejsp7Muuk6hm0g5iQ9cfhvKB4+hmZ4ErHVCA2a3C80jfmCjNODMGyA2h9SkYON2xkLMFsWXQeymDSORv3TfJke4GS/b7uXx3CVYigNEP3fPk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=scCu2Ntu; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="scCu2Ntu" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58OC0iLU025570; Wed, 24 Sep 2025 17:16:41 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=OZlRmUsoVE25Fkj72 3JgZQlQAimm2Ubmq5yCrK6J0m0=; b=scCu2Ntu1nc1ku4e3jOARspkMWax3VgFx GDinmUFCB8wE32/UjwSpzVSAw/CrhqbIXmOxbMYAlQd4ihcqRkc1Zo/R2qTUmG+P nYorzhRKHRW7SZ8lGME7oqW6WRShczZ2S+lA7QXkruHFz9kqeW6+lJJ5OIAHl3KX r8N28V7LcU8XxPv4G27CYZmBvxNWqyfBA3a1o5AQXNSdvYV7cGD/77W2yVfuS2py XyBtOYwMCajTvcbTjrUroaTTtVSWWJNGEec2Ite0ykHsgIlu8a00It0AkCm24lBI uy0L0cLhcqGU1fzflyLeZaAeS2CSZQnasTvU8YVEcHXuvhpxXWz4A== Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 499jpkgb41-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 24 Sep 2025 17:16:41 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58OEm7ha030336; Wed, 24 Sep 2025 17:16:40 GMT Received: from smtprelay04.wdc07v.mail.ibm.com ([172.16.1.71]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 49a9a19f4s-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 24 Sep 2025 17:16:40 +0000 Received: from smtpav03.dal12v.mail.ibm.com (smtpav03.dal12v.mail.ibm.com [10.241.53.102]) by smtprelay04.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58OHGc6k54133046 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Sep 2025 17:16:38 GMT Received: from smtpav03.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 35E5858063; Wed, 24 Sep 2025 17:16:38 +0000 (GMT) Received: from smtpav03.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7116F58056; Wed, 24 Sep 2025 17:16:37 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.252.148]) by smtpav03.dal12v.mail.ibm.com (Postfix) with ESMTP; Wed, 24 Sep 2025 17:16:37 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, clg@redhat.com, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v4 06/10] s390/pci: Update the logic for detecting passthrough device Date: Wed, 24 Sep 2025 10:16:24 -0700 Message-ID: <20250924171628.826-7-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250924171628.826-1-alifm@linux.ibm.com> References: <20250924171628.826-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=L50dQ/T8 c=1 sm=1 tr=0 ts=68d42779 cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=yJojWOMRYYMA:10 a=VnNF1IyMAAAA:8 a=MtaQWmmsotiDqj5H3ecA:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTIwMDAxMCBTYWx0ZWRfX7eql0a1R+kyt MMu2Roq5/CXaQdzLnLb4U4Icprod3C7n2BczXY8su2Gpbfw9cuOO6WyKztFWqL3KSgMwK2sMyRj f6psg33/x0Gx38dzXW75G5xlzDIpKO3HULGechliq43IRHY/MsRTuDsADmlJ851cLIgjzUbJnY+ ELIcSD7uIiCdil+xnkJEycQqOllJsEDyBvLoHvLUahnKqvmxOsR3LMHRMzzlDd+UUsIRA1zLutP Zs3yNqaIig/aPxevHCn2AzfEyWEJKR/MgjLwPW94Ll8sYH8ZLb6eEpXjwdpysFUzNrR44GpaFLJ ApngdQ5FOFNkm26m4p7easdmjATlux0lPQQKILL4qAA7VX4g0yDqgRvuKulh70r9gU67BNOsqlH VZpRNaQr X-Proofpoint-ORIG-GUID: 3vei9bZDTqjT46VOIIaOSgfcA2qWBcsy X-Proofpoint-GUID: 3vei9bZDTqjT46VOIIaOSgfcA2qWBcsy X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-24_04,2025-09-24_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 adultscore=0 phishscore=0 impostorscore=0 spamscore=0 priorityscore=1501 suspectscore=0 clxscore=1015 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509200010 Content-Type: text/plain; charset="utf-8" We can now have userspace drivers (vfio-pci based) on s390x. The userspace drivers will not have any KVM fd and so no kzdev associated with them. So we need to update the logic for detecting passthrough devices to not depend on struct kvm_zdev. Signed-off-by: Farhan Ali Reviewed-by: Niklas Schnelle --- arch/s390/include/asm/pci.h | 1 + arch/s390/pci/pci_event.c | 14 ++++---------- drivers/vfio/pci/vfio_pci_zdev.c | 9 ++++++++- 3 files changed, 13 insertions(+), 11 deletions(-) diff --git a/arch/s390/include/asm/pci.h b/arch/s390/include/asm/pci.h index aed19a1aa9d7..f47f62fc3bfd 100644 --- a/arch/s390/include/asm/pci.h +++ b/arch/s390/include/asm/pci.h @@ -169,6 +169,7 @@ struct zpci_dev { =20 char res_name[16]; bool mio_capable; + bool mediated_recovery; struct zpci_bar_struct bars[PCI_STD_NUM_BARS]; =20 u64 start_dma; /* Start of available DMA addresses */ diff --git a/arch/s390/pci/pci_event.c b/arch/s390/pci/pci_event.c index d930416d4c90..541d536be052 100644 --- a/arch/s390/pci/pci_event.c +++ b/arch/s390/pci/pci_event.c @@ -61,16 +61,10 @@ static inline bool ers_result_indicates_abort(pci_ers_r= esult_t ers_res) } } =20 -static bool is_passed_through(struct pci_dev *pdev) +static bool needs_mediated_recovery(struct pci_dev *pdev) { struct zpci_dev *zdev =3D to_zpci(pdev); - bool ret; - - mutex_lock(&zdev->kzdev_lock); - ret =3D !!zdev->kzdev; - mutex_unlock(&zdev->kzdev_lock); - - return ret; + return zdev->mediated_recovery; } =20 static bool is_driver_supported(struct pci_driver *driver) @@ -194,7 +188,7 @@ static pci_ers_result_t zpci_event_attempt_error_recove= ry(struct pci_dev *pdev) } pdev->error_state =3D pci_channel_io_frozen; =20 - if (is_passed_through(pdev)) { + if (needs_mediated_recovery(pdev)) { pr_info("%s: Cannot be recovered in the host because it is a pass-throug= h device\n", pci_name(pdev)); status_str =3D "failed (pass-through)"; @@ -277,7 +271,7 @@ static void zpci_event_io_failure(struct pci_dev *pdev,= pci_channel_state_t es) * we will inject the error event and let the guest recover the device * itself. */ - if (is_passed_through(pdev)) + if (needs_mediated_recovery(pdev)) goto out; driver =3D to_pci_driver(pdev->dev.driver); if (driver && driver->err_handler && driver->err_handler->error_detected) diff --git a/drivers/vfio/pci/vfio_pci_zdev.c b/drivers/vfio/pci/vfio_pci_z= dev.c index 0990fdb146b7..a7bc23ce8483 100644 --- a/drivers/vfio/pci/vfio_pci_zdev.c +++ b/drivers/vfio/pci/vfio_pci_zdev.c @@ -148,6 +148,8 @@ int vfio_pci_zdev_open_device(struct vfio_pci_core_devi= ce *vdev) if (!zdev) return -ENODEV; =20 + zdev->mediated_recovery =3D true; + if (!vdev->vdev.kvm) return 0; =20 @@ -161,7 +163,12 @@ void vfio_pci_zdev_close_device(struct vfio_pci_core_d= evice *vdev) { struct zpci_dev *zdev =3D to_zpci(vdev->pdev); =20 - if (!zdev || !vdev->vdev.kvm) + if (!zdev) + return; + + zdev->mediated_recovery =3D false; + + if (!vdev->vdev.kvm) return; =20 if (zpci_kvm_hook.kvm_unregister) --=20 2.43.0 From nobody Thu Oct 2 00:49:25 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C6954320A09; Wed, 24 Sep 2025 17:16:52 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758734214; cv=none; b=ZDAgOQY6lNgq5k58kPoP89dCeb4NZaXnSQpk2Nr2PkIbJydqJRmcfI1Ea0VWtsNbkRIKzG0Dfe5ferbdcfDkrFyx+PsiDqfNif/RzSPRNU4NjWCpUPujekkxf7OfLsKHqHFjC8XID/UT2MljPQsF3S7xu4Jis4xUQ17klhwhorM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758734214; c=relaxed/simple; bh=iT4hR+BUB+iLYg83Vw0N00TaV5CJbDPaMV1TvTMEa64=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=OX347F5SwxnvqlnkolEQxYuGr0cWpev+ee5MRUbFoWhgJrUBzcWojZRb2suWZOLz2UAtxhL6UFYH25m7d7xpAlnD0/V6lA9LHhwTbYQPE59BNTkAOQ1n+60ls4rxTCz8hUlXcYmy7mr9zNTiml3sKRwJeMYsyNyCDFRswCUeyes= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=bDUpzwge; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="bDUpzwge" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58O8iXTY020231; Wed, 24 Sep 2025 17:16:42 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=KzHak2VPVsBuiHLUE mTq7IvPdwA92EFx4JN7OIhDKz8=; b=bDUpzwgeb6g9DiNVxfpgeDobPLuB18wZp yq3T0DavZu0q3AjSUSJ/A/Lfva3TgGe1gSnxXzqqIDUqGzMOD6pR4iRMzsVQbr28 Mn9e6nLXi2gdRb3DT4rpND5BnYDmtQsDMq7D+Q2KPul/xWAB/lEc9ZAO37u+KaN9 n0psluA0koaZex9/7+s0xqHk15Rsyf7SiiEuNJjUlgOAXU3sr2ZF1eEMK2GfZ6a7 WWIs0k1J7dZXtEWcDnGLGzdg11CHFiE7ZUp56O26crKjyGYKKxqrMvYE4F8ZO5Nm ZYzpW9eWB6t6LMnsRtzjJTRYSfoXyXOzFsjj3aUqlgJ+amWG6RUQw== Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 499n0jru5w-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 24 Sep 2025 17:16:42 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58OEMWXR030370; Wed, 24 Sep 2025 17:16:41 GMT Received: from smtprelay05.wdc07v.mail.ibm.com ([172.16.1.72]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 49a9a19f4v-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 24 Sep 2025 17:16:41 +0000 Received: from smtpav03.dal12v.mail.ibm.com (smtpav03.dal12v.mail.ibm.com [10.241.53.102]) by smtprelay05.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58OHGd2p32703176 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Sep 2025 17:16:39 GMT Received: from smtpav03.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0D9675805A; Wed, 24 Sep 2025 17:16:39 +0000 (GMT) Received: from smtpav03.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 51C4B58056; Wed, 24 Sep 2025 17:16:38 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.252.148]) by smtpav03.dal12v.mail.ibm.com (Postfix) with ESMTP; Wed, 24 Sep 2025 17:16:38 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, clg@redhat.com, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v4 07/10] s390/pci: Store PCI error information for passthrough devices Date: Wed, 24 Sep 2025 10:16:25 -0700 Message-ID: <20250924171628.826-8-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250924171628.826-1-alifm@linux.ibm.com> References: <20250924171628.826-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTIwMDAzMyBTYWx0ZWRfX++wql110pv0o zGC6IUcTgCT2MlGItcbWiDjZe3/IHrECmjj7A5zOmbJGWcwHGt6D+NkFOPEUF1chnND4lPJHHKo wSOKucUDiqsu3b1ZCH16PMYZ7ZMq3q9jjd0Fj5ADhT4wP4HOXW4TNbpAr2jz5Tx3f5M3+kqq1q4 +eSVJqc5CTHTRS4hks0C6Zpd0uqiyejHbplKVwSs44m0o8gLAWmXQJKLrH/uQrYfckDsC7JpKsT krNceuZcWcmhAgFhVlLID1tYkhuC9z/oPY8YLadJrXtotVGfz2SmdCO0L6b9yk1UZ/2bnEEWYDJ tWyqvNBZfPAoI7iRLQ80EVNy27aazXLgo9NGwK6DX7Vsj+DHrf5oE23GLHOtU/PW30OIg+Ku5Wd yysoB48E X-Authority-Analysis: v=2.4 cv=TOlFS0la c=1 sm=1 tr=0 ts=68d4277a cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=yJojWOMRYYMA:10 a=VnNF1IyMAAAA:8 a=pyWyu9CeZPAOa7DPJWAA:9 X-Proofpoint-ORIG-GUID: hewvPnWLD7X1pKNLJbuJenxgXZ5J1C4t X-Proofpoint-GUID: hewvPnWLD7X1pKNLJbuJenxgXZ5J1C4t X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-24_04,2025-09-24_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 priorityscore=1501 phishscore=0 impostorscore=0 adultscore=0 suspectscore=0 spamscore=0 bulkscore=0 malwarescore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509200033 Content-Type: text/plain; charset="utf-8" For a passthrough device we need co-operation from user space to recover the device. This would require to bubble up any error information to user space. Let's store this error information for passthrough devices, so it can be retrieved later. Signed-off-by: Farhan Ali --- arch/s390/include/asm/pci.h | 28 ++++++++++ arch/s390/pci/pci.c | 1 + arch/s390/pci/pci_event.c | 95 +++++++++++++++++++------------- drivers/vfio/pci/vfio_pci_zdev.c | 2 + 4 files changed, 88 insertions(+), 38 deletions(-) diff --git a/arch/s390/include/asm/pci.h b/arch/s390/include/asm/pci.h index f47f62fc3bfd..40bfe5721109 100644 --- a/arch/s390/include/asm/pci.h +++ b/arch/s390/include/asm/pci.h @@ -116,6 +116,31 @@ struct zpci_bus { enum pci_bus_speed max_bus_speed; }; =20 +/* Content Code Description for PCI Function Error */ +struct zpci_ccdf_err { + u32 reserved1; + u32 fh; /* function handle */ + u32 fid; /* function id */ + u32 ett : 4; /* expected table type */ + u32 mvn : 12; /* MSI vector number */ + u32 dmaas : 8; /* DMA address space */ + u32 reserved2 : 6; + u32 q : 1; /* event qualifier */ + u32 rw : 1; /* read/write */ + u64 faddr; /* failing address */ + u32 reserved3; + u16 reserved4; + u16 pec; /* PCI event code */ +} __packed; + +#define ZPCI_ERR_PENDING_MAX 4 +struct zpci_ccdf_pending { + u8 count; + u8 head; + u8 tail; + struct zpci_ccdf_err err[ZPCI_ERR_PENDING_MAX]; +}; + /* Private data per function */ struct zpci_dev { struct zpci_bus *zbus; @@ -191,6 +216,8 @@ struct zpci_dev { struct iommu_domain *s390_domain; /* attached IOMMU domain */ struct kvm_zdev *kzdev; struct mutex kzdev_lock; + struct zpci_ccdf_pending pending_errs; + struct mutex pending_errs_lock; spinlock_t dom_lock; /* protect s390_domain change */ }; =20 @@ -316,6 +343,7 @@ void zpci_debug_exit_device(struct zpci_dev *); int zpci_report_error(struct pci_dev *, struct zpci_report_error_header *); int zpci_clear_error_state(struct zpci_dev *zdev); int zpci_reset_load_store_blocked(struct zpci_dev *zdev); +void zpci_cleanup_pending_errors(struct zpci_dev *zdev); =20 #ifdef CONFIG_NUMA =20 diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c index 3992ba44e1cf..759dfd9e0fb8 100644 --- a/arch/s390/pci/pci.c +++ b/arch/s390/pci/pci.c @@ -897,6 +897,7 @@ struct zpci_dev *zpci_create_device(u32 fid, u32 fh, en= um zpci_state state) mutex_init(&zdev->state_lock); mutex_init(&zdev->fmb_lock); mutex_init(&zdev->kzdev_lock); + mutex_init(&zdev->pending_errs_lock); =20 return zdev; =20 diff --git a/arch/s390/pci/pci_event.c b/arch/s390/pci/pci_event.c index 541d536be052..9f396a2847ab 100644 --- a/arch/s390/pci/pci_event.c +++ b/arch/s390/pci/pci_event.c @@ -18,23 +18,6 @@ #include "pci_bus.h" #include "pci_report.h" =20 -/* Content Code Description for PCI Function Error */ -struct zpci_ccdf_err { - u32 reserved1; - u32 fh; /* function handle */ - u32 fid; /* function id */ - u32 ett : 4; /* expected table type */ - u32 mvn : 12; /* MSI vector number */ - u32 dmaas : 8; /* DMA address space */ - u32 : 6; - u32 q : 1; /* event qualifier */ - u32 rw : 1; /* read/write */ - u64 faddr; /* failing address */ - u32 reserved3; - u16 reserved4; - u16 pec; /* PCI event code */ -} __packed; - /* Content Code Description for PCI Function Availability */ struct zpci_ccdf_avail { u32 reserved1; @@ -76,6 +59,41 @@ static bool is_driver_supported(struct pci_driver *drive= r) return true; } =20 +static void zpci_store_pci_error(struct pci_dev *pdev, + struct zpci_ccdf_err *ccdf) +{ + struct zpci_dev *zdev =3D to_zpci(pdev); + int i; + + mutex_lock(&zdev->pending_errs_lock); + if (zdev->pending_errs.count >=3D ZPCI_ERR_PENDING_MAX) { + pr_err("%s: Maximum number (%d) of pending error events queued", + pci_name(pdev), ZPCI_ERR_PENDING_MAX); + mutex_unlock(&zdev->pending_errs_lock); + return; + } + + i =3D zdev->pending_errs.tail % ZPCI_ERR_PENDING_MAX; + memcpy(&zdev->pending_errs.err[i], ccdf, sizeof(struct zpci_ccdf_err)); + zdev->pending_errs.tail++; + zdev->pending_errs.count++; + mutex_unlock(&zdev->pending_errs_lock); +} + +void zpci_cleanup_pending_errors(struct zpci_dev *zdev) +{ + struct pci_dev *pdev =3D NULL; + + mutex_lock(&zdev->pending_errs_lock); + pdev =3D pci_get_slot(zdev->zbus->bus, zdev->devfn); + if (zdev->pending_errs.count) + pr_info("%s: Unhandled PCI error events count=3D%d", + pci_name(pdev), zdev->pending_errs.count); + memset(&zdev->pending_errs, 0, sizeof(struct zpci_ccdf_pending)); + mutex_unlock(&zdev->pending_errs_lock); +} +EXPORT_SYMBOL_GPL(zpci_cleanup_pending_errors); + static pci_ers_result_t zpci_event_notify_error_detected(struct pci_dev *p= dev, struct pci_driver *driver) { @@ -169,7 +187,8 @@ static pci_ers_result_t zpci_event_do_reset(struct pci_= dev *pdev, * and the platform determines which functions are affected for * multi-function devices. */ -static pci_ers_result_t zpci_event_attempt_error_recovery(struct pci_dev *= pdev) +static pci_ers_result_t zpci_event_attempt_error_recovery(struct pci_dev *= pdev, + struct zpci_ccdf_err *ccdf) { pci_ers_result_t ers_res =3D PCI_ERS_RESULT_DISCONNECT; struct zpci_dev *zdev =3D to_zpci(pdev); @@ -188,13 +207,6 @@ static pci_ers_result_t zpci_event_attempt_error_recov= ery(struct pci_dev *pdev) } pdev->error_state =3D pci_channel_io_frozen; =20 - if (needs_mediated_recovery(pdev)) { - pr_info("%s: Cannot be recovered in the host because it is a pass-throug= h device\n", - pci_name(pdev)); - status_str =3D "failed (pass-through)"; - goto out_unlock; - } - driver =3D to_pci_driver(pdev->dev.driver); if (!is_driver_supported(driver)) { if (!driver) { @@ -210,12 +222,23 @@ static pci_ers_result_t zpci_event_attempt_error_reco= very(struct pci_dev *pdev) goto out_unlock; } =20 + if (needs_mediated_recovery(pdev)) + zpci_store_pci_error(pdev, ccdf); + ers_res =3D zpci_event_notify_error_detected(pdev, driver); if (ers_result_indicates_abort(ers_res)) { status_str =3D "failed (abort on detection)"; goto out_unlock; } =20 + if (needs_mediated_recovery(pdev)) { + pr_info("%s: Leaving recovery of pass-through device to user-space\n", + pci_name(pdev)); + ers_res =3D PCI_ERS_RESULT_RECOVERED; + status_str =3D "in progress"; + goto out_unlock; + } + if (ers_res !=3D PCI_ERS_RESULT_NEED_RESET) { ers_res =3D zpci_event_do_error_state_clear(pdev, driver); if (ers_result_indicates_abort(ers_res)) { @@ -258,25 +281,20 @@ static pci_ers_result_t zpci_event_attempt_error_reco= very(struct pci_dev *pdev) * @pdev: PCI function for which to report * @es: PCI channel failure state to report */ -static void zpci_event_io_failure(struct pci_dev *pdev, pci_channel_state_= t es) +static void zpci_event_io_failure(struct pci_dev *pdev, pci_channel_state_= t es, + struct zpci_ccdf_err *ccdf) { struct pci_driver *driver; =20 pci_dev_lock(pdev); pdev->error_state =3D es; - /** - * While vfio-pci's error_detected callback notifies user-space QEMU - * reacts to this by freezing the guest. In an s390 environment PCI - * errors are rarely fatal so this is overkill. Instead in the future - * we will inject the error event and let the guest recover the device - * itself. - */ + if (needs_mediated_recovery(pdev)) - goto out; + zpci_store_pci_error(pdev, ccdf); driver =3D to_pci_driver(pdev->dev.driver); if (driver && driver->err_handler && driver->err_handler->error_detected) driver->err_handler->error_detected(pdev, pdev->error_state); -out: + pci_dev_unlock(pdev); } =20 @@ -322,12 +340,13 @@ static void __zpci_event_error(struct zpci_ccdf_err *= ccdf) break; case 0x0040: /* Service Action or Error Recovery Failed */ case 0x003b: - zpci_event_io_failure(pdev, pci_channel_io_perm_failure); + zpci_event_io_failure(pdev, pci_channel_io_perm_failure, ccdf); break; default: /* PCI function left in the error state attempt to recover */ - ers_res =3D zpci_event_attempt_error_recovery(pdev); + ers_res =3D zpci_event_attempt_error_recovery(pdev, ccdf); if (ers_res !=3D PCI_ERS_RESULT_RECOVERED) - zpci_event_io_failure(pdev, pci_channel_io_perm_failure); + zpci_event_io_failure(pdev, pci_channel_io_perm_failure, + ccdf); break; } pci_dev_put(pdev); diff --git a/drivers/vfio/pci/vfio_pci_zdev.c b/drivers/vfio/pci/vfio_pci_z= dev.c index a7bc23ce8483..2be37eab9279 100644 --- a/drivers/vfio/pci/vfio_pci_zdev.c +++ b/drivers/vfio/pci/vfio_pci_zdev.c @@ -168,6 +168,8 @@ void vfio_pci_zdev_close_device(struct vfio_pci_core_de= vice *vdev) =20 zdev->mediated_recovery =3D false; =20 + zpci_cleanup_pending_errors(zdev); + if (!vdev->vdev.kvm) return; =20 --=20 2.43.0 From nobody Thu Oct 2 00:49:25 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4680031B108; Wed, 24 Sep 2025 17:16:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758734209; cv=none; b=jnVkH+FXdwLt9l/1dxa4j4Hjq5OZhRHce/oOPrJXJe91St2m5RT7kFm13CYsKVaLzoTgsrM9dvVS+13wH42XDwCOHZAZSIMSMojRXXmjKzcRgvarNRSnFzGK/KhUv+ooqfkQir5CX3VlzDjTvB5rg3VbHfWBrOzeZl3rPBT5/ds= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758734209; c=relaxed/simple; bh=DCL2uNhkT2T1GieYn26W3SH3cIX9duxF8ftiep2wZr4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=SAkhmnJ379MHbF2xJXKveqqj/JCBrUdVVbsewCUHsDbUg1GewnywQE141rlgvWscUL+oziP9QtNOCwwnBmJ8qc8sHN5NFEkWr+nFrc82Ylw2TNpTyPLKI+KR5pjazcbfQEDjJ6E57ikjtUIyJm3xRt/tFHMnX5lepjam1xCniU8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=f26qYk6I; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="f26qYk6I" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58OAOM0J006365; Wed, 24 Sep 2025 17:16:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=VnzLReFLvODQ7n9Cj a1s1rdfqgjqZkjn+xZ4yhNDWBQ=; b=f26qYk6Iu+Q2rurDiS7dzGjsU6O2O1Ehg yrh8KPcfS9NrkueG1jR81bj+LbrN9hR10hmkpyEdIjDqxAqoRl1XzlYFOW0yjGTB LDt0VAAy+j61oDJyZ9p+0L7s6h1nd5f+y7RXebObZK1mBHUSFHblHlEuwRFXekUL fPpsxcv/ipyyV6Vb7WnnkYSS2cXArVTgivPlk6blXzXcmRj7P8SCoGtOQqli7fJZ R5ljwqoGBDrtJ0qXy7uF/mQbFs7LJFbH7nOYP9UN809BnuIbUa7LWQ9QNzrgr1BJ UvRtODy68ydKSfGtE7UkY8PDCsebnkUBnEFZ+vcbp3qCuYORJBbeA== Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 499kwyr5v8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 24 Sep 2025 17:16:42 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58OGpF7H031139; Wed, 24 Sep 2025 17:16:41 GMT Received: from smtprelay05.wdc07v.mail.ibm.com ([172.16.1.72]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 49b9vdar40-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 24 Sep 2025 17:16:41 +0000 Received: from smtpav03.dal12v.mail.ibm.com (smtpav03.dal12v.mail.ibm.com [10.241.53.102]) by smtprelay05.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58OHGdv320972232 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Sep 2025 17:16:40 GMT Received: from smtpav03.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D5DC258064; Wed, 24 Sep 2025 17:16:39 +0000 (GMT) Received: from smtpav03.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 287A55803F; Wed, 24 Sep 2025 17:16:39 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.252.148]) by smtpav03.dal12v.mail.ibm.com (Postfix) with ESMTP; Wed, 24 Sep 2025 17:16:39 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, clg@redhat.com, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v4 08/10] vfio-pci/zdev: Add a device feature for error information Date: Wed, 24 Sep 2025 10:16:26 -0700 Message-ID: <20250924171628.826-9-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250924171628.826-1-alifm@linux.ibm.com> References: <20250924171628.826-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=J5Cq7BnS c=1 sm=1 tr=0 ts=68d4277a cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=yJojWOMRYYMA:10 a=VnNF1IyMAAAA:8 a=QIy8Zix3hhx26zbqT_4A:9 X-Proofpoint-GUID: Gw8VJVT5LJMgmQpsRZW-MrUmQxZXrjnj X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTIwMDAxNSBTYWx0ZWRfXw2xYmnTmIkpE RKVlqkxxd6vSyS+uNtOtqoh3BcwZQHQuK3U4XRBZX49MVktJk9lWoTStLDJPKhUVcaq1XkqRWE6 Zej9XGv/jjAtFRCCnP+Aet3DDHvFAvl8r9fAcofAmZf+VeEXikbbPvK1bnM8zPC/8cZckUNRhIk SB+wnbVUhJ2rLcCv3eVl2Zu1VDz/bk6/hJXX5VbTPJFdc6j4Dpfpvg9vtADdRvf0E0J14rz+D0n miIuZBjmxDQYiR6i1FcSDyTabS1HwFk/HWFQC+xQ+6a2WZFRRwIKVUDDd2WTJogi6TT+CDG27Kn Uu4XQQGlTqyF50m3vPpnGPwE68EVuFyaPIF5/SBWupv0EW298KBXNc9I/1oWT+yQILu9+Vnyliw /OGDIBeU X-Proofpoint-ORIG-GUID: Gw8VJVT5LJMgmQpsRZW-MrUmQxZXrjnj X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-24_04,2025-09-24_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 impostorscore=0 malwarescore=0 spamscore=0 bulkscore=0 phishscore=0 clxscore=1015 adultscore=0 suspectscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509200015 Content-Type: text/plain; charset="utf-8" For zPCI devices, we have platform specific error information. The platform firmware provides this error information to the operating system in an architecture specific mechanism. To enable recovery from userspace for these devices, we want to expose this error information to userspace. Add a new device feature to expose this information. Signed-off-by: Farhan Ali --- drivers/vfio/pci/vfio_pci_core.c | 2 ++ drivers/vfio/pci/vfio_pci_priv.h | 8 ++++++++ drivers/vfio/pci/vfio_pci_zdev.c | 34 ++++++++++++++++++++++++++++++++ include/uapi/linux/vfio.h | 15 ++++++++++++++ 4 files changed, 59 insertions(+) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index 7dcf5439dedc..378adb3226db 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -1514,6 +1514,8 @@ int vfio_pci_core_ioctl_feature(struct vfio_device *d= evice, u32 flags, return vfio_pci_core_pm_exit(device, flags, arg, argsz); case VFIO_DEVICE_FEATURE_PCI_VF_TOKEN: return vfio_pci_core_feature_token(device, flags, arg, argsz); + case VFIO_DEVICE_FEATURE_ZPCI_ERROR: + return vfio_pci_zdev_feature_err(device, flags, arg, argsz); default: return -ENOTTY; } diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_p= riv.h index a9972eacb293..808d92aab76b 100644 --- a/drivers/vfio/pci/vfio_pci_priv.h +++ b/drivers/vfio/pci/vfio_pci_priv.h @@ -86,6 +86,8 @@ int vfio_pci_info_zdev_add_caps(struct vfio_pci_core_devi= ce *vdev, struct vfio_info_cap *caps); int vfio_pci_zdev_open_device(struct vfio_pci_core_device *vdev); void vfio_pci_zdev_close_device(struct vfio_pci_core_device *vdev); +int vfio_pci_zdev_feature_err(struct vfio_device *device, u32 flags, + void __user *arg, size_t argsz); #else static inline int vfio_pci_info_zdev_add_caps(struct vfio_pci_core_device = *vdev, struct vfio_info_cap *caps) @@ -100,6 +102,12 @@ static inline int vfio_pci_zdev_open_device(struct vfi= o_pci_core_device *vdev) =20 static inline void vfio_pci_zdev_close_device(struct vfio_pci_core_device = *vdev) {} + +static int vfio_pci_zdev_feature_err(struct vfio_device *device, u32 flags, + void __user *arg, size_t argsz) +{ + return -ENODEV; +} #endif =20 static inline bool vfio_pci_is_vga(struct pci_dev *pdev) diff --git a/drivers/vfio/pci/vfio_pci_zdev.c b/drivers/vfio/pci/vfio_pci_z= dev.c index 2be37eab9279..261954039aa9 100644 --- a/drivers/vfio/pci/vfio_pci_zdev.c +++ b/drivers/vfio/pci/vfio_pci_zdev.c @@ -141,6 +141,40 @@ int vfio_pci_info_zdev_add_caps(struct vfio_pci_core_d= evice *vdev, return ret; } =20 +int vfio_pci_zdev_feature_err(struct vfio_device *device, u32 flags, + void __user *arg, size_t argsz) +{ + struct vfio_device_feature_zpci_err err; + struct vfio_pci_core_device *vdev =3D + container_of(device, struct vfio_pci_core_device, vdev); + struct zpci_dev *zdev =3D to_zpci(vdev->pdev); + int ret; + int head =3D 0; + + if (!zdev) + return -ENODEV; + + ret =3D vfio_check_feature(flags, argsz, VFIO_DEVICE_FEATURE_GET, + sizeof(err)); + if (ret !=3D 1) + return ret; + + mutex_lock(&zdev->pending_errs_lock); + if (zdev->pending_errs.count) { + head =3D zdev->pending_errs.head % ZPCI_ERR_PENDING_MAX; + err.pec =3D zdev->pending_errs.err[head].pec; + zdev->pending_errs.head++; + zdev->pending_errs.count--; + err.pending_errors =3D zdev->pending_errs.count; + } + mutex_unlock(&zdev->pending_errs_lock); + + if (copy_to_user(arg, &err, sizeof(err))) + return -EFAULT; + + return 0; +} + int vfio_pci_zdev_open_device(struct vfio_pci_core_device *vdev) { struct zpci_dev *zdev =3D to_zpci(vdev->pdev); diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h index 75100bf009ba..d72177bc3961 100644 --- a/include/uapi/linux/vfio.h +++ b/include/uapi/linux/vfio.h @@ -1478,6 +1478,21 @@ struct vfio_device_feature_bus_master { }; #define VFIO_DEVICE_FEATURE_BUS_MASTER 10 =20 +/** + * VFIO_DEVICE_FEATURE_ZPCI_ERROR feature provides PCI error information to + * userspace for vfio-pci devices on s390x. On s390x PCI error recovery in= volves + * platform firmware and notification to operating system is done by + * architecture specific mechanism. Exposing this information to userspace + * allows userspace to take appropriate actions to handle an error on the + * device. + */ +struct vfio_device_feature_zpci_err { + __u16 pec; + __u8 pending_errors; + __u8 pad; +}; +#define VFIO_DEVICE_FEATURE_ZPCI_ERROR 11 + /* -------- API for Type1 VFIO IOMMU -------- */ =20 /** --=20 2.43.0 From nobody Thu Oct 2 00:49:25 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AD17A31B800; Wed, 24 Sep 2025 17:16:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758734209; cv=none; b=tAr01h9+kmCywlxlW6pcHHr/QkxGH70Pu416gftKacc9Y546scMsfWw6CjTlA6Tji2Gtuthtar7mp8TNQVkfJzVfr8oEPsOqw63eDuphPAXgFg6s5YdRyXn5OkRtUiuemAYeuhPQyLGImsEod/HRaDV02xR8n7oBPZEZokiP/u8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758734209; c=relaxed/simple; bh=ytFXbNEKbWNWIa4e99DqMpdEgtgPube/J3VaS5HzAEQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JXP5Iho8F9oUp7PC3YxO1OxQqVAvSBKFS40b2YhiPme2Vywtp2fKJ5Satr8q2coGdTQ4gTYSoIZUdcqGOf5RGjcmfuYUZ9RtCe1yljwMA+dLUdvQHB5xZF0VopIpZSBzGsBJjHQScumUuT/fY2YT1mgDenh+lMGO5iJVczw2urQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=JHKdfkI8; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="JHKdfkI8" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58OCha1U028697; Wed, 24 Sep 2025 17:16:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=wJWMYHMfvbzU5BQsW YZErSvZaRLa/7xMVP6aVFziDXk=; b=JHKdfkI8gfLkmGRl0rsSiZasPB149HgbB 4JkjLG0GuScQF0pGeHm/Yudu/6Oh7myyOQ7Sfdsj5FX9VRH8BzKEuSPKHn5JPL/P DGxfICTjwBd9I7o83zSnoAnZFpzdLvngEvwjFx6sDf1Wzr4iDJYH8PEYmBc8Nx/z GuXXk8+gTiczT366Zus2wC/VxBIxotC6EA4jrdulNI+ADMIv4JPtT65pqhpi9uUR 2FPjkxtqJ1DeTLxyZ2RxW3f/8MLSyIJKovYmYPmXmG8f7TPOG47OreUvgQpJyS+1 brCz6vH8CBokeLZ2ZrS7rxuWOnJw8QDe4AM6usSGEoi53pMtMaJhQ== Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 499ksc11pq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 24 Sep 2025 17:16:43 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58OGqNCo031129; Wed, 24 Sep 2025 17:16:42 GMT Received: from smtprelay06.wdc07v.mail.ibm.com ([172.16.1.73]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 49b9vdar41-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 24 Sep 2025 17:16:42 +0000 Received: from smtpav03.dal12v.mail.ibm.com (smtpav03.dal12v.mail.ibm.com [10.241.53.102]) by smtprelay06.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58OHGeBf12845750 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Sep 2025 17:16:41 GMT Received: from smtpav03.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id AB0F258061; Wed, 24 Sep 2025 17:16:40 +0000 (GMT) Received: from smtpav03.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id F0E845803F; Wed, 24 Sep 2025 17:16:39 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.252.148]) by smtpav03.dal12v.mail.ibm.com (Postfix) with ESMTP; Wed, 24 Sep 2025 17:16:39 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, clg@redhat.com, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v4 09/10] vfio: Add a reset_done callback for vfio-pci driver Date: Wed, 24 Sep 2025 10:16:27 -0700 Message-ID: <20250924171628.826-10-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250924171628.826-1-alifm@linux.ibm.com> References: <20250924171628.826-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: 6cl494nTIHO2U0SAHq_P21FWyZhFF8cJ X-Proofpoint-GUID: 6cl494nTIHO2U0SAHq_P21FWyZhFF8cJ X-Authority-Analysis: v=2.4 cv=SdH3duRu c=1 sm=1 tr=0 ts=68d4277b cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=yJojWOMRYYMA:10 a=VnNF1IyMAAAA:8 a=hcTHx3Z5Akp3fEzgVBYA:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTIwMDAyMCBTYWx0ZWRfX4Ul4mXF3jdFd s67xrGyNw/2EDwiOV3id+g38FWnr82WZVQmiCJzJr8xPGXSBqSbrAxQOZ/bPEYaPxr/WvLViD4R i8zuhdEI7VLSZvqEJTS4JMGisUwRW/PubZKGs1/dq63Uj+Z/kNXStKb5zZV1WUqbtgC+L5MC6PB B+4ryQWI78oSDZYLgPE9wdEmOOmdBRgWbJxi0pEX2UjKwRQQLiysoG8v+Km1gOEx8+zbqTHMf4n SneDY1ffIvNcDUQaXOiQX3h2aqJ7FLuNKfSSMzjny2YYKr6oBhy4JSRenChz/srm/sv9P2T74Ry se7TWWIYG7rmM77cW4OaFlBKbHnXo1n5vg2d3MO2Idfpcy925ETfXCtwsT6xBHsOFf7hojEXqq+ Bug4JWw5 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-24_04,2025-09-24_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 suspectscore=0 phishscore=0 spamscore=0 bulkscore=0 priorityscore=1501 clxscore=1015 adultscore=0 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509200020 Content-Type: text/plain; charset="utf-8" On error recovery for a PCI device bound to vfio-pci driver, we want to recover the state of the device to its last known saved state. The callback restores the state of the device to its initial saved state. Signed-off-by: Farhan Ali --- drivers/vfio/pci/vfio_pci_core.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index 378adb3226db..f2fcb81b3e69 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -2241,6 +2241,17 @@ pci_ers_result_t vfio_pci_core_aer_err_detected(stru= ct pci_dev *pdev, } EXPORT_SYMBOL_GPL(vfio_pci_core_aer_err_detected); =20 +static void vfio_pci_core_aer_reset_done(struct pci_dev *pdev) +{ + struct vfio_pci_core_device *vdev =3D dev_get_drvdata(&pdev->dev); + + if (!vdev->pci_saved_state) + return; + + pci_load_saved_state(pdev, vdev->pci_saved_state); + pci_restore_state(pdev); +} + int vfio_pci_core_sriov_configure(struct vfio_pci_core_device *vdev, int nr_virtfn) { @@ -2305,6 +2316,7 @@ EXPORT_SYMBOL_GPL(vfio_pci_core_sriov_configure); =20 const struct pci_error_handlers vfio_pci_core_err_handlers =3D { .error_detected =3D vfio_pci_core_aer_err_detected, + .reset_done =3D vfio_pci_core_aer_reset_done, }; EXPORT_SYMBOL_GPL(vfio_pci_core_err_handlers); =20 --=20 2.43.0 From nobody Thu Oct 2 00:49:25 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8E99A320CB5; Wed, 24 Sep 2025 17:16:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758734217; cv=none; b=mHDZYh0oQjd/qMrMCmzqVVwf1bseLv/wpbwFRc6c8u2F2IGu8QOnpNeNfuHnKeWSL2fCC7yU0a4EGMq3VuetFSZMBUJBMEi92y2nzq7hW2lENcS54w4XyXuaqrYjhIPe+dVH/VQLHfZcgwC9x06QiSWeWo5uuyWMn6aFsCp8qTQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1758734217; c=relaxed/simple; bh=HvTt5221H5DQ1X+GZWXY3QkHHnBA6eW/stRkoiAL7gs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=luWwkTby4op1K2ureeaWYIV75a2WE/wZLEKsFln9eFS3wL8FA6nywlzXqT1TJEKpcgHmnYmQxIQyN/DXBD5PHZ7PwI/1AxYw4DbW9CgpmNPkD+B2FXqcTJpNqqvNXdZE17KeyIyiKdStMAT+3MxCBkiILr+pmQFnoDJeBZfAZI0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=nMTEGq3d; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="nMTEGq3d" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58OBC4E3030655; Wed, 24 Sep 2025 17:16:44 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=+c2kHIW89fFxZMF7i igDqHKPq5NHEczv6HmG2G6s5GY=; b=nMTEGq3dFYbBbQvJ9GXFiw6mQfvct6nxp aw3KigkApG5BUKVWVh2FLi7guzRJobT9vzJGKJD3pMnVh+yP3UN6o+SANjAexSsc lBtWB277l3UaoVmZrOvFWMTGX6E7cywfwHUFxDDwCBBSjoRci1Z2nhvjn/frKEpx +6FBbc0ZIsVo3+g2BLKcNgWXiUEZO57voI+LrTTcDv4lakN3q9smzMhQFemlLx5c 83BqZwoEEbeaqns43uNuuS6pMYgz0iUrmGZmvbr4x1hPuLbI0o78mAeNfkB6XbDy bQXF7j6uUBrRCMDWGKkBiohi0xjoYbouvIJ2/RtrFS8cZuxUsQxsw== Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 499n0jru62-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 24 Sep 2025 17:16:44 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58OGqNCp031129; Wed, 24 Sep 2025 17:16:43 GMT Received: from smtprelay07.wdc07v.mail.ibm.com ([172.16.1.74]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 49b9vdar45-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 24 Sep 2025 17:16:43 +0000 Received: from smtpav03.dal12v.mail.ibm.com (smtpav03.dal12v.mail.ibm.com [10.241.53.102]) by smtprelay07.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58OHGfTi25231924 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 24 Sep 2025 17:16:41 GMT Received: from smtpav03.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 82FD958056; Wed, 24 Sep 2025 17:16:41 +0000 (GMT) Received: from smtpav03.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C5F455803F; Wed, 24 Sep 2025 17:16:40 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.252.148]) by smtpav03.dal12v.mail.ibm.com (Postfix) with ESMTP; Wed, 24 Sep 2025 17:16:40 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, clg@redhat.com, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v4 10/10] vfio: Remove the pcie check for VFIO_PCI_ERR_IRQ_INDEX Date: Wed, 24 Sep 2025 10:16:28 -0700 Message-ID: <20250924171628.826-11-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250924171628.826-1-alifm@linux.ibm.com> References: <20250924171628.826-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTIwMDAzMyBTYWx0ZWRfX0X969P0a6iN4 bG850Eq2l3I5lvYOw5aezrN0xypGlm7dmRoS2mjgjFmVhsnKyn2SA41t7mp2VYdZ/cdbyfTJD46 VN+PKccWSd+5dSll+cCbxHLFpA5Vum/WLZcypoJZ1EXBXidPxLthqZ2ct3fN4gtcr9n5azWtyeR HNViRTycMLtwTQ30y3W/AUTjnjPeMQ0IJ5GfILbQ9KBFuQca5MujjiTizo7Hrvc9fOTzPShOgWb V26vx4SNTwUgpFWVVchi4f+Pm2ZfnYgx0gUvlvRUxxta5bs5RSE0VCb0Zm0UoIlLOl49tAYT5+M BUCDb26Ev0r6p271ZMz28JXiy/GUPjS5th0CfmdGvOgOeX2eZ8cSIGhUoJHVHG7BBab7ll0pbpq XGHe59DF X-Authority-Analysis: v=2.4 cv=TOlFS0la c=1 sm=1 tr=0 ts=68d4277c cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=yJojWOMRYYMA:10 a=VnNF1IyMAAAA:8 a=gONGJyW3jwFs4LhXXOUA:9 X-Proofpoint-ORIG-GUID: 48TChb8nh3rFSuN7J0ZHc9sSROnPXjw_ X-Proofpoint-GUID: 48TChb8nh3rFSuN7J0ZHc9sSROnPXjw_ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-24_04,2025-09-24_01,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 priorityscore=1501 phishscore=0 impostorscore=0 adultscore=0 suspectscore=0 spamscore=0 bulkscore=0 malwarescore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509200033 Content-Type: text/plain; charset="utf-8" We are configuring the error signaling on the vast majority of devices and it's extremely rare that it fires anyway. This allows userspace to be notified on errors for legacy PCI devices. The Internal Share Memory (ISM) device on s390x is one such device. For PCI devices on IBM s390x error recovery involves platform firmware and notification to operating system is done by architecture specific way. So the ISM device can still be recovered when notified of an error. Signed-off-by: Farhan Ali --- drivers/vfio/pci/vfio_pci_core.c | 6 ++---- drivers/vfio/pci/vfio_pci_intrs.c | 3 +-- 2 files changed, 3 insertions(+), 6 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index f2fcb81b3e69..d125471fd5ea 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -749,8 +749,7 @@ static int vfio_pci_get_irq_count(struct vfio_pci_core_= device *vdev, int irq_typ return (flags & PCI_MSIX_FLAGS_QSIZE) + 1; } } else if (irq_type =3D=3D VFIO_PCI_ERR_IRQ_INDEX) { - if (pci_is_pcie(vdev->pdev)) - return 1; + return 1; } else if (irq_type =3D=3D VFIO_PCI_REQ_IRQ_INDEX) { return 1; } @@ -1150,8 +1149,7 @@ static int vfio_pci_ioctl_get_irq_info(struct vfio_pc= i_core_device *vdev, case VFIO_PCI_REQ_IRQ_INDEX: break; case VFIO_PCI_ERR_IRQ_INDEX: - if (pci_is_pcie(vdev->pdev)) - break; + break; fallthrough; default: return -EINVAL; diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_= intrs.c index 123298a4dc8f..f2d13b6eb28f 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -838,8 +838,7 @@ int vfio_pci_set_irqs_ioctl(struct vfio_pci_core_device= *vdev, uint32_t flags, case VFIO_PCI_ERR_IRQ_INDEX: switch (flags & VFIO_IRQ_SET_ACTION_TYPE_MASK) { case VFIO_IRQ_SET_ACTION_TRIGGER: - if (pci_is_pcie(vdev->pdev)) - func =3D vfio_pci_set_err_trigger; + func =3D vfio_pci_set_err_trigger; break; } break; --=20 2.43.0