From nobody Thu Oct 2 19:27:06 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7FC56248891; Thu, 11 Sep 2025 18:33:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757615597; cv=none; b=A1thi1LXIDyrFyx+xPg+Fwa9ECBMzvvo0TQYKroqZM+cY6gz7ow9pzPM3XtE6PfagH7iyZ949AWZDcc1Hj1jh9zZ9AUehcVTJAXYGPt2VGC3SESyICVT0shfUUfZjIzNXWpPtnIE1qnTfIIqUkoSEPT/xA2zWep/ZlNUrusM1RM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757615597; c=relaxed/simple; bh=tVxKYpp9mqC89SBvbFtEfvrwRZWcJQBcFJ4zP92d8Hw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BDl8b7RJmOcS9pTmTRCn4v/nENez7Yz278jdHJays7cd1i77mONJokPob5eVZ8mmRTLylt0BbixBa7D0l19FKxENKC9k0PYUOoOY2qIHTGFQigdtu+UAKdsPRrQfjqrv+oECPG0FQL10g2CZ/YS0B1o7Vt/s7NSf7nvztZG/zYM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=WOnYSsoQ; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="WOnYSsoQ" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58BGMJAU030119; Thu, 11 Sep 2025 18:33:11 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=MB/690h1TmnHT3IRM wyhXMcnu5RN6hzXZqJq2XyjfTo=; b=WOnYSsoQQ665Ld9XOF4JoKKu5xc20pKby ikyX5CtpccPFV+9JxrStO52xiNkekP1rsZ9cByngQtL12UthbypExT5/XJe0fT0i XtESBuoYrG5azvpdlMOZlqmh39fbZncXcDMP4cafKMgrhwsrVE1ZyHm+tnrTYGOZ teN1aNStnkDqRBznTL0TUek/gm3OBArrndLeHFLKAlWTlYcHuN8xO5Dm4qqKFZ9Y Lpm+P6wmetdzALUUHDn6Zl5SlPdiRU4qa3W6HTfn6+4txFeJFayzQQ3aQlyeXx1G IJ1yQPMFTKycBmzBlquqnpyweAbi91CS45/nKN4ePwsdDSCKKua0w== Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490ukeu8j4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 18:33:11 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58BIJf1G001172; Thu, 11 Sep 2025 18:33:10 GMT Received: from smtprelay02.dal12v.mail.ibm.com ([172.16.1.4]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 491203q0ar-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 18:33:10 +0000 Received: from smtpav01.dal12v.mail.ibm.com (smtpav01.dal12v.mail.ibm.com [10.241.53.100]) by smtprelay02.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58BIX9uc18088628 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Sep 2025 18:33:09 GMT Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 38FD758059; Thu, 11 Sep 2025 18:33:09 +0000 (GMT) Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8157558058; Thu, 11 Sep 2025 18:33:08 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.249.32]) by smtpav01.dal12v.mail.ibm.com (Postfix) with ESMTP; Thu, 11 Sep 2025 18:33:08 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v3 01/10] PCI: Avoid saving error values for config space Date: Thu, 11 Sep 2025 11:32:58 -0700 Message-ID: <20250911183307.1910-2-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250911183307.1910-1-alifm@linux.ibm.com> References: <20250911183307.1910-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTA2MDE5NSBTYWx0ZWRfX/29pfwtsYXQo 6AOtezG1CEVVJMmemLRSRp3Dq3hyKEGce8Fcw/jNBKYBbGX7WsCEAYkv+FrM0fG2Rhp3hoP8P9T oi9qHbmwpJg6K3J50Yy+PdzBC8bAP7DF3gDm6VXHpNfi/PG9YwNlZQsU7N0hIWDHzUEV8nIKPpO RdApmWbrb+D40Z5WFJ6g0YUdhbSw4ELBX8jzzv09SZoH+N0maT5WcsT41clO12HrPfgk34/wDVw aj40efKV+6Hl50e0+c8eAoP2fxcCKGZX9sqDTJb5v1HP7mzVl7GQ6HplrV6zytTn0W5RRo6c2Tf msBzwvInfv/HX/PpNTdXBINFfN//j1VJq9CrvVHUcIyLT2eOK0wMyDpdrxFqDu0HH4cM8qjB+fm 9mWZK+oh X-Proofpoint-ORIG-GUID: Xs7WaiZhO0jIyDMYWySz7t3iesq7jDsX X-Proofpoint-GUID: Xs7WaiZhO0jIyDMYWySz7t3iesq7jDsX X-Authority-Analysis: v=2.4 cv=StCQ6OO0 c=1 sm=1 tr=0 ts=68c315e7 cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=yJojWOMRYYMA:10 a=VnNF1IyMAAAA:8 a=fYhOAy_zjFLNdzqrsEIA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-11_03,2025-09-11_02,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 malwarescore=0 bulkscore=0 clxscore=1011 adultscore=0 suspectscore=0 priorityscore=1501 impostorscore=0 phishscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509060195 Content-Type: text/plain; charset="utf-8" The current reset process saves the device's config space state before reset and restores it afterward. However, when a device is in an error state before reset, config space reads may return error values instead of valid data. This results in saving corrupted values that get written back to the device during state restoration. Avoid saving the state of the config space when the device is in error. While restoring we only restorei the state that can be restored through kernel data such as BARs or doesn't depend on the saved state. Signed-off-by: Farhan Ali --- drivers/pci/pci.c | 29 ++++++++++++++++++++++++++--- drivers/pci/pcie/aer.c | 5 +++++ drivers/pci/pcie/dpc.c | 5 +++++ drivers/pci/pcie/ptm.c | 5 +++++ drivers/pci/tph.c | 5 +++++ drivers/pci/vc.c | 5 +++++ 6 files changed, 51 insertions(+), 3 deletions(-) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index b0f4d98036cd..4b67d22faf0a 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -1720,6 +1720,11 @@ static void pci_restore_pcie_state(struct pci_dev *d= ev) struct pci_cap_saved_state *save_state; u16 *cap; =20 + if (!dev->state_saved) { + pci_warn(dev, "Not restoring pcie state, no saved state"); + return; + } + /* * Restore max latencies (in the LTR capability) before enabling * LTR itself in PCI_EXP_DEVCTL2. @@ -1775,6 +1780,11 @@ static void pci_restore_pcix_state(struct pci_dev *d= ev) struct pci_cap_saved_state *save_state; u16 *cap; =20 + if (!dev->state_saved) { + pci_warn(dev, "Not restoring pcix state, no saved state"); + return; + } + save_state =3D pci_find_saved_cap(dev, PCI_CAP_ID_PCIX); pos =3D pci_find_capability(dev, PCI_CAP_ID_PCIX); if (!save_state || !pos) @@ -1792,6 +1802,14 @@ static void pci_restore_pcix_state(struct pci_dev *d= ev) int pci_save_state(struct pci_dev *dev) { int i; + u16 val; + + pci_read_config_word(dev, PCI_DEVICE_ID, &val); + if (PCI_POSSIBLE_ERROR(val)) { + pci_warn(dev, "Device in error, not saving config space state\n"); + return -EIO; + } + /* XXX: 100% dword access ok here? */ for (i =3D 0; i < 16; i++) { pci_read_config_dword(dev, i * 4, &dev->saved_config_space[i]); @@ -1854,6 +1872,14 @@ static void pci_restore_config_space_range(struct pc= i_dev *pdev, =20 static void pci_restore_config_space(struct pci_dev *pdev) { + if (!pdev->state_saved) { + pci_warn(pdev, "No saved config space, restoring BARs\n"); + pci_restore_bars(pdev); + pci_write_config_word(pdev, PCI_COMMAND, + PCI_COMMAND_MEMORY | PCI_COMMAND_IO); + return; + } + if (pdev->hdr_type =3D=3D PCI_HEADER_TYPE_NORMAL) { pci_restore_config_space_range(pdev, 10, 15, 0, false); /* Restore BARs before the command register. */ @@ -1906,9 +1932,6 @@ static void pci_restore_rebar_state(struct pci_dev *p= dev) */ void pci_restore_state(struct pci_dev *dev) { - if (!dev->state_saved) - return; - pci_restore_pcie_state(dev); pci_restore_pasid_state(dev); pci_restore_pri_state(dev); diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c index e286c197d716..dca3502ef669 100644 --- a/drivers/pci/pcie/aer.c +++ b/drivers/pci/pcie/aer.c @@ -361,6 +361,11 @@ void pci_restore_aer_state(struct pci_dev *dev) if (!aer) return; =20 + if (!dev->state_saved) { + pci_warn(dev, "Not restoring aer state, no saved state"); + return; + } + save_state =3D pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_ERR); if (!save_state) return; diff --git a/drivers/pci/pcie/dpc.c b/drivers/pci/pcie/dpc.c index fc18349614d7..62c520af71a7 100644 --- a/drivers/pci/pcie/dpc.c +++ b/drivers/pci/pcie/dpc.c @@ -67,6 +67,11 @@ void pci_restore_dpc_state(struct pci_dev *dev) if (!pci_is_pcie(dev)) return; =20 + if (!dev->state_saved) { + pci_warn(dev, "Not restoring dpc state, no saved state"); + return; + } + save_state =3D pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_DPC); if (!save_state) return; diff --git a/drivers/pci/pcie/ptm.c b/drivers/pci/pcie/ptm.c index 65e4b008be00..7b5bcc23000d 100644 --- a/drivers/pci/pcie/ptm.c +++ b/drivers/pci/pcie/ptm.c @@ -112,6 +112,11 @@ void pci_restore_ptm_state(struct pci_dev *dev) if (!ptm) return; =20 + if (!dev->state_saved) { + pci_warn(dev, "Not restoring ptm state, no saved state"); + return; + } + save_state =3D pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_PTM); if (!save_state) return; diff --git a/drivers/pci/tph.c b/drivers/pci/tph.c index cc64f93709a4..f0f1bae46736 100644 --- a/drivers/pci/tph.c +++ b/drivers/pci/tph.c @@ -435,6 +435,11 @@ void pci_restore_tph_state(struct pci_dev *pdev) if (!pdev->tph_enabled) return; =20 + if (!pdev->state_saved) { + pci_warn(pdev, "Not restoring tph state, no saved state"); + return; + } + save_state =3D pci_find_saved_ext_cap(pdev, PCI_EXT_CAP_ID_TPH); if (!save_state) return; diff --git a/drivers/pci/vc.c b/drivers/pci/vc.c index a4ff7f5f66dd..fda435cd49c1 100644 --- a/drivers/pci/vc.c +++ b/drivers/pci/vc.c @@ -391,6 +391,11 @@ void pci_restore_vc_state(struct pci_dev *dev) { int i; =20 + if (!dev->state_saved) { + pci_warn(dev, "Not restoring vc state, no saved state"); + return; + } + for (i =3D 0; i < ARRAY_SIZE(vc_caps); i++) { int pos; struct pci_cap_saved_state *save_state; --=20 2.43.0 From nobody Thu Oct 2 19:27:06 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D783C36C06A; Thu, 11 Sep 2025 18:33:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757615598; cv=none; b=jlWRnYptbP2I19hVZGKY4g5AubQEHATYw6Lv/tEZIp62InMVXzXw3DHgK0NGMCCTgnYqSa+iN9M+p6uoarEm6+KLvL4yYml67Nq87nHYaz4UNH8kHfiz3gMxs7cpalvE8GGWpuAWcMBPOce1CS5zxk5fyGglhT8jW2nX0Lcb26E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757615598; c=relaxed/simple; bh=CRFypiDshZQc5tqdcgYif87R8KFZSyq2U5WddA5Orck=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=NtC5iLW8uTj2Uka/KpglTXNRMFyMPaXWKCjy36iQBRjrzxlhP75FaT8LKpAKk6yX9U9l98yx3h22r7wd/A9q4uTUAwps+7tLjQxKSxz+Szd/L+gxsRiRcx3EP7whPsZ8Tmf+nY/WcxI2e7JIvyXQGEq21ZtipCzv3gpmNKyGmAc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=jEdu7mJz; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="jEdu7mJz" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58BDTN9P025033; Thu, 11 Sep 2025 18:33:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=zjZiV8Hj6I7WyQ3jH AOZSNNkP5uu8kVo+lPuflgkoiM=; b=jEdu7mJz+6qBMGhIwDvEixjdYDXWZiWqC drc3vHFOWBjbWUbv1OnC4j3pCwtlLEheMUkoWyPikKsGiWUGxwjq27xYn3TGf9H3 XzrijGeEQX+zIOLlGzlao/WAxhOX/DcQmtPsw+dnmlUh2UZHpCClNOX5Wg39Uv9v 9Zh2cTVX63/h8ZcQDLJ3vicoob56MN7pgST4kLYwn5Papls9xNnPZbn57R3WXoWK Lgn6sxxTcoPt8SEFVLsi6B/ghavIg7jWaHJBIxvV/SD5PN4UDcxInq8vVgHEGXTS mumtpnHo3FBUgG/7ch9TOhPu6xAqTLLPTMX1WkBWyb5rpS2GG1fqg== Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490xydbgbj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 18:33:12 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58BHZtOL010613; Thu, 11 Sep 2025 18:33:11 GMT Received: from smtprelay06.dal12v.mail.ibm.com ([172.16.1.8]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4910sn76nj-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 18:33:11 +0000 Received: from smtpav01.dal12v.mail.ibm.com (smtpav01.dal12v.mail.ibm.com [10.241.53.100]) by smtprelay06.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58BIXAHS26673806 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Sep 2025 18:33:10 GMT Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 16DF958057; Thu, 11 Sep 2025 18:33:10 +0000 (GMT) Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5597F58058; Thu, 11 Sep 2025 18:33:09 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.249.32]) by smtpav01.dal12v.mail.ibm.com (Postfix) with ESMTP; Thu, 11 Sep 2025 18:33:09 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v3 02/10] PCI: Add additional checks for flr reset Date: Thu, 11 Sep 2025 11:32:59 -0700 Message-ID: <20250911183307.1910-3-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250911183307.1910-1-alifm@linux.ibm.com> References: <20250911183307.1910-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: 6WPPhbVhyCMxPWi0itOdS486Ru7FlZPM X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTA2MDIzNSBTYWx0ZWRfX7ACWFLMwwG3t L9k6MKZb/VEg9/Z/1lZ2NsbKAdA5nBVwXAnQlGcLEdMPt//II/r4tVkbS3MYdfqb+/GybzLOvk1 ad6VqMPxx2YY+YQij53dQi1gZhA0Q/EWUk2PYOq2ymdI6fimRPFIhwicd6hnjTgb7EFZcmik9w0 uMwl2aCI0dapDcAGNlR21/QyTg3/GAN0huCyY5ILaJgFkdxh3NmTQ5j8eUMZl1PZkx82FOn8J36 Yn5Ag9pxUOZVPByhHCIqGGp0xqbQYAERIr5KZAS4Y7RQ4nKJfsalndQXGlhx3YUkIc9iI3JZ9QC Dqp0QVZ2BbFeFWdd2XSiNTD6AohwMlm2iYTgrfBG0sR+i+t/qxYF/5SnCRGwJx7hhdswsnaw7P/ LRu8GKj4 X-Proofpoint-GUID: 6WPPhbVhyCMxPWi0itOdS486Ru7FlZPM X-Authority-Analysis: v=2.4 cv=F59XdrhN c=1 sm=1 tr=0 ts=68c315e8 cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=yJojWOMRYYMA:10 a=VnNF1IyMAAAA:8 a=8GLQ0EcgPjaQrAEZ2hkA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-11_03,2025-09-11_02,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 clxscore=1011 impostorscore=0 bulkscore=0 adultscore=0 spamscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509060235 Content-Type: text/plain; charset="utf-8" If a device is in an error state, then any reads of device registers can return error value. Add addtional checks to validate if a device is in an error state before doing an flr reset. Signed-off-by: Farhan Ali --- drivers/pci/pci.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 4b67d22faf0a..3994fa82df68 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -4580,12 +4580,19 @@ EXPORT_SYMBOL_GPL(pcie_flr); */ int pcie_reset_flr(struct pci_dev *dev, bool probe) { + u32 reg; + if (dev->dev_flags & PCI_DEV_FLAGS_NO_FLR_RESET) return -ENOTTY; =20 if (!(dev->devcap & PCI_EXP_DEVCAP_FLR)) return -ENOTTY; =20 + if (pcie_capability_read_dword(dev, PCI_EXP_DEVCAP, ®)) { + pci_warn(dev, "Device unable to do an FLR\n"); + return -ENOTTY; + } + if (probe) return 0; =20 --=20 2.43.0 From nobody Thu Oct 2 19:27:06 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4BC5C36C07F; Thu, 11 Sep 2025 18:33:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757615599; cv=none; b=Zo4BtzpQe8+zk42nslf/al2CvORBvdlNQul900d4IImb2+9SvirP2ZNiUPiBePpdxBnbRxtvw6pHcfHREJUcFWmwFX1y/zGHmurTvKLBF1Ah9oWl0DZ9Ni6A22nEjZPt6KJxTap0lsvS14A4/tFku7JCDavSrIZJfNg61qBiWOE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757615599; c=relaxed/simple; bh=A9xi35BB90o7/iR3ITPX43udqTkT3mvfgeueyjrnk3Y=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JqtBRgreXMZpDqx2cxHaWI0p2Y9RReqNeR13/MvbuIlqE8zshSQSP3rfYZTyztepbJyS9BIsXxUfUZEwYDfDi1ED9JjmPYHslvcmOZix6YVo4eW6jmWr9jyI+pswvj+Q4JvOIhJfNb51MqPpqJADE063moSD2uwPQov7I2BX2aE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=qvIhTzhf; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="qvIhTzhf" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58BEBo59024558; Thu, 11 Sep 2025 18:33:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=IiIsVKX9sgPnPaf1w huJxpybFaYB+X/PfUp+yLazCiE=; b=qvIhTzhf9+vR+dJY4ZC4b7y61nlWdeCm0 sokX1+D/EGxTw/W9mToO540S13VpMnI0vHuy+NVZp5D0rhYCIbXyQN+btf8idnqx H4g+JeVexCHJyhmq/y/LqhPMrmoC1Xnm/GxKA+eornny+72SWzsyKuHzFmeacxG0 kqjt8KS+o5uto3yRCHwkBRY/Iw0CJgqxPuUchahC1rLjMBd04d/lwFpcwKpmMk8Z KnUK6MYY7hyCGsPbpg6WQP7niEg1HzgLEAXq+kHqbA+Qxh81SGYqfjq9fa32yp6J BvePu17g5YSOdXigrUI8mMSufyBgL7UZn6FS9QAzoSMYNLN2T0fwA== Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490cmx6n90-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 18:33:12 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58BI6ZvG017218; Thu, 11 Sep 2025 18:33:12 GMT Received: from smtprelay06.dal12v.mail.ibm.com ([172.16.1.8]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 4911gmq2c6-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 18:33:11 +0000 Received: from smtpav01.dal12v.mail.ibm.com (smtpav01.dal12v.mail.ibm.com [10.241.53.100]) by smtprelay06.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58BIXAV326673812 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Sep 2025 18:33:11 GMT Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D69D158059; Thu, 11 Sep 2025 18:33:10 +0000 (GMT) Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 32D7158058; Thu, 11 Sep 2025 18:33:10 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.249.32]) by smtpav01.dal12v.mail.ibm.com (Postfix) with ESMTP; Thu, 11 Sep 2025 18:33:10 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v3 03/10] PCI: Allow per function PCI slots Date: Thu, 11 Sep 2025 11:33:00 -0700 Message-ID: <20250911183307.1910-4-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250911183307.1910-1-alifm@linux.ibm.com> References: <20250911183307.1910-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: HQVFu9CRau-j72qfXJG45PcMar8KGwTU X-Proofpoint-ORIG-GUID: HQVFu9CRau-j72qfXJG45PcMar8KGwTU X-Authority-Analysis: v=2.4 cv=J52q7BnS c=1 sm=1 tr=0 ts=68c315e8 cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=yJojWOMRYYMA:10 a=VnNF1IyMAAAA:8 a=K-1LnuYzVcpd1HOvht4A:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTA2MDAyNSBTYWx0ZWRfX/cuZaSKBQIp5 FzXTCQNI9zNbt2jwzsK+12hL4Y9IqSdhJUklaouWCV/X8iaCqym33DlXa1OxczZiV6yHHHdnwAU 8QSLk+2qAlC4Ul4SnX3QTGbhcWiOo8GEVGJmSxHRXLZ1Oz0Ru9V4spqtGkQ9lo7NpDHVTzBZE8S GsHBFtA3dlnTHQmKjBRaist6zh39btovCX9Tgn5ONyWNkZign5jv35uWlsOYuBPTYYz/+t5Nilf LsANlxNybyUB2ZmUHEy7qBRL6mDEGqCwhthtJma9MrsxZJVB23KpsGIuyb3ydxS71EXXU0hyEG3 DUUoBM1oyc8Gp4F9Ti1pSY1oDqsj4C0oMlrWoqqxLMyfMkSHC4qQ/iJVFMTAHc1OVbkTHagvKEB SkeSR2NV X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-11_03,2025-09-11_02,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 clxscore=1015 suspectscore=0 spamscore=0 phishscore=0 bulkscore=0 adultscore=0 malwarescore=0 priorityscore=1501 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509060025 Content-Type: text/plain; charset="utf-8" On s390 systems, which use a machine level hypervisor, PCI devices are always accessed through a form of PCI pass-through which fundamentally operates on a per PCI function granularity. This is also reflected in the s390 PCI hotplug driver which creates hotplug slots for individual PCI functions. Its reset_slot() function, which is a wrapper for zpci_hot_reset_device(), thus also resets individual functions. Currently, the kernel's PCI_SLOT() macro assigns the same pci_slot object to multifunction devices. This approach worked fine on s390 systems that only exposed virtual functions as individual PCI domains to the operating system. Since commit 44510d6fa0c0 ("s390/pci: Handling multifunctions") s390 supports exposing the topology of multifunction PCI devices by grouping them in a shared PCI domain. When attempting to reset a function through the hotplug driver, the shared slot assignment causes the wrong function to be reset instead of the intended one. It also leaks memory as we do create a pci_slot object for the function, but don't correctly free it in pci_slot_release(). Add a flag for struct pci_slot to allow per function PCI slots for functions managed through a hypervisor, which exposes individual PCI functions while retaining the topology. Fixes: 44510d6fa0c0 ("s390/pci: Handling multifunctions") Suggested-by: Niklas Schnelle Signed-off-by: Farhan Ali --- drivers/pci/hotplug/s390_pci_hpc.c | 10 ++++++++-- drivers/pci/pci.c | 4 +++- drivers/pci/slot.c | 14 +++++++++++--- include/linux/pci.h | 1 + 4 files changed, 23 insertions(+), 6 deletions(-) diff --git a/drivers/pci/hotplug/s390_pci_hpc.c b/drivers/pci/hotplug/s390_= pci_hpc.c index d9996516f49e..8b547de464bf 100644 --- a/drivers/pci/hotplug/s390_pci_hpc.c +++ b/drivers/pci/hotplug/s390_pci_hpc.c @@ -126,14 +126,20 @@ static const struct hotplug_slot_ops s390_hotplug_slo= t_ops =3D { =20 int zpci_init_slot(struct zpci_dev *zdev) { + int ret; char name[SLOT_NAME_SIZE]; struct zpci_bus *zbus =3D zdev->zbus; =20 zdev->hotplug_slot.ops =3D &s390_hotplug_slot_ops; =20 snprintf(name, SLOT_NAME_SIZE, "%08x", zdev->fid); - return pci_hp_register(&zdev->hotplug_slot, zbus->bus, - zdev->devfn, name); + ret =3D pci_hp_register(&zdev->hotplug_slot, zbus->bus, + zdev->devfn, name); + if (ret) + return ret; + + zdev->hotplug_slot.pci_slot->per_func_slot =3D 1; + return 0; } =20 void zpci_exit_slot(struct zpci_dev *zdev) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 3994fa82df68..70296d3b1cfc 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -5061,7 +5061,9 @@ static int pci_reset_hotplug_slot(struct hotplug_slot= *hotplug, bool probe) =20 static int pci_dev_reset_slot_function(struct pci_dev *dev, bool probe) { - if (dev->multifunction || dev->subordinate || !dev->slot || + if (dev->multifunction && !dev->slot->per_func_slot) + return -ENOTTY; + if (dev->subordinate || !dev->slot || dev->dev_flags & PCI_DEV_FLAGS_NO_BUS_RESET) return -ENOTTY; =20 diff --git a/drivers/pci/slot.c b/drivers/pci/slot.c index 50fb3eb595fe..51ee59e14393 100644 --- a/drivers/pci/slot.c +++ b/drivers/pci/slot.c @@ -63,6 +63,14 @@ static ssize_t cur_speed_read_file(struct pci_slot *slot= , char *buf) return bus_speed_read(slot->bus->cur_bus_speed, buf); } =20 +static bool pci_dev_matches_slot(struct pci_dev *dev, struct pci_slot *slo= t) +{ + if (slot->per_func_slot) + return dev->devfn =3D=3D slot->number; + + return PCI_SLOT(dev->devfn) =3D=3D slot->number; +} + static void pci_slot_release(struct kobject *kobj) { struct pci_dev *dev; @@ -73,7 +81,7 @@ static void pci_slot_release(struct kobject *kobj) =20 down_read(&pci_bus_sem); list_for_each_entry(dev, &slot->bus->devices, bus_list) - if (PCI_SLOT(dev->devfn) =3D=3D slot->number) + if (pci_dev_matches_slot(dev, slot)) dev->slot =3D NULL; up_read(&pci_bus_sem); =20 @@ -166,7 +174,7 @@ void pci_dev_assign_slot(struct pci_dev *dev) =20 mutex_lock(&pci_slot_mutex); list_for_each_entry(slot, &dev->bus->slots, list) - if (PCI_SLOT(dev->devfn) =3D=3D slot->number) + if (pci_dev_matches_slot(dev, slot)) dev->slot =3D slot; mutex_unlock(&pci_slot_mutex); } @@ -285,7 +293,7 @@ struct pci_slot *pci_create_slot(struct pci_bus *parent= , int slot_nr, =20 down_read(&pci_bus_sem); list_for_each_entry(dev, &parent->devices, bus_list) - if (PCI_SLOT(dev->devfn) =3D=3D slot_nr) + if (pci_dev_matches_slot(dev, slot)) dev->slot =3D slot; up_read(&pci_bus_sem); =20 diff --git a/include/linux/pci.h b/include/linux/pci.h index 59876de13860..9265f32d9786 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -78,6 +78,7 @@ struct pci_slot { struct list_head list; /* Node in list of slots */ struct hotplug_slot *hotplug; /* Hotplug info (move here) */ unsigned char number; /* PCI_SLOT(pci_dev->devfn) */ + unsigned int per_func_slot:1; /* Allow per function slot */ struct kobject kobj; }; =20 --=20 2.43.0 From nobody Thu Oct 2 19:27:06 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id F1AC536CC63; Thu, 11 Sep 2025 18:33:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757615599; cv=none; b=fZ3htuKBkR0FeaMIV48V31NpF2FnghhyxV0a5C7yvujMe4nQWE5waBOsVXTN7jMwebnvWsMdWt7nKplfh+W9vDDKctpYoQh0nGx0Yefy5HK5p1cxZ3QR8xRWyWMQpdOvcRNhkBlbsVeL8VE2Ph6xDBPVEKKvi7hLhO74gh2D4HI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757615599; c=relaxed/simple; bh=4D0+y0nuAv/oMRS2SJ1i+o2tSrq0Xli8o900mxZzDyo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=BWLA/C+R3fEIP4D7AbocDML5Zmyt+hqUn35vqxHDtk+Brw7aa10ripTZK8O498KJ53SmV/Ye4BCsmodaEdzLXot1DhebUwdurnVSvYRnJ4pgzbXCH47IolO5XHjn5xX1R6+5ut8U66A8h7nEkfjvWGJdmZXqrfsr46hCdoBeNA8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=JfFgvphP; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="JfFgvphP" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58BDlHmq002811; Thu, 11 Sep 2025 18:33:13 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=wwtqQl9dJaE0sfRmk 00O2pgfEIsW+XaywO1jJiYJhJQ=; b=JfFgvphPdLWN6VgaN6vH8gtmzT84TxetK eVqURP/pP0j9VrNqK9Nst8ufY0ozw1MfdyIzyI7UOoLxi3RX47yxcxt4tNLTmjFb Tq2y9QSE2MqmQ5epuU76iufO02jparlz8aneAM7Bgg8eoTVuEl0Uz1NXYfiql5Lf +97mDXgIdGbHDmy/rRoSBFzXbLiK7zAFXaHR/WVvYHgXxl3W2pkl5T9ataH1EFD1 ouoXlTjOmIXichG8ekNmNoBi4c9ldm1RC4Q83avVtL8KEgGIQfWveAbqHe43Ro6I wTxyjREtGi0DZm8LGS9DZKlmEi6YymthhBehqeGQhDjWg+LVecLFQ== Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490cffpp7h-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 18:33:13 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58BI6ZvH017218; Thu, 11 Sep 2025 18:33:12 GMT Received: from smtprelay07.dal12v.mail.ibm.com ([172.16.1.9]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 4911gmq2c7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 18:33:12 +0000 Received: from smtpav01.dal12v.mail.ibm.com (smtpav01.dal12v.mail.ibm.com [10.241.53.100]) by smtprelay07.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58BIXBt332375370 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Sep 2025 18:33:11 GMT Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A2AD258057; Thu, 11 Sep 2025 18:33:11 +0000 (GMT) Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id F243958058; Thu, 11 Sep 2025 18:33:10 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.249.32]) by smtpav01.dal12v.mail.ibm.com (Postfix) with ESMTP; Thu, 11 Sep 2025 18:33:10 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v3 04/10] s390/pci: Add architecture specific resource/bus address translation Date: Thu, 11 Sep 2025 11:33:01 -0700 Message-ID: <20250911183307.1910-5-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250911183307.1910-1-alifm@linux.ibm.com> References: <20250911183307.1910-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: 4Oz9sv7fwmDOIfBb-aVG6k-2PPrB1_Vv X-Proofpoint-GUID: 4Oz9sv7fwmDOIfBb-aVG6k-2PPrB1_Vv X-Authority-Analysis: v=2.4 cv=EYDIQOmC c=1 sm=1 tr=0 ts=68c315e9 cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=yJojWOMRYYMA:10 a=VnNF1IyMAAAA:8 a=yYx7OUe3D3zVL1aRsq0A:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTA2MDAyMCBTYWx0ZWRfXxompx4eB6Q1R e2kSJHEG5B456HbYZ7y0KWygZjOhwTIuFhtBDNF2UHPUljltFhBAbdvh4PTGjTvrdzWAXEWIb5q Nx2yn1fqHApB2tbhQGAG3gJeFQuAwzkW6ovrBK4sC7Srm/bXTlvrOSyVVwRPmDzKS8QctWL3SIf siH/FFxIHBpvL65AKJf5BGmSdIBWg2zSFD7g36+i5Dlf6rtpcT7s2T1rnvnD1OTGzovA10MSnxk Tg+c3ze5spU7rZPR5rqovKNUrVj2Suv09NXWL3z/TUA/gVZOBlypNqIBACJ4T6C3butRV8P5SBt dSwTUD3xuDfdafAICQacas+4L+g4Pij2TMm0f5LpfeohH28FPlbIdt4MUtr+9ecxGyA/9Ys/As0 sH5GYe2P X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-11_03,2025-09-11_02,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 adultscore=0 suspectscore=0 spamscore=0 impostorscore=0 priorityscore=1501 phishscore=0 clxscore=1011 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509060020 Content-Type: text/plain; charset="utf-8" On s390 today we overwrite the PCI BAR resource address to either an artificial cookie address or MIO address. However this address is different from the bus address of the BARs programmed by firmware. The artificial cookie address was created to index into an array of function handles (zpci_iomap_start). The MIO (mapped I/O) addresses are provided by firmware but maybe different from the bus address. This creates an issue when trying to convert the BAR resource address to bus address using the generic pcibios_resource_to_bus. Implement an architecture specific pcibios_resource_to_bus function to correctly translate PCI BAR resource address to bus address for s390. Similarly add architecture specific pcibios_bus_to_resource function to do the reverse translation. Signed-off-by: Farhan Ali --- arch/s390/pci/pci.c | 73 +++++++++++++++++++++++++++++++++++++++ drivers/pci/host-bridge.c | 4 +-- 2 files changed, 75 insertions(+), 2 deletions(-) diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c index cd6676c2d602..5baeb5f6f674 100644 --- a/arch/s390/pci/pci.c +++ b/arch/s390/pci/pci.c @@ -264,6 +264,79 @@ resource_size_t pcibios_align_resource(void *data, con= st struct resource *res, return 0; } =20 +void pcibios_resource_to_bus(struct pci_bus *bus, struct pci_bus_region *r= egion, + struct resource *res) +{ + struct zpci_bus *zbus =3D bus->sysdata; + struct zpci_bar_struct *zbar; + struct zpci_dev *zdev; + + region->start =3D res->start; + region->end =3D res->end; + + for (int i =3D 0; i < ZPCI_FUNCTIONS_PER_BUS; i++) { + int j =3D 0; + + zbar =3D NULL; + zdev =3D zbus->function[i]; + if (!zdev) + continue; + + for (j =3D 0; j < PCI_STD_NUM_BARS; j++) { + if (zdev->bars[j].res->start =3D=3D res->start && + zdev->bars[j].res->end =3D=3D res->end) { + zbar =3D &zdev->bars[j]; + break; + } + } + + if (zbar) { + /* only MMIO is supported */ + region->start =3D zbar->val & PCI_BASE_ADDRESS_MEM_MASK; + if (zbar->val & PCI_BASE_ADDRESS_MEM_TYPE_64) + region->start |=3D (u64)zdev->bars[j + 1].val << 32; + + region->end =3D region->start + (1UL << zbar->size) - 1; + return; + } + } +} + +void pcibios_bus_to_resource(struct pci_bus *bus, struct resource *res, + struct pci_bus_region *region) +{ + struct zpci_bus *zbus =3D bus->sysdata; + struct zpci_dev *zdev; + resource_size_t start, end; + + res->start =3D region->start; + res->end =3D region->end; + + for (int i =3D 0; i < ZPCI_FUNCTIONS_PER_BUS; i++) { + zdev =3D zbus->function[i]; + if (!zdev || !zdev->has_resources) + continue; + + for (int j =3D 0; j < PCI_STD_NUM_BARS; j++) { + if (!zdev->bars[j].val && !zdev->bars[j].size) + continue; + + /* only MMIO is supported */ + start =3D zdev->bars[j].val & PCI_BASE_ADDRESS_MEM_MASK; + if (zdev->bars[j].val & PCI_BASE_ADDRESS_MEM_TYPE_64) + start |=3D (u64)zdev->bars[j + 1].val << 32; + + end =3D start + (1UL << zdev->bars[j].size) - 1; + + if (start =3D=3D region->start && end =3D=3D region->end) { + res->start =3D zdev->bars[j].res->start; + res->end =3D zdev->bars[j].res->end; + return; + } + } + } +} + void __iomem *ioremap_prot(phys_addr_t phys_addr, size_t size, pgprot_t prot) { diff --git a/drivers/pci/host-bridge.c b/drivers/pci/host-bridge.c index afa50b446567..56d62afb3afe 100644 --- a/drivers/pci/host-bridge.c +++ b/drivers/pci/host-bridge.c @@ -48,7 +48,7 @@ void pci_set_host_bridge_release(struct pci_host_bridge *= bridge, } EXPORT_SYMBOL_GPL(pci_set_host_bridge_release); =20 -void pcibios_resource_to_bus(struct pci_bus *bus, struct pci_bus_region *r= egion, +void __weak pcibios_resource_to_bus(struct pci_bus *bus, struct pci_bus_re= gion *region, struct resource *res) { struct pci_host_bridge *bridge =3D pci_find_host_bridge(bus); @@ -73,7 +73,7 @@ static bool region_contains(struct pci_bus_region *region= 1, return region1->start <=3D region2->start && region1->end >=3D region2->e= nd; } =20 -void pcibios_bus_to_resource(struct pci_bus *bus, struct resource *res, +void __weak pcibios_bus_to_resource(struct pci_bus *bus, struct resource *= res, struct pci_bus_region *region) { struct pci_host_bridge *bridge =3D pci_find_host_bridge(bus); --=20 2.43.0 From nobody Thu Oct 2 19:27:06 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 42E7F36CC9C; Thu, 11 Sep 2025 18:33:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757615600; cv=none; b=QZC0zCZ9YKfOJ5AAznuxb12iLvx005H1Em8AifZv0oS+hSkrqbZ9tGLg4t59VO+yvA2f4QeiZ/DeDpF8iMZw/cCGvaLY5u99PiLvmXJief3Xv67xEgv7nx5QA5Rl4v3woxb0X9XLvioXFsN3THj+8olVox0bd+bjNz1LP9wfaVs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757615600; c=relaxed/simple; bh=BE/Fyb7bqjXOtPStqy41nzTS3JQmUv2CPtP6DDV37Ag=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=K0u12BTU1FpeToJQZ9Cm8UiWs8VwpsLlIOUlD+mRRdfOHk9P941oWDKAGE2Tk6FcRWN21nV+O5VxrMoM6J1bq/N9cQrtsglSNI4fom40djMlDR22WfL8pElf4HQgS5OTklbGd3X+LkH5HTQHVRi8PyRWVlyQjBa29bEeBS4Kx3Q= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=XwzImf9K; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="XwzImf9K" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58BDRAnK015943; Thu, 11 Sep 2025 18:33:15 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=Gtdx3YR1RuO0lD7+S BIUxw8c1f6dMRxwCpmb72Rlu48=; b=XwzImf9KBb0bS4gUSRcXEHZwa3BXhIgk0 QvPCMCtldmxWHn/AFuHQYVZ2TqbWcJABdqQe3i41SQ12DbjJdW70I4UNWCq3tGxN 5VCW9FsnaQEQGfj1Cq2hjr9JJagWQYtUm/vSxso4+vnW81HTq+E3rYJ0CF6qd2Z9 x7UWWADTBYS9u1FMJEpBLGS+FDH9XY9g8bkjai2/56vox0tsXL/E3UDFvOCdOtEe 8iZd8yLAJs1DRw/lcz7pVZkSx3bS1Sg/gkx3JfQWMM160aYbN6JuoEe8T9VZemYb 3PIvYXmXk2ZwcU25tatLZ+wfxBN2Xm/bd1jVsnIfZ/T/lFmbkKfFw== Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490cffpp7q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 18:33:15 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58BHNMPl007982; Thu, 11 Sep 2025 18:33:14 GMT Received: from smtprelay01.wdc07v.mail.ibm.com ([172.16.1.68]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 49109py8me-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 18:33:14 +0000 Received: from smtpav01.dal12v.mail.ibm.com (smtpav01.dal12v.mail.ibm.com [10.241.53.100]) by smtprelay01.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58BIXCjf33620234 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Sep 2025 18:33:12 GMT Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 733975805D; Thu, 11 Sep 2025 18:33:12 +0000 (GMT) Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BFE7558058; Thu, 11 Sep 2025 18:33:11 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.249.32]) by smtpav01.dal12v.mail.ibm.com (Postfix) with ESMTP; Thu, 11 Sep 2025 18:33:11 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v3 05/10] s390/pci: Restore IRQ unconditionally for the zPCI device Date: Thu, 11 Sep 2025 11:33:02 -0700 Message-ID: <20250911183307.1910-6-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250911183307.1910-1-alifm@linux.ibm.com> References: <20250911183307.1910-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: a-e4gsY4stPXg38BSndp-JldLYl69B9F X-Proofpoint-GUID: a-e4gsY4stPXg38BSndp-JldLYl69B9F X-Authority-Analysis: v=2.4 cv=EYDIQOmC c=1 sm=1 tr=0 ts=68c315eb cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=yJojWOMRYYMA:10 a=VnNF1IyMAAAA:8 a=ove13Onh8FEOJJhcmoIA:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTA2MDAyMCBTYWx0ZWRfXwvUc8iDYaR+E q2msXi8GLaY/1oqdnLX0SoXYwKAmqNbR++uDjI01SPhqjnE8TOEvILWkAkjkr18yDb6biepOrVT HFLHtM9wGFyReY6GWr7ZuK7U80Xfp4S3XR1PS+olsLOg4ux1qZxN+zTBZjwjQXPLVZd56Zzc9Ic zM+tU2oEpaUoXPx1R4U3wNxEdjw/pdymVqbHNK5IC6bojhLjYeoU1YfIyI+170pLJahx51LNeOb QmYZEkXecrtra6zQ/2+uGDWCKVA7yj4rKjzfRiZAhP6eLIU+2xfqJQ42iyL0uMyRgMmic109BCb Xc9GcDXqucF3YcuMp2YpnhK1at3LsZOMnhSksuhFxpyQ2bTVm42cT6k3X6gvkpMZOYml+glhgPV 7GFKnP3+ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-11_03,2025-09-11_02,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 adultscore=0 suspectscore=0 spamscore=0 impostorscore=0 priorityscore=1501 phishscore=0 clxscore=1015 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509060020 Content-Type: text/plain; charset="utf-8" Commit c1e18c17bda6 ("s390/pci: add zpci_set_irq()/zpci_clear_irq()"), introduced the zpci_set_irq() and zpci_clear_irq(), to be used while resetting a zPCI device. Commit da995d538d3a ("s390/pci: implement reset_slot for hotplug slot"), mentions zpci_clear_irq() being called in the path for zpci_hot_reset_devic= e(). But that is not the case anymore and these functions are not called outside of this file. However after a CLP disable/enable reset, the device's IRQ are unregistered, but the flag zdev->irq_registered does not get cleared. It creates an inconsistent state and so arch_restore_msi_irqs() doesn't correctly restore the device's IRQ. This becomes a problem when a PCI driver tries to restore the state of the device through pci_restore_state(). Restore IRQ unconditionally for the device and remove the irq_registered flag as its redundant. Signed-off-by: Farhan Ali Reviewed-by: Niklas Schnelle --- arch/s390/include/asm/pci.h | 1 - arch/s390/pci/pci_irq.c | 9 +-------- 2 files changed, 1 insertion(+), 9 deletions(-) diff --git a/arch/s390/include/asm/pci.h b/arch/s390/include/asm/pci.h index 41f900f693d9..aed19a1aa9d7 100644 --- a/arch/s390/include/asm/pci.h +++ b/arch/s390/include/asm/pci.h @@ -145,7 +145,6 @@ struct zpci_dev { u8 has_resources : 1; u8 is_physfn : 1; u8 util_str_avail : 1; - u8 irqs_registered : 1; u8 tid_avail : 1; u8 rtr_avail : 1; /* Relaxed translation allowed */ unsigned int devfn; /* DEVFN part of the RID*/ diff --git a/arch/s390/pci/pci_irq.c b/arch/s390/pci/pci_irq.c index 84482a921332..e73be96ce5fe 100644 --- a/arch/s390/pci/pci_irq.c +++ b/arch/s390/pci/pci_irq.c @@ -107,9 +107,6 @@ static int zpci_set_irq(struct zpci_dev *zdev) else rc =3D zpci_set_airq(zdev); =20 - if (!rc) - zdev->irqs_registered =3D 1; - return rc; } =20 @@ -123,9 +120,6 @@ static int zpci_clear_irq(struct zpci_dev *zdev) else rc =3D zpci_clear_airq(zdev); =20 - if (!rc) - zdev->irqs_registered =3D 0; - return rc; } =20 @@ -427,8 +421,7 @@ bool arch_restore_msi_irqs(struct pci_dev *pdev) { struct zpci_dev *zdev =3D to_zpci(pdev); =20 - if (!zdev->irqs_registered) - zpci_set_irq(zdev); + zpci_set_irq(zdev); return true; } =20 --=20 2.43.0 From nobody Thu Oct 2 19:27:06 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 23FE23705B0; Thu, 11 Sep 2025 18:33:19 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757615602; cv=none; b=lyFAhy6gEvi4CPfkBND51+by4nNu9MdUmeZdKjD8ila0ZsDJhn3twAQqH45hfpQSMqvyC5g6XEy+MHifzxdqnlBhfr+OgVFi9O/XTMemQqAyAnDpGfbMYvhEquGdlYP33NjogSr+f7A6YO+vR2M88ZNacnik88PAHRK+hAGoIAs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757615602; c=relaxed/simple; bh=HiF8yIXpdsl3GpcWw1U7Xg6P+sqRjiRYAkEPz33XzDQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Iu8NMcOHVaEkMW9UD+aWc/WQKDTNs5juPdw1EQsk9EVBDZhW0TkoEzR8NGqO568NX9GgEhZKF+zTM8EHqd5njrI3oBOjHelHVFLq33mp+/bzlmFdYTyt8VwIFwqP5eX9TspdWa+K3IZD1kQJO5R1Q4ffCRg7C7j0Jp8lyhxdZLA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=evVwYmY+; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="evVwYmY+" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58BFlX61017992; Thu, 11 Sep 2025 18:33:15 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=mmVlLOIe9nYcrSc3r Rudb0uYz3qoo02ifUZ1lexk/NU=; b=evVwYmY+XpcPy8hk54Jwnb5k2kgzhDcWb 9XKiEcxICJUGh4B/xst7Iu/tHXUpJycZpzweT4Uckxeoi+8kBH4NwFrkGmTmaRTA kPxQZsmEFssMC2DYM5RlvzNOEwIFrMYOkqJGC1MW3F9t56vL2ahsZ/LDl9am5PWQ +mAi+39s/aj/J/habHwm7zI4IVMEWBwUnz1XhXMCbqV9LDFw3x8XhUpRG3yAdlMQ Vkbe2oeNGxvLc+XZ6WWyewOW7FLCveJggzV0OcRvI4BlaO97BMwB3eXulmMFZHxW PatWslDLLs7Mqa4H32Iv7CaRXDT9KE4mc3AMJhVoQ6UFnpRRgRmvQ== Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490ukeu8jd-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 18:33:15 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58BHVSKV010666; Thu, 11 Sep 2025 18:33:14 GMT Received: from smtprelay02.wdc07v.mail.ibm.com ([172.16.1.69]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4910sn76np-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 18:33:14 +0000 Received: from smtpav01.dal12v.mail.ibm.com (smtpav01.dal12v.mail.ibm.com [10.241.53.100]) by smtprelay02.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58BIXDav26870456 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Sep 2025 18:33:13 GMT Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 41A0958057; Thu, 11 Sep 2025 18:33:13 +0000 (GMT) Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8F48958059; Thu, 11 Sep 2025 18:33:12 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.249.32]) by smtpav01.dal12v.mail.ibm.com (Postfix) with ESMTP; Thu, 11 Sep 2025 18:33:12 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v3 06/10] s390/pci: Update the logic for detecting passthrough device Date: Thu, 11 Sep 2025 11:33:03 -0700 Message-ID: <20250911183307.1910-7-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250911183307.1910-1-alifm@linux.ibm.com> References: <20250911183307.1910-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTA2MDE5NSBTYWx0ZWRfX45OE/jcCkoyO Rs4L9ICMbkk1DDP1gRSqU/5DOo1kMHYLigtxl7IV+xjBjqtsDLf72PGmtmJwB4AReb11+27sWW7 8RAThjivKVA+EKTYvV2Gmd56zph83ZZBc4YCBmhLzMCuowK/cJ1rTI9+7L2tBx3IxlQiGPGXPBQ hpO+7T2v4sWOH9RJTZPfNDd16Jufr0bQLBRX3S1ZJ1FoodPHcHgVcKPHmwY0+7iXuu9Bqm2SsRZ uXfpHT/aATNmRYT2zz2/zEMwaGRFnOlD1cVAMfnZnWtw13HnaBYVlb8yT6ezxmikxpM0KZNz4eY Ir5JA97kJu/HChjco1qvvK0u9bHaOCsgCuNNHEDPiW4nMdpUaJ/+8JydYAfZ40mqaXH/KKdRiry Fx/WoQ8c X-Proofpoint-ORIG-GUID: 82Exv7zzQ8yPjI-U5MYAJzU4W-FBQZDq X-Proofpoint-GUID: 82Exv7zzQ8yPjI-U5MYAJzU4W-FBQZDq X-Authority-Analysis: v=2.4 cv=StCQ6OO0 c=1 sm=1 tr=0 ts=68c315eb cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=yJojWOMRYYMA:10 a=VnNF1IyMAAAA:8 a=MtaQWmmsotiDqj5H3ecA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-11_03,2025-09-11_02,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 malwarescore=0 bulkscore=0 clxscore=1011 adultscore=0 suspectscore=0 priorityscore=1501 impostorscore=0 phishscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509060195 Content-Type: text/plain; charset="utf-8" We can now have userspace drivers (vfio-pci based) on s390x. The userspace drivers will not have any KVM fd and so no kzdev associated with them. So we need to update the logic for detecting passthrough devices to not depend on struct kvm_zdev. Signed-off-by: Farhan Ali Reviewed-by: Niklas Schnelle --- arch/s390/include/asm/pci.h | 1 + arch/s390/pci/pci_event.c | 14 ++++---------- drivers/vfio/pci/vfio_pci_zdev.c | 9 ++++++++- 3 files changed, 13 insertions(+), 11 deletions(-) diff --git a/arch/s390/include/asm/pci.h b/arch/s390/include/asm/pci.h index aed19a1aa9d7..f47f62fc3bfd 100644 --- a/arch/s390/include/asm/pci.h +++ b/arch/s390/include/asm/pci.h @@ -169,6 +169,7 @@ struct zpci_dev { =20 char res_name[16]; bool mio_capable; + bool mediated_recovery; struct zpci_bar_struct bars[PCI_STD_NUM_BARS]; =20 u64 start_dma; /* Start of available DMA addresses */ diff --git a/arch/s390/pci/pci_event.c b/arch/s390/pci/pci_event.c index d930416d4c90..541d536be052 100644 --- a/arch/s390/pci/pci_event.c +++ b/arch/s390/pci/pci_event.c @@ -61,16 +61,10 @@ static inline bool ers_result_indicates_abort(pci_ers_r= esult_t ers_res) } } =20 -static bool is_passed_through(struct pci_dev *pdev) +static bool needs_mediated_recovery(struct pci_dev *pdev) { struct zpci_dev *zdev =3D to_zpci(pdev); - bool ret; - - mutex_lock(&zdev->kzdev_lock); - ret =3D !!zdev->kzdev; - mutex_unlock(&zdev->kzdev_lock); - - return ret; + return zdev->mediated_recovery; } =20 static bool is_driver_supported(struct pci_driver *driver) @@ -194,7 +188,7 @@ static pci_ers_result_t zpci_event_attempt_error_recove= ry(struct pci_dev *pdev) } pdev->error_state =3D pci_channel_io_frozen; =20 - if (is_passed_through(pdev)) { + if (needs_mediated_recovery(pdev)) { pr_info("%s: Cannot be recovered in the host because it is a pass-throug= h device\n", pci_name(pdev)); status_str =3D "failed (pass-through)"; @@ -277,7 +271,7 @@ static void zpci_event_io_failure(struct pci_dev *pdev,= pci_channel_state_t es) * we will inject the error event and let the guest recover the device * itself. */ - if (is_passed_through(pdev)) + if (needs_mediated_recovery(pdev)) goto out; driver =3D to_pci_driver(pdev->dev.driver); if (driver && driver->err_handler && driver->err_handler->error_detected) diff --git a/drivers/vfio/pci/vfio_pci_zdev.c b/drivers/vfio/pci/vfio_pci_z= dev.c index 0990fdb146b7..a7bc23ce8483 100644 --- a/drivers/vfio/pci/vfio_pci_zdev.c +++ b/drivers/vfio/pci/vfio_pci_zdev.c @@ -148,6 +148,8 @@ int vfio_pci_zdev_open_device(struct vfio_pci_core_devi= ce *vdev) if (!zdev) return -ENODEV; =20 + zdev->mediated_recovery =3D true; + if (!vdev->vdev.kvm) return 0; =20 @@ -161,7 +163,12 @@ void vfio_pci_zdev_close_device(struct vfio_pci_core_d= evice *vdev) { struct zpci_dev *zdev =3D to_zpci(vdev->pdev); =20 - if (!zdev || !vdev->vdev.kvm) + if (!zdev) + return; + + zdev->mediated_recovery =3D false; + + if (!vdev->vdev.kvm) return; =20 if (zpci_kvm_hook.kvm_unregister) --=20 2.43.0 From nobody Thu Oct 2 19:27:06 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7D4E5369974; Thu, 11 Sep 2025 18:33:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757615602; cv=none; b=AbvQ5SASIwnh/vqsBoZGDUcIXAINrc29EvqWuTYhGtytChn61Fs/C3f8CzP3XegAJS+3Ss4QibSO/HDMjgK6Agh2E019f/tAnvEIFWKiMuxhwWKX+kLTEOYFfW8qemyEHAYkSsx4DYRzipr1QN/TRwQOBVY5FSvUDxcPRWNDh94= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757615602; c=relaxed/simple; bh=lISvvsVj3gZoCyb2hnU+9i3dJa2QHlhRVtFM4LwMPPU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lRPz6rIbZk2RACSY+gLa3XdUuanRbolMHFNsiTeDsjyR06IGdDPp6MnHOhDUa1qC8NA0CGmIkHPbiUj4tOxg5NcrvxWlrcUT60He8wruYAn85Dr2YWpnhSSufPDYrV/bltfX2dl6054ncFYYnWHiSBhuUJL0EnI6oghHYGzOwwY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=JsoPKZav; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="JsoPKZav" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58BClwkN026821; Thu, 11 Sep 2025 18:33:16 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=iQGNNINXa0BmjUmxW dUGnyQbGrZ+66cpiZoBLKw60i8=; b=JsoPKZavFjTPWF7I/MPrrIHSo1IJiz840 diYgU2Anwstrwq4zDlDEOzrxibRVB26/7lghUbCeNwVXPygjSJ6Fk6BKY6yh7TMa Ri0/PtnWHrh2tdJqbnKlxi980nArf5BSyfswBOXLnaTlQ1jwxcumvTZK8NMAf2LY UFfna9RwFik6GUj1PX9Lxaj++xvGCoKXxY3VUIpC+5YGmkDun51Fs33dP+bWUKlq Qm0Eo+O1n1uevbmG8qCKFulhXUzKn+pQDYVJYXufgBlFtVVnsNJ9ZuT6vKyHuJAe 7vVKfp5jAb9top6nKUO14KmfKSb8l4Oj6tjTV96jCvg/rmwZaRWbA== Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490bct5v8e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 18:33:16 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58BHlBZG008428; Thu, 11 Sep 2025 18:33:15 GMT Received: from smtprelay03.dal12v.mail.ibm.com ([172.16.1.5]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 49109py8mf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 18:33:15 +0000 Received: from smtpav01.dal12v.mail.ibm.com (smtpav01.dal12v.mail.ibm.com [10.241.53.100]) by smtprelay03.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58BIXEPn28771036 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Sep 2025 18:33:14 GMT Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1427F58057; Thu, 11 Sep 2025 18:33:14 +0000 (GMT) Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5FABC58058; Thu, 11 Sep 2025 18:33:13 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.249.32]) by smtpav01.dal12v.mail.ibm.com (Postfix) with ESMTP; Thu, 11 Sep 2025 18:33:13 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v3 07/10] s390/pci: Store PCI error information for passthrough devices Date: Thu, 11 Sep 2025 11:33:04 -0700 Message-ID: <20250911183307.1910-8-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250911183307.1910-1-alifm@linux.ibm.com> References: <20250911183307.1910-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTA2MDAxMCBTYWx0ZWRfX9Rx08+wgZfpQ L90VIx6Rl4mJeTa5BDK6oZgj98p0py8Z6o+jpe9dW0HrTOd04iEPa5W+4MswzXvnuNSD531fTUT R7kMX6gutI6tP7jREPRhPSSMBxYiWwx0BtrXQ3nrGRTiRavqnYOpgZSePODHnhMSK1FdOCUntvS /p42jza90Nk/s6OMzBeHyyHxeBOES4c6vh9y5pZkEGYrPEA7xhuw8/aAuWjxq+rU74kFN5GwZEC YTVtRRC2+pZ8XMUBy7ouTHTuglSm8Z1OXSI9uLT1rz7AiE2S6x4JZpOgX6YzWYyzQYs7Xk1REFS V+ZjC6c35ZgZL/jy7HnRHTSmyxjs9N7kkdcM+Mcmyw7N3qPWJVoq31hU2idHeUP2S16s6LaMfqF rLf4Dqct X-Authority-Analysis: v=2.4 cv=SKNCVPvH c=1 sm=1 tr=0 ts=68c315ec cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=yJojWOMRYYMA:10 a=VnNF1IyMAAAA:8 a=AhHhU-3714CzMtceQIIA:9 X-Proofpoint-GUID: 4QA-rC5EdO55zvAuNHiYqFNCIup10lXJ X-Proofpoint-ORIG-GUID: 4QA-rC5EdO55zvAuNHiYqFNCIup10lXJ X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-11_03,2025-09-11_02,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 spamscore=0 priorityscore=1501 bulkscore=0 malwarescore=0 adultscore=0 suspectscore=0 impostorscore=0 phishscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509060010 Content-Type: text/plain; charset="utf-8" For a passthrough device we need co-operation from user space to recover the device. This would require to bubble up any error information to user space. Let's store this error information for passthrough devices, so it can be retrieved later. Signed-off-by: Farhan Ali --- arch/s390/include/asm/pci.h | 28 ++++++++++ arch/s390/pci/pci.c | 1 + arch/s390/pci/pci_event.c | 95 +++++++++++++++++++------------- drivers/vfio/pci/vfio_pci_zdev.c | 2 + 4 files changed, 88 insertions(+), 38 deletions(-) diff --git a/arch/s390/include/asm/pci.h b/arch/s390/include/asm/pci.h index f47f62fc3bfd..72e05af90e08 100644 --- a/arch/s390/include/asm/pci.h +++ b/arch/s390/include/asm/pci.h @@ -116,6 +116,31 @@ struct zpci_bus { enum pci_bus_speed max_bus_speed; }; =20 +/* Content Code Description for PCI Function Error */ +struct zpci_ccdf_err { + u32 reserved1; + u32 fh; /* function handle */ + u32 fid; /* function id */ + u32 ett : 4; /* expected table type */ + u32 mvn : 12; /* MSI vector number */ + u32 dmaas : 8; /* DMA address space */ + u32 reserved2 : 6; + u32 q : 1; /* event qualifier */ + u32 rw : 1; /* read/write */ + u64 faddr; /* failing address */ + u32 reserved3; + u16 reserved4; + u16 pec; /* PCI event code */ +} __packed; + +#define ZPCI_ERR_PENDING_MAX 16 +struct zpci_ccdf_pending { + size_t count; + int head; + int tail; + struct zpci_ccdf_err err[ZPCI_ERR_PENDING_MAX]; +}; + /* Private data per function */ struct zpci_dev { struct zpci_bus *zbus; @@ -191,6 +216,8 @@ struct zpci_dev { struct iommu_domain *s390_domain; /* attached IOMMU domain */ struct kvm_zdev *kzdev; struct mutex kzdev_lock; + struct zpci_ccdf_pending pending_errs; + struct mutex pending_errs_lock; spinlock_t dom_lock; /* protect s390_domain change */ }; =20 @@ -316,6 +343,7 @@ void zpci_debug_exit_device(struct zpci_dev *); int zpci_report_error(struct pci_dev *, struct zpci_report_error_header *); int zpci_clear_error_state(struct zpci_dev *zdev); int zpci_reset_load_store_blocked(struct zpci_dev *zdev); +void zpci_cleanup_pending_errors(struct zpci_dev *zdev); =20 #ifdef CONFIG_NUMA =20 diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c index 5baeb5f6f674..02c7b06cfd0e 100644 --- a/arch/s390/pci/pci.c +++ b/arch/s390/pci/pci.c @@ -896,6 +896,7 @@ struct zpci_dev *zpci_create_device(u32 fid, u32 fh, en= um zpci_state state) mutex_init(&zdev->state_lock); mutex_init(&zdev->fmb_lock); mutex_init(&zdev->kzdev_lock); + mutex_init(&zdev->pending_errs_lock); =20 return zdev; =20 diff --git a/arch/s390/pci/pci_event.c b/arch/s390/pci/pci_event.c index 541d536be052..ac527410812b 100644 --- a/arch/s390/pci/pci_event.c +++ b/arch/s390/pci/pci_event.c @@ -18,23 +18,6 @@ #include "pci_bus.h" #include "pci_report.h" =20 -/* Content Code Description for PCI Function Error */ -struct zpci_ccdf_err { - u32 reserved1; - u32 fh; /* function handle */ - u32 fid; /* function id */ - u32 ett : 4; /* expected table type */ - u32 mvn : 12; /* MSI vector number */ - u32 dmaas : 8; /* DMA address space */ - u32 : 6; - u32 q : 1; /* event qualifier */ - u32 rw : 1; /* read/write */ - u64 faddr; /* failing address */ - u32 reserved3; - u16 reserved4; - u16 pec; /* PCI event code */ -} __packed; - /* Content Code Description for PCI Function Availability */ struct zpci_ccdf_avail { u32 reserved1; @@ -76,6 +59,41 @@ static bool is_driver_supported(struct pci_driver *drive= r) return true; } =20 +static void zpci_store_pci_error(struct pci_dev *pdev, + struct zpci_ccdf_err *ccdf) +{ + struct zpci_dev *zdev =3D to_zpci(pdev); + int i; + + mutex_lock(&zdev->pending_errs_lock); + if (zdev->pending_errs.count >=3D ZPCI_ERR_PENDING_MAX) { + pr_err("%s: Cannot store PCI error info for device", + pci_name(pdev)); + mutex_unlock(&zdev->pending_errs_lock); + return; + } + + i =3D zdev->pending_errs.tail % ZPCI_ERR_PENDING_MAX; + memcpy(&zdev->pending_errs.err[i], ccdf, sizeof(struct zpci_ccdf_err)); + zdev->pending_errs.tail++; + zdev->pending_errs.count++; + mutex_unlock(&zdev->pending_errs_lock); +} + +void zpci_cleanup_pending_errors(struct zpci_dev *zdev) +{ + struct pci_dev *pdev =3D NULL; + + mutex_lock(&zdev->pending_errs_lock); + pdev =3D pci_get_slot(zdev->zbus->bus, zdev->devfn); + if (zdev->pending_errs.count) + pr_err("%s: Unhandled PCI error events count=3D%zu", + pci_name(pdev), zdev->pending_errs.count); + memset(&zdev->pending_errs, 0, sizeof(struct zpci_ccdf_pending)); + mutex_unlock(&zdev->pending_errs_lock); +} +EXPORT_SYMBOL_GPL(zpci_cleanup_pending_errors); + static pci_ers_result_t zpci_event_notify_error_detected(struct pci_dev *p= dev, struct pci_driver *driver) { @@ -169,7 +187,8 @@ static pci_ers_result_t zpci_event_do_reset(struct pci_= dev *pdev, * and the platform determines which functions are affected for * multi-function devices. */ -static pci_ers_result_t zpci_event_attempt_error_recovery(struct pci_dev *= pdev) +static pci_ers_result_t zpci_event_attempt_error_recovery(struct pci_dev *= pdev, + struct zpci_ccdf_err *ccdf) { pci_ers_result_t ers_res =3D PCI_ERS_RESULT_DISCONNECT; struct zpci_dev *zdev =3D to_zpci(pdev); @@ -188,13 +207,6 @@ static pci_ers_result_t zpci_event_attempt_error_recov= ery(struct pci_dev *pdev) } pdev->error_state =3D pci_channel_io_frozen; =20 - if (needs_mediated_recovery(pdev)) { - pr_info("%s: Cannot be recovered in the host because it is a pass-throug= h device\n", - pci_name(pdev)); - status_str =3D "failed (pass-through)"; - goto out_unlock; - } - driver =3D to_pci_driver(pdev->dev.driver); if (!is_driver_supported(driver)) { if (!driver) { @@ -210,12 +222,22 @@ static pci_ers_result_t zpci_event_attempt_error_reco= very(struct pci_dev *pdev) goto out_unlock; } =20 + if (needs_mediated_recovery(pdev)) + zpci_store_pci_error(pdev, ccdf); + ers_res =3D zpci_event_notify_error_detected(pdev, driver); if (ers_result_indicates_abort(ers_res)) { status_str =3D "failed (abort on detection)"; goto out_unlock; } =20 + if (needs_mediated_recovery(pdev)) { + pr_info("%s: Recovering passthrough device\n", pci_name(pdev)); + ers_res =3D PCI_ERS_RESULT_RECOVERED; + status_str =3D "in progress"; + goto out_unlock; + } + if (ers_res !=3D PCI_ERS_RESULT_NEED_RESET) { ers_res =3D zpci_event_do_error_state_clear(pdev, driver); if (ers_result_indicates_abort(ers_res)) { @@ -258,25 +280,20 @@ static pci_ers_result_t zpci_event_attempt_error_reco= very(struct pci_dev *pdev) * @pdev: PCI function for which to report * @es: PCI channel failure state to report */ -static void zpci_event_io_failure(struct pci_dev *pdev, pci_channel_state_= t es) +static void zpci_event_io_failure(struct pci_dev *pdev, pci_channel_state_= t es, + struct zpci_ccdf_err *ccdf) { struct pci_driver *driver; =20 pci_dev_lock(pdev); pdev->error_state =3D es; - /** - * While vfio-pci's error_detected callback notifies user-space QEMU - * reacts to this by freezing the guest. In an s390 environment PCI - * errors are rarely fatal so this is overkill. Instead in the future - * we will inject the error event and let the guest recover the device - * itself. - */ + if (needs_mediated_recovery(pdev)) - goto out; + zpci_store_pci_error(pdev, ccdf); driver =3D to_pci_driver(pdev->dev.driver); if (driver && driver->err_handler && driver->err_handler->error_detected) driver->err_handler->error_detected(pdev, pdev->error_state); -out: + pci_dev_unlock(pdev); } =20 @@ -312,6 +329,7 @@ static void __zpci_event_error(struct zpci_ccdf_err *cc= df) pr_err("%s: Event 0x%x reports an error for PCI function 0x%x\n", pdev ? pci_name(pdev) : "n/a", ccdf->pec, ccdf->fid); =20 + if (!pdev) goto no_pdev; =20 @@ -322,12 +340,13 @@ static void __zpci_event_error(struct zpci_ccdf_err *= ccdf) break; case 0x0040: /* Service Action or Error Recovery Failed */ case 0x003b: - zpci_event_io_failure(pdev, pci_channel_io_perm_failure); + zpci_event_io_failure(pdev, pci_channel_io_perm_failure, ccdf); break; default: /* PCI function left in the error state attempt to recover */ - ers_res =3D zpci_event_attempt_error_recovery(pdev); + ers_res =3D zpci_event_attempt_error_recovery(pdev, ccdf); if (ers_res !=3D PCI_ERS_RESULT_RECOVERED) - zpci_event_io_failure(pdev, pci_channel_io_perm_failure); + zpci_event_io_failure(pdev, pci_channel_io_perm_failure, + ccdf); break; } pci_dev_put(pdev); diff --git a/drivers/vfio/pci/vfio_pci_zdev.c b/drivers/vfio/pci/vfio_pci_z= dev.c index a7bc23ce8483..2be37eab9279 100644 --- a/drivers/vfio/pci/vfio_pci_zdev.c +++ b/drivers/vfio/pci/vfio_pci_zdev.c @@ -168,6 +168,8 @@ void vfio_pci_zdev_close_device(struct vfio_pci_core_de= vice *vdev) =20 zdev->mediated_recovery =3D false; =20 + zpci_cleanup_pending_errors(zdev); + if (!vdev->vdev.kvm) return; =20 --=20 2.43.0 From nobody Thu Oct 2 19:27:06 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CCFD1371E93; Thu, 11 Sep 2025 18:33:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757615602; cv=none; b=ZRlZ+cnlc64ijYcOHZkCkfzFVum8IAfH6BsMU7sv6NcUtI/01VSx93r0aOCfSx+B8Qa+VdxRwJGO+0rLwSLERxxIm36UuVHE4fDXdH6Xek47lBvv4dvRn1oJ0yCKs7chrRJ+/pIdpF3LT04htdDuIoikhGPxXZzMXJUDkOxqsmQ= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757615602; c=relaxed/simple; bh=VKeAI6fRQ/t72OGx2CEndr/00kQZFDBkNl8vad6U7fM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZcMG9/ezVc8NcijxHiAKYese18v/4u6x3R97JK/cTrP3v95roIY8F1wKoU6fMn4Oi89cZoXSvUjGQf76UoHVK216/Ye3XNTHIwyQ83akT9uLkYsfYsieSh0oEt7qDn024r8FC3NY7hNUls1+QPQAFU6FivVYGKQOmHmTDkLbnj0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=BUiY4z3h; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="BUiY4z3h" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58BI960T021177; Thu, 11 Sep 2025 18:33:16 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=uNqMkd8x9H8pA4z+q I3lWSnlmkia9DOlDSAcCRov0BY=; b=BUiY4z3hIZBH/BBzekOZQSGax09wWL8+F IoRSnyhrlRxCRPseDxAA6pRKkHMB299yUYf5p/X621bcY8S/uASznJ9+wNPbmb71 P3ztK9NdqLYV9der2X+wQf/oSap+BPl2V1v4aHQ5Q3+VvLTtEK7iYNx5/8UEgbPt 0VLqszRrfe8+z8W+a/BbSBNazYVzSX0owBC+3LLCk8zBUCukdYSs+zqBo6TcHL1v s7hRdeyltBHXoCABP1uIG4+JUY0AiJ/GCec6NXbyzvzNxwnT8Sev3oKWsejheekS /r78ygBVYs2xkUMEwhna7TsLhJqO6JCMhZAqO9dOTlC7k/PuYGX9A== Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490bct5v8f-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 18:33:16 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58BI6ZvI017218; Thu, 11 Sep 2025 18:33:15 GMT Received: from smtprelay03.dal12v.mail.ibm.com ([172.16.1.5]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 4911gmq2cc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 18:33:15 +0000 Received: from smtpav01.dal12v.mail.ibm.com (smtpav01.dal12v.mail.ibm.com [10.241.53.100]) by smtprelay03.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58BIXEU824642150 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Sep 2025 18:33:15 GMT Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DAB535805D; Thu, 11 Sep 2025 18:33:14 +0000 (GMT) Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 30ECC58058; Thu, 11 Sep 2025 18:33:14 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.249.32]) by smtpav01.dal12v.mail.ibm.com (Postfix) with ESMTP; Thu, 11 Sep 2025 18:33:14 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v3 08/10] vfio-pci/zdev: Add a device feature for error information Date: Thu, 11 Sep 2025 11:33:05 -0700 Message-ID: <20250911183307.1910-9-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250911183307.1910-1-alifm@linux.ibm.com> References: <20250911183307.1910-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTA2MDAxMCBTYWx0ZWRfX1dYPr69qqEYR PpzgoYQauD+JZNADK1qz6Rol2nRXHQbN11aELKKo3B/ezS/bYzeyCN/uHVAWaTezscAk3npNPii huEygyvr/lWGZ/RYDngtXGdiWdozzNliOcrxRoftKI+//gHZSH2Bhzt2leTgZ4ZE0B2GhBHj6n0 1AfcX6IwekcM6nlV+mQ0aRskli++T+Vfvaf8vov4IFZSMBrw9WOEIft5UCY+I246UZAhm/pLEPf 9xEW8Wgihb0on1tw9/PNwdbf6lP/CUE4yz6Kx0J21sNiOrXA4dJVNPRjNzXChsrB+iuDHKu0+uV r/ZsrtFxdnVmw3jWXt4Y1u/8Jp2p0NGIxX5YaF+IAZCuonIHhdcL4ZyH9IMLITlBQT9pw3tw3pI Mwdhd4ki X-Authority-Analysis: v=2.4 cv=SKNCVPvH c=1 sm=1 tr=0 ts=68c315ec cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=yJojWOMRYYMA:10 a=VnNF1IyMAAAA:8 a=QIy8Zix3hhx26zbqT_4A:9 X-Proofpoint-GUID: maYmdmJIrT37lHslglJSrDLdN-26WcIV X-Proofpoint-ORIG-GUID: maYmdmJIrT37lHslglJSrDLdN-26WcIV X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-11_03,2025-09-11_02,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 spamscore=0 priorityscore=1501 bulkscore=0 malwarescore=0 adultscore=0 suspectscore=0 impostorscore=0 phishscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509060010 Content-Type: text/plain; charset="utf-8" For zPCI devices, we have platform specific error information. The platform firmware provides this error information to the operating system in an architecture specific mechanism. To enable recovery from userspace for these devices, we want to expose this error information to userspace. Add a new device feature to expose this information. Signed-off-by: Farhan Ali --- drivers/vfio/pci/vfio_pci_core.c | 2 ++ drivers/vfio/pci/vfio_pci_priv.h | 8 ++++++++ drivers/vfio/pci/vfio_pci_zdev.c | 34 ++++++++++++++++++++++++++++++++ include/uapi/linux/vfio.h | 14 +++++++++++++ 4 files changed, 58 insertions(+) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index 7dcf5439dedc..378adb3226db 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -1514,6 +1514,8 @@ int vfio_pci_core_ioctl_feature(struct vfio_device *d= evice, u32 flags, return vfio_pci_core_pm_exit(device, flags, arg, argsz); case VFIO_DEVICE_FEATURE_PCI_VF_TOKEN: return vfio_pci_core_feature_token(device, flags, arg, argsz); + case VFIO_DEVICE_FEATURE_ZPCI_ERROR: + return vfio_pci_zdev_feature_err(device, flags, arg, argsz); default: return -ENOTTY; } diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_p= riv.h index a9972eacb293..a4a7f97fdc2e 100644 --- a/drivers/vfio/pci/vfio_pci_priv.h +++ b/drivers/vfio/pci/vfio_pci_priv.h @@ -86,6 +86,8 @@ int vfio_pci_info_zdev_add_caps(struct vfio_pci_core_devi= ce *vdev, struct vfio_info_cap *caps); int vfio_pci_zdev_open_device(struct vfio_pci_core_device *vdev); void vfio_pci_zdev_close_device(struct vfio_pci_core_device *vdev); +int vfio_pci_zdev_feature_err(struct vfio_device *device, u32 flags, + void __user *arg, size_t argsz); #else static inline int vfio_pci_info_zdev_add_caps(struct vfio_pci_core_device = *vdev, struct vfio_info_cap *caps) @@ -100,6 +102,12 @@ static inline int vfio_pci_zdev_open_device(struct vfi= o_pci_core_device *vdev) =20 static inline void vfio_pci_zdev_close_device(struct vfio_pci_core_device = *vdev) {} + +static int vfio_pci_zdev_feature_err(struct vfio_device *device, u32 flags, + void __user *arg, size_t argsz); +{ + return -ENODEV; +} #endif =20 static inline bool vfio_pci_is_vga(struct pci_dev *pdev) diff --git a/drivers/vfio/pci/vfio_pci_zdev.c b/drivers/vfio/pci/vfio_pci_z= dev.c index 2be37eab9279..261954039aa9 100644 --- a/drivers/vfio/pci/vfio_pci_zdev.c +++ b/drivers/vfio/pci/vfio_pci_zdev.c @@ -141,6 +141,40 @@ int vfio_pci_info_zdev_add_caps(struct vfio_pci_core_d= evice *vdev, return ret; } =20 +int vfio_pci_zdev_feature_err(struct vfio_device *device, u32 flags, + void __user *arg, size_t argsz) +{ + struct vfio_device_feature_zpci_err err; + struct vfio_pci_core_device *vdev =3D + container_of(device, struct vfio_pci_core_device, vdev); + struct zpci_dev *zdev =3D to_zpci(vdev->pdev); + int ret; + int head =3D 0; + + if (!zdev) + return -ENODEV; + + ret =3D vfio_check_feature(flags, argsz, VFIO_DEVICE_FEATURE_GET, + sizeof(err)); + if (ret !=3D 1) + return ret; + + mutex_lock(&zdev->pending_errs_lock); + if (zdev->pending_errs.count) { + head =3D zdev->pending_errs.head % ZPCI_ERR_PENDING_MAX; + err.pec =3D zdev->pending_errs.err[head].pec; + zdev->pending_errs.head++; + zdev->pending_errs.count--; + err.pending_errors =3D zdev->pending_errs.count; + } + mutex_unlock(&zdev->pending_errs_lock); + + if (copy_to_user(arg, &err, sizeof(err))) + return -EFAULT; + + return 0; +} + int vfio_pci_zdev_open_device(struct vfio_pci_core_device *vdev) { struct zpci_dev *zdev =3D to_zpci(vdev->pdev); diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h index 75100bf009ba..a950c341602d 100644 --- a/include/uapi/linux/vfio.h +++ b/include/uapi/linux/vfio.h @@ -1478,6 +1478,20 @@ struct vfio_device_feature_bus_master { }; #define VFIO_DEVICE_FEATURE_BUS_MASTER 10 =20 +/** + * VFIO_DEVICE_FEATURE_ZPCI_ERROR feature provides PCI error information to + * userspace for vfio-pci devices on s390x. On s390x PCI error recovery in= volves + * platform firmware and notification to operating system is done by + * architecture specific mechanism. Exposing this information to userspace + * allows userspace to take appropriate actions to handle an error on the + * device. + */ +struct vfio_device_feature_zpci_err { + __u16 pec; + int pending_errors; +}; +#define VFIO_DEVICE_FEATURE_ZPCI_ERROR 11 + /* -------- API for Type1 VFIO IOMMU -------- */ =20 /** --=20 2.43.0 From nobody Thu Oct 2 19:27:06 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9A63B37C0F2; Thu, 11 Sep 2025 18:33:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757615605; cv=none; b=q6zthcm0Qxc/vKggk+hXV0Gh/QDgwiexn3pqKVOWQkuAj5wR1DU0AUWieivMjPiClpttxlbsq4MoYqTn6QQQttFJla/96s4wqeZbOnDYDw2FDb/AUo6K2G5lFhhNHZg/JzlvtxHUQemMj7tuUoaHXq8jV32xdCMoMrcZpQiSyIE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757615605; c=relaxed/simple; bh=ytFXbNEKbWNWIa4e99DqMpdEgtgPube/J3VaS5HzAEQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Me/JpsjhGAhCkZcTN53Z4QC+HLDV8rXOQm6HVE01THyV7rsYxSsmsIwU/taVTNq/ZXJv1TsAGUqziojyCY7uL/9q75U920TBgqKLKk2pAKRO/KPfhk0qA3k4WmEzA+nhCTULp4jIcywLlzCDhW5vgALL1Xh307pVV0irqWLOgCk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=daYGfCE+; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="daYGfCE+" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58BHarQO031789; Thu, 11 Sep 2025 18:33:19 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=wJWMYHMfvbzU5BQsW YZErSvZaRLa/7xMVP6aVFziDXk=; b=daYGfCE+GJxCX41C/ndtdQ5ZmzeZZ6LTW G5mVQVCWpxx3kiyW9A+N+93eWeqBPvlOLaHSPMxrKM55x/f+I82cbKoTQrd0eJ2m kxAKawfmJzMIvr4cmnOTwt4YTrCYzQPy400ZSRn+cfsBzoWisGmolnV9zWMBq0xm dg8jYuTfMPJ14yQCZIqpgHZuekwcvAsyHn8fWK8GmI+AV9ivg6QzCA1ZFJCLSQfG /MV8FVwXGfm4ZXjX6mCqb/8djcAYyEw3j9O6R9UO2e41A0uhDqi+hwWOSpXA8o+l d3ivcgbCNsL/R9eUb34HV9y/Op5YhdWMwJT0jE2n6YR4vm9hEhZBg== Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490acrdxay-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 18:33:18 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58BGF2dC011434; Thu, 11 Sep 2025 18:33:17 GMT Received: from smtprelay03.wdc07v.mail.ibm.com ([172.16.1.70]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 490y9uqgrh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 18:33:17 +0000 Received: from smtpav01.dal12v.mail.ibm.com (smtpav01.dal12v.mail.ibm.com [10.241.53.100]) by smtprelay03.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58BIX6cW29622840 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Sep 2025 18:33:06 GMT Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A89DC58059; Thu, 11 Sep 2025 18:33:15 +0000 (GMT) Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 04A1B58057; Thu, 11 Sep 2025 18:33:15 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.249.32]) by smtpav01.dal12v.mail.ibm.com (Postfix) with ESMTP; Thu, 11 Sep 2025 18:33:14 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v3 09/10] vfio: Add a reset_done callback for vfio-pci driver Date: Thu, 11 Sep 2025 11:33:06 -0700 Message-ID: <20250911183307.1910-10-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250911183307.1910-1-alifm@linux.ibm.com> References: <20250911183307.1910-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: 4uHlJCifPCGHmGXWRAptpuGsRZWxM_aj X-Authority-Analysis: v=2.4 cv=Mp1S63ae c=1 sm=1 tr=0 ts=68c315ee cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=yJojWOMRYYMA:10 a=VnNF1IyMAAAA:8 a=hcTHx3Z5Akp3fEzgVBYA:9 X-Proofpoint-ORIG-GUID: 4uHlJCifPCGHmGXWRAptpuGsRZWxM_aj X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTA2MDAwMCBTYWx0ZWRfX7TXy8smtZ86K zwV8zsC48Oir+y33umkd1k0Vi7uvVQpOJ55XchIPmf/sVjJelGGZHSRrPRwOAYnudio3Rpjxxpp SrB2Xb5Y4qYJ5Yd2yRplF8BdJcn9AYGE0X30xxgyAtsKaP41tEy7GHmZXLamO9kddAdRHG5H2Rz UWaL4MO9GlFGX5BzhRw78FqqfhfZ0PEbdhUHhvuqfAo/YA7gKG8MCc5EIJmQYmxwrBCyRHv0LBj cneQpb+6N7fRxJGAluvJZ+UXH4oKV9Pi7HcgvIsDsF9HIcexlGmtk6Auh93dd4rUV9Aj9JHyrD9 dmmIddBzUMzVR+/FDVLp7Pq702KVorBUIl2ftxbAsWPHzLauzeumj3xbQX0Sa7MG+JmhuNjznTx iZWp29QD X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-11_03,2025-09-11_02,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 malwarescore=0 clxscore=1011 phishscore=0 spamscore=0 adultscore=0 priorityscore=1501 bulkscore=0 suspectscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509060000 Content-Type: text/plain; charset="utf-8" On error recovery for a PCI device bound to vfio-pci driver, we want to recover the state of the device to its last known saved state. The callback restores the state of the device to its initial saved state. Signed-off-by: Farhan Ali --- drivers/vfio/pci/vfio_pci_core.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index 378adb3226db..f2fcb81b3e69 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -2241,6 +2241,17 @@ pci_ers_result_t vfio_pci_core_aer_err_detected(stru= ct pci_dev *pdev, } EXPORT_SYMBOL_GPL(vfio_pci_core_aer_err_detected); =20 +static void vfio_pci_core_aer_reset_done(struct pci_dev *pdev) +{ + struct vfio_pci_core_device *vdev =3D dev_get_drvdata(&pdev->dev); + + if (!vdev->pci_saved_state) + return; + + pci_load_saved_state(pdev, vdev->pci_saved_state); + pci_restore_state(pdev); +} + int vfio_pci_core_sriov_configure(struct vfio_pci_core_device *vdev, int nr_virtfn) { @@ -2305,6 +2316,7 @@ EXPORT_SYMBOL_GPL(vfio_pci_core_sriov_configure); =20 const struct pci_error_handlers vfio_pci_core_err_handlers =3D { .error_detected =3D vfio_pci_core_aer_err_detected, + .reset_done =3D vfio_pci_core_aer_reset_done, }; EXPORT_SYMBOL_GPL(vfio_pci_core_err_handlers); =20 --=20 2.43.0 From nobody Thu Oct 2 19:27:06 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9BB7A2DC770; Thu, 11 Sep 2025 18:33:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757615622; cv=none; b=QXQj1bWKRkyvtgwpp5ngf5LzKBo11JShuzikj5mCDfv/vjA0y1spGmoZXA9d9vxUDtJNqXX26ZKQmyc0llVXRQfwKpqeQ/AELDhZvs5wiP5fSWfGy9f5Z/gSay+u/SFqi6e76Xe2gXyC0FU/uThUG1xPIklu8xrt1Oo9bEiVHDA= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1757615622; c=relaxed/simple; bh=HvTt5221H5DQ1X+GZWXY3QkHHnBA6eW/stRkoiAL7gs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KZLRuzneqGbGVLuwfnQHPu8g5WkKyFP0JU9g6Vr8qefCsdSTItd3dBnhOz6dpfqPQIKU/KdYo0RmuMF50+KVp0rPRRMTyaeIeNnnr8Pi2EcTarNIi4kp59DcXuG818hMO7WiyGCxVMA4j87xlwCfYUDOpd7fz2+HyjPSHmML+j0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=gcb/4Lc3; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="gcb/4Lc3" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 58BA1afJ013271; Thu, 11 Sep 2025 18:33:19 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=+c2kHIW89fFxZMF7i igDqHKPq5NHEczv6HmG2G6s5GY=; b=gcb/4Lc3+ItXqNSeu6FlwZIQ9ORO7Xb7Y s+qmEWxKeaUIKqqD5bShw2HB3OvbXc0zNrBqyxFXehc0wj0ZkJ8tWkXqMhdNpknQ T3fbR+repueFwdTN8cNukLYFiexdqy463ZtriDTF/AvsAEtjQWjaTsxNkFWVjcxd JPvqNooy3oka/AdUyz6ujD64atDDA8NUhL6phaHFyAQWvCHDmAT50oLxS5LAxnfJ NBM7KpB3ZF5BF3Ey9tezNaRc0wXB+sev8gEru40o7V2g9z3jW9NV0GH0SeiVHcy8 W9hDd7y5vqYavJF5ZAw2xQsgf5UFodc1ug9I2y0FfP2sfOQUPR3sA== Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 490xydbgbu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 18:33:19 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 58BIK3a0001163; Thu, 11 Sep 2025 18:33:18 GMT Received: from smtprelay04.wdc07v.mail.ibm.com ([172.16.1.71]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 491203q0b1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 11 Sep 2025 18:33:18 +0000 Received: from smtpav01.dal12v.mail.ibm.com (smtpav01.dal12v.mail.ibm.com [10.241.53.100]) by smtprelay04.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 58BIXGYP35455596 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 11 Sep 2025 18:33:16 GMT Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7508C58058; Thu, 11 Sep 2025 18:33:16 +0000 (GMT) Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C467658057; Thu, 11 Sep 2025 18:33:15 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.249.32]) by smtpav01.dal12v.mail.ibm.com (Postfix) with ESMTP; Thu, 11 Sep 2025 18:33:15 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v3 10/10] vfio: Remove the pcie check for VFIO_PCI_ERR_IRQ_INDEX Date: Thu, 11 Sep 2025 11:33:07 -0700 Message-ID: <20250911183307.1910-11-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250911183307.1910-1-alifm@linux.ibm.com> References: <20250911183307.1910-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: lGVpjyA4ijHYw0BA5iwJdXUm3XqB5caw X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwOTA2MDIzNSBTYWx0ZWRfX94FDKZeZib8W N1zlDk3GxjoPxR/TFgIUd68U0rQuWOGtTuhE4jbOK6r6f2HsQ1CwvkyMHsix1SCVCQFZMpVpBj9 4LJacXV4Amrq3kDNhZU1tFFqArVkK8TyvzGmRzJ2rZZQBi+zLaHyrR5GlVk1lV5/XUG2OwemcSD X4WbTa74VxBWOHK36YujNk8pUh9sexdOqlKXb/31+by7pn/gaeX/KjIgGlkTR+lnmCITAUWRpgR pyYI13za69i3ecwmJ3GoO18Pel6uazJiHhoynmGz3sZcVk15zjh10jIHvT/8EF4WLLfAM89hdQV oWrk+X7b6e51YwVV33pRK69pXJL07IJZDWIA/5ZCqw5ePYqHujmMOqLCOXrUZmCUDzc+V+ejB+3 jbCbkJGY X-Proofpoint-GUID: lGVpjyA4ijHYw0BA5iwJdXUm3XqB5caw X-Authority-Analysis: v=2.4 cv=F59XdrhN c=1 sm=1 tr=0 ts=68c315ef cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=yJojWOMRYYMA:10 a=VnNF1IyMAAAA:8 a=gONGJyW3jwFs4LhXXOUA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1117,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-09-11_03,2025-09-11_02,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=0 phishscore=0 clxscore=1015 impostorscore=0 bulkscore=0 adultscore=0 spamscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2509060235 Content-Type: text/plain; charset="utf-8" We are configuring the error signaling on the vast majority of devices and it's extremely rare that it fires anyway. This allows userspace to be notified on errors for legacy PCI devices. The Internal Share Memory (ISM) device on s390x is one such device. For PCI devices on IBM s390x error recovery involves platform firmware and notification to operating system is done by architecture specific way. So the ISM device can still be recovered when notified of an error. Signed-off-by: Farhan Ali --- drivers/vfio/pci/vfio_pci_core.c | 6 ++---- drivers/vfio/pci/vfio_pci_intrs.c | 3 +-- 2 files changed, 3 insertions(+), 6 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index f2fcb81b3e69..d125471fd5ea 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -749,8 +749,7 @@ static int vfio_pci_get_irq_count(struct vfio_pci_core_= device *vdev, int irq_typ return (flags & PCI_MSIX_FLAGS_QSIZE) + 1; } } else if (irq_type =3D=3D VFIO_PCI_ERR_IRQ_INDEX) { - if (pci_is_pcie(vdev->pdev)) - return 1; + return 1; } else if (irq_type =3D=3D VFIO_PCI_REQ_IRQ_INDEX) { return 1; } @@ -1150,8 +1149,7 @@ static int vfio_pci_ioctl_get_irq_info(struct vfio_pc= i_core_device *vdev, case VFIO_PCI_REQ_IRQ_INDEX: break; case VFIO_PCI_ERR_IRQ_INDEX: - if (pci_is_pcie(vdev->pdev)) - break; + break; fallthrough; default: return -EINVAL; diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_= intrs.c index 123298a4dc8f..f2d13b6eb28f 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -838,8 +838,7 @@ int vfio_pci_set_irqs_ioctl(struct vfio_pci_core_device= *vdev, uint32_t flags, case VFIO_PCI_ERR_IRQ_INDEX: switch (flags & VFIO_IRQ_SET_ACTION_TYPE_MASK) { case VFIO_IRQ_SET_ACTION_TRIGGER: - if (pci_is_pcie(vdev->pdev)) - func =3D vfio_pci_set_err_trigger; + func =3D vfio_pci_set_err_trigger; break; } break; --=20 2.43.0