From nobody Fri Oct 3 20:25:18 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ED3F2301033; Mon, 25 Aug 2025 17:12:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756141958; cv=none; b=lF44LRjaacc3CmBGcp7IQmdqL0phZMBKSQgQ0QfCiMxYDBDB1VpgaQqXmCk5J/xrq6oZPWMa1cRehXqRiKrn/x3rzRjNPnjPpO7eDO0/yZrtQ3tG3XUbs5PceysnuH4rdwIZ4qQo67AeQ4jQOcm/Z4Go94PlDQE05rWR5W8/E1E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756141958; c=relaxed/simple; bh=xaZwswDwgMMYH9vEvK3X/1XYODWeoNo/veR2cT7F7+g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=knGF7V77qMPOKWvhDj5wTk2W7bJ6mAbV7SzEiP0T1SX3ZPjO7jolzg1FNgffx1lM7DqxKUkXb0dbXUsUzFqU/Vh7sJCkBE1+xpIRMZkzMXJP3aa1NV9SMIesLMrJbhwRCO822nZrAjlQxoK5GtGPCdJiGP82Jt0W6GON00JlIdQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=NDvdKq9u; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="NDvdKq9u" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57PC3x5t001647; Mon, 25 Aug 2025 17:12:32 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=gpWnExHb1upyt8hMs xgYZbWheuUsGOwmHno5X8ZygwE=; b=NDvdKq9u4Z6SuAShE+f+BETM9OJ17BTrK emFH9KFh1WwkZzWC5oSNMQzka0ESAF7aUPD2AiRirMmR6lL4QvH8NCDDZogVA3Nr ZapstH44H8AZkVw5bb4Ajz834HfOXruIEGGZ1loErRWy4yxn81eMbB7vkZqG8zu/ l6XrkJocVKJLZENzwu1Li5PpF6rt2wYowY80Aq782IUrunNsqoR/cocg+4UDKzBH b7dZEkLmh4zA2Wp4W5jZNAmjhPfkROX13flQRxMW7nKAHnRvLXGiPyjDIseOJlLf baWEwXcIoL5F7OF3gsIXhIHYYks9M6fKK5OTT+1u28cqZrYu8YtZA== Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48q557t98r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 25 Aug 2025 17:12:32 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57PFMVY9018009; Mon, 25 Aug 2025 17:12:31 GMT Received: from smtprelay05.wdc07v.mail.ibm.com ([172.16.1.72]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 48qtp36n68-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 25 Aug 2025 17:12:31 +0000 Received: from smtpav06.wdc07v.mail.ibm.com (smtpav06.wdc07v.mail.ibm.com [10.39.53.233]) by smtprelay05.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57PHCTLq26083948 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 25 Aug 2025 17:12:29 GMT Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 920DB58056; Mon, 25 Aug 2025 17:12:29 +0000 (GMT) Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 98EC65804E; Mon, 25 Aug 2025 17:12:28 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.255.253]) by smtpav06.wdc07v.mail.ibm.com (Postfix) with ESMTP; Mon, 25 Aug 2025 17:12:28 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v2 1/9] PCI: Avoid restoring error values in config space Date: Mon, 25 Aug 2025 10:12:18 -0700 Message-ID: <20250825171226.1602-2-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250825171226.1602-1-alifm@linux.ibm.com> References: <20250825171226.1602-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: _Fu7oOhOhzZd1Wv1j0pc4HkgObdpetU4 X-Proofpoint-ORIG-GUID: _Fu7oOhOhzZd1Wv1j0pc4HkgObdpetU4 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODIzMDAyMSBTYWx0ZWRfX9b2A7TGCSLnm efPLsC4T3Xzz3hiPFQh6nrRSaLBWKdSLWN/RT9lXKScLXeICtsLhYwj9uTeLagaunNRUxV/LS1V PWDRj0P2gXSy/bUWY+jqc4Mh8Cj+qHTphSxbJDef+qAhHNMHHm91Bb0NyDe799t4xcj2NXYNQMR g9p4DtuC3HNJgvnr7yiwhn0wGN14K95IVT2ZkHUQrxMN9wAnytE3SHEaMJ+5fQ8Hd7ZreFeNuF1 tASkpuJJYmttJhgdKNFsHx3gSHyq6jiO2d7qBVad4LRXM3S5q+ax/kD1mdw52YXgdb8vp8zcEFn XZDfDgSRnX6QtaRem2VP3wL8MXZb4AbUhqTjOSyCyoYBpGjuz5/Z6rzs/YMp/d7fHNDlh7kj91Y r3lUVhxD X-Authority-Analysis: v=2.4 cv=A8ZsP7WG c=1 sm=1 tr=0 ts=68ac9980 cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=2OwXVqhp2XgA:10 a=VnNF1IyMAAAA:8 a=m1QfOrpfu5IsAf0OiGAA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-25_07,2025-08-20_03,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 adultscore=0 priorityscore=1501 suspectscore=0 clxscore=1015 spamscore=0 impostorscore=0 malwarescore=0 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2508230021 Content-Type: text/plain; charset="utf-8" The current reset process saves the device's config space state before reset and restores it afterward. However, when a device is in an error state before reset, config space reads may return error values instead of valid data. This results in saving corrupted values that get written back to the device during state restoration. Add validation to prevent writing error values to the device when restoring the config space state after reset. Signed-off-by: Farhan Ali --- drivers/pci/pci.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index b0f4d98036cd..0dd95d782022 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -1825,6 +1825,9 @@ static void pci_restore_config_dword(struct pci_dev *= pdev, int offset, if (!force && val =3D=3D saved_val) return; =20 + if (PCI_POSSIBLE_ERROR(saved_val)) + return; + for (;;) { pci_dbg(pdev, "restore config %#04x: %#010x -> %#010x\n", offset, val, saved_val); --=20 2.43.0 From nobody Fri Oct 3 20:25:18 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EC5E026E175; Mon, 25 Aug 2025 17:12:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756141958; cv=none; b=gX15TPYCT4HRLsnO8/vejq/2rjrn480/VwSr2JVpFE80Z+B93dJVyh3CE7d+8Get6r8Q/Wi7VS6/3wyqO3DP10TxTTW/I1usVOljF3QL9VWRHUrVU6tO3BtXWPsF0DI4Hj6rc71Az45CQBYwAXloKg4AYj7bZrKnfcucZyTfhbI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756141958; c=relaxed/simple; bh=ltgFhqa+XtMSF+Bjmt4iYmt8qr47pAB7mkYgCNoJoDQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ppo+W0Mv9RupgM/nVu3gCU6fQNU/tzhB6lKOu4NYHrxJW0KB7vNzKVyhvjVbEtSn5zOieP413KVD0xxa93HtlfqRUPjhZQaMXwUp+zw7xiqoh4uKhyhujk4p8GtJYqEhMpHUt38yHUmt9+py8768W3MGDbDpHJVpF8nKBgiy68w= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=F/Ye7RKN; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="F/Ye7RKN" Received: from pps.filterd (m0356516.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57PFIbdK019576; Mon, 25 Aug 2025 17:12:33 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=N9e1A7GRC0k3UHgV9 UW5Oj57qEr5SNvLH8V/m4UDCpE=; b=F/Ye7RKNuEbS9yq81/L9Ugo+vDj3VpMeT PoS3riUhL+hLBvnmg/v/HsrZ59vCNWH0hGYytup/cCugdBEGy5jlAuck4lhfzk1I 8Eg5z3paoGaEHLaqAWeOW1GFbzUoeZ7EAEAgaHQvqDAADTIkIIkimofb/5e/l3Zy oUAnb/8qTppCWkF7h2KkDEB/C5K+lhOhfHn9aqXbCF3mZiGIcFxRrWPh7enRfirI HHvSkjo35J3PB0bgpugB6u82tf/kioWVF7vWYRmrTZg7UECHVdsdwynOBgHlrtDd wvUubHyJ/hEaSpUJWxw3G/900RvCpBUwzDgx12dPB+CxEFYduC/9Q== Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48q32vafy8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 25 Aug 2025 17:12:32 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57PDroos002561; Mon, 25 Aug 2025 17:12:32 GMT Received: from smtprelay06.wdc07v.mail.ibm.com ([172.16.1.73]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 48qrypex85-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 25 Aug 2025 17:12:32 +0000 Received: from smtpav06.wdc07v.mail.ibm.com (smtpav06.wdc07v.mail.ibm.com [10.39.53.233]) by smtprelay06.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57PHCUpg21234332 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 25 Aug 2025 17:12:30 GMT Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C8C525804E; Mon, 25 Aug 2025 17:12:30 +0000 (GMT) Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BD88158054; Mon, 25 Aug 2025 17:12:29 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.255.253]) by smtpav06.wdc07v.mail.ibm.com (Postfix) with ESMTP; Mon, 25 Aug 2025 17:12:29 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v2 2/9] PCI: Add additional checks for flr and pm reset Date: Mon, 25 Aug 2025 10:12:19 -0700 Message-ID: <20250825171226.1602-3-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250825171226.1602-1-alifm@linux.ibm.com> References: <20250825171226.1602-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: 3K2Au8MkW9NI1CBdIdT1wG-LY7i1zK_Z X-Proofpoint-GUID: 3K2Au8MkW9NI1CBdIdT1wG-LY7i1zK_Z X-Authority-Analysis: v=2.4 cv=AfSxH2XG c=1 sm=1 tr=0 ts=68ac9980 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=2OwXVqhp2XgA:10 a=VnNF1IyMAAAA:8 a=WWBBxJq8oJm8QP7TXnYA:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODIzMDAwMCBTYWx0ZWRfX4W5gxYNMnmOT gzcEu1tzNyF7zUnr8Hs8fJtQv4ydQV1tcFGrIt7l/R3sqMunsltCHnjUIbUcLe+rXG2Rx5tO43d PXfz8vn0VtWTXAR92hGDOy3oXNJuAYLS3miyPqyw9RoxrKGAql5tZ29gI6R4vXkCqqkUXeDO1Ub WV5uVCggjLjGR7I6YEnxP2bWQRAWN+LyN+CEM55i/wvuO3aHr95ZhzDvFKiWkV+bj65EJ1o+IDa SiTrrTDETc0jpnatfaEgQVO/x7UaVgQaA0fQqmL7/hw6rzjIhX3fVRyn7VsDeC9SdXqdMLMvucM G2KxRN9S1MW1H1XlRu8Dd4W4EQfnAPFszAOGwTDttPUtkw/rBm1KDDbMkM24fuQBZIVNLM7vC8L 679zn0Hp X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-25_08,2025-08-20_03,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 suspectscore=0 malwarescore=0 adultscore=0 spamscore=0 priorityscore=1501 bulkscore=0 impostorscore=0 clxscore=1015 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2508230000 Content-Type: text/plain; charset="utf-8" If a device is in an error state, then any reads of device registers can return error value. Add addtional checks to validate if a device is in an error state before doing an flr or pm reset. Signed-off-by: Farhan Ali --- drivers/pci/pci.c | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 0dd95d782022..a07bdb287cf3 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -4560,12 +4560,17 @@ EXPORT_SYMBOL_GPL(pcie_flr); */ int pcie_reset_flr(struct pci_dev *dev, bool probe) { + u32 reg; + if (dev->dev_flags & PCI_DEV_FLAGS_NO_FLR_RESET) return -ENOTTY; =20 if (!(dev->devcap & PCI_EXP_DEVCAP_FLR)) return -ENOTTY; =20 + if (pcie_capability_read_dword(dev, PCI_EXP_DEVCAP, ®)) + return -ENOTTY; + if (probe) return 0; =20 @@ -4640,6 +4645,8 @@ static int pci_pm_reset(struct pci_dev *dev, bool pro= be) return -ENOTTY; =20 pci_read_config_word(dev, dev->pm_cap + PCI_PM_CTRL, &csr); + if (PCI_POSSIBLE_ERROR(csr)) + return -ENOTTY; if (csr & PCI_PM_CTRL_NO_SOFT_RESET) return -ENOTTY; =20 --=20 2.43.0 From nobody Fri Oct 3 20:25:18 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C52FC303C9E; Mon, 25 Aug 2025 17:12:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756141961; cv=none; b=CUVWurF5WKdAW1HtQs89qL5LA+dLBWjL/Zhg9aik2l2JUGfuqG7aMiWIvVC0NnRw8Dz2286qXXZSKMKeVO1p5G1x6uPVYGnj6TEATcU576z9xUnIvYGn4uxiiISUjAT+Dg/ebW1tohJuilQqbzZnpwzlDNDczvLK9qjRcTrdE08= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756141961; c=relaxed/simple; bh=gxQBDm9CkrMlpO1c0Xc+fy37AaNeGPLTPSTrcJJYpss=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Iw3SV9pmShf9xxu9jj5BLYdZEBJzg6nD/FlHG2Moq4dg9HI3rnbwLzg+36FmeMqDoW9jMUrrc08MMcPXft1sl9+y6SAB+xcZ0hEOpkwXa+ybFkrFfXBTmxJ6X6p2pilB2gra9KW0klI9dRAIuMh7xXbN2M7fqQ95h138rsELNck= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=tPWMRt//; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="tPWMRt//" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57P9AkeK019463; Mon, 25 Aug 2025 17:12:34 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=AxGqz+wOzYX3hTjME Ke8dNpYA1otWEGI86p16x8pqJ4=; b=tPWMRt//8ry90MFMpLTK+d4D6aQQDAg9j 1ZvrhaxOBLEYSicQb/vxZ3m2zgydwvWnAevOTU+UEsLk036yQ4El99KCQYKxyIsc pfjAFWoXVNNy1VBWoQeB/XR4F4auTbetActA9GTJ0ol3CCQHY6K+v9g5avclwtGx yenimntNYDMhFDh0M464i1GeRWbi7OPD0+PG+tu17EMEBhisru4ubj8h0Qazcrpl zklaCIPWY+wbNUYl1Sqk+U9jSSgtpzsDmFUyNWVWjYkLfDCfFiug87dd7hrdYbap WaTBmtrJK8LBhkQeWpim9YJfGk/k+BziKVX/5T6KwPHIcHu3ZRCzg== Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48q557t98u-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 25 Aug 2025 17:12:34 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57PEkQG1002473; Mon, 25 Aug 2025 17:12:33 GMT Received: from smtprelay02.dal12v.mail.ibm.com ([172.16.1.4]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 48qt6m6q6r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 25 Aug 2025 17:12:33 +0000 Received: from smtpav06.wdc07v.mail.ibm.com (smtpav06.wdc07v.mail.ibm.com [10.39.53.233]) by smtprelay02.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57PHCW6q13632014 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 25 Aug 2025 17:12:32 GMT Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0A0A658054; Mon, 25 Aug 2025 17:12:32 +0000 (GMT) Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 011FD5804E; Mon, 25 Aug 2025 17:12:31 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.255.253]) by smtpav06.wdc07v.mail.ibm.com (Postfix) with ESMTP; Mon, 25 Aug 2025 17:12:30 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v2 3/9] PCI: Allow per function PCI slots for hypervisor isolated functions Date: Mon, 25 Aug 2025 10:12:20 -0700 Message-ID: <20250825171226.1602-4-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250825171226.1602-1-alifm@linux.ibm.com> References: <20250825171226.1602-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: G8-qkalkm8D9a4AG629WOFz6yyrtvCLj X-Proofpoint-ORIG-GUID: G8-qkalkm8D9a4AG629WOFz6yyrtvCLj X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODIzMDAyMSBTYWx0ZWRfX5VmsT49mF984 KkkZhRMO8UYbLczP22AdZ6cfH771DwbBV/4NMfnhPoOjyTZUsWQGbaVXFdYn58aGg/Hoz9KZGgO 63sCwkbc18t5kL3BPUOMjfbtQuQ/WSAmmA/6T/EKPd7PRRMOVxRUmUA//r1kwyvy+w/JvX2tbjU tx3qZy5GR98M+fe4Nv6xlfWBPp0rhoWshbazbTLE2K5mKQVCAOowqm928tjX4FE115RJySXkx0j LQQ8l/zhqgW54QstIPFObpfzzy8wr6ppLOAxP13FROsnglToV3ojcwemo6JCihl363hwWoIxOu7 HJCYL11bTdJNd8S8kYF/7yZaatdmHUhzyiHhbl7Svq/acRtl7soZkmSibii5GsOuC+doUJ7lgGA qAOXCmrv X-Authority-Analysis: v=2.4 cv=A8ZsP7WG c=1 sm=1 tr=0 ts=68ac9982 cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=2OwXVqhp2XgA:10 a=VnNF1IyMAAAA:8 a=cH6R9-kdAAAA:8 a=yKnoY3V09h4Zs0NVWIYA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-25_07,2025-08-20_03,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 adultscore=0 priorityscore=1501 suspectscore=0 clxscore=1015 spamscore=0 impostorscore=0 malwarescore=0 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2508230021 Content-Type: text/plain; charset="utf-8" On s390 systems, which use a machine level hypervisor, PCI devices are always accessed through a form of PCI pass-through which fundamentally operates on a per PCI function granularity. This is also reflected in the s390 PCI hotplug driver which creates hotplug slots for individual PCI functions. Its reset_slot() function, which is a wrapper for zpci_hot_reset_device(), thus also resets individual functions. Currently, the kernel's PCI_SLOT() macro assigns the same pci_slot object to multifunction devices. This approach worked fine on s390 systems that only exposed virtual functions as individual PCI domains to the operating system. Since commit 44510d6fa0c0 ("s390/pci: Handling multifunctions") s390 supports exposing the topology of multifunction PCI devices by grouping them in a shared PCI domain. When attempting to reset a function through the hotplug driver, the shared slot assignment causes the wrong function to be reset instead of the intended one. It also leaks memory as we do create a pci_slot object for the function, but don't correctly free it in pci_slot_release(). This patch adds a helper function to allow per function PCI slots for functions managed through a hypervisor which exposes individual PCI functions while retaining the topology. Fixes: 44510d6fa0c0 ("s390/pci: Handling multifunctions") Co-developed-by: Niklas Schnelle Signed-off-by: Niklas Schnelle Signed-off-by: Farhan Ali --- drivers/pci/slot.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/drivers/pci/slot.c b/drivers/pci/slot.c index 50fb3eb595fe..991526af0ffe 100644 --- a/drivers/pci/slot.c +++ b/drivers/pci/slot.c @@ -5,6 +5,7 @@ * Alex Chiang */ =20 +#include #include #include #include @@ -73,7 +74,7 @@ static void pci_slot_release(struct kobject *kobj) =20 down_read(&pci_bus_sem); list_for_each_entry(dev, &slot->bus->devices, bus_list) - if (PCI_SLOT(dev->devfn) =3D=3D slot->number) + if (dev->slot =3D=3D slot->number) dev->slot =3D NULL; up_read(&pci_bus_sem); =20 @@ -160,13 +161,25 @@ static int rename_slot(struct pci_slot *slot, const c= har *name) return result; } =20 +static bool pci_dev_matches_slot(struct pci_dev *dev, struct pci_slot *slo= t) +{ + if (hypervisor_isolated_pci_functions()) { + if (dev->devfn =3D=3D slot->number) + return true; + } else { + if (PCI_SLOT(dev->devfn) =3D=3D slot->number) + return true; + } + return false; +} + void pci_dev_assign_slot(struct pci_dev *dev) { struct pci_slot *slot; =20 mutex_lock(&pci_slot_mutex); list_for_each_entry(slot, &dev->bus->slots, list) - if (PCI_SLOT(dev->devfn) =3D=3D slot->number) + if (pci_dev_matches_slot(dev, slot)) dev->slot =3D slot; mutex_unlock(&pci_slot_mutex); } @@ -285,7 +298,7 @@ struct pci_slot *pci_create_slot(struct pci_bus *parent= , int slot_nr, =20 down_read(&pci_bus_sem); list_for_each_entry(dev, &parent->devices, bus_list) - if (PCI_SLOT(dev->devfn) =3D=3D slot_nr) + if (pci_dev_matches_slot(dev, slot)) dev->slot =3D slot; up_read(&pci_bus_sem); =20 --=20 2.43.0 From nobody Fri Oct 3 20:25:18 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 09BCE3019B0; Mon, 25 Aug 2025 17:12:39 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756141962; cv=none; b=XZu/wa9IY4GKLThsgOmd07x129sPcOJbEgdO+jLSRschV38n8jo7BNmVYy1e7d5glyGQa04GP1n+K6qu9OsRmAqkBtuHCfnDCUiMydjQShgSLwBJzfdXQAHl7j5JihX8urOzghvRvCp5JReZOtB79BY0A19K4r/Cz5mUHLAo730= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756141962; c=relaxed/simple; bh=ap8ZqAi3Gc/07bdX4XFTd8UIRfw9aj7EXhsR7evPuIU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=DvgXNg721YddINAYux+My3Lx4vkkkbbO+zaNLDMIM8UKyuvbu6j6XkpMJYvOyuz6yWpsq484FUWa2nj9xA/QylWEGKE5XoRMP5DhL7VPd4uqn4lYWJCYzNz5m8omiCfavZnR05ALZUFWGLsYTabeDj6KmB8S/ym5LasG5T6jIRc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=aFiv2eDY; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="aFiv2eDY" Received: from pps.filterd (m0353725.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57PGXDc0021533; Mon, 25 Aug 2025 17:12:35 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=qRiIPFXEObypK9lpw Whjwg8ySQsldPVM2VwQexmFjXM=; b=aFiv2eDYNFaR8/shXjLYiYNDYooRWeovx hs9qPtwXQJy1/O/uTvBJ7CyaNr7IaNf+HNhajuNDG7LuXv5Z4JxKJ0563WvvTmRv 7ETzESvyMeHXflDNW1L0G7IzUZ125Lj/uV5B4Oc9mAim53B1GiBpUEEO+elPBpRQ /HBSUGApY/UyIpHQ9R+syAAUoLsQF/uEgPAT4Yl5CoEyehtbtqIZ5Qqngo4CvhbI nWZUYYkLgTywu11WZvtI9T/vZV+HFydeEGg6KOuO7D1yLXMubDORE8bXDjGcxYLk 8K3jAAvO/LaU/slcNQz3gjq4NLhWhk1/loF83VQEh0/rA/3o+Ln+Q== Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48q42htapx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 25 Aug 2025 17:12:35 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57PE9S2r002531; Mon, 25 Aug 2025 17:12:35 GMT Received: from smtprelay03.dal12v.mail.ibm.com ([172.16.1.5]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 48qrypex8a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 25 Aug 2025 17:12:34 +0000 Received: from smtpav06.wdc07v.mail.ibm.com (smtpav06.wdc07v.mail.ibm.com [10.39.53.233]) by smtprelay03.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57PHCXG323397110 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 25 Aug 2025 17:12:33 GMT Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4106558068; Mon, 25 Aug 2025 17:12:33 +0000 (GMT) Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3709858055; Mon, 25 Aug 2025 17:12:32 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.255.253]) by smtpav06.wdc07v.mail.ibm.com (Postfix) with ESMTP; Mon, 25 Aug 2025 17:12:32 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v2 4/9] s390/pci: Restore airq unconditionally for the zPCI device Date: Mon, 25 Aug 2025 10:12:21 -0700 Message-ID: <20250825171226.1602-5-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250825171226.1602-1-alifm@linux.ibm.com> References: <20250825171226.1602-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODIzMDAxMCBTYWx0ZWRfXyAChJSIpk7qg mxqFFhJ8D3rzNKjZfLLkoM5edi27yIrfEkikIG8GNeBOWJD/pXH5qy0Dv+PLwhqvIc/SNMdEhFx Xlya6l9NKva5AO5KchG9e2Kam8GkgLEG7ZMrXL3KgeNjb2GfH6P/jwv6hvXlH5LvLv8p3UvGy3z ZzrJEhWE0fX+elmJJr8VrUg/t1eC97UBG2vqwKcOnPR8yzCmFiGr1IDt8b6k5JvxJ9uHTkWsysU Cu7p0Z5a0CXpECUHGlV20IZ2XwCBZKVhZtsmP0n8CBU0M6hIkvE9v4lo1hnYkFktLwZgBU+rQrp kKbZvezmtMnFmJna4K13GaAnDlfVFJXFfpWlYcFo69BxRGHKhJEy2Y0xIvw5FemAmxpSahcTu40 JD1uv4JX X-Proofpoint-ORIG-GUID: Bk8qRFMIwf-GQuUGUgcfDPsMK4lQKeSC X-Proofpoint-GUID: Bk8qRFMIwf-GQuUGUgcfDPsMK4lQKeSC X-Authority-Analysis: v=2.4 cv=evffzppX c=1 sm=1 tr=0 ts=68ac9983 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=2OwXVqhp2XgA:10 a=VnNF1IyMAAAA:8 a=gAMvz2k-Oe9gsD3n2XQA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-25_08,2025-08-20_03,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 adultscore=0 malwarescore=0 spamscore=0 bulkscore=0 impostorscore=0 clxscore=1015 priorityscore=1501 phishscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2508230010 Content-Type: text/plain; charset="utf-8" Commit c1e18c17bda6 ("s390/pci: add zpci_set_irq()/zpci_clear_irq()"), introduced the zpci_set_irq() and zpci_clear_irq(), to be used while resetting a zPCI device. Commit da995d538d3a ("s390/pci: implement reset_slot for hotplug slot"), mentions zpci_clear_irq() being called in the path for zpci_hot_reset_devic= e(). But that is not the case anymore and these functions are not called outside of this file. However after a CLP disable/enable reset (zpci_hot_reset_device),the airq setup of the device will need to be restored. Since we are no longer calling zpci_clear_airq() in the reset path, we should restore the airq for device unconditionally. Signed-off-by: Farhan Ali --- arch/s390/include/asm/pci.h | 1 - arch/s390/pci/pci_irq.c | 9 +-------- 2 files changed, 1 insertion(+), 9 deletions(-) diff --git a/arch/s390/include/asm/pci.h b/arch/s390/include/asm/pci.h index 41f900f693d9..aed19a1aa9d7 100644 --- a/arch/s390/include/asm/pci.h +++ b/arch/s390/include/asm/pci.h @@ -145,7 +145,6 @@ struct zpci_dev { u8 has_resources : 1; u8 is_physfn : 1; u8 util_str_avail : 1; - u8 irqs_registered : 1; u8 tid_avail : 1; u8 rtr_avail : 1; /* Relaxed translation allowed */ unsigned int devfn; /* DEVFN part of the RID*/ diff --git a/arch/s390/pci/pci_irq.c b/arch/s390/pci/pci_irq.c index 84482a921332..e73be96ce5fe 100644 --- a/arch/s390/pci/pci_irq.c +++ b/arch/s390/pci/pci_irq.c @@ -107,9 +107,6 @@ static int zpci_set_irq(struct zpci_dev *zdev) else rc =3D zpci_set_airq(zdev); =20 - if (!rc) - zdev->irqs_registered =3D 1; - return rc; } =20 @@ -123,9 +120,6 @@ static int zpci_clear_irq(struct zpci_dev *zdev) else rc =3D zpci_clear_airq(zdev); =20 - if (!rc) - zdev->irqs_registered =3D 0; - return rc; } =20 @@ -427,8 +421,7 @@ bool arch_restore_msi_irqs(struct pci_dev *pdev) { struct zpci_dev *zdev =3D to_zpci(pdev); =20 - if (!zdev->irqs_registered) - zpci_set_irq(zdev); + zpci_set_irq(zdev); return true; } =20 --=20 2.43.0 From nobody Fri Oct 3 20:25:18 2025 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 9EC07304BB4; Mon, 25 Aug 2025 17:12:40 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756141964; cv=none; b=NptN7NgaT1uvhDEHIH5k+K8QluEVx7RbLWB5PedYdbQL/PURBiNEOdoSYRcJXwGjLrYwRKY+sJREUW2V4CD1UPplsQcgJfcQ032r/fFSGlTMhiZSdIVHvyxZXiUTII4vC13+nOopzHrOhC74VMzDSloVyofUCQA4DT/7bdm58Wk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756141964; c=relaxed/simple; bh=HiF8yIXpdsl3GpcWw1U7Xg6P+sqRjiRYAkEPz33XzDQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lPxq498GSkIxfB04TzqObkxvYLUU5oso2FqMNv0AqC5dYL3qBBcvQcxhGChWwLVGQDz+jvpVTea/h7zgQoH+UzpwYJ/GGDVvMGGN7iXIc+90V1P9+RjR4oINUmc+Fv0ez0DlElkl+yIdC863vHefXovlLZpHiu17Jw+vwPNJaSg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=M661WLh6; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="M661WLh6" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57P8o7eF028452; Mon, 25 Aug 2025 17:12:36 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=mmVlLOIe9nYcrSc3r Rudb0uYz3qoo02ifUZ1lexk/NU=; b=M661WLh61+3qiD7CHn3CWLFFhLt/ffawh xr201lmQsznO7tJ3cHk1h/7rkhuaaOrs+ocnIsy8w8uBDfUnzz0nMNY49QHwbYgi nAUIlw5hBvmzE3PrHRwPB7mBvsyKKx29WKlSoZYDN+s0H9dJ7GeM+hw8EAd6USk/ ZLqcJAJblCeHENhGvjozpRaYomCTA3jqQaV8+d/t6ZZGTfjhiUDCTkSfmY4u/8pz 1KbsfyjJDfQAgZ7+3oDvoaN2V5ALlTq3H2okQVHJ5au3g8W43fctH3t6EtXMylTa 13DAYoqgWa7Zot9ysPNukCfoN8JpzilM+s8ufOhnyGEjfsWIzpwtQ== Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48q5hpt3qa-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 25 Aug 2025 17:12:36 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57PEf8A7002512; Mon, 25 Aug 2025 17:12:35 GMT Received: from smtprelay04.dal12v.mail.ibm.com ([172.16.1.6]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 48qt6m6q6x-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 25 Aug 2025 17:12:35 +0000 Received: from smtpav06.wdc07v.mail.ibm.com (smtpav06.wdc07v.mail.ibm.com [10.39.53.233]) by smtprelay04.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57PHCY3O3539462 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 25 Aug 2025 17:12:34 GMT Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 61B515804E; Mon, 25 Aug 2025 17:12:34 +0000 (GMT) Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6FF7E58055; Mon, 25 Aug 2025 17:12:33 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.255.253]) by smtpav06.wdc07v.mail.ibm.com (Postfix) with ESMTP; Mon, 25 Aug 2025 17:12:33 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v2 5/9] s390/pci: Update the logic for detecting passthrough device Date: Mon, 25 Aug 2025 10:12:22 -0700 Message-ID: <20250825171226.1602-6-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250825171226.1602-1-alifm@linux.ibm.com> References: <20250825171226.1602-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODIzMDAyMSBTYWx0ZWRfX6aY2olrKmA39 Y1zOyM7TQdd+yU0M9vXCWPzAXqqFFvZpPGw3B0r8CLbozklpEpX4l0MW5259ZG177zNIJDbXb3C EvwpsrDaIgH+omiOvcSHWBTfUu9IwJgc7YCU9hf7AhhWZGBamSfvapwddTkESP4G788hV7LRn0s yV5v2nwySvUfFlf9DAWUyAjdhdPVf9OjZ3gBrtmlfR+iMToa3vif5YX1+hWgG+eVBpyYAMGZ3EL 5XVctGqEyd7YLjPQFgAsJypqGGM+9vQxNA60lVejktre9MrfyCDU9Ayh+NJh/NApGAZDEqnWmpE DU+ukEdLJhxt1dUudIw1pnb7ukmoucmZ6qNpyNlRMTs9iRktBq+PewNQIPBq/FECHgSZ0Sh3NVT wHugfmPR X-Proofpoint-ORIG-GUID: Ta6eaxpeN5Qz0htYOnIh6Z0Z7hn3lzxg X-Proofpoint-GUID: Ta6eaxpeN5Qz0htYOnIh6Z0Z7hn3lzxg X-Authority-Analysis: v=2.4 cv=Ndbm13D4 c=1 sm=1 tr=0 ts=68ac9984 cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=2OwXVqhp2XgA:10 a=VnNF1IyMAAAA:8 a=MtaQWmmsotiDqj5H3ecA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-25_08,2025-08-20_03,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 impostorscore=0 clxscore=1015 phishscore=0 malwarescore=0 suspectscore=0 bulkscore=0 spamscore=0 adultscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2508230021 Content-Type: text/plain; charset="utf-8" We can now have userspace drivers (vfio-pci based) on s390x. The userspace drivers will not have any KVM fd and so no kzdev associated with them. So we need to update the logic for detecting passthrough devices to not depend on struct kvm_zdev. Signed-off-by: Farhan Ali --- arch/s390/include/asm/pci.h | 1 + arch/s390/pci/pci_event.c | 14 ++++---------- drivers/vfio/pci/vfio_pci_zdev.c | 9 ++++++++- 3 files changed, 13 insertions(+), 11 deletions(-) diff --git a/arch/s390/include/asm/pci.h b/arch/s390/include/asm/pci.h index aed19a1aa9d7..f47f62fc3bfd 100644 --- a/arch/s390/include/asm/pci.h +++ b/arch/s390/include/asm/pci.h @@ -169,6 +169,7 @@ struct zpci_dev { =20 char res_name[16]; bool mio_capable; + bool mediated_recovery; struct zpci_bar_struct bars[PCI_STD_NUM_BARS]; =20 u64 start_dma; /* Start of available DMA addresses */ diff --git a/arch/s390/pci/pci_event.c b/arch/s390/pci/pci_event.c index d930416d4c90..541d536be052 100644 --- a/arch/s390/pci/pci_event.c +++ b/arch/s390/pci/pci_event.c @@ -61,16 +61,10 @@ static inline bool ers_result_indicates_abort(pci_ers_r= esult_t ers_res) } } =20 -static bool is_passed_through(struct pci_dev *pdev) +static bool needs_mediated_recovery(struct pci_dev *pdev) { struct zpci_dev *zdev =3D to_zpci(pdev); - bool ret; - - mutex_lock(&zdev->kzdev_lock); - ret =3D !!zdev->kzdev; - mutex_unlock(&zdev->kzdev_lock); - - return ret; + return zdev->mediated_recovery; } =20 static bool is_driver_supported(struct pci_driver *driver) @@ -194,7 +188,7 @@ static pci_ers_result_t zpci_event_attempt_error_recove= ry(struct pci_dev *pdev) } pdev->error_state =3D pci_channel_io_frozen; =20 - if (is_passed_through(pdev)) { + if (needs_mediated_recovery(pdev)) { pr_info("%s: Cannot be recovered in the host because it is a pass-throug= h device\n", pci_name(pdev)); status_str =3D "failed (pass-through)"; @@ -277,7 +271,7 @@ static void zpci_event_io_failure(struct pci_dev *pdev,= pci_channel_state_t es) * we will inject the error event and let the guest recover the device * itself. */ - if (is_passed_through(pdev)) + if (needs_mediated_recovery(pdev)) goto out; driver =3D to_pci_driver(pdev->dev.driver); if (driver && driver->err_handler && driver->err_handler->error_detected) diff --git a/drivers/vfio/pci/vfio_pci_zdev.c b/drivers/vfio/pci/vfio_pci_z= dev.c index 0990fdb146b7..a7bc23ce8483 100644 --- a/drivers/vfio/pci/vfio_pci_zdev.c +++ b/drivers/vfio/pci/vfio_pci_zdev.c @@ -148,6 +148,8 @@ int vfio_pci_zdev_open_device(struct vfio_pci_core_devi= ce *vdev) if (!zdev) return -ENODEV; =20 + zdev->mediated_recovery =3D true; + if (!vdev->vdev.kvm) return 0; =20 @@ -161,7 +163,12 @@ void vfio_pci_zdev_close_device(struct vfio_pci_core_d= evice *vdev) { struct zpci_dev *zdev =3D to_zpci(vdev->pdev); =20 - if (!zdev || !vdev->vdev.kvm) + if (!zdev) + return; + + zdev->mediated_recovery =3D false; + + if (!vdev->vdev.kvm) return; =20 if (zpci_kvm_hook.kvm_unregister) --=20 2.43.0 From nobody Fri Oct 3 20:25:18 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CA4763054C8; Mon, 25 Aug 2025 17:12:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756141966; cv=none; b=LCW8UzV5s6gEBHuGBGFI5222LiCwNN05AxJ1Zapw0WIcHkCDLclvuhgddkaEfZpXMH3ANF7wg7lCVZFfvqcwgD4YBWBtD3+CsEdAWzE9EzFqkdCfW6k7Wo978i9ZDtiPHtz222/OJ4gQGooe2nBnm1d6r4nnfXgeWqRPlxqo9Hs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756141966; c=relaxed/simple; bh=NzQE0JrNlIZtKxZDh/CaFkHSC/E5PRD30PkB+w2hkB0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=PaYWErIX38MrlPGAeCKM6foAIpXvEJ8QlrT7MonCVDsLHcNsw4M/vG7NjufhuKv4ZYBjIS61w/buQSC8P/0YwCTU0VRuvzOK31kVRwXlsqXTaYakH3+grg+ijJoXpr8/lKsjOcQps2aFQPJP4S7u+5n41lUtCvsUJEEJx1g5mvU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=bfQhkosn; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="bfQhkosn" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57PCimsp021280; Mon, 25 Aug 2025 17:12:37 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=IjwZ53nzS+82KSHX1 vCRae+coPn2P/6vTVF240yzO2Q=; b=bfQhkosnnq/NMNb/bGkPpHbmxq1R5XgSR o9qUF9/yOsPyfZS0Du+LClfrofdW09YW6DootgTmam+dCalks1jAi91GdzCFPjXs NY//jm9pPxYNfIt8XRObp39O+4rL/vvycK3EKuiz7TODrPmnLMHJ1WlstUzhvkJn O3evcvr1JHOPEqqzQOyKjXGWiEgE9UFB9aqF3XNxL0Loc2MAmyZw+nz+bUul+rO5 qgfLamGTlTax/+AVLy4jFZm0SdVOUlSWICzBO0Nsy7JR1dBlk3Pj3KAAh4gftYyg IYQZOgizrWWtIHx0bNg41wpGKh4U81DNez3AQUq1gIaaBvzsaMuHQ== Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48q9751jxv-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 25 Aug 2025 17:12:37 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57PGg1nA007443; Mon, 25 Aug 2025 17:12:36 GMT Received: from smtprelay05.dal12v.mail.ibm.com ([172.16.1.7]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 48qqyu74qs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 25 Aug 2025 17:12:36 +0000 Received: from smtpav06.wdc07v.mail.ibm.com (smtpav06.wdc07v.mail.ibm.com [10.39.53.233]) by smtprelay05.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57PHCZmS590476 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 25 Aug 2025 17:12:35 GMT Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 7A22A5804E; Mon, 25 Aug 2025 17:12:35 +0000 (GMT) Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8D49A58054; Mon, 25 Aug 2025 17:12:34 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.255.253]) by smtpav06.wdc07v.mail.ibm.com (Postfix) with ESMTP; Mon, 25 Aug 2025 17:12:34 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v2 6/9] s390/pci: Store PCI error information for passthrough devices Date: Mon, 25 Aug 2025 10:12:23 -0700 Message-ID: <20250825171226.1602-7-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250825171226.1602-1-alifm@linux.ibm.com> References: <20250825171226.1602-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: r7YP_xOdU2giQxu_M-tgYySbnm_XzMvU X-Proofpoint-ORIG-GUID: r7YP_xOdU2giQxu_M-tgYySbnm_XzMvU X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODIzMDA3MSBTYWx0ZWRfX43/yvd5mSAlj Zwtk88nz5/6Rl+A2Y8PldarIHz0NmKKMXWxY5MzQGKiUtMElMIG1vJKpWegidHmaWGnEY6TCvX8 lWRYTeYHTDUuOM0mVntv5TFdwo8g5D0pnld7G+tbCQ3d0B9ih02SyN0016HM1ZnibcRF22g4+70 alcf+aqNVUnNcNJ1DCElLY9LS6WhJzsrR1oSvNtElD4nHsYDIYNRPpqkTBQ9dcdTwkmir06xPFW qUpCQos/wBl1aemI5j9VAqwyWSXoyFk8wt5OHfhNFzXdmfCoZ+5iM6o9GZltD4ZuLPep2eitu5p cZA0/2DqgJHoXcsG2SXsL/iWGs9yQtOHJFOMEq/LG/nxKWr06YNYoVtn3kgTJ7rlE2oltVq4cxX JFPA52Ba X-Authority-Analysis: v=2.4 cv=RtDFLDmK c=1 sm=1 tr=0 ts=68ac9985 cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=2OwXVqhp2XgA:10 a=VnNF1IyMAAAA:8 a=AhHhU-3714CzMtceQIIA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-25_08,2025-08-20_03,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 suspectscore=0 malwarescore=0 spamscore=0 priorityscore=1501 bulkscore=0 clxscore=1015 phishscore=0 adultscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2508230071 Content-Type: text/plain; charset="utf-8" For a passthrough device we need co-operation from user space to recover the device. This would require to bubble up any error information to user space. Let's store this error information for passthrough devices, so it can be retrieved later. Signed-off-by: Farhan Ali --- arch/s390/include/asm/pci.h | 28 ++++++++++ arch/s390/pci/pci.c | 1 + arch/s390/pci/pci_event.c | 95 +++++++++++++++++++------------- drivers/vfio/pci/vfio_pci_zdev.c | 2 + 4 files changed, 88 insertions(+), 38 deletions(-) diff --git a/arch/s390/include/asm/pci.h b/arch/s390/include/asm/pci.h index f47f62fc3bfd..72e05af90e08 100644 --- a/arch/s390/include/asm/pci.h +++ b/arch/s390/include/asm/pci.h @@ -116,6 +116,31 @@ struct zpci_bus { enum pci_bus_speed max_bus_speed; }; =20 +/* Content Code Description for PCI Function Error */ +struct zpci_ccdf_err { + u32 reserved1; + u32 fh; /* function handle */ + u32 fid; /* function id */ + u32 ett : 4; /* expected table type */ + u32 mvn : 12; /* MSI vector number */ + u32 dmaas : 8; /* DMA address space */ + u32 reserved2 : 6; + u32 q : 1; /* event qualifier */ + u32 rw : 1; /* read/write */ + u64 faddr; /* failing address */ + u32 reserved3; + u16 reserved4; + u16 pec; /* PCI event code */ +} __packed; + +#define ZPCI_ERR_PENDING_MAX 16 +struct zpci_ccdf_pending { + size_t count; + int head; + int tail; + struct zpci_ccdf_err err[ZPCI_ERR_PENDING_MAX]; +}; + /* Private data per function */ struct zpci_dev { struct zpci_bus *zbus; @@ -191,6 +216,8 @@ struct zpci_dev { struct iommu_domain *s390_domain; /* attached IOMMU domain */ struct kvm_zdev *kzdev; struct mutex kzdev_lock; + struct zpci_ccdf_pending pending_errs; + struct mutex pending_errs_lock; spinlock_t dom_lock; /* protect s390_domain change */ }; =20 @@ -316,6 +343,7 @@ void zpci_debug_exit_device(struct zpci_dev *); int zpci_report_error(struct pci_dev *, struct zpci_report_error_header *); int zpci_clear_error_state(struct zpci_dev *zdev); int zpci_reset_load_store_blocked(struct zpci_dev *zdev); +void zpci_cleanup_pending_errors(struct zpci_dev *zdev); =20 #ifdef CONFIG_NUMA =20 diff --git a/arch/s390/pci/pci.c b/arch/s390/pci/pci.c index cd6676c2d602..f795e05b5001 100644 --- a/arch/s390/pci/pci.c +++ b/arch/s390/pci/pci.c @@ -823,6 +823,7 @@ struct zpci_dev *zpci_create_device(u32 fid, u32 fh, en= um zpci_state state) mutex_init(&zdev->state_lock); mutex_init(&zdev->fmb_lock); mutex_init(&zdev->kzdev_lock); + mutex_init(&zdev->pending_errs_lock); =20 return zdev; =20 diff --git a/arch/s390/pci/pci_event.c b/arch/s390/pci/pci_event.c index 541d536be052..ac527410812b 100644 --- a/arch/s390/pci/pci_event.c +++ b/arch/s390/pci/pci_event.c @@ -18,23 +18,6 @@ #include "pci_bus.h" #include "pci_report.h" =20 -/* Content Code Description for PCI Function Error */ -struct zpci_ccdf_err { - u32 reserved1; - u32 fh; /* function handle */ - u32 fid; /* function id */ - u32 ett : 4; /* expected table type */ - u32 mvn : 12; /* MSI vector number */ - u32 dmaas : 8; /* DMA address space */ - u32 : 6; - u32 q : 1; /* event qualifier */ - u32 rw : 1; /* read/write */ - u64 faddr; /* failing address */ - u32 reserved3; - u16 reserved4; - u16 pec; /* PCI event code */ -} __packed; - /* Content Code Description for PCI Function Availability */ struct zpci_ccdf_avail { u32 reserved1; @@ -76,6 +59,41 @@ static bool is_driver_supported(struct pci_driver *drive= r) return true; } =20 +static void zpci_store_pci_error(struct pci_dev *pdev, + struct zpci_ccdf_err *ccdf) +{ + struct zpci_dev *zdev =3D to_zpci(pdev); + int i; + + mutex_lock(&zdev->pending_errs_lock); + if (zdev->pending_errs.count >=3D ZPCI_ERR_PENDING_MAX) { + pr_err("%s: Cannot store PCI error info for device", + pci_name(pdev)); + mutex_unlock(&zdev->pending_errs_lock); + return; + } + + i =3D zdev->pending_errs.tail % ZPCI_ERR_PENDING_MAX; + memcpy(&zdev->pending_errs.err[i], ccdf, sizeof(struct zpci_ccdf_err)); + zdev->pending_errs.tail++; + zdev->pending_errs.count++; + mutex_unlock(&zdev->pending_errs_lock); +} + +void zpci_cleanup_pending_errors(struct zpci_dev *zdev) +{ + struct pci_dev *pdev =3D NULL; + + mutex_lock(&zdev->pending_errs_lock); + pdev =3D pci_get_slot(zdev->zbus->bus, zdev->devfn); + if (zdev->pending_errs.count) + pr_err("%s: Unhandled PCI error events count=3D%zu", + pci_name(pdev), zdev->pending_errs.count); + memset(&zdev->pending_errs, 0, sizeof(struct zpci_ccdf_pending)); + mutex_unlock(&zdev->pending_errs_lock); +} +EXPORT_SYMBOL_GPL(zpci_cleanup_pending_errors); + static pci_ers_result_t zpci_event_notify_error_detected(struct pci_dev *p= dev, struct pci_driver *driver) { @@ -169,7 +187,8 @@ static pci_ers_result_t zpci_event_do_reset(struct pci_= dev *pdev, * and the platform determines which functions are affected for * multi-function devices. */ -static pci_ers_result_t zpci_event_attempt_error_recovery(struct pci_dev *= pdev) +static pci_ers_result_t zpci_event_attempt_error_recovery(struct pci_dev *= pdev, + struct zpci_ccdf_err *ccdf) { pci_ers_result_t ers_res =3D PCI_ERS_RESULT_DISCONNECT; struct zpci_dev *zdev =3D to_zpci(pdev); @@ -188,13 +207,6 @@ static pci_ers_result_t zpci_event_attempt_error_recov= ery(struct pci_dev *pdev) } pdev->error_state =3D pci_channel_io_frozen; =20 - if (needs_mediated_recovery(pdev)) { - pr_info("%s: Cannot be recovered in the host because it is a pass-throug= h device\n", - pci_name(pdev)); - status_str =3D "failed (pass-through)"; - goto out_unlock; - } - driver =3D to_pci_driver(pdev->dev.driver); if (!is_driver_supported(driver)) { if (!driver) { @@ -210,12 +222,22 @@ static pci_ers_result_t zpci_event_attempt_error_reco= very(struct pci_dev *pdev) goto out_unlock; } =20 + if (needs_mediated_recovery(pdev)) + zpci_store_pci_error(pdev, ccdf); + ers_res =3D zpci_event_notify_error_detected(pdev, driver); if (ers_result_indicates_abort(ers_res)) { status_str =3D "failed (abort on detection)"; goto out_unlock; } =20 + if (needs_mediated_recovery(pdev)) { + pr_info("%s: Recovering passthrough device\n", pci_name(pdev)); + ers_res =3D PCI_ERS_RESULT_RECOVERED; + status_str =3D "in progress"; + goto out_unlock; + } + if (ers_res !=3D PCI_ERS_RESULT_NEED_RESET) { ers_res =3D zpci_event_do_error_state_clear(pdev, driver); if (ers_result_indicates_abort(ers_res)) { @@ -258,25 +280,20 @@ static pci_ers_result_t zpci_event_attempt_error_reco= very(struct pci_dev *pdev) * @pdev: PCI function for which to report * @es: PCI channel failure state to report */ -static void zpci_event_io_failure(struct pci_dev *pdev, pci_channel_state_= t es) +static void zpci_event_io_failure(struct pci_dev *pdev, pci_channel_state_= t es, + struct zpci_ccdf_err *ccdf) { struct pci_driver *driver; =20 pci_dev_lock(pdev); pdev->error_state =3D es; - /** - * While vfio-pci's error_detected callback notifies user-space QEMU - * reacts to this by freezing the guest. In an s390 environment PCI - * errors are rarely fatal so this is overkill. Instead in the future - * we will inject the error event and let the guest recover the device - * itself. - */ + if (needs_mediated_recovery(pdev)) - goto out; + zpci_store_pci_error(pdev, ccdf); driver =3D to_pci_driver(pdev->dev.driver); if (driver && driver->err_handler && driver->err_handler->error_detected) driver->err_handler->error_detected(pdev, pdev->error_state); -out: + pci_dev_unlock(pdev); } =20 @@ -312,6 +329,7 @@ static void __zpci_event_error(struct zpci_ccdf_err *cc= df) pr_err("%s: Event 0x%x reports an error for PCI function 0x%x\n", pdev ? pci_name(pdev) : "n/a", ccdf->pec, ccdf->fid); =20 + if (!pdev) goto no_pdev; =20 @@ -322,12 +340,13 @@ static void __zpci_event_error(struct zpci_ccdf_err *= ccdf) break; case 0x0040: /* Service Action or Error Recovery Failed */ case 0x003b: - zpci_event_io_failure(pdev, pci_channel_io_perm_failure); + zpci_event_io_failure(pdev, pci_channel_io_perm_failure, ccdf); break; default: /* PCI function left in the error state attempt to recover */ - ers_res =3D zpci_event_attempt_error_recovery(pdev); + ers_res =3D zpci_event_attempt_error_recovery(pdev, ccdf); if (ers_res !=3D PCI_ERS_RESULT_RECOVERED) - zpci_event_io_failure(pdev, pci_channel_io_perm_failure); + zpci_event_io_failure(pdev, pci_channel_io_perm_failure, + ccdf); break; } pci_dev_put(pdev); diff --git a/drivers/vfio/pci/vfio_pci_zdev.c b/drivers/vfio/pci/vfio_pci_z= dev.c index a7bc23ce8483..2be37eab9279 100644 --- a/drivers/vfio/pci/vfio_pci_zdev.c +++ b/drivers/vfio/pci/vfio_pci_zdev.c @@ -168,6 +168,8 @@ void vfio_pci_zdev_close_device(struct vfio_pci_core_de= vice *vdev) =20 zdev->mediated_recovery =3D false; =20 + zpci_cleanup_pending_errors(zdev); + if (!vdev->vdev.kvm) return; =20 --=20 2.43.0 From nobody Fri Oct 3 20:25:18 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 798B63054FF; Mon, 25 Aug 2025 17:12:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756141965; cv=none; b=V/T65XAaeCNa5mX1clknf1eFq0TOz63MwaXqIkAMUtGHdjrCRZZ8MpOBKiA4jEC23DJpkLP+Y+nbpFUEPoTPjEIdfBDf1BNRR9WqkxIOSeJS5PndeHgL0WYi3zTBMUosaJ1viRgTrNFLk0krPve3KJz6Q2IgSXEbiUr/PCSXZ9c= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756141965; c=relaxed/simple; bh=VKeAI6fRQ/t72OGx2CEndr/00kQZFDBkNl8vad6U7fM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JHyItbxKaIXQcrzGOF54BciTfOWpaBIMiCjTWXdtkkCp8Q6yTFwLYZ1KJkrTkDvFw7p0c4+g2aqNOIxhWzNNMd2UwCC0oXXXCOZlzte0/tCTmUZpz2ERzr4vL+jz0Wlmgh5uQqjoQWHITifcB0JtVycU3KMua8htJ9Asg8s87IU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=pY4TYsS9; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="pY4TYsS9" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57PGZYgP004119; Mon, 25 Aug 2025 17:12:39 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=uNqMkd8x9H8pA4z+q I3lWSnlmkia9DOlDSAcCRov0BY=; b=pY4TYsS9l4//PxPqQc2tMa/hygPjfR681 7b2MLqpCV012sUabbzfFxY6jF9pu86KgIgDA0WaOa8V31/Yd4t7aGezQplCEH0JL yTUJSPhM6XKutaBOPcXQ5zXLIaBD1QQYqD55fgDHQWDWhNhzrfYmWZchYoKTSq60 mJMAJeGmv6CqbYrFisR1AJos/pkKNOWOQImQcwXTuSLzxK0C7NfrDX0iOrpqGo4/ sNx5Wuntsq/hVuUHk6GjU1rvWWNYYnwippVwHov7tbdvTYGhChoMN7Fw33v+5faJ 0PhmMpr5HB/TA9FAsYyXJzOMg2K+nbJyM6OuWLkNOezn9IQ0ufULg== Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48q557t991-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 25 Aug 2025 17:12:38 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57PEttEv017989; Mon, 25 Aug 2025 17:12:37 GMT Received: from smtprelay06.dal12v.mail.ibm.com ([172.16.1.8]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 48qtp36n6b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 25 Aug 2025 17:12:37 +0000 Received: from smtpav06.wdc07v.mail.ibm.com (smtpav06.wdc07v.mail.ibm.com [10.39.53.233]) by smtprelay06.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57PHCaTm60162366 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 25 Aug 2025 17:12:37 GMT Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8F62658064; Mon, 25 Aug 2025 17:12:36 +0000 (GMT) Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id A494958055; Mon, 25 Aug 2025 17:12:35 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.255.253]) by smtpav06.wdc07v.mail.ibm.com (Postfix) with ESMTP; Mon, 25 Aug 2025 17:12:35 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v2 7/9] vfio-pci/zdev: Add a device feature for error information Date: Mon, 25 Aug 2025 10:12:24 -0700 Message-ID: <20250825171226.1602-8-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250825171226.1602-1-alifm@linux.ibm.com> References: <20250825171226.1602-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: nCYwFIq9fAeIvEhwAJBOogQMUyrg9F8k X-Proofpoint-ORIG-GUID: nCYwFIq9fAeIvEhwAJBOogQMUyrg9F8k X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODIzMDAyMSBTYWx0ZWRfX/3mvOEseskWx nucxePvVa6x2WvizsDs7pKipG/H4cEYIL4NlynFPP2MDzGdCcW+/1wZPxcSGUBfASeZ8IhvMoVn v07C5KD2a6/GT2xbIpijFL3hD2/TL4I2XaoOmkrow6ot0XNBsFlkn7gweJlIeXO1gypMxme5qi3 yFD4sKgN0FOmhvBGuCLWwhBXyyBEkjKwiS4+dy5BFFhS5pdaSS7hffBAB92Ez2NY/8IX7BN1nU7 T/uNYdyRMTPVDHxbPbDy9QKc6J0NjQjFl9AMzCUNnNnwqFeZsa35dYQxKZ1pRhRsVfThfPGHNgN HSZzYUEl7BLbgP48Wg9gmwuXG1ky7jaSjD31hn9ZrEACTR7V8HMwatwKowcz5KDSS9559oc7U3w 8zeJKkDY X-Authority-Analysis: v=2.4 cv=A8ZsP7WG c=1 sm=1 tr=0 ts=68ac9986 cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=2OwXVqhp2XgA:10 a=VnNF1IyMAAAA:8 a=QIy8Zix3hhx26zbqT_4A:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-25_07,2025-08-20_03,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 phishscore=0 adultscore=0 priorityscore=1501 suspectscore=0 clxscore=1015 spamscore=0 impostorscore=0 malwarescore=0 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2508230021 Content-Type: text/plain; charset="utf-8" For zPCI devices, we have platform specific error information. The platform firmware provides this error information to the operating system in an architecture specific mechanism. To enable recovery from userspace for these devices, we want to expose this error information to userspace. Add a new device feature to expose this information. Signed-off-by: Farhan Ali --- drivers/vfio/pci/vfio_pci_core.c | 2 ++ drivers/vfio/pci/vfio_pci_priv.h | 8 ++++++++ drivers/vfio/pci/vfio_pci_zdev.c | 34 ++++++++++++++++++++++++++++++++ include/uapi/linux/vfio.h | 14 +++++++++++++ 4 files changed, 58 insertions(+) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index 7dcf5439dedc..378adb3226db 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -1514,6 +1514,8 @@ int vfio_pci_core_ioctl_feature(struct vfio_device *d= evice, u32 flags, return vfio_pci_core_pm_exit(device, flags, arg, argsz); case VFIO_DEVICE_FEATURE_PCI_VF_TOKEN: return vfio_pci_core_feature_token(device, flags, arg, argsz); + case VFIO_DEVICE_FEATURE_ZPCI_ERROR: + return vfio_pci_zdev_feature_err(device, flags, arg, argsz); default: return -ENOTTY; } diff --git a/drivers/vfio/pci/vfio_pci_priv.h b/drivers/vfio/pci/vfio_pci_p= riv.h index a9972eacb293..a4a7f97fdc2e 100644 --- a/drivers/vfio/pci/vfio_pci_priv.h +++ b/drivers/vfio/pci/vfio_pci_priv.h @@ -86,6 +86,8 @@ int vfio_pci_info_zdev_add_caps(struct vfio_pci_core_devi= ce *vdev, struct vfio_info_cap *caps); int vfio_pci_zdev_open_device(struct vfio_pci_core_device *vdev); void vfio_pci_zdev_close_device(struct vfio_pci_core_device *vdev); +int vfio_pci_zdev_feature_err(struct vfio_device *device, u32 flags, + void __user *arg, size_t argsz); #else static inline int vfio_pci_info_zdev_add_caps(struct vfio_pci_core_device = *vdev, struct vfio_info_cap *caps) @@ -100,6 +102,12 @@ static inline int vfio_pci_zdev_open_device(struct vfi= o_pci_core_device *vdev) =20 static inline void vfio_pci_zdev_close_device(struct vfio_pci_core_device = *vdev) {} + +static int vfio_pci_zdev_feature_err(struct vfio_device *device, u32 flags, + void __user *arg, size_t argsz); +{ + return -ENODEV; +} #endif =20 static inline bool vfio_pci_is_vga(struct pci_dev *pdev) diff --git a/drivers/vfio/pci/vfio_pci_zdev.c b/drivers/vfio/pci/vfio_pci_z= dev.c index 2be37eab9279..261954039aa9 100644 --- a/drivers/vfio/pci/vfio_pci_zdev.c +++ b/drivers/vfio/pci/vfio_pci_zdev.c @@ -141,6 +141,40 @@ int vfio_pci_info_zdev_add_caps(struct vfio_pci_core_d= evice *vdev, return ret; } =20 +int vfio_pci_zdev_feature_err(struct vfio_device *device, u32 flags, + void __user *arg, size_t argsz) +{ + struct vfio_device_feature_zpci_err err; + struct vfio_pci_core_device *vdev =3D + container_of(device, struct vfio_pci_core_device, vdev); + struct zpci_dev *zdev =3D to_zpci(vdev->pdev); + int ret; + int head =3D 0; + + if (!zdev) + return -ENODEV; + + ret =3D vfio_check_feature(flags, argsz, VFIO_DEVICE_FEATURE_GET, + sizeof(err)); + if (ret !=3D 1) + return ret; + + mutex_lock(&zdev->pending_errs_lock); + if (zdev->pending_errs.count) { + head =3D zdev->pending_errs.head % ZPCI_ERR_PENDING_MAX; + err.pec =3D zdev->pending_errs.err[head].pec; + zdev->pending_errs.head++; + zdev->pending_errs.count--; + err.pending_errors =3D zdev->pending_errs.count; + } + mutex_unlock(&zdev->pending_errs_lock); + + if (copy_to_user(arg, &err, sizeof(err))) + return -EFAULT; + + return 0; +} + int vfio_pci_zdev_open_device(struct vfio_pci_core_device *vdev) { struct zpci_dev *zdev =3D to_zpci(vdev->pdev); diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h index 75100bf009ba..a950c341602d 100644 --- a/include/uapi/linux/vfio.h +++ b/include/uapi/linux/vfio.h @@ -1478,6 +1478,20 @@ struct vfio_device_feature_bus_master { }; #define VFIO_DEVICE_FEATURE_BUS_MASTER 10 =20 +/** + * VFIO_DEVICE_FEATURE_ZPCI_ERROR feature provides PCI error information to + * userspace for vfio-pci devices on s390x. On s390x PCI error recovery in= volves + * platform firmware and notification to operating system is done by + * architecture specific mechanism. Exposing this information to userspace + * allows userspace to take appropriate actions to handle an error on the + * device. + */ +struct vfio_device_feature_zpci_err { + __u16 pec; + int pending_errors; +}; +#define VFIO_DEVICE_FEATURE_ZPCI_ERROR 11 + /* -------- API for Type1 VFIO IOMMU -------- */ =20 /** --=20 2.43.0 From nobody Fri Oct 3 20:25:18 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 408DE305E21; Mon, 25 Aug 2025 17:12:43 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756141965; cv=none; b=MUOdGThPOmtqgY3mmqi9Otxk8Dj7pl7vB1Vo5cB90Td5vpjznaLTdwRAQoi8d0VlHMShSBExGGpp96jL+33kv6nsMJrGmxXTXu4XVZ+wJQnMvKOhO1uTnnHLbb5TvRaRbNNf6439aNqWEDDWgISgDiavcQuU2r3645xTgHfgH5E= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756141965; c=relaxed/simple; bh=ytFXbNEKbWNWIa4e99DqMpdEgtgPube/J3VaS5HzAEQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=u30gbGN5PkfUD6XN3RgtJjF3gSEVOq6DKFRODxb9yJPOxk2c/UR4ifxCooE+0Gt/9iiet82KqhqE9onHNl9dRAJC/iVHEPHbAOLIRpiH7xqrTkidMSlANuKdPA0o53513QzQTwxoIBTUkjiHTkLJI6AX7W7C/tFVhwbn92kp+nY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=b+8Mxnqb; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="b+8Mxnqb" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57PBu3Vu032397; Mon, 25 Aug 2025 17:12:40 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=wJWMYHMfvbzU5BQsW YZErSvZaRLa/7xMVP6aVFziDXk=; b=b+8MxnqbxnHFXft1I8gfs8FtCKCXmERbp fIipRT/eTQBJHkn9/gj+lV3tF36tiypqM9qOeO7KTQ27BePhSPaohn916DGN0CLJ 953x3H65AkQlCVBISGiRT8q2A08W+9owK1Jo4jT5Ao0CfiYCneMHamla8x+d/3OV 77i0l7gOiGGoe1fqgu1h3MMnjnb8pYBy1oY6yDBB6p5ACSnmg8rp+fiSB4YKhfyT a2m4iGbQTb7pc1RkBWe2iNs0Zn9EFn48bNO1iSZ669YsmbV3A51/XUiF4sLQvXQi aK87uPRYbfp2ZIv9stjPEQSg+4VHct7v1UJj+gQ3tcGhXX78lJ/Aw== Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48q5ava7ka-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 25 Aug 2025 17:12:40 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57PDoxva002492; Mon, 25 Aug 2025 17:12:39 GMT Received: from smtprelay07.dal12v.mail.ibm.com ([172.16.1.9]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 48qrypex8c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 25 Aug 2025 17:12:39 +0000 Received: from smtpav06.wdc07v.mail.ibm.com (smtpav06.wdc07v.mail.ibm.com [10.39.53.233]) by smtprelay07.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57PHCbhO10945346 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 25 Aug 2025 17:12:38 GMT Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B0CD858062; Mon, 25 Aug 2025 17:12:37 +0000 (GMT) Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BA6225804E; Mon, 25 Aug 2025 17:12:36 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.255.253]) by smtpav06.wdc07v.mail.ibm.com (Postfix) with ESMTP; Mon, 25 Aug 2025 17:12:36 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v2 8/9] vfio: Add a reset_done callback for vfio-pci driver Date: Mon, 25 Aug 2025 10:12:25 -0700 Message-ID: <20250825171226.1602-9-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250825171226.1602-1-alifm@linux.ibm.com> References: <20250825171226.1602-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: y4Jlc0_SDS9QSYr3sj42wzE1B7ILMrZV X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODIzMDAyMSBTYWx0ZWRfXxfRH8MAuc+3D 9hiU9mJOWX/EBJNwBL4mWV7EGAZUS/aLqyVsNJver4nGAxEG6ihzhGkstt99hzyY/LrwI23Q/Gb VD1HNmnAWTjVpRYlzRVLU+hG6RtojroHEE5lkk2gYLtOB+rMULSQ/kThAxF+Ei+ga8Oq9LAk8EG rsOdDvo0eHi3u25OcqTcHKr+X2YLNYzPTXKdkF2Uk8SW6sbRlG8NTObmABTtSyu0NJFj+Adh8qp yJSDTkdi13DDNz9incwLNT3dZV5yTGHP4/9vK0BkDXKiIHQqv5ArphngMG7frd8yQSIBewrfrqF gtQ8kXQ5T3byJL2uWiuTpCZwOubmVMgwN4B/Wz1roV4Ba0qt8UC+n+AAc59Bk5GLiPJe4S/bBp9 WwUxG4CS X-Proofpoint-ORIG-GUID: y4Jlc0_SDS9QSYr3sj42wzE1B7ILMrZV X-Authority-Analysis: v=2.4 cv=SNNCVPvH c=1 sm=1 tr=0 ts=68ac9988 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=2OwXVqhp2XgA:10 a=VnNF1IyMAAAA:8 a=hcTHx3Z5Akp3fEzgVBYA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-25_08,2025-08-20_03,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 adultscore=0 bulkscore=0 phishscore=0 clxscore=1015 impostorscore=0 malwarescore=0 suspectscore=0 priorityscore=1501 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2508230021 Content-Type: text/plain; charset="utf-8" On error recovery for a PCI device bound to vfio-pci driver, we want to recover the state of the device to its last known saved state. The callback restores the state of the device to its initial saved state. Signed-off-by: Farhan Ali --- drivers/vfio/pci/vfio_pci_core.c | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index 378adb3226db..f2fcb81b3e69 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -2241,6 +2241,17 @@ pci_ers_result_t vfio_pci_core_aer_err_detected(stru= ct pci_dev *pdev, } EXPORT_SYMBOL_GPL(vfio_pci_core_aer_err_detected); =20 +static void vfio_pci_core_aer_reset_done(struct pci_dev *pdev) +{ + struct vfio_pci_core_device *vdev =3D dev_get_drvdata(&pdev->dev); + + if (!vdev->pci_saved_state) + return; + + pci_load_saved_state(pdev, vdev->pci_saved_state); + pci_restore_state(pdev); +} + int vfio_pci_core_sriov_configure(struct vfio_pci_core_device *vdev, int nr_virtfn) { @@ -2305,6 +2316,7 @@ EXPORT_SYMBOL_GPL(vfio_pci_core_sriov_configure); =20 const struct pci_error_handlers vfio_pci_core_err_handlers =3D { .error_detected =3D vfio_pci_core_aer_err_detected, + .reset_done =3D vfio_pci_core_aer_reset_done, }; EXPORT_SYMBOL_GPL(vfio_pci_core_err_handlers); =20 --=20 2.43.0 From nobody Fri Oct 3 20:25:18 2025 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1A2A6307ACB; Mon, 25 Aug 2025 17:12:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756141967; cv=none; b=kjmza6SEjseg6lHRQL3fsCRhEqXO0YeAaN0/vmulZ+wSQpwE+2Mlhl+5JDj6leD5sfKtHXC+7aL0qR0k/e01IriIQJQOQAnkVDXD+92/jfC81NSxRmTVpcBHkkC4YVQlMqbzd0ub2luNXLPgZ/fTITySxnbhmm5cxdrI/V1xH3U= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1756141967; c=relaxed/simple; bh=HvTt5221H5DQ1X+GZWXY3QkHHnBA6eW/stRkoiAL7gs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=QXTl2bM+KJE/IB8JBQid7SWRxxsykjFHxmysyqtKIWQLsziVP76GHz04tzLz45VoqAZAUKCIaRvMSEOM3goLdMqg5RutQ8EwTH63OWYVjBqDzsQE63BoIDpQx56KeYA2i7JoYWuiOj48hq2TnPmm7CqYyDO5jeCm0NQflLYvrlM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=NY6Le40z; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="NY6Le40z" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 57P7fjkm001667; Mon, 25 Aug 2025 17:12:41 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=+c2kHIW89fFxZMF7i igDqHKPq5NHEczv6HmG2G6s5GY=; b=NY6Le40zeoQN3qdi5QjEONZCPm7Z3qs/L s20NY4CqTOgSb/awXnst6umDuHf5DeDITLJwWRj2i/6wbSGdL3azQqRc09LZAQgD UGshtvi4dks3Icu+bH2sr7F1T6WLUuDNdAa3tX6ucTk6Xxs4gkB7zVOfSh3d+oYB dhphYxr277frQDIL0LRkzz6CysQy13Zu6gHtv/S4PtrhzWGkH1D/wXyJi3GMhsmX YDL0O71NlE38JPGVB/yP+J07MfnCZl9qRA5FDx/YahbMQSb9N3c/zx3YWGuTkfaQ I1MNPAsY952hWeGRU6IJNQgN37V5OIcQxpy305JHr8oO63Uq1pH/w== Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 48q9751jy2-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 25 Aug 2025 17:12:41 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 57PEUHSw029871; Mon, 25 Aug 2025 17:12:40 GMT Received: from smtprelay01.wdc07v.mail.ibm.com ([172.16.1.68]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 48qsfmeufn-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 25 Aug 2025 17:12:40 +0000 Received: from smtpav06.wdc07v.mail.ibm.com (smtpav06.wdc07v.mail.ibm.com [10.39.53.233]) by smtprelay01.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 57PHCcXh34341178 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 25 Aug 2025 17:12:38 GMT Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C7EE258064; Mon, 25 Aug 2025 17:12:38 +0000 (GMT) Received: from smtpav06.wdc07v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id DB13E5804E; Mon, 25 Aug 2025 17:12:37 +0000 (GMT) Received: from IBM-D32RQW3.ibm.com (unknown [9.61.255.253]) by smtpav06.wdc07v.mail.ibm.com (Postfix) with ESMTP; Mon, 25 Aug 2025 17:12:37 +0000 (GMT) From: Farhan Ali To: linux-s390@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Cc: alex.williamson@redhat.com, helgaas@kernel.org, alifm@linux.ibm.com, schnelle@linux.ibm.com, mjrosato@linux.ibm.com Subject: [PATCH v2 9/9] vfio: Remove the pcie check for VFIO_PCI_ERR_IRQ_INDEX Date: Mon, 25 Aug 2025 10:12:26 -0700 Message-ID: <20250825171226.1602-10-alifm@linux.ibm.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20250825171226.1602-1-alifm@linux.ibm.com> References: <20250825171226.1602-1-alifm@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: Ej2z1ELbrgA7KWIzy7Pu80JquLI43Ovk X-Proofpoint-ORIG-GUID: Ej2z1ELbrgA7KWIzy7Pu80JquLI43Ovk X-Proofpoint-Spam-Details-Enc: AW1haW4tMjUwODIzMDA3MSBTYWx0ZWRfXyxriXlNP19MQ FuKRjATPIajLpI3U4VGM8j4GvNRpECwkC+/hDgAwbCm23IYJAjHxOIvmd5YcEITU/aeUb0/MbTr J+PDmMcSpOJnzNu0Dxb7qxhNGGSqVQXPdIPxRSnw7dQm4ZmjsG7Q28f57sTk60GXWdMgPywO43o pne+zM/Qeq9spmKYYjeB9RouMMvsjG1EUt6W1fnyCRGr2gIOhf8rpauUGpfb1d0BotCasWJd1Yq mWW2IK8vyTF/CptG2NQd4EmcHuT4SuXIWgc0rK26i17SRB6pB+37xPvp4LG3CoCp2p0Y1UWeCBh wiKFemspAnc9zDdUwK7WsN4VoKh+76z/xIjYCEXoI0Z8ZqzBTBNHao8kIuhjRrKs2FhP70w+ZWZ K/ndlsbi X-Authority-Analysis: v=2.4 cv=RtDFLDmK c=1 sm=1 tr=0 ts=68ac9989 cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=2OwXVqhp2XgA:10 a=VnNF1IyMAAAA:8 a=gONGJyW3jwFs4LhXXOUA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1099,Hydra:6.1.9,FMLib:17.12.80.40 definitions=2025-08-25_08,2025-08-20_03,2025-03-28_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 suspectscore=0 malwarescore=0 spamscore=0 priorityscore=1501 bulkscore=0 clxscore=1015 phishscore=0 adultscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.19.0-2507300000 definitions=main-2508230071 Content-Type: text/plain; charset="utf-8" We are configuring the error signaling on the vast majority of devices and it's extremely rare that it fires anyway. This allows userspace to be notified on errors for legacy PCI devices. The Internal Share Memory (ISM) device on s390x is one such device. For PCI devices on IBM s390x error recovery involves platform firmware and notification to operating system is done by architecture specific way. So the ISM device can still be recovered when notified of an error. Signed-off-by: Farhan Ali --- drivers/vfio/pci/vfio_pci_core.c | 6 ++---- drivers/vfio/pci/vfio_pci_intrs.c | 3 +-- 2 files changed, 3 insertions(+), 6 deletions(-) diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_c= ore.c index f2fcb81b3e69..d125471fd5ea 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -749,8 +749,7 @@ static int vfio_pci_get_irq_count(struct vfio_pci_core_= device *vdev, int irq_typ return (flags & PCI_MSIX_FLAGS_QSIZE) + 1; } } else if (irq_type =3D=3D VFIO_PCI_ERR_IRQ_INDEX) { - if (pci_is_pcie(vdev->pdev)) - return 1; + return 1; } else if (irq_type =3D=3D VFIO_PCI_REQ_IRQ_INDEX) { return 1; } @@ -1150,8 +1149,7 @@ static int vfio_pci_ioctl_get_irq_info(struct vfio_pc= i_core_device *vdev, case VFIO_PCI_REQ_IRQ_INDEX: break; case VFIO_PCI_ERR_IRQ_INDEX: - if (pci_is_pcie(vdev->pdev)) - break; + break; fallthrough; default: return -EINVAL; diff --git a/drivers/vfio/pci/vfio_pci_intrs.c b/drivers/vfio/pci/vfio_pci_= intrs.c index 123298a4dc8f..f2d13b6eb28f 100644 --- a/drivers/vfio/pci/vfio_pci_intrs.c +++ b/drivers/vfio/pci/vfio_pci_intrs.c @@ -838,8 +838,7 @@ int vfio_pci_set_irqs_ioctl(struct vfio_pci_core_device= *vdev, uint32_t flags, case VFIO_PCI_ERR_IRQ_INDEX: switch (flags & VFIO_IRQ_SET_ACTION_TYPE_MASK) { case VFIO_IRQ_SET_ACTION_TRIGGER: - if (pci_is_pcie(vdev->pdev)) - func =3D vfio_pci_set_err_trigger; + func =3D vfio_pci_set_err_trigger; break; } break; --=20 2.43.0