From nobody Sat Sep 26 03:50:08 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D4A5F524AE6; Fri, 4 Sep 2026 22:35:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788561347; cv=none; b=Mxk9jem3mkw9B6C0OYeXUm4R15xEpsPMNHymAQ27CIZ/9zys3IgiEzVjh+x0C/CqWvGRK4aWhizbh5/EZnuUfDjM+ExW5adMG9fgzvEb8MOSNEAo0HxTV2dZOI2b54CSmxBVKVVSOm6aS6fEEboPjAo/1w0Pq4es2qk+6MRrKUI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788561347; c=relaxed/simple; bh=F9gk5BfzJvG4rH7kL57qJbaWkovs0vno87DSzyDGobA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=ZJhaoVh+0FzakHw/cng7y+dj/7oXHgLVodlUrEHnnBykX5Lda2WO/uotQXaj0+UnPFDF+U5S/UhmyrHzg3jO+qMIfbdkEyKJpUDH0RAicgEY2HOXMZx+kSrwMaNFQkzThNz/dPz6mTItPyNrXFfloojnu60oOvmnBT+mrEn8Vrw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=USNyjYvw; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="USNyjYvw" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 684L1vr62815882; Fri, 4 Sep 2026 22:35:36 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pp1; bh=Lfq6dI B0y0GS7qfsocQ5bwpe9EMZ4Zn7TnSFLlaWX5Y=; b=USNyjYvwVM17x1vFRLdvYz gje1MbFLeiZO6Y3F1WX6QlqVS2LxcX7W28jGuDLWzwzrXfZCQvG/kicKWGJVMtqr Uma2Pui+f+vz8FCVt1NwTDcQzNa1B6MBM29FYYVV3w3glcip1HbTL2fGD1zSnGwr O7gfguTH2hOKdaAQiffZaNsrnPiAaiC72y7IaFuutJl7AWYnsB/pkoGqPpvnkvs2 4j80piZc6iRREEBSclXhjZDaYkvvWYWWvkHvbSUm0IHllSy+uTZknfkCwq3VKox3 FzeibJopXcgHe9FwUiSBvfJLXSkKqz4u25Oh1DH3kPuGbSdLX687QYsfqozwP5+g == Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4gbpx65m3q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 04 Sep 2026 22:35:36 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 684MQKRZ000716; Fri, 4 Sep 2026 22:35:35 GMT Received: from smtprelay05.dal12v.mail.ibm.com ([172.16.1.7]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 4gcceyqdv1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 04 Sep 2026 22:35:35 +0000 (GMT) Received: from smtpav05.dal12v.mail.ibm.com (smtpav05.dal12v.mail.ibm.com [10.241.53.104]) by smtprelay05.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 684MZYsr32703194 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 4 Sep 2026 22:35:34 GMT Received: from smtpav05.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3269458067; Fri, 4 Sep 2026 22:35:34 +0000 (GMT) Received: from smtpav05.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0FADF58056; Fri, 4 Sep 2026 22:35:33 +0000 (GMT) Received: from li-4c4c4544-004d-4810-8043-b7c04f423534.ibm.com.com (unknown [9.61.80.215]) by smtpav05.dal12v.mail.ibm.com (Postfix) with ESMTP; Fri, 4 Sep 2026 22:35:32 +0000 (GMT) From: Anthony Krowiak To: linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jjherne@linux.ibm.com, borntraeger@de.ibm.com, mjrosato@linux.ibm.com, pasic@linux.ibm.com, alex@shazbot.org, kwankhede@nvidia.com, fiuczy@linux.ibm.com, pbonzini@redhat.com, frankja@linux.ibm.com, imbrenda@linux.ibm.com, agordeev@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com, stable@vger.kernel.org Subject: [PATCH v7 1/6] s390/vfio-ap: Fix leaks of pinned NIB and registered GISC Date: Fri, 4 Sep 2026 18:35:26 -0400 Message-ID: <20260904223531.1611088-2-akrowiak@linux.ibm.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260904223531.1611088-1-akrowiak@linux.ibm.com> References: <20260904223531.1611088-1-akrowiak@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=PPc/P/qC c=1 sm=1 tr=0 ts=6a9b47b8 cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=IkcTkHD0fZMA:10 a=VdqzKS8jKosA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=iQ6ETzBq9ecOQQE5vZCe:22 a=VwQbUJbxAAAA:8 a=VnNF1IyMAAAA:8 a=2K4mVM8Jay714jXeKF0A:9 a=3ZKOabzyN94A:10 a=QEXdDO2ut3YA:10 X-Proofpoint-GUID: mvfekuDBo-WpghpRFY0UU_NHT4vsOt5Q X-Proofpoint-ORIG-GUID: mvfekuDBo-WpghpRFY0UU_NHT4vsOt5Q X-Proofpoint-Spam-Info: AW1haW4tMjYwOTA0MDIwOSBTYWx0ZWRfX9AuquvM0CVwz 19fFvDocVNXbBnEtT0/TpYZb9qOKdVdOwiwO6MTrITwE+jkSbFIMIJH98z0LIaBkUAH6shBqCX+ rS1BjH7uRfQYUQtwMT68do9lwRmla6A= X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwOTA0MDIwOSBTYWx0ZWRfX+VB7wOfPJY9s lNzKDveTBoGDTuoogaBFaRoC1J1gxhU5zBAyDmMVQKNaTDpvhv8cE3gwNUeuYa5Y05kCOE1MtmF hJ/9aY+w19tZ5/wLTqqUi6Y81alCjguPX2pyxM3bYIoo98o0QGMGZaJ3zfzpLT8R2gTsr1Ovbug XzrAAsrgk52OpOhotcMbsG+UcbuAQwuqHXOokP06CWBHNUwCa5EhdV0Q2JKra1FBu7j83R6P6AW ZmlRNxganx6sEBMr0BLsGEwgDBxk5EPAiRjahxF1XKnVeI70WSFKmcakpNZ5ZuMGNFH8LxXXeFy nz9zdEgTlD3biL7HrtvMqxE+t8YZ3MK8EWKgDPC7lKzG/QLUd2jYhbeBVbgUMQLhI4NC8X5GzYW VopL9xdlhbxa3HRc3LR6qkBu1pg3/2nQIXJOos+MvO1gkOyAnpx4M0OLfcdMIQ98i6S0jr3E8TF gOFVWGIEtH98OrrJ31w== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1176,Hydra:6.1.134,FMLib:17.12.100.49 definitions=2026-09-04_06,2026-09-03_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 phishscore=0 adultscore=0 suspectscore=0 bulkscore=0 spamscore=0 priorityscore=1501 impostorscore=0 malwarescore=0 lowpriorityscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2606150000 definitions=main-2609040209 Several code paths in the vfio_ap driver failed to free the AQIC resources =E2=80=94 the pinned guest NIB page and the registered guest ISC used to enable interrupts for a queue =E2=80=94 when a queue became unavail= able or when unexpected response codes were returned. This could cause memory exhaustion and depletion of KVM interrupt subclass registrations over time with repeated dynamic AP reconfiguration. vfio_ap_mdev_reset_queue() ~~~~~~~~~~~~~~~~~~~~~~~~~~ AP_RESPONSE_Q_NOT_AVAIL (0x01) was not handled, causing it to fall through to the default case which only fired a WARN without calling vfio_ap_free_aqic_resources(). When ap_zapq() returns this response code the queue is physically unavailable and can no longer generate AP interrupts or DMA-write to the NIB. Add AP_RESPONSE_Q_NOT_AVAIL alongside AP_RESPONSE_DECONFIGURED and AP_RESPONSE_CHECKSTOPPED so that AQIC resources are freed immediately for all three non- operational cases. The default case itself also failed to free AQIC resources. The only valid response codes for PQAP-ZAPQ are 0x00, 0x01, 0x02, 0x03, 0x04, and 0x0a. Any other code indicates a hardware or firmware bug. A malfunctioning queue cannot generate AP interrupts or DMA-write to the NIB, so call vfio_ap_free_aqic_resources() in the default case rather than leaking the resources. vfio_ap_mdev_remove_queue() ~~~~~~~~~~~~~~~~~~~~~~~~~~~ When the AP bus scan detects a queue is no longer in the host AP configuration, vfio_ap_mdev_remove_queue() skips the ZAPQ since issuing it would return AP_RESPONSE_Q_NOT_AVAIL anyway. However, vfio_ap_free_aqic_resources() was also never called, leaking the pinned NIB page and registered guest ISC. Since the hardware is gone and can no longer DMA-write to the NIB, it is safe to call vfio_ap_free_aqic_resources() directly. Add an else branch to the host-config test_bit_inv guard to free AQIC resources when the queue is not in the host AP configuration. If the queue is not assigned to an mdev, vfio_ap_free_aqic_resources() is a no-op. apq_reset_check() ~~~~~~~~~~~~~~~~~ When apq_status_check() returns -EIO - indicating that PQAP(TAPQ) returned an invalid response code - apq_reset_check() returned immediately without freeing the AQIC resources. An invalid TAPQ response code indicates a hardware or firmware bug. A malfunctioning queue cannot generate AP interrupts or DMA-write to the NIB, so call vfio_ap_free_aqic_resources() before returning in the -EIO case. vfio_ap_irq_disable() ~~~~~~~~~~~~~~~~~~~~~ The valid response codes for PQAP(AQIC) include AP_RESPONSE_STATE_CHANGE_IN_PROGRESS (0x0a), AP_RESPONSE_INVALID_GISA (0x08), AP_RESPONSE_ASSOC_SECRET_NOT_UNIQUE (0x35), and AP_RESPONSE_ASSOC_FAILED (0x36), none of which were explicitly handled. All four fell through to the default case and jumped to end_fail without freeing resources. AP_RESPONSE_STATE_CHANGE_IN_PROGRESS indicates a transient condition, analogous to AP_RESPONSE_RESET_IN_PROGRESS and AP_RESPONSE_BUSY. Add it to the retry case. AP_RESPONSE_INVALID_GISA indicates the AQIC instruction was rejected due to an invalid GISA address. The disable did not take effect and the hardware still holds the NIB address. AP_RESPONSE_ASSOC_SECRET_NOT_UNIQUE and AP_RESPONSE_ASSOC_FAILED are asynchronous response codes from a previously executed association instruction. All subsequent AP instructions, including the current AQIC, end with the asynchronous response code until the queue is reset. The AQIC disable was therefore not executed and the hardware still holds the NIB address. For all three of the above, end_fail is the correct disposition since the NIB must not be freed while hardware may still write to it. Add these as explicit cases above default to document the intent. Fixes: ec89b55e3bce7 ("s390: ap: implement PAPQ AQIC interception in kernel= ") Cc: stable@vger.kernel.org Signed-off-by: Anthony Krowiak --- drivers/s390/crypto/vfio_ap_ops.c | 275 +++++++++++++++++++++++------- 1 file changed, 216 insertions(+), 59 deletions(-) diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_a= p_ops.c index 940c0ff668be..9d4f5b9e3301 100644 --- a/drivers/s390/crypto/vfio_ap_ops.c +++ b/drivers/s390/crypto/vfio_ap_ops.c @@ -31,6 +31,7 @@ #define AP_QUEUE_IN_USE "in use" =20 #define AP_RESET_INTERVAL 20 /* Reset sleep interval (20ms) */ +#define AP_RESET_MAX_WAIT 2000 /* Maximum wait for reset (2000ms) */ =20 static int vfio_ap_mdev_reset_queues(struct ap_matrix_mdev *matrix_mdev); static int vfio_ap_mdev_reset_qlist(struct list_head *qlist); @@ -226,16 +227,27 @@ static struct vfio_ap_queue *vfio_ap_mdev_get_queue( } =20 /** - * vfio_ap_wait_for_irqclear - clears the IR bit or gives up after 5 tries - * @apqn: The AP Queue number - * - * Checks the IRQ bit for the status of this APQN using ap_tapq. - * Returns if the ap_tapq function succeeded and the bit is clear. - * Returns if ap_tapq function failed with invalid, deconfigured or - * checkstopped AP. - * Otherwise retries up to 5 times after waiting 20ms. + * vfio_ap_wait_for_irqclear - wait for the IR bit to clear after a disable + * + * @apqn: the APQN of the queue + * + * Repeatedly polls the AP queue status via PQAP(TAPQ) every 20ms until th= e IR + * bit is clear, the queue becomes non-operational, or 5 retries are exhau= sted. + * + * Because PQAP(AQIC) disable initiates an asynchronous process, a + * condition-code 0 completion does not guarantee the IR bit has been clea= red. + * The host must confirm IR=3D0 before unpinning the NIB page to avoid a w= ild + * DMA write to a freed page. + * + * Return: + * - 0 if the IR bit is clear (i.e., interrupts are disabled). + * + * - -ENODEV if the PQAP-TAPQ response code indicates the queue is not ava= ilable, + * is deconfigured, or is checkstopped (i.e., not operational). + * + * - -ETIMEDOUT the function timed out before the IR bit was cleared. */ -static void vfio_ap_wait_for_irqclear(int apqn) +static int vfio_ap_wait_for_irqclear(int apqn) { struct ap_queue_status status; int retry =3D 5; @@ -246,7 +258,7 @@ static void vfio_ap_wait_for_irqclear(int apqn) case AP_RESPONSE_NORMAL: case AP_RESPONSE_RESET_IN_PROGRESS: if (!status.irq_enabled) - return; + return 0; fallthrough; case AP_RESPONSE_BUSY: msleep(20); @@ -257,12 +269,15 @@ static void vfio_ap_wait_for_irqclear(int apqn) default: WARN_ONCE(1, "%s: tapq rc %02x: %04x\n", __func__, status.response_code, apqn); - return; + return -ENODEV; } } while (--retry); =20 - WARN_ONCE(1, "%s: tapq rc %02x: %04x could not clear IR bit\n", - __func__, status.response_code, apqn); + WARN_ONCE(1, "%s: tapq rc %02x: timed out waiting for interrupts disabled= for %02x.%04x\n", + __func__, status.response_code, + AP_QID_CARD(apqn), AP_QID_QUEUE(apqn)); + + return -ETIMEDOUT; } =20 /** @@ -289,20 +304,28 @@ static void vfio_ap_free_aqic_resources(struct vfio_a= p_queue *q) } =20 /** - * vfio_ap_irq_disable - disables and clears an ap_queue interrupt - * @q: The vfio_ap_queue + * vfio_ap_irq_disable - disable interrupts for an AP queue + * @q: the vfio_ap_queue + * + * Issues PQAP(AQIC) to disable interrupts for the AP queue. On success + * (AP_RESPONSE_NORMAL or AP_RESPONSE_OTHERWISE_CHANGED), polls via + * vfio_ap_wait_for_irqclear() until the IR bit is confirmed clear before + * freeing the pinned NIB page and unregistering the guest ISC. This wait = is + * necessary because AQIC disable is asynchronous: freeing the NIB before = IR=3D0 + * is confirmed risks a wild DMA write to a freed host page. * - * Uses ap_aqic to disable the interruption and in case of success, reset - * in progress or IRQ disable command already proceeded: calls - * vfio_ap_wait_for_irqclear() to check for the IRQ bit to be clear - * and calls vfio_ap_free_aqic_resources() to free the resources associated - * with the AP interrupt handling. + * Retries up to 5 times (with 20ms sleep) if the queue is busy or a reset= is + * in progress. * - * In the case the AP is busy, or a reset is in progress, - * retries after 20ms, up to 5 times. + * If the IR bit cannot be confirmed clear (timeout), the NIB page and gue= st + * ISC are intentionally leaked. If the page were unpinned and returned to= the + * allocator, a subsequent hardware DMA write to that physical address wou= ld + * corrupt memory belonging to a new owner =E2=80=94 a wild DMA write that= could crash + * or compromise the host kernel. * - * Returns if ap_aqic function failed with invalid, deconfigured or - * checkstopped AP. + * If the queue is non-operational (deconfigured, checkstopped, not availa= ble), + * resources are freed immediately since the hardware can no longer write = to + * the NIB. * * Return: &struct ap_queue_status */ @@ -310,34 +333,90 @@ static struct ap_queue_status vfio_ap_irq_disable(str= uct vfio_ap_queue *q) { union ap_qirq_ctrl aqic_gisa =3D { .value =3D 0 }; struct ap_queue_status status; - int retries =3D 5; + int retries =3D 5, ret; =20 do { status =3D ap_aqic(q->apqn, aqic_gisa, 0); switch (status.response_code) { case AP_RESPONSE_OTHERWISE_CHANGED: case AP_RESPONSE_NORMAL: - vfio_ap_wait_for_irqclear(q->apqn); - goto end_free; + /* + * AQIC disable was accepted (NORMAL), or the queue was + * already disabled or a prior async request is still + * completing (OTHERWISE_CHANGED). In both cases, we must + * wait until interrupt processing has been disabled + * before proceeding. + */ + ret =3D vfio_ap_wait_for_irqclear(q->apqn); + if (ret =3D=3D 0 || ret =3D=3D -ENODEV) + goto end_free; + /* + * Timed out waiting to confirm interrupts are disabled. + * If ap_aqic returned NORMAL, the guest would incorrectly + * interpret that as a successful disable and may free or + * reuse the NIB while hardware can still write to it. + * Zero the status word and set OTHERWISE_CHANGED to mimic + * what the hardware does for that response code. This + * signals to the guest that the reset operation did not + * complete. + */ + if (status.response_code =3D=3D AP_RESPONSE_NORMAL) { + memset(&status, 0, sizeof(status)); + status.response_code =3D AP_RESPONSE_OTHERWISE_CHANGED; + } + goto end_fail; case AP_RESPONSE_RESET_IN_PROGRESS: case AP_RESPONSE_BUSY: + case AP_RESPONSE_STATE_CHANGE_IN_PROGRESS: msleep(20); break; case AP_RESPONSE_Q_NOT_AVAIL: case AP_RESPONSE_DECONFIGURED: case AP_RESPONSE_CHECKSTOPPED: + /* AP not operational; no further interrupts possible */ + WARN_ONCE(1, "%s: ap_aqic status %d\n", __func__, + status.response_code); + goto end_free; case AP_RESPONSE_INVALID_ADDRESS: + case AP_RESPONSE_INVALID_GISA: + case AP_RESPONSE_ASSOC_SECRET_NOT_UNIQUE: + case AP_RESPONSE_ASSOC_FAILED: default: - /* All cases in default means AP not operational */ + /* + * The AQIC disable was rejected; IRQ is still enabled + * and the hardware still holds the NIB address. Do not + * free resources. + */ WARN_ONCE(1, "%s: ap_aqic status %d\n", __func__, status.response_code); - goto end_free; + goto end_fail; } } while (retries--); =20 WARN_ONCE(1, "%s: ap_aqic status %d\n", __func__, status.response_code); + +end_fail: + /* + * We are here either because the AQIC instruction failed to disable + * interrupts, or because IR=3D0 could not be confirmed. In either case + * the NIB page and guest ISC cannot be freed: hardware may still write + * to the NIB, and unpinning the page would allow it to be reallocated + * to a new owner. A subsequent hardware DMA write to that physical + * address would corrupt the new owner's memory =E2=80=94 a wild DMA writ= e that + * could crash or compromise the host kernel. The resources are + * therefore intentionally leaked. + */ + return status; + end_free: + /* + * This label is reached because the queue was successfully disabled, + * or because the queue is not operational or not available, in which case + * interrupts can not be processed, so free the AQIC resources - the pinn= ed NIB + * page and the registered guest ISC - used to enable interrupts so they = will + * not be leaked. + */ vfio_ap_free_aqic_resources(q); return status; } @@ -401,22 +480,29 @@ static int ensure_nib_shared(unsigned long addr) } =20 /** - * vfio_ap_irq_enable - Enable Interruption for a APQN + * vfio_ap_irq_enable - enable interrupts for an AP queue on behalf of a g= uest * - * @q: the vfio_ap_queue holding AQIC parameters + * @q: the vfio_ap_queue for which interrupts are to be enabled * @isc: the guest ISC to register with the GIB interface - * @vcpu: the vcpu object containing the registers specifying the paramete= rs - * passed to the PQAP(AQIC) instruction. + * @vcpu: the vcpu whose registers contain the PQAP(AQIC) parameters * - * Pin the NIB saved in *q - * Register the guest ISC to GIB interface and retrieve the - * host ISC to issue the host side PQAP/AQIC + * Pins the guest NIB page, registers the guest ISC with the GIB to obtain= a + * host ISC, and reissues PQAP(AQIC) with the translated host-absolute NIB + * address and host ISC on behalf of the guest. * - * status.response_code may be set to AP_RESPONSE_INVALID_ADDRESS in case = the - * vfio_pin_pages or kvm_s390_gisc_register failed. + * The condition code and AP-queue status word returned by PQAP(AQIC) are + * reflected back to the guest as-is. IRQ state verification (polling until + * IR=3D1) is the responsibility of the guest AP bus, not the host. * - * Otherwise return the ap_queue_status returned by the ap_aqic(), - * all retry handling will be done by the guest. + * Resource management is based solely on whether hardware accepted the ne= w NIB: + * - AP_RESPONSE_NORMAL (CC=3D0): hardware accepted the new NIB; the old p= inned + * NIB page and registered guest ISC are freed and the new ones saved. + * - All other responses: hardware did not accept the new NIB; the newly p= inned + * page and registered ISC are freed and the previously saved resources = are + * left intact. + * + * AP_RESPONSE_INVALID_ADDRESS is returned if vfio_pin_pages() or + * kvm_s390_gisc_register() fails before the AQIC instruction is issued. * * Return: &struct ap_queue_status */ @@ -428,11 +514,10 @@ static struct ap_queue_status vfio_ap_irq_enable(stru= ct vfio_ap_queue *q, struct ap_queue_status status =3D {}; struct kvm_s390_gisa *gisa; struct page *h_page; - int nisc; + int nisc, ret; struct kvm *kvm; phys_addr_t h_nib; dma_addr_t nib; - int ret; =20 /* Verify that the notification indicator byte address is valid */ if (vfio_ap_validate_nib(vcpu, &nib)) { @@ -489,24 +574,33 @@ static struct ap_queue_status vfio_ap_irq_enable(stru= ct vfio_ap_queue *q, status =3D ap_aqic(q->apqn, aqic_gisa, h_nib); switch (status.response_code) { case AP_RESPONSE_NORMAL: - /* See if we did clear older IRQ configuration */ + /* + * Hardware accepted the new NIB address (CC=3D0). The old NIB and + * guest ISC are no longer used by hardware and can be freed. + * The new resources are saved for tracking and future teardown. + * + * IRQ state verification (polling until IR=3D1) is the + * responsibility of the guest AP bus, not the host. The + * condition code and status word are reflected back to the + * guest to respond to the PQAP-AQIC instruction. + */ vfio_ap_free_aqic_resources(q); q->saved_iova =3D nib; q->saved_isc =3D isc; break; - case AP_RESPONSE_OTHERWISE_CHANGED: - /* We could not modify IRQ settings: clear new configuration */ + default: + /* + * Hardware did not accept the new NIB (CC=3D3 or error). The + * previously saved NIB and guest ISC remain active and must + * not be freed. Release the newly pinned page and registered + * ISC that were prepared for this (rejected) request. + */ ret =3D kvm_s390_gisc_unregister(kvm, isc); if (ret) VFIO_AP_DBF_WARN("%s: kvm_s390_gisc_unregister: rc=3D%d isc=3D%d, apqn= =3D%#04x\n", __func__, ret, isc, q->apqn); vfio_unpin_pages(&q->matrix_mdev->vdev, nib, 1); break; - default: - pr_warn("%s: apqn %04x: response: %02x\n", __func__, q->apqn, - status.response_code); - vfio_ap_irq_disable(q); - break; } =20 if (status.response_code !=3D AP_RESPONSE_NORMAL) { @@ -635,7 +729,6 @@ static int handle_pqap(struct kvm_vcpu *vcpu) } =20 status =3D vcpu->run->s.regs.gprs[1]; - /* If IR bit(16) is set we enable the interrupt */ if ((status >> (63 - 16)) & 0x01) qstatus =3D vfio_ap_irq_enable(q, status & 0x07, vcpu); @@ -1919,22 +2012,57 @@ static int apq_status_check(int apqn, struct ap_que= ue_status *status) { switch (status->response_code) { case AP_RESPONSE_NORMAL: + /* + * This response code only indicates that the PQAP(ZAPQ) has + * been initiated. The following bit settings in the status + * returned from TAPQ must be verified to confirm that the + * asynchronous portion of the queue zeroization has completed. + */ + if (status->queue_empty && !status->replies_waiting && + !status->irq_enabled && !status->async) + return 0; + + /* Async zeroization still in progress; keep waiting */ + return -EBUSY; + case AP_RESPONSE_DECONFIGURED: case AP_RESPONSE_CHECKSTOPPED: - return 0; + /* + * The queue is non-operational: interrupts are not possible so + * AQIC resources can be safely freed. However, zeroization + * cannot be confirmed because all status bits are zeroed when + * these response codes are returned =E2=80=94 there is no way to + * distinguish a zeroized queue from one that has not been + * zeroized. Return -ENODEV to signal that AQIC resources should + * be freed but that zeroization has not been confirmed. + */ + return -ENODEV; + case AP_RESPONSE_RESET_IN_PROGRESS: - case AP_RESPONSE_BUSY: + /* + * A reset is in progress. It may be the reset we issued or one + * issued prior to ours; either way, once it completes the queue + * will be zeroized, so keep waiting. + */ return -EBUSY; + + case AP_RESPONSE_BUSY: case AP_RESPONSE_ASSOC_SECRET_NOT_UNIQUE: case AP_RESPONSE_ASSOC_FAILED: /* + * AP_RESPONSE_BUSY: + * The queue is busy with something unrelated to a reset and our + * ZAPQ was rejected outright. Re-issue the ZAPQ. + * + * AP_RESPONSE_ASSOC_SECRET_NOT_UNIQUE + * AP_RESPONSE_ASSOC_FAILED: * These asynchronous response codes indicate a PQAP(AAPQ) * instruction to associate a secret with the guest failed. All * subsequent AP instructions will end with the asynchronous - * response code until the AP queue is reset; so, let's return - * a value indicating a reset needs to be performed again. + * response code until the AP queue is reset. Re-issue the ZAPQ. */ return -EAGAIN; + default: WARN(true, "failed to verify reset of queue %02x.%04x: TAPQ rc=3D%u\n", @@ -1959,8 +2087,16 @@ static void apq_reset_check(struct work_struct *rese= t_work) elapsed +=3D AP_RESET_INTERVAL; status =3D ap_tapq(q->apqn, NULL); ret =3D apq_status_check(q->apqn, &status); - if (ret =3D=3D -EIO) + if (ret =3D=3D -EIO) { + /* + * TAPQ returned an invalid response code. This + * indicates a hardware or firmware bug; the queue + * cannot generate AP interrupts or DMA-write to the + * NIB, so free the AQIC resources rather than leak them. + */ + vfio_ap_free_aqic_resources(q); return; + } if (ret =3D=3D -EBUSY) { pr_notice_ratelimited(WAIT_MSG, elapsed, AP_QID_CARD(q->apqn), @@ -1977,8 +2113,12 @@ static void apq_reset_check(struct work_struct *rese= t_work) memcpy(&q->reset_status, &status, sizeof(status)); continue; } - if (q->saved_isc !=3D VFIO_AP_ISC_INVALID) - vfio_ap_free_aqic_resources(q); + /* + * We end up here when the ZAPQ has completed. ZAPQ + * disables interrupts, so if the AQIC resources must be + * freed; otherwise they may be leaked. + */ + vfio_ap_free_aqic_resources(q); break; } } @@ -1995,22 +2135,30 @@ static void vfio_ap_mdev_reset_queue(struct vfio_ap= _queue *q) switch (status.response_code) { case AP_RESPONSE_NORMAL: case AP_RESPONSE_RESET_IN_PROGRESS: - case AP_RESPONSE_BUSY: case AP_RESPONSE_STATE_CHANGE_IN_PROGRESS: /* * Let's verify whether the ZAPQ completed successfully on a work queue. */ queue_work(system_long_wq, &q->reset_work); break; + case AP_RESPONSE_Q_NOT_AVAIL: case AP_RESPONSE_DECONFIGURED: case AP_RESPONSE_CHECKSTOPPED: vfio_ap_free_aqic_resources(q); break; default: + /* + * The architecture defines only the response codes above as + * valid for ZAPQ. Any other response code indicates a hardware + * or firmware bug. Since a malfunctioning queue cannot generate + * AP interrupts or DMA-write to the NIB, free the AQIC resources + * rather than leak them. + */ WARN(true, "PQAP/ZAPQ for %02x.%04x failed with invalid rc=3D%u\n", AP_QID_CARD(q->apqn), AP_QID_QUEUE(q->apqn), status.response_code); + vfio_ap_free_aqic_resources(q); } } =20 @@ -2534,6 +2682,15 @@ void vfio_ap_mdev_remove_queue(struct ap_device *apd= ev) test_bit_inv(apqi, (unsigned long *)matrix_dev->info.aqm)) { vfio_ap_mdev_reset_queue(q); flush_work(&q->reset_work); + } else { + /* + * The queue is no longer in the host's AP configuration. + * The hardware cannot DMA-write to the NIB, so it is safe + * to free the AQIC resources directly without issuing a + * ZAPQ. If the queue is not assigned to an mdev, + * vfio_ap_free_aqic_resources() is a no-op. + */ + vfio_ap_free_aqic_resources(q); } =20 done: --=20 2.53.0 From nobody Sat Sep 26 03:50:08 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7148343B6F5; Fri, 4 Sep 2026 22:35:42 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788561344; cv=none; b=SQA5JBUt5YXvmKrwJ2LOIVWhIrgRZ9FE1T7xAkxUI6kQfIC0ylfFYQDCNC4jmWFRbSfU9VyDY1EtOya64nLBKU3auIfLOyUd+4txaSV6U4TmszIongvng8d4s9PZWW1cvWMTZd/l6B+XTsiWS3MjriRxyIuHg9ubqOC5/SOPnAg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788561344; c=relaxed/simple; bh=5VQ9Wz29+MojqNEjQal9MdmxWOl23AhQhuhOVeSvwYc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=I5l7c4J/YpIOkehaRWvjDDiUWnehB9SYtDN5vN1PyMRe0S8fsNhfEPEi0DOEhQFqq4Lu9Q1PSsB/dhMg5af+xFHszfoUt/MKfZfhqiTRzsQR7D5xwkikEoRpII7UxGwqkR4xA5AUWKeswJMlFaU7Um7IIsHhJyT8UMksEV3S4bI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=abXcK12R; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="abXcK12R" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 684L1ggR2857707; Fri, 4 Sep 2026 22:35:37 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=g7lzDEqLnLVDIee1n Koazw15T0aPMqgdpNwUZNR1R5U=; b=abXcK12Ri8mmB13PD8LthNyztR0dURgNe bWVr5w+ADV4OW4ybtnxHR+TTcwNZrdHnzKlz1vfaveFyt0B0pm2/x4lDsW4qGqpc lQUB4eCCx8f/AOZkFnJrb1PoY3QZrC96C/vd20qxSsPsX5lASVS6FIpWV6gH6zL4 T/gy47qwuuKQYcIEqao7KP6nR9tu2hB14OdrP+yotWlOPPN+ozzpv/waFFjCpUlp dZDbsj9YnfHBOtZZKrW+U+HR7mt7jzwdfn1BncdWYh7JyTEOK//b7fJ4EXVLCaTD JedCtt1BztnSGn40tTGduOYZQ9Fg5q6/V1R+ob0vPKiwiI5kHCIfw== Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4gbq2tvu8c-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 04 Sep 2026 22:35:37 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 684MQMbs017366; Fri, 4 Sep 2026 22:35:36 GMT Received: from smtprelay06.dal12v.mail.ibm.com ([172.16.1.8]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 4gcbygyf2r-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 04 Sep 2026 22:35:36 +0000 (GMT) Received: from smtpav05.dal12v.mail.ibm.com (smtpav05.dal12v.mail.ibm.com [10.241.53.104]) by smtprelay06.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 684MZZxi43581742 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 4 Sep 2026 22:35:35 GMT Received: from smtpav05.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 5DD2A5805D; Fri, 4 Sep 2026 22:35:35 +0000 (GMT) Received: from smtpav05.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4CD8458056; Fri, 4 Sep 2026 22:35:34 +0000 (GMT) Received: from li-4c4c4544-004d-4810-8043-b7c04f423534.ibm.com.com (unknown [9.61.80.215]) by smtpav05.dal12v.mail.ibm.com (Postfix) with ESMTP; Fri, 4 Sep 2026 22:35:34 +0000 (GMT) From: Anthony Krowiak To: linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jjherne@linux.ibm.com, borntraeger@de.ibm.com, mjrosato@linux.ibm.com, pasic@linux.ibm.com, alex@shazbot.org, kwankhede@nvidia.com, fiuczy@linux.ibm.com, pbonzini@redhat.com, frankja@linux.ibm.com, imbrenda@linux.ibm.com, agordeev@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com, stable@vger.kernel.org Subject: [PATCH v7 2/6] s390/vfio-ap: Fix failure to release IRQ notification eventfd contexts Date: Fri, 4 Sep 2026 18:35:27 -0400 Message-ID: <20260904223531.1611088-3-akrowiak@linux.ibm.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260904223531.1611088-1-akrowiak@linux.ibm.com> References: <20260904223531.1611088-1-akrowiak@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=bc1bluPB c=1 sm=1 tr=0 ts=6a9b47b9 cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=VdqzKS8jKosA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=RzCfie-kr_QcCd8fBx8p:22 a=VwQbUJbxAAAA:8 a=VnNF1IyMAAAA:8 a=fxeDLeT5AZ2qCvnLJ4gA:9 X-Proofpoint-Spam-Info: AW1haW4tMjYwOTA0MDIwOSBTYWx0ZWRfXwIlEtL4pDisu Dk7K8FPKufoRHadQLX7fQY6vIsJFB7dkoNsRXtkqXZWDJlMHQWp0CN+qKaXf7TkbbE50TBJd6UO ZGRWok+r7t0426DNIJOf57RRyU9ba8E= X-Proofpoint-ORIG-GUID: hRacaFsVQ5O319hKisxxYOixpx2Pwfue X-Proofpoint-GUID: hRacaFsVQ5O319hKisxxYOixpx2Pwfue X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwOTA0MDIwOSBTYWx0ZWRfX8zdv8M1SiUPz RgKoDXyTQ3be25/ul5CTskE/yjEbjJZVst8jTlRkCO2oLlThYJamhaVjhtPHsaSAIqFJ1SIHedN 9OnKimh3GuEr+faQxxzdYotSvpXq4daHJZ7SqPKgZBWLkFSqoZEHyE1cvpVb9da92h49IJQfq2e acGyLLTDuGus5DCP37DHslpX1vb4VAAe0q3Yi61No+xefIBvh6T91R6PZTdsKtOq15uaAywAHwo DkYnvqFHeTPHysmxjqL1AF110svmpePZlz5e04I8D8zZDqo0gci7MSUhBlVLXNkIKWcEXgHFmDw BSaD2iTYikYb2jIO4FS8vW1CWiFpz+ZwtvwVQ4QjQfNcH1IheraQlqUntxU4XgRjk171LiePsfh 9XtW4uP0Dl2BRv5bM/ySyUldoADU9IT4Vnn+aRLTbDb7pIfZeFrGq9dHkC5Yua5u+y5w3C3OL41 lA6VrExtduZGtX22YAA== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1176,Hydra:6.1.134,FMLib:17.12.100.49 definitions=2026-09-04_06,2026-09-03_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 suspectscore=0 adultscore=0 malwarescore=0 spamscore=0 lowpriorityscore=0 phishscore=0 clxscore=1015 priorityscore=1501 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2606150000 definitions=main-2609040209 Content-Type: text/plain; charset="utf-8" When userspace registers IRQ notification eventfds via the VFIO_DEVICE_SET_IRQS ioctl, vfio_ap_set_request_irq() and vfio_ap_set_cfg_change_irq() each call eventfd_ctx_fdget(), which takes a reference on the eventfd_ctx and stores it in matrix_mdev->req_trigger and matrix_mdev->cfg_chg_trigger respectively. These references are dropped only when userspace explicitly replaces or clears them via a subsequent SET_IRQS call. If the device is closed without that explicit teardown - because the guest exits, the VM process crashes, or the device file is simply closed - neither vfio_ap_mdev_close_device() nor the remove path releases these references. The eventfd_ctx backing objects and their associated file references therefore leak for the lifetime of the kernel. Fix this by introducing vfio_ap_mdev_release_eventfds() and calling it from vfio_ap_mdev_close_device() after vfio_ap_mdev_unset_kvm(). The VFIO core guarantees that close_device is called before vfio_unregister_group_dev() returns in the remove path, so fixing close_device is sufficient to cover both teardown paths. Note: ~~~~ The matrix_dev->mdevs lock must be held during the call to vfio_ap_mdev_release_eventfds(). There is a small window between the calls to vfio_ap_mdev_unset_kvm() which gets and releases the update locks and the acquisition of the matrix_dev->mdevs_lock mutex during which it is possible - although highly unlikely during normal operation - whereby a concurrent SET_IRQS call can get in. Taking matrix_dev->mdevs_lock around vfio_ap_mdev_release_eventfds() is sufficient to make this race-free. The SET_IRQS ioctl path writes req_trigger and cfg_chg_trigger only from vfio_ap_mdev_ioctl(), which holds mdevs_lock for its entire duration and always calls eventfd_ctx_put() on the previous value before storing the new one. Any number of concurrent SET_IRQS calls during the window between vfio_ap_mdev_unset_kvm() and the acquisition of mdevs_lock are therefore safe: each ioctl invocation puts the reference it found and installs a new one, leaving exactly one live reference in the field when it releases the lock. When release_eventfds subsequently acquires mdevs_lock it finds that single surviving reference and puts it. Conversely, a SET_IRQS call that loses the race and blocks on mdevs_lock will find the field NULL after release_eventfds finishes, take ownership of the reference it just created, and install it into a field that will never be read again - a transient leak. To close that final case, callers must ensure no new SET_IRQS ioctls can be issued after close_device() is called, which the VFIO core guarantees by releasing the device file before invoking close_device(). Fixes: bf48961f6f48e ("s390/vfio-ap: realize the VFIO_DEVICE_SET_IRQS ioctl= ") Cc: stable@vger.kernel.org Signed-off-by: Anthony Krowiak Reviewed-by: Matthew Rosato --- drivers/s390/crypto/vfio_ap_ops.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_a= p_ops.c index 9d4f5b9e3301..4a31454d725a 100644 --- a/drivers/s390/crypto/vfio_ap_ops.c +++ b/drivers/s390/crypto/vfio_ap_ops.c @@ -2209,12 +2209,28 @@ static int vfio_ap_mdev_open_device(struct vfio_dev= ice *vdev) return vfio_ap_mdev_set_kvm(matrix_mdev, vdev->kvm); } =20 +static void vfio_ap_mdev_release_eventfds(struct ap_matrix_mdev *matrix_md= ev) +{ + if (matrix_mdev->req_trigger) { + eventfd_ctx_put(matrix_mdev->req_trigger); + matrix_mdev->req_trigger =3D NULL; + } + if (matrix_mdev->cfg_chg_trigger) { + eventfd_ctx_put(matrix_mdev->cfg_chg_trigger); + matrix_mdev->cfg_chg_trigger =3D NULL; + } +} + static void vfio_ap_mdev_close_device(struct vfio_device *vdev) { struct ap_matrix_mdev *matrix_mdev =3D container_of(vdev, struct ap_matrix_mdev, vdev); =20 vfio_ap_mdev_unset_kvm(matrix_mdev); + + mutex_lock(&matrix_dev->mdevs_lock); + vfio_ap_mdev_release_eventfds(matrix_mdev); + mutex_unlock(&matrix_dev->mdevs_lock); } =20 static void vfio_ap_mdev_request(struct vfio_device *vdev, unsigned int co= unt) --=20 2.53.0 From nobody Sat Sep 26 03:50:08 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D49674F4740; Fri, 4 Sep 2026 22:35:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788561346; cv=none; b=ZfaYRSZGvb+5FzH+WgFXA0vzO6dGnA6EgK5jh655Keq4EKSnLNxYTjD/sf55FXZmj98HkgY2Tda0nwcO+4p0+VMC8nylqlj6CPCusolaRAuZwXBhVuzJFuCseLJtoOOIB2wG/y57pyo1gQuLhrA4lGg4B+WRaxQdEhekWdhPba0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788561346; c=relaxed/simple; bh=kURVJ5KEDyY5YH97k04lXU1lI5EnPeB52R2y/YWARo4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=p1Hs4JZXx7Q94N887/YodTljGtbxv/pZ/PkPpwuAFn6Lm/HJnNgX3YCmQ0kuo0qT/n2J0SJiA4PA0lZVjhHLeoI1aHgV2a3NoGZRjq9qv1cv/b9x4lx/7V8vGWQRoFXyTn7L21RP4Jo0t89n9ULOt470RkwWLKvSHNNSeLKgeiY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=A1WawDUm; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="A1WawDUm" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 684L2NiT2816748; Fri, 4 Sep 2026 22:35:39 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=TCMdEJXD1RUsYRptE uqZB7umVX95OPbgqOH3GpkS63U=; b=A1WawDUm0mio/XEWHGLPZc0sA+5Njxt65 +etk7t21OqwUQuwYGW5C/OPrOQBYibV/1hJOgrxIo7soexwbhHEJrCSwLVRahXde AfEmAhIRgaNR+skORzRvvEoh1XBnutj8m0UJuMr+DQje2ycnPXprNXQEOvbefvjs LOQR+xho1H/v4Hcbej5SmA8s2R5Ml8cVmZ3Ui6ds9whuyy9koRLl0zFtY7yJ8bc6 HBP9Anf4AnJLAjskwx+inXA+a4PUTgpAlV/I+b6FCNgRWs1PMN1iCd17Ctxs4eJC hlts7Aj6XvSYcaSQr71jAiMqoQhmbn3rwytSlInROM5v6nGXFeLlQ== Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4gbpx65m3t-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 04 Sep 2026 22:35:39 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 684MQRiI011802; Fri, 4 Sep 2026 22:35:38 GMT Received: from smtprelay03.dal12v.mail.ibm.com ([172.16.1.5]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4gecjb009q-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 04 Sep 2026 22:35:37 +0000 (GMT) Received: from smtpav05.dal12v.mail.ibm.com (smtpav05.dal12v.mail.ibm.com [10.241.53.104]) by smtprelay03.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 684MZaHA32506442 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 4 Sep 2026 22:35:36 GMT Received: from smtpav05.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 96C275805D; Fri, 4 Sep 2026 22:35:36 +0000 (GMT) Received: from smtpav05.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 786BA58056; Fri, 4 Sep 2026 22:35:35 +0000 (GMT) Received: from li-4c4c4544-004d-4810-8043-b7c04f423534.ibm.com.com (unknown [9.61.80.215]) by smtpav05.dal12v.mail.ibm.com (Postfix) with ESMTP; Fri, 4 Sep 2026 22:35:35 +0000 (GMT) From: Anthony Krowiak To: linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jjherne@linux.ibm.com, borntraeger@de.ibm.com, mjrosato@linux.ibm.com, pasic@linux.ibm.com, alex@shazbot.org, kwankhede@nvidia.com, fiuczy@linux.ibm.com, pbonzini@redhat.com, frankja@linux.ibm.com, imbrenda@linux.ibm.com, agordeev@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com, stable@vger.kernel.org Subject: [PATCH v7 3/6] s390/vfio-ap: Fix unbounded loop in apq_reset_check() Date: Fri, 4 Sep 2026 18:35:28 -0400 Message-ID: <20260904223531.1611088-4-akrowiak@linux.ibm.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260904223531.1611088-1-akrowiak@linux.ibm.com> References: <20260904223531.1611088-1-akrowiak@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=PPc/P/qC c=1 sm=1 tr=0 ts=6a9b47bb cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=VdqzKS8jKosA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=iQ6ETzBq9ecOQQE5vZCe:22 a=VwQbUJbxAAAA:8 a=VnNF1IyMAAAA:8 a=759lh7Qq2uJCZsb0SykA:9 X-Proofpoint-GUID: 0jU_mQXiAPZvDNJEugyUMJJkkkSFMj58 X-Proofpoint-ORIG-GUID: 0jU_mQXiAPZvDNJEugyUMJJkkkSFMj58 X-Proofpoint-Spam-Info: AW1haW4tMjYwOTA0MDIwOSBTYWx0ZWRfXz0LMHrDejxeF 91bTB50Pj/AwDMthbPlTcnvaNMPPyucahoAtrCPFMTLvd8uF7Tonct4/5JF/HYxqULyccQmRVcE 9erarazROv01niEoWjNGULbq1IU7zd0= X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwOTA0MDIwOSBTYWx0ZWRfX1D37CMuJ58Jz onKBJxtQOUnK/FkgIRHsxlflvJ+iGb7ESa5rEWAPIaEoSAADYUboDS3spK8E/3bpLBDDthQ+g39 5EpHHl7qKga5c4NTNAgjrd2e8UFVouhqHc87DiQM2ewTbS5cChyZKcjhjEg61xhPHuyUunXF7uZ jkabJUIt5Ra1bl83y97XSVU4xMOtuSGsC2RNuBVOn562A8aUu/KT8TzdK+FuaN6l1h1qkFk/uTO NNuUQ5VZ539WiEQ9x6b0nuSCH/7vtGIrofm3SsScPiRizdEcR2D1SI1zS8PWb9Hl40Cdb7H8oo8 vQpmbKMf7AJye0YS6jt9sr/651kD4QvSygJ71hwJR4oSG/2Ifqd1aaYPAtzvy5OUYxS1vRYqbNu qZzviiv1rZkyjCiacj6IVpLRBgW4VMH3WNwjYZFfBC6A5J513uQQjtY5Eeh4lXGd5D/rSb+sknq 3CTmYipnIXXCMLEqsCg== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1176,Hydra:6.1.134,FMLib:17.12.100.49 definitions=2026-09-04_06,2026-09-03_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 phishscore=0 adultscore=0 suspectscore=0 bulkscore=0 spamscore=0 priorityscore=1501 impostorscore=0 malwarescore=0 lowpriorityscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2606150000 definitions=main-2609040209 Content-Type: text/plain; charset="utf-8" s390/vfio-ap: Fix unbounded loop in apq_reset_check() The apq_reset_check() worker polls ap_tapq() in a while(true) loop waiting for a queue reset to complete. When ap_tapq() returns AP_RESPONSE_BUSY or AP_RESPONSE_RESET_IN_PROGRESS, apq_status_check() returns -EBUSY and the loop continues after sleeping AP_RESET_INTERVAL (20ms). There is no upper bound on how many times the loop iterates, so if the hardware continuously returns a busy response the worker runs indefinitely. This is particularly harmful because several callers of vfio_ap_mdev_reset_queue() - such as vfio_ap_mdev_reset_queues(), vfio_ap_mdev_reset_qlist() and vfio_ap_mdev_remove_queue() - call flush_work() on the queue's reset_work while holding one or more of the global matrix_dev locks (guests_lock, mdevs_lock) or the KVM lock. An indefinitely spinning worker permanently blocks access to all ap_matrix_mdev objects, which could hang other guests that are using them. Fix this by introducing AP_RESET_MAX_WAIT (2000ms) and breaking out of the poll loop when elapsed time reaches that threshold. If apq_reset_check() times out before verifying completion of the reset, the AQIC resources associated with the queue cannot be freed. The NIB is the active DMA target for AP interrupt delivery until the reset completes; freeing the pinned page would allow it to be reallocated to a new owner. A subsequent hardware DMA write to that physical address would corrupt the new owner's memory - a wild DMA write that could crash or compromise the host kernel. If the reset eventually completes, interrupts will be terminated, but the pinned NIB page and ISC registration will be leaked. This is preferable to a compromised kernel or kernel crash, or waiting indefinitely and blocking access to all mdevs, hanging the guests to which they are attached. On timeout, q->reset_status.response_code is set to AP_RESPONSE_RESET_IN_PROGRESS. This is used internally to signal that the reset did not complete, and ensures that if the queue is reset again, the re-issue logic in apq_reset_check() will re-issue the ZAPQ. This patch also fixes a bug whereby AP_RESPONSE_NORMAL (0) returned from PQAP(ZAPQ) was incorrectly treated as confirmation that the queue was zeroized. AP_RESPONSE_NORMAL only indicates that the ZAPQ was accepted; zeroization is performed asynchronously. To confirm completion, the following bits in the status word returned from PQAP(TAPQ) must all be verified: status->irq_enabled =3D=3D 0 status->queue_empty =3D=3D 1 status->replies_waiting =3D=3D 0 status->async =3D=3D 0 Fixes: dd174833e44e ("s390/vfio-ap: remove upper limit on wait for queue re= set to complete") Cc: stable@vger.kernel.org Signed-off-by: Anthony Krowiak --- drivers/s390/crypto/vfio_ap_ops.c | 91 ++++++++++++++++++++++++++++--- 1 file changed, 84 insertions(+), 7 deletions(-) diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_a= p_ops.c index 4a31454d725a..f9f35265b84f 100644 --- a/drivers/s390/crypto/vfio_ap_ops.c +++ b/drivers/s390/crypto/vfio_ap_ops.c @@ -2047,6 +2047,12 @@ static int apq_status_check(int apqn, struct ap_queu= e_status *status) return -EBUSY; =20 case AP_RESPONSE_BUSY: + /* + * The queue is busy with something unrelated to a reset and our + * ZAPQ was rejected outright. Re-issue the ZAPQ. + */ + return -EAGAIN; + case AP_RESPONSE_ASSOC_SECRET_NOT_UNIQUE: case AP_RESPONSE_ASSOC_FAILED: /* @@ -2072,6 +2078,33 @@ static int apq_status_check(int apqn, struct ap_queu= e_status *status) } } =20 +static void report_aqic_resource_leak(struct vfio_ap_queue *q) +{ + if (q->saved_isc !=3D VFIO_AP_ISC_INVALID || q->saved_iova) { + if (q->matrix_mdev) { + dev_warn_ratelimited(mdev_dev(q->matrix_mdev->mdev), + "Reset timed out for APQN %02x.%04x: leaking AQIC resources (NIB= page & GISC) to prevent host crash\n", + AP_QID_CARD(q->apqn), + AP_QID_QUEUE(q->apqn)); + } else { + pr_warn_ratelimited("Reset timed out for APQN %02x.%04x: leaking AQIC r= esources (NIB page & GISC) to prevent host crash\n", + AP_QID_CARD(q->apqn), + AP_QID_QUEUE(q->apqn)); + } + } else { + if (q->matrix_mdev) { + dev_warn_ratelimited(mdev_dev(q->matrix_mdev->mdev), + "Reset timed out for APQN %02x.%04x\n", + AP_QID_CARD(q->apqn), + AP_QID_QUEUE(q->apqn)); + } else { + pr_warn_ratelimited("Reset timed out for APQN %02x.%04x\n", + AP_QID_CARD(q->apqn), + AP_QID_QUEUE(q->apqn)); + } + } +} + #define WAIT_MSG "Waited %dms for reset of queue %02x.%04x (%u, %u, %u)" =20 static void apq_reset_check(struct work_struct *reset_work) @@ -2096,6 +2129,53 @@ static void apq_reset_check(struct work_struct *rese= t_work) */ vfio_ap_free_aqic_resources(q); return; + + if (elapsed >=3D AP_RESET_MAX_WAIT) { + /* + * Zeroization confirmed (ret =3D=3D 0): the TAPQ status bits + * indicate the async portion of the ZAPQ completed + * successfully. Free AQIC resources and return. + * + * Queue non-operational (ret =3D=3D -ENODEV): the queue is + * deconfigured or checkstopped; interrupts are not + * possible so AQIC resources can be safely freed. + * Zeroization cannot be confirmed in this state, but the + * queue cannot generate interrupts, so the NIB page is + * no longer a DMA target and it is safe to free it. + */ + if (!ret || ret =3D=3D -ENODEV) + goto done; + /* + * Timed out without being able to verify zapq completed. + * + * The AQIC resources associated with this queue - the pinned + * page containing the NIB and the registered guest ISC - + * cannot be freed here. The NIB is the active DMA target + * for AP interrupt delivery until the reset completes; + * freeing the pinned page while the hardware may still + * write to it would result in a wild DMA write that could + * corrupt host memory. + * + * If the reset eventually completes, interrupts will be + * terminated and the pinned NIB page and ISC registration + * will be leaked. This is preferable to either a wild DMA + * write or waiting indefinitely: flush_work() callers hold + * the matrix_dev->mdevs_lock mutex which serializes access + * to all mdev objects system-wide, so blocking here would + * hang all guests to which those mdevs are attached. + */ + report_aqic_resource_leak(q); + /* + * Zeroization could not be confirmed; set + * reset_status to AP_RESPONSE_RESET_IN_PROGRESS. + * This is used internally to signal that the reset + * did not complete, and ensures that if the queue + * is reset again, the re-issue logic in + * apq_reset_check() will re-issue the ZAPQ. + */ + q->reset_status.response_code =3D AP_RESPONSE_RESET_IN_PROGRESS; + + return; } if (ret =3D=3D -EBUSY) { pr_notice_ratelimited(WAIT_MSG, elapsed, @@ -2113,15 +2193,12 @@ static void apq_reset_check(struct work_struct *res= et_work) memcpy(&q->reset_status, &status, sizeof(status)); continue; } - /* - * We end up here when the ZAPQ has completed. ZAPQ - * disables interrupts, so if the AQIC resources must be - * freed; otherwise they may be leaked. - */ - vfio_ap_free_aqic_resources(q); - break; } } + +done: + if (q->saved_isc !=3D VFIO_AP_ISC_INVALID) + vfio_ap_free_aqic_resources(q); } =20 static void vfio_ap_mdev_reset_queue(struct vfio_ap_queue *q) --=20 2.53.0 From nobody Sat Sep 26 03:50:08 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BB2D252BE5D; Fri, 4 Sep 2026 22:35:46 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788561348; cv=none; b=h42yUFkQWS/HpyH1ZugJ+7BuCihN4zh9yt1B1l6BkwURxgLb0ZdtIiAiJS+Zf3wDcFPCupRZaQBTT3jf79FmB4z+VAly8jzxyZAf2NmV6Z5qL73qfLUMo7bsY3wnNE7PdZWnzKawwm+2vw+TodBV4KF79mGbbmqCsIsaXA4k4xc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788561348; c=relaxed/simple; bh=Pq5tIgbE8U9N+lb68/T1GoUSxmKiFGqYKP9oa9lNakY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lPIYbEUqgYkAwTA/hoTRZ0Pt0Ubnw3fxU13a+ePv3HvmajAn93AKA+KXkyxLOfL0JHrltuqABdZKkWqNcJyQwyimh6tnQtlkGBn/Vduah7b040Vbc0HMhhFsMPThO+KN5fZ/RGjSHRejwHvgAPVIXzTAegNZ55jvATnmKviuIwc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=S3csYYbQ; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="S3csYYbQ" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 684L1dJo2857489; Fri, 4 Sep 2026 22:35:40 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=5OP1YRGVIwlfMSzqY P0/PwIKv3UjBFBzE0gocjFnpIU=; b=S3csYYbQ/bDfnUggG8q5T7GvHHTB+iZPs VmwwWR2LOjNdyjVdcsyUSzdYOSu99svXvCNaPVTibIZIyu5jF6EAc/duKLvc1WVa LchGZ0oljzC69wwg2xnFwY26vq0QKzkcpu8/cM6otVFISzh97VAQ/7dWMqYmK7qQ Gkj3000ri8Ft43B9ueh37kuaaK6tXjYj1IB58XI/Q4dWCkPYFAs406IPLTGTc2r9 3wan7eYg9xPaGelr7Ci2FAhEk5wFxBfVXFVK7HB/DZLXn1ff8bVUCccxS7M9R7MC +oPGJLxrsgfVldGR5T7xr3A2eoDUyE58gPqQFcg6I5lIj/wBkzf4A== Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4gbq2tvu8g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 04 Sep 2026 22:35:40 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 684MQJjq026196; Fri, 4 Sep 2026 22:35:39 GMT Received: from smtprelay04.dal12v.mail.ibm.com ([172.16.1.6]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4gcarkqm4k-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 04 Sep 2026 22:35:38 +0000 (GMT) Received: from smtpav05.dal12v.mail.ibm.com (smtpav05.dal12v.mail.ibm.com [10.241.53.104]) by smtprelay04.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 684MZbWR19530372 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 4 Sep 2026 22:35:37 GMT Received: from smtpav05.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BE7E25805D; Fri, 4 Sep 2026 22:35:37 +0000 (GMT) Received: from smtpav05.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B348858056; Fri, 4 Sep 2026 22:35:36 +0000 (GMT) Received: from li-4c4c4544-004d-4810-8043-b7c04f423534.ibm.com.com (unknown [9.61.80.215]) by smtpav05.dal12v.mail.ibm.com (Postfix) with ESMTP; Fri, 4 Sep 2026 22:35:36 +0000 (GMT) From: Anthony Krowiak To: linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jjherne@linux.ibm.com, borntraeger@de.ibm.com, mjrosato@linux.ibm.com, pasic@linux.ibm.com, alex@shazbot.org, kwankhede@nvidia.com, fiuczy@linux.ibm.com, pbonzini@redhat.com, frankja@linux.ibm.com, imbrenda@linux.ibm.com, agordeev@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com Subject: [PATCH v7 4/6] s390/vfio-ap: Use AP_DOMAINS for adm_add bitmap size in vfio_ap_mdev_cfg_add() Date: Fri, 4 Sep 2026 18:35:29 -0400 Message-ID: <20260904223531.1611088-5-akrowiak@linux.ibm.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260904223531.1611088-1-akrowiak@linux.ibm.com> References: <20260904223531.1611088-1-akrowiak@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=bc1bluPB c=1 sm=1 tr=0 ts=6a9b47bc cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=VdqzKS8jKosA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=RzCfie-kr_QcCd8fBx8p:22 a=VnNF1IyMAAAA:8 a=gReD5cTbzgwmqpVxveEA:9 X-Proofpoint-Spam-Info: AW1haW4tMjYwOTA0MDIwOSBTYWx0ZWRfX8lwjvvVIs+3e M0DdqFS50v1HIpDzCY8RTBAPM3Hy/A/CBcnOZ6iGdwCNR3kWnbwUWnzMbnTPXer1f34gZaHSvB7 dLysAut2dXTvHKKe65bwfNhH1p2NB6o= X-Proofpoint-ORIG-GUID: 8XmXtRgpN5YMen4TGIwTq8LmcX1S76eW X-Proofpoint-GUID: 8XmXtRgpN5YMen4TGIwTq8LmcX1S76eW X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwOTA0MDIwOSBTYWx0ZWRfX55YjA+w+WvIP YFd1ZKe4BacJb6iQCEBKNs/Iaop1csjGiz6oYsFiQOWflLlWZx3N/0ds9dP26tkpyHsqlt13Iqw jP/+Yx5Na4YQS4p9oWqizvo74+TShMWuldK1Vr0C1ig1fAvXiQV+TswBohBSY0km/VAeMgLk31A nllpfeHo3Y+nsdAsR0BMQ3V2CwftH8pH2gQZwrLn41xMqKbBe6uO/tEtAMFCNYTY257XBFDz70G ZF7Dy4LBLOn19u0FclzqZEB9O03n4L1/ThUKHTnDFzTpaDvY20RZ4L3Tn8CzhChzl0dTEkFZ+VQ M3JAExj+sfl0BkrTn32hA5I5SHjMDSmE08gNsbkigpP5nK/fj7ZGkMLUBYsHyXST2EeVBT5xYKh 1lcjFUQ7TRdQuKjVjquybVl6tS610vtyEdxNHLzz1u75HpaqW1Rx4774bxvFrOieZzeqwJ8dsMi DIVZjDP//V4qM0v9BEQ== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1176,Hydra:6.1.134,FMLib:17.12.100.49 definitions=2026-09-04_06,2026-09-03_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 suspectscore=0 adultscore=0 malwarescore=0 spamscore=0 lowpriorityscore=0 phishscore=0 clxscore=1015 priorityscore=1501 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2606150000 definitions=main-2609040209 Content-Type: text/plain; charset="utf-8" Domain and control domain bitmaps are sized by the AP_DOMAINS constant, not AP_DEVICES. The two constants are both 256 today so there is no functional impact, but using the wrong constant is inconsistent with every operation on aqm/adm bitmaps. Use AP_DOMAINS to keep the code consistent and correct in case the two constants ever diverge. Note: This patch was submitted in response to a sashiko review comment pointing out there are other functions besides vfio_ap_mdev_cfg_add(), so there are fixes included here for those also. The subject line was kept the same since this is in v2 of this patch. Signed-off-by: Anthony Krowiak Reviewed-by: Jason J. Herne Reviewed-by: Matthew Rosato --- drivers/s390/crypto/vfio_ap_ops.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_a= p_ops.c index f9f35265b84f..672fae69739a 100644 --- a/drivers/s390/crypto/vfio_ap_ops.c +++ b/drivers/s390/crypto/vfio_ap_ops.c @@ -1503,7 +1503,7 @@ static void vfio_ap_mdev_hot_unplug_domain(struct ap_= matrix_mdev *matrix_mdev, { DECLARE_BITMAP(apqis, AP_DOMAINS); =20 - bitmap_zero(apqis, AP_DEVICES); + bitmap_zero(apqis, AP_DOMAINS); set_bit_inv(apqi, apqis); vfio_ap_mdev_hot_unplug_domains(matrix_mdev, apqis); } @@ -2947,11 +2947,11 @@ static void vfio_ap_mdev_on_cfg_remove(struct ap_co= nfig_info *cur_config_info, do_remove |=3D bitmap_andnot(aqrem, (unsigned long *)prev_config_info->aqm, (unsigned long *)cur_config_info->aqm, - AP_DEVICES); + AP_DOMAINS); do_remove |=3D bitmap_andnot(cdrem, (unsigned long *)prev_config_info->adm, (unsigned long *)cur_config_info->adm, - AP_DEVICES); + AP_DOMAINS); =20 if (do_remove) vfio_ap_mdev_cfg_remove(aprem, aqrem, cdrem); @@ -3062,7 +3062,7 @@ static void vfio_ap_mdev_cfg_add(unsigned long *apm_a= dd, unsigned long *aqm_add, bitmap_and(matrix_mdev->aqm_add, matrix_mdev->matrix.aqm, aqm_add, AP_DOMAINS); bitmap_and(matrix_mdev->adm_add, - matrix_mdev->matrix.adm, adm_add, AP_DEVICES); + matrix_mdev->matrix.adm, adm_add, AP_DOMAINS); =20 mutex_unlock(&matrix_dev->mdevs_lock); } --=20 2.53.0 From nobody Sat Sep 26 03:50:08 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BDFA55111B9; Fri, 4 Sep 2026 22:35:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788561349; cv=none; b=U7jq+Qgv0bkhtkOLhqjGwwRKRlJd/+Ov5K0ZWd3DxP/10cwOFbGcZ4oCr+b9n9Pn9sCtvD7tsLmvUxkfGj4hX9ule2U6ezptDemXfvc2ZRvElwjGPhtBCAnJ+dG0aK2MyKwUyKz6O6wHHq1ky1LHvBUQY4A6eAWY0v8d+XvIpBM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788561349; c=relaxed/simple; bh=BF2Xx4GiACU0nziEVVnvj6aLlm7UtavSM+domIxHv7Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=G1keYAbS8P/E+hdgp+W59fL+DXZ2Ys6A5pc8mguoI/iW+QKbxIoQK7fdyr2JX6rNr0EUSkPjKK0J/jHRtmMVDHR0RlN91qIbXnE/KeTxD5RvRqd7fdmmyFNcZyWR5R5pDObEkZBd7AIwEBBoX77f4d8X6mdFxCtCO29AkhJ8mUo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=NZl8b6uW; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="NZl8b6uW" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 684L1aGQ2815577; Fri, 4 Sep 2026 22:35:42 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=uDUF4yo28g5CCD87d 3y3cHN5gViuqCxCGofwtTuqsJE=; b=NZl8b6uWK7u221dJU8nLxE+LgJ0AiDg9r 8dUf1pyZ4bHrNluV8qkQjW/Oqjpl2vIt8j5IMW/wLF01HpVP85r2aqkyvzP0vEla diU+Fhkk9SuERHa4QjZuZUciCftU2FQQyiKrHjqixf6bmJPXs5YGRihqCaSHOYhD rkruzxmI1/9ZdboXF7KMnbyPxb+JLKIL7tHi1VjpzdaIICyIzhZ5iyycpoLks1HY b9LXbf2L0QlNv8h2oboB1VYeBXGahfB4nMmLbTcwZRmYG2u4p51CwXgazyjJBmDX P2ctLYenyXvrMcVGWh0yYB0ZyCqtcmEg418bR556XFBQ0sJJNCJpg== Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4gbpx65m41-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 04 Sep 2026 22:35:41 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 684MBMZn010196; Fri, 4 Sep 2026 22:35:40 GMT Received: from smtprelay05.wdc07v.mail.ibm.com ([172.16.1.72]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4gcb8hyj05-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 04 Sep 2026 22:35:40 +0000 (GMT) Received: from smtpav05.dal12v.mail.ibm.com (smtpav05.dal12v.mail.ibm.com [10.241.53.104]) by smtprelay05.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 684MZdKm2556494 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 4 Sep 2026 22:35:39 GMT Received: from smtpav05.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EA30058056; Fri, 4 Sep 2026 22:35:38 +0000 (GMT) Received: from smtpav05.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D8A8658065; Fri, 4 Sep 2026 22:35:37 +0000 (GMT) Received: from li-4c4c4544-004d-4810-8043-b7c04f423534.ibm.com.com (unknown [9.61.80.215]) by smtpav05.dal12v.mail.ibm.com (Postfix) with ESMTP; Fri, 4 Sep 2026 22:35:37 +0000 (GMT) From: Anthony Krowiak To: linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jjherne@linux.ibm.com, borntraeger@de.ibm.com, mjrosato@linux.ibm.com, pasic@linux.ibm.com, alex@shazbot.org, kwankhede@nvidia.com, fiuczy@linux.ibm.com, pbonzini@redhat.com, frankja@linux.ibm.com, imbrenda@linux.ibm.com, agordeev@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com, stable@vger.kernel.org Subject: [PATCH v7 5/6] s390/vfio-ap: fix queue state leakage to guest and host Date: Fri, 4 Sep 2026 18:35:30 -0400 Message-ID: <20260904223531.1611088-6-akrowiak@linux.ibm.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260904223531.1611088-1-akrowiak@linux.ibm.com> References: <20260904223531.1611088-1-akrowiak@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=PPc/P/qC c=1 sm=1 tr=0 ts=6a9b47bd cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=VdqzKS8jKosA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=iQ6ETzBq9ecOQQE5vZCe:22 a=VwQbUJbxAAAA:8 a=VnNF1IyMAAAA:8 a=HKRPhIxKZPI_7FFxOiYA:9 X-Proofpoint-GUID: PIyzCMSHuj5aAtK6S_VZj2h1JObK7B8V X-Proofpoint-ORIG-GUID: PIyzCMSHuj5aAtK6S_VZj2h1JObK7B8V X-Proofpoint-Spam-Info: AW1haW4tMjYwOTA0MDIwOSBTYWx0ZWRfX6J0GpZ5X8J7D fX6PCIXkIblxSUONyPzBVSQtHenqHX7iwmVk0Bf/VX3MRTxYtMfsfwpU8mc6MG12P8V+qpghxMC QZxt+NE7vSB/0bFkQxiXM3mOcInyLj8= X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwOTA0MDIwOSBTYWx0ZWRfX2RAL0TanlDDn yZ7aV27Qt4PfTROzvDUUstKmli8QxVf5AxW2uI88ok67cYSvpmlKhDUtfnWSGG+E71cKZ4BRutd 20Z+sC2SPQxytHgjLuVwEWJrDacZlbNYaqJfavyKSSQk4Yf9Fyu/EZ8qNX5RqnSpL7CP3Em+/RK 5/VmvMI3l7NQXwvFxQZKCNZ8K1fjs+RCMF06VAQkLApfDh40Vqt2bpV7KdbJb1P3OuMPqlPhDRP k2+CXBTpvZzfsyqAFZEOluXz+Jnq47AB7jHQXXCX94QchdU+8oqA1kUrihS/cNpSOyB/8nSPXkN I2twXdd6q04mflmd5vF2azIiggSwArw/PR9iBosjE0cOXM1vjWCb77j7yBWH5avBp696NpUbQSN AuFocIDcNVxcDDhzzsw7uGcC4lI9aQCkVL2lCazIkA529mwK6ZeF/KfEmfEbta8KKYxP8wRqyPk 0FUYTmBG0nzqT9gJd/g== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1176,Hydra:6.1.134,FMLib:17.12.100.49 definitions=2026-09-04_06,2026-09-03_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 phishscore=0 adultscore=0 suspectscore=0 bulkscore=0 spamscore=0 priorityscore=1501 impostorscore=0 malwarescore=0 lowpriorityscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2606150000 definitions=main-2609040209 Content-Type: text/plain; charset="utf-8" Commit dd174833e44e ("s390/vfio-ap: remove upper limit on wait for queue reset to complete") removed the upper bound on the wait for a queue reset to complete in apq_reset_check(), thus allowing the function to loop indefinitely. The reason given was to ensure both the security requirements and prevent resource leakage and corruption in the hypervisor. That is a legitimate concern; however, functions initiating the reset all hold the matrix_dev->mdevs_lock which guards access to all of the mdevs under the control of the vfio_ap device driver. Blocking of access prevents a system administrator from configuring the mdevs (i.e., assigning/unassigning adapters, domains and control domains via the mdev's sysfs interfaces) and may hang any guest that is started using one of the mdevs to supply its AP configuration. This patch limits the potential hang to the AP_RESET_MAX_WAIT (2000ms) timeout introduced in the preceding commit, and prevents leakage of queue state to a guest. _queue_passable() accepted AP_RESPONSE_DECONFIGURED and AP_RESPONSE_CHECKSTOPPED as passable states in addition to AP_RESPONSE_NORMAL. Neither DECONFIGURED nor CHECKSTOPPED confirms that the queue was zeroized; only AP_RESPONSE_NORMAL (0) does. A queue that is not confirmed zeroized must not be passed through to a guest, as it may contain key material from a previous guest or host operation. To fix this, _queue_passable() is limited to returning true only when reset_status.response_code =3D=3D AP_RESPONSE_NORMAL. A new helper, apq_reset_finalize(), is introduced to ensure q->reset_status correctly reflects the confirmed end state of the queue. It copies the full TAPQ status word to q->reset_status and sets q->reset_status.response_code to AP_RESPONSE_NORMAL only when apq_status_check() returns 0, confirming zeroization via TAPQ status bit verification. For -ENODEV (DECONFIGURED or CHECKSTOPPED), the non-zero response code is left intact, ensuring _queue_passable() correctly returns false. Additionally, vfio_ap_mdev_probe_queue() now calls vfio_ap_mdev_reset_queue() and flush_work() at probe time to guarantee a clean queue before it can be assigned to a guest. Fixes: dd174833e44e ("s390/vfio-ap: remove upper limit on wait for queue re= set to complete") Cc: stable@vger.kernel.org Signed-off-by: Anthony Krowiak --- drivers/s390/crypto/vfio_ap_ops.c | 63 +++++++++++++++++++++---------- 1 file changed, 44 insertions(+), 19 deletions(-) diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_a= p_ops.c index 672fae69739a..100ec011497f 100644 --- a/drivers/s390/crypto/vfio_ap_ops.c +++ b/drivers/s390/crypto/vfio_ap_ops.c @@ -785,14 +785,14 @@ static bool _queue_passable(struct vfio_ap_queue *q) if (!q) return false; =20 - switch (q->reset_status.response_code) { - case AP_RESPONSE_NORMAL: - case AP_RESPONSE_DECONFIGURED: - case AP_RESPONSE_CHECKSTOPPED: - return true; - default: - return false; - } + /* + * A queue is only passable if zeroization was confirmed by + * apq_reset_check() via TAPQ status bit verification. This is + * indicated by reset_status.response_code =3D=3D AP_RESPONSE_NORMAL (0). + * This is to protect against leaking the internal state of the queue + * to the guest. + */ + return q->reset_status.response_code =3D=3D AP_RESPONSE_NORMAL; } =20 /* @@ -2107,6 +2107,30 @@ static void report_aqic_resource_leak(struct vfio_ap= _queue *q) =20 #define WAIT_MSG "Waited %dms for reset of queue %02x.%04x (%u, %u, %u)" =20 +/** + * apq_reset_finalize - store final TAPQ status and free AQIC resources. + * @q: the vfio_ap_queue + * @status: the final AP queue status returned by PQAP(TAPQ) + * @ret: the return value from apq_status_check() + * + * Copies the full TAPQ status word to q->reset_status so that all status + * bits reflect the confirmed end state of the queue. If ret =3D=3D 0, + * zeroization was confirmed and the response code is overridden with + * AP_RESPONSE_NORMAL so that _queue_passable() returns true. For + * ret =3D=3D -ENODEV (DECONFIGURED or CHECKSTOPPED), the non-zero response + * code is left intact so _queue_passable() correctly returns false. + * AQIC resources are then freed. + */ +static void apq_reset_finalize(struct vfio_ap_queue *q, + struct ap_queue_status *status, int ret) +{ + memcpy(&q->reset_status, status, sizeof(*status)); + if (!ret) + q->reset_status.response_code =3D AP_RESPONSE_NORMAL; + if (q->saved_isc !=3D VFIO_AP_ISC_INVALID) + vfio_ap_free_aqic_resources(q); +} + static void apq_reset_check(struct work_struct *reset_work) { int ret =3D -EBUSY, elapsed =3D 0; @@ -2129,7 +2153,7 @@ static void apq_reset_check(struct work_struct *reset= _work) */ vfio_ap_free_aqic_resources(q); return; - + } if (elapsed >=3D AP_RESET_MAX_WAIT) { /* * Zeroization confirmed (ret =3D=3D 0): the TAPQ status bits @@ -2143,8 +2167,10 @@ static void apq_reset_check(struct work_struct *rese= t_work) * queue cannot generate interrupts, so the NIB page is * no longer a DMA target and it is safe to free it. */ - if (!ret || ret =3D=3D -ENODEV) - goto done; + if (!ret || ret =3D=3D -ENODEV) { + apq_reset_finalize(q, &status, ret); + return; + } /* * Timed out without being able to verify zapq completed. * @@ -2174,7 +2200,10 @@ static void apq_reset_check(struct work_struct *rese= t_work) * apq_reset_check() will re-issue the ZAPQ. */ q->reset_status.response_code =3D AP_RESPONSE_RESET_IN_PROGRESS; - + return; + } + if (!ret || ret =3D=3D -ENODEV) { + apq_reset_finalize(q, &status, ret); return; } if (ret =3D=3D -EBUSY) { @@ -2191,14 +2220,9 @@ static void apq_reset_check(struct work_struct *rese= t_work) ret =3D=3D -EAGAIN) { status =3D ap_zapq(q->apqn, 0); memcpy(&q->reset_status, &status, sizeof(status)); - continue; } } } - -done: - if (q->saved_isc !=3D VFIO_AP_ISC_INVALID) - vfio_ap_free_aqic_resources(q); } =20 static void vfio_ap_mdev_reset_queue(struct vfio_ap_queue *q) @@ -2698,8 +2722,9 @@ int vfio_ap_mdev_probe_queue(struct ap_device *apdev) =20 q->apqn =3D apqn; q->saved_isc =3D VFIO_AP_ISC_INVALID; - memset(&q->reset_status, 0, sizeof(q->reset_status)); INIT_WORK(&q->reset_work, apq_reset_check); + vfio_ap_mdev_reset_queue(q); + flush_work(&q->reset_work); =20 if (matrix_mdev) { vfio_ap_mdev_link_queue(matrix_mdev, q); @@ -2791,8 +2816,8 @@ void vfio_ap_mdev_remove_queue(struct ap_device *apde= v) vfio_ap_unlink_queue_fr_mdev(q); =20 dev_set_drvdata(&apdev->device, NULL); - kfree(q); release_update_locks_for_mdev(matrix_mdev); + kfree(q); } =20 /** --=20 2.53.0 From nobody Sat Sep 26 03:50:08 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7943F517BC6; Fri, 4 Sep 2026 22:35:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788561351; cv=none; b=dX3cITLw8HieEFeHYfTE1ax2t1h3z2n+Mb8NxlEoSL/uy58FE5s+D2JZubyR6y5zmpmoXZIiHVzBV424dj/4pUw0I6V6cftg+9D7lTUwoxR9wr8fzzW9wNokQURrDVhnNMK5in3m14XalGyWL8e4U/FeEOgcdHhp85C6I3s4SCs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788561351; c=relaxed/simple; bh=+xAjD3VNBKge45c8VIwjmsj0pIq0x8NRPc3aLsbltDs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=JElgwpE2o+j3Jm4gM0OCvPsNA+FQzxzSnG8CaSu5+UJScIb80HqZ2dfxDANq6/Jbbiq6ARPNBuEW3pARAmB5Mw4ellxVhst5g4bQn0Wtrk6AoWPAX4zXSFHwQJ9Y/mnmaZJV/8yeqEDU9NmgCjZ6isMZ5aY8DTgRSgYJpC20DP4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=GeW2WTfR; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="GeW2WTfR" Received: from pps.filterd (m0353729.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 684L2F5i940226; Fri, 4 Sep 2026 22:35:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=/hxX3p5Pzq5SDfpej FbvO1lArc0064pJJ9wwyi1axtU=; b=GeW2WTfRbiCAV6lwNeXYtioEPjz3Rds0t //tNcEfXzeIIfy0C7/nTEtbdaGfB85gKCVTuOHGkAXfckG3tfJ31DyT9jPLq2Kz0 xN7JtffIhRnpyyLbdi0I4GQxtfTwNrZIGO0hmS6YzP2j9uUQKYrBx+xOqwW96JlS RtjLJSNzySqn+yLDNJVNbaL2DrvRVzXKnn2AusacjmC/3+VhdInO648SuuaqwLJe PemirulDq9QddEe/FaqnnNuVBOZf746fb1Zv19UUnI+nFdtMEis7AIU4UjO6KJQ8 ldjj6P6DF71nedzDbDH0v4Ry844eGrW+l5QiXZObWCHOHj07Y2IlA== Received: from ppma12.dal12v.mail.ibm.com (dc.9e.1632.ip4.static.sl-reverse.com [50.22.158.220]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4gbq3rwkq5-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 04 Sep 2026 22:35:42 +0000 (GMT) Received: from pps.filterd (ppma12.dal12v.mail.ibm.com [127.0.0.1]) by ppma12.dal12v.mail.ibm.com (8.18.1.7/8.18.1.7) with ESMTP id 684MQIte028349; Fri, 4 Sep 2026 22:35:42 GMT Received: from smtprelay06.wdc07v.mail.ibm.com ([172.16.1.73]) by ppma12.dal12v.mail.ibm.com (PPS) with ESMTPS id 4gc9rqysqs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 04 Sep 2026 22:35:41 +0000 (GMT) Received: from smtpav05.dal12v.mail.ibm.com (smtpav05.dal12v.mail.ibm.com [10.241.53.104]) by smtprelay06.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 684MZekM65667486 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 4 Sep 2026 22:35:40 GMT Received: from smtpav05.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 12E9C5806A; Fri, 4 Sep 2026 22:35:40 +0000 (GMT) Received: from smtpav05.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 11E3458065; Fri, 4 Sep 2026 22:35:39 +0000 (GMT) Received: from li-4c4c4544-004d-4810-8043-b7c04f423534.ibm.com.com (unknown [9.61.80.215]) by smtpav05.dal12v.mail.ibm.com (Postfix) with ESMTP; Fri, 4 Sep 2026 22:35:38 +0000 (GMT) From: Anthony Krowiak To: linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: jjherne@linux.ibm.com, borntraeger@de.ibm.com, mjrosato@linux.ibm.com, pasic@linux.ibm.com, alex@shazbot.org, kwankhede@nvidia.com, fiuczy@linux.ibm.com, pbonzini@redhat.com, frankja@linux.ibm.com, imbrenda@linux.ibm.com, agordeev@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com Subject: [PATCH v7 6/6] s390/vfio-ap: replace guest-reachable WARNs with ratelimited warnings and add dmesg visibility to DBF_WARNs Date: Fri, 4 Sep 2026 18:35:31 -0400 Message-ID: <20260904223531.1611088-7-akrowiak@linux.ibm.com> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260904223531.1611088-1-akrowiak@linux.ibm.com> References: <20260904223531.1611088-1-akrowiak@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Authority-Analysis: v=2.4 cv=EIc2FVZC c=1 sm=1 tr=0 ts=6a9b47be cx=c_pps a=bLidbwmWQ0KltjZqbj+ezA==:117 a=bLidbwmWQ0KltjZqbj+ezA==:17 a=VdqzKS8jKosA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=uAbxVGIbfxUO_5tXvNgY:22 a=VnNF1IyMAAAA:8 a=IoIihqLE2SIwnEfGGcUA:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwOTA0MDIwOSBTYWx0ZWRfX0ZPP1dXu3zVq vGf+czjSiN7VJvV1bpt6TaDE+0Cxpb0t43R1/6n9z4OIfTqjyRrkjUcVtCswC1cP+xk4cSQXO9K HJpPYgjrb9RUgt+cqN6EXNpLi/1edXAS2a93cd8R4R/GScMTwuSOPVARFHjw9XZ8ds2X6gdJDi+ +AJ4vzQXXmwKClI5LSfXwr4gV28KKK44iHiwLHzNifN5pV2rESmSKj1UVKrzReQp/dWBpm1MYEB FJJqIs8+jOvcRRc4V3W1Jm9bWLNAwgX8hQG4wh8X1uWKpxygUa01yQZDP11+vgGmmnfJPKt7l4a 1nZ5dMYY56GrnM7wiYRK/TfDlsfNYWwH9lZxfsuvvZ/iH1NimmuH+9qDQbgmTArI0R9Wg646yiU o5zBT8S1p5OuQBp7z2aLe4tbPEheukMJ4VCgSzFkuq2BDjGdFBbagN683TNpiNabJx6hOnWj+hE /MUix/IXC04EngxXd5A== X-Proofpoint-GUID: ZYvP7p96ElAzMaouNy2wgKiFHUGkhrn2 X-Proofpoint-ORIG-GUID: ZYvP7p96ElAzMaouNy2wgKiFHUGkhrn2 X-Proofpoint-Spam-Info: AW1haW4tMjYwOTA0MDIwOSBTYWx0ZWRfX5x/H77ng6Mqv a6KKX7w0h/YFyxOa1nu+K7TUj2bddsgG+qRxqtuyyUGyvNhreMHevVbv8NpsE9bzg29n1nbVdiU LampvEt4tbOrLMewevVpV7ljE++UQGk= X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1176,Hydra:6.1.134,FMLib:17.12.100.49 definitions=2026-09-04_06,2026-09-03_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 impostorscore=0 suspectscore=0 priorityscore=1501 clxscore=1015 phishscore=0 spamscore=0 adultscore=0 lowpriorityscore=0 malwarescore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2606150000 definitions=main-2609040209 Content-Type: text/plain; charset="utf-8" WARN and WARN_ONCE macros in code paths reachable by a guest can be triggered repeatedly by a malicious or misbehaving guest, flooding the kernel log and potentially impacting system stability. Replace all WARN and WARN_ONCE calls reachable from the guest AP interrupt enable/disable and queue reset paths with ratelimited warning functions. When the queue is assigned to an mdev, dev_warn_ratelimited() is used so the mdev device name (which includes the UUID) appears in the message. Otherwise, pr_warn_ratelimited() is used. Four reporting functions are introduced: report_tapq_rc() - reports an invalid or unexpected response code from PQAP(TAPQ). Used in vfio_ap_wait_for_irqclear() and apq_status_check(). The signatures of both functions are changed to accept a struct vfio_ap_queue pointer instead of an apqn so the queue's mdev context is available for reporting. report_irqclear_timeout() - reports a timeout waiting for the IR bit to clear after a PQAP(AQIC) disable in vfio_ap_wait_for_irqclear(). report_aqic_disable_error() - reports a failed PQAP(AQIC) disable operation in vfio_ap_irq_disable(). Replaces three WARN_ONCE calls covering the non-operational queue, rejected disable, and retry exhaustion cases. report_zapq_rc() - reports an invalid response code from PQAP(ZAPQ) in vfio_ap_mdev_reset_queue(). The VFIO_AP_DBF_WARN() calls in vfio_ap_irq_enable() and handle_pqap() are retained but augmented with companion dev_warn_ratelimited() or pr_warn_ratelimited() calls. VFIO_AP_DBF_WARN() writes only to the s390 debug feature ring buffer, which requires a sysadmin to know to look in /sys/kernel/debug/s390dbf/ to find the messages. The companion dmesg log entries ensure that warning conditions are immediately visible in the kernel log without requiring familiarity with the s390 debug feature infrastructure. Signed-off-by: Anthony Krowiak --- drivers/s390/crypto/vfio_ap_ops.c | 128 +++++++++++++++++++++++------- 1 file changed, 98 insertions(+), 30 deletions(-) diff --git a/drivers/s390/crypto/vfio_ap_ops.c b/drivers/s390/crypto/vfio_a= p_ops.c index 100ec011497f..d037632df80e 100644 --- a/drivers/s390/crypto/vfio_ap_ops.c +++ b/drivers/s390/crypto/vfio_ap_ops.c @@ -247,13 +247,65 @@ static struct vfio_ap_queue *vfio_ap_mdev_get_queue( * * - -ETIMEDOUT the function timed out before the IR bit was cleared. */ -static int vfio_ap_wait_for_irqclear(int apqn) +static void report_tapq_rc(struct vfio_ap_queue *q, u8 rc) +{ + if (q->matrix_mdev) + dev_warn_ratelimited(mdev_dev(q->matrix_mdev->mdev), + "PQAP(TAPQ) for %02x.%04x failed with invalid rc=3D%#02x\n", + AP_QID_CARD(q->apqn), + AP_QID_QUEUE(q->apqn), rc); + else + pr_warn_ratelimited("PQAP(TAPQ) for %02x.%04x failed with invalid rc=3D%= #02x\n", + AP_QID_CARD(q->apqn), + AP_QID_QUEUE(q->apqn), rc); +} + +static void report_irqclear_timeout(struct vfio_ap_queue *q, u8 rc) +{ + if (q->matrix_mdev) + dev_warn_ratelimited(mdev_dev(q->matrix_mdev->mdev), + "PQAP(TAPQ) timed out waiting for IRQ clear on %02x.%04x: rc=3D%#= 02x\n", + AP_QID_CARD(q->apqn), + AP_QID_QUEUE(q->apqn), rc); + else + pr_warn_ratelimited("PQAP(TAPQ) timed out waiting for IRQ clear on %02x.= %04x: rc=3D%#02x\n", + AP_QID_CARD(q->apqn), + AP_QID_QUEUE(q->apqn), rc); +} + +static void report_aqic_disable_error(struct vfio_ap_queue *q, u8 rc) +{ + if (q->matrix_mdev) + dev_warn_ratelimited(mdev_dev(q->matrix_mdev->mdev), + "PQAP(AQIC) disable for %02x.%04x failed with rc=3D%#02x\n", + AP_QID_CARD(q->apqn), + AP_QID_QUEUE(q->apqn), rc); + else + pr_warn_ratelimited("PQAP(AQIC) disable for %02x.%04x failed with rc=3D%= #02x\n", + AP_QID_CARD(q->apqn), + AP_QID_QUEUE(q->apqn), rc); +} + +static void report_zapq_rc(struct vfio_ap_queue *q, u8 rc) +{ + if (q->matrix_mdev) + dev_warn_ratelimited(mdev_dev(q->matrix_mdev->mdev), + "PQAP(ZAPQ) for %02x.%04x failed with invalid rc=3D%#02x\n", + AP_QID_CARD(q->apqn), + AP_QID_QUEUE(q->apqn), rc); + else + pr_warn_ratelimited("PQAP(ZAPQ) for %02x.%04x failed with invalid rc=3D%= #02x\n", + AP_QID_CARD(q->apqn), + AP_QID_QUEUE(q->apqn), rc); +} + +static int vfio_ap_wait_for_irqclear(struct vfio_ap_queue *q) { struct ap_queue_status status; int retry =3D 5; =20 do { - status =3D ap_tapq(apqn, NULL); + status =3D ap_tapq(q->apqn, NULL); switch (status.response_code) { case AP_RESPONSE_NORMAL: case AP_RESPONSE_RESET_IN_PROGRESS: @@ -267,15 +319,12 @@ static int vfio_ap_wait_for_irqclear(int apqn) case AP_RESPONSE_DECONFIGURED: case AP_RESPONSE_CHECKSTOPPED: default: - WARN_ONCE(1, "%s: tapq rc %02x: %04x\n", __func__, - status.response_code, apqn); + report_tapq_rc(q, status.response_code); return -ENODEV; } } while (--retry); =20 - WARN_ONCE(1, "%s: tapq rc %02x: timed out waiting for interrupts disabled= for %02x.%04x\n", - __func__, status.response_code, - AP_QID_CARD(apqn), AP_QID_QUEUE(apqn)); + report_irqclear_timeout(q, status.response_code); =20 return -ETIMEDOUT; } @@ -347,7 +396,7 @@ static struct ap_queue_status vfio_ap_irq_disable(struc= t vfio_ap_queue *q) * wait until interrupt processing has been disabled * before proceeding. */ - ret =3D vfio_ap_wait_for_irqclear(q->apqn); + ret =3D vfio_ap_wait_for_irqclear(q); if (ret =3D=3D 0 || ret =3D=3D -ENODEV) goto end_free; /* @@ -374,8 +423,7 @@ static struct ap_queue_status vfio_ap_irq_disable(struc= t vfio_ap_queue *q) case AP_RESPONSE_DECONFIGURED: case AP_RESPONSE_CHECKSTOPPED: /* AP not operational; no further interrupts possible */ - WARN_ONCE(1, "%s: ap_aqic status %d\n", __func__, - status.response_code); + report_aqic_disable_error(q, status.response_code); goto end_free; case AP_RESPONSE_INVALID_ADDRESS: case AP_RESPONSE_INVALID_GISA: @@ -387,14 +435,12 @@ static struct ap_queue_status vfio_ap_irq_disable(str= uct vfio_ap_queue *q) * and the hardware still holds the NIB address. Do not * free resources. */ - WARN_ONCE(1, "%s: ap_aqic status %d\n", __func__, - status.response_code); + report_aqic_disable_error(q, status.response_code); goto end_fail; } } while (retries--); =20 - WARN_ONCE(1, "%s: ap_aqic status %d\n", __func__, - status.response_code); + report_aqic_disable_error(q, status.response_code); =20 end_fail: /* @@ -523,7 +569,10 @@ static struct ap_queue_status vfio_ap_irq_enable(struc= t vfio_ap_queue *q, if (vfio_ap_validate_nib(vcpu, &nib)) { VFIO_AP_DBF_WARN("%s: invalid NIB address: nib=3D%pad, apqn=3D%#04x\n", __func__, &nib, q->apqn); - + dev_warn_ratelimited(mdev_dev(q->matrix_mdev->mdev), + "PQAP(AQIC) enable for %02x.%04x: invalid NIB address %pad\n", + AP_QID_CARD(q->apqn), + AP_QID_QUEUE(q->apqn), &nib); status.response_code =3D AP_RESPONSE_INVALID_ADDRESS; return status; } @@ -538,7 +587,10 @@ static struct ap_queue_status vfio_ap_irq_enable(struc= t vfio_ap_queue *q, VFIO_AP_DBF_WARN("%s: vfio_pin_pages failed: rc=3D%d," "nib=3D%pad, apqn=3D%#04x\n", __func__, ret, &nib, q->apqn); - + dev_warn_ratelimited(mdev_dev(q->matrix_mdev->mdev), + "PQAP(AQIC) enable for %02x.%04x: vfio_pin_pages failed rc=3D%d\n= ", + AP_QID_CARD(q->apqn), + AP_QID_QUEUE(q->apqn), ret); status.response_code =3D AP_RESPONSE_INVALID_ADDRESS; return status; } @@ -561,7 +613,10 @@ static struct ap_queue_status vfio_ap_irq_enable(struc= t vfio_ap_queue *q, if (nisc < 0) { VFIO_AP_DBF_WARN("%s: gisc registration failed: nisc=3D%d, isc=3D%d, apq= n=3D%#04x\n", __func__, nisc, isc, q->apqn); - + dev_warn_ratelimited(mdev_dev(q->matrix_mdev->mdev), + "PQAP(AQIC) enable for %02x.%04x: GISC registration failed rc=3D%= d isc=3D%d\n", + AP_QID_CARD(q->apqn), + AP_QID_QUEUE(q->apqn), nisc, isc); vfio_unpin_pages(&q->matrix_mdev->vdev, nib, 1); status.response_code =3D AP_RESPONSE_INVALID_ADDRESS; return status; @@ -596,9 +651,14 @@ static struct ap_queue_status vfio_ap_irq_enable(struc= t vfio_ap_queue *q, * ISC that were prepared for this (rejected) request. */ ret =3D kvm_s390_gisc_unregister(kvm, isc); - if (ret) + if (ret) { VFIO_AP_DBF_WARN("%s: kvm_s390_gisc_unregister: rc=3D%d isc=3D%d, apqn= =3D%#04x\n", __func__, ret, isc, q->apqn); + dev_warn_ratelimited(mdev_dev(q->matrix_mdev->mdev), + "PQAP(AQIC) enable for %02x.%04x: GISC unregister failed rc=3D%d= isc=3D%d\n", + AP_QID_CARD(q->apqn), + AP_QID_QUEUE(q->apqn), ret, isc); + } vfio_unpin_pages(&q->matrix_mdev->vdev, nib, 1); break; } @@ -611,6 +671,11 @@ static struct ap_queue_status vfio_ap_irq_enable(struc= t vfio_ap_queue *q, aqic_gisa.zone, aqic_gisa.ir, aqic_gisa.gisc, aqic_gisa.gf, aqic_gisa.gisa, aqic_gisa.isc, q->apqn); + dev_warn_ratelimited(mdev_dev(q->matrix_mdev->mdev), + "PQAP(AQIC) enable for %02x.%04x failed with rc=3D%#02x\n", + AP_QID_CARD(q->apqn), + AP_QID_QUEUE(q->apqn), + status.response_code); } =20 return status; @@ -695,7 +760,8 @@ static int handle_pqap(struct kvm_vcpu *vcpu) if (!(vcpu->arch.sie_block->eca & ECA_AIV)) { VFIO_AP_DBF_WARN("%s: AIV facility not installed: apqn=3D0x%04x, eca=3D0= x%04x\n", __func__, apqn, vcpu->arch.sie_block->eca); - + pr_warn_ratelimited("PQAP(AQIC) for %02x.%04x: AIV facility not installe= d\n", + AP_QID_CARD(apqn), AP_QID_QUEUE(apqn)); return -EOPNOTSUPP; } =20 @@ -704,7 +770,8 @@ static int handle_pqap(struct kvm_vcpu *vcpu) if (!vcpu->kvm->arch.crypto.pqap_hook) { VFIO_AP_DBF_WARN("%s: PQAP(AQIC) hook not registered with the vfio_ap dr= iver: apqn=3D0x%04x\n", __func__, apqn); - + pr_warn_ratelimited("PQAP(AQIC) for %02x.%04x: hook not registered with = the vfio_ap driver\n", + AP_QID_CARD(apqn), AP_QID_QUEUE(apqn)); goto out_unlock; } =20 @@ -717,6 +784,9 @@ static int handle_pqap(struct kvm_vcpu *vcpu) VFIO_AP_DBF_WARN("%s: mdev %08lx-%04lx-%04lx-%04lx-%04lx%08lx not in use= : apqn=3D0x%04x\n", __func__, uuid[0], uuid[1], uuid[2], uuid[3], uuid[4], uuid[5], apqn); + dev_warn_ratelimited(mdev_dev(matrix_mdev->mdev), + "PQAP(AQIC) for %02x.%04x: mdev not in use\n", + AP_QID_CARD(apqn), AP_QID_QUEUE(apqn)); goto out_unlock; } =20 @@ -725,6 +795,9 @@ static int handle_pqap(struct kvm_vcpu *vcpu) VFIO_AP_DBF_WARN("%s: Queue %02x.%04x not bound to the vfio_ap driver\n", __func__, AP_QID_CARD(apqn), AP_QID_QUEUE(apqn)); + dev_warn_ratelimited(mdev_dev(matrix_mdev->mdev), + "PQAP(AQIC) for %02x.%04x: queue not bound to the vfio_ap driver\= n", + AP_QID_CARD(apqn), AP_QID_QUEUE(apqn)); goto out_unlock; } =20 @@ -2008,7 +2081,8 @@ static struct vfio_ap_queue *vfio_ap_find_queue(int a= pqn) return q; } =20 -static int apq_status_check(int apqn, struct ap_queue_status *status) +static int apq_status_check(struct vfio_ap_queue *q, + struct ap_queue_status *status) { switch (status->response_code) { case AP_RESPONSE_NORMAL: @@ -2070,10 +2144,7 @@ static int apq_status_check(int apqn, struct ap_queu= e_status *status) return -EAGAIN; =20 default: - WARN(true, - "failed to verify reset of queue %02x.%04x: TAPQ rc=3D%u\n", - AP_QID_CARD(apqn), AP_QID_QUEUE(apqn), - status->response_code); + report_tapq_rc(q, status->response_code); return -EIO; } } @@ -2143,7 +2214,7 @@ static void apq_reset_check(struct work_struct *reset= _work) msleep(AP_RESET_INTERVAL); elapsed +=3D AP_RESET_INTERVAL; status =3D ap_tapq(q->apqn, NULL); - ret =3D apq_status_check(q->apqn, &status); + ret =3D apq_status_check(q, &status); if (ret =3D=3D -EIO) { /* * TAPQ returned an invalid response code. This @@ -2255,10 +2326,7 @@ static void vfio_ap_mdev_reset_queue(struct vfio_ap_= queue *q) * AP interrupts or DMA-write to the NIB, free the AQIC resources * rather than leak them. */ - WARN(true, - "PQAP/ZAPQ for %02x.%04x failed with invalid rc=3D%u\n", - AP_QID_CARD(q->apqn), AP_QID_QUEUE(q->apqn), - status.response_code); + report_zapq_rc(q, status.response_code); vfio_ap_free_aqic_resources(q); } } --=20 2.53.0