From nobody Thu Apr 9 14:08:11 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 34A3424B28; Sun, 8 Mar 2026 03:04:49 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772939090; cv=none; b=f4ZpliMSeBkxqNJ86/JBd1Wi4PMuv4wL5GNLHU+qfNnIyGgpUx5IPtaQX0edYNzPY0DB/gWmYcRUoYQmSNixDYY9xfVvYIuUeIgksECg8LeCWbQmrkaOQ4twykYmHBYpgBFSzS/mh7MtudrzDhiG+grrukFz/PvoUgLvibcHlkw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772939090; c=relaxed/simple; bh=SOH99pkPWxLY+LxD4i1QK5JyubhYxu2fvavHq2wOGlE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=jH8LC/xuvF0APBs0n7kWJEt9qkqKbzYfAjfQYEmsMmgULgTzYeaW6vGXr6OeiT38KH0XrI/OAAZ4DeNlW02XKgflK4QBG1IF/92pQgmNhNRyxtqIoqVlfczz5lsGF8aHxEs6POOjudTUSlWC6/6pyIjgJzkeerjq7JyysB+RKk4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=VRax5P/z; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="VRax5P/z" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 627NOGOH3235469; Sun, 8 Mar 2026 03:04:45 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=SC2eg7UOHq1pRP0/v EprahfrAV7USBOZe9K1c1VEa8k=; b=VRax5P/zi6GaID4Qh8fiye3s6C4zyI6kb Ta1ZrStbdTDlDpXGM40jK/RlYSFacTuymnXCbO2eLhgxCBSrhaoo9n6JXCd5MVrO N2ZtL9IEAF3Isfis1zlUW0j+W+9NCnma7qkdh7aaqtTEszpmQ7M0sI9fR/bjMI0f haMFwgQKB2bFDpNL4M2EAIGGr1u/zXlhVRMeR0kof0IYxyRn2ux1FvqmofP+rGWw 4lZDHqwy4ro6lpiDYsPiFDCec2vu5lp5CFeUJE4Ju0pOX666m5MmllHPJ1BTkpG1 8JH1FS41YUjZb9l/3EiupdgtAmjgPjV77d2EZL8z0X/3BG/bp9cDw== Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4crcvr2ft9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 08 Mar 2026 03:04:45 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 6280bXDU009505; Sun, 8 Mar 2026 03:04:44 GMT Received: from smtprelay02.dal12v.mail.ibm.com ([172.16.1.4]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4crxqy0an1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 08 Mar 2026 03:04:44 +0000 Received: from smtpav01.dal12v.mail.ibm.com (smtpav01.dal12v.mail.ibm.com [10.241.53.100]) by smtprelay02.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 62834hdG15532582 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sun, 8 Mar 2026 03:04:43 GMT Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 56E4158058; Sun, 8 Mar 2026 03:04:43 +0000 (GMT) Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 84CF658057; Sun, 8 Mar 2026 03:04:42 +0000 (GMT) Received: from 9.60.13.83 (unknown [9.60.13.83]) by smtpav01.dal12v.mail.ibm.com (Postfix) with ESMTP; Sun, 8 Mar 2026 03:04:42 +0000 (GMT) From: Douglas Freimuth To: borntraeger@linux.ibm.com, imbrenda@linux.ibm.com, frankja@linux.ibm.com, david@kernel.org, hca@linux.ibm.com, gor@linux.ibm.com, agordeev@linux.ibm.com, svens@linux.ibm.com, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org Cc: mjrosato@linux.ibm.com, freimuth@linux.ibm.com Subject: [PATCH v1 1/3] Add map/unmap ioctl and clean mappings post-guest Date: Sun, 8 Mar 2026 04:04:36 +0100 Message-ID: <20260308030438.88580-2-freimuth@linux.ibm.com> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260308030438.88580-1-freimuth@linux.ibm.com> References: <20260308030438.88580-1-freimuth@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzA4MDAyNiBTYWx0ZWRfX0RQyci0PfNKC WMDbExmIYe8LvAWj75MM46hJcXZGEQmaWl7YqRn6RoLR/numdsMkgZveiCgwadqqled4mVSNAom AgMQWmnuU4L65P5QtmG/17wvJRMyTVWG6HVzFT8bBSAIcwcL7/yYn3Kz0df1waVK0TAe9JO0wcb uSItc0x3ew+28UpEEe5UvBYnY0GxLeUiNtj9v6TD9vg6O+he/I3+G5xBn/vne/Lj0wHyWCgl8sX 6vutdT9CumSTBa2JkOFUIN6UJrP0qRaXceaCsSkXiVaVaDfJVuclyrahdVkj0VgY53WeTQ7rLeF X3oiGeWnz4YLO4bZ7l4q3PdDCN+cZCAbt5u1RWXZZGTKl/0uZp7AK3vFtJbu78xArDkW3pZ0VvN aQwdL51roScl3Bgdc8z2S6NhXEEwsSa8Tam8QBRPbP3xC64HO2Z4CZZpeYpwG4ObzYrQraOIvK9 i3shJXxCc6L2E8FmNBg== X-Proofpoint-GUID: oTgqjZTeb3VomZOMyqZuWwNsadVGGWrH X-Proofpoint-ORIG-GUID: oTgqjZTeb3VomZOMyqZuWwNsadVGGWrH X-Authority-Analysis: v=2.4 cv=QoFTHFyd c=1 sm=1 tr=0 ts=69ace74d cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=RzCfie-kr_QcCd8fBx8p:22 a=VnNF1IyMAAAA:8 a=8Mf9-W6H7kIZzL-AiSkA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-08_01,2026-03-06_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 spamscore=0 priorityscore=1501 phishscore=0 lowpriorityscore=0 adultscore=0 clxscore=1011 malwarescore=0 suspectscore=0 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2602130000 definitions=main-2603080026 Content-Type: text/plain; charset="utf-8" Patch 1: This patch adds map/unmap ioctls which map the adapter set indicator pages so the pages can be accessed when interrupts are disabled. The mappings are cleaned up when the guest is removed. Fencing of Fast Inject in Secure Execution environments is enabled in this patch by not mapping adapter indicator pages. In Secure Execution environments the path of execution available before this patch is followed. Statistical counters to count map/unmap functions for adapter indicator pages are added in this patch. The counters can be used to analyze map/unmap functions in non-Secure Execution environments and similarly can be used to analyze Secure Execution environments where the counters should not be incremented as the adapter indicator pages are not mapped. Signed-off-by: Douglas Freimuth --- arch/s390/include/asm/kvm_host.h | 5 ++ arch/s390/kvm/interrupt.c | 143 +++++++++++++++++++++++++------ arch/s390/kvm/kvm-s390.c | 2 + 3 files changed, 124 insertions(+), 26 deletions(-) diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_h= ost.h index 64a50f0862aa..616be8ca4614 100644 --- a/arch/s390/include/asm/kvm_host.h +++ b/arch/s390/include/asm/kvm_host.h @@ -448,6 +448,8 @@ struct kvm_vcpu_arch { struct kvm_vm_stat { struct kvm_vm_stat_generic generic; u64 inject_io; + u64 io_390_adapter_map; + u64 io_390_adapter_unmap; u64 inject_float_mchk; u64 inject_pfault_done; u64 inject_service_signal; @@ -479,6 +481,9 @@ struct s390_io_adapter { bool masked; bool swap; bool suppressible; + struct rw_semaphore maps_lock; + struct list_head maps; + unsigned int nr_maps; }; =20 #define MAX_S390_IO_ADAPTERS ((MAX_ISC + 1) * 8) diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c index 18932a65ca68..cafc03e20f8f 100644 --- a/arch/s390/kvm/interrupt.c +++ b/arch/s390/kvm/interrupt.c @@ -2426,6 +2426,9 @@ static int register_io_adapter(struct kvm_device *dev, if (!adapter) return -ENOMEM; =20 + INIT_LIST_HEAD(&adapter->maps); + init_rwsem(&adapter->maps_lock); + adapter->nr_maps =3D 0; adapter->id =3D adapter_info.id; adapter->isc =3D adapter_info.isc; adapter->maskable =3D adapter_info.maskable; @@ -2450,12 +2453,103 @@ int kvm_s390_mask_adapter(struct kvm *kvm, unsigne= d int id, bool masked) return ret; } =20 +static struct page *get_map_page(struct kvm *kvm, u64 uaddr) +{ + struct mm_struct *mm =3D kvm->mm; + struct page *page =3D NULL; + int locked =3D 1; + + if (mmget_not_zero(mm)) { + mmap_read_lock(mm); + get_user_pages_remote(mm, uaddr, 1, FOLL_WRITE, + &page, &locked); + if (locked) + mmap_read_unlock(mm); + mmput(mm); + } + + return page; +} + +static int kvm_s390_adapter_map(struct kvm *kvm, unsigned int id, __u64 ad= dr) +{ + struct s390_io_adapter *adapter =3D get_io_adapter(kvm, id); + struct s390_map_info *map; + unsigned long flags; + int ret; + + if (!adapter || !addr) + return -EINVAL; + + map =3D kzalloc_obj(*map, GFP_KERNEL); + if (!map) + return -ENOMEM; + + map->page =3D get_map_page(kvm, addr); + if (!map->page) { + ret =3D -EINVAL; + goto out; + } + + INIT_LIST_HEAD(&map->list); + map->guest_addr =3D addr; + map->addr =3D addr; + down_write(&adapter->maps_lock); + if (adapter->nr_maps++ < MAX_S390_ADAPTER_MAPS) { + list_add_tail(&map->list, &adapter->maps); + ret =3D 0; + } else { + put_page(map->page); + ret =3D -EINVAL; + } + up_write(&adapter->maps_lock); +out: + if (ret) + kfree(map); + return ret; +} + +static int kvm_s390_adapter_unmap(struct kvm *kvm, unsigned int id, __u64 = addr) +{ + struct s390_io_adapter *adapter =3D get_io_adapter(kvm, id); + struct s390_map_info *map, *tmp; + int found =3D 0; + + if (!adapter || !addr) + return -EINVAL; + + down_write(&adapter->maps_lock); + list_for_each_entry_safe(map, tmp, &adapter->maps, list) { + if (map->guest_addr =3D=3D addr) { + found =3D 1; + adapter->nr_maps--; + list_del(&map->list); + put_page(map->page); + kfree(map); + break; + } + } + up_write(&adapter->maps_lock); + + return found ? 0 : -ENOENT; +} + void kvm_s390_destroy_adapters(struct kvm *kvm) { int i; + struct s390_map_info *map, *tmp; =20 - for (i =3D 0; i < MAX_S390_IO_ADAPTERS; i++) + for (i =3D 0; i < MAX_S390_IO_ADAPTERS; i++) { + if (!kvm->arch.adapters[i]) + continue; + list_for_each_entry_safe(map, tmp, + &kvm->arch.adapters[i]->maps, list) { + list_del(&map->list); + put_page(map->page); + kfree(map); + } kfree(kvm->arch.adapters[i]); + } } =20 static int modify_io_adapter(struct kvm_device *dev, @@ -2463,7 +2557,8 @@ static int modify_io_adapter(struct kvm_device *dev, { struct kvm_s390_io_adapter_req req; struct s390_io_adapter *adapter; - int ret; + __u64 host_addr; + int ret, idx; =20 if (copy_from_user(&req, (void __user *)attr->addr, sizeof(req))) return -EFAULT; @@ -2477,14 +2572,28 @@ static int modify_io_adapter(struct kvm_device *dev, if (ret > 0) ret =3D 0; break; - /* - * The following operations are no longer needed and therefore no-ops. - * The gpa to hva translation is done when an IRQ route is set up. The - * set_irq code uses get_user_pages_remote() to do the actual write. - */ case KVM_S390_IO_ADAPTER_MAP: case KVM_S390_IO_ADAPTER_UNMAP: - ret =3D 0; + mutex_lock(&dev->kvm->lock); + if (kvm_s390_pv_is_protected(dev->kvm)) { + mutex_unlock(&dev->kvm->lock); + break; + } + mutex_unlock(&dev->kvm->lock); + idx =3D srcu_read_lock(&dev->kvm->srcu); + host_addr =3D gpa_to_hva(dev->kvm, req.addr); + if (kvm_is_error_hva(host_addr)) { + srcu_read_unlock(&dev->kvm->srcu, idx); + return -EFAULT; + } + srcu_read_unlock(&dev->kvm->srcu, idx); + if (req.type =3D=3D KVM_S390_IO_ADAPTER_MAP) { + dev->kvm->stat.io_390_adapter_map++; + ret =3D kvm_s390_adapter_map(dev->kvm, req.id, host_addr); + } else { + dev->kvm->stat.io_390_adapter_unmap++; + ret =3D kvm_s390_adapter_unmap(dev->kvm, req.id, host_addr); + } break; default: ret =3D -EINVAL; @@ -2727,24 +2836,6 @@ static unsigned long get_ind_bit(__u64 addr, unsigne= d long bit_nr, bool swap) return swap ? (bit ^ (BITS_PER_LONG - 1)) : bit; } =20 -static struct page *get_map_page(struct kvm *kvm, u64 uaddr) -{ - struct mm_struct *mm =3D kvm->mm; - struct page *page =3D NULL; - int locked =3D 1; - - if (mmget_not_zero(mm)) { - mmap_read_lock(mm); - get_user_pages_remote(mm, uaddr, 1, FOLL_WRITE, - &page, &locked); - if (locked) - mmap_read_unlock(mm); - mmput(mm); - } - - return page; -} - static int adapter_indicators_set(struct kvm *kvm, struct s390_io_adapter *adapter, struct kvm_s390_adapter_int *adapter_int) diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c index bc7d6fa66eaf..8e6532f55a5a 100644 --- a/arch/s390/kvm/kvm-s390.c +++ b/arch/s390/kvm/kvm-s390.c @@ -68,6 +68,8 @@ const struct _kvm_stats_desc kvm_vm_stats_desc[] =3D { KVM_GENERIC_VM_STATS(), STATS_DESC_COUNTER(VM, inject_io), + STATS_DESC_COUNTER(VM, io_390_adapter_map), + STATS_DESC_COUNTER(VM, io_390_adapter_unmap), STATS_DESC_COUNTER(VM, inject_float_mchk), STATS_DESC_COUNTER(VM, inject_pfault_done), STATS_DESC_COUNTER(VM, inject_service_signal), --=20 2.52.0 From nobody Thu Apr 9 14:08:11 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 34AB328150F; Sun, 8 Mar 2026 03:04:48 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772939090; cv=none; b=eQnLiziEHhYgAxbO4B2gzbDL2r9/sWSBr0+GCdF+23n7tIjnHTot5ddVzqkSL1VuLZoMRix/GnLkdLi2k3608cd8w4e0XqpJOjFs2IFLhl/phiXv0OtkxGEH8FS05Z4cW9cR8OD+y4yGtYm0QbfjV18uXBeKFc3sl2DS9Pzn7cc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772939090; c=relaxed/simple; bh=X2Hx5h5OhgYSaF8Pj5faad2dNA/MMYHUK3JDDbRxai8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Blefjdiht3aYWKlb5FNpUgGC2QoxyqFz2KCS7/OEyFKgi1bQW7KI0ta2+mNvZstdkHOyweeZyQlwBtoGqNVyRu8kdaECgrPP99nOi+Y1wYyhab1NBtTDzmHrltWfxcCGN8nXn/qGCWl1+JGl3CJ2ngL00uHTlc4yZXBgwIjxtfU= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=Eyy75mf/; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="Eyy75mf/" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 6280U6SD3358408; Sun, 8 Mar 2026 03:04:46 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=/ZfZ+9e70WuiTmypy LwkrkJ1arXuxkss6L0U2LrOEXE=; b=Eyy75mf/qsAj7SDxYspIizMRl7Py4pfGa r8UCdHMI5y8s041NjSio4Fx01GJdrhaiomIA8a5ubTeXuT0dqE5vzVO7oQcrjrpX J7LkT/g2zLVC6vvGd7s/81izJc+Ht8iGNnooiXkpnl3X83rgB+nPRPOxjp+cZMfp 8o+Uu/pXfAJA+LLjBRlpMj/O+6u27Hqy9gDLoCwJIJusSBbCweJzW7/jmoV8lHF6 3EShlkBydwPbczvyyckb7u/WbfOc9M2q8tIOftPD8oj7e4tiWHq35PMzLSk2sDKJ DkS0hsjvgkQ/BI8re35qEg8YgS8gbOQhZHDeaRNQV5iBmNS/7E75Q== Received: from ppma13.dal12v.mail.ibm.com (dd.9e.1632.ip4.static.sl-reverse.com [50.22.158.221]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4crcvr2ftb-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 08 Mar 2026 03:04:45 +0000 (GMT) Received: from pps.filterd (ppma13.dal12v.mail.ibm.com [127.0.0.1]) by ppma13.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 6282gNw8024657; Sun, 8 Mar 2026 03:04:45 GMT Received: from smtprelay03.dal12v.mail.ibm.com ([172.16.1.5]) by ppma13.dal12v.mail.ibm.com (PPS) with ESMTPS id 4cs0jjr1u1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 08 Mar 2026 03:04:45 +0000 Received: from smtpav01.dal12v.mail.ibm.com (smtpav01.dal12v.mail.ibm.com [10.241.53.100]) by smtprelay03.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 62834iHV18678316 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sun, 8 Mar 2026 03:04:44 GMT Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 36ED758058; Sun, 8 Mar 2026 03:04:44 +0000 (GMT) Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 6F4CB58057; Sun, 8 Mar 2026 03:04:43 +0000 (GMT) Received: from 9.60.13.83 (unknown [9.60.13.83]) by smtpav01.dal12v.mail.ibm.com (Postfix) with ESMTP; Sun, 8 Mar 2026 03:04:43 +0000 (GMT) From: Douglas Freimuth To: borntraeger@linux.ibm.com, imbrenda@linux.ibm.com, frankja@linux.ibm.com, david@kernel.org, hca@linux.ibm.com, gor@linux.ibm.com, agordeev@linux.ibm.com, svens@linux.ibm.com, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org Cc: mjrosato@linux.ibm.com, freimuth@linux.ibm.com Subject: [PATCH v1 2/3] Enable adapter_indicators_set to use mapped pages Date: Sun, 8 Mar 2026 04:04:37 +0100 Message-ID: <20260308030438.88580-3-freimuth@linux.ibm.com> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260308030438.88580-1-freimuth@linux.ibm.com> References: <20260308030438.88580-1-freimuth@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzA4MDAyNiBTYWx0ZWRfXwUZu/2rsmROF 6YApjIzQqdDPS9yV1KOjYXC34H321IGDc0kGIELlt4rmKWtey8Ltt4hkETLKP7kW1Ln39mnySBc nmQOanGcSkpI+0WvBGln74W+VlrpjAX4NHc08zfHET+DXozUGLR7nsog7j4rwGl65lSA7Z3WXvm 0VmZck6DOoLZS7l3wv8OauctFo2hGKKtaIswHi9uQeFb3t0fFJbnz4HFn3Q396XbSiSZkozi5of RsEFnt/vowgLaz1BKt3MwQkkE4Slwdx2WP9TTsfwYanNPs/q//DiJ4FfgQPIVmk3Dvw61TaEiVd 62wu+F64hBt6xP2MnKyVxAtp9kvzt1faki+7EE5KUQK6d9chcfJgOlE0jNhdtf8lWjaTyi6CBQZ sa6gThHfzSkaKOFHP/+H88cvzkvLQ+VTRPtKkdUUSnwvnuX9KvXdrSzWaGFpFo44fAsoKeARvhy Dz8d2o/MOd00Z6KUltQ== X-Proofpoint-GUID: jbkGIAihYMSJUWtpPi6VOo4dys9QTMkQ X-Proofpoint-ORIG-GUID: jbkGIAihYMSJUWtpPi6VOo4dys9QTMkQ X-Authority-Analysis: v=2.4 cv=QoFTHFyd c=1 sm=1 tr=0 ts=69ace74d cx=c_pps a=AfN7/Ok6k8XGzOShvHwTGQ==:117 a=AfN7/Ok6k8XGzOShvHwTGQ==:17 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=RzCfie-kr_QcCd8fBx8p:22 a=VnNF1IyMAAAA:8 a=D0cRkWJsx0RdNmUHjycA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-08_01,2026-03-06_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 spamscore=0 priorityscore=1501 phishscore=0 lowpriorityscore=0 adultscore=0 clxscore=1011 malwarescore=0 suspectscore=0 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2602130000 definitions=main-2603080026 Content-Type: text/plain; charset="utf-8" Patch 2: This patch enables adapter_indicators_set to use mapped pages. If adapter indicator pages are not mapped then local mapping is done as it is prior to this patch. For example, Secure Execution environments will take the local mapping path as it does prior to this patch. Signed-off-by: Douglas Freimuth --- arch/s390/kvm/interrupt.c | 89 +++++++++++++++++++++++++++------------ 1 file changed, 62 insertions(+), 27 deletions(-) diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c index cafc03e20f8f..350106f84763 100644 --- a/arch/s390/kvm/interrupt.c +++ b/arch/s390/kvm/interrupt.c @@ -2836,41 +2836,74 @@ static unsigned long get_ind_bit(__u64 addr, unsign= ed long bit_nr, bool swap) return swap ? (bit ^ (BITS_PER_LONG - 1)) : bit; } =20 +static struct s390_map_info *get_map_info(struct s390_io_adapter *adapter, + u64 addr) +{ + struct s390_map_info *map; + + if (!adapter) + return NULL; + + list_for_each_entry(map, &adapter->maps, list) { + if (map->guest_addr =3D=3D addr) + return map; + } + return NULL; +} + static int adapter_indicators_set(struct kvm *kvm, - struct s390_io_adapter *adapter, - struct kvm_s390_adapter_int *adapter_int) + struct s390_io_adapter *adapter, + struct kvm_s390_adapter_int *adapter_int) { unsigned long bit; int summary_set, idx; - struct page *ind_page, *summary_page; + struct s390_map_info *ind_info, *summary_info; void *map; + struct page *ind_page, *summary_page; =20 - ind_page =3D get_map_page(kvm, adapter_int->ind_addr); - if (!ind_page) - return -1; - summary_page =3D get_map_page(kvm, adapter_int->summary_addr); - if (!summary_page) { - put_page(ind_page); - return -1; + ind_info =3D get_map_info(adapter, adapter_int->ind_addr); + if (!ind_info) { + ind_page =3D get_map_page(kvm, adapter_int->ind_addr); + if (!ind_page) + return -1; + idx =3D srcu_read_lock(&kvm->srcu); + map =3D page_address(ind_page); + bit =3D get_ind_bit(adapter_int->ind_addr, + adapter_int->ind_offset, adapter->swap); + set_bit(bit, map); + mark_page_dirty(kvm, adapter_int->ind_gaddr >> PAGE_SHIFT); + set_page_dirty_lock(ind_page); + srcu_read_unlock(&kvm->srcu, idx); + } else { + map =3D page_address(ind_info->page); + bit =3D get_ind_bit(ind_info->addr, adapter_int->ind_offset, adapter->sw= ap); + set_bit(bit, map); } =20 - idx =3D srcu_read_lock(&kvm->srcu); - map =3D page_address(ind_page); - bit =3D get_ind_bit(adapter_int->ind_addr, - adapter_int->ind_offset, adapter->swap); - set_bit(bit, map); - mark_page_dirty(kvm, adapter_int->ind_gaddr >> PAGE_SHIFT); - set_page_dirty_lock(ind_page); - map =3D page_address(summary_page); - bit =3D get_ind_bit(adapter_int->summary_addr, - adapter_int->summary_offset, adapter->swap); - summary_set =3D test_and_set_bit(bit, map); - mark_page_dirty(kvm, adapter_int->summary_gaddr >> PAGE_SHIFT); - set_page_dirty_lock(summary_page); - srcu_read_unlock(&kvm->srcu, idx); - - put_page(ind_page); - put_page(summary_page); + summary_page =3D get_map_page(kvm, adapter_int->summary_addr); + if (!summary_page) { + put_page(ind_page); + return -1; + } + idx =3D srcu_read_lock(&kvm->srcu); + map =3D page_address(summary_page); + bit =3D get_ind_bit(adapter_int->summary_addr, + adapter_int->summary_offset, adapter->swap); + summary_set =3D test_and_set_bit(bit, map); + mark_page_dirty(kvm, adapter_int->summary_gaddr >> PAGE_SHIFT); + set_page_dirty_lock(summary_page); + srcu_read_unlock(&kvm->srcu, idx); + } else { + map =3D page_address(summary_info->page); + bit =3D get_ind_bit(summary_info->addr, adapter_int->summary_offset, + adapter->swap); + summary_set =3D test_and_set_bit(bit, map); +} + + if (!ind_info) + put_page(ind_page); + if (!summary_info) + put_page(summary_page); return summary_set ? 0 : 1; } =20 @@ -2892,7 +2925,9 @@ static int set_adapter_int(struct kvm_kernel_irq_rout= ing_entry *e, adapter =3D get_io_adapter(kvm, e->adapter.adapter_id); if (!adapter) return -1; + down_read(&adapter->maps_lock); ret =3D adapter_indicators_set(kvm, adapter, &e->adapter); + up_read(&adapter->maps_lock); if ((ret > 0) && !adapter->masked) { ret =3D kvm_s390_inject_airq(kvm, adapter); if (ret =3D=3D 0) --=20 2.52.0 From nobody Thu Apr 9 14:08:11 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5F8DD2848A7; Sun, 8 Mar 2026 03:04:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772939097; cv=none; b=bq/E7vi+5wo5NeRIBNEWWIFBbqwBmfw4aaxqL6N+7we/Bu9IuxLB7zfEd6vcwzkDQHk4n0A+S0cBOAHUrKyCPiADF273YRboZaMGEwCVQVeibMMeAoiK5sxJrBoo6SwNcIgzQP8qhvZYFRzTiPRMd0aao1KxaqcmvZGcgAuycJY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1772939097; c=relaxed/simple; bh=ietlV3dLD3P+A4vdrvg8twFYI6qeNFV7VDLjQyrugyo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fGlGQY+MVG/bME88tAIkoXGvhQMCXG8REDcYsDk5IaLWjcmyULvT56fu69/7G2M450aM1reqMnw0jdGkSoAGFO2LCrEXM9Aq0665XYgGtdSJbFYkc7ZmX1ksGlhQp6KlvgXDOW2PVVCCghNncRVWmn66edGCrvuObCcZvghnGf8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=Sc7CiGPV; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="Sc7CiGPV" Received: from pps.filterd (m0360083.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 627NFCHY3509138; Sun, 8 Mar 2026 03:04:47 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=K1GlfDRADkkcx8fni eqpL+WVzINyWMatk5dWpZG5MD8=; b=Sc7CiGPV3W9mRnZBTu9A0StJb/vXOfjQA 3sU4+GqMO7Z5Q7pefCSFHtTwaAXJLRqLLTv2VDje8Lop8vt/9IjHaOTOhPDhikpj tZRn73E5x2/6IwS120qv55/PKpFsmp7t3X2khVmW53zkInqTA7aiGv0E4hxzqLxT 8CRFhRPLIkKtdiI5DGR90U5XQErkjMx7zkf5Kbm/yPViRUX1iq/RngxT8sX/YXF8 5Q+/TEphdkG6VhakZB62fS45ITONAaj/w5rxDwYE10ffgawImXgz027NnYTTQHNh 0vZW7P2TxgCOwMdee+mkMYwxKLDnyMC5kjUhtAMqGO/tTHZFeXeDA== Received: from ppma11.dal12v.mail.ibm.com (db.9e.1632.ip4.static.sl-reverse.com [50.22.158.219]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4crcvm2kp8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 08 Mar 2026 03:04:47 +0000 (GMT) Received: from pps.filterd (ppma11.dal12v.mail.ibm.com [127.0.0.1]) by ppma11.dal12v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 627KNAEo008779; Sun, 8 Mar 2026 03:04:46 GMT Received: from smtprelay04.dal12v.mail.ibm.com ([172.16.1.6]) by ppma11.dal12v.mail.ibm.com (PPS) with ESMTPS id 4cmdd1w9v4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 08 Mar 2026 03:04:46 +0000 Received: from smtpav01.dal12v.mail.ibm.com (smtpav01.dal12v.mail.ibm.com [10.241.53.100]) by smtprelay04.dal12v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 62834jgh27198096 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Sun, 8 Mar 2026 03:04:45 GMT Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 2128E58058; Sun, 8 Mar 2026 03:04:45 +0000 (GMT) Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 4F49C58057; Sun, 8 Mar 2026 03:04:44 +0000 (GMT) Received: from 9.60.13.83 (unknown [9.60.13.83]) by smtpav01.dal12v.mail.ibm.com (Postfix) with ESMTP; Sun, 8 Mar 2026 03:04:44 +0000 (GMT) From: Douglas Freimuth To: borntraeger@linux.ibm.com, imbrenda@linux.ibm.com, frankja@linux.ibm.com, david@kernel.org, hca@linux.ibm.com, gor@linux.ibm.com, agordeev@linux.ibm.com, svens@linux.ibm.com, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org Cc: mjrosato@linux.ibm.com, freimuth@linux.ibm.com Subject: [PATCH v1 3/3] Introducing kvm_arch_set_irq_inatomic fast inject Date: Sun, 8 Mar 2026 04:04:38 +0100 Message-ID: <20260308030438.88580-4-freimuth@linux.ibm.com> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260308030438.88580-1-freimuth@linux.ibm.com> References: <20260308030438.88580-1-freimuth@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: KFIiBZPp0qBUVdDqDv_-E3n4xR8oV-Jh X-Proofpoint-ORIG-GUID: KFIiBZPp0qBUVdDqDv_-E3n4xR8oV-Jh X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzA4MDAyNiBTYWx0ZWRfX/9lP+VcjsGJD yCDIggAegBN62H60qdJlz0ismJWbVzP6s8TG+he2Z7ZqCfPY7MxWlowR2eBAPoyGKPnqkh4FjdE 0YJ782+eYPPRR1M/lCisvCdFL6AmTSVUe2rWbLmfHRXmkIlLCWjhkIHR2HzULn4QEd+PWiCoTRV byBfL1V8pgm4by+D7ULZxCV1rP+i8DZXrNU+s+2yMghRxqwlcPeyV8pG5xSuJyIGT/ztkgaiAIj jt9+9r3DHwMzHxJ+jZQv39AXL5JRv67HRKoK7w4SV4jI1Xcu+q8cV0cvTyaQMDN4h8GahPMJkXI oo6rZi2053wo/d3WzyuWz4zLFRWFzVMAP6SrbC5EwnN/D6yxzRn7iBHawPLbFLWEV7dC81tEcw/ LUiJz18AM4Qr5X3C0f96ckhQh6u/jB+7B3CB8GAaC8DSkjhRdLifX6iM9Ngcqc/rYmV9O93NcVs Yeky69QZmIxpAIN1yiA== X-Authority-Analysis: v=2.4 cv=B5q0EetM c=1 sm=1 tr=0 ts=69ace74f cx=c_pps a=aDMHemPKRhS1OARIsFnwRA==:117 a=aDMHemPKRhS1OARIsFnwRA==:17 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=iQ6ETzBq9ecOQQE5vZCe:22 a=VnNF1IyMAAAA:8 a=Td7JJUAnW6VTKjkzrnUA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-08_01,2026-03-06_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 phishscore=0 adultscore=0 lowpriorityscore=0 bulkscore=0 priorityscore=1501 spamscore=0 clxscore=1015 impostorscore=0 suspectscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2602130000 definitions=main-2603080026 Content-Type: text/plain; charset="utf-8" Patch 3: This patch introduces kvm_arch_set_irq_inatomic, a fast path for irq injection. Instead of placing all interrupts on the global work queue as it does today, this patch provides a fast path for irq injection. Statistical counters have been added to enable analysis of irq i= njection on the fast path and slow path including io_390_inatomic, io_flic_inject_airq, io_set_adapter_int and io_390_inatomic_adapter_masked. In order to use this kernel series with virtio-pci, a QEMU that includes the 's390x/pci: set kvm_msi_via_irqfd_allowed' fix is needed. Additionally, the guest xml needs a thread pool and threads explicitly assigned per disk device using the common way of defining threads for disks. Signed-off-by: Douglas Freimuth --- arch/s390/include/asm/kvm_host.h | 8 +- arch/s390/kvm/interrupt.c | 169 ++++++++++++++++++++++++++----- arch/s390/kvm/kvm-s390.c | 24 ++++- arch/s390/kvm/kvm-s390.h | 3 +- 4 files changed, 170 insertions(+), 34 deletions(-) diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_h= ost.h index 616be8ca4614..a194e9808ae3 100644 --- a/arch/s390/include/asm/kvm_host.h +++ b/arch/s390/include/asm/kvm_host.h @@ -359,7 +359,7 @@ struct kvm_s390_float_interrupt { struct kvm_s390_mchk_info mchk; struct kvm_s390_ext_info srv_signal; int last_sleep_cpu; - struct mutex ais_lock; + spinlock_t ais_lock; u8 simm; u8 nimm; }; @@ -450,6 +450,10 @@ struct kvm_vm_stat { u64 inject_io; u64 io_390_adapter_map; u64 io_390_adapter_unmap; + u64 io_390_inatomic; + u64 io_flic_inject_airq; + u64 io_set_adapter_int; + u64 io_390_inatomic_adapter_masked; u64 inject_float_mchk; u64 inject_pfault_done; u64 inject_service_signal; @@ -481,7 +485,7 @@ struct s390_io_adapter { bool masked; bool swap; bool suppressible; - struct rw_semaphore maps_lock; + spinlock_t maps_lock; struct list_head maps; unsigned int nr_maps; }; diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c index 350106f84763..0429a1bc4b7a 100644 --- a/arch/s390/kvm/interrupt.c +++ b/arch/s390/kvm/interrupt.c @@ -1963,15 +1963,10 @@ static int __inject_vm(struct kvm *kvm, struct kvm_= s390_interrupt_info *inti) } =20 int kvm_s390_inject_vm(struct kvm *kvm, - struct kvm_s390_interrupt *s390int) + struct kvm_s390_interrupt *s390int, struct kvm_s390_interrupt_inf= o *inti) { - struct kvm_s390_interrupt_info *inti; int rc; =20 - inti =3D kzalloc_obj(*inti, GFP_KERNEL_ACCOUNT); - if (!inti) - return -ENOMEM; - inti->type =3D s390int->type; switch (inti->type) { case KVM_S390_INT_VIRTIO: @@ -2284,6 +2279,7 @@ static int flic_ais_mode_get_all(struct kvm *kvm, str= uct kvm_device_attr *attr) { struct kvm_s390_float_interrupt *fi =3D &kvm->arch.float_int; struct kvm_s390_ais_all ais; + unsigned long flags; =20 if (attr->attr < sizeof(ais)) return -EINVAL; @@ -2291,10 +2287,10 @@ static int flic_ais_mode_get_all(struct kvm *kvm, s= truct kvm_device_attr *attr) if (!test_kvm_facility(kvm, 72)) return -EOPNOTSUPP; =20 - mutex_lock(&fi->ais_lock); + spin_lock_irqsave(&fi->ais_lock, flags); ais.simm =3D fi->simm; ais.nimm =3D fi->nimm; - mutex_unlock(&fi->ais_lock); + spin_unlock_irqrestore(&fi->ais_lock, flags); =20 if (copy_to_user((void __user *)attr->addr, &ais, sizeof(ais))) return -EFAULT; @@ -2427,7 +2423,7 @@ static int register_io_adapter(struct kvm_device *dev, return -ENOMEM; =20 INIT_LIST_HEAD(&adapter->maps); - init_rwsem(&adapter->maps_lock); + spin_lock_init(&adapter->maps_lock); adapter->nr_maps =3D 0; adapter->id =3D adapter_info.id; adapter->isc =3D adapter_info.isc; @@ -2494,7 +2490,7 @@ static int kvm_s390_adapter_map(struct kvm *kvm, unsi= gned int id, __u64 addr) INIT_LIST_HEAD(&map->list); map->guest_addr =3D addr; map->addr =3D addr; - down_write(&adapter->maps_lock); + spin_lock_irqsave(&adapter->maps_lock, flags); if (adapter->nr_maps++ < MAX_S390_ADAPTER_MAPS) { list_add_tail(&map->list, &adapter->maps); ret =3D 0; @@ -2502,7 +2498,7 @@ static int kvm_s390_adapter_map(struct kvm *kvm, unsi= gned int id, __u64 addr) put_page(map->page); ret =3D -EINVAL; } - up_write(&adapter->maps_lock); + spin_unlock_irqrestore(&adapter->maps_lock, flags); out: if (ret) kfree(map); @@ -2514,11 +2510,12 @@ static int kvm_s390_adapter_unmap(struct kvm *kvm, = unsigned int id, __u64 addr) struct s390_io_adapter *adapter =3D get_io_adapter(kvm, id); struct s390_map_info *map, *tmp; int found =3D 0; + unsigned long flags; =20 if (!adapter || !addr) return -EINVAL; =20 - down_write(&adapter->maps_lock); + spin_lock_irqsave(&adapter->maps_lock, flags); list_for_each_entry_safe(map, tmp, &adapter->maps, list) { if (map->guest_addr =3D=3D addr) { found =3D 1; @@ -2529,7 +2526,7 @@ static int kvm_s390_adapter_unmap(struct kvm *kvm, un= signed int id, __u64 addr) break; } } - up_write(&adapter->maps_lock); + spin_unlock_irqrestore(&adapter->maps_lock, flags); =20 return found ? 0 : -ENOENT; } @@ -2630,6 +2627,7 @@ static int modify_ais_mode(struct kvm *kvm, struct kv= m_device_attr *attr) struct kvm_s390_float_interrupt *fi =3D &kvm->arch.float_int; struct kvm_s390_ais_req req; int ret =3D 0; + unsigned long flags; =20 if (!test_kvm_facility(kvm, 72)) return -EOPNOTSUPP; @@ -2646,7 +2644,7 @@ static int modify_ais_mode(struct kvm *kvm, struct kv= m_device_attr *attr) 2 : KVM_S390_AIS_MODE_SINGLE : KVM_S390_AIS_MODE_ALL, req.mode); =20 - mutex_lock(&fi->ais_lock); + spin_lock_irqsave(&fi->ais_lock, flags); switch (req.mode) { case KVM_S390_AIS_MODE_ALL: fi->simm &=3D ~AIS_MODE_MASK(req.isc); @@ -2659,7 +2657,7 @@ static int modify_ais_mode(struct kvm *kvm, struct kv= m_device_attr *attr) default: ret =3D -EINVAL; } - mutex_unlock(&fi->ais_lock); + spin_unlock_irqrestore(&fi->ais_lock, flags); =20 return ret; } @@ -2673,25 +2671,33 @@ static int kvm_s390_inject_airq(struct kvm *kvm, .parm =3D 0, .parm64 =3D isc_to_int_word(adapter->isc), }; + struct kvm_s390_interrupt_info *inti; + unsigned long flags; + int ret =3D 0; =20 + inti =3D kzalloc_obj(*inti, GFP_KERNEL_ACCOUNT); + if (!inti) + return -ENOMEM; + if (!test_kvm_facility(kvm, 72) || !adapter->suppressible) - return kvm_s390_inject_vm(kvm, &s390int); + return kvm_s390_inject_vm(kvm, &s390int, inti); =20 - mutex_lock(&fi->ais_lock); + spin_lock_irqsave(&fi->ais_lock, flags); if (fi->nimm & AIS_MODE_MASK(adapter->isc)) { trace_kvm_s390_airq_suppressed(adapter->id, adapter->isc); + kfree(inti); goto out; } =20 - ret =3D kvm_s390_inject_vm(kvm, &s390int); + ret =3D kvm_s390_inject_vm(kvm, &s390int, inti); if (!ret && (fi->simm & AIS_MODE_MASK(adapter->isc))) { fi->nimm |=3D AIS_MODE_MASK(adapter->isc); trace_kvm_s390_modify_ais_mode(adapter->isc, KVM_S390_AIS_MODE_SINGLE, 2); } out: - mutex_unlock(&fi->ais_lock); + spin_unlock_irqrestore(&fi->ais_lock, flags); return ret; } =20 @@ -2700,6 +2706,8 @@ static int flic_inject_airq(struct kvm *kvm, struct k= vm_device_attr *attr) unsigned int id =3D attr->attr; struct s390_io_adapter *adapter =3D get_io_adapter(kvm, id); =20 + kvm->stat.io_flic_inject_airq++; + if (!adapter) return -EINVAL; =20 @@ -2710,6 +2718,7 @@ static int flic_ais_mode_set_all(struct kvm *kvm, str= uct kvm_device_attr *attr) { struct kvm_s390_float_interrupt *fi =3D &kvm->arch.float_int; struct kvm_s390_ais_all ais; + unsigned long flags; =20 if (!test_kvm_facility(kvm, 72)) return -EOPNOTSUPP; @@ -2717,10 +2726,10 @@ static int flic_ais_mode_set_all(struct kvm *kvm, s= truct kvm_device_attr *attr) if (copy_from_user(&ais, (void __user *)attr->addr, sizeof(ais))) return -EFAULT; =20 - mutex_lock(&fi->ais_lock); + spin_lock_irqsave(&fi->ais_lock, flags); fi->simm =3D ais.simm; fi->nimm =3D ais.nimm; - mutex_unlock(&fi->ais_lock); + spin_unlock_irqrestore(&fi->ais_lock, flags); =20 return 0; } @@ -2852,17 +2861,20 @@ static struct s390_map_info *get_map_info(struct s3= 90_io_adapter *adapter, } =20 static int adapter_indicators_set(struct kvm *kvm, - struct s390_io_adapter *adapter, - struct kvm_s390_adapter_int *adapter_int) + struct s390_io_adapter *adapter, + struct kvm_s390_adapter_int *adapter_int) { unsigned long bit; int summary_set, idx; struct s390_map_info *ind_info, *summary_info; void *map; struct page *ind_page, *summary_page; + unsigned long flags; =20 + spin_lock_irqsave(&adapter->maps_lock, flags); ind_info =3D get_map_info(adapter, adapter_int->ind_addr); if (!ind_info) { + spin_unlock_irqrestore(&adapter->maps_lock, flags); ind_page =3D get_map_page(kvm, adapter_int->ind_addr); if (!ind_page) return -1; @@ -2878,8 +2890,13 @@ static int adapter_indicators_set(struct kvm *kvm, map =3D page_address(ind_info->page); bit =3D get_ind_bit(ind_info->addr, adapter_int->ind_offset, adapter->sw= ap); set_bit(bit, map); + spin_unlock_irqrestore(&adapter->maps_lock, flags); } =20 + spin_lock_irqsave(&adapter->maps_lock, flags); + summary_info =3D get_map_info(adapter, adapter_int->summary_addr); + if (!summary_info) { + spin_unlock_irqrestore(&adapter->maps_lock, flags); summary_page =3D get_map_page(kvm, adapter_int->summary_addr); if (!summary_page) { put_page(ind_page); @@ -2898,6 +2915,7 @@ static int adapter_indicators_set(struct kvm *kvm, bit =3D get_ind_bit(summary_info->addr, adapter_int->summary_offset, adapter->swap); summary_set =3D test_and_set_bit(bit, map); + spin_unlock_irqrestore(&adapter->maps_lock, flags); } =20 if (!ind_info) @@ -2907,6 +2925,37 @@ static int adapter_indicators_set(struct kvm *kvm, return summary_set ? 0 : 1; } =20 +static int adapter_indicators_set_fast(struct kvm *kvm, + struct s390_io_adapter *adapter, + struct kvm_s390_adapter_int *adapter_int) +{ + unsigned long bit; + int summary_set; + struct s390_map_info *ind_info, *summary_info; + void *map; + + spin_lock(&adapter->maps_lock); + ind_info =3D get_map_info(adapter, adapter_int->ind_addr); + if (!ind_info) { + spin_unlock(&adapter->maps_lock); + return -EWOULDBLOCK; + } + map =3D page_address(ind_info->page); + bit =3D get_ind_bit(ind_info->addr, adapter_int->ind_offset, adapter->swa= p); + set_bit(bit, map); + summary_info =3D get_map_info(adapter, adapter_int->summary_addr); + if (!summary_info) { + spin_unlock(&adapter->maps_lock); + return -EWOULDBLOCK; + } + map =3D page_address(summary_info->page); + bit =3D get_ind_bit(summary_info->addr, adapter_int->summary_offset, + adapter->swap); + summary_set =3D test_and_set_bit(bit, map); + spin_unlock(&adapter->maps_lock); + return summary_set ? 0 : 1; +} + /* * < 0 - not injected due to error * =3D 0 - coalesced, summary indicator already active @@ -2919,15 +2968,15 @@ static int set_adapter_int(struct kvm_kernel_irq_ro= uting_entry *e, int ret; struct s390_io_adapter *adapter; =20 + kvm->stat.io_set_adapter_int++; + /* We're only interested in the 0->1 transition. */ if (!level) return 0; adapter =3D get_io_adapter(kvm, e->adapter.adapter_id); if (!adapter) return -1; - down_read(&adapter->maps_lock); ret =3D adapter_indicators_set(kvm, adapter, &e->adapter); - up_read(&adapter->maps_lock); if ((ret > 0) && !adapter->masked) { ret =3D kvm_s390_inject_airq(kvm, adapter); if (ret =3D=3D 0) @@ -2982,7 +3031,6 @@ int kvm_set_routing_entry(struct kvm *kvm, int idx; =20 switch (ue->type) { - /* we store the userspace addresses instead of the guest addresses */ case KVM_IRQ_ROUTING_S390_ADAPTER: if (kvm_is_ucontrol(kvm)) return -EINVAL; @@ -3565,3 +3613,72 @@ int __init kvm_s390_gib_init(u8 nisc) out: return rc; } + +/* + * kvm_arch_set_irq_inatomic: fast-path for irqfd injection + */ +int kvm_arch_set_irq_inatomic(struct kvm_kernel_irq_routing_entry *e, + struct kvm *kvm, int irq_source_id, int level, + bool line_status) +{ + int ret; + struct s390_io_adapter *adapter; + struct kvm_s390_float_interrupt *fi =3D &kvm->arch.float_int; + struct kvm_s390_interrupt_info *inti; + struct kvm_s390_interrupt s390int =3D { + .type =3D KVM_S390_INT_IO(1, 0, 0, 0), + .parm =3D 0, + }; + + kvm->stat.io_390_inatomic++; + + /* We're only interested in the 0->1 transition. */ + if (!level) + return -EWOULDBLOCK; + if (e->type !=3D KVM_IRQ_ROUTING_S390_ADAPTER) + return -EWOULDBLOCK; + + adapter =3D get_io_adapter(kvm, e->adapter.adapter_id); + if (!adapter) + return -EWOULDBLOCK; + + s390int.parm64 =3D isc_to_int_word(adapter->isc); + ret =3D adapter_indicators_set_fast(kvm, adapter, &e->adapter); + if (ret < 0) + return -EWOULDBLOCK; + if (!ret || adapter->masked) { + kvm->stat.io_390_inatomic_adapter_masked++; + return 0; + } + + inti =3D kzalloc_obj(*inti, GFP_ATOMIC); + if (!inti) + return -EWOULDBLOCK; + + if (!test_kvm_facility(kvm, 72) || !adapter->suppressible) { + ret =3D kvm_s390_inject_vm(kvm, &s390int, inti); + if (ret =3D=3D 0) + return ret; + else + return -EWOULDBLOCK; + } + + spin_lock(&fi->ais_lock); + if (fi->nimm & AIS_MODE_MASK(adapter->isc)) { + trace_kvm_s390_airq_suppressed(adapter->id, adapter->isc); + kfree(inti); + goto out; + } + + ret =3D kvm_s390_inject_vm(kvm, &s390int, inti); + if (!ret && (fi->simm & AIS_MODE_MASK(adapter->isc))) { + fi->nimm |=3D AIS_MODE_MASK(adapter->isc); + trace_kvm_s390_modify_ais_mode(adapter->isc, + KVM_S390_AIS_MODE_SINGLE, 2); + } + goto out; + +out: + spin_unlock(&fi->ais_lock); + return 0; +} diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c index 8e6532f55a5a..29386afe6b29 100644 --- a/arch/s390/kvm/kvm-s390.c +++ b/arch/s390/kvm/kvm-s390.c @@ -70,6 +70,10 @@ const struct _kvm_stats_desc kvm_vm_stats_desc[] =3D { STATS_DESC_COUNTER(VM, inject_io), STATS_DESC_COUNTER(VM, io_390_adapter_map), STATS_DESC_COUNTER(VM, io_390_adapter_unmap), + STATS_DESC_COUNTER(VM, io_390_inatomic), + STATS_DESC_COUNTER(VM, io_flic_inject_airq), + STATS_DESC_COUNTER(VM, io_set_adapter_int), + STATS_DESC_COUNTER(VM, io_390_inatomic_adapter_masked), STATS_DESC_COUNTER(VM, inject_float_mchk), STATS_DESC_COUNTER(VM, inject_pfault_done), STATS_DESC_COUNTER(VM, inject_service_signal), @@ -2844,6 +2848,7 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int= ioctl, unsigned long arg) void __user *argp =3D (void __user *)arg; struct kvm_device_attr attr; int r; + struct kvm_s390_interrupt_info *inti; =20 switch (ioctl) { case KVM_S390_INTERRUPT: { @@ -2852,7 +2857,10 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned in= t ioctl, unsigned long arg) r =3D -EFAULT; if (copy_from_user(&s390int, argp, sizeof(s390int))) break; - r =3D kvm_s390_inject_vm(kvm, &s390int); + inti =3D kzalloc_obj(*inti, GFP_KERNEL_ACCOUNT); + if (!inti) + return -ENOMEM; + r =3D kvm_s390_inject_vm(kvm, &s390int, inti); break; } case KVM_CREATE_IRQCHIP: { @@ -3250,7 +3258,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long t= ype) mutex_unlock(&kvm->lock); } =20 - mutex_init(&kvm->arch.float_int.ais_lock); + spin_lock_init(&kvm->arch.float_int.ais_lock); spin_lock_init(&kvm->arch.float_int.lock); for (i =3D 0; i < FIRQ_LIST_COUNT; i++) INIT_LIST_HEAD(&kvm->arch.float_int.lists[i]); @@ -4371,11 +4379,16 @@ int kvm_s390_try_set_tod_clock(struct kvm *kvm, con= st struct kvm_s390_vm_tod_clo return 1; } =20 -static void __kvm_inject_pfault_token(struct kvm_vcpu *vcpu, bool start_to= ken, - unsigned long token) +static int __kvm_inject_pfault_token(struct kvm_vcpu *vcpu, bool start_tok= en, + unsigned long token) { struct kvm_s390_interrupt inti; struct kvm_s390_irq irq; + struct kvm_s390_interrupt_info *inti_mem; + + inti_mem =3D kzalloc_obj(*inti_mem, GFP_KERNEL_ACCOUNT); + if (!inti_mem) + return -ENOMEM; =20 if (start_token) { irq.u.ext.ext_params2 =3D token; @@ -4384,8 +4397,9 @@ static void __kvm_inject_pfault_token(struct kvm_vcpu= *vcpu, bool start_token, } else { inti.type =3D KVM_S390_INT_PFAULT_DONE; inti.parm64 =3D token; - WARN_ON_ONCE(kvm_s390_inject_vm(vcpu->kvm, &inti)); + WARN_ON_ONCE(kvm_s390_inject_vm(vcpu->kvm, &inti, inti_mem)); } + return true; } =20 bool kvm_arch_async_page_not_present(struct kvm_vcpu *vcpu, diff --git a/arch/s390/kvm/kvm-s390.h b/arch/s390/kvm/kvm-s390.h index bf1d7798c1af..2f2da868a040 100644 --- a/arch/s390/kvm/kvm-s390.h +++ b/arch/s390/kvm/kvm-s390.h @@ -373,7 +373,8 @@ int __must_check kvm_s390_deliver_pending_interrupts(st= ruct kvm_vcpu *vcpu); void kvm_s390_clear_local_irqs(struct kvm_vcpu *vcpu); void kvm_s390_clear_float_irqs(struct kvm *kvm); int __must_check kvm_s390_inject_vm(struct kvm *kvm, - struct kvm_s390_interrupt *s390int); + struct kvm_s390_interrupt *s390int, + struct kvm_s390_interrupt_info *inti); int __must_check kvm_s390_inject_vcpu(struct kvm_vcpu *vcpu, struct kvm_s390_irq *irq); static inline int kvm_s390_inject_prog_irq(struct kvm_vcpu *vcpu, --=20 2.52.0