From nobody Mon Apr 6 21:52:03 2026 Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7517F37649C; Mon, 6 Apr 2026 06:44:36 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.156.1 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775457877; cv=none; b=EjY+A6BGdBekHc9P+d0prV5UeqhGDfEXD5/Xg+iY0RiP/AbPffft6jvhR1XzuTGxMiRfivCuqw4rojAyOdTecryyav186ejMyc9vR+yCSHnM/IfFeYGxzpeDNWjAwK96nTAGCmsWjvWYR9eIzYpwCM8PMRrfKtMJkMeA4+Ls3xY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775457877; c=relaxed/simple; bh=c/zGYdb7SNr4xSTl7tzKhmEzxuPy3qwt/E0XSORa1T0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=rivtoqSDH0IsxIWOJYXonbA/w0JSeTA6G92KuMQAG5WITh+vbKsc8J3WUu8q5pOLYEYdgZ+RJ2AA3Rki9sC2oyZggFEd37YLPGIxDrdnqws3NNr8IPd3aZhLYOdYv31huYWToYD8jbb3UfSGRJFhwHJW6KItAsTE/+KnXFoCriM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=RtWyTUWF; arc=none smtp.client-ip=148.163.156.1 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="RtWyTUWF" Received: from pps.filterd (m0356517.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 635GlcG93095729; Mon, 6 Apr 2026 06:44:27 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=KxHFQpQTEAsHtUZ1N o1H682IzSQ+rJmhhqSdd0gwwRc=; b=RtWyTUWFZZNi4oysoDlO41/TWAhWDg/Y1 LiBFAcIxf7eBCHWTHUME+E6zQuAZ5S5o9W0qjhWpP24ypMYzsM5zYUF437Ekz0K/ 3JK6MUwox93ognnFlWmfHWX3GB68SgPYDiW5QWmvGYM/Ir/bBNilNduE8iFr7F9u 96rbf/HkrIIVex/2xAjcgQmmrAohCaM1Z7VkKli8ymOQlMz6sr9UmH5sq60NrDH4 9NOzcTpFtntkKQXnscOKCr1aRM48zilMqzxiejHtwLItByOfkuMdkIYvJpSPtp3n 1xsCkG3Es0cN5ZPqZSx6oZ1qj61f0zffX4vs106Kbv88xlMSO+bKg== Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4datc2nf47-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 06 Apr 2026 06:44:26 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 6364sTZx021109; Mon, 6 Apr 2026 06:44:25 GMT Received: from smtprelay02.wdc07v.mail.ibm.com ([172.16.1.69]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4dbdbybrrf-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 06 Apr 2026 06:44:25 +0000 Received: from smtpav01.dal12v.mail.ibm.com (smtpav01.dal12v.mail.ibm.com [10.241.53.100]) by smtprelay02.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 6366iOVj20578934 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 6 Apr 2026 06:44:24 GMT Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id EABCC58061; Mon, 6 Apr 2026 06:44:23 +0000 (GMT) Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 39C5E58057; Mon, 6 Apr 2026 06:44:23 +0000 (GMT) Received: from 9.60.13.83 (unknown [9.60.13.83]) by smtpav01.dal12v.mail.ibm.com (Postfix) with ESMTP; Mon, 6 Apr 2026 06:44:23 +0000 (GMT) From: Douglas Freimuth To: borntraeger@linux.ibm.com, imbrenda@linux.ibm.com, frankja@linux.ibm.com, david@kernel.org, hca@linux.ibm.com, gor@linux.ibm.com, agordeev@linux.ibm.com, svens@linux.ibm.com, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org Cc: mjrosato@linux.ibm.com, freimuth@linux.ibm.com Subject: [PATCH v3 1/3] KVM: s390: Add map/unmap ioctl and clean mappings post-guest Date: Mon, 6 Apr 2026 08:44:17 +0200 Message-ID: <20260406064419.14384-2-freimuth@linux.ibm.com> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260406064419.14384-1-freimuth@linux.ibm.com> References: <20260406064419.14384-1-freimuth@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNDA2MDA1OCBTYWx0ZWRfX2JB5bDP8hxva SW01x5F8e+byOP9khIIObP9PqZiVjgRys6AB/KWMtIcODLDAxj2Ergue+ByyiM45Gz4OhrxtC1A Gl4X60AYRG9vVftUckK+W/WXKxZz11h6CL/UWmOd96zebwougs2DEFRUS4OFXBegY803U9r87rM jQANlTgABC+14qekzm9HEhIRc7yyncDi8uJw0YcB7Kn/wosCocTDbE2YJmRDIXSAtJXfOPpp+jh MplNV5nieBY/oj2pi5bf9i5wYdyXdimdko4uL0y1cqDJYYphyJRQOhnV2QgwoHuOH/EafUzdYE0 46EPYq7NqMcBelmvRCXD8y4VbOWNVUyrWlthux5rT4lEC0I9+kYeShcIlA6vk6ClEDucl426DrB a773Cip3ohowOT4fA+xuaOwhbIgN/60VSoS26W4NM8VzDiIR4drqHnT8D90oSOZRH49YbjoEMTg /juffqNgFQQJrVx2q1w== X-Proofpoint-GUID: gvL-BebWAO0seKySbuM2g6Gq1Hxq0QK4 X-Proofpoint-ORIG-GUID: gvL-BebWAO0seKySbuM2g6Gq1Hxq0QK4 X-Authority-Analysis: v=2.4 cv=HJvO14tv c=1 sm=1 tr=0 ts=69d3564a cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=A5OVakUREuEA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=U7nrCbtTmkRpXpFmAIza:22 a=VnNF1IyMAAAA:8 a=gs1mDF41wL2eXU8JTBIA:9 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-04-06_01,2026-04-03_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 impostorscore=0 suspectscore=0 clxscore=1015 lowpriorityscore=0 adultscore=0 malwarescore=0 spamscore=0 phishscore=0 priorityscore=1501 bulkscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2604060058 Content-Type: text/plain; charset="utf-8" S390 needs map/unmap ioctls, which map the adapter set indicator pages, so the pages can be accessed when interrupts are disabled. The mappings are cleaned up when the guest is removed. Map/Unmap ioctls are fenced in order to avoid the longterm pinning in Secure Execution environments. In Secure Execution environments the path of execution available before this patch is followed. Statistical counters to count map/unmap functions for adapter indicator pages are added. The counters can be used to analyze map/unmap functions in non-Secure Execution environments and similarly can be used to analyze Secure Execution environments where the counters will not be incremented as the adapter indicator pages are not mapped. Signed-off-by: Douglas Freimuth --- arch/s390/include/asm/kvm_host.h | 5 + arch/s390/kvm/interrupt.c | 151 +++++++++++++++++++++++++------ arch/s390/kvm/kvm-s390.c | 27 ++++++ 3 files changed, 157 insertions(+), 26 deletions(-) diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_h= ost.h index 3039c88daa63..a078420751a1 100644 --- a/arch/s390/include/asm/kvm_host.h +++ b/arch/s390/include/asm/kvm_host.h @@ -448,6 +448,8 @@ struct kvm_vcpu_arch { struct kvm_vm_stat { struct kvm_vm_stat_generic generic; u64 inject_io; + u64 io_390_adapter_map; + u64 io_390_adapter_unmap; u64 inject_float_mchk; u64 inject_pfault_done; u64 inject_service_signal; @@ -479,6 +481,9 @@ struct s390_io_adapter { bool masked; bool swap; bool suppressible; + spinlock_t maps_lock; + struct list_head maps; + unsigned int nr_maps; }; =20 #define MAX_S390_IO_ADAPTERS ((MAX_ISC + 1) * 8) diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c index 7cb8ce833b62..47bd6361c849 100644 --- a/arch/s390/kvm/interrupt.c +++ b/arch/s390/kvm/interrupt.c @@ -2426,6 +2426,9 @@ static int register_io_adapter(struct kvm_device *dev, if (!adapter) return -ENOMEM; =20 + INIT_LIST_HEAD(&adapter->maps); + spin_lock_init(&adapter->maps_lock); + adapter->nr_maps =3D 0; adapter->id =3D adapter_info.id; adapter->isc =3D adapter_info.isc; adapter->maskable =3D adapter_info.maskable; @@ -2450,12 +2453,109 @@ int kvm_s390_mask_adapter(struct kvm *kvm, unsigne= d int id, bool masked) return ret; } =20 +static struct page *get_map_page(struct kvm *kvm, u64 uaddr) +{ + struct mm_struct *mm =3D kvm->mm; + struct page *page =3D NULL; + int locked =3D 1; + + if (mmget_not_zero(mm)) { + mmap_read_lock(mm); + get_user_pages_remote(mm, uaddr, 1, FOLL_WRITE, + &page, &locked); + if (locked) + mmap_read_unlock(mm); + mmput(mm); + } + + return page; +} + +static int kvm_s390_adapter_map(struct kvm *kvm, unsigned int id, __u64 ad= dr) +{ + struct s390_io_adapter *adapter =3D get_io_adapter(kvm, id); + struct s390_map_info *map; + unsigned long flags; + int ret; + + if (!adapter || !addr) + return -EINVAL; + + map =3D kzalloc_obj(*map, GFP_KERNEL_ACCOUNT); + if (!map) + return -ENOMEM; + + map->page =3D get_map_page(kvm, addr); + if (!map->page) { + ret =3D -EINVAL; + goto out; + } + + INIT_LIST_HEAD(&map->list); + map->guest_addr =3D addr; + map->addr =3D addr; + spin_lock_irqsave(&adapter->maps_lock, flags); + if (adapter->nr_maps < MAX_S390_ADAPTER_MAPS) { + list_add_tail(&map->list, &adapter->maps); + adapter->nr_maps++; + ret =3D 0; + } else { + put_page(map->page); + ret =3D -EINVAL; + } + spin_unlock_irqrestore(&adapter->maps_lock, flags); +out: + if (ret) + kfree(map); + return ret; +} + +static int kvm_s390_adapter_unmap(struct kvm *kvm, unsigned int id, __u64 = addr) +{ + struct s390_io_adapter *adapter =3D get_io_adapter(kvm, id); + struct s390_map_info *map, *tmp; + unsigned long flags; + int found =3D 0, idx; + + if (!adapter || !addr) + return -EINVAL; + + list_for_each_entry_safe(map, tmp, &adapter->maps, list) { + if (map->guest_addr =3D=3D addr) { + spin_lock_irqsave(&adapter->maps_lock, flags); + found =3D 1; + adapter->nr_maps--; + list_del(&map->list); + spin_unlock_irqrestore(&adapter->maps_lock, flags); + idx =3D srcu_read_lock(&kvm->srcu); + mark_page_dirty(kvm, map->addr >> PAGE_SHIFT); + set_page_dirty_lock(map->page); + srcu_read_unlock(&kvm->srcu, idx); + put_page(map->page); + kfree(map); + break; + } + } + + return found ? 0 : -ENOENT; +} + void kvm_s390_destroy_adapters(struct kvm *kvm) { int i; + struct s390_map_info *map, *tmp; =20 - for (i =3D 0; i < MAX_S390_IO_ADAPTERS; i++) + for (i =3D 0; i < MAX_S390_IO_ADAPTERS; i++) { + if (!kvm->arch.adapters[i]) + continue; + list_for_each_entry_safe(map, tmp, + &kvm->arch.adapters[i]->maps, list) { + list_del(&map->list); + put_page(map->page); + kfree(map); + } kfree(kvm->arch.adapters[i]); + } } =20 static int modify_io_adapter(struct kvm_device *dev, @@ -2463,7 +2563,8 @@ static int modify_io_adapter(struct kvm_device *dev, { struct kvm_s390_io_adapter_req req; struct s390_io_adapter *adapter; - int ret; + __u64 host_addr; + int ret, idx; =20 if (copy_from_user(&req, (void __user *)attr->addr, sizeof(req))) return -EFAULT; @@ -2477,14 +2578,30 @@ static int modify_io_adapter(struct kvm_device *dev, if (ret > 0) ret =3D 0; break; - /* - * The following operations are no longer needed and therefore no-ops. - * The gpa to hva translation is done when an IRQ route is set up. The - * set_irq code uses get_user_pages_remote() to do the actual write. - */ case KVM_S390_IO_ADAPTER_MAP: case KVM_S390_IO_ADAPTER_UNMAP: - ret =3D 0; + /* If in Secure Execution mode do not long term pin. */ + mutex_lock(&dev->kvm->lock); + if (kvm_s390_pv_is_protected(dev->kvm)) { + mutex_unlock(&dev->kvm->lock); + return 0; + } + idx =3D srcu_read_lock(&dev->kvm->srcu); + host_addr =3D gpa_to_hva(dev->kvm, req.addr); + if (kvm_is_error_hva(host_addr)) { + srcu_read_unlock(&dev->kvm->srcu, idx); + return -EFAULT; + } + srcu_read_unlock(&dev->kvm->srcu, idx); + if (req.type =3D=3D KVM_S390_IO_ADAPTER_MAP) { + dev->kvm->stat.io_390_adapter_map++; + ret =3D kvm_s390_adapter_map(dev->kvm, req.id, host_addr); + mutex_unlock(&dev->kvm->lock); + } else { + dev->kvm->stat.io_390_adapter_unmap++; + ret =3D kvm_s390_adapter_unmap(dev->kvm, req.id, host_addr); + mutex_unlock(&dev->kvm->lock); + } break; default: ret =3D -EINVAL; @@ -2730,24 +2847,6 @@ static unsigned long get_ind_bit(__u64 addr, unsigne= d long bit_nr, bool swap) return swap ? (bit ^ (BITS_PER_LONG - 1)) : bit; } =20 -static struct page *get_map_page(struct kvm *kvm, u64 uaddr) -{ - struct mm_struct *mm =3D kvm->mm; - struct page *page =3D NULL; - int locked =3D 1; - - if (mmget_not_zero(mm)) { - mmap_read_lock(mm); - get_user_pages_remote(mm, uaddr, 1, FOLL_WRITE, - &page, &locked); - if (locked) - mmap_read_unlock(mm); - mmput(mm); - } - - return page; -} - static int adapter_indicators_set(struct kvm *kvm, struct s390_io_adapter *adapter, struct kvm_s390_adapter_int *adapter_int) diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c index d7838334a338..4eada48c6e27 100644 --- a/arch/s390/kvm/kvm-s390.c +++ b/arch/s390/kvm/kvm-s390.c @@ -68,6 +68,8 @@ const struct kvm_stats_desc kvm_vm_stats_desc[] =3D { KVM_GENERIC_VM_STATS(), STATS_DESC_COUNTER(VM, inject_io), + STATS_DESC_COUNTER(VM, io_390_adapter_map), + STATS_DESC_COUNTER(VM, io_390_adapter_unmap), STATS_DESC_COUNTER(VM, inject_float_mchk), STATS_DESC_COUNTER(VM, inject_pfault_done), STATS_DESC_COUNTER(VM, inject_service_signal), @@ -2491,6 +2493,30 @@ static int kvm_s390_pv_dmp(struct kvm *kvm, struct k= vm_pv_cmd *cmd, return r; } =20 +static void kvm_s390_unmap_all_adapters_pv(struct kvm *kvm) +{ + unsigned long flags; + struct s390_map_info *map, *tmp; + int i, idx; + + for (i =3D 0; i < MAX_S390_IO_ADAPTERS; i++) { + if (!kvm->arch.adapters[i]) + continue; + list_for_each_entry_safe(map, tmp, + &kvm->arch.adapters[i]->maps, list) { + spin_lock_irqsave(&kvm->arch.adapters[i]->maps_lock, flags); + list_del(&map->list); + spin_unlock_irqrestore(&kvm->arch.adapters[i]->maps_lock, flags); + idx =3D srcu_read_lock(&kvm->srcu); + mark_page_dirty(kvm, map->addr >> PAGE_SHIFT); + set_page_dirty_lock(map->page); + srcu_read_unlock(&kvm->srcu, idx); + put_page(map->page); + kfree(map); + } + } +} + static int kvm_s390_handle_pv(struct kvm *kvm, struct kvm_pv_cmd *cmd) { const bool need_lock =3D (cmd->cmd !=3D KVM_PV_ASYNC_CLEANUP_PERFORM); @@ -2507,6 +2533,7 @@ static int kvm_s390_handle_pv(struct kvm *kvm, struct= kvm_pv_cmd *cmd) if (kvm_s390_pv_is_protected(kvm)) break; =20 + kvm_s390_unmap_all_adapters_pv(kvm); mmap_write_lock(kvm->mm); /* * Disable creation of new THPs. Existing THPs can stay, they --=20 2.52.0 From nobody Mon Apr 6 21:52:03 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 154F730F7F3; Mon, 6 Apr 2026 06:44:29 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775457872; cv=none; b=tUaM6+IkxvnySoZwj/66X0TvZGxHCN83kNIc8bOe4IHod0eVffo9bIBSc35j7TGU70qn1ntKvMq/uMLUKQND+ivrFxpPCYb132pXQhwohf66o/NBSMg4gl/JUERAmyV8XhYLEj/kI32HhHzQwlYLEAh1SxWDjiVNUPs/QnLKM9w= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775457872; c=relaxed/simple; bh=Ceo3umb562I4ul6Gc1yTKdDTRwk1D76OOZUTjBWjArg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=H0EdDYiegFuOFHmVnHv51+hI5psaWd4pNCyDcMcKRI/pV/plfJ4wghsrevXt02VbvCjNJCPvOS0WYLwYQpI1dC+NFF0s5TCFz52chaBTj7BApF722cbFFjCK1FLC/UrNzaUg2e+/EA/IUJ4Ogs2n29JIdTOHAtE4ZcYAhy/UYzI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=sJWfJS1g; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="sJWfJS1g" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 63611me1680208; Mon, 6 Apr 2026 06:44:27 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=m6IqFkveI+874/6rv g0vXNiZwnZ3MwW5fYrKRAtDJPQ=; b=sJWfJS1gLQXctVRMYtwvSlW046qfNv0tO A7BvHaDS7EIHEiMkugKBICMiOjTS+KO7T2lCmOBw4UcIpV5RrCimwx5OX4MD4/8m GZi+WXgNXlwdeGEJUZWFxiikRh2W4TjnI/qSU2IcWXpR++SC5G+kVU1vJ66FlN7O 8HYJin9V69EjH7Hh+eGCRY2cl8Gpq6bAe3l8mGLmUakfihSWbGDvzL6Ql15aH5Jr ZGeExIDC4NTwiORPtpXDZ4rTsLLwy5c8zzht2d4EYI1u5cOIgd21nksf1YPMm070 AG4j3IQXRLMiyc9kUabfDk5fln2jme/z6KqDecNb7snXzz2779zkg== Received: from ppma23.wdc07v.mail.ibm.com (5d.69.3da9.ip4.static.sl-reverse.com [169.61.105.93]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4dat9rd1by-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 06 Apr 2026 06:44:26 +0000 (GMT) Received: from pps.filterd (ppma23.wdc07v.mail.ibm.com [127.0.0.1]) by ppma23.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 6366GAQf022945; Mon, 6 Apr 2026 06:44:26 GMT Received: from smtprelay02.wdc07v.mail.ibm.com ([172.16.1.69]) by ppma23.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4dbefkbkxr-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 06 Apr 2026 06:44:26 +0000 Received: from smtpav01.dal12v.mail.ibm.com (smtpav01.dal12v.mail.ibm.com [10.241.53.100]) by smtprelay02.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 6366iOQs24248862 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 6 Apr 2026 06:44:25 GMT Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id BC65658062; Mon, 6 Apr 2026 06:44:24 +0000 (GMT) Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0D35958057; Mon, 6 Apr 2026 06:44:24 +0000 (GMT) Received: from 9.60.13.83 (unknown [9.60.13.83]) by smtpav01.dal12v.mail.ibm.com (Postfix) with ESMTP; Mon, 6 Apr 2026 06:44:23 +0000 (GMT) From: Douglas Freimuth To: borntraeger@linux.ibm.com, imbrenda@linux.ibm.com, frankja@linux.ibm.com, david@kernel.org, hca@linux.ibm.com, gor@linux.ibm.com, agordeev@linux.ibm.com, svens@linux.ibm.com, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org Cc: mjrosato@linux.ibm.com, freimuth@linux.ibm.com Subject: [PATCH v3 2/3] KVM: s390: Enable adapter_indicators_set to use mapped pages Date: Mon, 6 Apr 2026 08:44:18 +0200 Message-ID: <20260406064419.14384-3-freimuth@linux.ibm.com> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260406064419.14384-1-freimuth@linux.ibm.com> References: <20260406064419.14384-1-freimuth@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: 0LzzhHE353WOIxmce9JJxk7kQjjKykVR X-Proofpoint-ORIG-GUID: 0LzzhHE353WOIxmce9JJxk7kQjjKykVR X-Authority-Analysis: v=2.4 cv=bLYb4f+Z c=1 sm=1 tr=0 ts=69d3564b cx=c_pps a=3Bg1Hr4SwmMryq2xdFQyZA==:117 a=3Bg1Hr4SwmMryq2xdFQyZA==:17 a=A5OVakUREuEA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=RzCfie-kr_QcCd8fBx8p:22 a=VnNF1IyMAAAA:8 a=DOZO3_vCJS9djEhVyToA:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNDA2MDA1OCBTYWx0ZWRfXwvSgTsuKBgEj aDyNYrixBPEMOPPVmyDCJVjZOv6T2DE58qZ8xGeogUxsP1jCahLBdr2pwOBh3LONO6XqhWRpaEi 7fFvhcwlnUm8Yd4CRVRh548d1Q/vZGpix2rfJ7pPYvKahr4prvK8v6KImUhtiRwGZytw0uKsCfz ojJw9/6XiGo6PtIZrR0phY7VXeP4GNqhLkgxhBjmTfz3LFE0Bx/VSydH/eia3q/mHpxUrBo8C8v QD6RtK0B5W0Z+WJgoAJRBBtZbRmh33fMEn34PohQNas4a0Lpbj94ECcIG9GoboTeexcwsphPKQk xtp4HS8LSO11fiAR4oG/AvD/dULN51aJRW0SNJTjRpXVKJihvXx7GooGK4GfUTwzrVd7KefzYM4 egUZm489WXXcXvxz/CkLfgKRAUzkv7elXFmj8JNN9MKH9GXvXuBMvUa6hB0uw1UkMBruQCAh4SN jY7c8Xt6XxPPVu2/a2A== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-04-06_01,2026-04-03_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 adultscore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 priorityscore=1501 lowpriorityscore=0 malwarescore=0 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2604060058 Content-Type: text/plain; charset="utf-8" The S390 adapter_indicators_set function needs to be able to use mapped=20 pages so that worked can be processed,on a fast path when interrupts are disabled. If adapter indicator pages are not mapped then local mapping is=20 done on a slow path as it is prior to this patch. For example, Secure Execution environments will take the local mapping path as it does prior to this patch. Signed-off-by: Douglas Freimuth --- arch/s390/kvm/interrupt.c | 91 ++++++++++++++++++++++++++++----------- 1 file changed, 66 insertions(+), 25 deletions(-) diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c index 47bd6361c849..f3183c9ec7f1 100644 --- a/arch/s390/kvm/interrupt.c +++ b/arch/s390/kvm/interrupt.c @@ -2847,41 +2847,82 @@ static unsigned long get_ind_bit(__u64 addr, unsign= ed long bit_nr, bool swap) return swap ? (bit ^ (BITS_PER_LONG - 1)) : bit; } =20 +static struct s390_map_info *get_map_info(struct s390_io_adapter *adapter, + u64 addr) +{ + struct s390_map_info *map; + + if (!adapter) + return NULL; + + list_for_each_entry(map, &adapter->maps, list) { + if (map->guest_addr =3D=3D addr) + return map; + } + return NULL; +} + static int adapter_indicators_set(struct kvm *kvm, struct s390_io_adapter *adapter, struct kvm_s390_adapter_int *adapter_int) { unsigned long bit; int summary_set, idx; - struct page *ind_page, *summary_page; + struct s390_map_info *ind_info, *summary_info; void *map; + struct page *ind_page, *summary_page; + unsigned long flags; =20 - ind_page =3D get_map_page(kvm, adapter_int->ind_addr); - if (!ind_page) - return -1; - summary_page =3D get_map_page(kvm, adapter_int->summary_addr); - if (!summary_page) { - put_page(ind_page); - return -1; + spin_lock_irqsave(&adapter->maps_lock, flags); + ind_info =3D get_map_info(adapter, adapter_int->ind_addr); + if (!ind_info) { + spin_unlock_irqrestore(&adapter->maps_lock, flags); + ind_page =3D get_map_page(kvm, adapter_int->ind_addr); + if (!ind_page) + return -1; + idx =3D srcu_read_lock(&kvm->srcu); + map =3D page_address(ind_page); + bit =3D get_ind_bit(adapter_int->ind_addr, + adapter_int->ind_offset, adapter->swap); + set_bit(bit, map); + mark_page_dirty(kvm, adapter_int->ind_gaddr >> PAGE_SHIFT); + set_page_dirty_lock(ind_page); + srcu_read_unlock(&kvm->srcu, idx); + } else { + map =3D page_address(ind_info->page); + bit =3D get_ind_bit(ind_info->addr, adapter_int->ind_offset, adapter->sw= ap); + set_bit(bit, map); + spin_unlock_irqrestore(&adapter->maps_lock, flags); + } + spin_lock_irqsave(&adapter->maps_lock, flags); + summary_info =3D get_map_info(adapter, adapter_int->summary_addr); + if (!summary_info) { + spin_unlock_irqrestore(&adapter->maps_lock, flags); + summary_page =3D get_map_page(kvm, adapter_int->summary_addr); + if (!summary_page && !ind_info) { + put_page(ind_page); + return -1; + } + idx =3D srcu_read_lock(&kvm->srcu); + map =3D page_address(summary_page); + bit =3D get_ind_bit(adapter_int->summary_addr, + adapter_int->summary_offset, adapter->swap); + summary_set =3D test_and_set_bit(bit, map); + mark_page_dirty(kvm, adapter_int->summary_gaddr >> PAGE_SHIFT); + set_page_dirty_lock(summary_page); + srcu_read_unlock(&kvm->srcu, idx); + } else { + map =3D page_address(summary_info->page); + bit =3D get_ind_bit(summary_info->addr, adapter_int->summary_offset, + adapter->swap); + summary_set =3D test_and_set_bit(bit, map); + spin_unlock_irqrestore(&adapter->maps_lock, flags); } =20 - idx =3D srcu_read_lock(&kvm->srcu); - map =3D page_address(ind_page); - bit =3D get_ind_bit(adapter_int->ind_addr, - adapter_int->ind_offset, adapter->swap); - set_bit(bit, map); - mark_page_dirty(kvm, adapter_int->ind_gaddr >> PAGE_SHIFT); - set_page_dirty_lock(ind_page); - map =3D page_address(summary_page); - bit =3D get_ind_bit(adapter_int->summary_addr, - adapter_int->summary_offset, adapter->swap); - summary_set =3D test_and_set_bit(bit, map); - mark_page_dirty(kvm, adapter_int->summary_gaddr >> PAGE_SHIFT); - set_page_dirty_lock(summary_page); - srcu_read_unlock(&kvm->srcu, idx); - - put_page(ind_page); - put_page(summary_page); + if (!ind_info) + put_page(ind_page); + if (!summary_info) + put_page(summary_page); return summary_set ? 0 : 1; } =20 --=20 2.52.0 From nobody Mon Apr 6 21:52:03 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5EBEB318BA4; Mon, 6 Apr 2026 06:44:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775457873; cv=none; b=oWjaqTvOOSYiuIZlBSOiGE9DKBwHQyAFsq/3MMcjfb0BnZa4JXTkBJVuZmrKFtp3w2/+y48IussOKOPk/2zzItmJi8uYbs6cu80ywnfgzjICpAfZq/5vCIfLDzL5mVNmcGcTFhedLJmMWjh1DrsFeQUaZrMkIE56Pylnw9Pi0Xc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1775457873; c=relaxed/simple; bh=AQukvAaEpU9MUTSQ8qh1WS8+QGvnvfZqpJUScugoEo4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=K7TPCEihfhr7ZMpVDhBFm1gpHc9CYaMtq53lGjYYWP3k7z0Xyrwsue40Q/oR1IWb+kB7jjIRjUGGYie+iaykYW9Ahns6yno00jUnx5m//LyD9o5NcC8XqkMYmXH4ecKl1YZKxf1/KixdYF4JUpaD23i7nMlNPeaLRmAeVCTC0x4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=qYRiP0om; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="qYRiP0om" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 635Jm4Q13421027; Mon, 6 Apr 2026 06:44:28 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:date:from:in-reply-to:message-id :mime-version:references:subject:to; s=pp1; bh=SIUuhXJLriRqSNkUW 91RwG650An0AkzhcCo5RGwIkto=; b=qYRiP0om4SQVBQf7z1OAph7xB9Uh3o1iv +F03YfTHn3vA2bQKuRvhvmezhWi8F9fExEqTgO3uBF/VYUH5I1SCNBcOuIlJsLu3 HV4+LU+Ycd0U1cz2M5qCRXEnUH5tXUS9HVLycfn/JXeHSXjz+5g2oNmEjvbvBnjq LPAHM1TMol0AtQAUBz3swy6sj5/xFnvbh73uJvW62h3yiZ8sPUhNN5+vcLbK33MV ytRZGOq3vQDL6Jf3rGJHGLmRB+aMNozCC1veXdUJksjqgyulQNhWx8q2EpFV6tjd SSvC0ertj/2yZVEr+AJhjMdcGGWs77UGih82xOEwrofg2tf/2fR8Q== Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4dat9rd1c1-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 06 Apr 2026 06:44:27 +0000 (GMT) Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 6364t93U021881; Mon, 6 Apr 2026 06:44:27 GMT Received: from smtprelay03.wdc07v.mail.ibm.com ([172.16.1.70]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4dbdbybrrm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Mon, 06 Apr 2026 06:44:27 +0000 Received: from smtpav01.dal12v.mail.ibm.com (smtpav01.dal12v.mail.ibm.com [10.241.53.100]) by smtprelay03.wdc07v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 6366i1ww21299914 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 6 Apr 2026 06:44:01 GMT Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8FB2858058; Mon, 6 Apr 2026 06:44:25 +0000 (GMT) Received: from smtpav01.dal12v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D344558057; Mon, 6 Apr 2026 06:44:24 +0000 (GMT) Received: from 9.60.13.83 (unknown [9.60.13.83]) by smtpav01.dal12v.mail.ibm.com (Postfix) with ESMTP; Mon, 6 Apr 2026 06:44:24 +0000 (GMT) From: Douglas Freimuth To: borntraeger@linux.ibm.com, imbrenda@linux.ibm.com, frankja@linux.ibm.com, david@kernel.org, hca@linux.ibm.com, gor@linux.ibm.com, agordeev@linux.ibm.com, svens@linux.ibm.com, kvm@vger.kernel.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org Cc: mjrosato@linux.ibm.com, freimuth@linux.ibm.com Subject: [PATCH v3 3/3] KVM: s390: Introducing kvm_arch_set_irq_inatomic fast inject Date: Mon, 6 Apr 2026 08:44:19 +0200 Message-ID: <20260406064419.14384-4-freimuth@linux.ibm.com> X-Mailer: git-send-email 2.52.0 In-Reply-To: <20260406064419.14384-1-freimuth@linux.ibm.com> References: <20260406064419.14384-1-freimuth@linux.ibm.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-TM-AS-GCONF: 00 X-Proofpoint-GUID: 6QMrhQW_8oH0yb6FOhdkB9v-raLSqDBk X-Proofpoint-ORIG-GUID: 6QMrhQW_8oH0yb6FOhdkB9v-raLSqDBk X-Authority-Analysis: v=2.4 cv=bLYb4f+Z c=1 sm=1 tr=0 ts=69d3564b cx=c_pps a=5BHTudwdYE3Te8bg5FgnPg==:117 a=5BHTudwdYE3Te8bg5FgnPg==:17 a=A5OVakUREuEA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=RzCfie-kr_QcCd8fBx8p:22 a=VnNF1IyMAAAA:8 a=IXbFIl4MR9N5CccVwUwA:9 X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNDA2MDA1OCBTYWx0ZWRfX95NhwsQ/6+fH juvay9eYmgTlqnAFXw3Zv1vKVBDCYoC5sENA42KPb5eepy0v+N0lwuerEe5+0V7WCWvMw3i+wbE bRGYMBsVbIPMu1m026UQHzZqosWR2hpBkS9Fl7GmFhyAqL7n2EeP+63Uw1MgnmpGmXHb45jXDeP 1XzJXlB5xLbdBHtRJnwK/PBRi9gCs4aR79onZ4Ma5Oe2V/h8IYwO09ZiY2Cvflml8yJFCbOg8GL 017aHsuGrjqd4AaH/zjFvj17TypcN9RS4yAixsDpqmCQpJsXkofQ/4fjWlenT25V0zLlWoZBTPI qvp/6ZussAOBk/8aSVUxrBJ5k2bZZiZnUFF2DhZN4aK+sJ06FN+RdpsfsneKcorSdTI+FkLyP1c aBg4qgQ2N6h70RlFPdFDutu1DODe4/2uAMk9vqwv39qTS5jvHkChZmdx0zORO5P9j8dQhjlra3Q KkELqGWpaAnM7fw7MZw== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-04-06_01,2026-04-03_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 adultscore=0 suspectscore=0 phishscore=0 bulkscore=0 spamscore=0 priorityscore=1501 lowpriorityscore=0 malwarescore=0 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2604060058 Content-Type: text/plain; charset="utf-8" S390 needs a fast path for irq injection, and along those lines we=20 introduce kvm_arch_set_irq_inatomic. Instead of placing all interrupts on the global work queue as it does today, this patch provides a fast path for irq injection. The inatomic fast path cannot lose control since it is running with interrupts disabled. This meant making the following changes that exist on=20 the slow path today. First, the adapter_indicators page needs to be mapped since it is accessed with interrupts disabled, so we added map/unmap functions. Second, access to shared resources between the fast and slow=20 paths needed to be changed from mutex and semaphores to spin_lock's. Finally, the memory allocation on the slow path utilizes GFP_KERNEL_ACCOUNT but we had to implement the fast path with GFP_ATOMIC allocation. Each of these enhancements were required to prevent blocking on the fast inject path. Fencing of Fast Inject in Secure Execution environments is enabled in the patch series by not mapping adapter indicator pages. In Secure Execution environments the path of execution available before this patch is followed. Statistical counters have been added to enable analysis of irq injection on the fast path and slow path including io_390_inatomic, io_flic_inject_airq, io_set_adapter_int and io_390_inatomic_adapter_masked. Signed-off-by: Douglas Freimuth --- arch/s390/include/asm/kvm_host.h | 6 +- arch/s390/kvm/interrupt.c | 160 +++++++++++++++++++++++++++---- arch/s390/kvm/kvm-s390.c | 24 ++++- arch/s390/kvm/kvm-s390.h | 3 +- 4 files changed, 169 insertions(+), 24 deletions(-) diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_h= ost.h index a078420751a1..90b1a19074ce 100644 --- a/arch/s390/include/asm/kvm_host.h +++ b/arch/s390/include/asm/kvm_host.h @@ -359,7 +359,7 @@ struct kvm_s390_float_interrupt { struct kvm_s390_mchk_info mchk; struct kvm_s390_ext_info srv_signal; int last_sleep_cpu; - struct mutex ais_lock; + spinlock_t ais_lock; u8 simm; u8 nimm; }; @@ -450,6 +450,10 @@ struct kvm_vm_stat { u64 inject_io; u64 io_390_adapter_map; u64 io_390_adapter_unmap; + u64 io_390_inatomic; + u64 io_flic_inject_airq; + u64 io_set_adapter_int; + u64 io_390_inatomic_adapter_masked; u64 inject_float_mchk; u64 inject_pfault_done; u64 inject_service_signal; diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c index f3183c9ec7f1..ead54f968a79 100644 --- a/arch/s390/kvm/interrupt.c +++ b/arch/s390/kvm/interrupt.c @@ -1963,15 +1963,10 @@ static int __inject_vm(struct kvm *kvm, struct kvm_= s390_interrupt_info *inti) } =20 int kvm_s390_inject_vm(struct kvm *kvm, - struct kvm_s390_interrupt *s390int) + struct kvm_s390_interrupt *s390int, struct kvm_s390_interrupt_inf= o *inti) { - struct kvm_s390_interrupt_info *inti; int rc; =20 - inti =3D kzalloc_obj(*inti, GFP_KERNEL_ACCOUNT); - if (!inti) - return -ENOMEM; - inti->type =3D s390int->type; switch (inti->type) { case KVM_S390_INT_VIRTIO: @@ -2007,6 +2002,7 @@ int kvm_s390_inject_vm(struct kvm *kvm, 2); =20 rc =3D __inject_vm(kvm, inti); + /* memory allocation is done by the caller and inti is passed in, we free= it here */ if (rc) kfree(inti); return rc; @@ -2284,6 +2280,7 @@ static int flic_ais_mode_get_all(struct kvm *kvm, str= uct kvm_device_attr *attr) { struct kvm_s390_float_interrupt *fi =3D &kvm->arch.float_int; struct kvm_s390_ais_all ais; + unsigned long flags; =20 if (attr->attr < sizeof(ais)) return -EINVAL; @@ -2291,10 +2288,10 @@ static int flic_ais_mode_get_all(struct kvm *kvm, s= truct kvm_device_attr *attr) if (!test_kvm_facility(kvm, 72)) return -EOPNOTSUPP; =20 - mutex_lock(&fi->ais_lock); + spin_lock_irqsave(&fi->ais_lock, flags); ais.simm =3D fi->simm; ais.nimm =3D fi->nimm; - mutex_unlock(&fi->ais_lock); + spin_unlock_irqrestore(&fi->ais_lock, flags); =20 if (copy_to_user((void __user *)attr->addr, &ais, sizeof(ais))) return -EFAULT; @@ -2638,6 +2635,7 @@ static int modify_ais_mode(struct kvm *kvm, struct kv= m_device_attr *attr) struct kvm_s390_float_interrupt *fi =3D &kvm->arch.float_int; struct kvm_s390_ais_req req; int ret =3D 0; + unsigned long flags; =20 if (!test_kvm_facility(kvm, 72)) return -EOPNOTSUPP; @@ -2654,7 +2652,7 @@ static int modify_ais_mode(struct kvm *kvm, struct kv= m_device_attr *attr) 2 : KVM_S390_AIS_MODE_SINGLE : KVM_S390_AIS_MODE_ALL, req.mode); =20 - mutex_lock(&fi->ais_lock); + spin_lock_irqsave(&fi->ais_lock, flags); switch (req.mode) { case KVM_S390_AIS_MODE_ALL: fi->simm &=3D ~AIS_MODE_MASK(req.isc); @@ -2667,7 +2665,7 @@ static int modify_ais_mode(struct kvm *kvm, struct kv= m_device_attr *attr) default: ret =3D -EINVAL; } - mutex_unlock(&fi->ais_lock); + spin_unlock_irqrestore(&fi->ais_lock, flags); =20 return ret; } @@ -2681,25 +2679,33 @@ static int kvm_s390_inject_airq(struct kvm *kvm, .parm =3D 0, .parm64 =3D isc_to_int_word(adapter->isc), }; + struct kvm_s390_interrupt_info *inti; + unsigned long flags; + int ret =3D 0; =20 + inti =3D kzalloc_obj(*inti, GFP_KERNEL_ACCOUNT); + if (!inti) + return -ENOMEM; + if (!test_kvm_facility(kvm, 72) || !adapter->suppressible) - return kvm_s390_inject_vm(kvm, &s390int); + return kvm_s390_inject_vm(kvm, &s390int, inti); =20 - mutex_lock(&fi->ais_lock); + spin_lock_irqsave(&fi->ais_lock, flags); if (fi->nimm & AIS_MODE_MASK(adapter->isc)) { trace_kvm_s390_airq_suppressed(adapter->id, adapter->isc); + kfree(inti); goto out; } =20 - ret =3D kvm_s390_inject_vm(kvm, &s390int); + ret =3D kvm_s390_inject_vm(kvm, &s390int, inti); if (!ret && (fi->simm & AIS_MODE_MASK(adapter->isc))) { fi->nimm |=3D AIS_MODE_MASK(adapter->isc); trace_kvm_s390_modify_ais_mode(adapter->isc, KVM_S390_AIS_MODE_SINGLE, 2); } out: - mutex_unlock(&fi->ais_lock); + spin_unlock_irqrestore(&fi->ais_lock, flags); return ret; } =20 @@ -2708,6 +2714,8 @@ static int flic_inject_airq(struct kvm *kvm, struct k= vm_device_attr *attr) unsigned int id =3D attr->attr; struct s390_io_adapter *adapter =3D get_io_adapter(kvm, id); =20 + kvm->stat.io_flic_inject_airq++; + if (!adapter) return -EINVAL; =20 @@ -2718,6 +2726,7 @@ static int flic_ais_mode_set_all(struct kvm *kvm, str= uct kvm_device_attr *attr) { struct kvm_s390_float_interrupt *fi =3D &kvm->arch.float_int; struct kvm_s390_ais_all ais; + unsigned long flags; =20 if (!test_kvm_facility(kvm, 72)) return -EOPNOTSUPP; @@ -2725,10 +2734,10 @@ static int flic_ais_mode_set_all(struct kvm *kvm, s= truct kvm_device_attr *attr) if (copy_from_user(&ais, (void __user *)attr->addr, sizeof(ais))) return -EFAULT; =20 - mutex_lock(&fi->ais_lock); + spin_lock_irqsave(&fi->ais_lock, flags); fi->simm =3D ais.simm; fi->nimm =3D ais.nimm; - mutex_unlock(&fi->ais_lock); + spin_unlock_irqrestore(&fi->ais_lock, flags); =20 return 0; } @@ -2894,6 +2903,7 @@ static int adapter_indicators_set(struct kvm *kvm, set_bit(bit, map); spin_unlock_irqrestore(&adapter->maps_lock, flags); } + spin_lock_irqsave(&adapter->maps_lock, flags); summary_info =3D get_map_info(adapter, adapter_int->summary_addr); if (!summary_info) { @@ -2926,6 +2936,44 @@ static int adapter_indicators_set(struct kvm *kvm, return summary_set ? 0 : 1; } =20 +static int adapter_indicators_set_fast(struct kvm *kvm, + struct s390_io_adapter *adapter, + struct kvm_s390_adapter_int *adapter_int, + int setbit) +{ + unsigned long bit; + int summary_set; + struct s390_map_info *ind_info, *summary_info; + void *map; + + spin_lock(&adapter->maps_lock); + ind_info =3D get_map_info(adapter, adapter_int->ind_addr); + if (!ind_info) { + spin_unlock(&adapter->maps_lock); + return -EWOULDBLOCK; + } + map =3D page_address(ind_info->page); + bit =3D get_ind_bit(ind_info->addr, adapter_int->ind_offset, adapter->swa= p); + if (setbit) + set_bit(bit, map); + else + clear_bit(bit, map); + summary_info =3D get_map_info(adapter, adapter_int->summary_addr); + if (!summary_info) { + spin_unlock(&adapter->maps_lock); + return -EWOULDBLOCK; + } + map =3D page_address(summary_info->page); + bit =3D get_ind_bit(summary_info->addr, adapter_int->summary_offset, + adapter->swap); + if (setbit) + summary_set =3D test_and_set_bit(bit, map); + else + summary_set =3D test_and_clear_bit(bit, map); + spin_unlock(&adapter->maps_lock); + return summary_set ? 0 : 1; +} + /* * < 0 - not injected due to error * =3D 0 - coalesced, summary indicator already active @@ -2938,6 +2986,8 @@ static int set_adapter_int(struct kvm_kernel_irq_rout= ing_entry *e, int ret; struct s390_io_adapter *adapter; =20 + kvm->stat.io_set_adapter_int++; + /* We're only interested in the 0->1 transition. */ if (!level) return 0; @@ -3006,7 +3056,6 @@ int kvm_set_routing_entry(struct kvm *kvm, int idx; =20 switch (ue->type) { - /* we store the userspace addresses instead of the guest addresses */ case KVM_IRQ_ROUTING_S390_ADAPTER: if (kvm_is_ucontrol(kvm)) return -EINVAL; @@ -3597,3 +3646,80 @@ int __init kvm_s390_gib_init(u8 nisc) out: return rc; } + +/* + * kvm_arch_set_irq_inatomic: fast-path for irqfd injection + */ +int kvm_arch_set_irq_inatomic(struct kvm_kernel_irq_routing_entry *e, + struct kvm *kvm, int irq_source_id, int level, + bool line_status) +{ + int ret, setbit; + struct s390_io_adapter *adapter; + struct kvm_s390_float_interrupt *fi =3D &kvm->arch.float_int; + struct kvm_s390_interrupt_info *inti; + struct kvm_s390_interrupt s390int =3D { + .type =3D KVM_S390_INT_IO(1, 0, 0, 0), + .parm =3D 0, + }; + + kvm->stat.io_390_inatomic++; + + /* We're only interested in the 0->1 transition. */ + if (!level) + return -EWOULDBLOCK; + if (e->type !=3D KVM_IRQ_ROUTING_S390_ADAPTER) + return -EWOULDBLOCK; + + adapter =3D get_io_adapter(kvm, e->adapter.adapter_id); + if (!adapter) + return -EWOULDBLOCK; + + s390int.parm64 =3D isc_to_int_word(adapter->isc); + setbit =3D 1; + ret =3D adapter_indicators_set_fast(kvm, adapter, &e->adapter, setbit); + if (ret < 0) + return -EWOULDBLOCK; + if (!ret || adapter->masked) { + kvm->stat.io_390_inatomic_adapter_masked++; + return 0; + } + + inti =3D kzalloc_obj(*inti, GFP_ATOMIC); + if (!inti) + return -EWOULDBLOCK; + + if (!test_kvm_facility(kvm, 72) || !adapter->suppressible) { + ret =3D kvm_s390_inject_vm(kvm, &s390int, inti); + if (ret =3D=3D 0) { + return ret; + } else { + setbit =3D 0; + adapter_indicators_set_fast(kvm, adapter, &e->adapter, setbit); + return -EWOULDBLOCK; + } + } + + spin_lock(&fi->ais_lock); + if (fi->nimm & AIS_MODE_MASK(adapter->isc)) { + trace_kvm_s390_airq_suppressed(adapter->id, adapter->isc); + kfree(inti); + goto out; + } + + ret =3D kvm_s390_inject_vm(kvm, &s390int, inti); + if (!ret && (fi->simm & AIS_MODE_MASK(adapter->isc))) { + fi->nimm |=3D AIS_MODE_MASK(adapter->isc); + trace_kvm_s390_modify_ais_mode(adapter->isc, + KVM_S390_AIS_MODE_SINGLE, 2); + } else if (ret) { + spin_unlock(&fi->ais_lock); + setbit =3D 0; + adapter_indicators_set_fast(kvm, adapter, &e->adapter, setbit); + return -EWOULDBLOCK; + } + +out: + spin_unlock(&fi->ais_lock); + return 0; +} diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c index 4eada48c6e27..72d083e9afa8 100644 --- a/arch/s390/kvm/kvm-s390.c +++ b/arch/s390/kvm/kvm-s390.c @@ -70,6 +70,10 @@ const struct kvm_stats_desc kvm_vm_stats_desc[] =3D { STATS_DESC_COUNTER(VM, inject_io), STATS_DESC_COUNTER(VM, io_390_adapter_map), STATS_DESC_COUNTER(VM, io_390_adapter_unmap), + STATS_DESC_COUNTER(VM, io_390_inatomic), + STATS_DESC_COUNTER(VM, io_flic_inject_airq), + STATS_DESC_COUNTER(VM, io_set_adapter_int), + STATS_DESC_COUNTER(VM, io_390_inatomic_adapter_masked), STATS_DESC_COUNTER(VM, inject_float_mchk), STATS_DESC_COUNTER(VM, inject_pfault_done), STATS_DESC_COUNTER(VM, inject_service_signal), @@ -2869,6 +2873,7 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int= ioctl, unsigned long arg) void __user *argp =3D (void __user *)arg; struct kvm_device_attr attr; int r; + struct kvm_s390_interrupt_info *inti; =20 switch (ioctl) { case KVM_S390_INTERRUPT: { @@ -2877,7 +2882,10 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned in= t ioctl, unsigned long arg) r =3D -EFAULT; if (copy_from_user(&s390int, argp, sizeof(s390int))) break; - r =3D kvm_s390_inject_vm(kvm, &s390int); + inti =3D kzalloc_obj(*inti, GFP_KERNEL_ACCOUNT); + if (!inti) + return -ENOMEM; + r =3D kvm_s390_inject_vm(kvm, &s390int, inti); break; } case KVM_CREATE_IRQCHIP: { @@ -3275,7 +3283,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long t= ype) mutex_unlock(&kvm->lock); } =20 - mutex_init(&kvm->arch.float_int.ais_lock); + spin_lock_init(&kvm->arch.float_int.ais_lock); spin_lock_init(&kvm->arch.float_int.lock); for (i =3D 0; i < FIRQ_LIST_COUNT; i++) INIT_LIST_HEAD(&kvm->arch.float_int.lists[i]); @@ -4396,11 +4404,16 @@ int kvm_s390_try_set_tod_clock(struct kvm *kvm, con= st struct kvm_s390_vm_tod_clo return 1; } =20 -static void __kvm_inject_pfault_token(struct kvm_vcpu *vcpu, bool start_to= ken, - unsigned long token) +static int __kvm_inject_pfault_token(struct kvm_vcpu *vcpu, bool start_tok= en, + unsigned long token) { struct kvm_s390_interrupt inti; struct kvm_s390_irq irq; + struct kvm_s390_interrupt_info *inti_mem; + + inti_mem =3D kzalloc_obj(*inti_mem, GFP_KERNEL_ACCOUNT); + if (!inti_mem) + return -ENOMEM; =20 if (start_token) { irq.u.ext.ext_params2 =3D token; @@ -4409,8 +4422,9 @@ static void __kvm_inject_pfault_token(struct kvm_vcpu= *vcpu, bool start_token, } else { inti.type =3D KVM_S390_INT_PFAULT_DONE; inti.parm64 =3D token; - WARN_ON_ONCE(kvm_s390_inject_vm(vcpu->kvm, &inti)); + WARN_ON_ONCE(kvm_s390_inject_vm(vcpu->kvm, &inti, inti_mem)); } + return true; } =20 bool kvm_arch_async_page_not_present(struct kvm_vcpu *vcpu, diff --git a/arch/s390/kvm/kvm-s390.h b/arch/s390/kvm/kvm-s390.h index bf1d7798c1af..2f2da868a040 100644 --- a/arch/s390/kvm/kvm-s390.h +++ b/arch/s390/kvm/kvm-s390.h @@ -373,7 +373,8 @@ int __must_check kvm_s390_deliver_pending_interrupts(st= ruct kvm_vcpu *vcpu); void kvm_s390_clear_local_irqs(struct kvm_vcpu *vcpu); void kvm_s390_clear_float_irqs(struct kvm *kvm); int __must_check kvm_s390_inject_vm(struct kvm *kvm, - struct kvm_s390_interrupt *s390int); + struct kvm_s390_interrupt *s390int, + struct kvm_s390_interrupt_info *inti); int __must_check kvm_s390_inject_vcpu(struct kvm_vcpu *vcpu, struct kvm_s390_irq *irq); static inline int kvm_s390_inject_prog_irq(struct kvm_vcpu *vcpu, --=20 2.52.0