From nobody Thu Apr 2 20:28:23 2026 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8E72523E342; Wed, 25 Mar 2026 13:31:51 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=148.163.158.5 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774445513; cv=none; b=Javbid3/52ip2ul6ANRgMyCl10zgv7+3cghgcEEbyN+eV5saXZhwVj0270m91lWXKpcMP2tLFTQA70Cau2kDms3nSAwKNeko/8VDRuIXsrWLmm+EIxw+KqhhOjZkYj3m7jGHG1xV4mBvavolkjhpjXNUOohPF2fzNfsLr2BW5s4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1774445513; c=relaxed/simple; bh=e/WHK9eQHqJppvXVumAc5cape4zt0+8UGLMbgKfUplA=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=EqU76GwoO1HvBOAu7Y1vnppCiDtxEuOxiOCuErhP7BCd+SKLohAbvDekIF5L+wgvYMS49IDU4CSVeL/K0UEZfg66ypxX7WPq5AgTImx+lCtPfGThMSVIaWRNHcU+hLX1hCbhXQ4JOINWsA9U8VJ5hqIpPUKmqx0K3kHSXK+b9dw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com; spf=pass smtp.mailfrom=linux.ibm.com; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b=kMtWHNIQ; arc=none smtp.client-ip=148.163.158.5 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.ibm.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ibm.com header.i=@ibm.com header.b="kMtWHNIQ" Received: from pps.filterd (m0360072.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 62OLJvAG2477909; Wed, 25 Mar 2026 13:31:46 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s=pp1; bh=mi/XJW +UuxDWdQ3UFZILQbyAK8t0jctZSDE0vjfPAdI=; b=kMtWHNIQX5zr4xU93jD6JG gi+e8Nk6QxtS5pXD4Z5JS+aN5D5NHs/NNnxKVg9r7MtnB94uz2bVsRo2a7RW2t1m XNC63JSa/1vDhTjvRo+3voZej4mqi3fub7avmqSuhfklgRLMjzZoqjwwKSDAbf19 pqUo5u8lMVBGDVheNQlEPeoOz6u8sMg4y2icxI9SHaasI5SxLDLsGXwWMnAoUCFB Df52CjDHGRTbxTyyoeQQUE4DZ46WHHHZeC5EiF69S0kgsfIUvkrihJi9xhUASU64 6YAn5nRri+LjRQoR48x0RxUfeNH1NzbHMvQJ8WxGI699i6rZDXMMsw+Zu2uH6+Kg == Received: from ppma21.wdc07v.mail.ibm.com (5b.69.3da9.ip4.static.sl-reverse.com [169.61.105.91]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 4d1kumquff-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Mar 2026 13:31:46 +0000 (GMT) Received: from pps.filterd (ppma21.wdc07v.mail.ibm.com [127.0.0.1]) by ppma21.wdc07v.mail.ibm.com (8.18.1.2/8.18.1.2) with ESMTP id 62PAoUX7008732; Wed, 25 Mar 2026 13:31:46 GMT Received: from smtprelay02.fra02v.mail.ibm.com ([9.218.2.226]) by ppma21.wdc07v.mail.ibm.com (PPS) with ESMTPS id 4d26nnpq86-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 25 Mar 2026 13:31:45 +0000 Received: from smtpav01.fra02v.mail.ibm.com (smtpav01.fra02v.mail.ibm.com [10.20.54.100]) by smtprelay02.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 62PDVgEF51904894 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 25 Mar 2026 13:31:42 GMT Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0B7182004E; Wed, 25 Mar 2026 13:31:42 +0000 (GMT) Received: from smtpav01.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 9BE9B2004B; Wed, 25 Mar 2026 13:31:41 +0000 (GMT) Received: from tuxmaker.boeblingen.de.ibm.com (unknown [9.87.85.9]) by smtpav01.fra02v.mail.ibm.com (Postfix) with ESMTP; Wed, 25 Mar 2026 13:31:41 +0000 (GMT) From: Julian Ruess Date: Wed, 25 Mar 2026 14:31:24 +0100 Subject: [PATCH v8 2/3] vfio/ism: Implement vfio_pci driver for ISM devices Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <20260325-vfio_pci_ism-v8-2-ddc504cde914@linux.ibm.com> References: <20260325-vfio_pci_ism-v8-0-ddc504cde914@linux.ibm.com> In-Reply-To: <20260325-vfio_pci_ism-v8-0-ddc504cde914@linux.ibm.com> To: schnelle@linux.ibm.com, wintera@linux.ibm.com, ts@linux.ibm.com, oberpar@linux.ibm.com, gbayer@linux.ibm.com, Alex Williamson , Jason Gunthorpe , Yishai Hadas , Shameer Kolothum , Kevin Tian Cc: mjrosato@linux.ibm.com, alifm@linux.ibm.com, raspl@linux.ibm.com, hca@linux.ibm.com, agordeev@linux.ibm.com, gor@linux.ibm.com, julianr@linux.ibm.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-s390@vger.kernel.org, linux-pci@vger.kernel.org X-Mailer: b4 0.14.2 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: r3iXgog0fNFNFWffXWS7fGfnQjfkarHs X-Proofpoint-ORIG-GUID: r3iXgog0fNFNFWffXWS7fGfnQjfkarHs X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwMzI1MDA5NSBTYWx0ZWRfX3/SMhv93HUEe pCKzYxUbUW/+q6H9EbVSpSjJMRBN3DKU1lWIkq9/OQwz4U03n11UeFmf7FbppaWQUr8ANQ1/+nL /2rg6JBGjYhX3KEto3DLI6bFwlIppVDWxiEPlFwLkBi528amBB/UIxJFKO6LOHB1mgGaWKTY8cS 7hvQ7eTtTeF7M+vo7Rb3CjrKzwk6xdSc8uMHOgRZklyNgngYa+aOBQQYiLXBpk5LkRxGYHGd1zf JGveD61nonCzgu2M7PqkcGDp7LZ1hETNOhbwfOOEQXllesRCjlYnrV6GEUlhk1Btw9Q+qpOF9jk 4fKwboYGfKrRl5KCd45kKOp7lQ92idjs8aqZZW7YRxuCm1TJxl5Wc9hPiKk1cG9u4EyfybnAdVz KNRtD7nYoX4GRw5eMgS+fQOrM2T1GH9uXqzHQkgH30Pff3pDWF0axfbj5kgJcd9RTQct8NWlk+J S2LruIkpmMfCf/Mb0Gw== X-Authority-Analysis: v=2.4 cv=KbXfcAYD c=1 sm=1 tr=0 ts=69c3e3c2 cx=c_pps a=GFwsV6G8L6GxiO2Y/PsHdQ==:117 a=GFwsV6G8L6GxiO2Y/PsHdQ==:17 a=IkcTkHD0fZMA:10 a=Yq5XynenixoA:10 a=VkNPw1HP01LnGYTKEx00:22 a=RnoormkPH1_aCDwRdu11:22 a=RzCfie-kr_QcCd8fBx8p:22 a=VnNF1IyMAAAA:8 a=XdfKsd1DaXcHZAhJ8BgA:9 a=QEXdDO2ut3YA:10 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.51,FMLib:17.12.100.49 definitions=2026-03-25_04,2026-03-24_01,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 suspectscore=0 impostorscore=0 malwarescore=0 adultscore=0 clxscore=1015 priorityscore=1501 bulkscore=0 lowpriorityscore=0 phishscore=0 spamscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2603050001 definitions=main-2603250095 Add a vfio_pci variant driver for the s390-specific Internal Shared Memory (ISM) devices used for inter-VM communication. This enables the development of vfio-pci-based user space drivers for ISM devices. On s390, kernel primitives such as ioread() and iowrite() are switched over from function-handle-based PCI load/stores instructions to PCI memory-I/O (MIO) loads/stores when these are available and not explicitly disabled. Since these instructions cannot be used with ISM devices, ensure that classic function-handle-based PCI instructions are used instead. The driver is still required even when MIO instructions are disabled, as the ISM device relies on the PCI store block (PCISTB) instruction to perform write operations. Stores are not fragmented, therefore one ioctl corresponds to exactly one PCISTB instruction. User space must ensure to not write more than 4096 bytes at once to an ISM BAR which is the maximum payload of the PCISTB instruction. Reviewed-by: Alexandra Winter Reviewed-by: Niklas Schnelle Signed-off-by: Julian Ruess Reviewed-by: Farhan Ali --- drivers/vfio/pci/Kconfig | 2 + drivers/vfio/pci/Makefile | 2 + drivers/vfio/pci/ism/Kconfig | 10 ++ drivers/vfio/pci/ism/Makefile | 3 + drivers/vfio/pci/ism/main.c | 408 ++++++++++++++++++++++++++++++++++++++= ++++ 5 files changed, 425 insertions(+) diff --git a/drivers/vfio/pci/Kconfig b/drivers/vfio/pci/Kconfig index 1e82b44bda1a0a544e1add7f4b06edecf35aaf81..296bf01e185ecacc388ebc69e92= 706c99e47c814 100644 --- a/drivers/vfio/pci/Kconfig +++ b/drivers/vfio/pci/Kconfig @@ -60,6 +60,8 @@ config VFIO_PCI_DMABUF =20 source "drivers/vfio/pci/mlx5/Kconfig" =20 +source "drivers/vfio/pci/ism/Kconfig" + source "drivers/vfio/pci/hisilicon/Kconfig" =20 source "drivers/vfio/pci/pds/Kconfig" diff --git a/drivers/vfio/pci/Makefile b/drivers/vfio/pci/Makefile index e0a0757dd1d2b0bc69b7e4d79441d5cacf4e1cd8..6138f1bf241df04e7419f196b40= 4abdf9b194050 100644 --- a/drivers/vfio/pci/Makefile +++ b/drivers/vfio/pci/Makefile @@ -11,6 +11,8 @@ obj-$(CONFIG_VFIO_PCI) +=3D vfio-pci.o =20 obj-$(CONFIG_MLX5_VFIO_PCI) +=3D mlx5/ =20 +obj-$(CONFIG_ISM_VFIO_PCI) +=3D ism/ + obj-$(CONFIG_HISI_ACC_VFIO_PCI) +=3D hisilicon/ =20 obj-$(CONFIG_PDS_VFIO_PCI) +=3D pds/ diff --git a/drivers/vfio/pci/ism/Kconfig b/drivers/vfio/pci/ism/Kconfig new file mode 100644 index 0000000000000000000000000000000000000000..02f47d25fed2d34c732b67b3a36= 55b64a7625467 --- /dev/null +++ b/drivers/vfio/pci/ism/Kconfig @@ -0,0 +1,10 @@ +# SPDX-License-Identifier: GPL-2.0 +config ISM_VFIO_PCI + tristate "VFIO support for ISM devices" + depends on S390 + select VFIO_PCI_CORE + help + This provides user space support for IBM Internal Shared Memory (ISM) + Adapter devices using the VFIO framework. + + If you don't know what to do here, say N. diff --git a/drivers/vfio/pci/ism/Makefile b/drivers/vfio/pci/ism/Makefile new file mode 100644 index 0000000000000000000000000000000000000000..32cc3c66dd11395da85a2b6f05b= 3d97036ed8a35 --- /dev/null +++ b/drivers/vfio/pci/ism/Makefile @@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0 +obj-$(CONFIG_ISM_VFIO_PCI) +=3D ism-vfio-pci.o +ism-vfio-pci-y :=3D main.o diff --git a/drivers/vfio/pci/ism/main.c b/drivers/vfio/pci/ism/main.c new file mode 100644 index 0000000000000000000000000000000000000000..00bc81f7225f806eac1b99c4520= ab5a68137885e --- /dev/null +++ b/drivers/vfio/pci/ism/main.c @@ -0,0 +1,408 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * vfio-ISM driver for s390 + * + * Copyright IBM Corp. + */ + +#include "../vfio_pci_priv.h" +#include "linux/slab.h" + +#define ISM_VFIO_PCI_OFFSET_SHIFT 48 +#define ISM_VFIO_PCI_OFFSET_TO_INDEX(off) ((off) >> ISM_VFIO_PCI_OFFSET_SH= IFT) +#define ISM_VFIO_PCI_INDEX_TO_OFFSET(index) ((u64)(index) << ISM_VFIO_PCI_= OFFSET_SHIFT) +#define ISM_VFIO_PCI_OFFSET_MASK (((u64)(1) << ISM_VFIO_PCI_OFFSET_SHIFT) = - 1) + +/* + * Use __zpci_load() to bypass automatic use of + * PCI MIO instructions which are not supported on ISM devices + */ +#define ISM_READ(size) = \ + static int ism_read##size(struct zpci_dev *zdev, int bar, \ + size_t *filled, char __user *buf, \ + loff_t off) \ + { \ + u64 req, tmp; \ + u##size val; \ + int ret; \ + \ + req =3D ZPCI_CREATE_REQ(READ_ONCE(zdev->fh), bar, sizeof(val)); \ + ret =3D __zpci_load(&tmp, req, off); \ + if (ret) \ + return ret; \ + val =3D (u##size)tmp; \ + if (copy_to_user(buf, &val, sizeof(val))) \ + return -EFAULT; \ + *filled =3D sizeof(val); \ + return 0; \ + } + +ISM_READ(64); +ISM_READ(32); +ISM_READ(16); +ISM_READ(8); + +struct ism_vfio_pci_core_device { + struct vfio_pci_core_device core_device; + struct kmem_cache *store_block_cache; +}; + +static int ism_vfio_pci_open_device(struct vfio_device *core_vdev) +{ + struct ism_vfio_pci_core_device *ivpcd; + struct vfio_pci_core_device *vdev; + int ret; + + ivpcd =3D container_of(core_vdev, struct ism_vfio_pci_core_device, + core_device.vdev); + vdev =3D &ivpcd->core_device; + + ret =3D vfio_pci_core_enable(vdev); + if (ret) + return ret; + + vfio_pci_core_finish_enable(vdev); + return 0; +} + +/* + * ism_vfio_pci_do_io_r() + * + * On s390, kernel primitives such as ioread() and iowrite() are switched = over + * from function-handle-based PCI load/stores instructions to PCI memory-I= /O (MIO) + * loads/stores when these are available and not explicitly disabled. Sinc= e these + * instructions cannot be used with ISM devices, ensure that classic + * function-handle-based PCI instructions are used instead. + */ +static ssize_t ism_vfio_pci_do_io_r(struct vfio_pci_core_device *vdev, + char __user *buf, loff_t off, size_t count, + int bar) +{ + struct zpci_dev *zdev =3D to_zpci(vdev->pdev); + ssize_t done =3D 0; + int ret; + + while (count) { + size_t filled; + + if (count >=3D 8 && IS_ALIGNED(off, 8)) { + ret =3D ism_read64(zdev, bar, &filled, buf, off); + if (ret) + return ret; + } else if (count >=3D 4 && IS_ALIGNED(off, 4)) { + ret =3D ism_read32(zdev, bar, &filled, buf, off); + if (ret) + return ret; + } else if (count >=3D 2 && IS_ALIGNED(off, 2)) { + ret =3D ism_read16(zdev, bar, &filled, buf, off); + if (ret) + return ret; + } else { + ret =3D ism_read8(zdev, bar, &filled, buf, off); + if (ret) + return ret; + } + + count -=3D filled; + done +=3D filled; + off +=3D filled; + buf +=3D filled; + } + + return done; +} + +/* + * ism_vfio_pci_do_io_w() + * + * Ensure that the PCI store block (PCISTB) instruction is used as require= d by the + * ISM device. The ISM device also uses a 256 TiB BAR 0 for write operatio= ns, + * which requires a 48bit region address space (ISM_VFIO_PCI_OFFSET_SHIFT). + */ +static ssize_t ism_vfio_pci_do_io_w(struct vfio_pci_core_device *vdev, + char __user *buf, loff_t off, size_t count, + int bar) +{ + struct zpci_dev *zdev =3D to_zpci(vdev->pdev); + struct ism_vfio_pci_core_device *ivpcd; + ssize_t ret; + void *data; + u64 req; + + if (count > zdev->maxstbl) + return -EINVAL; + if (((off % PAGE_SIZE) + count) > PAGE_SIZE) + return -EINVAL; + + ivpcd =3D container_of(vdev, struct ism_vfio_pci_core_device, + core_device); + data =3D kmem_cache_alloc(ivpcd->store_block_cache, GFP_KERNEL); + if (!data) + return -ENOMEM; + + if (copy_from_user(data, buf, count)) { + ret =3D -EFAULT; + goto out_free; + } + + req =3D ZPCI_CREATE_REQ(READ_ONCE(zdev->fh), bar, count); + ret =3D __zpci_store_block(data, req, off); + if (ret) + goto out_free; + + ret =3D count; + +out_free: + kmem_cache_free(ivpcd->store_block_cache, data); + return ret; +} + +static ssize_t ism_vfio_pci_bar_rw(struct vfio_pci_core_device *vdev, + char __user *buf, size_t count, loff_t *ppos, + bool iswrite) +{ + int bar =3D ISM_VFIO_PCI_OFFSET_TO_INDEX(*ppos); + loff_t pos =3D *ppos & ISM_VFIO_PCI_OFFSET_MASK; + resource_size_t end; + ssize_t done =3D 0; + + if (pci_resource_start(vdev->pdev, bar)) + end =3D pci_resource_len(vdev->pdev, bar); + else + return -EINVAL; + + if (pos >=3D end) + return -EINVAL; + + count =3D min(count, (size_t)(end - pos)); + + if (iswrite) + done =3D ism_vfio_pci_do_io_w(vdev, buf, pos, count, bar); + else + done =3D ism_vfio_pci_do_io_r(vdev, buf, pos, count, bar); + + if (done >=3D 0) + *ppos +=3D done; + + return done; +} + +static ssize_t ism_vfio_pci_config_rw(struct vfio_pci_core_device *vdev, + char __user *buf, size_t count, + loff_t *ppos, bool iswrite) +{ + loff_t pos =3D *ppos; + size_t done =3D 0; + int ret =3D 0; + + pos &=3D ISM_VFIO_PCI_OFFSET_MASK; + + while (count) { + /* + * zPCI must not use MIO instructions for config space access, + * so we can use common code path here. + */ + ret =3D vfio_pci_config_rw_single(vdev, buf, count, &pos, iswrite); + if (ret < 0) + return ret; + + count -=3D ret; + done +=3D ret; + buf +=3D ret; + pos +=3D ret; + } + + *ppos +=3D done; + + return done; +} + +static ssize_t ism_vfio_pci_rw(struct vfio_device *core_vdev, char __user = *buf, + size_t count, loff_t *ppos, bool iswrite) +{ + unsigned int index =3D ISM_VFIO_PCI_OFFSET_TO_INDEX(*ppos); + struct vfio_pci_core_device *vdev; + int ret; + + vdev =3D container_of(core_vdev, struct vfio_pci_core_device, vdev); + + if (!count) + return 0; + + switch (index) { + case VFIO_PCI_CONFIG_REGION_INDEX: + ret =3D ism_vfio_pci_config_rw(vdev, buf, count, ppos, iswrite); + break; + + case VFIO_PCI_BAR0_REGION_INDEX ... VFIO_PCI_BAR5_REGION_INDEX: + ret =3D ism_vfio_pci_bar_rw(vdev, buf, count, ppos, iswrite); + break; + + default: + return -EINVAL; + } + + return ret; +} + +static ssize_t ism_vfio_pci_read(struct vfio_device *core_vdev, + char __user *buf, size_t count, loff_t *ppos) +{ + return ism_vfio_pci_rw(core_vdev, buf, count, ppos, false); +} + +static ssize_t ism_vfio_pci_write(struct vfio_device *core_vdev, + const char __user *buf, size_t count, + loff_t *ppos) +{ + return ism_vfio_pci_rw(core_vdev, (char __user *)buf, count, ppos, + true); +} + +static int ism_vfio_pci_ioctl_get_region_info(struct vfio_device *core_vde= v, + struct vfio_region_info *info, + struct vfio_info_cap *caps) +{ + struct vfio_pci_core_device *vdev =3D + container_of(core_vdev, struct vfio_pci_core_device, vdev); + struct pci_dev *pdev =3D vdev->pdev; + + switch (info->index) { + case VFIO_PCI_CONFIG_REGION_INDEX: + info->offset =3D ISM_VFIO_PCI_INDEX_TO_OFFSET(info->index); + info->size =3D pdev->cfg_size; + info->flags =3D VFIO_REGION_INFO_FLAG_READ | + VFIO_REGION_INFO_FLAG_WRITE; + break; + case VFIO_PCI_BAR0_REGION_INDEX ... VFIO_PCI_BAR5_REGION_INDEX: + info->offset =3D ISM_VFIO_PCI_INDEX_TO_OFFSET(info->index); + info->size =3D pci_resource_len(pdev, info->index); + if (!info->size) { + info->flags =3D 0; + break; + } + info->flags =3D VFIO_REGION_INFO_FLAG_READ | + VFIO_REGION_INFO_FLAG_WRITE; + break; + default: + info->offset =3D 0; + info->size =3D 0; + info->flags =3D 0; + return -EINVAL; + } + return 0; +} + +static int ism_vfio_pci_init_dev(struct vfio_device *core_vdev) +{ + struct zpci_dev *zdev =3D to_zpci(to_pci_dev(core_vdev->dev)); + struct ism_vfio_pci_core_device *ivpcd; + char cache_name[20]; + int ret; + + ivpcd =3D container_of(core_vdev, struct ism_vfio_pci_core_device, + core_device.vdev); + + snprintf(cache_name, sizeof(cache_name), "ism_sb_fid_%08x", zdev->fid); + + ivpcd->store_block_cache =3D + kmem_cache_create(cache_name, zdev->maxstbl, + (&(struct kmem_cache_args){ + .align =3D PAGE_SIZE, + .useroffset =3D 0, + .usersize =3D zdev->maxstbl, + }), + (SLAB_RECLAIM_ACCOUNT | SLAB_ACCOUNT)); + if (!ivpcd->store_block_cache) + return -ENOMEM; + + ret =3D vfio_pci_core_init_dev(core_vdev); + if (ret) + kmem_cache_destroy(ivpcd->store_block_cache); + + return ret; +} + +static void ism_vfio_pci_release_dev(struct vfio_device *core_vdev) +{ + struct ism_vfio_pci_core_device *ivpcd =3D container_of( + core_vdev, struct ism_vfio_pci_core_device, core_device.vdev); + + kmem_cache_destroy(ivpcd->store_block_cache); + vfio_pci_core_release_dev(core_vdev); +} + +static const struct vfio_device_ops ism_pci_ops =3D { + .name =3D "ism-vfio-pci", + .init =3D ism_vfio_pci_init_dev, + .release =3D ism_vfio_pci_release_dev, + .open_device =3D ism_vfio_pci_open_device, + .close_device =3D vfio_pci_core_close_device, + .ioctl =3D vfio_pci_core_ioctl, + .get_region_info_caps =3D ism_vfio_pci_ioctl_get_region_info, + .device_feature =3D vfio_pci_core_ioctl_feature, + .read =3D ism_vfio_pci_read, + .write =3D ism_vfio_pci_write, + .request =3D vfio_pci_core_request, + .match =3D vfio_pci_core_match, + .match_token_uuid =3D vfio_pci_core_match_token_uuid, + .bind_iommufd =3D vfio_iommufd_physical_bind, + .unbind_iommufd =3D vfio_iommufd_physical_unbind, + .attach_ioas =3D vfio_iommufd_physical_attach_ioas, + .detach_ioas =3D vfio_iommufd_physical_detach_ioas, +}; + +static int ism_vfio_pci_probe(struct pci_dev *pdev, + const struct pci_device_id *id) +{ + struct ism_vfio_pci_core_device *ivpcd; + int ret; + + ivpcd =3D vfio_alloc_device(ism_vfio_pci_core_device, core_device.vdev, + &pdev->dev, &ism_pci_ops); + if (IS_ERR(ivpcd)) + return PTR_ERR(ivpcd); + + dev_set_drvdata(&pdev->dev, &ivpcd->core_device); + + ret =3D vfio_pci_core_register_device(&ivpcd->core_device); + if (ret) + vfio_put_device(&ivpcd->core_device.vdev); + + return ret; +} + +static void ism_vfio_pci_remove(struct pci_dev *pdev) +{ + struct vfio_pci_core_device *core_device; + struct ism_vfio_pci_core_device *ivpcd; + + core_device =3D dev_get_drvdata(&pdev->dev); + ivpcd =3D container_of(core_device, struct ism_vfio_pci_core_device, + core_device); + + vfio_pci_core_unregister_device(&ivpcd->core_device); + vfio_put_device(&ivpcd->core_device.vdev); +} + +static const struct pci_device_id ism_device_table[] =3D { + { PCI_DRIVER_OVERRIDE_DEVICE_VFIO(PCI_VENDOR_ID_IBM, + PCI_DEVICE_ID_IBM_ISM) }, + {} +}; +MODULE_DEVICE_TABLE(pci, ism_device_table); + +static struct pci_driver ism_vfio_pci_driver =3D { + .name =3D KBUILD_MODNAME, + .id_table =3D ism_device_table, + .probe =3D ism_vfio_pci_probe, + .remove =3D ism_vfio_pci_remove, + .err_handler =3D &vfio_pci_core_err_handlers, + .driver_managed_dma =3D true, +}; + +module_pci_driver(ism_vfio_pci_driver); + +MODULE_LICENSE("GPL"); +MODULE_DESCRIPTION("vfio-pci variant driver for the IBM Internal Shared Me= mory (ISM) device"); +MODULE_AUTHOR("IBM Corporation"); --=20 2.51.0