From nobody Mon Feb 9 08:13:03 2026 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1521149654172779.5727360180713; Thu, 15 Mar 2018 14:34:14 -0700 (PDT) Received: from localhost ([::1]:53464 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ewaVp-0000DH-2k for importer@patchew.org; Thu, 15 Mar 2018 17:34:13 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:55455) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1ewaTo-0007ar-Jd for qemu-devel@nongnu.org; Thu, 15 Mar 2018 17:32:10 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1ewaTk-0001rs-KA for qemu-devel@nongnu.org; Thu, 15 Mar 2018 17:32:08 -0400 Received: from mx1.redhat.com ([209.132.183.28]:40284) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1ewaTk-0001qz-Bw for qemu-devel@nongnu.org; Thu, 15 Mar 2018 17:32:04 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 9B03A4DB14; Thu, 15 Mar 2018 21:32:03 +0000 (UTC) Received: from gimli.home (ovpn-117-203.phx2.redhat.com [10.3.117.203]) by smtp.corp.redhat.com (Postfix) with ESMTP id 5BEDA5D9CD; Thu, 15 Mar 2018 21:31:58 +0000 (UTC) From: Alex Williamson To: kvm@vger.kernel.org Date: Thu, 15 Mar 2018 15:31:58 -0600 Message-ID: <20180315213157.15150.86812.stgit@gimli.home> In-Reply-To: <20180315212634.15150.88094.stgit@gimli.home> References: <20180315212634.15150.88094.stgit@gimli.home> User-Agent: StGit/0.18-102-gdf9f MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.29]); Thu, 15 Mar 2018 21:32:03 +0000 (UTC) X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 209.132.183.28 Subject: [Qemu-devel] [PATCH v2 3/3] vfio/pci: Add ioeventfd support X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: aik@ozlabs.ru, linux-kernel@vger.kernel.org, peterx@redhat.com, qemu-devel@nongnu.org, eric.auger@redhat.com, alex.williamson@redhat.com Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 The ioeventfd here is actually irqfd handling of an ioeventfd such as supported in KVM. A user is able to pre-program a device write to occur when the eventfd triggers. This is yet another instance of eventfd-irqfd triggering between KVM and vfio. The impetus for this is high frequency writes to pages which are virtualized in QEMU. Enabling this near-direct write path for selected registers within the virtualized page can improve performance and reduce overhead. Specifically this is initially targeted at NVIDIA graphics cards where the driver issues a write to an MMIO register within a virtualized region in order to allow the MSI interrupt to re-trigger. Signed-off-by: Alex Williamson Reviewed-by: Peter Xu --- drivers/vfio/pci/vfio_pci.c | 35 +++++++++++ drivers/vfio/pci/vfio_pci_private.h | 19 ++++++ drivers/vfio/pci/vfio_pci_rdwr.c | 111 +++++++++++++++++++++++++++++++= ++++ include/uapi/linux/vfio.h | 27 +++++++++ 4 files changed, 192 insertions(+) diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c index b0f759476900..c6822149b394 100644 --- a/drivers/vfio/pci/vfio_pci.c +++ b/drivers/vfio/pci/vfio_pci.c @@ -305,6 +305,7 @@ static void vfio_pci_disable(struct vfio_pci_device *vd= ev) { struct pci_dev *pdev =3D vdev->pdev; struct vfio_pci_dummy_resource *dummy_res, *tmp; + struct vfio_pci_ioeventfd *ioeventfd, *ioeventfd_tmp; int i, bar; =20 /* Stop the device from further DMA */ @@ -314,6 +315,15 @@ static void vfio_pci_disable(struct vfio_pci_device *v= dev) VFIO_IRQ_SET_ACTION_TRIGGER, vdev->irq_type, 0, 0, NULL); =20 + /* Device closed, don't need mutex here */ + list_for_each_entry_safe(ioeventfd, ioeventfd_tmp, + &vdev->ioeventfds_list, next) { + vfio_virqfd_disable(&ioeventfd->virqfd); + list_del(&ioeventfd->next); + kfree(ioeventfd); + } + vdev->ioeventfds_nr =3D 0; + vdev->virq_disabled =3D false; =20 for (i =3D 0; i < vdev->num_regions; i++) @@ -1012,6 +1022,28 @@ static long vfio_pci_ioctl(void *device_data, =20 kfree(groups); return ret; + } else if (cmd =3D=3D VFIO_DEVICE_IOEVENTFD) { + struct vfio_device_ioeventfd ioeventfd; + int count; + + minsz =3D offsetofend(struct vfio_device_ioeventfd, fd); + + if (copy_from_user(&ioeventfd, (void __user *)arg, minsz)) + return -EFAULT; + + if (ioeventfd.argsz < minsz) + return -EINVAL; + + if (ioeventfd.flags & ~VFIO_DEVICE_IOEVENTFD_SIZE_MASK) + return -EINVAL; + + count =3D ioeventfd.flags & VFIO_DEVICE_IOEVENTFD_SIZE_MASK; + + if (hweight8(count) !=3D 1 || ioeventfd.fd < -1) + return -EINVAL; + + return vfio_pci_ioeventfd(vdev, ioeventfd.offset, + ioeventfd.data, count, ioeventfd.fd); } =20 return -ENOTTY; @@ -1174,6 +1206,8 @@ static int vfio_pci_probe(struct pci_dev *pdev, const= struct pci_device_id *id) vdev->irq_type =3D VFIO_PCI_NUM_IRQS; mutex_init(&vdev->igate); spin_lock_init(&vdev->irqlock); + mutex_init(&vdev->ioeventfds_lock); + INIT_LIST_HEAD(&vdev->ioeventfds_list); =20 ret =3D vfio_add_group_dev(&pdev->dev, &vfio_pci_ops, vdev); if (ret) { @@ -1215,6 +1249,7 @@ static void vfio_pci_remove(struct pci_dev *pdev) =20 vfio_iommu_group_put(pdev->dev.iommu_group, &pdev->dev); kfree(vdev->region); + mutex_destroy(&vdev->ioeventfds_lock); kfree(vdev); =20 if (vfio_pci_is_vga(pdev)) { diff --git a/drivers/vfio/pci/vfio_pci_private.h b/drivers/vfio/pci/vfio_pc= i_private.h index f561ac1c78a0..cde3b5d3441a 100644 --- a/drivers/vfio/pci/vfio_pci_private.h +++ b/drivers/vfio/pci/vfio_pci_private.h @@ -29,6 +29,19 @@ #define PCI_CAP_ID_INVALID 0xFF /* default raw access */ #define PCI_CAP_ID_INVALID_VIRT 0xFE /* default virt access */ =20 +/* Cap maximum number of ioeventfds per device (arbitrary) */ +#define VFIO_PCI_IOEVENTFD_MAX 1000 + +struct vfio_pci_ioeventfd { + struct list_head next; + struct virqfd *virqfd; + void __iomem *addr; + uint64_t data; + loff_t pos; + int bar; + int count; +}; + struct vfio_pci_irq_ctx { struct eventfd_ctx *trigger; struct virqfd *unmask; @@ -92,9 +105,12 @@ struct vfio_pci_device { bool nointx; struct pci_saved_state *pci_saved_state; int refcnt; + int ioeventfds_nr; struct eventfd_ctx *err_trigger; struct eventfd_ctx *req_trigger; struct list_head dummy_resources_list; + struct mutex ioeventfds_lock; + struct list_head ioeventfds_list; }; =20 #define is_intx(vdev) (vdev->irq_type =3D=3D VFIO_PCI_INTX_IRQ_INDEX) @@ -120,6 +136,9 @@ extern ssize_t vfio_pci_bar_rw(struct vfio_pci_device *= vdev, char __user *buf, extern ssize_t vfio_pci_vga_rw(struct vfio_pci_device *vdev, char __user *= buf, size_t count, loff_t *ppos, bool iswrite); =20 +extern long vfio_pci_ioeventfd(struct vfio_pci_device *vdev, loff_t offset, + uint64_t data, int count, int fd); + extern int vfio_pci_init_perm_bits(void); extern void vfio_pci_uninit_perm_bits(void); =20 diff --git a/drivers/vfio/pci/vfio_pci_rdwr.c b/drivers/vfio/pci/vfio_pci_r= dwr.c index 925419e0f459..a6029d0a5524 100644 --- a/drivers/vfio/pci/vfio_pci_rdwr.c +++ b/drivers/vfio/pci/vfio_pci_rdwr.c @@ -17,6 +17,7 @@ #include #include #include +#include #include =20 #include "vfio_pci_private.h" @@ -275,3 +276,113 @@ ssize_t vfio_pci_vga_rw(struct vfio_pci_device *vdev,= char __user *buf, =20 return done; } + +static int vfio_pci_ioeventfd_handler(void *opaque, void *unused) +{ + struct vfio_pci_ioeventfd *ioeventfd =3D opaque; + + switch (ioeventfd->count) { + case 1: + vfio_iowrite8(ioeventfd->data, ioeventfd->addr); + break; + case 2: + vfio_iowrite16(ioeventfd->data, ioeventfd->addr); + break; + case 4: + vfio_iowrite32(ioeventfd->data, ioeventfd->addr); + break; +#ifdef iowrite64 + case 8: + vfio_iowrite64(ioeventfd->data, ioeventfd->addr); + break; +#endif + } + + return 0; +} + +long vfio_pci_ioeventfd(struct vfio_pci_device *vdev, loff_t offset, + uint64_t data, int count, int fd) +{ + struct pci_dev *pdev =3D vdev->pdev; + loff_t pos =3D offset & VFIO_PCI_OFFSET_MASK; + int ret, bar =3D VFIO_PCI_OFFSET_TO_INDEX(offset); + struct vfio_pci_ioeventfd *ioeventfd; + + /* Only support ioeventfds into BARs */ + if (bar > VFIO_PCI_BAR5_REGION_INDEX) + return -EINVAL; + + if (pos + count > pci_resource_len(pdev, bar)) + return -EINVAL; + + /* Disallow ioeventfds working around MSI-X table writes */ + if (bar =3D=3D vdev->msix_bar && + !(pos + count <=3D vdev->msix_offset || + pos >=3D vdev->msix_offset + vdev->msix_size)) + return -EINVAL; + +#ifndef iowrite64 + if (count =3D=3D 8) + return -EINVAL; +#endif + + ret =3D vfio_pci_setup_barmap(vdev, bar); + if (ret) + return ret; + + mutex_lock(&vdev->ioeventfds_lock); + + list_for_each_entry(ioeventfd, &vdev->ioeventfds_list, next) { + if (ioeventfd->pos =3D=3D pos && ioeventfd->bar =3D=3D bar && + ioeventfd->data =3D=3D data && ioeventfd->count =3D=3D count) { + if (fd =3D=3D -1) { + vfio_virqfd_disable(&ioeventfd->virqfd); + list_del(&ioeventfd->next); + vdev->ioeventfds_nr--; + kfree(ioeventfd); + ret =3D 0; + } else + ret =3D -EEXIST; + + goto out_unlock; + } + } + + if (fd < 0) { + ret =3D -ENODEV; + goto out_unlock; + } + + if (vdev->ioeventfds_nr >=3D VFIO_PCI_IOEVENTFD_MAX) { + ret =3D -ENOSPC; + goto out_unlock; + } + + ioeventfd =3D kzalloc(sizeof(*ioeventfd), GFP_KERNEL); + if (!ioeventfd) { + ret =3D -ENOMEM; + goto out_unlock; + } + + ioeventfd->addr =3D vdev->barmap[bar] + pos; + ioeventfd->data =3D data; + ioeventfd->pos =3D pos; + ioeventfd->bar =3D bar; + ioeventfd->count =3D count; + + ret =3D vfio_virqfd_enable(ioeventfd, vfio_pci_ioeventfd_handler, + NULL, NULL, &ioeventfd->virqfd, fd); + if (ret) { + kfree(ioeventfd); + goto out_unlock; + } + + list_add(&ioeventfd->next, &vdev->ioeventfds_list); + vdev->ioeventfds_nr++; + +out_unlock: + mutex_unlock(&vdev->ioeventfds_lock); + + return ret; +} diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h index c74372163ed2..1aa7b82e8169 100644 --- a/include/uapi/linux/vfio.h +++ b/include/uapi/linux/vfio.h @@ -575,6 +575,33 @@ struct vfio_device_gfx_plane_info { =20 #define VFIO_DEVICE_GET_GFX_DMABUF _IO(VFIO_TYPE, VFIO_BASE + 15) =20 +/** + * VFIO_DEVICE_IOEVENTFD - _IOW(VFIO_TYPE, VFIO_BASE + 16, + * struct vfio_device_ioeventfd) + * + * Perform a write to the device at the specified device fd offset, with + * the specified data and width when the provided eventfd is triggered. + * vfio bus drivers may not support this for all regions, for all widths, + * or at all. vfio-pci currently only enables support for BAR regions, + * excluding the MSI-X vector table. + * + * Return: 0 on success, -errno on failure. + */ +struct vfio_device_ioeventfd { + __u32 argsz; + __u32 flags; +#define VFIO_DEVICE_IOEVENTFD_8 (1 << 0) /* 1-byte write */ +#define VFIO_DEVICE_IOEVENTFD_16 (1 << 1) /* 2-byte write */ +#define VFIO_DEVICE_IOEVENTFD_32 (1 << 2) /* 4-byte write */ +#define VFIO_DEVICE_IOEVENTFD_64 (1 << 3) /* 8-byte write */ +#define VFIO_DEVICE_IOEVENTFD_SIZE_MASK (0xf) + __u64 offset; /* device fd offset of write */ + __u64 data; /* data to be written */ + __s32 fd; /* -1 for de-assignment */ +}; + +#define VFIO_DEVICE_IOEVENTFD _IO(VFIO_TYPE, VFIO_BASE + 16) + /* -------- API for Type1 VFIO IOMMU -------- */ =20 /**