From nobody Wed Nov 13 05:08:32 2024 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 621F2C88C85 for ; Fri, 9 Jun 2023 08:53:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239467AbjFIIxU (ORCPT ); Fri, 9 Jun 2023 04:53:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52766 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S239270AbjFIIw0 (ORCPT ); Fri, 9 Jun 2023 04:52:26 -0400 Received: from mailgw02.mediatek.com (unknown [210.61.82.184]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id BFCBAB9; Fri, 9 Jun 2023 01:52:22 -0700 (PDT) X-UUID: ef81181a06a211eeb20a276fd37b9834-20230609 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=mediatek.com; s=dk; h=Content-Type:MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:CC:To:From; bh=deGZ6KSF2b7R6KdcQE1hvYLqiF/G0uzgUaB/CCA4efE=; b=fYtt5rRanMCn0xL61yJvQ4bvOFpCXebqDUEzNxi0vu1oIot0D2SJMgp3InEtaZ47zLEbXaNlF6LU7u2ByYbLp1vHY/plu7yU6Pbiz83UOg3jlyB/kBj2819le2CLubIKOlQdieut7suaGQqUd30gzufoxEbAXhtpAu5hHjDYuQQ=; X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.26,REQID:f46f8526-4f64-467d-b665-5e32ea5871b3,IP:0,U RL:0,TC:0,Content:-25,EDM:0,RT:0,SF:47,FILE:0,BULK:0,RULE:Release_Ham,ACTI ON:release,TS:22 X-CID-INFO: VERSION:1.1.26,REQID:f46f8526-4f64-467d-b665-5e32ea5871b3,IP:0,URL :0,TC:0,Content:-25,EDM:0,RT:0,SF:47,FILE:0,BULK:0,RULE:Release_Ham,ACTION :release,TS:22 X-CID-META: VersionHash:cb9a4e1,CLOUDID:b8cd1b3e-7aa7-41f3-a6bd-0433bee822f3,B ulkID:23060916521772YWOAW5,BulkQuantity:0,Recheck:0,SF:101|38|29|28|100|17 |19|48|102,TC:nil,Content:0,EDM:-3,IP:nil,URL:11|1,File:nil,Bulk:nil,QS:ni l,BEC:nil,COL:0,OSI:0,OSA:0,AV:0,LES:1,SPR:NO X-CID-BVR: 0 X-CID-BAS: 0,_,0,_ X-CID-FACTOR: TF_CID_SPAM_SNR,TF_CID_SPAM_SDM,TF_CID_SPAM_FAS,TF_CID_SPAM_FSD, TF_CID_SPAM_ULN X-UUID: ef81181a06a211eeb20a276fd37b9834-20230609 Received: from mtkmbs13n1.mediatek.inc [(172.21.101.193)] by mailgw02.mediatek.com (envelope-from ) (Generic MTA with TLSv1.2 ECDHE-RSA-AES256-GCM-SHA384 256/256) with ESMTP id 1560273463; Fri, 09 Jun 2023 16:52:16 +0800 Received: from mtkmbs13n1.mediatek.inc (172.21.101.193) by mtkmbs11n1.mediatek.inc (172.21.101.185) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.26; Fri, 9 Jun 2023 16:52:16 +0800 Received: from mtksdccf07.mediatek.inc (172.21.84.99) by mtkmbs13n1.mediatek.inc (172.21.101.73) with Microsoft SMTP Server id 15.2.1118.26 via Frontend Transport; Fri, 9 Jun 2023 16:52:16 +0800 From: Yi-De Wu To: Yingshiuan Pan , Ze-Yu Wang , Yi-De Wu , Jonathan Corbet , Catalin Marinas , Will Deacon , Arnd Bergmann , Matthias Brugger , AngeloGioacchino Del Regno CC: , , , , , Krzysztof Kozlowski , Rob Herring , Conor Dooley , Conor Dooley , Trilok Soni , David Bradil , Jade Shih , Miles Chen , Ivan Tseng , My Chuang , Shawn Hsiao , PeiLun Suei , Liju Chen , Willix Yeh Subject: [PATCH v4 6/9] virt: geniezone: Add ioeventfd support Date: Fri, 9 Jun 2023 16:52:11 +0800 Message-ID: <20230609085214.31071-7-yi-de.wu@mediatek.com> X-Mailer: git-send-email 2.18.0 In-Reply-To: <20230609085214.31071-1-yi-de.wu@mediatek.com> References: <20230609085214.31071-1-yi-de.wu@mediatek.com> MIME-Version: 1.0 X-MTK: N Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" From: "Yingshiuan Pan" Ioeventfd leverages eventfd to provide asynchronous notification mechanism for VMM. VMM can register a mmio address and bind with an eventfd. Once a mmio trap occurs on this registered region, its corresponding eventfd will be notified. Signed-off-by: Yingshiuan Pan Signed-off-by: Liju Chen Signed-off-by: Yi-De Wu --- drivers/virt/geniezone/Makefile | 2 +- drivers/virt/geniezone/gzvm_ioeventfd.c | 263 ++++++++++++++++++++++++ drivers/virt/geniezone/gzvm_vcpu.c | 27 ++- drivers/virt/geniezone/gzvm_vm.c | 17 ++ include/linux/gzvm_drv.h | 11 + include/uapi/linux/gzvm.h | 23 +++ 6 files changed, 341 insertions(+), 2 deletions(-) create mode 100644 drivers/virt/geniezone/gzvm_ioeventfd.c diff --git a/drivers/virt/geniezone/Makefile b/drivers/virt/geniezone/Makef= ile index aa52cee3ca8e..25493a4d1c63 100644 --- a/drivers/virt/geniezone/Makefile +++ b/drivers/virt/geniezone/Makefile @@ -8,4 +8,4 @@ GZVM_DIR ?=3D ../../../drivers/virt/geniezone =20 gzvm-y :=3D $(GZVM_DIR)/gzvm_main.o $(GZVM_DIR)/gzvm_vm.o \ $(GZVM_DIR)/gzvm_vcpu.o $(GZVM_DIR)/gzvm_irqchip.o \ - $(GZVM_DIR)/gzvm_irqfd.o + $(GZVM_DIR)/gzvm_irqfd.o $(GZVM_DIR)/gzvm_ioeventfd.o diff --git a/drivers/virt/geniezone/gzvm_ioeventfd.c b/drivers/virt/geniezo= ne/gzvm_ioeventfd.c new file mode 100644 index 000000000000..f5664cab98c3 --- /dev/null +++ b/drivers/virt/geniezone/gzvm_ioeventfd.c @@ -0,0 +1,263 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2023 MediaTek Inc. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +struct gzvm_ioevent { + struct list_head list; + __u64 addr; + __u32 len; + struct eventfd_ctx *evt_ctx; + __u64 datamatch; + bool wildcard; +}; + +/** + * ioeventfd_check_collision() - Check collison assumes gzvm->slots_lock h= eld. + * @gzvm: Pointer to gzvm. + * @p: Pointer to gzvm_ioevent. + */ +static bool ioeventfd_check_collision(struct gzvm *gzvm, struct gzvm_ioeve= nt *p) +{ + struct gzvm_ioevent *_p; + + list_for_each_entry(_p, &gzvm->ioevents, list) + if (_p->addr =3D=3D p->addr && + (!_p->len || !p->len || + (_p->len =3D=3D p->len && + (_p->wildcard || p->wildcard || + _p->datamatch =3D=3D p->datamatch)))) + return true; + + return false; +} + +static void gzvm_ioevent_release(struct gzvm_ioevent *p) +{ + eventfd_ctx_put(p->evt_ctx); + list_del(&p->list); + kfree(p); +} + +static bool gzvm_ioevent_in_range(struct gzvm_ioevent *p, __u64 addr, int = len, + const void *val) +{ + u64 _val; + + if (addr !=3D p->addr) + /* address must be precise for a hit */ + return false; + + if (!p->len) + /* length =3D 0 means only look at the address, so always a hit */ + return true; + + if (len !=3D p->len) + /* address-range must be precise for a hit */ + return false; + + if (p->wildcard) + /* all else equal, wildcard is always a hit */ + return true; + + /* otherwise, we have to actually compare the data */ + + WARN_ON_ONCE(!IS_ALIGNED((unsigned long)val, len)); + + switch (len) { + case 1: + _val =3D *(u8 *)val; + break; + case 2: + _val =3D *(u16 *)val; + break; + case 4: + _val =3D *(u32 *)val; + break; + case 8: + _val =3D *(u64 *)val; + break; + default: + return false; + } + + return _val =3D=3D p->datamatch; +} + +static int gzvm_deassign_ioeventfd(struct gzvm *gzvm, + struct gzvm_ioeventfd *args) +{ + struct gzvm_ioevent *p, *tmp; + struct eventfd_ctx *evt_ctx; + int ret =3D -ENOENT; + bool wildcard; + + evt_ctx =3D eventfd_ctx_fdget(args->fd); + if (IS_ERR(evt_ctx)) + return PTR_ERR(evt_ctx); + + wildcard =3D !(args->flags & GZVM_IOEVENTFD_FLAG_DATAMATCH); + + mutex_lock(&gzvm->lock); + + list_for_each_entry_safe(p, tmp, &gzvm->ioevents, list) { + if (p->evt_ctx !=3D evt_ctx || + p->addr !=3D args->addr || + p->len !=3D args->len || + p->wildcard !=3D wildcard) + continue; + + if (!p->wildcard && p->datamatch !=3D args->datamatch) + continue; + + gzvm_ioevent_release(p); + ret =3D 0; + break; + } + + mutex_unlock(&gzvm->lock); + + /* got in the front of this function */ + eventfd_ctx_put(evt_ctx); + + return ret; +} + +static int gzvm_assign_ioeventfd(struct gzvm *gzvm, struct gzvm_ioeventfd = *args) +{ + struct eventfd_ctx *evt_ctx; + struct gzvm_ioevent *evt; + int ret; + + evt_ctx =3D eventfd_ctx_fdget(args->fd); + if (IS_ERR(evt_ctx)) + return PTR_ERR(evt_ctx); + + evt =3D kmalloc(sizeof(*evt), GFP_KERNEL); + if (!evt) + return -ENOMEM; + *evt =3D (struct gzvm_ioevent) { + .addr =3D args->addr, + .len =3D args->len, + .evt_ctx =3D evt_ctx, + }; + if (args->flags & GZVM_IOEVENTFD_FLAG_DATAMATCH) { + evt->datamatch =3D args->datamatch; + evt->wildcard =3D false; + } else { + evt->wildcard =3D true; + } + + if (ioeventfd_check_collision(gzvm, evt)) { + ret =3D -EEXIST; + goto err_free; + } + + mutex_lock(&gzvm->lock); + list_add_tail(&evt->list, &gzvm->ioevents); + mutex_unlock(&gzvm->lock); + + return 0; + +err_free: + kfree(evt); + eventfd_ctx_put(evt_ctx); + return ret; +} + +/** + * gzvm_ioeventfd_check_valid() - Check user arguments is valid. + * @args: Pointer to gzvm_ioeventfd. + * + * Return true if user arguments are valid. + * Return false if user arguments are invalid. + */ +static bool gzvm_ioeventfd_check_valid(struct gzvm_ioeventfd *args) +{ + /* must be natural-word sized, or 0 to ignore length */ + switch (args->len) { + case 0: + case 1: + case 2: + case 4: + case 8: + break; + default: + return false; + } + + /* check for range overflow */ + if (args->addr + args->len < args->addr) + return false; + + /* check for extra flags that we don't understand */ + if (args->flags & ~GZVM_IOEVENTFD_VALID_FLAG_MASK) + return false; + + /* ioeventfd with no length can't be combined with DATAMATCH */ + if (!args->len && (args->flags & GZVM_IOEVENTFD_FLAG_DATAMATCH)) + return false; + + /* gzvm does not support pio bus ioeventfd */ + if (args->flags & GZVM_IOEVENTFD_FLAG_PIO) + return false; + + return true; +} + +/** + * gzvm_ioeventfd() - Register ioevent to ioevent list. + * @gzvm: Pointer to gzvm. + * @args: Pointer to gzvm_ioeventfd. + */ +int gzvm_ioeventfd(struct gzvm *gzvm, struct gzvm_ioeventfd *args) +{ + if (gzvm_ioeventfd_check_valid(args) =3D=3D false) + return -EINVAL; + + if (args->flags & GZVM_IOEVENTFD_FLAG_DEASSIGN) + return gzvm_deassign_ioeventfd(gzvm, args); + return gzvm_assign_ioeventfd(gzvm, args); +} + +/** + * gzvm_ioevent_write() - Travers this vm's registered ioeventfd to see if + * need notifying it. + * @vcpu: Pointer to vcpu. + * @addr: mmio address. + * @len: mmio size. + * @val: Pointer to void. + * + * Return true if this io is already sent to ioeventfd's listener. + * Return false if we cannot find any ioeventfd registering this mmio writ= e. + */ +bool gzvm_ioevent_write(struct gzvm_vcpu *vcpu, __u64 addr, int len, + const void *val) +{ + struct gzvm_ioevent *e; + + list_for_each_entry(e, &vcpu->gzvm->ioevents, list) { + if (gzvm_ioevent_in_range(e, addr, len, val)) { + eventfd_signal(e->evt_ctx, 1); + return true; + } + } + return false; +} + +int gzvm_init_ioeventfd(struct gzvm *gzvm) +{ + INIT_LIST_HEAD(&gzvm->ioevents); + + return 0; +} diff --git a/drivers/virt/geniezone/gzvm_vcpu.c b/drivers/virt/geniezone/gz= vm_vcpu.c index 94ead15fea3f..40b57d85eaa2 100644 --- a/drivers/virt/geniezone/gzvm_vcpu.c +++ b/drivers/virt/geniezone/gzvm_vcpu.c @@ -49,6 +49,30 @@ static long gzvm_vcpu_update_one_reg(struct gzvm_vcpu *v= cpu, void * __user argp, return 0; } =20 +/** + * gzvm_vcpu_handle_mmio() - Handle mmio in kernel space. + * @vcpu: Pointer to vcpu. + * + * Return: + * * true - This mmio exit has been processed. + * * false - This mmio exit has not been processed, require userspace. + */ +static bool gzvm_vcpu_handle_mmio(struct gzvm_vcpu *vcpu) +{ + __u64 addr; + __u32 len; + const void *val_ptr; + + /* So far, we don't have in-kernel mmio read handler */ + if (!vcpu->run->mmio.is_write) + return false; + addr =3D vcpu->run->mmio.phys_addr; + len =3D vcpu->run->mmio.size; + val_ptr =3D &vcpu->run->mmio.data; + + return gzvm_ioevent_write(vcpu, addr, len, val_ptr); +} + /** * gzvm_vcpu_run() - Handle vcpu run ioctl, entry point to guest and exit * point from guest @@ -70,7 +94,8 @@ static long gzvm_vcpu_run(struct gzvm_vcpu *vcpu, void * = __user argp) =20 switch (exit_reason) { case GZVM_EXIT_MMIO: - need_userspace =3D true; + if (!gzvm_vcpu_handle_mmio(vcpu)) + need_userspace =3D true; break; /** * it's geniezone's responsibility to fill corresponding data diff --git a/drivers/virt/geniezone/gzvm_vm.c b/drivers/virt/geniezone/gzvm= _vm.c index eedc2d5c24ad..ba5412acfa7d 100644 --- a/drivers/virt/geniezone/gzvm_vm.c +++ b/drivers/virt/geniezone/gzvm_vm.c @@ -370,6 +370,15 @@ static long gzvm_vm_ioctl(struct file *filp, unsigned = int ioctl, ret =3D gzvm_irqfd(gzvm, &data); break; } + case GZVM_IOEVENTFD: { + struct gzvm_ioeventfd data; + + ret =3D -EFAULT; + if (copy_from_user(&data, argp, sizeof(data))) + goto out; + ret =3D gzvm_ioeventfd(gzvm, &data); + break; + } case GZVM_ENABLE_CAP: { struct gzvm_enable_cap cap; =20 @@ -448,6 +457,14 @@ static struct gzvm *gzvm_create_vm(unsigned long vm_ty= pe) return ERR_PTR(ret); } =20 + ret =3D gzvm_init_ioeventfd(gzvm); + if (ret) { + dev_err(gzvm_debug_dev->this_device, + "Failed to initialize ioeventfd\n"); + kfree(gzvm); + return ERR_PTR(ret); + } + mutex_lock(&gzvm_list_lock); list_add(&gzvm->vm_list, &gzvm_list); mutex_unlock(&gzvm_list_lock); diff --git a/include/linux/gzvm_drv.h b/include/linux/gzvm_drv.h index 54b0a3d443c5..f0a172fb0dd8 100644 --- a/include/linux/gzvm_drv.h +++ b/include/linux/gzvm_drv.h @@ -6,6 +6,7 @@ #ifndef __GZVM_DRV_H__ #define __GZVM_DRV_H__ =20 +#include #include #include #include @@ -92,6 +93,8 @@ struct gzvm { struct mutex resampler_lock; } irqfds; =20 + struct list_head ioevents; + struct list_head vm_list; gzvm_id_t vm_id; =20 @@ -142,4 +145,12 @@ void gzvm_sync_hwstate(struct gzvm_vcpu *vcpu); =20 extern struct miscdevice *gzvm_debug_dev; =20 +int gzvm_init_ioeventfd(struct gzvm *gzvm); +int gzvm_ioeventfd(struct gzvm *gzvm, struct gzvm_ioeventfd *args); +bool gzvm_ioevent_write(struct gzvm_vcpu *vcpu, __u64 addr, int len, + const void *val); +void eventfd_ctx_do_read(struct eventfd_ctx *ctx, __u64 *cnt); +struct vm_area_struct *vma_lookup(struct mm_struct *mm, unsigned long addr= ); +void add_wait_queue_priority(struct wait_queue_head *wq_head, struct wait_= queue_entry *wq_entry); + #endif /* __GZVM_DRV_H__ */ diff --git a/include/uapi/linux/gzvm.h b/include/uapi/linux/gzvm.h index 3f1e829f855d..2af1b068947c 100644 --- a/include/uapi/linux/gzvm.h +++ b/include/uapi/linux/gzvm.h @@ -242,4 +242,27 @@ struct gzvm_irqfd { =20 #define GZVM_IRQFD _IOW(GZVM_IOC_MAGIC, 0x76, struct gzvm_irqfd) =20 +enum { + gzvm_ioeventfd_flag_nr_datamatch, + gzvm_ioeventfd_flag_nr_pio, + gzvm_ioeventfd_flag_nr_deassign, + gzvm_ioeventfd_flag_nr_max, +}; + +#define GZVM_IOEVENTFD_FLAG_DATAMATCH (1 << gzvm_ioeventfd_flag_nr_datamat= ch) +#define GZVM_IOEVENTFD_FLAG_PIO (1 << gzvm_ioeventfd_flag_nr_pio) +#define GZVM_IOEVENTFD_FLAG_DEASSIGN (1 << gzvm_ioeventfd_flag_nr_deassign) +#define GZVM_IOEVENTFD_VALID_FLAG_MASK ((1 << gzvm_ioeventfd_flag_nr_max) = - 1) + +struct gzvm_ioeventfd { + __u64 datamatch; + __u64 addr; /* legal pio/mmio address */ + __u32 len; /* 1, 2, 4, or 8 bytes; or 0 to ignore length */ + __s32 fd; + __u32 flags; + __u8 pad[36]; +}; + +#define GZVM_IOEVENTFD _IOW(GZVM_IOC_MAGIC, 0x79, struct gzvm_ioeventfd) + #endif /* __GZVM_H__ */ --=20 2.18.0