From nobody Wed May 1 22:32:46 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507182794629652.7574614626241; Wed, 4 Oct 2017 22:53:14 -0700 (PDT) Received: from localhost ([::1]:37908 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dzz5X-0003vC-3c for importer@patchew.org; Thu, 05 Oct 2017 01:52:51 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35601) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dzz42-0002ps-0m for qemu-devel@nongnu.org; Thu, 05 Oct 2017 01:51:19 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dzz3z-0006Um-DJ for qemu-devel@nongnu.org; Thu, 05 Oct 2017 01:51:18 -0400 Received: from ozlabs.ru ([107.173.13.209]:58876) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dzz3z-0006U5-6i; Thu, 05 Oct 2017 01:51:15 -0400 Received: from vpl1.ozlabs.ibm.com (localhost [IPv6:::1]) by ozlabs.ru (Postfix) with ESMTP id 649823A600A4; Thu, 5 Oct 2017 01:49:26 -0400 (EDT) From: Alexey Kardashevskiy To: qemu-devel@nongnu.org Date: Thu, 5 Oct 2017 16:50:36 +1100 Message-Id: <20171005055037.7767-2-aik@ozlabs.ru> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20171005055037.7767-1-aik@ozlabs.ru> References: <20171005055037.7767-1-aik@ozlabs.ru> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 107.173.13.209 Subject: [Qemu-devel] [PATCH qemu v5 1/2] memory/iommu/vfio: Define add_vfio_group() callback X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexey Kardashevskiy , Paolo Bonzini , Alex Williamson , qemu-ppc@nongnu.org, David Gibson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" The new callback will be called when a new VFIO IOMMU group is added. This should cause no behavioral change, the next patch will. Signed-off-by: Alexey Kardashevskiy --- This could be at the higher level - in MemoryRegionOps, makes sense? --- include/exec/memory.h | 4 ++++ hw/vfio/common.c | 15 +++++++++++++++ 2 files changed, 19 insertions(+) diff --git a/include/exec/memory.h b/include/exec/memory.h index 5ed4042f87..64e0b4fc96 100644 --- a/include/exec/memory.h +++ b/include/exec/memory.h @@ -210,6 +210,10 @@ typedef struct IOMMUMemoryRegionClass { IOMMUNotifierFlag new_flags); /* Set this up to provide customized IOMMU replay function */ void (*replay)(IOMMUMemoryRegion *iommu, IOMMUNotifier *notifier); + + /* Notifies IOMMUMR about attached VFIO IOMMU group */ + void (*add_vfio_group)(IOMMUMemoryRegion *iommu_mr, int vfio_kvm_fd, + int groupfd); } IOMMUMemoryRegionClass; =20 typedef struct CoalescedMemoryRange CoalescedMemoryRange; diff --git a/hw/vfio/common.c b/hw/vfio/common.c index 7b2924c0ef..9e861e0393 100644 --- a/hw/vfio/common.c +++ b/hw/vfio/common.c @@ -481,6 +481,21 @@ static void vfio_listener_region_add(MemoryListener *l= istener, VFIOGuestIOMMU *giommu; IOMMUMemoryRegion *iommu_mr =3D IOMMU_MEMORY_REGION(section->mr); =20 +#ifdef CONFIG_KVM + if (kvm_enabled()) { + VFIOGroup *group; + IOMMUMemoryRegionClass *imrc =3D + IOMMU_MEMORY_REGION_GET_CLASS(iommu_mr); + + QLIST_FOREACH(group, &container->group_list, container_next) { + if (imrc->add_vfio_group) { + imrc->add_vfio_group(iommu_mr, vfio_kvm_device_fd, + group->fd); + } + } + } +#endif + trace_vfio_listener_region_add_iommu(iova, end); /* * FIXME: For VFIO iommu types which have KVM acceleration to --=20 2.11.0 From nobody Wed May 1 22:32:46 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) client-ip=208.118.235.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Authentication-Results: mx.zohomail.com; spf=pass (zoho.com: domain of gnu.org designates 208.118.235.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org Return-Path: Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) by mx.zohomail.com with SMTPS id 1507182795158679.0836371687179; Wed, 4 Oct 2017 22:53:15 -0700 (PDT) Received: from localhost ([::1]:37909 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dzz5b-0003xF-F6 for importer@patchew.org; Thu, 05 Oct 2017 01:52:55 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:35611) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dzz43-0002pu-9c for qemu-devel@nongnu.org; Thu, 05 Oct 2017 01:51:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dzz41-0006Ys-D7 for qemu-devel@nongnu.org; Thu, 05 Oct 2017 01:51:19 -0400 Received: from ozlabs.ru ([107.173.13.209]:58884) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dzz41-0006YB-4Q; Thu, 05 Oct 2017 01:51:17 -0400 Received: from vpl1.ozlabs.ibm.com (localhost [IPv6:::1]) by ozlabs.ru (Postfix) with ESMTP id 86CAA3A600A5; Thu, 5 Oct 2017 01:49:28 -0400 (EDT) From: Alexey Kardashevskiy To: qemu-devel@nongnu.org Date: Thu, 5 Oct 2017 16:50:37 +1100 Message-Id: <20171005055037.7767-3-aik@ozlabs.ru> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20171005055037.7767-1-aik@ozlabs.ru> References: <20171005055037.7767-1-aik@ozlabs.ru> X-detected-operating-system: by eggs.gnu.org: GNU/Linux 3.x [fuzzy] X-Received-From: 107.173.13.209 Subject: [Qemu-devel] [PATCH qemu v5 2/2] spapr/iommu: Enable in-kernel TCE acceleration via VFIO KVM device X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexey Kardashevskiy , Paolo Bonzini , Alex Williamson , qemu-ppc@nongnu.org, David Gibson Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail: RSF_0 Z_629925259 SPT_0 Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" This implements a notification for a new IOMMU group attached to sPAPR's logical IO bus (LIOBN) to enable in-kernel TCE acceleration. This uses new kernel KVM_CAP_SPAPR_TCE_VFIO capability to enable in-kernel acceleration of TCE update requests which will go via the VFIO KVM device. Signed-off-by: Alexey Kardashevskiy --- target/ppc/kvm_ppc.h | 6 ++++++ hw/ppc/spapr_iommu.c | 34 ++++++++++++++++++++++++++++++++++ target/ppc/kvm.c | 7 ++++++- hw/ppc/trace-events | 1 + 4 files changed, 47 insertions(+), 1 deletion(-) diff --git a/target/ppc/kvm_ppc.h b/target/ppc/kvm_ppc.h index d6be38ecaf..2b985e1659 100644 --- a/target/ppc/kvm_ppc.h +++ b/target/ppc/kvm_ppc.h @@ -48,6 +48,7 @@ void *kvmppc_create_spapr_tce(uint32_t liobn, uint32_t pa= ge_shift, int kvmppc_remove_spapr_tce(void *table, int pfd, uint32_t window_size); int kvmppc_reset_htab(int shift_hint); uint64_t kvmppc_rma_size(uint64_t current_size, unsigned int hash_shift); +bool kvmppc_has_cap_spapr_vfio(void); #endif /* !CONFIG_USER_ONLY */ bool kvmppc_has_cap_epr(void); int kvmppc_define_rtas_kernel_token(uint32_t token, const char *function); @@ -231,6 +232,11 @@ static inline bool kvmppc_is_mem_backend_page_size_ok(= const char *obj_path) return true; } =20 +static inline bool kvmppc_has_cap_spapr_vfio(void) +{ + return false; +} + #endif /* !CONFIG_USER_ONLY */ =20 static inline bool kvmppc_has_cap_epr(void) diff --git a/hw/ppc/spapr_iommu.c b/hw/ppc/spapr_iommu.c index 5ccd785d5a..34baa881f0 100644 --- a/hw/ppc/spapr_iommu.c +++ b/hw/ppc/spapr_iommu.c @@ -17,6 +17,7 @@ * License along with this library; if not, see . */ #include "qemu/osdep.h" +#include #include "qemu/error-report.h" #include "hw/hw.h" #include "qemu/log.h" @@ -173,6 +174,34 @@ static void spapr_tce_notify_flag_changed(IOMMUMemoryR= egion *iommu, } } =20 +static void spapr_tce_add_vfio_group(IOMMUMemoryRegion *iommu_mr, + int vfio_kvm_fd, int groupfd) +{ + sPAPRTCETable *tcet =3D container_of(iommu_mr, sPAPRTCETable, iommu); + struct kvm_vfio_spapr_tce param =3D { + .tablefd =3D tcet->fd, + .groupfd =3D groupfd, + }; + struct kvm_device_attr attr =3D { + .group =3D KVM_DEV_VFIO_GROUP, + .attr =3D KVM_DEV_VFIO_GROUP_SET_SPAPR_TCE, + .addr =3D (uint64_t)(unsigned long)¶m, + }; + + if (!kvmppc_has_cap_spapr_vfio()) { + return; + } + + if (param.tablefd !=3D -1) { + if (ioctl(vfio_kvm_fd, KVM_SET_DEVICE_ATTR, &attr)) { + error_report("vfio: failed to setup fd %d for a group with fd = %d: %s", + param.tablefd, param.groupfd, strerror(errno)); + return; + } + } + trace_spapr_iommu_add_vfio_group(groupfd, param.tablefd); +} + static int spapr_tce_table_post_load(void *opaque, int version_id) { sPAPRTCETable *tcet =3D SPAPR_TCE_TABLE(opaque); @@ -284,6 +313,10 @@ void spapr_tce_set_need_vfio(sPAPRTCETable *tcet, bool= need_vfio) =20 tcet->need_vfio =3D need_vfio; =20 + if (!need_vfio || (tcet->fd !=3D -1 && kvmppc_has_cap_spapr_vfio())) { + return; + } + oldtable =3D tcet->table; =20 tcet->table =3D spapr_tce_alloc_table(tcet->liobn, @@ -643,6 +676,7 @@ static void spapr_iommu_memory_region_class_init(Object= Class *klass, void *data) imrc->translate =3D spapr_tce_translate_iommu; imrc->get_min_page_size =3D spapr_tce_get_min_page_size; imrc->notify_flag_changed =3D spapr_tce_notify_flag_changed; + imrc->add_vfio_group =3D spapr_tce_add_vfio_group; } =20 static const TypeInfo spapr_iommu_memory_region_info =3D { diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c index 171d3d8040..5438252bdc 100644 --- a/target/ppc/kvm.c +++ b/target/ppc/kvm.c @@ -136,7 +136,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s) cap_spapr_tce =3D kvm_check_extension(s, KVM_CAP_SPAPR_TCE); cap_spapr_tce_64 =3D kvm_check_extension(s, KVM_CAP_SPAPR_TCE_64); cap_spapr_multitce =3D kvm_check_extension(s, KVM_CAP_SPAPR_MULTITCE); - cap_spapr_vfio =3D false; + cap_spapr_vfio =3D kvm_vm_check_extension(s, KVM_CAP_SPAPR_TCE_VFIO); cap_one_reg =3D kvm_check_extension(s, KVM_CAP_ONE_REG); cap_hior =3D kvm_check_extension(s, KVM_CAP_PPC_HIOR); cap_epr =3D kvm_check_extension(s, KVM_CAP_PPC_EPR); @@ -2474,6 +2474,11 @@ bool kvmppc_has_cap_mmu_hash_v3(void) return cap_mmu_hash_v3; } =20 +bool kvmppc_has_cap_spapr_vfio(void) +{ + return cap_spapr_vfio; +} + PowerPCCPUClass *kvm_ppc_get_host_cpu_class(void) { uint32_t host_pvr =3D mfpvr(); diff --git a/hw/ppc/trace-events b/hw/ppc/trace-events index 4a6a6490fa..ab70d00582 100644 --- a/hw/ppc/trace-events +++ b/hw/ppc/trace-events @@ -36,6 +36,7 @@ spapr_iommu_ddw_query(uint64_t buid, uint32_t cfgaddr, un= signed wa, uint64_t win spapr_iommu_ddw_create(uint64_t buid, uint32_t cfgaddr, uint64_t pg_size, = uint64_t req_size, uint64_t start, uint32_t liobn) "buid=3D0x%"PRIx64" addr= =3D0x%"PRIx32", page size=3D0x%"PRIx64", requested=3D0x%"PRIx64", start add= r=3D0x%"PRIx64", liobn=3D0x%"PRIx32 spapr_iommu_ddw_remove(uint32_t liobn) "liobn=3D0x%"PRIx32 spapr_iommu_ddw_reset(uint64_t buid, uint32_t cfgaddr) "buid=3D0x%"PRIx64"= addr=3D0x%"PRIx32 +spapr_iommu_add_vfio_group(int groupfd, int tablefd) "Attached groupfd %d = to liobn fd %d" =20 # hw/ppc/spapr_drc.c spapr_drc_set_isolation_state(uint32_t index, int state) "drc: 0x%"PRIx32"= , state: 0x%"PRIx32 --=20 2.11.0