From nobody Thu Sep 19 16:05:15 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1726152874; cv=none; d=zohomail.com; s=zohoarc; b=TuS4geO0Eb413HIFZcbFdObADZb9LVKsJ4Y4a8SG+38bwGufP/xihOkbPmXHFEuHt2I9+/3PPiPs+4pKcpnnd9J747b0AR8cmFnlYa9kl1i8lbIQUdT3y62rd2oVi8xCzZfNrVauYaFY5oLEz8lS1pFvdfWeYrp0XSQGzC2EboU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1726152874; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=mdTcvG56M/0M/MW5T/zjnxeyEKF9XJOzplb4nYdwrAs=; b=NYkDSgOlreEhBXEsY252+fV7TLJ0qqVFqynEZkUcF3bOkhEf4hNT7gzfCRnYY6pViMfJsLQtEwVWr2GxwEW95OVLgai19RHYt3x1GpUwQzY2eyEY9MB2KpGD7knK/zONfqm11+lLpGww9E7yTCVCUafoR/B9KX753LMjcETjq9Y= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1726152874383879.15701605344; Thu, 12 Sep 2024 07:54:34 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1solCx-0000wG-4n; Thu, 12 Sep 2024 10:54:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1solCv-0000v9-5f for qemu-devel@nongnu.org; Thu, 12 Sep 2024 10:54:06 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1solCt-0000HN-0r for qemu-devel@nongnu.org; Thu, 12 Sep 2024 10:54:04 -0400 Received: from mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-212-IA0xUzprM0iNRtSXy49k9A-1; Thu, 12 Sep 2024 10:53:57 -0400 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-02.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 4E6921953941; Thu, 12 Sep 2024 14:53:56 +0000 (UTC) Received: from localhost.localdomain (unknown [10.45.224.195]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id C59C819560A3; Thu, 12 Sep 2024 14:53:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1726152841; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mdTcvG56M/0M/MW5T/zjnxeyEKF9XJOzplb4nYdwrAs=; b=gZCXkvdJuNaTfHBxsF4x2u+VRV/MExQamNT/7UXR+Qiv+J+21PokOmGsubGeZIC0SKGKZo 9aeUBtTR1ak1VI/FnRstNODFt0W2c77Ct816AiG9rfPqH5Sl1T2GXqs+uON22aSSDk1hRX ZEYMKp4XnQ5nC6S7TlybxHDLX247kBs= X-MC-Unique: IA0xUzprM0iNRtSXy49k9A-1 From: Albert Esteve To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , slp@redhat.com, hi@alyssa.is, mst@redhat.com, david@redhat.com, jasowang@redhat.com, stefanha@redhat.com, Stefano Garzarella , stevensd@chromium.org, Albert Esteve Subject: [PATCH v3 1/5] vhost-user: Add VIRTIO Shared Memory map request Date: Thu, 12 Sep 2024 16:53:31 +0200 Message-ID: <20240912145335.129447-2-aesteve@redhat.com> In-Reply-To: <20240912145335.129447-1-aesteve@redhat.com> References: <20240912145335.129447-1-aesteve@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=aesteve@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.143, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1726152876425116600 Content-Type: text/plain; charset="utf-8" Add SHMEM_MAP/UNMAP requests to vhost-user to handle VIRTIO Shared Memory mappings. This request allows backends to dynamically map fds into a VIRTIO Shared Memory Region indentified by its `shmid`. Then, the fd memory is advertised to the driver as a base addres + offset, so it can be read/written (depending on the mmap flags requested) while its valid. The backend can munmap the memory range in a given VIRTIO Shared Memory Region (again, identified by its `shmid`), to free it. Upon receiving this message, the front-end must mmap the regions with PROT_NONE to reserve the virtual memory space. The device model needs to create MemoryRegion instances for the VIRTIO Shared Memory Regions and add them to the `VirtIODevice` instance. Signed-off-by: Albert Esteve --- hw/virtio/vhost-user.c | 122 ++++++++++++++++++++++ hw/virtio/virtio.c | 13 +++ include/hw/virtio/virtio.h | 5 + subprojects/libvhost-user/libvhost-user.c | 60 +++++++++++ subprojects/libvhost-user/libvhost-user.h | 52 +++++++++ 5 files changed, 252 insertions(+) diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index 00561daa06..338cc942ec 100644 --- a/hw/virtio/vhost-user.c +++ b/hw/virtio/vhost-user.c @@ -115,6 +115,8 @@ typedef enum VhostUserBackendRequest { VHOST_USER_BACKEND_SHARED_OBJECT_ADD =3D 6, VHOST_USER_BACKEND_SHARED_OBJECT_REMOVE =3D 7, VHOST_USER_BACKEND_SHARED_OBJECT_LOOKUP =3D 8, + VHOST_USER_BACKEND_SHMEM_MAP =3D 9, + VHOST_USER_BACKEND_SHMEM_UNMAP =3D 10, VHOST_USER_BACKEND_MAX } VhostUserBackendRequest; =20 @@ -192,6 +194,24 @@ typedef struct VhostUserShared { unsigned char uuid[16]; } VhostUserShared; =20 +/* For the flags field of VhostUserMMap */ +#define VHOST_USER_FLAG_MAP_R (1u << 0) +#define VHOST_USER_FLAG_MAP_W (1u << 1) + +typedef struct { + /* VIRTIO Shared Memory Region ID */ + uint8_t shmid; + uint8_t padding[7]; + /* File offset */ + uint64_t fd_offset; + /* Offset within the VIRTIO Shared Memory Region */ + uint64_t shm_offset; + /* Size of the mapping */ + uint64_t len; + /* Flags for the mmap operation, from VHOST_USER_FLAG_* */ + uint64_t flags; +} VhostUserMMap; + typedef struct { VhostUserRequest request; =20 @@ -224,6 +244,7 @@ typedef union { VhostUserInflight inflight; VhostUserShared object; VhostUserTransferDeviceState transfer_state; + VhostUserMMap mmap; } VhostUserPayload; =20 typedef struct VhostUserMsg { @@ -1749,6 +1770,100 @@ vhost_user_backend_handle_shared_object_lookup(stru= ct vhost_user *u, return 0; } =20 +static int +vhost_user_backend_handle_shmem_map(struct vhost_dev *dev, + VhostUserMMap *vu_mmap, + int fd) +{ + void *addr =3D 0; + MemoryRegion *mr =3D NULL; + + if (fd < 0) { + error_report("Bad fd for map"); + return -EBADF; + } + + if (!dev->vdev->shmem_list || + dev->vdev->n_shmem_regions <=3D vu_mmap->shmid) { + error_report("Device only has %d VIRTIO Shared Memory Regions. " + "Requested ID: %d", + dev->vdev->n_shmem_regions, vu_mmap->shmid); + return -EFAULT; + } + + mr =3D &dev->vdev->shmem_list[vu_mmap->shmid]; + + if (!mr) { + error_report("VIRTIO Shared Memory Region at " + "ID %d unitialized", vu_mmap->shmid); + return -EFAULT; + } + + if ((vu_mmap->shm_offset + vu_mmap->len) < vu_mmap->len || + (vu_mmap->shm_offset + vu_mmap->len) > mr->size) { + error_report("Bad offset/len for mmap %" PRIx64 "+%" PRIx64, + vu_mmap->shm_offset, vu_mmap->len); + return -EFAULT; + } + + void *shmem_ptr =3D memory_region_get_ram_ptr(mr); + + addr =3D mmap(shmem_ptr + vu_mmap->shm_offset, vu_mmap->len, + ((vu_mmap->flags & VHOST_USER_FLAG_MAP_R) ? PROT_READ : 0) | + ((vu_mmap->flags & VHOST_USER_FLAG_MAP_W) ? PROT_WRITE : 0), + MAP_SHARED | MAP_FIXED, fd, vu_mmap->fd_offset); + + if (addr =3D=3D MAP_FAILED) { + error_report("Failed to mmap mem fd"); + return -EFAULT; + } + + return 0; +} + +static int +vhost_user_backend_handle_shmem_unmap(struct vhost_dev *dev, + VhostUserMMap *vu_mmap) +{ + void *addr =3D 0; + MemoryRegion *mr =3D NULL; + + if (!dev->vdev->shmem_list || + dev->vdev->n_shmem_regions <=3D vu_mmap->shmid) { + error_report("Device only has %d VIRTIO Shared Memory Regions. " + "Requested ID: %d", + dev->vdev->n_shmem_regions, vu_mmap->shmid); + return -EFAULT; + } + + mr =3D &dev->vdev->shmem_list[vu_mmap->shmid]; + + if (!mr) { + error_report("VIRTIO Shared Memory Region at " + "ID %d unitialized", vu_mmap->shmid); + return -EFAULT; + } + + if ((vu_mmap->shm_offset + vu_mmap->len) < vu_mmap->len || + (vu_mmap->shm_offset + vu_mmap->len) > mr->size) { + error_report("Bad offset/len for mmap %" PRIx64 "+%" PRIx64, + vu_mmap->shm_offset, vu_mmap->len); + return -EFAULT; + } + + void *shmem_ptr =3D memory_region_get_ram_ptr(mr); + + addr =3D mmap(shmem_ptr + vu_mmap->shm_offset, vu_mmap->len, + PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS | MAP_FIXED, -1, 0); + + if (addr =3D=3D MAP_FAILED) { + error_report("Failed to unmap memory"); + return -EFAULT; + } + + return 0; +} + static void close_backend_channel(struct vhost_user *u) { g_source_destroy(u->backend_src); @@ -1817,6 +1932,13 @@ static gboolean backend_read(QIOChannel *ioc, GIOCon= dition condition, ret =3D vhost_user_backend_handle_shared_object_lookup(dev->opaque= , ioc, &hdr, &payloa= d); break; + case VHOST_USER_BACKEND_SHMEM_MAP: + ret =3D vhost_user_backend_handle_shmem_map(dev, &payload.mmap, + fd ? fd[0] : -1); + break; + case VHOST_USER_BACKEND_SHMEM_UNMAP: + ret =3D vhost_user_backend_handle_shmem_unmap(dev, &payload.mmap); + break; default: error_report("Received unexpected msg type: %d.", hdr.request); ret =3D -EINVAL; diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index 9e10cbc058..ccc4f2cd75 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -3059,6 +3059,17 @@ int virtio_save(VirtIODevice *vdev, QEMUFile *f) return vmstate_save_state(f, &vmstate_virtio, vdev, NULL); } =20 +MemoryRegion *virtio_new_shmem_region(VirtIODevice *vdev) +{ + MemoryRegion *mr; + ++vdev->n_shmem_regions; + vdev->shmem_list =3D g_renew(MemoryRegion, vdev->shmem_list, + vdev->n_shmem_regions); + mr =3D &vdev->shmem_list[vdev->n_shmem_regions - 1]; + mr =3D g_new0(MemoryRegion, 1); + return mr; +} + /* A wrapper for use as a VMState .put function */ static int virtio_device_put(QEMUFile *f, void *opaque, size_t size, const VMStateField *field, JSONWriter *vmdes= c) @@ -3481,6 +3492,8 @@ void virtio_init(VirtIODevice *vdev, uint16_t device_= id, size_t config_size) virtio_vmstate_change, vdev); vdev->device_endian =3D virtio_default_endian(); vdev->use_guest_notifier_mask =3D true; + vdev->shmem_list =3D NULL; + vdev->n_shmem_regions =3D 0; } =20 /* diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h index 0fcbc5c0c6..d4a2f664d9 100644 --- a/include/hw/virtio/virtio.h +++ b/include/hw/virtio/virtio.h @@ -167,6 +167,9 @@ struct VirtIODevice */ EventNotifier config_notifier; bool device_iotlb_enabled; + /* Shared memory region for vhost-user mappings. */ + MemoryRegion *shmem_list; + int n_shmem_regions; }; =20 struct VirtioDeviceClass { @@ -286,6 +289,8 @@ void virtio_notify(VirtIODevice *vdev, VirtQueue *vq); =20 int virtio_save(VirtIODevice *vdev, QEMUFile *f); =20 +MemoryRegion *virtio_new_shmem_region(VirtIODevice *vdev); + extern const VMStateInfo virtio_vmstate_info; =20 #define VMSTATE_VIRTIO_DEVICE \ diff --git a/subprojects/libvhost-user/libvhost-user.c b/subprojects/libvho= st-user/libvhost-user.c index 9c630c2170..496268e12b 100644 --- a/subprojects/libvhost-user/libvhost-user.c +++ b/subprojects/libvhost-user/libvhost-user.c @@ -1592,6 +1592,66 @@ vu_rm_shared_object(VuDev *dev, unsigned char uuid[U= UID_LEN]) return vu_send_message(dev, &msg); } =20 +bool +vu_shmem_map(VuDev *dev, uint8_t shmid, uint64_t fd_offset, + uint64_t shm_offset, uint64_t len, uint64_t flags) +{ + VhostUserMsg vmsg =3D { + .request =3D VHOST_USER_BACKEND_SHMEM_MAP, + .size =3D sizeof(vmsg.payload.mmap), + .flags =3D VHOST_USER_VERSION, + .payload.mmap =3D { + .shmid =3D shmid, + .fd_offset =3D fd_offset, + .shm_offset =3D shm_offset, + .len =3D len, + .flags =3D flags, + }, + }; + + if (vu_has_protocol_feature(dev, VHOST_USER_PROTOCOL_F_REPLY_ACK)) { + vmsg.flags |=3D VHOST_USER_NEED_REPLY_MASK; + } + + pthread_mutex_lock(&dev->backend_mutex); + if (!vu_message_write(dev, dev->backend_fd, &vmsg)) { + pthread_mutex_unlock(&dev->backend_mutex); + return false; + } + + /* Also unlocks the backend_mutex */ + return vu_process_message_reply(dev, &vmsg); +} + +bool +vu_shmem_unmap(VuDev *dev, uint8_t shmid, uint64_t shm_offset, uint64_t le= n) +{ + VhostUserMsg vmsg =3D { + .request =3D VHOST_USER_BACKEND_SHMEM_UNMAP, + .size =3D sizeof(vmsg.payload.mmap), + .flags =3D VHOST_USER_VERSION, + .payload.mmap =3D { + .shmid =3D shmid, + .fd_offset =3D 0, + .shm_offset =3D shm_offset, + .len =3D len, + }, + }; + + if (vu_has_protocol_feature(dev, VHOST_USER_PROTOCOL_F_REPLY_ACK)) { + vmsg.flags |=3D VHOST_USER_NEED_REPLY_MASK; + } + + pthread_mutex_lock(&dev->backend_mutex); + if (!vu_message_write(dev, dev->backend_fd, &vmsg)) { + pthread_mutex_unlock(&dev->backend_mutex); + return false; + } + + /* Also unlocks the backend_mutex */ + return vu_process_message_reply(dev, &vmsg); +} + static bool vu_set_vring_call_exec(VuDev *dev, VhostUserMsg *vmsg) { diff --git a/subprojects/libvhost-user/libvhost-user.h b/subprojects/libvho= st-user/libvhost-user.h index deb40e77b3..ea4902e876 100644 --- a/subprojects/libvhost-user/libvhost-user.h +++ b/subprojects/libvhost-user/libvhost-user.h @@ -127,6 +127,8 @@ typedef enum VhostUserBackendRequest { VHOST_USER_BACKEND_SHARED_OBJECT_ADD =3D 6, VHOST_USER_BACKEND_SHARED_OBJECT_REMOVE =3D 7, VHOST_USER_BACKEND_SHARED_OBJECT_LOOKUP =3D 8, + VHOST_USER_BACKEND_SHMEM_MAP =3D 9, + VHOST_USER_BACKEND_SHMEM_UNMAP =3D 10, VHOST_USER_BACKEND_MAX } VhostUserBackendRequest; =20 @@ -186,6 +188,24 @@ typedef struct VhostUserShared { unsigned char uuid[UUID_LEN]; } VhostUserShared; =20 +/* For the flags field of VhostUserMMap */ +#define VHOST_USER_FLAG_MAP_R (1u << 0) +#define VHOST_USER_FLAG_MAP_W (1u << 1) + +typedef struct { + /* VIRTIO Shared Memory Region ID */ + uint8_t shmid; + uint8_t padding[7]; + /* File offset */ + uint64_t fd_offset; + /* Offset within the VIRTIO Shared Memory Region */ + uint64_t shm_offset; + /* Size of the mapping */ + uint64_t len; + /* Flags for the mmap operation, from VHOST_USER_FLAG_* */ + uint64_t flags; +} VhostUserMMap; + #if defined(_WIN32) && (defined(__x86_64__) || defined(__i386__)) # define VU_PACKED __attribute__((gcc_struct, packed)) #else @@ -214,6 +234,7 @@ typedef struct VhostUserMsg { VhostUserVringArea area; VhostUserInflight inflight; VhostUserShared object; + VhostUserMMap mmap; } payload; =20 int fds[VHOST_MEMORY_BASELINE_NREGIONS]; @@ -597,6 +618,37 @@ bool vu_add_shared_object(VuDev *dev, unsigned char uu= id[UUID_LEN]); */ bool vu_rm_shared_object(VuDev *dev, unsigned char uuid[UUID_LEN]); =20 +/** + * vu_shmem_map: + * @dev: a VuDev context + * @shmid: VIRTIO Shared Memory Region ID + * @fd_offset: File offset + * @shm_offset: Offset within the VIRTIO Shared Memory Region + * @len: Size of the mapping + * @flags: Flags for the mmap operation + * + * Advertises a new mapping to be made in a given VIRTIO Shared Memory Reg= ion. + * + * Returns: TRUE on success, FALSE on failure. + */ +bool vu_shmem_map(VuDev *dev, uint8_t shmid, uint64_t fd_offset, + uint64_t shm_offset, uint64_t len, uint64_t flags); + +/** + * vu_shmem_map: + * @dev: a VuDev context + * @shmid: VIRTIO Shared Memory Region ID + * @fd_offset: File offset + * @len: Size of the mapping + * + * The front-end un-mmaps a given range in the VIRTIO Shared Memory Region + * with the requested `shmid`. + * + * Returns: TRUE on success, FALSE on failure. + */ +bool vu_shmem_unmap(VuDev *dev, uint8_t shmid, uint64_t shm_offset, + uint64_t len); + /** * vu_queue_set_notification: * @dev: a VuDev context --=20 2.45.2 From nobody Thu Sep 19 16:05:15 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1726152898; cv=none; d=zohomail.com; s=zohoarc; b=ic4Gq6OgHzIJ5STMTEOV2Fgs5ROGv0wD2er8LAfWK9oFSMQMcAbZ+NnlmZU54ffDxqPjoLPszW0PnegEX9aWO6X7SoMwn1xg0SfNkoQBPkpO7X6MkyXnuhs9AvW5hs8BcCfwaau7ySl8tn24BXzVmuF383P5DvXn2Zrjir01bxM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1726152898; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=xG4rr6587vwjGLOmp2PGDY77mR8V0+e64UiAhyqmV34=; b=AQwAwPc/Z8mBXj7bPH9f4FtEM4zbB1gBX+MM1DKlVPgLNr/C10x6SJMavuvQeBGFIwenpEjx8kp8G4XdeCBL1WObkNc8uAGtkN2Ydwoq2H0TQCZd748dlIyExGZqK4AErHv/lg11BUSjEdlXAy2pKmuo60w817vD9hgtDIofTHw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1726152898497763.0056796937076; Thu, 12 Sep 2024 07:54:58 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1solD1-00015V-VV; Thu, 12 Sep 2024 10:54:11 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1solD0-00014t-Ce for qemu-devel@nongnu.org; Thu, 12 Sep 2024 10:54:10 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1solCx-0000Ha-2y for qemu-devel@nongnu.org; Thu, 12 Sep 2024 10:54:09 -0400 Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-115-UJsyyt2hOBqVSeMHn1NgHg-1; Thu, 12 Sep 2024 10:54:02 -0400 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 38D601955D59; Thu, 12 Sep 2024 14:54:01 +0000 (UTC) Received: from localhost.localdomain (unknown [10.45.224.195]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id B63461956052; Thu, 12 Sep 2024 14:53:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1726152846; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xG4rr6587vwjGLOmp2PGDY77mR8V0+e64UiAhyqmV34=; b=CGs/kq5/2xmAVBE+qdRqWLVG8tdjRlC255Rzmoqh70w+t1+vGj111Ozbr7HJtKtwgK22mr WsCUdIcTs88QSSXql4vjBuG8LEyqsSGOd9beVaCi30AGsU0UVkypnhTnM6HlrYIgTkqIPV zttyj/p6zt7klVWEWRpZ9jOeTd7nK68= X-MC-Unique: UJsyyt2hOBqVSeMHn1NgHg-1 From: Albert Esteve To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , slp@redhat.com, hi@alyssa.is, mst@redhat.com, david@redhat.com, jasowang@redhat.com, stefanha@redhat.com, Stefano Garzarella , stevensd@chromium.org, Albert Esteve Subject: [PATCH v3 2/5] virtio: Track shared memory mappings Date: Thu, 12 Sep 2024 16:53:32 +0200 Message-ID: <20240912145335.129447-3-aesteve@redhat.com> In-Reply-To: <20240912145335.129447-1-aesteve@redhat.com> References: <20240912145335.129447-1-aesteve@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=aesteve@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.143, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1726152900510116600 Content-Type: text/plain; charset="utf-8" Update shmem_list to be able to track active mappings on VIRTIO shared memory regions. This allows to verify that new mapping request received from backends do not overlap. If they do, the request shall fail in order to adhere to the specs. Signed-off-by: Albert Esteve --- hw/virtio/vhost-user.c | 31 +++++++++++++------- hw/virtio/virtio.c | 58 ++++++++++++++++++++++++++++++++++---- include/hw/virtio/virtio.h | 25 ++++++++++++++-- 3 files changed, 96 insertions(+), 18 deletions(-) diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index 338cc942ec..de0bb35257 100644 --- a/hw/virtio/vhost-user.c +++ b/hw/virtio/vhost-user.c @@ -1776,7 +1776,7 @@ vhost_user_backend_handle_shmem_map(struct vhost_dev = *dev, int fd) { void *addr =3D 0; - MemoryRegion *mr =3D NULL; + VirtSharedMemory *shmem =3D NULL; =20 if (fd < 0) { error_report("Bad fd for map"); @@ -1791,22 +1791,29 @@ vhost_user_backend_handle_shmem_map(struct vhost_de= v *dev, return -EFAULT; } =20 - mr =3D &dev->vdev->shmem_list[vu_mmap->shmid]; + shmem =3D &dev->vdev->shmem_list[vu_mmap->shmid]; =20 - if (!mr) { + if (!shmem) { error_report("VIRTIO Shared Memory Region at " "ID %d unitialized", vu_mmap->shmid); return -EFAULT; } =20 if ((vu_mmap->shm_offset + vu_mmap->len) < vu_mmap->len || - (vu_mmap->shm_offset + vu_mmap->len) > mr->size) { + (vu_mmap->shm_offset + vu_mmap->len) > shmem->mr->size) { error_report("Bad offset/len for mmap %" PRIx64 "+%" PRIx64, vu_mmap->shm_offset, vu_mmap->len); return -EFAULT; } =20 - void *shmem_ptr =3D memory_region_get_ram_ptr(mr); + if (virtio_shmem_map_overlaps(shmem, vu_mmap->shm_offset, vu_mmap->len= )) { + error_report("Requested memory (%" PRIx64 "+%" PRIx64 ") overalps " + "with previously mapped memory", + vu_mmap->shm_offset, vu_mmap->len); + return -EFAULT; + } + + void *shmem_ptr =3D memory_region_get_ram_ptr(shmem->mr); =20 addr =3D mmap(shmem_ptr + vu_mmap->shm_offset, vu_mmap->len, ((vu_mmap->flags & VHOST_USER_FLAG_MAP_R) ? PROT_READ : 0) | @@ -1818,6 +1825,8 @@ vhost_user_backend_handle_shmem_map(struct vhost_dev = *dev, return -EFAULT; } =20 + virtio_add_shmem_map(shmem, vu_mmap->shm_offset, vu_mmap->len); + return 0; } =20 @@ -1826,7 +1835,7 @@ vhost_user_backend_handle_shmem_unmap(struct vhost_de= v *dev, VhostUserMMap *vu_mmap) { void *addr =3D 0; - MemoryRegion *mr =3D NULL; + VirtSharedMemory *shmem =3D NULL; =20 if (!dev->vdev->shmem_list || dev->vdev->n_shmem_regions <=3D vu_mmap->shmid) { @@ -1836,22 +1845,22 @@ vhost_user_backend_handle_shmem_unmap(struct vhost_= dev *dev, return -EFAULT; } =20 - mr =3D &dev->vdev->shmem_list[vu_mmap->shmid]; + shmem =3D &dev->vdev->shmem_list[vu_mmap->shmid]; =20 - if (!mr) { + if (!shmem) { error_report("VIRTIO Shared Memory Region at " "ID %d unitialized", vu_mmap->shmid); return -EFAULT; } =20 if ((vu_mmap->shm_offset + vu_mmap->len) < vu_mmap->len || - (vu_mmap->shm_offset + vu_mmap->len) > mr->size) { + (vu_mmap->shm_offset + vu_mmap->len) > shmem->mr->size) { error_report("Bad offset/len for mmap %" PRIx64 "+%" PRIx64, vu_mmap->shm_offset, vu_mmap->len); return -EFAULT; } =20 - void *shmem_ptr =3D memory_region_get_ram_ptr(mr); + void *shmem_ptr =3D memory_region_get_ram_ptr(shmem->mr); =20 addr =3D mmap(shmem_ptr + vu_mmap->shm_offset, vu_mmap->len, PROT_NONE, MAP_PRIVATE | MAP_ANONYMOUS | MAP_FIXED, -1, 0); @@ -1861,6 +1870,8 @@ vhost_user_backend_handle_shmem_unmap(struct vhost_de= v *dev, return -EFAULT; } =20 + virtio_del_shmem_map(shmem, vu_mmap->shm_offset, vu_mmap->len); + return 0; } =20 diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index ccc4f2cd75..0e2cd62a15 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -3059,15 +3059,52 @@ int virtio_save(VirtIODevice *vdev, QEMUFile *f) return vmstate_save_state(f, &vmstate_virtio, vdev, NULL); } =20 -MemoryRegion *virtio_new_shmem_region(VirtIODevice *vdev) +VirtSharedMemory *virtio_new_shmem_region(VirtIODevice *vdev) { - MemoryRegion *mr; + VirtSharedMemory *shmem =3D NULL; ++vdev->n_shmem_regions; - vdev->shmem_list =3D g_renew(MemoryRegion, vdev->shmem_list, + vdev->shmem_list =3D g_renew(VirtSharedMemory, vdev->shmem_list, vdev->n_shmem_regions); - mr =3D &vdev->shmem_list[vdev->n_shmem_regions - 1]; - mr =3D g_new0(MemoryRegion, 1); - return mr; + shmem =3D &vdev->shmem_list[vdev->n_shmem_regions - 1]; + shmem =3D g_new0(VirtSharedMemory, 1); + QTAILQ_INIT(&shmem->mapped_regions); + return shmem; +} + +void virtio_add_shmem_map(VirtSharedMemory *shmem, hwaddr offset, + uint64_t size) +{ + MappedMemoryRegion *mmap =3D g_new0(MappedMemoryRegion, 1); + mmap->offset =3D offset; + mmap->size =3D int128_make64(size); + QTAILQ_REMOVE(&shmem->mapped_regions, mmap, link); + g_free(mmap); +} + +void virtio_del_shmem_map(VirtSharedMemory *shmem, hwaddr offset, + uint64_t size) +{ + MappedMemoryRegion *mmap =3D g_new0(MappedMemoryRegion, 1); + mmap->offset =3D offset; + mmap->size =3D int128_make64(size); + QTAILQ_INSERT_TAIL(&shmem->mapped_regions, mmap, link); + g_free(mmap); +} + +bool virtio_shmem_map_overlaps(VirtSharedMemory *shmem, hwaddr offset, + uint64_t size) +{ + MappedMemoryRegion *map_reg; + hwaddr new_reg_end =3D offset + size; + QTAILQ_FOREACH(map_reg, &shmem->mapped_regions, link) { + hwaddr region_end =3D map_reg->offset + map_reg->size; + if ((map_reg->offset =3D=3D offset) || + (map_reg->offset < offset && region_end >=3D offset) || + (offset < map_reg->offset && new_reg_end >=3D map_reg->offset = )) { + return true; + } + } + return false; =20 } =20 /* A wrapper for use as a VMState .put function */ @@ -4007,11 +4044,20 @@ static void virtio_device_free_virtqueues(VirtIODev= ice *vdev) static void virtio_device_instance_finalize(Object *obj) { VirtIODevice *vdev =3D VIRTIO_DEVICE(obj); + VirtSharedMemory *shmem =3D NULL; + int i; =20 virtio_device_free_virtqueues(vdev); =20 g_free(vdev->config); g_free(vdev->vector_queues); + for (i =3D 0; i< vdev->n_shmem_regions; i++) { + shmem =3D &vdev->shmem_list[i]; + while (!QTAILQ_EMPTY(&shmem->mapped_regions)) { + MappedMemoryRegion *mmap_reg =3D QTAILQ_FIRST(&shmem->mapped_r= egions); + QTAILQ_REMOVE(&shmem->mapped_regions, mmap_reg, link); + } + } } =20 static Property virtio_properties[] =3D { diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h index d4a2f664d9..5b801f33f5 100644 --- a/include/hw/virtio/virtio.h +++ b/include/hw/virtio/virtio.h @@ -98,6 +98,21 @@ enum virtio_device_endian { VIRTIO_DEVICE_ENDIAN_BIG, }; =20 +struct MappedMemoryRegion { + Int128 size; + hwaddr offset; + QTAILQ_ENTRY(MappedMemoryRegion) link; +}; + +typedef struct MappedMemoryRegion MappedMemoryRegion; + +struct VirtSharedMemory { + MemoryRegion *mr; + QTAILQ_HEAD(, MappedMemoryRegion) mapped_regions; +}; + +typedef struct VirtSharedMemory VirtSharedMemory; + /** * struct VirtIODevice - common VirtIO structure * @name: name of the device @@ -168,7 +183,7 @@ struct VirtIODevice EventNotifier config_notifier; bool device_iotlb_enabled; /* Shared memory region for vhost-user mappings. */ - MemoryRegion *shmem_list; + VirtSharedMemory *shmem_list; int n_shmem_regions; }; =20 @@ -289,7 +304,13 @@ void virtio_notify(VirtIODevice *vdev, VirtQueue *vq); =20 int virtio_save(VirtIODevice *vdev, QEMUFile *f); =20 -MemoryRegion *virtio_new_shmem_region(VirtIODevice *vdev); +VirtSharedMemory *virtio_new_shmem_region(VirtIODevice *vdev); +void virtio_add_shmem_map(VirtSharedMemory *shmem, hwaddr offset, + uint64_t size); +void virtio_del_shmem_map(VirtSharedMemory *shmem, hwaddr offset, + uint64_t size); +bool virtio_shmem_map_overlaps(VirtSharedMemory *shmem, hwaddr offset, + uint64_t size); =20 extern const VMStateInfo virtio_vmstate_info; =20 --=20 2.45.2 From nobody Thu Sep 19 16:05:15 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1726152933; cv=none; d=zohomail.com; s=zohoarc; b=H4St/Axw/0oBJuNJ7042QoNpLs0ZQF8oZPmz/5Y1iL6NHfEhBKchMYyhmtNngiSsUsEo2o+iD+ZXqMhnWWcTCI8QVXk+yJSKPOM++MAHu9SWqV/XyFzcVaMmiWt6UJ4dPk017JLh/d0mXLOfNadYy6Ho2mpDZwjGrv4mJJXixLA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1726152933; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=U+t26Tfe5xUE4swJAtZl77olEp6lICr9c8g8Yyw6mWE=; b=XBCiuIrGBna67Fmw0YbnfcotmcCHdCkuDD7polmQKh1AH9juyk6QYCNkPqLfsxz1Vws0/jJZgENXu1+cY8IRl5GRBr2amfU5ju8obc08wt5254CDeoYjSVDetOHj7S5Y7rsF+pgzn5cxZTDp7rmT9Lg8oEosp5JcTqsso3AoF98= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 172615293344085.87835196152241; Thu, 12 Sep 2024 07:55:33 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1solD5-0001Fm-5A; Thu, 12 Sep 2024 10:54:15 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1solD2-00016M-7T for qemu-devel@nongnu.org; Thu, 12 Sep 2024 10:54:12 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1solD0-0000Hq-KZ for qemu-devel@nongnu.org; Thu, 12 Sep 2024 10:54:11 -0400 Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-654-rpiAWRn6M16nReWyuTPQig-1; Thu, 12 Sep 2024 10:54:07 -0400 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 1E5BE19560BD; Thu, 12 Sep 2024 14:54:06 +0000 (UTC) Received: from localhost.localdomain (unknown [10.45.224.195]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id B179C1956052; Thu, 12 Sep 2024 14:54:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1726152850; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=U+t26Tfe5xUE4swJAtZl77olEp6lICr9c8g8Yyw6mWE=; b=f9EFS44AFQZgD6bhtC7gX1VZXRA3htnMA9Y/YrxNGOn0kblle1cwGE39Dlw5d0qLRvjjbt z1RXX/Gx046sGOccXGBjDjaHRDRW8d179QvlQBRGeslT3gc6ClkAWbFiq2hN3TcAMDqSiW nPMlK6ZX4ND6Wmk9qUioQGNeBF67+5w= X-MC-Unique: rpiAWRn6M16nReWyuTPQig-1 From: Albert Esteve To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , slp@redhat.com, hi@alyssa.is, mst@redhat.com, david@redhat.com, jasowang@redhat.com, stefanha@redhat.com, Stefano Garzarella , stevensd@chromium.org, Albert Esteve Subject: [PATCH v3 3/5] vhost_user: Add frontend command for shmem config Date: Thu, 12 Sep 2024 16:53:33 +0200 Message-ID: <20240912145335.129447-4-aesteve@redhat.com> In-Reply-To: <20240912145335.129447-1-aesteve@redhat.com> References: <20240912145335.129447-1-aesteve@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=aesteve@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.143, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1726152934594116600 Content-Type: text/plain; charset="utf-8" The frontend can use this command to retrieve VIRTIO Shared Memory Regions configuration from the backend. The response contains the number of shared memory regions, their size, and shmid. This is useful when the frontend is unaware of specific backend type and configuration, for example, in the `vhost-user-device` case. Signed-off-by: Albert Esteve --- hw/virtio/vhost-user.c | 45 +++++++++++++++++++++++++++++++ include/hw/virtio/vhost-backend.h | 6 +++++ include/hw/virtio/vhost-user.h | 1 + 3 files changed, 52 insertions(+) diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index de0bb35257..83f5c02bea 100644 --- a/hw/virtio/vhost-user.c +++ b/hw/virtio/vhost-user.c @@ -57,6 +57,8 @@ */ #define VHOST_USER_MAX_CONFIG_SIZE 256 =20 +#define VHOST_USER_MAX_SHMEM_REGIONS 256 + #define VHOST_USER_PROTOCOL_FEATURE_MASK ((1 << VHOST_USER_PROTOCOL_F_MAX)= - 1) =20 typedef enum VhostUserRequest { @@ -104,6 +106,7 @@ typedef enum VhostUserRequest { VHOST_USER_GET_SHARED_OBJECT =3D 41, VHOST_USER_SET_DEVICE_STATE_FD =3D 42, VHOST_USER_CHECK_DEVICE_STATE =3D 43, + VHOST_USER_GET_SHMEM_CONFIG =3D 44, VHOST_USER_MAX } VhostUserRequest; =20 @@ -138,6 +141,12 @@ typedef struct VhostUserMemRegMsg { VhostUserMemoryRegion region; } VhostUserMemRegMsg; =20 +typedef struct VhostUserShMemConfig { + uint32_t nregions; + uint32_t padding; + uint64_t memory_sizes[VHOST_MEMORY_BASELINE_NREGIONS]; +} VhostUserShMemConfig; + typedef struct VhostUserLog { uint64_t mmap_size; uint64_t mmap_offset; @@ -245,6 +254,7 @@ typedef union { VhostUserShared object; VhostUserTransferDeviceState transfer_state; VhostUserMMap mmap; + VhostUserShMemConfig shmem; } VhostUserPayload; =20 typedef struct VhostUserMsg { @@ -3134,6 +3144,40 @@ static int vhost_user_check_device_state(struct vhos= t_dev *dev, Error **errp) return 0; } =20 +static int vhost_user_get_shmem_config(struct vhost_dev *dev, + int *nregions, + uint64_t *memory_sizes, + Error **errp) +{ + int ret; + VhostUserMsg msg =3D { + .hdr.request =3D VHOST_USER_GET_SHMEM_CONFIG, + .hdr.flags =3D VHOST_USER_VERSION, + }; + + if (!virtio_has_feature(dev->protocol_features, + VHOST_USER_PROTOCOL_F_SHMEM)) { + return 0; + } + + ret =3D vhost_user_write(dev, &msg, NULL, 0); + if (ret < 0) { + return ret; + } + + ret =3D vhost_user_read(dev, &msg); + if (ret < 0) { + return ret; + } + + assert(msg.payload.shmem.nregions <=3D VHOST_USER_MAX_SHMEM_REGIONS); + *nregions =3D msg.payload.shmem.nregions; + memcpy(memory_sizes, + &msg.payload.shmem.memory_sizes, + sizeof(uint64_t) * VHOST_MEMORY_BASELINE_NREGIONS); + return 0; +} + const VhostOps user_ops =3D { .backend_type =3D VHOST_BACKEND_TYPE_USER, .vhost_backend_init =3D vhost_user_backend_init, @@ -3172,4 +3216,5 @@ const VhostOps user_ops =3D { .vhost_supports_device_state =3D vhost_user_supports_device_state, .vhost_set_device_state_fd =3D vhost_user_set_device_state_fd, .vhost_check_device_state =3D vhost_user_check_device_state, + .vhost_get_shmem_config =3D vhost_user_get_shmem_config, }; diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-ba= ckend.h index 70c2e8ffee..f9c2955420 100644 --- a/include/hw/virtio/vhost-backend.h +++ b/include/hw/virtio/vhost-backend.h @@ -159,6 +159,11 @@ typedef int (*vhost_set_device_state_fd_op)(struct vho= st_dev *dev, int *reply_fd, Error **errp); typedef int (*vhost_check_device_state_op)(struct vhost_dev *dev, Error **= errp); +typedef int (*vhost_get_shmem_config_op)(struct vhost_dev *dev, + int *nregions, + uint64_t *memory_sizes, + Error **errp); + =20 typedef struct VhostOps { VhostBackendType backend_type; @@ -214,6 +219,7 @@ typedef struct VhostOps { vhost_supports_device_state_op vhost_supports_device_state; vhost_set_device_state_fd_op vhost_set_device_state_fd; vhost_check_device_state_op vhost_check_device_state; + vhost_get_shmem_config_op vhost_get_shmem_config; } VhostOps; =20 int vhost_backend_update_device_iotlb(struct vhost_dev *dev, diff --git a/include/hw/virtio/vhost-user.h b/include/hw/virtio/vhost-user.h index 324cd8663a..d4bb2c3958 100644 --- a/include/hw/virtio/vhost-user.h +++ b/include/hw/virtio/vhost-user.h @@ -32,6 +32,7 @@ enum VhostUserProtocolFeature { /* Feature 17 reserved for VHOST_USER_PROTOCOL_F_XEN_MMAP. */ VHOST_USER_PROTOCOL_F_SHARED_OBJECT =3D 18, VHOST_USER_PROTOCOL_F_DEVICE_STATE =3D 19, + VHOST_USER_PROTOCOL_F_SHMEM =3D 20, VHOST_USER_PROTOCOL_F_MAX }; =20 --=20 2.45.2 From nobody Thu Sep 19 16:05:15 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1726152874; cv=none; d=zohomail.com; s=zohoarc; b=cfExA3tMSPiDboaW3TMOKpIEJ3mitG0tGgANb4DO72yCl6TtexDdpL9eD9aFkb8FymATVMju3uohgQMtkV7p6FSvGXdiLTJPctObx1fRJJR8n2OxE1a/shulpZHPpBnZaDkXXsA0MxE2Ky1ytXatu+A7BsihNfiohPNN3x3RLlM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1726152874; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=fCIUDDzY8Nmpe0C6EnAPszeGwSqsgVRES6S6Gvetf6Y=; b=Ug7zM1W9fevxdDKLOl20lRWT3uiuJCsPCZbYGltdw4XC/MSMOrrAAs3KqSQMzqfvlH5/cnOs+wLFizh4C/4VMCPwmXhl5NMKpB7l9rOl2HUFtYgFtuLpC/uAi4PNAEvO/qyWiJ4ZyBdW/hWxM1I0xxeRB1zWpaiObzF3WEmKVt4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1726152874226586.7134052886848; Thu, 12 Sep 2024 07:54:34 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1solD6-0001MK-UX; Thu, 12 Sep 2024 10:54:17 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1solD5-0001IU-T5 for qemu-devel@nongnu.org; Thu, 12 Sep 2024 10:54:15 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1solD3-0000IA-Qc for qemu-devel@nongnu.org; Thu, 12 Sep 2024 10:54:15 -0400 Received: from mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-561-yeLnVzR6M16vjycexhv8EA-1; Thu, 12 Sep 2024 10:54:12 -0400 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 389A5196CE1D; Thu, 12 Sep 2024 14:54:11 +0000 (UTC) Received: from localhost.localdomain (unknown [10.45.224.195]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 8A03019560A3; Thu, 12 Sep 2024 14:54:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1726152853; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fCIUDDzY8Nmpe0C6EnAPszeGwSqsgVRES6S6Gvetf6Y=; b=NecqnzenQLSkjdlwpXkfBKVQHQZ7TgdjUWAQGSZH2koZZt+0aN8u425FqaY9XFAHfzhSUG rN6nyYUUiOaPmptyAJY6SgwpjswuXttuKrvSDUJOdLApI+lS75VvhrXgs9QRvF6QSV+aol FF8S+A0nce1mnVMEV/RauaMuTtxAE68= X-MC-Unique: yeLnVzR6M16vjycexhv8EA-1 From: Albert Esteve To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , slp@redhat.com, hi@alyssa.is, mst@redhat.com, david@redhat.com, jasowang@redhat.com, stefanha@redhat.com, Stefano Garzarella , stevensd@chromium.org, Albert Esteve Subject: [PATCH v3 4/5] vhost-user-dev: Add cache BAR Date: Thu, 12 Sep 2024 16:53:34 +0200 Message-ID: <20240912145335.129447-5-aesteve@redhat.com> In-Reply-To: <20240912145335.129447-1-aesteve@redhat.com> References: <20240912145335.129447-1-aesteve@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=aesteve@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.143, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1726152876358116600 Content-Type: text/plain; charset="utf-8" Add a cache BAR in the vhost-user-device into which files can be directly mapped. The number, shmid, and size of the VIRTIO Shared Memory subregions is retrieved through a get_shmem_config message sent by the vhost-user-base module on the realize step, after virtio_init(). By default, if VHOST_USER_PROTOCOL_F_SHMEM feature is not supported by the backend, there is no cache. Signed-off-by: Albert Esteve --- hw/virtio/vhost-user-base.c | 37 +++++++++++++++++++++++++++-- hw/virtio/vhost-user-device-pci.c | 39 ++++++++++++++++++++++++++++--- 2 files changed, 71 insertions(+), 5 deletions(-) diff --git a/hw/virtio/vhost-user-base.c b/hw/virtio/vhost-user-base.c index 2bc3423326..f2597d021a 100644 --- a/hw/virtio/vhost-user-base.c +++ b/hw/virtio/vhost-user-base.c @@ -271,7 +271,9 @@ static void vub_device_realize(DeviceState *dev, Error = **errp) { VirtIODevice *vdev =3D VIRTIO_DEVICE(dev); VHostUserBase *vub =3D VHOST_USER_BASE(dev); - int ret; + uint64_t memory_sizes[8]; + void *cache_ptr; + int i, ret, nregions; =20 if (!vub->chardev.chr) { error_setg(errp, "vhost-user-base: missing chardev"); @@ -314,7 +316,7 @@ static void vub_device_realize(DeviceState *dev, Error = **errp) =20 /* Allocate queues */ vub->vqs =3D g_ptr_array_sized_new(vub->num_vqs); - for (int i =3D 0; i < vub->num_vqs; i++) { + for (i =3D 0; i < vub->num_vqs; i++) { g_ptr_array_add(vub->vqs, virtio_add_queue(vdev, vub->vq_size, vub_handle_output)); @@ -331,6 +333,37 @@ static void vub_device_realize(DeviceState *dev, Error= **errp) do_vhost_user_cleanup(vdev, vub); } =20 + ret =3D vub->vhost_dev.vhost_ops->vhost_get_shmem_config(&vub->vhost_d= ev, + &nregions, + memory_sizes, + errp); + + if (ret < 0) { + do_vhost_user_cleanup(vdev, vub); + } + + for (i =3D 0; i < nregions; i++) { + if (memory_sizes[i]) { + if (memory_sizes[i] % qemu_real_host_page_size() !=3D 0) { + error_setg(errp, "Shared memory %d size must be a power of= 2 " + "no smaller than the page size", i); + return; + } + + cache_ptr =3D mmap(NULL, memory_sizes[i], PROT_NONE, + MAP_ANONYMOUS | MAP_PRIVATE, -1, 0); + if (cache_ptr =3D=3D MAP_FAILED) { + error_setg_errno(errp, errno, "Unable to mmap blank cache"= ); + return; + } + + virtio_new_shmem_region(vdev); + memory_region_init_ram_ptr(vdev->shmem_list[i].mr, + OBJECT(vdev), "vub-shm-" + i, + memory_sizes[i], cache_ptr); + } + } + qemu_chr_fe_set_handlers(&vub->chardev, NULL, NULL, vub_event, NULL, dev, NULL, true); } diff --git a/hw/virtio/vhost-user-device-pci.c b/hw/virtio/vhost-user-devic= e-pci.c index efaf55d3dd..abf4e90c21 100644 --- a/hw/virtio/vhost-user-device-pci.c +++ b/hw/virtio/vhost-user-device-pci.c @@ -8,14 +8,18 @@ */ =20 #include "qemu/osdep.h" +#include "qapi/error.h" #include "hw/qdev-properties.h" #include "hw/virtio/vhost-user-base.h" #include "hw/virtio/virtio-pci.h" =20 +#define VIRTIO_DEVICE_PCI_CACHE_BAR 2 + struct VHostUserDevicePCI { VirtIOPCIProxy parent_obj; =20 VHostUserBase vub; + MemoryRegion cachebar; }; =20 #define TYPE_VHOST_USER_DEVICE_PCI "vhost-user-device-pci-base" @@ -25,10 +29,39 @@ OBJECT_DECLARE_SIMPLE_TYPE(VHostUserDevicePCI, VHOST_US= ER_DEVICE_PCI) static void vhost_user_device_pci_realize(VirtIOPCIProxy *vpci_dev, Error = **errp) { VHostUserDevicePCI *dev =3D VHOST_USER_DEVICE_PCI(vpci_dev); - DeviceState *vdev =3D DEVICE(&dev->vub); - + DeviceState *dev_state =3D DEVICE(&dev->vub); + VirtIODevice *vdev =3D VIRTIO_DEVICE(dev_state); + MemoryRegion *mr; + uint64_t offset =3D 0, cache_size =3D 0; + int i; + =20 vpci_dev->nvectors =3D 1; - qdev_realize(vdev, BUS(&vpci_dev->bus), errp); + qdev_realize(dev_state, BUS(&vpci_dev->bus), errp); + + for (i =3D 0; i < vdev->n_shmem_regions; i++) { + mr =3D vdev->shmem_list[i].mr; + if (mr->size > UINT64_MAX - cache_size) { + error_setg(errp, "Total shared memory required overflow"); + return; + } + cache_size =3D cache_size + mr->size; + } + if (cache_size) { + memory_region_init(&dev->cachebar, OBJECT(vpci_dev), + "vhost-device-pci-cachebar", cache_size); + for (i =3D 0; i < vdev->n_shmem_regions; i++) { + mr =3D vdev->shmem_list[i].mr; + memory_region_add_subregion(&dev->cachebar, offset, mr); + virtio_pci_add_shm_cap(vpci_dev, VIRTIO_DEVICE_PCI_CACHE_BAR, + offset, mr->size, i); + offset =3D offset + mr->size; + } + pci_register_bar(&vpci_dev->pci_dev, VIRTIO_DEVICE_PCI_CACHE_BAR, + PCI_BASE_ADDRESS_SPACE_MEMORY | + PCI_BASE_ADDRESS_MEM_PREFETCH | + PCI_BASE_ADDRESS_MEM_TYPE_64, + &dev->cachebar); + } } =20 static void vhost_user_device_pci_class_init(ObjectClass *klass, void *dat= a) --=20 2.45.2 From nobody Thu Sep 19 16:05:15 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1726152887; cv=none; d=zohomail.com; s=zohoarc; b=UCiHxn2zQEI6Y8SMdhi8KB+/EAjWj4Bo6SijYltNHG1qTxuVVgittoKab72ycGRFlbiV6YPEzTg60jEOyX2BPy2Fmo4KecpAVNCjV0Tdcqmw94CcXSuzGBcd87+Hs1dNWmNw4AGQog7zMSswcfQEOjIzu6bb8sDAS92twUNPUbI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1726152887; h=Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=begHxUsoQRgZERRTwqdiGrBOXuooiARyTfZVLIOy1Ww=; b=VVzY3u+S2TebO5kaNq1uBazsixj34k0AoRyoY76RPgPHWHoq0vVbmP1LnjkviPYTFNSP9H0bpO/7/X1VzqwEDzEYGKuAxCK2/r3Rdff7YlCoAJVPVh5dHYBPke8W46w4wDCJbm3USpS9Lhwxj/tlmGxc0pO8EkdgLWinJFVlDZs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1726152887064320.1912332845187; Thu, 12 Sep 2024 07:54:47 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1solDF-0001gA-11; Thu, 12 Sep 2024 10:54:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1solDB-0001f9-Ti for qemu-devel@nongnu.org; Thu, 12 Sep 2024 10:54:21 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1solD9-0000IU-Om for qemu-devel@nongnu.org; Thu, 12 Sep 2024 10:54:21 -0400 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-605-a1HWPKGuNc6bdcTrWiqgig-1; Thu, 12 Sep 2024 10:54:17 -0400 Received: from mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.17]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 757D91955F3C; Thu, 12 Sep 2024 14:54:15 +0000 (UTC) Received: from localhost.localdomain (unknown [10.45.224.195]) by mx-prod-int-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id B319719560A3; Thu, 12 Sep 2024 14:54:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1726152859; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=begHxUsoQRgZERRTwqdiGrBOXuooiARyTfZVLIOy1Ww=; b=Dqe2mRJn2REixov7KIU1jThqsgDZ2aCmmeqkLlKGXHNT0+lc5ZSaQ2VO0qL4Qnq6YA1nYb +zmtsl4O9Vk0pqsq5PoVS2MytUKajrTODHjSGYDdW9Qkbq6BinjzeqL06irSzdRScGM+oL gEPkM6TilCUzGFET5N2diwi+LYcLdD0= X-MC-Unique: a1HWPKGuNc6bdcTrWiqgig-1 From: Albert Esteve To: qemu-devel@nongnu.org Cc: =?UTF-8?q?Alex=20Benn=C3=A9e?= , slp@redhat.com, hi@alyssa.is, mst@redhat.com, david@redhat.com, jasowang@redhat.com, stefanha@redhat.com, Stefano Garzarella , stevensd@chromium.org, Albert Esteve Subject: [PATCH v3 5/5] vhost_user: Add MEM_READ/WRITE backend requests Date: Thu, 12 Sep 2024 16:53:35 +0200 Message-ID: <20240912145335.129447-6-aesteve@redhat.com> In-Reply-To: <20240912145335.129447-1-aesteve@redhat.com> References: <20240912145335.129447-1-aesteve@redhat.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.0 on 10.30.177.17 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=aesteve@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -21 X-Spam_score: -2.2 X-Spam_bar: -- X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.143, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1726152888475116600 Content-Type: text/plain; charset="utf-8" With SHMEM_MAP messages, sharing descriptors between devices will cause that these devices do not see the mappings, and fail to access these memory regions. To solve this, introduce MEM_READ/WRITE requests that will get triggered as a fallback when vhost-user memory translation fails. MEM_READ/WRITE requests have flexible array members, since we do not know in advance the number of bytes in the mapped region. Therefore, we need to allow bigger message sizes for these types, and ensure we allocate sufficient memory for them. Signed-off-by: Albert Esteve --- hw/virtio/vhost-user.c | 99 +++++++++++++++++------ subprojects/libvhost-user/libvhost-user.c | 84 +++++++++++++++++++ subprojects/libvhost-user/libvhost-user.h | 38 +++++++++ 3 files changed, 197 insertions(+), 24 deletions(-) diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index 83f5c02bea..8a4718ae10 100644 --- a/hw/virtio/vhost-user.c +++ b/hw/virtio/vhost-user.c @@ -120,6 +120,8 @@ typedef enum VhostUserBackendRequest { VHOST_USER_BACKEND_SHARED_OBJECT_LOOKUP =3D 8, VHOST_USER_BACKEND_SHMEM_MAP =3D 9, VHOST_USER_BACKEND_SHMEM_UNMAP =3D 10, + VHOST_USER_BACKEND_MEM_READ =3D 11, + VHOST_USER_BACKEND_MEM_WRITE =3D 12, VHOST_USER_BACKEND_MAX } VhostUserBackendRequest; =20 @@ -147,6 +149,12 @@ typedef struct VhostUserShMemConfig { uint64_t memory_sizes[VHOST_MEMORY_BASELINE_NREGIONS]; } VhostUserShMemConfig; =20 +typedef struct VhostUserMemRWMsg { + uint64_t guest_address; + uint32_t size; + uint8_t data[]; +} VhostUserMemRWMsg; + typedef struct VhostUserLog { uint64_t mmap_size; uint64_t mmap_offset; @@ -255,6 +263,7 @@ typedef union { VhostUserTransferDeviceState transfer_state; VhostUserMMap mmap; VhostUserShMemConfig shmem; + VhostUserMemRWMsg mem_rw; } VhostUserPayload; =20 typedef struct VhostUserMsg { @@ -343,17 +352,23 @@ static int vhost_user_read(struct vhost_dev *dev, Vho= stUserMsg *msg) return r; } =20 - /* validate message size is sane */ - if (msg->hdr.size > VHOST_USER_PAYLOAD_SIZE) { - error_report("Failed to read msg header." - " Size %d exceeds the maximum %zu.", msg->hdr.size, - VHOST_USER_PAYLOAD_SIZE); - return -EPROTO; - } - if (msg->hdr.size) { p +=3D VHOST_USER_HDR_SIZE; size =3D msg->hdr.size; + /* validate message size is sane */ + if (msg->hdr.size > VHOST_USER_PAYLOAD_SIZE) { + switch(msg->hdr.request) { + case VHOST_USER_BACKEND_MEM_READ: + case VHOST_USER_BACKEND_MEM_WRITE: + p =3D g_malloc0(size); + break; + default: + error_report("Failed to read msg header." + " Size %d exceeds the maximum %zu.", + size, VHOST_USER_PAYLOAD_SIZE); + return -EPROTO; + } + } r =3D qemu_chr_fe_read_all(chr, p, size); if (r !=3D size) { int saved_errno =3D errno; @@ -1885,6 +1900,28 @@ vhost_user_backend_handle_shmem_unmap(struct vhost_d= ev *dev, return 0; } =20 +static int +vhost_user_backend_handle_mem_read(struct vhost_dev *dev, + VhostUserMemRWMsg *mem_rw) +{ + MemTxResult result; + result =3D address_space_read(dev->vdev->dma_as, mem_rw->guest_address, + MEMTXATTRS_UNSPECIFIED, &mem_rw->data, + mem_rw->size); + return result; +} + +static int +vhost_user_backend_handle_mem_write(struct vhost_dev *dev, + VhostUserMemRWMsg *mem_rw) +{ + MemTxResult result; + result =3D address_space_write(dev->vdev->dma_as, mem_rw->guest_addres= s, + MEMTXATTRS_UNSPECIFIED, &mem_rw->data, + mem_rw->size); + return result; +} + static void close_backend_channel(struct vhost_user *u) { g_source_destroy(u->backend_src); @@ -1900,7 +1937,7 @@ static gboolean backend_read(QIOChannel *ioc, GIOCond= ition condition, struct vhost_dev *dev =3D opaque; struct vhost_user *u =3D dev->opaque; VhostUserHeader hdr =3D { 0, }; - VhostUserPayload payload =3D { 0, }; + VhostUserPayload *payload =3D g_new0(VhostUserPayload, 1); Error *local_err =3D NULL; gboolean rc =3D G_SOURCE_CONTINUE; int ret =3D 0; @@ -1919,46 +1956,59 @@ static gboolean backend_read(QIOChannel *ioc, GIOCo= ndition condition, } =20 if (hdr.size > VHOST_USER_PAYLOAD_SIZE) { - error_report("Failed to read msg header." - " Size %d exceeds the maximum %zu.", hdr.size, - VHOST_USER_PAYLOAD_SIZE); - goto err; + switch (hdr.request) { + case VHOST_USER_BACKEND_MEM_READ: + case VHOST_USER_BACKEND_MEM_WRITE: + payload =3D g_malloc0(hdr.size); + break; + default: + error_report("Failed to read msg header." + " Size %d exceeds the maximum %zu.", hdr.size, + VHOST_USER_PAYLOAD_SIZE); + goto err; + } } =20 /* Read payload */ - if (qio_channel_read_all(ioc, (char *) &payload, hdr.size, &local_err)= ) { + if (qio_channel_read_all(ioc, (char *) payload, hdr.size, &local_err))= { error_report_err(local_err); goto err; } =20 switch (hdr.request) { case VHOST_USER_BACKEND_IOTLB_MSG: - ret =3D vhost_backend_handle_iotlb_msg(dev, &payload.iotlb); + ret =3D vhost_backend_handle_iotlb_msg(dev, &payload->iotlb); break; case VHOST_USER_BACKEND_CONFIG_CHANGE_MSG: ret =3D vhost_user_backend_handle_config_change(dev); break; case VHOST_USER_BACKEND_VRING_HOST_NOTIFIER_MSG: - ret =3D vhost_user_backend_handle_vring_host_notifier(dev, &payloa= d.area, + ret =3D vhost_user_backend_handle_vring_host_notifier(dev, &payloa= d->area, fd ? fd[0] : -1); break; case VHOST_USER_BACKEND_SHARED_OBJECT_ADD: - ret =3D vhost_user_backend_handle_shared_object_add(dev, &payload.= object); + ret =3D vhost_user_backend_handle_shared_object_add(dev, &payload-= >object); break; case VHOST_USER_BACKEND_SHARED_OBJECT_REMOVE: ret =3D vhost_user_backend_handle_shared_object_remove(dev, - &payload.obje= ct); + &payload->obj= ect); break; case VHOST_USER_BACKEND_SHARED_OBJECT_LOOKUP: ret =3D vhost_user_backend_handle_shared_object_lookup(dev->opaque= , ioc, - &hdr, &payloa= d); + &hdr, payload= ); break; case VHOST_USER_BACKEND_SHMEM_MAP: - ret =3D vhost_user_backend_handle_shmem_map(dev, &payload.mmap, + ret =3D vhost_user_backend_handle_shmem_map(dev, &payload->mmap, fd ? fd[0] : -1); break; case VHOST_USER_BACKEND_SHMEM_UNMAP: - ret =3D vhost_user_backend_handle_shmem_unmap(dev, &payload.mmap); + ret =3D vhost_user_backend_handle_shmem_unmap(dev, &payload->mmap); + break; + case VHOST_USER_BACKEND_MEM_READ: + ret =3D vhost_user_backend_handle_mem_read(dev, &payload->mem_rw); + break; + case VHOST_USER_BACKEND_MEM_WRITE: + ret =3D vhost_user_backend_handle_mem_write(dev, &payload->mem_rw); break; default: error_report("Received unexpected msg type: %d.", hdr.request); @@ -1970,10 +2020,10 @@ static gboolean backend_read(QIOChannel *ioc, GIOCo= ndition condition, * directly in their request handlers. */ if (hdr.flags & VHOST_USER_NEED_REPLY_MASK) { - payload.u64 =3D !!ret; - hdr.size =3D sizeof(payload.u64); + payload->u64 =3D !!ret; + hdr.size =3D sizeof(payload->u64); =20 - if (!vhost_user_send_resp(ioc, &hdr, &payload, &local_err)) { + if (!vhost_user_send_resp(ioc, &hdr, payload, &local_err)) { error_report_err(local_err); goto err; } @@ -1991,6 +2041,7 @@ fdcleanup: close(fd[i]); } } + g_free(payload); return rc; } =20 diff --git a/subprojects/libvhost-user/libvhost-user.c b/subprojects/libvho= st-user/libvhost-user.c index 496268e12b..08de54896b 100644 --- a/subprojects/libvhost-user/libvhost-user.c +++ b/subprojects/libvhost-user/libvhost-user.c @@ -1652,6 +1652,90 @@ vu_shmem_unmap(VuDev *dev, uint8_t shmid, uint64_t s= hm_offset, uint64_t len) return vu_process_message_reply(dev, &vmsg); } =20 +bool +vu_send_mem_read(VuDev *dev, uint64_t guest_addr, uint32_t size, + uint8_t *data) +{ + VhostUserMsg msg_reply; + VhostUserMsg msg =3D { + .request =3D VHOST_USER_BACKEND_MEM_READ, + .size =3D sizeof(msg.payload.mem_rw), + .flags =3D VHOST_USER_VERSION | VHOST_USER_NEED_REPLY_MASK, + .payload =3D { + .mem_rw =3D { + .guest_address =3D guest_addr, + .size =3D size, + } + } + }; + + pthread_mutex_lock(&dev->backend_mutex); + if (!vu_message_write(dev, dev->backend_fd, &msg)) { + goto out_err; + } + + if (!vu_message_read_default(dev, dev->backend_fd, &msg_reply)) { + goto out_err; + } + + if (msg_reply.request !=3D msg.request) { + DPRINT("Received unexpected msg type. Expected %d, received %d", + msg.request, msg_reply.request); + goto out_err; + } + + if (msg_reply.payload.mem_rw.size !=3D size) { + DPRINT("Received unexpected number of bytes in the response. " + "Expected %d, received %d", + size, msg_reply.payload.mem_rw.size); + goto out_err; + } + + data =3D malloc(msg_reply.payload.mem_rw.size); + if (!data) { + DPRINT("Failed to malloc read memory data"); + goto out_err; + } + + memcpy(data, msg_reply.payload.mem_rw.data, size); + pthread_mutex_unlock(&dev->backend_mutex); + return true; + +out_err: + pthread_mutex_unlock(&dev->backend_mutex); + return false; +} + +bool +vu_send_mem_write(VuDev *dev, uint64_t guest_addr, uint32_t size, + uint8_t *data) +{ + VhostUserMsg msg =3D { + .request =3D VHOST_USER_BACKEND_MEM_WRITE, + .size =3D sizeof(msg.payload.mem_rw), + .flags =3D VHOST_USER_VERSION, + .payload =3D { + .mem_rw =3D { + .guest_address =3D guest_addr, + .size =3D size, + } + } + }; + memcpy(msg.payload.mem_rw.data, data, size); + + if (vu_has_protocol_feature(dev, VHOST_USER_PROTOCOL_F_REPLY_ACK)) { + msg.flags |=3D VHOST_USER_NEED_REPLY_MASK; + } + + if (!vu_message_write(dev, dev->backend_fd, &msg)) { + pthread_mutex_unlock(&dev->backend_mutex); + return false; + } + + /* Also unlocks the backend_mutex */ + return vu_process_message_reply(dev, &msg); +} + static bool vu_set_vring_call_exec(VuDev *dev, VhostUserMsg *vmsg) { diff --git a/subprojects/libvhost-user/libvhost-user.h b/subprojects/libvho= st-user/libvhost-user.h index ea4902e876..8ec49dcb1b 100644 --- a/subprojects/libvhost-user/libvhost-user.h +++ b/subprojects/libvhost-user/libvhost-user.h @@ -129,6 +129,8 @@ typedef enum VhostUserBackendRequest { VHOST_USER_BACKEND_SHARED_OBJECT_LOOKUP =3D 8, VHOST_USER_BACKEND_SHMEM_MAP =3D 9, VHOST_USER_BACKEND_SHMEM_UNMAP =3D 10, + VHOST_USER_BACKEND_MEM_READ =3D 11, + VHOST_USER_BACKEND_MEM_WRITE =3D 12, VHOST_USER_BACKEND_MAX } VhostUserBackendRequest; =20 @@ -152,6 +154,12 @@ typedef struct VhostUserMemRegMsg { VhostUserMemoryRegion region; } VhostUserMemRegMsg; =20 +typedef struct VhostUserMemRWMsg { + uint64_t guest_address; + uint32_t size; + uint8_t data[]; +} VhostUserMemRWMsg; + typedef struct VhostUserLog { uint64_t mmap_size; uint64_t mmap_offset; @@ -235,6 +243,7 @@ typedef struct VhostUserMsg { VhostUserInflight inflight; VhostUserShared object; VhostUserMMap mmap; + VhostUserMemRWMsg mem_rw; } payload; =20 int fds[VHOST_MEMORY_BASELINE_NREGIONS]; @@ -649,6 +658,35 @@ bool vu_shmem_map(VuDev *dev, uint8_t shmid, uint64_t = fd_offset, bool vu_shmem_unmap(VuDev *dev, uint8_t shmid, uint64_t shm_offset, uint64_t len); =20 +/** + * vu_send_mem_read: + * @dev: a VuDev context + * @guest_addr: guest physical address to read + * @size: number of bytes to read + * @data: head of an unitialized bytes array + * + * Reads `size` bytes of `guest_addr` in the frontend and stores + * them in `data`. + * + * Returns: TRUE on success, FALSE on failure. + */ +bool vu_send_mem_read(VuDev *dev, uint64_t guest_addr, uint32_t size, + uint8_t *data); + +/** + * vu_send_mem_write: + * @dev: a VuDev context + * @guest_addr: guest physical address to write + * @size: number of bytes to write + * @data: head of an array with `size` bytes to write + * + * Writes `size` bytes from `data` into `guest_addr` in the frontend. + * + * Returns: TRUE on success, FALSE on failure. + */ +bool vu_send_mem_write(VuDev *dev, uint64_t guest_addr, uint32_t size, + uint8_t *data); + /** * vu_queue_set_notification: * @dev: a VuDev context --=20 2.45.2