From nobody Mon Sep 16 19:04:07 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) client-ip=170.10.133.124; envelope-from=libvir-list-bounces@redhat.com; helo=us-smtp-delivery-124.mimecast.com; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1694469241; cv=none; d=zohomail.com; s=zohoarc; b=VqZxfmX4kLrmnvpFkdqBeR+ORUOl4te7aQ2B2Sab4fDbIiBMvkzJQNzDwPXFqH+WgXwqgyPmX5VTbbwpGMB/YirJuQTJn04Y/KgCuf2ssEYdos8Rm/j2KXVAkGfhd+q1SUMHGBXQrIwqd6Sk/ZaSjmiMcv/qxoR314xrU41MIIw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1694469241; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=kA/bMntDIOneXkinUNvxqMfYUKi6HgcARuo8sTMqajw=; b=SGHUZmOj+2etSciMwjVg3W/V1DW7d23/7U13AoiToCfywebSkmV994SPKfTIBzn+tJZ/BWFEj6KG8LdcDpl8ZHN3lBg7ny76+cgAv4e2dCzj3ItdIUV8SEUG7qBjcCMurMfOIcMWRvpApwe8PpLFOPpJqvGZK6aVDBqfL8RQ8DM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of redhat.com designates 170.10.133.124 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by mx.zohomail.com with SMTPS id 1694469241307335.84098155964614; Mon, 11 Sep 2023 14:54:01 -0700 (PDT) Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-618-rsYHpTuSMUOdEf_G23lcFg-1; Mon, 11 Sep 2023 17:53:56 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C50F63C100BB; Mon, 11 Sep 2023 21:53:52 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com [10.30.29.100]) by smtp.corp.redhat.com (Postfix) with ESMTP id A2AD6200E4BC; Mon, 11 Sep 2023 21:53:52 +0000 (UTC) Received: from mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (localhost [IPv6:::1]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 07B4419465B3; Mon, 11 Sep 2023 21:53:51 +0000 (UTC) Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.rdu2.redhat.com [10.11.54.2]) by mm-prod-listman-01.mail-001.prod.us-east-1.aws.redhat.com (Postfix) with ESMTP id 791831946597 for ; Mon, 11 Sep 2023 21:53:49 +0000 (UTC) Received: by smtp.corp.redhat.com (Postfix) id 69B4040C6EBF; Mon, 11 Sep 2023 21:53:49 +0000 (UTC) Received: from himantopus.redhat.com (unknown [10.22.34.54]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 3EC5740C6EA8 for ; Mon, 11 Sep 2023 21:53:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1694469239; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post; bh=kA/bMntDIOneXkinUNvxqMfYUKi6HgcARuo8sTMqajw=; b=hWTmQJ3sS06ksHZEsU2QUm/ZU4iz1idh4ovtzCwIByoJ6eM+ntOyx70EDjODTfJ34xXusb Z6g99EZbjdEAEPw10QCWFpEpQm5wFM5qhr7BlNDJL2si8uSloUT0emAYdxP6FTELWOAyVx Z5Z0bHxG3Q3zyJZoDcYVWzFXjquaO9o= X-MC-Unique: rsYHpTuSMUOdEf_G23lcFg-1 X-Original-To: libvir-list@listman.corp.redhat.com From: Jonathon Jongsma To: libvir-list@redhat.com Subject: [libvirt PATCH v2 5/5] qemu: Implement support for vDPA block devices Date: Mon, 11 Sep 2023 16:53:47 -0500 Message-ID: <20230911215347.1160427-6-jjongsma@redhat.com> In-Reply-To: <20230911215347.1160427-1-jjongsma@redhat.com> References: <20230911215347.1160427-1-jjongsma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 3.1 on 10.11.54.2 X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: libvir-list-bounces@redhat.com Sender: "libvir-list" X-Scanned-By: MIMEDefang 3.1 on 10.11.54.6 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1694469242066100009 Content-Type: text/plain; charset="utf-8"; x-default="true" Requires recent qemu with support for the virtio-blk-vhost-vdpa device and the ability to pass a /dev/fdset/N path for the vdpa path (8.1.0) Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=3D1900770 Signed-off-by: Jonathon Jongsma --- src/qemu/qemu_block.c | 20 ++++++++-- src/qemu/qemu_process.c | 34 +++++++++++++++++ src/qemu/qemu_validate.c | 32 ++++++++++++++-- .../disk-vhostvdpa.x86_64-latest.args | 37 +++++++++++++++++++ tests/qemuxml2argvdata/disk-vhostvdpa.xml | 21 +++++++++++ tests/qemuxml2argvtest.c | 34 +++++++++++++++++ tests/testutilsqemu.c | 11 ++++++ tests/testutilsqemu.h | 2 + 8 files changed, 185 insertions(+), 6 deletions(-) create mode 100644 tests/qemuxml2argvdata/disk-vhostvdpa.x86_64-latest.args create mode 100644 tests/qemuxml2argvdata/disk-vhostvdpa.xml diff --git a/src/qemu/qemu_block.c b/src/qemu/qemu_block.c index 0b4c2dbcf4..d31bbde0f4 100644 --- a/src/qemu/qemu_block.c +++ b/src/qemu/qemu_block.c @@ -778,6 +778,20 @@ qemuBlockStorageSourceGetNVMeProps(virStorageSource *s= rc) } =20 =20 +static virJSONValue * +qemuBlockStorageSourceGetVhostVdpaProps(virStorageSource *src) +{ + virJSONValue *ret =3D NULL; + qemuDomainStorageSourcePrivate *srcpriv =3D QEMU_DOMAIN_STORAGE_SOURCE= _PRIVATE(src); + + ignore_value(virJSONValueObjectAdd(&ret, + "s:driver", "virtio-blk-vhost-vdpa", + "s:path", qemuFDPassGetPath(srcpriv= ->fdpass), + NULL)); + return ret; +} + + static int qemuBlockStorageSourceGetBlockdevGetCacheProps(virStorageSource *src, virJSONValue *props) @@ -874,9 +888,9 @@ qemuBlockStorageSourceGetBackendProps(virStorageSource = *src, break; =20 case VIR_STORAGE_TYPE_VHOST_VDPA: - virReportError(VIR_ERR_INTERNAL_ERROR, "%s", - _("vhostvdpa disk type not yet supported")); - return NULL; + if (!(fileprops =3D qemuBlockStorageSourceGetVhostVdpaProps(src))) + return NULL; + break; =20 case VIR_STORAGE_TYPE_VHOST_USER: virReportError(VIR_ERR_INTERNAL_ERROR, "%s", diff --git a/src/qemu/qemu_process.c b/src/qemu/qemu_process.c index 7a1cdb0302..42837c4a8a 100644 --- a/src/qemu/qemu_process.c +++ b/src/qemu/qemu_process.c @@ -6820,6 +6820,28 @@ qemuProcessPrepareLaunchSecurityGuestInput(virDomain= Obj *vm) } =20 =20 +static int +qemuProcessPrepareHostStorageSourceVDPA(virStorageSource *src, + qemuDomainObjPrivate *priv) +{ + qemuDomainStorageSourcePrivate *srcpriv =3D NULL; + virStorageType actualType =3D virStorageSourceGetActualType(src); + int vdpafd =3D -1; + + if (actualType !=3D VIR_STORAGE_TYPE_VHOST_VDPA) + return 0; + + if ((vdpafd =3D qemuVDPAConnect(src->vdpadev)) < 0) + return -1; + + srcpriv =3D qemuDomainStorageSourcePrivateFetch(src); + + srcpriv->fdpass =3D qemuFDPassNew(src->nodestorage, priv); + qemuFDPassAddFD(srcpriv->fdpass, &vdpafd, "-vdpa"); + return 0; +} + + static int qemuProcessPrepareHostStorage(virQEMUDriver *driver, virDomainObj *vm, @@ -6856,6 +6878,18 @@ qemuProcessPrepareHostStorage(virQEMUDriver *driver, return -1; } =20 + /* connect to any necessary vdpa block devices */ + for (i =3D vm->def->ndisks; i > 0; i--) { + size_t idx =3D i - 1; + virDomainDiskDef *disk =3D vm->def->disks[idx]; + virStorageSource *src; + + for (src =3D disk->src; virStorageSourceIsBacking(src); src =3D sr= c->backingStore) { + if (qemuProcessPrepareHostStorageSourceVDPA(src, vm->privateDa= ta) < 0) + return -1; + } + } + return 0; } =20 diff --git a/src/qemu/qemu_validate.c b/src/qemu/qemu_validate.c index 165ab3a66a..5bae56b00f 100644 --- a/src/qemu/qemu_validate.c +++ b/src/qemu/qemu_validate.c @@ -3175,13 +3175,39 @@ qemuValidateDomainDeviceDefDisk(const virDomainDisk= Def *disk, } =20 if (disk->src->type =3D=3D VIR_STORAGE_TYPE_VHOST_USER) { + const char *vhosttype =3D virStorageTypeToString(disk->src->type); + if (!virQEMUCapsGet(qemuCaps, QEMU_CAPS_DEVICE_VHOST_USER_BLK)) { - virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s", - _("vhostuser disk is not supported with this QE= MU binary")); + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("%1$s disk is not supported with this QEMU bi= nary"), + vhosttype); + return -1; + } + + if (qemuValidateDomainDefVhostUserRequireSharedMemory(def, vhostty= pe) < 0) + return -1; + } + + if (disk->src->type =3D=3D VIR_STORAGE_TYPE_VHOST_VDPA) { + const char *vhosttype =3D virStorageTypeToString(disk->src->type); + + if (!virQEMUCapsGet(qemuCaps, QEMU_CAPS_DEVICE_VIRTIO_BLK_VHOST_VD= PA)) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("%1$s disk is not supported with this QEMU bi= nary"), + vhosttype); return -1; } =20 - if (qemuValidateDomainDefVhostUserRequireSharedMemory(def, "vhostu= ser") < 0) { + if (qemuValidateDomainDefVhostUserRequireSharedMemory(def, vhostty= pe) < 0) + return -1; + + if (disk->cachemode !=3D VIR_DOMAIN_DISK_CACHE_DIRECTSYNC && + disk->cachemode !=3D VIR_DOMAIN_DISK_CACHE_DISABLE) { + virReportError(VIR_ERR_CONFIG_UNSUPPORTED, + _("disk type '%1$s' requires cache mode '%2$s' = or '%3$s'"), + virStorageTypeToString(disk->src->type), + virDomainDiskCacheTypeToString(VIR_DOMAIN_DISK_= CACHE_DIRECTSYNC), + virDomainDiskCacheTypeToString(VIR_DOMAIN_DISK_= CACHE_DISABLE)); return -1; } } diff --git a/tests/qemuxml2argvdata/disk-vhostvdpa.x86_64-latest.args b/tes= ts/qemuxml2argvdata/disk-vhostvdpa.x86_64-latest.args new file mode 100644 index 0000000000..b987455ee4 --- /dev/null +++ b/tests/qemuxml2argvdata/disk-vhostvdpa.x86_64-latest.args @@ -0,0 +1,37 @@ +LC_ALL=3DC \ +PATH=3D/bin \ +HOME=3D/var/lib/libvirt/qemu/domain--1-QEMUGuest1 \ +USER=3Dtest \ +LOGNAME=3Dtest \ +XDG_DATA_HOME=3D/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.local/share \ +XDG_CACHE_HOME=3D/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.cache \ +XDG_CONFIG_HOME=3D/var/lib/libvirt/qemu/domain--1-QEMUGuest1/.config \ +/usr/bin/qemu-system-x86_64 \ +-name guest=3DQEMUGuest1,debug-threads=3Don \ +-S \ +-object '{"qom-type":"secret","id":"masterKey0","format":"raw","file":"/va= r/lib/libvirt/qemu/domain--1-QEMUGuest1/master-key.aes"}' \ +-machine pc,usb=3Doff,dump-guest-core=3Doff,memory-backend=3Dpc.ram,acpi= =3Doff \ +-accel tcg \ +-cpu qemu64 \ +-m size=3D219136k \ +-object '{"qom-type":"memory-backend-file","id":"pc.ram","mem-path":"/var/= lib/libvirt/qemu/ram/-1-QEMUGuest1/pc.ram","share":true,"x-use-canonical-pa= th-for-ramblock-id":false,"size":224395264}' \ +-overcommit mem-lock=3Doff \ +-smp 1,sockets=3D1,cores=3D1,threads=3D1 \ +-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \ +-display none \ +-no-user-config \ +-nodefaults \ +-chardev socket,id=3Dcharmonitor,fd=3D1729,server=3Don,wait=3Doff \ +-mon chardev=3Dcharmonitor,id=3Dmonitor,mode=3Dcontrol \ +-rtc base=3Dutc \ +-no-shutdown \ +-boot strict=3Don \ +-device '{"driver":"piix3-usb-uhci","id":"usb","bus":"pci.0","addr":"0x1.0= x2"}' \ +-add-fd set=3D0,fd=3D801,opaque=3Dlibvirt-1-storage-vdpa \ +-blockdev '{"driver":"virtio-blk-vhost-vdpa","path":"/dev/fdset/0","node-n= ame":"libvirt-1-storage","cache":{"direct":true,"no-flush":false},"auto-rea= d-only":true,"discard":"unmap"}' \ +-blockdev '{"node-name":"libvirt-1-format","read-only":false,"cache":{"dir= ect":true,"no-flush":false},"driver":"raw","file":"libvirt-1-storage"}' \ +-device '{"driver":"virtio-blk-pci","bus":"pci.0","addr":"0x2","drive":"li= bvirt-1-format","id":"virtio-disk0","bootindex":1,"write-cache":"on"}' \ +-audiodev '{"id":"audio1","driver":"none"}' \ +-device '{"driver":"virtio-balloon-pci","id":"balloon0","bus":"pci.0","add= r":"0x3"}' \ +-sandbox on,obsolete=3Ddeny,elevateprivileges=3Ddeny,spawn=3Ddeny,resource= control=3Ddeny \ +-msg timestamp=3Don diff --git a/tests/qemuxml2argvdata/disk-vhostvdpa.xml b/tests/qemuxml2argv= data/disk-vhostvdpa.xml new file mode 100644 index 0000000000..0ac3899a34 --- /dev/null +++ b/tests/qemuxml2argvdata/disk-vhostvdpa.xml @@ -0,0 +1,21 @@ + + QEMUGuest1 + c7a5fdbd-edaf-9455-926a-d65c16db1809 + 219136 + 219136 + + + + 1 + + hvm + + + /usr/bin/qemu-system-x86_64 + + + + + + + diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c index 2d06e2a0d1..0dd10f1fab 100644 --- a/tests/qemuxml2argvtest.c +++ b/tests/qemuxml2argvtest.c @@ -304,6 +304,7 @@ testCompareXMLToArgvCreateArgs(virQEMUDriver *drv, =20 for (i =3D 0; i < vm->def->ndisks; i++) { virDomainDiskDef *disk =3D vm->def->disks[i]; + virStorageSource *src; =20 /* host cdrom requires special treatment in qemu, mock it */ if (disk->device =3D=3D VIR_DOMAIN_DISK_DEVICE_CDROM && @@ -311,6 +312,37 @@ testCompareXMLToArgvCreateArgs(virQEMUDriver *drv, virStorageSourceIsBlockLocal(disk->src) && STREQ(disk->src->path, "/dev/cdrom")) disk->src->hostcdrom =3D true; + + if (info->args.vdpafds) { + for (src =3D disk->src; virStorageSourceIsBacking(src); src = =3D src->backingStore) { + gpointer value; + + if (src->type !=3D VIR_STORAGE_TYPE_VHOST_VDPA) + continue; + + if ((value =3D g_hash_table_lookup(info->args.vdpafds, src= ->vdpadev))) { + int fd =3D GPOINTER_TO_INT(value); + qemuDomainStorageSourcePrivate *srcpriv; + VIR_AUTOCLOSE fakefd =3D open("/dev/zero", O_RDWR); + + if (fcntl(fd, F_GETFD) !=3D -1) { + fprintf(stderr, "fd '%d' is already in use\n", fd); + abort(); + } + + if (dup2(fakefd, fd) < 0) { + fprintf(stderr, "failed to duplicate fake fd: %s", + g_strerror(errno)); + abort(); + } + + srcpriv =3D qemuDomainStorageSourcePrivateFetch(src); + + srcpriv->fdpass =3D qemuFDPassNew(src->nodestorage, pr= iv); + qemuFDPassAddFD(srcpriv->fdpass, &fd, "-vdpa"); + } + } + } } =20 if (vm->def->vsock) { @@ -1129,6 +1161,8 @@ mymain(void) DO_TEST_CAPS_VER("disk-vhostuser-numa", "4.2.0"); DO_TEST_CAPS_LATEST("disk-vhostuser-numa"); DO_TEST_CAPS_LATEST("disk-vhostuser"); + DO_TEST_CAPS_ARCH_LATEST_FULL("disk-vhostvdpa", "x86_64", + ARG_VDPA_FD, "/dev/vhost-vdpa-0", 801); DO_TEST_CAPS_LATEST_PARSE_ERROR("disk-device-lun-type-invalid"); DO_TEST_CAPS_LATEST_PARSE_ERROR("disk-attaching-partition-nosupport"); DO_TEST_CAPS_LATEST("disk-usb-device"); diff --git a/tests/testutilsqemu.c b/tests/testutilsqemu.c index bb3e1b52c5..7f8bf4c01c 100644 --- a/tests/testutilsqemu.c +++ b/tests/testutilsqemu.c @@ -709,6 +709,17 @@ testQemuInfoSetArgs(struct testQemuInfo *info, break; } =20 + case ARG_VDPA_FD: { + const char *vdpadev =3D va_arg(argptr, char *); + int vdpafd =3D va_arg(argptr, unsigned int); + + if (!info->args.vdpafds) + info->args.vdpafds =3D g_hash_table_new_full(g_str_hash, g= _str_equal, g_free, NULL); + + g_hash_table_insert(info->args.vdpafds, g_strdup(vdpadev), GIN= T_TO_POINTER(vdpafd)); + break; + } + case ARG_END: default: info->args.invalidarg =3D true; diff --git a/tests/testutilsqemu.h b/tests/testutilsqemu.h index 2d7e2f7779..8398627f71 100644 --- a/tests/testutilsqemu.h +++ b/tests/testutilsqemu.h @@ -49,6 +49,7 @@ typedef enum { ARG_CAPS_VARIANT, ARG_CAPS_HOST_CPU_MODEL, ARG_FD_GROUP, /* name, nfds, fd[0], ... fd[n-1] */ + ARG_VDPA_FD, /* vdpadev, fd */ ARG_END, } testQemuInfoArgName; =20 @@ -85,6 +86,7 @@ struct testQemuArgs { qemuTestCPUDef capsHostCPUModel; int gic; GHashTable *fds; + GHashTable *vdpafds; bool invalidarg; }; =20 --=20 2.41.0