[PATCH] qemu: implement NVMe device emulation support

Zhiyong Ye posted 1 patch 2 years, 11 months ago
Test syntax-check failed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/libvirt tags/patchew/20210513034526.26948-1-yezhiyong@bytedance.com
docs/formatdomain.rst                         | 11 ++++++--
docs/schemas/domaincommon.rng                 |  3 ++-
src/bhyve/bhyve_command.c                     |  1 +
src/bhyve/bhyve_domain.c                      |  1 +
src/conf/domain_conf.c                        |  5 ++++
src/conf/domain_conf.h                        |  1 +
src/conf/domain_validate.c                    |  4 ++-
src/hyperv/hyperv_driver.c                    |  2 ++
src/qemu/qemu_alias.c                         |  1 +
src/qemu/qemu_command.c                       |  8 ++++++
src/qemu/qemu_domain_address.c                | 11 +++++---
src/qemu/qemu_hotplug.c                       |  2 ++
src/qemu/qemu_validate.c                      | 13 ++++++++++
src/util/virutil.c                            |  3 ++-
src/vbox/vbox_common.c                        |  1 +
src/vz/vz_sdk.c                               |  2 ++
src/vz/vz_utils.c                             |  1 +
tests/qemuxml2argvdata/disk-nvme-device.args  | 28 +++++++++++++++++++++
tests/qemuxml2argvdata/disk-nvme-device.xml   | 30 ++++++++++++++++++++++
tests/qemuxml2argvtest.c                      |  1 +
tests/qemuxml2xmloutdata/disk-nvme-device.xml | 36 +++++++++++++++++++++++++++
tests/qemuxml2xmltest.c                       |  1 +
22 files changed, 157 insertions(+), 9 deletions(-)
create mode 100644 tests/qemuxml2argvdata/disk-nvme-device.args
create mode 100644 tests/qemuxml2argvdata/disk-nvme-device.xml
create mode 100644 tests/qemuxml2xmloutdata/disk-nvme-device.xml
[PATCH] qemu: implement NVMe device emulation support
Posted by Zhiyong Ye 2 years, 11 months ago
Implements QEMU support for NVMe device emulation, which was added in
qemu 1.6.0:

https://git.qemu.org/?p=qemu.git;a=commit;h=f3c507adcd7

It can be used by adding elements in XML like this:
<devices>
  ...
  <disk type='file' device='disk'>
    <driver name='qemu' type='raw'/>
    <source file='/var/lib/libvirt/images/nvme.img'/>
    <target dev='nvmea' bus='nvme'/>
    <serial>QEMU-WMAP9A966149</serial>
  </disk>
  ...
</devices>

Signed-off-by: Zhiyong Ye <yezhiyong@bytedance.com>
Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
---
 docs/formatdomain.rst                         | 11 ++++++--
 docs/schemas/domaincommon.rng                 |  3 ++-
 src/bhyve/bhyve_command.c                     |  1 +
 src/bhyve/bhyve_domain.c                      |  1 +
 src/conf/domain_conf.c                        |  5 ++++
 src/conf/domain_conf.h                        |  1 +
 src/conf/domain_validate.c                    |  4 ++-
 src/hyperv/hyperv_driver.c                    |  2 ++
 src/qemu/qemu_alias.c                         |  1 +
 src/qemu/qemu_command.c                       |  8 ++++++
 src/qemu/qemu_domain_address.c                | 11 +++++---
 src/qemu/qemu_hotplug.c                       |  2 ++
 src/qemu/qemu_validate.c                      | 13 ++++++++++
 src/util/virutil.c                            |  3 ++-
 src/vbox/vbox_common.c                        |  1 +
 src/vz/vz_sdk.c                               |  2 ++
 src/vz/vz_utils.c                             |  1 +
 tests/qemuxml2argvdata/disk-nvme-device.args  | 28 +++++++++++++++++++++
 tests/qemuxml2argvdata/disk-nvme-device.xml   | 30 ++++++++++++++++++++++
 tests/qemuxml2argvtest.c                      |  1 +
 tests/qemuxml2xmloutdata/disk-nvme-device.xml | 36 +++++++++++++++++++++++++++
 tests/qemuxml2xmltest.c                       |  1 +
 22 files changed, 157 insertions(+), 9 deletions(-)
 create mode 100644 tests/qemuxml2argvdata/disk-nvme-device.args
 create mode 100644 tests/qemuxml2argvdata/disk-nvme-device.xml
 create mode 100644 tests/qemuxml2xmloutdata/disk-nvme-device.xml

diff --git a/docs/formatdomain.rst b/docs/formatdomain.rst
index fa5c14febc..d7e5299664 100644
--- a/docs/formatdomain.rst
+++ b/docs/formatdomain.rst
@@ -2479,6 +2479,12 @@ paravirtualized driver is specified via the ``disk`` element.
        </source>
        <target dev='vdf' bus='virtio'/>
      </disk>
+     <disk type='file' device='disk'>
+       <driver name='qemu' type='raw'/>
+       <source file='/var/lib/libvirt/images/nvme.img'/>
+       <target dev='nvmea' bus='nvme'/>
+       <serial>QEMU-WMAP9A966149</serial>
+     </disk>
    </devices>
    ...
 
@@ -2880,8 +2886,9 @@ paravirtualized driver is specified via the ``disk`` element.
    name in the guest OS. Treat it as a device ordering hint. The optional
    ``bus`` attribute specifies the type of disk device to emulate; possible
    values are driver specific, with typical values being "ide", "scsi",
-   "virtio", "xen", "usb", "sata", or "sd" :since:`"sd" since 1.1.2` . If
-   omitted, the bus type is inferred from the style of the device name (e.g. a
+   "virtio", "xen", "usb", "sata", "sd" :since:`"sd" since 1.1.2`,
+   or "nvme" :since:`"nvme" since 7.3.0 (QEMU only)`. If omitted,
+   the bus type is inferred from the style of the device name (e.g. a
    device named 'sda' will typically be exported using a SCSI bus). The optional
    attribute ``tray`` indicates the tray status of the removable disks (i.e.
    CDROM or Floppy disk), the value can be either "open" or "closed", defaults
diff --git a/docs/schemas/domaincommon.rng b/docs/schemas/domaincommon.rng
index a2e5c50c1d..47c60d7247 100644
--- a/docs/schemas/domaincommon.rng
+++ b/docs/schemas/domaincommon.rng
@@ -2202,7 +2202,7 @@
 
   <define name="diskTarget">
     <data type="string">
-      <param name="pattern">(ioemu:)?(fd|hd|sd|vd|xvd|ubd)[a-zA-Z0-9_]+</param>
+      <param name="pattern">(ioemu:)?(fd|hd|sd|vd|xvd|ubd|nvme)[a-zA-Z0-9_]+</param>
     </data>
   </define>
   <define name="target">
@@ -2222,6 +2222,7 @@
             <value>uml</value> <!-- NOT USED ANYMORE -->
             <value>sata</value>
             <value>sd</value>
+            <value>nvme</value>
           </choice>
         </attribute>
       </optional>
diff --git a/src/bhyve/bhyve_command.c b/src/bhyve/bhyve_command.c
index f8e0ce5123..d601f15eae 100644
--- a/src/bhyve/bhyve_command.c
+++ b/src/bhyve/bhyve_command.c
@@ -320,6 +320,7 @@ bhyveBuildDiskArgStr(const virDomainDef *def,
     case VIR_DOMAIN_DISK_BUS_USB:
     case VIR_DOMAIN_DISK_BUS_UML:
     case VIR_DOMAIN_DISK_BUS_SD:
+    case VIR_DOMAIN_DISK_BUS_NVME:
     case VIR_DOMAIN_DISK_BUS_LAST:
     default:
         virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
diff --git a/src/bhyve/bhyve_domain.c b/src/bhyve/bhyve_domain.c
index 33e74e2e25..3972b57ba9 100644
--- a/src/bhyve/bhyve_domain.c
+++ b/src/bhyve/bhyve_domain.c
@@ -142,6 +142,7 @@ bhyveDomainDiskDefAssignAddress(struct _bhyveConn *driver,
     case VIR_DOMAIN_DISK_BUS_USB:
     case VIR_DOMAIN_DISK_BUS_UML:
     case VIR_DOMAIN_DISK_BUS_SD:
+    case VIR_DOMAIN_DISK_BUS_NVME:
     case VIR_DOMAIN_DISK_BUS_LAST:
     default:
         break;
diff --git a/src/conf/domain_conf.c b/src/conf/domain_conf.c
index 7044701fac..1c89309f29 100644
--- a/src/conf/domain_conf.c
+++ b/src/conf/domain_conf.c
@@ -350,6 +350,7 @@ VIR_ENUM_IMPL(virDomainDiskBus,
               "uml",
               "sata",
               "sd",
+              "nvme",
 );
 
 VIR_ENUM_IMPL(virDomainDiskCache,
@@ -5356,6 +5357,8 @@ virDomainDiskDefPostParse(virDomainDiskDef *disk,
                 disk->bus = VIR_DOMAIN_DISK_BUS_XEN;
             else if (STRPREFIX(disk->dst, "ubd"))
                 disk->bus = VIR_DOMAIN_DISK_BUS_UML;
+            else if (STRPREFIX(disk->dst, "nvme"))
+                disk->bus = VIR_DOMAIN_DISK_BUS_NVME;
         }
     }
 
@@ -7723,6 +7726,7 @@ virDomainDiskDefAssignAddress(virDomainXMLOption *xmlopt,
     case VIR_DOMAIN_DISK_BUS_USB:
     case VIR_DOMAIN_DISK_BUS_UML:
     case VIR_DOMAIN_DISK_BUS_SD:
+    case VIR_DOMAIN_DISK_BUS_NVME:
     case VIR_DOMAIN_DISK_BUS_LAST:
     default:
         /* Other disk bus's aren't controller based */
@@ -28737,6 +28741,7 @@ virDiskNameToBusDeviceIndex(virDomainDiskDef *disk,
         case VIR_DOMAIN_DISK_BUS_NONE:
         case VIR_DOMAIN_DISK_BUS_SATA:
         case VIR_DOMAIN_DISK_BUS_UML:
+        case VIR_DOMAIN_DISK_BUS_NVME:
         case VIR_DOMAIN_DISK_BUS_LAST:
         default:
             *busIdx = 0;
diff --git a/src/conf/domain_conf.h b/src/conf/domain_conf.h
index 2d5462bb55..b54284ee49 100644
--- a/src/conf/domain_conf.h
+++ b/src/conf/domain_conf.h
@@ -381,6 +381,7 @@ typedef enum {
     VIR_DOMAIN_DISK_BUS_UML,
     VIR_DOMAIN_DISK_BUS_SATA,
     VIR_DOMAIN_DISK_BUS_SD,
+    VIR_DOMAIN_DISK_BUS_NVME,
 
     VIR_DOMAIN_DISK_BUS_LAST
 } virDomainDiskBus;
diff --git a/src/conf/domain_validate.c b/src/conf/domain_validate.c
index 686b9e8d16..0e79698381 100644
--- a/src/conf/domain_validate.c
+++ b/src/conf/domain_validate.c
@@ -227,6 +227,7 @@ virDomainDiskAddressDiskBusCompatibility(virDomainDiskBus bus,
     case VIR_DOMAIN_DISK_BUS_USB:
     case VIR_DOMAIN_DISK_BUS_UML:
     case VIR_DOMAIN_DISK_BUS_SD:
+    case VIR_DOMAIN_DISK_BUS_NVME:
     case VIR_DOMAIN_DISK_BUS_NONE:
     case VIR_DOMAIN_DISK_BUS_LAST:
         return true;
@@ -721,7 +722,8 @@ virDomainDiskDefValidate(const virDomainDef *def,
         !STRPREFIX(disk->dst, "sd") &&
         !STRPREFIX(disk->dst, "vd") &&
         !STRPREFIX(disk->dst, "xvd") &&
-        !STRPREFIX(disk->dst, "ubd")) {
+        !STRPREFIX(disk->dst, "ubd") &&
+        !STRPREFIX(disk->dst, "nvme")) {
         virReportError(VIR_ERR_INTERNAL_ERROR,
                        _("Invalid harddisk device name: %s"), disk->dst);
         return -1;
diff --git a/src/hyperv/hyperv_driver.c b/src/hyperv/hyperv_driver.c
index ff20d5548b..f6fb0a610b 100644
--- a/src/hyperv/hyperv_driver.c
+++ b/src/hyperv/hyperv_driver.c
@@ -970,6 +970,7 @@ hypervDomainAttachStorage(virDomainPtr domain, virDomainDef *def, const char *ho
         case VIR_DOMAIN_DISK_BUS_UML:
         case VIR_DOMAIN_DISK_BUS_SATA:
         case VIR_DOMAIN_DISK_BUS_SD:
+        case VIR_DOMAIN_DISK_BUS_NVME:
         case VIR_DOMAIN_DISK_BUS_LAST:
         default:
             virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("Unsupported controller type"));
@@ -3125,6 +3126,7 @@ hypervDomainAttachDeviceFlags(virDomainPtr domain, const char *xml, unsigned int
         case VIR_DOMAIN_DISK_BUS_UML:
         case VIR_DOMAIN_DISK_BUS_SATA:
         case VIR_DOMAIN_DISK_BUS_SD:
+        case VIR_DOMAIN_DISK_BUS_NVME:
         case VIR_DOMAIN_DISK_BUS_LAST:
         default:
             virReportError(VIR_ERR_INTERNAL_ERROR, "%s", _("Invalid disk bus in definition"));
diff --git a/src/qemu/qemu_alias.c b/src/qemu/qemu_alias.c
index ed47fa335a..b4fd8f9ae6 100644
--- a/src/qemu/qemu_alias.c
+++ b/src/qemu/qemu_alias.c
@@ -254,6 +254,7 @@ qemuAssignDeviceDiskAlias(virDomainDef *def,
         case VIR_DOMAIN_DISK_BUS_XEN:
         case VIR_DOMAIN_DISK_BUS_UML:
         case VIR_DOMAIN_DISK_BUS_SD:
+        case VIR_DOMAIN_DISK_BUS_NVME:
         case VIR_DOMAIN_DISK_BUS_NONE:
         case VIR_DOMAIN_DISK_BUS_LAST:
             break;
diff --git a/src/qemu/qemu_command.c b/src/qemu/qemu_command.c
index d6c5308ef0..14d6bfcd89 100644
--- a/src/qemu/qemu_command.c
+++ b/src/qemu/qemu_command.c
@@ -1607,6 +1607,7 @@ qemuCheckIOThreads(const virDomainDef *def,
     case VIR_DOMAIN_DISK_BUS_UML:
     case VIR_DOMAIN_DISK_BUS_SATA:
     case VIR_DOMAIN_DISK_BUS_SD:
+    case VIR_DOMAIN_DISK_BUS_NVME:
     case VIR_DOMAIN_DISK_BUS_NONE:
     case VIR_DOMAIN_DISK_BUS_LAST:
         virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
@@ -1847,6 +1848,13 @@ qemuBuildDiskDeviceStr(const virDomainDef *def,
         virBufferAsprintf(&opt, "floppy,unit=%d", disk->info.addr.drive.unit);
         break;
 
+    case VIR_DOMAIN_DISK_BUS_NVME:
+        virBufferAddLit(&opt, "nvme");
+
+        if (qemuBuildDeviceAddressStr(&opt, def, &disk->info) < 0)
+            return NULL;
+        break;
+
     case VIR_DOMAIN_DISK_BUS_XEN:
     case VIR_DOMAIN_DISK_BUS_UML:
     case VIR_DOMAIN_DISK_BUS_SD:
diff --git a/src/qemu/qemu_domain_address.c b/src/qemu/qemu_domain_address.c
index a11e40d9b2..7249328b80 100644
--- a/src/qemu/qemu_domain_address.c
+++ b/src/qemu/qemu_domain_address.c
@@ -778,7 +778,6 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDef *dev,
     case VIR_DOMAIN_DEVICE_DISK:
         switch ((virDomainDiskBus) dev->data.disk->bus) {
         case VIR_DOMAIN_DISK_BUS_VIRTIO:
-            /* only virtio disks use PCI */
             switch ((virDomainDiskModel) dev->data.disk->model) {
             case VIR_DOMAIN_DISK_MODEL_VIRTIO_TRANSITIONAL:
                 /* Transitional devices only work in conventional PCI slots */
@@ -792,6 +791,9 @@ qemuDomainDeviceCalculatePCIConnectFlags(virDomainDeviceDef *dev,
             }
             return 0;
 
+        case VIR_DOMAIN_DISK_BUS_NVME:
+            return pciFlags;
+
         case VIR_DOMAIN_DISK_BUS_IDE:
         case VIR_DOMAIN_DISK_BUS_FDC:
         case VIR_DOMAIN_DISK_BUS_SCSI:
@@ -2235,10 +2237,11 @@ qemuDomainAssignDevicePCISlots(virDomainDef *def,
         }
     }
 
-    /* Disks (VirtIO only for now) */
+    /* Disks (VirtIO and NVMe-device only for now) */
     for (i = 0; i < def->ndisks; i++) {
-        /* Only VirtIO disks use PCI addrs */
-        if (def->disks[i]->bus != VIR_DOMAIN_DISK_BUS_VIRTIO)
+        /* Only VirtIO adn NVMe-device disks use PCI addrs */
+        if (def->disks[i]->bus != VIR_DOMAIN_DISK_BUS_VIRTIO &&
+            def->disks[i]->bus != VIR_DOMAIN_DISK_BUS_NVME)
             continue;
 
         /* don't touch s390 devices */
diff --git a/src/qemu/qemu_hotplug.c b/src/qemu/qemu_hotplug.c
index a64cddb9e7..862f30572d 100644
--- a/src/qemu/qemu_hotplug.c
+++ b/src/qemu/qemu_hotplug.c
@@ -1092,6 +1092,7 @@ qemuDomainAttachDeviceDiskLiveInternal(virQEMUDriver *driver,
         /* Note that SD card hotplug support should be added only once
          * they support '-device' (don't require -drive only).
          * See also: qemuDiskBusIsSD */
+    case VIR_DOMAIN_DISK_BUS_NVME:
     case VIR_DOMAIN_DISK_BUS_NONE:
     case VIR_DOMAIN_DISK_BUS_LAST:
         virReportError(VIR_ERR_OPERATION_UNSUPPORTED,
@@ -5312,6 +5313,7 @@ qemuDomainDetachPrepDisk(virDomainObj *vm,
         case VIR_DOMAIN_DISK_BUS_UML:
         case VIR_DOMAIN_DISK_BUS_SATA:
         case VIR_DOMAIN_DISK_BUS_SD:
+        case VIR_DOMAIN_DISK_BUS_NVME:
             virReportError(VIR_ERR_OPERATION_UNSUPPORTED, "%s",
                            _("This type of disk cannot be hot unplugged"));
             return -1;
diff --git a/src/qemu/qemu_validate.c b/src/qemu/qemu_validate.c
index 141203f979..73ce7cc617 100644
--- a/src/qemu/qemu_validate.c
+++ b/src/qemu/qemu_validate.c
@@ -2721,6 +2721,19 @@ qemuValidateDomainDeviceDefDiskFrontend(const virDomainDiskDef *disk,
 
         break;
 
+    case VIR_DOMAIN_DISK_BUS_NVME:
+        if (disk->info.type != VIR_DOMAIN_DEVICE_ADDRESS_TYPE_PCI) {
+            virReportError(VIR_ERR_INTERNAL_ERROR, "%s",
+                           _("unexpected address type for nvme disk"));
+            return -1;
+        }
+        if (!disk->serial) {
+            virReportError(VIR_ERR_INTERNAL_ERROR, "%s",
+                           _("serial property must be specified for nvme disk"));
+            return -1;
+        }
+        break;
+
     case VIR_DOMAIN_DISK_BUS_XEN:
     case VIR_DOMAIN_DISK_BUS_SD:
     case VIR_DOMAIN_DISK_BUS_NONE:
diff --git a/src/util/virutil.c b/src/util/virutil.c
index 3f49a469e5..a586b94c9a 100644
--- a/src/util/virutil.c
+++ b/src/util/virutil.c
@@ -369,7 +369,8 @@ int virDiskNameParse(const char *name, int *disk, int *partition)
     const char *ptr = NULL;
     char *rem;
     int idx = 0;
-    static char const* const drive_prefix[] = {"fd", "hd", "vd", "sd", "xvd", "ubd"};
+    static char const* const drive_prefix[] = {"fd", "hd", "vd", "sd",
+                                               "xvd", "ubd", "nvme"};
     size_t i;
     size_t n_digits;
 
diff --git a/src/vbox/vbox_common.c b/src/vbox/vbox_common.c
index 1ca521321c..11f057236c 100644
--- a/src/vbox/vbox_common.c
+++ b/src/vbox/vbox_common.c
@@ -1154,6 +1154,7 @@ vboxAttachDrives(virDomainDef *def, struct _vboxDriver *data, IMachine *machine)
         case VIR_DOMAIN_DISK_BUS_USB:
         case VIR_DOMAIN_DISK_BUS_UML:
         case VIR_DOMAIN_DISK_BUS_SD:
+        case VIR_DOMAIN_DISK_BUS_NVME:
         case VIR_DOMAIN_DISK_BUS_NONE:
         case VIR_DOMAIN_DISK_BUS_LAST:
             virReportError(VIR_ERR_CONFIG_UNSUPPORTED,
diff --git a/src/vz/vz_sdk.c b/src/vz/vz_sdk.c
index e09950812d..0a804aa284 100644
--- a/src/vz/vz_sdk.c
+++ b/src/vz/vz_sdk.c
@@ -3469,6 +3469,7 @@ static int prlsdkConfigureDisk(struct _vzDriver *driver,
     case VIR_DOMAIN_DISK_BUS_USB:
     case VIR_DOMAIN_DISK_BUS_UML:
     case VIR_DOMAIN_DISK_BUS_SD:
+    case VIR_DOMAIN_DISK_BUS_NVME:
     case VIR_DOMAIN_DISK_BUS_LAST:
     default:
         virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
@@ -4354,6 +4355,7 @@ prlsdkGetBlockStats(PRL_HANDLE sdkstats,
         case VIR_DOMAIN_DISK_BUS_USB:
         case VIR_DOMAIN_DISK_BUS_UML:
         case VIR_DOMAIN_DISK_BUS_SD:
+        case VIR_DOMAIN_DISK_BUS_NVME:
         case VIR_DOMAIN_DISK_BUS_LAST:
         default:
             virReportError(VIR_ERR_INTERNAL_ERROR,
diff --git a/src/vz/vz_utils.c b/src/vz/vz_utils.c
index 8fed875281..f7e4d174cf 100644
--- a/src/vz/vz_utils.c
+++ b/src/vz/vz_utils.c
@@ -244,6 +244,7 @@ vzCheckDiskAddressDriveUnsupportedParams(virDomainDiskDef *disk)
     case VIR_DOMAIN_DISK_BUS_USB:
     case VIR_DOMAIN_DISK_BUS_UML:
     case VIR_DOMAIN_DISK_BUS_SD:
+    case VIR_DOMAIN_DISK_BUS_NVME:
     case VIR_DOMAIN_DISK_BUS_LAST:
     default:
         virReportError(VIR_ERR_CONFIG_UNSUPPORTED, "%s",
diff --git a/tests/qemuxml2argvdata/disk-nvme-device.args b/tests/qemuxml2argvdata/disk-nvme-device.args
new file mode 100644
index 0000000000..594e1dea31
--- /dev/null
+++ b/tests/qemuxml2argvdata/disk-nvme-device.args
@@ -0,0 +1,28 @@
+LC_ALL=C \
+PATH=/bin \
+HOME=/tmp/lib/domain--1-QEMUGuest1 \
+USER=test \
+LOGNAME=test \
+XDG_DATA_HOME=/tmp/lib/domain--1-QEMUGuest1/.local/share \
+XDG_CACHE_HOME=/tmp/lib/domain--1-QEMUGuest1/.cache \
+XDG_CONFIG_HOME=/tmp/lib/domain--1-QEMUGuest1/.config \
+QEMU_AUDIO_DRV=none \
+/usr/bin/qemu-system-i386 \
+-name QEMUGuest1 \
+-S \
+-machine pc,accel=tcg,usb=off,dump-guest-core=off \
+-m 214 \
+-realtime mlock=off \
+-smp 1,sockets=1,cores=1,threads=1 \
+-uuid c7a5fdbd-edaf-9455-926a-d65c16db1809 \
+-display none \
+-no-user-config \
+-nodefaults \
+-chardev socket,id=charmonitor,path=/tmp/lib/domain--1-QEMUGuest1/monitor.sock,server=on,wait=off \
+-mon chardev=charmonitor,id=monitor,mode=control \
+-rtc base=utc \
+-no-shutdown \
+-no-acpi \
+-usb \
+-drive file=/tmp/data.img,format=raw,if=none,id=drive-nvme-disk0 \
+-device nvme,bus=pci.0,addr=0x2,drive=drive-nvme-disk0,id=nvme-disk0,bootindex=1,serial=QEMU-WMAP9A966149
diff --git a/tests/qemuxml2argvdata/disk-nvme-device.xml b/tests/qemuxml2argvdata/disk-nvme-device.xml
new file mode 100644
index 0000000000..18d9632f20
--- /dev/null
+++ b/tests/qemuxml2argvdata/disk-nvme-device.xml
@@ -0,0 +1,30 @@
+<domain type='qemu'>
+  <name>QEMUGuest1</name>
+  <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid>
+  <memory unit='KiB'>219136</memory>
+  <currentMemory unit='KiB'>219136</currentMemory>
+  <vcpu placement='static'>1</vcpu>
+  <os>
+    <type arch='i686' machine='pc'>hvm</type>
+    <boot dev='hd'/>
+  </os>
+  <clock offset='utc'/>
+  <on_poweroff>destroy</on_poweroff>
+  <on_reboot>restart</on_reboot>
+  <on_crash>destroy</on_crash>
+  <devices>
+    <emulator>/usr/bin/qemu-system-i386</emulator>
+    <disk type='file' device='disk'>
+      <driver name='qemu' type='raw'/>
+      <source file='/tmp/data.img'/>
+      <target dev='nvmea' bus='nvme'/>
+      <serial>QEMU-WMAP9A966149</serial>
+    </disk>
+    <controller type='usb' index='0'/>
+    <controller type='ide' index='0'/>
+    <controller type='pci' index='0' model='pci-root'/>
+    <input type='mouse' bus='ps2'/>
+    <input type='keyboard' bus='ps2'/>
+    <memballoon model='none'/>
+  </devices>
+</domain>
diff --git a/tests/qemuxml2argvtest.c b/tests/qemuxml2argvtest.c
index a9dafe226e..681158c73b 100644
--- a/tests/qemuxml2argvtest.c
+++ b/tests/qemuxml2argvtest.c
@@ -1493,6 +1493,7 @@ mymain(void)
     DO_TEST_CAPS_ARCH_VER("disk-arm-virtio-sd", "aarch64", "4.0.0");
     DO_TEST_CAPS_ARCH_LATEST("disk-arm-virtio-sd", "aarch64");
 
+    DO_TEST("disk-nvme-device", NONE);
     DO_TEST("graphics-egl-headless",
             QEMU_CAPS_DEVICE_CIRRUS_VGA);
     DO_TEST_CAPS_LATEST("graphics-egl-headless");
diff --git a/tests/qemuxml2xmloutdata/disk-nvme-device.xml b/tests/qemuxml2xmloutdata/disk-nvme-device.xml
new file mode 100644
index 0000000000..e016164d28
--- /dev/null
+++ b/tests/qemuxml2xmloutdata/disk-nvme-device.xml
@@ -0,0 +1,36 @@
+<domain type='qemu'>
+  <name>QEMUGuest1</name>
+  <uuid>c7a5fdbd-edaf-9455-926a-d65c16db1809</uuid>
+  <memory unit='KiB'>219136</memory>
+  <currentMemory unit='KiB'>219136</currentMemory>
+  <vcpu placement='static'>1</vcpu>
+  <os>
+    <type arch='i686' machine='pc'>hvm</type>
+    <boot dev='hd'/>
+  </os>
+  <clock offset='utc'/>
+  <on_poweroff>destroy</on_poweroff>
+  <on_reboot>restart</on_reboot>
+  <on_crash>destroy</on_crash>
+  <devices>
+    <emulator>/usr/bin/qemu-system-i386</emulator>
+    <disk type='file' device='disk'>
+      <driver name='qemu' type='raw'/>
+      <source file='/tmp/data.img'/>
+      <target dev='nvmea' bus='nvme'/>
+      <serial>QEMU-WMAP9A966149</serial>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
+    </disk>
+    <controller type='usb' index='0'>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
+    </controller>
+    <controller type='ide' index='0'>
+      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
+    </controller>
+    <controller type='pci' index='0' model='pci-root'/>
+    <input type='mouse' bus='ps2'/>
+    <input type='keyboard' bus='ps2'/>
+    <audio id='1' type='none'/>
+    <memballoon model='none'/>
+  </devices>
+</domain>
diff --git a/tests/qemuxml2xmltest.c b/tests/qemuxml2xmltest.c
index 7af6f90aee..f36819cd73 100644
--- a/tests/qemuxml2xmltest.c
+++ b/tests/qemuxml2xmltest.c
@@ -336,6 +336,7 @@ mymain(void)
     DO_TEST("disk-mirror-old", NONE);
     DO_TEST("disk-mirror", NONE);
     DO_TEST("disk-active-commit", NONE);
+    DO_TEST("disk-nvme-device", NONE);
     DO_TEST("graphics-listen-network",
             QEMU_CAPS_DEVICE_CIRRUS_VGA,
             QEMU_CAPS_VNC);
-- 
2.11.0

Re: [PATCH] qemu: implement NVMe device emulation support
Posted by Peter Krempa 2 years, 11 months ago
On Thu, May 13, 2021 at 11:45:25 +0800, Zhiyong Ye wrote:
> Implements QEMU support for NVMe device emulation, which was added in
> qemu 1.6.0:
> 
> https://git.qemu.org/?p=qemu.git;a=commit;h=f3c507adcd7
> 
> It can be used by adding elements in XML like this:
> <devices>
>   ...
>   <disk type='file' device='disk'>
>     <driver name='qemu' type='raw'/>
>     <source file='/var/lib/libvirt/images/nvme.img'/>
>     <target dev='nvmea' bus='nvme'/>
>     <serial>QEMU-WMAP9A966149</serial>
>   </disk>
>   ...
> </devices>
> 
> Signed-off-by: Zhiyong Ye <yezhiyong@bytedance.com>
> Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
> ---
>  docs/formatdomain.rst                         | 11 ++++++--
>  docs/schemas/domaincommon.rng                 |  3 ++-
>  src/bhyve/bhyve_command.c                     |  1 +
>  src/bhyve/bhyve_domain.c                      |  1 +
>  src/conf/domain_conf.c                        |  5 ++++
>  src/conf/domain_conf.h                        |  1 +
>  src/conf/domain_validate.c                    |  4 ++-
>  src/hyperv/hyperv_driver.c                    |  2 ++
>  src/qemu/qemu_alias.c                         |  1 +
>  src/qemu/qemu_command.c                       |  8 ++++++
>  src/qemu/qemu_domain_address.c                | 11 +++++---
>  src/qemu/qemu_hotplug.c                       |  2 ++
>  src/qemu/qemu_validate.c                      | 13 ++++++++++
>  src/util/virutil.c                            |  3 ++-
>  src/vbox/vbox_common.c                        |  1 +
>  src/vz/vz_sdk.c                               |  2 ++
>  src/vz/vz_utils.c                             |  1 +
>  tests/qemuxml2argvdata/disk-nvme-device.args  | 28 +++++++++++++++++++++
>  tests/qemuxml2argvdata/disk-nvme-device.xml   | 30 ++++++++++++++++++++++
>  tests/qemuxml2argvtest.c                      |  1 +
>  tests/qemuxml2xmloutdata/disk-nvme-device.xml | 36 +++++++++++++++++++++++++++
>  tests/qemuxml2xmltest.c                       |  1 +
>  22 files changed, 157 insertions(+), 9 deletions(-)
>  create mode 100644 tests/qemuxml2argvdata/disk-nvme-device.args
>  create mode 100644 tests/qemuxml2argvdata/disk-nvme-device.xml
>  create mode 100644 tests/qemuxml2xmloutdata/disk-nvme-device.xml

Hi, firstly it's customary in libvirt to split patches at least into the
following categories:

conf/ and XML additions

qemu/qemu_capabilities as separate change

and then required changes to the qemu driver possibly also split if
there are appropriate points.

Please note that after every commit the tree MUST compile cleanly
although obviously the new feature is not required to work properly
until the end of the series.

Additionally there was a discussion some time ago where we've discussed
some of the technicalies:

https://listman.redhat.com/archives/libvir-list/2020-November/msg00415.html

We've discussed there that the NVMe device should be more like a
'<controller>' and '<disk>' would map to a signle namespace on the
controller. According to that thread NVMe actually supports that.