From: hongleiwang <honglei.wang@smartx.com>

QEMU has supported nvme disk emulation for a long time,
see: https://qemu-project.gitlab.io/qemu/system/devices/nvme.html.

The following patches introduce nvme-ns disk bus type:
A disk with nvme-ns as bus is represented as an nvme namespace
and needs to be attached to an nvme controller. In XML, it can be
used like this:
<devices>
  ...
  <disk type='file' device='disk'>
    <driver name='qemu' type='raw'/>
    <source file='/tmp/data.img'/>
    <target dev='nvmensa' bus='nvme-ns'/>
    <address type='drive' controller='0' bus='0' target='0' unit='0'/>
  </disk>
  <controller type='nvme' index='0'>
    <serial>nvme-controller-serial-value</serial>
    <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
  </controller>
  ...
</devices>

Signed-off-by: ray <honglei.wang@smartx.com>

---
Compared to patch v1, this version removes the nvme bus type implementation
and keeps only the nvme controller + nvme-ns bus approach.

ray (5):
  qemu: Add support for NVMe namespace disk bus type
  qemu_capabilities: Add support for nvme-ns bus capabilities
  schema: Add nvme controller and nvme-ns bus defination
  tests: Add test case for nvme-ns device configuration
  NEWS: Document qemu nvme disk emulation feature

 NEWS.rst                                           | 17 +++++++++
 src/conf/domain_conf.c                             | 39 ++++++++++++++++++++
 src/conf/domain_conf.h                             |  7 ++++
 src/conf/domain_postparse.c                        |  2 ++
 src/conf/domain_validate.c                         |  4 ++-
 src/conf/schemas/domaincommon.rng                  | 11 +++++-
 src/conf/virconftypes.h                            |  2 ++
 src/hyperv/hyperv_driver.c                         |  2 ++
 src/qemu/qemu_alias.c                              |  1 +
 src/qemu/qemu_capabilities.c                       |  5 +++
 src/qemu/qemu_capabilities.h                       |  1 +
 src/qemu/qemu_command.c                            | 26 ++++++++++++++
 src/qemu/qemu_domain_address.c                     |  5 +++
 src/qemu/qemu_hotplug.c                            | 14 ++++++--
 src/qemu/qemu_postparse.c                          |  1 +
 src/qemu/qemu_validate.c                           | 18 ++++++++++
 src/test/test_driver.c                             |  2 ++
 src/util/virutil.c                                 |  2 +-
 src/vbox/vbox_common.c                             |  2 ++
 src/vmx/vmx.c                                      |  1 +
 .../qemu_10.0.0-q35.x86_64+amdsev.xml              |  1 +
 tests/domaincapsdata/qemu_10.0.0-q35.x86_64.xml    |  1 +
 .../qemu_10.0.0-tcg.x86_64+amdsev.xml              |  1 +
 tests/domaincapsdata/qemu_10.0.0-tcg.x86_64.xml    |  1 +
 tests/domaincapsdata/qemu_10.0.0.s390x.xml         |  1 +
 tests/domaincapsdata/qemu_10.0.0.x86_64+amdsev.xml |  1 +
 tests/domaincapsdata/qemu_10.0.0.x86_64.xml        |  1 +
 tests/domaincapsdata/qemu_4.2.0-virt.aarch64.xml   |  1 +
 tests/domaincapsdata/qemu_4.2.0.aarch64.xml        |  1 +
 tests/domaincapsdata/qemu_4.2.0.ppc64.xml          |  1 +
 .../domaincapsdata/qemu_5.0.0-tcg-virt.riscv64.xml |  1 +
 tests/domaincapsdata/qemu_5.0.0-virt.aarch64.xml   |  1 +
 tests/domaincapsdata/qemu_5.0.0-virt.riscv64.xml   |  1 +
 tests/domaincapsdata/qemu_5.0.0.aarch64.xml        |  1 +
 tests/domaincapsdata/qemu_5.0.0.ppc64.xml          |  1 +
 tests/domaincapsdata/qemu_5.1.0.sparc.xml          |  1 +
 tests/domaincapsdata/qemu_6.2.0-q35.x86_64.xml     |  1 +
 tests/domaincapsdata/qemu_6.2.0-tcg.x86_64.xml     |  1 +
 tests/domaincapsdata/qemu_6.2.0.ppc64.xml          |  1 +
 tests/domaincapsdata/qemu_6.2.0.x86_64.xml         |  1 +
 tests/domaincapsdata/qemu_7.0.0-q35.x86_64.xml     |  1 +
 tests/domaincapsdata/qemu_7.0.0-tcg.x86_64.xml     |  1 +
 tests/domaincapsdata/qemu_7.0.0.ppc64.xml          |  1 +
 tests/domaincapsdata/qemu_7.0.0.x86_64.xml         |  1 +
 tests/domaincapsdata/qemu_7.1.0-q35.x86_64.xml     |  1 +
 tests/domaincapsdata/qemu_7.1.0-tcg.x86_64.xml     |  1 +
 tests/domaincapsdata/qemu_7.1.0.ppc64.xml          |  1 +
 tests/domaincapsdata/qemu_7.1.0.x86_64.xml         |  1 +
 tests/domaincapsdata/qemu_7.2.0-hvf.x86_64+hvf.xml |  1 +
 tests/domaincapsdata/qemu_7.2.0-q35.x86_64.xml     |  1 +
 tests/domaincapsdata/qemu_7.2.0-tcg.x86_64+hvf.xml |  1 +
 tests/domaincapsdata/qemu_7.2.0-tcg.x86_64.xml     |  1 +
 tests/domaincapsdata/qemu_7.2.0.ppc.xml            |  1 +
 tests/domaincapsdata/qemu_7.2.0.x86_64.xml         |  1 +
 tests/domaincapsdata/qemu_8.0.0-q35.x86_64.xml     |  1 +
 tests/domaincapsdata/qemu_8.0.0-tcg.x86_64.xml     |  1 +
 tests/domaincapsdata/qemu_8.0.0.x86_64.xml         |  1 +
 tests/domaincapsdata/qemu_8.1.0-q35.x86_64.xml     |  1 +
 tests/domaincapsdata/qemu_8.1.0-tcg.x86_64.xml     |  1 +
 tests/domaincapsdata/qemu_8.1.0.x86_64.xml         |  1 +
 tests/domaincapsdata/qemu_8.2.0-q35.x86_64.xml     |  1 +
 .../qemu_8.2.0-tcg-virt.loongarch64.xml            |  1 +
 tests/domaincapsdata/qemu_8.2.0-tcg.x86_64.xml     |  1 +
 tests/domaincapsdata/qemu_8.2.0-virt.aarch64.xml   |  1 +
 .../domaincapsdata/qemu_8.2.0-virt.loongarch64.xml |  1 +
 tests/domaincapsdata/qemu_8.2.0.aarch64.xml        |  1 +
 tests/domaincapsdata/qemu_8.2.0.armv7l.xml         |  1 +
 tests/domaincapsdata/qemu_8.2.0.s390x.xml          |  1 +
 tests/domaincapsdata/qemu_8.2.0.x86_64.xml         |  1 +
 tests/domaincapsdata/qemu_9.0.0-q35.x86_64.xml     |  1 +
 tests/domaincapsdata/qemu_9.0.0-tcg.x86_64.xml     |  1 +
 tests/domaincapsdata/qemu_9.0.0.x86_64.xml         |  1 +
 tests/domaincapsdata/qemu_9.1.0-q35.x86_64.xml     |  1 +
 .../domaincapsdata/qemu_9.1.0-tcg-virt.riscv64.xml |  1 +
 tests/domaincapsdata/qemu_9.1.0-tcg.x86_64.xml     |  1 +
 tests/domaincapsdata/qemu_9.1.0-virt.riscv64.xml   |  1 +
 tests/domaincapsdata/qemu_9.1.0.s390x.xml          |  1 +
 tests/domaincapsdata/qemu_9.1.0.x86_64.xml         |  1 +
 .../domaincapsdata/qemu_9.2.0-hvf.aarch64+hvf.xml  |  1 +
 .../qemu_9.2.0-q35.x86_64+amdsev.xml               |  1 +
 tests/domaincapsdata/qemu_9.2.0-q35.x86_64.xml     |  1 +
 .../qemu_9.2.0-tcg.x86_64+amdsev.xml               |  1 +
 tests/domaincapsdata/qemu_9.2.0-tcg.x86_64.xml     |  1 +
 tests/domaincapsdata/qemu_9.2.0.s390x.xml          |  1 +
 tests/domaincapsdata/qemu_9.2.0.x86_64+amdsev.xml  |  1 +
 tests/domaincapsdata/qemu_9.2.0.x86_64.xml         |  1 +
 tests/qemucapabilitiesdata/caps_10.0.0_s390x.xml   |  1 +
 .../caps_10.0.0_x86_64+amdsev.xml                  |  1 +
 tests/qemucapabilitiesdata/caps_10.0.0_x86_64.xml  |  1 +
 tests/qemucapabilitiesdata/caps_6.2.0_ppc64.xml    |  1 +
 tests/qemucapabilitiesdata/caps_6.2.0_x86_64.xml   |  1 +
 tests/qemucapabilitiesdata/caps_7.0.0_ppc64.xml    |  1 +
 tests/qemucapabilitiesdata/caps_7.0.0_x86_64.xml   |  1 +
 tests/qemucapabilitiesdata/caps_7.1.0_ppc64.xml    |  1 +
 tests/qemucapabilitiesdata/caps_7.1.0_x86_64.xml   |  1 +
 tests/qemucapabilitiesdata/caps_7.2.0_ppc.xml      |  1 +
 .../qemucapabilitiesdata/caps_7.2.0_x86_64+hvf.xml |  1 +
 tests/qemucapabilitiesdata/caps_7.2.0_x86_64.xml   |  1 +
 tests/qemucapabilitiesdata/caps_8.0.0_x86_64.xml   |  1 +
 tests/qemucapabilitiesdata/caps_8.1.0_x86_64.xml   |  1 +
 tests/qemucapabilitiesdata/caps_8.2.0_aarch64.xml  |  1 +
 tests/qemucapabilitiesdata/caps_8.2.0_armv7l.xml   |  1 +
 .../caps_8.2.0_loongarch64.xml                     |  1 +
 tests/qemucapabilitiesdata/caps_8.2.0_s390x.xml    |  1 +
 tests/qemucapabilitiesdata/caps_8.2.0_x86_64.xml   |  1 +
 tests/qemucapabilitiesdata/caps_9.0.0_x86_64.xml   |  1 +
 tests/qemucapabilitiesdata/caps_9.1.0_riscv64.xml  |  1 +
 tests/qemucapabilitiesdata/caps_9.1.0_s390x.xml    |  1 +
 tests/qemucapabilitiesdata/caps_9.1.0_x86_64.xml   |  1 +
 .../caps_9.2.0_aarch64+hvf.xml                     |  1 +
 tests/qemucapabilitiesdata/caps_9.2.0_s390x.xml    |  1 +
 .../caps_9.2.0_x86_64+amdsev.xml                   |  1 +
 tests/qemucapabilitiesdata/caps_9.2.0_x86_64.xml   |  1 +
 .../disk-nvme-ns-device.x86_64-latest.args         | 36 +++++++++++++++++++
 .../disk-nvme-ns-device.x86_64-latest.xml          | 42 ++++++++++++++++++++++
 tests/qemuxmlconfdata/disk-nvme-ns-device.xml      | 41 +++++++++++++++++++++
 tests/qemuxmlconftest.c                            |  1 +
 117 files changed, 370 insertions(+), 5 deletions(-)
 create mode 100644 tests/qemuxmlconfdata/disk-nvme-ns-device.x86_64-latest.args
 create mode 100644 tests/qemuxmlconfdata/disk-nvme-ns-device.x86_64-latest.xml
 create mode 100644 tests/qemuxmlconfdata/disk-nvme-ns-device.xml

-- 
2.11.0

On Sun, Apr 27, 2025 at 07:48:02PM +0800, honglei.wang@smartx.com wrote:
>From: hongleiwang <honglei.wang@smartx.com>
>
>QEMU has supported nvme disk emulation for a long time,
>see: https://qemu-project.gitlab.io/qemu/system/devices/nvme.html.
>
>The following patches introduce nvme-ns disk bus type:

Thanks for the v2.  I have some suggestions which I'd like some more
feedback on, which might be controversial, hence the added Cc's for
people who replied to any of the versions.

>A disk with nvme-ns as bus is represented as an nvme namespace
>and needs to be attached to an nvme controller. In XML, it can be
>used like this:
><devices>
>  ...
>  <disk type='file' device='disk'>
>    <driver name='qemu' type='raw'/>
>    <source file='/tmp/data.img'/>
>    <target dev='nvmensa' bus='nvme-ns'/>

Since we're not using the bus='nvme', what if we abandoned the qemu
naming and not specify the -ns; on top of that it would be nicer to have
the device naming same as it actually happens on Linux.  That is kind of
biased, but we already do that for other disk types.

The above would then become:

     <target dev='nvme0n1' bus='nvme'/>

Because that means a slightly bigger change to how we do things
currently (bunch of the code depends on the original numbering) I'd be
willing to submit the XML part of this series with that change.  One of
the reasons is that I also need it for support in a different driver.

Let me know what you think.

Martin

On Wed, May 07, 2025 at 15:56:52 +0200, Martin Kletzander wrote:
> On Sun, Apr 27, 2025 at 07:48:02PM +0800, honglei.wang@smartx.com wrote:
> > From: hongleiwang <honglei.wang@smartx.com>
> > 
> > QEMU has supported nvme disk emulation for a long time,
> > see: https://qemu-project.gitlab.io/qemu/system/devices/nvme.html.
> > 
> > The following patches introduce nvme-ns disk bus type:
> 
> Thanks for the v2.  I have some suggestions which I'd like some more
> feedback on, which might be controversial, hence the added Cc's for
> people who replied to any of the versions.
> 
> > A disk with nvme-ns as bus is represented as an nvme namespace
> > and needs to be attached to an nvme controller. In XML, it can be
> > used like this:
> > <devices>
> >  ...
> >  <disk type='file' device='disk'>
> >    <driver name='qemu' type='raw'/>
> >    <source file='/tmp/data.img'/>
> >    <target dev='nvmensa' bus='nvme-ns'/>
> 
> Since we're not using the bus='nvme', what if we abandoned the qemu
> naming and not specify the -ns; on top of that it would be nicer to have
> the device naming same as it actually happens on Linux.  That is kind of
> biased, but we already do that for other disk types.

Keep in mind that we explicitly document that 'dev' may not match what
the device becomes in the guest ...

> 
> The above would then become:
> 
>     <target dev='nvme0n1' bus='nvme'/>

... in this case it could be more problematic because the disk
target/dev doesn't force you to put nvme0n1 and nvme0n2 on the same
controller nor does the namespace bit need to correspond to the
namespace id.

Yes it is true that with other buses you still can mix stuff up if you
configure it explicitly.

> Because that means a slightly bigger change to how we do things
> currently (bunch of the code depends on the original numbering) I'd be
> willing to submit the XML part of this series with that change.  One
> of the reasons is that I also need it for support in a different
> driver.

I guess that is fair. I still think that just doing 'nvmea', 'nvmeb',
'nvmec' etc should be sufficient here, but the code already has
provisions for partition numbers so this shouldn't be much different.

On Wed, May 07, 2025 at 04:43:29PM +0200, Peter Krempa wrote:
>On Wed, May 07, 2025 at 15:56:52 +0200, Martin Kletzander wrote:
>> On Sun, Apr 27, 2025 at 07:48:02PM +0800, honglei.wang@smartx.com wrote:
>> > From: hongleiwang <honglei.wang@smartx.com>
>> >
>> > QEMU has supported nvme disk emulation for a long time,
>> > see: https://qemu-project.gitlab.io/qemu/system/devices/nvme.html.
>> >
>> > The following patches introduce nvme-ns disk bus type:
>>
>> Thanks for the v2.  I have some suggestions which I'd like some more
>> feedback on, which might be controversial, hence the added Cc's for
>> people who replied to any of the versions.
>>
>> > A disk with nvme-ns as bus is represented as an nvme namespace
>> > and needs to be attached to an nvme controller. In XML, it can be
>> > used like this:
>> > <devices>
>> >  ...
>> >  <disk type='file' device='disk'>
>> >    <driver name='qemu' type='raw'/>
>> >    <source file='/tmp/data.img'/>
>> >    <target dev='nvmensa' bus='nvme-ns'/>
>>
>> Since we're not using the bus='nvme', what if we abandoned the qemu
>> naming and not specify the -ns; on top of that it would be nicer to have
>> the device naming same as it actually happens on Linux.  That is kind of
>> biased, but we already do that for other disk types.
>
>Keep in mind that we explicitly document that 'dev' may not match what
>the device becomes in the guest ...
>

Yes, I meant "naming it the same way" but specifically not "guaranteeing
it will match" =)

>>
>> The above would then become:
>>
>>     <target dev='nvme0n1' bus='nvme'/>
>
>... in this case it could be more problematic because the disk
>target/dev doesn't force you to put nvme0n1 and nvme0n2 on the same
>controller nor does the namespace bit need to correspond to the
>namespace id.
>

It does not, but I see our dev= naming as a shortcut for assigning
addresses.  Of course, that might be a very biased view due to knowing
how the code handles that, but still, the expectation should not be any
different to the expectation of what happens when you use dev='vdzz'.

>Yes it is true that with other buses you still can mix stuff up if you
>configure it explicitly.
>
>> Because that means a slightly bigger change to how we do things
>> currently (bunch of the code depends on the original numbering) I'd be
>> willing to submit the XML part of this series with that change.  One
>> of the reasons is that I also need it for support in a different
>> driver.
>
>I guess that is fair. I still think that just doing 'nvmea', 'nvmeb',
>'nvmec' etc should be sufficient here, but the code already has
>provisions for partition numbers so this shouldn't be much different.
>

The part of the code that needs changing to accommodate the nvme
handling would basically be special-cased with a is_this_nvme()
condition, but I, personally, see "nvme3n3" as an improvement compared
to "nvmeadr".  OTOH I'm fine with both because how often will I have to
type it manually...

Ideally, we should use names like nvme0n1, which follow the typical naming convention for NVMe devices (even if they don't necessarily match the actual device names inside the guest). However, using this approach in libvirt would tightly couple the device name with the controller relationship, meaning we'd need to handle the corresponding logic. If we can implement that logic, it would be the best outcome, allowing us to adopt the more familiar and user-friendly nvme0n1 style.

On the other hand, if we’re okay with using names that don't match the usual convention (nvme0n1), then I think nvme is a more appropriate and concise naming choice. This approach would simplify the handling logic.

I think we can evaluate the complexity of implementing the nvme0n1 format. If the scope of changes is acceptable, adopting this format would be more user-friendly.