hw/block/meson.build | 2 +- hw/block/nvme-ns.c | 32 ++++++++-- hw/block/nvme-ns.h | 13 ++++ hw/block/nvme-subsys.c | 111 +++++++++++++++++++++++++++++++++ hw/block/nvme-subsys.h | 32 ++++++++++ hw/block/nvme.c | 137 ++++++++++++++++++++++++++++++++++++++--- hw/block/nvme.h | 19 ++++++ include/block/nvme.h | 8 +++ 8 files changed, 340 insertions(+), 14 deletions(-) create mode 100644 hw/block/nvme-subsys.c create mode 100644 hw/block/nvme-subsys.h
Hello,
This patch series is third one to support multi-controller and namespace
sharing in multi-path. This series introduced subsystem scheme to
manage controller(s) and namespace(s) in the subsystem.
This series has new patches from the V2: 'detached' parameter has been
added to the nvme-ns device. This will decide whether to attach the
namespace to controller(s) in the current subsystem or not. If it's
given with true, then it will be just allocated in the subsystem, but
not attaching to any controllers in the subsystem. Otherwise, it will
automatically attach to all the controllers in the subsystem. The other
t hing is that the last patch implemented Identify Active Namespace ID
List command handler apart from the Allocated Namespace ID List.
Run with:
-device nvme,serial=qux,id=nvme3
-device nvme-ns,id=ns3,drive=drv12,nsid=3,bus=nvme3
-device nvme-subsys,id=subsys0
-device nvme,serial=foo,id=nvme0,subsys=subsys0
-device nvme,serial=bar,id=nvme1,subsys=subsys0
-device nvme,serial=baz,id=nvme2,subsys=subsys0
-device nvme-ns,id=ns1,drive=drv10,nsid=1,subsys=subsys0,detached=true
-device nvme-ns,id=ns2,drive=drv11,nsid=2,bus=nvme2
nvme-cli:
root@vm:~/work# nvme list -v
NVM Express Subsystems
Subsystem Subsystem-NQN Controllers
---------------- ------------------------------------------------------------------------------------------------ ----------------
nvme-subsys0 nqn.2019-08.org.qemu:qux nvme0
nvme-subsys1 nqn.2019-08.org.qemu:subsys0 nvme1, nvme2, nvme3
NVM Express Controllers
Device SN MN FR TxPort Address Subsystem Namespaces
-------- -------------------- ---------------------------------------- -------- ------ -------------- ------------ ----------------
nvme0 qux QEMU NVMe Ctrl 1.0 pcie 0000:00:06.0 nvme-subsys0
nvme1 foo QEMU NVMe Ctrl 1.0 pcie 0000:00:07.0 nvme-subsys1
nvme2 bar QEMU NVMe Ctrl 1.0 pcie 0000:00:08.0 nvme-subsys1
nvme3 baz QEMU NVMe Ctrl 1.0 pcie 0000:00:09.0 nvme-subsys1 nvme1n1
NVM Express Namespaces
Device NSID Usage Format Controllers
------------ -------- -------------------------- ---------------- ----------------
nvme0n1 3 268.44 MB / 268.44 MB 512 B + 0 B nvme0
nvme1n1 2 268.44 MB / 268.44 MB 512 B + 0 B nvme3
Now we have [allocated|not-allocated]/[attached/detached] scheme for
namespaces and controllers. The next series is going to be to support
namespace management and attachment with few Identify command handlers.
Please review.
Thanks!
Since RFC V2:
- Rebased on nvme-next branch with trivial patches from the previous
version(V2) applied. (Klaus)
- Fix enumeration type name convention with NvmeIdNs prefix. (Klaus)
- Put 'cntlid' to NvmeCtrl instance in nvme_init_ctrl() which was
missed in V2.
- Added 'detached' parameter to nvme-ns device to decide whether to
attach or not to controller(s) in the subsystem. (Klaus)
- Implemented Identify Active Namespace ID List aprt from Identify
Allocated Namespace ID List by removing fall-thru statement.
Since RFC V1:
- Updated namespace sharing scheme to be based on nvme-subsys
hierarchy.
Minwoo Im (8):
hw/block/nvme: introduce nvme-subsys device
hw/block/nvme: support to map controller to a subsystem
hw/block/nvme: add CMIC enum value for Identify Controller
hw/block/nvme: support for multi-controller in subsystem
hw/block/nvme: add NMIC enum value for Identify Namespace
hw/block/nvme: support for shared namespace in subsystem
hw/block/nvme: add 'detached' param not to attach namespace
hw/block/nvme: Add Identify Active Namespace ID List
hw/block/meson.build | 2 +-
hw/block/nvme-ns.c | 32 ++++++++--
hw/block/nvme-ns.h | 13 ++++
hw/block/nvme-subsys.c | 111 +++++++++++++++++++++++++++++++++
hw/block/nvme-subsys.h | 32 ++++++++++
hw/block/nvme.c | 137 ++++++++++++++++++++++++++++++++++++++---
hw/block/nvme.h | 19 ++++++
include/block/nvme.h | 8 +++
8 files changed, 340 insertions(+), 14 deletions(-)
create mode 100644 hw/block/nvme-subsys.c
create mode 100644 hw/block/nvme-subsys.h
--
2.17.1
On Jan 20 02:01, Minwoo Im wrote: > Hello, > > This patch series is third one to support multi-controller and namespace > sharing in multi-path. This series introduced subsystem scheme to > manage controller(s) and namespace(s) in the subsystem. > > This series has new patches from the V2: 'detached' parameter has been > added to the nvme-ns device. This will decide whether to attach the > namespace to controller(s) in the current subsystem or not. If it's > given with true, then it will be just allocated in the subsystem, but > not attaching to any controllers in the subsystem. Otherwise, it will > automatically attach to all the controllers in the subsystem. The other > t hing is that the last patch implemented Identify Active Namespace ID > List command handler apart from the Allocated Namespace ID List. > > Run with: > -device nvme,serial=qux,id=nvme3 > -device nvme-ns,id=ns3,drive=drv12,nsid=3,bus=nvme3 > > -device nvme-subsys,id=subsys0 > -device nvme,serial=foo,id=nvme0,subsys=subsys0 > -device nvme,serial=bar,id=nvme1,subsys=subsys0 > -device nvme,serial=baz,id=nvme2,subsys=subsys0 > -device nvme-ns,id=ns1,drive=drv10,nsid=1,subsys=subsys0,detached=true > -device nvme-ns,id=ns2,drive=drv11,nsid=2,bus=nvme2 > > nvme-cli: > root@vm:~/work# nvme list -v > NVM Express Subsystems > > Subsystem Subsystem-NQN Controllers > ---------------- ------------------------------------------------------------------------------------------------ ---------------- > nvme-subsys0 nqn.2019-08.org.qemu:qux nvme0 > nvme-subsys1 nqn.2019-08.org.qemu:subsys0 nvme1, nvme2, nvme3 > > NVM Express Controllers > > Device SN MN FR TxPort Address Subsystem Namespaces > -------- -------------------- ---------------------------------------- -------- ------ -------------- ------------ ---------------- > nvme0 qux QEMU NVMe Ctrl 1.0 pcie 0000:00:06.0 nvme-subsys0 Shouldn't nvme0n1 be listed under Namespaces for nvme0? > nvme1 foo QEMU NVMe Ctrl 1.0 pcie 0000:00:07.0 nvme-subsys1 > nvme2 bar QEMU NVMe Ctrl 1.0 pcie 0000:00:08.0 nvme-subsys1 > nvme3 baz QEMU NVMe Ctrl 1.0 pcie 0000:00:09.0 nvme-subsys1 nvme1n1 > > NVM Express Namespaces > > Device NSID Usage Format Controllers > ------------ -------- -------------------------- ---------------- ---------------- > nvme0n1 3 268.44 MB / 268.44 MB 512 B + 0 B nvme0 > nvme1n1 2 268.44 MB / 268.44 MB 512 B + 0 B nvme3 > > Now we have [allocated|not-allocated]/[attached/detached] scheme for > namespaces and controllers. The next series is going to be to support > namespace management and attachment with few Identify command handlers. > > Please review. > > Thanks! > > Since RFC V2: > - Rebased on nvme-next branch with trivial patches from the previous > version(V2) applied. (Klaus) > - Fix enumeration type name convention with NvmeIdNs prefix. (Klaus) > - Put 'cntlid' to NvmeCtrl instance in nvme_init_ctrl() which was > missed in V2. > - Added 'detached' parameter to nvme-ns device to decide whether to > attach or not to controller(s) in the subsystem. (Klaus) > - Implemented Identify Active Namespace ID List aprt from Identify > Allocated Namespace ID List by removing fall-thru statement. > > Since RFC V1: > - Updated namespace sharing scheme to be based on nvme-subsys > hierarchy. > > Minwoo Im (8): > hw/block/nvme: introduce nvme-subsys device > hw/block/nvme: support to map controller to a subsystem > hw/block/nvme: add CMIC enum value for Identify Controller > hw/block/nvme: support for multi-controller in subsystem > hw/block/nvme: add NMIC enum value for Identify Namespace > hw/block/nvme: support for shared namespace in subsystem > hw/block/nvme: add 'detached' param not to attach namespace > hw/block/nvme: Add Identify Active Namespace ID List > > hw/block/meson.build | 2 +- > hw/block/nvme-ns.c | 32 ++++++++-- > hw/block/nvme-ns.h | 13 ++++ > hw/block/nvme-subsys.c | 111 +++++++++++++++++++++++++++++++++ > hw/block/nvme-subsys.h | 32 ++++++++++ > hw/block/nvme.c | 137 ++++++++++++++++++++++++++++++++++++++--- > hw/block/nvme.h | 19 ++++++ > include/block/nvme.h | 8 +++ > 8 files changed, 340 insertions(+), 14 deletions(-) > create mode 100644 hw/block/nvme-subsys.c > create mode 100644 hw/block/nvme-subsys.h > > -- > 2.17.1 > > -- One of us - No more doubt, silence or taboo about mental illness.
On Tue, Jan 19, 2021 at 07:18:16PM +0100, Klaus Jensen wrote: > On Jan 20 02:01, Minwoo Im wrote: > > Run with: > > -device nvme,serial=qux,id=nvme3 > > -device nvme-ns,id=ns3,drive=drv12,nsid=3,bus=nvme3 > > > > -device nvme-subsys,id=subsys0 > > -device nvme,serial=foo,id=nvme0,subsys=subsys0 > > -device nvme,serial=bar,id=nvme1,subsys=subsys0 > > -device nvme,serial=baz,id=nvme2,subsys=subsys0 > > -device nvme-ns,id=ns1,drive=drv10,nsid=1,subsys=subsys0,detached=true > > -device nvme-ns,id=ns2,drive=drv11,nsid=2,bus=nvme2 > > > > nvme-cli: > > root@vm:~/work# nvme list -v > > NVM Express Subsystems > > > > Subsystem Subsystem-NQN Controllers > > ---------------- ------------------------------------------------------------------------------------------------ ---------------- > > nvme-subsys0 nqn.2019-08.org.qemu:qux nvme0 > > nvme-subsys1 nqn.2019-08.org.qemu:subsys0 nvme1, nvme2, nvme3 > > > > NVM Express Controllers > > > > Device SN MN FR TxPort Address Subsystem Namespaces > > -------- -------------------- ---------------------------------------- -------- ------ -------------- ------------ ---------------- > > nvme0 qux QEMU NVMe Ctrl 1.0 pcie 0000:00:06.0 nvme-subsys0 > > Shouldn't nvme0n1 be listed under Namespaces for nvme0? Minwoo, try pulling the most current nvme-cli. There was a sysfs scanning bug for non-mpath drives that should be fixed now.
> Minwoo, try pulling the most current nvme-cli. There was a sysfs > scanning bug for non-mpath drives that should be fixed now. Thank you, Keith! I've posted list result based on the latest one :)
On 21-01-19 19:18:16, Klaus Jensen wrote: > On Jan 20 02:01, Minwoo Im wrote: > > Hello, > > > > This patch series is third one to support multi-controller and namespace > > sharing in multi-path. This series introduced subsystem scheme to > > manage controller(s) and namespace(s) in the subsystem. > > > > This series has new patches from the V2: 'detached' parameter has been > > added to the nvme-ns device. This will decide whether to attach the > > namespace to controller(s) in the current subsystem or not. If it's > > given with true, then it will be just allocated in the subsystem, but > > not attaching to any controllers in the subsystem. Otherwise, it will > > automatically attach to all the controllers in the subsystem. The other > > t hing is that the last patch implemented Identify Active Namespace ID > > List command handler apart from the Allocated Namespace ID List. > > > > Run with: > > -device nvme,serial=qux,id=nvme3 > > -device nvme-ns,id=ns3,drive=drv12,nsid=3,bus=nvme3 > > > > -device nvme-subsys,id=subsys0 > > -device nvme,serial=foo,id=nvme0,subsys=subsys0 > > -device nvme,serial=bar,id=nvme1,subsys=subsys0 > > -device nvme,serial=baz,id=nvme2,subsys=subsys0 > > -device nvme-ns,id=ns1,drive=drv10,nsid=1,subsys=subsys0,detached=true > > -device nvme-ns,id=ns2,drive=drv11,nsid=2,bus=nvme2 > > > > nvme-cli: > > root@vm:~/work# nvme list -v > > NVM Express Subsystems > > > > Subsystem Subsystem-NQN Controllers > > ---------------- ------------------------------------------------------------------------------------------------ ---------------- > > nvme-subsys0 nqn.2019-08.org.qemu:qux nvme0 > > nvme-subsys1 nqn.2019-08.org.qemu:subsys0 nvme1, nvme2, nvme3 > > > > NVM Express Controllers > > > > Device SN MN FR TxPort Address Subsystem Namespaces > > -------- -------------------- ---------------------------------------- -------- ------ -------------- ------------ ---------------- > > nvme0 qux QEMU NVMe Ctrl 1.0 pcie 0000:00:06.0 nvme-subsys0 > > Shouldn't nvme0n1 be listed under Namespaces for nvme0? Oh, I missed that one from the output. As Keith mentioned, I ran the list command again based on the latest nvme-cli.git: Please refer the following result. I think it's okay not to send the cover letter again :) # nvme --version nvme version 1.13.48.g33c6 # nvme list -v NVM Express Subsystems Subsystem Subsystem-NQN Controllers ---------------- ------------------------------------------------------------------------------------------------ ---------------- nvme-subsys0 nqn.2019-08.org.qemu:qux nvme0 nvme-subsys1 nqn.2019-08.org.qemu:subsys0 nvme1, nvme2, nvme3 NVM Express Controllers Device SN MN FR TxPort Address Subsystem Namespaces -------- -------------------- ---------------------------------------- -------- ------ -------------- ------------ ---------------- nvme0 qux QEMU NVMe Ctrl 1.0 pcie 0000:00:06.0 nvme-subsys0 nvme0n1 nvme1 foo QEMU NVMe Ctrl 1.0 pcie 0000:00:07.0 nvme-subsys1 nvme2 bar QEMU NVMe Ctrl 1.0 pcie 0000:00:08.0 nvme-subsys1 nvme3 baz QEMU NVMe Ctrl 1.0 pcie 0000:00:09.0 nvme-subsys1 nvme1c3n1 NVM Express Namespaces Device NSID Usage Format Controllers ------------ -------- -------------------------- ---------------- ---------------- nvme0n1 3 268.44 MB / 268.44 MB 512 B + 0 B nvme0 nvme1n1 2 268.44 MB / 268.44 MB 512 B + 0 B nvme3
On Jan 20 09:44, Minwoo Im wrote: > On 21-01-19 19:18:16, Klaus Jensen wrote: > > On Jan 20 02:01, Minwoo Im wrote: > > > Hello, > > > > > > This patch series is third one to support multi-controller and namespace > > > sharing in multi-path. This series introduced subsystem scheme to > > > manage controller(s) and namespace(s) in the subsystem. > > > > > > This series has new patches from the V2: 'detached' parameter has been > > > added to the nvme-ns device. This will decide whether to attach the > > > namespace to controller(s) in the current subsystem or not. If it's > > > given with true, then it will be just allocated in the subsystem, but > > > not attaching to any controllers in the subsystem. Otherwise, it will > > > automatically attach to all the controllers in the subsystem. The other > > > t hing is that the last patch implemented Identify Active Namespace ID > > > List command handler apart from the Allocated Namespace ID List. > > > > > > Run with: > > > -device nvme,serial=qux,id=nvme3 > > > -device nvme-ns,id=ns3,drive=drv12,nsid=3,bus=nvme3 > > > > > > -device nvme-subsys,id=subsys0 > > > -device nvme,serial=foo,id=nvme0,subsys=subsys0 > > > -device nvme,serial=bar,id=nvme1,subsys=subsys0 > > > -device nvme,serial=baz,id=nvme2,subsys=subsys0 > > > -device nvme-ns,id=ns1,drive=drv10,nsid=1,subsys=subsys0,detached=true > > > -device nvme-ns,id=ns2,drive=drv11,nsid=2,bus=nvme2 > > > > > > nvme-cli: > > > root@vm:~/work# nvme list -v > > > NVM Express Subsystems > > > > > > Subsystem Subsystem-NQN Controllers > > > ---------------- ------------------------------------------------------------------------------------------------ ---------------- > > > nvme-subsys0 nqn.2019-08.org.qemu:qux nvme0 > > > nvme-subsys1 nqn.2019-08.org.qemu:subsys0 nvme1, nvme2, nvme3 > > > > > > NVM Express Controllers > > > > > > Device SN MN FR TxPort Address Subsystem Namespaces > > > -------- -------------------- ---------------------------------------- -------- ------ -------------- ------------ ---------------- > > > nvme0 qux QEMU NVMe Ctrl 1.0 pcie 0000:00:06.0 nvme-subsys0 > > > > Shouldn't nvme0n1 be listed under Namespaces for nvme0? > > Oh, I missed that one from the output. As Keith mentioned, I ran the > list command again based on the latest nvme-cli.git: > > Please refer the following result. I think it's okay not to send the > cover letter again :) > > # nvme --version > nvme version 1.13.48.g33c6 > > # nvme list -v > NVM Express Subsystems > > Subsystem Subsystem-NQN Controllers > ---------------- ------------------------------------------------------------------------------------------------ ---------------- > nvme-subsys0 nqn.2019-08.org.qemu:qux nvme0 > nvme-subsys1 nqn.2019-08.org.qemu:subsys0 nvme1, nvme2, nvme3 > > NVM Express Controllers > > Device SN MN FR TxPort Address Subsystem Namespaces > -------- -------------------- ---------------------------------------- -------- ------ -------------- ------------ ---------------- > nvme0 qux QEMU NVMe Ctrl 1.0 pcie 0000:00:06.0 nvme-subsys0 nvme0n1 > nvme1 foo QEMU NVMe Ctrl 1.0 pcie 0000:00:07.0 nvme-subsys1 > nvme2 bar QEMU NVMe Ctrl 1.0 pcie 0000:00:08.0 nvme-subsys1 > nvme3 baz QEMU NVMe Ctrl 1.0 pcie 0000:00:09.0 nvme-subsys1 nvme1c3n1 > > NVM Express Namespaces > > Device NSID Usage Format Controllers > ------------ -------- -------------------------- ---------------- ---------------- > nvme0n1 3 268.44 MB / 268.44 MB 512 B + 0 B nvme0 > nvme1n1 2 268.44 MB / 268.44 MB 512 B + 0 B nvme3 That looks better, but hmm. Shouldnt the namespace be named `nvme1c3n1` here has well? Is that also an issue with nvme-cli?
On 21-01-20 08:52:14, Klaus Jensen wrote: > On Jan 20 09:44, Minwoo Im wrote: > > On 21-01-19 19:18:16, Klaus Jensen wrote: > > > On Jan 20 02:01, Minwoo Im wrote: > > > > Hello, > > > > > > > > This patch series is third one to support multi-controller and namespace > > > > sharing in multi-path. This series introduced subsystem scheme to > > > > manage controller(s) and namespace(s) in the subsystem. > > > > > > > > This series has new patches from the V2: 'detached' parameter has been > > > > added to the nvme-ns device. This will decide whether to attach the > > > > namespace to controller(s) in the current subsystem or not. If it's > > > > given with true, then it will be just allocated in the subsystem, but > > > > not attaching to any controllers in the subsystem. Otherwise, it will > > > > automatically attach to all the controllers in the subsystem. The other > > > > t hing is that the last patch implemented Identify Active Namespace ID > > > > List command handler apart from the Allocated Namespace ID List. > > > > > > > > Run with: > > > > -device nvme,serial=qux,id=nvme3 > > > > -device nvme-ns,id=ns3,drive=drv12,nsid=3,bus=nvme3 > > > > > > > > -device nvme-subsys,id=subsys0 > > > > -device nvme,serial=foo,id=nvme0,subsys=subsys0 > > > > -device nvme,serial=bar,id=nvme1,subsys=subsys0 > > > > -device nvme,serial=baz,id=nvme2,subsys=subsys0 > > > > -device nvme-ns,id=ns1,drive=drv10,nsid=1,subsys=subsys0,detached=true > > > > -device nvme-ns,id=ns2,drive=drv11,nsid=2,bus=nvme2 > > > > > > > > nvme-cli: > > > > root@vm:~/work# nvme list -v > > > > NVM Express Subsystems > > > > > > > > Subsystem Subsystem-NQN Controllers > > > > ---------------- ------------------------------------------------------------------------------------------------ ---------------- > > > > nvme-subsys0 nqn.2019-08.org.qemu:qux nvme0 > > > > nvme-subsys1 nqn.2019-08.org.qemu:subsys0 nvme1, nvme2, nvme3 > > > > > > > > NVM Express Controllers > > > > > > > > Device SN MN FR TxPort Address Subsystem Namespaces > > > > -------- -------------------- ---------------------------------------- -------- ------ -------------- ------------ ---------------- > > > > nvme0 qux QEMU NVMe Ctrl 1.0 pcie 0000:00:06.0 nvme-subsys0 > > > > > > Shouldn't nvme0n1 be listed under Namespaces for nvme0? > > > > Oh, I missed that one from the output. As Keith mentioned, I ran the > > list command again based on the latest nvme-cli.git: > > > > Please refer the following result. I think it's okay not to send the > > cover letter again :) > > > > # nvme --version > > nvme version 1.13.48.g33c6 > > > > # nvme list -v > > NVM Express Subsystems > > > > Subsystem Subsystem-NQN Controllers > > ---------------- ------------------------------------------------------------------------------------------------ ---------------- > > nvme-subsys0 nqn.2019-08.org.qemu:qux nvme0 > > nvme-subsys1 nqn.2019-08.org.qemu:subsys0 nvme1, nvme2, nvme3 > > > > NVM Express Controllers > > > > Device SN MN FR TxPort Address Subsystem Namespaces > > -------- -------------------- ---------------------------------------- -------- ------ -------------- ------------ ---------------- > > nvme0 qux QEMU NVMe Ctrl 1.0 pcie 0000:00:06.0 nvme-subsys0 nvme0n1 > > nvme1 foo QEMU NVMe Ctrl 1.0 pcie 0000:00:07.0 nvme-subsys1 > > nvme2 bar QEMU NVMe Ctrl 1.0 pcie 0000:00:08.0 nvme-subsys1 > > nvme3 baz QEMU NVMe Ctrl 1.0 pcie 0000:00:09.0 nvme-subsys1 nvme1c3n1 > > > > NVM Express Namespaces > > > > Device NSID Usage Format Controllers > > ------------ -------- -------------------------- ---------------- ---------------- > > nvme0n1 3 268.44 MB / 268.44 MB 512 B + 0 B nvme0 > > nvme1n1 2 268.44 MB / 268.44 MB 512 B + 0 B nvme3 > > That looks better, but hmm. Shouldnt the namespace be named `nvme1c3n1` > here has well? Is that also an issue with nvme-cli? No it isn't in this series. The 'nvme3' controller has not been provided with 'subsys' parameter which means it does not support CMIC multi-controller as nvme_init_ctrl() does not set the flag for NVME_CMIC_MULTI_CTRL: -device nvme,serial=qux,id=nvme3 If this has been given with 'subsys' which means it supports for multi-controller, then it will be given like: Device SN MN FR TxPort Address Subsystem Namespaces -------- -------------------- ---------------------------------------- -------- ------ -------------- ------------ ---------------- nvme0 qux QEMU NVMe Ctrl 1.0 pcie 0000:00:06.0 nvme-subsys0 nvme0c0n1 In short, if 'nvme' device controller is not given with 'subsys', then it will not support multi-controller and it will never have GENHD_FL_HIDDEN (nvmeXcYnZ convention).
© 2016 - 2025 Red Hat, Inc.