NATIVE SCSI MULTIPATH:
This series introduces native SCSI multipath support. It is intended as
an alternative to dm-mpath.
This support aims to provide a multipath-enabled SCSI block device/
gendisk.
For a SCSI device to support native multipath, either of the following
conditions must be satisfied:
a. unique ID in VPD page 83 and ALUA support and scsi_multipath modparam
enabled
b. unique ID in VPD page 83 and scsi_multipath_always modparam enabled
This series relies on reading sdev->access_state to get path information.
This path information would be provided by ALUA. ALUA support which does
not rely on device handlers has already been discussed at
https://lore.kernel.org/linux-scsi/7755e98f-5619-48ba-bcfc-b64eec930c40@oracle.com/
and support will be added in the next phase.
New classes of devices are added:
- scsi_mpath_device
- scsi_mpath_disk
These are required since a multipath scsi_device has no common scsi host.
An example of the sysfs files and directories for these new classes is as
follows:
$ ls -l /sys/class/scsi_mpath_device/scsi_mpath_device0/
total 0
-rw-r--r-- 1 root root 4096 Feb 25 11:59 iopolicy
drwxr-xr-x 2 root root 0 Feb 25 11:59 multipath
drwxr-xr-x 2 root root 0 Feb 25 11:59 power
lrwxrwxrwx 1 root root 0 Feb 25 11:59 subsystem ->
../../../../class/scsi_mpath_device
-rw-r--r-- 1 root root 4096 Feb 25 11:58 uevent
-r--r--r-- 1 root root 4096 Feb 25 11:59 vpd_id
$ ls -l /sys/class/scsi_mpath_device/scsi_mpath_device0/multipath/
total 0
lrwxrwxrwx 1 root root 0 Feb 25 11:59 8:0:0:0 ->
../../../../platform/host8/session1/target8:0:0/8:0:0:0
lrwxrwxrwx 1 root root 0 Feb 25 11:59 9:0:0:0 ->
../../../../platform/host9/session2/target9:0:0/9:0:0:0
$ cat /sys/class/scsi_mpath_device/scsi_mpath_device0/vpd_id
naa.600140505200a986f0043c9afa1fd077
$ cat /sys/class/scsi_mpath_device/scsi_mpath_device0/iopolicy
numa
$
$ ls -l /sys/class/scsi_mpath_disk/scsi_mpath_disk0/
total 0
drwxr-xr-x 2 root root 0 Feb 25 12:00 power
drwxr-xr-x 11 root root 0 Feb 25 11:58 sdc
lrwxrwxrwx 1 root root 0 Feb 25 11:58 subsystem ->
../../../../class/scsi_mpath_disk
-rw-r--r-- 1 root root 4096 Feb 25 11:58 uevent
$ ls -l /sys/class/scsi_mpath_disk/scsi_mpath_disk0/sdc/multipath/
total 0
lrwxrwxrwx 1 root root 0 Feb 25 12:00 sdc:0 ->
../../../../../platform/host8/session1/target8:0:0/8:0:0:0/block/sdc:0
lrwxrwxrwx 1 root root 0 Feb 25 12:00 sdc:1 ->
../../../../../platform/host9/session2/target9:0:0/9:0:0:0/block/sdc:1
$ ls -l /dev/sdc
brw-rw---- 1 root disk 8, 32 Feb 25 11:58 /dev/sdc
The scsi_device and scsi_disk classes otherwise remain unmodified.
However, the per-path block device is hidden in /dev/. Furthermore,
multipathed block devices have a new naming scheme, sdX:Y, where
X is the scsi multipath device index and Y is the path index.
No multipath sg support is added. We still have a per-path sg device.
Since the SCSI block device is multipath enabled, we can access
multipathed scsi_ioctl() through that block device.
For failover, we take the approach of cloning bio's and re-submitting them
in full (for failover errors).
LIBMULTIPATH:
This series introduces also libmultipath. It is essentially a refactoring
of NVME multipath support, so we can have a common library to also support
native SCSI multipath.
Much of the code is taken directly from the NVMe multipath code. However,
NVMe specifics are removed. A template structure is provided so the driver
-may provide callbacks for driver specifics, like ANA support for NVMe.
Important new structures introduced include:
- mpath_head
These contain much of the multipath-specific functionality from
nvme_ns_head, including a pointer to the gendisk structure and
a path SRCU-based array.
- mpath_device
This is the per-path structure, and contains much the same
multipath-specific functionality in nvme_ns
libmultipath provides functionality for path management, path selection,
data path, and failover handling.
Full series also available at
https://github.com/johnpgarry/linux/commits/scsi-multipath-v7.2-v4/
Differences to v3:
- drop for now any libmultipath specific NVMe support, like cdev and PR
support
- deal with issues sashiko/codex found in libmultipath:
- add checks in mpath_add_device() for setting pointers
- add mpath_delete_device() -> mpath_synchronize() call
- add numa iopolicy check for disabled path
- fix trace in mpath_bdev_submit_bio()
- fix race in mpath_head_uninit() / mpath_put_head()
- add ioctl check in mpath_bdev_ioctl()
- fix sync ordering in mpath_remove_disk()
- drop unused arg in mpath_iopolicy_store()
- fix mpath_delayed_removal_secs_show() print format
- memset mpath_head in mpath_head_init()
- deal with issues sashiko/codex found in scsi-multipath support:
- change TPG check in scsi_mpath_dev_alloc()
- use scsi_multipath_param_set() -> __sysfs_match_string()
- add scsi_mpath_revalidate_paths()
- add sdev check in scsi_mpath_clone_end_io()
- fix sdev->access_state masking in scsi_mpath_is_disabled() and
scsi_mpath_is_optimized()
- properly set default iopolicy in scsi_mpath_alloc_head()
- fix error path in scsi_mpath_alloc_head()
- fix possible race in scsi_mpath_find_head()
- fix scsi_mpath_{start, end}_request() for partitions
- fix scsi_end_request() -> scsi_mpath_end_request() order to deal with
partial completions
- don't call __scsi_internal_device_block_nowait() ->
scsi_mpath_dev_clear_path()
- call scsi_internal_device_unblock() -> scsi_mpath_revalidate_paths()
- fix sd.c sys attr references to sd_mpath_disk
- handle delayed removal race in sd_mpath_remove_head()
- default index = -1 in sd_probe()
John Garry (28):
libmultipath: Add initial framework
libmultipath: Add basic gendisk support
libmultipath: Add path selection support
libmultipath: Add bio handling
libmultipath: Add support for mpath_device management
libmultipath: Add delayed removal support
libmultipath: Add sysfs helpers
libmultipath: Add mpath_bdev_report_zones()
libmultipath: Add support for block device IOCTL
libmultipath: Add mpath_bdev_getgeo()
libmultipath: Add mpath_bdev_get_unique_id()
scsi-multipath: introduce basic SCSI device support
scsi-multipath: introduce scsi_device head structure
scsi-multipath: provide sysfs link from to scsi_device
scsi-multipath: support iopolicy
scsi-multipath: clone each bio
scsi-multipath: clear path when device is blocked
scsi-multipath: revalidate paths upon device unblock
scsi-multipath: failover handling
scsi-multipath: provide callbacks for path state
scsi-multipath: add scsi_mpath_{start,end}_request()
scsi-multipath: add delayed disk removal support
scsi: sd: add multipath disk class
scsi: sd: add multipath disk attr groups
scsi: sd: support multipath disk
scsi: sd: add mpath_dev file
scsi: sd: add mpath_numa_nodes dev attribute
scsi: sd: add mpath_queue_depth dev attribute
drivers/scsi/Kconfig | 10 +
drivers/scsi/Makefile | 1 +
drivers/scsi/scsi.c | 8 +-
drivers/scsi/scsi_lib.c | 16 +-
drivers/scsi/scsi_multipath.c | 640 +++++++++++++++++++++++
drivers/scsi/scsi_scan.c | 4 +
drivers/scsi/scsi_sysfs.c | 10 +
drivers/scsi/sd.c | 585 ++++++++++++++++++++-
drivers/scsi/sd.h | 3 +
include/linux/multipath.h | 158 ++++++
include/scsi/scsi_cmnd.h | 6 +
include/scsi/scsi_device.h | 2 +
include/scsi/scsi_driver.h | 4 +
include/scsi/scsi_host.h | 4 +
include/scsi/scsi_multipath.h | 119 +++++
lib/Kconfig | 6 +
lib/Makefile | 2 +
lib/multipath.c | 960 ++++++++++++++++++++++++++++++++++
18 files changed, 2514 insertions(+), 24 deletions(-)
create mode 100644 drivers/scsi/scsi_multipath.c
create mode 100644 include/linux/multipath.h
create mode 100644 include/scsi/scsi_multipath.h
create mode 100644 lib/multipath.c
--
2.43.7