Hi Martin, reviewers,
This series adds functionality for NVMe initiator to the fnic driver.
The changes enable the fnic driver to act as an NVMe initiator over
Fibre Channel (FC-NVMe), allowing the host to discover and communicate
with NVMe targets using the existing fnic infrastructure. The patches
prepare the driver for full FC-NVMe initiator operation while
maintaining existing SCSI/FC functionality.
These are some of the salient patches:
o. Make fnic debug logging usable by SCSI and NVMe initiator roles.
o. Use fnic instance numbers for non-SCSI-facing identifiers.
o. Decode firmware roles for FCP, NVMe, and unsupported targets.
o. Advertise NVMe initiator service parameters during FC discovery.
o. Add FDLS role handling for NVMe initiator discovery flows.
o. Add the NVMe/FC transport path and port registration.
o. Route completions, resets, and LS frames by initiator role.
o. Add NVMe LS timeouts, statistics, and debugfs state reporting.
Even though the patches have been made into a series, some patches are
heavier than others. But, every effort has been made to keep the
purpose of each patch as a single-purpose, and to compile cleanly.
All the individual patches compile cleanly. The compiler used is GCC
14.2.
This patch set has been tested as a whole. Therefore, the tested-by
fields have been added only to one patch in the set.
I've refrained from adding tested-by to most of the patches, so as to
not mislead the reviewer/reader.
A brief note on the unit tests:
o. Configure multipathing, and run link flaps on single link. IOs drop
briefly, but pick up as expected.
o. Configure multipathing, and run link flaps on two links, with a 30
second delay in between. IOs drop briefly, but pick up as expected.
o. Repeat the above tests with 1 queue and 64 queues.
o. Perform tests with Netapp and Pure targets.
All tests were successful.
This set of patches was reviewed before submitting upstream,
and the following review comments were incorporated.
Incorporate review comments from Hannes Reinecke:
Decode target roles explicitly and report unsupported roles.
Remove the empty line before the FLOGI completion else block.
Add a short comment for the NVMe ERSP completion case.
Incorporate review comments from Lee Duncan:
Replace the NVMe LS OXID switch with a direct frame-type check.
Rename the NVMe frame helper to follow fnic function naming style.
Convert the NVMe opcode stats helper to a switch statement.
Share NVMe completion stats accounting and compute duration once.
Changes between v1 and v2:
Incorporate review comments from Marco Crivellari:
Explicitly use WQ_PERCPU for the fnic completion workqueue.
Changes between v2 and v3:
Fix issues reported by kernel bot.
Add the active OXID to the tport timeout debug print.
Guard NVMe I/O and LS request debug logs when tport is NULL.
Validate ERSP response length before copying the response.
Validate LS response frame and payload lengths before copying.
Limit nvmef_info debugfs output to the allocated buffer.
Changes between v3 and v4:
Incorporate review comments from Sashiko:
Keep role strings private and const.
Clear NVMe SGL DMA address on map failure.
Drop fnic lock around NVMe tport registration.
Unlink unregistered NVMe tports before freeing them.
Clear NVMe unload completion pointer on exit.
Take fnic lock before NVMe tport cleanup tag lookup.
Clear NVMe tport delete completion pointer on unregister failure.
Arm NVMe LS request timer before exposing the request.
Free LS ABTS frame when send fails.
Reuse NVMe LS request cleanup for abort failure.
Decrement NVMe completion wait queue counter when draining completions.
Preserve jiffies wrap when computing NVMe abort stats.
Changes between v4 and v5:
Incorporate review comments from Sashiko:
Set NVMe wait timestamp before queueing completions
Use async timer delete for NVMe LS responses
Complete failed NVMe ITMF aborts
Track NVMe SGL DMA mapping state
Detach NVMe LS cleanup before callbacks
Defer NVMe table frees until cleanup
Lock NVMe tport IO termination lookup
Count NVMe ERSP completions as firmware completions
Gate NVMe completions by initiator role
Check cleaned buffers before dereferencing them
Drain OXID reclaim state on reset
Avoid NVMe LS request send races
Arm NVMe LS abort timer after send succeeds
Clean up NVMe debugfs on probe errors
Clarify that FC-NVMe PRLI must not set Establish Image Pair.
Changes between v5 and v6:
Synchronize the tport retry timer before remote-port delete completes.
Avoid requeueing FDLS tport delete from the NVMe remote-port callback.
Drop the unnecessary else after the tport delete timeout return.
Name the tport cleanup retry limit instead of using a literal.
Publish can-free before completion and set cleanup-pending on timeout.
Incorporate review comments from Sashiko:
Flush NVMe completions after copy WQ cleanup
Reset NVMe SGL state before each I/O
Claim tport I/O before aborting it
Synchronize LS response timer before done
Preserve LS requests during abort start
Drain LS requests on rport unregister failure
Defer tport free after rport timeout
Avoid extra PCI device reference for lport
Thanks,
Karan
Karan Tilak Kumar (13):
scsi: fnic: Make debug logging protocol independent
scsi: fnic: Use fnic_num for non-SCSI identifiers
scsi: fnic: Decode firmware role configuration
scsi: fnic: Advertise NVMe initiator service parameters
scsi: fnic: Add FDLS role handling for NVMe initiators
scsi: fnic: Add the NVMe/FC transport path
scsi: fnic: Route completions and resets by initiator role
scsi: fnic: Handle NVMe LS frames in FDLS
scsi: fnic: Send NVMe LS requests through FDLS
scsi: fnic: Abort timed-out NVMe LS requests
scsi: fnic: Track NVMe transport statistics
scsi: fnic: Expose NVMe transport state in debugfs
scsi: fnic: Bump up version number
drivers/scsi/fnic/Makefile | 1 +
drivers/scsi/fnic/fcpio.h | 35 +
drivers/scsi/fnic/fdls_disc.c | 978 +++++++------
drivers/scsi/fnic/fdls_fc.h | 7 +
drivers/scsi/fnic/fip.c | 117 +-
drivers/scsi/fnic/fip.h | 2 +-
drivers/scsi/fnic/fnic.h | 104 +-
drivers/scsi/fnic/fnic_debugfs.c | 99 +-
drivers/scsi/fnic/fnic_fcs.c | 199 +--
drivers/scsi/fnic/fnic_fdls.h | 48 +-
drivers/scsi/fnic/fnic_io.h | 24 +-
drivers/scsi/fnic/fnic_isr.c | 28 +-
drivers/scsi/fnic/fnic_main.c | 214 ++-
drivers/scsi/fnic/fnic_nvme.c | 2268 ++++++++++++++++++++++++++++++
drivers/scsi/fnic/fnic_nvme.h | 214 +++
drivers/scsi/fnic/fnic_res.c | 31 +-
drivers/scsi/fnic/fnic_res.h | 32 +-
drivers/scsi/fnic/fnic_scsi.c | 251 ++--
drivers/scsi/fnic/fnic_stats.h | 28 +
drivers/scsi/fnic/fnic_trace.c | 3 +-
drivers/scsi/fnic/vnic_devcmd.h | 2 +-
21 files changed, 3906 insertions(+), 779 deletions(-)
create mode 100644 drivers/scsi/fnic/fnic_nvme.c
create mode 100644 drivers/scsi/fnic/fnic_nvme.h
--
2.47.1