This series adds the admin channel infrastructure and mailbox (MBOX)
protocol needed for V2 SR-IOV support in the enic driver.
The V2 SR-IOV design uses a direct PF-VF communication channel built on
dedicated WQ/RQ/CQ hardware resources and an MSI-X interrupt.
Firmware capability and admin channel infrastructure (patches 1-4):
- Probe-time firmware feature check for V2 SR-IOV support
- Admin channel open/close, RQ buffer management, CQ service
with MSI-X interrupt and NAPI polling
MBOX protocol and VF enable (patches 5-10):
- MBOX message types, core send/receive, PF and VF handlers
- V2 SR-IOV enable wiring with admin channel setup
- V2 VF probe with admin channel and PF registration
Signed-off-by: Satish Kharat <satishkh@cisco.com>
---
Changes in v7:
- Replace magic numbers in admin channel init with named macros
and inline comments for MBOX descriptor encoding
(patches 2, 6) [Paolo Abeni]
- Add defense-in-depth bounds check on admin RQ bytes_written (patch 4)
- Force NAPI reschedule on admin RQ refill failure (patch 4)
- Always unmask admin interrupt even with zero credits (patch 4)
- Reorder NAPI init before request_irq in admin channel open (patch 4)
- Remove redundant netdev_warn on admin msg enqueue kmalloc failure
(patch 4) [Paolo Abeni]
- Add netdev_warn on admin WQ/RQ disable failure in close path
(patch 2)
- Remove incorrect RES_TYPE_SRIOV_INTR interrupt allocation from
admin channel open (patch 2); interrupt setup handled entirely
in patch 4 using RES_TYPE_INTR_CTRL
- Rate-limit VF register/unregister log messages (patch 7) [Paolo Abeni]
- Add __aligned(8) to admin message data[] for strict-alignment
safety (patch 4)
- Rate-limit MBOX handler error warnings (patch 7)
- Pre-allocate port profile array before pci_disable_sriov in V1
disable path to avoid half-torn-down state on alloc failure (patch 9)
- Account for admin channel interrupt reservation in
enic_set_intr_mode() and enic_adjust_resources() (patch 9) [Paolo Abeni]
- Clear admin_rq_handler in enic_admin_channel_close (patch 9)
- Quiesce admin channel (mask interrupt, disable NAPI, block MBOX
sends) around soft reset (patch 9)
- Use WRITE_ONCE/READ_ONCE for mbox_send_disabled and
admin_rq_handler across data-path/reset boundaries
(patches 4, 6, 9)
- Fix commit message: reference enic_adjust_resources() alongside
enic_set_intr_mode() (patch 10)
Investigated findings from automated review (Simon Horman / Sashiko):
- Race between probe-time feature check and VF proxy: false positive;
detection runs at probe, enable runs from sriov_configure
- Struct alignment of __le32 after 2-byte mbox_hdr_embed: compiler
inserts correct padding, no manual alignment needed
- Stale MBOX reply matching / reinit_completion race: single-flight
design with mutex serialization prevents this
- cancel_work_sync vs MBOX unregister race: work cannot be
re-triggered during the close window
- Link to v6: https://patch.msgid.link/20260503-enic-sriov-v2-admin-channel-v2-v6-0-0af4fbc2d86d@cisco.com
Changes in v6:
- Add explanatory comments documenting admin_cq[0] (WQ CQE size) and
admin_cq[1] (RQ CQE size matching firmware enic_ext_cq() programming)
allocations (patch 2)
- Enforce bytes_written from CQ descriptor when enqueuing admin RQ
message; previously buf->len (allocation size) was passed, exposing
uninitialized buffer memory beyond the real payload (patch 4)
- Drop admin RQ messages with TRUNCATED set or FCS_OK clear, gated by
netdev_warn_once() (patch 4)
- Disable interrupt_enable on admin_cq[0]: WQ completions are polled
synchronously inside enic_mbox_send_msg() and never raise an
interrupt; matches admin_cq[1] (RQ) which does NAPI polling (patch 4)
- Add mbox_expected_reply gating in VF reply handlers (capability,
register, unregister): drop replies whose type does not match the
current waiter's expected type, avoiding spurious wakeup of an
unrelated waiter from a stale reply that arrives after timeout
(patch 8)
- Distinguish error returns in enic_mbox_vf_unregister(): -ETIMEDOUT
(no reply received), -EACCES (PF rejected the unregister), 0 on
success. Previously all paths collapsed to a single -ETIMEDOUT
(patch 8)
- Reserve one extra MSI-X slot in enic_set_intr_mode() when
has_admin_channel is set so enic_admin_setup_intr() always has room
to allocate at intr_count without exceeding intr_avail bounds when
data queue count is maxed out (patch 10)
- Clarify in commit messages that .sriov_configure is intentionally
not yet wired in this series and will be added in a follow-up after
the necessary devcmd hardening lands (patch 9)
- Link to v5: https://patch.msgid.link/20260423-enic-sriov-v2-admin-channel-v2-v5-0-caa9f504a3dc@cisco.com
Changes in v5:
- Fix DMA-into-freed-memory race: call enic_admin_qp_type_set() before
disabling RQ/WQ in both error and close paths (patch 3)
- Fix DMA mapping leak: enic_admin_wq_buf_clean() now unmaps and frees
WQ buffers still held at close time after a send timeout (patch 3)
- Log rate-limited warning on admin RQ refill failure (patch 4)
- Add missing linux/types.h and linux/bits.h includes to enic_mbox.h
(patch 5)
- Guard mbox_lock/mbox_comp init with mbox_initialized flag to prevent
re-initialization on sriov_configure re-entry (patch 7)
- Clear VF registered state before sending unregister reply so PF does
not treat a dead VF as still registered (patch 8)
- Gate VF-facing log messages with net_ratelimit() to prevent malicious
VF from flooding PF dmesg (patch 8)
- Reject VF port profile requests when V2 SR-IOV is active since
enic->pp is not reallocated for V2 VFs (patch 9)
- Move enic_sriov_detect_vf_type() before auto-enable check; skip
probe-time auto-enable for V2 VFs (patch 9)
- Move admin channel close and VF unregister before unregister_netdev()
in enic_remove() to prevent use-after-free on netdev (patch 10)
- Add comment in enic_reset() documenting that admin channel is not
recovered after soft reset (patch 10)
- Bypass RES_TYPE_SRIOV_INTR check for V2 VFs in admin channel
capability detection (patch 10)
- Link to v4: https://patch.msgid.link/20260411-enic-sriov-v2-admin-channel-v2-v4-0-f052326c2a57@cisco.com
Changes in v4:
- Fix reverse xmas tree variable ordering (patches 1, 6)
- Use kzalloc_obj instead of kzalloc with sizeof (patch 9)
- Add NULL check for pp allocation in V1 SR-IOV disable path (patch 9)
- Link to v3: https://lore.kernel.org/r/20260408-enic-sriov-v2-admin-channel-v2-v3-0-1d4999a03cec@cisco.com
Changes in v3:
- Use early-return pattern in enic_sriov_detect_vf_type to reduce
nesting (patch 1) [Breno Leitao]
- Link to v2: https://lore.kernel.org/r/20260408-enic-sriov-v2-admin-channel-v2-v2-0-d05dd3623fd3@cisco.com
Changes in v2:
- Fix lines exceeding 80 columns (patches 4, 6, 7, 8)
- Add __maybe_unused to enic_sriov_configure and enic_sriov_v2_enable;
.sriov_configure wiring deferred to a later series after devcmd
hardening is in place (patch 9)
- Guard probe-time auto-enable to skip V2 VFs (patch 9)
- Link to v1: https://lore.kernel.org/r/20260406-enic-sriov-v2-admin-channel-v2-v1-0-82cc47636a78@cisco.com
To: Satish Kharat <satishkh@cisco.com>
To: Andrew Lunn <andrew+netdev@lunn.ch>
To: "David S. Miller" <davem@davemloft.net>
To: Eric Dumazet <edumazet@google.com>
To: Jakub Kicinski <kuba@kernel.org>
To: Paolo Abeni <pabeni@redhat.com>
Cc: netdev@vger.kernel.org
Cc: linux-kernel@vger.kernel.org
Cc: Sesidhar Baddela <sebaddel@cisco.com>
---
Satish Kharat (10):
enic: verify firmware supports V2 SR-IOV at probe time
enic: add admin channel open and close for SR-IOV
enic: add admin RQ buffer management
enic: add admin CQ service with MSI-X interrupt and NAPI polling
enic: define MBOX message types and header structures
enic: add MBOX core send and receive for admin channel
enic: add MBOX PF handlers for VF register and capability
enic: add MBOX VF handlers for capability, register and link state
enic: wire V2 SR-IOV enable with admin channel and MBOX
enic: add V2 VF probe with admin channel and PF registration
drivers/net/ethernet/cisco/enic/Makefile | 3 +-
drivers/net/ethernet/cisco/enic/enic.h | 38 +-
drivers/net/ethernet/cisco/enic/enic_admin.c | 595 +++++++++++++++++++++++++
drivers/net/ethernet/cisco/enic/enic_admin.h | 27 ++
drivers/net/ethernet/cisco/enic/enic_main.c | 279 +++++++++++-
drivers/net/ethernet/cisco/enic/enic_mbox.c | 615 ++++++++++++++++++++++++++
drivers/net/ethernet/cisco/enic/enic_mbox.h | 95 ++++
drivers/net/ethernet/cisco/enic/enic_pp.c | 5 +
drivers/net/ethernet/cisco/enic/enic_res.c | 4 +-
drivers/net/ethernet/cisco/enic/vnic_cq.h | 9 +
drivers/net/ethernet/cisco/enic/vnic_devcmd.h | 13 +
drivers/net/ethernet/cisco/enic/vnic_enet.h | 4 +-
12 files changed, 1667 insertions(+), 20 deletions(-)
---
base-commit: 09942ddedcb960f9e78fd817ec33f501d1040c5b
change-id: 20260404-enic-sriov-v2-admin-channel-v2-c0aa3e988833
Best regards,
--
Satish Kharat <satishkh@cisco.com>