From nobody Fri Sep 25 22:18:50 2026 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6EA19314A9E; Tue, 8 Sep 2026 03:52:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=13.77.154.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788839565; cv=none; b=CYou6MptbQr1N+4dknS0iFjHc9L4mWB4WJraLs6r8MNn/xuFRMvbJ6DPYSdgWmiPnu52WRC9R71WShK3KcL7aspejacQTY5J4RxA4cyIIHKAjQVRFyGnd+vwPLT2fWaa6ruV9AsNdPcO/VmpnStyynAOF7AeJ5lbY7/YKPq/O1o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788839565; c=relaxed/simple; bh=tNM9k4wL7LDX/cy1e0ADPnamNjVN5oQ4REEUdBzChrg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=t8VZLCRZHizx97q9lYZkFYFC8bI/r6f+mayGS4dQaYSABMYVj1g7SVw5Xqcz9JnezsVpGPSwZgopaNxNRzRuOkhWvgUl3Xe/oRXrn/kjFr+knCbINOIT0SR7Uc8BK1r8Iz1Xtl9ucbC1MOqzqIYe0S+L3sQphqUDTRsZZbftm3M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=microsoft.com; spf=pass smtp.mailfrom=linux.microsoft.com; arc=none smtp.client-ip=13.77.154.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=microsoft.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.microsoft.com Received: by linux.microsoft.com (Postfix, from userid 1202) id 9605820B7128; Mon, 7 Sep 2026 20:52:04 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 9605820B7128 From: Long Li To: Long Li , Long Li , Konstantin Taranov , Jakub Kicinski , "David S . Miller" , Paolo Abeni , Eric Dumazet , Andrew Lunn , Jason Gunthorpe , Leon Romanovsky , Haiyang Zhang , "K . Y . Srinivasan" , Wei Liu , Dexuan Cui , shradhagupta@linux.microsoft.com, Simon Horman , ernis@linux.microsoft.com, stephen@networkplumber.org, shirazsaleem@microsoft.com Cc: netdev@vger.kernel.org, linux-rdma@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v5 1/4] net: mana: track when the HWC has been handed to the PF Date: Mon, 7 Sep 2026 20:51:55 -0700 Message-ID: <20260908035201.402424-2-longli@microsoft.com> X-Mailer: git-send-email 2.43.7 In-Reply-To: <20260908035201.402424-1-longli@microsoft.com> References: <20260908035201.402424-1-longli@microsoft.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Track possible PF queue ownership with setup_active for HWC reinit. Set it before submitting ESTABLISH_HWC. Cleanup now attempts DESTROY_HWC after submitted setup failures even before MAX_NUM_CQS arrives, adding potentially tens of seconds of polling. Resources are still freed if teardown fails. Signed-off-by: Long Li --- Changes in v5 (v4 -> v5): - Describe additional teardown attempts and polling after setup failure. - Clarify the flag's context lifetime and setup precondition. - Shorten comments; no executable changes from v4. Changes in v4 (standalone net-next rework after the v3 split): - Introduce this dedicated handover-tracking preparation patch. - Set the submission flag inside mana_smc_setup_hwc() and use it as the cleanup gate, including setup failures before MAX_NUM_CQS arrives. - Do not carry the separate net series' teardown-failure resource retentio= n. Changes in v3 (historical net fixes-only posting): - Handover tracking remained in the teardown-safety patch (5/6). - That patch dropped pcie_flr() recovery and retained resources after failed teardown. Those changes are not part of this preparation patch. Changes in v2 (v1 -> v2): - Handover tracking remained part of teardown-safety patch 5/7. - No separate preparation patch was posted. v1: - The combined series introduced setup_active in teardown-safety patch 5/7. drivers/net/ethernet/microsoft/mana/hw_channel.c | 14 +++++++------- drivers/net/ethernet/microsoft/mana/shm_channel.c | 8 +++++++- include/net/mana/hw_channel.h | 3 +++ include/net/mana/shm_channel.h | 2 +- 4 files changed, 18 insertions(+), 9 deletions(-) diff --git a/drivers/net/ethernet/microsoft/mana/hw_channel.c b/drivers/net= /ethernet/microsoft/mana/hw_channel.c index 263e7c4e2934186af037be4c80350a6e322b6771..88e92e94e2e90ff31ca6710a7e9= b8e34b5fa191c 100644 --- a/drivers/net/ethernet/microsoft/mana/hw_channel.c +++ b/drivers/net/ethernet/microsoft/mana/hw_channel.c @@ -683,7 +683,7 @@ static int mana_hwc_establish_channel(struct gdma_conte= xt *gc, u16 *q_depth, cq->mem_info.dma_handle, rq->mem_info.dma_handle, sq->mem_info.dma_handle, - eq->eq.msix_index); + eq->eq.msix_index, &hwc->setup_active); if (err) return err; =20 @@ -815,13 +815,13 @@ void mana_hwc_destroy_channel(struct gdma_context *gc) if (!hwc) return; =20 - /* gc->max_num_cqs is set in mana_hwc_init_event_handler(). If it's - * non-zero, the HWC worked and we should tear down the HWC here. - */ - if (gc->max_num_cqs > 0) { - mana_smc_teardown_hwc(&gc->shm_channel, false); - gc->max_num_cqs =3D 0; + if (hwc->setup_active) { + if (!mana_smc_teardown_hwc(&gc->shm_channel, false)) + hwc->setup_active =3D false; + else + dev_err(hwc->dev, "Failed to tear down HWC\n"); } + gc->max_num_cqs =3D 0; =20 if (hwc->txq) mana_hwc_destroy_wq(hwc, hwc->txq); diff --git a/drivers/net/ethernet/microsoft/mana/shm_channel.c b/drivers/ne= t/ethernet/microsoft/mana/shm_channel.c index d21b5db06e5092d82249fb1053d07f65aa38490c..3cf6a9f8e32c4ff5e5423fc950e= 88082941e21a8 100644 --- a/drivers/net/ethernet/microsoft/mana/shm_channel.c +++ b/drivers/net/ethernet/microsoft/mana/shm_channel.c @@ -129,9 +129,12 @@ void mana_smc_init(struct shm_channel *sc, struct devi= ce *dev, sc->base =3D base; } =20 +/* Requires no outstanding HWC handover. *submitted records possible PF + * ownership, including when setup fails after submission. + */ int mana_smc_setup_hwc(struct shm_channel *sc, bool reset_vf, u64 eq_addr, u64 cq_addr, u64 rq_addr, u64 sq_addr, - u32 eq_msix_index) + u32 eq_msix_index, bool *submitted) { union smc_proto_hdr *hdr; u16 all_addr_h4bits =3D 0; @@ -144,6 +147,8 @@ int mana_smc_setup_hwc(struct shm_channel *sc, bool res= et_vf, u64 eq_addr, int err; int i; =20 + *submitted =3D false; + /* Ensure VF already has possession of shared memory */ err =3D mana_smc_poll_register(sc->base, false); if (err) { @@ -229,6 +234,7 @@ int mana_smc_setup_hwc(struct shm_channel *sc, bool res= et_vf, u64 eq_addr, /* Write 256-message buffer to shared memory (final 32-bit write * triggers HW to set possession bit to PF). */ + *submitted =3D true; dword =3D (u32 *)shm_buf; for (i =3D 0; i < SMC_APERTURE_DWORDS; i++) writel(*dword++, sc->base + i * SMC_BASIC_UNIT); diff --git a/include/net/mana/hw_channel.h b/include/net/mana/hw_channel.h index 16feb39616c1bead1a043b3fadc2e18a90651516..befa09674ce5614a955441e75e4= 80a21aa8695fb 100644 --- a/include/net/mana/hw_channel.h +++ b/include/net/mana/hw_channel.h @@ -199,6 +199,9 @@ struct hw_channel_context { u32 pf_dest_vrcq_id; u32 hwc_timeout; =20 + /* PF may own the queue mappings; state lasts only for this context. */ + bool setup_active; + struct hwc_caller_ctx *caller_ctx; }; =20 diff --git a/include/net/mana/shm_channel.h b/include/net/mana/shm_channel.h index dbabcfb95daf3e87b39a657e3a5f23a1508d4f31..e96387d795259d52aeec7b3fe09= 81825d90a3822 100644 --- a/include/net/mana/shm_channel.h +++ b/include/net/mana/shm_channel.h @@ -20,7 +20,7 @@ void mana_smc_init(struct shm_channel *sc, struct device = *dev, =20 int mana_smc_setup_hwc(struct shm_channel *sc, bool reset_vf, u64 eq_addr, u64 cq_addr, u64 rq_addr, u64 sq_addr, - u32 eq_msix_index); + u32 eq_msix_index, bool *submitted); =20 int mana_smc_teardown_hwc(struct shm_channel *sc, bool reset_vf); =20 --=20 2.43.0 From nobody Fri Sep 25 22:18:50 2026 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 6E94A306741; Tue, 8 Sep 2026 03:52:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=13.77.154.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788839566; cv=none; b=NpZpCGpptIY5jfwHuXoakkCb2RQxjw6N924RlwjwGnGLH5sRyfX+fhkjmcMmWWqWxGZ+MxwE9QrE9r+oRYuLAMkM0h9NwjlrJknCAv+KThx5TOKuARUs22gxCD+93lj2usLkuINQfiNAE12vITwZzJhFCmLetk0rND+OZtyao3o= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788839566; c=relaxed/simple; bh=q3vF4G8WUe72pa0aU8bwBe2iXnaQ4fwoj4HVEAm5LU4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=OP21KlAJFztyBGCxOpsbZQB51J7Jmkmq7jU1gdw3GsUGrfhXjCk97727b5nCZyoLm6mf+rXpzygsh23R/aWBqjKpxYnZSi9Vtjrkj506z6jrDjV5Bu3B444gbe6kgStJaM8WkmB1YAoiUDSAkK7OvJ0EaiPFxAYrUZ7FaB78vI0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=microsoft.com; spf=pass smtp.mailfrom=linux.microsoft.com; arc=none smtp.client-ip=13.77.154.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=microsoft.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.microsoft.com Received: by linux.microsoft.com (Postfix, from userid 1202) id A351620B712B; Mon, 7 Sep 2026 20:52:04 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com A351620B712B From: Long Li To: Long Li , Long Li , Konstantin Taranov , Jakub Kicinski , "David S . Miller" , Paolo Abeni , Eric Dumazet , Andrew Lunn , Jason Gunthorpe , Leon Romanovsky , Haiyang Zhang , "K . Y . Srinivasan" , Wei Liu , Dexuan Cui , shradhagupta@linux.microsoft.com, Simon Horman , ernis@linux.microsoft.com, stephen@networkplumber.org, shirazsaleem@microsoft.com Cc: netdev@vger.kernel.org, linux-rdma@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v5 2/4] net: mana: give each HWC message slot its own completion state Date: Mon, 7 Sep 2026 20:51:56 -0700 Message-ID: <20260908035201.402424-3-longli@microsoft.com> X-Mailer: git-send-email 2.43.7 In-Reply-To: <20260908035201.402424-1-longli@microsoft.com> References: <20260908035201.402424-1-longli@microsoft.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Add per-slot locking, sender/response references and a responded flag; make ctx->error signed. Stop copying after buffer withdrawal or completion. Return a response recorded at the timeout check instead of -ETIMEDOUT, without shortening later waits. Ignore zero timeout-query replies; asynchronous updates remain unfiltered. Timed-out slots remain reusable; the next patch adds quarantine. Signed-off-by: Long Li --- Changes in v5 (v4 -> v5): - Describe buffer withdrawal and response precedence at the timeout check. - State that zero filtering applies only to timeout-query replies. - Shorten the message and comment; no executable changes from v4. Changes in v4 (standalone net-next rework after the v3 split): - Extract per-slot ownership into this preparation patch: lock, sender/response references, responded flag and signed error. - Initialize the slot before publishing its bitmap bit. - Return a recorded response when it races timeout; ignore zero timeout- query replies. Keep timed-out slot reuse here for the next patch to chan= ge. Changes in v3 (historical net fixes-only posting): - A separate stale-response fix (6/6) supplied per-slot locking/references and buffer withdrawal, together with a channel timeout latch and asynchronous zero-timeout filtering. The latter policy is not carried he= re. Changes in v2 (v1 -> v2): - Per-slot state remained within concurrency patch 6/7. - Its channel-lifetime accounting and teardown drain were revised; see the concurrency patch's history. v1: - Per-slot locking and references were introduced in concurrency patch 6/7. .../net/ethernet/microsoft/mana/gdma_main.c | 6 +- .../net/ethernet/microsoft/mana/hw_channel.c | 167 +++++++++++++++--- include/net/mana/hw_channel.h | 18 +- 3 files changed, 163 insertions(+), 28 deletions(-) diff --git a/drivers/net/ethernet/microsoft/mana/gdma_main.c b/drivers/net/= ethernet/microsoft/mana/gdma_main.c index f92b2d0bf926e1b715ff665d37f8173a2103e6fe..8d86de0a334b21d77ab6bfb5789= 17c56404bc856 100644 --- a/drivers/net/ethernet/microsoft/mana/gdma_main.c +++ b/drivers/net/ethernet/microsoft/mana/gdma_main.c @@ -331,7 +331,11 @@ static int mana_gd_query_hwc_timeout(struct pci_dev *p= dev, u32 *timeout_val) if (err || resp.hdr.status) return err ? err : -EPROTO; =20 - *timeout_val =3D resp.timeout_ms; + /* Keep the current timeout on a zero query reply. Asynchronous + * HWC_DATA_CFG_HWC_TIMEOUT updates remain unfiltered. + */ + if (resp.timeout_ms) + *timeout_val =3D resp.timeout_ms; =20 return 0; } diff --git a/drivers/net/ethernet/microsoft/mana/hw_channel.c b/drivers/net= /ethernet/microsoft/mana/hw_channel.c index 88e92e94e2e90ff31ca6710a7e9b8e34b5fa191c..6605e7a9c481bcb11c95f90f627= b7c422b62cc28 100644 --- a/drivers/net/ethernet/microsoft/mana/hw_channel.c +++ b/drivers/net/ethernet/microsoft/mana/hw_channel.c @@ -6,9 +6,11 @@ #include #include =20 +/* Acquire a free inflight message slot, waiting for one if all are in use= . */ static int mana_hwc_get_msg_index(struct hw_channel_context *hwc, u16 *msg= _id) { struct gdma_resource *r =3D &hwc->inflight_msg_res; + struct hwc_caller_ctx *ctx; unsigned long flags; u32 index; =20 @@ -19,6 +21,17 @@ static int mana_hwc_get_msg_index(struct hw_channel_cont= ext *hwc, u16 *msg_id) index =3D find_first_zero_bit(hwc->inflight_msg_res.map, hwc->inflight_msg_res.size); =20 + ctx =3D &hwc->caller_ctx[index]; + reinit_completion(&ctx->comp_event); + /* Take both references (sender + handle_resp) before publishing the + * slot, so an early response cannot free it under the sender. + */ + refcount_set(&ctx->refcnt, 2); + ctx->responded =3D false; + ctx->msg_id =3D index; + ctx->error =3D -EINPROGRESS; + + /* Publish the slot last, after it is fully initialised. */ bitmap_set(hwc->inflight_msg_res.map, index, 1); =20 spin_unlock_irqrestore(&r->lock, flags); @@ -40,6 +53,13 @@ static void mana_hwc_put_msg_index(struct hw_channel_con= text *hwc, u16 msg_id) up(&hwc->sema); } =20 +static void hwc_ctx_put(struct hw_channel_context *hwc, + struct hwc_caller_ctx *ctx) +{ + if (refcount_dec_and_test(&ctx->refcnt)) + mana_hwc_put_msg_index(hwc, ctx->msg_id); +} + static int mana_hwc_verify_resp_msg(const struct hwc_caller_ctx *caller_ct= x, const struct gdma_resp_hdr *resp_msg, u32 resp_len) @@ -90,22 +110,35 @@ static void mana_hwc_handle_resp(struct hw_channel_con= text *hwc, u32 resp_len, } =20 ctx =3D hwc->caller_ctx + msg_id; - err =3D mana_hwc_verify_resp_msg(ctx, resp_msg, resp_len); - if (err) - goto out; =20 - ctx->status_code =3D resp_msg->status; + spin_lock(&ctx->lock); =20 - memcpy(ctx->output_buf, resp_msg, resp_len); -out: + /* Honour a response only while the sender owns the slot (output_buf + * published) and has not already been answered; otherwise drop it as + * premature, stale or duplicate without touching the refcount. + */ + if (!ctx->output_buf || ctx->responded) { + spin_unlock(&ctx->lock); + mana_hwc_post_rx_wqe(hwc->rxq, rx_req); + return; + } + ctx->responded =3D true; + + err =3D mana_hwc_verify_resp_msg(ctx, resp_msg, resp_len); + if (!err) { + ctx->status_code =3D resp_msg->status; + memcpy(ctx->output_buf, resp_msg, resp_len); + } ctx->error =3D err; =20 - /* Must post rx wqe before complete(), otherwise the next rx may - * hit no_wqe error. + /* Post RX WQE before completing =E2=80=94 the next response may arrive + * immediately and needs a posted buffer. */ mana_hwc_post_rx_wqe(hwc->rxq, rx_req); - complete(&ctx->comp_event); + spin_unlock(&ctx->lock); + + hwc_ctx_put(hwc, ctx); } =20 static void mana_hwc_init_event_handler(void *ctx, struct gdma_queue *q_se= lf, @@ -657,8 +690,10 @@ static int mana_hwc_test_channel(struct hw_channel_con= text *hwc, u16 q_depth, if (!ctx) return -ENOMEM; =20 - for (i =3D 0; i < q_depth; ++i) + for (i =3D 0; i < q_depth; ++i) { + spin_lock_init(&ctx[i].lock); init_completion(&ctx[i].comp_event); + } =20 hwc->caller_ctx =3D ctx; =20 @@ -669,6 +704,12 @@ static int mana_hwc_establish_channel(struct gdma_cont= ext *gc, u16 *q_depth, u32 *max_req_msg_size, u32 *max_resp_msg_size) { + /* mana_hwc_init_event_handler() fills the bootstrap fields from hard + * IRQ on GDMA_EQE_HWC_INIT_DATA and then signals hwc_init_eqe_comp on + * GDMA_EQE_HWC_INIT_DONE. The wait_for_completion() below pairs with + * that complete(), so every value stored before INIT_DONE is ordered + * against the reads that follow it here. + */ struct hw_channel_context *hwc =3D gc->hwc.driver_data; struct gdma_queue *rq =3D hwc->rxq->gdma_wq; struct gdma_queue *sq =3D hwc->txq->gdma_wq; @@ -860,13 +901,19 @@ int mana_hwc_send_request(struct hw_channel_context *= hwc, u32 req_len, struct hwc_wq *txq =3D hwc->txq; struct gdma_req_hdr *req_msg; struct hwc_caller_ctx *ctx; + unsigned long flags; + bool drop_resp_ref; u32 dest_vrcq =3D 0; u32 dest_vrq =3D 0; u32 command; + u32 status; + u32 wait_ms; u16 msg_id; int err; =20 - mana_hwc_get_msg_index(hwc, &msg_id); + err =3D mana_hwc_get_msg_index(hwc, &msg_id); + if (err) + return err; =20 tx_wr =3D &txq->msg_buf->reqs[msg_id]; =20 @@ -878,8 +925,11 @@ int mana_hwc_send_request(struct hw_channel_context *h= wc, u32 req_len, } =20 ctx =3D hwc->caller_ctx + msg_id; + + spin_lock_irqsave(&ctx->lock, flags); ctx->output_buf =3D resp; ctx->output_buflen =3D resp_len; + spin_unlock_irqrestore(&ctx->lock, flags); =20 req_msg =3D (struct gdma_req_hdr *)tx_wr->buf_va; if (req) @@ -895,43 +945,108 @@ int mana_hwc_send_request(struct hw_channel_context = *hwc, u32 req_len, dest_vrcq =3D hwc->pf_dest_vrcq_id; } =20 + /* The response-side reference (from get_msg_index) keeps the slot + * alive if hardware responds right after the doorbell. + */ err =3D mana_hwc_post_tx_wqe(txq, tx_wr, dest_vrq, dest_vrcq, false); if (err) { dev_err(hwc->dev, "HWC: Failed to post send WQE: %d\n", err); goto out; } =20 + wait_ms =3D hwc->hwc_timeout; if (!wait_for_completion_timeout(&ctx->comp_event, - (msecs_to_jiffies(hwc->hwc_timeout)))) { - if (hwc->hwc_timeout !=3D 0) + msecs_to_jiffies(wait_ms))) { + /* Clear output_buf so a late response cannot write the caller's + * buffer, then check whether one already arrived + * (error !=3D -EINPROGRESS). + */ + spin_lock_irqsave(&ctx->lock, flags); + ctx->output_buf =3D NULL; + err =3D ctx->error; + status =3D ctx->status_code; + spin_unlock_irqrestore(&ctx->lock, flags); + + if (err !=3D -EINPROGRESS) { + /* A response raced in just after the timeout, so the + * hardware is alive: keep the channel and report what + * that response said rather than a timeout. It may + * itself be an error -- a malformed response leaves + * -EPROTO here -- which is still the answer to this + * command. + */ + hwc_ctx_put(hwc, ctx); + goto check_status; + } + + if (wait_ms !=3D 0) dev_err(hwc->dev, "Command 0x%x timed out: %u ms\n", - command, hwc->hwc_timeout); + command, wait_ms); =20 - /* Reduce further waiting if HWC no response */ + err =3D -ETIMEDOUT; + + /* No-wait teardown (hwc_timeout =3D=3D 0) is expected to expire; + * just release the slot so the next teardown command can reuse + * it. + */ + if (wait_ms =3D=3D 0) + goto out; + + /* Genuine timeout: shorten later waits so subsequent commands + * fail fast instead of each draining the full timeout. + */ if (hwc->hwc_timeout > 1) hwc->hwc_timeout =3D 1; =20 - err =3D -ETIMEDOUT; + /* Release the slot via out:; a late response no longer touches + * it, so the sender must drop the reference here. + */ goto out; } =20 - if (ctx->error) { - err =3D ctx->error; - goto out; - } + /* Clear output_buf and read the result under the lock; the slot may + * be reused after hwc_ctx_put(). + */ + spin_lock_irqsave(&ctx->lock, flags); + ctx->output_buf =3D NULL; + err =3D ctx->error; + status =3D ctx->status_code; + spin_unlock_irqrestore(&ctx->lock, flags); + hwc_ctx_put(hwc, ctx); + +check_status: + if (err) + goto done; =20 - if (ctx->status_code && ctx->status_code !=3D GDMA_STATUS_MORE_ENTRIES) { - if (ctx->status_code =3D=3D GDMA_STATUS_CMD_UNSUPPORTED) { + if (status && status !=3D GDMA_STATUS_MORE_ENTRIES) { + if (status =3D=3D GDMA_STATUS_CMD_UNSUPPORTED) { err =3D -EOPNOTSUPP; - goto out; + goto done; } + if (command !=3D MANA_QUERY_PHY_STAT) dev_err(hwc->dev, "Command 0x%x failed with status: 0x%x\n", - command, ctx->status_code); + command, status); err =3D -EPROTO; - goto out; + goto done; } + + err =3D 0; + goto done; out: - mana_hwc_put_msg_index(hwc, msg_id); + /* Error, no-wait teardown, or timeout: drop the sender's and the + * response-side references. Latch ->responded so a racing response + * is a no-op, and only drop the response-side ref if it has not. + */ + ctx =3D hwc->caller_ctx + msg_id; + spin_lock_irqsave(&ctx->lock, flags); + ctx->output_buf =3D NULL; + drop_resp_ref =3D !ctx->responded; + ctx->responded =3D true; + spin_unlock_irqrestore(&ctx->lock, flags); + if (drop_resp_ref) + refcount_dec(&ctx->refcnt); + hwc_ctx_put(hwc, ctx); +done: return err; } diff --git a/include/net/mana/hw_channel.h b/include/net/mana/hw_channel.h index befa09674ce5614a955441e75e480a21aa8695fb..b377e221aa5c8183825e65ac097= 2c6b9f959004c 100644 --- a/include/net/mana/hw_channel.h +++ b/include/net/mana/hw_channel.h @@ -171,8 +171,24 @@ struct hwc_caller_ctx { void *output_buf; u32 output_buflen; =20 - u32 error; /* Linux error code */ + int error; /* Linux error code (negative errno or 0) */ u32 status_code; + + /* Protects output_buf against concurrent access from + * handle_resp() (CQ interrupt) and the sender timeout path. + */ + spinlock_t lock; + + /* Tracks sender + handle_resp ownership. The last put + * (refcount reaches 0) releases the bitmap slot. + */ + refcount_t refcnt; + u16 msg_id; + + /* Set by the first handle_resp(), or by the sender's timeout path, + * so a later or duplicate response is dropped. + */ + bool responded; }; =20 struct hw_channel_context { --=20 2.43.0 From nobody Fri Sep 25 22:18:50 2026 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by smtp.subspace.kernel.org (Postfix) with ESMTP id AF0F532BF4B; Tue, 8 Sep 2026 03:52:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=13.77.154.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788839567; cv=none; b=ZDhddnAhbxHp+f+p7gskrPb1GUvRTnxRL0Nr09JYhBK+tiZVNysMDZicmL7kbJR8qfTJZFBFafZmzxruZohs0sp5lIC/kBRO5J41p0PMpBCmjwA1kPAEbKU8iFtOHOzG8MnHnYExeXhR9CNHUL0tB/sXMpFhMwtXzQpYThIJfkU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788839567; c=relaxed/simple; bh=XlKhyGXKosX93wop/obYqGozroIOE7whypy8YTAPc1Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=u7RjzRm1kW+E5XpPW1wuKMgGkZtm/UQ97+Mbj7YvIsSYSNaEcalep1kku+2Hogg3dhGhei4nmOEGmycYTCeS2yyfo6As358MJxtrAq4Zo8L54CH0mYbecTglEq2oxCQI650NmL/G3i3t2JoITWjje2paCvheH8XXaenI8SyN1mg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=microsoft.com; spf=pass smtp.mailfrom=linux.microsoft.com; arc=none smtp.client-ip=13.77.154.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=microsoft.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.microsoft.com Received: by linux.microsoft.com (Postfix, from userid 1202) id B050520B7135; Mon, 7 Sep 2026 20:52:04 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com B050520B7135 From: Long Li To: Long Li , Long Li , Konstantin Taranov , Jakub Kicinski , "David S . Miller" , Paolo Abeni , Eric Dumazet , Andrew Lunn , Jason Gunthorpe , Leon Romanovsky , Haiyang Zhang , "K . Y . Srinivasan" , Wei Liu , Dexuan Cui , shradhagupta@linux.microsoft.com, Simon Horman , ernis@linux.microsoft.com, stephen@networkplumber.org, shirazsaleem@microsoft.com Cc: netdev@vger.kernel.org, linux-rdma@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v5 3/4] net: mana: support concurrent HWC requests Date: Mon, 7 Sep 2026 20:51:57 -0700 Message-ID: <20260908035201.402424-4-longli@microsoft.com> X-Mailer: git-send-email 2.43.7 In-Reply-To: <20260908035201.402424-1-longli@microsoft.com> References: <20260908035201.402424-1-longli@microsoft.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Serialize SQ posting and protect HWC lookup and sender accounting with hwc_lock. Teardown stops admission, force-completes requests and drains senders before destroying CQ, TXQ and RXQ. Preserve cancellation errors; return -EPROTO instead of success for a response accepted before posting. Bound FIFO slot admission with down_timeout(), independently of the response wait. Admission expiry returns -ETIMEDOUT even for contention, so existing callers may reset a responsive channel. Quarantine posted requests on timeout, including zero-timeout cleanup, until a response or teardown releases their slot. This closes the late-response reuse window present at depth one. Keep depth one here. Existing lifecycle and timeout-field races are not resolved here. Signed-off-by: Long Li --- Changes in v5 (v4 -> v5): - Return -EPROTO rather than success if a response precedes submission; preserve nonzero cancellation errors and existing reference cleanup. - Document separate FIFO admission and response budgets, including contention-triggered recovery and zero-timeout quarantine. - Correct and shorten locking, publication and teardown comments. Changes in v4 (standalone net-next rework after the v3 split): - Rework former patch 6/7 as patch 3/4 on the new ownership preparation. - Use down_timeout() for semaphore admission and keep the slot until a timed-out request's response arrives or teardown releases it. - Retain quarantine for zero-timeout cleanup; do not use the former channel-wide timeout latch. - Serialize cancellation with SQ posting, preserve already recorded results during teardown, and retain guarded sender draining. Changes in v3 (historical net fixes-only posting): - Defer the concurrency feature from the net submission. - Related slot-timeout ownership work was posted separately as fix 6/6. Changes in v2 (v1 -> v2, former patch 6/7): - Replace atomic sender accounting with an hwc_lock-protected count and wait_event_lock_irq() drain to fence the final sender's wakeup. - Move force-completion and draining before teardown/FLR failure exits. v1: - Introduce waitqueue/bitmap admission, per-slot synchronization, posting serialization and channel teardown gating in patch 6/7. .../net/ethernet/microsoft/mana/gdma_main.c | 38 ++- .../net/ethernet/microsoft/mana/hw_channel.c | 229 ++++++++++++++---- include/net/mana/gdma.h | 9 + include/net/mana/hw_channel.h | 16 ++ 4 files changed, 239 insertions(+), 53 deletions(-) diff --git a/drivers/net/ethernet/microsoft/mana/gdma_main.c b/drivers/net/= ethernet/microsoft/mana/gdma_main.c index 8d86de0a334b21d77ab6bfb578917c56404bc856..eb88bae2b14d86de33e79eb597a= 076a7d6e54436 100644 --- a/drivers/net/ethernet/microsoft/mana/gdma_main.c +++ b/drivers/net/ethernet/microsoft/mana/gdma_main.c @@ -162,6 +162,8 @@ static int mana_gd_init_registers(struct pci_dev *pdev) bool mana_need_log(struct gdma_context *gc, int err) { struct hw_channel_context *hwc; + bool need_log =3D true; + unsigned long flags; =20 if (err !=3D -ETIMEDOUT) return true; @@ -169,11 +171,13 @@ bool mana_need_log(struct gdma_context *gc, int err) if (!gc) return true; =20 + spin_lock_irqsave(&gc->hwc_lock, flags); hwc =3D gc->hwc.driver_data; if (hwc && hwc->hwc_timeout =3D=3D 0) - return false; + need_log =3D false; + spin_unlock_irqrestore(&gc->hwc_lock, flags); =20 - return true; + return need_log; } =20 static int mana_gd_query_max_resources(struct pci_dev *pdev) @@ -391,9 +395,27 @@ static int mana_gd_detect_devices(struct pci_dev *pdev) int mana_gd_send_request(struct gdma_context *gc, u32 req_len, const void = *req, u32 resp_len, void *resp) { - struct hw_channel_context *hwc =3D gc->hwc.driver_data; + struct hw_channel_context *hwc; + unsigned long flags; + int err; + + spin_lock_irqsave(&gc->hwc_lock, flags); + hwc =3D gc->hwc.driver_data; + if (!hwc) { + spin_unlock_irqrestore(&gc->hwc_lock, flags); + return -ENODEV; + } + hwc->active_senders++; + spin_unlock_irqrestore(&gc->hwc_lock, flags); + + err =3D mana_hwc_send_request(hwc, req_len, req, resp_len, resp); + + spin_lock_irqsave(&gc->hwc_lock, flags); + if (--hwc->active_senders =3D=3D 0) + wake_up(&gc->hwc_drain_waitq); + spin_unlock_irqrestore(&gc->hwc_lock, flags); =20 - return mana_hwc_send_request(hwc, req_len, req, resp_len, resp); + return err; } EXPORT_SYMBOL_NS(mana_gd_send_request, "NET_MANA"); =20 @@ -714,6 +736,7 @@ static void mana_serv_reset(struct pci_dev *pdev) { struct gdma_context *gc =3D pci_get_drvdata(pdev); struct hw_channel_context *hwc; + unsigned long flags; int ret; =20 if (!gc) { @@ -723,14 +746,17 @@ static void mana_serv_reset(struct pci_dev *pdev) return; } =20 + spin_lock_irqsave(&gc->hwc_lock, flags); hwc =3D gc->hwc.driver_data; if (!hwc) { + spin_unlock_irqrestore(&gc->hwc_lock, flags); dev_err(&pdev->dev, "MANA service: no HWC\n"); goto out; } =20 /* HWC is not responding in this case, so don't wait */ hwc->hwc_timeout =3D 0; + spin_unlock_irqrestore(&gc->hwc_lock, flags); =20 dev_info(&pdev->dev, "MANA reset cycle start\n"); =20 @@ -1337,6 +1363,7 @@ static int mana_gd_create_dma_region(struct gdma_dev = *gd, if (gmi->nr_pages =3D=3D 0 && !MANA_PAGE_ALIGNED(gmi->virt_addr)) return -EINVAL; =20 + /* The caller must keep the HWC alive throughout queue creation. */ hwc =3D gc->hwc.driver_data; req_msg_size =3D struct_size(req, page_addr_list, num_page); if (req_msg_size > hwc->max_req_msg_size) @@ -1542,7 +1569,9 @@ int mana_gd_verify_vf_version(struct pci_dev *pdev) struct hw_channel_context *hwc; int err; =20 + /* The setup caller must exclude concurrent HWC teardown. */ hwc =3D gc->hwc.driver_data; + mana_gd_init_req_hdr(&req.hdr, GDMA_VERIFY_VF_DRIVER_VERSION, sizeof(req), sizeof(resp)); =20 @@ -2536,6 +2565,7 @@ static int mana_gd_probe(struct pci_dev *pdev, const = struct pci_device_id *ent) =20 mutex_init(&gc->eq_test_event_mutex); mutex_init(&gc->gic_mutex); + spin_lock_init(&gc->hwc_lock); pci_set_drvdata(pdev, gc); gc->bar0_pa =3D pci_resource_start(pdev, 0); gc->bar0_size =3D pci_resource_len(pdev, 0); diff --git a/drivers/net/ethernet/microsoft/mana/hw_channel.c b/drivers/net= /ethernet/microsoft/mana/hw_channel.c index 6605e7a9c481bcb11c95f90f627b7c422b62cc28..a4f7346d285f740c40f4f63c203= 48e30531f1435 100644 --- a/drivers/net/ethernet/microsoft/mana/hw_channel.c +++ b/drivers/net/ethernet/microsoft/mana/hw_channel.c @@ -6,7 +6,6 @@ #include #include =20 -/* Acquire a free inflight message slot, waiting for one if all are in use= . */ static int mana_hwc_get_msg_index(struct hw_channel_context *hwc, u16 *msg= _id) { struct gdma_resource *r =3D &hwc->inflight_msg_res; @@ -14,12 +13,30 @@ static int mana_hwc_get_msg_index(struct hw_channel_con= text *hwc, u16 *msg_id) unsigned long flags; u32 index; =20 - down(&hwc->sema); + /* FIFO slot admission has a separate budget from the response wait. + * Expiry reports -ETIMEDOUT even while earlier requests make progress, + * so callers may initiate recovery on contention alone. + */ + if (down_timeout(&hwc->sema, msecs_to_jiffies(hwc->hwc_timeout))) + return -ETIMEDOUT; =20 spin_lock_irqsave(&r->lock, flags); =20 - index =3D find_first_zero_bit(hwc->inflight_msg_res.map, - hwc->inflight_msg_res.size); + if (!hwc->channel_up) { + spin_unlock_irqrestore(&r->lock, flags); + up(&hwc->sema); + return -ENODEV; + } + + /* The semaphore admits at most r->size holders at a time, so a slot + * acquired above always has a free bit waiting for it here. + */ + index =3D find_first_zero_bit(r->map, r->size); + if (WARN_ON_ONCE(index >=3D r->size)) { + spin_unlock_irqrestore(&r->lock, flags); + up(&hwc->sema); + return -EIO; + } =20 ctx =3D &hwc->caller_ctx[index]; reinit_completion(&ctx->comp_event); @@ -28,11 +45,12 @@ static int mana_hwc_get_msg_index(struct hw_channel_con= text *hwc, u16 *msg_id) */ refcount_set(&ctx->refcnt, 2); ctx->responded =3D false; + ctx->resp_pending =3D true; ctx->msg_id =3D index; ctx->error =3D -EINPROGRESS; =20 /* Publish the slot last, after it is fully initialised. */ - bitmap_set(hwc->inflight_msg_res.map, index, 1); + bitmap_set(r->map, index, 1); =20 spin_unlock_irqrestore(&r->lock, flags); =20 @@ -101,6 +119,7 @@ static void mana_hwc_handle_resp(struct hw_channel_cont= ext *hwc, u32 resp_len, { const struct gdma_resp_hdr *resp_msg =3D rx_req->buf_va; struct hwc_caller_ctx *ctx; + bool release; int err; =20 if (!test_bit(msg_id, hwc->inflight_msg_res.map)) { @@ -113,13 +132,30 @@ static void mana_hwc_handle_resp(struct hw_channel_co= ntext *hwc, u32 resp_len, =20 spin_lock(&ctx->lock); =20 - /* Honour a response only while the sender owns the slot (output_buf - * published) and has not already been answered; otherwise drop it as - * premature, stale or duplicate without touching the refcount. + /* The sender has not published its buffer yet, so nothing asked for + * this response. Keep the slot reserved and drop the message. + */ + if (!ctx->output_buf && !ctx->responded) { + spin_unlock(&ctx->lock); + mana_hwc_post_rx_wqe(hwc->rxq, rx_req); + return; + } + + /* Take the response-side reference away exactly once: releasing it + * is what frees a slot whose sender has already given up. */ - if (!ctx->output_buf || ctx->responded) { + release =3D ctx->resp_pending; + ctx->resp_pending =3D false; + + if (ctx->responded) { + /* The sender timed out and abandoned the slot, or a response + * was already applied. Consume this one without writing + * anything, then release the slot it was holding. + */ spin_unlock(&ctx->lock); mana_hwc_post_rx_wqe(hwc->rxq, rx_req); + if (release) + hwc_ctx_put(hwc, ctx); return; } ctx->responded =3D true; @@ -138,7 +174,8 @@ static void mana_hwc_handle_resp(struct hw_channel_cont= ext *hwc, u32 resp_len, complete(&ctx->comp_event); spin_unlock(&ctx->lock); =20 - hwc_ctx_put(hwc, ctx); + if (release) + hwc_ctx_put(hwc, ctx); } =20 static void mana_hwc_init_event_handler(void *ctx, struct gdma_queue *q_se= lf, @@ -593,6 +630,7 @@ static int mana_hwc_create_wq(struct hw_channel_context= *hwc, hwc_wq->gdma_wq =3D queue; hwc_wq->queue_depth =3D q_depth; hwc_wq->hwc_cq =3D hwc_cq; + spin_lock_init(&hwc_wq->lock); =20 err =3D mana_hwc_alloc_dma_buf(hwc, q_depth, max_msg_size, &hwc_wq->msg_buf); @@ -610,7 +648,7 @@ static int mana_hwc_create_wq(struct hw_channel_context= *hwc, return err; } =20 -static int mana_hwc_post_tx_wqe(const struct hwc_wq *hwc_txq, +static int mana_hwc_post_tx_wqe(struct hwc_wq *hwc_txq, struct hwc_work_request *req, u32 dest_virt_rq_id, u32 dest_virt_rcq_id, bool dest_pf) @@ -649,7 +687,10 @@ static int mana_hwc_post_tx_wqe(const struct hwc_wq *h= wc_txq, req->wqe_req.inline_oob_data =3D tx_oob; req->wqe_req.client_data_unit =3D 0; =20 + spin_lock(&hwc_txq->lock); err =3D mana_gd_post_and_ring(hwc_txq->gdma_wq, &req->wqe_req, NULL); + spin_unlock(&hwc_txq->lock); + if (err) dev_err(dev, "Failed to post WQE on HWC SQ: %d\n", err); return err; @@ -675,6 +716,7 @@ static int mana_hwc_test_channel(struct hw_channel_cont= ext *hwc, u16 q_depth, struct hwc_wq *hwc_rxq =3D hwc->rxq; struct hwc_work_request *req; struct hwc_caller_ctx *ctx; + unsigned long flags; int err; int i; =20 @@ -697,7 +739,19 @@ static int mana_hwc_test_channel(struct hw_channel_con= text *hwc, u16 q_depth, =20 hwc->caller_ctx =3D ctx; =20 - return mana_gd_test_eq(gc, hwc->cq->gdma_eq); + /* Enable admission for the test EQ request. */ + spin_lock_irqsave(&hwc->inflight_msg_res.lock, flags); + hwc->channel_up =3D true; + spin_unlock_irqrestore(&hwc->inflight_msg_res.lock, flags); + + err =3D mana_gd_test_eq(gc, hwc->cq->gdma_eq); + if (err) { + spin_lock_irqsave(&hwc->inflight_msg_res.lock, flags); + hwc->channel_up =3D false; + spin_unlock_irqrestore(&hwc->inflight_msg_res.lock, flags); + } + + return err; } =20 static int mana_hwc_establish_channel(struct gdma_context *gc, u16 *q_dept= h, @@ -797,6 +851,7 @@ int mana_hwc_create_channel(struct gdma_context *gc) u32 max_req_msg_size, max_resp_msg_size; struct gdma_dev *gd =3D &gc->hwc; struct hw_channel_context *hwc; + unsigned long flags; u16 q_depth_max; int err; =20 @@ -805,10 +860,11 @@ int mana_hwc_create_channel(struct gdma_context *gc) return -ENOMEM; =20 gd->gdma_context =3D gc; - gd->driver_data =3D hwc; hwc->gdma_dev =3D gd; hwc->dev =3D gc->dev; hwc->hwc_timeout =3D HW_CHANNEL_WAIT_RESOURCE_TIMEOUT_MS; + hwc->active_senders =3D 0; + init_waitqueue_head(&gc->hwc_drain_waitq); =20 /* HWC's instance number is always 0. */ gd->dev_id.as_uint32 =3D 0; @@ -817,6 +873,11 @@ int mana_hwc_create_channel(struct gdma_context *gc) gd->pdid =3D INVALID_PDID; gd->doorbell =3D INVALID_DOORBELL; =20 + /* Publish for setup; queue initialization below must precede senders. */ + spin_lock_irqsave(&gc->hwc_lock, flags); + gc->hwc.driver_data =3D hwc; + spin_unlock_irqrestore(&gc->hwc_lock, flags); + /* mana_hwc_init_queues() only creates the required data structures, * and doesn't touch the HWC device. */ @@ -851,11 +912,60 @@ int mana_hwc_create_channel(struct gdma_context *gc) =20 void mana_hwc_destroy_channel(struct gdma_context *gc) { + /* The caller must serialize setup and teardown operations. */ struct hw_channel_context *hwc =3D gc->hwc.driver_data; + unsigned long flags; =20 if (!hwc) return; =20 + /* Nonzero num_inflight_msg means queue initialization completed. */ + if (hwc->num_inflight_msg) { + spin_lock_irqsave(&hwc->inflight_msg_res.lock, flags); + hwc->channel_up =3D false; + spin_unlock_irqrestore(&hwc->inflight_msg_res.lock, flags); + } + + /* Block new mana_gd_send_request() references before draining. */ + spin_lock_irqsave(&gc->hwc_lock, flags); + gc->hwc.driver_data =3D NULL; + spin_unlock_irqrestore(&gc->hwc_lock, flags); + + /* Complete occupied slots and drop pending response-side references. */ + if (hwc->caller_ctx) { + struct hwc_caller_ctx *ctx; + bool drop_resp_ref; + int i; + + for (i =3D 0; i < hwc->num_inflight_msg; i++) { + if (!test_bit(i, hwc->inflight_msg_res.map)) + continue; + + ctx =3D &hwc->caller_ctx[i]; + + spin_lock_irqsave(&ctx->lock, flags); + /* Preserve an already recorded result. */ + if (!ctx->responded) + ctx->error =3D -ENODEV; + drop_resp_ref =3D ctx->resp_pending; + ctx->resp_pending =3D false; + ctx->responded =3D true; + complete(&ctx->comp_event); + spin_unlock_irqrestore(&ctx->lock, flags); + + if (drop_resp_ref) + hwc_ctx_put(hwc, ctx); + } + } + + /* Pair with the last sender's wakeup under hwc_lock, so it finishes + * accessing gc before the drain returns. + */ + spin_lock_irq(&gc->hwc_lock); + wait_event_lock_irq(gc->hwc_drain_waitq, + hwc->active_senders =3D=3D 0, gc->hwc_lock); + spin_unlock_irq(&gc->hwc_lock); + if (hwc->setup_active) { if (!mana_smc_teardown_hwc(&gc->shm_channel, false)) hwc->setup_active =3D false; @@ -864,14 +974,28 @@ void mana_hwc_destroy_channel(struct gdma_context *gc) } gc->max_num_cqs =3D 0; =20 + /* Deregister the HWC EQ before freeing the work queues. */ + if (hwc->cq) + mana_hwc_destroy_cq(hwc->gdma_dev->gdma_context, hwc->cq); + if (hwc->txq) mana_hwc_destroy_wq(hwc, hwc->txq); =20 if (hwc->rxq) mana_hwc_destroy_wq(hwc, hwc->rxq); =20 - if (hwc->cq) - mana_hwc_destroy_cq(hwc->gdma_dev->gdma_context, hwc->cq); + if (hwc->caller_ctx) { + struct hwc_caller_ctx *ctx; + int i; + + for (i =3D 0; i < hwc->num_inflight_msg; i++) { + if (!test_bit(i, hwc->inflight_msg_res.map)) + continue; + + ctx =3D &hwc->caller_ctx[i]; + hwc_ctx_put(hwc, ctx); + } + } =20 kfree(hwc->caller_ctx); hwc->caller_ctx =3D NULL; @@ -886,7 +1010,6 @@ void mana_hwc_destroy_channel(struct gdma_context *gc) hwc->hwc_timeout =3D 0; =20 kfree(hwc); - gc->hwc.driver_data =3D NULL; gc->hwc.gdma_context =3D NULL; =20 vfree(gc->cq_table); @@ -903,6 +1026,8 @@ int mana_hwc_send_request(struct hw_channel_context *h= wc, u32 req_len, struct hwc_caller_ctx *ctx; unsigned long flags; bool drop_resp_ref; + bool abandoned =3D false; + bool cancelled; u32 dest_vrcq =3D 0; u32 dest_vrq =3D 0; u32 command; @@ -945,10 +1070,21 @@ int mana_hwc_send_request(struct hw_channel_context = *hwc, u32 req_len, dest_vrcq =3D hwc->pf_dest_vrcq_id; } =20 - /* The response-side reference (from get_msg_index) keeps the slot - * alive if hardware responds right after the doorbell. + /* Serialize cancellation with submission. An unsubmitted request + * cannot succeed, even if an unsolicited response was accepted. */ - err =3D mana_hwc_post_tx_wqe(txq, tx_wr, dest_vrq, dest_vrcq, false); + spin_lock_irqsave(&ctx->lock, flags); + cancelled =3D ctx->responded; + if (cancelled) + err =3D ctx->error ?: -EPROTO; + else + err =3D mana_hwc_post_tx_wqe(txq, tx_wr, dest_vrq, dest_vrcq, + false); + spin_unlock_irqrestore(&ctx->lock, flags); + + if (cancelled) + goto out; + if (err) { dev_err(hwc->dev, "HWC: Failed to post send WQE: %d\n", err); goto out; @@ -965,43 +1101,40 @@ int mana_hwc_send_request(struct hw_channel_context = *hwc, u32 req_len, ctx->output_buf =3D NULL; err =3D ctx->error; status =3D ctx->status_code; + if (err =3D=3D -EINPROGRESS) { + /* Publish abandonment with buffer withdrawal so a late + * response can reclaim the slot. Keep its reference. + */ + ctx->responded =3D true; + abandoned =3D true; + } spin_unlock_irqrestore(&ctx->lock, flags); =20 - if (err !=3D -EINPROGRESS) { - /* A response raced in just after the timeout, so the - * hardware is alive: keep the channel and report what - * that response said rather than a timeout. It may - * itself be an error -- a malformed response leaves - * -EPROTO here -- which is still the answer to this - * command. - */ + if (!abandoned) { + /* A completion won the race with timeout; use its result. */ hwc_ctx_put(hwc, ctx); goto check_status; } =20 - if (wait_ms !=3D 0) + if (wait_ms !=3D 0) { dev_err(hwc->dev, "Command 0x%x timed out: %u ms\n", command, wait_ms); =20 - err =3D -ETIMEDOUT; - - /* No-wait teardown (hwc_timeout =3D=3D 0) is expected to expire; - * just release the slot so the next teardown command can reuse - * it. - */ - if (wait_ms =3D=3D 0) - goto out; + /* Genuine timeout: shorten later waits so subsequent + * commands fail fast instead of each draining the + * full timeout. + */ + if (hwc->hwc_timeout > 1) + hwc->hwc_timeout =3D 1; + } =20 - /* Genuine timeout: shorten later waits so subsequent commands - * fail fast instead of each draining the full timeout. - */ - if (hwc->hwc_timeout > 1) - hwc->hwc_timeout =3D 1; + err =3D -ETIMEDOUT; =20 - /* Release the slot via out:; a late response no longer touches - * it, so the sender must drop the reference here. + /* Drop only the sender's reference; the response-side one is + * what keeps the slot reserved. */ - goto out; + hwc_ctx_put(hwc, ctx); + goto done; } =20 /* Clear output_buf and read the result under the lock; the slot may @@ -1034,14 +1167,12 @@ int mana_hwc_send_request(struct hw_channel_context= *hwc, u32 req_len, err =3D 0; goto done; out: - /* Error, no-wait teardown, or timeout: drop the sender's and the - * response-side references. Latch ->responded so a racing response - * is a no-op, and only drop the response-side ref if it has not. - */ + /* Release any references still held by this unsubmitted request. */ ctx =3D hwc->caller_ctx + msg_id; spin_lock_irqsave(&ctx->lock, flags); ctx->output_buf =3D NULL; - drop_resp_ref =3D !ctx->responded; + drop_resp_ref =3D ctx->resp_pending; + ctx->resp_pending =3D false; ctx->responded =3D true; spin_unlock_irqrestore(&ctx->lock, flags); if (drop_resp_ref) diff --git a/include/net/mana/gdma.h b/include/net/mana/gdma.h index 308950f9b54b0485bac66b80d63e257eaf5f787e..571a533e62e64790f9000d42ab0= e833fe36ccff6 100644 --- a/include/net/mana/gdma.h +++ b/include/net/mana/gdma.h @@ -468,6 +468,15 @@ struct gdma_context { /* Hardware communication channel (HWC) */ struct gdma_dev hwc; =20 + /* Sender drain; the final wakeup runs under hwc_lock. */ + wait_queue_head_t hwc_drain_waitq; + + /* Protects HWC publication, sender references, and short accesses in + * mana_need_log()/mana_serv_reset(). Setup and DMA-region readers + * still require lifecycle ordering. Not all timeout writers use it. + */ + spinlock_t hwc_lock; + /* Azure network adapter */ struct gdma_dev mana; =20 diff --git a/include/net/mana/hw_channel.h b/include/net/mana/hw_channel.h index b377e221aa5c8183825e65ac0972c6b9f959004c..fba27d8620a388a41ae7ddd3bf2= b4792f7beec3d 100644 --- a/include/net/mana/hw_channel.h +++ b/include/net/mana/hw_channel.h @@ -164,6 +164,9 @@ struct hwc_wq { u16 queue_depth; =20 struct hwc_cq *hwc_cq; + + /* Serializes SQ posting; unused for the RQ. */ + spinlock_t lock; }; =20 struct hwc_caller_ctx { @@ -189,6 +192,9 @@ struct hwc_caller_ctx { * so a later or duplicate response is dropped. */ bool responded; + + /* Response-side reference outstanding; protected by lock. */ + bool resp_pending; }; =20 struct hw_channel_context { @@ -196,6 +202,7 @@ struct hw_channel_context { struct device *dev; =20 u16 num_inflight_msg; + u32 max_req_msg_size; =20 u16 hwc_init_q_depth_max; @@ -208,6 +215,9 @@ struct hw_channel_context { struct hwc_wq *txq; struct hwc_cq *cq; =20 + /* Admission permits. Timed-out requests retain theirs until a + * response or teardown releases the slot. + */ struct semaphore sema; struct gdma_resource inflight_msg_res; =20 @@ -215,9 +225,15 @@ struct hw_channel_context { u32 pf_dest_vrcq_id; u32 hwc_timeout; =20 + /* Checked after slot acquisition; cleared on teardown to reject sends. */ + bool channel_up; + /* PF may own the queue mappings; state lasts only for this context. */ bool setup_active; =20 + /* mana_gd_send_request() callers, including waiters; under hwc_lock. */ + unsigned int active_senders; + struct hwc_caller_ctx *caller_ctx; }; =20 --=20 2.43.0 From nobody Fri Sep 25 22:18:50 2026 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by smtp.subspace.kernel.org (Postfix) with ESMTP id AF3B8330656; Tue, 8 Sep 2026 03:52:44 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=13.77.154.182 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788839566; cv=none; b=H0iTcBjyfToG+DxswDredcI58++QNuvZC/pPtvW66kqTyfd7lPFvMjX990PzDukGIPa9MYweuacKl/azDcoOWb8KHdaHH5BiqjhyrHZGTszRa6/hcVFza1mQbPOYeJvw1aV4sDBgxjvBFZoPtqhyU1EiQiUXFYUiB4e7Mi/hnys= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1788839566; c=relaxed/simple; bh=qh6fctz2WVtdB5Ok5ibBH18n9SD1GOZArmgZxdDplPc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=kezDr759/TeCsqo8sjSC4ttR0DZO/bO7cb1U+sbpuETgOEmOlfVayZMWSgirbm3f0rIcFsC1Yyf6SaP3TsyM9MIruLdFd+mQxA9EQD1dDf62miFrmIxERxzq+scSGu2AgcRBHd7rqbUzxYFgCz4xeNVg2GKD67V3H+rGx5BT5g0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=microsoft.com; spf=pass smtp.mailfrom=linux.microsoft.com; arc=none smtp.client-ip=13.77.154.182 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=microsoft.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.microsoft.com Received: by linux.microsoft.com (Postfix, from userid 1202) id BDC8720B7136; Mon, 7 Sep 2026 20:52:04 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com BDC8720B7136 From: Long Li To: Long Li , Long Li , Konstantin Taranov , Jakub Kicinski , "David S . Miller" , Paolo Abeni , Eric Dumazet , Andrew Lunn , Jason Gunthorpe , Leon Romanovsky , Haiyang Zhang , "K . Y . Srinivasan" , Wei Liu , Dexuan Cui , shradhagupta@linux.microsoft.com, Simon Horman , ernis@linux.microsoft.com, stephen@networkplumber.org, shirazsaleem@microsoft.com Cc: netdev@vger.kernel.org, linux-rdma@vger.kernel.org, linux-hyperv@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH net-next v5 4/4] net: mana: add dynamic HWC queue depth with reinit path Date: Mon, 7 Sep 2026 20:51:58 -0700 Message-ID: <20260908035201.402424-5-longli@microsoft.com> X-Mailer: git-send-email 2.43.7 In-Reply-To: <20260908035201.402424-1-longli@microsoft.com> References: <20260908035201.402424-1-longli@microsoft.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Rebuild depth-one HWC queues at the advertised maximum, capped at 128. Preserve the 24-bit depth in u32, bound allocations and round message buffers to a power of two. Advertise support in the driver-version request. Keep bootstrap queues for incompatible initial dimensions. A failed teardown or rebuild instead retries teardown of any submitted queues before restoring depth one; never test queues after an indeterminate destroy result. Abort creation if that teardown also fails. Reject incompatible message sizes after bootstrap fallback, and guard CQ unpublishing when a failed establish left no table. Reset dimensions and routing IDs before each establish. The missing- doorbell guard stops progression to the channel test on every attempt, but does not cover BAR bounds or earlier IRQ rearming. Signed-off-by: Long Li --- Changes in v5 (v4 -> v5): - Handle CQ cleanup when establishment failed before allocating cq_table. - Route initial destroy failures through confirmed teardown and fresh bootstrap reconstruction, rather than testing possibly destroyed queues. - Share the teardown gate with failed larger-depth establishment. - Correct dimension-validation and doorbell-guard scope descriptions; shorten comments without broadening validation policy. Changes in v4 (standalone net-next rework after the v3 split): - Rework former patch 7/7 as patch 4/4 without the net fixes as prerequisi= tes. - Cap depth at 128 and round message-buffer allocations to a power of two. - Require bootstrap message sizes before increasing depth, and validate the rebuilt channel's report against its allocations. - Reset dimensions/routing IDs on each establish and reject a missing doorbell after the handshake. - Retry teardown before restoring depth one after failed re-establishment. - Retain u32 storage of the 24-bit depth and allocation-overflow checks. Changes in v3 (historical net fixes-only posting): - Defer the dynamic-depth feature; it was not included in the net series. Changes in v2 (v1 -> v2): - Retain dynamic-depth patch 7/7 on the revised concurrency prerequisites; rebase the series onto net-next. v1: - Introduce depth-one bootstrap, rebuild at the reported depth, capability advertisement, allocation checks and bootstrap fallback in patch 7/7. .../net/ethernet/microsoft/mana/gdma_main.c | 8 +- .../net/ethernet/microsoft/mana/hw_channel.c | 197 +++++++++++++++++- include/net/mana/gdma.h | 4 + include/net/mana/hw_channel.h | 9 +- 4 files changed, 209 insertions(+), 9 deletions(-) diff --git a/drivers/net/ethernet/microsoft/mana/gdma_main.c b/drivers/net/= ethernet/microsoft/mana/gdma_main.c index eb88bae2b14d86de33e79eb597a076a7d6e54436..78424e1d884f18e6b216d627fc8= 0e77630b0b350 100644 --- a/drivers/net/ethernet/microsoft/mana/gdma_main.c +++ b/drivers/net/ethernet/microsoft/mana/gdma_main.c @@ -1260,15 +1260,17 @@ static void mana_gd_create_cq(const struct gdma_que= ue_spec *spec, static void mana_gd_destroy_cq(struct gdma_context *gc, struct gdma_queue *queue) { + struct gdma_queue **cq_table =3D READ_ONCE(gc->cq_table); u32 id =3D queue->id; =20 - if (id >=3D gc->max_num_cqs) + /* HWC re-establishment can fail before allocating the CQ table. */ + if (!cq_table || id >=3D gc->max_num_cqs) return; =20 - if (!gc->cq_table[id]) + if (!cq_table[id]) return; =20 - gc->cq_table[id] =3D NULL; + cq_table[id] =3D NULL; } =20 int mana_gd_create_hwc_queue(struct gdma_dev *gd, diff --git a/drivers/net/ethernet/microsoft/mana/hw_channel.c b/drivers/net= /ethernet/microsoft/mana/hw_channel.c index a4f7346d285f740c40f4f63c20348e30531f1435..750ee8f72999489a42b2bf6e856= 1480e6ce07133 100644 --- a/drivers/net/ethernet/microsoft/mana/hw_channel.c +++ b/drivers/net/ethernet/microsoft/mana/hw_channel.c @@ -216,7 +216,12 @@ static void mana_hwc_init_event_handler(void *ctx, str= uct gdma_queue *q_self, break; =20 case HWC_INIT_DATA_QUEUE_DEPTH: - hwc->hwc_init_q_depth_max =3D (u16)val; + /* HWC_INIT_DATA_QUEUE_DEPTH is a 24-bit field. Keep + * the full device-reported value here; it is clamped + * and validated in mana_hwc_create_channel() rather + * than silently truncated to u16. + */ + hwc->hwc_init_q_depth_max =3D val; break; =20 case HWC_INIT_DATA_MAX_REQUEST: @@ -546,7 +551,11 @@ static int mana_hwc_alloc_dma_buf(struct hw_channel_co= ntext *hwc, u16 q_depth, =20 dma_buf->num_reqs =3D q_depth; =20 - buf_size =3D MANA_PAGE_ALIGN(q_depth * max_msg_size); + /* mana_gd_alloc_memory() only accepts a power-of-two length, as + * already assumed for the EQ and CQ rings above. The slots are + * carved from the head of the buffer, so any tail is unused. + */ + buf_size =3D roundup_pow_of_two(MANA_PAGE_ALIGN(q_depth * max_msg_size)); =20 gmi =3D &dma_buf->mem_info; err =3D mana_gd_alloc_memory(gc, buf_size, gmi, false); @@ -754,7 +763,7 @@ static int mana_hwc_test_channel(struct hw_channel_cont= ext *hwc, u16 q_depth, return err; } =20 -static int mana_hwc_establish_channel(struct gdma_context *gc, u16 *q_dept= h, +static int mana_hwc_establish_channel(struct gdma_context *gc, u32 *q_dept= h, u32 *max_req_msg_size, u32 *max_resp_msg_size) { @@ -771,6 +780,15 @@ static int mana_hwc_establish_channel(struct gdma_cont= ext *gc, u16 *q_depth, struct gdma_queue *cq =3D hwc->cq->gdma_cq; int err; =20 + /* Do not reuse dimensions or routing IDs from a previous establish. */ + hwc->hwc_init_q_depth_max =3D 0; + hwc->hwc_init_max_req_msg_size =3D 0; + hwc->hwc_init_max_resp_msg_size =3D 0; + gc->hwc.doorbell =3D INVALID_DOORBELL; + gc->hwc.pdid =3D INVALID_PDID; + hwc->pf_dest_vrq_id =3D 0; + hwc->pf_dest_vrcq_id =3D 0; + init_completion(&hwc->hwc_init_eqe_comp); =20 err =3D mana_smc_setup_hwc(&gc->shm_channel, false, @@ -789,6 +807,14 @@ static int mana_hwc_establish_channel(struct gdma_cont= ext *gc, u16 *q_depth, *max_req_msg_size =3D hwc->hwc_init_max_req_msg_size; *max_resp_msg_size =3D hwc->hwc_init_max_resp_msg_size; =20 + /* Reject a missing doorbell before the channel test. This neither + * validates its BAR range nor protects earlier IRQ rearming. + */ + if (gc->hwc.doorbell =3D=3D INVALID_DOORBELL) { + dev_err(hwc->dev, "HWC: no doorbell in init data\n"); + return -EPROTO; + } + /* Both were set in mana_hwc_init_event_handler(). */ if (WARN_ON(cq->id >=3D gc->max_num_cqs)) return -EPROTO; @@ -807,6 +833,12 @@ static int mana_hwc_init_queues(struct hw_channel_cont= ext *hwc, u16 q_depth, { int err; =20 + /* CQ depth is q_depth * 2 (SQ + RQ) passed as u16 to create_cq. + * Cap to prevent u16 truncation. + */ + if (q_depth > U16_MAX / 2) + q_depth =3D U16_MAX / 2; + err =3D mana_hwc_init_inflight_msg(hwc, q_depth); if (err) return err; @@ -846,13 +878,44 @@ static int mana_hwc_init_queues(struct hw_channel_con= text *hwc, u16 q_depth, return err; } =20 +/* Bring-up only: requires no senders or concurrent lifecycle operations. + * This helper does not unpublish the HWC or drain senders. + */ +static void mana_hwc_destroy_queues(struct hw_channel_context *hwc) +{ + struct gdma_context *gc =3D hwc->gdma_dev->gdma_context; + + /* The CQ helper deregisters the HWC EQ before returning. */ + if (hwc->cq) { + mana_hwc_destroy_cq(gc, hwc->cq); + hwc->cq =3D NULL; + } + + kfree(hwc->caller_ctx); + hwc->caller_ctx =3D NULL; + + if (hwc->txq) { + mana_hwc_destroy_wq(hwc, hwc->txq); + hwc->txq =3D NULL; + } + + if (hwc->rxq) { + mana_hwc_destroy_wq(hwc, hwc->rxq); + hwc->rxq =3D NULL; + } + + mana_gd_free_res_map(&hwc->inflight_msg_res); + hwc->num_inflight_msg =3D 0; +} + int mana_hwc_create_channel(struct gdma_context *gc) { u32 max_req_msg_size, max_resp_msg_size; struct gdma_dev *gd =3D &gc->hwc; struct hw_channel_context *hwc; + struct gdma_queue **old_cq_table; unsigned long flags; - u16 q_depth_max; + u32 q_depth_max; int err; =20 hwc =3D kzalloc_obj(*hwc); @@ -896,8 +959,132 @@ int mana_hwc_create_channel(struct gdma_context *gc) goto out; } =20 + if (q_depth_max > HW_CHANNEL_VF_BOOTSTRAP_QUEUE_DEPTH) { + /* Bound DMA allocations before using the 24-bit depth. */ + if (q_depth_max > HW_CHANNEL_MAX_QUEUE_DEPTH) + q_depth_max =3D HW_CHANNEL_MAX_QUEUE_DEPTH; + + /* Keep bootstrap message sizes for mandatory commands. + * Incompatible reports skip rebuilding, not channel creation. + */ + if (max_req_msg_size !=3D HW_CHANNEL_MAX_REQUEST_SIZE || + max_resp_msg_size !=3D HW_CHANNEL_MAX_RESPONSE_SIZE || + (u64)q_depth_max * max_req_msg_size > + U32_MAX - MANA_PAGE_SIZE || + (u64)q_depth_max * max_resp_msg_size > + U32_MAX - MANA_PAGE_SIZE) { + dev_err(hwc->dev, + "HWC: invalid dims q=3D%u req=3D%u resp=3D%u\n", + q_depth_max, max_req_msg_size, + max_resp_msg_size); + q_depth_max =3D HW_CHANNEL_VF_BOOTSTRAP_QUEUE_DEPTH; + goto skip_reinit; + } + + err =3D mana_smc_teardown_hwc(&gc->shm_channel, false); + if (err) { + dev_err(hwc->dev, + "Failed to teardown HWC for reinit: %d\n", + err); + goto reinit_fallback; + } + + hwc->setup_active =3D false; + + /* Unpublish the CQ and drain its EQ before freeing the table. */ + mana_hwc_destroy_queues(hwc); + + old_cq_table =3D gc->cq_table; + gc->cq_table =3D NULL; + gc->max_num_cqs =3D 0; + synchronize_rcu(); + vfree(old_cq_table); + + err =3D mana_hwc_init_queues(hwc, q_depth_max, + max_req_msg_size, + max_resp_msg_size); + if (err) { + dev_err(hwc->dev, "Failed to reinit HWC: %d\n", err); + goto reinit_fallback; + } + + err =3D mana_hwc_establish_channel(gc, &q_depth_max, + &max_req_msg_size, + &max_resp_msg_size); + if (!err && + (q_depth_max < hwc->num_inflight_msg || + max_req_msg_size !=3D HW_CHANNEL_MAX_REQUEST_SIZE || + max_resp_msg_size !=3D HW_CHANNEL_MAX_RESPONSE_SIZE)) { + /* The rebuilt channel must support the allocated depth + * and message sizes. + */ + dev_err(hwc->dev, + "HWC: rebuilt q=3D%u req=3D%u resp=3D%u, built for %u/%u/%u\n", + q_depth_max, max_req_msg_size, + max_resp_msg_size, hwc->num_inflight_msg, + HW_CHANNEL_MAX_REQUEST_SIZE, + HW_CHANNEL_MAX_RESPONSE_SIZE); + err =3D -EPROTO; + } + if (err) { + dev_err(hwc->dev, "Failed to re-establish HWC: %d\n", + err); + goto reinit_fallback; + } + } + + goto skip_reinit; + +reinit_fallback: + /* A failed handshake leaves queue ownership uncertain. */ + if (hwc->setup_active) { + if (mana_smc_teardown_hwc(&gc->shm_channel, false)) { + dev_err(hwc->dev, + "Failed to tear down HWC before bootstrap fallback\n"); + goto out; + } + hwc->setup_active =3D false; + } + + /* The failed establish may not have allocated cq_table. */ + dev_warn(hwc->dev, "HWC reinit failed, falling back to bootstrap depth\n"= ); + + mana_hwc_destroy_queues(hwc); + + old_cq_table =3D gc->cq_table; + gc->cq_table =3D NULL; + gc->max_num_cqs =3D 0; + synchronize_rcu(); + vfree(old_cq_table); + + err =3D mana_hwc_init_queues(hwc, HW_CHANNEL_VF_BOOTSTRAP_QUEUE_DEPTH, + HW_CHANNEL_MAX_REQUEST_SIZE, + HW_CHANNEL_MAX_RESPONSE_SIZE); + if (err) { + dev_err(hwc->dev, "Failed to restore bootstrap HWC: %d\n", err); + goto out; + } + + err =3D mana_hwc_establish_channel(gc, &q_depth_max, &max_req_msg_size, + &max_resp_msg_size); + if (!err && + (max_req_msg_size !=3D HW_CHANNEL_MAX_REQUEST_SIZE || + max_resp_msg_size !=3D HW_CHANNEL_MAX_RESPONSE_SIZE)) { + /* The restored channel must report the allocated message sizes. */ + dev_err(hwc->dev, "HWC: bootstrap reports req=3D%u resp=3D%u\n", + max_req_msg_size, max_resp_msg_size); + err =3D -EPROTO; + } + if (err) { + dev_err(hwc->dev, "Failed to re-establish bootstrap HWC: %d\n", + err); + goto out; + } + +skip_reinit: + err =3D mana_hwc_test_channel(gc->hwc.driver_data, - HW_CHANNEL_VF_BOOTSTRAP_QUEUE_DEPTH, + hwc->num_inflight_msg, max_req_msg_size, max_resp_msg_size); if (err) { dev_err(hwc->dev, "Failed to test HWC: %d\n", err); diff --git a/include/net/mana/gdma.h b/include/net/mana/gdma.h index 571a533e62e64790f9000d42ab0e833fe36ccff6..22dd7c9cecc8a74ed08075bee2b= 6db6f39ea9cc0 100644 --- a/include/net/mana/gdma.h +++ b/include/net/mana/gdma.h @@ -692,6 +692,9 @@ enum { /* Driver supports dynamic interrupt moderation - DIM */ #define GDMA_DRV_CAP_FLAG_1_DYN_INTERRUPT_MODERATION BIT(28) =20 +/* Driver supports dynamic queue depth for HWC */ +#define GDMA_DRV_CAP_FLAG_1_DYN_HWC_QUEUE_DEPTH BIT(29) + /* Driver supports non-contiguous queue buffers */ #define GDMA_DRV_CAP_FLAG_1_NON_CONTIGUOUS_BUFFERS BIT(30) =20 @@ -710,6 +713,7 @@ enum { GDMA_DRV_CAP_FLAG_1_PROBE_RECOVERY | \ GDMA_DRV_CAP_FLAG_1_HANDLE_STALL_SQ_RECOVERY | \ GDMA_DRV_CAP_FLAG_1_HWC_TIMEOUT_RECOVERY | \ + GDMA_DRV_CAP_FLAG_1_DYN_HWC_QUEUE_DEPTH | \ GDMA_DRV_CAP_FLAG_1_EQ_MSI_UNSHARE_MULTI_VPORT | \ GDMA_DRV_CAP_FLAG_1_DYN_INTERRUPT_MODERATION | \ GDMA_DRV_CAP_FLAG_1_NON_CONTIGUOUS_BUFFERS) diff --git a/include/net/mana/hw_channel.h b/include/net/mana/hw_channel.h index fba27d8620a388a41ae7ddd3bf2b4792f7beec3d..dfd92336b2b4dc0e9943732f908= 0887683ebfbbc 100644 --- a/include/net/mana/hw_channel.h +++ b/include/net/mana/hw_channel.h @@ -11,6 +11,13 @@ =20 #define HW_CHANNEL_VF_BOOTSTRAP_QUEUE_DEPTH 1 =20 +/* Upper bound on the device-reported HWC queue depth. The HWC is a + * control channel, so a modest depth already keeps enough management + * commands in flight, while the depth directly scales two coherent DMA + * buffers of q_depth * max_msg_size bytes each. + */ +#define HW_CHANNEL_MAX_QUEUE_DEPTH 128 + #define HWC_INIT_DATA_CQID 1 #define HWC_INIT_DATA_RQID 2 #define HWC_INIT_DATA_SQID 3 @@ -205,7 +212,7 @@ struct hw_channel_context { =20 u32 max_req_msg_size; =20 - u16 hwc_init_q_depth_max; + u32 hwc_init_q_depth_max; u32 hwc_init_max_req_msg_size; u32 hwc_init_max_resp_msg_size; =20 --=20 2.43.0