From nobody Thu Sep 24 14:26:45 2026 Received: from mail-pj2-f12.google.com (mail-pj2-f12.google.com [74.125.227.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AEB913D4125 for ; Wed, 23 Sep 2026 21:07:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.227.140 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790197682; cv=none; b=inFP6h06FJh/ldZ+/Gkxxfc0pn/Izb+xZloHBXNwYuhIhFyY2IAuTysHKNwKWwgb9dA2RiWALJUbiG/LakQvqeMhuQajqbSaAmOz+hEd1EA5zb4h5Pj+9Ovcwj7SDy6otc1vrXmvvtduJ+v6FejyRCK5lZZaahymYc9UPmUAC+8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790197682; c=relaxed/simple; bh=+9ObR8sZ6oxPHgi0+oswXpBH/VC2XkRBGmM9k23GnkE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=CS+YEUOlzmqecSF+QSd6EwyIR36WM2kb37mlBWMXyu0xBUN5gjkH5lI0vbJNBmx4W7svZY0Qw+GbhED7oUZwahP4fr+tLs9or7DmB5cO9/v5k31VyrrOjfVrs08v56MoLqcmQVQ0WoRl1q9u4nBxSSl/uk4BZIIHIn3LjXwDn6E= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=dama.to; spf=none smtp.mailfrom=dama.to; dkim=pass (2048-bit key) header.d=dama-to.20251104.gappssmtp.com header.i=@dama-to.20251104.gappssmtp.com header.b=wfbQgxt9; arc=none smtp.client-ip=74.125.227.140 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=dama.to Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=dama.to Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=dama-to.20251104.gappssmtp.com header.i=@dama-to.20251104.gappssmtp.com header.b="wfbQgxt9" Received: by mail-pj2-f12.google.com with SMTP id d9443c01a7336-2d8fb334ddcso8046285ad.0 for ; Wed, 23 Sep 2026 14:07:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dama-to.20251104.gappssmtp.com; s=20251104; t=1790197674; x=1790802474; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=aWTLyua4wnNjgucbsZIX9QDTCvjLZHoMRXqqxyAwH6M=; b=wfbQgxt9wVGDq+cXS+L+LaX7wdr5CRtLon/t403RyCFwYXjB3hfJzoNqDQ96mfXdXh werfKpVRB+HKdp1JuTW3ssoTBjmLIcs9UDynfmavMxu8ao8bf8rt42nGF1x2/pMC5kg1 BrKPpvv3xxxLWZTvrCosPE7Qpu1RA9nQG7608Mb1ap9g+HRXafDLLMlzP0ZECrV2WQkE 7mPcKF5sdoCZ5BvNJ56FAKX8pHTSzoK9ho/4n885WsffeOsQBIRJIvZGePepgEDWnsZk YmiioLxGDBJUrVtggnHAAtS7bKBzi6NHNPK0ApogyxFThQv4Uj4RO6mpXcjFzCta6Yy7 s8Qw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790197674; x=1790802474; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=aWTLyua4wnNjgucbsZIX9QDTCvjLZHoMRXqqxyAwH6M=; b=hWLzpG7QfyIiT15HUmRCRQ+t8b59xsWXo3WOizaIzUemrrmrjSBfy3yn0BvdlEYeYv fDSfpAgxKGBT5x2S6X0axkbVgXL/DZxRjP31tyJbv8aTJslP7IZZOjqzEUyXu7Xhs6/H pP3Ij2TTzmq8Q9b+OliTURmkKAG36zHYM6ZtmR8cWo1GNDfZOAlViQox5J0XfsxAnUvO FWQtNYoK+KQ0KOMw4qI/Mrx0xmlIJrI3FXZPRMuWcjK6yeOBb5ZoJju7PmEwn5OLdbjR vQuwgspVruLtAhkJXwjICU5mlXhg/1Fzy3UU9AwMaPHPWbe/A0VpNZ2/OAaUg31bWpec 4ZKw== X-Forwarded-Encrypted: i=1; AKwUvBzJB91lskvA0Vx8o/XO0FIo8VhXh7F+7/T0mfccrrGryXISY/AZ2Gk5m17rCso47KupgqgCT8NIlBGM3Sw=@vger.kernel.org X-Gm-Message-State: AFuF++m+vEQUgs6uWUUGc4qw5LC6cdwzZaERvIey5SbYGW90xYwDXnDW Cdu5e9kitTQ5RZwQAXjB+prTrHcSde7CaYb2CmAp0tWh5pODKlBuSrdXuU3kBJCm79g= X-Gm-Gg: AYBFou0XWC72L5czIGMPxCMpp605AOLjLvDN2JKJZ/fJyPZJOArgmsd3YAHT907nMBg y8gEAZqOoFGt0z9Ac6/SI3goK+h32riJxpJkOlDkCjAj+lc0+aCPxMF2yJwK6SpU7z1eu4gVlQ0 6wkn9I3fRYlzKzKywOHQBmlcSw+j/op18vg/eITfIjX/qXFl/ef94dz0poPh5V3pQlujd4Qz/NV B8vvTD8mJkay0t8BOqWwtokQNWIJG36C0VOUDyfqdouful6wNkb3/XL4Bxtksa2/tkj+fMlL+Aj a6T6pXp0JcSVztQTrqWA9JhIdHZ4Jzry1orWv374PzIv0FKmOCzjZsu5KL7uJ1JS36i0off+TXI kFDvSPPyaBp3wpvqcMSlL6pgJNXCX+0N1xnhS83OABkIWniBRlLthpsyq7oLHAw6YRQ6X6qt2yi n+YTHYz7Di4+pTdzy77lZwJAXcpdizAEBTk5vgYakB+JXe+zWqsc5Z X-Received: by 2002:a17:903:908:b0:2dd:c053:a6f5 with SMTP id d9443c01a7336-2df7dc41ad5mr3196195ad.39.1790197674312; Wed, 23 Sep 2026 14:07:54 -0700 (PDT) Received: from localhost ([2a03:2880:2ff:40::]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2df6a516d58sm16931435ad.11.2026.09.23.14.07.53 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Sep 2026 14:07:53 -0700 (PDT) From: Joe Damato To: netdev@vger.kernel.org, Michael Chan , Pavan Chebbi , Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Prashant Sreedharan Cc: horms@kernel.org, linux-kernel@vger.kernel.org, Joe Damato Subject: [RFC net v3 1/3] bnxt_en: return the RING_FREE status to callers Date: Wed, 23 Sep 2026 14:07:40 -0700 Message-ID: <20260923210744.3406861-2-joe@dama.to> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260923210744.3406861-1-joe@dama.to> References: <20260923210744.3406861-1-joe@dama.to> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" hwrm_ring_free_send_msg() reports failure to its caller, returning -EIO when the firmware rejects HWRM_RING_FREE or never answers it. All three ring free helpers that send the command discard the value. Return it instead. No caller acts on it yet, so there is no functional change. Fixes: 74608fc98d28 ("bnxt_en: Ring free response from close path should us= e completion ring") Signed-off-by: Joe Damato --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 48 +++++++++++++---------- 1 file changed, 27 insertions(+), 21 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethern= et/broadcom/bnxt/bnxt.c index d7728d0c5b6e..a7f6facca7b4 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -7660,50 +7660,55 @@ static int hwrm_ring_free_send_msg(struct bnxt *bp, return 0; } =20 -static void bnxt_hwrm_tx_ring_free(struct bnxt *bp, - struct bnxt_tx_ring_info *txr, - bool close_path) +static int bnxt_hwrm_tx_ring_free(struct bnxt *bp, + struct bnxt_tx_ring_info *txr, + bool close_path) { struct bnxt_ring_struct *ring =3D &txr->tx_ring_struct; u32 cmpl_ring_id; + int rc; =20 if (ring->fw_ring_id =3D=3D INVALID_HW_RING_ID) - return; + return 0; =20 cmpl_ring_id =3D close_path ? bnxt_cp_ring_for_tx(bp, txr) : INVALID_HW_RING_ID; - hwrm_ring_free_send_msg(bp, ring, RING_FREE_REQ_RING_TYPE_TX, - cmpl_ring_id); + rc =3D hwrm_ring_free_send_msg(bp, ring, RING_FREE_REQ_RING_TYPE_TX, + cmpl_ring_id); ring->fw_ring_id =3D INVALID_HW_RING_ID; + return rc; } =20 -static void bnxt_hwrm_rx_ring_free(struct bnxt *bp, - struct bnxt_rx_ring_info *rxr, - bool close_path) +static int bnxt_hwrm_rx_ring_free(struct bnxt *bp, + struct bnxt_rx_ring_info *rxr, + bool close_path) { struct bnxt_ring_struct *ring =3D &rxr->rx_ring_struct; u32 grp_idx =3D rxr->bnapi->index; u32 cmpl_ring_id; + int rc; =20 if (ring->fw_ring_id =3D=3D INVALID_HW_RING_ID) - return; + return 0; =20 cmpl_ring_id =3D bnxt_cp_ring_for_rx(bp, rxr); - hwrm_ring_free_send_msg(bp, ring, - RING_FREE_REQ_RING_TYPE_RX, - close_path ? cmpl_ring_id : - INVALID_HW_RING_ID); + rc =3D hwrm_ring_free_send_msg(bp, ring, + RING_FREE_REQ_RING_TYPE_RX, + close_path ? cmpl_ring_id : + INVALID_HW_RING_ID); ring->fw_ring_id =3D INVALID_HW_RING_ID; bp->grp_info[grp_idx].rx_fw_ring_id =3D INVALID_HW_RING_ID; + return rc; } =20 -static void bnxt_hwrm_rx_agg_ring_free(struct bnxt *bp, - struct bnxt_rx_ring_info *rxr, - bool close_path) +static int bnxt_hwrm_rx_agg_ring_free(struct bnxt *bp, + struct bnxt_rx_ring_info *rxr, + bool close_path) { struct bnxt_ring_struct *ring =3D &rxr->rx_agg_ring_struct; u32 grp_idx =3D rxr->bnapi->index; u32 type, cmpl_ring_id; + int rc; =20 if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) type =3D RING_FREE_REQ_RING_TYPE_RX_AGG; @@ -7711,14 +7716,15 @@ static void bnxt_hwrm_rx_agg_ring_free(struct bnxt = *bp, type =3D RING_FREE_REQ_RING_TYPE_RX; =20 if (ring->fw_ring_id =3D=3D INVALID_HW_RING_ID) - return; + return 0; =20 cmpl_ring_id =3D bnxt_cp_ring_for_rx(bp, rxr); - hwrm_ring_free_send_msg(bp, ring, type, - close_path ? cmpl_ring_id : - INVALID_HW_RING_ID); + rc =3D hwrm_ring_free_send_msg(bp, ring, type, + close_path ? cmpl_ring_id : + INVALID_HW_RING_ID); ring->fw_ring_id =3D INVALID_HW_RING_ID; bp->grp_info[grp_idx].agg_fw_ring_id =3D INVALID_HW_RING_ID; + return rc; } =20 static void bnxt_hwrm_cp_ring_free(struct bnxt *bp, --=20 2.53.0-Meta From nobody Thu Sep 24 14:26:45 2026 Received: from mail-pj2-f12.google.com (mail-pj2-f12.google.com [74.125.227.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 986BA3D3CE2 for ; Wed, 23 Sep 2026 21:07:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.227.140 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790197684; cv=none; b=d5ZcSFk2HJiZ5GCkFldLULM3xZRf0VxURWU+/ZDKs60W773P1WMrGK6A4w2LjD9eaVU0B7WEhFsU3BgYlOBIISwBholbgunzZqoK3zaSEAJwn+UFF6n/lh4MHeb2g3l8udU4SLNTv/kqFJkIW7QoP69iLI69yweQnTaz9xXXfT8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790197684; c=relaxed/simple; bh=8B2fWGIDUgftWpMYtaxbZfUEWzXz+xAFmykfFulfqSY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=AKCmVZG67lxA2A56CdIgvEBBYo87+8QZoNtE+4f/e11ClxZvdD/73n5GrsrY59BvJRlNIcS239NA6u1TchisNk5xcHUGx8lC+YEj/k5+8KascIgzB+tcNQwitif5RDToNKy0ksJnDjasd9D54pQ3RAJoMpslXFkeOn887eCXA40= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=dama.to; spf=none smtp.mailfrom=dama.to; dkim=pass (2048-bit key) header.d=dama-to.20251104.gappssmtp.com header.i=@dama-to.20251104.gappssmtp.com header.b=w5/kP4rT; arc=none smtp.client-ip=74.125.227.140 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=dama.to Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=dama.to Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=dama-to.20251104.gappssmtp.com header.i=@dama-to.20251104.gappssmtp.com header.b="w5/kP4rT" Received: by mail-pj2-f12.google.com with SMTP id d9443c01a7336-2d747ed6d6eso9000775ad.2 for ; Wed, 23 Sep 2026 14:07:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dama-to.20251104.gappssmtp.com; s=20251104; t=1790197676; x=1790802476; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=aslm3Yn3BYgx8V54FZU0TWe6uKvDrpLLy9fax5L8J1M=; b=w5/kP4rTscb/mCMMHW3CPd1FkmxYDmkjz7VgryWm6h94s2pHak+1dkIi9RwHeSnmah EJkESKN2KfWrNM9esGF/sQr1oayAIpcBXUqNBtMVUlBjS/sWgCmKsglNZhM6iUngP1Iq hSg+QIYS3dt/JUV60gl4QUeHym/KnunP+asSN2uIq4wsgz7rVGboozEjBTL4cQ2BW0rj cdaCjOrUV+tkoWpb4SWWJr+W7qqitRDZ5XSdvlOJxaULsXE1jTZ9TAMNVK7k4D62TO0r I7gq1MvApRSRczysJZ+Wxi4QnFE4OngjIjIlOjfSE5R8IT/rgbLlkEzKfRq8fhNAH7t9 tijA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790197676; x=1790802476; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=aslm3Yn3BYgx8V54FZU0TWe6uKvDrpLLy9fax5L8J1M=; b=btrxf853KKblAsnjTZ8hWWDJ1ZkF+YsMABC9wdFaQ1K8iP7vu2iWZe/ZiERSexCpG2 cXFoexgA6uW0P6iQ07lp8D7EuxT4TSXighgDEgTaesZgOKQ8ZNPEq9tdedvugki1T6eI q3zvFpxjzA28WOP3jw5CohtFwfX0pEsmp9VHrlNotdPldi1sqQ+ZVdLh8tJ+VX4GDZ9s 8rLNVDa0tM7U8CRUXoxexcm3M5x3MlJeMR713D3WyzXi7lg7S0Nzr2M+542kjESck76z BV9HZqh44GaeiMN2yy5y5VC09qejZ/VSXmAeBCQig3Ei8tW2Z1KwRRML8DlysMc7ghIs 7w2A== X-Forwarded-Encrypted: i=1; AKwUvBxhLF9ULD7iIE2JMilcZHL2a532SAEoUrxzJ7P/ol45tOau/gUp718d4+viWC8WhVD1e++virQgPGVOr+8=@vger.kernel.org X-Gm-Message-State: AFuF++k9QCHr/YEWHvCylQiwsOOwmJnUXd4uZ/ELD2c75lXAy/RupobL X+V2N0Kio2+mosqtfAIOLeGoFVxQGkvlCbFcke3QyJtvSarE9VwtWfwAEGq2R0mD+qY= X-Gm-Gg: AYBFou1uvKEjx5CSczLysZuvBOlE20P2I5TP9C5PRKx6uy/DvG8t1DU6sKdMe3niJQE QoA/MVvTNmXwhCBiZ1H61OU7LS6wn6f+C/yqe3XFG5Z5v8QgqsQjG3u4vEX3Hky4OFBhzWNET9O o4aW/p0ldkmFcMaSrvP1LtEbWO0afslyz4p60v1FOaJvRuuU8rVeDrsJuU0WsoHCyehynPSYitT 81c4ebA38P3t5oK/iaD2tAnT/Yhb8ERN/p/1nepAuGsvlVwoZ8mBsJPGEUHFTFNTfk2AGmmPhZd ZWwq645SWNXbjDaVJ8Qd6RRIgfsmL86YHyVsA0wH/bvLUnH9qmQCopvs3n6GCc/LJbodPxhtC37 YDvzgjDoZ3MfZGqQZ0jwhEWFckiNH0pRgAXibRSJhVwKYToqUZW0SwE1Ixl8DyRaDRL3mTyxcCL If+eBiFy5BRwZUI8aQUPLCO20J3SYUQJNJ0ixyA3puJFEYYlwFEeF3PA== X-Received: by 2002:a17:902:f543:b0:2d8:d4d3:3fbb with SMTP id d9443c01a7336-2df7da839aemr2463045ad.15.1790197676232; Wed, 23 Sep 2026 14:07:56 -0700 (PDT) Received: from localhost ([2a03:2880:2ff:4c::]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-2df6a5eea15sm16450405ad.64.2026.09.23.14.07.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Sep 2026 14:07:55 -0700 (PDT) From: Joe Damato To: netdev@vger.kernel.org, Michael Chan , Pavan Chebbi , Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Prashant Sreedharan Cc: horms@kernel.org, linux-kernel@vger.kernel.org, Joe Damato Subject: [RFC net v3 2/3] bnxt_en: check HWRM response if completion never arrives Date: Wed, 23 Sep 2026 14:07:41 -0700 Message-ID: <20260923210744.3406861-3-joe@dama.to> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260923210744.3406861-1-joe@dama.to> References: <20260923210744.3406861-1-joe@dama.to> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When a command is sent over a completion ring, __hwrm_send() waits for NAPI to consume the completion and gives up if it never arrives, without looking at the response. If a completion is not posted within the timeout, check the response before giving up. If resp_len is set, the sequence id matches, and the valid byte is set then the firmware completed the command and only the notification was lost. Fall through to the normal error_code handling in that case. Several seconds are spent waiting for the completion, so a response that was written at all is complete by the time the wait gives up. There is no need to poll for the valid byte here the way the polling path below has to, where the poll is for a non-zero length and the valid byte at the end of the message may still be on its way. Log the response state on both paths so there is more data when this rare event occurs. Fixes: 74608fc98d28 ("bnxt_en: Ring free response from close path should us= e completion ring") Signed-off-by: Joe Damato --- .../net/ethernet/broadcom/bnxt/bnxt_hwrm.c | 33 ++++++++++++++++--- 1 file changed, 29 insertions(+), 4 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_hwrm.c b/drivers/net/e= thernet/broadcom/bnxt/bnxt_hwrm.c index 5bfabdca7d0e..4feba90f0bf6 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_hwrm.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_hwrm.c @@ -582,11 +582,36 @@ static int __hwrm_send(struct bnxt *bp, struct bnxt_h= wrm_ctx *ctx) } =20 if (READ_ONCE(token->state) !=3D BNXT_HWRM_COMPLETE) { - hwrm_err(bp, ctx, "Resp cmpl intr err msg: 0x%x\n", - req_type); - goto exit; + __le16 resp_seq_id; + u8 valid_byte =3D 0; + + /* The completion ring entry was not delivered for + * some reason. It might be possible that the command + * was carried out even without a completion being + * posted. Check the response before giving up and log + * the state. + */ + dma_rmb(); + resp_seq_id =3D READ_ONCE(ctx->resp->seq_id); + len =3D le16_to_cpu(READ_ONCE(ctx->resp->resp_len)); + if (len && resp_seq_id =3D=3D ctx->req->seq_id) + valid_byte =3D *((u8 *)ctx->resp + len - 1); + + if (!valid_byte) { + hwrm_err(bp, ctx, + "Resp cmpl intr err msg: 0x%x len:%d seq:0x%x/0x%x\n", + req_type, len, + le16_to_cpu(resp_seq_id), + le16_to_cpu(ctx->req->seq_id)); + goto exit; + } + netdev_warn(bp->dev, + "Resp cmpl intr not delivered, msg: 0x%x completed anyway (len:%d = valid:0x%x err:0x%x)\n", + req_type, len, valid_byte, + le16_to_cpu(ctx->resp->error_code)); + } else { + len =3D le16_to_cpu(READ_ONCE(ctx->resp->resp_len)); } - len =3D le16_to_cpu(READ_ONCE(ctx->resp->resp_len)); valid =3D ((u8 *)ctx->resp) + len - 1; } else { __le16 seen_out_of_seq =3D ctx->req->seq_id; /* will never see */ --=20 2.53.0-Meta From nobody Thu Sep 24 14:26:45 2026 Received: from mail-pj2-f12.google.com (mail-pj2-f12.google.com [74.125.227.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0CE2039D3FC for ; Wed, 23 Sep 2026 21:07:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.227.140 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790197684; cv=none; b=qiK0gkcoZgVJJ+LPbm8jk/TmIKdGFA7SGD29GW5VYFWfA8Qxh4IoGKhaSVrSLQuJtKNkEsesvGmdTDNqOX61awPMF8+h95aDc53poFIOa7i1Cbp/VnIa2amubaKa0rRcEYoD/JmRz2g73GJjuZl7fszGCOLhs1Zjg3K/p6EodJM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790197684; c=relaxed/simple; bh=etG2eVx3hAues7xCNYUP+hsrRAg5kIKuKmPWZ/y655k=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=lcF24X9H6PFmcgoLRrOTeb1n98pgJBjo3uiMyF4qziB3fMy4afk6CejXoUYElqJEf3DIJ2t2YgOLWIeXjdz2LaGX5ewk6WaVsdcLk7FipoJluDT8+UcLwOtA9wGlXU2H4elg+yqYOluVxV5ySlA8TlA+bm3w3ViSQZh1Ek7zKC4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=dama.to; spf=none smtp.mailfrom=dama.to; dkim=pass (2048-bit key) header.d=dama-to.20251104.gappssmtp.com header.i=@dama-to.20251104.gappssmtp.com header.b=zXe7/lPm; arc=none smtp.client-ip=74.125.227.140 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=dama.to Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=dama.to Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=dama-to.20251104.gappssmtp.com header.i=@dama-to.20251104.gappssmtp.com header.b="zXe7/lPm" Received: by mail-pj2-f12.google.com with SMTP id 98e67ed59e1d1-396ccd5cf03so1064786a91.1 for ; Wed, 23 Sep 2026 14:07:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dama-to.20251104.gappssmtp.com; s=20251104; t=1790197678; x=1790802478; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=HTfCkaT/O6XE7Szpivk/4PcLQiXftxMknWcst8LU/Z0=; b=zXe7/lPmay8ocyRg1ZzZCX6phcuCHho/Zf4qKmzp30/s6Ru5CMWhEvB+Yw2TUq68ZK aijLH/YV9H1K+OWqdHK4zNqxkX3wQ36I1BDWo7maHfd7/i5dH4/15AyZq1WbrcEn9fRu XTRskXiXSULFJICtf2KwsuLkWBta+vnaTb+wQDgLjHQ5XwebmZ5YxGKdicSNBCoJOzuk jzDxPGvFTo31YXGKu+JqBm3IzDFSEgKtF6EHJTQgv1rg6anILl3IIBjiz2pIukkzcbJm s9V9bjpkS8v59fZ8o5XdH9Eta9RxkLRJ4oeliRCtW6H5fdX3DXYgbhE2CfVZmbytr+ZG 5N4w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790197678; x=1790802478; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=HTfCkaT/O6XE7Szpivk/4PcLQiXftxMknWcst8LU/Z0=; b=KZTVtbKETrl8nt7ilGfGv4IuW32xOODcd472aeHty+F9VQTzgPD4vtiiB8kCThRBLD y+0qRRirLRNTEDTSgQMhdlcx0yUduKD6cb6ac6RTjxHHpLZR1I+FHD1SQQHzPt0nv4qU iFWst7+OADCKJFD3OK/oOrb9moT6pvptrbhywlfD+6Kr2NXCPh8thAMAYrsI0I3/98Pk C9d98mnyCxwyREHRpTc+ob06Dkh5kJ/HeDy67sIOIM45smjsvlafp2Wqu2a5pXbHOz6a 3qjesLkBI2vu1ts9217KTz6Iscj4PyMAPbhYDbGODrTuVuZszaK0GynEwtnbHuyOevEj Ykxg== X-Forwarded-Encrypted: i=1; AKwUvBxUh1bJfyRe6zR6rQSSwssDSYey91p+RweXVsMKVRU+lAeJXRuhOodpz3AbElc4XcCpBofFdv4JpYUejRo=@vger.kernel.org X-Gm-Message-State: AFuF++nkwD8yocWCv8Pu+WND0llaUYyqCtE+M9M5ADd2cG6E+5X23XNn 9QpQO+icnKKgoLyBJwTH/E9PrXdxAC6t4SiZDpcjSSH418m9nO3yqFR2pOKoM1mzxY4= X-Gm-Gg: AYBFou03/58ckfuXbw6Wrtx93X9Zr8rLb0d3AdWs7ZTEP6yQa5xQ6BFsJ79b0DF3fz8 tHDYJaNyxOBo/7cqTKu/2Uce9EWJNkML7SeLScRJ+pVXzYsIgv+6u6Pc1fQM2E8i3FIi+fD6XJ1 4qa6s6KTQRvb2zGr6ZdZrMHRqJ/Gm9IZVkgsx72dWYQXnsZiUNhBX7649no2QpZDOz4irYrwycd h4VQMSrzAWpe+FGamOAAnS2Q/xKC6g1Yp25W85iMU0y5Z+PY4T7iezqct1lAxdJL5lY9hQ9ifhp HsemZizYfKEmDgoNIeH1O+DX2yrKE3EUixsTJHK/BV2S/N4WDFq7iDGQJk5XJRgFLRv+LBUA0fx Dm+fnIWYnsO2bBNU4P1QNY9o1afOGILAhi1j37xhWl2tSjHRGzCs9mMRo/CCFaOMUKJ2kcjO2yj 52ahGO85WYty1O9/KbVypqykHwlNqZ3eKy7mrWdRvHdryi7B1k1oV9WLHdDNu1A/Q= X-Received: by 2002:a17:90b:4b83:b0:398:990e:1587 with SMTP id 98e67ed59e1d1-3a0986ed065mr287752a91.9.1790197678171; Wed, 23 Sep 2026 14:07:58 -0700 (PDT) Received: from localhost ([2a03:2880:2ff:42::]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3a0976c7435sm829146a91.11.2026.09.23.14.07.57 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 23 Sep 2026 14:07:57 -0700 (PDT) From: Joe Damato To: netdev@vger.kernel.org, Michael Chan , Pavan Chebbi , Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Prashant Sreedharan Cc: horms@kernel.org, linux-kernel@vger.kernel.org, Joe Damato Subject: [RFC net v3 3/3] bnxt_en: stop DMA before releasing rings the firmware did not free Date: Wed, 23 Sep 2026 14:07:42 -0700 Message-ID: <20260923210744.3406861-4-joe@dama.to> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260923210744.3406861-1-joe@dama.to> References: <20260923210744.3406861-1-joe@dama.to> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When HWRM_RING_FREE is not answered, bnxt_hwrm_ring_free() clears fw_ring_id and __bnxt_close_nic() goes on to call bnxt_free_mem(), which unmaps the ring memory and the RX buffers that the FW may still be using. This is reachable in production. On a BCM57504 the first sign is the TX watchdog; the close that follows times out a subset of its RING_FREEs and the driver releases those rings anyway: 05:30:12 NETDEV WATCHDOG: transmit queue 0 timed out 6073 ms 05:30:12 Resp cmpl intr err msg: 0x51 x20 05:30:12 hwrm_ring_free type 1 failed x12 05:30:12 hwrm_ring_free type 2 failed x8 05:30:12 AMD-Vi: IO_PAGE_FAULT x3 Count the rings the firmware did not free and report that to the caller so it can decide what to do. bnxt_hwrm_resource_free() still frees the remaining firmware resources before returning the error, so the shutdown can send every message it needs to before the device is stopped. The device stays unusable until a firmware reset. Fixes: 74608fc98d28 ("bnxt_en: Ring free response from close path should us= e completion ring") Signed-off-by: Joe Damato --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 48 +++++++++++++++++------ 1 file changed, 35 insertions(+), 13 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethern= et/broadcom/bnxt/bnxt.c index a7f6facca7b4..332fb374db5e 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -7754,21 +7754,25 @@ static void bnxt_clear_one_cp_ring(struct bnxt *bp,= struct bnxt_cp_ring_info *cp memset(cpr->cp_desc_ring[i], 0, size); } =20 -static void bnxt_hwrm_ring_free(struct bnxt *bp, bool close_path) +static int bnxt_hwrm_ring_free(struct bnxt *bp, bool close_path) { + int stuck =3D 0; u32 type; int i; =20 if (!bp->bnapi) - return; + return 0; =20 for (i =3D 0; i < bp->tx_nr_rings; i++) - bnxt_hwrm_tx_ring_free(bp, &bp->tx_ring[i], close_path); + if (bnxt_hwrm_tx_ring_free(bp, &bp->tx_ring[i], close_path)) + stuck++; =20 bnxt_cancel_dim(bp); for (i =3D 0; i < bp->rx_nr_rings; i++) { - bnxt_hwrm_rx_ring_free(bp, &bp->rx_ring[i], close_path); - bnxt_hwrm_rx_agg_ring_free(bp, &bp->rx_ring[i], close_path); + if (bnxt_hwrm_rx_ring_free(bp, &bp->rx_ring[i], close_path)) + stuck++; + if (bnxt_hwrm_rx_agg_ring_free(bp, &bp->rx_ring[i], close_path)) + stuck++; } =20 /* The completion rings are about to be freed. After that the @@ -7798,6 +7802,19 @@ static void bnxt_hwrm_ring_free(struct bnxt *bp, boo= l close_path) bp->grp_info[i].cp_fw_ring_id =3D INVALID_HW_RING_ID; } } + + if (!stuck) + return 0; + + netdev_err(bp->dev, "Firmware did not free %d ring(s)\n", stuck); + return -EIO; +} + +static void bnxt_stop_dma(struct bnxt *bp) +{ + netdev_err(bp->dev, + "Disabling DMA before releasing ring memory, a firmware reset is requ= ired\n"); + pci_clear_master(bp->pdev); } =20 static int __bnxt_trim_rings(struct bnxt *bp, int *rx, int *tx, int max, @@ -10832,16 +10849,19 @@ static void bnxt_clear_vnic(struct bnxt *bp) bnxt_hwrm_vnic_ctx_free(bp); } =20 -static void bnxt_hwrm_resource_free(struct bnxt *bp, bool close_path, - bool irq_re_init) +static int bnxt_hwrm_resource_free(struct bnxt *bp, bool close_path, + bool irq_re_init) { + int rc; + bnxt_clear_vnic(bp); - bnxt_hwrm_ring_free(bp, close_path); + rc =3D bnxt_hwrm_ring_free(bp, close_path); bnxt_hwrm_ring_grp_free(bp); if (irq_re_init) { bnxt_hwrm_stat_ctx_free(bp); bnxt_hwrm_free_tunnel_ports(bp); } + return rc; } =20 static int bnxt_hwrm_set_br_mode(struct bnxt *bp, u16 br_mode) @@ -11363,15 +11383,15 @@ static int bnxt_init_chip(struct bnxt *bp, bool i= rq_re_init) return 0; =20 err_out: - bnxt_hwrm_resource_free(bp, 0, true); + if (bnxt_hwrm_resource_free(bp, 0, true)) + bnxt_stop_dma(bp); =20 return rc; } =20 static int bnxt_shutdown_nic(struct bnxt *bp, bool irq_re_init) { - bnxt_hwrm_resource_free(bp, 1, irq_re_init); - return 0; + return bnxt_hwrm_resource_free(bp, 1, irq_re_init); } =20 static int bnxt_init_nic(struct bnxt *bp, bool irq_re_init) @@ -13422,7 +13442,8 @@ int bnxt_half_open_nic(struct bnxt *bp) */ void bnxt_half_close_nic(struct bnxt *bp) { - bnxt_hwrm_resource_free(bp, false, true); + if (bnxt_hwrm_resource_free(bp, false, true)) + bnxt_stop_dma(bp); bnxt_del_napi(bp); bnxt_free_skbs(bp); bnxt_free_mem(bp, true); @@ -13501,7 +13522,8 @@ static void __bnxt_close_nic(struct bnxt *bp, bool = irq_re_init, if (BNXT_SUPPORTS_MULTI_RSS_CTX(bp)) bnxt_clear_rss_ctxs(bp); /* Flush rings and disable interrupts */ - bnxt_shutdown_nic(bp, irq_re_init); + if (bnxt_shutdown_nic(bp, irq_re_init)) + bnxt_stop_dma(bp); =20 /* TODO CHIMP_FW: Link/PHY related cleanup if (link_re_init) */ =20 --=20 2.53.0-Meta