From nobody Thu Sep 24 15:11:03 2026 Received: from mail-pj2-f13.google.com (mail-pj2-f13.google.com [74.125.227.141]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7842648E0DE for ; Tue, 22 Sep 2026 18:24:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.227.141 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790101462; cv=none; b=I+zym0DAJQfY6X96bMljyAsbM0B61lHRM+iTypkUfsYQmi5iku/1AX9ZmhjSXXx8ORooxz3YH6/vzES083hvWrRMKgy7Y27dLr+T/O+bbrIgRF0TCRQqYLwLo/kIpCytAZxVil2Le4c5HDSfpf7ufiTtFOFH8q2LRHcqTxpbUZ0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790101462; c=relaxed/simple; bh=+9ObR8sZ6oxPHgi0+oswXpBH/VC2XkRBGmM9k23GnkE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=MvKkdDdHPYLuRbUyDky7iMacJDrC5Qhq/XEPFAxTtOh3L73HlKyKHF5wG3mgP92ZhzwFmjzRUXlj/jWv5v+jLLDUQQSs0q/+w0LAkSqK/dSqnsmbfFPb26OE2k8pt4gtoIlDuq3cie3icKfqk/GedL3ybipJ+bYQkUH9n2R1qG4= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=dama.to; spf=none smtp.mailfrom=dama.to; dkim=pass (2048-bit key) header.d=dama-to.20251104.gappssmtp.com header.i=@dama-to.20251104.gappssmtp.com header.b=h+bxILWJ; arc=none smtp.client-ip=74.125.227.141 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=dama.to Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=dama.to Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=dama-to.20251104.gappssmtp.com header.i=@dama-to.20251104.gappssmtp.com header.b="h+bxILWJ" Received: by mail-pj2-f13.google.com with SMTP id 98e67ed59e1d1-39d654f02baso153446a91.3 for ; Tue, 22 Sep 2026 11:24:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dama-to.20251104.gappssmtp.com; s=20251104; t=1790101459; x=1790706259; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=aWTLyua4wnNjgucbsZIX9QDTCvjLZHoMRXqqxyAwH6M=; b=h+bxILWJ0fcOr2kr70/9TAJ3oeIHe5ZsLkTkvIHLFRBXSqPFiEMI/FQAPR+OXLIX+G 2LHQiQ85hwpPcn90NmmG1qQG3vGypWMpSU8ffjh0nHpeXJt1efbnLZC3dl70HESEmLhj 1ejiEgrFjCDYV8iloZyCkSdG6VMKSy812Sin6Q9+YPGF591swkJneHWqZ0IJUR1bx3cb TYN8FUdPVv+JBWz2X/7X7kJiRd+vPv2KteUWdR8ce5dxdoNJqsAMpFI4x3oe+TYe/W0W QXOAsHdr+ytE/5Zv4KtKcDDvZdw9gf5h+/2qyli2janrqyyMucskgVHCbJoYdP9I0+sF UDzw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790101459; x=1790706259; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=aWTLyua4wnNjgucbsZIX9QDTCvjLZHoMRXqqxyAwH6M=; b=FgAZRx6O28rjKxhDWdb7Yq6KX/1zs3/md6UM3qceVJB5PFCbIGsk+SeSYri1RmXNNx pxfSIBwYkRXXNNacLB3XG0veKB4ajcJJaVke9vTINtvsv5Og8FG0NkUuNOnlWBSccicw jcgoUcfJtCu00LrlviBCTl9hpzzA9f3aQinxwrJzC67YrcnY25OyR8L1YyBd1bQoy4s3 8iP6pE8uK1PkKu/ydqHlZyDr8UJOjdTGg/hfbO805tBrG4vQV7nEU/vkbkcyzZUOLVj9 oqbXHfHXAGjY7y0vM4eTrtbb23vynNlZ/PJDLM/py3/lUo1RTQq+Y+xe3QhkfzhqQzQv pe/A== X-Forwarded-Encrypted: i=1; AKwUvBzrmWbQdm/OfbJPBS9JilqheTy8aWSyUQ/u5d/MNUMNTzJ46LHbzNGKtMJw1Yebf6bpx7BwmkloMR1vTyM=@vger.kernel.org X-Gm-Message-State: AFuF++mP6OaugsUOKqDGqAW7QAF8D1GLU5QwrgdMNU+PjJI3HrZ1j5Y9 drWX6oAr46NgXulM2+5OUcJFSDuzxV2I3uSCQVaA90llx6EwhGWra3pvGej2tJTvnNJyPFe/1Z9 CTVu6 X-Gm-Gg: AYBFou07hVKC0+kMLe/zo/AD3tWYnCgk8HNxNneaG3KivHSy93JZ5LgZz2EHZnEWw4o SMpBVoVJdh6kE5NzdemOJRAliFY+7JrvSkMROvLO0CVCoKhkC4Ys2e/ftooBOAyCnhZuc2Q/Anu /5qN8/V9p3UpOnDiQa2ghuoZAAs74kZ1hisowu2l8yMXWa/z0gRInJBEkJ3zQ3IyRcGgPVK7EaR 5mDps7nOYFJzqbw9CzwzjWpLpGMWgHpY+hPaX8f2BGTmErAKwIZ7PFTBfaYMDo3RCrc/4JUHQRf gkPCxE91Wldfh9wJHglbVWiYOZsp2uSkRx3oePC/bDiJahGHVx0HU2LkWk/p0GQpfQw0yTlfMQ4 rrXKpGHbpvk4nV0Oc15mnrltdX5eRDe74Gx16jKbzYxN8d2yyWlABlMOTksiYM7BabaES89i151 qqUjFIqpAqfQlXl8yA1baiPtJfBBsSRjeMhBEo1yndT5khbnLPkoip X-Received: by 2002:a17:90b:4c87:b0:3a0:516d:9f7e with SMTP id 98e67ed59e1d1-3a07e752ae8mr224806a91.45.1790101459604; Tue, 22 Sep 2026 11:24:19 -0700 (PDT) Received: from localhost ([2a03:2880:2ff:44::]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3a07ddf1cf3sm579154a91.9.2026.09.22.11.24.18 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Sep 2026 11:24:19 -0700 (PDT) From: Joe Damato To: netdev@vger.kernel.org, Michael Chan , Pavan Chebbi , Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Prashant Sreedharan Cc: horms@kernel.org, linux-kernel@vger.kernel.org, Joe Damato Subject: [RFC net v2 1/3] bnxt_en: return the RING_FREE status to callers Date: Tue, 22 Sep 2026 11:24:01 -0700 Message-ID: <20260922182405.1290749-2-joe@dama.to> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260922182405.1290749-1-joe@dama.to> References: <20260922182405.1290749-1-joe@dama.to> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" hwrm_ring_free_send_msg() reports failure to its caller, returning -EIO when the firmware rejects HWRM_RING_FREE or never answers it. All three ring free helpers that send the command discard the value. Return it instead. No caller acts on it yet, so there is no functional change. Fixes: 74608fc98d28 ("bnxt_en: Ring free response from close path should us= e completion ring") Signed-off-by: Joe Damato --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 48 +++++++++++++---------- 1 file changed, 27 insertions(+), 21 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethern= et/broadcom/bnxt/bnxt.c index d7728d0c5b6e..a7f6facca7b4 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -7660,50 +7660,55 @@ static int hwrm_ring_free_send_msg(struct bnxt *bp, return 0; } =20 -static void bnxt_hwrm_tx_ring_free(struct bnxt *bp, - struct bnxt_tx_ring_info *txr, - bool close_path) +static int bnxt_hwrm_tx_ring_free(struct bnxt *bp, + struct bnxt_tx_ring_info *txr, + bool close_path) { struct bnxt_ring_struct *ring =3D &txr->tx_ring_struct; u32 cmpl_ring_id; + int rc; =20 if (ring->fw_ring_id =3D=3D INVALID_HW_RING_ID) - return; + return 0; =20 cmpl_ring_id =3D close_path ? bnxt_cp_ring_for_tx(bp, txr) : INVALID_HW_RING_ID; - hwrm_ring_free_send_msg(bp, ring, RING_FREE_REQ_RING_TYPE_TX, - cmpl_ring_id); + rc =3D hwrm_ring_free_send_msg(bp, ring, RING_FREE_REQ_RING_TYPE_TX, + cmpl_ring_id); ring->fw_ring_id =3D INVALID_HW_RING_ID; + return rc; } =20 -static void bnxt_hwrm_rx_ring_free(struct bnxt *bp, - struct bnxt_rx_ring_info *rxr, - bool close_path) +static int bnxt_hwrm_rx_ring_free(struct bnxt *bp, + struct bnxt_rx_ring_info *rxr, + bool close_path) { struct bnxt_ring_struct *ring =3D &rxr->rx_ring_struct; u32 grp_idx =3D rxr->bnapi->index; u32 cmpl_ring_id; + int rc; =20 if (ring->fw_ring_id =3D=3D INVALID_HW_RING_ID) - return; + return 0; =20 cmpl_ring_id =3D bnxt_cp_ring_for_rx(bp, rxr); - hwrm_ring_free_send_msg(bp, ring, - RING_FREE_REQ_RING_TYPE_RX, - close_path ? cmpl_ring_id : - INVALID_HW_RING_ID); + rc =3D hwrm_ring_free_send_msg(bp, ring, + RING_FREE_REQ_RING_TYPE_RX, + close_path ? cmpl_ring_id : + INVALID_HW_RING_ID); ring->fw_ring_id =3D INVALID_HW_RING_ID; bp->grp_info[grp_idx].rx_fw_ring_id =3D INVALID_HW_RING_ID; + return rc; } =20 -static void bnxt_hwrm_rx_agg_ring_free(struct bnxt *bp, - struct bnxt_rx_ring_info *rxr, - bool close_path) +static int bnxt_hwrm_rx_agg_ring_free(struct bnxt *bp, + struct bnxt_rx_ring_info *rxr, + bool close_path) { struct bnxt_ring_struct *ring =3D &rxr->rx_agg_ring_struct; u32 grp_idx =3D rxr->bnapi->index; u32 type, cmpl_ring_id; + int rc; =20 if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) type =3D RING_FREE_REQ_RING_TYPE_RX_AGG; @@ -7711,14 +7716,15 @@ static void bnxt_hwrm_rx_agg_ring_free(struct bnxt = *bp, type =3D RING_FREE_REQ_RING_TYPE_RX; =20 if (ring->fw_ring_id =3D=3D INVALID_HW_RING_ID) - return; + return 0; =20 cmpl_ring_id =3D bnxt_cp_ring_for_rx(bp, rxr); - hwrm_ring_free_send_msg(bp, ring, type, - close_path ? cmpl_ring_id : - INVALID_HW_RING_ID); + rc =3D hwrm_ring_free_send_msg(bp, ring, type, + close_path ? cmpl_ring_id : + INVALID_HW_RING_ID); ring->fw_ring_id =3D INVALID_HW_RING_ID; bp->grp_info[grp_idx].agg_fw_ring_id =3D INVALID_HW_RING_ID; + return rc; } =20 static void bnxt_hwrm_cp_ring_free(struct bnxt *bp, --=20 2.53.0-Meta From nobody Thu Sep 24 15:11:03 2026 Received: from mail-pj2-f42.google.com (mail-pj2-f42.google.com [74.125.227.170]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6DEDC48F015 for ; Tue, 22 Sep 2026 18:24:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.227.170 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790101465; cv=none; b=jpmm+HzD35z2tmjEyIzgGGWfdMmnQw8Gv9SJz209dz/Lcpugc8E9NxioOYnjriqVsTuWnjgineYiVlmhWrz6W2aClWDvVwFZ0F3TEmTLKls57j7nnr0HVUW9VcVrJsH1sk7w2ISYowOxtaATF116DgBub9DA4uitlsueaLgm6Mg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790101465; c=relaxed/simple; bh=wJWRwJSOwUJCCW1ziry9J4YSKqISemZjbWRRzBnqnL4=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=otKxrk3ggQRpZG3xLLG75J2k4BwvXtRDu7hc6qlX89UU/GrQbfV6FPBtjXY2ORFHdEIGV0bIJE5Iv9MSJ/6RQslP2hQ1E7etHexIR1NNwgmIwlt7/om3iG1Cc7s5VE7CXlMHLUYyN8Oo+6LLrhR0xr7YBDbzuduUtHo8/3bN3FE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=dama.to; spf=none smtp.mailfrom=dama.to; dkim=pass (2048-bit key) header.d=dama-to.20251104.gappssmtp.com header.i=@dama-to.20251104.gappssmtp.com header.b=ehSt+aNC; arc=none smtp.client-ip=74.125.227.170 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=dama.to Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=dama.to Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=dama-to.20251104.gappssmtp.com header.i=@dama-to.20251104.gappssmtp.com header.b="ehSt+aNC" Received: by mail-pj2-f42.google.com with SMTP id 98e67ed59e1d1-396ccb65437so131423a91.3 for ; Tue, 22 Sep 2026 11:24:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dama-to.20251104.gappssmtp.com; s=20251104; t=1790101462; x=1790706262; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=xO1R65BfMNPc/29NEB8dWdJDbp318jymstIuoTuu6l8=; b=ehSt+aNCK/bSUvRlxLpdEeO7iKohkTqoNcwVpCMNl5h/8SyiZm9/7SunT93QhIsRZC EICjEjE6yChy1mY9QnF3/3vP2NtfS0Zl+dmPynkr0pQZd21MPA9n37KqxglL0X01K3N/ 2fb+bUfuwpD0hLIf3kNAklxVg/3rGOZgzC4ZIZ4P33K2etSmLdEbO2c5eeJ6f00BVE6r KpcTaAQMOX8hKDC51H6K80p3Ye1/Jb2j6jSRRzzO82AVym7G5Td5jIn7t7vMcGWsSfFh RfwKkHWcXW+W7jueoUSDwO0FBtftpOwZq9O4HZuGViLsoFLGQBc0FvxoQbqmnFGpoGht 7hHw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790101462; x=1790706262; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=xO1R65BfMNPc/29NEB8dWdJDbp318jymstIuoTuu6l8=; b=O7jsBQeENBuzgTbA+1nwZVbY+lTwoz6MTnsTbrs9YqExMhnyY4DIeEZu3u2jEaj8xV Fe6ipbvcPjKgE8/3QgoOCuIDqH2F8tSBNkYo+28VbfxdcIgSkYt8KtGttyabODVJqiKg reAS1sgCHjrd3J8YuM/lgCRTvXe6pss2VB1ZageRHYR5qEdmG97n4sQOnCvJVBsJVWou cN7wmN6JQHrRj5mSVw6I4Ja/C3d9hKAfENf/AdioQF1UWXaHDH8/e2ugV0PeIq3tyd5T 303sWa2QTmd1bucWpYjbL59SbFF5nqXOEtpHXwtSqJ1qvtfseElxclbwhJWv8mv4UPxW veWA== X-Forwarded-Encrypted: i=1; AKwUvBwwdrhogwwtGVKqj2ICvM9vFuKuc2IMlBBSFgm0LBKBaNp+36lW0tHap3NdXNrEWQSr2GbFEWwHy5h4o9E=@vger.kernel.org X-Gm-Message-State: AFuF++kiypwE78f0dO8jitZWE5eAlPFezNn7UnhGBloPgUMF237cHtAc +GDDV0ITed4XtWDDawgCYQT3NpMP/P66Qc4txh7eUNikqI+Gh1VyGzgm/9dbr4EVjzU= X-Gm-Gg: AYBFou3sy1V7cIb4uIdjidAvG/vTqTYunE15/WSV0ALF70zedqQr7waagXhQYxFXZGv xAmHEJPXrwlfG3C8bX2LqAT3bWRTPgjLgm6GvKCvMfFuUclk5QCfSuCHbF+Sc+vRqnWoctoHuoh xNvtLM6OYP1Au9NaDiQMJV/IL/I2SzGneGfhhhvqmzzeSlM9y1FvP9kTmMf3Lb/vjrVCgshFS/X cCu5EaEkci805vI5dRV8giOr0Nf89ieHcsVJ1MEoi69BRD0qB9WLDIr6zLiYNRwpbQXlx5vBBQ5 I/GP1cnNdH8OQgtcgjJDdpnceXPA8iwSrhmCw7DJzDuwTYMpXvWBsd1zmVYxiuMK3qg1WOfa2ee yJxjwRxwjWdbbwdksYk4qjc+YhwfVHJWW++Bh4B/8/HfnAgdCGokwKEXN5a5ylxhYTuHUFA/HJu OqgFL0uNu8i1vVgrsfT9Ocx2XXRMyqeoR62ViX1MhYnZT4GI/fVb8= X-Received: by 2002:a17:90b:4a45:b0:39e:6c6a:4b6c with SMTP id 98e67ed59e1d1-3a07e6bacfdmr240793a91.54.1790101461609; Tue, 22 Sep 2026 11:24:21 -0700 (PDT) Received: from localhost ([2a03:2880:2ff:4::]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-3a07db883b5sm663652a91.2.2026.09.22.11.24.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Sep 2026 11:24:21 -0700 (PDT) From: Joe Damato To: netdev@vger.kernel.org, Michael Chan , Pavan Chebbi , Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Prashant Sreedharan Cc: horms@kernel.org, linux-kernel@vger.kernel.org, Joe Damato Subject: [RFC net v2 2/3] bnxt_en: check HWRM response if completion never arrives Date: Tue, 22 Sep 2026 11:24:02 -0700 Message-ID: <20260922182405.1290749-3-joe@dama.to> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260922182405.1290749-1-joe@dama.to> References: <20260922182405.1290749-1-joe@dama.to> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When a command is sent over a completion ring, __hwrm_send() waits for NAPI to consume the completion and gives up if it never arrives, without looking at the response. If a completion is not posted within the timeout, check the response before giving up. If resp_len is set, the sequence id matches, and the valid byte appears then the firmware completed the command and only the notification was lost. Fall through to the normal error_code handling in that case. Log the response state on both paths so there is more data when this rare event occurs. Fixes: 74608fc98d28 ("bnxt_en: Ring free response from close path should us= e completion ring") Signed-off-by: Joe Damato --- .../net/ethernet/broadcom/bnxt/bnxt_hwrm.c | 75 ++++++++++++++----- 1 file changed, 57 insertions(+), 18 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_hwrm.c b/drivers/net/e= thernet/broadcom/bnxt/bnxt_hwrm.c index 5bfabdca7d0e..c494abb71c51 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_hwrm.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_hwrm.c @@ -456,6 +456,30 @@ static bool hwrm_wait_must_abort(struct bnxt *bp, u32 = req_type, u32 *fw_status) return *fw_status && !BNXT_FW_IS_HEALTHY(*fw_status); } =20 +/* Wait for the firmware to set the valid byte at the end of the response. + * Returns the number of usec spent waiting; a return of + * HWRM_VALID_BIT_DELAY_USEC or more means the byte never appeared. + */ +static int hwrm_wait_for_valid(u8 *valid) +{ + int j; + + for (j =3D 0; j < HWRM_VALID_BIT_DELAY_USEC; ) { + /* make sure we read from updated DMA memory */ + dma_rmb(); + if (*valid) + break; + if (j < 10) { + udelay(1); + j++; + } else { + usleep_range(20, 30); + j +=3D 20; + } + } + return j; +} + static int __hwrm_send(struct bnxt *bp, struct bnxt_hwrm_ctx *ctx) { u32 doorbell_offset =3D BNXT_GRCPF_REG_CHIMP_COMM_TRIGGER; @@ -582,12 +606,39 @@ static int __hwrm_send(struct bnxt *bp, struct bnxt_h= wrm_ctx *ctx) } =20 if (READ_ONCE(token->state) !=3D BNXT_HWRM_COMPLETE) { - hwrm_err(bp, ctx, "Resp cmpl intr err msg: 0x%x\n", - req_type); - goto exit; + bool completed =3D false; + u8 valid_byte =3D 0; + + /* The completion ring entry was not delivered for + * some reason. It might be possible that the command + * was carried out even without a completion being + * posted. Check the response before giving up and log + * the state. + */ + len =3D le16_to_cpu(READ_ONCE(ctx->resp->resp_len)); + if (len && + READ_ONCE(ctx->resp->seq_id) =3D=3D ctx->req->seq_id) { + valid =3D (u8 *)ctx->resp + len - 1; + completed =3D hwrm_wait_for_valid(valid) < + HWRM_VALID_BIT_DELAY_USEC; + valid_byte =3D *valid; + } + if (!completed) { + hwrm_err(bp, ctx, + "Resp cmpl intr err msg: 0x%x len:%d valid:0x%x seq:0x%x/0x%x\n", + req_type, len, valid_byte, + le16_to_cpu(READ_ONCE(ctx->resp->seq_id)), + le16_to_cpu(ctx->req->seq_id)); + goto exit; + } + netdev_warn(bp->dev, + "Resp cmpl intr not delivered, msg: 0x%x completed anyway (len:%d = valid:0x%x err:0x%x)\n", + req_type, len, valid_byte, + le16_to_cpu(ctx->resp->error_code)); + } else { + len =3D le16_to_cpu(READ_ONCE(ctx->resp->resp_len)); + valid =3D ((u8 *)ctx->resp) + len - 1; } - len =3D le16_to_cpu(READ_ONCE(ctx->resp->resp_len)); - valid =3D ((u8 *)ctx->resp) + len - 1; } else { __le16 seen_out_of_seq =3D ctx->req->seq_id; /* will never see */ int j; @@ -647,19 +698,7 @@ static int __hwrm_send(struct bnxt *bp, struct bnxt_hw= rm_ctx *ctx) =20 /* Last byte of resp contains valid bit */ valid =3D ((u8 *)ctx->resp) + len - 1; - for (j =3D 0; j < HWRM_VALID_BIT_DELAY_USEC; ) { - /* make sure we read from updated DMA memory */ - dma_rmb(); - if (*valid) - break; - if (j < 10) { - udelay(1); - j++; - } else { - usleep_range(20, 30); - j +=3D 20; - } - } + j =3D hwrm_wait_for_valid(valid); =20 if (j >=3D HWRM_VALID_BIT_DELAY_USEC) { hwrm_err(bp, ctx, "Error (timeout: %u) msg {0x%x 0x%x} len:%d v:%d\n", --=20 2.53.0-Meta From nobody Thu Sep 24 15:11:03 2026 Received: from mail-pz2-f42.google.com (mail-pz2-f42.google.com [74.125.228.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6A45048F000 for ; Tue, 22 Sep 2026 18:24:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=74.125.228.42 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790101466; cv=none; b=SwPbLytrGGlKX69gmuMzbQ8mvEaS8b6vw2qjevrj0pjwOy6kt2mQBhb4Eafc+lnHnjrp1d5cAQf+EMUM+32zi0fLg9VXChRl70U6DZwXly5bXqKE0QuSgHBAKKS+9GSxQaGffzPkdWD6jpQyUbPR4nEYmjmJ50psyplzZw20b60= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1790101466; c=relaxed/simple; bh=0U/i3+N7RWpmWdB2K1SYwzNCBGMxTC5Yl1aUMawg65s=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=giZvNIJQ5IWpHd5CaJ53uD7cl6n3ILztl+VlTBT3Gh+VoSTZle8b7Z2Zb4puM2xk2EVUMcP2lOBihZhsqbrCC4vmBCyJj37QP8CmGzFE9DCc24mxNQ8ku7qYqe8We0/ZYTtBK2WlkgGyq73DIHfTFwrrGgah9vsfGRxt4TsQSn0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=dama.to; spf=none smtp.mailfrom=dama.to; dkim=pass (2048-bit key) header.d=dama-to.20251104.gappssmtp.com header.i=@dama-to.20251104.gappssmtp.com header.b=UlrURzwG; arc=none smtp.client-ip=74.125.228.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=dama.to Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=dama.to Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=dama-to.20251104.gappssmtp.com header.i=@dama-to.20251104.gappssmtp.com header.b="UlrURzwG" Received: by mail-pz2-f42.google.com with SMTP id 41be03b00d2f7-cc1cea50dc3so200561a12.3 for ; Tue, 22 Sep 2026 11:24:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dama-to.20251104.gappssmtp.com; s=20251104; t=1790101464; x=1790706264; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to:content-type; bh=ak//GM/1EUqa8AvscB+gM/3Fh5QvD3sB1gF17iE3yxo=; b=UlrURzwG8R4Ue7r+jYLcgun72YCjcev4xA+SVqqb6s6GcCTbFpB/BEhl2bngcNQQL+ yvzfaRqNHQ5HSO3dlRwo3K/1G6pA53QY7gn5+iavm3c4m0FUdeShVVl0PJ8JwrUDxTBI l6J6fadvm6eR4EjzVR+PUHZ3H4CCKDdtTlVWxvmn4rkwemO6oI8Zylh5mCd3yWDIMHVt 24u+WdSD+5PhlzlFpZtK0sit6ETjy2zZGUuGnmUwR2jNUtQX5Zt7ukYClANQz/5OQufb PGFRKPnsCxHsYU3K1dQiUOTkCl8+11js7TBnUqDMqS1wLmhc43zUL3tkuDyKHlqMQo8i 7L/g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20260707; t=1790101464; x=1790706264; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-gg:x-gm-message-state:from :to:cc:subject:date:message-id:reply-to:content-type; bh=ak//GM/1EUqa8AvscB+gM/3Fh5QvD3sB1gF17iE3yxo=; b=B99GGbzVNGbjEprkMJt5ShTsRQC6te4LJk8Y/y3H1uuV7G3UhLw0r+Mvjxp9g0Lc2l +UT+B4XD3p8nsFHa1z5zgkf53ROnVJfm8ELxga8b7imOelH5NMZd3QWFPD+NJ6fz/Sy7 erUYg3i3YwBZKGdPZ/s1BbLc0q+ZKjGtizSG1s1Cfn7/Dghr7Du8zgi2xJFSXxAz2h4l 3lFtMmYGZQ+M6ycHWDQFJfZo8ePgKSaaoXUR2docPKOV+BVWYULAdRT1TJY+sGiqB8li hrJLEKvvuOYqLtwtD4mkMWFl5FxMNzniXqty6PJOhuHEZiwbhStCPkOXT9iRo8ddIR3X Totw== X-Forwarded-Encrypted: i=1; AKwUvBzFvH1ymqI+B0xGCMcJGTr7++Qjm0liw7CePuOBDwqku28h+fz6DEASK7pB5eu8pP5sZW3xdOevITV94iA=@vger.kernel.org X-Gm-Message-State: AFuF++myX8aTEk77fsTYLQf4IjGD5dKjJp7BFfxoE7icudmmODxij5XC aYBVKcIe1ggI/mbPCnJw8dVmm3+GQpLAHayW8ByEE5oHM+OmQXKyVbdLb58VYia6IVA= X-Gm-Gg: AYBFou1mxM734aAGD33iboIP2Jznl8YOn/GBrOYjlhp5p1NPpwv5RVmt7IDfGFdlNM1 +NbufcbCSvoU3GZTnDhCY1jYXnYpZ1g6fmBrvWQN6QV9IyGz64Lk0gFm7YXyF4MS2ONOaEm7Iad R1F0WHJOdB6nwRBsG1TsRl9kma8+Tusht9UGAwr2xtrwj4WSigg+PfMsfq0EvtmMXwK3y//Dh4E 3Kns5d0V6JYVs1QEId0KED4TZnjJ93wWN32g3icD0cnc3obcGyNAS8MV4fQXNA1zUpqe4LQkD55 tOiOXQJk3VBWsRLVT9Rav9odSfBWuezzXRyMh0PEg7kDg8d397kivfrddd5HYSefafj2PjgdO5f eXWUmCMmefXk9xZYL6m38LSm49SQg8SDhQMZMxo5YYb3b9dv9scbtVF59zcoqKqihF87xAsgnqK txiqNWAv8XjrJCXcjdsBRdJgyxcB86xcJKuivfkHFByACbqsAjzqHd X-Received: by 2002:a05:6a21:9082:b0:3dd:a195:dd60 with SMTP id adf61e73a8af0-3ddf82e5f1fmr256497637.66.1790101463748; Tue, 22 Sep 2026 11:24:23 -0700 (PDT) Received: from localhost ([2a03:2880:2ff:55::]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-cc75f3ba8b6sm12098a12.5.2026.09.22.11.24.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 22 Sep 2026 11:24:23 -0700 (PDT) From: Joe Damato To: netdev@vger.kernel.org, Michael Chan , Pavan Chebbi , Andrew Lunn , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Prashant Sreedharan Cc: horms@kernel.org, linux-kernel@vger.kernel.org, Joe Damato Subject: [RFC net v2 3/3] bnxt_en: stop DMA before releasing rings the firmware did not free Date: Tue, 22 Sep 2026 11:24:03 -0700 Message-ID: <20260922182405.1290749-4-joe@dama.to> X-Mailer: git-send-email 2.53.0 In-Reply-To: <20260922182405.1290749-1-joe@dama.to> References: <20260922182405.1290749-1-joe@dama.to> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" When HWRM_RING_FREE is not answered, bnxt_hwrm_ring_free() clears fw_ring_id and __bnxt_close_nic() goes on to call bnxt_free_mem(), which unmaps the ring memory and the RX buffers that the FW may still be using. This is reachable in production. On a BCM57504 the first sign is the TX watchdog; the close that follows times out a subset of its RING_FREEs and the driver releases those rings anyway: 05:30:12 NETDEV WATCHDOG: transmit queue 0 timed out 6073 ms 05:30:12 Resp cmpl intr err msg: 0x51 x20 05:30:12 hwrm_ring_free type 1 failed x12 05:30:12 hwrm_ring_free type 2 failed x8 05:30:12 AMD-Vi: IO_PAGE_FAULT x3 Count the rings the firmware did not free and, if there are any, disable the device before the close path releases memory. The device stays unusable until a firmware reset. Fixes: 74608fc98d28 ("bnxt_en: Ring free response from close path should us= e completion ring") Signed-off-by: Joe Damato --- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 21 ++++++++++++++++++--- 1 file changed, 18 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethern= et/broadcom/bnxt/bnxt.c index a7f6facca7b4..574532d7047b 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -7756,6 +7756,7 @@ static void bnxt_clear_one_cp_ring(struct bnxt *bp, s= truct bnxt_cp_ring_info *cp =20 static void bnxt_hwrm_ring_free(struct bnxt *bp, bool close_path) { + int stuck =3D 0; u32 type; int i; =20 @@ -7763,12 +7764,15 @@ static void bnxt_hwrm_ring_free(struct bnxt *bp, bo= ol close_path) return; =20 for (i =3D 0; i < bp->tx_nr_rings; i++) - bnxt_hwrm_tx_ring_free(bp, &bp->tx_ring[i], close_path); + if (bnxt_hwrm_tx_ring_free(bp, &bp->tx_ring[i], close_path)) + stuck++; =20 bnxt_cancel_dim(bp); for (i =3D 0; i < bp->rx_nr_rings; i++) { - bnxt_hwrm_rx_ring_free(bp, &bp->rx_ring[i], close_path); - bnxt_hwrm_rx_agg_ring_free(bp, &bp->rx_ring[i], close_path); + if (bnxt_hwrm_rx_ring_free(bp, &bp->rx_ring[i], close_path)) + stuck++; + if (bnxt_hwrm_rx_agg_ring_free(bp, &bp->rx_ring[i], close_path)) + stuck++; } =20 /* The completion rings are about to be freed. After that the @@ -7798,6 +7802,17 @@ static void bnxt_hwrm_ring_free(struct bnxt *bp, boo= l close_path) bp->grp_info[i].cp_fw_ring_id =3D INVALID_HW_RING_ID; } } + + if (!stuck) + return; + + /* FW never acknowledged freeing these rings, so it may still be + * DMAing to them. Stop the device before handing memory back. + */ + netdev_err(bp->dev, + "Firmware did not free %d ring(s); disabling DMA before releasing rin= g memory. A firmware reset is required.\n", + stuck); + pci_disable_device(bp->pdev); } =20 static int __bnxt_trim_rings(struct bnxt *bp, int *rx, int *tx, int max, --=20 2.53.0-Meta