Add a new selftest - ethtool_std_stats.sh - which validates the
eth-ctrl, eth-mac and pause standard statistics exported by an
interface. Not all counters can be validated in software, especially
those that are keeping track of errors. Counters such as
SingleCollisionFrames, FrameCheckSequenceErrors etc are not tested nor
included in this new selftest.

The central part of this patch is the traffic_test() function which
gathers the 'before' counter values, sends a batch of traffic and then
interrogates again the same counters in order to determine if the delta
is on target. The function receives an array through which the caller
can request what counters to be interrogated and, for each of them, what
is their target delta value.

The output from this selftest looks as follows on a LX2160ARDB board:

$ ./ethtool_std_stats.sh eth0 eth1
TEST: eth-ctrl tx on eth0 (MACControlFramesTransmitted on eth0)     [ OK ]
TEST: eth-ctrl tx on eth0 (MACControlFramesReceived on eth1)        [ OK ]
TEST: eth-ctrl tx on eth1 (MACControlFramesTransmitted on eth1)     [ OK ]
TEST: eth-ctrl tx on eth1 (MACControlFramesReceived on eth0)        [ OK ]
TEST: eth-mac bcast tx on eth0 (BroadcastFramesXmittedOK on eth0)   [ OK ]
TEST: eth-mac bcast tx on eth0 (FramesTransmittedOK on eth0)        [ OK ]
TEST: eth-mac bcast tx on eth0 (OctetsTransmittedOK on eth0)        [ OK ]
TEST: eth-mac bcast tx on eth0 (MulticastFramesXmittedOK on eth0)   [ OK ]
TEST: eth-mac bcast tx on eth0 (BroadcastFramesReceivedOK on eth1)  [ OK ]
TEST: eth-mac bcast tx on eth0 (FramesReceivedOK on eth1)           [ OK ]
TEST: eth-mac bcast tx on eth0 (OctetsReceivedOK on eth1)           [ OK ]
TEST: eth-mac bcast tx on eth0 (MulticastFramesReceivedOK on eth1)  [ OK ]
TEST: eth-mac mcast tx on eth0 (BroadcastFramesXmittedOK on eth0)   [ OK ]
TEST: eth-mac mcast tx on eth0 (FramesTransmittedOK on eth0)        [ OK ]
TEST: eth-mac mcast tx on eth0 (OctetsTransmittedOK on eth0)        [ OK ]
TEST: eth-mac mcast tx on eth0 (MulticastFramesXmittedOK on eth0)   [ OK ]
TEST: eth-mac mcast tx on eth0 (BroadcastFramesReceivedOK on eth1)  [ OK ]
TEST: eth-mac mcast tx on eth0 (FramesReceivedOK on eth1)           [ OK ]
TEST: eth-mac mcast tx on eth0 (OctetsReceivedOK on eth1)           [ OK ]
TEST: eth-mac mcast tx on eth0 (MulticastFramesReceivedOK on eth1)  [ OK ]
TEST: eth-mac ucast tx on eth0 (BroadcastFramesXmittedOK on eth0)   [ OK ]
TEST: eth-mac ucast tx on eth0 (FramesTransmittedOK on eth0)        [ OK ]
TEST: eth-mac ucast tx on eth0 (OctetsTransmittedOK on eth0)        [ OK ]
TEST: eth-mac ucast tx on eth0 (MulticastFramesXmittedOK on eth0)   [ OK ]
TEST: eth-mac ucast tx on eth0 (BroadcastFramesReceivedOK on eth1)  [ OK ]
TEST: eth-mac ucast tx on eth0 (FramesReceivedOK on eth1)           [ OK ]
TEST: eth-mac ucast tx on eth0 (OctetsReceivedOK on eth1)           [ OK ]
TEST: eth-mac ucast tx on eth0 (MulticastFramesReceivedOK on eth1)  [ OK ]
TEST: pause tx on eth0 (tx_pause_frames on eth0)                    [ OK ]
TEST: pause tx on eth0 (rx_pause_frames on eth1)                    [ OK ]
TEST: pause tx on eth1 (tx_pause_frames on eth1)                    [ OK ]
TEST: pause tx on eth1 (rx_pause_frames on eth0)                    [ OK ]

Please note that not all MACs are counting the software injected pause
frames as real Tx pause. For example, on a LS1028ARDB the selftest
output will reflect the fact that neither the ENETC MAC, nor the Felix
switch MAC are able to detect Tx pause frames injected by software.

$ ./ethtool_std_stats.sh eno0 swp0
(...)
TEST: Not all MACs detect injected pause frames                     [XFAIL]
TEST: pause tx on eno0 (rx_pause_frames on swp0)                    [ OK ]
TEST: Not all MACs detect injected pause frames                     [XFAIL]
TEST: pause tx on swp0 (rx_pause_frames on eno0)                    [ OK ]

Signed-off-by: Ioana Ciornei <ioana.ciornei@nxp.com>
---
 .../testing/selftests/drivers/net/hw/Makefile |   1 +
 .../drivers/net/hw/ethtool_std_stats.sh       | 192 ++++++++++++++++++
 2 files changed, 193 insertions(+)
 create mode 100755 tools/testing/selftests/drivers/net/hw/ethtool_std_stats.sh

diff --git a/tools/testing/selftests/drivers/net/hw/Makefile b/tools/testing/selftests/drivers/net/hw/Makefile
index a64140333a46..d447813a14b2 100644
--- a/tools/testing/selftests/drivers/net/hw/Makefile
+++ b/tools/testing/selftests/drivers/net/hw/Makefile
@@ -26,6 +26,7 @@ TEST_PROGS = \
 	ethtool_extended_state.sh \
 	ethtool_mm.sh \
 	ethtool_rmon.sh \
+	ethtool_std_stats.sh \
 	hw_stats_l3.sh \
 	hw_stats_l3_gre.sh \
 	iou-zcrx.py \
diff --git a/tools/testing/selftests/drivers/net/hw/ethtool_std_stats.sh b/tools/testing/selftests/drivers/net/hw/ethtool_std_stats.sh
new file mode 100755
index 000000000000..4ce8f140a18c
--- /dev/null
+++ b/tools/testing/selftests/drivers/net/hw/ethtool_std_stats.sh
@@ -0,0 +1,192 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+#shellcheck disable=SC2034 # SC does not see the global variables
+
+ALL_TESTS="
+	test_eth_ctrl_stats
+	test_eth_mac_stats
+	test_pause_stats
+"
+STABLE_MAC_ADDRS=yes
+NUM_NETIFS=2
+lib_dir=$(dirname "$0")
+# shellcheck source=./../../../net/forwarding/lib.sh
+source "$lib_dir"/../../../net/forwarding/lib.sh
+
+traffic_test()
+{
+	local iface=$1; shift
+	local neigh=$1; shift
+	local num_tx=$1; shift
+	local pkt_format="$1"; shift
+	local title="$1"; shift
+	declare -a counters=( "${@:2:$1}" ); shift "$(( $1 + 1 ))"
+	local before after delta target_high extra
+	local int grp counter target unit
+	local num_rx=$((num_tx * 2))
+	local xfail_message
+	local src="aggregate"
+
+	for i in "${!counters[@]}"; do
+		read -r int grp counter target unit xfail_message <<< "${counters[$i]}"
+		before[i]=$(ethtool_std_stats_get "$int" "$grp" "$counter" "$src")
+	done
+
+	# shellcheck disable=SC2086 # needs split options
+	$MZ "$iface" -q -c "$num_tx" $pkt_format
+
+	# shellcheck disable=SC2086 # needs split options
+	$MZ "$neigh" -q -c "$num_rx" $pkt_format
+
+	for i in "${!counters[@]}"; do
+		read -r int grp counter target unit xfail_message<<< "${counters[$i]}"
+
+		after[i]=$(ethtool_std_stats_get "$int" "$grp" "$counter" "$src")
+		if [[ "${after[$i]}" == "null" ]]; then
+			log_test_skip "$int does not support $grp-$counter"
+			continue;
+		fi
+
+		delta=$((after[i] - before[i]))
+
+		# Allow an extra 1% tolerance for random packets sent by the stack
+		extra=$((num_pkts * unit / 100))
+		target_high=$((target + extra))
+
+		RET=0
+		[ "$delta" -ge "$target" ] && [ "$delta" -le "$target_high" ]
+		err="$?"
+		if [[ $err != 0  ]] && [[ -n $xfail_message ]]; then
+			log_test_xfail "$xfail_message"
+			continue;
+		fi
+		check_err "$err" "$grp-$counter is not valid on $int (expected $target, got $delta)"
+		log_test "$title" "$counter on $int"
+	done
+}
+
+test_eth_ctrl_stats()
+{
+	local pkt_format="-a own -b bcast 88:08 -p 64"
+	local num_pkts=1000
+	local counters
+
+	counters=("$h1 eth-ctrl MACControlFramesTransmitted $num_pkts 1"
+		  "$h2 eth-ctrl MACControlFramesReceived $num_pkts 1")
+	traffic_test "$h1" "$h2" "$num_pkts" "$pkt_format" \
+		"eth-ctrl tx on $h1" \
+		"${#counters[@]}" "${counters[@]}"
+
+	counters=("$h2 eth-ctrl MACControlFramesTransmitted $num_pkts 1"
+		  "$h1 eth-ctrl MACControlFramesReceived $num_pkts 1")
+	traffic_test "$h2" "$h1" "$num_pkts" "$pkt_format" \
+		"eth-ctrl tx on $h2" \
+		"${#counters[@]}" "${counters[@]}"
+}
+
+test_eth_mac_stats()
+{
+	local pkt_size=100
+	local pkt_size_fcs=$((pkt_size + 4))
+	local bcast_pkt_format="-a own -b bcast -p $pkt_size"
+	local mcast_pkt_format="-a own -b -b 01:00:5E:00:00:01 -p $pkt_size"
+	local ucast_pkt_format="-a own -b $h2_mac -p $pkt_size"
+	local num_pkts=2000
+	local octets=$((pkt_size_fcs * num_pkts))
+	local counters
+
+	counters=("$h1 eth-mac BroadcastFramesXmittedOK $num_pkts 1"
+		  "$h1 eth-mac FramesTransmittedOK $num_pkts 1"
+		  "$h1 eth-mac OctetsTransmittedOK $octets $pkt_size_fcs"
+		  "$h1 eth-mac MulticastFramesXmittedOK 0 1"
+		  "$h2 eth-mac BroadcastFramesReceivedOK $num_pkts 1"
+		  "$h2 eth-mac FramesReceivedOK $num_pkts 1"
+		  "$h2 eth-mac OctetsReceivedOK $octets $pkt_size_fcs"
+		  "$h2 eth-mac MulticastFramesReceivedOK 0 1")
+	traffic_test "$h1" "$h2" "$num_pkts" "$bcast_pkt_format" \
+		"eth-mac bcast tx on $h1" \
+		"${#counters[@]}" "${counters[@]}"
+
+	counters=("$h1 eth-mac BroadcastFramesXmittedOK 0 1"
+		  "$h1 eth-mac FramesTransmittedOK $num_pkts 1"
+		  "$h1 eth-mac OctetsTransmittedOK $octets $pkt_size_fcs"
+		  "$h1 eth-mac MulticastFramesXmittedOK $num_pkts 1"
+		  "$h2 eth-mac BroadcastFramesReceivedOK 0 1"
+		  "$h2 eth-mac FramesReceivedOK $num_pkts 1"
+		  "$h2 eth-mac OctetsReceivedOK $octets $pkt_size_fcs"
+		  "$h2 eth-mac MulticastFramesReceivedOK $num_pkts 1")
+	traffic_test "$h1" "$h2" "$num_pkts" "$mcast_pkt_format" \
+		"eth-mac mcast tx on $h1" \
+		"${#counters[@]}" "${counters[@]}"
+
+	counters=("$h1 eth-mac BroadcastFramesXmittedOK 0 1"
+		  "$h1 eth-mac FramesTransmittedOK $num_pkts 1"
+		  "$h1 eth-mac OctetsTransmittedOK $octets $pkt_size_fcs"
+		  "$h1 eth-mac MulticastFramesXmittedOK 0 1"
+		  "$h2 eth-mac BroadcastFramesReceivedOK 0 1"
+		  "$h2 eth-mac FramesReceivedOK $num_pkts 1"
+		  "$h2 eth-mac OctetsReceivedOK $octets $pkt_size_fcs"
+		  "$h2 eth-mac MulticastFramesReceivedOK 0 1")
+	traffic_test "$h1" "$h2" "$num_pkts" "$ucast_pkt_format" \
+		"eth-mac ucast tx on $h1" \
+		"${#counters[@]}" "${counters[@]}"
+}
+
+test_pause_stats()
+{
+	local pkt_format="-a own -b 01:80:c2:00:00:01 88:08:00:01:00:01"
+	local xfail_message="Not all MACs detect injected pause frames"
+	local num_pkts=2000
+	local counters i
+
+	# Check that there is pause frame support
+	for ((i = 1; i <= NUM_NETIFS; ++i)); do
+		if ! ethtool -I --json -a "${NETIFS[p$i]}" > /dev/null 2>&1; then
+			log_test_skip "No support for pause frames, skip tests"
+			exit
+		fi
+	done
+
+	counters=("$h1 pause tx_pause_frames $num_pkts 1 $xfail_message"
+		  "$h2 pause rx_pause_frames $num_pkts 1")
+	traffic_test "$h1" "$h2" "$num_pkts" "$pkt_format" \
+		"pause tx on $h1" \
+		"${#counters[@]}" "${counters[@]}"
+
+	counters=("$h2 pause tx_pause_frames $num_pkts 1 $xfail_message"
+		  "$h1 pause rx_pause_frames $num_pkts 1")
+	traffic_test "$h2" "$h1" "$num_pkts" "$pkt_format" \
+		"pause tx on $h2" \
+		"${#counters[@]}" "${counters[@]}"
+}
+
+setup_prepare()
+{
+	h1=${NETIFS[p1]}
+	h2=${NETIFS[p2]}
+
+	h2_mac=$(mac_get "$h2")
+
+	for iface in $h1 $h2; do
+		ip link set dev "$iface" up
+	done
+}
+
+cleanup()
+{
+	pre_cleanup
+
+	for iface in $h2 $h1; do
+		ip link set dev "$iface" down
+	done
+}
+
+check_ethtool_counter_group_support
+trap cleanup EXIT
+
+setup_prepare
+setup_wait
+
+tests_run
+
+exit "$EXIT_STATUS"
-- 
2.25.1

Ioana Ciornei <ioana.ciornei@nxp.com> writes:

> Add a new selftest - ethtool_std_stats.sh - which validates the
> eth-ctrl, eth-mac and pause standard statistics exported by an
> interface. Not all counters can be validated in software, especially
> those that are keeping track of errors. Counters such as
> SingleCollisionFrames, FrameCheckSequenceErrors etc are not tested nor
> included in this new selftest.
>
> The central part of this patch is the traffic_test() function which
> gathers the 'before' counter values, sends a batch of traffic and then
> interrogates again the same counters in order to determine if the delta
> is on target. The function receives an array through which the caller
> can request what counters to be interrogated and, for each of them, what
> is their target delta value.
>
> The output from this selftest looks as follows on a LX2160ARDB board:
>
> $ ./ethtool_std_stats.sh eth0 eth1
> TEST: eth-ctrl tx on eth0 (MACControlFramesTransmitted on eth0)     [ OK ]
> TEST: eth-ctrl tx on eth0 (MACControlFramesReceived on eth1)        [ OK ]
> TEST: eth-ctrl tx on eth1 (MACControlFramesTransmitted on eth1)     [ OK ]
> TEST: eth-ctrl tx on eth1 (MACControlFramesReceived on eth0)        [ OK ]
> TEST: eth-mac bcast tx on eth0 (BroadcastFramesXmittedOK on eth0)   [ OK ]
> TEST: eth-mac bcast tx on eth0 (FramesTransmittedOK on eth0)        [ OK ]
> TEST: eth-mac bcast tx on eth0 (OctetsTransmittedOK on eth0)        [ OK ]
> TEST: eth-mac bcast tx on eth0 (MulticastFramesXmittedOK on eth0)   [ OK ]
> TEST: eth-mac bcast tx on eth0 (BroadcastFramesReceivedOK on eth1)  [ OK ]
> TEST: eth-mac bcast tx on eth0 (FramesReceivedOK on eth1)           [ OK ]
> TEST: eth-mac bcast tx on eth0 (OctetsReceivedOK on eth1)           [ OK ]
> TEST: eth-mac bcast tx on eth0 (MulticastFramesReceivedOK on eth1)  [ OK ]
> TEST: eth-mac mcast tx on eth0 (BroadcastFramesXmittedOK on eth0)   [ OK ]
> TEST: eth-mac mcast tx on eth0 (FramesTransmittedOK on eth0)        [ OK ]
> TEST: eth-mac mcast tx on eth0 (OctetsTransmittedOK on eth0)        [ OK ]
> TEST: eth-mac mcast tx on eth0 (MulticastFramesXmittedOK on eth0)   [ OK ]
> TEST: eth-mac mcast tx on eth0 (BroadcastFramesReceivedOK on eth1)  [ OK ]
> TEST: eth-mac mcast tx on eth0 (FramesReceivedOK on eth1)           [ OK ]
> TEST: eth-mac mcast tx on eth0 (OctetsReceivedOK on eth1)           [ OK ]
> TEST: eth-mac mcast tx on eth0 (MulticastFramesReceivedOK on eth1)  [ OK ]
> TEST: eth-mac ucast tx on eth0 (BroadcastFramesXmittedOK on eth0)   [ OK ]
> TEST: eth-mac ucast tx on eth0 (FramesTransmittedOK on eth0)        [ OK ]
> TEST: eth-mac ucast tx on eth0 (OctetsTransmittedOK on eth0)        [ OK ]
> TEST: eth-mac ucast tx on eth0 (MulticastFramesXmittedOK on eth0)   [ OK ]
> TEST: eth-mac ucast tx on eth0 (BroadcastFramesReceivedOK on eth1)  [ OK ]
> TEST: eth-mac ucast tx on eth0 (FramesReceivedOK on eth1)           [ OK ]
> TEST: eth-mac ucast tx on eth0 (OctetsReceivedOK on eth1)           [ OK ]
> TEST: eth-mac ucast tx on eth0 (MulticastFramesReceivedOK on eth1)  [ OK ]
> TEST: pause tx on eth0 (tx_pause_frames on eth0)                    [ OK ]
> TEST: pause tx on eth0 (rx_pause_frames on eth1)                    [ OK ]
> TEST: pause tx on eth1 (tx_pause_frames on eth1)                    [ OK ]
> TEST: pause tx on eth1 (rx_pause_frames on eth0)                    [ OK ]
>
> Please note that not all MACs are counting the software injected pause
> frames as real Tx pause. For example, on a LS1028ARDB the selftest
> output will reflect the fact that neither the ENETC MAC, nor the Felix
> switch MAC are able to detect Tx pause frames injected by software.
>
> $ ./ethtool_std_stats.sh eno0 swp0
> (...)
> TEST: Not all MACs detect injected pause frames                     [XFAIL]
> TEST: pause tx on eno0 (rx_pause_frames on swp0)                    [ OK ]
> TEST: Not all MACs detect injected pause frames                     [XFAIL]
> TEST: pause tx on swp0 (rx_pause_frames on eno0)                    [ OK ]
>
> Signed-off-by: Ioana Ciornei <ioana.ciornei@nxp.com>
> ---
>  .../testing/selftests/drivers/net/hw/Makefile |   1 +
>  .../drivers/net/hw/ethtool_std_stats.sh       | 192 ++++++++++++++++++
>  2 files changed, 193 insertions(+)
>  create mode 100755 tools/testing/selftests/drivers/net/hw/ethtool_std_stats.sh
>
> diff --git a/tools/testing/selftests/drivers/net/hw/Makefile b/tools/testing/selftests/drivers/net/hw/Makefile
> index a64140333a46..d447813a14b2 100644
> --- a/tools/testing/selftests/drivers/net/hw/Makefile
> +++ b/tools/testing/selftests/drivers/net/hw/Makefile
> @@ -26,6 +26,7 @@ TEST_PROGS = \
>  	ethtool_extended_state.sh \
>  	ethtool_mm.sh \
>  	ethtool_rmon.sh \
> +	ethtool_std_stats.sh \
>  	hw_stats_l3.sh \
>  	hw_stats_l3_gre.sh \
>  	iou-zcrx.py \
> diff --git a/tools/testing/selftests/drivers/net/hw/ethtool_std_stats.sh b/tools/testing/selftests/drivers/net/hw/ethtool_std_stats.sh
> new file mode 100755
> index 000000000000..4ce8f140a18c
> --- /dev/null
> +++ b/tools/testing/selftests/drivers/net/hw/ethtool_std_stats.sh
> @@ -0,0 +1,192 @@
> +#!/bin/bash
> +# SPDX-License-Identifier: GPL-2.0
> +#shellcheck disable=SC2034 # SC does not see the global variables
> +
> +ALL_TESTS="
> +	test_eth_ctrl_stats
> +	test_eth_mac_stats
> +	test_pause_stats
> +"
> +STABLE_MAC_ADDRS=yes
> +NUM_NETIFS=2
> +lib_dir=$(dirname "$0")
> +# shellcheck source=./../../../net/forwarding/lib.sh
> +source "$lib_dir"/../../../net/forwarding/lib.sh
> +
> +traffic_test()
> +{
> +	local iface=$1; shift
> +	local neigh=$1; shift
> +	local num_tx=$1; shift
> +	local pkt_format="$1"; shift
> +	local title="$1"; shift
> +	declare -a counters=( "${@:2:$1}" ); shift "$(( $1 + 1 ))"

Please make this local -a. declare inside a function also introduces a
local variable, so the effect is the same, but using local is more
obvious.

BTW, just a suggestion, personally I'd just make the counters a pure
"rest" argument:

	local -a counters=("$@")

simply because functions usually don't need more than one, and it's
easier to use on call site, and easier to understand what's going on.

> +	local before after delta target_high extra
> +	local int grp counter target unit
> +	local num_rx=$((num_tx * 2))
> +	local xfail_message
> +	local src="aggregate"

And local i.

> +
> +	for i in "${!counters[@]}"; do
> +		read -r int grp counter target unit xfail_message <<< "${counters[$i]}"
> +		before[i]=$(ethtool_std_stats_get "$int" "$grp" "$counter" "$src")
> +	done
> +
> +	# shellcheck disable=SC2086 # needs split options
> +	$MZ "$iface" -q -c "$num_tx" $pkt_format
> +
> +	# shellcheck disable=SC2086 # needs split options
> +	$MZ "$neigh" -q -c "$num_rx" $pkt_format
> +
> +	for i in "${!counters[@]}"; do

There should be a RET=0 here, so that the check_err below gets a clean slate.

> +		read -r int grp counter target unit xfail_message<<< "${counters[$i]}"
> +
> +		after[i]=$(ethtool_std_stats_get "$int" "$grp" "$counter" "$src")
> +		if [[ "${after[$i]}" == "null" ]]; then
> +			log_test_skip "$int does not support $grp-$counter"
> +			continue;
> +		fi
> +
> +		delta=$((after[i] - before[i]))
> +
> +		# Allow an extra 1% tolerance for random packets sent by the stack
> +		extra=$((num_pkts * unit / 100))
> +		target_high=$((target + extra))
> +
> +		RET=0
> +		[ "$delta" -ge "$target" ] && [ "$delta" -le "$target_high" ]
> +		err="$?"
> +		if [[ $err != 0  ]] && [[ -n $xfail_message ]]; then
> +			log_test_xfail "$xfail_message"

I think this should mention the $title as well. I think that the
xfail_message could be the log_test's opt_str, so:

                        log_test_xfail "$title" "$xfail_message"

A bit annoying that log_test_xfail clears the retmsg. It does so so that
messages from previous runs do not show up, which is desirable in
general, but here it would be handy.

> +			continue;
> +		fi
> +		check_err "$err" "$grp-$counter is not valid on $int (expected $target, got $delta)"
> +		log_test "$title" "$counter on $int"
> +	done
> +}
> +
> +test_eth_ctrl_stats()
> +{
> +	local pkt_format="-a own -b bcast 88:08 -p 64"
> +	local num_pkts=1000
> +	local counters

local -a

(Though yeah, bash doesn't care.)

> +
> +	counters=("$h1 eth-ctrl MACControlFramesTransmitted $num_pkts 1"
> +		  "$h2 eth-ctrl MACControlFramesReceived $num_pkts 1")
> +	traffic_test "$h1" "$h2" "$num_pkts" "$pkt_format" \
> +		"eth-ctrl tx on $h1" \
> +		"${#counters[@]}" "${counters[@]}"
> +
> +	counters=("$h2 eth-ctrl MACControlFramesTransmitted $num_pkts 1"
> +		  "$h1 eth-ctrl MACControlFramesReceived $num_pkts 1")
> +	traffic_test "$h2" "$h1" "$num_pkts" "$pkt_format" \
> +		"eth-ctrl tx on $h2" \
> +		"${#counters[@]}" "${counters[@]}"
> +}
> +
> +test_eth_mac_stats()
> +{
> +	local pkt_size=100
> +	local pkt_size_fcs=$((pkt_size + 4))
> +	local bcast_pkt_format="-a own -b bcast -p $pkt_size"
> +	local mcast_pkt_format="-a own -b -b 01:00:5E:00:00:01 -p $pkt_size"
> +	local ucast_pkt_format="-a own -b $h2_mac -p $pkt_size"
> +	local num_pkts=2000
> +	local octets=$((pkt_size_fcs * num_pkts))
> +	local counters

local -a

> +
> +	counters=("$h1 eth-mac BroadcastFramesXmittedOK $num_pkts 1"
> +		  "$h1 eth-mac FramesTransmittedOK $num_pkts 1"
> +		  "$h1 eth-mac OctetsTransmittedOK $octets $pkt_size_fcs"
> +		  "$h1 eth-mac MulticastFramesXmittedOK 0 1"
> +		  "$h2 eth-mac BroadcastFramesReceivedOK $num_pkts 1"
> +		  "$h2 eth-mac FramesReceivedOK $num_pkts 1"
> +		  "$h2 eth-mac OctetsReceivedOK $octets $pkt_size_fcs"
> +		  "$h2 eth-mac MulticastFramesReceivedOK 0 1")
> +	traffic_test "$h1" "$h2" "$num_pkts" "$bcast_pkt_format" \
> +		"eth-mac bcast tx on $h1" \
> +		"${#counters[@]}" "${counters[@]}"
> +
> +	counters=("$h1 eth-mac BroadcastFramesXmittedOK 0 1"
> +		  "$h1 eth-mac FramesTransmittedOK $num_pkts 1"
> +		  "$h1 eth-mac OctetsTransmittedOK $octets $pkt_size_fcs"
> +		  "$h1 eth-mac MulticastFramesXmittedOK $num_pkts 1"
> +		  "$h2 eth-mac BroadcastFramesReceivedOK 0 1"
> +		  "$h2 eth-mac FramesReceivedOK $num_pkts 1"
> +		  "$h2 eth-mac OctetsReceivedOK $octets $pkt_size_fcs"
> +		  "$h2 eth-mac MulticastFramesReceivedOK $num_pkts 1")
> +	traffic_test "$h1" "$h2" "$num_pkts" "$mcast_pkt_format" \
> +		"eth-mac mcast tx on $h1" \
> +		"${#counters[@]}" "${counters[@]}"
> +
> +	counters=("$h1 eth-mac BroadcastFramesXmittedOK 0 1"
> +		  "$h1 eth-mac FramesTransmittedOK $num_pkts 1"
> +		  "$h1 eth-mac OctetsTransmittedOK $octets $pkt_size_fcs"
> +		  "$h1 eth-mac MulticastFramesXmittedOK 0 1"
> +		  "$h2 eth-mac BroadcastFramesReceivedOK 0 1"
> +		  "$h2 eth-mac FramesReceivedOK $num_pkts 1"
> +		  "$h2 eth-mac OctetsReceivedOK $octets $pkt_size_fcs"
> +		  "$h2 eth-mac MulticastFramesReceivedOK 0 1")
> +	traffic_test "$h1" "$h2" "$num_pkts" "$ucast_pkt_format" \
> +		"eth-mac ucast tx on $h1" \
> +		"${#counters[@]}" "${counters[@]}"
> +}
> +
> +test_pause_stats()
> +{
> +	local pkt_format="-a own -b 01:80:c2:00:00:01 88:08:00:01:00:01"
> +	local xfail_message="Not all MACs detect injected pause frames"
> +	local num_pkts=2000
> +	local counters i

counters should be declared local -a

> +
> +	# Check that there is pause frame support
> +	for ((i = 1; i <= NUM_NETIFS; ++i)); do
> +		if ! ethtool -I --json -a "${NETIFS[p$i]}" > /dev/null 2>&1; then
> +			log_test_skip "No support for pause frames, skip tests"
> +			exit
> +		fi
> +	done
> +
> +	counters=("$h1 pause tx_pause_frames $num_pkts 1 $xfail_message"
> +		  "$h2 pause rx_pause_frames $num_pkts 1")
> +	traffic_test "$h1" "$h2" "$num_pkts" "$pkt_format" \
> +		"pause tx on $h1" \
> +		"${#counters[@]}" "${counters[@]}"
> +
> +	counters=("$h2 pause tx_pause_frames $num_pkts 1 $xfail_message"
> +		  "$h1 pause rx_pause_frames $num_pkts 1")
> +	traffic_test "$h2" "$h1" "$num_pkts" "$pkt_format" \
> +		"pause tx on $h2" \
> +		"${#counters[@]}" "${counters[@]}"
> +}
> +
> +setup_prepare()
> +{
> +	h1=${NETIFS[p1]}
> +	h2=${NETIFS[p2]}
> +
> +	h2_mac=$(mac_get "$h2")
> +
> +	for iface in $h1 $h2; do

These should be quoted.
iface should be local.

> +		ip link set dev "$iface" up

Plesae use defer:

		ip link set dev "$iface" up
		defer ip link set dev "$iface" down

Then drop the hand-rolled cleanup() altogether. Retain the "trap cleanup
EXIT" -- it will pick up forwarding/lib.sh's cleanup(), which calls
pre_cleanup and defer_scopes_cleanup().

> +	done
> +}
> +
> +cleanup()
> +{
> +	pre_cleanup
> +
> +	for iface in $h2 $h1; do
> +		ip link set dev "$iface" down
> +	done
> +}
> +
> +check_ethtool_counter_group_support
> +trap cleanup EXIT
> +
> +setup_prepare
> +setup_wait
> +
> +tests_run
> +
> +exit "$EXIT_STATUS"

On Fri, Feb 27, 2026 at 04:45:47PM +0100, Petr Machata wrote:
> 
> Ioana Ciornei <ioana.ciornei@nxp.com> writes:
> 
> > Add a new selftest - ethtool_std_stats.sh - which validates the
> > eth-ctrl, eth-mac and pause standard statistics exported by an
> > interface. Not all counters can be validated in software, especially
> > those that are keeping track of errors. Counters such as
> > SingleCollisionFrames, FrameCheckSequenceErrors etc are not tested nor
> > included in this new selftest.
> >
> > The central part of this patch is the traffic_test() function which
> > gathers the 'before' counter values, sends a batch of traffic and then
> > interrogates again the same counters in order to determine if the delta
> > is on target. The function receives an array through which the caller
> > can request what counters to be interrogated and, for each of them, what
> > is their target delta value.
> >
> > The output from this selftest looks as follows on a LX2160ARDB board:
> >
> > $ ./ethtool_std_stats.sh eth0 eth1
> > TEST: eth-ctrl tx on eth0 (MACControlFramesTransmitted on eth0)     [ OK ]
> > TEST: eth-ctrl tx on eth0 (MACControlFramesReceived on eth1)        [ OK ]
> > TEST: eth-ctrl tx on eth1 (MACControlFramesTransmitted on eth1)     [ OK ]
> > TEST: eth-ctrl tx on eth1 (MACControlFramesReceived on eth0)        [ OK ]
> > TEST: eth-mac bcast tx on eth0 (BroadcastFramesXmittedOK on eth0)   [ OK ]
> > TEST: eth-mac bcast tx on eth0 (FramesTransmittedOK on eth0)        [ OK ]
> > TEST: eth-mac bcast tx on eth0 (OctetsTransmittedOK on eth0)        [ OK ]
> > TEST: eth-mac bcast tx on eth0 (MulticastFramesXmittedOK on eth0)   [ OK ]
> > TEST: eth-mac bcast tx on eth0 (BroadcastFramesReceivedOK on eth1)  [ OK ]
> > TEST: eth-mac bcast tx on eth0 (FramesReceivedOK on eth1)           [ OK ]
> > TEST: eth-mac bcast tx on eth0 (OctetsReceivedOK on eth1)           [ OK ]
> > TEST: eth-mac bcast tx on eth0 (MulticastFramesReceivedOK on eth1)  [ OK ]
> > TEST: eth-mac mcast tx on eth0 (BroadcastFramesXmittedOK on eth0)   [ OK ]
> > TEST: eth-mac mcast tx on eth0 (FramesTransmittedOK on eth0)        [ OK ]
> > TEST: eth-mac mcast tx on eth0 (OctetsTransmittedOK on eth0)        [ OK ]
> > TEST: eth-mac mcast tx on eth0 (MulticastFramesXmittedOK on eth0)   [ OK ]
> > TEST: eth-mac mcast tx on eth0 (BroadcastFramesReceivedOK on eth1)  [ OK ]
> > TEST: eth-mac mcast tx on eth0 (FramesReceivedOK on eth1)           [ OK ]
> > TEST: eth-mac mcast tx on eth0 (OctetsReceivedOK on eth1)           [ OK ]
> > TEST: eth-mac mcast tx on eth0 (MulticastFramesReceivedOK on eth1)  [ OK ]
> > TEST: eth-mac ucast tx on eth0 (BroadcastFramesXmittedOK on eth0)   [ OK ]
> > TEST: eth-mac ucast tx on eth0 (FramesTransmittedOK on eth0)        [ OK ]
> > TEST: eth-mac ucast tx on eth0 (OctetsTransmittedOK on eth0)        [ OK ]
> > TEST: eth-mac ucast tx on eth0 (MulticastFramesXmittedOK on eth0)   [ OK ]
> > TEST: eth-mac ucast tx on eth0 (BroadcastFramesReceivedOK on eth1)  [ OK ]
> > TEST: eth-mac ucast tx on eth0 (FramesReceivedOK on eth1)           [ OK ]
> > TEST: eth-mac ucast tx on eth0 (OctetsReceivedOK on eth1)           [ OK ]
> > TEST: eth-mac ucast tx on eth0 (MulticastFramesReceivedOK on eth1)  [ OK ]
> > TEST: pause tx on eth0 (tx_pause_frames on eth0)                    [ OK ]
> > TEST: pause tx on eth0 (rx_pause_frames on eth1)                    [ OK ]
> > TEST: pause tx on eth1 (tx_pause_frames on eth1)                    [ OK ]
> > TEST: pause tx on eth1 (rx_pause_frames on eth0)                    [ OK ]
> >
> > Please note that not all MACs are counting the software injected pause
> > frames as real Tx pause. For example, on a LS1028ARDB the selftest
> > output will reflect the fact that neither the ENETC MAC, nor the Felix
> > switch MAC are able to detect Tx pause frames injected by software.
> >
> > $ ./ethtool_std_stats.sh eno0 swp0
> > (...)
> > TEST: Not all MACs detect injected pause frames                     [XFAIL]
> > TEST: pause tx on eno0 (rx_pause_frames on swp0)                    [ OK ]
> > TEST: Not all MACs detect injected pause frames                     [XFAIL]
> > TEST: pause tx on swp0 (rx_pause_frames on eno0)                    [ OK ]
> >
> > Signed-off-by: Ioana Ciornei <ioana.ciornei@nxp.com>
> > ---
> >  .../testing/selftests/drivers/net/hw/Makefile |   1 +
> >  .../drivers/net/hw/ethtool_std_stats.sh       | 192 ++++++++++++++++++
> >  2 files changed, 193 insertions(+)
> >  create mode 100755 tools/testing/selftests/drivers/net/hw/ethtool_std_stats.sh
> >
> > diff --git a/tools/testing/selftests/drivers/net/hw/Makefile b/tools/testing/selftests/drivers/net/hw/Makefile
> > index a64140333a46..d447813a14b2 100644
> > --- a/tools/testing/selftests/drivers/net/hw/Makefile
> > +++ b/tools/testing/selftests/drivers/net/hw/Makefile
> > @@ -26,6 +26,7 @@ TEST_PROGS = \
> >  	ethtool_extended_state.sh \
> >  	ethtool_mm.sh \
> >  	ethtool_rmon.sh \
> > +	ethtool_std_stats.sh \
> >  	hw_stats_l3.sh \
> >  	hw_stats_l3_gre.sh \
> >  	iou-zcrx.py \
> > diff --git a/tools/testing/selftests/drivers/net/hw/ethtool_std_stats.sh b/tools/testing/selftests/drivers/net/hw/ethtool_std_stats.sh
> > new file mode 100755
> > index 000000000000..4ce8f140a18c
> > --- /dev/null
> > +++ b/tools/testing/selftests/drivers/net/hw/ethtool_std_stats.sh
> > @@ -0,0 +1,192 @@
> > +#!/bin/bash
> > +# SPDX-License-Identifier: GPL-2.0
> > +#shellcheck disable=SC2034 # SC does not see the global variables
> > +
> > +ALL_TESTS="
> > +	test_eth_ctrl_stats
> > +	test_eth_mac_stats
> > +	test_pause_stats
> > +"
> > +STABLE_MAC_ADDRS=yes
> > +NUM_NETIFS=2
> > +lib_dir=$(dirname "$0")
> > +# shellcheck source=./../../../net/forwarding/lib.sh
> > +source "$lib_dir"/../../../net/forwarding/lib.sh
> > +
> > +traffic_test()
> > +{
> > +	local iface=$1; shift
> > +	local neigh=$1; shift
> > +	local num_tx=$1; shift
> > +	local pkt_format="$1"; shift
> > +	local title="$1"; shift
> > +	declare -a counters=( "${@:2:$1}" ); shift "$(( $1 + 1 ))"
> 
> Please make this local -a. declare inside a function also introduces a
> local variable, so the effect is the same, but using local is more
> obvious.

Ok.

> 
> BTW, just a suggestion, personally I'd just make the counters a pure
> "rest" argument:
> 
> 	local -a counters=("$@")
> 
> simply because functions usually don't need more than one, and it's
> easier to use on call site, and easier to understand what's going on.

Why I had this overly complicated retrieval of the array is because I
started out with multiple arrays and then restructured but forgot about
this. Will change to just retrieve the "rest".

> > +	local before after delta target_high extra
> > +	local int grp counter target unit
> > +	local num_rx=$((num_tx * 2))
> > +	local xfail_message
> > +	local src="aggregate"
> 
> And local i.

Sure.

> 
> > +
> > +	for i in "${!counters[@]}"; do
> > +		read -r int grp counter target unit xfail_message <<< "${counters[$i]}"
> > +		before[i]=$(ethtool_std_stats_get "$int" "$grp" "$counter" "$src")
> > +	done
> > +
> > +	# shellcheck disable=SC2086 # needs split options
> > +	$MZ "$iface" -q -c "$num_tx" $pkt_format
> > +
> > +	# shellcheck disable=SC2086 # needs split options
> > +	$MZ "$neigh" -q -c "$num_rx" $pkt_format
> > +
> > +	for i in "${!counters[@]}"; do
> 
> There should be a RET=0 here, so that the check_err below gets a clean slate.

You want me to move the RET=0 from some lines below to here, right?

> 
> > +		read -r int grp counter target unit xfail_message<<< "${counters[$i]}"
> > +
> > +		after[i]=$(ethtool_std_stats_get "$int" "$grp" "$counter" "$src")
> > +		if [[ "${after[$i]}" == "null" ]]; then
> > +			log_test_skip "$int does not support $grp-$counter"
> > +			continue;
> > +		fi
> > +
> > +		delta=$((after[i] - before[i]))
> > +
> > +		# Allow an extra 1% tolerance for random packets sent by the stack
> > +		extra=$((num_pkts * unit / 100))
> > +		target_high=$((target + extra))
> > +
> > +		RET=0
> > +		[ "$delta" -ge "$target" ] && [ "$delta" -le "$target_high" ]
> > +		err="$?"
> > +		if [[ $err != 0  ]] && [[ -n $xfail_message ]]; then
> > +			log_test_xfail "$xfail_message"
> 
> I think this should mention the $title as well. I think that the
> xfail_message could be the log_test's opt_str, so:
> 
>                         log_test_xfail "$title" "$xfail_message"

Good point.

> 
> A bit annoying that log_test_xfail clears the retmsg. It does so so that
> messages from previous runs do not show up, which is desirable in
> general, but here it would be handy.
> 
> > +			continue;
> > +		fi
> > +		check_err "$err" "$grp-$counter is not valid on $int (expected $target, got $delta)"
> > +		log_test "$title" "$counter on $int"
> > +	done
> > +}
> > +
> > +test_eth_ctrl_stats()
> > +{
> > +	local pkt_format="-a own -b bcast 88:08 -p 64"
> > +	local num_pkts=1000
> > +	local counters
> 
> local -a
> 
> (Though yeah, bash doesn't care.)

Ok.

> 
> > +
> > +	counters=("$h1 eth-ctrl MACControlFramesTransmitted $num_pkts 1"
> > +		  "$h2 eth-ctrl MACControlFramesReceived $num_pkts 1")
> > +	traffic_test "$h1" "$h2" "$num_pkts" "$pkt_format" \
> > +		"eth-ctrl tx on $h1" \
> > +		"${#counters[@]}" "${counters[@]}"
> > +
> > +	counters=("$h2 eth-ctrl MACControlFramesTransmitted $num_pkts 1"
> > +		  "$h1 eth-ctrl MACControlFramesReceived $num_pkts 1")
> > +	traffic_test "$h2" "$h1" "$num_pkts" "$pkt_format" \
> > +		"eth-ctrl tx on $h2" \
> > +		"${#counters[@]}" "${counters[@]}"
> > +}
> > +
> > +test_eth_mac_stats()
> > +{
> > +	local pkt_size=100
> > +	local pkt_size_fcs=$((pkt_size + 4))
> > +	local bcast_pkt_format="-a own -b bcast -p $pkt_size"
> > +	local mcast_pkt_format="-a own -b -b 01:00:5E:00:00:01 -p $pkt_size"
> > +	local ucast_pkt_format="-a own -b $h2_mac -p $pkt_size"
> > +	local num_pkts=2000
> > +	local octets=$((pkt_size_fcs * num_pkts))
> > +	local counters
> 
> local -a

Ok.

> 
> > +
> > +	counters=("$h1 eth-mac BroadcastFramesXmittedOK $num_pkts 1"
> > +		  "$h1 eth-mac FramesTransmittedOK $num_pkts 1"
> > +		  "$h1 eth-mac OctetsTransmittedOK $octets $pkt_size_fcs"
> > +		  "$h1 eth-mac MulticastFramesXmittedOK 0 1"
> > +		  "$h2 eth-mac BroadcastFramesReceivedOK $num_pkts 1"
> > +		  "$h2 eth-mac FramesReceivedOK $num_pkts 1"
> > +		  "$h2 eth-mac OctetsReceivedOK $octets $pkt_size_fcs"
> > +		  "$h2 eth-mac MulticastFramesReceivedOK 0 1")
> > +	traffic_test "$h1" "$h2" "$num_pkts" "$bcast_pkt_format" \
> > +		"eth-mac bcast tx on $h1" \
> > +		"${#counters[@]}" "${counters[@]}"
> > +
> > +	counters=("$h1 eth-mac BroadcastFramesXmittedOK 0 1"
> > +		  "$h1 eth-mac FramesTransmittedOK $num_pkts 1"
> > +		  "$h1 eth-mac OctetsTransmittedOK $octets $pkt_size_fcs"
> > +		  "$h1 eth-mac MulticastFramesXmittedOK $num_pkts 1"
> > +		  "$h2 eth-mac BroadcastFramesReceivedOK 0 1"
> > +		  "$h2 eth-mac FramesReceivedOK $num_pkts 1"
> > +		  "$h2 eth-mac OctetsReceivedOK $octets $pkt_size_fcs"
> > +		  "$h2 eth-mac MulticastFramesReceivedOK $num_pkts 1")
> > +	traffic_test "$h1" "$h2" "$num_pkts" "$mcast_pkt_format" \
> > +		"eth-mac mcast tx on $h1" \
> > +		"${#counters[@]}" "${counters[@]}"
> > +
> > +	counters=("$h1 eth-mac BroadcastFramesXmittedOK 0 1"
> > +		  "$h1 eth-mac FramesTransmittedOK $num_pkts 1"
> > +		  "$h1 eth-mac OctetsTransmittedOK $octets $pkt_size_fcs"
> > +		  "$h1 eth-mac MulticastFramesXmittedOK 0 1"
> > +		  "$h2 eth-mac BroadcastFramesReceivedOK 0 1"
> > +		  "$h2 eth-mac FramesReceivedOK $num_pkts 1"
> > +		  "$h2 eth-mac OctetsReceivedOK $octets $pkt_size_fcs"
> > +		  "$h2 eth-mac MulticastFramesReceivedOK 0 1")
> > +	traffic_test "$h1" "$h2" "$num_pkts" "$ucast_pkt_format" \
> > +		"eth-mac ucast tx on $h1" \
> > +		"${#counters[@]}" "${counters[@]}"
> > +}
> > +
> > +test_pause_stats()
> > +{
> > +	local pkt_format="-a own -b 01:80:c2:00:00:01 88:08:00:01:00:01"
> > +	local xfail_message="Not all MACs detect injected pause frames"
> > +	local num_pkts=2000
> > +	local counters i
> 
> counters should be declared local -a

Ok.

> 
> > +
> > +	# Check that there is pause frame support
> > +	for ((i = 1; i <= NUM_NETIFS; ++i)); do
> > +		if ! ethtool -I --json -a "${NETIFS[p$i]}" > /dev/null 2>&1; then
> > +			log_test_skip "No support for pause frames, skip tests"
> > +			exit
> > +		fi
> > +	done
> > +
> > +	counters=("$h1 pause tx_pause_frames $num_pkts 1 $xfail_message"
> > +		  "$h2 pause rx_pause_frames $num_pkts 1")
> > +	traffic_test "$h1" "$h2" "$num_pkts" "$pkt_format" \
> > +		"pause tx on $h1" \
> > +		"${#counters[@]}" "${counters[@]}"
> > +
> > +	counters=("$h2 pause tx_pause_frames $num_pkts 1 $xfail_message"
> > +		  "$h1 pause rx_pause_frames $num_pkts 1")
> > +	traffic_test "$h2" "$h1" "$num_pkts" "$pkt_format" \
> > +		"pause tx on $h2" \
> > +		"${#counters[@]}" "${counters[@]}"
> > +}
> > +
> > +setup_prepare()
> > +{
> > +	h1=${NETIFS[p1]}
> > +	h2=${NETIFS[p2]}
> > +
> > +	h2_mac=$(mac_get "$h2")
> > +
> > +	for iface in $h1 $h2; do
> 
> These should be quoted.
> iface should be local.

Ok.

> > +		ip link set dev "$iface" up
> 
> Plesae use defer:
> 
> 		ip link set dev "$iface" up
> 		defer ip link set dev "$iface" down
> 
> Then drop the hand-rolled cleanup() altogether. Retain the "trap cleanup
> EXIT" -- it will pick up forwarding/lib.sh's cleanup(), which calls
> pre_cleanup and defer_scopes_cleanup().

Sure! Thanks.

Ioana Ciornei <ioana.ciornei@nxp.com> writes:

> On Fri, Feb 27, 2026 at 04:45:47PM +0100, Petr Machata wrote:
>> 
>> Ioana Ciornei <ioana.ciornei@nxp.com> writes:
>> 
>> > +	for i in "${!counters[@]}"; do
>> 
>> There should be a RET=0 here, so that the check_err below gets a clean slate.
>
> You want me to move the RET=0 from some lines below to here, right?

My bad, I missed that there is a RET=0 below. Keep it as it is.

On Wed, 25 Feb 2026 17:06:48 +0200 Ioana Ciornei wrote:
> +ALL_TESTS="
> +	test_eth_ctrl_stats
> +	test_eth_mac_stats
> +	test_pause_stats
> +"
> +STABLE_MAC_ADDRS=yes
> +NUM_NETIFS=2
> +lib_dir=$(dirname "$0")
> +# shellcheck source=./../../../net/forwarding/lib.sh
> +source "$lib_dir"/../../../net/forwarding/lib.sh

Argh, at some point we should probably decide whether we have 
a preference on which "library" / set of env vars we use under 
drivers/net/hw. Adding Petr to CC.

The existing tests under drivers/net/hw which source forwarding/lib.sh
pre-date the "NIC setup" described in 
tools/testing/selftests/drivers/net/README.rst

Should we ask "NIC setup" to be used for all tests which only need
NUM_NETIFS=2 ? These are basically simple tests where one netif is 
a traffic generator and the other is the DUT. And IIUC the "forwarding
setup" can't be used when the traffic generator is controlled over SSH?

Jakub Kicinski <kuba@kernel.org> writes:

> On Wed, 25 Feb 2026 17:06:48 +0200 Ioana Ciornei wrote:
>> +ALL_TESTS="
>> +	test_eth_ctrl_stats
>> +	test_eth_mac_stats
>> +	test_pause_stats
>> +"
>> +STABLE_MAC_ADDRS=yes
>> +NUM_NETIFS=2
>> +lib_dir=$(dirname "$0")
>> +# shellcheck source=./../../../net/forwarding/lib.sh
>> +source "$lib_dir"/../../../net/forwarding/lib.sh
>
> Argh, at some point we should probably decide whether we have 
> a preference on which "library" / set of env vars we use under 
> drivers/net/hw. Adding Petr to CC.

I think the expectation is that by default, tests written in Bash are
run on one machine without remotes.

I think this fundamentally stems from the fact that running processes in
Python is a bit unwieldy, so it makes sense to have helpers, so
everybody uses them, so you can have helpers grow brains to do things
like over-the-ssh configuration. In Bash, running a traffic generator is
easier than working with arrays. So the helpers tend not to be as useful
and we don't generally have them. At least not in any consistent way.

Eyeing the file, the requirement for the remote interface to be up and
configured with an IP address is a bit surprising to me. I would think a
down state is the most natural, and have the test bring it up and
configure it in a way that it needs. I'm thinking maybe this is to allow
testing a sole interface on like an embedded device?

Anyway, that's a fairly strong differency between how Bash tests are
typically written and the NIC setup. I think basically all existing
tests assume the devices are theirs to tamper with.

> The existing tests under drivers/net/hw which source forwarding/lib.sh
> pre-date the "NIC setup" described in 
> tools/testing/selftests/drivers/net/README.rst
>
> Should we ask "NIC setup" to be used for all tests which only need
> NUM_NETIFS=2 ? These are basically simple tests where one netif is 
> a traffic generator and the other is the DUT. And IIUC the "forwarding
> setup" can't be used when the traffic generator is controlled over SSH?

In principle nothing prevents lib.sh from growing brains to support
these remote shenanigans. I think it's just that so far nobody cared
enough to actually do it.

I think that a helper that in effect does "run this on a machine where
$swp1 is" is mostly what is needed. That and "make sure $swp1 and $swp2
are on the same machine". It's going to be annoying to work with though,
because you need to annotate every single command. I bet there's a nice
syntax to make it not activelly annoying.

If we have this, it might make sense to require tests to make use of it.
(With an explicit opt-out for special cases.) But I do not want every
test to have to reinvent this wheel and cargo-cult snippets from other
tests.

BTW, my guess is that even many multi-port tests that I wrote boil down
to just a bunch of fairly independent loopbacks whose far ends could be
on remote machines. It's not a priori nonsense to me that one would run
a test like this, or whatever magic we'd use:

    ./test.sh ssh://petr@10.1.2.3:eth1 swp1 veth1 ns://foo:veth2

And it just works, because only swp1 and swp2 need to be bridged, the
rest can be remote, and the traffic generation helper knows that to pump
traffic to ssh://10.1.2.3:eth1, obviously you need to ssh there first.
But the library would need to have helpers for this, and the tests would
need to use them.

At least ethtool counters would cause problems obviously.

On Fri, 27 Feb 2026 14:53:06 +0100 Petr Machata wrote:
> Jakub Kicinski <kuba@kernel.org> writes:
> > On Wed, 25 Feb 2026 17:06:48 +0200 Ioana Ciornei wrote:  
> >> +ALL_TESTS="
> >> +	test_eth_ctrl_stats
> >> +	test_eth_mac_stats
> >> +	test_pause_stats
> >> +"
> >> +STABLE_MAC_ADDRS=yes
> >> +NUM_NETIFS=2
> >> +lib_dir=$(dirname "$0")
> >> +# shellcheck source=./../../../net/forwarding/lib.sh
> >> +source "$lib_dir"/../../../net/forwarding/lib.sh  
> >
> > Argh, at some point we should probably decide whether we have 
> > a preference on which "library" / set of env vars we use under 
> > drivers/net/hw. Adding Petr to CC.  
> 
> I think the expectation is that by default, tests written in Bash are
> run on one machine without remotes.

I think we need to define the guidance by properties of the test, not
the machine? Of course _tests_ which don't need a traffic source can
simply consume the NETIF evn variable, so its trivial to write them in
any language without much library support. But for tests which need
traffic input we need a different distinction than "does the author
care about single-interface machines", so to speak?

> I think this fundamentally stems from the fact that running processes in
> Python is a bit unwieldy, so it makes sense to have helpers, so
> everybody uses them, so you can have helpers grow brains to do things
> like over-the-ssh configuration. In Bash, running a traffic generator is
> easier than working with arrays. So the helpers tend not to be as useful
> and we don't generally have them. At least not in any consistent way.
> 
> Eyeing the file, the requirement for the remote interface to be up and
> configured with an IP address is a bit surprising to me. I would think a
> down state is the most natural, and have the test bring it up and
> configure it in a way that it needs. I'm thinking maybe this is to allow
> testing a sole interface on like an embedded device?
> 
> Anyway, that's a fairly strong differency between how Bash tests are
> typically written and the NIC setup. I think basically all existing
> tests assume the devices are theirs to tamper with.
> 
> > The existing tests under drivers/net/hw which source forwarding/lib.sh
> > pre-date the "NIC setup" described in 
> > tools/testing/selftests/drivers/net/README.rst
> >
> > Should we ask "NIC setup" to be used for all tests which only need
> > NUM_NETIFS=2 ? These are basically simple tests where one netif is 
> > a traffic generator and the other is the DUT. And IIUC the "forwarding
> > setup" can't be used when the traffic generator is controlled over SSH?  
> 
> In principle nothing prevents lib.sh from growing brains to support
> these remote shenanigans. I think it's just that so far nobody cared
> enough to actually do it.
> 
> I think that a helper that in effect does "run this on a machine where
> $swp1 is" is mostly what is needed. That and "make sure $swp1 and $swp2
> are on the same machine". It's going to be annoying to work with though,
> because you need to annotate every single command. I bet there's a nice
> syntax to make it not activelly annoying.
> 
> If we have this, it might make sense to require tests to make use of it.
> (With an explicit opt-out for special cases.) But I do not want every
> test to have to reinvent this wheel and cargo-cult snippets from other
> tests.
> 
> BTW, my guess is that even many multi-port tests that I wrote boil down
> to just a bunch of fairly independent loopbacks whose far ends could be
> on remote machines. It's not a priori nonsense to me that one would run
> a test like this, or whatever magic we'd use:
> 
>     ./test.sh ssh://petr@10.1.2.3:eth1 swp1 veth1 ns://foo:veth2
> 
> And it just works, because only swp1 and swp2 need to be bridged, the
> rest can be remote, and the traffic generation helper knows that to pump
> traffic to ssh://10.1.2.3:eth1, obviously you need to ssh there first.
> But the library would need to have helpers for this, and the tests would
> need to use them.

Right, to be clear I primarily care about being able to run these tests
with traffic generator on a remote system. Otherwise someone who cares
about single-port systems will have to add a separate test I guess?
And we will have to maintain two tests? :S

A nice to have is for all drivers/net/hw tests to use the "NIC" env
variables (move tests which really only make sense on multi-port
devices to drivers/net/forwarding?) But I think "translating"
forwarding variables to NIC if NUMIF=2 could easily be done
automatically by the test / lib so that's lower priority.

> At least ethtool counters would cause problems obviously.

Jakub Kicinski <kuba@kernel.org> writes:

> On Fri, 27 Feb 2026 14:53:06 +0100 Petr Machata wrote:
>> Jakub Kicinski <kuba@kernel.org> writes:
>> > On Wed, 25 Feb 2026 17:06:48 +0200 Ioana Ciornei wrote:  
>> >> +ALL_TESTS="
>> >> +	test_eth_ctrl_stats
>> >> +	test_eth_mac_stats
>> >> +	test_pause_stats
>> >> +"
>> >> +STABLE_MAC_ADDRS=yes
>> >> +NUM_NETIFS=2
>> >> +lib_dir=$(dirname "$0")
>> >> +# shellcheck source=./../../../net/forwarding/lib.sh
>> >> +source "$lib_dir"/../../../net/forwarding/lib.sh  
>> >
>> > Argh, at some point we should probably decide whether we have 
>> > a preference on which "library" / set of env vars we use under 
>> > drivers/net/hw. Adding Petr to CC.  
>> 
>> I think the expectation is that by default, tests written in Bash are
>> run on one machine without remotes.
>
> I think we need to define the guidance by properties of the test, not
> the machine? Of course _tests_ which don't need a traffic source can

Oh yeah, I'm just stating that's how it currently is and how we got here.

> simply consume the NETIF evn variable, so its trivial to write them in
> any language without much library support. But for tests which need
> traffic input we need a different distinction than "does the author
> care about single-interface machines", so to speak?

I agree that adherence to the drivers/net/README protocol is valuable to
some users and would be good to uphold if reasonable in a given tests.
If that's what you have in mind.

There are going to be tests where it's not a great fit, but I think that
of those NUM_NETIFS=2 tests that we currently have, maybe
ethtool_extended_state has a good reason to be obstinate, because it
sets up negotiations and needs an extra unplugged netdevice.

>> 
>> > The existing tests under drivers/net/hw which source forwarding/lib.sh
>> > pre-date the "NIC setup" described in 
>> > tools/testing/selftests/drivers/net/README.rst
>> >
>> > Should we ask "NIC setup" to be used for all tests which only need
>> > NUM_NETIFS=2 ? These are basically simple tests where one netif is 
>> > a traffic generator and the other is the DUT. And IIUC the "forwarding
>> > setup" can't be used when the traffic generator is controlled over SSH?  
>> 
>> In principle nothing prevents lib.sh from growing brains to support
>> these remote shenanigans. I think it's just that so far nobody cared
>> enough to actually do it.
>> 
>> I think that a helper that in effect does "run this on a machine where
>> $swp1 is" is mostly what is needed. That and "make sure $swp1 and $swp2
>> are on the same machine". It's going to be annoying to work with though,
>> because you need to annotate every single command. I bet there's a nice
>> syntax to make it not activelly annoying.
>> 
>> If we have this, it might make sense to require tests to make use of it.
>> (With an explicit opt-out for special cases.) But I do not want every
>> test to have to reinvent this wheel and cargo-cult snippets from other
>> tests.
>> 
>> BTW, my guess is that even many multi-port tests that I wrote boil down
>> to just a bunch of fairly independent loopbacks whose far ends could be
>> on remote machines. It's not a priori nonsense to me that one would run
>> a test like this, or whatever magic we'd use:
>> 
>>     ./test.sh ssh://petr@10.1.2.3:eth1 swp1 veth1 ns://foo:veth2
>> 
>> And it just works, because only swp1 and swp2 need to be bridged, the
>> rest can be remote, and the traffic generation helper knows that to pump
>> traffic to ssh://10.1.2.3:eth1, obviously you need to ssh there first.
>> But the library would need to have helpers for this, and the tests would
>> need to use them.
>
> Right, to be clear I primarily care about being able to run these tests
> with traffic generator on a remote system. Otherwise someone who cares
> about single-port systems will have to add a separate test I guess?
> And we will have to maintain two tests? :S
>
> A nice to have is for all drivers/net/hw tests to use the "NIC" env
> variables (move tests which really only make sense on multi-port
> devices to drivers/net/forwarding?) But I think "translating"
> forwarding variables to NIC if NUMIF=2 could easily be done
> automatically by the test / lib so that's lower priority.

The reason for my diatribe above was like, if forwarding/lib.sh is
clever enough to handle these things transparently, then the NIC case
might be supported through a wrapper that just does the right thing, and
the test can be overtly just a run of the mill forwarding test.

The requirement for the remote netdevice to be up and preconfigured
throws a wrench in this idea though, so dunno.

One way or another, _some_ brains need to reside somewhere in a library.
I'm not sure forwarding.sh is the place. But I don't want to end up in
the same situation as like lib/half_the_tests.sh that reinvent logging
from first principles again and again.

On Sat, Feb 28, 2026 at 10:11:26AM +0100, Petr Machata wrote:
> 
> Jakub Kicinski <kuba@kernel.org> writes:
> 
> > On Fri, 27 Feb 2026 14:53:06 +0100 Petr Machata wrote:
> >> Jakub Kicinski <kuba@kernel.org> writes:
> >> > On Wed, 25 Feb 2026 17:06:48 +0200 Ioana Ciornei wrote:  
> >> >> +ALL_TESTS="
> >> >> +	test_eth_ctrl_stats
> >> >> +	test_eth_mac_stats
> >> >> +	test_pause_stats
> >> >> +"
> >> >> +STABLE_MAC_ADDRS=yes
> >> >> +NUM_NETIFS=2
> >> >> +lib_dir=$(dirname "$0")
> >> >> +# shellcheck source=./../../../net/forwarding/lib.sh
> >> >> +source "$lib_dir"/../../../net/forwarding/lib.sh  
> >> >
> >> > Argh, at some point we should probably decide whether we have 
> >> > a preference on which "library" / set of env vars we use under 
> >> > drivers/net/hw. Adding Petr to CC.  
> >> 
> >> I think the expectation is that by default, tests written in Bash are
> >> run on one machine without remotes.
> >
> > I think we need to define the guidance by properties of the test, not
> > the machine? Of course _tests_ which don't need a traffic source can
> 
> Oh yeah, I'm just stating that's how it currently is and how we got here.
> 
> > simply consume the NETIF evn variable, so its trivial to write them in
> > any language without much library support. But for tests which need
> > traffic input we need a different distinction than "does the author
> > care about single-interface machines", so to speak?
> 
> I agree that adherence to the drivers/net/README protocol is valuable to
> some users and would be good to uphold if reasonable in a given tests.
> If that's what you have in mind.
> 
> There are going to be tests where it's not a great fit, but I think that
> of those NUM_NETIFS=2 tests that we currently have, maybe
> ethtool_extended_state has a good reason to be obstinate, because it
> sets up negotiations and needs an extra unplugged netdevice.

I would add here even ethtool_rmon.sh and this new test that I
submitted. If you are running with a traffic generator on another board
then you can no longer check that the counter's value is as expected
(with a 1% tolerance), you can only check the lower bound.

Additionally, if you are using the same single port also for control
traffic towards the remote traffic generator, then you surely cannot
reliably check that counters that should not be incremented are indeed
not incremented.

This means that some tests will not benefit at all from working with a
remote link partner, it will make them weaker.

Ioana

On Mon, 2 Mar 2026 14:11:21 +0200 Ioana Ciornei wrote:
> > I agree that adherence to the drivers/net/README protocol is valuable to
> > some users and would be good to uphold if reasonable in a given tests.
> > If that's what you have in mind.
> > 
> > There are going to be tests where it's not a great fit, but I think that
> > of those NUM_NETIFS=2 tests that we currently have, maybe
> > ethtool_extended_state has a good reason to be obstinate, because it
> > sets up negotiations and needs an extra unplugged netdevice.  
> 
> I would add here even ethtool_rmon.sh and this new test that I

I think I already told you that ethool_rmon predates the NIC tests
and bringing it up in this discussion is irrelevant.

> submitted. If you are running with a traffic generator on another board
> then you can no longer check that the counter's value is as expected
> (with a 1% tolerance), you can only check the lower bound.

1% tolerance is impractical for any CI with high test count.
The test will be flaky. And I really doubt that the 1% tolerance
is really necessary to catch most bugs. We're not trying to validate
silicon here.

> Additionally, if you are using the same single port also for control
> traffic towards the remote traffic generator, then you surely cannot
> reliably check that counters that should not be incremented are indeed
> not incremented.

I both told you in this conversation how to check the counters,
and written some existing tests for counters.

On Mon, Mar 02, 2026 at 04:07:43PM -0800, Jakub Kicinski wrote:
> On Mon, 2 Mar 2026 14:11:21 +0200 Ioana Ciornei wrote:
> > > I agree that adherence to the drivers/net/README protocol is valuable to
> > > some users and would be good to uphold if reasonable in a given tests.
> > > If that's what you have in mind.
> > > 
> > > There are going to be tests where it's not a great fit, but I think that
> > > of those NUM_NETIFS=2 tests that we currently have, maybe
> > > ethtool_extended_state has a good reason to be obstinate, because it
> > > sets up negotiations and needs an extra unplugged netdevice.  
> > 
> > I would add here even ethtool_rmon.sh and this new test that I
> 
> I think I already told you that ethool_rmon predates the NIC tests
> and bringing it up in this discussion is irrelevant.
> 
> > submitted. If you are running with a traffic generator on another board
> > then you can no longer check that the counter's value is as expected
> > (with a 1% tolerance), you can only check the lower bound.
> 
> 1% tolerance is impractical for any CI with high test count.
> The test will be flaky. And I really doubt that the 1% tolerance
> is really necessary to catch most bugs. We're not trying to validate
> silicon here.
> 
> > Additionally, if you are using the same single port also for control
> > traffic towards the remote traffic generator, then you surely cannot
> > reliably check that counters that should not be incremented are indeed
> > not incremented.
> 
> I both told you in this conversation how to check the counters,
> and written some existing tests for counters.

Judging by your response it's clear to me that you wanted to transmit
something that didn't actually get to me. I am afraid that it's not
clear to me what exactly is your feedback and what do you expect as a
next step.

What I did get:
- The new test should work with a single netdevice (and a remote
  endpoint for traffic generation).
- The test should not check for any upper bound for the ethtool counter
  value.
- The test is expected to follow drivers/net/README.rst.

Does this mean that your feedback is to convert the bash variant that I
submitted into a python one which uses lib.py?

If this is your intention, what is the plan for the rmon statistics (or
any drivers/net/hw/ bash tests that pre-date the README)? Do you see
those eventually getting converted to lib.py? I am merely asking so that
I know if I should convert them or they are to be left as is.

If I got this all wrong, I'm sorry.

On Tue, 3 Mar 2026 15:53:41 +0200 Ioana Ciornei wrote:
> > > I would add here even ethtool_rmon.sh and this new test that I  
> > 
> > I think I already told you that ethool_rmon predates the NIC tests
> > and bringing it up in this discussion is irrelevant.
> >   
> > > submitted. If you are running with a traffic generator on another board
> > > then you can no longer check that the counter's value is as expected
> > > (with a 1% tolerance), you can only check the lower bound.  
> > 
> > 1% tolerance is impractical for any CI with high test count.
> > The test will be flaky. And I really doubt that the 1% tolerance
> > is really necessary to catch most bugs. We're not trying to validate
> > silicon here.
> >   
> > > Additionally, if you are using the same single port also for control
> > > traffic towards the remote traffic generator, then you surely cannot
> > > reliably check that counters that should not be incremented are indeed
> > > not incremented.  
> > 
> > I both told you in this conversation how to check the counters,
> > and written some existing tests for counters.  
> 
> Judging by your response it's clear to me that you wanted to transmit
> something that didn't actually get to me. I am afraid that it's not
> clear to me what exactly is your feedback and what do you expect as a
> next step.
> 
> What I did get:
> - The new test should work with a single netdevice (and a remote
>   endpoint for traffic generation).

yes

> - The test should not check for any upper bound for the ethtool counter
>   value.

more or less.. you can check for crazy values (bitflips etc).
Either pick a value too high to be reasonable (100Gbps * time)
or hardcode some high threshold (2^31?)

> - The test is expected to follow drivers/net/README.rst.

yes

> Does this mean that your feedback is to convert the bash variant that I
> submitted into a python one which uses lib.py?

It'd certainly be easier for you, but I'm not against building out 
the necessary support to run traffic from remote in bash.

> If this is your intention, what is the plan for the rmon statistics (or
> any drivers/net/hw/ bash tests that pre-date the README)? Do you see
> those eventually getting converted to lib.py? I am merely asking so that
> I know if I should convert them or they are to be left as is.

Yes, I was planning on doing the conversion once we have some real HW
testing going in NIPA. If you're willing to convert them - that'd be
great!

On Wed, Feb 25, 2026 at 05:06:48PM +0200, Ioana Ciornei wrote:
> Add a new selftest - ethtool_std_stats.sh - which validates the
> eth-ctrl, eth-mac and pause standard statistics exported by an
> interface. Not all counters can be validated in software, especially
> those that are keeping track of errors. Counters such as
> SingleCollisionFrames, FrameCheckSequenceErrors etc are not tested nor
> included in this new selftest.

Hi Ioana

Thanks for the test!

Do we actually expect errors when running such a test? How many times
have you run this test and seen any of the error counters be anything
other than 0?

Which do you think is more likely:

1) A real error happens

2) Bug in the driver so that it reports a value in the wrong place?

Maybe we should check the error counters are zero? 

      Andrew

On Thu, Feb 26, 2026 at 12:38:58AM +0100, Andrew Lunn wrote:
> On Wed, Feb 25, 2026 at 05:06:48PM +0200, Ioana Ciornei wrote:
> > Add a new selftest - ethtool_std_stats.sh - which validates the
> > eth-ctrl, eth-mac and pause standard statistics exported by an
> > interface. Not all counters can be validated in software, especially
> > those that are keeping track of errors. Counters such as
> > SingleCollisionFrames, FrameCheckSequenceErrors etc are not tested nor
> > included in this new selftest.
> 
> Hi Ioana
> 
> Thanks for the test!
> 
> Do we actually expect errors when running such a test? How many times
> have you run this test and seen any of the error counters be anything
> other than 0?

No, we don't expect any errors with this test and I didn't see any error
counters incremenent in the tens of times that I ran the selftest. But,
to be fair, I was not looking for them thoroughly through testing.

> 
> Which do you think is more likely:
> 
> 1) A real error happens
> 
> 2) Bug in the driver so that it reports a value in the wrong place?
> 

I would say that having a driver bug is much likely than, for example,
having an FCS error.

> Maybe we should check the error counters are zero? 

Ok, I will extend the test to check the errors against zero and see how it behaves.

Thanks!

On Thu, Feb 26, 2026 at 09:03:32AM +0200, Ioana Ciornei wrote:
> On Thu, Feb 26, 2026 at 12:38:58AM +0100, Andrew Lunn wrote:
> > On Wed, Feb 25, 2026 at 05:06:48PM +0200, Ioana Ciornei wrote:
> > > Add a new selftest - ethtool_std_stats.sh - which validates the
> > > eth-ctrl, eth-mac and pause standard statistics exported by an
> > > interface. Not all counters can be validated in software, especially
> > > those that are keeping track of errors. Counters such as
> > > SingleCollisionFrames, FrameCheckSequenceErrors etc are not tested nor
> > > included in this new selftest.
> > 
> > Hi Ioana
> > 
> > Thanks for the test!
> > 
> > Do we actually expect errors when running such a test? How many times
> > have you run this test and seen any of the error counters be anything
> > other than 0?
> 
> No, we don't expect any errors with this test and I didn't see any error
> counters incremenent in the tens of times that I ran the selftest. But,
> to be fair, I was not looking for them thoroughly through testing.
> 
> > 
> > Which do you think is more likely:
> > 
> > 1) A real error happens
> > 
> > 2) Bug in the driver so that it reports a value in the wrong place?
> > 
> 
> I would say that having a driver bug is much likely than, for example,
> having an FCS error.
> 
> > Maybe we should check the error counters are zero? 
> 
> Ok, I will extend the test to check the errors against zero and see how it behaves.
> 

I am back with a bit more information. The counters which were not
checked in this version can be grouped in two categories:

- Error counters such as:

	u64 FrameCheckSequenceErrors;
	u64 AlignmentErrors;
	u64 FramesLostDueToIntMACXmitError;
	u64 CarrierSenseErrors;
	u64 FramesLostDueToIntMACRcvError;
	u64 InRangeLengthErrors;
	u64 OutOfRangeLengthField;
	u64 FrameTooLongErrors;
	u64 FramesAbortedDueToXSColls;

  I did extend the selftest with these ones so that we check them
  against zero. I ran the test hundreds of times and I did not see any
  problems.

- Collision related counters (not really errors):

	u64 SingleCollisionFrames;
	u64 MultipleCollisionFrames;
	u64 FramesWithDeferredXmissions;
	u64 LateCollisions;
	u64 FramesWithExcessiveDeferral;

  With these I don't know what to do. Theoretically, they could be
  non-zero in half-duplex circumstances which means that checking for
  zero would not be entirely accurate.


 Ioana

> I am back with a bit more information. The counters which were not
> checked in this version can be grouped in two categories:
> 
> - Error counters such as:
> 
> 	u64 FrameCheckSequenceErrors;
> 	u64 AlignmentErrors;
> 	u64 FramesLostDueToIntMACXmitError;
> 	u64 CarrierSenseErrors;
> 	u64 FramesLostDueToIntMACRcvError;
> 	u64 InRangeLengthErrors;
> 	u64 OutOfRangeLengthField;
> 	u64 FrameTooLongErrors;
> 	u64 FramesAbortedDueToXSColls;
> 
>   I did extend the selftest with these ones so that we check them
>   against zero. I ran the test hundreds of times and I did not see any
>   problems.

Great.

> 
> - Collision related counters (not really errors):
> 
> 	u64 SingleCollisionFrames;
> 	u64 MultipleCollisionFrames;
> 	u64 FramesWithDeferredXmissions;
> 	u64 LateCollisions;
> 	u64 FramesWithExcessiveDeferral;
> 
>   With these I don't know what to do. Theoretically, they could be
>   non-zero in half-duplex circumstances which means that checking for
>   zero would not be entirely accurate.

How about, fail the test if any are greater than 1% of the number of
packets transmitted/received? My _guess_ is, if you have 1% packet
loss, networking is not going to be happy anyway. It probably means
you have one end doing Half duplex and the other Full. That is a
typical configuration error you see causing collisions. Not that i've
actually seen this for maybe a decade!

Failing the test, with a comment about checking duplex configuration,
seems sensible.

     Andrew

On Thu, Feb 26, 2026 at 02:31:26PM +0100, Andrew Lunn wrote:
> > I am back with a bit more information. The counters which were not
> > checked in this version can be grouped in two categories:
> > 
> > - Error counters such as:
> > 
> > 	u64 FrameCheckSequenceErrors;
> > 	u64 AlignmentErrors;
> > 	u64 FramesLostDueToIntMACXmitError;
> > 	u64 CarrierSenseErrors;
> > 	u64 FramesLostDueToIntMACRcvError;
> > 	u64 InRangeLengthErrors;
> > 	u64 OutOfRangeLengthField;
> > 	u64 FrameTooLongErrors;
> > 	u64 FramesAbortedDueToXSColls;
> > 
> >   I did extend the selftest with these ones so that we check them
> >   against zero. I ran the test hundreds of times and I did not see any
> >   problems.
> 
> Great.
> 
> > 
> > - Collision related counters (not really errors):
> > 
> > 	u64 SingleCollisionFrames;
> > 	u64 MultipleCollisionFrames;
> > 	u64 FramesWithDeferredXmissions;
> > 	u64 LateCollisions;
> > 	u64 FramesWithExcessiveDeferral;
> > 
> >   With these I don't know what to do. Theoretically, they could be
> >   non-zero in half-duplex circumstances which means that checking for
> >   zero would not be entirely accurate.
> 
> How about, fail the test if any are greater than 1% of the number of
> packets transmitted/received? My _guess_ is, if you have 1% packet
> loss, networking is not going to be happy anyway. It probably means
> you have one end doing Half duplex and the other Full. That is a
> typical configuration error you see causing collisions. Not that i've
> actually seen this for maybe a decade!
> 
> Failing the test, with a comment about checking duplex configuration,
> seems sensible.

Seems reasonable. Thanks for the help!

On Thu, 26 Feb 2026 16:18:21 +0200 Ioana Ciornei wrote:
> > How about, fail the test if any are greater than 1% of the number of
> > packets transmitted/received? My _guess_ is, if you have 1% packet
> > loss, networking is not going to be happy anyway. It probably means
> > you have one end doing Half duplex and the other Full. That is a
> > typical configuration error you see causing collisions. Not that i've
> > actually seen this for maybe a decade!
> > 
> > Failing the test, with a comment about checking duplex configuration,
> > seems sensible.  
> 
> Seems reasonable. Thanks for the help!

FWIW the expectation is that the test should be able to run even on
systems / boards with a single interface. So the control traffic
(communicating with the traffic generator) will run over the same
interface as the test. 1% error is unachievable. I'd only check the
lower bound, and use some sanity value for the upper bound (2^30 ?)
if at all

On Thu, Feb 26, 2026 at 06:25:11PM -0800, Jakub Kicinski wrote:
> On Thu, 26 Feb 2026 16:18:21 +0200 Ioana Ciornei wrote:
> > > How about, fail the test if any are greater than 1% of the number of
> > > packets transmitted/received? My _guess_ is, if you have 1% packet
> > > loss, networking is not going to be happy anyway. It probably means
> > > you have one end doing Half duplex and the other Full. That is a
> > > typical configuration error you see causing collisions. Not that i've
> > > actually seen this for maybe a decade!
> > > 
> > > Failing the test, with a comment about checking duplex configuration,
> > > seems sensible.  
> > 
> > Seems reasonable. Thanks for the help!
> 
> FWIW the expectation is that the test should be able to run even on
> systems / boards with a single interface. So the control traffic
> (communicating with the traffic generator) will run over the same
> interface as the test. 1% error is unachievable. I'd only check the
> lower bound, and use some sanity value for the upper bound (2^30 ?)
> if at all

Really? I didn't know of that expectation at all.

I did take ethtool_rmon.sh as an example and that selftest as well
takes NUM_NETIFS=2 and does check for both a lower bound and upper bound
that takes into account a 1% deviance from the target.

How would the test even work with only a single interface?

On Fri, 27 Feb 2026 09:34:28 +0200 Ioana Ciornei wrote:
> On Thu, Feb 26, 2026 at 06:25:11PM -0800, Jakub Kicinski wrote:
> > On Thu, 26 Feb 2026 16:18:21 +0200 Ioana Ciornei wrote:  
> > > > How about, fail the test if any are greater than 1% of the number of
> > > > packets transmitted/received? My _guess_ is, if you have 1% packet
> > > > loss, networking is not going to be happy anyway. It probably means
> > > > you have one end doing Half duplex and the other Full. That is a
> > > > typical configuration error you see causing collisions. Not that i've
> > > > actually seen this for maybe a decade!
> > > > 
> > > > Failing the test, with a comment about checking duplex configuration,
> > > > seems sensible.    
> > > 
> > > Seems reasonable. Thanks for the help!  
> > 
> > FWIW the expectation is that the test should be able to run even on
> > systems / boards with a single interface. So the control traffic
> > (communicating with the traffic generator) will run over the same
> > interface as the test. 1% error is unachievable. I'd only check the
> > lower bound, and use some sanity value for the upper bound (2^30 ?)
> > if at all  
> 
> Really? I didn't know of that expectation at all.
> 
> I did take ethtool_rmon.sh as an example and that selftest as well
> takes NUM_NETIFS=2 and does check for both a lower bound and upper bound
> that takes into account a 1% deviance from the target.

I called out in the other thread that the bash scripts in this dir
pre-date any serious CI use. They are only there to get them out of 
the way of SW-only CI testing.

> How would the test even work with only a single interface?

Hopefully the readme mentioned in my other reply clarifies.

On Fri, Feb 27, 2026 at 09:34:28AM +0200, Ioana Ciornei wrote:
> On Thu, Feb 26, 2026 at 06:25:11PM -0800, Jakub Kicinski wrote:
> > On Thu, 26 Feb 2026 16:18:21 +0200 Ioana Ciornei wrote:
> > > > How about, fail the test if any are greater than 1% of the number of
> > > > packets transmitted/received? My _guess_ is, if you have 1% packet
> > > > loss, networking is not going to be happy anyway. It probably means
> > > > you have one end doing Half duplex and the other Full. That is a
> > > > typical configuration error you see causing collisions. Not that i've
> > > > actually seen this for maybe a decade!
> > > > 
> > > > Failing the test, with a comment about checking duplex configuration,
> > > > seems sensible.  
> > > 
> > > Seems reasonable. Thanks for the help!
> > 
> > FWIW the expectation is that the test should be able to run even on
> > systems / boards with a single interface. So the control traffic
> > (communicating with the traffic generator) will run over the same
> > interface as the test. 1% error is unachievable. I'd only check the
> > lower bound, and use some sanity value for the upper bound (2^30 ?)
> > if at all
> 
> Really? I didn't know of that expectation at all.
> 
> I did take ethtool_rmon.sh as an example and that selftest as well
> takes NUM_NETIFS=2 and does check for both a lower bound and upper bound
> that takes into account a 1% deviance from the target.
> 
> How would the test even work with only a single interface?

Just to add to this, for the 1% i was referring to counters for
collisions. If the control traffic is causing collisions the system it
just as wrongly configured as generated traffic causing collisions.

For 'everyday' systems, i doubt Half Duplex is ever used, but
automotive with a T1 PHY might. So we might need to review this 1%
once somebody runs this test on such a system.

     Andrew

On Fri, 27 Feb 2026 15:17:02 +0100 Andrew Lunn wrote:
> On Fri, Feb 27, 2026 at 09:34:28AM +0200, Ioana Ciornei wrote:
> > On Thu, Feb 26, 2026 at 06:25:11PM -0800, Jakub Kicinski wrote:  
> > > FWIW the expectation is that the test should be able to run even on
> > > systems / boards with a single interface. So the control traffic
> > > (communicating with the traffic generator) will run over the same
> > > interface as the test. 1% error is unachievable. I'd only check the
> > > lower bound, and use some sanity value for the upper bound (2^30 ?)
> > > if at all  
> > 
> > Really? I didn't know of that expectation at all.
> > 
> > I did take ethtool_rmon.sh as an example and that selftest as well
> > takes NUM_NETIFS=2 and does check for both a lower bound and upper bound
> > that takes into account a 1% deviance from the target.
> > 
> > How would the test even work with only a single interface?  
> 
> Just to add to this, for the 1% i was referring to counters for
> collisions. If the control traffic is causing collisions the system it
> just as wrongly configured as generated traffic causing collisions.
> 
> For 'everyday' systems, i doubt Half Duplex is ever used, but
> automotive with a T1 PHY might. So we might need to review this 1%
> once somebody runs this test on such a system.

Right, right, errors and exceptions are probably fine.
I was referring to checking overall byte / packet counters with 1%
tolerance. Sorry if I misread the discussion or the patch.