[PATCH net-next v10 7/7] selftests: netconsole: validate target resume

Andre Carvalho posted 7 patches 3 weeks, 6 days ago
There is a newer version of this series
[PATCH net-next v10 7/7] selftests: netconsole: validate target resume
Posted by Andre Carvalho 3 weeks, 6 days ago
Introduce a new netconsole selftest to validate that netconsole is able
to resume a deactivated target when the low level interface comes back.

The test setups the network using netdevsim, creates a netconsole target
and then remove/add netdevsim in order to bring the same interfaces
back. Afterwards, the test validates that the target works as expected.

Targets are created via cmdline parameters to the module to ensure that
we are able to resume targets that were bound by mac and interface name.

Reviewed-by: Breno Leitao <leitao@debian.org>
Signed-off-by: Andre Carvalho <asantostc@gmail.com>
---
 tools/testing/selftests/drivers/net/Makefile       |  1 +
 .../selftests/drivers/net/lib/sh/lib_netcons.sh    | 35 ++++++--
 .../selftests/drivers/net/netcons_resume.sh        | 97 ++++++++++++++++++++++
 3 files changed, 128 insertions(+), 5 deletions(-)

diff --git a/tools/testing/selftests/drivers/net/Makefile b/tools/testing/selftests/drivers/net/Makefile
index f5c71d993750..3eba569b3366 100644
--- a/tools/testing/selftests/drivers/net/Makefile
+++ b/tools/testing/selftests/drivers/net/Makefile
@@ -19,6 +19,7 @@ TEST_PROGS := \
 	netcons_cmdline.sh \
 	netcons_fragmented_msg.sh \
 	netcons_overflow.sh \
+	netcons_resume.sh \
 	netcons_sysdata.sh \
 	netcons_torture.sh \
 	netpoll_basic.py \
diff --git a/tools/testing/selftests/drivers/net/lib/sh/lib_netcons.sh b/tools/testing/selftests/drivers/net/lib/sh/lib_netcons.sh
index ae8abff4be40..b6093bcf2b06 100644
--- a/tools/testing/selftests/drivers/net/lib/sh/lib_netcons.sh
+++ b/tools/testing/selftests/drivers/net/lib/sh/lib_netcons.sh
@@ -203,19 +203,21 @@ function do_cleanup() {
 function cleanup_netcons() {
 	# delete netconsole dynamic reconfiguration
 	# do not fail if the target is already disabled
-	if [[ ! -d "${NETCONS_PATH}" ]]
+	local TARGET_PATH=${1:-${NETCONS_PATH}}
+
+	if [[ ! -d "${TARGET_PATH}" ]]
 	then
 		# in some cases this is called before netcons path is created
 		return
 	fi
-	if [[ $(cat "${NETCONS_PATH}"/enabled) != 0 ]]
+	if [[ $(cat "${TARGET_PATH}"/enabled) != 0 ]]
 	then
-		echo 0 > "${NETCONS_PATH}"/enabled || true
+		echo 0 > "${TARGET_PATH}"/enabled || true
 	fi
 	# Remove all the keys that got created during the selftest
-	find "${NETCONS_PATH}/userdata/" -mindepth 1 -type d -delete
+	find "${TARGET_PATH}/userdata/" -mindepth 1 -type d -delete
 	# Remove the configfs entry
-	rmdir "${NETCONS_PATH}"
+	rmdir "${TARGET_PATH}"
 }
 
 function cleanup() {
@@ -377,6 +379,29 @@ function check_netconsole_module() {
 	fi
 }
 
+function wait_target_state() {
+	local TARGET=${1}
+	local STATE=${2}
+	local TARGET_PATH="${NETCONS_CONFIGFS}"/"${TARGET}"
+	local ENABLED=0
+
+	if [ "${STATE}" == "enabled" ]
+	then
+		ENABLED=1
+	fi
+
+	if [ ! -d "$TARGET_PATH" ]; then
+		echo "FAIL: Target does not exist." >&2
+		exit "${ksft_fail}"
+	fi
+
+	local CHECK_CMD="grep \"$ENABLED\" \"$TARGET_PATH/enabled\""
+	slowwait 2 sh -c "test -n \"\$($CHECK_CMD)\"" || {
+		echo "FAIL: ${TARGET} is not ${STATE}." >&2
+		exit "${ksft_fail}"
+	}
+}
+
 # A wrapper to translate protocol version to udp version
 function wait_for_port() {
 	local NAMESPACE=${1}
diff --git a/tools/testing/selftests/drivers/net/netcons_resume.sh b/tools/testing/selftests/drivers/net/netcons_resume.sh
new file mode 100755
index 000000000000..383ad1149271
--- /dev/null
+++ b/tools/testing/selftests/drivers/net/netcons_resume.sh
@@ -0,0 +1,97 @@
+#!/usr/bin/env bash
+# SPDX-License-Identifier: GPL-2.0
+
+# This test validates that netconsole is able to resume a target that was
+# deactivated when its interface was removed when the interface is brought
+# back up.
+#
+# The test configures a netconsole target and then removes netdevsim module to
+# cause the interface to disappear. Targets are configured via cmdline to ensure
+# targets bound by interface name and mac address can be resumed.
+# The test verifies that the target moved to disabled state before adding
+# netdevsim and the interface back.
+#
+# Finally, the test verifies that the target is re-enabled automatically and
+# the message is received on the destination interface.
+#
+# Author: Andre Carvalho <asantostc@gmail.com>
+
+set -euo pipefail
+
+SCRIPTDIR=$(dirname "$(readlink -e "${BASH_SOURCE[0]}")")
+
+source "${SCRIPTDIR}"/lib/sh/lib_netcons.sh
+
+modprobe netdevsim 2> /dev/null || true
+rmmod netconsole 2> /dev/null || true
+
+check_netconsole_module
+
+function cleanup() {
+	cleanup_netcons "${NETCONS_CONFIGFS}/cmdline0"
+	do_cleanup
+	rmmod netconsole
+}
+
+trap cleanup EXIT
+
+# Run the test twice, with different cmdline parameters
+for BINDMODE in "ifname" "mac"
+do
+	echo "Running with bind mode: ${BINDMODE}" >&2
+	# Set current loglevel to KERN_INFO(6), and default to KERN_NOTICE(5)
+	echo "6 5" > /proc/sys/kernel/printk
+
+	# Create one namespace and two interfaces
+	set_network
+
+	# Create the command line for netconsole, with the configuration from
+	# the function above
+	CMDLINE=$(create_cmdline_str "${BINDMODE}")
+
+	# The content of kmsg will be save to the following file
+	OUTPUT_FILE="/tmp/${TARGET}-${BINDMODE}"
+
+	# Load the module, with the cmdline set
+	modprobe netconsole "${CMDLINE}"
+	# Expose cmdline target in configfs
+	mkdir "${NETCONS_CONFIGFS}/cmdline0"
+
+	# Target should be enabled
+	wait_target_state "cmdline0" "enabled"
+
+	# Remove low level module
+	rmmod netdevsim
+	# Target should be disabled
+	wait_target_state "cmdline0" "disabled"
+
+	# Add back low level module
+	modprobe netdevsim
+	# Recreate namespace and two interfaces
+	set_network
+	# Target should be enabled again
+	wait_target_state "cmdline0" "enabled"
+
+	# Listen for netconsole port inside the namespace and destination
+	# interface
+	listen_port_and_save_to "${OUTPUT_FILE}" &
+	# Wait for socat to start and listen to the port.
+	wait_local_port_listen "${NAMESPACE}" "${PORT}" udp
+	# Send the message
+	echo "${MSG}: ${TARGET}" > /dev/kmsg
+	# Wait until socat saves the file to disk
+	busywait "${BUSYWAIT_TIMEOUT}" test -s "${OUTPUT_FILE}"
+	# Make sure the message was received in the dst part
+	# and exit
+	validate_msg "${OUTPUT_FILE}"
+
+	# kill socat in case it is still running
+	pkill_socat
+	# Cleanup & unload the module
+	cleanup
+
+	echo "${BINDMODE} : Test passed" >&2
+done
+
+trap - EXIT
+exit "${EXIT_STATUS}"

-- 
2.52.0
Re: [PATCH net-next v10 7/7] selftests: netconsole: validate target resume
Posted by Jakub Kicinski 3 weeks, 6 days ago
On Mon, 12 Jan 2026 09:40:58 +0000 Andre Carvalho wrote:
> Introduce a new netconsole selftest to validate that netconsole is able
> to resume a deactivated target when the low level interface comes back.
> 
> The test setups the network using netdevsim, creates a netconsole target
> and then remove/add netdevsim in order to bring the same interfaces
> back. Afterwards, the test validates that the target works as expected.
> 
> Targets are created via cmdline parameters to the module to ensure that
> we are able to resume targets that were bound by mac and interface name.

The new test seems to be failing in netdev CI:

TAP version 13
1..1
# timeout set to 180
# selftests: drivers/net: netcons_resume.sh
# Running with bind mode: ifname
not ok 1 selftests: drivers/net: netcons_resume.sh # exit=1
-- 
pw-bot: cr
Re: [PATCH net-next v10 7/7] selftests: netconsole: validate target resume
Posted by Andre Carvalho 3 weeks, 1 day ago
On Mon, Jan 12, 2026 at 06:16:42AM -0800, Jakub Kicinski wrote:
> On Mon, 12 Jan 2026 09:40:58 +0000 Andre Carvalho wrote:
> > Introduce a new netconsole selftest to validate that netconsole is able
> > to resume a deactivated target when the low level interface comes back.
> > 
> > The test setups the network using netdevsim, creates a netconsole target
> > and then remove/add netdevsim in order to bring the same interfaces
> > back. Afterwards, the test validates that the target works as expected.
> > 
> > Targets are created via cmdline parameters to the module to ensure that
> > we are able to resume targets that were bound by mac and interface name.
> 
> The new test seems to be failing in netdev CI:
> 
> TAP version 13
> 1..1
> # timeout set to 180
> # selftests: drivers/net: netcons_resume.sh
> # Running with bind mode: ifname
> not ok 1 selftests: drivers/net: netcons_resume.sh # exit=1
> -- 
> pw-bot: cr

I've finally been able to reproduce this locally. The issue is caused by the
fact that the test currently expects that mac addresses for netdevsim devices are
deterministic. This is the case on my setup as systemd enforces it (MACAddressPolicy=persistent).

I was able to disable this behaviour by setting up /etc/systemd/network/50-netdevsim.link, with:

[Match]
Driver=netdevsim

[Link]
MACAddressPolicy=none

I'm assuming this is also the behaviour on CI hosts. I have started working on a fix
for this test and will submit v11 once that is ready. The approach I'm taking is saving and
restoring the mac addresses once I reload netdevsim module. Example code below (needs more testing):

function deactivate() {
	# Start by storing mac addresses so we can be restored in reactivate
	SAVED_DSTMAC=$(ip netns exec "${NAMESPACE}" \
		cat /sys/class/net/"$DSTIF"/address)
	SAVED_SRCMAC=$(mac_get "${SRCIF}")
	# Remove low level module
	rmmod netdevsim
}

function reactivate() {
	# Add back low level module
	modprobe netdevsim
	# Recreate namespace and two interfaces
	set_network
	# Restore MACs
	ip netns exec "${NAMESPACE}" ip link set "${DSTIF}" \
		address "${SAVED_DSTMAC}"
	if [ "${BINDMODE}" == "mac" ]; then
		ip link set dev "${SRCIF}" down
		ip link set dev "${SRCIF}" address "${SAVED_SRCMAC}"
		# Rename device in order to trigger target resume, as initial
		# when device was recreated it didnt have correct mac address.
		ip link set dev "${SRCIF}" name "${TARGET}"
	fi
}

The main annoyance is that to test resuming when a device was bound by mac I actually need
to change the name of the device after restoring the mac address (since when the device 
is registered after deactivation the mac won't match).


-- 
Andre Carvalho
Re: [PATCH net-next v10 7/7] selftests: netconsole: validate target resume
Posted by Jakub Kicinski 3 weeks, 1 day ago
On Fri, 16 Jan 2026 21:01:22 +0000 Andre Carvalho wrote:
> On Mon, Jan 12, 2026 at 06:16:42AM -0800, Jakub Kicinski wrote:
> > On Mon, 12 Jan 2026 09:40:58 +0000 Andre Carvalho wrote:  
> > > Introduce a new netconsole selftest to validate that netconsole is able
> > > to resume a deactivated target when the low level interface comes back.
> > > 
> > > The test setups the network using netdevsim, creates a netconsole target
> > > and then remove/add netdevsim in order to bring the same interfaces
> > > back. Afterwards, the test validates that the target works as expected.
> > > 
> > > Targets are created via cmdline parameters to the module to ensure that
> > > we are able to resume targets that were bound by mac and interface name.  
> > 
> > The new test seems to be failing in netdev CI:
> > 
> > TAP version 13
> > 1..1
> > # timeout set to 180
> > # selftests: drivers/net: netcons_resume.sh
> > # Running with bind mode: ifname
> > not ok 1 selftests: drivers/net: netcons_resume.sh # exit=1
> > -- 
> > pw-bot: cr  
> 
> I've finally been able to reproduce this locally. The issue is caused by the
> fact that the test currently expects that mac addresses for netdevsim devices are
> deterministic. This is the case on my setup as systemd enforces it (MACAddressPolicy=persistent).

Argh, systemd strikes again :(

> I was able to disable this behaviour by setting up /etc/systemd/network/50-netdevsim.link, with:
> 
> [Match]
> Driver=netdevsim
> 
> [Link]
> MACAddressPolicy=none
> 
> I'm assuming this is also the behaviour on CI hosts. 

Yes, systemd changing the MAC address is racy - it does it too slowly
and some tests start doing their thing, then systemd comes in and flips
the address. So indeed:

# cat /etc/systemd/network/99-default.link
[Match]
OriginalName=*

[Link]
NamePolicy=keep kernel database onboard slot path
AlternativeNamesPolicy=database onboard slot path mac
MACAddressPolicy=none

> I have started working on a fix
> for this test and will submit v11 once that is ready. The approach I'm taking is saving and
> restoring the mac addresses once I reload netdevsim module. Example code below (needs more testing):
> 
> function deactivate() {
> 	# Start by storing mac addresses so we can be restored in reactivate
> 	SAVED_DSTMAC=$(ip netns exec "${NAMESPACE}" \
> 		cat /sys/class/net/"$DSTIF"/address)
> 	SAVED_SRCMAC=$(mac_get "${SRCIF}")
> 	# Remove low level module
> 	rmmod netdevsim
> }
> 
> function reactivate() {
> 	# Add back low level module
> 	modprobe netdevsim
> 	# Recreate namespace and two interfaces
> 	set_network
> 	# Restore MACs
> 	ip netns exec "${NAMESPACE}" ip link set "${DSTIF}" \
> 		address "${SAVED_DSTMAC}"
> 	if [ "${BINDMODE}" == "mac" ]; then
> 		ip link set dev "${SRCIF}" down
> 		ip link set dev "${SRCIF}" address "${SAVED_SRCMAC}"
> 		# Rename device in order to trigger target resume, as initial
> 		# when device was recreated it didnt have correct mac address.
> 		ip link set dev "${SRCIF}" name "${TARGET}"
> 	fi
> }
> 
> The main annoyance is that to test resuming when a device was bound by mac I actually need
> to change the name of the device after restoring the mac address (since when the device 
> is registered after deactivation the mac won't match).

Workaround sounds reasonable, FWIW.
Re: [PATCH net-next v10 7/7] selftests: netconsole: validate target resume
Posted by Breno Leitao 3 weeks, 6 days ago
On Mon, Jan 12, 2026 at 06:16:42AM -0800, Jakub Kicinski wrote:
> On Mon, 12 Jan 2026 09:40:58 +0000 Andre Carvalho wrote:
> > Introduce a new netconsole selftest to validate that netconsole is able
> > to resume a deactivated target when the low level interface comes back.
> > 
> > The test setups the network using netdevsim, creates a netconsole target
> > and then remove/add netdevsim in order to bring the same interfaces
> > back. Afterwards, the test validates that the target works as expected.
> > 
> > Targets are created via cmdline parameters to the module to ensure that
> > we are able to resume targets that were bound by mac and interface name.
> 
> The new test seems to be failing in netdev CI:
> 
> TAP version 13
> 1..1
> # timeout set to 180
> # selftests: drivers/net: netcons_resume.sh
> # Running with bind mode: ifname
> not ok 1 selftests: drivers/net: netcons_resume.sh # exit=1

I was discussing this with Andre on private.

Also, do you know why we got:

	/srv/vmksft/testing/wt-18/tools/testing/selftests/kselftest/runner.sh: line 50: : No such file or directory

after the test failed?

Link: https://netdev-ctrl.bots.linux.dev/logs/vmksft/net-drv-dbg/results/470321/3-netcons-resume-sh/stdout
Re: [PATCH net-next v10 7/7] selftests: netconsole: validate target resume
Posted by Andre Carvalho 3 weeks, 5 days ago
On Mon, Jan 12, 2026 at 07:16:54AM -0800, Breno Leitao wrote:
> > On Mon, Jan 12, 2026 at 06:16:42AM -0800, Jakub Kicinski wrote:
> > The new test seems to be failing in netdev CI:
> > 
> > TAP version 13
> > 1..1
> > # timeout set to 180
> > # selftests: drivers/net: netcons_resume.sh
> > # Running with bind mode: ifname
> > not ok 1 selftests: drivers/net: netcons_resume.sh # exit=1
> 
> I was discussing this with Andre on private.

I have not been able to reproduce the failure locally yet. I've checkout out the
CI branch from linux-netdev/testing and followed the instructions in [1] to try
running it as close as possible to the CI setup and no luck yet.

I'll continue digging but appreciate any suggestions.

> Also, do you know why we got:
> 
> 	/srv/vmksft/testing/wt-18/tools/testing/selftests/kselftest/runner.sh: line 50: : No such file or directory
> 
> after the test failed?

I think this might be caused by a change[2] in run_kselftest.sh/runner.sh which
seems to cause runner.sh to fail when a test failed and runner.sh was not called
from run_kselftest.sh.

[1] https://github.com/linux-netdev/nipa/wiki/How-to-run-netdev-selftests-CI-style
[2] https://lore.kernel.org/all/20251111-b4-ksft-error-on-fail-v3-1-0951a51135f6@google.com/
-- 
Andre Carvalho
Re: [PATCH net-next v10 7/7] selftests: netconsole: validate target resume
Posted by Jakub Kicinski 3 weeks, 5 days ago
On Mon, 12 Jan 2026 20:49:02 +0000 Andre Carvalho wrote:
> > Also, do you know why we got:
> > 
> > 	/srv/vmksft/testing/wt-18/tools/testing/selftests/kselftest/runner.sh: line 50: : No such file or directory
> > 
> > after the test failed?  
> 
> I think this might be caused by a change[2] in run_kselftest.sh/runner.sh which
> seems to cause runner.sh to fail when a test failed and runner.sh was not called
> from run_kselftest.sh.

Yes! FWIW we see it in all tests in netdev :( 
Who tests the test infra, right?

It's not actually causing any functional issues for use so
I haven't followed up on this, but would be awesome to get it fixed..