From nobody Mon Feb 9 01:21:23 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 12FC81E25FA for ; Sun, 30 Mar 2025 21:54:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743371646; cv=none; b=QvW0XbeuQJcymv2CY9SDNwWCGc0/qGmNK5W8N+deb0swyM2CgDs//LwbXTX+kA/+8d8cIdCJOT476vSy/ax7lsvE+ziaLPwWbFWOYTJ8Dagj/whDDIcW13QkOljx3KnKtzMSWt2tmQUu1EjmIiuXCtEDKeaBoJwKYdyWPvH4zcs= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1743371646; c=relaxed/simple; bh=X6qEyyM62Rhm4nq9aLITTboCNm1IH474NxPhm04k+04=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=molxXWsm5duOncsQ3ZIjxaFUsPlJlGesEDsDWLH1aAJDJe7mGojZmePp0AKc8Dd794iIxB1EMQBka2fmA+388IUdgHr6O2rxa7qb+dntfJg4fI1ROR+7r8wv9zK6EpjCR0yXeMPt1FQ/+klxF+Hcu+0vfIhrbBa2jDnGTMIfvKM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=e1NYo4sV; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="e1NYo4sV" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1743371643; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=mrQjpFlShLNKMETTxcA/lOqCZh7cDFYxhNQyMaFasXA=; b=e1NYo4sVQFv8lHfdIQ1gUP5q3YJMtvwVQRCOJe15L/+B8nRMyipppa5yoJONsnl8garCb3 skByXM+ROsbqQ4v+TN8Mxufz3QKUWMeNIpSQb332RT2/CwuoVUKZHBQ4FfomJzYUQX05Bd Lvq/lsQjjnbmAWe0F/QQDOTDHhgI3Ww= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-177-Oxe5lqeuP3G8J4XLRwopuw-1; Sun, 30 Mar 2025 17:54:00 -0400 X-MC-Unique: Oxe5lqeuP3G8J4XLRwopuw-1 X-Mimecast-MFC-AGG-ID: Oxe5lqeuP3G8J4XLRwopuw_1743371639 Received: from mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.111]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 3BAE018EBE8A; Sun, 30 Mar 2025 21:53:59 +0000 (UTC) Received: from llong-thinkpadp16vgen1.westford.csb (unknown [10.22.64.34]) by mx-prod-int-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 913071801750; Sun, 30 Mar 2025 21:53:57 +0000 (UTC) From: Waiman Long To: Tejun Heo , Johannes Weiner , =?UTF-8?q?Michal=20Koutn=C3=BD?= , Shuah Khan Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org, Waiman Long Subject: [PATCH 09/10] selftest/cgroup: Clean up and restructure test_cpuset_prs.sh Date: Sun, 30 Mar 2025 17:52:47 -0400 Message-ID: <20250330215248.3620801-10-longman@redhat.com> In-Reply-To: <20250330215248.3620801-1-longman@redhat.com> References: <20250330215248.3620801-1-longman@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.111 Content-Type: text/plain; charset="utf-8" Cleaning up the test_cpuset_prs.sh script and restructure some of the functions so that a new test matrix with a different cgroup directory structure can be added in the next patch. Signed-off-by: Waiman Long --- .../selftests/cgroup/test_cpuset_prs.sh | 257 +++++++++++------- 1 file changed, 156 insertions(+), 101 deletions(-) diff --git a/tools/testing/selftests/cgroup/test_cpuset_prs.sh b/tools/test= ing/selftests/cgroup/test_cpuset_prs.sh index f11f347129d8..d99412e7d196 100755 --- a/tools/testing/selftests/cgroup/test_cpuset_prs.sh +++ b/tools/testing/selftests/cgroup/test_cpuset_prs.sh @@ -88,22 +88,30 @@ echo "" > test/cpuset.cpus # If isolated CPUs have been reserved at boot time (as shown in # cpuset.cpus.isolated), these isolated CPUs should be outside of CPUs 0-8 # that will be used by this script for testing purpose. If not, some of -# the tests may fail incorrectly. These pre-isolated CPUs should stay in -# an isolated state throughout the testing process for now. +# the tests may fail incorrectly. Wait a bit and retry again just in case +# these isolated CPUs are leftover from previous run and have just been +# cleaned up earlier in this script. +# +# These pre-isolated CPUs should stay in an isolated state throughout the +# testing process for now. # BOOT_ISOLCPUS=3D$(cat $CGROUP2/cpuset.cpus.isolated) +[[ -n "$BOOT_ISOLCPUS" ]] && { + sleep 0.5 + BOOT_ISOLCPUS=3D$(cat $CGROUP2/cpuset.cpus.isolated) +} if [[ -n "$BOOT_ISOLCPUS" ]] then [[ $(echo $BOOT_ISOLCPUS | sed -e "s/[,-].*//") -le 8 ]] && skip_test "Pre-isolated CPUs ($BOOT_ISOLCPUS) overlap CPUs to be tested" echo "Pre-isolated CPUs: $BOOT_ISOLCPUS" fi + cleanup() { online_cpus cd $CGROUP2 - rmdir A1/A2/A3 A1/A2 A1 B1 > /dev/null 2>&1 - rmdir test > /dev/null 2>&1 + rmdir A1/A2/A3 A1/A2 A1 B1 test/A1 test/B1 test > /dev/null 2>&1 [[ -n "$SCHED_DEBUG" ]] && echo "$SCHED_DEBUG" > /sys/kernel/debug/sched/verbose } @@ -173,14 +181,22 @@ test_add_proc() # # Cgroup test hierarchy # -# root -- A1 -- A2 -- A3 -# +- B1 +# root +# | +# +------+------+ +# | | +# A1 B1 +# | +# A2 +# | +# A3 # # P =3D set cpus.partition (0:member, 1:root, 2:isolated) # C =3D add cpu-list to cpuset.cpus # X =3D add cpu-list to cpuset.cpus.exclusive # S

=3D use prefix in subtree_control # T =3D put a task into cgroup +# CX =3D add cpu-list to both cpuset.cpus and cpuset.cpus.exclusive # O=3D =3D Write to CPU online file of # # ECPUs - effective CPUs of cpusets @@ -453,25 +469,26 @@ set_ctrl_state() PFILE=3D$CGRP/cpuset.cpus.partition CFILE=3D$CGRP/cpuset.cpus XFILE=3D$CGRP/cpuset.cpus.exclusive - S=3D$(expr substr $CMD 1 1) - if [[ $S =3D S ]] - then - PREFIX=3D${CMD#?} + case $CMD in + S*) PREFIX=3D${CMD#?} COMM=3D"echo ${PREFIX}${CTRL} > $SFILE" eval $COMM $REDIRECT - elif [[ $S =3D X ]] - then + ;; + X*) CPUS=3D${CMD#?} COMM=3D"echo $CPUS > $XFILE" eval $COMM $REDIRECT - elif [[ $S =3D C ]] - then - CPUS=3D${CMD#?} + ;; + CX*) + CPUS=3D${CMD#??} + COMM=3D"echo $CPUS > $CFILE; echo $CPUS > $XFILE" + eval $COMM $REDIRECT + ;; + C*) CPUS=3D${CMD#?} COMM=3D"echo $CPUS > $CFILE" eval $COMM $REDIRECT - elif [[ $S =3D P ]] - then - VAL=3D${CMD#?} + ;; + P*) VAL=3D${CMD#?} case $VAL in 0) VAL=3Dmember ;; @@ -486,15 +503,17 @@ set_ctrl_state() esac COMM=3D"echo $VAL > $PFILE" eval $COMM $REDIRECT - elif [[ $S =3D O ]] - then - VAL=3D${CMD#?} + ;; + O*) VAL=3D${CMD#?} write_cpu_online $VAL - elif [[ $S =3D T ]] - then - COMM=3D"echo 0 > $TFILE" + ;; + T*) COMM=3D"echo 0 > $TFILE" eval $COMM $REDIRECT - fi + ;; + *) echo "Unknown command: $CMD" + exit 1 + ;; + esac RET=3D$? [[ $RET -ne 0 ]] && { [[ -n "$SHOWERR" ]] && { @@ -532,21 +551,18 @@ online_cpus() } =20 # -# Return 1 if the list of effective cpus isn't the same as the initial lis= t. +# Remove all the test cgroup directories # reset_cgroup_states() { echo 0 > $CGROUP2/cgroup.procs online_cpus - rmdir A1/A2/A3 A1/A2 A1 B1 > /dev/null 2>&1 - pause 0.02 - set_ctrl_state . R- - pause 0.01 + rmdir $RESET_LIST > /dev/null 2>&1 } =20 dump_states() { - for DIR in . A1 A1/A2 A1/A2/A3 B1 + for DIR in $CGROUP_LIST do CPUS=3D$DIR/cpuset.cpus ECPUS=3D$DIR/cpuset.cpus.effective @@ -565,6 +581,21 @@ dump_states() done } =20 +# +# Set the actual cgroup directory into $CGRP_DIR +# $1 - cgroup name +# +set_cgroup_dir() +{ + CGRP_DIR=3D$1 + [[ $CGRP_DIR =3D A2 ]] && CGRP_DIR=3DA1/A2 + [[ $CGRP_DIR =3D A3 ]] && CGRP_DIR=3DA1/A2/A3 + [[ $CGRP_DIR =3D c11 ]] && CGRP_DIR=3Dp1/c11 + [[ $CGRP_DIR =3D c12 ]] && CGRP_DIR=3Dp1/c12 + [[ $CGRP_DIR =3D c21 ]] && CGRP_DIR=3Dp2/c21 + [[ $CGRP_DIR =3D c22 ]] && CGRP_DIR=3Dp2/c22 +} + # # Check effective cpus # $1 - check string, format: :[|:]* @@ -576,7 +607,8 @@ check_effective_cpus() do set -- $(echo $CHK | sed -e "s/:/ /g") CGRP=3D$1 - CPUS=3D$2 + EXPECTED_CPUS=3D$2 + ACTUAL_CPUS=3D if [[ $CGRP =3D X* ]] then CGRP=3D${CGRP#X} @@ -584,10 +616,10 @@ check_effective_cpus() else FILE=3Dcpuset.cpus.effective fi - [[ $CGRP =3D A2 ]] && CGRP=3DA1/A2 - [[ $CGRP =3D A3 ]] && CGRP=3DA1/A2/A3 - [[ -e $CGRP/$FILE ]] || return 1 - [[ $CPUS =3D $(cat $CGRP/$FILE) ]] || return 1 + set_cgroup_dir $CGRP + [[ -e $CGRP_DIR/$FILE ]] || return 1 + ACTUAL_CPUS=3D$(cat $CGRP_DIR/$FILE) + [[ $EXPECTED_CPUS =3D $ACTUAL_CPUS ]] || return 1 done } =20 @@ -602,23 +634,21 @@ check_cgroup_states() do set -- $(echo $CHK | sed -e "s/:/ /g") CGRP=3D$1 - CGRP_DIR=3D$CGRP - STATE=3D$2 + EXPECTED_STATE=3D$2 FILE=3D - EVAL=3D$(expr substr $STATE 2 2) - [[ $CGRP =3D A2 ]] && CGRP_DIR=3DA1/A2 - [[ $CGRP =3D A3 ]] && CGRP_DIR=3DA1/A2/A3 + EVAL=3D$(expr substr $EXPECTED_STATE 2 2) =20 - case $STATE in + set_cgroup_dir $CGRP + case $EXPECTED_STATE in P*) FILE=3D$CGRP_DIR/cpuset.cpus.partition ;; - *) echo "Unknown state: $STATE!" + *) echo "Unknown state: $EXPECTED_STATE!" exit 1 ;; esac - VAL=3D$(cat $FILE) + ACTUAL_STATE=3D$(cat $FILE) =20 - case "$VAL" in + case "$ACTUAL_STATE" in member) VAL=3D0 ;; root) VAL=3D1 @@ -642,7 +672,7 @@ check_cgroup_states() [[ $VAL -eq 1 && $VERBOSE -gt 0 ]] && { DOMS=3D$(cat $CGRP_DIR/cpuset.cpus.effective) [[ -n "$DOMS" ]] && - echo " [$CGRP] sched-domain: $DOMS" > $CONSOLE + echo " [$CGRP_DIR] sched-domain: $DOMS" > $CONSOLE } done return 0 @@ -665,22 +695,22 @@ check_cgroup_states() # check_isolcpus() { - EXPECT_VAL=3D$1 - ISOLCPUS=3D + EXPECTED_ISOLCPUS=3D$1 + ISCPUS=3D${CGROUP2}/cpuset.cpus.isolated + ISOLCPUS=3D$(cat $ISCPUS) LASTISOLCPU=3D SCHED_DOMAINS=3D/sys/kernel/debug/sched/domains - ISCPUS=3D${CGROUP2}/cpuset.cpus.isolated - if [[ $EXPECT_VAL =3D . ]] + if [[ $EXPECTED_ISOLCPUS =3D . ]] then - EXPECT_VAL=3D - EXPECT_VAL2=3D - elif [[ $(expr $EXPECT_VAL : ".*|.*") > 0 ]] + EXPECTED_ISOLCPUS=3D + EXPECTED_SDOMAIN=3D + elif [[ $(expr $EXPECTED_ISOLCPUS : ".*|.*") > 0 ]] then - set -- $(echo $EXPECT_VAL | sed -e "s/|/ /g") - EXPECT_VAL=3D$1 - EXPECT_VAL2=3D$2 + set -- $(echo $EXPECTED_ISOLCPUS | sed -e "s/|/ /g") + EXPECTED_ISOLCPUS=3D$2 + EXPECTED_SDOMAIN=3D$1 else - EXPECT_VAL2=3D$EXPECT_VAL + EXPECTED_SDOMAIN=3D$EXPECTED_ISOLCPUS fi =20 # @@ -689,20 +719,21 @@ check_isolcpus() # to make appending those CPUs easier. # [[ -n "$BOOT_ISOLCPUS" ]] && { - EXPECT_VAL=3D${EXPECT_VAL:+${EXPECT_VAL},}${BOOT_ISOLCPUS} - EXPECT_VAL2=3D${EXPECT_VAL2:+${EXPECT_VAL2},}${BOOT_ISOLCPUS} + EXPECTED_ISOLCPUS=3D${EXPECTED_ISOLCPUS:+${EXPECTED_ISOLCPUS},}${BOOT_IS= OLCPUS} + EXPECTED_SDOMAIN=3D${EXPECTED_SDOMAIN:+${EXPECTED_SDOMAIN},}${BOOT_ISOLC= PUS} } =20 # # Check cpuset.cpus.isolated cpumask # - [[ "$EXPECT_VAL2" !=3D "$ISOLCPUS" ]] && { + [[ "$EXPECTED_ISOLCPUS" !=3D "$ISOLCPUS" ]] && { # Take a 50ms pause and try again pause 0.05 ISOLCPUS=3D$(cat $ISCPUS) } - [[ "$EXPECT_VAL2" !=3D "$ISOLCPUS" ]] && return 1 + [[ "$EXPECTED_ISOLCPUS" !=3D "$ISOLCPUS" ]] && return 1 ISOLCPUS=3D + EXPECTED_ISOLCPUS=3D$EXPECTED_SDOMAIN =20 # # Use the sched domain in debugfs to check isolated CPUs, if available @@ -736,7 +767,7 @@ check_isolcpus() done [[ "$ISOLCPUS" =3D *- ]] && ISOLCPUS=3D${ISOLCPUS}$LASTISOLCPU =20 - [[ "$EXPECT_VAL" =3D "$ISOLCPUS" ]] + [[ "$EXPECTED_SDOMAIN" =3D "$ISOLCPUS" ]] } =20 test_fail() @@ -773,6 +804,63 @@ null_isolcpus_check() exit 1 } =20 +# +# Check state transition test result +# $1 - Test number +# $2 - Expected effective CPU values +# $3 - Expected partition states +# $4 - Expected isolated CPUs +# +check_test_results() +{ + _NR=3D$1 + _ECPUS=3D"$2" + _PSTATES=3D"$3" + _ISOLCPUS=3D"$4" + + [[ -n "$_ECPUS" && "$_ECPUS" !=3D . ]] && { + check_effective_cpus $_ECPUS + [[ $? -ne 0 ]] && test_fail $_NR "effective CPU" \ + "Cgroup $CGRP: expected $EXPECTED_CPUS, got $ACTUAL_CPUS" + } + + [[ -n "$_PSTATES" && "$_PSTATES" !=3D . ]] && { + check_cgroup_states $_PSTATES + [[ $? -ne 0 ]] && test_fail $_NR states \ + "Cgroup $CGRP: expected $EXPECTED_STATE, got $ACTUAL_STATE" + } + + # Compare the expected isolated CPUs with the actual ones, + # if available + [[ -n "$_ISOLCPUS" ]] && { + check_isolcpus $_ISOLCPUS + [[ $? -ne 0 ]] && { + [[ -n "$BOOT_ISOLCPUS" ]] && _ISOLCPUS=3D${_ISOLCPUS},${BOOT_ISOLCPUS} + test_fail $_NR "isolated CPU" \ + "Expect $_ISOLCPUS, get $ISOLCPUS instead" + } + } + reset_cgroup_states + # + # Check to see if effective cpu list changes + # + _NEWLIST=3D$(cat $CGROUP2/cpuset.cpus.effective) + RETRY=3D0 + while [[ $_NEWLIST !=3D $CPULIST && $RETRY -lt 8 ]] + do + # Wait a bit longer & recheck a few times + pause 0.02 + ((RETRY++)) + _NEWLIST=3D$(cat $CGROUP2/cpuset.cpus.effective) + done + [[ $_NEWLIST !=3D $CPULIST ]] && { + echo "Effective cpus changed to $_NEWLIST after test $_NR!" + exit 1 + } + null_isolcpus_check + [[ $VERBOSE -gt 0 ]] && echo "Test $I done." +} + # # Run cpuset state transition test # $1 - test matrix name @@ -785,6 +873,8 @@ run_state_test() { TEST=3D$1 CONTROLLER=3Dcpuset + CGROUP_LIST=3D". A1 A1/A2 A1/A2/A3 B1" + RESET_LIST=3D"A1/A2/A3 A1/A2 A1 B1" I=3D0 eval CNT=3D"\${#$TEST[@]}" =20 @@ -824,45 +914,7 @@ run_state_test() =20 [[ $RETVAL -ne $RESULT ]] && test_fail $I result =20 - [[ -n "$ECPUS" && "$ECPUS" !=3D . ]] && { - check_effective_cpus $ECPUS - [[ $? -ne 0 ]] && test_fail $I "effective CPU" - } - - [[ -n "$STATES" && "$STATES" !=3D . ]] && { - check_cgroup_states $STATES - [[ $? -ne 0 ]] && test_fail $I states - } - - # Compare the expected isolated CPUs with the actual ones, - # if available - [[ -n "$ICPUS" ]] && { - check_isolcpus $ICPUS - [[ $? -ne 0 ]] && { - [[ -n "$BOOT_ISOLCPUS" ]] && ICPUS=3D${ICPUS},${BOOT_ISOLCPUS} - test_fail $I "isolated CPU" \ - "Expect $ICPUS, get $ISOLCPUS instead" - } - } - reset_cgroup_states - # - # Check to see if effective cpu list changes - # - NEWLIST=3D$(cat cpuset.cpus.effective) - RETRY=3D0 - while [[ $NEWLIST !=3D $CPULIST && $RETRY -lt 8 ]] - do - # Wait a bit longer & recheck a few times - pause 0.02 - ((RETRY++)) - NEWLIST=3D$(cat cpuset.cpus.effective) - done - [[ $NEWLIST !=3D $CPULIST ]] && { - echo "Effective cpus changed to $NEWLIST after test $I!" - exit 1 - } - null_isolcpus_check - [[ $VERBOSE -gt 0 ]] && echo "Test $I done." + check_test_results $I "$ECPUS" "$STATES" "$ICPUS" ((I++)) done echo "All $I tests of $TEST PASSED." @@ -932,6 +984,7 @@ test_isolated() echo $$ > $CGROUP2/cgroup.procs [[ -d A1 ]] && rmdir A1 null_isolcpus_check + pause 0.05 } =20 # @@ -997,6 +1050,8 @@ test_inotify() else echo "Inotify test PASSED" fi + echo member > cpuset.cpus.partition + echo "" > cpuset.cpus } =20 trap cleanup 0 2 3 6 --=20 2.48.1