From nobody Sun Oct 5 09:04:14 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 54DF925B305 for ; Wed, 6 Aug 2025 17:24:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754501095; cv=none; b=LV94BjgT8UYNXSQfavCeBbdCF+sk0xxurFLR8P8hjZi9VWUnXiKAO7FKLPMbuOumeGz8JSLYUEvANWVD0SGAbv1tdKatJIaYmxV7DNO1n0Nc/8B4gQBV49wF5LY+HOOm/o19HCrLfX5cth9cGLLbDklWnjRg8/aQt/lPasdaPH8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754501095; c=relaxed/simple; bh=0C84Fhrf9HSphlR61yFsq5jnmsi1bukdWhhvd37/udQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=CIHHe+raOINrApDs/cswTHqH6qAS9YRxjNCYDsWaxeAkSASksnF/jn6WiWPd4eVa/mTPrgi7k8vxbWsbSYyDnEKXFTccRNAPsttMo6Alyxd6CbUrWN0OVNozdSyzWXRA3dtJuTRQRcyEZKliRNn8dfz+7aT35edjKcYiLJemq1U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=aLeUeG3x; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="aLeUeG3x" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1754501093; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Jy2+ThM5nEhHxxw4zaH5Ul3BDQX387i/DjD7CdpaNJo=; b=aLeUeG3xof2R+B8zjwUNwsf5Z18O/nwx9Ytb3BEUl7btBuGC21Y6WXdBJGXiRAVLZCWMrR AvZoRxYApI56jBc7Ozac1/fOY7euhIYPk1mwKeQ+m6JE1oRX7I7p4zUyGsb4QUot7aiFfe JR8DqCznnhzM1s4XyiO7gSAmlU6Npv4= Received: from mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-542-Xm6DUZZYNTKiJHVjAQydYQ-1; Wed, 06 Aug 2025 13:24:50 -0400 X-MC-Unique: Xm6DUZZYNTKiJHVjAQydYQ-1 X-Mimecast-MFC-AGG-ID: Xm6DUZZYNTKiJHVjAQydYQ_1754501088 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-04.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 6139519560AD; Wed, 6 Aug 2025 17:24:48 +0000 (UTC) Received: from llong-thinkpadp16vgen1.westford.csb (unknown [10.22.89.125]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 524AF30001A1; Wed, 6 Aug 2025 17:24:46 +0000 (UTC) From: Waiman Long To: Tejun Heo , Johannes Weiner , =?UTF-8?q?Michal=20Koutn=C3=BD?= Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Ingo Molnar , Juri Lelli , "Peter Zijlstra (Intel)" , Waiman Long Subject: [PATCH 1/3] cgroup/cpuset: Use static_branch_enable_cpuslocked() on cpusets_insane_config_key Date: Wed, 6 Aug 2025 13:24:28 -0400 Message-ID: <20250806172430.1155133-2-longman@redhat.com> In-Reply-To: <20250806172430.1155133-1-longman@redhat.com> References: <20250806172430.1155133-1-longman@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" The following lockdep splat was observed. [ 812.359086] =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D [ 812.359089] WARNING: possible recursive locking detected [ 812.359097] -------------------------------------------- [ 812.359100] runtest.sh/30042 is trying to acquire lock: [ 812.359105] ffffffffa7f27420 (cpu_hotplug_lock){++++}-{0:0}, at: static_= key_enable+0xe/0x20 [ 812.359131] [ 812.359131] but task is already holding lock: [ 812.359134] ffffffffa7f27420 (cpu_hotplug_lock){++++}-{0:0}, at: cpuset_= write_resmask+0x98/0xa70 : [ 812.359267] Call Trace: [ 812.359272] [ 812.359367] cpus_read_lock+0x3c/0xe0 [ 812.359382] static_key_enable+0xe/0x20 [ 812.359389] check_insane_mems_config.part.0+0x11/0x30 [ 812.359398] cpuset_write_resmask+0x9f2/0xa70 [ 812.359411] cgroup_file_write+0x1c7/0x660 [ 812.359467] kernfs_fop_write_iter+0x358/0x530 [ 812.359479] vfs_write+0xabe/0x1250 [ 812.359529] ksys_write+0xf9/0x1d0 [ 812.359558] do_syscall_64+0x5f/0xe0 Since commit d74b27d63a8b ("cgroup/cpuset: Change cpuset_rwsem and hotplug lock order"), the ordering of cpu hotplug lock and cpuset_mutex had been reversed. That patch correctly used the cpuslocked version of the static branch API to enable cpusets_pre_enable_key and cpusets_enabled_key, but it didn't do the same for cpusets_insane_config_key. The cpusets_insane_config_key can be enabled in the check_insane_mems_config() which is called from update_nodemask() or cpuset_hotplug_update_tasks() with both cpu hotplug lock and cpuset_mutex held. Deadlock can happen with a pending hotplug event that tries to acquire the cpu hotplug write lock which will block further cpus_read_lock() attempt from check_insane_mems_config(). Fix that by switching to use static_branch_enable_cpuslocked(). Fixes: d74b27d63a8b ("cgroup/cpuset: Change cpuset_rwsem and hotplug lock o= rder") Signed-off-by: Waiman Long Reviewed-by: Juri Lelli --- kernel/cgroup/cpuset.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index f74d04429a29..bf149246e001 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -280,7 +280,7 @@ static inline void check_insane_mems_config(nodemask_t = *nodes) { if (!cpusets_insane_config() && movable_only_nodes(nodes)) { - static_branch_enable(&cpusets_insane_config_key); + static_branch_enable_cpuslocked(&cpusets_insane_config_key); pr_info("Unsupported (movable nodes only) cpuset configuration detected = (nmask=3D%*pbl)!\n" "Cpuset allocations might fail even with a lot of memory available.\n", nodemask_pr_args(nodes)); --=20 2.50.0 From nobody Sun Oct 5 09:04:14 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8E53729CB4C for ; Wed, 6 Aug 2025 17:24:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754501098; cv=none; b=p4AOyfcttK38QsaJ6RSlpEEQAKSy8svHzLoEFN7VGAXExM/89SCgpgVPQhDUgVu9zJP4z/g+xyu9RJfJFhxONCwi8FNGPhE47BvBYVnIo2qiSCQPEUlalU8/+O33lNBfT74+BrWRx46bmHLyi+ITMHX8F7Gh6I73yQREW2O1o7A= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754501098; c=relaxed/simple; bh=Oj/vm0Gkw6edpCEs5FxHS4upaCO0j/bOzRv0a/F3qW8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nVSqAlFhUnI4Qp1lBddFNISbnthknSGYDSaDK6IjPUojWwniMpiF+SZvRdy21XY4xZwtIpD+4KOucCnq0FesOf1kB91Obin5zGTRIdn6uyjduIsR/wl2ENfC9y0GNWdhgtdHq2fQVEqX4THkfCelHhLqMXUI5KjC2v0yJALojlY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=a6s77UZc; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="a6s77UZc" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1754501095; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=5wrB6DGlnEetp0cy09fMeZaav/mSCyKZPlRkJ96Aoso=; b=a6s77UZcay+RL0Pw5qzkrGyDpfQzD2C4MBQGGQgfHYS5oKI+P6WqJTseUtg1m+CX+LvqJJ klUfEDkJt6dSZdSx+AGy0VK0ZUt8Lqv/V2GGDF3qEwL06PjCDEOI3Z4bu2yrcb89xPP6Lh uiAZAB1e9OSrIdtI7EUuxxaRPxzDDzg= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-684-O8SO42uDMRmlLRxO4Fthkw-1; Wed, 06 Aug 2025 13:24:52 -0400 X-MC-Unique: O8SO42uDMRmlLRxO4Fthkw-1 X-Mimecast-MFC-AGG-ID: O8SO42uDMRmlLRxO4Fthkw_1754501091 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 9AF721800357; Wed, 6 Aug 2025 17:24:50 +0000 (UTC) Received: from llong-thinkpadp16vgen1.westford.csb (unknown [10.22.89.125]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id AE33C3000198; Wed, 6 Aug 2025 17:24:48 +0000 (UTC) From: Waiman Long To: Tejun Heo , Johannes Weiner , =?UTF-8?q?Michal=20Koutn=C3=BD?= Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Ingo Molnar , Juri Lelli , "Peter Zijlstra (Intel)" , Waiman Long Subject: [PATCH 2/3] cgroup/cpuset.c: Fix a partition error with CPU hotplug Date: Wed, 6 Aug 2025 13:24:29 -0400 Message-ID: <20250806172430.1155133-3-longman@redhat.com> In-Reply-To: <20250806172430.1155133-1-longman@redhat.com> References: <20250806172430.1155133-1-longman@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" It was found during testing that an invalid leaf partition with an empty effective exclusive CPU list can become a valid empty partition with no CPU afer an offline/online operation of an unrelated CPU. An empty partition root is allowed in the special case that it has no task in its cgroup and has distributed out all its CPUs to its child partitions. That is certainly not the case here. The problem is in the cpumask_subsets() test in the hotplug case (update with no new mask) of update_parent_effective_cpumask() as it also returns true if the effective exclusive CPU list is empty. Fix that by addding the cpumask_empty() test to root out this exception case. Also add the cpumask_empty() test in cpuset_hotplug_update_tasks() to avoid calling update_parent_effective_cpumask() for this special case. Fixes: 0c7f293efc87 ("cgroup/cpuset: Add cpuset.cpus.exclusive.effective fo= r v2") Signed-off-by: Waiman Long --- kernel/cgroup/cpuset.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index bf149246e001..d993e058a663 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -1843,7 +1843,7 @@ static int update_parent_effective_cpumask(struct cpu= set *cs, int cmd, if (is_partition_valid(cs)) adding =3D cpumask_and(tmp->addmask, xcpus, parent->effective_xcpus); - } else if (is_partition_invalid(cs) && + } else if (is_partition_invalid(cs) && !cpumask_empty(xcpus) && cpumask_subset(xcpus, parent->effective_xcpus)) { struct cgroup_subsys_state *css; struct cpuset *child; @@ -3870,9 +3870,10 @@ static void cpuset_hotplug_update_tasks(struct cpuse= t *cs, struct tmpmasks *tmp) partcmd =3D partcmd_invalidate; /* * On the other hand, an invalid partition root may be transitioned - * back to a regular one. + * back to a regular one with a non-empty effective xcpus. */ - else if (is_partition_valid(parent) && is_partition_invalid(cs)) + else if (is_partition_valid(parent) && is_partition_invalid(cs) && + !cpumask_empty(cs->effective_xcpus)) partcmd =3D partcmd_update; =20 if (partcmd >=3D 0) { --=20 2.50.0 From nobody Sun Oct 5 09:04:14 2025 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7275229E0EC for ; Wed, 6 Aug 2025 17:24:58 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754501100; cv=none; b=FdkNvuB1ExXyfBP1icMxKmEwYd7rlSatabohoDanBUU4+Fi5ZlUkoxJaAxVx/NjnC0zDwjpnk2VhvuQsuNhAEkyZ5AGCZD0wh8T3IgH2hIGgWosWbDVQwasx9ne+mcJsdxydpY9b2vx14Ckhdaf+JsqW9myonuAG/LbJkqclef8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1754501100; c=relaxed/simple; bh=0Wm3Xoa5XLpfnubHs0aZbiFooFp3j7Dx1L7hNmkFDnI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=X9xkNuQA6JOTeiYXh95L0H9E7eDCRZq5I/CYQEvQojThl6SMnZC1+NpsnSUdILqFmi47ZkmwWI99jyk0sCjwZwhlW7QP7Q68CWuvZMgHcG/vbCXOAcWN1tI+Gen8MgKxpGEZ/Xndlik+BRJrE5wdAHNCLVHLfsBkVPEVQlNt6wI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=W+hzPkmO; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="W+hzPkmO" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1754501097; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sbFZQJRVLoxgj3cfMwEHUAeK89Crfm/u3SMh3N/Z9KU=; b=W+hzPkmO4dktOS0/fBiWcGfzeNZvmeBUHk/GOOtTLQoA++lqf9MW8vC8iQHnALqxRaXymY gN1ooa/kTqQoGwb89EEwdHwo57axBZbKvSCjZdY76wChxInJ2HCqLkHttN5KAvTiar552N nZ9SPgK/qNi3A+Ru/levAQVCbCg1Pas= Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-578-QyY7T9WjNmea-5KmOlD25w-1; Wed, 06 Aug 2025 13:24:54 -0400 X-MC-Unique: QyY7T9WjNmea-5KmOlD25w-1 X-Mimecast-MFC-AGG-ID: QyY7T9WjNmea-5KmOlD25w_1754501093 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id B187218002C6; Wed, 6 Aug 2025 17:24:52 +0000 (UTC) Received: from llong-thinkpadp16vgen1.westford.csb (unknown [10.22.89.125]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id E329B30001A6; Wed, 6 Aug 2025 17:24:50 +0000 (UTC) From: Waiman Long To: Tejun Heo , Johannes Weiner , =?UTF-8?q?Michal=20Koutn=C3=BD?= Cc: cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Ingo Molnar , Juri Lelli , "Peter Zijlstra (Intel)" , Waiman Long Subject: [PATCH 3/3] cgroup/cpuset: Remove the unnecessary css_get/put() in cpuset_partition_write() Date: Wed, 6 Aug 2025 13:24:30 -0400 Message-ID: <20250806172430.1155133-4-longman@redhat.com> In-Reply-To: <20250806172430.1155133-1-longman@redhat.com> References: <20250806172430.1155133-1-longman@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Content-Type: text/plain; charset="utf-8" The css_get/put() calls in cpuset_partition_write() are unnecessary as an active reference of the kernfs node will be taken which will prevent its removal and guarantee the existence of the css. Only the online check is needed. Signed-off-by: Waiman Long Reviewed-by: Michal Koutn=C3=BD --- kernel/cgroup/cpuset.c | 2 -- 1 file changed, 2 deletions(-) diff --git a/kernel/cgroup/cpuset.c b/kernel/cgroup/cpuset.c index d993e058a663..27adb04df675 100644 --- a/kernel/cgroup/cpuset.c +++ b/kernel/cgroup/cpuset.c @@ -3358,14 +3358,12 @@ static ssize_t cpuset_partition_write(struct kernfs= _open_file *of, char *buf, else return -EINVAL; =20 - css_get(&cs->css); cpus_read_lock(); mutex_lock(&cpuset_mutex); if (is_cpuset_online(cs)) retval =3D update_prstate(cs, val); mutex_unlock(&cpuset_mutex); cpus_read_unlock(); - css_put(&cs->css); return retval ?: nbytes; } =20 --=20 2.50.0