From nobody Fri Mar 29 09:07:53 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=citrix.com Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1581593631757393.5762438318151; Thu, 13 Feb 2020 03:33:51 -0800 (PST) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1j2CkH-0006Yg-5C; Thu, 13 Feb 2020 11:33:25 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1j2CkF-0006YF-KE for xen-devel@lists.xenproject.org; Thu, 13 Feb 2020 11:33:23 +0000 Received: from esa1.hc3370-68.iphmx.com (unknown [216.71.145.142]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id a1b795a0-4e54-11ea-bc8e-bc764e2007e4; Thu, 13 Feb 2020 11:33:18 +0000 (UTC) X-Inumbo-ID: a1b795a0-4e54-11ea-bc8e-bc764e2007e4 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1581593599; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Cl2Det7wNJ45tqMxqWuUyw67At2+V5OQKY6y0cC9lTM=; b=gdy+lTECWT8IHXiaQXaEp1SEmdMz2LV1ia28sMCIQf/OUzT8IdPV+K0M jcK4EFXkORhPkKVOXGuMSDvAZoGk9RGBUpvPlYodiunwab/Nc4BeKaO0I GoWQZPe1rqhNd8RhOWEBfV4Ae6oY8jgosBdNiVVlI162WCXHUmr9wR7nf s=; Authentication-Results: esa1.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=roger.pau@citrix.com; spf=Pass smtp.mailfrom=roger.pau@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Received-SPF: None (esa1.hc3370-68.iphmx.com: no sender authenticity information available from domain of roger.pau@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa1.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa1.hc3370-68.iphmx.com: domain of roger.pau@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa1.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa1.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa1.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: YRRRyRAXXPkQumE6KIu/yRwer9wZdiHD0Za0mRUCyann5GlEdJULTAzwB4rN/9TVN5DpL5CJYK nzLuTnXrZY0XGYSixRjfvDTxXhSIu9eqNDDrWyHCbMhqhwXGz0lkEA17b1dmxSNFDyten9LI5o bQ3AX9Y/fsjD/dw7ipgX0aM+dhbuW0l2uBSe6O3UvebnkEN2DCX+PRUq4jzA1hJmgZ4TrbyZne qEZ4p+Ysq8vge1F9Sa1bnSLKr8/mSOQFnxYoGUXLD6GTVMjkL20XJNlu+HjrJUoqJL0P2G75zM 22c= X-SBRS: 2.7 X-MesageID: 12566480 X-Ironport-Server: esa1.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.70,436,1574139600"; d="scan'208";a="12566480" From: Roger Pau Monne To: Date: Thu, 13 Feb 2020 12:32:36 +0100 Message-ID: <20200213113237.58795-2-roger.pau@citrix.com> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200213113237.58795-1-roger.pau@citrix.com> References: <20200213113237.58795-1-roger.pau@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH 1/2] smp: convert the cpu maps lock into a rw lock X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Ian Jackson , Jan Beulich , Roger Pau Monne Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Most users of the cpu maps just care about the maps not changing while the lock is being held, but don't actually modify the maps. Convert the lock into a rw lock, and take the lock in read mode in get_cpu_maps and in write mode in cpu_hotplug_begin. This will lower the contention around the lock, since plug and unplug operations that take the lock in write mode are not that common. Note that the read lock can be taken recursively (as it's a shared lock), and hence will keep the same behavior as the previously used recursive lock. As for the write lock, it's only used by CPU plug/unplug operations, and the lock is never taken recursively in that case. While there also change get_cpu_maps return type to bool. Reported-by: Julien Grall Suggested-also-by: Jan Beulich Signed-off-by: Roger Pau Monn=C3=A9 Reviewed-by: Julien Grall --- xen/common/cpu.c | 22 ++++++++++++++++------ xen/include/xen/cpu.h | 13 +++---------- 2 files changed, 19 insertions(+), 16 deletions(-) diff --git a/xen/common/cpu.c b/xen/common/cpu.c index 66c855c5d9..0d7a10878c 100644 --- a/xen/common/cpu.c +++ b/xen/common/cpu.c @@ -39,26 +39,36 @@ const unsigned long cpu_bit_bitmap[BITS_PER_LONG+1][BIT= S_TO_LONGS(NR_CPUS)] =3D { #endif }; =20 -static DEFINE_SPINLOCK(cpu_add_remove_lock); +static DEFINE_RWLOCK(cpu_add_remove_lock); =20 -bool_t get_cpu_maps(void) +bool get_cpu_maps(void) { - return spin_trylock_recursive(&cpu_add_remove_lock); + return read_trylock(&cpu_add_remove_lock); } =20 void put_cpu_maps(void) { - spin_unlock_recursive(&cpu_add_remove_lock); + read_unlock(&cpu_add_remove_lock); +} + +bool cpu_hotplug_begin(void) +{ + return write_trylock(&cpu_add_remove_lock); +} + +void cpu_hotplug_done(void) +{ + write_unlock(&cpu_add_remove_lock); } =20 static NOTIFIER_HEAD(cpu_chain); =20 void __init register_cpu_notifier(struct notifier_block *nb) { - if ( !spin_trylock(&cpu_add_remove_lock) ) + if ( !write_trylock(&cpu_add_remove_lock) ) BUG(); /* Should never fail as we are called only during boot. */ notifier_chain_register(&cpu_chain, nb); - spin_unlock(&cpu_add_remove_lock); + write_unlock(&cpu_add_remove_lock); } =20 static int cpu_notifier_call_chain(unsigned int cpu, unsigned long action, diff --git a/xen/include/xen/cpu.h b/xen/include/xen/cpu.h index 2c87db26f6..e49172f64c 100644 --- a/xen/include/xen/cpu.h +++ b/xen/include/xen/cpu.h @@ -6,19 +6,12 @@ #include =20 /* Safely access cpu_online_map, cpu_present_map, etc. */ -bool_t get_cpu_maps(void); +bool get_cpu_maps(void); void put_cpu_maps(void); =20 /* Safely perform CPU hotplug and update cpu_online_map, etc. */ -static inline bool cpu_hotplug_begin(void) -{ - return get_cpu_maps(); -} - -static inline void cpu_hotplug_done(void) -{ - put_cpu_maps(); -} +bool cpu_hotplug_begin(void); +void cpu_hotplug_done(void); =20 /* Receive notification of CPU hotplug events. */ void register_cpu_notifier(struct notifier_block *nb); --=20 2.25.0 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel From nobody Fri Mar 29 09:07:53 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=citrix.com Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1581593650453446.9014825845643; Thu, 13 Feb 2020 03:34:10 -0800 (PST) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1j2Ck7-0006XM-Jl; Thu, 13 Feb 2020 11:33:15 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1j2Ck6-0006XH-DN for xen-devel@lists.xenproject.org; Thu, 13 Feb 2020 11:33:14 +0000 Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 9e69e3e4-4e54-11ea-b887-12813bfff9fa; Thu, 13 Feb 2020 11:33:13 +0000 (UTC) X-Inumbo-ID: 9e69e3e4-4e54-11ea-b887-12813bfff9fa DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1581593593; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=7kzGedqMxIqgf+xQy1JZROZ0ogbeuglGr0w6S3J4cIs=; b=D6B5knrE4nl5Rq4bBPDv62k+GIzA/rb5Xx+MsmHgBc/RSWUYMASeb3JC AP9eSlfafOok7WGKEwHMthhEUjHcaOs7Gn7LzDNSwYhY3u8/QKxo0+fOI B+52BuaIT4BJJCjEdKpYITHOv9+YSa+KGvI9PdoA81l/wAhJi9D5JpTbH I=; Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=roger.pau@citrix.com; spf=Pass smtp.mailfrom=roger.pau@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Received-SPF: None (esa6.hc3370-68.iphmx.com: no sender authenticity information available from domain of roger.pau@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa6.hc3370-68.iphmx.com: domain of roger.pau@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa6.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa6.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: 45RgNY8mVo3e0fZb9nV2NT3TnCjXdsZrO2yvsdAByore05oe7M6+Z/v06gJPEeZXyZ21ErBtUQ juvLBHvAEQZhxas34Qhpdi4u/HxMM16h0lbPaLAiFm5LT/AtwVJ5ba+s4Z0wVDALsCkAghWFKF LdKFOxx9rogG3yILxug/NRkXncnmHE9/Ysp9ANgXSPy6rvMFxxeN7kaovzktIPlZVYW28vDEvV c3JVQHrP8dGbfenXQpNtqvqDTYAxMh53wHgHwPkKD+pSmgJWqWSr1miYhWioVNJMxCtQxrdURm l3w= X-SBRS: 2.7 X-MesageID: 12823952 X-Ironport-Server: esa6.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.70,436,1574139600"; d="scan'208";a="12823952" From: Roger Pau Monne To: Date: Thu, 13 Feb 2020 12:32:37 +0100 Message-ID: <20200213113237.58795-3-roger.pau@citrix.com> X-Mailer: git-send-email 2.25.0 In-Reply-To: <20200213113237.58795-1-roger.pau@citrix.com> References: <20200213113237.58795-1-roger.pau@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH 2/2] smp: convert cpu_hotplug_begin into a blocking lock acquisition X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Stefano Stabellini , Julien Grall , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Ian Jackson , Jan Beulich , Roger Pau Monne Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Don't allow cpu_hotplug_begin to fail by converting the trylock into a blocking lock acquisition. Write users of the cpu_add_remove_lock are limited to CPU plug/unplug operations, and cannot deadlock between themselves or other users taking the lock in read mode as cpu_add_remove_lock is always locked with interrupts enabled. There are also no other locks taken during the plug/unplug operations. The exclusive lock usage in register_cpu_notifier is also converted into a blocking lock acquisition, as it was previously not allowed to fail anyway. This is meaningful when running Xen in shim mode, since VCPU_{up/down} hypercalls use cpu hotplug/unplug operations in the background, and hence failing to take the lock results in VPCU_{up/down} failing with -EBUSY, which most users are not prepared to handle. Signed-off-by: Roger Pau Monn=C3=A9 --- I've tested this and seems to work fine AFAICT either when running on native or when used in the shim. I'm not sure if I'm missing something that would prevent the write lock acquisition from being made blocking. --- xen/arch/x86/smpboot.c | 3 +-- xen/common/cpu.c | 13 +++++-------- xen/include/xen/cpu.h | 2 +- 3 files changed, 7 insertions(+), 11 deletions(-) diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c index 67ee490f94..cc0d62f9e2 100644 --- a/xen/arch/x86/smpboot.c +++ b/xen/arch/x86/smpboot.c @@ -1286,8 +1286,7 @@ int cpu_add(uint32_t apic_id, uint32_t acpi_id, uint3= 2_t pxm) (pxm >=3D 256) ) return -EINVAL; =20 - if ( !cpu_hotplug_begin() ) - return -EBUSY; + cpu_hotplug_begin(); =20 /* Detect if the cpu has been added before */ if ( x86_acpiid_to_apicid[acpi_id] !=3D BAD_APICID ) diff --git a/xen/common/cpu.c b/xen/common/cpu.c index 0d7a10878c..31953f32e4 100644 --- a/xen/common/cpu.c +++ b/xen/common/cpu.c @@ -51,9 +51,9 @@ void put_cpu_maps(void) read_unlock(&cpu_add_remove_lock); } =20 -bool cpu_hotplug_begin(void) +void cpu_hotplug_begin(void) { - return write_trylock(&cpu_add_remove_lock); + write_lock(&cpu_add_remove_lock); } =20 void cpu_hotplug_done(void) @@ -65,8 +65,7 @@ static NOTIFIER_HEAD(cpu_chain); =20 void __init register_cpu_notifier(struct notifier_block *nb) { - if ( !write_trylock(&cpu_add_remove_lock) ) - BUG(); /* Should never fail as we are called only during boot. */ + write_lock(&cpu_add_remove_lock); notifier_chain_register(&cpu_chain, nb); write_unlock(&cpu_add_remove_lock); } @@ -100,8 +99,7 @@ int cpu_down(unsigned int cpu) int err; struct notifier_block *nb =3D NULL; =20 - if ( !cpu_hotplug_begin() ) - return -EBUSY; + cpu_hotplug_begin(); =20 err =3D -EINVAL; if ( (cpu >=3D nr_cpu_ids) || (cpu =3D=3D 0) ) @@ -142,8 +140,7 @@ int cpu_up(unsigned int cpu) int err; struct notifier_block *nb =3D NULL; =20 - if ( !cpu_hotplug_begin() ) - return -EBUSY; + cpu_hotplug_begin(); =20 err =3D -EINVAL; if ( (cpu >=3D nr_cpu_ids) || !cpu_present(cpu) ) diff --git a/xen/include/xen/cpu.h b/xen/include/xen/cpu.h index e49172f64c..e8eeb217a0 100644 --- a/xen/include/xen/cpu.h +++ b/xen/include/xen/cpu.h @@ -10,7 +10,7 @@ bool get_cpu_maps(void); void put_cpu_maps(void); =20 /* Safely perform CPU hotplug and update cpu_online_map, etc. */ -bool cpu_hotplug_begin(void); +void cpu_hotplug_begin(void); void cpu_hotplug_done(void); =20 /* Receive notification of CPU hotplug events. */ --=20 2.25.0 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel