From nobody Mon May 6 10:33:05 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail(p=none dis=none) header.from=citrix.com ARC-Seal: i=1; a=rsa-sha256; t=1577191575; cv=none; d=zohomail.com; s=zohoarc; b=eKNfOtM6p8UXXlX0PARq76RlpgdCoAE8F52yLhli9x9YdKEAWIDUqtFxoE9DgfM3iveiBcdb6bvLpdCrqCLmeHk+BVouBZgcqJM1lRoPaZDYIocliMyAz/ckRtmbQe4kjPSwtsn7ObEtj2UoDtefsapoIADzCuUH/A3s7Pbcwxo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1577191575; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:List-Subscribe:List-Post:List-Id:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Sender:Subject:To; bh=dh8D9sXp0w9Y2q0cmkvlrkcwszb2lADJQLz7Noubzws=; b=ZRqI359FeRGh5mxRS8bKewVBZjPf+ef8W2cWN5Ukb82gdBDyarCRC2AQINW5mhMWQZ6Cafv6fJaPgyw+uD3QqtYk707dr2nK96uWRECrMYChveUAQ8e8lpn0Bgacg8D6I3HSkBKCAML1143PXtIn8p82z0jj+RAHwXkv3GIF+GU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) smtp.mailfrom=xen-devel-bounces@lists.xenproject.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) by mx.zohomail.com with SMTPS id 1577191575000441.07112095460946; Tue, 24 Dec 2019 04:46:15 -0800 (PST) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ijjZ3-0003Xb-Bq; Tue, 24 Dec 2019 12:45:29 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1ijjZ1-0003XW-W5 for xen-devel@lists.xenproject.org; Tue, 24 Dec 2019 12:45:28 +0000 Received: from esa2.hc3370-68.iphmx.com (unknown [216.71.145.153]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 3d880c26-264b-11ea-a1e1-bc764e2007e4; Tue, 24 Dec 2019 12:45:19 +0000 (UTC) X-Inumbo-ID: 3d880c26-264b-11ea-a1e1-bc764e2007e4 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1577191520; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=RABxJAD09Xcvfv32cg3ovNKRC/mOqJQa6CUEM2fEOtw=; b=iKlHXETjP0aDMuf/q+ao8pMRwBfBGrjM1aGIZ1A8tbETfnxModwcWDGw vXweKbop69jJbi+ckqmtDr1JFPMTCeGAHls1dXc67yeaKzTpt8YrPWxEE y9mUi3l6VCHTvNS+70x8Zbj9XUynAKAeArTMTiSOkJ9xuBzLbOayuW65B 0=; Authentication-Results: esa2.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=roger.pau@citrix.com; spf=Pass smtp.mailfrom=roger.pau@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: none (zohomail.com: 192.237.175.120 is neither permitted nor denied by domain of lists.xenproject.org) client-ip=192.237.175.120; envelope-from=xen-devel-bounces@lists.xenproject.org; helo=lists.xenproject.org; Received-SPF: None (esa2.hc3370-68.iphmx.com: no sender authenticity information available from domain of roger.pau@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa2.hc3370-68.iphmx.com: domain of roger.pau@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="roger.pau@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ip4:168.245.78.127 ~all" Received-SPF: None (esa2.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa2.hc3370-68.iphmx.com; envelope-from="roger.pau@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: rqMozP5ekMuFiJXrxy9M+BCZSgZbcdW/jlyyvv+BlgLmgffSi3ZZS/b+togxw/ccR7jXdYUlKa FiwAgQB/aCeRyr2Gu1jzh78W25K1PtB0at0FlFKACFTBc4PfWjJ51hjA/oD3ArpZg8g6rDWiPm +nM3hVjl8Ft9EO5janvOroONGUTeLgFVM7dnjEJIRJrG7xm5Z7/Eqbg+ylCFvCi/HA5wlEHuZ8 7imZH1vIwgx3upv1d39FIIkYPHSNG/8xZI5TCU3vl9glMXK1LoOjcvV/8lfKAa2Vea8hKfK7y5 N54= X-SBRS: 2.7 X-MesageID: 10123062 X-Ironport-Server: esa2.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.69,351,1571716800"; d="scan'208";a="10123062" From: Roger Pau Monne To: Date: Tue, 24 Dec 2019 13:44:53 +0100 Message-ID: <20191224124453.47183-1-roger.pau@citrix.com> X-Mailer: git-send-email 2.24.1 MIME-Version: 1.0 Subject: [Xen-devel] [PATCH] x86/flush: use APIC ALLBUT destination shorthand when possible X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Andrew Cooper , Wei Liu , Jan Beulich , Roger Pau Monne Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) If the flush mask matches the mask of online CPUs use the APIC ALLBUT destination shorthand in order to send an IPI to all CPUs on the system except the current one. This can only be safely used when no CPU hotplug or unplug operations are taking place, no offline CPUs or those have been onlined and parked and finally when all CPUs in the system have been accounted for (ie: the number of CPUs doesn't exceed NR_CPUS and APIC IDs are below MAX_APICS). This is specially beneficial when using the PV shim, since using the shorthand avoids performing an APIC register write (or multiple ones if using xAPIC mode) for each destination in the flush mask. The lock time on a 32 vCPU guest using the shim without the shorthand is: Global lock flush_lock: addr=3Dffff82d0804b21c0, lockval=3Df602f602, not lo= cked lock:228455938(79406065573135), block:205908580(556416605761539) Average lock time: 347577ns While the same guest using the shorthand: Global lock flush_lock: addr=3Dffff82d0804b41c0, lockval=3Dd9c4d9bc, cpu=3D= 12 lock:1890775(416719148054), block:1663958(2500161282949) Average lock time: 220395ns Approximately a 1/3 improvement in the lock time. Signed-off-by: Roger Pau Monn=C3=A9 --- xen/arch/x86/acpi/boot.c | 1 + xen/arch/x86/mpparse.c | 5 +++++ xen/arch/x86/smp.c | 41 ++++++++++++++++++++++++++++++++++++++- xen/include/asm-x86/smp.h | 2 ++ 4 files changed, 48 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/acpi/boot.c b/xen/arch/x86/acpi/boot.c index 15542a9bdf..88e1a89ff0 100644 --- a/xen/arch/x86/acpi/boot.c +++ b/xen/arch/x86/acpi/boot.c @@ -103,6 +103,7 @@ acpi_parse_x2apic(struct acpi_subtable_header *header, = const unsigned long end) processor->lapic_flags & ACPI_MADT_ENABLED ? KERN_WARNING "WARNING: " : KERN_INFO, processor->local_apic_id, processor->uid); + cpu_overflow =3D true; /* * Must not return an error here, to prevent * acpi_table_parse_entries() from terminating early. diff --git a/xen/arch/x86/mpparse.c b/xen/arch/x86/mpparse.c index f057d9162f..8d7739fbf4 100644 --- a/xen/arch/x86/mpparse.c +++ b/xen/arch/x86/mpparse.c @@ -66,6 +66,9 @@ static unsigned int __initdata disabled_cpus; /* Bitmask of physically existing CPUs */ physid_mask_t phys_cpu_present_map; =20 +/* Record whether CPUs haven't been added due to overflows. */ +bool __read_mostly cpu_overflow; + void __init set_nr_cpu_ids(unsigned int max_cpus) { unsigned int tot_cpus =3D num_processors + disabled_cpus; @@ -160,6 +163,7 @@ static int MP_processor_info_x(struct mpc_config_proces= sor *m, printk_once(XENLOG_WARNING "WARNING: NR_CPUS limit of %u reached - ignoring further processors= \n", nr_cpu_ids); + cpu_overflow =3D true; return -ENOSPC; } =20 @@ -167,6 +171,7 @@ static int MP_processor_info_x(struct mpc_config_proces= sor *m, && genapic.name =3D=3D apic_default.name) { printk_once(XENLOG_WARNING "WARNING: CPUs limit of 8 reached - ignoring futher processors\n"); + cpu_overflow =3D true; return -ENOSPC; } =20 diff --git a/xen/arch/x86/smp.c b/xen/arch/x86/smp.c index 6fb39a0a24..427c33db9d 100644 --- a/xen/arch/x86/smp.c +++ b/xen/arch/x86/smp.c @@ -8,6 +8,7 @@ * later. */ =20 +#include #include #include #include @@ -123,6 +124,11 @@ void send_IPI_self_legacy(uint8_t vector) __default_send_IPI_shortcut(APIC_DEST_SELF, vector, APIC_DEST_PHYSICAL= ); } =20 +static void send_IPI_allbutself(unsigned int vector) +{ + __default_send_IPI_shortcut(APIC_DEST_ALLBUT, vector, APIC_DEST_PHYSIC= AL); +} + void send_IPI_mask_flat(const cpumask_t *cpumask, int vector) { unsigned long mask =3D cpumask_bits(cpumask)[0]; @@ -227,14 +233,47 @@ void flush_area_mask(const cpumask_t *mask, const voi= d *va, unsigned int flags) if ( (flags & ~FLUSH_ORDER_MASK) && !cpumask_subset(mask, cpumask_of(cpu)) ) { + bool cpus_locked =3D false; + spin_lock(&flush_lock); cpumask_and(&flush_cpumask, mask, &cpu_online_map); cpumask_clear_cpu(cpu, &flush_cpumask); flush_va =3D va; flush_flags =3D flags; - send_IPI_mask(&flush_cpumask, INVALIDATE_TLB_VECTOR); + + /* + * Prevent any CPU hot{un}plug while sending the IPIs if we are to= use + * a shorthand, also refuse to use a shorthand if not all CPUs are + * online or have been parked. + */ + if ( system_state > SYS_STATE_smp_boot && !cpu_overflow && + (cpus_locked =3D get_cpu_maps()) && + (park_offline_cpus || + cpumask_equal(&cpu_online_map, &cpu_present_map)) ) + { + cpumask_copy(this_cpu(scratch_cpumask), &cpu_online_map); + cpumask_clear_cpu(cpu, this_cpu(scratch_cpumask)); + } + else + { + if ( cpus_locked ) + { + put_cpu_maps(); + cpus_locked =3D false; + } + cpumask_clear(this_cpu(scratch_cpumask)); + } + + if ( cpumask_equal(&flush_cpumask, this_cpu(scratch_cpumask)) ) + send_IPI_allbutself(INVALIDATE_TLB_VECTOR); + else + send_IPI_mask(&flush_cpumask, INVALIDATE_TLB_VECTOR); + while ( !cpumask_empty(&flush_cpumask) ) cpu_relax(); + + if ( cpus_locked ) + put_cpu_maps(); spin_unlock(&flush_lock); } } diff --git a/xen/include/asm-x86/smp.h b/xen/include/asm-x86/smp.h index dbeed2fd41..3df4185744 100644 --- a/xen/include/asm-x86/smp.h +++ b/xen/include/asm-x86/smp.h @@ -84,6 +84,8 @@ extern cpumask_t **socket_cpumask; #define get_cpu_current(cpu) \ (get_cpu_info_from_stack((unsigned long)stack_base[cpu])->current_vcpu) =20 +extern bool cpu_overflow; + #endif /* !__ASSEMBLY__ */ =20 #endif --=20 2.24.1 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel