From nobody Fri Apr 26 14:50:44 2024 Delivered-To: importer@patchew.org Received-SPF: pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) client-ip=66.175.222.12; envelope-from=bounce+27952+63454+1787277+3901457@groups.io; helo=web01.groups.io; Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+63454+1787277+3901457@groups.io; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1596048751; cv=none; d=zohomail.com; s=zohoarc; b=ms2QFt6qcsOGyBsLXBOOyYBSiQSQ1c6Z8C4eS/Dq0ECMusIpmdETsm148v0NSw+CWeWadDjREcZNHGtl8ZqnBd18kzQmy5Q5ZOvP27MXFm0bzosHcROca7E0FGYDpO3RcmUoj9hT4oWzn0PHaebSR/y1oTBbVsC7DxTQ9GKAbZc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1596048751; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:List-Id:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Sender:Subject:To; bh=01a9H+dsfZ0SXAWj9Ysh3HArtR3qe+CKe9+bXUPZP6g=; b=EFrS3VUemTx33vr/ENJEL7eBYdG7UQKihFwV3oVmAZ3HDdqrv4QJJixr9Vq+jzkQ9FNzVohi+uSTfjpBBdytk58WTqKN+WGS2HsSV7rRGU5JuXjdLLLs9MNbwqj5FUulbTPYWw/dQsj/UNoiYZuFCrmuDqZGbJ4fZxEWAOGemcM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of groups.io designates 66.175.222.12 as permitted sender) smtp.mailfrom=bounce+27952+63454+1787277+3901457@groups.io; dmarc=fail header.from= (p=none dis=none) header.from= Received: from web01.groups.io (web01.groups.io [66.175.222.12]) by mx.zohomail.com with SMTPS id 1596048751655271.12390120678765; Wed, 29 Jul 2020 11:52:31 -0700 (PDT) Return-Path: X-Received: by 127.0.0.2 with SMTP id qPehYY1788612x3KEHt86hLz; Wed, 29 Jul 2020 11:52:31 -0700 X-Received: from us-smtp-1.mimecast.com (us-smtp-1.mimecast.com [205.139.110.120]) by mx.groups.io with SMTP id smtpd.web10.3165.1596048750558391097 for ; Wed, 29 Jul 2020 11:52:30 -0700 X-Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-240-PR4-7nMQMS-ST3cvtk2Ojw-1; Wed, 29 Jul 2020 14:52:21 -0400 X-MC-Unique: PR4-7nMQMS-ST3cvtk2Ojw-1 X-Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 547AD800685; Wed, 29 Jul 2020 18:52:20 +0000 (UTC) X-Received: from lacos-laptop-7.usersys.redhat.com (ovpn-114-234.ams2.redhat.com [10.36.114.234]) by smtp.corp.redhat.com (Postfix) with ESMTP id 1A23C1C6; Wed, 29 Jul 2020 18:52:18 +0000 (UTC) From: "Laszlo Ersek" To: edk2-devel-groups-io Cc: Eric Dong , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= , Rahul Kumar , Ray Ni Subject: [edk2-devel] [PATCH] UefiCpuPkg/PiSmmCpuDxeSmm: pause in WaitForSemaphore() before re-fetch Date: Wed, 29 Jul 2020 20:52:17 +0200 Message-Id: <20200729185217.10084-1-lersek@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Precedence: Bulk List-Unsubscribe: Sender: devel@edk2.groups.io List-Id: Mailing-List: list devel@edk2.groups.io; contact devel+owner@edk2.groups.io Reply-To: devel@edk2.groups.io,lersek@redhat.com X-Gm-Message-State: Vwpvtpp75xGhBcC5GfxhJMmux1787277AA= Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=groups.io; q=dns/txt; s=20140610; t=1596048751; bh=01a9H+dsfZ0SXAWj9Ysh3HArtR3qe+CKe9+bXUPZP6g=; h=Cc:Content-Type:Date:From:Reply-To:Subject:To; b=gF26iXFuVql53W5433g2Pu6EIkwBXIAuf9J4VtX6v1XDsfnNW2pTgTHn69ipO4QOwFP sFmQFwAxUgnpIjIGAmYBAla0rr7QeFf0pplxeG7o0l1D/oztlR1q950W2QCCnxyIzpu7i dKj6WO1yplx1Lhqy0/bS7N+S65e0Y7T+wMk= X-ZohoMail-DKIM: pass (identity @groups.io) Content-Type: text/plain; charset="utf-8" Most busy waits (spinlocks) in "UefiCpuPkg/PiSmmCpuDxeSmm/MpService.c" already call CpuPause() in their loop bodies; see SmmWaitForApArrival(), APHandler(), and SmiRendezvous(). However, the "main wait" within APHandler(): > // > // Wait for something to happen > // > WaitForSemaphore (mSmmMpSyncData->CpuData[CpuIndex].Run); doesn't do so, as WaitForSemaphore() keeps trying to acquire the semaphore without pausing. The performance impact is especially notable in QEMU/KVM + OVMF virtualization with CPU overcommit (that is, when the guest has significantly more VCPUs than the host has physical CPUs). The guest BSP is working heavily in: BSPHandler() [MpService.c] PerformRemainingTasks() [PiSmmCpuDxeSmm.c] SetUefiMemMapAttributes() [SmmCpuMemoryManagement.c] while the many guest APs are spinning in the "Wait for something to happen" semaphore acquisition, in APHandler(). The guest APs are generating useless memory traffic and saturating host CPUs, hindering the guest BSP's progress in SetUefiMemMapAttributes(). Rework the loop in WaitForSemaphore(): call CpuPause() in every iteration after the first check fails. Due to Pause Loop Exiting (known as Pause Filter on AMD), the host scheduler can favor the guest BSP over the guest APs. Running a 16 GB RAM + 512 VCPU guest on a 448 PCPU host, this patch reduces OVMF boot time (counted until reaching grub) from 20-30 minutes to less than 4 minutes. The patch should benefit physical machines as well -- according to the Intel SDM, PAUSE "Improves the performance of spin-wait loops". Adding PAUSE to the generic WaitForSemaphore() function is considered a general improvement. Cc: Eric Dong Cc: Philippe Mathieu-Daud=C3=A9 Cc: Rahul Kumar Cc: Ray Ni Ref: https://bugzilla.redhat.com/show_bug.cgi?id=3D1861718 Signed-off-by: Laszlo Ersek Reviewed-by: Eric Dong --- Notes: Repo: https://pagure.io/lersek/edk2.git Branch: sem_wait_pause_rhbz1861718 UefiCpuPkg/PiSmmCpuDxeSmm/MpService.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/UefiCpuPkg/PiSmmCpuDxeSmm/MpService.c b/UefiCpuPkg/PiSmmCpuDxe= Smm/MpService.c index 57e788c01b1f..4bcd217917d7 100644 --- a/UefiCpuPkg/PiSmmCpuDxeSmm/MpService.c +++ b/UefiCpuPkg/PiSmmCpuDxeSmm/MpService.c @@ -40,14 +40,18 @@ WaitForSemaphore ( { UINT32 Value; =20 - do { + for (;;) { Value =3D *Sem; - } while (Value =3D=3D 0 || - InterlockedCompareExchange32 ( - (UINT32*)Sem, - Value, - Value - 1 - ) !=3D Value); + if (Value !=3D 0 && + InterlockedCompareExchange32 ( + (UINT32*)Sem, + Value, + Value - 1 + ) =3D=3D Value) { + break; + } + CpuPause (); + } return Value - 1; } =20 --=20 2.19.1.3.g30247aa5d201 -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#63454): https://edk2.groups.io/g/devel/message/63454 Mute This Topic: https://groups.io/mt/75871294/1787277 Group Owner: devel+owner@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [importer@patchew.org] -=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-=3D-