From nobody Sat Nov 15 06:34:12 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1753885335; cv=none; d=zohomail.com; s=zohoarc; b=hN0zkXt8ApHzbz/vOdqmoCHDI7Gchi196I1sxcIxqKd6222LEV8WYFT+oCNu2ZQ+KIejb+MrYyCAn2YJMiUC+YfHkpBJQrhIPEBbn6pr3kiK/ncKzY0XeDXXIe9FUsTxk2KKRwaPd9P5z5HtPTD1nGtEAWUy+JGJLsLPG+1ONMw= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1753885335; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=6Y2ZCOhZ6YDQ0sgMGEW3nvxIQlwrgOR5ACfG/ksuE6E=; b=I6IYN0V0C4oEL/AegHuf5xa72/+0dS1iWqWKGzAWg5As7WA4DLGRjCkJ20AJwNhOtTxaw3z66XQAh9NMvCH92uR6ZKRbf5SpgXFAqYe/kt6Fd3LINZTkw6pkSXcqNe6aBtyjNvDtUvFjZPcLML9J7bVP513uf6Ci/bGgbCIUxtE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1753885334997923.8192988850561; Wed, 30 Jul 2025 07:22:14 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uh7gS-0007MC-V4; Wed, 30 Jul 2025 10:21:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uh663-0005ce-G3 for qemu-devel@nongnu.org; Wed, 30 Jul 2025 08:39:54 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uh660-0006tb-5K for qemu-devel@nongnu.org; Wed, 30 Jul 2025 08:39:50 -0400 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-643-24L8v2TbMMCZqtIdqbSKKg-1; Wed, 30 Jul 2025 08:39:43 -0400 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 25908180045B; Wed, 30 Jul 2025 12:39:42 +0000 (UTC) Received: from dell-r430-03.lab.eng.brq2.redhat.com (dell-r430-03.lab.eng.brq2.redhat.com [10.37.153.18]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 1E4B630001B1; Wed, 30 Jul 2025 12:39:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1753879186; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=6Y2ZCOhZ6YDQ0sgMGEW3nvxIQlwrgOR5ACfG/ksuE6E=; b=Ty7ACKsbcUQibbgiqDyzs7i6XsTKCKa2hwtS3VH3OdcntDGNXMebZelNo/5Z1iOJHNTj/I cRHDzHCajmBVd7faQ1zJ4Irb/YImsxvEHH4A8i8ZFoven1nzHr+R6iftYBsh5V20JQsrMA Qrn2DtT1RAV7UtgqhpNFp0L+QZ8eN3A= X-MC-Unique: 24L8v2TbMMCZqtIdqbSKKg-1 X-Mimecast-MFC-AGG-ID: 24L8v2TbMMCZqtIdqbSKKg_1753879182 From: Igor Mammedov To: qemu-devel@nongnu.org Cc: pbonzini@redhat.com, peterx@redhat.com, mst@redhat.com, mtosatti@redhat.com, kraxel@redhat.com, peter.maydell@linaro.org Subject: [PATCH v2 1/6] memory: reintroduce BQL-free fine-grained PIO/MMIO Date: Wed, 30 Jul 2025 14:39:29 +0200 Message-ID: <20250730123934.1787379-2-imammedo@redhat.com> In-Reply-To: <20250730123934.1787379-1-imammedo@redhat.com> References: <20250730123934.1787379-1-imammedo@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=imammedo@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1753885335992116600 This patch brings back Jan's idea [1] of BQL-free IO access This will let us make access to ACPI PM/HPET timers cheaper, and prevent BQL contention in case of workload that heavily uses the timers with a lot of vCPUs. 1) 196ea13104f (memory: Add global-locking property to memory regions) ... de7ea885c539 (kvm: Switch to unlocked MMIO) Signed-off-by: Igor Mammedov --- include/system/memory.h | 10 ++++++++++ system/memory.c | 6 ++++++ system/physmem.c | 2 +- 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/include/system/memory.h b/include/system/memory.h index e2cd6ed126..d04366c994 100644 --- a/include/system/memory.h +++ b/include/system/memory.h @@ -833,6 +833,7 @@ struct MemoryRegion { bool nonvolatile; bool rom_device; bool flush_coalesced_mmio; + bool lockless_io; bool unmergeable; uint8_t dirty_log_mask; bool is_iommu; @@ -2341,6 +2342,15 @@ void memory_region_set_flush_coalesced(MemoryRegion = *mr); */ void memory_region_clear_flush_coalesced(MemoryRegion *mr); =20 +/** + * memory_region_enable_lockless_io: Enable lockless (BQL free) acceess. + * + * Enable BQL-free access for devices with fine-grained locking. + * + * @mr: the memory region to be updated. + */ +void memory_region_enable_lockless_io(MemoryRegion *mr); + /** * memory_region_add_eventfd: Request an eventfd to be triggered when a wo= rd * is written to a location. diff --git a/system/memory.c b/system/memory.c index 5646547940..9a5a262112 100644 --- a/system/memory.c +++ b/system/memory.c @@ -2546,6 +2546,12 @@ void memory_region_clear_flush_coalesced(MemoryRegio= n *mr) } } =20 +void memory_region_enable_lockless_io(MemoryRegion *mr) +{ + mr->lockless_io =3D true; + mr->disable_reentrancy_guard =3D true; +} + void memory_region_add_eventfd(MemoryRegion *mr, hwaddr addr, unsigned size, diff --git a/system/physmem.c b/system/physmem.c index 130c148ffb..107871e2b3 100644 --- a/system/physmem.c +++ b/system/physmem.c @@ -2909,7 +2909,7 @@ bool prepare_mmio_access(MemoryRegion *mr) { bool release_lock =3D false; =20 - if (!bql_locked()) { + if (!bql_locked() && !mr->lockless_io) { bql_lock(); release_lock =3D true; } --=20 2.47.1 From nobody Sat Nov 15 06:34:12 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1753884865; cv=none; d=zohomail.com; s=zohoarc; b=ZMu5xWnV/nMYabf3bdl/yN3CZ/5XZqn2rM2t0pCfRQUTg+dx0NyfDDn0Wf6oefp6NqGly/fx1JunUnpezX+mWqKN/sEic3asoUrsi1gRqL3O51WV5L8fAx34WvbEA+dpCuoSSzeCokvytoeJgrAf+awUzZMPOu3RvmIYF2D3+ys= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1753884865; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=VRDAzl4q6o2ZOj98CGNsHZ+fXsoqdX1zXUVWLEyK8NA=; b=GCCQQvSYBzX38KRlZHM57nkRJ8XEl1CZsoXcrjHMhhtEfbHgcNWyLr4CcZu75yKuuyBgIXmURt40OgtRVeVi534Qft+vP+YGXJ+n4bsskLeyQh5cY9UtJqh6TdMwfelej/wcVjR4CxjDLoa8hPDg+9uYYxZsx2nt8up8CjdljIQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1753884865050820.5553003185912; Wed, 30 Jul 2025 07:14:25 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uh7Z0-0006HY-HN; Wed, 30 Jul 2025 10:13:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uh663-0005cf-Gk for qemu-devel@nongnu.org; Wed, 30 Jul 2025 08:39:53 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uh660-0006tl-HD for qemu-devel@nongnu.org; Wed, 30 Jul 2025 08:39:50 -0400 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-449-2OKdExiFP_aPweoNyNZmPA-1; Wed, 30 Jul 2025 08:39:45 -0400 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 9D2AC1956080; Wed, 30 Jul 2025 12:39:44 +0000 (UTC) Received: from dell-r430-03.lab.eng.brq2.redhat.com (dell-r430-03.lab.eng.brq2.redhat.com [10.37.153.18]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 8CB7330001BE; Wed, 30 Jul 2025 12:39:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1753879187; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VRDAzl4q6o2ZOj98CGNsHZ+fXsoqdX1zXUVWLEyK8NA=; b=Q7hnKKAm67WnFj0GqMbYN39g9jdVnctD5lb0jJ60EsDHcds6ea0jNsHQBipV4WrEcUP2v8 lOVizvY2iZORYpV0eznab1YEdmtKdGVJOSa++yDY8ET5adOp7y35X3YeZpD2hqgOPKY8a4 43fdnA8y5GheRIChD08Fa32RpNgkJ/4= X-MC-Unique: 2OKdExiFP_aPweoNyNZmPA-1 X-Mimecast-MFC-AGG-ID: 2OKdExiFP_aPweoNyNZmPA_1753879184 From: Igor Mammedov To: qemu-devel@nongnu.org Cc: pbonzini@redhat.com, peterx@redhat.com, mst@redhat.com, mtosatti@redhat.com, kraxel@redhat.com, peter.maydell@linaro.org Subject: [PATCH v2 2/6] acpi: mark PMTIMER as unlocked Date: Wed, 30 Jul 2025 14:39:30 +0200 Message-ID: <20250730123934.1787379-3-imammedo@redhat.com> In-Reply-To: <20250730123934.1787379-1-imammedo@redhat.com> References: <20250730123934.1787379-1-imammedo@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=imammedo@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 12 X-Spam_score: 1.2 X-Spam_bar: + X-Spam_report: (1.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1753884867655116600 Reading QEMU_CLOCK_VIRTUAL is thread-safe, write access is NOP. This makes possible to boot Windows with large vCPUs count when hv-time is not used. Reproducer: -M q35,hpet=3Doff -cpu host -enable-kvm -smp 240,sockets=3D4 -m 8G WS2025= .img fails to boot within 30min. With this fix it boots within 2-1min. Signed-off-by: Igor Mammedov --- hw/acpi/core.c | 1 + 1 file changed, 1 insertion(+) diff --git a/hw/acpi/core.c b/hw/acpi/core.c index 58f8964e13..ff16582803 100644 --- a/hw/acpi/core.c +++ b/hw/acpi/core.c @@ -547,6 +547,7 @@ void acpi_pm_tmr_init(ACPIREGS *ar, acpi_update_sci_fn = update_sci, ar->tmr.timer =3D timer_new_ns(QEMU_CLOCK_VIRTUAL, acpi_pm_tmr_timer, = ar); memory_region_init_io(&ar->tmr.io, memory_region_owner(parent), &acpi_pm_tmr_ops, ar, "acpi-tmr", 4); + memory_region_enable_lockless_io(&ar->tmr.io); memory_region_add_subregion(parent, 8, &ar->tmr.io); } =20 --=20 2.47.1 From nobody Sat Nov 15 06:34:12 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1753884701; cv=none; d=zohomail.com; s=zohoarc; b=azYKb9XknzuVVBk0YH6JUZf3V88NO5v9EbUZB00LGNt07BnlFHOakdr5XUY9GYdF/dnnwFnYVmlEsgR67sCE1XQy7X3CwobxMa6phgybw54/TNvLbozz1LO9D2oFM4gEr5nhFDMbhHZhKjSTN4wuiLRZRpF/u9QLF6aTWvUB16c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1753884701; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=YXMLdRLh9YwCZzYtukXOYEZwJEsXhn/JEVu/46+KzWE=; b=Z2yHgaXCY9NFeSmjoishJzTX3tlwFlxPcYV3AUKDiJhnu50fFr6Osy1/EXpa3crHReco36H1mZFIAp2d+WBAa61K/wiGZe8IUjlkvOm5Qz7vLnkTmICr6RLhgNo4mGqpT65qzzECfWaD0aYh9dgJ5yd71ASKig9sIitFd2J4IIQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1753884701518490.5789194790275; Wed, 30 Jul 2025 07:11:41 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uh7U8-000466-3X; Wed, 30 Jul 2025 10:08:48 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uh667-0005dp-LE for qemu-devel@nongnu.org; Wed, 30 Jul 2025 08:39:59 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uh663-0006uG-Vm for qemu-devel@nongnu.org; Wed, 30 Jul 2025 08:39:55 -0400 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-617-tAbn67rtOTGUZ6tHA2NjaA-1; Wed, 30 Jul 2025 08:39:48 -0400 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8E15A180045C; Wed, 30 Jul 2025 12:39:47 +0000 (UTC) Received: from dell-r430-03.lab.eng.brq2.redhat.com (dell-r430-03.lab.eng.brq2.redhat.com [10.37.153.18]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 3DC2930001B1; Wed, 30 Jul 2025 12:39:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1753879190; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YXMLdRLh9YwCZzYtukXOYEZwJEsXhn/JEVu/46+KzWE=; b=AEgH6MGAFVOHekHxgRYOwDPU9GzCeMqE50gPC5CvVp9G/ZcZr3XYFds4/GQioLS713xVUS aJNvsK4IIP22VdE1L+uiv1p0Ci5EKUlTVNlrcLoR9x7SCsfFunbKkjnHLTE45EayVsVoVN q4SGE0awytPUsjR3RmN+f6LmUDOuc2I= X-MC-Unique: tAbn67rtOTGUZ6tHA2NjaA-1 X-Mimecast-MFC-AGG-ID: tAbn67rtOTGUZ6tHA2NjaA_1753879187 From: Igor Mammedov To: qemu-devel@nongnu.org Cc: pbonzini@redhat.com, peterx@redhat.com, mst@redhat.com, mtosatti@redhat.com, kraxel@redhat.com, peter.maydell@linaro.org Subject: [PATCH v2 3/6] hpet: switch to fain-grained device locking Date: Wed, 30 Jul 2025 14:39:31 +0200 Message-ID: <20250730123934.1787379-4-imammedo@redhat.com> In-Reply-To: <20250730123934.1787379-1-imammedo@redhat.com> References: <20250730123934.1787379-1-imammedo@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=imammedo@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1753884703292116600 as a step towards lock-less HPET counter read, use per device locking instead of BQL. Signed-off-by: Igor Mammedov --- hw/timer/hpet.c | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/hw/timer/hpet.c b/hw/timer/hpet.c index cb48cc151f..ab5aa59ae4 100644 --- a/hw/timer/hpet.c +++ b/hw/timer/hpet.c @@ -38,6 +38,7 @@ #include "hw/timer/i8254.h" #include "system/address-spaces.h" #include "qom/object.h" +#include "qemu/lockable.h" #include "trace.h" =20 struct hpet_fw_config hpet_fw_cfg =3D {.count =3D UINT8_MAX}; @@ -69,6 +70,7 @@ struct HPETState { SysBusDevice parent_obj; /*< public >*/ =20 + QemuMutex lock; MemoryRegion iomem; uint64_t hpet_offset; bool hpet_offset_saved; @@ -428,6 +430,7 @@ static uint64_t hpet_ram_read(void *opaque, hwaddr addr, trace_hpet_ram_read(addr); addr &=3D ~4; =20 + QEMU_LOCK_GUARD(&s->lock); /*address range of all global regs*/ if (addr <=3D 0xff) { switch (addr) { @@ -482,6 +485,7 @@ static void hpet_ram_write(void *opaque, hwaddr addr, int len =3D MIN(size * 8, 64 - shift); uint64_t old_val, new_val, cleared; =20 + QEMU_LOCK_GUARD(&s->lock); trace_hpet_ram_write(addr, value); addr &=3D ~4; =20 @@ -679,8 +683,10 @@ static void hpet_init(Object *obj) SysBusDevice *sbd =3D SYS_BUS_DEVICE(obj); HPETState *s =3D HPET(obj); =20 + qemu_mutex_init(&s->lock); /* HPET Area */ memory_region_init_io(&s->iomem, obj, &hpet_ram_ops, s, "hpet", HPET_L= EN); + memory_region_enable_lockless_io(&s->iomem); sysbus_init_mmio(sbd, &s->iomem); } =20 --=20 2.47.1 From nobody Sat Nov 15 06:34:12 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1753884558; cv=none; d=zohomail.com; s=zohoarc; b=MUjgKRLYi+I7YNYNHiCXBd4JetQo0KQgvJEfmK5UexG40hIr5vNrajPCTD6Vy6n/5mXbAxxY7mXfscWcQmU1nBRq39YjzrRFTk9OzlChlxNPhQfJ0pYq9zzic0MrWqxXNhQcyO79yPRlzxR5lKKlh8MPM399w2n4m6W4HWMILyo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1753884558; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=y9K9gyDjJKbin0KwGCFLv4GYP/E25xDhDu7lZViWdjw=; b=A+NjK4CiAY6muzJLXhXKnXWDo2XlkGx/cQnF1E1CpyQkJxxJxzmytCoEBlAMmhToOtpyWCPVYBSn+Md7FTH+FmQc7Flkb6Vd6gMOi8rDtR4HGIAqCzK2VWKr1vpX36/GMMRDAFzhILkMrSQkCwdXMdGVUFb7w1RCLC2+OtLLQxk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 175388455800161.82165396719654; Wed, 30 Jul 2025 07:09:18 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uh7U9-0004Iw-ET; Wed, 30 Jul 2025 10:08:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uh66B-0005fk-CG for qemu-devel@nongnu.org; Wed, 30 Jul 2025 08:40:00 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uh665-0006v2-K8 for qemu-devel@nongnu.org; Wed, 30 Jul 2025 08:39:58 -0400 Received: from mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (ec2-35-165-154-97.us-west-2.compute.amazonaws.com [35.165.154.97]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-564-kT1SmgLoNOqgbmzU8U_o8w-1; Wed, 30 Jul 2025 08:39:51 -0400 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-08.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 0E5A9180048E; Wed, 30 Jul 2025 12:39:50 +0000 (UTC) Received: from dell-r430-03.lab.eng.brq2.redhat.com (dell-r430-03.lab.eng.brq2.redhat.com [10.37.153.18]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 011B330001B1; Wed, 30 Jul 2025 12:39:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1753879192; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=y9K9gyDjJKbin0KwGCFLv4GYP/E25xDhDu7lZViWdjw=; b=TYMU5vUkyAjPV8/2blRNJQX/V9iyIYEjphrRunSkd64hrRE2AwNjCGFR6RLa9V8g3wYqWw qyxzo2QfqWiWvsCaUzOv+x3KDmaLYEZb9WB0W/PZudzMBtNK3AaO+BGF+8w0YAU1Yt7FcF Hlq6fTwWo4NemEu8zk79WXzHSbOgFf8= X-MC-Unique: kT1SmgLoNOqgbmzU8U_o8w-1 X-Mimecast-MFC-AGG-ID: kT1SmgLoNOqgbmzU8U_o8w_1753879190 From: Igor Mammedov To: qemu-devel@nongnu.org Cc: pbonzini@redhat.com, peterx@redhat.com, mst@redhat.com, mtosatti@redhat.com, kraxel@redhat.com, peter.maydell@linaro.org Subject: [PATCH v2 4/6] hpet: move out main counter read into a separate block Date: Wed, 30 Jul 2025 14:39:32 +0200 Message-ID: <20250730123934.1787379-5-imammedo@redhat.com> In-Reply-To: <20250730123934.1787379-1-imammedo@redhat.com> References: <20250730123934.1787379-1-imammedo@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=imammedo@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_PASS=-0.001, T_SPF_HELO_TEMPERROR=0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1753884559656116600 Follow up patche will switch main counter read to lock-less mode. As preparation for that move relevant branch into a separate top level block to make followup patch cleaner/simplier by reducing contextual noise when lock-less read is introduced. no functional changes. Signed-off-by: Igor Mammedov --- hw/timer/hpet.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-) diff --git a/hw/timer/hpet.c b/hw/timer/hpet.c index ab5aa59ae4..97687697c9 100644 --- a/hw/timer/hpet.c +++ b/hw/timer/hpet.c @@ -431,6 +431,16 @@ static uint64_t hpet_ram_read(void *opaque, hwaddr add= r, addr &=3D ~4; =20 QEMU_LOCK_GUARD(&s->lock); + if ((addr <=3D 0xff) && (addr =3D=3D HPET_COUNTER)) { + if (hpet_enabled(s)) { + cur_tick =3D hpet_get_ticks(s); + } else { + cur_tick =3D s->hpet_counter; + } + trace_hpet_ram_read_reading_counter(addr & 4, cur_tick); + return cur_tick >> shift; + } + /*address range of all global regs*/ if (addr <=3D 0xff) { switch (addr) { @@ -438,14 +448,6 @@ static uint64_t hpet_ram_read(void *opaque, hwaddr add= r, return s->capability >> shift; case HPET_CFG: return s->config >> shift; - case HPET_COUNTER: - if (hpet_enabled(s)) { - cur_tick =3D hpet_get_ticks(s); - } else { - cur_tick =3D s->hpet_counter; - } - trace_hpet_ram_read_reading_counter(addr & 4, cur_tick); - return cur_tick >> shift; case HPET_STATUS: return s->isr >> shift; default: --=20 2.47.1 From nobody Sat Nov 15 06:34:12 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1753885336; cv=none; d=zohomail.com; s=zohoarc; b=lqV6WfXNRPKlO8MAtwQyo/s/x6CFfRboH1QPhHQKcBgubncCD0oBttk0J7ErPyd4fFx08mhNC+Vit3TOuzrKwO6xq0Bpx5il+VSavCdcpASrVK+Cat/4rZaaKg46NVinRUYPoEmopNcrni7tR3xVoE96CbKmVqmSgkTEDEAxsQs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1753885336; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=1zd1FxBLQ4yxr7ES3QONl2dWq+/99FTSEpJQKAVkOJI=; b=BrAngAF4iqhmEZ81qRKjRd+9ZqTV7xJ/Y5nGX6kQfFtApjlYpgaHsZ5wollupXCCi6jaW+eFZ1DD2YSexQX7M5KRzSoaX1b1JaLl+LpBQZxqF00/STIzcBsnBwLcAtXlZdcyjf4nWo9PXdJgEnKecmxvWL0BXRAs3fspyXCDDNU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1753885336235988.5131518915749; Wed, 30 Jul 2025 07:22:16 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uh7gc-00081c-Rw; Wed, 30 Jul 2025 10:21:43 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uh66D-0005ho-Ed for qemu-devel@nongnu.org; Wed, 30 Jul 2025 08:40:02 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uh669-0006vs-Ar for qemu-devel@nongnu.org; Wed, 30 Jul 2025 08:40:00 -0400 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-101-5mN1s-vyNlCk89WZ9v0Hmg-1; Wed, 30 Jul 2025 08:39:53 -0400 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 7C8D119560B5; Wed, 30 Jul 2025 12:39:52 +0000 (UTC) Received: from dell-r430-03.lab.eng.brq2.redhat.com (dell-r430-03.lab.eng.brq2.redhat.com [10.37.153.18]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 75A2B30001B1; Wed, 30 Jul 2025 12:39:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1753879196; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=1zd1FxBLQ4yxr7ES3QONl2dWq+/99FTSEpJQKAVkOJI=; b=e9ubpT/Blui4Bd49W2VuWPXNw9X79x5S+Ox+FWwlLAJyVyYbUh06Jhkr32T2p4/B9gijqM dw7YcQfpnpfBydimgzowVYGgkSXEDLdOiGEVHQr5Cegak+KsX7GSH94sboIcW7OaHxfrAt sEffTsT1YQ2z62/g1TmYsBPVpzf7rk0= X-MC-Unique: 5mN1s-vyNlCk89WZ9v0Hmg-1 X-Mimecast-MFC-AGG-ID: 5mN1s-vyNlCk89WZ9v0Hmg_1753879192 From: Igor Mammedov To: qemu-devel@nongnu.org Cc: pbonzini@redhat.com, peterx@redhat.com, mst@redhat.com, mtosatti@redhat.com, kraxel@redhat.com, peter.maydell@linaro.org Subject: [PATCH v2 5/6] hpet: make main counter read lock-less Date: Wed, 30 Jul 2025 14:39:33 +0200 Message-ID: <20250730123934.1787379-6-imammedo@redhat.com> In-Reply-To: <20250730123934.1787379-1-imammedo@redhat.com> References: <20250730123934.1787379-1-imammedo@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=imammedo@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 12 X-Spam_score: 1.2 X-Spam_bar: + X-Spam_report: (1.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1753885337640116600 Make access to main HPET counter lock-less when enable/disable state isn't changing (which is the most of the time). A read will fallback to locked access if the state change happens in the middle of read or read happens in the middle of the state change. This basically uses the same approach as cpu_get_clock(), modulo instead of busy wait it piggibacks to taking device lock to wait until HPET reaches consistent state. As result micro benchmark of concurrent reading of HPET counter with large number of vCPU shows over 80% better (less) latency. Signed-off-by: Igor Mammedov --- hw/timer/hpet.c | 48 ++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 44 insertions(+), 4 deletions(-) diff --git a/hw/timer/hpet.c b/hw/timer/hpet.c index 97687697c9..d822ca1cd0 100644 --- a/hw/timer/hpet.c +++ b/hw/timer/hpet.c @@ -74,6 +74,7 @@ struct HPETState { MemoryRegion iomem; uint64_t hpet_offset; bool hpet_offset_saved; + unsigned state_version; qemu_irq irqs[HPET_NUM_IRQ_ROUTES]; uint32_t flags; uint8_t rtc_irq_level; @@ -430,17 +431,44 @@ static uint64_t hpet_ram_read(void *opaque, hwaddr ad= dr, trace_hpet_ram_read(addr); addr &=3D ~4; =20 - QEMU_LOCK_GUARD(&s->lock); if ((addr <=3D 0xff) && (addr =3D=3D HPET_COUNTER)) { - if (hpet_enabled(s)) { - cur_tick =3D hpet_get_ticks(s); - } else { + unsigned version; + bool release_lock =3D false; +redo: + version =3D qatomic_load_acquire(&s->state_version); + if (unlikely(version & 1)) { + /* + * Updater is running, state can be inconsistent + * wait till it's done before reading counter + */ + release_lock =3D true; + qemu_mutex_lock(&s->lock); + } + + if (unlikely(!hpet_enabled(s))) { cur_tick =3D s->hpet_counter; + } else { + cur_tick =3D hpet_get_ticks(s); + } + + /* + * ensure counter math happens before we check version again + */ + smp_rmb(); + if (unlikely(version !=3D qatomic_load_acquire(&s->state_version))= ) { + /* + * counter state has changed, re-read counter again + */ + goto redo; + } + if (unlikely(release_lock)) { + qemu_mutex_unlock(&s->lock); } trace_hpet_ram_read_reading_counter(addr & 4, cur_tick); return cur_tick >> shift; } =20 + QEMU_LOCK_GUARD(&s->lock); /*address range of all global regs*/ if (addr <=3D 0xff) { switch (addr) { @@ -500,6 +528,11 @@ static void hpet_ram_write(void *opaque, hwaddr addr, old_val =3D s->config; new_val =3D deposit64(old_val, shift, len, value); new_val =3D hpet_fixup_reg(new_val, old_val, HPET_CFG_WRITE_MA= SK); + /* + * Odd versions mark the critical section, any readers will be + * forced into lock protected read if they come in the middle = of it + */ + qatomic_inc(&s->state_version); s->config =3D new_val; if (activating_bit(old_val, new_val, HPET_CFG_ENABLE)) { /* Enable main counter and interrupt generation. */ @@ -518,6 +551,13 @@ static void hpet_ram_write(void *opaque, hwaddr addr, hpet_del_timer(&s->timer[i]); } } + /* + * even versions mark the end of critical section, + * any readers started before config change, but were still ex= ecuted + * during the change, will be forced to re-read counter state + */ + qatomic_inc(&s->state_version); + /* i8254 and RTC output pins are disabled * when HPET is in legacy mode */ if (activating_bit(old_val, new_val, HPET_CFG_LEGACY)) { --=20 2.47.1 From nobody Sat Nov 15 06:34:12 2025 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=quarantine dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1753884699; cv=none; d=zohomail.com; s=zohoarc; b=lC81HJzeW1QGNZM92/RntQEZ4KNfQTypN7DKjSmbTdBhxs82ulr9O7mZfOfIBwPXURohrhvEBjIHjWoxw+qA8DOaeWrN0E6b6Ll+gx5MdoJ/BxmndaLkBzU7izD47ZJu7cugKjgj8QSBlNd+FtPERZIu7Jq+3ur4E3BA3Y17VCo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1753884699; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=vnkKV/qN9hIacD0TfY2P1fY0zKaP/nMy687lfMn0F8c=; b=iwfP/umpLN8iMyEZxgPO3MJvB6u20tDKVcPvV1TfAlQLHUUBsSAzqpfuKVayQXYETcbym1boYjy0Kp+I7+aMgnQQ1qZChVtNpT2I9VfSQXh8VC/jwM4dCjhDZR/R0aNwLihhhtab5rUB2L8ZV3W52/3Cr7fxv5taZBa6EV0nvmk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=quarantine dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1753884699438568.1986924689605; Wed, 30 Jul 2025 07:11:39 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1uh7Wl-00082i-KO; Wed, 30 Jul 2025 10:11:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uh66J-0005kf-3O for qemu-devel@nongnu.org; Wed, 30 Jul 2025 08:40:13 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uh66D-0006wZ-Fs for qemu-devel@nongnu.org; Wed, 30 Jul 2025 08:40:06 -0400 Received: from mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (ec2-54-186-198-63.us-west-2.compute.amazonaws.com [54.186.198.63]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-339-JPex7cTPPh6wE-oUrm9VtQ-1; Wed, 30 Jul 2025 08:39:56 -0400 Received: from mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com [10.30.177.4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mx-prod-mc-05.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTPS id 8453F1956088; Wed, 30 Jul 2025 12:39:55 +0000 (UTC) Received: from dell-r430-03.lab.eng.brq2.redhat.com (dell-r430-03.lab.eng.brq2.redhat.com [10.37.153.18]) by mx-prod-int-01.mail-002.prod.us-west-2.aws.redhat.com (Postfix) with ESMTP id 1D8C930001BE; Wed, 30 Jul 2025 12:39:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1753879199; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vnkKV/qN9hIacD0TfY2P1fY0zKaP/nMy687lfMn0F8c=; b=NvFGKUQDnCOTOYCFbvXfgRqOLKDvSj1Ld2VZrpNDvRzAGC2jWj3kbPRjpTg5+BgYyMaxmB cZbz+7zRGYRb+PQRw3kFDUNRvduGp/48PNix7T0Exoeqvx1SzrpxqLSKG97dZqxzOADFoq grB6mVJepjeurSSowQU2PEgEsLPvCkc= X-MC-Unique: JPex7cTPPh6wE-oUrm9VtQ-1 X-Mimecast-MFC-AGG-ID: JPex7cTPPh6wE-oUrm9VtQ_1753879195 From: Igor Mammedov To: qemu-devel@nongnu.org Cc: pbonzini@redhat.com, peterx@redhat.com, mst@redhat.com, mtosatti@redhat.com, kraxel@redhat.com, peter.maydell@linaro.org Subject: [PATCH v2 6/6] kvm: i386: irqchip: take BQL only if there is an interrupt Date: Wed, 30 Jul 2025 14:39:34 +0200 Message-ID: <20250730123934.1787379-7-imammedo@redhat.com> In-Reply-To: <20250730123934.1787379-1-imammedo@redhat.com> References: <20250730123934.1787379-1-imammedo@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.30.177.4 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=imammedo@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: 12 X-Spam_score: 1.2 X-Spam_bar: + X-Spam_report: (1.2 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_SBL_CSS=3.335, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_PASS=-0.001, T_SPF_HELO_TEMPERROR=0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1753884701382116600 when kernel-irqchip=3Dsplit is used, QEMU still hits BQL contention issue when reading ACPI PM/HPET timers (despite of timer[s] access being lock-less). So Windows with more than 255 cpus is still not able to boot (since it requires iommu -> split irqchip). Problematic path is in kvm_arch_pre_run() where BQL is taken unconditionally when split irqchip is in use. There are a few parts tha BQL protects there: 1. interrupt check and injecting however we do not take BQL when checking for pending interrupt (even within the same function), so the patch takes the same approach for cpu->interrupt_request checks and takes BQL only if there is a job to do. 2. request_interrupt_window access CPUState::kvm_run::request_interrupt_window doesn't need BQL as it's accessed on side QEMU only by its own vCPU thread. The only thing that BQL provides there is implict barrier. Which can be done by using cheaper explicit barrier there. 3. cr8/cpu_get_apic_tpr access the same (as #2) applies to CPUState::kvm_run::cr8 write, and APIC registers are also cached/synced (get/put) within the vCPU thread it belongs to. Taking BQL only when is necessary, eleminates BQL bottleneck on IO/MMIO only exit path, improoving latency by 80% on HPET micro benchmark. This lets Windows to boot succesfully (in case hv-time isn't used) when more than 255 vCPUs are in use. Signed-off-by: Igor Mammedov --- target/i386/kvm/kvm.c | 58 +++++++++++++++++++++++++++---------------- 1 file changed, 37 insertions(+), 21 deletions(-) diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c index 369626f8c8..32024d50f5 100644 --- a/target/i386/kvm/kvm.c +++ b/target/i386/kvm/kvm.c @@ -5450,6 +5450,7 @@ void kvm_arch_pre_run(CPUState *cpu, struct kvm_run *= run) { X86CPU *x86_cpu =3D X86_CPU(cpu); CPUX86State *env =3D &x86_cpu->env; + bool release_bql =3D 0; int ret; =20 /* Inject NMI */ @@ -5478,15 +5479,16 @@ void kvm_arch_pre_run(CPUState *cpu, struct kvm_run= *run) } } =20 - if (!kvm_pic_in_kernel()) { - bql_lock(); - } =20 /* Force the VCPU out of its inner loop to process any INIT requests * or (for userspace APIC, but it is cheap to combine the checks here) * pending TPR access reports. */ if (cpu->interrupt_request & (CPU_INTERRUPT_INIT | CPU_INTERRUPT_TPR))= { + if (!kvm_pic_in_kernel()) { + bql_lock(); + release_bql =3D true; + } if ((cpu->interrupt_request & CPU_INTERRUPT_INIT) && !(env->hflags & HF_SMM_MASK)) { cpu->exit_request =3D 1; @@ -5497,24 +5499,31 @@ void kvm_arch_pre_run(CPUState *cpu, struct kvm_run= *run) } =20 if (!kvm_pic_in_kernel()) { - /* Try to inject an interrupt if the guest can accept it */ - if (run->ready_for_interrupt_injection && - (cpu->interrupt_request & CPU_INTERRUPT_HARD) && - (env->eflags & IF_MASK)) { - int irq; - - cpu->interrupt_request &=3D ~CPU_INTERRUPT_HARD; - irq =3D cpu_get_pic_interrupt(env); - if (irq >=3D 0) { - struct kvm_interrupt intr; - - intr.irq =3D irq; - DPRINTF("injected interrupt %d\n", irq); - ret =3D kvm_vcpu_ioctl(cpu, KVM_INTERRUPT, &intr); - if (ret < 0) { - fprintf(stderr, - "KVM: injection failed, interrupt lost (%s)\n", - strerror(-ret)); + if (cpu->interrupt_request & CPU_INTERRUPT_HARD) { + if (!release_bql) { + bql_lock(); + release_bql =3D true; + } + + /* Try to inject an interrupt if the guest can accept it */ + if (run->ready_for_interrupt_injection && + (cpu->interrupt_request & CPU_INTERRUPT_HARD) && + (env->eflags & IF_MASK)) { + int irq; + + cpu->interrupt_request &=3D ~CPU_INTERRUPT_HARD; + irq =3D cpu_get_pic_interrupt(env); + if (irq >=3D 0) { + struct kvm_interrupt intr; + + intr.irq =3D irq; + DPRINTF("injected interrupt %d\n", irq); + ret =3D kvm_vcpu_ioctl(cpu, KVM_INTERRUPT, &intr); + if (ret < 0) { + fprintf(stderr, + "KVM: injection failed, interrupt lost (%s= )\n", + strerror(-ret)); + } } } } @@ -5531,7 +5540,14 @@ void kvm_arch_pre_run(CPUState *cpu, struct kvm_run = *run) =20 DPRINTF("setting tpr\n"); run->cr8 =3D cpu_get_apic_tpr(x86_cpu->apic_state); + /* + * make sure that request_interrupt_window/cr8 are set + * before KVM_RUN might read them + */ + smp_mb(); + } =20 + if (release_bql) { bql_unlock(); } } --=20 2.47.1