From nobody Mon Feb 9 15:10:19 2026 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1597765515; cv=none; d=zohomail.com; s=zohoarc; b=MCPcbG8rNyqdkNrevwFa2LqdNNFxyYrIxsmYNviAuX/OxH41RegjoRJTSMP20fXIwHdA9kFIdZQXHS+Dr9D2jT10lm2Ld0D1AO3PRwCCTkCscAxdANFmqO1Z7jIoFel5CgtQIBrGakv180AMaVYJYqZ5WPC5xG6OqR5OZKH0ns8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1597765515; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ENnQwlZiy2s0di8Jd6xc923V8L9TxtXZ4z/RSHMFSr8=; b=iyX6te40tu9/0Yr5TSB7binzhoLM18zdv+GElBgKFG+c/kzXtfFIwOu2xu0/oCSlDvN9a0nl+Wf0a3OL8DUNGT2lz6mu+aeDguvKenuLGaKY8hXMHYAFJ+clnYYowXpuvCJ9IKpQO4b45VKCflIvtq2Ic8luj1/RmiE+mt/bfEM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1597765515179699.3687305737204; Tue, 18 Aug 2020 08:45:15 -0700 (PDT) Received: from localhost ([::1]:36128 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1k83nV-00068i-Tn for importer@patchew.org; Tue, 18 Aug 2020 11:45:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:39494) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1k82h4-0004xn-KY for qemu-devel@nongnu.org; Tue, 18 Aug 2020 10:34:30 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:23125) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1k82h1-0002UL-Eq for qemu-devel@nongnu.org; Tue, 18 Aug 2020 10:34:30 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-511-KZGcSDqNOqiXxGbvOPpf-Q-1; Tue, 18 Aug 2020 10:34:23 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B8453AD32C; Tue, 18 Aug 2020 14:34:08 +0000 (UTC) Received: from localhost (ovpn-114-160.ams2.redhat.com [10.36.114.160]) by smtp.corp.redhat.com (Postfix) with ESMTP id 34E712CFC6; Tue, 18 Aug 2020 14:34:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1597761266; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ENnQwlZiy2s0di8Jd6xc923V8L9TxtXZ4z/RSHMFSr8=; b=RgTT7fZnqe3Ifq8Tfxo2L3bbuSmHBDIeTIQe97wrJBx+ctUTLljLZox3Ck0JPrKND7mtX6 zfW7pugnWqwFzJwkCuBJPGDFothgHdmtJgwOaI56I0kZn0Xzb4Qxh1Zz6xEMcELp8IAAuI zrAZpHIVAZm02eE4XkxzNyQeqaWZFuQ= X-MC-Unique: KZGcSDqNOqiXxGbvOPpf-Q-1 From: Stefan Hajnoczi To: qemu-devel@nongnu.org Subject: [PATCH v6 3/7] virtio-pci: add virtio_pci_optimal_num_queues() helper Date: Tue, 18 Aug 2020 15:33:44 +0100 Message-Id: <20200818143348.310613-4-stefanha@redhat.com> In-Reply-To: <20200818143348.310613-1-stefanha@redhat.com> References: <20200818143348.310613-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=stefanha@redhat.com X-Mimecast-Spam-Score: 0.001 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/08/18 03:22:11 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -23 X-Spam_score: -2.4 X-Spam_bar: -- X-Spam_report: (-2.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MIME_BASE64_TEXT=1.741, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=-1, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=unavailable autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Fam Zheng , Peter Maydell , "Michael S. Tsirkin" , Max Reitz , qemu-block@nongnu.org, David Hildenbrand , Halil Pasic , Christian Borntraeger , Richard Henderson , Thomas Huth , Eduardo Habkost , qemu-s390x@nongnu.org, qemu-arm@nongnu.org, Stefan Hajnoczi , David Gibson , Kevin Wolf , Pankaj Gupta , cohuck@redhat.com, Raphael Norwitz , qemu-ppc@nongnu.org, Paolo Bonzini Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Multi-queue devices achieve the best performance when each vCPU has a dedicated queue. This ensures that virtqueue used notifications are handled on the same vCPU that submitted virtqueue buffers. When another vCPU handles the the notification an IPI will be necessary to wake the submission vCPU and this incurs a performance overhead. Provide a helper function that virtio-pci devices will use in later patches to automatically select the optimal number of queues. The function handles guests with large numbers of CPUs by limiting the number of queues to fit within the following constraints: 1. The maximum number of MSI-X vectors. 2. The maximum number of virtqueues. Signed-off-by: Stefan Hajnoczi Reviewed-by: Cornelia Huck --- hw/virtio/virtio-pci.h | 9 +++++++++ hw/virtio/virtio-pci.c | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+) diff --git a/hw/virtio/virtio-pci.h b/hw/virtio/virtio-pci.h index e2eaaa9182..91096f0291 100644 --- a/hw/virtio/virtio-pci.h +++ b/hw/virtio/virtio-pci.h @@ -243,4 +243,13 @@ typedef struct VirtioPCIDeviceTypeInfo { /* Register virtio-pci type(s). @t must be static. */ void virtio_pci_types_register(const VirtioPCIDeviceTypeInfo *t); =20 +/** + * virtio_pci_optimal_num_queues: + * @fixed_queues: number of queues that are always present + * + * Returns: The optimal number of queues for a multi-queue device, excludi= ng + * @fixed_queues. + */ +unsigned virtio_pci_optimal_num_queues(unsigned fixed_queues); + #endif diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c index ccdf54e81c..fc69570dcc 100644 --- a/hw/virtio/virtio-pci.c +++ b/hw/virtio/virtio-pci.c @@ -19,6 +19,7 @@ =20 #include "exec/memop.h" #include "standard-headers/linux/virtio_pci.h" +#include "hw/boards.h" #include "hw/virtio/virtio.h" #include "migration/qemu-file-types.h" #include "hw/pci/pci.h" @@ -2058,6 +2059,37 @@ void virtio_pci_types_register(const VirtioPCIDevice= TypeInfo *t) g_free(base_name); } =20 +unsigned virtio_pci_optimal_num_queues(unsigned fixed_queues) +{ + /* + * 1:1 vq to vCPU mapping is ideal because the same vCPU that submitted + * virtqueue buffers can handle their completion. When a different vCPU + * handles completion it may need to IPI the vCPU that submitted the + * request and this adds overhead. + * + * Virtqueues consume guest RAM and MSI-X vectors. This is wasteful in + * guests with very many vCPUs and a device that is only used by a few + * vCPUs. Unfortunately optimizing that case requires manual pinning i= nside + * the guest, so those users might as well manually set the number of + * queues. There is no upper limit that can be applied automatically a= nd + * doing so arbitrarily would result in a sudden performance drop once= the + * threshold number of vCPUs is exceeded. + */ + unsigned num_queues =3D current_machine->smp.cpus; + + /* + * The maximum number of MSI-X vectors is PCI_MSIX_FLAGS_QSIZE + 1, bu= t the + * config change interrupt and the fixed virtqueues must be taken into + * account too. + */ + num_queues =3D MIN(num_queues, PCI_MSIX_FLAGS_QSIZE - fixed_queues); + + /* + * There is a limit to how many virtqueues a device can have. + */ + return MIN(num_queues, VIRTIO_QUEUE_MAX - fixed_queues); +} + /* virtio-pci-bus */ =20 static void virtio_pci_bus_new(VirtioBusState *bus, size_t bus_size, --=20 2.26.2