From nobody Thu May 16 03:16:30 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1594043977; cv=none; d=zohomail.com; s=zohoarc; b=Ij0TjbuHlTi0P389g19TVSSfQ0wQPKAFlT521C4TJFfXTAHLE0qhnb5jG9kAGzCE/SL62RgMQ90GSK/0xsaeAFUIi8zmPwc81b480FXRY9pOhqfJe9Ja59YEPSEZ3WbgqhIL14xRzls/z2/3hX7As1hn7150IkWuIQMLXIYhHQc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1594043977; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=aSP7/djMhy25kGi63d6m5Qu/kXKQoqJaN2oMWV9dQSg=; b=MyIIlAdtwp6xzvvh3AJ+odqAkU/5YF7t33/qrSJPPdLYDTt05NbMz2GHKlHibpLEmGFC7ng/p/Dic3n/4fV1l/W0B4mlF/Aw8nuyU+z1yCzxxGRTh7kYpEWXRx0bQiS4Tl4mHy7d6XV+qL1khZg2XXQeFbADZpIa2XzDMpHAWSI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1594043977429549.6586625060656; Mon, 6 Jul 2020 06:59:37 -0700 (PDT) Received: from localhost ([::1]:40274 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jsReh-0006Un-TI for importer@patchew.org; Mon, 06 Jul 2020 09:59:35 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46342) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jsRcT-0001bc-TE for qemu-devel@nongnu.org; Mon, 06 Jul 2020 09:57:17 -0400 Received: from us-smtp-2.mimecast.com ([207.211.31.81]:46389 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jsRcN-0006Sa-Sy for qemu-devel@nongnu.org; Mon, 06 Jul 2020 09:57:17 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-183-goE85UbSMSSoS1l_CdAbsg-1; Mon, 06 Jul 2020 09:57:07 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 1B4CF107ACCA; Mon, 6 Jul 2020 13:57:06 +0000 (UTC) Received: from localhost (ovpn-114-217.ams2.redhat.com [10.36.114.217]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9AF295C1B2; Mon, 6 Jul 2020 13:56:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594043831; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aSP7/djMhy25kGi63d6m5Qu/kXKQoqJaN2oMWV9dQSg=; b=aTYJSo26MV2RrGR6Vr1EAC9MWuXyymzSCxHiXQQbrGqG6E7toFCQPyBX54YL0kl9J8X7GM 4xDRiOmBHi9tvEP5WL3gBk+7erPZCemwE1jNd/6E8x932H6RW96EBGMSvzm+/ypDDeyOSv mexevkPvW3088wpVX3ObJUmaLx7NDzQ= X-MC-Unique: goE85UbSMSSoS1l_CdAbsg-1 From: Stefan Hajnoczi To: qemu-devel@nongnu.org Subject: [PATCH v5 1/5] virtio-pci: add virtio_pci_optimal_num_queues() helper Date: Mon, 6 Jul 2020 14:56:46 +0100 Message-Id: <20200706135650.438362-2-stefanha@redhat.com> In-Reply-To: <20200706135650.438362-1-stefanha@redhat.com> References: <20200706135650.438362-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=stefanha@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=207.211.31.81; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/06 01:39:15 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -13 X-Spam_score: -1.4 X-Spam_bar: - X-Spam_report: (-1.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MIME_BASE64_TEXT=1.741, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Eduardo Habkost , qemu-block@nongnu.org, "Michael S. Tsirkin" , cohuck@redhat.com, Max Reitz , Stefan Hajnoczi , Pankaj Gupta , Paolo Bonzini , Raphael Norwitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Multi-queue devices achieve the best performance when each vCPU has a dedicated queue. This ensures that virtqueue used notifications are handled on the same vCPU that submitted virtqueue buffers. When another vCPU handles the the notification an IPI will be necessary to wake the submission vCPU and this incurs a performance overhead. Provide a helper function that virtio-pci devices will use in later patches to automatically select the optimal number of queues. The function handles guests with large numbers of CPUs by limiting the number of queues to fit within the following constraints: 1. The maximum number of MSI-X vectors. 2. The maximum number of virtqueues. Signed-off-by: Stefan Hajnoczi Reviewed-by: Cornelia Huck --- hw/virtio/virtio-pci.h | 9 +++++++++ hw/virtio/virtio-pci.c | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+) diff --git a/hw/virtio/virtio-pci.h b/hw/virtio/virtio-pci.h index e2eaaa9182..91096f0291 100644 --- a/hw/virtio/virtio-pci.h +++ b/hw/virtio/virtio-pci.h @@ -243,4 +243,13 @@ typedef struct VirtioPCIDeviceTypeInfo { /* Register virtio-pci type(s). @t must be static. */ void virtio_pci_types_register(const VirtioPCIDeviceTypeInfo *t); =20 +/** + * virtio_pci_optimal_num_queues: + * @fixed_queues: number of queues that are always present + * + * Returns: The optimal number of queues for a multi-queue device, excludi= ng + * @fixed_queues. + */ +unsigned virtio_pci_optimal_num_queues(unsigned fixed_queues); + #endif diff --git a/hw/virtio/virtio-pci.c b/hw/virtio/virtio-pci.c index 7bc8c1c056..c48570c03f 100644 --- a/hw/virtio/virtio-pci.c +++ b/hw/virtio/virtio-pci.c @@ -19,6 +19,7 @@ =20 #include "exec/memop.h" #include "standard-headers/linux/virtio_pci.h" +#include "hw/boards.h" #include "hw/virtio/virtio.h" #include "migration/qemu-file-types.h" #include "hw/pci/pci.h" @@ -2028,6 +2029,37 @@ void virtio_pci_types_register(const VirtioPCIDevice= TypeInfo *t) g_free(base_name); } =20 +unsigned virtio_pci_optimal_num_queues(unsigned fixed_queues) +{ + /* + * 1:1 vq to vCPU mapping is ideal because the same vCPU that submitted + * virtqueue buffers can handle their completion. When a different vCPU + * handles completion it may need to IPI the vCPU that submitted the + * request and this adds overhead. + * + * Virtqueues consume guest RAM and MSI-X vectors. This is wasteful in + * guests with very many vCPUs and a device that is only used by a few + * vCPUs. Unfortunately optimizing that case requires manual pinning i= nside + * the guest, so those users might as well manually set the number of + * queues. There is no upper limit that can be applied automatically a= nd + * doing so arbitrarily would result in a sudden performance drop once= the + * threshold number of vCPUs is exceeded. + */ + unsigned num_queues =3D current_machine->smp.cpus; + + /* + * The maximum number of MSI-X vectors is PCI_MSIX_FLAGS_QSIZE + 1, bu= t the + * config change interrupt and the fixed virtqueues must be taken into + * account too. + */ + num_queues =3D MIN(num_queues, PCI_MSIX_FLAGS_QSIZE - fixed_queues); + + /* + * There is a limit to how many virtqueues a device can have. + */ + return MIN(num_queues, VIRTIO_QUEUE_MAX - fixed_queues); +} + /* virtio-pci-bus */ =20 static void virtio_pci_bus_new(VirtioBusState *bus, size_t bus_size, --=20 2.26.2 From nobody Thu May 16 03:16:30 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1594043894; cv=none; d=zohomail.com; s=zohoarc; b=KlphqATGrdPTzqlyABP5KndDPoK3cmecK0nYs3NSRYiKp5fpQMBYvcYdxh/hE0iVy0iEL8Cv37E8cFPh5ao1u0fX/C2VnRXpNMHipOpaZDF8WtxyM10V+eTzYCy8oR85bNpREVMVLMDVM8L0KpTG1hQ0Uc1de+dX+I5RASiB/MI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1594043894; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=uVeNRqM224/wT/cjolJi+5I426fp5SQFqRKucSZEUf4=; b=Y1CbVyC9O8oO+ONaK8/xbok3/7HXFZjbDeXINWLH6YZap7UkoNaBrvg8eTaXPPB4cU/IENtdaimkhBwa5d/lj0hCpuWrQqd+WSRSRhScPc4xin5lloYFM9mBp37GEw4SRvF9kzKR6CcIRRITBe+8lF5u6cARCorRpp2ovj2bL2Y= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1594043894193249.5756682151822; Mon, 6 Jul 2020 06:58:14 -0700 (PDT) Received: from localhost ([::1]:60770 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jsRdM-00030k-NT for importer@patchew.org; Mon, 06 Jul 2020 09:58:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46336) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jsRcT-0001bG-Ce for qemu-devel@nongnu.org; Mon, 06 Jul 2020 09:57:17 -0400 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:45152 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jsRcP-0006Sp-Jz for qemu-devel@nongnu.org; Mon, 06 Jul 2020 09:57:15 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-388-DIxKjo8APfiAlTrF-jI62A-1; Mon, 06 Jul 2020 09:57:09 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E98458015FA; Mon, 6 Jul 2020 13:57:07 +0000 (UTC) Received: from localhost (ovpn-114-217.ams2.redhat.com [10.36.114.217]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7F1465D9F5; Mon, 6 Jul 2020 13:57:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594043832; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uVeNRqM224/wT/cjolJi+5I426fp5SQFqRKucSZEUf4=; b=CZRleaVJdzBfGi8RVr4/i9Lrb64mtJzAwQQe1l7MyXfx/9BjbE6KcQ0rnBLuXu2TXiMk7g NZe7zXpgok+xoftKaNcBlYplkBbSFeBdhG5+vzk3v02y+xWB2iCSjiHJ4nuhJQQfZ3OFLh AhaM7NFOTiFNMusUFUPKzH462FHC7Uc= X-MC-Unique: DIxKjo8APfiAlTrF-jI62A-1 From: Stefan Hajnoczi To: qemu-devel@nongnu.org Subject: [PATCH v5 2/5] virtio-scsi: introduce a constant for fixed virtqueues Date: Mon, 6 Jul 2020 14:56:47 +0100 Message-Id: <20200706135650.438362-3-stefanha@redhat.com> In-Reply-To: <20200706135650.438362-1-stefanha@redhat.com> References: <20200706135650.438362-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=stefanha@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.61; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/06 01:59:39 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -30 X-Spam_score: -3.1 X-Spam_bar: --- X-Spam_report: (-3.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Eduardo Habkost , qemu-block@nongnu.org, "Michael S. Tsirkin" , cohuck@redhat.com, Max Reitz , Stefan Hajnoczi , Pankaj Gupta , Paolo Bonzini , Raphael Norwitz , =?UTF-8?q?Philippe=20Mathieu-Daud=C3=A9?= Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" The event and control virtqueues are always present, regardless of the multi-queue configuration. Define a constant so that virtqueue number calculations are easier to read. Signed-off-by: Stefan Hajnoczi Reviewed-by: Cornelia Huck Reviewed-by: Pankaj Gupta Reviewed-by: Philippe Mathieu-Daud=C3=A9 Reviewed-by: Raphael Norwitz --- include/hw/virtio/virtio-scsi.h | 3 +++ hw/scsi/vhost-user-scsi.c | 2 +- hw/scsi/virtio-scsi.c | 7 ++++--- hw/virtio/vhost-scsi-pci.c | 3 ++- hw/virtio/vhost-user-scsi-pci.c | 3 ++- hw/virtio/virtio-scsi-pci.c | 3 ++- 6 files changed, 14 insertions(+), 7 deletions(-) diff --git a/include/hw/virtio/virtio-scsi.h b/include/hw/virtio/virtio-scs= i.h index 24e768909d..9f293bcb80 100644 --- a/include/hw/virtio/virtio-scsi.h +++ b/include/hw/virtio/virtio-scsi.h @@ -36,6 +36,9 @@ #define VIRTIO_SCSI_MAX_TARGET 255 #define VIRTIO_SCSI_MAX_LUN 16383 =20 +/* Number of virtqueues that are always present */ +#define VIRTIO_SCSI_VQ_NUM_FIXED 2 + typedef struct virtio_scsi_cmd_req VirtIOSCSICmdReq; typedef struct virtio_scsi_cmd_resp VirtIOSCSICmdResp; typedef struct virtio_scsi_ctrl_tmf_req VirtIOSCSICtrlTMFReq; diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c index f2e524438a..a8b821466f 100644 --- a/hw/scsi/vhost-user-scsi.c +++ b/hw/scsi/vhost-user-scsi.c @@ -114,7 +114,7 @@ static void vhost_user_scsi_realize(DeviceState *dev, E= rror **errp) goto free_virtio; } =20 - vsc->dev.nvqs =3D 2 + vs->conf.num_queues; + vsc->dev.nvqs =3D VIRTIO_SCSI_VQ_NUM_FIXED + vs->conf.num_queues; vsc->dev.vqs =3D g_new0(struct vhost_virtqueue, vsc->dev.nvqs); vsc->dev.vq_index =3D 0; vsc->dev.backend_features =3D 0; diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c index b49775269e..eecdd05af5 100644 --- a/hw/scsi/virtio-scsi.c +++ b/hw/scsi/virtio-scsi.c @@ -191,7 +191,7 @@ static void virtio_scsi_save_request(QEMUFile *f, SCSIR= equest *sreq) VirtIOSCSIReq *req =3D sreq->hba_private; VirtIOSCSICommon *vs =3D VIRTIO_SCSI_COMMON(req->dev); VirtIODevice *vdev =3D VIRTIO_DEVICE(req->dev); - uint32_t n =3D virtio_get_queue_index(req->vq) - 2; + uint32_t n =3D virtio_get_queue_index(req->vq) - VIRTIO_SCSI_VQ_NUM_FI= XED; =20 assert(n < vs->conf.num_queues); qemu_put_be32s(f, &n); @@ -892,10 +892,11 @@ void virtio_scsi_common_realize(DeviceState *dev, sizeof(VirtIOSCSIConfig)); =20 if (s->conf.num_queues =3D=3D 0 || - s->conf.num_queues > VIRTIO_QUEUE_MAX - 2) { + s->conf.num_queues > VIRTIO_QUEUE_MAX - VIRTIO_SCSI_VQ_NUM_FIX= ED) { error_setg(errp, "Invalid number of queues (=3D %" PRIu32 "), " "must be a positive integer less than %d.", - s->conf.num_queues, VIRTIO_QUEUE_MAX - 2); + s->conf.num_queues, + VIRTIO_QUEUE_MAX - VIRTIO_SCSI_VQ_NUM_FIXED); virtio_cleanup(vdev); return; } diff --git a/hw/virtio/vhost-scsi-pci.c b/hw/virtio/vhost-scsi-pci.c index 095af23f3f..06e814d30e 100644 --- a/hw/virtio/vhost-scsi-pci.c +++ b/hw/virtio/vhost-scsi-pci.c @@ -50,7 +50,8 @@ static void vhost_scsi_pci_realize(VirtIOPCIProxy *vpci_d= ev, Error **errp) VirtIOSCSICommon *vs =3D VIRTIO_SCSI_COMMON(vdev); =20 if (vpci_dev->nvectors =3D=3D DEV_NVECTORS_UNSPECIFIED) { - vpci_dev->nvectors =3D vs->conf.num_queues + 3; + vpci_dev->nvectors =3D vs->conf.num_queues + + VIRTIO_SCSI_VQ_NUM_FIXED + 1; } =20 qdev_realize(vdev, BUS(&vpci_dev->bus), errp); diff --git a/hw/virtio/vhost-user-scsi-pci.c b/hw/virtio/vhost-user-scsi-pc= i.c index 4705cd54e8..ab6dfb71a9 100644 --- a/hw/virtio/vhost-user-scsi-pci.c +++ b/hw/virtio/vhost-user-scsi-pci.c @@ -56,7 +56,8 @@ static void vhost_user_scsi_pci_realize(VirtIOPCIProxy *v= pci_dev, Error **errp) VirtIOSCSICommon *vs =3D VIRTIO_SCSI_COMMON(vdev); =20 if (vpci_dev->nvectors =3D=3D DEV_NVECTORS_UNSPECIFIED) { - vpci_dev->nvectors =3D vs->conf.num_queues + 3; + vpci_dev->nvectors =3D vs->conf.num_queues + + VIRTIO_SCSI_VQ_NUM_FIXED + 1; } =20 qdev_realize(vdev, BUS(&vpci_dev->bus), errp); diff --git a/hw/virtio/virtio-scsi-pci.c b/hw/virtio/virtio-scsi-pci.c index c23a134202..3ff9eb7ef6 100644 --- a/hw/virtio/virtio-scsi-pci.c +++ b/hw/virtio/virtio-scsi-pci.c @@ -51,7 +51,8 @@ static void virtio_scsi_pci_realize(VirtIOPCIProxy *vpci_= dev, Error **errp) char *bus_name; =20 if (vpci_dev->nvectors =3D=3D DEV_NVECTORS_UNSPECIFIED) { - vpci_dev->nvectors =3D vs->conf.num_queues + 3; + vpci_dev->nvectors =3D vs->conf.num_queues + + VIRTIO_SCSI_VQ_NUM_FIXED + 1; } =20 /* --=20 2.26.2 From nobody Thu May 16 03:16:30 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1594043982; cv=none; d=zohomail.com; s=zohoarc; b=neXpbF3R8Hxg0jnCayR1UlM+1W7gOex9Q8TZ2V/f2vrAK2ega/fAvRvpSxc7bdwzfSeBoUygRt/wvRfJUu3MdPRgNmsBiPs3UCgpYIcf1yhiwVQbpFO/Kh3oVU9FuErnL/40Xky0Gqv7WzGtFm9j24EA5uYJr4tRmX4zY2/WWEc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1594043982; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=UKD3NH96DWiLHGfVGf9AwW0+0SJ9Fg0qLJeVaqWhJKI=; b=KSCeQyoOcsOhf+sjxN18Ios66nKGfFXWGl64BURobMSFyrPLziS0ccYw3YtRi/4ZfMw/wRHusVdUBatWzZ996FTuJopaoQxCloSWO4fDL0WMSSO6ogSTWhnpvAJKeKX7wLPLo/zBMUI1DGVtQtH1y8NKBDJFAjtec+9OV8B3dzc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1594043982658625.8366019697704; Mon, 6 Jul 2020 06:59:42 -0700 (PDT) Received: from localhost ([::1]:40586 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jsRen-0006cL-Aq for importer@patchew.org; Mon, 06 Jul 2020 09:59:41 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46350) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jsRcU-0001co-Tl for qemu-devel@nongnu.org; Mon, 06 Jul 2020 09:57:19 -0400 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:55015 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jsRcT-0006TC-5W for qemu-devel@nongnu.org; Mon, 06 Jul 2020 09:57:18 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-46-c2BY7VtxPbitRvuUk0pQ6w-1; Mon, 06 Jul 2020 09:57:11 -0400 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id C4AF3107ACF3; Mon, 6 Jul 2020 13:57:09 +0000 (UTC) Received: from localhost (ovpn-114-217.ams2.redhat.com [10.36.114.217]) by smtp.corp.redhat.com (Postfix) with ESMTP id 589235D9CC; Mon, 6 Jul 2020 13:57:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594043835; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UKD3NH96DWiLHGfVGf9AwW0+0SJ9Fg0qLJeVaqWhJKI=; b=C5I/llzY+3SCdrgCZottKcnwOG3l3jsXa+G+vIZ5fVvA3PjyLNz8RZvViRl22UhSaO0ac3 4dkiLuhYUrjefBWj/VV6/2LREdPtVa5+IN8EQQu0tQCTYaceVnDC0tQS/+mQ/Et+PKwaAR 3Wms/kJNZNo8LZ6ESgUPOmKsYFvK4Gk= X-MC-Unique: c2BY7VtxPbitRvuUk0pQ6w-1 From: Stefan Hajnoczi To: qemu-devel@nongnu.org Subject: [PATCH v5 3/5] virtio-scsi: default num_queues to -smp N Date: Mon, 6 Jul 2020 14:56:48 +0100 Message-Id: <20200706135650.438362-4-stefanha@redhat.com> In-Reply-To: <20200706135650.438362-1-stefanha@redhat.com> References: <20200706135650.438362-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=stefanha@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.61; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/06 01:59:39 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -13 X-Spam_score: -1.4 X-Spam_bar: - X-Spam_report: (-1.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MIME_BASE64_TEXT=1.741, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Eduardo Habkost , qemu-block@nongnu.org, "Michael S. Tsirkin" , cohuck@redhat.com, Max Reitz , Stefan Hajnoczi , Pankaj Gupta , Paolo Bonzini , Raphael Norwitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Automatically size the number of virtio-scsi-pci, vhost-scsi-pci, and vhost-user-scsi-pci request virtqueues to match the number of vCPUs. Other transports continue to default to 1 request virtqueue. A 1:1 virtqueue:vCPU mapping ensures that completion interrupts are handled on the same vCPU that submitted the request. No IPI is necessary to complete an I/O request and performance is improved. The maximum number of MSI-X vectors and virtqueues limit are respected. Signed-off-by: Stefan Hajnoczi Reviewed-by: Cornelia Huck --- include/hw/virtio/virtio-scsi.h | 2 ++ hw/core/machine.c | 3 +++ hw/scsi/vhost-scsi.c | 3 ++- hw/scsi/vhost-user-scsi.c | 3 ++- hw/scsi/virtio-scsi.c | 6 +++++- hw/virtio/vhost-scsi-pci.c | 10 +++++++--- hw/virtio/vhost-user-scsi-pci.c | 10 +++++++--- hw/virtio/virtio-scsi-pci.c | 10 +++++++--- 8 files changed, 35 insertions(+), 12 deletions(-) diff --git a/include/hw/virtio/virtio-scsi.h b/include/hw/virtio/virtio-scs= i.h index 9f293bcb80..c0b8e4dd7e 100644 --- a/include/hw/virtio/virtio-scsi.h +++ b/include/hw/virtio/virtio-scsi.h @@ -39,6 +39,8 @@ /* Number of virtqueues that are always present */ #define VIRTIO_SCSI_VQ_NUM_FIXED 2 =20 +#define VIRTIO_SCSI_AUTO_NUM_QUEUES UINT32_MAX + typedef struct virtio_scsi_cmd_req VirtIOSCSICmdReq; typedef struct virtio_scsi_cmd_resp VirtIOSCSICmdResp; typedef struct virtio_scsi_ctrl_tmf_req VirtIOSCSICtrlTMFReq; diff --git a/hw/core/machine.c b/hw/core/machine.c index 211b4e077a..3df534405b 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -29,7 +29,10 @@ #include "migration/vmstate.h" =20 GlobalProperty hw_compat_5_0[] =3D { + { "vhost-scsi", "num_queues", "1"}, + { "vhost-user-scsi", "num_queues", "1"}, { "virtio-balloon-device", "page-poison", "false" }, + { "virtio-scsi-device", "num_queues", "1"}, { "vmport", "x-read-set-eax", "off" }, { "vmport", "x-signal-unsupported-cmd", "off" }, { "vmport", "x-report-vmx-type", "off" }, diff --git a/hw/scsi/vhost-scsi.c b/hw/scsi/vhost-scsi.c index c1b012aea4..5e3bc319ab 100644 --- a/hw/scsi/vhost-scsi.c +++ b/hw/scsi/vhost-scsi.c @@ -272,7 +272,8 @@ static Property vhost_scsi_properties[] =3D { DEFINE_PROP_STRING("vhostfd", VirtIOSCSICommon, conf.vhostfd), DEFINE_PROP_STRING("wwpn", VirtIOSCSICommon, conf.wwpn), DEFINE_PROP_UINT32("boot_tpgt", VirtIOSCSICommon, conf.boot_tpgt, 0), - DEFINE_PROP_UINT32("num_queues", VirtIOSCSICommon, conf.num_queues, 1), + DEFINE_PROP_UINT32("num_queues", VirtIOSCSICommon, conf.num_queues, + VIRTIO_SCSI_AUTO_NUM_QUEUES), DEFINE_PROP_UINT32("virtqueue_size", VirtIOSCSICommon, conf.virtqueue_= size, 128), DEFINE_PROP_BOOL("seg_max_adjust", VirtIOSCSICommon, conf.seg_max_adju= st, diff --git a/hw/scsi/vhost-user-scsi.c b/hw/scsi/vhost-user-scsi.c index a8b821466f..7c0631656c 100644 --- a/hw/scsi/vhost-user-scsi.c +++ b/hw/scsi/vhost-user-scsi.c @@ -162,7 +162,8 @@ static void vhost_user_scsi_unrealize(DeviceState *dev) static Property vhost_user_scsi_properties[] =3D { DEFINE_PROP_CHR("chardev", VirtIOSCSICommon, conf.chardev), DEFINE_PROP_UINT32("boot_tpgt", VirtIOSCSICommon, conf.boot_tpgt, 0), - DEFINE_PROP_UINT32("num_queues", VirtIOSCSICommon, conf.num_queues, 1), + DEFINE_PROP_UINT32("num_queues", VirtIOSCSICommon, conf.num_queues, + VIRTIO_SCSI_AUTO_NUM_QUEUES), DEFINE_PROP_UINT32("virtqueue_size", VirtIOSCSICommon, conf.virtqueue_= size, 128), DEFINE_PROP_UINT32("max_sectors", VirtIOSCSICommon, conf.max_sectors, diff --git a/hw/scsi/virtio-scsi.c b/hw/scsi/virtio-scsi.c index eecdd05af5..3a71ea7097 100644 --- a/hw/scsi/virtio-scsi.c +++ b/hw/scsi/virtio-scsi.c @@ -891,6 +891,9 @@ void virtio_scsi_common_realize(DeviceState *dev, virtio_init(vdev, "virtio-scsi", VIRTIO_ID_SCSI, sizeof(VirtIOSCSIConfig)); =20 + if (s->conf.num_queues =3D=3D VIRTIO_SCSI_AUTO_NUM_QUEUES) { + s->conf.num_queues =3D 1; + } if (s->conf.num_queues =3D=3D 0 || s->conf.num_queues > VIRTIO_QUEUE_MAX - VIRTIO_SCSI_VQ_NUM_FIX= ED) { error_setg(errp, "Invalid number of queues (=3D %" PRIu32 "), " @@ -964,7 +967,8 @@ static void virtio_scsi_device_unrealize(DeviceState *d= ev) } =20 static Property virtio_scsi_properties[] =3D { - DEFINE_PROP_UINT32("num_queues", VirtIOSCSI, parent_obj.conf.num_queue= s, 1), + DEFINE_PROP_UINT32("num_queues", VirtIOSCSI, parent_obj.conf.num_queue= s, + VIRTIO_SCSI_AUTO_NUM_QUEUES), DEFINE_PROP_UINT32("virtqueue_size", VirtIOSCSI, parent_obj.conf.virtqueue_size, 2= 56), DEFINE_PROP_BOOL("seg_max_adjust", VirtIOSCSI, diff --git a/hw/virtio/vhost-scsi-pci.c b/hw/virtio/vhost-scsi-pci.c index 06e814d30e..a6bb0dc60d 100644 --- a/hw/virtio/vhost-scsi-pci.c +++ b/hw/virtio/vhost-scsi-pci.c @@ -47,11 +47,15 @@ static void vhost_scsi_pci_realize(VirtIOPCIProxy *vpci= _dev, Error **errp) { VHostSCSIPCI *dev =3D VHOST_SCSI_PCI(vpci_dev); DeviceState *vdev =3D DEVICE(&dev->vdev); - VirtIOSCSICommon *vs =3D VIRTIO_SCSI_COMMON(vdev); + VirtIOSCSIConf *conf =3D &dev->vdev.parent_obj.parent_obj.conf; + + if (conf->num_queues =3D=3D VIRTIO_SCSI_AUTO_NUM_QUEUES) { + conf->num_queues =3D + virtio_pci_optimal_num_queues(VIRTIO_SCSI_VQ_NUM_FIXED); + } =20 if (vpci_dev->nvectors =3D=3D DEV_NVECTORS_UNSPECIFIED) { - vpci_dev->nvectors =3D vs->conf.num_queues + - VIRTIO_SCSI_VQ_NUM_FIXED + 1; + vpci_dev->nvectors =3D conf->num_queues + VIRTIO_SCSI_VQ_NUM_FIXED= + 1; } =20 qdev_realize(vdev, BUS(&vpci_dev->bus), errp); diff --git a/hw/virtio/vhost-user-scsi-pci.c b/hw/virtio/vhost-user-scsi-pc= i.c index ab6dfb71a9..25e97ca54e 100644 --- a/hw/virtio/vhost-user-scsi-pci.c +++ b/hw/virtio/vhost-user-scsi-pci.c @@ -53,11 +53,15 @@ static void vhost_user_scsi_pci_realize(VirtIOPCIProxy = *vpci_dev, Error **errp) { VHostUserSCSIPCI *dev =3D VHOST_USER_SCSI_PCI(vpci_dev); DeviceState *vdev =3D DEVICE(&dev->vdev); - VirtIOSCSICommon *vs =3D VIRTIO_SCSI_COMMON(vdev); + VirtIOSCSIConf *conf =3D &dev->vdev.parent_obj.parent_obj.conf; + + if (conf->num_queues =3D=3D VIRTIO_SCSI_AUTO_NUM_QUEUES) { + conf->num_queues =3D + virtio_pci_optimal_num_queues(VIRTIO_SCSI_VQ_NUM_FIXED); + } =20 if (vpci_dev->nvectors =3D=3D DEV_NVECTORS_UNSPECIFIED) { - vpci_dev->nvectors =3D vs->conf.num_queues + - VIRTIO_SCSI_VQ_NUM_FIXED + 1; + vpci_dev->nvectors =3D conf->num_queues + VIRTIO_SCSI_VQ_NUM_FIXED= + 1; } =20 qdev_realize(vdev, BUS(&vpci_dev->bus), errp); diff --git a/hw/virtio/virtio-scsi-pci.c b/hw/virtio/virtio-scsi-pci.c index 3ff9eb7ef6..fa4b3bfb50 100644 --- a/hw/virtio/virtio-scsi-pci.c +++ b/hw/virtio/virtio-scsi-pci.c @@ -46,13 +46,17 @@ static void virtio_scsi_pci_realize(VirtIOPCIProxy *vpc= i_dev, Error **errp) { VirtIOSCSIPCI *dev =3D VIRTIO_SCSI_PCI(vpci_dev); DeviceState *vdev =3D DEVICE(&dev->vdev); - VirtIOSCSICommon *vs =3D VIRTIO_SCSI_COMMON(vdev); DeviceState *proxy =3D DEVICE(vpci_dev); + VirtIOSCSIConf *conf =3D &dev->vdev.parent_obj.conf; char *bus_name; =20 + if (conf->num_queues =3D=3D VIRTIO_SCSI_AUTO_NUM_QUEUES) { + conf->num_queues =3D + virtio_pci_optimal_num_queues(VIRTIO_SCSI_VQ_NUM_FIXED); + } + if (vpci_dev->nvectors =3D=3D DEV_NVECTORS_UNSPECIFIED) { - vpci_dev->nvectors =3D vs->conf.num_queues + - VIRTIO_SCSI_VQ_NUM_FIXED + 1; + vpci_dev->nvectors =3D conf->num_queues + VIRTIO_SCSI_VQ_NUM_FIXED= + 1; } =20 /* --=20 2.26.2 From nobody Thu May 16 03:16:30 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1594043900; cv=none; d=zohomail.com; s=zohoarc; b=Ze6bkCv5bU4D4VWcmh4QKdVMPREYI8U/cSHgDmQBTUrafhCLoNKi5jmWIQMySehdq5iWg5J7Bm1nEU4lb0pxVm6OizUnHuOg6IfHNcQXrVNYQ8LaO1XjLq8o1RIBo8DyGtzLWjtBdMsXhWGTdkYYnppg5FQRYc+IEUpsmMehMWs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1594043900; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=rVN2UUFevJOMtkf6B6bMrWRFpyvoNches0VgABND0vw=; b=K9PqoaJG8iBYg603gpR17keN9rWkSApyUjzVSi8JLXxKw7mjBWejcGbrSw21SXbaVvC0fVZfHHIfGU3vSgxSkj2adTV8ma0Ma5VWzewd7mW1mxy96OD93m2pxLuizR0GeREdU+RyWf8b78KYnZqlaB57Ye8IRq3HNpD+lmIKekI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1594043900915554.331804518803; Mon, 6 Jul 2020 06:58:20 -0700 (PDT) Received: from localhost ([::1]:33310 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jsRdT-0003LA-JK for importer@patchew.org; Mon, 06 Jul 2020 09:58:19 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46368) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jsRcY-0001ki-9L for qemu-devel@nongnu.org; Mon, 06 Jul 2020 09:57:22 -0400 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:34367 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jsRcW-0006Ty-Dc for qemu-devel@nongnu.org; Mon, 06 Jul 2020 09:57:21 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-168-4bs0ONVuO325tjyu6MJC9w-1; Mon, 06 Jul 2020 09:57:18 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2A38D64AD9; Mon, 6 Jul 2020 13:57:17 +0000 (UTC) Received: from localhost (ovpn-114-217.ams2.redhat.com [10.36.114.217]) by smtp.corp.redhat.com (Postfix) with ESMTP id 324AD60BEC; Mon, 6 Jul 2020 13:57:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594043839; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=rVN2UUFevJOMtkf6B6bMrWRFpyvoNches0VgABND0vw=; b=O1cpEBuaXTA4J9g0qb0T4LmyjLo7jUc3jK/2MI9FUAlGWA0rBdD4SP5NgS+B+NjH5Dx7LT drToZufIrmzSRsnOV/T/dNd9ckyRB2Pw969PdILaco5Ya9V1CTNjuf49F606zGtYQd+baW oW4lv19NZxA43zh+h7OjsOKn5eA/Q1I= X-MC-Unique: 4bs0ONVuO325tjyu6MJC9w-1 From: Stefan Hajnoczi To: qemu-devel@nongnu.org Subject: [PATCH v5 4/5] virtio-blk: default num_queues to -smp N Date: Mon, 6 Jul 2020 14:56:49 +0100 Message-Id: <20200706135650.438362-5-stefanha@redhat.com> In-Reply-To: <20200706135650.438362-1-stefanha@redhat.com> References: <20200706135650.438362-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=stefanha@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.61; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/06 01:59:39 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -13 X-Spam_score: -1.4 X-Spam_bar: - X-Spam_report: (-1.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MIME_BASE64_TEXT=1.741, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Eduardo Habkost , qemu-block@nongnu.org, "Michael S. Tsirkin" , cohuck@redhat.com, Max Reitz , Stefan Hajnoczi , Pankaj Gupta , Paolo Bonzini , Raphael Norwitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Automatically size the number of virtio-blk-pci request virtqueues to match the number of vCPUs. Other transports continue to default to 1 request virtqueue. A 1:1 virtqueue:vCPU mapping ensures that completion interrupts are handled on the same vCPU that submitted the request. No IPI is necessary to complete an I/O request and performance is improved. The maximum number of MSI-X vectors and virtqueues limit are respected. Performance improves from 78k to 104k IOPS on a 32 vCPU guest with 101 virtio-blk-pci devices (ioengine=3Dlibaio, iodepth=3D1, bs=3D4k, rw=3Drandr= ead with NVMe storage). Signed-off-by: Stefan Hajnoczi Reviewed-by: Cornelia Huck Reviewed-by: Pankaj Gupta --- include/hw/virtio/virtio-blk.h | 2 ++ hw/block/virtio-blk.c | 6 +++++- hw/core/machine.c | 1 + hw/virtio/virtio-blk-pci.c | 7 ++++++- 4 files changed, 14 insertions(+), 2 deletions(-) diff --git a/include/hw/virtio/virtio-blk.h b/include/hw/virtio/virtio-blk.h index b1334c3904..7539c2b848 100644 --- a/include/hw/virtio/virtio-blk.h +++ b/include/hw/virtio/virtio-blk.h @@ -30,6 +30,8 @@ struct virtio_blk_inhdr unsigned char status; }; =20 +#define VIRTIO_BLK_AUTO_NUM_QUEUES UINT16_MAX + struct VirtIOBlkConf { BlockConf conf; diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c index 413783693c..2204ba149e 100644 --- a/hw/block/virtio-blk.c +++ b/hw/block/virtio-blk.c @@ -1147,6 +1147,9 @@ static void virtio_blk_device_realize(DeviceState *de= v, Error **errp) error_setg(errp, "Device needs media, but drive is empty"); return; } + if (conf->num_queues =3D=3D VIRTIO_BLK_AUTO_NUM_QUEUES) { + conf->num_queues =3D 1; + } if (!conf->num_queues) { error_setg(errp, "num-queues property must be larger than 0"); return; @@ -1281,7 +1284,8 @@ static Property virtio_blk_properties[] =3D { #endif DEFINE_PROP_BIT("request-merging", VirtIOBlock, conf.request_merging, = 0, true), - DEFINE_PROP_UINT16("num-queues", VirtIOBlock, conf.num_queues, 1), + DEFINE_PROP_UINT16("num-queues", VirtIOBlock, conf.num_queues, + VIRTIO_BLK_AUTO_NUM_QUEUES), DEFINE_PROP_UINT16("queue-size", VirtIOBlock, conf.queue_size, 256), DEFINE_PROP_BOOL("seg-max-adjust", VirtIOBlock, conf.seg_max_adjust, t= rue), DEFINE_PROP_LINK("iothread", VirtIOBlock, conf.iothread, TYPE_IOTHREAD, diff --git a/hw/core/machine.c b/hw/core/machine.c index 3df534405b..845f6476cb 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -32,6 +32,7 @@ GlobalProperty hw_compat_5_0[] =3D { { "vhost-scsi", "num_queues", "1"}, { "vhost-user-scsi", "num_queues", "1"}, { "virtio-balloon-device", "page-poison", "false" }, + { "virtio-blk-device", "num-queues", "1"}, { "virtio-scsi-device", "num_queues", "1"}, { "vmport", "x-read-set-eax", "off" }, { "vmport", "x-signal-unsupported-cmd", "off" }, diff --git a/hw/virtio/virtio-blk-pci.c b/hw/virtio/virtio-blk-pci.c index 849cc7dfd8..37c6e0aeb4 100644 --- a/hw/virtio/virtio-blk-pci.c +++ b/hw/virtio/virtio-blk-pci.c @@ -50,9 +50,14 @@ static void virtio_blk_pci_realize(VirtIOPCIProxy *vpci_= dev, Error **errp) { VirtIOBlkPCI *dev =3D VIRTIO_BLK_PCI(vpci_dev); DeviceState *vdev =3D DEVICE(&dev->vdev); + VirtIOBlkConf *conf =3D &dev->vdev.conf; + + if (conf->num_queues =3D=3D VIRTIO_BLK_AUTO_NUM_QUEUES) { + conf->num_queues =3D virtio_pci_optimal_num_queues(0); + } =20 if (vpci_dev->nvectors =3D=3D DEV_NVECTORS_UNSPECIFIED) { - vpci_dev->nvectors =3D dev->vdev.conf.num_queues + 1; + vpci_dev->nvectors =3D conf->num_queues + 1; } =20 qdev_realize(vdev, BUS(&vpci_dev->bus), errp); --=20 2.26.2 From nobody Thu May 16 03:16:30 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1594044111; cv=none; d=zohomail.com; s=zohoarc; b=FXGrSVKKvEcvLzuu8zwr/maZkLqEIT/6n0atYdeInLhhxDZaQQR6QhI5U9073svhrKVN916SdeGDBPRWJhi019FXgyaTUsqHz9+UObg8+n+Zc5oJdoL2AqiUuL4ZcHtL1fur1skbFqUOT3/PCI2uzvfRNk9X5pX6WHd3o2/hwFA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1594044111; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=lYGCi0QB0SAc/mJAncRYzQLFnJOLAK+wSPTimfQliR0=; b=TmGxVGCcNPDW6CNXCzM+cIEAPRd8sC8vgAf0DWcvHlA6aeJbHvnIBJxgCGaghHfha9owJ88HhtN23z+V9f/YOeBpEsdZYmZqTteNCEi/zKKll6Zwlkt26bJOuXs7Nw/e37EtmM56QS90MguTedGq4VRbLpj1yVHcM893EaP4rn0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1594044111481614.2709546215486; Mon, 6 Jul 2020 07:01:51 -0700 (PDT) Received: from localhost ([::1]:45402 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jsRgb-0000F2-LM for importer@patchew.org; Mon, 06 Jul 2020 10:01:33 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:46440) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jsRci-00029K-Uq for qemu-devel@nongnu.org; Mon, 06 Jul 2020 09:57:32 -0400 Received: from us-smtp-2.mimecast.com ([205.139.110.61]:33397 helo=us-smtp-delivery-1.mimecast.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1jsRcg-0006V2-GI for qemu-devel@nongnu.org; Mon, 06 Jul 2020 09:57:32 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-350-tqlJc260MWelnA8ws-NxAw-1; Mon, 06 Jul 2020 09:57:26 -0400 Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E9AD219200C0; Mon, 6 Jul 2020 13:57:24 +0000 (UTC) Received: from localhost (ovpn-114-217.ams2.redhat.com [10.36.114.217]) by smtp.corp.redhat.com (Postfix) with ESMTP id BF539512FE; Mon, 6 Jul 2020 13:57:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1594043849; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=lYGCi0QB0SAc/mJAncRYzQLFnJOLAK+wSPTimfQliR0=; b=HKx7c2AyG+K8e0k9QvMmltp+XE+2rl0u7R9TIV7ZZbI3q8dHRrLVQSoGYCmUlTdjcAkPNb RG/f14JJjAIoXwuOAV632f/CCOAJOBrR9rTDztOjERq3yANiZsJdqd12uOcq2T0ARx+pqB 8kM0eEdRBPzLyXTyf7eMbC+688CJco0= X-MC-Unique: tqlJc260MWelnA8ws-NxAw-1 From: Stefan Hajnoczi To: qemu-devel@nongnu.org Subject: [PATCH v5 5/5] vhost-user-blk: default num_queues to -smp N Date: Mon, 6 Jul 2020 14:56:50 +0100 Message-Id: <20200706135650.438362-6-stefanha@redhat.com> In-Reply-To: <20200706135650.438362-1-stefanha@redhat.com> References: <20200706135650.438362-1-stefanha@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=stefanha@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=205.139.110.61; envelope-from=stefanha@redhat.com; helo=us-smtp-delivery-1.mimecast.com X-detected-operating-system: by eggs.gnu.org: First seen = 2020/07/06 01:59:39 X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy] X-Spam_score_int: -13 X-Spam_score: -1.4 X-Spam_bar: - X-Spam_report: (-1.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-1, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, MIME_BASE64_TEXT=1.741, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=_AUTOLEARN X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Kevin Wolf , Fam Zheng , Eduardo Habkost , qemu-block@nongnu.org, "Michael S. Tsirkin" , cohuck@redhat.com, Max Reitz , Stefan Hajnoczi , Pankaj Gupta , Paolo Bonzini , Raphael Norwitz Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Content-Type: text/plain; charset="utf-8" Automatically size the number of request virtqueues to match the number of vCPUs. This ensures that completion interrupts are handled on the same vCPU that submitted the request. No IPI is necessary to complete an I/O request and performance is improved. The maximum number of MSI-X vectors and virtqueues limit are respected. Signed-off-by: Stefan Hajnoczi Reviewed-by: Cornelia Huck Reviewed-by: Raphael Norwitz --- include/hw/virtio/vhost-user-blk.h | 2 ++ hw/block/vhost-user-blk.c | 6 +++++- hw/core/machine.c | 1 + hw/virtio/vhost-user-blk-pci.c | 4 ++++ 4 files changed, 12 insertions(+), 1 deletion(-) diff --git a/include/hw/virtio/vhost-user-blk.h b/include/hw/virtio/vhost-u= ser-blk.h index 34ad6f0c0e..292d17147c 100644 --- a/include/hw/virtio/vhost-user-blk.h +++ b/include/hw/virtio/vhost-user-blk.h @@ -25,6 +25,8 @@ #define VHOST_USER_BLK(obj) \ OBJECT_CHECK(VHostUserBlk, (obj), TYPE_VHOST_USER_BLK) =20 +#define VHOST_USER_BLK_AUTO_NUM_QUEUES UINT16_MAX + typedef struct VHostUserBlk { VirtIODevice parent_obj; CharBackend chardev; diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c index a00b854736..39aec42dae 100644 --- a/hw/block/vhost-user-blk.c +++ b/hw/block/vhost-user-blk.c @@ -420,6 +420,9 @@ static void vhost_user_blk_device_realize(DeviceState *= dev, Error **errp) return; } =20 + if (s->num_queues =3D=3D VHOST_USER_BLK_AUTO_NUM_QUEUES) { + s->num_queues =3D 1; + } if (!s->num_queues || s->num_queues > VIRTIO_QUEUE_MAX) { error_setg(errp, "vhost-user-blk: invalid number of IO queues"); return; @@ -531,7 +534,8 @@ static const VMStateDescription vmstate_vhost_user_blk = =3D { =20 static Property vhost_user_blk_properties[] =3D { DEFINE_PROP_CHR("chardev", VHostUserBlk, chardev), - DEFINE_PROP_UINT16("num-queues", VHostUserBlk, num_queues, 1), + DEFINE_PROP_UINT16("num-queues", VHostUserBlk, num_queues, + VHOST_USER_BLK_AUTO_NUM_QUEUES), DEFINE_PROP_UINT32("queue-size", VHostUserBlk, queue_size, 128), DEFINE_PROP_BIT("config-wce", VHostUserBlk, config_wce, 0, true), DEFINE_PROP_END_OF_LIST(), diff --git a/hw/core/machine.c b/hw/core/machine.c index 845f6476cb..31bfaacdb5 100644 --- a/hw/core/machine.c +++ b/hw/core/machine.c @@ -30,6 +30,7 @@ =20 GlobalProperty hw_compat_5_0[] =3D { { "vhost-scsi", "num_queues", "1"}, + { "vhost-user-blk", "num-queues", "1"}, { "vhost-user-scsi", "num_queues", "1"}, { "virtio-balloon-device", "page-poison", "false" }, { "virtio-blk-device", "num-queues", "1"}, diff --git a/hw/virtio/vhost-user-blk-pci.c b/hw/virtio/vhost-user-blk-pci.c index 4f5d5cbf44..a62a71e067 100644 --- a/hw/virtio/vhost-user-blk-pci.c +++ b/hw/virtio/vhost-user-blk-pci.c @@ -54,6 +54,10 @@ static void vhost_user_blk_pci_realize(VirtIOPCIProxy *v= pci_dev, Error **errp) VHostUserBlkPCI *dev =3D VHOST_USER_BLK_PCI(vpci_dev); DeviceState *vdev =3D DEVICE(&dev->vdev); =20 + if (dev->vdev.num_queues =3D=3D VHOST_USER_BLK_AUTO_NUM_QUEUES) { + dev->vdev.num_queues =3D virtio_pci_optimal_num_queues(0); + } + if (vpci_dev->nvectors =3D=3D DEV_NVECTORS_UNSPECIFIED) { vpci_dev->nvectors =3D dev->vdev.num_queues + 1; } --=20 2.26.2