From nobody Mon May 5 21:21:36 2025
Delivered-To: importer@patchew.org
Authentication-Results: mx.zohomail.com;
dkim=fail;
spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as
permitted sender)
smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org;
dmarc=fail(p=none dis=none) header.from=redhat.com
ARC-Seal: i=1; a=rsa-sha256; t=1603367502; cv=none;
d=zohomail.com; s=zohoarc;
b=la7Xd/w0nN1gTWeDABri1TJ277wagT0WwyO/YoCNCvDoChSwgM7TI4HcDGejtZmIrM/Tb8e/cKK5CiO1mTeL+WDcc9JVEhQWxsK8UBgotiY6bzFznJn0ohkMJRlwmTazxEKcOkPQVY1N/R/eJ6ewm4SU9fH16P4WZy0NQcghuSQ=
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com;
s=zohoarc;
t=1603367502;
h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To;
bh=PDGU8gIk8Pi7rgM0O/1+uxeO4tyTqhwMmraME3uYtfY=;
b=FhZUTUbq7+/Jow09btRqh/D5isc2my6yQDw6eXSuZ7l9IC9TICQlgiVkEAhhaa7x7J5MVlmjCQZnaHeg1W+CJvTFfRAQ/E36EK3GcJzuwzC70g0HVGbxIbCNbkoU4pEWMkjKPeZt/wTr/7mzzgoqkShawAxwNIO4zaapDP9vUlI=
ARC-Authentication-Results: i=1; mx.zohomail.com;
dkim=fail;
spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as
permitted sender)
smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org;
dmarc=fail header.from=<stefanha@redhat.com> (p=none dis=none)
header.from=<stefanha@redhat.com>
Return-Path: <qemu-devel-bounces+importer=patchew.org@nongnu.org>
Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by
mx.zohomail.com
with SMTPS id 1603367502911290.6508606573492;
Thu, 22 Oct 2020 04:51:42 -0700 (PDT)
Received: from localhost ([::1]:33190 helo=lists1p.gnu.org)
by lists.gnu.org with esmtp (Exim 4.90_1)
(envelope-from <qemu-devel-bounces+importer=patchew.org@nongnu.org>)
id 1kVZ89-0007bM-P7
for importer@patchew.org; Thu, 22 Oct 2020 07:51:41 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:36872)
by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <stefanha@redhat.com>)
id 1kVYmv-0006Tv-1X
for qemu-devel@nongnu.org; Thu, 22 Oct 2020 07:29:45 -0400
Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:25159)
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256)
(Exim 4.90_1) (envelope-from <stefanha@redhat.com>)
id 1kVYmp-0005Bn-0g
for qemu-devel@nongnu.org; Thu, 22 Oct 2020 07:29:42 -0400
Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com
[209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id
us-mta-493-RLZ2uVi9NuqCMYnPMKD10A-1; Thu, 22 Oct 2020 07:29:36 -0400
Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com
[10.5.11.22])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 45BA1AD741;
Thu, 22 Oct 2020 11:29:35 +0000 (UTC)
Received: from localhost (ovpn-114-229.ams2.redhat.com [10.36.114.229])
by smtp.corp.redhat.com (Postfix) with ESMTP id CD01110013D0;
Thu, 22 Oct 2020 11:29:34 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com;
s=mimecast20190719; t=1603366178;
h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
to:to:cc:cc:mime-version:mime-version:content-type:content-type:
content-transfer-encoding:content-transfer-encoding:
in-reply-to:in-reply-to:references:references;
bh=PDGU8gIk8Pi7rgM0O/1+uxeO4tyTqhwMmraME3uYtfY=;
b=DTyD6vlUdEKy274Z8IYDJJ9H5bJR2iUmn0VQRc0nnkZrWgpp8zuoKzPVP0KZKo8K8G4inK
8Nx6MKG/kDbGw7S+fPLgIF4FDcnz1F3chxeSx/tLeGEwGCV4DSv11HvvpTWm14+3HrWhWY
cap5jTE9MhZpXkl0245i3SvPS73lkvU=
X-MC-Unique: RLZ2uVi9NuqCMYnPMKD10A-1
From: Stefan Hajnoczi <stefanha@redhat.com>
To: qemu-devel@nongnu.org,
Peter Maydell <peter.maydell@linaro.org>
Subject: [PULL v2 23/28] block/export: add vhost-user-blk multi-queue support
Date: Thu, 22 Oct 2020 12:27:21 +0100
Message-Id: <20201022112726.736757-24-stefanha@redhat.com>
In-Reply-To: <20201022112726.736757-1-stefanha@redhat.com>
References: <20201022112726.736757-1-stefanha@redhat.com>
MIME-Version: 1.0
X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22
Authentication-Results: relay.mimecast.com;
auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=stefanha@redhat.com
X-Mimecast-Spam-Score: 0
X-Mimecast-Originator: redhat.com
Content-Transfer-Encoding: quoted-printable
Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17
as permitted sender) client-ip=209.51.188.17;
envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org;
helo=lists.gnu.org;
Received-SPF: pass client-ip=216.205.24.124;
envelope-from=stefanha@redhat.com;
helo=us-smtp-delivery-124.mimecast.com
X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/22 00:54:46
X-ACL-Warn: Detected OS = Linux 2.2.x-3.x [generic] [fuzzy]
X-Spam_score_int: -3
X-Spam_score: -0.4
X-Spam_bar: /
X-Spam_report: (-0.4 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001,
DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
MIME_BASE64_TEXT=1.741, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001,
RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001,
SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-BeenThere: qemu-devel@nongnu.org
X-Mailman-Version: 2.1.23
Precedence: list
List-Id: <qemu-devel.nongnu.org>
List-Unsubscribe: <https://lists.nongnu.org/mailman/options/qemu-devel>,
<mailto:qemu-devel-request@nongnu.org?subject=unsubscribe>
List-Archive: <https://lists.nongnu.org/archive/html/qemu-devel>
List-Post: <mailto:qemu-devel@nongnu.org>
List-Help: <mailto:qemu-devel-request@nongnu.org?subject=help>
List-Subscribe: <https://lists.nongnu.org/mailman/listinfo/qemu-devel>,
<mailto:qemu-devel-request@nongnu.org?subject=subscribe>
Cc: Kevin Wolf <kwolf@redhat.com>, Fam Zheng <fam@euphon.net>,
=?UTF-8?q?Daniel=20P=2E=20Berrang=C3=A9?= <berrange@redhat.com>,
Eduardo Habkost <ehabkost@redhat.com>, qemu-block@nongnu.org,
"Dr. David Alan Gilbert" <dgilbert@redhat.com>,
Coiby Xu <Coiby.Xu@gmail.com>,
Markus Armbruster <armbru@redhat.com>, Stefan Hajnoczi <stefanha@redhat.com>,
Paolo Bonzini <pbonzini@redhat.com>, Max Reitz <mreitz@redhat.com>
Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org
Sender: "Qemu-devel" <qemu-devel-bounces+importer=patchew.org@nongnu.org>
X-ZohoMail-DKIM: fail (Header signature does not verify)
Content-Type: text/plain; charset="utf-8"
Allow the number of queues to be configured using --export
vhost-user-blk,num-queues=3DN. This setting should match the QEMU --device
vhost-user-blk-pci,num-queues=3DN setting but QEMU vhost-user-blk.c lowers
its own value if the vhost-user-blk backend offers fewer queues than
QEMU.
The vhost-user-blk-server.c code is already capable of multi-queue. All
virtqueue processing runs in the same AioContext. No new locking is
needed.
Add the num-queues=3DN option and set the VIRTIO_BLK_F_MQ feature bit.
Note that the feature bit only announces the presence of the num_queues
configuration space field. It does not promise that there is more than 1
virtqueue, so we can set it unconditionally.
I tested multi-queue by running a random read fio test with numjobs=3D4 on
an -smp 4 guest. After the benchmark finished the guest /proc/interrupts
file showed activity on all 4 virtio-blk MSI-X. The /sys/block/vda/mq/
directory shows that Linux blk-mq has 4 queues configured.
An automated test is included in the next commit.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Message-id: 20201001144604.559733-2-stefanha@redhat.com
[Fixed accidental tab characters as suggested by Markus Armbruster
--Stefan]
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
qapi/block-export.json | 10 +++++++---
block/export/vhost-user-blk-server.c | 24 ++++++++++++++++++------
2 files changed, 25 insertions(+), 9 deletions(-)
diff --git a/qapi/block-export.json b/qapi/block-export.json
index 8a4ced817f..480c497690 100644
--- a/qapi/block-export.json
+++ b/qapi/block-export.json
@@ -93,11 +93,15 @@
# SocketAddress types are supported. Passed fds must be UNIX domain
# sockets.
# @logical-block-size: Logical block size in bytes. Defaults to 512 bytes.
+# @num-queues: Number of request virtqueues. Must be greater than 0. Defau=
lts
+# to 1.
#
# Since: 5.2
##
{ 'struct': 'BlockExportOptionsVhostUserBlk',
- 'data': { 'addr': 'SocketAddress', '*logical-block-size': 'size' } }
+ 'data': { 'addr': 'SocketAddress',
+ '*logical-block-size': 'size',
+ '*num-queues': 'uint16'} }
=20
##
# @NbdServerAddOptions:
@@ -233,8 +237,8 @@
{ 'union': 'BlockExportOptions',
'base': { 'type': 'BlockExportType',
'id': 'str',
- '*fixed-iothread': 'bool',
- '*iothread': 'str',
+ '*fixed-iothread': 'bool',
+ '*iothread': 'str',
'node-name': 'str',
'*writable': 'bool',
'*writethrough': 'bool' },
diff --git a/block/export/vhost-user-blk-server.c b/block/export/vhost-user=
-blk-server.c
index 286eb5fb9a..41f4933d6e 100644
--- a/block/export/vhost-user-blk-server.c
+++ b/block/export/vhost-user-blk-server.c
@@ -21,7 +21,7 @@
#include "util/block-helpers.h"
=20
enum {
- VHOST_USER_BLK_MAX_QUEUES =3D 1,
+ VHOST_USER_BLK_NUM_QUEUES_DEFAULT =3D 1,
};
struct virtio_blk_inhdr {
unsigned char status;
@@ -242,6 +242,7 @@ static uint64_t vu_blk_get_features(VuDev *dev)
1ull << VIRTIO_BLK_F_DISCARD |
1ull << VIRTIO_BLK_F_WRITE_ZEROES |
1ull << VIRTIO_BLK_F_CONFIG_WCE |
+ 1ull << VIRTIO_BLK_F_MQ |
1ull << VIRTIO_F_VERSION_1 |
1ull << VIRTIO_RING_F_INDIRECT_DESC |
1ull << VIRTIO_RING_F_EVENT_IDX |
@@ -338,7 +339,9 @@ static void blk_aio_detach(void *opaque)
=20
static void
vu_blk_initialize_config(BlockDriverState *bs,
- struct virtio_blk_config *config, uint32_t blk_=
size)
+ struct virtio_blk_config *config,
+ uint32_t blk_size,
+ uint16_t num_queues)
{
config->capacity =3D bdrv_getlength(bs) >> BDRV_SECTOR_BITS;
config->blk_size =3D blk_size;
@@ -346,7 +349,7 @@ vu_blk_initialize_config(BlockDriverState *bs,
config->seg_max =3D 128 - 2;
config->min_io_size =3D 1;
config->opt_io_size =3D 1;
- config->num_queues =3D VHOST_USER_BLK_MAX_QUEUES;
+ config->num_queues =3D num_queues;
config->max_discard_sectors =3D 32768;
config->max_discard_seg =3D 1;
config->discard_sector_alignment =3D config->blk_size >> 9;
@@ -368,6 +371,7 @@ static int vu_blk_exp_create(BlockExport *exp, BlockExp=
ortOptions *opts,
BlockExportOptionsVhostUserBlk *vu_opts =3D &opts->u.vhost_user_blk;
Error *local_err =3D NULL;
uint64_t logical_block_size;
+ uint16_t num_queues =3D VHOST_USER_BLK_NUM_QUEUES_DEFAULT;
=20
vexp->writable =3D opts->writable;
vexp->blkcfg.wce =3D 0;
@@ -385,15 +389,23 @@ static int vu_blk_exp_create(BlockExport *exp, BlockE=
xportOptions *opts,
}
vexp->blk_size =3D logical_block_size;
blk_set_guest_block_size(exp->blk, logical_block_size);
+
+ if (vu_opts->has_num_queues) {
+ num_queues =3D vu_opts->num_queues;
+ }
+ if (num_queues =3D=3D 0) {
+ error_setg(errp, "num-queues must be greater than 0");
+ return -EINVAL;
+ }
+
vu_blk_initialize_config(blk_bs(exp->blk), &vexp->blkcfg,
- logical_block_size);
+ logical_block_size, num_queues);
=20
blk_add_aio_context_notifier(exp->blk, blk_aio_attached, blk_aio_detac=
h,
vexp);
=20
if (!vhost_user_server_start(&vexp->vu_server, vu_opts->addr, exp->ctx,
- VHOST_USER_BLK_MAX_QUEUES, &vu_blk_iface,
- errp)) {
+ num_queues, &vu_blk_iface, errp)) {
blk_remove_aio_context_notifier(exp->blk, blk_aio_attached,
blk_aio_detach, vexp);
return -EADDRNOTAVAIL;
--=20
2.26.2