From nobody Tue May 14 17:24:35 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=nongnu.org ARC-Seal: i=1; a=rsa-sha256; t=1668783037; cv=none; d=zohomail.com; s=zohoarc; b=eE5dAByF34MeQKN9bLVj6QoKdBMcTUt2x8DTlVrPVZAMREihzniDAGwDnjGrvuaG7F9Ve/pdv2SgmKtr84yXU0M8hQDoKq0VX2XUWeESJKZFqjcCmWmQyFfZ0YyG96uhJ+haolaP3qXNJe4oh5J77nMTbGHaWMXsnUXHdMwVfPM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1668783037; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:Reply-To:Sender:Subject:To; bh=7LFVcOfRTuPBJW1DcVhzRZMswEavIEVo6WH/+nKyxDs=; b=UOIN9h+VZ0Gke9ftAGggyOODAFacDrysGSSB9pWl+sKej0i1+vi+MVVjKjlFERVo2GO5hzf5SV8bfSidf9TFxl/8tKb1giI1bfMU8t+qQK3+gdjyR7f09UjZ0bBHPthdZLsxyIyKGsIzf97nuKNKvhe7+ihQIN2spDCkVV51Qgc= ARC-Authentication-Results: i=1; mx.zohomail.com; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1668783037864360.142274661703; Fri, 18 Nov 2022 06:50:37 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ow2gq-0007hv-Jd; Fri, 18 Nov 2022 09:50:01 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ow2gZ-0007gS-D1 for qemu-devel@nongnu.org; Fri, 18 Nov 2022 09:49:53 -0500 Received: from szxga03-in.huawei.com ([45.249.212.189]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ow2gV-0005ky-HS for qemu-devel@nongnu.org; Fri, 18 Nov 2022 09:49:43 -0500 Received: from kwepemi100025.china.huawei.com (unknown [172.30.72.53]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4NDKNL05kKzFqL8; Fri, 18 Nov 2022 22:46:09 +0800 (CST) Received: from DESKTOP-27KDQMV.china.huawei.com (10.174.148.223) by kwepemi100025.china.huawei.com (7.221.188.158) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.31; Fri, 18 Nov 2022 22:49:20 +0800 To: , , , CC: , , , , , , Longpeng Subject: [PATCH] vhost: configure all host notifiers in a single MR transaction Date: Fri, 18 Nov 2022 22:49:15 +0800 Message-ID: <20221118144915.2009-1-longpeng2@huawei.com> X-Mailer: git-send-email 2.25.0.windows.1 MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.174.148.223] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To kwepemi100025.china.huawei.com (7.221.188.158) X-CFilter-Loop: Reflected Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=45.249.212.189; envelope-from=longpeng2@huawei.com; helo=szxga03-in.huawei.com X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-to: "Longpeng(Mike)" From: "Longpeng(Mike)" via Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZM-MESSAGEID: 1668783039143100001 Content-Type: text/plain; charset="utf-8" From: Longpeng This allows the vhost device to batch the setup of all its host notifiers. This significantly reduces the device starting time, e.g. the vhost-vDPA generic device [1] start time reduce from 376ms to 9.1ms for a VM with 64 vCPUs and 3 vDPA device(64vq per device). [1] https://www.mail-archive.com/qemu-devel@nongnu.org/msg921541.html Signed-off-by: Longpeng Reviewed-by: Stefan Hajnoczi --- hw/virtio/vhost.c | 39 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 38 insertions(+), 1 deletion(-) diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index d1c4c20b8c..bf82d9b176 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -1507,6 +1507,7 @@ void vhost_dev_cleanup(struct vhost_dev *hdev) int vhost_dev_enable_notifiers(struct vhost_dev *hdev, VirtIODevice *vdev) { BusState *qbus =3D BUS(qdev_get_parent_bus(DEVICE(vdev))); + int vq_init_count =3D 0; int i, r, e; =20 /* We will pass the notifiers to the kernel, make sure that QEMU @@ -1518,6 +1519,12 @@ int vhost_dev_enable_notifiers(struct vhost_dev *hde= v, VirtIODevice *vdev) goto fail; } =20 + /* + * Batch all the host notifiers in a single transaction to avoid + * quadratic time complexity in address_space_update_ioeventfds(). + */ + memory_region_transaction_begin(); + for (i =3D 0; i < hdev->nvqs; ++i) { r =3D virtio_bus_set_host_notifier(VIRTIO_BUS(qbus), hdev->vq_inde= x + i, true); @@ -1525,19 +1532,33 @@ int vhost_dev_enable_notifiers(struct vhost_dev *hd= ev, VirtIODevice *vdev) error_report("vhost VQ %d notifier binding failed: %d", i, -r); goto fail_vq; } + + vq_init_count++; } =20 + memory_region_transaction_commit(); + return 0; fail_vq: - while (--i >=3D 0) { + for (i =3D 0; i < vq_init_count; i++) { e =3D virtio_bus_set_host_notifier(VIRTIO_BUS(qbus), hdev->vq_inde= x + i, false); if (e < 0) { error_report("vhost VQ %d notifier cleanup error: %d", i, -r); } assert (e >=3D 0); + } + + /* + * The transaction expects the ioeventfds to be open when it + * commits. Do it now, before the cleanup loop. + */ + memory_region_transaction_commit(); + + for (i =3D 0; i < vq_init_count; i++) { virtio_bus_cleanup_host_notifier(VIRTIO_BUS(qbus), hdev->vq_index = + i); } + virtio_device_release_ioeventfd(vdev); fail: return r; @@ -1553,6 +1574,12 @@ void vhost_dev_disable_notifiers(struct vhost_dev *h= dev, VirtIODevice *vdev) BusState *qbus =3D BUS(qdev_get_parent_bus(DEVICE(vdev))); int i, r; =20 + /* + * Batch all the host notifiers in a single transaction to avoid + * quadratic time complexity in address_space_update_ioeventfds(). + */ + memory_region_transaction_begin(); + for (i =3D 0; i < hdev->nvqs; ++i) { r =3D virtio_bus_set_host_notifier(VIRTIO_BUS(qbus), hdev->vq_inde= x + i, false); @@ -1560,8 +1587,18 @@ void vhost_dev_disable_notifiers(struct vhost_dev *h= dev, VirtIODevice *vdev) error_report("vhost VQ %d notifier cleanup failed: %d", i, -r); } assert (r >=3D 0); + } + + /* + * The transaction expects the ioeventfds to be open when it + * commits. Do it now, before the cleanup loop. + */ + memory_region_transaction_commit(); + + for (i =3D 0; i < hdev->nvqs; ++i) { virtio_bus_cleanup_host_notifier(VIRTIO_BUS(qbus), hdev->vq_index = + i); } + virtio_device_release_ioeventfd(vdev); } =20 --=20 2.23.0