From nobody Fri May 3 02:25:11 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1667794476; cv=pass; d=zohomail.com; s=zohoarc; b=dnLebY3kQAVe9Hx2tfNow+ppdK43a73ajJvIkLFyPJQkjVAuj0ngY+mTpI2XeJ8/ILAATyyfDuQqTTVwZQ6skyHNIicrAf5OTYYS8ameIUTI4P/IBv8OukxB/CRBYl1vPpjFl89oN4FtYSjU9bhT4+B0YWCXhfmdnGn0E0IaMJg= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1667794476; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=GAZHzXwUDhmCQ5uIEE7zyz3xdGGGkWJX02k/u3/9hAE=; b=AOEGuJl3JqcLlJ5w2/JCze4O44Bi2EmxCcWFFLWFDVPAQb5/4SwFWOWFsStveEW85rNPbUGBA8U8JENSE1Sn6E4rTeDx+YJtdBKg47nMPtUmNyG+/gSNDuxxj/PXu3EoFhoQ451afBFBlSbd80TzexbehX44Pw9LRVmjR3BQgkI= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1667794476255246.23648916683373; Sun, 6 Nov 2022 20:14:36 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ortWH-0002Qy-ML; Sun, 06 Nov 2022 23:13:57 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ortWF-0002Q4-Cl for qemu-devel@nongnu.org; Sun, 06 Nov 2022 23:13:55 -0500 Received: from mail-mw2nam12on2061f.outbound.protection.outlook.com ([2a01:111:f400:fe5a::61f] helo=NAM12-MW2-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ortWC-0005Fj-Jf for qemu-devel@nongnu.org; Sun, 06 Nov 2022 23:13:55 -0500 Received: from DM6PR04CA0016.namprd04.prod.outlook.com (2603:10b6:5:334::21) by BN9PR12MB5194.namprd12.prod.outlook.com (2603:10b6:408:11b::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.26; Mon, 7 Nov 2022 04:08:48 +0000 Received: from DM6NAM11FT034.eop-nam11.prod.protection.outlook.com (2603:10b6:5:334:cafe::a) by DM6PR04CA0016.outlook.office365.com (2603:10b6:5:334::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.25 via Frontend Transport; Mon, 7 Nov 2022 04:08:48 +0000 Received: from mail.nvidia.com (216.228.117.161) by DM6NAM11FT034.mail.protection.outlook.com (10.13.173.47) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.20 via Frontend Transport; Mon, 7 Nov 2022 04:08:47 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Sun, 6 Nov 2022 20:08:40 -0800 Received: from nvidia.com (10.126.230.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Sun, 6 Nov 2022 20:08:39 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JK7ma4klIVfrmHijZ94ImHiwcxDXdqiO2rEGtDNqVm2+AqRyOGriGGe5w2bMtqwUm/1rsYRR1mZ2Yd4jidqrz7nq7P2QGv+95scqGaZObCcKx5aDJpSCAS57bs076xWHkMseBWAF9P+Fbi3mqkEHy5mbDx1x2C/hxOP7IS1Ebf6JVGm4G+ds54sniNiRkm+zQscawI6a37P+yQN6Kyf8IR3PDjajg/O4McvWNPvy3rYDy7sKFjHJowL54t/wLfgdLgY1eqYyColr/RfL+76e1xnvoJj9FXsPpb3oJZ7ySfPsrbZ2xj3l1holGicxu0GVo3qbKARZ2J1Ci6pJJuCnsg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=GAZHzXwUDhmCQ5uIEE7zyz3xdGGGkWJX02k/u3/9hAE=; b=BvsMblX6X+GgGuoKMVcVzS63AFLXsIMHOD9PAHQeHRcRgYojpFgCrTEpZOT63vou97nBmKUT3bmJoqU7KZUWgb9UdYeWl7Djw9NlyesmSYKg1xyWgD07beywuR8kgClNf6ZKT2gaxb0HR8L7jz9iagHy37pwR7ZOc26jFYQdRIgb6CBxLo1RjPHGJdQbVX3nKaaRybbFNpHfmK7UWT/9hz47rs6KQpsOEbLTSpKe5KNGwvClLD9+zg3cj06idYZ0cKCOj4mS8e7twkKLtzKiU/TUxzLqJ9EEVMjFfQ8l86H9xEtneFcoZ35iuYMkb5o6TX8sT832lLgCLO6WOOdyqA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=GAZHzXwUDhmCQ5uIEE7zyz3xdGGGkWJX02k/u3/9hAE=; b=jVPTcO9rouOZpE/ezUEIXov0Oiep9rqYsciQm0M+ZM0eBvwOhtA8nQvOomCDrAr08g+MFV4f100d/mLkuwnQ3vSF2ioH7ioPlJLgAmG83dVck73BTuowfgQW2kyCfU6TowVrVtLyEjIyMQvOTkWdBI/7+hJVxoWpOubp6DVp8XKTx/Hrq3dLC4i6KW2Auyc3/9Y2kJlQ/FSC+3/2RUOMw54qh//i+RnaAQFSYKAm7D9PgpTPP1MnNmEgXow5YqOlx6DvcNUa4LKpLCRiu/VWfo5+W9/xMo38/Y3R9N2i4LK8nm3wI97QVtX7dG3EE/jZSOX6oeeZDJVyxzf9dRc1ig== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C From: Yajun Wu To: , , , Subject: [PATCH v4 1/2] vhost: Change the sequence of device start Date: Mon, 7 Nov 2022 12:08:17 +0800 Message-ID: <20221107040818.619072-2-yajunw@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20221107040818.619072-1-yajunw@nvidia.com> References: <20220629022517.2600911-1-yajunw@nvidia.com> <20221107040818.619072-1-yajunw@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.126.230.35] X-ClientProxiedBy: rnnvmail201.nvidia.com (10.129.68.8) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT034:EE_|BN9PR12MB5194:EE_ X-MS-Office365-Filtering-Correlation-Id: cc68067f-0fce-4559-2dce-08dac075c51e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: YGd3jR/S1mSMquejQ1EnTSJc0rPeKU6UxXTF2z3OFLo1J9sxJXyWk9wBAzYYTWSaeyyKoSxXJ8iigwId0l5lD0MFeDjY8UJdRahqFkSsQY/QTw7BmKgb5Db6Yf9UJ6wxB9iC5DYGJ30UUHESBYRisHBRqlL1O/y8vkxJtiEPA06ZJhGPwrMbXm0XSncd2xMimycQdUaIcebtHR4GlV9CHnekc3zIY6C7D+E1nb1pDpxbwg+sCSy5M6nzNt+44WWs2kJyURUs4rlbyjulYMO/fA2V4ZJIc1FW2NGmEOqdRNEYlgNcsKURKsVYnBRUTCR9tXUxbQivM+f8tPKmgJ3B24wLiwSuDESlfnNFdblv3mtGgHyGv/QM9Q1pri7gt0wZJTZVFjF7a0Ew99offQjW/iYFcJIP8xnC1+uqr0foq2XYn2BgQRdFrrJO39I/3DDHB378C5CnYYkF4MjKnLL7H0v0ZNSUQeloL+1RKSq8VoPlwlBrgjEIlIrUkZSrzobv8RmD3KkGXHzXzTSFztBwz6YtDSNqWgoKct/5qRuTBwrvB5n50qxdkP1LzT+Q+CRGk9DDdZvkdYwKf1M25VfMvygo0KlGxdHJtts4VJWZ4xyQvcuMdpEF3SuhHKqFcudDsJs+pqwyMewqb0n7Cb5kf1rpiXAFqrSP28leT0KUtTzbLisNpTKq1DZLwe8u9nSD6Wyzyw2PiW3+vZSmzzEormKu52F9iSloAYY/Qj+IXttxmgkvGr3Zf7f4/OOG6AD32M9H+6Hwow4SpUciMyrp6w== X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(39860400002)(346002)(376002)(396003)(136003)(451199015)(36840700001)(40470700004)(46966006)(336012)(83380400001)(426003)(356005)(7636003)(47076005)(86362001)(36860700001)(82740400003)(5660300002)(2906002)(8936002)(41300700001)(82310400005)(40480700001)(55016003)(6666004)(7696005)(478600001)(2616005)(6286002)(16526019)(1076003)(186003)(26005)(40460700003)(110136005)(316002)(70206006)(70586007)(8676002)(6636002)(36756003); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Nov 2022 04:08:47.5580 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cc68067f-0fce-4559-2dce-08dac075c51e X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT034.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN9PR12MB5194 Received-SPF: softfail client-ip=2a01:111:f400:fe5a::61f; envelope-from=yajunw@nvidia.com; helo=NAM12-MW2-obe.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1667794477130100003 Content-Type: text/plain; charset="utf-8" This patch is part of adding vhost-user vhost_dev_start support. The motivation is to improve backend configuration speed and reduce live migration VM downtime. Moving the device start routines after finishing all the necessary device and VQ configuration, further aligning to the virtio specification for "device initialization sequence". Following patch will add vhost-user vhost_dev_start support. Signed-off-by: Yajun Wu Acked-by: Parav Pandit --- hw/block/vhost-user-blk.c | 18 +++++++++++------- hw/net/vhost_net.c | 14 ++++++++------ 2 files changed, 19 insertions(+), 13 deletions(-) diff --git a/hw/block/vhost-user-blk.c b/hw/block/vhost-user-blk.c index 13bf5cc47a..28409c90f7 100644 --- a/hw/block/vhost-user-blk.c +++ b/hw/block/vhost-user-blk.c @@ -168,13 +168,6 @@ static int vhost_user_blk_start(VirtIODevice *vdev, Er= ror **errp) goto err_guest_notifiers; } =20 - ret =3D vhost_dev_start(&s->dev, vdev); - if (ret < 0) { - error_setg_errno(errp, -ret, "Error starting vhost"); - goto err_guest_notifiers; - } - s->started_vu =3D true; - /* guest_notifier_mask/pending not used yet, so just unmask * everything here. virtio-pci will do the right thing by * enabling/disabling irqfd. @@ -183,9 +176,20 @@ static int vhost_user_blk_start(VirtIODevice *vdev, Er= ror **errp) vhost_virtqueue_mask(&s->dev, vdev, i, false); } =20 + s->dev.vq_index_end =3D s->dev.nvqs; + ret =3D vhost_dev_start(&s->dev, vdev); + if (ret < 0) { + error_setg_errno(errp, -ret, "Error starting vhost"); + goto err_guest_notifiers; + } + s->started_vu =3D true; + return ret; =20 err_guest_notifiers: + for (i =3D 0; i < s->dev.nvqs; i++) { + vhost_virtqueue_mask(&s->dev, vdev, i, true); + } k->set_guest_notifiers(qbus->parent, s->dev.nvqs, false); err_host_notifiers: vhost_dev_disable_notifiers(&s->dev, vdev); diff --git a/hw/net/vhost_net.c b/hw/net/vhost_net.c index d28f8b974b..0fe71ed309 100644 --- a/hw/net/vhost_net.c +++ b/hw/net/vhost_net.c @@ -387,21 +387,23 @@ int vhost_net_start(VirtIODevice *dev, NetClientState= *ncs, } else { peer =3D qemu_get_peer(ncs, n->max_queue_pairs); } - r =3D vhost_net_start_one(get_vhost_net(peer), dev); - - if (r < 0) { - goto err_start; - } =20 if (peer->vring_enable) { /* restore vring enable state */ r =3D vhost_set_vring_enable(peer, peer->vring_enable); =20 if (r < 0) { - vhost_net_stop_one(get_vhost_net(peer), dev); goto err_start; } } + + r =3D vhost_net_start_one(get_vhost_net(peer), dev); + if (r < 0) { + if (peer->vring_enable) { + vhost_set_vring_enable(peer, false); + } + goto err_start; + } } =20 return 0; --=20 2.27.0 From nobody Fri May 3 02:25:11 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass(p=reject dis=none) header.from=nvidia.com ARC-Seal: i=2; a=rsa-sha256; t=1667794204; cv=pass; d=zohomail.com; s=zohoarc; b=gejsZdRUXKh4H9fEe2ZLGgjFiKIggLmJX+C4NnPwXngCN/8okE0HiFTUvY13QYQpFySEsU+CIc3fgSwzn311KMw8++/CQcfEW7f255DdMIXearcNBszkJtBBBhbMt/2vzLHJDSU8wMw8+pnEF6ChqWVPEZDCfYBLhowkzTe2QGE= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1667794204; h=Content-Type:Content-Transfer-Encoding:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=RpOYEJpKQK9ijfjqnUAxvkZTuvtnmdix9wqdO6A93wM=; b=Uc0EQf+08n+zYnr8Pc9N7bkCQ1EWzpXsSUw/fSeCNR8961/7j2L9HLT964vQwYTqK7eSdHXSAJUB1LWymu2C2FHXhVpZKEC5XdOzmc8rufWEK7PKon+hmUmwfw3VNl39OGo5mlczO8//5bm+lRtdGypecX4+tDhWRZs+soUehXw= ARC-Authentication-Results: i=2; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; arc=pass (i=1 dmarc=pass fromdomain=nvidia.com); dmarc=pass header.from= (p=reject dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1667794204423729.143727453026; Sun, 6 Nov 2022 20:10:04 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1ortRV-0008PU-88; Sun, 06 Nov 2022 23:09:01 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ortRU-0008PK-4C for qemu-devel@nongnu.org; Sun, 06 Nov 2022 23:09:00 -0500 Received: from mail-bn7nam10on2078.outbound.protection.outlook.com ([40.107.92.78] helo=NAM10-BN7-obe.outbound.protection.outlook.com) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ortRP-0002Zx-Ku for qemu-devel@nongnu.org; Sun, 06 Nov 2022 23:08:59 -0500 Received: from DM6PR04CA0019.namprd04.prod.outlook.com (2603:10b6:5:334::24) by DM4PR12MB5867.namprd12.prod.outlook.com (2603:10b6:8:66::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.25; Mon, 7 Nov 2022 04:08:52 +0000 Received: from DM6NAM11FT034.eop-nam11.prod.protection.outlook.com (2603:10b6:5:334:cafe::a8) by DM6PR04CA0019.outlook.office365.com (2603:10b6:5:334::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.25 via Frontend Transport; Mon, 7 Nov 2022 04:08:52 +0000 Received: from mail.nvidia.com (216.228.117.161) by DM6NAM11FT034.mail.protection.outlook.com (10.13.173.47) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5791.20 via Frontend Transport; Mon, 7 Nov 2022 04:08:52 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by mail.nvidia.com (10.129.200.67) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.26; Sun, 6 Nov 2022 20:08:44 -0800 Received: from nvidia.com (10.126.230.35) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.29; Sun, 6 Nov 2022 20:08:42 -0800 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jVRxks9WdYsCLsHtCRoeyBtS+2Xzv529sULXqZh5p8P3ZR+68AyrfTq16yMNrrjcFvoQ7XVKEYOwQSiTYkM7+qeuQ+21KlH0cPtRbkGC3oympxAKPi1V2VlIfV1ZUDsANVJAnqIMcAK4Pmsx0VuZKWsafzCLtU4TiPUfEW9H6DGSCs4B5XP7InJGmt8FSZErYf8L8C7l0OIZLg2h5RHLr0M3OuFMhII1P/14HU5lJV+akYHuKcP7xTJp+Qpee2ZpWqQORT0EIi5Z0sYc5UD4sSKRR0G3tRlVwYtHRdJWEvKvS+aHzp5EyxWyuk1ryw3y3tixU6HK4saxdwbXv3/2xg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=RpOYEJpKQK9ijfjqnUAxvkZTuvtnmdix9wqdO6A93wM=; b=Y2ihqT0bHtn6Km0bjsalAj/iu2pMBLMMMoZcrCGfJdUrTL1UWbv+GCjqHsxm2GNVisaYWUpPF78KkOtBLIWy0s7styeHRzWIg5C07zPcdwG6f+6k/pduw8v+0j0RhrJPHuMzp/r73F9kupFVY7ibYaZdvjvyD2wqdmlyX4WzlW4a0GCWq4w9Ik0yrUVtWrAHHZBbak+jiMUEbB8eTqUEOTLe75yUi56mY015SGJ/vWK1QI2qms1vUST3L0v1B25czTAR3q5SlA61XzjB8WjfhHFVMN/taaiQgPt2YWHJwsicuXPCL7QyneuweRRBZgRkJ6uSKOOrbOPisdPkxL+WZg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 216.228.117.161) smtp.rcpttodomain=nongnu.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RpOYEJpKQK9ijfjqnUAxvkZTuvtnmdix9wqdO6A93wM=; b=RwnmZQkj1Fzf6FBbzmh2ska3kSFFdoum7wpFiZfs6zO9H/5ZfVZouXs5kAZNKClQ9zLbPa/iSPJYWQ+ypsn/3OieQVK6ak3IOBI+irqnEd8/i3cvd4GSAMuOmHKlJ1omRxKQfpbr2bh7v724nyETJjcSm9PCvmqmqHZKDsZechRL50EZt2Vqbd1jhqnEG2IhI9bGwCvvC6WNkKDUcG/8vhj43Ccx0gzEzgpZaMW99DhnEiLT/oiC3q7MqXiT3WGNLg7u0Yy9dvslB5DImXJDoC8GTRw51QQqEgpeAd9lYngxYopqZlcdNrkjkkpFjrUY8wbfRrMRZmZQXiR51nBSJA== X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 216.228.117.161) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 216.228.117.161 as permitted sender) receiver=protection.outlook.com; client-ip=216.228.117.161; helo=mail.nvidia.com; pr=C From: Yajun Wu To: , , , Subject: [PATCH v4 2/2] vhost-user: Support vhost_dev_start Date: Mon, 7 Nov 2022 12:08:18 +0800 Message-ID: <20221107040818.619072-3-yajunw@nvidia.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20221107040818.619072-1-yajunw@nvidia.com> References: <20220629022517.2600911-1-yajunw@nvidia.com> <20221107040818.619072-1-yajunw@nvidia.com> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [10.126.230.35] X-ClientProxiedBy: rnnvmail201.nvidia.com (10.129.68.8) To rnnvmail201.nvidia.com (10.129.68.8) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT034:EE_|DM4PR12MB5867:EE_ X-MS-Office365-Filtering-Correlation-Id: 5e8b9805-28a3-4634-7611-08dac075c7dd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: WOoGpU4Eo2O1tWgcQ5EcrO0X4qOXARPSZbsbcDVR0Tmdp328Kc9l7b3s2Rhi5GdaC+Qhll0g4Vyis1HDbxDsz8RCmiJASBpZ4WuejlfEs9/BD9q5e0eZxNxv9BSwc8BDNlqM48wvYTYyhJF1Ilu1r2HF1SKwhYXtfMU/HD1xkgQgKTURzjOKeJ8POcHiFg7jfvbNz0bVELPZ/SRw5uojSaT3SK0RQpybw1Uv1GECGqdpTWBRWYVMWLK0oeqoju/91yBl1X/Sig0LuFqj47ajAq0iM4bhvltm1Az++B59fphOYuoxe6LhhQmSldf9YPTK06ZTkYrl56gbM8Q1vBr9wjdVkFajrW7jSjbZA1Y5Oe1kJbAhLg+71bKAo4FxEgkpCoVNvrj9fNZFLC9ovn3Msz33Cj2a2IyEEMUquLXhP0MncQc+MFrW62M0X6xrezer1Ci8Sa1rdo8HNAGAyMJoQN6YvB+0MBJpjKLhyg0m8j2ZSrsiMLDBvc0EoQCKoIOkourxWiJNBUT+4QPay29oW3QbP0S6pfG0gDcn9v66qbs0X6DrS91LCwk1dKDdebPYM1AOUa7SXTW7WRPcl9GHW4KSbgVGsxcPL77f/7hmP1QR5w/twTYjaLdS/dyMHyPIhRndKSaFZqfP8JcAnV4mxXdPdIfWN7A0dpmPaHhTFUKdcOiK5/8boCCGHm9p8ukEtEh53gAuYzWXGfaPsdiJ6K3rv2ubsLiAiNPUIQ0O39DPAeYR2vqEk/JmPeXwm7af+0d6wLbKVnSGT8V7x6O4wg== X-Forefront-Antispam-Report: CIP:216.228.117.161; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:mail.nvidia.com; PTR:dc6edge2.nvidia.com; CAT:NONE; SFS:(13230022)(4636009)(39860400002)(346002)(376002)(396003)(136003)(451199015)(40470700004)(36840700001)(46966006)(966005)(478600001)(8936002)(5660300002)(36860700001)(7696005)(86362001)(8676002)(26005)(82310400005)(55016003)(41300700001)(2616005)(36756003)(6286002)(6636002)(110136005)(6666004)(82740400003)(7636003)(16526019)(1076003)(426003)(40460700003)(356005)(186003)(70586007)(70206006)(83380400001)(2906002)(47076005)(336012)(316002)(40480700001); DIR:OUT; SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Nov 2022 04:08:52.1826 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 5e8b9805-28a3-4634-7611-08dac075c7dd X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a; Ip=[216.228.117.161]; Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT034.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5867 Received-SPF: softfail client-ip=40.107.92.78; envelope-from=yajunw@nvidia.com; helo=NAM10-BN7-obe.outbound.protection.outlook.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @Nvidia.com) X-ZM-MESSAGEID: 1667794206068100001 Content-Type: text/plain; charset="utf-8" The motivation of adding vhost-user vhost_dev_start support is to improve backend configuration speed and reduce live migration VM downtime. Today VQ configuration is issued one by one. For virtio net with multi-queue support, backend needs to update RSS (Receive side scaling) on every rx queue enable. Updating RSS is time-consuming (typical time like 7ms). Implement already defined vhost status and message in the vhost specification [1]. (a) VHOST_USER_PROTOCOL_F_STATUS (b) VHOST_USER_SET_STATUS (c) VHOST_USER_GET_STATUS Send message VHOST_USER_SET_STATUS with VIRTIO_CONFIG_S_DRIVER_OK for device start and reset(0) for device stop. On reception of the DRIVER_OK message, backend can apply the needed setting only once (instead of incremental) and also utilize parallelism on enabling queues. This improves QEMU's live migration downtime with vhost user backend implementation by great margin, specially for the large number of VQs of 64 from 800 msec to 250 msec. [1] https://qemu-project.gitlab.io/qemu/interop/vhost-user.html Signed-off-by: Yajun Wu Acked-by: Parav Pandit --- hw/virtio/vhost-user.c | 74 +++++++++++++++++++++++++++++++++++++++++- 1 file changed, 73 insertions(+), 1 deletion(-) diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c index 03415b6c95..bb5164b753 100644 --- a/hw/virtio/vhost-user.c +++ b/hw/virtio/vhost-user.c @@ -81,6 +81,7 @@ enum VhostUserProtocolFeature { VHOST_USER_PROTOCOL_F_RESET_DEVICE =3D 13, /* Feature 14 reserved for VHOST_USER_PROTOCOL_F_INBAND_NOTIFICATIONS.= */ VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTS =3D 15, + VHOST_USER_PROTOCOL_F_STATUS =3D 16, VHOST_USER_PROTOCOL_F_MAX }; =20 @@ -126,6 +127,8 @@ typedef enum VhostUserRequest { VHOST_USER_GET_MAX_MEM_SLOTS =3D 36, VHOST_USER_ADD_MEM_REG =3D 37, VHOST_USER_REM_MEM_REG =3D 38, + VHOST_USER_SET_STATUS =3D 39, + VHOST_USER_GET_STATUS =3D 40, VHOST_USER_MAX } VhostUserRequest; =20 @@ -1452,6 +1455,43 @@ static int vhost_user_set_u64(struct vhost_dev *dev,= int request, uint64_t u64, return 0; } =20 +static int vhost_user_set_status(struct vhost_dev *dev, uint8_t status) +{ + return vhost_user_set_u64(dev, VHOST_USER_SET_STATUS, status, false); +} + +static int vhost_user_get_status(struct vhost_dev *dev, uint8_t *status) +{ + uint64_t value; + int ret; + + ret =3D vhost_user_get_u64(dev, VHOST_USER_GET_STATUS, &value); + if (ret < 0) { + return ret; + } + *status =3D value; + + return 0; +} + +static int vhost_user_add_status(struct vhost_dev *dev, uint8_t status) +{ + uint8_t s; + int ret; + + ret =3D vhost_user_get_status(dev, &s); + if (ret < 0) { + return ret; + } + + if ((s & status) =3D=3D status) { + return 0; + } + s |=3D status; + + return vhost_user_set_status(dev, s); +} + static int vhost_user_set_features(struct vhost_dev *dev, uint64_t features) { @@ -1460,6 +1500,7 @@ static int vhost_user_set_features(struct vhost_dev *= dev, * backend is actually logging changes */ bool log_enabled =3D features & (0x1ULL << VHOST_F_LOG_ALL); + int ret; =20 /* * We need to include any extra backend only feature bits that @@ -1467,9 +1508,18 @@ static int vhost_user_set_features(struct vhost_dev = *dev, * VHOST_USER_F_PROTOCOL_FEATURES bit for enabling protocol * features. */ - return vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES, + ret =3D vhost_user_set_u64(dev, VHOST_USER_SET_FEATURES, features | dev->backend_features, log_enabled); + + if (virtio_has_feature(dev->protocol_features, + VHOST_USER_PROTOCOL_F_STATUS)) { + if (!ret) { + return vhost_user_add_status(dev, VIRTIO_CONFIG_S_FEATURES_OK); + } + } + + return ret; } =20 static int vhost_user_set_protocol_features(struct vhost_dev *dev, @@ -2615,6 +2665,27 @@ void vhost_user_cleanup(VhostUserState *user) user->chr =3D NULL; } =20 +static int vhost_user_dev_start(struct vhost_dev *dev, bool started) +{ + if (!virtio_has_feature(dev->protocol_features, + VHOST_USER_PROTOCOL_F_STATUS)) { + return 0; + } + + /* Set device status only for last queue pair */ + if (dev->vq_index + dev->nvqs !=3D dev->vq_index_end) { + return 0; + } + + if (started) { + return vhost_user_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE | + VIRTIO_CONFIG_S_DRIVER | + VIRTIO_CONFIG_S_DRIVER_OK); + } else { + return vhost_user_set_status(dev, 0); + } +} + const VhostOps user_ops =3D { .backend_type =3D VHOST_BACKEND_TYPE_USER, .vhost_backend_init =3D vhost_user_backend_init, @@ -2649,4 +2720,5 @@ const VhostOps user_ops =3D { .vhost_backend_mem_section_filter =3D vhost_user_mem_section_filte= r, .vhost_get_inflight_fd =3D vhost_user_get_inflight_fd, .vhost_set_inflight_fd =3D vhost_user_set_inflight_fd, + .vhost_dev_start =3D vhost_user_dev_start, }; --=20 2.27.0