From nobody Sat Sep 21 05:18:21 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1704999846; cv=none; d=zohomail.com; s=zohoarc; b=XgzqQliglOpgqN1x2bxmg+5eAx6iDzm0vrqQzkL6oxzuSd7qh+O0PEggxxxyWvkUNarDX7wbyBf+qpbNJ38OZRs6XjFPaxHhGRONXN+Rseo4VjzGHvpiphVcmd1TLfxGaIoWAvyRAtN3SFwBIL+vDTmlxrIP39hyx302kBiI6BA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1704999846; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=qQKcPkY6/W3EsLgkAEJ6imao+rgKnctifkMPjHMrgNY=; b=J3WFETgfWm7J+BEX6nbQETmXHQgv0JHthd2eMNhKyVT28xPdu3aIU/NO+AYKy2OZr2HyM1E+uzAMAHA0w6wxap7AKFhRJOukdtBy1Zve1EXpir/1/0ntMz5g5vMa7Kg0LpRTpzquTd5nbzNIytfBSp2lK5ftS0t+yOKZDGfW1lc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1704999846460120.05072783795356; Thu, 11 Jan 2024 11:04:06 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rO0K5-0001xg-Os; Thu, 11 Jan 2024 14:02:37 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rO0K4-0001xY-J7 for qemu-devel@nongnu.org; Thu, 11 Jan 2024 14:02:36 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rO0K2-0002Lp-Dm for qemu-devel@nongnu.org; Thu, 11 Jan 2024 14:02:36 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-695-X8wHvCKHMbOjfGxl6mSmaQ-1; Thu, 11 Jan 2024 14:02:28 -0500 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 49EC18314F1; Thu, 11 Jan 2024 19:02:28 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.194.34]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7FBAA492BF0; Thu, 11 Jan 2024 19:02:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1704999753; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qQKcPkY6/W3EsLgkAEJ6imao+rgKnctifkMPjHMrgNY=; b=ZYi4jdQujvpFesC1bqcwJ2lUKrDwrWBhgVour2h8eoVvSNeZJ+1XgUYr84l/PDg6FZANdf 1DwsFyu/W/HiZRfjGNl8GRiKQ/ISTEeoGIV40Cjr9Laa/fmVUdWhQItYnjnhXryfQCliN8 OMeKWUv3QHyFyEQo4zUW4NbUD1M9MsE= X-MC-Unique: X8wHvCKHMbOjfGxl6mSmaQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: mst@redhat.com, qemu-devel@nongnu.org Cc: Peter Xu , Dragos Tatulea , Zhu Lingshan , Jason Wang , Lei Yang , Laurent Vivier , si-wei.liu@oracle.com, Stefano Garzarella , Parav Pandit Subject: [PATCH 1/6] vdpa: check for iova tree initialized at net_client_start Date: Thu, 11 Jan 2024 20:02:17 +0100 Message-Id: <20240111190222.496695-2-eperezma@redhat.com> In-Reply-To: <20240111190222.496695-1-eperezma@redhat.com> References: <20240111190222.496695-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.10 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -45 X-Spam_score: -4.6 X-Spam_bar: ---- X-Spam_report: (-4.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.467, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1704999847696100005 To map the guest memory while it is migrating we need to create the iova_tree, as long as the destination uses x-svq=3Don. Checking to not override it. The function vhost_vdpa_net_client_stop clear it if the device is stopped. If the guest starts the device again, the iova tree is recreated by vhost_vdpa_net_data_start_first or vhost_vdpa_net_cvq_start if needed, so old behavior is kept. Signed-off-by: Eugenio P=C3=A9rez --- net/vhost-vdpa.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 3726ee5d67..e11b390466 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -341,7 +341,9 @@ static void vhost_vdpa_net_data_start_first(VhostVDPASt= ate *s) =20 migration_add_notifier(&s->migration_state, vdpa_net_migration_state_notifier); - if (v->shadow_vqs_enabled) { + + /* iova_tree may be initialized by vhost_vdpa_net_load_setup */ + if (v->shadow_vqs_enabled && !v->shared->iova_tree) { v->shared->iova_tree =3D vhost_iova_tree_new(v->shared->iova_range= .first, v->shared->iova_range.l= ast); } --=20 2.39.3 From nobody Sat Sep 21 05:18:21 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1704999846; cv=none; d=zohomail.com; s=zohoarc; b=HiDecxvNF3ZA6GMqSBJTIscSOIKJgzClsfwF3bfbxhcYqKHCkUSGjGFdm/L4vlQzyKFyHBZdT8ZxDT9Z/8i8rwA+FlxqmHrum/UaGp5fg7xNNoC4PdFtIjRSVcSRbTMBGvQc8LGl1RV1U5oJEHJIMmPFI4FGx66iNTzeFZoEbV8= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1704999846; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=7hLfAp79U1FIsM1rjGCyedJBUUbAxAWLgcvsVFhJeTs=; b=P6mb9ZSifVqVyVxlNLx2NNKrkDDoGibGtGIMipxn0btQzy1tLReeSAdJkbdbNFHsBXoVpILNBmATkyXNjE0c9kf8nmKAOAc6WicyYU7s5gvRymHg4+NlvYyqpNkSYM4FnNSrmJiGU8Orb469kyjp/va7X2iWtSfbab3kZruUcRk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1704999846579352.7568543582936; Thu, 11 Jan 2024 11:04:06 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rO0K7-0001yR-PC; Thu, 11 Jan 2024 14:02:39 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rO0K6-0001xh-1K for qemu-devel@nongnu.org; Thu, 11 Jan 2024 14:02:38 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rO0K4-0002Ml-I3 for qemu-devel@nongnu.org; Thu, 11 Jan 2024 14:02:37 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-368-Vv7gGK6dNamWkiOyyqEWBg-1; Thu, 11 Jan 2024 14:02:31 -0500 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 584FE185A782; Thu, 11 Jan 2024 19:02:30 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.194.34]) by smtp.corp.redhat.com (Postfix) with ESMTP id 8ECE6492BF0; Thu, 11 Jan 2024 19:02:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1704999755; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7hLfAp79U1FIsM1rjGCyedJBUUbAxAWLgcvsVFhJeTs=; b=Ejj5VXwibYOPozO5J494SntUAroVEZds5VepwVthsTk3cHkr9euHR+E7AXPegH34dVrbbX ZMdhuHgT1DNrH5sxV37EUeNSTX+lJ3Xp/OSO8icvmgJoffusRkuV32NnwMGnkeZRvyi1lQ bHIXyqtiT/Hv3owLP7v4tRd7w6Z68AE= X-MC-Unique: Vv7gGK6dNamWkiOyyqEWBg-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: mst@redhat.com, qemu-devel@nongnu.org Cc: Peter Xu , Dragos Tatulea , Zhu Lingshan , Jason Wang , Lei Yang , Laurent Vivier , si-wei.liu@oracle.com, Stefano Garzarella , Parav Pandit Subject: [PATCH 2/6] vdpa: reorder vhost_vdpa_set_backend_cap Date: Thu, 11 Jan 2024 20:02:18 +0100 Message-Id: <20240111190222.496695-3-eperezma@redhat.com> In-Reply-To: <20240111190222.496695-1-eperezma@redhat.com> References: <20240111190222.496695-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.10 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -45 X-Spam_score: -4.6 X-Spam_bar: ---- X-Spam_report: (-4.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.467, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1704999847708100006 It will be used directly by vhost_vdpa_init. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-vdpa.c | 60 +++++++++++++++++++++--------------------- 1 file changed, 30 insertions(+), 30 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index ddae494ca8..33ae285f87 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -580,6 +580,36 @@ static void vhost_vdpa_init_svq(struct vhost_dev *hdev= , struct vhost_vdpa *v) v->shadow_vqs =3D g_steal_pointer(&shadow_vqs); } =20 +static int vhost_vdpa_set_backend_cap(struct vhost_dev *dev) +{ + struct vhost_vdpa *v =3D dev->opaque; + + uint64_t features; + uint64_t f =3D 0x1ULL << VHOST_BACKEND_F_IOTLB_MSG_V2 | + 0x1ULL << VHOST_BACKEND_F_IOTLB_BATCH | + 0x1ULL << VHOST_BACKEND_F_IOTLB_ASID | + 0x1ULL << VHOST_BACKEND_F_SUSPEND; + int r; + + if (vhost_vdpa_call(dev, VHOST_GET_BACKEND_FEATURES, &features)) { + return -EFAULT; + } + + features &=3D f; + + if (vhost_vdpa_first_dev(dev)) { + r =3D vhost_vdpa_call(dev, VHOST_SET_BACKEND_FEATURES, &features); + if (r) { + return -EFAULT; + } + } + + dev->backend_cap =3D features; + v->shared->backend_cap =3D features; + + return 0; +} + static int vhost_vdpa_init(struct vhost_dev *dev, void *opaque, Error **er= rp) { struct vhost_vdpa *v =3D opaque; @@ -827,36 +857,6 @@ static int vhost_vdpa_set_features(struct vhost_dev *d= ev, return vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_FEATURES_OK); } =20 -static int vhost_vdpa_set_backend_cap(struct vhost_dev *dev) -{ - struct vhost_vdpa *v =3D dev->opaque; - - uint64_t features; - uint64_t f =3D 0x1ULL << VHOST_BACKEND_F_IOTLB_MSG_V2 | - 0x1ULL << VHOST_BACKEND_F_IOTLB_BATCH | - 0x1ULL << VHOST_BACKEND_F_IOTLB_ASID | - 0x1ULL << VHOST_BACKEND_F_SUSPEND; - int r; - - if (vhost_vdpa_call(dev, VHOST_GET_BACKEND_FEATURES, &features)) { - return -EFAULT; - } - - features &=3D f; - - if (vhost_vdpa_first_dev(dev)) { - r =3D vhost_vdpa_call(dev, VHOST_SET_BACKEND_FEATURES, &features); - if (r) { - return -EFAULT; - } - } - - dev->backend_cap =3D features; - v->shared->backend_cap =3D features; - - return 0; -} - static int vhost_vdpa_get_device_id(struct vhost_dev *dev, uint32_t *device_id) { --=20 2.39.3 From nobody Sat Sep 21 05:18:21 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1704999864; cv=none; d=zohomail.com; s=zohoarc; b=Iw1qPx5IWXhnokqpvO3roV/8KeISEnOZrWXI4q5uTPKlYKoIFL802rzPgUNvsDNPTwBmXJkMwnLUuwKf/R+alZRsHHQHRYF3gDkgdbdFXn2asclvRUsLz1F7TJucRU5pT1z/n88ORMEd6oythunUg1rjA7cgGgt8OCDFzMnQcw0= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1704999864; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=ZNYtP0/fqUQrbz1uancjxE/XlEOpkN/LmQB3y+vqoIQ=; b=cCOBl2MuQdDXdsN031VS8GL+CTlisKISEalf+BwE8lnWpxF386NsyGCmRwk5TfDG4OgB9hIuBxtBnWmm4mhag6IaJSnP/FxI3odqxVVxizrocePpoUJdB1PPbWATXO/+rdMeuYnBr5zeFR0L/TJF41Jt/Azgrf2Z3woAf8+PrdA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1704999864273880.656821262037; Thu, 11 Jan 2024 11:04:24 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rO0K8-0001yZ-Ig; Thu, 11 Jan 2024 14:02:40 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rO0K7-0001y5-1D for qemu-devel@nongnu.org; Thu, 11 Jan 2024 14:02:39 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rO0K5-0002NJ-AL for qemu-devel@nongnu.org; Thu, 11 Jan 2024 14:02:38 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-582-AeBOwCZUP_O5KmmZjDJ-mQ-1; Thu, 11 Jan 2024 14:02:32 -0500 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 63B8C85A58A; Thu, 11 Jan 2024 19:02:32 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.194.34]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9BE65492C25; Thu, 11 Jan 2024 19:02:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1704999756; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZNYtP0/fqUQrbz1uancjxE/XlEOpkN/LmQB3y+vqoIQ=; b=B2ja8fD9k4WUgXeAWGLtCskCwCzGdBFhwL0c2w+7rReH8V85WbMeXYZkQk+IppEdKzcCpG rodEs36IOlgBYPk0Bv1t4G//eOKWGJDtDUnSh7P9Ak4X0EUkUQw/4tUXbM8DLxraVLoG5g Mcd8uQcvBWGvpwoA1ZzGH/DB6j7HQNo= X-MC-Unique: AeBOwCZUP_O5KmmZjDJ-mQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: mst@redhat.com, qemu-devel@nongnu.org Cc: Peter Xu , Dragos Tatulea , Zhu Lingshan , Jason Wang , Lei Yang , Laurent Vivier , si-wei.liu@oracle.com, Stefano Garzarella , Parav Pandit Subject: [PATCH 3/6] vdpa: set backend capabilities at vhost_vdpa_init Date: Thu, 11 Jan 2024 20:02:19 +0100 Message-Id: <20240111190222.496695-4-eperezma@redhat.com> In-Reply-To: <20240111190222.496695-1-eperezma@redhat.com> References: <20240111190222.496695-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.10 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -45 X-Spam_score: -4.6 X-Spam_bar: ---- X-Spam_report: (-4.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.467, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1704999865775100003 The backend does not reset them until the vdpa file descriptor is closed so there is no harm in doing it only once. This allows the destination of a live migration to premap memory in batches, using VHOST_BACKEND_F_IOTLB_BATCH. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-vdpa.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 33ae285f87..bd4db7ba5f 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -620,6 +620,12 @@ static int vhost_vdpa_init(struct vhost_dev *dev, void= *opaque, Error **errp) v->dev =3D dev; dev->opaque =3D opaque ; v->shared->listener =3D vhost_vdpa_memory_listener; + + ret =3D vhost_vdpa_set_backend_cap(dev); + if (unlikely(ret !=3D 0)) { + return ret; + } + vhost_vdpa_init_svq(dev, v); =20 error_propagate(&dev->migration_blocker, v->migration_blocker); @@ -1506,7 +1512,6 @@ const VhostOps vdpa_ops =3D { .vhost_set_vring_kick =3D vhost_vdpa_set_vring_kick, .vhost_set_vring_call =3D vhost_vdpa_set_vring_call, .vhost_get_features =3D vhost_vdpa_get_features, - .vhost_set_backend_cap =3D vhost_vdpa_set_backend_cap, .vhost_set_owner =3D vhost_vdpa_set_owner, .vhost_set_vring_endian =3D NULL, .vhost_backend_memslots_limit =3D vhost_vdpa_memslots_limit, --=20 2.39.3 From nobody Sat Sep 21 05:18:21 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1704999848; cv=none; d=zohomail.com; s=zohoarc; b=FEJ8KOU7cVC1hnhqyVIPG7vYWVM9ozUOlOgFRbesFBrQn58Br2iGzQEwAQ87iw5b2qzpNfVwTrKHncPVWv/wjhyhCPpLC4ZtnvC3TL3kiHGE9g1A+dW58Zfy/BKVxQbl52PX9GN1QjNNrRK7jokY7HQS89tPSnhzv+9VODohh3A= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1704999848; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=CGgCal5pEcp+8Ne5+Op8F9zTYYgmybpw7H9WYg4oxlA=; b=TehRvSV4txLBDqnVPIT86+LCVHEuU/FljNZRsH/sUk5EctYYcXbiV3CnZHaZN3p35bfiXLpRBJ7+v9fTL+pYHZVugVjtVw81CSSUj/coNNQvC3DqyO1lSmDdeBl1mIH42/2EkY7ozO5PcvUnBd3ODxT2s4HdJAE0SQk9qKYXev0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1704999848459389.8646666122313; Thu, 11 Jan 2024 11:04:08 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rO0KD-0001zm-FP; Thu, 11 Jan 2024 14:02:45 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rO0KB-0001z2-6s for qemu-devel@nongnu.org; Thu, 11 Jan 2024 14:02:43 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rO0K9-0002PH-HU for qemu-devel@nongnu.org; Thu, 11 Jan 2024 14:02:42 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-455-jT0-JY_MMGmOUjtxyTMXSg-1; Thu, 11 Jan 2024 14:02:35 -0500 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C36D485A58E; Thu, 11 Jan 2024 19:02:34 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.194.34]) by smtp.corp.redhat.com (Postfix) with ESMTP id A8707492BF0; Thu, 11 Jan 2024 19:02:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1704999760; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=CGgCal5pEcp+8Ne5+Op8F9zTYYgmybpw7H9WYg4oxlA=; b=E3g9NBYSaYQ7sLlSXoEwL1qkngU4133sn0plYnl2CpoSFA1UFIPmPUfgzuyvmOUAFdTuZT tI6/6QpM4LktWYSDdCeAR697kDfcjW/tlIJtGZNu1S52h2h1Y5A0d6dsoxAnX5fsgDFhDy cRLRPADrJYMXt9CbyPWvA9Ej+JCtdfs= X-MC-Unique: jT0-JY_MMGmOUjtxyTMXSg-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: mst@redhat.com, qemu-devel@nongnu.org Cc: Peter Xu , Dragos Tatulea , Zhu Lingshan , Jason Wang , Lei Yang , Laurent Vivier , si-wei.liu@oracle.com, Stefano Garzarella , Parav Pandit Subject: [PATCH 4/6] vdpa: add listener_registered Date: Thu, 11 Jan 2024 20:02:20 +0100 Message-Id: <20240111190222.496695-5-eperezma@redhat.com> In-Reply-To: <20240111190222.496695-1-eperezma@redhat.com> References: <20240111190222.496695-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.10 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -45 X-Spam_score: -4.6 X-Spam_bar: ---- X-Spam_report: (-4.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.467, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1704999849725100011 Check if the listener has been registered or not, so it needs to be registered again at start. Signed-off-by: Eugenio P=C3=A9rez --- include/hw/virtio/vhost-vdpa.h | 6 ++++++ hw/virtio/vhost-vdpa.c | 7 ++++++- 2 files changed, 12 insertions(+), 1 deletion(-) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index 8f54e5edd4..03ed2f2be3 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -45,6 +45,12 @@ typedef struct vhost_vdpa_shared { =20 bool iotlb_batch_begin_sent; =20 + /* + * The memory listener has been registered, so DMA maps have been sent= to + * the device. + */ + bool listener_registered; + /* Vdpa must send shadow addresses as IOTLB key for data queues, not G= PA */ bool shadow_data; } VhostVDPAShared; diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index bd4db7ba5f..b08349d57c 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -1331,7 +1331,10 @@ static int vhost_vdpa_dev_start(struct vhost_dev *de= v, bool started) "IOMMU and try again"); return -1; } - memory_listener_register(&v->shared->listener, dev->vdev->dma_as); + if (!v->shared->listener_registered) { + memory_listener_register(&v->shared->listener, dev->vdev->dma_= as); + v->shared->listener_registered =3D true; + } =20 return vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_DRIVER_OK); } @@ -1351,6 +1354,8 @@ static void vhost_vdpa_reset_status(struct vhost_dev = *dev) vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE | VIRTIO_CONFIG_S_DRIVER); memory_listener_unregister(&v->shared->listener); + v->shared->listener_registered =3D false; + } =20 static int vhost_vdpa_set_log_base(struct vhost_dev *dev, uint64_t base, --=20 2.39.3 From nobody Sat Sep 21 05:18:21 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1704999856; cv=none; d=zohomail.com; s=zohoarc; b=l+7jGnNe+qmryda5ILXrb9xVEOx8WHZOLCfGww4qjQdA7DXTVtAdOMB7jiUDaIWXxJBOsn5gae/8SX5mdzH4RKR1sjL+eAcnKEqSdv9AM7eKI3fuHckpwPdPadrWQZB3r3u0nAhAtvIZbxsNyR0olmqWqHh9XQ7LjfqwMxbS54Y= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1704999856; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=FDU5bDvjCD7qy3GlwYsAJnVVq36TVzsi9e3jkYL+vdU=; b=FE8mI9n5bWUj0wAKmEv6s30Jv8W4Os8CHxIwbaH4i/CJZ3R1tyFt0ksGEt10uyGx3KbUzlrYhqbMUGyLmfiZSS4geMagpFR10iNadnBpx63wz5uC2Z9feRPXm8Q/0SHFXubOMQOg4uaKrKRI4V9MF0ayOef3Ya8F2DXaQBy03XY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1704999856238436.4380864279175; Thu, 11 Jan 2024 11:04:16 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rO0KC-0001zR-Dd; Thu, 11 Jan 2024 14:02:44 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rO0KA-0001yt-JG for qemu-devel@nongnu.org; Thu, 11 Jan 2024 14:02:42 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rO0K8-0002Od-E3 for qemu-devel@nongnu.org; Thu, 11 Jan 2024 14:02:41 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-516-Lo9MZ2fIP_KsZNuy1evwRg-1; Thu, 11 Jan 2024 14:02:37 -0500 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id CEABE811E9C; Thu, 11 Jan 2024 19:02:36 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.194.34]) by smtp.corp.redhat.com (Postfix) with ESMTP id 13785492BF0; Thu, 11 Jan 2024 19:02:34 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1704999759; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=FDU5bDvjCD7qy3GlwYsAJnVVq36TVzsi9e3jkYL+vdU=; b=B8ROowOaM7bm4kiX58rqEeJz0jVIk/8q7phttZHX4suXzw4Uaztgoa/cUMv3eiSugK1snh U7qopsbFiUfl8bdgIHzxT9IbOpeFwT6w9sORzfxGcWSv9tAj1m+RlM7zPMAQPMRqMhkRsY j/TmqW847tkxfgs1y7apCcECV3e6kq8= X-MC-Unique: Lo9MZ2fIP_KsZNuy1evwRg-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: mst@redhat.com, qemu-devel@nongnu.org Cc: Peter Xu , Dragos Tatulea , Zhu Lingshan , Jason Wang , Lei Yang , Laurent Vivier , si-wei.liu@oracle.com, Stefano Garzarella , Parav Pandit Subject: [PATCH 5/6] vdpa: reorder listener assignment Date: Thu, 11 Jan 2024 20:02:21 +0100 Message-Id: <20240111190222.496695-6-eperezma@redhat.com> In-Reply-To: <20240111190222.496695-1-eperezma@redhat.com> References: <20240111190222.496695-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.10 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -45 X-Spam_score: -4.6 X-Spam_bar: ---- X-Spam_report: (-4.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.467, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1704999857795100003 Since commit f6fe3e333f ("vdpa: move memory listener to vhost_vdpa_shared") this piece of code repeatedly assign shared->listener members. This was not a problem as it was not used until device start. However next patches move the listener registration to this vhost_vdpa_init function. When the listener is registered it is added to an embedded linked list, so setting its members again will cause memory corruption to the linked list node. Do the right thing and only set it in the first vdpa device. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-vdpa.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index b08349d57c..521a889104 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -619,7 +619,6 @@ static int vhost_vdpa_init(struct vhost_dev *dev, void = *opaque, Error **errp) =20 v->dev =3D dev; dev->opaque =3D opaque ; - v->shared->listener =3D vhost_vdpa_memory_listener; =20 ret =3D vhost_vdpa_set_backend_cap(dev); if (unlikely(ret !=3D 0)) { @@ -661,6 +660,7 @@ static int vhost_vdpa_init(struct vhost_dev *dev, void = *opaque, Error **errp) vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE | VIRTIO_CONFIG_S_DRIVER); =20 + v->shared->listener =3D vhost_vdpa_memory_listener; return 0; } =20 --=20 2.39.3 From nobody Sat Sep 21 05:18:21 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1704999832; cv=none; d=zohomail.com; s=zohoarc; b=MoBCjI0Oq0b8R528ykaX4VNUV5gUKScjTOMlsGnsT1jhd4PkVWTNPC8nXDLLwQAKxZOPzTZysoekwom16BNHAHXs4dOsBNi7wMT1cjHCQVxKDYXSXb06V4th4Jky1JCFXJfqcqLu0hjgXDMyvc9+a3doTrckCgdZzCF6IW8TRoQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1704999832; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=xfHj4a+1kkW3IuoRJ+D3rPLEa+55ok5buRdNGYoGEhQ=; b=LiRwakJYeTCqIPKW1wPpPcy7P/7+s2u22ZmtnNmJmRuwH0q3Bztw7xIVUn79M/xyeUH9I2hFOOG35sI0ok2bYCaM6t27zNyoFSzpibtFy1cOv0Sjd5XArAoZen9/PJahk/z67zPi6amFFNyt7WBzeM9FWGb+owedcLctVcFXA0Y= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1704999832386996.316180241861; Thu, 11 Jan 2024 11:03:52 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rO0KG-0001zx-6a; Thu, 11 Jan 2024 14:02:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rO0KF-0001zo-2c for qemu-devel@nongnu.org; Thu, 11 Jan 2024 14:02:47 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rO0KD-0002RC-EN for qemu-devel@nongnu.org; Thu, 11 Jan 2024 14:02:46 -0500 Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-623-RWZ7LSwLOdeF1csfGnAqog-1; Thu, 11 Jan 2024 14:02:39 -0500 Received: from smtp.corp.redhat.com (int-mx10.intmail.prod.int.rdu2.redhat.com [10.11.54.10]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id DCD422999B28; Thu, 11 Jan 2024 19:02:38 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.194.34]) by smtp.corp.redhat.com (Postfix) with ESMTP id 20559492BF0; Thu, 11 Jan 2024 19:02:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1704999764; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=xfHj4a+1kkW3IuoRJ+D3rPLEa+55ok5buRdNGYoGEhQ=; b=cFs5RcuGJsCXX/briUVUbYdraCV1hz6xBV5fmk/P43OxGeaC4v5OCodxCoVDXY3qt0DXjL X2iNn2ckNaeJu5uN+LTa4u8HQMngAoaqmJccKaRTMzBciWw6J5fcDXM+xYsCeP4Lq+6SS3 nwdZiGHon4Yh56auIvYyEi6y0PngCr8= X-MC-Unique: RWZ7LSwLOdeF1csfGnAqog-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: mst@redhat.com, qemu-devel@nongnu.org Cc: Peter Xu , Dragos Tatulea , Zhu Lingshan , Jason Wang , Lei Yang , Laurent Vivier , si-wei.liu@oracle.com, Stefano Garzarella , Parav Pandit Subject: [PATCH 6/6] vdpa: move memory listener register to vhost_vdpa_init Date: Thu, 11 Jan 2024 20:02:22 +0100 Message-Id: <20240111190222.496695-7-eperezma@redhat.com> In-Reply-To: <20240111190222.496695-1-eperezma@redhat.com> References: <20240111190222.496695-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.10 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -45 X-Spam_score: -4.6 X-Spam_bar: ---- X-Spam_report: (-4.6 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.467, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1704999833615100003 Current memory operations like pinning may take a lot of time at the destination. Currently they are done after the source of the migration is stopped, and before the workload is resumed at the destination. This is a period where neigher traffic can flow, nor the VM workload can continue (downtime). We can do better as we know the memory layout of the guest RAM at the destination from the moment that all devices are initializaed. So moving that operation allows QEMU to communicate the kernel the maps while the workload is still running in the source, so Linux can start mapping them. As a small drawback, there is a time in the initialization where QEMU cannot respond to QMP etc. By some testing, this time is about 0.2seconds. This may be further reduced (or increased) depending on the vdpa driver and the platform hardware, and it is dominated by the cost of memory pinning. This matches the time that we move out of the called downtime window. The downtime is measured as checking the trace timestamp from the moment the source suspend the device to the moment the destination starts the eight and last virtqueue pair. For a 39G guest, it goes from ~2.2526 secs to 2.0949. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-vdpa.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 521a889104..eae8b790dd 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -660,7 +660,13 @@ static int vhost_vdpa_init(struct vhost_dev *dev, void= *opaque, Error **errp) vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE | VIRTIO_CONFIG_S_DRIVER); =20 + /* + * Being optimistic and listening address space memory. If the device + * uses vIOMMU, it is changed at vhost_vdpa_dev_start. + */ v->shared->listener =3D vhost_vdpa_memory_listener; + memory_listener_register(&v->shared->listener, &address_space_memory); + v->shared->listener_registered =3D true; return 0; } =20 @@ -1331,6 +1337,11 @@ static int vhost_vdpa_dev_start(struct vhost_dev *de= v, bool started) "IOMMU and try again"); return -1; } + if (v->shared->listener_registered && + dev->vdev->dma_as !=3D v->shared->listener.address_space) { + memory_listener_unregister(&v->shared->listener); + v->shared->listener_registered =3D false; + } if (!v->shared->listener_registered) { memory_listener_register(&v->shared->listener, dev->vdev->dma_= as); v->shared->listener_registered =3D true; --=20 2.39.3