From nobody Tue May 14 19:56:46 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1677254115; cv=none; d=zohomail.com; s=zohoarc; b=M5CFdqSfgWqyWkrz5kUpYuSdFdgzRW1KP3Du8C15zIgZLkprasaWz/2P2v3ukswrga0zYz+c3A5zyXZpdQ4QQFRUeVQxih9PGiEnP5raTj6awDfMgoQ5FIiilq/sJGJUQ8O4Gw6MtZBoJaFRlMmjWqUFxUao+7qTG/OaoI+bzvc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677254115; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=GqVK0RZiBVyglv7bn8UmLPqjZPwOnmJ9yoq1mwYLb2A=; b=XmdTnGY44wMxK9uXhy6k3AbEWO7ZdY6Hz9XMHEwA/PYz0l5WvboWkuTZJYr9LsV1ipbiTl+tU+ZzNn8lwCvZlBhAGDDwxs4wE9egfr+GP3p0PrZHb44X8uDTAtgjZhHJ37VCr1GzJ+2T05ea7eZCY6fOpOeinRu3s55o6ks+1iI= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677254115599492.62297274619505; Fri, 24 Feb 2023 07:55:15 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pVaPP-0005KX-Vt; Fri, 24 Feb 2023 10:54:56 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaPO-0005JI-07 for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:54:54 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaPM-0005Kq-4p for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:54:53 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-639-ajnuAosFNcSaTEGSAtDvhQ-1; Fri, 24 Feb 2023 10:54:48 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 7F4DC101A5B4; Fri, 24 Feb 2023 15:54:47 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.85]) by smtp.corp.redhat.com (Postfix) with ESMTP id A0C43C15BA0; Fri, 24 Feb 2023 15:54:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677254091; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=GqVK0RZiBVyglv7bn8UmLPqjZPwOnmJ9yoq1mwYLb2A=; b=OmXFdU0IJc3fJQomQgAyom7BQbnGVwHu36JYnUTO81qF8Ej20AhVeWQ4rn+7+9abt4G6lv pAkNeBvVUQPmGSb4+ELIIzK+NhoKTicRJ8fzd7twaCuO5wUnNNPCcm/SomRx91s0tg5Gmd bdgCMSQ6U+Z6FsNYO5cYvQL3C680e0k= X-MC-Unique: ajnuAosFNcSaTEGSAtDvhQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Stefano Garzarella , Shannon Nelson , Jason Wang , Gautam Dawar , Laurent Vivier , alvaro.karsz@solid-run.com, longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefan Hajnoczi , Cindy Lu , "Michael S. Tsirkin" , si-wei.liu@oracle.com, Liuxiangdong , Parav Pandit , Eli Cohen , Zhu Lingshan , Harpreet Singh Anand , "Gonglei (Arei)" , Lei Yang Subject: [PATCH v4 01/15] vdpa net: move iova tree creation from init to start Date: Fri, 24 Feb 2023 16:54:24 +0100 Message-Id: <20230224155438.112797-2-eperezma@redhat.com> In-Reply-To: <20230224155438.112797-1-eperezma@redhat.com> References: <20230224155438.112797-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1677254116796100001 Only create iova_tree if and when it is needed. The cleanup keeps being responsible of last VQ but this change allows it to merge both cleanup functions. Signed-off-by: Eugenio P=C3=A9rez Acked-by: Jason Wang Tested-by: Alvaro Karsz --- v4: * Remove leak of iova_tree because double allocation * Document better the sharing of IOVA tree between data and CVQ --- net/vhost-vdpa.c | 113 ++++++++++++++++++++++++++++++++++------------- 1 file changed, 83 insertions(+), 30 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index de5ed8ff22..b89c99066a 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -178,13 +178,9 @@ err_init: static void vhost_vdpa_cleanup(NetClientState *nc) { VhostVDPAState *s =3D DO_UPCAST(VhostVDPAState, nc, nc); - struct vhost_dev *dev =3D &s->vhost_net->dev; =20 qemu_vfree(s->cvq_cmd_out_buffer); qemu_vfree(s->status); - if (dev->vq_index + dev->nvqs =3D=3D dev->vq_index_end) { - g_clear_pointer(&s->vhost_vdpa.iova_tree, vhost_iova_tree_delete); - } if (s->vhost_net) { vhost_net_cleanup(s->vhost_net); g_free(s->vhost_net); @@ -234,10 +230,64 @@ static ssize_t vhost_vdpa_receive(NetClientState *nc,= const uint8_t *buf, return size; } =20 +/** From any vdpa net client, get the netclient of first queue pair */ +static VhostVDPAState *vhost_vdpa_net_first_nc_vdpa(VhostVDPAState *s) +{ + NICState *nic =3D qemu_get_nic(s->nc.peer); + NetClientState *nc0 =3D qemu_get_peer(nic->ncs, 0); + + return DO_UPCAST(VhostVDPAState, nc, nc0); +} + +static void vhost_vdpa_net_data_start_first(VhostVDPAState *s) +{ + struct vhost_vdpa *v =3D &s->vhost_vdpa; + + if (v->shadow_vqs_enabled) { + v->iova_tree =3D vhost_iova_tree_new(v->iova_range.first, + v->iova_range.last); + } +} + +static int vhost_vdpa_net_data_start(NetClientState *nc) +{ + VhostVDPAState *s =3D DO_UPCAST(VhostVDPAState, nc, nc); + struct vhost_vdpa *v =3D &s->vhost_vdpa; + + assert(nc->info->type =3D=3D NET_CLIENT_DRIVER_VHOST_VDPA); + + if (v->index =3D=3D 0) { + vhost_vdpa_net_data_start_first(s); + return 0; + } + + if (v->shadow_vqs_enabled) { + VhostVDPAState *s0 =3D vhost_vdpa_net_first_nc_vdpa(s); + v->iova_tree =3D s0->vhost_vdpa.iova_tree; + } + + return 0; +} + +static void vhost_vdpa_net_client_stop(NetClientState *nc) +{ + VhostVDPAState *s =3D DO_UPCAST(VhostVDPAState, nc, nc); + struct vhost_dev *dev; + + assert(nc->info->type =3D=3D NET_CLIENT_DRIVER_VHOST_VDPA); + + dev =3D s->vhost_vdpa.dev; + if (dev->vq_index + dev->nvqs =3D=3D dev->vq_index_end) { + g_clear_pointer(&s->vhost_vdpa.iova_tree, vhost_iova_tree_delete); + } +} + static NetClientInfo net_vhost_vdpa_info =3D { .type =3D NET_CLIENT_DRIVER_VHOST_VDPA, .size =3D sizeof(VhostVDPAState), .receive =3D vhost_vdpa_receive, + .start =3D vhost_vdpa_net_data_start, + .stop =3D vhost_vdpa_net_client_stop, .cleanup =3D vhost_vdpa_cleanup, .has_vnet_hdr =3D vhost_vdpa_has_vnet_hdr, .has_ufo =3D vhost_vdpa_has_ufo, @@ -351,7 +401,7 @@ dma_map_err: =20 static int vhost_vdpa_net_cvq_start(NetClientState *nc) { - VhostVDPAState *s; + VhostVDPAState *s, *s0; struct vhost_vdpa *v; uint64_t backend_features; int64_t cvq_group; @@ -415,8 +465,6 @@ static int vhost_vdpa_net_cvq_start(NetClientState *nc) return r; } =20 - v->iova_tree =3D vhost_iova_tree_new(v->iova_range.first, - v->iova_range.last); v->shadow_vqs_enabled =3D true; s->vhost_vdpa.address_space_id =3D VHOST_VDPA_NET_CVQ_ASID; =20 @@ -425,6 +473,27 @@ out: return 0; } =20 + s0 =3D vhost_vdpa_net_first_nc_vdpa(s); + if (s0->vhost_vdpa.iova_tree) { + /* + * SVQ is already configured for all virtqueues. Reuse IOVA tree = for + * simplicity, wether CVQ shares ASID with guest or not, because: + * - Memory listener need access to guest's memory addresses alloc= ated + * in the IOVA tree. + * - There should be plenty of IOVA address space for both ASID no= t to + * worry about collisions between them. Guest's translations are + * still validated with virtio virtqueue_pop so there is no risk= for + * the guest to access memory it shouldn't. + * + * To allocate a iova tree per ASID is doable but it complicates t= he + * code and it is not worth for the moment. + */ + v->iova_tree =3D s0->vhost_vdpa.iova_tree; + } else { + v->iova_tree =3D vhost_iova_tree_new(v->iova_range.first, + v->iova_range.last); + } + r =3D vhost_vdpa_cvq_map_buf(&s->vhost_vdpa, s->cvq_cmd_out_buffer, vhost_vdpa_net_cvq_cmd_page_len(), false); if (unlikely(r < 0)) { @@ -449,15 +518,9 @@ static void vhost_vdpa_net_cvq_stop(NetClientState *nc) if (s->vhost_vdpa.shadow_vqs_enabled) { vhost_vdpa_cvq_unmap_buf(&s->vhost_vdpa, s->cvq_cmd_out_buffer); vhost_vdpa_cvq_unmap_buf(&s->vhost_vdpa, s->status); - if (!s->always_svq) { - /* - * If only the CVQ is shadowed we can delete this safely. - * If all the VQs are shadows this will be needed by the time = the - * device is started again to register SVQ vrings and similar. - */ - g_clear_pointer(&s->vhost_vdpa.iova_tree, vhost_iova_tree_dele= te); - } } + + vhost_vdpa_net_client_stop(nc); } =20 static ssize_t vhost_vdpa_net_cvq_add(VhostVDPAState *s, size_t out_len, @@ -667,8 +730,7 @@ static NetClientState *net_vhost_vdpa_init(NetClientSta= te *peer, int nvqs, bool is_datapath, bool svq, - struct vhost_vdpa_iova_range iova_r= ange, - VhostIOVATree *iova_tree) + struct vhost_vdpa_iova_range iova_r= ange) { NetClientState *nc =3D NULL; VhostVDPAState *s; @@ -690,7 +752,6 @@ static NetClientState *net_vhost_vdpa_init(NetClientSta= te *peer, s->vhost_vdpa.shadow_vqs_enabled =3D svq; s->vhost_vdpa.iova_range =3D iova_range; s->vhost_vdpa.shadow_data =3D svq; - s->vhost_vdpa.iova_tree =3D iova_tree; if (!is_datapath) { s->cvq_cmd_out_buffer =3D qemu_memalign(qemu_real_host_page_size(), vhost_vdpa_net_cvq_cmd_page_le= n()); @@ -760,7 +821,6 @@ int net_init_vhost_vdpa(const Netdev *netdev, const cha= r *name, uint64_t features; int vdpa_device_fd; g_autofree NetClientState **ncs =3D NULL; - g_autoptr(VhostIOVATree) iova_tree =3D NULL; struct vhost_vdpa_iova_range iova_range; NetClientState *nc; int queue_pairs, r, i =3D 0, has_cvq =3D 0; @@ -812,12 +872,8 @@ int net_init_vhost_vdpa(const Netdev *netdev, const ch= ar *name, goto err; } =20 - if (opts->x_svq) { - if (!vhost_vdpa_net_valid_svq_features(features, errp)) { - goto err_svq; - } - - iova_tree =3D vhost_iova_tree_new(iova_range.first, iova_range.las= t); + if (opts->x_svq && !vhost_vdpa_net_valid_svq_features(features, errp))= { + goto err; } =20 ncs =3D g_malloc0(sizeof(*ncs) * queue_pairs); @@ -825,7 +881,7 @@ int net_init_vhost_vdpa(const Netdev *netdev, const cha= r *name, for (i =3D 0; i < queue_pairs; i++) { ncs[i] =3D net_vhost_vdpa_init(peer, TYPE_VHOST_VDPA, name, vdpa_device_fd, i, 2, true, opts->x_s= vq, - iova_range, iova_tree); + iova_range); if (!ncs[i]) goto err; } @@ -833,13 +889,11 @@ int net_init_vhost_vdpa(const Netdev *netdev, const c= har *name, if (has_cvq) { nc =3D net_vhost_vdpa_init(peer, TYPE_VHOST_VDPA, name, vdpa_device_fd, i, 1, false, - opts->x_svq, iova_range, iova_tree); + opts->x_svq, iova_range); if (!nc) goto err; } =20 - /* iova_tree ownership belongs to last NetClientState */ - g_steal_pointer(&iova_tree); return 0; =20 err: @@ -849,7 +903,6 @@ err: } } =20 -err_svq: qemu_close(vdpa_device_fd); =20 return -1; --=20 2.31.1 From nobody Tue May 14 19:56:46 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1677255866; cv=none; d=zohomail.com; s=zohoarc; b=azRInJuqiOtNGbUqjtBX4xOCiJklJgZ5YTpzzZpxPFlUXqfQxqImLRuVTpD4eGmcgI6nGNztDoUUlORiKsksntAtk0OTU3SQQdzUHV8RyOjXxn7qKRepDKe3Nh75nnKPMyCvCbEnob2RQGDKF2N1bHWnLByzCjG8Br7Vkqo7BUs= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677255866; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=X3d9BwbOucUxVtqeuEr+XHyMgQkL0UWAtdL7LMA7Nko=; b=LjN7d6r79IN1IHYnKGYDC/4OK8sevKRPqHWID4EXSJa9lY9zR0DwyMI2SRbciWMVKN/yogd8UJWB6lSSkEclTU9MSAliV94OcBhYBj30oUBT3IgCh7Pt7QxRCALnYORZXhsBoSLkGQicjPmTrC6r23WmPTSNHsiqPNOEdFH+jOg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677255866578630.5024193825212; Fri, 24 Feb 2023 08:24:26 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pVaPT-0005MC-95; Fri, 24 Feb 2023 10:54:59 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaPR-0005Kq-Au for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:54:57 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaPP-0005LS-Kp for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:54:56 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-232-laz-r6r2MZO45d8sZ3wUkg-1; Fri, 24 Feb 2023 10:54:51 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id AD40E882822; Fri, 24 Feb 2023 15:54:50 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.85]) by smtp.corp.redhat.com (Postfix) with ESMTP id BC5D5C15BA0; Fri, 24 Feb 2023 15:54:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677254095; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=X3d9BwbOucUxVtqeuEr+XHyMgQkL0UWAtdL7LMA7Nko=; b=V5st54cPMR0Lp34oIjdveYy4kleKpLVjtW+zL4wQ9/rM5gL8sZfwhIs88FSJqG4K24gQF6 FcLWWocol/fexLfSb9lZBvu5xXM+TO8vzJAJzssyP7ubtOPZ1f80Yw8XKv+2JYuY6JoZpj lYiQYxAdC5zDyKKvjQ+eXEN2i02onu8= X-MC-Unique: laz-r6r2MZO45d8sZ3wUkg-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Stefano Garzarella , Shannon Nelson , Jason Wang , Gautam Dawar , Laurent Vivier , alvaro.karsz@solid-run.com, longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefan Hajnoczi , Cindy Lu , "Michael S. Tsirkin" , si-wei.liu@oracle.com, Liuxiangdong , Parav Pandit , Eli Cohen , Zhu Lingshan , Harpreet Singh Anand , "Gonglei (Arei)" , Lei Yang Subject: [PATCH v4 02/15] vdpa: Remember last call fd set Date: Fri, 24 Feb 2023 16:54:25 +0100 Message-Id: <20230224155438.112797-3-eperezma@redhat.com> In-Reply-To: <20230224155438.112797-1-eperezma@redhat.com> References: <20230224155438.112797-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1677255868413100007 As SVQ can be enabled dynamically at any time, it needs to store call fd always. Signed-off-by: Eugenio P=C3=A9rez Tested-by: Alvaro Karsz --- hw/virtio/vhost-vdpa.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 542e003101..4f72a52a43 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -1240,16 +1240,16 @@ static int vhost_vdpa_set_vring_call(struct vhost_d= ev *dev, struct vhost_vring_file *file) { struct vhost_vdpa *v =3D dev->opaque; + int vdpa_idx =3D file->index - dev->vq_index; + VhostShadowVirtqueue *svq =3D g_ptr_array_index(v->shadow_vqs, vdpa_id= x); =20 + /* Remember last call fd because we can switch to SVQ anytime. */ + vhost_svq_set_svq_call_fd(svq, file->fd); if (v->shadow_vqs_enabled) { - int vdpa_idx =3D file->index - dev->vq_index; - VhostShadowVirtqueue *svq =3D g_ptr_array_index(v->shadow_vqs, vdp= a_idx); - - vhost_svq_set_svq_call_fd(svq, file->fd); return 0; - } else { - return vhost_vdpa_set_vring_dev_call(dev, file); } + + return vhost_vdpa_set_vring_dev_call(dev, file); } =20 static int vhost_vdpa_get_features(struct vhost_dev *dev, --=20 2.31.1 From nobody Tue May 14 19:56:46 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1677255442; cv=none; d=zohomail.com; s=zohoarc; b=IAQZuBXy4jfB2cmgik/Xt4uXrwcOcoiasP4lyeNR14cPj9BdqBlEhMePCTGMTtvpGEa33rpsVkYo8Dd5I9iEWlo/QX2rhdwVvltw6sThsFrUE1Ho8RbkjeyOj+g/n6q2aOxAqjgnGTwoy8+NM9Nh2JJPoWtK4FtpQ8YyW+48z5Y= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677255442; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=eEeRfwwIN44/Adg4INU1tbOT+msU8OVisKYfCUJYZrI=; b=THzp9AIuOomlJm/ldiykpqph+rdiowhcX1aVg1tvkC/S5x7VMrmSAdtgg1rOlGndgOhetYLLtcSzZAa96DUFhQ7+nGrS0nKJlfDDa0bsT9jV5iGCes27eDEiPX+AshKsTzjfUJT+C0NfBSHcteZRiftCMbxndAwpPOUOxOidZuw= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677255442937348.6665852040487; Fri, 24 Feb 2023 08:17:22 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pVaPb-0005P3-1X; Fri, 24 Feb 2023 10:55:07 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaPX-0005N9-Qe for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:04 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaPW-0005Wa-3b for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:03 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-364--KdNxvwhNM6j9Dufd5G8tg-1; Fri, 24 Feb 2023 10:54:55 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id CA0523804530; Fri, 24 Feb 2023 15:54:53 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.85]) by smtp.corp.redhat.com (Postfix) with ESMTP id EA0D7C15BA0; Fri, 24 Feb 2023 15:54:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677254101; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=eEeRfwwIN44/Adg4INU1tbOT+msU8OVisKYfCUJYZrI=; b=SJwdJFZDwhsNxjDh3oJ4kZfbQU/1CymqAzzoQnzCnUgvp+OovF3roeatT/e9KLPEaJ6a3R ft66eizvnByeuGa9VyhQxwUD69SRqut4v+Xo2al5DvzxoSP0+NNiloX1bIaawHI0rYkwUq nbw/l1WUQUbpEBV+OLFORlZWIjNOTg4= X-MC-Unique: -KdNxvwhNM6j9Dufd5G8tg-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Stefano Garzarella , Shannon Nelson , Jason Wang , Gautam Dawar , Laurent Vivier , alvaro.karsz@solid-run.com, longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefan Hajnoczi , Cindy Lu , "Michael S. Tsirkin" , si-wei.liu@oracle.com, Liuxiangdong , Parav Pandit , Eli Cohen , Zhu Lingshan , Harpreet Singh Anand , "Gonglei (Arei)" , Lei Yang Subject: [PATCH v4 03/15] vdpa: stop svq at vhost_vdpa_dev_start(false) Date: Fri, 24 Feb 2023 16:54:26 +0100 Message-Id: <20230224155438.112797-4-eperezma@redhat.com> In-Reply-To: <20230224155438.112797-1-eperezma@redhat.com> References: <20230224155438.112797-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1677255444792100003 It used to be done at vhost_vdpa_svq_cleanup, since a device couldn't switch to SVQ mode dynamically. Now that we need to fetch the state and trust SVQ will not modify guest's used_idx at migration, stop effectively SVQ at suspend time, as a real device would do. Leaving old vhost_svq_stop call at vhost_vdpa_svq_cleanup, as its supported to call it many times and it follows other operations that are called redundantly there too: * vhost_vdpa_host_notifiers_uninit * memory_listener_unregister Signed-off-by: Eugenio P=C3=A9rez Tested-by: Alvaro Karsz --- v3: New in v3 --- hw/virtio/vhost-vdpa.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 4f72a52a43..d9260191cc 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -1100,10 +1100,11 @@ static void vhost_vdpa_svqs_stop(struct vhost_dev *= dev) =20 for (unsigned i =3D 0; i < v->shadow_vqs->len; ++i) { VhostShadowVirtqueue *svq =3D g_ptr_array_index(v->shadow_vqs, i); + + vhost_svq_stop(svq); vhost_vdpa_svq_unmap_rings(dev, svq); =20 event_notifier_cleanup(&svq->hdev_kick); - event_notifier_cleanup(&svq->hdev_call); } } =20 --=20 2.31.1 From nobody Tue May 14 19:56:46 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1677255740; cv=none; d=zohomail.com; s=zohoarc; b=byEMTkI5yRJC/rvXowbMVymj9+/u5gAoC5Jc313dKM+9aM4gqa3n/BT46W29SFneFaqplrikHkijJsURI4zCo1HhH6q+qeWOKHvT8x1JMu9uhxeAH6VeK2rDG3xmsPWq+dYwq8TykEFlDmPY5npyuINnBEGciwBdDl8xFM0XLTU= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677255740; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=uVDfF3gE5yt1GKH79JIBjQUvPHWgAmQZJRipJiUPUtI=; b=IQ3gA4hj8v4IO2SxOIlIKg+gZX7Rr39V2oWdfxGphPvD0Ok32nQiDTm1Lnqer8EpVUONf0/IRU91hzajfTQD5qMTfOqbdv8MO0okyXtdIW+/r/5zUhtV8UNGrrKe/BanXayjUYPPdVUNDLeAtYxRwL94sWgn7bH+uQoGcUPEij8= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677255740976734.5996424356811; Fri, 24 Feb 2023 08:22:20 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pVaPa-0005NV-3h; Fri, 24 Feb 2023 10:55:06 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaPW-0005Mp-FQ for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:02 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaPU-0005WE-TB for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:02 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-20-oZtcjDPhP5OtX4b_MwnFQA-1; Fri, 24 Feb 2023 10:54:57 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E41C13C02527; Fri, 24 Feb 2023 15:54:56 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.85]) by smtp.corp.redhat.com (Postfix) with ESMTP id 11BBBC15BA0; Fri, 24 Feb 2023 15:54:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677254100; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=uVDfF3gE5yt1GKH79JIBjQUvPHWgAmQZJRipJiUPUtI=; b=Nv2s53HY4L02drMUBH7TjOFtQIqxsfvX0CoZFUiPDWyvkqOin+OeWNMwrp9ULbagxW/UAb Fh7NmbTE42IoGmdf1in0zMokY00PT22MuG8NbrxgOsSgjx0+LZjGTvox84//++AKO3mdcu 85nAX/aXWL4NNZN1Tzn6mbJi8HTh5EE= X-MC-Unique: oZtcjDPhP5OtX4b_MwnFQA-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Stefano Garzarella , Shannon Nelson , Jason Wang , Gautam Dawar , Laurent Vivier , alvaro.karsz@solid-run.com, longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefan Hajnoczi , Cindy Lu , "Michael S. Tsirkin" , si-wei.liu@oracle.com, Liuxiangdong , Parav Pandit , Eli Cohen , Zhu Lingshan , Harpreet Singh Anand , "Gonglei (Arei)" , Lei Yang Subject: [PATCH v4 04/15] vdpa: Negotiate _F_SUSPEND feature Date: Fri, 24 Feb 2023 16:54:27 +0100 Message-Id: <20230224155438.112797-5-eperezma@redhat.com> In-Reply-To: <20230224155438.112797-1-eperezma@redhat.com> References: <20230224155438.112797-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1677255741375100001 This is needed for qemu to know it can suspend the device to retrieve its status and enable SVQ with it, so all the process is transparent to the guest. Signed-off-by: Eugenio P=C3=A9rez Acked-by: Jason Wang Tested-by: Alvaro Karsz --- hw/virtio/vhost-vdpa.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index d9260191cc..4fac144169 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -659,7 +659,8 @@ static int vhost_vdpa_set_backend_cap(struct vhost_dev = *dev) uint64_t features; uint64_t f =3D 0x1ULL << VHOST_BACKEND_F_IOTLB_MSG_V2 | 0x1ULL << VHOST_BACKEND_F_IOTLB_BATCH | - 0x1ULL << VHOST_BACKEND_F_IOTLB_ASID; + 0x1ULL << VHOST_BACKEND_F_IOTLB_ASID | + 0x1ULL << VHOST_BACKEND_F_SUSPEND; int r; =20 if (vhost_vdpa_call(dev, VHOST_GET_BACKEND_FEATURES, &features)) { --=20 2.31.1 From nobody Tue May 14 19:56:46 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1677255510; cv=none; d=zohomail.com; s=zohoarc; b=VKRDQmoqwTHCXiVSmRRv4tinG0bMU523WJS/+T1pz6HdmCVMRGdsSHOvGeVhFVmyWMbOKQUIqi9dc6C9s85LYcfTGv5rxP+jIMtCA1iHaxthd7asNQQFlNc+e3+MDpu2FcROT6lHmpCTQfI+VuMsm7qlGcU6aHjqGbytWy01LrE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677255510; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=aMIjnkhrgHYDFpkFxARwsLF1Xtb7c1kMNSbetYSXft4=; b=eI4xgCFY8XwQELKAZcKeqHjBvqFE21FGNdKevsk2ydc5qokrmfSLYR3J75/Q40koVBa+O6mZy2ZDMJ29+1+Ql3ZSNArf3PCdOYWroQIrLuiZK7ThMoeb1UnHc41qgbS017Mrn4Z9z9gnn6OY1y7rgJ4YChj+n2lZ5RZAmBYylZU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677255510583602.6116437446278; Fri, 24 Feb 2023 08:18:30 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pVaQ7-0005gz-2W; Fri, 24 Feb 2023 10:55:39 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaPd-0005QM-7n for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:09 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaPa-0005l3-Hg for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:08 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-631-BYy73vIuMSCDuH25ETe3xQ-1; Fri, 24 Feb 2023 10:55:02 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E4E2A101A521; Fri, 24 Feb 2023 15:55:00 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.85]) by smtp.corp.redhat.com (Postfix) with ESMTP id 31715C15BA0; Fri, 24 Feb 2023 15:54:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677254104; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=aMIjnkhrgHYDFpkFxARwsLF1Xtb7c1kMNSbetYSXft4=; b=ex68pfXidLqu3okqLZsHwz1jUzA03QwlFzob5rRoqfQ7a6bDtWiT/hdP3z/t9XHvIsAhw9 Ius3C7C4gEfESRwjb8BioBATITw0gZd7wErAaZ7kk8mnGDw8R8zEyXoCtdrfpWpN6NcZTL y7tIiJ5LNxg9SpKo4440L8/nyo9+AaA= X-MC-Unique: BYy73vIuMSCDuH25ETe3xQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Stefano Garzarella , Shannon Nelson , Jason Wang , Gautam Dawar , Laurent Vivier , alvaro.karsz@solid-run.com, longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefan Hajnoczi , Cindy Lu , "Michael S. Tsirkin" , si-wei.liu@oracle.com, Liuxiangdong , Parav Pandit , Eli Cohen , Zhu Lingshan , Harpreet Singh Anand , "Gonglei (Arei)" , Lei Yang Subject: [PATCH v4 05/15] vdpa: move vhost reset after get vring base Date: Fri, 24 Feb 2023 16:54:28 +0100 Message-Id: <20230224155438.112797-6-eperezma@redhat.com> In-Reply-To: <20230224155438.112797-1-eperezma@redhat.com> References: <20230224155438.112797-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1677255511443100001 The function vhost.c:vhost_dev_stop calls vhost operation vhost_dev_start(false). In the case of vdpa it totally reset and wipes the device, making the fetching of the vring base (virtqueue state) totally useless. The kernel backend does not use vhost_dev_start vhost op callback, but vhost-user do. A patch to make vhost_user_dev_start more similar to vdpa is desirable, but it can be added on top. Signed-off-by: Eugenio P=C3=A9rez Tested-by: Alvaro Karsz --- include/hw/virtio/vhost-backend.h | 4 ++++ hw/virtio/vhost-vdpa.c | 22 ++++++++++++++++------ hw/virtio/vhost.c | 3 +++ 3 files changed, 23 insertions(+), 6 deletions(-) diff --git a/include/hw/virtio/vhost-backend.h b/include/hw/virtio/vhost-ba= ckend.h index c5ab49051e..ec3fbae58d 100644 --- a/include/hw/virtio/vhost-backend.h +++ b/include/hw/virtio/vhost-backend.h @@ -130,6 +130,9 @@ typedef bool (*vhost_force_iommu_op)(struct vhost_dev *= dev); =20 typedef int (*vhost_set_config_call_op)(struct vhost_dev *dev, int fd); + +typedef void (*vhost_reset_status_op)(struct vhost_dev *dev); + typedef struct VhostOps { VhostBackendType backend_type; vhost_backend_init vhost_backend_init; @@ -177,6 +180,7 @@ typedef struct VhostOps { vhost_get_device_id_op vhost_get_device_id; vhost_force_iommu_op vhost_force_iommu; vhost_set_config_call_op vhost_set_config_call; + vhost_reset_status_op vhost_reset_status; } VhostOps; =20 int vhost_backend_update_device_iotlb(struct vhost_dev *dev, diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 4fac144169..8cc9c98db9 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -1134,14 +1134,23 @@ static int vhost_vdpa_dev_start(struct vhost_dev *d= ev, bool started) if (started) { memory_listener_register(&v->listener, &address_space_memory); return vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_DRIVER_OK); - } else { - vhost_vdpa_reset_device(dev); - vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE | - VIRTIO_CONFIG_S_DRIVER); - memory_listener_unregister(&v->listener); + } =20 - return 0; + return 0; +} + +static void vhost_vdpa_reset_status(struct vhost_dev *dev) +{ + struct vhost_vdpa *v =3D dev->opaque; + + if (dev->vq_index + dev->nvqs !=3D dev->vq_index_end) { + return; } + + vhost_vdpa_reset_device(dev); + vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE | + VIRTIO_CONFIG_S_DRIVER); + memory_listener_unregister(&v->listener); } =20 static int vhost_vdpa_set_log_base(struct vhost_dev *dev, uint64_t base, @@ -1328,4 +1337,5 @@ const VhostOps vdpa_ops =3D { .vhost_vq_get_addr =3D vhost_vdpa_vq_get_addr, .vhost_force_iommu =3D vhost_vdpa_force_iommu, .vhost_set_config_call =3D vhost_vdpa_set_config_call, + .vhost_reset_status =3D vhost_vdpa_reset_status, }; diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index eb8c4c378c..a266396576 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -2049,6 +2049,9 @@ void vhost_dev_stop(struct vhost_dev *hdev, VirtIODev= ice *vdev, bool vrings) hdev->vqs + i, hdev->vq_index + i); } + if (hdev->vhost_ops->vhost_reset_status) { + hdev->vhost_ops->vhost_reset_status(hdev); + } =20 if (vhost_dev_has_iommu(hdev)) { if (hdev->vhost_ops->vhost_set_iotlb_callback) { --=20 2.31.1 From nobody Tue May 14 19:56:46 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1677255992; cv=none; d=zohomail.com; s=zohoarc; b=NwUXU6hEC0aJXFbWXRH9GuIf/OTPVLQUYGDLe0IFBC5/Lf52KdnvDOjc6nOTvXaOkkiu0CTyZGHm3tRFGm0eT+t58o7HTGY31wwWHa090N1XWJIJlieQpqCG67/Pm8U+gdo8aMmDh+jkI2FmWVXYrJfCSfIWq1NeD+tSgexduRY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677255992; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=9Sukp9MJ69DF3oRSxKsPhTznKScaRR+4KStzBHFuVP8=; b=icG7rqeFehfpGO875tQZJvi8UH2rUKTe6i+1qoqf7SHpzTSM9lABcAya9ffCZ7UHM+rFuc+KDZjzGaPfCK9MEM+OG/nokv8OAzvZV+8lOMUDFgTbrgFDzdFpp4M9awUyD3RRLVkkf4UIHs2uAyovdyMEq5tbBry8c3F3tkCa9Oc= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677255992849331.1078843429177; Fri, 24 Feb 2023 08:26:32 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pVaQ9-0005oK-9N; Fri, 24 Feb 2023 10:55:41 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaPf-0005Rf-6z for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:13 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaPd-0005qC-KA for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:10 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-214-APFbdkNjP3yQR3U48SMq4w-1; Fri, 24 Feb 2023 10:55:04 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 096FF848D92; Fri, 24 Feb 2023 15:55:04 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.85]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2CF45C15BA0; Fri, 24 Feb 2023 15:55:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677254109; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9Sukp9MJ69DF3oRSxKsPhTznKScaRR+4KStzBHFuVP8=; b=Y5n+WgMyxCyVCypyDqRI3p0IC0yTYcTrDrZBdB+icSw8rXS16vv6PavfFtEb8h8ZdrS5L7 oAVhHm5Uy2jnZpcxsvX3gekghrLwqP+dfC9/krQNwRS+DCwv95yZgSNLRpbatnW77QUOHm tslBnNnOE0rzlkRQ2mDxTe8px60ugCM= X-MC-Unique: APFbdkNjP3yQR3U48SMq4w-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Stefano Garzarella , Shannon Nelson , Jason Wang , Gautam Dawar , Laurent Vivier , alvaro.karsz@solid-run.com, longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefan Hajnoczi , Cindy Lu , "Michael S. Tsirkin" , si-wei.liu@oracle.com, Liuxiangdong , Parav Pandit , Eli Cohen , Zhu Lingshan , Harpreet Singh Anand , "Gonglei (Arei)" , Lei Yang Subject: [PATCH v4 06/15] vdpa: add vhost_vdpa->suspended parameter Date: Fri, 24 Feb 2023 16:54:29 +0100 Message-Id: <20230224155438.112797-7-eperezma@redhat.com> In-Reply-To: <20230224155438.112797-1-eperezma@redhat.com> References: <20230224155438.112797-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1677255995112100003 This allows vhost_vdpa to track if it is safe to get vring base from the device or not. If it is not, vhost can fall back to fetch idx from the guest buffer again. No functional change intended in this patch, later patches will use this field. Signed-off-by: Eugenio P=C3=A9rez Tested-by: Alvaro Karsz --- include/hw/virtio/vhost-vdpa.h | 2 ++ hw/virtio/vhost-vdpa.c | 8 ++++++++ 2 files changed, 10 insertions(+) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index 7997f09a8d..4a7d396674 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -42,6 +42,8 @@ typedef struct vhost_vdpa { bool shadow_vqs_enabled; /* Vdpa must send shadow addresses as IOTLB key for data queues, not G= PA */ bool shadow_data; + /* Device suspended successfully */ + bool suspended; /* IOVA mapping used by the Shadow Virtqueue */ VhostIOVATree *iova_tree; GPtrArray *shadow_vqs; diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 8cc9c98db9..228677895a 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -1227,6 +1227,14 @@ static int vhost_vdpa_get_vring_base(struct vhost_de= v *dev, return 0; } =20 + if (!v->suspended) { + /* + * Cannot trust in value returned by device, let vhost recover used + * idx from guest. + */ + return -1; + } + ret =3D vhost_vdpa_call(dev, VHOST_GET_VRING_BASE, ring); trace_vhost_vdpa_get_vring_base(dev, ring->index, ring->num); return ret; --=20 2.31.1 From nobody Tue May 14 19:56:46 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1677256516; cv=none; d=zohomail.com; s=zohoarc; b=PX+distDDIQVoU+3nT3c/JciWGlJ9ZaQyuKyeay4QnhHabFM9mGT9jpEec82TTNlg5kxtzIV7cFSVSebFowenYTUNgvsrHy+xyeCepo0TQVuisAl4exNOk1hM66wSSORPz3p+aeiWKGS8KN8295vdCIkLNK8fGLI3lvof+neaW4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677256516; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=BdfE0nCxLA0iEBNRdfCG3ox5I4m2523p5+WhZcH/Tzs=; b=ZDHrZIK1I6YrnL1QgKHONhjI6I5a0V4oYAMNJ4TmBldu8NPbO0mXu43DEsoqK+AxaYHuLqMkVyha/18Czc4cUGPz/tBACghjUdR8kSMYAyLaN4Ji0eDkqy3++DywtBM7csPbDQaGjQiu8USKICJBcjdSB2K11ombm14s7Cn39ew= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677256516135121.9698280435681; Fri, 24 Feb 2023 08:35:16 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pVaQB-00061J-CQ; Fri, 24 Feb 2023 10:55:43 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaPn-0005cZ-7F for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:21 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaPg-0005sL-DL for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:14 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-311-CBFGFpjNNOycquzJM8dPCA-1; Fri, 24 Feb 2023 10:55:08 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 2554F1C05B02; Fri, 24 Feb 2023 15:55:07 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.85]) by smtp.corp.redhat.com (Postfix) with ESMTP id 478BCC15BA0; Fri, 24 Feb 2023 15:55:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677254111; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=BdfE0nCxLA0iEBNRdfCG3ox5I4m2523p5+WhZcH/Tzs=; b=HUMqf3jhzQdDNFz12RLuZoNMjKtSl64XiSEnkRc8V1KS67cm2WB/cO6SR6gNAWJkEi7lVU ntXJhN/SV4dYQXTZsr6VBZ4ZVcNbe+cqYEQSkDPmsWv+ZbnjVmNI6yX05h5R4FA8NbTQ2a Vc8YnS38kORwHL7MyzoWRg+RH5OwvoI= X-MC-Unique: CBFGFpjNNOycquzJM8dPCA-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Stefano Garzarella , Shannon Nelson , Jason Wang , Gautam Dawar , Laurent Vivier , alvaro.karsz@solid-run.com, longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefan Hajnoczi , Cindy Lu , "Michael S. Tsirkin" , si-wei.liu@oracle.com, Liuxiangdong , Parav Pandit , Eli Cohen , Zhu Lingshan , Harpreet Singh Anand , "Gonglei (Arei)" , Lei Yang Subject: [PATCH v4 07/15] vdpa: add vhost_vdpa_suspend Date: Fri, 24 Feb 2023 16:54:30 +0100 Message-Id: <20230224155438.112797-8-eperezma@redhat.com> In-Reply-To: <20230224155438.112797-1-eperezma@redhat.com> References: <20230224155438.112797-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1677256517888100001 The function vhost.c:vhost_dev_stop fetches the vring base so the vq state can be migrated to other devices. However, this is unreliable in vdpa, since we didn't signal the device to suspend the queues, making the value fetched useless. Suspend the device if possible before fetching first and subsequent vring bases. Moreover, vdpa totally reset and wipes the device at the last device before fetch its vrings base, making that operation useless in the last device. This will be fixed in later patches of this series. Signed-off-by: Eugenio P=C3=A9rez Tested-by: Alvaro Karsz --- v4: * Look for _F_SUSPEND at vhost_dev->backend_cap, not backend_features * Fall back on reset & fetch used idx from guest's memory --- hw/virtio/vhost-vdpa.c | 25 +++++++++++++++++++++++++ hw/virtio/trace-events | 1 + 2 files changed, 26 insertions(+) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 228677895a..f542960a64 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -712,6 +712,7 @@ static int vhost_vdpa_reset_device(struct vhost_dev *de= v) =20 ret =3D vhost_vdpa_call(dev, VHOST_VDPA_SET_STATUS, &status); trace_vhost_vdpa_reset_device(dev, status); + v->suspended =3D false; return ret; } =20 @@ -1109,6 +1110,29 @@ static void vhost_vdpa_svqs_stop(struct vhost_dev *d= ev) } } =20 +static void vhost_vdpa_suspend(struct vhost_dev *dev) +{ + struct vhost_vdpa *v =3D dev->opaque; + int r; + + if (!vhost_vdpa_first_dev(dev)) { + return; + } + + if (!(dev->backend_cap & BIT_ULL(VHOST_BACKEND_F_SUSPEND))) { + trace_vhost_vdpa_suspend(dev); + r =3D ioctl(v->device_fd, VHOST_VDPA_SUSPEND); + if (unlikely(r)) { + error_report("Cannot suspend: %s(%d)", g_strerror(errno), errn= o); + } else { + v->suspended =3D true; + return; + } + } + + vhost_vdpa_reset_device(dev); +} + static int vhost_vdpa_dev_start(struct vhost_dev *dev, bool started) { struct vhost_vdpa *v =3D dev->opaque; @@ -1123,6 +1147,7 @@ static int vhost_vdpa_dev_start(struct vhost_dev *dev= , bool started) } vhost_vdpa_set_vring_ready(dev); } else { + vhost_vdpa_suspend(dev); vhost_vdpa_svqs_stop(dev); vhost_vdpa_host_notifiers_uninit(dev, dev->nvqs); } diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index a87c5f39a2..8f8d05cf9b 100644 --- a/hw/virtio/trace-events +++ b/hw/virtio/trace-events @@ -50,6 +50,7 @@ vhost_vdpa_set_vring_ready(void *dev) "dev: %p" vhost_vdpa_dump_config(void *dev, const char *line) "dev: %p %s" vhost_vdpa_set_config(void *dev, uint32_t offset, uint32_t size, uint32_t = flags) "dev: %p offset: %"PRIu32" size: %"PRIu32" flags: 0x%"PRIx32 vhost_vdpa_get_config(void *dev, void *config, uint32_t config_len) "dev: = %p config: %p config_len: %"PRIu32 +vhost_vdpa_suspend(void *dev) "dev: %p" vhost_vdpa_dev_start(void *dev, bool started) "dev: %p started: %d" vhost_vdpa_set_log_base(void *dev, uint64_t base, unsigned long long size,= int refcnt, int fd, void *log) "dev: %p base: 0x%"PRIx64" size: %llu refcn= t: %d fd: %d log: %p" vhost_vdpa_set_vring_addr(void *dev, unsigned int index, unsigned int flag= s, uint64_t desc_user_addr, uint64_t used_user_addr, uint64_t avail_user_ad= dr, uint64_t log_guest_addr) "dev: %p index: %u flags: 0x%x desc_user_addr:= 0x%"PRIx64" used_user_addr: 0x%"PRIx64" avail_user_addr: 0x%"PRIx64" log_g= uest_addr: 0x%"PRIx64 --=20 2.31.1 From nobody Tue May 14 19:56:46 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1677256359; cv=none; d=zohomail.com; s=zohoarc; b=LqoDOmmHqTd6rXmas2+KxXShatzvfajkmQ9PRXaFC5o+e8RdVLkB6wP5ZRYgRxDpLFs0Kf3P61e89PtreqRJsJaeHT+yPXhrtdAIIMV5BJeUq1NoIZzSBekm1nojLCwLvh7GBiLyhrjveaxQEPOM9QSyb5PRg0BtSKyBR/pJd78= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677256359; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Frk975xz6+Mcnrep3pgYyZfXIO/JS3EQ5Ult5wZhqr4=; b=SzlxSKOjVOuKNI4GrBfUdBUheTS4wwfLKHb4KbXgBpzj0rzQkfIHS1WmwRlIaUP//lzxR2E+lsmYICl9B5DutEIwZgWyVufU985Ruf3I1WVCGbZsuNwQ/5LPUBlJPm0XVoSoS1w6nxnJYLGSxKVc4N5xfqKO+Dw7QEUwpayqWfk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677256359748651.4772564472778; Fri, 24 Feb 2023 08:32:39 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pVaQD-0006Ho-8j; Fri, 24 Feb 2023 10:55:45 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaPp-0005cj-8N for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:32 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaPn-0005sz-09 for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:20 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-179-Fclhqr2UMbqu33C5kc0zuw-1; Fri, 24 Feb 2023 10:55:11 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 434923804530; Fri, 24 Feb 2023 15:55:10 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.85]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6461BC15BA0; Fri, 24 Feb 2023 15:55:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677254115; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Frk975xz6+Mcnrep3pgYyZfXIO/JS3EQ5Ult5wZhqr4=; b=TPutveKoZkJv7xKssZ6kNzyRSxpHTo+lCJOT4wi3CBA2fouy38pPThdFi5o/7zvoEMb2Kg jayBNTosuRZZ7xM3MsRqBovhnmsacW74o3Mt155l9e+8Tr920EzsHorPggPF5J2zgd4tZo wM4gsaMwjcWttK3Br61bBfj+ssEUwI0= X-MC-Unique: Fclhqr2UMbqu33C5kc0zuw-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Stefano Garzarella , Shannon Nelson , Jason Wang , Gautam Dawar , Laurent Vivier , alvaro.karsz@solid-run.com, longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefan Hajnoczi , Cindy Lu , "Michael S. Tsirkin" , si-wei.liu@oracle.com, Liuxiangdong , Parav Pandit , Eli Cohen , Zhu Lingshan , Harpreet Singh Anand , "Gonglei (Arei)" , Lei Yang Subject: [PATCH v4 08/15] vdpa: rewind at get_base, not set_base Date: Fri, 24 Feb 2023 16:54:31 +0100 Message-Id: <20230224155438.112797-9-eperezma@redhat.com> In-Reply-To: <20230224155438.112797-1-eperezma@redhat.com> References: <20230224155438.112797-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1677256361276100001 At this moment it is only possible to migrate to a vdpa device running with x-svq=3Don. As a protective measure, the rewind of the inflight descriptors was done at the destination. That way if the source sent a virtqueue with inuse descriptors they are always discarded. Since this series allows to migrate also to passthrough devices with no SVQ, the right thing to do is to rewind at the source so the base of vrings are correct. Support for inflight descriptors may be added in the future. Signed-off-by: Eugenio P=C3=A9rez Acked-by: Jason Wang Tested-by: Alvaro Karsz --- v4: * Use virtqueue_unpop at vhost_svq_stop instead of rewinding at vhost_vdpa_get_vring_base. --- hw/virtio/vhost-shadow-virtqueue.c | 8 ++++++-- hw/virtio/vhost-vdpa.c | 11 ----------- 2 files changed, 6 insertions(+), 13 deletions(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index 4307296358..523b379439 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -694,13 +694,17 @@ void vhost_svq_stop(VhostShadowVirtqueue *svq) g_autofree VirtQueueElement *elem =3D NULL; elem =3D g_steal_pointer(&svq->desc_state[i].elem); if (elem) { - virtqueue_detach_element(svq->vq, elem, 0); + /* + * TODO: This is ok for networking, but other kinds of devices + * might have problems with just unpop these. + */ + virtqueue_unpop(svq->vq, elem, 0); } } =20 next_avail_elem =3D g_steal_pointer(&svq->next_guest_avail_elem); if (next_avail_elem) { - virtqueue_detach_element(svq->vq, next_avail_elem, 0); + virtqueue_unpop(svq->vq, next_avail_elem, 0); } svq->vq =3D NULL; g_free(svq->desc_next); diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index f542960a64..71e3dc21fe 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -1218,18 +1218,7 @@ static int vhost_vdpa_set_vring_base(struct vhost_de= v *dev, struct vhost_vring_state *ring) { struct vhost_vdpa *v =3D dev->opaque; - VirtQueue *vq =3D virtio_get_queue(dev->vdev, ring->index); =20 - /* - * vhost-vdpa devices does not support in-flight requests. Set all of = them - * as available. - * - * TODO: This is ok for networking, but other kinds of devices might - * have problems with these retransmissions. - */ - while (virtqueue_rewind(vq, 1)) { - continue; - } if (v->shadow_vqs_enabled) { /* * Device vring base was set at device start. SVQ base is handled = by --=20 2.31.1 From nobody Tue May 14 19:56:46 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1677257074; cv=none; d=zohomail.com; s=zohoarc; b=FIRIDIwTqUDikvPGaOudIL7pA6lATlKHa8NfUc27QF4hpNS726ktx8eMs51JpVEm+ts5RlVEOED6hZNZ7yF6kLG12H7ZDuKIjq8xMGsPc693BZU78JlZPbtrKV6YgvelZYVgIgBL9vCTwNehaQOHWW7bxgC4+o0Zs/8C90c20sE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677257074; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=oV1Po9xq16gWsjHrzUgnPsIFEhHeMMm3iPS1GfTF2zY=; b=LgOhmyGZswjPtU4gWccAIU7viiTAw49hs4PGl99Z99S2NI0WyZcIbqL8u0NI5zqNUEJIR0K1TJlz3JJcdRdTxH1PtheOKGVLRdUB1zeH8ivW3Lo13+K1MCfSSsBrzKNeRGinWOFABTGPYl54PRFhrfjCU/l71e6UTLHQ19U0C8w= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677257074408657.3216849442397; Fri, 24 Feb 2023 08:44:34 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pVaQE-0006O6-Sv; Fri, 24 Feb 2023 10:55:47 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaPp-0005dE-Mz for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:32 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaPn-0005t5-0C for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:21 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-463-Kij6S0TkNzaPSZhZPugRkA-1; Fri, 24 Feb 2023 10:55:14 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 5F2D03804521; Fri, 24 Feb 2023 15:55:13 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.85]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7F5C0C15BA0; Fri, 24 Feb 2023 15:55:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677254118; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=oV1Po9xq16gWsjHrzUgnPsIFEhHeMMm3iPS1GfTF2zY=; b=ihWRV5TeA0Ew62Sam94riBdOcvHg6ngohhQbymwz4BL4p/Eq6EmmTAFad5c3OXKK70pvFg C7Ka4Ne2/BrbZ+v3Db/MM1l6fC8GwIFHNVGdeaF7zORhP4xvFo2WlU2Mkh7nwdwSX6696d 5onNtKtdBKbN32sA0bGCLrljy3MRndc= X-MC-Unique: Kij6S0TkNzaPSZhZPugRkA-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Stefano Garzarella , Shannon Nelson , Jason Wang , Gautam Dawar , Laurent Vivier , alvaro.karsz@solid-run.com, longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefan Hajnoczi , Cindy Lu , "Michael S. Tsirkin" , si-wei.liu@oracle.com, Liuxiangdong , Parav Pandit , Eli Cohen , Zhu Lingshan , Harpreet Singh Anand , "Gonglei (Arei)" , Lei Yang Subject: [PATCH v4 09/15] vdpa: add vdpa net migration state notifier Date: Fri, 24 Feb 2023 16:54:32 +0100 Message-Id: <20230224155438.112797-10-eperezma@redhat.com> In-Reply-To: <20230224155438.112797-1-eperezma@redhat.com> References: <20230224155438.112797-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1677257075813100002 This allows net to restart the device backend to configure SVQ on it. Ideally, these changes should not be net specific. However, the vdpa net backend is the one with enough knowledge to configure everything because of some reasons: * Queues might need to be shadowed or not depending on its kind (control vs data). * Queues need to share the same map translations (iova tree). Because of that it is cleaner to restart the whole net backend and configure again as expected, similar to how vhost-kernel moves between userspace and passthrough. If more kinds of devices need dynamic switching to SVQ we can create a callback struct like VhostOps and move most of the code there. VhostOps cannot be reused since all vdpa backend share them, and to personalize just for networking would be too heavy. Signed-off-by: Eugenio P=C3=A9rez Tested-by: Alvaro Karsz --- v4: * Delete duplication of set shadow_data and shadow_vqs_enabled moving it to data / cvq net start functions. v3: * Check for migration state at vdpa device start to enable SVQ in data vqs. v1 from RFC: * Add TODO to use the resume operation in the future. * Use migration_in_setup and migration_has_failed instead of a complicated switch case. --- net/vhost-vdpa.c | 72 ++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 69 insertions(+), 3 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index b89c99066a..c5512ddf10 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -26,12 +26,15 @@ #include #include "standard-headers/linux/virtio_net.h" #include "monitor/monitor.h" +#include "migration/migration.h" +#include "migration/misc.h" #include "hw/virtio/vhost.h" =20 /* Todo:need to add the multiqueue support here */ typedef struct VhostVDPAState { NetClientState nc; struct vhost_vdpa vhost_vdpa; + Notifier migration_state; VHostNetState *vhost_net; =20 /* Control commands shadow buffers */ @@ -239,10 +242,59 @@ static VhostVDPAState *vhost_vdpa_net_first_nc_vdpa(V= hostVDPAState *s) return DO_UPCAST(VhostVDPAState, nc, nc0); } =20 +static void vhost_vdpa_net_log_global_enable(VhostVDPAState *s, bool enabl= e) +{ + struct vhost_vdpa *v =3D &s->vhost_vdpa; + VirtIONet *n; + VirtIODevice *vdev; + int data_queue_pairs, cvq, r; + + /* We are only called on the first data vqs and only if x-svq is not s= et */ + if (s->vhost_vdpa.shadow_vqs_enabled =3D=3D enable) { + return; + } + + vdev =3D v->dev->vdev; + n =3D VIRTIO_NET(vdev); + if (!n->vhost_started) { + return; + } + + data_queue_pairs =3D n->multiqueue ? n->max_queue_pairs : 1; + cvq =3D virtio_vdev_has_feature(vdev, VIRTIO_NET_F_CTRL_VQ) ? + n->max_ncs - n->max_queue_pairs : 0; + /* + * TODO: vhost_net_stop does suspend, get_base and reset. We can be sm= arter + * in the future and resume the device if read-only operations between + * suspend and reset goes wrong. + */ + vhost_net_stop(vdev, n->nic->ncs, data_queue_pairs, cvq); + + /* Start will check migration setup_or_active to configure or not SVQ = */ + r =3D vhost_net_start(vdev, n->nic->ncs, data_queue_pairs, cvq); + if (unlikely(r < 0)) { + error_report("unable to start vhost net: %s(%d)", g_strerror(-r), = -r); + } +} + +static void vdpa_net_migration_state_notifier(Notifier *notifier, void *da= ta) +{ + MigrationState *migration =3D data; + VhostVDPAState *s =3D container_of(notifier, VhostVDPAState, + migration_state); + + if (migration_in_setup(migration)) { + vhost_vdpa_net_log_global_enable(s, true); + } else if (migration_has_failed(migration)) { + vhost_vdpa_net_log_global_enable(s, false); + } +} + static void vhost_vdpa_net_data_start_first(VhostVDPAState *s) { struct vhost_vdpa *v =3D &s->vhost_vdpa; =20 + add_migration_state_change_notifier(&s->migration_state); if (v->shadow_vqs_enabled) { v->iova_tree =3D vhost_iova_tree_new(v->iova_range.first, v->iova_range.last); @@ -256,6 +308,15 @@ static int vhost_vdpa_net_data_start(NetClientState *n= c) =20 assert(nc->info->type =3D=3D NET_CLIENT_DRIVER_VHOST_VDPA); =20 + if (s->always_svq || + migration_is_setup_or_active(migrate_get_current()->state)) { + v->shadow_vqs_enabled =3D true; + v->shadow_data =3D true; + } else { + v->shadow_vqs_enabled =3D false; + v->shadow_data =3D false; + } + if (v->index =3D=3D 0) { vhost_vdpa_net_data_start_first(s); return 0; @@ -276,6 +337,10 @@ static void vhost_vdpa_net_client_stop(NetClientState = *nc) =20 assert(nc->info->type =3D=3D NET_CLIENT_DRIVER_VHOST_VDPA); =20 + if (s->vhost_vdpa.index =3D=3D 0) { + remove_migration_state_change_notifier(&s->migration_state); + } + dev =3D s->vhost_vdpa.dev; if (dev->vq_index + dev->nvqs =3D=3D dev->vq_index_end) { g_clear_pointer(&s->vhost_vdpa.iova_tree, vhost_iova_tree_delete); @@ -412,11 +477,12 @@ static int vhost_vdpa_net_cvq_start(NetClientState *n= c) s =3D DO_UPCAST(VhostVDPAState, nc, nc); v =3D &s->vhost_vdpa; =20 - v->shadow_data =3D s->always_svq; + s0 =3D vhost_vdpa_net_first_nc_vdpa(s); + v->shadow_data =3D s0->vhost_vdpa.shadow_vqs_enabled; v->shadow_vqs_enabled =3D s->always_svq; s->vhost_vdpa.address_space_id =3D VHOST_VDPA_GUEST_PA_ASID; =20 - if (s->always_svq) { + if (s->vhost_vdpa.shadow_data) { /* SVQ is already configured for all virtqueues */ goto out; } @@ -473,7 +539,6 @@ out: return 0; } =20 - s0 =3D vhost_vdpa_net_first_nc_vdpa(s); if (s0->vhost_vdpa.iova_tree) { /* * SVQ is already configured for all virtqueues. Reuse IOVA tree = for @@ -749,6 +814,7 @@ static NetClientState *net_vhost_vdpa_init(NetClientSta= te *peer, s->vhost_vdpa.device_fd =3D vdpa_device_fd; s->vhost_vdpa.index =3D queue_pair_index; s->always_svq =3D svq; + s->migration_state.notify =3D vdpa_net_migration_state_notifier; s->vhost_vdpa.shadow_vqs_enabled =3D svq; s->vhost_vdpa.iova_range =3D iova_range; s->vhost_vdpa.shadow_data =3D svq; --=20 2.31.1 From nobody Tue May 14 19:56:46 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1677256492; cv=none; d=zohomail.com; s=zohoarc; b=lSoCkED4kv/1xOFtmf2NMQucVTMvx/aXLp1ZHULwBf+jzELWZIaO81sWpfxYT/8BXe8geumeOo0YgtktRRyBuKaJnsOm3aRqAMTW3V4SIvLecchzGrW0rnM3TNgXSHsikOnwFCxRw6ysWU95wyklP0XJIZp3xFFESu4B5HTPetY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677256492; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=X1C6L9nEFkeUTbPQ7OqGGoSRuulAeUc0p+h0r2E6tCY=; b=myF8yYajvu97IDpvN0ymAgHoKyAJM0GRWh2z4VF0YE3IHSbhfIlxMU0n5mGrjVw8r75hOpaaWRXFha2qADtHe36p0RpZ7aNBLeFQfyqB4Ji7c9XlXF4sV6LjR1muaM9ca7ra7HOmVz9KBUZQtDEiome4Gzy+vtqzNIuhOv2d65Y= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677256492664801.237230243041; Fri, 24 Feb 2023 08:34:52 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pVaQB-00064k-Ok; Fri, 24 Feb 2023 10:55:43 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaPs-0005dO-Dl for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:32 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaPo-0005tX-Sg for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:22 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-654-YMdPMAY9NkSfPHtu4aaVCA-1; Fri, 24 Feb 2023 10:55:17 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 8C05F3C02528; Fri, 24 Feb 2023 15:55:16 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.85]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9BE20C15BA0; Fri, 24 Feb 2023 15:55:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677254119; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=X1C6L9nEFkeUTbPQ7OqGGoSRuulAeUc0p+h0r2E6tCY=; b=LcOMOSG77V02333exn/16BQg5EQvigI0rbHuhYrwO0p7vX1HJU6HCswhTOuBocfznb+/vG /JleixnB1d/6zaZD6jNcBJq5wKuaKVOP/nPsBLC3dyS0SvaWCNTsNAE7OKnPgDitohCtNJ 5bucLFc3FQycrt2x9lsYIm2jjZUONTY= X-MC-Unique: YMdPMAY9NkSfPHtu4aaVCA-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Stefano Garzarella , Shannon Nelson , Jason Wang , Gautam Dawar , Laurent Vivier , alvaro.karsz@solid-run.com, longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefan Hajnoczi , Cindy Lu , "Michael S. Tsirkin" , si-wei.liu@oracle.com, Liuxiangdong , Parav Pandit , Eli Cohen , Zhu Lingshan , Harpreet Singh Anand , "Gonglei (Arei)" , Lei Yang Subject: [PATCH v4 10/15] vdpa: disable RAM block discard only for the first device Date: Fri, 24 Feb 2023 16:54:33 +0100 Message-Id: <20230224155438.112797-11-eperezma@redhat.com> In-Reply-To: <20230224155438.112797-1-eperezma@redhat.com> References: <20230224155438.112797-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1677256494444100003 Although it does not make a big difference, its more correct and simplifies the cleanup path in subsequent patches. Move ram_block_discard_disable(false) call to the top of vhost_vdpa_cleanup because: * We cannot use vhost_vdpa_first_dev after dev->opaque =3D NULL assignment. * Improve the stack order in cleanup: since it is the last action taken in init, it should be the first at cleanup. Signed-off-by: Eugenio P=C3=A9rez Tested-by: Alvaro Karsz --- hw/virtio/vhost-vdpa.c | 25 ++++++++++++++----------- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 71e3dc21fe..27655e7582 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -431,16 +431,6 @@ static int vhost_vdpa_init(struct vhost_dev *dev, void= *opaque, Error **errp) trace_vhost_vdpa_init(dev, opaque); int ret; =20 - /* - * Similar to VFIO, we end up pinning all guest memory and have to - * disable discarding of RAM. - */ - ret =3D ram_block_discard_disable(true); - if (ret) { - error_report("Cannot set discarding of RAM broken"); - return ret; - } - v =3D opaque; v->dev =3D dev; dev->opaque =3D opaque ; @@ -452,6 +442,16 @@ static int vhost_vdpa_init(struct vhost_dev *dev, void= *opaque, Error **errp) return 0; } =20 + /* + * Similar to VFIO, we end up pinning all guest memory and have to + * disable discarding of RAM. + */ + ret =3D ram_block_discard_disable(true); + if (ret) { + error_report("Cannot set discarding of RAM broken"); + return ret; + } + vhost_vdpa_add_status(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE | VIRTIO_CONFIG_S_DRIVER); =20 @@ -577,12 +577,15 @@ static int vhost_vdpa_cleanup(struct vhost_dev *dev) assert(dev->vhost_ops->backend_type =3D=3D VHOST_BACKEND_TYPE_VDPA); v =3D dev->opaque; trace_vhost_vdpa_cleanup(dev, v); + if (vhost_vdpa_first_dev(dev)) { + ram_block_discard_disable(false); + } + vhost_vdpa_host_notifiers_uninit(dev, dev->nvqs); memory_listener_unregister(&v->listener); vhost_vdpa_svq_cleanup(dev); =20 dev->opaque =3D NULL; - ram_block_discard_disable(false); =20 return 0; } --=20 2.31.1 From nobody Tue May 14 19:56:46 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1677256855; cv=none; d=zohomail.com; s=zohoarc; b=HbrhqZYho+luoUnbLKSAgGww35bSeeofk3DmlcGgNFlH8r8O1TN6PBvwrYBfZvCLYWfGtikOmnfLy3OlQH39O2sPiCUKhll5u0sUbjObOHXcmnulND6k7fBbwEwk7YpKxnqGrWp4bUHoqjz0sPvshkc8tZ3puUnxjJfaQm4PZnQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677256855; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=SFD1vTTGC6UTS0PaRXRY/vLSg/9vBPQI5vC68s4t4Ko=; b=FR6uZqoyUfLNG/1FdUFJsbmJRLtgq5/Vy53UFxmbY/eYuGSlMFlpbBtyCU1AGK3B/1RhvRvMK6nTekqQjJ7xJvhSpEIu6gxh7JSzdT4MpazKyZTiLXg4Gaar93SJpn4omNZ0FbRPciPocIagCMIFksuQ59VecfB8n3uCaNS2ROY= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677256855760782.4211532097241; Fri, 24 Feb 2023 08:40:55 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pVaQA-0005vc-8v; Fri, 24 Feb 2023 10:55:42 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaPv-0005gy-Bg for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:33 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaPt-0005uH-IG for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:27 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-330-DaTjsdxwMQG2qFcJXlXAxw-1; Fri, 24 Feb 2023 10:55:20 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id ABFA33C02528; Fri, 24 Feb 2023 15:55:19 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.85]) by smtp.corp.redhat.com (Postfix) with ESMTP id C8D6AC15BA0; Fri, 24 Feb 2023 15:55:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677254124; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SFD1vTTGC6UTS0PaRXRY/vLSg/9vBPQI5vC68s4t4Ko=; b=i/5Jjy9W9BJWaqRbdptiBsGBGJIi9KVS8O61N+uDNtufB7ztUu4BCFRzzVIaxZnYQIn5TX Fwt5FTPPg3ZaHDQ6VJfSTw4keQ+7cK4dyVAlyPi6Wd1Q4a0O2N9JULfPEAPqKnRuzJmpHg AHM8mROLsJRHH4syFXtLMbuuEBpJSiI= X-MC-Unique: DaTjsdxwMQG2qFcJXlXAxw-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Stefano Garzarella , Shannon Nelson , Jason Wang , Gautam Dawar , Laurent Vivier , alvaro.karsz@solid-run.com, longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefan Hajnoczi , Cindy Lu , "Michael S. Tsirkin" , si-wei.liu@oracle.com, Liuxiangdong , Parav Pandit , Eli Cohen , Zhu Lingshan , Harpreet Singh Anand , "Gonglei (Arei)" , Lei Yang Subject: [PATCH v4 11/15] vdpa net: block migration if the device has CVQ Date: Fri, 24 Feb 2023 16:54:34 +0100 Message-Id: <20230224155438.112797-12-eperezma@redhat.com> In-Reply-To: <20230224155438.112797-1-eperezma@redhat.com> References: <20230224155438.112797-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1677256856390100001 Devices with CVQ needs to migrate state beyond vq state. Leaving this to future series. Signed-off-by: Eugenio P=C3=A9rez Tested-by: Alvaro Karsz --- v3: Migration blocker is registered in vhost_dev. --- include/hw/virtio/vhost-vdpa.h | 1 + hw/virtio/vhost-vdpa.c | 1 + net/vhost-vdpa.c | 9 +++++++++ 3 files changed, 11 insertions(+) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index 4a7d396674..c278a2a8de 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -50,6 +50,7 @@ typedef struct vhost_vdpa { const VhostShadowVirtqueueOps *shadow_vq_ops; void *shadow_vq_ops_opaque; struct vhost_dev *dev; + Error *migration_blocker; VhostVDPAHostNotifier notifier[VIRTIO_QUEUE_MAX]; } VhostVDPA; =20 diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 27655e7582..25b64ae854 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -438,6 +438,7 @@ static int vhost_vdpa_init(struct vhost_dev *dev, void = *opaque, Error **errp) v->msg_type =3D VHOST_IOTLB_MSG_V2; vhost_vdpa_init_svq(dev, v); =20 + error_propagate(&dev->migration_blocker, v->migration_blocker); if (!vhost_vdpa_first_dev(dev)) { return 0; } diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index c5512ddf10..4f983df000 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -828,6 +828,15 @@ static NetClientState *net_vhost_vdpa_init(NetClientSt= ate *peer, =20 s->vhost_vdpa.shadow_vq_ops =3D &vhost_vdpa_net_svq_ops; s->vhost_vdpa.shadow_vq_ops_opaque =3D s; + + /* + * TODO: We cannot migrate devices with CVQ as there is no way to = set + * the device state (MAC, MQ, etc) before starting datapath. + * + * Migration blocker ownership now belongs to v + */ + error_setg(&s->vhost_vdpa.migration_blocker, + "net vdpa cannot migrate with CVQ feature"); } ret =3D vhost_vdpa_add(nc, (void *)&s->vhost_vdpa, queue_pair_index, n= vqs); if (ret) { --=20 2.31.1 From nobody Tue May 14 19:56:46 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1677257086; cv=none; d=zohomail.com; s=zohoarc; b=TdIl4XvkrXlTghsGzffZ5IIs4LKAYZ6SmSU0EbOUqUnBq6bsGSux4Q/9zcoVGbdZ3flg+dfLp6sN2LrefrIY31jLHoat59qGTs9z6S1QMGZV75i1ibXWMmgemKKlFuDt4fytjvAgTzvTQq36tD8q6RfcuxSWhuNNVJI7mWCfcyI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677257086; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=T4kHAJy6vW9/XOWabzAlBzZlC8VzGMVIaQSUJZtePAA=; b=QF/B29rAES5lQzt0Wi7c8OyYVW7uS8XKs/t//UgYV58bquHjsO5WekXZVZBDkp8zvUg30LHZW6x2u7dmPA94putupl182I84ck2UxTZ/RrzIhrb8VD/sYAET2FpdVCUIc9y0NeI+5TIqIOyvcuEyIV+ickupHdPR1cVbZg/oPxk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677257086890296.97684513008335; Fri, 24 Feb 2023 08:44:46 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pVaQE-0006Nk-97; Fri, 24 Feb 2023 10:55:46 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaPy-0005hC-M8 for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:33 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaPw-0005ua-Ju for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:29 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-557-YkRyiEKmO_CZOqHgyvT1sg-1; Fri, 24 Feb 2023 10:55:23 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id C673395D601; Fri, 24 Feb 2023 15:55:22 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.85]) by smtp.corp.redhat.com (Postfix) with ESMTP id E890BC15BA0; Fri, 24 Feb 2023 15:55:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677254127; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=T4kHAJy6vW9/XOWabzAlBzZlC8VzGMVIaQSUJZtePAA=; b=KHZpTNwUyKZW3oUASfE3o7UTUV9X7MfOqeV7u/L4cTxkpOX33Sc9bjCiFGnEsh6yQXthzp v9MH6vyO/XGXIh9cHX577xv+H+Utek4oo/XQ8lPm2Z0HuwM8fpOPc/83Whd7rnROmlG5oF R7nWojwWOKNqf+nFQa5gzQCIkg5bM3E= X-MC-Unique: YkRyiEKmO_CZOqHgyvT1sg-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Stefano Garzarella , Shannon Nelson , Jason Wang , Gautam Dawar , Laurent Vivier , alvaro.karsz@solid-run.com, longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefan Hajnoczi , Cindy Lu , "Michael S. Tsirkin" , si-wei.liu@oracle.com, Liuxiangdong , Parav Pandit , Eli Cohen , Zhu Lingshan , Harpreet Singh Anand , "Gonglei (Arei)" , Lei Yang Subject: [PATCH v4 12/15] vdpa: block migration if device has unsupported features Date: Fri, 24 Feb 2023 16:54:35 +0100 Message-Id: <20230224155438.112797-13-eperezma@redhat.com> In-Reply-To: <20230224155438.112797-1-eperezma@redhat.com> References: <20230224155438.112797-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1677257088791100003 A vdpa net device must initialize with SVQ in order to be migratable at this moment, and initialization code verifies some conditions. If the device is not initialized with the x-svq parameter, it will not expose _F_LOG so the vhost subsystem will block VM migration from its initialization. Next patches change this, so we need to verify migration conditions differently. QEMU only supports a subset of net features in SVQ, and it cannot migrate state that cannot track or restore in the destination. Add a migration blocker if the device offer an unsupported feature. Signed-off-by: Eugenio P=C3=A9rez Tested-by: Alvaro Karsz --- v3: add mirgation blocker properly so vhost_dev can handle it. --- net/vhost-vdpa.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 4f983df000..094dc1c2d0 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -795,7 +795,8 @@ static NetClientState *net_vhost_vdpa_init(NetClientSta= te *peer, int nvqs, bool is_datapath, bool svq, - struct vhost_vdpa_iova_range iova_r= ange) + struct vhost_vdpa_iova_range iova_r= ange, + uint64_t features) { NetClientState *nc =3D NULL; VhostVDPAState *s; @@ -818,7 +819,10 @@ static NetClientState *net_vhost_vdpa_init(NetClientSt= ate *peer, s->vhost_vdpa.shadow_vqs_enabled =3D svq; s->vhost_vdpa.iova_range =3D iova_range; s->vhost_vdpa.shadow_data =3D svq; - if (!is_datapath) { + if (queue_pair_index =3D=3D 0) { + vhost_vdpa_net_valid_svq_features(features, + &s->vhost_vdpa.migration_blocker= ); + } else if (!is_datapath) { s->cvq_cmd_out_buffer =3D qemu_memalign(qemu_real_host_page_size(), vhost_vdpa_net_cvq_cmd_page_le= n()); memset(s->cvq_cmd_out_buffer, 0, vhost_vdpa_net_cvq_cmd_page_len()= ); @@ -956,7 +960,7 @@ int net_init_vhost_vdpa(const Netdev *netdev, const cha= r *name, for (i =3D 0; i < queue_pairs; i++) { ncs[i] =3D net_vhost_vdpa_init(peer, TYPE_VHOST_VDPA, name, vdpa_device_fd, i, 2, true, opts->x_s= vq, - iova_range); + iova_range, features); if (!ncs[i]) goto err; } @@ -964,7 +968,7 @@ int net_init_vhost_vdpa(const Netdev *netdev, const cha= r *name, if (has_cvq) { nc =3D net_vhost_vdpa_init(peer, TYPE_VHOST_VDPA, name, vdpa_device_fd, i, 1, false, - opts->x_svq, iova_range); + opts->x_svq, iova_range, features); if (!nc) goto err; } --=20 2.31.1 From nobody Tue May 14 19:56:46 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1677255104; cv=none; d=zohomail.com; s=zohoarc; b=Frvcy4nv6khugK5Pumphk2gInLb7GbCOQhPgwj4Ermdh5kBfjPZmbcwxs2QOEKa4BcCgQEe/f0KdRyoDp10AEJFkio4j3Np1T4nVl4PsSpr+5D/rVPaP31FZ0fpU0tFm8ir0uxe2zwYuit4qOBQUVmcmrhikuKD58mwW4eVTvFI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677255104; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=ojO/eaemwdQdQXtGu0Anh3zWgg58HTq1RDYErsUN+MI=; b=OrCXrfXvTbsBZICY4NlLqeA9OIBiBzgfvjBj+gfh9qWKNZFKGYvd1oJKSSrvph+z4oaFbQTel2Kym0MxaCMpbewRn40GSj8BufgLw3dT3rlG6aKapVVQ9KlqWOmFCaNPPQZvrp8ckzhrIX+c4hCwEbs+wiW+q37xWWUgISNExQQ= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677255104851348.3026401268893; Fri, 24 Feb 2023 08:11:44 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pVaQG-0006VP-L9; Fri, 24 Feb 2023 10:55:48 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaQ1-0005ha-W9 for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:38 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaQ0-0005uo-Ca for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:33 -0500 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-588-qC_ettOoNcWvcfRIw85LLw-1; Fri, 24 Feb 2023 10:55:26 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id E22DC95D603; Fri, 24 Feb 2023 15:55:25 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.85]) by smtp.corp.redhat.com (Postfix) with ESMTP id 0F135C15BA0; Fri, 24 Feb 2023 15:55:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677254130; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ojO/eaemwdQdQXtGu0Anh3zWgg58HTq1RDYErsUN+MI=; b=Lr78fQDJkbNKX2IaDX6rROUcxXNZaNmYDHTE9XOxr+R69mOfnr3ysGi8ZcxVaZezK6Y/3V a0GFC4CqVI26fL41aM4XAAkuJD6oVMVqzXR//hJEQtfcHBvhpdVUwnOkaQMVJoroT2KyEV XFLqLCJnLTf5+pnbRTEg/cqsaePHKDs= X-MC-Unique: qC_ettOoNcWvcfRIw85LLw-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Stefano Garzarella , Shannon Nelson , Jason Wang , Gautam Dawar , Laurent Vivier , alvaro.karsz@solid-run.com, longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefan Hajnoczi , Cindy Lu , "Michael S. Tsirkin" , si-wei.liu@oracle.com, Liuxiangdong , Parav Pandit , Eli Cohen , Zhu Lingshan , Harpreet Singh Anand , "Gonglei (Arei)" , Lei Yang Subject: [PATCH v4 13/15] vdpa: block migration if SVQ does not admit a feature Date: Fri, 24 Feb 2023 16:54:36 +0100 Message-Id: <20230224155438.112797-14-eperezma@redhat.com> In-Reply-To: <20230224155438.112797-1-eperezma@redhat.com> References: <20230224155438.112797-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1677255106983100003 Next patches enable devices to be migrated even if vdpa netdev has not been started with x-svq. However, not all devices are migratable, so we need to block migration if we detect that. Block migration if we detect the device expose a feature SVQ does not know how to work with. Signed-off-by: Eugenio P=C3=A9rez Tested-by: Alvaro Karsz --- hw/virtio/vhost-vdpa.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 25b64ae854..8702780ad6 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -443,6 +443,21 @@ static int vhost_vdpa_init(struct vhost_dev *dev, void= *opaque, Error **errp) return 0; } =20 + /* + * If dev->shadow_vqs_enabled at initialization that means the device = has + * been started with x-svq=3Don, so don't block migration + */ + if (dev->migration_blocker =3D=3D NULL && !v->shadow_vqs_enabled) { + /* We don't have dev->features yet */ + uint64_t features; + ret =3D vhost_vdpa_get_dev_features(dev, &features); + if (unlikely(ret)) { + error_setg_errno(errp, -ret, "Could not get device features"); + return ret; + } + vhost_svq_valid_features(features, &dev->migration_blocker); + } + /* * Similar to VFIO, we end up pinning all guest memory and have to * disable discarding of RAM. --=20 2.31.1 From nobody Tue May 14 19:56:46 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1677256163; cv=none; d=zohomail.com; s=zohoarc; b=G5ZSxyOV1PPPJlSQDHCUtTRRt3s+mT3UrTaljU59glYocHLWlCMJqJm+KKndQUgx95omy5nRE1r/Of6Q+H8v1zyWoTCHIVr4mSImxE3fU1zi6+oihXccKtu618nK53OTqrXhMM/AGH9i3fmwqz5JnMs2tbT88AV7aYM9Ayg2slk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677256163; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=gY3WmHvLrnlgMMrQ7XZGGlUeCPnkDMedRKO6qTIuFJE=; b=Srg7Nt5T7fTYWy0jvgCaAnh0MARsrlEeeM8/ozbyt3Bkgx4UWIO4GQt2jHj1nNacpEsXCz2d2ue5tW/xWEOFlFYs9vjN6vcb03Z/SZzvwQn9IwoJUAMZHbEmItKO4ggtWxWA+zTlSgtfaFYbScriqquSbeK3L4b+6DoPVcvhBhE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677256163301885.0388567805738; Fri, 24 Feb 2023 08:29:23 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pVaQD-0006Hg-9J; Fri, 24 Feb 2023 10:55:45 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaQ7-0005oJ-Qr for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:40 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaQ5-0005vD-8R for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:39 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-185-AGTBGzDqOCqSo8opBD2sxg-1; Fri, 24 Feb 2023 10:55:29 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 07B30299E75E; Fri, 24 Feb 2023 15:55:29 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.85]) by smtp.corp.redhat.com (Postfix) with ESMTP id 2AB19C15BA0; Fri, 24 Feb 2023 15:55:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677254134; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=gY3WmHvLrnlgMMrQ7XZGGlUeCPnkDMedRKO6qTIuFJE=; b=BxOajgBJlsC4XrY977cvoy+LUFjaZ/ETTTh+bjGbLuzqFMKcMw8/vVgzAk7KLRF35J8x7/ 33Maon3vOXBkYqRsc0VIRvUAH16zF0x/Wk+us3WZvvVAmlb0Ksv/RNcC0NQyEOu1Q+myIb zXDF88Vn4138BQwLzlFqqFks6TG7Fyg= X-MC-Unique: AGTBGzDqOCqSo8opBD2sxg-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Stefano Garzarella , Shannon Nelson , Jason Wang , Gautam Dawar , Laurent Vivier , alvaro.karsz@solid-run.com, longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefan Hajnoczi , Cindy Lu , "Michael S. Tsirkin" , si-wei.liu@oracle.com, Liuxiangdong , Parav Pandit , Eli Cohen , Zhu Lingshan , Harpreet Singh Anand , "Gonglei (Arei)" , Lei Yang Subject: [PATCH v4 14/15] vdpa net: allow VHOST_F_LOG_ALL Date: Fri, 24 Feb 2023 16:54:37 +0100 Message-Id: <20230224155438.112797-15-eperezma@redhat.com> In-Reply-To: <20230224155438.112797-1-eperezma@redhat.com> References: <20230224155438.112797-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1677256164511100001 Since some actions move to the start function instead of init, the device features may not be the parent vdpa device's, but the one returned by vhost backend. If transition to SVQ is supported, the vhost backend will return _F_LOG_ALL to signal the device is migratable. Add VHOST_F_LOG_ALL. HW dirty page tracking can be added on top of this change if the device supports it in the future. Signed-off-by: Eugenio P=C3=A9rez Acked-by: Jason Wang Tested-by: Alvaro Karsz --- net/vhost-vdpa.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 094dc1c2d0..f55bb31400 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -101,6 +101,8 @@ static const uint64_t vdpa_svq_device_features =3D BIT_ULL(VIRTIO_NET_F_MQ) | BIT_ULL(VIRTIO_F_ANY_LAYOUT) | BIT_ULL(VIRTIO_NET_F_CTRL_MAC_ADDR) | + /* VHOST_F_LOG_ALL is exposed by SVQ */ + BIT_ULL(VHOST_F_LOG_ALL) | BIT_ULL(VIRTIO_NET_F_RSC_EXT) | BIT_ULL(VIRTIO_NET_F_STANDBY); =20 --=20 2.31.1 From nobody Tue May 14 19:56:46 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1677256783; cv=none; d=zohomail.com; s=zohoarc; b=eTMWJeMTrQAxkaIYFSMVYUVK5Nk3rMtaMFfycCnYfd1PnWviqcPHaj11296tXEfbW9dB3Y0MN5LVrsz6a/n2fns8URwibSVabM1/aLUHAo4uTm9s8DTwBfcegyuPyqUMF+QtxRH9d2Zc9lAufI3Z3V3Y0aMCVloIRR/ffSuajfA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1677256783; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=pS6Z/sVXz2QPRzFhXYU2YnsrDLbW6mC8TdAMsQ9cUGY=; b=k/95oZfU7BlTVCqTBAC8EMzu8jFgz1p+lnrlXv3TNptRNA7rZv7YiRQLvYNytUR1MIBSmyglW4qVsV0OLq7sos76pAgwIme17+ZFe8MIPXct0NSi1VTsDM+7Drqemn9L8pH3sf+NsHJLTmRLEqvNGPWftQQrsYolvyP1E20oabk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1677256783782908.706149795655; Fri, 24 Feb 2023 08:39:43 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1pVaQH-0006Zy-Gt; Fri, 24 Feb 2023 10:55:49 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaQB-00067B-FC for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:43 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1pVaQ9-0005vr-Fu for qemu-devel@nongnu.org; Fri, 24 Feb 2023 10:55:42 -0500 Received: from mimecast-mx02.redhat.com (mx3-rdu2.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-433-CaW9JJfxP0WN4eqga-830w-1; Fri, 24 Feb 2023 10:55:32 -0500 Received: from smtp.corp.redhat.com (int-mx08.intmail.prod.int.rdu2.redhat.com [10.11.54.8]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 218501C05B02; Fri, 24 Feb 2023 15:55:32 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.192.85]) by smtp.corp.redhat.com (Postfix) with ESMTP id 43F0CC15BA0; Fri, 24 Feb 2023 15:55:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1677254140; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=pS6Z/sVXz2QPRzFhXYU2YnsrDLbW6mC8TdAMsQ9cUGY=; b=ZlLxEL+fTEEVix1mwaCB2bS9lSSzrJlmDM4tMncrbEJErUds9i+b2TI72vURLN1esqJ6Wb aSC0iSlgO5RGHeR8j4TfiunU7tzIIF4W0TsJGj7yJSYIuW6uqFUk8k611dRVJr9TK481AK opnb9UYcsu1XhQdt5IVk+uIoMcpi2xI= X-MC-Unique: CaW9JJfxP0WN4eqga-830w-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Stefano Garzarella , Shannon Nelson , Jason Wang , Gautam Dawar , Laurent Vivier , alvaro.karsz@solid-run.com, longpeng2@huawei.com, virtualization@lists.linux-foundation.org, Stefan Hajnoczi , Cindy Lu , "Michael S. Tsirkin" , si-wei.liu@oracle.com, Liuxiangdong , Parav Pandit , Eli Cohen , Zhu Lingshan , Harpreet Singh Anand , "Gonglei (Arei)" , Lei Yang Subject: [PATCH v4 15/15] vdpa: return VHOST_F_LOG_ALL in vhost-vdpa devices Date: Fri, 24 Feb 2023 16:54:38 +0100 Message-Id: <20230224155438.112797-16-eperezma@redhat.com> In-Reply-To: <20230224155438.112797-1-eperezma@redhat.com> References: <20230224155438.112797-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.8 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1677256786064100003 vhost-vdpa devices can return this features now that blockers have been set in case some features are not met. Expose VHOST_F_LOG_ALL only in that case. Signed-off-by: Eugenio P=C3=A9rez Acked-by: Jason Wang Tested-by: Alvaro Karsz --- hw/virtio/vhost-vdpa.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 8702780ad6..2a66cb51fc 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -1307,10 +1307,9 @@ static int vhost_vdpa_set_vring_call(struct vhost_de= v *dev, static int vhost_vdpa_get_features(struct vhost_dev *dev, uint64_t *features) { - struct vhost_vdpa *v =3D dev->opaque; int ret =3D vhost_vdpa_get_dev_features(dev, features); =20 - if (ret =3D=3D 0 && v->shadow_vqs_enabled) { + if (ret =3D=3D 0) { /* Add SVQ logging capabilities */ *features |=3D BIT_ULL(VHOST_F_LOG_ALL); } --=20 2.31.1