From nobody Sat May 18 03:46:11 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1690565012; cv=none; d=zohomail.com; s=zohoarc; b=T3cvFybMUJriQ8iqvdNZ+otxMFPS0FAhzF8JzFmXWfhNxY+CAI0n4wOy3dJ0I4yHyd2UHVjkR1l7JQkYNmq5GTEVXayo002euT0RQGs8y/JC9/xsss1grARpxj1oL0O+nM/jpcKc9uB6LDdM0s3tOxhwBEHl58oXytpO0OB71gk= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1690565012; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=0AuE379vW/co2hmkYkUf0b8G8fUUpmfbw1dIONIw/9Q=; b=NUFWxr3U8btYTAY8446JKviOGD9ty2U87gWIfk5IB43TxwRh1F35iIgZN336rlRwoaaf3WwdZyp340i9l2U3LsunZG2N4VQVqD8oFsrHZCNRWf5dT09MINtZhffPtsIwjetAQXgdaxvVHxy/B1YU/s+yVAo17df+mpbK8PtF+qk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1690565011998204.7342334679921; Fri, 28 Jul 2023 10:23:31 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qPR8x-0001Oo-3B; Fri, 28 Jul 2023 13:20:47 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qPR8s-0001K6-V7 for qemu-devel@nongnu.org; Fri, 28 Jul 2023 13:20:43 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qPR8r-0008Mb-Bj for qemu-devel@nongnu.org; Fri, 28 Jul 2023 13:20:42 -0400 Received: from mimecast-mx02.redhat.com (66.187.233.73 [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-335-3mufYBFUPlW_-FfjBB4SQw-1; Fri, 28 Jul 2023 13:20:36 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 87B941C0513E; Fri, 28 Jul 2023 17:20:35 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.99]) by smtp.corp.redhat.com (Postfix) with ESMTP id 50C01111F3B0; Fri, 28 Jul 2023 17:20:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690564840; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0AuE379vW/co2hmkYkUf0b8G8fUUpmfbw1dIONIw/9Q=; b=VMiA+eHgMVfGS6XdNapohF9Euc7zk8q3GWh8NC4+QAa1mR3thXrwrERn1pYVqVpU4lJfyG zCtX4LvsjQ7yy+BL/zVihIYKdKLgy+pWa63BD40h/iqgWtcQW63bcS075pEsjHbtcyugZu COTOo0cRGR77dMTgbb6SdbTFSiI8W6A= X-MC-Unique: 3mufYBFUPlW_-FfjBB4SQw-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Jason Wang , Gautam Dawar , si-wei.liu@oracle.com, Zhu Lingshan , Stefano Garzarella , Parav Pandit , Cindy Lu , "Michael S. Tsirkin" , Harpreet Singh Anand , Laurent Vivier , Shannon Nelson , Lei Yang , Dragos Tatulea Subject: [PATCH 1/7] vdpa: export vhost_vdpa_set_vring_ready Date: Fri, 28 Jul 2023 19:20:22 +0200 Message-Id: <20230728172028.2074052-2-eperezma@redhat.com> In-Reply-To: <20230728172028.2074052-1-eperezma@redhat.com> References: <20230728172028.2074052-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1690565014468100005 The vhost-vdpa net backend needs to enable vrings in a different order than default, so export it. No functional change intended except for tracing, that now includes the (virtio) index being enabled and the return value of the ioctl. Still ignoring return value of this function if called from vhost_vdpa_dev_start, as reorganize calling code around it is out of the scope of this series. Signed-off-by: Eugenio P=C3=A9rez --- include/hw/virtio/vhost-vdpa.h | 1 + hw/virtio/vhost-vdpa.c | 28 ++++++++++++++++++++-------- hw/virtio/trace-events | 2 +- 3 files changed, 22 insertions(+), 9 deletions(-) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index e64bfc7f98..5407d54fd7 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -57,6 +57,7 @@ typedef struct vhost_vdpa { } VhostVDPA; =20 int vhost_vdpa_get_iova_range(int fd, struct vhost_vdpa_iova_range *iova_r= ange); +int vhost_vdpa_set_vring_ready(struct vhost_vdpa *v, unsigned idx); =20 int vhost_vdpa_dma_map(struct vhost_vdpa *v, uint32_t asid, hwaddr iova, hwaddr size, void *vaddr, bool readonly); diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 42f2a4bae9..bebcc9fe7c 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -532,6 +532,19 @@ int vhost_vdpa_get_iova_range(int fd, struct vhost_vdp= a_iova_range *iova_range) return ret < 0 ? -errno : 0; } =20 +int vhost_vdpa_set_vring_ready(struct vhost_vdpa *v, unsigned idx) +{ + struct vhost_dev *dev =3D v->dev; + struct vhost_vring_state state =3D { + .index =3D idx, + .num =3D 1, + }; + int r =3D vhost_vdpa_call(dev, VHOST_VDPA_SET_VRING_ENABLE, &state); + + trace_vhost_vdpa_set_vring_ready(dev, idx, r); + return r; +} + /* * The use of this function is for requests that only need to be * applied once. Typically such request occurs at the beginning @@ -876,16 +889,15 @@ static int vhost_vdpa_get_vq_index(struct vhost_dev *= dev, int idx) return idx; } =20 -static int vhost_vdpa_set_vring_ready(struct vhost_dev *dev) +static int vhost_vdpa_set_vrings_ready(struct vhost_dev *dev) { + struct vhost_vdpa *v =3D dev->opaque; int i; - trace_vhost_vdpa_set_vring_ready(dev); + + assert(dev->vhost_ops->backend_type =3D=3D VHOST_BACKEND_TYPE_VDPA); + for (i =3D 0; i < dev->nvqs; ++i) { - struct vhost_vring_state state =3D { - .index =3D dev->vq_index + i, - .num =3D 1, - }; - vhost_vdpa_call(dev, VHOST_VDPA_SET_VRING_ENABLE, &state); + vhost_vdpa_set_vring_ready(v, dev->vq_index + i); } return 0; } @@ -1298,7 +1310,7 @@ static int vhost_vdpa_dev_start(struct vhost_dev *dev= , bool started) if (unlikely(!ok)) { return -1; } - vhost_vdpa_set_vring_ready(dev); + vhost_vdpa_set_vrings_ready(dev); } else { vhost_vdpa_suspend(dev); vhost_vdpa_svqs_stop(dev); diff --git a/hw/virtio/trace-events b/hw/virtio/trace-events index 7109cf1a3b..1cb9027d1e 100644 --- a/hw/virtio/trace-events +++ b/hw/virtio/trace-events @@ -48,7 +48,7 @@ vhost_vdpa_set_features(void *dev, uint64_t features) "de= v: %p features: 0x%"PRI vhost_vdpa_get_device_id(void *dev, uint32_t device_id) "dev: %p device_id= %"PRIu32 vhost_vdpa_reset_device(void *dev) "dev: %p" vhost_vdpa_get_vq_index(void *dev, int idx, int vq_idx) "dev: %p idx: %d v= q idx: %d" -vhost_vdpa_set_vring_ready(void *dev) "dev: %p" +vhost_vdpa_set_vring_ready(void *dev, unsigned i, int r) "dev: %p, idx: %u= , r: %d" vhost_vdpa_dump_config(void *dev, const char *line) "dev: %p %s" vhost_vdpa_set_config(void *dev, uint32_t offset, uint32_t size, uint32_t = flags) "dev: %p offset: %"PRIu32" size: %"PRIu32" flags: 0x%"PRIx32 vhost_vdpa_get_config(void *dev, void *config, uint32_t config_len) "dev: = %p config: %p config_len: %"PRIu32 --=20 2.39.3 From nobody Sat May 18 03:46:11 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1690566286; cv=none; d=zohomail.com; s=zohoarc; b=Du3aq3N7wEBzxZ9rah0rxoOE1asond++8ts9AZwPXrePQMNLh1bzbPHmtVFaZZmNDAN7yttmujFQ80cQdgBK4jYCeEliBjrlocZTRwODUPZMuwkZB09/NQC7x4B5uzNfFD5G4k0F3dQhfSepBKVylUHpnbB2ncPnH+fNZkEXBgo= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1690566286; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=s1mv7uzcPXcJ81XaXK92RJXYFUYLlxlqghXDFZnxUMI=; b=KfuC6vDJmDk94o4Ep8UTKsFYeYUP/mKyAnXELqCK01CT6/LyJpVcKpp666fX1llZyqE/OztFY0nIuS5s6ex19GCnjFe7f6pDLP3gWaHAmCLTOCgQ8R2BPqaxMweIxYtWTxYGzhffWPzKdI0ay6BxxqD+bENeckajdxVL4X8v1d4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1690566286227721.2168688461087; Fri, 28 Jul 2023 10:44:46 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qPR8w-0001Np-FC; Fri, 28 Jul 2023 13:20:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qPR8u-0001N9-BX for qemu-devel@nongnu.org; Fri, 28 Jul 2023 13:20:44 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qPR8t-0008NU-0t for qemu-devel@nongnu.org; Fri, 28 Jul 2023 13:20:44 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-552-Q4i4enrrM5OeHy9A3OZL2A-1; Fri, 28 Jul 2023 13:20:38 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 0EA3F858EED; Fri, 28 Jul 2023 17:20:38 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.99]) by smtp.corp.redhat.com (Postfix) with ESMTP id C97DC1121330; Fri, 28 Jul 2023 17:20:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690564842; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=s1mv7uzcPXcJ81XaXK92RJXYFUYLlxlqghXDFZnxUMI=; b=T2hQhqckcpgHr3NWyKQsoAjXPKEOizNpivKdZRpXGAHNljmj3f32ByCrf0KG0KrSeG95ZU zrACnM/0lZshqF0Hls3De0uRxmu5hRlAHfqz28F75Xo84tkqr3F9ailL3FNgZI6w7TK8b6 C/o57CEAb2Z14ffvu/ghdUQLA3yAPwY= X-MC-Unique: Q4i4enrrM5OeHy9A3OZL2A-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Jason Wang , Gautam Dawar , si-wei.liu@oracle.com, Zhu Lingshan , Stefano Garzarella , Parav Pandit , Cindy Lu , "Michael S. Tsirkin" , Harpreet Singh Anand , Laurent Vivier , Shannon Nelson , Lei Yang , Dragos Tatulea Subject: [PATCH 2/7] vdpa: add should_enable op Date: Fri, 28 Jul 2023 19:20:23 +0200 Message-Id: <20230728172028.2074052-3-eperezma@redhat.com> In-Reply-To: <20230728172028.2074052-1-eperezma@redhat.com> References: <20230728172028.2074052-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1690566287515100001 To restore the device at the destination of a live migration we send the commands through control virtqueue. For a device to read CVQ it must have received the DRIVER_OK status bit. However this opens a window where the device could start receiving packets in rx queue 0 before it receives the RSS configuration. To avoid that, we will not send vring_enable until all configuration is used by the device. As a first step, enable a new vitio ops per vhost_vdpa device to know if we should enable a virtqueue or not. This srtuct can be reused in the future to add more actions to vhost_vdpa that depend on the virtIO kind of device. Signed-off-by: Eugenio P=C3=A9rez --- include/hw/virtio/vhost-vdpa.h | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index 5407d54fd7..3d330d439a 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -30,6 +30,13 @@ typedef struct VhostVDPAHostNotifier { void *addr; } VhostVDPAHostNotifier; =20 +struct vhost_vdpa; +typedef bool (*vhost_vdpa_virtio_should_enable_op)(const struct vhost_vdpa= *v); + +typedef struct VhostVDPAVirtIOOps { + vhost_vdpa_virtio_should_enable_op should_enable; +} VhostVDPAVirtIOOps; + typedef struct vhost_vdpa { int device_fd; int index; @@ -48,6 +55,7 @@ typedef struct vhost_vdpa { VhostIOVATree *iova_tree; GPtrArray *shadow_vqs; const VhostShadowVirtqueueOps *shadow_vq_ops; + const VhostVDPAVirtIOOps *virtio_ops; void *shadow_vq_ops_opaque; struct vhost_dev *dev; Error *migration_blocker; --=20 2.39.3 From nobody Sat May 18 03:46:11 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1690569571; cv=none; d=zohomail.com; s=zohoarc; b=oGpZkvBGg6YFybnVVatYlkT68lBpv4rFG4VA1WvG67BZ9oK5+knqUHf8fJsfarP8+ai/1VSIqJj22QulKbKGoYQO6srJK76ZyJE46NHllyzcJ0EjytRKlt5tVt4e87bNMofQpxxMtwBqiOoUCcVVWauVs5GBkCYjFEbdNPUVkKQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1690569571; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=fJnf01bbsUf53Izrt6GXdJzd9ID7pODZMAOQ02cP6nc=; b=dCVeEwAxRi/iuEbwL+rmgb8k0Kil/bFoNV3iifrCP7PDsVknVqVfhAwcOTdkmrnYDznCrQJMCatWzX1BULh/EE1gUs9oAmaYIyejulLfExiely3EC3BoTaoWSUkMhMD1zDbvt7tMUmCjRhcnMtGztamx3svnAVNpsZoXWpXxwHE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1690569571786642.8401687563168; Fri, 28 Jul 2023 11:39:31 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qPR8z-0001X9-LS; Fri, 28 Jul 2023 13:20:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qPR8x-0001PH-Tj for qemu-devel@nongnu.org; Fri, 28 Jul 2023 13:20:47 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qPR8w-0008OU-KI for qemu-devel@nongnu.org; Fri, 28 Jul 2023 13:20:47 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-487-qhkVz3lDO8ailwr81Z_qRQ-1; Fri, 28 Jul 2023 13:20:42 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 355E6805C3F; Fri, 28 Jul 2023 17:20:42 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.99]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4EBB11121330; Fri, 28 Jul 2023 17:20:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690564846; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=fJnf01bbsUf53Izrt6GXdJzd9ID7pODZMAOQ02cP6nc=; b=RgangJO+ouG9UUZpsws2SaADqS3V2UrAPOLgGgB6ACylku+pB0qJCU6oWslbQB2CLSZdqh TUIThYWqe6LfKE2oZzOg71ju1JLG4BrCZnyXawPXsMYun7yf7Z7WbkMjyVy7uva9PeJRF/ 8cV74g0e5BxHL9tit1tkuolX7B23nE0= X-MC-Unique: qhkVz3lDO8ailwr81Z_qRQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Jason Wang , Gautam Dawar , si-wei.liu@oracle.com, Zhu Lingshan , Stefano Garzarella , Parav Pandit , Cindy Lu , "Michael S. Tsirkin" , Harpreet Singh Anand , Laurent Vivier , Shannon Nelson , Lei Yang , Dragos Tatulea Subject: [PATCH 3/7] vdpa: use virtio_ops->should_enable at vhost_vdpa_set_vrings_ready Date: Fri, 28 Jul 2023 19:20:24 +0200 Message-Id: <20230728172028.2074052-4-eperezma@redhat.com> In-Reply-To: <20230728172028.2074052-1-eperezma@redhat.com> References: <20230728172028.2074052-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1690569573680100007 This allow to skip some rings that qemu does not want to enable. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-vdpa.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index bebcc9fe7c..1281502a71 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -896,6 +896,11 @@ static int vhost_vdpa_set_vrings_ready(struct vhost_de= v *dev) =20 assert(dev->vhost_ops->backend_type =3D=3D VHOST_BACKEND_TYPE_VDPA); =20 + if (v->virtio_ops && v->virtio_ops->should_enable + && !(v->virtio_ops->should_enable(v))) { + return 0; + } + for (i =3D 0; i < dev->nvqs; ++i) { vhost_vdpa_set_vring_ready(v, dev->vq_index + i); } --=20 2.39.3 From nobody Sat May 18 03:46:11 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1690568471; cv=none; d=zohomail.com; s=zohoarc; b=oCVZ/9CsLNah09UmC2gLBX13KcqVCA8U5900H5RyxbtWB70hadqpJFIUU+Eb033aRQpDC96efMDF46S/53rFIXyuqKThn+SR0a3x45B/z6n36vJiVTNsxd/qv7+LAepOjwL+TtwaN4nVMHsI81uak28uU9NI3I4yCVBMP2z3zds= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1690568471; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=HxuunUKvDmqwi3/RPgyAnob3gvg7QBjSxcGz824Vq1g=; b=VBzTjdRL7GH6VdbR+7RdRICOYs5K/0CHXGSfj86VhZAa+i8+AnkBIwYouYATV0yLdPq0StGZ9jrEmx5XqXUYonzj5siMeOHBSUVJrNwerkNUE5abwt+iJlgk10TdKNWTS2w/aUGpnFXQwQvrqqXMXWJ6gFLI8UaORuEaOmVp2b4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 16905684713493.393380574922844; Fri, 28 Jul 2023 11:21:11 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qPR93-0001Zf-8A; Fri, 28 Jul 2023 13:20:53 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qPR91-0001ZF-UO for qemu-devel@nongnu.org; Fri, 28 Jul 2023 13:20:51 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qPR90-0008TP-Es for qemu-devel@nongnu.org; Fri, 28 Jul 2023 13:20:51 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-553-VLNSlhkTP8S-yisbYSri1g-1; Fri, 28 Jul 2023 13:20:45 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id ACD5D803FDC; Fri, 28 Jul 2023 17:20:44 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.99]) by smtp.corp.redhat.com (Postfix) with ESMTP id 763211121330; Fri, 28 Jul 2023 17:20:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690564849; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=HxuunUKvDmqwi3/RPgyAnob3gvg7QBjSxcGz824Vq1g=; b=BcKm4o+tSaNE3TRODp5CRBMhnuY20zpTL17ARxOs2oX0BzZNChVwmUvwuO0jC/TJpwDOmi z5Zq3+yEthQ32dIM6bN4Gq4tBIYjm6kFq1imhXBPFoSon/YWDruzXVTt/RrhV2m1tJqwPS Z6Sae979/WooXALxYXuZjNSezA8XwnY= X-MC-Unique: VLNSlhkTP8S-yisbYSri1g-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Jason Wang , Gautam Dawar , si-wei.liu@oracle.com, Zhu Lingshan , Stefano Garzarella , Parav Pandit , Cindy Lu , "Michael S. Tsirkin" , Harpreet Singh Anand , Laurent Vivier , Shannon Nelson , Lei Yang , Dragos Tatulea Subject: [PATCH 4/7] vdpa: add stub vhost_vdpa_should_enable Date: Fri, 28 Jul 2023 19:20:25 +0200 Message-Id: <20230728172028.2074052-5-eperezma@redhat.com> In-Reply-To: <20230728172028.2074052-1-eperezma@redhat.com> References: <20230728172028.2074052-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1690568472210100001 To restore the device at the destination of a live migration we send the commands through control virtqueue. For a device to read CVQ it must have received the DRIVER_OK status bit. However this opens a window where the device could start receiving packets in rx queue 0 before it receives the RSS configuration. To avoid that, we will not send vring_enable until all configuration is used by the device. Add vhost-vdpa net vhost_vdpa_should_enable. Do not change the behavior in this commit, only introduce the op. Signed-off-by: Eugenio P=C3=A9rez --- net/vhost-vdpa.c | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 9795306742..3d7dc3e5c0 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -1255,6 +1255,15 @@ static const VhostShadowVirtqueueOps vhost_vdpa_net_= svq_ops =3D { .avail_handler =3D vhost_vdpa_net_handle_ctrl_avail, }; =20 +static bool vhost_vdpa_should_enable(const struct vhost_vdpa *v) +{ + return true; +} + +static const VhostVDPAVirtIOOps vhost_vdpa_virtio_net_ops =3D { + .should_enable =3D vhost_vdpa_should_enable, +}; + /** * Probe if CVQ is isolated * @@ -1378,6 +1387,7 @@ static NetClientState *net_vhost_vdpa_init(NetClientS= tate *peer, s->vhost_vdpa.shadow_vqs_enabled =3D svq; s->vhost_vdpa.iova_range =3D iova_range; s->vhost_vdpa.shadow_data =3D svq; + s->vhost_vdpa.virtio_ops =3D &vhost_vdpa_virtio_net_ops; if (queue_pair_index =3D=3D 0) { vhost_vdpa_net_valid_svq_features(features, &s->vhost_vdpa.migration_blocker= ); --=20 2.39.3 From nobody Sat May 18 03:46:11 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1690569571; cv=none; d=zohomail.com; s=zohoarc; b=SOc5fyyy5LqO5Ci8GDmL2MJwZ6QTvOLM11x7TTMNdghvdmNKetFZUaMgeaPGRQXSTReEI8we22qqwBC5zGj3N+em2/+joT6V1Y76p1SNzLmG7rl+dQmwsUk4llyVJmSaw+xs4rg7qeySBcGdsqRqRIfqYWcgU6E3X87MkJ3Hi98= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1690569571; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=SsZLuDbr8hKAlocP8fEDhU4P8chzoS8FDIBSDERk4N8=; b=bxgiQrpiRdou2sj1WNXpQm7OawfawRDkxvmMc8m98BsTPoY1T/3PW0u0x6tD7G5ETu5MaRzZJqeQxEHatRzTRA8j5mHYd2mfK+bp6mhCmBu2kNbgOg2xuNUT6c9sGUvUncuDoubJvZ9EMc0Vh8nzpdurt3V8KM7Xav1PNMN4yrk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 169056957138825.137008434288305; Fri, 28 Jul 2023 11:39:31 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qPR94-0001aY-Qn; Fri, 28 Jul 2023 13:20:54 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qPR93-0001Ze-4W for qemu-devel@nongnu.org; Fri, 28 Jul 2023 13:20:53 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qPR91-00009U-MG for qemu-devel@nongnu.org; Fri, 28 Jul 2023 13:20:52 -0400 Received: from mimecast-mx02.redhat.com (66.187.233.73 [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-682-VWUAtlpeMwq7sv7Ct-ub3Q-1; Fri, 28 Jul 2023 13:20:47 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 302543C0D84C; Fri, 28 Jul 2023 17:20:47 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.99]) by smtp.corp.redhat.com (Postfix) with ESMTP id ED9741121330; Fri, 28 Jul 2023 17:20:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690564851; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=SsZLuDbr8hKAlocP8fEDhU4P8chzoS8FDIBSDERk4N8=; b=dac8VJlrCwZPS1g2SY79U7+haZQPUl8zntI62oEK5ECEkomiQHZu7hGBLwWUH725qgrH8d vozb0jHmygZVaM0GfLJEdsgN8EWnHz1DoRrzt8dwcGv5JBbBGj+3kvamivktMIRN8R97Mt LRnaVR4XpixXz5F3lBlaVSavwthAo2A= X-MC-Unique: VWUAtlpeMwq7sv7Ct-ub3Q-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Jason Wang , Gautam Dawar , si-wei.liu@oracle.com, Zhu Lingshan , Stefano Garzarella , Parav Pandit , Cindy Lu , "Michael S. Tsirkin" , Harpreet Singh Anand , Laurent Vivier , Shannon Nelson , Lei Yang , Dragos Tatulea Subject: [PATCH 5/7] vdpa: delay enable of data vqs Date: Fri, 28 Jul 2023 19:20:26 +0200 Message-Id: <20230728172028.2074052-6-eperezma@redhat.com> In-Reply-To: <20230728172028.2074052-1-eperezma@redhat.com> References: <20230728172028.2074052-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1690569571910100001 To restore the device at the destination of a live migration we send the commands through control virtqueue. For a device to read CVQ it must have received the DRIVER_OK status bit. However this opens a window where the device could start receiving packets in rx queue 0 before it receives the RSS configuration. To avoid that, we do not send vring_enable until all configuration is used by the device. Signed-off-by: Eugenio P=C3=A9rez --- v2: Enable the dataplane vqs if cannot shadow CVQ because of device features or ASID. --- net/vhost-vdpa.c | 44 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 43 insertions(+), 1 deletion(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 3d7dc3e5c0..2c1cfda657 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -283,6 +283,15 @@ static VhostVDPAState *vhost_vdpa_net_first_nc_vdpa(Vh= ostVDPAState *s) return DO_UPCAST(VhostVDPAState, nc, nc0); } =20 +/** From any vdpa net client, get the netclient of the last queue pair */ +static VhostVDPAState *vhost_vdpa_net_last_nc_vdpa(VhostVDPAState *s) +{ + VirtIONet *n =3D qemu_get_nic_opaque(s->nc.peer); + NetClientState *nc =3D qemu_get_peer(n->nic->ncs, n->max_ncs - 1); + + return DO_UPCAST(VhostVDPAState, nc, nc); +} + static void vhost_vdpa_net_log_global_enable(VhostVDPAState *s, bool enabl= e) { struct vhost_vdpa *v =3D &s->vhost_vdpa; @@ -996,6 +1005,13 @@ static int vhost_vdpa_net_load(NetClientState *nc) return r; } =20 + for (int i =3D 0; i < v->dev->vq_index; ++i) { + r =3D vhost_vdpa_set_vring_ready(v, i); + if (unlikely(r)) { + return r; + } + } + return 0; } =20 @@ -1255,9 +1271,35 @@ static const VhostShadowVirtqueueOps vhost_vdpa_net_= svq_ops =3D { .avail_handler =3D vhost_vdpa_net_handle_ctrl_avail, }; =20 +/** + * Check if a vhost_vdpa device should enable before DRIVER_OK + * + * CVQ must always start first if we want to restore the state safely. Do = not + * start data vqs if the device has CVQ. + */ static bool vhost_vdpa_should_enable(const struct vhost_vdpa *v) { - return true; + struct vhost_dev *dev =3D v->dev; + VhostVDPAState *s =3D container_of(v, VhostVDPAState, vhost_vdpa); + VhostVDPAState *cvq_s =3D vhost_vdpa_net_last_nc_vdpa(s); + + if (!(dev->vq_index_end % 2)) { + /* vDPA device does not have CVQ */ + return true; + } + + if (dev->vq_index + 1 =3D=3D dev->vq_index_end) { + /* We're evaluating CVQ, that must always enable first */ + return true; + } + + if (!vhost_vdpa_net_valid_svq_features(v->dev->features, NULL) || + !cvq_s->cvq_isolated) { + /* CVQ is not isolated, so let's enable as usual */ + return true; + } + + return false; } =20 static const VhostVDPAVirtIOOps vhost_vdpa_virtio_net_ops =3D { --=20 2.39.3 From nobody Sat May 18 03:46:11 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1690570492; cv=none; d=zohomail.com; s=zohoarc; b=bzvB3ynQXJ0RRzON8SO4/pc3qNibbE/nSzDn66JQPtGtcKukuw/G9Rn9m6Dqeoh/99me5G8/mYDmB0k+aXdgIvtv5GNGyG3VELReyk/zpdop8bGb70rrzSqsDI/reAlYO1yunXbq5tdC3ytBYoqjlhZy53P7ghOklPmx1sAiKEM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1690570492; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=DLBGJawtkrYHfVbJApKuQXOsyi31D2kKEHtjvnCvcO8=; b=YfsMSpJliMOx+zYfys6TgCavzxopSos+QZt1AQihirBFvD0cfwAtgYHnjL0NclruQDIAKSfqRmTV7g9Hd9yx5k94sc74mnlktQyrUmC3O3QUOETmySkZNRPL2aKg80wNYF7upXADNLAddvpLDlCTQ7VKVqtLZiC3rA1LMbuNogM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1690570492447644.0572335174819; Fri, 28 Jul 2023 11:54:52 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qPR96-0001ao-Nm; Fri, 28 Jul 2023 13:20:56 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qPR95-0001ac-BM for qemu-devel@nongnu.org; Fri, 28 Jul 2023 13:20:55 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qPR94-0000Er-15 for qemu-devel@nongnu.org; Fri, 28 Jul 2023 13:20:55 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-112-wjjaCNrdMzqMgyaW8Rbbuw-1; Fri, 28 Jul 2023 13:20:50 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B00C78008A4; Fri, 28 Jul 2023 17:20:49 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.99]) by smtp.corp.redhat.com (Postfix) with ESMTP id 7244C1121330; Fri, 28 Jul 2023 17:20:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690564853; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=DLBGJawtkrYHfVbJApKuQXOsyi31D2kKEHtjvnCvcO8=; b=A5u/iCinbMr13vWK7FXnks1jbSVwQaDVBbJ153iHotHItK28U+EDPZmEvIbbw7QSEv9uK5 wFRJlJHi6IdS0mnwCD4Td5wCezfKZDGGCxgwlKau5d/l+3BI6oAnvjdRe/UnYzREQIOH9V rb7ma2aucBGCaBC4SI/cPV+GlHG8A/o= X-MC-Unique: wjjaCNrdMzqMgyaW8Rbbuw-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Jason Wang , Gautam Dawar , si-wei.liu@oracle.com, Zhu Lingshan , Stefano Garzarella , Parav Pandit , Cindy Lu , "Michael S. Tsirkin" , Harpreet Singh Anand , Laurent Vivier , Shannon Nelson , Lei Yang , Dragos Tatulea Subject: [PATCH 6/7] vdpa: enable cvq svq if data vq are shadowed Date: Fri, 28 Jul 2023 19:20:27 +0200 Message-Id: <20230728172028.2074052-7-eperezma@redhat.com> In-Reply-To: <20230728172028.2074052-1-eperezma@redhat.com> References: <20230728172028.2074052-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1690570492991100001 Previous to this commit, it was assumed that data can only be shadowed with x-cvq, or if a migration was in place. So CVQ did not need to check for migration. Signed-off-by: Eugenio P=C3=A9rez --- net/vhost-vdpa.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 2c1cfda657..118837c6b9 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -514,11 +514,10 @@ static int vhost_vdpa_net_cvq_start(NetClientState *n= c) =20 s0 =3D vhost_vdpa_net_first_nc_vdpa(s); v->shadow_data =3D s0->vhost_vdpa.shadow_vqs_enabled; - v->shadow_vqs_enabled =3D s->always_svq; + v->shadow_vqs_enabled =3D v->shadow_data; s->vhost_vdpa.address_space_id =3D VHOST_VDPA_GUEST_PA_ASID; =20 - if (s->vhost_vdpa.shadow_data) { - /* SVQ is already configured for all virtqueues */ + if (s->always_svq) { goto out; } =20 --=20 2.39.3 From nobody Sat May 18 03:46:12 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1690569696; cv=none; d=zohomail.com; s=zohoarc; b=hi6gBmBMnFLxQxgKA6KdcQmaYXCdkSbOwtZnwrnlIjJ5zftSTn4dS9Rwf3awX0d9GlZl9ZNcvMdgOHYh7AG8m9j2JotHpuLX5D/w8alMc8psBsaW/vCfztZyN7p2va6CPv5h1n3aofeROC3KYUpReGjOWtNLuI+lBku047x4bZg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1690569696; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=g/7pQzHfU65+0Q5ONhHthvSzKMuKtyj9JrntdEQCv4w=; b=cAMAX3NYJ1LIWQi8SMNC273ghre37vO1v9vbqIsSkcGGhv6q0hHaUolnNy2E9QLddkHwULmwXZwywdUdZ4Q7YXPDQz0uZwrrpB8v6hXdcKwM5gRSXb1vWIGAZiYzbkpG3c+AJIuVAYG6NtHSkPyYxr8dd1T+aMbyRyy1pMq/VU4= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1690569696274846.7587063564664; Fri, 28 Jul 2023 11:41:36 -0700 (PDT) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1qPR9A-0001bR-DI; Fri, 28 Jul 2023 13:21:00 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qPR98-0001b4-A4 for qemu-devel@nongnu.org; Fri, 28 Jul 2023 13:20:58 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1qPR97-0000MP-0L for qemu-devel@nongnu.org; Fri, 28 Jul 2023 13:20:58 -0400 Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-283-t0kkvNi7PzOZpCZk5arIIg-1; Fri, 28 Jul 2023 13:20:52 -0400 Received: from smtp.corp.redhat.com (int-mx03.intmail.prod.int.rdu2.redhat.com [10.11.54.3]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 3331D856F67; Fri, 28 Jul 2023 17:20:52 +0000 (UTC) Received: from eperezma.remote.csb (unknown [10.39.193.99]) by smtp.corp.redhat.com (Postfix) with ESMTP id F07F31121330; Fri, 28 Jul 2023 17:20:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1690564856; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=g/7pQzHfU65+0Q5ONhHthvSzKMuKtyj9JrntdEQCv4w=; b=YtPccQ+oxSYphSVgrU4856jORsk3LnWH4K40nx0gZvSroNOCVt4g9uWQ44E4fMQBgRx8T1 xQz9Sz+CrTJUZTRhd9uzZp+cxtdjvd0DADdKhPISo7WFwycjoYbP1IrHYN6VPAFINKsWWG T9NFF6zQ29RA8t1oc0++/InksRJ40eA= X-MC-Unique: t0kkvNi7PzOZpCZk5arIIg-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Cc: Jason Wang , Gautam Dawar , si-wei.liu@oracle.com, Zhu Lingshan , Stefano Garzarella , Parav Pandit , Cindy Lu , "Michael S. Tsirkin" , Harpreet Singh Anand , Laurent Vivier , Shannon Nelson , Lei Yang , Dragos Tatulea Subject: [PATCH 7/7] vdpa: remove net cvq migration blocker Date: Fri, 28 Jul 2023 19:20:28 +0200 Message-Id: <20230728172028.2074052-8-eperezma@redhat.com> In-Reply-To: <20230728172028.2074052-1-eperezma@redhat.com> References: <20230728172028.2074052-1-eperezma@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.1 on 10.11.54.3 Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1690569697697100003 Now that we have add migration blockers if the device does not support all the needed features, remove the general blocker applied to all net devices with CVQ. Signed-off-by: Eugenio P=C3=A9rez --- net/vhost-vdpa.c | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 118837c6b9..cc3455d131 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -1443,18 +1443,6 @@ static NetClientState *net_vhost_vdpa_init(NetClient= State *peer, s->vhost_vdpa.shadow_vq_ops =3D &vhost_vdpa_net_svq_ops; s->vhost_vdpa.shadow_vq_ops_opaque =3D s; s->cvq_isolated =3D cvq_isolated; - - /* - * TODO: We cannot migrate devices with CVQ and no x-svq enabled as - * there is no way to set the device state (MAC, MQ, etc) before - * starting the datapath. - * - * Migration blocker ownership now belongs to s->vhost_vdpa. - */ - if (!svq) { - error_setg(&s->vhost_vdpa.migration_blocker, - "net vdpa cannot migrate with CVQ feature"); - } } ret =3D vhost_vdpa_add(nc, (void *)&s->vhost_vdpa, queue_pair_index, n= vqs); if (ret) { --=20 2.39.3