From nobody Wed Nov 27 00:42:48 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1703582863; cv=none; d=zohomail.com; s=zohoarc; b=XTbxkgfeKwVZ0uuhf64/vpTIl3k1OL0j33Iom6CLSuJLxORE4wl/ljoFho40eZT1WZxsDOJh3wPCdDIbchLBXKViv1GEvUYsm3PyrJcuDEpjYF+9K+Y4/S8a4fty/0+qMZpRfykC8g5UY5PShH3QxF9HpqzjfruXsGxqe7PTS3w= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1703582863; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:Subject:To:To:Message-Id:Reply-To; bh=kho8Kn81JXFBrbbnbKhrQXI3g7g8qDF5iaK2v9gtOs0=; b=SH39dK1J+i5PE6pCca6d54/n7ghkXsSYC2r3DkMCihQBVJLfYlPkGIKyPqbfgV0L0Yl4XGdHHFSXznSEzJsjEop3ACH2CvUfh7hd6e1/6ri4yS5CfBsdnMSq3TF5KhBs4nEC44Xbn6InEic+GAlizJc6trxaJXp555UlXuu38wg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=pass header.from= (p=none dis=none) Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1703582863251801.9464582021645; Tue, 26 Dec 2023 01:27:43 -0800 (PST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rI3gP-0002nP-4G; Tue, 26 Dec 2023 04:25:05 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3gM-0002f8-Si for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:25:03 -0500 Received: from us-smtp-delivery-124.mimecast.com ([170.10.129.124]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rI3gI-0008K3-Vb for qemu-devel@nongnu.org; Tue, 26 Dec 2023 04:25:00 -0500 Received: from mail-wr1-f72.google.com (mail-wr1-f72.google.com [209.85.221.72]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-593-P0-2xfzVN7ijvjKc_3oHGw-1; Tue, 26 Dec 2023 04:24:56 -0500 Received: by mail-wr1-f72.google.com with SMTP id ffacd0b85a97d-33688a38636so3151679f8f.1 for ; Tue, 26 Dec 2023 01:24:56 -0800 (PST) Received: from redhat.com ([2.55.177.189]) by smtp.gmail.com with ESMTPSA id e25-20020adfa459000000b003365951cef9sm12261228wra.55.2023.12.26.01.24.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 26 Dec 2023 01:24:51 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1703582698; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=kho8Kn81JXFBrbbnbKhrQXI3g7g8qDF5iaK2v9gtOs0=; b=ZR+dNu6oTG+V/eotmO7qBAtkFEY90HAa3e4a7lS4UkWFXBj8OpyiyPPUjoEAMRTjM9LNAP HnRjmueRTlJFcuGQdWeKVZftHttXmemhsIhr7sIHBPDatU7I7sZXK1HDJ6F730byAFo0H7 c5JLcsiTqWO4xY9JWWOY6pP3NVEhmNI= X-MC-Unique: P0-2xfzVN7ijvjKc_3oHGw-1 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703582694; x=1704187494; h=in-reply-to:content-transfer-encoding:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=kho8Kn81JXFBrbbnbKhrQXI3g7g8qDF5iaK2v9gtOs0=; b=dMOf8vxj8qrpjcZqKr/lUQfvcWeCKCSfZJ1zkoANYzA8rwMlMHKQ8bkgHynGRudxrS 9kT/QV5RKBotu4LgAY/hsB5uev/709t1TKCBMGjH8BmbsXrLH+Tdcvri/hYdxLJVIs8A 4jJWUmz3k/WRw82pw35olK3RxZNPQqSHcsCE1XXH/8uK27Ff0TCiIcOizdmzUnmc3GZk VuPsev1Nb1heSLufyLF5RYmyqeMu/IR52PrtOj0LxXlE/MF405VnD+7CMrgwGpuCkV4q REkvNVyx1SQTarUyXfdSNkEWk67Tt96d7HhPOpUtfzZt/Kqs51n5HaSgJTYIomoiHcO2 Hh9Q== X-Gm-Message-State: AOJu0YyB9y1dL0yxN+W/sVt9CiNisc2ve4w+oXB9r3Xi0fToo0z/EvXv pveIBR4lElUd8AXMVrgxOit1244TxnktGOlH7YCIH7BByRfI9elHoNnyEdHMSms10xV8HFZq6pJ V9TjSucC2sZ/k56aRIP68VPD5LivDo4Gvwx/iDNOBq7aKVJeZCIWF+aJkS5iro786LmAAMluC/k 6O X-Received: by 2002:a05:600c:1987:b0:40d:5798:1797 with SMTP id t7-20020a05600c198700b0040d57981797mr1036205wmq.63.1703582694037; Tue, 26 Dec 2023 01:24:54 -0800 (PST) X-Google-Smtp-Source: AGHT+IFAWmUvPxc4Je98bTMHxr/II4hAz93qZvI+rO3dt2AgTAIBQQ99PHYcgWzVXK8axzSKfUm5jw== X-Received: by 2002:a05:600c:1987:b0:40d:5798:1797 with SMTP id t7-20020a05600c198700b0040d57981797mr1036197wmq.63.1703582693724; Tue, 26 Dec 2023 01:24:53 -0800 (PST) Date: Tue, 26 Dec 2023 04:24:49 -0500 From: "Michael S. Tsirkin" To: qemu-devel@nongnu.org Cc: Peter Maydell , Eugenio =?utf-8?B?UMOpcmV6?= , Jason Wang , Lei Yang Subject: [PULL 12/21] vdpa: move shadow_data to vhost_vdpa_shared Message-ID: <787ac82bcea98139204c26ccf2a488d2f9b54003.1703582625.git.mst@redhat.com> References: MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable In-Reply-To: X-Mailer: git-send-email 2.27.0.106.g8ac3dc51b1 X-Mutt-Fcc: =sent Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.129.124; envelope-from=mst@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -50 X-Spam_score: -5.1 X-Spam_bar: ----- X-Spam_report: (-5.1 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-2.977, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: qemu-devel-bounces+importer=patchew.org@nongnu.org X-ZohoMail-DKIM: pass (identity @redhat.com) X-ZM-MESSAGEID: 1703582865294100017 From: Eugenio P=C3=A9rez Next patches will register the vhost_vdpa memory listener while the VM is migrating at the destination, so we can map the memory to the device before stopping the VM at the source. The main goal is to reduce the downtime. However, the destination QEMU is unaware of which vhost_vdpa device will register its memory_listener. If the source guest has CVQ enabled, it will be the CVQ device. Otherwise, it will be the first one. Move the shadow_data member to VhostVDPAShared so all vhost_vdpa can use it, rather than always in the first or last vhost_vdpa. Signed-off-by: Eugenio P=C3=A9rez Acked-by: Jason Wang Message-Id: <20231221174322.3130442-5-eperezma@redhat.com> Tested-by: Lei Yang Reviewed-by: Michael S. Tsirkin Signed-off-by: Michael S. Tsirkin --- include/hw/virtio/vhost-vdpa.h | 5 +++-- hw/virtio/vhost-vdpa.c | 6 +++--- net/vhost-vdpa.c | 22 +++++----------------- 3 files changed, 11 insertions(+), 22 deletions(-) diff --git a/include/hw/virtio/vhost-vdpa.h b/include/hw/virtio/vhost-vdpa.h index 8d52a7e498..01e0f25e27 100644 --- a/include/hw/virtio/vhost-vdpa.h +++ b/include/hw/virtio/vhost-vdpa.h @@ -36,6 +36,9 @@ typedef struct vhost_vdpa_shared { =20 /* IOVA mapping used by the Shadow Virtqueue */ VhostIOVATree *iova_tree; + + /* Vdpa must send shadow addresses as IOTLB key for data queues, not G= PA */ + bool shadow_data; } VhostVDPAShared; =20 typedef struct vhost_vdpa { @@ -47,8 +50,6 @@ typedef struct vhost_vdpa { MemoryListener listener; uint64_t acked_features; bool shadow_vqs_enabled; - /* Vdpa must send shadow addresses as IOTLB key for data queues, not G= PA */ - bool shadow_data; /* Device suspended successfully */ bool suspended; VhostVDPAShared *shared; diff --git a/hw/virtio/vhost-vdpa.c b/hw/virtio/vhost-vdpa.c index 5ff1d43ba9..97588848fc 100644 --- a/hw/virtio/vhost-vdpa.c +++ b/hw/virtio/vhost-vdpa.c @@ -353,7 +353,7 @@ static void vhost_vdpa_listener_region_add(MemoryListen= er *listener, vaddr, section->readonly); =20 llsize =3D int128_sub(llend, int128_make64(iova)); - if (v->shadow_data) { + if (v->shared->shadow_data) { int r; =20 mem_region.translated_addr =3D (hwaddr)(uintptr_t)vaddr, @@ -380,7 +380,7 @@ static void vhost_vdpa_listener_region_add(MemoryListen= er *listener, return; =20 fail_map: - if (v->shadow_data) { + if (v->shared->shadow_data) { vhost_iova_tree_remove(v->shared->iova_tree, mem_region); } =20 @@ -435,7 +435,7 @@ static void vhost_vdpa_listener_region_del(MemoryListen= er *listener, =20 llsize =3D int128_sub(llend, int128_make64(iova)); =20 - if (v->shadow_data) { + if (v->shared->shadow_data) { const DMAMap *result; const void *vaddr =3D memory_region_get_ram_ptr(section->mr) + section->offset_within_region + diff --git a/net/vhost-vdpa.c b/net/vhost-vdpa.c index 7be2c30ad3..bf8e8327da 100644 --- a/net/vhost-vdpa.c +++ b/net/vhost-vdpa.c @@ -290,15 +290,6 @@ static ssize_t vhost_vdpa_receive(NetClientState *nc, = const uint8_t *buf, return size; } =20 -/** From any vdpa net client, get the netclient of the first queue pair */ -static VhostVDPAState *vhost_vdpa_net_first_nc_vdpa(VhostVDPAState *s) -{ - NICState *nic =3D qemu_get_nic(s->nc.peer); - NetClientState *nc0 =3D qemu_get_peer(nic->ncs, 0); - - return DO_UPCAST(VhostVDPAState, nc, nc0); -} - static void vhost_vdpa_net_log_global_enable(VhostVDPAState *s, bool enabl= e) { struct vhost_vdpa *v =3D &s->vhost_vdpa; @@ -369,13 +360,12 @@ static int vhost_vdpa_net_data_start(NetClientState *= nc) if (s->always_svq || migration_is_setup_or_active(migrate_get_current()->state)) { v->shadow_vqs_enabled =3D true; - v->shadow_data =3D true; } else { v->shadow_vqs_enabled =3D false; - v->shadow_data =3D false; } =20 if (v->index =3D=3D 0) { + v->shared->shadow_data =3D v->shadow_vqs_enabled; vhost_vdpa_net_data_start_first(s); return 0; } @@ -523,7 +513,7 @@ dma_map_err: =20 static int vhost_vdpa_net_cvq_start(NetClientState *nc) { - VhostVDPAState *s, *s0; + VhostVDPAState *s; struct vhost_vdpa *v; int64_t cvq_group; int r; @@ -534,12 +524,10 @@ static int vhost_vdpa_net_cvq_start(NetClientState *n= c) s =3D DO_UPCAST(VhostVDPAState, nc, nc); v =3D &s->vhost_vdpa; =20 - s0 =3D vhost_vdpa_net_first_nc_vdpa(s); - v->shadow_data =3D s0->vhost_vdpa.shadow_vqs_enabled; - v->shadow_vqs_enabled =3D s0->vhost_vdpa.shadow_vqs_enabled; + v->shadow_vqs_enabled =3D v->shared->shadow_data; s->vhost_vdpa.address_space_id =3D VHOST_VDPA_GUEST_PA_ASID; =20 - if (s->vhost_vdpa.shadow_data) { + if (v->shared->shadow_data) { /* SVQ is already configured for all virtqueues */ goto out; } @@ -1688,12 +1676,12 @@ static NetClientState *net_vhost_vdpa_init(NetClien= tState *peer, s->always_svq =3D svq; s->migration_state.notify =3D NULL; s->vhost_vdpa.shadow_vqs_enabled =3D svq; - s->vhost_vdpa.shadow_data =3D svq; if (queue_pair_index =3D=3D 0) { vhost_vdpa_net_valid_svq_features(features, &s->vhost_vdpa.migration_blocker= ); s->vhost_vdpa.shared =3D g_new0(VhostVDPAShared, 1); s->vhost_vdpa.shared->iova_range =3D iova_range; + s->vhost_vdpa.shared->shadow_data =3D svq; } else if (!is_datapath) { s->cvq_cmd_out_buffer =3D mmap(NULL, vhost_vdpa_net_cvq_cmd_page_l= en(), PROT_READ | PROT_WRITE, --=20 MST