From nobody Fri May 17 00:32:42 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1615837863; cv=none; d=zohomail.com; s=zohoarc; b=VwbLlxQbbYuTzeKBarDa2695M8FHNkJy/XBfh9IAi4lA/CNVz6YUQohXwkoEH/b0vBSMjZcvnLDaYUXyEIoi/TGj+6ah59Nda4wYv69KL2K0lVguNfC1cBs+/mNstHbmqzzHM381525Y3xy6nN58pfF/hGGIj7b1oFO38DTCZOI= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1615837863; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=0kky/RYc9A8VMyn0hQWQkkpfFNLpVtNdxBiYEiHpkaQ=; b=kLrjKC4E79GcrcplwHtCy/0FFVPQ6PNzJqvvq0oA/QJcPCAL1haVTgpL56nVgNeE5zNbTCq2gqZTb4zWw82eAxHAGNbxY+N6V/FCp89t36FEdGChUOu2umK8Q1p0Qv9vPaoIB7jC8WZGBEpI84XCwM+V1Nh/Z635j7yqYkTpo7Q= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 161583786309197.90158511295886; Mon, 15 Mar 2021 12:51:03 -0700 (PDT) Received: from localhost ([::1]:50350 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lLtEz-0006Fd-KF for importer@patchew.org; Mon, 15 Mar 2021 15:51:01 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37540) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLtD5-00050H-0R for qemu-devel@nongnu.org; Mon, 15 Mar 2021 15:49:03 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:36394) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1lLtD1-0003sO-BV for qemu-devel@nongnu.org; Mon, 15 Mar 2021 15:49:01 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-507-D4dPIXOoNhKXfvX26rTIVg-1; Mon, 15 Mar 2021 15:48:56 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id D068F92500; Mon, 15 Mar 2021 19:48:54 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-112-173.ams2.redhat.com [10.36.112.173]) by smtp.corp.redhat.com (Postfix) with ESMTP id 729D65F706; Mon, 15 Mar 2021 19:48:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1615837738; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=0kky/RYc9A8VMyn0hQWQkkpfFNLpVtNdxBiYEiHpkaQ=; b=M5Vt6lvog2a/iYoxOduNGk7q8Or7AL3mcWupF9Ul0H9VbRNT2DsoYKBaZV3tFLun5TOpga tqJ2ea/Wnia/fsV37H2pyLIiQhOg5c4zbZdTJCQ1JTRFYRfUDWRKXJVvgxx8MLqbuUqTmd 9JJIpvvVlwTb3n4cMKaLMTbGE18rLng= X-MC-Unique: D4dPIXOoNhKXfvX26rTIVg-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v2 01/13] virtio: Add virtio_queue_is_host_notifier_enabled Date: Mon, 15 Mar 2021 20:48:30 +0100 Message-Id: <20210315194842.277740-2-eperezma@redhat.com> In-Reply-To: <20210315194842.277740-1-eperezma@redhat.com> References: <20210315194842.277740-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.25, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Guru Prasad , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Stefano Garzarella , Michael Lilja , Jim Harford , Rob Miller Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) This allows shadow virtqueue code to assert the queue status before making changes. Signed-off-by: Eugenio P=C3=A9rez --- include/hw/virtio/virtio.h | 1 + hw/virtio/virtio.c | 5 +++++ 2 files changed, 6 insertions(+) diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h index b7ece7a6a8..c2c7cee993 100644 --- a/include/hw/virtio/virtio.h +++ b/include/hw/virtio/virtio.h @@ -316,6 +316,7 @@ void virtio_device_release_ioeventfd(VirtIODevice *vdev= ); bool virtio_device_ioeventfd_enabled(VirtIODevice *vdev); EventNotifier *virtio_queue_get_host_notifier(VirtQueue *vq); void virtio_queue_set_host_notifier_enabled(VirtQueue *vq, bool enabled); +bool virtio_queue_is_host_notifier_enabled(const VirtQueue *vq); void virtio_queue_host_notifier_read(EventNotifier *n); void virtio_queue_aio_set_host_notifier_handler(VirtQueue *vq, AioContext = *ctx, VirtIOHandleAIOOutput hand= le_output); diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index 07f4e60b30..a86b3f9c26 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -3594,6 +3594,11 @@ EventNotifier *virtio_queue_get_host_notifier(VirtQu= eue *vq) return &vq->host_notifier; } =20 +bool virtio_queue_is_host_notifier_enabled(const VirtQueue *vq) +{ + return vq->host_notifier_enabled; +} + void virtio_queue_set_host_notifier_enabled(VirtQueue *vq, bool enabled) { vq->host_notifier_enabled =3D enabled; --=20 2.27.0 From nobody Fri May 17 00:32:42 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1615837995; cv=none; d=zohomail.com; s=zohoarc; b=hbpahIBs+EXrZVMzKDGA32FNkE2vjM936s2HOR9a7Z0hpDRRlYhk6SZd7isbgYaDCIqSyULDf/f4rhCGG0el1DTNbQ0XWsWswkHSynHUjNj9pyvmq5IWu1QIOJ8+cC5wJz7H0lFGXm068KBQdAqLMBZ8X3n7qkWSqauxwpVaf5o= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1615837995; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=7e52shHXEbVUO5OUANPRBavCWxWLcLX8FMHXidmngsI=; b=bA8qcFvEDvWTzlpwbm2PRrt3gpVTn89V7cCsuxCKSGBwsz19Gw3vTqGXN8VtL6LpctKNty25IbLYvZyd8rpnwVsZ72ljrUlC2/kId3euRke0uf3cUEBko9sRxAEB1JOai4HusVcMuxw85okosTpor80lUw2AIR/vseteVGC0yLM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 161583799549231.46113949065557; Mon, 15 Mar 2021 12:53:15 -0700 (PDT) Received: from localhost ([::1]:58490 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lLtH7-0001Hf-SJ for importer@patchew.org; Mon, 15 Mar 2021 15:53:13 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37554) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLtD6-00050a-1O for qemu-devel@nongnu.org; Mon, 15 Mar 2021 15:49:04 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:56861) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1lLtD4-0003ss-FF for qemu-devel@nongnu.org; Mon, 15 Mar 2021 15:49:03 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-552-uSrrjxn-O4uGucdMVlleag-1; Mon, 15 Mar 2021 15:49:00 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id A10D919200C5; Mon, 15 Mar 2021 19:48:58 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-112-173.ams2.redhat.com [10.36.112.173]) by smtp.corp.redhat.com (Postfix) with ESMTP id 32B361876A; Mon, 15 Mar 2021 19:48:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1615837741; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7e52shHXEbVUO5OUANPRBavCWxWLcLX8FMHXidmngsI=; b=C3y5e0nqT0ySjPWGkMIFA5qor7KrTcC6kuXiBhrdoZ1PAWZS/hBPfLwyONzUZ3jpU+UxaC FuGvaUcSDfeU23emQ44RvtnNuXXvDBAiLVe2LNXA8sLTugg5+3T9Qxv2EChul7pbcHnAI9 9U2NnHxww1CxukMpe778bHKy08Wyi+Y= X-MC-Unique: uSrrjxn-O4uGucdMVlleag-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v2 02/13] vhost: Save masked_notifier state Date: Mon, 15 Mar 2021 20:48:31 +0100 Message-Id: <20210315194842.277740-3-eperezma@redhat.com> In-Reply-To: <20210315194842.277740-1-eperezma@redhat.com> References: <20210315194842.277740-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=63.128.21.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.25, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Guru Prasad , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Stefano Garzarella , Michael Lilja , Jim Harford , Rob Miller Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) It will be used to configure shadow virtqueue. Shadow virtqueue will relay the device->guest notifications, so vhost need to be able to tell the masking status. Signed-off-by: Eugenio P=C3=A9rez --- include/hw/virtio/vhost.h | 1 + hw/virtio/vhost.c | 2 ++ 2 files changed, 3 insertions(+) diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h index 4a8bc75415..ac963bf23d 100644 --- a/include/hw/virtio/vhost.h +++ b/include/hw/virtio/vhost.h @@ -28,6 +28,7 @@ struct vhost_virtqueue { unsigned avail_size; unsigned long long used_phys; unsigned used_size; + bool notifier_is_masked; EventNotifier masked_notifier; struct vhost_dev *dev; }; diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index e2163a0d63..4680c0cfcf 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -1527,6 +1527,8 @@ void vhost_virtqueue_mask(struct vhost_dev *hdev, Vir= tIODevice *vdev, int n, r =3D hdev->vhost_ops->vhost_set_vring_call(hdev, &file); if (r < 0) { VHOST_OPS_DEBUG("vhost_set_vring_call failed"); + } else { + hdev->vqs[index].notifier_is_masked =3D mask; } } =20 --=20 2.27.0 From nobody Fri May 17 00:32:42 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1615838009; cv=none; d=zohomail.com; s=zohoarc; b=dFWuPsmSQSJV5l+Nu5iiWKqg6osawztEUHYbVPzcTTlV7ALsPfhlLwCbTR79xTf147K6ZvKVKBr3CeyI3YzAdX1Imt5xX7qs4hgvxc09I/GAzVHgB2MH6UZR3inhq47WCF4WT/YZN7EpgaF3EuRGxaenosnHJKoacJcVYMxLB8g= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1615838009; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=yJUB2BKxUp+/hgtNUy8u9evkox43m4+lW00bD5Y36Zs=; b=OSF1Pf8FmnGTQNsUtCfUkKu1Ant0mFiiGRPEYOlNrt463O91NyB0yy6lJrOFp4QiElpmnfP1R6qQWML/ryAht0QLlGcIbQhvuOsNJ4BvsLwUlff12IbcvjmtaZDwuA+bD1IGpFymn7RRzAJs7rAUrz4pTTUcJ7FckbQQEcKC2+E= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1615838009675365.6151447262771; Mon, 15 Mar 2021 12:53:29 -0700 (PDT) Received: from localhost ([::1]:58702 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lLtHL-0001Mn-Fv for importer@patchew.org; Mon, 15 Mar 2021 15:53:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37566) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLtDC-0005A1-RJ for qemu-devel@nongnu.org; Mon, 15 Mar 2021 15:49:10 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:45059) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1lLtDB-0003vh-2T for qemu-devel@nongnu.org; Mon, 15 Mar 2021 15:49:10 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-428-F5x4cHkzODOYAErd37G0KA-1; Mon, 15 Mar 2021 15:49:06 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 351D219200C1; Mon, 15 Mar 2021 19:49:05 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-112-173.ams2.redhat.com [10.36.112.173]) by smtp.corp.redhat.com (Postfix) with ESMTP id 02C0E1C4; Mon, 15 Mar 2021 19:48:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1615837748; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=yJUB2BKxUp+/hgtNUy8u9evkox43m4+lW00bD5Y36Zs=; b=Slw9w4pdlHUY+2js8oQVm7L4IxR2rk5vrRr4H67OVD/jAU0lXVNhd17WuAWugn5eje9+hw NRskKbxt8TsCU09/s4VnnjNp0iwOoaRuZA3+v1S4TU56REyPQIuEz2AoKvs9bgaPuy6UB0 Xzh9tCrTXmH/rmp9FJqUx6oo/yN5nd8= X-MC-Unique: F5x4cHkzODOYAErd37G0KA-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v2 03/13] vhost: Add VhostShadowVirtqueue Date: Mon, 15 Mar 2021 20:48:32 +0100 Message-Id: <20210315194842.277740-4-eperezma@redhat.com> In-Reply-To: <20210315194842.277740-1-eperezma@redhat.com> References: <20210315194842.277740-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=63.128.21.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.25, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Guru Prasad , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Stefano Garzarella , Michael Lilja , Jim Harford , Rob Miller Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Vhost shadow virtqueue (SVQ)is an intermediate jump for virtqueue notifications and buffers, allowing qemu to track them. While qemu is forwarding the buffers and virtqueue changes, is able to commit the memory it's being dirtied, the same way regular qemu's VirtIO devices do. This commit only exposes basic SVQ allocation and free, so changes regarding different aspects of SVQ (notifications forwarding, buffer forwarding, starting/stopping) are more isolated and easier to bisect. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.h | 24 ++++++++++++ hw/virtio/vhost-shadow-virtqueue.c | 63 ++++++++++++++++++++++++++++++ hw/virtio/meson.build | 2 +- 3 files changed, 88 insertions(+), 1 deletion(-) create mode 100644 hw/virtio/vhost-shadow-virtqueue.h create mode 100644 hw/virtio/vhost-shadow-virtqueue.c diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-vi= rtqueue.h new file mode 100644 index 0000000000..6cc18d6acb --- /dev/null +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -0,0 +1,24 @@ +/* + * vhost software live migration ring + * + * SPDX-FileCopyrightText: Red Hat, Inc. 2021 + * SPDX-FileContributor: Author: Eugenio P=C3=A9rez + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#ifndef VHOST_SHADOW_VIRTQUEUE_H +#define VHOST_SHADOW_VIRTQUEUE_H + +#include "qemu/osdep.h" + +#include "hw/virtio/virtio.h" +#include "hw/virtio/vhost.h" + +typedef struct VhostShadowVirtqueue VhostShadowVirtqueue; + +VhostShadowVirtqueue *vhost_shadow_vq_new(struct vhost_dev *dev, int idx); + +void vhost_shadow_vq_free(VhostShadowVirtqueue *vq); + +#endif diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c new file mode 100644 index 0000000000..4512e5b058 --- /dev/null +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -0,0 +1,63 @@ +/* + * vhost software live migration ring + * + * SPDX-FileCopyrightText: Red Hat, Inc. 2021 + * SPDX-FileContributor: Author: Eugenio P=C3=A9rez + * + * SPDX-License-Identifier: GPL-2.0-or-later + */ + +#include "hw/virtio/vhost-shadow-virtqueue.h" + +#include "qemu/error-report.h" +#include "qemu/event_notifier.h" + +/* Shadow virtqueue to relay notifications */ +typedef struct VhostShadowVirtqueue { + /* Shadow kick notifier, sent to vhost */ + EventNotifier kick_notifier; + /* Shadow call notifier, sent to vhost */ + EventNotifier call_notifier; +} VhostShadowVirtqueue; + +/* + * Creates vhost shadow virtqueue, and instruct vhost device to use the sh= adow + * methods and file descriptors. + */ +VhostShadowVirtqueue *vhost_shadow_vq_new(struct vhost_dev *dev, int idx) +{ + g_autofree VhostShadowVirtqueue *svq =3D g_new0(VhostShadowVirtqueue, = 1); + int r; + + r =3D event_notifier_init(&svq->kick_notifier, 0); + if (r !=3D 0) { + error_report("Couldn't create kick event notifier: %s", + strerror(errno)); + goto err_init_kick_notifier; + } + + r =3D event_notifier_init(&svq->call_notifier, 0); + if (r !=3D 0) { + error_report("Couldn't create call event notifier: %s", + strerror(errno)); + goto err_init_call_notifier; + } + + return g_steal_pointer(&svq); + +err_init_call_notifier: + event_notifier_cleanup(&svq->kick_notifier); + +err_init_kick_notifier: + return NULL; +} + +/* + * Free the resources of the shadow virtqueue. + */ +void vhost_shadow_vq_free(VhostShadowVirtqueue *vq) +{ + event_notifier_cleanup(&vq->kick_notifier); + event_notifier_cleanup(&vq->call_notifier); + g_free(vq); +} diff --git a/hw/virtio/meson.build b/hw/virtio/meson.build index fbff9bc9d4..8b5a0225fe 100644 --- a/hw/virtio/meson.build +++ b/hw/virtio/meson.build @@ -11,7 +11,7 @@ softmmu_ss.add(when: 'CONFIG_ALL', if_true: files('vhost-= stub.c')) =20 virtio_ss =3D ss.source_set() virtio_ss.add(files('virtio.c')) -virtio_ss.add(when: 'CONFIG_VHOST', if_true: files('vhost.c', 'vhost-backe= nd.c')) +virtio_ss.add(when: 'CONFIG_VHOST', if_true: files('vhost.c', 'vhost-backe= nd.c', 'vhost-shadow-virtqueue.c')) virtio_ss.add(when: 'CONFIG_VHOST_USER', if_true: files('vhost-user.c')) virtio_ss.add(when: 'CONFIG_VHOST_VDPA', if_true: files('vhost-vdpa.c')) virtio_ss.add(when: 'CONFIG_VIRTIO_BALLOON', if_true: files('virtio-balloo= n.c')) --=20 2.27.0 From nobody Fri May 17 00:32:42 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1615838126; cv=none; d=zohomail.com; s=zohoarc; b=dm/AOcl11pVEZmLiD7N9BJllm4Zv8/tYRvPl/7TygyP5+eiy7S36Xhv82F5n7fWG7aCkieXuTzT9JUcEH8tKCP/j3qQVnDDsI93hQIqVVnkBV4mlAqCLfQInQoNxo1+27vIv7AazJLFs7ZR9iCRdYpqmTeU/S+kkOZU+do26sSM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1615838126; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=Gnqt716CWo5WYbtfXADF/wHUjv+rzznj/YbrsIdB6Tc=; b=mHG6x5KeUDfXiFIhO93AN0oSsuRTot5vVFGq38mcpnbwwv1ERgHX0benSdG0ndVA6nxizFKKFYYJRn8oaNLp/RltfwJm5WD7RUyKM1nmj83JgCAT83F2ic/OXzbHnu/m8yuSh6AN0sVz41+Yqe+Jg94uG3/ntq4SKcnFzBWimM0= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1615838126671532.1076078967476; Mon, 15 Mar 2021 12:55:26 -0700 (PDT) Received: from localhost ([::1]:36616 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lLtJF-0003u0-LL for importer@patchew.org; Mon, 15 Mar 2021 15:55:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37584) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLtDJ-0005LJ-To for qemu-devel@nongnu.org; Mon, 15 Mar 2021 15:49:17 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:45784) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1lLtDI-0003xv-2w for qemu-devel@nongnu.org; Mon, 15 Mar 2021 15:49:17 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-425-yhMKRI4WOzCTrJOZxSstmQ-1; Mon, 15 Mar 2021 15:49:13 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 0C46992502; Mon, 15 Mar 2021 19:49:12 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-112-173.ams2.redhat.com [10.36.112.173]) by smtp.corp.redhat.com (Postfix) with ESMTP id 88D5B5F706; Mon, 15 Mar 2021 19:49:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1615837755; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Gnqt716CWo5WYbtfXADF/wHUjv+rzznj/YbrsIdB6Tc=; b=CdIFahaeeFZPjJJ0YSR1y3Y2w3FHyHf7kaGbiQ9K+B9kWdccHaAtYID7TX+wzQX+arOoic G1xHlSFpEaSFSeYKZHqXghrRZqI6Iod6ygj47lWZEGQGSdgxVkkPjg1ZAiMi5ByfSvWR1t Pi7/tC/rcHDLNjkbYwNjQMpbdOQ+zic= X-MC-Unique: yhMKRI4WOzCTrJOZxSstmQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v2 04/13] vhost: Add x-vhost-enable-shadow-vq qmp Date: Mon, 15 Mar 2021 20:48:33 +0100 Message-Id: <20210315194842.277740-5-eperezma@redhat.com> In-Reply-To: <20210315194842.277740-1-eperezma@redhat.com> References: <20210315194842.277740-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.25, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Guru Prasad , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Stefano Garzarella , Michael Lilja , Jim Harford , Rob Miller Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Command to enable shadow virtqueue looks like: { "execute": "x-vhost-enable-shadow-vq", "arguments": { "name": "dev0", "en= able": true } } Signed-off-by: Eugenio P=C3=A9rez --- qapi/net.json | 22 ++++++++++++++++++++++ hw/virtio/vhost.c | 6 ++++++ 2 files changed, 28 insertions(+) diff --git a/qapi/net.json b/qapi/net.json index c31748c87f..4c5f65d021 100644 --- a/qapi/net.json +++ b/qapi/net.json @@ -77,6 +77,28 @@ ## { 'command': 'netdev_del', 'data': {'id': 'str'} } =20 +## +# @x-vhost-enable-shadow-vq: +# +# Use vhost shadow virtqueue. +# +# @name: the device name of the VirtIO device +# +# @enable: true to use he alternate shadow VQ notification path +# +# Returns: Error if failure, or 'no error' for success. Not found if vhost= is not enabled. +# +# Since: 6.0 +# +# Example: +# +# -> { "execute": "x-vhost-enable-shadow-vq", "arguments": { "name": "virt= io-net", "enable": false } } +# +## +{ 'command': 'x-vhost-enable-shadow-vq', + 'data': {'name': 'str', 'enable': 'bool'}, + 'if': 'defined(CONFIG_VHOST_KERNEL)' } + ## # @NetLegacyNicOptions: # diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index 4680c0cfcf..97f1bcfa42 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -15,6 +15,7 @@ =20 #include "qemu/osdep.h" #include "qapi/error.h" +#include "qapi/qapi-commands-net.h" #include "hw/virtio/vhost.h" #include "qemu/atomic.h" #include "qemu/range.h" @@ -1831,3 +1832,8 @@ int vhost_net_set_backend(struct vhost_dev *hdev, =20 return -1; } + +void qmp_x_vhost_enable_shadow_vq(const char *name, bool enable, Error **e= rrp) +{ + error_setg(errp, "Shadow virtqueue still not implemented"); +} --=20 2.27.0 From nobody Fri May 17 00:32:42 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1615837901; cv=none; d=zohomail.com; s=zohoarc; b=SfvQdap9wgRB33glsAs4XTF5oug65WWZY4qWD9oAjcUU/bLzgvGdRqpYp50yelmwDtX669ZToTXIb9DyEueNLkcgoDJWQgNJmXIPf4N0O0xziCj2b8WZ6qFfGQm3ic9+Wp5dWn7WJDW/zBMSpebqRaD2mT8q+aCHLW8l9M2obMg= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1615837901; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=7SOcpUXr4KJY4dwgxs4jw9yDkBbfxO7eDLVPHAfKXOo=; b=YweM57zRUZmaJKWKbW7pNjAXD9syrzzF8j7cFY/XmKW7LEHciWU81luJNJamryAsnYnHleqrWlORohDeDLkiaRoG4X4Rq21Gi9s7awfDhkFUKnggUTqf6PsPxJfwC4v0IPLmtDaL0Lw9/4YesORwAXed26f2Rm9lVfUs8SdjfrM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1615837901922634.4886092821362; Mon, 15 Mar 2021 12:51:41 -0700 (PDT) Received: from localhost ([::1]:52520 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lLtFc-0007FF-NC for importer@patchew.org; Mon, 15 Mar 2021 15:51:40 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37666) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLtDW-0005Rc-U9 for qemu-devel@nongnu.org; Mon, 15 Mar 2021 15:49:31 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:20785) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1lLtDM-00040s-J0 for qemu-devel@nongnu.org; Mon, 15 Mar 2021 15:49:26 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-253-DCqqoff8Ob6Ajf6esBBChg-1; Mon, 15 Mar 2021 15:49:17 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id B5C5A19200CD; Mon, 15 Mar 2021 19:49:15 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-112-173.ams2.redhat.com [10.36.112.173]) by smtp.corp.redhat.com (Postfix) with ESMTP id 623BD5F9B8; Mon, 15 Mar 2021 19:49:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1615837759; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7SOcpUXr4KJY4dwgxs4jw9yDkBbfxO7eDLVPHAfKXOo=; b=QA0MDt3C1pduRwdIYsbaMhwCu/Ms4eyTSDlR0gccnn7jZ+KpsCHbuYOVsz+XILXoUjpTD1 xErlsgUAKx2cJL9JbUa+bW5gnUKtDKinrawOAAWPWXSEZxLqNRTdFsNWjgIuUumsAyi5JA Tmt9rJ0GCEdOePthyMsjx7QIWUoMHzE= X-MC-Unique: DCqqoff8Ob6Ajf6esBBChg-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v2 05/13] vhost: Route guest->host notification through shadow virtqueue Date: Mon, 15 Mar 2021 20:48:34 +0100 Message-Id: <20210315194842.277740-6-eperezma@redhat.com> In-Reply-To: <20210315194842.277740-1-eperezma@redhat.com> References: <20210315194842.277740-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.25, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Guru Prasad , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Stefano Garzarella , Michael Lilja , Jim Harford , Rob Miller Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Shadow virtqueue notifications forwarding is disabled when vhost_dev stops, so code flow follows usual cleanup. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.h | 7 ++ include/hw/virtio/vhost.h | 4 + hw/virtio/vhost-shadow-virtqueue.c | 113 ++++++++++++++++++++++- hw/virtio/vhost.c | 143 ++++++++++++++++++++++++++++- 4 files changed, 265 insertions(+), 2 deletions(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-vi= rtqueue.h index 6cc18d6acb..c891c6510d 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -17,6 +17,13 @@ =20 typedef struct VhostShadowVirtqueue VhostShadowVirtqueue; =20 +bool vhost_shadow_vq_start(struct vhost_dev *dev, + unsigned idx, + VhostShadowVirtqueue *svq); +void vhost_shadow_vq_stop(struct vhost_dev *dev, + unsigned idx, + VhostShadowVirtqueue *svq); + VhostShadowVirtqueue *vhost_shadow_vq_new(struct vhost_dev *dev, int idx); =20 void vhost_shadow_vq_free(VhostShadowVirtqueue *vq); diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h index ac963bf23d..7ffdf9aea0 100644 --- a/include/hw/virtio/vhost.h +++ b/include/hw/virtio/vhost.h @@ -55,6 +55,8 @@ struct vhost_iommu { QLIST_ENTRY(vhost_iommu) iommu_next; }; =20 +typedef struct VhostShadowVirtqueue VhostShadowVirtqueue; + typedef struct VhostDevConfigOps { /* Vhost device config space changed callback */ @@ -83,7 +85,9 @@ struct vhost_dev { uint64_t backend_cap; bool started; bool log_enabled; + bool shadow_vqs_enabled; uint64_t log_size; + VhostShadowVirtqueue **shadow_vqs; Error *migration_blocker; const VhostOps *vhost_ops; void *opaque; diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index 4512e5b058..3e43399e9c 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -8,9 +8,12 @@ */ =20 #include "hw/virtio/vhost-shadow-virtqueue.h" +#include "hw/virtio/vhost.h" + +#include "standard-headers/linux/vhost_types.h" =20 #include "qemu/error-report.h" -#include "qemu/event_notifier.h" +#include "qemu/main-loop.h" =20 /* Shadow virtqueue to relay notifications */ typedef struct VhostShadowVirtqueue { @@ -18,14 +21,121 @@ typedef struct VhostShadowVirtqueue { EventNotifier kick_notifier; /* Shadow call notifier, sent to vhost */ EventNotifier call_notifier; + + /* + * Borrowed virtqueue's guest to host notifier. + * To borrow it in this event notifier allows to register on the event + * loop and access the associated shadow virtqueue easily. If we use t= he + * VirtQueue, we don't have an easy way to retrieve it. + * + * So shadow virtqueue must not clean it, or we would lose VirtQueue o= ne. + */ + EventNotifier host_notifier; + + /* Virtio queue shadowing */ + VirtQueue *vq; } VhostShadowVirtqueue; =20 +/* Forward guest notifications */ +static void vhost_handle_guest_kick(EventNotifier *n) +{ + VhostShadowVirtqueue *svq =3D container_of(n, VhostShadowVirtqueue, + host_notifier); + + if (unlikely(!event_notifier_test_and_clear(n))) { + return; + } + + event_notifier_set(&svq->kick_notifier); +} + +/* + * Restore the vhost guest to host notifier, i.e., disables svq effect. + */ +static int vhost_shadow_vq_restore_vdev_host_notifier(struct vhost_dev *de= v, + unsigned vhost_index, + VhostShadowVirtqueue = *svq) +{ + EventNotifier *vq_host_notifier =3D virtio_queue_get_host_notifier(svq= ->vq); + struct vhost_vring_file file =3D { + .index =3D vhost_index, + .fd =3D event_notifier_get_fd(vq_host_notifier), + }; + int r; + + /* Restore vhost kick */ + r =3D dev->vhost_ops->vhost_set_vring_kick(dev, &file); + return r ? -errno : 0; +} + +/* + * Start shadow virtqueue operation. + * @dev vhost device + * @hidx vhost virtqueue index + * @svq Shadow Virtqueue + */ +bool vhost_shadow_vq_start(struct vhost_dev *dev, + unsigned idx, + VhostShadowVirtqueue *svq) +{ + EventNotifier *vq_host_notifier =3D virtio_queue_get_host_notifier(svq= ->vq); + struct vhost_vring_file file =3D { + .index =3D idx, + .fd =3D event_notifier_get_fd(&svq->kick_notifier), + }; + int r; + + /* Check that notifications are still going directly to vhost dev */ + assert(virtio_queue_is_host_notifier_enabled(svq->vq)); + + /* + * event_notifier_set_handler already checks for guest's notifications= if + * they arrive in the switch, so there is no need to explicitely check= for + * them. + */ + event_notifier_init_fd(&svq->host_notifier, + event_notifier_get_fd(vq_host_notifier)); + event_notifier_set_handler(&svq->host_notifier, vhost_handle_guest_kic= k); + + r =3D dev->vhost_ops->vhost_set_vring_kick(dev, &file); + if (unlikely(r !=3D 0)) { + error_report("Couldn't set kick fd: %s", strerror(errno)); + goto err_set_vring_kick; + } + + return true; + +err_set_vring_kick: + event_notifier_set_handler(&svq->host_notifier, NULL); + + return false; +} + +/* + * Stop shadow virtqueue operation. + * @dev vhost device + * @idx vhost queue index + * @svq Shadow Virtqueue + */ +void vhost_shadow_vq_stop(struct vhost_dev *dev, + unsigned idx, + VhostShadowVirtqueue *svq) +{ + int r =3D vhost_shadow_vq_restore_vdev_host_notifier(dev, idx, svq); + if (unlikely(r < 0)) { + error_report("Couldn't restore vq kick fd: %s", strerror(-r)); + } + + event_notifier_set_handler(&svq->host_notifier, NULL); +} + /* * Creates vhost shadow virtqueue, and instruct vhost device to use the sh= adow * methods and file descriptors. */ VhostShadowVirtqueue *vhost_shadow_vq_new(struct vhost_dev *dev, int idx) { + int vq_idx =3D dev->vq_index + idx; g_autofree VhostShadowVirtqueue *svq =3D g_new0(VhostShadowVirtqueue, = 1); int r; =20 @@ -43,6 +153,7 @@ VhostShadowVirtqueue *vhost_shadow_vq_new(struct vhost_d= ev *dev, int idx) goto err_init_call_notifier; } =20 + svq->vq =3D virtio_get_queue(dev->vdev, vq_idx); return g_steal_pointer(&svq); =20 err_init_call_notifier: diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index 97f1bcfa42..4858a35df6 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -25,6 +25,7 @@ #include "exec/address-spaces.h" #include "hw/virtio/virtio-bus.h" #include "hw/virtio/virtio-access.h" +#include "hw/virtio/vhost-shadow-virtqueue.h" #include "migration/blocker.h" #include "migration/qemu-file-types.h" #include "sysemu/dma.h" @@ -1219,6 +1220,74 @@ static void vhost_virtqueue_stop(struct vhost_dev *d= ev, 0, virtio_queue_get_desc_size(vdev, idx)); } =20 +static int vhost_sw_live_migration_stop(struct vhost_dev *dev) +{ + int idx; + + dev->shadow_vqs_enabled =3D false; + + for (idx =3D 0; idx < dev->nvqs; ++idx) { + vhost_shadow_vq_stop(dev, idx, dev->shadow_vqs[idx]); + vhost_shadow_vq_free(dev->shadow_vqs[idx]); + } + + g_free(dev->shadow_vqs); + dev->shadow_vqs =3D NULL; + return 0; +} + +static int vhost_sw_live_migration_start(struct vhost_dev *dev) +{ + int idx, stop_idx; + + dev->shadow_vqs =3D g_new0(VhostShadowVirtqueue *, dev->nvqs); + for (idx =3D 0; idx < dev->nvqs; ++idx) { + dev->shadow_vqs[idx] =3D vhost_shadow_vq_new(dev, idx); + if (unlikely(dev->shadow_vqs[idx] =3D=3D NULL)) { + goto err_new; + } + } + + dev->shadow_vqs_enabled =3D true; + for (idx =3D 0; idx < dev->nvqs; ++idx) { + bool ok =3D vhost_shadow_vq_start(dev, idx, dev->shadow_vqs[idx]); + if (unlikely(!ok)) { + goto err_start; + } + } + + return 0; + +err_start: + dev->shadow_vqs_enabled =3D false; + for (stop_idx =3D 0; stop_idx < idx; stop_idx++) { + vhost_shadow_vq_stop(dev, idx, dev->shadow_vqs[stop_idx]); + } + +err_new: + for (idx =3D 0; idx < dev->nvqs; ++idx) { + vhost_shadow_vq_free(dev->shadow_vqs[idx]); + } + g_free(dev->shadow_vqs); + + return -1; +} + +static int vhost_sw_live_migration_enable(struct vhost_dev *dev, + bool enable_lm) +{ + int r; + + if (enable_lm =3D=3D dev->shadow_vqs_enabled) { + return 0; + } + + r =3D enable_lm ? vhost_sw_live_migration_start(dev) + : vhost_sw_live_migration_stop(dev); + + return r; +} + static void vhost_eventfd_add(MemoryListener *listener, MemoryRegionSection *section, bool match_data, uint64_t data, EventNotifie= r *e) @@ -1381,6 +1450,7 @@ int vhost_dev_init(struct vhost_dev *hdev, void *opaq= ue, hdev->log =3D NULL; hdev->log_size =3D 0; hdev->log_enabled =3D false; + hdev->shadow_vqs_enabled =3D false; hdev->started =3D false; memory_listener_register(&hdev->memory_listener, &address_space_memory= ); QLIST_INSERT_HEAD(&vhost_devices, hdev, entry); @@ -1484,6 +1554,10 @@ void vhost_dev_disable_notifiers(struct vhost_dev *h= dev, VirtIODevice *vdev) BusState *qbus =3D BUS(qdev_get_parent_bus(DEVICE(vdev))); int i, r; =20 + if (hdev->shadow_vqs_enabled) { + vhost_sw_live_migration_enable(hdev, false); + } + for (i =3D 0; i < hdev->nvqs; ++i) { r =3D virtio_bus_set_host_notifier(VIRTIO_BUS(qbus), hdev->vq_inde= x + i, false); @@ -1798,6 +1872,7 @@ fail_features: void vhost_dev_stop(struct vhost_dev *hdev, VirtIODevice *vdev) { int i; + bool is_shadow_vqs_enabled =3D hdev->shadow_vqs_enabled; =20 /* should only be called after backend is connected */ assert(hdev->vhost_ops); @@ -1805,7 +1880,16 @@ void vhost_dev_stop(struct vhost_dev *hdev, VirtIODe= vice *vdev) if (hdev->vhost_ops->vhost_dev_start) { hdev->vhost_ops->vhost_dev_start(hdev, false); } + if (is_shadow_vqs_enabled) { + /* Shadow virtqueue will be stopped */ + hdev->shadow_vqs_enabled =3D false; + } for (i =3D 0; i < hdev->nvqs; ++i) { + if (is_shadow_vqs_enabled) { + vhost_shadow_vq_stop(hdev, i, hdev->shadow_vqs[i]); + vhost_shadow_vq_free(hdev->shadow_vqs[i]); + } + vhost_virtqueue_stop(hdev, vdev, hdev->vqs + i, @@ -1819,6 +1903,8 @@ void vhost_dev_stop(struct vhost_dev *hdev, VirtIODev= ice *vdev) memory_listener_unregister(&hdev->iommu_listener); } vhost_log_put(hdev, true); + g_free(hdev->shadow_vqs); + hdev->shadow_vqs_enabled =3D false; hdev->started =3D false; hdev->vdev =3D NULL; } @@ -1835,5 +1921,60 @@ int vhost_net_set_backend(struct vhost_dev *hdev, =20 void qmp_x_vhost_enable_shadow_vq(const char *name, bool enable, Error **e= rrp) { - error_setg(errp, "Shadow virtqueue still not implemented"); + struct vhost_dev *hdev, *hdev_err; + VirtIODevice *vdev; + const char *err_cause =3D NULL; + int r; + ErrorClass err_class =3D ERROR_CLASS_GENERIC_ERROR; + + QLIST_FOREACH(hdev, &vhost_devices, entry) { + if (hdev->vdev && 0 =3D=3D strcmp(hdev->vdev->name, name)) { + vdev =3D hdev->vdev; + break; + } + } + + if (!hdev) { + err_class =3D ERROR_CLASS_DEVICE_NOT_FOUND; + err_cause =3D "Device not found"; + goto not_found_err; + } + + for ( ; hdev; hdev =3D QLIST_NEXT(hdev, entry)) { + if (vdev !=3D hdev->vdev) { + continue; + } + + if (!hdev->started) { + err_cause =3D "Device is not started"; + goto err; + } + + r =3D vhost_sw_live_migration_enable(hdev, enable); + if (unlikely(r)) { + err_cause =3D "Error enabling (see monitor)"; + goto err; + } + } + + return; + +err: + QLIST_FOREACH(hdev_err, &vhost_devices, entry) { + if (hdev_err =3D=3D hdev) { + break; + } + + if (vdev !=3D hdev->vdev) { + continue; + } + + vhost_sw_live_migration_enable(hdev, !enable); + } + +not_found_err: + if (err_cause) { + error_set(errp, err_class, + "Can't enable shadow vq on %s: %s", name, err_cause); + } } --=20 2.27.0 From nobody Fri May 17 00:32:42 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1615838133; cv=none; d=zohomail.com; s=zohoarc; b=Zx4s2NbzMuVSeyHsPerTX3rvF86CQKLdLVk4eDFF+MO3KdtxWMbEHWNEeZsMofzu64iRjAdY3lrbtRRE1TYN0sWsPKsfBT8GUvM2ICLIQjvOZvXAfmOPY8wTQP6txLaIeiomM4TuaRGiWRmS/jhZg3d20YiNTzqLD54aA4DX4CY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1615838133; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=IGRHCQ56HK95AEfpMnbCCCYS62wrUulHvDBG+Q8C3Jw=; b=abxj+8Fv3lQglMKTSdAIvNoCR13NrU94xfSaXnR9Z0XohwZ5s3XvfaKdD3RiiFn9NswQkftsCZmmqZ/JG8v3+FwzJrCTdvgQygen/kVsUbfXZYYktjBP451MPeW3vEr4E7IC/QrrKUXRLP6/Q3iA8E4nZPUCxGlCd3En91maOjE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1615838133271188.1014611293623; Mon, 15 Mar 2021 12:55:33 -0700 (PDT) Received: from localhost ([::1]:36906 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lLtJM-00041M-4I for importer@patchew.org; Mon, 15 Mar 2021 15:55:32 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37714) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLtDd-0005V3-Cw for qemu-devel@nongnu.org; Mon, 15 Mar 2021 15:49:37 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:46102) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1lLtDV-00042R-Or for qemu-devel@nongnu.org; Mon, 15 Mar 2021 15:49:35 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-571-V0QFJ42VPNyUhVRcfTJWXQ-1; Mon, 15 Mar 2021 15:49:24 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 6BCAE107ACCD; Mon, 15 Mar 2021 19:49:22 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-112-173.ams2.redhat.com [10.36.112.173]) by smtp.corp.redhat.com (Postfix) with ESMTP id 409651C4; Mon, 15 Mar 2021 19:49:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1615837767; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IGRHCQ56HK95AEfpMnbCCCYS62wrUulHvDBG+Q8C3Jw=; b=CD5HNv/stptUazRmDq78NIBfjrEpuQQKk/vUkRD0xZRsR0Io7CQhjkH0+hgUwG7IcMBMz9 Vu1XWtPBApC8SvAAtl5l12ih7UJqUAVZtpC8UwyMor3JuViuerJWIqh2d38vrtX+VdbuFg zL+3o5XP36UbaIBELXMvUNzW/FCy91M= X-MC-Unique: V0QFJ42VPNyUhVRcfTJWXQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v2 06/13] vhost: Route host->guest notification through shadow virtqueue Date: Mon, 15 Mar 2021 20:48:35 +0100 Message-Id: <20210315194842.277740-7-eperezma@redhat.com> In-Reply-To: <20210315194842.277740-1-eperezma@redhat.com> References: <20210315194842.277740-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.25, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Guru Prasad , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Stefano Garzarella , Michael Lilja , Jim Harford , Rob Miller Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) On one hand it uses a mutex to synchronize guest masking with SVQ start and stop, because otherwise guest mask could race with the SVQ stop code, sending an incorrect call notifier to vhost device. This would prevent further communication. On the other hand it needs to add an event to synchronize guest unmasking with call handling. Not doing that way could cause the guest to receive notifications after its unmask call. This could be done through the mutex but the event solution is cheaper for the buffer forwarding. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.h | 3 + include/hw/virtio/vhost.h | 1 + hw/virtio/vhost-shadow-virtqueue.c | 127 +++++++++++++++++++++++++++++ hw/virtio/vhost.c | 29 ++++++- 4 files changed, 157 insertions(+), 3 deletions(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-vi= rtqueue.h index c891c6510d..2ca4b92b12 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -17,6 +17,9 @@ =20 typedef struct VhostShadowVirtqueue VhostShadowVirtqueue; =20 +void vhost_shadow_vq_mask(VhostShadowVirtqueue *svq, EventNotifier *masked= ); +void vhost_shadow_vq_unmask(VhostShadowVirtqueue *svq); + bool vhost_shadow_vq_start(struct vhost_dev *dev, unsigned idx, VhostShadowVirtqueue *svq); diff --git a/include/hw/virtio/vhost.h b/include/hw/virtio/vhost.h index 7ffdf9aea0..2f556bd3d5 100644 --- a/include/hw/virtio/vhost.h +++ b/include/hw/virtio/vhost.h @@ -29,6 +29,7 @@ struct vhost_virtqueue { unsigned long long used_phys; unsigned used_size; bool notifier_is_masked; + QemuRecMutex masked_mutex; EventNotifier masked_notifier; struct vhost_dev *dev; }; diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index 3e43399e9c..8f6ffa729a 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -32,8 +32,22 @@ typedef struct VhostShadowVirtqueue { */ EventNotifier host_notifier; =20 + /* (Possible) masked notifier */ + struct { + EventNotifier *n; + + /* + * Event to confirm unmasking. + * set when the masked notifier has no uses + */ + QemuEvent is_free; + } masked_notifier; + /* Virtio queue shadowing */ VirtQueue *vq; + + /* Virtio device */ + VirtIODevice *vdev; } VhostShadowVirtqueue; =20 /* Forward guest notifications */ @@ -49,6 +63,70 @@ static void vhost_handle_guest_kick(EventNotifier *n) event_notifier_set(&svq->kick_notifier); } =20 +/* Forward vhost notifications */ +static void vhost_shadow_vq_handle_call_no_test(EventNotifier *n) +{ + VhostShadowVirtqueue *svq =3D container_of(n, VhostShadowVirtqueue, + call_notifier); + EventNotifier *masked_notifier; + + /* Signal start of using masked notifier */ + qemu_event_reset(&svq->masked_notifier.is_free); + masked_notifier =3D qatomic_load_acquire(&svq->masked_notifier.n); + if (!masked_notifier) { + qemu_event_set(&svq->masked_notifier.is_free); + } + + if (!masked_notifier) { + unsigned n =3D virtio_get_queue_index(svq->vq); + virtio_queue_invalidate_signalled_used(svq->vdev, n); + virtio_notify_irqfd(svq->vdev, svq->vq); + } else { + event_notifier_set(svq->masked_notifier.n); + } + + if (masked_notifier) { + /* Signal not using it anymore */ + qemu_event_set(&svq->masked_notifier.is_free); + } +} + +static void vhost_shadow_vq_handle_call(EventNotifier *n) +{ + + if (likely(event_notifier_test_and_clear(n))) { + vhost_shadow_vq_handle_call_no_test(n); + } +} + +/* + * Mask the shadow virtqueue. + * + * It can be called from a guest masking vmexit or shadow virtqueue start + * through QMP. + * + * @vq Shadow virtqueue + * @masked Masked notifier to signal instead of guest + */ +void vhost_shadow_vq_mask(VhostShadowVirtqueue *svq, EventNotifier *masked) +{ + qatomic_store_release(&svq->masked_notifier.n, masked); +} + +/* + * Unmask the shadow virtqueue. + * + * It can be called from a guest unmasking vmexit or shadow virtqueue start + * through QMP. + * + * @vq Shadow virtqueue + */ +void vhost_shadow_vq_unmask(VhostShadowVirtqueue *svq) +{ + qatomic_store_release(&svq->masked_notifier.n, NULL); + qemu_event_wait(&svq->masked_notifier.is_free); +} + /* * Restore the vhost guest to host notifier, i.e., disables svq effect. */ @@ -103,8 +181,39 @@ bool vhost_shadow_vq_start(struct vhost_dev *dev, goto err_set_vring_kick; } =20 + /* Set vhost call */ + file.fd =3D event_notifier_get_fd(&svq->call_notifier), + r =3D dev->vhost_ops->vhost_set_vring_call(dev, &file); + if (unlikely(r !=3D 0)) { + error_report("Couldn't set call fd: %s", strerror(errno)); + goto err_set_vring_call; + } + + + /* + * Lock to avoid a race condition between guest setting masked status = and + * us. + */ + QEMU_LOCK_GUARD(&dev->vqs[idx].masked_mutex); + /* Set shadow vq -> guest notifier */ + assert(dev->shadow_vqs_enabled); + vhost_virtqueue_mask(dev, dev->vdev, dev->vq_index + idx, + dev->vqs[idx].notifier_is_masked); + + if (dev->vqs[idx].notifier_is_masked && + event_notifier_test_and_clear(&dev->vqs[idx].masked_notifie= r)) { + /* Check for pending notifications from the device */ + vhost_shadow_vq_handle_call_no_test(&svq->call_notifier); + } + return true; =20 +err_set_vring_call: + r =3D vhost_shadow_vq_restore_vdev_host_notifier(dev, idx, svq); + if (unlikely(r < 0)) { + error_report("Couldn't restore vq kick fd: %s", strerror(-r)); + } + err_set_vring_kick: event_notifier_set_handler(&svq->host_notifier, NULL); =20 @@ -126,7 +235,19 @@ void vhost_shadow_vq_stop(struct vhost_dev *dev, error_report("Couldn't restore vq kick fd: %s", strerror(-r)); } =20 + assert(!dev->shadow_vqs_enabled); + event_notifier_set_handler(&svq->host_notifier, NULL); + + /* + * Lock to avoid a race condition between guest setting masked status = and + * us. + */ + QEMU_LOCK_GUARD(&dev->vqs[idx].masked_mutex); + + /* Restore vhost call */ + vhost_virtqueue_mask(dev, dev->vdev, dev->vq_index + idx, + dev->vqs[idx].notifier_is_masked); } =20 /* @@ -154,6 +275,10 @@ VhostShadowVirtqueue *vhost_shadow_vq_new(struct vhost= _dev *dev, int idx) } =20 svq->vq =3D virtio_get_queue(dev->vdev, vq_idx); + svq->vdev =3D dev->vdev; + event_notifier_set_handler(&svq->call_notifier, + vhost_shadow_vq_handle_call); + qemu_event_init(&svq->masked_notifier.is_free, true); return g_steal_pointer(&svq); =20 err_init_call_notifier: @@ -168,7 +293,9 @@ err_init_kick_notifier: */ void vhost_shadow_vq_free(VhostShadowVirtqueue *vq) { + qemu_event_destroy(&vq->masked_notifier.is_free); event_notifier_cleanup(&vq->kick_notifier); + event_notifier_set_handler(&vq->call_notifier, NULL); event_notifier_cleanup(&vq->call_notifier); g_free(vq); } diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index 4858a35df6..eab3e334f2 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -1224,7 +1224,8 @@ static int vhost_sw_live_migration_stop(struct vhost_= dev *dev) { int idx; =20 - dev->shadow_vqs_enabled =3D false; + /* Can be read by vhost_virtqueue_mask, from vm exit */ + qatomic_store_release(&dev->shadow_vqs_enabled, false); =20 for (idx =3D 0; idx < dev->nvqs; ++idx) { vhost_shadow_vq_stop(dev, idx, dev->shadow_vqs[idx]); @@ -1248,7 +1249,8 @@ static int vhost_sw_live_migration_start(struct vhost= _dev *dev) } } =20 - dev->shadow_vqs_enabled =3D true; + /* Can be read by vhost_virtqueue_mask, from vm exit */ + qatomic_store_release(&dev->shadow_vqs_enabled, true); for (idx =3D 0; idx < dev->nvqs; ++idx) { bool ok =3D vhost_shadow_vq_start(dev, idx, dev->shadow_vqs[idx]); if (unlikely(!ok)) { @@ -1259,7 +1261,7 @@ static int vhost_sw_live_migration_start(struct vhost= _dev *dev) return 0; =20 err_start: - dev->shadow_vqs_enabled =3D false; + qatomic_store_release(&dev->shadow_vqs_enabled, false); for (stop_idx =3D 0; stop_idx < idx; stop_idx++) { vhost_shadow_vq_stop(dev, idx, dev->shadow_vqs[stop_idx]); } @@ -1343,6 +1345,7 @@ static int vhost_virtqueue_init(struct vhost_dev *dev, goto fail_call; } =20 + qemu_rec_mutex_init(&vq->masked_mutex); vq->dev =3D dev; =20 return 0; @@ -1353,6 +1356,7 @@ fail_call: =20 static void vhost_virtqueue_cleanup(struct vhost_virtqueue *vq) { + qemu_rec_mutex_destroy(&vq->masked_mutex); event_notifier_cleanup(&vq->masked_notifier); } =20 @@ -1591,6 +1595,25 @@ void vhost_virtqueue_mask(struct vhost_dev *hdev, Vi= rtIODevice *vdev, int n, /* should only be called after backend is connected */ assert(hdev->vhost_ops); =20 + /* Avoid race condition with shadow virtqueue stop/start */ + QEMU_LOCK_GUARD(&hdev->vqs[index].masked_mutex); + + /* Set by QMP thread, so using acquire semantics */ + if (qatomic_load_acquire(&hdev->shadow_vqs_enabled)) { + if (mask) { + vhost_shadow_vq_mask(hdev->shadow_vqs[index], + &hdev->vqs[index].masked_notifier); + } else { + vhost_shadow_vq_unmask(hdev->shadow_vqs[index]); + } + + /* + * Vhost call fd must remain the same since shadow vq is not polli= ng + * for changes + */ + return; + } + if (mask) { assert(vdev->use_guest_notifier_mask); file.fd =3D event_notifier_get_fd(&hdev->vqs[index].masked_notifie= r); --=20 2.27.0 From nobody Fri May 17 00:32:42 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1615838028; cv=none; d=zohomail.com; s=zohoarc; b=APxCRtQmMRg4fcljNZbLC9vN//5Pd6KWOajvv2C3ciNN3YyPgfM8dSL2GDz0TrzdpmIOwKVMOY22XDtDHNTxiLnoNXKCmfQyi5pytOx8Es0+/6H71wnWP7E6YyPmmTrTY0gTVqIMf4e/kvE60GWRce+YBL+Oqxo3gRyTk2Iiz4U= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1615838028; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=AbWIH/kkEto2LZLZZe4BV9lZdunPIm7cSJBarpqzcYE=; b=Wrgg267CAhjUm/xB7uA/94xzuMv+69zMEmMwas9QSEb6uae7jVVhjQZVxYik0EXNLdtYf6Ewk6yTjYodXIMTuM907rvCiffJLQGgey2yJZbrvX8ARsmiUkgvyCASgf32Rlek5bMvt7GTuBOdHparPfcbsli+mTacuJjNKPC+bCM= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 16158380282231010.943992416484; Mon, 15 Mar 2021 12:53:48 -0700 (PDT) Received: from localhost ([::1]:60292 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lLtHe-00021t-8r for importer@patchew.org; Mon, 15 Mar 2021 15:53:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37730) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLtDf-0005X2-ID for qemu-devel@nongnu.org; Mon, 15 Mar 2021 15:49:41 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:48261) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1lLtDd-00045p-4p for qemu-devel@nongnu.org; Mon, 15 Mar 2021 15:49:39 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-564-aOCnnCaZOrCh2KrLGTjMAw-1; Mon, 15 Mar 2021 15:49:33 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 400F4801817; Mon, 15 Mar 2021 19:49:32 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-112-173.ams2.redhat.com [10.36.112.173]) by smtp.corp.redhat.com (Postfix) with ESMTP id C23E85F706; Mon, 15 Mar 2021 19:49:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1615837775; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=AbWIH/kkEto2LZLZZe4BV9lZdunPIm7cSJBarpqzcYE=; b=HPf3O8suTSvUPppCF40/PlgdgSqsw50ijNZwv6bLGyZyBW3VVCWaPiCRroF+AmJtBrSUsE i6p0ESAf1/Z0fp/cX2SzJdUIISqtfMl82fPUCA8BHn7m+JXTeeDDpSE7VknOC7jrv8njhD E/9KnWXyRc9sCYuqwyJ7/zbc0Hm6I34= X-MC-Unique: aOCnnCaZOrCh2KrLGTjMAw-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v2 07/13] vhost: Avoid re-set masked notifier in shadow vq Date: Mon, 15 Mar 2021 20:48:36 +0100 Message-Id: <20210315194842.277740-8-eperezma@redhat.com> In-Reply-To: <20210315194842.277740-1-eperezma@redhat.com> References: <20210315194842.277740-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=63.128.21.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.25, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Guru Prasad , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Stefano Garzarella , Michael Lilja , Jim Harford , Rob Miller Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Since all the shadow virtqueue device is done in software, we can avoid the write syscall. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.c | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index 8f6ffa729a..b6bab438d6 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -41,6 +41,9 @@ typedef struct VhostShadowVirtqueue { * set when the masked notifier has no uses */ QemuEvent is_free; + + /* Avoid re-sending signals */ + bool signaled; } masked_notifier; =20 /* Virtio queue shadowing */ @@ -81,7 +84,8 @@ static void vhost_shadow_vq_handle_call_no_test(EventNoti= fier *n) unsigned n =3D virtio_get_queue_index(svq->vq); virtio_queue_invalidate_signalled_used(svq->vdev, n); virtio_notify_irqfd(svq->vdev, svq->vq); - } else { + } else if (!svq->masked_notifier.signaled) { + svq->masked_notifier.signaled =3D true; event_notifier_set(svq->masked_notifier.n); } =20 @@ -110,6 +114,7 @@ static void vhost_shadow_vq_handle_call(EventNotifier *= n) */ void vhost_shadow_vq_mask(VhostShadowVirtqueue *svq, EventNotifier *masked) { + svq->masked_notifier.signaled =3D false; qatomic_store_release(&svq->masked_notifier.n, masked); } =20 --=20 2.27.0 From nobody Fri May 17 00:32:42 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1615838248; cv=none; d=zohomail.com; s=zohoarc; b=L2nGZa4UmVLShd/S3xaNInnu7m1EFB48zFInMB19OKSO8FA8i+48p28uRB1KHKEFEfInQFSTPs5DqDKns+sHiOe1XChh7C2iETO83Np2uIm7fmIgJNQUrjIfHReDeUOctVIediffMri83ohTakKdHndt2adQ1sDln5xD9Kp49kE= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1615838248; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=W7AqImQzQRN4d8y77E4TYLPjjtOJrVKWXw8C5p9lYeM=; b=MBnc0suKTlYP6SaXlqwOJSMtmgN5mylOBSW785Arzy/8sTZ2d2owvTgMBQS7A4ZnJ7i+RWvi5ZsA3tkvpkw2APuS7nV/YEr6ucfTGo7DfH0jPa47thlIxZ1PGuokXzTX1VlA1k1WOGrk90p2kQFZDog7p4U+kURCajEJUL05qhk= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1615838248728234.027214726274; Mon, 15 Mar 2021 12:57:28 -0700 (PDT) Received: from localhost ([::1]:43252 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lLtLD-0006np-Gw for importer@patchew.org; Mon, 15 Mar 2021 15:57:27 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37754) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLtDh-0005Y9-KO for qemu-devel@nongnu.org; Mon, 15 Mar 2021 15:49:41 -0400 Received: from us-smtp-delivery-124.mimecast.com ([63.128.21.124]:22204) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1lLtDf-00047L-TQ for qemu-devel@nongnu.org; Mon, 15 Mar 2021 15:49:41 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-312-D5KcFQtVPx6RnFXFAluDDw-1; Mon, 15 Mar 2021 15:49:37 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id E5B50801597; Mon, 15 Mar 2021 19:49:35 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-112-173.ams2.redhat.com [10.36.112.173]) by smtp.corp.redhat.com (Postfix) with ESMTP id 947895F9B8; Mon, 15 Mar 2021 19:49:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1615837779; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=W7AqImQzQRN4d8y77E4TYLPjjtOJrVKWXw8C5p9lYeM=; b=XsOVZ05snOkeWWtUiF+IdP+9AXYdEQafvolcxSf6EJrc8G0C4O1kjiop/YC/xYZNCz5Ks5 WxrerYdkEvpARLDFl24JdCDyNgoYiAHsokL8DcU+94lwpf+5NzriV2zJLNR6JME3MTfViT QPzAb53f+s9JynlWSCCULru0CCnBBTM= X-MC-Unique: D5KcFQtVPx6RnFXFAluDDw-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v2 08/13] virtio: Add vhost_shadow_vq_get_vring_addr Date: Mon, 15 Mar 2021 20:48:37 +0100 Message-Id: <20210315194842.277740-9-eperezma@redhat.com> In-Reply-To: <20210315194842.277740-1-eperezma@redhat.com> References: <20210315194842.277740-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=63.128.21.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.25, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Guru Prasad , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Stefano Garzarella , Michael Lilja , Jim Harford , Rob Miller Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) It reports the shadow virtqueue address from qemu virtual address space Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.h | 2 ++ hw/virtio/vhost-shadow-virtqueue.c | 24 +++++++++++++++++++++++- 2 files changed, 25 insertions(+), 1 deletion(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.h b/hw/virtio/vhost-shadow-vi= rtqueue.h index 2ca4b92b12..d82c35bccf 100644 --- a/hw/virtio/vhost-shadow-virtqueue.h +++ b/hw/virtio/vhost-shadow-virtqueue.h @@ -19,6 +19,8 @@ typedef struct VhostShadowVirtqueue VhostShadowVirtqueue; =20 void vhost_shadow_vq_mask(VhostShadowVirtqueue *svq, EventNotifier *masked= ); void vhost_shadow_vq_unmask(VhostShadowVirtqueue *svq); +void vhost_shadow_vq_get_vring_addr(const VhostShadowVirtqueue *svq, + struct vhost_vring_addr *addr); =20 bool vhost_shadow_vq_start(struct vhost_dev *dev, unsigned idx, diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index b6bab438d6..1460d1d5d1 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -17,6 +17,9 @@ =20 /* Shadow virtqueue to relay notifications */ typedef struct VhostShadowVirtqueue { + /* Shadow vring */ + struct vring vring; + /* Shadow kick notifier, sent to vhost */ EventNotifier kick_notifier; /* Shadow call notifier, sent to vhost */ @@ -51,6 +54,9 @@ typedef struct VhostShadowVirtqueue { =20 /* Virtio device */ VirtIODevice *vdev; + + /* Descriptors copied from guest */ + vring_desc_t descs[]; } VhostShadowVirtqueue; =20 /* Forward guest notifications */ @@ -132,6 +138,19 @@ void vhost_shadow_vq_unmask(VhostShadowVirtqueue *svq) qemu_event_wait(&svq->masked_notifier.is_free); } =20 +/* + * Get the shadow vq vring address. + * @svq Shadow virtqueue + * @addr Destination to store address + */ +void vhost_shadow_vq_get_vring_addr(const VhostShadowVirtqueue *svq, + struct vhost_vring_addr *addr) +{ + addr->desc_user_addr =3D (uint64_t)svq->vring.desc; + addr->avail_user_addr =3D (uint64_t)svq->vring.avail; + addr->used_user_addr =3D (uint64_t)svq->vring.used; +} + /* * Restore the vhost guest to host notifier, i.e., disables svq effect. */ @@ -262,7 +281,9 @@ void vhost_shadow_vq_stop(struct vhost_dev *dev, VhostShadowVirtqueue *vhost_shadow_vq_new(struct vhost_dev *dev, int idx) { int vq_idx =3D dev->vq_index + idx; - g_autofree VhostShadowVirtqueue *svq =3D g_new0(VhostShadowVirtqueue, = 1); + unsigned num =3D virtio_queue_get_num(dev->vdev, vq_idx); + size_t ring_size =3D vring_size(num, VRING_DESC_ALIGN_SIZE); + g_autofree VhostShadowVirtqueue *svq =3D g_malloc0(sizeof(*svq) + ring= _size); int r; =20 r =3D event_notifier_init(&svq->kick_notifier, 0); @@ -279,6 +300,7 @@ VhostShadowVirtqueue *vhost_shadow_vq_new(struct vhost_= dev *dev, int idx) goto err_init_call_notifier; } =20 + vring_init(&svq->vring, num, svq->descs, VRING_DESC_ALIGN_SIZE); svq->vq =3D virtio_get_queue(dev->vdev, vq_idx); svq->vdev =3D dev->vdev; event_notifier_set_handler(&svq->call_notifier, --=20 2.27.0 From nobody Fri May 17 00:32:42 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1615838151; cv=none; d=zohomail.com; s=zohoarc; b=mcmLYT2ExbwW86oeZtw6oLmj2Y9fyTHvixXzANk5AKtPn7vCYHP/9acZvsWYmLk+q6bHivtBv2umOdycjwPzJDNC+cYbHeAnn3OApgLQocze26o0KIeu3aQO7fTQ40hlNEr5JPnJ+IsAj0H4MHni0phr7v1dDjHFqFUuHFyfDGM= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1615838151; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=vd99cAaSU6abm6dJJrsG/VaQ4xBejc6I+hFkbFVbYYg=; b=SZAfmXX/exVx5kQIIWJN50op3AmDIad78+Fb+sKjthJjHzxsp67BUAqqAshi1xIhfg04ZaNsP88HrIlqGIClhbVdl0hjrp3+5HED7o/nc+yt3MJHahzfjfOqMyYSqItfpVfAJjFZqt5x5IqyEpOe+RVg1Lhdy7qhdqDl0id/J80= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 16158381519350.06989692897059285; Mon, 15 Mar 2021 12:55:51 -0700 (PDT) Received: from localhost ([::1]:38354 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lLtJe-0004fH-Qq for importer@patchew.org; Mon, 15 Mar 2021 15:55:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37770) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLtDl-0005gE-JI for qemu-devel@nongnu.org; Mon, 15 Mar 2021 15:49:45 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:56357) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1lLtDj-00048q-TM for qemu-devel@nongnu.org; Mon, 15 Mar 2021 15:49:45 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-581-12HN0WNgNReWuLyaKBKEAg-1; Mon, 15 Mar 2021 15:49:41 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 95DC392502; Mon, 15 Mar 2021 19:49:39 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-112-173.ams2.redhat.com [10.36.112.173]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4614A1C4; Mon, 15 Mar 2021 19:49:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1615837783; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=vd99cAaSU6abm6dJJrsG/VaQ4xBejc6I+hFkbFVbYYg=; b=YFQ7U0gfK1hGEMSb6raGMb2Vceekt6XNN3L4KhmpZUuBJkkRM7zR3lISZnBRgkOAM5YxTV h6OV8GNNX98YJIYtSaoZ46FpXAvxo6tqhk9zzsaZwCFotPXv/yi6V2MbNaQOkpUKRXN4Ub 2yYD9z9mM3G7GxPNreXUIgdBxZjj/0Q= X-MC-Unique: 12HN0WNgNReWuLyaKBKEAg-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v2 09/13] virtio: Add virtio_queue_full Date: Mon, 15 Mar 2021 20:48:38 +0100 Message-Id: <20210315194842.277740-10-eperezma@redhat.com> In-Reply-To: <20210315194842.277740-1-eperezma@redhat.com> References: <20210315194842.277740-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.25, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Guru Prasad , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Stefano Garzarella , Michael Lilja , Jim Harford , Rob Miller Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Check if all descriptors of the queue are available. In other words, is the complete opposite of virtio_queue_empty: If the queue is full, the driver cannot transfer more buffers to the device until the latter make some as used. In Shadow vq this situation happens with the correct guest network driver, since the rx queue is filled for the device to write. Since Shadow Virtqueue forward the available ones blindly, it will call the driver forever for them, reaching the point where no more descriptors are available. While a straightforward solution is to keep the count of them in SVQ, this specific issue is the only need for that counter. Exposing this check helps to keep the SVQ simpler storing as little status as possible. Signed-off-by: Eugenio P=C3=A9rez --- include/hw/virtio/virtio.h | 2 ++ hw/virtio/virtio.c | 18 ++++++++++++++++-- 2 files changed, 18 insertions(+), 2 deletions(-) diff --git a/include/hw/virtio/virtio.h b/include/hw/virtio/virtio.h index c2c7cee993..899c5e3506 100644 --- a/include/hw/virtio/virtio.h +++ b/include/hw/virtio/virtio.h @@ -232,6 +232,8 @@ int virtio_queue_ready(VirtQueue *vq); =20 int virtio_queue_empty(VirtQueue *vq); =20 +bool virtio_queue_full(const VirtQueue *vq); + /* Host binding interface. */ =20 uint32_t virtio_config_readb(VirtIODevice *vdev, uint32_t addr); diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c index a86b3f9c26..e9a4d9ffae 100644 --- a/hw/virtio/virtio.c +++ b/hw/virtio/virtio.c @@ -670,6 +670,20 @@ int virtio_queue_empty(VirtQueue *vq) } } =20 +/* + * virtio_queue_full: + * @vq The #VirtQueue + * + * Check if all descriptors of the queue are available. In other words, is= the + * complete opposite of virtio_queue_empty: If the queue is full, the driv= er + * cannot transfer more buffers to the device until the latter make some as + * used. + */ +bool virtio_queue_full(const VirtQueue *vq) +{ + return vq->inuse >=3D vq->vring.num; +} + static void virtqueue_unmap_sg(VirtQueue *vq, const VirtQueueElement *elem, unsigned int len) { @@ -1439,7 +1453,7 @@ static void *virtqueue_split_pop(VirtQueue *vq, size_= t sz) =20 max =3D vq->vring.num; =20 - if (vq->inuse >=3D vq->vring.num) { + if (unlikely(virtio_queue_full(vq))) { virtio_error(vdev, "Virtqueue size exceeded"); goto done; } @@ -1574,7 +1588,7 @@ static void *virtqueue_packed_pop(VirtQueue *vq, size= _t sz) =20 max =3D vq->vring.num; =20 - if (vq->inuse >=3D vq->vring.num) { + if (unlikely(virtio_queue_full(vq))) { virtio_error(vdev, "Virtqueue size exceeded"); goto done; } --=20 2.27.0 From nobody Fri May 17 00:32:42 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1615837907; cv=none; d=zohomail.com; s=zohoarc; b=fNr3lVyiMYHJ4TluETkgHOrLsK6z6ueIrskC4jI9qq5wmm5QQ6+ABmdtwvCwHvIMDqBjRKjp/8gHWHg5bYHC1uOsjsYBgYiXmvrqKdJYNhV99/qgdWkKHptIP5IeEZ6SjphYamUP5O7WCKGA45ghM4Z3iaMHzVnor0BtmnveNPc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1615837907; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=YXIH0j/rH7VLMaRzbXnWIbuRHR0Ubr2KYwFxlTMmxC8=; b=LCZf1oIqv7ZL9bqLP/mHB+V7EYezok5OuQlYMy3pU8MjEYy3uzlf24istvbfMJG7kdB02ovjXPMByHRN+4DCHdktSHWZCXAvTkKXxGNrgR2+IkGZNMi74Hkihey4yLLPiENSzjuPbrFiHSOT7YEgzrZybQku6J52KBc4B5UodlU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1615837907823398.3458806747542; Mon, 15 Mar 2021 12:51:47 -0700 (PDT) Received: from localhost ([::1]:53262 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lLtFi-0007YL-Oo for importer@patchew.org; Mon, 15 Mar 2021 15:51:46 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37802) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLtDp-0005kY-IX for qemu-devel@nongnu.org; Mon, 15 Mar 2021 15:49:49 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:49928) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1lLtDo-0004AA-19 for qemu-devel@nongnu.org; Mon, 15 Mar 2021 15:49:49 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-297-fd_d76CYOUSCL_43q9nttA-1; Mon, 15 Mar 2021 15:49:45 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 484298030B5; Mon, 15 Mar 2021 19:49:43 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-112-173.ams2.redhat.com [10.36.112.173]) by smtp.corp.redhat.com (Postfix) with ESMTP id EA0D35C5E0; Mon, 15 Mar 2021 19:49:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1615837786; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=YXIH0j/rH7VLMaRzbXnWIbuRHR0Ubr2KYwFxlTMmxC8=; b=S+uFn7kfTx+A1aQatN4ixV72aUd2SDcRuan+BglH0qtQltbcBIl2UFn0IfRXMvE0kGNdRB 2bCbmzsLRrr/0CNljkdQ4pH4xEqpSCsX+HHAV4DQ9VpuLGDckhMX4XeeOYUvU8mAahF/9n OQTKarWD+WdDjJnV8MZE3pFyc5TPz7U= X-MC-Unique: fd_d76CYOUSCL_43q9nttA-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v2 10/13] vhost: add vhost_kernel_set_vring_enable Date: Mon, 15 Mar 2021 20:48:39 +0100 Message-Id: <20210315194842.277740-11-eperezma@redhat.com> In-Reply-To: <20210315194842.277740-1-eperezma@redhat.com> References: <20210315194842.277740-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.25, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Guru Prasad , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Stefano Garzarella , Michael Lilja , Jim Harford , Rob Miller Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) This method is already present in vhost-user. This commit adapts it to vhost-net, so SVQ can use. vhost_kernel_set_enable stops the device, so qemu can ask for its status (next available idx the device was going to consume). When SVQ starts it can resume consuming the guest's driver ring, without notice from the latter. Not stopping the device before of the swapping could imply that it process more buffers than reported, what would duplicate the device action. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-backend.c | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/hw/virtio/vhost-backend.c b/hw/virtio/vhost-backend.c index 31b33bde37..1ac5c574a9 100644 --- a/hw/virtio/vhost-backend.c +++ b/hw/virtio/vhost-backend.c @@ -201,6 +201,34 @@ static int vhost_kernel_get_vq_index(struct vhost_dev = *dev, int idx) return idx - dev->vq_index; } =20 +static int vhost_kernel_set_vq_enable(struct vhost_dev *dev, unsigned idx, + bool enable) +{ + struct vhost_vring_file file =3D { + .index =3D idx, + }; + + if (!enable) { + file.fd =3D -1; /* Pass -1 to unbind from file. */ + } else { + struct vhost_net *vn_dev =3D container_of(dev, struct vhost_net, d= ev); + file.fd =3D vn_dev->backend; + } + + return vhost_kernel_net_set_backend(dev, &file); +} + +static int vhost_kernel_set_vring_enable(struct vhost_dev *dev, int enable) +{ + int i; + + for (i =3D 0; i < dev->nvqs; ++i) { + vhost_kernel_set_vq_enable(dev, i, enable); + } + + return 0; +} + #ifdef CONFIG_VHOST_VSOCK static int vhost_kernel_vsock_set_guest_cid(struct vhost_dev *dev, uint64_t guest_cid) @@ -317,6 +345,7 @@ static const VhostOps kernel_ops =3D { .vhost_set_owner =3D vhost_kernel_set_owner, .vhost_reset_device =3D vhost_kernel_reset_device, .vhost_get_vq_index =3D vhost_kernel_get_vq_index, + .vhost_set_vring_enable =3D vhost_kernel_set_vring_enable, #ifdef CONFIG_VHOST_VSOCK .vhost_vsock_set_guest_cid =3D vhost_kernel_vsock_set_guest_cid, .vhost_vsock_set_running =3D vhost_kernel_vsock_set_running, --=20 2.27.0 From nobody Fri May 17 00:32:42 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1615838248; cv=none; d=zohomail.com; s=zohoarc; b=DHbaV2wQj3cLXzxYkfnhw0Pi1KQvSgb/IDXovNv2otjKCUoJpdx7nikJZDxrHxswucJdj98VxV+TCm5QNJzUgDO9vMo7sVGb8NkAnlnLtR1fXYAVwvB4GLtZNmIqJqRyR9wZaqp+vuh/MjaRnubr7Z5qJLkj5HtoZJqunKQA54o= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1615838248; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=u9cqwMQ+rFXawpf7AL1LTcnF7tvSq1hGLTxiqTMVF4Q=; b=i8AHRquY/u7ZVUYmOFRJ7o7BLlzHOSLw4OkQ/MgwWA3n/prlNqxq9IdA6gZSqllJF6mjYb706yg+Lsr6Bw0GR9sy79logGjI2uneV3lpn3rn8orrvPhTJ9kxW2blG38YYOeySa1puvVjG7SkHnV6/1JXn1i1XKntw6l33U9Y1nA= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 16158382489581005.9649224186012; Mon, 15 Mar 2021 12:57:28 -0700 (PDT) Received: from localhost ([::1]:43040 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lLtLB-0006iV-E3 for importer@patchew.org; Mon, 15 Mar 2021 15:57:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37836) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLtDt-0005oO-VS for qemu-devel@nongnu.org; Mon, 15 Mar 2021 15:49:54 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:26015) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1lLtDr-0004Bm-EN for qemu-devel@nongnu.org; Mon, 15 Mar 2021 15:49:53 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-441-tZnxKVtWNuGC17YXl0FjGQ-1; Mon, 15 Mar 2021 15:49:49 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id 2F262108BD07; Mon, 15 Mar 2021 19:49:47 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-112-173.ams2.redhat.com [10.36.112.173]) by smtp.corp.redhat.com (Postfix) with ESMTP id 9D7125F9B8; Mon, 15 Mar 2021 19:49:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1615837790; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=u9cqwMQ+rFXawpf7AL1LTcnF7tvSq1hGLTxiqTMVF4Q=; b=LasuY81j4WWfl+keqSN3HD8ZiBSoqEz/D/N6zin7+k8a+ZSagmWRIEsZlnj0+0zTrPR+9T xeYHEJPaUjpLESbRSFPnEDx4u4RvXGn4n+MPE2AFrazdyGiz/zjWVf1WU/SgntL2mELs8M kZa6JuOJJ6uPUA+HbnZpm+EnFY7t+D0= X-MC-Unique: tZnxKVtWNuGC17YXl0FjGQ-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v2 11/13] vhost: Shadow virtqueue buffers forwarding Date: Mon, 15 Mar 2021 20:48:40 +0100 Message-Id: <20210315194842.277740-12-eperezma@redhat.com> In-Reply-To: <20210315194842.277740-1-eperezma@redhat.com> References: <20210315194842.277740-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.25, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Guru Prasad , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Stefano Garzarella , Michael Lilja , Jim Harford , Rob Miller Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Initial version of shadow virtqueue that actually forward buffers. It reuses the VirtQueue code for the device part. The driver part is based on Linux's virtio_ring driver, but with stripped functionality and optimizations so it's easier to review. These will be added in later commits. Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.c | 212 +++++++++++++++++++++++++++-- hw/virtio/vhost.c | 113 ++++++++++++++- 2 files changed, 312 insertions(+), 13 deletions(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index 1460d1d5d1..68ed0f2740 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -9,6 +9,7 @@ =20 #include "hw/virtio/vhost-shadow-virtqueue.h" #include "hw/virtio/vhost.h" +#include "hw/virtio/virtio-access.h" =20 #include "standard-headers/linux/vhost_types.h" =20 @@ -55,11 +56,96 @@ typedef struct VhostShadowVirtqueue { /* Virtio device */ VirtIODevice *vdev; =20 + /* Map for returning guest's descriptors */ + VirtQueueElement **ring_id_maps; + + /* Next head to expose to device */ + uint16_t avail_idx_shadow; + + /* Next free descriptor */ + uint16_t free_head; + + /* Last seen used idx */ + uint16_t shadow_used_idx; + + /* Next head to consume from device */ + uint16_t used_idx; + /* Descriptors copied from guest */ vring_desc_t descs[]; } VhostShadowVirtqueue; =20 -/* Forward guest notifications */ +static void vhost_vring_write_descs(VhostShadowVirtqueue *svq, + const struct iovec *iovec, + size_t num, bool more_descs, bool writ= e) +{ + uint16_t i =3D svq->free_head, last =3D svq->free_head; + unsigned n; + uint16_t flags =3D write ? virtio_tswap16(svq->vdev, VRING_DESC_F_WRIT= E) : 0; + vring_desc_t *descs =3D svq->vring.desc; + + if (num =3D=3D 0) { + return; + } + + for (n =3D 0; n < num; n++) { + if (more_descs || (n + 1 < num)) { + descs[i].flags =3D flags | virtio_tswap16(svq->vdev, + VRING_DESC_F_NEXT); + } else { + descs[i].flags =3D flags; + } + descs[i].addr =3D virtio_tswap64(svq->vdev, (hwaddr)iovec[n].iov_b= ase); + descs[i].len =3D virtio_tswap32(svq->vdev, iovec[n].iov_len); + + last =3D i; + i =3D virtio_tswap16(svq->vdev, descs[i].next); + } + + svq->free_head =3D virtio_tswap16(svq->vdev, descs[last].next); +} + +static unsigned vhost_shadow_vq_add_split(VhostShadowVirtqueue *svq, + VirtQueueElement *elem) +{ + int head; + unsigned avail_idx; + vring_avail_t *avail =3D svq->vring.avail; + + head =3D svq->free_head; + + /* We need some descriptors here */ + assert(elem->out_num || elem->in_num); + + vhost_vring_write_descs(svq, elem->out_sg, elem->out_num, + elem->in_num > 0, false); + vhost_vring_write_descs(svq, elem->in_sg, elem->in_num, false, true); + + /* + * Put entry in available array (but don't update avail->idx until they + * do sync). + */ + avail_idx =3D svq->avail_idx_shadow & (svq->vring.num - 1); + avail->ring[avail_idx] =3D virtio_tswap16(svq->vdev, head); + svq->avail_idx_shadow++; + + /* Expose descriptors to device */ + smp_wmb(); + avail->idx =3D virtio_tswap16(svq->vdev, svq->avail_idx_shadow); + + return head; + +} + +static void vhost_shadow_vq_add(VhostShadowVirtqueue *svq, + VirtQueueElement *elem) +{ + unsigned qemu_head =3D vhost_shadow_vq_add_split(svq, elem); + + svq->ring_id_maps[qemu_head] =3D elem; +} + +/* Handle guest->device notifications */ static void vhost_handle_guest_kick(EventNotifier *n) { VhostShadowVirtqueue *svq =3D container_of(n, VhostShadowVirtqueue, @@ -69,7 +155,72 @@ static void vhost_handle_guest_kick(EventNotifier *n) return; } =20 - event_notifier_set(&svq->kick_notifier); + /* Make available as many buffers as possible */ + do { + if (virtio_queue_get_notification(svq->vq)) { + /* No more notifications until process all available */ + virtio_queue_set_notification(svq->vq, false); + } + + while (true) { + VirtQueueElement *elem; + if (virtio_queue_full(svq->vq)) { + break; + } + + elem =3D virtqueue_pop(svq->vq, sizeof(*elem)); + if (!elem) { + break; + } + + vhost_shadow_vq_add(svq, elem); + event_notifier_set(&svq->kick_notifier); + } + + virtio_queue_set_notification(svq->vq, true); + } while (!virtio_queue_empty(svq->vq)); +} + +static bool vhost_shadow_vq_more_used(VhostShadowVirtqueue *svq) +{ + if (svq->used_idx !=3D svq->shadow_used_idx) { + return true; + } + + /* Get used idx must not be reordered */ + smp_rmb(); + svq->shadow_used_idx =3D virtio_tswap16(svq->vdev, svq->vring.used->id= x); + + return svq->used_idx !=3D svq->shadow_used_idx; +} + +static VirtQueueElement *vhost_shadow_vq_get_buf(VhostShadowVirtqueue *svq) +{ + vring_desc_t *descs =3D svq->vring.desc; + const vring_used_t *used =3D svq->vring.used; + vring_used_elem_t used_elem; + uint16_t last_used; + + if (!vhost_shadow_vq_more_used(svq)) { + return NULL; + } + + last_used =3D svq->used_idx & (svq->vring.num - 1); + used_elem.id =3D virtio_tswap32(svq->vdev, used->ring[last_used].id); + used_elem.len =3D virtio_tswap32(svq->vdev, used->ring[last_used].len); + + if (unlikely(used_elem.id >=3D svq->vring.num)) { + error_report("Device %s says index %u is available", svq->vdev->na= me, + used_elem.id); + return NULL; + } + + descs[used_elem.id].next =3D svq->free_head; + svq->free_head =3D used_elem.id; + + svq->used_idx++; + svq->ring_id_maps[used_elem.id]->len =3D used_elem.len; + return g_steal_pointer(&svq->ring_id_maps[used_elem.id]); } =20 /* Forward vhost notifications */ @@ -78,6 +229,7 @@ static void vhost_shadow_vq_handle_call_no_test(EventNot= ifier *n) VhostShadowVirtqueue *svq =3D container_of(n, VhostShadowVirtqueue, call_notifier); EventNotifier *masked_notifier; + VirtQueue *vq =3D svq->vq; =20 /* Signal start of using masked notifier */ qemu_event_reset(&svq->masked_notifier.is_free); @@ -86,14 +238,29 @@ static void vhost_shadow_vq_handle_call_no_test(EventN= otifier *n) qemu_event_set(&svq->masked_notifier.is_free); } =20 - if (!masked_notifier) { - unsigned n =3D virtio_get_queue_index(svq->vq); - virtio_queue_invalidate_signalled_used(svq->vdev, n); - virtio_notify_irqfd(svq->vdev, svq->vq); - } else if (!svq->masked_notifier.signaled) { - svq->masked_notifier.signaled =3D true; - event_notifier_set(svq->masked_notifier.n); - } + /* Make as many buffers as possible used. */ + do { + unsigned i =3D 0; + + /* TODO: Use VRING_AVAIL_F_NO_INTERRUPT */ + while (true) { + g_autofree VirtQueueElement *elem =3D vhost_shadow_vq_get_buf(= svq); + if (!elem) { + break; + } + + assert(i < svq->vring.num); + virtqueue_fill(vq, elem, elem->len, i++); + } + + virtqueue_flush(vq, i); + if (!masked_notifier) { + virtio_notify_irqfd(svq->vdev, svq->vq); + } else if (!svq->masked_notifier.signaled) { + svq->masked_notifier.signaled =3D true; + event_notifier_set(svq->masked_notifier.n); + } + } while (vhost_shadow_vq_more_used(svq)); =20 if (masked_notifier) { /* Signal not using it anymore */ @@ -103,7 +270,6 @@ static void vhost_shadow_vq_handle_call_no_test(EventNo= tifier *n) =20 static void vhost_shadow_vq_handle_call(EventNotifier *n) { - if (likely(event_notifier_test_and_clear(n))) { vhost_shadow_vq_handle_call_no_test(n); } @@ -254,7 +420,11 @@ void vhost_shadow_vq_stop(struct vhost_dev *dev, unsigned idx, VhostShadowVirtqueue *svq) { + int i; int r =3D vhost_shadow_vq_restore_vdev_host_notifier(dev, idx, svq); + + assert(!dev->shadow_vqs_enabled); + if (unlikely(r < 0)) { error_report("Couldn't restore vq kick fd: %s", strerror(-r)); } @@ -272,6 +442,18 @@ void vhost_shadow_vq_stop(struct vhost_dev *dev, /* Restore vhost call */ vhost_virtqueue_mask(dev, dev->vdev, dev->vq_index + idx, dev->vqs[idx].notifier_is_masked); + + + for (i =3D 0; i < svq->vring.num; ++i) { + g_autofree VirtQueueElement *elem =3D svq->ring_id_maps[i]; + /* + * Although the doc says we must unpop in order, it's ok to unpop + * everything. + */ + if (elem) { + virtqueue_unpop(svq->vq, elem, elem->len); + } + } } =20 /* @@ -284,7 +466,7 @@ VhostShadowVirtqueue *vhost_shadow_vq_new(struct vhost_= dev *dev, int idx) unsigned num =3D virtio_queue_get_num(dev->vdev, vq_idx); size_t ring_size =3D vring_size(num, VRING_DESC_ALIGN_SIZE); g_autofree VhostShadowVirtqueue *svq =3D g_malloc0(sizeof(*svq) + ring= _size); - int r; + int r, i; =20 r =3D event_notifier_init(&svq->kick_notifier, 0); if (r !=3D 0) { @@ -303,6 +485,11 @@ VhostShadowVirtqueue *vhost_shadow_vq_new(struct vhost= _dev *dev, int idx) vring_init(&svq->vring, num, svq->descs, VRING_DESC_ALIGN_SIZE); svq->vq =3D virtio_get_queue(dev->vdev, vq_idx); svq->vdev =3D dev->vdev; + for (i =3D 0; i < num - 1; i++) { + svq->descs[i].next =3D virtio_tswap16(dev->vdev, i + 1); + } + + svq->ring_id_maps =3D g_new0(VirtQueueElement *, num); event_notifier_set_handler(&svq->call_notifier, vhost_shadow_vq_handle_call); qemu_event_init(&svq->masked_notifier.is_free, true); @@ -324,5 +511,6 @@ void vhost_shadow_vq_free(VhostShadowVirtqueue *vq) event_notifier_cleanup(&vq->kick_notifier); event_notifier_set_handler(&vq->call_notifier, NULL); event_notifier_cleanup(&vq->call_notifier); + g_free(vq->ring_id_maps); g_free(vq); } diff --git a/hw/virtio/vhost.c b/hw/virtio/vhost.c index eab3e334f2..a373999bc4 100644 --- a/hw/virtio/vhost.c +++ b/hw/virtio/vhost.c @@ -1021,6 +1021,19 @@ int vhost_device_iotlb_miss(struct vhost_dev *dev, u= int64_t iova, int write) =20 trace_vhost_iotlb_miss(dev, 1); =20 + if (qatomic_load_acquire(&dev->shadow_vqs_enabled)) { + uaddr =3D iova; + len =3D 4096; + ret =3D vhost_backend_update_device_iotlb(dev, iova, uaddr, len, + IOMMU_RW); + if (ret) { + trace_vhost_iotlb_miss(dev, 2); + error_report("Fail to update device iotlb"); + } + + return ret; + } + iotlb =3D address_space_get_iotlb_entry(dev->vdev->dma_as, iova, write, MEMTXATTRS_UNSPECIFIED); @@ -1227,8 +1240,28 @@ static int vhost_sw_live_migration_stop(struct vhost= _dev *dev) /* Can be read by vhost_virtqueue_mask, from vm exit */ qatomic_store_release(&dev->shadow_vqs_enabled, false); =20 + dev->vhost_ops->vhost_set_vring_enable(dev, false); + if (vhost_backend_invalidate_device_iotlb(dev, 0, -1ULL)) { + error_report("Fail to invalidate device iotlb"); + } + for (idx =3D 0; idx < dev->nvqs; ++idx) { + /* + * Update used ring information for IOTLB to work correctly, + * vhost-kernel code requires for this. + */ + struct vhost_virtqueue *vq =3D dev->vqs + idx; + vhost_device_iotlb_miss(dev, vq->used_phys, true); + vhost_shadow_vq_stop(dev, idx, dev->shadow_vqs[idx]); + vhost_virtqueue_start(dev, dev->vdev, &dev->vqs[idx], + dev->vq_index + idx); + } + + /* Enable guest's vq vring */ + dev->vhost_ops->vhost_set_vring_enable(dev, true); + + for (idx =3D 0; idx < dev->nvqs; ++idx) { vhost_shadow_vq_free(dev->shadow_vqs[idx]); } =20 @@ -1237,6 +1270,59 @@ static int vhost_sw_live_migration_stop(struct vhost= _dev *dev) return 0; } =20 +/* + * Start shadow virtqueue in a given queue. + * In failure case, this function leaves queue working as regular vhost mo= de. + */ +static bool vhost_sw_live_migration_start_vq(struct vhost_dev *dev, + unsigned idx) +{ + struct vhost_vring_addr addr =3D { + .index =3D idx, + }; + struct vhost_vring_state s =3D { + .index =3D idx, + }; + int r; + bool ok; + + vhost_virtqueue_stop(dev, dev->vdev, &dev->vqs[idx], dev->vq_index + i= dx); + ok =3D vhost_shadow_vq_start(dev, idx, dev->shadow_vqs[idx]); + if (unlikely(!ok)) { + return false; + } + + /* From this point, vhost_virtqueue_start can reset these changes */ + vhost_shadow_vq_get_vring_addr(dev->shadow_vqs[idx], &addr); + r =3D dev->vhost_ops->vhost_set_vring_addr(dev, &addr); + if (unlikely(r !=3D 0)) { + VHOST_OPS_DEBUG("vhost_set_vring_addr for shadow vq failed"); + goto err; + } + + r =3D dev->vhost_ops->vhost_set_vring_base(dev, &s); + if (unlikely(r !=3D 0)) { + VHOST_OPS_DEBUG("vhost_set_vring_base for shadow vq failed"); + goto err; + } + + /* + * Update used ring information for IOTLB to work correctly, + * vhost-kernel code requires for this. + */ + r =3D vhost_device_iotlb_miss(dev, addr.used_user_addr, true); + if (unlikely(r !=3D 0)) { + /* Debug message already printed */ + goto err; + } + + return true; + +err: + vhost_virtqueue_start(dev, dev->vdev, &dev->vqs[idx], dev->vq_index + = idx); + return false; +} + static int vhost_sw_live_migration_start(struct vhost_dev *dev) { int idx, stop_idx; @@ -1249,24 +1335,35 @@ static int vhost_sw_live_migration_start(struct vho= st_dev *dev) } } =20 + dev->vhost_ops->vhost_set_vring_enable(dev, false); + if (vhost_backend_invalidate_device_iotlb(dev, 0, -1ULL)) { + error_report("Fail to invalidate device iotlb"); + } + /* Can be read by vhost_virtqueue_mask, from vm exit */ qatomic_store_release(&dev->shadow_vqs_enabled, true); for (idx =3D 0; idx < dev->nvqs; ++idx) { - bool ok =3D vhost_shadow_vq_start(dev, idx, dev->shadow_vqs[idx]); + bool ok =3D vhost_sw_live_migration_start_vq(dev, idx); if (unlikely(!ok)) { goto err_start; } } =20 + /* Enable shadow vq vring */ + dev->vhost_ops->vhost_set_vring_enable(dev, true); return 0; =20 err_start: qatomic_store_release(&dev->shadow_vqs_enabled, false); for (stop_idx =3D 0; stop_idx < idx; stop_idx++) { vhost_shadow_vq_stop(dev, idx, dev->shadow_vqs[stop_idx]); + vhost_virtqueue_start(dev, dev->vdev, &dev->vqs[idx], + dev->vq_index + stop_idx); } =20 err_new: + /* Enable guest's vring */ + dev->vhost_ops->vhost_set_vring_enable(dev, true); for (idx =3D 0; idx < dev->nvqs; ++idx) { vhost_shadow_vq_free(dev->shadow_vqs[idx]); } @@ -1970,6 +2067,20 @@ void qmp_x_vhost_enable_shadow_vq(const char *name, = bool enable, Error **errp) =20 if (!hdev->started) { err_cause =3D "Device is not started"; + } else if (!vhost_dev_has_iommu(hdev)) { + err_cause =3D "Does not support iommu"; + } else if (hdev->acked_features & BIT_ULL(VIRTIO_F_RING_PACKED)) { + err_cause =3D "Is packed"; + } else if (hdev->acked_features & BIT_ULL(VIRTIO_RING_F_EVENT_IDX)= ) { + err_cause =3D "Have event idx"; + } else if (hdev->acked_features & + BIT_ULL(VIRTIO_RING_F_INDIRECT_DESC)) { + err_cause =3D "Supports indirect descriptors"; + } else if (!hdev->vhost_ops->vhost_set_vring_enable) { + err_cause =3D "Cannot pause device"; + } + + if (err_cause) { goto err; } =20 --=20 2.27.0 From nobody Fri May 17 00:32:42 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1615838370; cv=none; d=zohomail.com; s=zohoarc; b=XgK5HKmsGnNthveo+9pqgsjoln1eZBiNXxUSq1K3TwJaUQ4SX6lHqoO/82JqF+WSN9jNKEnWzcRr3P3pNF7RFSOJaAx5tsGpfPyLbp02VRQqXr0QGXV5Z3vHi4epJE4jVyYiBcjSzGB6TSYPiUagsaZ73TyHL0vTVgbpHfCEiHQ= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1615838370; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=7Q6YRPAq+nVbXTQhJkh1RIqJxAZz53gcwQAYdX2IWNk=; b=TOlAyg05jvkZiaGHtUr0FIC/AxA3ilTxsJuAaXTFIqIPCySS+WAFY9SqOD4pfjP4lZgOlT64GRX7Wnqshk6UZMP+ZnaJrTgtHVyd2zwR0yupXIQNMPdUubtjWGM4fBraxDyPZ2OntvBUiGKHFeaTp8uKhdSVRgsa80o6Hl07IHg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1615838370234977.583797702026; Mon, 15 Mar 2021 12:59:30 -0700 (PDT) Received: from localhost ([::1]:50254 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lLtNB-0001HF-6S for importer@patchew.org; Mon, 15 Mar 2021 15:59:29 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37854) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLtE0-0005rF-1D for qemu-devel@nongnu.org; Mon, 15 Mar 2021 15:50:00 -0400 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]:52022) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1lLtDy-0004EK-Ez for qemu-devel@nongnu.org; Mon, 15 Mar 2021 15:49:59 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-162-ffhZUreKM52Mwh7s7tTbsg-1; Mon, 15 Mar 2021 15:49:55 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id EA5398042C1; Mon, 15 Mar 2021 19:49:53 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-112-173.ams2.redhat.com [10.36.112.173]) by smtp.corp.redhat.com (Postfix) with ESMTP id 820BE5F706; Mon, 15 Mar 2021 19:49:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1615837797; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=7Q6YRPAq+nVbXTQhJkh1RIqJxAZz53gcwQAYdX2IWNk=; b=WlxKjuoYv0vFYRDdlE5QogkITAEvoKaA7p6boBtf9MSMNzep8OthGK/ghssCxCjbI3DrK7 NVSFJYY7i6sSpkHMFVy5a4qiqT6U4WgwEkR62l3z7AgkQLuCYj6yjSBVWfHNwL42lsFBSR phS6R2g/UKnx81fQc4z+g7P7bbiVvJM= X-MC-Unique: ffhZUreKM52Mwh7s7tTbsg-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v2 12/13] vhost: Check for device VRING_USED_F_NO_NOTIFY at shadow virtqueue kick Date: Mon, 15 Mar 2021 20:48:41 +0100 Message-Id: <20210315194842.277740-13-eperezma@redhat.com> In-Reply-To: <20210315194842.277740-1-eperezma@redhat.com> References: <20210315194842.277740-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=170.10.133.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.25, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H4=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Guru Prasad , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Stefano Garzarella , Michael Lilja , Jim Harford , Rob Miller Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index 68ed0f2740..7df98fc43f 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -145,6 +145,15 @@ static void vhost_shadow_vq_add(VhostShadowVirtqueue *= svq, svq->ring_id_maps[qemu_head] =3D elem; } =20 +static void vhost_shadow_vq_kick(VhostShadowVirtqueue *svq) +{ + /* Make sure we are reading updated device flag */ + smp_rmb(); + if (!(svq->vring.used->flags & VRING_USED_F_NO_NOTIFY)) { + event_notifier_set(&svq->kick_notifier); + } +} + /* Handle guest->device notifications */ static void vhost_handle_guest_kick(EventNotifier *n) { @@ -174,7 +183,7 @@ static void vhost_handle_guest_kick(EventNotifier *n) } =20 vhost_shadow_vq_add(svq, elem); - event_notifier_set(&svq->kick_notifier); + vhost_shadow_vq_kick(svq); } =20 virtio_queue_set_notification(svq->vq, true); --=20 2.27.0 From nobody Fri May 17 00:32:42 2024 Delivered-To: importer@patchew.org Authentication-Results: mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail(p=none dis=none) header.from=redhat.com ARC-Seal: i=1; a=rsa-sha256; t=1615838369; cv=none; d=zohomail.com; s=zohoarc; b=Vs2VWR3dWNDA1YwbnxIytNojY0ASO+VczWbd0cQ0l6mJIEvbfXsMWA4OjvfE1fcga1oDhqtV3NbvySq/fc2npDC8CuqQWqBJYFOwYwfunLPDBW45TISRai0hxScFnWswqWkV16Yfxwjpm4aIy7WU32yQj6lScHLcSCrZQxLgn5A= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1615838369; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:List-Subscribe:List-Post:List-Id:List-Archive:List-Help:List-Unsubscribe:MIME-Version:Message-ID:References:Sender:Subject:To; bh=UiuTlgOtfMbdtiwceqwHYXYl3zo3bRNZ7YUjgCumRxI=; b=Ef+bX62OP8DIuNjLstJ+NsSZaM9fqKOoMzp+Rm8IyHwNlh99lpqiQ6a/OWo+8WNxqtSJtv2GHoPVsHx+qiV/ACVYPqAnMcNd2YxeWvDRr03H/vEqgJsFY2MywSSoX5ao4rwPoM/W/ywtvBUeM+DEoInnEoP6Pui+FDlal6RqbOE= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=fail; spf=pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=qemu-devel-bounces+importer=patchew.org@nongnu.org; dmarc=fail header.from= (p=none dis=none) header.from= Return-Path: Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mx.zohomail.com with SMTPS id 1615838369967466.9283563257553; Mon, 15 Mar 2021 12:59:29 -0700 (PDT) Received: from localhost ([::1]:50182 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lLtNA-0001FM-Qo for importer@patchew.org; Mon, 15 Mar 2021 15:59:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:37868) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lLtE3-0005ve-M2 for qemu-devel@nongnu.org; Mon, 15 Mar 2021 15:50:04 -0400 Received: from us-smtp-delivery-124.mimecast.com ([216.205.24.124]:27116) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_CBC_SHA1:256) (Exim 4.90_1) (envelope-from ) id 1lLtE2-0004Fh-1L for qemu-devel@nongnu.org; Mon, 15 Mar 2021 15:50:03 -0400 Received: from mimecast-mx01.redhat.com (mimecast-mx01.redhat.com [209.132.183.4]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-251-v1DqsurKN0-K1MpbphjFng-1; Mon, 15 Mar 2021 15:49:59 -0400 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mimecast-mx01.redhat.com (Postfix) with ESMTPS id BC049801597; Mon, 15 Mar 2021 19:49:57 +0000 (UTC) Received: from eperezma.remote.csb (ovpn-112-173.ams2.redhat.com [10.36.112.173]) by smtp.corp.redhat.com (Postfix) with ESMTP id 4B46B5F9B8; Mon, 15 Mar 2021 19:49:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1615837801; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=UiuTlgOtfMbdtiwceqwHYXYl3zo3bRNZ7YUjgCumRxI=; b=Wia2EJmYbkAcKaPDdcKFageLSt6M/LX94Jy4gxer/lqqu4Z1plKUFiG0wOH2BPK40rjhqs dJip/5z1eci/+mUkOofz856GRzkcO+CpYQytx1BwXLtvizcAtZjlM2mm8llAufzwvx50Tf byIEygZ86l2PkbBd0AGUwBoh4oDDPhE= X-MC-Unique: v1DqsurKN0-K1MpbphjFng-1 From: =?UTF-8?q?Eugenio=20P=C3=A9rez?= To: qemu-devel@nongnu.org Subject: [RFC v2 13/13] vhost: Use VRING_AVAIL_F_NO_INTERRUPT at device call on shadow virtqueue Date: Mon, 15 Mar 2021 20:48:42 +0100 Message-Id: <20210315194842.277740-14-eperezma@redhat.com> In-Reply-To: <20210315194842.277740-1-eperezma@redhat.com> References: <20210315194842.277740-1-eperezma@redhat.com> MIME-Version: 1.0 X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=eperezma@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Received-SPF: pass (zohomail.com: domain of gnu.org designates 209.51.188.17 as permitted sender) client-ip=209.51.188.17; envelope-from=qemu-devel-bounces+importer=patchew.org@nongnu.org; helo=lists.gnu.org; Received-SPF: pass client-ip=216.205.24.124; envelope-from=eperezma@redhat.com; helo=us-smtp-delivery-124.mimecast.com X-Spam_score_int: -29 X-Spam_score: -3.0 X-Spam_bar: --- X-Spam_report: (-3.0 / 5.0 requ) BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.25, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Parav Pandit , "Michael S. Tsirkin" , Guru Prasad , Jason Wang , Juan Quintela , Markus Armbruster , virtualization@lists.linux-foundation.org, Harpreet Singh Anand , Xiao W Wang , Eli Cohen , Stefano Garzarella , Michael Lilja , Jim Harford , Rob Miller Errors-To: qemu-devel-bounces+importer=patchew.org@nongnu.org Sender: "Qemu-devel" X-ZohoMail-DKIM: fail (Header signature does not verify) Signed-off-by: Eugenio P=C3=A9rez --- hw/virtio/vhost-shadow-virtqueue.c | 28 +++++++++++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/hw/virtio/vhost-shadow-virtqueue.c b/hw/virtio/vhost-shadow-vi= rtqueue.c index 7df98fc43f..e3879a4622 100644 --- a/hw/virtio/vhost-shadow-virtqueue.c +++ b/hw/virtio/vhost-shadow-virtqueue.c @@ -71,10 +71,35 @@ typedef struct VhostShadowVirtqueue { /* Next head to consume from device */ uint16_t used_idx; =20 + /* Cache for the exposed notification flag */ + bool notification; + /* Descriptors copied from guest */ vring_desc_t descs[]; } VhostShadowVirtqueue; =20 +static void vhost_shadow_vq_set_notification(VhostShadowVirtqueue *svq, + bool enable) +{ + uint16_t notification_flag; + + if (svq->notification =3D=3D enable) { + return; + } + + notification_flag =3D virtio_tswap16(svq->vdev, VRING_AVAIL_F_NO_INTER= RUPT); + + svq->notification =3D enable; + if (enable) { + svq->vring.avail->flags &=3D ~notification_flag; + } else { + svq->vring.avail->flags |=3D notification_flag; + } + + /* Make sure device reads our flag */ + smp_mb(); +} + static void vhost_vring_write_descs(VhostShadowVirtqueue *svq, const struct iovec *iovec, size_t num, bool more_descs, bool writ= e) @@ -251,7 +276,7 @@ static void vhost_shadow_vq_handle_call_no_test(EventNo= tifier *n) do { unsigned i =3D 0; =20 - /* TODO: Use VRING_AVAIL_F_NO_INTERRUPT */ + vhost_shadow_vq_set_notification(svq, false); while (true) { g_autofree VirtQueueElement *elem =3D vhost_shadow_vq_get_buf(= svq); if (!elem) { @@ -269,6 +294,7 @@ static void vhost_shadow_vq_handle_call_no_test(EventNo= tifier *n) svq->masked_notifier.signaled =3D true; event_notifier_set(svq->masked_notifier.n); } + vhost_shadow_vq_set_notification(svq, true); } while (vhost_shadow_vq_more_used(svq)); =20 if (masked_notifier) { --=20 2.27.0