From nobody Sun Feb 8 14:53:17 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A192F107B2 for ; Thu, 28 Mar 2024 00:22:15 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711585337; cv=none; b=eK2r4fc9iuEi/aHXPfSviNtTnGT7tzCRUyhO49BAkhDUz9D3EgBiDCDYmomdWaXafq+XRyRBYqBsJCUC9ZwiovFCJD+QsBAGu8nBC0iwbTXpGsTO3I1UVqTiqqGB1Fyb4Rn1oPAFNMK5oXnSFc+rdG1+hpBlnrsjUl4HfFOzyDo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711585337; c=relaxed/simple; bh=TpeLSXi6OQkQnOEw3Yi3HiVxO9pqlJkPXi+oAqf40MY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=utLXeY/rikFKUL6nrh6F5V51Ppa8ot9mUgEfbNB9zQ70fP2naJNbAeJ/4TVaLPBrW8Unew9sU5CtIqsYk/Q8ZOxucXUYEhm0mb3kQs/LjnoBI6RkMWJu02r3VZBSU2ZnZVQlnrKP3b+XV6HW1YIbVbTneSISFPOgqXiZ+f3FvCQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=MzQNU2JR; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="MzQNU2JR" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1711585334; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=WvpBOxCM54Ha0O04EGrQoRIE4s015ZHfzJODI+uMiUU=; b=MzQNU2JRUrVXisjn7lFLej/K/NdiCCBCaOznYOSXMQu/nkz4AB4y3bkNWEduACL3y9MgH/ 1ntJqWarDTtQngKAOxhMHCEtnwnDzuOTrt1OChPVdYXxh9Z/Hc/op3DcvF+JxOlrzcD4w4 91ZIvu066IgLvUzsA/8wyMFC11K1iMM= Received: from mimecast-mx02.redhat.com (mx-ext.redhat.com [66.187.233.73]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-357-UV_cAuXxPYu-HH8KBHS8nQ-1; Wed, 27 Mar 2024 20:22:11 -0400 X-MC-Unique: UV_cAuXxPYu-HH8KBHS8nQ-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B172B1C05AA3; Thu, 28 Mar 2024 00:22:10 +0000 (UTC) Received: from gshan-thinkpadx1nanogen2.remote.csb (unknown [10.64.136.81]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 6F9502166B31; Thu, 28 Mar 2024 00:22:06 +0000 (UTC) From: Gavin Shan To: virtualization@lists.linux.dev Cc: linux-kernel@vger.kernel.org, mst@redhat.com, jasowang@redhat.com, will@kernel.org, davem@davemloft.net, stefanha@redhat.com, sgarzare@redhat.com, keirf@google.com, yihyu@redhat.com, shan.gavin@gmail.com Subject: [PATCH v3 1/3] vhost: Add smp_rmb() in vhost_vq_avail_empty() Date: Thu, 28 Mar 2024 10:21:47 +1000 Message-ID: <20240328002149.1141302-2-gshan@redhat.com> In-Reply-To: <20240328002149.1141302-1-gshan@redhat.com> References: <20240328002149.1141302-1-gshan@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.6 Content-Type: text/plain; charset="utf-8" A smp_rmb() has been missed in vhost_vq_avail_empty(), spotted by Will. Otherwise, it's not ensured the available ring entries pushed by guest can be observed by vhost in time, leading to stale available ring entries fetched by vhost in vhost_get_vq_desc(), as reported by Yihuang Yu on NVidia's grace-hopper (ARM64) platform. /home/gavin/sandbox/qemu.main/build/qemu-system-aarch64 \ -accel kvm -machine virt,gic-version=3Dhost -cpu host \ -smp maxcpus=3D1,cpus=3D1,sockets=3D1,clusters=3D1,cores=3D1,threads=3D1 \ -m 4096M,slots=3D16,maxmem=3D64G \ -object memory-backend-ram,id=3Dmem0,size=3D4096M \ : \ -netdev tap,id=3Dvnet0,vhost=3Dtrue \ -device virtio-net-pci,bus=3Dpcie.8,netdev=3Dvnet0,mac=3D52:54:00:f1:26:b0 : guest# netperf -H 10.26.1.81 -l 60 -C -c -t UDP_STREAM virtio_net virtio0: output.0:id 100 is not a head! Add the missed smp_rmb() in vhost_vq_avail_empty(). When tx_can_batch() returns true, it means there's still pending tx buffers. Since it might read indices, so it still can bypass the smp_rmb() in vhost_get_vq_desc(). Note that it should be safe until vq->avail_idx is changed by commit 275bf960ac697 ("vhost: better detection of available buffers"). Fixes: 275bf960ac69 ("vhost: better detection of available buffers") Cc: # v4.11+ Reported-by: Yihuang Yu Suggested-by: Will Deacon Signed-off-by: Gavin Shan Acked-by: Jason Wang Reviewed-by: Stefano Garzarella --- drivers/vhost/vhost.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 045f666b4f12..29df65b2ebf2 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -2799,9 +2799,19 @@ bool vhost_vq_avail_empty(struct vhost_dev *dev, str= uct vhost_virtqueue *vq) r =3D vhost_get_avail_idx(vq, &avail_idx); if (unlikely(r)) return false; + vq->avail_idx =3D vhost16_to_cpu(vq, avail_idx); + if (vq->avail_idx !=3D vq->last_avail_idx) { + /* Since we have updated avail_idx, the following + * call to vhost_get_vq_desc() will read available + * ring entries. Make sure that read happens after + * the avail_idx read. + */ + smp_rmb(); + return false; + } =20 - return vq->avail_idx =3D=3D vq->last_avail_idx; + return true; } EXPORT_SYMBOL_GPL(vhost_vq_avail_empty); =20 --=20 2.44.0 From nobody Sun Feb 8 14:53:17 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6B9C118646 for ; Thu, 28 Mar 2024 00:22:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711585342; cv=none; b=HwRCPHixUuhtL4dhtF/5e53dUr5wvd8X1SeaVR4+vWZL9f25jB9H96vSPG1RxFTrivXxrQZOytMc4RIZVUHgC1OFseTD4l/yUpxPBYD+r8JX0zhmpjecWu6IQjQH+Zy6yfxoV1vDOxCs15y4QSSNi1A5U8pui27jqESvbjbSQDg= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711585342; c=relaxed/simple; bh=cU6kIJkiS3Tzc0gfxh5Vp6FcxfNzGBGQ0y/wawqvXsw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=BYRNSqAmS4DBLEgBFxtVPYAcJIU+nttOmIcVNLcRM0eDbXatMJw84wE588RvVGw2SHPfx5fyw7zqkYLwQlAkxZF78+p5XNT89T5LNS7w4islbWtbTepFYieGxWcWT0OkWnmLnZWeI91t3B5r56yRb/VwFJFmJYE7PCF/LPlYCJI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=EW6INL8J; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="EW6INL8J" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1711585340; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=+dym63xirAqdqlrrfe7SLDQLu/P+E/eWefAsaqG2hIU=; b=EW6INL8Jfn3mUBPAxysT4nHnFEQbrLqU+zjEkKWHO+rlt+HEgkfm9mSVcWsQquZzF9boPa ZSctlSJqxmvnEZwDdAIqeeNRXYAHWoeCGcxzHtOir/VWtyQkVPcbB1HN/7LQPIWjug1AJK qPb+lrtr+66pJP0LwehQ8CZFL3wNedk= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-223-fAi4iPYcOTaibOqMrTCOTw-1; Wed, 27 Mar 2024 20:22:15 -0400 X-MC-Unique: fAi4iPYcOTaibOqMrTCOTw-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id 44407800264; Thu, 28 Mar 2024 00:22:15 +0000 (UTC) Received: from gshan-thinkpadx1nanogen2.remote.csb (unknown [10.64.136.81]) by smtp.corp.redhat.com (Postfix) with ESMTPS id 4D3792166B34; Thu, 28 Mar 2024 00:22:11 +0000 (UTC) From: Gavin Shan To: virtualization@lists.linux.dev Cc: linux-kernel@vger.kernel.org, mst@redhat.com, jasowang@redhat.com, will@kernel.org, davem@davemloft.net, stefanha@redhat.com, sgarzare@redhat.com, keirf@google.com, yihyu@redhat.com, shan.gavin@gmail.com Subject: [PATCH v3 2/3] vhost: Add smp_rmb() in vhost_enable_notify() Date: Thu, 28 Mar 2024 10:21:48 +1000 Message-ID: <20240328002149.1141302-3-gshan@redhat.com> In-Reply-To: <20240328002149.1141302-1-gshan@redhat.com> References: <20240328002149.1141302-1-gshan@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.6 Content-Type: text/plain; charset="utf-8" A smp_rmb() has been missed in vhost_enable_notify(), inspired by Will. Otherwise, it's not ensured the available ring entries pushed by guest can be observed by vhost in time, leading to stale available ring entries fetched by vhost in vhost_get_vq_desc(), as reported by Yihuang Yu on NVidia's grace-hopper (ARM64) platform. /home/gavin/sandbox/qemu.main/build/qemu-system-aarch64 \ -accel kvm -machine virt,gic-version=3Dhost -cpu host \ -smp maxcpus=3D1,cpus=3D1,sockets=3D1,clusters=3D1,cores=3D1,threads=3D1 \ -m 4096M,slots=3D16,maxmem=3D64G \ -object memory-backend-ram,id=3Dmem0,size=3D4096M \ : \ -netdev tap,id=3Dvnet0,vhost=3Dtrue \ -device virtio-net-pci,bus=3Dpcie.8,netdev=3Dvnet0,mac=3D52:54:00:f1:26:b0 : guest# netperf -H 10.26.1.81 -l 60 -C -c -t UDP_STREAM virtio_net virtio0: output.0:id 100 is not a head! Add the missed smp_rmb() in vhost_enable_notify(). When it returns true, it means there's still pending tx buffers. Since it might read indices, so it still can bypass the smp_rmb() in vhost_get_vq_desc(). Note that it should be safe until vq->avail_idx is changed by commit d3bb267bbdcb ("vhost: cache avail index in vhost_enable_notify()"). Fixes: d3bb267bbdcb ("vhost: cache avail index in vhost_enable_notify()") Cc: # v5.18+ Reported-by: Yihuang Yu Suggested-by: Will Deacon Signed-off-by: Gavin Shan Acked-by: Jason Wang Reviewed-by: Stefano Garzarella --- drivers/vhost/vhost.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 29df65b2ebf2..32686c79c41d 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -2848,9 +2848,19 @@ bool vhost_enable_notify(struct vhost_dev *dev, stru= ct vhost_virtqueue *vq) &vq->avail->idx, r); return false; } + vq->avail_idx =3D vhost16_to_cpu(vq, avail_idx); + if (vq->avail_idx !=3D vq->last_avail_idx) { + /* Since we have updated avail_idx, the following + * call to vhost_get_vq_desc() will read available + * ring entries. Make sure that read happens after + * the avail_idx read. + */ + smp_rmb(); + return true; + } =20 - return vq->avail_idx !=3D vq->last_avail_idx; + return false; } EXPORT_SYMBOL_GPL(vhost_enable_notify); =20 --=20 2.44.0 From nobody Sun Feb 8 14:53:17 2026 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 82AD31BC49 for ; Thu, 28 Mar 2024 00:22:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.133.124 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711585347; cv=none; b=EeXVbjuNBcFA4ASBiFIf7mxD4d6h19bVHSNMc+UzTQGgfs0NrFkU2zIFOML1UTzBqnh7pPc7OduVXNEPbYPOHvHI7u/yYJwQH2JLeeU7sco8hUrpRrEh9IVeF0tOojSq8+mVkkJgVJ5ku8jnOuKxFo53YsX+hi5mXAgpXY+X3JI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711585347; c=relaxed/simple; bh=NEWB7Jyx6ZTqoQYpQmN8i2qbWCNLe6BWjFi5j4OmaIg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=J99R0m7qrWLxHXcD/dap0XqaBh+K6fDmX4LzV6YqCNy4uH5OXhJvK2xFlqO4W6F2Ix4ViZzCANTRrlbuheCJs9Wpra1Pt1wMMvss4L8m8glWu7nnAQB5Svys3UAONKcQn2lYMFAGU3ebdPhdtIl70UVXEkteK60naYKc4gfaQFQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=IR59b+Wo; arc=none smtp.client-ip=170.10.133.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="IR59b+Wo" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1711585344; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=4ak0Cq/D1N2M9Qn2X1+lt/FpcfjVoz3V3sPZ0fz5NBc=; b=IR59b+WozRgzp90VLsFGZzIBvACvL8X4ihIead5eUtVLH7vFHp9W0brA3RoWa93JunNUjL O8zYgfS+yf/lUyHt7Fw/HXN4julc9/bNCcDxpTj4GpW7b333dfAW1ILqtIEwV1givsQHEW dLT7Z+ZeSHJyk/UZJyqXE6it0RepPIw= Received: from mimecast-mx02.redhat.com (mimecast-mx02.redhat.com [66.187.233.88]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-526-Wz9J29WXMTaRPc9LhpVekA-1; Wed, 27 Mar 2024 20:22:21 -0400 X-MC-Unique: Wz9J29WXMTaRPc9LhpVekA-1 Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.rdu2.redhat.com [10.11.54.6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by mimecast-mx02.redhat.com (Postfix) with ESMTPS id B388B800265; Thu, 28 Mar 2024 00:22:19 +0000 (UTC) Received: from gshan-thinkpadx1nanogen2.remote.csb (unknown [10.64.136.81]) by smtp.corp.redhat.com (Postfix) with ESMTPS id BBCDE2166B31; Thu, 28 Mar 2024 00:22:15 +0000 (UTC) From: Gavin Shan To: virtualization@lists.linux.dev Cc: linux-kernel@vger.kernel.org, mst@redhat.com, jasowang@redhat.com, will@kernel.org, davem@davemloft.net, stefanha@redhat.com, sgarzare@redhat.com, keirf@google.com, yihyu@redhat.com, shan.gavin@gmail.com Subject: [PATCH v3 3/3] vhost: Improve vhost_get_avail_idx() with smp_rmb() Date: Thu, 28 Mar 2024 10:21:49 +1000 Message-ID: <20240328002149.1141302-4-gshan@redhat.com> In-Reply-To: <20240328002149.1141302-1-gshan@redhat.com> References: <20240328002149.1141302-1-gshan@redhat.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Scanned-By: MIMEDefang 3.4.1 on 10.11.54.6 Content-Type: text/plain; charset="utf-8" All the callers of vhost_get_avail_idx() are concerned to the memory barrier, imposed by smp_rmb() to ensure the order of the available ring entry read and avail_idx read. Improve vhost_get_avail_idx() so that smp_rmb() is executed when the avail_idx is advanced. With it, the callers needn't to worry about the memory barrier. Suggested-by: Michael S. Tsirkin Signed-off-by: Gavin Shan Acked-by: Jason Wang --- drivers/vhost/vhost.c | 75 +++++++++++++++---------------------------- 1 file changed, 26 insertions(+), 49 deletions(-) diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 32686c79c41d..e6882f4f6ce2 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -1290,10 +1290,28 @@ static void vhost_dev_unlock_vqs(struct vhost_dev *= d) mutex_unlock(&d->vqs[i]->mutex); } =20 -static inline int vhost_get_avail_idx(struct vhost_virtqueue *vq, - __virtio16 *idx) +static inline int vhost_get_avail_idx(struct vhost_virtqueue *vq) { - return vhost_get_avail(vq, *idx, &vq->avail->idx); + __virtio16 avail_idx; + int r; + + r =3D vhost_get_avail(vq, avail_idx, &vq->avail->idx); + if (unlikely(r)) { + vq_err(vq, "Failed to access avail idx at %p\n", + &vq->avail->idx); + return r; + } + + vq->avail_idx =3D vhost16_to_cpu(vq, avail_idx); + if (vq->avail_idx !=3D vq->last_avail_idx) { + /* Ensure the available ring entry read happens + * before the avail_idx read when the avail_idx + * is advanced. + */ + smp_rmb(); + } + + return 0; } =20 static inline int vhost_get_avail_head(struct vhost_virtqueue *vq, @@ -2499,7 +2517,6 @@ int vhost_get_vq_desc(struct vhost_virtqueue *vq, struct vring_desc desc; unsigned int i, head, found =3D 0; u16 last_avail_idx; - __virtio16 avail_idx; __virtio16 ring_head; int ret, access; =20 @@ -2507,12 +2524,8 @@ int vhost_get_vq_desc(struct vhost_virtqueue *vq, last_avail_idx =3D vq->last_avail_idx; =20 if (vq->avail_idx =3D=3D vq->last_avail_idx) { - if (unlikely(vhost_get_avail_idx(vq, &avail_idx))) { - vq_err(vq, "Failed to access avail idx at %p\n", - &vq->avail->idx); + if (unlikely(vhost_get_avail_idx(vq))) return -EFAULT; - } - vq->avail_idx =3D vhost16_to_cpu(vq, avail_idx); =20 if (unlikely((u16)(vq->avail_idx - last_avail_idx) > vq->num)) { vq_err(vq, "Guest moved used index from %u to %u", @@ -2525,11 +2538,6 @@ int vhost_get_vq_desc(struct vhost_virtqueue *vq, */ if (vq->avail_idx =3D=3D last_avail_idx) return vq->num; - - /* Only get avail ring entries after they have been - * exposed by guest. - */ - smp_rmb(); } =20 /* Grab the next descriptor number they're advertising, and increment @@ -2790,35 +2798,19 @@ EXPORT_SYMBOL_GPL(vhost_add_used_and_signal_n); /* return true if we're sure that avaiable ring is empty */ bool vhost_vq_avail_empty(struct vhost_dev *dev, struct vhost_virtqueue *v= q) { - __virtio16 avail_idx; - int r; - if (vq->avail_idx !=3D vq->last_avail_idx) return false; =20 - r =3D vhost_get_avail_idx(vq, &avail_idx); - if (unlikely(r)) - return false; - - vq->avail_idx =3D vhost16_to_cpu(vq, avail_idx); - if (vq->avail_idx !=3D vq->last_avail_idx) { - /* Since we have updated avail_idx, the following - * call to vhost_get_vq_desc() will read available - * ring entries. Make sure that read happens after - * the avail_idx read. - */ - smp_rmb(); + if (unlikely(vhost_get_avail_idx(vq))) return false; - } =20 - return true; + return vq->avail_idx =3D=3D vq->last_avail_idx; } EXPORT_SYMBOL_GPL(vhost_vq_avail_empty); =20 /* OK, now we need to know about added descriptors. */ bool vhost_enable_notify(struct vhost_dev *dev, struct vhost_virtqueue *vq) { - __virtio16 avail_idx; int r; =20 if (!(vq->used_flags & VRING_USED_F_NO_NOTIFY)) @@ -2842,25 +2834,10 @@ bool vhost_enable_notify(struct vhost_dev *dev, str= uct vhost_virtqueue *vq) /* They could have slipped one in as we were doing that: make * sure it's written, then check again. */ smp_mb(); - r =3D vhost_get_avail_idx(vq, &avail_idx); - if (r) { - vq_err(vq, "Failed to check avail idx at %p: %d\n", - &vq->avail->idx, r); + if (unlikely(vhost_get_avail_idx(vq))) return false; - } - - vq->avail_idx =3D vhost16_to_cpu(vq, avail_idx); - if (vq->avail_idx !=3D vq->last_avail_idx) { - /* Since we have updated avail_idx, the following - * call to vhost_get_vq_desc() will read available - * ring entries. Make sure that read happens after - * the avail_idx read. - */ - smp_rmb(); - return true; - } =20 - return false; + return vq->avail_idx !=3D vq->last_avail_idx; } EXPORT_SYMBOL_GPL(vhost_enable_notify); =20 --=20 2.44.0