[GIT PULL] virtio,vhost,vdpa: features, fixes, cleanups

Michael S. Tsirkin posted 1 patch 2 years, 8 months ago
Documentation/driver-api/index.rst                 |    1 +
Documentation/driver-api/virtio/index.rst          |   11 +
Documentation/driver-api/virtio/virtio.rst         |  144 +++
.../driver-api/virtio/writing_virtio_drivers.rst   |  197 ++++
MAINTAINERS                                        |    6 +
drivers/block/virtio_blk.c                         |  522 ++++++++-
.../crypto/virtio/virtio_crypto_skcipher_algs.c    |    3 +-
drivers/nvdimm/virtio_pmem.c                       |   11 +-
drivers/pci/quirks.c                               |    8 +
drivers/vdpa/Kconfig                               |   22 +
drivers/vdpa/Makefile                              |    1 +
drivers/vdpa/ifcvf/ifcvf_base.c                    |   32 +-
drivers/vdpa/ifcvf/ifcvf_base.h                    |   10 +-
drivers/vdpa/ifcvf/ifcvf_main.c                    |  162 ++-
drivers/vdpa/mlx5/Makefile                         |    2 +-
drivers/vdpa/mlx5/core/mlx5_vdpa.h                 |    5 +-
drivers/vdpa/mlx5/core/mr.c                        |   46 +-
drivers/vdpa/mlx5/net/debug.c                      |  152 +++
drivers/vdpa/mlx5/net/mlx5_vnet.c                  |  252 +++--
drivers/vdpa/mlx5/net/mlx5_vnet.h                  |   94 ++
drivers/vdpa/solidrun/Makefile                     |    6 +
drivers/vdpa/solidrun/snet_hwmon.c                 |  188 ++++
drivers/vdpa/solidrun/snet_main.c                  | 1111 ++++++++++++++++++++
drivers/vdpa/solidrun/snet_vdpa.h                  |  196 ++++
drivers/vdpa/vdpa.c                                |   11 +-
drivers/vdpa/vdpa_sim/vdpa_sim.c                   |    7 +-
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c               |    4 +-
drivers/vdpa/vdpa_sim/vdpa_sim_net.c               |    8 +-
drivers/vdpa/vdpa_user/vduse_dev.c                 |    3 +
drivers/vdpa/virtio_pci/vp_vdpa.c                  |    2 +-
drivers/vhost/vdpa.c                               |   52 +-
drivers/vhost/vhost.c                              |    4 +-
drivers/vhost/vringh.c                             |    5 +-
drivers/vhost/vsock.c                              |  224 ++--
drivers/virtio/virtio.c                            |   12 +-
drivers/virtio/virtio_pci_modern.c                 |    4 +-
drivers/virtio/virtio_ring.c                       |    2 +-
include/linux/pci_ids.h                            |    2 +
include/linux/virtio_config.h                      |    8 +-
include/linux/virtio_vsock.h                       |  126 ++-
include/uapi/linux/vdpa.h                          |    4 +-
include/uapi/linux/virtio_blk.h                    |  133 +++
include/uapi/linux/virtio_blk_ioctl.h              |   44 +
net/vmw_vsock/virtio_transport.c                   |  149 +--
net/vmw_vsock/virtio_transport_common.c            |  420 ++++----
net/vmw_vsock/vsock_loopback.c                     |   51 +-
tools/virtio/ringtest/main.h                       |   37 +-
tools/virtio/virtio-trace/trace-agent-ctl.c        |    2 +-
tools/virtio/virtio_test.c                         |    2 +-
tools/virtio/vringh_test.c                         |    2 +
50 files changed, 3661 insertions(+), 839 deletions(-)
create mode 100644 Documentation/driver-api/virtio/index.rst
create mode 100644 Documentation/driver-api/virtio/virtio.rst
create mode 100644 Documentation/driver-api/virtio/writing_virtio_drivers.rst
create mode 100644 drivers/vdpa/mlx5/net/debug.c
create mode 100644 drivers/vdpa/mlx5/net/mlx5_vnet.h
create mode 100644 drivers/vdpa/solidrun/Makefile
create mode 100644 drivers/vdpa/solidrun/snet_hwmon.c
create mode 100644 drivers/vdpa/solidrun/snet_main.c
create mode 100644 drivers/vdpa/solidrun/snet_vdpa.h
create mode 100644 include/uapi/linux/virtio_blk_ioctl.h
[GIT PULL] virtio,vhost,vdpa: features, fixes, cleanups
Posted by Michael S. Tsirkin 2 years, 8 months ago
The following changes since commit 830b3c68c1fb1e9176028d02ef86f3cf76aa2476:

  Linux 6.1 (2022-12-11 14:15:18 -0800)

are available in the Git repository at:

  https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git tags/for_linus

for you to fetch changes up to 98dd6b2ef50d6f7876606a86c8d8a767c9fef6f5:

  virtio_blk: mark all zone fields LE (2022-12-22 14:32:36 -0500)


Note: merging this upstream results in a conflict
between commit:

  de4eda9de2d9 ("use less confusing names for iov_iter direction initializers")

from Linus' tree and commit:

  ("virtio/vsock: replace virtio_vsock_pkt with sk_buff")

from this tree.

This resolution below fixes it up, due to Stephen Rothwell

diff --cc drivers/vhost/vsock.c
index cd6f7776013a,830bc823addc..000000000000
--- a/drivers/vhost/vsock.c
+++ b/drivers/vhost/vsock.c
@@@ -165,8 -157,9 +157,9 @@@ vhost_transport_do_send_pkt(struct vhos
  			break;
  		}
  
 -		iov_iter_init(&iov_iter, READ, &vq->iov[out], in, iov_len);
 +		iov_iter_init(&iov_iter, ITER_DEST, &vq->iov[out], in, iov_len);
- 		payload_len = pkt->len - pkt->off;
+ 		payload_len = skb->len;
+ 		hdr = virtio_vsock_hdr(skb);
  
  		/* If the packet is greater than the space available in the
  		 * buffer, we split it using multiple buffers.
@@@ -366,18 -340,21 +340,22 @@@ vhost_vsock_alloc_skb(struct vhost_virt
  		return NULL;
  	}
  
- 	pkt = kzalloc(sizeof(*pkt), GFP_KERNEL);
- 	if (!pkt)
+ 	len = iov_length(vq->iov, out);
+ 
+ 	/* len contains both payload and hdr */
+ 	skb = virtio_vsock_alloc_skb(len, GFP_KERNEL);
+ 	if (!skb)
  		return NULL;
  
 -	iov_iter_init(&iov_iter, WRITE, vq->iov, out, len);
 +	len = iov_length(vq->iov, out);
 +	iov_iter_init(&iov_iter, ITER_SOURCE, vq->iov, out, len);
  
- 	nbytes = copy_from_iter(&pkt->hdr, sizeof(pkt->hdr), &iov_iter);
- 	if (nbytes != sizeof(pkt->hdr)) {
+ 	hdr = virtio_vsock_hdr(skb);
+ 	nbytes = copy_from_iter(hdr, sizeof(*hdr), &iov_iter);
+ 	if (nbytes != sizeof(*hdr)) {
  		vq_err(vq, "Expected %zu bytes for pkt->hdr, got %zu bytes\n",
- 		       sizeof(pkt->hdr), nbytes);
- 		kfree(pkt);
+ 		       sizeof(*hdr), nbytes);
+ 		kfree_skb(skb);
  		return NULL;
  	}
  

It can also be found in linux-next, see next-20221220.


----------------------------------------------------------------
virtio,vhost,vdpa: features, fixes, cleanups

zoned block device support
lifetime stats support (for virtio devices backed by memory supporting that)
vsock rework to use skbuffs
ifcvf features provisioning
new SolidNET DPU driver

Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

----------------------------------------------------------------
Alvaro Karsz (5):
      Add SolidRun vendor id
      New PCI quirk for SolidRun SNET DPU.
      virtio: vdpa: new SolidNET DPU driver.
      virtio_blk: add VIRTIO_BLK_F_LIFETIME feature support
      virtio: vdpa: fix snprintf size argument in snet_vdpa driver

Angus Chen (2):
      virtio_pci: modify ENOENT to EINVAL
      virtio_blk: use UINT_MAX instead of -1U

Bobby Eshleman (1):
      virtio/vsock: replace virtio_vsock_pkt with sk_buff

Cindy Lu (2):
      vhost_vdpa: fix the crash in unmap a large memory
      vdpa_sim_net: should not drop the multicast/broadcast packet

Colin Ian King (1):
      RDMA/mlx5: remove variable i

Davidlohr Bueso (2):
      tools/virtio: remove stray characters
      tools/virtio: remove smp_read_barrier_depends()

Dawei Li (1):
      virtio: Implementing attribute show with sysfs_emit

Dmitry Fomichev (2):
      virtio-blk: use a helper to handle request queuing errors
      virtio-blk: add support for zoned block devices

Eli Cohen (8):
      vdpa/mlx5: Fix rule forwarding VLAN to TIR
      vdpa/mlx5: Return error on vlan ctrl commands if not supported
      vdpa/mlx5: Fix wrong mac address deletion
      vdpa/mlx5: Avoid using reslock in event_handler
      vdpa/mlx5: Avoid overwriting CVQ iotlb
      vdpa/mlx5: Move some definitions to a new header file
      vdpa/mlx5: Add debugfs subtree
      vdpa/mlx5: Add RX counters to debugfs

Eugenio Pérez (1):
      vdpa_sim_net: Offer VIRTIO_NET_F_STATUS

Harshit Mogalapalli (1):
      vduse: Validate vq_num in vduse_validate_config()

Jason Wang (2):
      vdpa: conditionally fill max max queue pair for stats
      vdpasim: fix memory leak when freeing IOTLBs

Michael S. Tsirkin (3):
      virtio_blk: temporary variable type tweak
      virtio_blk: zone append in header type tweak
      virtio_blk: mark all zone fields LE

Michael Sammler (1):
      virtio_pmem: populate numa information

Rafael Mendonca (1):
      virtio_blk: Fix signedness bug in virtblk_prep_rq()

Ricardo Cañuelo (2):
      tools/virtio: initialize spinlocks in vring_test.c
      docs: driver-api: virtio: virtio on Linux

Rong Wang (1):
      vdpa/vp_vdpa: fix kfree a wrong pointer in vp_vdpa_remove

Shaomin Deng (1):
      tools: Delete the unneeded semicolon after curly braces

Shaoqin Huang (2):
      virtio_pci: use helper function is_power_of_2()
      virtio_ring: use helper function is_power_of_2()

Si-Wei Liu (1):
      vdpa: merge functionally duplicated dev_features attributes

Stefano Garzarella (4):
      vringh: fix range used in iotlb_translate()
      vhost: fix range used in translate_desc()
      vhost-vdpa: fix an iotlb memory leak
      vdpa_sim: fix vringh initialization in vdpasim_queue_ready()

Wei Yongjun (1):
      virtio-crypto: fix memory leak in virtio_crypto_alg_skcipher_close_session()

Yuan Can (1):
      vhost/vsock: Fix error handling in vhost_vsock_init()

Zhu Lingshan (12):
      vDPA/ifcvf: decouple hw features manipulators from the adapter
      vDPA/ifcvf: decouple config space ops from the adapter
      vDPA/ifcvf: alloc the mgmt_dev before the adapter
      vDPA/ifcvf: decouple vq IRQ releasers from the adapter
      vDPA/ifcvf: decouple config IRQ releaser from the adapter
      vDPA/ifcvf: decouple vq irq requester from the adapter
      vDPA/ifcvf: decouple config/dev IRQ requester and vectors allocator from the adapter
      vDPA/ifcvf: ifcvf_request_irq works on ifcvf_hw
      vDPA/ifcvf: manage ifcvf_hw in the mgmt_dev
      vDPA/ifcvf: allocate the adapter in dev_add()
      vDPA/ifcvf: retire ifcvf_private_to_vf
      vDPA/ifcvf: implement features provisioning

ruanjinjie (1):
      vdpa_sim: fix possible memory leak in vdpasim_net_init() and vdpasim_blk_init()

wangjianli (1):
      tools/virtio: Variable type completion

 Documentation/driver-api/index.rst                 |    1 +
 Documentation/driver-api/virtio/index.rst          |   11 +
 Documentation/driver-api/virtio/virtio.rst         |  144 +++
 .../driver-api/virtio/writing_virtio_drivers.rst   |  197 ++++
 MAINTAINERS                                        |    6 +
 drivers/block/virtio_blk.c                         |  522 ++++++++-
 .../crypto/virtio/virtio_crypto_skcipher_algs.c    |    3 +-
 drivers/nvdimm/virtio_pmem.c                       |   11 +-
 drivers/pci/quirks.c                               |    8 +
 drivers/vdpa/Kconfig                               |   22 +
 drivers/vdpa/Makefile                              |    1 +
 drivers/vdpa/ifcvf/ifcvf_base.c                    |   32 +-
 drivers/vdpa/ifcvf/ifcvf_base.h                    |   10 +-
 drivers/vdpa/ifcvf/ifcvf_main.c                    |  162 ++-
 drivers/vdpa/mlx5/Makefile                         |    2 +-
 drivers/vdpa/mlx5/core/mlx5_vdpa.h                 |    5 +-
 drivers/vdpa/mlx5/core/mr.c                        |   46 +-
 drivers/vdpa/mlx5/net/debug.c                      |  152 +++
 drivers/vdpa/mlx5/net/mlx5_vnet.c                  |  252 +++--
 drivers/vdpa/mlx5/net/mlx5_vnet.h                  |   94 ++
 drivers/vdpa/solidrun/Makefile                     |    6 +
 drivers/vdpa/solidrun/snet_hwmon.c                 |  188 ++++
 drivers/vdpa/solidrun/snet_main.c                  | 1111 ++++++++++++++++++++
 drivers/vdpa/solidrun/snet_vdpa.h                  |  196 ++++
 drivers/vdpa/vdpa.c                                |   11 +-
 drivers/vdpa/vdpa_sim/vdpa_sim.c                   |    7 +-
 drivers/vdpa/vdpa_sim/vdpa_sim_blk.c               |    4 +-
 drivers/vdpa/vdpa_sim/vdpa_sim_net.c               |    8 +-
 drivers/vdpa/vdpa_user/vduse_dev.c                 |    3 +
 drivers/vdpa/virtio_pci/vp_vdpa.c                  |    2 +-
 drivers/vhost/vdpa.c                               |   52 +-
 drivers/vhost/vhost.c                              |    4 +-
 drivers/vhost/vringh.c                             |    5 +-
 drivers/vhost/vsock.c                              |  224 ++--
 drivers/virtio/virtio.c                            |   12 +-
 drivers/virtio/virtio_pci_modern.c                 |    4 +-
 drivers/virtio/virtio_ring.c                       |    2 +-
 include/linux/pci_ids.h                            |    2 +
 include/linux/virtio_config.h                      |    8 +-
 include/linux/virtio_vsock.h                       |  126 ++-
 include/uapi/linux/vdpa.h                          |    4 +-
 include/uapi/linux/virtio_blk.h                    |  133 +++
 include/uapi/linux/virtio_blk_ioctl.h              |   44 +
 net/vmw_vsock/virtio_transport.c                   |  149 +--
 net/vmw_vsock/virtio_transport_common.c            |  420 ++++----
 net/vmw_vsock/vsock_loopback.c                     |   51 +-
 tools/virtio/ringtest/main.h                       |   37 +-
 tools/virtio/virtio-trace/trace-agent-ctl.c        |    2 +-
 tools/virtio/virtio_test.c                         |    2 +-
 tools/virtio/vringh_test.c                         |    2 +
 50 files changed, 3661 insertions(+), 839 deletions(-)
 create mode 100644 Documentation/driver-api/virtio/index.rst
 create mode 100644 Documentation/driver-api/virtio/virtio.rst
 create mode 100644 Documentation/driver-api/virtio/writing_virtio_drivers.rst
 create mode 100644 drivers/vdpa/mlx5/net/debug.c
 create mode 100644 drivers/vdpa/mlx5/net/mlx5_vnet.h
 create mode 100644 drivers/vdpa/solidrun/Makefile
 create mode 100644 drivers/vdpa/solidrun/snet_hwmon.c
 create mode 100644 drivers/vdpa/solidrun/snet_main.c
 create mode 100644 drivers/vdpa/solidrun/snet_vdpa.h
 create mode 100644 include/uapi/linux/virtio_blk_ioctl.h

Re: [GIT PULL] virtio,vhost,vdpa: features, fixes, cleanups
Posted by Linus Torvalds 2 years, 8 months ago
On Thu, Dec 22, 2022 at 11:43 AM Michael S. Tsirkin <mst@redhat.com> wrote:
>
>   https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git tags/for_linus

I see none of this in linux-next.

               Linus
Re: [GIT PULL] virtio,vhost,vdpa: features, fixes, cleanups
Posted by Michael S. Tsirkin 2 years, 8 months ago
On Fri, Dec 23, 2022 at 11:54:41AM -0800, Linus Torvalds wrote:
> On Thu, Dec 22, 2022 at 11:43 AM Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> >   https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git tags/for_linus
> 
> I see none of this in linux-next.
> 
>                Linus

They were all there, just not as these commits, as I squashed fixups to
avoid bisect breakages with some configs. Did I do wrong?

-- 
MST
Re: [GIT PULL] virtio,vhost,vdpa: features, fixes, cleanups
Posted by Linus Torvalds 2 years, 8 months ago
On Fri, Dec 23, 2022 at 2:27 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> They were all there, just not as these commits, as I squashed fixups to
> avoid bisect breakages with some configs. Did I do wrong?

I am literally looking at the next-20221214 state right now, doing

    git log linus/master.. -- drivers/vhost/vsock.c
    git log linus/master.. -- drivers/vdpa/mlx5/
    git log --grep="temporary variable type tweak"

and seeing nothing.

So none of these commits - in *any* form - were in linux-next last
week as far as I can tell.

             Linus
Re: [GIT PULL] virtio,vhost,vdpa: features, fixes, cleanups
Posted by Michael S. Tsirkin 2 years, 8 months ago
On Fri, Dec 23, 2022 at 02:36:46PM -0800, Linus Torvalds wrote:
> On Fri, Dec 23, 2022 at 2:27 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > They were all there, just not as these commits, as I squashed fixups to
> > avoid bisect breakages with some configs. Did I do wrong?
> 
> I am literally looking at the next-20221214 state right now, doing
> 
>     git log linus/master.. -- drivers/vhost/vsock.c
>     git log linus/master.. -- drivers/vdpa/mlx5/
>     git log --grep="temporary variable type tweak"
> 
> and seeing nothing.
> 
> So none of these commits - in *any* form - were in linux-next last
> week as far as I can tell.
> 
>              Linus


They were in  next-20221220 though.

-- 
MST
Re: [GIT PULL] virtio,vhost,vdpa: features, fixes, cleanups
Posted by Linus Torvalds 2 years, 8 months ago
On Fri, Dec 23, 2022 at 9:35 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> They were in  next-20221220 though.

So, perfect for the *next* merge window.

Do you understand what the word "next" means? We don't call it
"linux-this", do we?

This is not a new rule. Things are supposed to be ready *before* the
merge window (that's what makes it "next", get it?).

I will also point you to to

  https://lore.kernel.org/lkml/CAHk-=wj_HcgFZNyZHTLJ7qC2613zphKDtLh6ndciwopZRfH0aQ@mail.gmail.com/

where I'm being pretty damn clear about things.

And before you start bleating about "I needed more heads up", never
mind that this isn't even a new rule, and never mind what that "next"
word means, let me just point to the 6.1-rc6 notice too:

  https://lore.kernel.org/lkml/CAHk-=wgUZwX8Sbb8Zvm7FxWVfX6CGuE7x+E16VKoqL7Ok9vv7g@mail.gmail.com/

and if the meaning of "next" has eluded you all these years, maybe it
was high time you learnt. Hmm?

              Linus
Re: [GIT PULL] virtio,vhost,vdpa: features, fixes, cleanups
Posted by Michael S. Tsirkin 2 years, 8 months ago
On Fri, Dec 23, 2022 at 10:10:30PM -0800, Linus Torvalds wrote:
> On Fri, Dec 23, 2022 at 9:35 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > They were in  next-20221220 though.
> 
> So, perfect for the *next* merge window.
> 
> Do you understand what the word "next" means? We don't call it
> "linux-this", do we?
> 
> This is not a new rule. Things are supposed to be ready *before* the
> merge window (that's what makes it "next", get it?).
> 
> I will also point you to to
> 
>   https://lore.kernel.org/lkml/CAHk-=wj_HcgFZNyZHTLJ7qC2613zphKDtLh6ndciwopZRfH0aQ@mail.gmail.com/
> 
> where I'm being pretty damn clear about things.
> 
> And before you start bleating about "I needed more heads up", never
> mind that this isn't even a new rule, and never mind what that "next"
> word means, let me just point to the 6.1-rc6 notice too:
> 
>   https://lore.kernel.org/lkml/CAHk-=wgUZwX8Sbb8Zvm7FxWVfX6CGuE7x+E16VKoqL7Ok9vv7g@mail.gmail.com/
> 
> and if the meaning of "next" has eluded you all these years, maybe it
> was high time you learnt. Hmm?
> 
>               Linus

Yea I really screwed up with this one. High time I learned that "no
fallout from testing" most likely does not mean "no bugs" but instead
"you forgot to push to next". Putting procedures in place now to
check automatically.


-- 
MST
Re: [GIT PULL] virtio,vhost,vdpa: features, fixes, cleanups
Posted by Slade Watkins 2 years, 8 months ago
On Sat, Dec 24, 2022 at 12:36 AM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Fri, Dec 23, 2022 at 02:36:46PM -0800, Linus Torvalds wrote:
> > On Fri, Dec 23, 2022 at 2:27 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > >
> > > They were all there, just not as these commits, as I squashed fixups to
> > > avoid bisect breakages with some configs. Did I do wrong?
> >
> > I am literally looking at the next-20221214 state right now, doing
> >
> >     git log linus/master.. -- drivers/vhost/vsock.c
> >     git log linus/master.. -- drivers/vdpa/mlx5/
> >     git log --grep="temporary variable type tweak"
> >
> > and seeing nothing.
> >
> > So none of these commits - in *any* form - were in linux-next last
> > week as far as I can tell.
> >
> >              Linus
>
>
> They were in  next-20221220 though.
>

+1, just checked and these appear to be in next-20221220.

-- Slade
Re: [GIT PULL] virtio,vhost,vdpa: features, fixes, cleanups
Posted by Michael S. Tsirkin 2 years, 8 months ago
On Fri, Dec 23, 2022 at 05:27:49PM -0500, Michael S. Tsirkin wrote:
> On Fri, Dec 23, 2022 at 11:54:41AM -0800, Linus Torvalds wrote:
> > On Thu, Dec 22, 2022 at 11:43 AM Michael S. Tsirkin <mst@redhat.com> wrote:
> > >
> > >   https://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost.git tags/for_linus
> > 
> > I see none of this in linux-next.
> > 
> >                Linus
> 
> They were all there, just not as these commits, as I squashed fixups to
> avoid bisect breakages with some configs. Did I do wrong?

More specifically, everything up to
458326ec10d1233399a342263d33878cb0afe710 lines up in next-20221220, and
then I decided to rebase to squash bugfixes.
Plus these are two trivial patches on top that just tweak sparse
tags so 0 chance of regressions, and an also trivial security-related bugfix.


> -- 
> MST