"-netdev socket" only supports inet sockets.

It's not a complex task to add support for unix sockets, but
the socket netdev parameters are not defined to manage well unix
socket parameters.

As discussed in:

  "socket.c added support for unix domain socket datagram transport"
  https://lore.kernel.org/qemu-devel/1C0E1BC5-904F-46B0-8044-68E43E67BE60@gmail.com/

This series adds support of unix socket type using SocketAddress QAPI structure.

Two new netdev backends, "stream" and "dgram" are added, that are barely a copy of "socket"
backend but they use the SocketAddress QAPI to provide socket parameters.
And then they also implement unix sockets (TCP and UDP).

Some examples of CLI syntax:

  for TCP:

  -netdev stream,id=socket0,addr.type=inet,addr.host=localhost,addr.port=1234
  -netdev stream,id=socket0,server=off,addr.type=inet,addr.host=localhost,addr.port=1234

  -netdev dgram,id=socket0,\
          local.type=inet,local.host=localhost,local.port=1234,\
          remote.type=inet,remote.host=localhost,remote.port=1235

  for UNIX:

  -netdev stream,id=socket0,addr.type=unix,addr.path=/tmp/qemu0
  -netdev stream,id=socket0,server=off,addr.type=unix,addr.path=/tmp/qemu0

  -netdev dgram,id=socket0,\
          local.type=unix,local.path=/tmp/qemu0,\
          remote.type=unix,remote.path=/tmp/qemu1

  for FD:

  -netdev stream,id=socket0,addr.type=fd,addr.str=4
  -netdev stream,id=socket0,server=off,addr.type=fd,addr.str=5

  -netdev dgram,id=socket0,local.type=fd,addr.str=4

v3:
  - remove support of "-net" for dgram and stream. They are only
    supported with "-netdev" option.
  - use &error_fatal directly in net_client_inits()
  - update qemu-options.hx
  - move to QIO for stream socket

v2:
  - use "stream" and "dgram" rather than "socket-ng,mode=stream"
    and ""socket-ng,mode=dgram"
  - extract code to bypass qemu_opts_parse_noisily() to
    a new patch
  - do not ignore EINVAL (Stefano)
  - fix "-net" option

CC: Ralph Schmieder <ralph.schmieder@gmail.com>
CC: Stefano Brivio <sbrivio@redhat.com>
CC: Daniel P. Berrangé <berrange@redhat.com>
CC: Markus Armbruster <armbru@redhat.com>

Laurent Vivier (10):
  net: introduce convert_host_port()
  net: remove the @errp argument of net_client_inits()
  qapi: net: introduce a way to bypass qemu_opts_parse_noisily()
  qapi: net: add stream and dgram netdevs
  net: stream: add unix socket
  net: dgram: make dgram_dst generic
  net: dgram: move mcast specific code from net_socket_fd_init_dgram()
  net: dgram: add unix socket
  qemu-sockets: introduce socket_uri()
  net: stream: move to QIO

Stefano Brivio (1):
  net: stream: Don't ignore EINVAL on netdev socket connection

 hmp-commands.hx        |   2 +-
 include/net/net.h      |   3 +-
 include/qemu/sockets.h |   4 +-
 monitor/hmp-cmds.c     |  23 +-
 net/clients.h          |   6 +
 net/dgram.c            | 706 +++++++++++++++++++++++++++++++++++++++++
 net/hub.c              |   2 +
 net/meson.build        |   2 +
 net/net.c              | 149 ++++++---
 net/stream.c           | 371 ++++++++++++++++++++++
 qapi/net.json          |  40 ++-
 qemu-options.hx        |  12 +
 softmmu/vl.c           |   5 +-
 util/qemu-sockets.c    |  20 ++
 14 files changed, 1277 insertions(+), 68 deletions(-)
 create mode 100644 net/dgram.c
 create mode 100644 net/stream.c

-- 
2.36.1

* Laurent Vivier (lvivier@redhat.com) wrote:
> "-netdev socket" only supports inet sockets.
> 
> It's not a complex task to add support for unix sockets, but
> the socket netdev parameters are not defined to manage well unix
> socket parameters.
> 
> As discussed in:
> 
>   "socket.c added support for unix domain socket datagram transport"
>   https://lore.kernel.org/qemu-devel/1C0E1BC5-904F-46B0-8044-68E43E67BE60@gmail.com/
> 
> This series adds support of unix socket type using SocketAddress QAPI structure.
> 
> Two new netdev backends, "stream" and "dgram" are added, that are barely a copy of "socket"
> backend but they use the SocketAddress QAPI to provide socket parameters.
> And then they also implement unix sockets (TCP and UDP).

Had you considered a -netdev chardev?

Dave

> Some examples of CLI syntax:
> 
>   for TCP:
> 
>   -netdev stream,id=socket0,addr.type=inet,addr.host=localhost,addr.port=1234
>   -netdev stream,id=socket0,server=off,addr.type=inet,addr.host=localhost,addr.port=1234
> 
>   -netdev dgram,id=socket0,\
>           local.type=inet,local.host=localhost,local.port=1234,\
>           remote.type=inet,remote.host=localhost,remote.port=1235
> 
>   for UNIX:
> 
>   -netdev stream,id=socket0,addr.type=unix,addr.path=/tmp/qemu0
>   -netdev stream,id=socket0,server=off,addr.type=unix,addr.path=/tmp/qemu0
> 
>   -netdev dgram,id=socket0,\
>           local.type=unix,local.path=/tmp/qemu0,\
>           remote.type=unix,remote.path=/tmp/qemu1
> 
>   for FD:
> 
>   -netdev stream,id=socket0,addr.type=fd,addr.str=4
>   -netdev stream,id=socket0,server=off,addr.type=fd,addr.str=5
> 
>   -netdev dgram,id=socket0,local.type=fd,addr.str=4
> 
> v3:
>   - remove support of "-net" for dgram and stream. They are only
>     supported with "-netdev" option.
>   - use &error_fatal directly in net_client_inits()
>   - update qemu-options.hx
>   - move to QIO for stream socket
> 
> v2:
>   - use "stream" and "dgram" rather than "socket-ng,mode=stream"
>     and ""socket-ng,mode=dgram"
>   - extract code to bypass qemu_opts_parse_noisily() to
>     a new patch
>   - do not ignore EINVAL (Stefano)
>   - fix "-net" option
> 
> CC: Ralph Schmieder <ralph.schmieder@gmail.com>
> CC: Stefano Brivio <sbrivio@redhat.com>
> CC: Daniel P. Berrangé <berrange@redhat.com>
> CC: Markus Armbruster <armbru@redhat.com>
> 
> Laurent Vivier (10):
>   net: introduce convert_host_port()
>   net: remove the @errp argument of net_client_inits()
>   qapi: net: introduce a way to bypass qemu_opts_parse_noisily()
>   qapi: net: add stream and dgram netdevs
>   net: stream: add unix socket
>   net: dgram: make dgram_dst generic
>   net: dgram: move mcast specific code from net_socket_fd_init_dgram()
>   net: dgram: add unix socket
>   qemu-sockets: introduce socket_uri()
>   net: stream: move to QIO
> 
> Stefano Brivio (1):
>   net: stream: Don't ignore EINVAL on netdev socket connection
> 
>  hmp-commands.hx        |   2 +-
>  include/net/net.h      |   3 +-
>  include/qemu/sockets.h |   4 +-
>  monitor/hmp-cmds.c     |  23 +-
>  net/clients.h          |   6 +
>  net/dgram.c            | 706 +++++++++++++++++++++++++++++++++++++++++
>  net/hub.c              |   2 +
>  net/meson.build        |   2 +
>  net/net.c              | 149 ++++++---
>  net/stream.c           | 371 ++++++++++++++++++++++
>  qapi/net.json          |  40 ++-
>  qemu-options.hx        |  12 +
>  softmmu/vl.c           |   5 +-
>  util/qemu-sockets.c    |  20 ++
>  14 files changed, 1277 insertions(+), 68 deletions(-)
>  create mode 100644 net/dgram.c
>  create mode 100644 net/stream.c
> 
> -- 
> 2.36.1
> 
-- 
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

On 20/06/2022 20:24, Dr. David Alan Gilbert wrote:
> * Laurent Vivier (lvivier@redhat.com) wrote:
>> "-netdev socket" only supports inet sockets.
>>
>> It's not a complex task to add support for unix sockets, but
>> the socket netdev parameters are not defined to manage well unix
>> socket parameters.
>>
>> As discussed in:
>>
>>    "socket.c added support for unix domain socket datagram transport"
>>    https://lore.kernel.org/qemu-devel/1C0E1BC5-904F-46B0-8044-68E43E67BE60@gmail.com/
>>
>> This series adds support of unix socket type using SocketAddress QAPI structure.
>>
>> Two new netdev backends, "stream" and "dgram" are added, that are barely a copy of "socket"
>> backend but they use the SocketAddress QAPI to provide socket parameters.
>> And then they also implement unix sockets (TCP and UDP).
> 
> Had you considered a -netdev chardev?
> 

I think by definition a "chardev" doesn't behave like a "netdev". Moreover "chardev" is 
already a frontend for several backends (socket, udp, ...), this would mean we use the 
frontend "chardev" as a backend of a "netdev". More and more layers...

And in the case of "-netdev dgram", we can use unix socket and sendto()/recv() while 
something like "-chardev udp,id=char0 -netdev chardev,chardev=char0,id=net0" doesn't 
support unix (see qio_channel_socket_dgram_sync()/socket_dgram()) and uses a 
"connect()/sendmsg()/recv()" (that really changes the behaviour of the backend)

The aim of this series is to add unix socket support.

Thanks,
Laurent

* Laurent Vivier (lvivier@redhat.com) wrote:
> On 20/06/2022 20:24, Dr. David Alan Gilbert wrote:
> > * Laurent Vivier (lvivier@redhat.com) wrote:
> > > "-netdev socket" only supports inet sockets.
> > > 
> > > It's not a complex task to add support for unix sockets, but
> > > the socket netdev parameters are not defined to manage well unix
> > > socket parameters.
> > > 
> > > As discussed in:
> > > 
> > >    "socket.c added support for unix domain socket datagram transport"
> > >    https://lore.kernel.org/qemu-devel/1C0E1BC5-904F-46B0-8044-68E43E67BE60@gmail.com/
> > > 
> > > This series adds support of unix socket type using SocketAddress QAPI structure.
> > > 
> > > Two new netdev backends, "stream" and "dgram" are added, that are barely a copy of "socket"
> > > backend but they use the SocketAddress QAPI to provide socket parameters.
> > > And then they also implement unix sockets (TCP and UDP).
> > 
> > Had you considered a -netdev chardev?
> > 
> 
> I think by definition a "chardev" doesn't behave like a "netdev". Moreover
> "chardev" is already a frontend for several backends (socket, udp, ...),
> this would mean we use the frontend "chardev" as a backend of a "netdev".
> More and more layers...

Yeh definitely more layers; but perhaps avoiding some duplication.

> And in the case of "-netdev dgram", we can use unix socket and
> sendto()/recv() while something like "-chardev udp,id=char0 -netdev
> chardev,chardev=char0,id=net0" doesn't support unix (see
> qio_channel_socket_dgram_sync()/socket_dgram()) and uses a
> "connect()/sendmsg()/recv()" (that really changes the behaviour of the
> backend)

It was -chardev socket, path=/unix/socket/path    that I was thinking
of; -chardev socket supports both tcp and unix already.

> The aim of this series is to add unix socket support.

Yes.

Dave

> Thanks,
> Laurent
> 
-- 
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

On Tue, Jun 21, 2022 at 11:31:36AM +0100, Dr. David Alan Gilbert wrote:
> * Laurent Vivier (lvivier@redhat.com) wrote:
> > On 20/06/2022 20:24, Dr. David Alan Gilbert wrote:
> > > * Laurent Vivier (lvivier@redhat.com) wrote:
> > > > "-netdev socket" only supports inet sockets.
> > > > 
> > > > It's not a complex task to add support for unix sockets, but
> > > > the socket netdev parameters are not defined to manage well unix
> > > > socket parameters.
> > > > 
> > > > As discussed in:
> > > > 
> > > >    "socket.c added support for unix domain socket datagram transport"
> > > >    https://lore.kernel.org/qemu-devel/1C0E1BC5-904F-46B0-8044-68E43E67BE60@gmail.com/
> > > > 
> > > > This series adds support of unix socket type using SocketAddress QAPI structure.
> > > > 
> > > > Two new netdev backends, "stream" and "dgram" are added, that are barely a copy of "socket"
> > > > backend but they use the SocketAddress QAPI to provide socket parameters.
> > > > And then they also implement unix sockets (TCP and UDP).
> > > 
> > > Had you considered a -netdev chardev?
> > > 
> > 
> > I think by definition a "chardev" doesn't behave like a "netdev". Moreover
> > "chardev" is already a frontend for several backends (socket, udp, ...),
> > this would mean we use the frontend "chardev" as a backend of a "netdev".
> > More and more layers...
> 
> Yeh definitely more layers; but perhaps avoiding some duplication.
> 
> > And in the case of "-netdev dgram", we can use unix socket and
> > sendto()/recv() while something like "-chardev udp,id=char0 -netdev
> > chardev,chardev=char0,id=net0" doesn't support unix (see
> > qio_channel_socket_dgram_sync()/socket_dgram()) and uses a
> > "connect()/sendmsg()/recv()" (that really changes the behaviour of the
> > backend)
> 
> It was -chardev socket, path=/unix/socket/path    that I was thinking
> of; -chardev socket supports both tcp and unix already.

IMHO we've over-used & abused chardevs in contexts where we really
should not have done. The chardev API is passable when all you need
is a persistent bidirectional channel, but is a really bad fit for
backends wanting to be aware of the dynamic connection oriented
semantics that sockets offer. The hoops we've had to jump through
in places to deal with having chardevs open asynchronously or deal
with automatic chardev re-connection is quite gross.

Chardev in the past was convenient to use, because we were not so
great at doing CLI syntax modelling & implementation, so it was
useful to re-use the chardev code for socket address handling on
the CLI.  We also didn't historically have nice APIs for dealing
with sockets - if you didn't use chardevs, you were stuck with
the raw sockets APIs. With our aim for CLI to be modelled &
implemented with QAPI these days, that benefit of re-using chardevs
for CLI is largely eliminated.  With our QIOChannel APIs, the
benefits of re-using chardevs from an impl POV is also largely
eliminated.


With regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|

* Daniel P. Berrangé (berrange@redhat.com) wrote:
> On Tue, Jun 21, 2022 at 11:31:36AM +0100, Dr. David Alan Gilbert wrote:
> > * Laurent Vivier (lvivier@redhat.com) wrote:
> > > On 20/06/2022 20:24, Dr. David Alan Gilbert wrote:
> > > > * Laurent Vivier (lvivier@redhat.com) wrote:
> > > > > "-netdev socket" only supports inet sockets.
> > > > > 
> > > > > It's not a complex task to add support for unix sockets, but
> > > > > the socket netdev parameters are not defined to manage well unix
> > > > > socket parameters.
> > > > > 
> > > > > As discussed in:
> > > > > 
> > > > >    "socket.c added support for unix domain socket datagram transport"
> > > > >    https://lore.kernel.org/qemu-devel/1C0E1BC5-904F-46B0-8044-68E43E67BE60@gmail.com/
> > > > > 
> > > > > This series adds support of unix socket type using SocketAddress QAPI structure.
> > > > > 
> > > > > Two new netdev backends, "stream" and "dgram" are added, that are barely a copy of "socket"
> > > > > backend but they use the SocketAddress QAPI to provide socket parameters.
> > > > > And then they also implement unix sockets (TCP and UDP).
> > > > 
> > > > Had you considered a -netdev chardev?
> > > > 
> > > 
> > > I think by definition a "chardev" doesn't behave like a "netdev". Moreover
> > > "chardev" is already a frontend for several backends (socket, udp, ...),
> > > this would mean we use the frontend "chardev" as a backend of a "netdev".
> > > More and more layers...
> > 
> > Yeh definitely more layers; but perhaps avoiding some duplication.
> > 
> > > And in the case of "-netdev dgram", we can use unix socket and
> > > sendto()/recv() while something like "-chardev udp,id=char0 -netdev
> > > chardev,chardev=char0,id=net0" doesn't support unix (see
> > > qio_channel_socket_dgram_sync()/socket_dgram()) and uses a
> > > "connect()/sendmsg()/recv()" (that really changes the behaviour of the
> > > backend)
> > 
> > It was -chardev socket, path=/unix/socket/path    that I was thinking
> > of; -chardev socket supports both tcp and unix already.
> 
> IMHO we've over-used & abused chardevs in contexts where we really
> should not have done. The chardev API is passable when all you need
> is a persistent bidirectional channel, but is a really bad fit for
> backends wanting to be aware of the dynamic connection oriented
> semantics that sockets offer. The hoops we've had to jump through
> in places to deal with having chardevs open asynchronously or deal
> with automatic chardev re-connection is quite gross.
> 
> Chardev in the past was convenient to use, because we were not so
> great at doing CLI syntax modelling & implementation, so it was
> useful to re-use the chardev code for socket address handling on
> the CLI.  We also didn't historically have nice APIs for dealing
> with sockets - if you didn't use chardevs, you were stuck with
> the raw sockets APIs. With our aim for CLI to be modelled &
> implemented with QAPI these days, that benefit of re-using chardevs
> for CLI is largely eliminated.  With our QIOChannel APIs, the
> benefits of re-using chardevs from an impl POV is also largely
> eliminated.

OK, fair enough.

Dave

> 
> With regards,
> Daniel
> -- 
> |: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org         -o-            https://fstop138.berrange.com :|
> |: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|
> 
-- 
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK