[PATCH v6 00/23] bsd-user: Implement mmap related system calls for FreeBSD.

Karim Taha posted 23 patches 7 months ago
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20230925182709.4834-1-kariem.taha2.7@gmail.com
Maintainers: Warner Losh <imp@bsdimp.com>, Kyle Evans <kevans@freebsd.org>
bsd-user/bsd-mem.c            | 104 ++++++++
bsd-user/bsd-mem.h            | 452 ++++++++++++++++++++++++++++++++++
bsd-user/freebsd/os-misc.h    |  94 +++++++
bsd-user/freebsd/os-syscall.c | 109 +++++++-
bsd-user/meson.build          |   1 +
bsd-user/mmap.c               |   2 +-
bsd-user/qemu-bsd.h           |  45 ++++
bsd-user/qemu.h               |   1 +
bsd-user/syscall_defs.h       |  39 +++
9 files changed, 842 insertions(+), 5 deletions(-)
create mode 100644 bsd-user/bsd-mem.c
create mode 100644 bsd-user/bsd-mem.h
create mode 100644 bsd-user/freebsd/os-misc.h
create mode 100644 bsd-user/qemu-bsd.h
[PATCH v6 00/23] bsd-user: Implement mmap related system calls for FreeBSD.
Posted by Karim Taha 7 months ago
Upstream the implementation of the following mmap system calls, from the
qemu-bsd-user fork:
   mmap(2), munmap(2),
   mprotect(2),
   msync(2),
   mlock(2), munlock(2), mlockall(2), munlockall(2), mincore(2),
   madvise(2),
   minherit(2),
   shm_open(2),shm_open2(2), shm_rename2(2), shm_unlink(2), shmget(2), shmctl(2), shmat(2),
   shmdt(2)
   brk(2)

Karim Taha (3):
  bsd-user: Implement shm_open2(2) system call
  bsd-user: Add bsd-mem.c to meson.build
  bsd-user: Implment madvise(2) to match the linux-user implementation.

Kyle Evans (1):
  bsd-user: Implement shm_rename(2) system call

Stacey Son (18):
  bsd-user: Implement struct target_ipc_perm
  bsd-user: Implement struct target_shmid_ds
  bsd-user: Declarations for ipc_perm and shmid_ds conversion functions
  bsd-user: Introduce freebsd/os-misc.h to the source tree
  bsd-user: Implement target_set_brk function in bsd-mem.c instead of
    os-syscall.c
  bsd-user: Implement ipc_perm conversion between host and target.
  bsd-user: Implement shmid_ds conversion between host and target.
  bsd-user: Introduce bsd-mem.h to the source tree
  bsd-user: Implement mmap(2) and munmap(2)
  bsd-user: Implement mprotect(2)
  bsd-user: Implement msync(2)
  bsd-user: Implement mlock(2), munlock(2), mlockall(2), munlockall(2),
    minherit(2)
  bsd-user: Implement mincore(2)
  bsd-user: Implement do_obreak function
  bsd-user: Implement shm_open(2)
  bsd-user: Implement shm_unlink(2) and shmget(2)
  bsd-user: Implement shmctl(2)
  bsd-user: Implement shmat(2) and shmdt(2)

Warner Losh (1):
  bsd-user: Add stubs for vadvise(), sbrk() and sstk()

 bsd-user/bsd-mem.c            | 104 ++++++++
 bsd-user/bsd-mem.h            | 452 ++++++++++++++++++++++++++++++++++
 bsd-user/freebsd/os-misc.h    |  94 +++++++
 bsd-user/freebsd/os-syscall.c | 109 +++++++-
 bsd-user/meson.build          |   1 +
 bsd-user/mmap.c               |   2 +-
 bsd-user/qemu-bsd.h           |  45 ++++
 bsd-user/qemu.h               |   1 +
 bsd-user/syscall_defs.h       |  39 +++
 9 files changed, 842 insertions(+), 5 deletions(-)
 create mode 100644 bsd-user/bsd-mem.c
 create mode 100644 bsd-user/bsd-mem.h
 create mode 100644 bsd-user/freebsd/os-misc.h
 create mode 100644 bsd-user/qemu-bsd.h

-- 
2.42.0
Re: [PATCH v6 00/23] bsd-user: Implement mmap related system calls for FreeBSD.
Posted by Warner Losh 7 months ago
On Mon, Sep 25, 2023 at 12:28 PM Karim Taha <kariem.taha2.7@gmail.com>
wrote:

> Upstream the implementation of the following mmap system calls, from the
> qemu-bsd-user fork:
>    mmap(2), munmap(2),
>    mprotect(2),
>    msync(2),
>    mlock(2), munlock(2), mlockall(2), munlockall(2), mincore(2),
>    madvise(2),
>    minherit(2),
>    shm_open(2),shm_open2(2), shm_rename2(2), shm_unlink(2), shmget(2),
> shmctl(2), shmat(2),
>    shmdt(2)
>    brk(2)
>
> Karim Taha (3):
>   bsd-user: Implement shm_open2(2) system call
>   bsd-user: Add bsd-mem.c to meson.build
>   bsd-user: Implment madvise(2) to match the linux-user implementation.
>
> Kyle Evans (1):
>   bsd-user: Implement shm_rename(2) system call
>
> Stacey Son (18):
>   bsd-user: Implement struct target_ipc_perm
>   bsd-user: Implement struct target_shmid_ds
>   bsd-user: Declarations for ipc_perm and shmid_ds conversion functions
>   bsd-user: Introduce freebsd/os-misc.h to the source tree
>   bsd-user: Implement target_set_brk function in bsd-mem.c instead of
>     os-syscall.c
>   bsd-user: Implement ipc_perm conversion between host and target.
>   bsd-user: Implement shmid_ds conversion between host and target.
>   bsd-user: Introduce bsd-mem.h to the source tree
>   bsd-user: Implement mmap(2) and munmap(2)
>   bsd-user: Implement mprotect(2)
>   bsd-user: Implement msync(2)
>   bsd-user: Implement mlock(2), munlock(2), mlockall(2), munlockall(2),
>     minherit(2)
>   bsd-user: Implement mincore(2)
>   bsd-user: Implement do_obreak function
>   bsd-user: Implement shm_open(2)
>   bsd-user: Implement shm_unlink(2) and shmget(2)
>   bsd-user: Implement shmctl(2)
>   bsd-user: Implement shmat(2) and shmdt(2)
>
> Warner Losh (1):
>   bsd-user: Add stubs for vadvise(), sbrk() and sstk()
>
>  bsd-user/bsd-mem.c            | 104 ++++++++
>  bsd-user/bsd-mem.h            | 452 ++++++++++++++++++++++++++++++++++
>  bsd-user/freebsd/os-misc.h    |  94 +++++++
>  bsd-user/freebsd/os-syscall.c | 109 +++++++-
>  bsd-user/meson.build          |   1 +
>  bsd-user/mmap.c               |   2 +-
>  bsd-user/qemu-bsd.h           |  45 ++++
>  bsd-user/qemu.h               |   1 +
>  bsd-user/syscall_defs.h       |  39 +++
>  9 files changed, 842 insertions(+), 5 deletions(-)
>  create mode 100644 bsd-user/bsd-mem.c
>  create mode 100644 bsd-user/bsd-mem.h
>  create mode 100644 bsd-user/freebsd/os-misc.h
>  create mode 100644 bsd-user/qemu-bsd.h
>

queued to bsd-user-topo

with minor conflicts from rebasing it and the proc system call changes onto
one branch.

I did fix one issue: in the blitz bsd-user fork branch, we called shm_open2
directly, which
you copied to this patch series. This works when compiling static, but not
when compiling
dynamically. In the blitz branch, we always do static building. But since
qemu's CI process
uses dynamic, there was an error. Turns out that shm_open2 is a 'hidden'
system call that's
used to implement other pseudo system calls. As such, it was purposely
hidden in the dynamic
case, exporting only the __sys_shm_open2 system call (normally there're
several ways to get
to these symbols for different threading models that aren't relevant for
this).  By the time I figured
out why it was failing, the history here, etc, it was easier to just make
the minor correction rather
than send it back to you for this one last trivial change. Especially since
the directions for
building bsd-user are recommend --static.

With these changes, I'm able to execute dynamically compiled hello-world:
% ./qemu-arm -L /vidpool/qemu/jails/jails/131armv7 ~/hello-13
Hello
%
which is the first time ever I've been able to run even a full trivial
program that's dynamically
linked. Well done!

Once I get these through the CI pipeline, I'll submit the pull request. And
then learn how to edit
the wiki page for the release notes :)


On Mon, Sep 25, 2023 at 12:28 PM Karim Taha <kariem.taha2.7@gmail.com>
wrote:

> Upstream the implementation of the following mmap system calls, from the
> qemu-bsd-user fork:
>    mmap(2), munmap(2),
>    mprotect(2),
>    msync(2),
>    mlock(2), munlock(2), mlockall(2), munlockall(2), mincore(2),
>    madvise(2),
>    minherit(2),
>    shm_open(2),shm_open2(2), shm_rename2(2), shm_unlink(2), shmget(2),
> shmctl(2), shmat(2),
>    shmdt(2)
>    brk(2)
>
> Karim Taha (3):
>   bsd-user: Implement shm_open2(2) system call
>   bsd-user: Add bsd-mem.c to meson.build
>   bsd-user: Implment madvise(2) to match the linux-user implementation.
>
> Kyle Evans (1):
>   bsd-user: Implement shm_rename(2) system call
>
> Stacey Son (18):
>   bsd-user: Implement struct target_ipc_perm
>   bsd-user: Implement struct target_shmid_ds
>   bsd-user: Declarations for ipc_perm and shmid_ds conversion functions
>   bsd-user: Introduce freebsd/os-misc.h to the source tree
>   bsd-user: Implement target_set_brk function in bsd-mem.c instead of
>     os-syscall.c
>   bsd-user: Implement ipc_perm conversion between host and target.
>   bsd-user: Implement shmid_ds conversion between host and target.
>   bsd-user: Introduce bsd-mem.h to the source tree
>   bsd-user: Implement mmap(2) and munmap(2)
>   bsd-user: Implement mprotect(2)
>   bsd-user: Implement msync(2)
>   bsd-user: Implement mlock(2), munlock(2), mlockall(2), munlockall(2),
>     minherit(2)
>   bsd-user: Implement mincore(2)
>   bsd-user: Implement do_obreak function
>   bsd-user: Implement shm_open(2)
>   bsd-user: Implement shm_unlink(2) and shmget(2)
>   bsd-user: Implement shmctl(2)
>   bsd-user: Implement shmat(2) and shmdt(2)
>
> Warner Losh (1):
>   bsd-user: Add stubs for vadvise(), sbrk() and sstk()
>
>  bsd-user/bsd-mem.c            | 104 ++++++++
>  bsd-user/bsd-mem.h            | 452 ++++++++++++++++++++++++++++++++++
>  bsd-user/freebsd/os-misc.h    |  94 +++++++
>  bsd-user/freebsd/os-syscall.c | 109 +++++++-
>  bsd-user/meson.build          |   1 +
>  bsd-user/mmap.c               |   2 +-
>  bsd-user/qemu-bsd.h           |  45 ++++
>  bsd-user/qemu.h               |   1 +
>  bsd-user/syscall_defs.h       |  39 +++
>  9 files changed, 842 insertions(+), 5 deletions(-)
>  create mode 100644 bsd-user/bsd-mem.c
>  create mode 100644 bsd-user/bsd-mem.h
>  create mode 100644 bsd-user/freebsd/os-misc.h
>  create mode 100644 bsd-user/qemu-bsd.h
>

queued to bsd-user-topo

with minor conflicts from rebasing it and the proc system call changes onto
one branch.

I did fix one issue: in the blitz bsd-user fork branch, we called shm_open2
directly, which
you copied to this patch series. This works when compiling static, but not
when compiling
dynamically. In the blitz branch, we always do static building. But since
qemu's CI process
uses dynamic, there was an error. Turns out that shm_open2 is a 'hidden'
system call that's
used to implement other pseudo system calls. As such, it was purposely
hidden in the dynamic
case, exporting only the __sys_shm_open2 system call (normally there's
several ways to get
to these symbols for different threading models that aren't relevant for
this).  By the time I figured
out why it was failing, the history here, etc, it was easier to just make
the minor correction rather
than send it back to you for this one last trivial change.

With these changes, I'm able to execute dynamically compiled hello-world:
% ./qemu-arm -L /vidpool/qemu/jails/jails/131armv7 ~/hello-13
Hello
%
which is one

Warner