[Qemu-devel] [PATCH v7 00/74] linux-user: Split do_syscall

Richard Henderson posted 74 patches 4 years, 11 months ago
Test docker-clang@ubuntu failed
Test asan passed
Test checkpatch failed
Test docker-mingw@fedora passed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20190519203726.20729-1-richard.henderson@linaro.org
Maintainers: Riku Voipio <riku.voipio@iki.fi>, Laurent Vivier <laurent@vivier.eu>
linux-user/syscall-defs.h      |  338 +++
linux-user/syscall.h           |  103 +
linux-user/strace.c            | 1569 +++-------
linux-user/syscall-fcntl.inc.c |  322 ++
linux-user/syscall-file.inc.c  | 1526 ++++++++++
linux-user/syscall-ioctl.inc.c |  873 ++++++
linux-user/syscall-ipc.inc.c   | 1088 +++++++
linux-user/syscall-mem.inc.c   |  244 ++
linux-user/syscall-proc.inc.c  |  758 +++++
linux-user/syscall-sig.inc.c   |  549 ++++
linux-user/syscall-time.inc.c  |   77 +
linux-user/syscall.c           | 5045 ++------------------------------
configure                      |   20 -
linux-user/strace.list         |  414 ---
14 files changed, 6512 insertions(+), 6414 deletions(-)
create mode 100644 linux-user/syscall-defs.h
create mode 100644 linux-user/syscall.h
create mode 100644 linux-user/syscall-fcntl.inc.c
create mode 100644 linux-user/syscall-file.inc.c
create mode 100644 linux-user/syscall-ioctl.inc.c
create mode 100644 linux-user/syscall-ipc.inc.c
create mode 100644 linux-user/syscall-mem.inc.c
create mode 100644 linux-user/syscall-proc.inc.c
create mode 100644 linux-user/syscall-sig.inc.c
create mode 100644 linux-user/syscall-time.inc.c
[Qemu-devel] [PATCH v7 00/74] linux-user: Split do_syscall
Posted by Richard Henderson 4 years, 11 months ago
Based-on: <20190519201953.20161-1-richard.henderson@linaro.org>
Aka "linux-user: path, clone, sparc, shmat fixes"

Version 6 was
https://lists.gnu.org/archive/html/qemu-devel/2019-01/msg04794.html

Since v5, I've fixed the ioctl failure that Laurent found, and
have done a few more syscalls.

I've tried to do more testing with LTP, but it's a really annoying
testsuite.  There are at least 3 tests that hang forever (before 
and after this patch set), and the host kernel's OOM handler kills
my login session, ending the test run.  This seems to be different
behaviour than I was seeing in January, when I did have test runs
that completed successfully.

OTOH, looking back at the v6 cover letter, I now see that I was
testing armhf and i386 as guests, whereas this time I was trying
aarch64 and ppc64 as guests.  32-bit guests unable to trigger the
host OOM killer?

Laurent, is there anything special that you're doing to see these
tests run to completion?


r~


Richard Henderson (74):
  linux-user: Setup split syscall infrastructure
  linux-user: Split out open, open_at
  linux-user: Share more code for open and openat
  linux-user: Tidy do_openat loop over fakes
  linux-user: Split out readlink, readlinkat
  linux-user: Split out close
  linux-user: Split out read, write
  linux-user: Reduce regpairs_aligned & target_offset64 ifdefs
  linux-user: Split out readv, writev
  linux-user: Split out pread64, pwrite64
  linux-user: Split out preadv, pwritev
  linux-user: Split out name_to_handle_at, open_by_handle_at
  linux-user: Split out ipc syscalls
  linux-user: Split out memory syscalls
  linux-user: Split out exit
  linux-user: Split out brk
  linux-user: Split out clone, fork, vfork
  linux-user: Split out wait4, waitid, waitpid
  linux-user: Implement rusage argument to waitid
  linux-user: Split out creat
  linux-user: Split out link, linkat
  linux-user: Split out unlink, unlinkat, rmdir
  linux-user: Split out execve
  linux-user: Implement execveat
  linux-user: Split out chdir
  linux-user: Split out time
  linux-user: Split out mknod, mknodat
  linux-user: Split out chmod, fchmod, fchmodat
  linux-user: Split out lseek, llseek
  linux-user: Split out getpid, getppid, getxpid
  linux-user: Split out mount
  linux-user: Split out umount, umount2
  linux-user: Split out stime
  linux-user: Split out alarm, pause
  linux-user: Split out utime, utimes, futimesat
  linux-user: Split out access, faccessat
  linux-user: Split out nice
  linux-user: Split out sync, syncfs
  linux-user: Split out kill
  linux-user: Split out rename, renameat, renameat2
  linux-user: Split out mkdir, mkdirat
  linux-user: Split out dup, dup2, dup3
  linux-user: Split out pipe, pipe2
  linux-user: Split out times
  linux-user: Split out acct
  linux-user: Move syscall_init to the end
  linux-user: Split out ioctl
  linux-user: Fix types in ioctl logging
  linux-user: Remove sentinel from ioctl_entries
  linux-user: Split out fcntl, fcntl64
  linux-user: Split out setpgid
  linux-user: Split out umask
  linux-user: Split out chroot
  linux-user: Split out getpgid, getpgrp
  linux-user: Split out getsid, setsid
  linux-user: Split out sigaction, rt_sigaction
  linux-user: Split out sgetmask, ssetmask
  linux-user: Split out sigprocmask, rt_sigprocmask
  linux-user: Split out sigpending, rt_sigpending
  linux-user: Split out sigsuspend, rt_sigsuspend
  linux-user: Split out rt_sigtimedwait
  linux-user: Split out rt_sigqueueinfo, rt_tgsigqueueinfo
  linux-user: Split out sigreturn, rt_sigreturn
  linux-user: Split out gethostname, sethostname
  linux-user: Split out getrlimit, setrlimit
  linux-user: Split out getrusage
  linux-user: Split out gettimeofday, settimeofday
  linux-user: Split out select, _newselect
  linux-user: Split out pselect6
  linux-user: Split out symlink, symlinkat
  linux-user: Split out swapon, swapoff
  linux-user: Split out reboot
  linux-user: Split out truncate, truncate64, ftruncate, ftruncate64
  linux-user: Split out getpriority, setpriority

 linux-user/syscall-defs.h      |  338 +++
 linux-user/syscall.h           |  103 +
 linux-user/strace.c            | 1569 +++-------
 linux-user/syscall-fcntl.inc.c |  322 ++
 linux-user/syscall-file.inc.c  | 1526 ++++++++++
 linux-user/syscall-ioctl.inc.c |  873 ++++++
 linux-user/syscall-ipc.inc.c   | 1088 +++++++
 linux-user/syscall-mem.inc.c   |  244 ++
 linux-user/syscall-proc.inc.c  |  758 +++++
 linux-user/syscall-sig.inc.c   |  549 ++++
 linux-user/syscall-time.inc.c  |   77 +
 linux-user/syscall.c           | 5045 ++------------------------------
 configure                      |   20 -
 linux-user/strace.list         |  414 ---
 14 files changed, 6512 insertions(+), 6414 deletions(-)
 create mode 100644 linux-user/syscall-defs.h
 create mode 100644 linux-user/syscall.h
 create mode 100644 linux-user/syscall-fcntl.inc.c
 create mode 100644 linux-user/syscall-file.inc.c
 create mode 100644 linux-user/syscall-ioctl.inc.c
 create mode 100644 linux-user/syscall-ipc.inc.c
 create mode 100644 linux-user/syscall-mem.inc.c
 create mode 100644 linux-user/syscall-proc.inc.c
 create mode 100644 linux-user/syscall-sig.inc.c
 create mode 100644 linux-user/syscall-time.inc.c

-- 
2.17.1


Re: [Qemu-devel] [PATCH v7 00/74] linux-user: Split do_syscall
Posted by Laurent Vivier 4 years, 11 months ago
On 19/05/2019 22:36, Richard Henderson wrote:
> Based-on: <20190519201953.20161-1-richard.henderson@linaro.org>
> Aka "linux-user: path, clone, sparc, shmat fixes"
> 
> Version 6 was
> https://lists.gnu.org/archive/html/qemu-devel/2019-01/msg04794.html
> 
> Since v5, I've fixed the ioctl failure that Laurent found, and
> have done a few more syscalls.
> 
> I've tried to do more testing with LTP, but it's a really annoying
> testsuite.  There are at least 3 tests that hang forever (before
> and after this patch set), and the host kernel's OOM handler kills
> my login session, ending the test run.  This seems to be different
> behaviour than I was seeing in January, when I did have test runs
> that completed successfully.
> 
> OTOH, looking back at the v6 cover letter, I now see that I was
> testing armhf and i386 as guests, whereas this time I was trying
> aarch64 and ppc64 as guests.  32-bit guests unable to trigger the
> host OOM killer?
> 
> Laurent, is there anything special that you're doing to see these
> tests run to completion?

I'm running them in a container ('unshare chroot') and aarch64 needs a 
patch that hasn't been merged:

   "[PATCH] linux-user: check valid address in access_ok()"
   https://patchew.org/QEMU/20190208173520.15007-1-remi@remlab.net/

I have a skipfile that contains:

futex_wait03
mmap13
mmap16
msgctl10
msgctl11
ptrace01
ptrace02
ptrace03
ptrace05
remap_file_pages01
remap_file_pages02
sendmsg01
fork13
fork14
creat07
mremap03

but I didn't update it for a while.

I will test as soon as possible your series in my environment.

Thanks,
Laurent

Re: [Qemu-devel] [PATCH v7 00/74] linux-user: Split do_syscall
Posted by Aleksandar Markovic 4 years, 11 months ago
On May 19, 2019 10:38 PM, "Richard Henderson" <richard.henderson@linaro.org>
wrote:
>
> Based-on: <20190519201953.20161-1-richard.henderson@linaro.org>
> Aka "linux-user: path, clone, sparc, shmat fixes"
>
> Version 6 was
> https://lists.gnu.org/archive/html/qemu-devel/2019-01/msg04794.html
>
> Since v5, I've fixed the ioctl failure that Laurent found, and
> have done a few more syscalls.
>
> I've tried to do more testing with LTP, but it's a really annoying
> testsuite.  There are at least 3 tests that hang forever (before
> and after this patch set), and the host kernel's OOM handler kills
> my login session, ending the test run.  This seems to be different
> behaviour than I was seeing in January, when I did have test runs
> that completed successfully.
>
> OTOH, looking back at the v6 cover letter, I now see that I was
> testing armhf and i386 as guests, whereas this time I was trying
> aarch64 and ppc64 as guests.  32-bit guests unable to trigger the
> host OOM killer?
>
> Laurent, is there anything special that you're doing to see these
> tests run to completion?
>

On surface, this series looks to be a refactoring series. However, this is
not the case: burried in various patches, are removals of some features
visible to the end user. Can you please list all such cases in the cover
letter, so that they can be easier identified and evaluated by reviewers?

Thanks,
Aleksandar

>
> r~
>
>
> Richard Henderson (74):
>   linux-user: Setup split syscall infrastructure
>   linux-user: Split out open, open_at
>   linux-user: Share more code for open and openat
>   linux-user: Tidy do_openat loop over fakes
>   linux-user: Split out readlink, readlinkat
>   linux-user: Split out close
>   linux-user: Split out read, write
>   linux-user: Reduce regpairs_aligned & target_offset64 ifdefs
>   linux-user: Split out readv, writev
>   linux-user: Split out pread64, pwrite64
>   linux-user: Split out preadv, pwritev
>   linux-user: Split out name_to_handle_at, open_by_handle_at
>   linux-user: Split out ipc syscalls
>   linux-user: Split out memory syscalls
>   linux-user: Split out exit
>   linux-user: Split out brk
>   linux-user: Split out clone, fork, vfork
>   linux-user: Split out wait4, waitid, waitpid
>   linux-user: Implement rusage argument to waitid
>   linux-user: Split out creat
>   linux-user: Split out link, linkat
>   linux-user: Split out unlink, unlinkat, rmdir
>   linux-user: Split out execve
>   linux-user: Implement execveat
>   linux-user: Split out chdir
>   linux-user: Split out time
>   linux-user: Split out mknod, mknodat
>   linux-user: Split out chmod, fchmod, fchmodat
>   linux-user: Split out lseek, llseek
>   linux-user: Split out getpid, getppid, getxpid
>   linux-user: Split out mount
>   linux-user: Split out umount, umount2
>   linux-user: Split out stime
>   linux-user: Split out alarm, pause
>   linux-user: Split out utime, utimes, futimesat
>   linux-user: Split out access, faccessat
>   linux-user: Split out nice
>   linux-user: Split out sync, syncfs
>   linux-user: Split out kill
>   linux-user: Split out rename, renameat, renameat2
>   linux-user: Split out mkdir, mkdirat
>   linux-user: Split out dup, dup2, dup3
>   linux-user: Split out pipe, pipe2
>   linux-user: Split out times
>   linux-user: Split out acct
>   linux-user: Move syscall_init to the end
>   linux-user: Split out ioctl
>   linux-user: Fix types in ioctl logging
>   linux-user: Remove sentinel from ioctl_entries
>   linux-user: Split out fcntl, fcntl64
>   linux-user: Split out setpgid
>   linux-user: Split out umask
>   linux-user: Split out chroot
>   linux-user: Split out getpgid, getpgrp
>   linux-user: Split out getsid, setsid
>   linux-user: Split out sigaction, rt_sigaction
>   linux-user: Split out sgetmask, ssetmask
>   linux-user: Split out sigprocmask, rt_sigprocmask
>   linux-user: Split out sigpending, rt_sigpending
>   linux-user: Split out sigsuspend, rt_sigsuspend
>   linux-user: Split out rt_sigtimedwait
>   linux-user: Split out rt_sigqueueinfo, rt_tgsigqueueinfo
>   linux-user: Split out sigreturn, rt_sigreturn
>   linux-user: Split out gethostname, sethostname
>   linux-user: Split out getrlimit, setrlimit
>   linux-user: Split out getrusage
>   linux-user: Split out gettimeofday, settimeofday
>   linux-user: Split out select, _newselect
>   linux-user: Split out pselect6
>   linux-user: Split out symlink, symlinkat
>   linux-user: Split out swapon, swapoff
>   linux-user: Split out reboot
>   linux-user: Split out truncate, truncate64, ftruncate, ftruncate64
>   linux-user: Split out getpriority, setpriority
>
>  linux-user/syscall-defs.h      |  338 +++
>  linux-user/syscall.h           |  103 +
>  linux-user/strace.c            | 1569 +++-------
>  linux-user/syscall-fcntl.inc.c |  322 ++
>  linux-user/syscall-file.inc.c  | 1526 ++++++++++
>  linux-user/syscall-ioctl.inc.c |  873 ++++++
>  linux-user/syscall-ipc.inc.c   | 1088 +++++++
>  linux-user/syscall-mem.inc.c   |  244 ++
>  linux-user/syscall-proc.inc.c  |  758 +++++
>  linux-user/syscall-sig.inc.c   |  549 ++++
>  linux-user/syscall-time.inc.c  |   77 +
>  linux-user/syscall.c           | 5045 ++------------------------------
>  configure                      |   20 -
>  linux-user/strace.list         |  414 ---
>  14 files changed, 6512 insertions(+), 6414 deletions(-)
>  create mode 100644 linux-user/syscall-defs.h
>  create mode 100644 linux-user/syscall.h
>  create mode 100644 linux-user/syscall-fcntl.inc.c
>  create mode 100644 linux-user/syscall-file.inc.c
>  create mode 100644 linux-user/syscall-ioctl.inc.c
>  create mode 100644 linux-user/syscall-ipc.inc.c
>  create mode 100644 linux-user/syscall-mem.inc.c
>  create mode 100644 linux-user/syscall-proc.inc.c
>  create mode 100644 linux-user/syscall-sig.inc.c
>  create mode 100644 linux-user/syscall-time.inc.c
>
> --
> 2.17.1
>
>
Re: [Qemu-devel] [PATCH v7 00/74] linux-user: Split do_syscall
Posted by Peter Maydell 4 years, 11 months ago
On Sun, 19 May 2019 at 21:39, Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> Based-on: <20190519201953.20161-1-richard.henderson@linaro.org>
> Aka "linux-user: path, clone, sparc, shmat fixes"
>
> Version 6 was
> https://lists.gnu.org/archive/html/qemu-devel/2019-01/msg04794.html
>
> Since v5, I've fixed the ioctl failure that Laurent found, and
> have done a few more syscalls.
>
> I've tried to do more testing with LTP, but it's a really annoying
> testsuite.  There are at least 3 tests that hang forever (before
> and after this patch set), and the host kernel's OOM handler kills
> my login session, ending the test run.  This seems to be different
> behaviour than I was seeing in January, when I did have test runs
> that completed successfully.

If these are different from the ones we're already skipping
in the skiplist documented at https://wiki.qemu.org/Testing/LTP
we should update that wiki page to also skip these new
hanging-tests (and whatever it is that eats all the memory).

thanks
-- PMM

Re: [Qemu-devel] [PATCH v7 00/74] linux-user: Split do_syscall
Posted by Aleksandar Markovic 4 years, 11 months ago
On May 20, 2019 11:43 AM, "Peter Maydell" <peter.maydell@linaro.org> wrote:
>
> On Sun, 19 May 2019 at 21:39, Richard Henderson
> <richard.henderson@linaro.org> wrote:
> >
> > Based-on: <20190519201953.20161-1-richard.henderson@linaro.org>
> > Aka "linux-user: path, clone, sparc, shmat fixes"
> >
> > Version 6 was
> > https://lists.gnu.org/archive/html/qemu-devel/2019-01/msg04794.html
> >
> > Since v5, I've fixed the ioctl failure that Laurent found, and
> > have done a few more syscalls.
> >
> > I've tried to do more testing with LTP, but it's a really annoying
> > testsuite.  There are at least 3 tests that hang forever (before
> > and after this patch set), and the host kernel's OOM handler kills
> > my login session, ending the test run.  This seems to be different
> > behaviour than I was seeing in January, when I did have test runs
> > that completed successfully.
>
> If these are different from the ones we're already skipping
> in the skiplist documented at https://wiki.qemu.org/Testing/LTP
> we should update that wiki page to also skip these new
> hanging-tests (and whatever it is that eats all the memory).
>

Peter, yes, it is definitely different. It has been a moving target lately.
May I suggest that we use a stable version of QEMU, and a stable version of
LTP, and create a skiplist for such stable configuration, include it in the
docs, explicitely noting used versions? We can continue doing that for each
QEMU release and whatever the latest LTP would be at the moment of that
QEMU release (maybe even keeping the previous skiplists - which could be
useful for debugging).

Regards,
Aleksandar

> thanks
> -- PMM
>