[PATCH] linux-user: manage binfmt-misc preserve-arg[0] flag

Laurent Vivier posted 1 patch 3 years, 2 months ago
Test checkpatch failed
Patches applied successfully (tree, apply log)
git fetch https://github.com/patchew-project/qemu tags/patchew/20210222105004.1642234-1-laurent@vivier.eu
Maintainers: Laurent Vivier <laurent@vivier.eu>
linux-user/main.c           | 24 ++++++++++++++++++++
scripts/qemu-binfmt-conf.sh | 44 +++++++++++++++++++++++--------------
2 files changed, 51 insertions(+), 17 deletions(-)
[PATCH] linux-user: manage binfmt-misc preserve-arg[0] flag
Posted by Laurent Vivier 3 years, 2 months ago
Add --preserve-argv0 in qemu-binfmt-conf.sh to configure the preserve-argv0
flag.

This patch allows to use new flag in AT_FLAGS to detect if
preserve-argv0 is configured for this interpreter:
argv[0] (the full pathname provided by binfmt-misc) is removed and
replaced by argv[1] (the original argv[0] provided by binfmt-misc when
'P'/preserve-arg[0] is set)

For instance with this patch and kernel support for AT_FLAGS:

  $ sudo chroot m68k-chroot sh -c 'echo $0'
  sh

without this patch:

  $ sudo chroot m68k-chroot sh -c 'echo $0'
  /usr/bin/sh

The new flag is available in kernel (v5.12) since:
2347961b11d4 ("binfmt_misc: pass binfmt_misc flags to the interpreter")

This can be tested with something like:

  # cp ..../qemu-ppc /chroot/powerpc/jessie

  # qemu-binfmt-conf.sh --qemu-path / --systemd ppc --credential yes \
                        --persistent no --preserve-argv0 yes
  # systemctl restart systemd-binfmt.service
  # cat /proc/sys/fs/binfmt_misc/qemu-ppc
  enabled
  interpreter //qemu-ppc
  flags: POC
  offset 0
  magic 7f454c4601020100000000000000000000020014
  mask ffffffffffffff00fffffffffffffffffffeffff
  # chroot /chroot/powerpc/jessie  sh -c 'echo $0'
  sh

  # qemu-binfmt-conf.sh --qemu-path / --systemd ppc --credential yes \
                        --persistent no --preserve-argv0 no
  # systemctl restart systemd-binfmt.service
  # cat /proc/sys/fs/binfmt_misc/qemu-ppc
  enabled
  interpreter //qemu-ppc
  flags: OC
  offset 0
  magic 7f454c4601020100000000000000000000020014
  mask ffffffffffffff00fffffffffffffffffffeffff
  # chroot /chroot/powerpc/jessie  sh -c 'echo $0'
  /bin/sh

Signed-off-by: Laurent Vivier <laurent@vivier.eu>
---
 linux-user/main.c           | 24 ++++++++++++++++++++
 scripts/qemu-binfmt-conf.sh | 44 +++++++++++++++++++++++--------------
 2 files changed, 51 insertions(+), 17 deletions(-)

diff --git a/linux-user/main.c b/linux-user/main.c
index 81f48ff54ed4..b9015a5dbd3e 100644
--- a/linux-user/main.c
+++ b/linux-user/main.c
@@ -26,6 +26,7 @@
 #include <sys/syscall.h>
 #include <sys/resource.h>
 #include <sys/shm.h>
+#include <linux/binfmts.h>
 
 #include "qapi/error.h"
 #include "qemu.h"
@@ -49,6 +50,11 @@
 #include "cpu_loop-common.h"
 #include "crypto/init.h"
 
+#ifndef AT_FLAGS_PRESERVE_ARGV0
+#define AT_FLAGS_PRESERVE_ARGV0_BIT 0
+#define AT_FLAGS_PRESERVE_ARGV0 (1 << AT_FLAGS_PRESERVE_ARGV0_BIT)
+#endif
+
 char *exec_path;
 
 int singlestep;
@@ -631,6 +637,7 @@ int main(int argc, char **argv, char **envp)
     int execfd;
     int log_mask;
     unsigned long max_reserved_va;
+    bool preserve_argv0;
 
     error_init(argv[0]);
     module_call_init(MODULE_INIT_TRACE);
@@ -687,6 +694,9 @@ int main(int argc, char **argv, char **envp)
 
     init_qemu_uname_release();
 
+    /*
+     * Manage binfmt-misc open-binary flag
+     */
     execfd = qemu_getauxval(AT_EXECFD);
     if (execfd == 0) {
         execfd = open(exec_path, O_RDONLY);
@@ -696,6 +706,20 @@ int main(int argc, char **argv, char **envp)
         }
     }
 
+    /*
+     * get binfmt_misc flags
+     */
+    preserve_argv0 = !!(qemu_getauxval(AT_FLAGS) & AT_FLAGS_PRESERVE_ARGV0);
+
+    /*
+     * Manage binfmt-misc preserve-arg[0] flag
+     *    argv[optind]     full path to the binary
+     *    argv[optind + 1] original argv[0]
+     */
+    if (optind + 1 < argc && preserve_argv0) {
+        optind++;
+    }
+
     if (cpu_model == NULL) {
         cpu_model = cpu_get_model(get_elf_eflags(execfd));
     }
diff --git a/scripts/qemu-binfmt-conf.sh b/scripts/qemu-binfmt-conf.sh
index 7b5d54b88741..573b5dc6acd7 100755
--- a/scripts/qemu-binfmt-conf.sh
+++ b/scripts/qemu-binfmt-conf.sh
@@ -178,25 +178,27 @@ usage() {
 Usage: qemu-binfmt-conf.sh [--qemu-path PATH][--debian][--systemd CPU]
                            [--help][--credential yes|no][--exportdir PATH]
                            [--persistent yes|no][--qemu-suffix SUFFIX]
+                           [--preserve-argv0 yes|no]
 
        Configure binfmt_misc to use qemu interpreter
 
-       --help:        display this usage
-       --qemu-path:   set path to qemu interpreter ($QEMU_PATH)
-       --qemu-suffix: add a suffix to the default interpreter name
-       --debian:      don't write into /proc,
-                      instead generate update-binfmts templates
-       --systemd:     don't write into /proc,
-                      instead generate file for systemd-binfmt.service
-                      for the given CPU. If CPU is "ALL", generate a
-                      file for all known cpus
-       --exportdir:   define where to write configuration files
-                      (default: $SYSTEMDDIR or $DEBIANDIR)
-       --credential:  if yes, credential and security tokens are
-                      calculated according to the binary to interpret
-       --persistent:  if yes, the interpreter is loaded when binfmt is
-                      configured and remains in memory. All future uses
-                      are cloned from the open file.
+       --help:          display this usage
+       --qemu-path:     set path to qemu interpreter ($QEMU_PATH)
+       --qemu-suffix:   add a suffix to the default interpreter name
+       --debian:        don't write into /proc,
+                        instead generate update-binfmts templates
+       --systemd:       don't write into /proc,
+                        instead generate file for systemd-binfmt.service
+                        for the given CPU. If CPU is "ALL", generate a
+                        file for all known cpus
+       --exportdir:     define where to write configuration files
+                        (default: $SYSTEMDDIR or $DEBIANDIR)
+       --credential:    if yes, credential and security tokens are
+                        calculated according to the binary to interpret
+       --persistent:    if yes, the interpreter is loaded when binfmt is
+                        configured and remains in memory. All future uses
+                        are cloned from the open file.
+       --preserve-argv0 preserve argv[0]
 
     To import templates with update-binfmts, use :
 
@@ -269,6 +271,9 @@ qemu_generate_register() {
     if [ "$PERSISTENT" = "yes" ] ; then
         flags="${flags}F"
     fi
+    if [ "$PRESERVE_ARG0" = "yes" ] ; then
+        flags="${flags}P"
+    fi
 
     echo ":qemu-$cpu:M::$magic:$mask:$qemu:$flags"
 }
@@ -330,9 +335,10 @@ DEBIANDIR="/usr/share/binfmts"
 QEMU_PATH=/usr/local/bin
 CREDENTIAL=no
 PERSISTENT=no
+PRESERVE_ARG0=no
 QEMU_SUFFIX=""
 
-options=$(getopt -o ds:Q:S:e:hc:p: -l debian,systemd:,qemu-path:,qemu-suffix:,exportdir:,help,credential:,persistent: -- "$@")
+options=$(getopt -o ds:Q:S:e:hc:p:g: -l debian,systemd:,qemu-path:,qemu-suffix:,exportdir:,help,credential:,persistent:,preserve-argv0: -- "$@")
 eval set -- "$options"
 
 while true ; do
@@ -388,6 +394,10 @@ while true ; do
         shift
         PERSISTENT="$1"
         ;;
+    -g|--preserve-argv0)
+        shift
+        PRESERVE_ARG0="$1"
+        ;;
     *)
         break
         ;;
-- 
2.29.2


Re: [PATCH] linux-user: manage binfmt-misc preserve-arg[0] flag
Posted by no-reply@patchew.org 3 years, 2 months ago
Patchew URL: https://patchew.org/QEMU/20210222105004.1642234-1-laurent@vivier.eu/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Message-id: 20210222105004.1642234-1-laurent@vivier.eu
Subject: [PATCH] linux-user: manage binfmt-misc preserve-arg[0] flag

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git rev-parse base > /dev/null || exit 0
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
From https://github.com/patchew-project/qemu
 - [tag update]      patchew/20210219184556.154972-1-berrange@redhat.com -> patchew/20210219184556.154972-1-berrange@redhat.com
 * [new tag]         patchew/20210222105004.1642234-1-laurent@vivier.eu -> patchew/20210222105004.1642234-1-laurent@vivier.eu
Switched to a new branch 'test'
a1035c1 linux-user: manage binfmt-misc preserve-arg[0] flag

=== OUTPUT BEGIN ===
ERROR: line over 90 characters
#195: FILE: scripts/qemu-binfmt-conf.sh:341:
+options=$(getopt -o ds:Q:S:e:hc:p:g: -l debian,systemd:,qemu-path:,qemu-suffix:,exportdir:,help,credential:,persistent:,preserve-argv0: -- "$@")

total: 1 errors, 0 warnings, 127 lines checked

Commit a1035c139838 (linux-user: manage binfmt-misc preserve-arg[0] flag) has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/20210222105004.1642234-1-laurent@vivier.eu/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [https://patchew.org/].
Please send your feedback to patchew-devel@redhat.com
Re: [PATCH] linux-user: manage binfmt-misc preserve-arg[0] flag
Posted by John Paul Adrian Glaubitz 3 years, 2 months ago
Hi Laurent!

On 2/22/21 11:50 AM, Laurent Vivier wrote:
> Add --preserve-argv0 in qemu-binfmt-conf.sh to configure the preserve-argv0
> flag.
> 
> This patch allows to use new flag in AT_FLAGS to detect if
> preserve-argv0 is configured for this interpreter:
> argv[0] (the full pathname provided by binfmt-misc) is removed and
> replaced by argv[1] (the original argv[0] provided by binfmt-misc when
> 'P'/preserve-arg[0] is set)

Would this patch finally fix the issue with the perl package in Debian? [1]

Adrian

> [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=974004

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913


Re: [PATCH] linux-user: manage binfmt-misc preserve-arg[0] flag
Posted by Laurent Vivier 3 years, 2 months ago
Le 22/02/2021 à 11:58, John Paul Adrian Glaubitz a écrit :
> Hi Laurent!
> 
> On 2/22/21 11:50 AM, Laurent Vivier wrote:
>> Add --preserve-argv0 in qemu-binfmt-conf.sh to configure the preserve-argv0
>> flag.
>>
>> This patch allows to use new flag in AT_FLAGS to detect if
>> preserve-argv0 is configured for this interpreter:
>> argv[0] (the full pathname provided by binfmt-misc) is removed and
>> replaced by argv[1] (the original argv[0] provided by binfmt-misc when
>> 'P'/preserve-arg[0] is set)
> 
> Would this patch finally fix the issue with the perl package in Debian? [1]

I think so. Helge can confirm that I guess.

But you need the kernel support (the one merged by Helge) as described in the patch message.

2347961b11d4 ("binfmt_misc: pass binfmt_misc flags to the interpreter")

> Adrian
> 
>> [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=974004
> 

Thanks,
Laurent

Re: [PATCH] linux-user: manage binfmt-misc preserve-arg[0] flag
Posted by Michael Tokarev 3 years, 2 months ago
22.02.2021 13:58, John Paul Adrian Glaubitz wrote:
> Hi Laurent!
> 
> On 2/22/21 11:50 AM, Laurent Vivier wrote:
>> Add --preserve-argv0 in qemu-binfmt-conf.sh to configure the preserve-argv0
>> flag.
>>
>> This patch allows to use new flag in AT_FLAGS to detect if
>> preserve-argv0 is configured for this interpreter:
>> argv[0] (the full pathname provided by binfmt-misc) is removed and
>> replaced by argv[1] (the original argv[0] provided by binfmt-misc when
>> 'P'/preserve-arg[0] is set)
> 
> Would this patch finally fix the issue with the perl package in Debian? [1]

It's been fixed a week or so ago.

/mjt

> Adrian
> 
>> [1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=974004
> 


Re: [PATCH] linux-user: manage binfmt-misc preserve-arg[0] flag
Posted by John Paul Adrian Glaubitz 3 years, 2 months ago
On 2/22/21 3:38 PM, Michael Tokarev wrote:
> 22.02.2021 13:58, John Paul Adrian Glaubitz wrote:
>> Hi Laurent!
>>
>> On 2/22/21 11:50 AM, Laurent Vivier wrote:
>>> Add --preserve-argv0 in qemu-binfmt-conf.sh to configure the preserve-argv0
>>> flag.
>>>
>>> This patch allows to use new flag in AT_FLAGS to detect if
>>> preserve-argv0 is configured for this interpreter:
>>> argv[0] (the full pathname provided by binfmt-misc) is removed and
>>> replaced by argv[1] (the original argv[0] provided by binfmt-misc when
>>> 'P'/preserve-arg[0] is set)
>>
>> Would this patch finally fix the issue with the perl package in Debian? [1]
> 
> It's been fixed a week or so ago.

Doesn't the patch require a kernel fix which is only present in Linux 5.12?

@Laurent: Could you help clarify the difference of both fixes?

Thanks,
Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913


Re: [PATCH] linux-user: manage binfmt-misc preserve-arg[0] flag
Posted by Michael Tokarev 3 years, 2 months ago
22.02.2021 17:43, John Paul Adrian Glaubitz wrote:
> On 2/22/21 3:38 PM, Michael Tokarev wrote:

>>
>> It's been fixed a week or so ago.
> 
> Doesn't the patch require a kernel fix which is only present in Linux 5.12?

No it does not. My approach does not require kernel support but it relies
on special argv[0] for the qemu binfmt interpreter.

/mjt

> @Laurent: Could you help clarify the difference of both fixes?
> 
> Thanks,
> Adrian
> 


Re: [PATCH] linux-user: manage binfmt-misc preserve-arg[0] flag
Posted by John Paul Adrian Glaubitz 3 years, 2 months ago
On 2/22/21 3:46 PM, Michael Tokarev wrote:
> 22.02.2021 17:43, John Paul Adrian Glaubitz wrote:
>> On 2/22/21 3:38 PM, Michael Tokarev wrote:
> 
>>>
>>> It's been fixed a week or so ago.
>>
>> Doesn't the patch require a kernel fix which is only present in Linux 5.12?
> 
> No it does not. My approach does not require kernel support but it relies
> on special argv[0] for the qemu binfmt interpreter.

OK, gotcha. Is it supposed to work with systemd-binfmt? It looks like it depends
on the old binfmt-support package.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913


Re: [PATCH] linux-user: manage binfmt-misc preserve-arg[0] flag
Posted by Michael Tokarev 3 years, 2 months ago
22.02.2021 17:54, John Paul Adrian Glaubitz wrote:

> OK, gotcha. Is it supposed to work with systemd-binfmt? It looks like it depends
> on the old binfmt-support package.

the qemu 4-line patch does not depend on any particular system, it relies on a
special name of its own argv[0] when registering the binfmt entry.  In order to
utilize it, we create a special-named symlink to qemu-foo and register that one
with the binfmt-misc subsystem, no matter if it is systemd or binfmt-support or
whatever else.

/mjt

Re: [PATCH] linux-user: manage binfmt-misc preserve-arg[0] flag
Posted by John Paul Adrian Glaubitz 3 years, 2 months ago
On 2/22/21 3:58 PM, Michael Tokarev wrote:
> 22.02.2021 17:54, John Paul Adrian Glaubitz wrote:
> 
>> OK, gotcha. Is it supposed to work with systemd-binfmt? It looks like it depends
>> on the old binfmt-support package.
> 
> the qemu 4-line patch does not depend on any particular system, it relies on a
> special name of its own argv[0] when registering the binfmt entry.  In order to
> utilize it, we create a special-named symlink to qemu-foo and register that one
> with the binfmt-misc subsystem, no matter if it is systemd or binfmt-support or
> whatever else.

OK, I was wondering this because qemu-user-static still pulls in the binfmt-support
package:

root@epyc:~> apt install qemu-user-static 
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  binfmt-support
The following NEW packages will be installed:
  binfmt-support qemu-user-static
0 upgraded, 2 newly installed, 0 to remove and 2 not upgraded.

Other distributions such as Fedora have already switched to systemd-binfmt.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913


Re: [PATCH] linux-user: manage binfmt-misc preserve-arg[0] flag
Posted by Michael Tokarev 3 years, 2 months ago
22.02.2021 18:09, John Paul Adrian Glaubitz wrote:

> OK, I was wondering this because qemu-user-static still pulls in the binfmt-support
> package:

sure. Debian does not _depend_ on systemd still (I don't want this flamewar). To me
it is binfmt-support which should adopt to systemd-binfmt, translating its actions
to the systemd equivalents, - this way there will be no need to implement *both*
binfmt-support and systemd-binfmt in every package using binfmts.

> Other distributions such as Fedora have already switched to systemd-binfmt.

This is true for distributions who completely switched to systemd. In Debian
it is optional and Debian still can use runit or sysvinit or something else.
At least in theory :)

Thanks,

/mjt

Re: [PATCH] linux-user: manage binfmt-misc preserve-arg[0] flag
Posted by John Paul Adrian Glaubitz 3 years, 2 months ago
Hi Michael!

On 2/22/21 3:58 PM, Michael Tokarev wrote:
> 22.02.2021 17:54, John Paul Adrian Glaubitz wrote:
> 
>> OK, gotcha. Is it supposed to work with systemd-binfmt? It looks like it depends
>> on the old binfmt-support package.
> 
> the qemu 4-line patch does not depend on any particular system, it relies on a
> special name of its own argv[0] when registering the binfmt entry.  In order to
> utilize it, we create a special-named symlink to qemu-foo and register that one
> with the binfmt-misc subsystem, no matter if it is systemd or binfmt-support or
> whatever else.

I have been trying to get qemu-user working with sbuild as it is shipped in Debian
unstable now but I didn't have any success.

Do you have some instructions somewhere how to get qemu-user working with sbuild?

My standard method [1] no longer works with the qemu-user-static package that is
shipped in unstable now:

> E: 15binfmt: update-binfmts: unable to open /var/run/schroot/mount/sid-m68k-sbuild-b1484996-cb57-436b-b491-60665add9bb8/bin/sh: No such file or directory
> E: Failed to execute “/usr/bin/getent”: No such file or directory
> E: Failed to execute “/usr/sbin/groupadd”: No such file or directory
> E: Failed to create group sbuild

Adrian

> [1] https://wiki.debian.org/M68k/sbuildQEMU

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913


Re: [PATCH] linux-user: manage binfmt-misc preserve-arg[0] flag
Posted by Michael Tokarev 3 years, 2 months ago
01.03.2021 13:35, John Paul Adrian Glaubitz wrote:
..
> I have been trying to get qemu-user working with sbuild as it is shipped in Debian
> unstable now but I didn't have any success.
> 
> Do you have some instructions somewhere how to get qemu-user working with sbuild?

Have you seen #983087 which I fixed yesterday?

/mjt

Re: [PATCH] linux-user: manage binfmt-misc preserve-arg[0] flag
Posted by John Paul Adrian Glaubitz 3 years, 2 months ago
On 3/1/21 11:40 AM, Michael Tokarev wrote:
> 01.03.2021 13:35, John Paul Adrian Glaubitz wrote:
> ..
>> I have been trying to get qemu-user working with sbuild as it is shipped in Debian
>> unstable now but I didn't have any success.
>>
>> Do you have some instructions somewhere how to get qemu-user working with sbuild?
> 
> Have you seen #983087 which I fixed yesterday?

Thanks, but it doesn't help, unfortunately.

Do I need to use qemu-user-static or qemu-user-binfmt?

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913


Re: [PATCH] linux-user: manage binfmt-misc preserve-arg[0] flag
Posted by Michael Tokarev 3 years, 2 months ago
01.03.2021 14:07, John Paul Adrian Glaubitz wrote:
> On 3/1/21 11:40 AM, Michael Tokarev wrote:
>> 01.03.2021 13:35, John Paul Adrian Glaubitz wrote:
>> ..
>>> I have been trying to get qemu-user working with sbuild as it is shipped in Debian
>>> unstable now but I didn't have any success.
>>>
>>> Do you have some instructions somewhere how to get qemu-user working with sbuild?
>>
>> Have you seen #983087 which I fixed yesterday?
> 
> Thanks, but it doesn't help, unfortunately.
> 
> Do I need to use qemu-user-static or qemu-user-binfmt?

Oh. You tried to use qemu-user, not qemu-user-static..
Well.

In order for it to work it should be registered with the
in-kernel binfmt-misc subsystem. qemu-user-static package
does this automatically for you, but qemu-user does not,
for that with qemu-user you'll have to additionally install
qemu-user-binfmt package.

But I guess it wont work with qemu-user even if it is registered
(by means of installing qemu-user-binfmt or manual registration
or whatever), - because regular qemu-user binaries aren't
statically linked and hence require all the shared libraries
within the chroot in order to run.

I never tried to use regular (non-static) qemu-user with
foreign chroot, and I suspect it wont work because of that
very reason, and at least extra setup is needed (like
copying appropriate /lib/ld.so and libc&glib&Co to the
chroot).

This is all about how qemu-user works, be it debian or
any other distribution, - it is basically the same.
I can only guess the wiki page you mentioned is wrong
here.

Thanks,

/mjt


Re: [PATCH] linux-user: manage binfmt-misc preserve-arg[0] flag
Posted by John Paul Adrian Glaubitz 3 years, 2 months ago
On 3/1/21 12:16 PM, Michael Tokarev wrote:
> Oh. You tried to use qemu-user, not qemu-user-static..
> Well.

I tried both packages. I also tried systemd-binfmt and binfmt-support,
neither worked.

> This is all about how qemu-user works, be it debian or
> any other distribution, - it is basically the same.
> I can only guess the wiki page you mentioned is wrong
> here.

There isn't really much in the wiki besides installing qemu-user-static.

I was, however, able to fix it by recreating the chroot.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaubitz@debian.org
`. `'   Freie Universitaet Berlin - glaubitz@physik.fu-berlin.de
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913


Re: [PATCH] linux-user: manage binfmt-misc preserve-arg[0] flag
Posted by Helge Deller 3 years, 2 months ago
On 2/22/21 3:58 PM, Michael Tokarev wrote:
> 22.02.2021 17:54, John Paul Adrian Glaubitz wrote:
>
>> OK, gotcha. Is it supposed to work with systemd-binfmt? It looks like it depends
>> on the old binfmt-support package.
>
> the qemu 4-line patch does not depend on any particular system, it relies on a
> special name of its own argv[0] when registering the binfmt entry.  In order to
> utilize it, we create a special-named symlink to qemu-foo and register that one
> with the binfmt-misc subsystem, no matter if it is systemd or binfmt-support or
> whatever else.

... which is pretty hackish (although it apparently works; I haven't tested myself).

The big question remains:
Is this "hack" just a temporary workaround which should be kept, or
is the support via the kernel-patch from Laurent the long-term and better solution?

Helge

Re: [PATCH] linux-user: manage binfmt-misc preserve-arg[0] flag
Posted by Michael Tokarev 3 years, 2 months ago
22.02.2021 18:02, Helge Deller wrote:
> On 2/22/21 3:58 PM, Michael Tokarev wrote:
>> 22.02.2021 17:54, John Paul Adrian Glaubitz wrote:
>>
>>> OK, gotcha. Is it supposed to work with systemd-binfmt? It looks like it depends
>>> on the old binfmt-support package.
>>
>> the qemu 4-line patch does not depend on any particular system, it relies on a
>> special name of its own argv[0] when registering the binfmt entry.  In order to
>> utilize it, we create a special-named symlink to qemu-foo and register that one
>> with the binfmt-misc subsystem, no matter if it is systemd or binfmt-support or
>> whatever else.
> 
> ... which is pretty hackish (although it apparently works; I haven't tested myself).

While it is hackish at first glance, the more I think about it the less hackish it
becomes.  The thing is - we're talking about different behavour of some program when
it sees different argv[0]. We can just as well use the same mechanism to support this.

> The big question remains:
> Is this "hack" just a temporary workaround which should be kept, or
> is the support via the kernel-patch from Laurent the long-term and better solution?

I don't care either way or the other, and I don't see the Laurent's one as better,
either. It's a long-term issue where quite some bullets has been spent already and
quite some spears has been broken. I come across a solution which actually works and
let others who waited for this issue to be solved to do their work.

At the very least, the one requiring kernel patch does not work on current distributions,
while my version works just fine, and we'll keep it in debian for now since it allows
to unbreak quite some other things. If qemu upstream will take another approach it is
entirely fine with me (after all that's why I sent an RFC in the first place, - to show
another (at least one) possible solution and to hear comments, and to develop something
which is good in the end). Hopefully we'll be able to switch to a better solution once
it's settled and will be available within current distributions.

Thanks,

/mjt

Re: [PATCH] linux-user: manage binfmt-misc preserve-arg[0] flag
Posted by Laurent Vivier 3 years, 1 month ago
Le 22/02/2021 à 11:50, Laurent Vivier a écrit :
> Add --preserve-argv0 in qemu-binfmt-conf.sh to configure the preserve-argv0
> flag.
> 
> This patch allows to use new flag in AT_FLAGS to detect if
> preserve-argv0 is configured for this interpreter:
> argv[0] (the full pathname provided by binfmt-misc) is removed and
> replaced by argv[1] (the original argv[0] provided by binfmt-misc when
> 'P'/preserve-arg[0] is set)
> 
> For instance with this patch and kernel support for AT_FLAGS:
> 
>   $ sudo chroot m68k-chroot sh -c 'echo $0'
>   sh
> 
> without this patch:
> 
>   $ sudo chroot m68k-chroot sh -c 'echo $0'
>   /usr/bin/sh
> 
> The new flag is available in kernel (v5.12) since:
> 2347961b11d4 ("binfmt_misc: pass binfmt_misc flags to the interpreter")
> 
> This can be tested with something like:
> 
>   # cp ..../qemu-ppc /chroot/powerpc/jessie
> 
>   # qemu-binfmt-conf.sh --qemu-path / --systemd ppc --credential yes \
>                         --persistent no --preserve-argv0 yes
>   # systemctl restart systemd-binfmt.service
>   # cat /proc/sys/fs/binfmt_misc/qemu-ppc
>   enabled
>   interpreter //qemu-ppc
>   flags: POC
>   offset 0
>   magic 7f454c4601020100000000000000000000020014
>   mask ffffffffffffff00fffffffffffffffffffeffff
>   # chroot /chroot/powerpc/jessie  sh -c 'echo $0'
>   sh
> 
>   # qemu-binfmt-conf.sh --qemu-path / --systemd ppc --credential yes \
>                         --persistent no --preserve-argv0 no
>   # systemctl restart systemd-binfmt.service
>   # cat /proc/sys/fs/binfmt_misc/qemu-ppc
>   enabled
>   interpreter //qemu-ppc
>   flags: OC
>   offset 0
>   magic 7f454c4601020100000000000000000000020014
>   mask ffffffffffffff00fffffffffffffffffffeffff
>   # chroot /chroot/powerpc/jessie  sh -c 'echo $0'
>   /bin/sh
> 
> Signed-off-by: Laurent Vivier <laurent@vivier.eu>
> ---
>  linux-user/main.c           | 24 ++++++++++++++++++++
>  scripts/qemu-binfmt-conf.sh | 44 +++++++++++++++++++++++--------------
>  2 files changed, 51 insertions(+), 17 deletions(-)
> 
> diff --git a/linux-user/main.c b/linux-user/main.c
> index 81f48ff54ed4..b9015a5dbd3e 100644
> --- a/linux-user/main.c
> +++ b/linux-user/main.c
> @@ -26,6 +26,7 @@
>  #include <sys/syscall.h>
>  #include <sys/resource.h>
>  #include <sys/shm.h>
> +#include <linux/binfmts.h>
>  
>  #include "qapi/error.h"
>  #include "qemu.h"
> @@ -49,6 +50,11 @@
>  #include "cpu_loop-common.h"
>  #include "crypto/init.h"
>  
> +#ifndef AT_FLAGS_PRESERVE_ARGV0
> +#define AT_FLAGS_PRESERVE_ARGV0_BIT 0
> +#define AT_FLAGS_PRESERVE_ARGV0 (1 << AT_FLAGS_PRESERVE_ARGV0_BIT)
> +#endif
> +
>  char *exec_path;
>  
>  int singlestep;
> @@ -631,6 +637,7 @@ int main(int argc, char **argv, char **envp)
>      int execfd;
>      int log_mask;
>      unsigned long max_reserved_va;
> +    bool preserve_argv0;
>  
>      error_init(argv[0]);
>      module_call_init(MODULE_INIT_TRACE);
> @@ -687,6 +694,9 @@ int main(int argc, char **argv, char **envp)
>  
>      init_qemu_uname_release();
>  
> +    /*
> +     * Manage binfmt-misc open-binary flag
> +     */
>      execfd = qemu_getauxval(AT_EXECFD);
>      if (execfd == 0) {
>          execfd = open(exec_path, O_RDONLY);
> @@ -696,6 +706,20 @@ int main(int argc, char **argv, char **envp)
>          }
>      }
>  
> +    /*
> +     * get binfmt_misc flags
> +     */
> +    preserve_argv0 = !!(qemu_getauxval(AT_FLAGS) & AT_FLAGS_PRESERVE_ARGV0);
> +
> +    /*
> +     * Manage binfmt-misc preserve-arg[0] flag
> +     *    argv[optind]     full path to the binary
> +     *    argv[optind + 1] original argv[0]
> +     */
> +    if (optind + 1 < argc && preserve_argv0) {
> +        optind++;
> +    }
> +
>      if (cpu_model == NULL) {
>          cpu_model = cpu_get_model(get_elf_eflags(execfd));
>      }
> diff --git a/scripts/qemu-binfmt-conf.sh b/scripts/qemu-binfmt-conf.sh
> index 7b5d54b88741..573b5dc6acd7 100755
> --- a/scripts/qemu-binfmt-conf.sh
> +++ b/scripts/qemu-binfmt-conf.sh
> @@ -178,25 +178,27 @@ usage() {
>  Usage: qemu-binfmt-conf.sh [--qemu-path PATH][--debian][--systemd CPU]
>                             [--help][--credential yes|no][--exportdir PATH]
>                             [--persistent yes|no][--qemu-suffix SUFFIX]
> +                           [--preserve-argv0 yes|no]
>  
>         Configure binfmt_misc to use qemu interpreter
>  
> -       --help:        display this usage
> -       --qemu-path:   set path to qemu interpreter ($QEMU_PATH)
> -       --qemu-suffix: add a suffix to the default interpreter name
> -       --debian:      don't write into /proc,
> -                      instead generate update-binfmts templates
> -       --systemd:     don't write into /proc,
> -                      instead generate file for systemd-binfmt.service
> -                      for the given CPU. If CPU is "ALL", generate a
> -                      file for all known cpus
> -       --exportdir:   define where to write configuration files
> -                      (default: $SYSTEMDDIR or $DEBIANDIR)
> -       --credential:  if yes, credential and security tokens are
> -                      calculated according to the binary to interpret
> -       --persistent:  if yes, the interpreter is loaded when binfmt is
> -                      configured and remains in memory. All future uses
> -                      are cloned from the open file.
> +       --help:          display this usage
> +       --qemu-path:     set path to qemu interpreter ($QEMU_PATH)
> +       --qemu-suffix:   add a suffix to the default interpreter name
> +       --debian:        don't write into /proc,
> +                        instead generate update-binfmts templates
> +       --systemd:       don't write into /proc,
> +                        instead generate file for systemd-binfmt.service
> +                        for the given CPU. If CPU is "ALL", generate a
> +                        file for all known cpus
> +       --exportdir:     define where to write configuration files
> +                        (default: $SYSTEMDDIR or $DEBIANDIR)
> +       --credential:    if yes, credential and security tokens are
> +                        calculated according to the binary to interpret
> +       --persistent:    if yes, the interpreter is loaded when binfmt is
> +                        configured and remains in memory. All future uses
> +                        are cloned from the open file.
> +       --preserve-argv0 preserve argv[0]
>  
>      To import templates with update-binfmts, use :
>  
> @@ -269,6 +271,9 @@ qemu_generate_register() {
>      if [ "$PERSISTENT" = "yes" ] ; then
>          flags="${flags}F"
>      fi
> +    if [ "$PRESERVE_ARG0" = "yes" ] ; then
> +        flags="${flags}P"
> +    fi
>  
>      echo ":qemu-$cpu:M::$magic:$mask:$qemu:$flags"
>  }
> @@ -330,9 +335,10 @@ DEBIANDIR="/usr/share/binfmts"
>  QEMU_PATH=/usr/local/bin
>  CREDENTIAL=no
>  PERSISTENT=no
> +PRESERVE_ARG0=no
>  QEMU_SUFFIX=""
>  
> -options=$(getopt -o ds:Q:S:e:hc:p: -l debian,systemd:,qemu-path:,qemu-suffix:,exportdir:,help,credential:,persistent: -- "$@")
> +options=$(getopt -o ds:Q:S:e:hc:p:g: -l debian,systemd:,qemu-path:,qemu-suffix:,exportdir:,help,credential:,persistent:,preserve-argv0: -- "$@")
>  eval set -- "$options"
>  
>  while true ; do
> @@ -388,6 +394,10 @@ while true ; do
>          shift
>          PERSISTENT="$1"
>          ;;
> +    -g|--preserve-argv0)
> +        shift
> +        PRESERVE_ARG0="$1"
> +        ;;
>      *)
>          break
>          ;;
> 

Applied to my linux-user-for-6.0 branch.

Thanks,
Laurent


Re: [PATCH] linux-user: manage binfmt-misc preserve-arg[0] flag
Posted by Michael Tokarev 2 years, 7 months ago
> Add --preserve-argv0 in qemu-binfmt-conf.sh to configure the preserve-argv0
> flag.
...
> diff --git a/linux-user/main.c b/linux-user/main.c
> @@ -697,6 +707,20 @@ int main(int argc, char **argv, char **envp)
>          }
>      }
>  
> +    /*
> +     * get binfmt_misc flags
> +     */
> +    preserve_argv0 = !!(qemu_getauxval(AT_FLAGS) & AT_FLAGS_PRESERVE_ARGV0);
> +
> +    /*
> +     * Manage binfmt-misc preserve-arg[0] flag
> +     *    argv[optind]     full path to the binary
> +     *    argv[optind + 1] original argv[0]
> +     */
> +    if (optind + 1 < argc && preserve_argv0) {
> +        optind++;
> +    }

Please note: this code is executed after parse_args() which is called
way up. And parse_args were able to mess up with the options & optind.

This is sort of a corner case really, but we rely on argv[1][0] being
!= '-'.  I think it is better to explicitly omit a call to parse_args()
for the AT_FLAGS_PRESERVE_ARGV0 case.  But parse_args() apparently is
a misnomer, since it also parses $ENVironment variables - this part
apparently should be run either way.

I noticed this because this interferes with my change in this area that
enables similar functionality (detecting the binfmt usage) but without
requiring any kernel changes and working with any version of kernel (it
has been discussed previously) - with both my code and this code in place
and the patched kernel, we update optind TWICE, one in parse_args() and
second here.  This caused someone filesystem to be wiped out already due
to wrong options being used.

Thanks,

/mjt

Re: [PATCH] linux-user: manage binfmt-misc preserve-arg[0] flag
Posted by Michael Tokarev 3 years, 2 months ago
22.02.2021 13:50, Laurent Vivier wrote:
> Add --preserve-argv0 in qemu-binfmt-conf.sh to configure the preserve-argv0
> flag.
> 
> This patch allows to use new flag in AT_FLAGS to detect if
> preserve-argv0 is configured for this interpreter:
> argv[0] (the full pathname provided by binfmt-misc) is removed and
> replaced by argv[1] (the original argv[0] provided by binfmt-misc when
> 'P'/preserve-arg[0] is set)

A few days ago I sent an RFC patch which fixes this issue without kernel
patch, by registering special binary name in binfmt (note: that patch had
argv[1] & argv[2] swapped by mistake).

Here it is: https://lists.gnu.org/archive/html/qemu-devel/2021-02/msg04639.html

/mjt

Re: [PATCH] linux-user: manage binfmt-misc preserve-arg[0] flag
Posted by Laurent Vivier 3 years, 2 months ago
Le 22/02/2021 à 15:45, Michael Tokarev a écrit :
> 22.02.2021 13:50, Laurent Vivier wrote:
>> Add --preserve-argv0 in qemu-binfmt-conf.sh to configure the preserve-argv0
>> flag.
>>
>> This patch allows to use new flag in AT_FLAGS to detect if
>> preserve-argv0 is configured for this interpreter:
>> argv[0] (the full pathname provided by binfmt-misc) is removed and
>> replaced by argv[1] (the original argv[0] provided by binfmt-misc when
>> 'P'/preserve-arg[0] is set)
> 
> A few days ago I sent an RFC patch which fixes this issue without kernel
> patch, by registering special binary name in binfmt (note: that patch had
> argv[1] & argv[2] swapped by mistake).
> 
> Here it is: https://lists.gnu.org/archive/html/qemu-devel/2021-02/msg04639.html
>
In this case, we don't want to modify QEMU to manage special case based on the binary name but
instead use a wrapper:

- add a proper parameter to manage the argv0 case
  [like this one https://patchew.org/QEMU/20191024153847.31815-1-laurent@vivier.eu/
   but I think now it would be better to use a new flag rather than using -0 ""]

- write a wrapper and name it "/usr/libexec/qemu-binfmt/foo-binfmt-P"

- the wrapper will call qemu-foo with the new parameter:

something like (with the example patch above):

#include <stdio.h>
#include <unistd.h>
#include <string.h>

static const char *baseargv[] = {
        "-0",
        "",
};

int main(int argc, char **argv, char **envp) {
        char *newargv[argc + sizeof(baseargv) / sizeof(char *) + 1];
        int current = 0;

        newargv[current] = argv[0];
        current++;

        memcpy(&newargv[current], baseargv, sizeof(baseargv));
        current += sizeof(baseargv) / sizeof(char *);

        memcpy(&newargv[current], &argv[1], sizeof(*argv) * (argc - 1));
        current += argc - 1;

        newargv[current] = NULL;

        return execve("/qemu-foo", newargv, envp);
}

Thanks,
Laurent

Re: [PATCH] linux-user: manage binfmt-misc preserve-arg[0] flag
Posted by Michael Tokarev 3 years, 2 months ago
22.02.2021 20:09, Laurent Vivier wrote:

>> Here it is: https://lists.gnu.org/archive/html/qemu-devel/2021-02/msg04639.html
>>
> In this case, we don't want to modify QEMU to manage special case based on the binary name but
> instead use a wrapper:

A wrapper immediately defeats the purpose of the fix-binary flag, unfortunately,
requiring the quemu-foo binary within the chroot again.  Such a wrapper has been
used for quite some time by Suse and others before the fix-binary flag appeared
in kernel. It was the first approach I considered and rejected.

Thanks,

/mjt