device: 0.948
other: 0.947
assembly: 0.933
semantic: 0.925
instruction: 0.920
boot: 0.909
graphic: 0.904
socket: 0.903
network: 0.890
vnc: 0.880
mistranslation: 0.874
KVM: 0.859

qemu-user: $0 incorrectly always reports absolute path

We just ran into an issue with the Perl package on Debian/m68k when being built with qemu-user [1].

The problem can be boiled down to qemu-user always reporting absolute paths for the shell variable $0 no matter on how the command was invoked.

A simple reproducer is this:

On normal system (no emulation):

root@nofan:~> sh -c 'echo $0'
sh
root@nofan:~>

On qemu-user:

(sid-m68k-sbuild)root@nofan:/# sh -c 'echo $0'
/bin/sh
(sid-m68k-sbuild)root@nofan:/#

> [1] https://lists.debian.org/debian-68k/2019/07/msg00007.html

Tentative patch

On 7/9/19 1:54 PM, Laurent Vivier wrote:
> ** Patch added: "Enable binfmt-misc preserve-arg[0] flag"
>    https://bugs.launchpad.net/qemu/+bug/1835839/+attachment/5275869/+files/0001-linux-user-manage-binfmt-misc-preserve-arg-0-flags.patch

Thanks! I just tried the patch and ran the setup script with:

./scripts/qemu-binfmt-conf.sh --debian --qemu-path=/usr/bin --qemu-suffix=-static --preserve-arg0 yes

and:

root@nofan:~/qemu> systemctl restart binfmt-support.service 
root@nofan:~/qemu>

But still don't get the correct path:

(sid-m68k-sbuild)root@nofan:/# sh -c 'echo $0'
/bin/sh
(sid-m68k-sbuild)root@nofan:/#

Do I need to do anything else?

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - <email address hidden>
`. `'   Freie Universitaet Berlin - <email address hidden>
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913


Is the proposed patch backwards compatible (ie "old QEMU binary works with newer binfmt-misc registration" and "new QEMU binary works with older binfmt-misc registration") ? Because binfmt-misc stuff is whole-system but QEMU binaries are per-chroot, this kind of thing is awkward to change if we don't have back-compat (and typically the kernel semantics for these things often don't allow back-compat or any kind of migration-path to the new better setup :-( )


Le 09/07/2019 à 14:12, John Paul Adrian Glaubitz a écrit :
> On 7/9/19 1:54 PM, Laurent Vivier wrote:
>> ** Patch added: "Enable binfmt-misc preserve-arg[0] flag"
>>    https://bugs.launchpad.net/qemu/+bug/1835839/+attachment/5275869/+files/0001-linux-user-manage-binfmt-misc-preserve-arg-0-flags.patch
> 
> Thanks! I just tried the patch and ran the setup script with:
> 
> ./scripts/qemu-binfmt-conf.sh --debian --qemu-path=/usr/bin --qemu-
> suffix=-static --preserve-arg0 yes
> 
> and:
> 
> root@nofan:~/qemu> systemctl restart binfmt-support.service 
> root@nofan:~/qemu>

if you use systemctl, the parameter of "./scripts/qemu-binfmt-conf.sh"
must be "--systemd m68k" rather than "--debian".

> 
> But still don't get the correct path:
> 
> (sid-m68k-sbuild)root@nofan:/# sh -c 'echo $0'
> /bin/sh
> (sid-m68k-sbuild)root@nofan:/#

Well, I've tested that, and it should work...

Thanks,
LAurent



On 7/9/19 2:51 PM, Laurent Vivier wrote:
> if you use systemctl, the parameter of "./scripts/qemu-binfmt-conf.sh"
> must be "--systemd m68k" rather than "--debian".

I tried that and I now get:

root@nofan:/local_scratch/sid-m68k-sbuild> chroot .
chroot: failed to run command ‘/bin/bash’: No such file or directory
root@nofan:/local_scratch/sid-m68k-sbuild>

>> But still don't get the correct path:
>>
>> (sid-m68k-sbuild)root@nofan:/# sh -c 'echo $0'
>> /bin/sh
>> (sid-m68k-sbuild)root@nofan:/#
> 
> Well, I've tested that, and it should work...

Oh, I'm not arguing that. I'm sure the error is on my side ;). I'm just trying
to find out what I'm doing wrong.

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - <email address hidden>
`. `'   Freie Universitaet Berlin - <email address hidden>
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913


Le 09/07/2019 à 14:16, Peter Maydell a écrit :
> Is the proposed patch backwards compatible (ie "old QEMU binary works
> with newer binfmt-misc registration" and "new QEMU binary works with
> older binfmt-misc registration") ? Because binfmt-misc stuff is whole-
> system but QEMU binaries are per-chroot, this kind of thing is awkward
> to change if we don't have back-compat (and typically the kernel
> semantics for these things often don't allow back-compat or any kind of
> migration-path to the new better setup :-( )
> 

If you don't enable the preserve-arg[0] flag, old and new QEMU will work.

If you enable the flag, only new QEMU with -0/QEMU_ARGV0 will work.

The best solution would be to force preserve-arg[0] with open-binary
flag and rely on AT_FDEXEC to detect the binfmt-misc environment, but
this breaks compatibility with existing environment and old QEMU.

Regarding the "binfmt-misc stuff is whole-system" problem, I've proposed
months ago a kernel based solution [1] to have a configuration per
namespace (chroot), but no one seems really interested (I think
maintainer is afraid by potential security issues).

[1] ns: introduce binfmt_misc namespace
    https://patchwork.kernel.org/cover/10634807/


Le 09/07/2019 à 15:07, John Paul Adrian Glaubitz a écrit :
> On 7/9/19 2:51 PM, Laurent Vivier wrote:
>> if you use systemctl, the parameter of "./scripts/qemu-binfmt-conf.sh"
>> must be "--systemd m68k" rather than "--debian".
> 
> I tried that and I now get:
> 
> root@nofan:/local_scratch/sid-m68k-sbuild> chroot .
> chroot: failed to run command ‘/bin/bash’: No such file or directory
> root@nofan:/local_scratch/sid-m68k-sbuild>

What is the content of:

/etc/binfmt.d/qemu-m68k.conf
/proc/sys/fs/binfmt_misc/qemu-m68k

what is the result of "file sid-m68k-sbuild/usr/bin/qemu-m68k-static"?

Bonus: if you don't want to copy qemu-m68k-static into the chroot, you
can use "--persistent" with qemu-binfmt-conf.sh.

Thanks,
Laurent


On 7/9/19 4:01 PM, Laurent Vivier wrote:
> What is the content of:
> 
> /etc/binfmt.d/qemu-m68k.conf
> /proc/sys/fs/binfmt_misc/qemu-m68k

root@nofan:~> cat /etc/binfmt.d/qemu-m68k.conf
:qemu-m68k:M::\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x04:\xff\xff\xff\xff\xff\xff\xfe\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff:/usr/bin/qemu-m68k-static:P
root@nofan:~> cat /proc/sys/fs/binfmt_misc/qemu-m68k
enabled
interpreter /usr/bin/qemu-m68k-static
flags: OCF
offset 0
magic 7f454c4601020100000000000000000000020004
mask ffffffffffffff00fffffffffffffffffffeffff
root@nofan:~>

> what is the result of "file sid-m68k-sbuild/usr/bin/qemu-m68k-static"?
> 
> Bonus: if you don't want to copy qemu-m68k-static into the chroot, you
> can use "--persistent" with qemu-binfmt-conf.sh.
I'm doing that and I don't have a copy of qemu in the chroot.

But I think I forgot to pass --persistent to the script while testing.

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - <email address hidden>
`. `'   Freie Universitaet Berlin - <email address hidden>
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913


After building qemu-m68k, I did:

root@nofan:~/qemu> scripts/qemu-binfmt-conf.sh --systemd m68k --qemu-path=/usr/bin --qemu-suffix=-static --preserve-arg0 yes --persistent yes
Setting /usr/bin/qemu-m68k-static as binfmt interpreter for m68k for systemd-binfmt.service
root@nofan:~/qemu> rm /usr/bin/qemu-m68k-static
root@nofan:~/qemu> cp -av m68k-linux-user/qemu-m68k /usr/bin/qemu-m68k-static 
'm68k-linux-user/qemu-m68k' -> '/usr/bin/qemu-m68k-static'
root@nofan:~/qemu> systemctl restart binfmt-support.service 
root@nofan:~/qemu> chroot /local_scratch/sid-m68k-sbuild/
(sid-m68k-sbuild)root@nofan:/# sh -c 'echo $0'
/bin/sh
(sid-m68k-sbuild)root@nofan:/#

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - <email address hidden>
`. `'   Freie Universitaet Berlin - <email address hidden>
  `-    GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913


Le 09/07/2019 à 17:09, John Paul Adrian Glaubitz a écrit :
> On 7/9/19 4:01 PM, Laurent Vivier wrote:
>> What is the content of:
>>
>> /etc/binfmt.d/qemu-m68k.conf
>> /proc/sys/fs/binfmt_misc/qemu-m68k
> 
> root@nofan:~> cat /etc/binfmt.d/qemu-m68k.conf
> :qemu-m68k:M::\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x04:\xff\xff\xff\xff\xff\xff\xfe\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff:/usr/bin/qemu-m68k-static:P
> root@nofan:~> cat /proc/sys/fs/binfmt_misc/qemu-m68k
> enabled
> interpreter /usr/bin/qemu-m68k-static
> flags: OCF
> offset 0
> magic 7f454c4601020100000000000000000000020004
> mask ffffffffffffff00fffffffffffffffffffeffff

This is not consistent: you have 'P' in /etc/binfmt.d/qemu-m68k.conf but
'OCF' in /proc/sys/fs/binfmt_misc/qemu-m68k.

Check "systemctl status binfmt-support.service"

With 'P' when you change the binary you must reload the configuration
because the binary is loaded in memory once when the configuration is
updated.

Thanks,
Laurent



The QEMU project is currently considering to move its bug tracking to
another system. For this we need to know which bugs are still valid
and which could be closed already. Thus we are setting older bugs to
"Incomplete" now.

If you still think this bug report here is valid, then please switch
the state back to "New" within the next 60 days, otherwise this report
will be marked as "Expired". Or please mark it as "Fix Released" if
the problem has been solved with a newer version of QEMU already.

Thank you and sorry for the inconvenience.


Laurent's patch has been included here:
https://gitlab.com/qemu-project/qemu/-/commit/6e1c0d7b951e19c53b8467e8bc4b71ee73a394ea
So I assume we can mark this as fixed now.