other: 0.711
KVM: 0.631
graphic: 0.566
vnc: 0.552
performance: 0.521
debug: 0.503
permissions: 0.474
device: 0.459
semantic: 0.396
network: 0.375
files: 0.366
PID: 0.360
boot: 0.358
socket: 0.292

multithreading not working right under qemu user mode for sh4

In a multithreaded program running under qemu-sh4 (version 2.9.0), thread termination and/or pthread_join is not working right.

The attached program works natively on all kinds of platforms, and under qemu user mode emulation for at least alpha, armelhf, aarch64, powerpc64le.

How to reproduce:
- Compile the program: sh4-linux-gnu-gcc-5 -O -Wall -lpthread -o test-tls test-tls.c
- Set environment variables for running qemu-sh4.
- ~/inst-qemu/2.9.0/bin/qemu-sh4 test-tls

Expected behaviour: After the "Worker xxxxx dying" line, the main() function prints "OK", and the program terminates.

Actual behaviour (only on sh4): After the "Worker xxxxx dying" line, it hangs. Attaching gdb to qemu shows 15 threads with a stack trace like
#0  safe_syscall_base () at /build/qemu-2.9.0/linux-user/host/x86_64/safe-syscall.inc.S:75
#1  0x00005584f86f4c48 in safe_futex (uaddr=<optimized out>, op=op@entry=128, val=val@entry=2, timeout=<optimized out>, uaddr2=uaddr2@entry=0x0, 
    val3=val3@entry=-161181992) at /build/qemu-2.9.0/linux-user/syscall.c:921
#2  0x00005584f870353b in do_futex (val3=-161181992, uaddr2=4134624624, timeout=0, val=<optimized out>, op=<optimized out>, uaddr=<optimized out>)
    at /build/qemu-2.9.0/linux-user/syscall.c:7147
#3  do_syscall (cpu_env=<optimized out>, num=240, arg1=<optimized out>, arg2=<optimized out>, arg3=<optimized out>, arg4=0, arg5=-160342672, 
    arg6=-161181992, arg7=0, arg8=0) at /build/qemu-2.9.0/linux-user/syscall.c:11692
#4  0x00005584f86f454a in cpu_loop (env=env@entry=0x5584fb3d04f8) at /build/qemu-2.9.0/linux-user/main.c:2676
#5  0x00005584f86f5dd5 in clone_func (arg=0x7fff4d485c20) at /build/qemu-2.9.0/linux-user/syscall.c:6234
#6  0x00007f08f05a46ba in start_thread (arg=0x7f08f1368700) at pthread_create.c:333
#7  0x00007f08f02da3dd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109

and 1 thread with a stack trace like
#0  safe_syscall_base () at /build/qemu-2.9.0/linux-user/host/x86_64/safe-syscall.inc.S:75
#1  0x00005584f86f4c48 in safe_futex (uaddr=<optimized out>, op=op@entry=0, val=val@entry=18875, timeout=<optimized out>, uaddr2=uaddr2@entry=0x0, 
    val3=val3@entry=-161180376) at /build/qemu-2.9.0/linux-user/syscall.c:921
#2  0x00005584f870353b in do_futex (val3=-161180376, uaddr2=4135101768, timeout=0, val=<optimized out>, op=<optimized out>, uaddr=<optimized out>)
    at /build/qemu-2.9.0/linux-user/syscall.c:7147
#3  do_syscall (cpu_env=<optimized out>, num=240, arg1=<optimized out>, arg2=<optimized out>, arg3=<optimized out>, arg4=0, arg5=-159865528, 
    arg6=-161180376, arg7=0, arg8=0) at /build/qemu-2.9.0/linux-user/syscall.c:11692
#4  0x00005584f86f454a in cpu_loop (env=0x5584fb3b99a8) at /build/qemu-2.9.0/linux-user/main.c:2676
#5  0x00005584f86c12d3 in main (argc=<optimized out>, argv=0x7fff4d4878b8, envp=<optimized out>)
    at /build/qemu-2.9.0/linux-user/main.c:4860

and 1 thread with a stack trace like
#0  syscall () at ../sysdeps/unix/sysv/linux/x86_64/syscall.S:38
#1  0x00005584f876eab5 in qemu_futex_wait (val=<optimized out>, f=<optimized out>) at /build/qemu-2.9.0/include/qemu/futex.h:26
#2  qemu_event_wait (ev=ev@entry=0x5584faa43d84 <rcu_call_ready_event>) at /build/qemu-2.9.0/util/qemu-thread-posix.c:399
#3  0x00005584f87748ce in call_rcu_thread (opaque=<optimized out>) at /build/qemu-2.9.0/util/rcu.c:249
#4  0x00007f08f05a46ba in start_thread (arg=0x7f08eff62700) at pthread_create.c:333
#5  0x00007f08f02da3dd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109





Another gnulib test (test-lock) fails with an assertion inside glibc:
test-lock: pthread_mutex_lock.c:81: __pthread_mutex_lock: Assertion `mutex->__data.__owner == 0' failed.
qemu: uncaught target signal 6 (Aborted) - core dumped

Based on this, I would speculate that the problem is in the qemu emulation of the Linux system calls that glibc uses for multithreading or in the implementation of the primitives used by qemu_event_wait().

On 07/02/2017 12:53 PM, Bruno Haible wrote:
> In a multithreaded program running under qemu-sh4 (version 2.9.0),
> thread termination and/or pthread_join is not working right.

QEMU does not support the roll-back atomic sequences used by linux on most 
uniprocessor SH.

Nor do we support the ll/sc form used by SH4A, although fixing that should be 
trivial.


r~


A patch for this has apparently been included in QEMU v2.10:
https://git.qemu.org/?p=qemu.git;a=commitdiff;h=4bfa602bc2227f5b5a506a4
Is this issue now completely fixed (so that we can close this ticket), or is there something left to do?

This works fine in qemu-2.10:
$ ~/inst-qemu/2.10.0/bin/qemu-sh4 test-tls
...
Worker 0xecdff4c0 doing value swapping
Worker 0xecdff4c0 doing value swapping
Worker 0xecdff4c0 before final verify
Worker 0xecdff4c0 after  final verify
Worker 0xecdff4c0 dying.
 OK