1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
|
graphic: 0.721
other: 0.700
instruction: 0.666
device: 0.646
mistranslation: 0.632
assembly: 0.603
KVM: 0.590
vnc: 0.575
network: 0.573
socket: 0.563
semantic: 0.558
boot: 0.464
qemu 2.7.0 receives SIGABRT in qemu_coroutine_enter()
I've been experiencing frequent SIGABRTs (in addition to segfaults in #1671876) lately with qemu 2.7.0 running Ubuntu 16.04 guests. The crash usually happens in qemu_coroutine_enter(). I haven't seen this so far with any other guests or distros.
Here is one stack trace I obtained
--------------------------------------------------------------------------
(gdb) bt
#0 0x00007fd7cc676067 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007fd7cc677448 in __GI_abort () at abort.c:89
#2 0x0000556aed247b6c in qemu_coroutine_enter (co=0x7fd574300df0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:113
#3 0x0000556aed247e55 in qemu_co_queue_run_restart (co=0x7fd574300ce0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#4 0x0000556aed2479a9 in qemu_coroutine_enter (co=0x7fd574300ce0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#5 0x0000556aed247e74 in qemu_co_queue_run_restart (co=0x7fd589111670) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#6 0x0000556aed2479a9 in qemu_coroutine_enter (co=0x7fd589111670) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#7 0x0000556aed247e74 in qemu_co_queue_run_restart (co=0x7fd57430dba0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#8 0x0000556aed2479a9 in qemu_coroutine_enter (co=0x7fd57430dba0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#9 0x0000556aed247e74 in qemu_co_queue_run_restart (co=0x7fd589119130) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#10 0x0000556aed2479a9 in qemu_coroutine_enter (co=0x7fd589119130) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#11 0x0000556aed247e74 in qemu_co_queue_run_restart (co=0x7fd589117410) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#12 0x0000556aed2479a9 in qemu_coroutine_enter (co=0x7fd589117410) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#13 0x0000556aed247e74 in qemu_co_queue_run_restart (co=0x7fd577f00e00) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#14 0x0000556aed2479a9 in qemu_coroutine_enter (co=0x7fd577f00e00) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#15 0x0000556aed247fa0 in qemu_co_enter_next (queue=queue@entry=0x556aef34e5e0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:106
#16 0x0000556aed1e6060 in timer_cb (blk=0x556aef34e590, is_write=<optimized out>) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/block/throttle-groups.c:400
#17 0x0000556aed1a3615 in timerlist_run_timers (timer_list=0x556aef3bad40) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/qemu-timer.c:528
#18 0x0000556aed1a3679 in timerlistgroup_run_timers (tlg=tlg@entry=0x556af0738758) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/qemu-timer.c:564
#19 0x0000556aed1a3f47 in aio_dispatch (ctx=ctx@entry=0x556af0738610) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/aio-posix.c:357
#20 0x0000556aed1a40e8 in aio_poll (ctx=0x556af0738610, blocking=<optimized out>) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/aio-posix.c:479
#21 0x0000556aed005c79 in iothread_run (opaque=0x556af07383c0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/iothread.c:46
#22 0x00007fd7cc9f40a4 in start_thread (arg=0x7fd7aafff700) at pthread_create.c:403
#23 0x00007fd7cc72962d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
--------------------------------------------------------------------------
The code crashes here
--------------------------------------------------------------------------
void qemu_coroutine_enter(Coroutine *co)
{
Coroutine *self = qemu_coroutine_self();
CoroutineAction ret;
trace_qemu_coroutine_enter(self, co, co->entry_arg);
if (co->caller) {
fprintf(stderr, "Co-routine re-entered recursively\n");
abort(); <--- Code aborts here
}
[...]
}
--------------------------------------------------------------------------
Debugging further we see:
--------------------------------------------------------------------------
(gdb) frame 2
#2 0x0000556aed247b6c in qemu_coroutine_enter (co=0x7fd574300df0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:113
113 /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c: No such file or directory.
(gdb) print *co
$1 = {entry = 0x7fd793e95a58, entry_arg = 0x1, caller = 0x7fd793e95a38, pool_next = {sle_next = 0x10}, co_queue_wakeup = {sqh_first = 0x7fd6ebbd2000, sqh_last = 0x1000}, co_queue_next = {
sqe_next = 0x7fd6ebbd1000}}
(gdb) print *co->caller
$2 = {entry = 0x400400000001, entry_arg = 0xc546a20, caller = 0x0, pool_next = {sle_next = 0x0}, co_queue_wakeup = {sqh_first = 0x0, sqh_last = 0xffffea00061f7480}, co_queue_next = {sqe_next = 0x100000000000}}
(gdb) frame 4
#4 0x0000556aed2479a9 in qemu_coroutine_enter (co=0x7fd574300ce0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
119 in /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c
(gdb) print *co
$3 = {entry = 0xc00000053, entry_arg = 0x7fd500000001, caller = 0x7fd574300d88, pool_next = {sle_next = 0x7fd574300d90}, co_queue_wakeup = {sqh_first = 0x7fd6ebbd1000, sqh_last = 0x7fd574300e00},
co_queue_next = {sqe_next = 0xc546a20}}
(gdb) print *co->caller
$4 = {entry = 0x230095a58, entry_arg = 0x230095a38, caller = 0x187dd2000, pool_next = {sle_next = 0x187dd1000}, co_queue_wakeup = {sqh_first = 0x187dd0000, sqh_last = 0x187dcf000}, co_queue_next = {
sqe_next = 0x184970000}}
--------------------------------------------------------------------------
The question is, why did qemu_coroutine_enter not complain when in earlier calls co->caller was not NULL?
Another stack trace:
--------------------------------------------------------------------------
(gdb) bt
#0 0x00007f2f34690067 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007f2f34691448 in __GI_abort () at abort.c:89
#2 0x00005629b8260b6c in qemu_coroutine_enter (co=0x7f2cd6a00940) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:113
#3 0x00005629b8260e74 in qemu_co_queue_run_restart (co=0x7f2cd6a00880) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#4 0x00005629b82609a9 in qemu_coroutine_enter (co=0x7f2cd6a00880) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#5 0x00005629b8260e74 in qemu_co_queue_run_restart (co=0x7f2cee202b00) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#6 0x00005629b82609a9 in qemu_coroutine_enter (co=0x7f2cee202b00) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#7 0x00005629b8260e74 in qemu_co_queue_run_restart (co=0x7f2cee2141d0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#8 0x00005629b82609a9 in qemu_coroutine_enter (co=0x7f2cee2141d0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#9 0x00005629b8260e74 in qemu_co_queue_run_restart (co=0x7f2cf300b370) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#10 0x00005629b82609a9 in qemu_coroutine_enter (co=0x7f2cf300b370) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#11 0x00005629b8260e74 in qemu_co_queue_run_restart (co=0x7f2cf1a03560) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#12 0x00005629b82609a9 in qemu_coroutine_enter (co=0x7f2cf1a03560) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#13 0x00005629b8260e74 in qemu_co_queue_run_restart (co=0x7f2cf3e15ba0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#14 0x00005629b82609a9 in qemu_coroutine_enter (co=0x7f2cf3e15ba0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#15 0x00005629b8260e74 in qemu_co_queue_run_restart (co=0x7f2ce80087f0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#16 0x00005629b82609a9 in qemu_coroutine_enter (co=0x7f2ce80087f0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#17 0x00005629b8260e74 in qemu_co_queue_run_restart (co=0x7f2cee20d9c0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#18 0x00005629b82609a9 in qemu_coroutine_enter (co=0x7f2cee20d9c0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#19 0x00005629b8260e74 in qemu_co_queue_run_restart (co=0x7f2ceff04850) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#20 0x00005629b82609a9 in qemu_coroutine_enter (co=0x7f2ceff04850) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#21 0x00005629b8260e74 in qemu_co_queue_run_restart (co=0x7f2cf21061c0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#22 0x00005629b82609a9 in qemu_coroutine_enter (co=0x7f2cf21061c0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#23 0x00005629b8260e74 in qemu_co_queue_run_restart (co=0x7f2cf2105c00) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#24 0x00005629b82609a9 in qemu_coroutine_enter (co=0x7f2cf2105c00) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#25 0x00005629b8260e74 in qemu_co_queue_run_restart (co=0x7f2cf3e1d590) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#26 0x00005629b82609a9 in qemu_coroutine_enter (co=0x7f2cf3e1d590) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#27 0x00005629b8260e74 in qemu_co_queue_run_restart (co=0x7f2cf3e16a00) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#28 0x00005629b82609a9 in qemu_coroutine_enter (co=0x7f2cf3e16a00) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#29 0x00005629b8260e74 in qemu_co_queue_run_restart (co=0x7f2ce8004da0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#30 0x00005629b82609a9 in qemu_coroutine_enter (co=0x7f2ce8004da0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#31 0x00005629b8260e74 in qemu_co_queue_run_restart (co=0x7f2cf3e15dc0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#32 0x00005629b82609a9 in qemu_coroutine_enter (co=0x7f2cf3e15dc0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#33 0x00005629b8260e74 in qemu_co_queue_run_restart (co=0x7f2ccff00420) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#34 0x00005629b82609a9 in qemu_coroutine_enter (co=0x7f2ccff00420) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#35 0x00005629b8260e74 in qemu_co_queue_run_restart (co=0x7f2cf1e04900) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#36 0x00005629b82609a9 in qemu_coroutine_enter (co=0x7f2cf1e04900) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#37 0x00005629b8260fa0 in qemu_co_enter_next (queue=queue@entry=0x5629ba5e35e0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:106
#38 0x00005629b81ff060 in timer_cb (blk=0x5629ba5e3590, is_write=<optimized out>) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/block/throttle-groups.c:400
#39 0x00005629b81bc615 in timerlist_run_timers (timer_list=0x5629ba64fd40) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/qemu-timer.c:528
#40 0x00005629b81bc679 in timerlistgroup_run_timers (tlg=tlg@entry=0x5629bb9cd758) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/qemu-timer.c:564
#41 0x00005629b81bcf47 in aio_dispatch (ctx=ctx@entry=0x5629bb9cd610) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/aio-posix.c:357
#42 0x00005629b81bd0e8 in aio_poll (ctx=0x5629bb9cd610, blocking=<optimized out>) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/aio-posix.c:479
#43 0x00005629b801ec79 in iothread_run (opaque=0x5629bb9cd3c0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/iothread.c:46
#44 0x00007f2f34a0e0a4 in start_thread (arg=0x7f2f12fff700) at pthread_create.c:403
#45 0x00007f2f3474362d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111
--------------------------------------------------------------------------
Looking at the data:
--------------------------------------------------------------------------
(gdb) frame 2
#2 0x00005629b8260b6c in qemu_coroutine_enter (co=0x7f2cd6a00940) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:113
113 /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c: No such file or directory.
(gdb) print *co
$1 = {entry = 0x7f2efbfbc4d8, entry_arg = 0x1, caller = 0x7f2efbfbc4b8, pool_next = {sle_next = 0x10}, co_queue_wakeup = {sqh_first = 0x7f2d217e2000, sqh_last = 0x1000}, co_queue_next = {sqe_next = 0x0}}
(gdb) print *co->caller
$2 = {entry = 0x1, entry_arg = 0xc21a480, caller = 0x0, pool_next = {sle_next = 0x0}, co_queue_wakeup = {sqh_first = 0x0, sqh_last = 0xffffea0000567882}, co_queue_next = {sqe_next = 0x100000000000}}
(gdb) frame 4
#4 0x00005629b82609a9 in qemu_coroutine_enter (co=0x7f2cd6a00880) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
119 in /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c
(gdb) print *co
$3 = {entry = 0x200000046, entry_arg = 0x7f2c00000001, caller = 0x7f2cd6a00928, pool_next = {sle_next = 0x7f2cd6a00930}, co_queue_wakeup = {sqh_first = 0x0, sqh_last = 0x7f2cd6a008a0}, co_queue_next = {
sqe_next = 0xc21a480}}
(gdb) print *co->caller
$4 = {entry = 0x2301bc4d8, entry_arg = 0x2301bc4b8, caller = 0x159e2000, pool_next = {sle_next = 0x7f2efbfbc4d8}, co_queue_wakeup = {sqh_first = 0x1, sqh_last = 0x7f2efbfbc4b8}, co_queue_next = {
sqe_next = 0x10}}
--------------------------------------------------------------------------
Same as above. If co->caller is not NULL in earlier calls, why have they succeeded?
Third stack trace:
--------------------------------------------------------------------------
#0 0x00007f4d5ad6a067 in raise () from /lib/x86_64-linux-gnu/libc.so.6
#1 0x00007f4d5ad6b448 in abort () from /lib/x86_64-linux-gnu/libc.so.6
#2 0x0000562a4c582b6c in qemu_coroutine_enter (co=0x7f4b1bf0a900) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:113
#3 0x0000562a4c582e55 in qemu_co_queue_run_restart (co=0x7f4b1bf0a830) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#4 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b1bf0a830) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#5 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b1bf0f4c0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#6 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b1bf0f4c0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#7 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b17e07c40) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#8 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b17e07c40) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#9 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b17e11420) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#10 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b17e11420) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#11 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b17e18c30) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#12 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b17e18c30) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#13 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b1bf07ea0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#14 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b1bf07ea0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#15 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b1000c0c0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#16 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b1000c0c0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#17 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b17e11b10) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#18 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b17e11b10) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#19 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b17e10500) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#20 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b17e10500) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#21 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b1bf0a610) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#22 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b1bf0a610) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#23 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b17e12820) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#24 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b17e12820) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#25 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b10002b10) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#26 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b10002b10) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#27 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b1000bfb0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#28 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b1000bfb0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#29 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b17e103f0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#30 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b17e103f0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#31 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b17e078b0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#32 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b17e078b0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#33 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4adfe02b00) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#34 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4adfe02b00) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#35 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b15701ae0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#36 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b15701ae0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#37 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b17e162f0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#38 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b17e162f0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#39 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b10009fe0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#40 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b10009fe0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#41 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b17e0b860) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#42 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b17e0b860) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#43 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b23f035c0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#44 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b23f035c0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#45 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b19e030c0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#46 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b19e030c0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#47 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b100051b0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#48 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b100051b0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#49 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4adfe03970) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#50 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4adfe03970) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#51 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b17e11a00) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#52 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b17e11a00) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#53 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b17e0e0a0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#54 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b17e0e0a0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#55 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b17e0ede0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#56 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b17e0ede0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#57 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4aeff00860) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#58 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4aeff00860) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
---Type <return> to continue, or q <return> to quit---
#59 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b17e0d6f0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#60 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b17e0d6f0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#61 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b17e0e490) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#62 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b17e0e490) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#63 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b17e17370) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#64 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b17e17370) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#65 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b17e15c40) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#66 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b17e15c40) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#67 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b07f00a80) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#68 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b07f00a80) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#69 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b15703250) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#70 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b15703250) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#71 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b17e17870) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#72 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b17e17870) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#73 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b15703140) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#74 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b15703140) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#75 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b17e0c210) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#76 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b17e0c210) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#77 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b17e08650) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#78 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b17e08650) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#79 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b17e07470) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#80 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b17e07470) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#81 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b15e03a10) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#82 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b15e03a10) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#83 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b17e11d90) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#84 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b17e11d90) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#85 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4b17e13d00) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#86 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b17e13d00) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#87 0x0000562a4c582e74 in qemu_co_queue_run_restart (co=0x7f4afbe02b00) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:60
#88 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4afbe02b00) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
#89 0x0000562a4c582fa0 in qemu_co_enter_next (queue=queue@entry=0x562a4d8e65e0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine-lock.c:106
#90 0x0000562a4c521060 in timer_cb (blk=0x562a4d8e6590, is_write=<optimized out>) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/block/throttle-groups.c:400
#91 0x0000562a4c4de615 in timerlist_run_timers (timer_list=0x562a4d952d40) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/qemu-timer.c:528
#92 0x0000562a4c4de679 in timerlistgroup_run_timers (tlg=tlg@entry=0x562a4ecd0758) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/qemu-timer.c:564
#93 0x0000562a4c4def47 in aio_dispatch (ctx=ctx@entry=0x562a4ecd0610) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/aio-posix.c:357
#94 0x0000562a4c4df0e8 in aio_poll (ctx=0x562a4ecd0610, blocking=<optimized out>) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/aio-posix.c:479
#95 0x0000562a4c340c79 in iothread_run (opaque=0x562a4ecd03c0) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/iothread.c:46
#96 0x00007f4d5b0e80a4 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#97 0x00007f4d5ae1d62d in clone () from /lib/x86_64-linux-gnu/libc.so.6
--------------------------------------------------------------------------
Looking at the data:
--------------------------------------------------------------------------
(gdb) frame 2
#2 0x0000562a4c582b6c in qemu_coroutine_enter (co=0x7f4b1bf0a900) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:113
113 /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c: No such file or directory.
(gdb) print *co
$1 = {entry = 0x7f4d23f20198, entry_arg = 0x1, caller = 0x7f4d23f20178, pool_next = {sle_next = 0x10}, co_queue_wakeup = {sqh_first = 0x7f4c605fb000, sqh_last = 0x1000}, co_queue_next = {
sqe_next = 0x7f4c584f5000}}
(gdb) print *co->caller
$2 = {entry = 0x400400000001, entry_arg = 0x5000ac0, caller = 0x0, pool_next = {sle_next = 0x0}, co_queue_wakeup = {sqh_first = 0x0, sqh_last = 0xffffea0005b1fec0}, co_queue_next = {sqe_next = 0x100000000000}}
(gdb) frame 4
#4 0x0000562a4c5829a9 in qemu_coroutine_enter (co=0x7f4b1bf0a830) at /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c:119
119 in /build/pb-qemu-pssKUp/pb-qemu-2.7.0/util/qemu-coroutine.c
(gdb) print *co
$5 = {entry = 0x7f4b180001d8, entry_arg = 0x7f4b180001d8, caller = 0x7f4b1bf0a8d8, pool_next = {sle_next = 0x7f4b1bf0a8e0}, co_queue_wakeup = {sqh_first = 0x7f4c584f5000, sqh_last = 0x7f4b1bf0a910},
co_queue_next = {sqe_next = 0x5000ac0}}
(gdb) print *co->caller
$6 = {entry = 0x230120198, entry_arg = 0x230120178, caller = 0x16c7fb000, pool_next = {sle_next = 0x1646f5000}, co_queue_wakeup = {sqh_first = 0x1718c2000, sqh_last = 0x7f4d23f20198}, co_queue_next = {
sqe_next = 0x1}}
--------------------------------------------------------------------------
Same thing.
On Wed, Mar 15, 2017 at 04:02:55PM -0000, Mohammed Gamal wrote:
> I've been experiencing frequent SIGABRTs (in addition to segfaults in
> #1671876) lately with qemu 2.7.0 running Ubuntu 16.04 guests. The crash
> usually happens in qemu_coroutine_enter(). I haven't seen this so far
> with any other guests or distros.
Please report this against the Ubuntu package, not the upstream QEMU
project.
If the abort reproduces with qemu.git/master or QEMU 2.9-rc0 then it
would be appropriate for the upstream QEMU bug tracker.
Thanks,
Stefan
Fixed by commit 528f449f590829b53ea01ed91817a695b540421d
|