diff options
Diffstat (limited to '')
| -rw-r--r-- | results/classifier/108/other/1378 | 35 | ||||
| -rw-r--r-- | results/classifier/108/other/1378407 | 29 | ||||
| -rw-r--r-- | results/classifier/108/other/1378554 | 203 |
3 files changed, 267 insertions, 0 deletions
diff --git a/results/classifier/108/other/1378 b/results/classifier/108/other/1378 new file mode 100644 index 000000000..09e35703f --- /dev/null +++ b/results/classifier/108/other/1378 @@ -0,0 +1,35 @@ +graphic: 0.821 +performance: 0.664 +semantic: 0.618 +device: 0.615 +KVM: 0.571 +PID: 0.456 +vnc: 0.425 +network: 0.338 +permissions: 0.334 +socket: 0.330 +other: 0.315 +boot: 0.286 +files: 0.211 +debug: 0.193 + +iSCSI causes memory corruption +Description of problem: +This is a compound problem, which most likely involves a combination of how TrueNAS SCALE handles iSCSI triggering a problem **and** some memory-handling issue in QEMU leading to a crash. In short any Linux machine started with iSCSI handled by QEMU directly leads to a hard crash within 30s-1h. I was able to find a pattern in logs: + +1. First, a message like `QEMU[53139]: kvm: iSCSI Busy/TaskSetFull/TimeOut (retry #1 in 0 ms): TASK_SET_FULL` is logged + - it is always `TASK_SET_FULL` + - it is always `retry #1 in ... ms`, where only number of miliseconds varies + - the line is repeated multiple times, sometimes 5x and sometimes >200x +2. It is followed by a single line with one of the following: + - `double free or corruption (out)` + - `double free or corruption (!prev)` + - `kvm: ../block/block-backend.c:1567: blk_aio_write_entry: Assertion `!qiov || qiov->size == acb->bytes' failed.` + - `kvm: malloc.c:2379: sysmalloc: Assertion `(old_top == initial_top (av) && old_size == 0) || ((unsigned long) (old_size) >= MINSIZE && prev_inuse (old_top) && ((unsigned long) old_end & (pagesize - 1)) == 0)' failed.` + - `kvm: iSCSI CheckCondition: SENSE KEY:UNIT_ATTENTION(6) ASCQ:BUS_RESET(0x2900)` + - `malloc(): invalid size (unsorted)` +3. The virtual machine crashes +Steps to reproduce: +I don't have a specific concrete steps, only clues really. This problem started happening after TrueNAS SCALE updated their iSCSI code in Bluefin release to a new upstream version. That iSCSI server still works when iSCSI is mounted by the kernel and QEMU uses a normal `/dev` entry. While there's probably some problem with it, QEMU shouldn't probably crash with memory errors. +Additional information: +While I'm a software developer, I don't code in C on a daily basis. However, looking at the errors, I have a suspicion the problem may be somewhere in the `iscsi_co_generic_cb()`, as it seems the struct is getting damaged (out of bound write?) and causes explosion somewhere down the line. diff --git a/results/classifier/108/other/1378407 b/results/classifier/108/other/1378407 new file mode 100644 index 000000000..b11208f25 --- /dev/null +++ b/results/classifier/108/other/1378407 @@ -0,0 +1,29 @@ +files: 0.885 +graphic: 0.706 +performance: 0.669 +boot: 0.649 +other: 0.639 +semantic: 0.638 +device: 0.543 +permissions: 0.520 +PID: 0.465 +network: 0.400 +vnc: 0.391 +debug: 0.336 +socket: 0.290 +KVM: 0.126 + +[feature request] Partition table wrapper for single-filesystem images + +Suppose you have a single filesystem image. It would be nice if QEMU could generate a virtual partition table for it and make it available to the guest as a partitioned disk. Otherwise you have to use workarounds like this: wiki.archlinux.org/index.php/QEMU#Simulate_virtual_disk_with_MBR_using_linear_RAID + +It should be relatively easy to do on top of existing vvfat code. + +This is a rather specific use case. Note that linux can use partitionless diskspace just fine, and depending on the bootmanager, one can use single partition as a virtual disk to boot linux too (syslinux supports this mode for one). Implementing this feature in qemu does not make much sense to me, unless it is a generic block device remapper like dm in kernel, which is rather complex. FWIW. + +The QEMU project is currently considering to move its bug tracking to another system. For this we need to know which bugs are still valid and which could be closed already. Thus we are setting older bugs to "Incomplete" now. +If you still think this bug report here is valid, then please switch the state back to "New" within the next 60 days, otherwise this report will be marked as "Expired". Or mark it as "Fix Released" if the problem has been solved with a newer version of QEMU already. Thank you and sorry for the inconvenience. + + +[Expired for QEMU because there has been no activity for 60 days.] + diff --git a/results/classifier/108/other/1378554 b/results/classifier/108/other/1378554 new file mode 100644 index 000000000..823c9c4d9 --- /dev/null +++ b/results/classifier/108/other/1378554 @@ -0,0 +1,203 @@ +other: 0.939 +graphic: 0.890 +KVM: 0.835 +performance: 0.834 +debug: 0.808 +vnc: 0.806 +semantic: 0.805 +permissions: 0.790 +device: 0.790 +boot: 0.780 +network: 0.775 +files: 0.773 +PID: 0.749 +socket: 0.712 + +qemu segfault in virtio_scsi_handle_cmd_req_submit on ARM 32 bit + +/home/rjones/d/qemu/arm-softmmu/qemu-system-arm \ + -global virtio-blk-device.scsi=off \ + -nodefconfig \ + -enable-fips \ + -nodefaults \ + -display none \ + -M virt \ + -machine accel=kvm:tcg \ + -m 500 \ + -no-reboot \ + -rtc driftfix=slew \ + -global kvm-pit.lost_tick_policy=discard \ + -kernel /home/rjones/d/libguestfs/tmp/.guestfs-1001/appliance.d/kernel \ + -initrd /home/rjones/d/libguestfs/tmp/.guestfs-1001/appliance.d/initrd \ + -device virtio-scsi-device,id=scsi \ + -drive file=/home/rjones/d/libguestfs/tmp/libguestfseV4fT5/scratch.1,cache=unsafe,format=raw,id=hd0,if=none \ + -device scsi-hd,drive=hd0 \ + -drive file=/home/rjones/d/libguestfs/tmp/.guestfs-1001/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none \ + -device scsi-hd,drive=appliance \ + -device virtio-serial-device \ + -serial stdio \ + -chardev socket,path=/home/rjones/d/libguestfs/tmp/libguestfseV4fT5/guestfsd.sock,id=channel0 \ + -device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \ + -append 'panic=1 mem=500M console=ttyAMA0 udevtimeout=6000 no_timer_check lpj=4464640 acpi=off printk.time=1 cgroup_disable=memory root=/dev/sdb selinux=0 guestfs_verbose=1 TERM=xterm-256color' + +The appliance boots, but segfaults as soon as the virtio-scsi driver is loaded: + +supermin: internal insmod virtio_scsi.ko +[ 3.992963] scsi0 : Virtio SCSI HBA +libguestfs: error: appliance closed the connection unexpectedly, see earlier error messages + +I captured a core dump: + +Core was generated by `/home/rjones/d/qemu/arm-softmmu/qemu-system-arm -global virtio-blk-device.scsi='. +Program terminated with signal SIGSEGV, Segmentation fault. +#0 0x000856bc in virtio_scsi_handle_cmd_req_submit (s=<optimized out>, + req=0x6c03acf8) at /home/rjones/d/qemu/hw/scsi/virtio-scsi.c:551 +551 bdrv_io_unplug(req->sreq->dev->conf.bs); +(gdb) bt +#0 0x000856bc in virtio_scsi_handle_cmd_req_submit (s=<optimized out>, + req=0x6c03acf8) at /home/rjones/d/qemu/hw/scsi/virtio-scsi.c:551 +#1 0x0008573a in virtio_scsi_handle_cmd (vdev=0xac4d68, vq=0xafe4b8) + at /home/rjones/d/qemu/hw/scsi/virtio-scsi.c:573 +#2 0x0004fdbe in access_with_adjusted_size (addr=80, + value=value@entry=0x4443e6c0, size=size@entry=4, access_size_min=1, + access_size_max=<optimized out>, access_size_max@entry=0, + access=access@entry=0x4fee9 <memory_region_write_accessor>, + mr=mr@entry=0xa53fa8) at /home/rjones/d/qemu/memory.c:480 +#3 0x00054234 in memory_region_dispatch_write (size=4, data=2, + addr=<optimized out>, mr=0xa53fa8) at /home/rjones/d/qemu/memory.c:1117 +#4 io_mem_write (mr=0xa53fa8, addr=<optimized out>, val=val@entry=2, + size=size@entry=4) at /home/rjones/d/qemu/memory.c:1958 +#5 0x00021c88 in address_space_rw (as=0x3b96b4 <address_space_memory>, + addr=167788112, buf=buf@entry=0x4443e790 "\002", len=len@entry=4, + is_write=is_write@entry=true) at /home/rjones/d/qemu/exec.c:2135 +#6 0x00021de6 in address_space_write (len=4, buf=0x4443e790 "\002", + addr=<optimized out>, as=<optimized out>) + at /home/rjones/d/qemu/exec.c:2202 +#7 subpage_write (opaque=<optimized out>, addr=<optimized out>, value=2, + len=4) at /home/rjones/d/qemu/exec.c:1811 +#8 0x0004fdbe in access_with_adjusted_size (addr=592, + value=value@entry=0x4443e820, size=size@entry=4, access_size_min=1, + access_size_max=<optimized out>, access_size_max@entry=0, + access=access@entry=0x4fee9 <memory_region_write_accessor>, + mr=mr@entry=0xaed980) at /home/rjones/d/qemu/memory.c:480 +#9 0x00054234 in memory_region_dispatch_write (size=4, data=2, + addr=<optimized out>, mr=0xaed980) at /home/rjones/d/qemu/memory.c:1117 +#10 io_mem_write (mr=0xaed980, addr=<optimized out>, val=2, size=size@entry=4) + at /home/rjones/d/qemu/memory.c:1958 +#11 0x00057f24 in io_writel (retaddr=1121296542, Cannot access memory at address 0x0 +addr=<optimized out>, val=2, + physaddr=592, env=0x9d6c50) at /home/rjones/d/qemu/softmmu_template.h:381 +#12 helper_le_stl_mmu (env=0x9d6c50, addr=<optimized out>, val=2, + mmu_idx=<optimized out>, retaddr=1121296542) + at /home/rjones/d/qemu/softmmu_template.h:419 +#13 0x42d5a0a0 in ?? () +Cannot access memory at address 0x0 +Backtrace stopped: previous frame identical to this frame (corrupt stack?) +(gdb) print req +$1 = (VirtIOSCSIReq *) 0x6c03acf8 +(gdb) print req->sreq +$2 = (SCSIRequest *) 0xc2c2c2c2 +(gdb) print req->sreq->dev +Cannot access memory at address 0xc2c2c2c6 +(gdb) print *req +$3 = { + dev = 0x6c000040, + vq = 0x6c000040, + qsgl = { + sg = 0x0, + nsg = 0, + nalloc = -1027423550, + size = 3267543746, + dev = 0xc2c2c2c2, + as = 0xc2c2c2c2 + }, + resp_iov = { + iov = 0xc2c2c2c2, + niov = -1027423550, + nalloc = -1027423550, + size = 3267543746 + }, + elem = { + index = 3267543746, + out_num = 3267543746, + in_num = 3267543746, + in_addr = {14033993530586874562 <repeats 1024 times>}, + out_addr = {14033993530586874562 <repeats 1024 times>}, + in_sg = {{ + iov_base = 0xc2c2c2c2, + iov_len = 3267543746 + } <repeats 1024 times>}, + out_sg = {{ + iov_base = 0xc2c2c2c2, + iov_len = 3267543746 + } <repeats 1024 times>} + }, + vring = 0xc2c2c2c2, + { + next = { + tqe_next = 0xc2c2c2c2, + tqe_prev = 0xc2c2c2c2 + }, + remaining = -1027423550 + }, + sreq = 0xc2c2c2c2, + resp_size = 3267543746, + mode = (SCSI_XFER_TO_DEV | unknown: 3267543744), + resp = { + cmd = { + sense_len = 3267543746, + resid = 3267543746, + status_qualifier = 49858, + status = 194 '\302', + response = 194 '\302' + }, + tmf = { + response = 194 '\302' + }, + an = { + event_actual = 3267543746, + response = 194 '\302' + }, + event = { + event = 3267543746, + lun = "\302\302\302\302\302\302\302", <incomplete sequence \302>, + reason = 3267543746 + } + }, + req = { + { + cmd = { + lun = "\302\302\302\302\302\302\302", <incomplete sequence \302>, + tag = 14033993530586874562, + task_attr = 194 '\302', + prio = 194 '\302', + crn = 194 '\302' + }, + cdb = 0x6c042d73 '\302' <repeats 36 times>, <incomplete sequence \302> + }, + tmf = { + type = 3267543746, + subtype = 3267543746, + lun = "\302\302\302\302\302\302\302", <incomplete sequence \302>, + tag = 14033993530586874562 + }, + an = { + type = 3267543746, + lun = "\302\302\302\302\302\302\302", <incomplete sequence \302>, + event_requested = 3267543746 + } + } +} + +This is qemu from git today (2014-10-07). + +The hardware is 32 bit ARM (ODROID-XU Samsung Exynos 5410). It is running Ubuntu 14.04 LTS as the main operating system, but I am NOT using qemu from Ubuntu (which is ancient). + +Richard, is this 3 year old bug still an issue? + + +Ah, my mail client found the thread that tells me this was fixed in commit 35e4e96c4d5bfcf. So we can close this. + + +Yes, qemu's working fine on aarch64 so this must have been fixed. + |