1 files changed, 50 insertions, 0 deletions
diff --git a/results/classifier/118/graphic/1378 b/results/classifier/118/graphic/1378
new file mode 100644
index 000000000..c4e14b240
--- /dev/null
+++ b/results/classifier/118/graphic/1378
@@ -0,0 +1,50 @@
+graphic: 0.821
+virtual: 0.736
+performance: 0.664
+kernel: 0.623
+semantic: 0.618
+device: 0.615
+KVM: 0.571
+architecture: 0.519
+PID: 0.456
+vnc: 0.425
+risc-v: 0.359
+ppc: 0.352
+VMM: 0.342
+network: 0.338
+permissions: 0.334
+socket: 0.330
+peripherals: 0.313
+boot: 0.286
+register: 0.277
+hypervisor: 0.240
+i386: 0.231
+files: 0.211
+x86: 0.198
+debug: 0.193
+arm: 0.187
+user-level: 0.158
+TCG: 0.154
+mistranslation: 0.107
+assembly: 0.035
+
+iSCSI causes memory corruption
+Description of problem:
+This is a compound problem, which most likely involves a combination of how TrueNAS SCALE handles iSCSI triggering a problem **and** some memory-handling issue in QEMU leading to a crash. In short any Linux machine started with iSCSI handled by QEMU directly leads to a hard crash within 30s-1h. I was able to find a pattern in logs:
+
+1. First, a message like `QEMU[53139]: kvm: iSCSI Busy/TaskSetFull/TimeOut (retry #1 in 0 ms): TASK_SET_FULL` is logged
+  - it is always `TASK_SET_FULL`
+  - it is always `retry #1 in ... ms`, where only number of miliseconds varies
+  - the line is repeated multiple times, sometimes 5x and sometimes >200x
+2. It is followed by a single line with one of the following:
+  - `double free or corruption (out)`
+  - `double free or corruption (!prev)`
+  - `kvm: ../block/block-backend.c:1567: blk_aio_write_entry: Assertion `!qiov || qiov->size == acb->bytes' failed.`
+  - `kvm: malloc.c:2379: sysmalloc: Assertion `(old_top == initial_top (av) && old_size == 0) || ((unsigned long) (old_size) >= MINSIZE && prev_inuse (old_top) && ((unsigned long) old_end & (pagesize - 1)) == 0)' failed.`
+  - `kvm: iSCSI CheckCondition: SENSE KEY:UNIT_ATTENTION(6) ASCQ:BUS_RESET(0x2900)`
+  - `malloc(): invalid size (unsorted)`
+3. The virtual machine crashes
+Steps to reproduce:
+I don't have a specific concrete steps, only clues really. This problem started happening after TrueNAS SCALE updated their iSCSI code in Bluefin release to a new upstream version. That iSCSI server still works when iSCSI is mounted by the kernel and QEMU uses a normal `/dev` entry. While there's probably some problem with it, QEMU shouldn't probably crash with memory errors.
+Additional information:
+While I'm a software developer, I don't code in C on a daily basis. However, looking at the errors, I have a suspicion the problem may be somewhere in the `iscsi_co_generic_cb()`, as it seems the struct is getting damaged (out of bound write?) and causes explosion somewhere down the line.