summary refs log tree commit diff stats
path: root/results/classifier/gemma3:12b/kvm/939027
diff options
context:
space:
mode:
Diffstat (limited to 'results/classifier/gemma3:12b/kvm/939027')
-rw-r--r--results/classifier/gemma3:12b/kvm/939027377
1 files changed, 377 insertions, 0 deletions
diff --git a/results/classifier/gemma3:12b/kvm/939027 b/results/classifier/gemma3:12b/kvm/939027
new file mode 100644
index 00000000..eb5a1f07
--- /dev/null
+++ b/results/classifier/gemma3:12b/kvm/939027
@@ -0,0 +1,377 @@
+
+Combining direct_io and non-direct_io leads to hang
+
+Version 0.12.2 - I know this isn't the latest but I looked through the changelogs and couldn't find any references to this being fixed.
+
+This is related to issues that many apps have, and is described in more detail here:
+    http://oss.sgi.com/archives/xfs/2010-07/msg00163.html
+and:
+    https://bugs.launchpad.net/percona-xtrabackup/+bug/606981
+
+When using both direct io and buffered io on the same file, you can cause corruption on the filesystem and other issues.
+XFS out right hangs, but the problem exists on ext3 and other filesystems which silently carry on.
+
+**this is a data corruption issue**.
+
+This is the full stack trace we got from a recent hang:
+
+Feb 22 19:55:14 virt11 kernel: INFO: task qemu-kvm:18360 blocked for more than 120 seconds.
+Feb 22 19:55:14 virt11 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
+Feb 22 19:55:14 virt11 kernel: qemu-kvm      D 0000000000000002     0 18360      1 0x00000080
+Feb 22 19:55:14 virt11 kernel: ffff8808614457e8 0000000000000086 ffff8800282d5f80 ffff8800282d5f80
+Feb 22 19:55:14 virt11 kernel: ffff880861445768 ffffffff810573ce ffff880861445768 0000000000000086
+Feb 22 19:55:14 virt11 kernel: ffff8808f15bba78 ffff880861445fd8 000000000000f598 ffff8808f15bba78
+Feb 22 19:55:14 virt11 kernel: Call Trace:
+Feb 22 19:55:14 virt11 kernel: [<ffffffff810573ce>] ? activate_task+0x2e/0x40
+Feb 22 19:55:14 virt11 kernel: [<ffffffff814dd755>] rwsem_down_failed_common+0x95/0x1d0
+Feb 22 19:55:14 virt11 kernel: [<ffffffff814dd8b3>] rwsem_down_write_failed+0x23/0x30
+Feb 22 19:55:14 virt11 kernel: [<ffffffff8126e573>] call_rwsem_down_write_failed+0x13/0x20
+Feb 22 19:55:14 virt11 kernel: [<ffffffff814dcdb2>] ? down_write+0x32/0x40
+Feb 22 19:55:14 virt11 kernel: [<ffffffffa0332d6e>] xfs_ilock+0x7e/0xd0 [xfs]
+Feb 22 19:55:14 virt11 kernel: [<ffffffffa033ab02>] xfs_iomap+0x2e2/0x440 [xfs]
+Feb 22 19:55:14 virt11 kernel: [<ffffffffa0354116>] __xfs_get_blocks+0x86/0x200 [xfs]
+Feb 22 19:55:14 virt11 kernel: [<ffffffffa03542aa>] xfs_get_blocks_direct+0x1a/0x20 [xfs]
+Feb 22 19:55:14 virt11 kernel: [<ffffffff811ac132>] __blockdev_direct_IO+0x872/0xc40
+Feb 22 19:55:14 virt11 kernel: [<ffffffffa0353f50>] xfs_vm_direct_IO+0xb0/0xf0 [xfs]
+Feb 22 19:55:14 virt11 kernel: [<ffffffffa0354290>] ? xfs_get_blocks_direct+0x0/0x20 [xfs]
+Feb 22 19:55:14 virt11 kernel: [<ffffffffa0353cc0>] ? xfs_end_io_direct+0x0/0xe0 [xfs]
+Feb 22 19:55:14 virt11 kernel: [<ffffffff8106dd57>] ? current_fs_time+0x27/0x30
+Feb 22 19:55:14 virt11 kernel: [<ffffffff8110df22>] generic_file_direct_write+0xc2/0x190
+Feb 22 19:55:14 virt11 kernel: [<ffffffffa034bfcf>] ? xfs_trans_unlocked_item+0x4f/0x60 [xfs]
+Feb 22 19:55:14 virt11 kernel: [<ffffffffa035dd1d>] xfs_write+0x4fd/0xb70 [xfs]
+Feb 22 19:55:14 virt11 kernel: [<ffffffff8105dc72>] ? default_wake_function+0x12/0x20
+Feb 22 19:55:14 virt11 kernel: [<ffffffffa03599a1>] xfs_file_aio_write+0x61/0x70 [xfs]
+Feb 22 19:55:14 virt11 kernel: [<ffffffff8117241a>] do_sync_write+0xfa/0x140
+Feb 22 19:55:14 virt11 kernel: [<ffffffff8107fbc2>] ? send_signal+0x42/0x80
+Feb 22 19:55:14 virt11 kernel: [<ffffffff8108e160>] ? autoremove_wake_function+0x0/0x40
+Feb 22 19:55:14 virt11 kernel: [<ffffffff8107ff96>] ? group_send_sig_info+0x56/0x70
+Feb 22 19:55:14 virt11 kernel: [<ffffffff81211d3b>] ? selinux_file_permission+0xfb/0x150
+Feb 22 19:55:14 virt11 kernel: [<ffffffff812051a6>] ? security_file_permission+0x16/0x20
+Feb 22 19:55:14 virt11 kernel: [<ffffffff81172718>] vfs_write+0xb8/0x1a0
+Feb 22 19:55:14 virt11 kernel: [<ffffffff810d1b62>] ? audit_syscall_entry+0x272/0x2a0
+Feb 22 19:55:14 virt11 kernel: [<ffffffff81173212>] sys_pwrite64+0x82/0xa0
+Feb 22 19:55:14 virt11 kernel: [<ffffffff8100b172>] system_call_fastpath+0x16/0x1b
+Feb 22 19:57:14 virt11 kernel: INFO: task qemu-kvm:18360 blocked for more than 120 seconds.
+Feb 22 19:57:14 virt11 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
+Feb 22 19:57:14 virt11 kernel: qemu-kvm      D 0000000000000002     0 18360      1 0x00000080
+Feb 22 19:57:14 virt11 kernel: ffff8808614457e8 0000000000000086 ffff8800282d5f80 ffff8800282d5f80
+Feb 22 19:57:14 virt11 kernel: ffff880861445768 ffffffff810573ce ffff880861445768 0000000000000086
+Feb 22 19:57:14 virt11 kernel: ffff8808f15bba78 ffff880861445fd8 000000000000f598 ffff8808f15bba78
+Feb 22 19:57:14 virt11 kernel: Call Trace:
+Feb 22 19:57:14 virt11 kernel: [<ffffffff810573ce>] ? activate_task+0x2e/0x40
+Feb 22 19:57:14 virt11 kernel: [<ffffffff814dd755>] rwsem_down_failed_common+0x95/0x1d0
+Feb 22 19:57:14 virt11 kernel: [<ffffffff814dd8b3>] rwsem_down_write_failed+0x23/0x30
+Feb 22 19:57:14 virt11 kernel: [<ffffffff8126e573>] call_rwsem_down_write_failed+0x13/0x20
+Feb 22 19:57:14 virt11 kernel: [<ffffffff814dcdb2>] ? down_write+0x32/0x40
+Feb 22 19:57:14 virt11 kernel: [<ffffffffa0332d6e>] xfs_ilock+0x7e/0xd0 [xfs]
+Feb 22 19:57:14 virt11 kernel: [<ffffffffa033ab02>] xfs_iomap+0x2e2/0x440 [xfs]
+Feb 22 19:57:14 virt11 kernel: [<ffffffffa0354116>] __xfs_get_blocks+0x86/0x200 [xfs]
+Feb 22 19:57:14 virt11 kernel: [<ffffffffa03542aa>] xfs_get_blocks_direct+0x1a/0x20 [xfs]
+Feb 22 19:57:14 virt11 kernel: [<ffffffff811ac132>] __blockdev_direct_IO+0x872/0xc40
+Feb 22 19:57:14 virt11 kernel: [<ffffffffa0353f50>] xfs_vm_direct_IO+0xb0/0xf0 [xfs]
+Feb 22 19:57:14 virt11 kernel: [<ffffffffa0354290>] ? xfs_get_blocks_direct+0x0/0x20 [xfs]
+Feb 22 19:57:14 virt11 kernel: [<ffffffffa0353cc0>] ? xfs_end_io_direct+0x0/0xe0 [xfs]
+Feb 22 19:57:14 virt11 kernel: [<ffffffff8106dd57>] ? current_fs_time+0x27/0x30
+Feb 22 19:57:14 virt11 kernel: [<ffffffff8110df22>] generic_file_direct_write+0xc2/0x190
+Feb 22 19:57:14 virt11 kernel: [<ffffffffa034bfcf>] ? xfs_trans_unlocked_item+0x4f/0x60 [xfs]
+Feb 22 19:57:14 virt11 kernel: [<ffffffffa035dd1d>] xfs_write+0x4fd/0xb70 [xfs]
+Feb 22 19:57:14 virt11 kernel: [<ffffffff8105dc72>] ? default_wake_function+0x12/0x20
+Feb 22 19:57:14 virt11 kernel: [<ffffffffa03599a1>] xfs_file_aio_write+0x61/0x70 [xfs]
+Feb 22 19:57:14 virt11 kernel: [<ffffffff8117241a>] do_sync_write+0xfa/0x140
+Feb 22 19:57:14 virt11 kernel: [<ffffffff8107fbc2>] ? send_signal+0x42/0x80
+Feb 22 19:57:14 virt11 kernel: [<ffffffff8108e160>] ? autoremove_wake_function+0x0/0x40
+Feb 22 19:57:14 virt11 kernel: [<ffffffff8107ff96>] ? group_send_sig_info+0x56/0x70
+Feb 22 19:57:14 virt11 kernel: [<ffffffff81211d3b>] ? selinux_file_permission+0xfb/0x150
+Feb 22 19:57:14 virt11 kernel: [<ffffffff812051a6>] ? security_file_permission+0x16/0x20
+Feb 22 19:57:14 virt11 kernel: [<ffffffff81172718>] vfs_write+0xb8/0x1a0
+Feb 22 19:57:14 virt11 kernel: [<ffffffff810d1b62>] ? audit_syscall_entry+0x272/0x2a0
+Feb 22 19:57:14 virt11 kernel: [<ffffffff81173212>] sys_pwrite64+0x82/0xa0
+Feb 22 19:57:14 virt11 kernel: [<ffffffff8100b172>] system_call_fastpath+0x16/0x1b
+Feb 22 19:59:14 virt11 kernel: INFO: task qemu-kvm:18360 blocked for more than 120 seconds.
+Feb 22 19:59:14 virt11 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
+Feb 22 19:59:14 virt11 kernel: qemu-kvm      D 0000000000000002     0 18360      1 0x00000080
+Feb 22 19:59:14 virt11 kernel: ffff8808614457e8 0000000000000086 ffff8800282d5f80 ffff8800282d5f80
+Feb 22 19:59:14 virt11 kernel: ffff880861445768 ffffffff810573ce ffff880861445768 0000000000000086
+Feb 22 19:59:14 virt11 kernel: ffff8808f15bba78 ffff880861445fd8 000000000000f598 ffff8808f15bba78
+Feb 22 19:59:14 virt11 kernel: Call Trace:
+Feb 22 19:59:14 virt11 kernel: [<ffffffff810573ce>] ? activate_task+0x2e/0x40
+Feb 22 19:59:14 virt11 kernel: [<ffffffff814dd755>] rwsem_down_failed_common+0x95/0x1d0
+Feb 22 19:59:14 virt11 kernel: [<ffffffff814dd8b3>] rwsem_down_write_failed+0x23/0x30
+Feb 22 19:59:14 virt11 kernel: [<ffffffff8126e573>] call_rwsem_down_write_failed+0x13/0x20
+Feb 22 19:59:14 virt11 kernel: [<ffffffff814dcdb2>] ? down_write+0x32/0x40
+Feb 22 19:59:14 virt11 kernel: [<ffffffffa0332d6e>] xfs_ilock+0x7e/0xd0 [xfs]
+Feb 22 19:59:14 virt11 kernel: [<ffffffffa033ab02>] xfs_iomap+0x2e2/0x440 [xfs]
+Feb 22 19:59:14 virt11 kernel: [<ffffffffa0354116>] __xfs_get_blocks+0x86/0x200 [xfs]
+Feb 22 19:59:14 virt11 kernel: [<ffffffffa03542aa>] xfs_get_blocks_direct+0x1a/0x20 [xfs]
+Feb 22 19:59:14 virt11 kernel: [<ffffffff811ac132>] __blockdev_direct_IO+0x872/0xc40
+Feb 22 19:59:14 virt11 kernel: [<ffffffffa0353f50>] xfs_vm_direct_IO+0xb0/0xf0 [xfs]
+Feb 22 19:59:14 virt11 kernel: [<ffffffffa0354290>] ? xfs_get_blocks_direct+0x0/0x20 [xfs]
+Feb 22 19:59:14 virt11 kernel: [<ffffffffa0353cc0>] ? xfs_end_io_direct+0x0/0xe0 [xfs]
+Feb 22 19:59:14 virt11 kernel: [<ffffffff8106dd57>] ? current_fs_time+0x27/0x30
+Feb 22 19:59:14 virt11 kernel: [<ffffffff8110df22>] generic_file_direct_write+0xc2/0x190
+Feb 22 19:59:14 virt11 kernel: [<ffffffffa034bfcf>] ? xfs_trans_unlocked_item+0x4f/0x60 [xfs]
+Feb 22 19:59:14 virt11 kernel: [<ffffffffa035dd1d>] xfs_write+0x4fd/0xb70 [xfs]
+Feb 22 19:59:14 virt11 kernel: [<ffffffff8105dc72>] ? default_wake_function+0x12/0x20
+Feb 22 19:59:14 virt11 kernel: [<ffffffffa03599a1>] xfs_file_aio_write+0x61/0x70 [xfs]
+Feb 22 19:59:14 virt11 kernel: [<ffffffff8117241a>] do_sync_write+0xfa/0x140
+Feb 22 19:59:14 virt11 kernel: [<ffffffff8107fbc2>] ? send_signal+0x42/0x80
+Feb 22 19:59:14 virt11 kernel: [<ffffffff8108e160>] ? autoremove_wake_function+0x0/0x40
+Feb 22 19:59:14 virt11 kernel: [<ffffffff8107ff96>] ? group_send_sig_info+0x56/0x70
+Feb 22 19:59:14 virt11 kernel: [<ffffffff81211d3b>] ? selinux_file_permission+0xfb/0x150
+Feb 22 19:59:14 virt11 kernel: [<ffffffff812051a6>] ? security_file_permission+0x16/0x20
+Feb 22 19:59:14 virt11 kernel: [<ffffffff81172718>] vfs_write+0xb8/0x1a0
+Feb 22 19:59:14 virt11 kernel: [<ffffffff810d1b62>] ? audit_syscall_entry+0x272/0x2a0
+Feb 22 19:59:14 virt11 kernel: [<ffffffff81173212>] sys_pwrite64+0x82/0xa0
+Feb 22 19:59:14 virt11 kernel: [<ffffffff8100b172>] system_call_fastpath+0x16/0x1b
+Feb 22 20:01:14 virt11 kernel: INFO: task qemu-kvm:18360 blocked for more than 120 seconds.
+Feb 22 20:01:14 virt11 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
+Feb 22 20:01:14 virt11 kernel: qemu-kvm      D 0000000000000002     0 18360      1 0x00000080
+Feb 22 20:01:14 virt11 kernel: ffff8808614457e8 0000000000000086 ffff8800282d5f80 ffff8800282d5f80
+Feb 22 20:01:14 virt11 kernel: ffff880861445768 ffffffff810573ce ffff880861445768 0000000000000086
+Feb 22 20:01:14 virt11 kernel: ffff8808f15bba78 ffff880861445fd8 000000000000f598 ffff8808f15bba78
+Feb 22 20:01:14 virt11 kernel: Call Trace:
+Feb 22 20:01:14 virt11 kernel: [<ffffffff810573ce>] ? activate_task+0x2e/0x40
+Feb 22 20:01:14 virt11 kernel: [<ffffffff814dd755>] rwsem_down_failed_common+0x95/0x1d0
+Feb 22 20:01:14 virt11 kernel: [<ffffffff814dd8b3>] rwsem_down_write_failed+0x23/0x30
+Feb 22 20:01:14 virt11 kernel: [<ffffffff8126e573>] call_rwsem_down_write_failed+0x13/0x20
+Feb 22 20:01:14 virt11 kernel: [<ffffffff814dcdb2>] ? down_write+0x32/0x40
+Feb 22 20:01:14 virt11 kernel: [<ffffffffa0332d6e>] xfs_ilock+0x7e/0xd0 [xfs]
+Feb 22 20:01:14 virt11 kernel: [<ffffffffa033ab02>] xfs_iomap+0x2e2/0x440 [xfs]
+Feb 22 20:01:14 virt11 kernel: [<ffffffffa0354116>] __xfs_get_blocks+0x86/0x200 [xfs]
+Feb 22 20:01:14 virt11 kernel: [<ffffffffa03542aa>] xfs_get_blocks_direct+0x1a/0x20 [xfs]
+Feb 22 20:01:14 virt11 kernel: [<ffffffff811ac132>] __blockdev_direct_IO+0x872/0xc40
+Feb 22 20:01:14 virt11 kernel: [<ffffffffa0353f50>] xfs_vm_direct_IO+0xb0/0xf0 [xfs]
+Feb 22 20:01:14 virt11 kernel: [<ffffffffa0354290>] ? xfs_get_blocks_direct+0x0/0x20 [xfs]
+Feb 22 20:01:14 virt11 kernel: [<ffffffffa0353cc0>] ? xfs_end_io_direct+0x0/0xe0 [xfs]
+Feb 22 20:01:14 virt11 kernel: [<ffffffff8106dd57>] ? current_fs_time+0x27/0x30
+Feb 22 20:01:14 virt11 kernel: [<ffffffff8110df22>] generic_file_direct_write+0xc2/0x190
+Feb 22 20:01:14 virt11 kernel: [<ffffffffa034bfcf>] ? xfs_trans_unlocked_item+0x4f/0x60 [xfs]
+Feb 22 20:01:14 virt11 kernel: [<ffffffffa035dd1d>] xfs_write+0x4fd/0xb70 [xfs]
+Feb 22 20:01:14 virt11 kernel: [<ffffffff8105dc72>] ? default_wake_function+0x12/0x20
+Feb 22 20:01:14 virt11 kernel: [<ffffffffa03599a1>] xfs_file_aio_write+0x61/0x70 [xfs]
+Feb 22 20:01:14 virt11 kernel: [<ffffffff8117241a>] do_sync_write+0xfa/0x140
+Feb 22 20:01:14 virt11 kernel: [<ffffffff8107fbc2>] ? send_signal+0x42/0x80
+Feb 22 20:01:14 virt11 kernel: [<ffffffff8108e160>] ? autoremove_wake_function+0x0/0x40
+Feb 22 20:01:14 virt11 kernel: [<ffffffff8107ff96>] ? group_send_sig_info+0x56/0x70
+Feb 22 20:01:14 virt11 kernel: [<ffffffff81211d3b>] ? selinux_file_permission+0xfb/0x150
+Feb 22 20:01:14 virt11 kernel: [<ffffffff812051a6>] ? security_file_permission+0x16/0x20
+Feb 22 20:01:14 virt11 kernel: [<ffffffff81172718>] vfs_write+0xb8/0x1a0
+Feb 22 20:01:14 virt11 kernel: [<ffffffff810d1b62>] ? audit_syscall_entry+0x272/0x2a0
+Feb 22 20:01:14 virt11 kernel: [<ffffffff81173212>] sys_pwrite64+0x82/0xa0
+Feb 22 20:01:14 virt11 kernel: [<ffffffff8100b172>] system_call_fastpath+0x16/0x1b
+Feb 22 20:03:14 virt11 kernel: INFO: task qemu-kvm:18360 blocked for more than 120 seconds.
+Feb 22 20:03:14 virt11 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
+Feb 22 20:03:14 virt11 kernel: qemu-kvm      D 0000000000000002     0 18360      1 0x00000080
+Feb 22 20:03:14 virt11 kernel: ffff8808614457e8 0000000000000086 ffff8800282d5f80 ffff8800282d5f80
+Feb 22 20:03:14 virt11 kernel: ffff880861445768 ffffffff810573ce ffff880861445768 0000000000000086
+Feb 22 20:03:14 virt11 kernel: ffff8808f15bba78 ffff880861445fd8 000000000000f598 ffff8808f15bba78
+Feb 22 20:03:14 virt11 kernel: Call Trace:
+Feb 22 20:03:14 virt11 kernel: [<ffffffff810573ce>] ? activate_task+0x2e/0x40
+Feb 22 20:03:14 virt11 kernel: [<ffffffff814dd755>] rwsem_down_failed_common+0x95/0x1d0
+Feb 22 20:03:14 virt11 kernel: [<ffffffff814dd8b3>] rwsem_down_write_failed+0x23/0x30
+Feb 22 20:03:14 virt11 kernel: [<ffffffff8126e573>] call_rwsem_down_write_failed+0x13/0x20
+Feb 22 20:03:14 virt11 kernel: [<ffffffff814dcdb2>] ? down_write+0x32/0x40
+Feb 22 20:03:14 virt11 kernel: [<ffffffffa0332d6e>] xfs_ilock+0x7e/0xd0 [xfs]
+Feb 22 20:03:14 virt11 kernel: [<ffffffffa033ab02>] xfs_iomap+0x2e2/0x440 [xfs]
+Feb 22 20:03:14 virt11 kernel: [<ffffffffa0354116>] __xfs_get_blocks+0x86/0x200 [xfs]
+Feb 22 20:03:14 virt11 kernel: [<ffffffffa03542aa>] xfs_get_blocks_direct+0x1a/0x20 [xfs]
+Feb 22 20:03:14 virt11 kernel: [<ffffffff811ac132>] __blockdev_direct_IO+0x872/0xc40
+Feb 22 20:03:14 virt11 kernel: [<ffffffffa0353f50>] xfs_vm_direct_IO+0xb0/0xf0 [xfs]
+Feb 22 20:03:14 virt11 kernel: [<ffffffffa0354290>] ? xfs_get_blocks_direct+0x0/0x20 [xfs]
+Feb 22 20:03:14 virt11 kernel: [<ffffffffa0353cc0>] ? xfs_end_io_direct+0x0/0xe0 [xfs]
+Feb 22 20:03:14 virt11 kernel: [<ffffffff8106dd57>] ? current_fs_time+0x27/0x30
+Feb 22 20:03:14 virt11 kernel: [<ffffffff8110df22>] generic_file_direct_write+0xc2/0x190
+Feb 22 20:03:14 virt11 kernel: [<ffffffffa034bfcf>] ? xfs_trans_unlocked_item+0x4f/0x60 [xfs]
+Feb 22 20:03:14 virt11 kernel: [<ffffffffa035dd1d>] xfs_write+0x4fd/0xb70 [xfs]
+Feb 22 20:03:14 virt11 kernel: [<ffffffff8105dc72>] ? default_wake_function+0x12/0x20
+Feb 22 20:03:14 virt11 kernel: [<ffffffffa03599a1>] xfs_file_aio_write+0x61/0x70 [xfs]
+Feb 22 20:03:14 virt11 kernel: [<ffffffff8117241a>] do_sync_write+0xfa/0x140
+Feb 22 20:03:14 virt11 kernel: [<ffffffff8107fbc2>] ? send_signal+0x42/0x80
+Feb 22 20:03:14 virt11 kernel: [<ffffffff8108e160>] ? autoremove_wake_function+0x0/0x40
+Feb 22 20:03:14 virt11 kernel: [<ffffffff8107ff96>] ? group_send_sig_info+0x56/0x70
+Feb 22 20:03:14 virt11 kernel: [<ffffffff81211d3b>] ? selinux_file_permission+0xfb/0x150
+Feb 22 20:03:14 virt11 kernel: [<ffffffff812051a6>] ? security_file_permission+0x16/0x20
+Feb 22 20:03:14 virt11 kernel: [<ffffffff81172718>] vfs_write+0xb8/0x1a0
+Feb 22 20:03:14 virt11 kernel: [<ffffffff810d1b62>] ? audit_syscall_entry+0x272/0x2a0
+Feb 22 20:03:14 virt11 kernel: [<ffffffff81173212>] sys_pwrite64+0x82/0xa0
+Feb 22 20:03:14 virt11 kernel: [<ffffffff8100b172>] system_call_fastpath+0x16/0x1b
+Feb 22 20:05:14 virt11 kernel: INFO: task qemu-kvm:18360 blocked for more than 120 seconds.
+Feb 22 20:05:14 virt11 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
+Feb 22 20:05:14 virt11 kernel: qemu-kvm      D 0000000000000002     0 18360      1 0x00000080
+Feb 22 20:05:14 virt11 kernel: ffff8808614457e8 0000000000000086 ffff8800282d5f80 ffff8800282d5f80
+Feb 22 20:05:14 virt11 kernel: ffff880861445768 ffffffff810573ce ffff880861445768 0000000000000086
+Feb 22 20:05:14 virt11 kernel: ffff8808f15bba78 ffff880861445fd8 000000000000f598 ffff8808f15bba78
+Feb 22 20:05:14 virt11 kernel: Call Trace:
+Feb 22 20:05:14 virt11 kernel: [<ffffffff810573ce>] ? activate_task+0x2e/0x40
+Feb 22 20:05:14 virt11 kernel: [<ffffffff814dd755>] rwsem_down_failed_common+0x95/0x1d0
+Feb 22 20:05:14 virt11 kernel: [<ffffffff814dd8b3>] rwsem_down_write_failed+0x23/0x30
+Feb 22 20:05:14 virt11 kernel: [<ffffffff8126e573>] call_rwsem_down_write_failed+0x13/0x20
+Feb 22 20:05:14 virt11 kernel: [<ffffffff814dcdb2>] ? down_write+0x32/0x40
+Feb 22 20:05:14 virt11 kernel: [<ffffffffa0332d6e>] xfs_ilock+0x7e/0xd0 [xfs]
+Feb 22 20:05:14 virt11 kernel: [<ffffffffa033ab02>] xfs_iomap+0x2e2/0x440 [xfs]
+Feb 22 20:05:14 virt11 kernel: [<ffffffffa0354116>] __xfs_get_blocks+0x86/0x200 [xfs]
+Feb 22 20:05:14 virt11 kernel: [<ffffffffa03542aa>] xfs_get_blocks_direct+0x1a/0x20 [xfs]
+Feb 22 20:05:14 virt11 kernel: [<ffffffff811ac132>] __blockdev_direct_IO+0x872/0xc40
+Feb 22 20:05:14 virt11 kernel: [<ffffffffa0353f50>] xfs_vm_direct_IO+0xb0/0xf0 [xfs]
+Feb 22 20:05:14 virt11 kernel: [<ffffffffa0354290>] ? xfs_get_blocks_direct+0x0/0x20 [xfs]
+Feb 22 20:05:14 virt11 kernel: [<ffffffffa0353cc0>] ? xfs_end_io_direct+0x0/0xe0 [xfs]
+Feb 22 20:05:14 virt11 kernel: [<ffffffff8106dd57>] ? current_fs_time+0x27/0x30
+Feb 22 20:05:14 virt11 kernel: [<ffffffff8110df22>] generic_file_direct_write+0xc2/0x190
+Feb 22 20:05:14 virt11 kernel: [<ffffffffa034bfcf>] ? xfs_trans_unlocked_item+0x4f/0x60 [xfs]
+Feb 22 20:05:14 virt11 kernel: [<ffffffffa035dd1d>] xfs_write+0x4fd/0xb70 [xfs]
+Feb 22 20:05:14 virt11 kernel: [<ffffffff8105dc72>] ? default_wake_function+0x12/0x20
+Feb 22 20:05:14 virt11 kernel: [<ffffffffa03599a1>] xfs_file_aio_write+0x61/0x70 [xfs]
+Feb 22 20:05:14 virt11 kernel: [<ffffffff8117241a>] do_sync_write+0xfa/0x140
+Feb 22 20:05:14 virt11 kernel: [<ffffffff8107fbc2>] ? send_signal+0x42/0x80
+Feb 22 20:05:14 virt11 kernel: [<ffffffff8108e160>] ? autoremove_wake_function+0x0/0x40
+Feb 22 20:05:14 virt11 kernel: [<ffffffff8107ff96>] ? group_send_sig_info+0x56/0x70
+Feb 22 20:05:14 virt11 kernel: [<ffffffff81211d3b>] ? selinux_file_permission+0xfb/0x150
+Feb 22 20:05:14 virt11 kernel: [<ffffffff812051a6>] ? security_file_permission+0x16/0x20
+Feb 22 20:05:14 virt11 kernel: [<ffffffff81172718>] vfs_write+0xb8/0x1a0
+Feb 22 20:05:14 virt11 kernel: [<ffffffff810d1b62>] ? audit_syscall_entry+0x272/0x2a0
+Feb 22 20:05:14 virt11 kernel: [<ffffffff81173212>] sys_pwrite64+0x82/0xa0
+Feb 22 20:05:14 virt11 kernel: [<ffffffff8100b172>] system_call_fastpath+0x16/0x1b
+Feb 22 20:07:14 virt11 kernel: INFO: task qemu-kvm:18360 blocked for more than 120 seconds.
+Feb 22 20:07:14 virt11 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
+Feb 22 20:07:14 virt11 kernel: qemu-kvm      D 0000000000000002     0 18360      1 0x00000080
+Feb 22 20:07:14 virt11 kernel: ffff8808614457e8 0000000000000086 ffff8800282d5f80 ffff8800282d5f80
+Feb 22 20:07:14 virt11 kernel: ffff880861445768 ffffffff810573ce ffff880861445768 0000000000000086
+Feb 22 20:07:14 virt11 kernel: ffff8808f15bba78 ffff880861445fd8 000000000000f598 ffff8808f15bba78
+Feb 22 20:07:14 virt11 kernel: Call Trace:
+Feb 22 20:07:14 virt11 kernel: [<ffffffff810573ce>] ? activate_task+0x2e/0x40
+Feb 22 20:07:14 virt11 kernel: [<ffffffff814dd755>] rwsem_down_failed_common+0x95/0x1d0
+Feb 22 20:07:14 virt11 kernel: [<ffffffff814dd8b3>] rwsem_down_write_failed+0x23/0x30
+Feb 22 20:07:14 virt11 kernel: [<ffffffff8126e573>] call_rwsem_down_write_failed+0x13/0x20
+Feb 22 20:07:14 virt11 kernel: [<ffffffff814dcdb2>] ? down_write+0x32/0x40
+Feb 22 20:07:14 virt11 kernel: [<ffffffffa0332d6e>] xfs_ilock+0x7e/0xd0 [xfs]
+Feb 22 20:07:14 virt11 kernel: [<ffffffffa033ab02>] xfs_iomap+0x2e2/0x440 [xfs]
+Feb 22 20:07:14 virt11 kernel: [<ffffffffa0354116>] __xfs_get_blocks+0x86/0x200 [xfs]
+Feb 22 20:07:14 virt11 kernel: [<ffffffffa03542aa>] xfs_get_blocks_direct+0x1a/0x20 [xfs]
+Feb 22 20:07:14 virt11 kernel: [<ffffffff811ac132>] __blockdev_direct_IO+0x872/0xc40
+Feb 22 20:07:14 virt11 kernel: [<ffffffffa0353f50>] xfs_vm_direct_IO+0xb0/0xf0 [xfs]
+Feb 22 20:07:14 virt11 kernel: [<ffffffffa0354290>] ? xfs_get_blocks_direct+0x0/0x20 [xfs]
+Feb 22 20:07:14 virt11 kernel: [<ffffffffa0353cc0>] ? xfs_end_io_direct+0x0/0xe0 [xfs]
+Feb 22 20:07:14 virt11 kernel: [<ffffffff8106dd57>] ? current_fs_time+0x27/0x30
+Feb 22 20:07:14 virt11 kernel: [<ffffffff8110df22>] generic_file_direct_write+0xc2/0x190
+Feb 22 20:07:14 virt11 kernel: [<ffffffffa034bfcf>] ? xfs_trans_unlocked_item+0x4f/0x60 [xfs]
+Feb 22 20:07:14 virt11 kernel: [<ffffffffa035dd1d>] xfs_write+0x4fd/0xb70 [xfs]
+Feb 22 20:07:14 virt11 kernel: [<ffffffff8105dc72>] ? default_wake_function+0x12/0x20
+Feb 22 20:07:14 virt11 kernel: [<ffffffffa03599a1>] xfs_file_aio_write+0x61/0x70 [xfs]
+Feb 22 20:07:14 virt11 kernel: [<ffffffff8117241a>] do_sync_write+0xfa/0x140
+Feb 22 20:07:14 virt11 kernel: [<ffffffff8107fbc2>] ? send_signal+0x42/0x80
+Feb 22 20:07:14 virt11 kernel: [<ffffffff8108e160>] ? autoremove_wake_function+0x0/0x40
+Feb 22 20:07:14 virt11 kernel: [<ffffffff8107ff96>] ? group_send_sig_info+0x56/0x70
+Feb 22 20:07:14 virt11 kernel: [<ffffffff81211d3b>] ? selinux_file_permission+0xfb/0x150
+Feb 22 20:07:14 virt11 kernel: [<ffffffff812051a6>] ? security_file_permission+0x16/0x20
+Feb 22 20:07:14 virt11 kernel: [<ffffffff81172718>] vfs_write+0xb8/0x1a0
+Feb 22 20:07:14 virt11 kernel: [<ffffffff810d1b62>] ? audit_syscall_entry+0x272/0x2a0
+Feb 22 20:07:14 virt11 kernel: [<ffffffff81173212>] sys_pwrite64+0x82/0xa0
+Feb 22 20:07:14 virt11 kernel: [<ffffffff8100b172>] system_call_fastpath+0x16/0x1b
+Feb 22 20:09:14 virt11 kernel: INFO: task qemu-kvm:18360 blocked for more than 120 seconds.
+Feb 22 20:09:14 virt11 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
+Feb 22 20:09:14 virt11 kernel: qemu-kvm      D 0000000000000002     0 18360      1 0x00000080
+Feb 22 20:09:14 virt11 kernel: ffff8808614457e8 0000000000000086 ffff8800282d5f80 ffff8800282d5f80
+Feb 22 20:09:14 virt11 kernel: ffff880861445768 ffffffff810573ce ffff880861445768 0000000000000086
+Feb 22 20:09:14 virt11 kernel: ffff8808f15bba78 ffff880861445fd8 000000000000f598 ffff8808f15bba78
+Feb 22 20:09:14 virt11 kernel: Call Trace:
+Feb 22 20:09:14 virt11 kernel: [<ffffffff810573ce>] ? activate_task+0x2e/0x40
+Feb 22 20:09:14 virt11 kernel: [<ffffffff814dd755>] rwsem_down_failed_common+0x95/0x1d0
+Feb 22 20:09:14 virt11 kernel: [<ffffffff814dd8b3>] rwsem_down_write_failed+0x23/0x30
+Feb 22 20:09:14 virt11 kernel: [<ffffffff8126e573>] call_rwsem_down_write_failed+0x13/0x20
+Feb 22 20:09:14 virt11 kernel: [<ffffffff814dcdb2>] ? down_write+0x32/0x40
+Feb 22 20:09:14 virt11 kernel: [<ffffffffa0332d6e>] xfs_ilock+0x7e/0xd0 [xfs]
+Feb 22 20:09:14 virt11 kernel: [<ffffffffa033ab02>] xfs_iomap+0x2e2/0x440 [xfs]
+Feb 22 20:09:14 virt11 kernel: [<ffffffffa0354116>] __xfs_get_blocks+0x86/0x200 [xfs]
+Feb 22 20:09:14 virt11 kernel: [<ffffffffa03542aa>] xfs_get_blocks_direct+0x1a/0x20 [xfs]
+Feb 22 20:09:14 virt11 kernel: [<ffffffff811ac132>] __blockdev_direct_IO+0x872/0xc40
+Feb 22 20:09:14 virt11 kernel: [<ffffffffa0353f50>] xfs_vm_direct_IO+0xb0/0xf0 [xfs]
+Feb 22 20:09:14 virt11 kernel: [<ffffffffa0354290>] ? xfs_get_blocks_direct+0x0/0x20 [xfs]
+Feb 22 20:09:14 virt11 kernel: [<ffffffffa0353cc0>] ? xfs_end_io_direct+0x0/0xe0 [xfs]
+Feb 22 20:09:14 virt11 kernel: [<ffffffff8106dd57>] ? current_fs_time+0x27/0x30
+Feb 22 20:09:14 virt11 kernel: [<ffffffff8110df22>] generic_file_direct_write+0xc2/0x190
+Feb 22 20:09:14 virt11 kernel: [<ffffffffa034bfcf>] ? xfs_trans_unlocked_item+0x4f/0x60 [xfs]
+Feb 22 20:09:14 virt11 kernel: [<ffffffffa035dd1d>] xfs_write+0x4fd/0xb70 [xfs]
+Feb 22 20:09:14 virt11 kernel: [<ffffffff8105dc72>] ? default_wake_function+0x12/0x20
+Feb 22 20:09:14 virt11 kernel: [<ffffffffa03599a1>] xfs_file_aio_write+0x61/0x70 [xfs]
+Feb 22 20:09:14 virt11 kernel: [<ffffffff8117241a>] do_sync_write+0xfa/0x140
+Feb 22 20:09:14 virt11 kernel: [<ffffffff8107fbc2>] ? send_signal+0x42/0x80
+Feb 22 20:09:14 virt11 kernel: [<ffffffff8108e160>] ? autoremove_wake_function+0x0/0x40
+Feb 22 20:09:14 virt11 kernel: [<ffffffff8107ff96>] ? group_send_sig_info+0x56/0x70
+Feb 22 20:09:14 virt11 kernel: [<ffffffff81211d3b>] ? selinux_file_permission+0xfb/0x150
+Feb 22 20:09:14 virt11 kernel: [<ffffffff812051a6>] ? security_file_permission+0x16/0x20
+Feb 22 20:09:14 virt11 kernel: [<ffffffff81172718>] vfs_write+0xb8/0x1a0
+Feb 22 20:09:14 virt11 kernel: [<ffffffff810d1b62>] ? audit_syscall_entry+0x272/0x2a0
+Feb 22 20:09:14 virt11 kernel: [<ffffffff81173212>] sys_pwrite64+0x82/0xa0
+Feb 22 20:09:14 virt11 kernel: [<ffffffff8100b172>] system_call_fastpath+0x16/0x1b
+Feb 22 20:11:14 virt11 kernel: INFO: task qemu-kvm:18360 blocked for more than 120 seconds.
+Feb 22 20:11:14 virt11 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
+Feb 22 20:11:14 virt11 kernel: qemu-kvm      D 0000000000000002     0 18360      1 0x00000080
+Feb 22 20:11:14 virt11 kernel: ffff8808614457e8 0000000000000086 ffff8800282d5f80 ffff8800282d5f80
+Feb 22 20:11:14 virt11 kernel: ffff880861445768 ffffffff810573ce ffff880861445768 0000000000000086
+Feb 22 20:11:14 virt11 kernel: ffff8808f15bba78 ffff880861445fd8 000000000000f598 ffff8808f15bba78
+Feb 22 20:11:14 virt11 kernel: Call Trace:
+Feb 22 20:11:14 virt11 kernel: [<ffffffff810573ce>] ? activate_task+0x2e/0x40
+Feb 22 20:11:14 virt11 kernel: [<ffffffff814dd755>] rwsem_down_failed_common+0x95/0x1d0
+Feb 22 20:11:14 virt11 kernel: [<ffffffff814dd8b3>] rwsem_down_write_failed+0x23/0x30
+Feb 22 20:11:14 virt11 kernel: [<ffffffff8126e573>] call_rwsem_down_write_failed+0x13/0x20
+Feb 22 20:11:14 virt11 kernel: [<ffffffff814dcdb2>] ? down_write+0x32/0x40
+Feb 22 20:11:14 virt11 kernel: [<ffffffffa0332d6e>] xfs_ilock+0x7e/0xd0 [xfs]
+Feb 22 20:11:14 virt11 kernel: [<ffffffffa033ab02>] xfs_iomap+0x2e2/0x440 [xfs]
+Feb 22 20:11:14 virt11 kernel: [<ffffffffa0354116>] __xfs_get_blocks+0x86/0x200 [xfs]
+Feb 22 20:11:14 virt11 kernel: [<ffffffffa03542aa>] xfs_get_blocks_direct+0x1a/0x20 [xfs]
+Feb 22 20:11:14 virt11 kernel: [<ffffffff811ac132>] __blockdev_direct_IO+0x872/0xc40
+Feb 22 20:11:14 virt11 kernel: [<ffffffffa0353f50>] xfs_vm_direct_IO+0xb0/0xf0 [xfs]
+Feb 22 20:11:14 virt11 kernel: [<ffffffffa0354290>] ? xfs_get_blocks_direct+0x0/0x20 [xfs]
+Feb 22 20:11:14 virt11 kernel: [<ffffffffa0353cc0>] ? xfs_end_io_direct+0x0/0xe0 [xfs]
+Feb 22 20:11:14 virt11 kernel: [<ffffffff8106dd57>] ? current_fs_time+0x27/0x30
+Feb 22 20:11:14 virt11 kernel: [<ffffffff8110df22>] generic_file_direct_write+0xc2/0x190
+Feb 22 20:11:14 virt11 kernel: [<ffffffffa034bfcf>] ? xfs_trans_unlocked_item+0x4f/0x60 [xfs]
+Feb 22 20:11:14 virt11 kernel: [<ffffffffa035dd1d>] xfs_write+0x4fd/0xb70 [xfs]
+Feb 22 20:11:14 virt11 kernel: [<ffffffff8105dc72>] ? default_wake_function+0x12/0x20
+Feb 22 20:11:14 virt11 kernel: [<ffffffffa03599a1>] xfs_file_aio_write+0x61/0x70 [xfs]
+Feb 22 20:11:14 virt11 kernel: [<ffffffff8117241a>] do_sync_write+0xfa/0x140
+Feb 22 20:11:14 virt11 kernel: [<ffffffff8107fbc2>] ? send_signal+0x42/0x80
+Feb 22 20:11:14 virt11 kernel: [<ffffffff8108e160>] ? autoremove_wake_function+0x0/0x40
+Feb 22 20:11:14 virt11 kernel: [<ffffffff8107ff96>] ? group_send_sig_info+0x56/0x70
+Feb 22 20:11:14 virt11 kernel: [<ffffffff81211d3b>] ? selinux_file_permission+0xfb/0x150
+Feb 22 20:11:14 virt11 kernel: [<ffffffff812051a6>] ? security_file_permission+0x16/0x20
+Feb 22 20:11:14 virt11 kernel: [<ffffffff81172718>] vfs_write+0xb8/0x1a0
+Feb 22 20:11:14 virt11 kernel: [<ffffffff810d1b62>] ? audit_syscall_entry+0x272/0x2a0
+Feb 22 20:11:14 virt11 kernel: [<ffffffff81173212>] sys_pwrite64+0x82/0xa0
+Feb 22 20:11:14 virt11 kernel: [<ffffffff8100b172>] system_call_fastpath+0x16/0x1b
+Feb 22 20:13:14 virt11 kernel: INFO: task qemu-kvm:18360 blocked for more than 120 seconds.
+Feb 22 20:13:14 virt11 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
+Feb 22 20:13:14 virt11 kernel: qemu-kvm      D 0000000000000002     0 18360      1 0x00000080
+Feb 22 20:13:14 virt11 kernel: ffff8808614457e8 0000000000000086 ffff8800282d5f80 ffff8800282d5f80
+Feb 22 20:13:14 virt11 kernel: ffff880861445768 ffffffff810573ce ffff880861445768 0000000000000086
+Feb 22 20:13:14 virt11 kernel: ffff8808f15bba78 ffff880861445fd8 000000000000f598 ffff8808f15bba78
+Feb 22 20:13:14 virt11 kernel: Call Trace:
+Feb 22 20:13:14 virt11 kernel: [<ffffffff810573ce>] ? activate_task+0x2e/0x40
+Feb 22 20:13:14 virt11 kernel: [<ffffffff814dd755>] rwsem_down_failed_common+0x95/0x1d0
+Feb 22 20:13:14 virt11 kernel: [<ffffffff814dd8b3>] rwsem_down_write_failed+0x23/0x30
+Feb 22 20:13:14 virt11 kernel: [<ffffffff8126e573>] call_rwsem_down_write_failed+0x13/0x20
+Feb 22 20:13:14 virt11 kernel: [<ffffffff814dcdb2>] ? down_write+0x32/0x40
+Feb 22 20:13:14 virt11 kernel: [<ffffffffa0332d6e>] xfs_ilock+0x7e/0xd0 [xfs]
+Feb 22 20:13:14 virt11 kernel: [<ffffffffa033ab02>] xfs_iomap+0x2e2/0x440 [xfs]
+Feb 22 20:13:14 virt11 kernel: [<ffffffffa0354116>] __xfs_get_blocks+0x86/0x200 [xfs]
+Feb 22 20:13:14 virt11 kernel: [<ffffffffa03542aa>] xfs_get_blocks_direct+0x1a/0x20 [xfs]
+Feb 22 20:13:14 virt11 kernel: [<ffffffff811ac132>] __blockdev_direct_IO+0x872/0xc40
+Feb 22 20:13:14 virt11 kernel: [<ffffffffa0353f50>] xfs_vm_direct_IO+0xb0/0xf0 [xfs]
+Feb 22 20:13:14 virt11 kernel: [<ffffffffa0354290>] ? xfs_get_blocks_direct+0x0/0x20 [xfs]
+Feb 22 20:13:14 virt11 kernel: [<ffffffffa0353cc0>] ? xfs_end_io_direct+0x0/0xe0 [xfs]
+Feb 22 20:13:14 virt11 kernel: [<ffffffff8106dd57>] ? current_fs_time+0x27/0x30
+Feb 22 20:13:14 virt11 kernel: [<ffffffff8110df22>] generic_file_direct_write+0xc2/0x190
+Feb 22 20:13:14 virt11 kernel: [<ffffffffa034bfcf>] ? xfs_trans_unlocked_item+0x4f/0x60 [xfs]
+Feb 22 20:13:14 virt11 kernel: [<ffffffffa035dd1d>] xfs_write+0x4fd/0xb70 [xfs]
+Feb 22 20:13:14 virt11 kernel: [<ffffffff8105dc72>] ? default_wake_function+0x12/0x20
+Feb 22 20:13:14 virt11 kernel: [<ffffffffa03599a1>] xfs_file_aio_write+0x61/0x70 [xfs]
+Feb 22 20:13:14 virt11 kernel: [<ffffffff8117241a>] do_sync_write+0xfa/0x140
+Feb 22 20:13:14 virt11 kernel: [<ffffffff8107fbc2>] ? send_signal+0x42/0x80
+Feb 22 20:13:14 virt11 kernel: [<ffffffff8108e160>] ? autoremove_wake_function+0x0/0x40
+Feb 22 20:13:14 virt11 kernel: [<ffffffff8107ff96>] ? group_send_sig_info+0x56/0x70
+Feb 22 20:13:14 virt11 kernel: [<ffffffff81211d3b>] ? selinux_file_permission+0xfb/0x150
+Feb 22 20:13:14 virt11 kernel: [<ffffffff812051a6>] ? security_file_permission+0x16/0x20
+Feb 22 20:13:14 virt11 kernel: [<ffffffff81172718>] vfs_write+0xb8/0x1a0
+Feb 22 20:13:14 virt11 kernel: [<ffffffff810d1b62>] ? audit_syscall_entry+0x272/0x2a0
+Feb 22 20:13:14 virt11 kernel: [<ffffffff81173212>] sys_pwrite64+0x82/0xa0
+Feb 22 20:13:14 virt11 kernel: [<ffffffff8100b172>] system_call_fastpath+0x16/0x1b
\ No newline at end of file