1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
|
mistranslation: 0.912
KVM: 0.861
graphic: 0.837
device: 0.836
other: 0.822
vnc: 0.819
boot: 0.793
network: 0.777
socket: 0.760
assembly: 0.750
instruction: 0.742
semantic: 0.706
Ubuntu 15.10: QEMU VM hang if memory >= 1T
1. Ubuntu 15.10 x86_64 installed on HP SuperDome X with 8CPUs and 4T memory.
2. Create a VM, install Ubuntu 15.10, if memory >= 1T , VM hang when start. If memory < 1T, it is good.
<domain type='kvm'>
<name>u1510-1</name>
<uuid>39eefe1e-4829-4843-b892-026d143f3ec7</uuid>
<memory unit='KiB'>1073741824</memory>
<currentMemory unit='KiB'>1073741824</currentMemory>
<vcpu placement='static'>16</vcpu>
<os>
<type arch='x86_64' machine='pc-i440fx-2.3'>hvm</type>
<boot dev='hd'/>
<boot dev='cdrom'/>
</os>
<features>
<acpi/>
<apic/>
<pae/>
</features>
<clock offset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<devices>
<emulator>/usr/bin/kvm</emulator>
<disk type='file' device='disk'>
<driver name='qemu' type='qcow2' cache='directsync'/>
<source file='/vms/images/u1510-1.img'/>
<target dev='vda' bus='virtio'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
</disk>
<disk type='file' device='cdrom'>
<driver name='qemu' type='raw'/>
<target dev='hdc' bus='ide'/>
<readonly/>
<address type='drive' controller='0' bus='1' target='0' unit='0'/>
</disk>
<controller type='pci' index='0' model='pci-root'/>
<controller type='ide' index='0'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
</controller>
<controller type='usb' index='0'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
</controller>
<interface type='bridge'>
<mac address='0c:da:41:1d:ae:f1'/>
<source bridge='vswitch0'/>
<model type='virtio'/>
<driver name='vhost'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
</interface>
<input type='mouse' bus='ps2'/>
<input type='keyboard' bus='ps2'/>
<graphics type='vnc' port='-1' autoport='yes' listen='0.0.0.0'>
<listen type='address' address='0.0.0.0'/>
</graphics>
<video>
<model type='cirrus' vram='16384' heads='1'/>
<address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
</video>
<memballoon model='virtio'>
<address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/>
</memballoon>
</devices>
</domain>
3. The panic stack is
... cannot show
async_page_fault+0x28
ioread32_rep+0x38
ata_sff_data_xfer32+0x8a
ata_pio_sector+0x93
ata_pio_sectors+0x34
ata_sff_hsm_move+0x226
RIP: kthread_data+0x10
CR2: FFFFFFFF_FFFFFFD8
4. Change the host os to Redhat 7.2 , the vm is good even memory >=3.8T.
I delete cdrom and IDE controller, the vm start sucessfully.
But when I increate memory to 1100G, vm hang at hpet_enable when start.
The panic is page_fault when execute hpet_period = hpet_readl(HPET_PERIOD);
It seems that ioremap_nocache does not works correctly.
hpet_virt_address = ioremap_nocache(hpet_address, HPET_MMAP_SIZE);
Hi,
just to be sure, if you run
kvm -vnc :1 -m 1.5G
kvm -vnc :1 -m 1.5G --no-hpet
do those also crash?
Can you please show the contents of
/var/log/libvirt/qemu/u1510-1.log
I mean vm hang when memory >= 1100G (1024G when enable ide cdrom) instead of 1.5G.
If disable hpet, the vm will hang at acpi_ex_system_memory_space_handler when memory >= 1100G
If disable kvm, vm is good when memory <= 1020G, but also hang when memory >= 1024G.
There is no critical information in vm.log:
After I changed PHYS_ADDR_MASK, qemu vm can start when memory >=1024G , but kvm vm still hang.
-# define PHYS_ADDR_MASK 0xffffffffffLL
+# define PHYS_ADDR_MASK 0xfffffffffffLL
The issue is sloved after change cpuid[80000008];
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -2547,7 +2547,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
if (env->features[FEAT_8000_0001_EDX] & CPUID_EXT2_LM) {
/* 64 bit processor */
/* XXX: The physical address space is limited to 42 bits in exec.c. */
- *eax = 0x00003028; /* 48 bits virtual, 40 bits physical */
+ *eax = 0x00003029; /* 48 bits virtual, 41 bits physical */
} else {
if (env->features[FEAT_1_EDX] & CPUID_PSE36) {
*eax = 0x00000024; /* 36 bits physical */
For cpus which have not EPT feature, should change CR3_L_MODE_RESERVED_BITS in arch/x86/include/asm/kvm_host.h:
-#define CR3_L_MODE_RESERVED_BITS 0xFFFFFF0000000000ULL
+#define CR3_L_MODE_RESERVED_BITS 0xFFFFFE0000000000ULL
Can you still reproduce this problem with the latest version of upstream QEMU?
[Expired for QEMU because there has been no activity for 60 days.]
[Expired for qemu (Ubuntu) because there has been no activity for 60 days.]
I only saw this because it expired now :-/
Anyone affected by this might want to take a look at bug 1776189 where Ubuntu added a special machine type to more easily set "host-phys-bits" which is the qemu flag to have more (usually the host has more) available (at the cost of migratability). That allows <1TB as the default bits in qemu are chosen on the base of TCG (to be able to emulate what is virtualized) and that is limited to 1TB.
|