1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
|
[regression][powerpc] some vcpus are found offline inside guest with different vsmt setting from qemu-cmdline and breaks subsequent vcpu hotplug operation (xive)
Env:
Host: Power9 HW ppc64le
# lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 128
On-line CPU(s) list: 24-31,40-159
Thread(s) per core: 4
Core(s) per socket: 16
Socket(s): 2
NUMA node(s): 2
Model: 2.3 (pvr 004e 1203)
Model name: POWER9, altivec supported
Frequency boost: enabled
CPU max MHz: 3800.0000
CPU min MHz: 2300.0000
L1d cache: 1 MiB
L1i cache: 1 MiB
L2 cache: 8 MiB
L3 cache: 160 MiB
NUMA node0 CPU(s): 24-31,40-79
NUMA node8 CPU(s): 80-159
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Mitigation; RFI Flush, L1D private per thread
Vulnerability Mds: Not affected
Vulnerability Meltdown: Mitigation; RFI Flush, L1D private per thread
Vulnerability Spec store bypass: Mitigation; Kernel entry/exit barrier (eieio)
Vulnerability Spectre v1: Mitigation; __user pointer sanitization, ori31 speculation barrier enabled
Vulnerability Spectre v2: Mitigation; Software count cache flush (hardware accelerated), Software link stack flush
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected
Host Kernel: 5.9.0-0.rc8.28.fc34.ppc64le (Fedora rawhide)
Guest Kernel: Fedora33(5.8.6-301.fc33.ppc64le)
Qemu: e12ce85b2c79d83a340953291912875c30b3af06 (qemu/master)
Steps to reproduce:
Boot below kvm guest: (-M pseries,vsmt=2 -smp 8,cores=8,threads=1)
/home/sath/qemu/build/qemu-system-ppc64 -name vm1 -M pseries,vsmt=2 -accel kvm -m 4096 -smp 8,cores=8,threads=1 -nographic -nodefaults -serial mon:stdio -vga none -nographic -device virtio-scsi-pci -drive file=/home/sath/tests/data/avocado-vt/images/fdevel-ppc64le.qcow2,if=none,id=hd0,format=qcow2,cache=none -device scsi-hd,drive=hd0
lscpu inside guest:
Actual:
[root@atest-guest ~]# lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0,2,4,6
Off-line CPU(s) list: 1,3,5,7 --------------------------NOK
Thread(s) per core: 1
Core(s) per socket: 4
Socket(s): 1
NUMA node(s): 1
Model: 2.3 (pvr 004e 1203)
Model name: POWER9 (architected), altivec supported
Hypervisor vendor: KVM
Virtualization type: para
L1d cache: 128 KiB
L1i cache: 128 KiB
NUMA node0 CPU(s): 0,2,4,6
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Mitigation; RFI Flush, L1D private per thread
Vulnerability Mds: Not affected
Vulnerability Meltdown: Mitigation; RFI Flush, L1D private per thread
Vulnerability Spec store bypass: Mitigation; Kernel entry/exit barrier (eieio)
Vulnerability Spectre v1: Mitigation; __user pointer sanitization, ori31
speculation barrier enabled
Vulnerability Spectre v2: Mitigation; Software count cache flush (hardwar
e accelerated), Software link stack flush
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected
Expected:
[root@atest-guest ~]# lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 1
Core(s) per socket: 8
Socket(s): 1
NUMA node(s): 1
Model: 2.3 (pvr 004e 1203)
Model name: POWER9 (architected), altivec supported
Hypervisor vendor: KVM
Virtualization type: para
L1d cache: 256 KiB
L1i cache: 256 KiB
NUMA node0 CPU(s): 0-7
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Mitigation; RFI Flush, L1D private per thread
Vulnerability Mds: Not affected
Vulnerability Meltdown: Mitigation; RFI Flush, L1D private per thread
Vulnerability Spec store bypass: Mitigation; Kernel entry/exit barrier (eieio)
Vulnerability Spectre v1: Mitigation; __user pointer sanitization, ori31
speculation barrier enabled
Vulnerability Spectre v2: Mitigation; Software count cache flush (hardwar
e accelerated), Software link stack flush
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected
There by further vcpuhotplug operation fails...
Did a git bisect and the bad commit is
acbdb9956fe93f4669141f103cb543d3025775db is the first bad commit
commit acbdb9956fe93f4669141f103cb543d3025775db
Author: Cédric Le Goater <email address hidden>
Date: Thu Aug 20 15:45:46 2020 +0200
spapr/xive: Allocate IPIs independently from the other sources
The vCPU IPIs are now allocated in kvmppc_xive_cpu_connect() when the
vCPU connects to the KVM device and not when all the sources are reset
in kvmppc_xive_source_reset()
This requires extra care for hotplug vCPUs and VM restore.
Signed-off-by: Cédric Le Goater <email address hidden>
Message-Id: <email address hidden>
Signed-off-by: David Gibson <email address hidden>
hw/intc/spapr_xive_kvm.c | 47 ++++++++++++++++++++++++++++++++++++++++++-----
1 file changed, 42 insertions(+), 5 deletions(-)
# git bisect log
git bisect start
# good: [d0ed6a69d399ae193959225cdeaa9382746c91cc] Update version for v5.1.0 release
git bisect good d0ed6a69d399ae193959225cdeaa9382746c91cc
# bad: [7daf8f8d011cdd5d3e86930ed2bde969425c790c] Merge remote-tracking branch 'remotes/kevin/tags/for-upstream' into staging
git bisect bad 7daf8f8d011cdd5d3e86930ed2bde969425c790c
# skip: [7595a65818ea9b49c36650a8c217a1ef9bd6e62a] hw/riscv: Sort the Kconfig options in alphabetical order
git bisect skip 7595a65818ea9b49c36650a8c217a1ef9bd6e62a
# skip: [3b65b742543bc6c2ad35e3b42401a26b48a87f26] target/hppa: Fix boot with old Linux installation CDs
git bisect skip 3b65b742543bc6c2ad35e3b42401a26b48a87f26
# bad: [f4ef8c9cc10b3bee829b9775879d4ff9f77c2442] Merge remote-tracking branch 'remotes/ehabkost/tags/machine-next-pull-request' into staging
git bisect bad f4ef8c9cc10b3bee829b9775879d4ff9f77c2442
# good: [4ee40a6b98c02b72fc5dd262df9d3ac8680d767b] hw/usb: Add U2F device check to passthru mode
git bisect good 4ee40a6b98c02b72fc5dd262df9d3ac8680d767b
# skip: [fe4b0b5bfa96c38ad1cad0689a86cca9f307e353] tcg: Implement 256-bit dup for tcg_gen_gvec_dup_mem
git bisect skip fe4b0b5bfa96c38ad1cad0689a86cca9f307e353
# skip: [287b1defeb44398d02669d97ebdc347d650f274d] target/microblaze: Cache mem_index in DisasContext
git bisect skip 287b1defeb44398d02669d97ebdc347d650f274d
# skip: [7a1fb2ef40df508e90eb756a80d67e6435246cae] block/nvme: Extract nvme_poll_queue()
git bisect skip 7a1fb2ef40df508e90eb756a80d67e6435246cae
# good: [536e340f464d7c2ef55cca47c7535d9409bf03c7] target/microblaze: Convert msrclr, msrset to decodetree
git bisect good 536e340f464d7c2ef55cca47c7535d9409bf03c7
# good: [227de21ed0759e275a469394af72c999d0134bb5] Merge remote-tracking branch 'remotes/rth/tags/pull-tcg-20200903' into staging
git bisect good 227de21ed0759e275a469394af72c999d0134bb5
# bad: [b95ba83fc56ebfc4b6869f21db0c757c0c191104] Merge remote-tracking branch 'remotes/dgibson/tags/ppc-for-5.2-20200908' into staging
git bisect bad b95ba83fc56ebfc4b6869f21db0c757c0c191104
# good: [789035f1239054331b335801a06bdbef026f02e1] oss-fuzz: fix rpath
git bisect good 789035f1239054331b335801a06bdbef026f02e1
# good: [00942071a0eabeb3ebc3bd594296859587f8f3c8] Merge remote-tracking branch 'remotes/rth/tags/pull-mb-20200907-2' into staging
git bisect good 00942071a0eabeb3ebc3bd594296859587f8f3c8
# bad: [554c2169e9251ca2829ab968bd9ba5641a5abe1d] ppc/spapr: Use start-powered-off CPUState property
git bisect bad 554c2169e9251ca2829ab968bd9ba5641a5abe1d
# good: [235d3b116213828f4206e2e4b199a32bffc96f35] spapr/xive: Modify kvm_cpu_is_enabled() interface
git bisect good 235d3b116213828f4206e2e4b199a32bffc96f35
# bad: [90d282d0858cf5a38f3e8a7e201aeab2a0ccbe88] ppc/spapr_nvdimm: use g_autofree in spapr_nvdimm_validate_opts()
git bisect bad 90d282d0858cf5a38f3e8a7e201aeab2a0ccbe88
# bad: [acbdb9956fe93f4669141f103cb543d3025775db] spapr/xive: Allocate IPIs independently from the other sources
git bisect bad acbdb9956fe93f4669141f103cb543d3025775db
# good: [fa94447a2cd6643609d5822d5b5f739dc8ad8a8c] spapr/xive: Use kvmppc_xive_source_reset() in post_load
git bisect good fa94447a2cd6643609d5822d5b5f739dc8ad8a8c
# first bad commit: [acbdb9956fe93f4669141f103cb543d3025775db] spapr/xive: Allocate IPIs independently from the other sources
Regards,
-Satheesh
Fixed by reverting the series that caused the regression.
https://git.qemu.org/?p=qemu.git;a=commit;h=6d24795ee7e3199d199d3c415312c93382ad1807
The optimization needs to be reworked later.
Tested with latest upstream and found the issue is fixed,
# git log -1
commit dd3d2340c4076d1735cd0f7cb61f4d8622b9562d (HEAD -> master, tag: v5.2.0-rc3, origin/master, origin/HEAD)
Author: Peter Maydell <email address hidden>
Date: Tue Nov 24 22:13:30 2020 +0000
Update version for v5.2.0-rc3 release
Signed-off-by: Peter Maydell <email address hidden>
/home/sath/qemu/build/qemu-system-ppc64 -name vm1 -M pseries,vsmt=2 -accel kvm -m 4096 -smp 8,cores=8,threads=1 -nographic -nodefaults -serial mon:stdio -vga none -nographic -device virtio-scsi-pci -drive file=/home/sath/tests/data/avocado-vt/images/fdevel-ppc64le.qcow2,if=none,id=hd0,format=qcow2,cache=none -device scsi-hd,drive=hd0
Fedora 33 (Thirty Three Prerelease)
Kernel 5.8.13-300.fc33.ppc64le on an ppc64le (hvc0)
atest-guest login: root
Password:
Login incorrect
atest-guest login: root
Password:
Last login: Wed Nov 18 09:03:24 on hvc0
[root@atest-guest ~]# lscpu
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 1
Core(s) per socket: 8
Socket(s): 1
NUMA node(s): 1
Regards,
-Satheesh
Released with QEMU v5.2.0.
|