1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
|
graphic: 0.915
permissions: 0.893
ppc: 0.891
semantic: 0.872
arm: 0.861
PID: 0.857
user-level: 0.854
peripherals: 0.845
performance: 0.845
debug: 0.842
register: 0.841
assembly: 0.840
x86: 0.831
device: 0.812
boot: 0.806
files: 0.802
hypervisor: 0.796
TCG: 0.794
architecture: 0.791
kernel: 0.785
virtual: 0.756
risc-v: 0.750
mistranslation: 0.745
socket: 0.726
network: 0.722
VMM: 0.715
vnc: 0.712
KVM: 0.686
i386: 0.511
qemu-system-x86_64: Linux kernel warning when CONFIG_NUMA_EMU is enabled
Description of problem:
When Linux kernel is run on qemu 6.2, it prints a warning when `NUMA_EMU` is used. When the same kernel is run on qemu 6.1.1 (`54e1f5be86dd11744e45da8be6afad01d01d59e7`) or earlier, no such warning is printed.
```
[ 0.341924] smpboot: CPU0: Intel QEMU Virtual CPU version 2.5+ (family: 0xf, model: 0x6b, stepping: 0x1)
[ 0.342371] Performance Events: unsupported Netburst CPU model 107 no PMU driver, software events only.
[ 0.343302] rcu: Hierarchical SRCU implementation.
[ 0.344470] smp: Bringing up secondary CPUs ...
[ 0.345349] x86: Booting SMP configuration:
[ 0.345945] .... node #1, CPUs: #1
[ 0.014099] ------------[ cut here ]------------
[ 0.014099] sched: CPU #1's llc-sibling CPU #0 is not on the same node! [node: 1 != 0]. Ignoring dependency.
[ 0.014099] WARNING: CPU: 1 PID: 0 at arch/x86/kernel/smpboot.c:423 topology_sane.isra.0+0x62/0x70
[ 0.014099] Modules linked in:
[ 0.014099] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.16.9 #6
[ 0.014099] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.15.0-0-g2dd4b9b3f840-prebuilt.qemu.org 04/01/2014
[ 0.014099] RIP: 0010:topology_sane.isra.0+0x62/0x70
[ 0.014099] Code: 80 3d e1 72 a1 01 00 75 f6 48 83 ec 08 4c 89 da 44 89 d6 48 c7 c7 c0 cd f4 8b 88 44 24 07 c6 05 c3 72 a1 01 01 e8 3c 16 b7 00 <0f> 0b 0f b6 44 24 07 48 83 c4 08 c3 66 90 48 8b 0d 21 95 a3 01 0f
[ 0.014099] RSP: 0000:ffffa8c3006a3ed8 EFLAGS: 00010086
[ 0.014099] RAX: 0000000000000000 RBX: ffffa335fdc15480 RCX: 0000000000000000
[ 0.014099] RDX: 0000000000000002 RSI: 00000000ffffffea RDI: 00000000ffffffff
[ 0.014099] RBP: ffffa3353dc15480 R08: ffffffff8c335ac8 R09: 00000000ffffdfff
[ 0.014099] R10: ffffffff8c255ae0 R11: ffffffff8c255ae0 R12: 0000000000000001
[ 0.014099] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000001
[ 0.014099] FS: 0000000000000000(0000) GS:ffffa335fdc00000(0000) knlGS:0000000000000000
[ 0.014099] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 0.014099] CR2: 0000000000000000 CR3: 0000000112a0c000 CR4: 00000000000006e0
[ 0.014099] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 0.014099] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[ 0.014099] Call Trace:
[ 0.014099] <TASK>
[ 0.014099] set_cpu_sibling_map+0x16a/0x560
[ 0.014099] start_secondary+0x42/0xf0
[ 0.014099] secondary_startup_64_no_verify+0xc2/0xcb
[ 0.014099] </TASK>
[ 0.014099] Kernel panic - not syncing: panic_on_warn set ...
[ 0.014099] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.16.9 #6
[ 0.014099] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS rel-1.15.0-0-g2dd4b9b3f840-prebuilt.qemu.org 04/01/2014
[ 0.014099] Call Trace:
[ 0.014099] <TASK>
[ 0.014099] dump_stack_lvl+0x34/0x44
[ 0.014099] panic+0xef/0x2a6
[ 0.014099] ? topology_sane.isra.0+0x62/0x70
[ 0.014099] __warn.cold+0x26/0x30
[ 0.014099] ? topology_sane.isra.0+0x62/0x70
[ 0.014099] report_bug+0x9a/0xc0
[ 0.014099] handle_bug+0x3c/0x60
[ 0.014099] exc_invalid_op+0x14/0x70
[ 0.014099] asm_exc_invalid_op+0x12/0x20
[ 0.014099] RIP: 0010:topology_sane.isra.0+0x62/0x70
[ 0.014099] Code: 80 3d e1 72 a1 01 00 75 f6 48 83 ec 08 4c 89 da 44 89 d6 48 c7 c7 c0 cd f4 8b 88 44 24 07 c6 05 c3 72 a1 01 01 e8 3c 16 b7 00 <0f> 0b 0f b6 44 24 07 48 83 c4 08 c3 66 90 48 8b 0d 21 95 a3 01 0f
[ 0.014099] RSP: 0000:ffffa8c3006a3ed8 EFLAGS: 00010086
[ 0.014099] RAX: 0000000000000000 RBX: ffffa335fdc15480 RCX: 0000000000000000
[ 0.014099] RDX: 0000000000000002 RSI: 00000000ffffffea RDI: 00000000ffffffff
[ 0.014099] RBP: ffffa3353dc15480 R08: ffffffff8c335ac8 R09: 00000000ffffdfff
[ 0.014099] R10: ffffffff8c255ae0 R11: ffffffff8c255ae0 R12: 0000000000000001
[ 0.014099] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000001
[ 0.014099] ? topology_sane.isra.0+0x62/0x70
[ 0.014099] set_cpu_sibling_map+0x16a/0x560
[ 0.014099] start_secondary+0x42/0xf0
[ 0.014099] secondary_startup_64_no_verify+0xc2/0xcb
[ 0.014099] </TASK>
[ 0.014099] ---[ end Kernel panic - not syncing: panic_on_warn set ... ]---
```
Steps to reproduce:
1. Check out the Linux kernel:
```
git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
cd linux-stable
git checkout v5.16.9
```
2. Generate the configuration file:
```
make defconfig
./scripts/config -e NUMA_EMU -e CMDLINE_BOOL --set-str CMDLINE "numa=fake=2 panic_on_warn=1" -d CONFIG_CMDLINE_OVERRIDE
```
3. Build the kernel
```
make -j32
```
4. Run qemu and wait for a couple of seconds:
```
./qemu-system-x86_64 -m 4G -smp 2 -kernel ~/linux-stable/arch/x86/boot/bzImage -append "console=ttyS0 root=/dev/sda earlyprintk=serial" -enable-kvm -nographic -snapshot
```
Additional information:
With explicit NUMA configuration, it boots fine:
```
./qemu-system-x86_64 -m 4G -smp 2 -object memory-backend-ram,size=8G,id=m0 -numa node,cpus=0-1,nodeid=0,memdev=m0 -kernel ~/linux-stable/arch/x86/boot/bzImage -append "console=ttyS0 root=/dev/sda earlyprintk=serial" -enable-kvm -nographic -snapshot
```
On the host machine:
```
$ numactl -H
available: 2 nodes (0-1)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53
node 0 size: 95259 MB
node 0 free: 1767 MB
node 1 cpus: 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71
node 1 size: 96757 MB
node 1 free: 2407 MB
node distances:
node 0 1
0: 10 21
1: 21 10
```
|