1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
|
id = 1705
title = "Illegal instruction when I want to numactl --cpubind=0 --membind=1 to CXL Memory"
state = "closed"
created_at = "2023-06-13T02:50:15.126Z"
closed_at = "2023-09-22T05:30:36.680Z"
labels = []
url = "https://gitlab.com/qemu-project/qemu/-/issues/1705"
host-os = "Ubuntu 20.04"
host-arch = "x86"
qemu-version = "8.0.50"
guest-os = "Ubuntu 22.04"
guest-arch = "x86"
description = """I ran QEMU for simulating CXL DRAM and when I tried to run `numactl --cpubind=0 --membind=1 ls` , I got `Illegal instruction`
The numa node 1 was the CXL DRAM simulated by QEMU.
> root@8003:~# numactl -H
> available: 2 nodes (0-1)
> node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47
> node 0 size: 32090 MB
> node 0 free: 31325 MB
> node 1 cpus:
> node 1 size: 32768 MB
> node 1 free: 32768 MB
> node distances:
> node 0 1
> 0: 10 20
> 1: 20 10
When I ran on numa node 1, no failed
> root@8003:~# numactl --membind=0 ls
> ndctl
When I ran on numa node 1(CXL DRAM),it failed.
> root@8003:~# numactl --membind=1 ls
> [ 913.975032] traps: ls[667] trap invalid opcode ip:7fdec255d180 sp:7ffd3c507288 error:0 in ld-linux-x86-64.so.2[7fdec2546000+2a000]
> **Illegal instruction**"""
reproduce = """1. start the guest
2. cxl list (we could see the simulated CXL DRAM)
> root@8003:~# cxl list
> [
> {
> "memdev":"mem0",
> "ram_size":34359738368,
> "serial":0,
> "host":"0000:0d:00.0"
> }
> ]
3. cxl create-region -t ram -d decoder0.0 -m mem0
4. daxctl reconfigure-device dax0.0 --mode=system-ram
5. numactl -H
> root@8003:~# numactl -H
> available: 2 nodes (0-1)
> node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47
> node 0 size: 32090 MB
> node 0 fr
> ee: 31254 MB
> node 1 cpus:
> node 1 size: 32768 MB
> node 1 free: 32768 MB
> node distances:
> node 0 1
> 0: 10 20
> 1: 20 10
6. numactl --membind=1 ls
> root@8003:~# numactl --membind=1 ls
> [38441.892140] **traps: ls[861] trap invalid opcode ip:7f15db6ac180 sp:7ffc648755c8 **error:0 in ld-linux-x86-64.so.2[7f15db695000+2a000]
> **Illegal instruction**"""
additional = """When I run dmesg, I found an error.
> root@8003:~# dmesg|grep error
> [ 2.321130] platform regulatory.0: Direct firmware load for regulatory.db failed with error -2
Since my CPU is a Xeon III, not a Xeon IV with CXL support, **I'm wondering if it's because the CPU doesn't support CXL instructions, or if the Xeon III can emulate it, just because my settings don't make sense**. If this is my settings problem, could you help me to deal this? Or it just caused by my Xeon III,I will update it to Xeon IV."""
|