1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
|
graphic: 0.545
semantic: 0.528
instruction: 0.518
device: 0.510
mistranslation: 0.471
other: 0.445
network: 0.396
boot: 0.396
KVM: 0.381
socket: 0.372
assembly: 0.233
vnc: 0.187
qemu-system-x86_64: "VFIO_MAP_DMA : -28" error when we attache 6 VF's to guest machine
We are trying to attach 6 VF's to the guest machine on 4.1.1 qemu emulator.
We are observing "VFIO_MAP_DMA : -28" error.
We are using w-bits=48 bits while lunching VM.
Please provide information how you started QEMU, and some information about your PCI device (e.g. the output of lspci).
qemu-system-x86_64 -name guest=fedora24 -machine q35,accel=kvm,kernel-irqchip=split \
-enable-kvm \
-m 4G \
-smp 8,sockets=1,cores=8,threads=1 \
-device intel-iommu,intremap=on,caching-mode=on,aw-bits=48 \
-drive file=<OS_IMAGE_FILE>,format=raw \
-device ioh3420,id=pcie.1,chassis=1 \
-device virtio-net-pci,bus=pcie.1,disable-legacy=on,disable-modern=off,iommu_platform=on,ats=on,netdev=net0 \
-netdev user,id=net0,hostfwd=tcp::1111-:22\
-device vfio-pci,host=3f:02.1 \
-device vfio-pci,host=3f:02.2 \
-device vfio-pci,host=3f:02.3 \
-device vfio-pci,host=3f:02.4 \
-device vfio-pci,host=3d:02.4 \
-device vfio-pci,host=3d:02.5 \
-device vfio-pci,host=3d:02.6 \
-nographic
Please find the above qemu command to lunch guest machine
Presumably w-bits (aw-bits?) implies using intel-iommu, there's a opportunity for the vfio iommu backend to return -ENOSPC (-28) if we exceed the default number of in-flight DMA mappings per container. The default limit is 65535. You can try increasing this by changing the dma_entry_limit module option on the vfio_iommu_type1 module. Note that in a typical vIOMMU config there's a container per device, so the number of VFs attached is possibly not a factor. It is however a lot of DMA mappings for a single device if this is the issue and you'd generally want to boot the guest with iommu=pt in order to have reasonable assigned device performance with a vIOMMU, which would also greatly reduce the number of mappings.
After increasing dma_entry_limit limit no issue observed.
But ideal senario device is getting hung and recovery happening only with host hard rebooting.
[Expired for QEMU because there has been no activity for 60 days.]
|