vfio-pci - Intel Arc DG2 - host errors
Description of problem:
The host continues to respond (slowly) after the VM is shutdown. Speeds back up to normal after about an hour. However, a reboot is required to get the host to operate normally.

When shutting down the VM, the host starts to display the following messages in dmesg:

[Thu Apr 13 01:30:47 2023] vfio-pci 0000:18:00.0: not ready 1023ms after FLR; waiting
[Thu Apr 13 01:30:49 2023] vfio-pci 0000:18:00.0: not ready 2047ms after FLR; waiting
[Thu Apr 13 01:30:52 2023] vfio-pci 0000:18:00.0: not ready 4095ms after FLR; waiting
[Thu Apr 13 01:30:57 2023] vfio-pci 0000:18:00.0: not ready 8191ms after FLR; waiting
[Thu Apr 13 01:31:06 2023] vfio-pci 0000:18:00.0: not ready 16383ms after FLR; waiting
[Thu Apr 13 01:31:25 2023] vfio-pci 0000:18:00.0: not ready 32767ms after FLR; waiting
[Thu Apr 13 01:31:59 2023] vfio-pci 0000:18:00.0: not ready 65535ms after FLR; giving up
[Thu Apr 13 01:32:11 2023] vfio-pci 0000:18:00.0: not ready 1023ms after bus reset; waiting
[Thu Apr 13 01:32:13 2023] vfio-pci 0000:18:00.0: not ready 2047ms after bus reset; waiting
[Thu Apr 13 01:32:16 2023] vfio-pci 0000:18:00.0: not ready 4095ms after bus reset; waiting
[Thu Apr 13 01:32:21 2023] vfio-pci 0000:18:00.0: not ready 8191ms after bus reset; waiting
[Thu Apr 13 01:32:31 2023] vfio-pci 0000:18:00.0: not ready 16383ms after bus reset; waiting
[Thu Apr 13 01:32:48 2023] vfio-pci 0000:18:00.0: not ready 32767ms after bus reset; waiting
[Thu Apr 13 01:33:22 2023] vfio-pci 0000:18:00.0: not ready 65535ms after bus reset; giving up
Steps to reproduce:
1. Shutdown VM.
Additional information:
I have startup and shutdown scripts that detach and reattach the card and these scripts work fine if I test them alone. It's only when I shutdown the VM that issue presents itself.

revert.sh

```
#!/bin/bash
set -x

systemctl reboot # to workaround host lockup on shutdown

# Load the config file with our environmental variables
source "/etc/libvirt/hooks/kvm.conf"
source "/etc/libvirt/hooks/vmPreBootSetup"

cpuSchedutil

# Unload VFIO-PCI Kernel Driver
modprobe -r vfio_pci
modprobe -r vfio_iommu_type1
modprobe -r vfio

# Re-Bind GPU to our display drivers
virsh nodedev-reattach $VIRSH_GPU_VIDEO
virsh nodedev-reattach $VIRSH_GPU_AUDIO

#modprobe drm_buddy intel_gtt video drm_display_helper cec ttm i915

# Restart Display Manager
systemctl restart sddm.service
```



Full dmesg log: 
[vfio_13_april_2023.txt](/uploads/5d5b642595c53cabb3c3608c07d59eb3/vfio_13_april_2023.txt)