Ubuntu 18.04 - vm disk i/o performance issue when using file system passthrough

== Comment: #0 - I-HSIN CHUNG <email address hidden> - 2019-11-15 12:35:05 ==
---Problem Description---
Ubuntu 18.04 - vm disk i/o performance issue when using file system passthrough
 
Contact Information = <email address hidden> 
 
---uname output---
Linux css-host-22 4.15.0-1039-ibm-gt #41-Ubuntu SMP Wed Oct 2 10:52:25 UTC 2019 ppc64le ppc64le ppc64le GNU/Linux (host) Linux ubuntu 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:08:54 UTC 2019 ppc64le ppc64le ppc64le GNU/Linux (vm)
 
Machine Type = p9/ac922 
 
---Debugger---
A debugger is not configured
 
---Steps to Reproduce---
 1. Env: Ubuntu 18.04.3 LTS?Genesis kernel linux-ibm-gt - 4.15.0-1039.41?qemu 1:2.11+dfsg-1ubuntu7.18 ibmcloud0.3 or 1:2.11+dfsg-1ubuntu7.19 ibm-cloud1?fio-3.15-4-g029b

2. execute run.sh to run fio benchmark:

2.1) run.sh:
#!/bin/bash
  
for bs in  4k 16m
do

for rwmixread in 0 25 50 75 100
do

for numjobs in 1 4 16 64
do
echo ./fio j1.txt --bs=$bs --rwmixread=$rwmixread --numjobs=$numjobs
./fio j1.txt --bs=$bs --rwmixread=$rwmixread --numjobs=$numjobs

done
done
done

2.2) j1.txt:

[global]
direct=1
rw=randrw
refill_buffers
norandommap
randrepeat=0
ioengine=libaio
iodepth=64
runtime=60

allow_mounted_write=1

[job2]
new_group
filename=/dev/vdb
filesize=1000g
cpus_allowed=0-63
numa_cpu_nodes=0
numa_mem_policy=bind:0

3. performance profile:
device passthrough performance for the nvme: 
http://css-host-22.watson.ibm.com/rundir/nvme_vm_perf_vm/20191011-112156/html/#/measurement/vm/ubuntu (I/O bandwidth achieved inside VM in GB/s range)

file system passthrough
http://css-host-22.watson.ibm.com/rundir/nvme_vm_perf_vm/20191106-123613/html/#/measurement/vm/ubuntu (I/o bandwidth achieved inside the VM is very low)

desired performance when using file system passthrough should be similar to the device passthrough
 
Userspace tool common name: fio 
 
The userspace tool has the following bit modes: should be 64 bit 

Userspace rpm: ? 

Userspace tool obtained from project website:  na 
 
*Additional Instructions for <email address hidden>: 
-Post a private note with access information to the machine that the bug is occuring on.
-Attach ltrace and strace of userspace application.