9pfs does not honor open file handles on unlinked files This was originally filed over here: https://bugzilla.redhat.com/show_bug.cgi?id=1114221 The open-unlink-fstat idiom used in some places to create an anonymous private temporary file does not work in a QEMU guest over a virtio-9p filesystem. Version-Release number of selected component (if applicable): qemu-kvm-1.6.2-6.fc20.x86_64 qemu-system-x86-1.6.2-6.fc20.x86_64 (those are fedora RPMs) How reproducible: Always. See this example C program: https://bugzilla.redhat.com/attachment.cgi?id=913069 Steps to Reproduce: 1. Export a filesystem with virt-manager for the guest. (type: mount, driver: default, mode: passthrough) 2. Start guest and mount that filesystem (mount -t 9p -o trans=virtio,version=9p2000.L ...) 3. Run a program that uses open-unlink-fstat (in my case it was trying to compile Perl 5.20) Actual results: fstat fails: open("/home/tst/filename", O_RDWR|O_CREAT|O_EXCL, 0600) = 3 unlink("/home/tst/filename") = 0 fstat(3, 0x23aa1a8) = -1 ENOENT (No such file or directory) close(3) Expected results: open("/home/tst/filename", O_RDWR|O_CREAT|O_EXCL, 0600) = 3 unlink("/home/tst/filename") = 0 fstat(3, {st_mode=S_IFREG|0600, st_size=0, ...}) = 0 fcntl(3, F_SETFD, FD_CLOEXEC) = 0 close(3) Additional info: There was a patch put into the kernel back in '07 to handle this very problem for other filesystems; maybe its helpful: http://lwn.net/Articles/251228/ There is also a thread on LKML from last December specifically about this very problem: https://lkml.org/lkml/2013/12/31/163 There was a discussion on the QEMU list back in '11 that doesn't seem to have come to a conclusion, but did provide the test program that i've attached to this report: http://marc.info/?l=qemu-devel&m=130443605720648&w=2 This bug makes it difficult to run a Debian Jessie guest with a 9pfs root filesystem, because "apt-get update" uses the open-unlink-fstat idiom. With this bug present, apt dies with the following error: E: Unable to determine file size for fd 7 - fstat (2: No such file or directory) I've done some digging from the client side. As is mentioned in Miklos' original patch (which appears to have been shot down) the higher level implementation appear to be broken for this. Here's what the code looks like for fstat in fs/stat.c: int vfs_fstat(unsigned int fd, struct kstat *stat) { struct fd f = fdget_raw(fd); int error = -EBADF; if (f.file) { error = vfs_getattr(&f.file->f_path, stat); fdput(f); } return error; } In other words, it only uses the open fd to derrive a path and then executes the getattr off of that path. If that path no longer exists (because of unlink or remove) then you are hosed. In my understanding, the "work around" I suppose is the so-called 'silly renaming' where remove/unlink simply does a rename until all open instances are closed. The frustrating thing is that the 9p protocol is setup to work off of the fid, which maps to the fd -- so its perfectly capable of the original semantic but the high level code in fs/stat.c only wants to give a path. I can do a work around on the client where a getattr "favors" open fids for the operation or I can do the sillyrename hack that NFS and CIFS has used but both of those look god-awful. -eric On Fri, Apr 10, 2015 at 7:30 AM, Mark Glines