qemu-nbd -r -c taints device for subsequent usage, even after -d Something about qemu-nbd -r -c /dev/nbd0 someimg leaves cruft behind - subsequent connections get marked readonly. This is on quantal, haven't checked precise or raring. To demonstrate: # use one image qemu-img create -f qcow2 /tmp/1.qcow2 100M sudo qemu-nbd -c /dev/nbd2 /tmp/1.qcow2 sudo mkfs -t ext4 /dev/nbd2 sudo qemu-nbd -d /dev/nbd2 # use a second one on the same nbd device, shows that reuse works: qemu-img create -f qcow2 /tmp/2.qcow2 100M sudo qemu-nbd -c /dev/nbd2 /tmp/2.qcow2 sudo mkfs -t ext4 /dev/nbd2 sudo qemu-nbd -d /dev/nbd2 # connect an image in read only mode sudo qemu-nbd -r -c /dev/nbd2 /tmp/2.qcow2 sudo dumpe2fs /dev/nbd2 | head -n 3 sudo qemu-nbd -d /dev/nbd2 # now try to reuse in read-write mode again: qemu-img create -f qcow2 /tmp/3.qcow2 100M sudo qemu-nbd -c /dev/nbd2 /tmp/3.qcow2 sudo mkfs -t ext4 /dev/nbd2 # here it goes boom: mke2fs 1.42.5 (29-Jul-2012) /dev/nbd2: Operation not permitted while setting up superblock # still need to cleanup sudo qemu-nbd -d /dev/nbd2 Happens on Precise as well. Quick code read - I think that this block: if (flags & NBD_FLAG_READ_ONLY) { int read_only = 1; TRACE("Setting readonly attribute"); if (ioctl(fd, BLKROSET, (unsigned long) &read_only) < 0) { int serrno = errno; LOG("Failed setting read-only attribute"); return -serrno; } } in nbd.c should be { int read_only = 0; if (flags & NBD_FLAG_READ_ONLY) read_only = 1; TRACE("Setting readonly attribute"); if (ioctl(fd, BLKROSET, (unsigned long) &read_only) < 0) { int serrno = errno; LOG("Failed setting read-only attribute"); return -serrno; } } http://paste.ubuntu.com/1352684/ is a debdiff, uploading the source format 3 patch as well Fixed patch - I had my sense inverted... http://paste.ubuntu.com/1352711/ Thanks, this still applies upstream as well. To some extent it is a bug in the upstream kernel, which doesn't reset state properly. However, the qemu patch is also good. Thanks! Thanks, Paul, I'll cherrypick commit c8969eded252058e90e91f12f75f32aceae46ec9 into the ubuntu packages This bug was fixed in the package qemu-kvm - 1.2.0+noroms-0ubuntu4 --------------- qemu-kvm (1.2.0+noroms-0ubuntu4) raring; urgency=low [ Serge Hallyn ] * debian/qemu-kvm.postinst: use udevadm trigger to change /dev/kvm perms as recommended by Steve Langasek (LP: #1057024) * apply debian/patches/nbd-fixes-to-read-only-handling.patch from upstream to make read-write mount after read-only mount work. (LP: #1077838) [ Robert Collins ] * Fix upstart job to succeed if ksm settings can't be altered in the same way other settings are handled. (LP: #1078530) -- Serge Hallyn