path: root/results/classifier/zero-shot/105/mistranslation/814222

mistranslation: 0.743
semantic: 0.641
assembly: 0.574
other: 0.555
graphic: 0.500
KVM: 0.499
instruction: 0.462
vnc: 0.413
socket: 0.410
device: 0.408
boot: 0.391
network: 0.382

kvm cannot use vhd files over 127GB

The primary use case for using vhds with KVM is to perform a conversion to a raw image file so that one could move from Hyper-V to Linux-KVM.  See more on this http://blog.allanglesit.com/2011/03/linux-kvm-migrating-hyper-v-vhd-images-to-kvm/

# kvm-img convert -f raw -O vpc /root/file.vhd /root/file.img

The above works great if you have VHDs smaller than 127GB, however if it is larger, then no error is generated during the conversion process, but it appears to just process up to that 127GB barrier and no more.  Also of note.  VHDs can also be run directly using KVM if they are smaller than 127GB.  VHDs can be read and function well using virtualbox as well as hyper-v, so I suspect the problem lies not with the VHD format (since that has a 2TB limitation).  But instead with how qemu-kvm is interpreting them.

BORING VERSION INFO:
# cat /etc/issue
Ubuntu 11.04 \n \l
# uname -rmiv
2.6.38-8-server #42-Ubuntu SMP Mon Apr 11 03:49:04 UTC 2011 x86_64 x86_64
# apt-cache policy kvm
kvm:
  Installed: 1:84+dfsg-0ubuntu16+0.14.0+noroms+0ubuntu4.1
  Candidate: 1:84+dfsg-0ubuntu16+0.14.0+noroms+0ubuntu4.1
  Version table:
 *** 1:84+dfsg-0ubuntu16+0.14.0+noroms+0ubuntu4.1 0
        500 http://apt.sonosite.com/ubuntu/ natty-updates/main amd64 Packages
        500 http://apt.sonosite.com/ubuntu/ natty-security/main amd64 Packages
        100 /var/lib/dpkg/status
     1:84+dfsg-0ubuntu16+0.14.0+noroms+0ubuntu4 0
        500 http://apt.sonosite.com/ubuntu/ natty/main amd64 Packages
# apt-cache policy libvirt-bin
libvirt-bin:
  Installed: 0.8.8-1ubuntu6.2
  Candidate: 0.8.8-1ubuntu6.2
  Version table:
 *** 0.8.8-1ubuntu6.2 0
        500 http://apt.sonosite.com/ubuntu/ natty-updates/main amd64 Packages
        500 http://apt.sonosite.com/ubuntu/ natty-security/main amd64 Packages
        100 /var/lib/dpkg/status
     0.8.8-1ubuntu6 0
        500 http://apt.sonosite.com/ubuntu/ natty/main amd64 Packages

qemu-img version 0.14.0

# vboxmanage -v
4.0.12r72916


REPRODUCTION STEPS (requires Windows 7 or Windows 2008 R2 with < 1GB of free space)

##  WINDOWS  MACHINE  ##

Use Computer Management > Disk Management
-Create 2 VHD files, both dynamically expanding 120GB and 140GB respectively.
-Do not initialize  or format.

These files will need to be transferred to an Ubuntu KVM machine (pscp is what I used but usb would work as well).

##  UBUNTU KVM MACHINE  ##

# ls *.vhd
120g-dyn.vhd  140g-dyn.vhd
# kvm-img info 120g-dyn.vhd 
image: 120g-dyn.vhd
file format: vpc
virtual size: 120G (128847052800 bytes)
disk size: 244K
# kvm-img info 140g-dyn.vhd 
image: 140g-dyn.vhd
file format: vpc
virtual size: 127G (136899993600 bytes)
disk size: 284K
# kvm-img info 120g-dyn.vhd | grep "virtual size"
virtual size: 120G (128847052800 bytes)
# kvm-img info 140g-dyn.vhd | grep "virtual size"
virtual size: 127G (136899993600 bytes)

Regardless of how big the second vhd is I always get a virtual size of 127G

Now if we use virtualbox to view the vhds we see markedly different results.

# VBoxManage showhdinfo 120g-dyn.vhd
UUID:                 e63681e0-ff12-4114-85de-7d13562b36db
Accessible:           yes
Logical size:         122880 MBytes
Current size on disk: 0 MBytes
Type:                 normal (base)
Storage format:       VHD
Format variant:       dynamic default
Location:             /root/120g-dyn.vhd
# VBoxManage showhdinfo 140g-dyn.vhd 
UUID:                 94531905-46b4-469f-bb44-7a7d388fb38f
Accessible:           yes
Logical size:         143360 MBytes
Current size on disk: 0 MBytes
Type:                 normal (base)
Storage format:       VHD
Format variant:       dynamic default
Location:             /root/140g-dyn.vhd

# kvm-img convert -f vpc -O raw 120g-dyn.vhd 120g-dyn.img
#
# kvm-img convert -f vpc -O raw 140g-dyn.vhd 140g-dyn.img
#

# kvm-img info 120g-dyn.img 
image: 120g-dyn.img
file format: raw
virtual size: 120G (128847052800 bytes)
disk size: 0
# kvm-img info 120g-dyn.img | grep "virtual size"
virtual size: 120G (128847052800 bytes)
# kvm-img info 140g-dyn.img 
image: 140g-dyn.img
file format: raw
virtual size: 127G (136899993600 bytes)
disk size: 0
# kvm-img info 140g-dyn.img | grep "virtual size"
virtual size: 127G (136899993600 bytes)

Notice after the conversion the raw image will the taken on the partial geometry of the vhd, thus rendering that image invalid.

vboxmanage has a clonehd option which allows you to successfully convert vhd to a raw image, which kvm then sees properly.

For giggles I also tested with a 140GB fixed VHD (in the same manner as above) and it displayed the virtual size as correct, so a good work around is to convert your VHDs to fixed, then use kvm-img to convert them.

Keep in mind that these reproduction steps will not have a file systems therefore no valid data, if there were for example NTFS with a text file the problem would still occur but more importantly the guest trying to use it would not be able to open the disk because of it being unable to find the final sector.

So long story short I think we are dealing with 2 issues here.

1) kvm not being able to deal with dynamic VHD files larger than 127GB
2) kvm-img not generating an error when it "fails" at converting or displaying information on dynamic VHDs larger than 127GB.  The error should be something like "qemu-kvm does not support dynamic VHD files larger that 127GB..."

Thanks for taking the time to submit this bug.

It looks like the 127G limit is known.  A recent patch went in to help with the symptom you are seeing, but unfortunately it only makes the failure detectable  :)  It's a start at least.

The following commit should be pulled in:

commit 6e9ea0c0629fe25723494a19498bedf4b781cbfa
Author: aurel32 <aurel32@c046a42c-6fe2-441c-8c8c-71466251a162>
Date:   Wed Apr 15 14:42:46 2009 +0000

    block-vpc: Don't silently create smaller image than requested
    
    The algorithm from the VHD specification for CHS calculation silently limits
    images to 127 GB which may confuse a user who requested a larger image. Better
    output an error message and abort.
    
    Signed-off-by: Kevin Wolf <email address hidden>
    Signed-off-by: Aurelien Jarno <email address hidden>
    
    git-svn-id: svn://svn.savannah.nongnu.org/qemu/trunk@7109 c046a42c-6fe2-441c-8c8c-71466251a162


Hm, no - that patch is already in natty and oneiric's qemu-kvm.

I'm afraid I'll need time to find a place where I can reproduce this.

As converting to fixed is listed as a workaround, I'm changing the priority to low per priority definitions.

Are the priority definitions documented somewhere?

I personally think you were right on when you had the priority at medium.  

Primarily because of the fact that no error is generated.  It can't just silently fail.  If it generated an error (so that people knew they needed to look for a work around) then I would agree that fixing the bug itself would be a low priority, as the work around is simple for anyone to implement.

@Matthew,

yes, the definitions are at https://wiki.ubuntu.com/Bugs/Importance.

Your reasoning makes sense.  I'll bump it back up, thanks.

This could be tough to reproduce, as I don't seem to have a way to
create a vpc image > 127G in the first place:

root@ip-10-36-186-165:/mnt# qemu-img create -f vpc vpc2.img 130G
Formatting 'vpc2.img', fmt=vpc size=139586437120 
qemu-img: The image size is too large for file format 'vpc'

root@ip-10-36-186-165:/mnt# qemu-img create -f raw raw1.img 130G
Formatting 'raw1.img', fmt=raw size=139586437120
root@ip-10-36-186-165:/mnt# qemu-img convert -f raw -O vpc raw1.img vpc1.img
qemu-img: The image size is too large for file format 'vpc'

root@ip-10-36-186-165:/mnt# qemu-img create -f vpc vpc1.img 127G
Formatting 'vpc1.img', fmt=vpc size=136365211648
root@ip-10-36-186-165:/mnt# qemu-img convert -f vpc -O raw vpc1.img raw2.img
root@ip-10-36-186-165:/mnt# qemu-img info raw2.img
image: raw2.img
file format: raw
virtual size: 127G (136365219840 bytes)
disk size: 0




This is a dynamically expanding VHD file created using the reproduction steps above on Windows 7.  This one is 120GB and converts correctly.

This has not been formatted or even initialized.

"kvm-img info 120g-dynamic.vhd" shows the proper geometry.


This is a dynamically expanding VHD file created using the reproduction steps above on Windows 7.  This one is 140GB and silently errors on conversion.

This has not been formatted or even initialized.

"kvm-img info 140g-dynamic.vhd" does not show the proper geometry.

I have attached a couple of VHDs that I created with Windows 7.  These should be helpful in your reproduction.

Also looking at your notes it looks like that previous patch which was committed only affected the creation.  So perhaps the same sort of check can be incorporated into the conversion process as well, so that you don't have the silent error.

@Matthew

thanks for the attachments.



This bug was fixed in the package qemu-kvm - 0.14.1+noroms-0ubuntu5

---------------
qemu-kvm (0.14.1+noroms-0ubuntu5) oneiric; urgency=low

  * debian/patches/vpc.patch: detect vpc files which are too big
    (LP: #814222)
 -- Serge Hallyn <email address hidden>   Mon, 12 Sep 2011 11:28:36 -0500

I came here from : http://lists.gnu.org/archive/html/qemu-devel/2011-07/msg02806.html

Actually, I experience an issue which may be useful to you.

I have a corrupted VHD file (as explained in that thread : https://forums.virtualbox.org/viewtopic.php?f=7&t=20614 ). I wanted to follow that procedure to solve my issue :

    qemu-img convert -O raw miimagen.vhd miimagen.bin
    VBoxManage convertdd miimagen.bin miimagen.vdi


but qemu-img convert -O raw miimagen.vhd miimagen.bin triggers the qemu-img: Could not open 'img.VHD': File too large error message.

Since, my file is 52,6 Go and the output is raw format. I guess it should not trigger that exception? or is that the normal behavior? Is there a way to bypass this limit? I use qemu-img 1.0 version.

Hope it can help your development (and it can help me back)
Thanks,
simon

Looks like Serge's fix has been included here:
http://git.qemu.org/?p=qemu.git;a=commitdiff;h=efc8243d00ab4cf4fa05a9b
... so let's close this bug now.