other: 0.961 permissions: 0.949 debug: 0.946 graphic: 0.946 semantic: 0.938 vnc: 0.929 network: 0.928 PID: 0.927 performance: 0.927 socket: 0.926 device: 0.925 boot: 0.911 KVM: 0.903 files: 0.873 cache-miss-rate / Invalid JSON Hi, We have VMs which were started with an older version than qemu 2.1 which added "cache-miss-rate" property for XBZRLECacheStats. While trying to migrate the VM to a new host which is running a higher version (2.3) of Qemu we got an exception: virJSONValueFromString:1642 : internal error: cannot parse json {"return": {"expected-downtime": 1, "xbzrle-cache": {"bytes": 0, "cache-size": 67108864, "cache-miss-rate": -nan, "pages": 0, "overflow": 0, "cache-miss": 8933}, "status": "active", "disk": {"total": 429496729600, "dirty-sync-count": 0, "remaining": 193896382464, "mbps": 0, "transferred": 235600347136, "duplicate": 0, "dirty-pages-rate": 0, "skipped": 0, "normal-bytes": 0, "normal": 0}, "setup-time": 13, "total-time": 1543124, "ram": {"total": 8599183360, "dirty-sync-count": 4, "remaining": 30695424, "mbps": 830.636997, "transferred": 3100448901, "duplicate": 1358341, "dirty-pages-rate": 7, "skipped": 0, "normal-bytes": 3082199040, "normal": 752490}}, "id": "libvirt-186200"}: lexical error: malformed number, a digit is required after the minus sign. 67108864, "cache-miss-rate": -nan, "pages": 0, "overflow": 0 (right here) ------^ virNetClientStreamRaiseError:191 : stream aborted at client request Would it be possible to improve the JSON parser to skip the key if the value is incorrect instead of throwing an exception? Then hopefully qemu 2.3 or higher is able to handle the data without this property, falling back to its default. Sorry, the bug is in libvirt On 05/25/2016 02:46 AM, Marc Brothier wrote: > Public bug reported: > > Hi, > > We have VMs which were started with an older version than qemu 2.1 which > added "cache-miss-rate" property for XBZRLECacheStats. While trying to > migrate the VM to a new host which is running a higher version (2.3) of > Qemu we got an exception: > > virJSONValueFromString:1642 : internal error: cannot parse json {"return": {"expected-downtime": 1, "xbzrle-cache": {"bytes": 0, "cache-size": 67108864, "cache-miss-rate": -nan, "pages": 0, "overflow": 0, "cache-miss": 8933}, "status": "active", "disk": {"total": 429496729600, "dirty-sync-count": 0, "remaining": 193896382464, "mbps": 0, "transferred": 235600347136, "duplicate": 0, "dirty-pages-rate": 0, "skipped": 0, "normal-bytes": 0, "normal": 0}, "setup-time": 13, "total-time": 1543124, "ram": {"total": 8599183360, "dirty-sync-count": 4, "remaining": 30695424, "mbps": 830.636997, "transferred": 3100448901, "duplicate": 1358341, "dirty-pages-rate": 7, "skipped": 0, "normal-bytes": 3082199040, "normal": 752490}}, "id": "libvirt-186200"}: lexical error: malformed number, a digit is required after the minus sign. > 67108864, "cache-miss-rate": -nan, "pages": 0, "overflow": 0 > (right here) ------^ > > virNetClientStreamRaiseError:191 : stream aborted at client request Wow - I've known we have a problem with qemu emitting non-compliant JSON, but this proves that it is fatal to libvirt. I guess my series on improving the JSON parser [1] should consider doing a fallback to s/NaN/0/ and s/Inf/DBL_MAX/ rather than completely erroring out when a client tries to request it. Meanwhile, it's an easy patch to qemu to avoid division by zero when generating cache-miss-rate. [1] https://lists.gnu.org/archive/html/qemu-devel/2016-05/msg03424.html > > > Would it be possible to improve the JSON parser to skip the key if the value is incorrect Libvirt uses libyajl to parse JSON, and libyajl has an outstanding bug request to support extensions to JSON such as parsing non-finite floats. Since there has been no upstream reaction to the bug request, I seriously doubt it will happen any time soon, so any change to tolerate NaN in libvirt would have to be a one-off patch. It sounds like the better fix is to make qemu emit valid JSON in the first place, rather than making libvirt deal with broken JSON from qemu. -- Eric Blake eblake redhat com +1-919-301-3266 Libvirt virtualization library http://libvirt.org Ok, then I'll reopen the bug with a request to fix JSON output in qemu. Also found a patch in qemu 2.4.0 (?) which fix a zero division: https://lists.gnu.org/archive/html/qemu-devel/2015-05/msg01173.html which might be the source of the problem. Marc, what distro are you running on? QEMU 2.3 is not maintained anymore upstream, so unless you are running on Ubuntu (and then we can reuse this bug tracker) you'll have to reopen the bug in your distro. We're using Ubuntu, and we manually patched the version 2.3 with the fix referenced. It will be soon deployed and I'll see if that fixes the problem. We tried to migrate a VM, and now we have a new error: error : virNetClientProgramDispatchError:177 : internal error: info migration reply was missing return status :( I didn't look properly, it's the same error with the patch applied. Well, to make this work we updated the libvirt code to correct such invalid value and put a "0" in that case for the "cache-miss-rate" value. Can someone confirm that it putting a "0" is a valid choice, or shall we have something else? Thanks Is there still something to be done for upstream QEMU here? ... otherwise, I assume we can close this bug now? I'm not able to test that issue anymore, you can close the ticket. [Expired for QEMU because there has been no activity for 60 days.]