id = 1307
title = "query-named-block-nodes, without flat=true, is massively slow as number of block nodes increases"
state = "opened"
created_at = "2022-11-09T10:40:24.888Z"
closed_at = "n/a"
labels = ["Storage", "kind::Bug"]
url = "https://gitlab.com/qemu-project/qemu/-/issues/1307"
host-os = "Fedora 36"
host-arch = "x86_64"
qemu-version = "v7.2.0-rc0"
guest-os = "n/a"
guest-arch = "n/a"
description = """The query-named-block-nodes command is insanely slow with deep backing chains when the flat=true arg is NOT given.

```
qemu-img create demo0.qcow2 1g
j=0
for i in `seq 1 199`
do 
    qemu-img create -f qcow2 -o backing_file=demo$j.qcow2 -o backing_fmt=qcow2 demo$i.qcow2
    j=$i
done
```

Now configure libvirt with

```
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' discard='unmap'/>
      <source file='/var/lib/libvirt/images/demo199.qcow2'/>
      <target dev='vdb' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
    </disk>
```

This results in `-blockdev` args

```
-blockdev '{"driver":"file","filename":"/var/lib/libvirt/images/demo0.qcow2","node-name":"libvirt-201-storage","auto-read-only":true,"discard":"unmap"}' \\
-blockdev '{"node-name":"libvirt-201-format","read-only":true,"discard":"unmap","driver":"qcow2","file":"libvirt-201-storage","backing":null}' \\
-blockdev '{"driver":"file","filename":"/var/lib/libvirt/images/demo1.qcow2","node-name":"libvirt-200-storage","auto-read-only":true,"discard":"unmap"}' \\
-blockdev '{"node-name":"libvirt-200-format","read-only":true,"discard":"unmap","driver":"qcow2","file":"libvirt-200-storage","backing":"libvirt-201-format"}' \\
-blockdev '{"driver":"file","filename":"/var/lib/libvirt/images/demo2.qcow2","node-name":"libvirt-199-storage","auto-read-only":true,"discard":"unmap"}' \\
-blockdev '{"node-name":"libvirt-199-format","read-only":true,"discard":"unmap","driver":"qcow2","file":"libvirt-199-storage","backing":"libvirt-200-format"}' \\
...snip...
-blockdev '{"driver":"file","filename":"/var/lib/libvirt/images/demo197.qcow2","node-name":"libvirt-4-storage","auto-read-only":true,"discard":"unmap"}' \\
-blockdev '{"node-name":"libvirt-4-format","read-only":true,"discard":"unmap","driver":"qcow2","file":"libvirt-4-storage","backing":"libvirt-5-format"}' \\
-blockdev '{"driver":"file","filename":"/var/lib/libvirt/images/demo198.qcow2","node-name":"libvirt-3-storage","auto-read-only":true,"discard":"unmap"}' \\
-blockdev '{"node-name":"libvirt-3-format","read-only":true,"discard":"unmap","driver":"qcow2","file":"libvirt-3-storage","backing":"libvirt-4-format"}' \\
-blockdev '{"driver":"file","filename":"/var/lib/libvirt/images/demo199.qcow2","node-name":"libvirt-1-storage","auto-read-only":true,"discard":"unmap"}' \\
-blockdev '{"node-name":"libvirt-1-format","read-only":false,"discard":"unmap","driver":"qcow2","file":"libvirt-1-storage","backing":"libvirt-3-format"}' \\
-device '{"driver":"virtio-blk-pci","bus":"pci.7","addr":"0x0","drive":"libvirt-1-format","id":"virtio-disk1"}' \\
```

Now stop libvirt

```
systemctl stop libvirtd
```

And speak directly to QMP

```
$ time socat UNIX:/var/lib/libvirt/qemu/domain-158-fedora38/monitor.sock - > /dev/null 
{ "execute": "qmp_capabilities", "arguments": { "enable": ["oob"] } }
{ "execute": "query-named-block-nodes"}
{ "execute": "quit" }

real\t2m19.276s
user\t0m0.006s
sys\t0m0.014s
```

If we save the 'query-named-block-nodes' output instead of sending it to /dev/null, we get a 86 MB file for the QMP response. This will break all known client apps since they limit QMP reply size.

It appears to have a combinatorial expansion of block nodes in the output.

Blocking the main event loop for 2 minutes is obviously not good either.

If we use '"flat": true' parameter to query-named-block-nodes, the command completes in just 15 seconds, and produces a large, but more manageable 2.7 MB

Since the non-flat  query-named-block-nodes output is so incredibly non-scalable, I think we should deprecate non-flat mode, and eventually make  flat the mandatory option."""
reproduce = "n/a"
additional = "n/a"