VNC outputs garbage in zlib mode

TL;DR: When QEMU is launched with VNC as the output and viewed with a client that defaults to zlib VNC encoding, the resulting output tends to accumulate artifacts.

Reproduction:
Launch QEMU (tried with versions 4.2.0 and 4.1.0 on Linux  64bit) with -vnc 0.0.0.0:0
Connect to it with a VNC client that allows you to select encoding, i.e. UltraVNC.
Set encoding to zlib (type 6), 32bit color.
As screen content changes it starts accumulating artifacts. Almost certain to appear if you open-close windows over a pattern.
Does not seem to depend on guest used, but easier to reproduce with a GUI.

Looks like this: https://orbides.org/img/vnc.png

It appears to be a deflate glitch of some sort - all of the bad pixels are generated by length/distance codes. Can't narrow it down any more.