path: root/results/scraper/launchpad/1868116

QEMU monitor no longer works

It was observed that the QEMU console (normally accessible using Ctrl+Alt+2) accepts no input, so it can't be used. This is being problematic because there are cases where it's required to send commands to the guest, or key combinations that the host would grab (as Ctrl-Alt-F1 or Alt-F4).

ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: qemu 1:4.2-3ubuntu2
Uname: Linux 5.6.0-rc6+ x86_64
ApportVersion: 2.20.11-0ubuntu20
Architecture: amd64
CurrentDesktop: XFCE
Date: Thu Mar 19 12:16:31 2020
Dependencies:
 
InstallationDate: Installed on 2017-06-13 (1009 days ago)
InstallationMedia: Xubuntu 17.04 "Zesty Zapus" - Release amd64 (20170412)
KvmCmdLine:
 COMMAND         STAT  EUID  RUID     PID    PPID %CPU COMMAND
 qemu-system-x86 Sl+   1000  1000   34275   25235 29.2 qemu-system-x86_64 -m 4G -cpu Skylake-Client -device virtio-vga,virgl=true,xres=1280,yres=720 -accel kvm -device nec-usb-xhci -serial vc -serial stdio -hda /home/usuario/Sistemas/androidx86.img -display gtk,gl=on -device usb-audio
 kvm-nx-lpage-re S        0     0   34284       2  0.0 [kvm-nx-lpage-re]
 kvm-pit/34275   S        0     0   34286       2  0.0 [kvm-pit/34275]
MachineType: LENOVO 80UG
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-5.6.0-rc6+ root=UUID=6b4ae5c0-c78c-49a6-a1ba-029192618a7a ro quiet ro kvm.ignore_msrs=1 kvm.report_ignored_msrs=0 kvm.halt_poll_ns=0 kvm.halt_poll_ns_grow=0 i915.enable_gvt=1 i915.fastboot=1 cgroup_enable=memory swapaccount=1 zswap.enabled=1 zswap.zpool=z3fold resume=UUID=a82e38a0-8d20-49dd-9cbd-de7216b589fc log_buf_len=16M usbhid.quirks=0x0079:0x0006:0x100000 config_scsi_mq_default=y scsi_mod.use_blk_mq=1 mtrr_gran_size=64M mtrr_chunk_size=64M nbd.nbds_max=2 nbd.max_part=63
SourcePackage: qemu
UpgradeStatus: Upgraded to focal on 2019-12-22 (87 days ago)
dmi.bios.date: 08/09/2018
dmi.bios.vendor: LENOVO
dmi.bios.version: 0XCN45WW
dmi.board.asset.tag: NO Asset Tag
dmi.board.name: Toronto 4A2
dmi.board.vendor: LENOVO
dmi.board.version: SDK0J40679 WIN
dmi.chassis.asset.tag: NO Asset Tag
dmi.chassis.type: 10
dmi.chassis.vendor: LENOVO
dmi.chassis.version: Lenovo ideapad 310-14ISK
dmi.modalias: dmi:bvnLENOVO:bvr0XCN45WW:bd08/09/2018:svnLENOVO:pn80UG:pvrLenovoideapad310-14ISK:rvnLENOVO:rnToronto4A2:rvrSDK0J40679WIN:cvnLENOVO:ct10:cvrLenovoideapad310-14ISK:
dmi.product.family: IDEAPAD
dmi.product.name: 80UG
dmi.product.sku: LENOVO_MT_80UG_BU_idea_FM_Lenovo ideapad 310-14ISK
dmi.product.version: Lenovo ideapad 310-14ISK
dmi.sys.vendor: LENOVO
mtime.conffile..etc.apport.crashdb.conf: 2019-08-29T08:39:36.787240



Hi Leonardo,

I first ran something in libvirt as usual, and I got monitor access through libvirt as usual.
For example:
 $ virsh qemu-monitor-command focal3 --pretty '{"execute":"query-block"}'

Then I started a most basic qemu to get the usual UI.
 $ qemu-system-x86_64 -enable-kvm --drive media=cdrom,file=http://archive.ubuntu.com/ubuntu/dists/bionic/main/installer-amd64/current/images/netboot/mini.iso

I can switch to the monitor there
 default grab is CTRL+ALT
 monitor on SHIFT+2

I was only once able to switch to the monitor, and then I failed to enter something just as you describe it. Setting to confirmed, but this needs a deeper look.

ok CTL+ALT+2 (without shift) is working well to switch - also the GTK UI has a menu to switch which works as well. But on all of those tries no way to enter to the monitor.

This still worked in Eoan, I double checked and it indeed does.
So it is not the change to GTK which I assumed first to be relate.

To ease repro of the case, this can be seen even if "ssh -XY" into Eoan/Focal containers.

I retried with the latest qemu as I recently did some misc fixes in the form of a stable update.
But with that it fails as well.

I started to consider bisecting:

#1 check if v4.2.0 from upstream exposes the behavior
  => yes it does
#2 check if v4.0.0 from upstream does not expose the behavior
  => No, v4.0.0 is as broken as th v4.2.0 build

Hmm, this used the same build options and the same build environment.
Might it again be an issue with the BIOS or the libraries?


Copying the binaries into an Eoan environment makes them work.

Since they use all sorts of local resources e.g. bios file lets make sure none in the system are avaiable.

It is not again seabios (bug 1866870), but what is it ... ?
I removed all of the packaged qemu and dependencies.
Behavior still is the same.

I took a snapshot to later revert and retry as needed.

$ lxc info e
Name: e
Location: none
Remote: unix://
Architecture: x86_64
Created: 2020/03/20 08:52 UTC
Status: Stopped
Type: container
Profiles: default, kvm
Snapshots:
  e-with-working-qemu-monitor (taken at 2020/03/24 10:52 UTC) (stateless)

Then I did a full upgrade eoan to focal and ran the same binary that formerly worked.
Note: 484 packages upgraded

Now the input is no more working, therefore the next step is to break down those 484 packages to identify which dependency it was that changed the behavior.

Step 1: libc/libpython
The following additional packages will be installed:
  alsa-topology-conf alsa-ucm-conf cryptsetup cryptsetup-initramfs gcc-10-base libasound2 libasound2-data libc-bin libc6 libcanberra0 libcrypt1 libffi7 libgcc-s1 libltdl7 libpython3.8
  libpython3.8-minimal libpython3.8-stdlib libtdb1 libvorbis0a libvorbisenc2 libvorbisfile3 locales sound-theme-freedesktop vim-common vim-runtime vim-tiny
Suggested packages:
  keyutils libasound2-plugins alsa-utils glibc-doc libcanberra-gtk0 libcanberra-pulse ctags vim-doc vim-scripts indent
The following NEW packages will be installed:
  alsa-topology-conf alsa-ucm-conf gcc-10-base libasound2 libasound2-data libcanberra0 libcrypt1 libffi7 libgcc-s1 libltdl7 libpython3.8 libpython3.8-minimal libpython3.8-stdlib libtdb1
  libvorbisfile3 sound-theme-freedesktop
The following packages will be upgraded:
  cryptsetup cryptsetup-initramfs libc-bin libc6 libvorbis0a libvorbisenc2 locales vim vim-common vim-runtime vim-tiny

=> Still working

Step 2:  zlib1g xxd xkb-data xfsprogs uuid-runtime util-linux usbutils usb.ids update-notifier-common update-manager-core unattended-upgrades ufw ubuntu-mono ubuntu-minimal ubuntu-keyring ubuntu-advantage-tools ubuntu-release-upgrader-core tzdata tmux telnet tcpdump (and dependencies)

The following additional packages will be installed:
  apt apt-utils libapt-pkg6.0 libevent-2.1-7 libgirepository-1.0-1 libglib2.0-0 libglib2.0-bin libicu66 libnewt0.52 libpython3-stdlib libseccomp2 python3 python3-apt python3-cffi-backend
  python3-dbus python3-distupgrade python3-gi python3-markupsafe python3-minimal python3-netifaces python3-newt python3-simplejson python3-systemd python3-twisted-bin
  python3-update-manager python3-yaml python3-zope.interface python3.8 python3.8-minimal
Suggested packages:
  apt-doc aptitude | synaptic | wajig dpkg-dev python3-doc python3-tk python3-venv python3-apt-dbg python-apt-doc python-dbus-doc python3-dbus-dbg python3-twisted-bin-dbg python3.8-venv
  python3.8-doc binutils binfmt-support bsd-mailx default-mta | mail-transport-agent needrestart util-linux-locales xfsdump attr quota
The following NEW packages will be installed:
  libapt-pkg6.0 libevent-2.1-7 libicu66 python3.8 python3.8-minimal
The following packages will be upgraded:
  apt apt-utils libgirepository-1.0-1 libglib2.0-0 libglib2.0-bin libnewt0.52 libpython3-stdlib libseccomp2 python3 python3-apt python3-cffi-backend python3-dbus python3-distupgrade
  python3-gi python3-markupsafe python3-minimal python3-netifaces python3-newt python3-simplejson python3-systemd python3-twisted-bin python3-update-manager python3-yaml
  python3-zope.interface tcpdump telnet tmux tzdata ubuntu-advantage-tools ubuntu-keyring ubuntu-mono ubuntu-release-upgrader-core ufw unattended-upgrades update-manager-core
  update-notifier-common usb.ids usbutils util-linux uuid-runtime xfsprogs xkb-data xxd zlib1g
44 upgraded, 5 newly installed, 0 to remove and 381 not upgraded.

=> Still working

Step 3: systemd systemd-sysv sysvinit-utils udev sudo ssh-import-id snapd


The following additional packages will be installed:
  libnss-systemd libpam-systemd libsystemd0 libudev1
Suggested packages:
  zenity | kdialog systemd-container
The following packages will be upgraded:
  libnss-systemd libpam-systemd libsystemd0 libudev1 snapd ssh-import-id sudo systemd systemd-sysv sysvinit-utils udev
11 upgraded, 0 newly installed, 0 to remove and 370 not upgraded.

The following packages will be REMOVED:
  libevent-2.1-6 libpython3.7 libpython3.7-minimal libpython3.7-stdlib libsodium23 python3-nacl python3-pymacaroons python3.7 python3.7-minimal


=> Still working

Step 4: python*

The following additional packages will be installed:
  command-not-found python3-distutils python3-hamcrest python3-importlib-metadata python3-lib2to3 python3-more-itertools python3-pyrsistent python3-setuptools python3-zipp
  software-properties-common
Suggested packages:
  python-attr-doc python-blinker-doc python-configobj-doc python-cryptography-doc python3-cryptography-vectors python3-gdbm-dbg python-jinja2-doc python-jsonschema-doc python3-crypto
  gnome-keyring libkf5wallet-bin python3-keyrings.alt python3-testresources python-openssl-doc python3-openssl-dbg python3-socks python-secretstorage-doc python3-wxgtk3.0 | python3-wxgtk
  python-setuptools-doc python3-tk python3-gtk2 python3-glade2 python3-pampy python3-qt4 python3-wxgtk2.8
The following NEW packages will be installed:
  python3-distutils python3-hamcrest python3-importlib-metadata python3-lib2to3 python3-more-itertools python3-pyrsistent python3-setuptools python3-zipp
The following packages will be upgraded:
  command-not-found python-apt-common python3-apport python3-asn1crypto python3-attr python3-automat python3-blinker python3-certifi python3-chardet python3-click python3-colorama
  python3-commandnotfound python3-configobj python3-constantly python3-cryptography python3-debian python3-distro python3-distro-info python3-entrypoints python3-gdbm python3-httplib2
  python3-hyperlink python3-idna python3-incremental python3-jinja2 python3-json-pointer python3-jsonschema python3-jwt python3-keyring python3-launchpadlib python3-lazr.restfulclient
  python3-lazr.uri python3-oauthlib python3-openssl python3-pkg-resources python3-problem-report python3-pyasn1 python3-pyasn1-modules python3-requests python3-requests-unixsocket
  python3-secretstorage python3-serial python3-service-identity python3-six python3-software-properties python3-twisted python3-urllib3 python3-wadllib software-properties-common
49 upgraded, 8 newly installed, 0 to remove and 320 not upgraded.

=> Still working

Step 5: further non lib binaries

The following additional packages will be installed:
  bind9-dnsutils bind9-libs libacl1 libbz2-1.0 libcurl4 libdbus-1-3 libdconf1 libdns-export1109 libext2fs2 libfuse2 libfwupd2 libfwupdplugin1 libgcc1 libgusb2 libip4tc2 libip6tc2
  libisc-export1105 libklibc libkmod2 liblocale-gettext-perl liblz4-1 libmagic-mgc libmagic1 libmaxminddb0 libncurses6 libncursesw6 libnftnl11 libpackagekit-glib2-18 libparted2 libpci3
  libperl5.30 libplymouth5 libprocps8 libstdc++6 libtext-charwidth-perl libtext-iconv-perl libtinfo6 libtss2-esys0 libuv1 libxmlb1 libxtables12 pci.ids perl-modules-5.30 tpm-udev
Suggested packages:
  apparmor-profiles-extra apparmor-utils apport-gtk | apport-kde bash-doc duperemove ccze gnome-terminal | xterm po-debconf speedometer ttf-ubuntu-font-family wireless-tools bzip2-doc
  libarchive1 anacron checksecurity default-mta | mail-transport-agent pinentry-gnome3 tor gpart fuse2fs e2fsck-static mlocate | locate gawk-doc git-daemon-run | git-daemon-sysvinit
  git-doc git-el git-email git-gui gitk gitweb git-cvs git-mediawiki git-svn parcimonie xloadimage scdaemon groff iproute2-doc firewalld nftables traceroute resolvconf avahi-autoipd
  isc-dhcp-client-ddns isoquery mmdb-bin libparted-dev libparted-i18n www-browser nfs-common multipath-tools-boot hunspell network-manager | wpasupplicant keychain libpam-ssh monkeysphere
  ssh-askpass molly-guard haveged appstream parted-doc perl-doc libterm-readline-gnu-perl | libterm-readline-perl-perl make libb-debug-perl liblocale-codes-perl desktop-base
  plymouth-themes torsocks rsyslog-mysql | rsyslog-pgsql rsyslog-mongodb rsyslog-doc rsyslog-openssl | rsyslog-gnutls rsyslog-gssapi rsyslog-relp
Recommended packages:
  e2fsprogs-l10n
The following NEW packages will be installed:
  bind9-dnsutils bind9-libs libdns-export1109 libfwupdplugin1 libisc-export1105 libmaxminddb0 libperl5.30 libplymouth5 libprocps8 libtss2-esys0 libuv1 pci.ids perl-modules-5.30 tpm-udev
The following packages will be upgraded:
  acl adwaita-icon-theme apparmor apport at-spi2-core base-files base-passwd bash bash-completion bind9-host bsdutils btrfs-progs busybox-initramfs busybox-static byobu bzip2
  cloud-guest-utils cloud-init cloud-initramfs-copymods cloud-initramfs-dyn-netconf console-setup console-setup-linux cpio cron cryptsetup-bin cryptsetup-run curl dbus dbus-user-session
  dconf-gsettings-backend dconf-service debianutils dirmngr distro-info-data dmidecode dmsetup dnsutils e2fsprogs ed ethtool fdisk file findutils fuse fwupd fwupd-signed gawk gcc-9-base
  gdisk geoip-database gettext-base git git-man gnupg gnupg-l10n gnupg-utils gpg gpg-agent gpg-wks-client gpg-wks-server gpgconf gpgsm gpgv grep groff-base gzip hostname htop info
  initramfs-tools initramfs-tools-bin initramfs-tools-core install-info iproute2 iptables iputils-ping iputils-tracepath irqbalance isc-dhcp-client isc-dhcp-common iso-codes
  keyboard-configuration klibc-utils kmod kpartx krb5-locales landscape-common language-selector-common less libacl1 libbz2-1.0 libcurl4 libdbus-1-3 libdconf1 libext2fs2 libfuse2 libfwupd2
  libgcc1 libgusb2 libip4tc2 libip6tc2 libklibc libkmod2 liblocale-gettext-perl liblz4-1 libmagic-mgc libmagic1 libncurses6 libncursesw6 libnftnl11 libpackagekit-glib2-18 libparted2
  libpci3 libstdc++6 libtext-charwidth-perl libtext-iconv-perl libtinfo6 libxmlb1 libxtables12 linux-base login logsave lsb-base lsb-release lshw lsof lz4 man-db manpages mawk mime-support
  mount multipath-tools nano ncurses-base ncurses-bin ncurses-term netbase netcat-openbsd netplan.io openssh-client openssh-server openssh-sftp-server openssl overlayroot packagekit
  packagekit-tools parted passwd pciutils perl perl-base plymouth plymouth-theme-ubuntu-text pollinate popularity-contest procps psmisc publicsuffix rsyslog screen sensible-utils sg3-utils
  sg3-utils-udev sosreport whiptail

The following packages will be REMOVED:
  at-spi2-core gcc-9-base geoip-database libapt-pkg5.90 libbind9-161 libdns-export1104 libdns1104 libgeoip1 libiptc0 libirs161 libisc-export1100 libisc1100 libisccc161 libisccfg163
  liblwres161 libperl5.28 libplymouth4 libxtst6 perl-modules-5.28 python3-asn1crypto x11-common

=> Still working

Step 6: non x11/wayland/gtk/readline libs

The following additional packages will be installed:
  libcap2 libhogweed5 libnettle7
Suggested packages:
  lrzip colord cups-common rng-tools gnutls-bin krb5-doc krb5-user gstreamer1.0-tools libpam-doc pulseaudio librsvg2-bin libsasl2-modules-gssapi-mit | libsasl2-modules-gssapi-heimdal
  libsasl2-modules-ldap libsasl2-modules-otp libsasl2-modules-sql libsmbios-doc
Recommended packages:
  at-spi2-core libgpg-error-l10n update-motd
The following NEW packages will be installed:
  libhogweed5 libnettle7
The following packages will be upgraded:
  libapparmor1 libappstream4 libarchive13 libasn1-8-heimdal libassuan0 libatk-bridge2.0-0 libatk1.0-0 libatk1.0-data libatspi2.0-0 libattr1 libaudit-common libaudit1 libavahi-client3
  libavahi-common-data libavahi-common3 libblkid1 libbrotli1 libcairo-gobject2 libcairo2 libcap2 libcolord2 libcom-err2 libcryptsetup12 libcups2 libcurl3-gnutls libdb5.3 libdebconfclient0
  libdevmapper1.02.1 libelf1 libepoxy0 liberror-perl libexpat1 libfdisk1 libflac8 libfreetype6 libfribidi0 libgcab-1.0-0 libgcrypt20 libgdk-pixbuf2.0-0 libgdk-pixbuf2.0-bin
  libgdk-pixbuf2.0-common libglib2.0-data libgmp10 libgnutls30 libgpg-error0 libgpgme11 libgraphite2-3 libgssapi-krb5-2 libgssapi3-heimdal libgstreamer1.0-0 libharfbuzz0b
  libhcrypto4-heimdal libheimbase1-heimdal libheimntlm0-heimdal libhx509-5-heimdal libjson-c4 libjson-glib-1.0-0 libjson-glib-1.0-common libk5crypto3 libkeyutils1 libkrb5-26-heimdal
  libkrb5-3 libkrb5support0 libldap-2.4-2 libldap-common liblzo2-2 libmount1 libmpdec2 libnghttp2-14 libp11-kit0 libpam-cap libpam-modules libpam-modules-bin libpam-runtime libpam0g
  libpango-1.0-0 libpangocairo-1.0-0 libpangoft2-1.0-0 libpcre2-8-0 libpipeline1 libpmem1 libpng16-16 libpopt0 libproxy1v5 libpulse0 libroken18-heimdal librsvg2-2 librsvg2-common librtmp1
  libsasl2-2 libsasl2-modules libsasl2-modules-db libselinux1 libsemanage-common libsemanage1 libsepol1 libsgutils2-2 libsmartcols1 libsmbios-c2 libsndfile1 libsoup-gnome2.4-1 libsoup2.4-1
  libsqlite3-0 libss2 libssh-4 libssl1.1 libstemmer0d libtasn1-6 libtiff5 libusb-1.0-0 libutempter0 libuuid1 libvte-2.91-0 libvte-2.91-common libwayland-client0 libwayland-cursor0
  libwayland-egl1 libwind0-heimdal libwrap0 libxcb-render0 libxcb-shm0 libxcb1 libxkbcommon0 libxml2 libyaml-0-2 libzstd1
126 upgraded, 2 newly installed, 0 to remove and 19 not upgraded.

=> Broken - so one of the above was it

Next - break step 6 into sub-sections

$ apt install libvte-2.91-0
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  libgnutls30 libhogweed5 libnettle7 libp11-kit0
Suggested packages:
  gnutls-bin
The following NEW packages will be installed:
  libhogweed5 libnettle7
The following packages will be upgraded:
  libgnutls30 libp11-kit0 libvte-2.91-0
3 upgraded, 2 newly installed, 0 to remove and 141 not upgraded.
Need to get 1429 kB of archives.
After this operation, 736 kB of additional disk space will be used.


And the counter experiment made it work as well, taking a full focal install and then installing the libvte from eoan:
 $ apt install libvte-2.91-0=0.58.2-1ubuntu2

Version difference of libvte-2.91-0 is 0.58.2-1ubuntu2 <-> 0.59.91-0ubuntu2

Debian isn't frozen yet and has 0.60.0-2, but that is broken as well.

Building vte from git on the tags matching eoan and focal.

LD_LIBRARY_PATH=/usr/local/lib/x86_64-linux-gnu/:$LD_LIBRARY_PATH ldd ./build/x86_64-softmmu/qemu-system-x86_64 | grep vte
        libvte-2.91.so.0 => /usr/local/lib/x86_64-linux-gnu/libvte-2.91.so.0 (0x00007ff9b4908000)

Eoans version 0.58.2 => works
root@e:~/vte# ll /usr/local/lib/x86_64-linux-gnu/
lrwxrwxrwx 1 root root     16 Mar 24 13:09 libvte-2.91.so -> libvte-2.91.so.0*
lrwxrwxrwx 1 root root     23 Mar 24 13:09 libvte-2.91.so.0 -> libvte-2.91.so.0.5800.2*
-rwxr-xr-x 1 root root 645240 Mar 24 13:08 libvte-2.91.so.0.5800.2*

Focals version 0.59.91 => Fails
lrwxrwxrwx 1 root root     16 Mar 24 13:13 libvte-2.91.so -> libvte-2.91.so.0*
lrwxrwxrwx 1 root root     23 Mar 24 13:13 libvte-2.91.so.0 -> libvte-2.91.so.0.5991.0*
-rwxr-xr-x 1 root root 695088 Mar 24 13:13 libvte-2.91.so.0.5991.0*

So we should be able to bisect the offending change in vte ...?!?

Bisect build
$ ninja -C _build uninstall; rm -rf _build; meson _build && ninja -C _build && ninja -C _build install; echo $?; ll /usr/local/lib/x86_64-linux-gnu/

Test
$ LD_LIBRARY_PATH=/usr/local/lib/x86_64-linux-gnu/:$LD_LIBRARY_PATH ./build/x86_64-softmmu/qemu-system-x86_64 -enable-kvm --drive media=cdrom,file=http://archive.ubuntu.com/ubuntu/dists/bionic/main/installer-amd64/current/images/netboot/mini.iso

After a while of testing, build errors and a wrong bisect start I eventually got this:

git bisect log
# bad: [a444c23eca931cadad321eaf06bc9c437ed9cee3] Version 0.59.91
# good: [b7440da36396cbdbaca005adef3789d2a8852985] ring: Fix an incorrect assertion
git bisect start '0.59.91' '0.58.2'
# good: [def2f0ee9dfbed821b859c9d59f2bb66ae76dd27] Version 0.57.90
git bisect good def2f0ee9dfbed821b859c9d59f2bb66ae76dd27
# bad: [1bdfc0fa1adf1225152068040c8bde0804297805] widget: Move scrollable adjustments up to Widget
git bisect bad 1bdfc0fa1adf1225152068040c8bde0804297805
# good: [7f6e48ce00eb0fd6992966a66df6464e85cfda84] ring: Tiny code cleanup: maintain GString integrity
git bisect good 7f6e48ce00eb0fd6992966a66df6464e85cfda84
# good: [416c41be417a051b1d20499754aaa0a827e43b6b] test: Document the changes made to UTF-8-test.txt
git bisect good 416c41be417a051b1d20499754aaa0a827e43b6b
# good: [623f3c66de097eefef44001a4ff70600c11313f9] build: Add configure switch to disable a11y code
git bisect good 623f3c66de097eefef44001a4ff70600c11313f9
# good: [cdccfe59db102628857086bd3b89de77b6de1d73] introspection: Fix signedness in vte_terminal_feed_child's annotation
git bisect good cdccfe59db102628857086bd3b89de77b6de1d73
# skip: [9e4fbae2cabcd937ac4d1a984ba844d24b44b83f] lib: Use ICU for legacy charset support
git bisect skip 9e4fbae2cabcd937ac4d1a984ba844d24b44b83f
# skip: [7888602c3a980eee093313b2c0f949c756668070] lib: Rework child exit and EOF handling
git bisect skip 7888602c3a980eee093313b2c0f949c756668070
# bad: [55e5d53676960feb5dc11400ecdc7c9d7c4ab13e] lib: Add missing files
git bisect bad 55e5d53676960feb5dc11400ecdc7c9d7c4ab13e
# good: [83cbe9998aa1c2babbf32eed0b5fa3909360a83b] widget: Deprecate vte_terminal_feed_child_binary
git bisect good 83cbe9998aa1c2babbf32eed0b5fa3909360a83b
# only skipped commits left to test
# possible first bad commit: [55e5d53676960feb5dc11400ecdc7c9d7c4ab13e] lib: Add missing files
# possible first bad commit: [7888602c3a980eee093313b2c0f949c756668070] lib: Rework child exit and EOF handling
# possible first bad commit: [9e4fbae2cabcd937ac4d1a984ba844d24b44b83f] lib: Use ICU for legacy charset support

So one of those three that won't build individually should be our candidate, lets take a look.

commit 55e5d53676960feb5dc11400ecdc7c9d7c4ab13e (refs/bisect/bad)
Author: Christian Persch <email address hidden>
Date:   Sun Nov 17 22:15:38 2019 +0100

    lib: Add missing files

commit 7888602c3a980eee093313b2c0f949c756668070 (refs/bisect/skip-7888602c3a980eee093313b2c0f949c756668070)
Author: Christian Persch <email address hidden>
Date:   Sun Nov 17 21:58:09 2019 +0100

    lib: Rework child exit and EOF handling
    
    When the child process exits, we used to immediately unset the PTY,
    which causes us to miss data written by the child but not yet read
    by vte.
    
    Instead, only store the child exit status, and defer emitting the
    'child-exited' signal until after all the pending data has been read
    and processed.
    
    Similarly, rework how EOF is processed. Instead of immediately
    queuing the emission of the 'eof' signal, only take note of the EOF,
    and process it after all pending data has processed. There also was
    a bug in that we took the first occurence of G_IO_HUP in
    Terminal::pty_io_read() to stop reading more data. Instead, only
    take a pure G_IO_HUP without G_IO_IN as EOF, or if reading data
    from the PTY returns the EIO error.
    
    This also fixes the bug where a(ny) partial character(s) not yet fully
    decoded by the UTF-8 and ICU decoder would not show in the output; this
    now correctly flushes the decoder, which inserts either a replacement
    character (for the UTF-8 decoder) or the character(s) in the ICU decoder
    internal state (most likely also a replacement character).
    
    https://bugzilla.gnome.org/show_bug.cgi?id=777686


commit 9e4fbae2cabcd937ac4d1a984ba844d24b44b83f (refs/bisect/skip-9e4fbae2cabcd937ac4d1a984ba844d24b44b83f)
Author: Christian Persch <email address hidden>
Date:   Sun Nov 17 21:58:09 2019 +0100

    lib: Use ICU for legacy charset support
    
    Instead of converting the whole chunk of input from the input
    charset to UTF-8 in one go, we need a decoder that consumes the
    input one byte at a time. Since the iconv API is not particularly
    suited to this (or, really, any) task, switch to using ICU for this.
    
    Add functions to get the list of supported legacy charsets, and
    to check whether a particular string is a supported charset.
    
    Fixes https://gitlab.gnome.org/GNOME/vte/issues/40

commit 83cbe9998aa1c2babbf32eed0b5fa3909360a83b (HEAD, refs/bisect/good-83cbe9998aa1c2babbf32eed0b5fa3909360a83b)
Author: Christian Persch <email address hidden>
Date:   Sun Nov 17 21:58:09 2019 +0100

    widget: Deprecate vte_terminal_feed_child_binary


Lets turn the order around a bit:
83cbe9998aa1c2babbf32eed0b5fa3909360a83b (known good)
 + cherry pick 55e5d53676960feb5dc11400ecdc7c9d7c4ab13e lib: Add missing files => Good
 + cherry pick 9e4fbae2cabcd937ac4d1a984ba844d24b44b83f Use ICU for legacy => Bad

83cbe9998aa1c2babbf32eed0b5fa3909360a83b (known good)
 + cherry pick 55e5d53676960feb5dc11400ecdc7c9d7c4ab13e lib: Add missing files => Good
 + cherry pick 7888602c3a980eee093313b2c0f949c756668070 Doesn't apply without 9e4fbae2

So it is this change that breaks things:
commit 9e4fbae2cabcd937ac4d1a984ba844d24b44b83f (refs/bisect/skip-9e4fbae2cabcd937ac4d1a984ba844d24b44b83f)
Author: Christian Persch <email address hidden>
Date:   Sun Nov 17 21:58:09 2019 +0100

    lib: Use ICU for legacy charset support


Last commit mentioning VTE is a while ago:
6415994 Thu Oct 11 17:30:39 2018 +0200 gtk: Don't vte_terminal_set_encoding() on new VTE versions

I built head of qemu against head of vte - to check if I even need to look for existing fixes.
=> That still fails, so it is probably time for a bug report to get other people to think with us.



FYI: Since this affects qemu (and VTE) git head I added an upstream-qemu task to the bug.

Thanks for this investigation so far!

We've opened an upstream VTE issue at https://gitlab.gnome.org/GNOME/vte/issues/222 .

We'd appreciate if QEMU developers joined us there. Apparently QEMU uses the "commit" signal in a way that it was not meant to be used, and thus it's unclear what the best solution would be.

Thank you Egmont for the bug for VTE in the gnome tracker!

Graphics isn't something I'm usually at home - the related qemu code is mostly in ui/gtk.c per Maintainers file Gerd Hoffmann is the expert. I subscribed him to the bug here to raise visibility for him.

For a bit of reverse-confirmation of the findings so far.
If I build qemu without VTE, like (configure)
GTK support       yes (3.24.14)
VTE support       no

It works, due to the fallback implemented by [1][2].
But obviously without all the VTE features, I'd prefer a more fine grained fix than disabling VTE :-)

[1]: https://git.qemu.org/?p=qemu.git;a=commit;h=f8c223f69ac58488ea830597281b7ddd33037c4c
[2]: https://git.qemu.org/?p=qemu.git;a=commit;h=bbbf9bfb9c27e389340cf50a11c22fa46c572150

I'm not really a UI guy, so I was checking what I might have lost by disabling VTE and found the very old [1]. That list of features really seems to make disabling VTE not an real option:
  "It's also screen reader accessible, supports copy/paste, proper scrolling and
   most of the other features you would expect from a terminal widget."

After seeing that Cole authored the "drop PTY" [3] patch I have subscribed him here as well.

I have tried to answer and ask a few questions on the VTE issue [2] to get it make progress, but it would really benefit getting the attention of Gerhard and Cole (or anyone else who feels the UI-power).

[1]: https://git.qemu.org/?p=qemu.git;a=commit;h=d861def367b516055dc4c46dc1305143ee653c84
[2]: https://gitlab.gnome.org/GNOME/vte/issues/222
[3]: https://git.qemu.org/?p=qemu.git;a=commit;h=d4370741402a97b8b6d0c38fef18ab38bf25ab22

same seems to happen on Fedora 32.

I'm not sure how many of you are tracking the Vte bug [1] so here a summary of the latest insight from there.

- Short term it seems that new behavior will be reverted in Vte 0.60.1.
- Long term the Vte devs might want to deprecate no-pty use cases or at least better understand why apps use it that way.

For more details please read [1].

[1]: https://gitlab.gnome.org/GNOME/vte/issues/222

Thank you for investigating this. I would bisect QEMU, but wouldn't investigate its libraries. Consequently, I would never find the cause of this problem.

For now, I am using -monitor telnet:127.0.0.1:55555,server,nowait to have access to the monitor on QEMU guests.

From IRC:
[16:10] <seb128> cpaelzer, @vte, we should get 0.60.1 for focal, 0.59.91 is a rc1 for 0.60, we are lacking behind merging the stable version from Debian but it's on our backlog (kenvandine was look at that one), the .1 is part of GNOME 3.36.1 which we plan to get before release (I would understand if you would like to backport a patch to help testing rather than waiting though)

From VTE Bug:
The standard Ubuntu freeze doesn't apply to GNOME packages. Usually Ubuntu aims to ship latest GNOME x.1. VTE is part of GNOME, VTE 0.60.0 is part of GNOME 3.36.0, VTE 0.60.1 belongs to GNOME 3.36.1 etc. Accordingly, 0.60.0 -> 0.60.1 contains important bugfixes only, no new features. In this particular case, 0.60.1 will bring a trivial shell script fix (quite important for non-VTE users), and hopefully this one. It would be outright ridiculous for an LTS distro to ship an unstable VTE. So, the only reasonable thing for Ubuntu 20.04 is to ship VTE 0.60.1. Anyway, this is not the right place to discuss it.

But gladly there now is a commit with a fix:
https://gitlab.gnome.org/GNOME/vte/-/commit/277ee003066b3993cf6d55a05606009caac69015

I agree that we need this for 20.04, and therefore will set this up in prio and assign it to the Desktop team.

Subscribed and Assigned to Ubuntu Desktop to get to 0.60.1 before Focal releases.
I'd be happy about an update here that this surely is on your todo list.

As Vte-upstream long term would want to get rid of this implementation style Christian Persch provided a qemu patch [1]. That is too much UI for me to really have an in-depth opinion, but I can say that it builds and input works fine with it.

I suggested on [2] to send it to qemu-devel, but in case that doesn't happen it might be great if Gerd Hoffmann and Cole Robinson could take a look at it.

[1]: https://gitlab.gnome.org/GNOME/vte/uploads/1e8ccb6aaf2e8fcef91dd67d23f47fae/qemu.patch
[2]: https://gitlab.gnome.org/GNOME/vte/issues/222

THe update should be part of GNOME 3.36.1 which is due this weekend

This bug was fixed in the package vte2.91 - 0.60.0-2ubuntu2

---------------
vte2.91 (0.60.0-2ubuntu2) focal; urgency=medium

  * debian/libvte-2.91-0.install
    - Dropped files duplicated in libvte-2.91-common
  * debian/control.in
    - Add appropriate Breaks/Replaces for moved files.

 -- Ken VanDine <email address hidden>  Fri, 27 Mar 2020 16:07:28 -0400

Thanks Ken!
I verified it and the new version indeed fixes the issue in focal.

Looking at the comments, it seems to me that this was an issue in VTE that got fixed. Is there still anything left to do for upstream QEMU here?

@Thomas - there is a leftover task here and I've filed [1] for it in the new tracker.
What is the right state to move this bug here into now?

[1]: https://gitlab.com/qemu-project/qemu/-/issues/137

Thanks for opening the new ticket. Let's close this one here as WontFix - since we will only look at the new issue now instead.