Replay/record does not work with `rrsnapshot`/`loadvm` Description of problem: Qemu's record/replay feature does not properly work when using snapshots (like rrsnapshot). Record/replay without snapshotting works just fine, but when using `rrsnapshot=...` the replay is stuck at boot. `loadvm` monitor command also gets qemu stuck. Record command: ``` $ qemu-system-x86_64 \ -cpu SandyBridge -smp 1 \ -serial stdio -display none \ -m 4096 \ -drive file=./empty.qcow2,id=rr \ -kernel ./boot/vmlinuz-lts \ -initrd ./boot/initramfs-lts . -monitor telnet::12345,server,nowait \ -append "console=ttyS0 root=/dev/ram0 alpine_dev=cdrom:iso9660 modules=loop,squashfs,sd-mod,usb-storage quiet" \ -icount shift=auto,rrfile=rr,rr=record,rrsnapshot=init ``` Broken replay command, which gets qemu stuck: ``` $ qemu-system-x86_64 \ -cpu SandyBridge -smp 1 \ -serial stdio -display none \ -m 4096 \ -drive file=./empty.qcow2,id=rr \ -kernel ./boot/vmlinuz-lts \ -initrd ./boot/initramfs-lts . -monitor telnet::12345,server,nowait \ -append "console=ttyS0 root=/dev/ram0 alpine_dev=cdrom:iso9660 modules=loop,squashfs,sd-mod,usb-storage quiet" \ -icount shift=auto,rrfile=rr,rr=replay,rrsnapshot=init qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.01H:ECX.tsc-deadline [bit 24] ``` Record/replay without `rrsnapshot`/`loadvm`/etc works as expected. Steps to reproduce: To reproduce i've used alpine linux kernel as the guest: ``` wget https://dl-cdn.alpinelinux.org/alpine/v3.20/releases/x86_64/alpine-standard-3.20.3-x86_64.iso 7z x alpine-standard-3.20.3-x86_64.iso ``` Prerequisites - an empty qcow2 file for snapshots: ``` qemu-img create -f qcow2 empty.qcow2 1G ``` Running an alpine linux kernel with `rr=record` - works just fine, kernel boots, accepts input. ``` $ qemu-system-x86_64 \ -cpu SandyBridge -smp 1 \ -serial stdio -display none \ -m 4096 \ -drive file=./empty.qcow2,id=rr \ -kernel ./boot/vmlinuz-lts \ -initrd ./boot/initramfs-lts . -monitor telnet::12345,server,nowait \ -append "console=ttyS0 root=/dev/ram0 alpine_dev=cdrom:iso9660 modules=loop,squashfs,sd-mod,usb-storage quiet" \ -icount shift=auto,rrfile=rr,rr=record,rrsnapshot=init qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.01H:ECX.tsc-deadline [bit 24] mount: mounting /dev/ram0 on /sysroot failed: Invalid argument Mounting root failed. initramfs emergency recovery shell launched. Type 'exit' to continue boot sh: can't access tty; job control turned off ~ # ls -alh total 32K drwx------ 18 root root 0 Oct 21 13:02 . drwx------ 18 root root 0 Oct 21 13:02 .. -rw------- 1 root root 8 Oct 21 13:02 .ash_history drwxr-xr-x 2 root root 0 Jun 18 12:44 .modloop drwxr-xr-x 2 root root 0 Oct 21 13:02 bin drwxr-xr-x 9 root root 2.5K Oct 21 13:02 dev drwxr-xr-x 4 root root 0 Oct 21 13:02 etc -rwxr-xr-x 1 root root 25.9K Jun 18 12:44 init drwxr-xr-x 5 root root 0 Jun 18 12:44 lib drwxr-xr-x 5 root root 0 Jun 18 12:44 media drwxr-xr-x 2 root root 0 Jun 18 12:44 newroot dr-xr-xr-x 114 root root 0 Oct 21 13:02 proc drwx------ 2 root root 0 Sep 4 12:53 root drwxr-xr-x 3 root root 0 Oct 21 13:02 run drwxr-xr-x 2 root root 0 Oct 21 13:02 sbin dr-xr-xr-x 13 root root 0 Oct 21 13:02 sys drwxr-xr-x 2 root root 0 Oct 21 13:02 sysroot drwxr-xr-x 2 root root 0 Oct 21 13:02 tmp drwxr-xr-x 5 root root 0 Oct 21 13:02 usr drwxr-xr-x 3 root root 0 Jun 18 12:44 var ~ # echo "AAAAAAAA?" AAAAAAAA? ~ # ``` `rr`-file is produced, which can be used for replaying **without** `rrsnapshot`-option: ``` $ qemu-system-x86_64 \ -cpu SandyBridge -smp 1 \ -serial stdio -display none \ -m 4096 \ -drive file=./empty.qcow2,id=rr \ -kernel ./boot/vmlinuz-lts \ -initrd ./boot/initramfs-lts . -monitor telnet::12345,server,nowait \ -append "console=ttyS0 root=/dev/ram0 alpine_dev=cdrom:iso9660 modules=loop,squashfs,sd-mod,usb-storage quiet" \ -icount shift=auto,rrfile=rr,rr=replay qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.01H:ECX.tsc-deadline [bit 24] mount: mounting /dev/ram0 on /sysroot failed: Invalid argument Mounting root failed. initramfs emergency recovery shell launched. Type 'exit' to continue boot sh: can't access tty; job control turned off ~ # ls -alh total 32K drwx------ 18 root root 0 Oct 21 13:02 . drwx------ 18 root root 0 Oct 21 13:02 .. -rw------- 1 root root 8 Oct 21 13:02 .ash_history drwxr-xr-x 2 root root 0 Jun 18 12:44 .modloop drwxr-xr-x 2 root root 0 Oct 21 13:02 bin drwxr-xr-x 9 root root 2.5K Oct 21 13:02 dev drwxr-xr-x 4 root root 0 Oct 21 13:02 etc -rwxr-xr-x 1 root root 25.9K Jun 18 12:44 init drwxr-xr-x 5 root root 0 Jun 18 12:44 lib drwxr-xr-x 5 root root 0 Jun 18 12:44 media drwxr-xr-x 2 root root 0 Jun 18 12:44 newroot dr-xr-xr-x 114 root root 0 Oct 21 13:02 proc drwx------ 2 root root 0 Sep 4 12:53 root drwxr-xr-x 3 root root 0 Oct 21 13:02 run drwxr-xr-x 2 root root 0 Oct 21 13:02 sbin dr-xr-xr-x 13 root root 0 Oct 21 13:02 sys drwxr-xr-x 2 root root 0 Oct 21 13:02 sysroot drwxr-xr-x 2 root root 0 Oct 21 13:02 tmp drwxr-xr-x 5 root root 0 Oct 21 13:02 usr drwxr-xr-x 3 root root 0 Jun 18 12:44 var ~ # echo "AAAAAAAA?" AAAAAAAA? ~ # ``` As you can see, replaying emulation session works as expected. How ever, if I add the `rrsnapshot`-option, it gets stuck: ``` $ qemu-system-x86_64 \ -cpu SandyBridge -smp 1 \ -serial stdio -display none \ -m 4096 \ -drive file=./empty.qcow2,id=rr \ -kernel ./boot/vmlinuz-lts \ -initrd ./boot/initramfs-lts . -monitor telnet::12345,server,nowait \ -append "console=ttyS0 root=/dev/ram0 alpine_dev=cdrom:iso9660 modules=loop,squashfs,sd-mod,usb-storage quiet" \ -icount shift=auto,rrfile=rr,rr=replay,rrsnapshot=init qemu-system-x86_64: warning: TCG doesn't support requested feature: CPUID.01H:ECX.tsc-deadline [bit 24] ``` This also can be reproduced without `rrsnapshot` option, by issuing `loadvm init` from qemu monitor: ``` $ telnet localhost 12345 qemu> loadvm init ... ``` Or, by using `gdb` and issuing reverse-commands that require `loadvm` to load previous state, like `reverse-stepi` or `reverse-continue`. Attaching a debugger & using debug-prints shows some thread being stuck in the [`rcu.c`](https://gitlab.com/qemu-project/qemu/-/blob/master/util/rcu.c), near the `qemu_event_wait(&rcu_call_ready_event);`. I've tried to wait for quite some time (about an hour) and there was no result. Additional information: **Qemu build.** Qemu binary built from sources of 9.1.0 with `--target-list=x86_64-softmmu`. **Host machine.** An almost clean Ubuntu 20.04 with necessary packages for building qemu from the latest release sources.