diff options
Diffstat (limited to 'results/classifier/zero-shot/105/assembly')
36 files changed, 4238 insertions, 0 deletions
diff --git a/results/classifier/zero-shot/105/assembly/1098729 b/results/classifier/zero-shot/105/assembly/1098729 new file mode 100644 index 000000000..13a8a87d5 --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/1098729 @@ -0,0 +1,177 @@ +assembly: 0.688 +device: 0.653 +boot: 0.646 +vnc: 0.643 +semantic: 0.636 +instruction: 0.636 +graphic: 0.631 +other: 0.591 +KVM: 0.563 +socket: 0.558 +network: 0.446 +mistranslation: 0.302 + +qemu-user-static for armhf: segfault in threaded code + + +Currently running QEMU from git (fedf2de31023) and running the armhf version of qemu-user-static which I have renamed qemu-armhf-static to follow the naming convention used in Debian. + +The host systems is a Debian testing x86_64-linux and I have an Debian testing armhf chroot which I invoke using schroot. + +Majority of program in the armhf chroot run fine, but I'm getting qemu segfaults in multi-threaded programs. + +As an example, I've grabbed the threads demo program here: + + https://computing.llnl.gov/tutorials/pthreads/samples/dotprod_mutex.c + +and changed NUMTHRDS from 4 to 10. I compile it as (same compile command on both x86_64 host and armhf guest): + + gcc -Wall -lpthread dotprod_mutex.c -o dotprod_mutex + +When compiled for x86_64 host it runs perfectly and even under Valgrind displays no errors whatsoever. + +However, when I compile the program in my armhs chroot and run it it usually (but not always) segaults or hangs or crashes. Example output: + + + (armhf) $ ./dotprod_mutex + Thread 1 did 100000 to 200000: mysum=100000.000000 global sum=100000.000000 + Thread 0 did 0 to 100000: mysum=100000.000000 global sum=200000.000000 + TCG temporary leak before f6731ca0 + qemu-arm-static: /home/erikd/Git/qemu-posix-timer-hacking/Upstream/tcg/tcg-op.h:2371: + tcg_gen_goto_tb: Assertion `(tcg_ctx.goto_tb_issue_mask & (1 << idx)) == 0' failed. + + + (armhf) $ ./dotprod_mutex + qemu: uncaught target signal 11 (Segmentation fault) - core dumped + Segmentation fault + + (armhf) $ ./dotprod_mutex + qemu-arm-static: /home/erikd/Git/qemu-posix-timer-hacking/Upstream/tcg/tcg.c:519: + tcg_temp_free_internal: Assertion `idx >= s->nb_globals && idx < s->nb_temps' failed. + + + (armhf) $ ./dotprod_mutex + Thread 1 did 100000 to 200000: mysum=100000.000000 global sum=100000.000000 + qemu: uncaught target signal 11 (Segmentation fault) - core dumped + Segmentation fault + +I can also comple a purely static version of the test program in the armhf chroot using: + + gcc -Wall -static -pthread dotprod_mutex.c -o dotprod-mutex-static + +and then run it simply using: + + qemu-arm-static dotprod-mutex-static + +which fails just like it does in the chroot. + +Begining to think this is memory corruption because of the number of different failure modes. In addition to the crashes in the initial report I have also seen the following: + + + qemu: uncaught target signal 4 (Illegal instruction) - core dumped + + More temporaries freed than allocated! + TCG temporary leak before 0001d1dc + + qemu-arm-static: /home/erikd/Git/qemu-pthread-hacking/tcg/tcg.c:1888: tcg_reg_alloc_op: + Assertion `ts->val_type == 1' failed. + + /home/erikd/Git/qemu-pthread-hacking/tcg/tcg.c:149: tcg fatal error + + +What's the best way to debug the qemu user space emulation? I read this: + + http://wiki.qemu.org/Documentation/Debugging + +but that seems to mainly refer to the qemu machine emulation. + +I added -ggdb to QEMU_CFLAGS in config-host.mak so it builds with debug symbols but gdb still doesn't provide any useful information beyond the following: + + Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". + [New Thread 0x7ffefdb6b700 (LWP 11210)] + [New Thread 0x7ffefdaf5700 (LWP 11211)] + [New Thread 0x7ffefda7f700 (LWP 11212)] + [New Thread 0x7ffefda09700 (LWP 11213)] + [New Thread 0x7ffefd993700 (LWP 11214)] + + Program received signal SIGSEGV, Segmentation fault. + [Switching to Thread 0x7ffefdaf5700 (LWP 11211)] + 0x0000000060363b58 in static_code_gen_buffer () + (gdb) bt + #0 0x0000000060363b58 in static_code_gen_buffer () + #1 0x00000000f50ba518 in ?? () + #2 0x00000000624a9360 in ?? () + #3 0x00007ffefdaf4b80 in ?? () + #4 0x326cebdf4a8e4700 in ?? () + #5 0x00007ffe00000000 in ?? () + #6 0x0000000000000000 in ?? () + +and valgrind doesn't help either. + +Yes, multithreaded guests are liable to crash; where they work they generally work more by luck than design. There is some discussion in LP:668799 of one of the known problems (whose symptoms are usually crashes or hangs). + + +At the top of function cpu_unlink_tb() in translate-all.c: + + /* FIXME: TB unchaining isn't SMP safe. For now just ignore the + problem and hope the cpu will stop of its own accord. For userspace + emulation this often isn't actually as bad as it sounds. Often + signals are used primarily to interrupt blocking syscalls. */ + + +The class of bugs exemplified by the symptoms described here are those where the multithreaded guest program causes QEMU to misbehave because we are sharing the code-translation globals (eg the generated code buffer) between multiple threads and they trod on each others' toes. + +(The race described in the comment in cpu_unlink_tb() has been fixed under LP:668799.) + +I also experimented the bug. +It may SIGSEGV or hang. Or it may work, very rarely. + +But I cannot reproduce it at all if change my app to stay on a single CPU: + +int +main(int argc, char * argv[] ) +{ + +#ifdef QEMU + cpu_set_t cpuSet; + CPU_ZERO(&cpuSet); + CPU_SET(0,&cpuSet); + if (sched_setaffinity(getpid(), sizeof(cpu_set_t), &cpuSet) !=0) + cerr << "sched_setaffinity failed" << endl; +#endif /* QEMU */ + +./build/buildd/qemu-linaro-1.5.0-2013.06/tcg/tcg.c:149: tcg fatal error +/build/buildd/qemu-linaro-1.5.0-2013.06/tcg/tcg.c:149: tcg fatal error + +same for me + +Same problem for me when executing msgmerge in qemu-arm-static. + +https://launchpadlibrarian.net/181070813/buildlog_ubuntu-utopic-armhf.hedgewars_0.9.21~alpha~7716~ubuntu14.10.1_FAILEDTOBUILD.txt.gz + +this started happening after the new deploy of trusty in buildds + +Also happening when manually built from the 2.1.2 release codebase. In my case it impacts the llvm-3.5.0 "make check" testsuite running an an armhf-emulated chroot--it immediately gets SIGSEGV and SIGILL as soon as it starts running tests. + +I cannot make dotprod_mutex.c to crash with the current master (git 8ffe756d). I've tried both linux-arm and linux-arm-static, the latter running under chroot. + +I've tried on three different machines, and have tested with different thread counts: 4, 10, 16, 64 (one of the machines has 64 cores). +I completed 1000 successive runs on each. + +Can you please retest on the current master? I certainly could trigger the bug on the qemu-arm-static that is packaged with Ubuntu 14.04, so it is possible that since then changes in qemu have at least made it harder to trigger the bug. + + + +I can confirm that building QEMU 2.5.0 from source, all the multi-thread issues seem to be fixed. + +Specifically, the mentioned dotprod_mutex.c example, even when modified to use 100 threads, is always running in the qemu-arm User mode emulator. + +Tested in Ubuntu 14.04 x86_64, with all the updates installed. + +Note that instead the QEMU 2.0.0 from the Ubuntu 14.04 repository is having issues even when using workarounds like running it with "taskset 0x1" to force the execution to a single CPU. + + + +We think we've fixed the multithreading issues in QEMU linux-user (in particular the test case that started this bug report works). If there are still problems with a QEMU version later than 2.10, please open fresh bug reports for specific guest programs that fail, giving detailed how-to-reproduce instructions. + + diff --git a/results/classifier/zero-shot/105/assembly/1396497 b/results/classifier/zero-shot/105/assembly/1396497 new file mode 100644 index 000000000..59c68c8f6 --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/1396497 @@ -0,0 +1,100 @@ +assembly: 0.467 +device: 0.444 +other: 0.422 +instruction: 0.377 +graphic: 0.272 +socket: 0.243 +boot: 0.232 +mistranslation: 0.218 +semantic: 0.196 +vnc: 0.196 +KVM: 0.181 +network: 0.116 + +'qemu-img snapshot' allows new snapshot to be created with the name of an existing snapshot + +qemu-img _may_ be working as designed, but it feels like this could be a bug. I'd certainly prefer to only allow unique snapshot names (unless maybe something like a "--force-non-unique-snapshot-names" was also specified). + +If this really is correct behaviour, it should be documented as qemu-img(1) currently specifies no details whatsoever regarding expected behaviour or valid snapshot names. + +$ qemu-img snapshot -l image.cow +$ qemu-img snapshot -c foo image.cow +$ qemu-img snapshot -l image.cow +Snapshot list: +ID TAG VM SIZE DATE VM CLOCK +1 foo 0 2014-11-26 08:30:53 00:00:00.000 +$ qemu-img snapshot -c foo image.cow +$ qemu-img snapshot -l image.cow +Snapshot list: +ID TAG VM SIZE DATE VM CLOCK +1 foo 0 2014-11-26 08:30:53 00:00:00.000 +2 foo 0 2014-11-26 08:30:58 00:00:00.000 +$ qemu-img snapshot -c foo image.cow +$ qemu-img snapshot -l image.cow +Snapshot list: +ID TAG VM SIZE DATE VM CLOCK +1 foo 0 2014-11-26 08:30:53 00:00:00.000 +2 foo 0 2014-11-26 08:30:58 00:00:00.000 +3 foo 0 2014-11-26 08:31:00 00:00:00.000 +$ qemu-img snapshot -d foo image.cow +$ qemu-img snapshot -l image.cow +Snapshot list: +ID TAG VM SIZE DATE VM CLOCK +2 foo 0 2014-11-26 08:30:58 00:00:00.000 +3 foo 0 2014-11-26 08:31:00 00:00:00.000 +$ qemu-img snapshot -d foo image.cow +$ qemu-img snapshot -l image.cow +Snapshot list: +ID TAG VM SIZE DATE VM CLOCK +3 foo 0 2014-11-26 08:31:00 00:00:00.000 +$ qemu-img snapshot -d foo image.cow +$ qemu-img snapshot -l image.cow +$ + +Note also how snapshot deletion works in reverse order - the oldest snapshot with a given name is deleted first. + +ProblemType: Bug +DistroRelease: Ubuntu 15.04 +Package: qemu-utils 2.1+dfsg-4ubuntu9 +ProcVersionSignature: Ubuntu 3.16.0-25.33-generic 3.16.7 +Uname: Linux 3.16.0-25-generic x86_64 +ApportVersion: 2.14.7-0ubuntu10 +Architecture: amd64 +CurrentDesktop: Unity +Date: Wed Nov 26 08:28:16 2014 +InstallationDate: Installed on 2014-04-11 (228 days ago) +InstallationMedia: Ubuntu 14.04 LTS "Trusty Tahr" - Daily amd64 (20140409) +KvmCmdLine: + COMMAND STAT EUID RUID PID PPID %CPU COMMAND + kvm-irqfd-clean S< 0 0 719 2 0.0 [kvm-irqfd-clean] +MachineType: LENOVO 20AQCTO1WW +ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-3.16.0-25-generic root=/dev/mapper/ubuntu--vg-root ro quiet splash vt.handoff=7 +SourcePackage: qemu +UpgradeStatus: Upgraded to vivid on 2014-05-08 (201 days ago) +dmi.bios.date: 02/10/2014 +dmi.bios.vendor: LENOVO +dmi.bios.version: GJET71WW (2.21 ) +dmi.board.asset.tag: Not Available +dmi.board.name: 20AQCTO1WW +dmi.board.vendor: LENOVO +dmi.board.version: 0B98405 STD +dmi.chassis.asset.tag: No Asset Information +dmi.chassis.type: 10 +dmi.chassis.vendor: LENOVO +dmi.chassis.version: Not Available +dmi.modalias: dmi:bvnLENOVO:bvrGJET71WW(2.21):bd02/10/2014:svnLENOVO:pn20AQCTO1WW:pvrThinkPadT440s:rvnLENOVO:rn20AQCTO1WW:rvr0B98405STD:cvnLENOVO:ct10:cvrNotAvailable: +dmi.product.name: 20AQCTO1WW +dmi.product.version: ThinkPad T440s +dmi.sys.vendor: LENOVO + + + +I'd agree that at least the last part - removing the oldest snapshot first - seems like a bug. + +The QEMU project is currently considering to move its bug tracking to another system. For this we need to know which bugs are still valid and which could be closed already. Thus we are setting older bugs to "Incomplete" now. +If you still think this bug report here is valid, then please switch the state back to "New" within the next 60 days, otherwise this report will be marked as "Expired". Or mark it as "Fix Released" if the problem has been solved with a newer version of QEMU already. Thank you and sorry for the inconvenience. + +[Expired for QEMU because there has been no activity for 60 days.] + +[Expired for qemu (Ubuntu) because there has been no activity for 60 days.] + diff --git a/results/classifier/zero-shot/105/assembly/1402755 b/results/classifier/zero-shot/105/assembly/1402755 new file mode 100644 index 000000000..6cd160e8e --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/1402755 @@ -0,0 +1,89 @@ +assembly: 0.901 +device: 0.883 +other: 0.879 +KVM: 0.874 +network: 0.873 +vnc: 0.871 +mistranslation: 0.870 +instruction: 0.867 +graphic: 0.854 +semantic: 0.846 +socket: 0.830 +boot: 0.803 + +qemu-kvm: e1000 RX ring is filled with partial-pkt of size 0 + +Hello, +We are using CentOS 6.5 with qemu-kvm-0.12.1.2-2.415 as a host of or VMs. +In the VM we use e1000 as the NIC emulation. +We've modified the e1000 driver to our needs. This modification start the RX engine while the RX ring is empty (RDH == RDT) +and at a later stage we fill the RX descriptors with buffers. This scheme works well on intel chips and VMware. +What we observe in this setup is that from time to time the RX ring is filled with "partial packets" of size 0 (meaning, DD bit is set, +No other status bits are set and packet size is also 0). + +Looking at the e1000 RX routine in qemu-kvm you can observe the following flow: +1. A packet is avail for receive: +2. The routine checks for RCTL_EN - it is enabled +3. The routine checks that the RDH equal RDT (they are as the ring is empty) but also checks if rxov is on (it is still off) so it doesn’t +Exit as it is supposed to. +4. The routine now updates the descriptor status with the DD bit (and vlan which we don’t care) +5. The routine checks if a buffer address is not NULL (it is as NULL since we haven’t filled it yet) – so is logs something. +6. The routine now updates the guest memory with this value (DD is 1) +7. The routine updates the check_rxov flag in order to allow ovf check the next time around. +(but ovf will not occur since in the next iteration RDH != RDT) +8. The routine loops over all the descriptors with the NULL buffer (which is all our ring) and writes the DD bit +9. We get this endless partial packet problem we see. + +qemu-kvm-0.12.1.2-2.415.el6_5.3/qemu-kvm-0.12.1.2/hw/e1000.c +static ssize_t +e1000_receive(VLANClientState *nc, const uint8_t *buf, size_t size) +{ +: : : +if (!(s->mac_reg[RCTL] & E1000_RCTL_EN)) +return -1; + +: : : +do { +if (s->mac_reg[RDH] == s->mac_reg[RDT] && s->check_rxov) { +set_ics(s, 0, E1000_ICS_RXO); +return -1; +} +base = ((uint64_t)s->mac_reg[RDBAH] << 32) + s->mac_reg[RDBAL] + +sizeof(desc) * s->mac_reg[RDH]; +cpu_physical_memory_read(base, (void *)&desc, sizeof(desc)); +desc.special = vlan_special; +desc.status |= (vlan_status | E1000_RXD_STAT_DD); +if (desc.buffer_addr) { +cpu_physical_memory_write(le64_to_cpu(desc.buffer_addr), +(void *)(buf + vlan_offset), size); +desc.length = cpu_to_le16(size); +desc.status |= E1000_RXD_STAT_EOP|E1000_RXD_STAT_IXSM; +} else // as per intel docs; skip descriptors with null buf addr +DBGOUT(RX, "Null RX descriptor!!\n"); +cpu_physical_memory_write(base, (void *)&desc, sizeof(desc)); + +: : : +if (++s->mac_reg[RDH] * sizeof(desc) >= s->mac_reg[RDLEN]) +s->mac_reg[RDH] = 0; +s->check_rxov = 1; +: : : +} while (desc.buffer_addr == 0); +} + + +A workaround is to enable the RX machine only after the descriptor ring is filled for the first time. + +Moti + +On Mon, Dec 15, 2014 at 04:59:55PM -0000, Moti wrote: +> We are using CentOS 6.5 with qemu-kvm-0.12.1.2-2.415 as a host of or VMs. + +Do you see the problem with qemu.git/master? + +Stefan + + +Looking through old bug tickets... can you still reproduce this issue with the latest version of QEMU? Or could we close this ticket nowadays? + +[Expired for QEMU because there has been no activity for 60 days.] + diff --git a/results/classifier/zero-shot/105/assembly/1490611 b/results/classifier/zero-shot/105/assembly/1490611 new file mode 100644 index 000000000..d29c92a6a --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/1490611 @@ -0,0 +1,356 @@ +assembly: 0.742 +other: 0.707 +graphic: 0.681 +device: 0.649 +network: 0.598 +instruction: 0.588 +semantic: 0.583 +vnc: 0.581 +KVM: 0.453 +socket: 0.434 +mistranslation: 0.412 +boot: 0.358 + +Using qemu >=2.2.1 to convert raw->VHD (fixed) adds extra padding to the result file, which Microsoft Azure rejects as invalid + +Starting with a raw disk image, using "qemu-img convert" to convert from raw to VHD results in the output VHD file's virtual size being aligned to the nearest 516096 bytes (16 heads x 63 sectors per head x 512 bytes per sector), instead of preserving the input file's size as the output VHD's virtual disk size. + +Microsoft Azure requires that disk images (VHDs) submitted for upload have virtual sizes aligned to a megabyte boundary. (Ex. 4096MB, 4097MB, 4098MB, etc. are OK, 4096.5MB is rejected with an error.) This is reflected in Microsoft's documentation: https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-linux-create-upload-vhd-generic/ + +This is reproducible with the following set of commands (including the Azure command line tools from https://github.com/Azure/azure-xplat-cli). For the following example, I used qemu version 2.2.1: + +$ dd if=/dev/zero of=source-disk.img bs=1M count=4096 + +$ stat source-disk.img + File: ‘source-disk.img’ + Size: 4294967296 Blocks: 798656 IO Block: 4096 regular file +Device: fc01h/64513d Inode: 13247963 Links: 1 +Access: (0644/-rw-r--r--) Uid: ( 1000/ smkent) Gid: ( 1000/ smkent) +Access: 2015-08-18 09:48:02.613988480 -0700 +Modify: 2015-08-18 09:48:02.825985646 -0700 +Change: 2015-08-18 09:48:02.825985646 -0700 + Birth: - + +$ qemu-img convert -f raw -o subformat=fixed -O vpc source-disk.img dest-disk.vhd + +$ stat dest-disk.vhd + File: ‘dest-disk.vhd’ + Size: 4296499712 Blocks: 535216 IO Block: 4096 regular file +Device: fc01h/64513d Inode: 13247964 Links: 1 +Access: (0644/-rw-r--r--) Uid: ( 1000/ smkent) Gid: ( 1000/ smkent) +Access: 2015-08-18 09:50:22.252077624 -0700 +Modify: 2015-08-18 09:49:24.424868868 -0700 +Change: 2015-08-18 09:49:24.424868868 -0700 + Birth: - + +$ azure vm image create testimage1 dest-disk.vhd -o linux -l "West US" +info: Executing command vm image create ++ Retrieving storage accounts +info: VHD size : 4097 MB +info: Uploading 4195800.5 KB +Requested:100.0% Completed:100.0% Running: 0 Time: 1m 0s Speed: 6744 KB/s +info: https://[redacted].blob.core.windows.net/vm-images/dest-disk.vhd was uploaded successfully +error: The VHD https://[redacted].blob.core.windows.net/vm-images/dest-disk.vhd has an unsupported virtual size of 4296499200 bytes. The size must be a whole number (in MBs). +info: Error information has been recorded to /home/smkent/.azure/azure.err +error: vm image create command failed + +I also ran the above commands using qemu 2.4.0, which resulted in the same error as the conversion behavior is the same. + +However, qemu 2.1.1 and earlier (including qemu 2.0.0 installed by Ubuntu 14.04) does not pad the virtual disk size during conversion. Using qemu-img convert from qemu versions <=2.1.1 results in a VHD that is exactly the size of the raw input file plus 512 bytes (for the VHD footer). Those qemu versions do not attempt to realign the disk. As a result, Azure accepts VHD files created using those versions of qemu-img convert for upload. + +Is there a reason why newer qemu realigns the converted VHD file? It would be useful if an option were added to disable this feature, as current versions of qemu cannot be used to create VHD files for Azure using Microsoft's official instructions. + +Which release contains this fix? + +Judging by their comments on bug 1399191, jan-wang1989 doesn't appear to be a QEMU developer. + +I bisected to this commit: +c70221df1f89953e85a3f1f96ceefbd6888bb55f + +Kevin Wolf, I added you since you were the author of the commit that I bisected this bug to. Can you advise at all on this bug? Thank you. + +In short: VHD is a mess and Microsoft isn't compatible with itself. That's the root cause of all the trouble we have with it. + +When the qemu VHD driver was written, it was designed to be compatible with Virtual PC, which used the disk geometry as the definite source for the image size. In order to achieve exactly the same results on qemu, we had to calculate the image size the same way, otherwise VMs would see a larger disk when run in qemu compared to Virtual PC. qemu-img always creates images so that real size and geometry match, which is the rounding you are seeing. The commit that you bisected to fixed that geometry and real size were inconsistent on fixed size disks. + +Now HyperV treats images differently. It still stores a geometry, but it doesn't actually use it to determine the size of a disk. Images created by qemu-img are generally okay because geometry and real size match, but when opening a HyperV image with qemu, the rounding we had to apply for Virtual PC is suddenly wrong. This has given us some trouble before. Now the requirement of Azure, which basically means we can't round any more even though we have to do it for Virtual PC compatibility, completes the mess. + +While we considered hacks for opening images correctly, like checking the creator application in the image, we can't do that automatically when creating an image. I'm afraid the best we can do here is to add an explicit subformat option that the user needs to specify manually. + +Jeff, any opinion on this? And should we finally implement the vpc_open() hack to distinguish by creator_app? + +If you create a subformat option I would humbly recommend focusing on Hyper-V and Azure compat, and then create an option that enforces the legacy behavior with your patch. This is essentially what some Linux distros have started to do. + + +First, I'd say that if you are converting an image over to use on Hyper-V, you would probably be better served using the VHDX format (completely different from VHD) - it is the newer format (and completely different from VHD), and is supported by QEMU as well. It is better defined and more consistent (at least so far) in its specification. + +That said, I think for the specific VHD problem we could look at the Creator field in the image. My only reservations on that are: + +1.) I haven't looked at the Creator field comprehensively across all revisions of Hyper-V and Virtual PC. But in my small sample size, it seems feasible. + +2.) It most likely won't be 100%, because of edge cases (e.g. I don't know what happens when Hyper-V opens a Virtual-PC produced VHD file, and under what circumstances it may or may not alter the Creator field) + +But the above two reservations can be overcome with the appropriate options that can be passed to the VHD format, to override the auto-detection method. + +I have access to both Virtual PC and Hyper-V, so I can put together a small patch series tomorrow to try that out. + +Unfortunately, VHDX is not supported in Azure. + +I would be happy to help test out any patches. + +I've posted a series to qemu-devel to hopefully address this issue. Cole, or anyone else that wants to test it out: http://lists.nongnu.org/archive/html/qemu-devel/2016-02/msg05511.html + +For the specific qemu-img convert case mentioned in this bug report, I believe using the new "force-size" option should address it, e.g.: + +# qemu-img convert -f raw -o subformat=fixed,force-size -O vpc source-disk.img dest-disk.vhd + + +It looks like the option is "force_size" rather than "force-size". + +It also seems to be having the intended effect. I regenerate my nixos iso with various configurations: + +qemu-master + jeffcody patches + force_size = 2147484160 +qemu-master + jeffcody patches (no force_size) = 2147992064 +qemu-stable: 2147992064 +qemu-220: 2147484160 + +2147992064 - 512 = 2147991552 (2048.484375 MiB) +2147484160 - 512 = 2147483648 (exactly 2048 MiB) + +It appears to be working. Thanks! + +(Note, I only applied the non-test patches. I have them applied on master in a fork under my user on GitHub if someone else wants to test easily). + +Thanks for testing! It is worth noting there is a v2 on the list now, that changes the creator app string to "qem2" from "qemu" when using the force_size option. That should only matter if you try to use your converted images on QEMU (so QEMU will know on that image to rely by default on the current_size field). + +Sounds good. I don't plan to retest unless you'd like me to; that change shouldn't affect me. + +For the status change, I am really sorry. +I think at that time, I just want to check the status/history of this bug, but I maybe click certain button by mistake. + +Fix is available with QEMU 2.6: +http://git.qemu.org/?p=qemu.git;a=commitdiff;h=fb9245c2610932d33ce14 + +Status changed to 'Confirmed' because the bug affects multiple users. + +Just did a source check of 1:2.6+dfsg-3ubuntu1 in yakkety and the upstream fix is already present (as it is 2.6 based). Working on the backport now to 16.04. + +I have submitted a test build of 2.5+dfsg-5ubuntu10.1~ppa1 to https://launchpad.net/~nacc/+archive/ubuntu/lp1490611. Please test that version on 16.04. + +Apologies, I uploaded an incorrect version to the PPA, please test 2.5+dfsg-5ubuntu10.3~ppa1. + + + +Uploaded to Xenial, thanks. Am I right in thinking that the new option force_size is required to be used with the patch to fix the problem? It's probably worth mentioning this in the Test Case in the bug description; otherwise it will appear to testers that the fix doesn't work. + +On 04.08.2016 [10:49:47 -0000], Robie Basak wrote: +> Uploaded to Xenial, thanks. Am I right in thinking that the new option +> force_size is required to be used with the patch to fix the problem? +> It's probably worth mentioning this in the Test Case in the bug +> description; otherwise it will appear to testers that the fix doesn't +> work. + +Agreed, updated. + + +Rejected: + +Rejected by Brian Murray: There was a security update to qemu today, so the SRU will need to be redone on top of that. + + + +Uploaded. + +It looks like there was a regression caused by that security update see bug 1612089. It might be best to coordinate with the security team, if they are doing a regular SRU, regarding the the fix for this bug. + +Can you rebase your fix on 1:2.5+dfsg-5ubuntu10.4 (due to the regression fix mentioned in #25)? +Another thing about your backport is that it dropped the qem2 bits from the patch. Is there a reason for this? If so please mention it in the debian/patch file. + +On 17.08.2016 [13:12:19 -0000], Chris J Arges wrote: +> Can you rebase your fix on 1:2.5+dfsg-5ubuntu10.4 (due to the +> regression fix mentioned in #25)? + +Will do! + +> Another thing about your backport is that it dropped the qem2 bits +> from the patch. Is there a reason for this? If so please mention it in +> the debian/patch file. + +Ah yes, those sections of the upstream fix do not apply cleanly due to +differing context (not even present). + +That does bring up another question, though: + +@smkbot or anyone else that might know. It seems the original series was +4 patches +(http://lists.nongnu.org/archive/html/qemu-devel/2016-02/msg06037.html), +and there was a follow-on of 7 patches that fixed a regression in that +series +(http://lists.nongnu.org/archive/html/qemu-devel/2016-03/msg05424.html). +Do we need to backport all 11 patches? In the first series, at least, +there is mention that patch 1/4 fixes an issue for reading VHD images. +While I realize that this particular bug is just for creating/converting +images, would it also be appropriate to backport the full set of fixes +for VHD/VPC? + + + + +On 17.08.2016 [10:20:26 -0700], Nish Aravamudan wrote: +> On 17.08.2016 [13:12:19 -0000], Chris J Arges wrote: +> > Can you rebase your fix on 1:2.5+dfsg-5ubuntu10.4 (due to the +> > regression fix mentioned in #25)? +> +> Will do! + +Done. + +> > Another thing about your backport is that it dropped the qem2 bits +> > from the patch. Is there a reason for this? If so please mention it in +> > the debian/patch file. +> +> Ah yes, those sections of the upstream fix do not apply cleanly due to +> differing context (not even present). + +The latest update includes the relevant context... + +> That does bring up another question, though: +> +> @smkbot or anyone else that might know. It seems the original series was +> 4 patches +> (http://lists.nongnu.org/archive/html/qemu-devel/2016-02/msg06037.html), +> and there was a follow-on of 7 patches that fixed a regression in that +> series +> (http://lists.nongnu.org/archive/html/qemu-devel/2016-03/msg05424.html). +> Do we need to backport all 11 patches? In the first series, at least, +> there is mention that patch 1/4 fixes an issue for reading VHD images. +> While I realize that this particular bug is just for creating/converting +> images, would it also be appropriate to backport the full set of fixes +> for VHD/VPC? + +On my own review, I believe we want to backport the 1 and 3 patch from +the first series, so that qemu-img is self-consistent, in terms of +reading and writing the new 'qem2' images. The second series does +include fixes, but not related to this bug, so if we were to include +them in a SRU, I'd rather they come in from a related bug report. + +@Stephen or anyone else affected, I've submitted an updated build at +https://launchpad.net/~nacc/+archive/ubuntu/lp1490611 for qemu +1:2.5+dfsg-5ubuntu10.5~ppa1 which should include the relevant. Please +test once it has finished building. + + +Is it correct to assume that current 16.04.2 Xenial with the 2.5 QEMU package, doesn't have this patch and can't generate MiB aligned Azure images with qemu-img (no force_size support) ? + +Any recommended PPA backport of QEMU from 16.10 (2.6+) ? + +cheers. + +Hello Alexandre, + +Yes, sorry, there have been several qemu SRUs pending and this one kept getting pushed back. Note that as far as end-users are concerned wrt. qemu, 16.04.2 is not really a relevant milestone. You'd still need to `apt update; apt upgrade` to get the latest from the repositories -- and yes, that version from the repositories does not yet have this fix. I believe Christian has it on his todo for the next SRU, though; Christian, could you confirm? + +On Fri, Feb 17, 2017 at 11:29 PM, Nish Aravamudan < +<email address hidden>> wrote: + +> I believe Christian has it on his todo for the next SRU, though; +> Christian, could you confirm? +> + +Yes, that is correct. +Sorry for the inconvenient delay due to the chain SRUs. +But at least the bigger ones we need to unbundle to make sure not making +things worse for end-users when SRU'ing. + + +Preliminary builds as preparation for an SRU currently building at + +Xenial https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/2502 + +My testing will start on those, but if you can please give it as much testing as you can as well. + +Tests for any regressions on the ppa look good so far, going to try to test the fixed case explicitly. + +I could confirm the fix on its raw alignment: +Image with 4G +Pre Fix (4295467520-512)/1024/1024 = 4096,4765625 +With Fix (4295467520-512)/1024/1024 = 4096,4765625 +With Fix + force_size set: (4294967808-512)/1024/1024 = 4096 + +That means +1. no change without opt in +2. the desired effect when option is set + +Waiting a few days for the other bugs in the SRU suggestion ppa to resolve/confirm fixes (or be dropped) and then moving to the actual SRU. + +Simplified steps to reproduce (without Azure cli / credentials) + +1. Start as in the SRU Template: +2. qemu-img convert -f raw -o subformat=fixed,force_size -O vpc source-disk.img dest-disk-old.vhd +3. upgrade +4. qemu-img convert -f raw -o subformat=fixed,force_size -O vpc source-disk.img dest-disk-new.vhd +5. qemu-img convert -f raw -o subformat=fixed -O vpc source-disk.img dest-disk-new-forced.vhd +6. check alignment: +$ stat dest-disk-old.vhd dest-disk-new.vhd dest-disk-new-forced.vhd | awk '/^ Size:/ {print ($2-512)/1024/1024}' +4096.48 +4096.48 +4096 + + +The other fixes I wanted to bundle turned out to have a too low rate of response and some show issues on testing. To not postpone this fix any longer (which it already had it share of) I un-bundled this fix and moved it into xenial-proposed now. + +It is now in the SRU review queue and given that the SRU Team acks it should show up here soon for final proposed verification. + +Hello Stephen, or anyone else affected, + +Accepted qemu into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/qemu/1:2.5+dfsg-5ubuntu10.10 in a few hours, and then in the -proposed repository. + +Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users. + +If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision. + +Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance! + +Verified Proposed: + +dd if=/dev/zero of=source-disk.img bs=1M count=4096 +qemu-img convert -f raw -o subformat=fixed -O vpc source-disk.img dest-disk-old.vhd +#upgrade +$qemu-img convert -f raw -o subformat=fixed -O vpc source-disk.img dest-disk-new.vhd +$qemu-img convert -f raw -o subformat=fixed,force_size -O vpc source-disk.img dest-disk-new-forced.vhd +#check alignment: +$ stat dest-disk-old.vhd dest-disk-new.vhd dest-disk-new-forced.vhd | awk '/^ *Size:/ {print ($2-512)/1024/1024}' +4096.48 +4096.48 +4096 + +That means: +1. without adding the new parm no behaviour change (good since it is SRU) +2. with the force_size parm the size is aligned + +That is enough for verification, but it would be even greater if that could also be tested on real azure. + +This bug was fixed in the package qemu - 1:2.5+dfsg-5ubuntu10.10 + +--------------- +qemu (1:2.5+dfsg-5ubuntu10.10) xenial; urgency=medium + + [Nishanth Aravamudan] + * debian/patches/ubuntu/add_force_size_option.patch: + block/vpc: fix VHD size calculation. (LP: #1490611) + + -- Christian Ehrhardt <email address hidden> Mon, 20 Feb 2017 13:09:53 +0100 + +The verification of the Stable Release Update for qemu has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions. + +I'm using this version on xenial, +andy@bastion:~/temp$ qemu-img -h +qemu-img version 2.5.0 (Debian 1:2.5+dfsg-5ubuntu10.31), Copyright (c) 2004-2008 Fabrice Bellard + +qemu-img convert -f raw -O vpc -o subformat=fixed,force_size /tmp/azure_config_disk_image20180901-22672-16zxelu papapa2.vhd + +unfortunately the papapa2.vhd size is 25166336!=25165824 which means it's not aligned in MiB. + +could you please help? + + diff --git a/results/classifier/zero-shot/105/assembly/1520 b/results/classifier/zero-shot/105/assembly/1520 new file mode 100644 index 000000000..74d083f87 --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/1520 @@ -0,0 +1,62 @@ +assembly: 0.885 +instruction: 0.877 +graphic: 0.876 +device: 0.871 +boot: 0.865 +mistranslation: 0.852 +other: 0.849 +semantic: 0.838 +network: 0.793 +socket: 0.761 +KVM: 0.699 +vnc: 0.678 + +x86 TCG acceleration running on s390x with -smp > host cpus slowed down by x10 +Description of problem: +This boots up a trivial guest using OVMF, when the conditions below are given it runs ~10x slower. + +I have found this breaking our tests of qemu 7.2 [(which due to Debian adding the offending change as backport is affected)](https://salsa.debian.org/qemu-team/qemu/-/blob/master/debian/patches/master/acpi-cpuhp-fix-guest-visible-maximum-access-size-to-.patch) by runnig an order of magnitude slower. + + +I was tracing it down (insert a long strange trip here) and found that it occurs: +- only with patch dab30fb "acpi: cpuhp: fix guest-visible maximum access size to the legacy reg block" applied + - latest master is still affetced +- only with s390x running emulation of x86 + - emulating x86 on ppc64 didn't show the same behavior +- only with -smp > host cpus + - smp 2 with 1 host cpu => slow + - smp 4 with 2 host cpu => slow + - any case where host cpu >= smp => fast + +On average good cases are on a 2964 s390x machine taking ~5-6 seconds for the good case. +The bad case is close to 60s which is the timeout of the automated tests. + +We all know -smp shouldn't be >host-cpus, and I totally admit that this is the definition of an edge case. +But I do not know what else might be affected and this just happened to be what the test does by default - and a slowdown by x10 seems too much even for edge cases to be just ignored. +And while we could just bump up the timeout (and probably will as an interim workaround) I wanted to file it here for your awareness. +Steps to reproduce: +You can recreate the same by using the commandline above and timing things on your own. + +Or you can use the [autopkgtest of edk2 in Ubuntu](https://git.launchpad.net/ubuntu/+source/edk2/tree/debian/tests/shell.py#n214) which have [shown this](https://autopkgtest.ubuntu.com/results/autopkgtest-lunar/lunar/s390x/e/edk2/20230224_094012_c95f4@/log.gz) first. +Additional information: +Only signed OVMF cases are affected, while aavmf and other OVMF are more or less on the same speed. + +``` +1 CPU / 1GB Memory +7.0 7.2 +6.54s 58.32s test_ovmf_ms +6.72s 56.96s test_ovmf_4m_ms +7.54s 55.47s test_ovmf_4m_secboot +7.56s 49.88s test_ovmf_secboot +7.01s 39.79s test_ovmf32_4m_secboot +7.38s 7.43s test_aavmf32 +7.27s 7.30s test_aavmf +7.26s 7.26s test_aavmf_snakeoil +5.83s 5.95s test_ovmf_4m +5.61s 5.81s test_ovmf_q35 +5.51s 5.64s test_ovmf_pc +5.26s 5.42s test_ovmf_snakeoil +``` + +Highlighting @cborntra since it is somewhat s390x related and @mjt0k as the patch is applied as backport in Debian. +I didn't find the handle of Laszlo (Author) to highlight him as well. diff --git a/results/classifier/zero-shot/105/assembly/1548170 b/results/classifier/zero-shot/105/assembly/1548170 new file mode 100644 index 000000000..acc45991f --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/1548170 @@ -0,0 +1,152 @@ +assembly: 0.684 +instruction: 0.679 +semantic: 0.677 +graphic: 0.668 +device: 0.642 +socket: 0.634 +network: 0.629 +other: 0.627 +KVM: 0.610 +mistranslation: 0.608 +boot: 0.566 +vnc: 0.504 + +qemu-kvm-spice spice server locks up when large user image is set + +QEMU emulator version 2.0.0 (Debian 2.0.0+dfsg-2ubuntu1.22), Copyright (c) 2003-2008 Fabrice Bellard + +I have QEMU kvm spice installed, running a Windows 10 1511, freshly installed this morning, using VirtIO root volume, Ethernet, and QXL graphics through Spice. It successfully installed the latest updates, and installed Visual Studio 2010 Professional plus Service Pack 1 and updates. + +Upon attempting to configure the following PNG as my account's user picture in the Settings control panel applet: + +https://static.kode54.net/pwywcomm_christopher_8bpp.png + +It added successfully, but then I noticed that Windows does not support alpha blended user pictures, and it blended it against white, so I quickly replaced it with the following: + +https://static.kode54.net/pwywcomm_christopher_blended.png + +Upon assigning that, with the other one still in the previous image buttons, Spice locked up completely. The VM was still running, as evidenced by a successful Remote Desktop session. + +Do I need to replace my entire Qemu setup with a Git or hand built official version to verify that this isn't Ubuntu's fault? + +On Mon, 02/22 04:06, Christopher Snowhill wrote: +> Public bug reported: +> +> QEMU emulator version 2.0.0 (Debian 2.0.0+dfsg-2ubuntu1.22), Copyright +> (c) 2003-2008 Fabrice Bellard +> +> I have QEMU kvm spice installed, running a Windows 10 1511, freshly +> installed this morning, using VirtIO root volume, Ethernet, and QXL +> graphics through Spice. It successfully installed the latest updates, +> and installed Visual Studio 2010 Professional plus Service Pack 1 and +> updates. +> +> Upon attempting to configure the following PNG as my account's user +> picture in the Settings control panel applet: +> +> https://static.kode54.net/pwywcomm_christopher_8bpp.png +> +> It added successfully, but then I noticed that Windows does not support +> alpha blended user pictures, and it blended it against white, so I +> quickly replaced it with the following: +> +> https://static.kode54.net/pwywcomm_christopher_blended.png +> +> Upon assigning that, with the other one still in the previous image +> buttons, Spice locked up completely. The VM was still running, as +> evidenced by a successful Remote Desktop session. +> +> Do I need to replace my entire Qemu setup with a Git or hand built +> official version to verify that this isn't Ubuntu's fault? + +I know next to nothing about graphic in QEMU but yes, it's worth to check the +lastest release (2.5) or even better the current git master. + +Fam + + +This does not appear to happen with 2.5, which I finally got installed through a full distribution upgrade to the latest beta of the upcoming LTS release. Of course, now I have another issue I didn’t notice before, where the Windows VM, which idles at about 20% of a single core most of the time, ends up using 80-100% of a single core on the host machine in the qemu-system-x86_64 process, but that’s probably what it was already doing before. + +> On Feb 22, 2016, at 5:31 PM, Fam Zheng <email address hidden> wrote: +> +> On Mon, 02/22 04:06, Christopher Snowhill wrote: +>> Public bug reported: +>> +>> QEMU emulator version 2.0.0 (Debian 2.0.0+dfsg-2ubuntu1.22), Copyright +>> (c) 2003-2008 Fabrice Bellard +>> +>> I have QEMU kvm spice installed, running a Windows 10 1511, freshly +>> installed this morning, using VirtIO root volume, Ethernet, and QXL +>> graphics through Spice. It successfully installed the latest updates, +>> and installed Visual Studio 2010 Professional plus Service Pack 1 and +>> updates. +>> +>> Upon attempting to configure the following PNG as my account's user +>> picture in the Settings control panel applet: +>> +>> https://static.kode54.net/pwywcomm_christopher_8bpp.png +>> +>> It added successfully, but then I noticed that Windows does not support +>> alpha blended user pictures, and it blended it against white, so I +>> quickly replaced it with the following: +>> +>> https://static.kode54.net/pwywcomm_christopher_blended.png +>> +>> Upon assigning that, with the other one still in the previous image +>> buttons, Spice locked up completely. The VM was still running, as +>> evidenced by a successful Remote Desktop session. +>> +>> Do I need to replace my entire Qemu setup with a Git or hand built +>> official version to verify that this isn't Ubuntu's fault? +> +> I know next to nothing about graphic in QEMU but yes, it's worth to check the +> lastest release (2.5) or even better the current git master. +> +> Fam +> +> -- +> You received this bug notification because you are subscribed to the bug +> report. +> https://bugs.launchpad.net/bugs/1548170 +> +> Title: +> qemu-kvm-spice spice server locks up when large user image is set +> +> Status in QEMU: +> New +> +> Bug description: +> QEMU emulator version 2.0.0 (Debian 2.0.0+dfsg-2ubuntu1.22), Copyright +> (c) 2003-2008 Fabrice Bellard +> +> I have QEMU kvm spice installed, running a Windows 10 1511, freshly +> installed this morning, using VirtIO root volume, Ethernet, and QXL +> graphics through Spice. It successfully installed the latest updates, +> and installed Visual Studio 2010 Professional plus Service Pack 1 and +> updates. +> +> Upon attempting to configure the following PNG as my account's user +> picture in the Settings control panel applet: +> +> https://static.kode54.net/pwywcomm_christopher_8bpp.png +> +> It added successfully, but then I noticed that Windows does not +> support alpha blended user pictures, and it blended it against white, +> so I quickly replaced it with the following: +> +> https://static.kode54.net/pwywcomm_christopher_blended.png +> +> Upon assigning that, with the other one still in the previous image +> buttons, Spice locked up completely. The VM was still running, as +> evidenced by a successful Remote Desktop session. +> +> Do I need to replace my entire Qemu setup with a Git or hand built +> official version to verify that this isn't Ubuntu's fault? +> +> To manage notifications about this bug go to: +> https://bugs.launchpad.net/qemu/+bug/1548170/+subscriptions + + + +If I get comment #2 right, the spice issue has been fixed sometime between v2.0 and v2.5, so I'm closing this ticket now. If you still encounter this issue with the latest version of QEMU (currently v4.0), feel free to open again (or report a new bug ticket for other issues). + diff --git a/results/classifier/zero-shot/105/assembly/1605 b/results/classifier/zero-shot/105/assembly/1605 new file mode 100644 index 000000000..e18742c75 --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/1605 @@ -0,0 +1,51 @@ +assembly: 0.734 +instruction: 0.734 +other: 0.725 +graphic: 0.703 +mistranslation: 0.695 +KVM: 0.688 +semantic: 0.681 +vnc: 0.676 +boot: 0.670 +socket: 0.634 +device: 0.634 +network: 0.554 + +On windows, 2nd kind vhdx-dyn bug, crash on Unexpected error in bdrv_check_qiov_request() in io.c +Description of problem: +On windows, 2nd kind vhdx-dyn bug, crash on Unexpected error in bdrv_check_qiov_request() in io.c +- qemu windows crashes during data copy + ```D:\tmpq\qemu\8.0.0-rc4\qemu\qemu-system-x86_64 -cpu qemu64 -m 4096 -machine "type=q35,kernel-irqchip=off" -accel whpx -smp "sockets=1,cores=8,threads=1" -bios D:\vstorage\win_m01_edk2-x8_64.fd -boot c -drive "index=0,if=virtio,media=disk,format=raw,file=D:\vstorage\m01_bootnoefi.raw.img" -drive "index=1,if=virtio,media=disk,format=raw,file=F:\m01_lnx.raw.img.vtoy" -drive "index=2,if=virtio,media=disk,format=vhdx,file=F:\gkpics01.vhdx" -drive "index=3,if=virtio,media=disk,format=vhdx,file=D:\test\sgdata.vhdx" -display sdl -vga virtio -rtc base=utc -netdev user,id=vmnic1,net=192.168.20.0/24,dns=192.168.20.3,dhcpstart=192.168.20.15,hostfwd=tcp::9551-:22 -device virtio-net,netdev=vmnic1 -chardev qemu-vdagent,id=ch1,name=vdagent,clipboard=on -device virtio-serial -device virtserialport,chardev=ch1,id=ch1,name=com.redhat.spice.0 -qmp "tcp:127.0.0.1:5955,server,nowait"``` + ``` ``` + ```Windows Hypervisor Platform accelerator is operational``` + ```Unexpected error in bdrv_check_qiov_request() at ../../../block/io.c:815:``` + ```D:\tmpq\qemu\8.0.0-rc4\qemu\qemu-system-x86_64.exe: offset is negative: -28983296``` + +. +- The **LINE NUMBER** : https://gitlab.com/qemu-project/qemu/-/blob/master/block/io.c#L815 +- qemu setup is ```qemu-w64-setup-20230414.exe ``` +Steps to reproduce: +1. have fresh vhdx ready create a vhdx in ```diskmgmt``` (also attached to [comment](https://gitlab.com/qemu-project/qemu/-/issues/727#note_1346341805)) +2. have vhdx with synthetic generated data ready (see process to generate sgdata in [comment](https://gitlab.com/qemu-project/qemu/-/issues/727#note_739930694) ) +3. start qemu, login, open terminal +4. Inside VM, start a terminal window, sudo root, +5. open```gdisk /dev/vdc``` create a ntfs partition +6. format as ntfs: ```mkfs.ntfs -Q -L fs_gkpics01 /dev/vdc1``` +7. mount the partition ```mount -t ntfs3 /dev/vdc1 /mnt/a -o uid=1000,gid=1000,defaults,umask=0002``` +8. mount the partition ```mount -t ntfs3 /dev/vdd2 /mnt/b -o uid=1000,gid=1000,defaults,umask=0002``` +9. In a user login, do rsync data-copy step + ```( fl="photos001" ; src="/mnt/b/sgdata" ; dst="/mnt/a" ; sdate=`date` ; echo "$sdate" ; cd "$src" ; rsync -avH "$fl" "$dst" ; echo "$sdate" ; date ; sudo -u gana DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus DISPLAY=:0.0 -- notify-send "$src/$fl" "rsync $src/$fl" )``` + + +The bug is easily reproducible. +The moment of the crash may seems spurious, but is almost certainly bound to happen. +When it happens, it can be seen to be the same error message. +Sometimes the crash happens in ```gdisk``` step, sometimes during ```mkfs.ntfs``` sometimes partway through the ```rsync```-copy, not very long into it. +Additional information: +- This has been happening for some time. I haven't used/tested vhdx much in windows much since 7.0.0 on account of other corruption bugs/lack of dependability. +- This does not happen in Linux, as tested in #727 +- The fix of #727 is unrelated to this. It doesn't have the same feel/reproduction intuitive-signature. + - Happens before (on doing the same test) + - on 8.0.0-rc1 (line number of io.c there is L811) + - on 7.2.0 (line no of io.c there is [L971](https://gitlab.com/qemu-project/qemu/-/blob/ace5a161ea1c09d8eaa8b2a717528457dc924e83/block/io.c#L971)) +- It may be caused by other changes going into block code since 7.0 . diff --git a/results/classifier/zero-shot/105/assembly/1612 b/results/classifier/zero-shot/105/assembly/1612 new file mode 100644 index 000000000..aa2f4e529 --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/1612 @@ -0,0 +1,64 @@ +assembly: 0.776 +instruction: 0.670 +graphic: 0.648 +semantic: 0.641 +device: 0.599 +other: 0.486 +network: 0.456 +socket: 0.453 +boot: 0.407 +vnc: 0.367 +mistranslation: 0.229 +KVM: 0.225 + +SVE first-faulting gather loads return incorrect data +Description of problem: +The results of `ldff1(b|h|w|d)` seem to be incorrect when `<Zt> == <Zm>`. The first element is duplicated throughout the vector while the FFR indicates that all elements were successfully loaded. This happens since https://gitlab.com/qemu-project/qemu/-/commit/50de9b78cec06e6d16e92a114a505779359ca532, and still happens on the latest master. +Steps to reproduce: +1. This assembly sequence loads data with an `ldff1d` instruction (and also loads the ffr). Note that with `ldff1d`, `<Zt> == <Zm>`. + +asmtest.s +``` + .type asmtest, @function + .balign 16 + .global asmtest +asmtest: + setffr + ptrue p0.d + index z1.d, #0, #1 + ldff1d z1.d, p0/z, [x0, z1.d, LSL #3] + rdffr p1.b + st1d {z1.d}, p0, [x1] + str p1, [x2] + ret +``` + +This harness for convenience intialises some data and checks the element at index 1, which should be 1. + +test.c +``` +#include <arm_sve.h> +#include <stdio.h> + +void asmtest(int64_t const * data, svint64_t * loaded, svbool_t * ffr); + +int main() { + const int64_t data[] = {42, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, + 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, + 22, 23, 24, 25, 26, 27, 28, 29, 30, 31}; + svint64_t loaded; + svbool_t ffr; + + asmtest(data, &loaded, &ffr); + + // Check value of element at index 1 + svuint64_t lanes = svindex_u64(0, 1); + svbool_t lane = svcmpeq_n_u64(svptrue_b64(), lanes, 1); + printf("%ld\n", svaddv_s64(lane, loaded)); +} +``` + +2. ```clang-15 -fuse-ld=lld -march=armv8-a+sve2 -target aarch64-unknown-linux-gnu -static *.c *.s -o svldffgathertest``` +3. ```qemu-aarch64 svldffgathertest``` - the value printed should be 1, but it can be seen that all values in the loaded vector are 42. +Additional information: +The above code was successfully tested on real SVE hardware. Normal gathers work fine in QEMU, as does a non-gather first-fault load. diff --git a/results/classifier/zero-shot/105/assembly/1620 b/results/classifier/zero-shot/105/assembly/1620 new file mode 100644 index 000000000..ecfad33cc --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/1620 @@ -0,0 +1,107 @@ +assembly: 0.775 +other: 0.747 +device: 0.720 +KVM: 0.715 +instruction: 0.707 +mistranslation: 0.702 +semantic: 0.688 +graphic: 0.672 +vnc: 0.639 +socket: 0.616 +boot: 0.561 +network: 0.554 + +SME FMOPA outer product instruction gives incorrect result +Description of problem: +The SME outer product instructions operate on tiles of elements. In the +below example we are performing an outer product of a vector of 1.0 +by itself. This naturally should produce a matrix filled with 1.0 +values, however if we read the values of the tile and printf them we +instead observe 0.0 values. + +Without digging into the underlying QEMU code this appears to be a bug +in how elements are set based on the tile number, since the same code +using za0.s rather than za1.s correctly reports all 1.0 values as output +as expected. + +main.c +``` +#include <stdio.h> + +void foo(float *dst); + +int main() { + float dst[16]; + foo(dst); + + // This should print: + // >>> 1.000000 1.000000 1.000000 1.000000 + // >>> 1.000000 1.000000 1.000000 1.000000 + // >>> 1.000000 1.000000 1.000000 1.000000 + // >>> 1.000000 1.000000 1.000000 1.000000 + for (int i=0; i<4; ++i) { + printf(">>> "); + for (int j=0; j<4; ++j) { + printf("%lf ", (double)dst[i * 4 + j]); + } + printf("\n"); + } +} +``` + +foo.S +``` +.global foo +foo: + stp x29, x30, [sp, -80]! + mov x29, sp + stp d8, d9, [sp, 16] + stp d10, d11, [sp, 32] + stp d12, d13, [sp, 48] + stp d14, d15, [sp, 64] + + smstart + + ptrue p0.s, vl4 + fmov z0.s, #1.0 + + // An outer product of a vector of 1.0 by itself should be a matrix of 1.0. + // Note that we are using tile 1 here (za1.s) rather than tile 0. + zero {za} + fmopa za1.s, p0/m, p0/m, z0.s, z0.s + + // Read the first 4x4 sub-matrix of elements from tile 1: + // Note that za1h should be interchangable here. + mov w12, #0 + mova z0.s, p0/m, za1v.s[w12, #0] + mova z1.s, p0/m, za1v.s[w12, #1] + mova z2.s, p0/m, za1v.s[w12, #2] + mova z3.s, p0/m, za1v.s[w12, #3] + + // And store them to the input pointer (dst in the C code): + st1w {z0.s}, p0, [x0] + add x0, x0, #16 + st1w {z1.s}, p0, [x0] + add x0, x0, #16 + st1w {z2.s}, p0, [x0] + add x0, x0, #16 + st1w {z3.s}, p0, [x0] + + smstop + + ldp d8, d9, [sp, 16] + ldp d10, d11, [sp, 32] + ldp d12, d13, [sp, 48] + ldp d14, d15, [sp, 64] + ldp x29, x30, [sp], 80 + ret +``` +Steps to reproduce: +``` +$ clang -target aarch64-linux-gnu -march=armv9-a+sme test.c -O1 -static +$ ~/qemu/build/qemu-aarch64 ./a.out +>>> 0.000000 0.000000 0.000000 0.000000 +>>> 0.000000 0.000000 0.000000 0.000000 +>>> 0.000000 0.000000 0.000000 0.000000 +>>> 0.000000 0.000000 0.000000 0.000000 +``` diff --git a/results/classifier/zero-shot/105/assembly/1649 b/results/classifier/zero-shot/105/assembly/1649 new file mode 100644 index 000000000..7d78cca98 --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/1649 @@ -0,0 +1,30 @@ +assembly: 0.984 +instruction: 0.900 +graphic: 0.900 +device: 0.747 +semantic: 0.642 +vnc: 0.610 +network: 0.592 +socket: 0.542 +mistranslation: 0.500 +boot: 0.379 +KVM: 0.352 +other: 0.209 + +"slli" instruction before "la" and "csrw" sequence leads to failure in setting the cs register +Description of problem: +slli a0, a0, 8 (1) + la a0, mtimvec (2) + csrw mtvec, a0 (3) + mtimvec: (4) + +For the above assembly snippet, the mtvec could be successfully set to the value of a0 +without the presence of the line (1) or with the shift amount being zero. However, +the mtvec can never be set successfully with the presence of line (1). +Steps to reproduce: +1. Create a test.s file and put these 4 lines of assembly into the file +2. In terminal, run: "riscv64-unknown-elf-gcc -Ttext 0x80000000 -c test.s -o test", "riscv64-unknown-elf-objcopy test -S -O binary test", and "qemu-system-riscv64 test -s -S" +3. In another terminal window, run [riscv64-unknown-elf-gdb -ex "target remote localhost:1234" -ex "layout asm"]. Keep running si command in gdb until you are at 0x80000000 where you shall see the first instruction as shown in line (1). Then keep going till you have stepped over the instruction shown in line (3). Now, run "p $mtvec" in gdb, you shall see its value being 0. +4. Redo the above steps without line (1), you shall see mtvec loaded successfully with the correct value. +Additional information: + diff --git a/results/classifier/zero-shot/105/assembly/1662050 b/results/classifier/zero-shot/105/assembly/1662050 new file mode 100644 index 000000000..1a1f36768 --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/1662050 @@ -0,0 +1,373 @@ +assembly: 0.865 +socket: 0.841 +other: 0.830 +instruction: 0.808 +device: 0.806 +semantic: 0.800 +graphic: 0.800 +KVM: 0.739 +mistranslation: 0.732 +boot: 0.693 +network: 0.645 +vnc: 0.587 + +qemu-img convert a overlay qcow2 image into a entire image + +I have a base image file "base.qcow2" and a delta qcow2 image file "delta.qcow2" whose backing file is "base.qcow2". + +Now I use qemu-img to convert "delta.qcow2" and will get a new image file "new.qcow2" which is complete and equivalent to combination of "base.qcow2" and "delta.qcow2". + +In fact, i just want to convert the delta qcow2 image file and get a new delta overlay qcow2 image file. So the "new.qcow2" is not what i want. I have to admit that this is not bug. Could you please take this as a new feature and enable qemu-img to convert images in-place? + +On 02/05/2017 09:19 PM, wayen wrote: +> +> I have a base image file "base.qcow2" and a delta qcow2 image file +> "delta.qcow2" whose backing file is "base.qcow2". +> +> Now I use qemu-img to convert "delta.qcow2" and will get a new image +> + file "new.qcow2" which is entire and equivalent to combination of +> "base.qcow2" and "delta.qcow2". +> +> In fact, i just want to convert the delta qcow2 image file and get a new +> delta overlay qcow2 image file. So the "new.qcow2" is not what i want. I +> have to admit that this is not bug. Could you please take this as a new +> feature and enable qemu-img to convert images in-place? + +This feature already exists. Use: + +qemu-img rebase -F $backing_fmt -b '' -f qcow2 delta.qcow2 + +and now delta.qcow2 will be a complete image. + +-- +Eric Blake eblake redhat com +1-919-301-3266 +Libvirt virtualization library http://libvirt.org + + + +@Eric Blake, Thanks very much for your help. In your way, I have verified that this feature already exists. + +@Eric Blake. Sorry, I didn't make it clear. In fact, I don't want to get a complete image. I just want to convert qcow2 overlay and get a new qcow2 overlay. Maybe you think my intention is meaningless， but this is what I want. + +Can't you simply do a "cp delta.qcow2 new.qcow2" ? + +@Thomas Huth, when we use qemu-img to convert a image "A" and will get a new image "B" which is equivalent to "A" . But "B" may be a lot different from "A",such as file size. The file size of "B" is usually less than "A" and this is very valuable to me. So I can't simply do a "cp delta.qcow2 new.qcow2" ? + +I meant to copy B, not A ... but I likely just haven't really understood what you're really trying to do here, so never mind. + +@Thomas Huth, I just want to reduce the file size of qcow2 overlay image by this conversion. + +On 02/08/2017 02:16 AM, wayen wrote: +> @Thomas Huth, I just want to reduce the file size of qcow2 overlay image +> by this conversion. + +Are you merely trying to sparsify the file (punch holes on the portion +of the file that is not mapped to disk), so that the apparent file size +is unchanged, but the actual disk usage is minimized? That's probably +already possible (virt-sparsify from libguestfs seems to be able to do +it; look at what steps it uses). + +Or are you asking for a way to defragment the file, so that the apparent +size shrinks because all occupied clusters are now contiguous, so that +there are no longer any need for holes? The end result is the same +amount of disk usage, but right now, the only way to defragment is to +convert from one image to a copy; we don't support in-place +defragmentation yet. So far, no one has come up with a compelling +reason why it is needed, or a patch to implement it, but there's no +technical reason why it can't be done. + +-- +Eric Blake eblake redhat com +1-919-301-3266 +Libvirt virtualization library http://libvirt.org + + + +@Eric Blake, I used virt-sparsify to sparsify the qcow2 overlay image. As you said, the actual disk usage is minimized and the apparent file size is unchanged.It is very valuable for me, because it means that my disk can hold more files. + +But we need to be careful when transfer the sparse qcow2 overlays. Because some tools do not support sparse file and will convert it into a common file,for example, scp. And I doubt what will happen when transfer sparse files between different file systems. + +So I want to use qemu-img to convert the sparse qcow2 overlay into common file. If this was feasible, the holes in the sparse overlay will be removed and the above problems will disappear. + +On Thu, Feb 09, 2017 at 02:05:54AM -0000, wayen wrote: +> @Eric Blake, I used virt-sparsify to sparsify the qcow2 overlay image. +> As you said, the actual disk usage is minimized and the apparent file +> size is unchanged.It is very valuable for me, because it means that my +> disk can hold more files. +> +> But we need to be careful when transfer the sparse qcow2 overlays. +> Because some tools do not support sparse file and will convert it into a +> common file,for example, scp. And I doubt what will happen when transfer +> sparse files between different file systems. +> +> So I want to use qemu-img to convert the sparse qcow2 overlay into +> common file. If this was feasible, the holes in the sparse overlay will +> be removed and the above problems will disappear. + +You can use the drive_mirror HMP command or drive-mirror QMP command on +a running QEMU instance to copy the data to a new file and switch to the +new file. + +I'm curious what exactly is being optimized by copying out a fresh qcow2 +image. + +Please post the output of: + + $ stat delta.qcow2 + $ qemu-img map delta.qcow2 + $ stat new.qcow2 + $ qemu-img map new.qcow2 + +This will allow us to see the size and data layout differences. + +Maybe a new command should be added to QEMU to do the optimization +in-place. This would avoid the disk space overhead associated with +drive-mirror, cp, qemu-img convert, etc. + +Stefan + + + + + + + + + + +On Tue, Feb 14, 2017 at 02:17:16AM -0000, wayen wrote: +> ** Attachment added: "qemu-img map new.qcow2 output" +> https://bugs.launchpad.net/qemu/+bug/1662050/+attachment/4818564/+files/qemu_img_map_new_qcow2.txt + +Thanks for posting the attachments. + +I ran a script to find unallocated clusters in the delta.qcow2 host +file. Most are actually qcow2 metadata (L1/L2 tables, refcount blocks). + +This output shows that any image file size reduction you are hoping to +achieve can only come from zero clusters. + +There are no holes in the files that would result in significant image +file size reduction if a new image were written out. + +Just wanted to share this info in case anyone else is thinking about how +to optimize qcow2 files. I still think rewriting images sequentially +can be useful - if internal snapshots were used and deleted then COW can +result in holes. + +Hole at 0 size 5.0 clusters +Hole at 393216 size 1.0 clusters +Hole at 589824 size 1.0 clusters +Hole at 1114112 size 1.0 clusters +Hole at 1310720 size 1.0 clusters +Hole at 1507328 size 1.0 clusters +Hole at 1703936 size 1.0 clusters +Hole at 2293760 size 1.0 clusters +Hole at 2621440 size 1.0 clusters +Hole at 3080192 size 1.0 clusters +Hole at 5111808 size 1.0 clusters +Hole at 6291456 size 1.0 clusters +Hole at 30408704 size 1.0 clusters +Hole at 47906816 size 1.0 clusters +Hole at 142671872 size 1.0 clusters +Hole at 219545600 size 1.0 clusters +Hole at 667090944 size 1.0 clusters +Hole at 853868544 size 1.0 clusters +Hole at 1562640384 size 1.0 clusters +Hole at 2147483648 size 1.0 clusters +Hole at 2617180160 size 1.0 clusters +Hole at 3411148800 size 1.0 clusters +Hole at 4107075584 size 1.0 clusters +Hole at 4294967296 size 1.0 clusters +Hole at 4452646912 size 1.0 clusters +Hole at 4792057856 size 1.0 clusters +Hole at 5494865920 size 1.0 clusters +Hole at 5645271040 size 1.0 clusters +Hole at 5702483968 size 1.0 clusters +Hole at 6187188224 size 1.0 clusters +Hole at 6442450944 size 1.0 clusters +Hole at 6862995456 size 1.0 clusters +Hole at 6987317248 size 1.0 clusters +Hole at 7567245312 size 1.0 clusters +Hole at 8135245824 size 1.0 clusters +Hole at 8590589952 size 1.0 clusters +Hole at 8613462016 size 1.0 clusters +Hole at 9055436800 size 1.0 clusters +Hole at 9703522304 size 1.0 clusters +Hole at 10279321600 size 1.0 clusters +Hole at 10737418240 size 3.0 clusters +Hole at 10844372992 size 1.0 clusters +Hole at 11167858688 size 1.0 clusters +Hole at 11209605120 size 1.0 clusters +Hole at 11209801728 size 1.0 clusters +Hole at 11730944000 size 1.0 clusters +Hole at 12183207936 size 1.0 clusters +Hole at 12705464320 size 1.0 clusters +Hole at 12884901888 size 1.0 clusters +Hole at 13444120576 size 1.0 clusters +Hole at 13910016000 size 1.0 clusters +Hole at 14182711296 size 1.0 clusters +Hole at 15025635328 size 1.0 clusters +Hole at 15032385536 size 1.0 clusters + +The following script draws the allocated clusters and holes in the image +file. I took your qemu-img map output, filtered out any lines with +base.qcow2, and sorted using sort -k3 -g to sort on the "Mapped to" +field. Then I ran ./qcow2-map-svg.py <filtered.txt >output.svg. + +#!/usr/bin/python3 +import sys +import io + +COLOR_ALLOCATED = '#ffaaaa' +COLOR_HOLE = '#999999' + +def svg_percentage(value, total): + return '{0}%'.format(100.0 * value / total) + +def svg_rect(x, width, color): + print('<rect x="{0}" y="0" width="{1}" height="40" fill="{2}" stroke="none" />'.format(x, width, color), file=out) + +def svg_text(x, y, text): + print('<text x="{0}" y="{1}">{2}</text>'.format(x, y, text), file=out) + +out = io.StringIO() + +print('''<?xml version="1.0" encoding="UTF-8" ?> +<svg xmlns="http://www.w3.org/2000/svg" version="1.1">''', file=out) + +file_map = [] +header = True +for line in sys.stdin: + if header: + header = False + continue + + offset, length, mapped, filename = line.split() + offset = int(offset, 16) + length = int(length, 16) + mapped = int(mapped, 16) + + file_map.append((offset, length, mapped, filename)) + +file_size = file_map[-1][2] + file_map[-1][1] +last_mapped = 0 +for _, length, mapped, _ in file_map: + if mapped > last_mapped: +# if mapped - last_mapped: +# print('Hole at {0} size {1} clusters'.format(last_mapped, (mapped - last_mapped) / 65536)) + svg_rect(svg_percentage(last_mapped, file_size), + svg_percentage(mapped - last_mapped, file_size), + COLOR_HOLE) + + svg_rect(svg_percentage(mapped, file_size), + svg_percentage(length, file_size), + COLOR_ALLOCATED) + last_mapped = mapped + length +if last_mapped < file_size: + svg_rect(svg_percentage(last_mapped, file_size), + svg_percentage(file_size - last_mapped, file_size), + COLOR_HOLE) + +for i in range(10): + svg_text(svg_percentage(i, 10), 60, svg_percentage(i, 10)) + +print('</svg>', file=out) + +print(out.getvalue()) + + +Is there any way to remove holes from qcow2 overlay images? It's very important to me. I am looking forward to your reply. + +Is there any way to remove holes from sparse qcow2 overlay? It's very important to me. I am looking forward to your reply. + +Hi wayen: + Which version are you used? + I also find this problem on old version qemu, and i write a patch +for it. It works. + I'm not sure the mainline version have solve this problem. + what command are you used? + +On Mon, Apr 10, 2017 at 10:14 AM, wayen <email address hidden> wrote: +> Is there any way to remove holes from qcow2 overlay images? It's very +> important to me. I am looking forward to your reply. +> +> -- +> You received this bug notification because you are a member of qemu- +> devel-ml, which is subscribed to QEMU. +> https://bugs.launchpad.net/bugs/1662050 +> +> Title: +> qemu-img convert a overlay qcow2 image into a entire image +> +> Status in QEMU: +> Incomplete +> +> Bug description: +> I have a base image file "base.qcow2" and a delta qcow2 image file +> "delta.qcow2" whose backing file is "base.qcow2". +> +> Now I use qemu-img to convert "delta.qcow2" and will get a new image +> file "new.qcow2" which is entire and equivalent to combination of +> "base.qcow2" and "delta.qcow2". +> +> In fact,I don't want to get a complete image.I just want to convert +> delta qcow2 image file "A" to a New delta overlay qcow2 image "B" +> which is equivalent to "A". So the "new.qcow2" is not what i want. I +> have to admit that this is not bug. Could you please take this as a +> new feature and enable qemu-img to convert qcow2 overlay itself? +> +> To manage notifications about this bug go to: +> https://bugs.launchpad.net/qemu/+bug/1662050/+subscriptions +> + + +Hi Lidong Chen: + I used QEMU 2.0.0 on Ubuntu 14.04. + Do you mean your patch can make qemu-img convert qcow2 overlay into a new overlay? + + +On Mon, Apr 10, 2017 at 1:07 PM, wayen <email address hidden> wrote: +> Hi Lidong Chen: +> I used QEMU 2.0.0 on Ubuntu 14.04. +> Do you mean your patch can make qemu-img convert qcow2 overlay into a new overlay? + +yes. but i find it already fixed in 2.0.0. +do you add the -o backing_file= option in the command? + +> +> -- +> You received this bug notification because you are a member of qemu- +> devel-ml, which is subscribed to QEMU. +> https://bugs.launchpad.net/bugs/1662050 +> +> Title: +> qemu-img convert a overlay qcow2 image into a entire image +> +> Status in QEMU: +> Incomplete +> +> Bug description: +> I have a base image file "base.qcow2" and a delta qcow2 image file +> "delta.qcow2" whose backing file is "base.qcow2". +> +> Now I use qemu-img to convert "delta.qcow2" and will get a new image +> file "new.qcow2" which is entire and equivalent to combination of +> "base.qcow2" and "delta.qcow2". +> +> In fact,I don't want to get a complete image.I just want to convert +> delta qcow2 image file "A" to a New delta overlay qcow2 image "B" +> which is equivalent to "A". So the "new.qcow2" is not what i want. I +> have to admit that this is not bug. Could you please take this as a +> new feature and enable qemu-img to convert qcow2 overlay itself? +> +> To manage notifications about this bug go to: +> https://bugs.launchpad.net/qemu/+bug/1662050/+subscriptions +> + + +Hi Lidong Chen: + + I forgot to use this option.I think the way you said is effective.I will try it.Thank you very much for your help + +[Expired for QEMU because there has been no activity for 60 days.] + diff --git a/results/classifier/zero-shot/105/assembly/1724570 b/results/classifier/zero-shot/105/assembly/1724570 new file mode 100644 index 000000000..4d4fabe5b --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/1724570 @@ -0,0 +1,95 @@ +assembly: 0.726 +semantic: 0.724 +other: 0.691 +mistranslation: 0.691 +instruction: 0.685 +graphic: 0.670 +device: 0.663 +socket: 0.647 +network: 0.619 +boot: 0.585 +vnc: 0.563 +KVM: 0.450 + +qemu-system-x86_64 generates ACPI tables with broken endianess when run on big-endian hosts + +The bios-tables-test always fails when run on a big-endian host, which has iasl installed. When it calls iasl to dumps the AML files into ASL files, iasl complains + +Intel ACPI Component Architecture +ASL+ Optimizing Compiler/Disassembler version 20170831 +Copyright (c) 2000 - 2017 Intel Corporation + +Input file aml-4L677Y, Length 0x38 (56) bytes +Table [TEPH] is too long for file - needs: 0x38000000, remaining in file: 0x38 +Could not get ACPI tables from aml-4L677Y, AE_BAD_HEADER + + +At first I thought this was an iasl bug, but the latest version of iasl in rawhide is ported to big endian. + +So I looked at the actual AML files that bios-tables-test extracts from the qemu-system-x86_64 memory space, when running on ppc64 host. These do indeed have different content from the AML files generated by qemu-system-x86_64 when running on an x86_64 host. + + +eg the AML file for the HPET shows + +< 0000000 T E P H nul nul nul 8 soh etx B O C H S sp +< 4554 4850 0000 3800 0301 4f42 4843 2053 +< 0000020 B X P C H P E T nul nul nul soh B X P C +< 5842 4350 5048 5445 0000 0100 5842 4350 +< 0000040 nul nul nul soh soh " ack nul nul nul nul nul nul nul P ~ +< 0000 0100 a201 8086 0000 0000 0000 fed0 +--- +> 0000000 H P E T 8 nul nul nul soh etx B O C H S sp +> 5048 5445 0038 0000 0301 4f42 4843 2053 +> 0000020 B X P C H P E T soh nul nul nul B X P C +> 5842 4350 5048 5445 0001 0000 5842 4350 +> 0000040 soh nul nul nul soh " ack nul nul nul nul nul nul nul P ~ +> 0001 0000 a201 8086 0000 0000 0000 fed0 + +so not only is the table name inverted, but the lenght is inverted, and several fields later on are inverted too. + +Other AML files for APIC and DSDT show similar brokenness + +This is seen with QEMU 2.10.0 + + + + + + + + + + + + + +I think something like this should fix this issue: + +diff a/tests/bios-tables-test.c b/tests/bios-tables-test.c +--- a/tests/bios-tables-test.c ++++ b/tests/bios-tables-test.c +@@ -279,8 +279,19 @@ static void dump_aml_files(test_data *data, bool rebuild) + } + g_assert(fd >= 0); + ++ sdt->header.signature = cpu_to_le32(sdt->header.signature); ++ sdt->header.length = cpu_to_le32(sdt->header.length); ++ sdt->header.oem_revision = cpu_to_le32(sdt->header.oem_revision); ++ sdt->header.asl_compiler_revision = cpu_to_le32(sdt->header.asl_compiler_revision); ++ + ret = qemu_write_full(fd, sdt, sizeof(AcpiTableHeader)); + g_assert(ret == sizeof(AcpiTableHeader)); ++ ++ sdt->header.signature = le32_to_cpu(sdt->header.signature); ++ sdt->header.length = le32_to_cpu(sdt->header.length); ++ sdt->header.oem_revision = le32_to_cpu(sdt->header.oem_revision); ++ sdt->header.asl_compiler_revision = le32_to_cpu(sdt->header.asl_compiler_revision); ++ + ret = qemu_write_full(fd, sdt->aml, sdt->aml_len); + g_assert(ret == sdt->aml_len); + + + +Fixed here: +https://git.qemu.org/?p=qemu.git;a=commitdiff;h=3831c07b89ab1f7aa1427 + diff --git a/results/classifier/zero-shot/105/assembly/1772262 b/results/classifier/zero-shot/105/assembly/1772262 new file mode 100644 index 000000000..8e29a24b2 --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/1772262 @@ -0,0 +1,67 @@ +assembly: 0.849 +device: 0.843 +instruction: 0.838 +other: 0.835 +semantic: 0.830 +graphic: 0.825 +network: 0.816 +boot: 0.804 +KVM: 0.795 +vnc: 0.784 +mistranslation: 0.781 +socket: 0.762 + +Adding -spice doesn't respect environment variable QEMU_AUDIO_DRV + +When -spice is added to the commandline, QEMU_AUDIO_DRV=alsa is not respected and I receive no audio from the guest using the alsa driver. When -spice options are omitted, audio works as usual. + +I want to channel mouse and keyboard events to the guest using spice (with looking-glass) instead of a third-party product over a network interface (with synergy), but audio ends up over the spice protocol as well instead of using alsa despite asking otherwise. For example, one will only hear audio when a program like spicy is running. Naturally it would be nice if this were not needed, since spice audio is gappier than regular audio on my machine. + +Perhaps if -spice added an option similar to agent-mouse=off but for audio, (say audio-pipe=off or so for example,) this might mitigate the issue in a more naive fashion UX-wise, without having to force the issue with environment variables that unfortunately for me don't seem to work. :( + +QEMU emulator version 2.12.0 + +Commandline + + qemu-system-x86_64 \ + -ctrl-grab \ + -enable-kvm \ + -cpu host,hv-time,kvm=off \ + -smp cores=4 \ + -m 8G \ + -M q35 \ + -vga none \ + -netdev tap,id=hostnet,ifname=tap1,script=no,downscript=no \ + -net nic,model=virtio,macaddr=52:54:FD:BF:F7:9A,netdev=hostnet \ + -netdev user,id=usernet,smb=/media/DRIVE-C/tux/vms/share \ + -net nic,model=virtio,macaddr=52:54:2E:40:4F:C8,netdev=usernet \ + -usb \ + -device usb-ehci,id=ehci \ + -device ioh3420,bus=pcie.0,addr=1c,multifunction=on,port=1,chassis=1,id=root.1 \ + -device vfio-pci,host=07:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on \ + -device ich9-intel-hda \ + -device hda-micro \ + -drive if=virtio,file=vm/win10/disk.img,media=disk \ + -boot menu=on,splash=splash/boot.jpg,splash-time=5000 \ + -name win10 \ + -device ivshmem-plain,memdev=ivshmem \ + -object memory-backend-file,id=ivshmem,share=on,mem-path=/dev/shm/looking-glass,size=32M \ + -device virtio-serial-pci \ + -device virtserialport,chardev=spicechannel0,name=com.redhat.spice.0 \ + -chardev spicevmc,id=spicechannel0,name=vdagent \ + -spice addr=127.0.0.1,port=5900,disable-ticketing + +When the last four options are not present, audio works as expected. This is with both a windows 10 guest and a windows 7 guest. + +Doesn't reproduce. The *default* driver is changed to spice when spice is enabled, but overriding using QEMU_AUDIO_DRV is not disabled. Any chance you didn't export QEMU_AUDIO_DRV? + +>Any chance you didn't export QEMU_AUDIO_DRV? + +Okay, I just checked by calling env in the same fashion. + +Well lash me bootlaces, it turns out that sudo needs to be invoked with the -E flag to preserve the environment. It's funny, the examples I read up on never mentioned this. + +Sorry for littering the bug tracker with a silly end-user problem. + +Cheers. + diff --git a/results/classifier/zero-shot/105/assembly/1787002 b/results/classifier/zero-shot/105/assembly/1787002 new file mode 100644 index 000000000..8ddd04df1 --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/1787002 @@ -0,0 +1,37 @@ +assembly: 0.914 +graphic: 0.775 +device: 0.753 +instruction: 0.728 +vnc: 0.613 +other: 0.605 +semantic: 0.604 +network: 0.552 +socket: 0.470 +boot: 0.335 +mistranslation: 0.288 +KVM: 0.196 + +disas/i386.c compile error + +QEMU Version: 2.12.1, 3.0.0-rc4 +Compiling with GCC 8.2.0 +System: Plop Linux, 32 bit + +Error: + CC disas/i386.o +/tmp/ccK8tHRs.s: Assembler messages: +/tmp/ccK8tHRs.s:53353: Error: can't resolve `L0' {*ABS* section} - `obuf' {.bss section} + + +The problematic line is in 'disas/i386.c' in the function 'INVLPG_Fixup (int bytemode, int sizeflag)': +strcpy (obuf + strlen (obuf) - 6, alt); + +If I comment out this line, then compiling works without problems. + + +The error comes only on 32 bit. On 64 bit, compiling works without problems. + +Looking through old bug tickets ... This sounds like it was rather a bug in your toolchain ... is it still reproducible with a newer version of your Linux distro and the latest version of QEMU? + +[Expired for QEMU because there has been no activity for 60 days.] + diff --git a/results/classifier/zero-shot/105/assembly/1806114 b/results/classifier/zero-shot/105/assembly/1806114 new file mode 100644 index 000000000..c4243b867 --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/1806114 @@ -0,0 +1,93 @@ +assembly: 0.860 +graphic: 0.823 +other: 0.817 +device: 0.798 +instruction: 0.793 +network: 0.746 +socket: 0.724 +KVM: 0.703 +boot: 0.697 +semantic: 0.675 +mistranslation: 0.659 +vnc: 0.631 + +Reading sectors from floppy with BIOS INT 13h is broken + +I'm developing a game bootable from a floppy disk, written in i386 assembly. I found out it doesn't work on newer QEMU versions. I managed to isolate the issue and it seems that there's a problem with handling of BIOS interrupt 13h when it comes to reading disk sectors (AH=02). + +I've written a simple test in assembly. It simply accesses four different floppy disk sectors and prints out the strings they contain. The problem is, the two latter strings don't show up at all nor the BIOS interrupt returns an error (CF set). I've attached the code to this bug report. I use following commands to compile it and run: + +$ nasm disk-test.asm -o disk-test.bin +$ qemu-system-i386 -boot a -fda disk-test.bin + +After running, the expected output is: + + I am a test string from boot sector + C:H:S 0:0:2 - Hello + C:H:S 0:0:3 - there! + C:H:S 0:1:4 - In QEMU you can't + C:H:S 1:0:5 - see these two lines! :( + +while the actual output is: + + I am a test string from boot sector + C:H:S 0:0:2 - Hello + C:H:S 0:0:3 - there! + + +So far, I found this problem in the current QEMU version for Ubuntu 18.04 (Debian 1:2.11+dfsg-1ubuntu7.8), as well as in the 3.1.0-rc3 and 2.12.1 versions, available on the www.qemu.org website. Thus, the issue doesn't seem to be very recent. + +To be sure, I ran the program on my other machine with older QEMU version (Debian 2.0.0+dfsg-2ubuntu1.43) and on my RaspberryPi 2 (Debian 1.1.2+dfsg-6+deb7u25) and everything works as expected there. It also works well in VirtualBox. + +I hope these information are useful to you, however, I feel like this bug may be more related to Seabios. If so, please let me know and I will report it somewhere else. + + + +Hello! Thank you for your kindly and detailed report. + +Since this is via the INT 13h mechanism, the problem is likely in SeaBIOS. + +If I'm reading the version strings right, it looks like QEMU 2.0.0 QEMU 1.1.2 appear to be working correctly for you, but do you know what versions of the SeaBIOS firmware they're running with? + +It might be interesting to see if, on QEMU 2.0.0 but a modern version of SeaBIOS, that it still breaks the same way -- that way we can tell for sure if it's a QEMU regression or a SeaBIOS one. + +Thank you, +--js + +I ran plenty of QEMU/SeaBIOS configurations and these are the results: + +Ubuntu 18.04: + OK --> QEMU qemu-2.0.0 + SEABIOS rel-1.7.4-0-g96917a8 + OK --> QEMU qemu-2.0.0 + SEABIOS 1.10.2-1ubuntu1 (default) + BAD --> QEMU Debian 1:2.11+dfsg-1ubuntu7.8 (default) + SEABIOS 1.10.2-1ubuntu1 (default) + BAD --> QEMU Debian 1:2.11+dfsg-1ubuntu7.8 (default) + SEABIOS 1.7.4-4ubuntu1 + BAD --> QEMU qemu-3.1.0-rc3 + SEABIOS rel-1.12.0-0-ga698c89 + +Ubuntu 14.04: + OK --> QEMU Debian 2.0.0+dfsg-2ubuntu1.44 (default) + SEABIOS 1.7.4-4ubuntu1 (default) + OK --> QEMU Debian 2.0.0+dfsg-2ubuntu1.44 (default) + SEABIOS rel-1.12.0-0-ga698c89 + BAD --> QEMU qemu-3.1.0-rc3 + SEABIOS 1.7.4-4ubuntu1 (default) + BAD --> QEMU qemu-3.1.0-rc3 + SEABIOS rel-1.12.0-0-ga698c89 + +Kali Linux (Debian 4.17.8-1kali1): + OK --> QEMU qemu-2.0.0 + SEABIOS rel-1.7.4-0-g96917a8 + OK --> QEMU qemu-2.0.0 + SEABIOS 1.11.1-1 (default) + BAD --> QEMU Debian 1:2.12+dfsg-3+b1 (default) + SEABIOS 1.11.1-1 (default) + BAD --> QEMU Debian 1:2.12+dfsg-3+b1 (default) + SEABIOS rel-1.7.4-0-g96917a8 + BAD --> QEMU qemu-3.1.0-rc3 + SEABIOS rel-1.12.0-0-ga698c89 + +Surprisingly, it seems that the problem is in QEMU. I didn't test versions between 2.0 and 2.12, because I had some issues with building (util/memfd.c:43:12: error: static declaration of ‘memfd_create’ follows non-static declaration). I'll try to fix it by hand and will let you know if I manage to determine what's the first version to cause problems. + +Alright, I managed to compile 2.5.0, 2.5.1 and 2.6.0 by simply making memfd_create() non-static. + +It turns out that 2.6.0 is the first version causing problems. I tested each of these with the provided SeaBIOS version and with SeaBIOS rel-1.7.4-0-g96917a8. + +The memfd_create() compilation failure is a bug in older QEMU versions that was only revealed by a newer glibc version (ie old QEMU built OK with older glibc but fails with newer glibc). It's fixed in git in commit 75e5b70e6b5dcc4f221, so backporting that to the older versions you're trying to build is probably the correct fix there, though it sounds like you've sufficiently worked around it anyway. + +I see. That wasn't much of a problem, but thank you for the information. I'll try to keep it in mind when building QEMU in future. + +The QEMU project is currently considering to move its bug tracking to another system. For this we need to know which bugs are still valid and which could be closed already. Thus we are setting older bugs to "Incomplete" now. +If you still think this bug report here is valid, then please switch the state back to "New" within the next 60 days, otherwise this report will be marked as "Expired". Or mark it as "Fix Released" if the problem has been solved with a newer version of QEMU already. Thank you and sorry for the inconvenience. + +[Expired for QEMU because there has been no activity for 60 days.] + diff --git a/results/classifier/zero-shot/105/assembly/1847793 b/results/classifier/zero-shot/105/assembly/1847793 new file mode 100644 index 000000000..580cbea22 --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/1847793 @@ -0,0 +1,265 @@ +assembly: 0.767 +device: 0.754 +instruction: 0.706 +KVM: 0.686 +boot: 0.680 +other: 0.651 +mistranslation: 0.609 +graphic: 0.608 +vnc: 0.590 +socket: 0.567 +network: 0.538 +semantic: 0.520 + +qemu 4.1.0 - Corrupt guest filesystem after new vm install + +When I install a new vm with qemu 4.1.0 all the guest filesystems are corrupt. The first boot from the install dvd iso is ok and the installer work fine. But the guest system hangs after the installer finishes and I reboot the guest. I can see the grub boot menue but the system cannot load the initramfs. + +Testet with: +- RedHat Enterprise Linux 7.5, 7.6 and 7.7 (RedHat uses xfs for the /boot and / partition) +Guided install with the graphical installer, no lvm selected. +- Debian Stable/Buster (Debian uses ext4 for / and /home partition) +Guidet install with the graphical installer and default options. + +Used commandline to create the vm disk image: +qemu-img create -f qcow2 /volumes/disk2-part2/vmdisks/vmtest10-1.qcow2 20G + +Used qemu commandline for vm installation: +#!/bin/sh +# vmtest10 Installation +# +/usr/bin/qemu-system-x86_64 -cpu SandyBridge-IBRS \ + -soundhw hda \ + -M q35 \ + -k de \ + -vga qxl \ + -machine accel=kvm \ + -m 4096 \ + -display gtk \ + -drive file=/volumes/disk2-part2/images/debian-10.0.0-amd64-DVD-1.iso,if=ide,media=cdrom \ + -drive file=/volumes/disk2-part2/images/vmtest10-1.qcow2,if=virtio,media=disk,cache=writeback \ + -boot once=d,menu=off \ + -device virtio-net-pci,mac=52:54:00:2c:02:6c,netdev=vlan0 \ + -netdev bridge,br=br0,id=vlan0 \ + -rtc base=localtime \ + -name "vmtest10" \ + -usb -device usb-tablet \ + -spice disable-ticketing \ + -device virtio-serial-pci \ + -device virtserialport,chardev=spicechannel0,name=com.redhat.spice.0 \ + -chardev spicevmc,id=spicechannel0,name=vdagent $* + +Host OS: +Archlinux (last updated at 10.10.2019) +Linux testing 5.3.5-arch1-1-ARCH #1 SMP PREEMPT Mon Oct 7 19:03:08 UTC 2019 x86_64 GNU/Linux +No libvirt in use. + + +With qemu 4.0.0 it works fine without any errors. + +Hi Claus, + Some things to try: + + a) after you quit qemu can you try qemu-img check on the qcow2 file to see if it's happy? + b) If you repeat your test using a raw image file rather than a qcow2 is it any happier? + c) How repeatable is it? If it's very repeatable it would be great if you could perform a git bisect to find which commit breaks it; we can walk you through it if you've not done it before. + +Hi David, + +a) +> qemu-img check /volumes/disk2-part2/images/vmtest10-1.qcow2 +No errors were found on the image. +24794/327680 = 7.57% allocated, 9.28% fragmented, 0.00% compressed clusters +Image end offset: 1625751552 + +> qemu-img info /volumes/disk2-part2/images/vmtest10-1.qcow2 +image: vmtest10-1.qcow2 +file format: qcow2 +virtual size: 20 GiB (21474836480 bytes) +disk size: 1.29 GiB +cluster_size: 65536 +Format specific information: + compat: 1.1 + lazy refcounts: false + refcount bits: 16 + corrupt: false + +b) +The raw image file works without any errors after install and reboot. +I created the image file with: +qemu-img create -f raw /volumes/disk2-part2/images/vmtest10-1.img 20G +Changes to the qemu commandline: +-drive format=raw,file=/volumes/disk2-part2/images/vmtest10-1.img,if=virtio,media=disk,cache=writeback \ + +c) +I can always repeat this behavior since 4.1.0 is out. +I could perform a git bisect. But I need help, I've never done that before. + +OK, thanks. + This might be the same problem as https://bugs.launchpad.net/qemu/+bug/1846427 +but we'll have to see. + +For a bisect; first of all, check out qemu from git and build 4.1.0; +check to make sure this breaks. +Now, see the git bisect instructions at: +https://git-scm.com/docs/git-bisect + +do: +git bisect start +git bisect bad +git bisect good v4.0.0 + +it'll then checkout a commit somewhere in between for you; build it, and then do either + +git bisect good or git bisect bad + +and it'll pick another commit + +It'll probably take 13 builds to nail the offending commit. + +Hi Claus, + +Do you use XFS on the host? + +Max + +I can confirm exactly the same issue on Arch linux with ext4 filesystem (qemu-4.1.0). + +After downgrading from 4.1.0 => 4.0.0 everything is running normal again, no corruption detected and all qcow2 images stays healthy. + +Hi Max, from my <https://bugs.launchpad.net/qemu/+bug/1846427/comments/8>: I've seen corruption on ext4. + +The bug reported here is not about qcow2 metadata corruption, but about guest data corruption (qemu-img check reports a clean image). It’s entirely possible (and I would even say likely) that there are two different causes. + +We know about two guest data corruptions already (which appeared in 4.1), and both seem to only appear on XFS. We have fixed one, the other we don’t quite know yet. + +Therefore, I’m wondering whether this is a guest data corruption that we probably already know about (because it’s on XFS), or whether we don’t (because it isn’t). + +In any case, I would separate these two bug reports on the basis that this one here is about guest data corruption, whereas 1846427 is about qcow2 metadata corruption. + +Max + +i've seen guest data corruption and qcow2 corruption on ext4. + +i've seen one case where the guest (win10) reports corruption but qemu-img check does not, but that's the outlier, usually both guest and qemu-img report corruption. + +for me the issue seems to only be win10 guests using virtio-scsi, i've not seen it on any of 25+ linux/solaris/macos/win2019 guests no matter what device driver/cache/trim i use. + +current workaround is convert from qcow2 to raw, everything else stays the same and i have no issues. + +I suppose that the problem described in bug 1846427 can also affect guest data, so I think it makes sense to divide based on whether there are only data corruptions or both data and metadata corruptions. + +So far, I don’t know of a report of pure guest data corruptions (without qcow2 metadata being affected) that didn’t happen on XFS, so I assume there is an issue that affects both data and metadata on all filesystems (described by 1846427; Kevin has sent a patch series upstream ot address it), and another one that only affects guest data and only occurs on XFS (this one). + +Actually, there are two problems we know of on XFS: + +The first one was a bug in qemu that has been fixed upstream by b2c6f23f4a9f6d8f1b648705cd46d3713b78d6a2. People that don’t use master but the 4.1 release instead are likely to hit that problem instead of the other one. + +The second one seems to be a kernel bug. When fallocating (writing zeroes in our case) and writing to a file in parallel, the write is discarded if: +- The fallocated area begins at or after the EOF, +- The written area begins after the fallocated area, +- The write is submitted through the AIO interface (io_submit()), +- The write and the fallocate operation are submitted before either one finishes (i.e. concurrently), +- The fallocate operation finishes after the write. + +In qemu, this happens only with aio=native, and then most of the time when an FALLOC_FL_ZERO_RANGE happens after the EOF while a write after that range is ongoing. + + +Claus as the reporter didn’t use aio=native, so if he’s indeed on XFS, he can’t have hit this second bug. If he’s on XFS, he will most likely have hit the first one that’s already fixed in master. + + +Still, we need to fix the second bug. As for how… It looks to me like a kernel bug, so in qemu we can’t do anything to fix it. But we should probably work around it. Kevin has proposed making zero-writes on XFS serializing until infinity, basically (i.e. UINT64_MAX in practice). That gives us some layering problems (either the file-posix block driver needs access to the TrackedRequest to extend its length, or the generic block layer needs to know whether a file-posix node is on XFS), and it yields the question of how to detect whether the bug has been fixed in the kernel. + +Max + +I have the same (related?) issue and wanted to add my experience with it. I had 3 qemu qcow2 VM running on ArchLinux. I never used snapshots or something like it. Just normal start&shutdown. 2 of these VMs were also ArchLinux running on ext4. Both of these VMs had a data corruption inside the quest. The data being corrupted were files I had not touched in month (large tar archives). One guest was running on a SSD with discard, the other VM was running on a normal hard drive without any discard. +The last VM was a Windows 10 VM. While the VM was running fine, after "fixing" the image issues with qemu-img -r all hdd.qcow2 the Windows 10 installation was unbootable and beyond repair with normal Windows tools. + +While the VMs are running I saw these lines printed by qemu (for all VMs in question): + +qcow2_free_clusters failed: Invalid argument +qcow2_free_clusters failed: Invalid argument +qcow2_free_clusters failed: Invalid argument + +I recreated my VMs and I now chose btrfs as a filesystem. No issues yet on the image. I also recreated the Windows 10 VM. It worked fine a couple of days. Today I checked the image, after I saw the free_clusters lines above again: + +Many many lines like this: +Leaked cluster 260703 refcount=1 reference=0 +ERROR cluster 260739 refcount=0 reference=1 +ERROR OFLAG_COPIED data cluster: l2_entry=800000038ec10000 refcount=0 + +638 errors were found on the image. +Data may be corrupted, or further writes to the image may corrupt it. + +339 leaked clusters were found on the image. +This means waste of disk space, but no harm to data. +314734/4096000 = 7.68% allocated, 26.70% fragmented, 0.00% compressed clusters +Image end offset: 21138374656 + +The installation itself still works but I don't know if there are any silently corrupted files in there. + +QEMU 4.1.0 from ArchLinux +Host-Filesystem is ext4 +Start-Parameter (the same on all VMs): + +qemu-system-x86_64 -cpu Haswell-noTSX -M q35 -enable-kvm -smp 4,cores=4,threads=1,sockets=1 -net nic,model=virtio -net user,hostname=WindowsKVM.local -drive if=none,id=hd,file=hdd.qcow2,discard=unmap -device virtio-scsi-pci,id=scsi --enable-kvm -device scsi-hd,drive=hd -m 4096 -drive if=pflash,format=raw,readonly,file=/usr/share/ovmf/x64/OVMF_CODE.fd -drive if=pflash,format=raw,file=./OVMF_VARS.fd -vga std -drive file=Windows10ISO/Windows.iso,index=0,media=cdrom -drive file=virtio-win-0.1.173.iso,index=1,media=cdrom -no-quit + +There is a patch for the XFS kernel driver to fix the bug: https://www.spinics.net/lists/linux-xfs/msg33429.html + +On the qemu side, we’re still discussing on how to work around the bug in the 4.2 release. + +Max + +Sorry for the delay,I was busy doing my job the last two weeks. + +I use XFS V5 on both main host (5.3.7-arch1-2-ARCH) and backup host (5.3.5-arch1-1-ARCH). + +It seems I ran in the first bug that has been fixed upstream. +With git master (git clone at 18.10.) I could not reproduce the failure on my backup host. +I installed an RedHat 7.6 VM as always and the VM works without any errors. The only thing I noticed was, the first boot after installation lasts longer as with qemu 4.0.0. + +After this I checked the archlinux repositories an found in AUR the qemu-git package. I removed the official qemu packages from my main host and installed this (qemu-git 8:v4.1.0.r1699.gf9bec78137-1). +The behavior is the same as on the backup host, the VM installation works without any errors as well as additional tasks (i. e. complete the basic installation to an full desktop installation). +The last days I used the main hosts with this package for my daily work. At the end of the day I checked the filesystems from the used existing, or new created VMs and didn't found any errors. + +May be for archlinux user who needs the 4.1.0 qemu version the qemu-git package from AUR is a possible workaround. + +Claus + + +Which filesystems does this apply to? Excludes ZFS? + + +Hi Claus, + +Thanks for the info! By now we know that the XFS bug can only be triggered with aio=native (for -drive), and since you aren’t using that, you won’t hit that. + +I suppose using git master works in the meantime, but in general of course it isn’t advisable for stability. (Yes, yes, I know, right now the released version is the broken one... :() + +4.1.1 and 4.2.0 will be released soon, which fix the qemu bug. + + +Hi Wayne, + +It applies only to XFS. There are two bugs, one in qemu 4.1.0 (will be fixed in 4.1.1 and 4.2.0), and one in XFS (we will have a workaround in 4.2.0, and I hope in 4.1.1, too). + + +Max + +Can we close this ticket now? + +The QEMU project is currently considering to move its bug tracking to +another system. For this we need to know which bugs are still valid +and which could be closed already. Thus we are setting older bugs to +"Incomplete" now. + +If you still think this bug report here is valid, then please switch +the state back to "New" within the next 60 days, otherwise this report +will be marked as "Expired". Or please mark it as "Fix Released" if +the problem has been solved with a newer version of QEMU already. + +Thank you and sorry for the inconvenience. + + +[Expired for QEMU because there has been no activity for 60 days.] + diff --git a/results/classifier/zero-shot/105/assembly/1850000 b/results/classifier/zero-shot/105/assembly/1850000 new file mode 100644 index 000000000..e1c287f6e --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/1850000 @@ -0,0 +1,156 @@ +assembly: 0.865 +device: 0.859 +semantic: 0.855 +other: 0.822 +instruction: 0.814 +mistranslation: 0.796 +vnc: 0.796 +graphic: 0.788 +KVM: 0.788 +boot: 0.783 +network: 0.628 +socket: 0.560 + +4.1.0 bogus QCOW2 corruption reported after compress + +Creating a compressed image then running `qemu-img check <..>.qcow2' on said image seems to report bogus corruption in some (but not all) cases: + +Step 1. + +# qemu-img info win7-base.qcow2 +image: win7-base.qcow2 +file format: qcow2 +virtual size: 20 GiB (21474836480 bytes) +disk size: 12.2 GiB +cluster_size: 65536 +Format specific information: + compat: 1.1 + lazy refcounts: true + refcount bits: 16 + corrupt: false + +# qemu-img check win7-base.qcow2 +No errors were found on the image. +327680/327680 = 100.00% allocated, 0.00% fragmented, 0.00% compressed clusters +Image end offset: 21478375424 + +Step 2. + +# qemu-img convert -f qcow2 -O qcow2 -c win7-base.qcow2 test1-z.qcow2 + +Step 3. + +# qemu-img info test1-z.qcow2 +image: test1-z.qcow2 +file format: qcow2 +virtual size: 20 GiB (21474836480 bytes) +disk size: 5.78 GiB +cluster_size: 65536 +Format specific information: + compat: 1.1 + lazy refcounts: false + refcount bits: 16 + corrupt: false + +# qemu-img check test1-z.qcow2 +ERROR cluster 1191 refcount=1 reference=2 +ERROR cluster 1194 refcount=1 reference=4 +ERROR cluster 1195 refcount=1 reference=7 +ERROR cluster 1196 refcount=1 reference=7 +ERROR cluster 1197 refcount=1 reference=6 +ERROR cluster 1198 refcount=1 reference=4 +ERROR cluster 1199 refcount=1 reference=4 +ERROR cluster 1200 refcount=1 reference=5 +ERROR cluster 1201 refcount=1 reference=3 +<...> snip many errors +Leaked cluster 94847 refcount=3 reference=0 +Leaked cluster 94848 refcount=3 reference=0 +Leaked cluster 94849 refcount=11 reference=0 +Leaked cluster 94850 refcount=14 reference=0 + +20503 errors were found on the image. +Data may be corrupted, or further writes to the image may corrupt it. + +20503 leaked clusters were found on the image. +This means waste of disk space, but no harm to data. +197000/327680 = 60.12% allocated, 89.32% fragmented, 88.50% compressed clusters +Image end offset: 6216220672 + + +The resultant image seems to work fine in a VM when used as a backing file. + +Interestingly, if I substitute a qemu-img binary from qemu-4.0 then no errors are reported. + +# /tmp/qemu-img check test1-z.qcow2 +No errors were found on the image. +197000/327680 = 60.12% allocated, 89.32% fragmented, 88.50% compressed clusters +Image end offset: 6216220672 + +Is the image corrupted or not? I'm guessing not. + +Just in case it matters, this is ext4 fs on rotational disk. Latest Arch Linux but self compiled 4.1.0 with recent QCOW2 corruption fixes added. + +I haven't tried latest trunk but might do so if time permits. + +Current trunk still displays the problem. + +A git bisection between 4.0.0 and 4.1.0 revealed: + +b6c246942b14d3e0dec46a6c5868ed84e7dbea19 is the first bad commit +commit b6c246942b14d3e0dec46a6c5868ed84e7dbea19 +Author: Alberto Garcia <email address hidden> +Date: Fri May 10 19:22:54 2019 +0300 + + qcow2: Define and use QCOW2_COMPRESSED_SECTOR_SIZE + + When an L2 table entry points to a compressed cluster the space used + by the data is specified in 512-byte sectors. This size is independent + from BDRV_SECTOR_SIZE and is specific to the qcow2 file format. + + The QCOW2_COMPRESSED_SECTOR_SIZE constant defined in this patch makes + this explicit. + + Signed-off-by: Alberto Garcia <email address hidden> + Signed-off-by: Kevin Wolf <email address hidden> + + block/qcow2-cluster.c | 5 +++-- + block/qcow2-refcount.c | 25 ++++++++++++++----------- + block/qcow2.c | 3 ++- + block/qcow2.h | 4 ++++ + 4 files changed, 23 insertions(+), 14 deletions(-) + + + + + +There is definitely a bug in that patch, and that is that QCOW2_COMPRESSED_SECTOR_MASK is an unsigned int instead of a uint64_t (so the mask is too small). + +It looks like the bug has existed in some places before that patch (because they use ~511 as a mask), but not in others. + +This would explain why the bug is visible only for some images, namely for those with a compressed size of more than 4 GB, I presume. + + +And indeed, fixing QCOW2_COMPRESSED_SECTOR_MASK to be an unsigned long long fixes the bug. I’ll send a patch (but I’ll have to write a more simple and quicker test case first). + +Max + +Forgot to say that I sent a patch: + +https://lists.nongnu.org/archive/html/qemu-block/2019-10/msg01764.html + + +Thanks for reporting! + +Max + +Thank you Max. + +Can confirm your patch fixes my issue (qemu-img check ...) + +Not sure about those other code paths. I don't use internal snapshots and I'm not sure under which circumstances qcow2_free_any_clusters() gets exercised. + +Just for good measure, with patch applied I created another >4GB compressed image then booted it a few times and all seems fine. + +Fix has been included in QEMU v4.2: +https://git.qemu.org/?p=qemu.git;a=commitdiff;h=24552feb6ae2f615b76c2 + diff --git a/results/classifier/zero-shot/105/assembly/1852196 b/results/classifier/zero-shot/105/assembly/1852196 new file mode 100644 index 000000000..be9bb8044 --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/1852196 @@ -0,0 +1,95 @@ +assembly: 0.893 +semantic: 0.886 +instruction: 0.872 +device: 0.864 +graphic: 0.856 +mistranslation: 0.855 +socket: 0.837 +other: 0.800 +vnc: 0.759 +boot: 0.739 +KVM: 0.723 +network: 0.689 + +update edk2 submodule & binaries to edk2-stable202008 + +edk2-stable201911 will be tagged soon: + + https://github.com/tianocore/tianocore.github.io/wiki/EDK-II-Release-Planning + + https://github.com/tianocore/edk2/releases/tag/edk2-stable201911 + [upcoming link] + +It should be picked up by QEMU, after the v4.2.0 release. + +Relevant fixes / features in edk2, since edk2-stable201905 (which is +what QEMU bundles at the moment, from LP#1831477): + +- enable UEFI HTTPS Boot in ArmVirtQemu* platforms + https://bugzilla.tianocore.org/show_bug.cgi?id=1009 + (this is from edk2-stable201908) + +- fix CVE-2019-14553 (Invalid server certificate accepted in HTTPS Boot) + https://bugzilla.tianocore.org/show_bug.cgi?id=960 + +- consume OpenSSL-1.1.1d, for fixing CVE-2019-1543, CVE-2019-1552 and + CVE-2019-1563 + https://bugzilla.tianocore.org/show_bug.cgi?id=2226 + +Hi Laszlo, + +Do you have a particular reason to update the submodule *after* the v4.2.0 release? +I'd rather see QEMU 4.2 released with edk2-stable201911, as it fixes various CVE (therefore a patch for 4.2-rc4 seems acceptable to me). + + +Yes, I do have a reason for delaying this LP until after 4.2.0 is out. + +When I filed this ticket (on 2019-Nov-12), QEMU had already entered the 4.2.0 soft feature freeze (on 2019-Oct-29). Despite possible appearances, this LP is actually a feature addition -- that's why I also set "Tags: feature-request" when I filed this LP. + +The reason this is not a fix but a feature addition is the following: +- CVE-2019-14553 is irrelevant (doesn't exist) until we enable HTTPS Boot, +- we have not enabled HTTPS Boot earlier exactly because of CVE-2019-14553, +- the plan is to enable HTTPS Boot now, with CVE-2019-14553 fixed, +- so what remains are CVE-2019-1543, CVE-2019-1552 and CVE-2019-1563, which are native OpenSSL problems. + +The upstream edk2 project advanced to OpenSSL 1.1.1d because of the last point (i.e. because of those three OpenSSL CVEs). That submodule update was tracked in: + +https://bugzilla.tianocore.org/show_bug.cgi?id=2226 + +As you can see: + +(1) there was zero analysis or explanation how those OpenSSL CVEs would *actually* affect edk2 platforms, + +(2) edk2 advanced to OpenSSL 1.1.1d (on 2019-Nov-05) approximately two months after upstream OpenSSL 1.1.1d was released (on 2019-Sep-10). + +Furthermore, + +(3) all the listed CVEs are marked "low severity": + +https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-1543 +https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-1552 +https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2019-1563 + +(The first two items are declared low severity on cve.mitre.org, while the last item is declared low severity in <https://www.openssl.org/news/secadv/20190910.txt>.) + +These points (1) through (3) tell me that the edk2 advance was more or less "better safe than sorry" or "cargo cult". + +While that approach is not necessarily wrong, if you have infinite amounts of time, my capacity falls near the other end of the spectrum. If someone runs QEMU in production, they should build their firmware from source anyway -- the bundling of edk2 binaries with QEMU is a convenience. + +If you'd like to submit a QEMU patch set (just for the sake of the CVE fixes, not the HTTPS Boot feature), and are willing to make the case for getting that into 4.2-rc4, I won't block it, but I don't think it's worth the churn, to be honest. + +Thanks! +Laszlo + +Posted + +* [qemu-devel] [PATCH 00/10] edk2: adopt the edk2-stable202008 release + +http://<email address hidden> + + + +Commit a68694cd1f3. + +Released with QEMU v5.2.0. + diff --git a/results/classifier/zero-shot/105/assembly/1862167 b/results/classifier/zero-shot/105/assembly/1862167 new file mode 100644 index 000000000..f176c7e38 --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/1862167 @@ -0,0 +1,32 @@ +assembly: 0.936 +graphic: 0.779 +instruction: 0.715 +device: 0.706 +other: 0.701 +semantic: 0.633 +vnc: 0.621 +network: 0.540 +mistranslation: 0.456 +socket: 0.431 +KVM: 0.390 +boot: 0.382 + +Variation of SVE register size (qemu-user-aarch64) + +Specification of ARMv8-A SVE extention allows various values for the size of the SVE register. On the other hand, it seems that the current qemu-aarch64 supports only the maximum length of 2048 bits as the SVE register size. I am writing an assembler program for a CPU that is compliant with ARMv8-A + SVE and has a 512-bit SVE register, but when this is run with qemu-user-aarch64, a 2048-bit load / store instruction is executed This causes a segmentation fault. Shouldn't qeum-user-aarch64 have an option to specify the SVE register size? + +This is already managed by a cpu property. + +0df9142d27d5 ("target/arm/cpu64: max cpu: Introduce sve<N> properties") + +See docs/arm-cpu-features.rst + +Try "-cpu max,sve512=on". + + + +Note also that the vector length in SVE is not fixed -- you should be writing your guest code to support arbitrary vector lengths, because otherwise it will not run on all SVE-supporting CPUs. + + +Thank you for your kind advice. I'll try it. + diff --git a/results/classifier/zero-shot/105/assembly/1877136 b/results/classifier/zero-shot/105/assembly/1877136 new file mode 100644 index 000000000..462edfe50 --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/1877136 @@ -0,0 +1,77 @@ +assembly: 0.260 +vnc: 0.236 +device: 0.233 +KVM: 0.232 +semantic: 0.213 +boot: 0.190 +instruction: 0.184 +other: 0.183 +network: 0.178 +mistranslation: 0.172 +graphic: 0.156 +socket: 0.150 + +Qemu GDB Arm core registers XML description not valid for M-profile + +When trying to debug an armv7-m binary running on Qemu, GDB makes some mistakes due to mistakenly believing the target is not M-profile. + +One observable is that backtraces over signal handlers are not handled correctly -- since the special M-profile EXC_RETURN value is not recognised. That happens because GDB doesn't think the target is M-profile. + +This happens because GDB sees a reported feature set from the Qemu remote connection that includes the feature `org.gnu.gdb.arm.core`. + +As described in the GDB online docs, for "M-profile targets (e.g. Cortex-M3), the ‘org.gnu.gdb.arm.core’ feature is replaced by ‘org.gnu.gdb.arm.m-profile’" +https://sourceware.org/gdb/current/onlinedocs/gdb/ARM-Features.html + +From a scan of the Qemu source code on commit ea1329bb3a8d5cd25b70e3dbf73e7ded4d5ad756 it seems that when emulating an arm core it uses `arm-core.xml` unconditionally for `CPUClass->gdb_core_xml_file`, and that means the only feature provided is `org.gnu.gdb.arm.core`. + +Note that even though there is a command to set the architecture in GDB, setting the target architecture to an M-profile core is still not a valid workaround. +This is because the target description overrides everything in setting the `is_m` attribute within GDB. + +Reproduction of the observable: +Using the examples here https://git.linaro.org/people/peter.maydell/m-profile-tests.git/tree/ . +Build the examples, and run +``` +qemu-system-arm -s -S -no-reboot -M lm3s6965evb -m 16 -serial stdio -display none -net nic -net user,restrict=on -d guest_errors,unimp -kernel test3-kern.bin +``` + +Then in a GDB session +``` +vshcmd: > arm-none-eabi-gdb -q +(gdb) +vshcmd: > file test3-kern.elf +Reading symbols from test3-kern.elf... +(gdb) +vshcmd: > target remote localhost:1234 +Remote debugging using localhost:1234 +_start () at init-m.S:53 +53 mov r0, #0 +(gdb) +vshcmd: > show architecture +The target architecture is set automatically (currently armv7) +(gdb) +vshcmd: > break svc +Breakpoint 1 at 0x6fc: svc. (2 locations) +(gdb) +vshcmd: > cont +Continuing. + +Breakpoint 1, svc () at test3.c:16 +16 int test = SEQ(); +(gdb) +vshcmd: > bt +#0 svc () at test3.c:16 +#1 0xfffffff8 in ?? () +Backtrace stopped: previous frame identical to this frame (corrupt stack?) +(gdb) +vshcmd: > print/x $lr +$1 = 0xfffffff9 +(gdb) +``` + +Patch submitted: https://<email address hidden>/ + + +Fix now in master, will be in QEMU 5.1. + +https://git.qemu.org/?p=qemu.git;a=commitdiff;h=c888f7e0fdcc09c8600 + diff --git a/results/classifier/zero-shot/105/assembly/1882671 b/results/classifier/zero-shot/105/assembly/1882671 new file mode 100644 index 000000000..bee22f728 --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/1882671 @@ -0,0 +1,797 @@ +assembly: 0.691 +instruction: 0.677 +semantic: 0.672 +graphic: 0.619 +device: 0.601 +socket: 0.599 +boot: 0.574 +other: 0.549 +mistranslation: 0.537 +network: 0.536 +vnc: 0.471 +KVM: 0.453 + +unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled + +The version of QEMU (4.2.0) packaged for Ubuntu 20.04 hangs indefinitely at boot if an OVMF bios is used. This happens ONLY with qemu-system-x86_64. qemu-system-i386 works fine with the latest ia32 OVMF bios. + +NOTE[1]: the same identical OVMF bios works fine on QEMU 2.x packaged with Ubuntu 18.04. +NOTE[2]: reproducing the fatal bug requires *no* operating system: + + qemu-system-x86_64 -bios OVMF-pure-efi.fd + +On its window QEMU gets stuck at the very first stage: + "Guest has not initialized the display (yet)." + +NOTE[3]: QEMU gets stuck no matter if KVM is used or not. + +NOTE[4]: By adding the `-d int` option it is possible to observe that QEMU is, apparently, stuck in an endless loop of interrupts. For the first few seconds, registers' values vary quickly, but at some point they reach a final value, while the interrupt counter increments: + + 2568: v=68 e=0000 i=0 cpl=0 IP=0038:0000000007f1d225 pc=0000000007f1d225 SP=0030:0000000007f0c8d0 env->regs[R_EAX]=0000000000000000 +RAX=0000000000000000 RBX=0000000007f0c920 RCX=0000000000000000 RDX=0000000000000001 +RSI=0000000006d18798 RDI=0000000000008664 RBP=0000000000000000 RSP=0000000007f0c8d0 +R8 =0000000000000001 R9 =0000000000000089 R10=0000000000000000 R11=0000000007f2c987 +R12=0000000000000000 R13=0000000000000000 R14=0000000007087901 R15=0000000000000000 +RIP=0000000007f1d225 RFL=00000246 [---Z-P-] CPL=0 II=0 A20=1 SMM=0 HLT=0 +ES =0030 0000000000000000 ffffffff 00cf9300 DPL=0 DS [-WA] +CS =0038 0000000000000000 ffffffff 00af9a00 DPL=0 CS64 [-R-] +SS =0030 0000000000000000 ffffffff 00cf9300 DPL=0 DS [-WA] +DS =0030 0000000000000000 ffffffff 00cf9300 DPL=0 DS [-WA] +FS =0030 0000000000000000 ffffffff 00cf9300 DPL=0 DS [-WA] +GS =0030 0000000000000000 ffffffff 00cf9300 DPL=0 DS [-WA] +LDT=0000 0000000000000000 0000ffff 00008200 DPL=0 LDT +TR =0000 0000000000000000 0000ffff 00008b00 DPL=0 TSS64-busy +GDT= 00000000079eea98 00000047 +IDT= 000000000758f018 00000fff +CR0=80010033 CR2=0000000000000000 CR3=0000000007c01000 CR4=00000668 +DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 +DR6=00000000ffff0ff0 DR7=0000000000000400 +CCS=0000000000000044 CCD=0000000000000000 CCO=EFLAGS +EFER=0000000000000d00 + + +NOTE[5]: Just to better help the investigation of the bug, I'd like to remark that the issue is NOT caused by an endless loop of triple-faults. I tried with -d cpu_reset and there is NO such loop. No triple fault whatsoever. + +NOTE[6]: The OVMF version used for the test has been downloaded from: +https://www.kraxel.org/repos/jenkins/edk2/edk2.git-ovmf-x64-0-20200515.1398.g6ff7c838d0.noarch.rpm + +but the issue is the same with older OVMF versions as well. + + +Please take a look at it, as the bug is NOT a corner case. QEMU 4.2.0 cannot boot with an UEFI firmware (OVMF) while virtualizing a x86_64 machine AT ALL. + +Thank you very much, +Vladislav K. Valtchev + +Please add + + -debugcon file:debug.log -global isa-debugcon.iobase=0x402 + +to the QEMU cmdline, and attach "debug.log". Thanks. + +Thanks for the quick response, I'm attaching the debug.log file. + + +Status changed to 'Confirmed' because the bug affects multiple users. + +Vladislav, + +The OVMF debug log ends like this (with UEFI protocol GUIDs decoded as +their textual identifiers in edk2): + +> [Security] 3rd party image[6D19D18] can be loaded after EndOfDxe: PciRoot(0x0)/Pci(0x3,0x0)/Offset(0x16400,0x4B1FF). +> InstallProtocolInterface: [EfiLoadedImageProtocol] 6D187C0 +> Loading driver at 0x00006B1F000 EntryPoint=0x00006B25497 82540em.efi +> InstallProtocolInterface: [EfiLoadedImageDevicePathProtocol] 6D18498 +> ProtectUefiImageCommon - 0x6D187C0 +> - 0x0000000006B1F000 - 0x00000000000B6E60 +> InstallProtocolInterface: [EfiDriverBindingProtocol] 6B50C00 +> InstallProtocolInterface: [EfiComponentName2Protocol] 6B50BD0 +> ASSERT /home/jenkins/workspace/edk2/rpms/build/edk2-g6ff7c838d0/MdeModulePkg/Core/Dxe/Image/Image.c(1676): Image->Tpl == gEfiCurrentTpl + +This final log snippet confirms that a UEFI device driver called +"82540em.efi" is being loaded and started from the option ROM BAR of the +PCI device that is at slot 3, function 0, of the root bridge. + +When this UEFI device driver is started, it trips an assert in the +platform firmware. Namely, in the CoreStartImage() function in the +"MdeModulePkg/Core/Dxe/Image/Image.c" source file of edk2: + + // + // Image has completed. Verify the tpl is the same + // + ASSERT (Image->Tpl == gEfiCurrentTpl); + +This suggests that the "82540em.efi" driver exits its entry point +function after having raised, but not having restored, the TPL (Task +Priority Level). In other words, the symptom indicates a bug in the UEFI +driver. + +I *suspect* (but am not sure) that you are using an e1000 emulated NIC, +and the "82540em.efi" driver exposed in its oprom comes from the iPXE +project: + +src/drivers/net/intel.c: PCI_ROM ( 0x8086, 0x100e, "82540em", "82540EM", 0 ), + +Therefore I suspect a bug in the iPXE version that the Ubuntu 20.04 +upgrade brought to you. + +(I can see a number of TPL-related patches in the iPXE git history, +around Feb-Mar 2018. And QEMU loads the iPXE oprom into the emulated +NICs ROM BAR.) + +Please try installing different versions of the iPXE package on your +Ubuntu host, and re-run your test (without changing any other elements +of your setup). + +Thanks. + + +(From the UEFI executable name "82540em.efi" in the log, I initially suspected an assigned physical NIC with a buggy flashed-on oprom. But grepping the iPXE tree for "82540em" yields a match, and QEMU loads the iPXE oproms by default into the emulated NICs' ROM BARs.) + +Hi Laszlo, +thanks for investigating the problem so rapidly. + +So, I downgraded the ipxe-qemu package from 1.0.0+git-20190109.133f4c4-0ubuntu3 (focal) to 1.0.0+git-20180124.fbe8c52d-0ubuntu2 (bionic) and the problem completely disappeared. Your theory looks absolutely correct to me. + +For what it's worth, I just discovered that, even with the buggy ipxe-qemu in Focal, the OVMF distributed with QEMU itself (/usr/share/qemu/OVMF.fd) worked, but ONLY with it. I tried with multiple other versions of OVMF and all of them caused QEMU to stuck at boot, probably because of that ASSERT in the 82540em.efi driver. + +Vlad + +Hi Vlad, + +the ipxe-qemu package in Ubuntu (1.0.0+git-20190109.133f4c4-0ubuntu3) is +built with DOWNLOAD_PROTO_HTTPS enabled (in "src/config/general.h"). +According to the Ubuntu changelog, this is a new feature added in +"1.0.0+git-20190109.133f4c4-0ubuntu1". + +With DOWNLOAD_PROTO_HTTPS enabled, I can reproduce the issue locally, +with iPXE built from source at git commit 133f4c4 (which you report the +issue for), and also at current iPXE master (9ee70fb95bc2). + +The issue does not reproduce (with DOWNLOAD_PROTO_HTTPS enabled) at +commit fbe8c52d. This suggests the problem should be bisectable. + +If I disable DOWNLOAD_PROTO_HTTPS, then the problem goes away even at +133f4c4 (i.e., the issue is masked). + +I've used current edk2 master to test with (14c7ed8b51f6). + +Viewed at 133f4c4: + +The DOWNLOAD_PROTO_HTTPS feature test macro seems to result in iPXE +attempting to gather entropy. (Likely for setting up TLS connections.) +For entropy gathering, iPXE seems to use an EFI timer, and to measure +jitter across one timer tick. In this, iPXE plays some tricks with the +UEFI TPL (Task Priority Level). + +In general, iPXE seems to want to run at TPL_CALLBACK most of the time, +to mask the timer interrupt in most code locations, and drops down to +TPL_APPLICATION only when it actively wants a timer callback (for the +jitter collection, see above). + +When the iPXE driver is launched, the StartImage() UEFI boot service +takes a note of the current TPL. It is TPL_APPLICATION (value 4). Then +iPXE seems to perform the above trickery with TPL_CALLBACK & entropy +collection. Finally, after installing EfiDriverBindingProtocol and +EfiComponentName2Protocol, the iPXE driver exits (as expected from a +UEFI driver model driver -- the entry point function is only supposed to +perform some setup steps & install some protocol interfaces). At this +point, StartImage() verifies whether the TPL has been restored to the +same as it was before launching the driver. + +Unfortunately, something about the TPL manipulations in iPXE is +unbalanced, because I see the following TPL changes: + +- raise: APPLICATION (4) -> CALLBACK (8) +- raise: CALLBACK (8) -> NOTIFY (16) +- raise: NOTIFY (16) -> NOTIFY (16) +- restore: NOTIFY (16) -> NOTIFY (16) +- restore: NOTIFY (16) -> CALLBACK (8) + +Note that the final "restore: CALLBACK (8) -> APPLICATION (4)" +transition is missing, before iPXE exits. This is what StartImage() +catches and reports with the failed ASSERT(). + +So, as I mentioned, the problem is bisectable. Here's the bisection log: + +> git bisect start +> # bad: [9ee70fb95bc266885ff88be228b044a2bb226eeb] [efi] Attempt to +> # connect our driver directly if ConnectController fails +> git bisect bad 9ee70fb95bc266885ff88be228b044a2bb226eeb +> # bad: [133f4c47baef6002b2ccb4904a035cda2303c6e5] [build] Handle +> # R_X86_64_PLT32 from binutils 2.31 +> git bisect bad 133f4c47baef6002b2ccb4904a035cda2303c6e5 +> # good: [fbe8c52d0d9cdb3d6f5fe8be8edab54618becc1f] [ena] Fix spurious +> # uninitialised variable warning on older versions of gcc +> git bisect good fbe8c52d0d9cdb3d6f5fe8be8edab54618becc1f +> # bad: [bc85368cdd311fe68ffcf251e7e8e90c14f8a9dc] [librm] Ensure that +> # inline code symbols are unique +> git bisect bad bc85368cdd311fe68ffcf251e7e8e90c14f8a9dc +> # bad: [0778418e29ea16fc897fc5b6e497054f5ba86ebd] [golan] Do not +> # assume all devices are identical +> git bisect bad 0778418e29ea16fc897fc5b6e497054f5ba86ebd +> # good: [f672a27b34220865b403df519593f382859559e0] [efi] Raise TPL +> # within EFI_USB_IO_PROTOCOL entry points +> git bisect good f672a27b34220865b403df519593f382859559e0 +> # bad: [d8c500b7945e57023dde5bd0be2b0e40963315d9] [efi] Drop to +> # TPL_APPLICATION when gathering entropy +> git bisect bad d8c500b7945e57023dde5bd0be2b0e40963315d9 +> # good: [c84f9d67272beaed98f98bf308471df16340a3be] [iscsi] Parse IPv6 +> # address in root path +> git bisect good c84f9d67272beaed98f98bf308471df16340a3be +> # first bad commit: [d8c500b7945e57023dde5bd0be2b0e40963315d9] [efi] +> # Drop to TPL_APPLICATION when gathering entropy + +The bisection fingers d8c500b7945e ("[efi] Drop to TPL_APPLICATION when + gathering entropy", 2018-03-12) as first bad commit. + +Feel free to report this problem on the upstream iPXE mailing list. + +Regarding Ubuntu downstream, you should be able to work around this +issue by #undef-ing DOWNLOAD_PROTO_HTTPS again, in +"src/config/general.h" -- *minimally* in the CONFIG=qemu build(s). That +is, in the ipxe-qemu subpackage. + +That's because in a CONFIG=qemu build, you totally don't need (or even +*use*) the iPXE HTTPS infrastructure (the entropy gathering that trips +the ASSERT seems spurious to me, with CONFIG=qemu). With CONFIG=qemu, +iPXE provides the UEFI SNP (Simple Network Protocol) interface on top of +the e1000 NIC, and the crypto stuff (if any) is done by the platform +firmware (edk2 / OVMF). + + + + +Thanks for the whole investigation, Laszlo. +I sent an e-mail to <email address hidden> forwarding your analysis with +a quick summary of mine on the top, indicating the probable first bad commit. + +Vlad + +Awesome debugging Lazlo and also a really well doen bug report Vladislav - thanks! + +As Ubutnu background DOWNLOAD_PROTO_HTTPS is enabled since quite a long time in Ubuntu since bug 1025239. + +I don't know if there are better ways nowadays that might allow to disable it in future versions of Ubuntu, but for the already released versions I can't just disable it anyway. +Therefore @Vladislav if the thread you started at <email address hidden> comes to a conclusion/patch please let us know here so that I can consider backporting whatever the final solution will be. + +I was happy to contribute, Christian :-) + +I just wanted to add that after sending the e-mail to <email address hidden>, I received an automatic response explaining that my e-mail will have to be approved by a moderator because I'm not a member of that mailing list. I just hope that my e-mail won't rot there indefinitely. + + +Vlad, + +you could subscribe to ipxe-devel at <https://lists.ipxe.org/mailman/listinfo/ipxe-devel>, wait until it's confirmed (I think it's automatic, so no moderator approval is needed for subscribing), and then resend your message. You can even stay subscribed -- if you don't want to get the ipxe-devel list traffic, you can click "unsubscribe or edit options", and then "disable mail delivery" (which is *different* from unsubscribing). That way you'd remain a subscriber (so you could post at any time), but you wouldn't get the general list traffic. + +hth +Laszlo + +Christian, + +you can enable DOWNLOAD_PROTO_HTTPS in the traditional BIOS image built from iPXE, and disable it in the UEFI driver built from iPXE. You can still combine both drivers into a combined option ROM. For SeaBIOS guests, there's not going to be any change. + +For UEFI guests, see my comment#7 -- you don't need the iPXE HTTPS implementation, because OVMF already contains the edk2 HTTPS BOOT feature (assuming you build OVMF with "-D NETWORK_HTTP_BOOT_ENABLE -D NETWORK_TLS_ENABLE"). The whole point of CONFIG=qemu is to strip the UEFI build of iPXE to a mere Simple Network Protocol driver. Such a build provides only the lowest level hardware abstraction for UEFI, and the rest (IPv[46], TCP/UDP, DHCP, PXE, TFTP, HTTP(S)) are provided on top by the edk2 implementations of various standard UEFI interfaces. + +If you want to run a full-featured iPXE build on a UEFI machine (including: in an OVMF guest), you still can, of course; lots of people do that, for good reasons. But that use case is best served by the *standalone UEFI application* build of iPXE (produced at "bin-x86_64-efi/ipxe.efi" by "make"). The UEFI *driver* build of iPXE should be as minimal as possible, in comparison -- just provide SNP for the desired NIC models. + +@Lazlo thanks a lot for that awesome experience and guidance! + +Config is a bit odd in this package anyway from too many people touching it with different mindset and a lot of history. + +There is this from upstream source: src/config/general.h +Which is patched to enable DOWNLOAD_PROTO_HTTPS via debian/patches/enable-https.patch + +But there also is $ cat debian/config/general.h: +#define ROM_BANNER_TIMEOUT 0 +#define NET_PROTO_IPV6 +#define DOWNLOAD_PROTO_NFS +This goes into ./config/local/general.h at the override_dh_auto_configure stage. + +I think I should merge with the latest version from Debian, then see how I can configure HTTPS with some but not the other builds and then run everything through a bigger regression check. +I'll start a merge of the latest version in bug 1884758 and then check out disabling DOWNLOAD_PROTO_HTTPS for EFI. + +The recent edk2 builds have -DNETWORK_HTTP_BOOT_ENABLE=TRUE -DNETWORK_TLS_ENABLE +Recent as in 2020.05-2 which means >=groovy. + +For the eventual SRU to Focal things are more complex as there -DNETWORK_TLS_ENABLE was missing. + +The above was an FYI, but is should be fine as outlined by Lazlo this isn't needed as since ipxe 1.0.0+git-20180124.fbe8c52d-0ubuntu4 we use CONFIG=qemu and in comment #7 he explained that in this case "totally don't need (or even *use*) the iPXE HTTPS infrastructure (the entropy gathering that trips the ASSERT seems spurious to me, with CONFIG=qemu). With CONFIG=qemu, iPXE provides the UEFI SNP (Simple Network Protocol) interface on top of the e1000 NIC, and the crypto stuff (if any) is done by the platform firmware (edk2 / OVMF)" + +Note: the bisection result of d8c500b7945e ("[efi] Drop to TPL_APPLICATION when gathering entropy" is in <=133f4c4 but > fbe8c52d which means for Ubuntu releases that would be affected >=Disco. + +@Lazlo - are combined roms breaking your suggestion to "just disable https in efi roms"? +In the build for the efi roms it uses this at some point: + src/util/catrom.pl src/bin-i386-pcbios/82540em.rom src/bin-x86_64-efi/82540em.efirom > src/bin-combined/82540em.efirom + +So the *efi* file in ipxe-qemu will be a combined rom (since 2013). +Of these the legacy one will have HTTPS enabled and the EFI part won't. +Would you expect that this is a problem for the suggested workaround (I don't but would be glad if you would let me know what you think about it)? + +Christian, + +what you describe seems to be exactly what I propose. Namely: +- build "82540em.rom" with HTTPS enabled, +- build "82540em.efirom" with CONFIG=qemu, and HTTPS disabled, +- create a combined option ROM image from the above two, using "catrom.pl". + +Thanks +Laszlo + +Thank you Lazlo, going forward that is the process that I have it execute then. +On the SRU I'll "only" disable HTTPS on EFI roms and we can take a look if nothing else stops working but this case here would be fixed. + +-- + +One question thou - asking for help to be able to make this an SRU at soem point. + +Vladislav initially reported the test could just be "qemu -bios ovmf". +I tried this in various Ubuntu releases like: + qemu-system-x86_64 -bios /usr/share/ovmf/OVMF.fd +But without any other setup that properly gives the edk tianocore logo followed by some PXE tries and failing to boot. After the timeouts all passed it eventually gives me the UEFI shell. + +I'd have hoped this behaves differently in Bionic (working) and Focal (failing) per the initial report - but it seems to be the same. + +My point is that I fail to recreate the bug in the existing releases, which is a requirement (to be able to verify the fix) for the SRU. + +@Vladislav (or @Lazlo since you said you can reproduce it) - is there now a better way known how to trigger this. + + +I used + +qemu-system-x86_64 \ + -enable-kvm \ + -monitor stdio \ + -drive if=pflash,snapshot=on,format=raw,file=OVMF.fd \ + -global e1000.romfile=82540em.combined.rom \ + -debugcon file:debug.log \ + -global isa-debugcon.iobase=0x402 + +where "OVMF.fd" was built from edk2 at then-master (14c7ed8b51f6 -- see comment 7), and "82540em.combined.rom" is the combined oprom (with HTTPS enabled in the EFI driver too) built from the problematic iPXE version(s). + +OVMF was built with "-b DEBUG". ("-b RELEASE" would remove the ASSERT()s from the firmware modules.) + +"debug.log" captures the firmware debug output. That's the file that ends with the ASSERT failure seen in comment 4. + +Following [1] I was building my test OVMF as guided by Lazlo. +That way I was able to use the combined e1000 EFI of the Ubuntu packages vs the debug OVMF build. + +Using that I can confirm the behavior (Bionic working, Focal failing). + +$ qemu-system-x86_64 -enable-kvm -monitor stdio -drive if=pflash,snapshot=on,format=raw,file=/root/edk2/Build/OvmfX64/DEBUG_GCC5/FV/OVMF.fd -global e1000.romfile=/usr/lib/ipxe/qemu/efi-e1000.rom -debugcon file:debug.log -global isa-debugcon.iobase=0x402 + +#/usr/lib/ipxe/qemu/efi-e1000.rom is from package ipxe-qemu +#/root/edk2/Build/OvmfX64/DEBUG_GCC5/FV/OVMF.fd is the debug build, I'll attach that file here to eas repo in other places + +Symptoms when failing: +- UI never leaves the "initializing" state +- in the debug.log the bad case asserts with + ASSERT /root/edk2/MdeModulePkg/Core/Dxe/Image/Image.c(1676): Image->Tpl == gEfiCurrentTpl + +That it seems to work fine with the ovmf build that is in Focal 0~20191122.bd85bf54-2ubuntu3 makes the SRU of this less urgent IMHO. And also resolves some of my remaining confusion since I've known that EFI boots in general work - and it seems (at least in this test POV) - that only a newer or different built OVMF triggers the issue. + +[1]: https://wiki.ubuntu.com/UEFI/EDK2#Set_up_build_environment + +I beg your pardon Lazlo, but one more question on CONFIG=qemu. +Since it was introduced config=QEMU was exported for efi and legacy roms. +But it seems for legacy roms like 82540em.rom CONFIG=qemu being set or not doesn't make a difference. + +I I just look at src/config/qemu/* vs src/config/ there is a massive difference and I see it showing up in the build log, so the export has "some" effect. + +Grepping for qemu.*82540em I see + +Old: +/tmp/buildlog_ubuntu-focal-amd64.ipxe_1.0.0+git-20190109.133f4c4-0ubuntu3_BUILDING.txt:35050:gcc -DARCH=i386 -DPLATFORM=pcbios -DSECUREBOOT=0 -march=i386 -fomit-frame-pointer -fstrength-reduce -falign-jumps=1 -falign-loops=1 -falign-functions=1 -mpreferred-stack-boundary=2 -mregparm=3 -mrtd -freg-struct-return -m32 -fshort-wchar -Ui386 -Ulinux -DNVALGRIND -Iinclude -I. -Iarch/x86/include -Iarch/i386/include -Iarch/i386/include/pcbios -Os -g -ffreestanding -Wall -W -Wformat-nonliteral -fno-stack-protector -fno-dwarf2-cfi-asm -fno-exceptions -fno-unwind-tables -fno-asynchronous-unwind-tables -Wno-address -Wno-stringop-truncation -fno-PIE -no-pie -ffunction-sections -fdata-sections -include include/compiler.h -DASM_TCHAR='@' -DASM_TCHAR_OPS='@' -DCONFIG=qemu -DLOCAL_CONFIG=qemu -DOBJECT=version -DBUILD_NAME="\"82540em.rom\"" \ +/tmp/buildlog_ubuntu-focal-amd64.ipxe_1.0.0+git-20190109.133f4c4-0ubuntu3_BUILDING.txt:47807:gcc -DARCH=x86_64 -DPLATFORM=efi -DSECUREBOOT=0 -fstrength-reduce -fomit-frame-pointer -falign-jumps=1 -falign-loops=1 -falign-functions=1 -m64 -mno-mmx -mno-sse -fshort-wchar -Ui386 -Ulinux -DNVALGRIND -fpie -mno-red-zone -Iinclude -I. -Iarch/x86/include -Iarch/x86_64/include -Iarch/x86_64/include/efi -Os -g -ffreestanding -Wall -W -Wformat-nonliteral -fno-stack-protector -fno-dwarf2-cfi-asm -fno-exceptions -fno-unwind-tables -fno-asynchronous-unwind-tables -Wno-address -Wno-stringop-truncation -ffunction-sections -fdata-sections -include include/compiler.h -DASM_TCHAR='@' -DASM_TCHAR_OPS='@' -DCONFIG=qemu -DLOCAL_CONFIG=qemu -DOBJECT=version -DBUILD_NAME="\"82540em.efidrv\"" \ + +New: +ipxe_1.0.0+git-20190125.36a4c85-5ubuntu1~ppa1_amd64.build:43490:gcc -DARCH=x86_64 -DPLATFORM=efi -DSECUREBOOT=0 -fstrength-reduce -fomit-frame-pointer -falign-jumps=1 -falign-loops=1 -falign-functions=1 -m64 -mno-mmx -mno-sse -fshort-wchar -Ui386 -Ulinux -DNVALGRIND -fpie -mno-red-zone -Iinclude -I. -Iarch/x86/include -Iarch/x86_64/include -Iarch/x86_64/include/efi -Os -g -ffreestanding -Wall -W -Wformat-nonliteral -fno-stack-protector -fno-dwarf2-cfi-asm -fno-exceptions -fno-unwind-tables -fno-asynchronous-unwind-tables -Wno-address -Wno-stringop-truncation -ffunction-sections -fdata-sections -include include/compiler.h -DASM_TCHAR='@' -DASM_TCHAR_OPS='@' -DCONFIG=qemu -DLOCAL_CONFIG=qemu -DOBJECT=version -DBUILD_NAME="\"82540em.efidrv\"" \ + +But at least on the output size of 82540em.rom I see no difference. +Hence my question - does CONFIG=qemu have no effect on the legacy roms? +If it has an effect what is the recommended config for them - setting it or not? +(Right now I have no more set CONFIG=qemu for legacy roms, but since it would be a change I want to be sure it really is a no-op or at least what is recommended). + +Christian, + +> But it seems for legacy roms like 82540em.rom CONFIG=qemu being set or not doesn't make a +> difference. + +(1) That's my understanding as well; from the following original iPXE commits anyway: + +- a15c0d7e868a ("[efi] Allow user experience to be downgraded", 2015-07-22), +- a200ad462e69 ("[build] Add named configuration for qemu", 2015-07-22). + +(I played a part in those commits existing.) + +(2) However, just to be 100% sure, I recommend *not* changing the CONFIG=<whatever> setting for the traditional BIOS build. If you have had CONFIG=qemu there, then keep it. If you haven't, then don't add it now. + +What matters is that the EFI build be performed with CONFIG=qemu. + +(FWIW, last time I looked, in RHEL downstream we used to build iPXE with CONFIG=qemu covering *both* the traditional BIOS image and the EFI image. But, I don't think that's enough reason for you to diverge now from your previous BIOS image settings in Ubuntu. I don't think you should risk regressions with that. Really, what matters is that the EFI image be built with CONFIG=qemu.) + +Thanks +Laszlo + +Thanks Lazlo, I'll keep the legacy roms as CONFIG=qemu then. +I didn't plan to change this on an SRU anyway, but going forward I wanted to adapt this to be correct. Hearing that in RH you also used CONFIG=qemu covering *both* is kind of re-assuring to keep it like that for now. + +This bug was fixed in the package ipxe - 1.0.0+git-20190125.36a4c85-5ubuntu1 + +--------------- +ipxe (1.0.0+git-20190125.36a4c85-5ubuntu1) groovy; urgency=medium + + * Merge with Debian unstable (LP: #1884758). Remaining changes: + - Split grub integration from ipxe->grub-ipxe. + - d/control: add package and adapt dependencies + - d/[grub-]ipxe.install: move some files to grub-ipxe + - rename d/ipxe.post* to d/grub-ipxe-post* + [updated to match 1.0.0+git-20190125.36a4c85-5] + - d/util/check-rom-sizes, d/rules: check sizes of generated roms to avoid + accidentally breaking KVM live migration on updates/fixes. + [updated for new and dropped rom file names] + - debian/copyright: update copyright information to satisfy lintian + dep5 checks (LP: 1747071) + - Build ROMs for QEMU with CONFIG=qemu (LP: 1789319) + [updated to match 1.0.0+git-20190125.36a4c85-5 and be more verbose] + - debian/patches/handle-dhcp-nack.patch: Handle DHCP NAK and send a + re-discover. (LP 1707999) + - d/p/0005-strip-802.1Q-VLAN-0-priority-tags.patch: Strip 802.1Q VLAN 0 + priority tags; Fixes PXE when VLAN tag is 0. (LP: 1805920) + - d/tree/ipxe/etc/grub.d/20_ipxe: Make grub-ipxe work under UEFI + (LP: 1811496) + - Use ipxe.efi under UEFI + - Save default entry when iPXE is selected + - d/tree/ipxe/etc/grub.d/20_ipxe: Identify ipxe grub menu entry in + an easier way (LP: 1858374) + * Dropped changes + - new upstream release 20190109.133f4c4 [superseded by new upstream] + - new upstream release 20180124.fbe8c52d [superseded by new upstream] + - Revert to the former git snapshot 20150424.a25a16d + [superseded by new upstream] + - This brings back debian/patches/0002-Don-t-use-libiberty.patch as needed + on the older source. + - Adapt d/p/0001-rom-change-banner-timeout.diff.patch to former state to + match old source. + - drop d/p/util-elf2efi-GNU_SOURCE.patch as it was not needed on old + source + - Fix FTBFS with newer perl versions [upstream] + - d/p/0004-fix_no-pie_option.patch: correct -no-pie option + to build without pie + [ still carried before ] + - Install ne.rom as pxe-ne2k_isa.rom + - d/ipxe-qemu.install: Install ne.rom as pxe-ne2k_isa.rom. + - d/ipxe-qemu.links: compat link for ne.rom + [ no more needed, was an old xen hvmloader bug ] + - d/ipxe-qemu.links: Add compat links from /usr/share/qemu + to /usr/lib/ipxe/qemu. + [ no more needed, transition from trusty ] + - add new rom for vmxnet3 (LP 1737211) [in Debian now] + - Add e1000e firmware for qemu. (closes 884240) [in Debian now] + - d/control: ipxe-qemu breaks qemu-system-x86 <2.11 [no more needed] + - d/p/enable-https.patch: Enable HTTPS support. + [resolved per rom type in d/rules now] + * Added changes + - d/rules: only enable https on non EFI roms. This lets EFI handle https + itself and avoids breakage in TPL manipulations (LP: #1882671) + - d/util/check-rom-sizes: fix if size does exactly match + +ipxe (1.0.0+git-20190125.36a4c85-5) unstable; urgency=medium + + * Cleanup src/bin correctly. (closes: #952275) + +ipxe (1.0.0+git-20190125.36a4c85-4) unstable; urgency=medium + + * Use new source format instead of own rules. + * Use debhelper 12. + * Move ipxe.efi into /boot. (closes: #947267) + +ipxe (1.0.0+git-20190125.36a4c85-3) unstable; urgency=medium + + * Combine legacy and EFI rom again. (closes: #947024) + +ipxe (1.0.0+git-20190125.36a4c85-2) unstable; urgency=medium + + * Add Vcs information. + * Include snponly.efi. (closes: #944321) + +ipxe (1.0.0+git-20190125.36a4c85-1) unstable; urgency=medium + + * New snapshot. (closes: #832765, #906365) + * Add e1000e and vmxnet3 firmware for qemu. (closes: #884240, #868124) + + -- Christian Ehrhardt <email address hidden> Tue, 23 Jun 2020 16:23:52 +0200 + +I have prepared a merge proposals and PPA test builds for Focal/Eoan +E-MP => https://code.launchpad.net/~paelzer/ubuntu/+source/ipxe/+git/ipxe/+merge/386647 +E-PPA => https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/4126/+packages +F-MP => https://code.launchpad.net/~paelzer/ubuntu/+source/ipxe/+git/ipxe/+merge/386648 +F-PPA => https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/4127/+packages + +For Eoan/Focal we need to be sure that the OVMF builds from edk2 can really take over the HTTPS functionality. Because edk2 itself for Debian/Ubuntu only got enabled later in >=Groovy: + + edk2 (2020.05-2) unstable; urgency=medium + * Enable https boot support, thanks to Dimitri John Ledkov. LP: #1883114. + +This comes down to: +-COMMON_FLAGS = -DNETWORK_HTTP_BOOT_ENABLE=TRUE -DSECURE_BOOT_ENABLE=TRUE ++COMMON_FLAGS = -DNETWORK_HTTP_BOOT_ENABLE=TRUE -DNETWORK_TLS_ENABLE -DSECURE_BOOT_ENABLE=TRUE + +Therefore once we drop HTTPS from the ipxe-qemu combined efi roms expecting that OVMF will take over this we need to ensure this can work without above enabling being available in Eoan/Focal as well. + +/me looks for a good way to verify that as I'm unsure if the test mentioned in bug 1883114 will really proved what we need in regard to dropping https here. Maybe an actual OVMF boot via HTTPS should be set up. If there are suggestions for a good way to test that this OVMF-HTTPS-takeover works as expected I'm open to them. + +If it turns out that we need to enable it in edk2/ovmf before we can go on in ipxe/ipxe-qemu we we can upload ipxe-qemu with a versioned BREAKS to the older ovmf package (to avoid https is dropped in 'ipxe-qemu', but not yet enabled in the 'ovmf'). But if needed backporting bug 1883114 becomes a pre-req to SRU this bug here. + +We will need quite some time to ensure this isn't breaking things. +The merge proposal was reviewed and I'll upload to Focal-unapproved now. +The intention is to not have me testing in advance and then having a short time in -proposed. Instead I think for this case it will be helpful to have it in proposed early, but longer to increase the amount of 3rd-party/people testing it. + +Therefore I have added block-proposed tags right away. + +Hello Vladislav, or anyone else affected, + +Accepted ipxe into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ipxe/1.0.0+git-20190109.133f4c4-0ubuntu3.1 in a few hours, and then in the -proposed repository. + +Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. + +If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed. + +Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping! + +N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days. + +Test #1: + + * Test the attached OVMF that triggers the bug: +qemu-system-x86_64 -enable-kvm -monitor stdio -drive if=pflash,snapshot=on,format=raw,file=OVMF-14c7ed8b51f6-DEBUG-enabled.fd -global e1000.romfile=/usr/lib/ipxe/qemu/efi-e1000.rom -debugcon file:debug.log -global isa-debugcon.iobase=0x402 + +Focal as-is with ipxe-qemu 1.0.0+git-20190109.133f4c4-0ubuntu3 fails: + +ASSERT /root/edk2/MdeModulePkg/Core/Dxe/Image/Image.c(1676): Image->Tpl == gEfiCurrentTpl + +Upgrading to proposed: +root@f-ipxe:~# apt install ipxe-qemu=1.0.0+git-20190109.133f4c4-0ubuntu3.1 +Reading package lists... Done +Building dependency tree +Reading state information... Done +The following packages will be upgraded: + ipxe-qemu +1 upgraded, 0 newly installed, 0 to remove and 28 not upgraded. +Need to get 903 kB of archives. +After this operation, 1749 kB of additional disk space will be used. +Get:1 http://archive.ubuntu.com/ubuntu focal-proposed/main amd64 ipxe-qemu all 1.0.0+git-20190109.133f4c4-0ubuntu3.1 [903 kB] +Fetched 903 kB in 0s (1963 kB/s) +(Reading database ... 47652 files and directories currently installed.) +Preparing to unpack .../ipxe-qemu_1.0.0+git-20190109.133f4c4-0ubuntu3.1_all.deb ... +Unpacking ipxe-qemu (1.0.0+git-20190109.133f4c4-0ubuntu3.1) over (1.0.0+git-20190109.133f4c4-0ubuntu3) ... +Setting up ipxe-qemu (1.0.0+git-20190109.133f4c4-0ubuntu3.1) ... + +With the version from proposed the crash/assert no more happens. + +Test #2 + * We pad the rom sizes to be sure, but never the less double check + migrations between Bionic <-> Focal (known to break on size changes) + +Manual size check (can be seen in build log): +OK: efi-e1000e.rom is exactly 524288 bytes as expected +... + +Seems ok, a regression test doing cross release migrations with proposed enabled is running + + +Test #3: + + * Extra Test: HTTPS boot a uEFI guest with the efi roms from ipxe-qemu + with old/new ipxe-qmeu code. This shall ensure that OVMF can really take + over as-is or if we need bug 1883114 before we can do so. + Details TBD when I'm doing these tests + +I created a q35 guest in libvirt without a disk and set it to run in uEFI mode via OVMF. +Starting that without further setup runs into an EFI loader that can't find anything to boot. + +Start PXE over IPv4 +... +Not Found +Start HTTP Boot over IPv4 +... +Not Found +-> into interactive boot-failed menu + +As I mentioned before in comment #26 Focals EDK2 didn't have HTTPS enabled yet, only in Groovy. + + +Therefure using the OVMF of groovy and the ipxe-qemu package from Focal-proposed I set up a test. + +$ cp ovmf-groovy/usr/share/OVMF/OVMF_VARS.fd test-vars.fd +$ qemu-system-x86_64 -enable-kvm -drive if=pflash,format=raw,readonly,file=/home/ubuntu/ovmf-groovy/usr/share/OVMF/OVMF_CODE.fd -drive if=pflash,format=raw,file=test-vars.fd -monitor stdio + +We can see that in this OVMF build the OVMF device manager has the option to enroll TLScerts. But TBH I haven't ever used this setup to then HTTPS boot through EFI/OVMF. + + +I found [1] but before going through all the lengths to set this up I wonder for further regression testing I wonder if there at all was a way to get HTTPS boot working in EFI mode with: +a) https enabled /usr/lib/ipxe/qemu/efi-e1000e.rom +b) not https enabled /usr/share/OVMF/OVMF_CODE.fd + + +I'm a bit lost in all the rom/boot/https/loader options. +I beg your pardon but @Lazlo do you know if above mentioned way existed and might - now that we take https away from (a) - be regressing? +If so which way would this need to be set up to be tested? +Is [1][2] a proper way to exercise this in Focal "using the https in e1000e" or would that only work with the HTTPS enabled OVMF of groovy? + +[1]: https://en.opensuse.org/UEFI_HTTPBoot_Server_Setup +[2]: https://edk2-docs.gitbook.io/getting-started-with-uefi-https-boot-on-edk-ii/introduction + +P.S. cross release migration tests still running + + +The TPL manipulation issue in iPXE is fixed as of commit http://github.com/ipxe/ipxe/commit/2ae5d4338 + +Building an iPXE ROM with HTTPS enabled will now initialise with no problems in qemu. + +Michael + + +Hello Christian, + +For *some* form of UEFI HTTPS boot, you have to enable *at least one* of +the {edk2, iPXE} HTTPS stacks. I'm unfamiliar with the Ubuntu releases, +but my understanding is the following: + +Ubuntu release edk2 HTTPS enabled iPXE HTTPS enabled iPXE TPL regression +-------------- ------------------ ------------------ ------------------- +Bionic no <don't know> no +Focal no yes yes +Groovy yes (bug 1883114) no (this bug) masked (this bug) + +In Groovy, you can work around the iPXE TPL regression by disabling the +iPXE HTTPS stack (i.e., in the efi-e1000e option ROM). Because, you can +effectively "replace" it with the edk2 HTTPS stack in the platform +firmware (in the OVMF binary), per bug 1883114. + +In Focal, if you do the same to iPXE, you can't fall back to the edk2 +HTTPS stack in OVMF -- because bug 1883114 is out of scope for Focal, +AIUI. + +However, disabling the iPXE HTTPS stack in Focal would not cause a +regression, in my opinion. That's because in Focal you can't boot the +"OVMF + efi-e1000e" combination *at all* -- you don't get far enough in +the boot process to even *attempt* HTTPS boot (or a boot from another +kind of media, for that matter). + +Thus in Focal, no form of *UEFI boot* (HTTPS or otherwise) has ever +worked, so there's nothing to regress by disabling the iPXE HTTPS stack +in "efi-e1000e.rom". + + +Sorry about the malformed table in comment 33 -- that's not my doing. I laid it out correctly; Launchpad messed it up by squeezing whitespace. Here it is again, using dots rather than spaces. + +Ubuntu.release..edk2.HTTPS.enabled..iPXE.HTTPS.enabled..iPXE.TPL.regression +--------------..------------------..------------------..------------------- +Bionic..........no..................<don't.know>........no................. +Focal...........no..................yes.................yes................ +Groovy..........yes.(bug.1883114)...no.(this.bug).......masked.(this.bug).. + + +Regression tests completed, no issues migrating in/out of updates nor between releases due to changing sizes (That mostly covers the non EFI roms thou). + +I want to also do some more manual tests with EFI guests in that regard. + +Hi Michael, thank you for the fix, and for mentioning it here. I didn't ignore your comment 32 when I was writing what would become comments 33 and 34 -- I think we must have been writing our comments in parallel, and I simply didn't see yours. + +Christian, now you should be able to resolve this LP ticket without modifying anything on the packaging side, by backporting upstream iPXE commit 2ae5d4338. + +Thanks! + + + <loader readonly='yes' type='pflash'>/usr/share/OVMF/OVMF_CODE.fd</loader> + <nvram template='/usr/share/OVMF/OVMF_VARS.fd'>/var/lib/uvtool/libvirt/images/bionic.VARS.fd</nvram> + +makes it run eif ovmf EFI like +-drive file=/usr/share/OVMF/OVMF_CODE.fd,if=pflash,format=raw,unit=0,readonly=on -drive file=/var/lib/uvtool/libvirt/images/bionic.VARS.fd,if=pflash,format=raw,unit=1 + +Still mgriates well from Bionic -> Focal and Back +B +$ virsh migrate --unsafe --live bionic qemu+ssh://10.225.185.44/system +F +$ virsh migrate --unsafe --live bionic qemu+ssh://10.225.185.35/system +B +$ virsh console bionic +Connected to domain bionic +Escape character is ^] +Ubuntu 18.04.4 LTS bionic ttyS0 +bionic login: + +So that also works well. +The only possible option missing to be tested is the real https boot via Focals OVMF (which didn't have https enabled), but unless someone comes back explaining that there is an odd combination that could have got that going we should be good. + +Let is sit a bit more time in proposed to be sure and wait for feedback. +Then we can declare it fully verified. + +I saw your update on refresh - yeah despite feeling sort of safe on the change as-is this fix seems even better for an SRU. + +Let me get that into groovy (there the packaging change made sense, no need to turn that back). +And from there SRU it to Focal. + + +Thank you Michael and Lazlo for the work and discussion on this. + +fix as URL => https://github.com/ipxe/ipxe/commit/2ae5d4338661b65c63eb5cb1a96e5b803fe7d620 + +This bug was fixed in the package ipxe - 1.0.0+git-20190125.36a4c85-5ubuntu2 + +--------------- +ipxe (1.0.0+git-20190125.36a4c85-5ubuntu2) groovy; urgency=medium + + * d/p/lp-1882671-efi-Raise-TPL-during-driver-entry-point.patch: fix the + formerly avoided efi TPL issues (LP: #1882671) + + -- Christian Ehrhardt <email address hidden> Thu, 16 Jul 2020 14:36:54 +0200 + +Fix uploaded for SRU to focal-unapproved. + +Hello Vladislav, or anyone else affected, + +Accepted ipxe into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/ipxe/1.0.0+git-20190109.133f4c4-0ubuntu3.2 in a few hours, and then in the -proposed repository. + +Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users. + +If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed. + +Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping! + +N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days. + +Testing the actual case: +$ dpkg -S /usr/lib/ipxe/qemu/efi-e1000.rom +ipxe-qemu: /usr/lib/ipxe/qemu/efi-e1000.rom + +root@f-ipxe:~# apt-cache policy ipxe-qemu +ipxe-qemu: + Installed: 1.0.0+git-20190109.133f4c4-0ubuntu3.2 + Candidate: 1.0.0+git-20190109.133f4c4-0ubuntu3.2 + Version table: + *** 1.0.0+git-20190109.133f4c4-0ubuntu3.2 500 + 500 http://archive.ubuntu.com/ubuntu focal-proposed/main amd64 Packages + 100 /var/lib/dpkg/status + 1.0.0+git-20190109.133f4c4-0ubuntu3.2 500 + 500 http://ppa.launchpad.net/ci-train-ppa-service/4156/ubuntu focal/main amd64 Packages + 1.0.0+git-20190109.133f4c4-0ubuntu3 500 + 500 http://archive.ubuntu.com/ubuntu focal/main amd64 Packages + +$ qemu-system-x86_64 -enable-kvm -monitor stdio -drive if=pflash,snapshot=on,format=raw,file=OVMF-14c7ed8b51f6-DEBUG-enabled.fd -global e1000.romfile=/usr/lib/ipxe/qemu/efi-e1000.rom -debugcon file:debug.log -global isa-debugcon.iobase=0x402 + +With the fix this now boots through to the actual EFI prompt and tries to initialize boot options. The log file no more lists the assertion. + +Testing the sizes went well, they are int he right size buckets as they were before (no major change by the patch). + +Note: We have not touched the HTTPs capability in the SRU which makes this much saver in this try#2. Therefore also the re-testing of these isn't needed this time. + +This overall LGTM and was in proposed for an extra-while without negative feedback, setting verification-done. + +This bug was fixed in the package ipxe - 1.0.0+git-20190109.133f4c4-0ubuntu3.2 + +--------------- +ipxe (1.0.0+git-20190109.133f4c4-0ubuntu3.2) focal; urgency=medium + + * Revert the changes of the non released version + 1.0.0+git-20190109.133f4c4-0ubuntu3.1 as there is a less impactful + fix more suited for an SRU. + * d/p/lp-1882671-efi-Raise-TPL-during-driver-entry-point.patch: fix the + formerly avoided efi TPL issues (LP: #1882671) + +ipxe (1.0.0+git-20190109.133f4c4-0ubuntu3.1) focal; urgency=medium + + * only enable https on non EFI roms. This lets EFI/OVMF handle https + itself and avoids breakage in TPL manipulations (LP: 1882671) + - d/p/enable-https.patch: drop old global way to Enable HTTPS support + - d/rules: enable https on non EFI roms. + - d/util/check-rom-sizes: fix if size does exactly match + + -- Christian Ehrhardt <email address hidden> Thu, 16 Jul 2020 16:51:30 +0200 + +The verification of the Stable Release Update for ipxe has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions. + +Per comment #32, fixed in upstream iPXE commit 2ae5d4338, setting ticket status for iPXE to "Fix Committed". + diff --git a/results/classifier/zero-shot/105/assembly/1883784 b/results/classifier/zero-shot/105/assembly/1883784 new file mode 100644 index 000000000..f5c17182f --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/1883784 @@ -0,0 +1,61 @@ +assembly: 0.972 +other: 0.911 +instruction: 0.906 +graphic: 0.881 +semantic: 0.818 +device: 0.696 +mistranslation: 0.631 +network: 0.625 +KVM: 0.570 +vnc: 0.551 +boot: 0.516 +socket: 0.469 + +[ppc64le] qemu behavior differs from ppc64le hardware + +I have some code which passes my test suite on PPC64LE hardware when compiled with GCC 10, but the saem binary fails with both qemu-ppc64le 4.2 (on Fedora 32) and qemu-ppc64le-static 5.0.0 (Debian testing). + +I'm not getting any errors about illegal instructions or anything, like that; the results are just silently different on qemu. + +I've generated a reduced test case, which is attached along with the binaries (both are the same code, one is just statically linked). They should execute successufully on PPC64LE hardware, but on qemu they hit a __builtin_abort (because the computed value doesn't match the expected value). + +Without being familiar with PPC assembly I'm not sure what else I can do, but if there is anything please let me know. + + + +Did you try to run it in a qemu-system-ppc64 guest? +It would help to know if it is a tcg or a linux-user bug. + +I just ran the provided binaries on a qemu-system-ppc64 version 5.0-5 from Debian Bullseye and they also aborted there + +The QEMU project is currently moving its bug tracking to another system. +For this we need to know which bugs are still valid and which could be +closed already. Thus we are setting older bugs to "Incomplete" now. + +If the bug has already been fixed in the latest upstream version of QEMU, +then please close this ticket as "Fix released". + +If it is not fixed yet and you think that this bug report here is still +valid, then you have two options: + +1) If you already have an account on gitlab.com, please open a new ticket +for this problem in our new tracker here: + + https://gitlab.com/qemu-project/qemu/-/issues + +and then close this ticket here on Launchpad (or let it expire auto- +matically after 60 days). Please mention the URL of this bug ticket on +Launchpad in the new ticket on GitLab. + +2) If you don't have an account on gitlab.com and don't intend to get +one, but still would like to keep this ticket opened, then please switch +the state back to "New" within the next 60 days (otherwise it will get +closed as "Expired"). We will then eventually migrate the ticket auto- +matically to the new system (but you won't be the reporter of the bug +in the new system and thus won't get notified on changes anymore). + +Thank you and sorry for the inconvenience. + + +[Expired for QEMU because there has been no activity for 60 days.] + diff --git a/results/classifier/zero-shot/105/assembly/2013 b/results/classifier/zero-shot/105/assembly/2013 new file mode 100644 index 000000000..167a54c76 --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/2013 @@ -0,0 +1,91 @@ +assembly: 0.918 +socket: 0.900 +vnc: 0.900 +boot: 0.881 +device: 0.875 +instruction: 0.870 +KVM: 0.861 +semantic: 0.851 +graphic: 0.827 +other: 0.822 +network: 0.803 +mistranslation: 0.707 + +The avocado test replay_kernel.py:ReplayKernelNormal.test_mips64el_malta is unreliable +Description of problem: +This test keeps hanging on CI +Steps to reproduce: +Run the test on GitLab's CI infrastructure and it will hang on replay. Examples: https://gitlab.com/stsquad/qemu/-/jobs/5664260736 +Additional information: +Excerpt from log: + +``` +18:02:49 DEBUG| Transitioning from 'Runstate.CONNECTING' to 'Runstate.RUNNING'. +18:02:49 DEBUG| Opening console file +18:02:49 DEBUG| Opening console socket +18:02:49 DEBUG| [ 0.000000] Initializing cgroup subsys cpuset +18:02:49 DEBUG| [ 0.000000] Initializing cgroup subsys cpu +18:02:49 DEBUG| [ 0.000000] Linux version 2.6.32-5-5kc-malta (Debian 2.6.32-48) (ben@decadent.org.uk) (gcc version 4.3.5 (Debian 4.3.5-4) ) #1 Fri Feb 15 21:38:11 UTC 2013 +18:02:49 DEBUG| [ 0.000000] +18:02:49 DEBUG| [ 0.000000] LINUX started... +18:02:49 DEBUG| [ 0.000000] bootconsole [early0] enabled +18:02:49 DEBUG| [ 0.000000] CPU revision is: 000182a0 (MIPS 20Kc) +18:02:49 DEBUG| [ 0.000000] FPU revision is: 000f8200 +18:02:49 DEBUG| [ 0.000000] Checking for the multiply/shift bug... no. +18:02:49 DEBUG| [ 0.000000] Checking for the daddiu bug... no. +18:02:49 DEBUG| [ 0.000000] Determined physical RAM map: +18:02:49 DEBUG| [ 0.000000] memory: 0000000000001000 @ 0000000000000000 (reserved) +18:02:49 DEBUG| [ 0.000000] memory: 00000000000ef000 @ 0000000000001000 (ROM data) +18:02:49 DEBUG| [ 0.000000] memory: 0000000000659000 @ 00000000000f0000 (reserved) +18:02:49 DEBUG| [ 0.000000] memory: 00000000078b7000 @ 0000000000749000 (usable) +18:02:49 DEBUG| [ 0.000000] Wasting 104440 bytes for tracking 1865 unused pages +18:02:49 DEBUG| [ 0.000000] Initrd not found or empty - disabling initrd +18:02:49 DEBUG| [ 0.000000] Zone PFN ranges: +18:02:49 DEBUG| [ 0.000000] DMA 0x00000000 -> 0x00001000 +18:02:49 DEBUG| [ 0.000000] Normal 0x00001000 -> 0x00008000 +18:02:49 DEBUG| [ 0.000000] Movable zone start PFN for each node +18:02:49 DEBUG| [ 0.000000] early_node_map[1] active PFN ranges +18:02:49 DEBUG| [ 0.000000] 0: 0x00000000 -> 0x00008000 +18:02:49 DEBUG| [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 32320 +18:02:49 DEBUG| [ 0.000000] Kernel command line: printk.time=1 panic=-1 console=ttyS0 +18:02:49 DEBUG| Shutting down VM appliance; timeout=30 +18:02:49 DEBUG| Attempting graceful termination +18:02:49 DEBUG| Closing console file +18:02:49 DEBUG| Closing console socket +18:02:49 DEBUG| Politely asking QEMU to terminate +... + +18:02:49 DEBUG| Transitioning from 'Runstate.CONNECTING' to 'Runstate.RUNNING'. +18:02:49 DEBUG| Opening console file +18:02:49 DEBUG| Opening console socket +18:02:49 DEBUG| [ 0.000000] Initializing cgroup subsys cpuset +18:02:49 DEBUG| [ 0.000000] Initializing cgroup subsys cpu +18:02:49 DEBUG| [ 0.000000] Linux version 2.6.32-5-5kc-malta (Debian 2.6.32-48) (ben@decadent.org.uk) (gcc version 4.3.5 (Debian 4.3.5-4) ) #1 Fri Feb 15 21:38:11 UTC 2013 +18:02:49 DEBUG| [ 0.000000] +18:02:49 DEBUG| [ 0.000000] LINUX started... +18:02:49 DEBUG| [ 0.000000] bootconsole [early0] enabled +18:02:49 DEBUG| [ 0.000000] CPU revision is: 000182a0 (MIPS 20Kc) +18:02:49 DEBUG| [ 0.000000] FPU revision is: 000f8200 +18:02:49 DEBUG| [ 0.000000] Checking for the multiply/shift bug... no. +18:02:49 DEBUG| [ 0.000000] Checking for the daddiu bug... no. +18:02:49 DEBUG| [ 0.000000] Determined physical RAM map: +18:02:49 DEBUG| [ 0.000000] memory: 0000000000001000 @ 0000000000000000 (reserved) +18:02:49 DEBUG| [ 0.000000] memory: 00000000000ef000 @ 0000000000001000 (ROM data) +18:02:49 DEBUG| [ 0.000000] memory: 0000000000659000 @ 00000000000f0000 (reserved) +18:02:49 DEBUG| [ 0.000000] m +18:04:48 ERROR| +18:04:48 ERROR| Reproduced traceback from: /builds/stsquad/qemu/build/pyvenv/lib/python3.10/site-packages/avocado/core/test.py:770 +18:04:48 ERROR| Traceback (most recent call last): +18:04:48 ERROR| File "/builds/stsquad/qemu/build/tests/avocado/replay_kernel.py", line 147, in test_mips64el_malta +18:04:48 ERROR| self.run_rr(kernel_path, kernel_command_line, console_pattern, shift=5) +18:04:48 ERROR| File "/builds/stsquad/qemu/build/tests/avocado/replay_kernel.py", line 78, in run_rr +18:04:48 ERROR| t2 = self.run_vm(kernel_path, kernel_command_line, console_pattern, +18:04:48 ERROR| File "/builds/stsquad/qemu/build/tests/avocado/replay_kernel.py", line 61, in run_vm +18:04:48 ERROR| self.wait_for_console_pattern(console_pattern, vm) +18:04:48 ERROR| File "/builds/stsquad/qemu/build/tests/avocado/boot_linux_console.py", line 52, in wait_for_console_pattern +18:04:48 ERROR| wait_for_console_pattern(self, success_message, +18:04:48 ERROR| File "/builds/stsquad/qemu/build/tests/avocado/avocado_qemu/__init__.py", line 199, in wait_for_console_pattern +18:04:48 ERROR| _console_interaction(test, success_message, failure_message, None, vm=vm) +18:04:48 ERROR| File "/builds/stsquad/qemu/build/tests/avocado/avocado_qemu/__init__.py", line 148, in _console_interaction +18:04:48 ERROR| msg = console.readline().decode().strip() +``` diff --git a/results/classifier/zero-shot/105/assembly/2180 b/results/classifier/zero-shot/105/assembly/2180 new file mode 100644 index 000000000..5cc18e796 --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/2180 @@ -0,0 +1,49 @@ +assembly: 0.960 +instruction: 0.823 +device: 0.808 +other: 0.766 +graphic: 0.744 +socket: 0.738 +KVM: 0.663 +vnc: 0.656 +boot: 0.619 +network: 0.609 +semantic: 0.550 +mistranslation: 0.301 + +QEMU crashes when an interrupt is triggered whose descriptor is not in physical memory +Description of problem: +When an interrupt is triggered whose descriptor is mapped but not in physical memory, QEMU crashes with the following message: +``` +** +ERROR:../system/cpus.c:524:bql_lock_impl: assertion failed: (!bql_locked()) +Bail out! ERROR:../system/cpus.c:524:bql_lock_impl: assertion failed: (!bql_locked()) +Aborted (core dumped) +``` + +The given code triggers the bug by moving the IDT's base address, but it can also be triggered by any other method of moving the IDT's physical memory location, f.ex paging. With KVM enabled, this specific example loops forever instead of crashing, but if the code is altered to use paging, an internal KVM error is reported and the VM is paused. +Steps to reproduce: +1. Assemble the code listed below using NASM: `nasm test.asm -o test.bin` +2. Run the code using `qemu-system-i386 -drive format=raw,file=test.bin`. Note that the given code only triggers the bug if the guest has 2 gigabytes or less of physical memory. +3. QEMU crashes. +Additional information: +NASM assembly of the code used: +``` +bits 16 +org 0x7c00 + +_start: + ; Disable interrupts and load new IDT + cli + o32 lidt [idtdesc] + ; Descriptor for INT 0 is in nonexistent physical memory, which crashes QEMU. + int 0x00 + +idtdesc: + dw 0x3ff ; Limit: 1 KiB for IDT + dd 0x80000000 ; Base: 2 GiB + +; Like most BIOSes, SeaBIOS requires this magic number to boot +times 510-($-$$) db 0 +dw 0xaa55 +``` diff --git a/results/classifier/zero-shot/105/assembly/2186 b/results/classifier/zero-shot/105/assembly/2186 new file mode 100644 index 000000000..2483d2d0e --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/2186 @@ -0,0 +1,47 @@ +assembly: 0.966 +instruction: 0.904 +graphic: 0.763 +device: 0.739 +other: 0.725 +socket: 0.587 +semantic: 0.570 +vnc: 0.566 +network: 0.501 +mistranslation: 0.395 +boot: 0.358 +KVM: 0.081 + +riscv virt pflash0 writes not supported +Description of problem: +I am using GDB to debug some Firmware related stuff. At some point in the execution my BIOS/Firmware writes into some global variable (at 0x2000525C) inside the .bss section which is linked to be inside the memory mapped pflash0. But when I step forward with GDB to the exact location where the store instruction (sw) is executed, QEMU prints the following: +``` +pflash_write: Unimplemented flash cmd sequence (offset 000000000000525c, wcycle 0x0 cmd 0x0 value 0x1) +``` +According to the top of `hw/block/pflash_cfi01.c` Flash writes are supported. I was also under the impression that the flash is memory mapped, but maybe that is not true? I am probably missing something here so it would be nice if someone could point me in the right direction. I would also gladly contribute if there is something missing in the riscv virt target. + +I made a simple program to more easily reproduce this: +``` +.section .text +.global _start +_start: + lui a5, 0x20000 + li a4, 5 + sw a4, 24(a5) + +``` +results in QEMU error msg: +``` +pflash_write: Unimplemented flash cmd sequence (offset 0000000000000018, wcycle 0x0 cmd 0x0 value 0x5) +``` +Steps to reproduce: +1. compile above assembly program like this: +``` +riscv64-unknown-elf-gcc -nostdlib -O0 bios.S +riscv64-unknown-elf-objcopy -O binary a.out +truncate -s 33554432 a.out +``` +2. start QEMU like this: +``` +qemu-system-riscv64 -M virt -bios none -drive if=pflash,format=raw,unit=0,file=a.out -nographic -d unimp +``` +3. notice the error message printed by QEMU diff --git a/results/classifier/zero-shot/105/assembly/2303 b/results/classifier/zero-shot/105/assembly/2303 new file mode 100644 index 000000000..6a9dde8c9 --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/2303 @@ -0,0 +1,84 @@ +assembly: 0.939 +graphic: 0.892 +other: 0.881 +device: 0.859 +semantic: 0.856 +instruction: 0.846 +vnc: 0.846 +KVM: 0.842 +socket: 0.831 +mistranslation: 0.807 +boot: 0.771 +network: 0.705 + +Multiple displays configuration supports +Additional information: +The following patch is a quick "hack" to make it work + +```patch + +From 18ad5058a18fa9f6db2c0c3058e25989908d95bb Mon Sep 17 00:00:00 2001 +From: Sergio Lopez <slp@redhat.com> +Date: Fri, 23 Jun 2023 13:15:15 +0200 +Subject: [PATCH 6/8] HACK: Set static resolutions for the VM + +--- + hw/display/virtio-gpu-base.c | 10 +++++++++- + ui/gtk.c | 6 ++++-- + 2 files changed, 13 insertions(+), 3 deletions(-) + +diff --git a/hw/display/virtio-gpu-base.c b/hw/display/virtio-gpu-base.c +index a29f191aa8..b1ccfa17b7 100644 +--- a/hw/display/virtio-gpu-base.c ++++ b/hw/display/virtio-gpu-base.c +@@ -47,6 +47,7 @@ virtio_gpu_base_fill_display_info(VirtIOGPUBase *g, + dpy_info->pmodes[i].enabled = 1; + dpy_info->pmodes[i].r.width = cpu_to_le32(g->req_state[i].width); + dpy_info->pmodes[i].r.height = cpu_to_le32(g->req_state[i].height); ++ fprintf(stderr, "display %d: %dx%d\n", i, dpy_info->pmodes[i].r.width, dpy_info->pmodes[i].r.height); + } + } + } +@@ -63,14 +64,17 @@ static void virtio_gpu_text_update(void *opaque, console_ch_t *chardata) + { + } + ++#if 0 + static void virtio_gpu_notify_event(VirtIOGPUBase *g, uint32_t event_type) + { + g->virtio_config.events_read |= event_type; + virtio_notify_config(&g->parent_obj); + } ++#endif + + static void virtio_gpu_ui_info(void *opaque, uint32_t idx, QemuUIInfo *info) + { ++#if 0 + VirtIOGPUBase *g = opaque; + + if (idx >= g->conf.max_outputs) { +@@ -94,6 +98,7 @@ static void virtio_gpu_ui_info(void *opaque, uint32_t idx, QemuUIInfo *info) + /* send event to guest */ + virtio_gpu_notify_event(g, VIRTIO_GPU_EVENT_DISPLAY); + return; ++#endif + } + + static void +@@ -186,11 +191,14 @@ virtio_gpu_base_device_realize(DeviceState *qdev, + virtio_add_queue(vdev, 16, cursor_cb); + } + +- g->enabled_output_bitmask = 1; ++ g->enabled_output_bitmask = 3; + + g->req_state[0].width = g->conf.xres; + g->req_state[0].height = g->conf.yres; + ++ g->req_state[1].width = 800; ++ g->req_state[1].height = 600; ++ + g->hw_ops = &virtio_gpu_ops; + for (i = 0; i < g->conf.max_outputs; i++) { + g->scanout[i].con = +``` diff --git a/results/classifier/zero-shot/105/assembly/2463 b/results/classifier/zero-shot/105/assembly/2463 new file mode 100644 index 000000000..410f50fcd --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/2463 @@ -0,0 +1,22 @@ +assembly: 0.829 +device: 0.760 +graphic: 0.752 +instruction: 0.594 +semantic: 0.578 +socket: 0.454 +boot: 0.363 +mistranslation: 0.360 +network: 0.338 +other: 0.326 +vnc: 0.307 +KVM: 0.091 + +allow sifive_e to use more RAM +Description of problem: +For users like me that are still learning RISC bare-metal assembly, searching online you will find many tutorials and examples using sifive_e with Qemu, so it is the easy way to get started. + +I quickly ran into crashes with my tests because I did not realize that sifive_e is limited to 16K of RAM. +I realize the 16K limit is hard coded so that it matches the real hardware, but that makes it very hard to run a variety of tests. +Additional information: +My fork of Qemu changes sifive_e to allow 256MB. +https://github.com/panjea/qemu/commit/97cb89d778ebe3407a969b8282e2e7adb4be2971 diff --git a/results/classifier/zero-shot/105/assembly/2677 b/results/classifier/zero-shot/105/assembly/2677 new file mode 100644 index 000000000..51b718506 --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/2677 @@ -0,0 +1,14 @@ +assembly: 0.335 +device: 0.272 +KVM: 0.229 +vnc: 0.207 +boot: 0.197 +semantic: 0.166 +network: 0.138 +mistranslation: 0.122 +graphic: 0.113 +instruction: 0.093 +other: 0.087 +socket: 0.071 + +edit doc on building diff --git a/results/classifier/zero-shot/105/assembly/2871 b/results/classifier/zero-shot/105/assembly/2871 new file mode 100644 index 000000000..07c8f71a1 --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/2871 @@ -0,0 +1,14 @@ +assembly: 0.834 +device: 0.776 +mistranslation: 0.637 +other: 0.613 +instruction: 0.512 +graphic: 0.488 +semantic: 0.394 +network: 0.353 +boot: 0.187 +vnc: 0.064 +KVM: 0.042 +socket: 0.034 + +loongarch64: unknown register name 'f0' in asm diff --git a/results/classifier/zero-shot/105/assembly/494 b/results/classifier/zero-shot/105/assembly/494 new file mode 100644 index 000000000..ed8669272 --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/494 @@ -0,0 +1,14 @@ +assembly: 0.840 +device: 0.742 +graphic: 0.638 +instruction: 0.629 +semantic: 0.151 +network: 0.138 +mistranslation: 0.092 +boot: 0.043 +socket: 0.028 +vnc: 0.025 +other: 0.021 +KVM: 0.002 + +cmake crashes on qemu-alpha-user with Illegal Instruction diff --git a/results/classifier/zero-shot/105/assembly/536 b/results/classifier/zero-shot/105/assembly/536 new file mode 100644 index 000000000..82f645095 --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/536 @@ -0,0 +1,14 @@ +assembly: 0.987 +instruction: 0.850 +device: 0.689 +graphic: 0.532 +network: 0.240 +semantic: 0.135 +mistranslation: 0.122 +boot: 0.120 +other: 0.076 +vnc: 0.046 +socket: 0.018 +KVM: 0.009 + +Null-ptr dereference in ich9_apm_ctrl_changed diff --git a/results/classifier/zero-shot/105/assembly/710 b/results/classifier/zero-shot/105/assembly/710 new file mode 100644 index 000000000..b79fa194c --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/710 @@ -0,0 +1,14 @@ +assembly: 0.819 +instruction: 0.771 +device: 0.709 +other: 0.623 +graphic: 0.448 +network: 0.319 +mistranslation: 0.238 +semantic: 0.205 +KVM: 0.139 +boot: 0.105 +socket: 0.100 +vnc: 0.024 + +maybe-uninitialized warning building target/m68k/ with -O3 diff --git a/results/classifier/zero-shot/105/assembly/811683 b/results/classifier/zero-shot/105/assembly/811683 new file mode 100644 index 000000000..dc859c53f --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/811683 @@ -0,0 +1,319 @@ +assembly: 0.912 +other: 0.911 +semantic: 0.906 +device: 0.904 +instruction: 0.901 +graphic: 0.891 +socket: 0.887 +boot: 0.848 +KVM: 0.846 +mistranslation: 0.842 +vnc: 0.841 +network: 0.832 + +7400,7410,7450 cpus vector have wrong exception prefix at reset + +I have a proprietary ROM implementing system calls that are executed via the 'SC' instruction. + +I use qemu-0.14.1, + +qemu-system-ppc -M prep -cpu $CPU -bios my_bios -kernel my_kernel + +That works fine on a 604 (CPU=0x00040103) - but does not on an emulated 7400 (CPU=0x000c0209) or 7450 (CPU=0x80000201). I found that the emulator jumps to 0x00000c00 instead of 0xfff00c00. +Probably this is due to a wrong setting in target-ppc/translate_init.c: + +init_excp_604() correctly sets env->hreset_vector=0xfff00000UL; + +but + +init_excp_7400() says env->hreset_vector=0x00000000UL; + +which seems wrong. (the 7400 manual says a hard-reset jumps initializes the +prefix to 0xfff00000.) + +Likewise, init_excp_7450() (and probably other, related CPUs) are wrong. + +Indeed, when I change the value in init_excp_7400() to 0xfff00000UL then +everything works as expected for me. + +Hi, + +Am 16.07.2011 um 23:49 schrieb till: + +> I have a proprietary ROM implementing system calls that are executed +> via +> the 'SC' instruction. +> +> I use qemu-0.14.1, +> +> qemu-system-ppc -M prep -cpu $CPU -bios my_bios -kernel my_kernel +> +> That works fine on a 604 (CPU=0x00040103) - but does not on an +> emulated 7400 (CPU=0x000c0209) or 7450 (CPU=0x80000201). I found +> that the emulator jumps to 0x00000c00 instead of 0xfff00c00. +> Probably this is due to a wrong setting in target-ppc/ +> translate_init.c: +> +> init_excp_604() correctly sets env->hreset_vector=0xfff00000UL; +> +> but +> +> init_excp_7400() says env->hreset_vector=0x00000000UL; +> +> which seems wrong. (the 7400 manual says a hard-reset jumps +> initializes the +> prefix to 0xfff00000.) + +Do you have a link to a spec saying so? Should be trivial to change +then. + +> Likewise, init_excp_7450() (and probably other, related CPUs) are +> wrong. +> +> Indeed, when I change the value in init_excp_7400() to 0xfff00000UL +> then +> everything works as expected for me. +> +> ** Affects: qemu +> Importance: Undecided +> Status: New + +> Bug description: +> I have a proprietary ROM implementing system calls that are executed +> via the 'SC' instruction. +> +> I use qemu-0.14.1, +> +> qemu-system-ppc -M prep -cpu $CPU -bios my_bios -kernel my_kernel + +We are currently in the process of revamping the PReP machine you are +using above. Is your BIOS available publicly so that we can test we +don't break anything for you? + +Andreas + + + +On 18.07.2011, at 00:34, Andreas Färber wrote: + +> Hi, +> +> Am 16.07.2011 um 23:49 schrieb till: +> +>> I have a proprietary ROM implementing system calls that are executed via +>> the 'SC' instruction. +>> +>> I use qemu-0.14.1, +>> +>> qemu-system-ppc -M prep -cpu $CPU -bios my_bios -kernel my_kernel +>> +>> That works fine on a 604 (CPU=0x00040103) - but does not on an emulated 7400 (CPU=0x000c0209) or 7450 (CPU=0x80000201). I found that the emulator jumps to 0x00000c00 instead of 0xfff00c00. +>> Probably this is due to a wrong setting in target-ppc/translate_init.c: +>> +>> init_excp_604() correctly sets env->hreset_vector=0xfff00000UL; +>> +>> but +>> +>> init_excp_7400() says env->hreset_vector=0x00000000UL; +>> +>> which seems wrong. (the 7400 manual says a hard-reset jumps initializes the +>> prefix to 0xfff00000.) +> +> Do you have a link to a spec saying so? Should be trivial to change then. + +According to MPC7450UM.pdf: + +MSR Bit Settings + +Bit: 25 +Name: IP + +Exception prefix. The setting of this bit specifies whether an exception vector offset is prepended with Fs or 0s. In the following description, nnnnn is the offset of the exception. + + 0 Exceptions are vectored to the physical address 0x000n_nnnn. + 1 Exceptions are vectored to the physical address 0xFFFn_nnnn. + +[...] + +9.9.1 Reset Inputs + +The MPC7450 has two reset inputs, described as follows: +• HRESET (hard reset)—The HRESET signal is used for power-on reset sequences, or for situations in which the MPC7450 must go through the entire cold start sequence of internal hardware initialization. The MPC7450 will initiate burst transactions after power-on reset in 60x bus mode. +• SRESET (soft reset)—The soft reset input provides warm reset capability. This input can be used to avoid forcing the MPC7450 to complete the cold start sequence. +When either reset input negates, the processor attempts to fetch code from the system reset exception vector. The vector is located at offset 0x00100 from the exception prefix (MSR[IP]). + +----> The MSR[IP] bit is set when HRESET negates. + + +So the correct implementation would be to set hreset_vector to 0xfff00000, but also set MSR_IP and clear hreset_vector when MSR_IP gets modified. + +I'll happily take patches :). + + +Alex + + + +Google for MPC7450UM.pdf and MPC7410UM.pdf. These two documents cover the + +7441, 7445, 7451, 7455, 7457, 7447, 7448 and the 7410 and 7400 CPUs, respectively. + +For all these, Alex' description applies. However, (and I made a mistake in my original post), +the setting affected is + +env->hreset_excp_prefix = 0xfff00000UL; + +in addition, hreset_vector should be: + +env->hreset_vector = 0x00000100UL; + +NOTE - I believe the other points raised by Alex (initialize MSR[IP] -- which BTW is called MSR_EP in qemu -- and switching the exception prefix when MSR[IP] is changed) are already correctly handled, see: + +target-ppc/helper.c: cpu_reset() +target-ppc/helper-hreg.h: hreg_store_msr() + +Should I post a patch to the mailing-list? + +Hi Andreas. + +I posted a reply to the bug database. Regarding my 'bios' - it is really +nothing. +I need it to boot RTEMS. It just mocks up a minimal residual and jumps to +the kernel load address. +You can take a look at + +http://www.rtems.org/viewvc/rtems/c/src/lib/libbsp/powerpc/shared/bootloader/ + +The stuff that goes into the dummy 'bios' is qemu_fakerom.S and +qemu_fakeres.c + +Regards +- Till + +On 07/17/2011 05:34 PM, Andreas Färber wrote: +> Hi, +> +> Am 16.07.2011 um 23:49 schrieb till: +> +>> I have a proprietary ROM implementing system calls that are executed +>> via +>> the 'SC' instruction. +>> +>> I use qemu-0.14.1, +>> +>> qemu-system-ppc -M prep -cpu $CPU -bios my_bios -kernel my_kernel +>> +>> That works fine on a 604 (CPU=0x00040103) - but does not on an +>> emulated 7400 (CPU=0x000c0209) or 7450 (CPU=0x80000201). I found +>> that the emulator jumps to 0x00000c00 instead of 0xfff00c00. +>> Probably this is due to a wrong setting in target-ppc/ +>> translate_init.c: +>> +>> init_excp_604() correctly sets env->hreset_vector=0xfff00000UL; +>> +>> but +>> +>> init_excp_7400() says env->hreset_vector=0x00000000UL; +>> +>> which seems wrong. (the 7400 manual says a hard-reset jumps +>> initializes the +>> prefix to 0xfff00000.) +> Do you have a link to a spec saying so? Should be trivial to change +> then. +> +>> Likewise, init_excp_7450() (and probably other, related CPUs) are +>> wrong. +>> +>> Indeed, when I change the value in init_excp_7400() to 0xfff00000UL +>> then +>> everything works as expected for me. +>> +>> ** Affects: qemu +>> Importance: Undecided +>> Status: New +>> Bug description: +>> I have a proprietary ROM implementing system calls that are executed +>> via the 'SC' instruction. +>> +>> I use qemu-0.14.1, +>> +>> qemu-system-ppc -M prep -cpu $CPU -bios my_bios -kernel my_kernel +> We are currently in the process of revamping the PReP machine you are +> using above. Is your BIOS available publicly so that we can test we +> don't break anything for you? +> +> Andreas +> + + + +Looking through old bug tickets... can you still reproduce this issue with the latest version of QEMU? Or could we close this ticket nowadays? + + +I no longer have the test readily available. So I tried to print the initial MSR and IP register contents from the QEMU monitor: + +qemu-system-ppc -machine none -cpu 7400 -S -monitor stdio +QEMU 5.0.93 monitor - type 'help' for more information +(qemu) info registers +NIP 00000000 LR 00000000 CTR 00000000 XER 00000000 CPU#0 +MSR 00000000 HID0 00000000 HF 00000000 iidx 0 didx 0 +Segmentation fault (core dumped) + +Unfortunately this lets qemu (tried 2.11.1(Debian 1:2.11+dfsg-1ubuntu7.29) as well as 5.1.0-rc3) segfault; apparently the time-base is not initialized but still accessed when -machine == none. Yet another bug, it seems. The NIP and MSR seem wrong, however. + +I can generate an empty ppc_rom.bin and fool a prep machine under 2.11.1: + +till@tillp1 $ ls -l empty.bin +-rw-r--r-- 1 till till 0 Aug 8 12:03 empty.bin + +till@tillp1 $ qemu-system-ppc -bios ./empty.bin -cpu 7400 -machine prep -S -monitor stdio +QEMU 2.11.1 monitor - type 'help' for more information +(qemu) info registers +NIP fff00100 LR 00000000 CTR 00000000 XER 00000000 CPU#0 +MSR 00000040 HID0 00000000 HF 00000000 iidx 3 didx 3 + +Here, the issue is fixed! Apparently it is fixed for the 'prep' machine but not 'none'. Unfortunately 'prep' is gone from 5.3.0 and 'none' is buggy; wait - it seems I can emulate 'prep' with '40p': + +till@tillp1 $ build/ppc-softmmu/qemu-system-ppc -machine 40p -cpu 7400 -S -monitor stdio +QEMU 5.0.93 monitor - type 'help' for more information +(qemu) info registers +NIP fff00100 LR 00000000 CTR 00000000 XER 00000000 CPU#0 +MSR 00000040 HID0 00000000 HF 00000000 iidx 3 didx 3 + +This looks good, so I suppose it is OK to close this bug. + + + + + +Ok, thanks for checking! I'll keep the bug open, though, in case someone wants to have a look at the segfault with the "none" machine. + +Please don't close ticket if there's a known problem just to at least +document there's a problem. Is this a CPU feature or board specific? + +Doesn't these CPUs have some way to select the exception vectors base and +could that be set wrong? I've also seen some problems with these CPUs but +last time I asked nobody answered: +https://lists.nongnu.org/archive/html/qemu-ppc/2020-03/msg00292.html +Could this bug be related to that? + + +Yes, it is a CPU feature, and yes you can select the exception vector prefix with the MSR[IP] bit which should be set by a hardware reset. The initial value seems wrong in qemu but that seems to fixed by the machine-specific initialization. The 'none' machine, however, just uses generic code and does not do anything PPC-specific. This means that + + - the MSR and probably other registers, too, are not initialized to what the hardware + documentation specifies as reset values. + - the time-base is not initialized at all (and this leads to a segfault when you start the + ppc 'none' machine) + - probably other things are not properly initialized. I wonder, e.g., about the MMU... + +It seems that all registers are simply initialized to zero. Then, there seems to be a 'reset' function which initializes the registers to the proper reset values (well - sort of bug 812398 reports that HID0 is not properly initialized by some CPU flavours). However, that reset function +is not executed by the 'none' machine initialization.... + + +This is an automated cleanup. This bug report has been moved to QEMU's +new bug tracker on gitlab.com and thus gets marked as 'expired' now. +Please continue with the discussion here: + + https://gitlab.com/qemu-project/qemu/-/issues/85 + + diff --git a/results/classifier/zero-shot/105/assembly/884401 b/results/classifier/zero-shot/105/assembly/884401 new file mode 100644 index 000000000..4b3206ab3 --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/884401 @@ -0,0 +1,83 @@ +assembly: 0.882 +graphic: 0.877 +socket: 0.869 +semantic: 0.866 +other: 0.865 +network: 0.851 +instruction: 0.850 +device: 0.832 +vnc: 0.769 +boot: 0.746 +mistranslation: 0.739 +KVM: 0.668 + +PCI Passthrough for Digium TCE400P Codec Card Not working + +trying to use a Digium TCE400P Codec card on a Virtual instance using the following information: + +lspci <enter> + +02:08.0 Ethernet controller: Digium, Inc. Wildcard TCE400P transcoder base card (rev 11) + +lspci -n <enter> + +02:08.0 0200: d161:8004 (rev 11) + +virsh nodedev-list | grep pci + +pci_0000_02_08_0 + +printf %x 02 +2 + +printf %x 08 +8 + +printf %x 0 +0 + +bus='0x02' +slot='0x08' +function='0x0' + +# virsh edit vmanager +<hostdev mode='subsystem' type='pci' managed='yes'> + <source> + <address domain='0x0000' bus='0x02' slot='0x08' function='0x0'/> + </source> +</hostdev> + +I have SELINUX disabled at this time. + +virsh start vmanager I get the following error message: + +[root@twins qemu]# virsh start vmanager +error: Failed to start domain vmanager +error: internal error Process exited while reading console log output: char device redirected to /dev/pts/2 +Unable to assign device: PCI region 1 at address 0xdf1fe000 has size 0x400, which is not a multiple of 4K +qemu-kvm: -device pci-assign,host=02:08.0,id=hostdev0,configfd=23,bus=pci.0,addr=0x6: Device 'pci-assign' could not be initialized + + + +Version Numbers: + +[root@twins qemu]# yum list | grep qemu +gpxe-roms-qemu.noarch 0.9.7-6.3.el6_0.1 @updates +qemu-img.x86_64 2:0.12.1.2-2.113.el6_0.8 @updates +qemu-kvm.x86_64 2:0.12.1.2-2.113.el6_0.8 @updates +qemu-kvm-tools.x86_64 2:0.12.1.2-2.113.el6_0.8 updates + +Here is what my grub.conf looks like (see the addition of the intel_iommu=on: + +title CentOS Linux (2.6.32-71.29.1.el6.x86_64) + root (hd0,0) + kernel /vmlinuz-2.6.32-71.29.1.el6.x86_64 ro root=/dev/mapper/vg_twins-lv_root rd_LVM_LV=vg_twins/lv_root rd_LVM_LV=vg_twins/lv_swap rd_NO_LUKS rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=us crashkernel=128M rhgb quiet intel_iommu=on + initrd /initramfs-2.6.32-71.29.1.el6.x86_64.img + + +This is a distro bug, not an upstream bug. The error message indicates the version of qemu-kvm you're using doesn't support sub-4k MMIO BARs. This has already been fixed in RHEL6.1. + +According to the comment to Alex, this should have been fixed in newer versions, so setting status to "Fix Released" now. + +Oh, I meant "comment from Alex", not "comment to Alex", sorry! + diff --git a/results/classifier/zero-shot/105/assembly/904 b/results/classifier/zero-shot/105/assembly/904 new file mode 100644 index 000000000..4946b8146 --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/904 @@ -0,0 +1,29 @@ +assembly: 0.991 +instruction: 0.939 +graphic: 0.875 +device: 0.846 +semantic: 0.662 +network: 0.599 +mistranslation: 0.507 +vnc: 0.485 +boot: 0.425 +socket: 0.381 +KVM: 0.219 +other: 0.191 + +RISC-V: Cannot set SEIP bit of mip csr register in M mode +Description of problem: + +Steps to reproduce: +1. run assembly instructions **in M mode**: +``` +not t0, x0 // set t0 to 0b11..11 +csrs mip, t0 // write mip with t0, mip registers are WARL(Write Any Values, Reads Legal Values) +csrr t1, mip // read value from mip to t1 +``` +2. GDB enters the command `display/z $t1` to see that the content of the t1 register is 0x466, which means that the SEIP bit of mip is not set. +3. According to page 47 of [riscv-privileged-20211203](https://github.com/riscv/riscv-isa-manual/releases/download/Priv-v1.12/riscv-privileged-20211203.pdf), `SEIP is writable in mip`. +4. According to page 81 of the same manual,`If implemented, SEIP is read-only in sip`. +5. However, the above code and results show that the SEIP bit of mip cannot be set by software **in M mode**. +Additional information: + diff --git a/results/classifier/zero-shot/105/assembly/968 b/results/classifier/zero-shot/105/assembly/968 new file mode 100644 index 000000000..3820ce2d6 --- /dev/null +++ b/results/classifier/zero-shot/105/assembly/968 @@ -0,0 +1,108 @@ +assembly: 0.718 +other: 0.711 +graphic: 0.650 +device: 0.637 +instruction: 0.603 +socket: 0.550 +semantic: 0.550 +vnc: 0.548 +boot: 0.527 +KVM: 0.514 +network: 0.497 +mistranslation: 0.457 + +QEMU guest agent fails to install if COM+ Application: QEMU Guest Agent VSS Provider not properly uninstalled +Description of problem: +QEMU guest agent fails to install if COM+ Application: QEMU Guest Agent VSS Provider not properly uninstalled +Steps to reproduce: +1. Install QEMU guest agent +2. Uninstall QEMU guest agent (in rare cases it didn't uninstall the COM+ component) +3. Install QEMU guest agent and get error: `Product: QEMU guest agent -- Error 1722. There is a problem with this Windows Installer package. A program run as part of the setup did not finish as expected. Contact your support personnel or package vendor. Action RegisterCom, location: cmd.exe, command: /c "C:\Program Files\Qemu-ga\qemu-ga.exe" -s vss-install` +Additional information: +1. **Qemu GA is already uninstalled:** + +``` +gwmi Win32_Product + + +IdentifyingNumber : {EE3877E4-07B0-41F2-ADB8-B45133DDCE37} +Name : Spice Agent 0.10.0-5 (64-bit) +Vendor : Red Hat, Inc. +Version : 0.10.5 +Caption : Spice Agent 0.10.0-5 (64-bit) + +IdentifyingNumber : {4C49C419-DE39-421B-B0F8-5F0DE1486869} +Name : Virtio-win-driver-installer +Vendor : Red Hat, Inc. +Version : 0.1.189 +Caption : Virtio-win-driver-installer + +IdentifyingNumber : {85F4CBCB-9BBC-4B50-A7D8-E1106771498D} +Name : Orca +Vendor : Microsoft Corporation +Version : 3.1.5299.0000 +Caption : Orca + +IdentifyingNumber : {89F4137D-6C26-4A84-BDB8-2E5A4BB71E00} +Name : Microsoft Silverlight +Vendor : Microsoft Corporation +Version : 5.1.50918.0 +Caption : Microsoft Silverlight + +IdentifyingNumber : {AB392F9F-0C0C-4098-B5BA-B1E84E62D6CE} +Name : Icinga 2 +Vendor : Icinga GmbH +Version : 2.11.0 +Caption : Icinga 2 +``` + +2. **Extract files from installer and run `qemu-ga.exe -s vss-install`** + +It fails with: `QGA VSS Provider is already installed. (Error: 80004004) Vorgang abgebrochen` + +3. **Uninstall COM+ component: `qemu-ga.exe -s vss-uninstall`** + +`Removing COM+ Application: QEMU Guest Agent VSS Provider` + +4. **Now you can install GA** + +``` +gwmi Win32_Product + + +IdentifyingNumber : {EE3877E4-07B0-41F2-ADB8-B45133DDCE37} +Name : Spice Agent 0.10.0-5 (64-bit) +Vendor : Red Hat, Inc. +Version : 0.10.5 +Caption : Spice Agent 0.10.0-5 (64-bit) + +IdentifyingNumber : {4C49C419-DE39-421B-B0F8-5F0DE1486869} +Name : Virtio-win-driver-installer +Vendor : Red Hat, Inc. +Version : 0.1.189 +Caption : Virtio-win-driver-installer + +IdentifyingNumber : {85F4CBCB-9BBC-4B50-A7D8-E1106771498D} +Name : Orca +Vendor : Microsoft Corporation +Version : 3.1.5299.0000 +Caption : Orca + +IdentifyingNumber : {99AD6A3C-F854-4E6E-865F-11D4A5E46172} +Name : QEMU guest agent +Vendor : RedHat +Version : 101.1.0 +Caption : QEMU guest agent + +IdentifyingNumber : {89F4137D-6C26-4A84-BDB8-2E5A4BB71E00} +Name : Microsoft Silverlight +Vendor : Microsoft Corporation +Version : 5.1.50918.0 +Caption : Microsoft Silverlight + +IdentifyingNumber : {AB392F9F-0C0C-4098-B5BA-B1E84E62D6CE} +Name : Icinga 2 +Vendor : Icinga GmbH +Version : 2.11.0 +Caption : Icinga 2 +``` |