path: root/results/classifier/zero-shot/118/all/1749393

register: 0.982
debug: 0.981
permissions: 0.978
virtual: 0.978
kernel: 0.977
device: 0.974
arm: 0.971
architecture: 0.971
semantic: 0.969
assembly: 0.967
performance: 0.967
graphic: 0.965
risc-v: 0.963
PID: 0.957
peripherals: 0.957
mistranslation: 0.957
x86: 0.956
user-level: 0.945
network: 0.933
hypervisor: 0.931
boot: 0.930
socket: 0.930
vnc: 0.928
files: 0.922
VMM: 0.922
ppc: 0.902
KVM: 0.855
i386: 0.855
TCG: 0.854

sbrk() not working under qemu-user with a PIE-compiled binary?

In Debian unstable, we recently switched bash to be a PIE-compiled binary (for hardening). Unfortunately this resulted in bash being broken when run under qemu-user (for all target architectures, host being amd64 for me).

$ sudo chroot /srv/chroots/sid-i386/ qemu-i386-static /bin/bash
bash: xmalloc: .././shell.c:1709: cannot allocate 10 bytes (0 bytes allocated)

bash has its own malloc implementation based on sbrk():
https://git.savannah.gnu.org/cgit/bash.git/tree/lib/malloc/malloc.c

When we disable this internal implementation and rely on glibc's malloc, then everything is fine. But it might be that glibc has a fallback when sbrk() is not working properly and it might hide the underlying problem in qemu-user.

This issue has also been reported to the bash upstream author and he suggested that the issue might be in qemu-user so I'm opening a ticket here. Here's the discussion with the bash upstream author:
https://lists.gnu.org/archive/html/bug-bash/2018-02/threads.html#00080

You can find the problematic bash binary in that .deb file:
http://snapshot.debian.org/archive/debian/20180206T154716Z/pool/main/b/bash/bash_4.4.18-1_i386.deb

The version of qemu I have been using is 2.11 (Debian package qemu-user-static version 1:2.11+dfsg-1) but I have had reports that the problem is reproducible with older versions (back to 2.8 at least).

Here are the related Debian bug reports:
https://bugs.debian.org/889869
https://bugs.debian.org/865599

It's worth noting that bash used to have this problem (when compiled as a PIE binary) even when run directly but then something got fixed in the kernel and now the problem only appears when run under qemu-user:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518483

Affected by the same bug.
target architecture arm
host architecture amd64

bug message
bash: xmalloc: .././locale.c:****: cannot allocate 2 bytes (0 bytes allocated)

This appears to be a problem in all PIE-compiled executables that use sbrk in qemu-user due to the way that position-independent code gets mmapped into adjacent ranges meaning there is no room for expansion. I've hacked my version of QEMU to force the program binary to mmap in a different range allowing for the region to be resized which fixes this issue. I don't know the most appropriate way to determine what range to use in generate though.

There seem to be two parts to this. Firstly, with a big reserved-region, which is the default for 32-bit-guest-on-64-bit-host, this code in main.c:

        if (reserved_va) {
            mmap_next_start = reserved_va;
        }

says to start trying for the next mmap address at the top of the reserved section, which is typically right at the top of the guest's address space. This means that for a PIE executable we'll try to load it at a very high address, which then means there's no space above the data section for the brk segment.

Secondly, for the no-reserved-region case (-R 0, or 64-on-64), we still fail, but this time because we've chosen to mmap the dynamic interpreter at an address just above the executable. Again, no space to expand the data segment and brk fails.

Linux kernel commit a87938b2e246b81 message says something about there being a guaranteed 128MB "gap" between data segment and stack on x86-64 which we're obviously not honourin; presumbably there's similar requirements for other archs. (As an aside, is bash really happy with only having perhaps 128MB of allocatable memory? Otherwise it really ought to use mmap rather than brk for its allocator.)


Could we over-allocate the data segment by QEMU_DATA_SIZE/getrlimit(RLIMIT_DATA)/128 MB depending on what's set - similar to how the stack size is managed? 

My current workaround for aarch64 on x86-64 is to mmap a dynamic main executable in some far-away part of the address space but I'm not sure how to find somewhere suitable on a 32-bit host/guest.

On 16 March 2018 at 10:34, Richard Henderson
<email address hidden> wrote:
> Limit this to 16M; there does not appear to be any special
> support for this in the kernel itself, at least for i686.
>
> Fixes: https://bugs.launchpad.net/bugs/1749393
> Signed-off-by: Richard Henderson <email address hidden>
> ---
>
> Commentary in the launchpad bug suggests 128M gap for x86_64, but that's
> somewhat irrelevant to the given i686 test case.  There's certainly nothing
> in the referenced kernel patch that does any more than we had been doing
> without this patch.

I think the 128MB is enforced by mmap_base() in arch/x86/mm/mmap.c:
since x86-64 sots HAVE_ARCH_UNMAPPED_AREA_TOPDOWN, mmap_base is the
highest address in memory where mmap is permitted, and mmap_base()
enforces that it goes at least 128MB below the bottom of the stack
(accounting for rlimit stack size requirements also). Since
binfmt_elf() loads ELF segments via mmap this means that they won't
go too close to the stack. (The commit a87938b2e246 ensures the
gap is honoured by using the full binary size when it does the first
mapping so that mmap picks an address that is sufficiently before the
end of the mmap region for everything to fit.)
The kernel also uses ELF_ET_DYN_BASE to ensure that PIE programs
themselves get loaded clear of the ELF interpreter, which we
don't have any equivalent of (so you can see that different values
of -R result in either the interpreter or the executable getting
loaded at lower addresses.)

PS: do you know what the intention of the
        if (reserved_va) {
            mmap_next_start = reserved_va;
        }
code in linux-user/main.c is? It seems a bit odd to say "ok,
we have reserved a big region. we will start trying to mmap
outside it.", especially when that region covers the full
4G that the guest can access...

thanks
-- PMM


Status changed to 'Confirmed' because the bug affects multiple users.

qemu patch proposed at http://lists.nongnu.org/archive/html/qemu-devel/2018-03/msg04700.html

Another proposed patch:
https://<email address hidden>/

Fixed here:
https://git.qemu.org/?p=qemu.git;a=commitdiff;h=6fd5944980f4

Will be merged in 20.10 with qemu >=5.0 where this came upstream.

This bug was fixed in the package qemu - 1:5.0-5ubuntu3

---------------
qemu (1:5.0-5ubuntu3) groovy; urgency=medium

  * d/p/ubuntu/lp-1887763-*: fix TCG sizing that OOMed many small CI
    environments (LP: #1887763)
  * Pick further changes for groovy from debian/master since 5.0-5
    - ati-vga-check-mm_index-before-recursive-call-CVE-2020-13800.patch
      Closes: CVE-2020-13800, ati-vga allows guest OS users to trigger
      infinite recursion via a crafted mm_index value during
      ati_mm_read or ati_mm_write call.
    - revert-memory-accept-mismatching-sizes-in-memory_region_access_valid...patch
      Closes: CVE-2020-13754, possible OOB memory accesses in a bunch of qemu
      devices which uses min_access_size and max_access_size Memory API fields.
      Also closes: CVE-2020-13791
    - exec-set-map-length-to-zero-when-returning-NULL-CVE-2020-13659.patch
      CVE-2020-13659: address_space_map in exec.c can trigger
      a NULL pointer dereference related to BounceBuffer
    - megasas-use-unsigned-type-for-reply_queue_head-and-check-index...patch
      Closes: #961887, CVE-2020-13362, megasas_lookup_frame in hw/scsi/megasas.c
      has an OOB read via a crafted reply_queue_head field from a guest OS user
    - megasas-use-unsigned-type-for-positive-numeric-fields.patch
      fix other possible cases like in CVE-2020-13362 (#961887)
    - megasas-fix-possible-out-of-bounds-array-access.patch
      Some tracepoints use a guest-controlled value as an index into the
      mfi_frame_desc[] array. Thus a malicious guest could cause a very low
      impact OOB errors here
    - nbd-server-avoid-long-error-message-assertions-CVE-2020-10761.patch
      Closes: CVE-2020-10761, An assertion failure issue in the QEMU NBD Server.
      This flaw occurs when an nbd-client sends a spec-compliant request that is
      near the boundary of maximum permitted request length. A remote nbd-client
      could use this flaw to crash the qemu-nbd server resulting in a DoS.
    - es1370-check-total-frame-count-against-current-frame-CVE-2020-13361.patch
      Closes: CVE-2020-13361, es1370_transfer_audio in hw/audio/es1370.c does not
      properly validate the frame count, which allows guest OS users to trigger
      an out-of-bounds access during an es1370_write() operation
    - a few patches from the stable series:
      - fix-tulip-breakage.patch
        The tulip network driver in a qemu-system-hppa emulation is broken in
        the sense that bigger network packages aren't received any longer and
        thus even running e.g. "apt update" inside the VM fails. Fix this.
      - 9p-lock-directory-streams-with-a-CoMutex.patch
        Prevent deadlocks in 9pfs readdir code
      - net-do-not-include-a-newline-in-the-id-of-nic-device.patch
        Fix newline accidentally sneaked into id string of a nic
      - qemu-nbd-close-inherited-stderr.patch
      - virtio-balloon-fix-free-page-hinting-check-on-unreal.patch
      - virtio-balloon-fix-free-page-hinting-without-an-iothread.patch
      - virtio-balloon-unref-the-iothread-when-unrealizing.patch
    - acpi-tmr-allow-2-byte-reads.patch (Closes: #964247)
    - reapply CVE-2020-13253 fixed from upstream:
      sdcard-simplify-realize-a-bit.patch (preparation for the next patch)
      sdcard-dont-allow-invalid-SD-card-sizes.patch (half part of CVE-2020-13253)
      sdcard-update-coding-style-to-make-checkpatch-happy.patch (preparational)
      sdcard-dont-switch-to-ReceivingData-if-address-is-in..-CVE-2020-13253.patch
      Closes: #961297, CVE-2020-13253
    - linux-user-refactor-ipc-syscall-and-support-of-semtimedop.patch
      (Closes: #965109)
    - linux-user-add-netlink-RTM_SETLINK-command.patch (Closes: #964289)
    - d/control: since qemu-system-data now contains module(s),
      it can't be multi-arch. Ditto for qemu-block-extra.
    - qemu-system-foo: depend on exact version of qemu-system-data,
      due to the latter having modules
    - acpi-allow-accessing-acpi-cnt-register-by-byte.patch' (Closes: #964793)
      This is another incarnation of the recent bugfix which actually enabled
      memory access constraints, like #964247
    - acpi-accept-byte-and-word-access-to-core-ACPI-registers.patch
      this replace acpi-allow-accessing-acpi-cnt-register-by-byte.patch
      and acpi-tmr-allow-2-byte-reads.patch, a more complete fix
    - xhci-fix-valid.max_access_size-to-access-address-registers.patch
      fix one more incarnation of the breakage after the CVE-2020-13754 fix
    - do not install outdated (0.12 and before) Changelog (Closes: #965381)
    - xgmac-fix-buffer-overflow-in-xgmac_enet_send-CVE-2020-15863.patch
      ARM-only XGMAC NIC, possible buffer overflow during packet transmission
      Closes: CVE-2020-15863
    - sm501 OOB read/write due to integer overflow in sm501_2d_operation()
      List of patches:
       sm501-convert-printf-abort-to-qemu_log_mask.patch
       sm501-shorten-long-variable-names-in-sm501_2d_operation.patch
       sm501-use-BIT-macro-to-shorten-constant.patch
       sm501-clean-up-local-variables-in-sm501_2d_operation.patch
       sm501-replace-hand-written-implementation-with-pixman-CVE-2020-12829.patch
      Closes: #961451, CVE-2020-12829
    - riscv-allow-64-bit-access-to-SiFive-CLINT.patch
      another fix for revert-memory-accept-.. CVE-2020-13754
    - seabios-hppa-fno-ipa-sra.patch fix ftbfs with gcc-10

qemu (1:5.0-5ubuntu2) groovy; urgency=medium

  * No change rebuild against new libnettle8 and libhogweed6 ABI.

qemu (1:5.0-5ubuntu1) groovy; urgency=medium

  * Merge with Debian testing (LP: #1749393), remaining changes:
    - qemu-kvm to systemd unit
      - d/qemu-kvm-init: script for QEMU KVM preparation modules, ksm,
        hugepages and architecture specifics
      - d/qemu-system-common.qemu-kvm.service: systemd unit to call
        qemu-kvm-init
      - d/qemu-system-common.install: install helper script
      - d/qemu-system-common.qemu-kvm.default: defaults for
        /etc/default/qemu-kvm
      - d/rules: call dh_installinit and dh_installsystemd for qemu-kvm
    - Distribution specific machine type (LP: 1304107 1621042)
      - d/p/ubuntu/define-ubuntu-machine-types.patch: define distro machine
        types
      - d/qemu-system-x86.NEWS Info on fixed machine type definitions
        for host-phys-bits=true (LP: 1776189)
      - add an info about -hpb machine type in debian/qemu-system-x86.NEWS
      - provide pseries-bionic-2.11-sxxm type as convenience with all
        meltdown/spectre workarounds enabled by default. (LP: 1761372).
      - ubuntu-q35 alias added to auto-select the most recent q35 ubuntu type
    - Enable nesting by default
      - d/p/ubuntu/enable-svm-by-default.patch: Enable nested svm by default
        in qemu64 on amd
        [ No more strictly needed, but required for backward compatibility ]
    - improved dependencies
      - Make qemu-system-common depend on qemu-block-extra
      - Make qemu-utils depend on qemu-block-extra
      - let qemu-utils recommend sharutils
    - arch aware kvm wrappers
    - tolerate ipxe size change on migrations to >=18.04 (LP: 1713490)
      - d/p/ubuntu/pre-bionic-256k-ipxe-efi-roms.patch: old machine types
        reference 256k path
      - d/control-in: depend on ipxe-qemu-256k-compat-efi-roms to be able to
        handle incoming migrations from former releases.
    - d/control-in: Disable capstone disassembler library support (universe)
    - d/qemu-system-x86.README.Debian: add info about updated nesting changes
    - d/control*, d/rules: disable xen by default, but provide universe
      package qemu-system-x86-xen as alternative
      [includes --disable-xen for user-static builds]
    - d/control-in: disable pmem on ppc64 as it is currently considered
      experimental on that architecture (pmdk v1.8-1)
    - d/rules: makefile definitions can't be recursive - sys_systems for s390x
    - d/rules: report config log from the correct subdir
    - allow qemu to load old modules post upgrade (LP 1847361)
      - d/qemu-block-extra.*.in, d/qemu-system-gui.*.in: save shared objects on
        upgrade
      - d/rules: generate maintainer scripts matching package version on build
      - d/rules: enable --enable-module-upgrades where --enable-modules is set
    - d/p/ubuntu/lp-1835546-*: backport the s390x protvirt feature (LP 1835546)
    - d/control-in: disable rbd support unavailable on riscv (LP: 1872931)
    - debian/patches/ubuntu/lp-1878973-*: fix assert in qemu-guest-agent that
      crashes it on shutdown (LP 1878973)
  * Dropped changes (no more needed)
    - d/qemu-system-common.maintscript: clean old sysv and upstart scripts
    - d/p/ubuntu/expose-vmx_qemu64cpu.patch: expose nested kvm by default
      in qemu64 cpu type.
    - d/control: avoid upgrade issues triggered by moving ivshmem tools after
      Debian. Fixed by bumping the related Breaks/Replaces to the
      Version Ubuntu introduced the change (LP 1862287)
  * Dropped changes (in Debian)
    - improved s390x support
    - d/binfmt-update-in: fix binfmt being called in some containers
      (LP 1840956)
    - qemu-system-x86-microvm package
      In addition to the generic multi-purpose qemu also provide a minimal
      feature binary that is loading faster for use cases with microvm machine
      type and qboot bios
      - d/control-in: add a new qemu-system-x86-microvm package
      - d/rules: add an extra config/build step to get the minimal qemu
    - Security and packaging fixes (LP 1872937)
      - arm-fix-PAuth-sbox-functions-CVE-2020-10702.patch
      - net-tulip-check-frame-size-and-r-w-data-length-CVE-2020-11102.patch
        CVE-2020-10702
        CVE-2020-11102
      - fix external spice UI
        + install ui-spice-app.so in qemu-system-common
        + install ui-spice-app.so only if built, spice is optional
      - switch binfmt registration to use update-binfmts --[un]import (#866756)
      - qemu-system-gui: Multi-Arch=same, not foreign (#956763)
      - qemu-system-data: s/highcolor/hicolor/ (#955741)
    - enable riscv build (LP 1872931)
      [ changes picked from Debian ]
      - enable support for riscv64 hosts
      - only enable librbd on architectures where it is built
      - ceph: do not list librados-dev as we only use librbd-dev and the latter
        depends on the former
      - seccomp grew up, no need in versioned build-dep
      - enable seccomp only on architectures where it can be built
  * Dropped changes (upstream)
    - d/p/ubuntu/lp-1857033-*: add support for Cooper Lake cpu model
      (LP 1857033)
    - d/p/lp-1859527-*: avoid breakage on high virtqueue counts (LP 1859527)
    - d/p/ubuntu/vhost-user-gpu-Drop-trailing-json-comma.patch: fix parsing of
      vhost-user-gpu
    - d/p/ubuntu/lp-1847361-vhost-correctly-turn-on-VIRTIO_F_IOMMU_PLATFORM.patch:
      avoid unnecessary IOTLB transactions (LP 1866207)
    - d/p/stable/lp-1867519-*: Stabilize qemu 4.2 with upstream
      patches @qemu-stable (LP 1867519)
    - remove d/p/ubuntu/expose-vmx_qemu64cpu.patch: Stop adding VMX to qemu64
      to avoid broken nesting (LP 1868692)
    - d/p/ubuntu/lp-1871830-*: avoid crash when using QEMU_MODULE_DIR
      (LP 1871830)
    - d/p/ubuntu/lp-1872107*: fix migration while rebooting guests (LP 1872107)
    - d/p/ubuntu/lp-1872931-*: fix build on non KVM platforms
    - d/p/ubuntu/lp-1872945-*: fix riscv emulation errors that e.g. hung ssh
      and clobbered doubles (LP 1872945)
    - SECURITY UPDATE: DoS via integer overflow in ati_2d_blt()
      - debian/patches/ubuntu/CVE-2020-11869.patch: fix checks in
        ati_2d_blt() to avoid crash in hw/display/ati_2d.c.
      - CVE-2020-11869
    - d/p/ubuntu/lp-1805256*: Fixes for QEMU on aarch64 ARM hosts
      - async: use explicit memory barriers (LP 1805256)
      - aio-wait: delegate polling of main AioContext if BQL not held
    - d/p/ubuntu/lp-1882774-*: fix issues with VMX subfeatures on systems not
      supporting to set them (LP 1882774)
    - d/p/ubuntu/lp-1847361-modules-load-upgrade.patch: to fallback module
      load to a versioned path
  * Added Changes:
    - d/control: regenerate debian/control out of control-in
    - update d/p/ubuntu/lp-1835546-* to the final versions
      - 11 patches dropped as they are in 5.0
      - 20 patches updated to how they will be in 5.1
    - d/p/ubuntu/virtio-net-fix-rsc_ext-compat-handling.patch: fix
      FTBFS in groovy
    - Make qemu-system-x86-microvm a transitional package as the binary is now
      in qemu-system-x86 itself.
    - d/control-in: build-dep libcap is no more needed
    - d/rules: update arch aware kvm wrappers
    - d/qemu-system-x86.README.Debian: fix typo

qemu (1:5.0-5) unstable; urgency=medium

  * more binfmt-install updates
  * CVE-2020-10717 fix from upstream:
    virtiofsd-add-rlimit-nofile-NUM-option.patch (preparational) and
    virtiofsd-stay-below-fs.file-max-CVE-2020-10717.patch
    (Closes: #959746, CVE-2020-10717)
  * 2 patches from upstream/stable to fix io_uring fd set buildup:
    aio-posix-dont-duplicate-fd-handler-deletion-in-fdmon_io_uring_destroy.patch
    aio-posix-disable-fdmon-io_uring-when-GSource-is-used.patch
  * upstream stable fix: hostmem-dont-use-mbind-if-host-nodes-is-empty.patch
  * upstream stable fix:
    net-use-peer-when-purging-queue-in-qemu_flush_or_purge_queue_packets.patch

qemu (1:5.0-4) unstable; urgency=medium

  * fix binfmt registration (Closes: #959222)
  * disable PIE for user-static build on x32 too, not only i386

qemu (1:5.0-3) unstable; urgency=medium

  * do not explicitly enable -static-pie on non-i386 architectures.
    Apparenly only amd64 actually support -static-pie for now, and
    it is correctly detected.

qemu (1:5.0-2) unstable; urgency=medium

  * (temporarily) disable pie on i386 static build
    For now -static-pie fails on i386 with the following error message:
      /usr/bin/ld: /usr/lib/i386-linux-gnu/libc.a(memset_chk-nonshared.o):
          unsupported non-PIC call to IFUNC `memset'
  * install qemu-system docs in qemu-system-common, not qemu-system-data,
    since docs require ./configure run

qemu (1:5.0-1) unstable; urgency=medium

  * new upstream release (5.0)
    Closes: #958926
    Closes: CVE-2020-11869
  * refresh patches, remove patches applied upstream
  * do not mention openhackware, it is not used anymore
  * do not disable bluez (support removed)
  * new system arch "rx"
  * dont install qemu-doc.* for now,
    but install virtiofsd & qemu-storage-daemon
  * add shared-lib-without-dependency-information tag
    to qemu-user-static.lintian-overrides
  * add html docs to qemu-system-data (to /usr/share/doc/qemu-system-common)
  * do not install usr/share/doc/qemu/specs & usr/share/doc/qemu/tools
  * install qemu-user html docs for qemu-user & qemu-user-static
  * build hppa-firmware.img from roms/seabios-hppa
    (and Build-Depeds-Indep on gcc-hppa-linux-gnu)
  * enable liburing on linux (build-depend on liburing-dev)
  * add upstream signing-key.asc (Michael Roth <email address hidden>)
  * build opensbi firmware
    (for riscv64 only, riscv32 is possible with compiler flags)
  * add source-level lintian-overrides for binaries-without-sources
    (lintian can't find sources for a few firmware images which are in roms/)

qemu (1:4.2-7) unstable; urgency=medium

  * qemu-system-gui: Multi-Arch=same, not foreign (Closes: #956763)
  * x32 arch is in the same family as i386 & x86_64, omit binfmt registration
  * check systemd-detect-virt before running update-binfmt
  * gluster is de-facto linux-only, do not build-depend on it on non-linux
  * virglrenderer is also essentially linux-specific
  * qemu-user-static does not depend on shlibs
  * disable parallel building of targets of d/rules
  * add lintian overrides (arch-dependent static binaries) for openbios binaries
  * separate binary-indep target into install-indep-prep and binary-indep
  * split out various components of qemu-system-data into independent
    build/install rules and add infrastructure for more components:
    x86-optionrom, sgabios, qboot, openbios, skiboot, palcode-clipper,
    slof, s390x-fw
  * iscsi-fix-heap-buffer-overflow-in-iscsi_aio_ioctl_cb.patch

qemu (1:4.2-6) unstable; urgency=medium

  * d/rules: fix FTBFS (brown-paper-bag bug) in last upload

qemu (1:4.2-5) unstable; urgency=medium

  * no error-out on address-of-packet-member in openbios
  * install ui-spice-app.so only if built, spice is optional
  * arm-fix-PAuth-sbox-functions-CVE-2020-10702.patch -
    Closes: CVE-2020-10702, weak signature generation
    in Pointer Authentication support for ARM
  * (temporarily) enable seccomp only on architectures where it can be built
    (Closes: #956624)
  * seccomp has grown up, no need in versioned build-dep
  * do not list librados-dev in build-dep as we only use librbd-dev
    and the latter depends on the former
  * only enable librbd on architectures where it is buildable

qemu (1:4.2-4) unstable; urgency=medium

  [ Michael Tokarev ]
  * d/rules: build minimal configuration for qboot/microvm usage
  * set microvm to be the default machine type for microvm case
  * install ui-spice-app.so in qemu-system-common
  * do not depend on libattr-dev, functions are now in libc6 (Closes: #953910)
  * net-tulip-check-frame-size-and-r-w-data-length-CVE-2020-11102.patch
    (Closes: #956145, CVE-2020-11102, tulip nic buffer overflow)
  * qemu-system-data: s/highcolor/hicolor/ (Closes: #955741)
  * switch binfmt registration to use update-binfmts --[un]import
    (Closes: #866756)
  * build openbios-ppc & openbios-sparc binaries in qemu-system-data,
    and replace corresponding binary packages.
    Add gcc-sparc64-linux-gnu, fcode-utils & xsltproc to build-depend-indep
  * build and provide/replace qemu-slof too

  [ Aurelien Jarno ]
  * enable support for riscv64 hosts

 -- Christian Ehrhardt <email address hidden>  Tue, 28 Jul 2020 13:21:31 +0200

There's a request for a backport of this fix to be made to Ubuntu 20.04 in duplicate bug 1924231, so I'm adding a task for that.

For Focal:
- SRU Template added to the bug
- MP: https://code.launchpad.net/~paelzer/ubuntu/+source/qemu/+git/qemu/+merge/401771
- PPA: https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/4535/+packages (still building)

I'd ask anyone affected by this on Focal to give it a try on the PPA and let us know if this fix would work for you.

Thank you for fixing the problem.

I confirmed that https://bugs.launchpad.net/bugs/1924231 is fixed with https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/4535/+packages.

Thank you.

I'm running qemu-arm version 4.2.1 (Debian 1:4.2-3ubuntu6.17) on Ubuntu 20.04.03, but I seem to still be affected by this (or something very much like it). In my case it is armhf exim4 crashing while creating a chroot on an amd64 host. The final command run from deeply within exim4's postinst is:

/usr/sbin/exim4 -C /var/lib/exim4/config.autogenerated.tmp -bV

and produces

Exim version 4.93 #5 built 28-Apr-2021 13:19:17
Copyright (c) University of Cambridge, 1995 - 2018
(c) The Exim Maintainers and contributors in ACKNOWLEDGMENTS file, 2007 - 2018
Berkeley DB: Berkeley DB 5.3.28: (September  9, 2013)
Support for: crypteq iconv() IPv6 GnuTLS move_frozen_messages DANE DKIM DNSSEC Event I18N OCSP PRDR SOCKS TCP_Fast_Open
Lookups (built-in): lsearch wildlsearch nwildlsearch iplsearch cdb dbm dbmjz dbmnz dnsdb dsearch nis nis0 passwd
Authenticators: cram_md5 plaintext
Routers: accept dnslookup ipliteral manualroute queryprogram redirect
Transports: appendfile/maildir/mailstore autoreply lmtp pipe smtp
Fixed never_users: 0
Configure owner: 0:0
Size of off_t: 8
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault (core dumped)

Interestingly, even

/usr/sbin/exim4 -C /dev/null -bV

produces the same result, so it likely doesn't depend on any configuration at my end and should be reproducible.

Please let me know if there is anything I can do to help debug further.

Should I create a separate ticket?

Yeah Sebastian, a new ticket (with a reference to this bug as being similar) would be preferred.

Hi,
sorry this has fallen through the cracks, but bug 1928075 made me re-discover it and it is time finally to complete that.

SRU template updated, PPA rebuilt, Merge requests updated.
Also bundled another bug fix.

Waiting for MR review now.

Uploaded to F-unapproved, waiting for the SRU team to accept it.

Hello Raphaël, or anyone else affected,

Accepted qemu into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/qemu/1:4.2-3ubuntu6.19 in a few hours, and then in the -proposed repository.

Please help us by testing this new package.  See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.  Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Focal

old

$ sudo apt install --reinstall qemu-user-static=1:4.2-3ubuntu6.18
Reading package lists... Done
Building dependency tree       
Reading state information... Done
0 upgraded, 0 newly installed, 1 reinstalled, 0 to remove and 0 not upgraded.
Need to get 21.3 MB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 qemu-user-static amd64 1:4.2-3ubuntu6.18 [21.3 MB]
Fetched 21.3 MB in 1s (16.4 MB/s)           
(Reading database ... 126154 files and directories currently installed.)
Preparing to unpack .../qemu-user-static_1%3a4.2-3ubuntu6.18_amd64.deb ...
Unpacking qemu-user-static (1:4.2-3ubuntu6.18) over (1:4.2-3ubuntu6.18) ...
Setting up qemu-user-static (1:4.2-3ubuntu6.18) ...
Processing triggers for man-db (2.9.1-1) ...

ubuntu@f-1928075-qemuuserstatic:~$ sudo chroot /home/ubuntu/bullseye-arm64 /bin/sh /debootstrap/debootstrap --second-stage
W: Failure trying to run:  /sbin/ldconfig
W: See //debootstrap/debootstrap.log for details
ubuntu@f-1928075-qemuuserstatic:~$ tail -n 2 bullseye-arm64/debootstrap/debootstrap.log
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
Segmentation fault (core dumped)

Upgrade


ubuntu@f-1928075-qemuuserstatic:~$ apt-cache policy qemu-user-static
qemu-user-static:
  Installed: 1:4.2-3ubuntu6.18
  Candidate: 1:4.2-3ubuntu6.19
  Version table:
     1:4.2-3ubuntu6.19 500
        500 http://archive.ubuntu.com/ubuntu focal-proposed/universe amd64 Packages
 *** 1:4.2-3ubuntu6.18 500
        500 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages
        100 /var/lib/dpkg/status
     1:4.2-3ubuntu6.17 500
        500 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages
     1:4.2-3ubuntu6 500
        500 http://archive.ubuntu.com/ubuntu focal/universe amd64 Packages
ubuntu@f-1928075-qemuuserstatic:~$ sudo apt install qemu-user-static
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages will be upgraded:
  qemu-user-static
1 upgraded, 0 newly installed, 0 to remove and 65 not upgraded.
Need to get 21.3 MB of archives.
After this operation, 0 B of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu focal-proposed/universe amd64 qemu-user-static amd64 1:4.2-3ubuntu6.19 [21.3 MB]
Fetched 21.3 MB in 2s (9092 kB/s)           
(Reading database ... 126160 files and directories currently installed.)
Preparing to unpack .../qemu-user-static_1%3a4.2-3ubuntu6.19_amd64.deb ...
Unpacking qemu-user-static (1:4.2-3ubuntu6.19) over (1:4.2-3ubuntu6.18) ...
Setting up qemu-user-static (1:4.2-3ubuntu6.19) ...
Processing triggers for man-db (2.9.1-1) ...
ubuntu@f-1928075-qemuuserstatic:~$ sudo update-binfmts  --test --display  qemu-aarch64
qemu-aarch64 (enabled):
     package = qemu-user-static
        type = magic
      offset = 0
       magic = \x7f\x45\x4c\x46\x02\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\xb7\x00
        mask = \xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff
 interpreter = /usr/bin/qemu-aarch64-static
    detector = 


Test with new versio

ubuntu@f-1928075-qemuuserstatic:~$ sudo chroot /home/ubuntu/bullseye-arm64 /bin/sh /debootstrap/debootstrap --second-stage
I: Installing core packages...
W: Failure trying to run:  dpkg --force-depends --install /var/cache/apt/archives/base-passwd_3.5.51_arm64.deb
W: See //debootstrap/debootstrap.log for details
ubuntu@f-1928075-qemuuserstatic:~$ tail -n 2 bullseye-arm64/debootstrap/debootstrap.log
dpkg: error: parsing file '/var/lib/dpkg/status' near line 5 package 'dpkg':
 duplicate value for 'Package' field


That is the good case and also a full run now completes.

$ sudo rm -rf bullseye-arm64; sudo qemu-debootstrap --arch=arm64 bullseye bullseye-arm64 http://ftp.debian.org/debian
I: Running command: debootstrap --arch arm64 --foreign bullseye bullseye-arm64 http://ftp.debian.org/debian
W: Cannot check Release signature; keyring file not available /usr/share/keyrings/debian-archive-keyring.gpg
I: Retrieving InRelease 
I: Retrieving Packages 
...
I: Configuring tasksel...
I: Configuring libc-bin...
I: Base system installed successfully.



I can't run the docker test due to networking restrictions, but it was the same fault and the same fix - so that should be ok. If anyone else can test -proposed with docker please feel free to do so.

FYI the release of this is slowed down by the slow verification of bug https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1929926

i can confirm that focal-proposed package fixes problems for arm64 and armhf on hostarch amd64

note: tried ppa listed here which fixes for arm64 but breaks armhf: https://bugs.launchpad.net/ubuntu/+source/qemu/+bug/1928075/comments/15

steps for installing proposed Package:

cat <<EOF >/etc/apt/sources.list.d/ubuntu-$(lsb_release -cs)-proposed.list
# Enable Ubuntu proposed archive

deb http://archive.ubuntu.com/ubuntu/ $(lsb_release -cs)-proposed restricted main multiverse universe
EOF

cat <<EOF >/etc/apt/preferences.d/proposed-updates
# Configure apt to allow selective installs of packages from proposed

Package: *
Pin: release a=$(lsb_release -cs)-proposed
Pin-Priority: 400
EOF

apt update
apt install qemu-user-static/focal-proposed

then build 2 bullseye-chroot (arm64 and armhf) including secondstage and no crash happens

Thank you Frank for that extra confirmation,
by now also all the blockers on the other bug fixed are good. I expect this to be released as soon as the SRU Team is back from the Christmas shutdown.

This bug was fixed in the package qemu - 1:4.2-3ubuntu6.19

---------------
qemu (1:4.2-3ubuntu6.19) focal; urgency=medium

  * d/p/u/lp-1749393-linux-user-Reserve-space-for-brk.patch: fix static
    use cases needing a lot of brk space (LP: #1749393)
  * d/p/u/lp-1929926-target-s390x-Fix-translation-exception-on-illegal-in.patch:
    fix uretprobe in s390x TCG (LP: #1929926)

 -- Christian Ehrhardt <email address hidden>  Mon, 26 Apr 2021 11:11:19 +0200

The verification of the Stable Release Update for qemu has completed successfully and the package is now being released to -updates.  Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report.  In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.