summary refs log tree commit diff stats
path: root/tcg (follow)
Commit message (Collapse)AuthorAgeFilesLines
* tcg: Fix error reporting on mprotect() failure in tcg_region_init()Markus Armbruster2025-09-301-2/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | tcg_region_init() calls one of qemu_mprotect_rwx(), qemu_mprotect_rw(), and mprotect(), then reports failure with error_setg_errno(&error_fatal, errno, ...). The use of &error_fatal is undesirable. qapi/error.h advises: * Please don't error_setg(&error_fatal, ...), use error_report() and * exit(), because that's more obvious. The use of errno is wrong. qemu_mprotect_rwx() and qemu_mprotect_rw() wrap around qemu_mprotect__osdep(). qemu_mprotect__osdep() calls mprotect() on POSIX, VirtualProtect() on Windows, and reports failure with error_report(). VirtualProtect() doesn't set errno. mprotect() does, but error_report() may clobber it. Fix tcg_region_init() to report errors only when it calls mprotect(), and rely on qemu_mprotect_rwx()'s and qemu_mprotect_rw()'s error reporting otherwise. Use error_report(), not error_setg(). Fixes: 22c6a9938f75 (tcg: Merge buffer protection and guard page protection) Fixes: 6bc144237a85 (tcg: Use Error with alloc_code_gen_buffer) Cc: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Markus Armbruster <armbru@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-ID: <20250923091000.3180122-3-armbru@redhat.com> Reviewed-by: Akihiko Odaki <odaki@rsg.ci.i.u-tokyo.ac.jp>
* include/hw/core/cpu: Invert the indexing into CPUTLBDescFastRichard Henderson2025-09-231-1/+2
| | | | | | | | | | | | | | | | This array is within CPUNegativeOffsetState, which means the last element of the array has an offset from env with the smallest magnitude. This can be encoded into fewer bits when generating TCG fast path memory references. When we changed the NB_MMU_MODES to be a global constant, rather than a per-target value, we pessimized the code generated for targets which use only a few mmu indexes. By inverting the array index, we counteract that. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* include/hw/core/cpu: Introduce cpu_tlb_fastRichard Henderson2025-09-232-2/+2
| | | | | | | | Encapsulate access to cpu->neg.tlb.f[] in a function. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/optimize: Fix folding of vector bitselWANG Rui2025-09-231-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | It looks like a typo. When the false value (C) is the constant -1, the correct fold should be: R = B | ~A Reproducer (LoongArch64 assembly): .text .globl _start _start: vldi $vr1, 3073 vldi $vr2, 1023 vbitsel.v $vr0, $vr2, $vr1, $vr2 vpickve2gr.d $a1, $vr0, 1 xori $a0, $a1, 1 li.w $a7, 93 syscall 0 Fixes: e58b977238e3 ("tcg/optimize: Optimize bitsel_vec") Link: https://github.com/llvm/llvm-project/issues/159610 Signed-off-by: WANG Rui <wangrui@loongson.cn> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-ID: <20250919124901.2756538-1-wangrui@loongson.cn>
* tcg/i386: Use vgf2p8affineqb for MO_8 vector shiftsRichard Henderson2025-09-041-4/+71
| | | | | | | | | | A constant matrix can describe the movement of the 8 bits, so these shifts can be performed with one instruction. Logic courtesy of Andi Kleen <ak@linux.intel.com>: https://gcc.gnu.org/pipermail/gcc-patches/2025-August/691624.html Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/i386: Add INDEX_op_x86_vgf2p8affineqb_vecRichard Henderson2025-09-042-0/+7
| | | | | | | | Add a backend-specific opcode for expanding the GFNI vgf2p8affineqb instruction, which we can use for expanding 8-bit immediate shifts and rotates. Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/i386: Use canonical operand ordering in expand_vec_sariRichard Henderson2025-09-041-2/+2
| | | | | | | | | The optimizer prefers to have constants as the second operand, so expand LT x,0 instead of GT 0,x. This will not affect the generated code at all. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/i386: Expand sari of bits-1 as pcmpgtRichard Henderson2025-09-041-0/+6
| | | | | | | Expand arithmetic right shift of bits-1 as a comparison vs 0. Suggested-by: Paolo Bonzini <pbonzini@redhat.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/arm: Fix tgen_depositRichard Henderson2025-09-041-1/+2
| | | | | | | | | | | | When converting from tcg_out_deposit, the arguments were not shuffled properly. Cc: qemu-stable@nongnu.org Fixes: cf4905c03135f1181e8 ("tcg: Convert deposit to TCGOutOpDeposit") Reported-by: Michael Tokarev <mjt@tls.msk.ru> Tested-by: Michael Tokarev <mjt@tls.msk.ru> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Add tcg_gen_atomic_{xchg,fetch_and,fetch_or}_i128Richard Henderson2025-08-301-3/+94
| | | | | | | Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Message-id: 20250815122653.701782-5-richard.henderson@linaro.org Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
* tcg/optimize: Don't fold INDEX_op_and_vec to extractRichard Henderson2025-07-211-1/+1
| | | | | | | | | | | There is no such thing as vector extract. Fixes: 932522a9ddc1 ("tcg/optimize: Fold and to extract during optimize") Resolves: https://gitlab.com/qemu-project/qemu/-/issues/3036 Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Tested-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org>
* tcg: Use uintptr_t in tcg_malloc implementationRichard Henderson2025-07-111-4/+5
| | | | | | | | | | Avoid ubsan failure with clang-20, tcg.h:715:19: runtime error: applying non-zero offset 64 to null pointer by not using pointers. Acked-by: Ilya Leoshkevich <iii@linux.ibm.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Fix constant propagation in tcg_reg_alloc_dupRichard Henderson2025-06-301-1/+1
| | | | | | | | | The scalar constant must be replicated for dup. Cc: qemu-stable@nongnu.org Fixes: bab1671f0fa ("tcg: Manually expand INDEX_op_dup_vec") Resolves: https://gitlab.com/qemu-project/qemu/-/issues/3002 Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/riscv: Fix typo in tgen_extractRichard Henderson2025-06-301-1/+1
| | | | | | | | | | | | Fix the direction of the shift, introduced when converting the codebase to TCGOutOp* and small tgen_* helpers. Fixes: 5a4d034f3cb ("tcg: Convert extract to TCGOutOpExtract") Reported-by: Andrea Bolognani <abologna@redhat.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Tested-by: Andrea Bolognani <abologna@redhat.com> Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
* tcg/optimize: Simplify fold_eqv constant checksRichard Henderson2025-06-301-3/+1
| | | | | | | Both cases are handled by fold_xor after conversion. Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/optimize: Simplify fold_orc constant checksRichard Henderson2025-06-301-5/+5
| | | | | | | | | | If operand 2 is constant, then the computation of z_mask and a_mask will produce the same results as the explicit check via fold_xi_to_i. Shift the calls of fold_xx_to_i and fold_ix_to_not down below the i2->is_const check. Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/optimize: Simplify fold_andc constant checksRichard Henderson2025-06-301-4/+5
| | | | | | | | | | If operand 2 is constant, then the computation of z_mask and a_mask will produce the same results as the explicit check via fold_xi_to_i. Shift the calls of fold_xx_to_i and fold_ix_to_not down below the i2->is_const check. Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/optimize: Simplify fold_and constant checksRichard Henderson2025-06-301-4/+3
| | | | | | | | | | If operand 2 is constant, then the computation of z_mask and a_mask will produce the same results as the explicit checks via fold_xi_to_i and fold_xi_to_x. Shift the call of fold_xx_to_x down below the ti_is_const(t2) check. Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/optimize: Fold and to extract during optimizeRichard Henderson2025-06-301-3/+30
| | | | | Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/optimize: Use fold_and in do_constant_folding_cond[12]Richard Henderson2025-06-301-0/+5
| | | | | | | | When lowering tst comparisons, completely fold the and opcode that we generate. Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/optimize: Build and use o_bits in fold_shiftRichard Henderson2025-06-301-2/+4
| | | | | Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/optimize: Build and use o_bits in fold_sextractRichard Henderson2025-06-301-24/+6
| | | | | | | | This was the last use of fold_affected_mask, now fully replaced by fold_masks_zosa. Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/optimize: Build and use o_bits in fold_movcondRichard Henderson2025-06-301-2/+3
| | | | | Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/optimize: Build and use o_bits in fold_extuRichard Henderson2025-06-301-3/+9
| | | | | Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/optimize: Build and use o_bits in fold_extsRichard Henderson2025-06-301-2/+4
| | | | | Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/optimize: Build and use z_bits and o_bits in fold_extract2Richard Henderson2025-06-301-13/+25
| | | | | Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/optimize: Build and use o_bits in fold_extractRichard Henderson2025-06-301-7/+5
| | | | | Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/optimize: Build and use o_bits in fold_depositRichard Henderson2025-06-301-2/+4
| | | | | Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/optimize: Build and use o_bits in fold_bswapRichard Henderson2025-06-301-25/+24
| | | | | Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/optimize: Build and use o_bits in fold_xorRichard Henderson2025-06-301-3/+6
| | | | | Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/optimize: Build and use zero, one and affected bits in fold_orcRichard Henderson2025-06-301-2/+9
| | | | | Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/optimize: Build and use one and affected bits in fold_orRichard Henderson2025-06-301-2/+8
| | | | | Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/optimize: Build and use z_bits and o_bits in fold_notRichard Henderson2025-06-301-1/+5
| | | | | Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/optimize: Build and use z_bits and o_bits in fold_norRichard Henderson2025-06-301-4/+10
| | | | | Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/optimize: Build and use z_bits and o_bits in fold_nandRichard Henderson2025-06-301-4/+10
| | | | | Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/optimize: Build and use z_bits and o_bits in fold_eqvRichard Henderson2025-06-301-2/+12
| | | | | Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/optimize: Build and use o_bits in fold_andcRichard Henderson2025-06-301-15/+8
| | | | | Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/optimize: Build and use o_bits in fold_andRichard Henderson2025-06-301-13/+7
| | | | | Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/optimize: Introduce fold_masks_zosaRichard Henderson2025-06-301-5/+11
| | | | | | | | | Add a new function with an affected mask. This will allow folding to a constant to happen before folding to a copy, without having to mind the ordering in all users. Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/optimize: Add one's mask to TempOptInfoRichard Henderson2025-06-301-16/+35
| | | | | | | | Add o_mask mirroring z_mask, but for 1's instead of 0's. Drop is_const and val fields, which now logically overlap. Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/optimize: Introduce arg_const_valRichard Henderson2025-06-301-37/+41
| | | | | | | | Use arg_const_val instead of direct access to the TempOptInfo val member. Rename both val and is_const to catch all direct accesses. Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Split out tcg_gen_gvec_dup_imm_varRichard Henderson2025-06-231-2/+8
| | | | | Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Split out tcg_gen_gvec_{add,sub}_varRichard Henderson2025-06-231-6/+26
| | | | | Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Split out tcg_gen_gvec_mov_varRichard Henderson2025-06-231-6/+15
| | | | | Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Split out tcg_gen_gvec_3_varRichard Henderson2025-06-231-38/+64
| | | | | Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Split out tcg_gen_gvec_2_varRichard Henderson2025-06-231-32/+53
| | | | | Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Add base arguments to check_overlap_[234]Richard Henderson2025-06-231-25/+36
| | | | | Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Add dbase argument to expand_clrRichard Henderson2025-06-231-18/+18
| | | | | Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Add dbase argument to do_dupRichard Henderson2025-06-231-16/+17
| | | | | Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Add dbase argument to do_dup_storeRichard Henderson2025-06-231-8/+8
| | | | | Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>