summary refs log tree commit diff stats
path: root/tcg/tcg-op-gvec.c (follow)
Commit message (Collapse)AuthorAgeFilesLines
* tcg: Split out tcg_gen_gvec_dup_imm_varRichard Henderson2025-06-231-2/+8
| | | | | Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Split out tcg_gen_gvec_{add,sub}_varRichard Henderson2025-06-231-6/+26
| | | | | Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Split out tcg_gen_gvec_mov_varRichard Henderson2025-06-231-6/+15
| | | | | Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Split out tcg_gen_gvec_3_varRichard Henderson2025-06-231-38/+64
| | | | | Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Split out tcg_gen_gvec_2_varRichard Henderson2025-06-231-32/+53
| | | | | Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Add base arguments to check_overlap_[234]Richard Henderson2025-06-231-25/+36
| | | | | Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Add dbase argument to expand_clrRichard Henderson2025-06-231-18/+18
| | | | | Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Add dbase argument to do_dupRichard Henderson2025-06-231-16/+17
| | | | | Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Add dbase argument to do_dup_storeRichard Henderson2025-06-231-8/+8
| | | | | Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Only include 'tcg-has.h' when necessaryRichard Henderson2025-01-161-0/+1
| | | | | | | | | TCG_TARGET_HAS_* definitions don't need to be exposed by "tcg/tcg.h". Only include 'tcg-has.h' when necessary. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-ID: <20250108215156.8731-15-philmd@linaro.org>
* tcg: Allow top bit of SIMD_DATA_BITS to be set in simd_desc()Peter Maydell2024-11-161-1/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | In simd_desc() we create a SIMD descriptor from various pieces including an arbitrary data value from the caller. We try to sanitize these to make sure everything will fit: the 'data' value needs to fit in the SIMD_DATA_BITS (== 22) sized field. However we do that sanitizing with: tcg_debug_assert(data == sextract32(data, 0, SIMD_DATA_BITS)); This works for the case where the data is supposed to be considered as a signed integer (which can then be returned via simd_data()). However, some callers want to treat the data value as unsigned. Specifically, for the Arm SVE operations, make_svemte_desc() assembles a data value as a collection of fields, and it needs to use all 22 bits. Currently if MTE is enabled then its MTEDESC SIZEM1 field may have the most significant bit set, and then it will trip this assertion. Loosen the assertion so that we only check that the data value will fit into the field in some way, either as a signed or as an unsigned value. This means we will fail to detect some kinds of bug in the callers, but we won't spuriously assert for intentional use of the data field as unsigned. Cc: qemu-stable@nongnu.org Fixes: db432672dc50e ("tcg: Add generic vector expanders") Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2601 Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Message-ID: <20241115172515.1229393-1-peter.maydell@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Fix iteration step in 32-bit gvec operationTANG Tiancheng2024-09-221-1/+1
| | | | | | | | | | | | | | | | The loop in the 32-bit case of the vector compare operation was incorrectly incrementing by 8 bytes per iteration instead of 4 bytes. This caused the function to process only half of the intended elements. Cc: qemu-stable@nongnu.org Fixes: 9622c697d1 (tcg: Add gvec compare with immediate and scalar operand) Signed-off-by: TANG Tiancheng <tangtiancheng.ttc@alibaba-inc.com> Reviewed-by: Liu Zhiwei <zhiwei_liu@linux.alibaba.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-ID: <20240904142739.854-2-zhiwei_liu@linux.alibaba.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
* tcg: Add write_aofs to GVecGen3iRichard Henderson2024-05-061-8/+22
| | | | | Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Don't free vector resultsRichard Henderson2023-11-061-74/+38
| | | | | | | | Avoid reusing vector temporaries so that we may re-use them when propagating stores to loads. Reviewed-by: Song Gao <gaosong@loongson.cn> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Rename cpu_env to tcg_envRichard Henderson2023-10-031-150/+150
| | | | | | | Allow the name 'cpu_env' to be used for something else. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Add gvec compare with immediate and scalar operandRichard Henderson2023-09-161-0/+149
| | | | | | | Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Tested-by: Song Gao <gaosong@loongson.cn> Reviewed-by: Song Gao <gaosong@loongson.cn> Message-Id: <20230831030904.1194667-2-richard.henderson@linaro.org>
* tcg: Use tcg_gen_negsetcond_*Richard Henderson2023-08-241-4/+2
| | | | | | Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Fix temporary variable in tcg_gen_gvec_andcsMax Chou2023-06-261-1/+1
| | | | | | | | | | The 5th parameter of tcg_gen_gvec_2s should be replaced by the temporary tmp variable in the tcg_gen_gvec_andcs function. Reviewed-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> Signed-off-by: Max Chou <max.chou@sifive.com> Message-Id: <20230622161646.32005-9-max.chou@sifive.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Split tcg/tcg-op-gvec.hRichard Henderson2023-06-051-1/+1
| | | | | | | | Create tcg/tcg-op-gvec-common.h, moving everything that does not concern TARGET_LONG_BITS. Adjust tcg-op-gvec.c to use the new header. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Split tcg/tcg-op-common.h from tcg/tcg-op.hRichard Henderson2023-06-051-1/+1
| | | | | | | | | Create tcg/tcg-op-common.h, moving everything that does not concern TARGET_LONG_BITS or TCGv. Adjust tcg/*.c to use the new header instead of tcg-op.h, in preparation for compiling tcg/ only once. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Add tcg_gen_gvec_rotrsNazar Kazakov2023-05-021-0/+11
| | | | | | | | | | Add tcg expander and helper functions for rotate right vector with scalar operand. Signed-off-by: Nazar Kazakov <nazar.kazakov@codethink.co.uk> Message-Id: <20230428144757.57530-10-lawrence.hunter@codethink.co.uk> [rth: Split out of larger patch; mask rotation count.] Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Add tcg_gen_gvec_andcsNazar Kazakov2023-05-021-0/+17
| | | | | | | | | | Add tcg expander and helper functions for and-compliment vector with scalar operand. Signed-off-by: Nazar Kazakov <nazar.kazakov@codethink.co.uk> Message-Id: <20230428144757.57530-10-lawrence.hunter@codethink.co.uk> [rth: Split out of larger patch.] Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Create tcg/tcg-temp-internal.hRichard Henderson2023-03-131-0/+1
| | | | | | | | | | | | Move the tcg_temp_free_* and tcg_temp_ebb_new_* declarations and inlines to the new header. These are private to the implementation, and will prevent tcg_temp_free_* from creeping back into the guest front ends. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Peter Maydell <peter.maydell@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Use tcg_constant_ptr in do_dupRichard Henderson2023-03-011-2/+1
| | | | | | Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Use tcg_temp_ebb_new_* in tcg/Richard Henderson2023-03-011-93/+93
| | | | | | | All of these have obvious and quite local scope. Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg/tcg-op-gvec: Remove unused "qemu/main-loop.h" headerPhilippe Mathieu-Daudé2023-02-271-1/+0
| | | | | | Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-Id: <20221219170806.60580-3-philmd@linaro.org>
* tcg/tcg-op-gvec.c: Introduce tcg_gen_gvec_4iMatheus Ferst2022-03-021-0/+146
| | | | | | | | | | Following the implementation of tcg_gen_gvec_3i, add a four-vector and immediate operand expansion method. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Matheus Ferst <matheus.ferst@eldorado.org.br> Message-Id: <20220225210936.1749575-34-matheus.ferst@eldorado.org.br> Signed-off-by: Cédric Le Goater <clg@kaod.org>
* tcg: Add tcg_gen_vec_shl{shr}{sar}8i_i32LIU Zhiwei2021-06-291-0/+28
| | | | | | | | Implement tcg_gen_vec_shl{shr}{sar}8i_tl by adding corresponging i32 OP. Signed-off-by: LIU Zhiwei <zhiwei_liu@c-sky.com> Message-Id: <20210624105023.3852-5-zhiwei_liu@c-sky.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Add tcg_gen_vec_shl{shr}{sar}16i_i32LIU Zhiwei2021-06-291-0/+28
| | | | | | | | Implement tcg_gen_vec_shl{shr}{sar}16i_tl by adding corresponging i32 OP. Signed-off-by: LIU Zhiwei <zhiwei_liu@c-sky.com> Message-Id: <20210624105023.3852-4-zhiwei_liu@c-sky.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Add tcg_gen_vec_add{sub}8_i32LIU Zhiwei2021-06-291-0/+38
| | | | | | | | Implement tcg_gen_vec_add{sub}8_tl by adding corresponging i32 OP. Signed-off-by: LIU Zhiwei <zhiwei_liu@c-sky.com> Message-Id: <20210624105023.3852-3-zhiwei_liu@c-sky.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Add tcg_gen_vec_add{sub}16_i32LIU Zhiwei2021-06-291-0/+28
| | | | | | | | Implement tcg_gen_vec_add{sub}16_tl by adding corresponding i32 OP. Signed-off-by: LIU Zhiwei <zhiwei_liu@c-sky.com> Message-Id: <20210624105023.3852-2-zhiwei_liu@c-sky.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Make gen_dup_i32/i64() public as tcg_gen_dup_i32/i64Peter Maydell2021-06-211-10/+10
| | | | | | | | | | | | | | | | | The Arm MVE VDUP implementation would like to be able to emit code to duplicate a byte or halfword value into an i32. We have code to do this already in tcg-op-gvec.c, so all we need to do is make the functions global. For consistency with other functions made available to the frontends: * we rename to tcg_gen_dup_* * we expose both the _i32 and _i64 forms * we provide the #define for a _tl form Suggested-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Peter Maydell <peter.maydell@linaro.org> Message-id: 20210617121628.20116-10-peter.maydell@linaro.org
* tcg: Use tcg_constant_{i32,i64,vec} with gvec expandersRichard Henderson2021-01-131-77/+50
| | | | Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Use memset for large vector byte replicationRichard Henderson2021-01-041-0/+32
| | | | | | | | | | | | | | | | | | In f47db80cc07, we handled odd-sized tail clearing for the case of hosts that have vector operations, but did not handle the case of hosts that do not have vector ops. This was ok until e2e7168a214b, which changed the encoding of simd_desc such that the odd sizes are impossible. Add memset as a tcg helper, and use that for all out-of-line byte stores to vectors. This includes, but is not limited to, the tail clearing operation in question. Cc: qemu-stable@nongnu.org Buglink: https://bugs.launchpad.net/bugs/1907817 Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Adjust simd_desc size encodingRichard Henderson2020-10-081-8/+27
| | | | | | | | | | | | | | With larger vector sizes, it turns out oprsz == maxsz, and we only need to represent mismatch for oprsz <= 32. We do, however, need to represent larger oprsz and do so without reducing SIMD_DATA_BITS. Reduce the size of the oprsz field and increase the maxsz field. Steal the oprsz value of 24 to indicate equality with maxsz. Tested-by: Frank Chang <frank.chang@sifive.com> Reviewed-by: Frank Chang <frank.chang@sifive.com> Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Implement 256-bit dup for tcg_gen_gvec_dup_memRichard Henderson2020-09-031-3/+49
| | | | | | | | We already support duplication of 128-bit blocks. This extends that support to 256-bit blocks. This will be needed by SVE2. Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Eliminate one store for in-place 128-bit dup_memRichard Henderson2020-09-031-2/+2
| | | | | | | Do not store back to the exact memory from which we just loaded. Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Fix tcg gen for vectorized absolute valueStephen Long2020-09-031-2/+3
| | | | | | | | | | | | | | The fallback inline expansion for vectorized absolute value, when the host doesn't support such an insn was flawed. E.g. when a vector of bytes has all elements negative, mask will be 0xffff_ffff_ffff_ffff. Subtracting mask only adds 1 to the low element instead of all elements becase -mask is 1 and not 0x0101_0101_0101_0101. Signed-off-by: Stephen Long <steplong@quicinc.com> Message-Id: <20200813161818.190-1-steplong@quicinc.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Implement gvec support for rotate by scalarRichard Henderson2020-06-021-0/+22
| | | | | | | | | | No host backend support yet, but the interfaces for rotls are in place. Only implement left-rotate for now, as the only known use of vector rotate by scalar is s390x, so any right-rotate would be unused and untestable. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Implement gvec support for rotate by vectorRichard Henderson2020-06-021-0/+122
| | | | | | | | | | | No host backend support yet, but the interfaces for rotlv and rotrv are in place. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> --- v3: Drop the generic expansion from rot to shift; we can do better for each backend, and then this code becomes unused.
* tcg: Implement gvec support for rotate by immediateRichard Henderson2020-06-021-0/+68
| | | | | | | | | | No host backend support yet, but the interfaces for rotli are in place. Canonicalize immediate rotate to the left, based on a survey of architectures, but provide both left and right shift interfaces to the translators. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Add load_dest parameter to GVecGen2Richard Henderson2020-05-061-13/+32
| | | | | | | | | We have this same parameter for GVecGen2i, GVecGen3, and GVecGen3i. This will make some SVE2 insns easier to parameterize. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Improve vector tail clearingRichard Henderson2020-05-061-20/+64
| | | | | | | | Better handling of non-power-of-2 tails as seen with Arm 8-byte vector operations. Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Remove tcg_gen_gvec_dup{8,16,32,64}iRichard Henderson2020-05-061-28/+0
| | | | | | | | | These interfaces are now unused. Reviewed-by: LIU Zhiwei <zhiwei_liu@c-sky.com> Reviewed-by: David Hildenbrand <david@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Use tcg_gen_gvec_dup_imm in logical simplificationsRichard Henderson2020-05-061-4/+4
| | | | | | | | Replace the outgoing interface. Reviewed-by: LIU Zhiwei <zhiwei_liu@c-sky.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Add tcg_gen_gvec_dup_immRichard Henderson2020-05-061-0/+7
| | | | | | | | | | | | Add a version of tcg_gen_dup_* that takes both immediate and a vector element size operand. This will replace the set of tcg_gen_gvec_dup{8,16,32,64}i functions that encode the element size within the function name. Reviewed-by: LIU Zhiwei <zhiwei_liu@c-sky.com> Reviewed-by: David Hildenbrand <david@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Add tcg_gen_gvec_5_ptrRichard Henderson2020-02-121-0/+32
| | | | | | | | | | Extend the vector generator infrastructure to handle 5 vector arguments. Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Taylor Simpson <tsimpson@quicinc.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* tcg: Search includes from the project root source directoryPhilippe Mathieu-Daudé2020-01-151-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | We currently search both the root and the tcg/ directories for tcg files: $ git grep '#include "tcg/' | wc -l 28 $ git grep '#include "tcg[^/]' | wc -l 94 To simplify the preprocessor search path, unify by expliciting the tcg/ directory. Patch created mechanically by running: $ for x in \ tcg.h tcg-mo.h tcg-op.h tcg-opc.h \ tcg-op-gvec.h tcg-gvec-desc.h; do \ sed -i "s,#include \"$x\",#include \"tcg/$x\"," \ $(git grep -l "#include \"$x\""); \ done Acked-by: David Gibson <david@gibson.dropbear.id.au> (ppc parts) Reviewed-by: Paolo Bonzini <pbonzini@redhat.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Stefan Weil <sw@weilnetz.de> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com> Message-Id: <20200101112303.20724-2-philmd@redhat.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* Include qemu/main-loop.h lessMarkus Armbruster2019-08-161-0/+1
| | | | | | | | | | | | | | | | | | | | In my "build everything" tree, changing qemu/main-loop.h triggers a recompile of some 5600 out of 6600 objects (not counting tests and objects that don't depend on qemu/osdep.h). It includes block/aio.h, which in turn includes qemu/event_notifier.h, qemu/notify.h, qemu/processor.h, qemu/qsp.h, qemu/queue.h, qemu/thread-posix.h, qemu/thread.h, qemu/timer.h, and a few more. Include qemu/main-loop.h only where it's needed. Touching it now recompiles only some 1700 objects. For block/aio.h and qemu/event_notifier.h, these numbers drop from 5600 to 2800. For the others, they shrink only slightly. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190812052359.30071-21-armbru@redhat.com> Reviewed-by: Alex Bennée <alex.bennee@linaro.org> Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com> Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
* Include qemu-common.h exactly where neededMarkus Armbruster2019-06-121-1/+0
| | | | | | | | | | | | | | | | No header includes qemu-common.h after this commit, as prescribed by qemu-common.h's file comment. Signed-off-by: Markus Armbruster <armbru@redhat.com> Message-Id: <20190523143508.25387-5-armbru@redhat.com> [Rebased with conflicts resolved automatically, except for include/hw/arm/xlnx-zynqmp.h hw/arm/nrf51_soc.c hw/arm/msf2-soc.c block/qcow2-refcount.c block/qcow2-cluster.c block/qcow2-cache.c target/arm/cpu.h target/lm32/cpu.h target/m68k/cpu.h target/mips/cpu.h target/moxie/cpu.h target/nios2/cpu.h target/openrisc/cpu.h target/riscv/cpu.h target/tilegx/cpu.h target/tricore/cpu.h target/unicore32/cpu.h target/xtensa/cpu.h; bsd-user/main.c and net/tap-bsd.c fixed up]