summary refs log tree commit diff stats
path: root/target/riscv/op_helper.c (follow)
Commit message (Collapse)AuthorAgeFilesLines
* target/riscv: Fix ssamoswap error handlingJim Shu2025-10-031-0/+49
| | | | | | | | | | | | Follow the RISC-V CFI v1.0 spec [1] to fix the exception type when ssamoswap is disabled by xSSE. [1] RISC-V CFI spec v1.0, ch2.7 Atomic Swap from a Shadow Stack Location Signed-off-by: Jim Shu <jim.shu@sifive.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Message-ID: <20250924074818.230010-4-jim.shu@sifive.com> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: do not call GETPC() in check_ret_from_m_mode()Daniel Henrique Barboza2025-07-301-6/+9
| | | | | | | | | | | | | | | | | | | | | | GETPC() should always be called from the top level helper, e.g. the first helper that is called by the translation code. We stopped doing that in commit 3157a553ec, and then we introduced problems when unwinding the exceptions being thrown by helper_mret(), as reported by [1]. Call GETPC() at the top level helper and pass the value along. [1] https://gitlab.com/qemu-project/qemu/-/issues/3020 Suggested-by: Richard Henderson <richard.henderson@linaro.org> Fixes: 3157a553ec ("target/riscv: Add Smrnmi mnret instruction") Closes: https://gitlab.com/qemu-project/qemu/-/issues/3020 Signed-off-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> Reviewed-by: Nutty Liu <liujingqi@lanxincomputing.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-ID: <20250714133739.1248296-1-dbarboza@ventanamicro.com> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: Fix MEPC/SEPC bit masking for IALIGNCharalampos Mitrodimas2025-07-041-2/+2
| | | | | | | | | | | | | | | | | | | | | | | According to the RISC-V Privileged Architecture specification, the low bit of MEPC/SEPC must always be zero. When IALIGN=32, the two low bits must be zero. This commit fixes the behavior of MEPC/SEPC CSR reads and writes, and the implicit reads by MRET/SRET instructions to properly mask the lowest bit(s) based on whether the C extension is enabled: - When C extension is enabled (IALIGN=16): mask bit 0 - When C extension is disabled (IALIGN=32): mask bits [1:0] Previously, when vectored mode bits from STVEC (which sets bit 0 for vectored mode) were written to MEPC, the bits would not be cleared correctly, causing incorrect behavior on MRET. Resolves: https://gitlab.com/qemu-project/qemu/-/issues/2855 Signed-off-by: Charalampos Mitrodimas <charmitro@posteo.net> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> Message-ID: <20250703182157.281320-2-charmitro@posteo.net> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: Pass ra to riscv_csrrw_i128Richard Henderson2025-05-191-4/+5
| | | | | | | | | Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-ID: <20250425152311.804338-6-richard.henderson@linaro.org> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: Pass ra to riscv_csrrwRichard Henderson2025-05-191-2/+2
| | | | | | | | | Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-ID: <20250425152311.804338-5-richard.henderson@linaro.org> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* include: Remove 'exec/exec-all.h'Philippe Mathieu-Daudé2025-04-301-1/+0
| | | | | | | | | | | | | "exec/exec-all.h" is now fully empty, let's remove it. Mechanical change running: $ sed -i '/exec\/exec-all.h/d' $(git grep -wl exec/exec-all.h) Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Mark Cave-Ayland <mark.caveayland@nutanix.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-ID: <20250424202412.91612-14-philmd@linaro.org>
* accel/tcg: Extract probe API out of 'exec/exec-all.h'Philippe Mathieu-Daudé2025-04-301-0/+1
| | | | | | | | | | | Declare probe methods in "accel/tcg/probe.h" to emphasize they are specific to TCG accelerator. Suggested-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Mark Cave-Ayland <mark.caveayland@nutanix.com> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-ID: <20250424202412.91612-13-philmd@linaro.org>
* exec: Restrict 'cpu_ldst.h' to accel/tcg/Philippe Mathieu-Daudé2025-04-231-1/+1
| | | | | | | | | | | | Mechanical change using: $ sed -i -e 's,exec/cpu_ldst,accel/tcg/cpu-ldst,' \ $(git grep -l exec/cpu_ldst.h) Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* exec/cpu-all: extract tlb flags defines to exec/tlb-flags.hPierrick Bouvier2025-04-231-0/+1
| | | | | | | Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Message-ID: <20250320223002.2915728-3-pierrick.bouvier@linaro.org>
* target/riscv: Add check for 16-bit aligned PC for different priv versions.Yu-Ming Chang2025-03-191-2/+6
| | | | | | | | | | | For privilege version 1.12 or newer, C always implies Zca. We can only check ext_zca to allow 16-bit aligned PC addresses. For older privilege versions, we only check C. Signed-off-by: Yu-Ming Chang <yumin686@andestech.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Message-ID: <174184718265.10540.10120024221661781046-0@git.sr.ht> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* exec: Declare tlb_flush*() in 'exec/cputlb.h'Philippe Mathieu-Daudé2025-03-081-0/+1
| | | | | | | | | | Move CPU TLB related methods to "exec/cputlb.h". Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> Message-ID: <20241114011310.3615-19-philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* target/riscv: Add CTR sctrclr instruction.Rajnesh Kanwal2025-03-041-0/+29
| | | | | | | | | | CTR extension adds a new instruction sctrclr to quickly clear the recorded entries buffer. Signed-off-by: Rajnesh Kanwal <rkanwal@rivosinc.com> Acked-by: Alistair Francis <alistair.francis@wdc.com> Message-ID: <20250205-b4-ctr_upstream_v6-v6-5-439d8e06c8ef@rivosinc.com> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: Add support to record CTR entries.Rajnesh Kanwal2025-03-041-0/+19
| | | | | | | | | | | | | | This commit adds logic to records CTR entries of different types and adds required hooks in TCG and interrupt/Exception logic to record events. This commit also adds support to invoke freeze CTR logic for breakpoint exceptions and counter overflow interrupts. Signed-off-by: Rajnesh Kanwal <rkanwal@rivosinc.com> Acked-by: Alistair Francis <alistair.francis@wdc.com> Message-ID: <20250205-b4-ctr_upstream_v6-v6-4-439d8e06c8ef@rivosinc.com> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: Implement Smdbltrp sret, mret and mnret behaviorClément Léger2025-01-191-0/+12
| | | | | | | | | | | When the Ssdbltrp extension is enabled, SSTATUS.MDT field is cleared when executing sret if executed in M-mode. When executing mret/mnret, SSTATUS.MDT is cleared. Signed-off-by: Clément Léger <cleger@rivosinc.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Message-ID: <20250110125441.3208676-8-cleger@rivosinc.com> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: Implement Ssdbltrp sret, mret and mnret behaviorClément Léger2025-01-191-1/+34
| | | | | | | | | | | | When the Ssdbltrp extension is enabled, SSTATUS.SDT field is cleared when executing sret. When executing mret/mnret, SSTATUS.SDT is cleared when returning to U, VS or VU and VSSTATUS.SDT is cleared when returning to VU from HS. Signed-off-by: Clément Léger <cleger@rivosinc.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Message-ID: <20250110125441.3208676-4-cleger@rivosinc.com> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: Add Zicfilp support for SmrnmiFrank Chang2025-01-191-0/+9
| | | | | | | | | | | | | | | | Zicfilp extension introduces the MNPELP (bit 9) in mnstatus. The MNPELP field holds the previous ELP. When a RNMI trap is delivered, the MNPELP is set to ELP and ELP set to NO_LP_EXPECTED. Upon a mnret, if the mnstatus.MNPP holds the value y, then ELP is set to the value of MNPELP if yLPE is 1; otherwise, it is set to NO_LP_EXPECTED. Signed-off-by: Frank Chang <frank.chang@sifive.com> Reviewed-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Message-ID: <20250106054336.1878291-7-frank.chang@sifive.com> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: Add Smrnmi mnret instructionTommy Wu2025-01-191-5/+40
| | | | | | | | | | | | | | This patch adds a new instruction 'mnret'. 'mnret' is an M-mode-only instruction that uses the values in `mnepc` and `mnstatus` to return to the program counter, privilege mode, and virtualization mode of the interrupted context. Signed-off-by: Frank Chang <frank.chang@sifive.com> Signed-off-by: Tommy Wu <tommy.wu@sifive.com> Reviewed-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Message-ID: <20250106054336.1878291-5-frank.chang@sifive.com> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: Apply pointer masking for virtualized memory accessesAlexey Baturo2025-01-191-8/+8
| | | | | | | | Signed-off-by: Alexey Baturo <baturo.alexey@gmail.com> Reviewed-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Message-ID: <20250106102346.1100149-7-baturo.alexey@gmail.com> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: add trace in riscv_raise_exception()Daniel Henrique Barboza2025-01-191-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When using system mode we can get the CPU traps being taken via the 'riscv_trap' trace or the "-d int" qemu log. User mode does not a way of logging/showing exceptions to users. Add a trace in riscv_raise_exception() to allow qemu-riscv(32/64) users to check all exceptions being thrown. This is particularly useful to help identifying insns that are throwing SIGILLs. As it is today we need to debug their binaries to identify where the illegal insns are: $ ~/work/qemu/build/qemu-riscv64 -cpu rv64 ./foo.out Illegal instruction (core dumped) After this change users can capture the trace and use EPC to pinpoint the insn: $ ~/work/qemu/build/qemu-riscv64 -cpu rv64 -trace riscv_exception ./foo.out riscv_exception 8 (user_ecall) on epc 0x17cd2 riscv_exception 8 (user_ecall) on epc 0x17cda riscv_exception 8 (user_ecall) on epc 0x17622 (...) riscv_exception 2 (illegal_instruction) on epc 0x1053a Illegal instruction (core dumped) Signed-off-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Message-ID: <20250106173734.412353-3-dbarboza@ventanamicro.com> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: use RISCVException enum in exception helpersDaniel Henrique Barboza2025-01-191-1/+2
| | | | | | | | | | | | | | | | Do a cosmetic change in riscv_raise_exception() to change 'exception' type from uint32_t to RISCVException, making it a bit clear that the arg is directly correlated to the RISCVException enum. As a side effect, change 'excp' type from int to RISCVException in generate_exception() to guarantee that all callers of riscv_raise_exception() will use the enum. Signed-off-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Message-ID: <20250106173734.412353-2-dbarboza@ventanamicro.com> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: save and restore elp state on priv transitionsDeepak Gupta2024-10-301-0/+17
| | | | | | | | | | | | | | | | | | elp state is recorded in *status on trap entry (less privilege to higher privilege) and restored in elp from *status on trap exit (higher to less privilege). Additionally this patch introduces a forward cfi helper function to determine if current privilege has forward cfi is enabled or not based on *envcfg (for U, VU, S, VU, HS) or mseccfg csr (for M). Signed-off-by: Deepak Gupta <debug@rivosinc.com> Co-developed-by: Jim Shu <jim.shu@sifive.com> Co-developed-by: Andy Chiu <andy.chiu@sifive.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Message-ID: <20241008225010.1861630-5-debug@rivosinc.com> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: raise an exception when CSRRS/CSRRC writes a read-only CSRYu-Ming Chang2024-07-181-4/+2
| | | | | | | | | | | | | | | | | Both CSRRS and CSRRC always read the addressed CSR and cause any read side effects regardless of rs1 and rd fields. Note that if rs1 specifies a register holding a zero value other than x0, the instruction will still attempt to write the unmodified value back to the CSR and will cause any attendant side effects. So if CSRRS or CSRRC tries to write a read-only CSR with rs1 which specifies a register holding a zero value, an illegal instruction exception should be raised. Signed-off-by: Yu-Ming Chang <yumin686@andestech.com> Signed-off-by: Alvin Chang <alvinga@andestech.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Message-ID: <172100444279.18077.6893072378718059541-0@git.sr.ht> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: Combine set_mode and set_virt functions.Rajnesh Kanwal2024-07-181-11/+6
| | | | | | | | | | | | | | | | | | | Combining riscv_cpu_set_virt_enabled() and riscv_cpu_set_mode() functions. This is to make complete mode change information available through a single function. This allows to easily differentiate between HS->VS, VS->HS and VS->VS transitions when executing state update codes. For example: One use-case which inspired this change is to update mode-specific instruction and cycle counters which requires information of both prev mode and current mode. Signed-off-by: Rajnesh Kanwal <rkanwal@rivosinc.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> Message-ID: <20240711-smcntrpmf_v7-v8-1-b7c38ae7b263@rivosinc.com> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: Raise exceptions on wrs.ntoAndrew Jones2024-06-031-0/+11
| | | | | | | | | | | | | | | | | | | | Implementing wrs.nto to always just return is consistent with the specification, as the instruction is permitted to terminate the stall for any reason, but it's not useful for virtualization, where we'd like the guest to trap to the hypervisor in order to allow scheduling of the lock holding VCPU. Change to always immediately raise exceptions when the appropriate conditions are present, otherwise continue to just return. Note, immediately raising exceptions is also consistent with the specification since the time limit that should expire prior to the exception is implementation-specific. Signed-off-by: Andrew Jones <ajones@ventanamicro.com> Reviewed-by: Christoph Müllner <christoph.muellner@vrull.eu> Reviewed-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Message-ID: <20240424142808.62936-2-ajones@ventanamicro.com> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: Replace cpu_mmu_index with riscv_env_mmu_indexRichard Henderson2024-02-031-2/+2
| | | | | | | | | Use the target-specific function name in preference to the generic name. Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
* target/helpers: Remove unnecessary 'qemu/main-loop.h' headerPhilippe Mathieu-Daudé2023-08-311-1/+0
| | | | | | | | | | "qemu/main-loop.h" declares functions related to QEMU's main loop mutex, which these files don't access. Remove the unused "qemu/main-loop.h" header. Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-Id: <20230828221314.18435-8-philmd@linaro.org>
* target/translate: Include missing 'exec/cpu_ldst.h' headerPhilippe Mathieu-Daudé2023-08-311-0/+1
| | | | | | | | All these files access the CPU LD/ST API declared in "exec/cpu_ldst.h". Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-Id: <20230828221314.18435-4-philmd@linaro.org>
* target/riscv: Make MPV only work when MPP != PRV_MWeiwei Li2023-07-101-1/+2
| | | | | | | | | | | | Upon MRET or explicit memory access with MPRV=1, MPV should be ignored when MPP=PRV_M. Signed-off-by: Weiwei Li <liweiwei@iscas.ac.cn> Signed-off-by: Junqiang Wang <wangjunqiang@iscas.ac.cn> Reviewed-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Message-Id: <20230603134236.15719-2-liweiwei@iscas.ac.cn> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: Check SUM in the correct registerRichard Henderson2023-05-051-1/+5
| | | | | | | | | | | | | | | | | Table 9.5 "Effect of MPRV..." specifies that MPV=1 uses VS-level vsstatus.SUM instead of HS-level sstatus.SUM. For HLV/HSV instructions, the HS-level register does not apply, but the VS-level register presumably does, though this is not mentioned explicitly in the manual. However, it matches the behavior for MPV. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Weiwei Li <liweiwei@iscas.ac.cn> Tested-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> Message-Id: <20230325105429.1142530-18-richard.henderson@linaro.org> Message-Id: <20230412114333.118895-18-richard.henderson@linaro.org> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: Move hstatus.spvp check to check_access_hlsvRichard Henderson2023-05-051-1/+1
| | | | | | | | | | | | | The current cpu_mmu_index value is really irrelevant to the HLV/HSV lookup. Provide the correct priv level directly. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Weiwei Li <liweiwei@iscas.ac.cn> Tested-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> Message-Id: <20230325105429.1142530-16-richard.henderson@linaro.org> Message-Id: <20230412114333.118895-16-richard.henderson@linaro.org> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: Rename MMU_HYP_ACCESS_BIT to MMU_2STAGE_BITRichard Henderson2023-05-051-1/+1
| | | | | | | | | | | | We will enable more uses of this bit in the future. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Weiwei Li <liweiwei@iscas.ac.cn> Tested-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> Message-Id: <20230325105429.1142530-12-richard.henderson@linaro.org> Message-Id: <20230412114333.118895-12-richard.henderson@linaro.org> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: Handle HLV, HSV via helpersRichard Henderson2023-05-051-6/+93
| | | | | | | | | | | | | | | Implement these instructions via helpers, in expectation of determining the mmu_idx to use at runtime. This allows the permission check to also be moved out of line, which allows HLSX to be removed from TB_FLAGS. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Weiwei Li <liweiwei@iscas.ac.cn> Tested-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> Message-Id: <20230325105429.1142530-11-richard.henderson@linaro.org> Message-Id: <20230412114333.118895-11-richard.henderson@linaro.org> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: Use cpu_ld*_code_mmu for HLVXRichard Henderson2023-05-051-2/+11
| | | | | | | | | | | | | Use the new functions to properly check execute permission for the read rather than read permission. Signed-off-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Weiwei Li <liweiwei@iscas.ac.cn> Tested-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> Message-Id: <20230325105429.1142530-10-richard.henderson@linaro.org> Message-Id: <20230412114333.118895-10-richard.henderson@linaro.org> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: Reduce overhead of MSTATUS_SUM changeFei Wu2023-05-051-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | Kernel needs to access user mode memory e.g. during syscalls, the window is usually opened up for a very limited time through MSTATUS.SUM, the overhead is too much if tlb_flush() gets called for every SUM change. This patch creates a separate MMU index for S+SUM, so that it's not necessary to flush tlb anymore when SUM changes. This is similar to how ARM handles Privileged Access Never (PAN). Result of 'pipe 10' from unixbench boosts from 223656 to 1705006. Many other syscalls benefit a lot from this too. Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Signed-off-by: Fei Wu <fei2.wu@intel.com> Message-Id: <20230324054154.414846-3-fei2.wu@intel.com> Reviewed-by: LIU Zhiwei <zhiwei_liu@linux.alibaba.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Weiwei Li <liweiwei@iscas.ac.cn> Tested-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> Message-Id: <20230325105429.1142530-8-richard.henderson@linaro.org> Message-Id: <20230412114333.118895-8-richard.henderson@linaro.org> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: fix H extension TVM trapYi Chen2023-05-051-6/+6
| | | | | | | | | | | | | | | | | | - Trap satp/hgatp accesses from HS-mode when MSTATUS.TVM is enabled. - Trap satp accesses from VS-mode when HSTATUS.VTVM is enabled. - Raise RISCV_EXCP_ILLEGAL_INST when U-mode executes SFENCE.VMA/SINVAL.VMA. - Raise RISCV_EXCP_VIRT_INSTRUCTION_FAULT when VU-mode executes SFENCE.VMA/SINVAL.VMA or VS-mode executes SFENCE.VMA/SINVAL.VMA with HSTATUS.VTVM enabled. - Raise RISCV_EXCP_VIRT_INSTRUCTION_FAULT when VU-mode executes HFENCE.GVMA/HFENCE.VVMA/HINVAL.GVMA/HINVAL.VVMA. Signed-off-by: Yi Chen <chenyi2000@zju.edu.cn> Reviewed-by: Weiwei Li <liweiwei@iscas.ac.cn> Reviewed-by: LIU Zhiwei <zhiwei_liu@linux.alibaba.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Message-Id: <20230406101559.39632-1-chenyi2000@zju.edu.cn> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: Use PRV_RESERVED instead of PRV_HWeiwei Li2023-05-051-1/+1
| | | | | | | | | | | | | | PRV_H has no real meaning, but just a reserved privilege mode currently. Signed-off-by: Weiwei Li <liweiwei@iscas.ac.cn> Signed-off-by: Junqiang Wang <wangjunqiang@iscas.ac.cn> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Message-Id: <20230407014743.18779-3-liweiwei@iscas.ac.cn> [ Changes by AF: - Convert one missing use of PRV_H ] Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: Fix the mstatus.MPP value after executing MRETWeiwei Li2023-05-051-1/+2
| | | | | | | | | | | The MPP will be set to the least-privileged supported mode (U if U-mode is implemented, else M). Signed-off-by: Weiwei Li <liweiwei@iscas.ac.cn> Signed-off-by: Junqiang Wang <wangjunqiang@iscas.ac.cn> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Message-Id: <20230407014743.18779-2-liweiwei@iscas.ac.cn> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: Fix format for indentationWeiwei Li2023-05-051-2/+2
| | | | | | | | | | | | | Fix identation problems, and try to use the same indentation strategy in the same file. Signed-off-by: Weiwei Li <liweiwei@iscas.ac.cn> Signed-off-by: Junqiang Wang <wangjunqiang@iscas.ac.cn> Reviewed-by: LIU Zhiwei <zhiwei_liu@linux.alibaba.com> Acked-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> Message-Id: <20230405085813.40643-3-liweiwei@iscas.ac.cn> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: Remove riscv_cpu_virt_enabled()Weiwei Li2023-05-051-9/+9
| | | | | | | | | | | | | | Directly use env->virt_enabled instead. Suggested-by: LIU Zhiwei <zhiwei_liu@linux.alibaba.com> Signed-off-by: Weiwei Li <liweiwei@iscas.ac.cn> Signed-off-by: Junqiang Wang <wangjunqiang@iscas.ac.cn> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: LIU Zhiwei <zhiwei_liu@linux.alibaba.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Reviewed-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> Message-Id: <20230405085813.40643-2-liweiwei@iscas.ac.cn> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: Remove redundant check on RVHWeiwei Li2023-05-051-2/+1
| | | | | | | | | | | | Check on riscv_cpu_virt_enabled contains the check on RVH. Signed-off-by: Weiwei Li <liweiwei@iscas.ac.cn> Signed-off-by: Junqiang Wang <wangjunqiang@iscas.ac.cn> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: LIU Zhiwei <zhiwei_liu@linux.alibaba.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Message-Id: <20230327080858.39703-3-liweiwei@iscas.ac.cn> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: implement Zicbom extensionChristoph Muellner2023-03-051-0/+67
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Zicbom is the Cache-Block Management extension defined in the already ratified RISC-V Base Cache Management Operation (CBO) ISA extension [1]. The extension contains three instructions: cbo.clean, cbo.flush and cbo.inval. All of them must be implemented in the same group as LQ and cbo.zero due to overlapping patterns. All these instructions can throw a Illegal Instruction/Virtual Instruction exception, similar to the existing cbo.zero. The same check_zicbo_envcfg() is used to handle these exceptions. Aside from that, these instructions also need to handle page faults and guest page faults. This is done in a new check_zicbom_access() helper. As with Zicboz, the cache block size for Zicbom is also configurable. Note that the spec determines that Zicbo[mp] and Zicboz can have different cache sizes (Section 2.7 of [1]), so we also include a 'cbom_blocksize' to go along with the existing 'cboz_blocksize'. They are set to the same size, so unless users want to play around with the settings both sizes will be the same. [1] https://github.com/riscv/riscv-CMOs/blob/master/specifications/cmobase-v1.0.1.pdf Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Weiwei Li <liweiwei@iscas.ac.cn> Co-developed-by: Philipp Tomsich <philipp.tomsich@vrull.eu> Signed-off-by: Christoph Muellner <cmuellner@linux.com> Signed-off-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> Message-ID: <20230224132536.552293-4-dbarboza@ventanamicro.com> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
* target/riscv: implement Zicboz extensionChristoph Muellner2023-03-051-0/+68
| | | | | | | | | | | | | | | | | | | | | | | | | | | The RISC-V base cache management operation (CBO) ISA extension has been ratified. It defines three extensions: Cache-Block Management, Cache-Block Prefetch and Cache-Block Zero. More information about the spec can be found at [1]. Let's start by implementing the Cache-Block Zero extension, Zicboz. It uses the cbo.zero instruction that, as with all CBO instructions that will be added later, needs to be implemented in an overlap group with the LQ instruction due to overlapping patterns. cbo.zero throws a Illegal Instruction/Virtual Instruction exception depending on CSR state. This is also the case for the remaining cbo instructions we're going to add next, so create a check_zicbo_envcfg() that will be used by all Zicbo[mz] instructions. [1] https://github.com/riscv/riscv-CMOs/blob/master/specifications/cmobase-v1.0.1.pdf Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Co-developed-by: Philipp Tomsich <philipp.tomsich@vrull.eu> Signed-off-by: Christoph Muellner <cmuellner@linux.com> Signed-off-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> Reviewed-by: Weiwei Li <liweiwei@iscas.ac.cn> Message-ID: <20230224132536.552293-3-dbarboza@ventanamicro.com> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
* target/riscv: remove RISCV_FEATURE_PMPDaniel Henrique Barboza2023-03-011-1/+1
| | | | | | | | | | | | | RISCV_FEATURE_PMP is being set via riscv_set_feature() by mirroring the cpu->cfg.pmp flag. Use the flag instead. Signed-off-by: Daniel Henrique Barboza <dbarboza@ventanamicro.com> Reviewed-by: Weiwei Li <liweiwei@iscas.ac.cn> Reviewed-by: Bin Meng <bmeng@tinylab.org> Reviewed-by: Andrew Jones <ajones@ventanamicro.com> Reviewed-by: LIU Zhiwei <zhiwei_liu@linux.alibaba.com> Message-ID: <20230222185205.355361-8-dbarboza@ventanamicro.com> Signed-off-by: Palmer Dabbelt <palmer@rivosinc.com>
* RISC-V: Adding XTheadSync ISA extensionChristoph Müllner2023-02-071-0/+6
| | | | | | | | | | | | | | This patch adds support for the XTheadSync ISA extension. The patch uses the T-Head specific decoder and translation. The implementation introduces a helper to execute synchronization tasks: helper_tlb_flush_all() performs a synchronized TLB flush on all CPUs. Co-developed-by: LIU Zhiwei <zhiwei_liu@linux.alibaba.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Signed-off-by: Christoph Müllner <christoph.muellner@vrull.eu> Message-Id: <20230131202013.2541053-3-christoph.muellner@vrull.eu> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: Clear mstatus.MPRV when leaving M-mode for priv spec 1.12+Bin Meng2023-01-061-0/+6
| | | | | | | | | | Since priv spec v1.12, MRET and SRET now clear mstatus.MPRV when leaving M-mode. Signed-off-by: Bin Meng <bmeng@tinylab.org> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Message-Id: <20221207090037.281452-2-bmeng@tinylab.org> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: Simplify helper_sret() a little bitBin Meng2023-01-061-14/+6
| | | | | | | | | | There are 2 paths in helper_sret() and the same mstatus update codes are replicated. Extract the common parts to simplify it a little bit. Signed-off-by: Bin Meng <bmeng@tinylab.org> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Message-Id: <20221207090037.281452-1-bmeng@tinylab.org> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: Fix mret exception cause when no pmp rule is configuredBin Meng2023-01-061-1/+1
| | | | | | | | | | | | | | | | | | | The priv spec v1.12 says: If no PMP entry matches an M-mode access, the access succeeds. If no PMP entry matches an S-mode or U-mode access, but at least one PMP entry is implemented, the access fails. Failed accesses generate an instruction, load, or store access-fault exception. At present the exception cause is set to 'illegal instruction' but should have been 'instruction access fault'. Fixes: d102f19a2085 ("target/riscv/pmp: Raise exception if no PMP entry is configured") Signed-off-by: Bin Meng <bmeng@tinylab.org> Reviewed-by: Wilfred Mallawa <wilfred.mallawa@wdc.com> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Message-Id: <20221205065303.204095-1-bmeng@tinylab.org> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* target/riscv: rvk: add CSR support for ZkrWeiwei Li2022-04-291-0/+9
| | | | | | | | | | | | | | | - add SEED CSR which must be accessed with a read-write instruction: A read-only instruction such as CSRRS/CSRRC with rs1=x0 or CSRRSI/CSRRCI with uimm=0 will raise an illegal instruction exception. - add USEED, SSEED fields for MSECCFG CSR Co-authored-by: Ruibo Lu <luruibo2000@163.com> Co-authored-by: Zewen Ye <lustrew@foxmail.com> Signed-off-by: Weiwei Li <liweiwei@iscas.ac.cn> Signed-off-by: Junqiang Wang <wangjunqiang@iscas.ac.cn> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Message-Id: <20220423023510.30794-13-liweiwei@iscas.ac.cn> Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
* compiler.h: replace QEMU_NORETURN with G_NORETURNMarc-André Lureau2022-04-211-2/+2
| | | | | | | | | | | | | G_NORETURN was introduced in glib 2.68, fallback to G_GNUC_NORETURN in glib-compat. Note that this attribute must be placed before the function declaration (bringing a bit of consistency in qemu codebase usage). Signed-off-by: Marc-André Lureau <marcandre.lureau@redhat.com> Reviewed-by: Daniel P. Berrangé <berrange@redhat.com> Reviewed-by: Warner Losh <imp@bsdimp.com> Message-Id: <20220420132624.2439741-20-marcandre.lureau@redhat.com>
* target/riscv: Adjust csr write mask with XLENLIU Zhiwei2022-01-211-1/+2
| | | | | | | | | | Write mask is representing the bits we care about. Signed-off-by: LIU Zhiwei <zhiwei_liu@c-sky.com> Reviewed-by: Richard Henderson <richard.henderson@linaro.org> Reviewed-by: Alistair Francis <alistair.francis@wdc.com> Message-id: 20220120122050.41546-11-zhiwei_liu@c-sky.com Signed-off-by: Alistair Francis <alistair.francis@wdc.com>