FakeJit was the native JIT for architectures without one, but the config
already forces those onto the IR interpreter, so CreateNativeJit() now
returns that instead. The vertex decoder fallback no longer needs
FakeEmitter.
JitBlockCache was the block cache of the pre-IR JITs. Its header keeps
the debug interface types the IR block caches use. Also removes the
JitInterface link hooks only it called, and the JitState fields tied to
it. This fixes a null dereference in the ImDebugger memory section,
which read GetBlockCache(), always null with the IR JITs.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The operands go in q0 and q1 and the result comes back in w1. The routine saves
nothing: the call site flushes whatever guest state is in the few registers it
clobbers (q4-q6, w0-w3), which come late in the allocation order, so usually
there's nothing to do. That's about 20 bytes a call, with nothing going through
memory, and as fast as having the whole thing inline, which made a vmmul.q about
5 KB of code. Inf and NaN go through a routine that calls vfpu_dot_reference().
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It was integer code, about four times the work. It also clobbers q0-q3 and
x16/x17 now, which both ARM64 JITs keep free, instead of saving them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GenerateArm64VfpuDot() generates vfpu_dot() as integer code, once per JIT, to
be called with BL. The operands and result go through mips->vfpuDotArgs and
vfpuDotResult, and the routine preserves every register but x30, so a call
site is two stores, the BL and a load, with nothing flushed. Inf and NaN
operands go to vfpu_dot_reference(), with the caller-saved registers saved.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The prefix fields were decoded by hand in the interpreter, the IR frontend,
JitState, MIPSState and the disassembler, with three copies of the constant
table. They now share small decoders and one table, in MIPS.h. The backends'
Vec4Shuffle lane decoding gets a helper of its own in IRInst.h.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A flush stores pending imms through SCRATCH1, so in a non-constant MemoryCheck, and in
ValidateAddress's bad-address path, the address computed into SCRATCH1 could be replaced by an imm
before IRRunMemCheck or ReportBadAddress saw it. It now waits in SCRATCH2, which flushes don't use.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Their IR-based replacements stay. The JIT core now creates the IR JIT on both, so "JIT using
IR" is the same as "JIT" on every architecture (ARM32 still uses its legacy JIT for both). The
settings no longer offer it, a saved JIT_IR loads as JIT, and the JIT harness and CPUEquivalence
fuzzer drop the duplicate run.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
So vcmovt never moves and vcmovf always moves (cpu/vfpu/undefined). The
interpreter did nothing for either, and the JITs took it as imm3 = 6,
one CC bit per lane.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On the PSP, zeros and denormals all compare equal, a tie returns the
second operand, and the result keeps its bits (cpu/vfpu/minmax_zero,
minmax_tie, fcr31). The interpreter got the ties and NaNs right but
compared denormals as numbers, and each JIT had its own fast path with
host min/max, which returned the first operand on a tie and flushed or
not depending on the host's mode.
All cores now compare by vfpu_minmax_key(), an integer key that is 0
for a zero exponent and orders the rest by sign and magnitude, and
select the operand by it, without branching on NaN.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
srl/srlv rotate when the low bit of their unused field (rs or sa) is
set and shift when it's clear, whatever the other bits hold
(cpu/cpu_alu/shift_encodings). The interpreter treated values above 1 as
invalid and the other cores shifted.
The Allegrex bit ops decode only some bits of sa, so all 32 values are
one of wsbh, wsbw, seb, seh and bitrev (cpu/cpu_alu/allegrex_encodings).
AllegrexBitOp() gives the canonical one for every core. Special3
function 0x18 does nothing on the PSP, where it was decoded as an alias
of 0x20.
The interpreter dispatch is regenerated, and its generator now lists
each instruction name once. CPUEquivalence no longer skips the srl/srlv
encodings.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A JIT only switched modes around syscalls, and only took the mode into account
in conversions, once it had compiled a ctc1 (hasSetRounding). A mode from an
interpreted ctc1, a savestate made on another core, a core switch or a thread
switch went unnoticed, and HLE then ran in the guest's mode. UpdateFCR31()
now sets the flag when fcr31 isn't the default, and clears the block cache at
the next safe point, since blocks compiled before don't switch either. A new
JIT gets an UpdateFCR31() too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Interpreted ops run in the guest's mode. A syscall left to the interpreter
(with JitDisable flags, say) is host code and now gets the host's mode, like a
compiled one, and an interpreted ctc1 to fcr31 now takes effect right away:
nothing set hasSetRounding for it, so the mode only reached the host at the
next time slice. Every other interpreted op skips the two calls it used to
make after it. The IR frontend's Comp_Generic, which the IR interpreter and the
IR native backends share, gets the same treatment.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comp_Generic restored the host's default mode around the interpreter call, so
an instruction the JIT doesn't compile rounded and flushed differently than in
the interpreter. Now the guest's mode stays in place, and is reapplied after the
call in case the instruction changed fcr31.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
None of the legacy JITs handled them: the destination was mapped without
being written, so it kept whatever the host register held.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mfhi/mflo into $zero mapped $zero as a destination and wrote HI or LO
into its host register, so a later instruction reading $zero (found by
CPUEquivalence as maddu/msub) saw that value. The other JITs and the IR
already skip them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ll into $zero mapped $zero for writing, which asserts. On ARM64 and
x86-64, sc wrote its 0/1 result into rt without marking it dirty, so it
was never stored (RISC-V and LoongArch already did this right).
See docs/cpu-equivalence.md.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- ext with pos + size past 32 and ins with msb below pos asserted in
UBFX/BFI. The results now match the other cores.
- A swizzle naming a lane past the op's size read an uninitialized
register in ApplyPrefixST. vdot, vcmp, vi2f, vi2x, vocp, vhdp, vcmov
and the matrix inits now leave odd prefixes to the interpreter, with
the same gates as the IR frontend.
- vdot, vhdp, vmmul and vqmul used fused multiply-add, which rounds
differently from every other core. They now multiply and add
separately, in the interpreter's order.
See docs/cpu-equivalence.md.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Config.h is edited about 100 times a year and each edit recompiled 198
files; now 172. SmallDataConvert.h pulls in arm_neon.h and was included by
headers that don't use it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With block entries out of memory, the only fake opcode left is the one the CPU cores fetch in
place of a hooked instruction. Call it MIPS_CALL_REPLACEMENT_OP, and its handlers
Int_CallReplacement and Dis_CallReplacement, and remove the sub-op defines and the unused
MIPS_IS_REPLACEMENT.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The JITs used to find a block by overwriting its first instruction with an "emuhack" holding
the offset into the code space, and replacement functions installed their own emuhack the same
way. Anything that read code (savestates, the debugger, memory dumps, cheats, the function
hasher, games reading their own code) had to translate them back.
Blocks now live in a block shadow: one u32 per guest instruction, indexed by the PC masked with
BLOCK_SHADOW_MASK (1GB on 64-bit, folding the kernel and uncached mirrors; 256MB on 32-bit, iOS
and Switch, where address space is short), with the low bits cleared so a misaligned PC can't
read across two entries. Only the ranges under memory views are committed, so a bad PC still
faults in the dispatcher, and the shadow is reserved before the memory map so it can't land in
a gap of the 4GB window. Entries hold full 32-bit offsets, lifting the 16MB limit. The
dispatchers on every backend, and the IR interpreter's loop, look blocks up there.
A hooked instruction without a compiled block gets the JIT's hook value in its entry: the offset
of the dispatcher's compile path (1 for the interpreters, the IR arena keeping that offset
empty), so dispatching there compiles a block, and the frontends substitute the hook. Compiling
replaces the entry with the block's, and destroying the block puts the hook value back. The hook
value is rewritten when the JIT changes or regenerates its dispatcher.
Hooks live in a map keyed by the masked address, and are dropped when the game overwrites the
hooked instruction, or when an icache invalidation finds the function's hash (immediates masked,
as when matching it) changed. Those checks run when queued invalidations are processed, since
cheats and the debugger queue them before writing. Installing and removing hooks invalidates
deferred, as it happens in HLE calls that can run inside a block. The cache instruction goes
through MIPSState, so it drops changed hooks too.
With clean memory, the save/clear/restore around savestates, memory reads and dumps goes away,
as do the source-range invalidations in the memcpy replacements, sceDmac and the cheats.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With the software renderer, which now stores depth as the GE does, a
16-bit read or write at VRAM's 0x04200000 or 0x04600000 mirror goes to
where the GE's 16-bit or 32-bit color layout puts that depth pixel
(ppsspp-re geprobe exp88: the mirrors are the layout's exact inverse).
Games access depth there with nothing else, so other widths stay plain.
The accessors check the address, then whether the GPU core is the
software renderer (CoreParameter); the hardware renderers keep depth plain
and handle the layout their own way. The renderer keeps the layouts up to
date with the EDRAM address translation. With it, the JITs send 16-bit
loads and stores to the interpreter, and the IR passes don't change a
load's width to or from 16 bits.
The pspautotests that read depth through the mirrors now do so 16 bits
at a time, re-recorded on hardware unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Only the rounding mode, flags, enables, cause, FCC and FS bits can be
written (0x0181FFFF, pspautotests cpu/fpu/fcr), as the interpreter, IR
and x86 already have it. Both ARM JITs stored the whole value.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
When a Load32, Store32, LoadFloat or StoreFloat is followed by the same op
on the next or previous word through the same base, and the base can be
mapped as a pointer, emit one LDP/STP for both. A struct-copying loop runs
about 24% faster; code without such pairs is unaffected.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The IR temps don't live past the block, but FlushAll stored them anyway at
every exit (the branch operands, lwl/lwr temps, VFPU temp lanes). At an exit,
discard the ones nothing later in the block reads.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A new FSinCos op writes both from one argument reduction. arm64 and x64 get
both back from a single call, packed in one double; RISC-V and LoongArch make
the two calls.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The cosine is then taken of what vrot wrote to that lane: the sine, or zero.
The IR looked at the sine lane instead of the lane holding the angle, and the
legacy JITs ignored the overlap. The assembler refuses such a vrot, so those
now leave it to the interpreter, and don't pair one with the vrot before it.
Covered by the new cpu/vfpu/vrot test.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Values in S8-S15 survive a call, but the full flush wrote them back and
the following instructions loaded them again. The VFPU math callouts,
vrot and vh2f now flush only the caller-saved registers, plus the few
callee-saved ones they stage values in, and map the destinations
afterwards. In a normalize loop with sixteen VFPU registers live, that
takes a vrsq.s from about 15 to 11.5 ns.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ll and sc always went to the interpreter; a few games use them
thousands of times. They now load and store directly with fast memory,
keeping llBit in MIPSState. vcmp's NaN and inf-or-NaN tests only look
at s, so they no longer need vt to be the same register: EN and NN
compare s with itself.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
vh2f always went to the interpreter in the IR. A new FHalfToFloat op
converts the lower or upper half of a word, and the native backends
call vfpu_h2f for it like FSin. The legacy arm64 JIT now makes the same
call instead of computing the conversion inline; vh2f is rare, and the
call is much less code.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
All went to the interpreter. vh2f works in integers to match vfpu_h2f,
since FCVTL neither flushes subnormal halves nor keeps inf/NaN mantissa
bits unshifted. The color conversions are bitfield extracts and inserts.
vbfy, vcrs and vdet follow the IR frontend, and vmscl scales element
by element. Results go through scratch registers, so a destination that
overlaps a source is fine.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Vec4ClampToZero and Vec2ClampToZero only ever fed Vec4Pack31To8 and
Vec2Pack31To16, for vi2uc and vi2us. The packs now clamp negative lanes
to zero themselves, which saves an op and a vector temp, and lets x64
clamp with PACKUSWB's saturation after an arithmetic shift.
While at it, RISC-V compiles Vec2Unpack16To31, Vec2Pack31To16 and
Vec4Pack32To8, and LoongArch Vec2Unpack16To31, Vec2Pack31To16 and the
non-LSX Vec4Pack32To8, all of which went to the IR interpreter.
LoongArch's Vec2Pack32To16 and Vec2Unpack16To32 now take their scalar
path with LSX too, instead of falling back.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
These went to the interpreter. vf2in/vf2iz/vf2iu/vf2id use FCVT, which
saturates like the PSP, and patch NaN to 0x7FFFFFFF like the IR
backend. vsgn keeps the sign bit on 1.0 and gives 0 below the smallest
normal. vsge and vslt select 1 or 0 on a compare whose condition is
false when unordered. EI and NI test |s| against infinity in integers.
All match the interpreter on special values, ties and denormals.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
These were always interpreted. They're now IR ops that the native
backends compile to calls to vfpu_exp2 and vfpu_log2, like FSin and
FAsin. vrexp2 is FNeg followed by FExp2, which is how vfpu_rexp2
computes it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
vsin, vcos, vnsin, vasin, vexp2, vlog2 and vrexp2 went to the
interpreter. They now take the same direct call as vrcp and friends.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
These went through the host's sqrt and division everywhere except the
interpreter's vrcp and vnrcp (vsqrt and vrsq there only behind
USE_VFPU_SQRT, now gone). They now always give the PSP's bits: the IR
gets FVSqrt (FSqrt stays the FPU's IEEE sqrt.s), and FRSqrt and FRecip,
which only the VFPU emits, become vfpu_rsqrt and vfpu_rcp; the IR
interpreter and the x64, arm64, RISC-V and LoongArch backends call them.
The old JITs call them directly, the ARM ones keeping the lanes in
callee-saved registers across the calls. cpu/vfpu/exact now passes on every core.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
In the interpreter, the IR interpreter, every backend's FSign and the x86
JIT's own vsgn. Recorded in cpu/vfpu/specials.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The other control registers were masked already; the CC path wasn't, so
a value with bits 6 or 7 set made bvt 6 and bvt 7 branch. Hardware never
sets those (cpu/vfpu/vbranch), and the IR masks them.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
mtvc to RCX0-7 keeps the low 20 bits of the value and forces the top
twelve to 0x3F8 (cpu/vfpu/vrnd), which is also the form vrnds and vrndi
leave them in. We masked with 0x3FFFFFFF. The generator itself matches
hardware for every seed and state in that test.
Interpreter, IR and the arm64 JIT. The x86 and ARM32 JITs don't mask
control register writes at all and are left as they were.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The arm64 JIT applied the plain prefix rules to every op. Several ops
don't follow them, and the interpreter and IR frontend know which:
- vabs and vneg are vmov with the abs/negate bit forced, so a negate in
the S prefix doesn't negate twice.
- vsat1 doesn't apply the D saturation.
- vrcp, vrsq, vsqrt, vnrcp (and the rest of that group) apply S and D to
the last lane only, from prefix position 0.
- vzero, vone and vidt are changed by a pending S prefix.
- vdiv, vscl, vfad/vavg and vrot each have their own rule.
- vuc2i and friends with an S prefix.
Hand these to the interpreter, exactly where the IR frontend does. Also
mask what mtvc writes to a control register (the low 20 bits of a prefix,
12 of the D prefix), as the IR already did.
Goes from 19 ops differing from hardware in cpu/vfpu/prefix_consume to
the 3 that differ on every core, and clears cpu/vfpu/prefix_ctrl.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The rule is LO = numerator <= 0xFFFF ? 0xFFFF : -1, unsigned. The IR
backend used CC_LE, so any numerator with the top bit set counted as
small and got 0xFFFF. The other backends compare unsigned. Also caught by
cpu/cpu_alu/cpu_div.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every path special-cased the one signed division overflow and set the
remainder to -1. cpu/cpu_alu/cpu_div, recorded on a PSP, says it's 0.
The classic arm64 JIT was the only one that got it right, by not
special-casing it at all.
arm64, RISC-V and LoongArch all produce INT_MIN with remainder 0 natively,
so their fixup blocks go away. x86 has to keep the check (IDIV traps) but
sets HI to 0 now, and so do both interpreters.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Comp_Vmfvc read vfpuCtrl[] straight from the context in all four
backends, while the mfvc path in Comp_Mftv flushes first, with the
comment "In case we have a saved prefix" - so "vpfxs X" followed by
"vmfvc sN, $128" returned the stale value in memory. The IR frontend
flushes only for the three prefix registers, which is the tighter form,
so vmfvc does the same there.
On ARM and ARM64 the fix alone wouldn't have been observable: those two
map the destination with no flags, leaving isDirty false, so the flush
dropped the loaded value without storing it. x86 already passes
MAP_DIRTY | MAP_NOINIT. That part is a fix of its own, but the two are
inseparable in this function - a vmfvc that reads the right value and
then throws it away is no better.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The ARM64 IR JIT crashed on any load/store to a constant address with the
top bit set and an offset too large for an immediate - the kernel RAM mirror
at 0x88000000, for instance.
PrepareSrc1Address is careful about this: a constant address like 0x89100010
arrives sign-extended as a negative int64_t, and the (imm & 0xC0000000) ==
0x80000000 check turns it back into the positive value it should be. But when
the offset doesn't fit an immediate we fall back to loading it into a register
and using the register-offset addressing mode, which only takes the W half of
that register plus an extend - and we asked for SXTW, undoing the fix and
pointing the access ~2GB below the memory view.
Sign extension is still right when the offset is genuinely negative, which
happens when the base is a pointerified register and the displacement is a
negative one from the MIPS instruction. So extend based on the sign of imm.
Found by the Jit unit test, which stores to 0x89100000 and segfaults in the
JIT_IR phase - only reproducible on an actual ARM64 CPU, which is why it never
showed up on CI. The RISC-V and LoongArch backends get this case right already.
step-over, step-out and run-until plant a one-shot breakpoint at the address
they want execution to return to. Keeping it in breakPoints_ alongside the
user's own meant the two kept colliding:
- Adding a log-only user breakpoint at the same address hijacked the temporary
one. AddBreakPoint() didn't match across temp-ness so both existed, and then
ChangeBreakPoint() looked up "the first enabled breakpoint at this address" -
a log-only breakpoint isn't enabled, so the temporary one won and had its
action overwritten to log-only. It lost PAUSE and the step never came back.
- RemoveBreakPoint() erased up to two entries per address to catch an
overlapping temporary one, so deleting either deleted both - including the
interpreter's cleanup path in CheckExecBreakpoints() taking the user's
breakpoint with it.
- ExecBreakPoint() handled one breakpoint per address, so with both at the same
address only one of them did anything: the step completed but the user's log
line never printed.
- Nothing dropped it when something *else* stopped us first, so an interrupted
step left a breakpoint armed at an address nobody was waiting for anymore,
which later fired as a phantom stop.
It's a single TempBreakPoint member now, invisible to the breakpoint lists and
untouched by user edits. One is enough: step over/out and cross-thread step into
all require the CPU to already be stepping and resume it immediately, so only
one can be in flight, and run-until now replaces rather than stacking (two
pending run-untils had no coherent meaning, and the loser stayed armed).
Behavior follows what other debuggers do. Both breakpoints at an address are
evaluated independently and their actions combine, so a log-only breakpoint
logs without stopping and still lets the step finish. Core_Break() drops the
temporary breakpoint on any stop, whatever the reason - the same way gdb deletes
its step-resume breakpoint and lldb discards the thread plan.
Two things to be careful of, both covered by the new TempBreakpoints test:
HasBreakPoints() has to account for it, or the interpreter's checked run loop
and the JIT skip breakpoint checking entirely and a step with no user
breakpoints set never returns; and IsAddressBreakPoint() (user-facing, for the
lists and disassembly markers) is now separate from NeedsBreakCheckAt() (what
the JIT frontends and interpreter ask), since only the latter should see it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
