HandleFault only handled faults in JIT code, so a game's bad load or store
under the IR interpreter crashed the app. Now IRInterpret marks itself
(a thread-local, cleared around HLE calls, replacements and the MIPS
interpreter fallback) and the handler steps over the faulting instruction,
reports the exception and stops the core, or just ignores the access when
IgnoreBadMemAccess is on. If the instruction can't be decoded it still
crashes, as before.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The native dispatcher generated for the ME leaves when its downcount
runs out instead of calling CoreTiming::Advance, and IRNativeJit gets
RunDowncount. Pointerify is decided from the CPU's own memory base. ARM32,
RISC-V and LoongArch64 still give the ME the IR interpreter.
LoadKernel/StoreKernel are now compiled on ARM64 and x64 rather than
going through the IR interpreter one op at a time: memory directly, or a
call to fixed code (which saves the caller-saved registers guest values
live in) for 0xBC000000-0xBFFFFFFF. ARM64's block size estimate counts
them as such. The 16-bit ones stay generic while the software renderer's
depth mirrors are active, as Load16/Store16 do. That matters for the main
CPU's kernel code too, and for all code while the ME is emulated, which
runs in kernel mode everywhere. mcidclan's me-soft-interrupt went from
476 billion to 6.4 billion host instructions for 3 emulated seconds,
ending with the same ME counter as the IR interpreter.
The ME unit tests run on the native JIT too.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The ME sees the main CPU's RAM, but has its own eDRAM at 0 (with
0x80000000 and 0xA0000000 mirrors) where the main CPU has its scratchpad,
its own memory where the main CPU has VRAM, and both CPUs share the boot
page at 0xBFC00000.
- The ME's own memory is views in the main arena flagged MV_ME_ONLY, which
only Memory::CreateMESpace() maps, into a second reservation along with
the shared views at their same arena positions. It gets its own block
shadow.
- The eDRAM (MV_EDRAM) is the ME's half of the eDRAM chip, the GE's VRAM
being the other: 2MB on the PSP-1000 (whose resident ME image says so),
4MB on the others (from mcidclan's libmecore, not hardware-checked).
- At VRAM's addresses, through 0x44000000 and 0xA4000000 alike, the ME
doesn't see VRAM but memory of its own (pspautotest me/vram, on a
PSP-1000): where homebrew keeps the VME's buffers. Its size isn't known;
2MB with VRAM's mirrors here, and no depth layout on 16-bit accesses.
- 0xA0000000 reaches physical memory uncached, so RAM is mapped at
0xA8000000 too.
- The boot page is a view too, which the main CPU reaches through a new
MMIO device (in savestates from MMIO section version 2). The ME runs
code from it, uncached, so the main CPU's writes invalidate code there.
On masked platforms it isn't mapped, and the device uses a plain
buffer.
- AddressSpace has IsValidAddress etc, with the ME's rules when edramSize
is set, and the interpreters and the IR interpreter check and fetch
instructions through their state's space. The IR compiler's checks still
use the main CPU's, until the ME gets a JIT.
- On Darwin, MemArena can hold several view reservations.
Masked platforms (iOS, 32-bit) don't get an ME space yet.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Memory::AddressSpace is where a CPU's addresses point (base) and its block
shadow, with the unchecked accessors and block shadow helpers as methods. The
main CPU's is Memory::g_mainSpace, kept in step with Memory::base and
blockShadow, and MIPSState::space points at it.
The CPU cores now go through their state's space instead of the globals: the
interpreter's and IR interpreter's loads and stores (which also saves the IR
interpreter reloading the global base), the IR interpreter's block lookup,
the block table's shadow entries, and the native JITs' memory base register,
block shadow lookups and pointerified registers. Address validity and
everything outside the cores still use the globals.
No change for the main CPU; this is so another CPU (the Media Engine) can see
memory differently. Benchmarked against master (God of War: Chains of
Olympus, Tekken 6): JIT instructions retired within noise, IR interpreter
2% fewer.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
HLEFunc and ReplaceFunc now take the MIPSState they run on. The dispatch
paths (both interpreters and every JIT backend) pass in the state they
belong to, and CallSyscall*, hleFinishSyscall and the HLE mips call
machinery use it instead of currentMIPS.
The wrappers in FunctionWrappers.h take the function as `auto` and call it
through HLECall, which also accepts an implementation with an extra leading
MIPSState * parameter and passes the state to it. Any other signature still
fails to compile, as before. PARAM/RETURN now refer to a local `mips`.
The HLE functions (and replacement functions) that used currentMIPS now take
the state that way. Callers outside a syscall's own frame - CoreTiming
events, the adhoc matching thread, dialogs, PPGe, the debugger UIs - pass
currentMIPS or &mipsr4k explicitly. The kernel's thread internals still use
currentMIPS.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Kernel code's loads and stores may reach hardware registers, which only the
interpreter handled, so the IR frontend sent every one of them (in kernel
blocks, only run by the VSH boot) to it.
They now become LoadKernel*/StoreKernel* IR ops (one per size, like the
plain loads and stores), which check at run time whether the address is
memory or a hardware register (MMIO). After constant
propagation, ResolveKernelMemory turns the ones at a constant address that's
plain memory into ordinary loads and stores, so only the hardware registers,
and addresses only known at run time, stay checked. The native JITs run the
two ops through their IR fallback. They're barriers, since a register access
has side effects, but can't change the GPRs, so constants stay known across.
The kernel mode check for MMIO moves to the interpreter, the only caller that
can run user code there. TestMMIO covers the new paths and which accesses
stay checked. Under --vsh, the JIT and the interpreter now make the same
unhandled register accesses, in the same numbers.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Fixes#12082
The bit-exact VFPU dot product is the default everywhere and fast enough,
so the plain multiply-add paths in the interpreter and the IR frontend go,
along with the AccurateVdot setting, --accurate-vdot, the ForceAccurateVdot
compat flag and the Vec4Dot IR op, which nothing emits any more.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The skip-first marker suppressed everything at the address you resume or
step from. That's needed for the pause, or you could never get off a
breakpoint, but it also dropped the log and hit count. So a log breakpoint
you'd stepped onto, or were parked on for some other reason, never logged
when execution moved on.
Now the skip only drops the pause, unless the breakpoint at that address is
the one that stopped us, which has already logged and counted this
execution. The check moves into the Exec* functions instead of being
repeated in the interpreter, IR interpreter and three JITs, where the
register breakpoint path only applied it to the pause and so logged twice
when stepping off a log+pause one. The marker gets an explicit valid flag,
since address 0 meant "none", and Core_Break() clears it.
Based on the earlier cleanup-skip-first branch, reduced to one marker plus
a flag. Checked over the WebSocket debugger on the interpreter, IR, x86 JIT
and JIT-IR: log breakpoints on a parked and a stepped-onto address each log
once, and stepping off a log+pause one doesn't report it again.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each block was a call into IRInterpret from RunBlocks: register save and restore, the first
dispatch through the shared switch, then the shadow lookup in RunBlocks. Now the exits look up the
next block themselves and keep dispatching while downcount lasts, the same check RunBlocks makes.
Anything else (a hook, an uncompiled block, an invalid address, downcount gone) still returns.
Without the vertex JIT, as on iOS: 4.5-10% fewer instructions and 1-4% fewer cycles in the games
measured.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
New IR ops do them four lanes at a time where the backend has them, the
ARM64 JIT and the IR interpreter so far. vmin and vmax compare by
vfpu_minmax_key() in NEON, which took about twelve instructions a lane
through the integer registers before; vslt and vsge are a compare and an
AND with 1.0.
vi2f becomes one SCVTF with fractional bits.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A new IR op, Vec4VfpuMultiDot, does four vfpu_dot()s against one vector,
the other operands being the rows or Vec4s of a 4x4 block of registers.
The frontend emits it for vmmul and vtfm/vhtfm, and for runs of vdot that
fit it, like a matrix times a vector written out.
On ARM64 it calls a routine that has the dots in lanes rather than the
terms, so nothing is horizontal and nothing goes through the integer
registers. The block comes in with LD1, or LD4 for the transposed form,
when none of it is in host registers. The IR interpreter runs it through
vfpu_dot(); the other backends don't get it yet.
The CPU fuzzer gets runs of vdot shaped for it, and finds no difference
between the cores on them.
There's a routine per number of terms, so vdot.t runs don't pay for a
fourth. The vector's significands are worked out once and multiplied by
element, and the rounding at the end is UCVTF's in round-to-nearest,
which is what games run in; the integer version stays for the other
modes. The routine saves nothing, and the call site keeps only those
registers it clobbers that hold guest state.
JitDotBench, ns per dot: vtfm4.q 3.40, vmmul.q 3.12, a run of vdot.t
2.83, against 4.6 for single dots.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The frontend lowers vdot, vhdp, vfad, vavg, vdet, vcrsp/vqmul, vtfm/vhtfm and
vmmul to it then, with the operands built from the same lanes, constants and
negations the interpreter feeds vfpu_dot(). BuildDotOperands() puts them in
Vec4 regs, using the free temps where they aren't one already, and the ops
fall back to the interpreter when there aren't enough. The IR interpreter runs
it through vfpu_dot(), and the native backends through their generic fallback
for now. vmmul takes the same path for the MoreAccurateVMMUL compat flag.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The prefix fields were decoded by hand in the interpreter, the IR frontend,
JitState, MIPSState and the disassembler, with three copies of the constant
table. They now share small decoders and one table, in MIPS.h. The backends'
Vec4Shuffle lane decoding gets a helper of its own in IRInst.h.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On the PSP, zeros and denormals all compare equal, a tie returns the
second operand, and the result keeps its bits (cpu/vfpu/minmax_zero,
minmax_tie, fcr31). The interpreter got the ties and NaNs right but
compared denormals as numbers, and each JIT had its own fast path with
host min/max, which returned the first operand on a tie and flushed or
not depending on the host's mode.
All cores now compare by vfpu_minmax_key(), an integer key that is 0
for a zero exponent and orders the rest by sign and magnitude, and
select the operand by it, without branching on NaN.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A native backend can run this op through its generic fallback after its own
asm applied the guest's mode, which RestoreHostRoundingMode() doesn't know
about, so it has to clear the bits even when it didn't save them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ARM32's libm ceilf/floorf don't honor flush-to-zero, so the interpreters flush
the input themselves there.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ARMv7 NEON always flushes denormals and rounds to nearest, whatever FPSCR
says, so on ARM32 the scalar paths are used, which follow the guest's mode
like everything else.
The Vec4 ops only come from VFPU code, and the VFPU itself always rounds to
nearest and flushes denormals (cpu/vfpu/fcr31, see docs/cpu-equivalence.md),
so the NEON paths were arguably closer to the PSP there. But they disagreed
with every other host and core, and agreement between those comes first.
Running VFPU ops in the VFPU's own mode on every core would get both, and is
still to do.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It only applied fcr31's rounding mode and flush-to-zero after a ctc1 in
the same block, so add.s, mul.s, cvt and friends rounded to nearest in
every other block. It's now applied once per time slice in RunLoopUntil,
where the native dispatchers do it, and the host's mode is put back
around CoreTiming, compiling, syscalls and replacement functions.
See docs/cpu-equivalence.md.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every op ended with a break back to one shared indirect jump, which the
CPU has to predict for every op in the program. With labels as values,
each op jumps through a table from its own site instead, which predicts
much better: an integer-heavy benchmark runs about 13% faster on an M1.
Other compilers keep the switch, and ops missing from the table fall back
to it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Blocks ending in a branch dispatch a conditional exit and then the
fallthrough ExitToConst. One op now returns either target, reading the
second from the ExitToConst, which stays behind unexecuted.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A new FSinCos op writes both from one argument reduction. arm64 and x64 get
both back from a single call, packed in one double; RISC-V and LoongArch make
the two calls.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
vh2f always went to the interpreter in the IR. A new FHalfToFloat op
converts the lower or upper half of a word, and the native backends
call vfpu_h2f for it like FSin. The legacy arm64 JIT now makes the same
call instead of computing the conversion inline; vh2f is rare, and the
call is much less code.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Vec4ClampToZero and Vec2ClampToZero only ever fed Vec4Pack31To8 and
Vec2Pack31To16, for vi2uc and vi2us. The packs now clamp negative lanes
to zero themselves, which saves an op and a vector temp, and lets x64
clamp with PACKUSWB's saturation after an arithmetic shift.
While at it, RISC-V compiles Vec2Unpack16To31, Vec2Pack31To16 and
Vec4Pack32To8, and LoongArch Vec2Unpack16To31, Vec2Pack31To16 and the
non-LSX Vec4Pack32To8, all of which went to the IR interpreter.
LoongArch's Vec2Pack32To16 and Vec2Unpack16To32 now take their scalar
path with LSX too, instead of falling back.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
These were always interpreted. They're now IR ops that the native
backends compile to calls to vfpu_exp2 and vfpu_log2, like FSin and
FAsin. vrexp2 is FNeg followed by FExp2, which is how vfpu_rexp2
computes it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
These went through the host's sqrt and division everywhere except the
interpreter's vrcp and vnrcp (vsqrt and vrsq there only behind
USE_VFPU_SQRT, now gone). They now always give the PSP's bits: the IR
gets FVSqrt (FSqrt stays the FPU's IEEE sqrt.s), and FRSqrt and FRecip,
which only the VFPU emits, become vfpu_rsqrt and vfpu_rcp; the IR
interpreter and the x64, arm64, RISC-V and LoongArch backends call them.
The old JITs call them directly, the ARM ones keeping the lanes in
callee-saved registers across the calls. cpu/vfpu/exact now passes on every core.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
x86 (SQRTSS and libm alike) returns 0xffc00000 for it, the PSP 0x7fc00000.
-0 stays -0 and a NaN input comes through as it is, so only a negative
input needs the sign cleared: a compare and an xor in the x64 JITs, a
branch in the interpreters. cpu/fpu/roundmode.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The C cast is undefined past the int32 range, and x86 makes it INT_MIN, so
round/trunc/ceil/floor/cvt.w.s of anything from 2^31 up gave 0x80000000 on
x86 hosts while the PSP saturates to 0x7fffffff (cpu/fpu/roundmode). Route
all of them through SaturatedFloatToInt, which also covers NaN and inf, and
drop the special cases that did.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
In the interpreter, the IR interpreter, every backend's FSign and the x86
JIT's own vsgn. Recorded in cpu/vfpu/specials.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Every path special-cased the one signed division overflow and set the
remainder to -1. cpu/cpu_alu/cpu_div, recorded on a PSP, says it's 0.
The classic arm64 JIT was the only one that got it right, by not
special-casing it at all.
arm64, RISC-V and LoongArch all produce INT_MIN with remainder 0 natively,
so their fixup blocks go away. x86 has to keep the check (IDIV traps) but
sets HI to 0 now, and so do both interpreters.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
FpCondFromReg's meta is "_G", so the register is in src1 - which is where
every native backend, PropagateConstants and ReorderLoadStore read it.
But IRInterpret read it from dest, and Comp_VecDo3 wrote it to dest to
match. The other emitter passes (0, MIPS_REG_ZERO), so both fields are
zero there and the disagreement stayed hidden.
The result was that vsge/vslt, which save fpcond to IRTEMP_0 and restore
it afterwards, restored r0 (always zero) on the native IR JITs instead of
the saved value, losing any c.cond.s result live across them. Fixed both
the emitter and the interpreter to use src1.
Vec4Pack31To8's SSE2 path computed (v >> 24) << 1, which is (v >> 23)
with bit 23 forced to zero - the scalar and NEON paths both do
(v >> 23) & 0xFF. Shift left first instead. The pspautotests inputs all
happen to have bit 23 clear after the clamp, so this wasn't caught.
Also:
- Evaluate() didn't mask constant-folded shift amounts to 5 bits, though
the neighbouring one-immediate path does.
- ApplyMemoryValidation only invalidated its address-check cache for ops
with a 'G' destination, while Interpret and CallReplacement can write
any GPR - both are barriers, so drop the whole cache when we see one.
- ReduceVec4Flush indexed isVec4 with (src2 & 3) instead of (src2 & ~3)
for Vec4Scale; a missed optimization rather than a miscompile.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Every JIT backend puts the host FPU into the mode fcr31 asks for (bits 0-1 and
24) before running emulated code, and takes it back out before calling any host
code. The plain interpreter did none of that, so all its float math rounded to
nearest with denormals intact no matter what the game had set - cpu/fpu/fpu
fails under -i and passes under the JIT on exactly this.
Move the helpers the IR interpreter already had for this out of IRInterpreter
and into MIPS.cpp as ApplyHostRoundingMode/RestoreHostRoundingMode, and use them
around the interpreter's run loop and single step, restoring around syscalls and
replacement functions, which are host code. ctc1 re-applies immediately, since
the interpreter has no block boundary to defer it to.
round.w.s changes with it: it was floorf(x + 0.5f), which is half-away-from-zero
rather than the half-to-even every JIT produces, and the add would now pick up
the guest's rounding mode on top of that. round_ieee_754 is both correct and
mode-independent, and is what cvt.w.s already used for the same rounding.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SfY7iFJEjmRXf1XGrTs4MF
