Yabause-PSP's me.prx ORs 0xA0000000 onto its own addresses to share a
buffer with the ME uncached, which is 0xA8xxxxxx for a module in kernel
RAM. The main CPU's space has no views there, so it crashed.
All of kernel segment 1 (0xA0000000-0xBFFFFFFF) now counts as MMIO, and a
device below the hardware registers maps RAM and VRAM onto their uncached
mirrors at 0x40000000. The main CPU's native kernel loads and stores send
the whole segment to that path; the ME keeps 0xBC000000 as the boundary,
since its space maps the rest.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The native dispatcher generated for the ME leaves when its downcount
runs out instead of calling CoreTiming::Advance, and IRNativeJit gets
RunDowncount. Pointerify is decided from the CPU's own memory base. ARM32,
RISC-V and LoongArch64 still give the ME the IR interpreter.
LoadKernel/StoreKernel are now compiled on ARM64 and x64 rather than
going through the IR interpreter one op at a time: memory directly, or a
call to fixed code (which saves the caller-saved registers guest values
live in) for 0xBC000000-0xBFFFFFFF. ARM64's block size estimate counts
them as such. The 16-bit ones stay generic while the software renderer's
depth mirrors are active, as Load16/Store16 do. That matters for the main
CPU's kernel code too, and for all code while the ME is emulated, which
runs in kernel mode everywhere. mcidclan's me-soft-interrupt went from
476 billion to 6.4 billion host instructions for 3 emulated seconds,
ending with the same ME counter as the IR interpreter.
The ME unit tests run on the native JIT too.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Memory::AddressSpace is where a CPU's addresses point (base) and its block
shadow, with the unchecked accessors and block shadow helpers as methods. The
main CPU's is Memory::g_mainSpace, kept in step with Memory::base and
blockShadow, and MIPSState::space points at it.
The CPU cores now go through their state's space instead of the globals: the
interpreter's and IR interpreter's loads and stores (which also saves the IR
interpreter reloading the global base), the IR interpreter's block lookup,
the block table's shadow entries, and the native JITs' memory base register,
block shadow lookups and pointerified registers. Address validity and
everything outside the cores still use the globals.
No change for the main CPU; this is so another CPU (the Media Engine) can see
memory differently. Benchmarked against master (God of War: Chains of
Olympus, Tekken 6): JIT instructions retired within noise, IR interpreter
2% fewer.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
HLEFunc and ReplaceFunc now take the MIPSState they run on. The dispatch
paths (both interpreters and every JIT backend) pass in the state they
belong to, and CallSyscall*, hleFinishSyscall and the HLE mips call
machinery use it instead of currentMIPS.
The wrappers in FunctionWrappers.h take the function as `auto` and call it
through HLECall, which also accepts an implementation with an extra leading
MIPSState * parameter and passes the state to it. Any other signature still
fails to compile, as before. PARAM/RETURN now refer to a local `mips`.
The HLE functions (and replacement functions) that used currentMIPS now take
the state that way. Callers outside a syscall's own frame - CoreTiming
events, the adhoc matching thread, dialogs, PPGe, the debugger UIs - pass
currentMIPS or &mipsr4k explicitly. The kernel's thread internals still use
currentMIPS.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
IRJit is now the base the IR JITs share: the frontend, the table of guest
blocks, and compiling and invalidating them. IRInterpreterJit and
IRNativeJit are the two kinds. Only IRInterpreterJit keeps IR, in its arena,
along with its dispatch loop and per-block profile.
The native JITs no longer keep the IR they compiled from. The debugger and
the --jit-profile report compile a native block's IR again when they want it,
on a copy of the frontend's state, which gives what ran unless the code or
that state has changed since.
IRBlockCache and IRBlock become JitBlockTable and JitBlock: where each block
came from, the page lookup for invalidation, and the block shadow entry (an
arena or native code offset, depending on the JIT). One IRJitDebugInterface
serves both JITs, taking the MIPS code from the table and the IR and host code
from the JIT, in place of the native JIT's own debug interface.
The native backends don't see the table at all. They compile a span of IR for
a block number and start PC, and keep what they need about their own blocks
(start PC, code offset, which block starts where, for linking) next to the
exits they already tracked. The register cache takes the instructions
directly.
The tracer's block index now goes into the IR before the block is stored, so
native blocks get it too. The unused FindByCookie goes. TestJit checks that
the native JIT keeps no IR, and can still show it. The JIT stats overlay no
longer shows IR for the native JITs. MIPSState can't be copied, now that it
owns its JIT.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The JIT backends' calls out of generated code (the MIPS and IR interpreter
fallbacks, and the bad address report) now take the state the backend was
built for instead of reading currentMIPS. Likewise the interpreter's tick
check, the IR dispatcher's Advance, and a couple of debugger helpers.
MIPSState::InitRegisters() no longer sets currentMIPS; the boot paths set it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Fixes#12082
The bit-exact VFPU dot product is the default everywhere and fast enough,
so the plain multiply-add paths in the interpreter and the IR frontend go,
along with the AccurateVdot setting, --accurate-vdot, the ForceAccurateVdot
compat flag and the Vec4Dot IR op, which nothing emits any more.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Big games fill 16 MB in normal play. It's now one constant shared by
every backend. ARM32's 32 MB branch range is the limit on going higher
there.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
DescribeCodePtr() called everything in the fixed code other than a few
routines "fixedCode". It now uses the routine list the profiler has
(moved to the backend as GetAllFixedCodeLabels()), with an offset when
not at the start, so a block ends in "b dispatcherCheckCoreState".
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The native IR JITs get the emulator thread's host PC sampled every 50 us (a
signal on macOS and Linux, SuspendThread on Windows), mapped to blocks, the
routines they call and host functions. The report has each hot block's MIPS,
IR and host code with the samples per host instruction. The IR interpreter's
block timing, previously behind IR_PROFILING, now runs on the same switch.
Both cost nothing when off.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
FakeJit was the native JIT for architectures without one, but the config
already forces those onto the IR interpreter, so CreateNativeJit() now
returns that instead. The vertex decoder fallback no longer needs
FakeEmitter.
JitBlockCache was the block cache of the pre-IR JITs. Its header keeps
the debug interface types the IR block caches use. Also removes the
JitInterface link hooks only it called, and the JitState fields tied to
it. This fixes a null dereference in the ImDebugger memory section,
which read GetBlockCache(), always null with the IR JITs.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
CTXREG points at mips->f[0] in the x86-64 IR JIT, 128 bytes into MIPSState,
but the dot routine's inf and NaN path stored its operands at
offsetof(MIPSState, vfpuDotArgs) from it. It read them back from the same
place, so the results were right, but it wrote over invalidateAll_ and 16
bytes past the end of mipsr4k. On Alpine that crashed the unit tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The operands go in xmm0 and xmm1 and the result comes back in edi. The routine
saves nothing: the call site flushes whatever guest state is in the registers it
clobbers, the xmms allocated last and rax, rcx, rdx, rsi and rdi, the last two
never allocated outside Windows. 32-bit x86 keeps going through the IR
interpreter.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GenerateX64VfpuDot() is the x86-64 counterpart of GenerateArm64VfpuDot(): the
same integer version of vfpu_dot(), preserving every register but the flags,
with inf and NaN operands going to vfpu_dot_reference(). A dest lane inside a
source Vec4 still takes the generic fallback, like Vec4Dot does.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GenerateArm64VfpuDot() generates vfpu_dot() as integer code, once per JIT, to
be called with BL. The operands and result go through mips->vfpuDotArgs and
vfpuDotResult, and the routine preserves every register but x30, so a call
site is two stores, the BL and a load, with nothing flushed. Inf and NaN
operands go to vfpu_dot_reference(), with the caller-saved registers saved.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FSat0_1's MAXSS had the known 0 second, so a NaN came out as 0, unlike the
interpreter and the other backends, and the PSP (cpu/vfpu/prefix_sat, nan_gen).
-0.0 is now cleared by a compare and mask rather than by adding 0.0, which would
quiet a signaling NaN.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Their IR-based replacements stay. The JIT core now creates the IR JIT on both, so "JIT using
IR" is the same as "JIT" on every architecture (ARM32 still uses its legacy JIT for both). The
settings no longer offer it, a saved JIT_IR loads as JIT, and the JIT harness and CPUEquivalence
fuzzer drop the duplicate run.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
So vcmovt never moves and vcmovf always moves (cpu/vfpu/undefined). The
interpreter did nothing for either, and the JITs took it as imm3 = 6,
one CC bit per lane.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On the PSP, zeros and denormals all compare equal, a tie returns the
second operand, and the result keeps its bits (cpu/vfpu/minmax_zero,
minmax_tie, fcr31). The interpreter got the ties and NaNs right but
compared denormals as numbers, and each JIT had its own fast path with
host min/max, which returned the first operand on a tie and flushed or
not depending on the host's mode.
All cores now compare by vfpu_minmax_key(), an integer key that is 0
for a zero exponent and orders the rest by sign and magnitude, and
select the operand by it, without branching on NaN.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
srl/srlv rotate when the low bit of their unused field (rs or sa) is
set and shift when it's clear, whatever the other bits hold
(cpu/cpu_alu/shift_encodings). The interpreter treated values above 1 as
invalid and the other cores shifted.
The Allegrex bit ops decode only some bits of sa, so all 32 values are
one of wsbh, wsbw, seb, seh and bitrev (cpu/cpu_alu/allegrex_encodings).
AllegrexBitOp() gives the canonical one for every core. Special3
function 0x18 does nothing on the PSP, where it was decoded as an alias
of 0x20.
The interpreter dispatch is regenerated, and its generator now lists
each instruction name once. CPUEquivalence no longer skips the srl/srlv
encodings.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A JIT only switched modes around syscalls, and only took the mode into account
in conversions, once it had compiled a ctc1 (hasSetRounding). A mode from an
interpreted ctc1, a savestate made on another core, a core switch or a thread
switch went unnoticed, and HLE then ran in the guest's mode. UpdateFCR31()
now sets the flag when fcr31 isn't the default, and clears the block cache at
the next safe point, since blocks compiled before don't switch either. A new
JIT gets an UpdateFCR31() too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Interpreted ops run in the guest's mode. A syscall left to the interpreter
(with JitDisable flags, say) is host code and now gets the host's mode, like a
compiled one, and an interpreted ctc1 to fcr31 now takes effect right away:
nothing set hasSetRounding for it, so the mode only reached the host at the
next time slice. Every other interpreted op skips the two calls it used to
make after it. The IR frontend's Comp_Generic, which the IR interpreter and the
IR native backends share, gets the same treatment.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comp_Generic restored the host's default mode around the interpreter call, so
an instruction the JIT doesn't compile rounded and flushed differently than in
the interpreter. Now the guest's mode stays in place, and is reapplied after the
call in case the instruction changed fcr31.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With FS set in fcr31, x86 flushed only denormal results (FTZ) while
ARM64's FZ also flushes denormal inputs, so the hosts disagreed whenever
a denormal reached an FPU or VFPU op. With DAZ too, x86 flushes the way
ARM64 does. That matches the PSP's VFPU, which always flushes inputs and
outputs (cpu/vfpu/fcr31); the PSP's FPU flushes only results
(cpu/fpu/denormal), which neither host does now.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
None of the legacy JITs handled them: the destination was mapped without
being written, so it kept whatever the host register held.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The other cores store 0 or 1. Branches and cfc1 only looked at bit 0, so
the guest couldn't tell, but the state differed between cores.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The System V ABI doesn't preserve them either, and the allocator hands
them out, so a helper call (vsin, vrsq and the like) could clobber a
mapped register. CPUEquivalence found it once it ran the IR JIT on
macOS: a k0 pointer in RDX came back as the helper's 1.0f.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ll into $zero mapped $zero for writing, which asserts. On ARM64 and
x86-64, sc wrote its 0/1 result into rt without marking it dirty, so it
was never stored (RISC-V and LoongArch already did this right).
See docs/cpu-equivalence.md.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
vdot, vhdp, vcmp, vf2i, vh2f, vi2f, vi2x, vocp, vsgn, vcmov, the color
conversions, the matrix inits, vmmov and vmscl now leave the prefixes
they don't handle to the interpreter, with the IR frontend's gates. A T
prefix on vcmov applies to D, which only the interpreter did.
sc into $zero wrote its result to an immediate. vmmul's SIMD path
crashed on vmmul.s and on matrices that wrap around the 4x4 block, which
MatrixWrapsAround() now detects.
See docs/cpu-equivalence.md.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With block entries out of memory, the only fake opcode left is the one the CPU cores fetch in
place of a hooked instruction. Call it MIPS_CALL_REPLACEMENT_OP, and its handlers
Int_CallReplacement and Dis_CallReplacement, and remove the sub-op defines and the unused
MIPS_IS_REPLACEMENT.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The JITs used to find a block by overwriting its first instruction with an "emuhack" holding
the offset into the code space, and replacement functions installed their own emuhack the same
way. Anything that read code (savestates, the debugger, memory dumps, cheats, the function
hasher, games reading their own code) had to translate them back.
Blocks now live in a block shadow: one u32 per guest instruction, indexed by the PC masked with
BLOCK_SHADOW_MASK (1GB on 64-bit, folding the kernel and uncached mirrors; 256MB on 32-bit, iOS
and Switch, where address space is short), with the low bits cleared so a misaligned PC can't
read across two entries. Only the ranges under memory views are committed, so a bad PC still
faults in the dispatcher, and the shadow is reserved before the memory map so it can't land in
a gap of the 4GB window. Entries hold full 32-bit offsets, lifting the 16MB limit. The
dispatchers on every backend, and the IR interpreter's loop, look blocks up there.
A hooked instruction without a compiled block gets the JIT's hook value in its entry: the offset
of the dispatcher's compile path (1 for the interpreters, the IR arena keeping that offset
empty), so dispatching there compiles a block, and the frontends substitute the hook. Compiling
replaces the entry with the block's, and destroying the block puts the hook value back. The hook
value is rewritten when the JIT changes or regenerates its dispatcher.
Hooks live in a map keyed by the masked address, and are dropped when the game overwrites the
hooked instruction, or when an icache invalidation finds the function's hash (immediates masked,
as when matching it) changed. Those checks run when queued invalidations are processed, since
cheats and the debugger queue them before writing. Installing and removing hooks invalidates
deferred, as it happens in HLE calls that can run inside a block. The cache instruction goes
through MIPSState, so it drops changed hooks too.
With clean memory, the save/clear/restore around savestates, memory reads and dumps goes away,
as do the source-range invalidations in the memcpy replacements, sceDmac and the cheats.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With the software renderer, which now stores depth as the GE does, a
16-bit read or write at VRAM's 0x04200000 or 0x04600000 mirror goes to
where the GE's 16-bit or 32-bit color layout puts that depth pixel
(ppsspp-re geprobe exp88: the mirrors are the layout's exact inverse).
Games access depth there with nothing else, so other widths stay plain.
The accessors check the address, then whether the GPU core is the
software renderer (CoreParameter); the hardware renderers keep depth plain
and handle the layout their own way. The renderer keeps the layouts up to
date with the EDRAM address translation. With it, the JITs send 16-bit
loads and stores to the interpreter, and the IR passes don't change a
load's width to or from 16 bits.
The pspautotests that read depth through the mirrors now do so 16 bits
at a time, re-recorded on hardware unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The IR temps don't live past the block, but FlushAll stored them anyway at
every exit (the branch operands, lwl/lwr temps, VFPU temp lanes). At an exit,
discard the ones nothing later in the block reads.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A new FSinCos op writes both from one argument reduction. arm64 and x64 get
both back from a single call, packed in one double; RISC-V and LoongArch make
the two calls.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The cosine is then taken of what vrot wrote to that lane: the sine, or zero.
The IR looked at the sine lane instead of the lane holding the angle, and the
legacy JITs ignored the overlap. The assembler refuses such a vrot, so those
now leave it to the interpreter, and don't pair one with the vrot before it.
Covered by the new cpu/vfpu/vrot test.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The thunk saves a fixed set of registers and MXCSR on every call. The
math helpers (vrcp through vrexp2, and vrot's sincos) leave MXCSR alone,
so they now go through CallProtectedLeaf, which saves only the
caller-saved registers the caches are using, around a direct call. vrot
also no longer flushes everything first. x86-64 only; 32-bit x86 keeps
the thunk.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
vh2f always went to the interpreter in the IR. A new FHalfToFloat op
converts the lower or upper half of a word, and the native backends
call vfpu_h2f for it like FSin. The legacy arm64 JIT now makes the same
call instead of computing the conversion inline; vh2f is rare, and the
call is much less code.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Vec4ClampToZero and Vec2ClampToZero only ever fed Vec4Pack31To8 and
Vec2Pack31To16, for vi2uc and vi2us. The packs now clamp negative lanes
to zero themselves, which saves an op and a vector temp, and lets x64
clamp with PACKUSWB's saturation after an arithmetic shift.
While at it, RISC-V compiles Vec2Unpack16To31, Vec2Pack31To16 and
Vec4Pack32To8, and LoongArch Vec2Unpack16To31, Vec2Pack31To16 and the
non-LSX Vec4Pack32To8, all of which went to the IR interpreter.
LoongArch's Vec2Pack32To16 and Vec2Unpack16To32 now take their scalar
path with LSX too, instead of falling back.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
vi2uc, vi2c, vi2us, vi2s, vuc2i, vc2i, vus2i and vs2i lowered to IR ops
that x64 left to the IR interpreter. They're now SSE2: shifts into
place, then PACKSSDW/PACKUSWB for the packs and self-unpacks for the
unpacks. An output overlapping its input still goes the slow way.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
These were always interpreted. They're now IR ops that the native
backends compile to calls to vfpu_exp2 and vfpu_log2, like FSin and
FAsin. vrexp2 is FNeg followed by FExp2, which is how vfpu_rexp2
computes it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
They were added in a different order, so they rounded differently from
every other backend.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
These went through the host's sqrt and division everywhere except the
interpreter's vrcp and vnrcp (vsqrt and vrsq there only behind
USE_VFPU_SQRT, now gone). They now always give the PSP's bits: the IR
gets FVSqrt (FSqrt stays the FPU's IEEE sqrt.s), and FRSqrt and FRecip,
which only the VFPU emits, become vfpu_rsqrt and vfpu_rcp; the IR
interpreter and the x64, arm64, RISC-V and LoongArch backends call them.
The old JITs call them directly, the ARM ones keeping the lanes in
callee-saved registers across the calls. cpu/vfpu/exact now passes on every core.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
x86 (SQRTSS and libm alike) returns 0xffc00000 for it, the PSP 0x7fc00000.
-0 stays -0 and a NaN input comes through as it is, so only a negative
input needs the sign cleared: a compare and an xor in the x64 JITs, a
branch in the interpreters. cpu/fpu/roundmode.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
EN/NN compared the source against an uninitialized temp, usually the
previous lane's all-ones mask (a NaN), so every lane came out as NaN. NI
used a less-than compare, which is false for a NaN. TR set the bit and
then replaced it with a garbage bit from the temp. Found by
cpu/vfpu/specials.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
In the interpreter, the IR interpreter, every backend's FSign and the x86
JIT's own vsgn. Recorded in cpu/vfpu/specials.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
vf2h truncates the mantissa (no rounding), gives a signed zero below 2^-14
and a signed inf from 65536 up, and keeps the low ten mantissa bits of a
NaN, so one with those clear becomes inf. vh2f flushes a subnormal half to
a signed zero and keeps inf/NaN mantissa bits unshifted. The x86 JIT's own
vh2f gets the subnormal flush; the other backends go through the
interpreter. Recorded in cpu/vfpu/specials.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The NaN check compared src1 with itself, so only a NaN in src1 reached
the slow path with the hardware's integer-compare rule. A NaN in src2 went
to MINSS/MAXSS, which just return it. Compare src1 with src2, as the arm64
backend does. Caught by cpu/vfpu/minmax.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
