Yabause-PSP's me.prx ORs 0xA0000000 onto its own addresses to share a
buffer with the ME uncached, which is 0xA8xxxxxx for a module in kernel
RAM. The main CPU's space has no views there, so it crashed.
All of kernel segment 1 (0xA0000000-0xBFFFFFFF) now counts as MMIO, and a
device below the hardware registers maps RAM and VRAM onto their uncached
mirrors at 0x40000000. The main CPU's native kernel loads and stores send
the whole segment to that path; the ME keeps 0xBC000000 as the boundary,
since its space maps the rest.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The native dispatcher generated for the ME leaves when its downcount
runs out instead of calling CoreTiming::Advance, and IRNativeJit gets
RunDowncount. Pointerify is decided from the CPU's own memory base. ARM32,
RISC-V and LoongArch64 still give the ME the IR interpreter.
LoadKernel/StoreKernel are now compiled on ARM64 and x64 rather than
going through the IR interpreter one op at a time: memory directly, or a
call to fixed code (which saves the caller-saved registers guest values
live in) for 0xBC000000-0xBFFFFFFF. ARM64's block size estimate counts
them as such. The 16-bit ones stay generic while the software renderer's
depth mirrors are active, as Load16/Store16 do. That matters for the main
CPU's kernel code too, and for all code while the ME is emulated, which
runs in kernel mode everywhere. mcidclan's me-soft-interrupt went from
476 billion to 6.4 billion host instructions for 3 emulated seconds,
ending with the same ME counter as the IR interpreter.
The ME unit tests run on the native JIT too.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Memory::AddressSpace is where a CPU's addresses point (base) and its block
shadow, with the unchecked accessors and block shadow helpers as methods. The
main CPU's is Memory::g_mainSpace, kept in step with Memory::base and
blockShadow, and MIPSState::space points at it.
The CPU cores now go through their state's space instead of the globals: the
interpreter's and IR interpreter's loads and stores (which also saves the IR
interpreter reloading the global base), the IR interpreter's block lookup,
the block table's shadow entries, and the native JITs' memory base register,
block shadow lookups and pointerified registers. Address validity and
everything outside the cores still use the globals.
No change for the main CPU; this is so another CPU (the Media Engine) can see
memory differently. Benchmarked against master (God of War: Chains of
Olympus, Tekken 6): JIT instructions retired within noise, IR interpreter
2% fewer.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
HLEFunc and ReplaceFunc now take the MIPSState they run on. The dispatch
paths (both interpreters and every JIT backend) pass in the state they
belong to, and CallSyscall*, hleFinishSyscall and the HLE mips call
machinery use it instead of currentMIPS.
The wrappers in FunctionWrappers.h take the function as `auto` and call it
through HLECall, which also accepts an implementation with an extra leading
MIPSState * parameter and passes the state to it. Any other signature still
fails to compile, as before. PARAM/RETURN now refer to a local `mips`.
The HLE functions (and replacement functions) that used currentMIPS now take
the state that way. Callers outside a syscall's own frame - CoreTiming
events, the adhoc matching thread, dialogs, PPGe, the debugger UIs - pass
currentMIPS or &mipsr4k explicitly. The kernel's thread internals still use
currentMIPS.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
IRJit is now the base the IR JITs share: the frontend, the table of guest
blocks, and compiling and invalidating them. IRInterpreterJit and
IRNativeJit are the two kinds. Only IRInterpreterJit keeps IR, in its arena,
along with its dispatch loop and per-block profile.
The native JITs no longer keep the IR they compiled from. The debugger and
the --jit-profile report compile a native block's IR again when they want it,
on a copy of the frontend's state, which gives what ran unless the code or
that state has changed since.
IRBlockCache and IRBlock become JitBlockTable and JitBlock: where each block
came from, the page lookup for invalidation, and the block shadow entry (an
arena or native code offset, depending on the JIT). One IRJitDebugInterface
serves both JITs, taking the MIPS code from the table and the IR and host code
from the JIT, in place of the native JIT's own debug interface.
The native backends don't see the table at all. They compile a span of IR for
a block number and start PC, and keep what they need about their own blocks
(start PC, code offset, which block starts where, for linking) next to the
exits they already tracked. The register cache takes the instructions
directly.
The tracer's block index now goes into the IR before the block is stored, so
native blocks get it too. The unused FindByCookie goes. TestJit checks that
the native JIT keeps no IR, and can still show it. The JIT stats overlay no
longer shows IR for the native JITs. MIPSState can't be copied, now that it
owns its JIT.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The JIT backends' calls out of generated code (the MIPS and IR interpreter
fallbacks, and the bad address report) now take the state the backend was
built for instead of reading currentMIPS. Likewise the interpreter's tick
check, the IR dispatcher's Advance, and a couple of debugger helpers.
MIPSState::InitRegisters() no longer sets currentMIPS; the boot paths set it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Fixes#12082
The bit-exact VFPU dot product is the default everywhere and fast enough,
so the plain multiply-add paths in the interpreter and the IR frontend go,
along with the AccurateVdot setting, --accurate-vdot, the ForceAccurateVdot
compat flag and the Vec4Dot IR op, which nothing emits any more.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Big games fill 16 MB in normal play. It's now one constant shared by
every backend. ARM32's 32 MB branch range is the limit on going higher
there.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
DescribeCodePtr() called everything in the fixed code other than a few
routines "fixedCode". It now uses the routine list the profiler has
(moved to the backend as GetAllFixedCodeLabels()), with an offset when
not at the start, so a block ends in "b dispatcherCheckCoreState".
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Mapping a quad whose halves were in registers as pairs (two vs2i.p
feeding a vi2f.q) flushed both and loaded the quad back, a store
forwarding stall.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
New IR ops do them four lanes at a time where the backend has them, the
ARM64 JIT and the IR interpreter so far. vmin and vmax compare by
vfpu_minmax_key() in NEON, which took about twelve instructions a lane
through the integer registers before; vslt and vsge are a compare and an
AND with 1.0.
vi2f becomes one SCVTF with fractional bits.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each jr and jalr now does the dispatcher's lookup itself, so each has its
own indirect branch, which the CPU predicts much better than the one in
the dispatcher that every one of them shared. Out of downcount or with no
block yet, it goes to the dispatcher as before.
Cycles over 10 emulated seconds: -2.3% in ATV, -1.4% Tekken 6 and Ridge
Racer, -1.2% Daxter, -1.1% Killzone, and -0.3% to -0.8% in the rest.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A new IR op, Vec4VfpuMultiDot, does four vfpu_dot()s against one vector,
the other operands being the rows or Vec4s of a 4x4 block of registers.
The frontend emits it for vmmul and vtfm/vhtfm, and for runs of vdot that
fit it, like a matrix times a vector written out.
On ARM64 it calls a routine that has the dots in lanes rather than the
terms, so nothing is horizontal and nothing goes through the integer
registers. The block comes in with LD1, or LD4 for the transposed form,
when none of it is in host registers. The IR interpreter runs it through
vfpu_dot(); the other backends don't get it yet.
The CPU fuzzer gets runs of vdot shaped for it, and finds no difference
between the cores on them.
There's a routine per number of terms, so vdot.t runs don't pay for a
fourth. The vector's significands are worked out once and multiplied by
element, and the rounding at the end is UCVTF's in round-to-nearest,
which is what games run in; the integer version stays for the other
modes. The routine saves nothing, and the call site keeps only those
registers it clobbers that hold guest state.
JitDotBench, ns per dot: vtfm4.q 3.40, vmmul.q 3.12, a run of vdot.t
2.83, against 4.6 for single dots.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The native IR JITs get the emulator thread's host PC sampled every 50 us (a
signal on macOS and Linux, SuspendThread on Windows), mapped to blocks, the
routines they call and host functions. The report has each hot block's MIPS,
IR and host code with the samples per host instruction. The IR interpreter's
block timing, previously behind IR_PROFILING, now runs on the same switch.
Both cost nothing when off.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
FakeJit was the native JIT for architectures without one, but the config
already forces those onto the IR interpreter, so CreateNativeJit() now
returns that instead. The vertex decoder fallback no longer needs
FakeEmitter.
JitBlockCache was the block cache of the pre-IR JITs. Its header keeps
the debug interface types the IR block caches use. Also removes the
JitInterface link hooks only it called, and the JitState fields tied to
it. This fixes a null dereference in the ImDebugger memory section,
which read GetBlockCache(), always null with the IR JITs.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The operands go in q0 and q1 and the result comes back in w1. The routine saves
nothing: the call site flushes whatever guest state is in the few registers it
clobbers (q4-q6, w0-w3), which come late in the allocation order, so usually
there's nothing to do. That's about 20 bytes a call, with nothing going through
memory, and as fast as having the whole thing inline, which made a vmmul.q about
5 KB of code. Inf and NaN go through a routine that calls vfpu_dot_reference().
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
It was integer code, about four times the work. It also clobbers q0-q3 and
x16/x17 now, which both ARM64 JITs keep free, instead of saving them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GenerateArm64VfpuDot() generates vfpu_dot() as integer code, once per JIT, to
be called with BL. The operands and result go through mips->vfpuDotArgs and
vfpuDotResult, and the routine preserves every register but x30, so a call
site is two stores, the BL and a load, with nothing flushed. Inf and NaN
operands go to vfpu_dot_reference(), with the caller-saved registers saved.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The prefix fields were decoded by hand in the interpreter, the IR frontend,
JitState, MIPSState and the disassembler, with three copies of the constant
table. They now share small decoders and one table, in MIPS.h. The backends'
Vec4Shuffle lane decoding gets a helper of its own in IRInst.h.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A flush stores pending imms through SCRATCH1, so in a non-constant MemoryCheck, and in
ValidateAddress's bad-address path, the address computed into SCRATCH1 could be replaced by an imm
before IRRunMemCheck or ReportBadAddress saw it. It now waits in SCRATCH2, which flushes don't use.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Their IR-based replacements stay. The JIT core now creates the IR JIT on both, so "JIT using
IR" is the same as "JIT" on every architecture (ARM32 still uses its legacy JIT for both). The
settings no longer offer it, a saved JIT_IR loads as JIT, and the JIT harness and CPUEquivalence
fuzzer drop the duplicate run.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
So vcmovt never moves and vcmovf always moves (cpu/vfpu/undefined). The
interpreter did nothing for either, and the JITs took it as imm3 = 6,
one CC bit per lane.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On the PSP, zeros and denormals all compare equal, a tie returns the
second operand, and the result keeps its bits (cpu/vfpu/minmax_zero,
minmax_tie, fcr31). The interpreter got the ties and NaNs right but
compared denormals as numbers, and each JIT had its own fast path with
host min/max, which returned the first operand on a tie and flushed or
not depending on the host's mode.
All cores now compare by vfpu_minmax_key(), an integer key that is 0
for a zero exponent and orders the rest by sign and magnitude, and
select the operand by it, without branching on NaN.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
srl/srlv rotate when the low bit of their unused field (rs or sa) is
set and shift when it's clear, whatever the other bits hold
(cpu/cpu_alu/shift_encodings). The interpreter treated values above 1 as
invalid and the other cores shifted.
The Allegrex bit ops decode only some bits of sa, so all 32 values are
one of wsbh, wsbw, seb, seh and bitrev (cpu/cpu_alu/allegrex_encodings).
AllegrexBitOp() gives the canonical one for every core. Special3
function 0x18 does nothing on the PSP, where it was decoded as an alias
of 0x20.
The interpreter dispatch is regenerated, and its generator now lists
each instruction name once. CPUEquivalence no longer skips the srl/srlv
encodings.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A JIT only switched modes around syscalls, and only took the mode into account
in conversions, once it had compiled a ctc1 (hasSetRounding). A mode from an
interpreted ctc1, a savestate made on another core, a core switch or a thread
switch went unnoticed, and HLE then ran in the guest's mode. UpdateFCR31()
now sets the flag when fcr31 isn't the default, and clears the block cache at
the next safe point, since blocks compiled before don't switch either. A new
JIT gets an UpdateFCR31() too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Interpreted ops run in the guest's mode. A syscall left to the interpreter
(with JitDisable flags, say) is host code and now gets the host's mode, like a
compiled one, and an interpreted ctc1 to fcr31 now takes effect right away:
nothing set hasSetRounding for it, so the mode only reached the host at the
next time slice. Every other interpreted op skips the two calls it used to
make after it. The IR frontend's Comp_Generic, which the IR interpreter and the
IR native backends share, gets the same treatment.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comp_Generic restored the host's default mode around the interpreter call, so
an instruction the JIT doesn't compile rounded and flushed differently than in
the interpreter. Now the guest's mode stays in place, and is reapplied after the
call in case the instruction changed fcr31.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
None of the legacy JITs handled them: the destination was mapped without
being written, so it kept whatever the host register held.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mfhi/mflo into $zero mapped $zero as a destination and wrote HI or LO
into its host register, so a later instruction reading $zero (found by
CPUEquivalence as maddu/msub) saw that value. The other JITs and the IR
already skip them.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ll into $zero mapped $zero for writing, which asserts. On ARM64 and
x86-64, sc wrote its 0/1 result into rt without marking it dirty, so it
was never stored (RISC-V and LoongArch already did this right).
See docs/cpu-equivalence.md.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- ext with pos + size past 32 and ins with msb below pos asserted in
UBFX/BFI. The results now match the other cores.
- A swizzle naming a lane past the op's size read an uninitialized
register in ApplyPrefixST. vdot, vcmp, vi2f, vi2x, vocp, vhdp, vcmov
and the matrix inits now leave odd prefixes to the interpreter, with
the same gates as the IR frontend.
- vdot, vhdp, vmmul and vqmul used fused multiply-add, which rounds
differently from every other core. They now multiply and add
separately, in the interpreter's order.
See docs/cpu-equivalence.md.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Config.h is edited about 100 times a year and each edit recompiled 198
files; now 172. SmallDataConvert.h pulls in arm_neon.h and was included by
headers that don't use it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With block entries out of memory, the only fake opcode left is the one the CPU cores fetch in
place of a hooked instruction. Call it MIPS_CALL_REPLACEMENT_OP, and its handlers
Int_CallReplacement and Dis_CallReplacement, and remove the sub-op defines and the unused
MIPS_IS_REPLACEMENT.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The JITs used to find a block by overwriting its first instruction with an "emuhack" holding
the offset into the code space, and replacement functions installed their own emuhack the same
way. Anything that read code (savestates, the debugger, memory dumps, cheats, the function
hasher, games reading their own code) had to translate them back.
Blocks now live in a block shadow: one u32 per guest instruction, indexed by the PC masked with
BLOCK_SHADOW_MASK (1GB on 64-bit, folding the kernel and uncached mirrors; 256MB on 32-bit, iOS
and Switch, where address space is short), with the low bits cleared so a misaligned PC can't
read across two entries. Only the ranges under memory views are committed, so a bad PC still
faults in the dispatcher, and the shadow is reserved before the memory map so it can't land in
a gap of the 4GB window. Entries hold full 32-bit offsets, lifting the 16MB limit. The
dispatchers on every backend, and the IR interpreter's loop, look blocks up there.
A hooked instruction without a compiled block gets the JIT's hook value in its entry: the offset
of the dispatcher's compile path (1 for the interpreters, the IR arena keeping that offset
empty), so dispatching there compiles a block, and the frontends substitute the hook. Compiling
replaces the entry with the block's, and destroying the block puts the hook value back. The hook
value is rewritten when the JIT changes or regenerates its dispatcher.
Hooks live in a map keyed by the masked address, and are dropped when the game overwrites the
hooked instruction, or when an icache invalidation finds the function's hash (immediates masked,
as when matching it) changed. Those checks run when queued invalidations are processed, since
cheats and the debugger queue them before writing. Installing and removing hooks invalidates
deferred, as it happens in HLE calls that can run inside a block. The cache instruction goes
through MIPSState, so it drops changed hooks too.
With clean memory, the save/clear/restore around savestates, memory reads and dumps goes away,
as do the source-range invalidations in the memcpy replacements, sceDmac and the cheats.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With the software renderer, which now stores depth as the GE does, a
16-bit read or write at VRAM's 0x04200000 or 0x04600000 mirror goes to
where the GE's 16-bit or 32-bit color layout puts that depth pixel
(ppsspp-re geprobe exp88: the mirrors are the layout's exact inverse).
Games access depth there with nothing else, so other widths stay plain.
The accessors check the address, then whether the GPU core is the
software renderer (CoreParameter); the hardware renderers keep depth plain
and handle the layout their own way. The renderer keeps the layouts up to
date with the EDRAM address translation. With it, the JITs send 16-bit
loads and stores to the interpreter, and the IR passes don't change a
load's width to or from 16 bits.
The pspautotests that read depth through the mirrors now do so 16 bits
at a time, re-recorded on hardware unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Only the rounding mode, flags, enables, cause, FCC and FS bits can be
written (0x0181FFFF, pspautotests cpu/fpu/fcr), as the interpreter, IR
and x86 already have it. Both ARM JITs stored the whole value.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
When a Load32, Store32, LoadFloat or StoreFloat is followed by the same op
on the next or previous word through the same base, and the base can be
mapped as a pointer, emit one LDP/STP for both. A struct-copying loop runs
about 24% faster; code without such pairs is unaffected.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The IR temps don't live past the block, but FlushAll stored them anyway at
every exit (the branch operands, lwl/lwr temps, VFPU temp lanes). At an exit,
discard the ones nothing later in the block reads.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A new FSinCos op writes both from one argument reduction. arm64 and x64 get
both back from a single call, packed in one double; RISC-V and LoongArch make
the two calls.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The cosine is then taken of what vrot wrote to that lane: the sine, or zero.
The IR looked at the sine lane instead of the lane holding the angle, and the
legacy JITs ignored the overlap. The assembler refuses such a vrot, so those
now leave it to the interpreter, and don't pair one with the vrot before it.
Covered by the new cpu/vfpu/vrot test.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Values in S8-S15 survive a call, but the full flush wrote them back and
the following instructions loaded them again. The VFPU math callouts,
vrot and vh2f now flush only the caller-saved registers, plus the few
callee-saved ones they stage values in, and map the destinations
afterwards. In a normalize loop with sixteen VFPU registers live, that
takes a vrsq.s from about 15 to 11.5 ns.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ll and sc always went to the interpreter; a few games use them
thousands of times. They now load and store directly with fast memory,
keeping llBit in MIPSState. vcmp's NaN and inf-or-NaN tests only look
at s, so they no longer need vt to be the same register: EN and NN
compare s with itself.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
vh2f always went to the interpreter in the IR. A new FHalfToFloat op
converts the lower or upper half of a word, and the native backends
call vfpu_h2f for it like FSin. The legacy arm64 JIT now makes the same
call instead of computing the conversion inline; vh2f is rare, and the
call is much less code.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
All went to the interpreter. vh2f works in integers to match vfpu_h2f,
since FCVTL neither flushes subnormal halves nor keeps inf/NaN mantissa
bits unshifted. The color conversions are bitfield extracts and inserts.
vbfy, vcrs and vdet follow the IR frontend, and vmscl scales element
by element. Results go through scratch registers, so a destination that
overlaps a source is fine.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Vec4ClampToZero and Vec2ClampToZero only ever fed Vec4Pack31To8 and
Vec2Pack31To16, for vi2uc and vi2us. The packs now clamp negative lanes
to zero themselves, which saves an op and a vector temp, and lets x64
clamp with PACKUSWB's saturation after an arithmetic shift.
While at it, RISC-V compiles Vec2Unpack16To31, Vec2Pack31To16 and
Vec4Pack32To8, and LoongArch Vec2Unpack16To31, Vec2Pack31To16 and the
non-LSX Vec4Pack32To8, all of which went to the IR interpreter.
LoongArch's Vec2Pack32To16 and Vec2Unpack16To32 now take their scalar
path with LSX too, instead of falling back.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
These went to the interpreter. vf2in/vf2iz/vf2iu/vf2id use FCVT, which
saturates like the PSP, and patch NaN to 0x7FFFFFFF like the IR
backend. vsgn keeps the sign bit on 1.0 and gives 0 below the smallest
normal. vsge and vslt select 1 or 0 on a compare whose condition is
false when unordered. EI and NI test |s| against infinity in integers.
All match the interpreter on special values, ties and denormals.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
These were always interpreted. They're now IR ops that the native
backends compile to calls to vfpu_exp2 and vfpu_log2, like FSin and
FAsin. vrexp2 is FNeg followed by FExp2, which is how vfpu_rexp2
computes it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
vsin, vcos, vnsin, vasin, vexp2, vlog2 and vrexp2 went to the
interpreter. They now take the same direct call as vrcp and friends.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
