Config.h is edited about 100 times a year and each edit recompiled 198
files; now 172. SmallDataConvert.h pulls in arm_neon.h and was included by
headers that don't use it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The crosshair turns red, and lock-on works, when stencil bit 3 is set at
the centre of the frame, which the game tests by reading the pixel from
VRAM with the CPU (#17936). Hook that and read back the stencil of that
row into the alpha bytes in RAM. The alpha the hardware backends keep
doesn't follow stencil everywhere, so reading color isn't enough.
Adds PerformReadbackStencilToMemory for this (8888 only for now).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The game converts each video frame into RAM and copies it into the
framebuffer with its own memcpy, which we didn't recognize. The hardware
backends never saw the copy and kept showing the last frame the GE drew.
Same function in the EU (ULES00192) and Korean (ULKS46038) versions.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The spinner copies the displayed framebuffer under it into the transparent
texels of its frames, which came out as stale VRAM around the disc. Hook it
after it gets the framebuffer and read back the rows it covers.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The first game builds its portrait like Golf 2 and Tennis, but its copy
function keeps y and h in different registers and loads the framebuffer pointer
at a different offset. Move the readbacks into a helper and add a hook for this
layout.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Tennis builds its portraits like Everybody's Golf 2: it memcpys a 64x64 rect of
the framebuffer and sets each texel's alpha bit from depth read straight from
VRAM, so the circles in the corners came out empty. Its copy function is
scheduled differently and hashes differently, so add it as a second entry for
the same hook, which is renamed to fit both.
Fixes#17826.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The game builds each 64x64 portrait by memcpying a rect of the framebuffer one
partial row at a time, then setting each texel's alpha bit from the depth
buffer, which it reads directly from VRAM. Neither got read back: the row
copies don't line up with a framebuffer, and depth is never downloaded.
Hook the copy function to read back both, whole rows at a time. That needed two
fixes to requested depth copies: they were dropped when the depth buffer lies
above all color buffers, and only an exact full-buffer size matched.
The portrait texture now matches the software renderer, though it still
doesn't show up on screen with Vulkan.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With block entries out of memory, the only fake opcode left is the one the CPU cores fetch in
place of a hooked instruction. Call it MIPS_CALL_REPLACEMENT_OP, and its handlers
Int_CallReplacement and Dis_CallReplacement, and remove the sub-op defines and the unused
MIPS_IS_REPLACEMENT.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Memory no longer holds the hooks, so the disassembly fetches the op the CPU runs, which is the
CallRepl pseudo-op at a hooked address. That lookup can come from the Win32 debugger's UI thread,
so the CPU thread now changes the hook map under a lock (and reads it without, being the only
writer).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The JITs used to find a block by overwriting its first instruction with an "emuhack" holding
the offset into the code space, and replacement functions installed their own emuhack the same
way. Anything that read code (savestates, the debugger, memory dumps, cheats, the function
hasher, games reading their own code) had to translate them back.
Blocks now live in a block shadow: one u32 per guest instruction, indexed by the PC masked with
BLOCK_SHADOW_MASK (1GB on 64-bit, folding the kernel and uncached mirrors; 256MB on 32-bit, iOS
and Switch, where address space is short), with the low bits cleared so a misaligned PC can't
read across two entries. Only the ranges under memory views are committed, so a bad PC still
faults in the dispatcher, and the shadow is reserved before the memory map so it can't land in
a gap of the 4GB window. Entries hold full 32-bit offsets, lifting the 16MB limit. The
dispatchers on every backend, and the IR interpreter's loop, look blocks up there.
A hooked instruction without a compiled block gets the JIT's hook value in its entry: the offset
of the dispatcher's compile path (1 for the interpreters, the IR arena keeping that offset
empty), so dispatching there compiles a block, and the frontends substitute the hook. Compiling
replaces the entry with the block's, and destroying the block puts the hook value back. The hook
value is rewritten when the JIT changes or regenerates its dispatcher.
Hooks live in a map keyed by the masked address, and are dropped when the game overwrites the
hooked instruction, or when an icache invalidation finds the function's hash (immediates masked,
as when matching it) changed. Those checks run when queued invalidations are processed, since
cheats and the debugger queue them before writing. Installing and removing hooks invalidates
deferred, as it happens in HLE calls that can run inside a block. The cache instruction goes
through MIPSState, so it drops changed hooks too.
With clean memory, the save/clear/restore around savestates, memory reads and dumps goes away,
as do the source-range invalidations in the memcpy replacements, sceDmac and the cheats.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
78c75e1a0b dropped IRNativeJit::InvalidateCacheAt as a duplicate, and with it the only call to
the backend's InvalidateBlock. Since then a block linked to an invalidated one kept jumping into
its old code, past the dispatcher. Call it from IRJit::InvalidateCacheAt through a virtual, like
the other native block hooks.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With the software renderer, which now stores depth as the GE does, a
16-bit read or write at VRAM's 0x04200000 or 0x04600000 mirror goes to
where the GE's 16-bit or 32-bit color layout puts that depth pixel
(ppsspp-re geprobe exp88: the mirrors are the layout's exact inverse).
Games access depth there with nothing else, so other widths stay plain.
The accessors check the address, then whether the GPU core is the
software renderer (CoreParameter); the hardware renderers keep depth plain
and handle the layout their own way. The renderer keeps the layouts up to
date with the EDRAM address translation. With it, the JITs send 16-bit
loads and stores to the interpreter, and the IR passes don't change a
load's width to or from 16 bits.
The pspautotests that read depth through the mirrors now do so 16 bits
at a time, re-recorded on hardware unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Only the rounding mode, flags, enables, cause, FCC and FS bits can be
written (0x0181FFFF, pspautotests cpu/fpu/fcr), as the interpreter, IR
and x86 already have it. Both ARM JITs stored the whole value.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An interpreter state always claimed an uneaten VFPU prefix, which made a
JIT loading it run in unknown-prefix mode for the rest of the session.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The list belongs to the sceGe call still in progress, whose end would have
run on the loaded CPU state. Also stop the camera and GPS when a state has
them off, don't restart capture when saving, and fix a double free of the
pmp frame queue (it only holds the media engine's own frame).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The function list is from before the load, where other code (an overlay
module) may have been. Hashing it again from the loaded memory keeps the
hooks to code that actually matches.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
When a Load32, Store32, LoadFloat or StoreFloat is followed by the same op
on the next or previous word through the same base, and the base can be
mapped as a pointer, emit one LDP/STP for both. A struct-copying loop runs
about 24% faster; code without such pairs is unaffected.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every op ended with a break back to one shared indirect jump, which the
CPU has to predict for every op in the program. With labels as values,
each op jumps through a table from its own site instead, which predicts
much better: an integer-heavy benchmark runs about 13% faster on an M1.
Other compilers keep the switch, and ops missing from the table fall back
to it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Blocks ending in a branch dispatch a conditional exit and then the
fallthrough ExitToConst. One op now returns either target, reading the
second from the ExitToConst, which stays behind unexecuted.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
It avoids flushes a native backend would need, at the cost of extra
instructions (a copy of each scalar before a Vec4Scale, for instance) that
the interpreter only has to dispatch.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Both were no-ops, so mfic left its destination unchanged. They read and
write the interrupt enable flag that sceKernelCpuSuspendIntr/ResumeIntr
use. Only bit 0 counts for mtic, which also goes for
sceKernelCpuResumeIntr, since on hardware it's just mtic.
Adds the intr/mfic test, recorded on hardware.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
OptimizeLoadsAfterStores only dropped a load right after a store of the same
reg. Now a load of anything the block stored or loaded before becomes a reg
move (with the extension for 8/16-bit loads), as long as nothing in between
may have changed the memory, the address reg, or the reg holding the value.
Only a store through the same base at a disjoint range is known not to alias,
and constant addresses outside RAM are left alone.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
When the cosine lane follows the sine lane, FSinCos can write both in place
instead of going through a temp and two FMovs.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The IR temps don't live past the block, but FlushAll stored them anyway at
every exit (the branch operands, lwl/lwr temps, VFPU temp lanes). At an exit,
discard the ones nothing later in the block reads.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- A constant stays known after it's written out for a read (by a store,
MovZ, a multiply...), so later uses still fold. Whatever an op writes is
forgotten after its inputs are written, and setting a reg to the value it
already holds isn't written twice.
- A conditional exit that isn't taken keeps the constants known.
- The saturating and min/max FP ops, FSign and the 31-bit Vec2 pack/unpack
no longer flush every GPR constant.
- A load through its own base is folded (lui v0, hi; lw v0, lo(v0)).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- An lwl/lwr pair was combined into one load even when the first half loads
into the base register, which changes the address of the second half.
- ApplyMemoryValidation shared one sp check across the block even past an
Interpret or CallReplacement, which may change sp.
- Drop a duplicate FSqrt meta entry, and name Load8Ext correctly.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Copy propagation through an FPR temp didn't stop when an instruction
rewrote the temp in place (it compared an FPR number against the +32
offset reg), so later reads lost that write.
- A read of the temp in both operands only had src1 replaced, yet the copy
into the temp was still removed.
- The replacement matched operands by number without checking their type,
so a StoreFloat whose GPR address had the temp's number got its address
replaced (IRVTEMP_PFX_S and IRTEMP_0 are both 192).
- A write to lanes 1-3 of a Vec4 temp wasn't noticed.
- IRReadsFromFPRs stopped after the F operands, missing Vec4Scale's vector.
- Exits and barriers didn't count as reading everything, so a write to a
real reg could be moved above an exit.
- Load32Linked and Store32Conditional were removed when their reg was
overwritten unread, losing LLBIT and the store.
Also fixes an off-by-one in the vec src3 read check. The unit test now
reports every failing case instead of stopping at the first.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
After the registers usable by compressed instructions, prefer s2-s7 and
fs2-fs11, so that fewer values have to be flushed around calls. The
dispatcher already saves them.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A new FSinCos op writes both from one argument reduction. arm64 and x64 get
both back from a single call, packed in one double; RISC-V and LoongArch make
the two calls.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The cosine is then taken of what vrot wrote to that lane: the sine, or zero.
The IR looked at the sine lane instead of the lane holding the angle, and the
legacy JITs ignored the overlap. The assembler refuses such a vrot, so those
now leave it to the interpreter, and don't pair one with the vrot before it.
Covered by the new cpu/vfpu/vrot test.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The ABI only preserves the low 64 bits of F24-F31, which are first in
the allocation order, so a four-lane vector there lost its upper half
across a call to a math helper. Flush those like arm64 does for
S8-S15.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The thunk saves a fixed set of registers and MXCSR on every call. The
math helpers (vrcp through vrexp2, and vrot's sincos) leave MXCSR alone,
so they now go through CallProtectedLeaf, which saves only the
caller-saved registers the caches are using, around a direct call. vrot
also no longer flushes everything first. x86-64 only; 32-bit x86 keeps
the thunk.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Values in S8-S15 survive a call, but the full flush wrote them back and
the following instructions loaded them again. The VFPU math callouts,
vrot and vh2f now flush only the caller-saved registers, plus the few
callee-saved ones they stage values in, and map the destinations
afterwards. In a normalize loop with sixteen VFPU registers live, that
takes a vrsq.s from about 15 to 11.5 ns.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With LSX a lane group can be mapped as one vector reg, and F() then returns
the same reg for every lane, so the per-lane code wrote only lane 0 (vs2i,
vus2i). Also drop the stale aliasing TODOs; each path reads its sources first.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Only 4x4 with source and destination transposed alike, and a scale
outside the destination, compiled; most vmscl in games are transposed
or 3x3. The rest now multiply element by element, with the scale
copied first, and only a partly overlapping source still falls back.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ll and sc always went to the interpreter; a few games use them
thousands of times. They now load and store directly with fast memory,
keeping llBit in MIPSState. vcmp's NaN and inf-or-NaN tests only look
at s, so they no longer need vt to be the same register: EN and NN
compare s with itself.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Without the Zbb extension, clz, rotr, wsbh, wsbw, bitrev, min and max
went to the IR interpreter, and wsbh always did. They're now base ISA
sequences: a branchless binary search for clz, paired shifts for the
rotates and byte swaps, mask-and-shift steps for bitrev, and a
compare-and-branch for min and max. Checked against a model of the
instructions over random inputs, since nothing here runs RISC-V.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
They always went to the interpreter. Each channel is now a shift, mask
and shift in the existing integer ops, so every backend handles them.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
vh2f always went to the interpreter in the IR. A new FHalfToFloat op
converts the lower or upper half of a word, and the native backends
call vfpu_h2f for it like FSin. The legacy arm64 JIT now makes the same
call instead of computing the conversion inline; vh2f is rare, and the
call is much less code.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
All went to the interpreter. vh2f works in integers to match vfpu_h2f,
since FCVTL neither flushes subnormal halves nor keeps inf/NaN mantissa
bits unshifted. The color conversions are bitfield extracts and inserts.
vbfy, vcrs and vdet follow the IR frontend, and vmscl scales element
by element. Results go through scratch registers, so a destination that
overlaps a source is fine.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Vec4ClampToZero and Vec2ClampToZero only ever fed Vec4Pack31To8 and
Vec2Pack31To16, for vi2uc and vi2us. The packs now clamp negative lanes
to zero themselves, which saves an op and a vector temp, and lets x64
clamp with PACKUSWB's saturation after an arithmetic shift.
While at it, RISC-V compiles Vec2Unpack16To31, Vec2Pack31To16 and
Vec4Pack32To8, and LoongArch Vec2Unpack16To31, Vec2Pack31To16 and the
non-LSX Vec4Pack32To8, all of which went to the IR interpreter.
LoongArch's Vec2Pack32To16 and Vec2Unpack16To32 now take their scalar
path with LSX too, instead of falling back.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
vi2uc, vi2c, vi2us, vi2s, vuc2i, vc2i, vus2i and vs2i lowered to IR ops
that x64 left to the IR interpreter. They're now SSE2: shifts into
place, then PACKSSDW/PACKUSWB for the packs and self-unpacks for the
unpacks. An output overlapping its input still goes the slow way.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
FCvtWS always went to the IR interpreter. Both now convert in the
current rounding mode, which ApplyRoundingMode keeps at the game's, as
x64 does. RISC-V's FCVT already saturates and gives INT_MAX for NaN;
LoongArch patches NaN to INT_MAX like its FRound.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
They always went to the IR interpreter. Now a signed compare-and-branch
on the normalized sources picks which one to move.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
