When a plane's values at both ends of the chunk fit in 32 bits, so do all between, and 32-bit
wrapping arithmetic gets them exactly: then in vector lanes, with the clamp.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The coverage stage was a scalar loop per pixel with the snap edge as a branch. A chunk starts a span,
so relative to it the bounds fit in 32 bits and the snap edge's pixel is a fixed offset per lane.
Also read the depth buffer for the early depth test with one load where SSE isn't used.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The span pixel function and the four-pixel bilinear sampler work in vector lanes, a channel per
vector, but took and returned the colors a pixel at a time, so every span was transposed twice on the
way. Now they take a channel at a time with a stride. The staged rows keep each chunk's colors that
way from the start (their stage walks a channel at a time anyway), and DrawRectangle builds them so.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
DrawSpanVector now covers everything but logic ops and the signed blend factors. Stencil tests pick
the SFail, ZFail or ZPass op per lane, and lanes that only get a new stencil keep their old color.
Pixels are read and written in the framebuffer's own format, converting the whole span at once, and
a span with all four pixels drawn is stored in one go.
Only drawn pixels are written: spans are aligned in screen coordinates, so with a draw offset one can
reach into a tile another thread is drawing.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The last stage of a staged row (texturing, the secondary color and the
pixels) is now a template on what's fixed per triangle: textured or not,
the UV planes fitting 32 bits, the mip level's q and through mode.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
When a triangle's q, s and t plane sums fit 32 bits wherever its spans
are walked, and the planes' exponents keep the GE's UV arithmetic in the
normal float range (GEUVSpanSafe), a span's plane values are one vector
add each and GEUVSpanCore runs without its range checks. Tested bit for
bit against the scalar code under those conditions.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Where the pixel function is the C++ one (and textures have planes), a
row is now drawn in stages over up to 64 pixels at a time: coverage,
depth and the early depth test, then the colors, secondary colors and fog
of all its pixels, then texturing and the pixels span by span. Each stage
keeps little state, where the single span loop spilled most of its.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The four-pixel sampler now runs the texture function (modulate, decal,
blend, replace and add, with color doubling) on all four pixels at once,
one vector per channel, instead of GetTextureFunctionOutput per pixel.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Where the pixel function is the C++ one, triangles and sprites now hand
it a whole span of four pixels. Unless the state has a stencil or color
test, a logic op or a signed blend factor, the span goes through one pixel
per vector lane: clamping, alpha test, fog, depth range, test and write,
blending, dither and the color write mask, as the per-pixel code does
them. Checked against the per-pixel code span by span in 8 games and 263
frame dumps.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Depth, colors, fog and the UV planes are now started once per row and
stepped a span at a time, instead of evaluated from scratch at every
span.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each row's covered pixels now come from the three edges directly, as a
range from 64-bit integer bounds at the pixel centers, and only the spans
in it are visited, with lane masks from the range. The edge functions
per span (32-bit, with a coarse row narrowing) are gone. The mip level's
q comes from the row's covered range instead of edge tests per pixel. A
tall triangle's long edge, which the GE checks at the other end of a span
for its first or last pixel, is still checked per pixel.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
GEUVSpan does the GE's per-pixel UV arithmetic for four pixels at once:
the plane values to float24s by exponent arithmetic, GERecip with its
table entries gathered, and the truncated 24-bit products from the 16-bit
significands' product. Lanes it doesn't cover (values past 2^24,
denormals, results out of range) send the span to the scalar code.
Tested bit for bit against it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
When the four pixels share their mip level (always, for a triangle's
span), their texel coordinates, clamping or wrapping and texel offsets
are computed in vector lanes, and only the texel and palette reads go
one by one. Adds Vec4S32::Min.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The GE walks rows in spans of four pixels and picks the mip level's q
once per span, so the rasterizer now does the same instead of 2x2
quads: triangle spans are aligned like the GE's, so the four pixels
share their level, which is computed once. Auto LOD without planes takes
its vertical derivative from the row below. Sprites, which always have
planes, lose their unused incremental UVs.
Without a sampler JIT, the four bilinear samples of a span are now taken
in one call. Same results on 263 frame dumps and 8 games.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A texel's byte offset is a sum of one for its row and one for its column,
swizzled or not, so the four texels of a bilinear sample now take two of
each, and the CLUT offset of a mip level is found once per sample. The
UV planes at a quad's four pixels are one sum plus steps, as for depth.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The rasterizer takes a GERecip per pixel, so it's inline now, with its
table exported. The q for the mip level is the same for the two pixels
of a quad row unless they're in different 4-pixel spans, so it's only
looked up again then.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Where there's no sampler JIT (everything but x86-64), bilinear sampling
switched on the texture format per pixel, on the CLUT format per texel,
and went through swizzle checks and vector-to-stack round trips. Now a
template instance per texture format, swizzling and CLUT format is
picked with the state, and the filter does all four channels at once in
16-bit lanes of a uint64_t. Same results (checked on 263 frame dumps).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The per-pixel texture coordinates called ldexp with a shift that's fixed
per triangle or line. The shift keeps the values in double's normal
range, so multiplying by a precomputed 2^-shift is exact. Lines also
convert their end values to fixed point once instead of per pixel.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The GE doesn't clip rectangles: one with a vertex behind the camera or outside
the screen range is dropped whole (the software renderer's ProcessRect, from
gpu/clipping/guardband). The hardware backends drew them, stretched to the far
off-screen vertex.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
N.L changes sign from vertex to vertex, so the branch on it was mispredicted a lot. A factor that isn't
positive scales the color to 0, so the term can always be added. Vertex reading about 6% faster in
Tekken 6.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every GEAdd4 and GEMulFloat24x4 checked its lanes for the cases it can't do and branched to a
fallback, a vector-to-scalar move and a branch per operation, about 14% of skinning. The unchecked
variants only collect those lanes; a vertex where any appears is skinned again with the checked ones.
The products use GEMulFloat24x4 (the same as a one-term GERowSum4 here), and the position's and
normal's add chains are interleaved. Skinning is about 20% faster.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The position is read once as (x, y, z, 1) for the clip, fog, texgen and
light vector rows, and lighting works on Vec4F32 throughout, without
going through arrays. Each light's constant term, its position minus
the world translation, is computed once per state instead of per
vertex. The texture matrix rows are now prepared with the transform
state, so loading that matrix dirties it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
GEDot3 forms one term per lane and sums across the lanes, about 1.5-2x
as fast as the scalar GEDot. Adds Vec4S32::GetLane.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Skinning did each component's weighted bone entries and its chain of
GEAdds separately in scalar code. The components now go in lanes, with
GEMulFloat24x4 for the weighted entries, one-term row sums for the
products and GEAdd4 for the sums, in the same order. Checked against the
scalar version on every vertex of 143 skinned frame dumps.
GEMulFloat24x4 matches ProductToFloat24, so inf and NaN bone entries
behave as before. GEAdd4 handles two zero operands without its fallback.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Adds the exponent instead of calling ldexpf, except for reciprocals that
can be denormal. Identical results for every normal float24 input.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
They're truncated to float24 with one AND, and their exponents, fractions
and signs worked out for all terms at once, each term then broadcasting its
lane (new CrossSIMD Vec4S32::SplatLane). The term count becomes a template
parameter so the lanes are constants. About 75-83 M transforms/s on an M1,
from 68; results unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The four clip space rows of a vertex, the texture matrix's three, the world
normal's three, the vector to a light, and each row of the combined matrix
now take one GERowSum4 call. Same results (pspautotests, frametests and
the probe regression set unchanged).
GERowSum4 scales its result by adding to the exponent instead of
multiplying, and CrossSIMD gets a Vec4S32::Max for the largest lsb. About
68 M transforms/s against 26 for four scalar GERowSums on an M1 (a pure
NEON version with per-lane shifts did about 79); replaying the frametest
dumps gets only about 1% faster, up to 11% on Taiko no Tatsujin.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Four rows sharing their left operands, like a vertex transform's: with
float24 operands each exact product is a 32-bit integer, (1.a' * 1.b') >> 15
= 32768 + a' + b' + (a' b' >> 15), and the terms are aligned by scaling with
a power of two and truncating. Lanes whose result exponent is out of a
single float scale's reach take the scalar GERowSum. Tested bit for bit
against GERowSum on random operands (arm64 and x86-64). Not used yet.
Also adds an arithmetic Vec4S32::Shr and the float/int bit casts to
CrossSIMD.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The GE reads the source in aligned 16-byte blocks, four at a time, into a
buffer of four destination-aligned 16-byte slots, and a read of a block a
slot holds returns the slot's bytes, stale ones included. So a transfer
shifting a row right by less than 64 bytes reads some of what it has just
written, in a pattern that isn't a forward or backward copy. Fitted to
ppsspp-re geprobe exp219 and exp220 on hardware (562 of 570 cases exact).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The fast matrix loads and the joined PRIMs read their commands themselves, then skipped them with
UpdatePC(pc, pc + n * 4), which counts the commands up to pc and resumes counting after the skipped
ones: those were never counted, as for a jump. Now pc moves first and UpdatePC counts up to it. The
software renderer runs those commands one at a time, so the backends now agree on the timing.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
gpu/vertices/carry, recorded on a PSP: a draw whose format has no UVs uses the last ones read, by any
earlier draw in draw order, textured or not. They're kept as read and get the scale and offset of the
draw that uses them. Normals carry over the same way.
We kept the UVs after scaling, only read them with texturing on, and after a cached indexed draw
carried the highest vertex instead of the last one drawn. The carried values move from statics in
ReadVertex to TransformUnit.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The same lerps as the GE-exact Bezier path, four at a time. At each v step,
four control columns per vector share the v blending factors; then each row
is evaluated at four u steps, each lane with its own six blending factors.
A vector's u steps can straddle a segment boundary, so the row's points are
gathered per lane, or splatted when all four are in one segment.
The unit test compares it bit for bit with the scalar code, including
tessellations that split a vector across segments. The GE-exact spline
benchmark runs about twice as fast on an M1.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Four of the GE's lerps at once: both operands go to 16-bit fixed point at
the larger exponent by scaling with a power of two and truncating, then the
integer lerp (split into byte halves so both multiplies fit 16 bits), then
back. The patch's four control columns run in the four lanes at each v
step, then four u steps of a row at once, with the row's first level of
lerps converted once per row. Lanes whose exponents are tiny take the scalar
lerp, and the normals' cross product stays scalar for now.
The scalar code stays as the reference (g_splineGEScalar), and the unit test
compares the two bit for bit on random control points. The GE-exact Bezier
benchmark goes from 3.4 to about 8.8 Mverts/s on an M1; the float SIMD path
does about 100.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The software renderer drops a color test that only rejects black under blending that adds to the
destination, since rejecting black then only skips adding nothing. With dithering on it doesn't:
a rejected pixel keeps the destination, an accepted black one gets the dither added. Ridge Racer 2
(20839) adds a glow that way and came out one step darker wherever the dither matrix was negative;
Burnout Legends' bloom (16119) too. Fog and logic ops would change the destination the same way.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
At the end of the 256 KB list buffer, playback jumped back to its start and kept writing, so when
the GE was more than a lap behind, the next lap overwrote commands it hadn't reached. Warriors
(14660) lost half its draws, and GTA LCS (15149) some. Finish the list, wait for it, and start a
new one instead, as the PSP replayer does. Both now match the PSP's replay exactly.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Dumps recorded before savedContextVersion 1 hold PPSSPP's old context layout, with the matrices
as raw floats, but GEState::Restore picks the layout from savedContextVersion, which only a
savestate sets. Playback restored them as the new layout, so the matrices became garbage and
the scene vanished (4140 drew only its HUD). Tell the layouts apart by the END the new one ends
with, as the PSP replayer does. Other callers are unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Older recorders reused data at any offset in the dump, and Prince of Persia: Rival Swords
(11587, 12943) has a CLUT stored one byte into the previous one. The GE ignores the low four
bits of a CLUT or texture address, so the LOADCLUT read 15 bytes early and the palette
channel-swap pass came out red. Map CLUTs, textures and transfer sources that aren't 16-byte
aligned to a separate aligned copy.
Also updates LocoRoco 12058's software reference, which now matches the PSP's replay exactly.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Before 71210f3fa2, a second dump request during a recording started it again, writing another
INIT (and initial CLUT) into the dump: the GE state of that moment, which already includes the
register writes the dump only has after it. Playback applied it, so the last draws ran with the
wrong state: GOD EATER BURST (13950) and GOD EATER 2 (6343) came out black and Street Riders
(14746) garbled. The recorder only writes an INIT when it starts, so skip any later one. The 15
such dumps in the GitHub issue set now match the PSP's replay, 10 of them exactly.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The recorder saves the CLUT the GE had loaded right after INIT, as a CLUT command with no
LOADCLUT to follow, unlike the CLUTs it records at a LOADCLUT. Playback only pointed the CLUT
address at it, so draws before the game's first LOADCLUT used whatever CLUT was loaded before:
HotBrain (16131, ULUS10268) drew its save screen background black. Load it, and put the game's
CLUT address back.
A dump whose last DISPLAY turns the display off (address 0), or has none, now shows the last
framebuffer drawn to (Auditorium 9213). The PSP replayer does both the same way.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A dump records a texture at a swizzled VRAM mirror as the game sampled
it, and the PSP's replayer writes it back with the CPU, which the PSP
sends through the depth layout at any access width. PPSSPP's playback
wrote it raw, so with depth stored in the GE's layout the texture came
back permuted: IL-2's depth texture at 0x04710000 and the two Gensou
Suikoden swizzle dumps. Texture copies, memcpys and memsets into a
mirror now go through the layout, 32 aligned bytes at a time. Against
PSP replays: both Suikoden dumps are now exact (from 1912 and 8083
pixels off), Star Wars #15832 is 67 off (from 2590) and IL-2 894 (from
70505, the rest from a write the next draw reads before it lands).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With the software renderer, which now stores depth as the GE does, a
16-bit read or write at VRAM's 0x04200000 or 0x04600000 mirror goes to
where the GE's 16-bit or 32-bit color layout puts that depth pixel
(ppsspp-re geprobe exp88: the mirrors are the layout's exact inverse).
Games access depth there with nothing else, so other widths stay plain.
The accessors check the address, then whether the GPU core is the
software renderer (CoreParameter); the hardware renderers keep depth plain
and handle the layout their own way. The renderer keeps the layouts up to
date with the EDRAM address translation. With it, the JITs send 16-bit
loads and stores to the interpreter, and the IR passes don't change a
load's width to or from 16 bits.
The pspautotests that read depth through the mirrors now do so 16 bits
at a time, re-recorded on hardware unchanged.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The software renderer kept depth linear in VRAM, where the GE stores it
in the layout GPU/Common/DepthSwizzle.h describes, from the EDRAM address
translation and the color format. So a texture or anything else sharing
VRAM with a depth buffer saw depth land in the wrong places (the player's
shadow in Silent Hill: Origins, #22415). Now every depth access goes
through the layout: the generic pixel functions, the 2x2 depth test, the
depth clear in 16-pixel runs, and the x86 pixel JIT, whose IDs carry the
translation so the layout's constants are compiled in. It changes with
the color format and sceGeEdramSetAddrTranslation, after a flush.
Reading VRAM through its 0x04200000 and 0x04600000 mirrors undoes the
layout, as the GE does: texturing from them (drawn from a linear copy
made after a flush), and block transfers from or to them (Hayate no
Gotoku restores its depth buffer through 0x04600000, #17878). Saved and
debugger depth buffers read through the layout, so they're linear.
Socom's lens flare dump now matches a PSP replay where it changes. About
2-3% more cycles in God of War, Tekken 6, Wipeout and Burnout.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Where the GE stores each depth pixel, from the EDRAM address translation
and the color format: with 32-bit color, address bits 5 up to log2(T) - 1
rotate left by one, then (T << 3) | 0x40 is XORed in (0x600, and bits 9
and 10 swapped, at T 0). The CPU sees the same mapping through VRAM's
0x04200000 and 0x04600000 mirrors. Measured in ppsspp-re geprobe exp8-10
(every pixel of 512x272 buffers at each T) and exp88; the unit test checks
it against the model fitted there.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The tessellator's output was queued as ordinary vertices and run through
a vertex decoder at flush, although the decoded layout is the same as
SimpleVertex. That decode did two things: the UV scale/offset, and
clearing gstate_c.vertexFullAlpha. The tessellator now does both, and
SubmitCurve writes its output where DecodeVerts would have put it and
queues it as predecoded. DecodeVerts only copies such a draw, when it
decodes elsewhere, like the Vulkan and GLES push buffers.
The software renderer decodes by itself and keeps the old path, as do
through mode curves and anything with draws already queued.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each row of vertices is evaluated by folding the patch's four control point
rows into four columns with the v weights, then weighting those along u,
four vertices per register with one attribute component each. Cross
products, normalization and the pole test become plain per-lane math, and
the result is transposed into SimpleVertex order on the way out.
The weights are stored one row per term so four steps load as one vector,
and the control points are padded to full vectors in a single array.
Colors now round to nearest on every platform. The SSE path already did,
NEON and the scalar path truncated.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The copy for texturing from the current render target was cropped to the
UV bounds, but bilinear filtering at the edges reads one texel beyond them,
which then came from whatever the temp FBO last held. On Vulkan that put
magenta streaks into Need for Speed Shift's sun glow, and also hit Call of
Duty, Burnout Dominator (#12831) and Assassin's Creed Bloodlines (#15821).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The fragment Z is the PSP's Z / 65536, and flat draws land exactly on an
integer Z. floor(z * 65535) put that Z - Z/65536 just above the boundary, so
float error (which differs with W) could drop it by one: Patlabor redraws a
HUD icon at the same depth with GEQUAL, and the second draw failed on
Vulkan. Floor Z + 0.01 instead, and store Z / 65535, as clears and the depth
clamp path do. Also fixes Heroes Phantasia #6485.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
