Where the mip level comes from q (slope mode, or auto with UV planes) and q is
the same over the whole triangle, its pixels all have one level. The staged
spans went through ApplyTexturing for each four pixels anyway, to work out that
level and call the quad sampler; a variant of DrawStagedSpans now works it out
once and calls the quad sampler directly. It's a separate variant, as the extra
code in the shared one cost other games about 1% of instructions on an M1.
LocoRoco takes 7.5% fewer instructions and 5% less wall time, Wipeout 2% fewer
instructions; the others are unchanged. The frame dump set renders the same.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GEDot3 and GERowSum called ldexpf for each result, and the UV plane
setup ldexp and ilogb for every triangle and tile, about 2% of the CPU
time between them. Adding to the exponent field gives the same values
while they stay normal (ScaleByPow2Exact, which calls ldexpf otherwise),
and the planes' values and shifts always are. 2-4% fewer instructions
in every game measured.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Vec4<int> and Vec4<float>'s AssignToAll went through the four value
constructor, which stores the values one by one, so whatever loaded the
vector next stalled: the edge functions' setup per triangle and tile,
among others. They now splat in a register on SSE and NEON.
ReadSpanDepth loads its four depths as one vector where they're stored
together, rather than through an array, and ComputePlane is inline, as
returning its plane through memory stalled the UV planes' setup the same
way.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The edge functions' start built a constant vector through Vec4<int>'s
four value constructor, which stores the values one by one, and the UV
ramps went through a stack array: each reloaded as a vector, a store
forwarding stall per triangle and tile. The constant is now loaded from
a static array, and the ramps {0, d, 2d, 3d} are made from d in lanes.
Wipeout takes 3.4 s for five emulated seconds (from 3.5 s), God of War
3.6 s (from 3.7 s).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ApplyTexturing's per-pixel path built each pixel's color from four
strided channel values, which the compiler stored in pieces and then
loaded as a vector, a store forwarding stall per pixel. On x86 and
ARM64, where the sampler takes its color in a register, the channel
rows are transposed in registers instead, and back afterwards.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The C++ paths draw and sample four pixels at a time in vector lanes,
while a JIT function only does one, and with a JIT function in use the
span and quad paths were skipped. On x86 the JITs had become slower:
Wipeout took 13% more wall time and 25% more CPU with them, God of War
3% more. They were also a second implementation of every GE rule, and
ARM never had them.
The SoftwareRendererJit setting goes with them. RasterizerRegCache.h
keeps only the types the pixel and sampler functions take, as
RasterizerTypes.h.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The GE doesn't divide by a q <= 0: it samples the last texel in u and v,
whatever s and t are, in clamp and repeat modes alike (gpu/texmtx/negq). The
hardware backends divided, which mirrors a projected texture behind its
projector: Fate/Extra's character shadows ran up the school's walls (#18667).
The software renderer sampled (0, 0) instead.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Staged spans without mip levels pass no level arrays, and the sampler goes
straight to its one level path. 4% fewer instructions in God of War.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Without mip levels and with one filter for both, a span's texturing is the
quad sampler at level 0: staged spans call it directly instead of going
through ApplyTexturing, which also skips the level calculation. God of War
retires 7% fewer instructions.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The coverage and the drawn range took the same three edge bounds, each a
64-bit division. 1% fewer cycles in God of War.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
CheckDepthTestPassed4 only has an SSE path, so on ARM it compared lane by
lane. Adds Vec4S32::LoadU16 for the depth values.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A flat secondary color of zero (no specular) isn't added to each span, and
a flat secondary color or absent fog fills its buffer once per triangle
rather than once per row. 3% fewer instructions in God of War.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
DrawRectangle computed each span's UVs from the planes in 64 bits through the general GEUVSpan. It
now fills the staged rows' buffers (flat colors, z and fog, the coverage and the early depth test)
and draws them with DrawStagedSpans, which takes the 32-bit UV path when the planes allow it. Each
pixel still picks its mip level from its own q.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
When a plane's values at both ends of the chunk fit in 32 bits, so do all between, and 32-bit
wrapping arithmetic gets them exactly: then in vector lanes, with the clamp.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The coverage stage was a scalar loop per pixel with the snap edge as a branch. A chunk starts a span,
so relative to it the bounds fit in 32 bits and the snap edge's pixel is a fixed offset per lane.
Also read the depth buffer for the early depth test with one load where SSE isn't used.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The span pixel function and the four-pixel bilinear sampler work in vector lanes, a channel per
vector, but took and returned the colors a pixel at a time, so every span was transposed twice on the
way. Now they take a channel at a time with a stride. The staged rows keep each chunk's colors that
way from the start (their stage walks a channel at a time anyway), and DrawRectangle builds them so.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The last stage of a staged row (texturing, the secondary color and the
pixels) is now a template on what's fixed per triangle: textured or not,
the UV planes fitting 32 bits, the mip level's q and through mode.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
When a triangle's q, s and t plane sums fit 32 bits wherever its spans
are walked, and the planes' exponents keep the GE's UV arithmetic in the
normal float range (GEUVSpanSafe), a span's plane values are one vector
add each and GEUVSpanCore runs without its range checks. Tested bit for
bit against the scalar code under those conditions.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Where the pixel function is the C++ one (and textures have planes), a
row is now drawn in stages over up to 64 pixels at a time: coverage,
depth and the early depth test, then the colors, secondary colors and fog
of all its pixels, then texturing and the pixels span by span. Each stage
keeps little state, where the single span loop spilled most of its.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Where the pixel function is the C++ one, triangles and sprites now hand
it a whole span of four pixels. Unless the state has a stencil or color
test, a logic op or a signed blend factor, the span goes through one pixel
per vector lane: clamping, alpha test, fog, depth range, test and write,
blending, dither and the color write mask, as the per-pixel code does
them. Checked against the per-pixel code span by span in 8 games and 263
frame dumps.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Depth, colors, fog and the UV planes are now started once per row and
stepped a span at a time, instead of evaluated from scratch at every
span.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each row's covered pixels now come from the three edges directly, as a
range from 64-bit integer bounds at the pixel centers, and only the spans
in it are visited, with lane masks from the range. The edge functions
per span (32-bit, with a coarse row narrowing) are gone. The mip level's
q comes from the row's covered range instead of edge tests per pixel. A
tall triangle's long edge, which the GE checks at the other end of a span
for its first or last pixel, is still checked per pixel.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
GEUVSpan does the GE's per-pixel UV arithmetic for four pixels at once:
the plane values to float24s by exponent arithmetic, GERecip with its
table entries gathered, and the truncated 24-bit products from the 16-bit
significands' product. Lanes it doesn't cover (values past 2^24,
denormals, results out of range) send the span to the scalar code.
Tested bit for bit against it.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The GE walks rows in spans of four pixels and picks the mip level's q
once per span, so the rasterizer now does the same instead of 2x2
quads: triangle spans are aligned like the GE's, so the four pixels
share their level, which is computed once. Auto LOD without planes takes
its vertical derivative from the row below. Sprites, which always have
planes, lose their unused incremental UVs.
Without a sampler JIT, the four bilinear samples of a span are now taken
in one call. Same results on 263 frame dumps and 8 games.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A texel's byte offset is a sum of one for its row and one for its column,
swizzled or not, so the four texels of a bilinear sample now take two of
each, and the CLUT offset of a mip level is found once per sample. The
UV planes at a quad's four pixels are one sum plus steps, as for depth.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The rasterizer takes a GERecip per pixel, so it's inline now, with its
table exported. The q for the mip level is the same for the two pixels
of a quad row unless they're in different 4-pixel spans, so it's only
looked up again then.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The per-pixel texture coordinates called ldexp with a shift that's fixed
per triangle or line. The shift keeps the values in double's normal
range, so multiplying by a precomputed 2^-shift is exact. Lines also
convert their end values to fixed point once instead of per pixel.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The software renderer kept depth linear in VRAM, where the GE stores it
in the layout GPU/Common/DepthSwizzle.h describes, from the EDRAM address
translation and the color format. So a texture or anything else sharing
VRAM with a depth buffer saw depth land in the wrong places (the player's
shadow in Silent Hill: Origins, #22415). Now every depth access goes
through the layout: the generic pixel functions, the 2x2 depth test, the
depth clear in 16-pixel runs, and the x86 pixel JIT, whose IDs carry the
translation so the layout's constants are compiled in. It changes with
the color format and sceGeEdramSetAddrTranslation, after a flush.
Reading VRAM through its 0x04200000 and 0x04600000 mirrors undoes the
layout, as the GE does: texturing from them (drawn from a linear copy
made after a flush), and block transfers from or to them (Hayate no
Gotoku restores its depth buffer through 0x04600000, #17878). Saved and
debugger depth buffers read through the layout, so they're linear.
Socom's lens flare dump now matches a PSP replay where it changes. About
2-3% more cycles in God of War, Tekken 6, Wipeout and Burnout.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Transform-mode lines take the triangle's per-vertex s, t, q (15 bits at the
larger exponent) along the line's gradient, like their color and depth, then
u = s * R(q). A pixel just inside a line's end samples short of the end
vertex's coordinate rather than at it (gpu/texmtx/prims: texel row 255, not a
wrap to 0).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The same way as Gouraud color: a gradient along the major axis from the
setup reciprocal, and each pixel's value from its center's signed distance
from v0, so an endpoint's pixel extrapolates past the endpoint's depth
(gpu/probe exp170, exact; it had 435 wrong pixels). Coded Arms' (ULUS10019)
last pixel, where a short line passed a GEQUAL test, and SOCOM's line depth.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Which side an edge is on decides whether a pixel center exactly on it is
inside. The test divided with truncation, so a vertex 0.2 subpixels left of
the opposite edge counted as on it, and a thin triangle's right edge became
inclusive. Peace Walker (ULUS10509), ULJM05302 and mali_missing_circle PSP
replays become exact.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With no steps along the line, both endpoint weights were zero, so fog (and
UVs) interpolated to zero: full fog. SOCOM's (UCUS98649) radar lines, a few
pixels long in an orthographic view, came out in the fog color.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An endpoint exactly on the left or right corner of a pixel's diamond (top
or bottom for y-major lines) doesn't count as inside it: the pixel is lit
only if the line crosses the diamond (gpu/probe exp149). Makes exp72,
Virtua Tennis and echochrome exact.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Next to an edge between two z = 0 vertices, the floored depth plane can
come out -1. The GE stores and range-tests 0 there, in through and
transform mode, with depth clamp on or off (gpu/probe exp148). Fixes a
seam across the sky in Frontier Gate Boost.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The forced alpha test for src alpha / inv src alpha blending assumed a
zero alpha pixel leaves the target unchanged, but with dithering on the
GE still writes dst + dither (gpu/probe exp141). Makes the Test Drive
(ULUS10249) frame dump exact.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A line's color gradient along its major axis comes from the triangle
setup's reciprocal, with 14 fraction bits per subpixel, floored, and each
pixel is the start color plus the gradient times its center's signed
distance from v0 along the line. A diagonal line (|dx| == |dy|) is
y-major. gpu/probe exp137, exact; fixes the line strips in a Final Fantasy
Zero frame dump.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
When a triangle's long edge (top to bottom vertex) has 3 dy > 2^17 in
screen subpixels, the GE lights the first pixel of each 4-pixel span along
it (the last, for a right edge) when any of the span is inside, likely an
overflowing 3 dy span offset. Bottom ties end the long edge at the left
vertex. Such triangles take a scalar mask path. gpu/probe exp130-135, exact;
none of 216 frame dumps has such a triangle.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The GE builds a sprite's UV plane from three corners whose s, t and q it
takes component by component from the two vertices: s from the vertex that
supplies the corner's x, t from the one that supplies its y (swapped when the
sprite is rotated), and q from the one that supplies its x. With different w
at the two vertices that bends the mapping; we used v1's w at every corner.
gpu/probe exp129: all four vertex orders at w = 1, 2.7 and differing w, exact.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A sprite drawn from bottom left to top right is rotated: its first row
rounds like a column ((y + 6) >> 4), and its UV plane comes from v0, v1 and
the bottom right corner, anchoring it at the bottom left. The other three
vertex orders were already exact (gpu/probe exp121-122, through and
transform mode). Fixes the last differences in an MGS frame dump (HUD
sprites).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Which boundary points of a pixel's diamond count as inside is transposed for
y-major lines: the left corner, the edges beside it and the top corner,
where x-major lines use the top corner, its edges and the left corner. exp72
went from 12 differing pixels to 1, and antialiased lines (exp112, same
pixels) from 117 to 101.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The float24 truncations, reciprocal and rsqrt tables, GE adder and row sums,
the setup reciprocal for planes, the mip log2, texture coordinate precision,
the lighting pow and color scaling, and the antialiased line alpha, from
TransformUnit, Rasterizer and Lighting, so they can be unit tested. No
behavior change.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Antialiased lines light the same pixels as plain ones, and each pixel's
alpha (replacing the vertex alpha) is 128 - |v|, v the largest integer
below 256 times the pixel center's offset from the line along the minor
axis (gpu/probe exp112: about 98% of line pixels exact, through and
transform mode; the rest look like the GE's stepper). Echochrome's MSE
against the PSP goes from 170 to 0.12.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The GE picks a triangle pixel's mip level (auto and slope modes) with the q
at the second pixel of its row's four-pixel span in the direction it walks
the row: x = 4k + 1 left to right, but 4k + 2 for triangles whose long edge
is the right side (the same condition as the plane anchor). When that pixel
is outside the triangle, the span's first covered pixel in that order. Auto
LOD under perspective is exact (gpu/probe exp93, exp103-106, the latter built
from a GTA LCS triangle), and the slope mode probes stay exact.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The check ANDs every CLUT entry, but took any common bit as proof that no
alpha is zero, so a white palette with alphas 0, 0x10, ... lost its alpha
test. Only the alpha bits count now. Ace Combat's subtitles become bit exact
against the PSP.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The auto level is the largest of the four UV gradients (du/dx, du/dy, dv/dx,
dv/dy) in texels per pixel, taken from the s and t planes, minus the pixel's
q, both through the GE's float-bits log2 (gpu/probe exp92: exact for affine
mappings at any rotation and anisotropy). Before, only du/dx and dv/dy from a
2x2 quad's differences counted. Under perspective (exp93) about 14% of
pixels are still a sixteenth of a level off.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With separate specular lighting and color doubling, the GE doubles the
secondary color as well (gpu/probe exp86, exact for modulate, add and
replace).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A perspective u = s * R(q) keeps 24 significant bits, truncated, instead of
a float24's 16 (gpu/probe exp82: large UVs from a texture matrix, as NFS Most
Wanted uses). Through-mode points keep 15 bits of their UVs, as the UV planes
do for triangles (exp83, exp84).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The 8-bit fog of each vertex goes through the same plane as depth and color.
Bit exact on gpu/probe exp21 (all 9 cases, from 94-98%).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
