Linux's uint64_t is unsigned long, which neither of its u32 and u64
(unsigned long long) overloads takes exactly, so the call was ambiguous.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The range check was written as tests for outside, which are all false for
NaN, so a NaN vertex passed and was converted to an integer. That differs
by CPU: arm64 gives 0, which lands off screen, while x86-64 gives INT_MIN,
which wraps, and the triangle's area overflowed into a huge sliver. In the
recording-opengl and recording-vulkan dumps that drew a beam across the
top right that a PSP replay of the dump doesn't show. Now such a vertex is
flagged, and its triangles dropped, on both.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A full state ring flushed the binner, and the next state change filled
it again in the same way. Now the flush also doubles the ring, up to
32768 states, so it happens at most a few times. Nothing draws right
after a flush, so the ring can move then. In the benchmark games, no
more than 512 states were queued between flushes, so 4096 stays the
starting size.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The binner kept a bounding box of every queued item's range, updated for
each item, only to tell whether anything had been queued since the last
flush. A flag says the same. Expand, which grew the box, is now
ItemQueued.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each queued primitive adds its vertex flags (alpha, fog, colors) to its
state, and states are optimized for those flags before their items are
distributed. A state can still gain primitives after some of its items
are in tiles. Re-optimizing it then changed its pixel and sampler
functions and IDs while threads drew with them. With the x86 pixel JIT,
the span function could turn null between checking and calling it, as
in Coded Arms' frame dump. God of War hit this about four times a frame.
Now a state is live once any of its items is in a tile, until the tiles
are drawn and reset, and a live state isn't changed. A primitive that
would change a live state's flags starts a new, unoptimized state from
the same registers instead, optimized for its own primitives only.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The pixel and sampler JITs only exist on x86-64, but elsewhere a cache
miss still flushed the binner before the compile that did nothing, and
the cache never filled. So on ARM every switch to a new pixel or sampler
state was a full flush: hundreds per frame.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Expanding a pending range by a rectangle starting below its start added
a negative row count to an unsigned height, which wrapped to millions of
rows. Overlap tests then overflowed and missed real overlaps. Now a
merged range is whole rows from the lower start to the higher end.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A draw whose texture rectangle overlapped what queued primitives write
flushed, once per draw. NeedsOrder already orders each primitive by the
texels it actually reads, so let it. God of War's bloom passes texture
from the main framebuffer declared as 512x512, which reaches into the
buffer they draw to, and flushed every 30 primitives: 6% faster there.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every tile a triangle touches sets it up again, so larger tiles do less
of that. The tile size is now picked at startup, as large as it can be
with at least sixteen tiles per thread over the screen. With fewer, a
scene whose drawing is concentrated on part of the screen leaves threads
idle: on eight threads, 128x32 tiles (four per thread) took LocoRoco a
third longer than 64x16, and God of War 5% longer. So eight threads keep
64x16, and four or fewer get larger tiles.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Returning to the CPU flushed all queued drawing, also when a list only reached its stall address.
Wipeout Pure builds its lists as it goes and stalls about a hundred times a frame, so the main thread
spent a third of its time finishing the drawing at those flushes. A stalled list isn't done, so now
only a finished one flushes there, which makes Wipeout Pure 15% faster.
What relied on that flush now flushes itself: memory copies, sets, readbacks and uploads that overlap
the queued drawing, and save states, before memory is saved or loaded (FlushPendingDrawing). Syncs and
the display already did. Queued drawing still reads textures from PSP memory when it runs, so a game
that rewrites a texture between stalls without syncing would see the new one; on hardware the GE has
usually drawn with the old one by then.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Waking a worker is a condition variable signal, a system call, and the main thread paid one per idle
worker every 48 queued items. With Tekken 6's tiny draws (about 5 vertices each) the workers keep
running dry, and those signals were about 10% of the main thread, which is what limits it there.
Now no more threads are woken than tiles have work, and only the first is woken from the main thread;
it wakes the rest.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A state that textures from what it draws to, or whose scissor reaches
past the stride, used to switch the binner to one thread, and every
primitive was drawn alone, copying the whole texture for its snapshot.
In Wipeout, God of War and GTA that was a third to half of the time.
Now it's decided per primitive. One is drawn after everything queued
and before what follows only if it needs that: it textures from texels
a queued primitive (or itself) writes, it writes texels a queued one
still has to read, or it reaches past the stride. Reads are bounded by
the vertices' texture coordinates plus the filter's neighbors and
mapped to tiles of the target; queued writes and reads are tracked per
tile. Everything else goes to the tiles as usual. The in-order ones are
split into bands of rows when large, since each pixel reads only the
snapshot, and the snapshot now only copies the previous primitive's
columns.
Same results on 263 frame dumps and 8 games.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each thread used to own a band of the screen, fixed when the work was
first split, and got its own copies of the primitives touching it. Bands
were rarely balanced (edge bands had half the work of middle ones, and
on big.LITTLE CPUs half the threads are slower), and primitives queued
up to the 2048 limit before the threads started.
Now primitives go to 64x16 tiles, as indices into the shared queue, in
batches of 16. A thread takes any tile with work and draws its items in
order; only one thread holds a tile at a time, so per-pixel order is
kept. A flush or a full queue has the main thread draw tiles too instead
of waiting, and when there's only a little work it does that alone
rather than wake the threads. Items are reclaimed once every tile has
drawn them. This also drops the per-thread queues, up to 23 MB.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The GE keeps the specular and spot exponents to 5 significant bits,
truncated, and from 512 up (infinity and NaN included) uses 496. Only
the specular one was truncated, and neither saturated. A huge or NaN
exponent overflowed pspPow's bits: highlights wrong on every backend,
and on D3D11, where the overflowing float to int conversion gives a
NaN, black (Everybody's Golf, #22418). With the exponent clamped, t
stays in range for every base the backends produce.
Measured, now bit exact in the software renderer and within 4 levels
on Vulkan.
Fixes#22418
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The x86 sampler JIT returned zero for a whole sample when a texture level
pointer was null, and for a mipmapped texture it checked the next level
too, even when it isn't blended in. Tactics Ogre draws its font from a
mipmapped texture whose next level has an invalid address, at level 0,
so all its text was missing on x86-64. The generic samplers read a null
level's texels as zero only where they use that level, and still apply
the texture function. Now samplers for such textures aren't compiled,
and the JIT's null checks are gone. Test: gpu/textures/mipinvalid.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
IsLargerThanHalf's non-SIMD loop assigned instead of or-ing, so only blue
counted: a yellow or red light gave no diffuse or specular on riscv64 and
loongarch64 (gpu/commands/light, gpu/exact/lighting and curves failed
there). The NEON version took a channel of 1 for none; it now agrees with
SSE, which counts any channel above zero.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Like z, a primitive whose vertices are all beyond the same x or y plane
(|x| > w) isn't drawn, for every primitive type and with clipping on or off;
a vertex on the plane is inside (gpu/probe exp176, gpu/clipping/xycull).
Immediate-mode vertices carry screen coordinates in their clip position, so
only z culls them.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Transform-mode lines take the triangle's per-vertex s, t, q (15 bits at the
larger exponent) along the line's gradient, like their color and depth, then
u = s * R(q). A pixel just inside a line's end samples short of the end
vertex's coordinate rather than at it (gpu/texmtx/prims: texel row 255, not a
wrap to 0).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
NaN and +inf directions light like (1, 1, 1), -NaN and -inf like (-1, -1, -1)
(gpu/commands/light).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A vertex with z exactly -w isn't clipped, so with clipping on the GE discards
a triangle whose such vertex lands outside the 4096 guard band. The check
skipped it (gpu/clipping/guardband).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A texture that fits in the GE's 8 KB texture cache keeps what was first
read until TEXFLUSH, across primitives and draws, so a later self-textured
draw doesn't see an earlier one's writes (gpu/probe exp174: three passes
over a 16x16 4444 buffer differ in 177 pixels with and without a flush
between them, both now exact). FF Type-0's (NPJH50443) blur; two of its
PSP replays become exact.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
It used (0, 0, 1) instead. The PSP keeps it zero, under every light kind
and type (gpu/probe exp173). SOCOM (UCES01242) lights meshes whose s8
normals are all zero; its PSP replay becomes exact.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The same way as Gouraud color: a gradient along the major axis from the
setup reciprocal, and each pixel's value from its center's signed distance
from v0, so an endpoint's pixel extrapolates past the endpoint's depth
(gpu/probe exp170, exact; it had 435 wrong pixels). Coded Arms' (ULUS10019)
last pixel, where a short line passed a GEQUAL test, and SOCOM's line depth.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Which side an edge is on decides whether a pixel center exactly on it is
inside. The test divided with truncation, so a vertex 0.2 subpixels left of
the opposite edge counted as on it, and a thin triangle's right edge became
inclusive. Peace Walker (ULUS10509), ULJM05302 and mali_missing_circle PSP
replays become exact.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Two triangles forming a rectangle each have their own UV planes. Scaled, a
sample can land right on a texel edge, where the plane's anchor decides the
side: Tokiden's (NPJH50878) 480x33 bar over 280x20 texels took the row
above in its second triangle (gpu/probe exp167). 1:1 samples are half a
texel from any edge.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Pixels past the framebuffer's stride land at the start of the next row,
which another binning task may be drawing, so the result raced. Tokimeki
Memorial (NPJH50127) blurs a 128 wide buffer under a full screen scissor;
its PSP replay is now exact (gpu/probe exp166).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With no steps along the line, both endpoint weights were zero, so fog (and
UVs) interpolated to zero: full fog. SOCOM's (UCUS98649) radar lines, a few
pixels long in an orthographic view, came out in the fog color.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
It used (0, 0, 1) instead, lighting everything facing the camera. The PSP
leaves the direction zero: no diffuse, and specular from the eye vector
alone (gpu/probe exp164). ULJM05302 has such a light on its cars, and the
Yu-Gi-Oh (ULJM05940) replay becomes exact.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The GE sets up UV planes per sprite, and their gradient is only exact for a
power of two area, so a 49 wide 1:1 sprite samples just below the texel
centers while the 128 wide sprite it was joined into hit them (gpu/probe
exp163, the ULJM05302 track map). Nearest sampling picks the same texels
either way.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The GE samples through its texture cache, so a primitive that textures from
its own render target mostly reads the old contents (geprobe exp81, exp159,
exp160). Give each such primitive a snapshot, refreshing only the rows the
previous one drew.
3rd Birthday (ULUS10567) bloom becomes exact on PSP replays, Burnout
Dominator's glare 93% -> 98% and Ridge Racer's blur 40% -> 94% of pixels.
The remaining differences are rows where the cache's lazily loaded 8-row
blocks already see the primitive's own writes.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The GE never forms a world position for lighting: the vector to a point
or spot light is the light position minus the world translation, minus
the model position times the world matrix, summed like a matrix row.
gpu/probe exp153-158 (point lights 0.02-20 units away, spot exponents up
to 40, game-like world matrices; exp153 is a Syphon Filter vertex) are
exact, and Syphon Filter (#13568) becomes exact.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
An endpoint exactly on the left or right corner of a pixel's diamond (top
or bottom for y-major lines) doesn't count as inside it: the pixel is lit
only if the line crosses the diamond (gpu/probe exp149). Makes exp72,
Virtua Tennis and echochrome exact.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The minor offset walks from v0 with a slope quantized like a triangle
plane's gradient (setup reciprocal, 14 fraction bits per subpixel,
floored), rather than one product per pixel. Makes gpu/probe exp112
exact, and Virtua Tennis and echochrome lose their AA line misses.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Next to an edge between two z = 0 vertices, the floored depth plane can
come out -1. The GE stores and range-tests 0 there, in through and
transform mode, with depth clamp on or off (gpu/probe exp148). Fixes a
seam across the sky in Frontier Gate Boost.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
De Boor at t = k/256 with the Bezier path's lerps. Each blending factor
(t - knot) / span is quantized in 1/256 from the nearer of its two knots,
and a segment's last point is evaluated as the next one's first. Colors,
texture coordinates, generated UVs and normals work as for Bezier
patches. Bit exact on gpu/probe exp144-146; software renderer only.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Both were fit as exceptions with a K off the formula, but the probes that
pinned them are as happy with the formula's K and a slope of -128, which
the large-w UV planes need (gpu/probe exp136, exp138 now exact).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The forced alpha test for src alpha / inv src alpha blending assumed a
zero alpha pixel leaves the target unchanged, but with dithering on the
GE still writes dst + dither (gpu/probe exp141). Makes the Test Drive
(ULUS10249) frame dump exact.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Columns before rows, de Casteljau at t = k/256 (rounded toward the patch
middle). Positions and UVs lerp in 16-bit fixed point at the larger
operand's exponent; colors in 15-bit fixed point. Generated UVs are k/256.
Bit exact on gpu/probe exp48-51 and exp139-140. Software renderer only,
since it's about 15x slower than the float weights.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The near-plane clipper's new vertices get their colors and fog (both 8-bit
per vertex) by lerping with t rounded to the nearest 1/256, not truncated
(gpu/probe exp136: every clipped color and fog pixel exact; the colors were
recovered exactly from the PSP's plane values per clipped triangle).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The minor-axis offset an antialiased line's alpha comes from is computed
with the triangle setup's reciprocal of the line's major length, walking
from v0, and floored; diagonal lines are y-major. exp112 went from 101
differing pixels to 23; the misses left sit at reciprocal segment starts.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A line's color gradient along its major axis comes from the triangle
setup's reciprocal, with 14 fraction bits per subpixel, floored, and each
pixel is the start color plus the gradient times its center's signed
distance from v0 along the line. A diagonal line (|dx| == |dy|) is
y-major. gpu/probe exp137, exact; fixes the line strips in a Final Fantasy
Zero frame dump.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The near-plane clipper's new vertices get their texture coordinates with
the same float24 arithmetic as their position, not a float lerp. Fixes the
texel ties under a clipped fan in a GTA frame dump (gpu/probe exp136; clipped
colors and fog still differ).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
When a triangle's long edge (top to bottom vertex) has 3 dy > 2^17 in
screen subpixels, the GE lights the first pixel of each 4-pixel span along
it (the last, for a right edge) when any of the span is inside, likely an
overflowing 3 dy span offset. Bottom ties end the long edge at the left
vertex. Such triangles take a scalar mask path. gpu/probe exp130-135, exact;
none of 216 frame dumps has such a triangle.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
With vertices far apart in the 4096 pixel screen space the cross product
overflowed 32 bits, flipping its sign, so both windings of a large triangle
were dropped (gpu/probe exp132: the PSP draws it).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
It ANDed the destination into a temporary instead of the color, so NAND
came out as NOT color. Found by gpu/probe exp125 on an x86-64 build (the
C++ path was already right).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The GE builds a sprite's UV plane from three corners whose s, t and q it
takes component by component from the two vertices: s from the vertex that
supplies the corner's x, t from the one that supplies its y (swapped when the
sprite is rotated), and q from the one that supplies its x. With different w
at the two vertices that bends the mapping; we used v1's w at every corner.
gpu/probe exp129: all four vertex orders at w = 1, 2.7 and differing w, exact.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A sprite drawn from bottom left to top right is rotated: its first row
rounds like a column ((y + 6) >> 4), and its UV plane comes from v0, v1 and
the bottom right corner, anchoring it at the bottom left. The other three
vertex orders were already exact (gpu/probe exp121-122, through and
transform mode). Fixes the last differences in an MGS frame dump (HUD
sprites).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
