softgpu: Fix unsigned wrap in BinDirtyRange::Expand and an OOB write in ClearDirty

Expand did `height += ((int)base - (int)newBase) / (stride * bpp)`. The right
operand is unsigned, so when newBase was above base the negative difference
converted to ~4 billion before the division and height wrapped. After that,
HasPendingWrite()'s `start >= base + height * strideBytes` early-out was taken
for every query, so the binner stopped noticing that a draw textures from the
framebuffer it's writing - skipping the flush and leaving maxTasks_ high, which
makes a self-sampling draw depend on which worker thread got there first.

Reachable without exotic state: scissor changes mark BINNER_RANGE dirty without
forcing a flush, so drawing with the scissor top at y=0 and then moving it down
is enough. Handle both directions explicitly instead.

ClearDirty was missing the bounds clamp its twin MarkDirty has. start is masked
to [0, 2047] but bytes isn't bounded - IsVRAMAddress accepts the whole mirrored
8MB window - so a large guest framebuffer near the top of VRAM runs the loop off
the end of vramDirty_[2048] and writes into whatever follows it. Only active with
frameskip enabled, but then it runs every flip.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vd8ntC2brCUtCrDJMqLbs8
This commit is contained in:
Henrik RydgårdandClaude Opus 5 committed 2026-08-29 11:55:23 +02:00
1 parent f901105bd9
commit 74dd222ea3
2 files changed
+18 -2

No files matched your search

+13 -2
View File
@@ -359,8 +359,19 @@ inline void BinDirtyRange::Expand(uint32_t newBase, uint32_t bpp, uint32_t strid
return;
}
if (stride != 0)
height += ((int)base - (int)newBase) / (stride * bpp);
if (stride != 0) {
// Careful with the subtraction: stride * bpp is unsigned, so a signed difference gets converted
// before the division. When newBase was above base that turned into ~4 billion rows, and the
// wrapped height made HasPendingWrite() report "no overlap" for every later query.
const uint32_t newStrideBytes = stride * bpp;
if (newBase < base) {
// The base moves up, so everything we already tracked shifts down by this many rows.
height += (base - newBase) / newStrideBytes;
} else {
// The new range starts further down - make sure the height still reaches its bottom.
height = std::max(height, (newBase - base) / newStrideBytes + h);
}
}
base = std::min(base, newBase);
strideBytes = std::max(strideBytes, stride * bpp);
widthBytes = strideBytes;
+5
View File
@@ -703,6 +703,11 @@ bool SoftGPU::ClearDirty(uint32_t addr, uint32_t bytes, SoftGPUVRAMDirty value)
uint32_t start = ((addr - PSP_GetVidMemBase()) & 0x001FFFFF) >> 10;
uint32_t end = start + ((bytes + 1023) >> 10);
// Same clamp as MarkDirty - the endpoint checks above accept the whole mirrored 8MB window, so
// bytes can carry us past the end of the array. Without this the loop below writes past it.
if (end > sizeof(vramDirty_)) {
end = sizeof(vramDirty_);
}
bool result = false;
for (uint32_t i = start; i < end; ++i) {
if (vramDirty_[i] & (uint8_t)value) {