The non-DXT decode paths read w texels per row from a source whose stride, range
check and unswizzle buffer were all sized from bufw alone. w and bufw are
independent GE registers, so w > bufw is reachable, and the last row then runs
off the end of both the validated guest range and the temp buffer.
The previous commit clamped w down to bufw, which stops the overrun but is the
wrong shape twice over: the game asked for w texels and the destination is sized
for w, so the tail of every row is left holding whatever was in the buffer, and
it silently narrows a texture the hardware would have decoded in full.
Size the checks from both instead. The extent a decode touches is bufw per row
plus however far the last row reaches past its own stride - the same adjustment
TextureReplacer::ComputeHash and the GE recorder already make - so the range
check, the height it falls back to when the range is short, and the unswizzle
buffer are all computed that way now.
Pull the five copies of "resize tmpTexBuf32_, unswizzle into it" into a helper
while we're here, since they all need the same sizing. It zeroes the buffer when
w reaches past bufw, as UnswizzleFromMem only fills the stride and the tail would
otherwise be stale heap.
ComputeTextureHash has the same bufw-only assumption, but it's left alone
deliberately - changing what goes into a texture hash isn't worth the risk here.
DXT is left alone too - it clamps to minw, but there the limit is the block index
within a bufw/4 block row, so its range check already covers what it reads.
Vuln 17: ReplacedTexture::LoadLevelData let a KTX2/DDS file at a higher
mip level resize the shared data_ vector to its own (attacker-controlled)
mip count, so data_[mipLevel + i] indexed out of bounds and the KTX2
branch resized a different element than it wrote to. Disallow mixing
image formats across mip levels, cap the container mip count, and resize
the same element that is used as the transcode destination.
Vuln 18: DecodeTextureLevel only validated the start address for
non-DXT textures, so guest-controlled w/h/bufw could drive reads past
mapped RAM. Validate the needed range like the DXT path does and clamp
the height; ReadIndexedTex now takes the clamped w/h.
Henrik Rydgård