rc_bittree()/rc_number() can index bm_dist_bits up to column 50 and
bm_dist up to row 43 (when decoding a "long distance" match, i.e.
match_len > 2), but the arrays were only sized for 39 and 18
respectively. A crafted LZRC-compressed block (reachable via
NPDRMDemoBlockDevice::ReadBlock) could drive these indices out of
range, corrupting adjacent probability tables within the same
LZRC_DECODE struct via rc_bit()'s read-modify-write. Size the arrays
for the indices the algorithm can actually produce, and initialize
them via sizeof() so the memset in rc_init stays correct.
The LZRC decompressor's only bounds check for output (and input) was a
debug-only _dbg_assert_msg_, which is a no-op in release builds. The
NPDRM demo block device also passed a hardcoded 1 MiB output length
while the real destination buffer (blockBuf_) could be as small as 2048
bytes, allowing a crafted NPDRM image to trigger an unbounded heap
overflow during game load.
Changes:
- rc_putbyte/rc_getbyte now enforce real bounds and set an error flag
instead of relying on debug asserts; decompression aborts with -1 on
overflow or truncated input.
- normalize() reads via rc_getbyte so it stays in bounds.
- Plain-text path clamps the copy size to both the output buffer and the
remaining input (and no longer interprets the size as signed).
- NPDRMDemoBlockDevice::ReadBlock passes blockSize_ (the real buffer
size) instead of 0x00100000 to lzrc_decompress.
- Add unittest/TestLzrc (synthetic input, no test data files): checks the
plain-text clamp, truncated input, and output overflow all fail safely.
- AGENTS.md: note to reuse existing format handlers/decompressors before
writing new ones.
Henrik Rydgård