Fixes cases where EBOOTs have more than 32 program headers. That limit came from
segmentVAddr being a 32-entry array indexed by program header number
Changes the table to a vector, and holds SEGMENT_NOT_LOADED for
headers that aren't PT_LOAD, so a relocation naming one is rejected and logged
rather than quietly relocating against zero. GetSegmentVaddr() still answers 0
for those, as it did when the table was a zero-initialized array.
Nothing here is known to misbehave on a real file - it's the input validation
around the fixes in the preceding commits.
ElfReader's constructor read e_phoff and e_shoff out of the header to find the
segment and section tables, before anything had established there was a header
there; LoadInto's size check only runs later. It leaves header null in that case
now, and the accessors that use it cope.
GetSegmentPtr didn't range-check the segment index at all, and both it and
GetSectionDataPtr accepted an offset exactly at the end of the file, which
addresses no bytes. GetSectionAddr and GetSectionSize took an index on trust.
LoadRelocations2 got most of this commit. Its segment end came from p_filesz
without checking the segment fits in the file, so the whole decode could run off
the end of the buffer. Within it, the flag and type tables are indexed by
bitfields out of each command word and were never checked against the table
sizes (which themselves come from the file); the loop only guaranteed one byte
was left before reading a two-byte command, and the branches that consume a
further two or four bytes checked nothing at all; and the offset segment number
- unlike the address segment number a few lines up - was used to index
segmentVAddr unchecked, though it's wide enough to exceed it. The command read
is byte-wise now too: how far buf has advanced depends on those file-supplied
table sizes, so it isn't necessarily even.
LoadSymbols only checked that a symbol name started inside the file, not that it
was terminated there.
Also dropped the atomic counter and the ParallelLoop.h include left over from
when LoadRelocations ran in parallel.
pspautotests 314/314 with --graphics=software.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
addrToHiLo's verification computed (hi<<16) + lo with hi a u16, which promotes
to int - so for any kernel module, loading at 0x88000000, the shift overflowed a
signed int. Undefined behaviour in the one place whose whole job is to check
that a relocation came out right.
A HI16 that found no matching LO16 logged an error and then wrote its zero-
initialized hi into the instruction anyway, blanking the immediate of a lui it
had just admitted it couldn't resolve. It leaves the instruction alone now: we
don't know the right value, and a zeroed lui produces a wrong address far from
here rather than a failure anyone can trace back.
And the candidate LO16's address was computed with the HI16's segment base
rather than its own, which is exactly the mismatch the warning a few lines below
exists to report - so when that warning fired, the IsValidAddress check guarding
the pairing had been applied to an address from the wrong segment.
pspautotests 314/314 with --graphics=software.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Both relocation branches took the section to modify straight from the file's
sh_info and only checked it wasn't negative, so a value like 1000 in a
ten-section file read past the end of the section table and decided what to
relocate based on whatever was there. sh_info is a u32, so ">= 0" only rejected
the half of the range above INT_MAX.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
st_shndx is a u16 that can hold a reserved value instead of a section number,
and SHN_ABS (0xFFF1) is common in real symbol tables - linker-script constants
like _gp end up there. LoadSymbols fed it straight to sectionAddrs, which has
GetNumSections() entries, so those symbols read a quarter of a megabyte past the
allocation and added whatever they found to the symbol's address. Unlike the
other bounds problems around here this one doesn't need a malformed file; any
ordinary ELF with an absolute symbol hits it.
Symbols that are undefined, absolute or common now get skipped instead - there's
nothing of ours to relocate them against - and a section number that's in range
for neither is skipped with a warning.
Also bail out if LoadInto hasn't run, since that's what fills in sectionAddrs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
segmentVAddr is a 32-entry array, but LoadInto filled it from e_phnum, which is
a u16 - so an ELF declaring 65535 program headers wrote 65535 u32s into it,
straight through the rest of the ElfReader object. The only check standing in
front of that verified the program headers fit in the file, which a ~2MB crafted
PRX satisfies. Rejected up front now: LoadRelocations already ignores segment
numbers at or past the array size, so a module with more than that couldn't be
relocated correctly anyway. Real modules have a handful - PSP_Header::nsegments
is a u8 and no more than 4 are ever used.
Second one from the same loop: with no PT_LOAD segment at all, totalStart stayed
0xFFFFFFFF and totalEnd 0, so totalSize came out as 1 (0 - 0xFFFFFFFF) and the
loader went on to allocate a 1-byte block at 0xFFFFFFFF.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Two things kept the companion ELF from doing its job.
The first is the one that mattered: fileToStart is the game's own *directory*
for folder-launched homebrew (IdentifiedFileType::PSP_PBP_DIRECTORY), which is
the normal case when you pick a homebrew in the UI. The search navigated up from
it regardless, landing in PSP/GAME and listing sibling games - all directories,
all skipped - so app.elf sitting right next to the EBOOT was never found. It only
ever worked when the path pointed at the EBOOT itself, which is how headless is
invoked, which is why it looked fine from there. Searches the directory itself
now when that's what it's given.
The second: line info didn't survive loading a savestate. Modules aren't just
re-registered there, they're destroyed and rebuilt (KernelObjectPool::Clear), so
removing a module's lines in ~PSPModule threw the table away on every state
load. The previous commit worked around it by re-reading the companion, which
was both wasteful and no help at all to an ELF launched directly - those bytes
are long gone by then.
SymbolMap already solves this and line info now does it the same way: keep what
you have, and let whatever next claims the address range replace it. AddModule
replaces by key and is called for every module load, including ones with no line
info of their own, so a range gets retired when it's genuinely reused. The whole
table goes when the game does, in PSP_Shutdown. That also means the savestate
path has nothing to re-read, so state loads no longer pay to re-parse a
multi-megabyte ELF.
The tradeoff is a window between a module unloading and its range being reclaimed
where a lookup can still answer for it. For a debugger that's a stale file:line
on an address nothing owns, against certain and total loss on every state load.
Verified with --auto-save-load-symbols off, launched both ways: by directory
(the case that was broken) and by EBOOT path, both give 3734 symbols and 98383
line rows.
pspautotests 314/314, UnitTest 55/55.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
bAutoSaveLoadSymbols is about writing .ppsym files back out and reading them in
again. It had also come to gate reading debug info that's simply sitting next to
the game, which is a different thing and shouldn't need asking for: the main
ELF's own symbols were already loaded unconditionally, but the companion ELF's
symbols and all line info were not.
Now the ELF is always the baseline - main or companion, symbols and line info -
and the setting only adds the .ppsym half on top of it.
Line info also loads from the module being loaded, not just from a companion,
so an ELF launched directly brings its own. A PRX has no .debug section for it
to find (prxgen strips them), so that's a cheap no-op for the usual EBOOT case,
which the companion path still covers.
That second source needs the two shapes distinguished, so AddModule takes an
explicit address delta rather than assuming a base: a companion links at zero
and wants the module's base added, while an ELF loaded at the addresses it asked
for already has final ones (bRelocate is just e_type != ET_EXEC). Rows that
don't land inside the module after that are dropped either way, which is a
better check than the old "offset smaller than the module" one.
Splitting the companion's identity check out of the symbol loader lets line info
reuse it, and drops an accidental requirement along the way: it used to reject
any companion without a symbol table, so an ELF built with -g but stripped of
its symbols would have contributed no line numbers either.
Verified with --auto-save-load-symbols off: CrossCraft's companion app.elf loads
3734 symbols and 98383 line rows where it previously loaded neither.
The direct-ELF path is not verified at runtime - it needs a bootable ELF that
carries DWARF, and there isn't one to hand. Both candidates here (pspautotests'
.elf builds and CrossCraft's own app.elf) are linked at address 0 and fail to
boot on that alone, which is pre-existing loader behaviour and nothing to do
with this.
pspautotests 314/314, UnitTest 55/55.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Homebrew commonly ships its unstripped ELF next to the EBOOT, which is already
how the symbol loader turns z_un_08841f98 into a function name. That same ELF
carries a DWARF .debug_line section, so the addresses can be mapped to source
files and lines too - and a backtrace stops being four hex numbers:
08841f98 move sp,fp mesh.zig:163
0883afa4 li v0,0x0 MenuState.zig:821
088260d8 andi at,v0,0xFFFF State.zig:40
0882a27c andi at,v0,0xFFFF engine.zig:468
Surfaced in three places: per frame in hle.backtrace, in the "hit" object that
cpu.breakpoint.hit and cpu.stepping share, and appended to the disassembly
window's status bar. The breakpoint case keys on the pc rather than the address,
since for a memory breakpoint the useful source location is the instruction that
did the access, not the data it touched.
Storage is a plain sorted table of absolute addresses per module. SymbolMap
keeps module-relative addresses because its .ppsym files are meant to be
reloaded by a different game that pulls in the same module; none of this is ever
written anywhere - it's regenerated from the ELF each boot - so there'd be
nothing for relative addresses to buy. Each module owns its own rows and file
names outright and is keyed the way SymbolMap::UnloadModule is, so unloading one
module drops its lines and nobody else's.
The subtle part is end-of-sequence markers. Without them a lookup for an address
in a gap - a compilation unit built without debug info - confidently reports the
last line of an unrelated file. A prototype run over one test binary
mis-attributed 70 of its 349 functions that way, so sequence ends are recorded
as rows with line 0 and a lookup landing on one reports nothing instead.
DWARF 2 through 4 are decoded (psp-gcc emits 2, Zig 4). Version 5 re-encoded the
file table, so those units are skipped with a warning rather than mis-parsed -
nothing targeting the PSP produces it today.
Scope, since it's narrower than it sounds: PRX conversion strips every .debug
section. I checked all 437 pspautotests .prx and CrossCraft's own app.prx -
none have any. Of 24 installed homebrew EBOOTs, zero carry debug info; CrossCraft
only does because it ships app.elf separately. So this helps someone developing
homebrew, and does nothing at all for a commercial game.
Costs about 1.2 MB for a large Zig binary (98383 rows, 438 files) and nothing
for anything without debug info. Follows bAutoSaveLoadSymbols like the symbols
do.
pspautotests 314/314, UnitTest 55/55, CoreUWP builds.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Homebrew almost always ships the ELF it was built from alongside the EBOOT -
app.elf next to app.prx - but prxgen strips the symbol table on the way to the
PRX, so the module PPSSPP loads has no names at all and MIPSAnalyst calls every
function it finds z_un_<address>. Working out what any of them are meant hand-
parsing that ELF with a throwaway script, which is how the CrossCraft
relocation bug got identified.
So read it directly. On module load, scan the game's own directory for an ELF
with a symbol table and add its STT_FUNC/STT_OBJECT entries at the module's
base. CrossCraft picks up 3734 symbols, and the disassembly turns from
z_un_088c00f0 into world.init_empty, with static_allocator.alloc at the vtable
entry it calls - the two functions that took the longest to identify by hand.
Matching is the part worth getting right, since a wrong match puts confident
nonsense at real addresses, which beats having no names only in the sense that
it's worse. A candidate has to be a 32-bit ELF with a symbol table whose
highest section ends within a page of the loaded module's size - the companion
links at base 0 and covers the same image, so that's a tight check, and
unrelated ELFs sitting in the same folder fail it. Symbols outside the module
are skipped individually too.
Names go in with updateName, so they win over the analyzer's placeholders
rather than losing to whichever got there first. Gated on the existing
bAutoSaveLoadSymbols setting (off by default), which already means "keep symbol
names around for me" and avoids a directory scan per module load otherwise.
pspautotests 314/314.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
ElfReader read the first loadable segment's p_align into firstSegAlign but only
ever used it to round down textStart for symbol bookkeeping - the allocation
itself used the block allocator's default grain. A module whose relocations are
only valid at a more strictly aligned base therefore got loaded somewhere it
couldn't work, and the symptom is addresses off by a multiple of 64KB rather
than an outright failure.
CrossCraft Classic (Zig) declares p_align 0x10000 and hits exactly that. It
declares it deliberately: a stage of Zig's PSP pipeline emits mispaired
HI16/LO16 relocations, and a 64KB-aligned base makes that harmless - with no
low bits in the base no carry is ever needed, so which of a symbol's LO16
entries a HI16 was paired with stops affecting the result. PPSSPP put it at
0x08804000 instead, where the carry does matter: 46 of its addresses came out
64KB low, and it jumped through a bogus vtable a few seconds in. It now loads
at 0x08810000 and all 8405 lui/addiu pairs resolve correctly.
Worth being clear that the relocation code was never wrong here - it matches
what the hardware does, pairing a run of HI16 with the next non-HI16 entry and
applying the carry. Only the load address differed.
The two AllocAt paths can't move the module, since the caller picked the
address, so they just report a misaligned one instead.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
* Rename LogType to Log
* Explicitly use the Log:: enum when logging. Allows for autocomplete when editing.
* Mac/ARM64 buildfix
* Do the same with the hle result log macros
* Rename the log names to mixed case while at it.
* iOS buildfix
* Qt buildfix attempt, ARM32 buildfix
Motorstorm: Arctic Edge US uses these pairs for some VFPU loads.
Without relocating these, strange shadows show underneath vehicles.
It appears as if actual firmware pairs with any non-HI16 relocation.
Henrik Rydgård