Two things kept the companion ELF from doing its job.
The first is the one that mattered: fileToStart is the game's own *directory*
for folder-launched homebrew (IdentifiedFileType::PSP_PBP_DIRECTORY), which is
the normal case when you pick a homebrew in the UI. The search navigated up from
it regardless, landing in PSP/GAME and listing sibling games - all directories,
all skipped - so app.elf sitting right next to the EBOOT was never found. It only
ever worked when the path pointed at the EBOOT itself, which is how headless is
invoked, which is why it looked fine from there. Searches the directory itself
now when that's what it's given.
The second: line info didn't survive loading a savestate. Modules aren't just
re-registered there, they're destroyed and rebuilt (KernelObjectPool::Clear), so
removing a module's lines in ~PSPModule threw the table away on every state
load. The previous commit worked around it by re-reading the companion, which
was both wasteful and no help at all to an ELF launched directly - those bytes
are long gone by then.
SymbolMap already solves this and line info now does it the same way: keep what
you have, and let whatever next claims the address range replace it. AddModule
replaces by key and is called for every module load, including ones with no line
info of their own, so a range gets retired when it's genuinely reused. The whole
table goes when the game does, in PSP_Shutdown. That also means the savestate
path has nothing to re-read, so state loads no longer pay to re-parse a
multi-megabyte ELF.
The tradeoff is a window between a module unloading and its range being reclaimed
where a lookup can still answer for it. For a debugger that's a stale file:line
on an address nothing owns, against certain and total loss on every state load.
Verified with --auto-save-load-symbols off, launched both ways: by directory
(the case that was broken) and by EBOOT path, both give 3734 symbols and 98383
line rows.
pspautotests 314/314, UnitTest 55/55.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
bAutoSaveLoadSymbols is about writing .ppsym files back out and reading them in
again. It had also come to gate reading debug info that's simply sitting next to
the game, which is a different thing and shouldn't need asking for: the main
ELF's own symbols were already loaded unconditionally, but the companion ELF's
symbols and all line info were not.
Now the ELF is always the baseline - main or companion, symbols and line info -
and the setting only adds the .ppsym half on top of it.
Line info also loads from the module being loaded, not just from a companion,
so an ELF launched directly brings its own. A PRX has no .debug section for it
to find (prxgen strips them), so that's a cheap no-op for the usual EBOOT case,
which the companion path still covers.
That second source needs the two shapes distinguished, so AddModule takes an
explicit address delta rather than assuming a base: a companion links at zero
and wants the module's base added, while an ELF loaded at the addresses it asked
for already has final ones (bRelocate is just e_type != ET_EXEC). Rows that
don't land inside the module after that are dropped either way, which is a
better check than the old "offset smaller than the module" one.
Splitting the companion's identity check out of the symbol loader lets line info
reuse it, and drops an accidental requirement along the way: it used to reject
any companion without a symbol table, so an ELF built with -g but stripped of
its symbols would have contributed no line numbers either.
Verified with --auto-save-load-symbols off: CrossCraft's companion app.elf loads
3734 symbols and 98383 line rows where it previously loaded neither.
The direct-ELF path is not verified at runtime - it needs a bootable ELF that
carries DWARF, and there isn't one to hand. Both candidates here (pspautotests'
.elf builds and CrossCraft's own app.elf) are linked at address 0 and fail to
boot on that alone, which is pre-existing loader behaviour and nothing to do
with this.
pspautotests 314/314, UnitTest 55/55.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Homebrew commonly ships its unstripped ELF next to the EBOOT, which is already
how the symbol loader turns z_un_08841f98 into a function name. That same ELF
carries a DWARF .debug_line section, so the addresses can be mapped to source
files and lines too - and a backtrace stops being four hex numbers:
08841f98 move sp,fp mesh.zig:163
0883afa4 li v0,0x0 MenuState.zig:821
088260d8 andi at,v0,0xFFFF State.zig:40
0882a27c andi at,v0,0xFFFF engine.zig:468
Surfaced in three places: per frame in hle.backtrace, in the "hit" object that
cpu.breakpoint.hit and cpu.stepping share, and appended to the disassembly
window's status bar. The breakpoint case keys on the pc rather than the address,
since for a memory breakpoint the useful source location is the instruction that
did the access, not the data it touched.
Storage is a plain sorted table of absolute addresses per module. SymbolMap
keeps module-relative addresses because its .ppsym files are meant to be
reloaded by a different game that pulls in the same module; none of this is ever
written anywhere - it's regenerated from the ELF each boot - so there'd be
nothing for relative addresses to buy. Each module owns its own rows and file
names outright and is keyed the way SymbolMap::UnloadModule is, so unloading one
module drops its lines and nobody else's.
The subtle part is end-of-sequence markers. Without them a lookup for an address
in a gap - a compilation unit built without debug info - confidently reports the
last line of an unrelated file. A prototype run over one test binary
mis-attributed 70 of its 349 functions that way, so sequence ends are recorded
as rows with line 0 and a lookup landing on one reports nothing instead.
DWARF 2 through 4 are decoded (psp-gcc emits 2, Zig 4). Version 5 re-encoded the
file table, so those units are skipped with a warning rather than mis-parsed -
nothing targeting the PSP produces it today.
Scope, since it's narrower than it sounds: PRX conversion strips every .debug
section. I checked all 437 pspautotests .prx and CrossCraft's own app.prx -
none have any. Of 24 installed homebrew EBOOTs, zero carry debug info; CrossCraft
only does because it ships app.elf separately. So this helps someone developing
homebrew, and does nothing at all for a commercial game.
Costs about 1.2 MB for a large Zig binary (98383 rows, 438 files) and nothing
for anything without debug info. Follows bAutoSaveLoadSymbols like the symbols
do.
pspautotests 314/314, UnitTest 55/55, CoreUWP builds.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Homebrew almost always ships the ELF it was built from alongside the EBOOT -
app.elf next to app.prx - but prxgen strips the symbol table on the way to the
PRX, so the module PPSSPP loads has no names at all and MIPSAnalyst calls every
function it finds z_un_<address>. Working out what any of them are meant hand-
parsing that ELF with a throwaway script, which is how the CrossCraft
relocation bug got identified.
So read it directly. On module load, scan the game's own directory for an ELF
with a symbol table and add its STT_FUNC/STT_OBJECT entries at the module's
base. CrossCraft picks up 3734 symbols, and the disassembly turns from
z_un_088c00f0 into world.init_empty, with static_allocator.alloc at the vtable
entry it calls - the two functions that took the longest to identify by hand.
Matching is the part worth getting right, since a wrong match puts confident
nonsense at real addresses, which beats having no names only in the sense that
it's worse. A candidate has to be a 32-bit ELF with a symbol table whose
highest section ends within a page of the loaded module's size - the companion
links at base 0 and covers the same image, so that's a tight check, and
unrelated ELFs sitting in the same folder fail it. Symbols outside the module
are skipped individually too.
Names go in with updateName, so they win over the analyzer's placeholders
rather than losing to whichever got there first. Gated on the existing
bAutoSaveLoadSymbols setting (off by default), which already means "keep symbol
names around for me" and avoids a directory scan per module load otherwise.
pspautotests 314/314.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
ElfReader read the first loadable segment's p_align into firstSegAlign but only
ever used it to round down textStart for symbol bookkeeping - the allocation
itself used the block allocator's default grain. A module whose relocations are
only valid at a more strictly aligned base therefore got loaded somewhere it
couldn't work, and the symptom is addresses off by a multiple of 64KB rather
than an outright failure.
CrossCraft Classic (Zig) declares p_align 0x10000 and hits exactly that. It
declares it deliberately: a stage of Zig's PSP pipeline emits mispaired
HI16/LO16 relocations, and a 64KB-aligned base makes that harmless - with no
low bits in the base no carry is ever needed, so which of a symbol's LO16
entries a HI16 was paired with stops affecting the result. PPSSPP put it at
0x08804000 instead, where the carry does matter: 46 of its addresses came out
64KB low, and it jumped through a bogus vtable a few seconds in. It now loads
at 0x08810000 and all 8405 lui/addiu pairs resolve correctly.
Worth being clear that the relocation code was never wrong here - it matches
what the hardware does, pairing a run of HI16 with the next non-HI16 entry and
applying the carry. Only the load address differed.
The two AllocAt paths can't move the module, since the caller picked the
address, so they just report a misaligned one instead.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
ReadSFO dereferenced index table entries without checking the table fit
within the buffer, and GetDataOffset had no bounds checks at all (reading
attacker-controlled offsets and strcmp'ing without a terminator guard).
- Validate the index table fits entirely within the buffer in ReadSFO.
- Add a size parameter to GetDataOffset and validate the index table,
key/data table positions, and key string termination before use.
* Rename LogType to Log
* Explicitly use the Log:: enum when logging. Allows for autocomplete when editing.
* Mac/ARM64 buildfix
* Do the same with the hle result log macros
* Rename the log names to mixed case while at it.
* iOS buildfix
* Qt buildfix attempt, ARM32 buildfix
BuildHash pads the buffer up to its 16-byte aligned size with zeroes,
so there needs to be space for that. Or, we should just remove that
write, but let's do the smallest change that fixes the bug for now.
