2.60 and 2.71 tag their flash0:/vsh/etc/index.dat 0x495BE403 rather than with
the 0x0B2Bxxx0 the rest of this family uses; 2.80, 2.81 and 2.82 use 0x0B2B05F0.
All five now reach an interactive XMB.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Those generations predate the per-model split and have a single
flash0:/vsh/etc/index.dat. 3.30 through 3.52 now reach an interactive XMB;
3.30 to 3.51 share 3.52's key.
3.03's mesg_led.prx is older than keys330_1, so its table was confirmed against
keys300_1 and keys280_1 instead - both match byte for byte.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
flash0:/vsh/etc/index_XXg.dat is the index of what the XMB shows, and each
firmware generation tags it with a key per PSP model. We only had the 6.6x
triple, so 5.03 through 6.39 decrypted nothing, drew no icons, and gave up with
the red error screen.
The keys come out of each firmware's own mesg_led_XXg.prx, whose tag table is
24-byte entries of tag plus 16-byte key. Extracting the 6.6x triple that way
reproduces the three keys already in this file byte for byte, which is what
establishes the layout.
5.03, 5.50, 5.55, 6.00, 6.20, 6.31 and 6.39 now reach an interactive XMB.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
flash0:/vsh/etc/index_XXg.dat is the index of what the XMB shows, and 5.50 tags
it 0x0B2B11F0, which we had no key for - so the shell decrypted nothing, drew
no icons, and gave up with the red error screen.
The key is read out of that firmware's own mesg_led_02g.prx, whose tag table is
24-byte entries of tag plus 16-byte key. The two neighbouring entries hold
keys330_1 and keys505_a byte for byte, which is how the layout was confirmed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The blocker below 6.60 wasn't offsets, it was that Sony renumbered the kernel
*_driver NIDs between versions. A function we HLE under its 6.6x NID is a
stranger on an older build, so the import lands in the real firmware module
instead - and that's where it goes wrong:
- sceRtc_driver sceRtcSetAlarmTick. Without the HLE the VSH's alarm call ran
the real rtc.prx, which called on into syscon.prx and blocked forever on a
SceSysconSync semaphore. That was the whole "stalls with every thread parked"
symptom; the tell was a fourth SceSysconSync waiter a healthy boot lacks.
- sceHprm_driver sceHprmReadLatch, called once a frame - so before this an
older firmware's 12-second boot logged ~20000 lines of one unresolved import.
Three extra NIDs each, found by disassembling the module from both firmwares
and matching on the address of the user-mode export whose NID never changed
(sceRtc/0x7D1FBED3, sceHprm/0x40D2F9F0).
5.55 additionally needed two PRX decryption keys we didn't have (0x4C941AF0
and 0x4C941BF0) - without them none of flash0:/kd decrypted and the shell came
up with no drivers behind it at all.
Checked one release at a time against every version that ships on a disc, plus
6.61. 4.05 and below still die on a null write inside vsh_module.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
KIRK CMD1 writes header + data_offset + align16(data_size) bytes into outbuf,
and all three come out of the header the decrypter just decrypted, not from the
caller. The SHA1 check doesn't bound them - it only covers the header, so it
passes just as happily for a block that's been cut short.
The PSAR walker has to guess how long an updater's second block is (nothing
records it, so it tries the sizes real updaters use), and a wrong guess sent
KIRK off the end of the buffer: unpacking a firmware crashed roughly half the
time, on every version and disc I tried, depending on the heap layout.
Bound the write against the size the caller gave us, in all six decrypt types.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
--re-decrypt runs pspDecryptPRX() over a file and unpacks the KL4E/KL3E
stream behind it. This opens up flash0:/kd/resource/*.img, the images the
Media Engine actually runs: they are ordinary tagged containers (tag
862648D1, which PrxDecrypter already has a key for) with the ~PSP
signature blanked, so the normal module loader never touches them.
--re-raw-base analyzes --re-module as a flat code image at a given
address rather than as a PRX. The decrypted ME images are raw MIPS with
no ELF around them; the address they were linked for is recoverable from
their own jal targets (0x08300000 for meimg.img).
Also makes PrxDecrypter.h self-contained - PSP_Header is built from _le
types, so it needs Common/Swap.h rather than relying on the includer.
Not every disc image is a whole number of 2048-byte sectors - tools that build
pre-patched ISOs write images that stop partway through their last one, with a
file legitimately ending there. Two things then conspired to lose that tail.
FileBlockDevice::GetNumBlocks() rounds down, so the partial sector isn't
counted, and the file size clamp in ISOFileSystem measured what the image holds
in whole blocks. A file running to the last byte of such an image got clamped
short - by up to a sector - before anything read it.
FileBlockDevice::ReadBlock() then returned false for a short read of that
sector, and ISOFileSystem::ReadFile substitutes an all-zero sector when a read
fails, so even the bytes that were there came back as zeroes.
Measure the clamp in bytes via GetUncompressedSize() instead of blocks, and
treat a short read at the end of the image as a success with the rest of the
sector zeroed. GetUncompressedSize() defaults to the block-based value and is
only overridden by FileBlockDevice, so nothing else changes behaviour.
Also report why a module was rejected. "Failed to load module" named the file
and nothing else, and the truncation check logged only the byte count, which
points at the executable when the real cause is that the loader was handed
fewer bytes than the file has. ElfReader now keeps the reason for a failed
LoadInto, __KernelLoadELFFromPtr puts it in the error string that reaches the
user, and both messages say which header table overran and by how much.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
kirk_engine.h and amctrl.h guard their declarations, but AES.h and SHA1.h
never did, and kirk_engine.h includes them from outside its own guard. So the
AES_* and SHA1* functions got C++ linkage in any C++ file that reached them
through there, and only linked for callers that happened to wrap the whole
header in an extern "C" of their own. Nothing had called AES_* from C++
before, so it stayed hidden until something did.
Guarding the two headers instead lets every caller include them plainly, and
the wrappers scattered around the tree come out. Both are pure declarations
over kirk_common.h's typedefs with no system headers behind them, so there's
nothing in there that shouldn't be wrapped.
kirk_engine.h also uses size_t without including anything that defines it,
which only held together because its includers happened to have it already.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A .sprx from one of these packages is an NPDRM "\0PSPEDAT" container: a
0x90-byte header, then an ordinary ~PSP PRX. The loader only ever saw the
EDAT magic and gave up with SCE_KERNEL_ERROR_UNSUPPORTED_PRX_TYPE.
Step over the header, then derive the key the PRX inside is really
encrypted against: sceNpDrmGetFixedKey() over the content ID, XOR in the
licensee key the game handed us through sceNpDrmSetLicenseeKey(), then AES
under a module key that had to be added. Both halves of that were already
lying around unused - sceNpDrmGetFixedKey() had no callers at all, and the
licensee key was being kept and never read.
The rest of it is a fixed XOR that the PRX header's decrypt_mode selects
rather than its tag, so it's applied on the mode the way JPCSP does it and
the tag table is left alone - tag 0x407810F0 carries no seed of its own
there either, so ours was never wrong about it. pspDecryptType5() already
had a slot for both XORs; no new decryption logic was needed.
Decryption is only half of it: these modules are KL4E-compressed rather
than gzipped, so they also need Core/Util/KL4E.cpp, which is already there
for the firmware modules that use the same compression. With both halves
Shiren 4 Plus loads its one big .sprx and runs. God Eater 2 needed one
further fix that isn't in this commit - the type-B relocation bug in
ElfReader::LoadRelocations2, issue #8075 - and then plays.
docs/pkg_notes.md has the container layout and the key derivation.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
LoadRelocations2 declared last_type, initialised it to -1, read it once - and
never assigned it. So the (flag & 0x38) == 0x08 case, which means "reuse the
lo16 the previous relocation carried", always saw last_type != 4 and reset
lo16 to 0 instead.
That matters because R_MIPS_HI16 computes ((op << 16) + lo16) + relocate_to and
then adds 0x10000 if bit 15 of the result is set, to pre-compensate the sign
extension the paired addiu will do. With lo16 wrongly 0 the carry decision is
made on the load address alone, so for any base whose low half has bit 15 set
the high half comes out one too high and the pointer lands 0x10000 past what it
should be.
A compiler emits exactly this pattern around a branch-likely: one lui in the
delay slot, another on the fall-through path, both for the same symbol, sharing
a single addiu after the paths converge. Only the second lui is adjacent to a
HI16, so the first one silently got the wrong high half.
last_type is assigned where JPCSP assigns its R_TYPE_OLD: at the end of the
branch that actually relocates something, so the commands that only move the
base around don't count as "the previous relocation" and a HI16/HI16/LO16 group
still pairs up across them. R_MIPS_NONE stops continuing the loop for the same
reason - it has to clear last_type, or a HI16 after it would reuse a lo16 that
isn't its own.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The disc-updater install ran when ltn0.pgf was missing, so a font set unpacked
from an old UMD looked complete forever, and a later game wanting a font its own
firmware added silently got a bundled substitute instead.
Requiring the whole registry doesn't work either: firmwares older than a font
can never satisfy it, so we'd unpack the same updater on every launch and
announce it each time. What settles it is that a game can't ask for a font that
didn't exist when it was made. Record the earliest firmware known to ship each
font in the registry, and only require the ones the running game's firmware
would have had.
The version comes from PARAM.SFO's PSP_SYSTEM_VER, with the bundled updater's
version as a fallback. That keeps the whole thing stateless - nothing recorded
that could go stale when flash0 or the ini gets moved around.
Survey of a large library, unpacking flash0:/font from each disc's updater
across firmware 1.50 to 6.60: jpn0 and ltn0..ltn15 are in every one of them, and
kr0.pgf is the only registry font that arrived later - absent in 1.50, present
from 1.52.
Uses one directory listing rather than a stat per font, since on Android's
scoped storage the individual checks are slow.
C++ homebrew has an unreadable symbol table -
everything is _ZN10PxRenderer7DrawImmE... - which makes the disassembly and
symbol list nearly useless. Add an Itanium C++ ABI demangler and run ELF
symbols through it on load, in both ElfReader::LoadSymbols (unstripped EXECs,
which is what a CMake pspdev EBOOT actually contains) and the companion-ELF
path.
The demangling standard is called Itanium for historical reasons - it
was defined for Itanium but ended up being almost universally
applicable.
Written from scratch rather than using __cxa_demangle, which doesn't exist on
MSVC/UWP, or vendoring LLVM's demangler, whose license doesn't fit. Anything
unrecognized (arbitrary constant expressions, decltype) aborts the parse and
the caller gets the original mangled name back, so a caller never sees a
half-parsed result. Recursion is depth-capped since the input comes from a
file we didn't write.
Checked against c++filt as an oracle: of 1089 mangled symbols in a real C++
homebrew EBOOT, one differs; of 55189 from libstdc++/libLLVM/cc1plus, 22
differ and 413 are declined. Fuzzed with 220k mutated and random inputs under
ASan/UBSan.
Also adds a right-click menu to the ImDebugger symbol list.
Note that SymbolMap stores names in char[128], so the longest STL names get
truncated in the UI. Still far more readable than the mangled form.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X3DbkJ8ShYiXU7q5Tv1LZu
When launching a file from outside the main screen (file association, shortcut,
drag-and-drop), the info hasn't been computed yet, so the file type and ID checks
that decide what to do with the file were reading empty data. Add
GameInfo::WaitUntilReady() - a condition variable signalled from
MarkReadyNoLock(), which every exit path of the work item goes through - and use
it there.
Also demote a noisy PRX decryption log line to DEBUG.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EMLTdwyyzU6Mze3w8VC2JL
pspDecryptPRX() tried types 0, 1, 2, 5 and 6. flash0:/vsh/etc/index_XXg.dat -
the index of what the XMB shows, fetched through sceResmgr_9DC14891 - needs
type 9, so it failed and the shell had no menu to build.
Type 9 is type 6 with three differences, all following from a type 9 file
carrying a real ECDSA signature at 0x104..0x12C where a type 6 file has nothing:
- The "must be empty" header check stops at 0x104 instead of 0x10C. The index's
signature starts there, so 8 of its bytes were failing type 6's check - the
original failure.
- The signature is left out of the hashed header rather than fed into it. JPCSP
zeroes buf2[0x34..0x5C), which is that same range once its header
rearrangement is undone, so PRXType9 just leaves the field zero.
- ecdsa_hash in the KIRK CMD1 header stays 0. Type 6/7 set it, but the branch
type 9 takes writes only the mode word, and setting it made KIRK reject the
block.
Tried last in the chain: its header check is a subset of type 6's, so a genuine
type 6 PRX would pass it and then fail on the hash, and trying it earlier would
shadow the real answer. False positives are not really possible either way - the
SHA1 check inside has to match before anything is decrypted.
Verified end to end: 496 bytes in, 159 out (the comp_size in the header),
starting "release:". sceResmgr checks that prefix and says so in its log line,
since a wrong-but-plausible decrypt would otherwise look like success here and
fail much later as an unreadable index.
The VSH now draws something different - the per-frame display list settles at 24
stall points rather than 38 - but what it shows is not visually confirmed;
framebuffer readback doesn't work under headless on either backend.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Found the cause of the red error screen the VSH ends on. Every resource load in
the boot succeeds - fonts, all the plugin RCOs, topmenu_icon.rco - and then:
sceIoOpen(flash0:/vsh/etc/index_02g.dat) -> fd 8
sceIoRead(8, 092a2d40, 496)
sceIoClose(8)
unresolved import sceResmgr/9dc14891, called from 'vsh_module'
sceKernelExitDeleteThread(1)
index_02g.dat is the index of what the XMB displays, and it is encrypted (it
starts "PSPsysGP"). sceResmgr_9DC14891 decrypts it. There was no sceResmgr module
at all, so the call trapped, the index stayed encrypted, and the ScePafJob thread
building the top menu exited - a shell with everything loaded and nothing to show.
This adds the module and the three tags it needs (0x0B2B90F0/91F0/92F0, keys and
code 0x5C) to PrxDecrypter.
It is not the whole fix yet: pspDecryptPRX() tries decryption types 0, 1, 2, 5
and 6, and this needs type 9, which JPCSP passes explicitly. So the call is now
reached and fails cleanly with a logged error instead of trapping, but does not
yet decrypt. Type 9 is a variant of type 2 and is the next job; the notes in
docs/VSHBootInvestigation.md say where it is in JPCSP and how to check a port
(159 bytes out, starting "release:").
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
An updater's DATA.PSAR is a flat sequence of records - each is 0x150 bytes of
PRX-style encryption header, a 0x110 byte entry describing one file, and then
its compressed contents. So to get the files, you don't actually have to run it
and let it self-unpack - we can just do it.
Two steps per record. First "demangle": the 0x130 bytes at +0x20 are AES-CBC
encrypted on top of everything else and hide the PRX tag at +0xD0, so a KIRK
CMD7 pass with keyseed 0x55 comes first. Then the record is an ordinary PRX blob
for the decrypter we already have, once it knows the tag - 0x0E000000, which is
new here. Its key needs the kirk7 scramble applied, unlike every other key in
that table, which are stored already scrambled; hence the flag on TAG_INFO.
UnpackPSAR() takes a prefix filter, since the planned main use for this is pulling
flash0:/font out of an updater the user supplies (or from an ISO) rather than
extracting whole firmwares, although that can also be interesting for running
the VSH.
Tested on a 6.61 updater: 436 entries, all 418 files decrypt and decompress,
nothing fails. The contents are what they should be - 295 ~PSP modules, 61 PRF
files, 18 PGF fonts, and the encrypted XMB indices.
Two things it doesn't do yet. Every entry in that archive is named with a
five-digit token rather than a path; the real names live in tables 00001-00012
inside the archive itself, under their own separate encryption, so files come
out under the short name for now and the prefix filter can't match them.
And only the zlib compression format is currently supported.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Fixes cases where EBOOTs have more than 32 program headers. That limit came from
segmentVAddr being a 32-entry array indexed by program header number
Changes the table to a vector, and holds SEGMENT_NOT_LOADED for
headers that aren't PT_LOAD, so a relocation naming one is rejected and logged
rather than quietly relocating against zero. GetSegmentVaddr() still answers 0
for those, as it did when the table was a zero-initialized array.
The "is this an ELF rather than a PBP" test compared against "\nFLE", which is
neither ELF's magic (\x7fELF) nor anything else - most likely a \x7f escape that
swallowed the E when it was written in 2013. Since no real file matches it,
every file that wasn't a PBP was reported as an ELF and the error branch was
unreachable. Compares against the real magic now, so something that's neither is
reported as neither. That error also printed the 4-byte magic with %s, which
isn't NUL-terminated - it's four hex bytes instead.
GetSubFileSize subtracted offsets that come straight out of the file without
checking they're ordered or even inside it, so a corrupt PBP produced a size
from an unsigned underflow - nearly 4GB, which the callers then had to catch by
size limit. It returns 0 for anything that doesn't make sense.
Also &(*out)[0] on a zero-length subfile, which is UB on an empty vector.
Plus one in ParamSFO: GetDataOffset mixed int and u32 for the data offset, so
its bounds check ran in whichever type the promotion landed on. It's size_t
throughout now, matching how ReadSFO does the same arithmetic.
Booted an EBOOT.PBP to check the PBP path end to end - loads, and generates the
same fake disc ID as before. pspautotests 314/314, UnitTest 55/55.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Nothing here is known to misbehave on a real file - it's the input validation
around the fixes in the preceding commits.
ElfReader's constructor read e_phoff and e_shoff out of the header to find the
segment and section tables, before anything had established there was a header
there; LoadInto's size check only runs later. It leaves header null in that case
now, and the accessors that use it cope.
GetSegmentPtr didn't range-check the segment index at all, and both it and
GetSectionDataPtr accepted an offset exactly at the end of the file, which
addresses no bytes. GetSectionAddr and GetSectionSize took an index on trust.
LoadRelocations2 got most of this commit. Its segment end came from p_filesz
without checking the segment fits in the file, so the whole decode could run off
the end of the buffer. Within it, the flag and type tables are indexed by
bitfields out of each command word and were never checked against the table
sizes (which themselves come from the file); the loop only guaranteed one byte
was left before reading a two-byte command, and the branches that consume a
further two or four bytes checked nothing at all; and the offset segment number
- unlike the address segment number a few lines up - was used to index
segmentVAddr unchecked, though it's wide enough to exceed it. The command read
is byte-wise now too: how far buf has advanced depends on those file-supplied
table sizes, so it isn't necessarily even.
LoadSymbols only checked that a symbol name started inside the file, not that it
was terminated there.
Also dropped the atomic counter and the ParallelLoop.h include left over from
when LoadRelocations ran in parallel.
pspautotests 314/314 with --graphics=software.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
The fake disc ID homebrew gets when it has no PARAM.SFO is built from the sum of
the bytes of its folder name, summed through a plain char - which is signed on
x86 and unsigned on ARM. So the same homebrew folder produced one ID on Windows
and a different one on Android, quietly splitting its savestates and per-game
config between platforms. Sum through unsigned char, which is what the ARM
builds (Android, iOS, Apple Silicon) already did.
Uppercasing is now explicit and ASCII-only rather than toupper(). Passing a
negative char to toupper() is undefined and trips MSVC's debug CRT assert, so a
folder with a non-ASCII name could stop a debug build dead, and what it did with
bytes above 0x7F otherwise depended on the locale.
ASCII folder names - very nearly all of them - produce exactly the same ID as
before. Non-ASCII ones change on the signed-char platforms, to what the
unsigned-char ones were already generating.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
addrToHiLo's verification computed (hi<<16) + lo with hi a u16, which promotes
to int - so for any kernel module, loading at 0x88000000, the shift overflowed a
signed int. Undefined behaviour in the one place whose whole job is to check
that a relocation came out right.
A HI16 that found no matching LO16 logged an error and then wrote its zero-
initialized hi into the instruction anyway, blanking the immediate of a lui it
had just admitted it couldn't resolve. It leaves the instruction alone now: we
don't know the right value, and a zeroed lui produces a wrong address far from
here rather than a failure anyone can trace back.
And the candidate LO16's address was computed with the HI16's segment base
rather than its own, which is exactly the mismatch the warning a few lines below
exists to report - so when that warning fired, the IsValidAddress check guarding
the pairing had been applied to an address from the wrong segment.
pspautotests 314/314 with --graphics=software.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Every decrypt type reads the tag at 0xD0, the compressed size at 0xB0 and key
data as far as 0x150, and writes a KIRK header into outbuf at a fixed offset
derived from sizeof(PSP_Header) - all without checking that either buffer is
that big. A PRX declaring a tiny psp_size therefore read past the end of its
input, wrote a 0xE0-byte header past the end of an equally tiny output buffer,
and handed KIRK "size - offset" as an unsigned underflow. The header write sits
behind the SHA-1 check, but the tag keys are compiled in and every hashed input
comes from the file, so that's arithmetic rather than luck. One size check at
the top of pspDecryptPRX covers all five types.
The module loader needed two things to go with it. Its "maybe it just isn't
encrypted" fallback checked for ELF magic at 0x150 of the *output* buffer, which
on the paths where decryption bails early has nothing written to it yet - so it
read uninitialized heap to decide, and then, if psp_size was under 0x150,
memcpy'd a negative length. It reads the input buffer now, which is what it goes
on to copy from anyway, and only when psp_size is big enough to hold what's
being tested.
Second, the returned size is just comp_size out of the file header, checked
against the allocated buffer by a _dbg_assert_ that isn't there in release. That
check is a real one now, folded into the existing sanity test next to it.
pspautotests cpu 11/11.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
header_ was declared const and value-initialized, and then the constructor read
the file into it via (u8 *)&header_. The C-style cast makes that compile, but
modifying a const object is undefined - the compiler is entitled to keep
assuming header_ still holds the zeroes it was initialized with, and fold reads
of it accordingly. It happens to work today; there's no reason to keep relying
on that.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Each entry gets param_max_len bytes in the data table, and that's what the
buffer is sized from - but nothing clamped what got written into it.
For VT_UTF8 the length written was s_value.size()+1, and then a terminator was
stored at data_ptr[param_len], one byte beyond that again. The memcpy already
copies the terminator (param_len counts it), so that store was both redundant
and always out of range. It doesn't take a malformed file to hit: several
callers pass the string's own length as max_size - see PSPLoaders.cpp's TITLE,
DISC_ID and DISC_VERSION - so the entry overran by two bytes every time, and a
128-character SAVEDATA_TITLE in a 128-byte slot wrote its terminator into the
next entry's data. VT_UTF8_SPE had the same missing clamp without the
off-by-one.
Both are clamped now and log when they truncate, and a negative max_size no
longer subtracts from the computed buffer size. Bytes written are unchanged for
values that do fit, which is every normal case - the terminator now comes from
the zero-fill instead of an explicit store - so this doesn't change any savedata
the emulator produces.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Both relocation branches took the section to modify straight from the file's
sh_info and only checked it wasn't negative, so a value like 1000 in a
ten-section file read past the end of the section table and decided what to
relocate based on whatever was there. sh_info is a u32, so ">= 0" only rejected
the half of the range above INT_MAX.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
st_shndx is a u16 that can hold a reserved value instead of a section number,
and SHN_ABS (0xFFF1) is common in real symbol tables - linker-script constants
like _gp end up there. LoadSymbols fed it straight to sectionAddrs, which has
GetNumSections() entries, so those symbols read a quarter of a megabyte past the
allocation and added whatever they found to the symbol's address. Unlike the
other bounds problems around here this one doesn't need a malformed file; any
ordinary ELF with an absolute symbol hits it.
Symbols that are undefined, absolute or common now get skipped instead - there's
nothing of ours to relocate them against - and a section number that's in range
for neither is skipped with a warning.
Also bail out if LoadInto hasn't run, since that's what fills in sectionAddrs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
segmentVAddr is a 32-entry array, but LoadInto filled it from e_phnum, which is
a u16 - so an ELF declaring 65535 program headers wrote 65535 u32s into it,
straight through the rest of the ElfReader object. The only check standing in
front of that verified the program headers fit in the file, which a ~2MB crafted
PRX satisfies. Rejected up front now: LoadRelocations already ignores segment
numbers at or past the array size, so a module with more than that couldn't be
relocated correctly anyway. Real modules have a handful - PSP_Header::nsegments
is a u8 and no more than 4 are ever used.
Second one from the same loop: with no PT_LOAD segment at all, totalStart stayed
0xFFFFFFFF and totalEnd 0, so totalSize came out as 1 (0 - 0xFFFFFFFF) and the
loader went on to allocate a 1-byte block at 0xFFFFFFFF.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Two things kept the companion ELF from doing its job.
The first is the one that mattered: fileToStart is the game's own *directory*
for folder-launched homebrew (IdentifiedFileType::PSP_PBP_DIRECTORY), which is
the normal case when you pick a homebrew in the UI. The search navigated up from
it regardless, landing in PSP/GAME and listing sibling games - all directories,
all skipped - so app.elf sitting right next to the EBOOT was never found. It only
ever worked when the path pointed at the EBOOT itself, which is how headless is
invoked, which is why it looked fine from there. Searches the directory itself
now when that's what it's given.
The second: line info didn't survive loading a savestate. Modules aren't just
re-registered there, they're destroyed and rebuilt (KernelObjectPool::Clear), so
removing a module's lines in ~PSPModule threw the table away on every state
load. The previous commit worked around it by re-reading the companion, which
was both wasteful and no help at all to an ELF launched directly - those bytes
are long gone by then.
SymbolMap already solves this and line info now does it the same way: keep what
you have, and let whatever next claims the address range replace it. AddModule
replaces by key and is called for every module load, including ones with no line
info of their own, so a range gets retired when it's genuinely reused. The whole
table goes when the game does, in PSP_Shutdown. That also means the savestate
path has nothing to re-read, so state loads no longer pay to re-parse a
multi-megabyte ELF.
The tradeoff is a window between a module unloading and its range being reclaimed
where a lookup can still answer for it. For a debugger that's a stale file:line
on an address nothing owns, against certain and total loss on every state load.
Verified with --auto-save-load-symbols off, launched both ways: by directory
(the case that was broken) and by EBOOT path, both give 3734 symbols and 98383
line rows.
pspautotests 314/314, UnitTest 55/55.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
bAutoSaveLoadSymbols is about writing .ppsym files back out and reading them in
again. It had also come to gate reading debug info that's simply sitting next to
the game, which is a different thing and shouldn't need asking for: the main
ELF's own symbols were already loaded unconditionally, but the companion ELF's
symbols and all line info were not.
Now the ELF is always the baseline - main or companion, symbols and line info -
and the setting only adds the .ppsym half on top of it.
Line info also loads from the module being loaded, not just from a companion,
so an ELF launched directly brings its own. A PRX has no .debug section for it
to find (prxgen strips them), so that's a cheap no-op for the usual EBOOT case,
which the companion path still covers.
That second source needs the two shapes distinguished, so AddModule takes an
explicit address delta rather than assuming a base: a companion links at zero
and wants the module's base added, while an ELF loaded at the addresses it asked
for already has final ones (bRelocate is just e_type != ET_EXEC). Rows that
don't land inside the module after that are dropped either way, which is a
better check than the old "offset smaller than the module" one.
Splitting the companion's identity check out of the symbol loader lets line info
reuse it, and drops an accidental requirement along the way: it used to reject
any companion without a symbol table, so an ELF built with -g but stripped of
its symbols would have contributed no line numbers either.
Verified with --auto-save-load-symbols off: CrossCraft's companion app.elf loads
3734 symbols and 98383 line rows where it previously loaded neither.
The direct-ELF path is not verified at runtime - it needs a bootable ELF that
carries DWARF, and there isn't one to hand. Both candidates here (pspautotests'
.elf builds and CrossCraft's own app.elf) are linked at address 0 and fail to
boot on that alone, which is pre-existing loader behaviour and nothing to do
with this.
pspautotests 314/314, UnitTest 55/55.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Homebrew commonly ships its unstripped ELF next to the EBOOT, which is already
how the symbol loader turns z_un_08841f98 into a function name. That same ELF
carries a DWARF .debug_line section, so the addresses can be mapped to source
files and lines too - and a backtrace stops being four hex numbers:
08841f98 move sp,fp mesh.zig:163
0883afa4 li v0,0x0 MenuState.zig:821
088260d8 andi at,v0,0xFFFF State.zig:40
0882a27c andi at,v0,0xFFFF engine.zig:468
Surfaced in three places: per frame in hle.backtrace, in the "hit" object that
cpu.breakpoint.hit and cpu.stepping share, and appended to the disassembly
window's status bar. The breakpoint case keys on the pc rather than the address,
since for a memory breakpoint the useful source location is the instruction that
did the access, not the data it touched.
Storage is a plain sorted table of absolute addresses per module. SymbolMap
keeps module-relative addresses because its .ppsym files are meant to be
reloaded by a different game that pulls in the same module; none of this is ever
written anywhere - it's regenerated from the ELF each boot - so there'd be
nothing for relative addresses to buy. Each module owns its own rows and file
names outright and is keyed the way SymbolMap::UnloadModule is, so unloading one
module drops its lines and nobody else's.
The subtle part is end-of-sequence markers. Without them a lookup for an address
in a gap - a compilation unit built without debug info - confidently reports the
last line of an unrelated file. A prototype run over one test binary
mis-attributed 70 of its 349 functions that way, so sequence ends are recorded
as rows with line 0 and a lookup landing on one reports nothing instead.
DWARF 2 through 4 are decoded (psp-gcc emits 2, Zig 4). Version 5 re-encoded the
file table, so those units are skipped with a warning rather than mis-parsed -
nothing targeting the PSP produces it today.
Scope, since it's narrower than it sounds: PRX conversion strips every .debug
section. I checked all 437 pspautotests .prx and CrossCraft's own app.prx -
none have any. Of 24 installed homebrew EBOOTs, zero carry debug info; CrossCraft
only does because it ships app.elf separately. So this helps someone developing
homebrew, and does nothing at all for a commercial game.
Costs about 1.2 MB for a large Zig binary (98383 rows, 438 files) and nothing
for anything without debug info. Follows bAutoSaveLoadSymbols like the symbols
do.
pspautotests 314/314, UnitTest 55/55, CoreUWP builds.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Homebrew almost always ships the ELF it was built from alongside the EBOOT -
app.elf next to app.prx - but prxgen strips the symbol table on the way to the
PRX, so the module PPSSPP loads has no names at all and MIPSAnalyst calls every
function it finds z_un_<address>. Working out what any of them are meant hand-
parsing that ELF with a throwaway script, which is how the CrossCraft
relocation bug got identified.
So read it directly. On module load, scan the game's own directory for an ELF
with a symbol table and add its STT_FUNC/STT_OBJECT entries at the module's
base. CrossCraft picks up 3734 symbols, and the disassembly turns from
z_un_088c00f0 into world.init_empty, with static_allocator.alloc at the vtable
entry it calls - the two functions that took the longest to identify by hand.
Matching is the part worth getting right, since a wrong match puts confident
nonsense at real addresses, which beats having no names only in the sense that
it's worse. A candidate has to be a 32-bit ELF with a symbol table whose
highest section ends within a page of the loaded module's size - the companion
links at base 0 and covers the same image, so that's a tight check, and
unrelated ELFs sitting in the same folder fail it. Symbols outside the module
are skipped individually too.
Names go in with updateName, so they win over the analyzer's placeholders
rather than losing to whichever got there first. Gated on the existing
bAutoSaveLoadSymbols setting (off by default), which already means "keep symbol
names around for me" and avoids a directory scan per module load otherwise.
pspautotests 314/314.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
ElfReader read the first loadable segment's p_align into firstSegAlign but only
ever used it to round down textStart for symbol bookkeeping - the allocation
itself used the block allocator's default grain. A module whose relocations are
only valid at a more strictly aligned base therefore got loaded somewhere it
couldn't work, and the symptom is addresses off by a multiple of 64KB rather
than an outright failure.
CrossCraft Classic (Zig) declares p_align 0x10000 and hits exactly that. It
declares it deliberately: a stage of Zig's PSP pipeline emits mispaired
HI16/LO16 relocations, and a 64KB-aligned base makes that harmless - with no
low bits in the base no carry is ever needed, so which of a symbol's LO16
entries a HI16 was paired with stops affecting the result. PPSSPP put it at
0x08804000 instead, where the carry does matter: 46 of its addresses came out
64KB low, and it jumped through a bogus vtable a few seconds in. It now loads
at 0x08810000 and all 8405 lui/addiu pairs resolve correctly.
Worth being clear that the relocation code was never wrong here - it matches
what the hardware does, pairing a run of HI16 with the next non-HI16 entry and
applying the carry. Only the load address differed.
The two AllocAt paths can't move the module, since the caller picked the
address, so they just report a misaligned one instead.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
ReadSFO dereferenced index table entries without checking the table fit
within the buffer, and GetDataOffset had no bounds checks at all (reading
attacker-controlled offsets and strcmp'ing without a terminator guard).
- Validate the index table fits entirely within the buffer in ReadSFO.
- Add a size parameter to GetDataOffset and validate the index table,
key/data table positions, and key string termination before use.
