Adds GameInfoFlags::BUNDLED_UPDATE_INFO, holding the version, title, size
and timestamp of the updater in PSP_GAME/SYSDIR/UPDATE. It comes from the
PARAM.SFO and the directory entry next to the archive, so it's a couple of
small reads on the ISOFileSystem the worker already has open - no
decryption, and DATA.BIN itself is only sniffed for its magic. Only
computed for ISOs; everything else is marked complete with an empty struct.
ISOFileSystem now parses the date out of the ISO9660 directory record,
stored as Unix UTC seconds and reported as the PSP's atime/ctime/mtime.
Those used to always read back as zero, so games calling sceIoGetstat on a
UMD file saw 1900 where hardware gives the mastering date.
Shown on GameScreen as e.g. "Firmware update on disc: 6.60 (2011-10-05),
25.6 MB".
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0149QcTVgZEXKXbgHyvXF4ZY
log.channel.set is meant for temporary, session-only diagnostic
tweaks (e.g. quieting a noisy channel while investigating something
over the WebSocket debugger). It was going through the same
SetLogLevel/SetEnabled calls the UI settings use, so a normal app
exit would persist whatever channels/levels the debugger last left
set, silently overwriting the user's real saved log preferences for
future runs - discovered when a later session's default logging
looked "off" for no apparent reason.
LogManager now tracks whether the debugger has touched channel
config this run and skips SaveConfig() entirely if so, leaving
whatever's already on disk untouched.
A firmware dumped off a console has its modules decrypted, while one unpacked
from an updater still has them as ~PSP blobs. IdentifyFile knew about ELF and
PBP but not that, so the loader refused those outright - even though
__KernelLoadELFFromPtr decrypts ~PSP perfectly well once it gets that far, which
is how every encrypted game EBOOT loads.
Checked against a vshmain.prx unpacked from the 6.61 updater: it now gets as far
as "Decrypting ~PSP file" in the module loader instead of stopping at
"CPU_Init didn't recognize file".
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Two small things I'd noticed as counters in a survey and written off as quirks
of the oldest firmwares. Both were bugs here.
The last entry of every 1.x and 2.x archive was dropped. Decoding a record hands
the decrypter 16 bytes of slack past the block, which was read out of the
archive - and the final record ends flush with the end, so there was nothing
there to read and the entry was refused. It copies what's there and zero-fills
only the remainder now. Worth noting the slack isn't decoration: filling it with
zeros unconditionally, which is what I tried first, breaks every archive that
does have those bytes, so the decrypter reads and uses them.
And 2.x names one entry "ipl:/psp_nandipl.bin", where the check for "is this
already a real path" only knew flash0: and flash1:. The file came out fine, but
it was counted unresolved, and - the part that matters - an entry that isn't a
real path is skipped whenever a prefix filter is set. Rather than collect device
names, the test is now for a "<dev>:/" shape, which 3.x's grouped short names
("com:00123") don't have.
Seven sources, firmware 1.50 through 6.61, now unpack with nothing unresolved
and nothing failed: 1.50 and 1.52 gain their missing last file, 2.81 loses its
unresolved count, and 3.95, 6.00, 6.20 and 6.61 are unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
The font extraction this is for happens while a game is running, when disc0: is
already mounted - so opening the image a second time to read one file out of it
is the wrong shape, and doesn't work at all for the cases that aren't an image,
like a folder-based disc. UnpackUpdaterFromMountedDisc() goes through
pspFileSystem instead, with MountedDiscHasUpdater() to ask cheaply first and
ReadMountedDiscUpdaterVersion() for the version out of the PARAM.SFO.
Testing it turned up that the prefix filter didn't work on 3.x archives at all:
their file lists write paths as "flash0/font/x.pgf" where 6.x writes
"flash0:/font/x.pgf", so a filter of "flash0:/font/" matched nothing and the
unpack quietly produced no files. Both forms are normalized to the 6.x one now,
so a caller only has to know one. The unfiltered output was already identical
either way, which is why the earlier disc tests looked fine.
Verified by booting Crisis Core and pulling flash0:/font out of its 3.95 updater
through the mounted disc: 21 files, nothing else, no failures. Booting homebrew
with no disc0: at all reports no updater rather than failing.
pspautotests 314/314 with --graphics=software, UnitTest 55/55, and the three
file-based sources (6.61 PBP, 3.95 and 6.20 discs) still unpack unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Most UMDs carry a firmware updater in PSP_GAME/SYSDIR/UPDATE, so the fonts can
come from a game the user already has instead of a separate download. Its
DATA.BIN turns out to be exactly the same archive as a downloaded updater's
DATA.PSAR, just without the PBP around it.
UnpackUpdater() replaces UnpackUpdaterPBP() and takes any of the three shapes: a
downloaded EBOOT.PBP, a bare DATA.BIN/PSAR, or a disc image, which it opens with
the block device and ISO filesystem we already have and looks in SYSDIR/UPDATE.
ReadUpdaterVersion() answers the version from the PARAM.SFO next to the archive
without decrypting anything, which is cheap enough to check every disc with.
Testing across eras turned up three things the 6.61 updater alone never showed:
Old archives name entries differently. 3.x groups them by model - "com:00123",
"01g:00005" - with "<group>:00000" as that group's file list, keyed on just the
number, separated by '|' rather than ',' and with paths written "flash0/font/x"
rather than "flash0:/font/x". 1.x skips the indirection and stores real paths.
Both are handled now.
Which numbers are file lists isn't fixed either. 6.61 uses 1-11, but 6.00 has
real files at 00010-00012, which were being taken for corrupt lists and dropped.
A list always decrypts, since the PRX layer under it validates a hash, so a
failure there now just means "this is a file" - which recovered 3 files each on
6.00 and 6.20.
And the walk ran one record past the end. The archive header says how long the
records really are, and both archives have a few bytes of padding after that.
Read from the discs of Coded Arms (1.50), Ace Combat X (2.81), Crisis Core
(3.95), Assassin's Creed Bloodlines (6.00) and BlazBlue (6.20), plus the
downloaded 6.61. Every one gives up its fonts - 17 of them on 1.50, 19 on 2.81,
21 from 3.95 on. All but two are clean: 1.50 has one .rco whose block won't
decrypt, and 2.81 has one name no list claims.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
An updater carries one file list per hardware revision, and which one you
resolve names against decides both what a file is called and whether it's part
of that model's firmware at all. That was hardcoded to "first list that names
it", which is right for extracting everything but wrong for reproducing what a
particular console would have installed.
PSARUnpackOptions::model takes a PSPModelGeneration now, and the lists are kept
per model rather than merged. Any (the default) keeps the old behaviour;
anything else uses only that model's list and skips what it doesn't name.
--unpack-updater-model on headless takes "01g".."12g" or "any".
On the 6.61 updater: any gives 411 files, 03g gives 330 with 81 belonging to
other models, 01g gives 313 with 98. The difference is what it should be - 03g
has the _03g.prx variants and arib.pgf, 01g has neither.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Every entry in an updater is named with a five-digit token. The paths live in
entries 00001-00012 of the same archive, one per PSP model, each a list of
"shortname,realpath" lines - so with those decrypted the walk can name
everything, and the prefix filter this was written for becomes usable.
The tables looked like they needed an unknown cipher, but the tables in the
reference implementations are DES's: a 56-entry PC-1, a 48-entry PC-2, 8x64
4-bit S-boxes and a 32-entry P, with the constants of a textbook IP/FP in the
bit-shuffling. It's DES-CBC decrypt with the key assembled from two words and
the IV alongside it, both per firmware series, and then an ordinary PRX blob
underneath. So this is a plain DES, one bit per byte, since the tables are a few
tens of KB in total and readable permutations matter more than speed here.
Confirmed by decrypting a table and finding a known PRX tag at 0xD0, which a
wrong key would not have produced.
On the 6.61 updater all 411 files now come out under their real paths -
flash0/font (21 files, ltn0-15, jpn0, kr0, arib, gb3s1518, imagefont),
flash0/kd (195), flash0/vsh/module (100), flash0/vsh/resource (67), and the
rest - with nothing unresolved. Checked the prefix filter separately with
"flash0:/font/": 21 files written and nothing else.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
An updater's DATA.PSAR is a flat sequence of records - each is 0x150 bytes of
PRX-style encryption header, a 0x110 byte entry describing one file, and then
its compressed contents. So to get the files, you don't actually have to run it
and let it self-unpack - we can just do it.
Two steps per record. First "demangle": the 0x130 bytes at +0x20 are AES-CBC
encrypted on top of everything else and hide the PRX tag at +0xD0, so a KIRK
CMD7 pass with keyseed 0x55 comes first. Then the record is an ordinary PRX blob
for the decrypter we already have, once it knows the tag - 0x0E000000, which is
new here. Its key needs the kirk7 scramble applied, unlike every other key in
that table, which are stored already scrambled; hence the flag on TAG_INFO.
UnpackPSAR() takes a prefix filter, since the planned main use for this is pulling
flash0:/font out of an updater the user supplies (or from an ISO) rather than
extracting whole firmwares, although that can also be interesting for running
the VSH.
Tested on a 6.61 updater: 436 entries, all 418 files decrypt and decompress,
nothing fails. The contents are what they should be - 295 ~PSP modules, 61 PRF
files, 18 PGF fonts, and the encrypted XMB indices.
Two things it doesn't do yet. Every entry in that archive is named with a
five-digit token rather than a path; the real names live in tables 00001-00012
inside the archive itself, under their own separate encryption, so files come
out under the short name for now and the prefix filter can't match them.
And only the zlib compression format is currently supported.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Fixes cases where EBOOTs have more than 32 program headers. That limit came from
segmentVAddr being a 32-entry array indexed by program header number
Changes the table to a vector, and holds SEGMENT_NOT_LOADED for
headers that aren't PT_LOAD, so a relocation naming one is rejected and logged
rather than quietly relocating against zero. GetSegmentVaddr() still answers 0
for those, as it did when the table was a zero-initialized array.
--launch starts PPSSPP itself, learns the debugger port from its output,
connects once the socket accepts, and kills it on exit. That replaces
some wrapper scripts, which had to background the emulator,
poll a log file for "Listening on port N", sleep a guessed interval before
connecting, and taskkill afterwards. The polling was also racy - a fixed port
plus a leftover process from an earlier run is a good way to drive the wrong
emulator - and nothing cleaned up, so --timeout's wall-clock budget left
processes alive for hours.
To make that dependable, the port is now also written straight to stderr from
Core/WebServer.cpp, outside the log system entirely. A tool that launches
PPSSPP has to learn the port before it can connect to anything, so that line
must not be losable to a log level or a disabled channel. wsdbg watches both
child streams for it, since which one it lands on depends on how a given build
routes logging.
--quiet sends the broadcast.config.set that every script was hand-writing,
disabling the logger and input broadcasts. The log one is expensive - each line
gets encoded as JSON and pushed down the socket.
The rest is one bug, in the docs rather than the code, which cost a lot of time:
the scripting example paired cpu.runUntilTime with ":wait cpu.stepping". --sync
already waits for the cpu.stepping that follows a resume-family command, so the
explicit :wait waits for a second one that never comes and burns the whole
--sync-timeout. With --sync-timeout 400 that turns a 3-second run into a
7-minute one that looks exactly like a slow boot, because the emulator really
has stopped where it was asked to. Example fixed, and both that and muting the
'stepping' category (same failure, different cause) are called out - wsdbg now
warns when a raw broadcast.config.set disables 'stepping'.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
NormalizePath's Windows branch ignored GetFullPathName failing. A zero return
matched neither of its two branches, so the 512-character buffer was handed back
with whatever was in it.
The content-URI branch then computed its relative path by substr()ing the
decoded FilePath() of the two URIs - but the containment check above it compared
the URIs themselves, which doesn't imply the same prefix relationship for the
decoded paths. When it doesn't hold, substr() throws out_of_range, uncaught.
Checked explicitly now.
More minor fixes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
__StartLogAudio ignored WaveFileWriter::Start()'s return value and set m_logAudio
either way, so a wav file that couldn't be opened - read-only or full audio
directory, most likely - left every subsequent mixed block calling
AddStereoSamples on a closed file. That opens with _assert_msg_(file, ...).
Also other minor fixes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
__KernelLoadELFFromPtr creates its PSPModule and inserts it into loadedModules
before it knows whether the file is loadable, so every failure exit has to
delete the decrypt buffer, Cleanup() the module and Destroy() it. Five of the
seven did. The "unreasonable decrypted size" exit and the decompression-failure
exit just returned - leaking the buffer, and leaving a live kernel object with
its UID stuck in loadedModules for the rest of the session.
While tracing that: the fake-module path frees newptr and then runs for another
sixty lines with ptr still pointing into it. Nothing reads it today - the exits
below use head, which points into the original input rather than the copy - so
there's no use-after-free and no double free, but that's a property of the
current code rather than anything enforced. Both pointers are nulled after the
delete so a future mistake there crashes instead of reading freed heap.
And the function read the magic, and in the ~SCE branch a second word after it,
before anything established the input was that big. The non-PBP caller
guarantees it, but the PBP path computes elfSize from two offsets in the file
and passes whatever comes out, including zero. Checked at the top, before the
module object exists, so that exit needs no cleanup of its own.
pspautotests 314/314 with --graphics=software, and an EBOOT.PBP still boots.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
The "is this an ELF rather than a PBP" test compared against "\nFLE", which is
neither ELF's magic (\x7fELF) nor anything else - most likely a \x7f escape that
swallowed the E when it was written in 2013. Since no real file matches it,
every file that wasn't a PBP was reported as an ELF and the error branch was
unreachable. Compares against the real magic now, so something that's neither is
reported as neither. That error also printed the 4-byte magic with %s, which
isn't NUL-terminated - it's four hex bytes instead.
GetSubFileSize subtracted offsets that come straight out of the file without
checking they're ordered or even inside it, so a corrupt PBP produced a size
from an unsigned underflow - nearly 4GB, which the callers then had to catch by
size limit. It returns 0 for anything that doesn't make sense.
Also &(*out)[0] on a zero-length subfile, which is UB on an empty vector.
Plus one in ParamSFO: GetDataOffset mixed int and u32 for the data offset, so
its bounds check ran in whichever type the promotion landed on. It's size_t
throughout now, matching how ReadSFO does the same arithmetic.
Booted an EBOOT.PBP to check the PBP path end to end - loads, and generates the
same fake disc ID as before. pspautotests 314/314, UnitTest 55/55.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Nothing here is known to misbehave on a real file - it's the input validation
around the fixes in the preceding commits.
ElfReader's constructor read e_phoff and e_shoff out of the header to find the
segment and section tables, before anything had established there was a header
there; LoadInto's size check only runs later. It leaves header null in that case
now, and the accessors that use it cope.
GetSegmentPtr didn't range-check the segment index at all, and both it and
GetSectionDataPtr accepted an offset exactly at the end of the file, which
addresses no bytes. GetSectionAddr and GetSectionSize took an index on trust.
LoadRelocations2 got most of this commit. Its segment end came from p_filesz
without checking the segment fits in the file, so the whole decode could run off
the end of the buffer. Within it, the flag and type tables are indexed by
bitfields out of each command word and were never checked against the table
sizes (which themselves come from the file); the loop only guaranteed one byte
was left before reading a two-byte command, and the branches that consume a
further two or four bytes checked nothing at all; and the offset segment number
- unlike the address segment number a few lines up - was used to index
segmentVAddr unchecked, though it's wide enough to exceed it. The command read
is byte-wise now too: how far buf has advanced depends on those file-supplied
table sizes, so it isn't necessarily even.
LoadSymbols only checked that a symbol name started inside the file, not that it
was terminated there.
Also dropped the atomic counter and the ParallelLoop.h include left over from
when LoadRelocations ran in parallel.
pspautotests 314/314 with --graphics=software.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
The fake disc ID homebrew gets when it has no PARAM.SFO is built from the sum of
the bytes of its folder name, summed through a plain char - which is signed on
x86 and unsigned on ARM. So the same homebrew folder produced one ID on Windows
and a different one on Android, quietly splitting its savestates and per-game
config between platforms. Sum through unsigned char, which is what the ARM
builds (Android, iOS, Apple Silicon) already did.
Uppercasing is now explicit and ASCII-only rather than toupper(). Passing a
negative char to toupper() is undefined and trips MSVC's debug CRT assert, so a
folder with a non-ASCII name could stop a debug build dead, and what it did with
bytes above 0x7F otherwise depended on the locale.
ASCII folder names - very nearly all of them - produce exactly the same ID as
before. Non-ASCII ones change on the signed-char platforms, to what the
unsigned-char ones were already generating.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
addrToHiLo's verification computed (hi<<16) + lo with hi a u16, which promotes
to int - so for any kernel module, loading at 0x88000000, the shift overflowed a
signed int. Undefined behaviour in the one place whose whole job is to check
that a relocation came out right.
A HI16 that found no matching LO16 logged an error and then wrote its zero-
initialized hi into the instruction anyway, blanking the immediate of a lui it
had just admitted it couldn't resolve. It leaves the instruction alone now: we
don't know the right value, and a zeroed lui produces a wrong address far from
here rather than a failure anyone can trace back.
And the candidate LO16's address was computed with the HI16's segment base
rather than its own, which is exactly the mismatch the warning a few lines below
exists to report - so when that warning fired, the IsValidAddress check guarding
the pairing had been applied to an address from the wrong segment.
pspautotests 314/314 with --graphics=software.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Every decrypt type reads the tag at 0xD0, the compressed size at 0xB0 and key
data as far as 0x150, and writes a KIRK header into outbuf at a fixed offset
derived from sizeof(PSP_Header) - all without checking that either buffer is
that big. A PRX declaring a tiny psp_size therefore read past the end of its
input, wrote a 0xE0-byte header past the end of an equally tiny output buffer,
and handed KIRK "size - offset" as an unsigned underflow. The header write sits
behind the SHA-1 check, but the tag keys are compiled in and every hashed input
comes from the file, so that's arithmetic rather than luck. One size check at
the top of pspDecryptPRX covers all five types.
The module loader needed two things to go with it. Its "maybe it just isn't
encrypted" fallback checked for ELF magic at 0x150 of the *output* buffer, which
on the paths where decryption bails early has nothing written to it yet - so it
read uninitialized heap to decide, and then, if psp_size was under 0x150,
memcpy'd a negative length. It reads the input buffer now, which is what it goes
on to copy from anyway, and only when psp_size is big enough to hold what's
being tested.
Second, the returned size is just comp_size out of the file header, checked
against the allocated buffer by a _dbg_assert_ that isn't there in release. That
check is a real one now, folded into the existing sanity test next to it.
pspautotests cpu 11/11.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
header_ was declared const and value-initialized, and then the constructor read
the file into it via (u8 *)&header_. The C-style cast makes that compile, but
modifying a const object is undefined - the compiler is entitled to keep
assuming header_ still holds the zeroes it was initialized with, and fold reads
of it accordingly. It happens to work today; there's no reason to keep relying
on that.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Each entry gets param_max_len bytes in the data table, and that's what the
buffer is sized from - but nothing clamped what got written into it.
For VT_UTF8 the length written was s_value.size()+1, and then a terminator was
stored at data_ptr[param_len], one byte beyond that again. The memcpy already
copies the terminator (param_len counts it), so that store was both redundant
and always out of range. It doesn't take a malformed file to hit: several
callers pass the string's own length as max_size - see PSPLoaders.cpp's TITLE,
DISC_ID and DISC_VERSION - so the entry overran by two bytes every time, and a
128-character SAVEDATA_TITLE in a 128-byte slot wrote its terminator into the
next entry's data. VT_UTF8_SPE had the same missing clamp without the
off-by-one.
Both are clamped now and log when they truncate, and a negative max_size no
longer subtracts from the computed buffer size. Bytes written are unchanged for
values that do fit, which is every normal case - the terminator now comes from
the zero-fill instead of an explicit store - so this doesn't change any savedata
the emulator produces.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Both relocation branches took the section to modify straight from the file's
sh_info and only checked it wasn't negative, so a value like 1000 in a
ten-section file read past the end of the section table and decided what to
relocate based on whatever was there. sh_info is a u32, so ">= 0" only rejected
the half of the range above INT_MAX.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
st_shndx is a u16 that can hold a reserved value instead of a section number,
and SHN_ABS (0xFFF1) is common in real symbol tables - linker-script constants
like _gp end up there. LoadSymbols fed it straight to sectionAddrs, which has
GetNumSections() entries, so those symbols read a quarter of a megabyte past the
allocation and added whatever they found to the symbol's address. Unlike the
other bounds problems around here this one doesn't need a malformed file; any
ordinary ELF with an absolute symbol hits it.
Symbols that are undefined, absolute or common now get skipped instead - there's
nothing of ours to relocate them against - and a section number that's in range
for neither is skipped with a warning.
Also bail out if LoadInto hasn't run, since that's what fills in sectionAddrs.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
segmentVAddr is a 32-entry array, but LoadInto filled it from e_phnum, which is
a u16 - so an ELF declaring 65535 program headers wrote 65535 u32s into it,
straight through the rest of the ElfReader object. The only check standing in
front of that verified the program headers fit in the file, which a ~2MB crafted
PRX satisfies. Rejected up front now: LoadRelocations already ignores segment
numbers at or past the array size, so a module with more than that couldn't be
relocated correctly anyway. Real modules have a handful - PSP_Header::nsegments
is a u8 and no more than 4 are ever used.
Second one from the same loop: with no PT_LOAD segment at all, totalStart stayed
0xFFFFFFFF and totalEnd 0, so totalSize came out as 1 (0 - 0xFFFFFFFF) and the
loader went on to allocate a 1-byte block at 0xFFFFFFFF.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
This is stuff encountered in the VSH boot research.
sceRtc_driver, scePower_driver, sceImpose_driver, ThreadManForKernel funcs,
sceRtcGetAlarmTick, sceHprm_driver/sceHprmReadLatch
sceVshBridge_Driver imports sceKernelResumeDispatchThread, SuspendDispatchThread,
and NotifyCallback from ThreadManForKernel, but they were only registered under
ThreadManForUser. Added sceKernelGetUserLevel and sceKernelIsUserModeThread (new).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Two things kept the companion ELF from doing its job.
The first is the one that mattered: fileToStart is the game's own *directory*
for folder-launched homebrew (IdentifiedFileType::PSP_PBP_DIRECTORY), which is
the normal case when you pick a homebrew in the UI. The search navigated up from
it regardless, landing in PSP/GAME and listing sibling games - all directories,
all skipped - so app.elf sitting right next to the EBOOT was never found. It only
ever worked when the path pointed at the EBOOT itself, which is how headless is
invoked, which is why it looked fine from there. Searches the directory itself
now when that's what it's given.
The second: line info didn't survive loading a savestate. Modules aren't just
re-registered there, they're destroyed and rebuilt (KernelObjectPool::Clear), so
removing a module's lines in ~PSPModule threw the table away on every state
load. The previous commit worked around it by re-reading the companion, which
was both wasteful and no help at all to an ELF launched directly - those bytes
are long gone by then.
SymbolMap already solves this and line info now does it the same way: keep what
you have, and let whatever next claims the address range replace it. AddModule
replaces by key and is called for every module load, including ones with no line
info of their own, so a range gets retired when it's genuinely reused. The whole
table goes when the game does, in PSP_Shutdown. That also means the savestate
path has nothing to re-read, so state loads no longer pay to re-parse a
multi-megabyte ELF.
The tradeoff is a window between a module unloading and its range being reclaimed
where a lookup can still answer for it. For a debugger that's a stale file:line
on an address nothing owns, against certain and total loss on every state load.
Verified with --auto-save-load-symbols off, launched both ways: by directory
(the case that was broken) and by EBOOT path, both give 3734 symbols and 98383
line rows.
pspautotests 314/314, UnitTest 55/55.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Modules are torn down and re-registered from PSPModule::DoState when a state is
loaded, which takes their symbols and line info with them. That path read the
per-module .ppsym file and nothing else, so coming back from a savestate lost
everything the companion ELF had contributed - every function back to
z_un_<address>, and no line numbers at all - with no way to get it back short of
rebooting the game.
It reads the companion again now, on the same terms as the initial load:
unconditionally, since the file is still sitting next to the game and only the
.ppsym half was ever meant to depend on the setting.
Costs a re-read and re-parse of the companion on every state load for games that
have one (CrossCraft: 3734 symbols and 98383 line rows), which is the price of
not silently losing them.
Line info that came from the main ELF rather than a companion still doesn't
survive a state load - those bytes aren't around at that point. Nothing to do
about that here, and it doesn't apply to the EBOOT case, where the companion is
the only source anyway.
pspautotests 314/314, UnitTest 55/55.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
bAutoSaveLoadSymbols is about writing .ppsym files back out and reading them in
again. It had also come to gate reading debug info that's simply sitting next to
the game, which is a different thing and shouldn't need asking for: the main
ELF's own symbols were already loaded unconditionally, but the companion ELF's
symbols and all line info were not.
Now the ELF is always the baseline - main or companion, symbols and line info -
and the setting only adds the .ppsym half on top of it.
Line info also loads from the module being loaded, not just from a companion,
so an ELF launched directly brings its own. A PRX has no .debug section for it
to find (prxgen strips them), so that's a cheap no-op for the usual EBOOT case,
which the companion path still covers.
That second source needs the two shapes distinguished, so AddModule takes an
explicit address delta rather than assuming a base: a companion links at zero
and wants the module's base added, while an ELF loaded at the addresses it asked
for already has final ones (bRelocate is just e_type != ET_EXEC). Rows that
don't land inside the module after that are dropped either way, which is a
better check than the old "offset smaller than the module" one.
Splitting the companion's identity check out of the symbol loader lets line info
reuse it, and drops an accidental requirement along the way: it used to reject
any companion without a symbol table, so an ELF built with -g but stripped of
its symbols would have contributed no line numbers either.
Verified with --auto-save-load-symbols off: CrossCraft's companion app.elf loads
3734 symbols and 98383 line rows where it previously loaded neither.
The direct-ELF path is not verified at runtime - it needs a bootable ELF that
carries DWARF, and there isn't one to hand. Both candidates here (pspautotests'
.elf builds and CrossCraft's own app.elf) are linked at address 0 and fail to
boot on that alone, which is pre-existing loader behaviour and nothing to do
with this.
pspautotests 314/314, UnitTest 55/55.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Following on from the DWARF line table: the lookup was only reachable from
hle.backtrace, the breakpoint hit object and the ImGui disassembly status bar.
Now also in
- the ImDebugger call stack (new Source column),
- the Win32 call stack (new Source column),
- the Win32 disassembly status bar, matching the ImGui one,
- the ImDisasmView right-click menu, which showed a bare address as its heading
and now leads with "mesh.zig:163 (08841f98)" when there's a line for it,
- breakpoint log lines - a log-only breakpoint's entire output is those lines,
and "BKP PC=08841f98 mesh.zig:163" reads a great deal better than an address
when you're scanning a few thousand of them,
- crash stack traces, via FormatStackTrace, which is what the crash screen and
crash reporting both use.
That last one is where it earns its keep, and it needed the invalid-jump path to
produce a stack trace at all - it was the one exec exception that didn't. It's
also the one that most deserves it: the address it jumped to tells you nothing,
the callers tell you everything. Execution has already moved to the bad address
by the time it's noticed, so a walk from pc finds no function to start from;
WalkCurrentStack takes an explicit starting pc now, and falling back to ra
recovers the chain. Reproducing the original CrossCraft bug:
CPU Jump: Invalid jump to ae870000 from PC ae870000(invalid) RA 08841f98
MIPS call stack:
rendering.mesh.Mesh(PspVertex).draw at mesh.zig:163 (08841c30+368, ...)
state.MenuState.draw at MenuState.zig:821 (0883ab90+414, ...)
engine.Engine.stepFrameInternal at State.zig:40 (08820f74+5164, ...)
utils.module._module_main_thread at engine.zig:468 (088272c4+2fb8, ...)
Fixed a pre-existing double-report while in there: every case in
Core_ExecException sent its message and then fell through to an unconditional
send of the same message, so each exec exception was logged twice. The message
is built in the switch and sent once at the end now.
pspautotests 314/314, UnitTest 55/55.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Homebrew commonly ships its unstripped ELF next to the EBOOT, which is already
how the symbol loader turns z_un_08841f98 into a function name. That same ELF
carries a DWARF .debug_line section, so the addresses can be mapped to source
files and lines too - and a backtrace stops being four hex numbers:
08841f98 move sp,fp mesh.zig:163
0883afa4 li v0,0x0 MenuState.zig:821
088260d8 andi at,v0,0xFFFF State.zig:40
0882a27c andi at,v0,0xFFFF engine.zig:468
Surfaced in three places: per frame in hle.backtrace, in the "hit" object that
cpu.breakpoint.hit and cpu.stepping share, and appended to the disassembly
window's status bar. The breakpoint case keys on the pc rather than the address,
since for a memory breakpoint the useful source location is the instruction that
did the access, not the data it touched.
Storage is a plain sorted table of absolute addresses per module. SymbolMap
keeps module-relative addresses because its .ppsym files are meant to be
reloaded by a different game that pulls in the same module; none of this is ever
written anywhere - it's regenerated from the ELF each boot - so there'd be
nothing for relative addresses to buy. Each module owns its own rows and file
names outright and is keyed the way SymbolMap::UnloadModule is, so unloading one
module drops its lines and nobody else's.
The subtle part is end-of-sequence markers. Without them a lookup for an address
in a gap - a compilation unit built without debug info - confidently reports the
last line of an unrelated file. A prototype run over one test binary
mis-attributed 70 of its 349 functions that way, so sequence ends are recorded
as rows with line 0 and a lookup landing on one reports nothing instead.
DWARF 2 through 4 are decoded (psp-gcc emits 2, Zig 4). Version 5 re-encoded the
file table, so those units are skipped with a warning rather than mis-parsed -
nothing targeting the PSP produces it today.
Scope, since it's narrower than it sounds: PRX conversion strips every .debug
section. I checked all 437 pspautotests .prx and CrossCraft's own app.prx -
none have any. Of 24 installed homebrew EBOOTs, zero carry debug info; CrossCraft
only does because it ships app.elf separately. So this helps someone developing
homebrew, and does nothing at all for a commercial game.
Costs about 1.2 MB for a large Zig binary (98383 rows, 438 files) and nothing
for anything without debug info. Follows bAutoSaveLoadSymbols like the symbols
do.
pspautotests 314/314, UnitTest 55/55, CoreUWP builds.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
