The disc-updater install ran when ltn0.pgf was missing, so a font set unpacked
from an old UMD looked complete forever, and a later game wanting a font its own
firmware added silently got a bundled substitute instead.
Requiring the whole registry doesn't work either: firmwares older than a font
can never satisfy it, so we'd unpack the same updater on every launch and
announce it each time. What settles it is that a game can't ask for a font that
didn't exist when it was made. Record the earliest firmware known to ship each
font in the registry, and only require the ones the running game's firmware
would have had.
The version comes from PARAM.SFO's PSP_SYSTEM_VER, with the bundled updater's
version as a fallback. That keeps the whole thing stateless - nothing recorded
that could go stale when flash0 or the ini gets moved around.
Survey of a large library, unpacking flash0:/font from each disc's updater
across firmware 1.50 to 6.60: jpn0 and ltn0..ltn15 are in every one of them, and
kr0.pgf is the only registry font that arrived later - absent in 1.50, present
from 1.52.
Uses one directory listing rather than a stat per font, since on Android's
scoped storage the individual checks are slow.
sceFont never looked at flash0:, so a firmware the user installed was
ignored and we always fell back to our bundled substitutes - despite
the "ignoring NAND" warning suggesting otherwise. It reads flash0:/font
now, after the game's own fonts and the classic ms0 override.
And if there's nothing in NAND, we unpack just flash0:/font out of the
firmware updater on the running disc, which most UMDs carry. That turns
"install a firmware first" into something that happens by itself for
anyone playing a retail game.
EmulatedModelGeneration moves from InstallUpdateScreen into PSARUnpack
so both callers pick the same firmware file list.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JvJR8oJNSCimCM9KXVLjfq
Three ways our rename differed from the PSP's:
- A wildcard in either path was passed through to the host, so
renaming "test*.txt" could quietly rename a real file. The PSP
doesn't expand them here, it rejects them outright.
- Renaming onto a file that already exists succeeded, because the host
rename() replaces the destination. The PSP refuses, and renaming a
file onto itself counts as that too.
- Crossing devices returned the right error, but after the same wait
as everything else. The hardware fails that one immediately.
Fixes io/file/rename, moved to tests_good.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JvJR8oJNSCimCM9KXVLjfq
GetFreeChannel counted down with an unsigned loop variable, so i >= 0
was always true. With every channel already reserved it wrapped past
zero and kept indexing g_audioChans until it walked off the end -
sceAudioChReserve(-1, ...) segfaulted the emulator instead of returning
"no channels available". Reproduces on audio/sceaudio/reserve, which
crashed before printing anything.
Also gives sceVaudioChReserve the parameter checks it never had. It
took any sample count, channel count and frequency; the hardware allows
256, 1024 or 2048 samples, stereo only, and the same sample rates the
SRC channel accepts. Every value in the test now matches - what's left
there is only reschedule markers.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JvJR8oJNSCimCM9KXVLjfq
Two things utility/systemparam caught:
A negative size passed to sceUtilityGetSystemParamString went through
Memory::IsValidRange, where it became an enormous range and came back
as a generic -1. The PSP just reports that the string doesn't fit, same
as any other size too small to hold it.
sceUtilityGetSystemParamInt returned 0x800ADF4 for an automatic adhoc
channel unconditionally. The FIXME there wondered whether the hardware
only does that once adhocctl is initialized - it does. Before any adhoc
module is up, which is the state nearly every game asks this in, the
hardware returns 0 and writes the channel out.
Fixes utility/systemparam/systemparam, moved to tests_good.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JvJR8oJNSCimCM9KXVLjfq
ReceiveMessage walks the packet list until it comes back around to the head.
Every pointer it follows is validated, so a list containing a loop that doesn't
include the head just keeps walking - the emulator hangs inside the syscall with
no way out. Bound the walk by the message count.
sceKernelCreateFpl and sceKernelCreateTlspl validate blockSize * count with the
formula from hardware, but that works in 4 byte alignment while the actual
allocation uses the alignment from the options struct, which the caller picks.
A large alignment inflates each block enough that the aligned total can wrap:
we'd then allocate a small block but keep the full block count, and
sceKernelAllocateTlspl / the Fpl allocate paths hand out
address + block * alignedSize well outside the allocation.
Also give PartitionMemoryBlock::address a default. Only the savestate
constructor leaves it unset, and DoState returns early when the section is
missing, which left the destructor freeing an uninitialized address.
Both wrote past the end of guest memory with game-supplied pointers:
- strcpy validated the destination with IsValidAddress, which covers a single
byte, then wrote strlen(src) + 1 bytes there.
- strcat validated both strings as they were, but the concatenation is longer
than either, and that's what gets written.
Check that the result actually fits, and copy with an explicit size instead of
the unbounded str* functions.
Also, snprintf with a negative size fell into result.resize(limit - 1), which
converts to a huge size_t and throws.
KernelImportModuleFuncs walks data straight out of the module being loaded, and
validated addresses without validating extents:
- The variable relocation list was scanned until it happened to hit a zero word,
with no bound at all - an unterminated list runs off the end of guest memory.
Now bounded by what's actually mapped from that address.
- nidData and varData were checked with IsValidAddress, which covers one word,
and then indexed numFuncs (up to 65535) and numVars times. Check the whole
array instead.
- The entry walk could read a final PspLibStubEntry that starts just short of
libstubend and extends past it.
All of these need a corrupt or crafted module, not a normal game.
Initialize the wokeThreads locals that were passed by reference uninitialized in
the event flag, VPL and semaphore timeout handlers. Harmless today since the
callee only ever assigns to them, but every other use in the same files starts
at false.
Do the semaphore overflow check in 64-bit, so a large signal value can't wrap
past it into currentCount.
sceKernelAllocHeapMemory and sceKernelAllocHeapMemoryWithOption return a
pointer, so failure has to be 0 - JPCSP documents it as "the address of the
allocated memory block, or NULL on error". We passed BlockAllocator::Alloc's
result straight out, which is (u32)-1 when the allocation fails, and returned
UID error codes for a bad heap id. Anything checking for a null pointer took
those for success.
AlarmIntrHandler::handleResult looked the alarm up again and passed the result
straight to __KernelScheduleAlarm without a null check - run() a few lines above
does check. An alarm handler that cancels its own alarm and then returns a
positive reschedule value dereferenced null.
Also, sceKernelSetSysClockAlarm validated four bytes and then read eight, and
returned a bare -1 instead of an error code.
CoreTiming::UnscheduleEvent returns the scheduled time minus the current time,
which is negative when the event is overdue but hasn't been processed yet - the
exact situation when a wait is satisfied right around its own timeout. Only the
semaphore clamped it; everywhere else we wrote (u32)cyclesToUs(negative) into
the game's timeout variable, i.e. a huge bogus "remaining time".
Pulled the shared shape into HLEKernel::WriteRemainingTimeout so it can't drift
apart again - event flags, mbx, fpl, vpl, msgpipe, mutex, lwmutex and semaphore
all go through it now. The two thread-end sites keep their own copy since they
unschedule even when the game passed no timeout pointer, and sceUsb just gets
the clamp.
The log was trimmed to 50 lines, which isn't enough to scroll back
through a conversation - now 250. Chat entries also carry the time they
arrived, shown as a dimmed HH:MM in front of the name, behind a new
"Show timestamps in chat" setting.
The timestamp is kept next to the text rather than baked into it, since
the chat view finds the sender by splitting the line at the first colon.
Fixes#15444
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JvJR8oJNSCimCM9KXVLjfq
The two long-standing bug reports had a shared root: the service loop could
end up in a state it never left.
- Exit hang: UPNP_CMD_EXIT was queued alongside port requests and only acted
on when it reached the front. A request that couldn't complete was never
popped, so exit sat behind it forever and join() blocked indefinitely.
Exit is a flag now, checked before anything else.
- CPU spin: wait_for() with a predicate returns immediately when the predicate
already holds, so a stuck queue head meant a tight loop. sceNetInet's bind()
queues UPnP_Add regardless of the setting, so this hit whenever UPnP was off
and a game used sockets. The loop always blocks now, and requests are dropped
while UPnP is off.
- Failed discovery was retried every 5s forever, each time a full 2s SSDP round
plus an error toast. Now backs off 5s -> 300s and reports once.
Other things found while in here:
- Every failed Initialize() leaked a UPNPUrls + IGDdatas, so ~every 5 seconds
for anyone with UPnP on and no router. The manual miniwget/parserootdesc/
GetUPNPUrls block was also redundant - UPNP_GetValidIGD does all of it and
memsets over the result, leaking the URLs and costing an extra HTTP round
trip per attempt.
- UPNP_GetValidIGD's status was never checked, so we could go DONE with no
usable IGD and hand a NULL controlURL to UPNP_GetConnectionTypeInfo.
- miniupnpc's strncpy into the port-mapping-entry buffers doesn't guarantee a
terminator; an 80-char description ran std::string off the end of desc[80].
- Add() marked another app's port "taken" only after our own add succeeded, so
a failed add left their mapping deleted and never restored.
- Clear() walked the router's entire table at exit, one HTTP round trip per
index. It now deletes only what we know we mapped, and the exit cleanup has
a time budget so an unreachable router can't stall shutdown.
- The in-flight request stayed in the queue during the router call, so a
same-port request arriving concurrently could erase it and be dropped
unexecuted.
- The queue is bounded, and last-write-wins per port collapses the churn from
games that rebind in a loop.
- The mapping description is built when the request is queued rather than read
off g_paramSFO from the UPnP thread later.
The thread now only exists while the setting is on - turning it off makes it
remove its mappings and exit, turning it on starts one. That means __UPnPInit()
has to run after the config is actually loaded; g_Config.Init() only builds a
lookup table. QueueRequest() reconciles too, so a per-game config or a libretro
core option enabling UPnP works without a notify at every call site.
The settings checkbox is disabled in-game, since sceNet latches related
settings at boot and a game that already mapped its ports wouldn't cope with
them disappearing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DCPmm7FoQUoqrbMdhfqhQ2
sceIoDevctl 0x01E18030 asks whether the disc's region matches the console's.
Unusually it answers through the return value rather than an output buffer -
1 matches, 0 doesn't - so leaving it unimplemented read as a mismatch, and the
VSH opened on "This disc cannot be started. The region code is not correct.",
asking 12 times. PPSSPP has no region-locked discs; anything it can load is
something it should run, so this always matches. The call now happens once.
Behind it is a second thing, not fixed here: the VSH believes a disc is inserted
at all because nothing in PPSSPP models an empty drive. sceUmd reports
PSP_UMD_PRESENT | PSP_UMD_READY unconditionally, and devctl 0x01F20001 always
answers "game disc". JPCSP answers "no disc" when no ISO is loaded, which is why
it never reaches the region question. Giving those two a notion of "no disc"
would be the real fix - low risk for games, which always have one, but it is on a
path every game uses, so it is written up in docs/VSHBootInvestigation.md rather
than done as a drive-by.
With this the shell reaches the interactive XMB: the error is dismissable with
circle and the menu behind it works. The per-frame display list stops settling
into one repeated frame and alternates between 45 and 48 stall points, which is
the headless-visible sign of a live menu rather than a static dialog.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
pspDecryptPRX() tried types 0, 1, 2, 5 and 6. flash0:/vsh/etc/index_XXg.dat -
the index of what the XMB shows, fetched through sceResmgr_9DC14891 - needs
type 9, so it failed and the shell had no menu to build.
Type 9 is type 6 with three differences, all following from a type 9 file
carrying a real ECDSA signature at 0x104..0x12C where a type 6 file has nothing:
- The "must be empty" header check stops at 0x104 instead of 0x10C. The index's
signature starts there, so 8 of its bytes were failing type 6's check - the
original failure.
- The signature is left out of the hashed header rather than fed into it. JPCSP
zeroes buf2[0x34..0x5C), which is that same range once its header
rearrangement is undone, so PRXType9 just leaves the field zero.
- ecdsa_hash in the KIRK CMD1 header stays 0. Type 6/7 set it, but the branch
type 9 takes writes only the mode word, and setting it made KIRK reject the
block.
Tried last in the chain: its header check is a subset of type 6's, so a genuine
type 6 PRX would pass it and then fail on the hash, and trying it earlier would
shadow the real answer. False positives are not really possible either way - the
SHA1 check inside has to match before anything is decrypted.
Verified end to end: 496 bytes in, 159 out (the comp_size in the header),
starting "release:". sceResmgr checks that prefix and says so in its log line,
since a wrong-but-plausible decrypt would otherwise look like success here and
fail much later as an unreadable index.
The VSH now draws something different - the per-frame display list settles at 24
stall points rather than 38 - but what it shows is not visually confirmed;
framebuffer readback doesn't work under headless on either backend.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Found the cause of the red error screen the VSH ends on. Every resource load in
the boot succeeds - fonts, all the plugin RCOs, topmenu_icon.rco - and then:
sceIoOpen(flash0:/vsh/etc/index_02g.dat) -> fd 8
sceIoRead(8, 092a2d40, 496)
sceIoClose(8)
unresolved import sceResmgr/9dc14891, called from 'vsh_module'
sceKernelExitDeleteThread(1)
index_02g.dat is the index of what the XMB displays, and it is encrypted (it
starts "PSPsysGP"). sceResmgr_9DC14891 decrypts it. There was no sceResmgr module
at all, so the call trapped, the index stayed encrypted, and the ScePafJob thread
building the top menu exited - a shell with everything loaded and nothing to show.
This adds the module and the three tags it needs (0x0B2B90F0/91F0/92F0, keys and
code 0x5C) to PrxDecrypter.
It is not the whole fix yet: pspDecryptPRX() tries decryption types 0, 1, 2, 5
and 6, and this needs type 9, which JPCSP passes explicitly. So the call is now
reached and fails cleanly with a logged error instead of trapping, but does not
yet decrypt. Type 9 is a variant of type 2 and is the next job; the notes in
docs/VSHBootInvestigation.md say where it is in JPCSP and how to check a port
(159 bytes out, starting "release:").
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Mostly small stubs:
- sceImpose: GetParam/SetParam/Changes/SetStatus, plus the 6.60 alias of
sceImposeGetBatteryIconStatus. Also fixes that function's first output - it
is a plain "is it charging" boolean, not a BATTICON_ value. We wrote
PSP_IMPOSE_BATTICON_NONE (0x80000000) there, which games ignore but which the
VSH reads as "no battery" and draws the empty-battery indicator for. These
are the bulk of the traffic: the VSH calls sceImposeChanges once a frame, so
this alone removes ~10000 trapped calls from a boot.
- SysMemForKernel: sceKernelSetRebootKernel, sceKernelSetUmdCacheOn.
- scePower_driver: scePowerSetWakeupCondition.
- sceHprm_driver, sceUsb: one NID-named call each, as in JPCSP.
Three groups are deliberately left unresolved, with comments explaining why,
because resolving them lets real flash0 drivers walk into hardware we do not
emulate and the boot dies where it used to reach the shell:
- ThreadManForKernel mutex/fpl NIDs: the NAND and ID storage drivers use these
to init, then poll the NAND controller at 0xbd101300 forever.
- InterruptManagerForKernel intr registration: 31 calls, then a stall in GE
list execution with no plugin module ever started.
73 unresolved import hits over 37 distinct module/NID pairs remain in a VSH
boot, mostly sceSysEventForKernel, sceSuspendForKernel and the various
*_driver modules that need real hardware behind them.
The sceImpose savestate section goes to v2 for the two new state variables.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
ModuleMgrForKernel/0xD5DDAB1F is how the VSH loads its own plugins. The XMB's
interface lives in flash0:/vsh/module/*_plugin.prx and vshmain pulls those in
through this kernel call rather than the user-mode sceKernelLoadModule, so the
existing note that vshmain never imports sceKernelLoadModule was true but
incomplete - it imports this instead, and it was unresolved.
The consequence was quiet: vshmain got no module id back and then called
sceKernelStartModule with id 0, which failed with UNKNOWN_MODULE. None of the
plugins that populate the XMB ever ran. The scene still had its containers,
which is why every frame set up render state per node and drew nothing inside
them - the "6x render-state-setup, 0 draws" symptom this investigation has been
chasing.
Also implements 0xD86DD11B sceKernelSearchModuleByName, the other unresolved
ModuleMgrForKernel import.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
For some reason, a pointer used to allocate the heap for scePaf is not
initialized. This hacks aroung that.
Additionally zero out the specific 4-byte "category 1 alarm count" address in vsh_module.
This gets us much further.
Extends LoadAndStartVshKernelModules() to load the 11 real kd/*.prx
kernel drivers for --vsh (dmacman, systimer, memlmd_01g,
loadexec_01g, lowio, idstorage, syscon, rtc, wlan, wlanfirm_01g, utility),
ahead of the existing 4 VSH-specific modules.
Only active when g_runningVSH, no effect on normal game boot.
Improve implementations of sceKernelSm1ReferOperations and sceKernelIsIntrContext.
Add some more MMIO stubs (GPIO, SYSCON).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
(cherry picked from commit 7f3168b7df85e47438900016c9ee7d7ef01a0a28)
Adds SceKernelLoadExecVSHParam and fills in the rest of LoadExecForKernel's
NIDs: real implementations for sceKernelExitVSHVSH/Kernel
(mirrors sceKernelExitGame) and sceKernelLoadExecBufferVSHUsbWlan (loads
an exec from an in-RAM buffer instead of a file, plus UNIMPL stubs.
sceKernelLoadExecBufferVSHUsbWlan needed __KernelLoadExec split into a
file-reading front end and a shared __KernelLoadExecFromPtr back end
that both it and the new buffer-based path call into.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSNaZnHCjmryS3ziVN9gZU
(cherry picked from commit 4adbedec9e2e3221113c9e5ad00b248bba1153e1)
Load and start VSH's kernel modules before booting vshmain.prx
A few flash0 modules (vshbridge.prx, paf.prx, common_gui.prx,
common_util.prx) should run for real once we know we're
actually booting the VSH rather than a game, since our fakes are unlikely
to be good substitutes for the genuine thing.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PSNaZnHCjmryS3ziVN9gZU
log.channel.set is meant for temporary, session-only diagnostic
tweaks (e.g. quieting a noisy channel while investigating something
over the WebSocket debugger). It was going through the same
SetLogLevel/SetEnabled calls the UI settings use, so a normal app
exit would persist whatever channels/levels the debugger last left
set, silently overwriting the user's real saved log preferences for
future runs - discovered when a later session's default logging
looked "off" for no apparent reason.
LogManager now tracks whether the debugger has touched channel
config this run and skips SaveConfig() entirely if so, leaving
whatever's already on disk untouched.
__StartLogAudio ignored WaveFileWriter::Start()'s return value and set m_logAudio
either way, so a wav file that couldn't be opened - read-only or full audio
directory, most likely - left every subsequent mixed block calling
AddStereoSamples on a closed file. That opens with _assert_msg_(file, ...).
Also other minor fixes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
__KernelLoadELFFromPtr creates its PSPModule and inserts it into loadedModules
before it knows whether the file is loadable, so every failure exit has to
delete the decrypt buffer, Cleanup() the module and Destroy() it. Five of the
seven did. The "unreasonable decrypted size" exit and the decompression-failure
exit just returned - leaking the buffer, and leaving a live kernel object with
its UID stuck in loadedModules for the rest of the session.
While tracing that: the fake-module path frees newptr and then runs for another
sixty lines with ptr still pointing into it. Nothing reads it today - the exits
below use head, which points into the original input rather than the copy - so
there's no use-after-free and no double free, but that's a property of the
current code rather than anything enforced. Both pointers are nulled after the
delete so a future mistake there crashes instead of reading freed heap.
And the function read the magic, and in the ~SCE branch a second word after it,
before anything established the input was that big. The non-PBP caller
guarantees it, but the PBP path computes elfSize from two offsets in the file
and passes whatever comes out, including zero. Checked at the top, before the
module object exists, so that exit needs no cleanup of its own.
pspautotests 314/314 with --graphics=software, and an EBOOT.PBP still boots.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Every decrypt type reads the tag at 0xD0, the compressed size at 0xB0 and key
data as far as 0x150, and writes a KIRK header into outbuf at a fixed offset
derived from sizeof(PSP_Header) - all without checking that either buffer is
that big. A PRX declaring a tiny psp_size therefore read past the end of its
input, wrote a 0xE0-byte header past the end of an equally tiny output buffer,
and handed KIRK "size - offset" as an unsigned underflow. The header write sits
behind the SHA-1 check, but the tag keys are compiled in and every hashed input
comes from the file, so that's arithmetic rather than luck. One size check at
the top of pspDecryptPRX covers all five types.
The module loader needed two things to go with it. Its "maybe it just isn't
encrypted" fallback checked for ELF magic at 0x150 of the *output* buffer, which
on the paths where decryption bails early has nothing written to it yet - so it
read uninitialized heap to decide, and then, if psp_size was under 0x150,
memcpy'd a negative length. It reads the input buffer now, which is what it goes
on to copy from anyway, and only when psp_size is big enough to hold what's
being tested.
Second, the returned size is just comp_size out of the file header, checked
against the allocated buffer by a _dbg_assert_ that isn't there in release. That
check is a real one now, folded into the existing sanity test next to it.
pspautotests cpu 11/11.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
This is stuff encountered in the VSH boot research.
sceRtc_driver, scePower_driver, sceImpose_driver, ThreadManForKernel funcs,
sceRtcGetAlarmTick, sceHprm_driver/sceHprmReadLatch
sceVshBridge_Driver imports sceKernelResumeDispatchThread, SuspendDispatchThread,
and NotifyCallback from ThreadManForKernel, but they were only registered under
ThreadManForUser. Added sceKernelGetUserLevel and sceKernelIsUserModeThread (new).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Two things kept the companion ELF from doing its job.
The first is the one that mattered: fileToStart is the game's own *directory*
for folder-launched homebrew (IdentifiedFileType::PSP_PBP_DIRECTORY), which is
the normal case when you pick a homebrew in the UI. The search navigated up from
it regardless, landing in PSP/GAME and listing sibling games - all directories,
all skipped - so app.elf sitting right next to the EBOOT was never found. It only
ever worked when the path pointed at the EBOOT itself, which is how headless is
invoked, which is why it looked fine from there. Searches the directory itself
now when that's what it's given.
The second: line info didn't survive loading a savestate. Modules aren't just
re-registered there, they're destroyed and rebuilt (KernelObjectPool::Clear), so
removing a module's lines in ~PSPModule threw the table away on every state
load. The previous commit worked around it by re-reading the companion, which
was both wasteful and no help at all to an ELF launched directly - those bytes
are long gone by then.
SymbolMap already solves this and line info now does it the same way: keep what
you have, and let whatever next claims the address range replace it. AddModule
replaces by key and is called for every module load, including ones with no line
info of their own, so a range gets retired when it's genuinely reused. The whole
table goes when the game does, in PSP_Shutdown. That also means the savestate
path has nothing to re-read, so state loads no longer pay to re-parse a
multi-megabyte ELF.
The tradeoff is a window between a module unloading and its range being reclaimed
where a lookup can still answer for it. For a debugger that's a stale file:line
on an address nothing owns, against certain and total loss on every state load.
Verified with --auto-save-load-symbols off, launched both ways: by directory
(the case that was broken) and by EBOOT path, both give 3734 symbols and 98383
line rows.
pspautotests 314/314, UnitTest 55/55.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Modules are torn down and re-registered from PSPModule::DoState when a state is
loaded, which takes their symbols and line info with them. That path read the
per-module .ppsym file and nothing else, so coming back from a savestate lost
everything the companion ELF had contributed - every function back to
z_un_<address>, and no line numbers at all - with no way to get it back short of
rebooting the game.
It reads the companion again now, on the same terms as the initial load:
unconditionally, since the file is still sitting next to the game and only the
.ppsym half was ever meant to depend on the setting.
Costs a re-read and re-parse of the companion on every state load for games that
have one (CrossCraft: 3734 symbols and 98383 line rows), which is the price of
not silently losing them.
Line info that came from the main ELF rather than a companion still doesn't
survive a state load - those bytes aren't around at that point. Nothing to do
about that here, and it doesn't apply to the EBOOT case, where the companion is
the only source anyway.
pspautotests 314/314, UnitTest 55/55.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
bAutoSaveLoadSymbols is about writing .ppsym files back out and reading them in
again. It had also come to gate reading debug info that's simply sitting next to
the game, which is a different thing and shouldn't need asking for: the main
ELF's own symbols were already loaded unconditionally, but the companion ELF's
symbols and all line info were not.
Now the ELF is always the baseline - main or companion, symbols and line info -
and the setting only adds the .ppsym half on top of it.
Line info also loads from the module being loaded, not just from a companion,
so an ELF launched directly brings its own. A PRX has no .debug section for it
to find (prxgen strips them), so that's a cheap no-op for the usual EBOOT case,
which the companion path still covers.
That second source needs the two shapes distinguished, so AddModule takes an
explicit address delta rather than assuming a base: a companion links at zero
and wants the module's base added, while an ELF loaded at the addresses it asked
for already has final ones (bRelocate is just e_type != ET_EXEC). Rows that
don't land inside the module after that are dropped either way, which is a
better check than the old "offset smaller than the module" one.
Splitting the companion's identity check out of the symbol loader lets line info
reuse it, and drops an accidental requirement along the way: it used to reject
any companion without a symbol table, so an ELF built with -g but stripped of
its symbols would have contributed no line numbers either.
Verified with --auto-save-load-symbols off: CrossCraft's companion app.elf loads
3734 symbols and 98383 line rows where it previously loaded neither.
The direct-ELF path is not verified at runtime - it needs a bootable ELF that
carries DWARF, and there isn't one to hand. Both candidates here (pspautotests'
.elf builds and CrossCraft's own app.elf) are linked at address 0 and fail to
boot on that alone, which is pre-existing loader behaviour and nothing to do
with this.
pspautotests 314/314, UnitTest 55/55.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Homebrew commonly ships its unstripped ELF next to the EBOOT, which is already
how the symbol loader turns z_un_08841f98 into a function name. That same ELF
carries a DWARF .debug_line section, so the addresses can be mapped to source
files and lines too - and a backtrace stops being four hex numbers:
08841f98 move sp,fp mesh.zig:163
0883afa4 li v0,0x0 MenuState.zig:821
088260d8 andi at,v0,0xFFFF State.zig:40
0882a27c andi at,v0,0xFFFF engine.zig:468
Surfaced in three places: per frame in hle.backtrace, in the "hit" object that
cpu.breakpoint.hit and cpu.stepping share, and appended to the disassembly
window's status bar. The breakpoint case keys on the pc rather than the address,
since for a memory breakpoint the useful source location is the instruction that
did the access, not the data it touched.
Storage is a plain sorted table of absolute addresses per module. SymbolMap
keeps module-relative addresses because its .ppsym files are meant to be
reloaded by a different game that pulls in the same module; none of this is ever
written anywhere - it's regenerated from the ELF each boot - so there'd be
nothing for relative addresses to buy. Each module owns its own rows and file
names outright and is keyed the way SymbolMap::UnloadModule is, so unloading one
module drops its lines and nobody else's.
The subtle part is end-of-sequence markers. Without them a lookup for an address
in a gap - a compilation unit built without debug info - confidently reports the
last line of an unrelated file. A prototype run over one test binary
mis-attributed 70 of its 349 functions that way, so sequence ends are recorded
as rows with line 0 and a lookup landing on one reports nothing instead.
DWARF 2 through 4 are decoded (psp-gcc emits 2, Zig 4). Version 5 re-encoded the
file table, so those units are skipped with a warning rather than mis-parsed -
nothing targeting the PSP produces it today.
Scope, since it's narrower than it sounds: PRX conversion strips every .debug
section. I checked all 437 pspautotests .prx and CrossCraft's own app.prx -
none have any. Of 24 installed homebrew EBOOTs, zero carry debug info; CrossCraft
only does because it ships app.elf separately. So this helps someone developing
homebrew, and does nothing at all for a commercial game.
Costs about 1.2 MB for a large Zig binary (98383 rows, 438 files) and nothing
for anything without debug info. Follows bAutoSaveLoadSymbols like the symbols
do.
pspautotests 314/314, UnitTest 55/55, CoreUWP builds.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Both mechanisms already existed and just weren't reachable from the WebSocket
API: PSP_CoreParameter().fastForward for unlimited, and an FPSLimit mode plus a
target frame rate for everything else. FrameTimingLimit() in sceDisplay.cpp is
where they all resolve to a single number.
The one thing worth being careful about is which knob to drive. Reusing
CUSTOM1/CUSTOM2 - the user's own alternative speeds - would have meant writing
g_Config.iFpsLimit1, which is persisted per game, so a debugger session would
permanently overwrite whatever speed the user had configured. Same class of
mistake as a debug setting leaking into the saved config. So this gets its own
FPSLimit::DEBUGGER mode and a debuggerFpsLimit field on CoreParameter, which
isn't persisted and is value-initialized on every boot. Two things fall out for
free: the analog-speed handler already backs off for any mode it doesn't own
(EmuScreen.cpp), and a debugger can't leave a game slowed down after a restart.
Percentages are relative to 60 FPS, the same convention GameSettingsScreen uses
when presenting the alternative speeds, so "200%" means one thing across the
app. Unlimited is fastForward rather than percent 0, so there's a single way to
express it.
The response reports limitFps straight from FrameTimingLimit(), exposed for the
purpose. That's the number the frame timing actually consumes, so a client never
has to reconstruct the interaction between fast-forward, this override and the
user's own hotkeys - which is exactly the sort of thing that goes stale.
Requests fail rather than being quietly ignored when something else owns the
speed: achievements hardcore mode, or netplay without the "allow speed control
while connected" option. Being ignored with a successful response is the worst
outcome for an automation client.
Verified against a running headless instance: percent 200 with fast-forward off
resolves to limitFps 120, fast-forward takes it to 0 while remembering the 200
underneath, an explicit null clears it, and out-of-range or empty requests are
rejected. The throttle *behaviour* is not verified end to end - a debug,
software-rendered headless build runs this game at about 1% of real time, so it
never reaches any of these targets and the limit can't be observed. That path is
shared with the existing CUSTOM1/CUSTOM2 speeds and unchanged.
sceNet.h is forward-declared rather than included: it reaches windows.h through
proAdhoc.h, which redefines the OPTIONAL macro that collides with
DebuggerParamType::OPTIONAL - the hazard this file's header comment already
warns about.
pspautotests 314/314, UnitTest 55/55.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Homebrew almost always ships the ELF it was built from alongside the EBOOT -
app.elf next to app.prx - but prxgen strips the symbol table on the way to the
PRX, so the module PPSSPP loads has no names at all and MIPSAnalyst calls every
function it finds z_un_<address>. Working out what any of them are meant hand-
parsing that ELF with a throwaway script, which is how the CrossCraft
relocation bug got identified.
So read it directly. On module load, scan the game's own directory for an ELF
with a symbol table and add its STT_FUNC/STT_OBJECT entries at the module's
base. CrossCraft picks up 3734 symbols, and the disassembly turns from
z_un_088c00f0 into world.init_empty, with static_allocator.alloc at the vtable
entry it calls - the two functions that took the longest to identify by hand.
Matching is the part worth getting right, since a wrong match puts confident
nonsense at real addresses, which beats having no names only in the sense that
it's worse. A candidate has to be a 32-bit ELF with a symbol table whose
highest section ends within a page of the loaded module's size - the companion
links at base 0 and covers the same image, so that's a tight check, and
unrelated ELFs sitting in the same folder fail it. Symbols outside the module
are skipped individually too.
Names go in with updateName, so they win over the analyzer's placeholders
rather than losing to whichever got there first. Gated on the existing
bAutoSaveLoadSymbols setting (off by default), which already means "keep symbol
names around for me" and avoids a directory scan per module load otherwise.
pspautotests 314/314.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
These KernelObject subclasses (and their Native* status structs) were
private implementation details of their respective .cpp files. Move them
into the matching .h instead, so external code - specifically the upcoming
WebSocket kernel-object introspection endpoints - can read a live object's
state directly via kernelObjects.Get<T>()/Iterate<T>(), the same way
PSPModule/PSPThread already can. Read-only by convention: nothing outside
each file should call DoState() or otherwise mutate these; the fields are
public here for that file's own pre-existing use, not an invitation to
write from elsewhere.
To avoid pulling each type's full dependency set (Memory::, BlockAllocator,
CoreTiming, HLEKernel::...) into headers many other files include, non-trivial
method bodies (DoState, and MsgPipe's buffer/wait-list management) are
declared in the header but still defined out-of-line in the .cpp, same as
before - only genuinely trivial one-liners went inline.
KernelObjectPool also gains IterateAll(), a type-agnostic sibling of the
existing Iterate<T>() - walks every live kernel object regardless of type,
for a coarse "what's alive right now" overview.
No behavior change - this is a pure visibility/declaration-vs-definition
move, not new functionality. That lands in a follow-up commit.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
SymbolMap:
- Fix GetModuleIndex(): it only checked the end of an active module's range
(via activeModuleEnds.upper_bound), never the start, so an address sitting
in the gap before a module was silently misattributed to it. Added
GetModuleIndexByName() as a companion lookup.
- AddModule() gains an optional crc param, stored per ModuleEntry. Reactivating
a module by name now also requires the crc to agree when both sides know it,
so two unrelated binaries that happen to share a name no longer get merged
into one symbol table (addresses the old TODO at the top of SymbolMap.h).
- AddLabel()/AddFunction() gain an updateName param (default false, preserving
existing "first writer wins" behavior) so a trusted source - like a loaded
symbol file - can be allowed to overwrite a name that a lower-confidence
automatic pass already assigned.
- New SaveModuleSymbols()/LoadModuleSymbols()/GetModuleSymbolsPath(): save or
restore one module's functions/data/labels to/from a small human-editable
text file, addressed relative to the module (so the file stays valid however
the module ends up positioned on a later run). Keyed by
PSP/SYSTEM/SYMBOLS/<moduleName>_<crc>.ppsym - deliberately by module+crc
rather than by game, so it's shared by every game/homebrew that loads the
exact same module. A "# game <id> <title>" comment records who last saved
it, informational only.
WebSocket debugger: hle.module.saveSymbols/loadSymbols expose the above.
sceKernelModule.cpp: auto-load a module's saved symbols right after it's
registered with the symbol map (both the real ELF-load path and the
savestate-load path), and auto-save on unload (before UnloadModule(), while
its symbols are still active) - gated behind the new bAutoSaveLoadSymbols
config setting (default off), with a matching Developer Tools checkbox and
a --auto-save-load-symbols command-line override for headless use.
Includes some in-progress cleanup already staged: DescribeAddress now calls
g_symbolMap->GetDescription() directly instead of through the now-removed
MIPSDebugInterface::getDescription() wrapper.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GZq8ZtJmFY7bkX5FVkr3P9
Henrik Rydgård