Another pass looking for gaps, all of it now recorded by audio/blocking/channels
and audio/blocking/oneshot.
sceAudioOneshotOutput was the last unimplemented entry in the module, returning
"library not linked" to anyone who called it. It plays one buffer on a channel it
never reserves, so the channel frees itself when the buffer runs out, and its
argument checks are their own set: any positive sample count, aligned or not, no
upper bound, a negative volume rejected rather than skipped, and no busy check at
all. No game is known to use it; it is implemented because tracing it turned out
to be cheap, not because anything needed it.
The channels test confirms four behaviors: sceAudioChReserve(-1) skips a released
channel that is still playing while a reserve of it succeeds, a second channel
joining a running mixer doesn't lose a block unlike a first, mono counts down in
the same 64-sample steps over the same time as stereo, and the panned blocking
output has no extra delay on the high channels.
Also: the SRC resampler now interpolates into the next buffer at a buffer join
instead of holding the last sample, since the codec reads the two descriptors as
one stream. That only shows up at non-native rates and no test can see it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two things the last pass looked at and left, now measured on hardware by the new
audio/blocking/vaudio and audio/blocking/overhead.
sceVaudioChRelease is not shaped like the Output2 and SRC releases. It hands the
channel a null pointer first, which waits for a buffer to finish, so it blocks for
one buffer and returns 0 where the others would refuse - and the buffer is played
out instead of being dropped, which is what we were doing. The reservation goes
now rather than when the drain finishes: the caller is parked either way and gets
the same answer at the same time, and the buffers keep playing because the mixer
does not look at the reservation.
The same test turned up two more: a *failed* sceVaudioChReserve still marks vaudio
reserved, so a caller that lost the channel to Output2 is told 0x80000021 next
time until a release clears it; and sceVaudioChRelease ignores that flag entirely
and releases whatever holds the SRC channel, which pspautotests already called the
"wrong release".
The flat 10000 cycles charged to every Output2 and SRC output turns out to be
right only for the refused case. Every other outcome ends up querying the codec
and costs over 100us on hardware, including finding the channel unreserved, so
those now cost 25000. Mixer channels are the other way round - cheap to refuse,
expensive on the one output that starts the DMA - so that charge moved to the DMA
start. F1 2009 behaves identically and Burnout Dominator's mixed output is
bit-identical to before.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
I deduced that this was the case, and attempted implementing this path long ago,
but I could never quite get it to work in all games. Set Claude on a quest to
research and implement it, and lo and behold, it works. A bit sobering.
Fixes#12888 and likely more. Additionally, audio latency is likely slightly
improved overall, and memory usage is down by 4.6MB.
Claude says:
The blocking output calls are not a queue that callers line up behind. Each
mixer channel holds exactly one buffer and at most one parked thread; a second
thread arriving while the first is waiting is told the channel is busy and is
expected to skip its turn. The Output2/SRC channel holds two DMA descriptors and
refuses a third caller outright, without waiting at all.
We blocked everyone instead, so a game running a movie thread and a sound-effect
thread over one output made the two alternate - a frame of movie audio, a frame
of effects silence - and the movie played at half rate. That is #12888, seen in
F1 2009 and Colin McRae: DiRT 2. With this, the movie thread keeps the channel
for the whole cutscene and the effects thread is refused, which is what the
hardware trace shows.
The driver also never copies a buffer on the way in: it stores the pointer and
its mixer walks it forward 64 samples at a time out of the game's own memory.
Modelling that fixes#20095 as a side effect, and drops the 4.6MB of per-channel
sample rings we were carrying. The mix event is re-phased to the moment a DMA
starts, since the mixer thread outranks its caller and gets a block in before the
output call returns.
Along the way: the two rest-length calls differ after a null-pointer output,
sceAudioChRelease reports not-reserved rather than not-init,
sceAudioChangeChannelConfig validates the format,
sceAudioChangeChannelVolume validates nothing, and sceAudioOutput2ChangeLength
takes a range of 17..4111. Details in docs/sceAudio.md.
Savestates: AudioChannel goes to version 4. Older ones stored mixed samples that
can't become a pointer and a position again, so they load with the pending audio
dropped and any parked threads released.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Cashing in the io/stat and io/shortname recordings.
__IoGetStat began with memset(stat, 0xfe, sizeof(SceIoStat)), which destroyed 24 bytes of the
caller's buffer that a real PSP never touches - it writes only as far as the timestamps and
leaves all six st_private words exactly as it found them. It also wrote a made-up sector number
into st_private[0] on the memory stick. That word carries the LBN on a UMD, which games read to
build disc0:/sce_lbn paths, so it stays for non-FAT and is left alone otherwise.
FAT has no permissions of its own and everything reads back as 0777. We were passing the host's
idea of the file through instead. The existing "all files look executable on FAT" hack for Beats
(issue #14812) was right in substance but lived only in sceIoDread, so sceIoGetstat and
sceIoDread disagreed about the same file where hardware has them agree. Both now go through one
path, which also gets the read-only case right: no write bits means mode 0555 and attr 0x21.
sceIoGetstat on the root of a volume is refused, as on hardware.
sceIoChstat was a logging stub. It now applies the read-only flag, which is what st_mode's write
bits and st_attr's 0x01 both mean on FAT - setting either produces both, and it's reversible.
That needs a new IFileSystem::SetFileWritable, defaulting to "can't" so read-only filesystems and
hosts that can't express it (Android content URIs) are unaffected; the call still succeeds there,
since hardware would have.
GenerateFatShortNames now accounts for capitalisation. FAT keeps a lowercase flag for the base and
another for the extension, but the PSP only honours the base one, so "shrt" becomes SHRT while
"readme.txt" becomes README~1.TXT. We were only adding a counter on collision. The unit test
carries the whole recorded set, including the corrected README~1.MD.
io/shortname stays in tests_next: its d_name column can't match while SimulateVFATBug is
uppercasing lowercase 8.3 names, which is deliberate and load-bearing for homebrew.
The new sysmem tests run the same partition sweep from both privilege levels, which settles
several things that were guesses:
The valid range is 1-6, not 1-9-except-7. sceKernelCreateVpl, CreateFpl, CreateMsgPipe and
AllocPartitionMemory all let 8 and 9 through to the permission check, so a caller asking for
partition 8 got ILLEGAL_PERM where hardware says ILLEGAL_ARGUMENT. Privilege changes the
permission check, not the range - 1, 3 and 4 are refused from user mode and work from kernel mode
in every one of these APIs, which is the evidence the earlier BlockAllocatorFromID change was
missing.
sceKernelAllocPartitionMemory reports an out-of-range partition differently depending on which
entry point was used - ILLEGAL_ARGUMENT through SysMemUserForUser, ILLEGAL_PARTITION through
SysMemForKernel. Both NIDs land on the same function here, and hleIsKernelMode() is precisely
"came in through the kernel NID", so it picks the right one.
sceKernelCreateHeap had four "TODO: Validate error code" comments and no test at all - it's
kernel-only, which is why. All four are now recorded: out-of-range partitions are
ILLEGAL_PARTITION, a size of zero or less is HEAPBLOCK_ALLOC_FAILED before anything is allocated,
a NULL name is refused with ERROR, and flags really are ignored. sceKernelAllocHeapMemoryWithOption
had its validation backwards: the option struct's size field isn't checked at all, while the
alignment must be a power of two from 4 to 0x80.
Not fixed, and split into sysmem/kernel/heapgrow in tests_next: a real heap will hand out a block
larger than the heap itself, so the size isn't a cap. Ours is a fixed allocator over the reserved
block. Worth establishing how far the real one grows before implementing that.
Risk: the range change makes partitions 8 and 9 fail earlier and with a different code than
before. Nothing in tests_good depended on the old behaviour except two expectations that had
drifted from hardware, corrected in the submodule.
PPSSPP decided whether a caller was privileged with hleIsKernelMode(), which reports whether the
syscall being executed is itself a kernel-only export. That's a different question from the one
the hardware answers: on a PSP the privilege belongs to the calling module, and a kernel module
reaches sceKernelCreateTlspl through the ordinary ThreadManForUser NID like anything else. So a
kernel module asking for partition 1, 3 or 4 got ILLEGAL_PERM where a real PSP hands it over,
which the new threads/tls/kernel/partition test shows directly.
BlockAllocatorFromID now also accepts a caller whose thread belongs to a kernel module, via a new
__KernelCurThreadIsKernelMode(). It checks the thread's own attribute first and then the owning
module, because a kernel module's main thread isn't necessarily flagged kernel - the attribute
comes from PSP_MAIN_THREAD_ATTR, which needn't set it. That mirrors how sceKernelCreateThread
already works out allowKernel.
This only ever widens access, and only for threads belonging to kernel modules, so games are
unaffected - they run in user modules and see exactly what they saw before.
sceKernelUtilsSha1Block* had the same single global context that MD5 did, so it gets the same
treatment: state, counters and block buffer now live at ctxAddr in the layout hash/sha1ctx
records off hardware. Unlike MD5, SHA-1 does not stream whole blocks through buf, which happens
to be what our sha1_update already does - so no fill-in step is needed there.
The Tlspl partition range from the last commit was too broad a cut. Hardware says only 1-6 exist,
but that recording is from user mode, and BlockAllocatorFromID deliberately maps 8 and 10 to the
user partition for a kernel-mode caller - rejecting them outright would have taken that away.
The tightened range now applies to user mode only and kernel mode keeps what it had.
threads/tls/partition also shows the answer doesn't depend on the compiled SDK version, checked
across 1.00 through 6.06, and that partition 5 is accepted - which no test had covered.
Three fixes, all of them things the new hardware tests turned up.
sceMd5Block* and sceKernelUtilsMd5Block* shared one static md5_context and ignored the context
pointer the caller passed in, with a TODO saying it would do "unless games do several MD5
concurrently". hash/md5ctx shows a real PSP keeps everything in the caller's 96 bytes and happily
runs two digests at once, so do that instead: the state, the counters and the block buffer now
live at ctxAddr in the game's own memory, in the layout the test pins down. Two interleaved
digests come out right, and a context that gets copied mid-digest carries on correctly. As a
side effect the state is now covered by savestates, which a file-static never was.
MersenneTwister masked both halves with 0x80000000 where the low half needs 0x7FFFFFFF, so
sceMt19937UInt and sceKernelUtilsMt19937UInt were returning a sequence that isn't MT19937 at
all - every number differed from hardware from the first draw. hash/mt19937ctx computes the
reference sequence itself and confirms the PSP is plain MT19937; with the mask fixed we match it
for both seeds tested. Init also twists the array immediately, as hardware does, so a context
that has been seeded but not drawn from now holds what a real one would.
sceKernelCreateTlspl accepted partitions up to 9 before falling through to the permission check.
Hardware draws the line at 6 - threads/tls/create records 7, 8, 9 and 10 all returning
ILLEGAL_ARGUMENT - so 8 and 9 were coming back ILLEGAL_PERM. Note this is genuinely different
from sceKernelCreateVpl right above it, which does let 8 and 9 through to ILLEGAL_PERM; the two
had been sharing a check that was only ever right for Vpl.
Risk worth naming: the MT19937 change alters the numbers any game gets from these calls. That's
the point - they were wrong - but a savestate taken mid-sequence will resume with a generator
that behaves differently from the one that made it.
We had a doc for running the existing tests against headless, but nothing on
the other half - bringing up PSPLink and usbhostfs_pc, what gentest.py does,
and how to get an .expected out of real hardware. Write that down, including
the parts that cost time to rediscover: usbhostfs_pc's working directory is
host0:/ so it has to start in the pspautotests root, gentest.py makes the
whole test directory and several old tests no longer build under pspdev's
GCC 15 (use -k), rebuilding a .prx with a newer toolchain balloons it, and
host0: is not FAT so anything testing FAT semantics needs ms0:.
Also adds the io/shortname test the doc uses as its worked example. It stays
in tests_next: hardware preserves the case of d_name where we uppercase it,
and appends ~1 to the short name of anything that isn't already valid
uppercase 8.3 where we only do that on a collision.
threads/tls/create moves to tests_next as well. It's collateral from the
submodule bump - upstream 1dcefeb regenerated its .expected on a PSP with
less free memory, so allocations at 1MB and above now expect failure, and
partitions 8 and 9 now expect 800200D2 where we return 800200D1.
SetFontPixel refused to write any pixel whose x fell outside
bytesPerLine, so a glyph drawn into a buffer with a bytesPerLine
narrower than its rows came out mostly blank. The hardware doesn't
bound it that way - it works out an address and writes, so the rows
overlap and the glyph smears across them. The declared bufWidth and
bufHeight, plus the address check, are what keep it in bounds.
Cache invalidation now covers the wider of bytesPerLine * bufHeight and
where the last row actually ends, since those are no longer the same
thing when the rows overlap.
Fixes font/charglyphimage and font/charglyphimageclip, moved from
tests_next to tests_good. The other font tests are unaffected, so
whatever ails them is something else.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JvJR8oJNSCimCM9KXVLjfq
A game can notify more data than the file actually had - audio/mp3/stream
asks for 3360 bytes and notifies all of them even when the read came up
short - so the tail of the buffer holds stale bytes from the previous half.
We happily decoded those, six frames past the end of the stream, because the
end flag only suppressed the zero fill and never stopped the decoder.
Check it before decoding too. The post-decode check stays where it was: the
hardware rewinds in the same call that decodes the last frame, so the sum
reads back as zero right after it, which is what audio/mp3/getsumdecoded
records. Moving the whole thing up front breaks that test.
Fixes audio/mp3/stream, added to tests_good - it walks 27 refills end to end,
so it also covers the half-buffer handout.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The area after the 0x5c0 workarea is double buffered - a half only becomes
writable again once the decoder has consumed past its end, so decoding a
single frame usually frees nothing at all. We instead reported every byte a
decode had just consumed, which made sceMp3CheckStreamDataNeeded() answer
"yes" after every single frame.
Beats sleeps 50ms whenever that call says the file thread is behind, so it
slept once per decoded frame and delivered audio at 46% of realtime - the
badly stuttering custom soundtracks. It now decodes 3-4 frames per 3360 byte
refill, with the write pointer alternating between the two halves exactly as
audio/mp3/stream records from hardware, and keeps up.
AuGetInfoToAddStreamData/AuNotifyAddStreamData now derive the write position
from how much has been added rather than from how much is still buffered,
since the write pointer walks the halves in turn and doesn't follow the
decoder.
Fixes audio/mp3/notifyadd, moved to tests_good, and the "after decode" case
in audio/mp3/checkneeded.
sceMp3: note that the half-buffer split is only verified at 8192 bytes
sceMp3GetInfoToAddStreamData always handed back the start of the work
area, so the pointer never moved as data was added - the hardware walks
it forward past what's already buffered. AuNotifyAddStreamData now
takes the new bytes from where the game was actually told to write, and
checks that range fits the buffer rather than just comparing the size.
Also compare readPos against endPos as signed. readPos is an int and a
game can notify a negative size, which made it promote to a huge u64
and look like the end of the stream, so we reported nothing left to
write where the hardware still wanted 6721 bytes.
Fixes audio/mp3/infotoadd, moved to tests_good. audio/mp3/notifyadd
gets both of its value differences fixed but still fails: after a
decode the hardware reports no space at all, while we free what the
decode consumed, so we do one round more than it does.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JvJR8oJNSCimCM9KXVLjfq
DirectoryFileSystem passed the host's permission bits and directory
size straight through to the game. The PSP has neither - its FAT driver
makes a mode up from the entry type and whether it's writable, and
reports no size for a directory. So a game saw 0644/0755 and a 4096
byte directory on Linux, but 0664/0777 and 0 on Windows, where the file
layer already synthesizes those bits.
Now both platforms report what the PSP does: 0777 for directories, 0664
for writable files, 0444 for read-only ones, and no size on a
directory. The parent ".." entry keeps its 4096, which is what the
hardware reports for that one.
Also fixes the synthetic PSP directory entry using 0x777 where 0777 was
meant.
Fixes io/directory/directory, moved to tests_good.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three ways our rename differed from the PSP's:
- A wildcard in either path was passed through to the host, so
renaming "test*.txt" could quietly rename a real file. The PSP
doesn't expand them here, it rejects them outright.
- Renaming onto a file that already exists succeeded, because the host
rename() replaces the destination. The PSP refuses, and renaming a
file onto itself counts as that too.
- Crossing devices returned the right error, but after the same wait
as everything else. The hardware fails that one immediately.
Fixes io/file/rename, moved to tests_good.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JvJR8oJNSCimCM9KXVLjfq
Two things utility/systemparam caught:
A negative size passed to sceUtilityGetSystemParamString went through
Memory::IsValidRange, where it became an enormous range and came back
as a generic -1. The PSP just reports that the string doesn't fit, same
as any other size too small to hold it.
sceUtilityGetSystemParamInt returned 0x800ADF4 for an automatic adhoc
channel unconditionally. The FIXME there wondered whether the hardware
only does that once adhocctl is initialized - it does. Before any adhoc
module is up, which is the state nearly every game asks this in, the
hardware returns 0 and writes the channel out.
Fixes utility/systemparam/systemparam, moved to tests_good.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JvJR8oJNSCimCM9KXVLjfq
Inverts/renames the setting to allow going back to the old sceAtrac
implementation, to work around any compatibility issue.
Note that we can never delete the old implementation, old savestates
will still use it - can't convert an existing session.
