The core and the headless tools no longer use mutexes, condition variables,
spinlocks, std::atomic, std::thread or volatile flags. Threads hand off through
libretro-common's retro_atomic and rthreads plus a few small pieces on top of
them in Common/Thread: Thread, MpscQueue, EventCounter, ParkingLot waits and
ReaderGate. Every wait blocks until it's woken; nothing polls on a timer.
State that only one thread touches just loses its lock. The PSP filesystem,
sceIo and the savedata dialog now belong to the emulation thread: async reads
and writes run when they're queued and complete later in emulated time, and
the IO and savedata threads are gone. The friend finder only talks to the
adhoc server and hands what it gets to the emulation thread. GPU init always
runs on the thread that starts the game.
The Vulkan render and compile queues, pipeline and render pass caches, delete
list and SPIR-V cache, the GL render queue, the software renderer's jit caches,
the file loader caches, the web debugger, UPnP, input, savestates, logging and
the GE debugger all hand off through queues and atomics. The IR jit's spinning
profiler thread goes.
A CPU thread with nothing to run now sleeps until work is queued instead of
spinning. Headless with --debugger waits for a client while stopped, and ends
the run once every client that connected has left. Web server shutdown keeps
servicing the CPU queue until the server has finished.
Socket waits now block in select() on the socket plus a wake or cancel
socket (Common/Net/Cancel), so a stop or cancel ends them at once.
- HTTP: cancel flags become net::CancelToken; Connect waits once on all
addresses, receives wait instead of spinning, no DNS retry sleep.
- Adhoc: the friend finder and adhoc server block on their sockets with
a wake socket and a deadline; matching runs on a CoreTiming tick
instead of two threads (savestate section bumped).
- sceNetInet select/poll check without blocking and park the HLE thread,
re-checked from a CoreTiming event until ready or the emulated
timeout.
- Vendor the aemu_postoffice client (was a submodule) with no sleeps:
waits on the socket, close via shutdown and an eventcount, cancelable
connects, and lock-free session slot claims.
- PortManager waits without a timeout; PSP_InitWait replaces the
headless boot poll.
- TimeUtil: sleep_us/sleep_ms only outside the core; sleep_precise,
sleep_random, yield and the Windows timer period are gone.
- Unit test for the socket waits.
Join() deliberately leaks the naett objects of a request still running at
shutdown and parks its sink, so a late chunk lands somewhere valid. But
HTTPSShutdown, which runs right after CancelAll, deleted those sinks, and
the POST body naett reads in place was owned by the request being
destroyed. On Android, where NativeShutdown isn't process exit, a late
WinHTTP/JNI callback could then write to or read from freed memory.
Only free the list when it's empty (that still fixes the leak the delete
was added for), and keep the POST body in the sink.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
'PPSSPPDebug64.exe' (Win32): Unloaded 'C:\Windows\System32\mfreadwrite.dll'
The thread 'RecentISOThreadFunc' (9920) has exited with code 0 (0x0).
The thread 'Console' (10488) has exited with code 0 (0x0).
Detected memory leaks!
Dumping objects ->
{17571338} normal block at 0x00000214CC4A3FE0, 16 bytes long.
Data: < Cf > E8 43 66 CD 14 02 00 00 00 00 00 00 00 00 00 00
{17571337} normal block at 0x00000214CC4A3B80, 16 bytes long.
Data: < Cf > C8 43 66 CD 14 02 00 00 00 00 00 00 00 00 00 00
D:\project\memory_leak\ppsspp\Windows\Debugger\Debugger_Disasm.cpp(174) : {17571336} normal block at 0x00000214CD664190, 640 bytes long.
Data: < C > 90 43 F2 C0 F6 7F 00 00 F6 0C 04 00 00 00 00 00
D:\project\memory_leak\ppsspp\UI\NativeApp.cpp(879) : {84582} normal block at 0x00000214CE8C5DB0, 4224 bytes long.
Data: < > 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
{2432} normal block at 0x00000214D6255C20, 16 bytes long.
Data: <0 $ > 30 E7 24 D6 14 02 00 00 00 00 00 00 00 00 00 00
D:\project\memory_leak\ppsspp\Common\Net\HTTPNaettRequest.cpp(43) : {2431} normal block at 0x00000214D624E730, 32 bytes long.
Data: < \% > 20 5C 25 D6 14 02 00 00 00 00 00 00 00 00 00 00
Object dump complete.
The thread 22000 has exited with code 0 (0x0).
The thread 21248 has exited with code 0 (0x0).
The thread 38104 has exited with code 0 (0x0).
The thread 27860 has exited with code 0 (0x0).
The thread 33240 has exited with code 0 (0x0).
The thread 29352 has exited with code 0 (0x0).
The thread 13964 has exited with code 0 (0x0).
The thread 5640 has exited with code 0 (0x0).
The thread 10528 has exited with code 0 (0x0).
The thread 15252 has exited with code 0 (0x0).
The thread 23016 has exited with code 0 (0x0).
The thread 31424 has exited with code 0 (0x0).
The thread 7000 has exited with code 0 (0x0).
The thread 11620 has exited with code 0 (0x0).
The thread 36264 has exited with code 0 (0x0).
The thread 27248 has exited with code 0 (0x0).
The thread 24780 has exited with code 0 (0x0).
The thread 26228 has exited with code 0 (0x0).
The thread 13676 has exited with code 0 (0x0).
The thread 16828 has exited with code 0 (0x0).
The thread 27388 has exited with code 0 (0x0).
The thread 1668 has exited with code 0 (0x0).
The thread 37272 has exited with code 0 (0x0).
The program '[23456] PPSSPPDebug64.exe' has exited with code 0 (0x0).
unique_ptr, not shared_ptr. Nothing ever shares it: naett holds a raw pointer
and takes no part in the counting, and the abandonment path hands the sink over
and lets go in the same breath. What's really going on is a single owner that
moves, which is what unique_ptr says and shared_ptr left you to work out from
reading all the uses.
naett never frees the pointer it's given for the writer - it only reads
bodyWriterData and hands it back to the callback - so deleting the sink is
ours to do. The things naett does allocate, the request and response, stay raw
pointers with explicit naettFree/naettClose.
The length field was just buffer.size() written down twice.
Found reviewing the commits before this one. On Windows a short write marked the
request complete and then queued another read regardless, which is worse than it
sounds: the caller is free to close a response the moment it sees complete, so
WinHTTP carried on writing into memory that was being freed. It also meant
cancelling didn't actually stop anything there. Stop at that point, and ignore
anything raised after the request is complete.
The Apple callbacks do the same check, so the cancellation we ask for after a
failed write doesn't overwrite the error that caused it.
naettMake only asserted that its request was non-NULL, and the request
constructors do return NULL when a platform can't set one up - an unparseable
URL is enough on Windows. Asserts are compiled out in release, so that was a
null dereference. It returns NULL now, and HTTPSRequest checks for it and fails
the request instead of handing it on.
Also: a request cancelled between construction and Start() didn't carry that
into the sink, and -1000 - our own cancellation code, not naett's - was logging
"Unhandled naett error" on a perfectly normal cancel.
Cancel() worked on the plain HTTP path - cancelled_ is threaded down as
progress->cancelled and checked while connecting and reading - but on the naett
path it only set a flag that nothing looked at. The transfer ran to completion
and cancelling just relabelled the result afterwards. Cancelling a store icon
that scrolled away, or a homebrew download the user gave up on, kept using the
bandwidth either way.
naett has one hook for this: a body writer that takes less than it was given
fails the request. So HTTPSRequest installs its own writer, which refuses
everything once cancelled. That works on all four backends and needs nothing
from naettClose, which is only really safe on Android.
The catch is lifetime. The writer runs on naett's transfer thread, and a request
that's still going when we're torn down would then be writing into a destroyed
HTTPSRequest - which is why the buffer it writes into is a separate refcounted
sink rather than a member. Join() on an unfinished request parks the sink where
it won't be freed and lets the request go, so a chunk that lands afterwards
writes somewhere that still exists.
That path is shutdown-only: RequestManager only cancels from its destructor, and
Update() waits for Done() before joining. Join now also notices a request that
finished while nobody was polling, and closes it properly instead of abandoning
it. What's left leaking at that point is a request still in flight as the
process exits, which is what already happened, just deliberate now and logged as
such rather than as an error.
naett has been a submodule pinned at v0.3.3; upstream has had no commits since
April 2024, and we want to carry local changes (next up: a libcurl-backed HTTPS
path for Linux). It's ~1500 lines of MIT C, smaller than several things we
already vendor, so bring it in-tree and drop the submodule.
Also drop the generated single-file amalgam (naett.c) that every build system
was compiling, and build src/*.c directly instead - otherwise the file you edit
isn't the file that gets compiled, which is a trap for anyone patching this.
example/ and testrig/ (a whole Android Studio project) are gone with it.
Two changes were needed to make the sources build on their own, both noted in
ext/naett/README-ppsspp.md along with the upstream commit:
- naett_internal.h now includes naett.h, which the amalgam pulled in first.
- naett_linux.c now includes stdio.h/stdlib.h. It calls exit/calloc/realloc/
free/fprintf without ever including either, and only got away with it because
naett_core.c sat above it in the concatenation.
No functional change - Linux still has HTTPS_NOT_AVAILABLE set, so it doesn't
build naett at all yet.
Rename naett to naett-lib
* Rename LogType to Log
* Explicitly use the Log:: enum when logging. Allows for autocomplete when editing.
* Mac/ARM64 buildfix
* Do the same with the hle result log macros
* Rename the log names to mixed case while at it.
* iOS buildfix
* Qt buildfix attempt, ARM32 buildfix
libretroadmin