These files use std::function, std::map, std::deque, std::unique_ptr or std::string_view and got
the header through others, which MSVC's standard library doesn't always do.
The GL contexts' CreateDrawContext made the thin3d presets and ContextReset made them again,
leaking the first set on every context reset. Only ContextReset makes them now.
The software renderer draws the same frame dumps on arm64 as on x86-64, so the frametests run
on the arm64 runner too. b.sh --aarch64 and setup-cross.sh aarch64 give a headless build to run
them under qemu locally.
A drained MpscQueue node goes onto a free stack shared by every queue of the same type, and a
producer takes that stack whole when its own thread's list runs dry, so once warmed up a push
no longer allocates. That takes the heap allocation back off memory block notifications, the
CPU queue, Vulkan deletes and the GL render queue. A unit test runs several producers against
one consumer over two rounds.
The NpSignin dialog's delays run on emulated time and are saved with the state, so the dialog
behaves the same at any speed and under run-ahead. The time since the last save and savestate
goes; only the standalone frontends' exit prompts read it.
Readback buffers are allocated when the readback is recorded rather than on the render thread,
so only the thread that records frames touches VMA. The allocator is created externally
synchronized, and VMA is built with no-op mutexes and plain counters instead of std::mutex and
std::atomic.
Headless runs Vulkan offscreen in builds without SDL too, and CI runs the headless tests on
lavapipe, skipping the pixel tests a hardware backend doesn't match the software references on.
sleep_ms and sleep_us go, with the thread manager test waiting on an event instead, and so does
the unused portable-file-dialogs header.
The core and the headless tools no longer use mutexes, condition variables,
spinlocks, std::atomic, std::thread or volatile flags. Threads hand off through
libretro-common's retro_atomic and rthreads plus a few small pieces on top of
them in Common/Thread: Thread, MpscQueue, EventCounter, ParkingLot waits and
ReaderGate. Every wait blocks until it's woken; nothing polls on a timer.
State that only one thread touches just loses its lock. The PSP filesystem,
sceIo and the savedata dialog now belong to the emulation thread: async reads
and writes run when they're queued and complete later in emulated time, and
the IO and savedata threads are gone. The friend finder only talks to the
adhoc server and hands what it gets to the emulation thread. GPU init always
runs on the thread that starts the game.
The Vulkan render and compile queues, pipeline and render pass caches, delete
list and SPIR-V cache, the GL render queue, the software renderer's jit caches,
the file loader caches, the web debugger, UPnP, input, savestates, logging and
the GE debugger all hand off through queues and atomics. The IR jit's spinning
profiler thread goes.
A CPU thread with nothing to run now sleeps until work is queued instead of
spinning. Headless with --debugger waits for a client while stopped, and ends
the run once every client that connected has left. Web server shutdown keeps
servicing the CPU queue until the server has finished.
The libretro core is the frontend, and the headless tools are the test and
debugging hosts. The Windows, UWP, Android, iOS, macOS and SDL apps, the UI
framework and imgui go, along with what only they used: the Win32 debugger
hooks (g_frameMutex, the shutdown lock), JNI thread attachment, the Android
text drawer, RAIntegration, Discord, recent files, game installs, tilt input,
the audio resamplers, the control mapper, memstick moving and the remote ISO
and upload server modes. The web server keeps the debugger. The MSVC, Gradle
and ndk-build projects go too.
Socket waits now block in select() on the socket plus a wake or cancel
socket (Common/Net/Cancel), so a stop or cancel ends them at once.
- HTTP: cancel flags become net::CancelToken; Connect waits once on all
addresses, receives wait instead of spinning, no DNS retry sleep.
- Adhoc: the friend finder and adhoc server block on their sockets with
a wake socket and a deadline; matching runs on a CoreTiming tick
instead of two threads (savestate section bumped).
- sceNetInet select/poll check without blocking and park the HLE thread,
re-checked from a CoreTiming event until ready or the emulated
timeout.
- Vendor the aemu_postoffice client (was a submodule) with no sleeps:
waits on the socket, close via shutdown and an eventcount, cancelable
connects, and lock-free session slot claims.
- PortManager waits without a timeout; PSP_InitWait replaces the
headless boot poll.
- TimeUtil: sleep_us/sleep_ms only outside the core; sleep_precise,
sleep_random, yield and the Windows timer period are gone.
- Unit test for the socket waits.
The core no longer sleeps to pace itself: the frontend runs retro_run at
its own rate, so the frame throttle, the idle-vblank delay and the real
clock sync event (and its core option) are gone, and a stepping CPU hands
straight back instead of waiting 2 ms. Waits that polled now park on a
lock-free eventcount through Common/Thread/ParkingLot and are woken by
whoever changes the state: a paused GL emu thread, which also queues a
marker behind its last frame so pausing runs the render queue up to it
instead of polling; async I/O results; the software renderer when its
queue is full; Vulkan pipeline compiles; and debugger requests once
stepping ends. Boot joins the loader thread, cache shutdown joins its
read-ahead thread, and the slow-I/O simulation is removed.
The path browser, the HTTP and WebSocket servers, the Windows console and
the game info cache are left out of the core build. libretro/test gains
lrgltest, which drives the GL backend on a surfaceless EGL context through
every path that pauses or stops the emu thread; run.sh and the image
checks CI job run it on llvmpipe.
Deflate, zlib and gzip streams go through rinflate and rdeflate, zstd
through rzstd, CHD images through rchd, zip archives through
rzip_archive and PNG through rpng. The bundled libraries, their build
glue and the system-library options are gone; adler32 and uncompress
keep zlib's ABI for the prebuilt FFmpeg archives that still call them.
libretro-common is one CMake library that the core, the headless tool
and the unit tests all link.
The reads take shorter paths on the way. CHD decodes straight from the
loader's reads and a range of blocks at a time, and a file read at any
alignment decodes into place; CSO keeps one inflate stream instead of
one per block; a plain ISO serves a file read with one read of its exact
bytes; a stored zip member is read in place and a deflated one through
restart points rather than a copy of the whole image in memory; zip
members decode into the caller's buffer; sequential reads through the
frontend's VFS skip the seek, which also keeps its buffering.
libretro/test reads one image as ISO, CSO, CHD and zip through the block
devices, boots each, and checks the codecs against Python-made vectors;
a CI job runs it.
The tree is built as a libretro core and nothing else, so the
standalone frontends lose their lanes: the Visual Studio and UWP
solutions, the Android app, iOS, the macOS app bundle, the AppImage
and the manual packaging workflows. PPSSPPHeadless and
PPSSPPUnitTest stay, built as named targets so a configure no longer
pulls the SDL frontend in, and keep feeding the headless, frametest,
qemu and Alpine jobs; the Docker image builds the same way.
Two lanes cover builds nothing ran before: the core on an arm64 host,
and the CMake build of the core on Linux.
The libretro-common submodule pinned the standalone repository at
76a3d54. It is replaced by an in-tree copy of RetroArch master
(b655a20) holding only what the core uses: the VFS, file, string,
encoding, compat and time sources it already built, the headers they and
the core include, the threading set - rthreads, retro_atomic, retro_spsc
and retro_waitable_spsc, mpsc_stack, retro_triple_buffer,
retro_eventcount, retro_asym_eventcount, retro_procbarrier and tpool,
with features_cpu and rstrtod, which they need - and the codecs that
replace the bundled zlib, zstd, libchdr, libzip and libpng: rinflate and
rdeflate, rzstd, rchd with its LZMA and FLAC decoders, rzip_archive,
rpng and crc32.
stdstring.c, which nothing calls, is no longer built. The threading sources are always built, by both the static Makefile and
CMake; the VFS set stays behind HAVE_LIBRETRO_VFS as before. The codecs
are built by the change that switches the core over to them.
On a native MSYS2 build FFmpeg's configure finds the Windows SDK headers
and enables DXVA2 and D3D11VA, whose dxva2.c fails with GCC 14 and later
(a pointer type mismatch in a conditional expression is now an error).
PPSSPP uses no FFmpeg hardware acceleration, so every autodetected
hwaccel and external library is disabled explicitly, which also makes
the result independent of what the build machine has installed.
The MinGW platform linked FFmpeg from the toolchain while compiling
against the bundled 3.x headers, which fails against any current FFmpeg.
The prebuilt ffmpeg/Windows libraries cannot stand in: they are MSVC
builds, and GNU ld leaves calls between their function-level COMDATs
unrelocated. The bundled FFmpeg source is now configured and built with
the MinGW toolchain out of tree, with the components from
windows_x64-build.sh, and its static libraries are linked by path, so
the core no longer depends on any FFmpeg installed on the system.
FFMPEG_CROSS_PREFIX names the binutils prefix when cross-compiling.
The build-libretro-mingw CI job no longer installs MSYS2's FFmpeg.
The MinGW platform compiled against the bundled Windows FFmpeg headers
(3.x) but linked the toolchain's FFmpeg, since the bundled libraries are
MSVC builds MinGW cannot link; against a current FFmpeg the old API
calls the 3.x headers select are unresolved. It now compiles against the
same FFmpeg it links, which the existing version guards already handle.
zstd's x86-64 asm decoder loop is enabled for any GNU C compiler, so the
asm source is now built for MinGW too rather than only skipped on MSVC,
and version and iphlpapi join the link for GetFileVersionInfo* and
GetAdaptersAddresses/GetBestInterfaceEx.
A build-libretro-mingw job builds the core with platform=win from
MSYS2 so these breaks show up in CI.
The zip_win32_file_operations slots take const void * names, so storing
CreateFileA, DeleteFileW and the rest relied on an implicit function
pointer conversion that GCC 14 rejects as an error, breaking the MinGW
build of the libretro core. The slot types are now named typedefs and
the ansi and utf16 tables cast each entry point to them explicitly.
SOFT_BMP_SIZE was a byte count used as the element count of a u16
array, keeping ~1 MB resident per context, and SwapBuffers memcpy'd a
full frame out of the vector ConvertFramebufferForLibretro returns empty
for unsupported formats, a heap OOB read. The intermediate buffer and
its per-frame memcpy are gone: the converted vector goes to video_cb
directly, with the 16:9 crop done by pointer offset. Before a GPU exists,
or when conversion fails, a frame dupe is sent instead of stale data.
System_AudioPushSamples converts straight into the output buffer: the
block loop never advanced the source pointer and copied numSamples*2
int16s out of a 2048-sample stack buffer, a stack OOB read for any push
over 1024 frames. Realloc failure drops samples instead of crashing and
log_cb is NULL-checked. retro_serialize refuses to write a state larger
than the frontend's buffer, retro_get_memory_data/size return nothing
until PSP memory is mapped, deferred unserialize no longer leaks the
previous buffer, and the cheat entry points stop leaking a CWCheatEngine
per call.
The emu thread acknowledges a pause with a CAS so it cannot clobber a
concurrent quit request. EmuThreadStop stops and joins from any live
state, so unloading while paused no longer leaves a thread running over
freed state. EmuThreadPause pumps queued frames until the thread parks,
via a new non-blocking GLRenderManager::HasQueuedThreadWork() check,
instead of draining one frame, which deadlocked when the thread was
blocked on a readback sync in the next frame. retro_run resumes a paused
thread and pauses it before check_variables() rewrites g_Config,
retro_reset stops it before PSP_Shutdown(), and the sleep_ms(4) after
resume is gone.
Linux's uint64_t is unsigned long, which neither of its u32 and u64
(unsigned long long) overloads takes exactly, so the call was ambiguous.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The range check was written as tests for outside, which are all false for
NaN, so a NaN vertex passed and was converted to an integer. That differs
by CPU: arm64 gives 0, which lands off screen, while x86-64 gives INT_MIN,
which wraps, and the triangle's area overflowed into a huge sliver. In the
recording-opengl and recording-vulkan dumps that drew a beam across the
top right that a PSP replay of the dump doesn't show. Now such a vertex is
flagged, and its triangles dropped, on both.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A full state ring flushed the binner, and the next state change filled
it again in the same way. Now the flush also doubles the ring, up to
32768 states, so it happens at most a few times. Nothing draws right
after a flush, so the ring can move then. In the benchmark games, no
more than 512 states were queued between flushes, so 4096 stays the
starting size.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The binner kept a bounding box of every queued item's range, updated for
each item, only to tell whether anything had been queued since the last
flush. A flag says the same. Expand, which grew the box, is now
ItemQueued.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each queued primitive adds its vertex flags (alpha, fog, colors) to its
state, and states are optimized for those flags before their items are
distributed. A state can still gain primitives after some of its items
are in tiles. Re-optimizing it then changed its pixel and sampler
functions and IDs while threads drew with them. With the x86 pixel JIT,
the span function could turn null between checking and calling it, as
in Coded Arms' frame dump. God of War hit this about four times a frame.
Now a state is live once any of its items is in a tile, until the tiles
are drawn and reset, and a live state isn't changed. A primitive that
would change a live state's flags starts a new, unoptimized state from
the same registers instead, optimized for its own primitives only.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The pixel and sampler JITs only exist on x86-64, but elsewhere a cache
miss still flushed the binner before the compile that did nothing, and
the cache never filled. So on ARM every switch to a new pixel or sampler
state was a full flush: hundreds per frame.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
headless_bench.py compares the speed of headless binaries on savestates
or frame dumps. It alternates the binaries between rounds and reports
medians of wall time, CPU time, and instructions and cycles where the OS
counts them. Instructions vary by well under 0.1% between identical runs,
where wall time swings by several percent. It checks for mains power
first, and stops on a run that failed or didn't get where it should.
dump_compare.py replays every frame dump in a directory with two binaries
and reports the dumps that render differently, or differently from run
to run.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Expanding a pending range by a rectangle starting below its start added
a negative row count to an unsigned height, which wrapped to millions of
rows. Overlap tests then overflowed and missed real overlaps. Now a
merged range is whole rows from the lower start to the higher end.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A draw whose texture rectangle overlapped what queued primitives write
flushed, once per draw. NeedsOrder already orders each primitive by the
texels it actually reads, so let it. God of War's bloom passes texture
from the main framebuffer declared as 512x512, which reaches into the
buffer they draw to, and flushed every 30 primitives: 6% faster there.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Every tile a triangle touches sets it up again, so larger tiles do less
of that. The tile size is now picked at startup, as large as it can be
with at least sixteen tiles per thread over the screen. With fewer, a
scene whose drawing is concentrated on part of the screen leaves threads
idle: on eight threads, 128x32 tiles (four per thread) took LocoRoco a
third longer than 64x16, and God of War 5% longer. So eight threads keep
64x16, and four or fewer get larger tiles.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Returning to the CPU flushed all queued drawing, also when a list only reached its stall address.
Wipeout Pure builds its lists as it goes and stalls about a hundred times a frame, so the main thread
spent a third of its time finishing the drawing at those flushes. A stalled list isn't done, so now
only a finished one flushes there, which makes Wipeout Pure 15% faster.
What relied on that flush now flushes itself: memory copies, sets, readbacks and uploads that overlap
the queued drawing, and save states, before memory is saved or loaded (FlushPendingDrawing). Syncs and
the display already did. Queued drawing still reads textures from PSP memory when it runs, so a game
that rewrites a texture between stalls without syncing would see the new one; on hardware the GE has
usually drawn with the old one by then.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Waking a worker is a condition variable signal, a system call, and the main thread paid one per idle
worker every 48 queued items. With Tekken 6's tiny draws (about 5 vertices each) the workers keep
running dry, and those signals were about 10% of the main thread, which is what limits it there.
Now no more threads are woken than tiles have work, and only the first is woken from the main thread;
it wakes the rest.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
A state that textures from what it draws to, or whose scissor reaches
past the stride, used to switch the binner to one thread, and every
primitive was drawn alone, copying the whole texture for its snapshot.
In Wipeout, God of War and GTA that was a third to half of the time.
Now it's decided per primitive. One is drawn after everything queued
and before what follows only if it needs that: it textures from texels
a queued primitive (or itself) writes, it writes texels a queued one
still has to read, or it reaches past the stride. Reads are bounded by
the vertices' texture coordinates plus the filter's neighbors and
mapped to tiles of the target; queued writes and reads are tracked per
tile. Everything else goes to the tiles as usual. The in-order ones are
split into bands of rows when large, since each pixel reads only the
snapshot, and the snapshot now only copies the previous primitive's
columns.
Same results on 263 frame dumps and 8 games.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Each thread used to own a band of the screen, fixed when the work was
first split, and got its own copies of the primitives touching it. Bands
were rarely balanced (edge bands had half the work of middle ones, and
on big.LITTLE CPUs half the threads are slower), and primitives queued
up to the 2048 limit before the threads started.
Now primitives go to 64x16 tiles, as indices into the shared queue, in
batches of 16. A thread takes any tile with work and draws its items in
order; only one thread holds a tile at a time, so per-pixel order is
kept. A flush or a full queue has the main thread draw tiles too instead
of waiting, and when there's only a little work it does that alone
rather than wake the threads. Items are reclaimed once every tile has
drawn them. This also drops the per-thread queues, up to 23 MB.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
HLSL types an unsuffixed hex literal as uint, so asint(x) - 0x3F800000
went unsigned and wrapped for every x below 1. The huge t then turned
into a NaN, and anything lit with specular or powered diffuse came out
black on D3D11 (the flags in Everybody's Golf's language select, #22418).
GLSL types it as int and was unaffected.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Texture pack installs took the destination directory from the archive's
textures.ini, and PKG update installs from the PBOOT's DISC_ID, both
unchecked, so a crafted archive or package could write outside
PSP/TEXTURES or PSP/GAME. Add IsSafePathComponent() to PathUtil and check
both, replacing the WebServer's own upload filename check with it too.
DISC_ID from any PARAM.SFO also names configs, cheats, textures, exdata
and savestates, so ReadSFO now drops one that isn't a safe name and the
game gets a fake ID like homebrew. The same goes for a frame dump's ID.
Combines #22419 and #22420 by Acts1631.
Co-Authored-By: Acts1631 <acts1631kjv@proton.me>
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The GE keeps the specular and spot exponents to 5 significant bits,
truncated, and from 512 up (infinity and NaN included) uses 496. Only
the specular one was truncated, and neither saturated. A huge or NaN
exponent overflowed pspPow's bits: highlights wrong on every backend,
and on D3D11, where the overflowing float to int conversion gives a
NaN, black (Everybody's Golf, #22418). With the exponent clamped, t
stays in range for every base the backends produce.
Measured, now bit exact in the software renderer and within 4 levels
on Vulkan.
Fixes#22418
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The x86 sampler JIT returned zero for a whole sample when a texture level
pointer was null, and for a mipmapped texture it checked the next level
too, even when it isn't blended in. Tactics Ogre draws its font from a
mipmapped texture whose next level has an invalid address, at level 0,
so all its text was missing on x86-64. The generic samplers read a null
level's texels as zero only where they use that level, and still apply
the texture function. Now samplers for such textures aren't compiled,
and the JIT's null checks are gone. Test: gpu/textures/mipinvalid.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
libretroadmin