naett has had a complete libcurl backend all along; we just never built it,
so Linux ran with HTTPS_NOT_AVAILABLE. That means no homebrew store over
HTTPS, and RetroAchievements talking to plain http://retroachievements.org.
libcurl is loaded with dlopen rather than linked, the same way we handle the
Vulkan loader, so it stays a soft dependency: we need the curl headers at
build time, but a build made here still starts on a machine without libcurl
installed - it just reports HTTPS as unavailable, exactly like today. Distro
packagers get the behavior they'd expect either way, and certificate
validation comes free from the system CA store.
New net::HTTPSAvailable() answers "did that work", and SDLMain folds it into
SYSPROP_SUPPORTS_HTTPS, which everything downstream already degrades on.
Four fixes to the backend itself, all noted in ext/naett/README-ppsspp.md:
- panic() called exit(1) on a pipe or curl_multi_perform failure. Taking the
emulator down because a download failed isn't acceptable - the backend now
disables itself and requests complete with naettGenericError.
- CURLINFO_RESPONSE_CODE writes a long into res->code, which is an int. Eight
bytes into four, getting away with it only because the next field absorbs
the zeroes.
- curl_easy_setopt is varargs and wants a long for these options; int literals
and int variables are UB on LP64.
- naettPlatformCloseResponse called through a null function pointer when
libcurl was missing. Found by testing that path, which segfaulted.
CI needs libcurl4-openssl-dev (curl-dev on Alpine) or it would quietly keep
building without HTTPS.
naett has been a submodule pinned at v0.3.3; upstream has had no commits since
April 2024, and we want to carry local changes (next up: a libcurl-backed HTTPS
path for Linux). It's ~1500 lines of MIT C, smaller than several things we
already vendor, so bring it in-tree and drop the submodule.
Also drop the generated single-file amalgam (naett.c) that every build system
was compiling, and build src/*.c directly instead - otherwise the file you edit
isn't the file that gets compiled, which is a trap for anyone patching this.
example/ and testrig/ (a whole Android Studio project) are gone with it.
Two changes were needed to make the sources build on their own, both noted in
ext/naett/README-ppsspp.md along with the upstream commit:
- naett_internal.h now includes naett.h, which the amalgam pulled in first.
- naett_linux.c now includes stdio.h/stdlib.h. It calls exit/calloc/realloc/
free/fprintf without ever including either, and only got away with it because
naett_core.c sat above it in the concatenation.
No functional change - Linux still has HTTPS_NOT_AVAILABLE set, so it doesn't
build naett at all yet.
Rename naett to naett-lib
NewThreadExecutor::Run pushed a std::thread per connection and only ever joined them
in the destructor, so a server leaked a joinable thread object for every connection
it had ever served. Measured with 60 connect/disconnect cycles against the debugger:
handle count +60 before, +1 after. Each worker now flags itself done as its last act,
and Run() reaps the finished ones first. Only the accept thread calls Run(), so the
flag is the only thing that needs to be atomic.
Note this doesn't bound how many connections can be in flight at once - it just stops
the finished ones from piling up.
Separately, a received close code was echoed straight back. RFC 6455 7.4.1 reserves
1004, 1005, 1006 and 1015 for describing how a connection ended locally, so they must
never go on the wire - echoing one back would be our protocol violation rather than
the client's. Send PROTOCOL_ERROR when they give us something we can't repeat.
RequestHeader::GetParamValue indexed parts[1] without checking the size. A query
parameter with no '=' at all ("?foo") makes SplitString return a single element, so
both the DEBUG_LOG and the assignment read off the end of the vector. Nothing calls
GetParamValue today, so this is latent rather than live, but it's driven straight
off the request line.
The 64-bit frame length was assembled with header[n] << 24 on uint8_t values, which
promote to int - a byte >= 0x80 in the top position shifts into the sign bit and then
sign-extends when widened to uint64_t. The resulting size was always rejected, just
by the wrong check and via signed overflow to get there. Cast first.
OutputSink::Block() had the same shape as the InputSink one this branch already
fixed: a broken socket is reported ready immediately and forever, so waiting on it
is a spin. Bail if the sink already knows it's broken.
Follow-up to the previous commit, from Nemoumbra's questions - which found a worse
spin than the one that fix addressed.
InputSink couldn't tell "nothing right now" from "peer is gone": Fill() treats
recv() == 0 as no data and only sets hasError_ on a real error. Block() then waits
with WaitUntilReady(), which reports a closed socket as ready immediately and
forever, so TakeExact() looped on it without ever returning. A client that
disconnects with half a frame buffered - easy to do while blasting messages - put
the server in an infinite loop inside TakeExact, never even returning to Process().
Measured 7.95 CPU-seconds over 8 seconds; 0.08 after.
So: track EOF explicitly (sticky atEnd_, exposed as AtEnd()), and have Block() give
up when nothing more can arrive.
That information was being thrown away in three more places:
* Process() only tried to fill when the sink was already empty, so a disconnect went
unnoticed for as long as there were leftovers - and if those leftovers were a
partial frame, the read above never completed. Always fill, and close once the
peer is gone and we've consumed what it sent.
* ReadPending() uses TakeAtMost(), which returns 0 both for "nothing right now" and
"nothing ever again", and then reported success having consumed nothing. Ask the
sink which it was.
* Both TakeExact() call sites answered a failed read with POLICY_VIOLATION, blaming
the client for a protocol error when it had simply disconnected. Check the sink
and report ABNORMAL when that's what happened.
Also stop queueing data once our own close frame is queued. RFC 6455 5.5.1 forbids
data frames after a close, and beyond the protocol, anything appended afterwards
keeps the buffers non-empty and starves the "everything is flushed" check that ends
the connection. Observed the server pumping 167MB of log broadcasts after being
asked to close.
The repeated close-and-discard is now one helper.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DCPmm7FoQUoqrbMdhfqhQ2
Reported by Nemoumbra: the debugger server could get stuck in a tight select()
loop after a lot of traffic, burning a core.
Once OutputSink hits a real send() error it latches hasError_, after which Flush()
returns immediately without consuming anything, so out_->Empty() is false forever.
Process() waited for that to empty before finishing the close, kept the fd in the
write set, and select() reports an errored socket as ready every time - so it
returned true on every lap without ever making progress, and WebSocketDebuggerLoop
span. This needs sentClose_ to be set for it to be unrecoverable, since otherwise
the read side notices the disconnect and closes; a client that sends CLOSE (or
trips a protocol error) while output is backed up gets exactly that. Reproduced
with a client that queues ~120MB of responses, sends CLOSE, then resets the
connection without reading: 6.02 CPU-seconds over 6 seconds before, 0.06 after.
Treat an output error as fatal to the connection instead.
Also, select() returning -1 always returned true, so any error that doesn't fix
itself (a bad fd rather than EINTR) was a second busy-loop with no wait at all.
EINTR retries, everything else closes.
Finally, SendFlush() erased the drained bytes off the front of outBuf_ every lap.
With a backlog that's a memmove of the whole buffer per lap, i.e. quadratic in the
backlog, which burns CPU on its own while draining a slow client. Track a consumed
offset and only compact once the dead prefix is worth reclaiming.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DCPmm7FoQUoqrbMdhfqhQ2
UriDecode() formed SRC_END - 2 unconditionally, a pointer before the
start of the buffer (UB) for a 0- or 1-byte input. IsLocalAbsolutePath()
indexed path[0]/path[1] on a std::string_view with no bounds check,
UB for an empty path (path[0]) or a 1-byte path on Windows (path[1]).
Neither was known to crash in practice, but both are real UB flagged
by hardened/UBSan builds and easy to trigger (e.g. an empty query
string, or listing the VFS root).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L4QAoxV2KY7ek4PcZw3WvY
ReadFrame() accepted a 64-bit client-supplied payload length with only
a top-bit check, and ReadPending() immediately resized a buffer by it
before any data had arrived - a single frame claiming a huge length
(reachable via the WebSocket debugger endpoint) could trigger a
multi-exabyte allocation attempt. Now rejected up front (both the
single frame and the fragmented-message total) against a 64MB cap.
Also replaced &payload[0]/&vector[0] with .data() in the send/receive
paths - operator[] on a possibly-empty vector (e.g. an empty PING) is
UB even when the result is never dereferenced.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L4QAoxV2KY7ek4PcZw3WvY
ParseHttpHeader() used strchr(buffer, ' ') unconditionally as endptr,
even though the parser explicitly supports HTTP/0.9-style requests
with no trailing space/version (type = SIMPLE). A request line like
"GET /" with no space made strchr return null, and nullptr - buffer
truncated to a garbage length driving new[]/memcpy. Falls back to the
end of the line when no space is found, and clamps param_length to
avoid a similar issue when '?' appears after the (missing) space.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L4QAoxV2KY7ek4PcZw3WvY
Henrik Rydgård