NewThreadExecutor::Run pushed a std::thread per connection and only ever joined them
in the destructor, so a server leaked a joinable thread object for every connection
it had ever served. Measured with 60 connect/disconnect cycles against the debugger:
handle count +60 before, +1 after. Each worker now flags itself done as its last act,
and Run() reaps the finished ones first. Only the accept thread calls Run(), so the
flag is the only thing that needs to be atomic.
Note this doesn't bound how many connections can be in flight at once - it just stops
the finished ones from piling up.
Separately, a received close code was echoed straight back. RFC 6455 7.4.1 reserves
1004, 1005, 1006 and 1015 for describing how a connection ended locally, so they must
never go on the wire - echoing one back would be our protocol violation rather than
the client's. Send PROTOCOL_ERROR when they give us something we can't repeat.
RequestHeader::GetParamValue indexed parts[1] without checking the size. A query
parameter with no '=' at all ("?foo") makes SplitString return a single element, so
both the DEBUG_LOG and the assignment read off the end of the vector. Nothing calls
GetParamValue today, so this is latent rather than live, but it's driven straight
off the request line.
The 64-bit frame length was assembled with header[n] << 24 on uint8_t values, which
promote to int - a byte >= 0x80 in the top position shifts into the sign bit and then
sign-extends when widened to uint64_t. The resulting size was always rejected, just
by the wrong check and via signed overflow to get there. Cast first.
OutputSink::Block() had the same shape as the InputSink one this branch already
fixed: a broken socket is reported ready immediately and forever, so waiting on it
is a spin. Bail if the sink already knows it's broken.
Follow-up to the previous commit, from Nemoumbra's questions - which found a worse
spin than the one that fix addressed.
InputSink couldn't tell "nothing right now" from "peer is gone": Fill() treats
recv() == 0 as no data and only sets hasError_ on a real error. Block() then waits
with WaitUntilReady(), which reports a closed socket as ready immediately and
forever, so TakeExact() looped on it without ever returning. A client that
disconnects with half a frame buffered - easy to do while blasting messages - put
the server in an infinite loop inside TakeExact, never even returning to Process().
Measured 7.95 CPU-seconds over 8 seconds; 0.08 after.
So: track EOF explicitly (sticky atEnd_, exposed as AtEnd()), and have Block() give
up when nothing more can arrive.
That information was being thrown away in three more places:
* Process() only tried to fill when the sink was already empty, so a disconnect went
unnoticed for as long as there were leftovers - and if those leftovers were a
partial frame, the read above never completed. Always fill, and close once the
peer is gone and we've consumed what it sent.
* ReadPending() uses TakeAtMost(), which returns 0 both for "nothing right now" and
"nothing ever again", and then reported success having consumed nothing. Ask the
sink which it was.
* Both TakeExact() call sites answered a failed read with POLICY_VIOLATION, blaming
the client for a protocol error when it had simply disconnected. Check the sink
and report ABNORMAL when that's what happened.
Also stop queueing data once our own close frame is queued. RFC 6455 5.5.1 forbids
data frames after a close, and beyond the protocol, anything appended afterwards
keeps the buffers non-empty and starves the "everything is flushed" check that ends
the connection. Observed the server pumping 167MB of log broadcasts after being
asked to close.
The repeated close-and-discard is now one helper.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DCPmm7FoQUoqrbMdhfqhQ2
Reported by Nemoumbra: the debugger server could get stuck in a tight select()
loop after a lot of traffic, burning a core.
Once OutputSink hits a real send() error it latches hasError_, after which Flush()
returns immediately without consuming anything, so out_->Empty() is false forever.
Process() waited for that to empty before finishing the close, kept the fd in the
write set, and select() reports an errored socket as ready every time - so it
returned true on every lap without ever making progress, and WebSocketDebuggerLoop
span. This needs sentClose_ to be set for it to be unrecoverable, since otherwise
the read side notices the disconnect and closes; a client that sends CLOSE (or
trips a protocol error) while output is backed up gets exactly that. Reproduced
with a client that queues ~120MB of responses, sends CLOSE, then resets the
connection without reading: 6.02 CPU-seconds over 6 seconds before, 0.06 after.
Treat an output error as fatal to the connection instead.
Also, select() returning -1 always returned true, so any error that doesn't fix
itself (a bad fd rather than EINTR) was a second busy-loop with no wait at all.
EINTR retries, everything else closes.
Finally, SendFlush() erased the drained bytes off the front of outBuf_ every lap.
With a backlog that's a memmove of the whole buffer per lap, i.e. quadratic in the
backlog, which burns CPU on its own while draining a slow client. Track a consumed
offset and only compact once the dead prefix is worth reclaiming.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DCPmm7FoQUoqrbMdhfqhQ2
ReadFrame() accepted a 64-bit client-supplied payload length with only
a top-bit check, and ReadPending() immediately resized a buffer by it
before any data had arrived - a single frame claiming a huge length
(reachable via the WebSocket debugger endpoint) could trigger a
multi-exabyte allocation attempt. Now rejected up front (both the
single frame and the fragmented-message total) against a 64MB cap.
Also replaced &payload[0]/&vector[0] with .data() in the send/receive
paths - operator[] on a possibly-empty vector (e.g. an empty PING) is
UB even when the result is never dereferenced.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L4QAoxV2KY7ek4PcZw3WvY
* Move and rename file_util/fd_util to Common/File/FileUtil and DirListing
Let's also move net while we're at it.
Move the ZIM/PNG loaders over to Common.
Move the UI framework into Common
iOS buildfix
* Buildfix
* Buildfixes
* Apple buildfix
* This typo again..
* UWP buildfix
* Fix build of PPSSPPQt, such as it is (it's not in good condition...)
* Guess what? Another buildfix.
Henrik Rydgård