mirror of
https://github.com/pewdiepie-archdaemon/odysseus.git
synced 2026-06-15 17:25:26 -04:00
Lucas Daniel
adc6ac9394
The previous approach polled request.is_disconnected() inside the async-for body of the chat/agent streaming loops. That happens too late: by the time the poll runs, __anext__() has already awaited and consumed the next upstream chunk, so a slow or silent generation could still run for a full round-trip (or until a read timeout) after the client disconnected. It was also unconditional, which would have made ordinary chat navigation/refresh/tab-close stop a run that the detached-run design intentionally keeps going server-side. Both problems trace back to the same root cause: chat_stream always wraps its generator in agent_runs (the detached-run manager), which decouples the generator's lifetime from the SSE response on purpose so normal chat/agent streams survive the client going away. Polling disconnection inside a detached generator can never be "prompt" — the generator isn't tied to that request anymore — and doing so defeats the whole point of detaching it. Compare panes don't need (or want) that: each pane's session exists only to drive that one generation, there's nothing meaningful to /resume, and the user expects the pane's Stop button — which aborts the fetch and closes the SSE — to cancel the upstream call right away. So route compare-mode requests around the agent_runs wrapper entirely and stream the generator directly as the SSE body. Starlette already cancels a streaming response's body iterator (raising CancelledError/GeneratorExit into it) the instant it notices the client disconnected — including while the generator is mid-await on the next upstream chunk — and the existing except (CancelledError, GeneratorExit) handlers in both the chat-mode and agent-mode loops already save the partial response exactly once. No polling needed; the redesign just stops getting in its own way. Normal (non-Compare) chat and agent streams are untouched and keep going through agent_runs, preserving detached-run semantics (surviving tab close / navigation / refresh, reconnect via /api/chat/resume). Replaces the source-text assertions in tests/test_compare_stop_disconnect_poll.py with runtime tests that actually exercise the cancellation contract: a Compare-shaped generator is cancelled mid-await (not after the next chunk arrives) and saves its partial exactly once; a normal completion still saves exactly once via the completion path; agent_runs keeps a detached run alive when its subscriber disconnects and only stops it on an explicit stop()/cancel (also saving the partial exactly once); and the cancellation contract is pinned for both chat-mode- and agent-mode-shaped chunk sequences.