* feat(email): add Google OAuth2 for Google Workspace / .edu IMAP & SMTP
Google deprecated basic-auth (password) access for Google Workspace
accounts in May 2025. This means any .edu or org Google email account
could no longer connect via IMAP/SMTP with a username + password —
the email feature was silently broken for a large class of users.
This PR adds full OAuth2 (XOAUTH2) support for Google accounts so
Workspace / .edu emails work out of the box.
## What changed
### Backend
- `core/database.py`: add `oauth_provider`, `oauth_access_token`,
`oauth_refresh_token`, `oauth_token_expiry`, and `display_name`
columns to `EmailAccount` + idempotent migration
- `routes/email_helpers.py`: XOAUTH2 auth in `_imap_connect()` and
`_send_smtp_message()`, automatic token refresh, OAuth fields in
`_get_email_config()`
- `routes/email_routes.py`: OAuth authorize + callback routes,
`_smtp_ready()` fix, OAuth fields through `_deliver()` closure,
`display_name` in `From:` header
### Frontend
- `static/js/settings.js`: "Google Workspace / .edu" provider preset,
"Connect with Google" button, success/error banner, display name field
- `static/js/document.js`: `_accountCanSend()` recognises OAuth accounts
as SMTP-capable
* security: sign OAuth state, scope callback by owner, fix quotes & logs
Addresses reviewer feedback on the email OAuth2 PR:
- OAuth state is now HMAC-SHA256 signed (keyed with the app secret from
secret_storage) encoding account_id + owner + a random nonce, and is
verified with constant-time comparison in the callback before any
token write. Replaces the bare account_id state, closing the CSRF /
state-guessing gap.
- Callback extracts the owner from the verified state and re-checks it
against EmailAccount.owner before writing tokens, matching the
ownership guards used elsewhere in the email routes. Single-user mode
(owner == "") still accepts any account, consistent with
_assert_owns_account.
- Replaced curly/smart quotes in the Name/Email/Display Name input rows
with plain ASCII so getElementById lookups and event wiring work.
- Stripped account name, SMTP host/user, owner, and raw provider error
text from send-config and OAuth logs; failures now surface as generic
error codes in the redirect instead of raw exception strings.
* test(email): add OAuth2 state, _smtp_ready, and XOAUTH2 tests
Move the OAuth state sign/verify helpers out of the setup_email_routes
closure into module-level make_oauth_state/verify_oauth_state in
email_helpers.py so they can be unit-tested, then add tests/test_email_oauth.py:
- signed state round-trips account_id + owner, nonce is unique per call
- tampered account_id, forged signature, and garbage states are rejected
- _smtp_ready treats an OAuth account (no password) as send-capable, and
still rejects host+user-only accounts with neither password nor OAuth
- _xoauth2_string / _xoauth2_bytes produce the correct SASL XOAUTH2 framing
14 new tests; existing test_security_regressions.py still passes (28).
* refactor(email): single XOAUTH2 frame helper, use RuntimeError
Polish from self-review before merge:
- Collapse the XOAUTH2 framing to one source of truth: _xoauth2_raw()
returns the unencoded SASL string used by both the SMTP and IMAP auth
callbacks (each library base64-encodes it), and _xoauth2_bytes() is
just its .encode(). Removes the unused base64 _xoauth2_string helper
and the duplicated inline frame in _send_smtp_message.
- Raise RuntimeError (not bare Exception) for the "OAuth token
unavailable" path, matching the convention used across src/.
- Update tests accordingly.
All 14 OAuth tests + 28 security regressions pass; SMTP/IMAP XOAUTH2
verified live against a real Workspace account.
* tests(email-oauth): cover the security-sensitive OAuth paths before merge
The previous tests only exercised pure helpers (state signing, _smtp_ready,
XOAUTH2 framing). This adds coverage for the actual token-custody and
ownership behaviour, pinning the real route handlers rather than
re-implementations of their logic.
Real OAuth callback route (pulled live from setup_email_routes()):
- missing code -> generic missing_code redirect, no account id / owner in URL
- provider error -> generic google_error redirect, raw error not echoed
- tampered/invalid state -> invalid_state redirect, auth code never leaked
- signed state with owner mismatch -> token write refused (ownership_error),
DB row left untouched
- signed state with matching owner -> tokens written encrypted, and only to
the intended account (a second account stays untouched)
Real accounts-list route:
- exposes oauth_provider status but never the access/refresh token values,
encrypted or otherwise
Token storage / refresh helpers (isolated in-memory SQLite, mocked HTTP):
- refreshed access token stored encrypted; expiry is a timestamp, not a token
- fresh token uses cache (no refresh call); expired token triggers refresh
- refresh HTTP failure returns None silently, no exception or secret surfaced
- missing client credentials short-circuits to None
Password-account regression:
- password IMAP accounts call conn.login(); OAuth accounts call XOAUTH2
authenticate() and never login()
28 tests pass (14 prior + 14 new).
* fix(email-oauth): drop raw exception text from token-refresh log
Google token refresh failures now log the account id only, matching
the conservative logging used elsewhere on the OAuth path — no raw
provider/exception details surfacing in logs.
* fix(email-oauth): bring OAuth UI parity to the Integrations email form
The Google Workspace / .edu provider preset, Display Name field, and
Connect-with-Google flow were only wired into the Email-tab account
form. The Integrations-tab form (a separate code path for the same
account type) was missing all three, so the OAuth option was invisible
from that entry point. Mirrors the same PROVIDERS entry, OAuth section,
and connect handler so both forms behave identically.
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Alexandre Teixeira <111787685+alteixeira20@users.noreply.github.com>
* feat(auth): add per-user admin promote/demote toggle
Admin-only API and Users-tab control to grant/revoke admin rights; refuses to demote the last admin.
* fix(auth): restore pre-admin privilege restrictions on demotion
Promoting now stashes the user's privilege map (privileges_before_admin)
and demoting restores it instead of resetting to defaults, so a
promote/demote round trip can no longer broaden a restricted user's
access. Users without a stash (created as admin, or promoted before this
fix) still demote to DEFAULT_PRIVILEGES so a born-admin's stored all-True
map — including can_use_bash — can't survive demotion.
---------
Co-authored-by: K M Merajul Arefin <merajul.arefin@therapservices.net>
find_bash() rejected the WindowsApps WSL stub and then probed only %LocalAppData%\Git, so per-user Git for Windows installs (winget / Inno Setup {userpf}) under %LocalAppData%\Programs\Git were never found and the Cookbook reported "needs Git Bash" despite Git being installed.
Add the Programs\Git subfolder to the LocalAppData fallback root.
The _migrate_* startup helpers in core/database.py opened a raw
sqlite3.connect() inside a try and called conn.close() as the last
statement in that try. If any earlier statement raised (locked DB,
unexpected schema, a failed ALTER), close() was skipped and the bare
except only logged the error — leaking the connection (file handle +
lock) for the lifetime of the process. These migrations run on every
startup.
Wrap each in the conn = None + try/except/finally pattern already used
by _migrate_chat_messages_fts in this same file, so the connection is
closed on all exit paths. 25 functions; no change on the success path.
Helpers that already close safely are left untouched: _migrate_chat_messages_fts
and _migrate_backfill_task_folders (the latter uses SQLAlchemy's
engine.connect() context manager).
Same bug class as the previously merged DB-connection-leak fix (#64)
and the IMAP logout-on-all-paths fix (#1530).
* docs: add implementation plan for fixing chat context drifting (#135)
* fix: make Session.history immutable + fix {}.history crash
- Session.history now exposes a COPY of the internal _history list
- add_message() replaces history with a fresh copy each time
- get_context_messages() derives from _history directly
- replace_messages() updates both _history and history
- truncate_messages() updates both _history and history
- _persist_message() line 207: fixed {}.history fallback crash
- Added 11 tests for session isolation and edge cases
Addresses #135 root cause #1: shared mutable references
* fix: task scheduler uses SessionManager methods instead of overwriting sessions
- Added ensure_task_session() to SessionManager (checks cache first)
- Task scheduler now uses ensure_task_session() instead of direct dict assignment
- Task scheduler now uses SessionManager.add_message() for message persistence
- Removed direct sess_obj.history.append() that was silently losing data
Addresses #135 root causes #2 and #3
* fix: add age guard to cleanup_empty_sessions — don't delete sessions <1h old
Prevents the cleanup task from deleting sessions that were just created
and haven't received any messages yet (message_count == 0).
Addresses #135 root cause #5
* test: comprehensive session isolation tests (10/10 passing)
* refactor: consolidate _session_manager into singleton pattern
- Added set_session_manager_instance / get_session_manager_instance to core/models
- kept backward-compat aliases (set_session_manager, get_session_manager)
- session_manager.py re-exports the singleton functions
- ai_interaction.set_session_manager now syncs with the core singleton
- context_compactor uses get_session_manager_instance() instead of getattr hack
- app.py initializes the singleton once
Addresses #135 root cause #4: fragile global wiring
* test: add concurrent session isolation integration tests
Verifies:
- Concurrent add_message to different sessions doesn't cross-contaminate
- Rapid parallel writes maintain isolation
- Read-write concurrent access is safe
All 3 async tests pass, proving the immutable history fix works under concurrency
* fix: pre-import core.models in conftest to prevent test pollution
test_agent_loop.py stubs sys.modules['core.models'] = MagicMock() at
module level during collection. Any test collected after it imports
Session as a MagicMock. Pre-importing core.models in conftest.py
before test_agent_loop.py's module-level code runs prevents this.
* fix: make .history authoritative mutable list, address PR review
Per review feedback: keep .history as the authoritative mutable list so
existing code doing .history.pop(), .history = [...], etc. still works.
Fix the cross-contamination bug by ensuring __post_init__() gives each
Session its OWN unique history list (never shared).
Changes:
- core/models.py: .history IS the authoritative list. _history aliases it.
Each Session gets its own list in __post_init__.
- core/session_manager.py: add_message() delegates to Session.add_message()
instead of appending directly — no double-append, single source of truth.
- tests/test_session_manager.py: updated test to reflect that .history
references see new messages (same list, not a snapshot).
- docs/plans/2026-06-01-fix-chat-context-drifting.md: removed (not for
shipping — useful design context but too much process/doc to ship).
All 272 tests pass (3 pre-existing failures unrelated).
* Fix session manager message persistence
* Fix session history alias regressions
* Fix session history aliasing and task delivery
Three issues combined to make the per-user 'Allowed models' checklist
unreliable (#3032):
1. admin.js _loadModelsForUser fetched /api/models, which is backed by
cached_models — endpoints that haven't been probed yet (e.g. a
freshly-added DeepSeek API endpoint) simply didn't show up in the
checklist. Switched to /api/model-endpoints, which always reflects
every configured endpoint regardless of cache state.
2. _saveModels sent allowed_models: [] both when the admin clicked
[All] (no restriction) and [None] (block everything) — the backend
had no way to distinguish the two.
3. _enforce_chat_privileges treated an empty allowed_models list as
'no restriction' (falsy -> skip the check), so [None] had no effect.
Added an explicit block_all_models privilege flag (defaulting to False,
and forced to False for admins) that admin.js now sets when zero models
are checked. _enforce_chat_privileges checks it first and 403s
regardless of allowed_models contents.
* feat: Add ChatGPT Subscription support and related features
- Introduced a new provider option for ChatGPT Subscription in the endpoint selection UI.
- Implemented OAuth flow for ChatGPT Subscription sign-in, including polling for authorization status.
- Updated admin interface to handle ChatGPT Subscription, including disabling API key input and providing user guidance.
- Enhanced cost tracking logic to differentiate between subscription and non-subscription endpoints.
- Added new slash commands for managing skills, including listing, searching, and invoking skills.
- Implemented caching for skill catalog to optimize performance.
- Updated tests to cover new ChatGPT Subscription functionality and ensure proper endpoint probing.
- Refactored existing code to accommodate new features and improve maintainability.
* refactor: share provider device-flow setup
- reuse one device-flow backend for Copilot and ChatGPT Subscription
- add one frontend device-flow helper for Settings and /setup
- put GitHub Copilot back into Add Models, now as a dropdown option
- make provider selection just select; clicking Add starts sign-in
- stop ChatGPT Subscription setup from opening auth tabs automatically
- make /setup copilot and /setup chatgpt-subscription work from chat
- show ChatGPT Subscription in the /setup suggestions
- show the real error message when setup fails
- add focused tests for the shared flow and setup UI
* feat(chatgpt-subscription): harden credential lifecycle and streamline auth UX
Backend:
- Resolve runtime bearer for provider-auth endpoints at probe time via a
shared _resolve_probe_key() that delegates to resolve_endpoint_runtime,
applied across all probe/refresh call sites.
- Skip live completion probes and health pings for discovery-only providers
(centralized behind _is_discovery_only_provider) — the Codex/Responses API
has no such endpoints, so status is derived from cached models.
- Never persist the short lived ChatGPT bearer to the plaintext sessions
table; proactively clear any stale bearer left by an earlier code path.
- Revoke orphaned ProviderAuthSession credentials when the last endpoint
backing them is deleted (_delete_orphaned_provider_auth), surfaced via
cleared_provider_auth in the delete response.
Frontend (admin.js):
- Auto-start the device-auth flow on provider selection so the authorization
panel (code + Authorize) shows immediately instead of behind a "Sign in" click.
- Remove the redundant top button for device auth providers, move retry
into the panel via an inline "Try again".
- Drop the self-evident hint text and add an execCommand clipboard fallback so
Copy works in non-secure (HTTP/LAN) contexts.
* fix: harden chatgpt subscription provider
* chore: remove PR media from branch
* Fix chatgpt subscription recovery and token handling
---------
Co-authored-by: 5p00kyy <admin@5p00ky.dev>
* refactor(constants): single source of truth for data dir + merge core/src constants
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(contributing): use named src.constants for data paths, drop core/constants references
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(platform): add WSL compatibility functions and path translation
fix(cookbook): enhance model scan script to support additional HuggingFace cache paths
fix(hardware): improve cache key generation for remote SSH context
test(tests): add tests for WSL detection and path translation functionality
* fix(cookbook): prefer prebuilt wheels for llama-cpp-python and normalize package aliases
* fix: enable StrictHostKeyChecking in nvidia probe
refactor: consolidate ssh & powershell command execution to utility functions in core module
refactor: consolidate nvidia path candidates in to single variables in core module
tests: add tests for new utility functions
* fix: correct wrong variable name
PDF uploads are stored as markdown wrappers with pdf_source or pdf_form_source markers so the editor can preserve extracted text, form fields, and annotations. The library exposed that internal wrapper: auto-created PDF documents used the hashed storage filename as the title, and row/facet language reported markdown instead of pdf.
Derive chat-upload PDF titles from the original upload name, derive document-library display language from the PDF source marker for rows, filters, and facets, and keep markdown wrappers excluded from the markdown facet when they represent PDFs.
The expanded library card already renders PDF-backed documents through /api/document/{id}/render-pdf. Allow only that inline PDF preview endpoint to be framed by same-origin app pages while leaving normal routes on X-Frame-Options: DENY and frame-ancestors none.
Also tighten the existing PDF marker regression assertion so it matches the actual historical corruption signature instead of contradicting the preserved [Page 1 text]: marker.
Fixes#2468
#2753 made the agent loopback base port-configurable but only for
_COOKBOOK_BASE in tool_implementations. Several other in-process loopback
calls still hardcoded http://localhost:7000 and broke off port 7000:
cookbook_serve_lifecycle (model-endpoints x2, shell/exec), builtin_actions
(model/serve), task_routes (calendar x3), and the gallery/email calls in
tool_implementations.
Extract the resolution (ODYSSEUS_INTERNAL_BASE / APP_PORT / 7000 fallback,
127.0.0.1 to avoid IPv6 ambiguity) into core.constants.internal_api_base()
and route every call site through it. Rename the now-misnamed _COOKBOOK_BASE
to _INTERNAL_BASE since it serves gallery/email/calendar/serve too. Adds a
test for the resolver plus a regression guard against reintroducing the
literal.
Part of #2752.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* feat(platform): add support for Apple Silicon detection in platform compatibility
test(tests): enhance shell_routes tests for Apple Silicon compatibility
* fix issues with missing import
* fix: correct package name in package-lock.json and enhance package installation commands in shell_routes.py and cookbook.js
* feat: add Apfel startup and health checks on macOS
- bootstrap Apfel via Homebrew on arm64 macOS
- start `apfel --serve --port 11435` detached for Odysseus
- verify readiness via `/health`
- clean up the Apfel process on exit or Ctrl+C
* fix: duplicate variable declaration post-merge conflict
- Should fix `node` CI issues.
* fix: issues with the update status of the APFEL dependency.
- fixed by changing the main conditional that determines the update.
* Fix: Remove unnecessary whitespaces and formatting for the model_routes.py file.
* Fix: whitespace issues with the model_routes file
* Fix: Remove unnecessary whitespaces and formatting for the model_routes.py file. Final
* Fix: Fixed updates using PIP for APFEL instead of custom cmd
* feat: assign folder='Tasks' to task sessions at creation
Task sessions (LLM, action, research) now set folder='Tasks' on their
DbSession row, matching the pattern used by the Assistant folder. This
enables sidebar lens filtering without changing existing session
behaviour.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add backfill script for task session folders
One-shot script to set folder='Tasks' on existing [Task]/[Research]
sessions that predate the folder assignment in task_scheduler.py.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: replace standalone backfill script with automatic migration
Convert scripts/backfill_task_folders.py into _migrate_backfill_task_folders()
in core/database.py, called from init_db(). The migration is idempotent (only
touches rows where folder IS NULL/empty) and runs automatically on upgrade,
so operators no longer need a manual step to tag pre-existing task sessions.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
AuthMiddleware is the outermost middleware, so a credential-less CORS preflight
(OPTIONS + Access-Control-Request-Method) was rejected with 401 before
CORSMiddleware could answer it. That blocks every cross-origin browser/WebView
client: the preflight fails, so the real request is never sent.
Let a genuine preflight through at the top of AuthMiddleware.dispatch via a pure,
unit-tested predicate (core.middleware.is_cors_preflight). Precise -- only
OPTIONS carrying Access-Control-Request-Method; a credentialed request is never
matched -- and no data access.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(security): add HSTS and Permissions-Policy headers to SecurityHeadersMiddleware
Strict-Transport-Security is sent only when the connection is HTTPS
(detected via request.url.scheme or X-Forwarded-Proto: https), so
plain-HTTP dev deployments behind a reverse proxy are unaffected.
Permissions-Policy disables camera, microphone, and geolocation APIs
unconditionally — Odysseus does not use them, and this prevents a
successful XSS from requesting browser-native sensor access.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix(security): scope Permissions-Policy microphone directive to same-origin
Reviewers on PR #3081 (alteixeira20, NubsCarson) flagged that
microphone=() blocks mic access for same-origin (self) too, breaking
Odysseus's own voice/STT flow (getUserMedia({audio: true}) in
static/js/voiceRecorder.js). Scope it to microphone=(self) so
third-party origins stay locked out while the app's own UI keeps mic
access; camera and geolocation remain fully disabled as unused.
Adds focused middleware tests covering HSTS scoping (HTTPS direct,
X-Forwarded-Proto, absent on plain HTTP) and the Permissions-Policy
same-origin microphone contract.
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat(calendar): support multiple CalDAV accounts
Replaces the single CalDAV credential slot with a named account list so
users can sync both a personal and work calendar simultaneously.
- Add `account_id` column to `CalendarCal` + startup migration
- `_load_caldav_accounts()` in caldav_sync.py reads `caldav_accounts`
list from prefs, auto-migrating the legacy single `caldav` key on
first use (no user action required)
- `sync_caldav()` iterates all accounts and aggregates counts/errors
- `writeback_event()` resolves credentials via `CalendarCal.account_id`,
falling back to the first account for legacy rows
- New REST endpoints: GET/POST/PUT/DELETE `/api/calendar/config/accounts`
- Legacy GET/POST `/api/calendar/config` preserved for backward compat
- Settings UI: one card per account with Label, URL, Username, Password
fields; Test button works for both unsaved (inline creds) and saved
(by account_id) accounts; delete removes only that account
- Update test_caldav_url_hardening.py mock to include `_save_for_user`
and updated `_sync_blocking` signature
* fix(calendar): restore #2765 PK scoping and #2819 writeback URL validation
Two regressions introduced by the multi-account refactor:
1. PK collision (#2765): _stable_cal_id was back to hashing only the URL,
so two users — or one user with two accounts on the same server — would
collide on the primary key. Restore owner+account_id in the hash key
(format: "{owner}\n{account_id}\n{url}") and thread both values through
_sync_blocking → _writeback_blocking → push_event → find_remote_calendar
so the hash round-trips correctly on write-back.
2. URL validation dropped (#2819): _load_caldav_accounts imported
_save_for_user at function scope, causing an ImportError on test mocks
that only provide _load_for_user, which prevented writeback_event from
reaching the validate_caldav_url call. Move the import inside the
migration branch and wrap in try/except (best-effort save; next call
re-migrates from the still-present legacy key).
Update fake_writeback_blocking in test_caldav_writeback.py to accept the
new owner/account_id optional params.
This commit consolidates all Windows Cookbook background fixes into a single comprehensive commit based on the latest main branch.
Key fixes included:
1. React looksSuccessful Mismatch: Append 'DOWNLOAD_OK' for pip install commands in routes/cookbook_routes.py.
2. Local Windows SSH Wrapper & Log Directory Mismatch: Bypassed ssh wrappers and dynamically selected odysseus-tmux logs for local tasks in static/js/cookbookRunning.js.
3. WSL Bash Filtration: Filtered out the WSL bash stub at C:\Windows\System32\bash.exe in core/platform_compat.py.
4. Drive-Colon Path Normalization: Replaced .as_posix() with git_bash_path() in routes/shell_routes.py and src/bg_jobs.py.
5. GGUF-Only Hardware Fitting: Restructured local Windows recommendations to rank GGUF only in services/hwfit/fit.py.
6. Safe Win32 Process Liveness Probe: Replaced os.kill(pid, 0) with a safe Win32 API probe using GetExitCodeProcess in core/platform_compat.py.
7. Prebuilt llama-cpp-python Wheels: Supply the CPU extra index during compilation failure fallback.
8. Enforce UTF-8 log encoding: Set PYTHONIOENCODING=utf-8 on Windows bootstrap runners.
9. Fix Linux Llama.cpp Build script syntax error in routes/cookbook_helpers.py.
10. Page Reload Status Check: Run sys.executable instead of 'python3' to bypass Microsoft Store execution stubs on local Windows hosts.
11. Llama.cpp serve build bypass: Bypassed cmake compilation checks on local Windows and verified python bindings directly.
12. Serve Command Path Validation: Masked safe GGUF path printf subshells '' inside the serve command validator.
13. CPU Mismatch Diagnostics: Intercepted AVX2-lacking '0xc000001d' (Illegal Instruction) crashes in static/js/cookbook-diagnosis.js and guided users to Ollama.
14. Windows Pytest stability: Fixed stub import leakage in test files.
The CalDAV pull prunes events in the synced calendar+window whose UID the
server didn't just return, to propagate upstream deletions. But CalendarEvent
had no field distinguishing a server-pulled row from a locally-created one, so
the prune also deleted events that were never on the server: events created by
the agent / email triage (which never write back to the server) and UI events
whose best-effort write-back failed. Result: silent, unrecoverable loss of the
user's appointments (hard db.delete, no soft-delete).
Add an 'origin' column to calendar_events (lightweight idempotent migration,
mirroring _migrate_add_calendar_is_utc), set origin='caldav' on rows the sync
inserts/updates, and gate the prune on origin == 'caldav'. Locally-created
events carry origin NULL and are never pruned. On the first sync after the
migration nothing is pruned (all rows NULL until re-marked), erring toward
keeping data.
Fixes#2704
* feat(mcp): add Streamable HTTP transport with OAuth 2.0
Odysseus could only reach MCP servers over stdio and SSE, so modern
remote servers like https://mcp.higgsfield.ai/mcp (Streamable HTTP,
gated behind OAuth) could not be connected.
Add an `http` transport that connects via the SDK's
streamablehttp_client and authenticates with the SDK's
OAuthClientProvider: RFC 9728 protected-resource discovery, RFC 8414
authorization-server metadata, Dynamic Client Registration,
authorization-code + PKCE, and token refresh. A small bridge
(src/mcp_oauth.py) connects the SDK's blocking callback to the existing
web callback route via an asyncio.Future keyed by the OAuth `state`,
and the dynamic client registration plus tokens persist per-server in a
new encrypted `oauth_tokens` column.
The connect runs as a bounded background task so the "Add server"
request returns immediately; redirect_handler publishes needs_auth +
auth_url to connection state as soon as discovery/DCR completes (which
can exceed the bounded wait), and the UI polls until connected. Remote
users finish via the existing paste-back flow. The Google OAuth path is
left unchanged.
- core/database.py: encrypted oauth_tokens column + migration
- src/mcp_oauth.py: OAuth provider, DB-backed TokenStorage, state registry
- src/mcp_manager.py: http dispatch, background connect, _connect_http
- routes/mcp_routes.py: http validation, needs_auth/auth_url, callback bridge
- static/js/settings.js: Streamable HTTP option + OAuth flow with polling
- tests: 5 new unit tests (transport dispatch, registry, token storage)
Verified against the live Higgsfield server: discovery, DCR (client_id
issued), loopback redirect accepted, and a PKCE authorization URL with
needs_auth status. No regressions (full suite delta is only the 5 added
passing tests).
* fix(mcp): address PR #1033 review feedback
- mcp_oauth: derive redirect URI from OAUTH_REDIRECT_BASE_URL/APP_PUBLIC_URL
(default http://localhost:7000) instead of hardcoding the port
- mcp_oauth: leave OAuth scope unset so the SDK derives it from the server's
WWW-Authenticate/protected-resource metadata; hardcoding an OIDC scope broke
non-OpenID MCP servers (verified: Higgsfield still gets its server-derived
scope)
- mcp_oauth: prune abandoned OAuth flows (_prune_stale + _pending_ts) so the
module-level registries can't grow unbounded
- mcp_oauth: persist tokens/client-info in a single DB session/commit
(_update) instead of a load+save double round-trip
- mcp_manager: cancel and drop the background connect task in
disconnect_server so a deleted server stops publishing status
- database: document why the oauth_tokens migration uses TEXT while the model
declares EncryptedText (encryption is applied at the Python layer)
- settings.js: surface persistent OAuth-poll failures and an explicit timeout
message instead of silently swallowing errors
- tests: cover the stale-flow pruning
* static/js/settings.js now shows an in-flight loading state on the buttons that fire requests:
Slash-command replies and the echoed /setup command are persisted to session
history so they render in the transcript, but they are UI chatter the user
never meant as conversation. They were sent to the model on the next turn,
which then commented on '/setup ...' and exposed transient values (e.g. the
Copilot device user_code) to the LLM.
- get_context_messages() (the LLM-API view) now skips messages tagged
metadata.source == 'slash'. Display/history-load paths use raw history and
are unaffected.
- slashCommands.js tags the echoed user command with source:'slash' too (the
assistant replies already carried it); the user line was the one untagged
path that still reached context.
Fixes#2634.
truncate_messages deletes db_messages[keep_count:] (a no-op when
keep_count >= the real message total) then unconditionally wrote
db_session.message_count = keep_count. When keep_count exceeds the
number of messages that actually exist — e.g. the manage_session AI
tool defaults keep_count to 10, and the HTTP truncate endpoint passes
any client value — the persisted count is set too high (10 on a
3-message session), diverging from the real row count. That column
gates lazy DB-hydration in get_session (message_count > 0) and is
surfaced to the history UI, so it is correctness-relevant. Clamp to
min(keep_count, len(db_messages)); the in-memory slice already caps
naturally.
* fix: support large proxy model endpoint refresh
Large OpenAI-compatible proxy endpoints can expose hundreds of models and make /v1/models slow. Treating those endpoints like local model servers caused model picker opens and background probes to repeatedly hit /models, producing timeouts and making otherwise usable endpoints appear offline.
Make model endpoint discovery cached-first for normal UI usage, add explicit proxy/API classification and refresh policy fields, exclude proxy/API endpoints from aggressive local probing, and preserve cached models when refresh fails.
Manual Test/Add/Refresh actions still fetch the full model list with longer timeouts so users can intentionally import large proxy model lists without blocking normal model picker usage.
* fix: preserve endpoint ping status semantics
* fix: revoke API bearer tokens when their owner is deleted
* Re-run CI
* Invalidate bearer-token cache on user delete so warmed cached tokens stop working
Use func.lower() when updating SQL owner columns, match prefs keys
case-insensitively, and normalize session usernames before comparing
during rename. Prevents silently skipping legacy mixed-case owner data.
Fixes#1165
Multimodal content (list of {type, text/image_url} blocks) couldn't be
stored in the DB Text column, causing silent persist failures. On reload
the frontend fell back to String() on the array, rendering
[object Object],[object Object] in the chat.
- Serialize list content as JSON in _persist_message()
- Deserialize back to list in _db_to_session() via _parse_msg_content()
- Extract text parts from multimodal arrays in sessions.js instead of
String() coercion
A session that exists only in the in-memory SessionManager — never persisted,
or whose DB row was removed out-of-band — was listed by GET /api/sessions (the
list is built from the in-memory manager) but 404'd on every per-session
operation, so it could never be deleted.
Two causes, both fixed:
1. _verify_session_owner() only consulted the DB and raised 404 when no row
existed. It now falls back to the in-memory session's owner when (and only
when) a session_manager is supplied and the caller actually owns the ghost.
The DB row stays authoritative when present, and a ghost owned by another
user still 404s, so the ownership/security model is unchanged. The new
parameter defaults to None, preserving behavior for all other callers.
2. SessionManager.delete_session() only removed the in-memory entry when a DB
row was found, so memory-only ghosts survived. It now drops the in-memory
copy regardless and reports success when either the DB row or the in-memory
entry was removed.
Added tests/test_session_ghost_delete.py covering both layers, including the
cross-owner 404, the unauthenticated 403, DB-row-wins precedence, and backward
compatibility when no manager is passed.
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(db): enable SQLite foreign keys so ondelete cascades actually fire
core/database.py declares DB-level FK actions throughout
(ondelete="CASCADE" / "SET NULL"), but SQLite disables foreign-key
enforcement per connection by default and the engine had no connect-event
listener turning it on. So every one of those ondelete actions was dead.
Concrete impact: cleanup_old_sessions() in src/cleanup_service.py removes
old sessions with a bulk `query(Session).delete()`, which bypasses the
ORM-level relationship cascade and relies solely on the DB-level
ondelete="CASCADE" on ChatMessage.session_id. With foreign keys off, the
messages are never deleted — they pile up as orphaned rows on every
cleanup cycle.
Add the standard SQLAlchemy connect listener issuing `PRAGMA
foreign_keys=ON`, guarded by `isinstance(conn, sqlite3.Connection)` so it
only affects SQLite and leaves other backends untouched.
tests/test_sqlite_foreign_keys.py inserts a Session + ChatMessage, deletes
the Session via bulk `query().delete()`, and asserts the ChatMessage is
cascade-deleted. Fails before this change (orphan remains).
* docs(db): clarify FK pragma scope per review; trim test comments
Address review feedback on the foreign_keys PRAGMA change:
- Note that the class-level connect listener fires for every Engine in the
process and is a no-op on non-SQLite backends (isinstance guard).
- Warn near init_db() that FK enforcement is now global, so a migration
that temporarily violates FK constraints must disable foreign_keys around
that work.
- Drop the step-by-step narration comments from the regression test.
No behavior change.
After a successful password change, revoke all browser sessions for the
same user except the one that submitted the request. This prevents stale
sessions on other devices from remaining valid after credentials are
updated.
Keep API-token behavior unchanged. The current browser session is
preserved so the user can continue from the tab that changed the
password.
Add focused regression tests for preserving the current session, revoking
other sessions, persisting revocation, and avoiding revocation when the
current password is incorrect.
`core.middleware.require_admin` grants admin to any request whose
`request.state.current_user == "internal-tool"` — the sentinel meant only
for the in-process tool-loopback path. But the normal cookie auth path
(app.py) sets `current_user` to the raw username, and neither `create_user`
nor the signup route reserved that name. As a result an account literally
named "internal-tool" was silently treated as admin by every
`require_admin`-gated route. With self-service signup enabled this is an
anonymous -> admin privilege escalation.
Reserve the full synthetic-owner set the codebase already special-cases —
"internal-tool", "api", "demo", "system" (see `_SYNTHETIC_OWNERS` in
routes/assistant_routes.py and the matching guards in src/task_scheduler.py
and routes/research_routes.py). "api" collides with the bearer-token owner
sentinel; "demo"/"system" would leave a real account denied an assistant
and inconsistently owner-scoped.
Refuse to create or rename into any reserved name (case/space-normalized),
and reject empty usernames while we're here. Adds a regression test.
Co-authored-by: Claude <noreply@anthropic.com>
verify_password() and create_session() both call .strip().lower() on
the incoming username, but _load() stored keys verbatim from auth.json.
Any mixed-case key (e.g. written by manual edit or a future migration)
would never match, producing a permanent 'Invalid credentials' error.
Fix: lowercase all keys at load time so the in-memory dict always
matches what the login path expects.
Fixes#423
Hriday Ranka