* fix(routes): serve 404 instead of 500 when an HTML page file is missing
_serve_html_with_nonce opened the HTML file with no error handling, and
callers such as /backgrounds and /login pass their paths in with no
existence check, so a missing or unreadable file raised an unhandled
OSError that surfaced as a 500. Wrap the read and raise HTTPException(404)
instead; the normal render path (CSP-nonce substitution) is unchanged.
Fixes#4594
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(routes): distinguish missing page (404) from read failure (500)
The previous fix caught a broad OSError and returned 404 for every
failure, which masks real server-side problems (permission errors, I/O
failures) as "not found" and lets them slip past error alerting. Split
FileNotFoundError (genuine 404) from other OSError, which now logs the
exception and returns a generic 500 — without leaking the OS error
string or file path into the response body.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(routes): treat unreadable bundled HTML page as logged 500, not 404
Per PR #4637 review: every caller of the page-render helper serves a fixed,
server-owned template (index/login/backgrounds), never a client-supplied
path. So a missing or unreadable file is a server fault (broken deployment),
not a client "not found" — a 404 there mislabels a server error and hides a
missing core template from 5xx alerting, contradicting the OSError->500
rationale this PR is built on. Collapse both branches into a single logged,
leak-free 500.
Move the helper to src.app_helpers.serve_html_with_nonce so the behavior can
be unit-tested without importing the whole app (app.py is the slim
orchestrator; the test harness stubs src.database, so importing app in tests
is not viable). Add tests pinning missing/unreadable -> 500 (not 404) and
nonce injection on the happy path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(chat): strip executed email tool fences from the live stream (#3993)
The backend strips every fenced tool block from persisted text (the regex in
src/tool_parsing.py is built from the full TOOL_TAGS set, which includes the
email tools), so a reloaded session renders cleanly. The live frontend path
uses a separate hardcoded EXEC_FENCE_RE in static/js/chatRenderer.js that only
listed web_search/read_file/write_file/create_document/edit_document/
update_document — so executed email tool fences (list_emails, etc.) lingered as
raw code blocks in the live assistant bubble until the user reloaded.
Add the nine email tool tags to EXEC_FENCE_RE so the live render settles into
the same clean layout as the history reload. bash/python stay excluded on
purpose: those are languages a user may legitimately have asked the model to
show as code, not tool invocations.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(chat): single-source live exec-fence tool list from TOOL_TAGS (#3993)
Per review: EXEC_FENCE_RE was a second, hand-maintained copy of the
executable-tool list, so any tool not in it — and every future tool added to
TOOL_TAGS — would leave its executed fence lingering in the live bubble until
reload (the original #3993 bug, recurring one tool at a time).
EXEC_FENCE_RE is now built from an explicit EXEC_TOOL_TAGS list that mirrors
TOOL_TAGS (src/agent_tools/__init__.py) minus bash/python, which stay excluded
as legitimate code-example languages. A new regression test
(test_exec_fence_re_covers_all_executable_tools) extracts both lists from
source and fails if they drift, so the whole class is caught in CI instead of
by a user — the "minimum acceptable middle ground" from the review, made exact
(set equality, not just coverage).
Verified: pytest tests/test_live_strip_email_tool_fences.py (5 passed);
node --check static/js/chatRenderer.js; and a node run of the built regex
confirms email/generate_image/manage_memory/ls fences strip while
bash/python/sh are preserved.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(chat): build live exec-fence list from /api/tools at runtime (#3993)
Make TOOL_TAGS the single source for live exec-fence stripping. chatRenderer.js
no longer hard-codes a tool list; it fetches the backend's authoritative set
once from GET /api/tools (sorted(TOOL_TAGS)) and builds EXEC_FENCE_RE from it at
load, minus bash/python. No second list to drift, and a future tool added to
TOOL_TAGS is covered automatically — without touching the streaming path.
Until the fetch resolves EXEC_FENCE_RE is null and exec fences aren't stripped
(a sub-second window before the first stream); the backend already strips
persisted history, so a reload always renders clean.
Drop test_exec_fence_re_covers_all_executable_tools (no hand-maintained list to
guard) and add source-level guards: the frontend keeps no hard-coded list and
fetches /api/tools, and the endpoint serves the full sorted(TOOL_TAGS).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CVCKth4g8pWh7pwFDVm4iL
* fix(chat): warn on /api/tools fetch failure instead of swallowing it (#3993)
A fresh-context review flagged that loadExecFenceRegex's catch silently
discarded errors: if the one-shot fetch fails, EXEC_FENCE_RE stays null for the
whole session and live exec fences go unstripped until reload, with zero signal.
console.warn it, and correct the comment to describe the failure mode honestly
(was understated as just a sub-second startup window).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CVCKth4g8pWh7pwFDVm4iL
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: Images cannot be seen by model that is vision capable
* fix: skip http(s) image_url for Ollama (images[] is base64-only)
---------
Co-authored-by: michaelxer <michaelxer@users.noreply.github.com>
* fix(llm): detect mistral.ai provider and support reasoning_effort
Four coupled bugs broke Mistral thinking model support:
1. _detect_provider() had no mistral.ai host check, so all Mistral
endpoints fell through to the generic 'openai' provider string.
_provider_display_name() correctly identified them as 'Mistral',
making any 'if provider == "Mistral"' check elsewhere dead code.
2. reasoning_effort parameter was never sent in the request payload,
so Mistral never activated thinking mode even when the user
configured a thinking-capable model (mistral-small-latest,
mistral-medium-latest, magistral-*).
3. Mistral returns content as a typed array
([{"type":"thinking",...},{"type":"text",...}]) when
reasoning is on, not as a plain string. Both the streaming and
non-streaming parsers expected strings and silently dropped the
thinking content.
4. _THINKING_MODEL_PATTERNS didn't include magistral or mistral-*
model prefixes, so the frontend wouldn't tag reasoning output
as thinking even after the above were fixed.
Fix:
- Add mistral.ai to _detect_provider() host checks
- Add a _normalize_mistral_content() helper that splits the typed
array into (text, thinking) strings
- Inject payload["reasoning_effort"] = "high" when provider is
Mistral and _supports_thinking(model) is true, in both stream_llm
and llm_call_async payload construction
- Wire the normalizer into both response parsers
- Extend _THINKING_MODEL_PATTERNS to include magistral,
mistral-small, mistral-medium, mistral-large
Tested on Docker install with mistral-small-latest +
reasoning_effort=high. Reasoning streams correctly into the
thinking panel after the fix.
Fixes#4678
* fix(llm): address review — lowercase provider id, configurable effort, tests
Addresses vdmkenny's review on PR #4698:
1. Removed duplicate 'if provider == "mistral"' block in stream_llm
— two back-to-back copies, one was dead-redundant.
2. Dropped personal-context comment ('free-tier limits are generous
for this user') and made reasoning_effort configurable via env var
ODYSSEUS_MISTRAL_REASONING_EFFORT (high / medium / low / none).
Default remains 'high' for backward compat with the tested behavior.
3. Recased provider id from 'Mistral' to 'mistral' to match the
lowercase convention used by every other provider id in the file
(openai, anthropic, ollama, copilot, ...). _provider_display_name()
still returns the Title-Case 'Mistral' for UI labels — only the
runtime id used in 'if provider == ...' checks was recased.
4. Added tests/test_llm_core_mistral_content.py with 13 tests pinning
_normalize_mistral_content()'s contract: string passthrough, the
Mistral array format (thinking + text blocks), and edge cases
(empty, garbage, None, wrong types, missing fields, string-vs-array
inner thinking field).
Also fixed a gap the review didn't catch: the non-streaming paths
(llm_call sync + llm_call_async) were missing the reasoning_effort
injection entirely. Added the same injection to both, so Deep Research
and agent tool calls also activate Mistral thinking.
All 13 new tests pass. Existing reasoning/streaming/ollama-thinking
tests still pass (38 tests, no regressions).
Fixes#4678
* fix(memory): keep the Brain memory item menu above the modal at any stack depth
The memory item "⋮" dropdown is portaled to <body> with a hardcoded
z-index of 10001. Tool modals, however, get a monotonically increasing
z-index from modalManager's bring-to-front counter (_modalTopZ), which
climbs unbounded as modals are opened/restored over a session. Once that
counter passes 10001, the Brain modal stacks above the body-portaled
dropdown, so the menu renders behind the panel — visible only where it
spills past the modal's edge (#4720).
Derive the dropdown's z-index from the owning modal's current z-index
(+1), keeping 10001 as a floor for the common low-counter case, so the
menu always sits just above its modal however high the counter has climbed.
Verified with document.elementFromPoint at the dropdown's location: with a
high modal z-index the old build returns the modal at every sampled point
(menu behind); the fixed build returns the dropdown (menu on top). The
default low-counter case is unchanged (z stays 10001).
* refactor(modal): route body-portaled dropdowns through a shared topPortalZ() helper
The hardcoded z-index:10001 the Brain memory menu used (#4720) is the same
literal shared by ~16 body-portaled dropdowns across calendar, cookbook,
cookbookServe, documentLibrary, emailLibrary, gallery, notes, emojiPicker and
memory — each renders behind its owning tool modal once modalManager's
bring-to-front counter climbs past the literal over a long session.
Promote the per-dropdown fix into a single topPortalZ() helper in
toolWindowZOrder.js — the existing source of truth for tool-window z, already
imported by modalManager's _bringToFront and notes.js — returning
max(topToolWindowZ(), dock-chip floor) + 1, so a portaled dropdown always sits
just above the live tool-window stack however high the counter has climbed.
Route all 16 sites through it. The slashCommands tour tooltips and the
cookbookServe VRAM dialog are intentionally left out (neither is a modal-owned
portaled dropdown).
Add tests/test_portal_dropdown_z_js.py covering the helper, including the #4720
scenario (modal counter at 99999 -> dropdown at 100000). Existing
test_notes_z_order_js.py stays green.
* fix(security): redact credential-bearing URLs and PII from logs
Several log statements emitted sensitive data in clear text:
- model_routes / chat_routes / contacts_routes logged endpoint URLs raw.
Admin-configured URLs can embed credentials in userinfo or query
(e.g. https://user:pass@host, ?api_key=...). Route them through a
shared core.log_safety.redact_url() that drops userinfo/query/fragment.
- note_routes / task_scheduler logged operator email addresses (smtp_user,
recipient). Replaced with presence booleans, which keeps the diagnostic
("why didn't this send") without writing PII to logs.
model_routes already had a local redactor on its HTTPStatusError branch;
the generic except branch was missed, so reuse the existing helper there.
Clears CodeQL py/clear-text-logging-sensitive-data alerts 264, 317, 324,
325, 343, 344, 528.
* fix(security): re-bracket IPv6 hosts and single-source the URL redactor
Address review on #4750:
- redact_url now re-brackets IPv6 literals so host:port stays
unambiguous (https://[2001:db8::1]:8443/v1, not the bracket-less
ambiguous form).
- point model_routes._redact_url_for_log at the shared helper so the
two redactors are single-sourced (also picks up the IPv6 fix).
* fix(security): escape backslashes in calendar bg-image CSS url()
The calendar event-background CSS escaped ' -> \' for a bg: image URL but
not backslashes first. Inside a single-quoted url('...'), \ is the CSS
escape char, so a URL value ending in/containing a backslash escapes the
closing quote and breaks out of the string, injecting arbitrary CSS. The
bg:<url> value is per-event and CalDAV-syncable, hence untrusted (CodeQL
js/incomplete-sanitization).
Add a single canonical _cssUrlEscape() in calendar/utils.js that escapes
backslashes FIRST, then quotes, and route all four sinks through it:
calendar.js:416 / :1263 (the flagged #463/#464), the event-form preview
(:2931), and _calBgCss() in utils.js — the latter two share the identical
bug but were unflagged. Output is byte-identical to the old escaping for
legitimate URLs (which contain no backslashes); only malicious input differs.
Resolves CodeQL js/incomplete-sanitization #463, #464.
* fix(security): route remaining calendar bg url() sinks through _cssUrlEscape
Review (vdmkenny) flagged that the centralization missed an injectable
sibling sink: the edit-form color-picker swatch (calendar.js:2856) built
`url('${url}')` from `existing.color` (a CalDAV-syncable, untrusted `bg:`
value) raw, then interpolated it into `style="background:..."` via innerHTML
- the same `'`/`\` breakout class as the sinks already fixed. The custom-dot
preview (:2953) was likewise raw (non-exploitable - a CSSOM `.style`
assignment of a URL the current user just picked - but it broke the invariant).
Route both through `_cssUrlEscape`, and normalize the two pre-escaped-variable
sites (_calItemBgStyle, _renderWeek) to the same inline form so all five
url() interpolations in calendar.js follow one rule. Add a whole-file
invariant test asserting every `url('${...}')` calls `_cssUrlEscape` - this
catches a future missed sink, the exact failure mode here. Behavior-identical
for legitimate URLs (no visual change).
* fix: document read fails with 403 when auth is disabled
Add _auth_disabled() bypass in _verify_doc_owner() and the
/api/documents/{session_id} route guard so documents remain accessible
in single-user / no-auth mode.
Minimal change: only adds the auth-disabled check alongside existing
403 raises — preserves existing formatting and line endings.
* refactor: hoist _auth_disabled import to module level
Address reviewer feedback on PR #4623 — no circular import exists
(src.auth_helpers only imports stdlib + fastapi), so the inline
imports are unnecessary. Moves the import to module top in both
document_helpers.py and document_routes.py.
* test: add regression tests for auth-disabled document access (PR #4623)
Remote Cookbook hwfit probes failed on Windows hosts because the PowerShell script was sent as nested -Command quoting through OpenSSH. Use -EncodedCommand for remote probes, auto-detect platform when omitted (including Darwin for Mac SSH hosts), and return a clearer error when SSH works but the probe fails.
Co-authored-by: Cursor <cursoragent@cursor.com>
Inline backtick spans were converted to <code> only at the end of
mdToHtml, after the bare-URL autolink and <a>/allowed-HTML passes. A URL
inside inline code is preceded by a space, so the autolink wrapped it in
an <a> tag and swapped it for an ___ALLOWED_HTML_ placeholder, corrupting
commands like `irm http://127.0.0.1:3000/x`.
Extract inline code into placeholders before the link passes, mirroring
the existing fenced-code-block handling, and restore them last so
placeholders carried inside restored <a> blocks resolve. Escape the code
at extraction time since it now bypasses the global escape pass.
The fallback regex in email_pollers.py that recovers a
[{"action": ...}, ...] JSON array from raw model output used lazy
[^[\]]*? runs inside a (?:,\s*\{...\}\s*)* repetition, which backtracks
exponentially (CodeQL py/redos) on inputs like [{"action"},{ + }},{{ * N.
It runs on the LLM reply to an email→calendar prompt embedding the
untrusted email body, so a crafted email can stall the background poller.
Extract the pattern to a module-level _CAL_ACTION_ARRAY_RE and rewrite the
object-content class from the lazy [^[\]]*? to a greedy brace-delimited
[^{}], which removes the quantifier ambiguity. The match is linear (a 500KB
adversarial input now resolves in <1ms) and equivalent on well-formed
arrays; it is also strictly more robust for values containing '[' or ']'
(the old class bailed on those and extracted nothing).
Resolves CodeQL py/redos #198.
The DELETE /api/personal/file disk-delete containment check used the
shared PERSONAL_UPLOADS_DIR root, so one admin could delete another
user's personal upload by passing its path (uploads are partitioned per
owner under <root>/<owner>/). Confine the check to the caller's own
per-owner subdir via _personal_upload_dir_for_owner(owner). RAG removal
and listing exclusion are unchanged (they still serve non-upload indexed
sources). Adds a regression test for the cross-owner case.
Moves create_session, list_sessions, send_to_session and manage_session out of
ai_interaction.py into src/agent_tools/session_tools.py (the do_ prefix
dropped) and registers them in TOOL_HANDLERS, so dispatch flows through the
registry instead of the dispatch_ai_tool elif in tool_execution.py. Same
pattern as the model-interaction move.
The bodies move verbatim; each fetches the runtime-set session manager via a
get_session_manager() shim, and reuses _resolve_model / AI_CHAT_TIMEOUT from
ai_interaction. manage_session's internal 'list' alias is repointed from the
old do_list_sessions to the moved list_sessions. stream_ai_tool (dead, no
callers) and do_pipeline stay put. dispatch_ai_tool loses its four now-unused
branches.
Tests: test_session_tools_registry covers registration, owner threading, the
manage_session->list_sessions delegation, graceful no-manager handling, and
registry dispatch. Verified end-to-end against a live SessionManager.
Detached bash jobs (#!bg) could be launched and auto-reported on completion,
but the agent had no way to act on a running one: no on-demand output read and
no kill (it blocked until the 1h max-runtime). bg_jobs had the pieces
(_read_output, list_for_session, internal _kill) but none was exposed.
Adds:
- bg_jobs.kill(job_id): tears down the process tree, marks the job killed, and
sets followed_up so the monitor does not also auto-continue a deliberate kill.
- manage_bg_jobs registry tool with actions list / output / kill, scoped to the
chat that launched the job (cross-session access reads as not found).
- Wiring: TOOL_HANDLERS/TAGS, function schema, RAG index + keyword hints, parser
name map, dispatch (threads session_id via _direct_fallback). Gated like bash
(NON_ADMIN_BLOCKED_TOOLS; plan-mode mutator).
- agent_loop: background-job intent regex maps to the files domain (and the tool
joins _DOMAIN_TOOL_MAP[files]) so short commands like 'kill that job' are not
dropped by the low-signal gate that skips tool retrieval.
- bg launch message tells the model to call manage_bg_jobs itself for check/stop
rather than printing raw tool syntax to the user.
Tests: tests/test_bg_job_tools.py (kill semantics, per-chat scoping, actions,
and the intent classifier).
* fix(tools): prune skipped dirs before descending in glob tool
GlobTool used pathlib.Path.rglob which descends into every directory
(including node_modules, .git, dist, etc.) and filters AFTER the walk.
On repos with large junk directories this causes the glob tool to hang
for minutes.
Replace rglob with os.walk that prunes _CODENAV_SKIP_DIRS before
descending — matching the approach GrepTool already uses. Also add a
fast path for literal patterns (no wildcards → direct path lookup).
Fixes#4493
* fix(tools): use regex glob matching to fix * semantics and literal fallback
Replace fnmatch with _glob_to_regex so that * stays within a single
path segment (matching pathlib/rglob semantics) and **/ spans zero or
more directories. Literal patterns now fall through to os.walk when
the direct path lookup misses, so e.g. 'foo.py' still finds files at
any depth.
Add tests for:
- bare literal matching in subdirectories
- multi-segment single-star patterns (sub/*.txt)
- * not crossing / boundaries
- ** matching at arbitrary depth
Closes#4493
---------
Co-authored-by: michaelxer <michaelxer@users.noreply.github.com>
The Dependencies tab's llama.cpp docker recipe surfaced
\`docker pull ghcr.io/ggerganov/llama.cpp:server-cuda\`. The upstream
repo moved from github.com/ggerganov/llama.cpp to
github.com/ggml-org/llama.cpp and the old GHCR namespace no longer
publishes images, so copying the recipe failed with:
failed to resolve reference "ghcr.io/ggerganov/llama.cpp:server-cuda":
not found
Point the recipe at \`ghcr.io/ggml-org/llama.cpp:server-cuda\`, which is
already the namespace routes/cookbook_routes.py uses for the source
clone. Adds a regression test in the same shape as
test_cookbook_diagnosis_js.py asserting the new namespace and forbidding
the dead one.
No CSS/HTML/SVG/style changes — the file is a pure data module
(no DOM access) consumed by other renderers; only the displayed command
text changes.
Two background tasks scheduled on every chat completion in
routes/chat_helpers.py — the memory/skill extraction dispatch and the
session auto-namer — are created via bare asyncio.create_task(...).
asyncio only holds a weak reference to the outer task, so the GC can
collect it mid-execution and the work silently never runs.
Add a module-private _BG_TASKS set and a _spawn_bg() helper that mirrors
WebhookManager._spawn_tracked (the pattern #3964 / #4336 established for
the webhook emitters two lines apart in the same function). Route both
call sites through it so the lifecycle owner is explicit.
Adds an AST-level guard test that fails on any bare
asyncio.create_task(...) statement in routes/chat_helpers.py to prevent
a regression — same shape as test_webhook_emitters_use_manager.py from
#4336.
The same bare pattern exists in routes/email_routes.py and
routes/cookbook_routes.py; left out of this PR per CONTRIBUTING.md's
"one fix per PR" and tracked in #4443's "Additional Information" for a
follow-up.
The persistent login cookie's max_age hardcoded 60 * 60 * 24 * 7, an
independent copy of the session token lifetime that core/auth.py already
defines once as TOKEN_TTL (and reports to the frontend via /api/auth/policy
as session_days). If TOKEN_TTL changes, the cookie silently drifts: the
browser keeps a cookie for a token whose lifetime no longer matches.
Import TOKEN_TTL and use it for the cookie max_age so the session lifetime
has a single source of truth. No behaviour change at the current value.
Fixes#4471
The harmony stream router only recognized the analysis and final channels, so
gpt-oss's standard `commentary` channel (tool-call preambles / function-arg
bodies) was unhandled: the literal `<|channel|>commentary` marker, the
`to=functions.*` recipient, and the commentary body all leaked into the
visible answer. Add commentary to the marker regex + the suffix-hold table, and
route its body to thinking (only `final` is user-facing). Adds a regression
test (split-chunk + recipient + body), verified to fail without the fix.
_patch_prefs installs a fake routes.prefs_routes with a bare
sys.modules[...] = assignment that is never undone. The stub is an empty
ModuleType without _save_for_user, so a later test whose code path runs
`from routes.prefs_routes import _save_for_user` (e.g. test_backup_import_skills)
fails with ImportError under an unfavorable test order.
Install the stub with monkeypatch.setitem instead (the helper already takes
monkeypatch and uses it for DATA_DIR) so it is reverted at teardown.
Repro: pytest tests/test_skill_index_prompt_injection.py tests/test_backup_import_skills.py
(1 failed before, 5 passed after).
* fix(agent): index api_call so RAG tool selection can retrieve it
api_call exists in FUNCTION_TOOL_SCHEMAS and the agent's system prompt
advertises configured API integrations, but the tool had no entry in
BUILTIN_TOOL_DESCRIPTIONS. RAG tool selection embeds those descriptions and
retrieves the top-K per message, so a tool without one can never be selected:
the agent claims it can call Home Assistant/Miniflux/Gitea/etc. and then
never receives the api_call schema (unless the Personal Assistant
ASSISTANT_ALWAYS_AVAILABLE path applies).
Add a retrieval-rich description for api_call, plus an ast-based parity test
asserting every FUNCTION_TOOL_SCHEMAS tool has an index description so the
next added tool cannot silently drift the same way.
Fixes#3794
* fix(agent): route API-integration intent to api_call at selection time
Addresses review (RaresKeY) on #3923: indexing api_call in the ToolIndex
description was necessary but not sufficient — the #3794 repro ('Use the
api_call tool to call Home Assistant GET /api/states') matched no domain in
_classify_agent_request, classified as low-signal, so the agent loop skipped
retrieval entirely and the schema filter sent only ALWAYS_AVAILABLE
(manage_memory/ask_user/update_plan). api_call never reached the model.
- _classify_agent_request: detect API-integration intent (api_call,
integration(s), Home Assistant/Miniflux/Gitea/Linkding/Jellyfin) -> new
'integrations' domain, so the turn is no longer low-signal.
- _DOMAIN_TOOL_MAP['integrations'] = {api_call}: deterministically seeds
api_call into relevant tools after retrieval, independent of embeddings.
- _DOMAIN_RULES['integrations']: rule pack (required — _domain_rules_for_tools
indexes _DOMAIN_RULES[domain] directly).
- tool_index _KEYWORD_HINTS: parity hint for the retrieval / keyword-fallback
paths.
- Regression drives the real classifier -> domain-map -> FUNCTION_TOOL_SCHEMAS
filter chain and asserts api_call is advertised for the #3794 prompt.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(document): allow render-pdf to be framed and 503 cleanly on missing PyMuPDF
Fixes#2101.
Two related bugs in the PDF-form library preview flow:
1. SecurityHeadersMiddleware was sending X-Frame-Options: DENY and
frame-ancestors 'none' on /api/document/{doc_id}/render-pdf, but
static/js/documentLibrary.js embeds the response in an <iframe> for
the library card preview. The browser blocked the load with
ERR_BLOCKED_BY_RESPONSE, leaving the user with a blank panel.
Extend the existing is_tool_render exemption to also cover
/api/document/.../render-pdf. Per-document owner checks still run in
the route handler, so the exemption is scoped the same way as the
tool-render exemption it mirrors. /api/document/.../export-pdf is
left untouched — it's a download (Content-Disposition: attachment),
not an iframe embed.
2. routes/document_routes.py:render_pdf called fill_fields, which
raises RuntimeError via _require_fitz() when the optional PyMuPDF
dependency isn't installed. That RuntimeError bubbled out as a
generic 500 with a cryptic 'PDF render failed' detail.
Reuse the existing _load_pdf_viewer_fitz() helper to fail fast with
a 503 and a user-actionable install hint (mentions
requirements-optional.txt and AGPL-3.0), matching the convention
used by the other PDF endpoints.
Tests cover both fixes:
- middleware headers on /api/document/.../render-pdf (iframeable, but
X-Content-Type-Options and Referrer-Policy are still set)
- middleware headers on /api/document/.../export-pdf (must stay strict)
- middleware path matching precision (similar-but-different paths stay
strict)
- middleware headers on /api/tools/.../render (no regression)
- middleware headers on /api/chat (no regression)
- render-pdf returns 503 with install hint when PyMuPDF is missing
- 503 is raised before any file I/O (fail-fast ordering)
* chore: address maintainer feedback on PDF previews same-origin framing and comment trimming
* chore: make render-pdf regression tests order-independent
Moves chat_with_model, ask_teacher and list_models out of ai_interaction.py
into src/agent_tools/model_interaction_tools.py (the do_ prefix dropped) and
registers them in TOOL_HANDLERS, so dispatch flows through the registry instead
of the dispatch_ai_tool elif in tool_execution.py.
The implementations are relocated, not wrapped. ai_interaction.py keeps only
the shared helpers they reuse (_resolve_model, AI_CHAT_TIMEOUT), still used by
the not-yet-migrated session/pipeline tools. dispatch_ai_tool loses its three
now-unused branches.
Also removes the dead do_second_opinion: it was already off the live tool
surface (no tag/schema/parsing/dispatch; tool_index.py notes it was removed),
so the function and its stale frontend catalog entries (admin.js, assistant.js)
are deleted.
Tests: owner-scope test points at the new list_models location and drops the
moved tools from the dispatch_ai_tool parametrize; a new
test_model_interaction_registry covers registration, owner threading, and
registry dispatch.
* fix(security): allowlist manage_mcp 'add' to close the agent-path RCE
do_manage_mcp('add') passed model- and prompt-injection-controlled command,
args, and env straight to a stdio subprocess spawn with no validation, and it
persisted an enabled server row before connecting (so a payload also survived
to re-execute on restart). A string smuggled into a skill description, memory
entry, fetched page, or email body could register a server running arbitrary
code as the app UID, e.g. command='sh' args=['-c','...'].
Add _validate_mcp_command, applied on the agent path before any DB write or
spawn:
- Hard-deny interpreters, runtimes, package runners, shells, and exec-wrappers
(even if an operator lists one in ODYSSEUS_MCP_ALLOWED_COMMANDS).
- Require a bare basename (no path components, no shell metacharacters) that is
present in the operator allowlist (empty by default).
- Reject code-exec argv flags by prefix so glued forms are caught too
(-c/-e/-m/--eval/--exec/--print/--module/--command/--require), remote-URL
args, and env keys that inject code into the child (LD_PRELOAD, NODE_OPTIONS,
PYTHONPATH, DYLD_*, PATH, ...).
A rejected registration returns an error, writes no row, and makes no
connection. The trusted admin route is unchanged. Mirrors the policy intent of
_validate_serve_cmd but inverted for the model-reachable surface.
Supersedes #438; incorporates the bypass forms found in its review (interpreter
script paths, -m pip, glued -c/-e, --eval=, eval subcommands, package runners,
remote URLs) and adds integration coverage on the real do_manage_mcp path.
Closes#2891
* fix(security): deny versioned/alias runtimes in manage_mcp allowlist
Addresses RaresKeY's review on #4433. The hard-deny matched command names
exactly, so versioned or alias runtime forms (python3.11, node18, pip3,
ruby3.2, java, javac, bunx, tsx, ts-node, pypy3, ...) slipped past and, if an
operator allowlisted one, re-opened the prompt-injection-controlled MCP
registration path.
- Canonicalize a trailing version suffix before the deny check so versioned
forms collapse to the family (python3.11 -> python, node18 -> node, pip3 ->
pip); both the raw basename and the canonical form are denied.
- Broaden the denied-family set (java/javac/jshell/jbang/kotlin/dotnet/mono/
swift/osascript/tsx/ts-node/bunx/pypy/jruby/raku/luajit/wish/expect/iex).
Deny runs before the operator allowlist, so an alias cannot be allowlisted back
in. Canonicalization only feeds the deny check, so a legit name that ends in a
digit still reaches the normal allowlist check rather than being mis-denied.
Adds validator + integration regressions for versioned/alias runtimes asserting
no DB row and no connection, including the allowlisted-anyway case.
* fix(hwfit): use CPU fallback for cpu_only speed estimates
* fix(hwfit): preserve ARM fallback for cpu_only estimates
---------
Co-authored-by: Cata <cata@bigjohn.local>
The scheduled-task runner built the agent's tool set from RAG retrieval plus
ASSISTANT_ALWAYS_AVAILABLE. Neither includes bash/python (nor the file tools),
and no keyword hint force-includes them, so a task only saw the shell when the
tool-embedding index happened to surface it. On hosts where that index is empty
or degraded (e.g. a fresh Docker deploy), retrieval returns nothing and the task
agent never receives bash/python — telling the user the shell is disabled even
for an admin owner.
Offer the shell/file group to task agents by default, mirroring the chat agent
where these are on unless a privilege or global setting turns them off. The
existing blocked_tools_for_owner() gate in stream_agent_loop still strips the
whole group for non-admin multi-user owners and only admits it for admins and
single-user (AUTH_ENABLED=false) deployments, so this changes what is offered,
not who is allowed. A crew that defines an explicit enabled_tools allowlist
still has its restriction honored.
Also merge the operator's global disabled_tools setting into the scheduler's
disabled set before composing relevant_tools and before entering the agent
loop, matching what chat already does. Without it, the global tool-disable
contract did not reach unattended scheduled tasks: an admin or AUTH_ENABLED=false
task could still see and call shell/file tools the operator had turned off
globally, since the prompt/schema/execution gates only enforce the disabled
tools passed in.
cmd_list filtered on the event START falling inside the window
(dtstart >= start AND dtstart < end). The canonical web route
(routes/calendar_routes.py) and the recurrence contract test use
OVERLAP semantics for non-recurring events: dtstart < end AND
dtend > start. So an event that began before the window but is still
ongoing inside it — e.g. a 09:00-17:00 conference listed at 14:00, or
any multi-day event spanning the window — was silently dropped by the
CLI even though the web UI shows it. Use overlap, matching the route.
dtend is NOT NULL in the schema, so no null-end regression.
The non-native (prompted) tool-call path fed tool output back to the model as a plain "[Tool execution results]" user message, bypassing the untrusted_context_message wrapper that THREAT_MODEL.md requires for tool output. That path is what models without native tool-calling (many smaller local models) use, so prompt-injection inside a tool result (fetched page, file read, MCP/email output) could be read as instructions there.
Wrap it via untrusted_context_message("tool execution results", ...), the same hardening already applied to skills (#788) and escalation traces (#275). Also update _recent_context_for_retrieval, which used the old "[Tool execution results]" prefix as a sentinel to keep tool envelopes out of the retrieval query, to recognise the wrapped envelope via metadata.trusted.
The native path keeps returning tool-role messages (a user-role wrapper would break the native tool-call contract); it is covered by UNTRUSTED_CONTEXT_POLICY. Adds tests/test_tool_output_prompt_injection.py.
Fixes#1627.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Added PASSWORD_MIN_LENGTH and RESERVED_USERNAMES to src/constants.py as the
single source of truth. Previously PASSWORD_MIN_LENGTH was hardcoded as 8 in
four route handlers and all three JS validation paths; RESERVED_USERNAMES was
an inline frozenset duplicated in core/auth.py, routes/assistant_routes.py,
routes/research_routes.py, and src/task_scheduler.py.
Added GET /api/auth/policy (unauthenticated) so the frontend reads the real
values from the server instead of hardcoding them in JS.
Added missing empty-username guard to /setup and admin POST /users. Both
returned a misleading 500/409 on whitespace-only input. /signup already had the
check; this makes all three consistent.
Ahmed Dlshad