Guard the agent_max_tool_calls settings read so hand-edited or agent-written non-numeric settings.json values fall back to 0 instead of crashing agent-mode chat stream initialization. Add regression coverage for guarded coercion.
Use the upload handler's tolerant index loader when reading upload metadata so corrupt uploads.json degrades to missing metadata instead of a 500. Return 400 for malformed vision JSON request bodies and add regression coverage for both paths.
Accept calendar datetime phrases such as "3pm tomorrow" by adding a time-first natural-language parser branch mirroring the reminder parser. Add regression coverage proving time-first forms match their existing day-first equivalents.
* fix(security): prevent ReDoS in LLM-output tool/think parsers
The regexes that parse untrusted model output in text_helpers.py and
tool_parsing.py are delimiter-bounded with a lazy [\s\S]*? (or an
ambiguous (\s+[^>]*)?). Applied with re.sub/re.finditer over a whole
response, they degrade to O(n^2) when the closing delimiter is absent:
the engine rescans to end-of-string from every opener. Model output is
untrusted, so a prompt-injected or malicious model can stall the agent
loop with many unclosed openers (measured ~25s on a 60KB <thought flood).
- text_helpers.py: replace ambiguous <thought(\s+[^>]*)?> with
<thought([^>]*)> (identical capture, no \s+/[^>]* overlap); skip the
Gemma <|channel>...<channel|> subs when no <channel|> closer is present.
- tool_parsing.py: gate _TOOL_CALL_RE, _XML_TOOL_CALL_RE and _TOOL_CODE_RE
(in parse_tool_blocks and strip_tool_blocks) on a cheap presence check
for their closing delimiter. With no closer the regex cannot match, so
skipping is equivalent; only the wasted O(n^2) rescan is removed.
Resolves CodeQL py/polynomial-redos #230, #231, #232, #233, #235, #236,
#524. The _XML_OPEN_TOOL_CALL_RE alerts (#234, #477) are false positives
(its greedy [\s\S]*\Z is linear) and left untouched.
* fix(security): close ReDoS gaps in tool/think parsers from review
Addresses two review findings on the closer-guard approach:
- Whole-string "closer exists?" checks were bypassable: a stale closer
before an opener flood, or a closer with no reachable inner `}`, kept
the guard true while every opener still rescanned to end-of-string
(O(n^2)). Replace the substring guards with `_iter_delimited`, a
forward-only scan that pairs each opener with a *later* closer and
stops once none is reachable (O(n)). `parse_tool_blocks` and
`strip_tool_blocks` (via `_strip_delimited`) both use it for the
[TOOL_CALL], <tool_call>/<function_call>, and <tool_code> formats.
Verified equivalent to the original regexes on well-formed inputs.
- `<thought([^>]*)>` dropped the tag-name boundary and corrupted
unrelated tags (`<thoughtful>` -> `<thinkful>`). Use `<thought(\s[^>]*)?>`:
the single fixed `\s` keeps the pattern linear (no `\s+`/`[^>]*`
overlap) while restoring the boundary; capture is byte-for-byte
identical for real `<thought ...>` openers.
Adds regressions for stale-closer-before-opener, closer-present-without-
inner-brace, and the <thoughtful>/<thoughts> passthrough.
* fix(security): close Gemma channel ReDoS guard flagged in review
vdmkenny noted the same bypassable whole-string guard remained in
text_helpers.py: `if "<channel|>" in out.lower()` gating the Gemma
thought/response channel subs. A stale `<channel|>` before a
`<|channel>thought` opener flood keeps the guard true while every opener
still rescans to end-of-string (measured ~7.3s at 4k openers).
Replace it with `_sub_delimited`, the same forward-only scan used for the
tool-call parsers: pair each opener with a later closer, stop when none is
reachable (O(n)). Verified output-equivalent to the original capture regexes
on well-formed multi-channel inputs; the stale-closer case now runs in <2ms.
Adds a regression for stale-closer-before-opener on the Gemma path.
* fix(security): harden strip_think() think-tag ReDoS flagged in review
The earlier fixes hardened normalize_thinking_markup and the delimiter
scanners, but the production entrypoint strip_think() still ran
_THINK_CLOSED_RE / _THINK_ATTR_RE / _THINK_OPEN_RE (and the stray-tag
_THINK_TAG_RE) over untrusted model output. Those kept the same ReDoS
shapes: the lazy `<open>[\s\S]*?</close>` rescanned to end-of-string from
every opener, and `(?:\s+[^>]*)?` / `[^>]*` attribute scans ran to
end-of-string from every opener on a "many openers, no closer" flood. On
the prior head, malformed `<think` / `<thinking` / `<thought` floods took
6-14s through strip_think(). The shipped `<thought>` normalization had the
same residual: the single-opener case was linear but an opener flood was
still O(n^2) (~4.4s).
- Replace the lazy multi-pass _THINK_CLOSED_RE loop with the existing
forward-only _sub_delimited scan (pair each opener with the first
reachable closer, stop when none is reachable). One pass collapses
sequential and nested blocks as before.
- Bound every opener/stray-tag attribute scan at `<` (`[^<>]` not `[^>]`)
so a no-`>` opener flood can't drive a single match attempt to
end-of-string. Identical capture for well-formed think/thought tags.
- email_helpers._strip_think: compute had_think from the single linear
_THINK_TAG_RE instead of the lazy closed/open `.search()` calls, which
had the same O(n^2) on the email reply/summary/extraction paths.
All flood variants now finish in <10ms (were 6-14s). Output verified
byte-for-byte identical to the prior implementation over a 34-case corpus
(nested, mismatched, attr, uppercase, Gemma, prose, prompt-echo). Adds
strip_think() timing regressions for malformed openers, opener floods
(all three tag names), the closed-opener flood, and the malformed-closer
flood.
* docs: trim verbose comments in think-tag ReDoS fix
Recognize api.cerebras.ai as a Cerebras cloud provider so llama.cpp/LM Studio cache-affinity fields are not attached even when endpoint_kind is misconfigured as local. Add regression coverage for provider detection, self-hosted classification, and payload field exclusion.
Strip fenced code blocks before extracting visual-report headings so heading-looking lines inside code fences do not desync TOC anchors. Add regression coverage for backtick and tilde fences while preserving normal heading extraction.
Install libmagic1 and image-scoped python-magic in the Docker image so upload MIME detection can use content sniffing. Add regression coverage for the Dockerfile dependency pair and the libmagic-present sniffing path.
A single-day all-day event whose source writes DTEND equal to DTSTART
(treating DTEND as an inclusive bound rather than the RFC 5545 exclusive
one) was stored verbatim as a zero-duration row. list_events selects
events overlapping the window with `dtstart < end AND dtend > start`, so
that row is filtered out for any window starting at or after its date and
the event never appears, even though the import reported success.
Events created via the API never hit this because creation always
synthesizes a positive duration; only the two import paths can persist a
non-positive one. Clamp a non-positive end at import (import_ics and the
CalDAV pull) to the same default span used when DTEND is absent: one day
for all-day events, one hour otherwise.
Also repair the persisted state for users who already imported before this
clamp existed. Their stored zero-duration row is invisible, and re-importing
the same ICS hit the duplicate branch and skipped without touching it, so
the event stayed hidden. The duplicate branch now backfills the clamp onto
the matched row before skipping, and the response reports a `repaired` count.
(The CalDAV pull already rewrites dtend on re-sync, so it self-heals.)
_parse_msg_content deserializes stored multimodal content (image/audio
blocks) back into a list. It treated ANY string starting with '[{' and
containing the substring "type" as serialized content, requiring only
that each element be a dict — never that "type" be a real content-block
kind. So a plain text message whose content happens to be a JSON array
of typed objects (e.g. a user pasting an API schema sample like
[{"type": "object", ...}]) was silently parsed from str into a list on
the next hydration, destroying the original string. This runs on every
session load from the DB (_db_to_session -> get_session). Restrict the
round-trip to non-empty lists whose every element is a dict whose
"type" is a recognized block kind (text/image/image_url/audio/...);
real multimodal content (verified: document_processor emits exactly
these) still round-trips, JSON-looking text is left untouched.
The Codex cookbook bridge authorized cookie sessions with require_user()
only, allowing non-admin accounts to read cookbook task state, server
topology, task logs, tmux sessions, and model presets. The stop/adopt
routes also execute local or SSH-backed tmux commands.
Add _require_cookbook_scope() that enforces require_admin() for
cookie-session callers while preserving the existing API-token scope
checks. Apply it to all nine /api/codex/cookbook/* routes.
Fixes#4542
Co-authored-by: michaelxer <michaelxer@users.noreply.github.com>
Prevent untrusted source/context guard text from being merged into the current visible user request during provider message sanitization.
Changes:
- Detect untrusted context blocks during LLM message sanitization
- Insert a short assistant boundary before the current user request
- Keep the visible user prompt as its own user message
- Preserve normal consecutive user-message merging for non-untrusted cases
- Strengthen prompt-security wording to avoid mentioning guard wrappers
- Add regression coverage for untrusted context followed by a user prompt
Notes:
- Untrusted context remains role:user for safety
- This does not add prompt debug logging
- This does not change frontend draft persistence
The lazy `<think>.*?</think>` pattern (one compiled `_THINK_RE`, one inline
copy) is applied with `re.sub` over whole model responses. With a `<think>`
opener and no closer, the engine rescans to end-of-string from every opener
-> O(n^2) on attacker-influenced output (prompt injection can echo thousands
of openers via tool output / retrieved content). CodeQL py/polynomial-redos.
Replace both with `_strip_think_blocks`, a forward-only linear scan that is
byte-for-byte equivalent to the original narrow regex: only literal
`<think>`/`</think>` (any case) match, a dangling opener with no closer is
left intact, and an orphan `</think>` is never stripped. Routing through the
broader `text_helpers.strip_think` was avoided on purpose -- it also strips
`<thinking>`, attributes and prompt echoes, which would change what the
loop's progress/circling heuristics see.
Adds tests/test_redos_think_blocks.py pinning regex-equivalence on a battery
of well-formed/edge inputs plus a linear-time bound on hostile input.
The email-account endpoints coerced user-supplied ports with a bare int(data.get("imap_port") or 993), so a non-numeric port (e.g. "imap") raised ValueError and surfaced as an HTTP 500 in the create, update, and test-config endpoints.
Add a _coerce_port(value, default) -> (port, error) helper and use it in all three endpoints, returning the endpoints standard {"ok": False, "error": ...} response (matching the existing "name required" validation) instead of crashing. A blank or missing port still falls back to the default (993/465).
* test(tools): add shim protection test for tool_implementations split
Covers all 48 top-level functions (33 do_* + 15 _helpers) extracted from
the original module. Guards the upcoming split: the shim must re-export
every symbol so existing 'from src.tool_implementations import X' imports
keep working. Passes on baseline (pre-split).
* refactor(tools): add src/tools/ package with shared _common
Slice 1 Task 2 (#4082/#4071). Adds the package skeleton and moves the
shared _parse_tool_args helper into src/tools/_common.py. Domain modules
will import from here. tool_implementations.py is untouched at this step.
* refactor(tools): extract system domain into src/tools/system.py
Slice 1 (#4082/#4071), Task 3: move the system-domain tool functions
(do_manage_skills/_skill_dump/do_manage_tasks/do_manage_endpoints/
do_manage_mcp/do_manage_webhooks/do_manage_tokens/do_manage_settings/
do_api_call/do_app_api) and the app_api blocklist constants out of
tool_implementations.py into a new src/tools/system.py module.
tool_implementations.py re-imports all of them so it stays a working
backward-compatible facade (shim test stays green).
- do_manage_mcp resolves get_mcp_manager via a function-local import
from tool_implementations so the test that patches
src.tool_implementations.get_mcp_manager still applies post-move.
- do_app_api imports _internal_headers and _INTERNAL_BASE (still in
tool_implementations) function-locally to avoid a circular import.
- Repoint test_context_budget introspection assertion to the moved
code's new home in src/tools/system.py.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* refactor(tools): extract cookbook domain into src/tools/cookbook.py
Moves the model-serving (cookbook) tool domain out of tool_implementations.py
into src/tools/cookbook.py as part of slice 1 (#4082/#4071):
- 13 do_* tools: download/serve/list/stop/tail/search/adopt/cached models,
list downloads/cancel, list cookbook servers, serve presets
- 9 private helpers: _cookbook_servers, _resolve_cookbook_host,
_cookbook_env_for_host, _infer_serve_{port,host}, _ensure_served_endpoint,
_cookbook_register_task, _cookbook_apply_retry_suggestion,
_scan_running_model_processes, _cookbook_kill_session
- _MODEL_PROCESS_PATTERNS constant (used only by _scan_running_model_processes)
tool_implementations.py stays a backward-compatible facade via a re-import
from src.tools.cookbook; src/tools/__init__ re-exports the same symbols.
_internal_headers and _INTERNAL_BASE stay in tool_implementations.py (shared
by system.py's do_app_api and many cookbook funcs). Each cookbook function
that needs them does a function-local import to avoid a top-level circular
dependency, matching the system-domain split.
Verified: compileall clean; shim test green; cookbook-touching suite
(652 passed, 1 skipped); full suite 3587 passed, 2 failed
(pre-existing test_api_chat_security, unrelated).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* refactor(tools): extract search domain into src/tools/search.py
* refactor(tools): extract notes domain into src/tools/notes.py
* refactor(tools): extract calendar domain into src/tools/calendar.py
Repoints tests/test_caldav_bidirectional_sync.py source-introspection
to src/tools/calendar.py (do_manage_calendar moved there).
* refactor(tools): extract image domain into src/tools/image.py
* refactor(tools): extract research domain into src/tools/research.py
* refactor(tools): extract contacts domain into src/tools/contacts.py
* refactor(tools): extract vault domain into src/tools/vault.py
Repoints tests/test_vault_password_not_in_argv.py source-introspection
to src/tools/vault.py (the vault do_* helpers moved there).
* refactor(tools): collapse tool_implementations to clean re-export shim
Move shared _INTERNAL_BASE/_internal_headers to src/tools/_common.py and
drop the duplicate _parse_tool_args (already in _common). tool_implementations.py
is now a pure re-export facade (+ 3 pre-existing email-context helpers, out of
scope). Domain files' function-local imports of these names still resolve via
the facade re-export.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(tools): port upstream cookbook workflow changes to split module
Rebase onto dev dropped c504214 ("Cookbook model workflow fixes") edits
to do_serve_model / do_tail_serve_output: the extraction commit moved
the pre-edit bodies into src/tools/cookbook.py and git auto-accepted the
deletion from tool_implementations.py, losing dev's changes. Restore them
in their post-split home:
- do_serve_model: add where/log_path/next_tools and the expanded
"Next required check" output message
- do_tail_serve_output: empty-output fallback message replacing
"(empty pane)"
(do_manage_settings web_fetch alias edit was already applied to
src/tools/system.py during the system-extract conflict resolution.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(tools): break admin_tools circular import in split facade
After rebasing onto dev (#3629 moved the admin manage_* tools into
src/agent_tools/admin_tools), the facade re-exported them via a top-level
`from src.agent_tools.admin_tools import ...`. But src.agent_tools.__init__
imports this facade at top level, so the eager import re-entered the
partially-initialized agent_tools package and broke collection.
Re-export the admin symbols (do_manage_endpoints/mcp/webhooks/tokens/
settings, _MCP_DENIED_COMMANDS, _validate_mcp_command) lazily through
module __getattr__ instead, and drop them from src/tools/__init__ (they
no longer live in the src.tools package). system.py now holds only the
skills/tasks/api bridges; admin tools live solely in admin_tools.py,
matching upstream.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(tools): re-export dropped helpers through the split shim
Address review finding from #4423: the compatibility facade claimed to
preserve every original top-level symbol but omitted three helpers the
old src.tool_implementations exposed. Re-export them and pin them in
the shim protection test:
- _string_arg, _validate_cookbook_ssh_target <- src/tools/cookbook.py
- _mcp_allowed_commands <- src/agent_tools/admin_tools.py (lazily via
__getattr__, to keep the agent_tools.__init__ <-> facade import acyclic
after the #3629 admin-tools migration)
All three added to tests/test_tool_implementations_shim.py _EXPECTED so
the test contract now matches its "every original top-level function"
comment.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test(tools): self-verify shim re-exports every domain do_*
The hand-maintained _EXPECTED list in the shim protection test can drift
silently when a new tool is added to a domain module but not re-exported
by the facade — exactly the omission a reviewer flagged post-split.
Add an auto-discovering test that enumerates every do_* from the domain
modules (incl. admin_tools) and asserts reachability through the shim,
so a forgotten re-export fails the build automatically.
Uses hasattr (not dir(ti)) because the admin symbols are re-exported
lazily via module __getattr__ and don't appear in dir(ti).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* test(tools): self-verify every in-repo facade import resolves
RaresKeY's P3 on the shim test was a claim-vs-reality gap: the docstring
said it protected "every from src.tool_implementations import X" but the
hand-maintained _EXPECTED list omitted three underscore helpers, so the
claim wasn't enforced. Re-exporting the three (cf1f5e3) fixed the known
gap; this closes the structural one.
Add test_every_facade_import_in_repo_resolves: ast-enumerate every
`from src.tool_implementations import X` site in src/ and tests/ and
assert hasattr(ti, X) for each. A forgotten re-export that anything in
the repo imports now fails the build automatically — including underscore
helpers, which the do_* discovery test does not cover.
Together with test_shim_reexports_every_domain_do_function, the shim
contract is now self-verifying. Demote _EXPECTED in the docstring to the
curated historical/downstream surface (the three helpers have no in-repo
consumer, so they stay manual by necessity) instead of "ground truth".
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(tools): dedupe _parse_tool_args + align shim guard with route consumers
Addresses two P3s from review (RaresKeY, 2026-06-26):
1. maintainability — _common carried a full copy of _parse_tool_args
alongside the canonical src.tool_utils one; future parser fixes could
diverge. The two bodies were byte-identical in logic, so _common now
re-exports from tool_utils (a leaf module, no circular-import risk).
The single-source test is extended to assert _common._parse_tool_args
and tool_implementations._parse_tool_args are the same object as
tool_utils._parse_tool_args.
2. test — the shim guard's import-site scan only walked src/ and tests/,
missing routes/chat_routes.py's clear_active_email/set_active_email
imports, and _EXPECTED omitted the active-email facade helpers. The
scan now walks every first-party Python dir (pruning venvs/caches/data
in-place), and set/get/clear_active_email are added to _EXPECTED
(get_active_email has no in-repo importer, so the scan alone can't see
it).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: yuandonghao <yuandonghao@cohl.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* feat: add dismiss (×) button to all toast notifications (#1355)
* Refresh README presentation
* fix: reset pointer-events on toast dismiss button click
Action toasts set pointer-events:auto on #toast for their clickable
button, but the × close-button handler only cleared the auto-hide timer
without resetting pointer-events. This left an invisible fixed overlay
blocking clicks in the top-right area after manual dismissal.
- Add pointerEvents reset in both showToast and showError close handlers
- Add DOM behavior tests for pointer-events across all toast types
---------
Co-authored-by: pewdiepie-archdaemon <pewdiepie-archdaemon@users.noreply.github.com>
* fix: include in-memory templates in group participant character list
_getCharacterList() only fetched user templates from the /api/presets/templates
endpoint. When a character was just created in the Character tab, the async
auto-save to the templates API might not have completed by the time the Group
tab loaded its participant dropdown — causing newly created characters to be
missing.
Now also merges the in-memory userTemplates array from presets.js as a
fallback. These are updated as soon as the async save completes (via the
loadUserTemplates callback), so they bridge the gap between character creation
and API persistence.
Fixes#3207
* fix: optimistic userTemplates update on character save
Update the in-memory userTemplates array immediately when saveCustomPreset()
succeeds, before the fire-and-forget templates API POST completes. This
bridges the timing gap where _getCharacterList() calls getUserTemplates()
and gets stale data because loadUserTemplates() hasn't been triggered yet.
* test: verify group participant dropdown merges in-memory templates
Source-level guards for the #3207 fix:
- group.js imports and calls getUserTemplates() to merge in-memory templates
- presets.js exports getUserTemplates and does optimistic in-memory update on save
5 tests ensuring the fix can't be silently reverted.
* fix: generate client-side id for optimistic update, return shallow copy from getUserTemplates
1. New characters now get a 'user-<hex>' id immediately on save, matching
the server's convention (uuid.uuid4().hex[:8]). Previously the id was ''
which the merge guard in _getCharacterList filtered as falsy.
2. getUserTemplates() now returns [...userTemplates] so callers cannot
accidentally mutate module state.
* fix(group.js): fix selection drop-downs behavior
- add an identifier to the selection drop-downs
based on what type it is.
- fix behavior of continuously adding a row
when a user clicks the "Group" tab button.
- fix behavior of not repopulating existing
selection drop-downs whenever a user
clicks the "Group" tab button.
* fix(#3207): remove duplicate of latest persona
- fix the duplication of the latest persona
or character being shown in selection
drop-downs.
- remove unnecessary blocks of code in
`_getCharacterList()`
- add functionality to show error toast if saving
a preset template/character fails.
- add functionality to revert optimistic update
of preset template/character if saving fails.
* chore(group.js,preset.js): fix test & format errors
remove trailing whitespaces in lines 230 and 232
in /static/group.js
add back the expected syntax from
tests/test_group_character_dropdown.py
* fix(presets.js,group.js): fix runtime errors
as stated in a comment by @alteixeira20,
runtime errors exist for the applied fixes.
fixes:
- missing ending `]`
querySelectorAll("select.preset-input[data-selection-type=character")
in `group.js`
- spelling error in `modelSelection.vale` in `group.js`
- fix the ordering logic error in optimistic rollback where `Object.assign` is called first before the clone happens in `saveCustomPreset` in `presets.js`.
- add tests for the cloning logic bug with the same format as previous tests by checking the order of LOC in `tests/test_group_character_dropdown.py`.
---------
Co-authored-by: michaelxer <michaelxer@users.noreply.github.com>
Co-authored-by: Alexandre Teixeira <111787685+alteixeira20@users.noreply.github.com>
* fix(cookbook): prefer native llama-server on local Windows
* fix(cookbook): harden local llama-server launch commands
* fix(cookbook): build serve commands for selected target
Fixes#4767. #4724 routed 16 body-portaled dropdowns through the shared
topPortalZ() helper so they always render just above the currently-raised tool
modal, but two were missed and still used a hardcoded z-index, so they hit the
same #4720 bug once a modal's bring-to-front counter climbed past the literal:
- tasks.js _showTaskDropdown(): inline z-index:100000 on .task-dropdown
- skills.js kebab menu (.skill-kebab-menu): z-index:100002 in style.css
Both now set zIndex from topPortalZ() after they are appended to the body,
matching the other migrated sites. The dead CSS z-index on .skill-kebab-menu is
removed (the inline value always wins). test_portal_dropdown_z_js.py gains a
source guard asserting both files use topPortalZ() and that no hardcoded
100000/100002 portal literal survives in either file or style.css.
* Fix#4507: only block model launch on real port collisions
Quick-run hardcoded port 8000 and never called _nextAvailablePort(), so
every launch collided. Both pre-launch guards (serve panel + quick-run)
were count-based and fired regardless of port.
- quick-run now auto-assigns a free port (8080 for llama.cpp)
- both guards parse the new port and only prompt on a real overlap,
stopping only the colliding serve
- dialog reports the actual port instead of a hardcoded 8000
* refactor(cookbook): share _taskPort for port parsing; auto-assign llama.cpp port
Addresses review on #4760:
- _taskPort regex now matches --port= as well as --port (space)
- _nextAvailablePort and both launch guards reuse _taskPort instead of inline regex
- quick-run llama.cpp no longer pins 8080, so two can run concurrently
* fix(cookbook): _taskPort also parses -p; add port-parsing tests
Addresses review on #4760:
- _taskPort now matches -p <n> too, so it's the complete single reader
(was missing the short flag that other readers already handle)
- add tests/test_cookbook_port_parsing_js.py covering the port forms,
shared-reader reuse, and llama.cpp auto-assign
* test(cookbook): extract pure port helpers and test behavior
Addresses review on #4760: the prior tests only asserted source strings.
- extract portOf() and nextFreePort() into static/js/cookbookPorts.js
- cookbookRunning.js imports them; _taskPort and _nextAvailablePort delegate
- tests run the helpers via node and assert real behavior: all port forms
(--port, --port=, -p, -p=), next-free-port skipping taken ports, and the
same-port-clash / different-port-coexist outcome
---------
Co-authored-by: samy <samy@odysseus.boukouro.com>
* fix(model-routes): harden _probe_endpoint against malformed model-list responses
_probe_endpoint parsed model lists with data.get(...) at four sites without
checking that data is a dict, and built the list with a truthiness-only
filter. A /models (or /api/tags) endpoint returning HTTP 200 with valid but
non-dict JSON ([], "x", null, 123) made data.get(...) raise AttributeError,
and a non-string id like 123 passed the filter and then hit .startswith() /
.lower() in the Z.AI/Kimi curated merge and _is_chat_model(). Both errors are
swallowed by the broad except Exception, but the comprehension dies mid-list
so the ENTIRE probed model list is discarded and the endpoint silently
degrades — masking a misconfigured/non-compliant upstream as "no models".
- Guard each data.get(...) with isinstance(data, dict) so a non-dict body
falls through the existing `or []` default.
- Restrict the OpenAI and Ollama model-list comprehensions to non-empty str
values, protecting the .startswith() merges and both _is_chat_model calls.
- Add an isinstance guard at the top of _is_chat_model (defense in depth for
all four call sites).
No behavior change for well-formed {"data":[...]} / {"models":[...]}
responses. Adds regression tests (non-dict body via caplog, mixed/all
non-string ids, _is_chat_model boundary) that fail before the fix and pass
after.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(model-routes): extract _openai_model_ids / _ollama_model_names helpers
Per review on #4789: the malformed-response guards were inlined four times in
_probe_endpoint (two OpenAI-id comprehensions, two Ollama-name comprehensions).
Pull each into a small, directly-testable helper so the security-relevant
parsing lives in one place and a future malformed-shape fix doesn't have to be
applied in four spots (CONTRIBUTING flags repeated logic for this reason).
Behavior is unchanged. Adds direct unit tests for both helpers (non-dict body,
non-string ids, non-dict entries, name>model precedence).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Upstream bug (present in pewdiepie-archdaemon/odysseus main): the task
executor passes task.endpoint_url VERBATIM to the model HTTP call, unlike
the chat path which stores build_chat_url(normalize_base(base)) on the
session. A task carrying an explicit bare OpenAI-compatible base such as
"http://host:11434/v1" therefore POSTs to a 404 ("page not found"); the
agent loop swallows the empty body into "The model returned an empty
response" and marks the run success, so nothing surfaces the failure.
Tasks that omit an endpoint dodge this only because _resolve_defaults()
cribs an already-full URL from a recent chat session. The API/token path
(e.g. an external client that POSTs /api/tasks with endpoint_url=".../v1")
hits it every time.
Fix: route every resolved task endpoint through _normalize_chat_endpoint()
at the three resolution sites (_execute_llm_task, the persona/research
session path, and _execute_research_task). The helper is idempotent
(strips any existing chat suffix, re-appends the correct one) and leaves
native-Ollama (/api...) and already-concrete URLs untouched, so other
providers are unaffected. Proven via isolated repro: ".../v1" -> 404 ->
empty; ".../v1/chat/completions" -> 200 -> real gemma4:31b output.
Regression test asserts the bare-/v1 -> full-chat-URL mapping, idempotency,
and the native-Ollama/empty passthroughs.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Part of #3629 (the `admin_tools.py` bullet). Moves the config/integration admin
tools off the legacy elif dispatch chain in tool_implementations.py onto the
agent_tools registry:
manage_endpoints, manage_mcp, manage_webhooks, manage_tokens, manage_settings
The do_* implementations (and manage_mcp's command-allowlist / RCE guard:
_validate_mcp_command, _mcp_allowed_commands, and the _MCP_* constants) move
verbatim into the new src/agent_tools/admin_tools.py. They register through a
single ADMIN_TOOL_HANDLERS map that TOOL_HANDLERS.update()s, and the five elif
branches plus their imports are dropped from tool_execution.py, so these tools
now flow through _direct_fallback like the other migrated clusters. The names
are re-exported from src.agent_tools for back-compat.
Dedup:
- _parse_tool_args was duplicated in tool_implementations.py and
document_tools.py. It now lives once in src.tool_utils (which imports nothing
from the project beyond src.constants, so this introduces no cycle) and both
call sites import it from there. The orphaned `import json` in document_tools
is removed with it.
- The five tools share one _owner_adapter(fn) factory that threads ctx["owner"]
into the owner-taking do_* signature, instead of five near-identical wrappers.
Tests: new tests/test_admin_tools_registry.py pins the registration, the
re-export back-compat, the owner-threading adapter, and the single-source
_parse_tool_args (across admin_tools and document_tools). Existing MCP /
settings / webhook suites are repointed at the new module.
* feat(discovery): detect llama.cpp servers and label local providers
Scan port 8080 (llama-server) and 11435 (APFEL) during discovery, fingerprint
llama.cpp via its native /props endpoint, and label well-known local serving
ports (8080 llama.cpp, 8000 vLLM, 1234 LM Studio, 11434 Ollama) consistently
in both the Python provider helper and the JS endpoint UI. Adds a llama.cpp
hint to the /setup slash command.
* fix(discovery): don't infer the serving tool from the port alone
Per review: vLLM, SGLang, llama.cpp and plain OpenAI-compatible servers all
share 8000/8080, so labeling by port mislabels real setups (a vLLM box on 8080
shown as llama.cpp). Drop the port->tool assertions from _provider_label and
providerLabel; the authoritative signal is the /props fingerprint done during
discovery, which is unchanged. Loopback now reads a neutral 'local endpoint' /
'Local'. Tests updated to assert the neutral labels.
* fix: use atomic write in APIKeyManager.save() to prevent data loss
Opening api_keys.json with 'w' truncates the file before writing, so a
crash, disk-full, or mid-write error leaves all stored provider API keys
corrupted. Switch to atomic write (temp file + fsync + os.replace) so
the original file is always intact on any failure.
Fixes#4591
* chore: trigger CI re-run
* chore: update PR description
* chore: fix how-to-test section for description check
---------
Co-authored-by: michaelxer <michaelxer@users.noreply.github.com>
* First bare fix
* Adding the option toggle
* toggle function fix
* Final fix, added missing /auth/
* Extended toggle text & added tests
* Comments change
* Description toggle change
* br tag fix
* description change based on suggestion
setup.py read ODYSSEUS_ADMIN_USER / ODYSSEUS_ADMIN_PASSWORD via os.getenv()
but never loaded .env, so on native Linux/macOS installs a password
pre-seeded in .env (documented in docs/setup.md and .env.example) was
silently ignored and a random one generated, breaking the first login.
Docker was unaffected because compose passes the vars into the container env.
Call load_dotenv(BASE_DIR/.env, encoding="utf-8-sig") at the top of main(),
mirroring app.py (utf-8-sig tolerates a Notepad UTF-8 BOM). load_dotenv does
not override already-exported OS vars, so the existing precedence is kept.
python-dotenv is already a required dependency.
Adds a regression test that pre-seeds credentials only in .env (not the
shell) and asserts the stored bcrypt hash matches the pre-seeded password.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add official Gemma 4 12B-it plus QAT-INT4/INT8 catalog entries (with their
GGUF sources), QAT quantization support across the quant tables and the
prequantized-prefix list, and the missing RTX 3050 / 3050 Ti memory
bandwidth so speed estimates stop falling back to the generic cuda value.
* fix(routes): serve 404 instead of 500 when an HTML page file is missing
_serve_html_with_nonce opened the HTML file with no error handling, and
callers such as /backgrounds and /login pass their paths in with no
existence check, so a missing or unreadable file raised an unhandled
OSError that surfaced as a 500. Wrap the read and raise HTTPException(404)
instead; the normal render path (CSP-nonce substitution) is unchanged.
Fixes#4594
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(routes): distinguish missing page (404) from read failure (500)
The previous fix caught a broad OSError and returned 404 for every
failure, which masks real server-side problems (permission errors, I/O
failures) as "not found" and lets them slip past error alerting. Split
FileNotFoundError (genuine 404) from other OSError, which now logs the
exception and returns a generic 500 — without leaking the OS error
string or file path into the response body.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(routes): treat unreadable bundled HTML page as logged 500, not 404
Per PR #4637 review: every caller of the page-render helper serves a fixed,
server-owned template (index/login/backgrounds), never a client-supplied
path. So a missing or unreadable file is a server fault (broken deployment),
not a client "not found" — a 404 there mislabels a server error and hides a
missing core template from 5xx alerting, contradicting the OSError->500
rationale this PR is built on. Collapse both branches into a single logged,
leak-free 500.
Move the helper to src.app_helpers.serve_html_with_nonce so the behavior can
be unit-tested without importing the whole app (app.py is the slim
orchestrator; the test harness stubs src.database, so importing app in tests
is not viable). Add tests pinning missing/unreadable -> 500 (not 404) and
nonce injection on the happy path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* fix(chat): strip executed email tool fences from the live stream (#3993)
The backend strips every fenced tool block from persisted text (the regex in
src/tool_parsing.py is built from the full TOOL_TAGS set, which includes the
email tools), so a reloaded session renders cleanly. The live frontend path
uses a separate hardcoded EXEC_FENCE_RE in static/js/chatRenderer.js that only
listed web_search/read_file/write_file/create_document/edit_document/
update_document — so executed email tool fences (list_emails, etc.) lingered as
raw code blocks in the live assistant bubble until the user reloaded.
Add the nine email tool tags to EXEC_FENCE_RE so the live render settles into
the same clean layout as the history reload. bash/python stay excluded on
purpose: those are languages a user may legitimately have asked the model to
show as code, not tool invocations.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(chat): single-source live exec-fence tool list from TOOL_TAGS (#3993)
Per review: EXEC_FENCE_RE was a second, hand-maintained copy of the
executable-tool list, so any tool not in it — and every future tool added to
TOOL_TAGS — would leave its executed fence lingering in the live bubble until
reload (the original #3993 bug, recurring one tool at a time).
EXEC_FENCE_RE is now built from an explicit EXEC_TOOL_TAGS list that mirrors
TOOL_TAGS (src/agent_tools/__init__.py) minus bash/python, which stay excluded
as legitimate code-example languages. A new regression test
(test_exec_fence_re_covers_all_executable_tools) extracts both lists from
source and fails if they drift, so the whole class is caught in CI instead of
by a user — the "minimum acceptable middle ground" from the review, made exact
(set equality, not just coverage).
Verified: pytest tests/test_live_strip_email_tool_fences.py (5 passed);
node --check static/js/chatRenderer.js; and a node run of the built regex
confirms email/generate_image/manage_memory/ls fences strip while
bash/python/sh are preserved.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* refactor(chat): build live exec-fence list from /api/tools at runtime (#3993)
Make TOOL_TAGS the single source for live exec-fence stripping. chatRenderer.js
no longer hard-codes a tool list; it fetches the backend's authoritative set
once from GET /api/tools (sorted(TOOL_TAGS)) and builds EXEC_FENCE_RE from it at
load, minus bash/python. No second list to drift, and a future tool added to
TOOL_TAGS is covered automatically — without touching the streaming path.
Until the fetch resolves EXEC_FENCE_RE is null and exec fences aren't stripped
(a sub-second window before the first stream); the backend already strips
persisted history, so a reload always renders clean.
Drop test_exec_fence_re_covers_all_executable_tools (no hand-maintained list to
guard) and add source-level guards: the frontend keeps no hard-coded list and
fetches /api/tools, and the endpoint serves the full sorted(TOOL_TAGS).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CVCKth4g8pWh7pwFDVm4iL
* fix(chat): warn on /api/tools fetch failure instead of swallowing it (#3993)
A fresh-context review flagged that loadExecFenceRegex's catch silently
discarded errors: if the one-shot fetch fails, EXEC_FENCE_RE stays null for the
whole session and live exec fences go unstripped until reload, with zero signal.
console.warn it, and correct the comment to describe the failure mode honestly
(was understated as just a sub-second startup window).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CVCKth4g8pWh7pwFDVm4iL
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: Images cannot be seen by model that is vision capable
* fix: skip http(s) image_url for Ollama (images[] is base64-only)
---------
Co-authored-by: michaelxer <michaelxer@users.noreply.github.com>
* fix(llm): detect mistral.ai provider and support reasoning_effort
Four coupled bugs broke Mistral thinking model support:
1. _detect_provider() had no mistral.ai host check, so all Mistral
endpoints fell through to the generic 'openai' provider string.
_provider_display_name() correctly identified them as 'Mistral',
making any 'if provider == "Mistral"' check elsewhere dead code.
2. reasoning_effort parameter was never sent in the request payload,
so Mistral never activated thinking mode even when the user
configured a thinking-capable model (mistral-small-latest,
mistral-medium-latest, magistral-*).
3. Mistral returns content as a typed array
([{"type":"thinking",...},{"type":"text",...}]) when
reasoning is on, not as a plain string. Both the streaming and
non-streaming parsers expected strings and silently dropped the
thinking content.
4. _THINKING_MODEL_PATTERNS didn't include magistral or mistral-*
model prefixes, so the frontend wouldn't tag reasoning output
as thinking even after the above were fixed.
Fix:
- Add mistral.ai to _detect_provider() host checks
- Add a _normalize_mistral_content() helper that splits the typed
array into (text, thinking) strings
- Inject payload["reasoning_effort"] = "high" when provider is
Mistral and _supports_thinking(model) is true, in both stream_llm
and llm_call_async payload construction
- Wire the normalizer into both response parsers
- Extend _THINKING_MODEL_PATTERNS to include magistral,
mistral-small, mistral-medium, mistral-large
Tested on Docker install with mistral-small-latest +
reasoning_effort=high. Reasoning streams correctly into the
thinking panel after the fix.
Fixes#4678
* fix(llm): address review — lowercase provider id, configurable effort, tests
Addresses vdmkenny's review on PR #4698:
1. Removed duplicate 'if provider == "mistral"' block in stream_llm
— two back-to-back copies, one was dead-redundant.
2. Dropped personal-context comment ('free-tier limits are generous
for this user') and made reasoning_effort configurable via env var
ODYSSEUS_MISTRAL_REASONING_EFFORT (high / medium / low / none).
Default remains 'high' for backward compat with the tested behavior.
3. Recased provider id from 'Mistral' to 'mistral' to match the
lowercase convention used by every other provider id in the file
(openai, anthropic, ollama, copilot, ...). _provider_display_name()
still returns the Title-Case 'Mistral' for UI labels — only the
runtime id used in 'if provider == ...' checks was recased.
4. Added tests/test_llm_core_mistral_content.py with 13 tests pinning
_normalize_mistral_content()'s contract: string passthrough, the
Mistral array format (thinking + text blocks), and edge cases
(empty, garbage, None, wrong types, missing fields, string-vs-array
inner thinking field).
Also fixed a gap the review didn't catch: the non-streaming paths
(llm_call sync + llm_call_async) were missing the reasoning_effort
injection entirely. Added the same injection to both, so Deep Research
and agent tool calls also activate Mistral thinking.
All 13 new tests pass. Existing reasoning/streaming/ollama-thinking
tests still pass (38 tests, no regressions).
Fixes#4678
* fix(memory): keep the Brain memory item menu above the modal at any stack depth
The memory item "⋮" dropdown is portaled to <body> with a hardcoded
z-index of 10001. Tool modals, however, get a monotonically increasing
z-index from modalManager's bring-to-front counter (_modalTopZ), which
climbs unbounded as modals are opened/restored over a session. Once that
counter passes 10001, the Brain modal stacks above the body-portaled
dropdown, so the menu renders behind the panel — visible only where it
spills past the modal's edge (#4720).
Derive the dropdown's z-index from the owning modal's current z-index
(+1), keeping 10001 as a floor for the common low-counter case, so the
menu always sits just above its modal however high the counter has climbed.
Verified with document.elementFromPoint at the dropdown's location: with a
high modal z-index the old build returns the modal at every sampled point
(menu behind); the fixed build returns the dropdown (menu on top). The
default low-counter case is unchanged (z stays 10001).
* refactor(modal): route body-portaled dropdowns through a shared topPortalZ() helper
The hardcoded z-index:10001 the Brain memory menu used (#4720) is the same
literal shared by ~16 body-portaled dropdowns across calendar, cookbook,
cookbookServe, documentLibrary, emailLibrary, gallery, notes, emojiPicker and
memory — each renders behind its owning tool modal once modalManager's
bring-to-front counter climbs past the literal over a long session.
Promote the per-dropdown fix into a single topPortalZ() helper in
toolWindowZOrder.js — the existing source of truth for tool-window z, already
imported by modalManager's _bringToFront and notes.js — returning
max(topToolWindowZ(), dock-chip floor) + 1, so a portaled dropdown always sits
just above the live tool-window stack however high the counter has climbed.
Route all 16 sites through it. The slashCommands tour tooltips and the
cookbookServe VRAM dialog are intentionally left out (neither is a modal-owned
portaled dropdown).
Add tests/test_portal_dropdown_z_js.py covering the helper, including the #4720
scenario (modal counter at 99999 -> dropdown at 100000). Existing
test_notes_z_order_js.py stays green.
* fix(security): redact credential-bearing URLs and PII from logs
Several log statements emitted sensitive data in clear text:
- model_routes / chat_routes / contacts_routes logged endpoint URLs raw.
Admin-configured URLs can embed credentials in userinfo or query
(e.g. https://user:pass@host, ?api_key=...). Route them through a
shared core.log_safety.redact_url() that drops userinfo/query/fragment.
- note_routes / task_scheduler logged operator email addresses (smtp_user,
recipient). Replaced with presence booleans, which keeps the diagnostic
("why didn't this send") without writing PII to logs.
model_routes already had a local redactor on its HTTPStatusError branch;
the generic except branch was missed, so reuse the existing helper there.
Clears CodeQL py/clear-text-logging-sensitive-data alerts 264, 317, 324,
325, 343, 344, 528.
* fix(security): re-bracket IPv6 hosts and single-source the URL redactor
Address review on #4750:
- redact_url now re-brackets IPv6 literals so host:port stays
unambiguous (https://[2001:db8::1]:8443/v1, not the bracket-less
ambiguous form).
- point model_routes._redact_url_for_log at the shared helper so the
two redactors are single-sourced (also picks up the IPv6 fix).
* fix(security): escape backslashes in calendar bg-image CSS url()
The calendar event-background CSS escaped ' -> \' for a bg: image URL but
not backslashes first. Inside a single-quoted url('...'), \ is the CSS
escape char, so a URL value ending in/containing a backslash escapes the
closing quote and breaks out of the string, injecting arbitrary CSS. The
bg:<url> value is per-event and CalDAV-syncable, hence untrusted (CodeQL
js/incomplete-sanitization).
Add a single canonical _cssUrlEscape() in calendar/utils.js that escapes
backslashes FIRST, then quotes, and route all four sinks through it:
calendar.js:416 / :1263 (the flagged #463/#464), the event-form preview
(:2931), and _calBgCss() in utils.js — the latter two share the identical
bug but were unflagged. Output is byte-identical to the old escaping for
legitimate URLs (which contain no backslashes); only malicious input differs.
Resolves CodeQL js/incomplete-sanitization #463, #464.
* fix(security): route remaining calendar bg url() sinks through _cssUrlEscape
Review (vdmkenny) flagged that the centralization missed an injectable
sibling sink: the edit-form color-picker swatch (calendar.js:2856) built
`url('${url}')` from `existing.color` (a CalDAV-syncable, untrusted `bg:`
value) raw, then interpolated it into `style="background:..."` via innerHTML
- the same `'`/`\` breakout class as the sinks already fixed. The custom-dot
preview (:2953) was likewise raw (non-exploitable - a CSSOM `.style`
assignment of a URL the current user just picked - but it broke the invariant).
Route both through `_cssUrlEscape`, and normalize the two pre-escaped-variable
sites (_calItemBgStyle, _renderWeek) to the same inline form so all five
url() interpolations in calendar.js follow one rule. Add a whole-file
invariant test asserting every `url('${...}')` calls `_cssUrlEscape` - this
catches a future missed sink, the exact failure mode here. Behavior-identical
for legitimate URLs (no visual change).
* fix: document read fails with 403 when auth is disabled
Add _auth_disabled() bypass in _verify_doc_owner() and the
/api/documents/{session_id} route guard so documents remain accessible
in single-user / no-auth mode.
Minimal change: only adds the auth-disabled check alongside existing
403 raises — preserves existing formatting and line endings.
* refactor: hoist _auth_disabled import to module level
Address reviewer feedback on PR #4623 — no circular import exists
(src.auth_helpers only imports stdlib + fastapi), so the inline
imports are unnecessary. Moves the import to module top in both
document_helpers.py and document_routes.py.
* test: add regression tests for auth-disabled document access (PR #4623)
Remote Cookbook hwfit probes failed on Windows hosts because the PowerShell script was sent as nested -Command quoting through OpenSSH. Use -EncodedCommand for remote probes, auto-detect platform when omitted (including Darwin for Mac SSH hosts), and return a clearer error when SSH works but the probe fails.
Co-authored-by: Cursor <cursoragent@cursor.com>
Inline backtick spans were converted to <code> only at the end of
mdToHtml, after the bare-URL autolink and <a>/allowed-HTML passes. A URL
inside inline code is preceded by a space, so the autolink wrapped it in
an <a> tag and swapped it for an ___ALLOWED_HTML_ placeholder, corrupting
commands like `irm http://127.0.0.1:3000/x`.
Extract inline code into placeholders before the link passes, mirroring
the existing fenced-code-block handling, and restore them last so
placeholders carried inside restored <a> blocks resolve. Escape the code
at extraction time since it now bypasses the global escape pass.
muhamed hamed