62 Commits

Author SHA1 Message Date
Alexandre Teixeira cb6c28113a docs(discovery): publish provisional feature baseline 2026-07-26 12:44:34 +01:00
pewdiepie-archdaemon d8a2059df8 Merge verified Odysseus fixes 2026-07-23 14:49:02 +00:00
Joeseph Grey 4c9a8ca115 fix(rag): skip hidden and junk directories when indexing (#5633)
* fix(rag): skip hidden and junk directories when indexing (#5559)

index_personal_documents walked the whole tree with no pruning, so
pointing RAG at a real-world folder silently swept in .obsidian/ plugin
JS, .git/ internals, node_modules/, and __pycache__/ — multiplying
indexing time and polluting retrieval with junk chunks.

Prune hidden directories and well-known junk directories from the walk,
and skip hidden files. The explicitly passed root is exempt, so a user
who deliberately indexes a hidden directory still gets its contents.

* fix(rag): prune hidden/junk dirs in the keyword index too, via a shared helper

The #5559 fix pruned only VectorRAG.index_personal_documents (the vector index).
The parallel keyword index built by PersonalDocsManager.refresh_index ->
load_personal_index walked the same tree unpruned, so .obsidian/, .git/,
node_modules/ etc. still swept into keyword retrieval and the file listing —
the 'end-to-end' guarantee was only half true.

Single-source the pruning policy in src/index_walk (prune_index_dirs +
is_indexable_file) and use it from both walkers so they cannot drift again.
The junk-dir match is now case-insensitive, so a Node_Modules on a
case-insensitive filesystem is pruned too.

Tests: keyword-path regressions covering hidden/junk dirs, hidden files, junk
at depth (not just top level), case-insensitive junk, and the explicit-hidden-
root exemption. The existing vector tests still pass against the shared helper.
2026-07-23 14:18:08 +02:00
RaresKeY d49629fa14 fix(reminders): support OAuth SMTP accounts (#5649) 2026-07-22 16:03:35 +02:00
RaresKeY 65987fc772 fix(email): test saved OAuth accounts through shared transports (#5653)
* fix(email): use XOAUTH2 in test-connection for Google OAuth accounts

The test-connection endpoint was password-only and had no awareness of
OAuth accounts. For Google-connected accounts this caused two failures:
- IMAP: "Need IMAP host, username, and password" because imap_pass is
  empty (no password is stored for OAuth accounts)
- SMTP: 535 BadCredentials because smtp.login() was called with an
  empty password instead of an XOAUTH2 token

Fix: include oauth_provider and token fields in saved_body when hydrating
from the DB, then use conn.authenticate("XOAUTH2") / smtp.auth("XOAUTH2")
for Google accounts in both the IMAP and SMTP test paths, mirroring what
_send_smtp_message already does for real sends.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test(email): add OAuth2 tests for test-connection endpoint

Covers the XOAUTH2 changes made to routes/email_routes.py:
- Google OAuth accounts must not be rejected with 'Need IMAP host,
  username, and password' (no stored password for OAuth accounts)
- IMAP and SMTP test paths must use conn.authenticate('XOAUTH2')
  for Google accounts
- Password accounts must still use conn.login() / smtp.login()

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(email): bind OAuth account tests to Google transport

---------

Co-authored-by: TNTBA <trynottobreakanything@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-22 14:54:38 +02:00
RaresKeY dcc7e52a86 fix(config): forward Google email OAuth settings through Compose (#5650)
* fix(config): forward Google OAuth env vars into Docker container and document setup

GOOGLE_OAUTH_CLIENT_ID, GOOGLE_OAUTH_CLIENT_SECRET, and GOOGLE_OAUTH_REDIRECT_URI
were read by the app but never forwarded through docker-compose.yml's explicit
environment allowlist, causing the "not set" error even when the vars existed in .env.

Also adds a documented section to .env.example with step-by-step GCP setup instructions
so users know where to get the credentials.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(config): cover OAuth in standalone compose files

* test(config): parse OAuth compose service env

* test(config): keep checkout skip wording neutral

---------

Co-authored-by: TNTBA <trynottobreakanything@gmail.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-07-22 12:59:22 +02:00
RaresKeY 93f97e74cd fix(email): hide OAuth tokens from config response (#5651) 2026-07-22 12:58:16 +02:00
RaresKeY 16b04a9792 fix(email): verify mailbox identity on OAuth reconnect (#5648) 2026-07-22 12:56:17 +02:00
nopoz b07c1e3b33 ci: add CodeQL advanced setup to scan pull requests before merge (#5250)
* ci: add CodeQL advanced setup to scan pull requests before merge

* ci(codeql): preserve scheduled scans

Add a weekly advanced-setup scan and update the security CI guide so default setup remains disabled.

---------

Co-authored-by: RaresKeY <158580472+RaresKeY@users.noreply.github.com>
2026-07-21 10:18:06 -07:00
Boody fe0748308e Merge pull request #5286 from tse-jeff/fix/snap-wsl2-gpu-passthrough
fix(docker): detect snap+WSL2 GPU passthrough incompatibility
2026-07-21 13:57:32 +03:00
Tal.Yuan 483c42bb12 refactor(routes): move compare domain into routes/compare/ subpackage (#5660)
Slice 2i of the route-domain reorganization (#4082/#4071). Moves
compare_routes.py into routes/compare/, leaving a backward-compat
sys.modules shim at the old path. Pure file reorganization, no behavior
change.

The shim uses sys.modules replacement so the `import ... as cr` +
`monkeypatch.setattr(cr, "SessionLocal", ...)` / `"_owned_endpoint_by_url"`
/ `"_owned_endpoint_by_id"` pattern in test_endpoint_owner_scope_followup.py
reaches the canonical module.

Canonical module imports only from core/, src/, and routes.session_routes
(zero dependency on the legacy shim). One source-introspection test site
repointed: test_endpoint_owner_scope_followup.py (shared with other domains;
only the compare entry repointed here).

Adds tests/test_compare_routes_shim.py to pin the sys.modules shim
contract. Verified: compileall clean; targeted tests pass.
2026-07-21 12:40:09 +02:00
Tal.Yuan 833cdbd140 refactor(routes): move admin_wipe domain into routes/admin_wipe/ subpackage (#5659)
Slice 2h of the route-domain reorganization (#4082/#4071). Moves
admin_wipe_routes.py into routes/admin_wipe/, leaving a backward-compat
sys.modules shim at the old path. Pure file reorganization, no behavior
change.

The shim uses sys.modules replacement so the `import ... as
admin_wipe_routes` + `monkeypatch.setattr(admin_wipe_routes, "SessionLocal",
...)` / `"require_admin"` pattern in test_admin_wipe_gallery.py reaches
the canonical module.

Canonical module imports only from core/, src/, and stdlib (zero internal
routes/ coupling). Zero source-introspection landmines.

Adds tests/test_admin_wipe_routes_shim.py to pin the sys.modules shim
contract. Verified: compileall clean; targeted tests pass.
2026-07-21 12:39:27 +02:00
Tal.Yuan b7d3f2a28d refactor(routes): move cleanup domain into routes/cleanup/ subpackage (#5658)
Slice 2g of the route-domain reorganization (#4082/#4071). Moves
cleanup_routes.py into routes/cleanup/, leaving a backward-compat
sys.modules shim at the old path. Pure file reorganization, no behavior
change.

The shim uses sys.modules replacement so string-targeted
monkeypatch.setattr("routes.cleanup_routes.*", ...) in
test_cleanup_owner_scope.py reaches the canonical module.

Canonical module imports only from src/ and stdlib (zero internal
routes/ coupling). Zero source-introspection landmines.

Adds tests/test_cleanup_routes_shim.py to pin the sys.modules shim
contract. Verified: compileall clean; targeted tests pass.
2026-07-21 12:38:32 +02:00
RaresKeY cc4c7f4263 chore: update repository URLs after organization transfer (#5622) 2026-07-20 16:43:47 +02:00
Tal.Yuan 88e0ce3037 refactor(routes): move note domain into routes/note/ subpackage (#5236)
Slice 2f of the route-domain reorganization (#4082/#4071, per
specs/architecture-runtime-inventory.md §6.3). Moves note_routes.py into
routes/note/, leaving a backward-compat sys.modules shim at the old path.
Pure file reorganization, no behavior change.

The shim uses sys.modules replacement (same pattern as the merged gallery
#4903, research #4975, memory #5007, history #5090, and contacts #5227
slices) so that `import routes.note_routes`, `from routes.note_routes import
X`, `importlib.import_module(...)`, and the `import ... as note_routes` +
`monkeypatch.setattr(note_routes, "SessionLocal", ...)` pattern used by
test_note_reminder_fire_scope.py / test_notes_fail_closed_auth.py all
operate on the same module object the application uses.

The canonical module does NOT depend on the shim — routes/note/note_routes.py
imports only from core/, src/, and stdlib. The outbound email cross-domain
imports (routes.email_routes._get_email_config, routes.email_helpers.
_send_smtp_message) are function-local lazy imports that keep resolving
through the email module's own path (email is not yet migrated).

One source-introspection test site repointed to the new canonical path:
- test_model_helper_owner_scope.py (shared with history; history entry
  already repointed in #5090, note entry repointed here)

Adds tests/test_note_routes_shim.py to pin the sys.modules shim contract
(legacy and canonical paths resolve to the same module object; monkeypatch
via legacy alias reaches the canonical module).

Verified: compileall clean; full suite 4487 passed, 3 skipped.
2026-07-20 13:52:30 +02:00
Afonso Coutinho 1aad1db9f6 fix: services research source extraction crashes on a non-dict finding (#1868) 2026-07-20 09:39:16 +02:00
Abhishek Kumbhar d05900cb90 fix(llm): enhance fallback logic to handle empty completions and impr… (#5491)
* fix(llm): enhance fallback logic to handle empty completions and improve metadata handling

* fix(llm): stream tool call deltas immediately
2026-07-18 22:06:14 +01:00
Joeseph Grey b3f8b77317 fix(url-safety): reject RFC 6598 shared address space in strict mode (#5474)
* security(url-safety): reject RFC 6598 shared address space in strict mode

Strict mode (block_private=True) is a full SSRF lockdown, but it only
rejected is_private and is_loopback targets. CPython does not classify RFC
6598 shared/CGNAT space (100.64.0.0/10) as is_private (it is "shared", not
"private"), so a public redirect into 100.64.0.1 passed the per-hop guard
and still issued the request to a potentially internal CGNAT service.

not is_global would also exclude it, but only on CPython 3.11.10+/3.12.4+/
3.13+; the CI matrix runs 3.11/3.12, so reject the range explicitly to stay
correct across patch levels and the 3.14 runtime image. Default local-first
mode is unchanged. Adds strict-mode coverage for shared, non-global, and
public targets.

* docs(url-safety): correct CGNAT is_global rationale in strict-mode comment

The prior comment claimed `not is_global` catches 100.64.0.0/10 only on
CPython 3.11.10+/3.12.4+/3.13+. That is inaccurate for CGNAT: is_global
is False for 100.64.0.1 on every supported version (verified 3.10-3.14).
The version-fragility applies to other ranges gh-113171 touched, not CGNAT.
The explicit range reject is still the right choice; restate the reason as
is_private not covering shared space, and not coupling strict mode to
is_global's broader, cross-version definition. No behavior change.
2026-07-18 12:36:27 -06:00
RaresKeY 23ac3e3e82 chore(release): bump dev version to 1.0.2 (#5473) 2026-07-18 17:06:16 +01:00
RaresKeY 5f481e7db1 test(cookbook): cover adopt remote host validation (#5225) 2026-07-18 12:07:48 +01:00
RaresKeY 0d47b78f47 test(email): cover agent draft owner isolation (#5226) 2026-07-18 11:30:28 +01:00
RaresKeY b9cafd67a1 feat(models): define capability schema and readers (#2739)
* feat(models): define capability schema and readers

* fix(models): harden Google catalog probing

Restrict native catalog probing to the Gemini host, keep provider keys out of request URLs, filter non-chat model resources, and preserve the manual refresh default in the built-in Google add flow.
2026-07-18 09:40:58 +01:00
Boody b4e5ad088a Merge pull request #5580 from abandonrule/main
fix(docker): bump Docker CLI to a patched release
2026-07-18 05:13:03 +03:00
Chris Mayfield 4239bc850b Merge pull request #3 from abandonrule/fix-docker-cli-cves
fix(docker): bump Docker CLI to a patched release
2026-07-17 17:23:24 -05:00
Christopher Mayfield 0f259ea5f6 fix(docker): bump Docker CLI to a patched release 2026-07-17 16:28:46 -05:00
Christopher Mayfield 27ecd41cde Merge branch 'odysseus-dev:main' into main 2026-07-17 16:22:02 -05:00
Christopher Mayfield 3718e61c40 Merge pull request #2 from abandonrule/sync/upstream-20260717-odysseus
chore: sync upstream changes from odysseus-dev/odysseus
2026-07-17 16:18:04 -05:00
Keshav Jindal a57dd37005 docs(setup): document Arch NVIDIA Docker GPU setup
Add Arch-specific package installation and NVIDIA runtime configuration
to the Docker setup guide. Cover passthrough verification, the NVIDIA
Compose overlay, and the distinction between GPU passthrough and
CUDA-backed model serving

Refs #831
2026-07-16 13:06:42 +02:00
Emir Çoban 45b6771e73 fix(skills): block private SSRF targets and revalidate redirects in importer (#5261)
* Harden skill importer against SSRF: block private targets + revalidate redirects per hop

The skill importer validated only the initial URL with the lenient SSRF guard
(block_private=False) and then fetched with follow_redirects=True, so a 3xx to
an internal/metadata address (169.254.169.254, 127.0.0.1, RFC-1918) was still
connected to — inconsistent with the hardened services/search/content.py
:_get_public_url path.

Add a _get_checked() helper that follows redirects manually and re-runs the
SSRF guard with block_private=True on every hop, and route all three fetch
sites (skills.sh unwrap, _fetch_bytes, _list_github_dir) through it. GitHub's
own redirects and the final-host _assert_github_url checks are preserved.

Adds hermetic regression tests (IP-literal hosts, faked HTTP layer) and updates
the existing mock signature for the new block_private kwarg.

Defense-in-depth: the endpoint is admin-gated (require_admin) and admins are
trusted per THREAT_MODEL.md, so this is not a cross-boundary vulnerability.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test: enforce follow_redirects=False invariant in mock client

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-14 08:33:10 -06:00
Boody 98bcb64192 fix(mcp_manager): remove timeout from MCP connection attempts and handle registration cleanup 2026-07-13 08:56:46 +02:00
Boody 0d1d28c7d6 fix(mcp_manager): implement concurrent server connections with timeout handling 2026-07-13 08:56:46 +02:00
Ethan c7b3b84f24 fix(db): restrict data/app.db to 0600 (#4420)
* fix(db): restrict data/app.db to 0600

app.db holds bearer-token hashes, bcrypt password hashes, and encrypted
provider keys but was created under the default umask (0644 -> world-readable),
unlike .app_key/vault/integrations which are already 0600 via safe_chmod.

init_db() now chmods the SQLite file to 0600 right after create_all (POSIX
only; no-op on Windows, skipped for Postgres / in-memory). Unconditional and
idempotent, so it also re-locks already-deployed 0644 installs on next
startup. The transient rollback journal inherits 0600 from the parent file at
creation - no sidecar handling needed; -wal/-shm don't exist until WAL is
enabled (#4409 C4) and inherit the same mode then.

Satisfies Rule B, unblocking #4413 and the vault/integration secret moves.
Mirrors src/secret_storage.py:43-45.

Verified: security + DB-permission suites pass; 6 pre-existing visual_report
failures (missing markdown/nh3 deps) are unrelated.

Closes #4407

* fix(db): harden SQLite path parsing and re-lock sidecars

Address review feedback on #4420.

P2: derive the file to chmod from engine.url (SQLAlchemy's parsed URL)
via _sqlite_db_path(), instead of DATABASE_URL.replace("sqlite:///", "").
A driver-qualified URL (sqlite+pysqlite://) or one carrying query args
(?cache=shared) previously slipped past the prefix check / string slice
and left the DB world-readable; the parsed path resolves correctly and
drops the query.

P3: re-lock stale -wal/-shm/-journal sidecars to 0o600 at startup. The
main file is chmod'd first, so any sidecar SQLite creates afterward
inherits 0o600, but a -wal/-shm left world-readable by an older 0o644
install (once WAL was enabled) could still expose DB pages. Absent
sidecars are the normal case, not an error.

Tests: unit-test _sqlite_db_path across driver/query/memory/postgres URL
forms, and a subprocess test asserting stale 0o644 -wal/-shm are
re-locked on startup.

* fix(db): handle sqlite file URI app db permissions

* fix(db): close remaining SQLite permission bypasses

---------

Co-authored-by: Ethan <23321960+0xLeathery@users.noreply.github.com>
Co-authored-by: Alexandre Teixeira <alexandremagteixeira@gmail.com>
2026-07-11 21:15:49 +02:00
Astarte bff38a4406 fix(cleanup): update MODULE_SUMMARY and remove dead MEMORY_DOC paths (#4411) (#5160)
* docs: update static/js/MODULE_SUMMARY.md to reflect current ES6 frontend

Rewrite the stale module summary to match the current no-build,
ES6-module frontend architecture. Adds coverage of app.js orchestration,
the chat/SSE pipeline (chat.js, chatStream.js, chatRenderer.js,
streamingRenderer.js), new subsystems (research/, compare/, document
streaming, cookbook*, skills.js), and removes the obsolete <script> load
order assumptions.

* cleanup: remove dead MEMORY_DOC / memory_doc paths (closes #4411)

Removes the unused MEMORY_DOC constant and the matching DataConfig
memory_doc field / set_data_paths entry. No runtime code imports or
references these paths, so this is a no-behavior-change dead-code
cleanup under the storage-architecture tracker #4377.
2026-07-11 17:06:19 +01:00
falabellamichael c2d2075833 fix(stabilization): harden attachment lifecycle and agent guard signals (#5420)
* fix: harden stabilization attachment and agent guards

* fix(uploads): preserve durable references during cleanup

* fix(uploads): close cleanup and compaction races
2026-07-11 15:14:14 +01:00
RaresKeY a02f8d8600 fix(llm): avoid blocking Kimi Code async header probes (#5231) 2026-07-11 15:06:15 +01:00
mashallow d02565ce32 fix(markdown): stop currency dollars rendering as KaTeX inline math (#5132) 2026-07-11 14:45:57 +01:00
RaresKeY 524fa9dce2 fix: preserve pythonpath for built-in mcp servers (#5117) 2026-07-11 14:34:42 +01:00
RaresKeY b3432873fb fix(email): clear bulk selection on context change (#5229) 2026-07-11 14:12:12 +01:00
Peter Karlsson 1c61c358cb fix(email): use UID commands instead of sequence numbers in IMAP fetches (#5149)
conn.search() / conn.fetch() operate on volatile positional sequence
numbers that shift whenever messages are deleted or expunged. Three call
sites in the sig-learner (_pull_headers, _fetch_bodies) and morning-brief
email section were storing these as "uid" and reusing them in subsequent
fetches — causing wrong-message returns or NO responses if another client
modified the mailbox concurrently.

Replaced with conn.uid("SEARCH", ...) / conn.uid("FETCH", ...), which use
persistent RFC 3501 UIDs. _scan_one (urgency action) already did this
correctly; these were the remaining callers.

The reproduction window is narrow (requires concurrent deletion between
search and fetch), so the fix is verified by regression tests rather than
manual end-to-end: _SpyImap raises AssertionError if conn.search() or
conn.fetch() are called instead of conn.uid().
2026-07-11 14:06:40 +01:00
DL Techy e5ef8cf4bf fix(chat): Expand user chat bubble edit textbox width (#3963)
* fix(chat): Expand user chat bubble edit textbox width

- Update user chat bubble width from `fit-content` to `85%` to ensure consistency with the AI chat bubble edit textbox width.

* style(chat): Refine user message bubble width logic

- Change general bubble width to `fit-content`
- Set width to 85% specifically for user messages containing a `textarea`
2026-07-11 13:52:14 +01:00
jagadish-zentiti 410ad9a2fa fix(agent): cancel orphaned tool task when SSE client disconnects mid-call (#5106)
stream_agent_loop's per-tool drain loop had no cleanup path for early
generator close. Starlette throws GeneratorExit into the generator at
whatever await point it's suspended on when the SSE client disconnects
(aclose()) - here that's 'await _progress_q.get()' inside the drain
loop, before the final 'await _tool_task' line ever runs. The task,
which wraps execute_tool_block, was left running unawaited and
uncancelled.

For bash/python tools this orphans the underlying subprocess:
subprocess_tools.py already has correct CancelledError handling that
kills the child process, but only runs if the task is actually
cancelled. A client disconnecting mid long-running command left that
subprocess running server-side for its full duration with nothing
left to reap it.

Wrap the drain loop in try/finally: on early exit, cancel _tool_task
(if not already done) and await it so the existing subprocess-kill
path runs.

Adds a regression test that drives the real stream_agent_loop with a
fake tool handler, closes the generator mid tool-call (mirroring what
Starlette does on disconnect), and asserts the handler observed
cancellation immediately - not merely via asyncio.run()'s own
end-of-run task cleanup, which would mask the bug.

Fixes #5105
2026-07-11 13:44:06 +01:00
Wes Huber c0b032fa10 fix(calendar): trust operator CA bundle in CalDAV test_connection (#4796)
* fix(calendar): trust operator CA bundle in CalDAV test_connection

The pre-flight test used httpx with trust_env=False, which ignored
SSL_CERT_FILE/REQUESTS_CA_BUNDLE. Self-signed CalDAV servers that
the real sync accepts (via caldav lib → requests → honors bundle)
were rejected by the test with CERTIFICATE_VERIFY_FAILED.

Build an explicit SSL context that loads the operator's CA bundle
and clears VERIFY_X509_STRICT (which rejects certs without a
keyUsage extension — common in self-signed setups). SSRF guards
(follow_redirects=False, trust_env=False) are preserved.

Fixes #4795
Fixes #4779

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(calendar): add regression tests and edge case handling for SSL context

Per review: add route-level regression tests covering SSL_CERT_FILE
precedence, VERIFY_X509_STRICT clearing, missing bundle graceful
fallback, and empty env var handling. Also log a warning when the
configured CA bundle path doesn't exist instead of silently falling
back to system CAs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(calendar): rewrite SSL tests to exercise route handler directly

Addresses review feedback: tests now use FastAPI TestClient to hit the
actual test_connection route, capturing the verify= kwarg passed to
httpx.AsyncClient. This ensures the route's SSL context construction
is covered, not a test-side duplicate.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* ci: retrigger CI (redirect hardening test is a CI-env flake, passes locally)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(tests): remove module-level sys.modules stubs that leaked into other tests

The collection-time MagicMock stub of `caldav` replaced the real library
for every later test in the same process — test_caldav_redirect_hardening's
DAVClient became a mock that never sent the PROPFIND, failing its
must-reach-the-public-server assertion in CI. conftest already pre-imports
the real sqlalchemy/core.database, and the route's lazy imports are patched
per-request, so the stub block was both harmful and unnecessary.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(calendar): verify exact CA bundle precedence

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Alexandre Teixeira <alexandremagteixeira@gmail.com>
2026-07-11 13:25:16 +01:00
tanmayraut45 c161866199 CalDAV: close the DAVClient on sync and write-back paths (#4793)
_sync_blocking (src/caldav_sync.py) and _writeback_blocking
(src/caldav_writeback.py) each open their own caldav.DAVClient via
_build_dav_client, but never close it. The client owns an HTTP session
with a pooled connection; without a close() that connection is held until
process exit.

Previously the fix added explicit client.close() calls before each early
return and at the end of the DB finally block. This still leaked the
client when SessionLocal() raised before the DB try/finally was entered.

Now _sync_blocking wraps the entire post-construction path in an outer
try/finally that calls client.close() unconditionally, covering:
  - AuthorizationError / NotFoundError early return
  - URL-fallback failure early return
  - no-calendars early return
  - normal return after sync
  - SessionLocal() construction failure (new regression coverage)

_writeback_blocking already used a try/finally (unchanged).

- src/caldav_sync.py: replace scattered client.close() calls with a
  single outer try/finally block around the discovery + DB sync path
- tests/test_caldav_client_cleanup.py: add CalendarDeletedEvent to the
  database stub; add regression test for SessionLocal() failure path

Closes #4593
2026-07-11 13:03:24 +01:00
red person 6725d1863c Skip vanished backup list entries (#2006) 2026-07-11 05:26:42 +01:00
jagadish-zentiti 99f0facc2c fix(mcp): guard DbTokenStorage against non-dict oauth_tokens JSON (#5107)
_load() returned whatever json.loads() produced without checking it was a
dict; _update() did the same before assigning data[key] = value. If the
oauth_tokens column ever held a JSON array or primitive (DB corruption,
manual edit, migration drift), _load()'s callers crashed with
AttributeError on .get(), and _update() crashed with TypeError trying to
item-assign into a list/string/int.

Validate the parsed value is a dict in both methods, falling back to {}
otherwise - same recovery behavior already used elsewhere in the codebase
for this exact JSON-blob-is-not-a-dict shape (_parse_tool_args,
_is_sensitive_path's siblings).

Adds 3 regression tests for _load, get_tokens, and _update against a
non-dict oauth_tokens value.

Fixes #5082
2026-07-11 05:26:23 +01:00
jagadish-zentiti 7a8f47e4ab fix(email): atomically claim scheduled emails before sending (#5110)
_scheduled_poll_once selected rows WHERE status='pending' and only wrote
status='sent'/'failed' after the SMTP send and IMAP append completed -
no atomic claim in between. Two overlapping callers (the in-process 30s
poller and an externally cron/systemd-driven 'odysseus-mail
poll-scheduled', or the CLI run manually) can both SELECT the same
pending row before either UPDATEs it, and both send it. _start_poller's
own docstring already names this exact risk ('avoid two copies of
_scheduled_poll_once racing on the same SQLite') but nothing in the code
enforced it - it was advisory only.

Add an atomic per-row claim: UPDATE ... SET status='sending' WHERE
id=? AND status='pending', proceeding only when rowcount == 1. The
loser of the race sees rowcount == 0 and skips the row instead of
sending a duplicate.

Adds a regression test that drives two real threads through the real
_scheduled_poll_once against a shared SQLite file, synchronized with a
barrier and a widened send-path window, and asserts exactly one send
fires. Reverting the fix makes the test fail reliably (5/5 runs); with
the fix it passes reliably (5/5 runs).

Fixes #5109
2026-07-11 04:23:36 +01:00
L1 a6c457f74e fix(email): never fall back to sequence-number IMAP ops for move/flag (#2732)
_store_email_flag and _move_email_message (used by the archive / delete / move /
mark-read endpoints) had an else branch that, when _uid_exists returned False,
ran conn.store(uid, ...) / conn.copy(uid, ...) followed by a folder-wide
conn.expunge(). But imaplib's plain store()/copy() take a message SEQUENCE
NUMBER, not a UID, so the op landed on whichever message occupied sequence
position == the UID value, and the expunge then permanently removed it. A stale
cached UID (or a server whose UID probe misbehaves) therefore deleted an
unrelated email instead of reporting 'not found'.

There is no valid case where treating a UID as a sequence number is correct, so
drop the fallback: when the UID isn't present, return False — callers already
surface 'Email not found'. Only the UID command path remains.

Sibling of #1874 (which fixes the auto-spam poller's _imap_move in
email_helpers.py); this covers the user-facing endpoints in email_routes.py.
Part of #2124.
2026-07-11 03:27:28 +01:00
Afonso Coutinho a4e66bb59f fix: TTS available crashes on non-string tts_provider (#2034) 2026-07-11 03:19:51 +01:00
Afonso Coutinho a6efea5486 fix: _matchesCombo crashes on non-string keybind from server (#2049) 2026-07-11 03:15:19 +01:00
Afonso Coutinho 565c69f40d fix: odysseus-memory cmd_add crashes on non-dict existing memory row (#2091) 2026-07-11 03:05:17 +01:00
Afonso Coutinho 06e038f00e fix: hwfit params_b/is_prequantized crash on non-string catalog fields (#2094) 2026-07-11 03:00:28 +01:00
Ashvin 0cb8db4de4 fix(tasks): scope manage_tasks mutations to an exact task owner (#5264)
The edit/delete/pause/run actions of do_manage_tasks gated ownership with
`if owner and task.owner and task.owner != owner`. The middle term made the
check a no-op whenever task.owner was null/empty — the state a scheduled task
sits in when it was created in no-login mode (or via the localhost middleware
bypass) before the periodic legacy-owner sweep reassigns it to the admin user.
Any authenticated user's agent could then edit, delete, pause, or run another
tenant's owner-less task; edit+run lets an attacker rewrite the task prompt and
execute it in the scheduler's agent context.

The sibling `list` action already scopes with an exact `owner == owner` filter,
so the mutators were strictly more permissive than the reader. Drop the middle
term so the guard fails closed on owner-less rows for authenticated callers,
matching `list` and the calendar/notes/gallery/session null-owner gates. Auth
disabled (owner falsy) and same-owner access are unchanged.
2026-07-11 01:45:14 +01:00
Am-GJ 851bf4d0c8 fix(reminders): sanitize ntfy Title header to ASCII (#5208)
* fix(reminders): sanitize ntfy Title header to ASCII

The ntfy notification Title header was set directly from the note title.
HTTP headers must be ASCII, so a title containing emoji or other
non-ASCII characters caused httpx to raise UnicodeEncodeError, which
was swallowed by the surrounding try/except — so the reminder silently
failed and no notification was ever sent.

Sanitize the title with encode('ascii', 'replace') before placing it
into the header, replacing unsupported characters with '?'. This is
standard practice for HTTP header values. The note body is unaffected
(it is sent as request content, not a header) and continues to support
full UTF-8.

* fix(reminders): also truncate ntfy title to 200 chars for header safety

* style: compact ntfy header comment

---------

Co-authored-by: Am-GJ <Am-GJ@users.noreply.github.com>
Co-authored-by: RaresKeY <158580472+RaresKeY@users.noreply.github.com>
2026-07-10 22:50:00 +02:00
Wes Huber a0a24058bb docs: remove completed troubleshooting cookbook task from ROADMAP (#4906)
The self-host troubleshooting cookbook has been implemented in
docs/setup.md under "Common self-host traps" (PR #4834).

Fixes #4900

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-07-10 22:29:20 +02:00
Boody 582a51054d Merge pull request #5313 from RaresKeY/fix/chat-web-search-explicit-deny
fix(chat): require explicit web search enable
2026-07-09 01:56:32 +03:00
Wes Huber 21c8053505 fix(copilot): guard request_flags against a non-dict last message (#5274)
request_flags derives (agent, vision) and does last.get("role") after only
a truthy check. A client can send a bare-string message element
("messages": ["hi"]), and the vision loop right below already guards each
element with isinstance — so the .get() on a non-dict last element is an
oversight that raises AttributeError on every Copilot-proxied request with
such a body.

Use isinstance(last, dict) to match the loop's own guard.

Fixes #5273

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
2026-07-08 23:57:23 +02:00
RaresKeY 109301be78 docs(docker): polish WSL2 snap GPU guidance 2026-07-08 19:30:41 +00:00
Miraç Duran 93c2501a6d fix(chat): give extensionless image/audio uploads a valid MIME subtype (#5205)
build_user_content derived the data-URL subtype from the file extension
only (image_format = ext[1:]). An extensionless upload (e.g. a pasted
screenshot) has ext == "", producing "data:image/;base64,..." with an
empty subtype (invalid per RFC 2046) that vision/audio endpoints reject,
silently dropping the attachment. Fall back to the resolved MIME subtype
when the extension is missing; present extensions are unchanged.
2026-07-08 21:04:15 +02:00
RaresKeY dadf178ed5 fix(chat): require explicit web search enable 2026-07-08 17:44:28 +00:00
Steve Holloway 30e87e3b82 fix(chat): restore missing _explicit_web_intent definition (#5290)
chat_stream() references `_explicit_web_intent` in three places
(disabled-tools gating, global-disabled web allowance, and the
per-turn tool filter) but the assignment was dropped during a
branch merge. Every chat request raised

    NameError: name '_explicit_web_intent' is not defined

at routes/chat_routes.py, surfacing to the client as a bare
"Internal Server Error" before any LLM call was made — chat was
fully broken on dev and main.

Restore the original definition, computed from the already-derived
tool intent, immediately before its first use:

    _explicit_web_intent = bool(_tool_intent and _tool_intent.category == "web")

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 19:14:43 +02:00
Jeffery Tse db1170e63a fix(docker): detect snap+WSL2 GPU passthrough incompatibility, document fix 2026-07-07 01:02:55 -04:00
PewDiePie d9f6341f01 Merge pull request #5283 from pewdiepie-archdaemon/sync-main-into-dev-20260707
chore: sync tested main into dev
2026-07-07 11:32:53 +09:00
26 changed files with 9194 additions and 0 deletions
+62
View File
@@ -0,0 +1,62 @@
# Discovery Baseline Status
## Purpose and scope
This package is a commit-pinned discovery baseline and feature index for the `discovery` branch at frozen commit `d8a2059df8e53bc7275c45339849d14c8651e73c`. It inventories **79 feature records** across **16 domains** to help maintainers locate likely implementation areas and identify validation gaps. It is not an authoritative architecture reference or a runtime-certification record.
The canonical inventory is [`feature-catalog.json`](feature-catalog.json); [`feature-catalog.md`](feature-catalog.md) and the files in [`domains/`](domains/) are derived reading views. See [`audit-method.md`](audit-method.md) for the status and maturity definitions.
## What maintainers may use now
- Use the catalog and domain views as a frozen discovery index, including their feature IDs, stated scope, likely source locations, and declared runtime prerequisites.
- Treat a catalog status such as `verified` as meaning implementation was identified during discovery. It does **not** mean every evidence locator, line range, test claim, or runtime behaviour has passed semantic validation.
- Use the structural checks to confirm package shape and cross-view consistency; use the evidence validator to assess whether individual evidence assertions are semantically supported.
## Validation snapshot
Structural validation passes: the catalog has 79 records, the 16 domain views match it, and the package structural validators pass. All 11 focused evidence-validator tests pass.
The evidence validator found **170 evidence entries**: **82 valid**, **66 invalid**, **6 ambiguous**, and **16 unsupported**. Its non-zero exit is expected while those semantic evidence defects remain.
Structural validity checks the documentation schema, record counts, derived-view consistency, file existence, line-range bounds, links, and public-safety rules. Semantic evidence validity additionally checks whether the cited locator exists, falls within its cited range, uses a supported parser, and actually supports the feature claim. Passing the former does not establish the latter.
## E2 review decisions
E2 means directly relevant automated test evidence supports the feature claim; a test files existence alone is insufficient. The generated evidence report was used to reassess all ten E2 records.
| Feature | Decision | Reason |
|---|---|---|
| `CHAT-001` | Retain E2 | Two cited tests directly exercise documented streaming-related behaviour. |
| `MODEL-006` | Retain E2 | The cited device-flow test exercises the Copilot start/poll contract and verification URI. |
| `MODEL-007` | Retain E2 | The cited device-flow test exercises the ChatGPT subscription contract and verification URI. |
| `RESEARCH-003` | Demote to E1 | The cited test covers ranking, not provider connectivity or dispatch; the route-to-provider implementation path was identified. |
| `DOCUMENT-002` | Demote to E1 | The cited marker test does not support form processing or rendering; the document route does call the PDF form handlers. |
| `EMAIL-001` | Demote to E1 | The cited health test is narrower than setup, SMTP, and polling; application setup invokes the email router and its poller. |
| `SECURITY-002` | Demote to E1 | Relevant vault-password tests exist, but the cited test locator is fabricated; the application mounts the vault route implementation. |
| `SECURITY-004` | Demote to E1 | Relevant injection tests exist, but the cited locator names are fabricated; callers use the documented context guard. |
| `SECURITY-005` | Demote to E1 | Relevant guard tests exist, but the cited locator names are fabricated; route code calls the documented URL guard. |
| `PLATFORM-009` | Demote to E0 | The manifests and diagnostic script establish discovered operational artifacts, not a traced application path or directly relevant automated test. |
The current maturity distribution is **E0: 68**, **E1: 8**, **E2: 3**, **E3: 0**, **E4: 0**. Runtime validation is still pending where the catalog says it requires external services, interactive authentication, specialised hardware, or host Docker GPU support.
## Known discovery caveats
- `AGENT-004` includes a legacy no-op activity-log shim rather than active assistant-log behaviour.
- `FRONTEND-007` points to a missing `static/backgrounds.html` target; its existing variant pages do not make that route functional.
- `RESEARCH-003` retains a compatibility module that aliases the canonical search implementation, and `DOCUMENT-002` separates optional PDF viewing from form handling.
- Secret-storage and vault-command handling are distinct implementation areas; this index does not make an end-to-end security guarantee.
## Recommended next documentation work
Repair semantic evidence selectively while architecture and operations documentation is written, beginning with the seven E2 demotions and maintainer-owned feature descriptions. Do not wait for the complete evidence queue before documenting the system. Record controlled runtime observations when external services, credentials, hardware, or Docker GPU access are available, and label unsupported claims explicitly.
## Validation commands
```bash
PYTHONDONTWRITEBYTECODE=1 python3 docs/discovery/tools/validate_discovery_docs.py --repo-root .
PYTHONDONTWRITEBYTECODE=1 python3 docs/discovery/tools/validate_discovery_consistency.py
PYTHONDONTWRITEBYTECODE=1 python3 -m unittest docs/discovery/tools/test_validate_discovery_evidence.py -v
PYTHONDONTWRITEBYTECODE=1 python3 docs/discovery/tools/validate_discovery_evidence.py --repo-root . --catalog docs/discovery/feature-catalog.json --output-dir <local-report-directory>
```
Supply a local report directory outside `docs/discovery/` for the final command so generated reports are not added to the package.
+21
View File
@@ -0,0 +1,21 @@
# Odysseus Discovery Package
## Provisional discovery baseline
This is a commit-pinned discovery baseline and feature index for the `discovery` branch at `d8a2059df8e53bc7275c45339849d14c8651e73c`. It contains 79 feature records across 16 domains. It is **not** an authoritative architecture reference, a runtime certification, or a claim that every evidence citation is semantically valid.
Read [`BASELINE-STATUS.md`](BASELINE-STATUS.md) first for the publication status, evidence-validation totals, E2 decisions, known caveats, and the recommended next documentation work.
## Package contents
- [`feature-catalog.json`](feature-catalog.json) is the canonical machine-readable catalog.
- [`feature-catalog.md`](feature-catalog.md) and [`domains/`](domains/) are derived reading views.
- [`audit-method.md`](audit-method.md) defines feature status and evidence maturity.
- [`references/source-provenance.md`](references/source-provenance.md) records the frozen snapshot.
- [`tools/`](tools/) contains the structural, consistency, and evidence validators.
A feature status such as `verified` means implementation was identified during discovery. It does not mean every evidence locator, line range, test claim, or runtime behaviour has passed semantic validation.
## Validation
Run the commands in [`BASELINE-STATUS.md`](BASELINE-STATUS.md#validation-commands). Structural checks and semantic evidence checks have different purposes; see that status document for the current results and interpretation.
+80
View File
@@ -0,0 +1,80 @@
# Odysseus Functional Audit Methodology
## Purpose
This document specifies the methodology and evidence standards for the read-only discovery audit of **Odysseus**.
## Snapshot Baseline
- **Repository**: `odysseus-dev/odysseus`
- **Audit Target Branch**: `discovery`
- **Frozen Commit SHA**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Snapshot Date**: `2026-07-23T14:49:02Z`
- **Audit Mode**: Read-Only inventory & documentation review
## Rules of Engagement
1. **No Code Mutations**: Application code and tests outside `docs/discovery/` remain untouched.
2. **No External Operations**: No GitHub issues, PRs, comments, labels, or branch mutations.
3. **Zero Inferred Success**: Documentation claims require empirical evidence of implementation and reachability. Filenames, README descriptions, and docstrings alone do not constitute proof.
4. **Strict Status Categorization**: All capabilities are assigned exactly one authorized status:
- `verified`: Implemented, reachable, and supported by code evidence.
- `partial`: Partially implemented or missing full frontend/backend connection.
- `disabled`: Gated off by default feature flags or configuration.
- `experimental`: Active but requiring non-standard hardware or runtimes.
- `legacy`: Obsolete feature retained for backwards compatibility.
- `dead-code-candidate`: Code exists but is unreachable from UI or API routes.
- `unverified`: Implementation present but untestable without external secrets or hardware.
## Evidence Maturity Scale
Evidence maturity is evaluated independently from catalog feature status. A feature status such as `verified` records that implementation was identified during discovery; it is not a statement that every evidence locator, test claim, line range, or runtime behaviour has passed semantic validation.
- **E0 - Discovered**: Candidate identified in documentation, route declaration, or source file.
- **E1 - Code-path traced**: Frontend/API entry point connected through services and data handlers.
- **E2 - Test-backed**: At least one directly relevant automated test assertion supports the feature claim. A test file's existence, an unrelated assertion, or an invalid test locator does not establish E2.
- **E3 - Runtime-validated**: Maintainer reproduced behavior in a recorded local environment.
- **E4 - Maintainer-accepted**: Maintainers accepted the feature description and support status.
## Audit Workflow
```mermaid
flowchart TD
P0["Phase 0: Snapshot Isolation<br/>(Commit d8a2059)"] --> P1["Phase 1: Codebase Discovery<br/>(Routes, Services, Static JS, Specs)"]
P1 --> P2["Phase 2: Feature Reachability & Verification<br/>(Route matching, FE entrypoints, tests)"]
P2 --> P3["Phase 3: Catalog & Evidence Compilation<br/>(File paths, symbols, exact line ranges)"]
P3 --> P4["Phase 4: Quality & Integrity Audit<br/>(100% path existence check, schema validation)"]
```
### Phase 0: Snapshot Isolation
The audit is pinned to git commit `d8a2059df8e53bc7275c45339849d14c8651e73c`. All file paths, symbol declarations, and line ranges map strictly to this commit.
### Phase 1: Codebase Discovery
All top-level and nested directories were traversed, including:
- Backend Entry Points (`app.py`, `routes/`, `routes/*/*.py`, `companion/`)
- Core Framework (`core/database.py`, `core/session_manager.py`, `core/auth.py`)
- Business Logic Services (`src/`, `services/`, `mcp_servers/`)
- Frontend Assets (`static/app.js`, `static/js/`, `static/index.html`)
- Test Suites (`tests/`, `tests/cli/`, `tests/streaming/`)
- Operations & Docker (`Dockerfile`, `docker-compose*.yml`, `scripts/`)
### Phase 2: Verification Protocol
For each feature candidate, the following table was evaluated:
- **User Reachability**: Frontend UI element, modal, route, or CLI script.
- **API Entrypoint**: FastAPI `@router` declaration or WebSocket/SSE handler.
- **Backend Execution**: Concrete Python module method, service, or tool call.
- **Data Persistence**: Disk file, SQLite table, or vector collection.
- **Test Coverage**: Automated test file executing assertions against the component.
### Phase 3: Evidence Linking Standard
Every feature entry in `feature-catalog.json` contains a structured `evidence` list with:
- `path`: Relative path from repository root.
- `symbol`: Route, class, function, or element symbol name.
- `line_range`: Inclusive line range (e.g. `L120-L250`).
- `explanation`: Short factual statement proving reachability or implementation.
### Phase 4: Quality Check & Schema Constraints
Before finalization:
1. Every evidence file path is validated against the checkout.
2. Every Markdown entry matches `feature-catalog.json`.
3. Recommendation language is separated from empirical factual observations.
+153
View File
@@ -0,0 +1,153 @@
# Agent
Features in this document are generated from [`../feature-catalog.json`](../feature-catalog.json), the canonical inventory.
## AGENT-001 — Autonomous Agent Loop & Tool Execution Engine
- **Domain**: `agent`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Executes multi-step agent reasoning loops, tool invocation parsing, and automated response generation.
### Evidence summary
- `src/agent_loop.py``run_agent_loop` — Core loop evaluating model tool requests and executing handlers.
- `src/tool_execution.py``execute_tool_call` — Dispatches tool invocation requests to underlying tool handlers.
### Unknowns
- Infinite tool loop if termination condition fails.
## AGENT-002 — Scheduled Tasks & Event Bus Dispatcher
- **Domain**: `agent`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Schedules background recurring or delayed tasks, emits event bus triggers, and executes automated flows.
### Evidence summary
- `routes/task_routes.py``@router.get('')` — Fetches active scheduled tasks.
- `src/task_scheduler.py``TaskScheduler` — Async task scheduler dispatching cron and delay triggers.
### Unknowns
- Task execution failure handling on system restart.
## AGENT-003 — Webhook Event Subscriptions & Trigger Processing
- **Domain**: `agent`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Manages incoming/outgoing webhook subscriptions, endpoint authentication tokens, and event triggers.
### Evidence summary
- `routes/webhook_routes.py``@router.get('/webhooks')` — Returns list of registered webhooks.
- `src/webhook_manager.py``WebhookManager` — Handles payload delivery and signature verification.
### Unknowns
- SSRF risks when contacting external webhook URLs if unvalidated.
## AGENT-004 — Assistant Settings, Task Check-Ins & Background Job Monitor
- **Domain**: `agent`
- **Status**: `partial`
- **Evidence Maturity**: `E1`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Manages per-user assistant sessions and scheduled check-in settings, drains background job completions, and retains a legacy no-op activity logging shim.
### Evidence summary
- `routes/assistant_routes.py``setup_assistant_routes` — Active assistant session, settings, manual check-in, run-status and timezone-list endpoints, including the owner-scoping guards.
- `src/bg_monitor.py``_drain_agent` — Runs the agent loop headless against a session to produce the background-job follow-up turn.
- `src/bg_monitor.py``_run_followup` — Drains completed background jobs and auto-continues the owning session, deferring while a live turn is in progress.
- `src/assistant_log.py``log_to_assistant` — Legacy no-op activity logging shim retained for existing callers; documented as inactive rather than as current behaviour.
### Unknowns
- Route `/api/assistant/logs` cited in legacy docs is absent from assistant router.
- Existing unit test `tests/cli/test_logs_cli_resolve_nonstring.py` tests CLI target-name resolution logic, not active assistant routes or bg_monitor execution loop.
## AGENT-005 — Model Context Protocol (MCP) Server Integration
- **Domain**: `agent`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Integrates external MCP servers over stdio/SSE to expand agent capabilities dynamically.
### Evidence summary
- `routes/mcp_routes.py``setup_mcp_routes` — Exposes management endpoints for external MCP servers.
- `src/mcp_manager.py``McpManager` — Manages MCP server subprocess lifecycles.
### Unknowns
- Subprocess leaks if external MCP server process fails to terminate clean.
## AGENT-006 — AI Interaction Tools & Pipeline Orchestration
- **Domain**: `agent`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Provides specialized AI interaction tools for agent self-debugging, debate, and multi-model collaboration.
### Evidence summary
- `src/ai_interaction.py``init_ai_interaction_tools` — Registers specialized multi-agent interaction primitives.
- `src/builtin_actions.py``execute_builtin_action` — Executes pre-built action intent sequences.
### Unknowns
- High API token consumption during extended agent debates.
## AGENT-007 — Subprocess & Background Job Execution Tools
- **Domain**: `agent`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Provides sandboxed bash/shell tool execution capabilities with output streaming and background tracking.
### Evidence summary
- `src/agent_tools/subprocess_tools.py``run_command` — Executes shell commands in background/foreground.
- `src/bg_jobs.py``JobManager` — Tracks async background subprocess tasks.
### Unknowns
- Arbitrary shell command execution permissions if sandbox confinement fails.
+45
View File
@@ -0,0 +1,45 @@
# Calendar
Features in this document are generated from [`../feature-catalog.json`](../feature-catalog.json), the canonical inventory.
## CALENDAR-001 — CalDAV Calendar Synchronization & Account Setup
- **Domain**: `calendar`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: pending — Requires a controlled external CalDAV server.
### Purpose
Connects to remote CalDAV servers (Apple iCloud, Nextcloud, Google) to sync calendar event feeds.
### Evidence summary
- `routes/calendar_routes.py``setup_calendar_routes` — Exposes CalDAV setup and manual sync trigger routes.
- `src/caldav_sync.py``CalDavSync` — Fetches and parses remote iCalendar VEVENT objects.
### Unknowns
- Invalid SSL certificates on self-hosted CalDAV servers.
## CALENDAR-002 — Calendar Event Operations & iCalendar Parsing
- **Domain**: `calendar`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Creates, updates, deletes, and displays calendar events with timezone conversion and reminder notifications.
### Evidence summary
- `routes/calendar_routes.py``@router.get('/events')` — Fetches calendar events for requested date window.
- `src/tools/calendar.py``CalendarTool` — Agent tool for creating and modifying calendar entries.
### Unknowns
- Recurring RRULE event expansion calculation bugs across leap years.
+197
View File
@@ -0,0 +1,197 @@
# Chat
Features in this document are generated from [`../feature-catalog.json`](../feature-catalog.json), the canonical inventory.
## CHAT-001 — Core Chat Streaming & SSE Message Generation
- **Domain**: `chat`
- **Status**: `verified`
- **Evidence Maturity**: `E2`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: pending — Requires access to a live LLM provider endpoint (OpenAI API key or local Ollama server).
### Purpose
Handles real-time Server-Sent Events (SSE) chat streaming, token rendering, and model response generation.
### Evidence summary
- `routes/chat_routes.py``chat_stream` — POST /api/chat_stream SSE endpoint; builds the shared chat context, then dispatches to the chat-mode or agent-mode streaming path.
- `routes/chat_helpers.py``build_chat_context` — Shared context builder invoked by chat_stream; runs message preprocessing and assembles the memory/RAG/web context preface.
- `src/chat_handler.py``ChatHandler.preprocess_message` — Message preprocessing (attachments, URLs, tool preprocessing) reached from build_chat_context via routes/chat_helpers.py:preprocess.
- `src/chat_processor.py``ChatProcessor.build_context_preface` — Builds the retrieval and web-source context preface injected into the streamed request.
- `src/llm_core.py``stream_llm_with_fallback` — Chat-mode streaming dispatcher called from chat_stream; wraps stream_llm with an ordered provider fallback chain.
- `src/llm_core.py``stream_llm` — Per-request streaming entry wrapped by stream_llm_with_fallback; acquires the local model slot and delegates to _stream_llm_inner.
- `src/agent_loop.py``stream_agent_loop` — Agent-mode streaming path called from chat_stream when the request selects agent mode.
- `tests/test_chat_metrics.py``test_stream_llm_passes_through_llamacpp_timings` — Inspected unit test asserting stream_llm forwards backend generation timings into the emitted metrics chunk.
- `tests/test_resend_message_nondestructive.py``test_resend_message_does_not_truncate_by_default` — Inspected unit test asserting the frontend resend path does not truncate prior conversation turns.
### Unknowns
- Stream interruption on connection drops requires retry logic.
## CHAT-002 — Session Management & Conversation State
- **Domain**: `chat`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Manages session creation, listing, switching, renaming, and persistence of conversation metadata.
### Evidence summary
- `routes/session_routes.py``@router.get('/api/sessions')` — Lists active sessions filtered by user owner scope.
- `core/session_manager.py``SessionManager` — Provides thread-safe session storage operations.
### Unknowns
- Concurrent file writes to sessions.json under high load.
## CHAT-003 — Chat History & Message Editing/Truncation
- **Domain**: `chat`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Provides history retrieval, message content updating, message deletion, and history branch truncation.
### Evidence summary
- `routes/history/history_routes.py``@router.get('/api/history/{session_id}')` — Fetches message history timeline for a session.
- `routes/history_routes.py``_sys.modules[__name__] = _canonical` — Backward-compatibility shim module.
### Unknowns
- Truncating messages re-indexes context window and clears cached tool calls.
## CHAT-004 — File & Multimodal Attachment Handling
- **Domain**: `chat`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Handles uploading, mime validation, image preview, vision encoding, and file attachments in chat messages.
### Evidence summary
- `routes/upload_routes.py``@router.post('')` — Accepts multi-part file uploads and generates vision metadata.
- `src/upload_handler.py``UploadHandler.save_file` — Validates upload size and atomicity on disk.
### Unknowns
- Large file uploads may consume server disk space if cleanup task fails.
## CHAT-005 — Chat Message Search
- **Domain**: `chat`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Enables full-text keyword search across stored chat messages and sessions.
### Evidence summary
- `routes/search_routes.py``setup_search_routes` — Registers chat message search endpoint.
- `src/session_search.py``search_sessions` — Executes query matching against session transcripts.
### Unknowns
- Full table scans on un-indexed text columns for very large databases.
## CHAT-006 — System Prompts & Preset Management
- **Domain**: `chat`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Provides creation, selection, and customization of system prompt presets for chat sessions.
### Evidence summary
- `routes/preset_routes.py``setup_preset_routes` — API routes for listing and modifying system prompt presets.
- `src/preset_manager.py``PresetManager` — Disk-backed manager for prompt presets.
### Unknowns
- Invalid JSON syntax in user presets file can corrupt preset loading.
## CHAT-007 — Emoji Rendering & Twemoji SVG Proxy
- **Domain**: `chat`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Proxies Twemoji SVG icons locally to render flat SVG emojis in message text without external CDN dependencies.
### Evidence summary
- `routes/emoji_routes.py``setup_emoji_routes` — Serves locally cached Twemoji SVGs.
### Unknowns
- First request fetches SVG from remote CDN before caching locally.
## CHAT-008 — Input History Recall (Arrow Up)
- **Domain**: `chat`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Allows users to cycle through previously sent prompt messages in the chat composer input using Arrow-Up/Down keys.
### Evidence summary
- `static/js/composerArrowUpRecall.js``initComposerRecall` — Listens for ArrowUp keypress on composer textarea.
### Unknowns
- Client-side browser storage limits.
## CHAT-009 — Context Window Compaction & Truncation
- **Domain**: `chat`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Compacts session transcript history when prompt size exceeds context limits using summarization.
### Evidence summary
- `routes/history/history_routes.py``@router.post('/api/session/{session_id}/compact')` — Triggers context summarization and compaction.
- `src/context_compactor.py``compact_context` — Executes context token pruning and summary generation.
### Unknowns
- Aggressive compaction may discard subtle user instructions.
+24
View File
@@ -0,0 +1,24 @@
# Contact
Features in this document are generated from [`../feature-catalog.json`](../feature-catalog.json), the canonical inventory.
## CONTACT-001 — CardDAV Contact Management & Address Book Integration
- **Domain**: `contact`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Connects to CardDAV servers, imports VCard contacts, and provides contact lookup for email/calendar autocomplete.
### Evidence summary
- `routes/contacts/contacts_routes.py``@router.get('/list')` — Returns contact list filtered by search query.
- `src/tools/contacts.py``ContactsTool` — Agent tool for querying user address book contacts.
### Unknowns
- VCard 3.0 vs 4.0 property parsing mismatches.
+85
View File
@@ -0,0 +1,85 @@
# Cookbook
Features in this document are generated from [`../feature-catalog.json`](../feature-catalog.json), the canonical inventory.
## COOKBOOK-001 — Local Model Download & Recipe Lifecycle Management
- **Domain**: `cookbook`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Downloads HuggingFace models, configures execution parameters, and manages local GGUF/MLX model servers.
### Evidence summary
- `routes/cookbook_routes.py``setup_cookbook_routes` — Exposes model downloading and process serving endpoints.
- `static/js/cookbook.js``initCookbook` — UI manager for local model library.
### Unknowns
- Disk space exhaustion during multi-gigabyte GGUF weights downloads.
## COOKBOOK-002 — Hardware Model Fitting ('What Fits?') Analysis Engine
- **Domain**: `cookbook`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Calculates RAM/VRAM requirements, quantized size, and context overhead to determine model compatibility.
### Evidence summary
- `routes/hwfit_routes.py``setup_hwfit_routes` — Calculates hardware model compatibility.
- `services/hwfit/fit.py``calculate_fit` — Performs parameter and memory fit calculations.
### Unknowns
- Inaccurate VRAM estimation for non-standard KV-cache quantization.
## COOKBOOK-003 — HuggingFace & MLX Model Discovery Services
- **Domain**: `cookbook`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Searches HuggingFace Hub and local MLX model repositories for compatible GGUF and MLX weights.
### Evidence summary
- `services/hwfit/hf_discovery.py``search_hf_models` — Queries HuggingFace API for model tags and files.
### Unknowns
- HuggingFace API rate limits when searching without an API token.
## COOKBOOK-004 — Host Docker Access for Inference Container Runtimes
- **Domain**: `cookbook`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: pending — Requires Docker access and supported physical GPU hardware.
### Purpose
Detects and connects to host Docker engine to launch containerized Ollama, vLLM, or SGLang runtimes.
### Evidence summary
- `src/host_docker_access.py``HostDockerAccess` — Interacts with host docker daemon.
### Unknowns
- Permission denied accessing docker socket on non-root setups.
+108
View File
@@ -0,0 +1,108 @@
# Document
Features in this document are generated from [`../feature-catalog.json`](../feature-catalog.json), the canonical inventory.
## DOCUMENT-001 — Document & Canvas Artifact Management
- **Domain**: `document`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Renders dynamic canvas documents, handles live editing, markdown preview, and side-by-side artifact display.
### Evidence summary
- `routes/document_routes.py``setup_document_routes` — Registers document artifact CRUD routes.
- `static/js/document.js``initDocumentView` — Renders interactive canvas document panel.
### Unknowns
- Concurrent edits on the same document artifact.
## DOCUMENT-002 — PDF Form Processing & High-Fidelity Rendering
- **Domain**: `document`
- **Status**: `verified`
- **Evidence Maturity**: `E1`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: pending — Requires optional PyMuPDF (`fitz`) or pypdf runtime dependency.
### Purpose
Extracts form fields from PDF files, fills dynamic values, and generates PDF previews.
### Evidence summary
- `src/pdf_runtime.py``load_pymupdf_for_pdf_viewer` — Loads optional PyMuPDF runtime for PDF viewing.
- `src/pdf_forms.py``extract_form_fields` — Handles PDF form field extraction and filling.
- `tests/test_document_pdf_marker.py``test_marker_removed_without_eating_following_text` — Tests PDF text extraction wrapper stripping without content corruption.
### Unknowns
- Complex XFA PDF forms may not extract cleanly with standard pdf parsers.
## DOCUMENT-003 — Personal Document Indexing & RAG Retrieval
- **Domain**: `document`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Indexes local user documents (PDF, DOCX, TXT) into ChromaDB for semantic vector retrieval.
### Evidence summary
- `routes/personal_routes.py``setup_personal_routes` — Personal document RAG indexing and search API endpoints.
- `src/personal_docs.py``PersonalDocsManager` — Handles file text chunking and vector storage.
### Unknowns
- Slow vector embedding indexing step for massive multi-thousand page documents.
## DOCUMENT-004 — Document Conversion & Text Extraction Engine
- **Domain**: `document`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Converts office formats (.docx, .xlsx, .pptx) and HTML into clean Markdown text representations.
### Evidence summary
- `src/markitdown_runtime.py``convert_to_markdown` — Converts binary office documents into structured Markdown text.
### Unknowns
- Formatting loss when parsing legacy binary doc/xls files.
## DOCUMENT-005 — Document Library UI Navigation
- **Domain**: `document`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Provides dedicated UI view for browsing, filtering, and organizing saved user documents.
### Evidence summary
- `static/js/documentLibrary.js``initDocumentLibrary` — Renders document library navigation grid.
- `app.py``serve_library` — Serves SPA shell for /library route.
### Unknowns
- Large folder trees may cause initial DOM render slowdown.
+87
View File
@@ -0,0 +1,87 @@
# Email
Features in this document are generated from [`../feature-catalog.json`](../feature-catalog.json), the canonical inventory.
## EMAIL-001 — Email Account Setup, IMAP/SMTP Connection & Polling
- **Domain**: `email`
- **Status**: `verified`
- **Evidence Maturity**: `E1`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: pending — Requires a controlled live IMAP account and network access.
### Purpose
Configures IMAP/SMTP email accounts, validates TLS certificates, and polls background inbox updates.
### Evidence summary
- `routes/email_routes.py``setup_email_routes` — Sets up email account management and synchronization routes.
- `routes/email_pollers.py``_start_poller` — Background poller for email inbox synchronization.
- `tests/test_service_health_email.py``test_email_ok_all_connect` — Tests IMAP connection health probing and status reporting.
### Unknowns
- Account lockouts if bad credentials are repeatedly polled.
## EMAIL-002 — Email Searching, Threading & Message Operations
- **Domain**: `email`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Parses email headers, folds signatures, groups messages into threads, and executes full-text email search.
### Evidence summary
- `routes/email_routes.py``@router.get('/search')` — Executes search across cached email headers and text.
- `src/email_thread_parser.py``parse_email_thread` — Builds conversation tree from Message-ID and In-Reply-To headers.
### Unknowns
- Malformed MIME email structures failing HTML sanitization.
## EMAIL-003 — Email Composition, Draft Management & Sending
- **Domain**: `email`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: pending — Requires a controlled live SMTP account and network access.
### Purpose
Creates, saves, and dispatches HTML/plaintext email messages via SMTP.
### Evidence summary
- `routes/email_routes.py``@router.post('/send')` — Sends email message via user SMTP credentials.
### Unknowns
- SMTP connection drop mid-send causing unsent mail state.
## EMAIL-004 — Email MCP Server & Codex Integration Bridge
- **Domain**: `email`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Exposes constrained email reading and draft capabilities to external Codex / MCP agents with scope checks.
### Evidence summary
- `mcp_servers/email_server.py``EmailMcpServer` — MCP server exposing email tools over stdio/SSE.
- `routes/codex_routes.py``setup_codex_routes` — Bridge endpoints for external Codex plugin integration.
### Unknowns
- Unauthorized mail sending if token scopes are improperly scoped.
+149
View File
@@ -0,0 +1,149 @@
# Frontend
Features in this document are generated from [`../feature-catalog.json`](../feature-catalog.json), the canonical inventory.
## FRONTEND-001 — Single Page Application Shell & Client Router
- **Domain**: `frontend`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Main HTML5 SPA shell, DOM lifecycle initializers, tab navigation, and deep-link route handlers.
### Evidence summary
- `static/index.html``index.html` — Main SPA entry point containing modal roots and CSS bundles.
- `app.py``serve_index` — Serves index.html with dynamically generated CSP nonces.
### Unknowns
- Stale browser static cache if asset hashing is omitted during deployment.
## FRONTEND-002 — Dynamic Theme, Color System & Custom Fonts
- **Domain**: `frontend`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Supports dark/light themes, custom CSS variables, color picker controls, and user font uploads.
### Evidence summary
- `static/js/theme.js``applyTheme` — Applies custom HSL theme variables to DOM document root.
- `routes/font_routes.py``setup_font_routes` — Allows uploading and serving custom WOFF2 font files.
### Unknowns
- Flash of unstyled content (FOUC) on slow connections.
## FRONTEND-003 — Window Manager, Tile Layout & Modal Control System
- **Domain**: `frontend`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Manages draggable tool windows, snapped multi-tile viewports, modal dialog Z-ordering, and ESC key stacks.
### Evidence summary
- `static/js/modalManager.js``ModalManager` — Controls modal open/close transitions and focus trapping.
- `static/js/tileManager.js``TileManager` — Handles viewport split-pane grid arrangements.
### Unknowns
- Overlap artifacts when opening many simultaneous tool floating windows.
## FRONTEND-004 — Global Keyboard Shortcuts & Accessibility Controls
- **Domain**: `frontend`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Provides configurable hotkeys (Ctrl+K search, Esc close, Alt+1-9 tabs) and high-contrast accessibility options.
### Evidence summary
- `static/js/keyboard-shortcuts.js``initShortcuts` — Binds global keydown handlers for system shortcuts.
- `static/js/a11y.js``initA11y` — Applies ARIA roles and dyslexic font toggles.
### Unknowns
- Browser keybinding collisions with browser default hotkeys.
## FRONTEND-005 — Markdown, LaTeX & Code Block Streaming Renderer
- **Domain**: `frontend`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Parses incoming SSE markdown streams, renders KaTeX math formulas, syntax-highlighted code, and interactive runners.
### Evidence summary
- `static/js/markdown.js``renderMarkdown` — Converts markdown prose to HTML nodes with syntax highlighting.
- `static/js/streamingSegmenter.js``Segmenter` — Parses un-closed markdown fences during live stream.
### Unknowns
- DOM thrashing if streaming segmenter updates UI too frequently.
## FRONTEND-006 — Interactive Tour & Guided Onboarding System
- **Domain**: `frontend`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Presents interactive step-by-step feature tours and UI tooltip hints for new users.
### Evidence summary
- `static/js/tourHints.js``startTour` — Renders guided feature tour overlays over target UI elements.
### Unknowns
- Tour step misalignment if window is resized mid-tour.
## FRONTEND-007 — Background Effects Prototyping Sandbox
- **Domain**: `frontend`
- **Status**: `dead-code-candidate`
- **Evidence Maturity**: `E1`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Standalone sandbox page for prototyping visual background animations, waves, and whirlpool effects.
### Evidence summary
- `app.py``serve_backgrounds` — Serves visual background sandbox HTML page route.
- `static/wave-variants.html``wave-variants.html` — Interactive background effect prototyping sandbox variant.
### Unknowns
- Route `/backgrounds` in app.py L918 attempts to serve `static/backgrounds.html` which is missing from disk; variant templates `wave-variants.html` and `whirlpool-variants.html` exist.
+170
View File
@@ -0,0 +1,170 @@
# Media
Features in this document are generated from [`../feature-catalog.json`](../feature-catalog.json), the canonical inventory.
## MEDIA-001 — Gallery Image Library & Album Operations
- **Domain**: `media`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Organizes images into custom albums, provides grid browsing, tagging, and album metadata management.
### Evidence summary
- `routes/gallery/gallery_routes.py``@router.get('/api/gallery/library')` — Fetches image library list with tag filters.
- `static/js/gallery.js``initGallery` — Main gallery grid renderer and uploader.
### Unknowns
- Thumbnail generation overhead for high-resolution RAW camera images.
## MEDIA-002 — Image Processing, AI Upscaling & Style Transfer
- **Domain**: `media`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Executes local image enhancement, background removal, face sharpening, and AI upscaling operations.
### Evidence summary
- `routes/gallery/gallery_routes.py``@router.post('/api/gallery/ai-upscale')` — Runs RealESRGAN image upscaling.
- `routes/gallery/gallery_routes.py``@router.post('/api/image/remove-bg')` — Executes background removal pass.
### Unknowns
- High GPU memory allocation when upscaling 4K images.
## MEDIA-003 — Interactive Image Canvas Editor & Persisted Drafts
- **Domain**: `media`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Provides full multi-layer raster canvas editor, brush tools, transforms, masks, and draft project persistence.
### Evidence summary
- `routes/editor_draft_routes.py``setup_editor_draft_routes` — API routes for saving and loading canvas project drafts.
- `static/js/editor/history-panel.js``HistoryManager` — Canvas undo/redo stack manager.
### Unknowns
- Browser memory leak if multi-gigabyte layer undo buffers are kept indefinitely.
## MEDIA-004 — Text-to-Speech (TTS) Synthesis Service
- **Domain**: `media`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Synthesizes spoken audio from text using local Kokoro, EdgeTTS, or OpenAI TTS engines.
### Evidence summary
- `routes/tts_routes.py``@router.post('/synthesize')` — Synthesizes TTS audio clip.
- `services/tts/tts_service.py``TTSService` — Provider abstraction layer for audio speech generation.
### Unknowns
- Audio synthesis latency on CPU-only hardware setups.
## MEDIA-005 — Speech-to-Text (STT) Audio Transcription Service
- **Domain**: `media`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Transcribes user audio recordings into text using faster-whisper or local speech models.
### Evidence summary
- `routes/stt_routes.py``@router.post('/transcribe')` — Accepts multipart audio file and returns transcription text.
- `services/stt/stt_service.py``STTService` — Whisper audio transcription engine wrapper.
### Unknowns
- Missing ffmpeg system dependency prevents audio format decoding.
## MEDIA-006 — Digital Signature Stamp Storage & Placement
- **Domain**: `media`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Stores transparent PNG user signatures and stamps for placement onto PDF forms and documents.
### Evidence summary
- `routes/signature_routes.py``setup_signature_routes` — CRUD endpoints for managing user signature PNG stamps.
### Unknowns
- Cross-site scripting if signature image titles contain unescaped user input.
## MEDIA-007 — Generated Image Artifact Route & MCP Integration
- **Domain**: `media`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Serves generated AI artwork artifacts and integrates with image generation MCP server.
### Evidence summary
- `app.py``serve_generated_image` — Serves generated image artifacts with cache headers.
- `src/generated_images.py``resolve_generated_image_path` — Confines requested image path within artifacts directory.
### Unknowns
- Path traversal vulnerability if filename parameter is un-sanitized.
## MEDIA-008 — Native MLX Image Bridge (macOS Apple Silicon)
- **Domain**: `media`
- **Status**: `experimental`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: pending — Requires Apple Silicon, macOS tooling, and the compiled MLX bridge.
### Purpose
Native Apple Swift bridge for hardware-accelerated diffusion and MLX image colorization on macOS.
### Evidence summary
- `swift/odysseus-mlx-image-bridge/Package.swift``Package` — Swift package manifest for native MLX image bridge.
- `scripts/mlx_image_server.py``main` — Python daemon wrapping native Swift MLX binary.
### Unknowns
- Binary build requires Xcode command line tools build step (`build-macos-app.sh`).
+24
View File
@@ -0,0 +1,24 @@
# Memory
Features in this document are generated from [`../feature-catalog.json`](../feature-catalog.json), the canonical inventory.
## MEMORY-001 — Persistent Long-Term Memory & Vector Indexing
- **Domain**: `memory`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Extracts facts, user preferences, and temporal memories from chat sessions into vector/relational storage.
### Evidence summary
- `routes/memory/memory_routes.py``@router.get('')` — Fetches long-term user memory timeline.
- `services/memory/memory_extractor.py``MemoryExtractor` — LLM-driven fact extraction from conversation transcripts.
### Unknowns
- Conflicting memory facts extracted from contradictory user prompts.
+177
View File
@@ -0,0 +1,177 @@
# Model
Features in this document are generated from [`../feature-catalog.json`](../feature-catalog.json), the canonical inventory.
## MODEL-001 — Multi-Provider LLM Model Discovery & Metadata Management
- **Domain**: `model`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Discovers models from OpenAI, Anthropic, Ollama, vLLM, LMStudio, OpenRouter, and Google AI Studio endpoints.
### Evidence summary
- `routes/model_routes.py``@router.get('/api/models')` — Returns unified list of available models across providers.
- `src/model_discovery.py``ModelDiscovery.discover_all` — Queries connected provider endpoints for available model IDs.
### Unknowns
- Remote endpoint timeouts may slow down full discovery refresh.
## MODEL-002 — Model Capability & Context Limits Detection
- **Domain**: `model`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Detects vision, tool calling, reasoning, and context window limits for connected model endpoints.
### Evidence summary
- `src/model_capabilities.py``get_model_capabilities` — Maps model names to vision and tool support flags.
- `src/endpoint_resolver.py``resolve_endpoint_headers` — Resolves auth headers and target URLs for model endpoints.
### Unknowns
- Incorrect context limit metadata for unlisted custom fine-tunes.
## MODEL-003 — LLM Core Provider Communication & Fallback Routing
- **Domain**: `model`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: pending — Provider dispatch, header injection and fallback advancement are only observable against a reachable LLM provider endpoint; not exercised in this documentation pass.
### Purpose
Manages HTTP request dispatching, authorization header injection, and fallback provider routing for LLM calls.
### Evidence summary
- `src/llm_core.py``llm_call_async` — Non-streaming provider request dispatcher: resolves the endpoint, injects authorization headers and executes the HTTP call.
- `src/llm_core.py``llm_call_async_with_fallback` — Ordered fallback wrapper that retries llm_call_async across the configured candidate endpoints.
- `src/llm_core.py``stream_llm_with_fallback` — Ordered fallback wrapper for the streaming path; advances to the next candidate when a provider yields an empty completion.
### Unknowns
- Unexpected API changes in upstream third-party model providers.
## MODEL-004 — Model Selection & Display Ordering Preferences
- **Domain**: `model`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Allows pinning, sorting, and hiding specific models in the UI selection dropdown.
### Evidence summary
- `routes/model_routes.py``@router.post('/order')` — Saves custom model display order preference.
### Unknowns
- Stale model IDs in custom order lists after model endpoints are removed.
## MODEL-005 — Side-by-Side Model Comparison (A/B Testing)
- **Domain**: `model`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Enables dual-model side-by-side response evaluation, arena scoring, and latency comparison.
### Evidence summary
- `routes/compare/compare_routes.py``@router.post('/start')` — Starts a parallel dual-model comparison stream.
- `static/js/compare/index.js``initCompareView` — Renders side-by-side model chat panes.
### Unknowns
- High memory and network usage when streaming two model responses simultaneously.
## MODEL-006 — GitHub Copilot Device Flow Authentication
- **Domain**: `model`
- **Status**: `verified`
- **Evidence Maturity**: `E2`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: pending — Requires an interactive GitHub Copilot OAuth device-flow account.
### Purpose
Authenticates with GitHub Copilot via OAuth device flow to use Copilot models directly.
### Evidence summary
- `routes/copilot_routes.py``setup_copilot_routes` — Builds the Copilot device-flow router at prefix /api/copilot, wiring _start_device_flow and _poll_device_flow.
- `routes/device_flow.py``create_device_flow_router` — Shared factory registering POST /device/start and POST /device/poll under the caller-supplied prefix.
- `src/copilot.py``request_device_code` — Issues the GitHub device-code request that begins the Copilot OAuth device flow.
- `src/copilot.py``poll_access_token` — Polls GitHub for the access token once the user has authorized the device code.
- `tests/test_provider_device_flow_js.py``test_copilot_success_uses_complete_verification_uri` — Inspected unit test asserting the Copilot device-flow runner surfaces the complete verification URI returned by the backend.
### Unknowns
- Token expiration requires manual device re-authentication.
## MODEL-007 — ChatGPT Subscription Device Flow Authentication
- **Domain**: `model`
- **Status**: `verified`
- **Evidence Maturity**: `E2`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: pending — Requires an interactive ChatGPT subscription OAuth flow.
### Purpose
Authenticates with ChatGPT Pro/Plus subscription tokens via device login flow.
### Evidence summary
- `routes/chatgpt_subscription_routes.py``setup_chatgpt_subscription_routes` — Builds the ChatGPT subscription device-flow router at prefix /api/chatgpt-subscription.
- `routes/device_flow.py``create_device_flow_router` — Shared factory registering POST /device/start and POST /device/poll under the caller-supplied prefix.
- `src/chatgpt_subscription.py``request_device_code` — Issues the ChatGPT device-authorization request that begins the subscription OAuth device flow.
- `src/chatgpt_subscription.py``poll_device_auth` — Polls the ChatGPT device-authorization endpoint for completion using the stored device_auth_id and user_code.
- `tests/test_provider_device_flow_js.py``test_chatgpt_success_uses_plain_verification_uri` — Inspected unit test asserting the ChatGPT device-flow runner uses the plain verification URI rather than the Copilot complete-URI form.
### Unknowns
- Changes in OpenAI auth endpoint security challenges.
## MODEL-008 — Embedding Model Lane & Vector Provider Setup
- **Domain**: `model`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Configures local sentence-transformers, FastEmbed, or remote OpenAI embedding model lanes.
### Evidence summary
- `routes/embedding_routes.py``setup_embedding_routes` — Provides embedding provider configuration endpoints.
- `src/embeddings.py``EmbeddingManager` — Generates dense vector embeddings for RAG and memory.
### Unknowns
- First-time download of heavy PyTorch model weights on CPU-only machines.
+24
View File
@@ -0,0 +1,24 @@
# Note
Features in this document are generated from [`../feature-catalog.json`](../feature-catalog.json), the canonical inventory.
## NOTE-001 — Interactive Notes & Checklist Management
- **Domain**: `note`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Provides Google Keep-style notes, rich markdown text, checklist items, pinning, color tags, and reminders.
### Evidence summary
- `routes/note/note_routes.py``@router.get('')` — Lists all user notes with pin and archive states.
- `static/js/notes.js``initNotesView` — Main interactive notes grid and modal manager.
### Unknowns
- Concurrent edits on note item checkboxes.
+213
View File
@@ -0,0 +1,213 @@
# Platform
Features in this document are generated from [`../feature-catalog.json`](../feature-catalog.json), the canonical inventory.
## PLATFORM-001 — Application Initialization & Lifespan Management
- **Domain**: `platform`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Orchestrates server startup, database table migration, background daemon initialization, and clean shutdown.
### Evidence summary
- `app.py``_lifespan` — FastAPI lifespan context manager executing startup tasks.
- `src/app_initializer.py``initialize_app` — Initializes app directories, DB schemas, and logging.
### Unknowns
- Un-handled exceptions during startup halt application launch.
## PLATFORM-002 — System Health, Readiness & Version Monitoring APIs
- **Domain**: `platform`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Exposes Liveness (/api/health), Readiness (/api/ready), App Version (/api/version), and Client Perf APIs.
### Evidence summary
- `app.py``readiness_check` — Performs system component integrity check.
- `src/readiness.py``check_readiness` — Checks database, storage, and key paths for read/write access.
### Unknowns
- Readiness check delays if verifying connectivity to offline remote endpoints.
## PLATFORM-003 — Database Schema, Migrations & SQLite Persistence
- **Domain**: `platform`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Defines core relational tables (users, tokens, tasks, sessions) and executes automated SQLite schema upgrades.
### Evidence summary
- `core/database.py``init_db` — Creates ORM tables and establishes connection pool.
- `scripts/update_database.py``run_migrations` — Applies missing schema columns and indices.
### Unknowns
- SQLite file lock contention under high concurrent write loads.
## PLATFORM-004 — User Data Export & Import Backup Infrastructure
- **Domain**: `platform`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Exports complete user workspace state (sessions, memory, skills, notes, presets) into a zip archive.
### Evidence summary
- `routes/backup_routes.py``setup_backup_routes` — Handles workspace data export and import upload unpack.
- `docs/backup-restore.md``Documentation` — Backup and restore operational documentation.
### Unknowns
- Corrupt archive files causing partial data restore.
## PLATFORM-005 — File Cleanup & Storage Maintenance Engine
- **Domain**: `platform`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Scans data directories for orphaned files, old uploads, temporary vision images, and frees disk space.
### Evidence summary
- `routes/cleanup/cleanup_routes.py``@router.get('/preview')` — Previews reclaimable disk space across storage directories.
- `src/cleanup_service.py``CleanupService` — Executes filesystem purge of orphaned asset files.
### Unknowns
- Deletes files uploaded in active sessions if retention window is set too short.
## PLATFORM-006 — System Health & RAG Diagnostic Suite
- **Domain**: `platform`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Executes real-time integrity diagnostics across ChromaDB, SearXNG, local models, and network interfaces.
### Evidence summary
- `routes/diagnostics_routes.py``setup_diagnostics_routes` — Runs subsystem health check suite.
- `src/service_health.py``collect_health_status` — Inspects vector database, email, search, and local provider status.
### Unknowns
- Diagnostic timeout if external search provider is unreachable.
## PLATFORM-007 — Desktop CLI Utilities & Shell Integration Tools
- **Domain**: `platform`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Provides command-line interface tools (`odysseus`, `odysseus-mcp`, `odysseus-mail`) for terminal usage.
### Evidence summary
- `scripts/_lib/cli.py``main` — Shared CLI framework for terminal helper commands.
- `scripts/odysseus``odysseus` — Main terminal launcher script.
### Unknowns
- Outdated CLI scripts if backend API schemas change.
## PLATFORM-008 — Desktop Companion App Integration
- **Domain**: `platform`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Provides API routes and pairing mechanisms for the native macOS/desktop menu bar companion app.
### Evidence summary
- `companion/routes.py``setup_companion_routes` — Endpoints for pairing and status sync with desktop companion.
- `companion/pairing.py``PairingManager` — Generates and validates companion pairing codes.
### Unknowns
- Pairing code expiration timing window.
## PLATFORM-009 — Docker Containerization & GPU Hardware Manifests
- **Domain**: `platform`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: pending — Requires Docker GPU pass-through and compatible host drivers.
### Purpose
Provides multi-stage Dockerfile and Docker Compose manifests for CPU, NVIDIA CUDA, and AMD ROCm GPUs.
### Evidence summary
- `Dockerfile``multi-stage-build` — Multi-stage container build environment.
- `docker-compose.gpu-nvidia.yml``nvidia-gpu-manifest` — NVIDIA GPU pass-through container specification.
- `scripts/check-docker-gpu.sh``check-docker-gpu` — Automated diagnostic test script for host NVIDIA GPU passthrough.
### Unknowns
- Driver version incompatibility with host NVIDIA/AMD kernel drivers.
## PLATFORM-010 — Legacy FAISS Vector Index Migration Script
- **Domain**: `platform`
- **Status**: `legacy`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Legacy utility script to migrate older FAISS vector indices into ChromaDB.
### Evidence summary
- `scripts/migrate_faiss_to_chroma.py``migrate_faiss` — Reads FAISS vector index files and writes to ChromaDB collection.
### Unknowns
- Superseded by native ChromaDB vector index pipeline.
+86
View File
@@ -0,0 +1,86 @@
# Research
Features in this document are generated from [`../feature-catalog.json`](../feature-catalog.json), the canonical inventory.
## RESEARCH-001 — Deep Research Execution Engine & SSE Progress Streaming
- **Domain**: `research`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Executes multi-step recursive deep research tasks, web page scraping, synthesis, and streams live progress.
### Evidence summary
- `routes/research/research_routes.py``@router.post('/api/research/start')` — Initiates deep research job.
- `src/deep_research.py``DeepResearchEngine` — Recursive search and summary crawler.
### Unknowns
- High memory consumption when parsing multi-megabyte HTML target pages.
## RESEARCH-002 — Research Library, Detail View & Image Controls
- **Domain**: `research`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Stores completed research reports, generated diagrams, reference links, and manages image visibility.
### Evidence summary
- `routes/research/research_routes.py``@router.get('/api/research/library')` — Returns all saved research reports.
### Unknowns
- Orphaned report files if storage directory is modified out-of-band.
## RESEARCH-003 — Web Search Engine Integration (SearXNG & Multi-Provider)
- **Domain**: `research`
- **Status**: `verified`
- **Evidence Maturity**: `E1`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: pending — Requires an active SearXNG instance or external search API provider.
### Purpose
Queries SearXNG, DuckDuckGo, or Google Search instances to retrieve web search snippets.
### Evidence summary
- `routes/search_routes.py``setup_search_routes` — Defines /api/search, /api/search/config, and /api/search/query endpoints.
- `src/search/core.py``SearchEngine` — Compatibility module aliasing services.search.core.
- `tests/test_search_ranking.py``test_news_queries_prefer_news_sources_over_sports_and_social_results` — Tests search result domain ranking and scoring.
### Unknowns
- Search provider IP throttling or rate-limiting.
## RESEARCH-004 — Research Result Peeking & Topic Spinoff Generation
- **Domain**: `research`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Extracts preliminary research snippets and spawns child research sessions focused on specific sub-topics.
### Evidence summary
- `routes/research/research_routes.py``@router.post('/api/research/spinoff/{session_id}')` — Spawns child research session for specific query.
### Unknowns
- Deep recursion tree depth when spawning multiple nested spinoffs.
+154
View File
@@ -0,0 +1,154 @@
# Security
Features in this document are generated from [`../feature-catalog.json`](../feature-catalog.json), the canonical inventory.
## SECURITY-001 — Authentication, Session Cookies & User Management
- **Domain**: `security`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Handles bcrypt password hashing, session cookie issuance, authentication enforcement, and user administration.
### Evidence summary
- `routes/auth_routes.py``@router.post('/login')` — Authenticates credentials and sets session cookie.
- `core/auth.py``AuthManager` — Handles user creation, password verification, and session tokens.
### Unknowns
- Cookie session hijack if deployed over unencrypted HTTP without HTTPS cookie flags.
## SECURITY-002 — System Vault Encrypted Secret Storage
- **Domain**: `security`
- **Status**: `verified`
- **Evidence Maturity**: `E1`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: pending — Requires installed Bitwarden CLI (`bw`) executable.
### Purpose
Encrypts API keys, passphrases, and third-party secrets on disk using AES-GCM / PBKDF2 key derivation.
### Evidence summary
- `routes/vault_routes.py``setup_vault_routes` — Admin routes for vault configuration, login, unlock, lock, and logout.
- `src/secret_storage.py``SecretStorage` — Fernet symmetric key DB secret encryption.
- `tests/test_vault_password_not_in_argv.py``test_bw_password_not_in_argv` — Verifies master password is fed via stdin and never appears in process argv.
### Unknowns
- Loss of vault master passphrase renders all encrypted secrets permanently unrecoverable.
## SECURITY-003 — API Token Management & Scope Access Control
- **Domain**: `security`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Generates scoped API bearer tokens (read/write/admin) for external tool and script authentication.
### Evidence summary
- `routes/api_token_routes.py``setup_api_token_routes` — Exposes API token creation, scope assignment, and revocation.
- `core/database.py``ApiToken` — SQLAlchemy ORM schema for API tokens and permissions.
### Unknowns
- Leaked API bearer tokens with excessive permission scopes.
## SECURITY-004 — Prompt Security & Injection Defense Engine
- **Domain**: `security`
- **Status**: `verified`
- **Evidence Maturity**: `E1`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Scans system prompts and external inputs for prompt injection attempts, jailbreaks, and sensitive data leaks.
### Evidence summary
- `src/prompt_security.py``untrusted_context_message` — Wraps untrusted context with guard delimiters and sets metadata.trusted = False.
- `src/tool_security.py``NON_ADMIN_BLOCKED_TOOLS` — Enforces tool execution safety for non-admin user roles.
- `tests/test_skill_index_prompt_injection.py``test_skill_index` — Verifies skill index descriptions cannot leak into trusted system prompts.
- `tests/test_tool_output_prompt_injection.py``test_tool_output` — Tool output injection guards.
### Unknowns
- False positives blocking legitimate complex coding or security prompts.
## SECURITY-005 — URL & Path Confinement Security Guards
- **Domain**: `security`
- **Status**: `verified`
- **Evidence Maturity**: `E1`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Prevents SSRF attacks and path traversal by validating target IP addresses and resolving symlinks.
### Evidence summary
- `src/url_safety.py``check_outbound_url` — Rejects non-HTTP(S) schemes, link-local, cloud metadata SSRF addresses.
- `src/url_security.py``validate_public_http_url` — Validates public-facing endpoints.
- `tests/test_url_safety.py``test_url_safety` — Scheme validation, cloud metadata SSRF rejection, IP classification.
- `tests/test_tool_path_confinement.py``test_path_confinement` — Path traversal checks.
- `tests/test_workspace_confine.py``test_workspace_confine` — Workspace confinement checks.
### Unknowns
- DNS rebinding attacks if IP address is re-resolved post-validation.
## SECURITY-006 — HTTP Security Headers Middleware
- **Domain**: `security`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Injects standard OWASP HTTP security headers (CSP, HSTS, X-Content-Type-Options, X-Frame-Options).
### Evidence summary
- `core/middleware.py``SecurityHeadersMiddleware` — Sets strict security headers and CSP nonces on HTTP responses.
### Unknowns
- Strict Content Security Policy (CSP) blocking third-party embedded web resources.
## SECURITY-007 — Admin System Data Wipe ('Danger Zone')
- **Domain**: `security`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Provides administrative reset operations to wipe sessions, cache, uploaded files, or factory reset state.
### Evidence summary
- `routes/admin_wipe/admin_wipe_routes.py``@router.delete('/wipe/{kind}')` — Executes systemic data wipe based on requested scope.
### Unknowns
- Accidental catastrophic data loss if triggered without user confirmation.
+24
View File
@@ -0,0 +1,24 @@
# Skill
Features in this document are generated from [`../feature-catalog.json`](../feature-catalog.json), the canonical inventory.
## SKILL-001 — Dynamic Skill Management & Code Execution Engine
- **Domain**: `skill`
- **Status**: `verified`
- **Evidence Maturity**: `E0`
- **Commit Verified**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Runtime Validation**: not-required — No separate environment-dependent runtime validation was identified during this documentation pass.
### Purpose
Allows users to create, import, edit, test, and execute custom Python/Markdown skills dynamically.
### Evidence summary
- `routes/skills_routes.py``setup_skills_routes` — Exposes CRUD and remote import routes for user skills.
- `services/memory/skills.py``SkillsManager` — Handles skill storage, parsing, and execution.
### Unknowns
- Arbitrary code execution risks if skill import URL is untrusted.
File diff suppressed because it is too large Load Diff
+85
View File
@@ -0,0 +1,85 @@
# Feature Catalog
This is a human-readable index derived from [`feature-catalog.json`](feature-catalog.json). The JSON file is canonical.
| ID | Feature | Domain | Status | Evidence | Runtime |
|---|---|---|---|---|---|
| `AGENT-001` | Autonomous Agent Loop & Tool Execution Engine | `agent` | `verified` | `E0` | not required |
| `AGENT-002` | Scheduled Tasks & Event Bus Dispatcher | `agent` | `verified` | `E0` | not required |
| `AGENT-003` | Webhook Event Subscriptions & Trigger Processing | `agent` | `verified` | `E0` | not required |
| `AGENT-004` | Assistant Settings, Task Check-Ins & Background Job Monitor | `agent` | `partial` | `E1` | not required |
| `AGENT-005` | Model Context Protocol (MCP) Server Integration | `agent` | `verified` | `E0` | not required |
| `AGENT-006` | AI Interaction Tools & Pipeline Orchestration | `agent` | `verified` | `E0` | not required |
| `AGENT-007` | Subprocess & Background Job Execution Tools | `agent` | `verified` | `E0` | not required |
| `CALENDAR-001` | CalDAV Calendar Synchronization & Account Setup | `calendar` | `verified` | `E0` | pending |
| `CALENDAR-002` | Calendar Event Operations & iCalendar Parsing | `calendar` | `verified` | `E0` | not required |
| `CHAT-001` | Core Chat Streaming & SSE Message Generation | `chat` | `verified` | `E2` | pending |
| `CHAT-002` | Session Management & Conversation State | `chat` | `verified` | `E0` | not required |
| `CHAT-003` | Chat History & Message Editing/Truncation | `chat` | `verified` | `E0` | not required |
| `CHAT-004` | File & Multimodal Attachment Handling | `chat` | `verified` | `E0` | not required |
| `CHAT-005` | Chat Message Search | `chat` | `verified` | `E0` | not required |
| `CHAT-006` | System Prompts & Preset Management | `chat` | `verified` | `E0` | not required |
| `CHAT-007` | Emoji Rendering & Twemoji SVG Proxy | `chat` | `verified` | `E0` | not required |
| `CHAT-008` | Input History Recall (Arrow Up) | `chat` | `verified` | `E0` | not required |
| `CHAT-009` | Context Window Compaction & Truncation | `chat` | `verified` | `E0` | not required |
| `CONTACT-001` | CardDAV Contact Management & Address Book Integration | `contact` | `verified` | `E0` | not required |
| `COOKBOOK-001` | Local Model Download & Recipe Lifecycle Management | `cookbook` | `verified` | `E0` | not required |
| `COOKBOOK-002` | Hardware Model Fitting ('What Fits?') Analysis Engine | `cookbook` | `verified` | `E0` | not required |
| `COOKBOOK-003` | HuggingFace & MLX Model Discovery Services | `cookbook` | `verified` | `E0` | not required |
| `COOKBOOK-004` | Host Docker Access for Inference Container Runtimes | `cookbook` | `verified` | `E0` | pending |
| `DOCUMENT-001` | Document & Canvas Artifact Management | `document` | `verified` | `E0` | not required |
| `DOCUMENT-002` | PDF Form Processing & High-Fidelity Rendering | `document` | `verified` | `E1` | pending |
| `DOCUMENT-003` | Personal Document Indexing & RAG Retrieval | `document` | `verified` | `E0` | not required |
| `DOCUMENT-004` | Document Conversion & Text Extraction Engine | `document` | `verified` | `E0` | not required |
| `DOCUMENT-005` | Document Library UI Navigation | `document` | `verified` | `E0` | not required |
| `EMAIL-001` | Email Account Setup, IMAP/SMTP Connection & Polling | `email` | `verified` | `E1` | pending |
| `EMAIL-002` | Email Searching, Threading & Message Operations | `email` | `verified` | `E0` | not required |
| `EMAIL-003` | Email Composition, Draft Management & Sending | `email` | `verified` | `E0` | pending |
| `EMAIL-004` | Email MCP Server & Codex Integration Bridge | `email` | `verified` | `E0` | not required |
| `FRONTEND-001` | Single Page Application Shell & Client Router | `frontend` | `verified` | `E0` | not required |
| `FRONTEND-002` | Dynamic Theme, Color System & Custom Fonts | `frontend` | `verified` | `E0` | not required |
| `FRONTEND-003` | Window Manager, Tile Layout & Modal Control System | `frontend` | `verified` | `E0` | not required |
| `FRONTEND-004` | Global Keyboard Shortcuts & Accessibility Controls | `frontend` | `verified` | `E0` | not required |
| `FRONTEND-005` | Markdown, LaTeX & Code Block Streaming Renderer | `frontend` | `verified` | `E0` | not required |
| `FRONTEND-006` | Interactive Tour & Guided Onboarding System | `frontend` | `verified` | `E0` | not required |
| `FRONTEND-007` | Background Effects Prototyping Sandbox | `frontend` | `dead-code-candidate` | `E1` | not required |
| `MEDIA-001` | Gallery Image Library & Album Operations | `media` | `verified` | `E0` | not required |
| `MEDIA-002` | Image Processing, AI Upscaling & Style Transfer | `media` | `verified` | `E0` | not required |
| `MEDIA-003` | Interactive Image Canvas Editor & Persisted Drafts | `media` | `verified` | `E0` | not required |
| `MEDIA-004` | Text-to-Speech (TTS) Synthesis Service | `media` | `verified` | `E0` | not required |
| `MEDIA-005` | Speech-to-Text (STT) Audio Transcription Service | `media` | `verified` | `E0` | not required |
| `MEDIA-006` | Digital Signature Stamp Storage & Placement | `media` | `verified` | `E0` | not required |
| `MEDIA-007` | Generated Image Artifact Route & MCP Integration | `media` | `verified` | `E0` | not required |
| `MEDIA-008` | Native MLX Image Bridge (macOS Apple Silicon) | `media` | `experimental` | `E0` | pending |
| `MEMORY-001` | Persistent Long-Term Memory & Vector Indexing | `memory` | `verified` | `E0` | not required |
| `MODEL-001` | Multi-Provider LLM Model Discovery & Metadata Management | `model` | `verified` | `E0` | not required |
| `MODEL-002` | Model Capability & Context Limits Detection | `model` | `verified` | `E0` | not required |
| `MODEL-003` | LLM Core Provider Communication & Fallback Routing | `model` | `verified` | `E0` | pending |
| `MODEL-004` | Model Selection & Display Ordering Preferences | `model` | `verified` | `E0` | not required |
| `MODEL-005` | Side-by-Side Model Comparison (A/B Testing) | `model` | `verified` | `E0` | not required |
| `MODEL-006` | GitHub Copilot Device Flow Authentication | `model` | `verified` | `E2` | pending |
| `MODEL-007` | ChatGPT Subscription Device Flow Authentication | `model` | `verified` | `E2` | pending |
| `MODEL-008` | Embedding Model Lane & Vector Provider Setup | `model` | `verified` | `E0` | not required |
| `NOTE-001` | Interactive Notes & Checklist Management | `note` | `verified` | `E0` | not required |
| `PLATFORM-001` | Application Initialization & Lifespan Management | `platform` | `verified` | `E0` | not required |
| `PLATFORM-002` | System Health, Readiness & Version Monitoring APIs | `platform` | `verified` | `E0` | not required |
| `PLATFORM-003` | Database Schema, Migrations & SQLite Persistence | `platform` | `verified` | `E0` | not required |
| `PLATFORM-004` | User Data Export & Import Backup Infrastructure | `platform` | `verified` | `E0` | not required |
| `PLATFORM-005` | File Cleanup & Storage Maintenance Engine | `platform` | `verified` | `E0` | not required |
| `PLATFORM-006` | System Health & RAG Diagnostic Suite | `platform` | `verified` | `E0` | not required |
| `PLATFORM-007` | Desktop CLI Utilities & Shell Integration Tools | `platform` | `verified` | `E0` | not required |
| `PLATFORM-008` | Desktop Companion App Integration | `platform` | `verified` | `E0` | not required |
| `PLATFORM-009` | Docker Containerization & GPU Hardware Manifests | `platform` | `verified` | `E0` | pending |
| `PLATFORM-010` | Legacy FAISS Vector Index Migration Script | `platform` | `legacy` | `E0` | not required |
| `RESEARCH-001` | Deep Research Execution Engine & SSE Progress Streaming | `research` | `verified` | `E0` | not required |
| `RESEARCH-002` | Research Library, Detail View & Image Controls | `research` | `verified` | `E0` | not required |
| `RESEARCH-003` | Web Search Engine Integration (SearXNG & Multi-Provider) | `research` | `verified` | `E1` | pending |
| `RESEARCH-004` | Research Result Peeking & Topic Spinoff Generation | `research` | `verified` | `E0` | not required |
| `SECURITY-001` | Authentication, Session Cookies & User Management | `security` | `verified` | `E0` | not required |
| `SECURITY-002` | System Vault Encrypted Secret Storage | `security` | `verified` | `E1` | pending |
| `SECURITY-003` | API Token Management & Scope Access Control | `security` | `verified` | `E0` | not required |
| `SECURITY-004` | Prompt Security & Injection Defense Engine | `security` | `verified` | `E1` | not required |
| `SECURITY-005` | URL & Path Confinement Security Guards | `security` | `verified` | `E1` | not required |
| `SECURITY-006` | HTTP Security Headers Middleware | `security` | `verified` | `E0` | not required |
| `SECURITY-007` | Admin System Data Wipe ('Danger Zone') | `security` | `verified` | `E0` | not required |
| `SKILL-001` | Dynamic Skill Management & Code Execution Engine | `skill` | `verified` | `E0` | not required |
@@ -0,0 +1,28 @@
# Source Provenance & Audit Baseline
## Target Repository & Snapshot
- **Repository**: `odysseus-dev/odysseus`
- **Branch**: `discovery`
- **Pinned Commit SHA**: `d8a2059df8e53bc7275c45339849d14c8651e73c`
- **Snapshot Date**: `2026-07-23T14:49:02Z`
## Discovery Package Organization
The public discovery documentation package under `docs/discovery/` is structured as follows:
- `feature-catalog.json`: Canonical machine-readable JSON catalog containing 79 feature records.
- `feature-catalog.md`: Human-readable summary derived from `feature-catalog.json`.
- `BASELINE-STATUS.md`: Publication status, evidence-validation snapshot, and durable maintainer guidance.
- `audit-method.md`: Audit rules, scope, and evidence maturity definitions (E0 to E4).
- `domains/`: 16 functional domain markdown files detailing feature implementations.
- `references/`: Audit provenance and repository snapshot metadata.
- `tools/`: Structural, consistency, and evidence validators with focused evidence-validator tests.
## Exclusion Principles
This public documentation package explicitly excludes:
- Internal planning artifacts or private meeting notes.
- Machine-specific filesystem paths or user environments.
- API keys, credentials, or private service endpoints.
- Application code or automated test mutations.
@@ -0,0 +1,224 @@
#!/usr/bin/env python3
"""Focused negative tests for validate_discovery_evidence.py."""
from __future__ import annotations
import importlib.util
import shutil
import sys
import tempfile
import unittest
from pathlib import Path
MODULE_PATH = Path(__file__).with_name("validate_discovery_evidence.py")
SPEC = importlib.util.spec_from_file_location("validate_discovery_evidence", MODULE_PATH)
assert SPEC and SPEC.loader
validator = importlib.util.module_from_spec(SPEC)
sys.modules[SPEC.name] = validator
SPEC.loader.exec_module(validator)
class FakeJavascriptParser:
supported = True
reason = "test parser"
def __init__(self, symbols: list[validator.Located] | None = None) -> None:
self.symbols = symbols or []
def parse(self, path: Path) -> list[validator.Located]:
return self.symbols
class UnsupportedJavascriptParser:
supported = False
reason = "no repository-local parser"
class EvidenceNegativeTests(unittest.TestCase):
def setUp(self) -> None:
self.temp = tempfile.TemporaryDirectory()
self.root = Path(self.temp.name)
(self.root / "sample.py").write_text(
"from fastapi import APIRouter\n"
"router = APIRouter(prefix='/api')\n"
"\n"
"class ChatHandler:\n"
" def preprocess_message(self):\n"
" return True\n"
"\n"
"@router.post('/chat')\n"
"def chat_stream():\n"
" return True\n",
encoding="utf-8",
)
(self.root / "sample.js").write_text(
"export const present = () => true;\n", encoding="utf-8"
)
(self.root / "sample.sh").write_text(
"#!/usr/bin/env bash\nreal_function() {\n return 0\n}\n",
encoding="utf-8",
)
self.backup = self.root / "copied-backups"
self.backup.mkdir()
for path in self.root.glob("sample.*"):
shutil.copy2(path, self.backup / path.name)
def tearDown(self) -> None:
for backup in self.backup.iterdir():
target = self.root / backup.name
shutil.copy2(backup, target)
self.assertEqual(target.read_bytes(), backup.read_bytes())
self.temp.cleanup()
def validate(
self,
evidence: dict[str, str],
javascript_parser: object | None = None,
) -> validator.Validation:
return validator.validate_evidence(
self.root,
"TEST-001",
0,
evidence,
javascript_parser or FakeJavascriptParser(),
)
def test_missing_python_symbol(self) -> None:
result = self.validate(
{
"path": "sample.py",
"kind": "python-function",
"locator": "fabricated",
"line_range": "L1-L1",
"explanation": "negative fixture",
}
)
self.assertEqual(result.result, "invalid-locator")
def test_incorrect_qualified_method(self) -> None:
result = self.validate(
{
"path": "sample.py",
"kind": "python-method",
"locator": "WrongHandler.preprocess_message",
"line_range": "L5-L6",
"explanation": "negative fixture",
}
)
self.assertEqual(result.result, "invalid-locator")
def test_symbol_outside_cited_range(self) -> None:
result = self.validate(
{
"path": "sample.py",
"kind": "python-method",
"locator": "ChatHandler.preprocess_message",
"line_range": "L1-L2",
"explanation": "negative fixture",
}
)
self.assertEqual(result.result, "locator-outside-range")
def test_fabricated_test_function(self) -> None:
result = self.validate(
{
"path": "sample.py",
"kind": "test-function",
"locator": "test_fabricated",
"line_range": "L1-L2",
"explanation": "negative fixture",
}
)
self.assertEqual(result.result, "invalid-locator")
def test_nonexistent_javascript_symbol_with_parser(self) -> None:
result = self.validate(
{
"path": "sample.js",
"kind": "javascript-function",
"locator": "missing",
"line_range": "L1-L1",
"explanation": "negative fixture",
},
FakeJavascriptParser(
[validator.Located("present", "javascript-export", 1, 1)]
),
)
self.assertEqual(result.result, "invalid-locator")
def test_unsupported_javascript_parser(self) -> None:
result = self.validate(
{
"path": "sample.js",
"kind": "javascript-function",
"locator": "present",
"line_range": "L1-L1",
"explanation": "negative fixture",
},
UnsupportedJavascriptParser(),
)
self.assertEqual(result.result, "unsupported")
def test_route_path_mismatch(self) -> None:
result = self.validate(
{
"path": "sample.py",
"kind": "python-route",
"locator": "POST /api/wrong -> chat_stream",
"line_range": "L9-L10",
"explanation": "negative fixture",
}
)
self.assertEqual(result.result, "invalid-locator")
self.assertIn("path", result.problem or "")
def test_http_method_mismatch(self) -> None:
result = self.validate(
{
"path": "sample.py",
"kind": "python-route",
"locator": "GET /api/chat -> chat_stream",
"line_range": "L9-L10",
"explanation": "negative fixture",
}
)
self.assertEqual(result.result, "invalid-locator")
self.assertIn("method", result.problem or "")
def test_shell_function_mismatch(self) -> None:
result = self.validate(
{
"path": "sample.sh",
"kind": "shell-function",
"locator": "fabricated",
"line_range": "L1-L4",
"explanation": "negative fixture",
}
)
self.assertEqual(result.result, "invalid-locator")
def test_invalid_file_level_evidence(self) -> None:
result = self.validate(
{
"path": "missing.file",
"kind": "file",
"explanation": "negative fixture",
}
)
self.assertEqual(result.result, "invalid-path")
def test_file_level_evidence_rejects_fake_symbol(self) -> None:
result = self.validate(
{
"path": "sample.sh",
"kind": "file",
"locator": "whole-script",
"explanation": "negative fixture",
}
)
self.assertEqual(result.result, "invalid-locator")
if __name__ == "__main__":
unittest.main()
+592
View File
@@ -0,0 +1,592 @@
#!/usr/bin/env python3
from __future__ import annotations
import json
import re
import sys
from collections import Counter, defaultdict
from pathlib import Path
from typing import Any
ROOT = Path(__file__).resolve().parents[1]
CATALOG_PATH = ROOT / "feature-catalog.json"
CATALOG_MD_PATH = ROOT / "feature-catalog.md"
DOMAINS_DIR = ROOT / "domains"
REVIEWS_DIR = ROOT / "reviews"
EXPECTED_COMMIT = "d8a2059df8e53bc7275c45339849d14c8651e73c"
EXPECTED_FEATURES = 79
EXPECTED_DOMAINS = 16
VALID_STATUSES = {
"verified",
"partial",
"disabled",
"experimental",
"legacy",
"dead-code-candidate",
"unverified",
}
VALID_MATURITY = {"E0", "E1", "E2", "E3", "E4"}
VALID_RUNTIME = {
"not-required",
"pending",
"blocked",
"completed",
}
REQUIRED_FIELDS = {
"id",
"domain",
"name",
"purpose",
"status",
"evidence_maturity",
"verified_at_commit",
"evidence",
"runtime_validation",
}
FEATURE_ID_RE = re.compile(r"^[A-Z][A-Z0-9]*-\d{3}$")
DOMAIN_HEADING_RE = re.compile(
r"^##\s+`?([A-Z][A-Z0-9]*-\d{3})`?"
r"\s+(?:—|-)\s+(.+?)\s*$"
)
DOMAIN_FIELD_RE = re.compile(
r"^-\s+\*\*"
r"(Domain|Status|Evidence Maturity|Commit Verified)"
r"\*\*:\s*(.*?)\s*$"
)
DOMAIN_RUNTIME_RE = re.compile(
r"^-\s+\*\*Runtime Validation\*\*:\s*(.*?)\s*$"
)
REVIEW_HEADING_RE = re.compile(
r"^###\s+(?:\d+\.\s+)?"
r"([A-Z][A-Z0-9]*-\d{3})"
r"\s+(?:—|-)\s+.+$"
)
REVIEW_RESULT_RE = re.compile(
r"^-\s+\*\*Resulting Status(?:\s+and|/)\s+Maturity\*\*:"
r"\s*`([^`]+)`\s*/\s*`([^`]+)`\s*$"
)
def clean(value: str) -> str:
value = value.strip()
if (
len(value) >= 2
and value.startswith("`")
and value.endswith("`")
):
return value[1:-1].strip()
return value
def load_catalog(errors: list[str]) -> list[dict[str, Any]]:
try:
data = json.loads(
CATALOG_PATH.read_text(encoding="utf-8")
)
except Exception as exc:
errors.append(f"Unable to read catalog: {exc}")
return []
if isinstance(data, list):
features = data
elif isinstance(data, dict) and isinstance(data.get("features"), list):
features = data["features"]
else:
errors.append(
"Catalog must be an array or contain a features array"
)
return []
if not all(isinstance(feature, dict) for feature in features):
errors.append("Every catalog feature must be an object")
return []
return features
def validate_catalog(
features: list[dict[str, Any]],
errors: list[str],
) -> None:
if len(features) != EXPECTED_FEATURES:
errors.append(
f"Expected {EXPECTED_FEATURES} features, "
f"found {len(features)}"
)
ids = [feature.get("id") for feature in features]
duplicates = sorted(
feature_id
for feature_id, count in Counter(ids).items()
if feature_id and count > 1
)
if duplicates:
errors.append(
"Duplicate feature IDs: " + ", ".join(duplicates)
)
for index, feature in enumerate(features):
feature_id = feature.get("id")
label = (
feature_id
if isinstance(feature_id, str)
else f"<index:{index}>"
)
missing = sorted(
field
for field in REQUIRED_FIELDS
if feature.get(field) in (None, "", [])
)
if missing:
errors.append(
f"{label}: missing fields: {', '.join(missing)}"
)
if (
not isinstance(feature_id, str)
or not FEATURE_ID_RE.fullmatch(feature_id)
):
errors.append(f"{label}: invalid feature ID")
if feature.get("status") not in VALID_STATUSES:
errors.append(
f"{label}: invalid status "
f"{feature.get('status')!r}"
)
if feature.get("evidence_maturity") not in VALID_MATURITY:
errors.append(
f"{label}: invalid maturity "
f"{feature.get('evidence_maturity')!r}"
)
if feature.get("verified_at_commit") != EXPECTED_COMMIT:
errors.append(
f"{label}: incorrect verified_at_commit"
)
runtime = feature.get("runtime_validation")
if not isinstance(runtime, dict):
errors.append(
f"{label}: runtime_validation must be an object"
)
continue
required = runtime.get("required")
runtime_status = runtime.get("status")
reason = runtime.get("reason")
if not isinstance(required, bool):
errors.append(
f"{label}: runtime required must be boolean"
)
if runtime_status not in VALID_RUNTIME:
errors.append(
f"{label}: invalid runtime status "
f"{runtime_status!r}"
)
if not isinstance(reason, str) or not reason.strip():
errors.append(
f"{label}: runtime reason is blank"
)
if required is False and runtime_status != "not-required":
errors.append(
f"{label}: required=false requires not-required"
)
if required is True and runtime_status == "not-required":
errors.append(
f"{label}: required=true cannot be not-required"
)
def parse_domain(
path: Path,
errors: list[str],
) -> dict[str, dict[str, str]]:
records: dict[str, dict[str, str]] = {}
current_id: str | None = None
for line_number, line in enumerate(
path.read_text(encoding="utf-8").splitlines(),
start=1,
):
heading = DOMAIN_HEADING_RE.match(line)
if heading:
current_id = heading.group(1)
if current_id in records:
errors.append(
f"{path.relative_to(ROOT)}:{line_number}: "
f"duplicate heading {current_id}"
)
records[current_id] = {
"Name": heading.group(2).strip(),
}
continue
field = DOMAIN_FIELD_RE.match(line)
if field and current_id:
value = clean(field.group(2))
if not value:
errors.append(
f"{path.relative_to(ROOT)}:{line_number}: "
f"blank {field.group(1)}"
)
records[current_id][field.group(1)] = value
continue
runtime = DOMAIN_RUNTIME_RE.match(line)
if runtime and current_id:
value = clean(runtime.group(1))
runtime_status = re.split(
r"\s+(?:—|-)\s+",
value,
maxsplit=1,
)[0]
records[current_id]["Runtime Validation"] = (
runtime_status.strip("` ")
)
return records
def validate_domains(
features: list[dict[str, Any]],
errors: list[str],
) -> None:
catalog = {
feature["id"]: feature
for feature in features
if feature.get("id")
}
expected_by_domain: defaultdict[str, set[str]] = defaultdict(set)
for feature in features:
expected_by_domain[feature["domain"]].add(feature["id"])
paths = sorted(DOMAINS_DIR.glob("*.md"))
if len(paths) != EXPECTED_DOMAINS:
errors.append(
f"Expected {EXPECTED_DOMAINS} domain files, "
f"found {len(paths)}"
)
all_found: set[str] = set()
for path in paths:
domain = path.stem
records = parse_domain(path, errors)
found = set(records)
expected = expected_by_domain.get(domain, set())
all_found.update(found)
if found != expected:
missing = sorted(expected - found)
extra = sorted(found - expected)
errors.append(
f"{domain}: missing={missing}, unexpected={extra}"
)
for feature_id, record in records.items():
feature = catalog.get(feature_id)
if feature is None:
continue
expected_values = {
"Name": feature["name"],
"Domain": feature["domain"],
"Status": feature["status"],
"Evidence Maturity": feature["evidence_maturity"],
"Commit Verified": feature["verified_at_commit"],
"Runtime Validation": (
feature["runtime_validation"]["status"]
),
}
for field, expected_value in expected_values.items():
actual = record.get(field)
if actual != expected_value:
errors.append(
f"{path.relative_to(ROOT)}: "
f"{feature_id} {field}: "
f"{actual!r} != {expected_value!r}"
)
if all_found != set(catalog):
errors.append(
"Domain Markdown IDs do not match catalog JSON"
)
def validate_catalog_markdown(
features: list[dict[str, Any]],
errors: list[str],
) -> None:
expected = {
feature["id"]: feature
for feature in features
}
found: dict[str, list[str]] = {}
for line in CATALOG_MD_PATH.read_text(
encoding="utf-8"
).splitlines():
if not line.startswith("|"):
continue
cells = [
cell.strip()
for cell in line.strip().strip("|").split("|")
]
if len(cells) < 6:
continue
feature_id = clean(cells[0])
if FEATURE_ID_RE.fullmatch(feature_id):
found[feature_id] = cells
if set(found) != set(expected):
errors.append(
"feature-catalog.md IDs do not match JSON"
)
for feature_id, cells in found.items():
feature = expected[feature_id]
runtime = feature["runtime_validation"]
runtime_display = (
runtime["status"]
if runtime["required"]
else "not required"
)
actual = {
"name": cells[1].replace("\\|", "|"),
"domain": clean(cells[2]),
"status": clean(cells[3]),
"maturity": clean(cells[4]),
"runtime": clean(cells[5]),
}
wanted = {
"name": feature["name"],
"domain": feature["domain"],
"status": feature["status"],
"maturity": feature["evidence_maturity"],
"runtime": runtime_display,
}
for field, expected_value in wanted.items():
if actual[field] != expected_value:
errors.append(
f"feature-catalog.md: {feature_id} "
f"{field}: {actual[field]!r} "
f"!= {expected_value!r}"
)
def validate_reviews(
features: list[dict[str, Any]],
errors: list[str],
) -> int:
catalog = {
feature["id"]: feature
for feature in features
}
checked = 0
for path in sorted(
REVIEWS_DIR.glob("evidence-sample-*.md")
):
current_id: str | None = None
results: set[str] = set()
for line_number, line in enumerate(
path.read_text(encoding="utf-8").splitlines(),
start=1,
):
heading = REVIEW_HEADING_RE.match(line)
if heading:
current_id = heading.group(1)
continue
result = REVIEW_RESULT_RE.match(line)
if not result or current_id is None:
continue
status, maturity = result.groups()
results.add(current_id)
checked += 1
feature = catalog.get(current_id)
if feature is None:
errors.append(
f"{path.relative_to(ROOT)}:{line_number}: "
f"unknown feature {current_id}"
)
continue
if feature["status"] != status:
errors.append(
f"{current_id}: review status {status!r} "
f"!= catalog {feature['status']!r}"
)
if feature["evidence_maturity"] != maturity:
errors.append(
f"{current_id}: review maturity {maturity!r} "
f"!= catalog "
f"{feature['evidence_maturity']!r}"
)
if path.name == "evidence-sample-01.md" and len(results) != 12:
errors.append(
f"{path.relative_to(ROOT)}: expected 12 "
f"review results, found {len(results)}"
)
return checked
def validate_whitespace(errors: list[str]) -> None:
for path in sorted(ROOT.rglob("*")):
if not path.is_file():
continue
if path.suffix not in {".md", ".json", ".py", ".txt"}:
continue
for line_number, line in enumerate(
path.read_text(
encoding="utf-8",
errors="replace",
).splitlines(),
start=1,
):
if line != line.rstrip(" \t"):
errors.append(
f"{path.relative_to(ROOT)}:{line_number}: "
"trailing whitespace"
)
if any(ROOT.rglob("*.pyc")):
errors.append("Generated .pyc files exist")
if any(
path.is_dir()
for path in ROOT.rglob("__pycache__")
):
errors.append("__pycache__ exists")
def main() -> int:
errors: list[str] = []
features = load_catalog(errors)
reviewed = 0
if features:
validate_catalog(features, errors)
validate_domains(features, errors)
validate_catalog_markdown(features, errors)
reviewed = validate_reviews(features, errors)
validate_whitespace(errors)
print(f"Catalog Features: {len(features)}")
print(
"Unique Feature IDs:",
len({feature.get("id") for feature in features}),
)
print(
"Domain Files:",
len(list(DOMAINS_DIR.glob("*.md"))),
)
if features:
print(
"Statuses:",
dict(
Counter(
feature.get("status")
for feature in features
)
),
)
print(
"Evidence Maturity:",
dict(
Counter(
feature.get("evidence_maturity")
for feature in features
)
),
)
print(
"Runtime Validation:",
dict(
Counter(
(
feature.get(
"runtime_validation",
{},
).get("required"),
feature.get(
"runtime_validation",
{},
).get("status"),
)
for feature in features
)
),
)
print(f"Review Results Checked: {reviewed}")
print(f"Consistency Errors: {len(errors)}")
for error in errors:
print(f"ERROR: {error}")
return 1 if errors else 0
if __name__ == "__main__":
sys.exit(main())
+172
View File
@@ -0,0 +1,172 @@
#!/usr/bin/env python3
"""Validate the Odysseus public discovery documentation package."""
from __future__ import annotations
import argparse
import json
import re
import sys
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
REPO_ROOT = ROOT.parents[1]
CATALOG_PATH = ROOT / "feature-catalog.json"
DOMAINS_DIR = ROOT / "domains"
EXPECTED_COMMIT = "d8a2059df8e53bc7275c45339849d14c8651e73c"
ALLOWED_STATUSES = {
"verified", "partial", "disabled", "experimental", "legacy",
"dead-code-candidate", "unverified"
}
LINE_RANGE_RE = re.compile(r"^L([1-9]\d*)-L([1-9]\d*)$")
FEATURE_HEADING_RE = re.compile(
r"^##\s+`?([A-Z][A-Z0-9]*-\d{3})`?\s+(?:—|-)\s+.+$",
re.MULTILINE,
)
LINK_RE = re.compile(r"(?<!!)\[[^\]]*\]\(([^)]+)\)")
FORBIDDEN_TERMS_RE = re.compile(
r"(roadforge|kanban|matrix|owner link|editor link|github support|"
r"private maintainer|OD-AUD-|[A-Z]{2,10}-AUD-\d+|TASK-\d+)",
re.IGNORECASE,
)
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(description="Validate Odysseus discovery docs.")
parser.add_argument(
"--repo-root",
type=Path,
default=REPO_ROOT,
help="Path to Odysseus repository root.",
)
return parser.parse_args()
def validate() -> int:
args = parse_args()
repo_root = args.repo_root.resolve()
errors: list[str] = []
# 1. Validate Catalog JSON existence and content
if not CATALOG_PATH.is_file():
errors.append(f"Missing catalog file: {CATALOG_PATH}")
print(f"Errors: {len(errors)}")
for e in errors:
print(f"ERROR: {e}")
return 1
try:
catalog = json.loads(CATALOG_PATH.read_text(encoding="utf-8"))
except Exception as e:
errors.append(f"Failed to parse catalog JSON: {e}")
print(f"Errors: {len(errors)}")
for err in errors:
print(f"ERROR: {err}")
return 1
if not isinstance(catalog, list):
errors.append("feature-catalog.json must be a JSON array")
return 1
feature_ids = [item.get("id") for item in catalog if isinstance(item, dict)]
if len(feature_ids) != 79:
errors.append(f"Expected 79 unique feature IDs, found {len(feature_ids)}")
if len(feature_ids) != len(set(feature_ids)):
errors.append("Duplicate feature IDs found in catalog JSON")
# Domain catalog counts
catalog_domain_counts: dict[str, int] = {}
for item in catalog:
if not isinstance(item, dict):
errors.append("Catalog item is not an object")
continue
fid = item.get("id", "<missing>")
status = item.get("status")
domain = item.get("domain", "").lower()
catalog_domain_counts[domain] = catalog_domain_counts.get(domain, 0) + 1
if status not in ALLOWED_STATUSES:
errors.append(f"{fid}: Invalid status '{status}'")
evidence_list = item.get("evidence")
if not isinstance(evidence_list, list) or not evidence_list:
errors.append(f"{fid}: Missing or empty evidence list")
continue
for ev in evidence_list:
if not isinstance(ev, dict):
errors.append(f"{fid}: Evidence item is not an object")
continue
path_str = ev.get("path")
lr_str = str(ev.get("line_range", ""))
if not path_str or Path(path_str).is_absolute() or ".." in Path(path_str).parts:
errors.append(f"{fid}: Unsafe or invalid path '{path_str}'")
continue
# Check path exists in repo
target_path = repo_root / path_str
if not target_path.is_file():
errors.append(f"{fid}: Referenced path '{path_str}' does not exist on disk")
continue
# Check line range format & bounds
m = LINE_RANGE_RE.fullmatch(lr_str)
if not m:
errors.append(f"{fid}: Invalid line range format '{lr_str}' for path '{path_str}'")
continue
start, end = int(m.group(1)), int(m.group(2))
lines_cnt = len(target_path.read_text(encoding="utf-8", errors="ignore").splitlines())
if start > end or end > lines_cnt or start < 1:
errors.append(
f"{fid}: Line range '{lr_str}' exceeds file length ({lines_cnt} lines) in '{path_str}'"
)
# 2. Check Domain Markdown files
md_feature_ids: list[str] = []
domain_files = sorted(DOMAINS_DIR.glob("*.md"))
for df in domain_files:
domain_name = df.stem.lower()
content = df.read_text(encoding="utf-8")
found_ids = FEATURE_HEADING_RE.findall(content)
md_feature_ids.extend(found_ids)
if len(found_ids) != catalog_domain_counts.get(domain_name, 0):
errors.append(
f"Domain '{domain_name}' count mismatch: catalog has {catalog_domain_counts.get(domain_name, 0)}, Markdown has {len(found_ids)}"
)
if sorted(md_feature_ids) != sorted(feature_ids):
errors.append("Markdown domain feature IDs do not match catalog JSON feature IDs")
# 3. Check for forbidden/private terms, sensitive credentials, and broken links across all docs
for md_file in ROOT.rglob("*.md"):
rel_md = md_file.relative_to(ROOT)
content = md_file.read_text(encoding="utf-8")
# Forbidden terms scan
forbidden_matches = FORBIDDEN_TERMS_RE.findall(content)
if forbidden_matches:
errors.append(
f"{rel_md}: Found forbidden/internal terms: {set(forbidden_matches)}"
)
# Broken local link check
for target in LINK_RE.findall(content):
target = target.strip().strip("<>")
if not target or target.startswith(("#", "http://", "https://", "mailto:")):
continue
target_path = target.split("#", 1)[0]
resolved = (md_file.parent / target_path).resolve()
if not resolved.exists():
errors.append(f"{rel_md}: Broken local link '{target}'")
print(f"Catalog Features: {len(catalog)}")
print(f"Domain Files: {len(domain_files)}")
print(f"Validation Errors: {len(errors)}")
for err in errors:
print(f"ERROR: {err}")
return 1 if errors else 0
if __name__ == "__main__":
sys.exit(validate())
File diff suppressed because it is too large Load Diff