diff --git a/specs/discovery/README.md b/discovery/README.md similarity index 94% rename from specs/discovery/README.md rename to discovery/README.md index 4343f9ec7..f4c205837 100644 --- a/specs/discovery/README.md +++ b/discovery/README.md @@ -31,4 +31,4 @@ Compact, code-grounded discovery maps of cross-cutting systems in the checked-in 4. When no owning specification exists, retain the verified finding here and record missing documentation ownership as a follow-up; otherwise retain unresolved context here and correct stale wording. > [!NOTE] -> This package intentionally contains no generator, validator, maturity scale, feature database, or parallel work tracker. The [architecture runtime inventory](../architecture-runtime-inventory.md) remains useful structural context, but is an explicitly draft snapshot. +> This package intentionally contains no generator, validator, maturity scale, feature database, or parallel work tracker. The [architecture runtime inventory](../specs/architecture-runtime-inventory.md) remains useful structural context, but is an explicitly draft snapshot. diff --git a/specs/discovery/safety-boundaries.md b/discovery/safety-boundaries.md similarity index 79% rename from specs/discovery/safety-boundaries.md rename to discovery/safety-boundaries.md index 4340aa8d5..fa3ab7f3b 100644 --- a/specs/discovery/safety-boundaries.md +++ b/discovery/safety-boundaries.md @@ -37,7 +37,7 @@ - **User-controlled inputs:** Tool paths, patches, file contents, search patterns, and workspace selection passed into the tool dispatcher. -- **Current safeguards:** [`src/tool_execution.py`](../../src/tool_execution.py) resolves paths, blocks sensitive subpaths, applies allowlist containment, and tightens paths to the active workspace when one is bound. File tools use those resolvers. +- **Current safeguards:** [`src/tool_execution.py`](../src/tool_execution.py) resolves paths, blocks sensitive subpaths, applies allowlist containment, and tightens paths to the active workspace when one is bound. File tools use those resolvers. - **Confirmed risks or gaps:** Intentional authority with safeguards. The file-tool policy does not sandbox the shell; treating a workspace as a whole-process containment boundary would be incorrect. @@ -51,7 +51,7 @@ - **User-controlled inputs:** Chat content, attached/retrieved content that may influence the model, tool arguments, per-request tool selection, and policy toggles. -- **Current safeguards:** [`src/tool_security.py`](../../src/tool_security.py) blocks protected tools for non-admin users and fails closed for malformed tool names; [`src/tool_policy.py`](../../src/tool_policy.py) supports disabled and guide-only policy; prompt-security helpers label untrusted context. +- **Current safeguards:** [`src/tool_security.py`](../src/tool_security.py) blocks protected tools for non-admin users and fails closed for malformed tool names; [`src/tool_policy.py`](../src/tool_policy.py) supports disabled and guide-only policy; prompt-security helpers label untrusted context. - **Confirmed risks or gaps:** Credible risk requiring verification: aliases, legacy text tools, native function calls, and MCP-qualified names must all reach the same policy outcome. The code has specific alias handling for email/MCP names, which makes this a sensitive compatibility seam. @@ -65,7 +65,7 @@ - **User-controlled inputs:** Server configuration, remote OAuth completion, tool arguments, and model-selected MCP calls. -- **Current safeguards:** MCP routes are registered through [`routes/mcp_routes.py`](../../routes/mcp_routes.py); MCP-qualified tools are denied to non-admin users by [`src/tool_security.py`](../../src/tool_security.py). OAuth state and token persistence are handled in [`src/mcp_oauth.py`](../../src/mcp_oauth.py). +- **Current safeguards:** MCP routes are registered through [`routes/mcp_routes.py`](../routes/mcp_routes.py); MCP-qualified tools are denied to non-admin users by [`src/tool_security.py`](../src/tool_security.py). OAuth state and token persistence are handled in [`src/mcp_oauth.py`](../src/mcp_oauth.py). - **Confirmed risks or gaps:** Credible risk requiring verification: an MCP server authority is broader than the application can infer from its tool name. This map does not establish a trust or approval model for server installation and individual tool invocation. @@ -79,7 +79,7 @@ - **User-controlled inputs:** Search/fetch URLs, imported skill URLs, webhook configuration, and some endpoint settings. -- **Current safeguards:** [`src/url_security.py`](../../src/url_security.py) validates untrusted public HTTP URLs and fails closed on unsuitable schemes or private addresses. [`services/search/content.py`](../../services/search/content.py) resolves and rejects non-public hosts, pins resolved addresses for fetches, caps bodies, and limits redirects. +- **Current safeguards:** [`src/url_security.py`](../src/url_security.py) validates untrusted public HTTP URLs and fails closed on unsuitable schemes or private addresses. [`services/search/content.py`](../services/search/content.py) resolves and rejects non-public hosts, pins resolved addresses for fetches, caps bodies, and limits redirects. - **Confirmed risks or gaps:** Intentional split: administrator-created model endpoints may target private providers, while untrusted URLs use public-address checks. That distinction is required for self-hosted deployments but needs explicit call-site review. @@ -93,9 +93,9 @@ - **User-controlled inputs:** Administrative configuration, login/unlock requests, imported settings, and agent vault tool arguments. -- **Current safeguards:** [`src/secret_storage.py`](../../src/secret_storage.py) uses a locally stored Fernet key with restrictive permissions for supported database secrets. Vault routes require an administrator, avoid passing master passwords in command arguments, and set restrictive permissions on the vault-session file. +- **Current safeguards:** [`src/secret_storage.py`](../src/secret_storage.py) uses a locally stored Fernet key with restrictive permissions for supported database secrets. Vault routes require an administrator, avoid passing master passwords in command arguments, and set restrictive permissions on the vault-session file. -- **Confirmed risks or gaps:** Confirmed current boundary: vault session data is persisted through the vault path, not through [`src/secret_storage.py`](../../src/secret_storage.py). This is an unresolved question about current security semantics, not a confirmed exposure. +- **Confirmed risks or gaps:** Confirmed current boundary: vault session data is persisted through the vault path, not through [`src/secret_storage.py`](../src/secret_storage.py). This is an unresolved question about current security semantics, not a confirmed exposure. - **Unverified behaviour:** Current encryption-at-rest, owner scope, rotation, lock/logout, backup/restore, and log/tool-result exposure behaviour remains unverified. @@ -107,7 +107,7 @@ - **User-controlled inputs:** Login/signup data, session cookies, API tokens, authentication configuration, and requests to privileged routes. -- **Current safeguards:** [`core/auth.py`](../../core/auth.py), [`core/middleware.py`](../../core/middleware.py), and route-level checks establish identity and administrator gates. [`app.py`](../../app.py) warns when localhost bypass is configured; [`SECURITY.md`](../../SECURITY.md) documents deployment requirements. +- **Current safeguards:** [`core/auth.py`](../core/auth.py), [`core/middleware.py`](../core/middleware.py), and route-level checks establish identity and administrator gates. [`app.py`](../app.py) warns when localhost bypass is configured; [`SECURITY.md`](../SECURITY.md) documents deployment requirements. - **Confirmed risks or gaps:** Intentional authority with safeguards. Security depends on deployments keeping authentication enabled and internal services private; this map does not audit reverse-proxy or environment configuration. @@ -123,7 +123,7 @@ - **Current safeguards:** Administrative wipe routes use the administrative boundary. Cleanup exposes a preview route before mutation. The documented backup tool requires explicit restore confirmation, stages the old data directory, and validates archive members before extraction. -- **Confirmed risks or gaps:** Intentional destructive authority. Backup archives contain secrets by design, as documented in [`docs/backup-restore.md`](../../docs/backup-restore.md); this is an operator confidentiality responsibility, not a code defect established here. +- **Confirmed risks or gaps:** Intentional destructive authority. Backup archives contain secrets by design, as documented in [`docs/backup-restore.md`](../docs/backup-restore.md); this is an operator confidentiality responsibility, not a code defect established here. - **Unverified behaviour:** Role-gate, confirmation, archive-rejection, staged-recovery, and owner-isolation behaviour remains unverified. No destructive runtime test was performed. @@ -135,7 +135,7 @@ - **User-controlled inputs:** Stored task prompt, schedule, model/crew selection, enabled-tool configuration, output target, and prior persisted state. -- **Current safeguards:** [`src/task_scheduler.py`](../../src/task_scheduler.py) serializes execution, records task runs, associates work with an owner, and applies the agent owner-based tool gate. [`src/bg_jobs.py`](../../src/bg_jobs.py) keeps bounded state and can terminate overlong subprocess jobs. +- **Current safeguards:** [`src/task_scheduler.py`](../src/task_scheduler.py) serializes execution, records task runs, associates work with an owner, and applies the agent owner-based tool gate. [`src/bg_jobs.py`](../src/bg_jobs.py) keeps bounded state and can terminate overlong subprocess jobs. - **Confirmed risks or gaps:** Credible risk requiring verification: authority is inherited and exercised later, so changes to roles, task configuration, and disabled tools must be checked at execution time rather than assumed from task creation. diff --git a/discovery/system-map.md b/discovery/system-map.md new file mode 100644 index 000000000..938af76f3 --- /dev/null +++ b/discovery/system-map.md @@ -0,0 +1,139 @@ +# Current system map + +> [!NOTE] +> This non-canonical discovery map is an evidence guide, not an exhaustive feature catalog or runtime certification. Verify the cited source before relying on a finding. Each section records local implementation observations, evidence locations, confirmed current problems, and unresolved factual questions. + +## Navigate the system + +- [Startup and application composition](#startup-and-application-composition) +- [Frontend shell and browser interaction](#frontend-shell-and-browser-interaction) +- [Chat, sessions, and streaming](#chat-sessions-and-streaming) +- [Agents, tools, and execution](#agents-tools-and-execution) +- [Models, providers, and local serving](#models-providers-and-local-serving) +- [Search and research](#search-and-research) +- [Documents, retrieval, and personal knowledge](#documents-retrieval-and-personal-knowledge) +- [Memory and skills](#memory-and-skills) +- [Email, calendar, contacts, notes, and tasks](#email-calendar-contacts-notes-and-tasks) +- [Media, speech, and image work](#media-speech-and-image-work) +- [Authentication, secrets, and privileged administration](#authentication-secrets-and-privileged-administration) +- [Persistence, background work, and operations](#persistence-background-work-and-operations) + +## Startup and application composition + +- **How it works:** [`app.py`](../app.py) creates the application, mounts static assets, constructs shared services, registers route factories, and owns lifespan startup and shutdown. [`src/app_initializer.py`](../src/app_initializer.py) prepares application state; [`core/`](../core/) provides persistence, authentication, middleware, sessions, and platform helpers. + +- **Evidence locations:** [`app.py`](../app.py); [`src/app_initializer.py`](../src/app_initializer.py); [`core/database.py`](../core/database.py); [`core/auth.py`](../core/auth.py); [`core/middleware.py`](../core/middleware.py); [`routes/`](../routes/). + +- **Known problems:** None recorded by this mapping. + +- **Open question:** Which component currently owns startup and shutdown for each long-lived service? + +## Frontend shell and browser interaction + +- **How it works:** [`static/index.html`](../static/index.html) is served by the root and SPA deep-link routes in [`app.py`](../app.py); [`static/app.js`](../static/app.js), [`static/style.css`](../static/style.css), and [`static/js/`](../static/js/) implement the client surface. + +- **Evidence locations:** [`static/index.html`](../static/index.html); [`static/app.js`](../static/app.js); [`static/js/`](../static/js/); [`static/style.css`](../static/style.css); [`app.py`](../app.py) deep-link handlers. + +- **Known problems:** The `/backgrounds` route in [`app.py`](../app.py) calls `serve_html_with_nonce` for `static/backgrounds.html`, but that file is absent from [`static/`](../static/). This is a confirmed broken prototype route, not evidence about the rest of the frontend. + +- **Open question:** Is `/backgrounds` currently an intentionally supported route or an obsolete prototype? + +## Chat, sessions, and streaming + +- **How it works:** [`routes/chat_routes.py`](../routes/chat_routes.py) and [`routes/chat_helpers.py`](../routes/chat_helpers.py) coordinate requests, session state, and SSE delivery. [`src/chat_handler.py`](../src/chat_handler.py), [`src/chat_processor.py`](../src/chat_processor.py), [`src/llm_core.py`](../src/llm_core.py), and [`src/session_actions.py`](../src/session_actions.py) provide message preparation, provider interaction, and session operations. + +- **Evidence locations:** [`routes/chat_routes.py`](../routes/chat_routes.py); [`routes/chat_helpers.py`](../routes/chat_helpers.py); [`routes/session_routes.py`](../routes/session_routes.py); [`src/chat_handler.py`](../src/chat_handler.py); [`src/chat_processor.py`](../src/chat_processor.py); [`src/llm_core.py`](../src/llm_core.py); [`core/session_manager.py`](../core/session_manager.py). + +- **Known problems:** [`src/agent_loop.py`](../src/agent_loop.py) annotates `_resolved_tool_event_name` with `Any` but imports no `Any` and does not enable postponed annotation evaluation. Python evaluates that annotation while importing the module, so this is an import-time defect at the checked baseline. + +- **Open question:** No end-to-end provider or browser streaming run was performed for this map. + +## Agents, tools, and execution + +- **How it works:** [`src/agent_loop.py`](../src/agent_loop.py) drives multi-round tool use. [`src/tool_execution.py`](../src/tool_execution.py) dispatches calls and binds workspace context. [`src/agent_tools/`](../src/agent_tools/) contains individual implementations; [`src/tool_security.py`](../src/tool_security.py) and [`src/tool_policy.py`](../src/tool_policy.py) apply role and request policies. Long-running command work is represented by [`src/bg_jobs.py`](../src/bg_jobs.py). + +- **Evidence locations:** [`src/agent_loop.py`](../src/agent_loop.py); [`src/tool_execution.py`](../src/tool_execution.py); [`src/agent_tools/`](../src/agent_tools/); [`src/tool_security.py`](../src/tool_security.py); [`src/tool_policy.py`](../src/tool_policy.py); [`src/tool_schemas.py`](../src/tool_schemas.py); [`src/bg_jobs.py`](../src/bg_jobs.py). + +- **Known problems:** The import-time annotation defect above blocks the main agent/tool path. The shell is intentionally not a filesystem or network sandbox; that is an authority boundary, not by itself a vulnerability claim. + +- **Open question:** Which native, legacy, and MCP-qualified invocation paths reach each policy gate? + +## Models, providers, and local serving + +- **How it works:** Model routes delegate to discovery, capabilities, endpoint resolution, and LLM core modules. Cookbook routes and hardware-fit services handle model lifecycle and local-serving support. + +- **Evidence locations:** [`routes/model_routes.py`](../routes/model_routes.py); [`src/model_discovery.py`](../src/model_discovery.py); [`src/model_capabilities.py`](../src/model_capabilities.py); [`src/endpoint_resolver.py`](../src/endpoint_resolver.py); [`src/llm_core.py`](../src/llm_core.py); [`routes/cookbook_routes.py`](../routes/cookbook_routes.py); [`src/cookbook_serve_lifecycle.py`](../src/cookbook_serve_lifecycle.py); [`services/hwfit/`](../services/hwfit/). + +- **Known problems:** None recorded by this mapping. + +- **Open question:** Which endpoint inputs are administrator-created and permitted to use private provider addresses? + +## Search and research + +- **How it works:** HTTP search routes use [`services/search/`](../services/search/); research is exposed through [`routes/research/`](../routes/research/) and implemented in [`services/research/`](../services/research/), [`src/deep_research.py`](../src/deep_research.py), and related helpers. [`src/search/`](../src/search/) remains an import-compatibility layer for callers not yet moved to `services.search`. + +- **Evidence locations:** [`routes/search_routes.py`](../routes/search_routes.py); [`services/search/`](../services/search/); [`routes/research/research_routes.py`](../routes/research/research_routes.py); [`services/research/`](../services/research/); [`src/deep_research.py`](../src/deep_research.py); [`src/search/`](../src/search/). + +- **Known problems:** None recorded by this mapping. + +- **Open question:** No live provider request was made; provider configuration and network access remain unverified. + +## Documents, retrieval, and personal knowledge + +- **How it works:** Document routes coordinate upload handling, document processing, and editor actions. Personal-document and RAG modules use Chroma and embedding clients. PDF viewing uses the optional-dependency loader in [`src/pdf_runtime.py`](../src/pdf_runtime.py); form extraction and filling live separately in [`src/pdf_forms.py`](../src/pdf_forms.py) and [`src/pdf_form_doc.py`](../src/pdf_form_doc.py). + +- **Evidence locations:** [`routes/document_routes.py`](../routes/document_routes.py); [`src/upload_handler.py`](../src/upload_handler.py); [`src/document_processor.py`](../src/document_processor.py); [`src/document_actions.py`](../src/document_actions.py); [`src/personal_docs.py`](../src/personal_docs.py); [`src/rag_manager.py`](../src/rag_manager.py); [`src/embeddings.py`](../src/embeddings.py); [`src/pdf_runtime.py`](../src/pdf_runtime.py); [`src/pdf_forms.py`](../src/pdf_forms.py); [`src/pdf_form_doc.py`](../src/pdf_form_doc.py). + +- **Known problems:** PDF viewing/runtime loading and PDF form processing are separate implementations. That separation is confirmed and intentional in the source; it is not a defect without a reported behavioural failure. + +- **Open question:** Optional PDF dependencies and representative uploaded documents were not exercised. + +## Memory and skills + +- **How it works:** Memory routes use [`services/memory/`](../services/memory/) and vector helpers. Skills are exposed through [`routes/skills_routes.py`](../routes/skills_routes.py), stored and managed in [`services/memory/skills.py`](../services/memory/skills.py), and may be imported through [`services/memory/skill_importer.py`](../services/memory/skill_importer.py). + +- **Evidence locations:** [`routes/memory/memory_routes.py`](../routes/memory/memory_routes.py); [`services/memory/`](../services/memory/); [`src/memory.py`](../src/memory.py); [`src/memory_vector.py`](../src/memory_vector.py); [`routes/skills_routes.py`](../routes/skills_routes.py); [`services/memory/skills.py`](../services/memory/skills.py); [`services/memory/skill_importer.py`](../services/memory/skill_importer.py). + +- **Known problems:** None recorded by this mapping. + +- **Open question:** Which imported skill content can reach execution-capable paths, and which validation occurs before that point? + +## Email, calendar, contacts, notes, and tasks + +- **How it works:** Dedicated route modules own email, CalDAV calendar, CardDAV contacts, notes, and tasks. Supporting modules include email helpers and pollers, CalDAV sync and writeback, and the task scheduler. + +- **Evidence locations:** [`routes/email_routes.py`](../routes/email_routes.py); [`routes/calendar_routes.py`](../routes/calendar_routes.py); [`routes/contacts/contacts_routes.py`](../routes/contacts/contacts_routes.py); [`routes/note/note_routes.py`](../routes/note/note_routes.py); [`routes/task_routes.py`](../routes/task_routes.py); [`routes/assistant_routes.py`](../routes/assistant_routes.py); [`src/caldav_sync.py`](../src/caldav_sync.py); [`src/caldav_writeback.py`](../src/caldav_writeback.py); [`src/task_scheduler.py`](../src/task_scheduler.py). + +- **Known problems:** None recorded by this mapping. + +- **Open question:** External account behaviour, writeback, and delivery require controlled credentials and are not runtime-validated here. + +## Media, speech, and image work + +- **How it works:** Gallery and image routes coordinate media features. Service modules own speech and media integrations; [`src/generated_images.py`](../src/generated_images.py) and [`src/visual_report.py`](../src/visual_report.py) support artifact handling and presentation. + +- **Evidence locations:** [`routes/gallery/gallery_routes.py`](../routes/gallery/gallery_routes.py); [`routes/stt_routes.py`](../routes/stt_routes.py); [`routes/tts_routes.py`](../routes/tts_routes.py); [`src/generated_images.py`](../src/generated_images.py); [`services/stt/`](../services/stt/); [`services/tts/`](../services/tts/); [`services/faces/`](../services/faces/); [`src/visual_report.py`](../src/visual_report.py). + +- **Known problems:** None recorded by this mapping. + +- **Open question:** Hardware- and provider-dependent media workflows were not exercised. + +## Authentication, secrets, and privileged administration + +- **How it works:** [`core/auth.py`](../core/auth.py) and [`core/middleware.py`](../core/middleware.py) provide identity and request gates. [`src/secret_storage.py`](../src/secret_storage.py) encrypts application-managed database secrets with a local Fernet key. Vault handling is separate: [`routes/vault_routes.py`](../routes/vault_routes.py) and [`src/tools/vault.py`](../src/tools/vault.py) invoke the Bitwarden CLI and persist its session data in the application data area. + +- **Evidence locations:** [`core/auth.py`](../core/auth.py); [`core/middleware.py`](../core/middleware.py); [`routes/auth_routes.py`](../routes/auth_routes.py); [`routes/api_token_routes.py`](../routes/api_token_routes.py); [`src/secret_storage.py`](../src/secret_storage.py); [`routes/vault_routes.py`](../routes/vault_routes.py); [`src/tools/vault.py`](../src/tools/vault.py); [`routes/admin_wipe/admin_wipe_routes.py`](../routes/admin_wipe/admin_wipe_routes.py). + +- **Known problems:** Vault-command handling and local application secret storage are distinct paths with different storage mechanisms. This is a source-confirmed boundary, not evidence that either path is compromised. + +- **Open question:** What are the current confidentiality, ownership, rotation, and backup semantics for vault session data? + +## Persistence, background work, and operations + +- **How it works:** SQLite models and persistence are centred in [`core/database.py`](../core/database.py); managers use application data paths. The scheduler and background-job monitor can continue work outside a live browser request. Operational routes cover cleanup, backup, and administrative wipe; the repository also provides a backup script and user documentation. + +- **Evidence locations:** [`core/database.py`](../core/database.py); [`src/runtime_paths.py`](../src/runtime_paths.py); [`src/task_scheduler.py`](../src/task_scheduler.py); [`src/bg_jobs.py`](../src/bg_jobs.py); [`src/bg_monitor.py`](../src/bg_monitor.py); [`routes/backup_routes.py`](../routes/backup_routes.py); [`routes/cleanup/cleanup_routes.py`](../routes/cleanup/cleanup_routes.py); [`routes/admin_wipe/admin_wipe_routes.py`](../routes/admin_wipe/admin_wipe_routes.py); [`scripts/odysseus-backup`](../scripts/odysseus-backup); [`docs/backup-restore.md`](../docs/backup-restore.md). + +- **Known problems:** None recorded by this mapping. + +- **Open question:** What current behaviour applies to background execution, cancellation, retries, and authority inheritance? diff --git a/specs/discovery/system-map.md b/specs/discovery/system-map.md deleted file mode 100644 index ce65e2b02..000000000 --- a/specs/discovery/system-map.md +++ /dev/null @@ -1,139 +0,0 @@ -# Current system map - -> [!NOTE] -> This non-canonical discovery map is an evidence guide, not an exhaustive feature catalog or runtime certification. Verify the cited source before relying on a finding. Each section records local implementation observations, evidence locations, confirmed current problems, and unresolved factual questions. - -## Navigate the system - -- [Startup and application composition](#startup-and-application-composition) -- [Frontend shell and browser interaction](#frontend-shell-and-browser-interaction) -- [Chat, sessions, and streaming](#chat-sessions-and-streaming) -- [Agents, tools, and execution](#agents-tools-and-execution) -- [Models, providers, and local serving](#models-providers-and-local-serving) -- [Search and research](#search-and-research) -- [Documents, retrieval, and personal knowledge](#documents-retrieval-and-personal-knowledge) -- [Memory and skills](#memory-and-skills) -- [Email, calendar, contacts, notes, and tasks](#email-calendar-contacts-notes-and-tasks) -- [Media, speech, and image work](#media-speech-and-image-work) -- [Authentication, secrets, and privileged administration](#authentication-secrets-and-privileged-administration) -- [Persistence, background work, and operations](#persistence-background-work-and-operations) - -## Startup and application composition - -- **How it works:** [`app.py`](../../app.py) creates the application, mounts static assets, constructs shared services, registers route factories, and owns lifespan startup and shutdown. [`src/app_initializer.py`](../../src/app_initializer.py) prepares application state; [`core/`](../../core/) provides persistence, authentication, middleware, sessions, and platform helpers. - -- **Evidence locations:** [`app.py`](../../app.py); [`src/app_initializer.py`](../../src/app_initializer.py); [`core/database.py`](../../core/database.py); [`core/auth.py`](../../core/auth.py); [`core/middleware.py`](../../core/middleware.py); [`routes/`](../../routes/). - -- **Known problems:** None recorded by this mapping. - -- **Open question:** Which component currently owns startup and shutdown for each long-lived service? - -## Frontend shell and browser interaction - -- **How it works:** [`static/index.html`](../../static/index.html) is served by the root and SPA deep-link routes in [`app.py`](../../app.py); [`static/app.js`](../../static/app.js), [`static/style.css`](../../static/style.css), and [`static/js/`](../../static/js/) implement the client surface. - -- **Evidence locations:** [`static/index.html`](../../static/index.html); [`static/app.js`](../../static/app.js); [`static/js/`](../../static/js/); [`static/style.css`](../../static/style.css); [`app.py`](../../app.py) deep-link handlers. - -- **Known problems:** The `/backgrounds` route in [`app.py`](../../app.py) calls `serve_html_with_nonce` for `static/backgrounds.html`, but that file is absent from [`static/`](../../static/). This is a confirmed broken prototype route, not evidence about the rest of the frontend. - -- **Open question:** Is `/backgrounds` currently an intentionally supported route or an obsolete prototype? - -## Chat, sessions, and streaming - -- **How it works:** [`routes/chat_routes.py`](../../routes/chat_routes.py) and [`routes/chat_helpers.py`](../../routes/chat_helpers.py) coordinate requests, session state, and SSE delivery. [`src/chat_handler.py`](../../src/chat_handler.py), [`src/chat_processor.py`](../../src/chat_processor.py), [`src/llm_core.py`](../../src/llm_core.py), and [`src/session_actions.py`](../../src/session_actions.py) provide message preparation, provider interaction, and session operations. - -- **Evidence locations:** [`routes/chat_routes.py`](../../routes/chat_routes.py); [`routes/chat_helpers.py`](../../routes/chat_helpers.py); [`routes/session_routes.py`](../../routes/session_routes.py); [`src/chat_handler.py`](../../src/chat_handler.py); [`src/chat_processor.py`](../../src/chat_processor.py); [`src/llm_core.py`](../../src/llm_core.py); [`core/session_manager.py`](../../core/session_manager.py). - -- **Known problems:** [`src/agent_loop.py`](../../src/agent_loop.py) annotates `_resolved_tool_event_name` with `Any` but imports no `Any` and does not enable postponed annotation evaluation. Python evaluates that annotation while importing the module, so this is an import-time defect at the checked baseline. - -- **Open question:** No end-to-end provider or browser streaming run was performed for this map. - -## Agents, tools, and execution - -- **How it works:** [`src/agent_loop.py`](../../src/agent_loop.py) drives multi-round tool use. [`src/tool_execution.py`](../../src/tool_execution.py) dispatches calls and binds workspace context. [`src/agent_tools/`](../../src/agent_tools/) contains individual implementations; [`src/tool_security.py`](../../src/tool_security.py) and [`src/tool_policy.py`](../../src/tool_policy.py) apply role and request policies. Long-running command work is represented by [`src/bg_jobs.py`](../../src/bg_jobs.py). - -- **Evidence locations:** [`src/agent_loop.py`](../../src/agent_loop.py); [`src/tool_execution.py`](../../src/tool_execution.py); [`src/agent_tools/`](../../src/agent_tools/); [`src/tool_security.py`](../../src/tool_security.py); [`src/tool_policy.py`](../../src/tool_policy.py); [`src/tool_schemas.py`](../../src/tool_schemas.py); [`src/bg_jobs.py`](../../src/bg_jobs.py). - -- **Known problems:** The import-time annotation defect above blocks the main agent/tool path. The shell is intentionally not a filesystem or network sandbox; that is an authority boundary, not by itself a vulnerability claim. - -- **Open question:** Which native, legacy, and MCP-qualified invocation paths reach each policy gate? - -## Models, providers, and local serving - -- **How it works:** Model routes delegate to discovery, capabilities, endpoint resolution, and LLM core modules. Cookbook routes and hardware-fit services handle model lifecycle and local-serving support. - -- **Evidence locations:** [`routes/model_routes.py`](../../routes/model_routes.py); [`src/model_discovery.py`](../../src/model_discovery.py); [`src/model_capabilities.py`](../../src/model_capabilities.py); [`src/endpoint_resolver.py`](../../src/endpoint_resolver.py); [`src/llm_core.py`](../../src/llm_core.py); [`routes/cookbook_routes.py`](../../routes/cookbook_routes.py); [`src/cookbook_serve_lifecycle.py`](../../src/cookbook_serve_lifecycle.py); [`services/hwfit/`](../../services/hwfit/). - -- **Known problems:** None recorded by this mapping. - -- **Open question:** Which endpoint inputs are administrator-created and permitted to use private provider addresses? - -## Search and research - -- **How it works:** HTTP search routes use [`services/search/`](../../services/search/); research is exposed through [`routes/research/`](../../routes/research/) and implemented in [`services/research/`](../../services/research/), [`src/deep_research.py`](../../src/deep_research.py), and related helpers. [`src/search/`](../../src/search/) remains an import-compatibility layer for callers not yet moved to `services.search`. - -- **Evidence locations:** [`routes/search_routes.py`](../../routes/search_routes.py); [`services/search/`](../../services/search/); [`routes/research/research_routes.py`](../../routes/research/research_routes.py); [`services/research/`](../../services/research/); [`src/deep_research.py`](../../src/deep_research.py); [`src/search/`](../../src/search/). - -- **Known problems:** None recorded by this mapping. - -- **Open question:** No live provider request was made; provider configuration and network access remain unverified. - -## Documents, retrieval, and personal knowledge - -- **How it works:** Document routes coordinate upload handling, document processing, and editor actions. Personal-document and RAG modules use Chroma and embedding clients. PDF viewing uses the optional-dependency loader in [`src/pdf_runtime.py`](../../src/pdf_runtime.py); form extraction and filling live separately in [`src/pdf_forms.py`](../../src/pdf_forms.py) and [`src/pdf_form_doc.py`](../../src/pdf_form_doc.py). - -- **Evidence locations:** [`routes/document_routes.py`](../../routes/document_routes.py); [`src/upload_handler.py`](../../src/upload_handler.py); [`src/document_processor.py`](../../src/document_processor.py); [`src/document_actions.py`](../../src/document_actions.py); [`src/personal_docs.py`](../../src/personal_docs.py); [`src/rag_manager.py`](../../src/rag_manager.py); [`src/embeddings.py`](../../src/embeddings.py); [`src/pdf_runtime.py`](../../src/pdf_runtime.py); [`src/pdf_forms.py`](../../src/pdf_forms.py); [`src/pdf_form_doc.py`](../../src/pdf_form_doc.py). - -- **Known problems:** PDF viewing/runtime loading and PDF form processing are separate implementations. That separation is confirmed and intentional in the source; it is not a defect without a reported behavioural failure. - -- **Open question:** Optional PDF dependencies and representative uploaded documents were not exercised. - -## Memory and skills - -- **How it works:** Memory routes use [`services/memory/`](../../services/memory/) and vector helpers. Skills are exposed through [`routes/skills_routes.py`](../../routes/skills_routes.py), stored and managed in [`services/memory/skills.py`](../../services/memory/skills.py), and may be imported through [`services/memory/skill_importer.py`](../../services/memory/skill_importer.py). - -- **Evidence locations:** [`routes/memory/memory_routes.py`](../../routes/memory/memory_routes.py); [`services/memory/`](../../services/memory/); [`src/memory.py`](../../src/memory.py); [`src/memory_vector.py`](../../src/memory_vector.py); [`routes/skills_routes.py`](../../routes/skills_routes.py); [`services/memory/skills.py`](../../services/memory/skills.py); [`services/memory/skill_importer.py`](../../services/memory/skill_importer.py). - -- **Known problems:** None recorded by this mapping. - -- **Open question:** Which imported skill content can reach execution-capable paths, and which validation occurs before that point? - -## Email, calendar, contacts, notes, and tasks - -- **How it works:** Dedicated route modules own email, CalDAV calendar, CardDAV contacts, notes, and tasks. Supporting modules include email helpers and pollers, CalDAV sync and writeback, and the task scheduler. - -- **Evidence locations:** [`routes/email_routes.py`](../../routes/email_routes.py); [`routes/calendar_routes.py`](../../routes/calendar_routes.py); [`routes/contacts/contacts_routes.py`](../../routes/contacts/contacts_routes.py); [`routes/note/note_routes.py`](../../routes/note/note_routes.py); [`routes/task_routes.py`](../../routes/task_routes.py); [`routes/assistant_routes.py`](../../routes/assistant_routes.py); [`src/caldav_sync.py`](../../src/caldav_sync.py); [`src/caldav_writeback.py`](../../src/caldav_writeback.py); [`src/task_scheduler.py`](../../src/task_scheduler.py). - -- **Known problems:** None recorded by this mapping. - -- **Open question:** External account behaviour, writeback, and delivery require controlled credentials and are not runtime-validated here. - -## Media, speech, and image work - -- **How it works:** Gallery and image routes coordinate media features. Service modules own speech and media integrations; [`src/generated_images.py`](../../src/generated_images.py) and [`src/visual_report.py`](../../src/visual_report.py) support artifact handling and presentation. - -- **Evidence locations:** [`routes/gallery/gallery_routes.py`](../../routes/gallery/gallery_routes.py); [`routes/stt_routes.py`](../../routes/stt_routes.py); [`routes/tts_routes.py`](../../routes/tts_routes.py); [`src/generated_images.py`](../../src/generated_images.py); [`services/stt/`](../../services/stt/); [`services/tts/`](../../services/tts/); [`services/faces/`](../../services/faces/); [`src/visual_report.py`](../../src/visual_report.py). - -- **Known problems:** None recorded by this mapping. - -- **Open question:** Hardware- and provider-dependent media workflows were not exercised. - -## Authentication, secrets, and privileged administration - -- **How it works:** [`core/auth.py`](../../core/auth.py) and [`core/middleware.py`](../../core/middleware.py) provide identity and request gates. [`src/secret_storage.py`](../../src/secret_storage.py) encrypts application-managed database secrets with a local Fernet key. Vault handling is separate: [`routes/vault_routes.py`](../../routes/vault_routes.py) and [`src/tools/vault.py`](../../src/tools/vault.py) invoke the Bitwarden CLI and persist its session data in the application data area. - -- **Evidence locations:** [`core/auth.py`](../../core/auth.py); [`core/middleware.py`](../../core/middleware.py); [`routes/auth_routes.py`](../../routes/auth_routes.py); [`routes/api_token_routes.py`](../../routes/api_token_routes.py); [`src/secret_storage.py`](../../src/secret_storage.py); [`routes/vault_routes.py`](../../routes/vault_routes.py); [`src/tools/vault.py`](../../src/tools/vault.py); [`routes/admin_wipe/admin_wipe_routes.py`](../../routes/admin_wipe/admin_wipe_routes.py). - -- **Known problems:** Vault-command handling and local application secret storage are distinct paths with different storage mechanisms. This is a source-confirmed boundary, not evidence that either path is compromised. - -- **Open question:** What are the current confidentiality, ownership, rotation, and backup semantics for vault session data? - -## Persistence, background work, and operations - -- **How it works:** SQLite models and persistence are centred in [`core/database.py`](../../core/database.py); managers use application data paths. The scheduler and background-job monitor can continue work outside a live browser request. Operational routes cover cleanup, backup, and administrative wipe; the repository also provides a backup script and user documentation. - -- **Evidence locations:** [`core/database.py`](../../core/database.py); [`src/runtime_paths.py`](../../src/runtime_paths.py); [`src/task_scheduler.py`](../../src/task_scheduler.py); [`src/bg_jobs.py`](../../src/bg_jobs.py); [`src/bg_monitor.py`](../../src/bg_monitor.py); [`routes/backup_routes.py`](../../routes/backup_routes.py); [`routes/cleanup/cleanup_routes.py`](../../routes/cleanup/cleanup_routes.py); [`routes/admin_wipe/admin_wipe_routes.py`](../../routes/admin_wipe/admin_wipe_routes.py); [`scripts/odysseus-backup`](../../scripts/odysseus-backup); [`docs/backup-restore.md`](../../docs/backup-restore.md). - -- **Known problems:** None recorded by this mapping. - -- **Open question:** What current behaviour applies to background execution, cancellation, retries, and authority inheritance?