mirror of
https://github.com/pewdiepie-archdaemon/odysseus.git
synced 2026-08-02 03:28:36 -04:00
Compare commits
7 Commits
discovery
...
56d0e95a67
| Author | SHA1 | Date | |
|---|---|---|---|
| 56d0e95a67 | |||
| 7480545dd1 | |||
| e36393145b | |||
| 6ab6f7b0b1 | |||
| dfb62c4fba | |||
| 749b8a949a | |||
| cb6c28113a |
@@ -8,8 +8,8 @@ body:
|
||||
value: |
|
||||
**Before submitting:** search [open issues](https://github.com/odysseus-dev/odysseus/issues)
|
||||
and [discussions](https://github.com/odysseus-dev/odysseus/discussions) first.
|
||||
Feature requests that duplicate [ROADMAP.md](https://github.com/odysseus-dev/odysseus/blob/main/ROADMAP.md)
|
||||
or an existing open issue will be closed as duplicates.
|
||||
The [roadmap](https://github.com/odysseus-dev/odysseus/blob/main/ROADMAP.md) is directional rather than a complete backlog.
|
||||
Feature requests that duplicate an existing issue or accepted proposal may be closed as duplicates.
|
||||
|
||||
If your idea needs community input before it becomes a concrete proposal,
|
||||
start a [discussion](https://github.com/odysseus-dev/odysseus/discussions/categories/ideas) instead.
|
||||
|
||||
@@ -55,7 +55,7 @@ A full hover-to-play tour lives on the landing page: [`docs/index.html`](docs/in
|
||||
|
||||
## Contributing
|
||||
|
||||
Help is welcome. The best entry points are fresh-install testing, provider setup bugs, mobile/editor polish, docs, and small focused refactors. See [CONTRIBUTING.md](CONTRIBUTING.md) and [ROADMAP.md](ROADMAP.md).
|
||||
Help is welcome. The best entry points are fresh-install testing, provider setup bugs, mobile/editor polish, documentation, and small focused refactors. See [CONTRIBUTING.md](CONTRIBUTING.md), the [public roadmap](ROADMAP.md), and the open [GitHub issues](https://github.com/odysseus-dev/odysseus/issues).
|
||||
|
||||
## Security
|
||||
|
||||
|
||||
+43
-75
@@ -1,87 +1,55 @@
|
||||
# Roadmap / Help Wanted
|
||||
# Roadmap
|
||||
|
||||
Odysseus is on a voyage, but not home yet. It works great for me (lol), but this ship is moving fast and feedback/help would be appreciated! (I don't know what I'm doing, help).
|
||||
This document provides a high-level view of the areas Odysseus is currently improving.
|
||||
|
||||
If you see weird CSS, strange layout behavior, or a suspiciously murky corner of
|
||||
the codebase, you are probably right to stay away.
|
||||
It is directional rather than exhaustive. Priorities may change as the project evolves, defects are discovered, and maintainers learn more from implementation work and user feedback.
|
||||
|
||||
## High Priority
|
||||
For current implementation work, see the open [GitHub issues](https://github.com/odysseus-dev/odysseus/issues). Accepted behaviour should be documented in the repository alongside the code.
|
||||
|
||||
- SQUASH BUGS
|
||||
- Fresh install smoke tests on Linux, macOS, and Windows. Docker, native Python,
|
||||
and WSL all need coverage.
|
||||
## Current priorities
|
||||
|
||||
- Integration audit: do integrations even work? Confirm what works, what needs setup docs, and what should be removed or hidden.
|
||||
- Cookbook reliability on other computers. This is probably the area most likely to need work across different machines, GPUs, drivers, shells, and Python environments.
|
||||
- Cookbook SGLang support across platforms. Make sure SGLang setup/serve works
|
||||
predictably on Linux, Windows/WSL, macOS where possible, Docker, and common
|
||||
NVIDIA/AMD hardware paths.
|
||||
- Deep Research model presets by hardware. Recommend approved model/parameter
|
||||
profiles for small, medium, and large local setups so people with different
|
||||
hardware can use Deep Research without guessing. Surface this either in Deep
|
||||
Research settings or as a Cookbook scan/dropdown suggestion.
|
||||
- Cookbook model scan/download ranking. Prioritize newer architectures and
|
||||
better hardware-fit models instead of scoring everything almost the same.
|
||||
Ranking should account for architecture age, quant format, VRAM/RAM fit,
|
||||
backend support, vision/mmproj requirements, and likely serve reliability.
|
||||
- Cookbook error feedback and logging. Failed downloads, dependency installs,
|
||||
preflights, and serve jobs should show the actual command/output/error in the
|
||||
UI, with copyable logs and clear next steps instead of just "crashed".
|
||||
- Agent prompt/context bloat. Agent mode is too heavy for smaller local models:
|
||||
tool schemas, skills, memory, documents, and instructions can eat the context
|
||||
before the user request really starts. We need slimmer prompts, better tool
|
||||
selection, smaller default tool sets, and clearer guidance for models with
|
||||
4k/8k/16k context windows.
|
||||
- Local model speculative decoding support. For Odysseus-tuned local models,
|
||||
plan to ship or recommend a small same-tokenizer draft model when the serving
|
||||
backend supports it. Early vLLM testing showed a generic `Qwen3-0.6B` draft
|
||||
beside `Qwen3-8B` can materially reduce wall time, while an unsupported
|
||||
DSpark conversion performed poorly. Treat this as a supported draft-model lane
|
||||
first; keep MTP-specific packaging as future work only when the architecture
|
||||
and runtime support are real. Judge this by time-to-success, tool correctness,
|
||||
grammar, and unchanged target output, not tokens/sec alone.
|
||||
- Skill/tool prompt-injection audit. User-editable skills, notes, documents,
|
||||
fetched pages, and memories should be treated as untrusted data. Keep testing
|
||||
whether models follow malicious instructions from those surfaces.
|
||||
- Better degraded-state reporting for ChromaDB, SearXNG, email, ntfy, and provider probes.
|
||||
- Email performance audit. Fetching, searching, opening, deleting, and sending
|
||||
email can feel slow, especially over IMAP/SMTP providers with high latency.
|
||||
Need someone who knows mail performance to profile the current flow, identify
|
||||
whether the bottleneck is IMAP folder select/fetch, cache invalidation,
|
||||
attachment/body loading, SMTP handshakes, or frontend refresh behavior, then
|
||||
propose safer caching/prefetch/batching without breaking multi-account state.
|
||||
- Provider setup/probing audit for Anthropic, Gemini, Groq, xAI, OpenRouter, OpenAI, and DeepSeek.
|
||||
### Reliability and setup
|
||||
|
||||
## Refactor Targets
|
||||
- CSS cleanup. `static/style.css` basically Calypso's island atm.
|
||||
- Tour core helper. The onboarding tours have too much copy-pasted scaffolding; promote a shared `tour-core.js` helper before adding more tours.
|
||||
- Modal/window positioning cleanup. Some window controls have improved, but the
|
||||
underlying popup/dropdown/fixed-position behavior is still too fragile.
|
||||
- Mobile media override discoverability. A lot of "CSS did not move" bugs are mobile `@media` overrides of the same selector; comments or linting around desktop/mobile paired rules would help.
|
||||
- Dead code pass for old routes, stale feature flags, and unused UI states.
|
||||
- Improve fresh-install and smoke-test coverage across supported environments.
|
||||
- Make provider setup, probing, and failure states more predictable.
|
||||
- Improve Cookbook reliability across hardware, operating systems, drivers, shells, and serving backends.
|
||||
- Improve degraded-state reporting and recovery guidance when optional services are unavailable.
|
||||
|
||||
## Frontend
|
||||
### Local model workflows
|
||||
|
||||
- Expand the Editor for quicker, more robust everyday use. Better file/document
|
||||
handling, smoother window behavior, clearer save/export flows, stronger image
|
||||
editing affordances, and fewer brittle edge cases.
|
||||
- Better AI integration for Notes and Todos. Notes should be easier for the
|
||||
agent to read, update, summarize, and turn into actions. Todos should be
|
||||
assignable to an agent from the UI, possibly through a button, task action,
|
||||
or dedicated skill/tool flow.
|
||||
- Mobile gallery/editor polish. Easier to launch/download inpaint model or any missing pieces.
|
||||
- Accessibility pass: keyboard navigation, focus states, contrast, reduced motion.
|
||||
- Improve empty states and error messages on fresh installs.
|
||||
- Tighten first-run setup, hints, and tours so they do not repeat or fight each other.
|
||||
- Vendor CDN assets eventually for a more fully self-hosted/offline mode.
|
||||
- Improve hardware-aware model recommendations and compatibility guidance.
|
||||
- Evaluate serving optimizations, including speculative decoding, through reproducible benchmarks.
|
||||
- Improve installation, preflight checks, logging, and error reporting for local model serving.
|
||||
- Reduce prompt and context overhead for smaller local models.
|
||||
|
||||
## Backend
|
||||
### Safety and resilience
|
||||
|
||||
- More tests around endpoint probing and provider setup.
|
||||
- Better task scheduler defaults and visibility.
|
||||
- Backup/restore guide and helper flow for `data/`.
|
||||
- Security hardening around admin-only tools and clear docs for their risk.
|
||||
- Continue hardening tool execution, filesystem access, credentials, networking, and destructive operations.
|
||||
- Treat content from documents, notes, memories, skills, and fetched pages as potentially untrusted.
|
||||
- Improve security-focused regression coverage and operational guidance.
|
||||
- Review integrations that expand access to sensitive data or privileged operations.
|
||||
|
||||
## Not The Focus Right Now
|
||||
### Product usability
|
||||
|
||||
I prob shouldnt add more themes.
|
||||
- Improve first-run setup, onboarding, hints, and tours.
|
||||
- Improve accessibility, keyboard navigation, focus behaviour, contrast, and reduced-motion support.
|
||||
- Improve empty states, error messages, and recovery paths.
|
||||
- Strengthen Notes, Todos, Editor, mobile, and everyday workspace flows.
|
||||
|
||||
### Architecture and maintainability
|
||||
|
||||
- Reduce duplication and technical debt through focused, reviewable refactors.
|
||||
- Improve subsystem documentation as behaviour and architecture become stable.
|
||||
- Remove stale code, obsolete feature flags, and unsupported integrations.
|
||||
- Keep implementation decisions grounded in current code and verified behaviour.
|
||||
|
||||
## Tracking work
|
||||
|
||||
Concrete implementation tasks, defects, proposals, and technical investigations are tracked in:
|
||||
|
||||
- [GitHub Issues](https://github.com/odysseus-dev/odysseus/issues)
|
||||
- [Contributing Guide](CONTRIBUTING.md)
|
||||
|
||||
Maintainers may use additional private coordination tools for ownership, planning, and unresolved decisions.
|
||||
|
||||
This roadmap is not a complete backlog or a guarantee that a particular item will be delivered.
|
||||
|
||||
+5
-5
@@ -68,14 +68,14 @@ External content that reaches the LLM is treated as untrusted via `src/prompt_se
|
||||
- `X-Content-Type-Options: nosniff` and `Referrer-Policy: no-referrer` everywhere.
|
||||
- **CSP:** nonce-based `script-src 'self' 'nonce-{nonce}' https://cdn.jsdelivr.net`. `style-src 'unsafe-inline'` is intentionally kept — `static/index.html` ships inline `<style>` blocks and JS modules set `style=""` attributes at runtime. Inline styles do not execute script so the risk is visual-only. Removing this requires templating the HTML files and auditing all JS-set style attributes.
|
||||
|
||||
## Token-Supplied Model Endpoints
|
||||
|
||||
Direct `/api/v1/chat` requests with a token-supplied `base_url` must use a public HTTP(S) endpoint. This restriction applies only to untrusted direct values; administrator-configured endpoints may intentionally use local or LAN URLs for private model providers.
|
||||
|
||||
## Known Gaps
|
||||
|
||||
These are open, acknowledged, and contributor help is welcome:
|
||||
|
||||
1. **No shell/filesystem sandbox.** The agent `bash` and `read_file`/`write_file` tools run as the app process user with no network egress filtering or filesystem confinement. A successful prompt-injection reaching a shell-enabled admin session can make outbound requests to internal services. See #1058 for the sandbox proposal.
|
||||
|
||||
2. **SSRF via `/api/v1/chat` `base_url` parameter.** A chat-scoped API token can supply an arbitrary `base_url`; the server forwards the LLM request to that host without validating the scheme or address. PR #1039 fixes this.
|
||||
|
||||
3. **`src/search/` partial consolidation.** `src.search.core` and `src.search.providers` correctly alias `services.search` via `sys.modules` replacement. `analytics`, `cache`, `content`, `query`, and `ranking` are still independent copies that can drift. The SSRF regression tests in `tests/test_webhook_ssrf_resilience.py` test `src.webhook_manager` directly (separate from search), so the safety net there is intact. See #1058.
|
||||
|
||||
4. **Token scopes are coarse.** There is no way to grant a session a subset of the owning user's privileges. Companion/mobile tokens carry either `chat` or `admin` scope with no per-capability granularity.
|
||||
2. **Token scopes are coarse.** There is no way to grant a session a subset of the owning user's privileges. Companion/mobile tokens carry either `chat` or `admin` scope with no per-capability granularity.
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
# Odysseus discovery maps
|
||||
|
||||
Compact, code-grounded discovery maps of cross-cutting systems in the checked-in Odysseus codebase. They preserve investigation context and open factual questions; they are not canonical subsystem specifications, a feature certification, or a substitute for normal testing.
|
||||
|
||||
> [!IMPORTANT]
|
||||
> Checked-in code is the source of truth for current behaviour. Mature subsystem specifications, where they exist, are the canonical documentation of accepted subsystem behaviour. Check code, tests, and configuration before reconciling a discovery finding. Discovery remains non-canonical.
|
||||
|
||||
## Explore the maps
|
||||
|
||||
| Document | Purpose |
|
||||
|---|---|
|
||||
| [Current system map](system-map.md) | Records evidence locations, confirmed local observations, and factual open questions about subsystem boundaries. |
|
||||
| [Safety boundaries](safety-boundaries.md) | Records evidence about broad authority, safeguards, confirmed risks or gaps, and unverified behaviour. |
|
||||
|
||||
## Working rules
|
||||
|
||||
- **Trace the code first.** Confirm the current path in source before recording a claim.
|
||||
- **Promote selectively.** When an owning mature specification exists, add a fact only when it is verified, useful, and not already represented there.
|
||||
- **Record missing ownership.** When no owning specification exists, retain the verified finding in discovery and record missing documentation ownership as a follow-up.
|
||||
- **Retain uncertainty here.** Keep unresolved questions and useful investigation context in discovery rather than treating them as canonical truth.
|
||||
- **Keep specifications current-state only.** Do not record intentions, design direction, refactor plans, decision history, priority, ownership, or sequencing here.
|
||||
- **Investigate with cause.** Do not exhaustively revalidate existing functionality without a report, visible failure, relevant change, or high-authority review need.
|
||||
- **Review authority carefully.** Give execution, data access, external tools, credentials, destructive operations, and unattended work focused review.
|
||||
- **Use stable locations.** Cite modules, routes, classes, and functions instead of fragile line ranges or generated evidence tables.
|
||||
|
||||
## Reconciliation flow
|
||||
|
||||
1. Start with the relevant map and trace the cited code.
|
||||
2. Classify the finding against current source evidence and an owning mature specification where one exists.
|
||||
3. Promote only verified, useful facts that are missing from an existing owning specification.
|
||||
4. When no owning specification exists, retain the verified finding here and record missing documentation ownership as a follow-up; otherwise retain unresolved context here and correct stale wording.
|
||||
|
||||
> [!NOTE]
|
||||
> This package intentionally contains no generator, validator, maturity scale, feature database, or parallel work tracker. The [architecture runtime inventory](../specs/architecture-runtime-inventory.md) remains useful structural context, but is an explicitly draft snapshot.
|
||||
@@ -0,0 +1,142 @@
|
||||
# Safety boundaries
|
||||
|
||||
> [!IMPORTANT]
|
||||
> This non-canonical discovery map records code-grounded safeguards, confirmed risks or gaps, and unverified behaviour. Broad authority does not by itself establish a vulnerability. Verify the cited source before relying on a finding. No destructive test, external connection, or real credential was used for this map.
|
||||
|
||||
## Navigate the boundaries
|
||||
|
||||
- [Shell and subprocess execution](#shell-and-subprocess-execution)
|
||||
- [Filesystem access and workspace confinement](#filesystem-access-and-workspace-confinement)
|
||||
- [Agent-controlled tool dispatch](#agent-controlled-tool-dispatch)
|
||||
- [MCP and external tool servers](#mcp-and-external-tool-servers)
|
||||
- [Outbound network requests and URL validation](#outbound-network-requests-and-url-validation)
|
||||
- [Secrets, credentials, and vault sessions](#secrets-credentials-and-vault-sessions)
|
||||
- [Authentication and privileged administration](#authentication-and-privileged-administration)
|
||||
- [Deletion, wipe, backup, and restore](#deletion-wipe-backup-and-restore)
|
||||
- [Background jobs and unattended task execution](#background-jobs-and-unattended-task-execution)
|
||||
|
||||
## Shell and subprocess execution
|
||||
|
||||
- **Boundary:** Shell routes, agent `bash` and `python` tools, local model serving, and detached background jobs.
|
||||
|
||||
- **Available authority:** Commands run as the application process user and can create child processes.
|
||||
|
||||
- **User-controlled inputs:** Direct shell requests, model-produced tool arguments, scheduled-task prompts, and model-serving configuration.
|
||||
|
||||
- **Current safeguards:** Agent dispatch applies owner/admin checks and tool policy; process helpers use timeouts or bounded background-job lifecycle where implemented.
|
||||
|
||||
- **Confirmed risks or gaps:** Intentional authority with a confirmed gap: the agent shell starts in its workspace but is not sandboxed to it, and has no egress sandbox. This is documented in source and the threat model; it is not a newly demonstrated bypass.
|
||||
|
||||
- **Unverified behaviour:** Role-gate and disabled-tool outcomes, direct shell-route behaviour, and timeout, cancellation, and output handling for foreground and detached processes remain unverified.
|
||||
|
||||
## Filesystem access and workspace confinement
|
||||
|
||||
- **Boundary:** Agent read, write, patch, listing, glob, and grep tools.
|
||||
|
||||
- **Available authority:** Read and modify files within active workspace confinement or fallback allowlisted roots.
|
||||
|
||||
- **User-controlled inputs:** Tool paths, patches, file contents, search patterns, and workspace selection passed into the tool dispatcher.
|
||||
|
||||
- **Current safeguards:** [`src/tool_execution.py`](../src/tool_execution.py) resolves paths, blocks sensitive subpaths, applies allowlist containment, and tightens paths to the active workspace when one is bound. File tools use those resolvers.
|
||||
|
||||
- **Confirmed risks or gaps:** Intentional authority with safeguards. The file-tool policy does not sandbox the shell; treating a workspace as a whole-process containment boundary would be incorrect.
|
||||
|
||||
- **Unverified behaviour:** Traversal, symlink, sensitive-name, absolute-path, and workspace-switch behaviour remains unverified.
|
||||
|
||||
## Agent-controlled tool dispatch
|
||||
|
||||
- **Boundary:** Model output becomes native or parsed tool calls and is dispatched by the agent loop.
|
||||
|
||||
- **Available authority:** The authority of every enabled tool, including privileged built-ins and external tools.
|
||||
|
||||
- **User-controlled inputs:** Chat content, attached/retrieved content that may influence the model, tool arguments, per-request tool selection, and policy toggles.
|
||||
|
||||
- **Current safeguards:** [`src/tool_security.py`](../src/tool_security.py) blocks protected tools for non-admin users and fails closed for malformed tool names; [`src/tool_policy.py`](../src/tool_policy.py) supports disabled and guide-only policy; prompt-security helpers label untrusted context.
|
||||
|
||||
- **Confirmed risks or gaps:** Credible risk requiring verification: aliases, legacy text tools, native function calls, and MCP-qualified names must all reach the same policy outcome. The code has specific alias handling for email/MCP names, which makes this a sensitive compatibility seam.
|
||||
|
||||
- **Unverified behaviour:** The current policy outcomes for owner role, request mode, disabled state, native versus parsed invocation, qualified aliases, and external-content entry points remain unverified.
|
||||
|
||||
## MCP and external tool servers
|
||||
|
||||
- **Boundary:** Configured MCP servers and their tools are exposed to the agent through the MCP manager and routes.
|
||||
|
||||
- **Available authority:** Depends on the server: external network access, local process access, messaging, or data mutation may be delegated outside the application.
|
||||
|
||||
- **User-controlled inputs:** Server configuration, remote OAuth completion, tool arguments, and model-selected MCP calls.
|
||||
|
||||
- **Current safeguards:** MCP routes are registered through [`routes/mcp_routes.py`](../routes/mcp_routes.py); MCP-qualified tools are denied to non-admin users by [`src/tool_security.py`](../src/tool_security.py). OAuth state and token persistence are handled in [`src/mcp_oauth.py`](../src/mcp_oauth.py).
|
||||
|
||||
- **Confirmed risks or gaps:** Credible risk requiring verification: an MCP server authority is broader than the application can infer from its tool name. This map does not establish a trust or approval model for server installation and individual tool invocation.
|
||||
|
||||
- **Unverified behaviour:** Server onboarding, credential storage, server-origin trust, OAuth callback deployment, tool disablement, and invocation audit behaviour remain unverified.
|
||||
|
||||
## Outbound network requests and URL validation
|
||||
|
||||
- **Boundary:** Search/content fetch, research, webhooks, skill import, provider endpoints, and other HTTP clients.
|
||||
|
||||
- **Available authority:** The application can make outbound requests from its network position.
|
||||
|
||||
- **User-controlled inputs:** Search/fetch URLs, imported skill URLs, webhook configuration, and some endpoint settings.
|
||||
|
||||
- **Current safeguards:** [`src/url_security.py`](../src/url_security.py) validates untrusted public HTTP URLs and fails closed on unsuitable schemes or private addresses. [`services/search/content.py`](../services/search/content.py) resolves and rejects non-public hosts, pins resolved addresses for fetches, caps bodies, and limits redirects.
|
||||
|
||||
- **Confirmed risks or gaps:** Intentional split: administrator-created model endpoints may target private providers, while untrusted URLs use public-address checks. That distinction is required for self-hosted deployments but needs explicit call-site review.
|
||||
|
||||
- **Unverified behaviour:** The URL-source classification for outbound clients and the current handling of redirects and DNS changes remain unverified.
|
||||
|
||||
## Secrets, credentials, and vault sessions
|
||||
|
||||
- **Boundary:** Application-managed encrypted secrets, API keys, provider credentials, and Bitwarden/Vaultwarden CLI sessions.
|
||||
|
||||
- **Available authority:** Credentials unlock remote providers and connected personal services.
|
||||
|
||||
- **User-controlled inputs:** Administrative configuration, login/unlock requests, imported settings, and agent vault tool arguments.
|
||||
|
||||
- **Current safeguards:** [`src/secret_storage.py`](../src/secret_storage.py) uses a locally stored Fernet key with restrictive permissions for supported database secrets. Vault routes require an administrator, avoid passing master passwords in command arguments, and set restrictive permissions on the vault-session file.
|
||||
|
||||
- **Confirmed risks or gaps:** Confirmed current boundary: vault session data is persisted through the vault path, not through [`src/secret_storage.py`](../src/secret_storage.py). This is an unresolved question about current security semantics, not a confirmed exposure.
|
||||
|
||||
- **Unverified behaviour:** Current encryption-at-rest, owner scope, rotation, lock/logout, backup/restore, and log/tool-result exposure behaviour remains unverified.
|
||||
|
||||
## Authentication and privileged administration
|
||||
|
||||
- **Boundary:** Session authentication, API tokens, privileged routes, and internal tool loopback.
|
||||
|
||||
- **Available authority:** Administrative identity can access execution, settings, integrations, data deletion, and secrets.
|
||||
|
||||
- **User-controlled inputs:** Login/signup data, session cookies, API tokens, authentication configuration, and requests to privileged routes.
|
||||
|
||||
- **Current safeguards:** [`core/auth.py`](../core/auth.py), [`core/middleware.py`](../core/middleware.py), and route-level checks establish identity and administrator gates. [`app.py`](../app.py) warns when localhost bypass is configured; [`SECURITY.md`](../SECURITY.md) documents deployment requirements.
|
||||
|
||||
- **Confirmed risks or gaps:** Intentional authority with safeguards. Security depends on deployments keeping authentication enabled and internal services private; this map does not audit reverse-proxy or environment configuration.
|
||||
|
||||
- **Unverified behaviour:** Setup, anonymous, non-admin, admin, token, and internal-loopback behaviour, including privileged-route gate consistency, remains unverified.
|
||||
|
||||
## Deletion, wipe, backup, and restore
|
||||
|
||||
- **Boundary:** Administrative wipe, cleanup, backup import/export, and the backup restore command.
|
||||
|
||||
- **Available authority:** Delete or replace user data and credentials.
|
||||
|
||||
- **User-controlled inputs:** Administrative HTTP requests, cleanup choices, backup payloads, archive paths, and restore command options.
|
||||
|
||||
- **Current safeguards:** Administrative wipe routes use the administrative boundary. Cleanup exposes a preview route before mutation. The documented backup tool requires explicit restore confirmation, stages the old data directory, and validates archive members before extraction.
|
||||
|
||||
- **Confirmed risks or gaps:** Intentional destructive authority. Backup archives contain secrets by design, as documented in [`docs/backup-restore.md`](../docs/backup-restore.md); this is an operator confidentiality responsibility, not a code defect established here.
|
||||
|
||||
- **Unverified behaviour:** Role-gate, confirmation, archive-rejection, staged-recovery, and owner-isolation behaviour remains unverified. No destructive runtime test was performed.
|
||||
|
||||
## Background jobs and unattended task execution
|
||||
|
||||
- **Boundary:** Scheduled tasks, background-job monitor, startup tasks, and notification/delivery work that continue without an active browser request.
|
||||
|
||||
- **Available authority:** Scheduled agent work can obtain model access and, for eligible owners, shell and file tools; task output can interact with connected services.
|
||||
|
||||
- **User-controlled inputs:** Stored task prompt, schedule, model/crew selection, enabled-tool configuration, output target, and prior persisted state.
|
||||
|
||||
- **Current safeguards:** [`src/task_scheduler.py`](../src/task_scheduler.py) serializes execution, records task runs, associates work with an owner, and applies the agent owner-based tool gate. [`src/bg_jobs.py`](../src/bg_jobs.py) keeps bounded state and can terminate overlong subprocess jobs.
|
||||
|
||||
- **Confirmed risks or gaps:** Credible risk requiring verification: authority is inherited and exercised later, so changes to roles, task configuration, and disabled tools must be checked at execution time rather than assumed from task creation.
|
||||
|
||||
- **Unverified behaviour:** Creation, editing, role-change, scheduling, cancellation, restart-recovery, and execution behaviour remains unverified, including whether current policy is re-evaluated before privileged action.
|
||||
@@ -0,0 +1,139 @@
|
||||
# Current system map
|
||||
|
||||
> [!NOTE]
|
||||
> This non-canonical discovery map is an evidence guide, not an exhaustive feature catalog or runtime certification. Verify the cited source before relying on a finding. Each section records local implementation observations, evidence locations, confirmed current problems, and unresolved factual questions.
|
||||
|
||||
## Navigate the system
|
||||
|
||||
- [Startup and application composition](#startup-and-application-composition)
|
||||
- [Frontend shell and browser interaction](#frontend-shell-and-browser-interaction)
|
||||
- [Chat, sessions, and streaming](#chat-sessions-and-streaming)
|
||||
- [Agents, tools, and execution](#agents-tools-and-execution)
|
||||
- [Models, providers, and local serving](#models-providers-and-local-serving)
|
||||
- [Search and research](#search-and-research)
|
||||
- [Documents, retrieval, and personal knowledge](#documents-retrieval-and-personal-knowledge)
|
||||
- [Memory and skills](#memory-and-skills)
|
||||
- [Email, calendar, contacts, notes, and tasks](#email-calendar-contacts-notes-and-tasks)
|
||||
- [Media, speech, and image work](#media-speech-and-image-work)
|
||||
- [Authentication, secrets, and privileged administration](#authentication-secrets-and-privileged-administration)
|
||||
- [Persistence, background work, and operations](#persistence-background-work-and-operations)
|
||||
|
||||
## Startup and application composition
|
||||
|
||||
- **How it works:** [`app.py`](../app.py) creates the application, mounts static assets, constructs shared services, registers route factories, and owns lifespan startup and shutdown. [`src/app_initializer.py`](../src/app_initializer.py) prepares application state; [`core/`](../core/) provides persistence, authentication, middleware, sessions, and platform helpers.
|
||||
|
||||
- **Evidence locations:** [`app.py`](../app.py); [`src/app_initializer.py`](../src/app_initializer.py); [`core/database.py`](../core/database.py); [`core/auth.py`](../core/auth.py); [`core/middleware.py`](../core/middleware.py); [`routes/`](../routes/).
|
||||
|
||||
- **Known problems:** None recorded by this mapping.
|
||||
|
||||
- **Open question:** Which component currently owns startup and shutdown for each long-lived service?
|
||||
|
||||
## Frontend shell and browser interaction
|
||||
|
||||
- **How it works:** [`static/index.html`](../static/index.html) is served by the root and SPA deep-link routes in [`app.py`](../app.py); [`static/app.js`](../static/app.js), [`static/style.css`](../static/style.css), and [`static/js/`](../static/js/) implement the client surface.
|
||||
|
||||
- **Evidence locations:** [`static/index.html`](../static/index.html); [`static/app.js`](../static/app.js); [`static/js/`](../static/js/); [`static/style.css`](../static/style.css); [`app.py`](../app.py) deep-link handlers.
|
||||
|
||||
- **Known problems:** The `/backgrounds` route in [`app.py`](../app.py) calls `serve_html_with_nonce` for `static/backgrounds.html`, but that file is absent from [`static/`](../static/). This is a confirmed broken prototype route, not evidence about the rest of the frontend.
|
||||
|
||||
- **Open question:** Is `/backgrounds` currently an intentionally supported route or an obsolete prototype?
|
||||
|
||||
## Chat, sessions, and streaming
|
||||
|
||||
- **How it works:** [`routes/chat_routes.py`](../routes/chat_routes.py) and [`routes/chat_helpers.py`](../routes/chat_helpers.py) coordinate requests, session state, and SSE delivery. [`src/chat_handler.py`](../src/chat_handler.py), [`src/chat_processor.py`](../src/chat_processor.py), [`src/llm_core.py`](../src/llm_core.py), and [`src/session_actions.py`](../src/session_actions.py) provide message preparation, provider interaction, and session operations.
|
||||
|
||||
- **Evidence locations:** [`routes/chat_routes.py`](../routes/chat_routes.py); [`routes/chat_helpers.py`](../routes/chat_helpers.py); [`routes/session_routes.py`](../routes/session_routes.py); [`src/chat_handler.py`](../src/chat_handler.py); [`src/chat_processor.py`](../src/chat_processor.py); [`src/llm_core.py`](../src/llm_core.py); [`core/session_manager.py`](../core/session_manager.py).
|
||||
|
||||
- **Known problems:** [`src/agent_loop.py`](../src/agent_loop.py) annotates `_resolved_tool_event_name` with `Any` but imports no `Any` and does not enable postponed annotation evaluation. Python evaluates that annotation while importing the module, so this is an import-time defect at the checked baseline.
|
||||
|
||||
- **Open question:** No end-to-end provider or browser streaming run was performed for this map.
|
||||
|
||||
## Agents, tools, and execution
|
||||
|
||||
- **How it works:** [`src/agent_loop.py`](../src/agent_loop.py) drives multi-round tool use. [`src/tool_execution.py`](../src/tool_execution.py) dispatches calls and binds workspace context. [`src/agent_tools/`](../src/agent_tools/) contains individual implementations; [`src/tool_security.py`](../src/tool_security.py) and [`src/tool_policy.py`](../src/tool_policy.py) apply role and request policies. Long-running command work is represented by [`src/bg_jobs.py`](../src/bg_jobs.py).
|
||||
|
||||
- **Evidence locations:** [`src/agent_loop.py`](../src/agent_loop.py); [`src/tool_execution.py`](../src/tool_execution.py); [`src/agent_tools/`](../src/agent_tools/); [`src/tool_security.py`](../src/tool_security.py); [`src/tool_policy.py`](../src/tool_policy.py); [`src/tool_schemas.py`](../src/tool_schemas.py); [`src/bg_jobs.py`](../src/bg_jobs.py).
|
||||
|
||||
- **Known problems:** The import-time annotation defect above blocks the main agent/tool path. The shell is intentionally not a filesystem or network sandbox; that is an authority boundary, not by itself a vulnerability claim.
|
||||
|
||||
- **Open question:** Which native, legacy, and MCP-qualified invocation paths reach each policy gate?
|
||||
|
||||
## Models, providers, and local serving
|
||||
|
||||
- **How it works:** Model routes delegate to discovery, capabilities, endpoint resolution, and LLM core modules. Cookbook routes and hardware-fit services handle model lifecycle and local-serving support.
|
||||
|
||||
- **Evidence locations:** [`routes/model_routes.py`](../routes/model_routes.py); [`src/model_discovery.py`](../src/model_discovery.py); [`src/model_capabilities.py`](../src/model_capabilities.py); [`src/endpoint_resolver.py`](../src/endpoint_resolver.py); [`src/llm_core.py`](../src/llm_core.py); [`routes/cookbook_routes.py`](../routes/cookbook_routes.py); [`src/cookbook_serve_lifecycle.py`](../src/cookbook_serve_lifecycle.py); [`services/hwfit/`](../services/hwfit/).
|
||||
|
||||
- **Known problems:** None recorded by this mapping.
|
||||
|
||||
- **Open question:** Which endpoint inputs are administrator-created and permitted to use private provider addresses?
|
||||
|
||||
## Search and research
|
||||
|
||||
- **How it works:** HTTP search routes use [`services/search/`](../services/search/); research is exposed through [`routes/research/`](../routes/research/) and implemented in [`services/research/`](../services/research/), [`src/deep_research.py`](../src/deep_research.py), and related helpers. [`src/search/`](../src/search/) remains an import-compatibility layer for callers not yet moved to `services.search`.
|
||||
|
||||
- **Evidence locations:** [`routes/search_routes.py`](../routes/search_routes.py); [`services/search/`](../services/search/); [`routes/research/research_routes.py`](../routes/research/research_routes.py); [`services/research/`](../services/research/); [`src/deep_research.py`](../src/deep_research.py); [`src/search/`](../src/search/).
|
||||
|
||||
- **Known problems:** None recorded by this mapping.
|
||||
|
||||
- **Open question:** No live provider request was made; provider configuration and network access remain unverified.
|
||||
|
||||
## Documents, retrieval, and personal knowledge
|
||||
|
||||
- **How it works:** Document routes coordinate upload handling, document processing, and editor actions. Personal-document and RAG modules use Chroma and embedding clients. PDF viewing uses the optional-dependency loader in [`src/pdf_runtime.py`](../src/pdf_runtime.py); form extraction and filling live separately in [`src/pdf_forms.py`](../src/pdf_forms.py) and [`src/pdf_form_doc.py`](../src/pdf_form_doc.py).
|
||||
|
||||
- **Evidence locations:** [`routes/document_routes.py`](../routes/document_routes.py); [`src/upload_handler.py`](../src/upload_handler.py); [`src/document_processor.py`](../src/document_processor.py); [`src/document_actions.py`](../src/document_actions.py); [`src/personal_docs.py`](../src/personal_docs.py); [`src/rag_manager.py`](../src/rag_manager.py); [`src/embeddings.py`](../src/embeddings.py); [`src/pdf_runtime.py`](../src/pdf_runtime.py); [`src/pdf_forms.py`](../src/pdf_forms.py); [`src/pdf_form_doc.py`](../src/pdf_form_doc.py).
|
||||
|
||||
- **Known problems:** PDF viewing/runtime loading and PDF form processing are separate implementations. That separation is confirmed and intentional in the source; it is not a defect without a reported behavioural failure.
|
||||
|
||||
- **Open question:** Optional PDF dependencies and representative uploaded documents were not exercised.
|
||||
|
||||
## Memory and skills
|
||||
|
||||
- **How it works:** Memory routes use [`services/memory/`](../services/memory/) and vector helpers. Skills are exposed through [`routes/skills_routes.py`](../routes/skills_routes.py), stored and managed in [`services/memory/skills.py`](../services/memory/skills.py), and may be imported through [`services/memory/skill_importer.py`](../services/memory/skill_importer.py).
|
||||
|
||||
- **Evidence locations:** [`routes/memory/memory_routes.py`](../routes/memory/memory_routes.py); [`services/memory/`](../services/memory/); [`src/memory.py`](../src/memory.py); [`src/memory_vector.py`](../src/memory_vector.py); [`routes/skills_routes.py`](../routes/skills_routes.py); [`services/memory/skills.py`](../services/memory/skills.py); [`services/memory/skill_importer.py`](../services/memory/skill_importer.py).
|
||||
|
||||
- **Known problems:** None recorded by this mapping.
|
||||
|
||||
- **Open question:** Which imported skill content can reach execution-capable paths, and which validation occurs before that point?
|
||||
|
||||
## Email, calendar, contacts, notes, and tasks
|
||||
|
||||
- **How it works:** Dedicated route modules own email, CalDAV calendar, CardDAV contacts, notes, and tasks. Supporting modules include email helpers and pollers, CalDAV sync and writeback, and the task scheduler.
|
||||
|
||||
- **Evidence locations:** [`routes/email_routes.py`](../routes/email_routes.py); [`routes/calendar_routes.py`](../routes/calendar_routes.py); [`routes/contacts/contacts_routes.py`](../routes/contacts/contacts_routes.py); [`routes/note/note_routes.py`](../routes/note/note_routes.py); [`routes/task_routes.py`](../routes/task_routes.py); [`routes/assistant_routes.py`](../routes/assistant_routes.py); [`src/caldav_sync.py`](../src/caldav_sync.py); [`src/caldav_writeback.py`](../src/caldav_writeback.py); [`src/task_scheduler.py`](../src/task_scheduler.py).
|
||||
|
||||
- **Known problems:** None recorded by this mapping.
|
||||
|
||||
- **Open question:** External account behaviour, writeback, and delivery require controlled credentials and are not runtime-validated here.
|
||||
|
||||
## Media, speech, and image work
|
||||
|
||||
- **How it works:** Gallery and image routes coordinate media features. Service modules own speech and media integrations; [`src/generated_images.py`](../src/generated_images.py) and [`src/visual_report.py`](../src/visual_report.py) support artifact handling and presentation.
|
||||
|
||||
- **Evidence locations:** [`routes/gallery/gallery_routes.py`](../routes/gallery/gallery_routes.py); [`routes/stt_routes.py`](../routes/stt_routes.py); [`routes/tts_routes.py`](../routes/tts_routes.py); [`src/generated_images.py`](../src/generated_images.py); [`services/stt/`](../services/stt/); [`services/tts/`](../services/tts/); [`services/faces/`](../services/faces/); [`src/visual_report.py`](../src/visual_report.py).
|
||||
|
||||
- **Known problems:** None recorded by this mapping.
|
||||
|
||||
- **Open question:** Hardware- and provider-dependent media workflows were not exercised.
|
||||
|
||||
## Authentication, secrets, and privileged administration
|
||||
|
||||
- **How it works:** [`core/auth.py`](../core/auth.py) and [`core/middleware.py`](../core/middleware.py) provide identity and request gates. [`src/secret_storage.py`](../src/secret_storage.py) encrypts application-managed database secrets with a local Fernet key. Vault handling is separate: [`routes/vault_routes.py`](../routes/vault_routes.py) and [`src/tools/vault.py`](../src/tools/vault.py) invoke the Bitwarden CLI and persist its session data in the application data area.
|
||||
|
||||
- **Evidence locations:** [`core/auth.py`](../core/auth.py); [`core/middleware.py`](../core/middleware.py); [`routes/auth_routes.py`](../routes/auth_routes.py); [`routes/api_token_routes.py`](../routes/api_token_routes.py); [`src/secret_storage.py`](../src/secret_storage.py); [`routes/vault_routes.py`](../routes/vault_routes.py); [`src/tools/vault.py`](../src/tools/vault.py); [`routes/admin_wipe/admin_wipe_routes.py`](../routes/admin_wipe/admin_wipe_routes.py).
|
||||
|
||||
- **Known problems:** Vault-command handling and local application secret storage are distinct paths with different storage mechanisms. This is a source-confirmed boundary, not evidence that either path is compromised.
|
||||
|
||||
- **Open question:** What are the current confidentiality, ownership, rotation, and backup semantics for vault session data?
|
||||
|
||||
## Persistence, background work, and operations
|
||||
|
||||
- **How it works:** SQLite models and persistence are centred in [`core/database.py`](../core/database.py); managers use application data paths. The scheduler and background-job monitor can continue work outside a live browser request. Operational routes cover cleanup, backup, and administrative wipe; the repository also provides a backup script and user documentation.
|
||||
|
||||
- **Evidence locations:** [`core/database.py`](../core/database.py); [`src/runtime_paths.py`](../src/runtime_paths.py); [`src/task_scheduler.py`](../src/task_scheduler.py); [`src/bg_jobs.py`](../src/bg_jobs.py); [`src/bg_monitor.py`](../src/bg_monitor.py); [`routes/backup_routes.py`](../routes/backup_routes.py); [`routes/cleanup/cleanup_routes.py`](../routes/cleanup/cleanup_routes.py); [`routes/admin_wipe/admin_wipe_routes.py`](../routes/admin_wipe/admin_wipe_routes.py); [`scripts/odysseus-backup`](../scripts/odysseus-backup); [`docs/backup-restore.md`](../docs/backup-restore.md).
|
||||
|
||||
- **Known problems:** None recorded by this mapping.
|
||||
|
||||
- **Open question:** What current behaviour applies to background execution, cancellation, retries, and authority inheritance?
|
||||
+1
-1
@@ -441,7 +441,7 @@ uv pip sync requirements.lock # reproduce it exactly la
|
||||
### Outlook / Office 365 email
|
||||
Odysseus email accounts currently use IMAP/SMTP username-password auth. Outlook
|
||||
and Microsoft 365 generally require OAuth instead, so normal Microsoft mailbox
|
||||
passwords will fail. See [docs/email-outlook.md](docs/email-outlook.md) for the
|
||||
passwords will fail. See [email-outlook.md](email-outlook.md) for the
|
||||
current limitation and the planned integration direction.
|
||||
|
||||
## Security Notes
|
||||
|
||||
@@ -1,8 +1,8 @@
|
||||
"""Provider endpoint URL-building tests.
|
||||
|
||||
Covers ``build_chat_url`` and ``build_models_url`` for every provider named in
|
||||
ROADMAP.md: Anthropic, Gemini, Groq, xAI, OpenRouter, OpenAI, DeepSeek, Ollama
|
||||
(local + cloud).
|
||||
Covers ``build_chat_url`` and ``build_models_url`` for the provider families
|
||||
exercised by this module: Anthropic, Gemini, Groq, xAI, OpenRouter, OpenAI,
|
||||
DeepSeek, and Ollama (local and cloud).
|
||||
"""
|
||||
import pytest
|
||||
|
||||
|
||||
Reference in New Issue
Block a user